mARTin-B78
813e06c914
Magazine P1:
- Images shown automatically during playback (assign_images_to_articles
links each figure to the article whose column it falls in; the client
shows the last image reached in reading order as playback advances).
- "Play all articles" queues every approved article and advances
automatically at the end of each rather than stopping.
- Word-level highlighting, explicitly documented as an ESTIMATE (no
forced-alignment/timestamp source exists): each word's share of the
sentence's real measured audio duration is proportional to its
character length. A checkbox turns it off; sentence highlighting
(measured, not estimated) is unaffected.
- Driving mode: full-screen overlay, two oversized controls, reuses
whatever is already playing/queued.
- Sidebars/pull-out boxes excluded from narration automatically
(mark_sidebars: an anomalously narrow column vs. the page's regular
grid). Fixed a real defect found while building this: a full-width
headline was left in the column-width analysis and could bridge two
genuinely separate columns into one, hiding a real sidebar column
entirely -- caught by a failing test before the fix.
Second OCR engine + fallback ladder:
- KDL Frontier Parser nano wired in via vision_ocr_page_chain (ordered
engine list, first usable result wins, tesseract last resort).
Confirmed live: ~3x faster than Baidu (~12s vs ~40s/page) but returns
plain text with NO coordinate markers, unlike Baidu's labeled regions.
_whole_page_region treats the whole response as one ambiguous
full-page block (gated by ocr_quality_score) rather than discarding
usable text just because it lacks layout structure.
OCR engines page + a real UI bug:
- The OCR & Document Parsing page had no .s-engines-page[data-page="ocr"]
markup at all -- renderLocalContainers already supported an OCR grid
generically, but $('dc-grid-ocr') was always null, so the card grid
silently never rendered. Added an Active OCR Engine panel mirroring
Active Language Model exactly, plus the Local grid -- both configured
engines now appear as ready-made cards, no "Add Custom" dialog needed.
- Fixed the audiobook pipeline's "Previous/Next" nav bar bleeding into
every .section-head in the document (confirmed live on the Engines
page: a "Cast Audiobook" button and "STEP 1/8" hint that made no sense
there). Changed from a blocklist (naming each unrelated section as
noticed) to an allowlist of the pipeline's three actual destinations.
Verified live end-to-end: play-all queue building and manual advance,
word-span markup, driving-mode toggle, OCR page showing both engines as
local cards, zero stray nav bars on the Engines page, no console errors.
20 new backend tests (156 total in the magazine module).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
89 lines
3.8 KiB
Markdown
89 lines
3.8 KiB
Markdown
# Project map
|
|
|
|
Living index of where things are and what state they are in.
|
|
Update in the same commit as the change it describes.
|
|
|
|
## Current phase
|
|
|
|
Magazine projects, **P0 and P1 complete** (frontend reachable at Studio → Magazines). Audiobook/Rehearser/voice-library
|
|
features are mature and in daily use.
|
|
|
|
## Deployment
|
|
|
|
Self-hosted Docker on the DGX Spark, port 7890, image
|
|
`ghcr.io/martin-b78/tts-voice-creator-clone-and-design-2`.
|
|
Python and static files are bind-mounted, so code changes need only a container
|
|
recreate; rebuild and push only when `Dockerfile` or `requirements.txt` change.
|
|
|
|
## Layout
|
|
|
|
| Path | Role |
|
|
|---|---|
|
|
| `server.py` | FastAPI app, router registration, auth middleware |
|
|
| `core/` | domain logic — config, database, audio, voice, routing, magazine |
|
|
| `routes/` | HTTP boundary, one module per feature area |
|
|
| `static/js/` | frontend, one module per feature area (no build step for most) |
|
|
| `tests/` | `test_*.py` (unittest) and `*.spec.js` (Playwright) |
|
|
| `architecture/` | SOPs for subsystems whose design is not obvious from code |
|
|
| `workflows/` | ComfyUI graphs for character portrait generation |
|
|
|
|
## Feature areas
|
|
|
|
| Area | Server | Frontend | State |
|
|
|---|---|---|---|
|
|
| Voice library / clone / design | `routes/library.py`, `core/voice.py` | `library.js` | mature |
|
|
| Read Aloud (books, PDFs) | `routes/reader.py` | `reader.js` | mature |
|
|
| Audiobook casting | `routes/conversation.py` | `audiobook*.js` | mature |
|
|
| Rehearser | `routes/rehearsals_db.py` | `rehearser-*.js` | mature |
|
|
| Studio (phase wizard) | — | `studio.js` | mature |
|
|
| TTS engines / routing | `routes/tts.py`, `core/routing.py` | `engines.js` | mature |
|
|
| STT | `routes/stt.py` | — | mature |
|
|
| **Magazines** | `core/magazine.py`, `routes/magazine.py` | `magazine.js`, `s-magazine.html` | **P0 + P1** |
|
|
|
|
## Data model
|
|
|
|
Documents (books and magazines alike) live under
|
|
`<CONFIG_DIR>/reader_library/<doc_id>/`, discriminated by `meta.json → kind`:
|
|
|
|
- `kind: "pdf"` / text — Read Aloud book. Text is extracted in the browser and
|
|
not persisted; `meta.json` holds counts and a flat `idx` resume position.
|
|
- `kind: "magazine"` — adds `layout.json`, `articles.json`, `pages/`, `images/`,
|
|
`podcast/`, and a per-article `progress` map. See
|
|
[architecture/magazine-ingestion.md](architecture/magazine-ingestion.md).
|
|
|
|
Settings, presets, routes and the SQLite database also live under `CONFIG_DIR`,
|
|
which is the writable volume.
|
|
|
|
## Environment
|
|
|
|
| Variable | Purpose |
|
|
|---|---|
|
|
| `CONFIG_DIR` | writable state (settings, library, database) |
|
|
| `VOICES_DIR` / `OUTPUT_DIR` | voice WAV library |
|
|
| `TTS_CONTAINER_NAMES` | containers restarted after a voice change |
|
|
| `DOCKER_HOST` | points at the socket proxy; the raw socket is not mounted |
|
|
| `llm_url` / `llm_model` / `llm_api_key` | settings.json, OpenAI-compatible LLM |
|
|
|
|
## Pending decisions
|
|
|
|
- Container registry: ghcr costs quota against a private package (≈427 MB per
|
|
push, 500 MB free on the Free plan). Local registry vs. ghcr vs. no re-pull is
|
|
**unresolved** — the compose files currently point at ghcr.
|
|
## Known limitations
|
|
|
|
- `gitea` backup remote is 157 commits behind `origin`; `main` tracks `gitea`.
|
|
- Magazine sidebar exclusion is a narrow-column geometry heuristic
|
|
(`mark_sidebars`), not certain — review always shows every sidebar block.
|
|
- Magazine word-level highlighting is a character-length estimate against
|
|
real measured audio duration, not true forced alignment.
|
|
- No formal cross-engine OCR accuracy benchmark exists (two engines wired:
|
|
ocr-baidu-unlimited-ocr primary, KDL Frontier Parser nano fallback).
|
|
- No `npm run qa` aggregate command yet; Python tests run via `unittest`,
|
|
browser tests via `playwright test`.
|
|
|
|
## Next logical step
|
|
|
|
Magazine P2 per TODO.md: image descriptions, table/chart handling, EPUB 3
|
|
Media Overlay export, and a formal OCR-engine benchmark (see
|
|
architecture/magazine-ingestion.md).
|