tts-voice-creator-clone-and-design-2/PROJECT_MAP.md

Project map

Living index of where things are and what state they are in. Update in the same commit as the change it describes.

Current phase

Magazine projects, P0 and P1 complete (frontend reachable at Studio → Magazines). Audiobook/Rehearser/voice-library features are mature and in daily use.

Deployment

Two self-hosted Docker instances, both port 7890:

• DGX Spark (192.168.178.8) — primary. docker-compose.yml builds locally and tags the result ghcr.io/martin-b78/tts-voice-creator-clone-and-design-2, but that registry copy is not kept current on every deploy (several versions behind as of 2026-09-14) — treat it as a snapshot, not a source of truth. Python and static files are bind-mounted, so a code change needs only a container recreate; rebuild only when Dockerfile or requirements.txt change.
• correspondence-nuc (192.168.178.19) — secondary, live since 2026-09-14. No TTS/OCR engines of its own; every generation call crosses the LAN to the DGX (docker-compose.nuc.yml, tracked here, pins vision_ocr_url/vision_ocr_fallback_url explicitly to the DGX's IP — their default resolves relative to whichever host the container runs on, silently wrong here otherwise). Voice library is the DGX's own directory, NFS-mounted read-write (/etc/exports on the DGX, /etc/fstab on the NUC) — never a separate copy that could drift. Deployed by rsyncing the working tree (excluding .git, config/, logs/) and building on the NUC itself, since the ghcr.io image is both stale and private with no registry login there. See TODO.md's "P2 — deployment" entry for the full setup and the one accepted limitation (voice-change auto-restart of the DGX's TTS containers doesn't reach across hosts — manual restart on the DGX, same as before this existed).

Layout

Path	Role
server.py	FastAPI app, router registration, auth middleware
core/	domain logic — config, database, audio, voice, routing, magazine
routes/	HTTP boundary, one module per feature area
static/js/	frontend, one module per feature area (no build step for most)
tests/	test_*.py (unittest, run with python3 -m unittest tests.test_x), *.test.mjs (npm run test:unit, Node) and *.spec.js (Playwright — must mock every write, the app on :7890 is the real one)
scripts/	score_attribution.py (word-aligned cast scoring), release/minify/frontend-gate tooling
architecture/	SOPs for subsystems whose design is not obvious from code
workflows/	ComfyUI graphs for character portrait generation

Feature areas

Area	Server	Frontend	State
Voice library / clone / design	routes/library.py, core/voice.py	library.js	mature
Read Aloud (books, PDFs)	routes/reader.py	reader.js	mature
Audiobook casting (span mode: deterministic quote spans, LLM names speakers only; verify/recast)	routes/conversation.py, core/attribution_spans.py	audiobook*.js, audiobook-spans.js, audiobook-cast-spans.js	new 2026-09-19 — see architecture/audiobook-attribution.md
Characters (avatar/concept identity, appearance forms)	routes/characters.py, routes/conversation.py	library-characters.js, character-sheets.js	reworked 2026-09-19 — deterministic age/species proportions and period-only scene objects; see architecture/character-visual-identity.md
Rehearser	routes/rehearsals_db.py	rehearser-*.js	mature
Studio (phase wizard)	—	studio.js	mature
TTS engines / routing	routes/tts.py, core/routing.py	engines.js	mature
STT	routes/stt.py	—	mature
Magazines	core/magazine.py, routes/magazine.py	magazine.js, s-magazine.html	P0 + P1

Data model

Documents (books and magazines alike) live under <CONFIG_DIR>/reader_library/<doc_id>/, discriminated by meta.json → kind:

• kind: "pdf" / text — Read Aloud book. Text is extracted in the browser and not persisted; meta.json holds counts and a flat idx resume position.
• kind: "magazine" — adds layout.json, articles.json, pages/, images/, podcast/, and a per-article progress map. See architecture/magazine-ingestion.md.

Settings, presets, routes and the SQLite database also live under CONFIG_DIR, which is the writable volume.

Environment

Variable	Purpose
CONFIG_DIR	writable state (settings, library, database)
VOICES_DIR / OUTPUT_DIR	voice WAV library
TTS_CONTAINER_NAMES	containers restarted after a voice change
DOCKER_HOST	points at the socket proxy; the raw socket is not mounted
llm_url / llm_model / llm_api_key	settings.json, OpenAI-compatible LLM

Pending decisions

• Container registry: ghcr costs quota against a private package (≈427 MB per push, 500 MB free on the Free plan). Local registry vs. ghcr vs. no re-pull is unresolved — the compose files currently point at ghcr.

Known limitations

• Audiobook attribution (span mode) is measured, not perfect: on three hand-corrected novels about 90-92 % of all words end up with the right voice and ~75-80 % of spoken words with the right speaker. The references themselves contain label noise (1-2 points is not meaningful). Open, real problems: a person known only by a role that means different people in different scenes ("der Fremde") is not resolved to a character; look-alike characters in one scene are the usual mix-up; very long monologues are inconsistent in the references.
• A speech is only found where the text marks it with a recognised quotation style (»…«, „…“, “…”, «…»); a book with none of them uses the older, slower whole-passage engine.
• Character image generation needs ComfyUI/an image provider; the deterministic prompts, forms and review logic are covered by tests without one, but a real pair render is not part of the suite.
• gitea backup remote is 157 commits behind origin; main tracks gitea.
• Magazine sidebar exclusion is a narrow-column geometry heuristic (mark_sidebars), not certain — review always shows every sidebar block.
• Magazine word-level highlighting is a character-length estimate against real measured audio duration, not true forced alignment.
• No formal cross-engine OCR accuracy benchmark exists (two engines wired: ocr-baidu-unlimited-ocr primary, KDL Frontier Parser nano fallback).
• Figure description generation (Magazine review, "Describe") needs a genuinely caption-capable vision model — confirmed live that the primary engine (baidu/Unlimited-OCR) is transcription-only and returns empty content for any descriptive prompt, regardless of phrasing (three prompts tried). Falls through to vision_ocr_fallback_url/_model, which worked correctly once its container was started — that container (ocr-kdl-frontier-parser-nano) is not running by default.
• Figure boundaries are re-derived from the page render (snap_visual_bboxes) because vision OCR grounds figures loosely; it corrects roughly 100 of the 147 figure regions on c't KI Wissen 2026 and deliberately declines the rest. A pale chart or line drawing on white is below its ink threshold and keeps the engine's box — draw those by hand in review. The pass is idempotent (safe to re-run via "Fix figure boundaries" any number of times), including the one confirmed pair that cannot mathematically converge — two adjacent figures sharing a dark caption strip settle on the same answer every run rather than continuing to drift between two states.
• No npm run qa aggregate command yet; Python tests run via unittest, browser tests via playwright test.
• The app has no authentication for the browser UI and no per-user data isolation — single-tenant/LAN-trust only (one global settings/database/voice library, no accounts). See architecture/multi-tenant-and-identity.md and TODO.md's "Deferred — public multi-user readiness" section.

Next logical step

Magazine P2 per TODO.md: image descriptions shipped; remaining — table/chart handling depth, EPUB 3 Media Overlay export, and a formal OCR-engine benchmark (see architecture/magazine-ingestion.md). Then Character system, then Audiobook speaker attribution, per the user's stated priority order.