tts-voice-creator-clone-and-design-2/PROJECT_MAP.md

# Project map

Living index of where things are and what state they are in.
Update in the same commit as the change it describes.

## Current phase

Magazine projects, **P0 and P1 complete** (frontend reachable at Studio → Magazines). Audiobook/Rehearser/voice-library
features are mature and in daily use. Latest release v1.59.0: global "All characters" view in the sidebar and per-book
characters inside Studio, untitled-page cleanup with an exclude option, duplicate/clean tools. v1.58.0 added the
driving-mode menu, auto-advance, article index with page labels, merge suggestions, distinct face profiles, English
prompt translation, vision-response cache and the review Approve fix. v1.57.0 added driving mode with sentence/word-level
sync via `tts-aligner`.

## Deployment

Two self-hosted Docker instances, both port 7890:

- **DGX Spark** (`192.168.178.8`) — primary. `docker-compose.yml` builds locally and tags the result
  `ghcr.io/martin-b78/tts-voice-creator-clone-and-design-2`, but that registry copy is not kept current on every
  deploy (several versions behind as of 2026-09-14) — treat it as a snapshot, not a source of truth. Python and
  static files are bind-mounted, so a code change needs only a container recreate; rebuild only when `Dockerfile`
  or `requirements.txt` change.
- **`correspondence-nuc`** (`192.168.178.19`) — secondary, live since 2026-09-14. No TTS/OCR engines of its own;
  every generation call crosses the LAN to the DGX (`docker-compose.nuc.yml`, tracked here, pins
  `vision_ocr_url`/`vision_ocr_fallback_url` explicitly to the DGX's IP — their default resolves relative to
  whichever host the container runs on, silently wrong here otherwise). Voice library is the DGX's own directory,
  NFS-mounted read-write (`/etc/exports` on the DGX, `/etc/fstab` on the NUC) — never a separate copy that could
  drift. Deployed by rsyncing the working tree (excluding `.git`, `config/`, `logs/`) and building on the NUC
  itself, since the `ghcr.io` image is both stale and private with no registry login there. See TODO.md's
  "P2 — deployment" entry for the full setup and the one accepted limitation (voice-change auto-restart of the
  DGX's TTS containers doesn't reach across hosts — manual restart on the DGX, same as before this existed).

- **`tts-aligner`** (forced-aligner sidecar, `aligner/`) exists only in the DGX `docker-compose.yml`. The NUC compose
  file deliberately omits it: with `ALIGNER_URL` unset the app quietly falls back to estimated word timing, so the NUC
  works unchanged. Known limitation: no measured word times on the NUC.
- **Rollback** (either host): check out the previous tag (`git checkout v1.58.0`; on the NUC re-rsync that tree), then
  `docker compose up -d --force-recreate` (rebuild only if `Dockerfile`/`requirements.txt` changed). Stop the aligner
  with `docker compose stop tts-aligner`; `audio/<id>.timings.json` version 2 files stay readable by v1 readers.

## Layout

| Path | Role |
|---|---|
| `server.py` | FastAPI app, router registration, auth middleware |
| `core/` | domain logic — config, database, audio, voice, routing, magazine |
| `routes/` | HTTP boundary, one module per feature area |
| `static/js/` | frontend, one module per feature area (no build step for most) |
| `tests/` | `test_*.py` (unittest, run with `python3 -m unittest tests.test_x`), `*.test.mjs` (`npm run test:unit`, Node) and `*.spec.js` (Playwright — must mock every write, the app on :7890 is the real one) |
| `scripts/` | `score_attribution.py` (word-aligned cast scoring), release/minify/frontend-gate tooling |
| `architecture/` | SOPs for subsystems whose design is not obvious from code |
| `workflows/` | ComfyUI graphs for character portrait generation |

## Feature areas

| Area | Server | Frontend | State |
|---|---|---|---|
| Voice library / clone / design | `routes/library.py`, `core/voice.py` | `library.js` | mature |
| Voice archetype gallery (40 own German-first design voices, cached previews) | `core/voice_archetypes.py`, `routes/archetypes.py` | `voice-archetypes.js`, `s-design.html` | **new** — see [architecture/voice-archetypes.md](architecture/voice-archetypes.md) |
| Read Aloud (books, PDFs) | `routes/reader.py` | `reader.js` | mature |
| Conversation metrics in the Latency column (TTFB charts STT/LLM/TTS, token usage) | `routes/conversation.py` (`_usage_from_chunk`, `_post_llm_stream`) | `conversation.js`, `s-conversation.html` | **new 2026-10-06** — in-memory, last 50 turns |
| Audiobook casting (span mode: deterministic quote spans, LLM names speakers only; verify/recast) | `routes/conversation.py`, `core/attribution_spans.py` | `audiobook*.js`, `audiobook-spans.js`, `audiobook-cast-spans.js` | **new 2026-09-19** — see [architecture/audiobook-attribution.md](architecture/audiobook-attribution.md) |
| Characters (avatar/concept identity, appearance forms) | `routes/characters.py`, `routes/conversation.py` | `library-characters.js`, `character-sheets.js` | **reworked 2026-09-19** — see [architecture/character-visual-identity.md](architecture/character-visual-identity.md) |
| Rehearser | `routes/rehearsals_db.py` | `rehearser-*.js` | mature |
| Studio (phase wizard) | — | `studio.js` | mature |
| TTS engines / routing | `routes/tts.py`, `core/routing.py` | `engines.js` | mature |
| STT | `routes/stt.py` | — | mature |
| **Magazines** (incl. paragraph fusion, acronym glossary, spellcheck: `core/pronunciations.py`, `core/spelling.py`) | `core/magazine.py`, `routes/magazine.py` | `magazine.js`, `s-magazine.html` | **P0 + P1** |

## Data model

Documents (books and magazines alike) live under
`<CONFIG_DIR>/reader_library/<doc_id>/`, discriminated by `meta.json → kind`:

- `kind: "pdf"` / text — Read Aloud book. Text is extracted in the browser and
  not persisted; `meta.json` holds counts and a flat `idx` resume position.
- `kind: "magazine"` — adds `layout.json`, `articles.json`, `pages/`, `images/`,
  `podcast/`, `dictionary.json` (words kept as valid by the reviewer or the spelling agent),
  per-article `glossary` (acronym → spoken form, in `articles.json`), and a per-article `progress` map. See
  [architecture/magazine-ingestion.md](architecture/magazine-ingestion.md).

Settings, presets, routes and the SQLite database also live under `CONFIG_DIR`,
which is the writable volume.

## Environment

| Variable | Purpose |
|---|---|
| `CONFIG_DIR` | writable state (settings, library, database) |
| `VOICES_DIR` / `OUTPUT_DIR` | voice WAV library |
| `TTS_CONTAINER_NAMES` | containers restarted after a voice change |
| `DOCKER_HOST` | points at the socket proxy; the raw socket is not mounted |
| `ALIGNER_URL` | forced-aligner sidecar for word timing (`http://tts-aligner:8000`; empty = off) |
| `llm_url` / `llm_model` / `llm_api_key` | settings.json, OpenAI-compatible LLM |

QA: `npm run qa -- --fast` runs lint, unit, security, build checks (see `architecture/qa.md`); optional `--sonar` needs `SONAR_HOST_URL` + `SONAR_TOKEN` in the shell/CI secrets.

## Pending decisions

- Container registry: ghcr costs quota against a private package (≈427 MB per
  push, 500 MB free on the Free plan). Local registry vs. ghcr vs. no re-pull is
  **unresolved** — the compose files currently point at ghcr.
## Known limitations

- Live data changes made on the DGX only (not on the NUC, which has its own DB/config): book KONTROLLE removed, 5 duplicate
  characters resolved, untitled-page cleanup applied (backups under `config/backups/`).
- 18 re-rendered/stale audio articles are pending the owner's voice choice.
- `scripts/clean_character_records.py` was applied for 5 pairs; the 17 remaining duplicate-sheet pairs need owner decisions.
- Audiobook attribution (span mode) is measured, not perfect: on three hand-corrected novels about
  90-92 % of all words end up with the right voice and ~75-80 % of spoken words with the right
  speaker. The references themselves contain label noise (1-2 points is not meaningful). Open, real
  problems: a person known only by a role that means different people in different scenes ("der
  Fremde") is not resolved to a character; look-alike characters in one scene are the usual mix-up;
  very long monologues are inconsistent in the references.
- A speech is only found where the text marks it with a recognised quotation style (»…«, „…“,
  “…”, «…»); a book with none of them uses the older, slower whole-passage engine.
- Character image generation needs ComfyUI/an image provider; the deterministic prompts, forms and
  review logic are covered by tests without one, but a real pair render is not part of the suite.
- `gitea` backup remote is 157 commits behind `origin`; `main` tracks `gitea`.
- Magazine sidebar exclusion is a narrow-column geometry heuristic
  (`mark_sidebars`), not certain — review always shows every sidebar block.
- Magazine spelling relies on pyspellchecker word lists (installed in the container, not necessarily on a host
  Python): German has no morphological analysis, so unusual compounds beyond a two-part split are flagged and
  need *Keep*. Paragraph fusion is heuristic (`paragraph_continues`); a wrongly joined pair is fixed in review.
- Magazine word-level highlighting is a character-length estimate against
  real measured audio duration, not true forced alignment.
- No formal cross-engine OCR accuracy benchmark exists (two engines wired:
  ocr-baidu-unlimited-ocr primary, KDL Frontier Parser nano fallback).
- Figure description generation (Magazine review, "Describe") needs a
  genuinely caption-capable vision model — confirmed live that the primary
  engine (baidu/Unlimited-OCR) is transcription-only and returns empty
  content for any descriptive prompt, regardless of phrasing (three prompts
  tried). Falls through to `vision_ocr_fallback_url`/`_model`, which worked
  correctly once its container was started — that container
  (`ocr-kdl-frontier-parser-nano`) is not running by default.
- Figure boundaries are re-derived from the page render (`snap_visual_bboxes`)
  because vision OCR grounds figures loosely; it corrects roughly 100 of the
  147 figure regions on `c't KI Wissen 2026` and deliberately declines the
  rest. A pale chart or line drawing on white is below its ink threshold and
  keeps the engine's box — draw those by hand in review. The pass is
  idempotent (safe to re-run via "Fix figure boundaries" any number of
  times), including the one confirmed pair that cannot mathematically
  converge — two adjacent figures sharing a dark caption strip settle on the
  same answer every run rather than continuing to drift between two states.
- No `npm run qa` aggregate command yet; Python tests run via `unittest`,
  browser tests via `playwright test`.
- The app has no authentication for the browser UI and no per-user data
  isolation — single-tenant/LAN-trust only (one global settings/database/voice
  library, no accounts). See `architecture/multi-tenant-and-identity.md` and
  TODO.md's "Deferred — public multi-user readiness" section.

## Next logical step

Magazine P2 per TODO.md: image descriptions shipped; remaining — table/chart
handling depth, EPUB 3 Media Overlay export, and a formal OCR-engine
benchmark (see architecture/magazine-ingestion.md). Then Character system,
then Audiobook speaker attribution, per the user's stated priority order.