tts-voice-creator-clone-and-design-2/TODO.md

TTS Voice Creator Roadmap

This roadmap is ordered by priority. Refactors should be incremental and covered by tests before behavior is changed.

Priority levels:

• P0 — safety: prevents crashes, data loss, broken releases, or security incidents.
• P1 — high impact: improves memory use, responsiveness, and maintainability.
• P2 — quality: improves diagnostics, polish, and long-term operations.

Prioritized execution backlog

Effort estimates are for one developer familiar with this codebase: S = a few hours, M = 1–2 days, L = several days, XL = multi-week or iterative. Within each priority, tasks are ordered from smallest to largest effort. Complete the first block before investing in architecture or polish.

Current status — 2026-09-04

All P0 safety, security, data-integrity, and release-protection work is complete. P1 implementation is active: pagination, shared voice caching, bounded decoded-audio caching, background jobs, canonical character identity, forms, and the frontend namespace/manifest foundations and section-scoped lazy loading are working and regression-tested. Remaining P1 work is intentionally open where it needs a real architecture change: the larger Audiobook/Rehearser/style split.

Work order — set 2026-09-14

User-set priority for picking up feature work, in this order:

1. Magazine reader roadmap (below) — contents-view redesign is blocked on a layout decision from the user; the other open items (image/chart descriptions, legal source browsing) are actionable without one.
2. Character system — portrait/concept-sheet identity mismatch, then character-form UX polish.
3. Audiobook speaker attribution — resume from Book 1 re-measurement, then the never-yet-run Book 2 generalization regression.

Deployment and the deferred multi-user initiative below are explicitly parked, not part of this order.

P0 — do first

• S — Stop provider API keys from reaching frontend JavaScript or logs. Provider keys are masked in settings responses, probes resolve credentials server-side, and browser storage/log paths are clean. The separate external caller key is available only through deliberate regeneration, uses no-store, and is not retained by the UI.
• S — Validate uploaded filenames and paths. Upload endpoints reject traversal, path separators and control characters before deriving extensions; ZIP imports reject absolute, traversal and malformed archive members.
• M — Add upload, request-size, temporary-file and concurrent-job limits. Endpoint-specific upload/request caps, bounded temporary work, cache eviction, and bounded TTS/STT/image job pools are implemented.
• M — Add automatic backups before bulk deletes, voice moves and recasts. Make recovery explicit and verify that backups can be restored.
• M — Strictly validate custom URLs and prevent SSRF. HTTP downloads and provider/container probes share scheme, hostname and private-destination validation with explicit local-network opt-in.
• M — Add authentication and rate limits for destructive/API endpoints. External API authentication is enabled by default and destructive/general mutation limits return bounded 429 responses.
• M — Validate character sheets and voice metadata with schemas. Character/form and voice metadata are bounded and type-checked at API boundaries.
• L — Add database migrations and prompt/generation history. Preserve old record shapes and expose protected, metadata-only generation history for auditing and follow-up recovery workflows.
• L — Harden container and Docker access. The app runs non-root with dropped capabilities, no-new-privileges and a read-only root filesystem; Compose Docker access is isolated behind a minimal container proxy with only container inspection/control enabled.

P1 — next, highest user impact

• S — Re-measure audiobook segmentation against cast_run3.json. Run 3 has 2,227 segments, 1,043 dialogue segments and 585 newline-fused segments, but zero adjacent same-speaker dialogue pairs for _audiobookMergeAdjacentDialogue to merge. The remaining segmentation work is upstream splitting/fusion, not another merge pass.
• S — Finish the remaining Playwright smoke checks. Characters/Cast, profile sorting, column visibility, voice ratings, and starting/cancelling a generation job are covered.
• M — Fix the Schwester Larissa unclosed-guillemet split. Attribution normalization now repairs dialogue split before mixed German guillemet closes (»…‹ / «…›), with an epigraph regression test.
• M — Make profile portraits and character sheets use one canonical identity. Shared structured prompts now version consistent pairs as v3 and fingerprint canonical appearance fields; add cross-record validation so mismatched profile/reference files are detected even when both are current-version. The UI now flags missing, stale, changed, or explicitly inconsistent pair metadata; visual similarity itself remains a human review concern, while generated/replaced assets are now detected using stored SHA-256 digests.
• M — Add bounded duplicate-generation protection. Consistent portrait/reference, voice-design, and casting jobs now deduplicate repeated requests with bounded, expiring Idempotency-Key contracts.
• M — Improve large-table performance. Character productions and the flat voice library paginate at 50 visible rows; low-memory mode also reduces rendered rows and image work.
• M — Add explicit cache invalidation and eviction. Fish.audio search responses, persisted synthesized audio, image-generation job mappings, and browser object URLs have bounded cleanup.
• M — Add constrained-device validation. Chrome CPU throttling, a 500-character workload, browser DOM/audio/heap/image-cache bounds, and a hard 4 GiB current-release container check are covered. The 1.24.21 image served successfully at 52.54 MiB idle usage, 0.10% CPU, 6 PIDs, and OOMKilled=false; cgroup memory was confirmed as 4,294,967,296 bytes and temporary/root filesystem usage was sampled.
• L — Add background jobs and bounded queues. Image generation already has the full interactive job path, and transcription, voice design, and casting now have optional ID/status/cancel/timeout paths while preserving synchronous callers. Job metadata now persists in SQLite and marks interrupted work after restart; durable binary result storage is bounded and authenticated; outbound TTS and LLM calls now share a bounded transient-failure retry policy, and /api/jobs exposes sanitized recent job history. Completed voice-design audio and consistent-image outputs now use bounded, atomic artifact files with authenticated download URLs; transcription text remains deliberately non-persistent. Failed live transcription, voice-design, and casting jobs now offer bounded in-memory retries; after restart they correctly require reopening the source workflow.
• L — Run the Book 2 generalization regression. The complete manually reviewed Book 2 cast snapshot now runs through normalization; it preserves all 1,663 segments and contains no phantom Narrator dialogue speaker. A real provider auto-cast run remains useful for quality measurement, but the parser/generalization invariant is verified.
• L — Add character forms inside one character record. sheet.forms[] stores bounded alternate appearances, prompts, and per-form visual assets; the detail-page switcher preserves one character ID and voice, and selected forms feed prompt generation and identity fingerprints.
• L — Modularize the frontend incrementally. The shared manifest, compatibility namespace bridge, section-scoped lazy loading, and responsibility-level Audiobook/Rehearser/style extraction are complete; a full ES-module migration remains a separate follow-up.
• M — Add structured logs with request/job IDs. API/MCP requests now receive sanitized correlation IDs and duration logs; route-specific logs retain provider/job context without request bodies.
• L — Add linting and duplicate-function checks. The manifest-driven frontend syntax/declaration gate now runs locally and in the SonarQube quality workflow alongside the existing Python and bundle checks.
• Expose extracted Rehearser and Audiobook parser APIs through the window.TTVC namespace bridge.

P2 — deployment

• S — Deploy the app to correspondence-nuc (192.168.178.19), pointed at the DGX Spark's TTS engine over the network. Live since 2026-09-14, v1.27.5. NFS export set up on the DGX (/home/sparky/Media/_Sounds/TTS_Voices → 192.168.178.19 only, rw,no_root_squash), mounted read-write on the NUC at /mnt/dgx-tts-voices and persisted in /etc/fstab there — confirmed the NUC instance sees the same 344 voices the DGX does, not a separate copy. Deployed by rsyncing the working tree from the DGX (excluding .git, config/, logs/ and other local-only paths) to /home/sparky/Docker/tts-voice-creator-clone-and-design-2 on the NUC and building the image there directly — the published ghcr.io image turned out to be both several versions stale and private with no registry credentials on the NUC, not worth setting up for an image that would need re-pushing on every deploy anyway when the source is right there to build from. One real gap found along the way: vision_ocr_url/vision_ocr_fallback_url default to host.docker.internal, which resolves to whichever host the container is running on — correct on the DGX itself, silently wrong on the NUC (would have pointed at the NUC's own, nonexistent OCR engine). Pinned both to the DGX's LAN IP explicitly via environment overrides in the NUC's compose file. Also needed: the NUC's config/logs bind-mount directories were created root-owned by docker compose (this session's NUC access is a root SSH key) while the container's own app user is uid 1000 — chown -R 1000:1000 on both before the app could open its SQLite state DB. Confirmed the accepted limitation from before still holds as expected: no docker-socket-proxy on the NUC instance (nothing local for it to control, and cross-host container restart never worked anyway), so a voice change still needs a manual restart on the DGX. NUC compose file tracked here as docker-compose.nuc.yml (the plain docker-compose.yml in this repo stays the DGX's own, per its build:/dgx_net/docker-socket-proxy shape) — deployed to the NUC as its docker-compose.yml, since that host has no .git checkout of its own to diff against, only the rsynced tree. A code change means: re-rsync the tree (excluding .git, config/, logs/), docker compose build, docker compose up -d — no separate push/pull step, same as the DGX's own bind-mount-and-recreate pattern.

Deferred — public multi-user readiness (cross-app initiative)

Not scheduled; blocked on a decision the user wants to make once, shared across all self-hosted apps, not per-app. See architecture/multi-tenant-and-identity.md for the full research findings and design sketch. Do not start without a fresh planning pass — this is a multi-week, cross-repo effort.

• Design and stand up a shared identity service across all self-hosted apps (interconnected user profile, not per-app accounts), building on the existing OurFlashcards GoTrue + Postgres + PostgREST stack on the NUC rather than a new auth system per app.
• Once shared identity exists: add user_id/owner_id columns to every table in core/database.py, move voices/output/reader_library/audio-cache to per-user paths, replace the single global settings.json with per-user settings, replace the single shared external_api_key with per-user API keys.
  • Includes provider API keys (llm_api_key, groq_api_key, elevenlabs_api_key, engine_api_keys, etc.) — today one value per provider for the whole install (gitignored + 0600 settings.json, never echoed to the browser once saved). Each user needs their own key per provider, scoped under their identity- service user id; see architecture/multi-tenant-and-identity.md §1.
• Replace the global TTS/STT/image concurrency semaphores (routes/tts.py, routes/stt.py, routes/conversation.py) with per-user fair-share admission control.
• Required consent attestation + audit trail for voice cloning (decided: required). Checkbox at voice upload time, logged with timestamp/account/IP; add a user_id column to generation_history so misuse is traceable per account.
• Add a reverse-proxy/TLS edge layer (CORS, CSRF, CSP/HSTS) for internet exposure — nothing in this repo handles this today.

P2 — after behavior and safety stabilize

• M — Add structured logs with request/job IDs. Report provider, endpoint, duration and failure reason without logging prompts, keys or book contents.
• M — Separate provider health from configuration status. TTS/STT discovery now returns independent configured, available, and bounded health details; backend help text shows actionable connectivity errors.
• M — Add visible job history and retry controls. Expose recent failures and safe retry state to users.
• L — Add memory diagnostics. Show browser heap where supported, rendered rows, cache size, active jobs, server/container memory and recent operation sizes.
• L — Add linting and duplicate-function checks. Run them in CI alongside the existing compile, bundle and SonarQube checks.

P0 — Testing, data safety, and resource limits

• Establish a Playwright baseline before large refactors (app shell, bundle, i18n, clone tabs, and Rehearser parsing; installed Chromium supported).
• Add explicit Docker memory limits where appropriate; the web container defaults to an 8 GB limit and 2 GB reservation with environment overrides.
• Limit upload sizes, temporary files, and concurrent jobs. Upload/request caps, bounded caches, and bounded TTS/STT/image-generation pools are implemented; durable background-job conversion remains a separate P1 item.
• Use atomic writes for .meta.json files.
• Create automatic backups before bulk deletes, voice moves, and recasts.
• Add request-size limits, rate limits, and authentication for destructive/API endpoints.

P1 — Modular frontend architecture

• Split audiobook.js, rehearser.js, and style.css by responsibility; Rehearser casting, playback, training, voice design, I/O, library, stage, and recording blocks plus Audiobook casting, preview, export, and feature styles are isolated. Core Reader state and compatibility event wiring remain ordered in the legacy entry points.
• Extract Rehearser stage display and cast-sidebar controls into rehearser-stage.js.
• Replace broad globals with namespaces such as window.TTVC.audiobook; pure leaves, Audiobook/Reader/Rehearser feature modules, Studio, and character-library persistence now use private scopes with explicit compatibility exports. Voice-library loading, rendering, benchmarking, import workflows, character-sheet extraction, character-library rendering/actions, voice-inspector selection, shared utility helpers, cast compatibility, i18n, and settings/log UI are also scoped; legacy entry points remain only as explicit compatibility exports.
• Generate the feature-module list for loader.js and scripts/minify.mjs from one shared manifest; larger responsibility splits remain.
• Add a dependency-free frontend syntax and top-level declaration collision gate (npm run check:frontend); a full style/lint policy remains a follow-up.
• Extract Rehearser voice matching and voice design; matching/import helpers now live with the voice-picker module, while timing-sensitive DOM wiring remains in the cast renderer.
• Extract Rehearser voice-picker presentation and preview state into rehearser-voice-picker.js.
• Extract Rehearser cast view, sorting, filtering, and toolbar wiring into rehearser-cast.js.
• Extract Rehearser training transcript comparison helpers into rehearser-train.js.
• Extract Rehearser character development and backend-selection controls into rehearser-voice-design.js.
• Extract Audiobook quote segmentation and conservative turn-taking into audiobook-segmentation.js.
• Extract Rehearser library persistence and IndexedDB migration into rehearser-library.js.
• Extract Audiobook parsing and casting; parsing, segmentation, cast normalization/serialization, live cast view, LLM orchestration, preview, and export now live in focused modules. Core Reader state and compatibility wiring remain in audiobook.js.
• Extract Audiobook parsing primitives and speaker-heuristic constants into audiobook-parse.js.
• Extract Audiobook cast serialization and page-stamping helpers into audiobook-cast.js.
• Split the Performance/resource diagnostic styles into static/css/resource-diagnostics.css.
• Split background-job history and retry-panel styles into static/css/background-jobs.css.
• Split Voice Design form and sample-grid styles into static/css/voice-design.css.
• Split style.css by feature; Rehearser rules now live in static/css/rehearser.css alongside the existing diagnostics, background-job, voice-design, and Audiobook/Reader feature styles.
• Decision — do not migrate to TypeScript or rewrite the frontend until behavior is covered by tests.

P1 — Frontend performance and memory safety

• Lazy-load Audiobook, Rehearser, Studio, and image-generation modules only when opened. The shared manifest keeps the foundation in the core bundle and loads Reader/Audiobook, Rehearser, Library/Characters, and Studio groups once on first navigation; deep links load their group before activation.
• Paginate large character and voice tables; render approximately 25–50 visible rows, with a constrained-device regression covering a 250-character production.
• Cache /api/voices with a shared 30-second session cache, one in-flight request, explicit refresh, and invalidation after library metadata/folder mutations; the SQLite voice index remains authoritative.
• Lazy-load profile images and concept art.
• Update only affected table rows instead of rebuilding entire tables where the operation is row-local; benchmark, normalization, rating, and voice-note updates now patch the existing visible rows. Structural changes still trigger a full filtered rebuild so sorting, grouping, and pagination remain correct.
• Add an LRU cache for decoded Rehearser audio and remove old entries automatically. Persisted line-audio is already bounded; profile/concept and voice-library images now use a bounded viewport-aware browser cache that removes old decoded sources while retaining restoration URLs.
• Revoke discarded object URLs with URL.revokeObjectURL(objectUrl); completed synthesized voice previews and character/Rehearser local auditions now release their Blob URLs when playback ends or is replaced.
• Add low-memory mode using navigator.deviceMemory where available; shared worker pools cap at two tasks on devices reporting approximately 4 GB RAM or less.
• Disable image preloading and reduce table/cache sizes on devices with approximately 4 GB RAM. Images use lazy decoding, and character/voice pages reduce from 50 to 25 rows when navigator.deviceMemory <= 4.

P1 — Backend stability and bounded concurrency

• Introduce background jobs for voice design, transcription, casting, and image generation. Synchronous callers remain supported while each long-running path has an optional bounded background job.
• Give every job an ID, progress, cancellation, timeout, retry, and idempotency key. Failed live jobs offer bounded retries from stripped in-memory inputs; retry data is never returned or persisted, and restart correctly requires reopening the source workflow.
• Prevent duplicate generation when a user clicks twice for image, voice-design, and casting submissions.
• Centralize provider health checks and error handling. TTS, STT, and container URL probes now share the bounded core.provider_health.probe_provider mechanics and normalized connection/timeout/HTTP/JSON-shape errors; their provider-specific endpoint lists and authentication policies remain at the route layer.
• Limit concurrency to 1–2 image generations, 1–2 voice generations, and 1 STT request per user. Consistent portrait/reference rendering, preview/style/seed-finder synthesis, and the OpenAI-compatible speech proxy are capped at two active jobs; STT transcription and benchmarks are capped at one active job.
• Use bounded server queues for the implemented synchronous generation paths; durable background-job queues remain a separate outstanding item above.

P0 — Security

• Never expose provider API keys to frontend JavaScript or logs. Provider keys are masked on read and absent from browser storage; the external-client key is an intentional, explicit handoff for caller authentication and its regeneration response is non-cacheable.
• Validate uploaded filenames and paths.
• Strictly validate custom URLs to prevent SSRF, with private destinations available only when explicitly allowed for local provider/container operation.
• Add request-size limits, rate limits, and authentication for destructive/API endpoints.
• Run containers as non-root with dropped capabilities and read-only filesystems where practical.
• Isolate Docker socket access behind a minimal privileged service.

P0 — Data integrity and recovery

• Validate character sheets and voice metadata with schemas.
• Add database migrations instead of silently changing record shapes.
• Use atomic writes for .meta.json files.
• Create automatic backups before bulk deletes, voice moves, and recasts.
• Store prompt versions and generation history so bad regenerations can be audited and follow-up recovery can be built.

P0 — Testing

Automated tests

• Character parsing and phantom-speaker prevention, including malformed-response prose preservation.
• Voice-matching constraints reject invented voice IDs and avoid unnecessary duplicate assignments.
• Profile-completeness counting for empty, partial, and fully populated sheets.
• Rating persistence.
• STT endpoint normalization and backend-response shape handling.
• ComfyUI workflow submission, prompt injection, polling, and image retrieval.
• API timeout clamping and bounded upstream error text; live-provider failure drills remain environment-specific.

Playwright smoke suite

• Load the app.
• Verify the production bundle loads.
• Verify language switching and Rehearser parsing.
• Verify clone source tabs switch correctly.
• Open Characters / Cast.
• Sort the profile column.
• Hide and show columns.
• Rate a voice.
• Start and cancel a generation job.

P2 — Observability

• Add structured logs with request and job IDs.
• Report provider, endpoint, duration, and error reason clearly.
• Separate health status from “configured”; discovery APIs now preserve both states and expose sanitized probe errors.
• Add visible job history and safe retry state; the authenticated /api/jobs metadata endpoint now feeds a rendered panel with status, elapsed time, errors, cancellation, and an explicit reopen-source retry boundary.
• Avoid logging prompts, API keys, or book contents by default.

P1 — Resource limits and constrained-device testing

• Add explicit Docker memory limits; the web container defaults to mem_limit: 8g and mem_reservation: 2g, configurable per host.
• Limit upload sizes and clean converted upload intermediates; continue expanding endpoint-specific temporary-file and job limits.
• Test with Chrome DevTools-equivalent CPU throttling; desktop Playwright uses Chromium CDP at 4× CPU and asserts navigation plus bounded DOM/audio counts, with JS heap captured when Chromium exposes it.
• Test the current image in a container limited to 4 GB RAM; the app served / with a 4 GiB cgroup limit, ~50.6 MiB idle memory, 0.09% CPU, and no OOM kill.
• Monitor browser-tab memory, JavaScript heap where supported, DOM-node count, audio elements, persisted jobs, server RSS, configured-data disk, and temporary disk usage through the read-only Performance diagnostics panel.
• Add a memory diagnostics page showing cache count/size, Docker/container memory, and largest recent operations. The authenticated panel reports browser cache counters, cgroup memory current/max, server RSS, configured/temp storage, persisted jobs, and sanitized operation durations; host-wide Docker metrics remain unavailable by design.

P1 — Speaker attribution: measured state and next steps

Status 2026-09-19 — reworked, see architecture/audiobook-attribution.md. Span mode (deterministic quote spans, the LLM only names speakers) replaced the segmentation-heavy engine: Book 1 every-word-right 92.3 % (was 87.0 %), narration/dialogue agreement 97.4 % (was 93.6 %), ~10 min instead of ~4 h; Books 2/3 (never tuned on) 90.8 % / 89.9 %. The word-aligned scorer is scripts/score_attribution.py; the references have label noise, so 1–2 points is not meaningful. Everything below the line is the historical record of the classic engine's Runs 1–3.

Still open (each needs a real decision or new evidence, not more prompt tuning):

• A person known only by a role that means different people in different scenes ("der Fremde") is not resolved to a character — needs scene-level identity tracking; the label is kept as its own cast entry until the book itself ties it to a name.
• Look-alike main characters in one scene (Marcian/Zerwas/Lancorian) remain the biggest real error class (~1/5 of spoken words). Tried and rejected: reasoning mode (25× slower, no gain), larger/smaller windows, tag-based overrides, reasoning adjudication of disputes (made disputed lines worse). Untried: an explicit scene-cast constraint (characters present in the scene) built from the text.

---

Ordered follow-up queue

These follow-ups are ordered by dependency and expected benefit. Estimates assume one developer familiar with this codebase; benefit is relative user/reliability impact.

• P1 / M — High benefit — Identity-anchor regression tests. Add automated coverage proving avatar and concept-sheet requests share canonical identity anchors and persist the prompt version used for each generated asset.
• P1 / S–M — Medium benefit — Preserve dialogue-only merging. Keep the dialogue-only merge for recast/verify outputs where adjacent pairs still occur, with regression coverage for the non-dialogue boundaries.
• P1 / L–XL — Very high benefit — Rework segmentation from Run 3. The initial-cast final normalization now merges narration only, preserving dialogue turn boundaries; Run 3 showed same-speaker dialogue joins were obscuring upstream split/fusion errors. Recast/verify retains its separately tested dialogue-only merge.
• P1 / M — High benefit — Re-run Book 1. Done 2026-09-19 with the span-mode engine: overall 92.2 % (was 87.0 %), type agreement 97.4 % (was 93.6 %), ≈11 min instead of ≈4 h. See architecture/audiobook-attribution.md.
• P1 / L–XL — Very high benefit — Run Book 2 generalization. Done 2026-09-19: Book 2 90.8 % and Book 3 89.9 % overall (neither tuned on) vs 92.2 % on Book 1 — the approach generalises. Note the references carry label noise, so 1–2 points is not meaningful.
• P1 / L — Very high benefit — Unify visual identity. Portrait and concept-sheet generation now use one canonical identity description, stored prompt version, identity fingerprint, asset digests, and stale-asset review path; shared-anchor and metadata persistence are covered by automated tests.
• P2 / M–L — Medium–high benefit — Enrich character-form UX. Done 2026-09-19: forms panel (add/rename/delete/show, per-form stale flag, per-form and all-forms image generation), Physical/Clothing edits scoped to the active form. See architecture/character-visual-identity.md.

Magazine reader roadmap

Magazines are a separate Studio document type: they use a narrator and layout-aware article extraction, not character casting. The original PDF remains the visual source of truth while approved articles become linked, reflowable HTML.

P0 — Magazine foundation

• Add a Magazine project type beside Books and Theater Plays.

• [~] Add PDF/JPG/PNG import with page previews and safe local storage. (PDF + page renders done; JPG/PNG not done)

• Add OCR-engine selection from Engines > OCR & Document Parsing, with page-image input and structured output.

• Detect contents/index pages and propose article titles, authors, and page ranges.

• Build a review screen where users approve or correct article boundaries before TTS. (includes "Edit boxes" — user-requested 2026-09-14: drag an already-detected region to move or resize it, or delete it outright, with an optional re-OCR of the corrected area for text regions. "Draw it" remains the separate tool for a region detection missed entirely.)

• Preserve source coordinates for every extracted block, image, caption, and page reference.

• Generate per-article podcast scripts from a connected LLM with two or three user-picked host voices, with a Studio panel to pick hosts, generate, and play the script back turn by turn.

• Make the ingestion "watching it work" panel more visually appealing — user-requested 2026-09-14. Today it's a static side-by-side of the current page image and a plain text/label list (#mag-ingest-preview, static/sections/s-magazine.html, populated by routes/magazine.py's preview-HTML builder around line 158). Wanted instead: draw the actual detected region boxes (image/table/code/figure/etc.) live, overlaid directly on the page image as each region is classified — watching the layout engine work through the page, not just a before/after. Likely an SVG or absolutely-positioned overlay using each block's bbox (already computed and already used for the review screen's region boxes, magReviewMarkup-adjacent code in static/js/magazine.js) drawn incrementally, one box per region as it's classified, rather than all boxes appearing at once at the end. Purely cosmetic/decorative per the existing comment on that panel — must not become the only place the essential progress info (percentage, page count, done/failed) lives; that stays in #mag-ingest-status.

P1 — Magazine listening experience

• [~] Generate clean article HTML with headings, captions, figures, sidebars, and page links. (headings/byline/captions styled and page-linked, figures now shown during playback; sidebars are auto-excluded from narration rather than rendered inline, per the P2 sidebar-exclusion item below)
• Generate narrator audio per article with resume/progress and a play-all queue.
• Synchronize paragraph/sentence highlighting with playback and keep PDF and article views linked. (sentence-level; clicking a sentence or switching to the original-page view jumps to its source bbox)
• Show article images or page crops during playback, with optional caption reading. (images shown, matched to reading position; caption text is not separately spoken)
• [~] Add confidence warnings, extraction diffs, and manual text correction before audio generation. (confidence badges + block-level boundary editing done; no extraction diff view or free-text correction)
• Redesign the magazine contents view (#s-magazine, the article list for one issue). Today it is a flat list of title + page range + badge + Review button, which shows nothing of the issue itself: no page, no opening text, no figure, no sign of which articles already have audio. Requested directions, none decided:
  • Three columns. Page thumbnail · headline + opening text · the article's figures. Reads down the issue page by page in column one, with the text and the pictures beside it.
  • Synthesis state per row. A "synthesised / not synthesised" marker, the MP3's duration as mm:ss, and an inline play button — so the list doubles as the playlist instead of having to enter an article to find out.
  • Flipbook. The PDF as a page-turning spread (issuu.com as the reference), with the extracted content of the visible pages in two columns underneath it. Decide the shape first — the three ideas are not variants of one layout, and the middle one (synthesis state) is worth having under any of them. Whatever is chosen has to keep working at phone width and stay navigable by keyboard and screen reader; a flipbook in particular needs a non-gestural alternative.

P2 — Advanced magazine intelligence

• [~] Improve cross-page continuation detection and automatic sidebar/advertisement exclusion. (sidebar exclusion done via mark_sidebars, a narrow-column geometry heuristic; cross-page continuation still only uses the forward "continued on page N" reference, not "continued from")
• Add image descriptions, chart/table handling, and optional spoken figure summaries. (per-figure "Describe" in review generates a type-aware description — chart/table/code/figure each get their own prompt — via the configured vision engine with fallback; editable before approval; spoken only if "Read figure descriptions" is enabled. Live-verified against a real chart: the configured primary vision engine turned out to be a pure-OCR model that returns empty content for any non-transcription prompt regardless of phrasing — confirmed with three different prompts — so description generation only works when a genuinely caption-capable model is reachable as the fallback; documented as a config-dependent limitation in Manual.md, not a code defect.)
• [~] Add word-level karaoke timing and EPUB 3 Media Overlay export. (word-level highlighting done as a character-length estimate against the real measured audio duration, not true forced alignment; no EPUB 3 export)
• [~] Add mobile/driving mode with large controls, automatic article advance, and hands-free resume. (driving mode done: two oversized controls, reuses whatever is already playing/queued; not mobile-specific)
• [~] Add OCR-engine benchmark comparisons and per-page fallback selection. (per-page fallback done, now a real two-engine chain: vision OCR escalates when a page needs OCR or scores poorly, KDL Frontier Parser nano is the vision fallback, tesseract is the last resort; no formal benchmark)
• Evaluate additional OCR/vision candidates against the two already wired (baidu/Unlimited-OCR, KDL-Frontier-Parser-nano) and pick the best fit per task — user-requested 2026-09-14. Goal: low RAM/VRAM footprint with excellent speed and quality, for both page-text transcription and open-ended figure/chart description (the latter is the harder requirement — confirmed live that baidu/Unlimited-OCR is transcription-only and cannot do it at all, see architecture/magazine-ingestion.md and CHANGELOG 1.29.0). Candidates to install as Docker containers alongside the existing engines, each added through the Engines tab (routes/docker.py's _LOCAL_CONTAINER_DEFS, role ocr) so they're selectable/connectable/benchmarkable like any other engine, not hardcoded: - [ ] docling — document conversion/layout toolkit. - [ ] PaddleOCR — established general-purpose OCR. - [ ] DeepSeek-OCR — vision-language OCR model. - [ ] Qwen3-VL-30B-A3B-Instruct-FP8 — general vision-language model; check whether it fits the RAM/VRAM budget at 30B-A3B FP8 (likely the heaviest candidate — weigh against the "low footprint" goal specifically) and whether it's meaningfully better at open-ended description than the nano-class candidates above. For each: measure page-transcription accuracy and figure/chart-description quality against the existing c't KI Wissen 2026 test corpus, plus idle/active VRAM footprint and latency, before deciding what replaces or joins the current primary/fallback pair.

P2 — Finding magazines to read

• Browse and search online PDF magazine sources from inside the app, so an issue can be found and pulled in without leaving for a browser and uploading by hand. Wanted: a catalogue of sources, a search across them, and a one-click "ingest this issue". Blocked on a source decision, and that decision is the whole task. The two sites named as examples (freemagazines.top, pdf-magazines.org) redistribute current commercial titles — c't, BBC Wildlife, AirForces Monthly — without the publishers' licence, so building a scraper against them is out: it is copyright infringement regardless of who presses the button, and both would be trivially blocked or taken down anyway. Sources that can be built against, in rough order of usefulness here:
  • OPDS catalogues — the open standard for exactly this. Calibre-Web, Kavita and Komga all serve OPDS, so a generic OPDS client also covers the user's own library on the NAS. Start here.
  • Internet Archive (archive.org API) — a genuinely large public-domain and openly-licensed periodical collection, with a documented search API and no scraping needed.
  • Publisher-run free issues and RSS/Atom feeds — heise gives away sample c't issues, many trade titles publish a free archive. Per-publisher work, but legitimate and high quality.
  • Open-access journals (DOAJ, arXiv, PubMed Central) — if the appetite is reading rather than magazines specifically, these are the largest legal corpus by far and all have real APIs. Whatever is added, check licensing, rate limits and terms per source before wiring it (core rule 8), honour robots.txt, and keep each source behind its own provider interface so one going away does not break the rest.

Three full regression runs of Der Sturm against the hand-corrected reference. Everything below is measured, not estimated. Artefacts live in config/regression_snapshots/der_sturm/ (*_manual.json are the human reference — never overwrite them).

Run	Segments	Headline agreement	Fully correct	Wrong speaker	Unmatched
1	2143	72.6%	54.2%	438	544
2	2273	77.7%	52.8%	345	727
3	2227	78.7%	52.2%	314	750
human reference	2110	—	—	0	—

Read the headline with care. Agreement is computed only over rows whose text matches the human's, and that pool keeps shrinking (1598 → 1545 → 1475). Absolute correct segments went 1160 → 1200 → 1161. The rising headline is largely a shrinking denominator; true end-to-end accuracy has drifted slightly down.

What genuinely improved

• Unknown speakers 121 → 5 → 8; the addressee/Unknown prompt rewrite did not overcorrect (369 after cast collapsed to 19 after one recast pass).
• Wrong speaker names 438 → 345 → 314 — consistent, real.
• Verify pass now completes (~65 LLM calls) instead of never finishing.

The blocker: segmentation, not attribution

750 of ~1060 problem segments are text that doesn't match the human's at all: roughly 400 split finer than the human, ~210 merged where the human split, ~70 OCR/wording differences. Attribution work cannot fix these.

• fused dialogue segments hit 212 (human: 6, run 2: 59). These are _audiobookMergeAdjacentDialogue joins using \n\n; 137 of them match no human segment. Splitting them back was tested and is worse (52.2% → 43.4%), so the merge is not the primary cause — but merging blocks the human kept separate is contributing.
• The merge was predicted (on run 2 data) to cut unmatched 727 → 588. On run 3 it went to 750. That prediction did not hold and needs re-deriving on run 3.
• Fixed: attribution normalization repairs the Schwester Larissa epigraph when the LLM splits it before a mixed guillemet close; the regression is covered by tests/test_conversation.py. (unclosed-guillemet handling).

Next steps, highest value first

1. Attack segmentation directly — it is now the dominant error class. Run 3 has no merge candidates, so inspect the earlier dialogue split/fusion stages and the 585 already-fused segments instead of changing _audiobookMergeAdjacentDialogue.
2. Preserve the dialogue-only merge for recast/verify outputs where adjacent pairs can still occur; do not claim it explains the run-3 mismatch.
3. Only then re-run book 1. Re-running before segmentation moves will re-measure rules already known not to shift the number that matters.
4. Book 2 generalization run never started. Reference is backed up at config/regression_snapshots/book2/. Its point is to test whether these rules generalise or were overfit to book 1 — still unanswered, and the confidence scorer in particular was tuned solely on book 1.

Character profiles

48 human vs 64 auto (run 3), 35 matched, 13 missed. Auto over-generates and got worse than run 2's 51. Generic role labels (Fremder 19 lines, Vampir 19, Inquisitor 15, Ork 13, and the adverb Unsicher) still become cast members with their own voices; the appositive resolver only fires where the book supplies grammatical evidence (Darrag der Schmied), and pre-reveal aliases like "der Fremde" deliberately withhold it.

Harness notes (cost real time; do not relearn)

• Casting auto-starts audiobookAutoRepairCast 800 ms after finishing, which re-sets _audiobook.running. Guards now explain the refusal instead of returning silently, but an automated driver must still wait for running to clear and treat a sub-60s phase as a bailout, not a success.
• Stall detection must key on the passage counter (#ab-cv-count). segments, done and unknown do not move during a verify pass — watching them killed a healthy run twice.
• Character Sheets only saves after its passage loop ends; poll /api/characters until records appear before closing the browser, or a 4.4 h run is discarded.

P1 — Character forms (implemented 2026-09-04)

Shape-shifting characters (Zerwas: human / vampire / wolf-demon, aliases "Fürst der Nacht", "der Wolfsmensch") need more than one appearance per character.

Decided: forms live INSIDE one character record, not as separate records — one voice, one attribution target, so cast lines saying "Zerwas" always resolve. The sheet gains a form switcher; each form carries its own physical, image_prompt and portrait. Detection is automatic: the sheet prompt asks the LLM to identify whether a character has distinct forms and describe each, so it generalises to any book (werewolves, disguises, transformations) without per-book rules.

Implemented as bounded sheet.forms[] data inside the existing character record. The extraction prompt asks for distinct forms, the detail page switches the active appearance, selected forms feed prompt/fingerprint generation, and generated/uploaded portrait/reference assets remain attached to that form. One character ID and one voice are preserved for attribution. A future polish pass can add richer inline form editing and per-form bulk-generation UX.

P1 — Portrait / concept-sheet identity mismatch

The avatar and the concept sheet can show two different people (Lancorian: dark-haired man vs white-haired mage). Cause is understood: image_prompt has two sources — the LLM writes it as one of the 53 character-sheet fields, while _build_character_portrait_prompt builds it deterministically from the same structured facts the concept sheet uses. The deterministic one keeps both images consistent; the LLM-written one omits identity anchors (age, hair, build) so the model invents them.

Status — resolved 2026-09-19

Verified against the code and covered by tests (see architecture/character-visual-identity.md):

• One canonical identity description feeds both the avatar and the concept sheet (_character_visual_facts; portrait and reference prompts are built deterministically, the active appearance form is overlaid first).
• image_prompt is no longer authored by the LLM during sheet extraction (server and browser prompts); the deterministic prompt is generated right after extraction, before avatar-only generation and before per-form generation. Hand-edited prompts are never overwritten.
• Prompt version, identity fingerprint and asset digests are stored with every generated pair — for the base appearance too (they used to be lost on reload) and per form.
• Review indicator: per-character banner and GET /api/characters/visual-pair-review, now form-aware; a pair is also flagged when one image was regenerated alone.
• Tests: server/browser fingerprint parity, prompt anchors shared by both builders, review endpoint (base + forms), and Playwright persistence tests that fail on the previous code.

Still true, and deliberately not "fixed": the resolved prompt text is not stored per image (only its version and the fingerprint of the facts it was built from).

Known regression from the 2026-09-02 batch

Regenerating the top-10 profiles from evidence windows (passages around each character's own dialogue) produced weaker physical text than the app's full-book pass — Lancorian's became behaviour ("peers through small windows, moves nimbly on spiral staircases") rather than appearance, where the older sheet had "shoulder-length white hair, faded scar across forehead". Prefer the app's own Character Sheets flow over evidence-window shortcuts when appearance quality matters.