mARTin-B78/tts-voice-creator-clone-and-design-2

mARTin-B78 9ad881ada8 feat(magazine): manage OPDS catalogues in the app, env-based login, result paging

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

2026-09-30 11:03:03 +02:00

72 KiB

Raw Blame History

TTS Voice Creator Roadmap

This roadmap is ordered by priority. Refactors should be incremental and covered by tests before behavior is changed.

Priority levels:

P0 — safety: prevents crashes, data loss, broken releases, or security incidents.
P1 — high impact: improves memory use, responsiveness, and maintainability.
P2 — quality: improves diagnostics, polish, and long-term operations.

Prioritized execution backlog

Effort estimates are for one developer familiar with this codebase: S = a few hours, M = 1–2 days, L = several days, XL = multi-week or iterative. Within each priority, tasks are ordered from smallest to largest effort. Complete the first block before investing in architecture or polish.

Current status — 2026-09-04

All P0 safety, security, data-integrity, and release-protection work is complete. P1 implementation is active: pagination, shared voice caching, bounded decoded-audio caching, background jobs, canonical character identity, forms, and the frontend namespace/manifest foundations and section-scoped lazy loading are working and regression-tested. Remaining P1 work is intentionally open where it needs a real architecture change: the larger Audiobook/Rehearser/style split.

Work order — set 2026-09-14

User-set priority for picking up feature work, in this order:

Magazine reader roadmap (below) — contents-view redesign is blocked on a layout decision from the user; the other open items (image/chart descriptions, legal source browsing) are actionable without one.
Character system — portrait/concept-sheet identity mismatch, then character-form UX polish.
Audiobook speaker attribution — resume from Book 1 re-measurement, then the never-yet-run Book 2 generalization regression.

Deployment and the deferred multi-user initiative below are explicitly parked, not part of this order.

Magazine MP3 quality pass — set 2026-09-23

Work these in order; each item needs regression coverage against German and English magazine text before it is marked complete.

Remove printed line-break hyphens before narration. Fuse words split only because they crossed a printed line, while preserving real compounds such as KI-Modell.
Expand German and English acronyms for TTS. Keep a maintainable pronunciation database and expand whole tokens such as KI, AI, LLM and API in spoken text without changing the displayed article.
Conservatively restore OCR-missed German umlauts. Use a spellchecker only where an umlaut variant is known and unambiguous; never rewrite names or unknown technical vocabulary on a guess.
Keep multi-page articles as one playlist item. Prefer contents-page anchors and explicit continuation markers, and prevent running heads or internal headings from starting a new article.
Support two-voice printed interviews. Detect repeated speaker labels, use the selected narrator and optional second interview voice, and render the turns as one article.

Magazine MP3 quality pass, part 2 — set 2026-09-28

Findings from the first whole-issue quality run. Every issue found must either be fixed by an agent (vision re-read / LLM adjudication, never a blind rewrite) or be marked on the page for a human. Order of work:

Fuse one article's paragraphs into single narration blocks. A paragraph that continues into the next column or onto a later page is separate regions today, so playback and the MP3 split the sentence at the seam. Join the fragments into one block (keeping the source regions for highlighting) before sentence splitting.
Context-aware acronym converter. Use the article's language (not a per-fragment guess) and a per-article glossary: known acronyms come from the pronunciation database, unknown ones are classified by the LLM from their sentence (expansion / spelled letters / read as a word) and stay editable.
German and English spellchecker with fix-or-mark. Per-block language, tokens unknown in both dictionaries and the user's own dictionary are findings; a vision re-read and an LLM adjudication fix what they can prove, everything left is marked on the page with suggestions (accept / ignore / add to dictionary).

Magazine: verify, re-import and compare — set 2026-09-28

The current issue (c't KI-Wissen 2026) is the test bed. The point is to measure whether the newest rules beat what was built up by hand, and to feed every difference back into the rules. Do these in order.

A — Finish verifying this magazine (automatic part). 2026-09-28: full Auto-fix with the language model (thinking off, ~4s/call) and vision re-read ran to completion: 94 misspellings fixed, 77 marked, 52 blocks linked, 27 pages still need a human decision. Snapshot after this run is config/magazine_backups/bd871c7c22f0449984d3b0e3d603d6d6-after-autofix-20260928-1810.

Healing re-run with both G fixes, 2026-09-29 (~17 min; the same "Auto-fix all pages" the user runs by hand, triggered once to apply the two fixes retroactively): re-running clean_up_regions on the result found 0 additional duplicates, confirming the 6 residual ones found before this run are gone. Also surfaced (pre-existing behaviour, not part of either fix) one duplicate article pair sharing page 75 (article-020 "AVRunner im nächsten Desinfec't" / article-021 "Wofür Softwareentwickler KI-Tools nutzen", both approved) -- added to the pages needing review, since resolving two approved articles that both claim the same page is a human decision. Still open: a human must work the 27 flagged pages (same list as before, plus this new duplicate-article pair), approve/reject the not-reviewed articles, and decide which of article-020/021 page 75 actually belongs to.
B — Make a backup copy. Taken 2026-09-28 after A's automatic pass (see snapshot above; layout, articles, dictionary, quality report; 62 articles). Audio stays in the earlier 17:33 snapshot.
C — Clone the PDF and re-import it. Cloned as doc 345b2a841f104b03b3cdf61faceb66cc from the same source.pdf, ingested from nothing (no reuse of the reviewed layout).
D — Process the new import to the maximum. Full ingest (~2h27m) + Auto-fix with LLM and vision (~35m) on the clone. Result: 60 pages need a decision (vs. 27 for the hand-reviewed magazine) -- expected, since the backup carries months of manual box/order/approval fixes a from-scratch run cannot reproduce.
E — Compare the new result with the backup. Ran scripts/compare_magazines.py; report saved at architecture/evidence/compare-345b2a84-vs-backup-20260928.{md,json}. Headline: text similarity 0.937, box recall/precision 0.88/0.85, reading-order agreement 0.967, 51/56 articles matched.
F — Show the differences and decide what is better. Decided not needed: a side-by-side review UI adds no value the markdown/JSON report from E doesn't already give for this workflow.
[~] G — Adjust the rules so the next magazine needs less work. Two universal fixes landed 2026-09-28/29, each traced from the worst-differing pages E flagged, not from one page in isolation:
1. Vision refusal hallucinations (pages 82, 87, 124, 202): the vision OCR engine sometimes answers with a sentence describing its own failure ("The image is too blurry to recognize...", "The Ground Truth image displays...") instead of leaving an unreadable photo/screenshot region empty, tiled across dozens of regions and narrated as real page text. Fixed in core/magazine.py (_drop_repeated_region_text/_looks_like_vision_refusal): text repeated verbatim across 3+ regions that also reads like a refusal is blanked before it becomes blocks. Verified live: re-OCR of page 82 discarded 29 hallucinated regions, page 87 discarded 122.
2. Duplicate check runs too early (pages 31, 42, 103, 151, 152, and 6 more on the already-reviewed backup): the duplicate-box detector (clean_up_regions) only ran once, before the box-fitting/seam- mending loop that can grow or merge a box afterward -- a box that becomes a duplicate only because of that growth was left unmarked, so its text was narrated twice. Fixed by re-running the check after that loop at ingest, and again after recover_missing_text in the whole-issue quality run (routes/magazine.py). Verified live: re-running the check on the current layout alone found 13 such duplicates in the fresh import and 6 in the already-reviewed backup (the bug predates this fix, so the backup itself is not clean either -- its next Auto-fix/quality run will self-heal now). Regression tests: tests/test_magazine.py::VisionOcrRefusalHallucinationTests, tests/test_magazine_routes.py::QualityRunJobTests::test_a_duplicate_created_by_recovering_missing_text_is_still_caught, tests/test_magazine_routes.py::IngestSyncDuplicateRecheckTests.
D→E re-run with both fixes, 2026-09-29 (re-running Auto-fix on the same clone; report at architecture/evidence/compare-345b2a84-vs-backup-20260929-after-fixes.{md,json}): pages needing a decision 60→53, overlapping pairs 188→14, clipped edges 127→96 -- real improvement. Narratable word count also dropped slightly (68680→64946), consistent with duplicate text no longer being counted twice. Caveat: this re-run's own "re-reading suspicious words" step calls the vision engine again on flagged boxes, and that engine is not deterministic -- pages 82 and 87 got worse on this pass (re-hallucinated differently) for reasons unrelated to either fix, which is why some individual pages regressed even as the totals improved. A from-scratch re-ingest (true D) would be a cleaner test but costs ~2h27m; not run again this pass.

Still open: page 44's screenshot-in-article OCR (garbled but partly real transcription of a mockup image) was looked at and deliberately left alone -- a known-word-ratio filter would also catch genuine quoted text, and no reliable rule to tell them apart was found; page 3's biggest "difference" turned out to be baseline swallowing real editorial text into a figure's alt text, i.e. new is better there, not a regression to fix. A decision on whether to run the healed quality pass on the live production magazine (6 residual duplicates found there, not yet corrected) is still open.

Ingest now runs Auto-fix itself, 2026-09-29. A fresh import used to stop laid out but not worked, leaving Auto-fix for a separate manual click; _ingest_sync now runs _quality_run_sync itself under the same job (routes/magazine.py), so a fresh import opens already-fixed. An Auto-fix failure is reported as an ingest failure without losing the ingest's own result. Regression tests: tests/test_magazine_routes.py::IngestSyncAutoQualityRunTests.

Clean from-scratch re-ingest, 2026-09-29 (doc 64fe2b4def774c29a44c14a5c4d35571, both fixes and the auto-chained Auto-fix in place; ~2h39m combined; report at architecture/evidence/compare-64fe2b4d-vs-backup-20260929-fresh-reingest.{md,json}): the ingest-time ordering fix is proven in isolation -- re-running clean_up_regions on the finished layout found 0 additional duplicates (13 were found this way on the previous, unfixed clone). The hallucination filter also fired identically (29 + 122 regions discarded on pages 82/87, matching the first run). But the whole-magazine comparison against the backup came back worse than the previous clone's (76 pages needing a decision vs. 53, overlapping pairs 170 vs. 14) for two traced, non-regression reasons: (1) the vision OCR engine is not deterministic across independent full-magazine passes -- already seen with pages 82/87 specifically; (2) scripts/compare_magazines.py was comparing pages the backup has hand-marked as full-page advertisements (skipped by its quality run) against the same pages freshly OCR'd in the new import, which had never had that one-time manual step -- a review-state difference, not a pipeline one. Fixed (2): the compare tool now excludes any page either side calls an advertisement from every difference statistic, reporting the count separately instead (pages_skipped_as_advertisement; 18 on this pair). Regression tests: tests/test_compare_magazines.py (advertisement handling). Re-run with the fix: text similarity 0.837→0.870, box precision 0.62→0.66 -- a real but partial improvement, confirming (1) still accounts for most of the remaining gap. Not fixable here: engine non-determinism is a property of the external OCR service, not this pipeline; a fair A/B would need the same fixed OCR responses on both sides, which the current tooling has no way to freeze.

Voice Studio (github.com/debpalash/voicestudio) — evaluation, container, comparison

1 — Study Voice Studio. 2026-09-29: read the repo (README, LICENSE-NOTICE.md, feature-catalog.md, engines/README.md, speech-platform.md, performance.md, mcp.md, docker.md) and the GitHub API record (46.2k stars, 3,470+ commits, pushed today, license AGPL-3.0-only — quoted exactly from the API, not the fetch tool's summary). AGPL-3.0: commercial use of the app is explicitly allowed; the Affero clause only applies to a modified version offered to others over a network. No scraping beyond the repo's own published files and its /health endpoint.
2 — Run it as a Docker container. Already running (user's own doing, confirmed live, not started by this session): http://192.168.178.8:3900/ on the same DGX Spark this app runs on -- GET /health → {"status":"ok","device":"cuda (NVIDIA GB10)","version":"0.5.6"}. Real, non-hypothetical risk noted in the comparison doc: it shares the DGX's one GPU with this app's Qwen3-TTS/vision-OCR/LLM calls, so a heavy VoiceStudio job and a magazine Auto-fix run at the same time will contend for VRAM.
3 — Detailed comparison with this software. Written up at architecture/voicestudio-comparison.md: side-by-side table, what each is ahead on, the shared-GPU risk, and a narrow recommendation (not a replacement -- the magazine pipeline has no counterpart in VoiceStudio to compare against; worth revisiting only as an alternate TTS backend for a second opinion on narration quality, or its MCP server for a quick engine A/B). No integration started.

UI/UX and feature ideas from the VoiceStudio comparison — set 2026-09-29

From the deep-dive UI/UX comparison (architecture/voicestudio-comparison.md, "UI/UX and feature-parity deep dive"). User picked these to build; telephony integrations (Twilio/ Plivo/Telnyx) were in the same gap list but deliberately not picked -- leave that one alone.

Projects menu: group Audiobooks, Magazines, Script Rehearsals and Video Dubbing under one "Projects" nav section. Currently Reader (audiobook), Magazine and Rehearser are three separate top-level nav-tree-head sections in the sidebar (static/index.html/served nav — see data-nav-section="s-reader" / "s-magazine" / "s-rehearser"). Add a new "Projects" parent with these three as submenu items, plus a fourth submenu item for the new Video Dubbing feature below (a placeholder entry is fine until that feature exists, but say so in its UI rather than linking to nothing silently). Update Manual.md's navigation description and any Playwright specs that assert the current nav structure.
Video Dubbing feature. New capability, not currently in this app at all: upload a video/audio file or paste a URL, transcribe it, translate to a target language (offer a local/offline model, not only an LLM call -- see VoiceStudio's NLLB-200 option), then re-synthesize speech timed to the original and mux it back into the video. This is a multi-day feature (needs its own data model, a job pipeline, and a review UI, likely similar in shape to the magazine pipeline's box/undo conventions) -- plan it properly (goals, data model, API contract, error/offline behaviour) before implementing, per the planning workflow, rather than bolting it onto an existing route file.
Live STT Dictation. A global hotkey that starts listening, transcribes speech, and inserts the resulting text into whatever field/app currently has focus -- distinct from the existing "global hotkey daemon" in Connect Apps, which triggers TTS playback, not speech-to-text insertion; do not conflate the two or reuse its name for this. Needs a small native/OS-level component (focus capture, text insertion) similar in spirit to VoiceStudio's Rust control sidecar -- decide the platform(s) to support first (this household's own machines) rather than building a generic cross-platform solution speculatively.
Onboarding / setup wizard. Either (a) detect the local stack already configured (Qwen3-TTS, vision OCR, the LLM router -- these already have URLs in Settings) and walk a new install through pointing at them with a live check per engine, or (b) a single one-click installer/setup flow that provisions everything needed, mirroring VoiceStudio's tiered model packs (Fast/Balanced/Quality/Max, disk size shown before committing). Decide which of (a)/(b) fits this app's actual deployment story (self-hosted, one household, engines usually already running on the DGX) before building -- (a) is likely the better fit and the smaller change; (b) only makes sense if this app starts bundling/installing its own models rather than calling existing engine URLs.
Global command palette + cross-app "recent" history. A Ctrl/Cmd+K palette to jump to any section/action, and a unified "recent generations" view spanning Voice Clone/ Design/Conversation/Magazine/Reader, not just each feature's own history (the magazine's 20-step undo stack stays separate -- this is about finding a past generation across features, not undoing one). Survey existing per-feature history/list endpoints first (/api/*/history-shaped routes already exist for several features) so the palette reuses them rather than duplicating state.
Curated voice-design preset gallery, integrated into the existing preset system. s-design.html already has save/load/delete for named presets (design-preset-*, routes behind design-preset-save/-load/-delete) -- add a small built-in gallery of named example personas (in VoiceStudio's style: "The Librarian", "The Anchor", …), each with a one-line pitch and a pre-rendered demo clip, "Use this design →" filling the description box from it. Ship a handful covering common needs (audiobook narrator, news anchor, casual conversational, storyteller) rather than trying to match VoiceStudio's exact catalog. Generate and store the demo clips once (not on every page load) and keep them small (the existing 24 kHz mono WAV convention).
General UI/UX simplification pass -- reduce how overwhelming this app's settings/ configuration screens are. User's own observation comparing live: VoiceStudio's integrations screen (http://192.168.178.8:3900/#/integrations) is "way cleaner" than this app's Settings/Engines equivalent. Concrete evidence for the gap: s-llms.html ("Engines") is 880 lines with 31+ labelled fields all visible on one page at once, versus VoiceStudio's per-category model cards (TTS/ASR/Dictation/Diarisation/Translation/LLM) that each expand on demand -- see the models.png screenshot referenced in architecture/voicestudio-comparison.md. Direction: progressive disclosure (group into collapsed cards/categories, expand one at a time) instead of one long flat form; fewer fields visible per step; a simpler default view with advanced options tucked behind a toggle, mirroring the "Fast/Balanced/Quality/Max" tiering idea from the onboarding-wizard item above rather than exposing every knob unconditionally. Not limited to Engines -- once a pattern is settled there, audit Settings' other pages and any other dense single-page form (Payloads, the magazine quality-run's settings, etc.) against it. Needs its own before/after review with the user (screenshots or a walkthrough) since "simpler" is a judgement call, not a number to hit.

P0 — do first

S — Stop provider API keys from reaching frontend JavaScript or logs. Provider keys are masked in settings responses, probes resolve credentials server-side, and browser storage/log paths are clean. The separate external caller key is available only through deliberate regeneration, uses no-store, and is not retained by the UI.
S — Validate uploaded filenames and paths. Upload endpoints reject traversal, path separators and control characters before deriving extensions; ZIP imports reject absolute, traversal and malformed archive members.
M — Add upload, request-size, temporary-file and concurrent-job limits. Endpoint-specific upload/request caps, bounded temporary work, cache eviction, and bounded TTS/STT/image job pools are implemented.
M — Add automatic backups before bulk deletes, voice moves and recasts. Make recovery explicit and verify that backups can be restored.
M — Strictly validate custom URLs and prevent SSRF. HTTP downloads and provider/container probes share scheme, hostname and private-destination validation with explicit local-network opt-in.
M — Add authentication and rate limits for destructive/API endpoints. External API authentication is enabled by default and destructive/general mutation limits return bounded 429 responses.
M — Validate character sheets and voice metadata with schemas. Character/form and voice metadata are bounded and type-checked at API boundaries.
L — Add database migrations and prompt/generation history. Preserve old record shapes and expose protected, metadata-only generation history for auditing and follow-up recovery workflows.
L — Harden container and Docker access. The app runs non-root with dropped capabilities, no-new-privileges and a read-only root filesystem; Compose Docker access is isolated behind a minimal container proxy with only container inspection/control enabled.

P1 — next, highest user impact

L — Magazine: model-based layout detection and reading order. Evaluate replacing the hand-written region and order rules with dedicated models, as in the approach from Building agentic document understanding with OCR and LLMs: PaddleOCR PP-DocLayout (regions: text, title, figure, chart, table, footnote) and LayoutReader (a LayoutLMv3 fine-tune that orders regions from text + boxes). Before building: licences (PaddleOCR Apache-2.0; LayoutLMv3 weights CC BY-NC-SA 4.0 -- non-commercial only), GPU/RAM on the DGX (already near its limit), and a measurement on the same 204-page issue with the existing continuation score (current heuristics: 89 confirmed continuations; any model must beat that). A tagged PDF, when a publisher ships one, already carries the reading order in its structure tree -- read that first when present (reading-order best practice).
M — Magazine podcasts via Open Notebook (multi-speaker). Open Notebook has an advanced multi-speaker podcast generator ("Generate professional podcasts") that could make the magazine podcast mode much better. It already runs on the LAN at 192.168.178.19 (Docker: API :5055, UI :8500, SurrealDB :8501). Two options, decide after reading its API and licence: (a) hook it up — send an approved article to Open Notebook as a source, ask it for a podcast script/episode, and play or voice the result here (a new external integration: check licence, data flow — article text leaves this app —, auth, rate/cost, and availability; keep it optional with the current podcast mode as fallback); or (b) copy the approach — port its speaker/episode-profile design into the existing podcast mode and voice it with this app's own TTS voices. Prefer (a) if its API exposes script generation cleanly, since the voices would still come from here.
M — Magazine: let the reader decide the region order. Done 2026-09-21 (type a position, or Order by clicking; hand-ordered pages are locked). Original note: On the boundary-review page, give each region an explicit reading position: click the regions in the order they should be read, or type a number into each one. The order drives playback and the OCR-results list. Today the order is whatever the layout analysis produced, and a page with two text columns comes out interleaved, so a listener hears half a sentence from the wrong column.
L — Magazine: overhaul region boxes and how they are handled in OCR and approval. (Done 2026-09-21: type picker and delete per row; Clean up boxes — text in pictures, OCR noise, duplicates, run-on author lines. Done too: keyboard navigation, merge with next, split at cursor, per-row re-OCR and text editing. Done 2026-09-28: Clean up boxes now fits text boxes to their ink (grows over clipped edges, trims empty margins; hand-placed boxes untouched; on the measured issue only 5 of 658 boxes on pages 1-59 needed it). Still open: the 333 overlapping boxes with different text on the measured issue need a human.) The boxes the layout analysis produces need a pass of their own, on top of the ordering work above. Seen on real pages: a byline box that runs on into the first body line ("Von Hartmut Gieselmann Ein guter Butler kennt die Vorlieben seines"), so author and paragraph both come out wrong; boxes nested inside a figure and OCRed as separate headlines/captions ("Apple Intelligence" on the slide in the photo); sliver boxes holding OCR noise ("mtavt;-:,!«"); overlapping boxes that fight over the same lines; and running heads/footers picked up as regions every time. Wanted: snap boxes to the real text block, split or merge where a box crosses a paragraph or column edge, drop or fold in regions inside a figure, discard degenerate slivers, and make the approval screen quick to correct — keyboard moves between regions, merge/split in place, and a retype/re-OCR of one region without leaving the page.
M — Magazine: read text columns top-to-bottom, then left-to-right. Done 2026-09-21 (column anchors from left edges, bands, pull-quotes last; 85→89 continuations on a 204-page issue; Fix reading order for existing issues). Original note: Group detected text regions into columns first, order each column downwards, then take the columns left to right (mirroring for RTL). Confirmed wrong on page 38 of issue 204: the left column's paragraphs and the right column's are mixed into one sequence. Should land before the manual ordering above, which is then only a correction tool rather than routine work.
S — Re-measure audiobook segmentation against cast_run3.json. Run 3 has 2,227 segments, 1,043 dialogue segments and 585 newline-fused segments, but zero adjacent same-speaker dialogue pairs for _audiobookMergeAdjacentDialogue to merge. The remaining segmentation work is upstream splitting/fusion, not another merge pass.
S — Finish the remaining Playwright smoke checks. Characters/Cast, profile sorting, column visibility, voice ratings, and starting/cancelling a generation job are covered.
M — Fix the Schwester Larissa unclosed-guillemet split. Attribution normalization now repairs dialogue split before mixed German guillemet closes (»…‹ / «…›), with an epigraph regression test.
M — Make profile portraits and character sheets use one canonical identity. Shared structured prompts now version consistent pairs as v3 and fingerprint canonical appearance fields; add cross-record validation so mismatched profile/reference files are detected even when both are current-version. The UI now flags missing, stale, changed, or explicitly inconsistent pair metadata; visual similarity itself remains a human review concern, while generated/replaced assets are now detected using stored SHA-256 digests.
M — Add bounded duplicate-generation protection. Consistent portrait/reference, voice-design, and casting jobs now deduplicate repeated requests with bounded, expiring Idempotency-Key contracts.
M — Improve large-table performance. Character productions and the flat voice library paginate at 50 visible rows; low-memory mode also reduces rendered rows and image work.
M — Add explicit cache invalidation and eviction. Fish.audio search responses, persisted synthesized audio, image-generation job mappings, and browser object URLs have bounded cleanup.
M — Add constrained-device validation. Chrome CPU throttling, a 500-character workload, browser DOM/audio/heap/image-cache bounds, and a hard 4 GiB current-release container check are covered. The 1.24.21 image served successfully at 52.54 MiB idle usage, 0.10% CPU, 6 PIDs, and OOMKilled=false; cgroup memory was confirmed as 4,294,967,296 bytes and temporary/root filesystem usage was sampled.
L — Add background jobs and bounded queues. Image generation already has the full interactive job path, and transcription, voice design, and casting now have optional ID/status/cancel/timeout paths while preserving synchronous callers. Job metadata now persists in SQLite and marks interrupted work after restart; durable binary result storage is bounded and authenticated; outbound TTS and LLM calls now share a bounded transient-failure retry policy, and /api/jobs exposes sanitized recent job history. Completed voice-design audio and consistent-image outputs now use bounded, atomic artifact files with authenticated download URLs; transcription text remains deliberately non-persistent. Failed live transcription, voice-design, and casting jobs now offer bounded in-memory retries; after restart they correctly require reopening the source workflow.
L — Run the Book 2 generalization regression. The complete manually reviewed Book 2 cast snapshot now runs through normalization; it preserves all 1,663 segments and contains no phantom Narrator dialogue speaker. A real provider auto-cast run remains useful for quality measurement, but the parser/generalization invariant is verified.
L — Add character forms inside one character record. sheet.forms[] stores bounded alternate appearances, prompts, and per-form visual assets; the detail-page switcher preserves one character ID and voice, and selected forms feed prompt generation and identity fingerprints.
L — Modularize the frontend incrementally. The shared manifest, compatibility namespace bridge, section-scoped lazy loading, and responsibility-level Audiobook/Rehearser/style extraction are complete; a full ES-module migration remains a separate follow-up.
M — Add structured logs with request/job IDs. API/MCP requests now receive sanitized correlation IDs and duration logs; route-specific logs retain provider/job context without request bodies.
L — Add linting and duplicate-function checks. The manifest-driven frontend syntax/declaration gate now runs locally and in the SonarQube quality workflow alongside the existing Python and bundle checks.
Expose extracted Rehearser and Audiobook parser APIs through the window.TTVC namespace bridge.

P2 — deployment

S — Deploy the app to correspondence-nuc (192.168.178.19), pointed at the DGX Spark's TTS engine over the network. Live since 2026-09-14, v1.27.5. NFS export set up on the DGX (/home/sparky/Media/_Sounds/TTS_Voices → 192.168.178.19 only, rw,no_root_squash), mounted read-write on the NUC at /mnt/dgx-tts-voices and persisted in /etc/fstab there — confirmed the NUC instance sees the same 344 voices the DGX does, not a separate copy. Deployed by rsyncing the working tree from the DGX (excluding .git, config/, logs/ and other local-only paths) to /home/sparky/Docker/tts-voice-creator-clone-and-design-2 on the NUC and building the image there directly — the published ghcr.io image turned out to be both several versions stale and private with no registry credentials on the NUC, not worth setting up for an image that would need re-pushing on every deploy anyway when the source is right there to build from. One real gap found along the way: vision_ocr_url/vision_ocr_fallback_url default to host.docker.internal, which resolves to whichever host the container is running on — correct on the DGX itself, silently wrong on the NUC (would have pointed at the NUC's own, nonexistent OCR engine). Pinned both to the DGX's LAN IP explicitly via environment overrides in the NUC's compose file. Also needed: the NUC's config/logs bind-mount directories were created root-owned by docker compose (this session's NUC access is a root SSH key) while the container's own app user is uid 1000 — chown -R 1000:1000 on both before the app could open its SQLite state DB. Confirmed the accepted limitation from before still holds as expected: no docker-socket-proxy on the NUC instance (nothing local for it to control, and cross-host container restart never worked anyway), so a voice change still needs a manual restart on the DGX. NUC compose file tracked here as docker-compose.nuc.yml (the plain docker-compose.yml in this repo stays the DGX's own, per its build:/dgx_net/docker-socket-proxy shape) — deployed to the NUC as its docker-compose.yml, since that host has no .git checkout of its own to diff against, only the rsynced tree. A code change means: re-rsync the tree (excluding .git, config/, logs/), docker compose build, docker compose up -d — no separate push/pull step, same as the DGX's own bind-mount-and-recreate pattern. Footgun confirmed 2026-09-27: docker-compose.yml must ALSO be excluded from the rsync (--exclude docker-compose.yml) — otherwise it silently overwrites the NUC's own compose file with the DGX's (wrong build/image, dgx_net/docker-socket-proxy the NUC doesn't have), and docker compose up -d then fails with network dgx_net declared as external, but could not be found. Recovery: cp docker-compose.nuc.yml docker-compose.yml on the NUC, then up -d --force-recreate.

Deferred — public multi-user readiness (cross-app initiative)

Not scheduled; blocked on a decision the user wants to make once, shared across all self-hosted apps, not per-app. See architecture/multi-tenant-and-identity.md for the full research findings and design sketch. Do not start without a fresh planning pass — this is a multi-week, cross-repo effort.

Design and stand up a shared identity service across all self-hosted apps (interconnected user profile, not per-app accounts), building on the existing OurFlashcards GoTrue + Postgres + PostgREST stack on the NUC rather than a new auth system per app.
Once shared identity exists: add user_id/owner_id columns to every table in core/database.py, move voices/output/reader_library/audio-cache to per-user paths, replace the single global settings.json with per-user settings, replace the single shared external_api_key with per-user API keys.
- Includes provider API keys (llm_api_key, groq_api_key, elevenlabs_api_key, engine_api_keys, etc.) — today one value per provider for the whole install (gitignored + 0600 settings.json, never echoed to the browser once saved). Each user needs their own key per provider, scoped under their identity- service user id; see architecture/multi-tenant-and-identity.md §1.
Replace the global TTS/STT/image concurrency semaphores (routes/tts.py, routes/stt.py, routes/conversation.py) with per-user fair-share admission control.
Required consent attestation + audit trail for voice cloning (decided: required). Checkbox at voice upload time, logged with timestamp/account/IP; add a user_id column to generation_history so misuse is traceable per account.
Add a reverse-proxy/TLS edge layer (CORS, CSRF, CSP/HSTS) for internet exposure — nothing in this repo handles this today.

P2 — after behavior and safety stabilize

M — Add structured logs with request/job IDs. Report provider, endpoint, duration and failure reason without logging prompts, keys or book contents.
M — Separate provider health from configuration status. TTS/STT discovery now returns independent configured, available, and bounded health details; backend help text shows actionable connectivity errors.
M — Add visible job history and retry controls. Expose recent failures and safe retry state to users.
L — Add memory diagnostics. Show browser heap where supported, rendered rows, cache size, active jobs, server/container memory and recent operation sizes.
L — Add linting and duplicate-function checks. Run them in CI alongside the existing compile, bundle and SonarQube checks.

P0 — Testing, data safety, and resource limits

Establish a Playwright baseline before large refactors (app shell, bundle, i18n, clone tabs, and Rehearser parsing; installed Chromium supported).
Add explicit Docker memory limits where appropriate; the web container defaults to an 8 GB limit and 2 GB reservation with environment overrides.
Limit upload sizes, temporary files, and concurrent jobs. Upload/request caps, bounded caches, and bounded TTS/STT/image-generation pools are implemented; durable background-job conversion remains a separate P1 item.
Use atomic writes for .meta.json files.
Create automatic backups before bulk deletes, voice moves, and recasts.
Add request-size limits, rate limits, and authentication for destructive/API endpoints.

P1 — Modular frontend architecture

Split audiobook.js, rehearser.js, and style.css by responsibility; Rehearser casting, playback, training, voice design, I/O, library, stage, and recording blocks plus Audiobook casting, preview, export, and feature styles are isolated. Core Reader state and compatibility event wiring remain ordered in the legacy entry points.
Extract Rehearser stage display and cast-sidebar controls into rehearser-stage.js.
Replace broad globals with namespaces such as window.TTVC.audiobook; pure leaves, Audiobook/Reader/Rehearser feature modules, Studio, and character-library persistence now use private scopes with explicit compatibility exports. Voice-library loading, rendering, benchmarking, import workflows, character-sheet extraction, character-library rendering/actions, voice-inspector selection, shared utility helpers, cast compatibility, i18n, and settings/log UI are also scoped; legacy entry points remain only as explicit compatibility exports.
Generate the feature-module list for loader.js and scripts/minify.mjs from one shared manifest; larger responsibility splits remain.
Add a dependency-free frontend syntax and top-level declaration collision gate (npm run check:frontend); a full style/lint policy remains a follow-up.
Extract Rehearser voice matching and voice design; matching/import helpers now live with the voice-picker module, while timing-sensitive DOM wiring remains in the cast renderer.
Extract Rehearser voice-picker presentation and preview state into rehearser-voice-picker.js.
Extract Rehearser cast view, sorting, filtering, and toolbar wiring into rehearser-cast.js.
Extract Rehearser training transcript comparison helpers into rehearser-train.js.
Extract Rehearser character development and backend-selection controls into rehearser-voice-design.js.
Extract Audiobook quote segmentation and conservative turn-taking into audiobook-segmentation.js.
Extract Rehearser library persistence and IndexedDB migration into rehearser-library.js.
Extract Audiobook parsing and casting; parsing, segmentation, cast normalization/serialization, live cast view, LLM orchestration, preview, and export now live in focused modules. Core Reader state and compatibility wiring remain in audiobook.js.
Extract Audiobook parsing primitives and speaker-heuristic constants into audiobook-parse.js.
Extract Audiobook cast serialization and page-stamping helpers into audiobook-cast.js.
Split the Performance/resource diagnostic styles into static/css/resource-diagnostics.css.
Split background-job history and retry-panel styles into static/css/background-jobs.css.
Split Voice Design form and sample-grid styles into static/css/voice-design.css.
Split style.css by feature; Rehearser rules now live in static/css/rehearser.css alongside the existing diagnostics, background-job, voice-design, and Audiobook/Reader feature styles.
Decision — do not migrate to TypeScript or rewrite the frontend until behavior is covered by tests.

P1 — Frontend performance and memory safety

Lazy-load Audiobook, Rehearser, Studio, and image-generation modules only when opened. The shared manifest keeps the foundation in the core bundle and loads Reader/Audiobook, Rehearser, Library/Characters, and Studio groups once on first navigation; deep links load their group before activation.
Paginate large character and voice tables; render approximately 25–50 visible rows, with a constrained-device regression covering a 250-character production.
Cache /api/voices with a shared 30-second session cache, one in-flight request, explicit refresh, and invalidation after library metadata/folder mutations; the SQLite voice index remains authoritative.
Lazy-load profile images and concept art.
Update only affected table rows instead of rebuilding entire tables where the operation is row-local; benchmark, normalization, rating, and voice-note updates now patch the existing visible rows. Structural changes still trigger a full filtered rebuild so sorting, grouping, and pagination remain correct.
Add an LRU cache for decoded Rehearser audio and remove old entries automatically. Persisted line-audio is already bounded; profile/concept and voice-library images now use a bounded viewport-aware browser cache that removes old decoded sources while retaining restoration URLs.
Revoke discarded object URLs with URL.revokeObjectURL(objectUrl); completed synthesized voice previews and character/Rehearser local auditions now release their Blob URLs when playback ends or is replaced.
Add low-memory mode using navigator.deviceMemory where available; shared worker pools cap at two tasks on devices reporting approximately 4 GB RAM or less.
Disable image preloading and reduce table/cache sizes on devices with approximately 4 GB RAM. Images use lazy decoding, and character/voice pages reduce from 50 to 25 rows when navigator.deviceMemory <= 4.

P1 — Backend stability and bounded concurrency

Introduce background jobs for voice design, transcription, casting, and image generation. Synchronous callers remain supported while each long-running path has an optional bounded background job.
Give every job an ID, progress, cancellation, timeout, retry, and idempotency key. Failed live jobs offer bounded retries from stripped in-memory inputs; retry data is never returned or persisted, and restart correctly requires reopening the source workflow.
Prevent duplicate generation when a user clicks twice for image, voice-design, and casting submissions.
Centralize provider health checks and error handling. TTS, STT, and container URL probes now share the bounded core.provider_health.probe_provider mechanics and normalized connection/timeout/HTTP/JSON-shape errors; their provider-specific endpoint lists and authentication policies remain at the route layer.
Limit concurrency to 1–2 image generations, 1–2 voice generations, and 1 STT request per user. Consistent portrait/reference rendering, preview/style/seed-finder synthesis, and the OpenAI-compatible speech proxy are capped at two active jobs; STT transcription and benchmarks are capped at one active job.
Use bounded server queues for the implemented synchronous generation paths; durable background-job queues remain a separate outstanding item above.

P0 — Security

Never expose provider API keys to frontend JavaScript or logs. Provider keys are masked on read and absent from browser storage; the external-client key is an intentional, explicit handoff for caller authentication and its regeneration response is non-cacheable.
Validate uploaded filenames and paths.
Strictly validate custom URLs to prevent SSRF, with private destinations available only when explicitly allowed for local provider/container operation.
Add request-size limits, rate limits, and authentication for destructive/API endpoints.
Run containers as non-root with dropped capabilities and read-only filesystems where practical.
Isolate Docker socket access behind a minimal privileged service.

P0 — Data integrity and recovery

Validate character sheets and voice metadata with schemas.
Add database migrations instead of silently changing record shapes.
Use atomic writes for .meta.json files.
Create automatic backups before bulk deletes, voice moves, and recasts.
Store prompt versions and generation history so bad regenerations can be audited and follow-up recovery can be built.

P0 — Testing

Automated tests

Character parsing and phantom-speaker prevention, including malformed-response prose preservation.
Voice-matching constraints reject invented voice IDs and avoid unnecessary duplicate assignments.
Profile-completeness counting for empty, partial, and fully populated sheets.
Rating persistence.
STT endpoint normalization and backend-response shape handling.
ComfyUI workflow submission, prompt injection, polling, and image retrieval.
API timeout clamping and bounded upstream error text; live-provider failure drills remain environment-specific.

Playwright smoke suite

Load the app.
Verify the production bundle loads.
Verify language switching and Rehearser parsing.
Verify clone source tabs switch correctly.
Open Characters / Cast.
Sort the profile column.
Hide and show columns.
Rate a voice.
Start and cancel a generation job.

P2 — Observability

Add structured logs with request and job IDs.
Report provider, endpoint, duration, and error reason clearly.
Separate health status from “configured”; discovery APIs now preserve both states and expose sanitized probe errors.
Add visible job history and safe retry state; the authenticated /api/jobs metadata endpoint now feeds a rendered panel with status, elapsed time, errors, cancellation, and an explicit reopen-source retry boundary.
Avoid logging prompts, API keys, or book contents by default.

P1 — Resource limits and constrained-device testing

Add explicit Docker memory limits; the web container defaults to mem_limit: 8g and mem_reservation: 2g, configurable per host.
Limit upload sizes and clean converted upload intermediates; continue expanding endpoint-specific temporary-file and job limits.
Test with Chrome DevTools-equivalent CPU throttling; desktop Playwright uses Chromium CDP at 4× CPU and asserts navigation plus bounded DOM/audio counts, with JS heap captured when Chromium exposes it.
Test the current image in a container limited to 4 GB RAM; the app served / with a 4 GiB cgroup limit, ~50.6 MiB idle memory, 0.09% CPU, and no OOM kill.
Monitor browser-tab memory, JavaScript heap where supported, DOM-node count, audio elements, persisted jobs, server RSS, configured-data disk, and temporary disk usage through the read-only Performance diagnostics panel.
Add a memory diagnostics page showing cache count/size, Docker/container memory, and largest recent operations. The authenticated panel reports browser cache counters, cgroup memory current/max, server RSS, configured/temp storage, persisted jobs, and sanitized operation durations; host-wide Docker metrics remain unavailable by design.

P1 — Speaker attribution: measured state and next steps

Status 2026-09-19 — reworked, see architecture/audiobook-attribution.md. Span mode (deterministic quote spans, the LLM only names speakers) replaced the segmentation-heavy engine: Book 1 every-word-right 92.3 % (was 87.0 %), narration/dialogue agreement 97.4 % (was 93.6 %), ~10 min instead of ~4 h; Books 2/3 (never tuned on) 90.8 % / 89.9 %. The word-aligned scorer is scripts/score_attribution.py; the references have label noise, so 1–2 points is not meaningful. Everything below the line is the historical record of the classic engine's Runs 1–3.

Still open (each needs a real decision or new evidence, not more prompt tuning):

A person known only by a role that means different people in different scenes ("der Fremde") is not resolved to a character — needs scene-level identity tracking; the label is kept as its own cast entry until the book itself ties it to a name.
Look-alike main characters in one scene (Marcian/Zerwas/Lancorian) remain the biggest real error class (~1/5 of spoken words). Tried and rejected: reasoning mode (25× slower, no gain), larger/smaller windows, tag-based overrides, reasoning adjudication of disputes (made disputed lines worse), an explicit scene-cast hint (names mentioned in the passage: +0.3 points, noise). What remains is mostly model capacity plus reference noise; the useful lever left is human review guided by the agreed/disputed flags.

Ordered follow-up queue

These follow-ups are ordered by dependency and expected benefit. Estimates assume one developer familiar with this codebase; benefit is relative user/reliability impact.

P1 / M — High benefit — Identity-anchor regression tests. Add automated coverage proving avatar and concept-sheet requests share canonical identity anchors and persist the prompt version used for each generated asset.
P1 / S–M — Medium benefit — Preserve dialogue-only merging. Keep the dialogue-only merge for recast/verify outputs where adjacent pairs still occur, with regression coverage for the non-dialogue boundaries.
P1 / L–XL — Very high benefit — Rework segmentation from Run 3. The initial-cast final normalization now merges narration only, preserving dialogue turn boundaries; Run 3 showed same-speaker dialogue joins were obscuring upstream split/fusion errors. Recast/verify retains its separately tested dialogue-only merge.
P1 / M — High benefit — Re-run Book 1. Done 2026-09-19 with the span-mode engine: overall 92.2 % (was 87.0 %), type agreement 97.4 % (was 93.6 %), ≈11 min instead of ≈4 h. See architecture/audiobook-attribution.md.
P1 / L–XL — Very high benefit — Run Book 2 generalization. Done 2026-09-19: Book 2 90.8 % and Book 3 89.9 % overall (neither tuned on) vs 92.2 % on Book 1 — the approach generalises. Note the references carry label noise, so 1–2 points is not meaningful.
P1 / L — Very high benefit — Unify visual identity. Portrait and concept-sheet generation now use one canonical identity description, stored prompt version, identity fingerprint, asset digests, and stale-asset review path; shared-anchor and metadata persistence are covered by automated tests.
P2 / M–L — Medium–high benefit — Enrich character-form UX. Done 2026-09-19: forms panel (add/rename/delete/show, per-form stale flag, per-form and all-forms image generation), Physical/Clothing edits scoped to the active form. See architecture/character-visual-identity.md.

Magazine reader roadmap

Magazines are a separate Studio document type: they use a narrator and layout-aware article extraction, not character casting. The original PDF remains the visual source of truth while approved articles become linked, reflowable HTML.

P0 — Magazine foundation

Add a Magazine project type beside Books and Theater Plays.
Add PDF/JPG/PNG import with page previews and safe local storage. (2026-09-29: JPG/PNG page images become one scan-style PDF, see architecture/magazine-ingestion.md)
Add OCR-engine selection from Engines > OCR & Document Parsing, with page-image input and structured output.
Detect contents/index pages and propose article titles, authors, and page ranges.
Build a review screen where users approve or correct article boundaries before TTS. (includes "Edit boxes" — user-requested 2026-09-14: drag an already-detected region to move or resize it, or delete it outright, with an optional re-OCR of the corrected area for text regions. "Draw it" remains the separate tool for a region detection missed entirely.)
Preserve source coordinates for every extracted block, image, caption, and page reference.
Generate per-article podcast scripts from a connected LLM with two or three user-picked host voices, with a Studio panel to pick hosts, generate, and play the script back turn by turn.
Make the ingestion "watching it work" panel more visually appealing (done: live region boxes drawn over the page image, colour legend, reduced-motion safe; commit 1edc941, covered by tests/magazine.spec.js) — user-requested 2026-09-14. Today it's a static side-by-side of the current page image and a plain text/label list (#mag-ingest-preview, static/sections/s-magazine.html, populated by routes/magazine.py's preview-HTML builder around line 158). Wanted instead: draw the actual detected region boxes (image/table/code/figure/etc.) live, overlaid directly on the page image as each region is classified — watching the layout engine work through the page, not just a before/after. Likely an SVG or absolutely-positioned overlay using each block's bbox (already computed and already used for the review screen's region boxes, magReviewMarkup-adjacent code in static/js/magazine.js) drawn incrementally, one box per region as it's classified, rather than all boxes appearing at once at the end. Purely cosmetic/decorative per the existing comment on that panel — must not become the only place the essential progress info (percentage, page count, done/failed) lives; that stays in #mag-ingest-status.

P1 — Magazine listening experience

Generate clean article HTML with headings, captions, figures, sidebars, and page links. (headings/byline/captions styled and page-linked, figures shown during playback, and (2026-09-29) sidebars shown in place as labelled, never-narrated asides)
Generate narrator audio per article with resume/progress and a play-all queue.
Synchronize paragraph/sentence highlighting with playback and keep PDF and article views linked. (sentence-level; clicking a sentence or switching to the original-page view jumps to its source bbox)
Show article images or page crops during playback, with optional caption reading. (images shown, matched to reading position; caption text is not separately spoken)
Add confidence warnings, extraction diffs, and manual text correction before audio generation. (confidence badges, boundary editing, per-region text editing, and (2026-09-29) an Edited mark with a word diff against the recognised text and one-click restore)
Redesign the magazine contents view (#s-magazine, the article list for one issue). Today it is a flat list of title + page range + badge + Review button, which shows nothing of the issue itself: no page, no opening text, no figure, no sign of which articles already have audio. Requested directions, none decided:
- Three columns. Page thumbnail · headline + opening text · the article's figures. Reads down the issue page by page in column one, with the text and the pictures beside it.
- Synthesis state per row. A "synthesised / not synthesised" marker, the MP3's duration as mm:ss, and an inline play button — so the list doubles as the playlist instead of having to enter an article to find out.
- Flipbook. The PDF as a page-turning spread (issuu.com as the reference), with the extracted content of the visible pages in two columns underneath it. Decide the shape first — the three ideas are not variants of one layout, and the middle one (synthesis state) is worth having under any of them. Whatever is chosen has to keep working at phone width and stay navigable by keyboard and screen reader; a flipbook in particular needs a non-gestural alternative.

P2 — Advanced magazine intelligence

Improve cross-page continuation detection and automatic sidebar/advertisement exclusion. (sidebar exclusion via mark_sidebars, a narrow-column geometry heuristic; cross-page continuation now follows both "continued on page N" and "continued from page N", 2026-09-29, unit-tested, not yet seen on a real issue)
Add image descriptions, chart/table handling, and optional spoken figure summaries. (per-figure "Describe" in review generates a type-aware description — chart/table/code/figure each get their own prompt — via the configured vision engine with fallback; editable before approval; spoken only if "Read figure descriptions" is enabled. Live-verified against a real chart: the configured primary vision engine turned out to be a pure-OCR model that returns empty content for any non-transcription prompt regardless of phrasing — confirmed with three different prompts — so description generation only works when a genuinely caption-capable model is reachable as the fallback; documented as a config-dependent limitation in Manual.md, not a code defect.)
[~] Add word-level karaoke timing and EPUB 3 Media Overlay export. (EPUB 3 Media Overlay export done 2026-09-30, EPUBCheck-clean, with exact sentence timings for newly rendered audio; word-level highlighting is still a character-length estimate within each sentence, not forced alignment; the in-app MP3 player still aligns its transcript by estimate and could use the new timing records)
Add mobile/driving mode with large controls, automatic article advance, and hands-free resume. (driving mode with large controls, queue auto-advance and resume at the saved position; 2026-09-30: Media Session for lock screen / headset / car controls and artwork, screen wake lock while playing)
Add OCR-engine benchmark comparisons and per-page fallback selection. (per-page fallback is a real engine chain: vision OCR escalates when a page needs OCR or scores poorly, KDL nano is the vision fallback, tesseract the last resort; 2026-09-29: scripts/bench_ocr.py scores any engine against a native-text PDF's own text layer, first results for tesseract, Unlimited-OCR, docling and PaddleOCR in architecture/evidence/ocr-bench-2026-09-29/)
Evaluate additional OCR/vision candidates against the two already wired (baidu/Unlimited-OCR, KDL-Frontier-Parser-nano) and pick the best fit per task — user-requested 2026-09-14. Goal: low RAM/VRAM footprint with excellent speed and quality, for both page-text transcription and open-ended figure/chart description (the latter is the harder requirement — confirmed live that baidu/Unlimited-OCR is transcription-only and cannot do it at all, see architecture/magazine-ingestion.md and CHANGELOG 1.29.0). Candidates to install as Docker containers alongside the existing engines, each added through the Engines tab (routes/docker.py's _LOCAL_CONTAINER_DEFS, role ocr) so they're selectable/connectable/benchmarkable like any other engine, not hardcoded: - [x] docling — document conversion/layout toolkit. (2026-09-29: benchmarked on CPU, F1 0.867, 8.9 s/page, 2.4 GB; see architecture/evidence/ocr-bench-2026-09-29/) - [x] PaddleOCR — established general-purpose OCR. (2026-09-29: benchmarked on CPU, F1 0.881, 36.5 s/page, 1.4 GB; not yet wired as an engine) - [ ] DeepSeek-OCR — vision-language OCR model. (not run 2026-09-29: only ~14 GB of the unified memory is free; needs a running model stopped for a test window) - [ ] (not run 2026-09-29: ~30 GB does not fit beside the running sglang model) Qwen3-VL-30B-A3B-Instruct-FP8 — general vision-language model; check whether it fits the RAM/VRAM budget at 30B-A3B FP8 (likely the heaviest candidate — weigh against the "low footprint" goal specifically) and whether it's meaningfully better at open-ended description than the nano-class candidates above. For each: measure page-transcription accuracy and figure/chart-description quality against the existing c't KI Wissen 2026 test corpus, plus idle/active VRAM footprint and latency, before deciding what replaces or joins the current primary/fallback pair.

P2 — Finding magazines to read

[~] Browse and search online PDF magazine sources from inside the app (done 2026-09-29/30: Internet Archive + OPDS catalogues with in-app management, login via an environment variable, and result paging, see architecture/magazine-sources.md; still open: publisher free issues/feeds, open-access journals) without leaving for a browser and uploading by hand. Wanted: a catalogue of sources, a search across them, and a one-click "ingest this issue". Blocked on a source decision, and that decision is the whole task. The two sites named as examples (freemagazines.top, pdf-magazines.org) redistribute current commercial titles — c't, BBC Wildlife, AirForces Monthly — without the publishers' licence, so building a scraper against them is out: it is copyright infringement regardless of who presses the button, and both would be trivially blocked or taken down anyway. Sources that can be built against, in rough order of usefulness here:
- OPDS catalogues — the open standard for exactly this. Calibre-Web, Kavita and Komga all serve OPDS, so a generic OPDS client also covers the user's own library on the NAS. Start here.
- Internet Archive (archive.org API) — a genuinely large public-domain and openly-licensed periodical collection, with a documented search API and no scraping needed.
- Publisher-run free issues and RSS/Atom feeds — heise gives away sample c't issues, many trade titles publish a free archive. Per-publisher work, but legitimate and high quality.
- Open-access journals (DOAJ, arXiv, PubMed Central) — if the appetite is reading rather than magazines specifically, these are the largest legal corpus by far and all have real APIs. Whatever is added, check licensing, rate limits and terms per source before wiring it (core rule 8), honour robots.txt, and keep each source behind its own provider interface so one going away does not break the rest.

Three full regression runs of Der Sturm against the hand-corrected reference. Everything below is measured, not estimated. Artefacts live in config/regression_snapshots/der_sturm/ (*_manual.json are the human reference — never overwrite them).

Run	Segments	Headline agreement	Fully correct	Wrong speaker	Unmatched
1	2143	72.6%	54.2%	438	544
2	2273	77.7%	52.8%	345	727
3	2227	78.7%	52.2%	314	750
human reference	2110	—	—	0	—

Read the headline with care. Agreement is computed only over rows whose text matches the human's, and that pool keeps shrinking (1598 → 1545 → 1475). Absolute correct segments went 1160 → 1200 → 1161. The rising headline is largely a shrinking denominator; true end-to-end accuracy has drifted slightly down.

What genuinely improved

Unknown speakers 121 → 5 → 8; the addressee/Unknown prompt rewrite did not overcorrect (369 after cast collapsed to 19 after one recast pass).
Wrong speaker names 438 → 345 → 314 — consistent, real.
Verify pass now completes (~65 LLM calls) instead of never finishing.

The blocker: segmentation, not attribution

750 of ~1060 problem segments are text that doesn't match the human's at all: roughly 400 split finer than the human, ~210 merged where the human split, ~70 OCR/wording differences. Attribution work cannot fix these.

fused dialogue segments hit 212 (human: 6, run 2: 59). These are _audiobookMergeAdjacentDialogue joins using \n\n; 137 of them match no human segment. Splitting them back was tested and is worse (52.2% → 43.4%), so the merge is not the primary cause — but merging blocks the human kept separate is contributing.
The merge was predicted (on run 2 data) to cut unmatched 727 → 588. On run 3 it went to 750. That prediction did not hold and needs re-deriving on run 3.
Fixed: attribution normalization repairs the Schwester Larissa epigraph when the LLM splits it before a mixed guillemet close; the regression is covered by tests/test_conversation.py. (unclosed-guillemet handling).

Next steps, highest value first

Attack segmentation directly — it is now the dominant error class. Run 3 has no merge candidates, so inspect the earlier dialogue split/fusion stages and the 585 already-fused segments instead of changing _audiobookMergeAdjacentDialogue.
Preserve the dialogue-only merge for recast/verify outputs where adjacent pairs can still occur; do not claim it explains the run-3 mismatch.
Only then re-run book 1. Re-running before segmentation moves will re-measure rules already known not to shift the number that matters.
Book 2 generalization run never started. Reference is backed up at config/regression_snapshots/book2/. Its point is to test whether these rules generalise or were overfit to book 1 — still unanswered, and the confidence scorer in particular was tuned solely on book 1.

Character profiles

48 human vs 64 auto (run 3), 35 matched, 13 missed. Auto over-generates and got worse than run 2's 51. Generic role labels (Fremder 19 lines, Vampir 19, Inquisitor 15, Ork 13, and the adverb Unsicher) still become cast members with their own voices; the appositive resolver only fires where the book supplies grammatical evidence (Darrag der Schmied), and pre-reveal aliases like "der Fremde" deliberately withhold it.

Harness notes (cost real time; do not relearn)

Casting auto-starts audiobookAutoRepairCast 800 ms after finishing, which re-sets _audiobook.running. Guards now explain the refusal instead of returning silently, but an automated driver must still wait for running to clear and treat a sub-60s phase as a bailout, not a success.
Stall detection must key on the passage counter (#ab-cv-count). segments, done and unknown do not move during a verify pass — watching them killed a healthy run twice.
Character Sheets only saves after its passage loop ends; poll /api/characters until records appear before closing the browser, or a 4.4 h run is discarded.

P1 — Character forms (implemented 2026-09-04)

Shape-shifting characters (Zerwas: human / vampire / wolf-demon, aliases "Fürst der Nacht", "der Wolfsmensch") need more than one appearance per character.

Decided: forms live INSIDE one character record, not as separate records — one voice, one attribution target, so cast lines saying "Zerwas" always resolve. The sheet gains a form switcher; each form carries its own physical, image_prompt and portrait. Detection is automatic: the sheet prompt asks the LLM to identify whether a character has distinct forms and describe each, so it generalises to any book (werewolves, disguises, transformations) without per-book rules.

Implemented as bounded sheet.forms[] data inside the existing character record. The extraction prompt asks for distinct forms, the detail page switches the active appearance, selected forms feed prompt/fingerprint generation, and generated/uploaded portrait/reference assets remain attached to that form. One character ID and one voice are preserved for attribution. A future polish pass can add richer inline form editing and per-form bulk-generation UX.

P1 — Portrait / concept-sheet identity mismatch

The avatar and the concept sheet can show two different people (Lancorian: dark-haired man vs white-haired mage). Cause is understood: image_prompt has two sources — the LLM writes it as one of the 53 character-sheet fields, while _build_character_portrait_prompt builds it deterministically from the same structured facts the concept sheet uses. The deterministic one keeps both images consistent; the LLM-written one omits identity anchors (age, hair, build) so the model invents them.

Status — resolved 2026-09-19

Verified against the code and covered by tests (see architecture/character-visual-identity.md):

One canonical identity description feeds both the avatar and the concept sheet (_character_visual_facts; portrait and reference prompts are built deterministically, the active appearance form is overlaid first).
image_prompt is no longer authored by the LLM during sheet extraction (server and browser prompts); the deterministic prompt is generated right after extraction, before avatar-only generation and before per-form generation. Hand-edited prompts are never overwritten.
Prompt version, identity fingerprint and asset digests are stored with every generated pair — for the base appearance too (they used to be lost on reload) and per form.
Review indicator: per-character banner and GET /api/characters/visual-pair-review, now form-aware; a pair is also flagged when one image was regenerated alone.
Tests: server/browser fingerprint parity, prompt anchors shared by both builders, review endpoint (base + forms), and Playwright persistence tests that fail on the previous code.

Still true, and deliberately not "fixed": the resolved prompt text is not stored per image (only its version and the fingerprint of the facts it was built from).

Known regression from the 2026-09-02 batch

Regenerating the top-10 profiles from evidence windows (passages around each character's own dialogue) produced weaker physical text than the app's full-book pass — Lancorian's became behaviour ("peers through small windows, moves nimbly on spiral staircases") rather than appearance, where the older sheet had "shoulder-length white hair, faded scar across forehead". Prefer the app's own Character Sheets flow over evidence-window shortcuts when appearance quality matters.

Magazine: deleting a picture/chart/table region leaves its crop file and article reference behind. Found 2026-09-22 while cleaning up two invented figure regions by hand. delete_region()/DELETE .../regions/{block_id} matches only the block's own id; a picture/chart/table's derived crop has a different id (p{page}-region-{block_id}, from crop_visual_regions), so deleting the block leaves that crop file on disk and its id in any article's images list (the article would still try to show a picture of content that's no longer recognised as one). Fix: when the deleted id is a block with a crop derived from it, also remove the p{page}-region-{id} image entry, delete its file, and strip that derived id (not just the block id) from every article's images list.
Audiobook character image runs must be isolated per character. Before generating a new character's profile image or character sheet, clear/reset every generation field, reference image and transient conditioning value inherited from the preceding character. Profile and sheet images for the same character should continue to share identity, but no appearance traits (for example dwarf stature) may leak into the next character. Add a regression test that generates two deliberately dissimilar characters consecutively and verifies that the second request contains only its own identity/features/references.
Characters table on phones: the sticky name column covers the rating stars. At iPhone width the star buttons in the Characters table cannot be tapped (Playwright: td.lib-chars-tbl-name intercepts the click). Found 2026-09-29; the phone-width case of tests/characters.spec.js "profile sorting, column visibility, and voice rating" is test.fixme until fixed.

72 KiB Raw Blame History Unescape Escape