mARTin-B78/tts-voice-creator-clone-and-design-2

Test and SonarQube / quality (push) Waiting to run

Details

fix(magazine): derive figure boundaries from the page render

A vision OCR engine grounds a figure far less precisely than it
transcribes one. Over all 138 figure regions of a 204-page issue the
transcribed contents were right while the boxes were routinely offset,
undersized, or sitting on the neighbouring text column -- page 23's card
grid was reported at x 0.24-0.54 where the graphic runs 0.41-0.95.

snap_visual_bboxes() reduces the page render to an ink-coverage grid,
seeds a connected-component search with the engine's box, and re-joins
tiles a gutter apart so a card grid comes back as one figure. Text
blocks are excluded from the mask, which is what stops a figure
swallowing the column beside it. A candidate must be solidly inked or
framed on nearly all of its rows and columns (admitting a white-interior
screenshot), may not shrink below 30% of the seed, and reverts when two
figures land on the same graphic; anything else keeps the box it had.

Runs before the figure crop in both OCR paths, so the images shown
during playback are framed correctly too. Replaces refine_picture_
bounds(), which only recognised coloured panels and had no text fence.

Also adds "Fix figure boundaries" in review and POST
/api/magazine/{id}/figures/refine behind it, so an issue ingested before
this pass can be corrected without re-running OCR. Verified live against
c't KI Wissen 2026: 78 of 138 boxes corrected, the rest declined.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-09-12 10:17:00 +02:00

4.5 KiB

Raw Blame History

Project map

Living index of where things are and what state they are in. Update in the same commit as the change it describes.

Current phase

Magazine projects, P0 and P1 complete (frontend reachable at Studio → Magazines). Audiobook/Rehearser/voice-library features are mature and in daily use.

Deployment

Self-hosted Docker on the DGX Spark, port 7890, image ghcr.io/martin-b78/tts-voice-creator-clone-and-design-2. Python and static files are bind-mounted, so code changes need only a container recreate; rebuild and push only when Dockerfile or requirements.txt change.

Layout

Path	Role
`server.py`	FastAPI app, router registration, auth middleware
`core/`	domain logic — config, database, audio, voice, routing, magazine
`routes/`	HTTP boundary, one module per feature area
`static/js/`	frontend, one module per feature area (no build step for most)
`tests/`	`test_.py` (unittest) and `.spec.js` (Playwright)
`architecture/`	SOPs for subsystems whose design is not obvious from code
`workflows/`	ComfyUI graphs for character portrait generation

Feature areas

Area	Server	Frontend	State
Voice library / clone / design	`routes/library.py`, `core/voice.py`	`library.js`	mature
Read Aloud (books, PDFs)	`routes/reader.py`	`reader.js`	mature
Audiobook casting	`routes/conversation.py`	`audiobook*.js`	mature
Rehearser	`routes/rehearsals_db.py`	`rehearser-*.js`	mature
Studio (phase wizard)	—	`studio.js`	mature
TTS engines / routing	`routes/tts.py`, `core/routing.py`	`engines.js`	mature
STT	`routes/stt.py`	—	mature
Magazines	`core/magazine.py`, `routes/magazine.py`	`magazine.js`, `s-magazine.html`	P0 + P1

Data model

Documents (books and magazines alike) live under <CONFIG_DIR>/reader_library/<doc_id>/, discriminated by meta.json → kind:

kind: "pdf" / text — Read Aloud book. Text is extracted in the browser and not persisted; meta.json holds counts and a flat idx resume position.
kind: "magazine" — adds layout.json, articles.json, pages/, images/, podcast/, and a per-article progress map. See architecture/magazine-ingestion.md.

Settings, presets, routes and the SQLite database also live under CONFIG_DIR, which is the writable volume.

Environment

Variable	Purpose
`CONFIG_DIR`	writable state (settings, library, database)
`VOICES_DIR` / `OUTPUT_DIR`	voice WAV library
`TTS_CONTAINER_NAMES`	containers restarted after a voice change
`DOCKER_HOST`	points at the socket proxy; the raw socket is not mounted
`llm_url` / `llm_model` / `llm_api_key`	settings.json, OpenAI-compatible LLM

Pending decisions

Container registry: ghcr costs quota against a private package (≈427 MB per push, 500 MB free on the Free plan). Local registry vs. ghcr vs. no re-pull is unresolved — the compose files currently point at ghcr.

Known limitations

gitea backup remote is 157 commits behind origin; main tracks gitea.
Magazine sidebar exclusion is a narrow-column geometry heuristic (mark_sidebars), not certain — review always shows every sidebar block.
Magazine word-level highlighting is a character-length estimate against real measured audio duration, not true forced alignment.
No formal cross-engine OCR accuracy benchmark exists (two engines wired: ocr-baidu-unlimited-ocr primary, KDL Frontier Parser nano fallback).
Figure boundaries are re-derived from the page render (snap_visual_bboxes) because vision OCR grounds figures loosely; it corrected 78 of 138 figure boxes on c't KI Wissen 2026 and deliberately declines the rest. A pale chart or line drawing on white is below its ink threshold and keeps the engine's box — draw those by hand in review.
No npm run qa aggregate command yet; Python tests run via unittest, browser tests via playwright test.
The app has no authentication for the browser UI and no per-user data isolation — single-tenant/LAN-trust only (one global settings/database/voice library, no accounts). See architecture/multi-tenant-and-identity.md and TODO.md's "Deferred — public multi-user readiness" section.

Next logical step

Magazine P2 per TODO.md: image descriptions, table/chart handling, EPUB 3 Media Overlay export, and a formal OCR-engine benchmark (see architecture/magazine-ingestion.md).

4.5 KiB Raw Blame History