mARTin-B78
55f147d7a7
A vision OCR engine grounds a figure far less precisely than it
transcribes one. Over all 138 figure regions of a 204-page issue the
transcribed contents were right while the boxes were routinely offset,
undersized, or sitting on the neighbouring text column -- page 23's card
grid was reported at x 0.24-0.54 where the graphic runs 0.41-0.95.
snap_visual_bboxes() reduces the page render to an ink-coverage grid,
seeds a connected-component search with the engine's box, and re-joins
tiles a gutter apart so a card grid comes back as one figure. Text
blocks are excluded from the mask, which is what stops a figure
swallowing the column beside it. A candidate must be solidly inked or
framed on nearly all of its rows and columns (admitting a white-interior
screenshot), may not shrink below 30% of the seed, and reverts when two
figures land on the same graphic; anything else keeps the box it had.
Runs before the figure crop in both OCR paths, so the images shown
during playback are framed correctly too. Replaces refine_picture_
bounds(), which only recognised coloured panels and had no text fence.
Also adds "Fix figure boundaries" in review and POST
/api/magazine/{id}/figures/refine behind it, so an issue ingested before
this pass can be corrected without re-running OCR. Verified live against
c't KI Wissen 2026: 78 of 138 boxes corrected, the rest declined.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
7.2 KiB
User manual
Living manual. Every feature updates this file in the same commit as the code.
Sections for the mature features (voice library, Read Aloud, audiobook casting, Rehearser) are still to be written — this file was created alongside the magazine feature and currently documents that area only.
Magazines
Purpose
Listen to a magazine or newspaper one article at a time, from its true beginning to its true end, without adverts, sidebars or a neighbouring article spliced into the middle. Each article can be narrated by a single voice, or turned into a podcast discussion between two or three voices you pick.
Prerequisites
- The magazine as a PDF (native text or scanned — both work).
- At least one voice in the library for narration.
- For podcast mode: an LLM configured in Settings (
llm_url,llm_model), and two or three voices chosen as hosts.
Step by step
- Add the magazine. Studio → Add Magazine / Paper. Give it a title and issue, then upload the PDF.
- Ingest. Ingestion runs in the background — you can close the browser. It renders every page, extracts text (falling back to OCR on scanned pages), classifies each region of the layout, finds the contents page, and proposes articles with page ranges. While it runs, a live preview shows each page's image next to its recognised text as soon as that page finishes, so you can see ingestion actually working page by page instead of watching a bare progress bar.
- Review the articles. This step is not optional and cannot be skipped. Each proposed article shows its title, author, page range, estimated listening time and a confidence score. The review screen is split: the scanned page on the left with a numbered, icon-labelled box per detected region (title, byline, body text, caption, figure…), and the same regions as text on the right — click either side to highlight the other. Sidebars and pull-out boxes are already unticked for you — tick one back in if you actually want it read. A region already claimed by a different article on the same page is marked with that article's number, so it is never mistaken for unclaimed text. If OCR quality is poor, "OCR again" re-reads just the current page through the configured vision engine (or "OCR again (whole magazine)" from the contents screen for every page, which also resets any articles already reviewed). If a figure's box is drawn in the wrong place or at the wrong size — a common miss, because an OCR engine locates a photo far less precisely than it reads type — "Fix figure boundaries" re-derives every figure box in the issue from the scanned pages themselves. It calls no OCR engine, changes no text and touches no approval, so it is safe to run at any point, including on an issue you are halfway through reviewing; boxes it cannot place confidently are left exactly as they were. If the layout engine missed a region entirely, "Draw it" lets you click-drag a box directly on the page image and add it as text (typed or OCRed) or as a new figure. Confirm the boundaries, correct the title, then Approve.
- Listen. An approved article can be narrated with the default voice, or sent to podcast mode. Use Play all articles (on the contents screen) to listen through every approved article back to back without returning to this screen between them. A figure from the page is shown automatically as playback reaches it. Word highlighting can be turned off if you find it distracting — it is an estimate based on word length, not a measurement, so it will not always land exactly on the spoken word. Driving mode (the button next to the article title) switches to two large controls and nothing else, for listening without looking at the screen.
Expected result: the article plays from its headline to its final paragraph. Page furniture (running heads, page numbers), adverts, sidebars and pull quotes are never read aloud unless you explicitly include them in review.
Confidence, and what it means
| Shown | Meaning |
|---|---|
| ~90% | The contents page listed this article — title and start page are known. |
| ~55% | Detected from a headline alone; check the boundaries carefully. |
| Low | No body copy found. Usually an advert or a section divider — reject it. |
An article is never narrated until you approve it, whatever its confidence.
Podcast mode
Choose two or three voices as hosts. The configured LLM writes a discussion of that article for listeners who have not read it, and each host's turns are synthesised with their voice.
The prompt forbids inventing facts, figures or quotes that are not in the article; if the article reaches no conclusion the hosts say so. It is still a generated discussion, not the author's words — treat it as a summary, and read the original for anything that matters.
Podcast generation is refused for an article you have not approved, because unreviewed boundaries may include a neighbouring article or an advert.
Errors
| What you see | What happened | What to do |
|---|---|---|
| "Upload the PDF first" | The project has no source.pdf. |
Upload it, then ingest. |
| Ingestion failed | The PDF is encrypted or corrupt. | Check it opens in a viewer. |
| "Approve the article before generating a podcast" | Review gate. | Approve it first. |
| "No LLM model configured" | llm_model is empty. |
Set it in Settings. |
| "LLM did not return usable JSON" | The model ignored the format. | Retry, or pick a stronger model. |
| "Article has too little text" | Under 200 characters — usually a stub. | Check the boundaries in review. |
| A figure box sits over the wrong part of the page | The OCR engine grounded the figure loosely. | Press "Fix figure boundaries"; if the graphic is too pale to detect, draw the region by hand. |
| "Fix figure boundaries" reports nothing corrected | Every box already matched its page, or none could be placed with confidence. | Nothing is wrong; correct any remaining box by drawing it. |
Privacy
Ingestion, OCR and storage are entirely local to the container. Podcast mode is
the exception: the article's text is sent to whatever LLM llm_url points
at. If that is a hosted API rather than a model on your own network, the
article leaves your machine. Narration uses your local TTS stack and sends
nothing outward.
Accessibility and mobile
Sentence and word highlighting both respect prefers-reduced-motion (no
smooth-scroll animation when it is set; the highlight itself is a colour
change, not motion). Driving mode gives two large, high-contrast controls
(skip, play/pause) with nothing else to interact with, for use without
looking at the screen — it does not, and should not, replace watching the
road. Word-level highlighting can be switched off in the reading controls if
it is more distracting than helpful; sentence highlighting keeps working
either way. Keyboard navigation of the contents tree and review screen is
not yet built.
Offline
Ingested magazines are stored server-side, so any device on the network sees the same library and reading progress. Playback needs the server; there is no offline cache yet.