tts-voice-creator-clone-and-design-2/architecture/magazine-ingestion.md

SOP — Magazine ingestion

Status: P0 complete. P1 mostly complete — Studio -> Magazines has the full library/contents/review/read/podcast flow. Remaining P1: images shown during playback and a cross-article play-all queue.

Why magazines are not books

The audiobook pipeline assumes one continuous text in reading order. A magazine breaks every part of that assumption: several unrelated articles share a page, columns run in parallel, adverts and sidebars interrupt body copy, and a feature routinely jumps twenty pages to its tail. Narrating a magazine page-by-page produces audio that walks out of an article mid-sentence and into an advert.

The canonical object is therefore the article, not the page. Everything below exists to reconstruct articles from a flat PDF.

Pipeline

Implemented in core/magazine.py; driven by routes/magazine.py.

PDF
 → extract_pages()        PyMuPDF: text blocks with bbox, font size/name, weight
 → vision_ocr_page()      connected vision OCR engine, when a page needs OCR or its
                          text scores poorly (ocr_quality_score) -- native or
                          tesseract alike. Labels footer/title/page_number/caption
                          directly. Off unless configured; never fatal.
 → ocr_page()             tesseract TSV fallback: no vision engine configured, or
                          the vision call failed/timed out/was unreachable
 → build_vision_blocks()  vision regions -> Blocks; a direct label bypasses
                          classify_blocks entirely (see below)
 → classify_blocks()      title / deck / byline / body / caption / pullquote /
                          sidebar / furniture / ad / figure -- for native/tesseract
                          blocks, and any vision region left ambiguous ("text")
 → mark_advert_pages()    spread an "ADVERTISEMENT" label across its whole page
 → mark_pullquotes()      demote display text that merely repeats body copy
 → assign_reading_order() column detection, then top-to-bottom within a column
 → cluster_running_furniture()  fuzzy-match near-edge repeats tesseract's noisy
                          OCR defeated exact matching on (see below)
 → score/parse_contents() Stage A: locate and parse the table of contents
 → group_articles()       Stage C: group blocks, link "continued on page N"
 → human review           Stage D, in the UI — nothing is narrated until approved

Why server-side, not in the browser

Text extraction for plain PDFs runs client-side (PDF.js), and that stays. Layout analysis does not, for three reasons: PyMuPDF exposes real font metrics and embedded-image rectangles that PDF.js does not; OCR coordinates from tesseract merge with those bboxes in one coordinate space; and the result is expensive enough (minutes for an issue) that it must be computed once and persisted, not recomputed per browser session.

PDF.js remains responsible for rendering in the PDF view.

Docling/Unstructured were considered and rejected: they pull torch plus layout model weights (multi-GB, ARM64 wheel risk on the DGX Spark, GPU contention with the TTS stack) to improve on heuristics that already work on the cases we hit. Revisit only if real issues show grouping failures the heuristics cannot fix.

Decisions that are easy to get wrong

Font baseline is document-wide, never per page. classify_blocks judges size as a ratio against base_size. A page carrying a full-bleed photograph and one caption has no body copy of its own, so a per-page baseline makes that caption its own reference and classifies it as body text. extract_pages measures the baseline once across every block in the issue.

Reading order is column-major. Sorting blocks by y interleaves parallel columns, splicing the left column's sentence into the right column's. Columns are inferred from vertical whitespace (where blocks are absent), because magazines change column count per page — a fixed grid is wrong. Blocks spanning

60% of the page width sort ahead of both columns, so a full-width headline precedes the body it introduces.

Furniture is identified by repetition, not position. A running head and a headline both sit at the top of a page. What separates them is that the running head recurs on many pages at the same edge; find_repeated_lines needs the whole document, so classification waits until every page has been read.

Exact-match repetition is not enough once OCR is involved. Confirmed live on a scanned c't magazine: the running footer "c't Wissen 2026" was read differently by tesseract on nearly every page (cE KI-WISSEN, ce Wissen 2136, cA Wissen 2036, …), so find_repeated_lines's exact match never recognised them as the same line — each became its own spurious title and started a new article, taking a 28-page scan from a handful of real sections to 26 near-1-per-page "articles". cluster_running_furniture fuzzy-matches near-edge blocks (digit-stripped, difflib.SequenceMatcher similarity ≥ 0.6, requiring ≥3 occurrences) as a second pass over the whole document. A vision OCR engine that labels the footer directly (see below) makes this pass unnecessary for the pages it covers, but tesseract-sourced pages still need it.

OCR font size is a measurement for native text, an estimate for OCR text. PyMuPDF gives an exact digital font size per span. Tesseract only gives a line bounding box, and its own line SEGMENTATION is unreliable on a scanned magazine page — ocr_page groups by paragraph (matching PyMuPDF's own grouping) and uses the MEDIAN of the paragraph's line heights, not its total bbox height, which would scale with line count instead of type size. A vision OCR engine gives one box per paragraph with no per-line data at all, so its "text"-labeled (ambiguous) blocks get a synthesized estimate instead (estimate_font_size_from_area, derived from the box's area and the text's length) — good enough to feed the same title/deck/pullquote thresholds, not a real measurement.

A vision-engine label is trusted outright, never re-derived. When the engine labels a region footer/title/page_number/caption directly (build_vision_blocks), classify_blocks skips it (source == "vision_label") rather than reclassifying it from an estimated font size — that would throw away a more direct signal than any heuristic can reconstruct after the fact. Only an ambiguous "text" label (or one the engine invents) goes through the normal classification pass.

A page is claimed only if it contributes. An advert between two features starts no article, so without an explicit check it gets swallowed by the preceding one and highlighted as part of it in the PDF view.

Pull quotes are demoted. Narrating one reads the same sentence twice a few seconds apart. A display-sized block whose text already appears in the body copy is a pull quote, not a deck.

Confidence, and why review is mandatory

Signal	Confidence
Anchored by a contents entry	0.90
Started by a headline, no contents entry	0.55
Article whose blocks contain no body copy	×0.4

Nothing is narrated until a human sets status: "approved". This is deliberate: a five-second correction beats a 30-minute recording with a sidebar in the middle. POST …/podcast refuses unapproved articles outright.

Storage

Magazines live in the Read Aloud store and are distinguished by meta.json → kind: "magazine", so the existing library listing, cover, progress and audio endpoints keep working.

<CONFIG_DIR>/reader_library/<doc_id>/
    meta.json           kind, title, issue, ingest status, default_voice,
                        podcast hosts, per-article progress
    source.pdf          the original
    layout.json         every classified block, with bbox + reading order
    articles.json       detected articles; status gates narration
    pages/NNNN.png      rendered pages (PDF view + OCR input)
    images/*.png        extracted figures ≥64px
    podcast/<id>.json   generated host scripts
    audio/<idx>.mp3     synthesised audio (existing reader endpoints)

layout.json is the join table between audio, text, image and PDF coordinates: every block carries page + bbox, so a sentence in the reflowed view can jump to its source region on the rendered page.

Podcast mode

Per article, alongside narration — not instead of it. An approved article's narratable blocks are sent to the configured LLM (llm_url / llm_model, reusing _post_llm_chat_completion from routes/conversation.py) with a system prompt that forbids inventing facts and demands strict JSON. The parser tolerates a markdown fence and surrounding prose, and reassigns unknown host labels rather than dropping the turn — an unrecognised label would otherwise leave a line with no voice. 2–3 hosts, each bound to a voice the user picks; synthesis reuses the multi-speaker path the audiobook casting already uses.

Frontend

static/js/magazine.js + static/sections/s-magazine.html, wired into the same lazy-module system as the other sections (static/module-manifest.json, static/loader.js, static/nav.js). Two independent section lists gate visibility (loader.js fetches the HTML, nav.js decides navTo will reveal it) — both needed updating; missing either one leaves the section perpetually hidden even though its HTML loaded correctly.

State machine: library -> contents -> review -> read, plus a pane toggle (reflow | pdf) within read. Sentences are split from narratable blocks client-side, each tagged with its source blockId/page/bbox, so clicking a sentence or switching to the PDF pane draws the highlight rectangle from the same coordinate the layout pipeline recorded — no separate sync data.

Reuses /api/tts-preview for both narration and podcast turns; no new audio pipeline. magazine_articles marks each article has_podcast so the read view skips a doomed status probe when nothing was generated, rather than a routine 404 on every open.

The pane toggle is a button group with aria-pressed, not role="tab" — this app's tab pattern would additionally require aria-selected, roving tabindex and arrow-key navigation, none of which plain click switching provides. role="tab" without that whole pattern fails WCAG 2.2 AA guidance and, in practice, hides the buttons from getByRole('tab')-oriented tooling.

Tests

• tests/test_magazine.py — 80 tests on the pure geometry/classification logic: column reading order, the document-baseline regression, furniture, pull quotes, contents parsing, continuation linking, advert pages, paragraph-grouped OCR, fuzzy running-furniture matching, and the vision OCR quality score / region parsing / block building / HTTP call (network mocked).
• tests/test_magazine_routes.py — 10 tests: path traversal rejection, review gating, LLM script parsing including malformed output.
• tests/magazine.spec.js — Playwright, backend mocked to the real response contract: library listing, review pre-selection, approval, sentence highlighting + PDF-page linking, the podcast approval gate, and a full-flow console-error check.

Both run without PyMuPDF: the I/O functions import it lazily so the logic worth testing needs no native dependency or sample PDF.

Verified end-to-end against a synthetic 8-page issue (contents page, two-column feature with a byline and a page-8 jump, full-page advert, second feature, reviews) and against an image-only rendering of the same file to exercise OCR.

Vision OCR escalation

Engines → OCR & Document Parsing lists layout-aware parsers (KDL Frontier Parser nano, Baidu Unlimited-OCR) as connection-probed cards; those cards are still presentation/health-check only. Ingestion itself now dispatches to whichever OpenAI-compatible vision engine is configured (vision_ocr_url / vision_ocr_model in Settings, vision_ocr_enabled gate) — vision_ocr_page sends the rendered page image with a plain transcription prompt and parses the engine's own labeled-region reply (parse_vision_ocr_regions). Escalation triggers per page, not for the whole document: a page needing OCR at all (native_chars < 120), or one whose already-extracted text — native or tesseract — scores below ocr_quality_score's coarse garbage threshold. Any failure (unreachable engine, timeout, unparseable output) returns None and the page falls back to tesseract exactly as if vision OCR were not configured; ingestion never fails because of it.

Verified live against ocr-baidu-unlimited-ocr on a real 28-page scanned c't magazine excerpt (Epson ES-580W, no scanner-side OCR) — every garbled title tesseract had produced was replaced by the real headline:

Before (tesseract)	After (vision OCR)
K | soll Arbeit abnehmen…	1. Irrtum: KI ist ein allgemeiner Jobkiller
. |	(furniture, correctly excluded)
@)X	(furniture, correctly excluded)
Owns mee	Das Team: „Hive Mind" frisst Konzentration
Wie KI Entwickler-Jobs verandert	Wie KI Entwickler-Jobs verändert

Furniture detection went from 44 to 151 blocks on the same document, because the engine labels running headers/footers/page numbers directly instead of cluster_running_furniture having to recover them after the fact from noisy OCR variants. 26 of 28 pages used the vision path; the remaining 2 fell back to tesseract automatically (a transient failure or timeout — not investigated further, since the fallback is exactly the intended behaviour). Article count did not drop much (26 → 25): the residual "over-fragmentation" is now clearly attributable to the document itself — a dossier of numbered "1. Irrtum" / "2. Irrtum" sub-sections and an interview with c't: … question headers, all of which read as headline-fallback article starts absent a contents page — not to OCR quality, which the review screen is exactly the tool for.

ocr_quality_score is deliberately coarse: it discriminates "mostly symbol noise" from "plausible prose" well enough to gate an expensive vision-OCR call, but a single plausible-looking misread inside an otherwise normal sentence (confirmed live: tesseract's "KI" → "K |" substitution, 7 of 8 words still genuine German) scores just as high as clean text. Catching that class of error needs a dictionary or language-model check, out of scope here.

Routing to KDL Frontier Parser nano, or a genuine multi-engine fallback ladder with benchmark comparison, remains TODO P2.

Known limitations

• Sidebar detection (BLOCK_SIDEBAR) is defined but not yet inferred; sidebars currently classify as body and must be excluded by hand in review. P2.
• continued from page N is parsed but not used for linking — only the forward continued on page N reference drives it.
• Reviews-style pages with no headline are absorbed by the preceding article when the contents page does not list them.
• Word-level timing is not available; P1 targets sentence highlighting.