mARTin-B78
71a398b82f
Studio -> Magazines: library, contents tree grouped by section with
confidence/pages/listening-time/audio status, the Stage D review
screen (tick/untick layout blocks, correct title/author, approve or
reject), a reflowed article view with click-to-seek sentence
highlighting, a linked original-page view showing the source bounding
box, and a podcast panel (2-3 host pickers, script generation,
turn-by-turn playback).
Wired into the existing lazy-module system (module-manifest.json,
loader.js, nav.js) rather than a separate app. Reuses /api/tts-preview
for both narration and podcast turns -- no new audio pipeline.
Fixes found building this:
- The pane toggle used role="tab" without the full ARIA tabs pattern
(aria-selected, roving tabindex, arrow-key nav) -- fails WCAG 2.2 AA
and hid the buttons from role-based tooling. Replaced with a button
group + aria-pressed, matching the plain-click interaction it
actually has.
- Print layout cues ("(continued on page 8)") were being sent to TTS
verbatim -- narration would have spoken the parenthetical aloud.
Stripped from the narration copy only; layout.json keeps the
original for the review screen.
- has_podcast now comes from the articles listing so opening an
article without a generated script skips a doomed GET instead of a
routine 404 on every open.
Verified against the real API contract with Playwright (library,
review pre-selection, approval, sentence highlighting + PDF-page
linking, podcast approval gate, zero console errors across the flow)
and manually end-to-end against a real ingested fixture, screenshots
included. Full suite: 94 Python tests, 33/34 desktop Playwright tests
(one unrelated pre-existing flaky DOM-count assertion).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
193 lines
9.5 KiB
Markdown
193 lines
9.5 KiB
Markdown
# SOP — Magazine ingestion
|
||
|
||
Status: **P0 complete. P1 mostly complete** — Studio -> Magazines has the full
|
||
library/contents/review/read/podcast flow. Remaining P1: images shown during
|
||
playback and a cross-article play-all queue.
|
||
|
||
## Why magazines are not books
|
||
|
||
The audiobook pipeline assumes one continuous text in reading order. A magazine
|
||
breaks every part of that assumption: several unrelated articles share a page,
|
||
columns run in parallel, adverts and sidebars interrupt body copy, and a feature
|
||
routinely jumps twenty pages to its tail. Narrating a magazine page-by-page
|
||
produces audio that walks out of an article mid-sentence and into an advert.
|
||
|
||
**The canonical object is therefore the article, not the page.** Everything
|
||
below exists to reconstruct articles from a flat PDF.
|
||
|
||
## Pipeline
|
||
|
||
Implemented in [`core/magazine.py`](../core/magazine.py); driven by
|
||
[`routes/magazine.py`](../routes/magazine.py).
|
||
|
||
```
|
||
PDF
|
||
→ extract_pages() PyMuPDF: text blocks with bbox, font size/name, weight
|
||
→ ocr_page() tesseract TSV fallback when a page has <120 native chars
|
||
→ classify_blocks() title / deck / byline / body / caption / pullquote /
|
||
sidebar / furniture / ad / figure
|
||
→ mark_advert_pages() spread an "ADVERTISEMENT" label across its whole page
|
||
→ mark_pullquotes() demote display text that merely repeats body copy
|
||
→ assign_reading_order() column detection, then top-to-bottom within a column
|
||
→ score/parse_contents() Stage A: locate and parse the table of contents
|
||
→ group_articles() Stage C: group blocks, link "continued on page N"
|
||
→ human review Stage D, in the UI — nothing is narrated until approved
|
||
```
|
||
|
||
### Why server-side, not in the browser
|
||
|
||
Text extraction for plain PDFs runs client-side (PDF.js), and that stays. Layout
|
||
analysis does not, for three reasons: PyMuPDF exposes real font metrics and
|
||
embedded-image rectangles that PDF.js does not; OCR coordinates from tesseract
|
||
merge with those bboxes in one coordinate space; and the result is expensive
|
||
enough (minutes for an issue) that it must be computed once and persisted, not
|
||
recomputed per browser session.
|
||
|
||
PDF.js remains responsible for *rendering* in the PDF view.
|
||
|
||
Docling/Unstructured were considered and rejected: they pull torch plus layout
|
||
model weights (multi-GB, ARM64 wheel risk on the DGX Spark, GPU contention with
|
||
the TTS stack) to improve on heuristics that already work on the cases we hit.
|
||
Revisit only if real issues show grouping failures the heuristics cannot fix.
|
||
|
||
## Decisions that are easy to get wrong
|
||
|
||
**Font baseline is document-wide, never per page.** `classify_blocks` judges
|
||
size as a ratio against `base_size`. A page carrying a full-bleed photograph and
|
||
one caption has no body copy of its own, so a per-page baseline makes that
|
||
caption its own reference and classifies it as body text. `extract_pages`
|
||
measures the baseline once across every block in the issue.
|
||
|
||
**Reading order is column-major.** Sorting blocks by `y` interleaves parallel
|
||
columns, splicing the left column's sentence into the right column's. Columns
|
||
are inferred from vertical whitespace (where blocks are *absent*), because
|
||
magazines change column count per page — a fixed grid is wrong. Blocks spanning
|
||
>60% of the page width sort ahead of both columns, so a full-width headline
|
||
precedes the body it introduces.
|
||
|
||
**Furniture is identified by repetition, not position.** A running head and a
|
||
headline both sit at the top of a page. What separates them is that the running
|
||
head recurs on many pages at the same edge; `find_repeated_lines` needs the
|
||
whole document, so classification waits until every page has been read.
|
||
|
||
**A page is claimed only if it contributes.** An advert between two features
|
||
starts no article, so without an explicit check it gets swallowed by the
|
||
preceding one and highlighted as part of it in the PDF view.
|
||
|
||
**Pull quotes are demoted.** Narrating one reads the same sentence twice a few
|
||
seconds apart. A display-sized block whose text already appears in the body copy
|
||
is a pull quote, not a deck.
|
||
|
||
## Confidence, and why review is mandatory
|
||
|
||
| Signal | Confidence |
|
||
|---|---|
|
||
| Anchored by a contents entry | 0.90 |
|
||
| Started by a headline, no contents entry | 0.55 |
|
||
| Article whose blocks contain no body copy | ×0.4 |
|
||
|
||
Nothing is narrated until a human sets `status: "approved"`. This is deliberate:
|
||
a five-second correction beats a 30-minute recording with a sidebar in the
|
||
middle. `POST …/podcast` refuses unapproved articles outright.
|
||
|
||
## Storage
|
||
|
||
Magazines live in the Read Aloud store and are distinguished by
|
||
`meta.json → kind: "magazine"`, so the existing library listing, cover, progress
|
||
and audio endpoints keep working.
|
||
|
||
```
|
||
<CONFIG_DIR>/reader_library/<doc_id>/
|
||
meta.json kind, title, issue, ingest status, default_voice,
|
||
podcast hosts, per-article progress
|
||
source.pdf the original
|
||
layout.json every classified block, with bbox + reading order
|
||
articles.json detected articles; status gates narration
|
||
pages/NNNN.png rendered pages (PDF view + OCR input)
|
||
images/*.png extracted figures ≥64px
|
||
podcast/<id>.json generated host scripts
|
||
audio/<idx>.mp3 synthesised audio (existing reader endpoints)
|
||
```
|
||
|
||
`layout.json` is the join table between audio, text, image and PDF coordinates:
|
||
every block carries `page` + `bbox`, so a sentence in the reflowed view can jump
|
||
to its source region on the rendered page.
|
||
|
||
## Podcast mode
|
||
|
||
Per article, alongside narration — not instead of it. An approved article's
|
||
narratable blocks are sent to the configured LLM (`llm_url` / `llm_model`,
|
||
reusing `_post_llm_chat_completion` from `routes/conversation.py`) with a system
|
||
prompt that forbids inventing facts and demands strict JSON. The parser tolerates
|
||
a markdown fence and surrounding prose, and reassigns unknown host labels rather
|
||
than dropping the turn — an unrecognised label would otherwise leave a line with
|
||
no voice. 2–3 hosts, each bound to a voice the user picks; synthesis reuses the
|
||
multi-speaker path the audiobook casting already uses.
|
||
|
||
## Frontend
|
||
|
||
`static/js/magazine.js` + `static/sections/s-magazine.html`, wired into the same
|
||
lazy-module system as the other sections (`static/module-manifest.json`,
|
||
`static/loader.js`, `static/nav.js`). Two independent section lists gate
|
||
visibility (`loader.js` fetches the HTML, `nav.js` decides `navTo` will reveal
|
||
it) — both needed updating; missing either one leaves the section perpetually
|
||
hidden even though its HTML loaded correctly.
|
||
|
||
State machine: `library -> contents -> review -> read`, plus a `pane` toggle
|
||
(`reflow` | `pdf`) within `read`. Sentences are split from narratable blocks
|
||
client-side, each tagged with its source `blockId`/`page`/`bbox`, so clicking a
|
||
sentence or switching to the PDF pane draws the highlight rectangle from the
|
||
same coordinate the layout pipeline recorded — no separate sync data.
|
||
|
||
Reuses `/api/tts-preview` for both narration and podcast turns; no new audio
|
||
pipeline. `magazine_articles` marks each article `has_podcast` so the read view
|
||
skips a doomed status probe when nothing was generated, rather than a routine
|
||
404 on every open.
|
||
|
||
The pane toggle is a button *group* with `aria-pressed`, not `role="tab"` —
|
||
this app's tab pattern would additionally require `aria-selected`, roving
|
||
`tabindex` and arrow-key navigation, none of which plain click switching
|
||
provides. `role="tab"` without that whole pattern fails WCAG 2.2 AA guidance
|
||
and, in practice, hides the buttons from `getByRole('tab')`-oriented tooling.
|
||
|
||
## Tests
|
||
|
||
- [`tests/test_magazine.py`](../tests/test_magazine.py) — 34 tests on the pure
|
||
geometry/classification logic: column reading order, the document-baseline
|
||
regression, furniture, pull quotes, contents parsing, continuation linking,
|
||
advert pages.
|
||
- [`tests/test_magazine_routes.py`](../tests/test_magazine_routes.py) — 10 tests: path
|
||
traversal rejection, review gating, LLM script parsing including malformed
|
||
output.
|
||
- [`tests/magazine.spec.js`](../tests/magazine.spec.js) — Playwright, backend mocked to the
|
||
real response contract: library listing, review pre-selection, approval,
|
||
sentence highlighting + PDF-page linking, the podcast approval gate, and a
|
||
full-flow console-error check.
|
||
|
||
Both run without PyMuPDF: the I/O functions import it lazily so the logic worth
|
||
testing needs no native dependency or sample PDF.
|
||
|
||
Verified end-to-end against a synthetic 8-page issue (contents page, two-column
|
||
feature with a byline and a page-8 jump, full-page advert, second feature,
|
||
reviews) and against an image-only rendering of the same file to exercise OCR.
|
||
|
||
## OCR engine selection (not yet wired)
|
||
|
||
Engines → OCR & Document Parsing lists layout-aware parsers (KDL Frontier Parser
|
||
nano, Baidu Unlimited-OCR) as connection-probed cards. Those are presentation
|
||
and health-check only — there is no server-side dispatch, so ingestion currently
|
||
always uses local tesseract via `ocr_lang`. Routing a page image to one of those
|
||
vLLM endpoints and mapping its structured output onto `Block` is real work, not
|
||
a config change; it is the natural home for TODO P2's per-page fallback
|
||
selection and benchmark comparison.
|
||
|
||
## Known limitations
|
||
|
||
- Sidebar detection (`BLOCK_SIDEBAR`) is defined but not yet inferred; sidebars
|
||
currently classify as body and must be excluded by hand in review. P2.
|
||
- `continued from page N` is parsed but not used for linking — only the forward
|
||
`continued on page N` reference drives it.
|
||
- Reviews-style pages with no headline are absorbed by the preceding article
|
||
when the contents page does not list them.
|
||
- Word-level timing is not available; P1 targets sentence highlighting.
|