Ingestion now streams each page's image and recognised text into a
console-styled preview panel (Baidu Unlimited-OCR demo style) as pages
finish, instead of a bare progress bar -- raw text is written to a
per-job ingest_preview.json file since the background-job result field
strips text-like keys for data minimization.
Split-screen review boxes and their matching list rows now show a
centred icon for the region's kind and, when a region on a shared page
belongs to a different article, a badge naming that article, so a
claimed region is never mistaken for unclaimed text.
Added a "Draw it" mode: click-drag a box directly on the page image to
add a region the layout engine missed, as typed/OCRed text or as a new
figure (add_manual_region, POST /api/magazine/{doc_id}/pages/{page}/regions).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Split-screen review: the rendered page (left) with a numbered box per
detected block, the extracted text (right) with matching numbers.
Clicking either side highlights the other. Checkboxes moved from a flat
list into this layout without changing their semantics (still gate
what gets narrated).
"OCR again": reocr_page() forces one page through the configured
vision engine again, replacing only that page's blocks -- deliberately
does NOT call build_articles(), so an already-approved article is not
silently re-grouped. A magazine-level option re-reads every page and
re-runs detection (with a confirmation, since that does reset article
status). extract_pages() gained force_vision_ocr to skip its quality
gate for this case.
Building the numbered-box review immediately did what it was built to
do -- surfaced a real bug on live production data (real magazine, real
scan, not a synthetic fixture): a photo caption at 49.9% of the page
width sat in the gutter between two genuinely separate columns and
merged them into one, so every block on the page came back column=0
and reading order fell back to pure top-to-bottom, interleaving the
two columns' sentences. Fixed two compounding causes: column boundary
detection now excludes blocks much wider than the page's median width,
and detect_columns' gutter threshold (3.5%) was itself narrower than
some real magazine gutters (~2.1% measured) -- lowered to 1.5%.
10 new/updated tests, including one built directly from the real
measured page-17 block coordinates that exposed the bug.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Magazine P1:
- Images shown automatically during playback (assign_images_to_articles
links each figure to the article whose column it falls in; the client
shows the last image reached in reading order as playback advances).
- "Play all articles" queues every approved article and advances
automatically at the end of each rather than stopping.
- Word-level highlighting, explicitly documented as an ESTIMATE (no
forced-alignment/timestamp source exists): each word's share of the
sentence's real measured audio duration is proportional to its
character length. A checkbox turns it off; sentence highlighting
(measured, not estimated) is unaffected.
- Driving mode: full-screen overlay, two oversized controls, reuses
whatever is already playing/queued.
- Sidebars/pull-out boxes excluded from narration automatically
(mark_sidebars: an anomalously narrow column vs. the page's regular
grid). Fixed a real defect found while building this: a full-width
headline was left in the column-width analysis and could bridge two
genuinely separate columns into one, hiding a real sidebar column
entirely -- caught by a failing test before the fix.
Second OCR engine + fallback ladder:
- KDL Frontier Parser nano wired in via vision_ocr_page_chain (ordered
engine list, first usable result wins, tesseract last resort).
Confirmed live: ~3x faster than Baidu (~12s vs ~40s/page) but returns
plain text with NO coordinate markers, unlike Baidu's labeled regions.
_whole_page_region treats the whole response as one ambiguous
full-page block (gated by ocr_quality_score) rather than discarding
usable text just because it lacks layout structure.
OCR engines page + a real UI bug:
- The OCR & Document Parsing page had no .s-engines-page[data-page="ocr"]
markup at all -- renderLocalContainers already supported an OCR grid
generically, but $('dc-grid-ocr') was always null, so the card grid
silently never rendered. Added an Active OCR Engine panel mirroring
Active Language Model exactly, plus the Local grid -- both configured
engines now appear as ready-made cards, no "Add Custom" dialog needed.
- Fixed the audiobook pipeline's "Previous/Next" nav bar bleeding into
every .section-head in the document (confirmed live on the Engines
page: a "Cast Audiobook" button and "STEP 1/8" hint that made no sense
there). Changed from a blocklist (naming each unrelated section as
noticed) to an allowlist of the pipeline's three actual destinations.
Verified live end-to-end: play-all queue building and manual advance,
word-span markup, driving-mode toggle, OCR page showing both engines as
local cards, zero stray nav bars on the Engines page, no console errors.
20 new backend tests (156 total in the magazine module).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A page needing OCR (or whose already-extracted text -- native or
tesseract alike -- scores as mostly garbage via ocr_quality_score) is
now escalated to a connected vision OCR engine before falling back to
tesseract. Falls back automatically on any failure (unreachable engine,
timeout, unparseable output); never fatal to ingestion.
Verified live against ocr-baidu-unlimited-ocr on a real 28-page scanned
c't magazine excerpt (Epson ES-580W, no scanner OCR): every garbled
article title tesseract produced ("K | soll Arbeit abnehmen", ". |",
"@)X", "Owns mee") was replaced by the real headline ("1. Irrtum: KI
ist ein allgemeiner Jobkiller", etc.), and furniture detection jumped
from 44 to 151 blocks because the engine labels running headers/
footers/page numbers directly instead of needing fuzzy post-hoc
matching. 26 of 28 pages used the vision path; the remaining 2 fell
back to tesseract automatically.
Also fixes two real defects found investigating the same real-world
data, both regression-tested against the actual garbled strings
observed:
- Exact-match repeated-line detection (running header/footer) silently
failed once tesseract OCR'd the same line differently each time --
cluster_running_furniture fuzzy-matches near-edge blocks instead.
- ocr_page() built one Block per OCR text-LINE rather than per
paragraph, so a single anomalous line (a chart number, scan noise, a
wrapped headline continuation) could independently trigger title
classification -- now grouped by paragraph like the native PyMuPDF
path, using the paragraph's median line height rather than its total
bbox height for font size.
A vision-engine label (footer/title/page_number/caption) is trusted
outright and skips classify_blocks' font-size heuristics entirely,
since the engine gives no reliable font metric; an ambiguous "text"
region gets a synthesized estimate from its paragraph area and length
(estimate_font_size_from_area) and still goes through normal
classification.
Off unless vision_ocr_url/vision_ocr_model are configured
(vision_ocr_enabled setting, default on if reachable); per-request
override via the ingest endpoint body. Default engine URL uses
host.docker.internal, confirmed live -- the OCR container runs on its
own Docker network, not dgx_net, so a container-name default would not
resolve from the app container.
28 new tests (136 total in the module). ocr_quality_score is
deliberately coarse and documented as such: it catches text that is
mostly symbol noise, not a plausible single-word misread inside an
otherwise normal sentence (confirmed live: a misread "KI" -> "K |"
still scored 0.81) -- that class of error needs a dictionary or
language-model check, out of scope here.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Studio -> Magazines: library, contents tree grouped by section with
confidence/pages/listening-time/audio status, the Stage D review
screen (tick/untick layout blocks, correct title/author, approve or
reject), a reflowed article view with click-to-seek sentence
highlighting, a linked original-page view showing the source bounding
box, and a podcast panel (2-3 host pickers, script generation,
turn-by-turn playback).
Wired into the existing lazy-module system (module-manifest.json,
loader.js, nav.js) rather than a separate app. Reuses /api/tts-preview
for both narration and podcast turns -- no new audio pipeline.
Fixes found building this:
- The pane toggle used role="tab" without the full ARIA tabs pattern
(aria-selected, roving tabindex, arrow-key nav) -- fails WCAG 2.2 AA
and hid the buttons from role-based tooling. Replaced with a button
group + aria-pressed, matching the plain-click interaction it
actually has.
- Print layout cues ("(continued on page 8)") were being sent to TTS
verbatim -- narration would have spoken the parenthetical aloud.
Stripped from the narration copy only; layout.json keeps the
original for the review screen.
- has_podcast now comes from the articles listing so opening an
article without a generated script skips a doomed GET instead of a
routine 404 on every open.
Verified against the real API contract with Playwright (library,
review pre-selection, approval, sentence highlighting + PDF-page
linking, podcast approval gate, zero console errors across the flow)
and manually end-to-end against a real ingested fixture, screenshots
included. Full suite: 94 Python tests, 33/34 desktop Playwright tests
(one unrelated pre-existing flaky DOM-count assertion).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Magazines break the audiobook pipeline's assumptions: several unrelated
articles share a page, columns run in parallel, adverts and sidebars
interrupt body copy, and a feature routinely jumps twenty pages to its
tail. Narrating page-by-page walks out of an article mid-sentence and
into an advert, so the canonical object is the article, not the page.
core/magazine.py reconstructs articles from a flat PDF: PyMuPDF extracts
blocks with bboxes and font metrics, tesseract supplies a TSV fallback so
scanned pages join the same column grouping, blocks are classified and
ordered column-major, and articles are anchored by the parsed table of
contents with "continued on page N" tails linked back.
routes/magazine.py adds 13 endpoints. Ingestion is a restart-safe
background job. Nothing is narrated until a human approves the
boundaries; podcast generation refuses unapproved articles.
Podcast mode renders an approved article as a 2-3 host discussion using
the configured LLM and voices the user picks, alongside narration rather
than instead of it.
Verified end-to-end over HTTP on a synthetic 8-page issue: the feature
came back as pages [3, 8] with its jump tail absorbed and the advert page
excluded, and an image-only rendering recovered all 47 blocks via OCR.
44 new tests; full suite 90 passing.
The Studio UI is not built — this is reachable only via the API.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- Deterministic hash-based character portraits and reference sheets
- Implemented strict early Renaissance aesthetic controls and role-specific headwear rules
- Added audiobook batch export scripts and progress tooling
- Verified book profiles with a new 'confirmed' state tracking mechanism
- Updated Pillow dependency for robust image handling
- Configured .gitignore for audiobook and audio cache exports
Re-cast all three volumes end to end and compared each against its existing
cast. Book 03 finished at 9 unknown, below the 25-line cast it replaced, and
its fused multi-turn segments — one voice reading both sides of an exchange —
fell from 60 to 6. Speaker counts stayed level with the hand-corrected books
instead of inflating, confirming the phantom-speaker filter and the label
consolidation both hold.
Agreement with the hand-corrected books is 81-82%. The largest remaining
error is systematic ping-pong inversion, where a whole exchange alternates
onto the wrong speaker, followed by epithets that share no tokens with the
canonical name.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Consolidation ran only at cast completion, but recast/repair passes assign
speakers of their own and can re-introduce the variants just removed —
confirmed live: a freshly cast book gained "Alrik" alongside "Alrik von
Blautann" during auto-repair, splitting one character across two voices again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
One character is referred to by several labels across a book — bare vs full
name, a title on its own, a stray fragment — and each variant became its own
cast entry with its own voice, so the character audibly changed voice
mid-scene. Measured against a hand-corrected book this accounted for more of
the disagreements than genuine misattributions.
Variants are folded together only when one label's words are a strict subset
of another's after stripping articles and titles, so "Weber" and "Alter Weber"
stay separate and unrelated names never merge. Applied to a real cast:
Sharraz -> Sharraz Garthai, Baronin -> Baronin Ira von Seewiesen,
Von -> Oberst Alrik von Blautann (40 lines, 58 -> 54 speakers).
Also moves the quotation-mark toggle into the A- / A+ toolbar row.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CSS cannot target a character inside a text node, so the marks are now wrapped
in their own .ab-q element at render time and hidden with a single class on
body. Toggling is instant regardless of segment count, and the stored text is
never modified.
Also fixes the toggle looking inert on a freshly cast book: it previously only
added missing marks and never hid the ones the book already carried. Button
moved bottom-right as a larger labelled pill, clear of the search controls.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
German prose never quotes narration, so displaying the guillemets makes a
mislabeled line obvious at a glance. Extraction strips them from ~80% of
segments, so they are rendered rather than stored: dialogue rows display
wrapped in » «, lines that kept their own marks are left as-is, and the saved
text is untouched so synthesis and exports are unaffected. A toolbar button
toggles visibility and the choice persists.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adjacent segments merge when they share a speaker, but an unattributed line
carries the speaker "Unknown", so two consecutive Unknown lines compared as
equal and were fused into a single block joined by a blank line. Confirmed
against the source text: Lysandra's line and Perdia's interruption — two
separate paragraphs in the book — became one segment, after which no
attribution pass could separate them and one voice would read both sides of
the exchange.
"Unknown" means the speaker is not known, never that two lines share one.
Narration and lines with a real speaker name merge exactly as before.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A trailing inquit names only the last speaker, so applying it to a block
holding multiple turns silently mis-credited the earlier one — a wrong
attribution is invisible where an Unknown is not. Such blocks are now skipped
by the grammar rules and left for the LLM pass or a manual split.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"Darrags Stimme klang tonlos." after a quote names who just spoke, but the
voice-announcement rule only looked at the narration before a line, so this
mirror image stayed Unknown. Verified on the book (-> Darrag).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Driven by unresolved lines in a real German novel:
- AB_SPEECH_VERBS was missing many common inquit verbs (grunzte, versetzte,
unterbrach, drängte, herrschte, keuchte, erklärte, widersprach, warf ein …)
and had no thought verbs at all, so inner monologue ("dachte Lysandra") was
never attributed.
- A trailing inquit naming two people ("unterbrach die blonde Perdia Lysandra")
resolved to the wrong one; the grammatical subject speaks, so articles and
adjectives are skipped and the first name after the verb wins.
- A quote whose own inquit had been mistyped as dialogue could never resolve,
because the rule required the following segment to be narration. A segment
opening with a lower-case speech verb is an inquit whatever its type; it is
now accepted and its type corrected.
Verified on the book: resolves further lines with no LLM call.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Opening a book always restored the localStorage draft and then pushed it to the
server as a backfill, so a tab holding an older copy silently overwrote newer
work saved from anywhere else — confirmed live: a cast improved from 79 Unknown
lines to 25 was destroyed by opening the book in a stale tab. Local and server
drafts are now compared by savedAt, the newest wins and is cached locally, and
the server is only backfilled when the local copy is actually newer.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Identify/Cast flyouts were hard-anchored to open above their trigger,
which suited the casting panel's bottom footer but clipped them off the top of
the screen in Studio, where the same toolbar sits near the top — the first two
entries were unreachable. They now pick the side with space, prefer opening
downwards, stay clamped inside the viewport and scroll if taller than it.
Also reorders the Identify menu to the sequence the passes should run in, with
the one-click Auto-repair cast first.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Spoken lines that PDF extraction typed as narration were invisible to
"Identify unknown characters" (it only scans type==='dialogue'), so no number
of re-runs could fix them, while the pass that could re-sent all ~1000
narration segments to the LLM. Adds a detector that scores narration for
direct-speech signals and re-checks only those: 36 candidates instead of 1036
on a real novel. Adds "Auto-repair cast" to chain the passes in the right
order automatically after casting.
Also fixes a rollback gap where stopping a run skipped the safety net that
undoes a pass which increased Unknown speakers (confirmed live: 80 -> 159
with no warning), corrects an attribution rule that taught the model the
listener is the speaker, and stops the threshold loop from re-verifying every
narration segment each round.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fish-Speech clones from the app's own saved WAV library but was missing from
the set of backends populated with those voices, so the UI probed the Fish
server for a voice-listing endpoint it does not have, showed "Fetched 0
voices" and left the engine unusable despite being healthy. It now reports
the full library (218 voices).
Also adds a Fish-only panel listing all 49 documented emotion tags with
click-to-insert and a multi-emotion example, and corrects the backend's
advertised capabilities: it was flagged style_aware with "emotion markers are
honoured per request", but measurement shows a reference clip's in-context
prosody overwhelms inline tags (10x loudness spread across emotions without a
reference, 1.2x with one). It is now described by what it does well:
deterministic, byte-reproducible cloning.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The emotion quick-pickers added in 1.20.5 guarded with
`typeof REH_EMOTIONS === 'undefined'`, but REH_EMOTIONS is a const declared
later in the bundle's single shared scope — `typeof` on a const in its
temporal dead zone throws instead of returning "undefined", which aborted
top-level initialization for every module bundled after tts-preview.js.
The pickers now read window.REH_EMOTIONS on a deferred macrotask.
Also: emotion instructions are now always built in English (spoken text and
the native-accent clause stay in the book's language), which controlled A/B
testing showed produces a far cleaner prosodic gradient from Qwen3-TTS; and
Fish-Speech now receives temperature/top_p/repetition_penalty, which it was
the only backend never to have forwarded.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fish-Speech emotion tags were silently ignored on non-English books: per-line
emotions are LLM-generated in the book's own language, but Fish-Speech only
recognizes English [tag] markers, and a double-tagging bug was stacking a
broken server-derived tag on top of the client's own. Added a DE->EN
translation table and removed the double-tagging. Also wires the existing
book-profile context and race_species field into character portrait prompts
(previously only used for voice design), adds a recast-until-threshold loop
for casting, and adds backend-aware emotion quick-picks to Read Aloud, Try a
Voice, and Conversation.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
For a designed voice the instruct prompt is the voice's identity — the TTS
engine reproduces the voice from that text alone. The only copy saved was the
`note` display summary, clipped to 240 characters, which left 43 of 73 voices
cut off mid-sentence. Save the complete prompt in its own field so the engine
can register a voice from the whole description.
Existing voices keep working from the clipped copy (it still carries gender,
accent and timbre) and pick up the full text when next redesigned.
Pairs with the engine-side fix in tts-dgx-spark-faster-qwen3-tts.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
