Real failures found by re-inspecting a fresh re-ingest of c't KI Wissen
2026 after the previous figure-boundary-snapping release: a bar chart
was still truncated to roughly a third of its real height, and a
chart's top edge cut into an unrelated paragraph above it.
Root causes, each independently confirmed against the real page render
before fixing:
- A bar chart's bars/gridlines/axis labels rarely reach the "solid
ink" density a photo does (measured 0.25-0.5 vs the 0.55 threshold),
so the existing pass anchored only on the one fully-solid element
(the zero-line) and never grew further. Added a second, looser pass
that grows an anchor -- or the seed itself -- through any real mark
at all, fenced by the same blocked cells.
- Anchor selection ("which solid component does the seed overlap
most") was seed-size-dependent: a small seed touching only the
zero-line anchored there, but re-running the SAME function on the
box it had just produced -- now large enough to also touch a dense
axis-label strip -- picked that instead and shrank an already-good
box back down. Fixed by dilating the solid mask and labelling once,
BEFORE any seed is considered, so which components count as one
figure is a property of the page alone.
- A merged bounding box can enclose real text sitting between two
components even though neither component's own cells touched it (a
rectangle union, not the components' actual shape). Added
_trim_blocked_edges() to pull each edge back to the nearest text
wall found searching outward from the box's own centre.
- Two adjacent figures fencing each other's growth made each one's
result depend on the other's CURRENT box -- for two dark photos
sharing a caption strip, this never settled, swinging by over 100pt
call after call. snap_visual_bboxes() now re-seeds every visual on a
page from every other's last result for a bounded number of rounds,
deciding same-graphic ambiguity once up front (fenced against the
original seeds, before any iteration) so two disputed seeds can't
mutually fence each other into a tidy-looking split instead of being
recognised as the same disputed claim.
- A result is now only accepted once it reproduces itself when fed
back through the same rules (IoU-checked), and the minimum
seed/graphic overlap for repositioning a badly-offset box was
lowered slightly -- it was narrowly rejecting some real cases.
Verified against the full 204-page/147-figure document: idempotency
holds except for one confirmed non-convergent pair, which now lands on
the same deterministic answer every run instead of continuing to
drift.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Resumes an in-progress, uncommitted change from a previous session: the
live "page image + recognised text" preview shown during ingestion was
reworked to render formatted HTML (styled headings, a figure
placeholder) instead of a plain-text dump, but was left with a
duplication bug, dead code, inline styles, and no tests.
- Fixed duplication: a figure block's own transcribed text (usually its
on-screen UI copy) was rendered as an ordinary paragraph in addition
to the new figure placeholder box. Figure/chart/table/code blocks are
now represented only by the placeholder.
- Furniture/sidebar/pullquote/ad text is shown muted instead of being
dropped with no replacement, and is never promoted to a heading
regardless of font size.
- Factored the formatting logic out of the progress closure into a
standalone, unit-tested _render_ingest_preview_html().
- Replaced dead getattr(bbox, "y0", ...) calls (bbox is always a plain
[x, y, w, h] list) with direct indexing, moved statistics/logging
imports to module level, switched manual character replacement to
html.escape(), and bounded the preview by an accumulated line budget
instead of a blind slice that could truncate mid-tag.
- Replaced inline style="..." with CSS classes matching the panel's
existing dark console treatment.
- The frontend now always uses innerHTML (every text node is escaped
server-side), dropping the startsWith('<') content-type sniffing.
- Marked the preview panel aria-hidden -- decorative, and would
otherwise churn noisily for screen reader users on top of the
adjacent progress bar.
Verified against the real c't KI Wissen 2026 layout: a figure's raw
OCR text no longer appears twice, a real headline renders as <h3>, a
footer/pull-quote render muted, and all text is correctly escaped.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
No code change. Added to TODO.md:
- Magazine contents view (#s-magazine): logged three candidate layouts
as open options rather than picking one -- three-column page/text/
figures, synthesis-state-and-inline-play (the recommended quick win
under any layout), and a flipbook with content columns underneath.
Needs a decision before implementation.
- Browsing free PDF magazine sources: the two sites given as examples
(freemagazines.top, pdf-magazines.org) redistribute current commercial
titles without licence, so no scraper was built against them. Scoped
instead against OPDS catalogues, archive.org's public-domain
periodicals, publisher-run free issues, and open-access journals.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A vision OCR engine grounds a figure far less precisely than it
transcribes one. Over all 138 figure regions of a 204-page issue the
transcribed contents were right while the boxes were routinely offset,
undersized, or sitting on the neighbouring text column -- page 23's card
grid was reported at x 0.24-0.54 where the graphic runs 0.41-0.95.
snap_visual_bboxes() reduces the page render to an ink-coverage grid,
seeds a connected-component search with the engine's box, and re-joins
tiles a gutter apart so a card grid comes back as one figure. Text
blocks are excluded from the mask, which is what stops a figure
swallowing the column beside it. A candidate must be solidly inked or
framed on nearly all of its rows and columns (admitting a white-interior
screenshot), may not shrink below 30% of the seed, and reverts when two
figures land on the same graphic; anything else keeps the box it had.
Runs before the figure crop in both OCR paths, so the images shown
during playback are framed correctly too. Replaces refine_picture_
bounds(), which only recognised coloured panels and had no text fence.
Also adds "Fix figure boundaries" in review and POST
/api/magazine/{id}/figures/refine behind it, so an issue ingested before
this pass can be corrected without re-running OCR. Verified live against
c't KI Wissen 2026: 78 of 138 boxes corrected, the rest declined.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
