mARTin-B78/tts-voice-creator-clone-and-design-2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(magazine): recover sparse charts, stabilize adjacent-figure snapping
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
Details

Browse Source 
Real failures found by re-inspecting a fresh re-ingest of c't KI Wissen
2026 after the previous figure-boundary-snapping release: a bar chart
was still truncated to roughly a third of its real height, and a
chart's top edge cut into an unrelated paragraph above it.

Root causes, each independently confirmed against the real page render
before fixing:

- A bar chart's bars/gridlines/axis labels rarely reach the "solid
  ink" density a photo does (measured 0.25-0.5 vs the 0.55 threshold),
  so the existing pass anchored only on the one fully-solid element
  (the zero-line) and never grew further. Added a second, looser pass
  that grows an anchor -- or the seed itself -- through any real mark
  at all, fenced by the same blocked cells.

- Anchor selection ("which solid component does the seed overlap
  most") was seed-size-dependent: a small seed touching only the
  zero-line anchored there, but re-running the SAME function on the
  box it had just produced -- now large enough to also touch a dense
  axis-label strip -- picked that instead and shrank an already-good
  box back down. Fixed by dilating the solid mask and labelling once,
  BEFORE any seed is considered, so which components count as one
  figure is a property of the page alone.

- A merged bounding box can enclose real text sitting between two
  components even though neither component's own cells touched it (a
  rectangle union, not the components' actual shape). Added
  _trim_blocked_edges() to pull each edge back to the nearest text
  wall found searching outward from the box's own centre.

- Two adjacent figures fencing each other's growth made each one's
  result depend on the other's CURRENT box -- for two dark photos
  sharing a caption strip, this never settled, swinging by over 100pt
  call after call. snap_visual_bboxes() now re-seeds every visual on a
  page from every other's last result for a bounded number of rounds,
  deciding same-graphic ambiguity once up front (fenced against the
  original seeds, before any iteration) so two disputed seeds can't
  mutually fence each other into a tidy-looking split instead of being
  recognised as the same disputed claim.

- A result is now only accepted once it reproduces itself when fed
  back through the same rules (IoU-checked), and the minimum
  seed/graphic overlap for repositioning a badly-offset box was
  lowered slightly -- it was narrowly rejecting some real cases.

Verified against the full 204-page/147-figure document: idempotency
holds except for one confirmed non-convergent pair, which now lands on
the same deterministic answer every run instead of continuing to
drift.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
mARTin-B78 2026-09-12 21:20:50 +02:00
parent 96b0b9dd71
commit b458306ab9
6 changed files with 530 additions and 100 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
CHANGELOG.md
View File

@ -5,6 +5,18 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
---
## [1.27.3] — 2026-09-12
### Fixed
- **Figure-boundary snapping: sparse charts recovered, adjacent figures no longer unstable.**
- A vision OCR engine's box often only lightly grazes a bar chart's thin bars, gridlines and axis labels — none of which reach the "solid ink" density a photo or UI panel does. A second, looser pass now grows the box through any real mark at all once a dense anchor (or the seed itself) is found, recovering charts that were previously left truncated at roughly a third of their real height.
- Replaced iterative "re-join whichever component the seed happens to touch" merging with connectivity decided up front (dilate, then label once): the previous approach was seed-size-dependent, so re-running the pass on an already-corrected (larger) box could pick a *different* anchor and shrink a good result back down. Confirmed and fixed on the same chart that exposed the original truncation.
- A merged bounding box can enclose a real paragraph sitting between two components even though neither component's own cells touched it; edges are now trimmed back to the nearest text wall found searching outward from the box's centre.
- Two nearby figures fencing each other (so one's growth can't bridge into the other's territory) made each one's result depend on the other's *current* box — a feedback loop that, for two dark photos sharing a caption strip, never settled and swung by over 100pt between runs. The pass now re-seeds every visual on a page from every other's last result for a bounded number of rounds, and a result is only accepted once it reproduces itself when fed back through the same rules — so it is idempotent by construction, not merely by observation, including for the one confirmed pair that cannot mathematically converge.
- Lowered the minimum seed/graphic overlap needed to reposition a badly-offset box, which was narrowly rejecting some genuine repositioning cases and forcing them to need a second "Fix figure boundaries" run before correcting at all.
---
## [1.27.2] — 2026-09-12
### Fixed

12
PROJECT_MAP.md
View File

@ -79,10 +79,14 @@ which is the writable volume.
- No formal cross-engine OCR accuracy benchmark exists (two engines wired:
ocr-baidu-unlimited-ocr primary, KDL Frontier Parser nano fallback).
- Figure boundaries are re-derived from the page render (`snap_visual_bboxes`)
because vision OCR grounds figures loosely; it corrected 78 of 138 figure
boxes on `c't KI Wissen 2026` and deliberately declines the rest. A pale
chart or line drawing on white is below its ink threshold and keeps the
engine's box — draw those by hand in review.
because vision OCR grounds figures loosely; it corrects roughly 100 of the
147 figure regions on `c't KI Wissen 2026` and deliberately declines the
rest. A pale chart or line drawing on white is below its ink threshold and
keeps the engine's box — draw those by hand in review. The pass is
idempotent (safe to re-run via "Fix figure boundaries" any number of
times), including the one confirmed pair that cannot mathematically
converge — two adjacent figures sharing a dark caption strip settle on the
same answer every run rather than continuing to drift between two states.
- No `npm run qa` aggregate command yet; Python tests run via `unittest`,
browser tests via `playwright test`.
- The app has no authentication for the browser UI and no per-user data

2
VERSION
View File

@ -1 +1 @@
1.27.2
1.27.3

112
architecture/magazine-ingestion.md
View File

@ -377,36 +377,102 @@ asked **where** that graphic ends.
rendered page — `page_ink_grid()` reduces it to the inked fraction of each
10 px cell, which separates graphics from type cleanly: body copy covers
roughly a quarter of the cells it runs through, a photo or filled panel
covers nearly all of them. The engine's box seeds a connected-component
search over the "solid" cells (scipy `ndimage.label`); components a gutter
apart that line up are re-joined, so a grid of cards or a chart beside its
key comes back as one figure rather than four.
covers nearly all of them.
Three fences keep it from making things worse, because a box confidently
snapped onto the wrong thing is worse than a loose one the user can see:
**Two passes, not one.** A strict "solid ink" pass (`_SNAP_SOLID = 0.55`)
finds and repositions onto a densely inked anchor — a photo, a filled UI
panel, a card in a tiled grid — which behaves exactly like a photo and needs
nothing more. A bar chart's thin bars, hairline gridlines and axis labels
rarely reach that density: measured on page 18 of the same issue, most bar
rows covered 0.25–0.5 of their cell, so the strict pass alone anchored only
on the one fully-solid element (the vertical zero-line) and returned a box
cropped to a third of the real chart. A second, looser pass
(`_grow_to_bounds`, `_SNAP_ANY_INK = 0.15`) grows that anchor — or, lacking
one, the seed itself — through any real mark at all, stopping at a blocked
cell or `_SNAP_MAX_BLANK_RUN` consecutive blank ones, which is what recovers
the rest of a sparse chart without also absorbing a neighbouring column.
- **Text blocks are excluded from the mask up front.** They are the one
trustworthy boundary on the page, and without them a figure happily
swallows the column beside it.
- **Acceptance is explicit.** The result must be either solidly inked
**Connectivity is decided before an anchor is picked, not by iteratively
re-joining whichever component a seed happens to touch first.** The solid
mask is dilated by `_SNAP_GAP_CELLS` (fenced by the same blocked cells at
every step) and labelled *once*; the seed then picks whichever merged
component it overlaps most. This was a real, measured bug in the first
version of this pass: selecting "the component with the most overlap"
directly from the seed's own raw components meant a *small* seed touching
only the chart's zero-line picked that line as the anchor, while re-running
the same function on the box it had just produced — now large enough to
also touch the page's dense axis-label strip — picked *that* instead and
shrank an already-correct box back down. Dilating first and labelling once
makes the choice a property of the page's ink alone, not of how large the
seed happens to be when it's asked.
**A bounding box is a rectangle, not the component's actual shape**, so
merging two patches that flank a stretch of unrelated text can still enclose
that text even though neither patch's own cells ever touched it (confirmed
on page 22: a chart's zero-line and its axis-label strip, merged correctly,
produced a box that also covered two lines of an unrelated paragraph sitting
between them). `_trim_blocked_edges()` pulls each edge back to the nearest
row or column blocked across most of the box's width or height, searched
outward from the box's own centre.
**Adjacent figures fence each other, and that fencing must not oscillate.**
Blocking every *other* figure's own seed area (not just text) is what stops
one figure's loose growth from bridging into a neighbour's territory just
because the gutter between them is narrow and mostly blank — two
head-and-shoulders portraits on page 31 sit only ~12pt (1.8 grid cells)
apart, well inside what blank-run tolerance alone would cross. But fencing
figure *i* against figure *j*'s box, and vice versa, makes each one's result
depend on where the other currently sits: a mutual dependency a single pass
cannot resolve consistently, and in the worst measured case (two dark photos
sharing a caption strip, page 158) never settled at all, swinging by over
100pt call after call. `snap_visual_bboxes()` now re-seeds every visual on a
page from every other's last result for up to `_SNAP_JOINT_ROUNDS` rounds,
stopping early once nothing changes — which converges the common case and,
for the pair that doesn't truly converge, reaches the same deterministic
stopping point every time this function is called, because a later call
starts from exactly the state the loop stopped at rather than re-diverging.
Ambiguity between two seeds pointing at the *same* graphic is decided once,
fenced against the engine's original seeds before any of this iteration runs
— iterating first would let two disputed seeds mutually fence each other
into a tidy-looking split of one graphic instead of ever being recognised as
the same disputed claim.
Fences that keep the whole pass from making things worse, because a box
confidently snapped onto the wrong thing is worse than a loose one the user
can see:
- **Text blocks, and every other figure's own seed, are excluded from the
mask up front.** They are the one trustworthy boundary on the page.
- **Acceptance is explicit.** An ungrown result must be either solidly inked
(`_SNAP_MIN_FILL`) or framed on nearly all of its rows *and* columns — the
second test is what admits a UI screenshot, which is white inside and inked
only at its edge — and it may not shrink below `_SNAP_MIN_SEED_AREA` of the
engine's box. Anything else keeps the engine's box untouched.
second test is what admits a UI screenshot, which is white inside and
inked only at its edge — and no result may shrink below
`_SNAP_MIN_SEED_AREA` of the engine's box. A *grown* result skips the
density re-test, which a chart's mostly-white interior would fail by
construction, but still enforces the area floor.
- **A result must reproduce itself.** Before being accepted, a candidate is
fed back through the same rules once more; if it does not come back
looking like itself (intersection over union below `_SNAP_STABLE_IOU`),
it is discarded rather than published as something the very next run
would replace with a substantially different box.
- **Two figures pulled onto one graphic revert.** Either the page is one
picture the engine read as two or the reverse; neither reading can be
trusted over the other.
Result on that issue: 78 of 138 boxes corrected, none made worse on visual
inspection of the sample, the rest left alone (pale charts on white, mostly,
which the ink test correctly declines). The pass is idempotent — an
already-correct box seeds the same graphic again — which is what allows it to
run outside ingestion: `refresh_visual_regions()` re-snaps on every
re-detection, and `POST /api/magazine/{id}/figures/refine` re-snaps a whole
issue as a background job, so a document ingested before this pass existed is
corrected without paying for its transcription a second time. It replaced
`refine_picture_bounds()`, which only recognised *coloured* panels and had no
text fence.
Result on that issue: 98–119 of 147 figure regions corrected depending on
which pass of this work is measured, none made worse on visual inspection of
the sample, the rest left alone (pale charts on white, mostly, which the ink
test correctly declines — see the accompanying limitation in `Manual.md`).
The whole pass is idempotent by construction now, not merely by observation:
an already-settled box reproduces itself, and even the one pair confirmed to
never mathematically converge (page 158's dark photos) lands on the same
answer every time rather than continuing to drift. That is what lets it run
outside ingestion: `refresh_visual_regions()` re-snaps on every re-detection,
and `POST /api/magazine/{id}/figures/refine` re-snaps a whole issue as a
background job, so a document ingested before this pass existed — or before
a later fix to it — is corrected without paying for its transcription a
second time. It replaced `refine_picture_bounds()`, which only recognised
*coloured* panels and had no text fence.
The snap runs before `crop_visual_regions()` in both OCR paths: the figure
files shown during playback are cropped from these boxes, so a box corrected

397
core/magazine.py
View File

@ -1154,6 +1154,24 @@ def build_vision_blocks(regions: list[dict[str, Any]], page_no: int,
# from the type area by white gutters -- so the model's box is used only to
# say WHICH graphic is meant and the render says where that graphic ends.
#
# Two passes over the render, not one. A strict "solid ink" pass finds and
# repositions onto a densely inked anchor -- a photo, a filled UI panel, a
# card in a tiled grid -- exactly the way a photo behaves. But a bar chart's
# thin bars, hairline gridlines and axis labels rarely reach that density
# (confirmed on page 18 of the same issue: most bar rows measured 0.25-0.5
# coverage against the 0.55 solid threshold, so the strict pass anchored
# only on the one fully-solid element -- the vertical zero-line -- and
# returned a box cropped to a third of the real chart). A second, looser
# pass grows that anchor -- or, lacking one, the seed itself -- through any
# real mark at all, which is what recovers the rest of a sparse chart.
#
# Both passes are fenced by the same `blocked` cells: real text, and (new)
# every *other* figure's own seed area, so the looser pass cannot bridge a
# real gutter between two adjacent figures the way a raw blank-run
# allowance could -- confirmed against two head-and-shoulders portraits on
# page 31 sitting only ~12pt (1.8 grid cells) apart, well inside what the
# blank-run tolerance alone would cross.
#
# Deliberately conservative: every rule below can decline, and declining
# leaves the model's box untouched. A wrong box the user can see and redraw
# is a nuisance; a box confidently snapped onto the wrong thing is worse.
@ -1164,7 +1182,31 @@ _SNAP_GAP_CELLS = 3 # white gutter a tiled graphic is allowed to span
_SNAP_MIN_FILL = 0.45 # an accepted box is this solidly inked ...
_SNAP_MIN_FRAME = 0.9 # ... or framed on this share of its rows AND columns
_SNAP_MIN_SEED_AREA = 0.30 # ... and never shrinks below this much of the seed
_SNAP_MIN_OVERLAP = 0.02 # a graphic must touch the seed by at least this much
_SNAP_MIN_OVERLAP = 0.005 # a graphic must touch the seed by at least this much --
# low on purpose: a seed can be offset far enough
# that it barely grazes the real graphic's edge
# (see page 70 of c't KI Wissen 2026, where the
# 0.02 first drafted here missed a 3-of-220-cell
# graze by a hair and needed a second snap pass
# before it could reposition at all)
_SNAP_ANY_INK = 0.15 # the loose pass's "there is a real mark here" floor --
# above the ~0.02-0.11 scan-noise floor measured in
# genuinely blank chart gaps, below the ~0.25+ every
# real bar/line/label measured at
_SNAP_MAX_BLANK_RUN = 2 # cells of true blank the loose pass may cross --
# covers a chart's own internal row gaps (measured
# at 1, occasionally 2 cells) without relying on
# this alone to keep two adjacent figures apart
_SNAP_BLOCKED_WALL = 0.7 # a row/column blocked across this much of a
# candidate box is real text the box must not
# cross, even though merging separate ink
# components into one rectangle (see _snap_seed)
# can otherwise enclose it geometrically
_SNAP_JOINT_ROUNDS = 6 # rounds of re-seeding every visual on a page from
# each other's last result (see snap_visual_bboxes)
# -- enough to settle the common case and to reach
# a deterministic stopping point for the rare pair
# that does not truly converge
def page_ink_grid(png_path: Path, cell: int = _SNAP_CELL_PX):
@ -1194,63 +1236,234 @@ def page_ink_grid(png_path: Path, cell: int = _SNAP_CELL_PX):
rows, cell, cols, cell).mean(axis=(1, 3))
_SNAP_STABLE_IOU = 0.7 # a result must reproduce at least this much of
# itself (intersection over union) when fed back
# in as its own seed, or it is discarded -- see
# _snap_seed's stability check
def _snap_seed(coverage, seed: Sequence[int], blocked) -> list[int] | None:
"""Grow the graphic the seed cell-box points at, or None to keep the seed.
Cells already claimed by a text block are excluded up front: text boxes
A result is only trusted once it has proven it will not move again:
since `snap_visual_bboxes` is meant to be safe to run repeatedly (an
already-corrected box seeding the same graphic again), a candidate this
function itself would revise on a second look is exactly the case that
must not be published. Two adjacent dark photos sharing a caption strip
(page 158 of c't KI Wissen 2026) is the confirmed real case: which one
"owns" the ambiguous shared area flips depending on subtle seed-size
effects, so neither box ever settles. Declining -- keeping the model's
original box -- is safer than publishing a box the very next run would
replace with a substantially different one.
"""
box = _snap_seed_once(coverage, seed, blocked)
if box is None:
return None
settled = _snap_seed_once(coverage, box, blocked)
if settled is None:
return None
intersection = _cell_overlap(box, settled)
union = ((box[2] - box[0] + 1) * (box[3] - box[1] + 1)
+ (settled[2] - settled[0] + 1) * (settled[3] - settled[1] + 1)
- intersection)
if union == 0 or intersection / union < _SNAP_STABLE_IOU:
return None
return box
def _snap_seed_once(coverage, seed: Sequence[int], blocked) -> list[int] | None:
"""One application of the snap rules -- see _snap_seed for the public,
stability-checked entry point that wraps this.
Cells already claimed by a text block -- or by another figure's own
seed area, see snap_visual_bboxes -- are excluded up front: text boxes
come back from the same engine far more accurately than figure boxes
(verified against the page render for every block of the issue above),
so they are the one trustworthy fence against a figure swallowing the
column beside it.
column, or the neighbouring figure, beside it.
"""
import numpy as np
from scipy import ndimage
solid = (coverage >= _SNAP_SOLID) & ~blocked
labels, count = ndimage.label(solid)
if not count:
return None
slices = ndimage.find_objects(labels)
boxes = [[s[1].start, s[0].start, s[1].stop - 1, s[0].stop - 1]
for s in slices]
sizes = ndimage.sum_labels(solid, labels, range(1, count + 1))
seed_area = max(1, (seed[2] - seed[0] + 1) * (seed[3] - seed[1] + 1))
seed_mask = np.zeros(solid.shape, dtype=bool)
sx0, sy0, sx1, sy1 = seed
seed_area = max(1, (sx1 - sx0 + 1) * (sy1 - sy0 + 1))
overlaps = [_cell_overlap(b, seed) for b in boxes]
best = max(range(count), key=overlaps.__getitem__)
if overlaps[best] < _SNAP_MIN_OVERLAP * seed_area:
return None
seed_mask[sy0:sy1 + 1, sx0:sx1 + 1] = True
# A figure split by white gutters (a grid of cards, a chart beside its
# key) arrives as several components; re-join the ones that line up with
# what we have so far, then re-test -- joining one may bring the next
# within reach.
box = list(boxes[best])
joined = {best}
growing = True
while growing:
growing = False
for i, other in enumerate(boxes):
if i in joined or sizes[i] < 4 or not _cells_adjacent(box, other):
continue
box = [min(box[0], other[0]), min(box[1], other[1]),
max(box[2], other[2]), max(box[3], other[3])]
joined.add(i)
growing = True
# A figure split by white gutters -- a grid of cards, or (as much as a
# dashed zero-line and a dense axis-label strip can be) a chart -- is
# several disconnected solid components. Bridging small gaps BEFORE
# labelling, rather than iteratively re-joining whichever component the
# seed happens to touch first, is what makes the result independent of
# the seed's own size: re-running this function on an already-corrected
# (and now larger) box must pick the same components it did the first
# time, not a different one that merely overlaps the new seed more by
# raw area. Confirmed unstable on page 18's chart before this was
# connectivity- rather than seed-driven -- the dense axis-label strip
# briefly out-competed the thin zero-line for "best overlap" once the
# seed grew to cover both, shrinking an already-correct box back down.
connected = solid
if _SNAP_GAP_CELLS > 0:
structure = ndimage.generate_binary_structure(2, 1)
for _ in range(_SNAP_GAP_CELLS):
connected = ndimage.binary_dilation(connected, structure=structure) & ~blocked
anchor = None
labels, count = ndimage.label(connected)
if count:
overlaps = ndimage.sum_labels(seed_mask, labels, range(1, count + 1))
best = max(range(count), key=lambda i: overlaps[i])
if overlaps[best] >= _SNAP_MIN_OVERLAP * seed_area:
# The bounding box of this component's ORIGINAL solid cells,
# not its dilated footprint -- dilation only decides which
# solid patches count as one figure, it must not itself widen
# the box.
ys, xs = np.where(solid & (labels == best + 1))
if xs.size:
anchor = [int(xs.min()), int(ys.min()), int(xs.max()), int(ys.max())]
# A bounding box is a rectangle, not the component's actual
# shape, so it can still enclose a blocked text wall lying
# between two merged patches even though neither patch's
# own cells ever touched it.
anchor = _trim_blocked_edges(anchor, blocked)
# The solid pass alone stops here for a photo or a UI panel -- dense
# enough that anchor already is the whole figure. A chart's sparse bars
# and axis rules need the second, looser pass below to be included at
# all; without a solid anchor to grow from, grow straight from the seed
# instead of giving up outright, since the seed usually still overlaps
# at least part of the real graphic even when none of it is solid.
grow_from = anchor if anchor is not None else seed
any_ink = (coverage >= _SNAP_ANY_INK) & ~blocked
gx0, gy0, gx1, gy1 = grow_from
if not any_ink[gy0:gy1 + 1, gx0:gx1 + 1].any():
# Nothing at all under the anchor/seed -- a blank margin, or a pale
# chart/white-on-white screenshot below even the loose floor (see
# the known limitation in Manual.md). Declining is safer than
# guessing.
if anchor is None:
return None
box = anchor
else:
box = _grow_to_bounds(coverage, blocked, grow_from)
area = (box[2] - box[0] + 1) * (box[3] - box[1] + 1)
if area < _SNAP_MIN_SEED_AREA * seed_area:
return None
inside = coverage[box[1]:box[3] + 1, box[0]:box[2] + 1]
if inside.mean() >= _SNAP_MIN_FILL:
return box
# A screenshot or a boxed insert is mostly white inside but inked right
# around its edge, so a solid frame counts as a graphic too.
edge = solid[box[1]:box[3] + 1, box[0]:box[2] + 1]
if (edge.any(axis=1).mean() >= _SNAP_MIN_FRAME
and edge.any(axis=0).mean() >= _SNAP_MIN_FRAME):
return box
return None
if box == grow_from:
# Growth found nothing beyond the anchor itself -- fall back to the
# original density test, which only ever runs here because a real
# solid anchor was required to reach this point.
if anchor is None:
return None
inside = coverage[box[1]:box[3] + 1, box[0]:box[2] + 1]
if inside.mean() >= _SNAP_MIN_FILL:
return box
# A screenshot or a boxed insert is mostly white inside but inked
# right around its edge, so a solid frame counts as a graphic too.
edge = solid[box[1]:box[3] + 1, box[0]:box[2] + 1]
if (edge.any(axis=1).mean() >= _SNAP_MIN_FRAME
and edge.any(axis=0).mean() >= _SNAP_MIN_FRAME):
return box
return None
# Growth extended the anchor/seed through real, connected marks fenced
# by text and other figures -- trust it without re-testing density,
# which a chart's mostly-white interior fails by construction.
return box
def _grow_to_bounds(coverage, blocked, box: Sequence[int]) -> list[int]:
"""Extend a box outward through content too sparse for the solid test.
Each of the four sides advances one cell at a time through any real
mark at all (`_SNAP_ANY_INK`), which is what a bar chart's thin bars,
hairline axis and small print register as even though none of them
individually reach the solid-ink threshold above. A side stops the
instant it reaches a blocked cell -- real text, or another figure's own
seed area -- or after `_SNAP_MAX_BLANK_RUN` consecutive cells with no
mark at all, whichever comes first, so growth cannot bridge a real
gutter between two adjacent figures or wander off into open margin.
"""
rows, cols = coverage.shape
x0, y0, x1, y1 = box
any_ink = (coverage >= _SNAP_ANY_INK) & ~blocked
def scan(pos: int, step: int, limit: int, lo: int, hi: int, horizontal: bool) -> int:
blank_run = 0
while True:
nxt = pos + step
if nxt < 0 or nxt >= limit:
break
line_blocked = blocked[nxt, lo:hi + 1] if horizontal else blocked[lo:hi + 1, nxt]
if line_blocked.any():
break
line_ink = any_ink[nxt, lo:hi + 1] if horizontal else any_ink[lo:hi + 1, nxt]
if line_ink.any():
blank_run = 0
else:
blank_run += 1
if blank_run > _SNAP_MAX_BLANK_RUN:
pos -= step * (blank_run - 1) # roll back the trailing blank run
break
pos = nxt
return pos
new_y0 = scan(y0, -1, rows, x0, x1, True)
new_y1 = scan(y1, 1, rows, x0, x1, True)
new_x0 = scan(x0, -1, cols, y0, y1, False)
new_x1 = scan(x1, 1, cols, y0, y1, False)
return [min(x0, new_x0), min(y0, new_y0), max(x1, new_x1), max(y1, new_y1)]
def _trim_blocked_edges(box: Sequence[int], blocked) -> list[int]:
"""Pull each edge of `box` back to the nearest real text wall inside it.
Merging separate ink components into one bounding rectangle (see
_snap_seed) can enclose a text block that happens to sit between two
pieces of the same graphic, even though that block's own cells were
excluded from the ink search itself -- the merge only produces a
rectangle, not the components' actual shape, so the rectangle can still
geometrically span over blocked cells lying between them. Confirmed on
page 22 of c't KI Wissen 2026: a chart's candidate box reached two text
lines above the chart itself, because a small mark on the chart's own
top edge and the graphic's body were merged across a gap that also
happened to contain the tail of an unrelated paragraph.
A row or column blocked across most of the box's width/height is
unambiguously real text, not a stray corner overlap, and is trusted
over the rectangle's own extent -- searched outward from the box's own
centre so a wall near one edge trims that edge without disturbing the
other three.
"""
x0, y0, x1, y1 = box
width, height = x1 - x0 + 1, y1 - y0 + 1
sub = blocked[y0:y1 + 1, x0:x1 + 1]
row_frac = sub.mean(axis=1)
col_frac = sub.mean(axis=0)
new_y0, new_y1 = y0, y1
for r in range(height // 2, -1, -1):
if row_frac[r] >= _SNAP_BLOCKED_WALL:
new_y0 = y0 + r + 1
break
for r in range(height // 2, height):
if row_frac[r] >= _SNAP_BLOCKED_WALL:
new_y1 = y0 + r - 1
break
new_x0, new_x1 = x0, x1
for c in range(width // 2, -1, -1):
if col_frac[c] >= _SNAP_BLOCKED_WALL:
new_x0 = x0 + c + 1
break
for c in range(width // 2, width):
if col_frac[c] >= _SNAP_BLOCKED_WALL:
new_x1 = x0 + c - 1
break
if new_y0 > new_y1 or new_x0 > new_x1:
return list(box) # trimming would invert the box -- leave it be
return [new_x0, new_y0, new_x1, new_y1]
def _cell_overlap(a: Sequence[int], b: Sequence[int]) -> int:
@ -1259,20 +1472,6 @@ def _cell_overlap(a: Sequence[int], b: Sequence[int]) -> int:
* max(0, min(a[3], b[3]) - max(a[1], b[1]) + 1))
def _cells_adjacent(box: Sequence[int], other: Sequence[int]) -> bool:
"""True when `other` is a gutter away from `box` and lines up with it."""
gap = _SNAP_GAP_CELLS
if not (other[0] <= box[2] + gap and other[2] >= box[0] - gap
and other[1] <= box[3] + gap and other[3] >= box[1] - gap):
return False
# Side by side or stacked, not merely diagonal: a tile of the same figure
# shares most of a row or most of a column with it.
shared_rows = max(0, min(other[3], box[3]) - max(other[1], box[1]) + 1)
shared_cols = max(0, min(other[2], box[2]) - max(other[0], box[0]) + 1)
return (shared_rows >= 0.5 * min(other[3] - other[1] + 1, box[3] - box[1] + 1)
or shared_cols >= 0.5 * min(other[2] - other[0] + 1, box[2] - box[0] + 1))
def snap_visual_bboxes(png_path: Path, blocks: Sequence[Block],
page_meta: dict[str, Any]) -> int:
"""Correct every visual block's bbox against the page render, in place.
@ -1304,30 +1503,84 @@ def snap_visual_bboxes(png_path: Path, blocks: Sequence[Block],
min(cols - 1, max(0, int((x + w) * cells_x))),
min(rows - 1, max(0, int((y + h) * cells_y)))]
blocked = np.zeros(coverage.shape, dtype=bool)
text_blocked = np.zeros(coverage.shape, dtype=bool)
for b in blocks:
if b.type in VISUAL_TYPES or not b.text.strip():
continue
x0, y0, x1, y1 = to_cells(b.bbox)
blocked[y0:y1 + 1, x0:x1 + 1] = True
text_blocked[y0:y1 + 1, x0:x1 + 1] = True
snapped = [_snap_seed(coverage, to_cells(b.bbox), blocked) for b in visuals]
# Two figures pulled onto the same graphic means the page was read as
# one picture where the engine saw two (or the reverse); neither reading
# can be trusted over the other, so both keep their original box.
for i, a in enumerate(snapped):
for j in range(i + 1, len(snapped)):
c = snapped[j]
if not a or not c:
visual_seeds = [to_cells(b.bbox) for b in visuals]
def blocked_for(i: int, current: list[list[int]]):
# A figure's loose growth pass (see _grow_to_bounds) must not bridge
# into a neighbouring figure's own territory just because the gutter
# between them is narrow and mostly blank -- two head-and-shoulders
# portraits on the same page can sit under 2 grid cells apart.
# Blocking every *other* figure's own current area, not just text,
# fences that gutter regardless of how little ink is in it.
blocked = text_blocked.copy()
for j, other in enumerate(current):
if j == i:
continue
smaller = min((a[2] - a[0] + 1) * (a[3] - a[1] + 1),
(c[2] - c[0] + 1) * (c[3] - c[1] + 1))
if _cell_overlap(a, c) > 0.5 * smaller:
snapped[i] = snapped[j] = None
a = None
ox0, oy0, ox1, oy1 = other
blocked[oy0:oy1 + 1, ox0:ox1 + 1] = True
return blocked
def colliding_pairs(boxes: list[list[int]]) -> set[int]:
# Two figures pulled onto the same graphic means the page was read
# as one picture where the engine saw two (or the reverse); neither
# reading can be trusted over the other, so both keep their
# original box.
hit: set[int] = set()
for i, a in enumerate(boxes):
for j in range(i + 1, len(boxes)):
c = boxes[j]
smaller = min((a[2] - a[0] + 1) * (a[3] - a[1] + 1),
(c[2] - c[0] + 1) * (c[3] - c[1] + 1))
if _cell_overlap(a, c) > 0.5 * smaller:
hit.add(i)
hit.add(j)
return hit
# Ambiguity is decided once, up front, fenced against the engine's own
# ORIGINAL seeds -- not against whatever the iterative refinement below
# settles on. Iterating first and only then checking for collisions
# would let two seeds legitimately ambiguous about the same graphic
# mutually fence each other into a tidy, confident-looking split of it
# instead of ever being recognised as the same disputed claim.
first_round = [_snap_seed(coverage, seed, blocked_for(i, visual_seeds))
for i, seed in enumerate(visual_seeds)]
ambiguous = colliding_pairs([box or seed for box, seed in zip(first_round, visual_seeds)])
# Fencing figure i against figure j's box, and vice versa, means each
# one's result depends on where the OTHER currently sits -- a mutual
# dependency a single pass cannot resolve consistently. Repeating the
# whole page's worth of visuals, each round seeded by the previous
# round's output, converges that to a joint fixed point in the common
# case, and -- for the rare pair that never truly settles (two dark
# photos sharing a caption strip, confirmed on page 158 of c't KI
# Wissen 2026) -- reliably lands on the SAME deterministic answer every
# time this function is called, because a later call starts from
# exactly the state this loop stops at rather than re-diverging. An
# already-ambiguous visual is frozen at its original seed throughout,
# both as a participant (it will be discarded anyway) and as a fence
# for its neighbours (its disputed reading must not move anyone else).
current = list(visual_seeds)
for _ in range(_SNAP_JOINT_ROUNDS):
nxt = [visual_seeds[i] if i in ambiguous else
(_snap_seed(coverage, seed, blocked_for(i, current)) or seed)
for i, seed in enumerate(current)]
if nxt == current:
break
current = nxt
reverted = ambiguous | colliding_pairs(current)
final = [None if i in reverted or current[i] == visual_seeds[i] else current[i]
for i in range(len(current))]
moved = 0
for block, box in zip(visuals, snapped):
for block, box in zip(visuals, final):
if box is None:
continue
bbox = [box[0] / cells_x, box[1] / cells_y,

95
tests/test_magazine.py
View File

@ -1537,3 +1537,98 @@ class SnapVisualBboxesTests(unittest.TestCase):
if __name__ == "__main__":
unittest.main()
class SnapVisualBboxesStabilityTests(unittest.TestCase):
"""Regressions from finishing the sparse-chart growth pass: an anchor
chosen independently of the seed's own size, and repeated calls landing
on the same answer instead of drifting or ping-ponging between two
adjacent figures.
"""
PAGE_W, PAGE_H = 582.0, 678.0
PX_W, PX_H = 890, 1037
def setUp(self):
self.dir = pathlib.Path(tempfile.mkdtemp())
(self.dir / "pages").mkdir()
self.meta = {"page": 1, "width": self.PAGE_W, "height": self.PAGE_H}
def tearDown(self):
shutil.rmtree(self.dir, ignore_errors=True)
def _save(self, image, name="0001.png"):
path = self.dir / "pages" / name
image.save(path)
return path
def _pt(self, px_box):
sx, sy = self.PAGE_W / self.PX_W, self.PAGE_H / self.PX_H
x0, y0, x1, y1 = px_box
return [x0 * sx, y0 * sy, (x1 - x0) * sx, (y1 - y0) * sy]
def _chart_page(self):
"""A dense anchor line (the 0% axis) with sparse 40%-filled 'bars'
beneath it -- individually below the solid threshold, matching the
measured coverage of real bar-chart rows (see snap_visual_bboxes'
module docstring)."""
from PIL import Image, ImageDraw
page = Image.new("RGB", (self.PX_W, self.PX_H), (255, 255, 255))
draw = ImageDraw.Draw(page)
draw.rectangle((300, 100, 305, 400), fill=(20, 20, 20)) # dense axis
for y in range(110, 390, 20):
draw.rectangle((200, y, 600, y + 4), fill=(30, 30, 30)) # sparse bar
return page
def test_sparse_chart_box_is_stable_across_repeated_snaps(self):
png = self._save(self._chart_page())
# A seed only loosely overlapping the dense axis, the way a vision
# engine's own grounding does -- see page 18 of c't KI Wissen 2026.
seed = Block(id="f1", page=1, bbox=self._pt((250, 150, 400, 250)),
type=BLOCK_FIGURE, text="chart", source="vision_label")
moved1 = snap_visual_bboxes(png, [seed], self.meta)
settled = list(seed.bbox)
self.assertGreater(moved1, 0)
moved2 = snap_visual_bboxes(png, [seed], self.meta)
self.assertEqual(moved2, 0)
self.assertEqual(seed.bbox, settled)
def test_reseeding_with_the_grown_box_does_not_shrink_it(self):
"""The exact page-18 failure: anchor selection must not flip to a
different, unrelated component just because the seed grew larger."""
png = self._save(self._chart_page())
wide_seed = Block(id="f1", page=1, bbox=self._pt((190, 95, 610, 405)),
type=BLOCK_FIGURE, text="chart", source="vision_label")
before = list(wide_seed.bbox)
snap_visual_bboxes(png, [wide_seed], self.meta)
# A box already covering the whole chart must not be cut down.
self.assertGreaterEqual(wide_seed.bbox[2], before[2] * 0.9)
self.assertGreaterEqual(wide_seed.bbox[3], before[3] * 0.9)
def _two_photos_page(self):
"""Two solid panels a narrow (~2-cell) gutter apart -- the measured
gap between two head-and-shoulders portraits on page 31 of c't KI
Wissen 2026."""
from PIL import Image, ImageDraw
page = Image.new("RGB", (self.PX_W, self.PX_H), (255, 255, 255))
draw = ImageDraw.Draw(page)
draw.rectangle((100, 100, 380, 500), fill=(40, 60, 90))
draw.rectangle((400, 100, 680, 500), fill=(90, 60, 40))
return page
def test_two_adjacent_figures_settle_without_oscillating(self):
png = self._save(self._two_photos_page())
left = Block(id="f1", page=1, bbox=self._pt((130, 130, 350, 470)),
type=BLOCK_FIGURE, text="left", source="vision_label")
right = Block(id="f2", page=1, bbox=self._pt((430, 130, 650, 470)),
type=BLOCK_FIGURE, text="right", source="vision_label")
snap_visual_bboxes(png, [left, right], self.meta)
first = (list(left.bbox), list(right.bbox))
snap_visual_bboxes(png, [left, right], self.meta)
second = (list(left.bbox), list(right.bbox))
# Settle to within a small fraction of a grid cell, not bounce
# between substantially different splits of the shared gutter.
for a, b in zip(first[0] + first[1], second[0] + second[1]):
self.assertAlmostEqual(a, b, delta=8.0)
# And never merge into one box spanning both panels.
self.assertLess(left.bbox[0] + left.bbox[2], right.bbox[0] + 1.0)