fix(magazine): derive figure boundaries from the page render
Some checks are pending
Test and SonarQube / quality (push) Waiting to run

A vision OCR engine grounds a figure far less precisely than it
transcribes one. Over all 138 figure regions of a 204-page issue the
transcribed contents were right while the boxes were routinely offset,
undersized, or sitting on the neighbouring text column -- page 23's card
grid was reported at x 0.24-0.54 where the graphic runs 0.41-0.95.

snap_visual_bboxes() reduces the page render to an ink-coverage grid,
seeds a connected-component search with the engine's box, and re-joins
tiles a gutter apart so a card grid comes back as one figure. Text
blocks are excluded from the mask, which is what stops a figure
swallowing the column beside it. A candidate must be solidly inked or
framed on nearly all of its rows and columns (admitting a white-interior
screenshot), may not shrink below 30% of the seed, and reverts when two
figures land on the same graphic; anything else keeps the box it had.

Runs before the figure crop in both OCR paths, so the images shown
during playback are framed correctly too. Replaces refine_picture_
bounds(), which only recognised coloured panels and had no text fence.

Also adds "Fix figure boundaries" in review and POST
/api/magazine/{id}/figures/refine behind it, so an issue ingested before
this pass can be corrected without re-running OCR. Verified live against
c't KI Wissen 2026: 78 of 138 boxes corrected, the rest declined.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
mARTin-B78 2026-09-12 10:17:00 +02:00
parent a4b7c18a2f
commit 55f147d7a7
10 changed files with 550 additions and 46 deletions

View File

@ -5,6 +5,23 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
---
## [1.27.0] — 2026-09-12
### Fixed
- **Figure boundaries are now derived from the scanned page, not from the OCR engine's guess.**
- A vision OCR engine locates a photo far less precisely than it reads type: over all 138 figure regions of a 204-page issue, figure boxes were routinely offset, undersized, or sitting on the neighbouring text column while the transcribed text of the same regions was correct.
- New `snap_visual_bboxes()` pass reduces the page render to an ink-coverage grid, seeds a connected-component search with the engine's box, and re-joins tiles a gutter apart — so a grid of cards or a chart beside its key comes back as one figure. Text blocks are excluded from the mask, which stops a figure swallowing the column beside it.
- Conservative by design: a candidate must be solidly inked *or* framed on nearly all of its rows and columns (which is what admits a white-interior UI screenshot), may not shrink below 30% of the engine's box, and reverts when two figures are pulled onto the same graphic. Anything it cannot place confidently keeps the box it had.
- Runs before the figure crop in both OCR paths, so the images shown during playback are framed correctly too. Corrected 78 of 138 figure boxes on the reference issue.
- Replaces `refine_picture_bounds()`, which only recognised coloured panels and had no text fence.
### Added
- **"Fix figure boundaries" in magazine review**, and `POST /api/magazine/{id}/figures/refine` behind it.
- Re-snaps every figure box in an issue as a restart-safe background job. No OCR engine is called, so an issue ingested before this release is corrected without paying for its transcription again.
- Text, article grouping and existing approvals are untouched — safe to run halfway through a review. Reports how many boundaries moved.
---
## [1.26.14] — 2026-09-10
### Added

View File

@ -47,7 +47,14 @@ turned into a podcast discussion between two or three voices you pick.
mistaken for unclaimed text. If OCR quality is poor, **"OCR again"**
re-reads just the current page through the configured vision engine
(or **"OCR again (whole magazine)"** from the contents screen for every
page, which also resets any articles already reviewed). If the layout
page, which also resets any articles already reviewed). If a figure's
box is drawn in the wrong place or at the wrong size — a common miss,
because an OCR engine locates a photo far less precisely than it reads
type — **"Fix figure boundaries"** re-derives every figure box in the
issue from the scanned pages themselves. It calls no OCR engine, changes
no text and touches no approval, so it is safe to run at any point,
including on an issue you are halfway through reviewing; boxes it cannot
place confidently are left exactly as they were. If the layout
engine missed a region entirely, **"Draw it"** lets you click-drag a box
directly on the page image and add it as text (typed or OCRed) or as a
new figure. Confirm the boundaries, correct the title, then **Approve**.
@ -99,6 +106,8 @@ unreviewed boundaries may include a neighbouring article or an advert.
| "No LLM model configured" | `llm_model` is empty. | Set it in Settings. |
| "LLM did not return usable JSON" | The model ignored the format. | Retry, or pick a stronger model. |
| "Article has too little text" | Under 200 characters — usually a stub. | Check the boundaries in review. |
| A figure box sits over the wrong part of the page | The OCR engine grounded the figure loosely. | Press **"Fix figure boundaries"**; if the graphic is too pale to detect, draw the region by hand. |
| "Fix figure boundaries" reports nothing corrected | Every box already matched its page, or none could be placed with confidence. | Nothing is wrong; correct any remaining box by drawing it. |
### Privacy

View File

@ -78,6 +78,11 @@ which is the writable volume.
real measured audio duration, not true forced alignment.
- No formal cross-engine OCR accuracy benchmark exists (two engines wired:
ocr-baidu-unlimited-ocr primary, KDL Frontier Parser nano fallback).
- Figure boundaries are re-derived from the page render (`snap_visual_bboxes`)
because vision OCR grounds figures loosely; it corrected 78 of 138 figure
boxes on `c't KI Wissen 2026` and deliberately declines the rest. A pale
chart or line drawing on white is below its ink threshold and keeps the
engine's box — draw those by hand in review.
- No `npm run qa` aggregate command yet; Python tests run via `unittest`,
browser tests via `playwright test`.
- The app has no authentication for the browser UI and no per-user data

View File

@ -1 +1 @@
1.26.14
1.27.0

View File

@ -34,6 +34,9 @@ PDF
the vision call failed/timed out/was unreachable
→ build_vision_blocks() vision regions -> Blocks; a direct label bypasses
classify_blocks entirely (see below)
→ snap_visual_bboxes() figure/chart/table/code boxes re-derived from the page
render, because grounding is the weakest part of a
vision engine's answer (see below)
→ classify_blocks() title / deck / byline / body / caption / pullquote /
sidebar / furniture / ad / figure -- for native/tesseract
blocks, and any vision region left ambiguous ("text")
@ -357,6 +360,58 @@ fallback and not the default.
A formal cross-engine accuracy benchmark remains TODO P2.
### Figure bounds come from the render, not the engine
A vision OCR engine grounds a figure far less precisely than it transcribes
one. Measured over all 138 figure regions of `c't KI Wissen 2026` (204 pages,
Baidu Unlimited-OCR): text boxes matched the page render closely enough to
use as-is, while figure boxes were routinely offset, undersized, or landed on
the neighbouring text column — page 23's card grid was reported at
`x 0.24–0.54` of the page where the graphic actually runs `0.41–0.95`, and
page 20's full-width illustration was cut off at a third of its height. The
transcribed *contents* of those same regions were right, which is the useful
signal: the model reliably says **which** graphic it means, so the render is
asked **where** that graphic ends.
`snap_visual_bboxes()` (core/magazine.py) does that, on a coarse grid of the
rendered page — `page_ink_grid()` reduces it to the inked fraction of each
10 px cell, which separates graphics from type cleanly: body copy covers
roughly a quarter of the cells it runs through, a photo or filled panel
covers nearly all of them. The engine's box seeds a connected-component
search over the "solid" cells (scipy `ndimage.label`); components a gutter
apart that line up are re-joined, so a grid of cards or a chart beside its
key comes back as one figure rather than four.
Three fences keep it from making things worse, because a box confidently
snapped onto the wrong thing is worse than a loose one the user can see:
- **Text blocks are excluded from the mask up front.** They are the one
trustworthy boundary on the page, and without them a figure happily
swallows the column beside it.
- **Acceptance is explicit.** The result must be either solidly inked
(`_SNAP_MIN_FILL`) or framed on nearly all of its rows *and* columns — the
second test is what admits a UI screenshot, which is white inside and inked
only at its edge — and it may not shrink below `_SNAP_MIN_SEED_AREA` of the
engine's box. Anything else keeps the engine's box untouched.
- **Two figures pulled onto one graphic revert.** Either the page is one
picture the engine read as two or the reverse; neither reading can be
trusted over the other.
Result on that issue: 78 of 138 boxes corrected, none made worse on visual
inspection of the sample, the rest left alone (pale charts on white, mostly,
which the ink test correctly declines). The pass is idempotent — an
already-correct box seeds the same graphic again — which is what allows it to
run outside ingestion: `refresh_visual_regions()` re-snaps on every
re-detection, and `POST /api/magazine/{id}/figures/refine` re-snaps a whole
issue as a background job, so a document ingested before this pass existed is
corrected without paying for its transcription a second time. It replaced
`refine_picture_bounds()`, which only recognised *coloured* panels and had no
text fence.
The snap runs before `crop_visual_regions()` in both OCR paths: the figure
files shown during playback are cropped from these boxes, so a box corrected
late would leave a mis-framed image on disk.
The Active OCR Engine panel (Settings → Engines → OCR & Document Parsing)
mirrors the Active Language Model panel exactly — same endpoint/model
selects, same `/api/conversation/llm-models` fetch (it just proxies to

View File

@ -1143,6 +1143,201 @@ def build_vision_blocks(regions: list[dict[str, Any]], page_no: int,
return out
# ── Figure bounds from the page render ───────────────────────────────────────
#
# A vision OCR engine grounds a figure far less precisely than it transcribes
# one: on c't KI Wissen 2026 the label and the read-out contents were right
# while the four numbers landed on the neighbouring text column at two thirds
# of the real size (page 23), or covered only the top third of a full-width
# illustration (page 20). The rendered page itself carries the answer -- a
# photo, a chart or a UI screenshot is a densely inked rectangle fenced off
# from the type area by white gutters -- so the model's box is used only to
# say WHICH graphic is meant and the render says where that graphic ends.
#
# Deliberately conservative: every rule below can decline, and declining
# leaves the model's box untouched. A wrong box the user can see and redraw
# is a nuisance; a box confidently snapped onto the wrong thing is worse.
_SNAP_CELL_PX = 10 # grid resolution -- ~2 mm at the 110 dpi page render
_SNAP_SOLID = 0.55 # inked fraction marking a cell as graphic, not type
_SNAP_GAP_CELLS = 3 # white gutter a tiled graphic is allowed to span
_SNAP_MIN_FILL = 0.45 # an accepted box is this solidly inked ...
_SNAP_MIN_FRAME = 0.9 # ... or framed on this share of its rows AND columns
_SNAP_MIN_SEED_AREA = 0.30 # ... and never shrinks below this much of the seed
_SNAP_MIN_OVERLAP = 0.02 # a graphic must touch the seed by at least this much
def page_ink_grid(png_path: Path, cell: int = _SNAP_CELL_PX):
"""Coarse map of the page render: inked fraction of each `cell`-px square.
"Inked" means visibly darker than the paper or visibly coloured, so a
grey scan background does not count as ink. Body type covers roughly a
quarter of the cells it runs through; a photo or a filled panel covers
nearly all of them, which is the separation snap_visual_bboxes() relies
on. Returns None when the render cannot be read.
"""
try:
import numpy as np
from PIL import Image
with Image.open(png_path) as im:
rgb = np.asarray(im.convert("RGB"), dtype=np.float32)
except Exception:
return None
lightness = rgb.max(axis=2)
saturation = lightness - rgb.min(axis=2)
paper = float(np.percentile(lightness, 90)) or 255.0
ink = (lightness < paper * 0.88) | (saturation > 40)
rows, cols = (n // cell for n in ink.shape)
if rows < 4 or cols < 4:
return None
return ink[:rows * cell, :cols * cell].reshape(
rows, cell, cols, cell).mean(axis=(1, 3))
def _snap_seed(coverage, seed: Sequence[int], blocked) -> list[int] | None:
"""Grow the graphic the seed cell-box points at, or None to keep the seed.
Cells already claimed by a text block are excluded up front: text boxes
come back from the same engine far more accurately than figure boxes
(verified against the page render for every block of the issue above),
so they are the one trustworthy fence against a figure swallowing the
column beside it.
"""
from scipy import ndimage
solid = (coverage >= _SNAP_SOLID) & ~blocked
labels, count = ndimage.label(solid)
if not count:
return None
slices = ndimage.find_objects(labels)
boxes = [[s[1].start, s[0].start, s[1].stop - 1, s[0].stop - 1]
for s in slices]
sizes = ndimage.sum_labels(solid, labels, range(1, count + 1))
sx0, sy0, sx1, sy1 = seed
seed_area = max(1, (sx1 - sx0 + 1) * (sy1 - sy0 + 1))
overlaps = [_cell_overlap(b, seed) for b in boxes]
best = max(range(count), key=overlaps.__getitem__)
if overlaps[best] < _SNAP_MIN_OVERLAP * seed_area:
return None
# A figure split by white gutters (a grid of cards, a chart beside its
# key) arrives as several components; re-join the ones that line up with
# what we have so far, then re-test -- joining one may bring the next
# within reach.
box = list(boxes[best])
joined = {best}
growing = True
while growing:
growing = False
for i, other in enumerate(boxes):
if i in joined or sizes[i] < 4 or not _cells_adjacent(box, other):
continue
box = [min(box[0], other[0]), min(box[1], other[1]),
max(box[2], other[2]), max(box[3], other[3])]
joined.add(i)
growing = True
area = (box[2] - box[0] + 1) * (box[3] - box[1] + 1)
if area < _SNAP_MIN_SEED_AREA * seed_area:
return None
inside = coverage[box[1]:box[3] + 1, box[0]:box[2] + 1]
if inside.mean() >= _SNAP_MIN_FILL:
return box
# A screenshot or a boxed insert is mostly white inside but inked right
# around its edge, so a solid frame counts as a graphic too.
edge = solid[box[1]:box[3] + 1, box[0]:box[2] + 1]
if (edge.any(axis=1).mean() >= _SNAP_MIN_FRAME
and edge.any(axis=0).mean() >= _SNAP_MIN_FRAME):
return box
return None
def _cell_overlap(a: Sequence[int], b: Sequence[int]) -> int:
"""Area, in grid cells, shared by two inclusive [x0, y0, x1, y1] boxes."""
return (max(0, min(a[2], b[2]) - max(a[0], b[0]) + 1)
* max(0, min(a[3], b[3]) - max(a[1], b[1]) + 1))
def _cells_adjacent(box: Sequence[int], other: Sequence[int]) -> bool:
"""True when `other` is a gutter away from `box` and lines up with it."""
gap = _SNAP_GAP_CELLS
if not (other[0] <= box[2] + gap and other[2] >= box[0] - gap
and other[1] <= box[3] + gap and other[3] >= box[1] - gap):
return False
# Side by side or stacked, not merely diagonal: a tile of the same figure
# shares most of a row or most of a column with it.
shared_rows = max(0, min(other[3], box[3]) - max(other[1], box[1]) + 1)
shared_cols = max(0, min(other[2], box[2]) - max(other[0], box[0]) + 1)
return (shared_rows >= 0.5 * min(other[3] - other[1] + 1, box[3] - box[1] + 1)
or shared_cols >= 0.5 * min(other[2] - other[0] + 1, box[2] - box[0] + 1))
def snap_visual_bboxes(png_path: Path, blocks: Sequence[Block],
page_meta: dict[str, Any]) -> int:
"""Correct every visual block's bbox against the page render, in place.
Returns how many boxes moved. Silent no-op when the render is missing or
the imaging stack is unavailable -- the layout stays exactly as the OCR
engine reported it, which is what happened before this pass existed.
"""
visuals = [b for b in blocks if b.type in VISUAL_TYPES]
if not visuals:
return 0
try:
import numpy as np
import scipy.ndimage # noqa: F401 -- required by _snap_seed
except ImportError:
return 0
coverage = page_ink_grid(png_path)
if coverage is None:
return 0
rows, cols = coverage.shape
page_w = float(page_meta.get("width") or 0) or 1.0
page_h = float(page_meta.get("height") or 0) or 1.0
cells_x, cells_y = cols / page_w, rows / page_h
def to_cells(bbox: Sequence[float]) -> list[int]:
x, y, w, h = bbox
return [min(cols - 1, max(0, int(x * cells_x))),
min(rows - 1, max(0, int(y * cells_y))),
min(cols - 1, max(0, int((x + w) * cells_x))),
min(rows - 1, max(0, int((y + h) * cells_y)))]
blocked = np.zeros(coverage.shape, dtype=bool)
for b in blocks:
if b.type in VISUAL_TYPES or not b.text.strip():
continue
x0, y0, x1, y1 = to_cells(b.bbox)
blocked[y0:y1 + 1, x0:x1 + 1] = True
snapped = [_snap_seed(coverage, to_cells(b.bbox), blocked) for b in visuals]
# Two figures pulled onto the same graphic means the page was read as
# one picture where the engine saw two (or the reverse); neither reading
# can be trusted over the other, so both keep their original box.
for i, a in enumerate(snapped):
for j in range(i + 1, len(snapped)):
c = snapped[j]
if not a or not c:
continue
smaller = min((a[2] - a[0] + 1) * (a[3] - a[1] + 1),
(c[2] - c[0] + 1) * (c[3] - c[1] + 1))
if _cell_overlap(a, c) > 0.5 * smaller:
snapped[i] = snapped[j] = None
a = None
moved = 0
for block, box in zip(visuals, snapped):
if box is None:
continue
bbox = [box[0] / cells_x, box[1] / cells_y,
(box[2] + 1 - box[0]) / cells_x, (box[3] + 1 - box[1]) / cells_y]
if any(abs(a - b) > 1.0 for a, b in zip(bbox, block.bbox)):
moved += 1
block.bbox = bbox
return moved
def ocr_page(png_path: Path, lang: str = "deu+eng",
min_confidence: float = 40.0) -> list[dict[str, Any]]:
"""OCR one rendered page, returning paragraph boxes from tesseract's TSV.
@ -1355,8 +1550,13 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110,
# conflicting reading on top of it.
vision_page_count += 1
blocks = vision_blocks
page_meta = {"width": rect.width, "height": rect.height}
# Before cropping: the crop is taken from the block's bbox,
# so a box the engine placed loosely would be saved as a
# mis-framed figure file as well as a mis-drawn review box.
snap_visual_bboxes(png, blocks, page_meta)
detected_images = crop_visual_regions(out_dir, page_no,
{"width": rect.width, "height": rect.height}, blocks)
page_meta, blocks)
if detected_images:
page_images = detected_images
elif needs_ocr:
@ -1604,6 +1804,7 @@ def reocr_page(doc_dir: Path, layout: dict[str, Any], page_no: int,
scale_x = page_meta["width"] / max(1.0, _png_width(png, fitz))
new_blocks = build_vision_blocks(regions, page_no, scale_x, id_prefix="r")
snap_visual_bboxes(png, new_blocks, page_meta)
all_blocks = [Block(**b) for b in layout.get("blocks", [])]
kept = [b for b in all_blocks if b.page != page_no]
@ -1734,54 +1935,23 @@ def _crop_page_png(doc_dir: Path, page_no: int, page_meta: dict[str, Any],
im.crop(box_px).save(dest)
def refine_picture_bounds(png: Path, page_meta: dict, bbox: Sequence[float]) -> list[float]:
"""Snap an approximate picture box to a large, solid coloured panel.
Text and small decorations do not qualify. Grayscale or fragmented
artwork keeps the model's box instead of guessing a new boundary.
"""
import numpy as np
from PIL import Image
from scipy import ndimage
with Image.open(png) as image:
rgb = np.asarray(image.convert("RGB"), dtype=np.int16)
height, width = rgb.shape[:2]
mask = (rgb.max(axis=2) - rgb.min(axis=2) > 28) & (rgb.min(axis=2) < 225)
mask = ndimage.binary_closing(mask, iterations=3)
labels, _ = ndimage.label(mask)
candidates = []
x, y, w, h = bbox
for i, slices in enumerate(ndimage.find_objects(labels), 1):
if slices is None:
continue
sy, sx = slices
area = (sy.stop - sy.start) * (sx.stop - sx.start)
if area < width * height * .025 or area > width * height * .85:
continue
if np.count_nonzero(labels[slices] == i) / area < .7:
continue
box = [sx.start / width * page_meta["width"], sy.start / height * page_meta["height"],
(sx.stop - sx.start) / width * page_meta["width"], (sy.stop - sy.start) / height * page_meta["height"]]
overlap = max(0, min(x+w, box[0]+box[2])-max(x, box[0])) * max(0, min(y+h, box[1]+box[3])-max(y, box[1]))
score = overlap / max(1, min(w*h, box[2]*box[3]))
if score > .6:
candidates.append((score, box))
return max(candidates, key=lambda c: c[0])[1] if candidates else list(bbox)
def refresh_visual_regions(doc_dir: Path, layout: dict, page: int | None = None) -> None:
"""Regenerate semantic crops and avoid treating a scanned page as a figure."""
"""Re-snap figure bounds, regenerate the crops taken from them, and avoid
treating a whole scanned page as one figure."""
for meta in layout.get("pages", []):
page_no = meta["page"]
if page is not None and page_no != page:
continue
blocks = [Block(**b) for b in layout.get("blocks", []) if b["page"] == page_no]
for block in blocks:
if block.type == BLOCK_FIGURE and block.source == "llm_layout":
block.bbox = refine_picture_bounds(doc_dir / "pages" / f"{page_no:04d}.png", meta, block.bbox)
# Re-snapping here, not only at OCR time, is what lets an issue
# ingested before this pass existed pick up corrected figure boxes
# without paying for a second OCR run of the whole document. It is
# idempotent: an already-correct box seeds the same graphic again.
if snap_visual_bboxes(doc_dir / "pages" / f"{page_no:04d}.png", blocks, meta):
snapped = {b.id: b.bbox for b in blocks if b.type in VISUAL_TYPES}
for saved in layout["blocks"]:
if saved["id"] == block.id:
saved["bbox"] = block.bbox
if saved["id"] in snapped:
saved["bbox"] = snapped[saved["id"]]
crops = crop_visual_regions(doc_dir, page_no, meta, blocks)
kept = []
for img in layout.get("images", []):

View File

@ -646,6 +646,53 @@ async def magazine_reocr_page(doc_id: str, page: int, request: Request):
return {"job_id": job_id}
def _refine_figures_sync(doc_id: str, job_id: str) -> None:
"""Re-snap every figure box in a document against its page renders.
Restart-safe like the other layout jobs: a 200-page issue means 200 page
renders read and re-cropped, far too long for a request/response cycle.
Cheap compared with re-ingesting, though -- no OCR engine is called, so
an issue read before the snapping pass existed can be corrected without
paying for the transcription again, and without disturbing the text,
the article grouping, or any approval the user has already given.
"""
d = _LIB / doc_id
try:
layout = mag.load_layout(d)
if not layout:
raise FileNotFoundError("No layout yet -- run ingestion first")
pages = layout.get("pages", [])
before = {b["id"]: list(b["bbox"]) for b in layout.get("blocks", [])}
for i, meta in enumerate(pages, 1):
background_job_update(job_id, stage="figures",
detail=f"Checking figure boundaries on page {i} of {len(pages)}")
mag.refresh_visual_regions(d, layout, meta["page"])
moved = sum(1 for b in layout.get("blocks", [])
if before.get(b["id"]) and b["bbox"] != before[b["id"]])
mag.save_layout(d, layout)
background_job_update(
job_id, stage="done", done=True, result={"moved": moved},
detail=f"{moved} figure boundar{'y' if moved == 1 else 'ies'} corrected"
if moved else "Every figure boundary already matched its page")
except Exception as exc: # noqa: BLE001 - surfaced to UI
background_job_update(job_id, stage="failed", done=True, error=str(exc)[:1000])
@router.post("/api/magazine/{doc_id}/figures/refine")
async def magazine_refine_figures(doc_id: str):
"""Re-check every figure boundary in this issue against the page renders.
For an issue whose figure boxes came back loose from the OCR engine: the
boxes are re-derived from the rendered pages and the figure crops taken
from them are rebuilt. Text, articles and approvals are untouched.
"""
_require_magazine(doc_id)
job_id = uuid.uuid4().hex
background_job_create(job_id, "magazine-refine-figures", fingerprint=doc_id)
asyncio.create_task(asyncio.to_thread(_refine_figures_sync, doc_id, job_id))
return {"job_id": job_id}
def _add_region_sync(doc_id: str, job_id: str, page: int, bbox: list[float], kind: str,
text: str, vision_ocr_url: str | None, vision_ocr_model: str | None,
vision_ocr_fallback_url: str | None, vision_ocr_fallback_model: str | None) -> None:

View File

@ -740,9 +740,12 @@ function renderReview() {
'<span class="mdi mdi-advertisements-outline" aria-hidden="true"></span> Advertisement</button>' +
'<button type="button" class="btn-sm" onclick="magReviewClearCurrentPage()"><span class="mdi mdi-checkbox-multiple-blank-outline" aria-hidden="true"></span> Clear all fields</button></div>' +
'<span class="mag-review-pagenav-spacer"></span>' +
'<button type="button" class="btn-sm" id="mag-refine-figures-btn" onclick="magRefineFigures()" ' +
'title="Re-check every figure boundary in this issue against the scanned pages">' +
'<span class="mdi mdi-image-auto-adjust" aria-hidden="true"></span> Fix figure boundaries</button>' +
'<button type="button" class="btn-sm" id="mag-reocr-btn" onclick="magReocrPage()">' +
'<span class="mdi mdi-text-recognition"></span> OCR again (this page)</button>' +
'<span id="mag-reocr-status" class="muted"></span>' +
'<span id="mag-reocr-status" class="muted" role="status"></span>' +
'</div>';
html += '<div class="mag-review-fields">';
@ -1173,6 +1176,45 @@ async function magReocrPage() {
renderReview();
}
// Re-derives every figure box in the issue from the page renders. Separate
// from "OCR again" because no OCR engine is involved: it only corrects where
// the figures are, never what the page says, so it is safe to run on an issue
// that is half reviewed already.
async function magRefineFigures() {
const btn = $('mag-refine-figures-btn');
const statusEl = $('mag-reocr-status');
if (btn) btn.disabled = true;
if (statusEl) statusEl.textContent = 'Checking figure boundaries…';
let jobId;
try {
const r = await api('/api/magazine/' + state.doc.id + '/figures/refine', { method: 'POST' });
jobId = r.job_id;
} catch (e) {
if (btn) btn.disabled = false;
if (statusEl) statusEl.textContent = '';
return say('Could not start the check: ' + e.message, 'err');
}
let failed = false;
for (;;) {
let job;
try { job = await api('/api/jobs/' + jobId); } catch (_) { break; }
if (job.done) {
if (job.error) { failed = true; say('Figure check failed: ' + job.error, 'err'); }
else say(job.detail || 'Figure boundaries checked', 'ok');
break;
}
if (statusEl && job.detail) statusEl.textContent = job.detail;
await new Promise((r) => setTimeout(r, 2000));
}
if (btn) btn.disabled = false;
if (statusEl) statusEl.textContent = '';
if (failed) return;
state.layout = null;
await ensureLayout();
_magReviewChecked = null;
renderReview();
}
async function magApprove() {
const a = state.article;
if (!a) return;
@ -1805,6 +1847,7 @@ root.magReviewFocusBlock = magReviewFocusBlock;
root.magReviewHighlightBox = magReviewHighlightBox;
root.magReviewToggleBox = magReviewToggleBox;
root.magReocrPage = magReocrPage;
root.magRefineFigures = magRefineFigures;
root.magReviewToggleImage = magReviewToggleImage;
root.magReviewToggleDraw = magReviewToggleDraw;
root.magReviewCancelDrawPanel = magReviewCancelDrawPanel;

View File

@ -19,6 +19,9 @@ os.environ.setdefault("CONFIG_DIR", tempfile.mkdtemp(prefix="ttvc-magazine-test-
from core.magazine import (
ocr_page,
cluster_running_furniture,
snap_visual_bboxes,
page_ink_grid,
BLOCK_FIGURE,
looks_like_real_title,
Article,
BLOCK_AD,
@ -1435,5 +1438,102 @@ class ManualRegionTests(unittest.TestCase):
self.assertEqual(len(names), 2)
class SnapVisualBboxesTests(unittest.TestCase):
"""A vision OCR engine grounds a figure loosely; the render is the truth.
The synthetic page mirrors the real failure seen on c't KI Wissen 2026:
a solid graphic on the right, a text column on the left, and a reported
figure box that straddles both at a fraction of the real size.
"""
PAGE_W, PAGE_H = 582.0, 678.0 # PDF points
PX_W, PX_H = 890, 1037 # rendered pixels
FIGURE_PX = (400, 100, 850, 400) # x0, y0, x1, y1
TEXT_PX = (60, 100, 350, 500)
def setUp(self):
from PIL import Image, ImageDraw
self.dir = pathlib.Path(tempfile.mkdtemp())
(self.dir / "pages").mkdir()
page = Image.new("RGB", (self.PX_W, self.PX_H), (255, 255, 255))
draw = ImageDraw.Draw(page)
draw.rectangle(self.FIGURE_PX, fill=(40, 90, 200))
x0, y0, x1, y1 = self.TEXT_PX
for y in range(y0, y1, 12): # type covers ~a third of its cells
draw.rectangle((x0, y, x1, y + 3), fill=(20, 20, 20))
self.png = self.dir / "pages" / "0001.png"
page.save(self.png)
self.meta = {"page": 1, "width": self.PAGE_W, "height": self.PAGE_H}
def tearDown(self):
shutil.rmtree(self.dir, ignore_errors=True)
def _pt(self, px_box):
"""Pixel [x0, y0, x1, y1] as a PDF-point [x, y, w, h] bbox."""
sx, sy = self.PAGE_W / self.PX_W, self.PAGE_H / self.PX_H
x0, y0, x1, y1 = px_box
return [x0 * sx, y0 * sy, (x1 - x0) * sx, (y1 - y0) * sy]
def _text_block(self):
return Block(id="t1", page=1, bbox=self._pt(self.TEXT_PX),
type=BLOCK_BODY, text="body copy " * 40)
def _figure_block(self, px_box):
return Block(id="f1", page=1, bbox=self._pt(px_box), type=BLOCK_FIGURE,
text="figure", source="vision_label")
def test_grid_separates_graphic_from_type(self):
grid = page_ink_grid(self.png)
self.assertIsNotNone(grid)
rows, cols = grid.shape
fig = grid[int(200 / self.PX_H * rows), int(600 / self.PX_W * cols)]
txt = grid[int(200 / self.PX_H * rows), int(200 / self.PX_W * cols)]
self.assertGreater(fig, 0.9)
self.assertLess(txt, 0.5)
def test_offset_box_snaps_onto_the_graphic(self):
block = self._figure_block((200, 60, 600, 250))
moved = snap_visual_bboxes(self.png, [block, self._text_block()], self.meta)
self.assertEqual(moved, 1)
want = self._pt(self.FIGURE_PX)
for got, expected in zip(block.bbox, want):
self.assertAlmostEqual(got, expected, delta=12.0)
def test_snap_does_not_swallow_the_text_column(self):
block = self._figure_block((200, 60, 600, 250))
snap_visual_bboxes(self.png, [block, self._text_block()], self.meta)
self.assertGreater(block.bbox[0], self._pt(self.TEXT_PX)[0] + 20)
def test_snapping_is_idempotent(self):
block = self._figure_block((200, 60, 600, 250))
blocks = [block, self._text_block()]
snap_visual_bboxes(self.png, blocks, self.meta)
settled = list(block.bbox)
self.assertEqual(snap_visual_bboxes(self.png, blocks, self.meta), 0)
self.assertEqual(block.bbox, settled)
def test_box_over_type_only_is_left_alone(self):
block = self._figure_block((60, 100, 350, 500))
before = list(block.bbox)
self.assertEqual(
snap_visual_bboxes(self.png, [block, self._text_block()], self.meta), 0)
self.assertEqual(block.bbox, before)
def test_two_figures_on_one_graphic_keep_their_boxes(self):
left = self._figure_block((410, 110, 600, 380))
right = Block(id="f2", page=1, bbox=self._pt((610, 110, 840, 380)),
type=BLOCK_FIGURE, text="figure", source="vision_label")
before = [list(left.bbox), list(right.bbox)]
self.assertEqual(snap_visual_bboxes(self.png, [left, right], self.meta), 0)
self.assertEqual([left.bbox, right.bbox], before)
def test_missing_render_is_a_no_op(self):
block = self._figure_block((200, 60, 600, 250))
before = list(block.bbox)
self.assertEqual(
snap_visual_bboxes(self.dir / "pages" / "0099.png", [block], self.meta), 0)
self.assertEqual(block.bbox, before)
if __name__ == "__main__":
unittest.main()

View File

@ -160,3 +160,61 @@ class AdvertisementReviewTests(unittest.TestCase):
self.assertEqual(len(layout['blocks']), 1)
self.assertEqual(layout['blocks'][0]['type'], 'ad')
self.assertEqual(layout['blocks'][0]['bbox'], [0, 0, 600, 800])
class RefineFiguresJobTests(unittest.TestCase):
"""Document-wide figure re-snap: reports what moved, keeps text untouched."""
def _layout(self):
return {
"pages": [{"page": 1, "width": 600, "height": 800},
{"page": 2, "width": 600, "height": 800}],
"blocks": [
{"id": "fig", "page": 1, "type": "figure", "bbox": [10, 10, 50, 50], "text": ""},
{"id": "copy", "page": 2, "type": "body", "bbox": [0, 0, 100, 100], "text": "Body"},
],
}
def test_moved_boxes_are_counted_and_saved(self):
from unittest.mock import patch
from routes.magazine import _refine_figures_sync
layout = self._layout()
def refresh(_dir, doc, page=None):
if page == 1:
doc["blocks"][0]["bbox"] = [20, 20, 90, 90]
with patch('routes.magazine.mag.load_layout', return_value=layout), \
patch('routes.magazine.mag.refresh_visual_regions', side_effect=refresh), \
patch('routes.magazine.mag.save_layout') as save, \
patch('routes.magazine.background_job_update') as update:
_refine_figures_sync('a' * 32, 'job')
save.assert_called_once()
done = [c.kwargs for c in update.call_args_list if c.kwargs.get('done')]
self.assertEqual(done[-1]['stage'], 'done')
self.assertEqual(done[-1]['result'], {'moved': 1})
self.assertEqual(layout['blocks'][1]['bbox'], [0, 0, 100, 100])
def test_nothing_to_correct_still_finishes_cleanly(self):
from unittest.mock import patch
from routes.magazine import _refine_figures_sync
with patch('routes.magazine.mag.load_layout', return_value=self._layout()), \
patch('routes.magazine.mag.refresh_visual_regions'), \
patch('routes.magazine.mag.save_layout'), \
patch('routes.magazine.background_job_update') as update:
_refine_figures_sync('a' * 32, 'job')
done = [c.kwargs for c in update.call_args_list if c.kwargs.get('done')]
self.assertEqual(done[-1]['result'], {'moved': 0})
self.assertNotIn('error', done[-1])
def test_a_document_with_no_layout_reports_failure(self):
from unittest.mock import patch
from routes.magazine import _refine_figures_sync
with patch('routes.magazine.mag.load_layout', return_value=None), \
patch('routes.magazine.mag.save_layout') as save, \
patch('routes.magazine.background_job_update') as update:
_refine_figures_sync('a' * 32, 'job')
save.assert_not_called()
done = [c.kwargs for c in update.call_args_list if c.kwargs.get('done')]
self.assertEqual(done[-1]['stage'], 'failed')
self.assertIn('No layout', done[-1]['error'])