fix(magazine): screen invented picture/chart/table regions against page ink

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

CHANGELOG.md

@@ -7,6 +7,11 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
 
 ## [Unreleased]
 
+## [1.42.4] — 2026-09-22
+
+### Fixed
+- **Magazine ingestion: a proposed picture, chart or table sometimes wasn't one.** The semantic-layout pass can invent a new picture region for a graphic OCR missed; on a real issue it sometimes drew that box over a column gap and the text in it instead (a table-of-contents heading and intro paragraph in one case), burying real paragraphs under an empty "Figure" placeholder. A proposed region is now checked against the page render's own ink density before it is created — a real photo, chart or table is solidly inked, a text area is not — confirmed against the actual false positives and true positives found on that issue (0.10-0.14 vs 0.56-0.97 mean ink, no overlap). QA also spot-checked headline, paragraph, column, page-number and advertisement recognition across a sample of pages of that issue; no other errors found.
+
 ## [1.42.3] — 2026-09-22
 
 ### Fixed

Manual.md

@@ -113,7 +113,12 @@ turned into a podcast discussion between two or three voices you pick.
    real edges of the picture for long. The recognised-text panel shows the
    actual cropped picture, chart or table for the same reason, rather than
    a plain "Figure" placeholder, as soon as it has been cropped. The page
-   scan itself (a scanned magazine is one picture per page) is not framed. If a run is interrupted
+   scan itself (a scanned magazine is one picture per page) is not framed.
+   A proposed picture, chart or table is checked against the page render
+   itself before it is created, so a text area or column gap is not shown
+   as a picture. A real picture can still get a loose or shifted box in
+   some cases; use *Refine figures* (or re-run detection on the page) to
+   try snapping it again, or fix the box by hand on the review page. If a run is interrupted
    (for example when the server restarts), the status now says so instead
    of spinning forever; start it again with *Re-run detection*.
 3. **Review the articles.** This step is not optional and cannot be skipped.

VERSION

@@ -1 +1 @@
-1.42.3
+1.42.4

architecture/magazine-ingestion.md

@@ -615,6 +615,40 @@ rendered. Confirmed live: the page now shows both engines as ready-made
 local cards (Connect/Start/Restart/Use as OCR/Show code snippet), matching
 the LLM page, with no "Add Custom" dialog needed for either.
 
+### A new figure the model invents is screened against the page's own ink
+
+The semantic-layout call is allowed to propose a picture/chart/table region
+with `block_ids: []` -- a graphic OCR never produced a block for. Confirmed
+on a real issue: it sometimes drew that box over a column gap and the text
+sitting in it instead -- a table-of-contents heading and its intro paragraph
+in one case, a photo caption's column in another -- which buried real,
+already-recognised paragraphs under an empty "Figure" placeholder rather
+than adding a picture that was actually missing.
+
+`region_looks_like_a_picture()` (core/magazine.py) screens every such
+invented region against `page_ink_grid_points()` before it is created:
+mean ink coverage over the proposed box's own area must reach
+`_INVENTED_VISUAL_MIN_INK` (0.4). Measured on the confirmed false positives
+and true positives of the same issue, the two cases do not come close to
+overlapping -- 0.10-0.14 for the invented regions that were actually text,
+0.56-0.97 for real photos, charts and tables -- so this is a wide floor, not
+a fine-tuned one. `apply_semantic_regions()` takes `page_ink` as an optional
+parameter (`None` skips the check, e.g. in a unit test with no page render
+to check against); `_recognise_semantic_layout()` computes it once per page
+from the same render already loaded for the vision prompt.
+
+This is a *different* signal from `snap_visual_bboxes()` above: that one
+corrects a real graphic's loose boundary once OCR/the model has already
+agreed something is there. This one decides whether the "something" should
+have existed at all. A region that survives this check can still get a loose
+box from the snap pass afterwards -- confirmed on the same issue, several
+real photos kept a box shifted or oversized enough to miss half the actual
+picture, which the snap pass's own stability check declined to correct
+rather than risk publishing a box the next run would replace (see the
+"pulled onto one graphic" fence above). That remaining imprecision is a
+candidate for the model-based layout detection already on the TODO list
+(PP-DocLayout gives a native picture bbox instead of an LLM's guess).
+
 ### A workflow-pipeline nav bar was bleeding into unrelated pages
 
 `_wfUpdateHeaderNav` (utils.js) injects the audiobook pipeline's "Previous /

core/magazine.py

@@ -1527,6 +1527,24 @@ def page_ink_grid(png_path: Path, cell: int = _SNAP_CELL_PX):
         rows, cell, cols, cell).mean(axis=(1, 3))
 
 
+def page_ink_grid_points(png_path: Path, page_meta: dict, cell: int = _SNAP_CELL_PX):
+    """page_ink_grid(), plus each cell's size in the page's own point units (as bboxes are stored
+    in), for a caller that has no reason to know the render's pixel size -- see
+    region_looks_like_a_picture, which screens a proposed visual region against this."""
+    grid = page_ink_grid(png_path, cell)
+    if grid is None:
+        return None
+    try:
+        from PIL import Image
+        with Image.open(png_path) as im:
+            w_px, h_px = im.size
+    except Exception:
+        return None
+    if not w_px or not h_px or not page_meta.get("width") or not page_meta.get("height"):
+        return None
+    return grid, page_meta["width"] * cell / w_px, page_meta["height"] * cell / h_px
+
+
 _SNAP_STABLE_IOU = 0.7    # a result must reproduce at least this much of
                           # itself (intersection over union) when fed back
                           # in as its own seed, or it is discarded -- see
@@ -2385,7 +2403,34 @@ def reocr_page(doc_dir: Path, layout: dict[str, Any], page_no: int,
     return layout
 
 
-def apply_semantic_regions(blocks: list[dict], regions: list[dict], page_meta: dict | None = None) -> list[dict]:
+_INVENTED_VISUAL_MIN_INK = 0.4   # see the guard below apply_semantic_regions; a real photo, chart
+                                  # or table measured 0.56-0.97 on confirmed real pages of the same
+                                  # issue, a text area the model mistook for one 0.10-0.14 -- wide
+                                  # margin either side of this floor
+
+
+def region_looks_like_a_picture(bbox: Sequence[float], page_ink) -> bool:
+    """Whether a NEW visual region (see apply_semantic_regions) is inked densely enough, over its
+    own area, to plausibly be a photo, chart or table rather than a text area or column gap the
+    model mistook for one. `page_ink` is (coverage_grid, cell_w, cell_h) in the same page-point
+    units as `bbox`, as produced by page_ink_grid() -- or None to skip the check (no page render to
+    check against, e.g. in a unit test), which always passes.
+    """
+    if page_ink is None:
+        return True
+    grid, cell_w, cell_h = page_ink
+    if grid is None or not cell_w or not cell_h:
+        return True
+    x, y, w, h = bbox
+    r0, r1 = max(0, int(y / cell_h)), min(grid.shape[0], int(math.ceil((y + h) / cell_h)))
+    c0, c1 = max(0, int(x / cell_w)), min(grid.shape[1], int(math.ceil((x + w) / cell_w)))
+    if r1 <= r0 or c1 <= c0:
+        return True   # too small to sample a page-render cell -- err toward keeping it
+    return float(grid[r0:r1, c0:c1].mean()) >= _INVENTED_VISUAL_MIN_INK
+
+
+def apply_semantic_regions(blocks: list[dict], regions: list[dict], page_meta: dict | None = None,
+                           page_ink=None) -> list[dict]:
     """Group existing OCR fragments without allowing invented IDs or text."""
     known = {b["id"]: b for b in blocks}
     used = set()
@@ -2409,6 +2454,14 @@ def apply_semantic_regions(blocks: list[dict], regions: list[dict], page_meta: d
             visual_box = [x0 / 1000 * page_meta["width"], y0 / 1000 * page_meta["height"],
                           (x1 - x0) / 1000 * page_meta["width"], (y1 - y0) / 1000 * page_meta["height"]]
         if not ids and kind in VISUAL_TYPES and visual_box:
+            # A "new" picture the model invents (block_ids: []) is meant for a graphic OCR never
+            # produced a block for. Confirmed on a real issue: it sometimes instead draws that box
+            # over a column gap and the text below it (a table-of-contents heading and its intro
+            # paragraph in one case) -- which would otherwise bury real paragraphs under an empty
+            # "Figure" placeholder. A real photo/chart/table is a densely inked area of the page
+            # render; that case is not, so page_ink screens it out before it is ever created.
+            if not region_looks_like_a_picture(visual_box, page_ink):
+                continue
             out.append(Block(id=f"p{page_meta['page']:04d}-visual-{len(out)}", page=page_meta['page'],
                              bbox=visual_box, type=kind, text="", reading_order=len(out),
                              source="llm_layout", region_label=label).to_json())

routes/magazine.py

@@ -720,6 +720,7 @@ def _recognise_semantic_layout(layout: dict, settings: dict, page: int | None =
                                             for b in batch], ensure_ascii=False)}]}
                 page_meta = next((p for p in layout.get("pages", []) if p["page"] == page_no), None)
                 png = doc_dir / "pages" / f"{page_no:04d}.png" if doc_dir else None
+                page_ink = mag.page_ink_grid_points(png, page_meta) if png and png.is_file() and page_meta else None
                 if png and png.is_file():
                     prompt = payload["messages"][1]["content"]
                     payload["messages"][1]["content"] = [
@@ -740,7 +741,7 @@ def _recognise_semantic_layout(layout: dict, settings: dict, page: int | None =
                             isinstance(confidence, (int, float)) and .9 <= confidence <= 1:
                         batch = [dict(b, type=mag.BLOCK_AD) for b in batch]
                         regions = [{"type": "advertisement", "block_ids": [b["id"] for b in batch], "bbox": [0,0,1000,1000]}]
-                    grouped = mag.apply_semantic_regions(batch, regions, page_meta)
+                    grouped = mag.apply_semantic_regions(batch, regions, page_meta, page_ink=page_ink)
                     ids = {b["id"] for b in batch}
                     layout["blocks"] = [b for b in layout["blocks"] if b["id"] not in ids] + grouped
                 except Exception:

tests/test_magazine.py

@@ -1825,6 +1825,90 @@ class DeleteRegionTests(unittest.TestCase):
         self.assertEqual(len(layout["blocks"]), 1)
 
 
+class InventedVisualRegionScreenTests(unittest.TestCase):
+    """A semantic-layout call can propose a brand-new picture/chart/table region (block_ids: [])
+    for a graphic OCR never produced a block for. Confirmed on a real issue (c't KI Wissen 2026):
+    it sometimes instead draws that box over a column gap and the text below it, which would bury
+    real paragraphs under an empty "Figure" placeholder. region_looks_like_a_picture and
+    apply_semantic_regions' page_ink screen that out using the page render's own ink density --
+    a real photo/chart/table is solid, a text area is not (confirmed: 0.56-0.97 vs 0.10-0.14 on
+    the actual false positives and true positives found on that issue)."""
+
+    PAGE_W, PAGE_H = 582.0, 678.0
+    PX_W, PX_H = 890, 1037
+    FIGURE_PX = (400, 100, 850, 400)
+    TEXT_PX = (60, 100, 350, 500)
+
+    def setUp(self):
+        from PIL import Image, ImageDraw
+        self.dir = pathlib.Path(tempfile.mkdtemp())
+        (self.dir / "pages").mkdir()
+        page = Image.new("RGB", (self.PX_W, self.PX_H), (255, 255, 255))
+        draw = ImageDraw.Draw(page)
+        draw.rectangle(self.FIGURE_PX, fill=(40, 90, 200))
+        x0, y0, x1, y1 = self.TEXT_PX
+        for y in range(y0, y1, 12):
+            draw.rectangle((x0, y, x1, y + 3), fill=(20, 20, 20))
+        self.png = self.dir / "pages" / "0001.png"
+        page.save(self.png)
+        self.meta = {"page": 1, "width": self.PAGE_W, "height": self.PAGE_H}
+
+    def tearDown(self):
+        shutil.rmtree(self.dir, ignore_errors=True)
+
+    def _pt(self, px_box):
+        sx, sy = self.PAGE_W / self.PX_W, self.PAGE_H / self.PX_H
+        x0, y0, x1, y1 = px_box
+        return [x0 * sx, y0 * sy, (x1 - x0) * sx, (y1 - y0) * sy]
+
+    def test_a_dense_region_looks_like_a_picture(self):
+        from core.magazine import region_looks_like_a_picture, page_ink_grid_points
+        page_ink = page_ink_grid_points(self.png, self.meta)
+        self.assertIsNotNone(page_ink)
+        self.assertTrue(region_looks_like_a_picture(self._pt(self.FIGURE_PX), page_ink))
+
+    def test_a_sparse_text_region_does_not_look_like_a_picture(self):
+        from core.magazine import region_looks_like_a_picture, page_ink_grid_points
+        page_ink = page_ink_grid_points(self.png, self.meta)
+        self.assertFalse(region_looks_like_a_picture(self._pt(self.TEXT_PX), page_ink))
+
+    def test_no_page_ink_always_passes(self):
+        from core.magazine import region_looks_like_a_picture
+        self.assertTrue(region_looks_like_a_picture(self._pt(self.TEXT_PX), None))
+
+    def test_apply_semantic_regions_keeps_a_real_picture_and_drops_a_text_area(self):
+        from core.magazine import apply_semantic_regions, page_ink_grid_points
+        page_ink = page_ink_grid_points(self.png, self.meta)
+
+        def permille(px_box):
+            x, y, w, h = self._pt(px_box)
+            return [x / self.PAGE_W * 1000, y / self.PAGE_H * 1000,
+                   (x + w) / self.PAGE_W * 1000, (y + h) / self.PAGE_H * 1000]
+
+        grouped = apply_semantic_regions([], [
+            {"type": "image", "block_ids": [], "bbox": permille(self.FIGURE_PX)},
+            {"type": "image", "block_ids": [], "bbox": permille(self.TEXT_PX)},
+        ], self.meta, page_ink=page_ink)
+        self.assertEqual(len(grouped), 1)
+        self.assertEqual(grouped[0]["type"], "figure")
+        for got, want in zip(grouped[0]["bbox"], self._pt(self.FIGURE_PX)):
+            self.assertAlmostEqual(got, want, delta=1.0)
+
+    def test_without_page_ink_both_are_kept_as_before(self):
+        from core.magazine import apply_semantic_regions
+
+        def permille(px_box):
+            x, y, w, h = self._pt(px_box)
+            return [x / self.PAGE_W * 1000, y / self.PAGE_H * 1000,
+                   (x + w) / self.PAGE_W * 1000, (y + h) / self.PAGE_H * 1000]
+
+        grouped = apply_semantic_regions([], [
+            {"type": "image", "block_ids": [], "bbox": permille(self.FIGURE_PX)},
+            {"type": "image", "block_ids": [], "bbox": permille(self.TEXT_PX)},
+        ], self.meta)
+        self.assertEqual(len(grouped), 2)
+
+
 class SnapVisualBboxesTests(unittest.TestCase):
     """A vision OCR engine grounds a figure loosely; the render is the truth.