fix(magazine): finish and harden the live ingest preview

Resumes an in-progress, uncommitted change from a previous session: the
live "page image + recognised text" preview shown during ingestion was
reworked to render formatted HTML (styled headings, a figure
placeholder) instead of a plain-text dump, but was left with a
duplication bug, dead code, inline styles, and no tests.

- Fixed duplication: a figure block's own transcribed text (usually its
  on-screen UI copy) was rendered as an ordinary paragraph in addition
  to the new figure placeholder box. Figure/chart/table/code blocks are
  now represented only by the placeholder.
- Furniture/sidebar/pullquote/ad text is shown muted instead of being
  dropped with no replacement, and is never promoted to a heading
  regardless of font size.
- Factored the formatting logic out of the progress closure into a
  standalone, unit-tested _render_ingest_preview_html().
- Replaced dead getattr(bbox, "y0", ...) calls (bbox is always a plain
  [x, y, w, h] list) with direct indexing, moved statistics/logging
  imports to module level, switched manual character replacement to
  html.escape(), and bounded the preview by an accumulated line budget
  instead of a blind slice that could truncate mid-tag.
- Replaced inline style="..." with CSS classes matching the panel's
  existing dark console treatment.
- The frontend now always uses innerHTML (every text node is escaped
  server-side), dropping the startsWith('<') content-type sniffing.
- Marked the preview panel aria-hidden -- decorative, and would
  otherwise churn noisily for screen reader users on top of the
  adjacent progress bar.

Verified against the real c't KI Wissen 2026 layout: a figure's raw
OCR text no longer appears twice, a real headline renders as <h3>, a
footer/pull-quote render muted, and all text is correctly escaped.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

CHANGELOG.md

@@ -5,6 +5,21 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
 
 ---
 
+## [1.27.2] — 2026-09-12
+
+### Fixed
+- **Live ingest preview finished and hardened** (was left as in-progress, uncommitted work from a previous session).
+  - Formatting logic factored out into a standalone, unit-tested `_render_ingest_preview_html()` instead of living inline inside the progress callback.
+  - Fixed a duplication bug: a figure's own transcribed text (usually its on-screen UI copy, e.g. "Post your new Instagram photos...") was rendered as an ordinary paragraph *in addition to* the new figure placeholder box. Figure/chart/table/code blocks are now represented only by the placeholder.
+  - Furniture, sidebar, pull-quote and advertisement text is shown muted (previously dropped the `[LABEL]` bracket format entirely with no replacement) and is never promoted to a heading regardless of font size.
+  - Replaced dead `getattr(bbox, "y0", ...)` calls (bbox is always a plain `[x, y, w, h]` list) with direct indexing, moved `import statistics`/`logging` to module level, and switched manual `&`/`<`/`>` replacement to `html.escape()`.
+  - Replaced inline `style="..."` attributes with CSS classes (`.mag-ingest-preview-figure`, `.mag-ingest-preview-excluded`, `.mag-ingest-preview-more`) in `style.css`, matching the panel's existing dark console treatment.
+  - Bounded the preview by content budget (accumulated per line, capped with an ellipsis marker) instead of a blind character slice that could truncate mid-tag.
+  - The always-HTML output let the frontend drop its `startsWith('<')` content-type sniffing in favor of always using `innerHTML` (every text node is escaped server-side).
+  - Marked the preview panel `aria-hidden` — it duplicates information already in the adjacent progress bar and would otherwise churn noisily for screen reader users.
+
+---
+
 ## [1.27.1] — 2026-09-12
 
 ### Added

Manual.md

@@ -32,9 +32,14 @@ turned into a podcast discussion between two or three voices you pick.
    renders every page, extracts text (falling back to OCR on scanned pages),
    classifies each region of the layout, finds the contents page, and proposes
    articles with page ranges. While it runs, a live preview shows each page's
-   image next to its recognised text as soon as that page finishes, so you can
+   image next to its recognised text as soon as that page finishes -- a
-   see ingestion actually working page by page instead of watching a bare
+   detected headline is shown larger, a figure appears as its own picture
-   progress bar.
+   marker rather than as garbled on-image text, and page furniture is shown
+   dimmed -- so you can see ingestion actually working page by page instead
+   of watching a bare progress bar. This preview is a quick approximation,
+   not the final read: it runs before the layout is fully classified, so a
+   heading guess or an excluded marking here can still change once review
+   opens.
 3. **Review the articles.** This step is not optional and cannot be skipped.
    Each proposed article shows its title, author, page range, estimated
    listening time and a confidence score. The review screen is split: the

VERSION

@@ -1 +1 @@
-1.27.1
+1.27.2

core/magazine.py

@@ -1593,7 +1593,10 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110,
                 # one side, recognised text on the other, filled in as each
                 # page finishes) without waiting for the document-wide passes
                 # below.
-                progress(page_no, doc.page_count, blocks)
+                try:
+                    progress(page_no, doc.page_count, blocks, page_images)
+                except TypeError:
+                    progress(page_no, doc.page_count, blocks)
 
         # Furniture can only be identified across the whole document, so
         # classification waits until every page has been read.

routes/magazine.py

@@ -18,8 +18,11 @@ from __future__ import annotations
 
 import asyncio
 import base64
+import html
 import json
+import logging
 import re
+import statistics
 import time
 import uuid
 from pathlib import Path
@@ -109,6 +112,71 @@ async def magazine_create(request: Request):
 
 # ── Stage A–C: ingestion ─────────────────────────────────────────────────────
 
+# Character budget for one page's live ingest preview (see
+# _render_ingest_preview_html). Generous enough for a dense page, small
+# enough that a runaway page (a poem-like layout with hundreds of short
+# fragments) cannot make the poll response slow.
+_INGEST_PREVIEW_BUDGET = 8000
+
+
+def _render_ingest_preview_html(blocks: list, images: list | None) -> str:
+    """Format one page's just-read blocks as simple HTML, in reading order.
+
+    Best-effort presentation only, for the live "here is what was just
+    read" preview during ingestion -- classify_blocks has not run yet at
+    this point, so the only firm type information available is whatever a
+    vision-labelling engine already assigned (footer/title/page_number/...
+    via build_vision_blocks); everything else is still the raw BLOCK_BODY
+    the extractor started with. A font-size heuristic against this page's
+    own median stands in for a heading distinction until classification
+    catches up.
+
+    A figure gets a placeholder box instead of its own transcribed text:
+    that text is usually the figure's on-screen copy (button labels, UI
+    chrome -- see the "Post your new Instagram photos" example in
+    architecture/magazine-ingestion.md) and reads as noise next to a box
+    that already says "this is a picture". Furniture/sidebar/pullquote/ad
+    text is shown, muted, so the preview still doubles as an early check
+    that vision labelling is working -- but never promoted to a heading,
+    regardless of its font size.
+
+    Every text node is escaped; the caller writes the result straight into
+    the page as innerHTML.
+    """
+    items: list[tuple[float, str, Any]] = [(b.bbox[1], "text", b) for b in blocks]
+    items += [(img["bbox"][1], "image", img) for img in (images or [])]
+    items.sort(key=lambda entry: entry[0])
+
+    font_sizes = [b.font_size for b in blocks if b.font_size > 0]
+    median_size = statistics.median(font_sizes) if font_sizes else 0.0
+
+    excluded_types = (mag.BLOCK_FURNITURE, mag.BLOCK_SIDEBAR, mag.BLOCK_PULLQUOTE, mag.BLOCK_AD)
+    parts: list[str] = []
+    budget_used = 0
+    for _, kind, item in items:
+        if kind == "image":
+            line = ('<div class="mag-ingest-preview-figure">'
+                    '<span class="mdi mdi-image-outline" aria-hidden="true"></span>'
+                    'Figure</div>')
+        else:
+            block = item
+            if block.type in mag.VISUAL_TYPES:
+                continue  # represented by its own figure placeholder instead
+            text = (block.text or "").strip()
+            if not text:
+                continue
+            excluded = block.type in excluded_types
+            tag = "h3" if not excluded and block.font_size > median_size + 2.0 else "div"
+            css_class = ' class="mag-ingest-preview-excluded"' if excluded else ""
+            line = f"<{tag}{css_class}>{html.escape(text)}</{tag}>"
+        if parts and budget_used + len(line) > _INGEST_PREVIEW_BUDGET:
+            parts.append('<div class="mag-ingest-preview-more">…</div>')
+            break
+        parts.append(line)
+        budget_used += len(line)
+    return "\n".join(parts)
+
+
 def _ingest_sync(doc_id: str, job_id: str, ocr_lang: str, render_dpi: int,
                  vision_ocr_url: str | None, vision_ocr_model: str | None,
                  vision_ocr_fallback_url: str | None = None,
@@ -121,35 +189,23 @@ def _ingest_sync(doc_id: str, job_id: str, ocr_lang: str, render_dpi: int,
         if not pdf.is_file():
             raise FileNotFoundError("No source.pdf uploaded for this magazine")
 
-        def progress(page: int, total: int, blocks: list) -> None:
+        def progress(page: int, total: int, blocks: list, images: list | None = None) -> None:
             background_job_update(
                 job_id, stage="extract",
                 detail=f"Analysing page {page} of {total}",
             )
-            # A live "here is what was just read" preview (page image +
+            # Written to its own small file rather than the generic job
-            # recognised text), mirroring ocr-baidu-unlimited-ocr's own demo
-            # UI. Written to its own small file rather than the generic job
             # result: that field strips any key resembling "text"/"content"
             # (data-minimisation for other job kinds) and caps at 500 chars,
             # both wrong for this. Best-effort -- a write failure must never
             # abort ingestion.
             try:
-                lines = []
+                preview_html = _render_ingest_preview_html(blocks, images)
-                for b in blocks:
-                    b_type = getattr(b, "type", "")
-                    label = getattr(b, "region_label", None) or b_type or "region"
-                    txt = (getattr(b, "text", "") or "").strip()
-                    if b_type in mag.VISUAL_TYPES or b_type in (mag.BLOCK_FURNITURE, mag.BLOCK_SIDEBAR, mag.BLOCK_PULLQUOTE, mag.BLOCK_AD):
-                        line_str = f"[{label.upper()}] {txt}".strip()
-                        lines.append(line_str)
-                    elif txt:
-                        lines.append(txt)
-                preview_text = "\n\n".join(lines)[:4000]
                 (d / "ingest_preview.json").write_text(
-                    json.dumps({"page": page, "text": preview_text}, ensure_ascii=False),
+                    json.dumps({"page": page, "text": preview_html}, ensure_ascii=False),
                     "utf-8")
             except Exception:
-                pass
+                logging.exception("Failed to write live ingest preview for page %s", page)
 
         background_job_update(job_id, stage="extract", detail="Opening PDF")
         layout = mag.extract_pages(pdf, d, render_dpi=render_dpi, ocr_lang=ocr_lang,

static/js/magazine.js

@@ -500,7 +500,10 @@ async function pollIngest() {
           if (img) img.src = '/api/magazine/' + state.doc.id + '/pages/' + data.preview.page;
         }
         const textEl = $('mag-ingest-preview-text');
-        if (textEl) textEl.textContent = data.preview.text || '';
+        // Server-built HTML (see _render_ingest_preview_html in
+        // routes/magazine.py) -- every text node in it is already escaped,
+        // so this is the one place that inner-HTMLs a live server value.
+        if (textEl) textEl.innerHTML = data.preview.text || '';
       } else if (ing.status !== 'running') {
         preview.hidden = true;
       }

static/sections/s-magazine.html

@@ -70,7 +70,11 @@
           </div>
         </div>
         <div id="mag-ingest-status" class="mag-status" hidden></div>
-        <div id="mag-ingest-preview" class="mag-ingest-preview" hidden>
+        <!-- Decorative "watch it work" visualisation -- the essential
+             progress information (percentage, page count, done/failed) is
+             in #mag-ingest-status above, which is what a screen reader user
+             needs; this panel adds nothing beyond that but noisy churn. -->
+        <div id="mag-ingest-preview" class="mag-ingest-preview" hidden aria-hidden="true">
           <div class="mag-ingest-preview-side">
             <span class="mag-ingest-preview-label">Page</span>
             <img id="mag-ingest-preview-img" alt="Page currently being read">

static/style.css

@@ -4830,6 +4830,18 @@ input[list="lang-codes"] {
 .mag-ingest-preview-side img { width: 100%; min-width: 0; flex: 1 1 auto; min-height: 0; object-fit: contain; margin: 0 auto; border-radius: 4px; }
 .mag-ingest-preview-text { color: #d7dae0; font-size: 13px; line-height: 1.6; white-space: pre-wrap;
   overflow-y: auto; min-height: 0; flex: 1 1 auto; font-family: ui-monospace, 'SF Mono', Consolas, monospace; }
+/* Elements the live preview's HTML formatting inserts (see
+   _render_ingest_preview_html in routes/magazine.py) -- sans-serif, unlike
+   the monospace body text, so a detected headline visibly reads as one. */
+.mag-ingest-preview-text h3 { margin: 14px 0 6px; font: 600 14px/1.35 -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif; color: #e8eaef; }
+.mag-ingest-preview-text h3:first-child { margin-top: 0; }
+.mag-ingest-preview-text .mag-ingest-preview-figure { display: flex; flex-direction: column; align-items: center; gap: 4px;
+  margin: 12px 0; padding: 14px; border: 1px dashed #3a3f47; border-radius: var(--radius);
+  background: rgba(255,255,255,.04); color: #8a8f98; font: 500 11px/1.3 -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif;
+  text-transform: uppercase; letter-spacing: .05em; }
+.mag-ingest-preview-text .mag-ingest-preview-figure .mdi { font-size: 22px; }
+.mag-ingest-preview-text .mag-ingest-preview-excluded { color: #6b7078; font-style: italic; }
+.mag-ingest-preview-text .mag-ingest-preview-more { color: #6b7078; text-align: center; padding: 4px 0 0; }
 .mag-ingest-preview-resize { background: var(--border); cursor: col-resize; position: relative; touch-action: none; }
 .mag-ingest-preview-resize::after { content: ''; position: absolute; inset: 0 2px; background: transparent; }
 .mag-ingest-preview.is-resizing { user-select: none; cursor: col-resize; }

tests/test_magazine_routes.py

@@ -218,3 +218,76 @@ class RefineFiguresJobTests(unittest.TestCase):
         done = [c.kwargs for c in update.call_args_list if c.kwargs.get('done')]
         self.assertEqual(done[-1]['stage'], 'failed')
         self.assertIn('No layout', done[-1]['error'])
+
+
+class IngestPreviewHtmlTests(unittest.TestCase):
+    """The live ingest preview: a fast, best-effort HTML render of one
+    page's just-read blocks, well before classify_blocks or article
+    grouping has run."""
+
+    def _block(self, text, font_size=10.0, type="body", bbox=None):
+        from core.magazine import Block
+        return Block(id="b", page=1, bbox=bbox or [0, 0, 100, 10],
+                    type=type, text=text, font_size=font_size)
+
+    def test_reading_order_follows_vertical_position(self):
+        from routes.magazine import _render_ingest_preview_html
+        top = self._block("First", bbox=[0, 10, 100, 10])
+        bottom = self._block("Second", bbox=[0, 200, 100, 10])
+        html = _render_ingest_preview_html([bottom, top], None)
+        self.assertLess(html.index("First"), html.index("Second"))
+
+    def test_large_font_becomes_a_heading(self):
+        from routes.magazine import _render_ingest_preview_html
+        body = self._block("Normal copy", font_size=10.0)
+        heading = self._block("Big Headline", font_size=24.0, bbox=[0, 50, 100, 10])
+        html = _render_ingest_preview_html([body, heading], None)
+        self.assertIn("<h3>Big Headline</h3>", html)
+        self.assertNotIn("<h3>Normal copy</h3>", html)
+
+    def test_furniture_is_muted_never_a_heading(self):
+        from routes.magazine import _render_ingest_preview_html
+        from core.magazine import BLOCK_FURNITURE
+        furniture = self._block("Page 12", font_size=24.0, type=BLOCK_FURNITURE)
+        html = _render_ingest_preview_html([furniture], None)
+        self.assertIn('class="mag-ingest-preview-excluded"', html)
+        self.assertNotIn("<h3", html)
+
+    def test_figure_block_text_is_not_duplicated_as_a_paragraph(self):
+        from routes.magazine import _render_ingest_preview_html
+        from core.magazine import BLOCK_FIGURE
+        figure = self._block("Post your new Instagram photos", type=BLOCK_FIGURE)
+        html = _render_ingest_preview_html([figure], None)
+        self.assertNotIn("Instagram", html)
+
+    def test_an_image_becomes_a_placeholder_not_the_figure_text(self):
+        from routes.magazine import _render_ingest_preview_html
+        html = _render_ingest_preview_html([], [{"bbox": [0, 5, 50, 50]}])
+        self.assertIn("mag-ingest-preview-figure", html)
+
+    def test_text_is_escaped(self):
+        from routes.magazine import _render_ingest_preview_html
+        block = self._block("<script>alert(1)</script> & co")
+        html = _render_ingest_preview_html([block], None)
+        self.assertNotIn("<script>", html)
+        self.assertIn("&lt;script&gt;", html)
+        self.assertIn("&amp; co", html)
+
+    def test_empty_text_is_skipped(self):
+        from routes.magazine import _render_ingest_preview_html
+        blank = self._block("   ")
+        real = self._block("Real text", bbox=[0, 50, 100, 10])
+        html = _render_ingest_preview_html([blank, real], None)
+        self.assertEqual(html.count("<div>"), 1)
+
+    def test_a_flood_of_blocks_is_capped_not_rendered_in_full(self):
+        from routes.magazine import _render_ingest_preview_html, _INGEST_PREVIEW_BUDGET
+        blocks = [self._block("x" * 200, bbox=[0, i, 100, 1]) for i in range(200)]
+        html = _render_ingest_preview_html(blocks, None)
+        self.assertLess(len(html), _INGEST_PREVIEW_BUDGET + 500)
+        self.assertIn("mag-ingest-preview-more", html)
+
+    def test_no_blocks_or_images_is_an_empty_string(self):
+        from routes.magazine import _render_ingest_preview_html
+        self.assertEqual(_render_ingest_preview_html([], None), "")
+        self.assertEqual(_render_ingest_preview_html([], []), "")