feat(magazine): live region boxes in the import preview; interrupted imports no longer spin for ever

The preview carries each page's regions (percent of the page, type, reading
order) for the reading pass and the labelling pass, drawn over the page image
with a legend. A server restart that killed the import left it "running" for
ever; the status route now reports it as failed. v1.42.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

CHANGELOG.md

@@ -7,6 +7,21 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
 
 ## [Unreleased]
 
+## [1.42.0] — 2026-09-22
+
+### Added
+- **Magazine import: watch the regions being found.** The live page preview draws every region over the page as it is recognised — coloured by type, numbered in reading order, with a legend — and follows the second pass (*Recognising paragraphs, headings and advertisements*) page by page, so you can see boxes change type; the progress line shows which page it is on.
+
+### Fixed
+- **An interrupted magazine import spun for ever.** When a server restart killed the import job, the magazine stayed "running" and the page kept showing "Recognising paragraphs…"; it now reports the interruption so the import can be started again.
+- The live preview wrote nothing during the second pass (its regions were in a format the text preview could not read).
+
+## [1.41.1] — 2026-09-22
+
+### Fixed
+- **Magazine: "Could not add region: … unexpected keyword argument 'original_text'".** After *Clean up boxes* split an author line, drawing a region (and re-OCR of a page) crashed on the extra field kept with that author line. The field is now part of the region model and survives every path.
+- **Undo could roll back more than the change it belonged to.** An Undo offered for a delete stayed valid while other changes that record no undo point of their own (drawing a region, fixing the reading order, clean-up, approving) happened afterwards — pressing it would have reverted those too. Undo now refuses as soon as anything else has changed, and those actions clear the Undo button.
+
 ## [1.41.0] — 2026-09-21
 
 ### Changed

Manual.md

@@ -97,7 +97,15 @@ turned into a podcast discussion between two or three voices you pick.
    of watching a bare progress bar. This preview is a quick approximation,
    not the final read: it runs before the layout is fully classified, so a
    heading guess or an excluded marking here can still change once review
-   opens.
+   opens. The page image also shows the **regions as they are found**: each
+   box is drawn over the page, coloured by type (headline, paragraph,
+   caption, picture, advert, running head …, see the legend under the page)
+   and numbered in reading order. After all pages are read, the second pass
+   (*Recognising paragraphs, headings and advertisements*) walks the pages
+   again and the preview follows it, so you can watch boxes change type —
+   the progress bar then reads "… — page 12 of 204". If a run is interrupted
+   (for example when the server restarts), the status now says so instead
+   of spinning forever; start it again with *Re-run detection*.
 3. **Review the articles.** This step is not optional and cannot be skipped.
    Each proposed article shows its title, author, page range, estimated
    listening time and a confidence score. The review screen is split: the

VERSION

@@ -1 +1 @@
-1.41.0
+1.42.0

architecture/magazine-ingestion.md

@@ -347,7 +347,10 @@ membership like the single-region routes. Every destructive route (delete, type
 split, bulk) writes `undo.json` = {token, label, layout, articles} *after* validating and *before* saving
 (the files on disk still hold the old state), and returns the token; `POST .../undo {token}` restores both
 files only if that token is still the latest (409 otherwise), so an old Undo can never roll back a newer
-change. One level. UI: *Select boxes* mode (click or drag a rectangle; a box ≥ 50% inside is selected),
+change. One level. After the change is saved the snapshot is sealed with a digest of layout.json +
+articles.json (`_seal_undo`); undo also requires the current state to match that digest, so a later change that
+records no undo point (drawing, fix reading order, clean-up, approve) makes the old Undo invalid instead of being
+rolled back with it. UI: *Select boxes* mode (click or drag a rectangle; a box ≥ 50% inside is selected),
 a bulk bar, and an Undo bar replacing the per-delete confirmation dialog (per NN/g-style guidance: frequent,
 reversible actions get Undo, not "are you sure?").
 
@@ -368,6 +371,22 @@ image gives the region a new id, and the route moves its **article membership**
 are listed separately on an article), so a converted figure stays in the article being reviewed. The
 page's order is recomputed afterwards (unless hand-ordered), since the type decides where a region is read.
 
+## Live region boxes in the ingest preview
+
+`ingest_preview.json` now carries `stage` ("reading" | "labelling") and `boxes`: each region in percent of the
+page, its type (`page_number` for page-number furniture) and its reading-order number; pictures come from the
+page's images (`_ingest_preview_boxes`, capped at 400). One writer, `_write_ingest_preview`, serves both stages:
+the reader's progress hook (it now also passes the page size) and the labelling passes, which call `on_page`
+after every page in a `try/finally`, so a page whose model call fails is still reported. The labelling pass holds
+blocks as dicts; they are adapted to the attributes the text renderer reads (it previously failed silently on
+dicts and wrote nothing). The browser fits the box layer to the part of the frame the image covers ("contain"
+scaling leaves bars) and re-fits on resize; new boxes fade in (not with reduced motion); the panel stays
+`aria-hidden` -- the status line carries the progress for screen readers.
+
+**Interrupted runs.** A server restart kills the ingest job but left the magazine marked `running`, so the page
+spun for ever. `GET .../ingest/status` now turns `running` + a finished job with an error into `failed` (with the
+error), and the page offers a re-run.
+
 ## Tests
 
 - [`tests/test_magazine.py`](../tests/test_magazine.py) — 100 tests on the pure

core/magazine.py

@@ -116,9 +116,16 @@ class Block:
     region_label: str = ""
     continues_from_previous_page: bool = False
     source: str = "native"          # "native" | "ocr" | "vision_ocr" | "vision_label"
+    # The author line's text before an author line that ran into the first sentence was split
+    # (clean_up_regions). Part of the model so every path that rebuilds blocks keeps it -- and does
+    # not crash on it (drawing a region did, "unexpected keyword argument 'original_text'").
+    original_text: str = ""
 
     def to_json(self) -> dict[str, Any]:
-        return asdict(self)
+        data = asdict(self)
+        if not data["original_text"]:
+            del data["original_text"]
+        return data
 
 
 @dataclass
@@ -2130,6 +2137,9 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110,
                 # one side, recognised text on the other, filled in as each
                 # page finishes) without waiting for the document-wide passes
                 # below.
+                try:
+                    progress(page_no, doc.page_count, blocks, page_images, page_size=(rect.width, rect.height))
+                except TypeError:
                     try:
                         progress(page_no, doc.page_count, blocks, page_images)
                     except TypeError:

routes/magazine.py

@@ -19,6 +19,7 @@ from __future__ import annotations
 import asyncio
 import base64
 import html
+import hashlib
 import json
 import logging
 import re
@@ -119,6 +120,51 @@ async def magazine_create(request: Request):
 _INGEST_PREVIEW_BUDGET = 8000
 
 
+def _ingest_preview_boxes(blocks: list, images: list | None, size: tuple[float, float] | None) -> list[dict]:
+    """The page's regions as boxes in percent of the page, in reading order, for the live preview."""
+    if not size or not size[0] or not size[1]:
+        return []
+    w, h = float(size[0]), float(size[1])
+
+    def get(b, key, default=None):
+        return b.get(key, default) if isinstance(b, dict) else getattr(b, key, default)
+
+    boxes = []
+    ordered = sorted(blocks or [], key=lambda b: (get(b, "reading_order", 0) or 0))
+    for n, b in enumerate(ordered, 1):
+        x, y, bw, bh = (float(v) for v in get(b, "bbox", [0, 0, 0, 0]))
+        label = get(b, "region_label", "") or ""
+        kind = "page_number" if get(b, "type") == mag.BLOCK_FURNITURE and label == "page_number" else get(b, "type", "body")
+        boxes.append({"x": round(100 * x / w, 2), "y": round(100 * y / h, 2), "w": round(100 * bw / w, 2),
+                      "h": round(100 * bh / h, 2), "type": str(kind)[:20], "n": n})
+    for img in images or []:
+        bbox = get(img, "bbox")
+        if not bbox:
+            continue
+        x, y, bw, bh = (float(v) for v in bbox)
+        boxes.append({"x": round(100 * x / w, 2), "y": round(100 * y / h, 2), "w": round(100 * bw / w, 2),
+                      "h": round(100 * bh / h, 2), "type": "image", "n": None})
+    return boxes[:400]
+
+
+def _write_ingest_preview(d: Path, page: int, blocks: list, images: list | None,
+                          size: tuple[float, float] | None, stage: str) -> None:
+    """The live "page being read" preview: the recognised text plus the page's boxes. Best-effort --
+    a write failure never aborts ingestion."""
+    from types import SimpleNamespace
+    # The labelling pass holds blocks as dicts; the text renderer reads attributes.
+    as_objects = [b if not isinstance(b, dict) else SimpleNamespace(
+        bbox=b.get("bbox") or [0, 0, 0, 0], type=b.get("type") or mag.BLOCK_BODY,
+        text=b.get("text") or "", font_size=float(b.get("font_size") or 0.0)) for b in blocks or []]
+    try:
+        (d / "ingest_preview.json").write_text(json.dumps({
+            "page": page, "stage": stage, "text": _render_ingest_preview_html(as_objects, images),
+            "boxes": _ingest_preview_boxes(blocks, images, size),
+        }, ensure_ascii=False), "utf-8")
+    except Exception:
+        logging.exception("Failed to write live ingest preview for page %s", page)
+
+
 def _render_ingest_preview_html(blocks: list, images: list | None) -> str:
     """Format one page's just-read blocks as simple HTML, in reading order.
 
@@ -189,7 +235,8 @@ def _ingest_sync(doc_id: str, job_id: str, ocr_lang: str, render_dpi: int,
         if not pdf.is_file():
             raise FileNotFoundError("No source.pdf uploaded for this magazine")
 
-        def progress(page: int, total: int, blocks: list, images: list | None = None) -> None:
+        def progress(page: int, total: int, blocks: list, images: list | None = None,
+                     page_size: tuple[float, float] | None = None) -> None:
             background_job_update(
                 job_id, stage="extract",
                 detail=f"Analysing page {page} of {total}",
@@ -197,15 +244,8 @@ def _ingest_sync(doc_id: str, job_id: str, ocr_lang: str, render_dpi: int,
             # Written to its own small file rather than the generic job
             # result: that field strips any key resembling "text"/"content"
             # (data-minimisation for other job kinds) and caps at 500 chars,
-            # both wrong for this. Best-effort -- a write failure must never
+            # both wrong for this.
-            # abort ingestion.
+            _write_ingest_preview(d, page, blocks, images, page_size, "reading")
-            try:
-                preview_html = _render_ingest_preview_html(blocks, images)
-                (d / "ingest_preview.json").write_text(
-                    json.dumps({"page": page, "text": preview_html}, ensure_ascii=False),
-                    "utf-8")
-            except Exception:
-                logging.exception("Failed to write live ingest preview for page %s", page)
 
         background_job_update(job_id, stage="extract", detail="Opening PDF")
         layout = mag.extract_pages(pdf, d, render_dpi=render_dpi, ocr_lang=ocr_lang,
@@ -215,9 +255,21 @@ def _ingest_sync(doc_id: str, job_id: str, ocr_lang: str, render_dpi: int,
                                    vision_ocr_fallback_model=vision_ocr_fallback_model,
                                    force_vision_ocr=force_vision_ocr,
                                    progress=progress)
+        page_sizes = {p["page"]: (p.get("width"), p.get("height")) for p in layout.get("pages", [])}
+        total_pages = len(page_sizes)
+
+        def labelled(step: str):
+            def on_page(page_no: int) -> None:
+                background_job_update(job_id, stage="advertisements",
+                                      detail=f"{step} — page {page_no} of {total_pages}")
+                _write_ingest_preview(d, page_no, [b for b in layout.get("blocks", []) if b["page"] == page_no],
+                                      [i for i in layout.get("images", []) if i["page"] == page_no],
+                                      page_sizes.get(page_no), "labelling")
+            return on_page
+
         background_job_update(job_id, stage="advertisements", detail="Recognising paragraphs, headings and advertisements")
-        _recognise_advertisements(layout, _load_settings())
+        _recognise_advertisements(layout, _load_settings(), on_page=labelled("Finding advertisements"))
-        _recognise_semantic_layout(layout, _load_settings(), doc_dir=d)
+        _recognise_semantic_layout(layout, _load_settings(), doc_dir=d, on_page=labelled("Recognising paragraphs and headings"))
         mag.refresh_visual_regions(d, layout)
         layout, _ = mag.clean_up_regions(layout)
         mag.save_layout(d, layout)
@@ -317,6 +369,13 @@ async def magazine_ingest_status(doc_id: str):
     d, meta = _require_magazine(doc_id)
     job_id = (meta.get("ingest") or {}).get("job_id")
     job = background_job_get(job_id) if job_id else None
+    # A job that ended without finishing the ingest (a server restart kills it mid-run) left the
+    # magazine marked "running" for ever, and the page spun on "Recognising paragraphs…" with no
+    # way to see it had stopped. Report it as failed so it can be started again.
+    ingest = meta.get("ingest") or {}
+    if ingest.get("status") == "running" and job and job.get("done") and job.get("error"):
+        meta["ingest"] = {"status": "failed", "job_id": job_id, "error": str(job.get("error"))[:300]}
+        _write_meta(d, meta)
     preview = None
     try:
         preview_path = d / "ingest_preview.json"
@@ -469,7 +528,7 @@ async def magazine_page_image(doc_id: str, page: int):
     return FileResponse(png, media_type="image/png")
 
 
-def _recognise_advertisements(layout: dict, settings: dict, page: int | None = None) -> None:
+def _recognise_advertisements(layout: dict, settings: dict, page: int | None = None, on_page=None) -> None:
     """Use the configured LLM to identify advertising, region by region."""
     from routes.conversation import _post_llm_chat_completion
     model = str(settings.get("llm_model") or "").strip()
@@ -479,6 +538,7 @@ def _recognise_advertisements(layout: dict, settings: dict, page: int | None = N
     pages = sorted({b["page"] for b in layout.get("blocks", []) if page is None or b["page"] == page})
     failures = 0
     for page_no in pages:
+        try:
             blocks = [b for b in layout.get("blocks", []) if b["page"] == page_no]
             for start in range(0, len(blocks), 40):
                 batch = blocks[start:start + 40]
@@ -520,6 +580,9 @@ def _recognise_advertisements(layout: dict, settings: dict, page: int | None = N
                         block.update(type=mag.BLOCK_AD, source="llm_ad", confidence=confidence)
                 except Exception:
                     failures += 1
+        finally:
+            if on_page:
+                on_page(page_no)
     ads = {b["id"] for b in layout.get("blocks", []) if b["type"] == mag.BLOCK_AD}
     for image in layout.get("images", []):
         if image.get("source_block") in ads:
@@ -527,7 +590,8 @@ def _recognise_advertisements(layout: dict, settings: dict, page: int | None = N
     layout["advertisement_review"] = "partial_failure" if failures else "done"
 
 
-def _recognise_semantic_layout(layout: dict, settings: dict, page: int | None = None, doc_dir: Path | None = None) -> None:
+def _recognise_semantic_layout(layout: dict, settings: dict, page: int | None = None, doc_dir: Path | None = None,
+                               on_page=None) -> None:
     from routes.conversation import _post_llm_chat_completion
     # OCR transcription and page-layout interpretation are different jobs.
     # The latter needs a vision-language model because a column boundary,
@@ -542,6 +606,7 @@ def _recognise_semantic_layout(layout: dict, settings: dict, page: int | None =
     failures = 0
     pages = sorted({b["page"] for b in layout.get("blocks", []) if page is None or b["page"] == page})
     for page_no in pages:
+        try:
             blocks = [b for b in layout["blocks"] if b["page"] == page_no]
             content = [b for b in blocks if b["type"] != mag.BLOCK_FURNITURE and b.get("text", "").strip()]
             ad_blocks = [b for b in content if b["type"] == mag.BLOCK_AD]
@@ -628,6 +693,9 @@ def _recognise_semantic_layout(layout: dict, settings: dict, page: int | None =
                     layout["blocks"] = [b for b in layout["blocks"] if b["id"] not in ids] + grouped
                 except Exception:
                     failures += 1
+        finally:
+            if on_page:
+                on_page(page_no)
     status = "partial_failure" if failures else "done"
     layout["semantic_review"] = status
     layout["advertisement_review"] = status
@@ -756,13 +824,36 @@ def _add_split_paragraphs_to_articles(articles: list[dict], report: dict) -> boo
 
 def _snapshot_for_undo(d: Path, label: str) -> str:
     """Keep the layout and articles as they are now, so the change about to happen can be undone.
-    One level: the next snapshot replaces this one. Returns the token the client undoes with."""
+    One level: the next snapshot replaces this one. Returns the token the client undoes with.
+    The caller seals it with _seal_undo once the change is saved."""
     token = uuid.uuid4().hex
     snapshot = {"token": token, "label": label, "layout": mag.load_layout(d), "articles": mag.load_articles(d)}
     (d / "undo.json").write_text(json.dumps(snapshot, ensure_ascii=False), "utf-8")
     return token
 
 
+def _state_digest(d: Path) -> str:
+    """A fingerprint of the saved layout and articles."""
+    h = hashlib.sha256()
+    for name in ("layout.json", "articles.json"):
+        path = d / name
+        h.update(path.read_bytes() if path.is_file() else b"-")
+    return h.hexdigest()
+
+
+def _seal_undo(d: Path | None, token: str | None) -> None:
+    """Record what the state looks like right after the undoable change. Undo is refused once the
+    state differs from that -- any later change (drawing a region, fixing the reading order,
+    approving) would otherwise be rolled back silently with it."""
+    path = d / "undo.json" if d is not None and token else None
+    if path is None or not path.is_file():
+        return
+    snapshot = json.loads(path.read_text("utf-8"))
+    if snapshot.get("token") == token:
+        snapshot["after"] = _state_digest(d)
+        path.write_text(json.dumps(snapshot, ensure_ascii=False), "utf-8")
+
+
 @router.post("/api/magazine/{doc_id}/undo")
 async def magazine_undo(doc_id: str, request: Request):
     """Undo the last region change (delete, type change, merge, split, bulk). Body: {token}.
@@ -774,7 +865,7 @@ async def magazine_undo(doc_id: str, request: Request):
     if not isinstance(token, str) or not path.is_file():
         raise HTTPException(409, "Nothing to undo")
     snapshot = json.loads(path.read_text("utf-8"))
-    if snapshot.get("token") != token:
+    if snapshot.get("token") != token or snapshot.get("after") != _state_digest(d):
         raise HTTPException(409, "Something else has changed since -- this can no longer be undone")
     mag.save_layout(d, snapshot["layout"])
     mag.save_articles(d, snapshot["articles"])
@@ -840,6 +931,7 @@ async def magazine_bulk_regions(doc_id: str, page: int, request: Request):
                     changed = True
     if changed:
         mag.save_articles(d, articles)
+    _seal_undo(d, token)
     return {"ok": True, "undo": token, "remap": remap, "count": len(ids)}
 
 
@@ -866,6 +958,7 @@ async def magazine_merge_region(doc_id: str, page: int, block_id: str):
             changed = True
     if changed:
         mag.save_articles(d, articles)
+    _seal_undo(d, token)
     return {"ok": True, "id": block_id, "removed": removed, "undo": token}
 
 
@@ -890,6 +983,7 @@ async def magazine_split_region(doc_id: str, page: int, block_id: str, request:
     articles = mag.load_articles(d)
     if _add_split_paragraphs_to_articles(articles, {"byline_split": [{"id": block_id, "new_id": new_id}]}):
         mag.save_articles(d, articles)
+    _seal_undo(d, token)
     return {"ok": True, "id": block_id, "new_id": new_id, "undo": token}
 
 
@@ -1095,6 +1189,7 @@ async def magazine_update_region(doc_id: str, page: int, block_id: str, request:
     mag.save_layout(d, layout)
     if region_id != block_id:
         _move_article_membership(d, block_id, region_id, to_image=new_type == "image")
+    _seal_undo(d, token)
     return {"ok": True, "text": text, "id": region_id, "type": new_type, "undo": token}
 
 
@@ -1142,6 +1237,7 @@ async def magazine_delete_region(doc_id: str, page: int, block_id: str):
                 changed = True
     if changed:
         mag.save_articles(d, articles)
+    _seal_undo(d, token)
     return {"ok": True, "undo": token}
 
 

static/index.html

@@ -27,12 +27,12 @@
 
   <!-- ── Core styles (local — no CDN dependency for first paint) ────────── -->
   <link rel="stylesheet" href="/static/vendor/mdi/materialdesignicons.min.css">
-<link rel="stylesheet" href="/static/style.css?v=1.41.0">
+<link rel="stylesheet" href="/static/style.css?v=1.42.0">
-<link rel="stylesheet" href="/static/css/resource-diagnostics.css?v=1.41.0">
+<link rel="stylesheet" href="/static/css/resource-diagnostics.css?v=1.42.0">
-<link rel="stylesheet" href="/static/css/background-jobs.css?v=1.41.0">
+<link rel="stylesheet" href="/static/css/background-jobs.css?v=1.42.0">
-<link rel="stylesheet" href="/static/css/voice-design.css?v=1.41.0">
+<link rel="stylesheet" href="/static/css/voice-design.css?v=1.42.0">
-<link rel="stylesheet" href="/static/css/rehearser.css?v=1.41.0">
+<link rel="stylesheet" href="/static/css/rehearser.css?v=1.42.0">
-<link rel="stylesheet" href="/static/css/audiobook.css?v=1.41.0">
+<link rel="stylesheet" href="/static/css/audiobook.css?v=1.42.0">
 
 
   <!-- ── Flag icons — non-blocking (loaded async, icons appear after JS) ── -->

static/js/magazine.js

@@ -458,6 +458,60 @@ function magParsePageProgress(detail) {
   return { page, total, pct: Math.min(100, Math.round((page / total) * 100)) };
 }
 
+// ── Live boxes on the page being read ─────────────────────────────────────
+// The regions of the page currently being read (and, in the labelling pass, being re-typed) are
+// drawn over the page image as they arrive, coloured by type and numbered in reading order.
+
+const MAG_PREVIEW_TYPES = {
+  title: 'Headline', heading: 'Subheading', deck: 'Standfirst', byline: 'Author', body: 'Paragraph',
+  caption: 'Caption', pullquote: 'Pull quote', sidebar: 'Sidenote', ad: 'Advertisement',
+  furniture: 'Running head / footer', page_number: 'Page number', image: 'Picture',
+  figure: 'Picture', chart: 'Chart', table: 'Table', code: 'Code',
+};
+let _magPreviewSeen = { page: null, keys: new Set() };
+
+// Fit the box layer to the part of the frame the page image actually covers ("contain" scaling
+// leaves bars around it), so a box drawn at 50% sits at 50% of the page, not of the frame.
+function magFitPreviewLayer() {
+  const img = $('mag-ingest-preview-img'), layer = $('mag-ingest-preview-boxes');
+  if (!img || !layer || !img.naturalWidth || !img.clientWidth) return;
+  const scale = Math.min(img.clientWidth / img.naturalWidth, img.clientHeight / img.naturalHeight);
+  const w = img.naturalWidth * scale, h = img.naturalHeight * scale;
+  layer.style.left = (img.offsetLeft + (img.clientWidth - w) / 2) + 'px';
+  layer.style.top = (img.offsetTop + (img.clientHeight - h) / 2) + 'px';
+  layer.style.width = w + 'px';
+  layer.style.height = h + 'px';
+}
+
+function magRenderPreviewBoxes(page, boxes, stage) {
+  const layer = $('mag-ingest-preview-boxes'), img = $('mag-ingest-preview-img');
+  if (!layer) return;
+  const stageEl = $('mag-ingest-preview-stage');
+  if (stageEl) stageEl.textContent = page ? '· ' + page + (stage === 'labelling' ? ' — labelling regions' : ' — reading') : '';
+  if (_magPreviewSeen.page !== page) _magPreviewSeen = { page, keys: new Set() };
+  const key = (b) => [b.x, b.y, b.w, b.h].join(',');
+  layer.innerHTML = boxes.map((b) => {
+    const isNew = !_magPreviewSeen.keys.has(key(b));
+    const type = MAG_PREVIEW_TYPES[b.type] ? b.type : 'body';
+    return '<div class="mag-preview-box mpt-' + type + (isNew ? ' is-new' : '') + '" style="left:' + b.x + '%;top:' + b.y +
+      '%;width:' + b.w + '%;height:' + b.h + '%" title="' + esc(MAG_PREVIEW_TYPES[type]) + '">' +
+      (b.n ? '<span class="mag-preview-box-num">' + b.n + '</span>' : '') + '</div>';
+  }).join('');
+  boxes.forEach((b) => _magPreviewSeen.keys.add(key(b)));
+  const legend = $('mag-ingest-preview-legend');
+  if (legend) {
+    const present = [...new Set(boxes.map((b) => (MAG_PREVIEW_TYPES[b.type] ? b.type : 'body')))];
+    legend.innerHTML = present.map((t) => '<span class="mag-preview-legend-item"><span class="mag-preview-swatch mpt-' + t + '"></span>' +
+      esc(MAG_PREVIEW_TYPES[t]) + '</span>').join('') + (boxes.length ? '<span class="mag-preview-legend-count">' + boxes.length + ' regions</span>' : '');
+  }
+  if (img && !img.dataset.fitBound) {
+    img.dataset.fitBound = '1';
+    img.addEventListener('load', magFitPreviewLayer);
+    if (typeof ResizeObserver === 'function') new ResizeObserver(magFitPreviewLayer).observe(img);
+  }
+  magFitPreviewLayer();
+}
+
 async function pollIngest() {
   const bar = $('mag-ingest-status');
   const preview = $('mag-ingest-preview');
@@ -504,6 +558,7 @@ async function pollIngest() {
           const img = $('mag-ingest-preview-img');
           if (img) img.src = '/api/magazine/' + state.doc.id + '/pages/' + data.preview.page;
         }
+        magRenderPreviewBoxes(data.preview.page, data.preview.boxes || [], data.preview.stage);
         const textEl = $('mag-ingest-preview-text');
         // Server-built HTML (see _render_ingest_preview_html in
         // routes/magazine.py) -- every text node in it is already escaped,
@@ -1217,6 +1272,7 @@ async function magOrderPut(order) {
 }
 
 async function magOrderSave() {
+  magOfferUndo(null);   // this change records no undo point, so an older Undo no longer applies
   const seq = (state.orderSeq || []).slice();
   if (!seq.length) return;
   try {
@@ -1231,6 +1287,7 @@ async function magOrderSave() {
 }
 
 async function magOrderMove(id, value) {
+  magOfferUndo(null);   // this change records no undo point, so an older Undo no longer applies
   const order = (state.reviewRegionOrder || []).filter((x) => x !== id);
   const to = Math.min(Math.max(1, parseInt(value, 10) || 1), order.length + 1) - 1;
   order.splice(to, 0, id);
@@ -1245,6 +1302,7 @@ async function magOrderMove(id, value) {
 }
 
 async function magOrderReset() {
+  magOfferUndo(null);   // this change records no undo point, so an older Undo no longer applies
   try {
     await api('/api/magazine/' + state.doc.id + '/pages/' + state.reviewPage + '/reading-order', { method: 'DELETE' });
   } catch (e) {
@@ -1596,6 +1654,7 @@ async function magReviewSubmitEdit(reocr) {
 }
 
 async function magCleanUpBoxes() {
+  magOfferUndo(null);   // this change records no undo point, so an older Undo no longer applies
   const ok = await confirmDialog('Clean up the boxes of this whole issue: text inside pictures, OCR noise (boxes of one or two characters) ' +
     'and boxes that repeat a bigger box\'s text stop being read aloud, and author lines that run into the first sentence are split. ' +
     'Nothing is deleted — every change shows in the review list and can be undone with the type picker — and a backup of the layout is kept.',
@@ -1620,6 +1679,7 @@ async function magCleanUpBoxes() {
 }
 
 async function magFixReadingOrder() {
+  magOfferUndo(null);   // this change records no undo point, so an older Undo no longer applies
   const ok = await confirmDialog('Re-read the reading order of every page in this issue, column by column. ' +
     'Text, boxes and approvals stay as they are, but the order in which approved articles are read aloud can change. ' +
     'Pages you ordered by hand are kept.', { title: 'Fix reading order?', okLabel: 'Fix reading order' });
@@ -1911,6 +1971,7 @@ function magReviewOpenDrawPanel(box) {
 }
 
 async function magReviewSubmitDraw() {
+  magOfferUndo(null);   // this change records no undo point, so an older Undo no longer applies
   const box = $('mag-draw-rect');
   const pageMeta = (state.layout.pages || []).find((p) => p.page === state.reviewPage);
   if (!box || !pageMeta) return;
@@ -2020,6 +2081,7 @@ function selectedBlockIds() {
 }
 
 async function magReocrPage() {
+  magOfferUndo(null);   // this change records no undo point, so an older Undo no longer applies
   const page = state.reviewPage;
   if (page == null) return;
   const btn = $('mag-reocr-btn');
@@ -2102,6 +2164,7 @@ async function magRefineFigures() {
 }
 
 async function magApprove() {
+  magOfferUndo(null);   // this change records no undo point, so an older Undo no longer applies
   const a = state.article;
   if (!a) return;
   const blocks = selectedBlockIds();
@@ -2865,6 +2928,7 @@ root.magSelectToggle = magSelectToggle;
 root.magSelectNone = magSelectNone;
 root.magBulk = magBulk;
 root.magUndo = magUndo;
+root.magRenderPreviewBoxes = magRenderPreviewBoxes;
 root.magRowReocr = magRowReocr;
 root.magRowMerge = magRowMerge;
 root.magListKey = magListKey;

static/sections/s-magazine.html

@@ -76,8 +76,12 @@
              needs; this panel adds nothing beyond that but noisy churn. -->
         <div id="mag-ingest-preview" class="mag-ingest-preview" hidden aria-hidden="true">
           <div class="mag-ingest-preview-side">
-            <span class="mag-ingest-preview-label">Page</span>
+            <span class="mag-ingest-preview-label">Page <span id="mag-ingest-preview-stage"></span></span>
+            <div class="mag-ingest-preview-canvas">
               <img id="mag-ingest-preview-img" alt="Page currently being read">
+              <div id="mag-ingest-preview-boxes" class="mag-ingest-preview-boxes"></div>
+            </div>
+            <div id="mag-ingest-preview-legend" class="mag-ingest-preview-legend"></div>
           </div>
           <div class="mag-ingest-preview-resize" id="mag-ingest-preview-resize" role="separator" aria-orientation="vertical" aria-label="Resize page and recognised text panes" title="Drag to resize page and recognised text panes"></div>
           <div class="mag-ingest-preview-side">

static/style.css

@@ -5189,3 +5189,22 @@ input[list="lang-codes"] {
 .mag-review-box.is-selected { outline: 3px solid var(--warning, #d97706); outline-offset: -1px;
   background: color-mix(in srgb, var(--warning, #d97706) 18%, transparent); }
 .mag-blocks [data-block-row].is-selected { border-color: var(--warning, #d97706); box-shadow: inset 3px 0 0 var(--warning, #d97706); }
+
+/* Live region boxes on the page being read (magazine ingestion preview) */
+.mag-ingest-preview-canvas { position: relative; flex: 1 1 auto; min-height: 0; display: flex; }
+.mag-ingest-preview-canvas img { width: 100%; height: 100%; object-fit: contain; }
+.mag-ingest-preview-boxes { position: absolute; pointer-events: none; }
+.mag-preview-box { position: absolute; border: 2px solid var(--c, #3b82f6); border-radius: 2px;
+  background: color-mix(in srgb, var(--c, #3b82f6) 14%, transparent); }
+.mag-preview-box.is-new { animation: mag-preview-in .45s ease-out; }
+@keyframes mag-preview-in { from { opacity: 0; transform: scale(1.04); } to { opacity: 1; transform: none; } }
+@media (prefers-reduced-motion: reduce) { .mag-preview-box.is-new { animation: none; } }
+.mag-preview-box-num { position: absolute; left: -2px; top: -2px; min-width: 16px; padding: 0 3px; font: 600 10px/16px system-ui, sans-serif;
+  color: #fff; background: var(--c, #3b82f6); border-radius: 2px 0 2px 0; text-align: center; }
+.mag-ingest-preview-legend { display: flex; flex-wrap: wrap; gap: 4px 10px; font-size: 11px; color: #b6bac2; }
+.mag-preview-legend-item { display: inline-flex; align-items: center; gap: 4px; }
+.mag-preview-swatch { width: 10px; height: 10px; border-radius: 2px; background: var(--c); }
+.mag-preview-legend-count { margin-left: auto; color: #8a8f98; }
+.mpt-title { --c: #a855f7; } .mpt-heading { --c: #d8b4fe; } .mpt-deck { --c: #06b6d4; } .mpt-byline { --c: #10b981; } .mpt-body { --c: #3b82f6; }
+.mpt-caption { --c: #14b8a6; } .mpt-pullquote { --c: #ec4899; } .mpt-sidebar { --c: #f59e0b; } .mpt-ad { --c: #ef4444; }
+.mpt-furniture, .mpt-page_number { --c: #9ca3af; } .mpt-image, .mpt-figure, .mpt-chart, .mpt-table, .mpt-code { --c: #f97316; }

tests/magazine.spec.js

@@ -629,3 +629,55 @@ test('clicking boxes in select mode builds a selection whose type changes at onc
   await page.locator('#mag-bulk-type').selectOption('ocr_noise');
   await expect.poll(() => sent).toEqual({ ids: ['b1', 'b2'], action: 'type', type: 'ocr_noise' });
 });
+
+test('an action without its own undo point clears the Undo button', async ({ page }) => {
+  await mockBackend(page);
+  await page.route(`**/api/magazine/${DOC_ID}/pages/3/regions/b3`, route => route.fulfill({ json: { ok: true, undo: 'tok-9' } }));
+  await page.route(`**/api/magazine/${DOC_ID}/reading-order`, route => route.fulfill({ json: { ok: true, pages_changed: [], pages_locked: 0 } }));
+  await openReview(page);
+  await page.locator('[data-block-row="b3"]').getByRole('button', { name: /Delete region/ }).click();
+  await expect(page.locator('#mag-undo-bar')).toContainText('Region deleted.');
+  await page.getByRole('button', { name: 'Fix reading order', exact: true }).click();
+  await page.getByRole('alertdialog').getByRole('button', { name: 'Fix reading order', exact: true }).click();
+  await expect(page.locator('#mag-undo-bar')).toBeEmpty();
+});
+
+test('the live preview draws the page regions over the page image as they arrive', async ({ page }) => {
+  await mockBackend(page);
+  await page.goto('/', { waitUntil: 'domcontentloaded' });
+  await page.waitForSelector('#app-skeleton', { state: 'detached' });
+  await navTo(page, 's-magazine');
+  await page.locator('#mag-library-list .mag-card').first().click();
+  // A 100x200 page image in a frame much wider than the page: the boxes must follow the image,
+  // not the frame (contain scaling leaves bars left and right).
+  await page.evaluate(async () => {
+    const preview = document.getElementById('mag-ingest-preview');
+    preview.hidden = false;
+    preview.style.width = '800px';
+    const canvas = document.createElement('canvas'); canvas.width = 100; canvas.height = 200;
+    const img = document.getElementById('mag-ingest-preview-img');
+    img.style.height = '400px';
+    await new Promise(r => { img.onload = r; img.src = canvas.toDataURL('image/png'); });
+  });
+  await page.evaluate(() => window.magRenderPreviewBoxes(5, [
+    { x: 0, y: 0, w: 100, h: 10, type: 'title', n: 1 },
+    { x: 0, y: 20, w: 50, h: 50, type: 'body', n: 2 },
+  ], 'reading'));
+  await expect(page.locator('#mag-ingest-preview-boxes .mag-preview-box')).toHaveCount(2);
+  await expect(page.locator('#mag-ingest-preview-stage')).toContainText('5 — reading');
+  await expect(page.locator('#mag-ingest-preview-legend')).toContainText('Headline');
+  await expect(page.locator('#mag-ingest-preview-legend')).toContainText('2 regions');
+  const [imgBox, layerBox] = await Promise.all([
+    page.locator('#mag-ingest-preview-img').boundingBox(), page.locator('#mag-ingest-preview-boxes').boundingBox()]);
+  // The layer has the page's 1:2 shape and is centred in the wider frame.
+  expect(Math.abs(layerBox.height - 2 * layerBox.width)).toBeLessThan(2);
+  expect(Math.abs((layerBox.x + layerBox.width / 2) - (imgBox.x + imgBox.width / 2))).toBeLessThan(2);
+  // A region that appears on the next poll is marked new; the ones already shown are not.
+  await page.evaluate(() => window.magRenderPreviewBoxes(5, [
+    { x: 0, y: 0, w: 100, h: 10, type: 'title', n: 1 },
+    { x: 0, y: 20, w: 50, h: 50, type: 'body', n: 2 },
+    { x: 50, y: 20, w: 50, h: 50, type: 'ad', n: 3 },
+  ], 'labelling'));
+  await expect(page.locator('#mag-ingest-preview-boxes .mag-preview-box.is-new')).toHaveCount(1);
+  await expect(page.locator('#mag-ingest-preview-stage')).toContainText('labelling regions');
+});

tests/test_magazine_routes.py

@@ -6,6 +6,7 @@ import os
 import shutil
 import tempfile
 import unittest
+from pathlib import Path
 
 _CONFIG = tempfile.mkdtemp(prefix="ttvc-magazine-routes-test-")
 os.environ.setdefault("CONFIG_DIR", _CONFIG)
@@ -1123,3 +1124,123 @@ class BulkUndoFoldTests(unittest.TestCase):
                 self.assertEqual(err.exception.status_code, 400)
         self.assertEqual(len(mag.load_layout(self.dir)["blocks"]), 4)
         self.assertFalse((self.dir / "undo.json").exists())
+
+
+class UndoSafetyAndExtraFieldTests(BulkUndoFoldTests):
+    """Undo never rolls back a later change; a split author line's extra field survives every path."""
+
+    def test_undo_is_refused_once_anything_else_changed(self):
+        from core import magazine as mag
+        from routes.magazine import magazine_bulk_regions, magazine_undo
+        out = self.run_route(magazine_bulk_regions, 1, body={"ids": ["lg1"], "action": "delete"})
+        # A change that records no undo point of its own, e.g. drawing a region or fixing the order.
+        layout = mag.load_layout(self.dir)
+        layout["blocks"][0]["text"] = "edited later"
+        mag.save_layout(self.dir, layout)
+        with self.assertRaises(HTTPException) as err:
+            self.run_route(magazine_undo, body={"token": out["undo"]})
+        self.assertEqual(err.exception.status_code, 409)
+        self.assertEqual(mag.load_layout(self.dir)["blocks"][0]["text"], "edited later")
+
+    def test_an_unsealed_old_snapshot_is_refused(self):
+        import json
+        from routes.magazine import magazine_undo
+        from core import magazine as mag
+        (self.dir / "undo.json").write_text(json.dumps({"token": "t", "label": "x", "layout": mag.load_layout(self.dir),
+                                                        "articles": mag.load_articles(self.dir)}), "utf-8")
+        with self.assertRaises(HTTPException):
+            self.run_route(magazine_undo, body={"token": "t"})
+
+    def test_drawing_a_region_on_a_layout_with_a_split_author_line_works(self):
+        """"Could not add region: Block.__init__() got an unexpected keyword argument 'original_text'"."""
+        from core.magazine import add_manual_region, clean_up_regions
+        self.layout["blocks"].append({"id": "by", "page": 1, "bbox": [50, 20, 300, 30], "type": "byline",
+                                      "text": "Von Jo Bager KI gilt als Antwort", "reading_order": 9})
+        cleaned, report = clean_up_regions(self.layout)
+        self.assertEqual(len(report["byline_split"]), 1)
+        after = add_manual_region(self.dir, cleaned, 1, [400, 700, 100, 40], "text", "New text")
+        byline = next(b for b in after["blocks"] if b["id"] == "by")
+        self.assertIn("KI gilt als Antwort", byline["original_text"])
+        self.assertTrue(any(b["text"] == "New text" for b in after["blocks"]))
+
+    def test_blocks_without_the_field_do_not_gain_it(self):
+        from core.magazine import Block
+        self.assertNotIn("original_text", Block(id="x", page=1, bbox=[0, 0, 1, 1], type="body", text="t").to_json())
+
+
+class LivePreviewBoxTests(unittest.TestCase):
+    """The ingest preview carries the page's boxes, so they can be drawn as they are made."""
+
+    def test_boxes_are_in_percent_of_the_page_and_numbered_in_reading_order(self):
+        from core.magazine import Block
+        from routes.magazine import _ingest_preview_boxes
+        blocks = [Block(id="b", page=1, bbox=[300, 400, 150, 100], type="body", text="t", reading_order=1),
+                  Block(id="a", page=1, bbox=[0, 0, 600, 80], type="title", text="t", reading_order=0),
+                  Block(id="n", page=1, bbox=[280, 780, 40, 10], type="furniture", text="12", reading_order=2,
+                        region_label="page_number")]
+        boxes = _ingest_preview_boxes(blocks, [{"bbox": [60, 100, 240, 300]}], (600, 800))
+        self.assertEqual([(b["type"], b["n"]) for b in boxes], [("title", 1), ("body", 2), ("page_number", 3), ("image", None)])
+        self.assertEqual({k: boxes[1][k] for k in ("x", "y", "w", "h")}, {"x": 50.0, "y": 50.0, "w": 25.0, "h": 12.5})
+
+    def test_dict_blocks_from_the_labelling_pass_work_too(self):
+        from routes.magazine import _ingest_preview_boxes
+        boxes = _ingest_preview_boxes([{"bbox": [0, 0, 100, 100], "type": "ad", "reading_order": 0}], [], (200, 200))
+        self.assertEqual(boxes[0]["type"], "ad")
+
+    def test_no_page_size_means_no_boxes_rather_than_an_error(self):
+        from routes.magazine import _ingest_preview_boxes
+        self.assertEqual(_ingest_preview_boxes([{"bbox": [0, 0, 1, 1], "type": "body"}], [], None), [])
+
+    def test_the_preview_file_holds_page_stage_text_and_boxes(self):
+        import json
+        from pathlib import Path
+        from routes.magazine import _write_ingest_preview
+        d = Path(tempfile.mkdtemp(prefix="ttvc-preview-"))
+        try:
+            _write_ingest_preview(d, 7, [{"bbox": [0, 0, 50, 50], "type": "body", "text": "Hallo", "reading_order": 0}],
+                                  [], (100, 100), "labelling")
+            data = json.loads((d / "ingest_preview.json").read_text("utf-8"))
+            self.assertEqual((data["page"], data["stage"], len(data["boxes"])), (7, "labelling", 1))
+            self.assertIn("Hallo", data["text"])
+        finally:
+            shutil.rmtree(d, ignore_errors=True)
+
+    def test_the_labelling_pass_reports_every_page_even_when_its_model_call_fails(self):
+        from unittest import mock
+        import routes.magazine as rm
+        layout = {"pages": [{"page": 1, "width": 1, "height": 1}, {"page": 2, "width": 1, "height": 1}],
+                  "blocks": [{"id": "a", "page": 1, "bbox": [0, 0, 1, 1], "type": "body", "text": "x"},
+                             {"id": "b", "page": 2, "bbox": [0, 0, 1, 1], "type": "body", "text": "y"}], "images": []}
+        seen = []
+        with mock.patch("routes.conversation._post_llm_chat_completion", side_effect=RuntimeError("down")):
+            rm._recognise_advertisements(layout, {"llm_model": "m", "llm_url": "http://x"}, on_page=seen.append)
+        self.assertEqual(seen, [1, 2])
+
+
+class InterruptedIngestTests(unittest.TestCase):
+    def test_an_ingest_killed_by_a_restart_is_reported_as_failed_not_running_for_ever(self):
+        import asyncio
+        from unittest.mock import patch
+        from routes.magazine import magazine_ingest_status
+        meta = {"kind": "magazine", "ingest": {"status": "running", "job_id": "j1"}}
+        written = {}
+        with patch('routes.magazine._require_magazine', return_value=(Path(tempfile.mkdtemp()), meta)), \
+             patch('routes.magazine.background_job_get', return_value={"done": True, "stage": "error",
+                                                                      "error": "Background job interrupted by server restart"}), \
+             patch('routes.magazine._write_meta', side_effect=lambda d, m: written.update(m)):
+            out = asyncio.run(magazine_ingest_status('a' * 32))
+        self.assertEqual(out["ingest"]["status"], "failed")
+        self.assertIn("server restart", out["ingest"]["error"])
+        self.assertEqual(written["ingest"]["status"], "failed")
+
+    def test_a_job_still_running_stays_running(self):
+        import asyncio
+        from unittest.mock import patch
+        from routes.magazine import magazine_ingest_status
+        meta = {"kind": "magazine", "ingest": {"status": "running", "job_id": "j1"}}
+        with patch('routes.magazine._require_magazine', return_value=(Path(tempfile.mkdtemp()), meta)), \
+             patch('routes.magazine.background_job_get', return_value={"done": False, "stage": "extract"}), \
+             patch('routes.magazine._write_meta') as write:
+            out = asyncio.run(magazine_ingest_status('a' * 32))
+        self.assertEqual(out["ingest"]["status"], "running")
+        write.assert_not_called()