feat: OCR benchmark harness and first candidate results (docling, PaddleOCR)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 17:55:48 +02:00 · 2026-09-29 17:55:48 +02:00 · 35e0a64665
commit 35e0a64665
parent c3c5c55f17
7 changed files with 576 additions and 4 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -9,6 +9,7 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
 ### Fixed
 - **Voice benchmark failures now say why.** Designed voices are benchmarked through the Voice Design engine; when that service is down the row showed a bare "-" and the raw "Max retries exceeded" text. The error now names the service and the container to start, the Factor and WPM columns show `ERR` beside the time column, and a batch stops after three identical "unreachable" failures instead of retrying every remaining voice.
 ### Added
+- `scripts/bench_ocr.py`: engine-agnostic OCR benchmark that scores an engine's transcription of rendered pages against a native-text PDF's own text layer (word F1, sequence similarity, seconds per page). First results (tesseract, Unlimited-OCR, docling, PaddleOCR) in `architecture/evidence/ocr-bench-2026-09-29/`: transcription accuracy is equal within noise (F1 0.865-0.881), tesseract is 4-20x faster; DeepSeek-OCR and Qwen3-VL-30B were not run (no memory headroom).
 - **Find an issue online (magazines).** A search panel in the magazine library finds issues in the Internet Archive and in any OPDS catalogue you configure (Calibre-Web, Kavita, Komga), shows the source's licence, and imports the PDF straight into a new magazine project, then starts ingestion. SSRF-safe (downloads only from the provider's own host), size/time bounded, feeds with DTDs refused. See `architecture/magazine-sources.md`.
 - **Magazine contents list shows synthesis state.** Each article row says "Synthesised · m:ss" (the MP3's exact length) or "Not synthesised" (approved, no MP3 yet), and has an inline play/pause button that streams the MP3 without opening the article. Icon plus words, screen-reader labelled, works at phone width.
 - `scripts/compare_magazines.py`: read-only comparison of two magazine folders (words, boxes, reading order, articles, quality numbers) for re-import checks.
--- a/TODO.md
+++ b/TODO.md
@ -732,10 +732,10 @@ casting. The original PDF remains the visual source of truth while approved arti
      Candidates to install as Docker containers alongside the existing engines, each added through the Engines tab
      (`routes/docker.py`'s `_LOCAL_CONTAINER_DEFS`, role `ocr`) so they're selectable/connectable/benchmarkable like
      any other engine, not hardcoded:
-      - [ ] [docling](https://github.com/docling-project/docling) — document conversion/layout toolkit.
-      - [ ] [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR) — established general-purpose OCR.
-      - [ ] [DeepSeek-OCR](https://github.com/deepseek-ai/DeepSeek-OCR) — vision-language OCR model.
-      - [ ] Qwen3-VL-30B-A3B-Instruct-FP8 — general vision-language model; check whether it fits the RAM/VRAM
+      - [x] [docling](https://github.com/docling-project/docling) — document conversion/layout toolkit. *(2026-09-29: benchmarked on CPU, F1 0.867, 8.9 s/page, 2.4 GB; see architecture/evidence/ocr-bench-2026-09-29/)*
+      - [x] [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR) — established general-purpose OCR. *(2026-09-29: benchmarked on CPU, F1 0.881, 36.5 s/page, 1.4 GB; not yet wired as an engine)*
+      - [ ] [DeepSeek-OCR](https://github.com/deepseek-ai/DeepSeek-OCR) — vision-language OCR model. *(not run 2026-09-29: only ~14 GB of the unified memory is free; needs a running model stopped for a test window)*
+      - [ ] *(not run 2026-09-29: ~30 GB does not fit beside the running sglang model)* Qwen3-VL-30B-A3B-Instruct-FP8 — general vision-language model; check whether it fits the RAM/VRAM
            budget at 30B-A3B FP8 (likely the heaviest candidate — weigh against the "low footprint" goal
            specifically) and whether it's meaningfully better at open-ended description than the nano-class
            candidates above.
--- a/architecture/evidence/ocr-bench-2026-09-29/README.md
+++ b/architecture/evidence/ocr-bench-2026-09-29/README.md
@ -0,0 +1,44 @@
+# OCR candidate benchmark — 2026-09-29
+
+Harness: `scripts/bench_ocr.py`. Corpus: *c't KI Wissen 2026* (native text layer, 204 pages). Six evenly spaced
+pages with at least 250 words (5, 32, 67, 101, 138, 169), rendered at 110 dpi (the app's ingest default). The
+PDF's own text layer is the ground truth. Raw numbers: `bench_*.json` (metrics only, no magazine text).
+
+| Engine | Where it ran | Mean word F1 | Seq. similarity | Median s/page | Memory | Licence |
+|---|---|---:|---:|---:|---|---|
+| tesseract deu+eng (current last resort) | CPU, app container | 0.880 | 0.841 | 2.2 | negligible | Apache-2.0 |
+| baidu/Unlimited-OCR (current primary) | GPU container, port 8098 | 0.865 (5/6 pages) | 0.865 | 44.1 | see `docker stats` | as published |
+| docling (default pipeline + RapidOCR) | CPU, venv | 0.867 | 0.783 | 8.9 (first page 66 s warm-up) | 2.4 GB RSS | MIT |
+| PaddleOCR 3.2 (PP-OCRv6 medium, `de`) | CPU, venv | 0.881 | 0.551 | 36.5 | 1.4 GB RSS | Apache-2.0 |
+| DeepSeek-OCR | **not run** | | | | ~7 GB weights | MIT |
+| Qwen3-VL-30B-A3B-Instruct-FP8 | **not run** | | | | ~30 GB weights | Apache-2.0 |
+| KDL-Frontier-Parser-nano (current fallback) | **not run**: container exited (137) 6 days ago | | | | | |
+
+Metrics: *word F1* is order-insensitive overlap of lower-cased word tokens; *sequence similarity* is a difflib ratio
+over the token sequence and so also rewards correct reading order. Both are capped below 1.0 by things a text layer
+contains that a page transcription need not (running heads, hyphenation, sidebars), so compare engines to each other,
+not to 1.0.
+
+## Reading
+- **Plain transcription accuracy is a wash.** Every engine lands at F1 0.865–0.881 on these pages; the differences are
+  within page-to-page variation (0.77–0.96). Nothing here justifies replacing tesseract for text quality alone.
+- **Speed differs by 20×.** Tesseract 2.2 s/page beats the GPU vision engine (44 s/page) and both CPU candidates.
+  The vision engine also returned nothing for one of six pages (page 101), which the app's fallback chain absorbs.
+- **What the heavier engines add is structure, not characters:** Unlimited-OCR labels regions and furniture,
+  docling returns layout-ordered text (sequence similarity 0.78 vs PaddleOCR's 0.55, which is raw line order).
+  Neither is scored here; the app's own layout stage already does this work, so the case for a candidate rests on
+  region/table/figure quality, which needs a different benchmark (labelled boxes), not this one.
+- **Open-ended figure description** (the harder requirement in TODO) is untested: none of the engines run here is a
+  captioning model, and the two that are (DeepSeek-OCR, Qwen3-VL) could not be loaded — see below.
+
+## Why two candidates were not run
+The host has 121 GB of unified memory with about 107 GB in use (the running sglang model dominates; GPU and system
+memory are the same pool on this machine), leaving roughly 14 GB. Qwen3-VL-30B FP8 (~30 GB) cannot be loaded without
+stopping another model, and DeepSeek-OCR (~7 GB, pinned dependencies on a GB10 GPU) would leave almost no margin: an
+out-of-memory kill there could take down the running stack. Both need a decision from the owner: which running model to
+stop for a test window.
+
+## Caveats
+Six pages of one German-language, native-text magazine; CPU timings for docling and PaddleOCR (both would be faster on
+GPU); one run each. Enough to rule out an obvious accuracy win, not to pick a winner for scanned or image-heavy issues —
+this magazine's pages are born-digital, so a scanned issue would separate the engines more.
--- a/architecture/evidence/ocr-bench-2026-09-29/bench_baseline.json
+++ b/architecture/evidence/ocr-bench-2026-09-29/bench_baseline.json
@ -0,0 +1,136 @@
+{
+ "pdf": "source.pdf",
+ "pages": [
+  5,
+  32,
+  67,
+  101,
+  138,
+  169
+ ],
+ "dpi": 110,
+ "results": [
+  {
+   "engine": "tesseract",
+   "pages": [
+    {
+     "page": 5,
+     "sec": 2.2,
+     "ref_words": 709,
+     "hyp_words": 687,
+     "f1": 0.8954,
+     "seq": 0.884,
+     "error": null
+    },
+    {
+     "page": 32,
+     "sec": 2.3,
+     "ref_words": 681,
+     "hyp_words": 659,
+     "f1": 0.8791,
+     "seq": 0.6925,
+     "error": null
+    },
+    {
+     "page": 67,
+     "sec": 1.3,
+     "ref_words": 457,
+     "hyp_words": 376,
+     "f1": 0.7899,
+     "seq": 0.7731,
+     "error": null
+    },
+    {
+     "page": 101,
+     "sec": 1.2,
+     "ref_words": 365,
+     "hyp_words": 360,
+     "f1": 0.9572,
+     "seq": 0.9517,
+     "error": null
+    },
+    {
+     "page": 138,
+     "sec": 2.3,
+     "ref_words": 653,
+     "hyp_words": 615,
+     "f1": 0.8864,
+     "seq": 0.877,
+     "error": null
+    },
+    {
+     "page": 169,
+     "sec": 2.2,
+     "ref_words": 667,
+     "hyp_words": 631,
+     "f1": 0.8737,
+     "seq": 0.8706,
+     "error": null
+    }
+   ],
+   "load_sec": 0.0,
+   "process_rss_growth_mb": 0
+  },
+  {
+   "engine": "vision:http://host.docker.internal:8098|baidu/Unlimited-OCR",
+   "pages": [
+    {
+     "page": 5,
+     "sec": 50.6,
+     "ref_words": 709,
+     "hyp_words": 691,
+     "f1": 0.8886,
+     "seq": 0.8886,
+     "error": null
+    },
+    {
+     "page": 32,
+     "sec": 46.3,
+     "ref_words": 681,
+     "hyp_words": 652,
+     "f1": 0.8792,
+     "seq": 0.8762,
+     "error": null
+    },
+    {
+     "page": 67,
+     "sec": 26.7,
+     "ref_words": 457,
+     "hyp_words": 361,
+     "f1": 0.7971,
+     "seq": 0.7971,
+     "error": null
+    },
+    {
+     "page": 101,
+     "sec": 3.5,
+     "ref_words": 365,
+     "hyp_words": 0,
+     "f1": 0.0,
+     "seq": 0.0,
+     "error": "RuntimeError: engine returned nothing"
+    },
+    {
+     "page": 138,
+     "sec": 44.1,
+     "ref_words": 653,
+     "hyp_words": 621,
+     "f1": 0.8917,
+     "seq": 0.8917,
+     "error": null
+    },
+    {
+     "page": 169,
+     "sec": 37.3,
+     "ref_words": 667,
+     "hyp_words": 640,
+     "f1": 0.8707,
+     "seq": 0.8707,
+     "error": null
+    }
+   ],
+   "load_sec": 0.1,
+   "process_rss_growth_mb": 13
+  }
+ ]
+}
--- a/architecture/evidence/ocr-bench-2026-09-29/bench_docling.json
+++ b/architecture/evidence/ocr-bench-2026-09-29/bench_docling.json
@ -0,0 +1,75 @@
+{
+ "pdf": "source.pdf",
+ "pages": [
+  5,
+  32,
+  67,
+  101,
+  138,
+  169
+ ],
+ "dpi": 110,
+ "results": [
+  {
+   "engine": "docling",
+   "pages": [
+    {
+     "page": 5,
+     "sec": 66.2,
+     "ref_words": 709,
+     "hyp_words": 679,
+     "f1": 0.8847,
+     "seq": 0.7695,
+     "error": null
+    },
+    {
+     "page": 32,
+     "sec": 9.9,
+     "ref_words": 681,
+     "hyp_words": 653,
+     "f1": 0.8861,
+     "seq": 0.6762,
+     "error": null
+    },
+    {
+     "page": 67,
+     "sec": 6.7,
+     "ref_words": 457,
+     "hyp_words": 340,
+     "f1": 0.7729,
+     "seq": 0.7704,
+     "error": null
+    },
+    {
+     "page": 101,
+     "sec": 6.2,
+     "ref_words": 365,
+     "hyp_words": 350,
+     "f1": 0.9231,
+     "seq": 0.9175,
+     "error": null
+    },
+    {
+     "page": 138,
+     "sec": 7.9,
+     "ref_words": 653,
+     "hyp_words": 607,
+     "f1": 0.8825,
+     "seq": 0.7873,
+     "error": null
+    },
+    {
+     "page": 169,
+     "sec": 12.4,
+     "ref_words": 667,
+     "hyp_words": 622,
+     "f1": 0.8534,
+     "seq": 0.7742,
+     "error": null
+    }
+   ],
+   "load_sec": 3.5,
+   "process_rss_growth_mb": 2397
+  }
+ ]
+}
--- a/architecture/evidence/ocr-bench-2026-09-29/bench_paddle.json
+++ b/architecture/evidence/ocr-bench-2026-09-29/bench_paddle.json
@ -0,0 +1,75 @@
+{
+ "pdf": "source.pdf",
+ "pages": [
+  5,
+  32,
+  67,
+  101,
+  138,
+  169
+ ],
+ "dpi": 110,
+ "results": [
+  {
+   "engine": "paddleocr",
+   "pages": [
+    {
+     "page": 5,
+     "sec": 49.0,
+     "ref_words": 709,
+     "hyp_words": 688,
+     "f1": 0.889,
+     "seq": 0.4839,
+     "error": null
+    },
+    {
+     "page": 32,
+     "sec": 46.4,
+     "ref_words": 681,
+     "hyp_words": 664,
+     "f1": 0.8907,
+     "seq": 0.4848,
+     "error": null
+    },
+    {
+     "page": 67,
+     "sec": 29.7,
+     "ref_words": 457,
+     "hyp_words": 466,
+     "f1": 0.7822,
+     "seq": 0.4225,
+     "error": null
+    },
+    {
+     "page": 101,
+     "sec": 27.9,
+     "ref_words": 365,
+     "hyp_words": 363,
+     "f1": 0.9533,
+     "seq": 0.9533,
+     "error": null
+    },
+    {
+     "page": 138,
+     "sec": 41.1,
+     "ref_words": 653,
+     "hyp_words": 622,
+     "f1": 0.9035,
+     "seq": 0.4863,
+     "error": null
+    },
+    {
+     "page": 169,
+     "sec": 31.8,
+     "ref_words": 667,
+     "hyp_words": 638,
+     "f1": 0.869,
+     "seq": 0.4736,
+     "error": null
+    }
+   ],
+   "load_sec": 13.7,
+   "process_rss_growth_mb": 1355
+  }
+ ]
+}
--- a/scripts/bench_ocr.py
+++ b/scripts/bench_ocr.py
@ -0,0 +1,241 @@
+#!/usr/bin/env python3
+"""Benchmark OCR engines on pages of a native-text PDF, using its text layer as ground truth.
+
+    python3 scripts/bench_ocr.py --pdf source.pdf --pages 12,20,30 --engine tesseract \
+        --engine "vision:http://localhost:8098|baidu/Unlimited-OCR" --engine docling --engine paddleocr
+
+The PDF's own text layer is what a perfect transcription of the rendered page would say, so no
+hand-labelled data is needed. Two scores per page, both on lower-cased word tokens:
+  * bag F1   -- order-insensitive overlap (robust to differing column reading order)
+  * seq sim  -- difflib ratio over the token sequence (penalises scrambled order)
+Also wall-clock seconds per page and the peak resident memory of THIS process (a remote engine's own
+footprint is not visible here; read it from `docker stats`).
+
+Engines are optional: one whose library or server is missing is reported as unavailable, not an error.
+Read-only: renders into a temp directory and writes nothing else unless --out is given.
+"""
+from __future__ import annotations
+
+import argparse
+import difflib
+import json
+import re
+import resource
+import statistics
+import sys
+import tempfile
+import time
+from collections import Counter
+from pathlib import Path
+
+_TOKEN = re.compile(r"[\wÄÖÜäöüß]+", re.UNICODE)
+
+
+def tokens(text: str) -> list[str]:
+    # Hyphenation at line ends ("zu-\nführen") is joined so it does not count against an engine
+    # that correctly re-joins it (or for one that keeps the break).
+    text = re.sub(r"-\s*\n\s*", "", text or "")
+    return [t.lower() for t in _TOKEN.findall(text)]
+
+
+def bag_f1(ref: list[str], hyp: list[str]) -> float:
+    if not ref or not hyp:
+        return 0.0
+    overlap = sum((Counter(ref) & Counter(hyp)).values())
+    if not overlap:
+        return 0.0
+    p, r = overlap / len(hyp), overlap / len(ref)
+    return 2 * p * r / (p + r)
+
+
+def seq_sim(ref: list[str], hyp: list[str]) -> float:
+    if not ref or not hyp:
+        return 0.0
+    return difflib.SequenceMatcher(None, ref, hyp, autojunk=False).ratio()
+
+
+def render(pdf: Path, page: int, dpi: int, out: Path) -> Path:
+    import pymupdf
+    with pymupdf.open(str(pdf)) as doc:
+        doc[page - 1].get_pixmap(dpi=dpi).save(str(out))
+    return out
+
+
+def truth(pdf: Path, page: int) -> str:
+    import pymupdf
+    with pymupdf.open(str(pdf)) as doc:
+        return doc[page - 1].get_text()
+
+
+def pick_pages(pdf: Path, count: int, min_words: int) -> list[int]:
+    """Evenly spaced pages with enough text to score meaningfully (adverts and photo pages drop out)."""
+    import pymupdf
+    with pymupdf.open(str(pdf)) as doc:
+        eligible = [n + 1 for n in range(len(doc)) if len(tokens(doc[n].get_text())) >= min_words]
+    if len(eligible) <= count:
+        return eligible
+    step = len(eligible) / count
+    return [eligible[int(i * step)] for i in range(count)]
+
+
+# ── engines: each is a callable (png_path) -> transcribed text, or raises Unavailable ─────────────
+
+class Unavailable(Exception):
+    pass
+
+
+def engine_tesseract(lang: str = "deu+eng"):
+    import shutil
+    import subprocess
+    if not shutil.which("tesseract"):
+        raise Unavailable("tesseract is not installed here")
+
+    def run(png: Path) -> str:
+        return subprocess.run(["tesseract", str(png), "stdout", "-l", lang], capture_output=True,
+                              text=True, timeout=180, check=True).stdout
+    return run
+
+
+def engine_vision(spec: str):
+    """`vision:<base_url>|<model>` -- an OpenAI-compatible vision server, called exactly as the app calls it."""
+    try:
+        base, model = spec.split("|", 1)
+    except ValueError:
+        raise Unavailable("use vision:<url>|<model>")
+    sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+    try:
+        from core import magazine as mag
+    except Exception as e:  # the app's own module needs its environment
+        raise Unavailable(f"cannot import the app's OCR client ({type(e).__name__})")
+    import requests
+    try:
+        requests.get(base.rstrip("/") + ("/v1/models" if "/v1" not in base else "/models"), timeout=5).raise_for_status()
+    except Exception as e:
+        raise Unavailable(f"server not reachable ({type(e).__name__})")
+
+    def run(png: Path) -> str:
+        regions = mag.vision_ocr_page(png, base, model, timeout=300.0)
+        if regions is None:
+            raise RuntimeError("engine returned nothing")
+        return "\n".join(str(r.get("text") or "") for r in regions)
+    return run
+
+
+def engine_docling():
+    try:
+        from docling.document_converter import DocumentConverter
+    except Exception as e:
+        raise Unavailable(f"docling not installed ({type(e).__name__})")
+    conv = DocumentConverter()
+
+    def run(png: Path) -> str:
+        return conv.convert(str(png)).document.export_to_text()
+    return run
+
+
+def engine_paddleocr(lang: str = "de"):
+    try:
+        from paddleocr import PaddleOCR
+    except Exception as e:
+        raise Unavailable(f"paddleocr not installed ({type(e).__name__})")
+    try:  # 3.x: page orientation/unwarping models are extra work a straight scan does not need
+        ocr = PaddleOCR(lang=lang, use_doc_orientation_classify=False, use_doc_unwarping=False,
+                        use_textline_orientation=False)
+    except TypeError:  # 2.x
+        ocr = PaddleOCR(lang=lang, use_angle_cls=False)
+
+    def run(png: Path) -> str:
+        lines: list[str] = []
+        res = ocr.predict(str(png)) if hasattr(ocr, "predict") else ocr.ocr(str(png))
+        for page in res or []:
+            if isinstance(page, dict) or hasattr(page, "get"):          # 3.x result object
+                lines.extend(str(t) for t in (page.get("rec_texts") or []))
+                continue
+            for item in page or []:                                      # 2.x [[box, (text, score)], ...]
+                try:
+                    lines.append(item[1][0])
+                except (IndexError, TypeError):
+                    pass
+        return "\n".join(lines)
+    return run
+
+
+def build_engine(spec: str):
+    if spec == "tesseract":
+        return engine_tesseract()
+    if spec.startswith("vision:"):
+        return engine_vision(spec[len("vision:"):])
+    if spec == "docling":
+        return engine_docling()
+    if spec == "paddleocr":
+        return engine_paddleocr()
+    raise Unavailable(f"unknown engine {spec!r}")
+
+
+def peak_rss_mb() -> float:
+    return resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024.0
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--pdf", required=True, type=Path)
+    ap.add_argument("--pages", default="", help="comma-separated 1-based page numbers (default: evenly spaced)")
+    ap.add_argument("--count", type=int, default=8, help="pages to sample when --pages is not given")
+    ap.add_argument("--min-words", type=int, default=250)
+    ap.add_argument("--dpi", type=int, default=110, help="110 matches the app's ingest default")
+    ap.add_argument("--engine", action="append", required=True)
+    ap.add_argument("--out", type=Path, help="write the full results as JSON")
+    args = ap.parse_args()
+
+    pages = [int(p) for p in args.pages.split(",") if p.strip()] or pick_pages(args.pdf, args.count, args.min_words)
+    print(f"pages: {pages}")
+    results = []
+    with tempfile.TemporaryDirectory(prefix="bench-ocr-") as tmp:
+        pngs = {p: render(args.pdf, p, args.dpi, Path(tmp) / f"p{p}.png") for p in pages}
+        refs = {p: tokens(truth(args.pdf, p)) for p in pages}
+        for spec in args.engine:
+            entry = {"engine": spec, "pages": []}
+            try:
+                base_rss = peak_rss_mb()
+                t0 = time.perf_counter()
+                run = build_engine(spec)
+                entry["load_sec"] = round(time.perf_counter() - t0, 1)
+            except Unavailable as e:
+                entry["unavailable"] = str(e)
+                results.append(entry)
+                print(f"{spec}: unavailable — {e}")
+                continue
+            for p in pages:
+                t0 = time.perf_counter()
+                try:
+                    hyp = tokens(run(pngs[p]))
+                    err = None
+                except Exception as e:
+                    hyp, err = [], f"{type(e).__name__}: {e}"[:200]
+                sec = time.perf_counter() - t0
+                entry["pages"].append({
+                    "page": p, "sec": round(sec, 1), "ref_words": len(refs[p]), "hyp_words": len(hyp),
+                    "f1": round(bag_f1(refs[p], hyp), 4), "seq": round(seq_sim(refs[p], hyp), 4), "error": err})
+                print(f"  {spec} p{p}: f1={entry['pages'][-1]['f1']:.3f} seq={entry['pages'][-1]['seq']:.3f} {sec:.1f}s"
+                      + (f" ERROR {err}" if err else ""))
+            entry["process_rss_growth_mb"] = round(peak_rss_mb() - base_rss)
+            results.append(entry)
+
+    print("\n| engine | pages ok | mean F1 | mean seq | median s/page | RSS growth MB |")
+    print("|---|---:|---:|---:|---:|---:|")
+    for e in results:
+        ok = [r for r in e.get("pages", []) if not r["error"]]
+        if not ok:
+            print(f"| {e['engine']} | 0/{len(e.get('pages', []))} | – | – | – | – | {e.get('unavailable', '')}")
+            continue
+        print(f"| {e['engine']} | {len(ok)}/{len(e['pages'])} | {statistics.mean(r['f1'] for r in ok):.3f} | "
+              f"{statistics.mean(r['seq'] for r in ok):.3f} | {statistics.median(r['sec'] for r in ok):.1f} | "
+              f"{e.get('process_rss_growth_mb', 0)} |")
+    if args.out:
+        args.out.write_text(json.dumps({"pdf": args.pdf.name, "pages": pages, "dpi": args.dpi, "results": results},
+                                       ensure_ascii=False, indent=1), "utf-8")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())