mARTin-B78/tts-voice-creator-clone-and-design-2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: escalate magazine OCR to a connected vision engine on garbled/scanned pages

Browse Source 
A page needing OCR (or whose already-extracted text -- native or
tesseract alike -- scores as mostly garbage via ocr_quality_score) is
now escalated to a connected vision OCR engine before falling back to
tesseract. Falls back automatically on any failure (unreachable engine,
timeout, unparseable output); never fatal to ingestion.

Verified live against ocr-baidu-unlimited-ocr on a real 28-page scanned
c't magazine excerpt (Epson ES-580W, no scanner OCR): every garbled
article title tesseract produced ("K | soll Arbeit abnehmen", ". |",
"@)X", "Owns mee") was replaced by the real headline ("1. Irrtum: KI
ist ein allgemeiner Jobkiller", etc.), and furniture detection jumped
from 44 to 151 blocks because the engine labels running headers/
footers/page numbers directly instead of needing fuzzy post-hoc
matching. 26 of 28 pages used the vision path; the remaining 2 fell
back to tesseract automatically.

Also fixes two real defects found investigating the same real-world
data, both regression-tested against the actual garbled strings
observed:
- Exact-match repeated-line detection (running header/footer) silently
  failed once tesseract OCR'd the same line differently each time --
  cluster_running_furniture fuzzy-matches near-edge blocks instead.
- ocr_page() built one Block per OCR text-LINE rather than per
  paragraph, so a single anomalous line (a chart number, scan noise, a
  wrapped headline continuation) could independently trigger title
  classification -- now grouped by paragraph like the native PyMuPDF
  path, using the paragraph's median line height rather than its total
  bbox height for font size.

A vision-engine label (footer/title/page_number/caption) is trusted
outright and skips classify_blocks' font-size heuristics entirely,
since the engine gives no reliable font metric; an ambiguous "text"
region gets a synthesized estimate from its paragraph area and length
(estimate_font_size_from_area) and still goes through normal
classification.

Off unless vision_ocr_url/vision_ocr_model are configured
(vision_ocr_enabled setting, default on if reachable); per-request
override via the ingest endpoint body. Default engine URL uses
host.docker.internal, confirmed live -- the OCR container runs on its
own Docker network, not dgx_net, so a container-name default would not
resolve from the app container.

28 new tests (136 total in the module). ocr_quality_score is
deliberately coarse and documented as such: it catches text that is
mostly symbol noise, not a plausible single-word misread inside an
otherwise normal sentence (confirmed live: a misread "KI" -> "K |"
still scored 0.81) -- that class of error needs a dictionary or
language-model check, out of scope here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
mARTin-B78 2026-09-06 19:06:11 +02:00
parent 7227c8f89e
commit 477cb43d4a
8 changed files with 887 additions and 40 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
CHANGELOG.md
View File

@ -8,6 +8,23 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
## [Unreleased]
### Added
- **Vision OCR Escalation For Magazine Ingestion.** A page that needs OCR, or whose already-extracted text scores as
mostly garbage (`ocr_quality_score`) — native or tesseract alike — is now escalated to a connected vision OCR engine
(`vision_ocr_url`/`vision_ocr_model` in Settings, defaulting to `ocr-baidu-unlimited-ocr` via `host.docker.internal`),
with tesseract kept only as the automatic fallback if the call fails or the engine is unreachable. Verified live end
to end on a real 28-page scanned c't magazine excerpt (Epson ES-580W, no scanner OCR): every previously garbled
article title — `K | soll Arbeit abnehmen…`, `. |`, `@)X`, `Owns mee`, `Add | Postnew` — was replaced by the real
headline (`1. Irrtum: KI ist ein allgemeiner Jobkiller`, `3. Irrtum: KI entlastet Fachkräfte`, `Wie KI Entwickler-Jobs
verändert`), and furniture block detection jumped from 44 to 151 blocks because the engine labels running
headers/footers/page numbers directly (`text [x0,y0,x1,y1]content`, `footer […]…`, `page_number […]…`) instead of
needing after-the-fact fuzzy matching. 26 of 28 pages used the vision path; the remaining 2 fell back to tesseract
automatically. `ocr_quality_score` is deliberately coarse — it catches text that is mostly symbol noise, not a
plausible-looking single-word misread inside an otherwise normal sentence, which needs a dictionary or
language-model check and is out of scope here. A vision-labeled block (`source: "vision_label"`) is trusted outright
and skips `classify_blocks`' font-size heuristics entirely, since the engine gives no reliable font metric — an
ambiguous `"text"` region still gets a font-size *estimate* from its paragraph area and length
(`estimate_font_size_from_area`) and goes through normal classification. Off unless a URL and model are configured
(`vision_ocr_enabled` setting, default on if reachable; per-request override via the ingest endpoint). 28 new tests.
- **Magazine Projects (backend + API).** A second document workflow beside books, for magazines and papers where the
audiobook assumptions break down: several unrelated articles share a page, columns run in parallel, adverts and sidebars
interrupt body copy, and a feature routinely jumps twenty pages to its tail. Narrating such a PDF page-by-page walks out

4
TODO.md
View File

@ -318,7 +318,9 @@ casting. The original PDF remains the visual source of truth while approved arti
- [ ] Add image descriptions, chart/table handling, and optional spoken figure summaries.
- [ ] Add word-level karaoke timing and EPUB 3 Media Overlay export.
- [ ] Add mobile/driving mode with large controls, automatic article advance, and hands-free resume.
- [ ] Add OCR-engine benchmark comparisons and per-page fallback selection.
- [~] Add OCR-engine benchmark comparisons and per-page fallback selection.
*(per-page fallback done: vision OCR escalates when a page needs OCR or scores poorly, tesseract is the automatic
fallback; no formal cross-engine benchmark or KDL Frontier Parser nano routing yet)*
Three full regression runs of *Der Sturm* against the hand-corrected reference.
Everything below is measured, not estimated. Artefacts live in

107
architecture/magazine-ingestion.md
View File

@ -23,12 +23,22 @@ Implemented in [`core/magazine.py`](../core/magazine.py); driven by
```
PDF
→ extract_pages() PyMuPDF: text blocks with bbox, font size/name, weight
→ ocr_page() tesseract TSV fallback when a page has <120 native chars
→ vision_ocr_page() connected vision OCR engine, when a page needs OCR or its
text scores poorly (ocr_quality_score) -- native or
tesseract alike. Labels footer/title/page_number/caption
directly. Off unless configured; never fatal.
→ ocr_page() tesseract TSV fallback: no vision engine configured, or
the vision call failed/timed out/was unreachable
→ build_vision_blocks() vision regions -> Blocks; a direct label bypasses
classify_blocks entirely (see below)
→ classify_blocks() title / deck / byline / body / caption / pullquote /
sidebar / furniture / ad / figure
sidebar / furniture / ad / figure -- for native/tesseract
blocks, and any vision region left ambiguous ("text")
→ mark_advert_pages() spread an "ADVERTISEMENT" label across its whole page
→ mark_pullquotes() demote display text that merely repeats body copy
→ assign_reading_order() column detection, then top-to-bottom within a column
→ cluster_running_furniture() fuzzy-match near-edge repeats tesseract's noisy
OCR defeated exact matching on (see below)
→ score/parse_contents() Stage A: locate and parse the table of contents
→ group_articles() Stage C: group blocks, link "continued on page N"
→ human review Stage D, in the UI — nothing is narrated until approved
@ -70,6 +80,39 @@ headline both sit at the top of a page. What separates them is that the running
head recurs on many pages at the same edge; `find_repeated_lines` needs the
whole document, so classification waits until every page has been read.
**Exact-match repetition is not enough once OCR is involved.** Confirmed live
on a scanned c't magazine: the running footer "c't Wissen 2026" was read
differently by tesseract on nearly every page (`cE KI-WISSEN`, `ce Wissen
2136`, `cA Wissen 2036`, …), so `find_repeated_lines`'s exact match never
recognised them as the same line — each became its own spurious title and
started a new article, taking a 28-page scan from a handful of real sections
to 26 near-1-per-page "articles". `cluster_running_furniture` fuzzy-matches
near-edge blocks (digit-stripped, `difflib.SequenceMatcher` similarity ≥ 0.6,
requiring ≥3 occurrences) as a second pass over the whole document. A vision
OCR engine that labels the footer directly (see below) makes this pass
unnecessary for the pages it covers, but tesseract-sourced pages still need
it.
**OCR font size is a measurement for native text, an estimate for OCR text.**
PyMuPDF gives an exact digital font size per span. Tesseract only gives a line
bounding box, and its own line SEGMENTATION is unreliable on a scanned
magazine page — `ocr_page` groups by paragraph (matching PyMuPDF's own
grouping) and uses the MEDIAN of the paragraph's line heights, not its total
bbox height, which would scale with line count instead of type size. A vision
OCR engine gives one box per paragraph with no per-line data at all, so its
"text"-labeled (ambiguous) blocks get a synthesized estimate instead
(`estimate_font_size_from_area`, derived from the box's area and the text's
length) — good enough to feed the same title/deck/pullquote thresholds, not a
real measurement.
**A vision-engine label is trusted outright, never re-derived.** When the
engine labels a region `footer`/`title`/`page_number`/`caption` directly
(`build_vision_blocks`), `classify_blocks` skips it (`source ==
"vision_label"`) rather than reclassifying it from an estimated font size —
that would throw away a more direct signal than any heuristic can reconstruct
after the fact. Only an ambiguous `"text"` label (or one the engine invents)
goes through the normal classification pass.
**A page is claimed only if it contributes.** An advert between two features
starts no article, so without an explicit check it gets swallowed by the
preceding one and highlighted as part of it in the PDF view.
@ -152,10 +195,12 @@ and, in practice, hides the buttons from `getByRole('tab')`-oriented tooling.
## Tests
- [`tests/test_magazine.py`](../tests/test_magazine.py) — 34 tests on the pure
- [`tests/test_magazine.py`](../tests/test_magazine.py) — 80 tests on the pure
geometry/classification logic: column reading order, the document-baseline
regression, furniture, pull quotes, contents parsing, continuation linking,
advert pages.
advert pages, paragraph-grouped OCR, fuzzy running-furniture matching, and
the vision OCR quality score / region parsing / block building / HTTP call
(network mocked).
- [`tests/test_magazine_routes.py`](../tests/test_magazine_routes.py) — 10 tests: path
traversal rejection, review gating, LLM script parsing including malformed
output.
@ -171,15 +216,55 @@ Verified end-to-end against a synthetic 8-page issue (contents page, two-column
feature with a byline and a page-8 jump, full-page advert, second feature,
reviews) and against an image-only rendering of the same file to exercise OCR.
## OCR engine selection (not yet wired)
## Vision OCR escalation
Engines → OCR & Document Parsing lists layout-aware parsers (KDL Frontier Parser
nano, Baidu Unlimited-OCR) as connection-probed cards. Those are presentation
and health-check only — there is no server-side dispatch, so ingestion currently
always uses local tesseract via `ocr_lang`. Routing a page image to one of those
vLLM endpoints and mapping its structured output onto `Block` is real work, not
a config change; it is the natural home for TODO P2's per-page fallback
selection and benchmark comparison.
nano, Baidu Unlimited-OCR) as connection-probed cards; those cards are still
presentation/health-check only. Ingestion itself now dispatches to whichever
OpenAI-compatible vision engine is configured (`vision_ocr_url` /
`vision_ocr_model` in Settings, `vision_ocr_enabled` gate) — `vision_ocr_page`
sends the rendered page image with a plain transcription prompt and parses the
engine's own labeled-region reply (`parse_vision_ocr_regions`). Escalation
triggers per page, not for the whole document: a page needing OCR at all
(`native_chars < 120`), or one whose already-extracted text — native
*or* tesseract — scores below `ocr_quality_score`'s coarse garbage threshold.
Any failure (unreachable engine, timeout, unparseable output) returns `None`
and the page falls back to tesseract exactly as if vision OCR were not
configured; ingestion never fails because of it.
**Verified live** against `ocr-baidu-unlimited-ocr` on a real 28-page scanned
c't magazine excerpt (Epson ES-580W, no scanner-side OCR) — every garbled
title tesseract had produced was replaced by the real headline:
| Before (tesseract) | After (vision OCR) |
|---|---|
| `K \| soll Arbeit abnehmen…` | `1. Irrtum: KI ist ein allgemeiner Jobkiller` |
| `. \|` | *(furniture, correctly excluded)* |
| `@)X` | *(furniture, correctly excluded)* |
| `Owns mee` | `Das Team: „Hive Mind" frisst Konzentration` |
| `Wie KI Entwickler-Jobs verandert` | `Wie KI Entwickler-Jobs verändert` |
Furniture detection went from 44 to 151 blocks on the same document, because
the engine labels running headers/footers/page numbers directly instead of
`cluster_running_furniture` having to recover them after the fact from noisy
OCR variants. 26 of 28 pages used the vision path; the remaining 2 fell back
to tesseract automatically (a transient failure or timeout — not investigated
further, since the fallback is exactly the intended behaviour). Article count
did not drop much (26 → 25): the residual "over-fragmentation" is now clearly
attributable to the document itself — a dossier of numbered "1. Irrtum" /
"2. Irrtum" sub-sections and an interview with `c't: …` question headers, all
of which read as headline-fallback article starts absent a contents page —
not to OCR quality, which the review screen is exactly the tool for.
`ocr_quality_score` is deliberately coarse: it discriminates "mostly symbol
noise" from "plausible prose" well enough to gate an expensive vision-OCR
call, but a single plausible-looking misread inside an otherwise normal
sentence (confirmed live: tesseract's "KI" → "K |" substitution, 7 of 8 words
still genuine German) scores just as high as clean text. Catching that class
of error needs a dictionary or language-model check, out of scope here.
Routing to KDL Frontier Parser nano, or a genuine multi-engine fallback
ladder with benchmark comparison, remains TODO P2.
## Known limitations

5
core/config.py
View File

@ -16,6 +16,7 @@ from core.constants import (
_NVIDIA_CLONE_DEFAULT, _NVIDIA_ZEROSHOT_DEFAULT, _NVIDIA_FLOW_DEFAULT,
_FASTER_WHISPER_DEFAULT, _WHISPER_CPP_DEFAULT,
_KOKORO_DEFAULT, _VIBEVOICE_DEFAULT, _XTTS_DEFAULT, _FISHSPEECH_DEFAULT,
_VISION_OCR_DEFAULT, _VISION_OCR_MODEL_DEFAULT,
_TTS_STREAM_DEFAULT,
)
from core.retry import request_with_retries
@ -39,6 +40,7 @@ _SETTINGS_KEYS = {
"captures_default_voice",
"client_voice_bindings",
"llm_url", "llm_model", "llm_api_key",
"vision_ocr_url", "vision_ocr_model", "vision_ocr_enabled",
"engine_local_urls", "engine_container_names", "engine_api_keys", "custom_engine_cards",
# Character portrait generation (cloud APIs + local ComfyUI)
"image_gen_provider", "image_gen_model",
@ -330,6 +332,9 @@ def _load_settings() -> dict:
"llm_url": "http://localhost:11434/v1",
"llm_api_key": "",
"llm_model": "",
"vision_ocr_url": _VISION_OCR_DEFAULT,
"vision_ocr_model": _VISION_OCR_MODEL_DEFAULT,
"vision_ocr_enabled": True,
"engine_local_urls": {},
"engine_container_names": {},
"engine_api_keys": {},

9
core/constants.py
View File

@ -37,6 +37,15 @@ _FASTER_WHISPER_DEFAULT = os.environ.get("FASTER_WHISPER_URL", "http://192.168.1
_WHISPER_CPP_DEFAULT = os.environ.get("WHISPER_CPP_URL", "http://192.168.178.8:8085")
_GROQ_STT_ENDPOINT = "https://api.groq.com/openai/v1"
_KOKORO_DEFAULT = os.environ.get("KOKORO_URL", "http://192.168.178.8:8880/v1")
# Layout-aware OCR engine for magazine ingestion (see routes/magazine.py). Not
# a hardware URL like the others above -- points at whichever "ocr-*" engine
# card (Engines > OCR & Document Parsing) is configured. Defaults to the
# Baidu Unlimited-OCR container's HOST-published port via host.docker.internal
# (wired in docker-compose.yml's extra_hosts) rather than its container name --
# confirmed live that it runs on its own Docker network, not dgx_net, so a
# container-name default would not resolve from the app container.
_VISION_OCR_DEFAULT = os.environ.get("VISION_OCR_URL", "http://host.docker.internal:8098/v1")
_VISION_OCR_MODEL_DEFAULT = os.environ.get("VISION_OCR_MODEL", "baidu/Unlimited-OCR")
_VIBEVOICE_DEFAULT = os.environ.get("VIBEVOICE_URL", "http://192.168.178.8:8027")
_XTTS_DEFAULT = os.environ.get("XTTS_URL", "http://192.168.178.8:8005")
_FISHSPEECH_DEFAULT = os.environ.get("FISHSPEECH_URL", "http://192.168.178.8:38080")

382
core/magazine.py
View File

@ -24,11 +24,17 @@ can be exercised without the native dependency or a sample PDF.
"""
from __future__ import annotations
import base64
import difflib
import json
import re
import statistics
import subprocess
import unicodedata
import requests
from core.retry import request_with_retries
from dataclasses import dataclass, field, asdict
from pathlib import Path
from typing import Any, Iterable, Sequence
@ -98,7 +104,7 @@ class Block:
font_name: str = ""
bold: bool = False
column: int = 0
source: str = "native" # "native" | "ocr"
source: str = "native" # "native" | "ocr" | "vision_ocr" | "vision_label"
def to_json(self) -> dict[str, Any]:
return asdict(self)
@ -267,6 +273,13 @@ def classify_blocks(blocks: list[Block], page_height: float, page_width: float,
base = base_size if base_size and base_size > 0 else body_font_size(blocks)
for b in blocks:
# A vision OCR engine that explicitly labeled this region (footer,
# title, page_number, caption -- see build_vision_blocks) is a more
# direct signal than any font-size heuristic can reconstruct after
# the fact; second-guessing it here would throw that away.
if b.source == "vision_label":
continue
text = _normalise(b.text)
if not text:
b.type = BLOCK_FURNITURE
@ -372,6 +385,69 @@ def find_repeated_lines(pages_text: Sequence[Sequence[str]],
return {t for t, n in counts.items() if n >= minimum and len(t) <= 80}
def looks_like_real_title(text: str) -> bool:
"""Guards the headline-fallback article start against OCR noise.
A stray misread character or chart-axis fragment ("I", "@)X", ". |") can
classify as BLOCK_TITLE by size alone; without this it would spawn its
own article. Confirmed live on a scanned magazine: single-character OCR
artifacts at running-footer size otherwise start empty articles.
"""
letters = sum(1 for c in (text or "") if c.isalpha())
return letters >= 4
def _digit_stripped(text: str) -> str:
return re.sub(r"\d+", "", text or "")
def cluster_running_furniture(blocks: Sequence[Block], page_heights: dict[int, float],
similarity: float = 0.6, min_count: int = 3) -> None:
"""Reclassify near-edge blocks that recur across pages as furniture, even
when OCR reads the running header/footer differently each time.
find_repeated_lines() needs an EXACT match, which OCR defeats: a magazine
tag like "c't Wissen 2026" came back as "cE KI-WISSEN", "ce Wissen 2136",
"cA Wissen 2036" across different pages of the same scan -- confirmed
live, and the dominant cause of a 28-page scan producing 26 near-1-per-
page "articles" instead of grouping around real headlines. Only the page
number reliably varies page to page, so digit-stripped fuzzy similarity
catches what exact matching misses. Mutates blocks in place.
"""
candidates: list[tuple[Block, str]] = []
for b in blocks:
if b.type in (BLOCK_FURNITURE, BLOCK_AD):
continue
text = _normalise(b.text)
if not text or len(text) > 80:
continue
height = page_heights.get(b.page, 842.0)
near_top = b.bbox[1] < height * 0.07
near_bottom = b.bbox[1] + b.bbox[3] > height * 0.93
if not (near_top or near_bottom):
continue
key = _digit_stripped(text).lower().strip()
if len(key) < 3:
continue
candidates.append((b, key))
assigned = [False] * len(candidates)
for i, (_, key_i) in enumerate(candidates):
if assigned[i]:
continue
cluster = [i]
for j in range(i + 1, len(candidates)):
if assigned[j]:
continue
if difflib.SequenceMatcher(None, key_i, candidates[j][1]).ratio() >= similarity:
cluster.append(j)
if len(cluster) >= min_count:
for idx in cluster:
candidates[idx][0].type = BLOCK_FURNITURE
candidates[idx][0].confidence = 0.75
assigned[idx] = True
# ── Stage A: the table of contents ───────────────────────────────────────────
def score_contents_page(lines: Sequence[str]) -> float:
@ -455,7 +531,8 @@ def group_articles(blocks: Sequence[Block],
if not any(b.type in NARRATABLE and b.type != BLOCK_TITLE
for b in by_page[page]):
continue
titles = [b for b in by_page[page] if b.type == BLOCK_TITLE]
titles = [b for b in by_page[page] if b.type == BLOCK_TITLE
and looks_like_real_title(b.text)]
if titles:
starts[page] = {
"title": _normalise(titles[0].text),
@ -571,10 +648,201 @@ def _require_fitz():
) from exc
# Labels a vision OCR engine may return for a region, mapped onto the same
# taxonomy tesseract/native blocks use. Anything else ("text", or a label the
# engine invents) is treated as ambiguous body copy and still goes through
# classify_blocks() for finer distinctions (byline, deck, ...).
_VISION_LABEL_TYPES = {
"footer": BLOCK_FURNITURE,
"header": BLOCK_FURNITURE,
"page_number": BLOCK_FURNITURE,
"title": BLOCK_TITLE,
"heading": BLOCK_TITLE,
"caption": BLOCK_CAPTION,
}
# One label + its four-number bbox, anchoring a region; content runs until
# the next such marker. Confirmed live against Baidu Unlimited-OCR's actual
# output shape: "text [241, 90, 588, 249]Battery technology is entering...".
_VISION_REGION = re.compile(r"(?m)^(\w+)\s*\[\s*(\d+)\s*,\s*(\d+)\s*,\s*(\d+)\s*,\s*(\d+)\s*\]")
def ocr_quality_score(text: str) -> float:
"""Rough plausibility that OCR/extracted text is real prose, 0..1.
Coarse by design: it catches text that is mostly symbol noise (confirmed
live on tesseract line-height artifacts -- ". |", "@)X" scored 0.0/0.33)
well enough to gate an expensive vision-OCR escalation, but it cannot and
is not meant to catch a plausible-looking character substitution inside
an otherwise normal sentence (confirmed live: "K | soll Arbeit abnehmen"
-- a single misread "KI" -- still scored 0.81, since 7 of its 8 words are
genuine German). Catching that class of error needs a dictionary or
language-model check, which is out of scope for a cheap heuristic.
"""
text = (text or "").strip()
if not text:
return 0.0
words = text.split()
if not words:
return 0.0
def _stripped(w: str) -> str:
return w.strip(".,;:!?()[]{}\"'|-–—")
garbage = sum(1 for w in words if not _stripped(w))
letters = sum(1 for c in text if c.isalpha())
total = len(text.replace(" ", "")) or 1
score = (letters / total) * (1.0 - garbage / len(words))
return round(max(0.0, min(1.0, score)), 3)
def estimate_font_size_from_area(width: float, height: float, text_length: int,
char_width_ratio: float = 0.5,
line_height_ratio: float = 1.2) -> float:
"""Estimate a representative font size for a paragraph-level box with no
per-line data (a vision OCR engine gives one box per paragraph, not per
line, unlike tesseract -- see ocr_page()).
Derived from the standard area relationship for a wrapped paragraph:
box_height ~= (text_length / (box_width / (char_width_ratio*size))) *
(line_height_ratio*size), which solves to size ~= sqrt(area / (text_length
* char_width_ratio * line_height_ratio)). Using total bbox height directly
(as if it were one line) would scale with paragraph length instead of
type size -- the same bug already fixed for tesseract's line grouping.
"""
if width <= 0 or height <= 0 or text_length <= 0:
return 0.0
denom = text_length * char_width_ratio * line_height_ratio
return (width * height / denom) ** 0.5 if denom > 0 else 0.0
def parse_vision_ocr_regions(raw: str) -> list[dict[str, Any]]:
"""Parse a vision OCR engine's labeled-region transcription.
Tolerates a preamble the model sometimes prepends before the first region
(confirmed live: instruction-echoing junk, and the literal decoy phrase
"[No text detected]" ahead of real output) by only ever anchoring on the
"label [x0, y0, x1, y1]" marker -- anything before the first match, or
between calls with zero matches, is discarded rather than misread as
content.
"""
matches = list(_VISION_REGION.finditer(raw or ""))
regions = []
for i, m in enumerate(matches):
label = m.group(1).lower()
x0, y0, x1, y1 = (int(m.group(g)) for g in (2, 3, 4, 5))
start = m.end()
end = matches[i + 1].start() if i + 1 < len(matches) else len(raw or "")
text = (raw or "")[start:end].strip()
if not text or x1 <= x0 or y1 <= y0:
continue
regions.append({
"label": label,
"bbox": [float(x0), float(y0), float(x1 - x0), float(y1 - y0)],
"text": text,
})
return regions
def vision_ocr_page(png_path: Path, base_url: str, model: str,
timeout: float = 150.0) -> list[dict[str, Any]] | None:
"""OCR one rendered page with a connected vision-capable OCR engine.
Returns None on any failure (unreachable engine, timeout, unusable
output) so the caller can fall back to tesseract -- this must never be
the reason ingestion fails outright. Confirmed live against
ocr-baidu-unlimited-ocr: markedly more accurate German text than
tesseract on the same scanned page, and it labels running headers/
footers and page numbers directly, which is the dominant source of the
over-fragmentation cluster_running_furniture() otherwise has to recover
after the fact from noisy OCR variants of the same repeating line.
"""
try:
image_b64 = base64.b64encode(png_path.read_bytes()).decode("ascii")
except OSError:
return None
payload = {
"model": model,
"messages": [{"role": "user", "content": [
{"type": "text", "text": (
"Transcribe all visible text on this page verbatim, preserving "
"paragraph breaks. Output only the transcription, no commentary."
)},
{"type": "image_url", "image_url": {
"url": f"data:image/png;base64,{image_b64}",
}},
]}],
"max_tokens": 4000,
"temperature": 0.0,
}
try:
resp = request_with_retries(
"POST", f"{base_url.rstrip('/')}/chat/completions",
json=payload, timeout=timeout, attempts=2,
)
resp.raise_for_status()
content = resp.json()["choices"][0]["message"]["content"]
except (requests.exceptions.RequestException, ValueError, KeyError, IndexError):
return None
regions = parse_vision_ocr_regions(content)
return regions or None
def build_vision_blocks(regions: list[dict[str, Any]], page_no: int,
scale_x: float, id_prefix: str) -> list[Block]:
"""Turn one page's vision OCR regions into Blocks.
A label the engine gives directly (footer/page_number/header/title/
caption) is trusted outright -- source="vision_label" makes
classify_blocks() skip it rather than second-guess it from font size,
which vision OCR does not reliably provide (see estimate_font_size_from_
area's docstring: it is a paragraph-area estimate, not a measurement).
An unrecognised or "text" label is left ambiguous (BLOCK_BODY,
source="vision_ocr") and still goes through the normal classification
pass for finer distinctions such as byline or deck.
"""
out = []
for i, region in enumerate(regions):
x, y, w, h = (v * scale_x for v in region["bbox"])
label = region["label"]
text = region["text"]
mapped = _VISION_LABEL_TYPES.get(label)
if mapped is not None:
block = Block(
id=f"{id_prefix}{page_no:04d}-v{i:03d}",
page=page_no, bbox=[x, y, w, h], type=mapped, text=text,
confidence=0.85, source="vision_label",
)
else:
block = Block(
id=f"{id_prefix}{page_no:04d}-v{i:03d}",
page=page_no, bbox=[x, y, w, h], type=BLOCK_BODY, text=text,
confidence=0.8,
font_size=estimate_font_size_from_area(w, h, len(text)),
source="vision_ocr",
)
out.append(block)
return out
def ocr_page(png_path: Path, lang: str = "deu+eng",
min_confidence: float = 40.0) -> list[dict[str, Any]]:
"""OCR one rendered page, returning line boxes from tesseract's TSV.
"""OCR one rendered page, returning paragraph boxes from tesseract's TSV.
Grouped by (block, paragraph), not by line: PyMuPDF's native path already
aggregates a paragraph's spans into one Block with one representative font
size, and OCR must match that shape or the two paths behave inconsistently
downstream. Confirmed live on a scanned magazine -- one Block per LINE let
a single anomalous line (a chart axis number, a headline's wrapped second
line, scan noise) independently trigger BLOCK_TITLE classification, and a
28-page scan produced ~11 "title" blocks per page instead of grouping
around real headlines. font_size uses the MEDIAN of the paragraph's own
line heights rather than the paragraph's total bbox height (which would
scale with line count, not type size) -- a genuine multi-line headline
still measures large because each of its few lines is uniformly large;
one noisy line inside an otherwise normal paragraph no longer dominates.
TSV rather than plain text because the coordinates are the whole point:
they are what lets an OCR'd page take part in the same column and article
grouping as a native-text page.
@ -587,7 +855,7 @@ def ocr_page(png_path: Path, lang: str = "deu+eng",
except (subprocess.SubprocessError, FileNotFoundError, OSError):
return []
lines: dict[tuple[int, int, int], dict[str, Any]] = {}
paragraphs: dict[tuple[int, int], dict[str, Any]] = {}
for row in proc.stdout.splitlines()[1:]:
parts = row.split("\t")
if len(parts) < 12:
@ -602,34 +870,61 @@ def ocr_page(png_path: Path, lang: str = "deu+eng",
word = parts[11].strip()
if not word or conf < min_confidence:
continue
key = (block_num, par_num, line_num)
rec = lines.setdefault(key, {
para_key = (block_num, par_num)
para = paragraphs.setdefault(para_key, {"lines": {}})
line = para["lines"].setdefault(line_num, {
"text": [], "x0": left, "y0": top,
"x1": left + width, "y1": top + height, "conf": [],
})
rec["text"].append(word)
rec["x0"] = min(rec["x0"], left)
rec["y0"] = min(rec["y0"], top)
rec["x1"] = max(rec["x1"], left + width)
rec["y1"] = max(rec["y1"], top + height)
rec["conf"].append(conf)
line["text"].append(word)
line["x0"] = min(line["x0"], left)
line["y0"] = min(line["y0"], top)
line["x1"] = max(line["x1"], left + width)
line["y1"] = max(line["y1"], top + height)
line["conf"].append(conf)
out = []
for (_, _, _), rec in sorted(lines.items()):
for _, para in sorted(paragraphs.items()):
lines = [l for _, l in sorted(para["lines"].items()) if l["text"]]
if not lines:
continue
x0 = min(l["x0"] for l in lines)
y0 = min(l["y0"] for l in lines)
x1 = max(l["x1"] for l in lines)
y1 = max(l["y1"] for l in lines)
heights = [float(l["y1"] - l["y0"]) for l in lines]
confs = [c for l in lines for c in l["conf"]]
out.append({
"text": " ".join(rec["text"]),
"bbox": [float(rec["x0"]), float(rec["y0"]),
float(rec["x1"] - rec["x0"]), float(rec["y1"] - rec["y0"])],
"confidence": round(sum(rec["conf"]) / len(rec["conf"]) / 100.0, 3),
"text": " ".join(" ".join(l["text"]) for l in lines),
"bbox": [float(x0), float(y0), float(x1 - x0), float(y1 - y0)],
"line_height": statistics.median(heights),
"confidence": round(sum(confs) / len(confs) / 100.0, 3),
})
return out
_VISION_QUALITY_THRESHOLD = 0.5 # below this, existing native/OCR text is escalated
def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110,
ocr_lang: str = "deu+eng",
vision_ocr_url: str | None = None,
vision_ocr_model: str | None = None,
vision_ocr_timeout: float = 150.0,
progress: Any = None) -> dict[str, Any]:
"""Run the full extraction for one PDF, writing page renders and images.
When vision_ocr_url/model are given, a page whose text needs OCR (or
whose already-extracted text scores as mostly garbage, native or
tesseract alike -- see ocr_quality_score) is escalated to that engine
first, with tesseract only as the fallback if the call fails or the
engine is unreachable. Confirmed live against ocr-baidu-unlimited-ocr:
markedly more accurate transcription of a scanned German magazine than
tesseract, and it labels running headers/footers/page numbers directly
(see build_vision_blocks), which is the dominant source of the
over-fragmentation cluster_running_furniture() otherwise has to recover
after the fact from noisy OCR variants of the same repeating line.
Returns the layout document: every block on every page, classified and in
reading order, plus the raw contents-page candidates.
"""
@ -638,6 +933,8 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110,
images_dir = out_dir / "images"
pages_dir.mkdir(parents=True, exist_ok=True)
images_dir.mkdir(parents=True, exist_ok=True)
vision_enabled = bool(vision_ocr_url and vision_ocr_model)
vision_page_count = 0
doc = fitz.open(str(pdf_path))
try:
@ -693,22 +990,51 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110,
))
# A page whose native text is negligible is a scan (or is drawn as
# outlines); OCR is the only way to reach its words.
# outlines); OCR is the only way to reach its words. A page WITH
# native/embedded text is not necessarily good text -- a
# pre-OCR'd scan (e.g. from scanner-bundled OCR software) can
# carry a poor text layer that this same escalation should reach.
native_chars = sum(len(b.text) for b in blocks)
if native_chars < 120:
needs_ocr = native_chars < 120
native_quality = (
ocr_quality_score(" ".join(b.text for b in blocks))
if not needs_ocr else 0.0
)
needs_escalation = needs_ocr or native_quality < _VISION_QUALITY_THRESHOLD
vision_blocks: list[Block] | None = None
if vision_enabled and needs_escalation:
regions = vision_ocr_page(png, vision_ocr_url, vision_ocr_model,
timeout=vision_ocr_timeout)
if regions:
scale_x = rect.width / max(1.0, _png_width(png, fitz))
vision_blocks = build_vision_blocks(
regions, page_no, scale_x, id_prefix="b")
if vision_blocks is not None:
# Replaces whatever was already there -- including a poor
# native/pre-OCR'd text layer, rather than layering a second,
# conflicting reading on top of it.
vision_page_count += 1
blocks = vision_blocks
elif needs_ocr:
scale_x = rect.width / max(1.0, _png_width(png, fitz))
for li, line in enumerate(ocr_page(png, ocr_lang)):
bx, by, bw, bh = line["bbox"]
for li, para in enumerate(ocr_page(png, ocr_lang)):
bx, by, bw, bh = para["bbox"]
blocks.append(Block(
id=f"b{page_no:04d}-o{li:03d}",
page=page_no,
bbox=[bx * scale_x, by * scale_x, bw * scale_x, bh * scale_x],
type=BLOCK_BODY,
text=line["text"],
confidence=line["confidence"],
font_size=bh * scale_x,
text=para["text"],
confidence=para["confidence"],
# The paragraph's own line height, not its full bbox
# height -- that would scale with line count instead
# of type size (see ocr_page's docstring).
font_size=para["line_height"] * scale_x,
source="ocr",
))
# else: native text present and scored well enough -- unchanged.
raw_pages.append({
"page": page_no,
@ -742,6 +1068,13 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110,
assign_reading_order(p["blocks"], p["width"])
all_blocks.extend(p["blocks"])
# Cross-page, so it runs once over every block rather than per page
# (see cluster_running_furniture's docstring for why exact-match
# find_repeated_lines above is not enough once OCR is involved).
page_heights = {p["page"]: p["height"] for p in raw_pages}
cluster_running_furniture(all_blocks, page_heights)
for p in raw_pages:
lines = [b.text for b in sorted(p["blocks"], key=lambda b: b.reading_order)]
score = score_contents_page(lines)
if score >= 0.45:
@ -757,6 +1090,7 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110,
for p in raw_pages],
"blocks": [b.to_json() for b in all_blocks],
"contents_candidates": contents_candidates,
"vision_ocr_pages": vision_page_count,
}
finally:
doc.close()

30
routes/magazine.py
View File

@ -108,7 +108,8 @@ async def magazine_create(request: Request):
# ── Stage A–C: ingestion ─────────────────────────────────────────────────────
def _ingest_sync(doc_id: str, job_id: str, ocr_lang: str, render_dpi: int) -> None:
def _ingest_sync(doc_id: str, job_id: str, ocr_lang: str, render_dpi: int,
vision_ocr_url: str | None, vision_ocr_model: str | None) -> None:
"""Run the extraction pipeline; recorded as a restart-safe job."""
d = _LIB / doc_id
pdf = d / "source.pdf"
@ -123,8 +124,10 @@ def _ingest_sync(doc_id: str, job_id: str, ocr_lang: str, render_dpi: int) -> No
)
background_job_update(job_id, stage="extract", detail="Opening PDF")
layout = mag.extract_pages(pdf, d, render_dpi=render_dpi,
ocr_lang=ocr_lang, progress=progress)
layout = mag.extract_pages(pdf, d, render_dpi=render_dpi, ocr_lang=ocr_lang,
vision_ocr_url=vision_ocr_url,
vision_ocr_model=vision_ocr_model,
progress=progress)
mag.save_layout(d, layout)
background_job_update(job_id, stage="articles", detail="Detecting articles")
@ -140,6 +143,7 @@ def _ingest_sync(doc_id: str, job_id: str, ocr_lang: str, render_dpi: int) -> No
"ocr_pages": sum(
1 for b in layout.get("blocks", []) if b.get("source") == "ocr"
),
"vision_ocr_pages": layout.get("vision_ocr_pages", 0),
}
_write_meta(d, meta)
background_job_update(job_id, stage="done", done=True,
@ -171,13 +175,31 @@ async def magazine_ingest(doc_id: str, request: Request):
except (TypeError, ValueError):
render_dpi = 110
# A page needing OCR (or whose existing text scores as mostly garbage --
# see ocr_quality_score) is escalated to this engine before falling back
# to tesseract. Off by default per-request unless the deployment enabled
# it in Settings; a request can still force it off with vision_ocr:false
# even when enabled, e.g. for a fast re-run once boundaries are already
# approved and only re-detection is wanted.
settings = _load_settings()
vision_ocr_url = vision_ocr_model = None
want_vision = body.get("vision_ocr")
if want_vision is None:
want_vision = bool(settings.get("vision_ocr_enabled"))
if want_vision:
vision_ocr_url = str(body.get("vision_ocr_url") or settings.get("vision_ocr_url") or "").strip()
vision_ocr_model = str(body.get("vision_ocr_model") or settings.get("vision_ocr_model") or "").strip()
if not (vision_ocr_url and vision_ocr_model):
vision_ocr_url = vision_ocr_model = None
job_id = uuid.uuid4().hex
background_job_create(job_id, "magazine-ingest", fingerprint=doc_id)
meta["ingest"] = {"status": "running", "job_id": job_id}
_write_meta(d, meta)
asyncio.create_task(
asyncio.to_thread(_ingest_sync, doc_id, job_id, ocr_lang, render_dpi)
asyncio.to_thread(_ingest_sync, doc_id, job_id, ocr_lang, render_dpi,
vision_ocr_url, vision_ocr_model)
)
return {"job_id": job_id}

373
tests/test_magazine.py
View File

@ -9,11 +9,16 @@ from __future__ import annotations
import os
import tempfile
import pathlib
import unittest
from unittest import mock
os.environ.setdefault("CONFIG_DIR", tempfile.mkdtemp(prefix="ttvc-magazine-test-config-"))
from core.magazine import (
ocr_page,
cluster_running_furniture,
looks_like_real_title,
BLOCK_AD,
BLOCK_BODY,
BLOCK_BYLINE,
@ -27,8 +32,10 @@ from core.magazine import (
assign_reading_order,
body_font_size,
build_articles,
build_vision_blocks,
classify_blocks,
detect_columns,
estimate_font_size_from_area,
estimate_listen_seconds,
find_continuation,
find_repeated_lines,
@ -37,8 +44,11 @@ from core.magazine import (
looks_like_byline,
mark_advert_pages,
mark_pullquotes,
ocr_quality_score,
parse_contents,
parse_vision_ocr_regions,
score_contents_page,
vision_ocr_page,
)
PAGE_W, PAGE_H = 595.0, 842.0 # A4 in points
@ -254,6 +264,157 @@ class ContinuationTests(unittest.TestCase):
self.assertEqual(find_continuation("The page turned slowly."), (None, None))
class OcrParagraphGroupingTests(unittest.TestCase):
"""ocr_page() groups tesseract's TSV by paragraph, not by line -- matching
how the native PyMuPDF path already aggregates a paragraph's spans into
one Block with one representative font size. One Block per line let a
single anomalous line (a chart number, scan noise, a headline's wrapped
second line) independently trigger BLOCK_TITLE classification; confirmed
live as the dominant cause of a 28-page scan producing ~11 "title" blocks
per page instead of grouping around real headlines."""
TSV_HEADER = "level\tpage_num\tblock_num\tpar_num\tline_num\tword_num\tleft\ttop\twidth\theight\tconf\ttext"
def _row(self, block, par, line, word_num, left, top, width, height, conf, text):
return f"5\t1\t{block}\t{par}\t{line}\t{word_num}\t{left}\t{top}\t{width}\t{height}\t{conf}\t{text}"
def _run(self, rows):
tsv = "\n".join([self.TSV_HEADER] + rows)
fake = mock.Mock(stdout=tsv)
with mock.patch("subprocess.run", return_value=fake):
return ocr_page(pathlib.Path("/fake.png"))
def test_two_lines_of_one_paragraph_become_a_single_block(self):
rows = [
self._row(1, 1, 1, 1, 10, 100, 40, 20, 92, "Battery"),
self._row(1, 1, 1, 2, 55, 100, 40, 20, 91, "technology"),
self._row(1, 1, 2, 1, 10, 125, 60, 20, 90, "continues"),
]
out = self._run(rows)
self.assertEqual(len(out), 1)
self.assertEqual(out[0]["text"], "Battery technology continues")
def test_font_size_uses_median_line_height_not_total_bbox_height(self):
"""The regression this guards: a paragraph's bbox spans every line it
contains, so using that height as font_size scales with line count.
A 5-line normal paragraph would then misread as giant display type."""
rows = [self._row(1, 1, i, 1, 10, 100 + i * 20, 200, 18, 90, "word")
for i in range(1, 6)]
out = self._run(rows)
self.assertEqual(len(out), 1)
# Combined bbox height spans 5 lines (~118px); the real type size is
# each line's own ~18px height.
self.assertLess(out[0]["line_height"], out[0]["bbox"][3])
self.assertAlmostEqual(out[0]["line_height"], 18.0, delta=1.0)
def test_one_anomalous_line_does_not_dominate_the_paragraph_median(self):
"""A single chart-axis number or scan artifact inside an otherwise
normal paragraph must not make the whole paragraph read as a title."""
rows = [
self._row(1, 1, 1, 1, 10, 100, 40, 18, 90, "normal"),
self._row(1, 1, 2, 1, 10, 120, 40, 95, 90, "GIANT"), # one outlier line
self._row(1, 1, 3, 1, 10, 220, 40, 19, 90, "normal"),
]
out = self._run(rows)
self.assertLess(out[0]["line_height"], 30.0)
def test_low_confidence_words_are_dropped(self):
rows = [
self._row(1, 1, 1, 1, 10, 100, 40, 18, 90, "good"),
self._row(1, 1, 1, 2, 55, 100, 40, 18, 5, "garbage"),
]
out = self._run(rows)
self.assertEqual(out[0]["text"], "good")
def test_different_paragraphs_stay_separate_blocks(self):
rows = [
self._row(1, 1, 1, 1, 10, 100, 40, 18, 90, "first"),
self._row(1, 2, 1, 1, 10, 300, 40, 18, 90, "second"),
]
out = self._run(rows)
self.assertEqual(len(out), 2)
def test_tesseract_failure_returns_empty_not_an_exception(self):
with mock.patch("subprocess.run", side_effect=FileNotFoundError()):
self.assertEqual(ocr_page(pathlib.Path("/fake.png")), [])
class RunningFurnitureFuzzyMatchTests(unittest.TestCase):
"""Grounded in real OCR output from a scanned c't magazine: a running
footer ("c't Wissen 2026") was read differently on nearly every page,
defeating exact-match repetition and producing 26 near-1-per-page
"articles" on a 28-page scan instead of grouping around real headlines."""
def _footer(self, page, text, y=800):
b = block(f"f{page}", page, 60, y, 200, 18, text, size=18.3, btype=BLOCK_TITLE)
return b
def test_ocr_noisy_footer_variants_cluster_as_furniture(self):
variants = ["c't Wissen 2026", "cE KI-WISSEN", "ce Wissen 2136",
"cA Wissen 2036", "Wissen 205", "Wissen 2026",
"Wissen 2025", "ce Wissen 206"]
blocks = [self._footer(i + 1, v) for i, v in enumerate(variants)]
heights = {b.page: PAGE_H for b in blocks}
cluster_running_furniture(blocks, heights)
self.assertTrue(all(b.type == BLOCK_FURNITURE for b in blocks),
[b.type for b in blocks])
def test_a_real_headline_appearing_once_is_not_reclassified(self):
blocks = [self._footer(1, "Wissen 2026"), self._footer(2, "Wissen 2026"),
self._footer(3, "Wissen 2026")]
headline = block("h1", 4, 50, 60, 470, 50,
"The Future of Solid State Batteries", size=26.0,
btype=BLOCK_TITLE)
blocks.append(headline)
heights = {b.page: PAGE_H for b in blocks}
cluster_running_furniture(blocks, heights)
self.assertEqual(headline.type, BLOCK_TITLE)
def test_fewer_than_min_count_repeats_are_left_alone(self):
"""Two coincidental near-matches should not be enough -- avoids
reclassifying a genuinely rare pair of similar short headlines."""
blocks = [self._footer(1, "Wissen 2026"), self._footer(2, "Wissen 2036")]
heights = {b.page: PAGE_H for b in blocks}
cluster_running_furniture(blocks, heights)
self.assertTrue(all(b.type == BLOCK_TITLE for b in blocks))
def test_body_position_text_is_never_reclassified_regardless_of_repetition(self):
"""Only near-edge blocks are candidates -- a repeated body phrase
(e.g. a slogan) must not be swept into furniture."""
blocks = [block(f"b{i}", i + 1, 50, 400, 200, 20, "Wissen 2026", size=18.3)
for i in range(5)]
heights = {b.page: PAGE_H for b in blocks}
cluster_running_furniture(blocks, heights)
self.assertTrue(all(b.type == BLOCK_BODY for b in blocks))
def test_heavily_truncated_ocr_fragment_does_not_falsely_join(self):
variants = ["Arbeitswelt & Wirtschaft"] * 3 + ["& Wi"]
blocks = [self._footer(i + 1, v) for i, v in enumerate(variants)]
heights = {b.page: PAGE_H for b in blocks}
cluster_running_furniture(blocks, heights)
# The three clean repeats still cluster...
self.assertEqual(sum(1 for b in blocks if b.type == BLOCK_FURNITURE), 3)
# ...the badly truncated one is left for a human to catch in review.
self.assertEqual(blocks[3].type, BLOCK_TITLE)
class TitlePlausibilityTests(unittest.TestCase):
def test_ocr_noise_fragments_are_rejected(self):
for junk in ("I", "@)X", ". |", "12", ""):
self.assertFalse(looks_like_real_title(junk), junk)
def test_real_headlines_are_accepted(self):
for title in ("Wie KI Entwickler-Jobs verändert", "AI Jobs", "Batteries"):
self.assertTrue(looks_like_real_title(title), title)
def test_single_garbled_title_does_not_start_an_empty_article(self):
"""The exact failure seen live: a stray 'I' at running-footer size
on an otherwise bodyless page must not spawn its own article."""
blocks = [block("b1", 10, 50, 800, 40, 18, "I", size=18.3, btype=BLOCK_TITLE)]
blocks[0].reading_order = 0
self.assertEqual(group_articles(blocks, [], page_count=28), [])
class ArticleGroupingTests(unittest.TestCase):
def _issue_blocks(self):
blocks = [
@ -366,6 +527,218 @@ class ArticleGroupingTests(unittest.TestCase):
self.assertTrue(all(a.pages[0] != 2 for a in arts))
class OcrQualityScoreTests(unittest.TestCase):
"""Grounded in real output from both OCR paths on a scanned c't magazine.
Deliberately coarse: it only needs to tell "mostly garbage" from
"plausible prose" well enough to gate an expensive vision-OCR escalation
-- not to catch a subtle character substitution inside an otherwise
normal sentence (confirmed live: a misread "KI" -> "K |" inside a real
German sentence still scores high, since 7 of its 8 words are genuine;
that class of error needs a dictionary or language-model check).
"""
def test_pure_symbol_noise_scores_at_or_near_zero(self):
for junk in (". |", "@)X", ""):
self.assertLess(ocr_quality_score(junk), 0.4, junk)
def test_clean_prose_scores_high(self):
for text in (
"Damit stellen die Forscher das Hauptversprechen der KI-Branche infrage.",
"1. Irrtum: KI ist ein allgemeiner Jobkiller",
):
self.assertGreater(ocr_quality_score(text), 0.85, text)
def test_a_single_misread_word_in_an_otherwise_real_sentence_is_not_caught(self):
"""Documents the known limitation rather than hiding it: this is a
coarse gate, not an accuracy detector."""
text = "K | soll Arbeit abnehmen, die Produktivität stei-"
self.assertGreater(ocr_quality_score(text), 0.7, text)
def test_empty_and_whitespace_only_are_zero(self):
self.assertEqual(ocr_quality_score(""), 0.0)
self.assertEqual(ocr_quality_score(" "), 0.0)
class FontSizeAreaEstimateTests(unittest.TestCase):
"""A vision OCR engine returns one box per paragraph, not per line (see
vision_ocr_page's docstring) -- there is no per-line height to read a
font size from, so it must be estimated from the paragraph's area and
its own text length."""
def test_longer_text_in_the_same_box_implies_a_smaller_font(self):
short = estimate_font_size_from_area(300, 100, text_length=50)
long_ = estimate_font_size_from_area(300, 100, text_length=500)
self.assertGreater(short, long_)
def test_degenerate_inputs_return_zero_not_an_error(self):
self.assertEqual(estimate_font_size_from_area(0, 100, 50), 0.0)
self.assertEqual(estimate_font_size_from_area(100, 0, 50), 0.0)
self.assertEqual(estimate_font_size_from_area(100, 100, 0), 0.0)
def test_a_plausible_headline_box_yields_a_plausible_display_size(self):
# "1. Irrtum: KI ist ein allgemeiner Jobkiller" in a 309x42 box.
size = estimate_font_size_from_area(309, 42, text_length=44)
self.assertGreater(size, 10.0)
self.assertLess(size, 40.0)
class VisionOcrRegionParsingTests(unittest.TestCase):
"""Grounded in ocr-baidu-unlimited-ocr's actual response shape, confirmed
live against a real scanned c't magazine page."""
REAL_SAMPLE = (
" Do not use special punctuation or formatting as visual depts. "
"Special formats such as Oalemium namespace.\n"
"text [260, 86, 589, 248]Killo Bertine in Brackmur et B. Bruche der "
"IT-Branche.\n"
"title [605, 294, 914, 336]1. Irrtum: KI ist ein allgemeiner Jobkiller\n"
"footer [65, 953, 143, 965]c.t KI Wissen 2026\n"
"footer [810, 948, 924, 960]Arbeitswelt & Wirtschaft\n"
"page_number [945, 945, 959, 959]7"
)
def test_preamble_before_the_first_marker_is_discarded(self):
"""The engine sometimes prepends instruction-echoing junk, including
the literal decoy phrase '[No text detected]', before real output."""
regions = parse_vision_ocr_regions(self.REAL_SAMPLE)
self.assertNotIn("Do not use", regions[0]["text"])
self.assertNotIn("namespace", regions[0]["text"])
def test_every_labeled_region_is_recovered(self):
regions = parse_vision_ocr_regions(self.REAL_SAMPLE)
labels = [r["label"] for r in regions]
self.assertEqual(labels, ["text", "title", "footer", "footer", "page_number"])
def test_bbox_is_corner_to_corner_not_xywh(self):
regions = parse_vision_ocr_regions(self.REAL_SAMPLE)
footer = regions[2]
self.assertEqual(footer["bbox"], [65.0, 953.0, 78.0, 12.0])
def test_page_number_text_is_preserved(self):
regions = parse_vision_ocr_regions(self.REAL_SAMPLE)
self.assertEqual(regions[-1]["text"], "7")
def test_completely_unstructured_output_yields_no_regions(self):
self.assertEqual(parse_vision_ocr_regions("just some prose, no markers"), [])
def test_empty_input_yields_no_regions(self):
self.assertEqual(parse_vision_ocr_regions(""), [])
def test_a_degenerate_zero_area_box_is_dropped(self):
self.assertEqual(parse_vision_ocr_regions("text [10,10,10,10]junk"), [])
class VisionBlockBuildingTests(unittest.TestCase):
"""An explicit engine label (footer/title/page_number/caption) is trusted
outright rather than re-derived from font size, which vision OCR gives no
reliable measurement of -- only "text" (or an unrecognised label) is left
ambiguous for the normal classification pass."""
def test_footer_label_becomes_furniture_and_is_protected_from_reclassification(self):
regions = [{"label": "footer", "bbox": [10, 900, 100, 20], "text": "c.t Wissen 2026"}]
blocks = build_vision_blocks(regions, page_no=5, scale_x=1.0, id_prefix="b")
self.assertEqual(blocks[0].type, BLOCK_FURNITURE)
self.assertEqual(blocks[0].source, "vision_label")
# classify_blocks must leave a vision_label block alone even though
# its heuristics, run in isolation, would call it something else.
classify_blocks(blocks, page_height=1049, page_width=897)
self.assertEqual(blocks[0].type, BLOCK_FURNITURE)
def test_title_label_becomes_title_and_survives_classification(self):
regions = [{"label": "title", "bbox": [605, 294, 914, 336],
"text": "1. Irrtum: KI ist ein allgemeiner Jobkiller"}]
blocks = build_vision_blocks(regions, page_no=5, scale_x=1.0, id_prefix="b")
classify_blocks(blocks, page_height=1049, page_width=897)
self.assertEqual(blocks[0].type, BLOCK_TITLE)
def test_page_number_label_becomes_furniture(self):
regions = [{"label": "page_number", "bbox": [945, 945, 959, 959], "text": "7"}]
blocks = build_vision_blocks(regions, page_no=5, scale_x=1.0, id_prefix="b")
self.assertEqual(blocks[0].type, BLOCK_FURNITURE)
def test_caption_label_becomes_caption(self):
regions = [{"label": "caption", "bbox": [10, 10, 100, 20], "text": "A lithium cell."}]
blocks = build_vision_blocks(regions, page_no=5, scale_x=1.0, id_prefix="b")
self.assertEqual(blocks[0].type, BLOCK_CAPTION)
def test_text_label_is_ambiguous_and_goes_through_normal_classification(self):
regions = [{"label": "text", "bbox": [50, 100, 400, 300],
"text": "Ordinary paragraph copy. " * 10}]
blocks = build_vision_blocks(regions, page_no=5, scale_x=1.0, id_prefix="b")
self.assertEqual(blocks[0].source, "vision_ocr")
classify_blocks(blocks, page_height=1049, page_width=897, base_size=10.0)
self.assertEqual(blocks[0].type, BLOCK_BODY)
def test_an_unrecognised_label_is_also_treated_as_ambiguous(self):
regions = [{"label": "sidebar_thing", "bbox": [50, 100, 400, 300], "text": "Some text"}]
blocks = build_vision_blocks(regions, page_no=5, scale_x=1.0, id_prefix="b")
self.assertEqual(blocks[0].source, "vision_ocr")
def test_scale_factor_is_applied_to_the_bbox(self):
regions = [{"label": "footer", "bbox": [100, 200, 300, 220], "text": "x"}]
blocks = build_vision_blocks(regions, page_no=1, scale_x=0.5, id_prefix="b")
self.assertEqual(blocks[0].bbox, [50.0, 100.0, 150.0, 110.0])
def test_block_ids_are_unique_and_page_scoped(self):
regions = [{"label": "text", "bbox": [0, 0, 10, 10], "text": "a"},
{"label": "text", "bbox": [0, 20, 10, 10], "text": "b"}]
blocks = build_vision_blocks(regions, page_no=3, scale_x=1.0, id_prefix="b")
self.assertEqual(len({b.id for b in blocks}), 2)
self.assertTrue(all("0003" in b.id for b in blocks))
class VisionOcrPageCallTests(unittest.TestCase):
"""The HTTP call itself, with the network mocked -- vision_ocr_page must
degrade to None on any failure so ingestion can fall back to tesseract
rather than fail outright."""
def _resp(self, status=200, json_body=None):
import requests
resp = mock.Mock(status_code=status)
resp.raise_for_status = mock.Mock(
side_effect=None if status < 400
else requests.exceptions.HTTPError(f"HTTP {status}"))
resp.json = mock.Mock(return_value=json_body or {})
return resp
def test_successful_call_returns_parsed_regions(self):
body = {"choices": [{"message": {"content":
"text [10,10,110,60]Hello there, this is a real sentence."}}]}
with mock.patch("core.magazine.request_with_retries",
return_value=self._resp(json_body=body)):
out = vision_ocr_page(pathlib.Path(__file__), "http://engine/v1", "model-x")
self.assertIsNotNone(out)
self.assertEqual(out[0]["label"], "text")
def test_unreachable_engine_returns_none(self):
import requests
with mock.patch("core.magazine.request_with_retries",
side_effect=requests.exceptions.ConnectionError()):
self.assertIsNone(vision_ocr_page(pathlib.Path(__file__), "http://engine/v1", "m"))
def test_http_error_status_returns_none(self):
with mock.patch("core.magazine.request_with_retries",
return_value=self._resp(status=500)):
self.assertIsNone(vision_ocr_page(pathlib.Path(__file__), "http://engine/v1", "m"))
def test_unparseable_response_shape_returns_none(self):
with mock.patch("core.magazine.request_with_retries",
return_value=self._resp(json_body={"unexpected": "shape"})):
self.assertIsNone(vision_ocr_page(pathlib.Path(__file__), "http://engine/v1", "m"))
def test_output_with_zero_usable_regions_returns_none_not_empty_list(self):
"""An empty-but-truthy list would look like success to the caller."""
body = {"choices": [{"message": {"content": "no markers in here at all"}}]}
with mock.patch("core.magazine.request_with_retries",
return_value=self._resp(json_body=body)):
self.assertIsNone(vision_ocr_page(pathlib.Path(__file__), "http://engine/v1", "m"))
def test_missing_file_returns_none(self):
self.assertIsNone(vision_ocr_page(pathlib.Path("/no/such/file.png"),
"http://engine/v1", "m"))
class ContinuationMarkerStrippingTests(unittest.TestCase):
def test_a_print_layout_cue_is_removed_before_narration(self):
"""Spoken aloud this would read as literal parenthesised text -- a