feat(magazine): live per-page issue markers and panel, one-click page auto-fix, run fixes boxes too

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
mARTin-B78 2026-09-28 10:41:39 +02:00
parent efaab0590e
commit b85f40e263
8 changed files with 419 additions and 6 deletions

View File

@ -18,6 +18,9 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
- **A paragraph that wraps around an inset pull quote is cut into a full-width part and a narrow part**, so its box no longer sits over the quote (as the right-hand column already did). The text lines are measured from the page image; the text is cut in proportion to line width at a space, within a word or two of the printed break, and the two pieces are read in order and rejoined for narration. A paragraph whose lines are all narrow just gets a narrower box. Runs after **OCR again (this page)** and in **Clean up boxes**; new pieces join the same articles. - **A paragraph that wraps around an inset pull quote is cut into a full-width part and a narrow part**, so its box no longer sits over the quote (as the right-hand column already did). The text lines are measured from the page image; the text is cut in proportion to line width at a space, within a word or two of the printed break, and the two pieces are read in order and rejoined for narration. A paragraph whose lines are all narrow just gets a narrower box. Runs after **OCR again (this page)** and in **Clean up boxes**; new pieces join the same articles.
- **Review shows what is wrong with each page, on the page.** A *What is wrong on this page* panel (measured live from the page render, not from the last run) explains each problem in plain language with its own fix button, and marks it on the boxes: red dashed frame = text outside any box, orange edge = a box edge cutting through text, purple dashed = two boxes over the same text, yellow dotted = words that look mis-read, red wavy underline = unknown words. **Fix this page automatically** fits boxes, adds inset pull quotes, cuts paragraphs that wrap around them, finds missed text, re-reads suspicious words and (with a language model) fixes provable misspellings, with one Undo.
- **The whole-magazine run now fixes as well as checks:** *Quality check / run* also fits boxes, finds inset pull quotes and cuts wrapped paragraphs on every page before the missed-text, re-read and spelling steps, and reports how many boxes it fixed.
### Fixed ### Fixed
- **The review list shows each region's whole text**, not the first 220 characters, so every misspelling gets its red wavy underline (like a word processor). The engine's `[Non-Text]` marker and hairline boxes (under 3 pt tall) are now treated as OCR noise by *Clean up boxes* and after a page re-OCR, so they no longer show up as near-invisible regions. Fitted boxes keep a little air on grown sides. - **The review list shows each region's whole text**, not the first 220 characters, so every misspelling gets its red wavy underline (like a word processor). The engine's `[Non-Text]` marker and hairline boxes (under 3 pt tall) are now treated as OCR noise by *Clean up boxes* and after a page re-OCR, so they no longer show up as near-invisible regions. Fitted boxes keep a little air on grown sides.
- **The quality check no longer flags correct text as "suspect OCR".** The word check guessed the language per fragment (`iPhones statt Token` read as English) and flagged valid German; it now uses the page's language, the same lenient German rules as the spellchecker, and treats words that recur on the page (names, brands) as vocabulary. Page 61 of the measured issue went from 11 suspect regions to 3 (one a genuinely truncated word), and the issue from 758 to 610 before the language fix's full effect. - **The quality check no longer flags correct text as "suspect OCR".** The word check guessed the language per fragment (`iPhones statt Token` read as English) and flagged valid German; it now uses the page's language, the same lenient German rules as the spellchecker, and treats words that recur on the page (names, brands) as vocabulary. Page 61 of the measured issue went from 11 suspect regions to 3 (one a genuinely truncated word), and the issue from 758 to 610 before the language fix's full effect.

View File

@ -282,6 +282,14 @@ the original for anything that matters.
Podcast generation is refused for an article you have not approved, because Podcast generation is refused for an article you have not approved, because
unreviewed boundaries may include a neighbouring article or an advert. unreviewed boundaries may include a neighbouring article or an advert.
### What is wrong on this page, and fixing it automatically
**Purpose:** see at a glance what is wrong with a page and fix it without hunting for the right button.
**Steps:** open a page in review. The **What is wrong on this page** panel lists each problem in plain language and marks it on the page: **red dashed frame** = printed text outside any box (never read); **orange edge** = a box edge cutting through text; **purple dashed outline** = two boxes over the same text; **yellow dotted outline** = words that look mis-read; **red wavy underline** (in the list) = words no dictionary knows. Each row has its own button (*Find missed text*, *Fit boxes to the text*, *Verify OCR text*, *Fix with language model*). **Fix this page automatically** does all of them in one go: fits boxes, adds an inset pull quote, cuts a paragraph that wraps around it, finds missed text, re-reads suspicious words and, when a language model is configured, fixes provable misspellings.
**Result:** the page reloads with what was fixed; **Undo** (above the list) restores the page as it was. Overlapping boxes have no automatic fix: delete or merge one.
**Whole magazine:** **Quality check / run** in the contents view now applies the same fixes to every page and then lists the pages that still need a look (purple badges).
**Errors:** a failed fix shows the reason and changes nothing. **Offline / privacy:** the box fixes and the audit are local; the OCR re-read and spelling steps use only the engines you configured.
### Clean up boxes: fitting boxes to the text ### Clean up boxes: fitting boxes to the text
**Clean up boxes** (review toolbar) also fits each text box to the ink of its text: a side that cuts through letters grows a little, empty margin is trimmed. Boxes you drew or resized are never changed; a backup of the layout is kept and the message reports how many boxes were fitted. **Clean up boxes** (review toolbar) also fits each text box to the ink of its text: a side that cuts through letters grows a little, empty margin is trimmed. Boxes you drew or resized are never changed; a backup of the layout is kept and the message reports how many boxes were fitted.

View File

@ -988,6 +988,36 @@ async def magazine_verify_ocr_text(doc_id: str, page: int):
_QUALITY_REPORT = "quality_report.json" _QUALITY_REPORT = "quality_report.json"
def _vision_args(settings: dict[str, Any]) -> tuple[str | None, str | None, str | None, str | None]:
def pick(key: str) -> str | None:
return str(settings.get(key) or "").strip() or None
return (pick("vision_ocr_url"), pick("vision_ocr_model"),
pick("vision_ocr_fallback_url"), pick("vision_ocr_fallback_model"))
def _fix_page_geometry(d: Path, layout: dict[str, Any], articles: list[dict[str, Any]], page: int,
settings: dict[str, Any]) -> tuple[dict[str, Any], dict[str, int], bool]:
"""The deterministic box fixes for one page, in the order that lets each rely on the last.
Retype noise/duplicates, add an inset pull quote, cut a paragraph that wraps around it, then
fit every text box to its ink (re-reading the ones that gained lines). Returns the layout,
counts of what was done, and whether the articles changed (new pieces join their article).
"""
url, model, fb_url, fb_model = _vision_args(settings)
layout, cleanup = mag.clean_up_regions(layout, pages=[page])
noise = len(cleanup.get("ocr_noise", [])) + len(cleanup.get("duplicate", [])) + len(cleanup.get("in_picture", []))
layout, quotes = mag.detect_inset_pullquotes(d, layout, [page], url, model, fb_url, fb_model)
layout, wrap = mag.split_around_pullquotes(d, layout, [page])
cleanup["wrap_split"] = wrap
layout, snapped = mag.snap_text_boxes(d, layout, pages=[page])
if url and model and any(m.get("lines_added") for m in snapped):
layout, _ = mag.reread_grown_regions(d, layout, snapped, url, model, fb_url, fb_model)
changed = _add_split_paragraphs_to_articles(articles, cleanup)
counts = {"noise_retyped": noise, "pullquotes_found": len(quotes), "paragraphs_cut": len(wrap),
"boxes_fitted": len(snapped), "bylines_split": len(cleanup.get("byline_split", []))}
return layout, counts, changed
def _quality_run_sync(doc_id: str, job_id: str, settings: dict[str, Any]) -> None: def _quality_run_sync(doc_id: str, job_id: str, settings: dict[str, Any]) -> None:
"""Whole-issue completeness run: find missed text, verify OCR, then measure coverage. """Whole-issue completeness run: find missed text, verify OCR, then measure coverage.
@ -1009,6 +1039,9 @@ def _quality_run_sync(doc_id: str, job_id: str, settings: dict[str, Any]) -> Non
agent = bool(str(settings.get("llm_model") or "").strip()) agent = bool(str(settings.get("llm_model") or "").strip())
issue_language = _issue_language(d, layout) issue_language = _issue_language(d, layout)
pages = [meta["page"] for meta in layout.get("pages", [])] pages = [meta["page"] for meta in layout.get("pages", [])]
articles = mag.load_articles(d)
articles_changed = False
fixed_boxes = 0
rows: list[dict[str, Any]] = [] rows: list[dict[str, Any]] = []
for i, page in enumerate(pages, 1): for i, page in enumerate(pages, 1):
def step(text: str, page: int = page, i: int = i) -> None: def step(text: str, page: int = page, i: int = i) -> None:
@ -1025,6 +1058,10 @@ def _quality_run_sync(doc_id: str, job_id: str, settings: dict[str, Any]) -> Non
"needs_review": False}) "needs_review": False})
continue continue
try: try:
step("fixing boxes")
layout, geometry, moved_articles = _fix_page_geometry(d, layout, articles, page, settings)
articles_changed = articles_changed or moved_articles
fixed_boxes += geometry["boxes_fitted"] + geometry["paragraphs_cut"] + geometry["pullquotes_found"]
step("finding missed text") step("finding missed text")
layout, added = mag.recover_missing_text(d, layout, page) layout, added = mag.recover_missing_text(d, layout, page)
if verify: if verify:
@ -1049,7 +1086,8 @@ def _quality_run_sync(doc_id: str, job_id: str, settings: dict[str, Any]) -> Non
if i % 5 == 0: if i % 5 == 0:
mag.save_layout(d, layout) mag.save_layout(d, layout)
mag.save_layout(d, layout) mag.save_layout(d, layout)
articles = mag.load_articles(d) if articles_changed:
mag.save_articles(d, articles)
duplicates = mag.duplicate_article_groups(articles) duplicates = mag.duplicate_article_groups(articles)
by_id = {a["id"]: a for a in articles} by_id = {a["id"]: a for a in articles}
for row in rows: for row in rows:
@ -1061,6 +1099,7 @@ def _quality_run_sync(doc_id: str, job_id: str, settings: dict[str, Any]) -> Non
summary = { summary = {
"pages": len(rows), "pages": len(rows),
"verified_with_vision": verify, "verified_with_vision": verify,
"boxes_fixed": fixed_boxes,
"regions_added": sum(r["added"] for r in rows), "regions_added": sum(r["added"] for r in rows),
"regions_corrected": sum(r["corrected"] for r in rows), "regions_corrected": sum(r["corrected"] for r in rows),
"spelling_fixed": sum(r.get("spelling_fixed", 0) for r in rows), "spelling_fixed": sum(r.get("spelling_fixed", 0) for r in rows),
@ -1074,7 +1113,7 @@ def _quality_run_sync(doc_id: str, job_id: str, settings: dict[str, Any]) -> Non
flagged = len(summary["pages_needing_review"]) flagged = len(summary["pages_needing_review"])
background_job_update( background_job_update(
job_id, stage="done", done=True, result=summary, job_id, stage="done", done=True, result=summary,
detail=(f"{summary['regions_added']} missed regions added, " detail=(f"{summary['boxes_fixed']} boxes fixed, {summary['regions_added']} missed regions added, "
f"{summary['regions_corrected']} words corrected, " f"{summary['regions_corrected']} words corrected, "
f"{summary['spelling_fixed']} misspellings fixed, {summary['spelling_marked']} marked; " f"{summary['spelling_fixed']} misspellings fixed, {summary['spelling_marked']} marked; "
f"{flagged} page{'s' if flagged != 1 else ''} still need a look" f"{flagged} page{'s' if flagged != 1 else ''} still need a look"
@ -1093,6 +1132,73 @@ async def magazine_quality_run(doc_id: str):
return {"job_id": job_id} return {"job_id": job_id}
@router.get("/api/magazine/{doc_id}/pages/{page}/audit")
async def magazine_page_audit(doc_id: str, page: int):
"""What is wrong with one page right now, measured live from its render (not the last run)."""
d, _ = _require_magazine(doc_id)
layout = mag.load_layout(d)
meta = next((p for p in (layout or {}).get("pages", []) if p["page"] == page), None)
if meta is None:
raise HTTPException(404, "Page not found")
audit = await asyncio.to_thread(mag.audit_page_coverage, d, layout, page)
spell = await asyncio.to_thread(_spelling_findings, d, layout, None, page, _issue_language(d, layout))
audit["spelling"] = len(spell)
audit["advertisement"] = mag.page_is_advertisement(layout, page)
return audit
def _autofix_page_sync(doc_id: str, job_id: str, page: int, mode: str, settings: dict[str, Any]) -> None:
"""Fix one page: boxes always; missed text, OCR verification and spelling too unless mode is 'boxes'."""
d = _LIB / doc_id
try:
layout = mag.load_layout(d)
if not layout:
raise FileNotFoundError("No layout yet -- run ingestion first")
articles = mag.load_articles(d)
token = _snapshot_for_undo(d, f"Auto-fixed page {page}")
background_job_update(job_id, stage="fixing", detail=f"Fixing the boxes on page {page}")
layout, done, moved = _fix_page_geometry(d, layout, articles, page, settings)
if mode != "boxes":
url, model, fb_url, fb_model = _vision_args(settings)
background_job_update(job_id, stage="fixing", detail="Looking for missed text")
layout, done["regions_added"] = mag.recover_missing_text(d, layout, page)
if url and model:
background_job_update(job_id, stage="fixing", detail="Re-reading suspicious words")
layout, _, done["words_corrected"] = mag.verify_ocr_recognition(d, layout, page, url, model, fb_url, fb_model)
if str(settings.get("llm_model") or "").strip():
background_job_update(job_id, stage="fixing", detail="Checking spelling")
outcome = spelling_agent_pass(d, layout, settings, page=page, language=_issue_language(d, layout))
layout, done["misspellings_fixed"] = outcome["layout"], outcome["fixed"]
mag.save_layout(d, layout)
if moved:
mag.save_articles(d, articles)
_seal_undo(d, token)
done["undo"] = token
fixed = sum(v for k, v in done.items() if k != "undo" and isinstance(v, int))
background_job_update(job_id, stage="done", done=True, result=done,
detail=f"Page {page}: {fixed} fix{'es' if fixed != 1 else ''} applied")
except Exception as exc: # noqa: BLE001 - surfaced to UI
background_job_update(job_id, stage="failed", done=True, error=str(exc)[:1000])
@router.post("/api/magazine/{doc_id}/pages/{page}/autofix")
async def magazine_autofix_page(doc_id: str, page: int, request: Request):
"""Background job that fixes what can be fixed on one page. Body: {mode: "all" | "boxes"}."""
d, _ = _require_magazine(doc_id)
layout = mag.load_layout(d)
if not any(p["page"] == page for p in (layout or {}).get("pages", [])):
raise HTTPException(404, "Page not found")
try:
body = await request.json()
except ValueError:
body = {}
mode = "boxes" if isinstance(body, dict) and body.get("mode") == "boxes" else "all"
job_id = uuid.uuid4().hex
background_job_create(job_id, "magazine-autofix", fingerprint=f"{doc_id}:{page}")
asyncio.create_task(asyncio.to_thread(_autofix_page_sync, doc_id, job_id, page, mode, _load_settings()))
return {"job_id": job_id}
@router.get("/api/magazine/{doc_id}/quality/report") @router.get("/api/magazine/{doc_id}/quality/report")
async def magazine_quality_report(doc_id: str): async def magazine_quality_report(doc_id: str):
"""Result of the last whole-issue quality run (404 if none yet).""" """Result of the last whole-issue quality run (404 if none yet)."""

View File

@ -43,6 +43,7 @@ const state = {
spell: null, // last spelling findings shown (page or article scope) spell: null, // last spelling findings shown (page or article scope)
spellByBlock: new Map(), spellByBlock: new Map(),
spellOpen: false, spellOpen: false,
audit: null, // live audit of the reviewed page ({page, data})
acronyms: null, // the open article's glossary state acronyms: null, // the open article's glossary state
queue: [], // article ids for "play all" (empty = not queued) queue: [], // article ids for "play all" (empty = not queued)
queueIndex: -1, queueIndex: -1,
@ -1650,7 +1651,8 @@ function renderReview() {
'<span class="mdi mdi-spellcheck" aria-hidden="true"></span> Spelling <span id="mag-spell-count" class="mag-pill"></span></button>' + '<span class="mdi mdi-spellcheck" aria-hidden="true"></span> Spelling <span id="mag-spell-count" class="mag-pill"></span></button>' +
'<span id="mag-reocr-status" class="muted" role="status"></span>' + '<span id="mag-reocr-status" class="muted" role="status"></span>' +
'</div>' + '</div>' +
'<div id="mag-spell-panel" class="mag-spell-panel" ' + (state.spellOpen ? '' : 'hidden') + '></div>'; '<div id="mag-spell-panel" class="mag-spell-panel" ' + (state.spellOpen ? '' : 'hidden') + '></div>' +
'<div id="mag-issues-panel" class="mag-issues-panel" role="region" aria-label="What is wrong on this page" aria-live="polite"></div>';
html += '<div class="mag-review-fields">'; html += '<div class="mag-review-fields">';
html += '<div class="mag-review-field-row">' + html += '<div class="mag-review-field-row">' +
@ -1726,6 +1728,7 @@ function renderReview() {
setupMagBoxEdit(); setupMagBoxEdit();
magReviewAutofillTitleAuthor(); magReviewAutofillTitleAuthor();
magSpellLoadPage(); magSpellLoadPage();
magAuditLoad();
} }
// A Headline or Author region belonging to this article is exactly what its // A Headline or Author region belonging to this article is exactly what its
@ -3063,9 +3066,9 @@ function magRenderQualityReport(report) {
async function magQualityRun() { async function magQualityRun() {
if (!state.doc) return; if (!state.doc) return;
const ok = await confirmDialog( const ok = await confirmDialog(
'Scan every page for text no box covers and re-read suspicious OCR? This can take a while on a long issue. ' + 'Fix every page automatically (fit boxes, find inset pull quotes, add missed text, re-read suspicious words, fix provable misspellings) and list what still needs a look? This can take a while on a long issue. ' +
'Added regions start unticked, so nothing new is read aloud until you approve it.', 'Added regions start unticked, so nothing new is read aloud until you approve it.',
{ title: 'Quality check the whole magazine?', okLabel: 'Run check' }); { title: 'Auto-fix and check the whole magazine?', okLabel: 'Run' });
if (!ok) return; if (!ok) return;
const btn = $('mag-quality-run-btn'); const btn = $('mag-quality-run-btn');
const box = $('mag-quality-report'); const box = $('mag-quality-report');
@ -3180,6 +3183,148 @@ async function magSetStatus(status) {
} }
} }
// ── What is wrong on this page: markers on the boxes + one-click fixes ──────────
// The audit is measured live from the page render (GET .../pages/N/audit), not read from the last
// whole-issue run, so it always matches what is on screen. Each kind of problem has its own
// marker on the page and its own plain-language row with the button that fixes it.
let _magAuditSeq = 0;
const MAG_ISSUE_KINDS = [
{ key: 'uncovered', cls: 'is-uncovered', label: 'Text outside any box',
what: (n) => n + ' area' + (n === 1 ? '' : 's') + ' of printed text ' + (n === 1 ? 'is' : 'are') + ' not inside any box, so ' + (n === 1 ? 'it' : 'they') + ' would never be read (red dashed frame).',
fix: 'Find missed text', run: "magRunPageQuality('missing')" },
{ key: 'clipped', cls: 'is-issue-clipped', label: 'Edge cuts text',
what: (n) => n + ' box edge' + (n === 1 ? '' : 's') + ' cut' + (n === 1 ? 's' : '') + ' through the text, so letters or words at the border can be lost (orange edge).',
fix: 'Fit boxes to the text', run: "magAutofixPage('boxes')" },
{ key: 'overlaps', cls: 'is-issue-overlap', label: 'Overlaps another box',
what: (n) => n + ' pair' + (n === 1 ? '' : 's') + ' of boxes cover the same text, so it may be read twice (purple dashed outline).',
fix: null, hint: 'Delete or merge one of the two (Select boxes, then Delete), or run Clean up boxes.' },
{ key: 'suspect_blocks', cls: 'is-issue-suspect', label: 'Words look mis-read',
what: (n) => n + ' box' + (n === 1 ? ' has' : 'es have') + ' words that look like OCR mistakes (yellow dotted outline).',
fix: 'Verify OCR text', run: "magRunPageQuality('recognition')" },
{ key: 'spelling', cls: null, label: 'Unknown words',
what: (n) => n + ' word' + (n === 1 ? '' : 's') + ' no German or English dictionary knows (red wavy underline in the list).',
fix: 'Fix with language model', run: 'magSpellOpenAndFix()' },
];
function magIssueCount(audit, key) {
const v = audit[key];
return Array.isArray(v) ? v.length : (Number(v) || 0);
}
async function magAuditLoad() {
const page = state.reviewPage;
if (page == null || !state.doc) return;
const seq = ++_magAuditSeq;
const panel = $('mag-issues-panel');
if (panel && !(state.audit && state.audit.page === page)) panel.innerHTML = '<p class="muted">Checking this page…</p>';
let data;
try { data = await api('/api/magazine/' + state.doc.id + '/pages/' + page + '/audit'); } catch (_) {
if (panel) panel.innerHTML = '';
return;
}
if (seq !== _magAuditSeq || state.view !== 'review') return;
state.audit = { page, data };
magAuditPaint();
}
function magAuditPaint() {
const wrap = $('mag-review-page-wrap');
const audit = state.audit && state.audit.page === state.reviewPage ? state.audit.data : null;
document.querySelectorAll('#mag-review .mag-issue-box').forEach((el) => el.remove());
document.querySelectorAll('#mag-review .mag-review-box').forEach((el) => {
el.classList.remove('is-issue-clipped', 'is-issue-overlap', 'is-issue-suspect',
'side-left', 'side-right', 'side-top', 'side-bottom');
el.removeAttribute('data-issue');
});
const panel = $('mag-issues-panel');
if (!audit || !panel) return;
const boxOf = (id) => document.querySelector('#mag-review-page-wrap .mag-review-box[data-block="' + CSS.escape(id) + '"]');
const tag = (el, cls, text) => {
if (!el) return;
el.classList.add(cls);
el.setAttribute('data-issue', [el.getAttribute('data-issue'), text].filter(Boolean).join(' · '));
};
(audit.clipped || []).forEach((c) => {
const el = boxOf(c.id);
tag(el, 'is-issue-clipped', 'edge cuts text');
if (el) el.classList.add('side-' + c.side);
});
(audit.overlaps || []).forEach(([a, b]) => { tag(boxOf(a), 'is-issue-overlap', 'overlaps another box'); tag(boxOf(b), 'is-issue-overlap', 'overlaps another box'); });
(audit.suspect_blocks || []).forEach((id) => tag(boxOf(id), 'is-issue-suspect', 'words look mis-read'));
const meta = (state.layout?.pages || []).find((p) => p.page === state.reviewPage);
if (wrap && meta) {
(audit.uncovered || []).forEach((b) => {
const el = document.createElement('div');
el.className = 'mag-issue-box is-uncovered';
el.setAttribute('data-issue', 'text outside any box');
el.style.cssText = 'left:' + (b[0] / meta.width * 100).toFixed(2) + '%;top:' + (b[1] / meta.height * 100).toFixed(2) +
'%;width:' + (b[2] / meta.width * 100).toFixed(2) + '%;height:' + (b[3] / meta.height * 100).toFixed(2) + '%';
wrap.appendChild(el);
});
}
const found = MAG_ISSUE_KINDS.filter((k) => magIssueCount(audit, k.key) > 0);
const total = found.reduce((n, k) => n + magIssueCount(audit, k.key), 0);
let html = '<div class="mag-issues-head"><strong>' + (total ? 'What is wrong on this page' : '✓ Nothing wrong found on this page') + '</strong>' +
'<button type="button" class="btn-sm primary" id="mag-autofix-btn" onclick="magAutofixPage(\'all\')" ' +
'title="Fits boxes to their text, adds inset pull quotes, cuts paragraphs that wrap around them, finds missed text, re-reads suspicious words and (with a language model) fixes provable misspellings. One Undo brings the page back.">' +
'<span class="mdi mdi-auto-fix" aria-hidden="true"></span> Fix this page automatically</button>' +
'<span id="mag-issues-status" class="muted" role="status"></span></div>';
if (total) {
html += '<ul class="mag-issues-list">' + found.map((k) => {
const n = magIssueCount(audit, k.key);
return '<li><span class="mag-issue-swatch ' + (k.cls || 'is-spell') + '" aria-hidden="true"></span>' +
'<span class="mag-issues-text"><strong>' + esc(k.label) + '</strong> — ' + esc(k.what(n)) +
(k.fix ? '' : ' <span class="muted">' + esc(k.hint) + '</span>') + '</span>' +
(k.fix ? '<button type="button" class="btn-sm" onclick="' + k.run + '">' + esc(k.fix) + '</button>' : '') + '</li>';
}).join('') + '</ul>';
}
if (audit.advertisement) html += '<p class="muted">This page is an advertisement: nothing on it is read aloud.</p>';
panel.innerHTML = html;
}
async function magAutofixPage(mode) {
const page = state.reviewPage;
if (page == null) return;
const btn = $('mag-autofix-btn');
const status = () => $('mag-issues-status');
if (btn) btn.disabled = true;
if (status()) status().textContent = 'Starting…';
let jobId;
try {
jobId = (await api('/api/magazine/' + state.doc.id + '/pages/' + page + '/autofix', {
method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ mode }),
})).job_id;
} catch (e) {
if (btn) btn.disabled = false;
return say('Could not start the fix: ' + e.message, 'err');
}
let job;
for (;;) {
try { job = await api('/api/jobs/' + jobId); } catch (_) { job = null; break; }
if (job.done) break;
if (status() && job.detail) status().textContent = job.detail;
await new Promise((r) => setTimeout(r, 1500));
}
if (btn) btn.disabled = false;
if (!job || job.error) return say('Fixing the page failed: ' + (job && job.error || 'lost the job'), 'err');
say(job.detail || 'Page fixed', 'ok');
if (job.result && job.result.undo) state.undo = { token: job.result.undo, label: 'Auto-fixed page ' + page };
state.layout = null;
await ensureLayout();
const articleId = state.article.id;
await magRefreshArticles();
state.article = state.articles.find((a) => a.id === articleId) || state.article;
_magReviewChecked = null;
renderReview();
}
async function magSpellOpenAndFix() {
if (!state.spellOpen) magSpellToggle();
await magSpellAgent();
}
// ── Spelling and pronunciation checks ──────────────────────────────────────── // ── Spelling and pronunciation checks ────────────────────────────────────────
// Spelling: dictionary-unknown words are MARKED (review list) and listed with // Spelling: dictionary-unknown words are MARKED (review list) and listed with
// suggestions; nothing is rewritten unless the reviewer picks a suggestion or the // suggestions; nothing is rewritten unless the reviewer picks a suggestion or the
@ -4684,6 +4829,8 @@ root.magDrivingCycleSpeed = magDrivingCycleSpeed;
root.magDrivingToggleText = magDrivingToggleText; root.magDrivingToggleText = magDrivingToggleText;
root.magDrivingToggleImages = magDrivingToggleImages; root.magDrivingToggleImages = magDrivingToggleImages;
root.magDrivingPrevious = magDrivingPrevious; root.magDrivingPrevious = magDrivingPrevious;
root.magAutofixPage = magAutofixPage;
root.magSpellOpenAndFix = magSpellOpenAndFix;
root.magSpellToggle = magSpellToggle; root.magSpellToggle = magSpellToggle;
root.magSpellFix = magSpellFix; root.magSpellFix = magSpellFix;
root.magSpellKeep = magSpellKeep; root.magSpellKeep = magSpellKeep;

View File

@ -65,7 +65,7 @@
onkeydown="if(event.key==='Enter'){this.blur();}"> onkeydown="if(event.key==='Enter'){this.blur();}">
<div class="mag-bar-actions"> <div class="mag-bar-actions">
<button type="button" class="btn-sm" id="mag-quality-run-btn" onclick="magQualityRun()" <button type="button" class="btn-sm" id="mag-quality-run-btn" onclick="magQualityRun()"
title="Check every page: add text no box covers, re-read suspicious OCR, and list pages that still need a look"> title="Fix every page automatically (boxes, inset pull quotes, missed text, misread words, spelling) and list the pages that still need a look">
<span class="mdi mdi-clipboard-check-outline"></span> <span class="mag-qrun-label">Quality check / run</span> <span class="mdi mdi-clipboard-check-outline"></span> <span class="mag-qrun-label">Quality check / run</span>
</button> </button>
<button type="button" class="btn-sm" id="mag-contents-view-toggle" onclick="magToggleContentsView()" title="Switch between an article list and a page-by-page thumbnail grid"> <button type="button" class="btn-sm" id="mag-contents-view-toggle" onclick="magToggleContentsView()" title="Switch between an article list and a page-by-page thumbnail grid">

View File

@ -5472,3 +5472,29 @@ input[list="lang-codes"] {
.mag-acr-table thead { display: none; } .mag-acr-table thead { display: none; }
.mag-acr-table tr { display: grid; gap: 2px; padding: 6px 0; border-bottom: 1px solid var(--border, #ddd); } .mag-acr-table tr { display: grid; gap: 2px; padding: 6px 0; border-bottom: 1px solid var(--border, #ddd); }
} }
/* Magazine review: what is wrong on this page */
.mag-issues-panel { margin: 10px 0; padding: 10px 12px; border: 1px solid var(--border); border-radius: 8px; background: var(--surface); }
.mag-issues-head { display: flex; flex-wrap: wrap; align-items: center; gap: 10px; }
.mag-issues-list { list-style: none; margin: 8px 0 0; padding: 0; display: grid; gap: 6px; }
.mag-issues-list li { display: flex; flex-wrap: wrap; align-items: center; gap: 8px; }
.mag-issues-text { flex: 1 1 320px; min-width: 0; }
.mag-issue-swatch { width: 18px; height: 14px; border-radius: 3px; flex: 0 0 auto; border: 3px solid transparent; }
.mag-issue-swatch.is-uncovered { border: 2px dashed #dc2626; }
.mag-issue-swatch.is-issue-clipped { border-right: 5px solid #f97316; border-top-color: #bbb; border-bottom-color: #bbb; border-left-color: #bbb; }
.mag-issue-swatch.is-issue-overlap { border: 2px dashed #9333ea; }
.mag-issue-swatch.is-issue-suspect { border: 2px dotted #ca8a04; }
.mag-issue-swatch.is-spell { border: 0; background: repeating-linear-gradient(135deg, transparent 0 3px, #d93025 3px 5px); height: 4px; margin-top: 8px; }
.mag-issue-box { position: absolute; pointer-events: none; z-index: 3; box-sizing: border-box; }
.mag-issue-box.is-uncovered { border: 3px dashed #dc2626; background: rgba(220,38,38,.10); }
.mag-issue-box::after, .mag-review-box[data-issue]::after { content: attr(data-issue); position: absolute; left: -2px; bottom: 100%; max-width: 240px;
padding: 1px 5px; font-size: 11px; line-height: 1.3; color: #fff; background: #b91c1c; border-radius: 3px 3px 0 0; white-space: nowrap; pointer-events: none; }
.mag-review-box.is-issue-clipped[data-issue]::after { background: #c2410c; }
.mag-review-box.is-issue-overlap[data-issue]::after { background: #7e22ce; }
.mag-review-box.is-issue-suspect[data-issue]::after { background: #a16207; }
.mag-review-box.is-issue-overlap { outline: 3px dashed #9333ea; outline-offset: 1px; }
.mag-review-box.is-issue-suspect { outline: 3px dotted #ca8a04; outline-offset: 1px; }
.mag-review-box.is-issue-clipped.side-right { box-shadow: 5px 0 0 0 #f97316; }
.mag-review-box.is-issue-clipped.side-left { box-shadow: -5px 0 0 0 #f97316; }
.mag-review-box.is-issue-clipped.side-top { box-shadow: 0 -5px 0 0 #f97316; }
.mag-review-box.is-issue-clipped.side-bottom { box-shadow: 0 5px 0 0 #f97316; }

View File

@ -85,6 +85,11 @@ async function mockBackend(page, { articleStatus = 'detected', hasAudio = false,
body: silentWav(2000), contentType: 'audio/wav', body: silentWav(2000), contentType: 'audio/wav',
})); }));
await page.route(`**/api/magazine/${DOC_ID}/progress`, route => route.fulfill({ json: { ok: true } })); await page.route(`**/api/magazine/${DOC_ID}/progress`, route => route.fulfill({ json: { ok: true } }));
// Reads the review screen makes on its own; tests that care override these routes.
await page.route(`**/api/magazine/${DOC_ID}/pages/*/audit`, route => route.fulfill({
json: { page: 3, uncovered: [], uncovered_ratio: 0, suspect_blocks: [], overlaps: [], clipped: [], measured: true, spelling: 0, advertisement: false },
}));
await page.route(`**/api/magazine/${DOC_ID}/spelling**`, route => route.fulfill({ json: { issues: [], dictionary: {} } }));
await page.route(`**/api/magazine/${DOC_ID}/pages/*`, route => route.fulfill({ await page.route(`**/api/magazine/${DOC_ID}/pages/*`, route => route.fulfill({
// 1x1 PNG — only geometry/wiring is under test, not real page rendering. // 1x1 PNG — only geometry/wiring is under test, not real page rendering.
body: Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=', 'base64'), body: Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=', 'base64'),
@ -457,6 +462,59 @@ test.describe('Magazine pronunciation and spelling checks', () => {
}); });
}); });
test.describe('Magazine: what is wrong on this page', () => {
async function openReview(page) {
await page.goto('/', { waitUntil: 'domcontentloaded' });
await page.waitForSelector('#app-skeleton', { state: 'detached', timeout: 15_000 });
await navTo(page, 's-magazine');
await page.locator('#mag-library-list .mag-card').first().click();
await page.locator('#mag-contents-view-toggle').click();
await page.locator('.mag-card-wrap').filter({ hasText: 'Page 3' })
.getByRole('button', { name: 'Review', exact: true }).click();
}
test('each problem gets its own marker on the page and a plain-language row with a fix', async ({ page }) => {
await mockBackend(page);
await page.route(`**/api/magazine/${DOC_ID}/pages/*/audit`, route => route.fulfill({ json: {
page: 3, uncovered: [[300, 200, 100, 50]], uncovered_ratio: 0.02, suspect_blocks: ['b3'], overlaps: [['b1', 'b2']],
clipped: [{ id: 'b3', side: 'right' }], measured: true, spelling: 2, advertisement: false } }));
await openReview(page);
const panel = page.locator('#mag-issues-panel');
await expect(panel).toContainText('What is wrong on this page');
await expect(panel).toContainText('Text outside any box');
await expect(panel).toContainText('Edge cuts text');
await expect(panel).toContainText('Overlaps another box');
await expect(panel).toContainText('Words look mis-read');
await expect(panel).toContainText('Unknown words');
await expect(page.locator('#mag-review-page-wrap .mag-issue-box.is-uncovered')).toHaveCount(1);
await expect(page.locator('#mag-review-page-wrap .mag-review-box.is-issue-clipped.side-right')).toHaveCount(1);
await expect(page.locator('#mag-review-page-wrap .mag-review-box.is-issue-overlap')).toHaveCount(2);
await expect(page.locator('#mag-review-page-wrap .mag-review-box.is-issue-suspect')).toHaveCount(1);
});
test('Fix this page automatically starts the job, offers Undo and reloads the page', async ({ page }) => {
await mockBackend(page);
let started = null;
await page.route(`**/api/magazine/${DOC_ID}/pages/3/autofix`, (route) => {
started = route.request().postDataJSON();
return route.fulfill({ json: { job_id: 'fix1' } });
});
await page.route('**/api/jobs/fix1', route => route.fulfill({ json: {
done: true, detail: 'Page 3: 2 fixes applied', result: { boxes_fitted: 2, undo: 'tok' } } }));
await openReview(page);
await page.getByRole('button', { name: 'Fix this page automatically' }).click();
await expect.poll(() => started).toEqual({ mode: 'all' });
await expect(page.locator('#mag-undo-bar')).toContainText('Auto-fixed page 3');
});
test('a clean page says so', async ({ page }) => {
await mockBackend(page);
await openReview(page);
await expect(page.locator('#mag-issues-panel')).toContainText('Nothing wrong found on this page');
});
});
test('drawing a chart links the crop and excludes enclosed narration on approval', async ({ page }) => { test('drawing a chart links the crop and excludes enclosed narration on approval', async ({ page }) => {
await mockBackend(page); await mockBackend(page);
let cropped = false; let cropped = false;

View File

@ -1690,3 +1690,68 @@ class SpellingRouteTests(unittest.TestCase):
with patch('routes.magazine._load_settings', return_value={}), self.assertRaises(HTTPException) as ctx: with patch('routes.magazine._load_settings', return_value={}), self.assertRaises(HTTPException) as ctx:
self.call(magazine_spelling_agent, body={}) self.call(magazine_spelling_agent, body={})
self.assertEqual(ctx.exception.status_code, 400) self.assertEqual(ctx.exception.status_code, 400)
class PageAuditAndAutofixTests(unittest.TestCase):
def setUp(self):
from PIL import Image, ImageDraw
from core import magazine as mag
self.dir = Path(tempfile.mkdtemp(prefix="ttvc-autofix-"))
(self.dir / "pages").mkdir()
im = Image.new("RGB", (600, 800), (248, 242, 244))
draw = ImageDraw.Draw(im)
for y in range(100, 200, 10): # text x 100..300
draw.rectangle([100, y, 300, y + 5], fill=(93, 91, 92))
im.save(self.dir / "pages" / "0001.png")
# the box stops 20 px short of the text's right end: a clipped edge
mag.save_layout(self.dir, {"pages": [{"page": 1, "width": 600, "height": 800}], "images": [], "blocks": [
{"id": "b1", "page": 1, "type": "body", "reading_order": 0, "bbox": [98, 98, 182, 108],
"text": "Ein Absatz mit genug Text, damit er ordentlich gelesen wird.", "source": "vision_ocr", "region_label": "text"}]})
mag.save_articles(self.dir, [{"id": "a1", "blocks": ["b1"], "images": []}])
def tearDown(self):
shutil.rmtree(self.dir, ignore_errors=True)
def call(self, fn, *args, body=None):
import asyncio
from unittest.mock import AsyncMock, patch
request = AsyncMock(); request.json.return_value = body if body is not None else {}
with patch('routes.magazine._require_magazine', return_value=(self.dir, {})), \
patch('routes.magazine._load_settings', return_value={}), \
patch('routes.magazine.asyncio.to_thread', new=_run_sync), \
patch('routes.magazine.asyncio.create_task', side_effect=lambda coro: asyncio.ensure_future(coro)):
return asyncio.run(fn('a' * 32, *args, *([request] if body is not None else [])))
def test_the_live_audit_reports_the_clipped_edge(self):
from routes.magazine import magazine_page_audit
audit = self.call(magazine_page_audit, 1)
self.assertEqual(audit["clipped"], [{"id": "b1", "side": "right"}])
self.assertEqual(audit["spelling"], 0)
self.assertFalse(audit["advertisement"])
def test_the_audit_of_an_unknown_page_is_a_404(self):
from routes.magazine import magazine_page_audit
with self.assertRaises(HTTPException) as ctx:
self.call(magazine_page_audit, 9)
self.assertEqual(ctx.exception.status_code, 404)
def test_the_box_fix_removes_the_clipped_edge_and_can_be_undone(self):
from unittest.mock import patch
from core import magazine as mag
from routes.magazine import _autofix_page_sync, magazine_page_audit, magazine_undo
with patch('routes.magazine._LIB', self.dir.parent), patch('routes.magazine.background_job_update') as job:
_autofix_page_sync(self.dir.name, "j", 1, "boxes", {})
result = job.call_args.kwargs
self.assertTrue(result["done"], result)
self.assertGreaterEqual(result["result"]["boxes_fitted"], 1)
self.assertEqual(self.call(magazine_page_audit, 1)["clipped"], [])
self.call(magazine_undo, body={"token": result["result"]["undo"]})
self.assertEqual(mag.load_layout(self.dir)["blocks"][0]["bbox"], [98, 98, 182, 108])
def test_a_failure_is_reported_on_the_job_not_raised(self):
from unittest.mock import patch
from routes.magazine import _autofix_page_sync
(self.dir / "layout.json").unlink()
with patch('routes.magazine._LIB', self.dir.parent), patch('routes.magazine.background_job_update') as job:
_autofix_page_sync(self.dir.name, "j", 1, "all", {})
self.assertTrue(job.call_args.kwargs["done"] and job.call_args.kwargs["error"])