diff --git a/CHANGELOG.md b/CHANGELOG.md index 685c54c..fb806f7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,18 +38,20 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi - **Articles nobody has reviewed are blue.** In the article list an unapproved, unrejected article has a blue background and edge and a *not reviewed* pill (approved keeps its green edge), so what still needs a person stands out. - **Overlapping boxes are fixed automatically.** Where a lower-ranked box runs over a higher-ranked one (headline > deck/byline > body > caption/sidebar/pull quote), the lower one is cut back to the largest part that no longer overlaps, unless that would leave under 40% of it (then both stay for a human). Hand-placed boxes and boxes of equal rank are never touched. - **Fewer false alarms and better fits:** a strip of solid ink beside a headline (a photo or filled panel behind it) is no longer counted as letters cut off; headline and deck boxes may grow up to 70 px sideways (body boxes 16 px); an inset pull-quote box that repeats text a box already holds becomes a duplicate. +- **Lines and frames are not missed text.** Beyond dotted leaders and thin rules, the audit now tells letters from lines by shape: text is many small separate marks, whereas a screenshot or chart frame, a vertical rule, a chart axis or a signature is a few large connected shapes (long and thin, or big and hollow). Only such shapes are ignored; a paragraph of text inside a frame, and a solid photo, are still found. Unboxed areas on the measured issue fell from 105 to 27. - **Rules and dotted leaders are not missed text.** The dots between a table-of-contents entry and its page number (and thin printed rules) are ink but no letters; the audit now ignores ink that is only a few pixel rows tall. - **Stacked boxes of a column now meet at the gap between two lines** (*seam mending*). Detection often cuts a column into slices that overlap or leave a line in the seam, so that line was in neither box and both reported a clipped edge. The shared boundary moves to the nearest blank gap (below the line for a paragraph that is still running, above it after a finished one, in the middle of an overlap); a line that had fallen into the seam is read on its own and attached to the box that now owns it, and the box takes the column's width. Page 32 of the measured issue: clipped edges 2 → 0 and the lost line "Entwicklungen auseinander. „KI ist bei uns in allen" is back in the text. 57 seams were mended across the issue. - **"Edge cuts text" needs the box's own lines.** Ink beside a left/right edge only counts as cut-off letters when it sits on the rows where the box itself has text and is intermittent (letters and gaps); a solid vertical line (a screenshot frame, a chart border, a photo edge) or ink between the box's lines does not, and a thin horizontal rule above or below is not a cut line. On the measured issue clipped edges fell from 82 to 48 and pages needing a decision from 53 to 42. - **Unboxed ink in or beside a picture, chart or advert is no longer reported as missed text** (labels of a chart, text set into a photo, an advert's own wording). -- **Undo and Redo buttons in the sticky review bar** (right after *Contents*), also on Ctrl+Z and Ctrl+Shift+Z / Ctrl+Y (not while typing in a text field, where the browser undoes the typing). The last 20 recorded changes are kept (compressed, in the magazine's `undo/` folder) and each button's tooltip names the change it would undo or redo. More changes are now recorded: approving/rejecting or editing an article, moving or resizing a region, drawing a region, re-reading a page, setting or releasing a page's reading order, *Fix reading order* and *Clean up boxes*, on top of region edits, linking, auto-fix and spelling corrections. A new change ends the redo history. As before, an undo (or redo) is refused if something else changed the magazine in between, so it can never roll back work it does not know about (`GET /history`, `POST /undo`, `POST /redo`). +- **Undo and Redo buttons in the sticky review bar**, at the left end of its second row (placed by measuring the bar, so they stay there at any window width), also on Ctrl+Z and Ctrl+Shift+Z / Ctrl+Y (not while typing in a text field, where the browser undoes the typing). The last 20 recorded changes are kept (compressed, in the magazine's `undo/` folder) and each button's tooltip names the change it would undo or redo. More changes are now recorded: approving/rejecting or editing an article, moving or resizing a region, drawing a region, re-reading a page, setting or releasing a page's reading order, *Fix reading order* and *Clean up boxes*, on top of region edits, linking, auto-fix and spelling corrections. A new change ends the redo history. As before, an undo (or redo) is refused if something else changed the magazine in between, so it can never roll back work it does not know about (`GET /history`, `POST /undo`, `POST /redo`). ### Changed - **The Contents button is the first item of the sticky review bar**, so it stays in reach while scrolling a long page. - **The page thumbnail is the Review button.** In the thumbnail grid, clicking the (big) page image opens the page in review; the separate *Review* / *Review this page* buttons are gone. A page shared by two or more articles keeps one labelled *Review* button per article, so it stays clear which is meant. ### Fixed +- **The new-box popup is no longer cut off.** Drawing a box near the bottom of a page put the *Region type / Add region* panel below the window edge. It is now placed in the window: below the box when it fits, above it otherwise, always fully inside, and it follows the box when the page scrolls. - **Changed scripts and stylesheets now reach the browser immediately.** They were requested under a version that only moved at release time (`…js?v=1.53.4`) and cached as immutable for a year, so a browser or proxy could keep running old code after an update (a button added to a script simply did not appear). The app shell now requests every asset under `-`, where the build id follows the newest script/stylesheet/manifest change. - **An approved page is no longer purple.** A page the quality check flagged stayed purple even after you approved its article; now your approval wins and the page shows its review colour (the warning badge on the card still lists what the check found). Purple means flagged *and* nobody has approved anything on the page. An old MP3 on an article nobody approved (left over from before pages were linked) no longer turns a page green either: audio only counts for approved articles. - **Reading order is now applied automatically after import, *OCR again* and every per-page fix**, not only by the manual *Fix reading order* button. Until now the order after import was whatever the vision/LLM step returned, so a column's short last box could be read after the whole other column; on the measured issue 101 of 204 pages had a saved order that differs from the column-aware order. Pages whose order you fixed by hand stay untouched, except after *OCR again*, when every region is new and a hand order can no longer apply. diff --git a/Manual.md b/Manual.md index f7e55f8..91784b4 100644 --- a/Manual.md +++ b/Manual.md @@ -298,7 +298,7 @@ unreviewed boundaries may include a neighbouring article or an advert. ### Undo and Redo **Purpose:** take back a change (or put it back) without leaving review. -**Steps:** use **Undo** / **Redo** in the sticky bar (next to *Contents*), or Ctrl+Z / Ctrl+Shift+Z (Ctrl+Y). The button tooltip names the change, e.g. "Undo: Deleted 2 regions". The last 20 recorded changes are kept per magazine: region edits and moves, drawing a region, approving/rejecting an article, page linking, auto-fix, reading-order changes, box clean-up and spelling corrections. +**Steps:** use **Undo** / **Redo** in the sticky bar (at the left of its second row), or Ctrl+Z / Ctrl+Shift+Z (Ctrl+Y). The button tooltip names the change, e.g. "Undo: Deleted 2 regions". The last 20 recorded changes are kept per magazine: region edits and moves, drawing a region, approving/rejecting an article, page linking, auto-fix, reading-order changes, box clean-up and spelling corrections. **Errors:** a button stays disabled when there is nothing to undo/redo, or when something else has changed the magazine since (for example a whole-magazine run), because undoing then would roll that work back too. A new change ends the redo history. **Privacy:** the saved states live in the magazine's own `undo/` folder; nothing is sent anywhere. diff --git a/core/magazine.py b/core/magazine.py index 0d02f0a..21cd2dc 100644 --- a/core/magazine.py +++ b/core/magazine.py @@ -2893,6 +2893,54 @@ def _is_rule_only(pixels, blob: Sequence[float], page_meta: dict[str, Any]) -> b return int((region.sum(axis=1) >= 3).sum()) <= _RULE_MAX_INK_ROWS +_LINE_MIN_LONG_PX, _LINE_MAX_THIN_PX = 45, 8 # a thin, long connected shape is a rule +_FRAME_MIN_LONG_PX, _FRAME_MAX_FILL = 90, 0.2 # a big, mostly hollow shape is a frame or a scribble +_GLYPH_MIN_PX, _GLYPH_MAX_PX, _GLYPH_MIN_INK = 4, 80, 6 +_MIN_GLYPHS, _MIN_GLYPH_SHARE = 6, 0.25 +_PICTURE_MIN_SIDE_PX, _PICTURE_MIN_FILL = 20, 0.5 + + +def _is_lines_or_frames_only(pixels, blob: Sequence[float], page_meta: dict[str, Any]) -> bool: + """True when the ink inside `blob` is rules, frames, chart axes or a signature rather than text. + + Text is many small separate marks (letters) sitting in rows. Rules, table and chart lines, + screenshot outlines and a signature are a few large connected shapes: long and thin, or big and + mostly hollow. The blob is dropped when fewer than six letter-sized marks are left, or they hold + under a quarter of the ink. Framed *text* is kept: its letters are many small marks. + """ + try: + import numpy as np + from scipy import ndimage + except Exception: + return False + height, width = pixels.shape + sx, sy = width / max(1.0, float(page_meta["width"])), height / max(1.0, float(page_meta["height"])) + x, y, w, h = blob + region = pixels[max(0, int(y * sy)):int((y + h) * sy) + 1, max(0, int(x * sx)):int((x + w) * sx) + 1] + if region.size == 0: + return False + labels, count = ndimage.label(region, structure=np.ones((3, 3), dtype=int)) + if count == 0: + return True + sizes = np.bincount(labels.ravel())[1:] + glyphs = glyph_ink = total = 0 + for index, sl in enumerate(ndimage.find_objects(labels), 1): + ink = int(sizes[index - 1]) + if ink < _GLYPH_MIN_INK: + continue + box_h, box_w = sl[0].stop - sl[0].start, sl[1].stop - sl[1].start + long_side, thin_side = max(box_h, box_w), min(box_h, box_w) + total += ink + if thin_side >= _PICTURE_MIN_SIDE_PX and ink / float(box_h * box_w) >= _PICTURE_MIN_FILL: + return False # a solid picture: the figure logic decides, not this filter + is_line = (long_side >= _LINE_MIN_LONG_PX and thin_side <= _LINE_MAX_THIN_PX) or ( + long_side >= _FRAME_MIN_LONG_PX and ink / float(box_h * box_w) < _FRAME_MAX_FILL) + if not is_line and _GLYPH_MIN_PX <= long_side <= _GLYPH_MAX_PX: + glyphs += 1 + glyph_ink += ink + return total == 0 or glyphs < _MIN_GLYPHS or glyph_ink / float(total) < _MIN_GLYPH_SHARE + + def _clipped_edges(pixels, page_meta: dict[str, Any], text_blocks: list[dict[str, Any]], boxes: list[Sequence[float]]) -> list[dict[str, Any]]: """Text boxes whose edge cuts through text: ink in a thin strip just outside a side, ignoring @@ -3740,7 +3788,7 @@ def audit_page_coverage(doc_dir: Path, layout: dict[str, Any], page_no: int) -> # graphic, not narratable text a box forgot: a blob in or beside one is not reported. if any(bbox_overlaps(blob, g, tol=_AUDIT_GRAPHIC_MARGIN) for g in graphic_boxes): continue - if pixels is not None and _is_rule_only(pixels, blob, page_meta): + if pixels is not None and (_is_rule_only(pixels, blob, page_meta) or _is_lines_or_frames_only(pixels, blob, page_meta)): continue # a printed rule or dotted leader, not letters kept_cells += len(rows) report["uncovered"].append(blob) diff --git a/static/js/magazine.js b/static/js/magazine.js index 60ca688..5bfc7ab 100644 --- a/static/js/magazine.js +++ b/static/js/magazine.js @@ -1616,13 +1616,6 @@ function renderReview() { // Back to the contents lives in the sticky bar, so it stays in reach while scrolling a long page. '' + - '
' + - '' + - '
' + // One page stepper for the whole magazine, not a separate one bounded // to just this article's own pages — the two almost always agreed and // having both doubled the toolbar's width for no real benefit. @@ -1653,7 +1646,15 @@ function renderReview() { '' + '' + - '' + + // Undo / Redo: magPlaceUndoRedo() moves this group to the start of the bar's second row. + '
' + + '' + + '
' + + '' + @@ -1756,6 +1757,7 @@ function renderReview() { magAuditLoad(); magFlowLoad(); magHistoryRefresh(); + magPlaceUndoRedo(); } // A Headline or Author region belonging to this article is exactly what its @@ -2155,6 +2157,38 @@ async function magBulk(action, type) { await magReloadReview(); } +// Undo / Redo belong at the left end of the bar's SECOND row, wherever the rows happen to break at +// this window width, so the group is placed by measuring: take it out, find the first control that +// sits below the first row, put the group in front of it (or at the end if the bar is one row). +function magPlaceUndoRedo() { + const bar = document.querySelector('#mag-review .mag-review-pagenav'); + const group = $('mag-undo-group'); + if (!bar || !group) return; + const place = () => { + group.remove(); + bar.querySelectorAll('.mag-pagenav-break').forEach((el) => el.remove()); + const items = Array.from(bar.children).filter((el) => !el.classList.contains('mag-review-pagenav-spacer')); + const firstTop = items.length ? items[0].offsetTop : 0; + const secondRowStart = items.find((el) => el.offsetTop > firstTop + 8); + if (!secondRowStart) { bar.appendChild(group); return; } // one row only: keep it at the end + // A forced line break in front of the group, so it opens the second row instead of sliding + // back into free space at the end of the first. + const lineBreak = document.createElement('span'); + lineBreak.className = 'mag-pagenav-break'; + lineBreak.setAttribute('aria-hidden', 'true'); + bar.insertBefore(lineBreak, secondRowStart); + bar.insertBefore(group, secondRowStart); + }; + place(); + if (!bar._magUndoObserver && typeof ResizeObserver === 'function') { + let width = bar.clientWidth; + bar._magUndoObserver = new ResizeObserver(() => { + if (bar.clientWidth !== width) { width = bar.clientWidth; place(); } + }); + bar._magUndoObserver.observe(bar); + } +} + // ── Undo / redo (server keeps the last 20 recorded changes) ─────────────────── function magHistoryPaint() { const h = state.history || {}; @@ -2783,13 +2817,40 @@ function magReviewOpenDrawPanel(box) { '' + ''; -wrap.appendChild(panel); - const top = parseFloat(box.style.top) || 0, left = parseFloat(box.style.left) || 0, height = parseFloat(box.style.height) || 0; - panel.style.top = Math.min(85, top + height + 1) + '%'; - panel.style.left = Math.min(60, left) + '%'; + wrap.appendChild(panel); + magPlaceDrawPanel(panel, box); + // The panel is placed in the window, not in the page image, so it follows the rectangle when the + // page scrolls or the window changes size, and goes away with its panel. + const follow = () => { + if (!panel.isConnected) { + window.removeEventListener('scroll', follow, true); + window.removeEventListener('resize', follow); + return; + } + magPlaceDrawPanel(panel, box); + }; + window.addEventListener('scroll', follow, true); + window.addEventListener('resize', follow); if ($('mag-draw-kind-picker')) _magSetupDrawKindPicker(drawKindOptions); } +// Below the drawn rectangle when it fits, above it when not, and always fully inside the window +// (a rectangle drawn near the bottom of the page used to push the panel past the edge, cutting off +// the Add region button). +function magPlaceDrawPanel(panel, box) { + const margin = 8; + const r = box.getBoundingClientRect(); + const width = Math.min(panel.offsetWidth, window.innerWidth - 2 * margin); + const height = Math.min(panel.offsetHeight, window.innerHeight - 2 * margin); + let top = r.bottom + 6; + if (top + height > window.innerHeight - margin) { + const above = r.top - height - 6; + top = above >= margin ? above : Math.max(margin, window.innerHeight - height - margin); + } + panel.style.top = Math.round(top) + 'px'; + panel.style.left = Math.round(Math.min(Math.max(margin, r.left), window.innerWidth - width - margin)) + 'px'; +} + // A 16-option