diff --git a/CHANGELOG.md b/CHANGELOG.md index dbd8831..0a628c4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,9 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi --- ## [Unreleased] +### Added +- **Turn a page that was scanned sideways or upside down.** *Turn left*, *Turn right* and *Turn 180°* in the review toolbar turn the page image, carry its boxes along so reviewed articles still match, and read the page again (`POST /api/magazine/{id}/pages/{page}/rotate`). The turn is remembered in the magazine's metadata and applied whenever pages are rendered, so re-importing does not undo it. If the re-read fails, the page goes back exactly as it was. Not covered by Undo (it changes the image): the opposite button turns it back. Measured on a real page scanned sideways: word-F1 0.016 as scanned, 0.876 after turning and re-reading. Page images now revalidate (`Cache-Control: no-cache`) so a turned page shows at once. + ### Changed - **Fixing flagged words in the review list no longer restyles the paragraph or hides the underlines.** Clicking an underlined word now edits just that word in place (a small field in the paragraph's own type, suggestion chips, Enter applies, Esc cancels, *Keep* for names), through the same checked one-word fix as the spelling list. Clicking elsewhere in the text opens the whole-text editor, which now keeps the paragraph's font size, line height, width and wrapping (it was 13 px in a full-row box against 14 px text in a narrower column), grows with its text instead of scrolling in a small box, and keeps the doubtful words underlined while you type. On a phone the row text and editors are both 16 px (what iOS needs to avoid zooming in). diff --git a/Manual.md b/Manual.md index e6512df..2cbd8b2 100644 --- a/Manual.md +++ b/Manual.md @@ -393,6 +393,32 @@ reads the text directly; the overlay is an addition, not a replacement. **Offline.** The file is self-contained. **Privacy.** It contains the magazine's text and your generated audio, so share it only where the magazine's licence allows. +### A page that was scanned sideways or upside down + +**Purpose.** Some scans arrive turned: the page looks sideways, and what OCR read from it is gibberish. + +**Prerequisites.** A vision OCR engine set up in Settings (the same one *OCR again* uses), because turning re-reads the page. + +**Steps.** Open the page in Review and press **Turn right** (90° clockwise), **Turn left** (90° counter-clockwise) or **Turn 180°**. +The app asks first, then turns the page image, reads the page again, and shows the result. Tick **two-page spread** to choose which +of the two pages the buttons act on, as with the other page tools. + +**Expected result.** The page stands upright, its boxes and text are new readings of the upright page, and the page's quality findings +are re-measured. On a test page, reading it sideways scored word-F1 0.02 and reading it after turning 0.88 (an ordinary page scores +about 0.88). The turn is remembered: re-running detection or re-importing does not flip the page back. + +**What is replaced.** The page's boxes and text are replaced by the new reading, so corrections you made to that page by hand are lost, and +articles that used the page go back to "not approved" where the new boxes do not line up with the old ones. **Undo does not cover a turn** +(it changes the page image); if you turned the wrong way, press the opposite button. + +**Errors.** Without a vision OCR engine nothing is changed and you are told why. If the engine is down or returns nothing, the page image is +put back exactly as it was and nothing is saved. A page that has a real text layer in the PDF is read from its image after a turn, because the +PDF's own text positions belong to the unturned page. + +**Mobile / accessibility.** The three buttons are real buttons in a labelled group, each with its direction in words, and the confirmation is a dialog you can answer with the keyboard. + +**Privacy.** The page image goes to the OCR engine you configured, exactly as for *OCR again*. + ### Editing recognised text, and seeing what you changed **Purpose.** Fix what OCR got wrong and always be able to see, and undo, what you changed. diff --git a/architecture/magazine-ingestion.md b/architecture/magazine-ingestion.md index a7fe4e7..dddcf7e 100644 --- a/architecture/magazine-ingestion.md +++ b/architecture/magazine-ingestion.md @@ -477,6 +477,22 @@ Verified end-to-end against a synthetic 8-page issue (contents page, two-column feature with a byline and a page-8 jump, full-page advert, second feature, reviews) and against an image-only rendering of the same file to exercise OCR. +## Turning a page scanned sideways + +`POST /api/magazine/{id}/pages/{page}/rotate {degrees: 90|180|270|-90}` starts the same background job as "OCR again" +(`_reocr_page_sync`) with a turn in front of it. The job turns the rendered page PNG (`rotate_page_image`, clockwise, via a temporary +file), carries the page's existing boxes and figure boxes to their new places (`layout_with_rotated_page` / `rotate_bbox`, which also swaps +the page's width and height for a quarter turn and accumulates `rotation` in the page's layout entry), and re-reads the page. Carrying the old +boxes matters: `remap_reviewed_page` decides which article a new region joins by overlap with the old ones, which would find none if the old +boxes were still in the sideways geometry. If the read fails, raises or returns nothing, the original PNG bytes are written back and neither +layout, articles nor metadata are saved. On success the total turn per page is stored in `meta.json` `page_rotations` (four quarter turns delete +the entry); `extract_pages(page_rotations=...)` applies it with `page.set_rotation` whenever a page is rendered, so re-ingesting does not flip the +page back, and on a turned page the PDF's own text and embedded-image boxes (unturned coordinates) are ignored in favour of OCR of the render. +A turn takes no undo snapshot: Undo restores layout and articles but not the image, which would put old-orientation boxes on the turned page. The page +image endpoint now answers `Cache-Control: no-cache` so a turned page shows at once. Measured on a real page scanned sideways (Baidu Unlimited-OCR): +word-F1 0.016 read as scanned, 0.876 after a turn and re-read. Limitation: an automatic "this page looks sideways" hint would need tesseract's +orientation data (`osd.traineddata`), which the image does not ship. + ## Import from page images `POST /api/magazine/{id}/source-images` (multipart `files`, in the order sent) builds `source.pdf` with diff --git a/core/magazine.py b/core/magazine.py index 95ff25e..cb12940 100644 --- a/core/magazine.py +++ b/core/magazine.py @@ -2485,9 +2485,13 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110, vision_ocr_fallback_model: str | None = None, vision_ocr_timeout: float = 150.0, force_vision_ocr: bool = False, - progress: Any = None) -> dict[str, Any]: + progress: Any = None, + page_rotations: dict[int, int] | None = None) -> dict[str, Any]: """Run the full extraction for one PDF, writing page renders and images. + `page_rotations` ({page: clockwise degrees}) are turns the reader applied to pages scanned sideways; a + rendered page is turned the same way whenever it is (re)rendered, so a re-import does not undo them. + When vision_ocr_url/model are given, a page whose text needs OCR (or whose already-extracted text scores as mostly garbage, native or tesseract alike -- see ocr_quality_score) is escalated to that engine @@ -2522,8 +2526,11 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110, for pno in range(doc.page_count): page = doc.load_page(pno) - rect = page.rect page_no = pno + 1 + extra_turn = int((page_rotations or {}).get(page_no, 0)) % 360 + if extra_turn: + page.set_rotation((page.rotation + extra_turn) % 360) + rect = page.rect png = pages_dir / f"{page_no:04d}.png" if not png.exists(): @@ -2532,7 +2539,9 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110, image_boxes: list[list[float]] = [] page_images: list[dict[str, Any]] = [] - for img_index, info in enumerate(page.get_images(full=True)): + # The PDF's own text and image boxes are in the page's unturned coordinates; on a page the reader + # turned, the render (and OCR of it) is the truth. + for img_index, info in enumerate([] if extra_turn else page.get_images(full=True)): rects: list[list[float]] = [] try: for r in page.get_image_rects(info[0]): @@ -2554,7 +2563,7 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110, }) blocks: list[Block] = [] - data = page.get_text("dict") + data = {} if extra_turn else page.get_text("dict") for bi, blk in enumerate(data.get("blocks", [])): if blk.get("type") != 0: # 0 = text, 1 = image continue @@ -2832,6 +2841,67 @@ def build_articles(layout: dict[str, Any]) -> list[dict[str, Any]]: return out +# ── Turning a page that was scanned sideways or upside down ─────────────────── + +ROTATIONS = (90, 180, 270) # clockwise degrees a reader can ask for + + +def normalise_rotation(degrees: Any) -> int: + """90, 180 or 270 (clockwise). -90 means 270; anything else is refused.""" + try: + d = int(degrees) % 360 + except (TypeError, ValueError): + raise ValueError("degrees must be 90, 180 or 270") from None + if d not in ROTATIONS: + raise ValueError("degrees must be 90, 180 or 270") + return d + + +def rotate_bbox(bbox: Sequence[float], width: float, height: float, degrees: int) -> list[float]: + """Where a [x, y, w, h] box on a page of width x height lands after the page is turned clockwise.""" + x, y, w, h = (float(v) for v in bbox) + d = normalise_rotation(degrees) + if d == 90: + return [height - (y + h), x, h, w] + if d == 180: + return [width - (x + w), height - (y + h), w, h] + return [y, width - (x + w), h, w] # 270 + + +def rotate_page_image(png: Path, degrees: int) -> None: + """Turn a rendered page clockwise in place (written to a temporary file first).""" + from PIL import Image + d = normalise_rotation(degrees) + op = {90: Image.Transpose.ROTATE_270, 180: Image.Transpose.ROTATE_180, 270: Image.Transpose.ROTATE_90}[d] + tmp = png.with_suffix(".rot.tmp") + try: + with Image.open(png) as im: + im.transpose(op).save(tmp, format="PNG") + tmp.replace(png) + finally: + tmp.unlink(missing_ok=True) + + +def layout_with_rotated_page(layout: dict[str, Any], page_no: int, degrees: int) -> dict[str, Any]: + """The layout with one page turned: its size swapped for a quarter turn, `rotation` accumulated, and the + boxes already on it carried to where they are now, so whatever was approved still lines up with them.""" + d = normalise_rotation(degrees) + meta = next((p for p in layout.get("pages", []) if p["page"] == page_no), None) + if meta is None: + raise ValueError("Page not found") + w, h = float(meta["width"]), float(meta["height"]) + out = dict(layout) + out["pages"] = [ + ({**p, "width": h, "height": w, "rotation": (int(p.get("rotation") or 0) + d) % 360} if d != 180 + else {**p, "rotation": (int(p.get("rotation") or 0) + d) % 360}) if p["page"] == page_no else p + for p in layout.get("pages", [])] + out["blocks"] = [{**b, "bbox": rotate_bbox(b["bbox"], w, h, d)} if b.get("page") == page_no else b + for b in layout.get("blocks", [])] + out["images"] = [{**i, "bbox": rotate_bbox(i["bbox"], w, h, d)} if i.get("page") == page_no and i.get("bbox") else i + for i in layout.get("images", [])] + return out + + def reocr_page(doc_dir: Path, layout: dict[str, Any], page_no: int, vision_ocr_url: str, vision_ocr_model: str, vision_ocr_fallback_url: str | None = None, diff --git a/routes/magazine.py b/routes/magazine.py index 49f0c9e..a56778f 100644 --- a/routes/magazine.py +++ b/routes/magazine.py @@ -314,6 +314,19 @@ def _render_ingest_preview_html(blocks: list, images: list | None, doc_id: str = return "\n".join(parts) +def _page_rotations(meta: dict[str, Any]) -> dict[int, int]: + """{page: clockwise degrees} the reader applied to pages scanned sideways (meta.json `page_rotations`).""" + out: dict[int, int] = {} + for k, v in (meta.get("page_rotations") or {}).items(): + try: + page, deg = int(k), int(v) % 360 + except (TypeError, ValueError): + continue + if page >= 1 and deg in mag.ROTATIONS: + out[page] = deg + return out + + def _ingest_sync(doc_id: str, job_id: str, ocr_lang: str, render_dpi: int, vision_ocr_url: str | None, vision_ocr_model: str | None, vision_ocr_fallback_url: str | None = None, @@ -346,7 +359,8 @@ def _ingest_sync(doc_id: str, job_id: str, ocr_lang: str, render_dpi: int, vision_ocr_fallback_url=vision_ocr_fallback_url, vision_ocr_fallback_model=vision_ocr_fallback_model, force_vision_ocr=force_vision_ocr, - progress=progress) + progress=progress, + page_rotations=_page_rotations(_read_meta(d))) page_sizes = {p["page"]: (p.get("width"), p.get("height")) for p in layout.get("pages", [])} total_pages = len(page_sizes) @@ -774,7 +788,8 @@ async def magazine_page_image(doc_id: str, page: int): png = d / "pages" / f"{page:04d}.png" if not png.is_file(): raise HTTPException(404, "Page not rendered") - return FileResponse(png, media_type="image/png") + # "no-cache" = keep a copy but check it is still current (ETag): a page turned by the reader must show at once. + return FileResponse(png, media_type="image/png", headers={"Cache-Control": "no-cache"}) def _recognise_advertisements(layout: dict, settings: dict, page: int | None = None, on_page=None) -> None: @@ -954,16 +969,25 @@ def _recognise_semantic_layout(layout: dict, settings: dict, page: int | None = def _reocr_page_sync(doc_id: str, job_id: str, page: int, vision_ocr_url: str, vision_ocr_model: str, vision_ocr_fallback_url: str | None, - vision_ocr_fallback_model: str | None) -> None: + vision_ocr_fallback_model: str | None, + rotate_degrees: int = 0) -> None: """"OCR again" on one page with the engine the user picked. Restart-safe job for the same reason ingestion is: a vision OCR call can take up to ~40s (confirmed live), too long for a plain request/response cycle. """ d = _LIB / doc_id + png = d / "pages" / f"{page:04d}.png" + original_png: bytes | None = None # kept while a turn is in flight, so a failed re-read can put the page back try: layout = mag.load_layout(d) if not layout: raise FileNotFoundError("No layout yet -- run ingestion first") + if rotate_degrees: + background_job_update(job_id, stage="rotate", detail=f"Turning page {page}") + original_png = png.read_bytes() + mag.rotate_page_image(png, rotate_degrees) + # The old boxes move with the page, so what was approved still lines up when the new ones are matched. + layout = mag.layout_with_rotated_page(layout, page, rotate_degrees) background_job_update(job_id, stage="reocr", detail=f"Re-reading page {page}") updated = mag.reocr_page(d, layout, page, vision_ocr_url, vision_ocr_model, vision_ocr_fallback_url, vision_ocr_fallback_model) @@ -990,13 +1014,33 @@ def _reocr_page_sync(doc_id: str, job_id: str, page: int, articles = mag.load_articles(d) mag.remap_reviewed_page(articles, layout, updated, page) _add_split_paragraphs_to_articles(articles, cleanup) - token = _snapshot_for_undo(d, f"Re-read page {page}") + # A turn rewrites the page image too, which an undo snapshot of layout + articles does not cover. + token = None if rotate_degrees else _snapshot_for_undo(d, f"Re-read page {page}") mag.save_layout(d, updated) mag.save_articles(d, articles) - _seal_undo(d, token) + if rotate_degrees: + meta = _read_meta(d) + turns = dict(meta.get("page_rotations") or {}) + total = (int(turns.get(str(page), 0)) + rotate_degrees) % 360 + if total: + turns[str(page)] = total + else: + turns.pop(str(page), None) # four quarter turns are no turn at all + meta["page_rotations"] = turns + _write_meta(d, meta) + _refresh_quality_rows(d, updated, [page]) # the old findings described the sideways page + original_png = None # saved: nothing to restore any more. Turning back is its own button. + else: + _seal_undo(d, token) background_job_update(job_id, stage="done", done=True, - detail=f"Page {page} re-read", result={"page": page, "undo": token}) + detail=f"Page {page} " + ("turned and re-read" if rotate_degrees else "re-read"), + result={"page": page, "undo": None if rotate_degrees else token, "rotation": rotate_degrees}) except Exception as exc: # noqa: BLE001 - surfaced to UI + if original_png is not None: + try: + png.write_bytes(original_png) # nothing was saved: the page goes back as it was + except OSError: + logging.exception("Could not restore page %s after a failed turn", page) background_job_update(job_id, stage="failed", done=True, error=str(exc)[:1000]) @@ -1039,6 +1083,38 @@ async def magazine_reocr_page(doc_id: str, page: int, request: Request): return {"job_id": job_id} +@router.post("/api/magazine/{doc_id}/pages/{page}/rotate") +async def magazine_rotate_page(doc_id: str, page: int, request: Request): + """Turn a page that was scanned sideways or upside down and read it again. + + Body: {"degrees": 90 | 180 | 270 | -90} clockwise. The rendered page is turned, the page's old boxes + are carried with it so reviewed articles still match, and the page is re-read like "OCR again". The turn + is remembered (meta.json `page_rotations`) so re-importing does not undo it. Needs a vision OCR engine, + the same as "OCR again". + """ + d, _ = _require_magazine(doc_id) + if page < 1 or page > 10000: + raise HTTPException(400, "Bad page") + try: + body = await request.json() + degrees = mag.normalise_rotation(body.get("degrees")) + except (ValueError, AttributeError, TypeError) as exc: + raise HTTPException(400, str(exc) or "degrees must be 90, 180 or 270") + if not (d / "pages" / f"{page:04d}.png").is_file(): + raise HTTPException(404, "Page not rendered -- run ingestion first") + layout = mag.load_layout(d) + if not layout or not any(p["page"] == page for p in layout.get("pages", [])): + raise HTTPException(404, "Page not found in the layout") + settings = _load_settings() + url, model, fb_url, fb_model = _vision_args(settings) + if not (url and model): + raise HTTPException(400, "No vision OCR engine configured -- set one in Settings (turning re-reads the page)") + job_id = uuid.uuid4().hex + background_job_create(job_id, "magazine-rotate-page", fingerprint=f"{doc_id}:{page}") + asyncio.create_task(asyncio.to_thread(_reocr_page_sync, doc_id, job_id, page, url, model, fb_url, fb_model, degrees)) + return {"job_id": job_id, "degrees": degrees} + + def _quality_page_sync(doc_id: str, job_id: str, page: int, mode: str, vision_ocr_url: str | None = None, vision_ocr_model: str | None = None, diff --git a/static/js/magazine.js b/static/js/magazine.js index d59371b..7664f57 100644 --- a/static/js/magazine.js +++ b/static/js/magazine.js @@ -1992,6 +1992,17 @@ function renderReview() { '' + + '' + + '' + + '' + + '' + + '' + '' + '