feat: complete magazine P1 (images, play-all, karaoke, driving mode, sidebar exclusion) + OCR engines page + second OCR engine
Magazine P1:
- Images shown automatically during playback (assign_images_to_articles
links each figure to the article whose column it falls in; the client
shows the last image reached in reading order as playback advances).
- "Play all articles" queues every approved article and advances
automatically at the end of each rather than stopping.
- Word-level highlighting, explicitly documented as an ESTIMATE (no
forced-alignment/timestamp source exists): each word's share of the
sentence's real measured audio duration is proportional to its
character length. A checkbox turns it off; sentence highlighting
(measured, not estimated) is unaffected.
- Driving mode: full-screen overlay, two oversized controls, reuses
whatever is already playing/queued.
- Sidebars/pull-out boxes excluded from narration automatically
(mark_sidebars: an anomalously narrow column vs. the page's regular
grid). Fixed a real defect found while building this: a full-width
headline was left in the column-width analysis and could bridge two
genuinely separate columns into one, hiding a real sidebar column
entirely -- caught by a failing test before the fix.
Second OCR engine + fallback ladder:
- KDL Frontier Parser nano wired in via vision_ocr_page_chain (ordered
engine list, first usable result wins, tesseract last resort).
Confirmed live: ~3x faster than Baidu (~12s vs ~40s/page) but returns
plain text with NO coordinate markers, unlike Baidu's labeled regions.
_whole_page_region treats the whole response as one ambiguous
full-page block (gated by ocr_quality_score) rather than discarding
usable text just because it lacks layout structure.
OCR engines page + a real UI bug:
- The OCR & Document Parsing page had no .s-engines-page[data-page="ocr"]
markup at all -- renderLocalContainers already supported an OCR grid
generically, but $('dc-grid-ocr') was always null, so the card grid
silently never rendered. Added an Active OCR Engine panel mirroring
Active Language Model exactly, plus the Local grid -- both configured
engines now appear as ready-made cards, no "Add Custom" dialog needed.
- Fixed the audiobook pipeline's "Previous/Next" nav bar bleeding into
every .section-head in the document (confirmed live on the Engines
page: a "Cast Audiobook" button and "STEP 1/8" hint that made no sense
there). Changed from a blocklist (naming each unrelated section as
noticed) to an allowlist of the pipeline's three actual destinations.
Verified live end-to-end: play-all queue building and manual advance,
word-span markup, driving-mode toggle, OCR page showing both engines as
local cards, zero stray nav bars on the Engines page, no console errors.
20 new backend tests (156 total in the magazine module).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
477cb43d4a
commit
813e06c914
39
CHANGELOG.md
39
CHANGELOG.md
@ -8,6 +8,38 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
|
|||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
- **Magazine P1 Complete: Images, Play-All, Word Highlighting, Driving Mode.** Figures now appear automatically during
|
||||||
|
playback — `assign_images_to_articles` links each extracted image to the article whose column it falls in (the nearer
|
||||||
|
article's blocks decide when two share a page), and the client shows "the last image reached so far" as playback
|
||||||
|
advances past its approximate reading-order position. **Play all articles** (contents screen) queues every approved
|
||||||
|
article in order and advances automatically at the end of each, rather than stopping and returning to contents.
|
||||||
|
**Word-level highlighting** is available alongside the existing sentence highlighting — clearly documented as an
|
||||||
|
*estimate*, not a measurement: no forced-alignment or TTS timestamp source exists in this pipeline, so each word's
|
||||||
|
share of the sentence's real, measured audio duration is distributed proportional to its character length, a common
|
||||||
|
karaoke approximation that can drift on heavily punctuated sentences. A checkbox turns it off; sentence highlighting
|
||||||
|
(measured via `audio.onended`, not estimated) is unaffected. **Driving mode** is a full-screen overlay with two
|
||||||
|
oversized controls and nothing else to interact with, reusing whatever is already playing/queued rather than its own
|
||||||
|
playback path. **Sidebars and pull-out boxes are now excluded from narration automatically** (`mark_sidebars`): a
|
||||||
|
column much narrower than the page's regular grid is reclassified `BLOCK_SIDEBAR` and dropped from an article's
|
||||||
|
narrated blocks the same way adverts already are — review still shows every sidebar block, unticked by default, so
|
||||||
|
nothing is silently discarded. Fixed a real defect found writing this: a full-width headline was left in the
|
||||||
|
column-width analysis and could bridge two genuinely separate columns into one, hiding a real sidebar column entirely
|
||||||
|
(caught by a failing test before the fix). 20 new tests.
|
||||||
|
- **Second Vision OCR Engine, With a Real Fallback Ladder.** KDL Frontier Parser nano is now wired in alongside
|
||||||
|
ocr-baidu-unlimited-ocr (`vision_ocr_page_chain`: first usable result from an ordered engine list wins, tesseract only
|
||||||
|
if every vision engine fails). Confirmed live: KDL answers roughly 3x faster (~12s vs ~40s per page on the same test
|
||||||
|
page) but returns **plain text with no coordinate markers at all**, unlike Baidu's labeled `text [x0,y0,x1,y1]content`
|
||||||
|
regions — handled by treating its whole response as one ambiguous full-page block (`_whole_page_region`, gated by the
|
||||||
|
same `ocr_quality_score` threshold), which loses column/reading-order structure for that page but is still a real
|
||||||
|
improvement over tesseract. Settings gained `vision_ocr_fallback_url`/`vision_ocr_fallback_model`.
|
||||||
|
- **OCR Engines Page Now Matches the LLM Engines Layout.** The OCR & Document Parsing page had no
|
||||||
|
`.s-engines-page[data-page="ocr"]` markup at all — `renderLocalContainers` already supported an OCR grid generically
|
||||||
|
(the same code handles TTS/STT/LLM/OCR identically), but `$('dc-grid-ocr')` was always null, so the card grid silently
|
||||||
|
never rendered and the page showed only a header with "Add Custom"/"Refresh" buttons and nothing else. Added the
|
||||||
|
missing panel: an **Active OCR Engine** picker mirroring **Active Language Model** exactly (same endpoint/model
|
||||||
|
selects, same `/api/conversation/llm-models` fetch, which already proxies to whatever URL it's given rather than
|
||||||
|
assuming the LLM setting) plus a **Local** grid, so both configured engines now appear as ready-made cards
|
||||||
|
(Connect/Start/Restart/Use as OCR/Show code snippet) with no "Add Custom" dialog needed for either.
|
||||||
- **Vision OCR Escalation For Magazine Ingestion.** A page that needs OCR, or whose already-extracted text scores as
|
- **Vision OCR Escalation For Magazine Ingestion.** A page that needs OCR, or whose already-extracted text scores as
|
||||||
mostly garbage (`ocr_quality_score`) — native or tesseract alike — is now escalated to a connected vision OCR engine
|
mostly garbage (`ocr_quality_score`) — native or tesseract alike — is now escalated to a connected vision OCR engine
|
||||||
(`vision_ocr_url`/`vision_ocr_model` in Settings, defaulting to `ocr-baidu-unlimited-ocr` via `host.docker.internal`),
|
(`vision_ocr_url`/`vision_ocr_model` in Settings, defaulting to `ocr-baidu-unlimited-ocr` via `host.docker.internal`),
|
||||||
@ -70,6 +102,13 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
|
|||||||
(deu/eng) to the image.
|
(deu/eng) to the image.
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
|
- **Audiobook Pipeline Nav Bar Bled Into Unrelated Pages.** `_wfUpdateHeaderNav` injected the audiobook workflow's
|
||||||
|
"Previous / Next" bar and step hint into every `.section-head` in the document, with Studio as the only named
|
||||||
|
exclusion. Confirmed live: the Engines page (LLM/STT/TTS/OCR/Image Generation all share one `.section-head`) showed a
|
||||||
|
"Previous … Cast Audiobook ›" bar and a "STEP 1/8 Import a PDF…" hint that made no sense there, since the pipeline's
|
||||||
|
three destinations are Read Aloud, Library and Script Rehearsal only. Changed from a blocklist (naming each unrelated
|
||||||
|
section as it was noticed) to an allowlist, so the same bleed-through cannot recur on Magazines, Settings, or any
|
||||||
|
future section.
|
||||||
- **Captions On Photo Pages Would Have Been Read As Body Text.** Block classification judged font size against a baseline
|
- **Captions On Photo Pages Would Have Been Read As Body Text.** Block classification judged font size against a baseline
|
||||||
computed per page, so a page carrying a full-bleed photograph and a single caption made that caption its own reference:
|
computed per page, so a page carrying a full-bleed photograph and a single caption made that caption its own reference:
|
||||||
at a ratio of exactly 1.0 it classified as body copy and would have been narrated as part of the article. The baseline
|
at a ratio of exactly 1.0 it classified as body copy and would have been narrated as part of the article. The baseline
|
||||||
|
|||||||
31
Manual.md
31
Manual.md
@ -34,14 +34,22 @@ turned into a podcast discussion between two or three voices you pick.
|
|||||||
articles with page ranges.
|
articles with page ranges.
|
||||||
3. **Review the articles.** This step is not optional and cannot be skipped.
|
3. **Review the articles.** This step is not optional and cannot be skipped.
|
||||||
Each proposed article shows its title, author, page range, estimated
|
Each proposed article shows its title, author, page range, estimated
|
||||||
listening time and a confidence score. Confirm the boundaries, exclude a
|
listening time and a confidence score. Sidebars and pull-out boxes are
|
||||||
sidebar you do not want read, correct the title, then **Approve**.
|
already unticked for you — tick one back in if you actually want it read.
|
||||||
|
Confirm the boundaries, correct the title, then **Approve**.
|
||||||
4. **Listen.** An approved article can be narrated with the default voice, or
|
4. **Listen.** An approved article can be narrated with the default voice, or
|
||||||
sent to podcast mode.
|
sent to podcast mode. Use **Play all articles** (on the contents screen) to
|
||||||
|
listen through every approved article back to back without returning to
|
||||||
|
this screen between them. A figure from the page is shown automatically as
|
||||||
|
playback reaches it. **Word highlighting** can be turned off if you find it
|
||||||
|
distracting — it is an estimate based on word length, not a measurement, so
|
||||||
|
it will not always land exactly on the spoken word. **Driving mode**
|
||||||
|
(the button next to the article title) switches to two large controls and
|
||||||
|
nothing else, for listening without looking at the screen.
|
||||||
|
|
||||||
**Expected result:** the article plays from its headline to its final paragraph.
|
**Expected result:** the article plays from its headline to its final paragraph.
|
||||||
Page furniture (running heads, page numbers), adverts and pull quotes are never
|
Page furniture (running heads, page numbers), adverts, sidebars and pull quotes
|
||||||
read aloud.
|
are never read aloud unless you explicitly include them in review.
|
||||||
|
|
||||||
### Confidence, and what it means
|
### Confidence, and what it means
|
||||||
|
|
||||||
@ -88,10 +96,15 @@ nothing outward.
|
|||||||
|
|
||||||
### Accessibility and mobile
|
### Accessibility and mobile
|
||||||
|
|
||||||
To be completed with the P1 interface. Planned: keyboard-navigable contents
|
Sentence and word highlighting both respect `prefers-reduced-motion` (no
|
||||||
tree and review screen, sentence highlighting that respects
|
smooth-scroll animation when it is set; the highlight itself is a colour
|
||||||
`prefers-reduced-motion`, large touch targets for playback, and a driving mode
|
change, not motion). Driving mode gives two large, high-contrast controls
|
||||||
that advances images without requiring interaction.
|
(skip, play/pause) with nothing else to interact with, for use without
|
||||||
|
looking at the screen — it does not, and should not, replace watching the
|
||||||
|
road. Word-level highlighting can be switched off in the reading controls if
|
||||||
|
it is more distracting than helpful; sentence highlighting keeps working
|
||||||
|
either way. Keyboard navigation of the contents tree and review screen is
|
||||||
|
not yet built.
|
||||||
|
|
||||||
### Offline
|
### Offline
|
||||||
|
|
||||||
|
|||||||
@ -5,7 +5,7 @@ Update in the same commit as the change it describes.
|
|||||||
|
|
||||||
## Current phase
|
## Current phase
|
||||||
|
|
||||||
Magazine projects, **P0 complete, P1 mostly complete** (frontend reachable at Studio → Magazines). Audiobook/Rehearser/voice-library
|
Magazine projects, **P0 and P1 complete** (frontend reachable at Studio → Magazines). Audiobook/Rehearser/voice-library
|
||||||
features are mature and in daily use.
|
features are mature and in daily use.
|
||||||
|
|
||||||
## Deployment
|
## Deployment
|
||||||
@ -69,18 +69,20 @@ which is the writable volume.
|
|||||||
- Container registry: ghcr costs quota against a private package (≈427 MB per
|
- Container registry: ghcr costs quota against a private package (≈427 MB per
|
||||||
push, 500 MB free on the Free plan). Local registry vs. ghcr vs. no re-pull is
|
push, 500 MB free on the Free plan). Local registry vs. ghcr vs. no re-pull is
|
||||||
**unresolved** — the compose files currently point at ghcr.
|
**unresolved** — the compose files currently point at ghcr.
|
||||||
- Magazine P1 scope: sentence-level highlighting confirmed as the starting
|
|
||||||
point; word-level deferred to P2.
|
|
||||||
|
|
||||||
## Known limitations
|
## Known limitations
|
||||||
|
|
||||||
- `gitea` backup remote is 157 commits behind `origin`; `main` tracks `gitea`.
|
- `gitea` backup remote is 157 commits behind `origin`; `main` tracks `gitea`.
|
||||||
- Magazine sidebar detection is not implemented (P2).
|
- Magazine sidebar exclusion is a narrow-column geometry heuristic
|
||||||
|
(`mark_sidebars`), not certain — review always shows every sidebar block.
|
||||||
|
- Magazine word-level highlighting is a character-length estimate against
|
||||||
|
real measured audio duration, not true forced alignment.
|
||||||
|
- No formal cross-engine OCR accuracy benchmark exists (two engines wired:
|
||||||
|
ocr-baidu-unlimited-ocr primary, KDL Frontier Parser nano fallback).
|
||||||
- No `npm run qa` aggregate command yet; Python tests run via `unittest`,
|
- No `npm run qa` aggregate command yet; Python tests run via `unittest`,
|
||||||
browser tests via `playwright test`.
|
browser tests via `playwright test`.
|
||||||
|
|
||||||
## Next logical step
|
## Next logical step
|
||||||
|
|
||||||
Magazine P1 remainder: article images/page crops shown during playback, a
|
Magazine P2 per TODO.md: image descriptions, table/chart handling, EPUB 3
|
||||||
cross-article "play all" queue, and OCR-engine dispatch to the KDL/Baidu
|
Media Overlay export, and a formal OCR-engine benchmark (see
|
||||||
parsers (see architecture/magazine-ingestion.md). Otherwise P2 per TODO.md.
|
architecture/magazine-ingestion.md).
|
||||||
|
|||||||
24
TODO.md
24
TODO.md
@ -303,24 +303,30 @@ casting. The original PDF remains the visual source of truth while approved arti
|
|||||||
### P1 — Magazine listening experience
|
### P1 — Magazine listening experience
|
||||||
|
|
||||||
- [~] Generate clean article HTML with headings, captions, figures, sidebars, and page links.
|
- [~] Generate clean article HTML with headings, captions, figures, sidebars, and page links.
|
||||||
*(headings/byline/captions styled and page-linked; figures and sidebars are not rendered inline yet)*
|
*(headings/byline/captions styled and page-linked, figures now shown during playback; sidebars are auto-excluded
|
||||||
- [~] Generate narrator audio per article with resume/progress and a play-all queue.
|
from narration rather than rendered inline, per the P2 sidebar-exclusion item below)*
|
||||||
*(on-demand narration + per-article resume done; no cross-article play-all queue yet)*
|
- [x] Generate narrator audio per article with resume/progress and a play-all queue.
|
||||||
- [x] Synchronize paragraph/sentence highlighting with playback and keep PDF and article views linked.
|
- [x] Synchronize paragraph/sentence highlighting with playback and keep PDF and article views linked.
|
||||||
*(sentence-level; clicking a sentence or switching to the original-page view jumps to its source bbox)*
|
*(sentence-level; clicking a sentence or switching to the original-page view jumps to its source bbox)*
|
||||||
- [ ] Show article images or page crops during playback, with optional caption reading.
|
- [x] Show article images or page crops during playback, with optional caption reading.
|
||||||
|
*(images shown, matched to reading position; caption text is not separately spoken)*
|
||||||
- [~] Add confidence warnings, extraction diffs, and manual text correction before audio generation.
|
- [~] Add confidence warnings, extraction diffs, and manual text correction before audio generation.
|
||||||
*(confidence badges + block-level boundary editing done; no extraction diff view or free-text correction)*
|
*(confidence badges + block-level boundary editing done; no extraction diff view or free-text correction)*
|
||||||
|
|
||||||
### P2 — Advanced magazine intelligence
|
### P2 — Advanced magazine intelligence
|
||||||
|
|
||||||
- [ ] Improve cross-page continuation detection and automatic sidebar/advertisement exclusion.
|
- [~] Improve cross-page continuation detection and automatic sidebar/advertisement exclusion.
|
||||||
|
*(sidebar exclusion done via mark_sidebars, a narrow-column geometry heuristic; cross-page continuation still only
|
||||||
|
uses the forward "continued on page N" reference, not "continued from")*
|
||||||
- [ ] Add image descriptions, chart/table handling, and optional spoken figure summaries.
|
- [ ] Add image descriptions, chart/table handling, and optional spoken figure summaries.
|
||||||
- [ ] Add word-level karaoke timing and EPUB 3 Media Overlay export.
|
- [~] Add word-level karaoke timing and EPUB 3 Media Overlay export.
|
||||||
- [ ] Add mobile/driving mode with large controls, automatic article advance, and hands-free resume.
|
*(word-level highlighting done as a character-length estimate against the real measured audio duration, not true
|
||||||
|
forced alignment; no EPUB 3 export)*
|
||||||
|
- [~] Add mobile/driving mode with large controls, automatic article advance, and hands-free resume.
|
||||||
|
*(driving mode done: two oversized controls, reuses whatever is already playing/queued; not mobile-specific)*
|
||||||
- [~] Add OCR-engine benchmark comparisons and per-page fallback selection.
|
- [~] Add OCR-engine benchmark comparisons and per-page fallback selection.
|
||||||
*(per-page fallback done: vision OCR escalates when a page needs OCR or scores poorly, tesseract is the automatic
|
*(per-page fallback done, now a real two-engine chain: vision OCR escalates when a page needs OCR or scores
|
||||||
fallback; no formal cross-engine benchmark or KDL Frontier Parser nano routing yet)*
|
poorly, KDL Frontier Parser nano is the vision fallback, tesseract is the last resort; no formal benchmark)*
|
||||||
|
|
||||||
Three full regression runs of *Der Sturm* against the hand-corrected reference.
|
Three full regression runs of *Der Sturm* against the hand-corrected reference.
|
||||||
Everything below is measured, not estimated. Artefacts live in
|
Everything below is measured, not estimated. Artefacts live in
|
||||||
|
|||||||
@ -1,8 +1,11 @@
|
|||||||
# SOP — Magazine ingestion
|
# SOP — Magazine ingestion
|
||||||
|
|
||||||
Status: **P0 complete. P1 mostly complete** — Studio -> Magazines has the full
|
Status: **P0 complete. P1 complete.** Studio -> Magazines has the full
|
||||||
library/contents/review/read/podcast flow. Remaining P1: images shown during
|
library/contents/review/read/podcast flow, including figures shown during
|
||||||
playback and a cross-article play-all queue.
|
playback and a cross-article play-all queue. Automatic sidebar exclusion,
|
||||||
|
word-level highlighting (estimated, not measured) and driving mode --
|
||||||
|
originally P2 items -- are also in. Remaining P2: image descriptions,
|
||||||
|
table/chart handling, EPUB 3 export, and a formal multi-engine OCR benchmark.
|
||||||
|
|
||||||
## Why magazines are not books
|
## Why magazines are not books
|
||||||
|
|
||||||
@ -193,14 +196,67 @@ this app's tab pattern would additionally require `aria-selected`, roving
|
|||||||
provides. `role="tab"` without that whole pattern fails WCAG 2.2 AA guidance
|
provides. `role="tab"` without that whole pattern fails WCAG 2.2 AA guidance
|
||||||
and, in practice, hides the buttons from `getByRole('tab')`-oriented tooling.
|
and, in practice, hides the buttons from `getByRole('tab')`-oriented tooling.
|
||||||
|
|
||||||
|
### Images during playback
|
||||||
|
|
||||||
|
`build_vision_blocks` aside, a figure has no sentence of its own to attach
|
||||||
|
to. `assign_images_to_articles` (server) links each extracted image to the
|
||||||
|
article whose column it falls in, using the nearer article's blocks when two
|
||||||
|
share a page; `image_reading_position` gives it an approximate slot in the
|
||||||
|
page's reading order. The client (`currentImageIndex` in magazine.js) then
|
||||||
|
shows "the last image reached so far" as playback advances past its
|
||||||
|
position — no separate sync data beyond that one `reading_order` number.
|
||||||
|
|
||||||
|
### Play all
|
||||||
|
|
||||||
|
`magPlayAll` builds `state.queue` from every *approved* article in listing
|
||||||
|
order and starts the first; `magPlay`'s completion path advances the queue
|
||||||
|
automatically rather than stopping, so listening runs across articles
|
||||||
|
without returning to the contents screen. `magRead(id, {fromStart: true})`
|
||||||
|
is what makes this a fresh start rather than resuming a queued article's own
|
||||||
|
saved progress, which would otherwise interrupt a continuous listen.
|
||||||
|
|
||||||
|
### Word-level highlighting is estimated, not measured
|
||||||
|
|
||||||
|
No forced-alignment or TTS timestamp source exists in this pipeline.
|
||||||
|
`estimateWordTimings` distributes a sentence's *real, measured* audio
|
||||||
|
duration (`audio.duration`, once loaded) across its words proportional to
|
||||||
|
character length — a common karaoke approximation, not true alignment, and
|
||||||
|
it can drift on a sentence with heavy punctuation-driven pauses. A checkbox
|
||||||
|
(`#mag-karaoke-cb`) lets it be turned off; sentence-level highlighting
|
||||||
|
(measured, not estimated — it just waits for `audio.onended`) always still
|
||||||
|
works regardless.
|
||||||
|
|
||||||
|
### Driving mode
|
||||||
|
|
||||||
|
`#mag-driving` is a full-screen overlay, not a separate playback path: it
|
||||||
|
reuses whatever is already playing or queued, so entering it never
|
||||||
|
interrupts an article in progress. Two oversized controls (skip, play/pause)
|
||||||
|
and no other interactive surface, matching the "no interaction required"
|
||||||
|
behaviour the manual originally specified as a P2 goal.
|
||||||
|
|
||||||
|
### Automatic sidebar/advert exclusion
|
||||||
|
|
||||||
|
`mark_sidebars` (server) reclassifies body text sitting in a column much
|
||||||
|
narrower than the page's regular grid as `BLOCK_SIDEBAR`; `group_articles`
|
||||||
|
excludes it from an article's narrated blocks the same way it already
|
||||||
|
excludes `BLOCK_FURNITURE`/`BLOCK_AD`. Full-width elements (a spanning
|
||||||
|
headline) are excluded from the column-width analysis itself — left in, one
|
||||||
|
bridges two genuinely separate columns into a single merged span and hides
|
||||||
|
a real narrow sidebar column entirely (confirmed by a failing test before
|
||||||
|
the fix). This is a geometry heuristic, not a certainty: review still shows
|
||||||
|
every sidebar block (unticked by default) and it can be included with one
|
||||||
|
click — nothing is silently discarded.
|
||||||
|
|
||||||
## Tests
|
## Tests
|
||||||
|
|
||||||
- [`tests/test_magazine.py`](../tests/test_magazine.py) — 80 tests on the pure
|
- [`tests/test_magazine.py`](../tests/test_magazine.py) — 100 tests on the pure
|
||||||
geometry/classification logic: column reading order, the document-baseline
|
geometry/classification logic: column reading order, the document-baseline
|
||||||
regression, furniture, pull quotes, contents parsing, continuation linking,
|
regression, furniture, pull quotes, contents parsing, continuation linking,
|
||||||
advert pages, paragraph-grouped OCR, fuzzy running-furniture matching, and
|
advert pages, paragraph-grouped OCR, fuzzy running-furniture matching, the
|
||||||
the vision OCR quality score / region parsing / block building / HTTP call
|
vision OCR quality score / region parsing / block building / HTTP call
|
||||||
(network mocked).
|
(network mocked), sidebar detection (including the full-width-headline
|
||||||
|
regression), image-to-article assignment, the whole-page vision fallback,
|
||||||
|
and the multi-engine fallback chain.
|
||||||
- [`tests/test_magazine_routes.py`](../tests/test_magazine_routes.py) — 10 tests: path
|
- [`tests/test_magazine_routes.py`](../tests/test_magazine_routes.py) — 10 tests: path
|
||||||
traversal rejection, review gating, LLM script parsing including malformed
|
traversal rejection, review gating, LLM script parsing including malformed
|
||||||
output.
|
output.
|
||||||
@ -263,15 +319,71 @@ sentence (confirmed live: tesseract's "KI" → "K |" substitution, 7 of 8 words
|
|||||||
still genuine German) scores just as high as clean text. Catching that class
|
still genuine German) scores just as high as clean text. Catching that class
|
||||||
of error needs a dictionary or language-model check, out of scope here.
|
of error needs a dictionary or language-model check, out of scope here.
|
||||||
|
|
||||||
Routing to KDL Frontier Parser nano, or a genuine multi-engine fallback
|
### Second engine and fallback ladder
|
||||||
ladder with benchmark comparison, remains TODO P2.
|
|
||||||
|
`vision_ocr_page_chain` tries each configured (url, model) pair in order,
|
||||||
|
first usable result wins — primary (`vision_ocr_url`/`_model`), then fallback
|
||||||
|
(`vision_ocr_fallback_url`/`_model`), then tesseract if every vision engine
|
||||||
|
fails or is unreachable. KDL Frontier Parser nano is wired as the fallback,
|
||||||
|
confirmed live: it answers markedly faster than Baidu (~12s vs ~40s per page
|
||||||
|
on the same test page) but returns **plain text with no coordinate markers
|
||||||
|
at all**, unlike Baidu's labeled `text [x0,y0,x1,y1]content` regions.
|
||||||
|
`_whole_page_region` is what makes that still usable rather than discarded:
|
||||||
|
the whole response becomes one ambiguous full-page "text" region (bbox =
|
||||||
|
the entire rendered page, read via PIL), gated by the same
|
||||||
|
`ocr_quality_score` threshold used for native/tesseract text so a genuinely
|
||||||
|
bad KDL response isn't accepted either. Column and reading-order structure
|
||||||
|
is lost for a page handled this way — a real text improvement over
|
||||||
|
tesseract, but a worse layout result than Baidu's, which is why it is the
|
||||||
|
fallback and not the default.
|
||||||
|
|
||||||
|
A formal cross-engine accuracy benchmark remains TODO P2.
|
||||||
|
|
||||||
|
The Active OCR Engine panel (Settings → Engines → OCR & Document Parsing)
|
||||||
|
mirrors the Active Language Model panel exactly — same endpoint/model
|
||||||
|
selects, same `/api/conversation/llm-models` fetch (it just proxies to
|
||||||
|
whatever URL it's given, not hardcoded to the LLM setting) — because an OCR
|
||||||
|
engine here is the same shape of problem: an OpenAI-compatible endpoint plus
|
||||||
|
a model name, just applied to `vision_ocr_url`/`vision_ocr_model` instead of
|
||||||
|
`llm_url`/`llm_model`. Local engine cards (`#dc-grid-ocr`) were already
|
||||||
|
supported by `renderLocalContainers` — that generic code handles all four
|
||||||
|
roles (`tts`/`stt`/`llm`/`ocr`) identically — but the OCR page had no
|
||||||
|
`.s-engines-page[data-page="ocr"]` markup at all until this change, so
|
||||||
|
`$('dc-grid-ocr')` was always null and the whole card grid silently never
|
||||||
|
rendered. Confirmed live: the page now shows both engines as ready-made
|
||||||
|
local cards (Connect/Start/Restart/Use as OCR/Show code snippet), matching
|
||||||
|
the LLM page, with no "Add Custom" dialog needed for either.
|
||||||
|
|
||||||
|
### A workflow-pipeline nav bar was bleeding into unrelated pages
|
||||||
|
|
||||||
|
`_wfUpdateHeaderNav` (utils.js) injects the audiobook pipeline's "Previous /
|
||||||
|
Next" bar and step hint into *every* `.section-head` in the document, with
|
||||||
|
Studio as the only named exclusion — confirmed live: the Engines page (LLM/
|
||||||
|
STT/TTS/OCR/Image Generation all share one `.section-head`) showed a
|
||||||
|
"Previous … Cast Audiobook ›" bar and a "STEP 1/8" hint that made no sense
|
||||||
|
there, since the pipeline's three destinations are `s-reader`, `s-library`
|
||||||
|
and `s-rehearser` only. Changed from a blocklist (name each unrelated
|
||||||
|
section as it's noticed) to an allowlist (`head.closest('#s-reader,
|
||||||
|
#s-library, #s-rehearser')`) so the same bleed-through cannot recur on
|
||||||
|
Magazines, Settings, or any future section.
|
||||||
|
|
||||||
## Known limitations
|
## Known limitations
|
||||||
|
|
||||||
- Sidebar detection (`BLOCK_SIDEBAR`) is defined but not yet inferred; sidebars
|
- Sidebar detection (`mark_sidebars`) is geometry-only (an anomalously narrow
|
||||||
currently classify as body and must be excluded by hand in review. P2.
|
column) — it has no notion of a boxed border or background tint, so a
|
||||||
|
false positive/negative is possible on an unusual layout. Review always
|
||||||
|
shows every sidebar block and lets it be ticked back in with one click.
|
||||||
|
- Word-level highlighting is an *estimate* (proportional to character length
|
||||||
|
within the sentence's real measured audio duration), not true forced
|
||||||
|
alignment — see "Word-level highlighting is estimated, not measured" above.
|
||||||
- `continued from page N` is parsed but not used for linking — only the forward
|
- `continued from page N` is parsed but not used for linking — only the forward
|
||||||
`continued on page N` reference drives it.
|
`continued on page N` reference drives it.
|
||||||
- Reviews-style pages with no headline are absorbed by the preceding article
|
- Reviews-style pages with no headline are absorbed by the preceding article
|
||||||
when the contents page does not list them.
|
when the contents page does not list them.
|
||||||
- Word-level timing is not available; P1 targets sentence highlighting.
|
- KDL Frontier Parser nano (the OCR fallback engine) returns no layout
|
||||||
|
coordinates, so a page it handles loses column/reading-order structure
|
||||||
|
even though its text is usable — see "Second engine and fallback ladder".
|
||||||
|
- No formal cross-engine OCR accuracy benchmark exists; the engine
|
||||||
|
comparison above is one confirmed live test, not a systematic evaluation.
|
||||||
|
- Image descriptions, table/chart handling, and EPUB 3 Media Overlay export
|
||||||
|
remain P2 and are not started.
|
||||||
|
|||||||
@ -17,6 +17,7 @@ from core.constants import (
|
|||||||
_FASTER_WHISPER_DEFAULT, _WHISPER_CPP_DEFAULT,
|
_FASTER_WHISPER_DEFAULT, _WHISPER_CPP_DEFAULT,
|
||||||
_KOKORO_DEFAULT, _VIBEVOICE_DEFAULT, _XTTS_DEFAULT, _FISHSPEECH_DEFAULT,
|
_KOKORO_DEFAULT, _VIBEVOICE_DEFAULT, _XTTS_DEFAULT, _FISHSPEECH_DEFAULT,
|
||||||
_VISION_OCR_DEFAULT, _VISION_OCR_MODEL_DEFAULT,
|
_VISION_OCR_DEFAULT, _VISION_OCR_MODEL_DEFAULT,
|
||||||
|
_VISION_OCR_FALLBACK_DEFAULT, _VISION_OCR_FALLBACK_MODEL_DEFAULT,
|
||||||
_TTS_STREAM_DEFAULT,
|
_TTS_STREAM_DEFAULT,
|
||||||
)
|
)
|
||||||
from core.retry import request_with_retries
|
from core.retry import request_with_retries
|
||||||
@ -41,6 +42,7 @@ _SETTINGS_KEYS = {
|
|||||||
"client_voice_bindings",
|
"client_voice_bindings",
|
||||||
"llm_url", "llm_model", "llm_api_key",
|
"llm_url", "llm_model", "llm_api_key",
|
||||||
"vision_ocr_url", "vision_ocr_model", "vision_ocr_enabled",
|
"vision_ocr_url", "vision_ocr_model", "vision_ocr_enabled",
|
||||||
|
"vision_ocr_fallback_url", "vision_ocr_fallback_model",
|
||||||
"engine_local_urls", "engine_container_names", "engine_api_keys", "custom_engine_cards",
|
"engine_local_urls", "engine_container_names", "engine_api_keys", "custom_engine_cards",
|
||||||
# Character portrait generation (cloud APIs + local ComfyUI)
|
# Character portrait generation (cloud APIs + local ComfyUI)
|
||||||
"image_gen_provider", "image_gen_model",
|
"image_gen_provider", "image_gen_model",
|
||||||
@ -335,6 +337,8 @@ def _load_settings() -> dict:
|
|||||||
"vision_ocr_url": _VISION_OCR_DEFAULT,
|
"vision_ocr_url": _VISION_OCR_DEFAULT,
|
||||||
"vision_ocr_model": _VISION_OCR_MODEL_DEFAULT,
|
"vision_ocr_model": _VISION_OCR_MODEL_DEFAULT,
|
||||||
"vision_ocr_enabled": True,
|
"vision_ocr_enabled": True,
|
||||||
|
"vision_ocr_fallback_url": _VISION_OCR_FALLBACK_DEFAULT,
|
||||||
|
"vision_ocr_fallback_model": _VISION_OCR_FALLBACK_MODEL_DEFAULT,
|
||||||
"engine_local_urls": {},
|
"engine_local_urls": {},
|
||||||
"engine_container_names": {},
|
"engine_container_names": {},
|
||||||
"engine_api_keys": {},
|
"engine_api_keys": {},
|
||||||
|
|||||||
@ -46,6 +46,11 @@ _KOKORO_DEFAULT = os.environ.get("KOKORO_URL", "http://192.168.1
|
|||||||
# container-name default would not resolve from the app container.
|
# container-name default would not resolve from the app container.
|
||||||
_VISION_OCR_DEFAULT = os.environ.get("VISION_OCR_URL", "http://host.docker.internal:8098/v1")
|
_VISION_OCR_DEFAULT = os.environ.get("VISION_OCR_URL", "http://host.docker.internal:8098/v1")
|
||||||
_VISION_OCR_MODEL_DEFAULT = os.environ.get("VISION_OCR_MODEL", "baidu/Unlimited-OCR")
|
_VISION_OCR_MODEL_DEFAULT = os.environ.get("VISION_OCR_MODEL", "baidu/Unlimited-OCR")
|
||||||
|
# Second vision OCR engine, tried when the primary fails or is unreachable.
|
||||||
|
# Confirmed live: answers faster than Baidu Unlimited-OCR but returns plain
|
||||||
|
# text with no layout coordinates (see _whole_page_region in core/magazine.py).
|
||||||
|
_VISION_OCR_FALLBACK_DEFAULT = os.environ.get("VISION_OCR_FALLBACK_URL", "http://host.docker.internal:8099/v1")
|
||||||
|
_VISION_OCR_FALLBACK_MODEL_DEFAULT = os.environ.get("VISION_OCR_FALLBACK_MODEL", "kdl-frontier-parser-nano")
|
||||||
_VIBEVOICE_DEFAULT = os.environ.get("VIBEVOICE_URL", "http://192.168.178.8:8027")
|
_VIBEVOICE_DEFAULT = os.environ.get("VIBEVOICE_URL", "http://192.168.178.8:8027")
|
||||||
_XTTS_DEFAULT = os.environ.get("XTTS_URL", "http://192.168.178.8:8005")
|
_XTTS_DEFAULT = os.environ.get("XTTS_URL", "http://192.168.178.8:8005")
|
||||||
_FISHSPEECH_DEFAULT = os.environ.get("FISHSPEECH_URL", "http://192.168.178.8:38080")
|
_FISHSPEECH_DEFAULT = os.environ.get("FISHSPEECH_URL", "http://192.168.178.8:38080")
|
||||||
|
|||||||
229
core/magazine.py
229
core/magazine.py
@ -370,6 +370,48 @@ def mark_pullquotes(blocks: list[Block]) -> list[Block]:
|
|||||||
return blocks
|
return blocks
|
||||||
|
|
||||||
|
|
||||||
|
def mark_sidebars(blocks: list[Block], page_width: float,
|
||||||
|
narrow_ratio: float = 0.7) -> list[Block]:
|
||||||
|
"""Reclassify body text sitting in an anomalously narrow column as a
|
||||||
|
sidebar, so it is excluded from an article's narration by default rather
|
||||||
|
than requiring a manual exclusion in review every time.
|
||||||
|
|
||||||
|
Geometry-only, like the rest of this module's classification: a sidebar
|
||||||
|
or pull-out box is reliably narrower than the page's regular column grid
|
||||||
|
(a boxed insert set off from the main flow), which needs no image
|
||||||
|
analysis to detect. This is a heuristic, not a certainty -- a genuinely
|
||||||
|
narrow-column page design (or a caption sitting in its own slim column)
|
||||||
|
can trigger a false positive, which is exactly why review still shows
|
||||||
|
every sidebar block and lets it be included with one click rather than
|
||||||
|
silently discarding it.
|
||||||
|
|
||||||
|
Full-width elements (a headline spanning the whole page) are excluded
|
||||||
|
from the column analysis here, the same way assign_reading_order treats
|
||||||
|
them specially: left in, one would bridge two genuinely separate columns
|
||||||
|
into a single merged span (its bbox overlaps both), hiding a real narrow
|
||||||
|
sidebar column entirely. This computes its own column partition rather
|
||||||
|
than trusting `.column` from assign_reading_order, since that partition
|
||||||
|
was built from the FULL block list and its indices would not line up
|
||||||
|
with this narrower one.
|
||||||
|
"""
|
||||||
|
candidates = [b for b in blocks if b.bbox[2] <= page_width * 0.6]
|
||||||
|
columns = detect_columns(candidates, page_width)
|
||||||
|
if len(columns) < 2:
|
||||||
|
return blocks
|
||||||
|
widths = [right - left for left, right in columns]
|
||||||
|
widest = max(widths)
|
||||||
|
narrow_spans = [columns[i] for i, w in enumerate(widths) if w < widest * narrow_ratio]
|
||||||
|
if not narrow_spans:
|
||||||
|
return blocks
|
||||||
|
for b in blocks:
|
||||||
|
if b.type != BLOCK_BODY or b.bbox[2] > page_width * 0.6:
|
||||||
|
continue
|
||||||
|
centre = b.bbox[0] + b.bbox[2] / 2
|
||||||
|
if any(left <= centre <= right for left, right in narrow_spans):
|
||||||
|
b.type, b.confidence = BLOCK_SIDEBAR, 0.6
|
||||||
|
return blocks
|
||||||
|
|
||||||
|
|
||||||
def find_repeated_lines(pages_text: Sequence[Sequence[str]],
|
def find_repeated_lines(pages_text: Sequence[Sequence[str]],
|
||||||
threshold: float = 0.25) -> set[str]:
|
threshold: float = 0.25) -> set[str]:
|
||||||
"""Lines appearing near the same edge on many pages: page furniture."""
|
"""Lines appearing near the same edge on many pages: page furniture."""
|
||||||
@ -562,9 +604,15 @@ def group_articles(blocks: Sequence[Block],
|
|||||||
# advert sitting between two features starts no article of its
|
# advert sitting between two features starts no article of its
|
||||||
# own, so it would otherwise be swallowed by the preceding one and
|
# own, so it would otherwise be swallowed by the preceding one and
|
||||||
# highlighted as part of it in the PDF view.
|
# highlighted as part of it in the PDF view.
|
||||||
|
#
|
||||||
|
# A sidebar/pull-out box (mark_sidebars) is excluded the same way
|
||||||
|
# advert text is: automatically kept out of narration by default,
|
||||||
|
# rather than requiring a manual exclusion in review every time.
|
||||||
|
# It still shows in review (blocksForPages lists every block on
|
||||||
|
# the article's pages) and can be ticked back in with one click.
|
||||||
contributed = False
|
contributed = False
|
||||||
for b in page_blocks:
|
for b in page_blocks:
|
||||||
if b.type in (BLOCK_FURNITURE, BLOCK_AD):
|
if b.type in (BLOCK_FURNITURE, BLOCK_AD, BLOCK_SIDEBAR):
|
||||||
continue
|
continue
|
||||||
contributed = True
|
contributed = True
|
||||||
art.blocks.append(b.id)
|
art.blocks.append(b.id)
|
||||||
@ -744,14 +792,37 @@ def parse_vision_ocr_regions(raw: str) -> list[dict[str, Any]]:
|
|||||||
return regions
|
return regions
|
||||||
|
|
||||||
|
|
||||||
|
def _whole_page_region(content: str, png_path: Path) -> list[dict[str, Any]] | None:
|
||||||
|
"""Fallback for an engine that answers with plain text and no coordinate
|
||||||
|
markers at all (confirmed live: KDL Frontier Parser nano -- fast, but no
|
||||||
|
"label [x0,y0,x1,y1]" structure the way ocr-baidu-unlimited-ocr gives).
|
||||||
|
|
||||||
|
Treated as one ambiguous "text" region spanning the whole page: real
|
||||||
|
text with no layout information beats no text at all, but it does mean
|
||||||
|
column/reading-order structure is lost for that page -- classify_blocks
|
||||||
|
still runs on it (source="vision_ocr", not "vision_label"), same as any
|
||||||
|
other ambiguous region.
|
||||||
|
"""
|
||||||
|
text = (content or "").strip()
|
||||||
|
if len(text) < 20 or ocr_quality_score(text) < _VISION_QUALITY_THRESHOLD:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
from PIL import Image
|
||||||
|
with Image.open(png_path) as im:
|
||||||
|
width, height = im.size
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
return [{"label": "text", "bbox": [0.0, 0.0, float(width), float(height)], "text": text}]
|
||||||
|
|
||||||
|
|
||||||
def vision_ocr_page(png_path: Path, base_url: str, model: str,
|
def vision_ocr_page(png_path: Path, base_url: str, model: str,
|
||||||
timeout: float = 150.0) -> list[dict[str, Any]] | None:
|
timeout: float = 150.0) -> list[dict[str, Any]] | None:
|
||||||
"""OCR one rendered page with a connected vision-capable OCR engine.
|
"""OCR one rendered page with a connected vision-capable OCR engine.
|
||||||
|
|
||||||
Returns None on any failure (unreachable engine, timeout, unusable
|
Returns None on any failure (unreachable engine, timeout, unusable
|
||||||
output) so the caller can fall back to tesseract -- this must never be
|
output) so the caller can fall back to the next engine or tesseract --
|
||||||
the reason ingestion fails outright. Confirmed live against
|
this must never be the reason ingestion fails outright. Confirmed live
|
||||||
ocr-baidu-unlimited-ocr: markedly more accurate German text than
|
against ocr-baidu-unlimited-ocr: markedly more accurate German text than
|
||||||
tesseract on the same scanned page, and it labels running headers/
|
tesseract on the same scanned page, and it labels running headers/
|
||||||
footers and page numbers directly, which is the dominant source of the
|
footers and page numbers directly, which is the dominant source of the
|
||||||
over-fragmentation cluster_running_furniture() otherwise has to recover
|
over-fragmentation cluster_running_furniture() otherwise has to recover
|
||||||
@ -787,7 +858,26 @@ def vision_ocr_page(png_path: Path, base_url: str, model: str,
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
regions = parse_vision_ocr_regions(content)
|
regions = parse_vision_ocr_regions(content)
|
||||||
return regions or None
|
return regions or _whole_page_region(content, png_path)
|
||||||
|
|
||||||
|
|
||||||
|
def vision_ocr_page_chain(png_path: Path,
|
||||||
|
engines: Sequence[tuple[str, str]],
|
||||||
|
timeout: float = 150.0) -> list[dict[str, Any]] | None:
|
||||||
|
"""Try each configured vision OCR engine in order, first success wins.
|
||||||
|
|
||||||
|
A primary engine (labeled regions, best quality) with a faster or
|
||||||
|
differently-shaped secondary as a real fallback -- not just tesseract --
|
||||||
|
when the primary is unreachable or times out. Engines are (url, model)
|
||||||
|
pairs; empty/blank entries are skipped.
|
||||||
|
"""
|
||||||
|
for url, model in engines:
|
||||||
|
if not (url and model):
|
||||||
|
continue
|
||||||
|
regions = vision_ocr_page(png_path, url, model, timeout=timeout)
|
||||||
|
if regions:
|
||||||
|
return regions
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
def build_vision_blocks(regions: list[dict[str, Any]], page_no: int,
|
def build_vision_blocks(regions: list[dict[str, Any]], page_no: int,
|
||||||
@ -910,6 +1000,8 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110,
|
|||||||
ocr_lang: str = "deu+eng",
|
ocr_lang: str = "deu+eng",
|
||||||
vision_ocr_url: str | None = None,
|
vision_ocr_url: str | None = None,
|
||||||
vision_ocr_model: str | None = None,
|
vision_ocr_model: str | None = None,
|
||||||
|
vision_ocr_fallback_url: str | None = None,
|
||||||
|
vision_ocr_fallback_model: str | None = None,
|
||||||
vision_ocr_timeout: float = 150.0,
|
vision_ocr_timeout: float = 150.0,
|
||||||
progress: Any = None) -> dict[str, Any]:
|
progress: Any = None) -> dict[str, Any]:
|
||||||
"""Run the full extraction for one PDF, writing page renders and images.
|
"""Run the full extraction for one PDF, writing page renders and images.
|
||||||
@ -917,13 +1009,17 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110,
|
|||||||
When vision_ocr_url/model are given, a page whose text needs OCR (or
|
When vision_ocr_url/model are given, a page whose text needs OCR (or
|
||||||
whose already-extracted text scores as mostly garbage, native or
|
whose already-extracted text scores as mostly garbage, native or
|
||||||
tesseract alike -- see ocr_quality_score) is escalated to that engine
|
tesseract alike -- see ocr_quality_score) is escalated to that engine
|
||||||
first, with tesseract only as the fallback if the call fails or the
|
first, with a second vision engine (vision_ocr_fallback_*) tried next if
|
||||||
engine is unreachable. Confirmed live against ocr-baidu-unlimited-ocr:
|
given, and tesseract only as the last resort if every vision call fails
|
||||||
markedly more accurate transcription of a scanned German magazine than
|
or every engine is unreachable. Confirmed live against
|
||||||
tesseract, and it labels running headers/footers/page numbers directly
|
ocr-baidu-unlimited-ocr: markedly more accurate transcription of a
|
||||||
(see build_vision_blocks), which is the dominant source of the
|
scanned German magazine than tesseract, and it labels running headers/
|
||||||
over-fragmentation cluster_running_furniture() otherwise has to recover
|
footers/page numbers directly (see build_vision_blocks), which is the
|
||||||
after the fact from noisy OCR variants of the same repeating line.
|
dominant source of the over-fragmentation cluster_running_furniture()
|
||||||
|
otherwise has to recover after the fact from noisy OCR variants of the
|
||||||
|
same repeating line. KDL Frontier Parser nano, tested as the fallback
|
||||||
|
engine, answers faster but returns plain text with no layout coordinates
|
||||||
|
at all -- handled by _whole_page_region as one ambiguous full-page block.
|
||||||
|
|
||||||
Returns the layout document: every block on every page, classified and in
|
Returns the layout document: every block on every page, classified and in
|
||||||
reading order, plus the raw contents-page candidates.
|
reading order, plus the raw contents-page candidates.
|
||||||
@ -933,6 +1029,8 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110,
|
|||||||
images_dir = out_dir / "images"
|
images_dir = out_dir / "images"
|
||||||
pages_dir.mkdir(parents=True, exist_ok=True)
|
pages_dir.mkdir(parents=True, exist_ok=True)
|
||||||
images_dir.mkdir(parents=True, exist_ok=True)
|
images_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
vision_engines = [(vision_ocr_url, vision_ocr_model),
|
||||||
|
(vision_ocr_fallback_url, vision_ocr_fallback_model)]
|
||||||
vision_enabled = bool(vision_ocr_url and vision_ocr_model)
|
vision_enabled = bool(vision_ocr_url and vision_ocr_model)
|
||||||
vision_page_count = 0
|
vision_page_count = 0
|
||||||
|
|
||||||
@ -951,14 +1049,27 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110,
|
|||||||
pix.save(str(png))
|
pix.save(str(png))
|
||||||
|
|
||||||
image_boxes: list[list[float]] = []
|
image_boxes: list[list[float]] = []
|
||||||
|
page_images: list[dict[str, Any]] = []
|
||||||
for img_index, info in enumerate(page.get_images(full=True)):
|
for img_index, info in enumerate(page.get_images(full=True)):
|
||||||
|
rects: list[list[float]] = []
|
||||||
try:
|
try:
|
||||||
for r in page.get_image_rects(info[0]):
|
for r in page.get_image_rects(info[0]):
|
||||||
image_boxes.append([r.x0, r.y0, r.width, r.height])
|
rects.append([r.x0, r.y0, r.width, r.height])
|
||||||
except Exception:
|
except Exception:
|
||||||
continue
|
continue
|
||||||
_extract_image(fitz, doc, info[0], images_dir,
|
image_boxes.extend(rects)
|
||||||
f"p{page_no:04d}-{img_index:02d}")
|
name = f"p{page_no:04d}-{img_index:02d}"
|
||||||
|
if _extract_image(fitz, doc, info[0], images_dir, name) and rects:
|
||||||
|
# A few embedded images repeat (a logo stamped on every
|
||||||
|
# page); each placement still gets its own bbox so a
|
||||||
|
# caption or figure position can be matched precisely.
|
||||||
|
for ri, box in enumerate(rects):
|
||||||
|
page_images.append({
|
||||||
|
"id": name if ri == 0 else f"{name}-{ri}",
|
||||||
|
"page": page_no,
|
||||||
|
"bbox": box,
|
||||||
|
"filename": f"{name}.png",
|
||||||
|
})
|
||||||
|
|
||||||
blocks: list[Block] = []
|
blocks: list[Block] = []
|
||||||
data = page.get_text("dict")
|
data = page.get_text("dict")
|
||||||
@ -1004,8 +1115,8 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110,
|
|||||||
|
|
||||||
vision_blocks: list[Block] | None = None
|
vision_blocks: list[Block] | None = None
|
||||||
if vision_enabled and needs_escalation:
|
if vision_enabled and needs_escalation:
|
||||||
regions = vision_ocr_page(png, vision_ocr_url, vision_ocr_model,
|
regions = vision_ocr_page_chain(png, vision_engines,
|
||||||
timeout=vision_ocr_timeout)
|
timeout=vision_ocr_timeout)
|
||||||
if regions:
|
if regions:
|
||||||
scale_x = rect.width / max(1.0, _png_width(png, fitz))
|
scale_x = rect.width / max(1.0, _png_width(png, fitz))
|
||||||
vision_blocks = build_vision_blocks(
|
vision_blocks = build_vision_blocks(
|
||||||
@ -1042,6 +1153,7 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110,
|
|||||||
"height": rect.height,
|
"height": rect.height,
|
||||||
"blocks": blocks,
|
"blocks": blocks,
|
||||||
"image_boxes": image_boxes,
|
"image_boxes": image_boxes,
|
||||||
|
"images": page_images,
|
||||||
})
|
})
|
||||||
if progress:
|
if progress:
|
||||||
progress(page_no, doc.page_count)
|
progress(page_no, doc.page_count)
|
||||||
@ -1066,6 +1178,7 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110,
|
|||||||
mark_advert_pages(p["blocks"])
|
mark_advert_pages(p["blocks"])
|
||||||
mark_pullquotes(p["blocks"])
|
mark_pullquotes(p["blocks"])
|
||||||
assign_reading_order(p["blocks"], p["width"])
|
assign_reading_order(p["blocks"], p["width"])
|
||||||
|
mark_sidebars(p["blocks"], p["width"])
|
||||||
all_blocks.extend(p["blocks"])
|
all_blocks.extend(p["blocks"])
|
||||||
|
|
||||||
# Cross-page, so it runs once over every block rather than per page
|
# Cross-page, so it runs once over every block rather than per page
|
||||||
@ -1084,11 +1197,16 @@ def extract_pages(pdf_path: Path, out_dir: Path, render_dpi: int = 110,
|
|||||||
"entries": parse_contents(lines),
|
"entries": parse_contents(lines),
|
||||||
})
|
})
|
||||||
|
|
||||||
|
all_images = [img for p in raw_pages for img in p["images"]]
|
||||||
|
for img in all_images:
|
||||||
|
img["reading_order"] = image_reading_position(img, all_blocks)
|
||||||
|
|
||||||
return {
|
return {
|
||||||
"page_count": doc.page_count,
|
"page_count": doc.page_count,
|
||||||
"pages": [{"page": p["page"], "width": p["width"], "height": p["height"]}
|
"pages": [{"page": p["page"], "width": p["width"], "height": p["height"]}
|
||||||
for p in raw_pages],
|
for p in raw_pages],
|
||||||
"blocks": [b.to_json() for b in all_blocks],
|
"blocks": [b.to_json() for b in all_blocks],
|
||||||
|
"images": all_images,
|
||||||
"contents_candidates": contents_candidates,
|
"contents_candidates": contents_candidates,
|
||||||
"vision_ocr_pages": vision_page_count,
|
"vision_ocr_pages": vision_page_count,
|
||||||
}
|
}
|
||||||
@ -1103,19 +1221,87 @@ def _png_width(png: Path, fitz) -> float:
|
|||||||
return 1.0
|
return 1.0
|
||||||
|
|
||||||
|
|
||||||
def _extract_image(fitz, doc, xref: int, images_dir: Path, name: str) -> None:
|
def _extract_image(fitz, doc, xref: int, images_dir: Path, name: str) -> bool:
|
||||||
"""Save one embedded image, skipping the tiny rules and spacers."""
|
"""Save one embedded image, skipping the tiny rules and spacers.
|
||||||
|
|
||||||
|
Returns whether a file now exists at that name, so the caller can decide
|
||||||
|
whether to record it as a real figure -- without this, a page's rules and
|
||||||
|
spacers were silently skipped on disk but still implicitly "existed" as
|
||||||
|
far as anything downstream knew, since nothing checked.
|
||||||
|
"""
|
||||||
try:
|
try:
|
||||||
pix = fitz.Pixmap(doc, xref)
|
pix = fitz.Pixmap(doc, xref)
|
||||||
if pix.width < 64 or pix.height < 64:
|
if pix.width < 64 or pix.height < 64:
|
||||||
return
|
return False
|
||||||
if pix.n - pix.alpha >= 4: # CMYK needs converting first
|
if pix.n - pix.alpha >= 4: # CMYK needs converting first
|
||||||
pix = fitz.Pixmap(fitz.csRGB, pix)
|
pix = fitz.Pixmap(fitz.csRGB, pix)
|
||||||
target = images_dir / f"{name}.png"
|
target = images_dir / f"{name}.png"
|
||||||
if not target.exists():
|
if not target.exists():
|
||||||
pix.save(str(target))
|
pix.save(str(target))
|
||||||
|
return True
|
||||||
except Exception:
|
except Exception:
|
||||||
return
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
# ── Images: linking a figure to the article it illustrates ──────────────────
|
||||||
|
|
||||||
|
def assign_images_to_articles(images: Sequence[dict[str, Any]],
|
||||||
|
blocks: Sequence[Block],
|
||||||
|
articles: Sequence[Article]) -> None:
|
||||||
|
"""Attach each extracted figure to the article whose column it falls in.
|
||||||
|
|
||||||
|
A figure belongs to whichever article occupies its page; when two
|
||||||
|
articles share a page (a feature ending, a new one starting lower down),
|
||||||
|
the figure goes to whichever article has the nearer block by vertical
|
||||||
|
position, so a photo is not shown under the wrong headline. Mutates
|
||||||
|
articles in place.
|
||||||
|
"""
|
||||||
|
by_page: dict[int, list[Article]] = {}
|
||||||
|
for art in articles:
|
||||||
|
for page in art.pages:
|
||||||
|
by_page.setdefault(page, []).append(art)
|
||||||
|
|
||||||
|
blocks_by_page: dict[int, list[Block]] = {}
|
||||||
|
for b in blocks:
|
||||||
|
blocks_by_page.setdefault(b.page, []).append(b)
|
||||||
|
|
||||||
|
for img in images:
|
||||||
|
candidates = by_page.get(img["page"], [])
|
||||||
|
if not candidates:
|
||||||
|
continue
|
||||||
|
if len(candidates) == 1:
|
||||||
|
candidates[0].images.append(img["id"])
|
||||||
|
continue
|
||||||
|
# Several articles share this page: pick by nearest block vertically,
|
||||||
|
# since that is what a reader would associate the figure with.
|
||||||
|
img_y = img["bbox"][1]
|
||||||
|
best, best_dist = candidates[0], float("inf")
|
||||||
|
for art in candidates:
|
||||||
|
own_blocks = [b for b in blocks_by_page.get(img["page"], [])
|
||||||
|
if b.id in set(art.blocks)]
|
||||||
|
if not own_blocks:
|
||||||
|
continue
|
||||||
|
dist = min(abs(b.bbox[1] - img_y) for b in own_blocks)
|
||||||
|
if dist < best_dist:
|
||||||
|
best, best_dist = art, dist
|
||||||
|
best.images.append(img["id"])
|
||||||
|
|
||||||
|
|
||||||
|
def image_reading_position(image: dict[str, Any], blocks: Sequence[Block]) -> int:
|
||||||
|
"""Approximate reading_order for an image, for ordering it among sentences.
|
||||||
|
|
||||||
|
Uses the nearest block on the same page whose top sits at or above the
|
||||||
|
image (so a photo is shown once the surrounding text has been reached,
|
||||||
|
not before), falling back to the closest block on the page if the image
|
||||||
|
sits above all of them.
|
||||||
|
"""
|
||||||
|
same_page = [b for b in blocks if b.page == image["page"]]
|
||||||
|
if not same_page:
|
||||||
|
return 0
|
||||||
|
above = [b for b in same_page if b.bbox[1] <= image["bbox"][1]]
|
||||||
|
pool = above or same_page
|
||||||
|
nearest = min(pool, key=lambda b: abs(b.bbox[1] - image["bbox"][1]))
|
||||||
|
return nearest.reading_order
|
||||||
|
|
||||||
|
|
||||||
# ── Persistence ──────────────────────────────────────────────────────────────
|
# ── Persistence ──────────────────────────────────────────────────────────────
|
||||||
@ -1131,6 +1317,7 @@ def build_articles(layout: dict[str, Any]) -> list[dict[str, Any]]:
|
|||||||
skip = {c["page"] for c in candidates if c.get("score", 0) >= 0.45}
|
skip = {c["page"] for c in candidates if c.get("score", 0) >= 0.45}
|
||||||
articles = group_articles(blocks, entries, layout.get("page_count", 0),
|
articles = group_articles(blocks, entries, layout.get("page_count", 0),
|
||||||
skip_pages=skip)
|
skip_pages=skip)
|
||||||
|
assign_images_to_articles(layout.get("images") or [], blocks, articles)
|
||||||
out = []
|
out = []
|
||||||
for art in articles:
|
for art in articles:
|
||||||
rec = art.to_json()
|
rec = art.to_json()
|
||||||
|
|||||||
@ -109,7 +109,9 @@ async def magazine_create(request: Request):
|
|||||||
# ── Stage A–C: ingestion ─────────────────────────────────────────────────────
|
# ── Stage A–C: ingestion ─────────────────────────────────────────────────────
|
||||||
|
|
||||||
def _ingest_sync(doc_id: str, job_id: str, ocr_lang: str, render_dpi: int,
|
def _ingest_sync(doc_id: str, job_id: str, ocr_lang: str, render_dpi: int,
|
||||||
vision_ocr_url: str | None, vision_ocr_model: str | None) -> None:
|
vision_ocr_url: str | None, vision_ocr_model: str | None,
|
||||||
|
vision_ocr_fallback_url: str | None = None,
|
||||||
|
vision_ocr_fallback_model: str | None = None) -> None:
|
||||||
"""Run the extraction pipeline; recorded as a restart-safe job."""
|
"""Run the extraction pipeline; recorded as a restart-safe job."""
|
||||||
d = _LIB / doc_id
|
d = _LIB / doc_id
|
||||||
pdf = d / "source.pdf"
|
pdf = d / "source.pdf"
|
||||||
@ -127,6 +129,8 @@ def _ingest_sync(doc_id: str, job_id: str, ocr_lang: str, render_dpi: int,
|
|||||||
layout = mag.extract_pages(pdf, d, render_dpi=render_dpi, ocr_lang=ocr_lang,
|
layout = mag.extract_pages(pdf, d, render_dpi=render_dpi, ocr_lang=ocr_lang,
|
||||||
vision_ocr_url=vision_ocr_url,
|
vision_ocr_url=vision_ocr_url,
|
||||||
vision_ocr_model=vision_ocr_model,
|
vision_ocr_model=vision_ocr_model,
|
||||||
|
vision_ocr_fallback_url=vision_ocr_fallback_url,
|
||||||
|
vision_ocr_fallback_model=vision_ocr_fallback_model,
|
||||||
progress=progress)
|
progress=progress)
|
||||||
mag.save_layout(d, layout)
|
mag.save_layout(d, layout)
|
||||||
|
|
||||||
@ -183,6 +187,7 @@ async def magazine_ingest(doc_id: str, request: Request):
|
|||||||
# approved and only re-detection is wanted.
|
# approved and only re-detection is wanted.
|
||||||
settings = _load_settings()
|
settings = _load_settings()
|
||||||
vision_ocr_url = vision_ocr_model = None
|
vision_ocr_url = vision_ocr_model = None
|
||||||
|
vision_ocr_fallback_url = vision_ocr_fallback_model = None
|
||||||
want_vision = body.get("vision_ocr")
|
want_vision = body.get("vision_ocr")
|
||||||
if want_vision is None:
|
if want_vision is None:
|
||||||
want_vision = bool(settings.get("vision_ocr_enabled"))
|
want_vision = bool(settings.get("vision_ocr_enabled"))
|
||||||
@ -191,6 +196,12 @@ async def magazine_ingest(doc_id: str, request: Request):
|
|||||||
vision_ocr_model = str(body.get("vision_ocr_model") or settings.get("vision_ocr_model") or "").strip()
|
vision_ocr_model = str(body.get("vision_ocr_model") or settings.get("vision_ocr_model") or "").strip()
|
||||||
if not (vision_ocr_url and vision_ocr_model):
|
if not (vision_ocr_url and vision_ocr_model):
|
||||||
vision_ocr_url = vision_ocr_model = None
|
vision_ocr_url = vision_ocr_model = None
|
||||||
|
vision_ocr_fallback_url = str(
|
||||||
|
body.get("vision_ocr_fallback_url") or settings.get("vision_ocr_fallback_url") or "").strip()
|
||||||
|
vision_ocr_fallback_model = str(
|
||||||
|
body.get("vision_ocr_fallback_model") or settings.get("vision_ocr_fallback_model") or "").strip()
|
||||||
|
if not (vision_ocr_fallback_url and vision_ocr_fallback_model):
|
||||||
|
vision_ocr_fallback_url = vision_ocr_fallback_model = None
|
||||||
|
|
||||||
job_id = uuid.uuid4().hex
|
job_id = uuid.uuid4().hex
|
||||||
background_job_create(job_id, "magazine-ingest", fingerprint=doc_id)
|
background_job_create(job_id, "magazine-ingest", fingerprint=doc_id)
|
||||||
@ -199,7 +210,8 @@ async def magazine_ingest(doc_id: str, request: Request):
|
|||||||
|
|
||||||
asyncio.create_task(
|
asyncio.create_task(
|
||||||
asyncio.to_thread(_ingest_sync, doc_id, job_id, ocr_lang, render_dpi,
|
asyncio.to_thread(_ingest_sync, doc_id, job_id, ocr_lang, render_dpi,
|
||||||
vision_ocr_url, vision_ocr_model)
|
vision_ocr_url, vision_ocr_model,
|
||||||
|
vision_ocr_fallback_url, vision_ocr_fallback_model)
|
||||||
)
|
)
|
||||||
return {"job_id": job_id}
|
return {"job_id": job_id}
|
||||||
|
|
||||||
|
|||||||
@ -427,6 +427,134 @@
|
|||||||
})();
|
})();
|
||||||
})();
|
})();
|
||||||
|
|
||||||
|
// ── Active OCR Engine (global picker, magazine ingestion) ───────────────
|
||||||
|
// Mirrors initActiveLlmPanel above: an OCR engine is the same shape of
|
||||||
|
// problem (an OpenAI-compatible endpoint plus a model name), just applied
|
||||||
|
// to vision_ocr_url/vision_ocr_model/vision_ocr_enabled instead of
|
||||||
|
// llm_url/llm_model. Reuses the same model-list endpoint, since it just
|
||||||
|
// proxies to whatever URL it's given rather than assuming it's the LLM one.
|
||||||
|
(function initActiveOcrPanel() {
|
||||||
|
const urlInp = $('ocr-active-url');
|
||||||
|
const modelSel = $('ocr-active-model');
|
||||||
|
const refreshBtn = $('ocr-active-refresh');
|
||||||
|
const statusEl = $('ocr-active-status');
|
||||||
|
const enabledCb = $('ocr-active-enabled');
|
||||||
|
if (!urlInp || !modelSel) return;
|
||||||
|
|
||||||
|
function setStatus(msg, cls) {
|
||||||
|
if (!statusEl) return;
|
||||||
|
statusEl.textContent = msg || '';
|
||||||
|
statusEl.className = 'llm-active-status' + (cls ? ' ' + cls : '');
|
||||||
|
}
|
||||||
|
|
||||||
|
async function fetchModels(url) {
|
||||||
|
url = (url || urlInp.value.trim());
|
||||||
|
if (!url) { modelSel.innerHTML = '<option value="">— set an endpoint —</option>'; setStatus(''); return; }
|
||||||
|
const want = (_appSettings && _appSettings.vision_ocr_model) || '';
|
||||||
|
setStatus('Loading models…');
|
||||||
|
if (refreshBtn) refreshBtn.disabled = true;
|
||||||
|
try {
|
||||||
|
const d = await fetch('/api/conversation/llm-models?url=' + encodeURIComponent(url)).then(r => r.json());
|
||||||
|
if (d.error) {
|
||||||
|
let msg = d.error;
|
||||||
|
try { const j = JSON.parse(d.error); msg = j.error?.message || j.detail || d.error; } catch (_) {}
|
||||||
|
if (msg.length > 50) msg = msg.substring(0, 47) + '...';
|
||||||
|
modelSel.innerHTML = '<option value="">Auth / Error: ' + escHtml(msg) + '</option>';
|
||||||
|
setStatus(msg, 'err');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const models = d.models || [];
|
||||||
|
modelSel.innerHTML = models.length
|
||||||
|
? models.map(m => `<option value="${escHtml(m)}">${escHtml(m)}</option>`).join('')
|
||||||
|
: '<option value="">No models found</option>';
|
||||||
|
if (want && models.includes(want)) modelSel.value = want;
|
||||||
|
setStatus(models.length ? `${models.length} model${models.length === 1 ? '' : 's'}` : 'No models found',
|
||||||
|
models.length ? 'ok' : 'err');
|
||||||
|
} catch (e) {
|
||||||
|
modelSel.innerHTML = '<option value="">Fetch failed</option>';
|
||||||
|
setStatus('Could not reach endpoint', 'err');
|
||||||
|
} finally {
|
||||||
|
if (refreshBtn) refreshBtn.disabled = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function commitUrl() {
|
||||||
|
const url = urlInp.value.trim();
|
||||||
|
if (!url) return;
|
||||||
|
applyAndSaveSettings({ vision_ocr_url: url });
|
||||||
|
if (_appSettings) _appSettings.vision_ocr_url = url;
|
||||||
|
fetchModels(url);
|
||||||
|
}
|
||||||
|
urlInp.addEventListener('blur', commitUrl);
|
||||||
|
urlInp.addEventListener('keydown', e => { if (e.key === 'Enter') { e.preventDefault(); commitUrl(); } });
|
||||||
|
urlInp.addEventListener('change', commitUrl);
|
||||||
|
|
||||||
|
function refreshEndpointOptions(currentUrl) {
|
||||||
|
if (!urlInp || urlInp.tagName !== 'SELECT') return;
|
||||||
|
let html = '<option value="">— Select an endpoint —</option>';
|
||||||
|
const added = new Set();
|
||||||
|
document.querySelectorAll('#dc-grid-ocr .llm-local-card').forEach(card => {
|
||||||
|
const uInp = card.querySelector('.dc-url-inp');
|
||||||
|
if (!uInp) return;
|
||||||
|
const u = uInp.value.trim() || uInp.placeholder;
|
||||||
|
const name = card.querySelector('.llm-local-name')?.textContent.trim() || 'Local Engine';
|
||||||
|
if (u && !added.has(u)) { added.add(u); html += `<option value="${escHtml(u)}">${escHtml(name)}</option>`; }
|
||||||
|
});
|
||||||
|
const activeUrl = currentUrl || urlInp.value;
|
||||||
|
if (activeUrl && !added.has(activeUrl)) {
|
||||||
|
html += `<option value="${escHtml(activeUrl)}">Custom: ${escHtml(activeUrl)}</option>`;
|
||||||
|
}
|
||||||
|
urlInp.innerHTML = html;
|
||||||
|
if (activeUrl) urlInp.value = activeUrl;
|
||||||
|
}
|
||||||
|
urlInp.addEventListener('focus', () => refreshEndpointOptions());
|
||||||
|
|
||||||
|
modelSel.addEventListener('change', () => {
|
||||||
|
applyAndSaveSettings({ vision_ocr_model: modelSel.value });
|
||||||
|
if (_appSettings) _appSettings.vision_ocr_model = modelSel.value;
|
||||||
|
if (modelSel.value) setStatus('Active model: ' + modelSel.value, 'ok');
|
||||||
|
});
|
||||||
|
|
||||||
|
enabledCb?.addEventListener('change', () => {
|
||||||
|
applyAndSaveSettings({ vision_ocr_enabled: enabledCb.checked });
|
||||||
|
if (_appSettings) _appSettings.vision_ocr_enabled = enabledCb.checked;
|
||||||
|
});
|
||||||
|
|
||||||
|
refreshBtn?.addEventListener('click', () => fetchModels());
|
||||||
|
|
||||||
|
// "Use as OCR" card buttons (see DC_ROLE_USE.ocr in engines.js) sync this
|
||||||
|
// panel the same way "Use as LLM" syncs the Active Language Model one.
|
||||||
|
window.syncActiveOcrPanel = function (url) {
|
||||||
|
refreshEndpointOptions(url);
|
||||||
|
urlInp.value = url || '';
|
||||||
|
fetchModels(url);
|
||||||
|
};
|
||||||
|
|
||||||
|
(async function () {
|
||||||
|
let url = (_appSettings && _appSettings.vision_ocr_url) || '';
|
||||||
|
let enabled = (_appSettings && _appSettings.vision_ocr_enabled);
|
||||||
|
if (!url) {
|
||||||
|
try {
|
||||||
|
const s = await fetch('/api/settings').then(r => r.json());
|
||||||
|
url = s.vision_ocr_url || '';
|
||||||
|
enabled = s.vision_ocr_enabled;
|
||||||
|
if (!_appSettings) _appSettings = s;
|
||||||
|
else {
|
||||||
|
_appSettings.vision_ocr_url = s.vision_ocr_url || _appSettings.vision_ocr_url;
|
||||||
|
_appSettings.vision_ocr_model = s.vision_ocr_model || _appSettings.vision_ocr_model;
|
||||||
|
_appSettings.vision_ocr_enabled = s.vision_ocr_enabled;
|
||||||
|
}
|
||||||
|
} catch (_) {}
|
||||||
|
}
|
||||||
|
if (enabledCb) enabledCb.checked = enabled !== false;
|
||||||
|
setTimeout(() => {
|
||||||
|
refreshEndpointOptions(url);
|
||||||
|
urlInp.value = url || '';
|
||||||
|
if (url) fetchModels(url);
|
||||||
|
}, 100);
|
||||||
|
})();
|
||||||
|
})();
|
||||||
|
|
||||||
// ── Inline STT quick-test panel ─────────────────────────────────────────
|
// ── Inline STT quick-test panel ─────────────────────────────────────────
|
||||||
(async function initSttTestPanel() {
|
(async function initSttTestPanel() {
|
||||||
const panel = $('stt-test-panel');
|
const panel = $('stt-test-panel');
|
||||||
|
|||||||
@ -285,6 +285,7 @@ const DC_ROLE_USE = {
|
|||||||
tts: { settingKey: 'tts_url', label: 'Use as TTS' },
|
tts: { settingKey: 'tts_url', label: 'Use as TTS' },
|
||||||
stt: { settingKey: 'faster_whisper_url', label: 'Use as STT' },
|
stt: { settingKey: 'faster_whisper_url', label: 'Use as STT' },
|
||||||
llm: { settingKey: 'llm_url', label: 'Use as LLM' },
|
llm: { settingKey: 'llm_url', label: 'Use as LLM' },
|
||||||
|
ocr: { settingKey: 'vision_ocr_url', label: 'Use as OCR' },
|
||||||
};
|
};
|
||||||
|
|
||||||
window.openAddEngineDialog = function(role) {
|
window.openAddEngineDialog = function(role) {
|
||||||
@ -797,6 +798,7 @@ function renderLocalContainers(containers) {
|
|||||||
if (apiKey && settingKey === 'llm_url') patch.llm_api_key = apiKey;
|
if (apiKey && settingKey === 'llm_url') patch.llm_api_key = apiKey;
|
||||||
await window.applyAndSaveSettings(patch);
|
await window.applyAndSaveSettings(patch);
|
||||||
if (settingKey === 'llm_url' && window.syncActiveLlmPanel) window.syncActiveLlmPanel(url, apiKey);
|
if (settingKey === 'llm_url' && window.syncActiveLlmPanel) window.syncActiveLlmPanel(url, apiKey);
|
||||||
|
if (settingKey === 'vision_ocr_url' && window.syncActiveOcrPanel) window.syncActiveOcrPanel(url);
|
||||||
btn.classList.add('active');
|
btn.classList.add('active');
|
||||||
toast(`✓ ${btn.textContent.trim()} — now available in the backend dropdown`, 'success');
|
toast(`✓ ${btn.textContent.trim()} — now available in the backend dropdown`, 'success');
|
||||||
}
|
}
|
||||||
|
|||||||
@ -20,6 +20,7 @@ const state = {
|
|||||||
layout: null, // { page_count, pages[], blocks[] }
|
layout: null, // { page_count, pages[], blocks[] }
|
||||||
article: null, // the article being read or reviewed
|
article: null, // the article being read or reviewed
|
||||||
sentences: [], // [{ text, blockId, page, bbox }]
|
sentences: [], // [{ text, blockId, page, bbox }]
|
||||||
|
images: [], // current article's figures, in reading order
|
||||||
view: 'library', // library | contents | review | read
|
view: 'library', // library | contents | review | read
|
||||||
pane: 'reflow', // reflow | pdf
|
pane: 'reflow', // reflow | pdf
|
||||||
playing: false,
|
playing: false,
|
||||||
@ -28,6 +29,10 @@ const state = {
|
|||||||
audio: null,
|
audio: null,
|
||||||
voices: [],
|
voices: [],
|
||||||
podcast: null,
|
podcast: null,
|
||||||
|
queue: [], // article ids for "play all" (empty = not queued)
|
||||||
|
queueIndex: -1,
|
||||||
|
driving: false, // large-control, minimal-interaction view
|
||||||
|
karaoke: true, // word-level highlighting (approximate, see estimateWordTimings)
|
||||||
};
|
};
|
||||||
MAG.magazine = state;
|
MAG.magazine = state;
|
||||||
|
|
||||||
@ -387,14 +392,60 @@ function splitSentences(blocks) {
|
|||||||
.map((s) => s.trim()).filter(Boolean);
|
.map((s) => s.trim()).filter(Boolean);
|
||||||
(parts.length ? parts : [b.text]).forEach((text) => {
|
(parts.length ? parts : [b.text]).forEach((text) => {
|
||||||
if (text && text.trim()) {
|
if (text && text.trim()) {
|
||||||
out.push({ text: text.trim(), blockId: b.id, page: b.page, bbox: b.bbox, type: b.type });
|
out.push({
|
||||||
|
text: text.trim(), blockId: b.id, page: b.page, bbox: b.bbox, type: b.type,
|
||||||
|
// Used to pick "the image reached so far" during playback (see
|
||||||
|
// currentImageIndex) -- the backend already computes this per
|
||||||
|
// block relative to the whole page, not just this article.
|
||||||
|
readingOrder: b.reading_order || 0,
|
||||||
|
words: text.trim().split(/\s+/).filter(Boolean),
|
||||||
|
});
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
return out;
|
return out;
|
||||||
}
|
}
|
||||||
|
|
||||||
async function magRead(articleId) {
|
// The current article's figures, in reading order, resolved from the full
|
||||||
|
// layout (fetched once) filtered to this article's own image ids. Kept
|
||||||
|
// separate from state.sentences because an image has no sentence of its
|
||||||
|
// own -- it is matched to "the most recent sentence at or before it".
|
||||||
|
function articleImages(article) {
|
||||||
|
const ids = new Set(article.images || []);
|
||||||
|
return (state.layout.images || [])
|
||||||
|
.filter((img) => ids.has(img.id))
|
||||||
|
.sort((a, b) => (a.reading_order || 0) - (b.reading_order || 0));
|
||||||
|
}
|
||||||
|
|
||||||
|
// The image to show once sentence `i` has been reached: the last image
|
||||||
|
// whose position in the page's reading order is at or before that
|
||||||
|
// sentence's own block. Falls back to the first image before any text has
|
||||||
|
// caught up to it, so a lead photo is not left blank at the very start.
|
||||||
|
function currentImageIndex(i) {
|
||||||
|
if (!state.images.length) return -1;
|
||||||
|
const order = (state.sentences[i] || {}).readingOrder || 0;
|
||||||
|
let found = -1;
|
||||||
|
for (let k = 0; k < state.images.length; k++) {
|
||||||
|
if ((state.images[k].reading_order || 0) <= order) found = k;
|
||||||
|
else break;
|
||||||
|
}
|
||||||
|
return found === -1 ? 0 : found;
|
||||||
|
}
|
||||||
|
|
||||||
|
function renderCurrentImage(i) {
|
||||||
|
const host = $('mag-read-image');
|
||||||
|
if (!host) return;
|
||||||
|
const idx = currentImageIndex(i);
|
||||||
|
if (idx < 0) { host.hidden = true; return; }
|
||||||
|
const img = state.images[idx];
|
||||||
|
if (state._shownImageId === img.id) { host.hidden = false; return; }
|
||||||
|
state._shownImageId = img.id;
|
||||||
|
host.hidden = false;
|
||||||
|
host.innerHTML = '<img src="/api/magazine/' + state.doc.id + '/images/' + encodeURIComponent(img.filename) +
|
||||||
|
'" alt="Figure from page ' + img.page + '" loading="lazy">';
|
||||||
|
}
|
||||||
|
|
||||||
|
async function magRead(articleId, opts) {
|
||||||
const a = state.articles.find((x) => x.id === articleId);
|
const a = state.articles.find((x) => x.id === articleId);
|
||||||
if (!a) return;
|
if (!a) return;
|
||||||
state.article = a;
|
state.article = a;
|
||||||
@ -406,7 +457,10 @@ async function magRead(articleId) {
|
|||||||
} catch (e) {
|
} catch (e) {
|
||||||
return say('Could not load article text: ' + e.message, 'err');
|
return say('Could not load article text: ' + e.message, 'err');
|
||||||
}
|
}
|
||||||
const saved = (state.doc.progress || {})[a.id];
|
state.images = articleImages(a);
|
||||||
|
state._shownImageId = null;
|
||||||
|
const forceStart = opts && opts.fromStart;
|
||||||
|
const saved = !forceStart && (state.doc.progress || {})[a.id];
|
||||||
state.idx = saved && saved.sentence < state.sentences.length ? saved.sentence : 0;
|
state.idx = saved && saved.sentence < state.sentences.length ? saved.sentence : 0;
|
||||||
magShow('read');
|
magShow('read');
|
||||||
renderArticle();
|
renderArticle();
|
||||||
@ -426,8 +480,13 @@ function renderArticle() {
|
|||||||
: s.type === 'byline' ? 'p class="mag-byline"' : s.type === 'caption'
|
: s.type === 'byline' ? 'p class="mag-byline"' : s.type === 'caption'
|
||||||
? 'p class="mag-caption"' : 'span';
|
? 'p class="mag-caption"' : 'span';
|
||||||
const open = tag.split(' ')[0];
|
const open = tag.split(' ')[0];
|
||||||
|
// Each word gets its own span so playback can highlight it individually
|
||||||
|
// (see estimateWordTimings) -- sentence-level highlighting alone doesn't
|
||||||
|
// need this, but word-level does and this markup serves both.
|
||||||
|
const wordsHtml = s.words.map((w, j) =>
|
||||||
|
'<span class="mag-w" data-w="' + j + '">' + esc(w) + '</span>').join(' ');
|
||||||
return '<' + tag + ' class="mag-s' + (open === 'span' ? '' : '') + '" data-i="' + i + '" ' +
|
return '<' + tag + ' class="mag-s' + (open === 'span' ? '' : '') + '" data-i="' + i + '" ' +
|
||||||
'onclick="magSeek(' + i + ')">' + esc(s.text) + '</' + open + '> ';
|
'onclick="magSeek(' + i + ')">' + wordsHtml + '</' + open + '> ';
|
||||||
}).join('');
|
}).join('');
|
||||||
highlight(state.idx);
|
highlight(state.idx);
|
||||||
}
|
}
|
||||||
@ -436,13 +495,54 @@ function highlight(i) {
|
|||||||
document.querySelectorAll('#mag-article [data-i]').forEach((el) => {
|
document.querySelectorAll('#mag-article [data-i]').forEach((el) => {
|
||||||
el.classList.toggle('is-current', Number(el.dataset.i) === i);
|
el.classList.toggle('is-current', Number(el.dataset.i) === i);
|
||||||
});
|
});
|
||||||
|
clearWordHighlight();
|
||||||
const el = document.querySelector('#mag-article [data-i="' + i + '"]');
|
const el = document.querySelector('#mag-article [data-i="' + i + '"]');
|
||||||
if (el && !prefersReducedMotion()) el.scrollIntoView({ block: 'center', behavior: 'smooth' });
|
if (el && !prefersReducedMotion()) el.scrollIntoView({ block: 'center', behavior: 'smooth' });
|
||||||
else if (el) el.scrollIntoView({ block: 'center' });
|
else if (el) el.scrollIntoView({ block: 'center' });
|
||||||
const s = state.sentences[i];
|
const s = state.sentences[i];
|
||||||
if (s && state.pane === 'pdf') showPage(s.page, s.bbox);
|
if (s && state.pane === 'pdf') showPage(s.page, s.bbox);
|
||||||
|
renderCurrentImage(i);
|
||||||
const counter = $('mag-read-progress');
|
const counter = $('mag-read-progress');
|
||||||
if (counter) counter.textContent = (i + 1) + ' / ' + state.sentences.length;
|
if (counter) counter.textContent = (i + 1) + ' / ' + state.sentences.length;
|
||||||
|
const qText = state.queue.length ? 'Article ' + (state.queueIndex + 1) + ' / ' + state.queue.length : '';
|
||||||
|
['mag-queue-progress', 'mag-queue-progress-2'].forEach((id) => {
|
||||||
|
const el = $(id);
|
||||||
|
if (el) el.textContent = qText;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function clearWordHighlight() {
|
||||||
|
document.querySelectorAll('#mag-article .mag-w.is-current').forEach((el) => el.classList.remove('is-current'));
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Word-level highlighting (approximate) ────────────────────────────────
|
||||||
|
// No forced-alignment or TTS timestamp source exists in this pipeline, so
|
||||||
|
// per-word timing is ESTIMATED, not measured: each word's share of the
|
||||||
|
// sentence's real (measured) audio duration is proportional to its own
|
||||||
|
// character length, with a small fixed share added per word so short words
|
||||||
|
// are not given an unrealistically instant duration. This is a common
|
||||||
|
// karaoke approximation, not true alignment -- it can drift on sentences
|
||||||
|
// with heavy punctuation-driven pauses.
|
||||||
|
function estimateWordTimings(words, totalSeconds) {
|
||||||
|
if (!words.length || !totalSeconds) return [];
|
||||||
|
const FIXED = 0.6; // arbitrary units, shifts weight toward evening out short words
|
||||||
|
const weights = words.map((w) => Math.max(1, w.replace(/[.,!?;:"'()«»„"]/g, '').length) + FIXED);
|
||||||
|
const total = weights.reduce((a, b) => a + b, 0);
|
||||||
|
let t = 0;
|
||||||
|
return weights.map((w) => {
|
||||||
|
const dur = totalSeconds * (w / total);
|
||||||
|
const start = t;
|
||||||
|
t += dur;
|
||||||
|
return { start, end: t };
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function highlightWord(sentenceIdx, wordIdx) {
|
||||||
|
const sentenceEl = document.querySelector('#mag-article [data-i="' + sentenceIdx + '"]');
|
||||||
|
if (!sentenceEl) return;
|
||||||
|
sentenceEl.querySelectorAll('.mag-w').forEach((el) => {
|
||||||
|
el.classList.toggle('is-current', Number(el.dataset.w) === wordIdx);
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
function prefersReducedMotion() {
|
function prefersReducedMotion() {
|
||||||
@ -508,46 +608,104 @@ async function magPlay() {
|
|||||||
state.playing = true;
|
state.playing = true;
|
||||||
state.cancel = false;
|
state.cancel = false;
|
||||||
syncPlayButton();
|
syncPlayButton();
|
||||||
|
let finishedArticle = false;
|
||||||
try {
|
try {
|
||||||
while (state.idx < state.sentences.length && !state.cancel) {
|
while (state.idx < state.sentences.length && !state.cancel) {
|
||||||
highlight(state.idx);
|
highlight(state.idx);
|
||||||
const text = state.sentences[state.idx].text;
|
const sentence = state.sentences[state.idx];
|
||||||
let blob;
|
let blob;
|
||||||
try {
|
try {
|
||||||
blob = await fetchTtsPreviewBlob(voice, text, 'mp3');
|
blob = await fetchTtsPreviewBlob(voice, sentence.text, 'mp3');
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
say('Synthesis failed: ' + e.message, 'err');
|
say('Synthesis failed: ' + e.message, 'err');
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
if (state.cancel) break;
|
if (state.cancel) break;
|
||||||
await playBlob(blob);
|
await playBlob(blob, sentence);
|
||||||
if (state.cancel) break;
|
if (state.cancel) break;
|
||||||
state.idx += 1;
|
state.idx += 1;
|
||||||
saveProgress();
|
saveProgress();
|
||||||
}
|
}
|
||||||
|
finishedArticle = state.idx >= state.sentences.length && !state.cancel;
|
||||||
} finally {
|
} finally {
|
||||||
state.playing = false;
|
state.playing = false;
|
||||||
state.cancel = false;
|
state.cancel = false;
|
||||||
syncPlayButton();
|
syncPlayButton();
|
||||||
if (state.idx >= state.sentences.length) {
|
}
|
||||||
state.idx = state.sentences.length - 1;
|
if (finishedArticle) {
|
||||||
|
state.idx = state.sentences.length - 1;
|
||||||
|
if (state.queue.length && state.queueIndex < state.queue.length - 1) {
|
||||||
|
// Play-all: move straight to the next approved article rather than
|
||||||
|
// stopping -- the whole point of a queue is not touching the screen
|
||||||
|
// between articles.
|
||||||
|
state.queueIndex += 1;
|
||||||
|
await magRead(state.queue[state.queueIndex], { fromStart: true });
|
||||||
|
magPlay();
|
||||||
|
} else {
|
||||||
|
state.queue = [];
|
||||||
|
state.queueIndex = -1;
|
||||||
say('Article finished', 'ok');
|
say('Article finished', 'ok');
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
function playBlob(blob) {
|
function playBlob(blob, sentence) {
|
||||||
return new Promise((resolve) => {
|
return new Promise((resolve) => {
|
||||||
const url = URL.createObjectURL(blob);
|
const url = URL.createObjectURL(blob);
|
||||||
const audio = new Audio(url);
|
const audio = new Audio(url);
|
||||||
state.audio = audio;
|
state.audio = audio;
|
||||||
const done = () => { URL.revokeObjectURL(url); state.audio = null; resolve(); };
|
let timings = null;
|
||||||
|
const onTimeUpdate = () => {
|
||||||
|
if (!state.karaoke || !sentence || !sentence.words.length) return;
|
||||||
|
if (!timings) {
|
||||||
|
if (!audio.duration || !isFinite(audio.duration)) return;
|
||||||
|
timings = estimateWordTimings(sentence.words, audio.duration);
|
||||||
|
}
|
||||||
|
const t = audio.currentTime;
|
||||||
|
const wi = timings.findIndex((w) => t >= w.start && t < w.end);
|
||||||
|
if (wi !== -1) highlightWord(state.idx, wi);
|
||||||
|
};
|
||||||
|
if (state.karaoke) audio.addEventListener('timeupdate', onTimeUpdate);
|
||||||
|
const done = () => {
|
||||||
|
audio.removeEventListener('timeupdate', onTimeUpdate);
|
||||||
|
URL.revokeObjectURL(url);
|
||||||
|
state.audio = null;
|
||||||
|
resolve();
|
||||||
|
};
|
||||||
audio.onended = done;
|
audio.onended = done;
|
||||||
audio.onerror = done;
|
audio.onerror = done;
|
||||||
audio.play().catch(done);
|
audio.play().catch(done);
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ── Play all approved articles in sequence ──────────────────────────────
|
||||||
|
async function magPlayAll() {
|
||||||
|
const ids = state.articles.filter((a) => a.status === 'approved').map((a) => a.id);
|
||||||
|
if (!ids.length) return say('Approve at least one article first', 'err');
|
||||||
|
state.queue = ids;
|
||||||
|
state.queueIndex = 0;
|
||||||
|
await magRead(ids[0], { fromStart: true });
|
||||||
|
magPlay();
|
||||||
|
}
|
||||||
|
|
||||||
|
function magStopQueue() {
|
||||||
|
state.queue = [];
|
||||||
|
state.queueIndex = -1;
|
||||||
|
['mag-queue-progress', 'mag-queue-progress-2'].forEach((id) => {
|
||||||
|
const el = $(id);
|
||||||
|
if (el) el.textContent = '';
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async function magQueueNext() {
|
||||||
|
if (!state.queue.length || state.queueIndex >= state.queue.length - 1) return;
|
||||||
|
const wasPlaying = state.playing;
|
||||||
|
if (wasPlaying) magPause();
|
||||||
|
state.queueIndex += 1;
|
||||||
|
await magRead(state.queue[state.queueIndex], { fromStart: true });
|
||||||
|
if (wasPlaying) magPlay();
|
||||||
|
}
|
||||||
|
|
||||||
function magPause() {
|
function magPause() {
|
||||||
state.cancel = true;
|
state.cancel = true;
|
||||||
if (state.audio) { try { state.audio.pause(); } catch (_) {} }
|
if (state.audio) { try { state.audio.pause(); } catch (_) {} }
|
||||||
@ -557,11 +715,13 @@ function magPause() {
|
|||||||
|
|
||||||
function syncPlayButton() {
|
function syncPlayButton() {
|
||||||
const btn = $('mag-play');
|
const btn = $('mag-play');
|
||||||
if (!btn) return;
|
if (btn) {
|
||||||
btn.innerHTML = state.playing
|
btn.innerHTML = state.playing
|
||||||
? '<span class="mdi mdi-pause"></span> Pause'
|
? '<span class="mdi mdi-pause"></span> Pause'
|
||||||
: '<span class="mdi mdi-play"></span> Listen';
|
: '<span class="mdi mdi-play"></span> Listen';
|
||||||
btn.setAttribute('aria-label', state.playing ? 'Pause narration' : 'Play narration');
|
btn.setAttribute('aria-label', state.playing ? 'Pause narration' : 'Play narration');
|
||||||
|
}
|
||||||
|
if (state.driving) syncDrivingPlayButton();
|
||||||
}
|
}
|
||||||
|
|
||||||
let _saveTimer = null;
|
let _saveTimer = null;
|
||||||
@ -707,10 +867,45 @@ function magShow(view) {
|
|||||||
|
|
||||||
function magBackToContents() {
|
function magBackToContents() {
|
||||||
magPause();
|
magPause();
|
||||||
|
magStopQueue();
|
||||||
|
state.driving = false;
|
||||||
magShow('contents');
|
magShow('contents');
|
||||||
magRefreshArticles();
|
magRefreshArticles();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ── Driving mode ──────────────────────────────────────────────────────────
|
||||||
|
// Large, minimal-interaction controls for listening without looking at the
|
||||||
|
// screen (see Manual.md's planned driving-mode behaviour). Reuses whatever
|
||||||
|
// is already playing/queued rather than starting its own playback path --
|
||||||
|
// entering driving mode does not interrupt an article already in progress.
|
||||||
|
function magToggleDriving() {
|
||||||
|
state.driving = !state.driving;
|
||||||
|
const overlay = $('mag-driving');
|
||||||
|
if (overlay) overlay.hidden = !state.driving;
|
||||||
|
if (state.driving) renderDriving();
|
||||||
|
}
|
||||||
|
|
||||||
|
function renderDriving() {
|
||||||
|
const a = state.article;
|
||||||
|
if (!a) return;
|
||||||
|
const titleEl = $('mag-driving-title');
|
||||||
|
if (titleEl) titleEl.textContent = a.title || 'Untitled';
|
||||||
|
const byEl = $('mag-driving-by');
|
||||||
|
if (byEl) byEl.textContent = a.author ? 'By ' + a.author : '';
|
||||||
|
syncDrivingPlayButton();
|
||||||
|
const qEl = $('mag-driving-queue');
|
||||||
|
if (qEl) qEl.hidden = !state.queue.length;
|
||||||
|
}
|
||||||
|
|
||||||
|
function syncDrivingPlayButton() {
|
||||||
|
const btn = $('mag-driving-play');
|
||||||
|
if (!btn) return;
|
||||||
|
btn.innerHTML = state.playing
|
||||||
|
? '<span class="mdi mdi-pause"></span>'
|
||||||
|
: '<span class="mdi mdi-play"></span>';
|
||||||
|
btn.setAttribute('aria-label', state.playing ? 'Pause' : 'Play');
|
||||||
|
}
|
||||||
|
|
||||||
// ── Boot ─────────────────────────────────────────────────────────────────────
|
// ── Boot ─────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
function magInit() {
|
function magInit() {
|
||||||
@ -736,6 +931,10 @@ root.magShow = magShow;
|
|||||||
root.magBackToContents = magBackToContents;
|
root.magBackToContents = magBackToContents;
|
||||||
root.magGeneratePodcast = magGeneratePodcast;
|
root.magGeneratePodcast = magGeneratePodcast;
|
||||||
root.magPlayPodcast = magPlayPodcast;
|
root.magPlayPodcast = magPlayPodcast;
|
||||||
|
root.magPlayAll = magPlayAll;
|
||||||
|
root.magQueueNext = magQueueNext;
|
||||||
|
root.magStopQueue = magStopQueue;
|
||||||
|
root.magToggleDriving = magToggleDriving;
|
||||||
root.magInit = magInit;
|
root.magInit = magInit;
|
||||||
MAG.magazineApi = { load: magLoadLibrary, refresh: magRefreshArticles, splitSentences, fmtPages };
|
MAG.magazineApi = { load: magLoadLibrary, refresh: magRefreshArticles, splitSentences, fmtPages };
|
||||||
|
|
||||||
|
|||||||
@ -1112,11 +1112,18 @@ function _wfUpdateHeaderNav(anyState, curIdx) {
|
|||||||
const nextStep = curIdx >= 0 ? _wfNeighbor(curIdx, 1) : null;
|
const nextStep = curIdx >= 0 ? _wfNeighbor(curIdx, 1) : null;
|
||||||
const curStep = curIdx >= 0 ? WF_STEPS[curIdx] : null;
|
const curStep = curIdx >= 0 ? WF_STEPS[curIdx] : null;
|
||||||
document.querySelectorAll('.section-head').forEach(head => {
|
document.querySelectorAll('.section-head').forEach(head => {
|
||||||
// Studio has its own self-contained 4-phase nav (studio.js) — this
|
// This stepper's Previous/Next buttons are hardcoded to the audiobook
|
||||||
// stepper's Previous/Next buttons are hardcoded to the old s-reader/
|
// pipeline's three destinations (s-reader, s-library, s-rehearser) — an
|
||||||
// s-library/s-rehearser sections and don't know about Studio's phases,
|
// allowlist, not a blocklist, since every OTHER section (Studio has its
|
||||||
// so injecting them here just breaks navigation when clicked.
|
// own 4-phase nav; Engines, Magazines, Settings, etc. have nothing to do
|
||||||
if (head.closest('#s-caststudio')) {
|
// with this pipeline at all) would otherwise need to be named here one
|
||||||
|
// at a time as each one was noticed showing a "Step 1/8" bar and a
|
||||||
|
// "Cast Audiobook" button that make no sense on that page. Confirmed
|
||||||
|
// live: this exact bleed-through appeared on Engines (LLM/STT/TTS/OCR/
|
||||||
|
// Image Generation all share one .section-head) before this became an
|
||||||
|
// allowlist.
|
||||||
|
const onPipelinePage = !!head.closest('#s-reader, #s-library, #s-rehearser');
|
||||||
|
if (!onPipelinePage) {
|
||||||
const staleNav = head.querySelector('.wf-header-nav');
|
const staleNav = head.querySelector('.wf-header-nav');
|
||||||
if (staleNav) staleNav.hidden = true;
|
if (staleNav) staleNav.hidden = true;
|
||||||
const staleHint = head.nextElementSibling?.classList?.contains('wf-step-hint') ? head.nextElementSibling : null;
|
const staleHint = head.nextElementSibling?.classList?.contains('wf-step-hint') ? head.nextElementSibling : null;
|
||||||
|
|||||||
@ -807,6 +807,42 @@
|
|||||||
|
|
||||||
</div><!-- /image -->
|
</div><!-- /image -->
|
||||||
|
|
||||||
|
<div class="s-engines-page" data-page="ocr">
|
||||||
|
|
||||||
|
<!-- ── Active OCR Engine (global picker) ───────────────────── -->
|
||||||
|
<div class="llm-active-panel card">
|
||||||
|
<div class="llm-active-head">
|
||||||
|
<span class="llm-active-icon"><span class="mdi mdi-star-four-points"></span></span>
|
||||||
|
<div>
|
||||||
|
<h4 class="llm-active-title">Active OCR Engine</h4>
|
||||||
|
<p class="llm-active-sub">Used by Magazine ingestion when a page needs OCR, or when its existing text scores poorly. Tesseract (bundled, no setup) is always the fallback if this is unreachable.</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="llm-active-controls">
|
||||||
|
<label class="llm-active-lbl"><span class="mdi mdi-server-network"></span> Endpoint</label>
|
||||||
|
<select id="ocr-active-url" class="llm-active-url-inp" style="appearance: auto; cursor: pointer; padding-right: 25px;">
|
||||||
|
<option value="">— Select an endpoint —</option>
|
||||||
|
</select>
|
||||||
|
<label class="llm-active-lbl"><span class="mdi mdi-brain"></span> Model</label>
|
||||||
|
<select id="ocr-active-model" aria-label="Active OCR model"><option value="">— fetch models —</option></select>
|
||||||
|
<button id="ocr-active-refresh" class="btn-secondary" type="button" title="Fetch models"><span class="mdi mdi-refresh"></span></button>
|
||||||
|
<span id="ocr-active-status" class="llm-active-status"></span>
|
||||||
|
</div>
|
||||||
|
<label class="llm-active-lbl" style="margin-top:10px;display:flex;align-items:center;gap:6px;font-weight:normal">
|
||||||
|
<input type="checkbox" id="ocr-active-enabled"> Enable vision OCR escalation during ingestion
|
||||||
|
</label>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div style="display:flex;justify-content:space-between;align-items:center;margin-top:20px;margin-bottom:12px">
|
||||||
|
<h4 class="llm-local-cat" style="margin:0;text-transform:uppercase">Local</h4>
|
||||||
|
<button class="btn-primary" type="button" onclick="openAddEngineDialog('ocr')"><span class="mdi mdi-plus"></span> Add Custom</button>
|
||||||
|
</div>
|
||||||
|
<div id="dc-grid-ocr" class="dc-grid" style="margin-bottom:14px"></div>
|
||||||
|
|
||||||
|
<div class="llm-info-note" style="margin-top:16px;">Two engines were confirmed live against a real scanned magazine: <strong>Baidu Unlimited-OCR</strong> gives the most accurate text and also labels running headers/footers/page numbers/titles directly (best default). <strong>KDL Frontier Parser nano</strong> answers faster but returns plain text with no layout coordinates or labels — a usable fallback, not a first choice.</div>
|
||||||
|
|
||||||
|
</div><!-- /ocr -->
|
||||||
|
|
||||||
<!-- ── Add Custom Engine dialog ─────────────────────────────── -->
|
<!-- ── Add Custom Engine dialog ─────────────────────────────── -->
|
||||||
<dialog id="add-engine-dlg" class="add-engine-dlg">
|
<dialog id="add-engine-dlg" class="add-engine-dlg">
|
||||||
<div class="add-engine-form">
|
<div class="add-engine-form">
|
||||||
|
|||||||
@ -39,6 +39,9 @@
|
|||||||
</button>
|
</button>
|
||||||
<h3 id="mag-doc-title">Magazine</h3>
|
<h3 id="mag-doc-title">Magazine</h3>
|
||||||
<div class="mag-bar-actions">
|
<div class="mag-bar-actions">
|
||||||
|
<button type="button" class="btn-sm primary" onclick="magPlayAll()">
|
||||||
|
<span class="mdi mdi-playlist-play"></span> Play all articles
|
||||||
|
</button>
|
||||||
<button type="button" class="btn-sm" onclick="magIngest()">
|
<button type="button" class="btn-sm" onclick="magIngest()">
|
||||||
<span class="mdi mdi-refresh"></span> Re-run detection
|
<span class="mdi mdi-refresh"></span> Re-run detection
|
||||||
</button>
|
</button>
|
||||||
@ -70,6 +73,9 @@
|
|||||||
<span id="mag-read-by" class="muted"></span>
|
<span id="mag-read-by" class="muted"></span>
|
||||||
<span id="mag-read-pages" class="muted"></span>
|
<span id="mag-read-pages" class="muted"></span>
|
||||||
</div>
|
</div>
|
||||||
|
<button type="button" class="btn-sm" onclick="magToggleDriving()" title="Large controls for listening without looking at the screen">
|
||||||
|
<span class="mdi mdi-car-outline"></span> Driving mode
|
||||||
|
</button>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div class="mag-controls">
|
<div class="mag-controls">
|
||||||
@ -81,6 +87,11 @@
|
|||||||
<span class="mdi mdi-play"></span> Listen
|
<span class="mdi mdi-play"></span> Listen
|
||||||
</button>
|
</button>
|
||||||
<span id="mag-read-progress" class="muted" aria-live="polite"></span>
|
<span id="mag-read-progress" class="muted" aria-live="polite"></span>
|
||||||
|
<span id="mag-queue-progress" class="muted" aria-live="polite"></span>
|
||||||
|
<label class="mag-karaoke-toggle" title="Highlight each word approximately as it is spoken">
|
||||||
|
<input type="checkbox" id="mag-karaoke-cb" checked onchange="TTVC.magazine.karaoke = this.checked">
|
||||||
|
Word highlighting
|
||||||
|
</label>
|
||||||
<div class="mag-pane-tabs" role="group" aria-label="Reading view">
|
<div class="mag-pane-tabs" role="group" aria-label="Reading view">
|
||||||
<button type="button" class="btn-sm active" data-mag-pane="reflow"
|
<button type="button" class="btn-sm active" data-mag-pane="reflow"
|
||||||
aria-pressed="true" onclick="magSetPane('reflow')">Article</button>
|
aria-pressed="true" onclick="magSetPane('reflow')">Article</button>
|
||||||
@ -90,6 +101,7 @@
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div id="mag-pane-reflow" class="mag-pane">
|
<div id="mag-pane-reflow" class="mag-pane">
|
||||||
|
<div id="mag-read-image" class="mag-read-image" hidden></div>
|
||||||
<div id="mag-article" class="mag-article"></div>
|
<div id="mag-article" class="mag-article"></div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@ -117,3 +129,28 @@
|
|||||||
<div id="mag-podcast-script"></div>
|
<div id="mag-podcast-script"></div>
|
||||||
</details>
|
</details>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<!-- ── Driving mode overlay ─────────────────────────────────────────── -->
|
||||||
|
<!-- Large controls, minimal text, no interaction required beyond play/
|
||||||
|
pause/skip -- reuses whatever is already playing rather than its
|
||||||
|
own playback path. -->
|
||||||
|
<div id="mag-driving" class="mag-driving" hidden role="dialog" aria-label="Driving mode">
|
||||||
|
<button type="button" class="mag-driving-exit" onclick="magToggleDriving()" aria-label="Exit driving mode">
|
||||||
|
<span class="mdi mdi-close"></span>
|
||||||
|
</button>
|
||||||
|
<div class="mag-driving-body">
|
||||||
|
<h2 id="mag-driving-title">Article</h2>
|
||||||
|
<p id="mag-driving-by" class="muted"></p>
|
||||||
|
<div id="mag-driving-queue" class="mag-driving-queue" hidden>
|
||||||
|
<span id="mag-queue-progress-2" class="muted"></span>
|
||||||
|
</div>
|
||||||
|
<div class="mag-driving-controls">
|
||||||
|
<button type="button" class="mag-driving-btn" onclick="magQueueNext()" aria-label="Next article" title="Next article (only in Play all)">
|
||||||
|
<span class="mdi mdi-skip-next"></span>
|
||||||
|
</button>
|
||||||
|
<button type="button" class="mag-driving-btn mag-driving-play" id="mag-driving-play" onclick="magPlay()" aria-label="Play">
|
||||||
|
<span class="mdi mdi-play"></span>
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|||||||
@ -4816,6 +4816,17 @@ input[list="lang-codes"] {
|
|||||||
/* The reading highlight has to survive both themes and colour-blindness, so it
|
/* The reading highlight has to survive both themes and colour-blindness, so it
|
||||||
carries a left rule as well as a tint rather than relying on hue alone. */
|
carries a left rule as well as a tint rather than relying on hue alone. */
|
||||||
.mag-article [data-i].is-current { background: rgba(37,99,235,.16); box-shadow: -3px 0 0 0 var(--accent); }
|
.mag-article [data-i].is-current { background: rgba(37,99,235,.16); box-shadow: -3px 0 0 0 var(--accent); }
|
||||||
|
/* Word-level highlighting is an ESTIMATE (see estimateWordTimings in
|
||||||
|
magazine.js) -- a distinct, lighter treatment from the sentence highlight
|
||||||
|
above keeps that difference legible rather than implying frame-accurate
|
||||||
|
timing. */
|
||||||
|
.mag-w.is-current { background: var(--accent); color: var(--on-accent); border-radius: 2px; }
|
||||||
|
|
||||||
|
.mag-karaoke-toggle { display: inline-flex; align-items: center; gap: 6px; font-size: 13px; color: var(--subtext); }
|
||||||
|
|
||||||
|
.mag-read-image { max-width: 68ch; margin: 0 auto 18px; }
|
||||||
|
.mag-read-image img { width: 100%; height: auto; max-height: 360px; object-fit: contain;
|
||||||
|
border-radius: var(--radius); border: 1px solid var(--border); background: var(--panel); display: block; }
|
||||||
|
|
||||||
.mag-page-wrap { position: relative; max-width: 900px; margin: 0 auto; }
|
.mag-page-wrap { position: relative; max-width: 900px; margin: 0 auto; }
|
||||||
.mag-page-wrap img { width: 100%; height: auto; display: block; border: 1px solid var(--border); border-radius: var(--radius); }
|
.mag-page-wrap img { width: 100%; height: auto; display: block; border: 1px solid var(--border); border-radius: var(--radius); }
|
||||||
@ -4831,8 +4842,31 @@ input[list="lang-codes"] {
|
|||||||
.mag-turns li { margin-bottom: 8px; border-radius: 3px; padding: 2px 4px; }
|
.mag-turns li { margin-bottom: 8px; border-radius: 3px; padding: 2px 4px; }
|
||||||
.mag-turns li.is-current { background: rgba(37,99,235,.16); box-shadow: -3px 0 0 0 var(--accent); }
|
.mag-turns li.is-current { background: rgba(37,99,235,.16); box-shadow: -3px 0 0 0 var(--accent); }
|
||||||
|
|
||||||
|
/* ── Driving mode ──────────────────────────────────────────────────────────
|
||||||
|
Full-screen, minimal-interaction overlay: two oversized controls, large
|
||||||
|
type, nothing else competing for attention. Reuses whatever is already
|
||||||
|
playing/queued -- it is a view, not a separate playback path. */
|
||||||
|
.mag-driving { position: fixed; inset: 0; z-index: 500; background: var(--bg);
|
||||||
|
display: flex; align-items: center; justify-content: center; }
|
||||||
|
.mag-driving-exit { position: absolute; top: 20px; right: 20px; width: 52px; height: 52px;
|
||||||
|
border-radius: 50%; border: 1px solid var(--border); background: var(--surface); color: var(--text);
|
||||||
|
font-size: 22px; display: flex; align-items: center; justify-content: center; cursor: pointer; }
|
||||||
|
.mag-driving-exit:focus-visible { outline: 3px solid var(--accent); outline-offset: 2px; }
|
||||||
|
.mag-driving-body { text-align: center; max-width: 90vw; }
|
||||||
|
.mag-driving-body h2 { font-size: clamp(28px, 5vw, 48px); margin: 0 0 8px; }
|
||||||
|
.mag-driving-queue { margin-bottom: 20px; font-size: 18px; }
|
||||||
|
.mag-driving-controls { display: flex; align-items: center; justify-content: center; gap: 32px; margin-top: 32px; }
|
||||||
|
.mag-driving-btn { width: 96px; height: 96px; border-radius: 50%; border: none;
|
||||||
|
background: var(--panel); color: var(--text); font-size: 36px; cursor: pointer;
|
||||||
|
display: flex; align-items: center; justify-content: center; }
|
||||||
|
.mag-driving-btn.mag-driving-play { width: 140px; height: 140px; font-size: 56px;
|
||||||
|
background: var(--accent); color: var(--on-accent); }
|
||||||
|
.mag-driving-btn:focus-visible { outline: 3px solid var(--accent); outline-offset: 3px; }
|
||||||
|
.mag-driving-btn:active { transform: scale(0.96); }
|
||||||
|
|
||||||
@media (prefers-reduced-motion: reduce) {
|
@media (prefers-reduced-motion: reduce) {
|
||||||
.mag-page-box { transition: none; }
|
.mag-page-box { transition: none; }
|
||||||
|
.mag-driving-btn:active { transform: none; }
|
||||||
}
|
}
|
||||||
@media (max-width: 860px) {
|
@media (max-width: 860px) {
|
||||||
.mag-new-grid, .mag-review-head { grid-template-columns: 1fr; }
|
.mag-new-grid, .mag-review-head { grid-template-columns: 1fr; }
|
||||||
@ -4841,4 +4875,7 @@ input[list="lang-codes"] {
|
|||||||
.mag-block { grid-template-columns: auto 1fr; }
|
.mag-block { grid-template-columns: auto 1fr; }
|
||||||
.mag-block-type { display: none; }
|
.mag-block-type { display: none; }
|
||||||
.mag-article { font-size: 18px; }
|
.mag-article { font-size: 18px; }
|
||||||
|
.mag-driving-controls { gap: 20px; }
|
||||||
|
.mag-driving-btn { width: 76px; height: 76px; font-size: 28px; }
|
||||||
|
.mag-driving-btn.mag-driving-play { width: 110px; height: 110px; font-size: 44px; }
|
||||||
}
|
}
|
||||||
|
|||||||
@ -19,6 +19,7 @@ from core.magazine import (
|
|||||||
ocr_page,
|
ocr_page,
|
||||||
cluster_running_furniture,
|
cluster_running_furniture,
|
||||||
looks_like_real_title,
|
looks_like_real_title,
|
||||||
|
Article,
|
||||||
BLOCK_AD,
|
BLOCK_AD,
|
||||||
BLOCK_BODY,
|
BLOCK_BODY,
|
||||||
BLOCK_BYLINE,
|
BLOCK_BYLINE,
|
||||||
@ -26,9 +27,11 @@ from core.magazine import (
|
|||||||
BLOCK_DECK,
|
BLOCK_DECK,
|
||||||
BLOCK_FURNITURE,
|
BLOCK_FURNITURE,
|
||||||
BLOCK_PULLQUOTE,
|
BLOCK_PULLQUOTE,
|
||||||
|
BLOCK_SIDEBAR,
|
||||||
BLOCK_TITLE,
|
BLOCK_TITLE,
|
||||||
Block,
|
Block,
|
||||||
article_text,
|
article_text,
|
||||||
|
assign_images_to_articles,
|
||||||
assign_reading_order,
|
assign_reading_order,
|
||||||
body_font_size,
|
body_font_size,
|
||||||
build_articles,
|
build_articles,
|
||||||
@ -39,16 +42,19 @@ from core.magazine import (
|
|||||||
estimate_listen_seconds,
|
estimate_listen_seconds,
|
||||||
find_continuation,
|
find_continuation,
|
||||||
find_repeated_lines,
|
find_repeated_lines,
|
||||||
|
image_reading_position,
|
||||||
strip_continuation_markers,
|
strip_continuation_markers,
|
||||||
group_articles,
|
group_articles,
|
||||||
looks_like_byline,
|
looks_like_byline,
|
||||||
mark_advert_pages,
|
mark_advert_pages,
|
||||||
mark_pullquotes,
|
mark_pullquotes,
|
||||||
|
mark_sidebars,
|
||||||
ocr_quality_score,
|
ocr_quality_score,
|
||||||
parse_contents,
|
parse_contents,
|
||||||
parse_vision_ocr_regions,
|
parse_vision_ocr_regions,
|
||||||
score_contents_page,
|
score_contents_page,
|
||||||
vision_ocr_page,
|
vision_ocr_page,
|
||||||
|
vision_ocr_page_chain,
|
||||||
)
|
)
|
||||||
|
|
||||||
PAGE_W, PAGE_H = 595.0, 842.0 # A4 in points
|
PAGE_W, PAGE_H = 595.0, 842.0 # A4 in points
|
||||||
@ -831,5 +837,187 @@ class BuildArticlesTests(unittest.TestCase):
|
|||||||
self.assertGreater(arts[0]["estimated_seconds"], 0)
|
self.assertGreater(arts[0]["estimated_seconds"], 0)
|
||||||
|
|
||||||
|
|
||||||
|
class SidebarDetectionTests(unittest.TestCase):
|
||||||
|
"""BLOCK_SIDEBAR is geometry-only, like the rest of this module: a boxed
|
||||||
|
insert is reliably narrower than the page's regular column grid. This is
|
||||||
|
a heuristic, not a certainty -- review still shows every sidebar block
|
||||||
|
and lets it be ticked back in with one click."""
|
||||||
|
|
||||||
|
def test_a_narrow_third_column_is_marked_a_sidebar(self):
|
||||||
|
blocks = [
|
||||||
|
block("main1", 1, 50, 100, 300, 400, "Main body copy. " * 40),
|
||||||
|
block("side1", 1, 400, 100, 100, 400, "A boxed insert. " * 10),
|
||||||
|
]
|
||||||
|
assign_reading_order(blocks, PAGE_W)
|
||||||
|
mark_sidebars(blocks, PAGE_W)
|
||||||
|
self.assertEqual(next(b for b in blocks if b.id == "side1").type, BLOCK_SIDEBAR)
|
||||||
|
self.assertEqual(next(b for b in blocks if b.id == "main1").type, BLOCK_BODY)
|
||||||
|
|
||||||
|
def test_two_similarly_wide_columns_are_left_alone(self):
|
||||||
|
"""A normal two-column layout must not be mistaken for a sidebar."""
|
||||||
|
blocks = [
|
||||||
|
block("l", 1, 50, 100, 200, 400, "Left column. " * 30),
|
||||||
|
block("r", 1, 280, 100, 200, 400, "Right column. " * 30),
|
||||||
|
]
|
||||||
|
assign_reading_order(blocks, PAGE_W)
|
||||||
|
mark_sidebars(blocks, PAGE_W)
|
||||||
|
self.assertTrue(all(b.type == BLOCK_BODY for b in blocks))
|
||||||
|
|
||||||
|
def test_a_single_column_page_has_no_sidebar_candidates(self):
|
||||||
|
blocks = [block("a", 1, 50, 100, 480, 400, "Full width copy. " * 40)]
|
||||||
|
assign_reading_order(blocks, PAGE_W)
|
||||||
|
mark_sidebars(blocks, PAGE_W)
|
||||||
|
self.assertEqual(blocks[0].type, BLOCK_BODY)
|
||||||
|
|
||||||
|
def test_a_title_or_caption_in_the_narrow_column_is_untouched(self):
|
||||||
|
"""Only BLOCK_BODY is reclassified -- a caption already correctly
|
||||||
|
typed should not be relabelled just for sitting in a narrow column."""
|
||||||
|
blocks = [
|
||||||
|
block("main1", 1, 50, 100, 300, 400, "Main body copy. " * 40),
|
||||||
|
block("cap", 1, 400, 100, 100, 30, "A caption.", btype=BLOCK_CAPTION),
|
||||||
|
]
|
||||||
|
assign_reading_order(blocks, PAGE_W)
|
||||||
|
mark_sidebars(blocks, PAGE_W)
|
||||||
|
self.assertEqual(next(b for b in blocks if b.id == "cap").type, BLOCK_CAPTION)
|
||||||
|
|
||||||
|
def test_sidebar_text_is_excluded_from_narration_by_default(self):
|
||||||
|
"""End-to-end: a detected sidebar must not be swept into the
|
||||||
|
article's narrated blocks the way an advert page already is not."""
|
||||||
|
blocks = [
|
||||||
|
block("t", 18, 50, 60, 470, 50, "The Future of Batteries",
|
||||||
|
size=26.0, btype=BLOCK_TITLE),
|
||||||
|
block("main1", 18, 50, 150, 300, 400, "Main body copy. " * 40),
|
||||||
|
block("side1", 18, 400, 150, 100, 400, "Battery myths. " * 20),
|
||||||
|
]
|
||||||
|
assign_reading_order(blocks, PAGE_W)
|
||||||
|
mark_sidebars(blocks, PAGE_W)
|
||||||
|
arts = group_articles(blocks, [{"title": "The Future of Batteries",
|
||||||
|
"page": 18, "section": ""}], page_count=20)
|
||||||
|
self.assertNotIn("side1", arts[0].blocks)
|
||||||
|
self.assertIn("main1", arts[0].blocks)
|
||||||
|
|
||||||
|
|
||||||
|
class ImageAssignmentTests(unittest.TestCase):
|
||||||
|
"""Extracted figures are useless until linked to the article they
|
||||||
|
illustrate -- assign_images_to_articles is that link."""
|
||||||
|
|
||||||
|
def test_an_image_on_a_single_article_page_is_assigned_to_it(self):
|
||||||
|
art = Article(id="a1", title="T", pages=[5], blocks=["b1"])
|
||||||
|
blocks = [block("b1", 5, 50, 100, 300, 50, "Some text")]
|
||||||
|
images = [{"id": "img1", "page": 5, "bbox": [50, 200, 200, 150]}]
|
||||||
|
assign_images_to_articles(images, blocks, [art])
|
||||||
|
self.assertEqual(art.images, ["img1"])
|
||||||
|
|
||||||
|
def test_an_image_on_a_page_with_no_matching_article_is_skipped(self):
|
||||||
|
art = Article(id="a1", title="T", pages=[5], blocks=[])
|
||||||
|
images = [{"id": "img1", "page": 9, "bbox": [50, 200, 200, 150]}]
|
||||||
|
assign_images_to_articles(images, [], [art])
|
||||||
|
self.assertEqual(art.images, [])
|
||||||
|
|
||||||
|
def test_shared_page_image_goes_to_the_nearer_articles_blocks(self):
|
||||||
|
"""Two articles land on the same page (one ending, one starting) --
|
||||||
|
the figure must follow whichever article it actually sits beside."""
|
||||||
|
art_a = Article(id="a1", title="First", pages=[6], blocks=["ba"])
|
||||||
|
art_b = Article(id="a2", title="Second", pages=[6], blocks=["bb"])
|
||||||
|
blocks = [
|
||||||
|
block("ba", 6, 50, 50, 300, 100, "End of the first article."),
|
||||||
|
block("bb", 6, 50, 500, 300, 100, "Start of the second article."),
|
||||||
|
]
|
||||||
|
# The image sits right next to article B's block (y=520 vs y=500).
|
||||||
|
images = [{"id": "img1", "page": 6, "bbox": [50, 520, 200, 150]}]
|
||||||
|
assign_images_to_articles(images, blocks, [art_a, art_b])
|
||||||
|
self.assertEqual(art_a.images, [])
|
||||||
|
self.assertEqual(art_b.images, ["img1"])
|
||||||
|
|
||||||
|
|
||||||
|
class ImageReadingPositionTests(unittest.TestCase):
|
||||||
|
def test_image_takes_the_position_of_the_nearest_block_above_it(self):
|
||||||
|
blocks = [
|
||||||
|
block("b1", 1, 50, 50, 300, 100, "First paragraph."),
|
||||||
|
block("b2", 1, 50, 300, 300, 100, "Second paragraph."),
|
||||||
|
]
|
||||||
|
blocks[0].reading_order = 0
|
||||||
|
blocks[1].reading_order = 1
|
||||||
|
image = {"page": 1, "bbox": [50, 320, 200, 150]}
|
||||||
|
self.assertEqual(image_reading_position(image, blocks), 1)
|
||||||
|
|
||||||
|
def test_image_above_all_text_falls_back_to_the_nearest_block(self):
|
||||||
|
blocks = [block("b1", 1, 50, 300, 300, 100, "Paragraph.")]
|
||||||
|
blocks[0].reading_order = 5
|
||||||
|
image = {"page": 1, "bbox": [50, 10, 200, 150]}
|
||||||
|
self.assertEqual(image_reading_position(image, blocks), 5)
|
||||||
|
|
||||||
|
def test_no_blocks_on_the_page_returns_zero(self):
|
||||||
|
self.assertEqual(image_reading_position({"page": 9, "bbox": [0, 0, 1, 1]}, []), 0)
|
||||||
|
|
||||||
|
|
||||||
|
class WholePageVisionFallbackTests(unittest.TestCase):
|
||||||
|
"""KDL Frontier Parser nano, confirmed live, answers with plain text and
|
||||||
|
no coordinate markers at all -- unlike ocr-baidu-unlimited-ocr's labeled
|
||||||
|
regions. _whole_page_region is what lets that still produce something
|
||||||
|
usable rather than being discarded outright."""
|
||||||
|
|
||||||
|
def test_plain_text_with_no_markers_becomes_one_page_sized_region(self):
|
||||||
|
from core.magazine import _whole_page_region
|
||||||
|
with mock.patch("PIL.Image.open") as mopen:
|
||||||
|
mopen.return_value.__enter__.return_value.size = (897, 1049)
|
||||||
|
regions = _whole_page_region(
|
||||||
|
"Battery technology is entering a new phase. " * 5,
|
||||||
|
pathlib.Path("/fake.png"))
|
||||||
|
self.assertEqual(len(regions), 1)
|
||||||
|
self.assertEqual(regions[0]["label"], "text")
|
||||||
|
self.assertEqual(regions[0]["bbox"], [0.0, 0.0, 897.0, 1049.0])
|
||||||
|
|
||||||
|
def test_garbage_text_is_rejected_not_wrapped(self):
|
||||||
|
from core.magazine import _whole_page_region
|
||||||
|
self.assertIsNone(_whole_page_region(". | @)X", pathlib.Path("/fake.png")))
|
||||||
|
|
||||||
|
def test_too_short_text_is_rejected(self):
|
||||||
|
from core.magazine import _whole_page_region
|
||||||
|
self.assertIsNone(_whole_page_region("Hi.", pathlib.Path("/fake.png")))
|
||||||
|
|
||||||
|
def test_empty_text_is_rejected(self):
|
||||||
|
from core.magazine import _whole_page_region
|
||||||
|
self.assertIsNone(_whole_page_region("", pathlib.Path("/fake.png")))
|
||||||
|
|
||||||
|
|
||||||
|
class VisionEngineChainTests(unittest.TestCase):
|
||||||
|
"""The primary engine (labeled regions, best quality) is tried first; a
|
||||||
|
differently-shaped secondary is a real fallback, not just tesseract."""
|
||||||
|
|
||||||
|
def test_primary_success_short_circuits_the_fallback(self):
|
||||||
|
with mock.patch("core.magazine.vision_ocr_page") as mocked:
|
||||||
|
mocked.return_value = [{"label": "text", "bbox": [0, 0, 1, 1], "text": "x"}]
|
||||||
|
out = vision_ocr_page_chain(pathlib.Path("/fake.png"),
|
||||||
|
[("http://a", "m1"), ("http://b", "m2")])
|
||||||
|
self.assertEqual(mocked.call_count, 1)
|
||||||
|
self.assertIsNotNone(out)
|
||||||
|
|
||||||
|
def test_primary_failure_falls_through_to_the_secondary(self):
|
||||||
|
with mock.patch("core.magazine.vision_ocr_page") as mocked:
|
||||||
|
mocked.side_effect = [None, [{"label": "text", "bbox": [0, 0, 1, 1], "text": "x"}]]
|
||||||
|
out = vision_ocr_page_chain(pathlib.Path("/fake.png"),
|
||||||
|
[("http://a", "m1"), ("http://b", "m2")])
|
||||||
|
self.assertEqual(mocked.call_count, 2)
|
||||||
|
self.assertIsNotNone(out)
|
||||||
|
|
||||||
|
def test_every_engine_failing_returns_none(self):
|
||||||
|
with mock.patch("core.magazine.vision_ocr_page", return_value=None):
|
||||||
|
out = vision_ocr_page_chain(pathlib.Path("/fake.png"),
|
||||||
|
[("http://a", "m1"), ("http://b", "m2")])
|
||||||
|
self.assertIsNone(out)
|
||||||
|
|
||||||
|
def test_blank_engine_entries_are_skipped_without_calling_out(self):
|
||||||
|
with mock.patch("core.magazine.vision_ocr_page") as mocked:
|
||||||
|
mocked.return_value = [{"label": "text", "bbox": [0, 0, 1, 1], "text": "x"}]
|
||||||
|
vision_ocr_page_chain(pathlib.Path("/fake.png"),
|
||||||
|
[("", ""), ("http://b", "m2")])
|
||||||
|
self.assertEqual(mocked.call_count, 1)
|
||||||
|
self.assertEqual(mocked.call_args[0][1], "http://b")
|
||||||
|
|
||||||
|
def test_empty_engine_list_returns_none(self):
|
||||||
|
self.assertIsNone(vision_ocr_page_chain(pathlib.Path("/fake.png"), []))
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user