- fix: Edit boxes and page-nav controls were dead clicks (handlers never exported to root); resize from any edge/corner instead of only bottom-right - feat: two-page spread in review (page images above, OCR lists beneath) - feat: searchable voice picker for narrator and podcast hosts - feat: Prepare audio synthesises the whole article ahead of playback; playback also stays three sentences ahead - fix: sync index.html cache-busting version with VERSION (was stuck at 1.26.12) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
188 lines
12 KiB
Markdown
188 lines
12 KiB
Markdown
# User manual
|
||
|
||
Living manual. Every feature updates this file in the same commit as the code.
|
||
|
||
> Sections for the mature features (voice library, Read Aloud, audiobook
|
||
> casting, Rehearser) are still to be written — this file was created alongside
|
||
> the magazine feature and currently documents that area only.
|
||
|
||
---
|
||
|
||
## Magazines
|
||
|
||
### Purpose
|
||
|
||
Listen to a magazine or newspaper one **article** at a time, from its true
|
||
beginning to its true end, without adverts, sidebars or a neighbouring article
|
||
spliced into the middle. Each article can be narrated by a single voice, or
|
||
turned into a podcast discussion between two or three voices you pick.
|
||
|
||
### Prerequisites
|
||
|
||
- The magazine as a PDF (native text or scanned — both work).
|
||
- At least one voice in the library for narration.
|
||
- For podcast mode: an LLM configured in Settings (`llm_url`, `llm_model`), and
|
||
two or three voices chosen as hosts.
|
||
|
||
### Step by step
|
||
|
||
1. **Add the magazine.** Studio → *Add Magazine / Paper*. Give it a title and
|
||
issue, then upload the PDF.
|
||
2. **Ingest.** Ingestion runs in the background — you can close the browser. It
|
||
renders every page, extracts text (falling back to OCR on scanned pages),
|
||
classifies each region of the layout, finds the contents page, and proposes
|
||
articles with page ranges. While it runs, a live preview shows each page's
|
||
image next to its recognised text as soon as that page finishes -- a
|
||
detected headline is shown larger, a figure appears as its own picture
|
||
marker rather than as garbled on-image text, and page furniture is shown
|
||
dimmed -- so you can see ingestion actually working page by page instead
|
||
of watching a bare progress bar. This preview is a quick approximation,
|
||
not the final read: it runs before the layout is fully classified, so a
|
||
heading guess or an excluded marking here can still change once review
|
||
opens.
|
||
3. **Review the articles.** This step is not optional and cannot be skipped.
|
||
Each proposed article shows its title, author, page range, estimated
|
||
listening time and a confidence score. The review screen is split: the
|
||
scanned page on the left with a numbered, icon-labelled box per detected
|
||
region (title, byline, body text, caption, figure…), and the same regions
|
||
as text on the right — click either side to highlight the other. Sidebars
|
||
and pull-out boxes are already unticked for you — tick one back in if you
|
||
actually want it read. A region already claimed by a different article on
|
||
the same page is marked with that article's number, so it is never
|
||
mistaken for unclaimed text. If OCR quality is poor, **"OCR again"**
|
||
re-reads just the current page through the configured vision engine
|
||
(or **"OCR again (whole magazine)"** from the contents screen for every
|
||
page, which also resets any articles already reviewed). If a figure's
|
||
box is drawn in the wrong place or at the wrong size — a common miss,
|
||
because an OCR engine locates a photo far less precisely than it reads
|
||
type — **"Fix figure boundaries"** re-derives every figure box in the
|
||
issue from the scanned pages themselves. It calls no OCR engine, changes
|
||
no text and touches no approval, so it is safe to run at any point,
|
||
including on an issue you are halfway through reviewing; boxes it cannot
|
||
place confidently are left exactly as they were. If the layout
|
||
engine missed a region entirely, **"Draw it"** lets you click-drag a box
|
||
directly on the page image and add it as text (typed or OCRed) or as a
|
||
new figure. If a region was found but its box is misplaced, the wrong
|
||
size, or shouldn't exist at all, **"Edit boxes"** lets you drag an
|
||
existing box to move it, drag any of its eight edge/corner handles to
|
||
resize just that side, or press its **×** to delete it outright. The
|
||
per-edge handles matter most when two regions share a boundary — for
|
||
example a paragraph split wrongly between two boxes — since dragging just
|
||
the top or bottom handle moves that one shared line without disturbing
|
||
either box's width or its opposite edge. Moving or resizing opens a small
|
||
confirm panel — **Save** keeps the region's existing text at its new
|
||
position; for a text region, **Save & re-OCR** also reads the corrected
|
||
area again and replaces the text with that fresh reading (useful when the
|
||
box was too small or too large to read cleanly the first time, or after
|
||
moving a shared boundary). Deleting a region removes it from the page
|
||
entirely, not just from the article currently under review. Use
|
||
**"Two-page spread"** to see this page next to its facing page (e.g. 22
|
||
and 23), each with its own OCR-results list directly beneath it — useful
|
||
for lining up a figure or a jump that crosses the gutter. Only one page
|
||
is editable at a time; click **"Edit this page"** on the other half to
|
||
switch which one the drawing and editing tools apply to. Confirm the
|
||
boundaries, correct the title, then **Approve**.
|
||
Optionally, click **"Describe"** under a figure, chart, table or code block
|
||
to have the configured vision engine generate a one- or two-sentence
|
||
description for listeners who cannot see the image — e.g. what a chart
|
||
measures and its general trend, or a table's row/column headings and
|
||
what stands out. You can edit the generated text, or write your own from
|
||
scratch, before approving the article. Generating a description changes
|
||
nothing about existing narration by itself — it is only spoken if you
|
||
also turn on "Read figure descriptions" in step 4.
|
||
4. **Listen.** An approved article can be narrated with the default voice, or
|
||
sent to podcast mode. Pick the narrator (and the podcast hosts) from the
|
||
same searchable voice picker used elsewhere in the app — open it and type
|
||
to filter the list. Playback synthesises one sentence at a time, staying a
|
||
few sentences ahead, so you only wait for the first. To avoid waiting at
|
||
all, press **Prepare audio** first: it synthesises the whole article now
|
||
(starting from where you are, then the part before it), shows progress as
|
||
"Preparing audio 12 / 24", and can be cancelled with the same button. Once
|
||
it says "Audio ready", **Listen** plays straight through from the prepared
|
||
audio. Prepared audio is held in the browser only — closing or reloading
|
||
the tab, opening another article, or changing the narrator discards it, and
|
||
Prepare audio must be run again. Use **Play all articles** (on the contents screen) to
|
||
listen through every approved article back to back without returning to
|
||
this screen between them. A figure from the page is shown automatically as
|
||
playback reaches it. **Word highlighting** can be turned off if you find it
|
||
distracting — it is an estimate based on word length, not a measurement, so
|
||
it will not always land exactly on the spoken word. Turn on **"Read figure
|
||
descriptions"** in the reading controls to have any figure descriptions
|
||
generated during review spoken aloud where they occur in the article;
|
||
figures without a generated description are silently skipped, exactly as
|
||
before this option existed. **Driving mode**
|
||
(the button next to the article title) switches to two large controls and
|
||
nothing else, for listening without looking at the screen.
|
||
|
||
**Expected result:** the article plays from its headline to its final paragraph.
|
||
Page furniture (running heads, page numbers), adverts, sidebars and pull quotes
|
||
are never read aloud unless you explicitly include them in review.
|
||
|
||
### Confidence, and what it means
|
||
|
||
| Shown | Meaning |
|
||
|---|---|
|
||
| ~90% | The contents page listed this article — title and start page are known. |
|
||
| ~55% | Detected from a headline alone; check the boundaries carefully. |
|
||
| Low | No body copy found. Usually an advert or a section divider — reject it. |
|
||
|
||
An article is never narrated until you approve it, whatever its confidence.
|
||
|
||
### Podcast mode
|
||
|
||
Choose two or three voices as hosts. The configured LLM writes a discussion of
|
||
that article for listeners who have not read it, and each host's turns are
|
||
synthesised with their voice.
|
||
|
||
The prompt forbids inventing facts, figures or quotes that are not in the
|
||
article; if the article reaches no conclusion the hosts say so. It is still a
|
||
generated discussion, not the author's words — treat it as a summary, and read
|
||
the original for anything that matters.
|
||
|
||
Podcast generation is refused for an article you have not approved, because
|
||
unreviewed boundaries may include a neighbouring article or an advert.
|
||
|
||
### Errors
|
||
|
||
| What you see | What happened | What to do |
|
||
|---|---|---|
|
||
| "Upload the PDF first" | The project has no `source.pdf`. | Upload it, then ingest. |
|
||
| Ingestion failed | The PDF is encrypted or corrupt. | Check it opens in a viewer. |
|
||
| "Approve the article before generating a podcast" | Review gate. | Approve it first. |
|
||
| "No LLM model configured" | `llm_model` is empty. | Set it in Settings. |
|
||
| "LLM did not return usable JSON" | The model ignored the format. | Retry, or pick a stronger model. |
|
||
| "Article has too little text" | Under 200 characters — usually a stub. | Check the boundaries in review. |
|
||
| A figure box sits over the wrong part of the page | The OCR engine grounded the figure loosely. | Press **"Fix figure boundaries"**; if the graphic is too pale to detect, draw the region by hand. |
|
||
| "Fix figure boundaries" reports nothing corrected | Every box already matched its page, or none could be placed with confidence. | Nothing is wrong; correct any remaining box by drawing it. |
|
||
| "No vision OCR engine configured" (Describe) | `vision_ocr_url`/`vision_ocr_model` are empty. | Set a vision engine in Settings. |
|
||
| "The vision engine returned nothing usable for this figure" | The configured engine (and fallback, if set) returned an empty or degenerate response for this image. Some vision models are transcription-only and cannot produce open-ended descriptions at all, however the prompt is phrased — confirmed with a pure-OCR model that answers its own "Text Recognition:" prompt but returns nothing for any descriptive one. | Configure a genuinely caption-capable vision model as the fallback (`vision_ocr_fallback_url`/`vision_ocr_fallback_model`), or write the description by hand. |
|
||
| "Save & re-OCR" on an edited box leaves the old text in place | The corrected area produced an empty or unreadable OCR result — the same engine limitation that can affect figure descriptions applies here too. | Press **Save** to keep the corrected box position with the existing text, then correct the text by hand if needed, or try "OCR again (this page)" once the box position is saved. |
|
||
|
||
### Privacy
|
||
|
||
Ingestion, OCR and storage are entirely local to the container. Podcast mode is
|
||
one exception: **the article's text is sent to whatever LLM `llm_url` points
|
||
at.** Generating a figure description is the other: **the cropped image is
|
||
sent to whatever `vision_ocr_url` (and, on failure, `vision_ocr_fallback_url`)
|
||
points at.** If either is a hosted API rather than a model on your own
|
||
network, that content leaves your machine. Narration uses your local TTS
|
||
stack and sends nothing outward.
|
||
|
||
### Accessibility and mobile
|
||
|
||
Sentence and word highlighting both respect `prefers-reduced-motion` (no
|
||
smooth-scroll animation when it is set; the highlight itself is a colour
|
||
change, not motion). Driving mode gives two large, high-contrast controls
|
||
(skip, play/pause) with nothing else to interact with, for use without
|
||
looking at the screen — it does not, and should not, replace watching the
|
||
road. Word-level highlighting can be switched off in the reading controls if
|
||
it is more distracting than helpful; sentence highlighting keeps working
|
||
either way. Keyboard navigation of the contents tree and review screen is
|
||
not yet built.
|
||
|
||
### Offline
|
||
|
||
Ingested magazines are stored server-side, so any device on the network sees the
|
||
same library and reading progress. Playback needs the server; there is no
|
||
offline cache yet.
|