tts-voice-creator-clone-and-design-2/Manual.md

# User manual

Living manual. Every feature updates this file in the same commit as the code.

> Sections for the mature features (voice library, Read Aloud, audiobook
> casting, Rehearser) are still to be written — this file was created alongside
> the magazine feature and currently documents that area only.

---

## Audiobook casting (Read Aloud → Cast as Audiobook)

### Purpose

Turn a book into a script with one narrator and one voice per character, ready for
the Rehearser and MP3 export.

### Prerequisites

- A book open in Read Aloud (PDF or text) and an LLM configured in Settings.

### Step by step

1. Open the book, then **Cast as Audiobook**.
2. The app finds the quotation marks in the text and splits it into narration and
   spoken lines by itself — this part never changes a word of the book. The LLM is
   then asked only *who speaks* each spoken line, in passages of a few thousand
   characters. A whole novel takes roughly ten minutes (the previous method took
   hours).
3. Progress shows per passage; **Stop Casting** keeps what is done and **Continue
   casting** resumes.
4. Review the cast as before. **Recast unknown** re-reads only the lines still marked
   Unknown, with more context; a line the model cannot place stays Unknown rather than
   getting a guessed voice. **Verify** (the second quality run) reads every spoken line a
   second time with different passage boundaries — about as long as the first cast — and
   **never changes a speaker**: lines both readings agree on are marked confirmed (about
   4 in 5 of those were right in testing), and lines where they disagree keep the first
   reading, are flagged in the review list and show what the other reading said. The
   percentage next to each line means what it says in testing: confirmed lines were about
   82 % right, disputed lines about 48 %, lines not yet verified about 76 %. Use the flags
   to decide where to spend your review time. Neither step can change the book's
   text, and a stopped or failed run leaves the cast as it was.

**Expected result:** every word of the book appears exactly once, quoted speech is
typed as dialogue and everything else as narration. When the text only refers to a
person by role ("der Fremde", "die Baronin"), the line is labelled with that role;
later passes merge it into the named character when the book itself ties the two
together (e.g. "der Inquisitor Marcian"). Some role labels stay separate — use the
character merge in the cast view for those.

### Errors, limits and privacy

- If the LLM cannot be reached for a passage, its lines are named from the speech
  tags around them where possible and otherwise marked Unknown; nothing is lost.
- A book whose speech is not marked with quotation marks (»…«, „…“, “…”, «…») uses
  the previous whole-passage method automatically. To force the previous method in
  this browser: `localStorage['ab-span-mode'] = '0'`.
- The custom attribution prompt in Settings only applies to the older whole-passage method; span
  mode uses its own fixed instructions (it only asks who speaks each line).
- Book text goes to the LLM you configured, exactly as before; nothing else is sent
  anywhere. Nothing works offline without a reachable LLM.
- Accuracy: measured against hand-corrected casts of three novels, roughly 90–92 % of
  all words end up with the right voice and about three quarters of spoken words with
  the right speaker. Similar-looking characters in the same scene are the usual
  mix-up; check those in review.

---

## Magazines

### Purpose

Listen to a magazine or newspaper one **article** at a time, from its true
beginning to its true end, without adverts, sidebars or a neighbouring article
spliced into the middle. Each article can be narrated by a single voice, or
turned into a podcast discussion between two or three voices you pick.

### Prerequisites

- The magazine as a PDF (native text or scanned — both work).
- At least one voice in the library for narration.
- For podcast mode: an LLM configured in Settings (`llm_url`, `llm_model`), and
  two or three voices chosen as hosts.

### Step by step

1. **Add the magazine.** Studio → *Add Magazine / Paper*. Give it a title and
   issue, then upload the PDF.
2. **Ingest.** Ingestion runs in the background — you can close the browser. It
   renders every page, extracts text (falling back to OCR on scanned pages),
   classifies each region of the layout, finds the contents page, and proposes
   articles with page ranges. While it runs, a live preview shows each page's
   image next to its recognised text as soon as that page finishes -- a
   detected headline is shown larger, a figure appears as its own picture
   marker rather than as garbled on-image text, and page furniture is shown
   dimmed -- so you can see ingestion actually working page by page instead
   of watching a bare progress bar. This preview is a quick approximation,
   not the final read: it runs before the layout is fully classified, so a
   heading guess or an excluded marking here can still change once review
   opens. The page image also shows the **regions as they are found**: each
   box is drawn over the page, coloured by type (headline, paragraph,
   caption, picture, advert, running head …, see the legend under the page)
   and numbered in reading order. After all pages are read, the second pass
   (*Recognising paragraphs, headings and advertisements*) walks the pages
   again and the preview follows it, so you can watch boxes change type —
   the progress bar then reads "… — page 12 of 204". Pictures, charts,
   tables and code are only recognised in this second pass (the first pass
   only reads text, and says so under the page); they are drawn as dashed,
   filled boxes in their own colours — picture orange, chart green, table
   yellow, code grey — so they stand out from the text boxes, and fitted
   to the actual picture as soon as that page is labelled (not only once
   the whole magazine has finished), so the box does not lag behind the
   real edges of the picture for long. The recognised-text panel shows the
   actual cropped picture, chart or table for the same reason, rather than
   a plain "Figure" placeholder, as soon as it has been cropped. The page
   scan itself (a scanned magazine is one picture per page) is not framed.
   A proposed picture, chart or table is checked against the page render
   itself before it is created, so a text area or column gap is not shown
   as a picture. A real picture can still get a loose or shifted box in
   some cases; use *Refine figures* (or re-run detection on the page) to
   try snapping it again, or fix the box by hand on the review page. If a run is interrupted
   (for example when the server restarts), the status now says so instead
   of spinning forever; start it again with *Re-run detection*.
3. **Review the articles.** This step is not optional and cannot be skipped.
   Each proposed article shows its title, author, page range, estimated
   listening time and a confidence score. The review screen is split: the
   scanned page on the left with a numbered, icon-labelled box per detected
   region (title, byline, body text, caption, figure…), and the same regions
   as text on the right — click either side to highlight the other. Sidebars
   and pull-out boxes are already unticked for you — tick one back in if you
   actually want it read. A region already claimed by a different article on
   the same page is marked with that article's number, so it is never
   mistaken for unclaimed text. If OCR quality is poor, **"OCR again"**
   re-reads just the current page through the configured vision engine
   (or **"OCR again (whole magazine)"** from the contents screen for every
   page, which also resets any articles already reviewed). If a figure's
   box is drawn in the wrong place or at the wrong size — a common miss,
   because an OCR engine locates a photo far less precisely than it reads
   type — **"Fix figure boundaries"** re-derives every figure box in the
   issue from the scanned pages themselves. It calls no OCR engine, changes
   no text and touches no approval, so it is safe to run at any point,
   including on an issue you are halfway through reviewing; boxes it cannot
   place confidently are left exactly as they were. If the layout
   engine missed a region entirely, **"Draw it"** lets you click-drag a box
   directly on the page image and add it as text (typed or OCRed) or as a
   new figure. If a region was found but its box is misplaced, the wrong
   size, or shouldn't exist at all, **"Edit boxes"** lets you drag an
   existing box to move it, drag any of its eight edge/corner handles to
   resize just that side, or press its **×** to delete it outright. The
   per-edge handles matter most when two regions share a boundary — for
   example a paragraph split wrongly between two boxes — since dragging just
   the top or bottom handle moves that one shared line without disturbing
   either box's width or its opposite edge. Moving or resizing opens a small
   confirm panel — **Save** keeps the region's existing text at its new
   position; for a text region, **Save & re-OCR** also reads the corrected
   area again and replaces the text with that fresh reading (useful when the
   box was too small or too large to read cleanly the first time, or after
   moving a shared boundary). Deleting a region removes it from the page
   entirely, not just from the article currently under review. Every row in
   the list on the right also has its own **type picker** and **delete
   button** (the bin icon), so you can fix a region without switching into
   Edit boxes: change a paragraph into a *Headline*, a *Page number*, an
   *Advertisement* or any other kind, or turn a text box into an *Image*
   (the box is cut out as a figure) and back (the box is read with OCR).
   Delete asks for confirmation first. The page's reading order is updated
   after a type change, because a page number or advert is read differently
   from running text. **"Fix reading order"** re-reads the order of every
   page in the issue column by column — left column top to bottom, then the
   next column — for issues imported before the column detection improved.
   It changes only the order: text, boxes and approvals stay as they are,
   although an approved article may now be read in a different (correct)
   order. Short pull-quotes are read at the end of the page instead of in
   the middle of a sentence. To **set the order yourself**, either type a new
   number into the position box at the front of a row (the region moves
   there), or press **"Order by clicking"** and click the boxes on the page
   in the order they should be read — each clicked box shows its new number,
   *Undo last* takes one back, and boxes you skip follow in their current
   order — then *Save order*. A page ordered by hand says so above the list
   and is never reshuffled by *Fix reading order*, a type change or a newly
   drawn box (a new box is read last until you move it); **"Automatic
   order"** hands the page back to the automatic column order.
   **"Clean up boxes"** fixes four problems across the whole issue at once:
   short text inside a picture (a label on a screenshot, a word on a slide in
   a photo) becomes *Text in picture*; boxes of one or two characters become
   *OCR noise*; a box that only repeats a bigger box's text (typically a
   subheading the paragraph box also read) becomes *Duplicate*, so it is not
   read twice; and an author line that runs into the article's first
   sentence is split into the name and a paragraph of its own. None of these
   are read aloud, nothing is deleted, every change shows in the list and can
   be undone with the type picker, and a backup of the page layout is kept
   (`layout.backup-<date>.json` in the issue's folder). New imports and
   *OCR again* do this automatically. Longer text inside a picture is left
   for you to judge: it can be real text caught in a picture frame that is
   too big. Each text row also has three quick actions: **Edit text** (the
   pencil) opens the region's text in place — fix OCR mistakes and *Save
   text*, or put the cursor where one paragraph ends and press **Split
   here** to make two regions (the second is read right after the first);
   **Read again** re-runs OCR on just that region; and **Merge with the next
   region** joins a paragraph the OCR cut in two (a word broken by a hyphen
   at the line end is closed up again). Everything works from the keyboard
   too: ↑/↓ move between regions, Space ticks or unticks, Enter edits the
   text, R reads again, M merges with the next region and Delete deletes
   (with Undo); letters typed in the text editor are never taken as
   shortcuts. To fix **many boxes at once**, press **"Select boxes"**, then
   click boxes on the page or drag a rectangle over them (every box at least
   half inside is selected; X selects the focused row from the keyboard). A
   bar shows how many are selected: change their type in one go, or delete
   them. Deleting, changing a type, merging and splitting no longer ask
   "are you sure?" — they happen at once and an **Undo** button appears
   above the list (it undoes the latest change, and only while nothing else
   has changed since). When a region becomes a picture — drawn as one or
   changed into one — the short text inside it (a chart's legend and axis
   labels) automatically becomes *Text in picture*, so there is nothing left
   to delete by hand. Use
   **"Two-page spread"** to see this page next to its facing page (e.g. 22
   and 23), each with its own OCR-results list directly beneath it — useful
   for lining up a figure or a jump that crosses the gutter. Only one page
   is editable at a time; click **"Edit this page"** on the other half to
   switch which one the drawing and editing tools apply to. Confirm the
   boundaries, correct the title, then **Approve**.
   Optionally, click **"Describe"** under a figure, chart, table or code block
   to have the configured vision engine generate a one- or two-sentence
   description for listeners who cannot see the image — e.g. what a chart
   measures and its general trend, or a table's row/column headings and
   what stands out. You can edit the generated text, or write your own from
   scratch, before approving the article. Generating a description changes
   nothing about existing narration by itself — it is only spoken if you
   also turn on "Read figure descriptions" in step 4.
4. **Listen.** An approved article can be narrated with the default voice, or
   sent to podcast mode. Pick the narrator (and the podcast hosts) from the
   same searchable voice picker used elsewhere in the app — open it and type
   to filter the list. Playback synthesises one sentence at a time, staying a
   few sentences ahead, so you only wait for the first. To avoid waiting at
   all, press **Prepare audio** first: it synthesises the whole article now
   (starting from where you are, then the part before it), shows progress as
   "Preparing audio 12 / 24", and can be cancelled with the same button. Once
   it says "Audio ready", **Listen** plays straight through from the prepared
   audio. Prepared audio is held in the browser only — closing or reloading
   the tab, opening another article, or changing the narrator discards it, and
   Prepare audio must be run again. Use **Play all articles** (on the contents screen) to
   listen through every approved article back to back without returning to
   this screen between them. A figure from the page is shown automatically as
   playback reaches it. **Word highlighting** can be turned off if you find it
   distracting — it is an estimate based on word length, not a measurement, so
   it will not always land exactly on the spoken word. Turn on **"Read figure
   descriptions"** in the reading controls to have any figure descriptions
   generated during review spoken aloud where they occur in the article;
   figures without a generated description are silently skipped, exactly as
   before this option existed. **Driving mode**
   (the button next to the article title) is a full-screen, podcast-style player
   for the generated MP3s: previous/next article, **back 15 s / forward 15 s**,
   large play/pause, a seek bar, a **speed** button (0.75×–2×, remembered), and
   **Text** / **Images** toggles that show or hide the sentence being read (with
   the lines before and after) and the article's current picture. Everything is a
   large touch target and keyboard reachable; the speed also applies to the
   playlist player.

**Expected result:** the article plays from its headline to its final paragraph.
Page furniture (running heads, page numbers), adverts, sidebars and pull quotes
are never read aloud unless you explicitly include them in review.

### Confidence, and what it means

| Shown | Meaning |
|---|---|
| ~90% | The contents page listed this article — title and start page are known. |
| ~55% | Detected from a headline alone; check the boundaries carefully. |
| Low | No body copy found. Usually an advert or a section divider — reject it. |

An article is never narrated until you approve it, whatever its confidence.

### Podcast mode

Choose two or three voices as hosts. The configured LLM writes a discussion of
that article for listeners who have not read it, and each host's turns are
synthesised with their voice.

The prompt forbids inventing facts, figures or quotes that are not in the
article; if the article reaches no conclusion the hosts say so. It is still a
generated discussion, not the author's words — treat it as a summary, and read
the original for anything that matters.

Podcast generation is refused for an article you have not approved, because
unreviewed boundaries may include a neighbouring article or an advert.

### Quality check / run (whole magazine)

**Purpose:** find every page where text is not inside a box, boxes overlap or cut off letters, or OCR still looks wrong.
**Prerequisites:** the magazine has been ingested. A vision OCR engine (Settings) is optional; without it the word re-reading step is skipped and the result says so.
**Steps:** on the contents screen press **"Quality check / run"** (first button in the toolbar) and confirm. Progress shows "Checking page N of M".
**Result:** the run (1) adds text regions that independent OCR found outside every box (unticked, so nothing new is read aloud until you approve it), (2) re-reads suspicious words from the page image and only accepts a clearly better reading, (3) measures the page image for ink outside every box, overlapping text boxes, and boxes whose edge cuts through text. Pages that still need a look are listed as buttons under the toolbar; each opens that page in review.
**Errors:** one page failing does not stop the run; it is listed as "check failed". The report is kept in the magazine's folder (`quality_report.json`) and served at `GET /api/magazine/{id}/quality/report`.
**Page colours and live indicator:** in the thumbnail grid, pages the run found clean turn blue and pages with problems turn purple, whatever their review state (only a page whose every article was rejected keeps its grey). A purple warning badge with the problem count sits on the thumbnail; hover or focus it to read what was found, click it to open the page in review. The colour reflects the last run — run the check again after fixing pages. While the run is going, the page being worked on pulses and names its step (finding missed text / re-reading suspicious words / measuring box coverage), and finished pages show "✓ checked". Pulsing is disabled under reduced-motion. The same live indicator works for every batch step while the thumbnail grid is showing: reading pages (OCR), finding advertisements / recognising paragraphs during detection, fixing all figure boundaries, and rendering audio (the pages of the article being rendered pulse with "rendering audio n/total").
**Advertisement pages:** press **Advertisement** on a thumbnail (or **Approve page as advertisement** in review) to approve a whole page as an ad. The page turns green with a dashed border, nothing on it is read aloud (even if an article claims it), and the quality check skips it. Press the button again to undo; no boxes or articles are changed.
**Mobile / accessibility:** the list is a labelled live region of ordinary buttons, keyboard reachable.
**Offline / privacy:** the missed-text pass uses local OCR; the word re-read uses only the vision engine you configured. Nothing else leaves the machine.

### Errors

| What you see | What happened | What to do |
|---|---|---|
| "Upload the PDF first" | The project has no `source.pdf`. | Upload it, then ingest. |
| Ingestion failed | The PDF is encrypted or corrupt. | Check it opens in a viewer. |
| "Approve the article before generating a podcast" | Review gate. | Approve it first. |
| "No LLM model configured" | `llm_model` is empty. | Set it in Settings. |
| "LLM did not return usable JSON" | The model ignored the format. | Retry, or pick a stronger model. |
| "Article has too little text" | Under 200 characters — usually a stub. | Check the boundaries in review. |
| A figure box sits over the wrong part of the page | The OCR engine grounded the figure loosely. | Press **"Fix figure boundaries"**; if the graphic is too pale to detect, draw the region by hand. |
| "Fix figure boundaries" reports nothing corrected | Every box already matched its page, or none could be placed with confidence. | Nothing is wrong; correct any remaining box by drawing it. |
| "No vision OCR engine configured" (Describe) | `vision_ocr_url`/`vision_ocr_model` are empty. | Set a vision engine in Settings. |
| "The vision engine returned nothing usable for this figure" | The configured engine (and fallback, if set) returned an empty or degenerate response for this image. Some vision models are transcription-only and cannot produce open-ended descriptions at all, however the prompt is phrased — confirmed with a pure-OCR model that answers its own "Text Recognition:" prompt but returns nothing for any descriptive one. | Configure a genuinely caption-capable vision model as the fallback (`vision_ocr_fallback_url`/`vision_ocr_fallback_model`), or write the description by hand. |
| "Save & re-OCR" on an edited box leaves the old text in place | The corrected area produced an empty or unreadable OCR result — the same engine limitation that can affect figure descriptions applies here too. | Press **Save** to keep the corrected box position with the existing text, then correct the text by hand if needed, or try "OCR again (this page)" once the box position is saved. |

### Privacy

Ingestion, OCR and storage are entirely local to the container. Podcast mode is
one exception: **the article's text is sent to whatever LLM `llm_url` points
at.** Generating a figure description is the other: **the cropped image is
sent to whatever `vision_ocr_url` (and, on failure, `vision_ocr_fallback_url`)
points at.** If either is a hosted API rather than a model on your own
network, that content leaves your machine. Narration uses your local TTS
stack and sends nothing outward.

### Accessibility and mobile

Sentence and word highlighting both respect `prefers-reduced-motion` (no
smooth-scroll animation when it is set; the highlight itself is a colour
change, not motion). Driving mode gives large, high-contrast controls
(skip, ±15 s, play/pause, speed) — hide Text and Images for a bare screen — for use without
looking at the screen — it does not, and should not, replace watching the
road. Word-level highlighting can be switched off in the reading controls if
it is more distracting than helpful; sentence highlighting keeps working
either way. Keyboard navigation of the contents tree and review screen is
not yet built.

### Offline

Ingested magazines are stored server-side, so any device on the network sees the
same library and reading progress. Playback needs the server; there is no
offline cache yet.