tts-voice-creator-clone-and-design-2/architecture/audiobook-attribution.md

# SOP — Audiobook speaker attribution (span mode)

Status: **implemented, default on.** Switch off with `localStorage['ab-span-mode'] = '0'`
(the classic whole-passage engine then runs). A book with no recognisable quotation
style automatically uses the classic engine.

## Why it changed

The classic engine sends ~3,000-character passages to an LLM and asks it to re-emit
the whole passage as JSON segments (`/api/attribute-dialogue`). That has three costs:

* **Text corruption.** The model fuses, duplicates or drops text; a stack of
  post-processing passes (dedupe, recover-dropped-text, prefix-dup cut, orphan-quote
  repair) exists only to undo it. Measured on a hand-corrected book: narration/dialogue
  typing agreed on 93.6 % of words.
* **Hours per book.** Output tokens ≈ input tokens (a 4 h run for one novel).
* **Unmeasurable.** Exact-text matching counts any boundary difference as an error.

## Design

```
book text ──► segmentByQuotes()      deterministic (static/js/audiobook-spans.js)
              narration / dialogue spans with exact source offsets
          ──► buildWindows()         ~2.4–3.6k chars per LLM call, cut at paragraphs
          ──► POST /api/attribute-spans   LLM answers ONLY {id, speaker, emotion} per dialogue span
          ──► assemble segments      text is taken from the spans, never from the LLM
          ──► existing finalisation  phantom drop, role→name aliases, name consolidation,
                                     confidence, autosave, Rehearser save, auto-repair
```

The LLM can no longer change, drop, duplicate or fuse a single word; it can only choose
a speaker, so its output is a few tokens per dialogue line.

### Segmenter rules (`audiobook-spans.js`)

* One quote style per book, chosen by counting well-formed pairs (`»…«` German inward,
  `„…“`, `“…”`, `«…»` French). No style with ≥ 5 pairs → classic engine.
* A closing mark with no opener (opener lost in PDF extraction): the speech starts at the
  earliest run of speech-like sentences directly before it (1st/2nd-person pronouns, `?`,
  short `!`).
* An opener whose closer is lost: the speech ends after the leading run of speech-like
  sentences.
* A speech continues across paragraph breaks while a closing mark still follows before the
  next opener (some typesetting re-opens every paragraph, some only closes the last).
* A quote that is a term/title inside a sentence (no sentence-final punctuation, lower-case
  continuation) is narration.
* Adjacent narration merges; each quote stays its own dialogue span.

### `POST /api/attribute-spans`

Request: `{spans:[{id, kind, text}], known_characters, recent, following, language, llm_url,
model, timeout_seconds, want_reasoning}`. Ids are `N<n>` / `D<n>`; `kind` must match the id
prefix; ≤ 400 spans, ≤ 20 000 chars per span, ≤ 60 000 total, at least one dialogue span
(HTTP 400 otherwise).
Response: `{speakers:{id:{speaker, emotion}}, missing:[ids], characters:[names]}`, or
`{failed:true, error, missing:[…]}` (HTTP 200) when the LLM is unavailable — the client then
falls back to speech-tag guessing. The text is never returned.

Behaviour: thinking is disabled by default (`chat_template_kwargs.enable_thinking=false`,
retried without the key on HTTP 400/422). Measured: reasoning mode was ~25× slower with no
accuracy gain on the same windows. An answer that parses to nothing is retried once with a
larger token budget; lines the model skipped are retried once on their own. Calls are
serialised with every other attribution call.

Naming rule: the model answers with the label the text uses ("Der Fremde", "Baronin",
"Oberst") when a person is only referred to by role in the scene, instead of guessing another
known character's name. Role → name resolution then happens book-wide in the existing
`_audiobookResolveRoleAliases` / `_audiobookConsolidateSpeakerAliases` passes.

## Measuring

`scripts/score_attribution.py <manual.json> <auto.json> [--characters <manual chars>]` aligns
the two casts word by word (insensitive to chunking) and reports coverage, narration/dialogue
type agreement, dialogue speaker accuracy (strict and alias-tolerant) and the largest
confusions. **Read it with care:** the human references contain label noise (inline narration
folded into the neighbouring dialogue, inconsistent handling of very long monologues,
occasional wrong speakers), so differences of ~1–2 points are noise.

Book 1 (*Der Sturm*), same reference, same scorer:

| | classic (Run 3) | span mode |
|---|---|---|
| type agreement | 93.6 % | 97.4 % |
| dialogue words, right speaker (tolerant) | 60.6 % | 75.4 % |
| overall every word right (tolerant) | 87.0 % | 92.2 % |
| wall time | ≈ 4 h | ≈ 11 min |
| Unknown-speaker words | 16 | 114 |

## Security / privacy

Book text is sent to the configured LLM exactly as before (same endpoint, same settings);
the server never returns text. Inputs are validated at the boundary (see limits above). No
new credentials, storage or network destinations.

## Rollback

`localStorage['ab-span-mode'] = '0'` restores the classic engine per browser; removing the
dispatch at the top of `audiobookCast()` (`static/js/audiobook-pipeline.js`) removes it for
everyone. Casts made by either engine are the same shape and remain readable by the other.