test(audiobook): cover emotion to instruct path and document audiobook emotions

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -7,6 +7,7 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
 
 ## [Unreleased]
 ### Added
+- **Audiobook emotion tests.** Playwright `tests/audiobook-emotion.spec.js` (mocked routes) covers span-cast emotion -> `instruct` in `/api/tts-preview` (dialogue only, English tag, narration empty) and the line cache key; `tests/test_tts_instruct_payload.py` covers the instruct passthrough.
 - **LibriVox and Wikimedia Commons voice sources.** New tab in Get Voices Online searches the official LibriVox feed and the Commons MediaWiki API through `GET /api/voice-sources/{librivox|wikimedia}?q=&lang=` (fixed hosts, query cap 80, 10 results, 30 s timeout, 10 min cache, 1 s throttle, descriptive User-Agent, result URLs host-checked). Results show licence and a preview; "Import clip" trims (start/length) and stores provenance (LibriVox: public domain + check-the-reader note; Wikimedia: licence + attribution). `archive.org` and `upload.wikimedia.org` added to the audio proxy allowlist. Tests: `tests/test_voice_sources_new.py`, `tests/voice-sources-new.spec.js`. See `architecture/voice-sources.md`.
 - **Provenance tags on exported audio.** Magazine MP3 downloads, `audio.zip` and the MP3s inside the EPUB get ID3 tags (`synthetic_speech=true`, `generator`, `generator_version`, `generated_date`, comment; voice name only on opt-in) via one ffmpeg stream copy of a temp copy; stored audio is untouched. Per-export checkboxes on the magazine export row ("Mark as synthetic speech", "include voice name"), default from new setting `export_provenance_default` (on). `POST /api/export/provenance` and `python -m core.export_provenance file.mp3` read the tags back. Inaudible AudioSeal watermarking is researched (MIT code and weights, torch needed, weak message recovery after MP3/AAC) but only a stub `watermark(path)` that reports unavailable. Spoken disclosure is not implemented yet. Tests: `tests/test_watermark_tags.py`, `tests/magazine.spec.js`. See `architecture/watermarking.md`.
 - **Inline text tags for TTS (setting `inline_tags_enabled`, default OFF).** `[pause 500ms]`/`[pause 1.2s]` (max 10 s each, 60 s per text), `[spell]USA[/spell]` (German letter names for German text), `[slow]..[/slow]` / `[fast]..[/fast]` (0.85 / 1.15 via ffmpeg atempo, innermost wins, unclosed ends at paragraph). Known control tags (e.g. `[laugh]`) are stripped, never spoken; `[1]` and `[sic]` stay. Rendered in `POST /api/tts-preview` (also used by audiobook line previews), `/v1/audio/speech` (non-streaming), `/speak` and the MCP speak tool through one shared `ssml_lite.render_with_tags`; stripped only (never spoken) in streaming, spoken chat replies and magazine MP3 (timings stay exact). Settings -> General switch (accessible, DE/EN) and a syntax hint under the Try It Out and Read Aloud text boxes. Tests: `tests/test_ssml_lite.py`, `tests/test_inline_tags_paths.py`, `tests/inline-tags.spec.js`.
@@ -17,6 +18,9 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
 - **Voice Design: attribute chips.** Toggle chips (gender, age, pitch, pace, accent/region, tone/timbre, emotion; German + English labels) above the design prompt add or remove only their own phrase in the free-text description. The textarea stays fully free-form. Test: `tests/design-chips.spec.js`.
 - **Voice health: timestamped audio problems.** `core.audio.audio_problems` flags long interior silence, clipping and sudden level jumps (`problems` list with `kind`, `start_s`, `end_s`, `detail`, plus one warning per kind) in the route-test voice health; existing keys unchanged. Computed on demand and cached by file mtime, not in the voice list. Shown in the routing test result and in the Try-out voice panel (new `GET /api/voices/{id}/health`); imports store the result in meta (`audio_problems`). Tests: `tests/test_audio_problems.py`, `tests/voice-health.spec.js`.
 
+### Docs
+- **Audiobook emotions** section in `Manual.md` (filling, editing, voice types that honour them, limits) and an Emotions section in `architecture/audiobook-attribution.md`.
+
 ### Changed
 - **Magazine interviews: journalist/guest voices.** Interview labels such as `c't:` (also `c’t`, `ct`, OCR `Cc't`), `Redaktion`, `Interviewer`, `Frage` are the interviewing side and are read in the narrator voice; every other recurring label is a guest and gets the second voice. Unlabelled blocks continue the current speaker's turn; headings, decks and bylines stay narrator. Detection stays strict (label starts a block and recurs at least twice). Audio-timing records now store each segment's voice and speaker. Tests: `tests/test_magazine.py` (`InterviewJournalistTests`).
 

Manual.md

@@ -284,6 +284,26 @@ locally against your own database; nothing is sent anywhere.
 
 **Errors:** if the book title cannot be read, the prompt to pick a book is shown. **Mobile/accessibility:** bands are headings with a show/hide button (`aria-expanded`); the search field is labelled; layouts fit 320 px. **Offline/privacy:** nothing new is sent; the view reads the local library.
 
+## Audiobook emotions
+
+**Purpose:** dialogue lines can be spoken with a mood (angry, whispering, sad) instead of flat.
+
+**How it is filled:** during casting the LLM answers `{speaker, emotion}` for every dialogue span, with 1-2 words in the book's language (e.g. "wütend"). Narration never gets an emotion. If the LLM fails or returns nothing for a line, the emotion stays empty (line is read neutrally); nothing is invented. The emotion is shown in brackets after the speaker name in the cast view.
+
+**Editing:** edit the emotion per line in the Rehearser (tone field), or reassign the speaker in the cast view. Changing it invalidates the cached audio of that line, so the next synthesis/export re-reads it.
+
+**What happens to it:** the Rehearser turns the word into an English style instruction (German words are translated; unknown words are dropped) and sends it as `instruct` to the TTS engine. Fish-Speech gets an inline `[tag]` instead.
+
+**Which voices honour it:** designed voices (Voice Design backend) and Fish-Speech react clearly. Cloned voices (Voice Clone backend) may ignore the instruction; the Rehearser shows a tone warning for such backends.
+
+**Limits:**
+- In span mode a failed LLM call leaves the emotion empty.
+- A custom attribution prompt is ignored in span mode (the fixed span prompt is used); it applies only to the classic engine.
+- The merge pass keeps one emotion per merged block (the first non-empty one), so neighbouring lines of the same speaker can share a mood.
+- Emotion words outside the built-in German-to-English table are not translated and have no effect on Fish-Speech.
+
+**Mobile/accessibility:** the emotion field is a normal labelled text input (keyboard and screen reader operable). **Offline/privacy:** the emotion comes from the same LLM call as the speaker; no extra data is sent.
+
 ## Magazines
 
 ### Purpose

architecture/audiobook-attribution.md

@@ -142,6 +142,20 @@ just before it) gave +0.3 points — noise — and was not adopted.
 Request additions: `attribute_only` `[ids]` — the other spans are context; unrequested lines are never
 reported missing.
 
+## Emotions
+
+`/api/attribute-spans` returns `{speaker, emotion}` per dialogue id (1-2 words in the book's language; narration ids never
+get one). Path to audio: segment `emotion` -> `audiobookBuildScript` / `rehState.lines[].emotion` -> `audiobookLineVoice`
+-> `_buildInstruct` (English clause; German words via `_rehEmotionEnglishTag`) -> `fetchTtsPreviewBlob` ->
+`payload["instruct"]` (`core/tts_helpers.py`). The line audio cache key includes the instruct, so an edited emotion never
+serves stale audio. Fish-Speech uses `_rehInlineTone` ([tag]) instead.
+
+Limits: engine support differs (`routes/tts.py` `_backend_capabilities`: `voice_clone` style_aware false, instruct may be
+ignored; `voice_design` strong). Span-mode LLM failure leaves emotion empty. A custom attribution prompt is ignored in span
+mode. The merge pass keeps one emotion per block. Tests: `tests/audiobook-emotion.spec.js`,
+`tests/test_tts_instruct_payload.py`. No per-voice UI hint yet: the voice is only chosen later in the Rehearser, which already
+shows a tone warning for non-style-aware backends.
+
 ## Security / privacy
 
 Book text is sent to the configured LLM exactly as before (same endpoint, same settings);

tests/audiobook-emotion.spec.js

@@ -0,0 +1,65 @@
+// @ts-check
+// Audiobook emotion path: span-cast segment emotion -> script -> rehearser line -> instruct -> /api/tts-preview.
+// Routes are mocked; nothing here reaches the live TTS API.
+const { test, expect } = require('@playwright/test');
+
+const WAV = Buffer.from('UklGRiQAAABXQVZFZm10IBAAAAABAAEAQB8AAIA+AAACABAAZGF0YQAAAAA=', 'base64');
+
+async function openReader(page) {
+  await page.goto('/', { waitUntil: 'domcontentloaded' });
+  await page.waitForSelector('#app-skeleton', { state: 'detached', timeout: 15_000 });
+  await page.waitForFunction(() => typeof window.navTo === 'function', null, { timeout: 15_000 });
+  await page.evaluate(() => window.navTo('s-reader'));
+  await page.waitForFunction(() => typeof window.audiobookBuildScript === 'function' && typeof window.fetchTtsPreviewBlob === 'function', null, { timeout: 15_000 });
+}
+
+test('span-cast emotion reaches the tts-preview instruct for dialogue only; export shares the path', async ({ page }) => {
+  const bodies = [];
+  await page.route('**/api/tts-preview', route => { bodies.push(route.request().postDataJSON()); return route.fulfill({ status: 200, contentType: 'audio/wav', body: WAV }); });
+  await page.route('**/api/attribute-dialogue', route => route.fulfill({ json: { segments: [] } }));
+  await openReader(page);
+
+  // segments exactly as audiobook-cast-spans.js builds them (emotion from the LLM, narration always '')
+  const res = await page.evaluate(async () => {
+    const segs = [
+      { speaker: 'Narrator', type: 'narration', text: 'Der Zwerg schneuzte sich.', emotion: '', spanId: 'N0' },
+      { speaker: 'Linosch', type: 'dialogue', text: 'Na los!', emotion: 'wütend', spanId: 'D1' },
+      { speaker: 'Alrik', type: 'dialogue', text: 'Vielen Dank.', emotion: '', spanId: 'D2' },
+    ];
+    const { script, emotions } = window.audiobookBuildScript(segs);
+    rehState.lines = window.parseScript(script);
+    rehState.cast = { LINOSCH: { voice: 'DE_M_test', instruct: '' }, ALRIK: { voice: 'DE_M_test2', instruct: '' } };
+    rehState.narratorVoice = 'DE_F_narr';
+    let k = 0;
+    rehState.lines.forEach(l => { if (l.type === 'dialog') { const e = emotions[k++]; if (e) l.emotion = e; } });
+    const out = [];
+    for (const l of rehState.lines) {
+      const { voice, instruct } = window.audiobookLineVoice(l);
+      await window.fetchTtsPreviewBlob(voice, l.text, 'wav', instruct, 'voice_design');
+      out.push({ text: l.text, type: l.type, emotion: l.emotion || '' });
+    }
+    return { out, exportUsesIt: /audiobookLineVoice\(/.test(window.audiobookExport.toString()) };
+  });
+  expect(res.exportUsesIt).toBe(true);        // the export loop builds its instruct via the same helper
+  expect(bodies).toHaveLength(3);
+  const byText = Object.fromEntries(bodies.map(b => [b.text, b.instruct]));
+  expect(byText['Der Zwerg schneuzte sich.']).toBe('');
+  expect(byText['Na los!']).toContain('angry');          // German "wütend" translated to the English tag
+  expect(byText['Na los!']).not.toContain('wütend');
+  expect(byText['Vielen Dank.']).not.toMatch(/angry/);   // empty emotion: no emotion clause
+});
+
+test('changing the emotion changes the line cache key (no stale flat audio)', async ({ page }) => {
+  await openReader(page);
+  const keys = await page.evaluate(async () => {
+    window.rehState.cast = { LINOSCH: { voice: 'DE_M_test', instruct: '' } };
+    const k = async (emo) => {
+      const { instruct } = window.audiobookLineVoice({ type: 'dialog', speaker: 'LINOSCH', text: 'Na los!', emotion: emo });
+      return window._lineAudioCacheKey('Na los!', 'DE_M_test', instruct);
+    };
+    return [await k(''), await k('wütend'), await k('wütend'), await k('traurig')];
+  });
+  expect(keys[0]).not.toBe(keys[1]);
+  expect(keys[1]).toBe(keys[2]);
+  expect(keys[1]).not.toBe(keys[3]);
+});

tests/test_tts_instruct_payload.py

@@ -0,0 +1,29 @@
+"""Emotion -> instruct passthrough in the TTS payload builders (core/tts_helpers.py)."""
+import os
+import tempfile
+import unittest
+
+os.environ.setdefault("CONFIG_DIR", tempfile.mkdtemp(prefix="tip-cfg-"))
+
+from core.tts_helpers import _tts_request_config, _tts_stream_request_config
+
+S = {"tts_url": "http://localhost:9999", "tts_backend": "openai"}
+
+
+class InstructPayloadTests(unittest.TestCase):
+    def test_instruct_is_sent_stripped(self):
+        _, p, _ = _tts_request_config("Hi", "DE_M_x", S, instruct="  Speak in an angry tone.  ")
+        self.assertEqual(p["instruct"], "Speak in an angry tone.")
+        _, p, _ = _tts_stream_request_config("Hi", "DE_M_x", S, "  angry  ")
+        self.assertEqual(p["instruct"], "angry")
+
+    def test_empty_instruct_is_omitted(self):
+        for ins in ("", "   "):
+            _, p, _ = _tts_request_config("Hi", "DE_M_x", S, instruct=ins)
+            self.assertNotIn("instruct", p)
+            _, p, _ = _tts_stream_request_config("Hi", "DE_M_x", S, ins)
+            self.assertNotIn("instruct", p)
+
+
+if __name__ == "__main__":
+    unittest.main()