feat(tts): inline pause, spell and speed tags

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -7,6 +7,7 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
 
 ## [Unreleased]
 ### Added
+- **Inline text tags for TTS (setting `inline_tags_enabled`, default OFF).** `[pause 500ms]`/`[pause 1.2s]` (max 10 s each, 60 s per text), `[spell]USA[/spell]` (German letter names for German text), `[slow]..[/slow]` / `[fast]..[/fast]` (0.85 / 1.15 via ffmpeg atempo, innermost wins, unclosed ends at paragraph). Known control tags (e.g. `[laugh]`) are stripped, never spoken; `[1]` and `[sic]` stay. Only `POST /api/tts-preview` renders them; other paths are listed in `architecture/inline-tags.md`. Tests: `tests/test_ssml_lite.py`.
 - **Optional vocal isolation for URL/YouTube imports (default OFF).** `POST /api/isolate-vocals {id}` runs the Demucs CLI (`htdemucs`, two stems) on a registered temp audio and registers the vocals as a new id; the original stays valid. Needs `VOCAL_ISOLATION=1`, a `demucs` binary (or `VOCAL_ISOLATION_CMD`) and the weights already on disk; the app never downloads anything and nothing is added to the main image. Limits: 15 min input, 15 min runtime (`VOCAL_ISOLATION_MAX_SECONDS`/`_TIMEOUT`), one job at a time, cancelled on client disconnect, temp files always removed. `GET /api/version` reports `features.vocal_isolation`; the "Remove background music" button (clone page URL card and voice library add-URL) appears only then, with an aria-live status and a play original / play without music toggle. Tests: `tests/test_vocal_isolation.py`, `tests/vocal-isolation.spec.js`. See `architecture/vocal-isolation.md`.
 - **Voice export/import bundles.** `GET /api/voices/{id}/export` streams a `<id>.voice.zip` (audio, `reference.txt`, `meta.json` with seed, design prompt, rating/tags, `provenance`, `audio_problems`; optional picture and consent recording; `manifest.json` with format version, app version and sha256 per file; no temp files). `POST /api/voices/import` validates at the boundary: 150 MB upload cap, fixed flat member names only (zip-slip/path traversal rejected), per-file, total and compression-ratio caps, hashes verified, audio decoded, meta and provenance re-validated; a taken name gets a `_2` suffix, an identical voice is a no-op, nothing is overwritten. Voice library toolbar: "Export selected" and "Import bundle" (44px). Tests: `tests/test_voice_bundle.py`, `tests/voice-bundle.spec.js`. See `architecture/voice-bundle.md`.
 - **Consent/provenance field per voice.** Each voice gets `provenance` in its `.meta.json` (`basis`: own_voice, consented, licensed, public_domain or unknown; `source`, `licence`, `note`, optional `recorded_consent`, `set_at`). Existing voices count as `unknown`; `quick-import-voice` prefills the source URL (basis stays `unknown`). New `PUT /api/voices/{id}/provenance` (enum, length caps, control characters stripped, http(s) URLs only, no credentials) and `POST`/`GET /api/voices/{id}/consent-recording` (audio only, 10 MB cap, stored as `<voice>.consent`, moves/deletes with the voice). The voice inspector has a "Source & consent" section (typed note or MediaRecorder recording) and voices without a recorded basis show a "Source unknown" badge. Generation is not blocked. Data stays local. Tests: `tests/test_voice_provenance.py`, `tests/voice-provenance.spec.js`.

Manual.md

@@ -8,6 +8,17 @@ Living manual. Every feature updates this file in the same commit as the code.
 
 ---
 
+## Inline tags in the text (optional)
+Purpose: pauses, spelled-out acronyms and slower/faster passages inside one text. Prerequisite: set `inline_tags_enabled` to true in the settings (default off; when off the tags are spoken literally).
+
+| Tag | Result |
+|---|---|
+| `[pause 500ms]` / `[pause 1.2s]` | silence, max 10 s each and 60 s per text |
+| `[spell]USA[/spell]` | letters one by one (German letter names for German text) |
+| `[slow]..[/slow]` / `[fast]..[/fast]` | 0.85x / 1.15x speed, same pitch; innermost wins; unclosed ends at the paragraph |
+
+Known control tags such as `[laugh]` are removed, never read; `[1]` and `[sic]` stay. Works with every engine in the Voice Generator preview/generate path (speed via ffmpeg, so no engine support needed). Not yet in the OpenAI-compatible proxy, streaming, chat or audiobook/magazine audio. Errors: a failing engine reports the usual TTS error. Mobile/accessibility/privacy: no UI change, text stays local.
+
 ## Design a Voice: attribute chips
 
 **Purpose:** build a voice description quickly. **Use:** open Design a Voice, press chips (gender, age, pitch, pace, accent/region, tone/timbre, emotion). Each chip adds its phrase to the description; pressing it again removes only that phrase. Gender, age, pitch and pace allow one choice each. **Free text:** type anything in the box; chips never overwrite your text and are not a limit on what you can describe. **Keyboard/touch:** chips are buttons (Tab, Space/Enter, `aria-pressed`, 44 px targets). **Languages:** chip labels follow the UI language (German/English); the inserted phrase is English. **Offline/privacy:** purely local, nothing is sent until you generate.

architecture/inline-tags.md

@@ -0,0 +1,23 @@
+# Inline text tags (feature flag `inline_tags_enabled`, default off)
+
+Engine-independent preprocessing in `core/ssml_lite.py` (pure parser plus `render`). Our engines have no native tags, so the app splits text, synthesises each piece and joins the audio itself. Idea only is borrowed from other tools; no third-party code.
+
+## Syntax
+| Tag | Effect |
+|---|---|
+| `[pause 500ms]`, `[pause 1.2s]` | generated silence; max 10 s each, 60 s per text; invalid argument is dropped |
+| `[spell]USA[/spell]` | letters spoken one by one (`U S A`; German letter names `Uh Ess Ah` when language is German) |
+| `[slow]..[/slow]`, `[fast]..[/fast]` | speed factor 0.85 / 1.15 via ffmpeg `atempo` (`core.audio._change_tempo`) on the rendered segment; innermost wins |
+
+Unclosed `[slow]`/`[fast]`/`[spell]` end at the paragraph break (blank line).
+
+## Stripping rule
+Only these names are removed and never spoken (opening or closing, optional short argument): the four tags above plus `laugh(s/ing) chuckle sigh gasp cough breath breathe inhale exhale whisper shout music sfx noise beat silence emphasis excited sad angry happy calm sniff gulp clears_throat`. Everything else in brackets (`[1]`, `[sic]`, `[Anm.]`) is ordinary prose and untouched.
+
+## Flow
+`parse(text, language)` -> segments `{text, speed, pause_ms}` (tags never reach a segment, so any sentence chunker run on a segment cannot cut a tag) -> `render()` synthesises per segment, stretches, joins with silence -> one WAV. Duration therefore already includes pauses and stretching.
+
+## Coverage
+Covered: `POST /api/tts-preview` (main generate path; seed-finder calls only get tags stripped via `plain_text`).
+Not covered: `/v1/audio/speech` proxy, streaming playback, conversation/chat, audiobook casting and magazine MP3/read-aloud (timing records `write_audio_timings` assume untagged text; tags there would be spoken literally). Engine speed params are not used: `atempo` works the same for every engine.
+Rollback: set `inline_tags_enabled` to false (default).

core/config.py

@@ -45,6 +45,7 @@ _SETTINGS_KEYS = {
     # LLM so page images never accidentally go to a text-only model.
     "magazine_layout_url", "magazine_layout_model",
     "vision_ocr_url", "vision_ocr_model", "vision_ocr_enabled",
+    "inline_tags_enabled",
     "vision_ocr_fallback_url", "vision_ocr_fallback_model",
     # OPDS catalogues to search for magazine issues: [{"name": ..., "url": ...}], no credentials.
     "magazine_opds_catalogs",
@@ -346,6 +347,7 @@ def _load_settings() -> dict:
         "vision_ocr_url":         _VISION_OCR_DEFAULT,
         "vision_ocr_model":       _VISION_OCR_MODEL_DEFAULT,
         "vision_ocr_enabled":     True,
+        "inline_tags_enabled":    False,
         "vision_ocr_fallback_url":   _VISION_OCR_FALLBACK_DEFAULT,
         "vision_ocr_fallback_model": _VISION_OCR_FALLBACK_MODEL_DEFAULT,
         "engine_local_urls":      {},

core/ssml_lite.py

@@ -0,0 +1,167 @@
+"""Inline text tags for TTS, engine independent (idea only; no third-party code).
+
+Tags: [pause 500ms] [pause 1.2s], [spell]USA[/spell], [slow]..[/slow], [fast]..[/fast].
+parse() turns text into segments {text, speed, pause_ms}; render() synthesises and joins them.
+Only a fixed set of tag names is ever stripped, so "[1]" or "[sic]" stay untouched.
+"""
+from __future__ import annotations
+
+import re
+from typing import Callable
+
+MAX_PAUSE_MS = 10_000
+MAX_TOTAL_PAUSE_MS = 60_000
+SPEEDS = {"slow": 0.85, "fast": 1.15}
+
+# Control-style tags from other tools that must never be spoken. Allowlist, not "any [word]".
+STRIP_NAMES = frozenset({
+    "laugh", "laughs", "laughing", "chuckle", "sigh", "gasp", "cough", "breath", "breathe",
+    "inhale", "exhale", "whisper", "shout", "music", "sfx", "noise", "beat", "silence",
+    "emphasis", "excited", "sad", "angry", "happy", "calm", "sniff", "gulp", "clears_throat",
+})
+
+_TAG_RE = re.compile(r"\[(/?)([A-Za-z_]+)(?:[ \t]+([^\[\]\n]{0,24}))?\]")
+_PAUSE_ARG_RE = re.compile(r"^(\d+(?:\.\d+)?)\s*(ms|s)$", re.I)
+_PARA_RE = re.compile(r"(\n\s*\n)")
+
+_DE_LETTERS = {
+    "a": "Ah", "b": "Be", "c": "Ze", "d": "De", "e": "Eh", "f": "Eff", "g": "Ge", "h": "Ha",
+    "i": "Ih", "j": "Jott", "k": "Ka", "l": "Ell", "m": "Emm", "n": "Enn", "o": "Oh", "p": "Pe",
+    "q": "Ku", "r": "Err", "s": "Ess", "t": "Te", "u": "Uh", "v": "Fau", "w": "We", "x": "Iks",
+    "y": "Üpsilon", "z": "Zett", "ä": "Ä", "ö": "Ö", "ü": "Ü", "ß": "Ess-Zett",
+}
+
+
+def _spell(content: str, language: str) -> str:
+    chars = [c for c in content if not c.isspace()]
+    if language == "de":
+        return " ".join(_DE_LETTERS.get(c.lower(), c) for c in chars)
+    from core.pronunciations import spell_letters
+    return spell_letters("".join(chars))
+
+
+def _lang(text: str, language: str) -> str:
+    low = (language or "auto").strip().lower()
+    if low in {"de", "german", "deutsch"} or low.startswith("de-"):
+        return "de"
+    if low in {"", "auto"}:
+        from core.pronunciations import detect_text_language
+        return detect_text_language(text)
+    return "en"
+
+
+def _pause_ms(arg: str | None) -> int | None:
+    m = _PAUSE_ARG_RE.match((arg or "").strip())
+    if not m:
+        return None
+    ms = float(m.group(1)) * (1000 if m.group(2).lower() == "s" else 1)
+    return min(int(ms), MAX_PAUSE_MS)
+
+
+def has_tags(text: str) -> bool:
+    return any(m.group(2).lower() in {"pause", "spell", "slow", "fast"} | STRIP_NAMES
+               for m in _TAG_RE.finditer(text or ""))
+
+
+def _parse_paragraph(text: str, lang: str) -> list[dict]:
+    out: list[dict] = []
+    speeds: list[float] = []
+    spell_from: int | None = None          # index in `out` where an open [spell] started
+    pos = 0
+
+    def add(chunk: str) -> None:
+        if chunk:
+            out.append({"text": chunk, "speed": speeds[-1] if speeds else 1.0, "pause_ms": 0})
+
+    def close_spell() -> None:
+        nonlocal spell_from
+        if spell_from is not None:
+            joined = "".join(s["text"] for s in out[spell_from:])
+            del out[spell_from:]
+            add(_spell(joined, lang))
+            spell_from = None
+
+    for m in _TAG_RE.finditer(text):
+        name, closing = m.group(2).lower(), m.group(1) == "/"
+        known = name in {"pause", "spell", "slow", "fast"} or name in STRIP_NAMES
+        if not known:
+            continue                       # ordinary prose such as "[1]" or "[sic]": handled below
+        add(text[pos:m.start()])
+        pos = m.end()
+        if name == "pause" and not closing:
+            ms = _pause_ms(m.group(3))
+            if ms:
+                out.append({"text": "", "speed": 1.0, "pause_ms": ms})
+        elif name == "spell":
+            if closing:
+                close_spell()
+            elif spell_from is None:
+                spell_from = len(out)
+        elif name in SPEEDS:
+            if closing:
+                if speeds:
+                    speeds.pop()
+            else:
+                speeds.append(SPEEDS[name])
+        # anything else in STRIP_NAMES (and a stray [/pause]) is dropped
+    add(text[pos:])
+    close_spell()                          # unclosed [spell] runs to the end of the paragraph
+    return out
+
+
+def parse(text: str, language: str = "auto") -> list[dict]:
+    """Split text into segments; unclosed tags end at the paragraph break; pauses are capped."""
+    lang = _lang(text, language)
+    segments: list[dict] = []
+    for part in _PARA_RE.split(text or ""):
+        if _PARA_RE.fullmatch(part):
+            if segments and not segments[-1]["pause_ms"]:
+                segments[-1]["text"] += part
+            continue
+        segments.extend(_parse_paragraph(part, lang))
+    budget = MAX_TOTAL_PAUSE_MS
+    merged: list[dict] = []
+    for seg in segments:
+        if seg["pause_ms"]:
+            seg["pause_ms"] = min(seg["pause_ms"], budget)
+            budget -= seg["pause_ms"]
+            if not seg["pause_ms"]:
+                continue
+        elif merged and not merged[-1]["pause_ms"] and merged[-1]["speed"] == seg["speed"]:
+            merged[-1]["text"] += seg["text"]
+            continue
+        merged.append(seg)
+    return [s for s in merged if s["pause_ms"] or s["text"].strip()]
+
+
+def plain_text(text: str, language: str = "auto") -> str:
+    """Text with all tags resolved and no timing (for paths that cannot render pauses/speed)."""
+    return re.sub(r"[ \t]+", " ", " ".join(s["text"].strip() for s in parse(text, language) if s["text"].strip()))
+
+
+def render(text: str, synth: Callable[[str], tuple[bytes, str]], tempo: Callable[[bytes, float], bytes],
+           language: str = "auto") -> tuple[bytes, str]:
+    """Synthesise every segment, stretch slow/fast ones, join with generated silence -> WAV bytes.
+
+    The result's duration already includes pauses and stretching, so timings measured on it stay right.
+    """
+    import io
+    from pydub import AudioSegment
+
+    combined = AudioSegment.empty()
+    for seg in parse(text, language):
+        if seg["pause_ms"]:
+            rate = combined.frame_rate if len(combined) else 24000
+            combined += AudioSegment.silent(duration=seg["pause_ms"], frame_rate=rate)
+            continue
+        audio, media_type = synth(seg["text"].strip())
+        fmt = "wav" if audio[:4] == b"RIFF" or "wav" in media_type.lower() else None
+        piece = AudioSegment.from_file(io.BytesIO(audio), format=fmt)
+        if abs(seg["speed"] - 1.0) > 0.01:
+            buf = io.BytesIO()
+            piece.export(buf, format="wav")
+            piece = AudioSegment.from_file(io.BytesIO(tempo(buf.getvalue(), seg["speed"])), format="wav")
+        combined += piece
+    out = io.BytesIO()
+    combined.export(out, format="wav")
+    return out.getvalue(), "audio/wav"

routes/tts.py

@@ -39,6 +39,7 @@ from core.voice import (
 )
 from core.voice_index import indexed_voices, refresh_voice_index_background
 from core.validation import _validate_http_url
+from core import ssml_lite
 from core.provider_health import probe_provider
 from core.registry import _registry_put, TEMP_DIR
 from core.tts_helpers import (
@@ -828,9 +829,18 @@ async def tts_preview(request: Request):
 
     try:
         if backend == "voice_clone" and data.get("seed_finder") and "seed" in _overrides:
+            if settings.get("inline_tags_enabled"):
+                text = ssml_lite.plain_text(text, str(data.get("language") or "auto"))
             audio, media_type = await _bounded_tts_call(
                 _seed_finder_request_audio, text, voice, settings, int(_overrides["seed"])
             )
+        elif settings.get("inline_tags_enabled") and ssml_lite.has_tags(text):
+            from core.audio import _change_tempo
+            audio, media_type = await _bounded_tts_call(
+                ssml_lite.render, text,
+                lambda t: _preview_request_audio(t, voice, settings, instruct, backend),
+                _change_tempo, str(data.get("language") or "auto"),
+            )
         else:
             audio, media_type = await _bounded_tts_call(_preview_request_audio, text, voice, settings, instruct, backend)
     except HTTPException:

tests/test_ssml_lite.py

@@ -0,0 +1,84 @@
+import io
+import unittest
+
+from pydub import AudioSegment
+from pydub.generators import Sine
+
+from core import ssml_lite as S
+from core.magazine import split_sentences_for_tts
+
+
+def texts(segs):
+    return [(s["text"], s["speed"], s["pause_ms"]) for s in segs]
+
+
+class ParseTests(unittest.TestCase):
+    def test_plain_text_is_one_segment(self):
+        self.assertEqual(texts(S.parse("Hello there.")), [("Hello there.", 1.0, 0)])
+
+    def test_pause_units_and_split(self):
+        self.assertEqual(texts(S.parse("A [pause 500ms] B [pause 1.2s] C")),
+                         [("A ", 1.0, 0), ("", 1.0, 500), (" B ", 1.0, 0), ("", 1.0, 1200), (" C", 1.0, 0)])
+
+    def test_pause_caps(self):
+        self.assertEqual(S.parse("A[pause 99s]B")[1]["pause_ms"], 10_000)
+        many = "A" + "[pause 10s]B" * 10
+        self.assertEqual(sum(s["pause_ms"] for s in S.parse(many)), 60_000)
+
+    def test_bad_pause_is_stripped_not_spoken(self):
+        self.assertEqual(S.plain_text("A [pause soon] B"), "A B")
+
+    def test_speed_nesting_innermost_wins(self):
+        segs = S.parse("a [slow]b [fast]c[/fast] d[/slow] e")
+        self.assertEqual([(s["text"].strip(), s["speed"]) for s in segs],
+                         [("a", 1.0), ("b", 0.85), ("c", 1.15), ("d", 0.85), ("e", 1.0)])
+
+    def test_unclosed_runs_to_paragraph_end(self):
+        segs = S.parse("[slow]one\n\ntwo")
+        self.assertEqual([(s["text"].strip(), s["speed"]) for s in segs], [("one", 0.85), ("two", 1.0)])
+
+    def test_unknown_control_tags_stripped_citations_kept(self):
+        self.assertEqual(S.plain_text("Hi [laugh] there [/emphasis] [sic] cited [1]."), "Hi there [sic] cited [1].")
+        self.assertNotIn("[", S.plain_text("[whisper]psst[/whisper]"))
+
+    def test_spell_english_and_german(self):
+        self.assertEqual(S.plain_text("[spell]USA[/spell]", "en"), "U S A")
+        self.assertEqual(S.plain_text("[spell]USA[/spell]", "de"), "Uh Ess Ah")
+        self.assertEqual(S.plain_text("[spell]ZDF[/spell] Nachrichten über Straße"), "Zett De Eff Nachrichten über Straße")
+        self.assertEqual(S.plain_text("[spell]ab", "en"), "a b")
+
+    def test_segments_never_contain_tags_and_chunker_keeps_them_whole(self):
+        text = "[slow]First. Second sentence here.[/slow] Third. [pause 1s] Fourth."
+        for seg in S.parse(text):
+            self.assertNotIn("[", seg["text"])
+        slow = [s for s in S.parse(text) if s["speed"] != 1.0][0]
+        self.assertEqual(slow["speed"], 0.85)
+        self.assertIn("First.", slow["text"])
+        self.assertIn("Second", slow["text"])
+        self.assertGreaterEqual(len(split_sentences_for_tts(slow["text"])), 1)
+
+    def test_has_tags(self):
+        self.assertTrue(S.has_tags("a [pause 1s]"))
+        self.assertFalse(S.has_tags("see [1] and [sic]"))
+
+
+class RenderTests(unittest.TestCase):
+    def test_render_joins_with_silence_and_stretch(self):
+        tone = Sine(440).to_audio_segment(duration=1000).set_frame_rate(24000)
+        buf = io.BytesIO()
+        tone.export(buf, format="wav")
+        wav = buf.getvalue()
+
+        def tempo(b, speed):                       # fake: just trims, proves it was called with the factor
+            tempo.calls.append(speed)
+            return b
+        tempo.calls = []
+        out, media = S.render("A [pause 500ms] [slow]B[/slow]", lambda t: (wav, "audio/wav"), tempo, "en")
+        dur = len(AudioSegment.from_file(io.BytesIO(out), format="wav"))
+        self.assertEqual(media, "audio/wav")
+        self.assertEqual(tempo.calls, [0.85])
+        self.assertAlmostEqual(dur, 2500, delta=30)
+
+
+if __name__ == "__main__":
+    unittest.main()