feat(tts): inline tags in generate, audiobook and magazine paths

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
mARTin-B78 2026-10-04 17:35:18 +02:00
parent 5a7c73ea42
commit 9b169037a4
14 changed files with 256 additions and 17 deletions

View File

@ -8,7 +8,7 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
## [Unreleased] ## [Unreleased]
### Added ### Added
- **Provenance tags on exported audio.** Magazine MP3 downloads, `audio.zip` and the MP3s inside the EPUB get ID3 tags (`synthetic_speech=true`, `generator`, `generator_version`, `generated_date`, comment; voice name only on opt-in) via one ffmpeg stream copy of a temp copy; stored audio is untouched. Per-export checkboxes on the magazine export row ("Mark as synthetic speech", "include voice name"), default from new setting `export_provenance_default` (on). `POST /api/export/provenance` and `python -m core.export_provenance file.mp3` read the tags back. Inaudible AudioSeal watermarking is researched (MIT code and weights, torch needed, weak message recovery after MP3/AAC) but only a stub `watermark(path)` that reports unavailable. Spoken disclosure is not implemented yet. Tests: `tests/test_watermark_tags.py`, `tests/magazine.spec.js`. See `architecture/watermarking.md`. - **Provenance tags on exported audio.** Magazine MP3 downloads, `audio.zip` and the MP3s inside the EPUB get ID3 tags (`synthetic_speech=true`, `generator`, `generator_version`, `generated_date`, comment; voice name only on opt-in) via one ffmpeg stream copy of a temp copy; stored audio is untouched. Per-export checkboxes on the magazine export row ("Mark as synthetic speech", "include voice name"), default from new setting `export_provenance_default` (on). `POST /api/export/provenance` and `python -m core.export_provenance file.mp3` read the tags back. Inaudible AudioSeal watermarking is researched (MIT code and weights, torch needed, weak message recovery after MP3/AAC) but only a stub `watermark(path)` that reports unavailable. Spoken disclosure is not implemented yet. Tests: `tests/test_watermark_tags.py`, `tests/magazine.spec.js`. See `architecture/watermarking.md`.
- **Inline text tags for TTS (setting `inline_tags_enabled`, default OFF).** `[pause 500ms]`/`[pause 1.2s]` (max 10 s each, 60 s per text), `[spell]USA[/spell]` (German letter names for German text), `[slow]..[/slow]` / `[fast]..[/fast]` (0.85 / 1.15 via ffmpeg atempo, innermost wins, unclosed ends at paragraph). Known control tags (e.g. `[laugh]`) are stripped, never spoken; `[1]` and `[sic]` stay. Only `POST /api/tts-preview` renders them; other paths are listed in `architecture/inline-tags.md`. Tests: `tests/test_ssml_lite.py`. - **Inline text tags for TTS (setting `inline_tags_enabled`, default OFF).** `[pause 500ms]`/`[pause 1.2s]` (max 10 s each, 60 s per text), `[spell]USA[/spell]` (German letter names for German text), `[slow]..[/slow]` / `[fast]..[/fast]` (0.85 / 1.15 via ffmpeg atempo, innermost wins, unclosed ends at paragraph). Known control tags (e.g. `[laugh]`) are stripped, never spoken; `[1]` and `[sic]` stay. Rendered in `POST /api/tts-preview` (also used by audiobook line previews), `/v1/audio/speech` (non-streaming), `/speak` and the MCP speak tool through one shared `ssml_lite.render_with_tags`; stripped only (never spoken) in streaming, spoken chat replies and magazine MP3 (timings stay exact). Settings -> General switch (accessible, DE/EN) and a syntax hint under the Try It Out and Read Aloud text boxes. Tests: `tests/test_ssml_lite.py`, `tests/test_inline_tags_paths.py`, `tests/inline-tags.spec.js`.
- **Optional vocal isolation for URL/YouTube imports (default OFF).** `POST /api/isolate-vocals {id}` runs the Demucs CLI (`htdemucs`, two stems) on a registered temp audio and registers the vocals as a new id; the original stays valid. Needs `VOCAL_ISOLATION=1`, a `demucs` binary (or `VOCAL_ISOLATION_CMD`) and the weights already on disk; the app never downloads anything and nothing is added to the main image. Limits: 15 min input, 15 min runtime (`VOCAL_ISOLATION_MAX_SECONDS`/`_TIMEOUT`), one job at a time, cancelled on client disconnect, temp files always removed. `GET /api/version` reports `features.vocal_isolation`; the "Remove background music" button (clone page URL card and voice library add-URL) appears only then, with an aria-live status and a play original / play without music toggle. Tests: `tests/test_vocal_isolation.py`, `tests/vocal-isolation.spec.js`. See `architecture/vocal-isolation.md`. - **Optional vocal isolation for URL/YouTube imports (default OFF).** `POST /api/isolate-vocals {id}` runs the Demucs CLI (`htdemucs`, two stems) on a registered temp audio and registers the vocals as a new id; the original stays valid. Needs `VOCAL_ISOLATION=1`, a `demucs` binary (or `VOCAL_ISOLATION_CMD`) and the weights already on disk; the app never downloads anything and nothing is added to the main image. Limits: 15 min input, 15 min runtime (`VOCAL_ISOLATION_MAX_SECONDS`/`_TIMEOUT`), one job at a time, cancelled on client disconnect, temp files always removed. `GET /api/version` reports `features.vocal_isolation`; the "Remove background music" button (clone page URL card and voice library add-URL) appears only then, with an aria-live status and a play original / play without music toggle. Tests: `tests/test_vocal_isolation.py`, `tests/vocal-isolation.spec.js`. See `architecture/vocal-isolation.md`.
- **Voice export/import bundles.** `GET /api/voices/{id}/export` streams a `<id>.voice.zip` (audio, `reference.txt`, `meta.json` with seed, design prompt, rating/tags, `provenance`, `audio_problems`; optional picture and consent recording; `manifest.json` with format version, app version and sha256 per file; no temp files). `POST /api/voices/import` validates at the boundary: 150 MB upload cap, fixed flat member names only (zip-slip/path traversal rejected), per-file, total and compression-ratio caps, hashes verified, audio decoded, meta and provenance re-validated; a taken name gets a `_2` suffix, an identical voice is a no-op, nothing is overwritten. Voice library toolbar: "Export selected" and "Import bundle" (44px). Tests: `tests/test_voice_bundle.py`, `tests/voice-bundle.spec.js`. See `architecture/voice-bundle.md`. - **Voice export/import bundles.** `GET /api/voices/{id}/export` streams a `<id>.voice.zip` (audio, `reference.txt`, `meta.json` with seed, design prompt, rating/tags, `provenance`, `audio_problems`; optional picture and consent recording; `manifest.json` with format version, app version and sha256 per file; no temp files). `POST /api/voices/import` validates at the boundary: 150 MB upload cap, fixed flat member names only (zip-slip/path traversal rejected), per-file, total and compression-ratio caps, hashes verified, audio decoded, meta and provenance re-validated; a taken name gets a `_2` suffix, an identical voice is a no-op, nothing is overwritten. Voice library toolbar: "Export selected" and "Import bundle" (44px). Tests: `tests/test_voice_bundle.py`, `tests/voice-bundle.spec.js`. See `architecture/voice-bundle.md`.
- **Consent/provenance field per voice.** Each voice gets `provenance` in its `.meta.json` (`basis`: own_voice, consented, licensed, public_domain or unknown; `source`, `licence`, `note`, optional `recorded_consent`, `set_at`). Existing voices count as `unknown`; `quick-import-voice` prefills the source URL (basis stays `unknown`). New `PUT /api/voices/{id}/provenance` (enum, length caps, control characters stripped, http(s) URLs only, no credentials) and `POST`/`GET /api/voices/{id}/consent-recording` (audio only, 10 MB cap, stored as `<voice>.consent`, moves/deletes with the voice). The voice inspector has a "Source & consent" section (typed note or MediaRecorder recording) and voices without a recorded basis show a "Source unknown" badge. Generation is not blocked. Data stays local. Tests: `tests/test_voice_provenance.py`, `tests/voice-provenance.spec.js`. - **Consent/provenance field per voice.** Each voice gets `provenance` in its `.meta.json` (`basis`: own_voice, consented, licensed, public_domain or unknown; `source`, `licence`, `note`, optional `recorded_consent`, `set_at`). Existing voices count as `unknown`; `quick-import-voice` prefills the source URL (basis stays `unknown`). New `PUT /api/voices/{id}/provenance` (enum, length caps, control characters stripped, http(s) URLs only, no credentials) and `POST`/`GET /api/voices/{id}/consent-recording` (audio only, 10 MB cap, stored as `<voice>.consent`, moves/deletes with the voice). The voice inspector has a "Source & consent" section (typed note or MediaRecorder recording) and voices without a recorded basis show a "Source unknown" badge. Generation is not blocked. Data stays local. Tests: `tests/test_voice_provenance.py`, `tests/voice-provenance.spec.js`.

View File

@ -9,7 +9,7 @@ Living manual. Every feature updates this file in the same commit as the code.
--- ---
## Inline tags in the text (optional) ## Inline tags in the text (optional)
Purpose: pauses, spelled-out acronyms and slower/faster passages inside one text. Prerequisite: set `inline_tags_enabled` to true in the settings (default off; when off the tags are spoken literally). Purpose: pauses, spelled-out acronyms and slower/faster passages inside one text. Prerequisite: switch on **Settings -> General -> Inline text tags** (default off; when off the tags are spoken literally). A syntax hint sits under the text boxes in Try It Out and Read Aloud.
| Tag | Result | | Tag | Result |
|---|---| |---|---|
@ -17,7 +17,7 @@ Purpose: pauses, spelled-out acronyms and slower/faster passages inside one text
| `[spell]USA[/spell]` | letters one by one (German letter names for German text) | | `[spell]USA[/spell]` | letters one by one (German letter names for German text) |
| `[slow]..[/slow]` / `[fast]..[/fast]` | 0.85x / 1.15x speed, same pitch; innermost wins; unclosed ends at the paragraph | | `[slow]..[/slow]` / `[fast]..[/fast]` | 0.85x / 1.15x speed, same pitch; innermost wins; unclosed ends at the paragraph |
Known control tags such as `[laugh]` are removed, never read; `[1]` and `[sic]` stay. Works with every engine in the Voice Generator preview/generate path (speed via ffmpeg, so no engine support needed). Not yet in the OpenAI-compatible proxy, streaming, chat or audiobook/magazine audio. Errors: a failing engine reports the usual TTS error. Mobile/accessibility/privacy: no UI change, text stays local. Known control tags such as `[laugh]` are removed, never read; `[1]` and `[sic]` stay. Rendered (pauses, spell, speed) in: Try It Out/generate, Read Aloud and audiobook line previews (both call `/api/tts-preview`), the OpenAI-compatible `/v1/audio/speech` (non-streaming), and `/speak` and the MCP speak tool. Tags are only removed, never spoken, in: streaming playback and streaming `/v1/audio/speech`, spoken chat replies, and magazine MP3 (so recorded sentence timings stay exact). Speed uses ffmpeg, so no engine support is needed. Errors: a failing engine reports the usual TTS error. Mobile/accessibility: the switch is a labelled checkbox with `role=switch` (keyboard Space, 44 px row); German and English labels. Privacy: text stays local.
## Design a Voice: attribute chips ## Design a Voice: attribute chips

View File

@ -18,6 +18,10 @@ Only these names are removed and never spoken (opening or closing, optional shor
`parse(text, language)` -> segments `{text, speed, pause_ms}` (tags never reach a segment, so any sentence chunker run on a segment cannot cut a tag) -> `render()` synthesises per segment, stretches, joins with silence -> one WAV. Duration therefore already includes pauses and stretching. `parse(text, language)` -> segments `{text, speed, pause_ms}` (tags never reach a segment, so any sentence chunker run on a segment cannot cut a tag) -> `render()` synthesises per segment, stretches, joins with silence -> one WAV. Duration therefore already includes pauses and stretching.
## Coverage ## Coverage
Covered: `POST /api/tts-preview` (main generate path; seed-finder calls only get tags stripped via `plain_text`). `ssml_lite.render_with_tags(text, synth, language, enabled)` is the one shared entry (plain `synth(text)` unless enabled and tagged); async routes bridge to it through `routes.tts._generate_with_tags`.
Not covered: `/v1/audio/speech` proxy, streaming playback, conversation/chat, audiobook casting and magazine MP3/read-aloud (timing records `write_audio_timings` assume untagged text; tags there would be spoken literally). Engine speed params are not used: `atempo` works the same for every engine. - Rendered: `POST /api/tts-preview` (normal generate; audiobook line previews and Read Aloud call it from the browser, there is no server-side audiobook synthesis), `POST /v1/audio/speech` non-streaming branches (virtual, voice_design, magpie, nvidia, fish, default), `/speak` and the MCP speak tool. Route sounds and `_prepare_proxy_audio` run after rendering, so durations include pauses.
Rollback: set `inline_tags_enabled` to false (default). - Stripped only (`plain_text`, never spoken): streaming `/v1/audio/speech`, sentence-streamed chat replies (`_make_tts_task`; a tag cut by the sentence splitter before it reaches the task can leak), seed-finder calls.
- Magazine MP3 (`synthesize_article_audio`): tags are stripped from each prepared segment before sentence splitting, never rendered. Reason: `write_audio_timings` records per-sentence spans and character counts; inserted silence or stretched clips would need extra span types. Spans are measured on the audio actually joined, so they stay exact.
- Off (default): text is passed through unchanged, tags are spoken literally.
Engine speed params are not used: `atempo` works the same for every engine.
Rollback: set `inline_tags_enabled` to false (Settings -> General switch).

View File

@ -5753,6 +5753,11 @@ def synthesize_article_audio(doc_dir: Path, article: dict[str, Any], layout: dic
utterance_segment: list[int] = [] # which segment each utterance came from, for the timing record utterance_segment: list[int] = [] # which segment each utterance came from, for the timing record
for n, segment in enumerate(segments): for n, segment in enumerate(segments):
spoken = segment.get("spoken") or prepare_text_for_tts(str(segment["text"]), language, glossary) spoken = segment.get("spoken") or prepare_text_for_tts(str(segment["text"]), language, glossary)
if settings.get("inline_tags_enabled"):
# Tags are stripped here, never rendered: pauses/stretch would shift the recorded sentence timings.
from core.ssml_lite import has_tags, plain_text
if has_tags(spoken):
spoken = plain_text(spoken, language)
segment_voice = interview_voice if segment.get("slot") == 1 else voice segment_voice = interview_voice if segment.get("slot") == 1 else voice
segment["voice"] = segment_voice segment["voice"] = segment_voice
for sentence in split_sentences_for_tts(spoken): for sentence in split_sentences_for_tts(spoken):

View File

@ -165,3 +165,12 @@ def render(text: str, synth: Callable[[str], tuple[bytes, str]], tempo: Callable
out = io.BytesIO() out = io.BytesIO()
combined.export(out, format="wav") combined.export(out, format="wav")
return out.getvalue(), "audio/wav" return out.getvalue(), "audio/wav"
def render_with_tags(text: str, synth: Callable[[str], tuple[bytes, str]], language: str = "auto",
enabled: bool = False) -> tuple[bytes, str]:
"""Shared entry for every path that can render tags: plain `synth(text)` unless enabled and tagged."""
if not (enabled and has_tags(text)):
return synth(text)
from core.audio import _change_tempo
return render(text, synth, _change_tempo, language)

View File

@ -45,6 +45,7 @@ from core.attribution_spans import (
parse_response as _span_parse_response, validate_spans as _span_validate, parse_response as _span_parse_response, validate_spans as _span_validate,
validate_only_ids as _span_validate_only_ids, validate_only_ids as _span_validate_only_ids,
) )
from core import ssml_lite
from core.tts_helpers import _preview_request_audio from core.tts_helpers import _preview_request_audio
from routes.stt import _transcribe_audio, _clean_stt_backend from routes.stt import _transcribe_audio, _clean_stt_backend
@ -5670,7 +5671,9 @@ async def speak(request: Request):
instruct = str(data.get("instruct") or "") instruct = str(data.get("instruct") or "")
try: try:
audio, media_type = await asyncio.to_thread( audio, media_type = await asyncio.to_thread(
_preview_request_audio, text, voice, settings, instruct, backend ssml_lite.render_with_tags, text,
lambda t: _preview_request_audio(t, voice, settings, instruct, backend),
"auto", bool(settings.get("inline_tags_enabled")),
) )
except Exception as e: except Exception as e:
raise HTTPException(502, f"TTS error: {e}") raise HTTPException(502, f"TTS error: {e}")
@ -5813,7 +5816,9 @@ async def _mcp_tool_speak(args: dict, client_id: str) -> dict:
llm_model = settings.get("llm_model") or "" llm_model = settings.get("llm_model") or ""
text = await asyncio.to_thread(_rewrite_with_persona_sync, text, persona, llm_url, llm_model) text = await asyncio.to_thread(_rewrite_with_persona_sync, text, persona, llm_url, llm_model)
audio, _media_type = await asyncio.to_thread( audio, _media_type = await asyncio.to_thread(
_preview_request_audio, text, voice, settings, "", "voice_clone" ssml_lite.render_with_tags, text,
lambda t: _preview_request_audio(t, voice, settings, "", "voice_clone"),
"auto", bool(settings.get("inline_tags_enabled")),
) )
audio_b64 = base64.b64encode(audio).decode() audio_b64 = base64.b64encode(audio).decode()
return { return {
@ -6103,6 +6108,8 @@ def _make_tts_task(
sem: "asyncio.Semaphore | None", sem: "asyncio.Semaphore | None",
emotion: str = "", emotion: str = "",
) -> "asyncio.Task": ) -> "asyncio.Task":
if settings.get("inline_tags_enabled"):
text = ssml_lite.plain_text(text) or text # sentence-streamed chat reply: tags stripped, never spoken
tts_text, instruct = _conv_tts_text_and_instruct(text, emotion, backend) tts_text, instruct = _conv_tts_text_and_instruct(text, emotion, backend)
if sem is None: if sem is None:
return asyncio.create_task( return asyncio.create_task(

View File

@ -87,6 +87,17 @@ async def _bounded_tts_call(fn, *args):
_TTS_GENERATION_SLOTS.release() _TTS_GENERATION_SLOTS.release()
async def _generate_with_tags(text: str, language: str, settings: dict, gen_one):
"""Run `gen_one(text)` (async, one synthesis) through inline-tag rendering when the flag is on."""
if not (settings.get("inline_tags_enabled") and ssml_lite.has_tags(text)):
return await gen_one(text)
loop = asyncio.get_running_loop()
return await asyncio.to_thread(
ssml_lite.render_with_tags, text,
lambda t: asyncio.run_coroutine_threadsafe(gen_one(t), loop).result(), language, True,
)
def _voice_design_job_set(job_id: str, **fields) -> None: def _voice_design_job_set(job_id: str, **fields) -> None:
job = _VOICE_DESIGN_JOBS.get(job_id) job = _VOICE_DESIGN_JOBS.get(job_id)
if job is not None: if job is not None:
@ -834,15 +845,10 @@ async def tts_preview(request: Request):
audio, media_type = await _bounded_tts_call( audio, media_type = await _bounded_tts_call(
_seed_finder_request_audio, text, voice, settings, int(_overrides["seed"]) _seed_finder_request_audio, text, voice, settings, int(_overrides["seed"])
) )
elif settings.get("inline_tags_enabled") and ssml_lite.has_tags(text):
from core.audio import _change_tempo
audio, media_type = await _bounded_tts_call(
ssml_lite.render, text,
lambda t: _preview_request_audio(t, voice, settings, instruct, backend),
_change_tempo, str(data.get("language") or "auto"),
)
else: else:
audio, media_type = await _bounded_tts_call(_preview_request_audio, text, voice, settings, instruct, backend) async def _one(t):
return await _bounded_tts_call(_preview_request_audio, t, voice, settings, instruct, backend)
audio, media_type = await _generate_with_tags(text, str(data.get("language") or "auto"), settings, _one)
except HTTPException: except HTTPException:
raise raise
except Exception as e: except Exception as e:
@ -1145,6 +1151,8 @@ async def openai_speech_proxy(request: Request):
route_has_speed = abs(_route_speed(route) - 1.0) > 0.01 route_has_speed = abs(_route_speed(route) - 1.0) > 0.01
if backend == "streaming" and not virtual and response_format == "wav" and not route_has_sounds and not route_has_speed: if backend == "streaming" and not virtual and response_format == "wav" and not route_has_sounds and not route_has_speed:
if settings.get("inline_tags_enabled"):
text = ssml_lite.plain_text(text, explicit_lang or "auto") or text # streaming: tags stripped, never spoken
try: try:
resp = await _bounded_tts_call(_open_tts_stream_response, text, voice, settings, style_instruction) resp = await _bounded_tts_call(_open_tts_stream_response, text, voice, settings, style_instruction)
except HTTPException: except HTTPException:
@ -1179,7 +1187,7 @@ async def openai_speech_proxy(request: Request):
}, },
) )
try: async def _gen_one(text):
if virtual: if virtual:
_name, preset = virtual _name, preset = virtual
instruct = str(preset.get("description", "")).strip() instruct = str(preset.get("description", "")).strip()
@ -1222,6 +1230,11 @@ async def openai_speech_proxy(request: Request):
audio, media_type = await _bounded_tts_call( audio, media_type = await _bounded_tts_call(
_tts_request_audio, text, voice, settings, style_instruction, _tts_request_audio, text, voice, settings, style_instruction,
) )
return audio, media_type
try:
audio, media_type = await _generate_with_tags(
text, str(data.get("language") or data.get("lang") or "auto"), settings, _gen_one)
except HTTPException: except HTTPException:
raise raise
except Exception as e: except Exception as e:

View File

@ -37,6 +37,10 @@ const I18N_DICT = {
'No candidate could be generated.': 'Es konnte kein Kandidat erzeugt werden.', 'No candidate could be generated.': 'Es konnte kein Kandidat erzeugt werden.',
'Speech-to-text unavailable: scored by level and audio problems only.': 'Sprache-zu-Text nicht verfügbar: Wertung nur nach Pegel und Audioproblemen.', 'Speech-to-text unavailable: scored by level and audio problems only.': 'Sprache-zu-Text nicht verfügbar: Wertung nur nach Pegel und Audioproblemen.',
'Auto-pick failed': 'Automatische Wahl fehlgeschlagen', 'Auto-pick failed': 'Automatische Wahl fehlgeschlagen',
// Inline text tags
'Inline text tags': 'Inline-Texttags',
'Off by default. When on, tags shape the speech in Try It Out, Read Aloud, the speech API and audiobooks; streaming and magazine audio only remove them.': 'Standardmäßig aus. Wenn aktiv, steuern Tags die Sprache in „Ausprobieren“, „Vorlesen“, der Sprach-API und Hörbüchern; Streaming und Magazin-Audio entfernen sie nur.',
'Inline tags (enable in Settings → General):': 'Inline-Tags (aktivieren unter Einstellungen → Allgemein):',
// Brand / sidebar groups // Brand / sidebar groups
'Voice Creator': 'Voice Creator', 'Voice Creator': 'Voice Creator',
'Clone · Design · Deploy': 'Klonen · Entwerfen · Bereitstellen', 'Clone · Design · Deploy': 'Klonen · Entwerfen · Bereitstellen',

View File

@ -579,6 +579,12 @@ async function loadSettings() {
restoreEngineApiKeyInputs(); restoreEngineApiKeyInputs();
const extKeyInp = $('s-external-api-key'); const extKeyInp = $('s-external-api-key');
if (extKeyInp) { extKeyInp.value = ''; extKeyInp.placeholder = s.external_api_key === '__configured__' ? 'Configured — regenerate to replace' : ''; } if (extKeyInp) { extKeyInp.value = ''; extKeyInp.placeholder = s.external_api_key === '__configured__' ? 'Configured — regenerate to replace' : ''; }
const inlineTags = $('s-inline-tags-enabled');
if (inlineTags) inlineTags.checked = !!s.inline_tags_enabled;
if (inlineTags && !inlineTags.dataset.wired) {
inlineTags.dataset.wired = '1';
inlineTags.addEventListener('change', () => _patchSettings({ inline_tags_enabled: inlineTags.checked }));
}
const extKeyReq = $('s-external-api-key-required'); const extKeyReq = $('s-external-api-key-required');
if (extKeyReq) extKeyReq.checked = !!s.external_api_key_required; if (extKeyReq) extKeyReq.checked = !!s.external_api_key_required;
if (extKeyReq && !extKeyReq.dataset.wired) { if (extKeyReq && !extKeyReq.dataset.wired) {

View File

@ -79,6 +79,7 @@
<div><strong>Paste text</strong><span>Plain text, Markdown, chapters, notes, or draft prose.</span></div> <div><strong>Paste text</strong><span>Plain text, Markdown, chapters, notes, or draft prose.</span></div>
</div> </div>
<textarea id="reader-paste-text" class="reader-paste-text" rows="5" placeholder="Paste text here..." aria-label="Paste text to read aloud"></textarea> <textarea id="reader-paste-text" class="reader-paste-text" rows="5" placeholder="Paste text here..." aria-label="Paste text to read aloud"></textarea>
<span class="s-hint tag-syntax-hint"><span>Inline tags (enable in Settings → General):</span> <code>[pause 500ms]</code> <code>[spell]USA[/spell]</code> <code>[slow]…[/slow]</code> <code>[fast]…[/fast]</code></span>
<div class="reader-paste-actions"> <div class="reader-paste-actions">
<button class="btn-primary btn-sm" id="reader-paste-load" type="button"><span class="mdi mdi-text-box-check-outline"></span> Use pasted text</button> <button class="btn-primary btn-sm" id="reader-paste-load" type="button"><span class="mdi mdi-text-box-check-outline"></span> Use pasted text</button>
</div> </div>

View File

@ -34,6 +34,14 @@
<textarea id="s-seed-finder-text" placeholder="Leave empty for language-specific defaults..." style="min-height:60px; font-family:inherit; resize:vertical;"></textarea> <textarea id="s-seed-finder-text" placeholder="Leave empty for language-specific defaults..." style="min-height:60px; font-family:inherit; resize:vertical;"></textarea>
<span class="s-hint">Custom test sentence used when opening the Seed Finder.</span> <span class="s-hint">Custom test sentence used when opening the Seed Finder.</span>
</div> </div>
<label class="s-toggle-row" style="grid-column: 1 / -1">
<input type="checkbox" role="switch" id="s-inline-tags-enabled">
<div>
<span class="s-toggle-title">Inline text tags</span>
<span class="s-hint">Off by default. When on, tags shape the speech in Try It Out, Read Aloud, the speech API and audiobooks; streaming and magazine audio only remove them.</span>
<span class="s-hint"><code>[pause 500ms]</code> <code>[spell]USA[/spell]</code> <code>[slow]…[/slow]</code> <code>[fast]…[/fast]</code></span>
</div>
</label>
</div> </div>
<div class="s-page-actions"> <div class="s-page-actions">
<button class="btn-primary s-save-btn">Save settings</button> <button class="btn-primary s-save-btn">Save settings</button>

View File

@ -61,6 +61,7 @@
<h2><span class="mdi mdi-text-long"></span> Text to synthesize</h2> <h2><span class="mdi mdi-text-long"></span> Text to synthesize</h2>
<div class="field"> <div class="field">
<textarea id="preview-text-area" placeholder="Enter the text you want to synthesize…" style="min-height:120px">Hello! This is a voice preview from TTS Voice Creator - Clone and Design.</textarea> <textarea id="preview-text-area" placeholder="Enter the text you want to synthesize…" style="min-height:120px">Hello! This is a voice preview from TTS Voice Creator - Clone and Design.</textarea>
<span class="s-hint tag-syntax-hint"><span>Inline tags (enable in Settings → General):</span> <code>[pause 500ms]</code> <code>[spell]USA[/spell]</code> <code>[slow]…[/slow]</code> <code>[fast]…[/fast]</code></span>
</div> </div>
<div class="field" style="margin-top:10px"> <div class="field" style="margin-top:10px">
<label style="display:flex;align-items:center;gap:8px"> <label style="display:flex;align-items:center;gap:8px">

44
tests/inline-tags.spec.js Normal file
View File

@ -0,0 +1,44 @@
// @ts-check
// Inline-tags settings switch + syntax hints (all /api routes mocked).
const { test, expect } = require('@playwright/test');
async function boot(page, lang) {
/** @type {any[]} */ const patches = [];
await page.route('**/api/**', route => route.request().method() === 'GET' ? route.fulfill({ json: {} }) : route.fulfill({ json: { ok: true } }));
await page.route('**/api/settings', route => {
if (route.request().method() === 'GET') return route.fulfill({ json: { inline_tags_enabled: false } });
patches.push(route.request().postDataJSON());
return route.fulfill({ json: { ok: true } });
});
await page.route('**/static/dist/main.min.js*', r => r.abort());
if (lang) await page.addInitScript(l => { try { localStorage.setItem('app-lang', l); } catch (e) {} }, lang);
await page.goto('/', { waitUntil: 'domcontentloaded' });
await page.waitForFunction(() => typeof window.navTo === 'function', null, { timeout: 15_000 });
await page.waitForSelector('#app-skeleton', { state: 'detached', timeout: 15_000 });
return patches;
}
test('switch is labelled, keyboard-operable and saves the flag', async ({ page }) => {
const patches = await boot(page);
await page.evaluate(() => window.navSettingsCat('general'));
const sw = page.getByRole('switch', { name: /Inline text tags/ });
await expect(sw).not.toBeChecked();
await sw.focus();
await page.keyboard.press('Space');
await expect(sw).toBeChecked();
await expect.poll(() => patches.some(p => p && p.inline_tags_enabled === true)).toBe(true);
});
test('syntax hint sits under the Try It Out text box', async ({ page }) => {
await boot(page);
await page.evaluate(() => window.navTo('s-tryout'));
await expect(page.locator('#preview-text-area ~ .tag-syntax-hint')).toContainText('[pause 500ms]');
});
test('German labels', async ({ page }) => {
await boot(page, 'de');
await page.evaluate(() => window.navSettingsCat('general'));
await expect(page.getByText('Inline-Texttags')).toBeVisible();
await page.evaluate(() => window.navTo('s-tryout'));
await expect(page.locator('#preview-text-area ~ .tag-syntax-hint')).toContainText('Inline-Tags');
});

View File

@ -0,0 +1,137 @@
"""Inline tags in the generate, /v1/audio/speech, conversation and magazine paths (fake synth, no network)."""
import io
import os
import tempfile
import unittest
from pathlib import Path
from unittest import mock
os.environ.setdefault("CONFIG_DIR", tempfile.mkdtemp(prefix="itp-cfg-")) # routes.* would otherwise mkdir /home/app
from pydub import AudioSegment
from pydub.generators import Sine
from core import ssml_lite as S
SPOKEN: list[str] = []
def fake_synth(text, *_a, **_k):
SPOKEN.append(text)
buf = io.BytesIO()
Sine(440).to_audio_segment(duration=10 * len(text)).export(buf, format="wav")
return buf.getvalue(), "audio/wav"
def wav_ms(data: bytes) -> int:
return len(AudioSegment.from_file(io.BytesIO(data), format="wav"))
class HelperTests(unittest.TestCase):
def setUp(self):
SPOKEN.clear()
def test_flag_off_passes_text_through(self):
S.render_with_tags("A [pause 1s] B", fake_synth, "en", False)
self.assertEqual(SPOKEN, ["A [pause 1s] B"])
def test_flag_on_renders_pause_and_never_speaks_tags(self):
audio, _ = S.render_with_tags("Aaaa [pause 1s] Bbbb [laughs]", fake_synth, "en", True)
self.assertFalse(any("[" in t for t in SPOKEN))
self.assertGreaterEqual(wav_ms(audio), 1000 + 40 + 40 - 5)
class RouteTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
from fastapi import FastAPI
from fastapi.testclient import TestClient
import routes.tts as T
cls.T = T
app = FastAPI()
app.include_router(T.router)
cls.client = TestClient(app)
def setUp(self):
SPOKEN.clear()
def _post(self, url, body, flag):
T = self.T
settings = {"inline_tags_enabled": flag}
with mock.patch.object(T, "_load_settings", return_value=settings), \
mock.patch.object(T, "_preview_request_audio", side_effect=fake_synth), \
mock.patch.object(T, "_tts_request_audio", side_effect=fake_synth), \
mock.patch.object(T, "_resolve_tts_route", side_effect=lambda app, v, t, l: (v, None)), \
mock.patch.object(T, "_route_backend", return_value="customvoice"), \
mock.patch.object(T, "_apply_route_sounds", side_effect=lambda a, m, r, s: (a, m, [])):
return self.client.post(url, json=body)
def test_speech_proxy_flag_on_has_pause_and_no_tag_text(self):
r = self._post("/v1/audio/speech", {"input": "Aaaa [pause 1s] Bbbb", "voice": "v", "response_format": "wav"}, True)
self.assertEqual(r.status_code, 200, r.text)
self.assertFalse(any("[" in t for t in SPOKEN), SPOKEN)
self.assertGreaterEqual(wav_ms(r.content), 1000)
def test_speech_proxy_flag_off_unchanged(self):
r = self._post("/v1/audio/speech", {"input": "Aaaa [pause 1s] Bbbb", "voice": "v", "response_format": "wav"}, False)
self.assertEqual(r.status_code, 200, r.text)
self.assertEqual(SPOKEN, ["Aaaa [pause 1s] Bbbb"])
def test_tts_preview_flag_on(self):
r = self._post("/api/tts-preview", {"text": "Aaaa [pause 500ms] Bbbb", "voice": "v", "backend": "customvoice"}, True)
self.assertEqual(r.status_code, 200, r.text)
self.assertFalse(any("[" in t for t in SPOKEN), SPOKEN)
self.assertGreaterEqual(wav_ms(r.content), 500)
class MagazineTests(unittest.TestCase):
"""Magazine path strips tags (never speaks them) so recorded timings equal the audio actually joined."""
def _render(self, flag):
import core.magazine as M
SPOKEN.clear()
blocks = [{"text": "First line here. [pause 5s] Second line here.", "type": "paragraph"}]
with tempfile.TemporaryDirectory() as d, \
mock.patch.object(M, "article_text", return_value=blocks), \
mock.patch("core.tts_helpers._tts_request_audio", side_effect=fake_synth), \
mock.patch.object(M, "tag_article_mp3"):
doc = Path(d)
try:
out = M.synthesize_article_audio(doc, {"id": "a1"}, {}, "v", {"inline_tags_enabled": flag})
except Exception as e: # mp3 export needs ffmpeg
self.skipTest(f"magazine render unavailable here: {e}")
t = M.read_audio_timings(out)
return t, wav_ms_of(out)
def test_flag_on_strips_tags_and_timings_match_audio(self):
t, total = self._render(True)
self.assertFalse(any("[" in s or "pause" in s for s in SPOKEN), SPOKEN)
self.assertLess(abs(t["duration_ms"] - total), 150) # mp3 frame padding only; no 5 s hole
self.assertLess(t["duration_ms"], 2000)
def test_flag_off_keeps_legacy_text(self):
self._render(False)
self.assertTrue(any("[pause 5s]" in s for s in SPOKEN))
def wav_ms_of(path):
return len(AudioSegment.from_file(path))
class ConversationTests(unittest.TestCase):
def test_chat_reply_chunk_strips_tags_when_on(self):
import asyncio
import routes.conversation as C
SPOKEN.clear()
async def both():
for flag in (True, False):
with mock.patch.object(C, "_preview_request_audio", side_effect=fake_synth):
await C._make_tts_task("Hello [pause 1s] there.", "v", {"inline_tags_enabled": flag}, "customvoice", None)
await asyncio.sleep(0.05)
asyncio.run(both())
self.assertEqual(SPOKEN, ["Hello there.", "Hello [pause 1s] there."])
if __name__ == "__main__":
unittest.main()