feat(tts): inline tags in generate, audiobook and magazine paths
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
5a7c73ea42
commit
9b169037a4
@ -8,7 +8,7 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
|
||||
## [Unreleased]
|
||||
### Added
|
||||
- **Provenance tags on exported audio.** Magazine MP3 downloads, `audio.zip` and the MP3s inside the EPUB get ID3 tags (`synthetic_speech=true`, `generator`, `generator_version`, `generated_date`, comment; voice name only on opt-in) via one ffmpeg stream copy of a temp copy; stored audio is untouched. Per-export checkboxes on the magazine export row ("Mark as synthetic speech", "include voice name"), default from new setting `export_provenance_default` (on). `POST /api/export/provenance` and `python -m core.export_provenance file.mp3` read the tags back. Inaudible AudioSeal watermarking is researched (MIT code and weights, torch needed, weak message recovery after MP3/AAC) but only a stub `watermark(path)` that reports unavailable. Spoken disclosure is not implemented yet. Tests: `tests/test_watermark_tags.py`, `tests/magazine.spec.js`. See `architecture/watermarking.md`.
|
||||
- **Inline text tags for TTS (setting `inline_tags_enabled`, default OFF).** `[pause 500ms]`/`[pause 1.2s]` (max 10 s each, 60 s per text), `[spell]USA[/spell]` (German letter names for German text), `[slow]..[/slow]` / `[fast]..[/fast]` (0.85 / 1.15 via ffmpeg atempo, innermost wins, unclosed ends at paragraph). Known control tags (e.g. `[laugh]`) are stripped, never spoken; `[1]` and `[sic]` stay. Only `POST /api/tts-preview` renders them; other paths are listed in `architecture/inline-tags.md`. Tests: `tests/test_ssml_lite.py`.
|
||||
- **Inline text tags for TTS (setting `inline_tags_enabled`, default OFF).** `[pause 500ms]`/`[pause 1.2s]` (max 10 s each, 60 s per text), `[spell]USA[/spell]` (German letter names for German text), `[slow]..[/slow]` / `[fast]..[/fast]` (0.85 / 1.15 via ffmpeg atempo, innermost wins, unclosed ends at paragraph). Known control tags (e.g. `[laugh]`) are stripped, never spoken; `[1]` and `[sic]` stay. Rendered in `POST /api/tts-preview` (also used by audiobook line previews), `/v1/audio/speech` (non-streaming), `/speak` and the MCP speak tool through one shared `ssml_lite.render_with_tags`; stripped only (never spoken) in streaming, spoken chat replies and magazine MP3 (timings stay exact). Settings -> General switch (accessible, DE/EN) and a syntax hint under the Try It Out and Read Aloud text boxes. Tests: `tests/test_ssml_lite.py`, `tests/test_inline_tags_paths.py`, `tests/inline-tags.spec.js`.
|
||||
- **Optional vocal isolation for URL/YouTube imports (default OFF).** `POST /api/isolate-vocals {id}` runs the Demucs CLI (`htdemucs`, two stems) on a registered temp audio and registers the vocals as a new id; the original stays valid. Needs `VOCAL_ISOLATION=1`, a `demucs` binary (or `VOCAL_ISOLATION_CMD`) and the weights already on disk; the app never downloads anything and nothing is added to the main image. Limits: 15 min input, 15 min runtime (`VOCAL_ISOLATION_MAX_SECONDS`/`_TIMEOUT`), one job at a time, cancelled on client disconnect, temp files always removed. `GET /api/version` reports `features.vocal_isolation`; the "Remove background music" button (clone page URL card and voice library add-URL) appears only then, with an aria-live status and a play original / play without music toggle. Tests: `tests/test_vocal_isolation.py`, `tests/vocal-isolation.spec.js`. See `architecture/vocal-isolation.md`.
|
||||
- **Voice export/import bundles.** `GET /api/voices/{id}/export` streams a `<id>.voice.zip` (audio, `reference.txt`, `meta.json` with seed, design prompt, rating/tags, `provenance`, `audio_problems`; optional picture and consent recording; `manifest.json` with format version, app version and sha256 per file; no temp files). `POST /api/voices/import` validates at the boundary: 150 MB upload cap, fixed flat member names only (zip-slip/path traversal rejected), per-file, total and compression-ratio caps, hashes verified, audio decoded, meta and provenance re-validated; a taken name gets a `_2` suffix, an identical voice is a no-op, nothing is overwritten. Voice library toolbar: "Export selected" and "Import bundle" (44px). Tests: `tests/test_voice_bundle.py`, `tests/voice-bundle.spec.js`. See `architecture/voice-bundle.md`.
|
||||
- **Consent/provenance field per voice.** Each voice gets `provenance` in its `.meta.json` (`basis`: own_voice, consented, licensed, public_domain or unknown; `source`, `licence`, `note`, optional `recorded_consent`, `set_at`). Existing voices count as `unknown`; `quick-import-voice` prefills the source URL (basis stays `unknown`). New `PUT /api/voices/{id}/provenance` (enum, length caps, control characters stripped, http(s) URLs only, no credentials) and `POST`/`GET /api/voices/{id}/consent-recording` (audio only, 10 MB cap, stored as `<voice>.consent`, moves/deletes with the voice). The voice inspector has a "Source & consent" section (typed note or MediaRecorder recording) and voices without a recorded basis show a "Source unknown" badge. Generation is not blocked. Data stays local. Tests: `tests/test_voice_provenance.py`, `tests/voice-provenance.spec.js`.
|
||||
|
||||
@ -9,7 +9,7 @@ Living manual. Every feature updates this file in the same commit as the code.
|
||||
---
|
||||
|
||||
## Inline tags in the text (optional)
|
||||
Purpose: pauses, spelled-out acronyms and slower/faster passages inside one text. Prerequisite: set `inline_tags_enabled` to true in the settings (default off; when off the tags are spoken literally).
|
||||
Purpose: pauses, spelled-out acronyms and slower/faster passages inside one text. Prerequisite: switch on **Settings -> General -> Inline text tags** (default off; when off the tags are spoken literally). A syntax hint sits under the text boxes in Try It Out and Read Aloud.
|
||||
|
||||
| Tag | Result |
|
||||
|---|---|
|
||||
@ -17,7 +17,7 @@ Purpose: pauses, spelled-out acronyms and slower/faster passages inside one text
|
||||
| `[spell]USA[/spell]` | letters one by one (German letter names for German text) |
|
||||
| `[slow]..[/slow]` / `[fast]..[/fast]` | 0.85x / 1.15x speed, same pitch; innermost wins; unclosed ends at the paragraph |
|
||||
|
||||
Known control tags such as `[laugh]` are removed, never read; `[1]` and `[sic]` stay. Works with every engine in the Voice Generator preview/generate path (speed via ffmpeg, so no engine support needed). Not yet in the OpenAI-compatible proxy, streaming, chat or audiobook/magazine audio. Errors: a failing engine reports the usual TTS error. Mobile/accessibility/privacy: no UI change, text stays local.
|
||||
Known control tags such as `[laugh]` are removed, never read; `[1]` and `[sic]` stay. Rendered (pauses, spell, speed) in: Try It Out/generate, Read Aloud and audiobook line previews (both call `/api/tts-preview`), the OpenAI-compatible `/v1/audio/speech` (non-streaming), and `/speak` and the MCP speak tool. Tags are only removed, never spoken, in: streaming playback and streaming `/v1/audio/speech`, spoken chat replies, and magazine MP3 (so recorded sentence timings stay exact). Speed uses ffmpeg, so no engine support is needed. Errors: a failing engine reports the usual TTS error. Mobile/accessibility: the switch is a labelled checkbox with `role=switch` (keyboard Space, 44 px row); German and English labels. Privacy: text stays local.
|
||||
|
||||
## Design a Voice: attribute chips
|
||||
|
||||
|
||||
@ -18,6 +18,10 @@ Only these names are removed and never spoken (opening or closing, optional shor
|
||||
`parse(text, language)` -> segments `{text, speed, pause_ms}` (tags never reach a segment, so any sentence chunker run on a segment cannot cut a tag) -> `render()` synthesises per segment, stretches, joins with silence -> one WAV. Duration therefore already includes pauses and stretching.
|
||||
|
||||
## Coverage
|
||||
Covered: `POST /api/tts-preview` (main generate path; seed-finder calls only get tags stripped via `plain_text`).
|
||||
Not covered: `/v1/audio/speech` proxy, streaming playback, conversation/chat, audiobook casting and magazine MP3/read-aloud (timing records `write_audio_timings` assume untagged text; tags there would be spoken literally). Engine speed params are not used: `atempo` works the same for every engine.
|
||||
Rollback: set `inline_tags_enabled` to false (default).
|
||||
`ssml_lite.render_with_tags(text, synth, language, enabled)` is the one shared entry (plain `synth(text)` unless enabled and tagged); async routes bridge to it through `routes.tts._generate_with_tags`.
|
||||
- Rendered: `POST /api/tts-preview` (normal generate; audiobook line previews and Read Aloud call it from the browser, there is no server-side audiobook synthesis), `POST /v1/audio/speech` non-streaming branches (virtual, voice_design, magpie, nvidia, fish, default), `/speak` and the MCP speak tool. Route sounds and `_prepare_proxy_audio` run after rendering, so durations include pauses.
|
||||
- Stripped only (`plain_text`, never spoken): streaming `/v1/audio/speech`, sentence-streamed chat replies (`_make_tts_task`; a tag cut by the sentence splitter before it reaches the task can leak), seed-finder calls.
|
||||
- Magazine MP3 (`synthesize_article_audio`): tags are stripped from each prepared segment before sentence splitting, never rendered. Reason: `write_audio_timings` records per-sentence spans and character counts; inserted silence or stretched clips would need extra span types. Spans are measured on the audio actually joined, so they stay exact.
|
||||
- Off (default): text is passed through unchanged, tags are spoken literally.
|
||||
Engine speed params are not used: `atempo` works the same for every engine.
|
||||
Rollback: set `inline_tags_enabled` to false (Settings -> General switch).
|
||||
|
||||
@ -5753,6 +5753,11 @@ def synthesize_article_audio(doc_dir: Path, article: dict[str, Any], layout: dic
|
||||
utterance_segment: list[int] = [] # which segment each utterance came from, for the timing record
|
||||
for n, segment in enumerate(segments):
|
||||
spoken = segment.get("spoken") or prepare_text_for_tts(str(segment["text"]), language, glossary)
|
||||
if settings.get("inline_tags_enabled"):
|
||||
# Tags are stripped here, never rendered: pauses/stretch would shift the recorded sentence timings.
|
||||
from core.ssml_lite import has_tags, plain_text
|
||||
if has_tags(spoken):
|
||||
spoken = plain_text(spoken, language)
|
||||
segment_voice = interview_voice if segment.get("slot") == 1 else voice
|
||||
segment["voice"] = segment_voice
|
||||
for sentence in split_sentences_for_tts(spoken):
|
||||
|
||||
@ -165,3 +165,12 @@ def render(text: str, synth: Callable[[str], tuple[bytes, str]], tempo: Callable
|
||||
out = io.BytesIO()
|
||||
combined.export(out, format="wav")
|
||||
return out.getvalue(), "audio/wav"
|
||||
|
||||
|
||||
def render_with_tags(text: str, synth: Callable[[str], tuple[bytes, str]], language: str = "auto",
|
||||
enabled: bool = False) -> tuple[bytes, str]:
|
||||
"""Shared entry for every path that can render tags: plain `synth(text)` unless enabled and tagged."""
|
||||
if not (enabled and has_tags(text)):
|
||||
return synth(text)
|
||||
from core.audio import _change_tempo
|
||||
return render(text, synth, _change_tempo, language)
|
||||
|
||||
@ -45,6 +45,7 @@ from core.attribution_spans import (
|
||||
parse_response as _span_parse_response, validate_spans as _span_validate,
|
||||
validate_only_ids as _span_validate_only_ids,
|
||||
)
|
||||
from core import ssml_lite
|
||||
from core.tts_helpers import _preview_request_audio
|
||||
from routes.stt import _transcribe_audio, _clean_stt_backend
|
||||
|
||||
@ -5670,7 +5671,9 @@ async def speak(request: Request):
|
||||
instruct = str(data.get("instruct") or "")
|
||||
try:
|
||||
audio, media_type = await asyncio.to_thread(
|
||||
_preview_request_audio, text, voice, settings, instruct, backend
|
||||
ssml_lite.render_with_tags, text,
|
||||
lambda t: _preview_request_audio(t, voice, settings, instruct, backend),
|
||||
"auto", bool(settings.get("inline_tags_enabled")),
|
||||
)
|
||||
except Exception as e:
|
||||
raise HTTPException(502, f"TTS error: {e}")
|
||||
@ -5813,7 +5816,9 @@ async def _mcp_tool_speak(args: dict, client_id: str) -> dict:
|
||||
llm_model = settings.get("llm_model") or ""
|
||||
text = await asyncio.to_thread(_rewrite_with_persona_sync, text, persona, llm_url, llm_model)
|
||||
audio, _media_type = await asyncio.to_thread(
|
||||
_preview_request_audio, text, voice, settings, "", "voice_clone"
|
||||
ssml_lite.render_with_tags, text,
|
||||
lambda t: _preview_request_audio(t, voice, settings, "", "voice_clone"),
|
||||
"auto", bool(settings.get("inline_tags_enabled")),
|
||||
)
|
||||
audio_b64 = base64.b64encode(audio).decode()
|
||||
return {
|
||||
@ -6103,6 +6108,8 @@ def _make_tts_task(
|
||||
sem: "asyncio.Semaphore | None",
|
||||
emotion: str = "",
|
||||
) -> "asyncio.Task":
|
||||
if settings.get("inline_tags_enabled"):
|
||||
text = ssml_lite.plain_text(text) or text # sentence-streamed chat reply: tags stripped, never spoken
|
||||
tts_text, instruct = _conv_tts_text_and_instruct(text, emotion, backend)
|
||||
if sem is None:
|
||||
return asyncio.create_task(
|
||||
|
||||
@ -87,6 +87,17 @@ async def _bounded_tts_call(fn, *args):
|
||||
_TTS_GENERATION_SLOTS.release()
|
||||
|
||||
|
||||
async def _generate_with_tags(text: str, language: str, settings: dict, gen_one):
|
||||
"""Run `gen_one(text)` (async, one synthesis) through inline-tag rendering when the flag is on."""
|
||||
if not (settings.get("inline_tags_enabled") and ssml_lite.has_tags(text)):
|
||||
return await gen_one(text)
|
||||
loop = asyncio.get_running_loop()
|
||||
return await asyncio.to_thread(
|
||||
ssml_lite.render_with_tags, text,
|
||||
lambda t: asyncio.run_coroutine_threadsafe(gen_one(t), loop).result(), language, True,
|
||||
)
|
||||
|
||||
|
||||
def _voice_design_job_set(job_id: str, **fields) -> None:
|
||||
job = _VOICE_DESIGN_JOBS.get(job_id)
|
||||
if job is not None:
|
||||
@ -834,15 +845,10 @@ async def tts_preview(request: Request):
|
||||
audio, media_type = await _bounded_tts_call(
|
||||
_seed_finder_request_audio, text, voice, settings, int(_overrides["seed"])
|
||||
)
|
||||
elif settings.get("inline_tags_enabled") and ssml_lite.has_tags(text):
|
||||
from core.audio import _change_tempo
|
||||
audio, media_type = await _bounded_tts_call(
|
||||
ssml_lite.render, text,
|
||||
lambda t: _preview_request_audio(t, voice, settings, instruct, backend),
|
||||
_change_tempo, str(data.get("language") or "auto"),
|
||||
)
|
||||
else:
|
||||
audio, media_type = await _bounded_tts_call(_preview_request_audio, text, voice, settings, instruct, backend)
|
||||
async def _one(t):
|
||||
return await _bounded_tts_call(_preview_request_audio, t, voice, settings, instruct, backend)
|
||||
audio, media_type = await _generate_with_tags(text, str(data.get("language") or "auto"), settings, _one)
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
@ -1145,6 +1151,8 @@ async def openai_speech_proxy(request: Request):
|
||||
route_has_speed = abs(_route_speed(route) - 1.0) > 0.01
|
||||
|
||||
if backend == "streaming" and not virtual and response_format == "wav" and not route_has_sounds and not route_has_speed:
|
||||
if settings.get("inline_tags_enabled"):
|
||||
text = ssml_lite.plain_text(text, explicit_lang or "auto") or text # streaming: tags stripped, never spoken
|
||||
try:
|
||||
resp = await _bounded_tts_call(_open_tts_stream_response, text, voice, settings, style_instruction)
|
||||
except HTTPException:
|
||||
@ -1179,7 +1187,7 @@ async def openai_speech_proxy(request: Request):
|
||||
},
|
||||
)
|
||||
|
||||
try:
|
||||
async def _gen_one(text):
|
||||
if virtual:
|
||||
_name, preset = virtual
|
||||
instruct = str(preset.get("description", "")).strip()
|
||||
@ -1222,6 +1230,11 @@ async def openai_speech_proxy(request: Request):
|
||||
audio, media_type = await _bounded_tts_call(
|
||||
_tts_request_audio, text, voice, settings, style_instruction,
|
||||
)
|
||||
return audio, media_type
|
||||
|
||||
try:
|
||||
audio, media_type = await _generate_with_tags(
|
||||
text, str(data.get("language") or data.get("lang") or "auto"), settings, _gen_one)
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
|
||||
@ -37,6 +37,10 @@ const I18N_DICT = {
|
||||
'No candidate could be generated.': 'Es konnte kein Kandidat erzeugt werden.',
|
||||
'Speech-to-text unavailable: scored by level and audio problems only.': 'Sprache-zu-Text nicht verfügbar: Wertung nur nach Pegel und Audioproblemen.',
|
||||
'Auto-pick failed': 'Automatische Wahl fehlgeschlagen',
|
||||
// Inline text tags
|
||||
'Inline text tags': 'Inline-Texttags',
|
||||
'Off by default. When on, tags shape the speech in Try It Out, Read Aloud, the speech API and audiobooks; streaming and magazine audio only remove them.': 'Standardmäßig aus. Wenn aktiv, steuern Tags die Sprache in „Ausprobieren“, „Vorlesen“, der Sprach-API und Hörbüchern; Streaming und Magazin-Audio entfernen sie nur.',
|
||||
'Inline tags (enable in Settings → General):': 'Inline-Tags (aktivieren unter Einstellungen → Allgemein):',
|
||||
// Brand / sidebar groups
|
||||
'Voice Creator': 'Voice Creator',
|
||||
'Clone · Design · Deploy': 'Klonen · Entwerfen · Bereitstellen',
|
||||
|
||||
@ -579,6 +579,12 @@ async function loadSettings() {
|
||||
restoreEngineApiKeyInputs();
|
||||
const extKeyInp = $('s-external-api-key');
|
||||
if (extKeyInp) { extKeyInp.value = ''; extKeyInp.placeholder = s.external_api_key === '__configured__' ? 'Configured — regenerate to replace' : ''; }
|
||||
const inlineTags = $('s-inline-tags-enabled');
|
||||
if (inlineTags) inlineTags.checked = !!s.inline_tags_enabled;
|
||||
if (inlineTags && !inlineTags.dataset.wired) {
|
||||
inlineTags.dataset.wired = '1';
|
||||
inlineTags.addEventListener('change', () => _patchSettings({ inline_tags_enabled: inlineTags.checked }));
|
||||
}
|
||||
const extKeyReq = $('s-external-api-key-required');
|
||||
if (extKeyReq) extKeyReq.checked = !!s.external_api_key_required;
|
||||
if (extKeyReq && !extKeyReq.dataset.wired) {
|
||||
|
||||
@ -79,6 +79,7 @@
|
||||
<div><strong>Paste text</strong><span>Plain text, Markdown, chapters, notes, or draft prose.</span></div>
|
||||
</div>
|
||||
<textarea id="reader-paste-text" class="reader-paste-text" rows="5" placeholder="Paste text here..." aria-label="Paste text to read aloud"></textarea>
|
||||
<span class="s-hint tag-syntax-hint"><span>Inline tags (enable in Settings → General):</span> <code>[pause 500ms]</code> <code>[spell]USA[/spell]</code> <code>[slow]…[/slow]</code> <code>[fast]…[/fast]</code></span>
|
||||
<div class="reader-paste-actions">
|
||||
<button class="btn-primary btn-sm" id="reader-paste-load" type="button"><span class="mdi mdi-text-box-check-outline"></span> Use pasted text</button>
|
||||
</div>
|
||||
|
||||
@ -34,6 +34,14 @@
|
||||
<textarea id="s-seed-finder-text" placeholder="Leave empty for language-specific defaults..." style="min-height:60px; font-family:inherit; resize:vertical;"></textarea>
|
||||
<span class="s-hint">Custom test sentence used when opening the Seed Finder.</span>
|
||||
</div>
|
||||
<label class="s-toggle-row" style="grid-column: 1 / -1">
|
||||
<input type="checkbox" role="switch" id="s-inline-tags-enabled">
|
||||
<div>
|
||||
<span class="s-toggle-title">Inline text tags</span>
|
||||
<span class="s-hint">Off by default. When on, tags shape the speech in Try It Out, Read Aloud, the speech API and audiobooks; streaming and magazine audio only remove them.</span>
|
||||
<span class="s-hint"><code>[pause 500ms]</code> <code>[spell]USA[/spell]</code> <code>[slow]…[/slow]</code> <code>[fast]…[/fast]</code></span>
|
||||
</div>
|
||||
</label>
|
||||
</div>
|
||||
<div class="s-page-actions">
|
||||
<button class="btn-primary s-save-btn">Save settings</button>
|
||||
|
||||
@ -61,6 +61,7 @@
|
||||
<h2><span class="mdi mdi-text-long"></span> Text to synthesize</h2>
|
||||
<div class="field">
|
||||
<textarea id="preview-text-area" placeholder="Enter the text you want to synthesize…" style="min-height:120px">Hello! This is a voice preview from TTS Voice Creator - Clone and Design.</textarea>
|
||||
<span class="s-hint tag-syntax-hint"><span>Inline tags (enable in Settings → General):</span> <code>[pause 500ms]</code> <code>[spell]USA[/spell]</code> <code>[slow]…[/slow]</code> <code>[fast]…[/fast]</code></span>
|
||||
</div>
|
||||
<div class="field" style="margin-top:10px">
|
||||
<label style="display:flex;align-items:center;gap:8px">
|
||||
|
||||
44
tests/inline-tags.spec.js
Normal file
44
tests/inline-tags.spec.js
Normal file
@ -0,0 +1,44 @@
|
||||
// @ts-check
|
||||
// Inline-tags settings switch + syntax hints (all /api routes mocked).
|
||||
const { test, expect } = require('@playwright/test');
|
||||
|
||||
async function boot(page, lang) {
|
||||
/** @type {any[]} */ const patches = [];
|
||||
await page.route('**/api/**', route => route.request().method() === 'GET' ? route.fulfill({ json: {} }) : route.fulfill({ json: { ok: true } }));
|
||||
await page.route('**/api/settings', route => {
|
||||
if (route.request().method() === 'GET') return route.fulfill({ json: { inline_tags_enabled: false } });
|
||||
patches.push(route.request().postDataJSON());
|
||||
return route.fulfill({ json: { ok: true } });
|
||||
});
|
||||
await page.route('**/static/dist/main.min.js*', r => r.abort());
|
||||
if (lang) await page.addInitScript(l => { try { localStorage.setItem('app-lang', l); } catch (e) {} }, lang);
|
||||
await page.goto('/', { waitUntil: 'domcontentloaded' });
|
||||
await page.waitForFunction(() => typeof window.navTo === 'function', null, { timeout: 15_000 });
|
||||
await page.waitForSelector('#app-skeleton', { state: 'detached', timeout: 15_000 });
|
||||
return patches;
|
||||
}
|
||||
|
||||
test('switch is labelled, keyboard-operable and saves the flag', async ({ page }) => {
|
||||
const patches = await boot(page);
|
||||
await page.evaluate(() => window.navSettingsCat('general'));
|
||||
const sw = page.getByRole('switch', { name: /Inline text tags/ });
|
||||
await expect(sw).not.toBeChecked();
|
||||
await sw.focus();
|
||||
await page.keyboard.press('Space');
|
||||
await expect(sw).toBeChecked();
|
||||
await expect.poll(() => patches.some(p => p && p.inline_tags_enabled === true)).toBe(true);
|
||||
});
|
||||
|
||||
test('syntax hint sits under the Try It Out text box', async ({ page }) => {
|
||||
await boot(page);
|
||||
await page.evaluate(() => window.navTo('s-tryout'));
|
||||
await expect(page.locator('#preview-text-area ~ .tag-syntax-hint')).toContainText('[pause 500ms]');
|
||||
});
|
||||
|
||||
test('German labels', async ({ page }) => {
|
||||
await boot(page, 'de');
|
||||
await page.evaluate(() => window.navSettingsCat('general'));
|
||||
await expect(page.getByText('Inline-Texttags')).toBeVisible();
|
||||
await page.evaluate(() => window.navTo('s-tryout'));
|
||||
await expect(page.locator('#preview-text-area ~ .tag-syntax-hint')).toContainText('Inline-Tags');
|
||||
});
|
||||
137
tests/test_inline_tags_paths.py
Normal file
137
tests/test_inline_tags_paths.py
Normal file
@ -0,0 +1,137 @@
|
||||
"""Inline tags in the generate, /v1/audio/speech, conversation and magazine paths (fake synth, no network)."""
|
||||
import io
|
||||
import os
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
os.environ.setdefault("CONFIG_DIR", tempfile.mkdtemp(prefix="itp-cfg-")) # routes.* would otherwise mkdir /home/app
|
||||
|
||||
from pydub import AudioSegment
|
||||
from pydub.generators import Sine
|
||||
|
||||
from core import ssml_lite as S
|
||||
|
||||
SPOKEN: list[str] = []
|
||||
|
||||
|
||||
def fake_synth(text, *_a, **_k):
|
||||
SPOKEN.append(text)
|
||||
buf = io.BytesIO()
|
||||
Sine(440).to_audio_segment(duration=10 * len(text)).export(buf, format="wav")
|
||||
return buf.getvalue(), "audio/wav"
|
||||
|
||||
|
||||
def wav_ms(data: bytes) -> int:
|
||||
return len(AudioSegment.from_file(io.BytesIO(data), format="wav"))
|
||||
|
||||
|
||||
class HelperTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
SPOKEN.clear()
|
||||
|
||||
def test_flag_off_passes_text_through(self):
|
||||
S.render_with_tags("A [pause 1s] B", fake_synth, "en", False)
|
||||
self.assertEqual(SPOKEN, ["A [pause 1s] B"])
|
||||
|
||||
def test_flag_on_renders_pause_and_never_speaks_tags(self):
|
||||
audio, _ = S.render_with_tags("Aaaa [pause 1s] Bbbb [laughs]", fake_synth, "en", True)
|
||||
self.assertFalse(any("[" in t for t in SPOKEN))
|
||||
self.assertGreaterEqual(wav_ms(audio), 1000 + 40 + 40 - 5)
|
||||
|
||||
|
||||
class RouteTests(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
from fastapi import FastAPI
|
||||
from fastapi.testclient import TestClient
|
||||
import routes.tts as T
|
||||
cls.T = T
|
||||
app = FastAPI()
|
||||
app.include_router(T.router)
|
||||
cls.client = TestClient(app)
|
||||
|
||||
def setUp(self):
|
||||
SPOKEN.clear()
|
||||
|
||||
def _post(self, url, body, flag):
|
||||
T = self.T
|
||||
settings = {"inline_tags_enabled": flag}
|
||||
with mock.patch.object(T, "_load_settings", return_value=settings), \
|
||||
mock.patch.object(T, "_preview_request_audio", side_effect=fake_synth), \
|
||||
mock.patch.object(T, "_tts_request_audio", side_effect=fake_synth), \
|
||||
mock.patch.object(T, "_resolve_tts_route", side_effect=lambda app, v, t, l: (v, None)), \
|
||||
mock.patch.object(T, "_route_backend", return_value="customvoice"), \
|
||||
mock.patch.object(T, "_apply_route_sounds", side_effect=lambda a, m, r, s: (a, m, [])):
|
||||
return self.client.post(url, json=body)
|
||||
|
||||
def test_speech_proxy_flag_on_has_pause_and_no_tag_text(self):
|
||||
r = self._post("/v1/audio/speech", {"input": "Aaaa [pause 1s] Bbbb", "voice": "v", "response_format": "wav"}, True)
|
||||
self.assertEqual(r.status_code, 200, r.text)
|
||||
self.assertFalse(any("[" in t for t in SPOKEN), SPOKEN)
|
||||
self.assertGreaterEqual(wav_ms(r.content), 1000)
|
||||
|
||||
def test_speech_proxy_flag_off_unchanged(self):
|
||||
r = self._post("/v1/audio/speech", {"input": "Aaaa [pause 1s] Bbbb", "voice": "v", "response_format": "wav"}, False)
|
||||
self.assertEqual(r.status_code, 200, r.text)
|
||||
self.assertEqual(SPOKEN, ["Aaaa [pause 1s] Bbbb"])
|
||||
|
||||
def test_tts_preview_flag_on(self):
|
||||
r = self._post("/api/tts-preview", {"text": "Aaaa [pause 500ms] Bbbb", "voice": "v", "backend": "customvoice"}, True)
|
||||
self.assertEqual(r.status_code, 200, r.text)
|
||||
self.assertFalse(any("[" in t for t in SPOKEN), SPOKEN)
|
||||
self.assertGreaterEqual(wav_ms(r.content), 500)
|
||||
|
||||
|
||||
class MagazineTests(unittest.TestCase):
|
||||
"""Magazine path strips tags (never speaks them) so recorded timings equal the audio actually joined."""
|
||||
|
||||
def _render(self, flag):
|
||||
import core.magazine as M
|
||||
SPOKEN.clear()
|
||||
blocks = [{"text": "First line here. [pause 5s] Second line here.", "type": "paragraph"}]
|
||||
with tempfile.TemporaryDirectory() as d, \
|
||||
mock.patch.object(M, "article_text", return_value=blocks), \
|
||||
mock.patch("core.tts_helpers._tts_request_audio", side_effect=fake_synth), \
|
||||
mock.patch.object(M, "tag_article_mp3"):
|
||||
doc = Path(d)
|
||||
try:
|
||||
out = M.synthesize_article_audio(doc, {"id": "a1"}, {}, "v", {"inline_tags_enabled": flag})
|
||||
except Exception as e: # mp3 export needs ffmpeg
|
||||
self.skipTest(f"magazine render unavailable here: {e}")
|
||||
t = M.read_audio_timings(out)
|
||||
return t, wav_ms_of(out)
|
||||
|
||||
def test_flag_on_strips_tags_and_timings_match_audio(self):
|
||||
t, total = self._render(True)
|
||||
self.assertFalse(any("[" in s or "pause" in s for s in SPOKEN), SPOKEN)
|
||||
self.assertLess(abs(t["duration_ms"] - total), 150) # mp3 frame padding only; no 5 s hole
|
||||
self.assertLess(t["duration_ms"], 2000)
|
||||
|
||||
def test_flag_off_keeps_legacy_text(self):
|
||||
self._render(False)
|
||||
self.assertTrue(any("[pause 5s]" in s for s in SPOKEN))
|
||||
|
||||
|
||||
def wav_ms_of(path):
|
||||
return len(AudioSegment.from_file(path))
|
||||
|
||||
|
||||
class ConversationTests(unittest.TestCase):
|
||||
def test_chat_reply_chunk_strips_tags_when_on(self):
|
||||
import asyncio
|
||||
import routes.conversation as C
|
||||
SPOKEN.clear()
|
||||
|
||||
async def both():
|
||||
for flag in (True, False):
|
||||
with mock.patch.object(C, "_preview_request_audio", side_effect=fake_synth):
|
||||
await C._make_tts_task("Hello [pause 1s] there.", "v", {"inline_tags_enabled": flag}, "customvoice", None)
|
||||
await asyncio.sleep(0.05)
|
||||
asyncio.run(both())
|
||||
self.assertEqual(SPOKEN, ["Hello there.", "Hello [pause 1s] there."])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Loading…
Reference in New Issue
Block a user