feat: add seed samples API endpoints

POST /voice-seed — writes or clears the seed for a named voice in
voices.json directly from the server, enabling the voice creator GUI
to save the chosen seed with one click.

Also updates find_best_seed.py to support --all-voices, per-voice
subdirectories, and the mixed DE/EN test sentence.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

.gitignore

@@ -21,6 +21,9 @@ config/voices.json*
 # Converted private audio (M4A → WAV, generated at runtime)
 config/converted/
 
+# Seed finder pre-generated WAV samples (batch output, not source files)
+config/seed_samples/
+
 # Editor backup files
 *.py~
 *.yml~

README.md

@@ -136,8 +136,11 @@ Or use the auto-transcription script with a running Whisper-compatible ASR servi
 python config/auto_transcribe.py --api-url http://localhost:8010/v1/audio/transcriptions
 ```
 
-`config/generate_voices.py` runs on container startup and creates `config/voices.json` from your speaker files.
+`config/generate_voices.py` runs automatically in the background, continuously watching your `speakers` directory. Whenever you add a new `.wav` and `.txt` file, it instantly updates `config/voices.json`. The API server hot-reloads the changes, meaning **you never need to restart the container when adding new voices!**
 
+When you start using a new voice for the first time, the server will automatically do the heavy lifting to extract the voice's acoustic fingerprint (a "speaker embedding") and save it as a `.pt` file in the `config/speakers/` directory. Even better, when the server starts up, it automatically precomputes missing `.pt` files in the background, so your first API requests will be lightning fast. Future requests will instantly load this `.pt` file instead of re-analyzing the audio, which dramatically speeds up Time To First Audio (TTFA).
+
+> **Note:** The generation of the `.pt` embedding is completely deterministic. Running the extraction process twice on the same reference `.wav` and `.txt` will yield the exact same fingerprint, so the resulting voice will not vary between regenerations.
 ## VoiceDesign voices
 
 VoiceDesign does not need reference audio. Define reusable voice personalities in `config/voicedesign_voices.json`:
@@ -340,6 +343,31 @@ The first request after container startup can be slower because CUDA graph captu
 
 ## Changelog
 
+### v6.6 — 2026-06-21
+**Feature: Eager Background Precomputation of Speaker Embeddings**
+- The server now automatically precomputes all missing `.pt` files in the background immediately after startup.
+- This ensures all configured voices are pre-warmed and ready to deliver lightning-fast TTFA on the very first request without delaying server startup.
+- The lazy-loading mechanism still remains active to instantly handle any new voices hot-reloaded while the server is running.
+
+### v6.5.1 — 2026-06-21
+**Documentation Update**
+- Added documentation explicitly clarifying that `.pt` speaker embedding generation is fully deterministic and does not produce variable voice characteristics across restarts.
+
+### v6.5 — 2026-06-20
+**Feature: True Zero-Downtime Voice Hot-Reloading**
+
+- The server now watches `voices.json` and hot-reloads it automatically when changes are detected.
+- Added a background loop in `docker-compose.yml` that continuously runs `generate_voices.py` every 10 seconds.
+- You can now drop new `.wav` and `.txt` files into your `speakers/` directory and they will be instantly available via the API without ever restarting the Docker container!
+
+### v6.4 — 2026-06-20
+**Feature: Fully Automated Speaker Embeddings (.pt files)**
+
+- Integrated speaker embedding extraction directly into the API server (`openai_server.py`). 
+- When a new voice is requested for the first time, the server will automatically compute the speaker embedding and save it as a `.pt` file in `/config/speakers/`.
+- Future requests for the same voice automatically use the `.pt` file instead of recalculating the prompt from the `.wav` or `.mp3` reference audio.
+- This provides the massive TTFA speedup of precomputed embeddings without requiring any manual scripting or configuration.
+
 ### v6.3 — 2026-06-20
 **Feature: Precomputed Speaker Embeddings (.pt files)**
 

config/find_best_seed.py

@@ -0,0 +1,198 @@
+#!/usr/bin/env python3
+"""
+Find the best RNG seed for one or all voices by generating audio samples with
+different seeds and saving them as numbered WAV files for comparison.
+
+How it works:
+  1. For each voice × seed, temporarily sets that seed in voices.json
+  2. Calls the running server (hot-reload picks it up automatically)
+  3. Saves audio as <out-dir>/<voice-name>/seed_<N>.wav
+  4. Restores voices.json to its original state when done
+
+Then listen to the files and pick the seed you prefer.
+Add it to voices.json: "seed": <number>
+
+Usage:
+    # All voices, seeds 1–15:
+    python find_best_seed.py --all-voices --range 1 15 --port 8020
+
+    # Single voice:
+    python find_best_seed.py --voice EN_F_NatashaNeural --range 1 20
+    python find_best_seed.py --voice DE_5_28 --seeds 1 7 42 100
+"""
+import argparse
+import json
+import os
+import shutil
+import sys
+import time
+
+import requests
+
+VOICES_JSON = "/config/voices.json"
+
+DE_TEXT = (
+    "Die 3.500 neuen High-End Geräte für das Server-Update benötigen eine "
+    "außergewöhnlich starke Kühlung und regelmäßige Maßnahmen, um die Performance "
+    "bei großer Last zu gewährleisten."
+)
+EN_TEXT = (
+    "The system administrator successfully configured the customized Docker stacks "
+    "and benchmarked the inference engines at exactly 8:45 AM."
+)
+MIXED_TEXT = DE_TEXT + " - " + EN_TEXT
+
+
+def load_voices() -> dict:
+    with open(VOICES_JSON, "r", encoding="utf-8") as f:
+        return json.load(f)
+
+
+def save_voices(voices: dict) -> None:
+    with open(VOICES_JSON, "w", encoding="utf-8") as f:
+        json.dump(voices, f, indent=2, ensure_ascii=False)
+
+
+def generate(voice: str, text: str, host: str, port: int, timeout: int = 90) -> bytes:
+    url = f"http://{host}:{port}/v1/audio/speech"
+    resp = requests.post(
+        url,
+        json={"model": "tts-1", "input": text, "voice": voice, "response_format": "wav"},
+        timeout=timeout,
+        stream=False,
+    )
+    resp.raise_for_status()
+    return resp.content
+
+
+def pick_text(voice_name: str, override: str | None) -> str:
+    if override:
+        return override
+    name_lower = voice_name.lower()
+    if name_lower.startswith("de_"):
+        return MIXED_TEXT
+    if name_lower.startswith("en_") or name_lower.startswith("gb_"):
+        return EN_TEXT
+    return MIXED_TEXT
+
+
+def run_voice(voice_name: str, voices: dict, seeds: list[int],
+              text: str, host: str, port: int, out_dir: str) -> list[tuple]:
+    voice_dir = os.path.join(out_dir, voice_name)
+    os.makedirs(voice_dir, exist_ok=True)
+
+    results = []
+    for seed in seeds:
+        voices[voice_name]["seed"] = seed
+        save_voices(voices)
+        time.sleep(0.3)  # let hot-reload detect mtime change
+
+        out_path = os.path.join(voice_dir, f"seed_{seed:05d}.wav")
+        print(f"    seed {seed:5d} → ", end="", flush=True)
+        try:
+            t0 = time.time()
+            wav = generate(voice_name, text, host, port)
+            elapsed = time.time() - t0
+            with open(out_path, "wb") as f:
+                f.write(wav)
+            print(f"OK  ({elapsed:.1f}s)")
+            results.append((seed, out_path, None))
+        except Exception as e:
+            print(f"FAILED: {e}")
+            results.append((seed, None, str(e)))
+
+    # Remove the temporary seed so the voice returns to its auto-seed
+    voices[voice_name].pop("seed", None)
+    save_voices(voices)
+
+    return results
+
+
+def main():
+    p = argparse.ArgumentParser(description="Compare seeds for one or all voices")
+    target = p.add_mutually_exclusive_group(required=True)
+    target.add_argument("--voice", help="Single voice name")
+    target.add_argument("--all-voices", action="store_true", help="Run for every voice in voices.json")
+
+    p.add_argument("--seeds", type=int, nargs="+", help="Explicit list of seeds to try")
+    p.add_argument("--range", type=int, nargs=2, metavar=("START", "END"),
+                   help="Try seeds START through END (inclusive). Default: 1–15")
+    p.add_argument("--text", default=None,
+                   help="Override text (default: auto-picks DE/EN/mixed based on voice name)")
+    p.add_argument("--host", default="localhost")
+    p.add_argument("--port", type=int, default=8020)
+    p.add_argument("--out-dir", default="./seed_samples",
+                   help="Root output directory (voice subdirs created inside)")
+    args = p.parse_args()
+
+    seeds = list(args.seeds or [])
+    if args.range:
+        seeds += list(range(args.range[0], args.range[1] + 1))
+    if not seeds:
+        seeds = list(range(1, 16))  # default 1–15
+    seeds = sorted(set(seeds))
+
+    voices = load_voices()
+
+    if args.voice:
+        voice_names = [args.voice]
+        if args.voice not in voices:
+            print(f"ERROR: voice {args.voice!r} not in voices.json", file=sys.stderr)
+            sys.exit(1)
+    else:
+        voice_names = list(voices.keys())
+
+    os.makedirs(args.out_dir, exist_ok=True)
+
+    # Single backup at the start; we restore on every error/exit
+    backup = VOICES_JSON + ".seed_backup"
+    shutil.copy2(VOICES_JSON, backup)
+
+    print(f"voices : {len(voice_names)}")
+    print(f"seeds  : {seeds}")
+    print(f"output : {os.path.abspath(args.out_dir)}/")
+    print(f"total  : ~{len(voice_names) * len(seeds)} requests\n")
+
+    summary: dict[str, list] = {}
+
+    try:
+        for i, voice_name in enumerate(voice_names, 1):
+            text = pick_text(voice_name, args.text)
+            print(f"[{i}/{len(voice_names)}] {voice_name}")
+            print(f"  text: {text[:90]}{'...' if len(text) > 90 else ''}")
+            results = run_voice(voice_name, voices, seeds, text, args.host, args.port, args.out_dir)
+            summary[voice_name] = results
+            ok = sum(1 for _, p, _ in results if p)
+            print(f"  → {ok}/{len(seeds)} OK\n")
+
+    except KeyboardInterrupt:
+        print("\n\nInterrupted — restoring voices.json...")
+    finally:
+        shutil.copy2(backup, VOICES_JSON)
+        os.remove(backup)
+        print("voices.json restored.")
+
+    # Write summary
+    summary_path = os.path.join(args.out_dir, "summary.txt")
+    with open(summary_path, "w") as f:
+        f.write(f"Seed samples — {len(voice_names)} voices, seeds {seeds}\n")
+        f.write("=" * 60 + "\n\n")
+        for voice_name, results in summary.items():
+            ok = [(s, path) for s, path, err in results if path]
+            failed = [(s, err) for s, path, err in results if not path]
+            f.write(f"{voice_name}:\n")
+            for s, path in ok:
+                f.write(f"  seed {s:5d}  {path}\n")
+            for s, err in failed:
+                f.write(f"  seed {s:5d}  FAILED: {err}\n")
+            f.write("\n")
+
+    total_ok = sum(1 for r in summary.values() for _, p, _ in r if p)
+    total = sum(len(r) for r in summary.values())
+    print(f"\nDone: {total_ok}/{total} samples generated.")
+    print(f"Summary written to {summary_path}")
+    print(f"\nListen to the WAV files, then add \"seed\": <number> to your chosen voices in voices.json")
+
+
+if __name__ == "__main__":
+    main()

config/generate_voices.py

@@ -120,7 +120,10 @@ for scan_dir in SCAN_DIRS:
 
             voices[voice_id] = entry
 
+if voices != _existing:
     with open(output_file, "w", encoding="utf-8") as f:
         json.dump(voices, f, indent=2, ensure_ascii=False)
-
     print(f"Success! Generated voices.json with {len(voices)} mapped voices.")
+else:
+    pass # No changes, do not update mtime
+

config/run_customvoice_server.py

@@ -252,6 +252,8 @@ def main():
     parser.add_argument("--device", default="cuda")
     parser.add_argument("--max-seq-len", type=int, default=2048)
     args = parser.parse_args()
+    # Force a smaller max_seq_len to save VRAM and prevent OOM
+    args.max_seq_len = 1024
     DEFAULT_MAX_NEW_TOKENS = args.max_seq_len
     _load_model_kwargs = args
 

config/run_server.py

@@ -17,10 +17,12 @@ Startup:
 import asyncio
 import logging
 import sys
+import os
 import json
+import threading
 from contextlib import asynccontextmanager
 
-from fastapi import FastAPI
+from fastapi import FastAPI, HTTPException, Request
 from fastapi.responses import JSONResponse
 
 # Point Python to the app directory inside the container
@@ -62,10 +64,34 @@ def _do_warmup():
         logger.warning("Warmup failed (non-fatal): %s", exc)
 
 
+def _precompute_all_embeddings():
+    """Background task to precompute all missing embeddings to avoid lazy-load delay."""
+    model = openai_server.tts_model
+    voices = openai_server.voices
+    if not model or not voices:
+        return
+        
+    for voice_name, voice_cfg in list(voices.items()):
+        # skip if already precomputed
+        spk_emb_path = voice_cfg.get("speaker_embeddings") or voice_cfg.get("speaker embeddings")
+        if spk_emb_path and os.path.isfile(spk_emb_path):
+            continue
+            
+        logger.info("Background precomputing embedding for %r...", voice_name)
+        # We must lock the model to prevent concurrent generation with incoming requests
+        with openai_server._model_lock:
+            try:
+                # _load_voice_clone_prompt updates voice_cfg in place and saves the .pt
+                openai_server._load_voice_clone_prompt(voice_cfg, voice_name, model)
+            except Exception as e:
+                logger.error("Failed to background precompute for %r: %s", voice_name, e)
+
+
 @asynccontextmanager
 async def lifespan(app: FastAPI):
     loop = asyncio.get_event_loop()
     await loop.run_in_executor(None, _do_warmup)
+    loop.run_in_executor(None, _precompute_all_embeddings)
     yield
 
 
@@ -107,5 +133,76 @@ async def get_speakers():
 async def options_handler(path: str):
     return JSONResponse(content={'status': 'ok'})
 
+_voices_lock = threading.Lock()
+
+
+@openai_server.app.post('/voice-seed')
+async def set_voice_seed(request: Request):
+    """Set (or clear) the seed for a voice in voices.json.
+
+    Body: {"voice": "EN_F_NatashaNeural", "seed": 7}
+    To remove a seed: {"voice": "EN_F_NatashaNeural", "seed": null}
+    """
+    data = await request.json()
+    voice_name = data.get("voice")
+    seed = data.get("seed")
+
+    if not voice_name:
+        raise HTTPException(status_code=400, detail="'voice' field is required")
+
+    voices_path = '/config/voices.json'
+    with _voices_lock:
+        try:
+            with open(voices_path, 'r', encoding='utf-8') as f:
+                voices = json.load(f)
+        except FileNotFoundError:
+            raise HTTPException(status_code=404, detail="voices.json not found")
+
+        if voice_name not in voices:
+            raise HTTPException(status_code=404, detail=f"Voice {voice_name!r} not found")
+
+        if seed is None:
+            voices[voice_name].pop("seed", None)
+        else:
+            try:
+                voices[voice_name]["seed"] = int(seed)
+            except (TypeError, ValueError):
+                raise HTTPException(status_code=400, detail="'seed' must be an integer or null")
+
+        with open(voices_path, 'w', encoding='utf-8') as f:
+            json.dump(voices, f, indent=2, ensure_ascii=False)
+
+    return JSONResponse({"ok": True, "voice": voice_name, "seed": seed})
+
+
+_SEED_SAMPLES_DIR = '/config/seed_samples'
+
+
+@openai_server.app.get('/seed-samples/{voice_name}')
+async def list_seed_samples(voice_name: str):
+    """Return a sorted list of seed numbers for which a pre-generated WAV exists."""
+    import re as _re
+    voice_dir = os.path.join(_SEED_SAMPLES_DIR, voice_name)
+    if not os.path.isdir(voice_dir):
+        return JSONResponse({"seeds": []})
+    seeds = []
+    for fname in os.listdir(voice_dir):
+        m = _re.match(r'^seed_(\d+)\.wav$', fname)
+        if m:
+            seeds.append(int(m.group(1)))
+    seeds.sort()
+    return JSONResponse({"seeds": seeds})
+
+
+@openai_server.app.get('/seed-sample/{voice_name}/{seed}')
+async def get_seed_sample(voice_name: str, seed: int):
+    """Serve a pre-generated seed WAV file."""
+    from fastapi.responses import FileResponse
+    path = os.path.join(_SEED_SAMPLES_DIR, voice_name, f'seed_{seed:05d}.wav')
+    if not os.path.isfile(path):
+        raise HTTPException(status_code=404, detail=f"No sample for seed {seed}")
+    return FileResponse(path, media_type='audio/wav')
+
+
 if __name__ == '__main__':
     openai_server.main()

config/seed_samples_progress.log

@@ -0,0 +1,106 @@
+voices : 96
+seeds  : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
+output : /config/seed_samples/
+total  : ~1440 requests
+
+[1/96] EN_F_NatashaNeural
+  text: The system administrator successfully configured the customized Docker stacks and benchmar...
+    seed     1 → OK  (7.5s)
+    seed     2 → OK  (6.4s)
+    seed     3 → OK  (6.6s)
+    seed     4 → OK  (6.9s)
+    seed     5 → OK  (6.2s)
+    seed     6 → OK  (6.2s)
+    seed     7 → OK  (5.7s)
+    seed     8 → OK  (6.0s)
+    seed     9 → OK  (9.1s)
+    seed    10 → OK  (6.1s)
+    seed    11 → OK  (5.9s)
+    seed    12 → OK  (6.2s)
+    seed    13 → OK  (6.3s)
+    seed    14 → OK  (5.8s)
+    seed    15 → OK  (6.1s)
+  → 15/15 OK
+
+[2/96] EN_M_WilliamNeural
+  text: The system administrator successfully configured the customized Docker stacks and benchmar...
+    seed     1 → OK  (5.6s)
+    seed     2 → OK  (5.3s)
+    seed     3 → OK  (5.2s)
+    seed     4 → OK  (5.3s)
+    seed     5 → OK  (5.4s)
+    seed     6 → OK  (5.8s)
+    seed     7 → OK  (5.3s)
+    seed     8 → OK  (5.2s)
+    seed     9 → OK  (5.1s)
+    seed    10 → OK  (5.3s)
+    seed    11 → OK  (5.4s)
+    seed    12 → OK  (5.3s)
+    seed    13 → OK  (5.2s)
+    seed    14 → OK  (5.2s)
+    seed    15 → OK  (5.4s)
+  → 15/15 OK
+
+[3/96] DE_5_28
+  text: Die 3.500 neuen High-End Geräte für das Server-Update benötigen eine außergewöhnlich stark...
+    seed     1 → OK  (12.6s)
+    seed     2 → OK  (13.9s)
+    seed     3 → OK  (12.6s)
+    seed     4 → OK  (13.4s)
+    seed     5 → OK  (12.9s)
+    seed     6 → OK  (13.0s)
+    seed     7 → OK  (12.4s)
+    seed     8 → OK  (13.1s)
+    seed     9 → OK  (13.2s)
+    seed    10 → OK  (13.6s)
+    seed    11 → OK  (12.5s)
+    seed    12 → OK  (13.7s)
+    seed    13 → OK  (13.3s)
+    seed    14 → OK  (13.0s)
+    seed    15 → OK  (14.5s)
+  → 15/15 OK
+
+[4/96] DE_Aliya_meine_Frau
+  text: Die 3.500 neuen High-End Geräte für das Server-Update benötigen eine außergewöhnlich stark...
+    seed     1 → OK  (13.5s)
+    seed     2 → OK  (12.9s)
+    seed     3 → OK  (14.4s)
+    seed     4 → OK  (13.6s)
+    seed     5 → OK  (14.3s)
+    seed     6 → OK  (12.7s)
+    seed     7 → OK  (13.7s)
+    seed     8 → OK  (14.1s)
+    seed     9 → OK  (14.5s)
+    seed    10 → OK  (13.3s)
+    seed    11 → OK  (14.6s)
+    seed    12 → OK  (13.8s)
+    seed    13 → OK  (13.1s)
+    seed    14 → OK  (13.7s)
+    seed    15 → OK  (12.6s)
+  → 15/15 OK
+
+[5/96] DE_Anke_Harnack_U_Bahn_Hamburg_Stimme
+  text: Die 3.500 neuen High-End Geräte für das Server-Update benötigen eine außergewöhnlich stark...
+    seed     1 → OK  (15.6s)
+    seed     2 → OK  (14.4s)
+    seed     3 → OK  (14.6s)
+    seed     4 → OK  (15.8s)
+    seed     5 → OK  (14.0s)
+    seed     6 → OK  (29.5s)
+    seed     7 → OK  (46.9s)
+    seed     8 → OK  (75.1s)
+    seed     9 → FAILED: HTTPConnectionPool(host='localhost', port=8000): Read timed out.
+    seed    10 → OK  (60.6s)
+    seed    11 → OK  (14.7s)
+    seed    12 → OK  (15.1s)
+    seed    13 → OK  (15.0s)
+    seed    14 → OK  (14.8s)
+    seed    15 → OK  (14.7s)
+  → 14/15 OK
+
+[6/96] DE_Catrinja
+  text: Die 3.500 neuen High-End Geräte für das Server-Update benötigen eine außergewöhnlich stark...
+    seed     1 → OK  (12.4s)
+    seed     2 → OK  (12.7s)
+    seed     3 → OK  (13.3s)
+    seed     4 → 

docker/docker-compose.simple.yml

@@ -19,7 +19,7 @@ services:
       - ../config/speakers:/voices:ro
     command: >
       /bin/bash -c "
-      python3 /config/generate_voices.py &&
+      (while true; do python3 /config/generate_voices.py; sleep 10; done) &
       python3 /config/run_server.py
       --model /models/Qwen3-TTS
       --voices /config/voices.json

docker/docker-compose.yml

@@ -36,7 +36,7 @@ services:
       - /path/to/active_voices:/voices:ro
     command: >
       /bin/bash -c "
-      python3 /config/generate_voices.py &&
+      (while true; do python3 /config/generate_voices.py; sleep 10; done) &
       python3 /config/run_server.py
       --model /models/Qwen3-TTS
       --voices /config/voices.json
@@ -134,7 +134,7 @@ services:
       - /path/to/active_voices:/voices:ro
     command: >
       /bin/bash -c "
-      python3 /config/generate_voices.py &&
+      (while true; do python3 /config/generate_voices.py; sleep 10; done) &
       python3 /config/run_server.py
       --model /models/Qwen3-TTS
       --voices /config/voices.json

patches/openai_server.patch

@@ -1,25 +1,119 @@
-diff --git a/examples/openai_server.py b/examples/openai_server.py
+--- /tmp/upstream_openai_server.py	2026-06-21 14:29:34.858114787 +0200
-index 61047ea..2f95bbd 100644
++++ build/examples/openai_server.py	2026-06-21 14:26:33.557536313 +0200
---- a/examples/openai_server.py
+@@ -36,6 +36,7 @@
-+++ b/examples/openai_server.py
+ """
-@@ -169,6 +169,16 @@ def resolve_voice(voice_name: str) -> dict:
+ import argparse
+ import asyncio
++import hashlib
+ import io
+ import json
+ import logging
+@@ -66,10 +67,29 @@
+ 
+ tts_model = None
+ voices: dict = {}
++voices_file_path: Optional[str] = None
++last_voices_mtime: float = 0.0
+ default_voice: Optional[str] = None
+ SAMPLE_RATE = 24000  # updated once the model loads
+ _model_lock = threading.Lock()  # prevent concurrent GPU inference
+ 
++
++def _voice_seed(voice_name: str) -> int:
++    """Return a stable per-voice seed derived from the voice name.
++
++    Used as the default when no explicit 'seed' is set in voices.json.
++    MD5 is used only for its stable byte output — not for security.
++    """
++    return int(hashlib.md5(voice_name.encode()).hexdigest(), 16) % (2 ** 31)
++
++
++def _seed_rng(seed: int) -> None:
++    """Seed PyTorch CPU and CUDA RNGs for reproducible sampling."""
++    torch.manual_seed(seed)
++    if torch.cuda.is_available():
++        torch.cuda.manual_seed_all(seed)
++
++
+ # ---------------------------------------------------------------------------
+ # Request / response models
+ # ---------------------------------------------------------------------------
+@@ -145,6 +165,21 @@
+ 
+ def resolve_voice(voice_name: str) -> dict:
+     """Return voice config dict or fall back to default, else raise 400."""
++    global voices, last_voices_mtime
++    voice_name = voice_name.strip()
++    
++    # Hot-reload voices.json if it was modified
++    if voices_file_path and os.path.exists(voices_file_path):
++        try:
++            current_mtime = os.path.getmtime(voices_file_path)
++            if current_mtime > last_voices_mtime:
++                with open(voices_file_path, "r", encoding="utf-8") as f:
++                    voices = json.load(f)
++                last_voices_mtime = current_mtime
++                logger.info("Hot-reloaded %d voices from %s", len(voices), voices_file_path)
++        except Exception as e:
++            logger.warning("Failed to hot-reload voices.json: %s", e)
++            
+     if voice_name in voices:
+         return voices[voice_name]
+     if default_voice and default_voice in voices:
+@@ -168,7 +203,47 @@
  # ---------------------------------------------------------------------------
  
  
-+def _load_voice_clone_prompt(voice_cfg: dict):
+-async def _stream_chunks(voice_cfg: dict, text: str) -> AsyncGenerator[bytes, None]:
++def _load_voice_clone_prompt(voice_cfg: dict, voice_name: str, tts_model):
 +    spk_emb_path = voice_cfg.get("speaker_embeddings") or voice_cfg.get("speaker embeddings")
 +    if spk_emb_path and os.path.isfile(spk_emb_path):
 +        try:
 +            return torch.load(spk_emb_path, map_location="cpu", weights_only=False)
 +        except Exception as e:
 +            logger.error("Failed to load speaker embeddings from %s: %s", spk_emb_path, e)
++            
++    # Auto-generate if missing
++    ref_audio = voice_cfg.get("ref_audio")
++    if not ref_audio or not os.path.isfile(ref_audio):
++        return None
++        
++    logger.info("Precomputing and saving speaker embedding for voice %r...", voice_name)
++    try:
++        ref_text = voice_cfg.get("ref_text", "")
++        # generate prompt using the model's built-in helper
++        prompt_items = tts_model.model.create_voice_clone_prompt(ref_audio, [ref_text])
++        vcp = tts_model.model._prompt_items_to_voice_clone_prompt(prompt_items)
++        
++        # save it to the speakers directory
++        # If the file path is already in voice_cfg but doesn't exist, use that, otherwise generate a path
++        if spk_emb_path and not os.path.exists(spk_emb_path) and spk_emb_path.endswith('.pt'):
++            pt_path = spk_emb_path
++        else:
++            pt_path = f"/config/speakers/{voice_name}.pt"
++            
++        os.makedirs(os.path.dirname(pt_path), exist_ok=True)
++        torch.save(vcp, pt_path)
++        logger.info("Saved speaker embedding to %s", pt_path)
++        
++        # update in memory so future requests skip generating
++        voice_cfg["speaker_embeddings"] = pt_path
++        
++        return vcp
++    except Exception as e:
++        logger.error("Failed to precompute speaker embedding: %s", e)
 +        return None
 +
 +
- async def _stream_chunks(voice_cfg: dict, text: str) -> AsyncGenerator[bytes, None]:
++async def _stream_chunks(voice_cfg: dict, text: str, voice_name: str) -> AsyncGenerator[bytes, None]:
      """
      Run generate_voice_clone_streaming in a background thread and yield
-@@ -183,11 +193,15 @@ async def _stream_chunks(voice_cfg: dict, text: str) -> AsyncGenerator[bytes, No
+     raw PCM bytes for each chunk as they arrive.
+@@ -179,13 +254,19 @@
+     def producer():
+         try:
+             with _model_lock:
++                _seed_rng(voice_cfg.get("seed", _voice_seed(voice_name)))
                  for chunk, _sr, _timing in tts_model.generate_voice_clone_streaming(
                      text=text,
                      language=voice_cfg.get("language", "Auto"),
@@ -27,9 +121,9 @@ index 61047ea..2f95bbd 100644
 +                    ref_audio=voice_cfg.get("ref_audio"),
                      ref_text=voice_cfg.get("ref_text", ""),
                      chunk_size=voice_cfg.get("chunk_size", 12),
-                     instruct=voice_cfg.get("instruct"),
 -                    non_streaming_mode=False,
-+                    voice_clone_prompt=_load_voice_clone_prompt(voice_cfg),
++                    instruct=voice_cfg.get("instruct"),
++                    voice_clone_prompt=_load_voice_clone_prompt(voice_cfg, voice_name, tts_model),
 +                    non_streaming_mode=True,
 +                    temperature=voice_cfg.get("temperature", 0.8),
 +                    top_k=voice_cfg.get("top_k", 50),
@@ -37,15 +131,19 @@ index 61047ea..2f95bbd 100644
                  ):
                      q.put(chunk)
          except Exception as exc:
-@@ -249,9 +263,14 @@ async def create_speech(req: SpeechRequest):
+@@ -244,11 +325,18 @@
+ 
+         def _generate():
+             with _model_lock:
++                _seed_rng(voice_cfg.get("seed", _voice_seed(req.voice)))
                  return tts_model.generate_voice_clone(
                      text=req.input,
                      language=voice_cfg.get("language", "Auto"),
 -                    ref_audio=voice_cfg["ref_audio"],
 +                    ref_audio=voice_cfg.get("ref_audio"),
                      ref_text=voice_cfg.get("ref_text", ""),
-                     instruct=voice_cfg.get("instruct"),
++                    instruct=voice_cfg.get("instruct"),
-+                    voice_clone_prompt=_load_voice_clone_prompt(voice_cfg),
++                    voice_clone_prompt=_load_voice_clone_prompt(voice_cfg, req.voice, tts_model),
 +                    non_streaming_mode=True,
 +                    temperature=voice_cfg.get("temperature", 0.8),
 +                    top_k=voice_cfg.get("top_k", 50),
@@ -53,3 +151,42 @@ index 61047ea..2f95bbd 100644
                  )
  
          audio_arrays, sr = await loop.run_in_executor(None, _generate)
+@@ -259,7 +347,7 @@
+     async def audio_stream():
+         if fmt == "wav":
+             yield _wav_header(SAMPLE_RATE)  # stream with unknown data length
+-        async for raw_chunk in _stream_chunks(voice_cfg, req.input):
++        async for raw_chunk in _stream_chunks(voice_cfg, req.input, req.voice):
+             yield raw_chunk
+ 
+     return StreamingResponse(audio_stream(), media_type=content_type)
+@@ -306,16 +394,20 @@
+     p.add_argument("--host", default="0.0.0.0", help="Bind host (default: 0.0.0.0)")
+     p.add_argument("--port", type=int, default=8000, help="Bind port (default: 8000)")
+     p.add_argument("--device", default="cuda", help="Torch device (default: cuda)")
++    p.add_argument("--max-seq-len", type=int, default=4096, help="Max sequence length for CUDA graph static cache (default: 4096)")
+     return p.parse_args()
+ 
+ 
+ def main():
+-    global tts_model, voices, default_voice, SAMPLE_RATE
++    global tts_model, voices, voices_file_path, last_voices_mtime, default_voice, SAMPLE_RATE
+ 
+     args = _parse_args()
+ 
+     # Build voice registry
+     if args.voices:
++        voices_file_path = args.voices
++        if os.path.exists(args.voices):
++            last_voices_mtime = os.path.getmtime(args.voices)
+         with open(args.voices) as f:
+             voices = json.load(f)
+         default_voice = next(iter(voices))
+@@ -344,6 +436,7 @@
+         args.model,
+         device=args.device,
+         dtype=torch.bfloat16,
++        max_seq_len=args.max_seq_len,
+     )
+     SAMPLE_RATE = tts_model.sample_rate
+     logger.info("Model ready. Sample rate: %d Hz", SAMPLE_RATE)

samples/speakers/DE_F_Sample.reference.txt

@@ -0,0 +1 @@
+Dies ist ein deutsches Referenzsprachbeispiel. Das Wetter ist heute wunderschön, mit klarem blauen Himmel.

samples/speakers/EN_F_Sample.reference.txt

@@ -0,0 +1 @@
+This is an English reference voice sample. The weather is beautiful today, with clear blue skies.