.gitignore

@@ -21,9 +21,6 @@ config/voices.json*
 # Converted private audio (M4A → WAV, generated at runtime)
 config/converted/
 
-# Seed finder pre-generated WAV samples (batch output, not source files)
-config/seed_samples/
-
 # Editor backup files
 *.py~
 *.yml~

README.md

@@ -136,11 +136,8 @@ Or use the auto-transcription script with a running Whisper-compatible ASR servi
 python config/auto_transcribe.py --api-url http://localhost:8010/v1/audio/transcriptions
 ```
 
-`config/generate_voices.py` runs automatically in the background, continuously watching your `speakers` directory. Whenever you add a new `.wav` and `.txt` file, it instantly updates `config/voices.json`. The API server hot-reloads the changes, meaning **you never need to restart the container when adding new voices!**
+`config/generate_voices.py` runs on container startup and creates `config/voices.json` from your speaker files.
 
-When you start using a new voice for the first time, the server will automatically do the heavy lifting to extract the voice's acoustic fingerprint (a "speaker embedding") and save it as a `.pt` file in the `config/speakers/` directory. Even better, when the server starts up, it automatically precomputes missing `.pt` files in the background, so your first API requests will be lightning fast. Future requests will instantly load this `.pt` file instead of re-analyzing the audio, which dramatically speeds up Time To First Audio (TTFA).
-
-> **Note:** The generation of the `.pt` embedding is completely deterministic. Running the extraction process twice on the same reference `.wav` and `.txt` will yield the exact same fingerprint, so the resulting voice will not vary between regenerations.
 ## VoiceDesign voices
 
 VoiceDesign does not need reference audio. Define reusable voice personalities in `config/voicedesign_voices.json`:
@@ -343,31 +340,6 @@ The first request after container startup can be slower because CUDA graph captu
 
 ## Changelog
 
-### v6.6 — 2026-06-21
-**Feature: Eager Background Precomputation of Speaker Embeddings**
-- The server now automatically precomputes all missing `.pt` files in the background immediately after startup.
-- This ensures all configured voices are pre-warmed and ready to deliver lightning-fast TTFA on the very first request without delaying server startup.
-- The lazy-loading mechanism still remains active to instantly handle any new voices hot-reloaded while the server is running.
-
-### v6.5.1 — 2026-06-21
-**Documentation Update**
-- Added documentation explicitly clarifying that `.pt` speaker embedding generation is fully deterministic and does not produce variable voice characteristics across restarts.
-
-### v6.5 — 2026-06-20
-**Feature: True Zero-Downtime Voice Hot-Reloading**
-
-- The server now watches `voices.json` and hot-reloads it automatically when changes are detected.
-- Added a background loop in `docker-compose.yml` that continuously runs `generate_voices.py` every 10 seconds.
-- You can now drop new `.wav` and `.txt` files into your `speakers/` directory and they will be instantly available via the API without ever restarting the Docker container!
-
-### v6.4 — 2026-06-20
-**Feature: Fully Automated Speaker Embeddings (.pt files)**
-
-- Integrated speaker embedding extraction directly into the API server (`openai_server.py`). 
-- When a new voice is requested for the first time, the server will automatically compute the speaker embedding and save it as a `.pt` file in `/config/speakers/`.
-- Future requests for the same voice automatically use the `.pt` file instead of recalculating the prompt from the `.wav` or `.mp3` reference audio.
-- This provides the massive TTFA speedup of precomputed embeddings without requiring any manual scripting or configuration.
-
 ### v6.3 — 2026-06-20
 **Feature: Precomputed Speaker Embeddings (.pt files)**
 

config/find_best_seed.py

@@ -1,198 +0,0 @@
-#!/usr/bin/env python3
-"""
-Find the best RNG seed for one or all voices by generating audio samples with
-different seeds and saving them as numbered WAV files for comparison.
-
-How it works:
-  1. For each voice × seed, temporarily sets that seed in voices.json
-  2. Calls the running server (hot-reload picks it up automatically)
-  3. Saves audio as <out-dir>/<voice-name>/seed_<N>.wav
-  4. Restores voices.json to its original state when done
-
-Then listen to the files and pick the seed you prefer.
-Add it to voices.json: "seed": <number>
-
-Usage:
-    # All voices, seeds 1–15:
-    python find_best_seed.py --all-voices --range 1 15 --port 8020
-
-    # Single voice:
-    python find_best_seed.py --voice EN_F_NatashaNeural --range 1 20
-    python find_best_seed.py --voice DE_5_28 --seeds 1 7 42 100
-"""
-import argparse
-import json
-import os
-import shutil
-import sys
-import time
-
-import requests
-
-VOICES_JSON = "/config/voices.json"
-
-DE_TEXT = (
-    "Die 3.500 neuen High-End Geräte für das Server-Update benötigen eine "
-    "außergewöhnlich starke Kühlung und regelmäßige Maßnahmen, um die Performance "
-    "bei großer Last zu gewährleisten."
-)
-EN_TEXT = (
-    "The system administrator successfully configured the customized Docker stacks "
-    "and benchmarked the inference engines at exactly 8:45 AM."
-)
-MIXED_TEXT = DE_TEXT + " - " + EN_TEXT
-
-
-def load_voices() -> dict:
-    with open(VOICES_JSON, "r", encoding="utf-8") as f:
-        return json.load(f)
-
-
-def save_voices(voices: dict) -> None:
-    with open(VOICES_JSON, "w", encoding="utf-8") as f:
-        json.dump(voices, f, indent=2, ensure_ascii=False)
-
-
-def generate(voice: str, text: str, host: str, port: int, timeout: int = 90) -> bytes:
-    url = f"http://{host}:{port}/v1/audio/speech"
-    resp = requests.post(
-        url,
-        json={"model": "tts-1", "input": text, "voice": voice, "response_format": "wav"},
-        timeout=timeout,
-        stream=False,
-    )
-    resp.raise_for_status()
-    return resp.content
-
-
-def pick_text(voice_name: str, override: str | None) -> str:
-    if override:
-        return override
-    name_lower = voice_name.lower()
-    if name_lower.startswith("de_"):
-        return MIXED_TEXT
-    if name_lower.startswith("en_") or name_lower.startswith("gb_"):
-        return EN_TEXT
-    return MIXED_TEXT
-
-
-def run_voice(voice_name: str, voices: dict, seeds: list[int],
-              text: str, host: str, port: int, out_dir: str) -> list[tuple]:
-    voice_dir = os.path.join(out_dir, voice_name)
-    os.makedirs(voice_dir, exist_ok=True)
-
-    results = []
-    for seed in seeds:
-        voices[voice_name]["seed"] = seed
-        save_voices(voices)
-        time.sleep(0.3)  # let hot-reload detect mtime change
-
-        out_path = os.path.join(voice_dir, f"seed_{seed:05d}.wav")
-        print(f"    seed {seed:5d} → ", end="", flush=True)
-        try:
-            t0 = time.time()
-            wav = generate(voice_name, text, host, port)
-            elapsed = time.time() - t0
-            with open(out_path, "wb") as f:
-                f.write(wav)
-            print(f"OK  ({elapsed:.1f}s)")
-            results.append((seed, out_path, None))
-        except Exception as e:
-            print(f"FAILED: {e}")
-            results.append((seed, None, str(e)))
-
-    # Remove the temporary seed so the voice returns to its auto-seed
-    voices[voice_name].pop("seed", None)
-    save_voices(voices)
-
-    return results
-
-
-def main():
-    p = argparse.ArgumentParser(description="Compare seeds for one or all voices")
-    target = p.add_mutually_exclusive_group(required=True)
-    target.add_argument("--voice", help="Single voice name")
-    target.add_argument("--all-voices", action="store_true", help="Run for every voice in voices.json")
-
-    p.add_argument("--seeds", type=int, nargs="+", help="Explicit list of seeds to try")
-    p.add_argument("--range", type=int, nargs=2, metavar=("START", "END"),
-                   help="Try seeds START through END (inclusive). Default: 1–15")
-    p.add_argument("--text", default=None,
-                   help="Override text (default: auto-picks DE/EN/mixed based on voice name)")
-    p.add_argument("--host", default="localhost")
-    p.add_argument("--port", type=int, default=8020)
-    p.add_argument("--out-dir", default="./seed_samples",
-                   help="Root output directory (voice subdirs created inside)")
-    args = p.parse_args()
-
-    seeds = list(args.seeds or [])
-    if args.range:
-        seeds += list(range(args.range[0], args.range[1] + 1))
-    if not seeds:
-        seeds = list(range(1, 16))  # default 1–15
-    seeds = sorted(set(seeds))
-
-    voices = load_voices()
-
-    if args.voice:
-        voice_names = [args.voice]
-        if args.voice not in voices:
-            print(f"ERROR: voice {args.voice!r} not in voices.json", file=sys.stderr)
-            sys.exit(1)
-    else:
-        voice_names = list(voices.keys())
-
-    os.makedirs(args.out_dir, exist_ok=True)
-
-    # Single backup at the start; we restore on every error/exit
-    backup = VOICES_JSON + ".seed_backup"
-    shutil.copy2(VOICES_JSON, backup)
-
-    print(f"voices : {len(voice_names)}")
-    print(f"seeds  : {seeds}")
-    print(f"output : {os.path.abspath(args.out_dir)}/")
-    print(f"total  : ~{len(voice_names) * len(seeds)} requests\n")
-
-    summary: dict[str, list] = {}
-
-    try:
-        for i, voice_name in enumerate(voice_names, 1):
-            text = pick_text(voice_name, args.text)
-            print(f"[{i}/{len(voice_names)}] {voice_name}")
-            print(f"  text: {text[:90]}{'...' if len(text) > 90 else ''}")
-            results = run_voice(voice_name, voices, seeds, text, args.host, args.port, args.out_dir)
-            summary[voice_name] = results
-            ok = sum(1 for _, p, _ in results if p)
-            print(f"  → {ok}/{len(seeds)} OK\n")
-
-    except KeyboardInterrupt:
-        print("\n\nInterrupted — restoring voices.json...")
-    finally:
-        shutil.copy2(backup, VOICES_JSON)
-        os.remove(backup)
-        print("voices.json restored.")
-
-    # Write summary
-    summary_path = os.path.join(args.out_dir, "summary.txt")
-    with open(summary_path, "w") as f:
-        f.write(f"Seed samples — {len(voice_names)} voices, seeds {seeds}\n")
-        f.write("=" * 60 + "\n\n")
-        for voice_name, results in summary.items():
-            ok = [(s, path) for s, path, err in results if path]
-            failed = [(s, err) for s, path, err in results if not path]
-            f.write(f"{voice_name}:\n")
-            for s, path in ok:
-                f.write(f"  seed {s:5d}  {path}\n")
-            for s, err in failed:
-                f.write(f"  seed {s:5d}  FAILED: {err}\n")
-            f.write("\n")
-
-    total_ok = sum(1 for r in summary.values() for _, p, _ in r if p)
-    total = sum(len(r) for r in summary.values())
-    print(f"\nDone: {total_ok}/{total} samples generated.")
-    print(f"Summary written to {summary_path}")
-    print(f"\nListen to the WAV files, then add \"seed\": <number> to your chosen voices in voices.json")
-
-
-if __name__ == "__main__":
-    main()

config/generate_voices.py

@@ -120,10 +120,7 @@ for scan_dir in SCAN_DIRS:
 
             voices[voice_id] = entry
 
-if voices != _existing:
+with open(output_file, "w", encoding="utf-8") as f:
-    with open(output_file, "w", encoding="utf-8") as f:
+    json.dump(voices, f, indent=2, ensure_ascii=False)
-        json.dump(voices, f, indent=2, ensure_ascii=False)
-    print(f"Success! Generated voices.json with {len(voices)} mapped voices.")
-else:
-    pass # No changes, do not update mtime
 
+print(f"Success! Generated voices.json with {len(voices)} mapped voices.")

config/run_customvoice_server.py

@@ -252,8 +252,6 @@ def main():
     parser.add_argument("--device", default="cuda")
     parser.add_argument("--max-seq-len", type=int, default=2048)
     args = parser.parse_args()
-    # Force a smaller max_seq_len to save VRAM and prevent OOM
-    args.max_seq_len = 1024
     DEFAULT_MAX_NEW_TOKENS = args.max_seq_len
     _load_model_kwargs = args
 

config/run_server.py

@@ -17,12 +17,10 @@ Startup:
 import asyncio
 import logging
 import sys
-import os
 import json
-import threading
 from contextlib import asynccontextmanager
 
-from fastapi import FastAPI, HTTPException, Request
+from fastapi import FastAPI
 from fastapi.responses import JSONResponse
 
 # Point Python to the app directory inside the container
@@ -64,34 +62,10 @@ def _do_warmup():
         logger.warning("Warmup failed (non-fatal): %s", exc)
 
 
-def _precompute_all_embeddings():
-    """Background task to precompute all missing embeddings to avoid lazy-load delay."""
-    model = openai_server.tts_model
-    voices = openai_server.voices
-    if not model or not voices:
-        return
-        
-    for voice_name, voice_cfg in list(voices.items()):
-        # skip if already precomputed
-        spk_emb_path = voice_cfg.get("speaker_embeddings") or voice_cfg.get("speaker embeddings")
-        if spk_emb_path and os.path.isfile(spk_emb_path):
-            continue
-            
-        logger.info("Background precomputing embedding for %r...", voice_name)
-        # We must lock the model to prevent concurrent generation with incoming requests
-        with openai_server._model_lock:
-            try:
-                # _load_voice_clone_prompt updates voice_cfg in place and saves the .pt
-                openai_server._load_voice_clone_prompt(voice_cfg, voice_name, model)
-            except Exception as e:
-                logger.error("Failed to background precompute for %r: %s", voice_name, e)
-
-
 @asynccontextmanager
 async def lifespan(app: FastAPI):
     loop = asyncio.get_event_loop()
     await loop.run_in_executor(None, _do_warmup)
-    loop.run_in_executor(None, _precompute_all_embeddings)
     yield
 
 
@@ -133,76 +107,5 @@ async def get_speakers():
 async def options_handler(path: str):
     return JSONResponse(content={'status': 'ok'})
 
-_voices_lock = threading.Lock()
-
-
-@openai_server.app.post('/voice-seed')
-async def set_voice_seed(request: Request):
-    """Set (or clear) the seed for a voice in voices.json.
-
-    Body: {"voice": "EN_F_NatashaNeural", "seed": 7}
-    To remove a seed: {"voice": "EN_F_NatashaNeural", "seed": null}
-    """
-    data = await request.json()
-    voice_name = data.get("voice")
-    seed = data.get("seed")
-
-    if not voice_name:
-        raise HTTPException(status_code=400, detail="'voice' field is required")
-
-    voices_path = '/config/voices.json'
-    with _voices_lock:
-        try:
-            with open(voices_path, 'r', encoding='utf-8') as f:
-                voices = json.load(f)
-        except FileNotFoundError:
-            raise HTTPException(status_code=404, detail="voices.json not found")
-
-        if voice_name not in voices:
-            raise HTTPException(status_code=404, detail=f"Voice {voice_name!r} not found")
-
-        if seed is None:
-            voices[voice_name].pop("seed", None)
-        else:
-            try:
-                voices[voice_name]["seed"] = int(seed)
-            except (TypeError, ValueError):
-                raise HTTPException(status_code=400, detail="'seed' must be an integer or null")
-
-        with open(voices_path, 'w', encoding='utf-8') as f:
-            json.dump(voices, f, indent=2, ensure_ascii=False)
-
-    return JSONResponse({"ok": True, "voice": voice_name, "seed": seed})
-
-
-_SEED_SAMPLES_DIR = '/config/seed_samples'
-
-
-@openai_server.app.get('/seed-samples/{voice_name}')
-async def list_seed_samples(voice_name: str):
-    """Return a sorted list of seed numbers for which a pre-generated WAV exists."""
-    import re as _re
-    voice_dir = os.path.join(_SEED_SAMPLES_DIR, voice_name)
-    if not os.path.isdir(voice_dir):
-        return JSONResponse({"seeds": []})
-    seeds = []
-    for fname in os.listdir(voice_dir):
-        m = _re.match(r'^seed_(\d+)\.wav$', fname)
-        if m:
-            seeds.append(int(m.group(1)))
-    seeds.sort()
-    return JSONResponse({"seeds": seeds})
-
-
-@openai_server.app.get('/seed-sample/{voice_name}/{seed}')
-async def get_seed_sample(voice_name: str, seed: int):
-    """Serve a pre-generated seed WAV file."""
-    from fastapi.responses import FileResponse
-    path = os.path.join(_SEED_SAMPLES_DIR, voice_name, f'seed_{seed:05d}.wav')
-    if not os.path.isfile(path):
-        raise HTTPException(status_code=404, detail=f"No sample for seed {seed}")
-    return FileResponse(path, media_type='audio/wav')
-
-
 if __name__ == '__main__':
     openai_server.main()

config/seed_samples_progress.log

@@ -1,106 +0,0 @@
-voices : 96
-seeds  : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
-output : /config/seed_samples/
-total  : ~1440 requests
-
-[1/96] EN_F_NatashaNeural
-  text: The system administrator successfully configured the customized Docker stacks and benchmar...
-    seed     1 → OK  (7.5s)
-    seed     2 → OK  (6.4s)
-    seed     3 → OK  (6.6s)
-    seed     4 → OK  (6.9s)
-    seed     5 → OK  (6.2s)
-    seed     6 → OK  (6.2s)
-    seed     7 → OK  (5.7s)
-    seed     8 → OK  (6.0s)
-    seed     9 → OK  (9.1s)
-    seed    10 → OK  (6.1s)
-    seed    11 → OK  (5.9s)
-    seed    12 → OK  (6.2s)
-    seed    13 → OK  (6.3s)
-    seed    14 → OK  (5.8s)
-    seed    15 → OK  (6.1s)
-  → 15/15 OK
-
-[2/96] EN_M_WilliamNeural
-  text: The system administrator successfully configured the customized Docker stacks and benchmar...
-    seed     1 → OK  (5.6s)
-    seed     2 → OK  (5.3s)
-    seed     3 → OK  (5.2s)
-    seed     4 → OK  (5.3s)
-    seed     5 → OK  (5.4s)
-    seed     6 → OK  (5.8s)
-    seed     7 → OK  (5.3s)
-    seed     8 → OK  (5.2s)
-    seed     9 → OK  (5.1s)
-    seed    10 → OK  (5.3s)
-    seed    11 → OK  (5.4s)
-    seed    12 → OK  (5.3s)
-    seed    13 → OK  (5.2s)
-    seed    14 → OK  (5.2s)
-    seed    15 → OK  (5.4s)
-  → 15/15 OK
-
-[3/96] DE_5_28
-  text: Die 3.500 neuen High-End Geräte für das Server-Update benötigen eine außergewöhnlich stark...
-    seed     1 → OK  (12.6s)
-    seed     2 → OK  (13.9s)
-    seed     3 → OK  (12.6s)
-    seed     4 → OK  (13.4s)
-    seed     5 → OK  (12.9s)
-    seed     6 → OK  (13.0s)
-    seed     7 → OK  (12.4s)
-    seed     8 → OK  (13.1s)
-    seed     9 → OK  (13.2s)
-    seed    10 → OK  (13.6s)
-    seed    11 → OK  (12.5s)
-    seed    12 → OK  (13.7s)
-    seed    13 → OK  (13.3s)
-    seed    14 → OK  (13.0s)
-    seed    15 → OK  (14.5s)
-  → 15/15 OK
-
-[4/96] DE_Aliya_meine_Frau
-  text: Die 3.500 neuen High-End Geräte für das Server-Update benötigen eine außergewöhnlich stark...
-    seed     1 → OK  (13.5s)
-    seed     2 → OK  (12.9s)
-    seed     3 → OK  (14.4s)
-    seed     4 → OK  (13.6s)
-    seed     5 → OK  (14.3s)
-    seed     6 → OK  (12.7s)
-    seed     7 → OK  (13.7s)
-    seed     8 → OK  (14.1s)
-    seed     9 → OK  (14.5s)
-    seed    10 → OK  (13.3s)
-    seed    11 → OK  (14.6s)
-    seed    12 → OK  (13.8s)
-    seed    13 → OK  (13.1s)
-    seed    14 → OK  (13.7s)
-    seed    15 → OK  (12.6s)
-  → 15/15 OK
-
-[5/96] DE_Anke_Harnack_U_Bahn_Hamburg_Stimme
-  text: Die 3.500 neuen High-End Geräte für das Server-Update benötigen eine außergewöhnlich stark...
-    seed     1 → OK  (15.6s)
-    seed     2 → OK  (14.4s)
-    seed     3 → OK  (14.6s)
-    seed     4 → OK  (15.8s)
-    seed     5 → OK  (14.0s)
-    seed     6 → OK  (29.5s)
-    seed     7 → OK  (46.9s)
-    seed     8 → OK  (75.1s)
-    seed     9 → FAILED: HTTPConnectionPool(host='localhost', port=8000): Read timed out.
-    seed    10 → OK  (60.6s)
-    seed    11 → OK  (14.7s)
-    seed    12 → OK  (15.1s)
-    seed    13 → OK  (15.0s)
-    seed    14 → OK  (14.8s)
-    seed    15 → OK  (14.7s)
-  → 14/15 OK
-
-[6/96] DE_Catrinja
-  text: Die 3.500 neuen High-End Geräte für das Server-Update benötigen eine außergewöhnlich stark...
-    seed     1 → OK  (12.4s)
-    seed     2 → OK  (12.7s)
-    seed     3 → OK  (13.3s)
-    seed     4 → 

docker/docker-compose.simple.yml

@@ -19,7 +19,7 @@ services:
       - ../config/speakers:/voices:ro
     command: >
       /bin/bash -c "
-      (while true; do python3 /config/generate_voices.py; sleep 10; done) &
+      python3 /config/generate_voices.py &&
       python3 /config/run_server.py
       --model /models/Qwen3-TTS
       --voices /config/voices.json

docker/docker-compose.yml

@@ -36,7 +36,7 @@ services:
       - /path/to/active_voices:/voices:ro
     command: >
       /bin/bash -c "
-      (while true; do python3 /config/generate_voices.py; sleep 10; done) &
+      python3 /config/generate_voices.py &&
       python3 /config/run_server.py
       --model /models/Qwen3-TTS
       --voices /config/voices.json
@@ -134,7 +134,7 @@ services:
       - /path/to/active_voices:/voices:ro
     command: >
       /bin/bash -c "
-      (while true; do python3 /config/generate_voices.py; sleep 10; done) &
+      python3 /config/generate_voices.py &&
       python3 /config/run_server.py
       --model /models/Qwen3-TTS
       --voices /config/voices.json

patches/openai_server.patch

@@ -1,119 +1,25 @@
---- /tmp/upstream_openai_server.py	2026-06-21 14:29:34.858114787 +0200
+diff --git a/examples/openai_server.py b/examples/openai_server.py
-+++ build/examples/openai_server.py	2026-06-21 14:26:33.557536313 +0200
+index 61047ea..2f95bbd 100644
-@@ -36,6 +36,7 @@
+--- a/examples/openai_server.py
- """
++++ b/examples/openai_server.py
- import argparse
+@@ -169,6 +169,16 @@ def resolve_voice(voice_name: str) -> dict:
- import asyncio
-+import hashlib
- import io
- import json
- import logging
-@@ -66,10 +67,29 @@
- 
- tts_model = None
- voices: dict = {}
-+voices_file_path: Optional[str] = None
-+last_voices_mtime: float = 0.0
- default_voice: Optional[str] = None
- SAMPLE_RATE = 24000  # updated once the model loads
- _model_lock = threading.Lock()  # prevent concurrent GPU inference
- 
-+
-+def _voice_seed(voice_name: str) -> int:
-+    """Return a stable per-voice seed derived from the voice name.
-+
-+    Used as the default when no explicit 'seed' is set in voices.json.
-+    MD5 is used only for its stable byte output — not for security.
-+    """
-+    return int(hashlib.md5(voice_name.encode()).hexdigest(), 16) % (2 ** 31)
-+
-+
-+def _seed_rng(seed: int) -> None:
-+    """Seed PyTorch CPU and CUDA RNGs for reproducible sampling."""
-+    torch.manual_seed(seed)
-+    if torch.cuda.is_available():
-+        torch.cuda.manual_seed_all(seed)
-+
-+
- # ---------------------------------------------------------------------------
- # Request / response models
- # ---------------------------------------------------------------------------
-@@ -145,6 +165,21 @@
- 
- def resolve_voice(voice_name: str) -> dict:
-     """Return voice config dict or fall back to default, else raise 400."""
-+    global voices, last_voices_mtime
-+    voice_name = voice_name.strip()
-+    
-+    # Hot-reload voices.json if it was modified
-+    if voices_file_path and os.path.exists(voices_file_path):
-+        try:
-+            current_mtime = os.path.getmtime(voices_file_path)
-+            if current_mtime > last_voices_mtime:
-+                with open(voices_file_path, "r", encoding="utf-8") as f:
-+                    voices = json.load(f)
-+                last_voices_mtime = current_mtime
-+                logger.info("Hot-reloaded %d voices from %s", len(voices), voices_file_path)
-+        except Exception as e:
-+            logger.warning("Failed to hot-reload voices.json: %s", e)
-+            
-     if voice_name in voices:
-         return voices[voice_name]
-     if default_voice and default_voice in voices:
-@@ -168,7 +203,47 @@
  # ---------------------------------------------------------------------------
  
  
--async def _stream_chunks(voice_cfg: dict, text: str) -> AsyncGenerator[bytes, None]:
++def _load_voice_clone_prompt(voice_cfg: dict):
-+def _load_voice_clone_prompt(voice_cfg: dict, voice_name: str, tts_model):
 +    spk_emb_path = voice_cfg.get("speaker_embeddings") or voice_cfg.get("speaker embeddings")
 +    if spk_emb_path and os.path.isfile(spk_emb_path):
 +        try:
 +            return torch.load(spk_emb_path, map_location="cpu", weights_only=False)
 +        except Exception as e:
 +            logger.error("Failed to load speaker embeddings from %s: %s", spk_emb_path, e)
-+            
++    return None
-+    # Auto-generate if missing
-+    ref_audio = voice_cfg.get("ref_audio")
-+    if not ref_audio or not os.path.isfile(ref_audio):
-+        return None
-+        
-+    logger.info("Precomputing and saving speaker embedding for voice %r...", voice_name)
-+    try:
-+        ref_text = voice_cfg.get("ref_text", "")
-+        # generate prompt using the model's built-in helper
-+        prompt_items = tts_model.model.create_voice_clone_prompt(ref_audio, [ref_text])
-+        vcp = tts_model.model._prompt_items_to_voice_clone_prompt(prompt_items)
-+        
-+        # save it to the speakers directory
-+        # If the file path is already in voice_cfg but doesn't exist, use that, otherwise generate a path
-+        if spk_emb_path and not os.path.exists(spk_emb_path) and spk_emb_path.endswith('.pt'):
-+            pt_path = spk_emb_path
-+        else:
-+            pt_path = f"/config/speakers/{voice_name}.pt"
-+            
-+        os.makedirs(os.path.dirname(pt_path), exist_ok=True)
-+        torch.save(vcp, pt_path)
-+        logger.info("Saved speaker embedding to %s", pt_path)
-+        
-+        # update in memory so future requests skip generating
-+        voice_cfg["speaker_embeddings"] = pt_path
-+        
-+        return vcp
-+    except Exception as e:
-+        logger.error("Failed to precompute speaker embedding: %s", e)
-+        return None
 +
 +
-+async def _stream_chunks(voice_cfg: dict, text: str, voice_name: str) -> AsyncGenerator[bytes, None]:
+ async def _stream_chunks(voice_cfg: dict, text: str) -> AsyncGenerator[bytes, None]:
      """
      Run generate_voice_clone_streaming in a background thread and yield
-     raw PCM bytes for each chunk as they arrive.
+@@ -183,11 +193,15 @@ async def _stream_chunks(voice_cfg: dict, text: str) -> AsyncGenerator[bytes, No
-@@ -179,13 +254,19 @@
-     def producer():
-         try:
-             with _model_lock:
-+                _seed_rng(voice_cfg.get("seed", _voice_seed(voice_name)))
                  for chunk, _sr, _timing in tts_model.generate_voice_clone_streaming(
                      text=text,
                      language=voice_cfg.get("language", "Auto"),
@@ -121,9 +27,9 @@
 +                    ref_audio=voice_cfg.get("ref_audio"),
                      ref_text=voice_cfg.get("ref_text", ""),
                      chunk_size=voice_cfg.get("chunk_size", 12),
+                     instruct=voice_cfg.get("instruct"),
 -                    non_streaming_mode=False,
-+                    instruct=voice_cfg.get("instruct"),
++                    voice_clone_prompt=_load_voice_clone_prompt(voice_cfg),
-+                    voice_clone_prompt=_load_voice_clone_prompt(voice_cfg, voice_name, tts_model),
 +                    non_streaming_mode=True,
 +                    temperature=voice_cfg.get("temperature", 0.8),
 +                    top_k=voice_cfg.get("top_k", 50),
@@ -131,19 +37,15 @@
                  ):
                      q.put(chunk)
          except Exception as exc:
-@@ -244,11 +325,18 @@
+@@ -249,9 +263,14 @@ async def create_speech(req: SpeechRequest):
- 
-         def _generate():
-             with _model_lock:
-+                _seed_rng(voice_cfg.get("seed", _voice_seed(req.voice)))
                  return tts_model.generate_voice_clone(
                      text=req.input,
                      language=voice_cfg.get("language", "Auto"),
 -                    ref_audio=voice_cfg["ref_audio"],
 +                    ref_audio=voice_cfg.get("ref_audio"),
                      ref_text=voice_cfg.get("ref_text", ""),
-+                    instruct=voice_cfg.get("instruct"),
+                     instruct=voice_cfg.get("instruct"),
-+                    voice_clone_prompt=_load_voice_clone_prompt(voice_cfg, req.voice, tts_model),
++                    voice_clone_prompt=_load_voice_clone_prompt(voice_cfg),
 +                    non_streaming_mode=True,
 +                    temperature=voice_cfg.get("temperature", 0.8),
 +                    top_k=voice_cfg.get("top_k", 50),
@@ -151,42 +53,3 @@
                  )
  
          audio_arrays, sr = await loop.run_in_executor(None, _generate)
-@@ -259,7 +347,7 @@
-     async def audio_stream():
-         if fmt == "wav":
-             yield _wav_header(SAMPLE_RATE)  # stream with unknown data length
--        async for raw_chunk in _stream_chunks(voice_cfg, req.input):
-+        async for raw_chunk in _stream_chunks(voice_cfg, req.input, req.voice):
-             yield raw_chunk
- 
-     return StreamingResponse(audio_stream(), media_type=content_type)
-@@ -306,16 +394,20 @@
-     p.add_argument("--host", default="0.0.0.0", help="Bind host (default: 0.0.0.0)")
-     p.add_argument("--port", type=int, default=8000, help="Bind port (default: 8000)")
-     p.add_argument("--device", default="cuda", help="Torch device (default: cuda)")
-+    p.add_argument("--max-seq-len", type=int, default=4096, help="Max sequence length for CUDA graph static cache (default: 4096)")
-     return p.parse_args()
- 
- 
- def main():
--    global tts_model, voices, default_voice, SAMPLE_RATE
-+    global tts_model, voices, voices_file_path, last_voices_mtime, default_voice, SAMPLE_RATE
- 
-     args = _parse_args()
- 
-     # Build voice registry
-     if args.voices:
-+        voices_file_path = args.voices
-+        if os.path.exists(args.voices):
-+            last_voices_mtime = os.path.getmtime(args.voices)
-         with open(args.voices) as f:
-             voices = json.load(f)
-         default_voice = next(iter(voices))
-@@ -344,6 +436,7 @@
-         args.model,
-         device=args.device,
-         dtype=torch.bfloat16,
-+        max_seq_len=args.max_seq_len,
-     )
-     SAMPLE_RATE = tts_model.sample_rate
-     logger.info("Model ready. Sample rate: %d Hz", SAMPLE_RATE)

samples/speakers/DE_F_Sample.reference.txt

@@ -1 +0,0 @@
-Dies ist ein deutsches Referenzsprachbeispiel. Das Wetter ist heute wunderschön, mit klarem blauen Himmel.

samples/speakers/EN_F_Sample.reference.txt

@@ -1 +0,0 @@
-This is an English reference voice sample. The weather is beautiful today, with clear blue skies.