diff --git a/README.md b/README.md index 360e304..beb8a3e 100644 --- a/README.md +++ b/README.md @@ -138,7 +138,7 @@ python config/auto_transcribe.py --api-url http://localhost:8010/v1/audio/transc `config/generate_voices.py` runs automatically in the background, continuously watching your `speakers` directory. Whenever you add a new `.wav` and `.txt` file, it instantly updates `config/voices.json`. The API server hot-reloads the changes, meaning **you never need to restart the container when adding new voices!** -When you start using a new voice for the first time, the server will automatically do the heavy lifting to extract the voice's acoustic fingerprint (a "speaker embedding") and save it as a `.pt` file in the `config/speakers/` directory. Future requests will instantly load this `.pt` file instead of re-analyzing the audio, which dramatically speeds up Time To First Audio (TTFA). +When you start using a new voice for the first time, the server will automatically do the heavy lifting to extract the voice's acoustic fingerprint (a "speaker embedding") and save it as a `.pt` file in the `config/speakers/` directory. Even better, when the server starts up, it automatically precomputes missing `.pt` files in the background, so your first API requests will be lightning fast. Future requests will instantly load this `.pt` file instead of re-analyzing the audio, which dramatically speeds up Time To First Audio (TTFA). > **Note:** The generation of the `.pt` embedding is completely deterministic. Running the extraction process twice on the same reference `.wav` and `.txt` will yield the exact same fingerprint, so the resulting voice will not vary between regenerations. ## VoiceDesign voices @@ -343,6 +343,12 @@ The first request after container startup can be slower because CUDA graph captu ## Changelog +### v6.6 — 2026-06-21 +**Feature: Eager Background Precomputation of Speaker Embeddings** +- The server now automatically precomputes all missing `.pt` files in the background immediately after startup. +- This ensures all configured voices are pre-warmed and ready to deliver lightning-fast TTFA on the very first request without delaying server startup. +- The lazy-loading mechanism still remains active to instantly handle any new voices hot-reloaded while the server is running. + ### v6.5.1 — 2026-06-21 **Documentation Update** - Added documentation explicitly clarifying that `.pt` speaker embedding generation is fully deterministic and does not produce variable voice characteristics across restarts. diff --git a/config/run_server.py b/config/run_server.py index 51ee83b..c7ccb87 100644 --- a/config/run_server.py +++ b/config/run_server.py @@ -17,6 +17,7 @@ Startup: import asyncio import logging import sys +import os import json from contextlib import asynccontextmanager @@ -62,10 +63,34 @@ def _do_warmup(): logger.warning("Warmup failed (non-fatal): %s", exc) +def _precompute_all_embeddings(): + """Background task to precompute all missing embeddings to avoid lazy-load delay.""" + model = openai_server.tts_model + voices = openai_server.voices + if not model or not voices: + return + + for voice_name, voice_cfg in list(voices.items()): + # skip if already precomputed + spk_emb_path = voice_cfg.get("speaker_embeddings") or voice_cfg.get("speaker embeddings") + if spk_emb_path and os.path.isfile(spk_emb_path): + continue + + logger.info("Background precomputing embedding for %r...", voice_name) + # We must lock the model to prevent concurrent generation with incoming requests + with openai_server._model_lock: + try: + # _load_voice_clone_prompt updates voice_cfg in place and saves the .pt + openai_server._load_voice_clone_prompt(voice_cfg, voice_name, model) + except Exception as e: + logger.error("Failed to background precompute for %r: %s", voice_name, e) + + @asynccontextmanager async def lifespan(app: FastAPI): loop = asyncio.get_event_loop() await loop.run_in_executor(None, _do_warmup) + loop.run_in_executor(None, _precompute_all_embeddings) yield