- run_server.py: warm up CUDA graphs at server startup via lifespan
event so the first real request does not pay the 7-8s graph-
compilation penalty; uses modern lifespan API instead of deprecated
on_event
- generate_voices.py: set chunk_size=4 per voice so streaming clients
receive first audio after ~333ms instead of ~1s
- docker-compose.yml: add --max-seq-len 2048 (halves static KV cache,
reduces VRAM and graph-capture time)
- config/benchmark_api.py: new script to measure TTFA, RTF and speed
against the live API endpoint
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Run Qwen3-TTS with CUDA graph acceleration on NVIDIA DGX Spark
(ARM64 / SM 121 / CUDA 13) as an OpenAI-compatible TTS API server.
- Dockerfile targeting nvidia/cuda:13.0.2-base-ubuntu24.04 with ARM64 cu130 PyTorch wheels
- Patch for max-seq-len support to handle long reference audio without crashes
- OpenWebUI + SillyTavern compatible API endpoints (/v1/models, /v1/audio/voices, /speakers)
- Voice management: auto-generate voices.json from speaker reference audio files
- Auto-transcription helper script for generating reference text from audio
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
