mARTin-B78
ccccdf820a
- Fixed-window CUDA-graph codec decode for streaming (TTS_GRAPH_DECODE=0 disables) - Per-request chunk_size (2-24) on /v1/audio/speech - prebuffer_ms (0-10000) to hold back audio before sending - Stop generation on client disconnect - Recommend --max-seq-len 2048 for the streaming container Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
123 lines
3.7 KiB
Markdown
123 lines
3.7 KiB
Markdown
# Qwen3-TTS Streaming — DGX Spark (GB10)
|
|
|
|
Low-latency, OpenAI-compatible streaming TTS server for the **NVIDIA DGX Spark GB10**
|
|
(ARM64 / SM 121 / CUDA 13), powered by [faster-qwen3-tts](https://github.com/andimarafioti/faster-qwen3-tts)
|
|
with CUDA graph acceleration.
|
|
|
|
Streams WAV audio chunks to the client while generation is still running —
|
|
first audio arrives in under a second for typical sentences.
|
|
|
|
Part of a four-backend TTS stack documented on the NVIDIA Developer Forum:
|
|
[Three times (VoiceClone | VoiceDesign | CustomVoice) — Faster-Qwen3-TTS for NVIDIA DGX Spark (GB10)](https://forums.developer.nvidia.com/t/three-times-voiceclone-voicedesign-customvoice-faster-qwen3-tts-for-nvidia-dgx-spark-gb10/370530)
|
|
|
|
---
|
|
|
|
## Quick start
|
|
|
|
```bash
|
|
docker run -d \
|
|
--runtime nvidia \
|
|
--name qwen3-tts-streaming \
|
|
-p 8023:8000 \
|
|
-e NVIDIA_VISIBLE_DEVICES=all \
|
|
-v /path/to/Qwen3-TTS-12Hz-0.6B-Base:/models/Qwen3-TTS:ro \
|
|
-v /path/to/faster-qwen3-tts/config:/config:rw \
|
|
-v /path/to/active_voices:/voices:ro \
|
|
martinb78/faster-qwen3-tts-dgx-spark:streaming \
|
|
/bin/bash -c "
|
|
python3 /config/generate_voices.py &&
|
|
python3 /config/run_server.py
|
|
--model /models/Qwen3-TTS
|
|
--voices /config/voices.json
|
|
--port 8000
|
|
--max-seq-len 2048
|
|
"
|
|
```
|
|
|
|
Download the 0.6B Base weights before starting the container:
|
|
|
|
```bash
|
|
hf download Qwen/Qwen3-TTS-12Hz-0.6B-Base --local-dir /path/to/Qwen3-TTS-12Hz-0.6B-Base
|
|
```
|
|
|
|
For an existing stack, change only the streaming service's model bind mount.
|
|
`--model /models/Qwen3-TTS` loads that directory; `QWEN_TTS_MODEL` does not
|
|
override this explicit argument. Use Base for reference-audio voice cloning.
|
|
|
|
Check it's running:
|
|
|
|
```bash
|
|
curl http://localhost:8023/health
|
|
```
|
|
|
|
---
|
|
|
|
## Voice configuration
|
|
|
|
Create a `voices.json` in your config directory. Each entry maps a voice ID to a
|
|
reference audio file and transcript:
|
|
|
|
```json
|
|
{
|
|
"william": {
|
|
"ref_audio": "/voices/william.wav",
|
|
"ref_text": "The quick brown fox jumps over the lazy dog.",
|
|
"language": "English",
|
|
"temperature": 0.75,
|
|
"top_k": 40,
|
|
"top_p": 0.85
|
|
},
|
|
"natasha": {
|
|
"ref_audio": "/voices/natasha.wav",
|
|
"ref_text": "She sells seashells by the seashore.",
|
|
"language": "English"
|
|
}
|
|
}
|
|
```
|
|
|
|
`temperature`, `top_k`, and `top_p` are optional — defaults are `0.8 / 50 / 0.9`.
|
|
|
|
---
|
|
|
|
## API
|
|
|
|
OpenAI-compatible `/v1/audio/speech`:
|
|
|
|
```bash
|
|
curl http://localhost:8023/v1/audio/speech \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"model": "tts-1", "input": "Hello world!", "voice": "william", "response_format": "wav"}' \
|
|
--output speech.wav
|
|
```
|
|
|
|
| Endpoint | Method | Description |
|
|
|---|---|---|
|
|
| `/v1/audio/speech` | POST | Generate speech (WAV / PCM / MP3) |
|
|
| `/v1/models` | GET | List available voice IDs |
|
|
| `/v1/audio/voices` | GET | Voice list (OpenWebUI fallback) |
|
|
| `/speakers` | GET | Voice list (SillyTavern) |
|
|
| `/health` | GET | Liveness check |
|
|
|
|
Works with **OpenWebUI**, **SillyTavern**, **llama-swap**, and any OpenAI-compatible client.
|
|
|
|
---
|
|
|
|
## Requirements
|
|
|
|
- NVIDIA DGX Spark GB10 or another ARM64 system with CUDA 13
|
|
- CUDA driver 580+
|
|
- Docker + NVIDIA Container Toolkit
|
|
- [Qwen3-TTS-12Hz-0.6B-Base](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-0.6B-Base) weights downloaded locally
|
|
|
|
---
|
|
|
|
## Related images
|
|
|
|
| Image | Description |
|
|
|---|---|
|
|
| `martinb78/faster-qwen3-tts-dgx-spark:latest` / `:v5` | VoiceClone, VoiceDesign, and CustomVoice backends |
|
|
| `martinb78/faster-qwen3-tts-dgx-spark:streaming` | This tag — streaming VoiceClone |
|
|
|
|
Full four-backend `docker-compose` setup in `docker/docker-compose.yml` and detailed documentation on GitHub and the
|
|
[NVIDIA Developer Forum](https://forums.developer.nvidia.com/t/three-times-voiceclone-voicedesign-customvoice-faster-qwen3-tts-for-nvidia-dgx-spark-gb10/370530).
|