mARTin-B78/tts-dgx-spark-faster-qwen3-tts
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: clamp chunk_size to 2..24 and require 1.7B-Base for streaming (v6.13.1)

Browse Source 
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
mARTin-B78 2026-10-04 01:02:01 +02:00
parent ccccdf820a
commit 37fb8c062e
3 changed files with 15 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
DOCKERHUB_STREAMING.md
View File

@ -20,7 +20,7 @@ docker run -d \
--name qwen3-tts-streaming \ --name qwen3-tts-streaming \
-p 8023:8000 \ -p 8023:8000 \
-e NVIDIA_VISIBLE_DEVICES=all \ -e NVIDIA_VISIBLE_DEVICES=all \
-v /path/to/Qwen3-TTS-12Hz-0.6B-Base:/models/Qwen3-TTS:ro \ -v /path/to/Qwen3-TTS-12Hz-1.7B-Base:/models/Qwen3-TTS:ro \
-v /path/to/faster-qwen3-tts/config:/config:rw \ -v /path/to/faster-qwen3-tts/config:/config:rw \
-v /path/to/active_voices:/voices:ro \ -v /path/to/active_voices:/voices:ro \
martinb78/faster-qwen3-tts-dgx-spark:streaming \ martinb78/faster-qwen3-tts-dgx-spark:streaming \
@ -34,12 +34,14 @@ docker run -d \
" "
``` ```
Download the 0.6B Base weights before starting the container: Download the 1.7B Base weights before starting the container:
```bash ```bash
hf download Qwen/Qwen3-TTS-12Hz-0.6B-Base --local-dir /path/to/Qwen3-TTS-12Hz-0.6B-Base hf download Qwen/Qwen3-TTS-12Hz-1.7B-Base --local-dir /path/to/Qwen3-TTS-12Hz-1.7B-Base
``` ```
Streaming must use the same model size as the one voices.json was built for (1.7B-Base), otherwise requests fail with a 1024 vs 2048 tensor size mismatch.
For an existing stack, change only the streaming service's model bind mount. For an existing stack, change only the streaming service's model bind mount.
`--model /models/Qwen3-TTS` loads that directory; `QWEN_TTS_MODEL` does not `--model /models/Qwen3-TTS` loads that directory; `QWEN_TTS_MODEL` does not
override this explicit argument. Use Base for reference-audio voice cloning. override this explicit argument. Use Base for reference-audio voice cloning.
@ -107,7 +109,7 @@ Works with **OpenWebUI**, **SillyTavern**, **llama-swap**, and any OpenAI-compat
- NVIDIA DGX Spark GB10 or another ARM64 system with CUDA 13 - NVIDIA DGX Spark GB10 or another ARM64 system with CUDA 13
- CUDA driver 580+ - CUDA driver 580+
- Docker + NVIDIA Container Toolkit - Docker + NVIDIA Container Toolkit
- [Qwen3-TTS-12Hz-0.6B-Base](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-0.6B-Base) weights downloaded locally - [Qwen3-TTS-12Hz-1.7B-Base](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) weights downloaded locally
--- ---

11
README.md
View File

@ -19,8 +19,8 @@ One Docker image covers all four backends, with two semantic tag aliases:
| Tag | Use for | | Tag | Use for |
|---|---| |---|---|
| `:latest` / `:v6` / `:v6.13` | VoiceClone, VoiceDesign, CustomVoice | | `:latest` / `:v6` / `:v6.13.1` | VoiceClone, VoiceDesign, CustomVoice |
| `:latest-streaming` / `:v6-streaming` / `:v6.13-streaming` | Streaming VoiceClone | | `:latest-streaming` / `:v6-streaming` / `:v6.13.1-streaming` | Streaming VoiceClone |
Both tags point to the same image — the `-streaming` suffix is a semantic convention so compose files and version pins are unambiguous. Both tags point to the same image — the `-streaming` suffix is a semantic convention so compose files and version pins are unambiguous.
@ -354,7 +354,7 @@ The first request after container startup can be slower because CUDA graph captu
- The 1.7B Qwen3-TTS models use about 6 GB of GPU memory each in bfloat16. - The 1.7B Qwen3-TTS models use about 6 GB of GPU memory each in bfloat16.
- The forum playbook shows the four API containers running together on DGX Spark with low visible memory pressure, but exact usage depends on model size, sequence length, and warmup state. - The forum playbook shows the four API containers running together on DGX Spark with low visible memory pressure, but exact usage depends on model size, sequence length, and warmup state.
- Use the 0.6B Qwen3-TTS variants if you want a lighter multi-service setup. - The 0.6B Qwen3-TTS variants are lighter but need their own `voices.json` regenerated with that model's embeddings. Do not use them for the streaming/clone containers, which share the default `voices.json` (1.7B-Base).
- `--max-seq-len 2048` handles most sentence-style TTS requests. Long-form narration may need `4096`, with more memory required. - `--max-seq-len 2048` handles most sentence-style TTS requests. Long-form narration may need `4096`, with more memory required.
- Pin services to different GPUs with `NVIDIA_VISIBLE_DEVICES=0`, `NVIDIA_VISIBLE_DEVICES=1`, and so on if your system has more than one GPU. - Pin services to different GPUs with `NVIDIA_VISIBLE_DEVICES=0`, `NVIDIA_VISIBLE_DEVICES=1`, and so on if your system has more than one GPU.
@ -378,6 +378,11 @@ The first request after container startup can be slower because CUDA graph captu
## Changelog ## Changelog
### v6.13.1 — 2026-10-04
- **Fix:** Streaming must use the same model size as the one voices.json was built for (1.7B-Base), otherwise requests fail with a 1024 vs 2048 tensor size mismatch. Streaming compose examples now default to the 1.7B-Base model.
- **Fix:** Out-of-range `chunk_size` values are clamped to 2–24 (e.g. -5/0/1 -> 2, 1000 -> 24) and still return 200; only non-integers return 422. Covered by an offline check of the patched `SpeechRequest` + `_clamp_chunk_size`.
### v6.13 — 2026-10-04 ### v6.13 — 2026-10-04
**Feature: Per-Request `chunk_size` for Streaming** **Feature: Per-Request `chunk_size` for Streaming**
- `/v1/audio/speech` accepts an optional integer `chunk_size` (codec frames per streamed chunk, 12 = 1 s), clamped to 2–24. It applies to the streaming wav/pcm path only; omitted keeps the voice's `voices.json` value (default unchanged). - `/v1/audio/speech` accepts an optional integer `chunk_size` (codec frames per streamed chunk, 12 = 1 s), clamped to 2–24. It applies to the streaming wav/pcm path only; omitted keeps the voice's `voices.json` value (default unchanged).

2
docker/docker-compose.yml
View File

@ -129,7 +129,7 @@ services:
ports: ports:
- "8023:8000" - "8023:8000"
volumes: volumes:
- /path/to/Qwen3-TTS-12Hz-0.6B-Base:/models/Qwen3-TTS:ro - /path/to/Qwen3-TTS-12Hz-1.7B-Base:/models/Qwen3-TTS:ro
- /path/to/faster-qwen3-tts/config:/config:rw - /path/to/faster-qwen3-tts/config:/config:rw
- /path/to/active_voices:/voices:ro - /path/to/active_voices:/voices:ro
command: > command: >