fix: clamp chunk_size to 2..24 and require 1.7B-Base for streaming (v6.13.1)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
ccccdf820a
commit
37fb8c062e
@ -20,7 +20,7 @@ docker run -d \
|
||||
--name qwen3-tts-streaming \
|
||||
-p 8023:8000 \
|
||||
-e NVIDIA_VISIBLE_DEVICES=all \
|
||||
-v /path/to/Qwen3-TTS-12Hz-0.6B-Base:/models/Qwen3-TTS:ro \
|
||||
-v /path/to/Qwen3-TTS-12Hz-1.7B-Base:/models/Qwen3-TTS:ro \
|
||||
-v /path/to/faster-qwen3-tts/config:/config:rw \
|
||||
-v /path/to/active_voices:/voices:ro \
|
||||
martinb78/faster-qwen3-tts-dgx-spark:streaming \
|
||||
@ -34,12 +34,14 @@ docker run -d \
|
||||
"
|
||||
```
|
||||
|
||||
Download the 0.6B Base weights before starting the container:
|
||||
Download the 1.7B Base weights before starting the container:
|
||||
|
||||
```bash
|
||||
hf download Qwen/Qwen3-TTS-12Hz-0.6B-Base --local-dir /path/to/Qwen3-TTS-12Hz-0.6B-Base
|
||||
hf download Qwen/Qwen3-TTS-12Hz-1.7B-Base --local-dir /path/to/Qwen3-TTS-12Hz-1.7B-Base
|
||||
```
|
||||
|
||||
Streaming must use the same model size as the one voices.json was built for (1.7B-Base), otherwise requests fail with a 1024 vs 2048 tensor size mismatch.
|
||||
|
||||
For an existing stack, change only the streaming service's model bind mount.
|
||||
`--model /models/Qwen3-TTS` loads that directory; `QWEN_TTS_MODEL` does not
|
||||
override this explicit argument. Use Base for reference-audio voice cloning.
|
||||
@ -107,7 +109,7 @@ Works with **OpenWebUI**, **SillyTavern**, **llama-swap**, and any OpenAI-compat
|
||||
- NVIDIA DGX Spark GB10 or another ARM64 system with CUDA 13
|
||||
- CUDA driver 580+
|
||||
- Docker + NVIDIA Container Toolkit
|
||||
- [Qwen3-TTS-12Hz-0.6B-Base](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-0.6B-Base) weights downloaded locally
|
||||
- [Qwen3-TTS-12Hz-1.7B-Base](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) weights downloaded locally
|
||||
|
||||
---
|
||||
|
||||
|
||||
11
README.md
11
README.md
@ -19,8 +19,8 @@ One Docker image covers all four backends, with two semantic tag aliases:
|
||||
|
||||
| Tag | Use for |
|
||||
|---|---|
|
||||
| `:latest` / `:v6` / `:v6.13` | VoiceClone, VoiceDesign, CustomVoice |
|
||||
| `:latest-streaming` / `:v6-streaming` / `:v6.13-streaming` | Streaming VoiceClone |
|
||||
| `:latest` / `:v6` / `:v6.13.1` | VoiceClone, VoiceDesign, CustomVoice |
|
||||
| `:latest-streaming` / `:v6-streaming` / `:v6.13.1-streaming` | Streaming VoiceClone |
|
||||
|
||||
Both tags point to the same image — the `-streaming` suffix is a semantic convention so compose files and version pins are unambiguous.
|
||||
|
||||
@ -354,7 +354,7 @@ The first request after container startup can be slower because CUDA graph captu
|
||||
|
||||
- The 1.7B Qwen3-TTS models use about 6 GB of GPU memory each in bfloat16.
|
||||
- The forum playbook shows the four API containers running together on DGX Spark with low visible memory pressure, but exact usage depends on model size, sequence length, and warmup state.
|
||||
- Use the 0.6B Qwen3-TTS variants if you want a lighter multi-service setup.
|
||||
- The 0.6B Qwen3-TTS variants are lighter but need their own `voices.json` regenerated with that model's embeddings. Do not use them for the streaming/clone containers, which share the default `voices.json` (1.7B-Base).
|
||||
- `--max-seq-len 2048` handles most sentence-style TTS requests. Long-form narration may need `4096`, with more memory required.
|
||||
- Pin services to different GPUs with `NVIDIA_VISIBLE_DEVICES=0`, `NVIDIA_VISIBLE_DEVICES=1`, and so on if your system has more than one GPU.
|
||||
|
||||
@ -378,6 +378,11 @@ The first request after container startup can be slower because CUDA graph captu
|
||||
|
||||
## Changelog
|
||||
|
||||
### v6.13.1 — 2026-10-04
|
||||
|
||||
- **Fix:** Streaming must use the same model size as the one voices.json was built for (1.7B-Base), otherwise requests fail with a 1024 vs 2048 tensor size mismatch. Streaming compose examples now default to the 1.7B-Base model.
|
||||
- **Fix:** Out-of-range `chunk_size` values are clamped to 2–24 (e.g. -5/0/1 -> 2, 1000 -> 24) and still return 200; only non-integers return 422. Covered by an offline check of the patched `SpeechRequest` + `_clamp_chunk_size`.
|
||||
|
||||
### v6.13 — 2026-10-04
|
||||
**Feature: Per-Request `chunk_size` for Streaming**
|
||||
- `/v1/audio/speech` accepts an optional integer `chunk_size` (codec frames per streamed chunk, 12 = 1 s), clamped to 2–24. It applies to the streaming wav/pcm path only; omitted keeps the voice's `voices.json` value (default unchanged).
|
||||
|
||||
@ -129,7 +129,7 @@ services:
|
||||
ports:
|
||||
- "8023:8000"
|
||||
volumes:
|
||||
- /path/to/Qwen3-TTS-12Hz-0.6B-Base:/models/Qwen3-TTS:ro
|
||||
- /path/to/Qwen3-TTS-12Hz-1.7B-Base:/models/Qwen3-TTS:ro
|
||||
- /path/to/faster-qwen3-tts/config:/config:rw
|
||||
- /path/to/active_voices:/voices:ro
|
||||
command: >
|
||||
|
||||
Loading…
Reference in New Issue
Block a user