diff --git a/DOCKERHUB_STREAMING.md b/DOCKERHUB_STREAMING.md index 9aaf963..5d39e75 100644 --- a/DOCKERHUB_STREAMING.md +++ b/DOCKERHUB_STREAMING.md @@ -20,7 +20,7 @@ docker run -d \ --name qwen3-tts-streaming \ -p 8023:8000 \ -e NVIDIA_VISIBLE_DEVICES=all \ - -v /path/to/Qwen3-TTS-12Hz-0.6B-Base:/models/Qwen3-TTS:ro \ + -v /path/to/Qwen3-TTS-12Hz-1.7B-Base:/models/Qwen3-TTS:ro \ -v /path/to/faster-qwen3-tts/config:/config:rw \ -v /path/to/active_voices:/voices:ro \ martinb78/faster-qwen3-tts-dgx-spark:streaming \ @@ -34,12 +34,14 @@ docker run -d \ " ``` -Download the 0.6B Base weights before starting the container: +Download the 1.7B Base weights before starting the container: ```bash -hf download Qwen/Qwen3-TTS-12Hz-0.6B-Base --local-dir /path/to/Qwen3-TTS-12Hz-0.6B-Base +hf download Qwen/Qwen3-TTS-12Hz-1.7B-Base --local-dir /path/to/Qwen3-TTS-12Hz-1.7B-Base ``` +Streaming must use the same model size as the one voices.json was built for (1.7B-Base), otherwise requests fail with a 1024 vs 2048 tensor size mismatch. + For an existing stack, change only the streaming service's model bind mount. `--model /models/Qwen3-TTS` loads that directory; `QWEN_TTS_MODEL` does not override this explicit argument. Use Base for reference-audio voice cloning. @@ -107,7 +109,7 @@ Works with **OpenWebUI**, **SillyTavern**, **llama-swap**, and any OpenAI-compat - NVIDIA DGX Spark GB10 or another ARM64 system with CUDA 13 - CUDA driver 580+ - Docker + NVIDIA Container Toolkit -- [Qwen3-TTS-12Hz-0.6B-Base](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-0.6B-Base) weights downloaded locally +- [Qwen3-TTS-12Hz-1.7B-Base](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) weights downloaded locally --- diff --git a/README.md b/README.md index 6173d13..f2d16be 100644 --- a/README.md +++ b/README.md @@ -19,8 +19,8 @@ One Docker image covers all four backends, with two semantic tag aliases: | Tag | Use for | |---|---| -| `:latest` / `:v6` / `:v6.13` | VoiceClone, VoiceDesign, CustomVoice | -| `:latest-streaming` / `:v6-streaming` / `:v6.13-streaming` | Streaming VoiceClone | +| `:latest` / `:v6` / `:v6.13.1` | VoiceClone, VoiceDesign, CustomVoice | +| `:latest-streaming` / `:v6-streaming` / `:v6.13.1-streaming` | Streaming VoiceClone | Both tags point to the same image — the `-streaming` suffix is a semantic convention so compose files and version pins are unambiguous. @@ -354,7 +354,7 @@ The first request after container startup can be slower because CUDA graph captu - The 1.7B Qwen3-TTS models use about 6 GB of GPU memory each in bfloat16. - The forum playbook shows the four API containers running together on DGX Spark with low visible memory pressure, but exact usage depends on model size, sequence length, and warmup state. -- Use the 0.6B Qwen3-TTS variants if you want a lighter multi-service setup. +- The 0.6B Qwen3-TTS variants are lighter but need their own `voices.json` regenerated with that model's embeddings. Do not use them for the streaming/clone containers, which share the default `voices.json` (1.7B-Base). - `--max-seq-len 2048` handles most sentence-style TTS requests. Long-form narration may need `4096`, with more memory required. - Pin services to different GPUs with `NVIDIA_VISIBLE_DEVICES=0`, `NVIDIA_VISIBLE_DEVICES=1`, and so on if your system has more than one GPU. @@ -378,6 +378,11 @@ The first request after container startup can be slower because CUDA graph captu ## Changelog +### v6.13.1 — 2026-10-04 + +- **Fix:** Streaming must use the same model size as the one voices.json was built for (1.7B-Base), otherwise requests fail with a 1024 vs 2048 tensor size mismatch. Streaming compose examples now default to the 1.7B-Base model. +- **Fix:** Out-of-range `chunk_size` values are clamped to 2–24 (e.g. -5/0/1 -> 2, 1000 -> 24) and still return 200; only non-integers return 422. Covered by an offline check of the patched `SpeechRequest` + `_clamp_chunk_size`. + ### v6.13 — 2026-10-04 **Feature: Per-Request `chunk_size` for Streaming** - `/v1/audio/speech` accepts an optional integer `chunk_size` (codec frames per streamed chunk, 12 = 1 s), clamped to 2–24. It applies to the streaming wav/pcm path only; omitted keeps the voice's `voices.json` value (default unchanged). diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml index 89179bc..2679ba0 100644 --- a/docker/docker-compose.yml +++ b/docker/docker-compose.yml @@ -129,7 +129,7 @@ services: ports: - "8023:8000" volumes: - - /path/to/Qwen3-TTS-12Hz-0.6B-Base:/models/Qwen3-TTS:ro + - /path/to/Qwen3-TTS-12Hz-1.7B-Base:/models/Qwen3-TTS:ro - /path/to/faster-qwen3-tts/config:/config:rw - /path/to/active_voices:/voices:ro command: >