A fresh build pulled torch 2.14.0 from the cu130 index, which has not
been validated with CUDA graph capture on SM121. Pin torch 2.12.1,
torchvision 0.27.1 and torchaudio 2.11.0 (the versions in the running
images) and document the v6.11 image tags.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
OpenAI-compatible clients may send a route alias such as FR, FR_F or
FR_M as the voice. Map it (case-insensitively) to a configured
<LANG>_<M/F>_* speaker instead of falling back to the default English
voice; a bare language code prefers a female voice.
Regenerated patches/openai_server.patch from source so it applies
cleanly to both the pinned upstream commit and upstream main.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fresh builds cloned upstream main, which since v0.4.0 requires
Transformers 5 (qwen-tts-hf). qwen-asr then downgraded transformers to
4.57.6, breaking Qwen3TTSModel.from_pretrained at startup.
Pin upstream to 7cdef7e (v0.2.6, as in the released images) and pin
qwen-tts==0.1.1 / qwen-asr==0.0.6. FASTER_QWEN3_TTS_REF now accepts a
commit SHA, tag or branch.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A single generate call was capped at --max-seq-len codec tokens, so text
long enough to need more audio than that got cut off wherever generation
happened to be. Long input is now split into paragraph/sentence-sized
segments and generated back-to-back, so --max-seq-len can be sized for
VRAM budget instead of the longest expected input.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The clone server passed instruct=voice_cfg.get("instruct") at both call sites,
so a voice's delivery was fixed in voices.json for its whole lifetime. For an
audiobook that means a character can be consistent or expressive, not both:
VoiceDesign takes a per-request instruct but has no seed, so its identity is
re-sampled on every call, while VoiceClone is deterministic but had no way to
direct a line.
SpeechRequest now accepts instruct, merged identity-first via _merge_instruct()
— the voice's own description followed by the line's direction — so directing
one line cannot re-roll the character. Same rule run_voicedesign_server.py got
in v6.8.
This is safe on the clone path specifically because extract_embeddings.py writes
full ICL prompts (ref codec tokens + ref_text), not x-vector-only ones;
model.py warns that instruct is unreliable in the latter mode.
The change lives in patches/openai_server.patch, regenerated against the
pristine upstream blob and verified to apply cleanly. README documents the
request field and adds a section on the cast-in-VoiceDesign,
narrate-from-VoiceClone workflow.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
config/voicedesign_voices.json is generated on every container start from the
voice library's designed voices, so it carries the user's own character design
prompts and made the working tree permanently dirty. It is now gitignored.
The 8 bundled vd_* presets were only ever stored in that file, so untracking it
alone would leave a fresh clone with no built-in voices at all. They now live in
config/voicedesign_voices.presets.json, which is tracked:
- generate_voices.py seeds the registry from the presets file when no vd_*
entries survive in the generated one (i.e. on a fresh checkout).
- run_voicedesign_server.py starts from the presets file when the registry does
not exist yet, instead of crashing on the missing path. Hot-reload picks up
the real registry as soon as the voice-clone container writes it.
Also documents the v6.8 engine fixes from 9836261 in the README changelog, which
that commit did not touch.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add :latest-streaming and :v6-streaming as explicit tags for the streaming
service — same image as :latest/:v6 but semantically distinct so compose
files and version pins are unambiguous. Suggested by forum user.
Update docker/docker-compose.yml streaming service to :latest-streaming.
Update README image table and tag summary.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Re-add --max-seq-len 2048 to voiceclone command now that v6 image supports it.
Update README changelog to accurately describe all v6 changes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The VoiceClone server was using non_streaming_mode=False, a mode designed
for streaming LLM->TTS pipelines. In that mode only one text token enters
the model's KV cache during prefill; the rest feed via trailing_text_hiddens
at one step per codec frame. For a 54-word paragraph this provides only ~4s
of text guidance for ~18s of speech — 77% generated with no text conditioning.
Without text context the model free-runs and drifts, sometimes changing gender.
Fix: switch to non_streaming_mode=True (already the default for VoiceDesign
and CustomVoice) so the full text is in the prefill throughout generation.
Also lower default temperature 0.9->0.8 and add top_p=0.9 to reduce
accumulated sampling noise over long runs. Temperature, top_k, and top_p
are now configurable per voice in voices.json.
- patches/openai_server.patch: updated for new upstream HEAD; both streaming
(WAV/PCM) and non-streaming (MP3) paths now use non_streaming_mode=True
- config/run_server.py: align warmup call to non_streaming_mode=True
- README.md: bump image tags v4->v5, add changelog section
- Version: v5
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Run Qwen3-TTS with CUDA graph acceleration on NVIDIA DGX Spark
(ARM64 / SM 121 / CUDA 13) as an OpenAI-compatible TTS API server.
- Dockerfile targeting nvidia/cuda:13.0.2-base-ubuntu24.04 with ARM64 cu130 PyTorch wheels
- Patch for max-seq-len support to handle long reference audio without crashes
- OpenWebUI + SillyTavern compatible API endpoints (/v1/models, /v1/audio/voices, /speakers)
- Voice management: auto-generate voices.json from speaker reference audio files
- Auto-transcription helper script for generating reference text from audio
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
