- Fixed-window CUDA-graph codec decode for streaming (TTS_GRAPH_DECODE=0 disables)
- Per-request chunk_size (2-24) on /v1/audio/speech
- prebuffer_ms (0-10000) to hold back audio before sending
- Stop generation on client disconnect
- Recommend --max-seq-len 2048 for the streaming container
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
OpenAI-compatible clients may send a route alias such as FR, FR_F or
FR_M as the voice. Map it (case-insensitively) to a configured
<LANG>_<M/F>_* speaker instead of falling back to the default English
voice; a bare language code prefers a female voice.
Regenerated patches/openai_server.patch from source so it applies
cleanly to both the pinned upstream commit and upstream main.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A single generate call was capped at --max-seq-len codec tokens, so text
long enough to need more audio than that got cut off wherever generation
happened to be. Long input is now split into paragraph/sentence-sized
segments and generated back-to-back, so --max-seq-len can be sized for
VRAM budget instead of the longest expected input.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The clone server passed instruct=voice_cfg.get("instruct") at both call sites,
so a voice's delivery was fixed in voices.json for its whole lifetime. For an
audiobook that means a character can be consistent or expressive, not both:
VoiceDesign takes a per-request instruct but has no seed, so its identity is
re-sampled on every call, while VoiceClone is deterministic but had no way to
direct a line.
SpeechRequest now accepts instruct, merged identity-first via _merge_instruct()
— the voice's own description followed by the line's direction — so directing
one line cannot re-roll the character. Same rule run_voicedesign_server.py got
in v6.8.
This is safe on the clone path specifically because extract_embeddings.py writes
full ICL prompts (ref codec tokens + ref_text), not x-vector-only ones;
model.py warns that instruct is unreliable in the latter mode.
The change lives in patches/openai_server.patch, regenerated against the
pristine upstream blob and verified to apply cleanly. README documents the
request field and adds a section on the cast-in-VoiceDesign,
narrate-from-VoiceClone workflow.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Each API request now seeds PyTorch's CPU and CUDA RNGs before generation.
The seed is auto-derived from the voice name (stable MD5 hash) if not set
in voices.json, so all existing voices get consistent voice character with
zero config changes. Override per-voice with "seed": <int> in voices.json.
Previously the sampling RNG was unseeded, causing pitch/modulation drift
across requests even when speaker embeddings (.pt files) were identical.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The upstream openai_server.py removed the instruct parameter from the
generate_voice_clone call, breaking the old patch context. Regenerated
from a fresh upstream clone with all four changes:
- non_streaming_mode=True in _stream_chunks and create_speech
- per-voice temperature/top_k/top_p in both generation paths
- --max-seq-len argument added to _parse_args
- max_seq_len=args.max_seq_len passed to FasterQwen3TTS.from_pretrained
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The upstream openai_server.py gained --max-seq-len after v5 was built.
Add it to the patch so every build gets explicit sequence length control.
Also add max_seq_len=args.max_seq_len to FasterQwen3TTS.from_pretrained()
so the argument is actually used.
Remove || true from git apply so patch failures fail the build instead
of silently producing a broken image.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The VoiceClone server was using non_streaming_mode=False, a mode designed
for streaming LLM->TTS pipelines. In that mode only one text token enters
the model's KV cache during prefill; the rest feed via trailing_text_hiddens
at one step per codec frame. For a 54-word paragraph this provides only ~4s
of text guidance for ~18s of speech — 77% generated with no text conditioning.
Without text context the model free-runs and drifts, sometimes changing gender.
Fix: switch to non_streaming_mode=True (already the default for VoiceDesign
and CustomVoice) so the full text is in the prefill throughout generation.
Also lower default temperature 0.9->0.8 and add top_p=0.9 to reduce
accumulated sampling noise over long runs. Temperature, top_k, and top_p
are now configurable per voice in voices.json.
- patches/openai_server.patch: updated for new upstream HEAD; both streaming
(WAV/PCM) and non-streaming (MP3) paths now use non_streaming_mode=True
- config/run_server.py: align warmup call to non_streaming_mode=True
- README.md: bump image tags v4->v5, add changelog section
- Version: v5
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Run Qwen3-TTS with CUDA graph acceleration on NVIDIA DGX Spark
(ARM64 / SM 121 / CUDA 13) as an OpenAI-compatible TTS API server.
- Dockerfile targeting nvidia/cuda:13.0.2-base-ubuntu24.04 with ARM64 cu130 PyTorch wheels
- Patch for max-seq-len support to handle long reference audio without crashes
- OpenWebUI + SillyTavern compatible API endpoints (/v1/models, /v1/audio/voices, /speakers)
- Voice management: auto-generate voices.json from speaker reference audio files
- Auto-transcription helper script for generating reference text from audio
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
