Two engine-level bugs made custom voices unusable in different ways.
Stale speaker embeddings (voice clone):
A .pt embedding is a cache baked from a specific pairing of reference audio
and reference transcript. Re-recording or re-designing a voice replaces those
sources but leaves the old .pt in place, and generate_voices.py pointed at it
unconditionally — so the server kept cloning from an embedding whose audio
tokens no longer matched the transcript stored alongside them. Confirmed on
73 of 155 voices: generation ignored the requested text entirely, emitting
short unrelated filler ("Thank you.") or fragments of the previous reference
transcript. Voices that had never been re-recorded were unaffected, which is
why this looked specific to designed voices.
Now an embedding older than its reference audio/transcript is treated as
invalid and removed so the server recomputes it. Verified: 73 embeddings
regenerated, and voices that previously returned unrelated text now transcribe
back to exactly the requested input.
Unregistered Voice Design voices:
For the VoiceDesign model a voice's identity IS its instruct prompt, but
designed voices were never written into voicedesign_voices.json, so the server
only knew its 8 bundled presets. resolve_voice() silently substituted the first
one, answering requests for a German male character with 'vd_british_male' —
the source of the apparent gender flips between takes.
- generate_voices.py now mirrors designed voices into the VoiceDesign registry
(it runs in the clone container, which is the one with the voice library
mounted; /config is shared with the VoiceDesign container).
- The VoiceDesign server hot-reloads its registry, matching what the clone
server already did, so voices designed while it is running resolve without
a restart.
- An unknown voice no longer becomes a different one: if the request carries
its own instruct that is used, otherwise it is a 404 rather than a silent
substitution.
- A request instruct is now combined with the voice's registered instruct
instead of replacing it. Previously any line carrying an emotion discarded
the character's identity and re-rolled a voice from a few words of
direction, which made a character drift between lines.
- Per-voice temperature/top_p/top_k are honored. Deliberately no seed:
generate_voice_design() takes no seed parameter, so designed voices cannot
be pinned that way — consistency comes from low temperature/top_p.
Verified end to end: three consecutive takes of the same designed voice now
hold 86-91 Hz median F0 (was flipping register between takes), and emotional
lines stay within 84-86 Hz instead of losing the character entirely.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
POST /voice-seed — writes or clears the seed for a named voice in
voices.json directly from the server, enabling the voice creator GUI
to save the chosen seed with one click.
Also updates find_best_seed.py to support --all-voices, per-voice
subdirectories, and the mixed DE/EN test sentence.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Generates WAV samples with different seeds for a given voice so the
user can listen and pick the most natural-sounding one. Uses the
server's hot-reload to switch seeds without restarting.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each API request now seeds PyTorch's CPU and CUDA RNGs before generation.
The seed is auto-derived from the voice name (stable MD5 hash) if not set
in voices.json, so all existing voices get consistent voice character with
zero config changes. Override per-voice with "seed": <int> in voices.json.
Previously the sampling RNG was unseeded, causing pitch/modulation drift
across requests even when speaker embeddings (.pt files) were identical.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add :latest-streaming and :v6-streaming as explicit tags for the streaming
service — same image as :latest/:v6 but semantically distinct so compose
files and version pins are unambiguous. Suggested by forum user.
Update docker/docker-compose.yml streaming service to :latest-streaming.
Update README image table and tag summary.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The :streaming tag was the pre-v5 image with non_streaming_mode=False,
causing the same voice drift bug as voiceclone had before v5. The streaming
service runs identical code (run_server.py -> openai_server.py) so it uses
:latest which has the non_streaming_mode=True fix. The QWEN_TTS_* env vars
were unused since the command overrides them; removed for clarity.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
generate_voices.py previously overwrote voices.json completely on every
start, discarding any manually added temperature/top_k/top_p fields.
Now loads existing voices.json first and merges user-added fields into
freshly generated entries, so per-voice sampling params survive restarts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Re-add --max-seq-len 2048 to voiceclone command now that v6 image supports it.
Update README changelog to accurately describe all v6 changes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The upstream openai_server.py removed the instruct parameter from the
generate_voice_clone call, breaking the old patch context. Regenerated
from a fresh upstream clone with all four changes:
- non_streaming_mode=True in _stream_chunks and create_speech
- per-voice temperature/top_k/top_p in both generation paths
- --max-seq-len argument added to _parse_args
- max_seq_len=args.max_seq_len passed to FasterQwen3TTS.from_pretrained
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The upstream openai_server.py gained --max-seq-len after v5 was built.
Add it to the patch so every build gets explicit sequence length control.
Also add max_seq_len=args.max_seq_len to FasterQwen3TTS.from_pretrained()
so the argument is actually used.
Remove || true from git apply so patch failures fail the build instead
of silently producing a broken image.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
--max-seq-len is not in the current :latest image's openai_server.py (upstream
added it after v5 was built). Remove it from voiceclone command so the
container starts; upstream default (4096) applies. Will be re-added explicitly
once v6 image is built with current upstream.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The VoiceClone server was using non_streaming_mode=False, a mode designed
for streaming LLM->TTS pipelines. In that mode only one text token enters
the model's KV cache during prefill; the rest feed via trailing_text_hiddens
at one step per codec frame. For a 54-word paragraph this provides only ~4s
of text guidance for ~18s of speech — 77% generated with no text conditioning.
Without text context the model free-runs and drifts, sometimes changing gender.
Fix: switch to non_streaming_mode=True (already the default for VoiceDesign
and CustomVoice) so the full text is in the prefill throughout generation.
Also lower default temperature 0.9->0.8 and add top_p=0.9 to reduce
accumulated sampling noise over long runs. Temperature, top_k, and top_p
are now configurable per voice in voices.json.
- patches/openai_server.patch: updated for new upstream HEAD; both streaming
(WAV/PCM) and non-streaming (MP3) paths now use non_streaming_mode=True
- config/run_server.py: align warmup call to non_streaming_mode=True
- README.md: bump image tags v4->v5, add changelog section
- Version: v5
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add run_customvoice_server.py: OpenAI-compatible TTS server for Qwen3-TTS
CustomVoice models with speaker-based voice selection and per-request
language/instruct/max_new_tokens overrides
- Add customvoice_voices.json: voice config for 9 built-in speakers across
English, Chinese, Japanese, and Korean
- Add benchmark_api.py: API benchmarking tool reporting TTFA, total time,
RTF, and speed multiplier across short/medium/long sentences
- Refactor run_voicedesign_server.py: extract _request_generation_params()
helper, add per-request language/instruct/max_new_tokens override support,
and wire DEFAULT_MAX_NEW_TOKENS to --max-seq-len arg
- Add faster-qwen3-tts.code-workspace for VS Code project config
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- generate_voices.py: scan /config/speakers and /voices recursively,
support .ogg and .m4a (M4A auto-converted via ffmpeg), sanitise voice IDs
- auto_transcribe.py: scan both host paths recursively, support all formats,
use parakeet-asr on port 8010
- docker-compose.yml: mount /home/sparky/Projekte/TTS_Voices/speakers as
/voices, add faster-qwen3-tts-voicedesign service on port 8021
- run_voicedesign_server.py: OpenAI-compatible server for VoiceDesign model
- voicedesign_voices.json: 8 British/German VoiceDesign voices
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- run_server.py: warm up CUDA graphs at server startup via lifespan
event so the first real request does not pay the 7-8s graph-
compilation penalty; uses modern lifespan API instead of deprecated
on_event
- generate_voices.py: set chunk_size=4 per voice so streaming clients
receive first audio after ~333ms instead of ~1s
- docker-compose.yml: add --max-seq-len 2048 (halves static KV cache,
reduces VRAM and graph-capture time)
- config/benchmark_api.py: new script to measure TTFA, RTF and speed
against the live API endpoint
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Run Qwen3-TTS with CUDA graph acceleration on NVIDIA DGX Spark
(ARM64 / SM 121 / CUDA 13) as an OpenAI-compatible TTS API server.
- Dockerfile targeting nvidia/cuda:13.0.2-base-ubuntu24.04 with ARM64 cu130 PyTorch wheels
- Patch for max-seq-len support to handle long reference audio without crashes
- OpenWebUI + SillyTavern compatible API endpoints (/v1/models, /v1/audio/voices, /speakers)
- Voice management: auto-generate voices.json from speaker reference audio files
- Auto-transcription helper script for generating reference text from audio
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
