mARTin-B78
98405f681d
The VoiceClone server was using non_streaming_mode=False, a mode designed for streaming LLM->TTS pipelines. In that mode only one text token enters the model's KV cache during prefill; the rest feed via trailing_text_hiddens at one step per codec frame. For a 54-word paragraph this provides only ~4s of text guidance for ~18s of speech — 77% generated with no text conditioning. Without text context the model free-runs and drifts, sometimes changing gender. Fix: switch to non_streaming_mode=True (already the default for VoiceDesign and CustomVoice) so the full text is in the prefill throughout generation. Also lower default temperature 0.9->0.8 and add top_p=0.9 to reduce accumulated sampling noise over long runs. Temperature, top_k, and top_p are now configurable per voice in voices.json. - patches/openai_server.patch: updated for new upstream HEAD; both streaming (WAV/PCM) and non-streaming (MP3) paths now use non_streaming_mode=True - config/run_server.py: align warmup call to non_streaming_mode=True - README.md: bump image tags v4->v5, add changelog section - Version: v5 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
28 lines
1.3 KiB
Diff
28 lines
1.3 KiB
Diff
diff --git a/examples/openai_server.py b/examples/openai_server.py
|
|
index 61047ea..2b0c8bb 100644
|
|
--- a/examples/openai_server.py
|
|
+++ b/examples/openai_server.py
|
|
@@ -187,7 +187,10 @@ async def _stream_chunks(voice_cfg: dict, text: str) -> AsyncGenerator[bytes, No
|
|
ref_text=voice_cfg.get("ref_text", ""),
|
|
chunk_size=voice_cfg.get("chunk_size", 12),
|
|
instruct=voice_cfg.get("instruct"),
|
|
- non_streaming_mode=False,
|
|
+ non_streaming_mode=True,
|
|
+ temperature=voice_cfg.get("temperature", 0.8),
|
|
+ top_k=voice_cfg.get("top_k", 50),
|
|
+ top_p=voice_cfg.get("top_p", 0.9),
|
|
):
|
|
q.put(chunk)
|
|
except Exception as exc:
|
|
@@ -252,6 +255,10 @@ async def create_speech(req: SpeechRequest):
|
|
ref_audio=voice_cfg["ref_audio"],
|
|
ref_text=voice_cfg.get("ref_text", ""),
|
|
instruct=voice_cfg.get("instruct"),
|
|
+ non_streaming_mode=True,
|
|
+ temperature=voice_cfg.get("temperature", 0.8),
|
|
+ top_k=voice_cfg.get("top_k", 50),
|
|
+ top_p=voice_cfg.get("top_p", 0.9),
|
|
)
|
|
|
|
audio_arrays, sr = await loop.run_in_executor(None, _generate)
|