Two engine-level bugs made custom voices unusable in different ways.
Stale speaker embeddings (voice clone):
A .pt embedding is a cache baked from a specific pairing of reference audio
and reference transcript. Re-recording or re-designing a voice replaces those
sources but leaves the old .pt in place, and generate_voices.py pointed at it
unconditionally — so the server kept cloning from an embedding whose audio
tokens no longer matched the transcript stored alongside them. Confirmed on
73 of 155 voices: generation ignored the requested text entirely, emitting
short unrelated filler ("Thank you.") or fragments of the previous reference
transcript. Voices that had never been re-recorded were unaffected, which is
why this looked specific to designed voices.
Now an embedding older than its reference audio/transcript is treated as
invalid and removed so the server recomputes it. Verified: 73 embeddings
regenerated, and voices that previously returned unrelated text now transcribe
back to exactly the requested input.
Unregistered Voice Design voices:
For the VoiceDesign model a voice's identity IS its instruct prompt, but
designed voices were never written into voicedesign_voices.json, so the server
only knew its 8 bundled presets. resolve_voice() silently substituted the first
one, answering requests for a German male character with 'vd_british_male' —
the source of the apparent gender flips between takes.
- generate_voices.py now mirrors designed voices into the VoiceDesign registry
(it runs in the clone container, which is the one with the voice library
mounted; /config is shared with the VoiceDesign container).
- The VoiceDesign server hot-reloads its registry, matching what the clone
server already did, so voices designed while it is running resolve without
a restart.
- An unknown voice no longer becomes a different one: if the request carries
its own instruct that is used, otherwise it is a 404 rather than a silent
substitution.
- A request instruct is now combined with the voice's registered instruct
instead of replacing it. Previously any line carrying an emotion discarded
the character's identity and re-rolled a voice from a few words of
direction, which made a character drift between lines.
- Per-voice temperature/top_p/top_k are honored. Deliberately no seed:
generate_voice_design() takes no seed parameter, so designed voices cannot
be pinned that way — consistency comes from low temperature/top_p.
Verified end to end: three consecutive takes of the same designed voice now
hold 86-91 Hz median F0 (was flipping register between takes), and emotional
lines stay within 84-86 Hz instead of losing the character entirely.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
mARTin-B78
9836261d28