mARTin-B78/tts-voice-creator-clone-and-design-2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Convert conversation audio to 16kHz for STT (fixes whisperx alignment crash)

Browse Source 
STT models (Whisper, WhisperX VAD, wav2vec2 alignment) all expect 16kHz.
Sending 24kHz caused whisperx's VAD to miss speech segments, leaving
alignment with None inputs → 'NoneType has no attribute to' crash.

Added _to_wav_16k() and use it in conversation/turn and transcribe-bytes
endpoints. Health check probe also uses 16kHz silence for consistency.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
mARTin-B78 2026-05-28 09:13:12 +02:00
parent d16be50cc9
commit 27a949f1ff
1 changed files with 14 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
server.py
View File

@ -906,6 +906,15 @@ def _to_wav_24k(src: Path) -> Path:
return out
def _to_wav_16k(src: Path) -> Path:
"""16kHz mono WAV — required by Whisper/WhisperX VAD and wav2vec2 alignment."""
out = TEMP_DIR / f"{src.stem}_16k.wav"
seg = AudioSegment.from_file(str(src))
seg = seg.set_frame_rate(16000).set_channels(1).set_sample_width(2)
seg.export(str(out), format="wav")
return out
def _trim(src: Path, start_s: float, end_s: float) -> Path:
seg = AudioSegment.from_file(str(src))
trimmed = seg[int(start_s * 1000):int(end_s * 1000)]
@ -1903,9 +1912,8 @@ def _stt_backend_api_key(settings: dict, backend: str) -> str:
return settings.get("whisper_api_key", "").strip()
def _make_minimal_wav(duration_ms: int = 500) -> bytes:
"""Minimal WAV: mono 16-bit 16kHz silence of given duration."""
sample_rate = 16000
def _make_minimal_wav(duration_ms: int = 500, sample_rate: int = 16000) -> bytes:
"""Minimal WAV: mono 16-bit silence of given duration at given sample rate."""
num_frames = sample_rate * duration_ms // 1000
data = b"\x00\x00" * num_frames
header = struct.pack(
@ -2131,7 +2139,7 @@ async def transcribe_bytes(
with tmp.open("wb") as f:
_copy_limited(file.file, f, _MAX_UPLOAD_BYTES)
if suffix != ".wav":
wav_tmp = _to_wav_24k(tmp)
wav_tmp = _to_wav_16k(tmp)
settings = _load_settings()
stt_backend = _clean_stt_backend(backend)
text, used_backend = await asyncio.to_thread(_transcribe_audio, wav_tmp, settings, stt_backend)
@ -5377,7 +5385,8 @@ async def conversation_turn(
try:
with tmp.open("wb") as f:
_copy_limited(audio.file, f, _MAX_UPLOAD_BYTES)
wav_tmp = tmp if suffix == ".wav" else _to_wav_24k(tmp)
# STT models (Whisper, WhisperX, wav2vec2 alignment) all expect 16kHz
wav_tmp = _to_wav_16k(tmp)
except Exception as e:
tmp.unlink(missing_ok=True)
raise HTTPException(400, f"Audio upload failed: {e}")