tts-voice-creator-clone-and-design-2/architecture/vocal-isolation.md

Vocal isolation for URL/YouTube imports

Status: implemented behind a flag, default OFF. Goal: remove music/background from a downloaded clip before trimming and cloning (/api/download-yt -> /api/isolate-vocals -> trim -> quick-import-voice).

Findings (read-only inspection, 2026-10-04)

• App image: python:3.11-slim-bookworm, requirements.txt has no torch/onnx/demucs. The live app container has no torch (import torch fails). It is read_only: true, cap_drop: ALL, 8 GB mem limit, no GPU reservation. Source dirs are bind-mounted read-only.
• GPU/torch live in other service containers (tts-faster-qwen3-*, aligner, ...), not reusable: different images, no shared API for source separation.
• There is no VAD, denoise or separation anywhere in the app today.

Options

Option	Footprint	Verdict
Demucs htdemucs (MIT code; weights MIT, ~80 MB, from dl.fbaipublicfiles.com via torch hub) in the main image	torch + torchaudio on ARM64 ~ 2-3 GB image growth	rejected: heavy, unconditional
Demucs in a separate sidecar container	no growth of main image, GPU optional	best long-term, but needs a new image, compose service and network contract
Optional external Demucs CLI, called as a subprocess, only when the flag is on and the binary + weights exist	zero deps in the main image; the owner picks where Demucs lives (venv on a bind mount, or a derived image)	chosen
Lighter: RNNoise/DeepFilterNet/spectral gating	small, but removes noise only, not music/singing	does not meet the goal

Why the CLI: demucs --two-stems=vocals -n htdemucs is the stable public interface; no import-time cost, no behaviour change when absent. ARM64: torch/torchaudio publish aarch64 wheels (CUDA wheels exist for DGX Spark / GB10; CPU wheels work everywhere), demucs itself is pure Python (pip install demucs). A sidecar can later expose the same CLI via VOCAL_ISOLATION_CMD (e.g. docker exec ... wrapper) without code changes.

Safety rules

• VOCAL_ISOLATION=1 required; otherwise the feature reports unavailable and the API answers 503.
• Available only if the command resolves AND the htdemucs weights file (955717e8-*.th) is already in $TORCH_HOME/hub/checkpoints (default ~/.cache/torch). The app never downloads weights: the owner installs them once (documented in Manual.md). This prevents Demucs' silent first-run download.
• Input bounded to VOCAL_ISOLATION_MAX_SECONDS (default 900), runtime to VOCAL_ISOLATION_TIMEOUT (default 900 s), one job at a time (409 when busy), client disconnect cancels and kills the process.
• Work happens in a private temp dir that is always removed; the result is converted to 24 kHz mono WAV and registered as a new temp id. The original id is untouched (compare / undo in the UI).
• Subprocess started with an argument list (no shell); the path comes from the registry, never from the client.

Contract

• GET /api/version gains features.vocal_isolation: bool.
• POST /api/isolate-vocals {id} -> {id, duration}. Errors: 400 bad id, 404 unknown id, 413 too long, 409 busy, 503 disabled/unavailable, 504 timeout, 500 failed (message without internals).
• Offline behaviour: fully local; no network access.