43 lines
3.2 KiB
Markdown
43 lines
3.2 KiB
Markdown
# Vocal isolation for URL/YouTube imports
|
|
|
|
Status: implemented behind a flag, default OFF. Goal: remove music/background from a downloaded
|
|
clip before trimming and cloning (`/api/download-yt` -> `/api/isolate-vocals` -> trim -> `quick-import-voice`).
|
|
|
|
## Findings (read-only inspection, 2026-10-04)
|
|
- App image: `python:3.11-slim-bookworm`, `requirements.txt` has no torch/onnx/demucs. The live app
|
|
container has no `torch` (`import torch` fails). It is `read_only: true`, `cap_drop: ALL`, 8 GB mem
|
|
limit, no GPU reservation. Source dirs are bind-mounted read-only.
|
|
- GPU/torch live in other service containers (tts-faster-qwen3-*, aligner, ...), not reusable: different
|
|
images, no shared API for source separation.
|
|
- There is no VAD, denoise or separation anywhere in the app today.
|
|
|
|
## Options
|
|
| Option | Footprint | Verdict |
|
|
|---|---|---|
|
|
| Demucs `htdemucs` (MIT code; weights MIT, ~80 MB, from dl.fbaipublicfiles.com via torch hub) in the main image | torch + torchaudio on ARM64 ~ 2-3 GB image growth | rejected: heavy, unconditional |
|
|
| Demucs in a separate sidecar container | no growth of main image, GPU optional | best long-term, but needs a new image, compose service and network contract |
|
|
| **Optional external Demucs CLI**, called as a subprocess, only when the flag is on and the binary + weights exist | zero deps in the main image; the owner picks where Demucs lives (venv on a bind mount, or a derived image) | **chosen** |
|
|
| Lighter: RNNoise/DeepFilterNet/spectral gating | small, but removes noise only, not music/singing | does not meet the goal |
|
|
|
|
Why the CLI: `demucs --two-stems=vocals -n htdemucs` is the stable public interface; no import-time cost,
|
|
no behaviour change when absent. ARM64: `torch`/`torchaudio` publish aarch64 wheels (CUDA wheels exist for
|
|
DGX Spark / GB10; CPU wheels work everywhere), `demucs` itself is pure Python (`pip install demucs`).
|
|
A sidecar can later expose the same CLI via `VOCAL_ISOLATION_CMD` (e.g. `docker exec ...` wrapper) without code changes.
|
|
|
|
## Safety rules
|
|
- `VOCAL_ISOLATION=1` required; otherwise the feature reports unavailable and the API answers 503.
|
|
- Available only if the command resolves AND the htdemucs weights file (`955717e8-*.th`) is already in
|
|
`$TORCH_HOME/hub/checkpoints` (default `~/.cache/torch`). The app never downloads weights: the owner
|
|
installs them once (documented in Manual.md). This prevents Demucs' silent first-run download.
|
|
- Input bounded to `VOCAL_ISOLATION_MAX_SECONDS` (default 900), runtime to `VOCAL_ISOLATION_TIMEOUT`
|
|
(default 900 s), one job at a time (409 when busy), client disconnect cancels and kills the process.
|
|
- Work happens in a private temp dir that is always removed; the result is converted to 24 kHz mono WAV
|
|
and registered as a new temp id. The original id is untouched (compare / undo in the UI).
|
|
- Subprocess started with an argument list (no shell); the path comes from the registry, never from the client.
|
|
|
|
## Contract
|
|
- `GET /api/version` gains `features.vocal_isolation: bool`.
|
|
- `POST /api/isolate-vocals {id}` -> `{id, duration}`. Errors: 400 bad id, 404 unknown id, 413 too long,
|
|
409 busy, 503 disabled/unavailable, 504 timeout, 500 failed (message without internals).
|
|
- Offline behaviour: fully local; no network access.
|