mARTin-B78
5a7c73ea42
ID3 provenance tags (synthetic speech, generator, version, date, opt-in voice name) on magazine MP3/ZIP/EPUB exports via a stream-copied temp copy; verify endpoint and CLI; AudioSeal tier is a documented stub. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
27 lines
3.6 KiB
Markdown
27 lines
3.6 KiB
Markdown
# Provenance tags and watermarking for exported audio
|
|
|
|
Status: tier 1 implemented (`core/export_provenance.py`); tier 2 (AudioSeal) stubbed, not installed.
|
|
|
|
## Research (checked 2026-10-04)
|
|
|
|
**AudioSeal (facebookresearch/audioseal)**
|
|
- Licence: README states code AND model weights are MIT ("full MIT license, including the license for the model weights", since April 2024). Commercial use allowed. Verdict: licence acceptable.
|
|
- Requirements: Python >= 3.8, PyTorch >= 1.13, OmegaConf, NumPy (einops for streaming). Weights are fetched from the Hugging Face hub on first use (network download, so must be baked in or mounted for an offline box).
|
|
- ARM64/CPU: the host has torch 2.10 (aarch64, cu130), but the app image has no torch (nothing in Dockerfile/requirements). The generator is a small conv/LSTM net at 16 kHz and runs on CPU; no CPU benchmark was found in the docs, so cost per hour of audio is unverified (measure before promising). Docs flag torchaudio backend quirks (needs `soundfile` or torchaudio 2.1).
|
|
- Robustness (paper "Proactive Detection of Voice Cloning with Localized Watermarking", arXiv 2401.17264, and third-party comparisons): detection (is-watermarked) stays high under MP3/AAC (mean AUC about 0.97; 128 kbps AAC detection 1.0), but exact recovery of the 16-bit message under AAC 128k falls to about 0 and newer comparisons report < 60% message accuracy under MP3/AAC. So: only "watermarked yes/no" is dependable after re-encode, not a voice/version id.
|
|
- What is detectable: per-sample (1/16000 s) localized probability that audio was watermarked, plus optional 16-bit message. Needs the AudioSeal detector (python+torch); no public third party checks it. Trained at 16 kHz, resampling needed for 22-44 kHz audio. Adversarial removal is possible.
|
|
|
|
**Tier 1 alternatives (no dependencies)**
|
|
- Provenance tags in the container (ID3v2.3 TXXX/COMM via ffmpeg `-metadata`): `synthetic_speech=true`, `generator`, `generator_version`, `generated_date`, optional `voice` (name only), `comment`. Honest, trivially strippable, survives only while the container is kept.
|
|
- Spoken disclosure ("Dieses Hörbuch wurde synthetisch erzeugt"): not strippable by metadata removal, but needs a TTS call plus a re-encoding concat; NOT implemented yet (see Open decisions).
|
|
|
|
## Design
|
|
|
|
1. Tier 1 (implemented): on export, one ffmpeg stream copy (`-c copy -map_metadata 0 -id3v2_version 3`) of the MP3 into a temp copy with the tags. The stored article audio is never changed. Applies to: single article download, `audio.zip`, and the MP3s inside the EPUB. Query `?provenance=1|0` and `&voice=1` per export; absent -> setting `export_provenance_default` (default true). No user names, paths or secrets are written; voice only when opted in, basename only; ffmpeg errors are not forwarded (they can contain paths).
|
|
2. Verify: `POST /api/export/provenance` (raw audio body, 100 MB cap) returns the tags and `synthetic`; CLI `python -m core.export_provenance file.mp3` (exit 0 if tagged).
|
|
3. Tier 2: `watermark(path)` raises `NotImplementedError`; `watermark_available()` is False. Enabling it later means: optional extra image/venv with torch + audioseal and pre-mounted weights, a feature flag, lazy import inside `watermark()`, and a decision on re-encode (watermark must be applied to PCM before the MP3 encode, i.e. at synthesis time, not on the finished MP3 copy).
|
|
|
|
## Open decisions
|
|
- Spoken disclosure: render once per language via the normal TTS path and concat at export (re-encode) - needs owner go-ahead.
|
|
- Tier 2: needs torch in a sidecar container (not the main image); recommended only if an inaudible mark is really required, given weak message recovery after AAC/MP3.
|