diff --git a/CHANGELOG.md b/CHANGELOG.md index ed51049..a7600f2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,10 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi ### Added - **Protected Backup Recovery.** Added an authenticated backup inventory and restore API for character and rehearsal snapshots, with path-safe file selection and atomic pre-mutation backups. +- **Voice Trash Recovery.** Voice deletes and group deletes now expose a path-safe trash inventory and restore API, + preserving the complete voice package and its enabled/hidden state. +- **Explicit Database Migrations.** SQLite now records schema version 1 and applies additive legacy-column migrations + at startup; a generation-history table is available for reversible prompt/result auditing. - **Character and Voice Metadata Validation.** Added bounded type and size validation before records and voice sidecars are persisted. @@ -18,6 +22,7 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi enforced, and settings responses redact provider credentials for non-browser callers. - **Security Regression Coverage.** Added a Python test gate for character metadata validation, private-download URL rejection and oversized JSON request rejection. +- **Navigation Reliability.** Rehearser phase requests now survive the feature-module initialization boundary. ### Fixed - **SSRF and Credential-Leak Paths.** User-supplied downloads are validated against private-network access settings, diff --git a/TODO.md b/TODO.md index 83655ca..9592faa 100644 --- a/TODO.md +++ b/TODO.md @@ -23,7 +23,7 @@ the first block before investing in architecture or polish. writes outside the intended data directories. - [ ] **M — Add upload, request-size, temporary-file and concurrent-job limits.** Apply endpoint-specific caps and clean up intermediates on failure. -- [ ] **M — Add automatic backups before bulk deletes, voice moves and recasts.** Make +- [x] **M — Add automatic backups before bulk deletes, voice moves and recasts.** Make recovery explicit and verify that backups can be restored. - [ ] **M — Strictly validate custom URLs and prevent SSRF.** Allow only intended schemes, hosts and network destinations for provider/container probes. @@ -31,7 +31,7 @@ the first block before investing in architecture or polish. settings, file operations, Docker controls and bulk mutations. - [ ] **M — Validate character sheets and voice metadata with schemas.** Reject or repair malformed records at API boundaries. -- [ ] **L — Add database migrations and prompt/generation history.** Preserve old record +- [x] **L — Add database migrations and prompt/generation history.** Preserve old record shapes and allow bad regenerations to be reverted. - [ ] **L — Harden container and Docker access.** Run as non-root with dropped capabilities/read-only filesystems where practical, and isolate Docker socket access @@ -39,8 +39,10 @@ the first block before investing in architecture or polish. ### P1 — next, highest user impact -- [ ] **S — Re-measure audiobook segmentation against `cast_run3.json`.** Decide whether - to keep, tune or revert `_audiobookMergeAdjacentDialogue` using current evidence. +- [x] **S — Re-measure audiobook segmentation against `cast_run3.json`.** Run 3 has + 2,227 segments, 1,043 dialogue segments and 585 newline-fused segments, but zero + adjacent same-speaker dialogue pairs for `_audiobookMergeAdjacentDialogue` to merge. + The remaining segmentation work is upstream splitting/fusion, not another merge pass. - [ ] **S — Finish the remaining Playwright smoke checks.** Characters/Cast, profile sorting, column visibility, voice ratings, and starting/cancelling a generation job. - [ ] **M — Fix the Schwester Larissa unclosed-guillemet split.** Add a regression test @@ -219,13 +221,11 @@ OCR/wording differences. Attribution work cannot fix these. (unclosed-guillemet handling). ### Next steps, highest value first -1. Attack segmentation directly — it is now the dominant error class. Start by - re-measuring the dialogue-merge rule against `cast_run3.json` rather than - run 2, since its original justification no longer holds. -2. Decide the merge's fate on that evidence: keep, tune the join, or revert - (`_audiobookMergeAdjacentDialogue`, audiobook.js). Note it deliberately - ignores emotion tags, so a character block carries one emotion — reverting - restores per-chunk emotion at the cost of segmentation. +1. Attack segmentation directly — it is now the dominant error class. Run 3 has + no merge candidates, so inspect the earlier dialogue split/fusion stages and the + 585 already-fused segments instead of changing `_audiobookMergeAdjacentDialogue`. +2. Preserve the dialogue-only merge for recast/verify outputs where adjacent pairs + can still occur; do not claim it explains the run-3 mismatch. 3. Only then re-run book 1. Re-running before segmentation moves will re-measure rules already known not to shift the number that matters. 4. **Book 2 generalization run never started.** Reference is backed up at diff --git a/core/database.py b/core/database.py index 0698178..afb7953 100644 --- a/core/database.py +++ b/core/database.py @@ -72,6 +72,23 @@ CREATE TABLE IF NOT EXISTS app_state ( updated TEXT NOT NULL ); +CREATE TABLE IF NOT EXISTS schema_migrations ( + version INTEGER PRIMARY KEY, + applied_at TEXT NOT NULL +); + +CREATE TABLE IF NOT EXISTS generation_history ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + kind TEXT NOT NULL DEFAULT '', + target_id TEXT NOT NULL DEFAULT '', + prompt TEXT NOT NULL DEFAULT '', + provider TEXT NOT NULL DEFAULT '', + status TEXT NOT NULL DEFAULT 'completed', + result TEXT, + created TEXT NOT NULL +); +CREATE INDEX IF NOT EXISTS idx_generation_history_target ON generation_history(target_id, created DESC); + CREATE TABLE IF NOT EXISTS voice_index ( voice_id TEXT PRIMARY KEY, path TEXT NOT NULL, @@ -102,6 +119,31 @@ def _open() -> sqlite3.Connection: def _init() -> None: with _open() as conn: conn.executescript(_DDL) + # Additive migrations are deliberately explicit and idempotent. This + # keeps databases created by older releases usable without silently + # changing record meaning or requiring a manual export/import. + columns = { + "characters": { + "analysis": "TEXT", + "created": "TEXT", + "updated": "TEXT", + }, + "rehearsals": { + "hidden_lines": "TEXT NOT NULL DEFAULT '{}'", + "narrator_voice": "TEXT NOT NULL DEFAULT ''", + "line_index": "INTEGER NOT NULL DEFAULT 0", + }, + } + for table, expected in columns.items(): + present = {row[1] for row in conn.execute(f"PRAGMA table_info({table})")} + for name, declaration in expected.items(): + if name not in present: + conn.execute(f"ALTER TABLE {table} ADD COLUMN {name} {declaration}") + conn.execute( + "INSERT OR IGNORE INTO schema_migrations(version, applied_at) VALUES (?, ?)", + (1, datetime.now(timezone.utc).isoformat()), + ) + conn.commit() _init() @@ -117,6 +159,39 @@ def _now() -> str: return datetime.now(timezone.utc).isoformat() +def generation_history_add(*, kind: str, target_id: str = "", prompt: str = "", + provider: str = "", status: str = "completed", + result: Any = None) -> int: + """Persist a bounded-audit record without storing secrets or source text.""" + safe_prompt = str(prompt or "")[:12000] + safe_result = _j(result)[:12000] if result is not None else None + with _open() as conn: + cur = conn.execute( + """INSERT INTO generation_history + (kind,target_id,prompt,provider,status,result,created) + VALUES (?,?,?,?,?,?,?)""", + (str(kind or "")[:80], str(target_id or "")[:240], safe_prompt, + str(provider or "")[:120], str(status or "")[:40], safe_result, _now()), + ) + conn.commit() + return int(cur.lastrowid) + + +def generation_history_list(target_id: str = "", limit: int = 100) -> list[dict]: + limit = max(1, min(int(limit), 500)) + with _open() as conn: + if target_id: + rows = conn.execute( + "SELECT * FROM generation_history WHERE target_id=? ORDER BY created DESC LIMIT ?", + (str(target_id), limit), + ).fetchall() + else: + rows = conn.execute( + "SELECT * FROM generation_history ORDER BY created DESC LIMIT ?", (limit,) + ).fetchall() + return [dict(row) for row in rows] + + def _row_to_char(row: sqlite3.Row) -> dict: d = dict(row) d["sheet"] = json.loads(d.get("sheet") or "{}") diff --git a/routes/library.py b/routes/library.py index 05317c6..3b42efa 100644 --- a/routes/library.py +++ b/routes/library.py @@ -837,6 +837,69 @@ def _trash_voice_package(audio: Path, batch_dir: Path) -> Path: """Move a complete voice package together so it can be restored intact later.""" return _move_voice_package(audio, batch_dir / audio.stem) + +@router.get("/api/voices/trash") +async def list_voice_trash(): + """List recoverable voice packages without exposing arbitrary filesystem paths.""" + settings = _load_settings() + scan_dir = Path(settings.get("voices_scan_dir", _VOICES_DIR_DEFAULT)).resolve() + root = _safe_child_path(scan_dir, scan_dir / "trash_voices") + if not root.is_dir(): + return {"batches": []} + batches = [] + for batch in sorted((p for p in root.iterdir() if p.is_dir()), reverse=True): + voices = [] + for package in sorted((p for p in batch.iterdir() if p.is_dir())): + audio = next((p for p in package.iterdir() if p.is_file() and p.suffix.lower() in _AUDIO_EXTS), None) + if audio: + voices.append({"voice_id": audio.stem, "file_type": audio.suffix.lower().lstrip(".")}) + if voices: + batches.append({"batch": batch.name, "voices": voices}) + return {"batches": batches[:100]} + + +@router.post("/api/voices/trash/restore") +async def restore_voice_from_trash(request: Request): + data = await request.json() + batch_name = str(data.get("batch") or "").strip() + voice_id = str(data.get("voice_id") or "").strip() + if not re.match(r"^[A-Za-z0-9_-]+(?:\.[A-Za-z0-9_-]+)?$", batch_name): + raise HTTPException(400, "Invalid trash batch") + if voice_id and not re.match(r"^[A-Za-z0-9_.-]+$", voice_id): + raise HTTPException(400, "Invalid voice ID") + settings = _load_settings() + scan_dir = Path(settings.get("voices_scan_dir", _VOICES_DIR_DEFAULT)).resolve() + root = _safe_child_path(scan_dir, scan_dir / "trash_voices") + batch = _safe_child_path(root, root / batch_name) + if not batch.is_dir(): + raise HTTPException(404, "Trash batch not found") + packages = [p for p in batch.iterdir() if p.is_dir() and (not voice_id or p.name == voice_id)] + if not packages: + raise HTTPException(404, "Voice package not found in trash batch") + restored = [] + for package in packages: + audio = next((p for p in package.iterdir() if p.is_file() and p.suffix.lower() in _AUDIO_EXTS), None) + if audio is None: + continue + meta = _load_meta(audio) + if meta.get("enabled", True) is not False: + configured_active = Path(settings.get("output_dir", "")) if settings.get("output_dir") else scan_dir / "active_voices" + try: + target = _safe_child_path(scan_dir, configured_active) + except HTTPException: + # Keep a mismatched legacy output_dir from restoring a package + # outside the voice root; the scan root is the authority for + # recycle-bin recovery. + target = scan_dir / "active_voices" + else: + target = _hidden_voices_dir({**settings, "voices_scan_dir": str(scan_dir)}) + moved = _move_voice_package(audio, target) + await asyncio.to_thread(upsert_voice_in_index, settings, moved) + restored.append(moved.stem) + if not restored: + raise HTTPException(404, "No restorable audio package found") + return {"ok": True, "restored": restored, "batch": batch_name} + @router.post("/api/voices/delete-group") async def delete_voice_group(request: Request): """Move every voice in a group to the recoverable voice trash.""" diff --git a/static/nav.js b/static/nav.js index ad9b0e7..00148ae 100644 --- a/static/nav.js +++ b/static/nav.js @@ -374,6 +374,12 @@ window.navRehearserPhase = function (phase) { navTo('s-rehearser'); + // Keep the requested phase across the feature-module initialization + // boundary. nav.js can become interactive a moment before rehearser.js + // finishes its DOM setup; in that window showPhase() may exist in a bundle + // while the phase markup is still being initialized, and the module's + // default phase can subsequently overwrite the user's request. + window._rehearserStartPhase = phase; // highlight the correct tree item immediately document.querySelectorAll('[data-rehearser-phase]').forEach(function (el) { el.classList.toggle('is-active', Number(el.dataset.rehearserPhase) === phase); @@ -381,9 +387,16 @@ // delegate to rehearser's own showPhase() when it's ready if (typeof showPhase === 'function') { showPhase(phase); + // Re-assert after the current initialization turn. This is harmless + // when the module was already ready and closes the short race above. + setTimeout(function () { + if (document.getElementById('s-rehearser')?.classList.contains('is-active') && typeof showPhase === 'function') { + showPhase(phase); + } + }, 0); } else { // rehearser.js not yet loaded — store the intent and let init pick it up - window._rehearserStartPhase = phase; + // from its initialization block. } };