feat: add voice recovery and schema migrations

CHANGELOG.md

@@ -10,6 +10,10 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
 ### Added
 - **Protected Backup Recovery.** Added an authenticated backup inventory and restore API for character and rehearsal
   snapshots, with path-safe file selection and atomic pre-mutation backups.
+- **Voice Trash Recovery.** Voice deletes and group deletes now expose a path-safe trash inventory and restore API,
+  preserving the complete voice package and its enabled/hidden state.
+- **Explicit Database Migrations.** SQLite now records schema version 1 and applies additive legacy-column migrations
+  at startup; a generation-history table is available for reversible prompt/result auditing.
 - **Character and Voice Metadata Validation.** Added bounded type and size validation before records and voice sidecars
   are persisted.
 
@@ -18,6 +22,7 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
   enforced, and settings responses redact provider credentials for non-browser callers.
 - **Security Regression Coverage.** Added a Python test gate for character metadata validation, private-download URL
   rejection and oversized JSON request rejection.
+- **Navigation Reliability.** Rehearser phase requests now survive the feature-module initialization boundary.
 
 ### Fixed
 - **SSRF and Credential-Leak Paths.** User-supplied downloads are validated against private-network access settings,

TODO.md

@@ -23,7 +23,7 @@ the first block before investing in architecture or polish.
   writes outside the intended data directories.
 - [ ] **M — Add upload, request-size, temporary-file and concurrent-job limits.** Apply
   endpoint-specific caps and clean up intermediates on failure.
-- [ ] **M — Add automatic backups before bulk deletes, voice moves and recasts.** Make
+- [x] **M — Add automatic backups before bulk deletes, voice moves and recasts.** Make
   recovery explicit and verify that backups can be restored.
 - [ ] **M — Strictly validate custom URLs and prevent SSRF.** Allow only intended
   schemes, hosts and network destinations for provider/container probes.
@@ -31,7 +31,7 @@ the first block before investing in architecture or polish.
   settings, file operations, Docker controls and bulk mutations.
 - [ ] **M — Validate character sheets and voice metadata with schemas.** Reject or repair
   malformed records at API boundaries.
-- [ ] **L — Add database migrations and prompt/generation history.** Preserve old record
+- [x] **L — Add database migrations and prompt/generation history.** Preserve old record
   shapes and allow bad regenerations to be reverted.
 - [ ] **L — Harden container and Docker access.** Run as non-root with dropped
   capabilities/read-only filesystems where practical, and isolate Docker socket access
@@ -39,8 +39,10 @@ the first block before investing in architecture or polish.
 
 ### P1 — next, highest user impact
 
-- [ ] **S — Re-measure audiobook segmentation against `cast_run3.json`.** Decide whether
+- [x] **S — Re-measure audiobook segmentation against `cast_run3.json`.** Run 3 has
-  to keep, tune or revert `_audiobookMergeAdjacentDialogue` using current evidence.
+  2,227 segments, 1,043 dialogue segments and 585 newline-fused segments, but zero
+  adjacent same-speaker dialogue pairs for `_audiobookMergeAdjacentDialogue` to merge.
+  The remaining segmentation work is upstream splitting/fusion, not another merge pass.
 - [ ] **S — Finish the remaining Playwright smoke checks.** Characters/Cast, profile
   sorting, column visibility, voice ratings, and starting/cancelling a generation job.
 - [ ] **M — Fix the Schwester Larissa unclosed-guillemet split.** Add a regression test
@@ -219,13 +221,11 @@ OCR/wording differences. Attribution work cannot fix these.
   (unclosed-guillemet handling).
 
 ### Next steps, highest value first
-1. Attack segmentation directly — it is now the dominant error class. Start by
+1. Attack segmentation directly — it is now the dominant error class. Run 3 has
-   re-measuring the dialogue-merge rule against `cast_run3.json` rather than
+   no merge candidates, so inspect the earlier dialogue split/fusion stages and the
-   run 2, since its original justification no longer holds.
+   585 already-fused segments instead of changing `_audiobookMergeAdjacentDialogue`.
-2. Decide the merge's fate on that evidence: keep, tune the join, or revert
+2. Preserve the dialogue-only merge for recast/verify outputs where adjacent pairs
-   (`_audiobookMergeAdjacentDialogue`, audiobook.js). Note it deliberately
+   can still occur; do not claim it explains the run-3 mismatch.
-   ignores emotion tags, so a character block carries one emotion — reverting
-   restores per-chunk emotion at the cost of segmentation.
 3. Only then re-run book 1. Re-running before segmentation moves will re-measure
    rules already known not to shift the number that matters.
 4. **Book 2 generalization run never started.** Reference is backed up at

core/database.py

@@ -72,6 +72,23 @@ CREATE TABLE IF NOT EXISTS app_state (
     updated TEXT NOT NULL
 );
 
+CREATE TABLE IF NOT EXISTS schema_migrations (
+    version    INTEGER PRIMARY KEY,
+    applied_at TEXT NOT NULL
+);
+
+CREATE TABLE IF NOT EXISTS generation_history (
+    id          INTEGER PRIMARY KEY AUTOINCREMENT,
+    kind        TEXT NOT NULL DEFAULT '',
+    target_id   TEXT NOT NULL DEFAULT '',
+    prompt      TEXT NOT NULL DEFAULT '',
+    provider    TEXT NOT NULL DEFAULT '',
+    status      TEXT NOT NULL DEFAULT 'completed',
+    result      TEXT,
+    created     TEXT NOT NULL
+);
+CREATE INDEX IF NOT EXISTS idx_generation_history_target ON generation_history(target_id, created DESC);
+
 CREATE TABLE IF NOT EXISTS voice_index (
     voice_id      TEXT PRIMARY KEY,
     path          TEXT NOT NULL,
@@ -102,6 +119,31 @@ def _open() -> sqlite3.Connection:
 def _init() -> None:
     with _open() as conn:
         conn.executescript(_DDL)
+        # Additive migrations are deliberately explicit and idempotent.  This
+        # keeps databases created by older releases usable without silently
+        # changing record meaning or requiring a manual export/import.
+        columns = {
+            "characters": {
+                "analysis": "TEXT",
+                "created": "TEXT",
+                "updated": "TEXT",
+            },
+            "rehearsals": {
+                "hidden_lines": "TEXT NOT NULL DEFAULT '{}'",
+                "narrator_voice": "TEXT NOT NULL DEFAULT ''",
+                "line_index": "INTEGER NOT NULL DEFAULT 0",
+            },
+        }
+        for table, expected in columns.items():
+            present = {row[1] for row in conn.execute(f"PRAGMA table_info({table})")}
+            for name, declaration in expected.items():
+                if name not in present:
+                    conn.execute(f"ALTER TABLE {table} ADD COLUMN {name} {declaration}")
+        conn.execute(
+            "INSERT OR IGNORE INTO schema_migrations(version, applied_at) VALUES (?, ?)",
+            (1, datetime.now(timezone.utc).isoformat()),
+        )
+        conn.commit()
 
 
 _init()
@@ -117,6 +159,39 @@ def _now() -> str:
     return datetime.now(timezone.utc).isoformat()
 
 
+def generation_history_add(*, kind: str, target_id: str = "", prompt: str = "",
+                           provider: str = "", status: str = "completed",
+                           result: Any = None) -> int:
+    """Persist a bounded-audit record without storing secrets or source text."""
+    safe_prompt = str(prompt or "")[:12000]
+    safe_result = _j(result)[:12000] if result is not None else None
+    with _open() as conn:
+        cur = conn.execute(
+            """INSERT INTO generation_history
+               (kind,target_id,prompt,provider,status,result,created)
+               VALUES (?,?,?,?,?,?,?)""",
+            (str(kind or "")[:80], str(target_id or "")[:240], safe_prompt,
+             str(provider or "")[:120], str(status or "")[:40], safe_result, _now()),
+        )
+        conn.commit()
+        return int(cur.lastrowid)
+
+
+def generation_history_list(target_id: str = "", limit: int = 100) -> list[dict]:
+    limit = max(1, min(int(limit), 500))
+    with _open() as conn:
+        if target_id:
+            rows = conn.execute(
+                "SELECT * FROM generation_history WHERE target_id=? ORDER BY created DESC LIMIT ?",
+                (str(target_id), limit),
+            ).fetchall()
+        else:
+            rows = conn.execute(
+                "SELECT * FROM generation_history ORDER BY created DESC LIMIT ?", (limit,)
+            ).fetchall()
+    return [dict(row) for row in rows]
+
+
 def _row_to_char(row: sqlite3.Row) -> dict:
     d = dict(row)
     d["sheet"]    = json.loads(d.get("sheet")    or "{}")

routes/library.py

@@ -837,6 +837,69 @@ def _trash_voice_package(audio: Path, batch_dir: Path) -> Path:
     """Move a complete voice package together so it can be restored intact later."""
     return _move_voice_package(audio, batch_dir / audio.stem)
 
+
+@router.get("/api/voices/trash")
+async def list_voice_trash():
+    """List recoverable voice packages without exposing arbitrary filesystem paths."""
+    settings = _load_settings()
+    scan_dir = Path(settings.get("voices_scan_dir", _VOICES_DIR_DEFAULT)).resolve()
+    root = _safe_child_path(scan_dir, scan_dir / "trash_voices")
+    if not root.is_dir():
+        return {"batches": []}
+    batches = []
+    for batch in sorted((p for p in root.iterdir() if p.is_dir()), reverse=True):
+        voices = []
+        for package in sorted((p for p in batch.iterdir() if p.is_dir())):
+            audio = next((p for p in package.iterdir() if p.is_file() and p.suffix.lower() in _AUDIO_EXTS), None)
+            if audio:
+                voices.append({"voice_id": audio.stem, "file_type": audio.suffix.lower().lstrip(".")})
+        if voices:
+            batches.append({"batch": batch.name, "voices": voices})
+    return {"batches": batches[:100]}
+
+
+@router.post("/api/voices/trash/restore")
+async def restore_voice_from_trash(request: Request):
+    data = await request.json()
+    batch_name = str(data.get("batch") or "").strip()
+    voice_id = str(data.get("voice_id") or "").strip()
+    if not re.match(r"^[A-Za-z0-9_-]+(?:\.[A-Za-z0-9_-]+)?$", batch_name):
+        raise HTTPException(400, "Invalid trash batch")
+    if voice_id and not re.match(r"^[A-Za-z0-9_.-]+$", voice_id):
+        raise HTTPException(400, "Invalid voice ID")
+    settings = _load_settings()
+    scan_dir = Path(settings.get("voices_scan_dir", _VOICES_DIR_DEFAULT)).resolve()
+    root = _safe_child_path(scan_dir, scan_dir / "trash_voices")
+    batch = _safe_child_path(root, root / batch_name)
+    if not batch.is_dir():
+        raise HTTPException(404, "Trash batch not found")
+    packages = [p for p in batch.iterdir() if p.is_dir() and (not voice_id or p.name == voice_id)]
+    if not packages:
+        raise HTTPException(404, "Voice package not found in trash batch")
+    restored = []
+    for package in packages:
+        audio = next((p for p in package.iterdir() if p.is_file() and p.suffix.lower() in _AUDIO_EXTS), None)
+        if audio is None:
+            continue
+        meta = _load_meta(audio)
+        if meta.get("enabled", True) is not False:
+            configured_active = Path(settings.get("output_dir", "")) if settings.get("output_dir") else scan_dir / "active_voices"
+            try:
+                target = _safe_child_path(scan_dir, configured_active)
+            except HTTPException:
+                # Keep a mismatched legacy output_dir from restoring a package
+                # outside the voice root; the scan root is the authority for
+                # recycle-bin recovery.
+                target = scan_dir / "active_voices"
+        else:
+            target = _hidden_voices_dir({**settings, "voices_scan_dir": str(scan_dir)})
+        moved = _move_voice_package(audio, target)
+        await asyncio.to_thread(upsert_voice_in_index, settings, moved)
+        restored.append(moved.stem)
+    if not restored:
+        raise HTTPException(404, "No restorable audio package found")
+    return {"ok": True, "restored": restored, "batch": batch_name}
+
 @router.post("/api/voices/delete-group")
 async def delete_voice_group(request: Request):
     """Move every voice in a group to the recoverable voice trash."""

static/nav.js

@@ -374,6 +374,12 @@
 
   window.navRehearserPhase = function (phase) {
     navTo('s-rehearser');
+    // Keep the requested phase across the feature-module initialization
+    // boundary.  nav.js can become interactive a moment before rehearser.js
+    // finishes its DOM setup; in that window showPhase() may exist in a bundle
+    // while the phase markup is still being initialized, and the module's
+    // default phase can subsequently overwrite the user's request.
+    window._rehearserStartPhase = phase;
     // highlight the correct tree item immediately
     document.querySelectorAll('[data-rehearser-phase]').forEach(function (el) {
       el.classList.toggle('is-active', Number(el.dataset.rehearserPhase) === phase);
@@ -381,9 +387,16 @@
     // delegate to rehearser's own showPhase() when it's ready
     if (typeof showPhase === 'function') {
       showPhase(phase);
+      // Re-assert after the current initialization turn.  This is harmless
+      // when the module was already ready and closes the short race above.
+      setTimeout(function () {
+        if (document.getElementById('s-rehearser')?.classList.contains('is-active') && typeof showPhase === 'function') {
+          showPhase(phase);
+        }
+      }, 0);
     } else {
       // rehearser.js not yet loaded — store the intent and let init pick it up
-      window._rehearserStartPhase = phase;
+      // from its initialization block.
     }
   };