diff --git a/PROJECT_MAP.md b/PROJECT_MAP.md index a8f3c9a..f7956b1 100644 --- a/PROJECT_MAP.md +++ b/PROJECT_MAP.md @@ -80,6 +80,10 @@ which is the writable volume. ocr-baidu-unlimited-ocr primary, KDL Frontier Parser nano fallback). - No `npm run qa` aggregate command yet; Python tests run via `unittest`, browser tests via `playwright test`. +- The app has no authentication for the browser UI and no per-user data + isolation — single-tenant/LAN-trust only (one global settings/database/voice + library, no accounts). See `architecture/multi-tenant-and-identity.md` and + TODO.md's "Deferred — public multi-user readiness" section. ## Next logical step diff --git a/TODO.md b/TODO.md index a4ff319..56ad9fe 100644 --- a/TODO.md +++ b/TODO.md @@ -115,6 +115,30 @@ architecture change: the larger Audiobook/Rehearser/style split. sudo systemctl enable --now nfs-kernel-server ``` +## Deferred — public multi-user readiness (cross-app initiative) + +Not scheduled; blocked on a decision the user wants to make once, shared across +all self-hosted apps, not per-app. See `architecture/multi-tenant-and-identity.md` +for the full research findings and design sketch. Do not start without a fresh +planning pass — this is a multi-week, cross-repo effort. + +- [ ] Design and stand up a shared identity service across all self-hosted apps + (interconnected user profile, not per-app accounts), building on the existing + OurFlashcards GoTrue + Postgres + PostgREST stack on the NUC rather than a new + auth system per app. +- [ ] Once shared identity exists: add user_id/owner_id columns to every table in + core/database.py, move voices/output/reader_library/audio-cache to per-user + paths, replace the single global settings.json with per-user settings, replace + the single shared external_api_key with per-user API keys. +- [ ] Replace the global TTS/STT/image concurrency semaphores (routes/tts.py, + routes/stt.py, routes/conversation.py) with per-user fair-share admission + control. +- [ ] Required consent attestation + audit trail for voice cloning (decided: + required). Checkbox at voice upload time, logged with timestamp/account/IP; + add a user_id column to generation_history so misuse is traceable per account. +- [ ] Add a reverse-proxy/TLS edge layer (CORS, CSRF, CSP/HSTS) for internet + exposure — nothing in this repo handles this today. + ### P2 — after behavior and safety stabilize - [x] **M — Add structured logs with request/job IDs.** Report provider, endpoint, diff --git a/architecture/multi-tenant-and-identity.md b/architecture/multi-tenant-and-identity.md new file mode 100644 index 0000000..790decd --- /dev/null +++ b/architecture/multi-tenant-and-identity.md @@ -0,0 +1,174 @@ +# Multi-tenant and identity — deferred, cross-app initiative + +> This is a deferred, cross-app initiative — see `TODO.md`'s "Deferred — public +> multi-user readiness" section. Do not start implementation without a fresh +> planning pass: it depends on a shared identity service that does not exist +> yet in any app, and touches almost every data path in this repo. + +## Why this exists + +The user asked what it would take to make this app ready for public, +multi-user use. Three research passes over this codebase established that it +is currently single-tenant and LAN-trusted end to end, with no accounts of any +kind. The user then redirected the auth-backend decision specifically: rather +than building a login system scoped to this app alone, they want **one +shared, interconnected login/user-profile system across all their +self-hosted apps** (this app, OurFlashcards, and others still to come), so a +person's profile is one thing, not N per-app copies. That is a decision for +the whole app portfolio, not this repo, so it is recorded here rather than +implemented. + +One policy decision was made and should carry forward unchanged into +whatever implementation eventually happens: voice cloning must require a +**consent attestation with an audit trail** once accounts exist (see below). + +## Current state (as of 2026-09-07) + +### 1. No multi-tenancy exists anywhere + +Everything hangs off one process-wide `CONFIG_DIR` +(`core/constants.py`, default `/home/app/.config/tts-voice-creator`): + +- `settings.json` — one global settings dict, loaded/saved by + `core/config.py`'s `_load_settings()`/`_save_settings()`, cached in a single + module-level variable. No per-user key anywhere in this path. +- `tts_creator.db` — a real SQLite database (`core/database.py`, WAL mode, + explicitly commented as "designed for easy migration to + PostgreSQL/Supabase later"). Tables: `characters`, `rehearsals`, + `reader_scripts`, `app_state`, `schema_migrations`, `generation_history`, + `background_jobs`, `voice_index`. **None have a `user_id`/`owner`/`tenant` + column.** `characters` is keyed by `book`, `rehearsals` by `title` — nothing + scopes a row to a caller. +- Voice library — one global `voices_scan_dir` setting (default `/voices`), + read throughout `routes/library.py`; active/output voices resolved via + `core/voice.py`'s `_hidden_voices_dir`/`_active_voices_dir`. +- Magazines/books — flat filesystem tree at `CONFIG_DIR/reader_library//` + (`routes/reader.py`, `routes/magazine.py`); `id` is a document id, not a + user id. +- Generated audio — `CONFIG_DIR/line_audio_cache/`, + `CONFIG_DIR/audiobook_exports/`, `CONFIG_DIR/background-artifacts/` — all + global, unscoped. +- `background_jobs` (`core/database.py`) has no owner column either — + `background_job_list()` returns every job process-wide. + +### 2. The browser UI has no authentication at all + +`ApiKeyGateMiddleware` (`server.py:273-330`) is the only inbound auth +mechanism, and it only applies to non-browser callers: + +- A request is treated as "browser, same-origin" (and let through with **no + key check whatsoever**) if its `Sec-Fetch-Site`/`Origin`/`Referer` matches + `Host` (`server.py` around line 304-308). +- Everything else (scripts, external API callers, `/mcp`) must send + `X-API-Key` or `Authorization: Bearer ` matching one single shared + secret, `external_api_key`, lazily generated and stored in the same global + settings blob (`core/config.py`, `_ensure_external_api_key`; + regeneration via `routes/settings.py`). +- There is no session, cookie, or login system anywhere. This is a "gate the + robots, trust the browser" design that only makes sense when every browser + on the network is already trusted — i.e. today's LAN deployment. It is the + hard blocker for any public exposure, independent of every other gap below. + +### 3. Concurrency is one global pool, not per-user + +- TTS: `asyncio.Semaphore(2)` (`routes/tts.py:60`, + `_TTS_GENERATION_SLOTS`) — 2 concurrent TTS jobs for the **entire service**, + 30s wait then HTTP 429. +- STT: `asyncio.Semaphore(1)` (`routes/stt.py:29`) — 1 concurrent + transcription, service-wide. +- Image/portrait generation: `threading.BoundedSemaphore(2)` + (`routes/conversation.py:67`). +- No dedicated vision-OCR concurrency slot; OCR rides the general LLM/provider + path. +- `MutationRateLimitMiddleware` (`server.py:96-140`) is already keyed by + `(client_ip, is_destructive)` — this is the one piece of infrastructure + already shaped like a per-caller limiter and would be the natural place to + swap the key to `(user_id, is_destructive)` once accounts exist. +- **No per-caller quota or fairness mechanism exists today.** One user + submitting many requests can exhaust the global slots and 429 every other + user. TODO.md's existing P0 resource-limit entries describe this as + "per user" in wording but the implementation is a single process-global + semaphore — that gap needs closing for real multi-user hosting. + +### 4. No voice-cloning consent or misuse control exists + +- The only guardrail anywhere is a documentation sentence + (`docs/USER_MANUAL.md:284`): "Always review licensing and consent before + using a voice outside personal testing." Nothing in the app enforces, + prompts for, or records this. +- No consent-capture UI, no click-through attestation, no per-generation + audit trail linking a clone/generation to who requested it. +- `generation_history` (`core/database.py`) has columns + `id, kind, target_id, prompt, provider, status, result, created` — **no + identity column** — and deliberately discards the actual prompt + (`safe_prompt = ""`) and strips prompt/text-like keys from `result` for + data minimization. Good for secret-leakage avoidance, but it means today's + history can't answer "who generated this." + +### 5. No edge security layer + +No `CORSMiddleware`, no CSRF token, no CSP/HSTS/`X-Frame-Options` anywhere in +`server.py`. SSRF protection (`core/validation.py`'s `_validate_http_url`) +exists but most call sites pass `allow_private=True` since they intentionally +target LAN TTS/OCR backends — correct for today's trust model, would need +tightening/scoping by caller for internet exposure. No TLS termination or +reverse proxy exists in this repo; port `7890` is published directly. + +Container-level hardening is solid and should be **kept unchanged**: the app +runs as a non-root user, `read_only: true`, `cap_drop: ALL`, +`no-new-privileges`, with a locked-down `docker-socket-proxy` sidecar +(`docker-compose.yml`). None of that needs to change for multi-tenancy — it's +orthogonal to the identity/data-isolation gap. + +### 6. Prior art already exists — reuse it + +The NUC (`correspondence-nuc`, 192.168.178.19) already runs a working, +self-hosted Supabase-style identity stack for OurFlashcards: +`OurFlashcards-Auth` (Supabase GoTrue), `OurFlashcards-PostgresSQL`, +`OurFlashcards-DataAPI` (PostgREST), fronted by nginx gateways. ` +core/database.py` in this repo is already explicitly written to ease a +future Postgres/Supabase migration. This is the natural foundation for a +**shared cross-app identity service** — a separate GoTrue+Postgres instance +(or a shared one, scoped correctly) that every self-hosted app, including +this one, validates sessions/JWTs against — rather than inventing a new +per-app auth system. + +## Target design sketch (not yet decided in detail — for the next planning pass) + +Once a shared identity service exists across the app portfolio: + +- **Data model**: add `user_id`/`owner_id` columns to every table in + `core/database.py` (`characters`, `rehearsals`, `reader_scripts`, + `generation_history`, `background_jobs`, `voice_index`), with migrations + that backfill existing rows to a single "legacy/admin" account so nothing + is lost. +- **Storage layout**: per-user subdirectories under voices/output/ + reader_library/audio-cache instead of the single global paths in + `core/constants.py`/`core/voice.py`. +- **Settings**: a per-user settings row instead of one global blob in + `core/config.py`; per-user API keys replacing the single shared + `external_api_key`. +- **Concurrency**: per-user fair-share admission control replacing the + global semaphores in `routes/tts.py`, `routes/stt.py`, + `routes/conversation.py`; extend `MutationRateLimitMiddleware`'s existing + `(client_ip, is_destructive)` key to include the account. +- **Voice-cloning consent (decided)**: a required attestation checkbox at + voice-upload time ("I have the right to clone this voice"), stored with a + timestamp, account id, and IP alongside the voice sample; add a `user_id` + column to `generation_history` so a specific account's generations are + traceable and the account can be suspended if misused. This is the one + piece of policy the user has already settled — implement it as designed + here, don't re-litigate it. +- **Edge layer**: a reverse proxy (matching whatever pattern the NUC's + nginx gateways already use) for TLS termination, HSTS, CSP, and real + CORS/CSRF posture appropriate for cookie/JWT-based sessions. + +## Explicitly still open (not decided, don't assume an answer) + +- Which apps join the shared identity service and in what order. +- Whether existing OurFlashcards accounts become the same accounts, or a + separate but interconnected account store. +- Reverse-proxy technology/domain/TLS certificate approach for public + exposure. +- Any billing, quota tiers, or admin/abuse-response tooling beyond the + consent attestation and audit trail described above.