diff --git a/.gitignore b/.gitignore index 98ea295..74f2bdf 100644 --- a/.gitignore +++ b/.gitignore @@ -37,6 +37,7 @@ APP_ERKLΓ„RUNG.md config/settings.json config/voice_design_presets.json config/tts_routes.json +config/reader_library/ logs/*.log logs/*.log.* diff --git a/CHANGELOG.md b/CHANGELOG.md index cc3e351..0816232 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,41 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) Β· versioned wi ## [Unreleased] +## [1.7.0] β€” 2026-06-21 + +### Added +- **Seed Finder** β€” in **My Voices**, each voice's inspector has a 🎲 **Seed Finder** panel: generate a sample for a range of seeds (the TTS engine produces a slightly different take per seed), play each result, and click **β˜… Use seed N** to pin your favourite β€” saved immediately to the TTS server's `voices.json`. New `/api/tts-voice-seed` proxy, module `static/js/seed-finder.js`. +- **Voice library β€” Precompute embeddings**: a **Precompute** button warms all active voices so the TTS engine builds and caches each voice's speaker embedding (`.pt`) ahead of time, making the *first* playback of a voice instant (better Time-To-First-Audio) instead of paying the one-time analysis cost on first use. Runs with bounded concurrency, progress, and cancel. (The faster-qwen3-tts engine already prefers a cached `.pt` and auto-creates it from the reference wav when missing β€” the wav stays the source of truth; this just pre-warms the cache.) +- **Character sheets** β€” a **Character sheets** button in both **Read Aloud** and the **Script Rehearser** uses your configured LLM to extract actor-facing, RPG-style profiles for every character: Archetype, Physical Stats (metric-only), Alignment & Ethos, Core Attributes (highest/lowest), Trained Skills, Signature Inventory, Dark Secret / Fatal Flaw, Conflict Style, and Win Condition. Deduced details are marked with `*`, characters are grouped **main vs supporting**, and each sheet cites its **sources** (page number from the PDF + a short verbatim quote). Long texts are processed in chunks and merged per character (fields filled in, inventory/sources de-duplicated). Results render as scannable cards in an overlay with **Copy as Markdown**, and are cached so re-opening is instant. New endpoint `POST /api/character-sheets`, module `static/js/character-sheets.js`. +- **Book β†’ multi-speaker audiobook** β€” a **Cast as audiobook** button in Read Aloud turns a novel into a cast-able script. An LLM scans the current scope (selection / page range / whole book, chunked with a running character roster so the same speaker keeps one name throughout) and attributes every segment to a **Narrator** or a **character**, with a per-line **emotion**. New endpoint `POST /api/attribute-dialogue` and module `static/js/audiobook.js`; reuses the rehearser's casting, per-line tone, and synthesis. Failed chunks fall back to narration so a book always casts; progress is shown and cancellable. + - **Live casting view**: while attributing, a **wide, non-blocking, minimisable panel** shows a scrolling feed of each line with its assigned speaker (and emotion) plus a **character roster that fills up** with per-character line counts β€” instead of a bare modal progress bar. You can **minimise it and keep using the app** (any tab), then re-open to watch progress; when it finishes it parks as a **"βœ“ Review & cast"** panel rather than auto-popping, so it waits for you if you wandered off. + - **Calmer messaging**: a passage the LLM can't attribute is no longer shown as a red "attribution failed: Error" toast β€” it's a quiet "read by the narrator" note in the feed, with a single neutral summary ("N passages had no detected dialogue") in the review step. + - **Editable preview**: before handing off, a review overlay lists every segment with an editable **speaker** (autocompletes from detected characters) and **emotion** so mis-attributions are fixed in seconds; "Open in Rehearser" applies the edits and lands you at the Cast phase. + - **Audiobook export** (rehearser): an **Audiobook** button synthesises every cast line as MP3 (bounded concurrency, progress, cancel) and downloads **one MP3 per chapter** (split on Chapter/Kapitel/Part/Prologue… headings or act/scene markers), or a single file when no chapters are detected. + - **Saved as a reopenable rehearsal**: handing the cast off to the Rehearser now also **saves it to the Script Rehearser library (Bibliothek)** automatically, so the attributed script + cast + per-line emotions persist β€” reopen it anytime to change speakers/voices/lines and synthesise or export the audiobook. +- **Read Aloud** β€” a new sidebar tab that turns the app into a text-to-speech document reader. Import a **PDF** (rendered to its real page layout via pdf.js) or a **.txt / .md** file, pick any voice + backend and a **reading speed** (0.5×–2Γ—), then press play: the document is read sentence-by-sentence while the **word being spoken is highlighted** in place (overlay box on the PDF page, inline highlight in text mode), with the view auto-scrolling to follow. Click any word to jump there. Reuses the rehearser's word-timing + pdf.js loader and the existing `/api/tts-preview` pipeline β€” no backend changes. New files `static/sections/s-reader.html` and `static/js/reader.js`. +- **Read Aloud β€” synthesis-state overlay**: every sentence is colour-coded by state β€” **red** (not synthesised), **yellow** (synthesising), **green** (ready/cached), **blue** (currently reading) β€” shown as a translucent overlay on the PDF page and as a tint in text mode, with a legend. +- **Read Aloud β€” PDF zoom controls**: Fit width, Fit height, Two-page spread, and zoom in/out with a live percentage. Word geometry is stored scale-independently so zoom re-renders instantly and highlights stay aligned; fit modes track window resizes. +- **Read Aloud β€” resume**: the last reading position is remembered per document, so re-opening the same file resumes where you left off. +- **Read Aloud β€” book library**: **Save to library** stores the original PDF/text together with its **synthesised audio** and reading position in the browser (IndexedDB). A "My books" shelf lists saved documents with audio- and read-progress bars; reopen one to continue right where you left off with the already-synthesised pages intact β€” handy for working through long books. Reading position auto-saves on pause / stop / leaving the tab. +- **Read Aloud β€” MP3 storage & export**: audio is now synthesised and stored as **MP3** (far smaller than WAV, so books fit comfortably in the browser library). An **Export MP3** control downloads the synthesised audio as **one file per page** (sections combined) or **one file per sentence**, with meaningful filenames like `Title - p01 - 03.mp3` / `Title - p01.mp3`. Any not-yet-synthesised sentences in scope are rendered first. +- **Read Aloud β€” voice consistency**: addresses the slight timbre/prosody drift you hear when each sentence is generated separately. A **"Voice consistency"** selector synthesises in larger continuous chunks β€” *per sentence* (responsive), *per paragraph* (steadier), or *per page* (steadiest) β€” so a whole passage is one generation. Optional **Seed** and **Temperature** inputs pin the generation (forwarded to backends that support them, with graceful fallback), **Normalise loudness** evens out volume between chunks on playback, and a backend hint flags cloned/zero-shot engines that re-sample per request and suggests remedies. Chunk mode + seed/temperature/normalise are saved with library books. +- **Read Aloud β€” synthesise ahead**: a **Synthesise** button pre-renders audio for gap-free reading, scoped to **all**, a **page range** (PDF), or a **click-selected sentence range**. Select mode is a guided, persistent step flow β€” click a start sentence (it pulses as the anchor), then the end; the mode stays active with step hints so you can keep refining, and you leave it with the **Done** button or **Esc**. Synthesis runs with bounded concurrency, shows progress, and can be cancelled; the synthesis-state colours fill in green as each sentence completes. + +### Changed +- **Audiobook casting β€” smarter speaker attribution**: the LLM prompt now reasons about **conversational turn-taking** (in a two-person exchange speakers alternate, so untagged lines are attributed by context rather than dumped as β€œUnknown”), and each passage is given the **recent dialogue** from the previous one so a conversation continues correctly across passage boundaries. The deterministic fallback (used when the LLM is unavailable) also got a conservative two-person turn-taking fill and a stop-list that rejects common German non-name words (Sofort, Stimme, Frage, PlΓΆtzlich…), so it no longer invents bogus characters. +- **Character sheets β€” self-filling across the book**: sheets now build up progressively β€” each passage receives the **sheet-so-far** (with which fields each character still needs) and the model fills gaps and refines instead of starting from scratch, so details accumulate as more of the book is read. +- **Read Aloud β€” library now lives on the server (syncs across devices)**: saved books, their synthesised audio, and reading position were previously stored only in the browser (IndexedDB), so a book saved on the laptop never appeared on the desktop. The library now persists under the server's config volume (`reader_library//` with `meta.json`, the source document, and per-unit MP3s) via new `/api/reader/docs…` endpoints. Any device pointed at the same server sees the same "My books" shelf; opening a book is instant and its audio streams **per chunk on demand** (nothing is bulk-downloaded), and saves stay incremental (only new chunks upload). +- **Language switcher** β€” replaced the sidebar language dropdown with a **flag toggle** next to the "Voice Creator" headline (click to switch interface language). Added German strings for **Read Aloud** ("Vorlesen") and its UI. + +### Performance +- **Read Aloud β€” memory & smoothness for long books**: decoded audio (uncompressed PCM) is now kept only for a small window around the playhead and re-decoded from the cached MP3 on demand; off-screen PDF page canvases are released and re-rastered on return β€” together these bound memory on big books (previously both grew unbounded and could crash long sessions). The next chunk is **pre-decoded** during playback for gapless transitions, transport actions no longer scan every unit (single tracked "reading" index), PDF sentences/units are built **incrementally per page** (no end-of-parse spike), and library **saves are incremental** β€” only newly-synthesised chunks are written (a separate per-unit audio store), instead of rewriting the whole book each save. + ### Fixed +- **Audiobook casting β€” German (and other) quote styles not recognised**: dialogue marked with German guillemets `»…«` / `β€žβ€¦β€œ` / `›…‹`, French `«…»`, curly `β€œβ€¦β€`, CJK `γ€Œβ€¦γ€`, or em-dash speech was treated as narration, so books like German novels cast everything to the narrator. The LLM prompt now explicitly handles all these styles (with guillemets called out), passages with **no quotation marks skip the LLM entirely** (so genuine narration isn't shown as a failure), PDF line-break **hyphenation is mended** (`Schwer- tes` β†’ `Schwertes`) for clean speech, and if the LLM call fails on a passage that *does* contain quotes, a **deterministic fallback** splits out the dialogue **and attributes speakers from speech tags** (`»…«, sagte Riskan` β†’ Riskan; pronouns rejected) so the book stays castable with real names even when the LLM is offline. +- **Read Aloud β€” auto-scroll**: while reading, the view now scrolls only the document pane instead of the whole window, so the currently-spoken line no longer slides up under the app header out of view. +- **Read Aloud β€” backend dropdown stuck on "Checking…"**: the reader's TTS-backend select is now populated by the shared backend refresh and fetched on demand when the section opens, so it fills reliably even if backends finish loading after you're already on the tab. +- **Read Aloud β€” large PDFs froze the page** ("this page is not responding"): the page-parse loop now yields to the browser periodically (with a "Reading PDF… page x / n" indicator), and per-sentence status overlays are created lazily per page instead of all at once. A 60-page book now imports with a max main-thread stall of ~40 ms (was multi-second), creating only the visible pages' overlays. - **Chunked TTS β€” "Failed to fetch" on long text**: `splitTextIntoChunks` only split on sentence terminators (`.!?`), so newline-delimited text (e.g. German bullet lists or care-plan notes) was never split β€” the full page was sent as one request, causing a TCP timeout that the browser surfaced as "Failed to fetch". Fixed by processing each line individually before applying the sentence regex. Also moved `generation.js` from deferred batch E into the main feature batch so `generateChunkedTts` is always defined before the user can click Generate. --- diff --git a/VERSION b/VERSION index dc1e644..9dbb0c0 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.6.0 +1.7.0 \ No newline at end of file diff --git a/routes/conversation.py b/routes/conversation.py index 9bde3f2..ef343f4 100644 --- a/routes/conversation.py +++ b/routes/conversation.py @@ -435,6 +435,227 @@ def _extract_json_block(text: str) -> str: return "" +@router.post("/api/character-sheets") +async def character_sheets(request: Request): + """Extract actor-facing RPG-style character sheets from a passage. + + Body: {text, known_characters:[...], language, llm_url, model} + The text may contain "[p.N]" page markers so the model can cite sources. + Returns: {sheets:[{name, aliases, archetype, physical, alignment, + attribute_high, attribute_low, skills, inventory:[...], secret, + conflict_style, win_condition, tier:"main"|"supporting", + sources:[{page, quote}]}], characters:[names]} + Deduced (not explicit) values are marked with a trailing " *". + """ + data = await request.json() + text: str = (data.get("text") or "").strip() + known: list = data.get("known_characters") or [] + existing: str = (data.get("existing") or "").strip() # partial sheets so far (progressive fill) + language: str = (data.get("language") or "").strip() + _settings = _load_settings() + llm_url: str = (data.get("llm_url") or _settings.get("llm_url") or "http://localhost:11434/v1").rstrip("/") + model: str = (data.get("model") or _settings.get("llm_model") or "").strip() + if not text: + raise HTTPException(400, "No text provided") + + lang_hint = f" The text language is {language}; write the sheet in that language." if language else "" + system = ( + "You are an expert dramaturge and tabletop RPG game master building character sheets that FILL UP " + "as a book is read passage by passage. Read this passage and extract playable, action-oriented " + "character sheets β€” sheets an actor can use to immediately know how to PLAY the character.\n" + "PROGRESSIVE FILLING: you may be given the sheets built so far. For returning characters, ADD any " + "NEW detail this passage reveals and refine vague fields; do not contradict solid earlier facts or " + "blank out a field you cannot improve. Add brand-new characters as they appear. Leave a field empty " + "if the book genuinely hasn't shown it yet (a later passage can fill it). Extrapolate from dialogue " + "and actions when reasonable, and mark any deduced value with a trailing ' *'.\n" + f"{lang_hint}\n" + "For each character output these fields:\n" + "- name, aliases\n" + "- archetype: a two-word role summary (e.g. 'Ruthless Scholar')\n" + "- physical: age, build, vocal quality, posture. Use ONLY the metric system for any height/weight.\n" + "- alignment: strict moral code + the one line they will never cross\n" + "- attribute_high / attribute_low: highest and lowest natural attribute (Charisma, Intelligence, Wisdom, Agility…)\n" + "- skills: what they are demonstrably good at in the story\n" + "- inventory: 1-3 defining items/props/clothing (array of short strings)\n" + "- secret: dark secret or fatal flaw\n" + "- conflict_style: fight, flight, or manipulate β€” how they act when cornered\n" + "- win_condition: the specific event that would make them feel they have won\n" + "- tier: 'main' or 'supporting'\n" + "- sources: array of {page, quote} β€” the page number from the nearest [p.N] marker and a short " + "verbatim quote that supports the sheet (1-3 entries). Use null page if unknown.\n" + "Reuse the EXACT names from the known-characters list for returning characters. " + "Respond with STRICT JSON only:\n" + '{"sheets":[{"name":"","aliases":"","archetype":"","physical":"","alignment":"",' + '"attribute_high":"","attribute_low":"","skills":"","inventory":[],"secret":"",' + '"conflict_style":"","win_condition":"","tier":"main","sources":[{"page":1,"quote":""}]}]}\n/no-think' + ) + user = ( + ("Known characters so far: " + ", ".join(str(n) for n in known) + "\n\n" if known else "") + + ("Sheets so far (fill gaps / refine; keep solid facts):\n" + existing + "\n\n" if existing else "") + + "Passage:\n" + text + ) + payload: dict = { + "messages": [ + {"role": "system", "content": system}, + {"role": "user", "content": user}, + ], + "temperature": 0.4, + "max_tokens": 3500, + } + if model: + payload["model"] = model + try: + resp = requests.post( + f"{llm_url}/chat/completions", json=payload, + headers={"Authorization": "Bearer no-key"}, timeout=240, + ) + resp.raise_for_status() + raw = resp.json()["choices"][0]["message"]["content"].strip() + except Exception as e: + raise HTTPException(502, f"LLM character-sheet generation failed: {e}") + + content = re.sub(r".*?", "", raw, flags=re.DOTALL).strip() or raw + sheets = [] + for cand in (content, _extract_json_block(content)): + if not cand: + continue + try: + parsed = json.loads(cand) + if isinstance(parsed, dict) and isinstance(parsed.get("sheets"), list): + sheets = parsed["sheets"] + break + except Exception: + continue + clean, names = [], [] + for s in sheets: + if not isinstance(s, dict): + continue + name = str(s.get("name") or "").strip() + if not name: + continue + inv = s.get("inventory") + if isinstance(inv, str): + inv = [x.strip() for x in inv.split(",") if x.strip()] + elif not isinstance(inv, list): + inv = [] + src = s.get("sources") if isinstance(s.get("sources"), list) else [] + s.update({"name": name, "inventory": inv[:3], + "tier": "main" if str(s.get("tier") or "").lower().startswith("main") else "supporting", + "sources": src[:3]}) + clean.append(s) + names.append(name) + return {"sheets": clean, "characters": names} + + +@router.post("/api/attribute-dialogue") +async def attribute_dialogue(request: Request): + """Split a prose passage into attributed segments for a multi-voice audiobook. + + Body: {text, known_characters:[...], language, llm_url, model} + Returns: {segments:[{speaker, type:"narration"|"dialogue", text, emotion}], characters:[names]} + The frontend calls this per chunk, passing the running character roster so the + same speaker keeps the same name across the whole book. + """ + data = await request.json() + text: str = (data.get("text") or "").strip() + known: list = data.get("known_characters") or [] + recent: str = (data.get("recent") or "").strip() # last few attributed lines, for continuity + language: str = (data.get("language") or "").strip() + _settings = _load_settings() + llm_url: str = (data.get("llm_url") or _settings.get("llm_url") or "http://localhost:11434/v1").rstrip("/") + model: str = (data.get("model") or _settings.get("llm_model") or "").strip() + if not text: + raise HTTPException(400, "No text provided") + + lang_hint = f" The text language is {language}; keep names and wording in that language." if language else "" + system = ( + "You attribute dialogue in prose fiction for a multi-voice audiobook. " + "Split the passage into consecutive segments in reading order. For each segment output:\n" + "- speaker: 'Narrator' for narration/description, or the character's name for spoken dialogue. " + "Use 'Unknown' ONLY as an absolute last resort.\n" + "- type: 'narration' or 'dialogue'\n" + "- text: the verbatim spoken words for dialogue (WITHOUT the surrounding quotation marks), or the verbatim prose for narration\n" + "- emotion: for dialogue, one or two words (e.g. neutral, angry, sad, excited, whisper, tender); '' for narration\n" + f"{lang_hint}\n" + "QUOTATION STYLES β€” books mark speech in many ways; treat ALL of these as spoken dialogue:\n" + " English straight \"...\" and curly β€œ...”; German Β»...Β« (guillemets pointing inward) and β€ž...β€œ; " + "French Β«...Β» (pointing outward); single β€˜...’; CJK γ€Œ...」 γ€Ž...』; and em-dash speech where a line " + "starts with β€” or – (Spanish/French/Polish style).\n" + "German guillemets are the MOST IMPORTANT to detect: Β»Was schaust du dir an?Β« is a spoken line.\n" + "ATTRIBUTING THE SPEAKER (this is the hard, important part β€” be decisive):\n" + "1. If there is a dialogue tag ('sagte Riskan', 'fragte sie', 'Peter said'), use it. Resolve pronouns " + "(er/sie/he/she) to the actual name from nearby context.\n" + "2. UNTAGGED lines: use **conversational turn-taking**. In a two-person exchange the speaker ALTERNATES " + "every line β€” if Riskan just spoke, the next untagged quote is the other person, then back to Riskan, and so on.\n" + "3. Use the scene context, action beats around a quote (the person doing the action usually speaks), the " + "'Recent dialogue' below (continue the same conversation/alternation across the passage boundary), and the " + "known-characters list. Reuse the EXACT known names.\n" + "4. Only output 'Unknown' if the speaker is genuinely indeterminable even after applying turn-taking and " + "context β€” this should be rare. Prefer the most likely named character over 'Unknown'.\n" + "RULES:\n" + "- Put dialogue tags and action beats in a NARRATION segment, never inside the dialogue text.\n" + "- If a quote is interrupted by a tag (Β»Die PauseΒ«, sagte Peter, Β»ist vorbei.Β«), stitch the spoken parts " + "into ONE dialogue segment ('Die Pause ist vorbei.') with the tag as a separate narration segment.\n" + "- Strip the quotation marks/guillemets from dialogue text. Keep every word otherwise, in order.\n" + "Respond with STRICT JSON only:\n" + '{"segments":[{"speaker":"Narrator","type":"narration","text":"...","emotion":""}]}\n/no-think' + ) + user = ( + ("Known characters so far: " + ", ".join(str(n) for n in known) + "\n\n" if known else "") + + ("Recent dialogue (the immediately preceding lines β€” continue the same conversation/turn-taking):\n" + recent + "\n\n" if recent else "") + + "Passage:\n" + text + ) + payload: dict = { + "messages": [ + {"role": "system", "content": system}, + {"role": "user", "content": user}, + ], + "temperature": 0.2, + "max_tokens": min(4096, max(700, len(text) // 2 + 800)), + } + if model: + payload["model"] = model + try: + resp = requests.post( + f"{llm_url}/chat/completions", json=payload, + headers={"Authorization": "Bearer no-key"}, timeout=180, + ) + resp.raise_for_status() + raw = resp.json()["choices"][0]["message"]["content"].strip() + except Exception as e: + raise HTTPException(502, f"LLM attribution failed: {e}") + + content = re.sub(r".*?", "", raw, flags=re.DOTALL).strip() or raw + segments = [] + for cand in (content, _extract_json_block(content)): + if not cand: + continue + try: + parsed = json.loads(cand) + if isinstance(parsed, dict) and isinstance(parsed.get("segments"), list): + segments = parsed["segments"] + break + except Exception: + continue + # Normalise + collect speaker roster + clean, chars = [], [] + for seg in segments: + if not isinstance(seg, dict): + continue + t = str(seg.get("text") or "").strip() + if not t: + continue + sp = str(seg.get("speaker") or "Narrator").strip() or "Narrator" + typ = "dialogue" if str(seg.get("type") or "").lower().startswith("dial") else "narration" + if typ == "narration": + sp = "Narrator" + emo = str(seg.get("emotion") or "").strip() + clean.append({"speaker": sp, "type": typ, "text": t, "emotion": emo}) + if typ == "dialogue" and sp.lower() != "narrator" and sp not in chars: + chars.append(sp) + return {"segments": clean, "characters": chars} + + # ── Audio effects ───────────────────────────────────────────────────────────── def _apply_audio_effects(audio_bytes: bytes, effects: list) -> bytes: diff --git a/routes/reader.py b/routes/reader.py new file mode 100644 index 0000000..3b7833e --- /dev/null +++ b/routes/reader.py @@ -0,0 +1,172 @@ +"""Server-side Read Aloud library so saved books sync across devices. + +Stores each document under the writable config volume: + /reader_library// + meta.json metadata (title, kind, idx, voice, counts, …) + source.pdf | source.txt the original document (for re-rendering) + audio/.mp3 per-unit synthesised audio (incremental) + +Endpoints are deliberately small (file I/O) and mirror the previous IndexedDB +shape so the frontend swap is mechanical. +""" +import json +import shutil +import uuid +from pathlib import Path + +from fastapi import APIRouter, HTTPException, Request +from fastapi.responses import FileResponse, Response + +from core.constants import CONFIG_DIR + +router = APIRouter() + +_LIB = CONFIG_DIR / "reader_library" + + +def _lib() -> Path: + _LIB.mkdir(parents=True, exist_ok=True) + return _LIB + + +def _doc_dir(doc_id: str) -> Path: + # ids are server-generated uuid hex; reject anything else (path safety) + if not doc_id or not doc_id.isalnum() or len(doc_id) > 40: + raise HTTPException(400, "Bad document id") + return _lib() / doc_id + + +def _read_meta(d: Path) -> dict: + try: + return json.loads((d / "meta.json").read_text("utf-8")) + except Exception: + return {} + + +def _audio_indices(d: Path) -> list[int]: + ad = d / "audio" + if not ad.is_dir(): + return [] + out = [] + for f in ad.glob("*.mp3"): + try: + out.append(int(f.stem)) + except ValueError: + pass + return sorted(out) + + +@router.get("/api/reader/docs") +async def reader_list_docs(): + lib = _lib() + docs = [] + for d in lib.iterdir(): + if not d.is_dir(): + continue + meta = _read_meta(d) + if not meta: + continue + meta["id"] = d.name + meta["synthCount"] = len(_audio_indices(d)) + docs.append(meta) + docs.sort(key=lambda m: m.get("updated") or "", reverse=True) + return {"docs": docs} + + +@router.post("/api/reader/docs") +async def reader_save_doc(request: Request): + meta = await request.json() + doc_id = str(meta.get("id") or "").strip() + if doc_id: + d = _doc_dir(doc_id) + if not d.is_dir(): + raise HTTPException(404, "Document not found") + else: + doc_id = uuid.uuid4().hex + d = _lib() / doc_id + d.mkdir(parents=True, exist_ok=True) + meta["created"] = meta.get("created") + meta.pop("id", None) + (d / "meta.json").write_text(json.dumps(meta, ensure_ascii=False), "utf-8") + return {"id": doc_id} + + +@router.get("/api/reader/docs/{doc_id}") +async def reader_get_doc(doc_id: str): + d = _doc_dir(doc_id) + meta = _read_meta(d) + if not meta: + raise HTTPException(404, "Document not found") + meta["id"] = doc_id + meta["audioIdx"] = _audio_indices(d) + return meta + + +@router.delete("/api/reader/docs/{doc_id}") +async def reader_delete_doc(doc_id: str): + d = _doc_dir(doc_id) + if d.is_dir(): + shutil.rmtree(d, ignore_errors=True) + return {"ok": True} + + +@router.patch("/api/reader/docs/{doc_id}/progress") +async def reader_progress(doc_id: str, request: Request): + d = _doc_dir(doc_id) + meta = _read_meta(d) + if not meta: + raise HTTPException(404, "Document not found") + body = await request.json() + if "idx" in body: + meta["idx"] = body["idx"] + if "updated" in body: + meta["updated"] = body["updated"] + (d / "meta.json").write_text(json.dumps(meta, ensure_ascii=False), "utf-8") + return {"ok": True} + + +@router.put("/api/reader/docs/{doc_id}/source") +async def reader_put_source(doc_id: str, request: Request, ext: str = "pdf"): + d = _doc_dir(doc_id) + if not d.is_dir(): + raise HTTPException(404, "Document not found") + ext = "txt" if ext.lower() == "txt" else "pdf" + body = await request.body() + # clear any existing source of the other type + for old in d.glob("source.*"): + old.unlink() + (d / f"source.{ext}").write_bytes(body) + return {"ok": True, "bytes": len(body)} + + +@router.get("/api/reader/docs/{doc_id}/source") +async def reader_get_source(doc_id: str): + d = _doc_dir(doc_id) + for ext, media in (("pdf", "application/pdf"), ("txt", "text/plain")): + f = d / f"source.{ext}" + if f.is_file(): + return FileResponse(str(f), media_type=media) + raise HTTPException(404, "Source not found") + + +@router.get("/api/reader/docs/{doc_id}/audio") +async def reader_list_audio(doc_id: str): + return {"idx": _audio_indices(_doc_dir(doc_id))} + + +@router.put("/api/reader/docs/{doc_id}/audio/{idx}") +async def reader_put_audio(doc_id: str, idx: int, request: Request): + d = _doc_dir(doc_id) + if not d.is_dir(): + raise HTTPException(404, "Document not found") + (d / "audio").mkdir(exist_ok=True) + (d / "audio" / f"{idx}.mp3").write_bytes(await request.body()) + return {"ok": True} + + +@router.get("/api/reader/docs/{doc_id}/audio/{idx}") +async def reader_get_audio(doc_id: str, idx: int): + f = _doc_dir(doc_id) / "audio" / f"{idx}.mp3" + if not f.is_file(): + raise HTTPException(404, "Audio not found") + return FileResponse(str(f), media_type="audio/mpeg") diff --git a/scripts/minify.mjs b/scripts/minify.mjs index 04627ef..c24d8e6 100644 --- a/scripts/minify.mjs +++ b/scripts/minify.mjs @@ -22,7 +22,7 @@ mkdirSync(outDir, { recursive: true }); const MAIN = [ 'voice-picker', 'voice-inspector', 'voice-sources', 'fishaudio-browser', 'integrations', 'routing', 'voice-clone', 'voice-library', 'tts-preview', - 'benchmark', 'stt', 'rehearser-parse', 'rehearser', + 'benchmark', 'stt', 'rehearser-parse', 'rehearser', 'reader', 'audiobook', 'character-sheets', ].map(n => join(jsDir, n + '.js')); const source = MAIN.map(f => `\n/* ==== ${f.split('/').pop()} ==== */\n` + readFileSync(f, 'utf8')).join('\n'); diff --git a/server.py b/server.py index a498fcd..8a877aa 100644 --- a/server.py +++ b/server.py @@ -12,7 +12,7 @@ from fastapi.staticfiles import StaticFiles from fastapi.middleware.gzip import GZipMiddleware from core.constants import STATIC_DIR, _BufferHandler -from routes import admin, settings, library, stt, sources, docker, tts, conversation +from routes import admin, settings, library, stt, sources, docker, tts, conversation, reader logger = logging.getLogger("uvicorn.error") @@ -84,6 +84,7 @@ app.include_router(sources.router) app.include_router(docker.router) app.include_router(tts.router) app.include_router(conversation.router) +app.include_router(reader.router) # ── Static files ────────────────────────────────────────────────────────────── diff --git a/static/index.html b/static/index.html index 9077425..b03b224 100644 --- a/static/index.html +++ b/static/index.html @@ -146,6 +146,7 @@ + @@ -259,6 +260,7 @@ + diff --git a/static/js/audiobook.js b/static/js/audiobook.js new file mode 100644 index 0000000..76eb04d --- /dev/null +++ b/static/js/audiobook.js @@ -0,0 +1,468 @@ +// ── Book β†’ multi-speaker audiobook ────────────────────────────────────────── +// +// Bridges the Read Aloud reader and the Script Rehearser: an LLM scans the +// document (in the current scope β€” selection / page range / whole book), +// attributes every segment to a speaker ("Narrator" or a character) with an +// emotion, then hands the result to the Script Rehearser as a cast-able script +// so each character gets its own voice. The rehearser is the editable preview: +// you fix any mis-attribution, cast voices, and synthesise there. +// +// Reuses: /api/attribute-dialogue (LLM), readerState + readerScopeIndices() +// (reader.js), parseScript / detectCharacters / rehState / rehDefaultLlmUrl +// (rehearser.js), splitTextIntoChunks (generation.js), $ / toast (utils.js). + +const AUDIOBOOK_CHUNK_CHARS = 3000; // passage size per LLM attribution call +const _audiobook = { running: false, cancel: false }; + +// Opening/closing quote glyphs across book conventions: English "..."/β€œ...”, +// German Β»...Β«/β€ž...β€œ, French Β«...Β», single β€˜...’/β€Ί...β€Ή, CJK γ€Œ...γ€γ€Ž...』, em-dash speech. +const AB_DIALOGUE_RE = /[Β«Β»β€žβ€œβ€"β€Ÿβ€šβ€˜β€™β€Ίβ€Ήγ€Žγ€Œ]|(?:^|\n)\s*[—–]\s/; +function audiobookHasDialogue(t) { return AB_DIALOGUE_RE.test(t || ''); } + +// Join words hyphenated across a PDF line break ("Schwer- tes" β†’ "Schwertes") +// so the audiobook reads cleanly and speaker tags aren't split. +function audiobookDehyphenate(t) { return (t || '').replace(/([a-zÀâüß])-\s+(?=[a-zÀâüßA-ZΓ„Γ–Γœ])/g, '$1'); } + +// Speech-tag heuristic so the book still casts with REAL names when the LLM is down. +const AB_SPEECH_VERBS = '(?:sagte|fragte|rief|antwortete|erwiderte|entgegnete|meinte|flΓΌsterte|wisperte|raunte|murmelte|brummte|knurrte|brΓΌllte|schrie|stammelte|fauchte|zischte|seufzte|lachte|kicherte|befahl|wiederholte|fuhr\\s+fort|said|asked|replied|answered|whispered|murmured|muttered|shouted|cried|called|exclaimed|added|continued)'; +const AB_NOTNAME = new Set([ + 'Der', 'Die', 'Das', 'Den', 'Dem', 'Ein', 'Eine', 'Einen', 'Er', 'Sie', 'Es', 'Ich', 'Du', 'Wir', 'Ihr', 'Man', + 'Und', 'Aber', 'Da', 'Dann', 'Doch', 'So', 'Nun', 'Jetzt', 'The', 'He', 'She', 'It', 'They', 'A', 'An', 'And', 'But', 'Then', 'Now', + // common sentence-initial adverbs / interjections / abstractions that are NOT characters + 'Sofort', 'PlΓΆtzlich', 'Endlich', 'Schließlich', 'Stille', 'Schweigen', 'Stimme', 'Stimmen', 'Frage', 'Antwort', + 'GelΓ€chter', 'Wieder', 'Gleich', 'Sogleich', 'Langsam', 'Leise', 'Laut', 'Kaum', 'Vielleicht', 'NatΓΌrlich', 'Wirklich', + 'Ja', 'Nein', 'Komm', 'Warte', 'Halt', 'Geh', 'Hier', 'Dort', 'Oben', 'Unten', 'Schon', 'Noch', 'Auch', 'Nur', 'Immer', 'Nie']); +const _AB_NAME = "([A-ZΓ„Γ–Γœ][A-Za-zÀâüß'\\-]+)"; +function audiobookGuessSpeaker(after, before) { + let m; + const ok = n => (n && !AB_NOTNAME.has(n)) ? n : null; + // NOTE: no 'i' flag β€” names must be genuinely capitalized; German speech verbs + // after a quote are lowercase, so this rejects pronouns like "sagte er". + // after the quote: ", sagte Riskan" / "sagte Riskan" (verb β†’ name) + if ((m = new RegExp('^[\\s,;–-]*' + AB_SPEECH_VERBS + '\\s+(?:der|die|das|ein|eine)?\\s*' + _AB_NAME).exec(after || ''))) { const r = ok(m[1]); if (r) return r; } + // after the quote: ", Riskan sagte" (name β†’ verb) + if ((m = new RegExp('^[\\s,;–-]*' + _AB_NAME + '\\s+' + AB_SPEECH_VERBS).exec(after || ''))) { const r = ok(m[1]); if (r) return r; } + // before the quote: "Riskan sagte:" / "Riskan fragte" + if ((m = new RegExp(_AB_NAME + '\\s+' + AB_SPEECH_VERBS + '[\\s:,–-]*$').exec(before || ''))) { const r = ok(m[1]); if (r) return r; } + return null; +} + +// Deterministic fallback: split a passage into narration + dialogue by quotation +// spans and attribute speakers from the surrounding speech tags. Used when the LLM +// is unavailable so dialogue β€” and as many speakers as possible β€” are never lost. +const AB_QUOTE_SPAN = /Β»([^Β«]+)Β«|Β«([^Β»]+)Β»|β€ž([^β€œβ€]+)[β€œβ€]|β€œ([^”]+)”|"([^"]+)"|γ€Œ([^」]+)」|γ€Ž([^』]+)』/g; +function audiobookSplitByQuotes(text) { + const spans = []; let m; + AB_QUOTE_SPAN.lastIndex = 0; + while ((m = AB_QUOTE_SPAN.exec(text))) { + spans.push({ start: m.index, end: AB_QUOTE_SPAN.lastIndex, quote: (m[1] || m[2] || m[3] || m[4] || m[5] || m[6] || m[7] || '').trim() }); + } + if (!spans.length) return [{ speaker: 'Narrator', type: 'narration', text, emotion: '' }]; + const out = []; let last = 0; + for (let k = 0; k < spans.length; k++) { + const sp = spans[k]; + const pre = text.slice(last, sp.start); + if (pre.trim()) out.push({ speaker: 'Narrator', type: 'narration', text: pre.trim(), emotion: '' }); + if (sp.quote) { + const after = text.slice(sp.end, k + 1 < spans.length ? spans[k + 1].start : text.length); + const speaker = audiobookGuessSpeaker(after, pre) || 'Unknown'; + out.push({ speaker, type: 'dialogue', text: sp.quote, emotion: '' }); + } + last = sp.end; + } + const tail = text.slice(last); + if (tail.trim()) out.push({ speaker: 'Narrator', type: 'narration', text: tail.trim(), emotion: '' }); + return audiobookTurnTaking(out); +} + +// Fill 'Unknown' dialogue speakers by two-person alternation β€” but only once TWO +// distinct named speakers are established nearby (conservative: won't guess in a +// monologue, so it rarely invents a wrong name). +function audiobookTurnTaking(segs) { + let a = null, b = null; // two most recent distinct named speakers (b = latest) + for (const s of segs) { + if (s.type !== 'dialogue') continue; + if (s.speaker && s.speaker !== 'Unknown') { + if (s.speaker !== b) { a = b; b = s.speaker; } + } else if (a && b && a !== b) { + s.speaker = a; // the other of the two β†’ alternate + const t = a; a = b; b = t; // rotate so the next Unknown alternates back + } + } + return segs; +} + +function audiobookLlmUrl() { return $('reh-llm-url')?.value.trim() || (typeof rehDefaultLlmUrl === 'function' ? rehDefaultLlmUrl() : ''); } +function audiobookLlmModel() { return $('reh-llm-model')?.value || ''; } +function audiobookLang() { return $('reh-design-lang')?.value || ''; } + +// Gather the plain text of the current reader scope (selection > page range > all). +function audiobookScopeText() { + if (typeof readerScopeIndices !== 'function' || !readerState?.sentences?.length) return ''; + const raw = readerScopeIndices().map(i => readerState.sentences[i].text).join(' ').replace(/\s+/g, ' ').trim(); + return audiobookDehyphenate(raw); // mend PDF line-break hyphenation for clean speech + tag matching +} + +// ── Progress overlay ───────────────────────────────────────────────────────── + +function audiobookProgress(total) { + let ov = document.getElementById('audiobook-overlay'); + if (!ov) { + ov = document.createElement('div'); + ov.id = 'audiobook-overlay'; + ov.className = 'audiobook-overlay'; + ov.innerHTML = `
+
Casting audiobook
+
Analysing…
+
+
+
`; + document.body.appendChild(ov); + ov.querySelector('#audiobook-cancel').addEventListener('click', () => { _audiobook.cancel = true; }); + } + ov.hidden = false; + const fill = ov.querySelector('#audiobook-fill'); + const msg = ov.querySelector('#audiobook-msg'); + return { + update(done, label) { if (fill) fill.style.width = (done / total * 100) + '%'; if (msg && label) msg.textContent = label; }, + done() { ov.hidden = true; }, + }; +} + +const _AB_PALETTE = ['#3b82f6', '#10b981', '#8b5cf6', '#f59e0b', '#ef4444', '#ec4899', '#06b6d4', '#84cc16', '#f97316', '#14b8a6', '#6366f1', '#d946ef']; + +// Live casting view: a scrolling feed of attributed lines + a character roster +// that fills up as speakers are discovered. Far clearer than a bare bar. +function audiobookCastView(total) { + document.getElementById('audiobook-overlay')?.remove(); + // A floating, NON-blocking, minimisable panel β€” keep working in the app and + // come back to watch progress (no backdrop, no modal lock). + const ov = document.createElement('div'); + ov.id = 'audiobook-overlay'; ov.className = 'ab-castpanel'; + ov.innerHTML = ` +
+ + Casting audiobook + passage 0 / ${total} + + + +
+
+
+
+
+
Characters found
+
listening…
+
+
+ `; + document.body.appendChild(ov); + const setMin = min => { + ov.classList.toggle('minimized', min); + const ic = ov.querySelector('#ab-cv-min .mdi'); + if (ic) ic.className = 'mdi ' + (min ? 'mdi-window-maximize' : 'mdi-window-minimize'); + }; + ov.querySelector('#ab-cv-min').addEventListener('click', () => setMin(!ov.classList.contains('minimized'))); + ov.querySelector('#ab-cv-x').addEventListener('click', () => { if (_audiobook.running) _audiobook.cancel = true; else ov.remove(); }); + ov.querySelector('#ab-cv-head').addEventListener('click', e => { if (ov.classList.contains('minimized') && !e.target.closest('button')) setMin(false); }); + const fill = ov.querySelector('#ab-cv-fill'), count = ov.querySelector('#ab-cv-count'); + const feed = ov.querySelector('#ab-cv-feed'), chars = ov.querySelector('#ab-cv-chars'); + const roster = new Map(); // name -> { count, color } + const colorFor = name => { + if (!roster.has(name)) roster.set(name, { count: 0, color: _AB_PALETTE[roster.size % _AB_PALETTE.length] }); + return roster.get(name).color; + }; + const renderRoster = () => { + const items = [...roster.entries()].sort((a, b) => b[1].count - a[1].count); + chars.innerHTML = items.length + ? items.map(([n, info]) => `${escHtml(n)}${info.count}`).join('') + : 'listening…'; + }; + const MAXROWS = 80; + const trim = () => { while (feed.childElementCount > MAXROWS) feed.removeChild(feed.firstChild); feed.scrollTop = feed.scrollHeight; }; + return { + update(done) { if (fill) fill.style.width = (done / total * 100) + '%'; if (count) count.textContent = `passage ${done} / ${total}`; }, + addSegments(segs) { + const frag = document.createDocumentFragment(); + for (const s of segs) { + const dialog = s.type === 'dialogue' && s.speaker && s.speaker.toLowerCase() !== 'narrator'; + const row = document.createElement('div'); + row.className = 'ab-cv-row' + (dialog ? '' : ' is-narr'); + if (dialog) { const c = colorFor(s.speaker); roster.get(s.speaker).count++; + row.innerHTML = `${escHtml(s.speaker)}${s.emotion ? ' Β· ' + escHtml(s.emotion) : ''}${escHtml((s.text || '').slice(0, 160))}`; + } else { + row.innerHTML = `Narrator${escHtml((s.text || '').slice(0, 160))}`; + } + frag.appendChild(row); + } + feed.appendChild(frag); trim(); renderRoster(); + }, + note(text) { const r = document.createElement('div'); r.className = 'ab-cv-note'; r.textContent = text; feed.appendChild(r); trim(); }, + // Park the panel in a "done" state with a Review button instead of auto-popping + // the preview β€” so it waits for you if you wandered off to do something else. + complete(summary, onOpen) { + if (count) count.textContent = 'done'; + if (fill) fill.style.width = '100%'; + ov.querySelector('#ab-cv-min').hidden = true; + ov.querySelector('#ab-cv-x').title = 'Dismiss'; + const foot = ov.querySelector('#ab-cv-foot'); + foot.hidden = false; + foot.innerHTML = ` ${escHtml(summary)}`; + foot.querySelector('#ab-cv-review').addEventListener('click', () => { ov.remove(); onOpen(); }); + // gently nudge the panel open if it was minimised while you were away + ov.classList.add('ab-castpanel-done'); + }, + done() { ov.remove(); }, + }; +} + +// ── Attribution β†’ script handoff ───────────────────────────────────────────── + +async function audiobookCast() { + if (_audiobook.running) return; + const text = audiobookScopeText(); + if (!text) { toast('Import a document first', 'error'); return; } + if (typeof parseScript !== 'function') { toast('Rehearser not loaded yet β€” try again in a moment', 'error'); return; } + + const chunks = (typeof splitTextIntoChunks === 'function') + ? splitTextIntoChunks(text, AUDIOBOOK_CHUNK_CHARS) + : [text]; + + _audiobook.running = true; _audiobook.cancel = false; + const view = audiobookCastView(chunks.length); + const llm_url = audiobookLlmUrl(), model = audiobookLlmModel(), language = audiobookLang(); + + const allSegments = []; + const roster = []; + let narrationOnly = 0; // passages with no quotes at all β€” legitimately all narration + let degraded = 0; // passages with dialogue the LLM couldn't analyse β†’ quotes auto-extracted + try { + for (let i = 0; i < chunks.length; i++) { + if (_audiobook.cancel) break; + view.update(i); + // No quotation marks anywhere β†’ pure narration; skip the LLM entirely (faster, not an error) + if (!audiobookHasDialogue(chunks[i])) { + const seg = { speaker: 'Narrator', type: 'narration', text: chunks[i], emotion: '' }; + allSegments.push(seg); narrationOnly++; view.addSegments([seg]); + continue; + } + // recent attributed dialogue β†’ lets the LLM continue turn-taking across the boundary + const recent = allSegments.filter(s => s.type === 'dialogue' && s.speaker && s.speaker !== 'Unknown') + .slice(-6).map(s => `${s.speaker}: ${(s.text || '').slice(0, 80)}`).join('\n'); + let data = null; + try { + const r = await fetch('/api/attribute-dialogue', { + method: 'POST', headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ text: chunks[i], known_characters: roster.slice(-40), recent, language, llm_url, model }), + }); + if (!r.ok) { const e = await r.json().catch(() => ({})); throw new Error(e.detail || r.statusText); } + data = await r.json(); + } catch (_) { data = null; } + + let segs = data && Array.isArray(data.segments) ? data.segments : []; + if (!segs.length) { + // LLM unavailable or returned nothing, but this passage HAS quotes β€” + // extract dialogue + attribute speakers from speech tags so it isn't lost. + segs = audiobookSplitByQuotes(chunks[i]); + degraded++; + const named = segs.filter(s => s.type === 'dialogue' && s.speaker !== 'Unknown').length; + view.note(`Passage ${i + 1} β€” auto-detected dialogue${named ? ` (${named} speaker${named !== 1 ? 's' : ''} from tags)` : ' (set speakers in review)'}`); + } + // harvest speaker names (from LLM or tag heuristic) into the running roster + (data && data.characters || []).forEach(n => { if (n && n !== 'Unknown' && !roster.includes(n)) roster.push(n); }); + segs.forEach(s => { if (s.type === 'dialogue' && s.speaker && s.speaker !== 'Unknown' && !roster.includes(s.speaker)) roster.push(s.speaker); }); + segs.forEach(s => allSegments.push(s)); + view.addSegments(segs); + } + view.update(chunks.length); + } finally { + _audiobook.running = false; + } + + if (_audiobook.cancel) { view.done(); toast('Casting cancelled', 'error'); return; } + if (!allSegments.length) { view.done(); toast('No segments produced', 'error'); return; } + + _audiobook.segments = allSegments; + _audiobook.roster = roster; + _audiobook.narratedPassages = narrationOnly; + _audiobook.degraded = degraded; + // Park the panel with a "Review & cast" button (don't auto-pop, in case you wandered off) + const speakers = new Set(allSegments.filter(s => s.type === 'dialogue' && s.speaker).map(s => s.speaker)); + const summary = `${speakers.size} character${speakers.size !== 1 ? 's' : ''} Β· ${allSegments.length} segments`; + view.complete(summary, audiobookShowPreview); +} + +// ── Editable attribution preview ───────────────────────────────────────────── + +function audiobookShowPreview() { + const segs = _audiobook.segments || []; + const roster = _audiobook.roster || []; + document.getElementById('audiobook-preview')?.remove(); + const ov = document.createElement('div'); + ov.id = 'audiobook-preview'; + ov.className = 'audiobook-overlay'; + // datalist = Narrator + LLM roster + any speakers present in the segments (incl. Unknown) + const speakerSet = [...new Set(['Narrator', ...roster, ...segs.filter(s => s.type === 'dialogue' && s.speaker).map(s => s.speaker)])]; + const charCount = speakerSet.filter(n => n !== 'Narrator' && n !== 'Unknown').length; + const opts = speakerSet.map(n => ``).join(''); - sel.addEventListener('change', () => window.setAppLang(sel.value)); - brand.appendChild(sel); + const head = brand && brand.querySelector('h1'); + if (!head || document.getElementById('app-lang-toggle')) return; + const codes = Object.keys(I18N_LANGS); + const btn = document.createElement('button'); + btn.id = 'app-lang-toggle'; + btn.className = 'app-lang-toggle'; + btn.type = 'button'; + const nextOf = () => codes[(codes.indexOf(_appLang) + 1) % codes.length]; + const render = () => { + btn.innerHTML = ``; + const label = 'Language: ' + I18N_LANGS[_appLang] + ' β€” switch to ' + I18N_LANGS[nextOf()]; + btn.title = label; btn.setAttribute('aria-label', label); + }; + render(); + btn.addEventListener('click', () => window.setAppLang(nextOf())); + head.appendChild(btn); }; diff --git a/static/js/reader.js b/static/js/reader.js new file mode 100644 index 0000000..11643b4 --- /dev/null +++ b/static/js/reader.js @@ -0,0 +1,1324 @@ +// ── Read Aloud β€” PDF/TXT document reader with live word highlighting ──────── +// +// Imports a PDF (rendered to canvas via pdf.js with a highlight overlay on the +// real text) or a plain-text file, then reads it sentence-by-sentence with a +// chosen voice/backend/speed while highlighting the word being spoken. +// +// Per-sentence synthesis state is colour-coded: red = not synthesised, +// yellow = synthesising, green = ready (cached), blue = currently reading. +// PDF view supports zoom (fit-width / fit-height / two-page / Β±). Reading +// position is remembered so the same document resumes where it left off. +// +// Reuses existing globals: loadPdfJs() + computeWordTimings() (rehearser.js), +// fetchTtsPreviewBlob() (tts-preview.js), ttsBackendOptions()/availableTtsBackends() +// (settings.js), and $ / escHtml / toast (utils.js). + +const readerState = { + mode: null, // 'pdf' | 'text' + title: '', + pdfDoc: null, + pages: [], // [{ pageDiv, overlay, page, base, rendered, renderTask }] + sentences: [], // [{ text, words:[{page,x,top,w,h} | {el}], status, _stat }] + idx: 0, + readingIdx: -1, // the single unit currently shown as "reading" (avoids full scans) + playing: false, + speed: 1, + scale: 1, + zoomMode:'fit-width', // 'fit-width' | 'fit-height' | 'two' | 'custom' + audioCtx: null, + currentSource: null, + raf: null, + blobCache: new Map(), // idx -> Blob + bufCache: new Map(), // idx -> AudioBuffer + gainCache: new Map(), // idx -> normalisation gain + unitsByPage: new Map(),// page -> [unit indices] for lazy status-box creation + baseSentences: [], // raw per-sentence units (before chunk grouping) + chunkMode: 'sentence',// 'sentence' | 'paragraph' | 'page' β€” synthesis chunk size + normalize: true, // even out loudness between chunks on playback + io: null, // IntersectionObserver for lazy page raster + _seq: 0, // import generation guard + synthRunning: false, + synthCancel: false, + selecting: false, // click-to-select-range mode (stays on until Done/Esc) + selStart: null, + selEnd: null, + selAnchored: false, // true after the first click, waiting for the end click + fileBlob: null, // original imported file (Blob) β€” kept so PDFs can be re-rendered + docText: '', // original text (for text-mode docs) + savedId: null, // server library doc id when this doc came from / was saved to the library + savedAudioIdx: new Set(), // unit indices whose audio is already on the server (incremental save / lazy fetch) + sourceUploaded: false, // the source document has been uploaded to the server +}; +window.readerState = readerState; + +const READER_RESUME_KEY = 'reader-resume'; +const READER_FMT = 'mp3'; // synthesise + store as MP3 (compact; decodeAudioData plays it) +const READER_BUF_WINDOW = 3; // keep decoded PCM only for Β±N units around the playhead +const READER_CANVAS_MARGIN = 2200; // px; raster canvases beyond this from view are released + +function readerCtx() { + if (!readerState.audioCtx) readerState.audioCtx = new (window.AudioContext || window.webkitAudioContext)(); + if (readerState.audioCtx.state === 'suspended') readerState.audioCtx.resume().catch(() => {}); + return readerState.audioCtx; +} + +// ── Section enter / leave hooks (wired from nav.js) ───────────────────────── + +window.readerOnShow = async function () { + const sel = $('reader-backend-select'); + if (sel && (!sel.value || sel.options.length <= 1)) { + // Backends may not have loaded yet (or loaded before this section existed) β€” + // fetch them on demand so the dropdown fills instead of staying on "Checking…". + if (typeof availableTtsBackends === 'function' && !availableTtsBackends().length && + typeof refreshTtsBackendAvailability === 'function') { + try { await refreshTtsBackendAvailability(); } catch (_) {} + } + if (typeof availableTtsBackends === 'function' && availableTtsBackends().length) { + sel.innerHTML = ttsBackendOptions(sel.value); + sel.disabled = false; + } + } + readerUpdateBackendHint(); + readerRenderLibrary(); +}; + +window.readerStop = function () { readerSaveResume(); readerPersistProgress(); readerStopPlayback(); }; + +// ── Backend + voice pickers ───────────────────────────────────────────────── + +async function readerFetchVoices() { + const btn = $('reader-fetch-voices-btn'); + const backend = $('reader-backend-select')?.value; + if (!backend) { toast('No available TTS backend', 'error'); return; } + if (btn) btn.disabled = true; + try { + const raw = await fetch('/api/tts-voices?backend=' + encodeURIComponent(backend)).then(r => r.json()); + const ids = (Array.isArray(raw) ? raw : []).map(v => (typeof v === 'string' ? v : (v.id || v.name || v.voice_id))).filter(Boolean); + const sel = $('reader-voice-select'); if (!sel) return; + const prev = sel.value; + sel.innerHTML = ''; + ids.forEach(id => { const o = document.createElement('option'); o.value = o.textContent = id; sel.appendChild(o); }); + if (prev && ids.includes(prev)) sel.value = prev; + toast('Fetched ' + ids.length + ' voices', 'success'); + } catch (e) { toast('Fetch failed: ' + (e.message || e), 'error'); } + finally { if (btn) btn.disabled = false; } +} + +// Guidance for steady long-form narration (addresses voice drift between chunks): +// some backends re-sample timbre per request; flag those and suggest remedies. +function readerUpdateBackendHint() { + const el = $('reader-backend-hint'); if (!el) return; + const id = $('reader-backend-select')?.value || ''; + const b = (typeof backendById === 'function') ? backendById(id) : null; + const steady = id && ((b && b.style_aware) || /design|custom|kokoro|magpie/i.test(id)); + el.innerHTML = steady + ? ' Good for long-form narration. For maximum steadiness, increase β€œVoice consistency”.' + : ' Cloned / zero-shot voices re-sample each chunk, so the voice can drift between sections. To steady it: raise β€œVoice consistency” to paragraph/page, set a fixed Seed with a low Temperature (e.g. 0.3), keep β€œNormalise loudness” on, or pick a style-aware backend.'; + el.classList.toggle('reader-hint-warn', !steady); +} + +// ── Import ────────────────────────────────────────────────────────────────── + +async function readerImportFile(file) { + if (!file) return; + const name = (file.name || '').toLowerCase(); + readerResetDoc(); + readerState.title = file.name.replace(/\.[^.]+$/, ''); + readerState.fileBlob = file; + const titleEl = $('reader-doc-title'); if (titleEl) titleEl.textContent = readerState.title; + try { + if (name.endsWith('.pdf')) { + toast('Reading PDF…', 'success'); + await readerLoadPdf(file); + } else { + const text = await file.text(); + readerState.docText = text; + readerLoadText(text); + } + if (!readerState.sentences.length) { toast('No readable text found in this document', 'error'); return; } + if (readerState.mode === 'text') readerState.sentences.forEach((s, i) => readerSetStatus(i, 'pending')); + $('reader-transport').hidden = false; + $('reader-dropzone').hidden = true; + $('reader-toolbar').hidden = readerState.mode !== 'pdf'; + $('reader-synthbar').hidden = false; + const pr = $('reader-page-range'); + if (pr) { + pr.hidden = readerState.mode !== 'pdf'; + if (readerState.mode === 'pdf') { + const n = readerState.pages.length; + const pf = $('reader-page-from'), pt = $('reader-page-to'); + if (pf) { pf.max = n; pf.value = 1; } + if (pt) { pt.max = n; pt.value = n; } + } + } + readerUpdateScopeLabel(); + const resumed = readerLoadResume(); + readerUpdateProgress(); + readerHighlightSentence(readerState.sentences[readerState.idx]); + toast(resumed + ? 'Resumed at sentence ' + (readerState.idx + 1) + ' / ' + readerState.sentences.length + : 'Loaded Β· ' + readerState.sentences.length + ' sentences β€” press play', 'success'); + } catch (e) { toast('Import failed: ' + (e.message || e), 'error'); } +} + +function readerResetDoc() { + readerStopPlayback(); + readerState._seq++; + readerState.mode = null; + readerState.pdfDoc = null; + readerState.pages = []; + readerState.sentences = []; + readerState.idx = 0; + readerState.readingIdx = -1; + readerState.scale = 1; + readerState.zoomMode = 'fit-width'; + readerState.blobCache.clear(); + readerState.bufCache.clear(); + readerState.gainCache.clear(); + readerState.unitsByPage = new Map(); + readerState.baseSentences = []; + readerState.fileBlob = null; + readerState.docText = ''; + readerState.savedId = null; + readerState.savedAudioIdx = new Set(); + readerState.sourceUploaded = false; + if (readerState.io) { readerState.io.disconnect(); readerState.io = null; } + readerState.synthRunning = false; + readerState.synthCancel = false; + readerSetSelecting(false); + readerState.selStart = readerState.selEnd = null; readerState.selAnchored = false; + const sb = $('reader-synthbar'); if (sb) sb.hidden = true; + const sp = $('reader-synth-prog'); if (sp) sp.hidden = true; + const si = $('reader-sel-info'); if (si) si.hidden = true; + const doc = $('reader-doc'); if (doc) { doc.innerHTML = ''; doc.classList.remove('two-page', 'reader-selecting'); } +} + +// Build sentence list from a flat array of word objects (each carries geometry). +function readerBuildSentences(words) { + const sentences = []; + let cur = null; + const endRe = /[.!?]["'”’)\]]?$/; + const abbrev = /^(mr|mrs|ms|dr|prof|sr|jr|vs|etc|e\.g|i\.e|no|vol|st|fig)\.?$/i; + for (const w of words) { + if (!cur) cur = { text: '', words: [], status: 'pending', _stat: null }; + cur.words.push(w); + cur.text += (cur.text ? ' ' : '') + w.text; + const isEnd = endRe.test(w.text) && !abbrev.test(w.text.replace(/[^a-z.]/gi, '')); + if ((isEnd && cur.words.length >= 2) || cur.words.length >= 45) { + sentences.push(cur); cur = null; + } + } + if (cur && cur.words.length) sentences.push(cur); + return sentences; +} + +// Group base sentences into synthesis units. Larger units = one continuous +// generation = steadier voice (fewer seams where timbre/prosody can drift). +// β€’ sentence β€” one unit per sentence (most responsive) +// β€’ paragraph β€” merge sentences within a paragraph (text) / page (PDF), ~500 chars +// β€’ page β€” merge a whole PDF page (or ~2000-char blocks of text) +function readerGroupUnits(base, mode) { + if (mode === 'sentence' || !base.length) { + return base.map(s => ({ text: s.text, words: s.words, status: 'pending', _stat: null })); + } + const maxChars = mode === 'page' ? 2000 : 500; + const isPdf = readerState.mode === 'pdf'; + const keyOf = s => isPdf ? (s.words[0]?.page ?? 0) : (s.words[0]?.para ?? 0); + const units = []; + let cur = null; + for (const s of base) { + const k = keyOf(s); + // boundary break: paragraph mode breaks on para/page change; page mode breaks + // only on PDF page change (text page mode flows by size alone) + const boundary = cur && ( + (mode === 'paragraph' && k !== cur._key) || + (mode === 'page' && isPdf && k !== cur._key) + ); + const tooLong = cur && cur.words.length && (cur.text.length + s.text.length + 1 > maxChars); + if (boundary || tooLong) { units.push(cur); cur = null; } + if (!cur) cur = { text: '', words: [], status: 'pending', _stat: null, _key: k }; + cur.text += (cur.text ? ' ' : '') + s.text; + cur.words.push(...s.words); + } + if (cur) units.push(cur); + return units; +} + +// ── PDF import: build scale-1 geometry; render lazily at current scale ─────── + +async function readerLoadPdf(file) { + await loadPdfJs(); + const seq = readerState._seq; + const ab = await file.arrayBuffer(); + const pdf = await pdfjsLib.getDocument({ data: ab }).promise; + if (seq !== readerState._seq) return; + readerState.mode = 'pdf'; + readerState.pdfDoc = pdf; + const doc = $('reader-doc'); + const nPages = pdf.numPages; + const progress = readerShowParseProgress(nPages); + + // Sentences/units are built incrementally per page (page-local) and appended, + // so there is no big synchronous spike at the end and memory work is spread + // across yields β€” keeps huge books responsive. Earlier indices never shift + // (later pages only append), so caches/status stay aligned. + for (let p = 1; p <= nPages; p++) { + const page = await pdf.getPage(p); + if (seq !== readerState._seq) { progress.done(); return; } + const base = page.getViewport({ scale: 1 }); // geometry reference (scale-independent) + + const pageDiv = document.createElement('div'); + pageDiv.className = 'reader-page'; + const overlay = document.createElement('div'); + overlay.className = 'reader-overlay'; + pageDiv.appendChild(overlay); + doc.appendChild(pageDiv); + + const pageIdx = readerState.pages.length; + readerState.pages.push({ pageDiv, overlay, page, base, rendered: false, renderTask: null }); + + // Per-word geometry in scale-1 device px (top-left origin) β†’ multiply by scale later + const content = await page.getTextContent(); + if (seq !== readerState._seq) return; + const pageWords = []; + for (const item of content.items) { + const str = (item.str || ''); + if (!str.trim()) continue; + const m = pdfjsLib.Util.transform(base.transform, item.transform); + const h = Math.hypot(m[2], m[3]) || Math.hypot(m[0], m[1]) || 12; + const x = m[4]; + const top = m[5] - h; + const w = (item.width || 0); + const tokens = str.split(/(\s+)/); // keep spaces for width proportion + const totalLen = str.length || 1; + let offX = x; + for (const tok of tokens) { + const tw = (tok.length / totalLen) * w; + if (tok.trim()) pageWords.push({ page: pageIdx, x: offX, top, w: Math.max(tw, 2), h, text: tok }); + offX += tw; + } + } + // Build + append this page's sentences and units + const pageBase = readerBuildSentences(pageWords); + const pageUnits = readerGroupUnits(pageBase, readerState.chunkMode); + const startUnit = readerState.sentences.length; + readerState.baseSentences.push(...pageBase); + readerState.sentences.push(...pageUnits); + const idxs = []; + for (let u = startUnit; u < readerState.sentences.length; u++) idxs.push(u); + readerState.unitsByPage.set(pageIdx, idxs); + + // Yield to the event loop every few pages so the browser stays responsive + // on big books (prevents the "page is not responding" prompt). + progress.update(p); + if (p % 5 === 0) await new Promise(r => setTimeout(r, 0)); + } + progress.done(); + if (seq !== readerState._seq) return; + readerApplyZoom('fit-width'); + readerSetupLazyRaster(); +} + +// Lightweight parse-progress line shown in the document pane for big PDFs. +function readerShowParseProgress(total) { + const doc = $('reader-doc'); if (!doc) return { update() {}, done() {} }; + const el = document.createElement('div'); + el.className = 'reader-parsing'; + el.innerHTML = ' Reading PDF… page 0 / ' + total + ''; + doc.appendChild(el); + const label = el.querySelector('span:last-child'); + return { + update(p) { if (label) label.textContent = 'Reading PDF… page ' + p + ' / ' + total; }, + done() { el.remove(); }, + }; +} + +// ── Zoom ───────────────────────────────────────────────────────────────────── + +function readerComputeScale(mode) { + const doc = $('reader-doc'); + const base = readerState.pages[0]?.base; + if (!doc || !base) return 1; + const padW = 36, gap = 16; + const availW = (doc.clientWidth || 900) - padW; + const availH = (doc.clientHeight || 600) - 36; + if (mode === 'fit-width') return Math.max(0.2, availW / base.width); + if (mode === 'fit-height') return Math.max(0.2, availH / base.height); + if (mode === 'two') return Math.max(0.2, (availW - gap) / 2 / base.width); + return readerState.scale; +} + +function readerApplyZoom(mode) { + if (!readerState.pages.length) return; + if (mode) readerState.zoomMode = mode; + if (mode && mode !== 'custom') readerState.scale = readerComputeScale(mode); + const scale = readerState.scale; + const doc = $('reader-doc'); + doc.classList.toggle('two-page', readerState.zoomMode === 'two'); + + readerState.pages.forEach(pg => { + pg.pageDiv.style.width = (pg.base.width * scale) + 'px'; + pg.pageDiv.style.height = (pg.base.height * scale) + 'px'; + const old = pg.pageDiv.querySelector('canvas.reader-canvas'); + if (old) old.remove(); + if (pg.renderTask) { try { pg.renderTask.cancel(); } catch (_) {} pg.renderTask = null; } + pg.rendered = false; + }); + // reposition only the status boxes that actually exist (lazy/visible pages) + readerState.pages.forEach(pg => pg.overlay.querySelectorAll('.reader-stat').forEach(el => { + const i = parseInt(el.dataset.si); if (!isNaN(i)) readerPaintStatus(i); + })); + if (readerState.idx < readerState.sentences.length) readerHighlightSentence(readerState.sentences[readerState.idx]); + const pct = $('reader-zoom-pct'); if (pct) pct.textContent = Math.round(scale * 100) + '%'; + readerRenderVisible(); +} + +function readerSetupLazyRaster() { + if (readerState.io) readerState.io.disconnect(); + readerState.io = new IntersectionObserver(entries => { + entries.forEach(en => { + if (!en.isIntersecting) return; + const idx = readerState.pages.findIndex(p => p.pageDiv === en.target); + if (idx >= 0) readerRenderPage(idx); + }); + }, { root: $('reader-doc'), rootMargin: '600px 0px' }); + readerState.pages.forEach(p => readerState.io.observe(p.pageDiv)); +} + +function readerRenderVisible() { + const doc = $('reader-doc'); if (!doc) return; + const dr = doc.getBoundingClientRect(); + readerState.pages.forEach((pg, i) => { + const r = pg.pageDiv.getBoundingClientRect(); + if (r.bottom > dr.top - 800 && r.top < dr.bottom + 800) readerRenderPage(i); + }); + readerEvictCanvases(dr); +} + +// Release canvas bitmaps for pages far outside the viewport so a long book's +// raster memory stays bounded; they re-render when scrolled back into view. +function readerEvictCanvases(dr) { + const doc = $('reader-doc'); if (!doc) return; + dr = dr || doc.getBoundingClientRect(); + readerState.pages.forEach(pg => { + if (!pg.rendered) return; + const r = pg.pageDiv.getBoundingClientRect(); + if (r.bottom < dr.top - READER_CANVAS_MARGIN || r.top > dr.bottom + READER_CANVAS_MARGIN) { + if (pg.renderTask) { try { pg.renderTask.cancel(); } catch (_) {} pg.renderTask = null; } + const c = pg.pageDiv.querySelector('canvas.reader-canvas'); + if (c) c.remove(); + pg.rendered = false; // allow re-raster when it returns to view + } + }); +} + +async function readerRenderPage(idx) { + const pg = readerState.pages[idx]; + if (!pg || pg.rendered) return; + pg.rendered = true; // claim immediately to avoid double render + const scale = readerState.scale; + const viewport = pg.page.getViewport({ scale }); + const canvas = document.createElement('canvas'); + canvas.className = 'reader-canvas'; + canvas.width = Math.floor(viewport.width); + canvas.height = Math.floor(viewport.height); + pg.pageDiv.insertBefore(canvas, pg.overlay); + readerCreatePageStatus(idx); // create this page's status boxes lazily (cheap for big books) + try { + pg.renderTask = pg.page.render({ canvasContext: canvas.getContext('2d'), viewport }); + await pg.renderTask.promise; + } catch (_) { pg.rendered = false; canvas.remove(); } + finally { pg.renderTask = null; } +} + +// ── Text import: render word spans, highlight via classes ──────────────────── + +function readerLoadText(text) { + readerState.mode = 'text'; + readerState.docText = text; // keep the source so it can be saved to the library + const doc = $('reader-doc'); + const pane = document.createElement('div'); + pane.className = 'reader-text'; + doc.appendChild(pane); + + const words = []; + const paras = text.replace(/\r\n/g, '\n').split(/\n{2,}/); + paras.forEach((para, pi) => { + const pEl = document.createElement('p'); + pEl.className = 'reader-text-para'; + para.split('\n').forEach((line, li, arr) => { + line.trim().split(/\s+/).forEach(tok => { + if (!tok) return; + const span = document.createElement('span'); + span.className = 'reader-word'; + span.textContent = tok; + pEl.appendChild(span); + pEl.appendChild(document.createTextNode(' ')); + words.push({ el: span, text: tok, page: 0, para: pi }); + }); + if (li < arr.length - 1) pEl.appendChild(document.createElement('br')); + }); + pane.appendChild(pEl); + }); + readerState.baseSentences = readerBuildSentences(words); + readerState.sentences = readerGroupUnits(readerState.baseSentences, readerState.chunkMode); + pane.addEventListener('click', e => { + const span = e.target.closest('.reader-word'); if (!span) return; + const si = readerState.sentences.findIndex(s => s.words.some(w => w.el === span)); + if (si < 0) return; + if (readerState.selecting) readerPickSentence(si); + else readerJumpTo(si); + }); +} + +// ── Per-sentence synthesis-state visualization ─────────────────────────────── + +function readerSentBBox(sentence) { + const pageIdx = sentence.words[0]?.page ?? 0; + const on = sentence.words.filter(w => w.page === pageIdx); + return { + page: pageIdx, + x: Math.min(...on.map(w => w.x)), + top: Math.min(...on.map(w => w.top)), + w: Math.max(...on.map(w => w.x + w.w)) - Math.min(...on.map(w => w.x)), + h: Math.max(...on.map(w => w.top + w.h)) - Math.min(...on.map(w => w.top)), + }; +} + +// Index units by their page so status boxes can be created lazily per page +// (creating thousands of overlay divs up front would jank big books). +function readerBuildUnitIndex() { + readerState.unitsByPage = new Map(); + if (readerState.mode !== 'pdf') return; + readerState.sentences.forEach((s, i) => { + const pg = s.words[0]?.page ?? 0; + if (!readerState.unitsByPage.has(pg)) readerState.unitsByPage.set(pg, []); + readerState.unitsByPage.get(pg).push(i); + }); +} + +// Create + paint the status boxes for one page (called when the page rasters). +function readerCreatePageStatus(pageIdx) { + if (readerState.mode !== 'pdf') return; + const pg = readerState.pages[pageIdx]; if (!pg) return; + const idxs = readerState.unitsByPage?.get(pageIdx) || []; + for (const i of idxs) { + const s = readerState.sentences[i]; + if (!s || s._stat) continue; + const el = document.createElement('div'); + el.className = 'reader-stat'; + el.dataset.si = i; + pg.overlay.appendChild(el); + s._stat = el; + readerPaintStatus(i); + } +} + +// effective colour: reading wins, else cached/synth/pending +function readerEffectiveStatus(s) { return s.status; } + +function readerSetStatus(idx, status) { + const s = readerState.sentences[idx]; if (!s) return; + s.status = status; + readerPaintStatus(idx); +} + +function readerPaintStatus(idx) { + const s = readerState.sentences[idx]; if (!s) return; + const eff = readerEffectiveStatus(s); + if (readerState.mode === 'pdf') { + if (!s._stat) return; + const b = readerSentBBox(s); + const scale = readerState.scale; + Object.assign(s._stat.style, { left: (b.x * scale) + 'px', top: (b.top * scale) + 'px', width: (b.w * scale) + 'px', height: (b.h * scale) + 'px' }); + s._stat.className = 'reader-stat reader-stat-' + eff; + } else { + s.words.forEach(w => { + if (!w.el) return; + w.el.classList.remove('stat-pending', 'stat-synth', 'stat-ready', 'stat-reading'); + w.el.classList.add('stat-' + eff); + }); + } +} + +// ── Active reading highlight (word + scroll) ───────────────────────────────── + +let _readerWordBox = null; + +function readerEnsureBox() { + if (!_readerWordBox) { _readerWordBox = document.createElement('div'); _readerWordBox.className = 'reader-hl-word'; } +} + +function readerClearHighlights() { + if (readerState.mode === 'text') { + document.querySelectorAll('.reader-word.is-word').forEach(el => el.classList.remove('is-word')); + } else if (_readerWordBox) { + _readerWordBox.hidden = true; + } +} + +// Scroll ONLY the #reader-doc pane (never the window) so the active text stays +// inside the visible frame and not under the app header. Keeps the target at +// ~38% from the top when it would otherwise be clipped top or bottom. +function readerEnsureVisible(el) { + const doc = $('reader-doc'); if (!doc || !el) return; + const dr = doc.getBoundingClientRect(); + const er = el.getBoundingClientRect(); + if (er.top < dr.top + 70 || er.bottom > dr.bottom - 50) { + const delta = (er.top - dr.top) - doc.clientHeight * 0.38; + doc.scrollTo({ top: doc.scrollTop + delta, behavior: 'smooth' }); + } +} + +function readerHighlightSentence(sentence) { + if (!sentence) return; + if (readerState.mode === 'text') readerEnsureVisible(sentence.words[0]?.el); + else if (sentence._stat) readerEnsureVisible(sentence._stat); +} + +function readerHighlightWord(sentence, wi) { + const w = sentence.words[wi]; if (!w) return; + if (readerState.mode === 'text') { + document.querySelectorAll('.reader-word.is-word').forEach(el => el.classList.remove('is-word')); + if (w.el) { w.el.classList.add('is-word'); readerEnsureVisible(w.el); } + return; + } + readerEnsureBox(); + const pg = readerState.pages[w.page]; if (!pg) return; + const scale = readerState.scale; + if (_readerWordBox.parentNode !== pg.overlay) pg.overlay.appendChild(_readerWordBox); + Object.assign(_readerWordBox.style, { left: (w.x * scale) + 'px', top: (w.top * scale) + 'px', width: (w.w * scale) + 'px', height: (w.h * scale) + 'px' }); + _readerWordBox.hidden = false; + readerEnsureVisible(_readerWordBox); +} + +// ── Scope: select range / page range / all + batch synthesis ───────────────── + +// Select mode STAYS ON until the user explicitly finishes (Done button / Esc). +// Step 1: click a start sentence (the "anchor"). Step 2: click the end sentence. +// The anchor pulses while waiting; after a range is set, the next click starts a +// fresh anchor so you can keep refining without leaving the mode. +function readerSetSelecting(on) { + readerState.selecting = !!on; + readerState.selAnchored = false; + const doc = $('reader-doc'); if (doc) doc.classList.toggle('reader-selecting', readerState.selecting); + const btn = $('reader-select-toggle'); if (btn) btn.classList.toggle('active', readerState.selecting); + const lbl = $('reader-select-label'); if (lbl) lbl.textContent = readerState.selecting ? 'Done' : 'Select range'; + const ic = $('reader-select-icon'); if (ic) ic.className = 'mdi ' + (readerState.selecting ? 'mdi-check' : 'mdi-cursor-default-click-outline'); + readerSelHint(readerState.selecting ? 'Click the start sentence…' : ''); + if (!readerState.selecting) readerPaintSelection(); // drop the transient anchor styling +} + +function readerSelHint(html) { + const h = $('reader-sel-hint'); if (!h) return; + h.innerHTML = html; h.hidden = !html; +} + +function readerPickSentence(si) { + if (!readerState.selAnchored) { + // start a fresh range + readerState.selStart = si; readerState.selEnd = si; readerState.selAnchored = true; + readerSelHint('Start at sentence ' + (si + 1) + ' β€” now click the end sentence'); + } else { + readerState.selEnd = si; readerState.selAnchored = false; + const r = readerSelRange(); + readerSelHint('Selected ' + (r[1] - r[0] + 1) + ' sentences β€” click a new start, or Done'); + } + readerPaintSelection(); + readerUpdateScopeLabel(); +} + +function readerClearSelection() { + readerState.selStart = readerState.selEnd = null; readerState.selAnchored = false; + readerPaintSelection(); + readerUpdateScopeLabel(); + if (readerState.selecting) readerSelHint('Click the start sentence…'); +} + +function readerSelRange() { + if (readerState.selStart === null || readerState.selEnd === null) return null; + return [Math.min(readerState.selStart, readerState.selEnd), Math.max(readerState.selStart, readerState.selEnd)]; +} + +function readerPaintSelection() { + const range = readerSelRange(); + const inSel = i => range && i >= range[0] && i <= range[1]; + // While waiting for the second click, the lone start sentence is the "anchor". + const anchorI = (readerState.selAnchored && range && range[0] === range[1]) ? range[0] : -1; + readerState.sentences.forEach((s, i) => { + const sel = inSel(i) && i !== anchorI; + const anchor = i === anchorI; + if (readerState.mode === 'pdf') { + if (s._stat) { s._stat.classList.toggle('sel', sel); s._stat.classList.toggle('sel-anchor', anchor); } + } else { + s.words.forEach(w => { if (w.el) { w.el.classList.toggle('in-sel', sel); w.el.classList.toggle('sel-anchor', anchor); } }); + } + }); +} + +// Indices to act on: an active selection wins, else PDF page range, else everything. +function readerScopeIndices() { + const all = readerState.sentences.map((_, i) => i); + const range = readerSelRange(); + if (range) return all.filter(i => i >= range[0] && i <= range[1]); + if (readerState.mode === 'pdf') { + const n = readerState.pages.length; + let from = parseInt($('reader-page-from')?.value) || 1; + let to = parseInt($('reader-page-to')?.value) || n; + from = Math.max(1, Math.min(from, n)); + to = Math.max(from, Math.min(to, n)); + return all.filter(i => { const p = (readerState.sentences[i].words[0]?.page ?? 0) + 1; return p >= from && p <= to; }); + } + return all; +} + +function readerUpdateScopeLabel() { + const scopeEl = $('reader-synth-scope'); + const range = readerSelRange(); + if (scopeEl) { + if (range) scopeEl.textContent = 'selection'; + else if (readerState.mode === 'pdf') scopeEl.textContent = 'pages'; + else scopeEl.textContent = 'all'; + } + const info = $('reader-sel-info'), txt = $('reader-sel-text'); + if (info) info.hidden = !range; + if (txt && range) txt.textContent = 'sentences ' + (range[0] + 1) + '–' + (range[1] + 1) + ' (' + (range[1] - range[0] + 1) + ')'; +} + +// Synthesise the given sentence indices (skips already-cached). Returns count done. +async function readerSynthIndices(targets) { + targets = targets.filter(i => !readerState.blobCache.has(i)); + if (!targets.length || readerState.synthRunning) return 0; + const voice = $('reader-voice-select')?.value; + const backend = $('reader-backend-select')?.value; + if (!voice) { toast('Pick a voice first', 'error'); return 0; } + if (!backend) { toast('No TTS backend selected', 'error'); return 0; } + const instruct = $('reader-instruct')?.value.trim() || ''; + + readerState.synthRunning = true; readerState.synthCancel = false; + const prog = $('reader-synth-prog'); if (prog) prog.hidden = false; + const total = targets.length; let done = 0; + const update = () => { + const f = $('reader-synth-fill'); if (f) f.style.width = (done / total * 100) + '%'; + const l = $('reader-synth-label'); if (l) l.textContent = done + ' / ' + total; + }; + update(); + + const queue = targets.slice(); + const worker = async () => { + while (queue.length && !readerState.synthCancel) { + const i = queue.shift(); + if (readerState.blobCache.has(i)) { done++; update(); continue; } + if (readerState.sentences[i].status === 'pending') readerSetStatus(i, 'synth'); + try { + const blob = await fetchTtsPreviewBlob(voice, readerState.sentences[i].text, READER_FMT, instruct, backend, false, readerGenParams()); + readerState.blobCache.set(i, blob); + if (readerState.sentences[i].status === 'synth') readerSetStatus(i, 'ready'); + } catch (_) { + if (readerState.sentences[i].status === 'synth') readerSetStatus(i, 'pending'); + } + done++; update(); + } + }; + const N = Math.min(2, targets.length); // bounded concurrency β€” don't overload the engine + await Promise.all(Array.from({ length: N }, worker)); + + readerState.synthRunning = false; + if (prog) prog.hidden = true; + return done; +} + +async function readerSynthAll() { + const targets = readerScopeIndices().filter(i => !readerState.blobCache.has(i)); + if (!targets.length) { toast('Selected range is already synthesised', 'success'); return; } + const done = await readerSynthIndices(targets); + if (done || !readerState.synthCancel) toast(readerState.synthCancel ? 'Synthesis cancelled (' + done + ' done)' : 'Synthesised ' + done + ' sentences', readerState.synthCancel ? 'error' : 'success'); +} + +// ── Export as MP3 (per page, or per sentence) with meaningful filenames ────── + +function readerSafeName(s) { + return (s || 'audio').replace(/[\/\\:*?"<>|]+/g, '_').replace(/\s+/g, ' ').trim().slice(0, 60) || 'audio'; +} +function readerPad(n) { return String(n).padStart(2, '0'); } +function readerPageOf(i) { return readerState.mode === 'pdf' ? (readerState.sentences[i].words[0]?.page ?? 0) + 1 : 1; } + +function readerDownload(blob, filename) { + const a = document.createElement('a'); + a.href = URL.createObjectURL(blob); + a.download = filename; + document.body.appendChild(a); + a.click(); + setTimeout(() => { URL.revokeObjectURL(a.href); a.remove(); }, 1500); +} +const _readerDelay = ms => new Promise(r => setTimeout(r, ms)); + +async function readerExport(mode) { + if (!readerState.sentences.length) { toast('Import a document first', 'error'); return; } + const indices = readerScopeIndices(); + const missing = indices.filter(i => !readerState.blobCache.has(i)); + if (missing.length) { + toast('Synthesising ' + missing.length + ' missing sentence(s) before export…', 'success'); + await readerSynthIndices(missing); + if (readerState.synthCancel) { toast('Export cancelled', 'error'); return; } + } + const ready = indices.filter(i => readerState.blobCache.has(i)); + if (!ready.length) { toast('Nothing to export', 'error'); return; } + const title = readerSafeName(readerState.title); + + if (mode === 'sentence') { + const perPage = {}; + for (const i of ready) { + const pg = readerPageOf(i); + perPage[pg] = (perPage[pg] || 0) + 1; + const name = readerState.mode === 'pdf' + ? `${title} - p${readerPad(pg)} - ${readerPad(perPage[pg])}.mp3` + : `${title} - ${readerPad(perPage[pg])}.mp3`; + readerDownload(readerState.blobCache.get(i), name); + await _readerDelay(350); + } + toast('Exported ' + ready.length + ' MP3 files', 'success'); + return; + } + + // mode === 'page' β†’ one combined MP3 per page (text mode β†’ one file for the doc) + const byPage = new Map(); + ready.forEach(i => { const pg = readerPageOf(i); if (!byPage.has(pg)) byPage.set(pg, []); byPage.get(pg).push(i); }); + let files = 0; + for (const [pg, idxs] of [...byPage.entries()].sort((a, b) => a[0] - b[0])) { + const blob = new Blob(idxs.map(i => readerState.blobCache.get(i)), { type: 'audio/mpeg' }); + const name = readerState.mode === 'pdf' ? `${title} - p${readerPad(pg)}.mp3` : `${title}.mp3`; + readerDownload(blob, name); + files++; + await _readerDelay(400); + } + toast('Exported ' + files + (readerState.mode === 'pdf' ? ' page MP3 file(s)' : ' MP3'), 'success'); +} + +// ── Chunk mode (voice consistency) ─────────────────────────────────────────── + +function readerRechunk(mode) { + readerState.chunkMode = mode; + if (!readerState.baseSentences.length) return; + readerStopPlayback(); + // drop synthesised audio (chunk text changed) and old per-unit status boxes + readerState.blobCache.clear(); readerState.bufCache.clear(); readerState.gainCache.clear(); + readerState.savedAudioIdx = new Set(); // chunking changed β†’ previously-saved audio no longer aligns + readerState.pages.forEach(p => p.overlay.querySelectorAll('.reader-stat').forEach(e => e.remove())); + readerClearSelection(); + readerState.sentences = readerGroupUnits(readerState.baseSentences, mode); + readerBuildUnitIndex(); + // recreate status boxes only for pages already rendered (rest come on raster) + readerState.pages.forEach((p, i) => { if (p.rendered) readerCreatePageStatus(i); }); + if (readerState.mode === 'text') readerState.sentences.forEach((s, i) => readerSetStatus(i, 'pending')); + readerState.idx = 0; + readerUpdateScopeLabel(); + readerUpdateProgress(); + readerHighlightSentence(readerState.sentences[0]); + toast('Voice consistency: ' + mode + ' β€” audio cleared, re-synthesise', 'success'); +} + +// ── Generation params (seed / temperature) + loudness normalisation ────────── + +function readerGenParams() { + const out = {}; + const seed = $('reader-seed')?.value.trim(); + const temp = $('reader-temp')?.value.trim(); + if (seed !== '' && seed != null && !isNaN(+seed)) out.seed = parseInt(seed, 10); + if (temp !== '' && temp != null && !isNaN(+temp)) out.temperature = parseFloat(temp); + return Object.keys(out).length ? out : null; +} + +// RMS-based gain so each chunk plays at a similar perceived loudness (clamped, +// peak-limited to avoid clipping). Cached per unit. +function readerComputeGain(idx, buf) { + if (readerState.gainCache.has(idx)) return readerState.gainCache.get(idx); + const ch = buf.getChannelData(0); + const step = Math.max(1, Math.floor(ch.length / 8000)); + let sum = 0, n = 0, peak = 1e-6; + for (let i = 0; i < ch.length; i += step) { const v = ch[i]; sum += v * v; n++; if (Math.abs(v) > peak) peak = Math.abs(v); } + const rms = Math.sqrt(sum / Math.max(1, n)) || 0.0001; + let gain = 0.10 / rms; // target RMS β‰ˆ 0.10 + gain = Math.max(0.5, Math.min(gain, 4)); + gain = Math.min(gain, 0.99 / peak); // never clip + readerState.gainCache.set(idx, gain); + return gain; +} + +// ── Playback ───────────────────────────────────────────────────────────────── + +// Fetch a chunk's audio that's already persisted on the server (synced device). +async function readerFetchServerAudio(idx) { + if (!readerState.savedId || !readerState.savedAudioIdx.has(idx)) return null; + try { + const r = await fetch(`${READER_API}/${readerState.savedId}/audio/${idx}`); + if (r.ok) { const b = await r.blob(); readerState.blobCache.set(idx, b); return b; } + } catch (_) {} + return null; +} + +async function readerGetBuffer(idx) { + if (readerState.bufCache.has(idx)) return readerState.bufCache.get(idx); + let blob = readerState.blobCache.get(idx) || await readerFetchServerAudio(idx); + if (!blob) { + const voice = $('reader-voice-select')?.value; + const backend = $('reader-backend-select')?.value; + if (!voice) throw new Error('Pick a voice first'); + if (!backend) throw new Error('No TTS backend selected'); + const instruct = $('reader-instruct')?.value.trim() || ''; + if (readerState.sentences[idx].status !== 'reading') readerSetStatus(idx, 'synth'); + blob = await fetchTtsPreviewBlob(voice, readerState.sentences[idx].text, READER_FMT, instruct, backend, false, readerGenParams()); + readerState.blobCache.set(idx, blob); + if (readerState.sentences[idx].status === 'synth') readerSetStatus(idx, 'ready'); + } + const buf = await readerCtx().decodeAudioData(await blob.arrayBuffer()); + readerState.bufCache.set(idx, buf); + return buf; +} + +// Keep decoded PCM (large) only for a small window around the playhead; the +// compact MP3 blobs stay cached so re-entry just re-decodes (cheap). +function readerEvictBuffers(center) { + if (readerState.bufCache.size <= READER_BUF_WINDOW * 2 + 1) return; + for (const i of readerState.bufCache.keys()) { + if (Math.abs(i - center) > READER_BUF_WINDOW) readerState.bufCache.delete(i); + } +} + +// Fetch the next chunk's audio AND pre-decode it so playback starts gaplessly. +async function readerPrefetch(idx) { + if (idx < 0 || idx >= readerState.sentences.length) return; + if (!readerState.blobCache.has(idx)) { + // prefer audio already persisted on the server, else synthesise + if (!(await readerFetchServerAudio(idx))) { + const voice = $('reader-voice-select')?.value; + const backend = $('reader-backend-select')?.value; + if (!voice || !backend) return; + const instruct = $('reader-instruct')?.value.trim() || ''; + if (readerState.sentences[idx].status === 'pending') readerSetStatus(idx, 'synth'); + try { + const b = await fetchTtsPreviewBlob(voice, readerState.sentences[idx].text, READER_FMT, instruct, backend, false, readerGenParams()); + readerState.blobCache.set(idx, b); + if (readerState.sentences[idx].status === 'synth') readerSetStatus(idx, 'ready'); + } catch (_) { if (readerState.sentences[idx].status === 'synth') readerSetStatus(idx, 'pending'); return; } + } + } + // pre-decode (only matters while it's the immediate look-ahead within the window) + if (!readerState.bufCache.has(idx) && Math.abs(idx - readerState.idx) <= READER_BUF_WINDOW) { + try { readerState.bufCache.set(idx, await readerCtx().decodeAudioData(await readerState.blobCache.get(idx).arrayBuffer())); } + catch (_) {} + } +} + +// Revert the single "reading" unit to its idle colour (no full-array scan). +function readerClearReading() { + const i = readerState.readingIdx; + if (i >= 0 && i < readerState.sentences.length && readerState.sentences[i].status === 'reading') { + readerSetStatus(i, readerState.blobCache.has(i) ? 'ready' : 'pending'); + } + readerState.readingIdx = -1; +} + +async function readerPlayCurrent() { + if (!readerState.playing) return; + if (readerState.idx >= readerState.sentences.length) { readerStopPlayback(); readerState.idx = 0; readerUpdateProgress(); return; } + const idx = readerState.idx; + const sentence = readerState.sentences[idx]; + readerUpdateProgress(); + readerSaveResume(); + readerState.readingIdx = idx; + readerSetStatus(idx, 'reading'); + readerHighlightSentence(sentence); + + let buf; + try { buf = await readerGetBuffer(idx); } + catch (e) { toast(e.message || String(e), 'error'); readerSetStatus(idx, 'pending'); readerStopPlayback(); return; } + if (!readerState.playing || readerState.idx !== idx) return; + readerEvictBuffers(idx); + + const timings = computeWordTimings(sentence.text, buf.duration); + const ctx = readerCtx(); + readerStopSource(); + const src = ctx.createBufferSource(); + src.buffer = buf; + src.playbackRate.value = readerState.speed; + if (readerState.normalize) { + const g = ctx.createGain(); + g.gain.value = readerComputeGain(idx, buf); + src.connect(g); g.connect(ctx.destination); + } else { + src.connect(ctx.destination); + } + readerState.currentSource = src; + const t0 = ctx.currentTime; + + readerPrefetch(idx + 1); + + src.onended = () => { + if (readerState.currentSource !== src) return; + readerState.currentSource = null; + if (readerState.raf) { cancelAnimationFrame(readerState.raf); readerState.raf = null; } + readerSetStatus(idx, 'ready'); // synthesised β†’ green + if (readerState.readingIdx === idx) readerState.readingIdx = -1; + if (!readerState.playing) return; + readerState.idx++; + readerPlayCurrent(); + }; + src.start(0); + + const tick = () => { + if (readerState.currentSource !== src) return; + const elapsed = (ctx.currentTime - t0) * readerState.speed; + let active = 0; + for (let i = timings.length - 1; i >= 0; i--) { if (elapsed >= timings[i].start) { active = i; break; } } + readerHighlightWord(sentence, Math.min(active, sentence.words.length - 1)); + readerState.raf = requestAnimationFrame(tick); + }; + readerState.raf = requestAnimationFrame(tick); +} + +function readerStopSource() { + if (readerState.currentSource) { try { readerState.currentSource.onended = null; readerState.currentSource.stop(0); } catch (_) {} readerState.currentSource = null; } + if (readerState.raf) { cancelAnimationFrame(readerState.raf); readerState.raf = null; } +} + +function readerStopPlayback() { + readerState.playing = false; + readerStopSource(); + readerClearHighlights(); + // a sentence left mid-read reverts to its cached/uncached colour + readerClearReading(); + readerUpdatePlayBtn(); +} + +function readerPlay() { + if (!readerState.sentences.length) { toast('Import a document first', 'error'); return; } + if (!$('reader-voice-select')?.value) { toast('Pick a voice first', 'error'); return; } + readerCtx(); + readerState.playing = true; + readerUpdatePlayBtn(); + readerPlayCurrent(); +} + +function readerPause() { + readerState.playing = false; + readerStopSource(); + readerSaveResume(); + readerPersistProgress(); + readerClearReading(); + readerUpdatePlayBtn(); +} + +function readerJumpTo(idx) { + readerState.idx = Math.max(0, Math.min(idx, readerState.sentences.length - 1)); + readerStopSource(); + readerClearHighlights(); + readerClearReading(); + readerUpdateProgress(); + readerSaveResume(); + if (readerState.playing) readerPlayCurrent(); + else readerHighlightSentence(readerState.sentences[readerState.idx]); +} + +function readerUpdatePlayBtn() { + const btn = $('reader-play'); if (!btn) return; + const ic = btn.querySelector('.mdi'); + if (ic) ic.className = 'mdi ' + (readerState.playing ? 'mdi-pause' : 'mdi-play'); +} + +function readerUpdateProgress() { + const total = readerState.sentences.length; + const cur = total ? readerState.idx + 1 : 0; + const lbl = $('reader-progress-label'); if (lbl) lbl.textContent = cur + ' / ' + total; + const fill = $('reader-progress-fill'); if (fill) fill.style.width = (total ? (readerState.idx / total) * 100 : 0) + '%'; +} + +// ── Resume (persist last position per document title) ──────────────────────── + +function readerSaveResume() { + if (!readerState.title || !readerState.sentences.length) return; + try { localStorage.setItem(READER_RESUME_KEY, JSON.stringify({ title: readerState.title, idx: readerState.idx, total: readerState.sentences.length })); } catch (_) {} +} + +function readerLoadResume() { + try { + const r = JSON.parse(localStorage.getItem(READER_RESUME_KEY) || 'null'); + if (r && r.title === readerState.title && r.idx > 0 && r.idx < readerState.sentences.length) { + readerState.idx = r.idx; + return true; + } + } catch (_) {} + return false; +} + +// ── Library (server-side under the config volume β†’ syncs across devices) ────── +// Each book lives at /reader_library// (meta.json + source + audio/). +// Audio is fetched per-unit on demand so opening a book is instant on any device. + +const READER_API = '/api/reader/docs'; + +function readerSyntheticCount() { + let n = 0; + for (let i = 0; i < readerState.sentences.length; i++) if (readerState.blobCache.has(i)) n++; + return n; +} + +async function readerSaveLibrary() { + if (!readerState.sentences.length) { toast('Import a document first', 'error'); return; } + if (readerState.mode === 'pdf' && !readerState.fileBlob) { toast('Original PDF unavailable to save', 'error'); return; } + const meta = { + title: readerState.title || 'Untitled', + kind: readerState.mode, + idx: readerState.idx, + voice: $('reader-voice-select')?.value || '', + backend: $('reader-backend-select')?.value || '', + speed: readerState.speed, + instruct: $('reader-instruct')?.value.trim() || '', + chunkMode: readerState.chunkMode, + seed: $('reader-seed')?.value.trim() || '', + temperature: $('reader-temp')?.value.trim() || '', + normalize: readerState.normalize, + sentenceCount: readerState.sentences.length, + pageCount: readerState.pages.length, + synthCount: readerSyntheticCount(), + updated: new Date().toISOString(), + }; + const btn = $('reader-save-lib'); if (btn) btn.disabled = true; + try { + if (readerState.savedId) meta.id = readerState.savedId; + const r = await fetch(READER_API, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(meta) }); + if (!r.ok) throw new Error((await r.json().catch(() => ({}))).detail || r.statusText); + const fresh = !readerState.savedId; + readerState.savedId = (await r.json()).id; + // upload the source document once (it's needed to re-render on another device) + if (fresh || !readerState.sourceUploaded) { + const ext = readerState.mode === 'pdf' ? 'pdf' : 'txt'; + const body = readerState.mode === 'pdf' ? readerState.fileBlob : new Blob([readerState.docText || ''], { type: 'text/plain' }); + const sr = await fetch(`${READER_API}/${readerState.savedId}/source?ext=${ext}`, { method: 'PUT', body }); + if (sr.ok) readerState.sourceUploaded = true; + } + // upload ONLY chunks not already on the server (incremental) + let uploaded = 0; + for (let i = 0; i < readerState.sentences.length; i++) { + if (readerState.blobCache.has(i) && !readerState.savedAudioIdx.has(i)) { + const ar = await fetch(`${READER_API}/${readerState.savedId}/audio/${i}`, { method: 'PUT', body: readerState.blobCache.get(i) }); + if (ar.ok) { readerState.savedAudioIdx.add(i); uploaded++; } + } + } + toast('Saved (' + meta.synthCount + ' / ' + meta.sentenceCount + ' synthesised' + (uploaded ? ', +' + uploaded + ' new' : '') + ')', 'success'); + readerRenderLibrary(); + } catch (e) { toast('Save failed: ' + (e.message || e), 'error'); } + finally { if (btn) btn.disabled = false; } +} + +// Light update of just the reading position (called on stop/pause/leave) +async function readerPersistProgress() { + if (!readerState.savedId) return; + try { + await fetch(`${READER_API}/${readerState.savedId}/progress`, { + method: 'PATCH', headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ idx: readerState.idx, updated: new Date().toISOString() }), + }); + } catch (_) {} +} + +async function readerDeleteLibraryDoc(id) { + try { + await fetch(`${READER_API}/${id}`, { method: 'DELETE' }); + if (readerState.savedId === id) { readerState.savedId = null; readerState.savedAudioIdx = new Set(); readerState.sourceUploaded = false; } + readerRenderLibrary(); + } catch (e) { toast('Delete failed: ' + (e.message || e), 'error'); } +} + +async function readerOpenLibraryDoc(id) { + let rec; + try { const r = await fetch(`${READER_API}/${id}`); if (!r.ok) throw new Error(r.statusText); rec = await r.json(); } + catch (e) { toast('Open failed: ' + (e.message || e), 'error'); return; } + readerResetDoc(); + readerState.title = rec.title; + const titleEl = $('reader-doc-title'); if (titleEl) titleEl.textContent = rec.title; + // restore controls (chunk mode must be set BEFORE the document is grouped) + readerState.chunkMode = rec.chunkMode || 'sentence'; + if ($('reader-chunk-mode')) $('reader-chunk-mode').value = readerState.chunkMode; + if ($('reader-seed')) $('reader-seed').value = rec.seed || ''; + if ($('reader-temp')) $('reader-temp').value = rec.temperature || ''; + readerState.normalize = rec.normalize !== false; + if ($('reader-normalize')) $('reader-normalize').checked = readerState.normalize; + if (rec.backend && $('reader-backend-select')) readerSetSelectValue('reader-backend-select', rec.backend); + if (rec.voice && $('reader-voice-select')) readerSetSelectValue('reader-voice-select', rec.voice); + if (typeof rec.speed === 'number') { + readerState.speed = rec.speed; + if ($('reader-speed')) $('reader-speed').value = rec.speed; + const sl = $('reader-speed-label'); if (sl) sl.textContent = rec.speed.toFixed(2).replace(/0$/, '') + 'Γ—'; + } + if ($('reader-instruct')) $('reader-instruct').value = rec.instruct || ''; + // fetch the source document from the server + let sourceBlob = null; + try { const sr = await fetch(`${READER_API}/${id}/source`); if (sr.ok) sourceBlob = await sr.blob(); } catch (_) {} + try { + if (rec.kind === 'pdf') { + if (!sourceBlob) { toast('Saved PDF data missing', 'error'); return; } + readerState.fileBlob = sourceBlob; + toast('Opening book…', 'success'); + await readerLoadPdf(sourceBlob); + } else { + const text = sourceBlob ? await sourceBlob.text() : (rec.text || ''); + readerState.docText = text; + readerLoadText(text); + } + } catch (e) { toast('Open failed: ' + (e.message || e), 'error'); return; } + if (!readerState.sentences.length) { toast('Document had no readable text', 'error'); return; } + if (readerState.mode === 'text') readerState.sentences.forEach((s, i) => readerSetStatus(i, 'pending')); + readerState.savedId = id; + readerState.sourceUploaded = true; // already on the server + readerState.savedAudioIdx = new Set(); + // mark which units already have audio on the server (fetched lazily on play) + if (rec.sentenceCount === readerState.sentences.length) { + (rec.audioIdx || []).forEach(i => { if (i < readerState.sentences.length) { readerState.savedAudioIdx.add(i); readerSetStatus(i, 'ready'); } }); + } else if ((rec.audioIdx || []).length) { + toast('Document changed β€” re-synthesis needed', 'error'); + } + readerState.idx = Math.max(0, Math.min(rec.idx || 0, readerState.sentences.length - 1)); + $('reader-transport').hidden = false; + $('reader-dropzone').hidden = true; + $('reader-toolbar').hidden = readerState.mode !== 'pdf'; + $('reader-synthbar').hidden = false; + const pr = $('reader-page-range'); + if (pr) { + pr.hidden = readerState.mode !== 'pdf'; + if (readerState.mode === 'pdf') { + const n = readerState.pages.length; + if ($('reader-page-from')) { $('reader-page-from').max = n; $('reader-page-from').value = 1; } + if ($('reader-page-to')) { $('reader-page-to').max = n; $('reader-page-to').value = n; } + } + } + readerUpdateScopeLabel(); + readerUpdateProgress(); + readerHighlightSentence(readerState.sentences[readerState.idx]); + toast('Opened "' + rec.title + '" Β· resuming at sentence ' + (readerState.idx + 1), 'success'); +} + +function readerSetSelectValue(id, value) { + const sel = $(id); if (!sel) return; + if (![...sel.options].some(o => o.value === value)) { + const o = document.createElement('option'); o.value = o.textContent = value; sel.appendChild(o); + } + sel.value = value; +} + +async function readerRenderLibrary() { + const card = $('reader-library-card'), list = $('reader-lib-list'); + if (!card || !list) return; + let all = []; + try { const r = await fetch(READER_API); if (r.ok) all = (await r.json()).docs || []; } catch (_) { all = []; } + if (!all.length) { card.hidden = true; list.innerHTML = ''; return; } + card.hidden = false; + list.innerHTML = all.map(rec => { + const total = rec.sentenceCount || 0; + const synth = rec.synthCount || 0; + const readPct = total ? Math.round(((rec.idx || 0) / total) * 100) : 0; + const synthPct = total ? Math.round((synth / total) * 100) : 0; + const date = rec.updated ? new Date(rec.updated).toLocaleDateString() : ''; + return `
+ +
+
${escHtml(rec.title)}
+
${total} sentences${rec.pageCount ? ' Β· ' + rec.pageCount + ' pg' : ''} Β· ${date}
+
+
${synthPct}% audio ${readPct}% read
+
`; + }).join(''); + list.querySelectorAll('.reader-book').forEach(el => el.addEventListener('click', e => { + if (e.target.closest('.reader-book-del')) return; + readerOpenLibraryDoc(el.dataset.id); + })); + list.querySelectorAll('.reader-book-del').forEach(btn => btn.addEventListener('click', e => { + e.stopPropagation(); + if (confirm('Delete this saved document and its audio?')) readerDeleteLibraryDoc(btn.dataset.del); + })); +} + +// ── Wiring ─────────────────────────────────────────────────────────────────── + +$('reader-file-input')?.addEventListener('change', function () { + const f = this.files?.[0]; this.value = ''; + readerImportFile(f); +}); + +$('reader-fetch-voices-btn')?.addEventListener('click', readerFetchVoices); +$('reader-backend-select')?.addEventListener('change', () => { + const sel = $('reader-voice-select'); if (sel) sel.innerHTML = ''; + readerUpdateBackendHint(); +}); +$('reader-chunk-mode')?.addEventListener('change', function () { readerRechunk(this.value); }); +$('reader-normalize')?.addEventListener('change', function () { readerState.normalize = this.checked; }); + +$('reader-play')?.addEventListener('click', () => { readerState.playing ? readerPause() : readerPlay(); }); +$('reader-stop')?.addEventListener('click', () => { readerStopPlayback(); readerState.idx = 0; readerSaveResume(); readerUpdateProgress(); }); +$('reader-prev')?.addEventListener('click', () => readerJumpTo(readerState.idx - 1)); +$('reader-next')?.addEventListener('click', () => readerJumpTo(readerState.idx + 1)); + +$('reader-speed')?.addEventListener('input', function () { + readerState.speed = parseFloat(this.value) || 1; + const lbl = $('reader-speed-label'); if (lbl) lbl.textContent = readerState.speed.toFixed(2).replace(/0$/, '') + 'Γ—'; + if (readerState.currentSource) readerState.currentSource.playbackRate.value = readerState.speed; +}); + +// Zoom controls +$('reader-zoom-fitw')?.addEventListener('click', () => readerApplyZoom('fit-width')); +$('reader-zoom-fith')?.addEventListener('click', () => readerApplyZoom('fit-height')); +$('reader-zoom-two') ?.addEventListener('click', () => readerApplyZoom('two')); +$('reader-zoom-in') ?.addEventListener('click', () => { readerState.scale = Math.min(4, readerState.scale * 1.2); readerApplyZoom('custom'); }); +$('reader-zoom-out') ?.addEventListener('click', () => { readerState.scale = Math.max(0.2, readerState.scale / 1.2); readerApplyZoom('custom'); }); + +// Library + export +$('reader-save-lib')?.addEventListener('click', readerSaveLibrary); +$('reader-export-btn')?.addEventListener('click', () => readerExport($('reader-export-mode')?.value || 'page')); + +// Batch synthesis + range selection +$('reader-synth-all')?.addEventListener('click', readerSynthAll); +$('reader-synth-cancel')?.addEventListener('click', () => { readerState.synthCancel = true; }); +$('reader-select-toggle')?.addEventListener('click', () => { + if (readerState.selecting) readerSetSelecting(false); // "Done" + else { readerClearSelection(); readerSetSelecting(true); } +}); +$('reader-sel-clear')?.addEventListener('click', readerClearSelection); +// Esc leaves select mode +document.addEventListener('keydown', e => { + if (e.key === 'Escape' && readerState.selecting) { readerSetSelecting(false); } +}); +$('reader-page-from')?.addEventListener('change', readerUpdateScopeLabel); +$('reader-page-to') ?.addEventListener('change', readerUpdateScopeLabel); +// Click a PDF sentence's status box to set a selection point (in select mode) +$('reader-doc')?.addEventListener('click', e => { + if (!readerState.selecting || readerState.mode !== 'pdf') return; + const stat = e.target.closest('.reader-stat'); if (!stat) return; + const si = parseInt(stat.dataset.si); + if (!isNaN(si)) readerPickSentence(si); +}); + +// Re-raster pages that scroll into view (zoom invalidates cached canvases) +$('reader-doc')?.addEventListener('scroll', (function () { + let t = null; + return () => { if (t) return; t = setTimeout(() => { t = null; readerRenderVisible(); }, 120); }; +})()); + +// Keep fit-width/height/two correct on window resize +window.addEventListener('resize', (function () { + let t = null; + return () => { if (readerState.mode !== 'pdf' || readerState.zoomMode === 'custom') return; clearTimeout(t); t = setTimeout(() => readerApplyZoom(readerState.zoomMode), 200); }; +})()); + +// Drag & drop over the dropzone + the whole reader doc +(function setupReaderDropzone() { + const targets = ['reader-dropzone', 'reader-doc'].map(id => $(id)).filter(Boolean); + targets.forEach(el => { + el.addEventListener('dragover', e => { if (e.dataTransfer && [...(e.dataTransfer.types || [])].includes('Files')) { e.preventDefault(); el.classList.add('dragover'); } }); + el.addEventListener('dragleave', () => el.classList.remove('dragover')); + el.addEventListener('drop', e => { + if (!(e.dataTransfer && [...(e.dataTransfer.types || [])].includes('Files'))) return; + e.preventDefault(); el.classList.remove('dragover'); + readerImportFile(e.dataTransfer.files?.[0]); + }); + }); +})(); diff --git a/static/js/settings.js b/static/js/settings.js index aa221ca..b435f5a 100644 --- a/static/js/settings.js +++ b/static/js/settings.js @@ -171,6 +171,13 @@ async function refreshTtsBackendAvailability(selected = '') { libraryTts.innerHTML = ttsBackendOptions(prev); libraryTts.disabled = !availableTtsBackends().length; } + const readerBackend = $('reader-backend-select'); + if (readerBackend) { + const prev = selected || readerBackend.value; + readerBackend.innerHTML = ttsBackendOptions(prev); + readerBackend.disabled = !availableTtsBackends().length; + if (typeof readerUpdateBackendHint === 'function') readerUpdateBackendHint(); + } document.querySelectorAll('.opt-style-backend').forEach(sel => { const prev = sel.value; sel.innerHTML = styleBackendOptions(prev); diff --git a/static/js/tts-preview.js b/static/js/tts-preview.js index 5070e7f..60672d2 100644 --- a/static/js/tts-preview.js +++ b/static/js/tts-preview.js @@ -211,9 +211,10 @@ async function createTtsStreamUrl(voice, text, instruct = '') { const data = await r.json(); return data.url; } -async function fetchTtsPreviewBlob(voice, text, responseFormat = 'wav', instruct = '', backend = 'voice_clone', applyPersona = false) { +async function fetchTtsPreviewBlob(voice, text, responseFormat = 'wav', instruct = '', backend = 'voice_clone', applyPersona = false, extra = null) { const body = {text, voice, response_format: responseFormat, instruct, backend}; if (applyPersona) body.apply_persona = true; + if (extra && typeof extra === 'object') Object.assign(body, extra); // e.g. {seed, temperature} const r = await fetch('/api/tts-preview', {method:'POST', headers:{'Content-Type':'application/json'}, body: JSON.stringify(body)}); if (!r.ok) { const e=await r.json().catch(()=>({})); throw new Error(e.detail || r.statusText); } return await r.blob(); diff --git a/static/js/voice-library.js b/static/js/voice-library.js index f202c08..d8a0e73 100644 --- a/static/js/voice-library.js +++ b/static/js/voice-library.js @@ -799,6 +799,44 @@ $('copy-active-voices-btn').addEventListener('click', async () => { status('Copied active voices to clipboard'); }); +// Precompute speaker embeddings: fire a tiny synth per active voice so the TTS +// engine computes + caches each voice's .pt (speaker fingerprint) ahead of time, +// making first real playback instant. The engine prefers the cached .pt and only +// (re)builds it from the wav when missing β€” so this is purely a warm-up. +$('precompute-embeddings-btn')?.addEventListener('click', async () => { + const backend = libraryTtsBackend(); + const ids = activeVoiceIds(); + if (!ids.length) { toast('No active voices to precompute', 'error'); return; } + if (!confirm(`Precompute speaker embeddings for ${ids.length} active voice(s) via β€œ${backend}”?\n\nThis warms each voice so the engine caches its .pt and first playback is instant.`)) return; + + const btn = $('precompute-embeddings-btn'); if (btn) btn.disabled = true; + const ov = document.createElement('div'); + ov.className = 'audiobook-overlay'; ov.id = 'precompute-overlay'; + ov.innerHTML = `
Precomputing embeddings
+
0 / ${ids.length}
+
+
`; + document.body.appendChild(ov); + let cancel = false; + ov.querySelector('#pc-cancel').addEventListener('click', () => { cancel = true; }); + const fill = ov.querySelector('#pc-fill'), msg = ov.querySelector('#pc-msg'); + let done = 0, ok = 0, failed = 0; + const queue = ids.slice(); + const worker = async () => { + while (queue.length && !cancel) { + const id = queue.shift(); + if (msg) msg.textContent = `${done} / ${ids.length} Β· ${id}`; + try { await fetchTtsPreviewBlob(id, 'Hallo.', 'wav', '', backend); ok++; } + catch (_) { failed++; } + done++; if (fill) fill.style.width = (done / ids.length * 100) + '%'; + } + }; + try { await Promise.all(Array.from({ length: Math.min(2, ids.length) }, worker)); } + finally { ov.remove(); if (btn) btn.disabled = false; } + toast(cancel ? `Cancelled β€” ${ok} warmed` : `Precomputed ${ok} embedding(s)${failed ? `, ${failed} skipped/failed` : ''}`, + (!ok && failed) ? 'error' : 'success'); +}); + const LIB_ADD_SAMPLE_TEXTS = { EN: 'The clear morning light warmed the quiet studio as I described a silver train, a bright red apple, and the gentle rhythm of rain on the window.', DE: 'Das klare Morgenlicht waermte das ruhige Studio, waehrend ich einen silbernen Zug, einen roten Apfel und den sanften Rhythmus des Regens am Fenster beschrieb.', diff --git a/static/nav.js b/static/nav.js index 8426899..8c40dca 100644 --- a/static/nav.js +++ b/static/nav.js @@ -18,7 +18,7 @@ llms: 's-llms' }; - const SECTIONS = ['s-voices', 's-clone', 's-design', 's-studio', 's-tryout', 's-rehearser', 's-performance', 's-routing', 's-connect', 's-settings', 's-llms', 's-conversation']; + const SECTIONS = ['s-voices', 's-clone', 's-design', 's-studio', 's-tryout', 's-rehearser', 's-reader', 's-performance', 's-routing', 's-connect', 's-settings', 's-llms', 's-conversation']; function storedSection() { try { return localStorage.getItem('ttsvc_section') || ''; } catch (_) { return ''; } @@ -94,6 +94,9 @@ function showSection(sectionId) { if (!SECTIONS.includes(sectionId)) sectionId = 's-voices'; + // Leaving the Read Aloud reader: stop playback so audio doesn't keep running + if (sectionId !== 's-reader' && typeof window.readerStop === 'function') window.readerStop(); + if (sectionId === 's-reader' && typeof window.readerOnShow === 'function') window.readerOnShow(); setStoredSection(sectionId); setSectionHash(sectionId); SECTIONS.forEach(function (id) { diff --git a/static/sections/s-reader.html b/static/sections/s-reader.html new file mode 100644 index 0000000..617cc42 --- /dev/null +++ b/static/sections/s-reader.html @@ -0,0 +1,147 @@ +
+ +
+

Read Aloud

+

Import a PDF or text document, pick a voice and speed, then have it read to you while the word being spoken is highlighted.

+
+
+ +
+ + +
+
+
+ + +
+
+ +
+ + +
+
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+ + +
+
+
+ + + + + +
+ + Drag & drop a document here + .pdf Β· .txt Β· .md β€” or use the Import button above +
+ + + + + + + + + + + +
+ +
diff --git a/static/sections/s-rehearser.html b/static/sections/s-rehearser.html index ecb5f56..99506f9 100644 --- a/static/sections/s-rehearser.html +++ b/static/sections/s-rehearser.html @@ -311,6 +311,12 @@ + + diff --git a/static/sections/s-voices.html b/static/sections/s-voices.html index abbf6a9..97db8d4 100644 --- a/static/sections/s-voices.html +++ b/static/sections/s-voices.html @@ -102,6 +102,7 @@ +