91 lines
18 KiB
Markdown
91 lines
18 KiB
Markdown
Script Rehearsal — Feature Inventory
|
|
1. Library phase (phase 1)
|
|
Files: s-rehearser.html #reh-phase-1, JS renderLibraryList(), rehDbGetAll/rehDbAdd/rehDbPut, loadRecord() (exposed globally as window.rehLoadRecord), bookCover(), rehUniqueLibraryRecords(), rehTitleKey().
|
|
Shows saved rehearsals as a "bookshelf" (#reh-library-list.reh-bookshelf, list/shelf toggle #reh-lib-view-toggle), each with a generated cover (bookCover), dedup'd by title (rehUniqueLibraryRecords).
|
|
Script entry points: paste text into #reh-script-text + #reh-script-title → #reh-parse-btn → parseScript(); file upload (#reh-file-input txt/md, #reh-pdf-input pdf via importPDFScript, #reh-fdx-input fdx/osf via importFDX); drag-and-drop onto #reh-dropzone; .reh session import (#reh-lib-import-file, rehImportAnyFile); IMSDb/Scripts.com browser modal (#reh-imsdb-modal) fetching scripts from the web; and opening a saved record from the Library section or Audiobook (loadRecord/window.rehLoadRecord, called by library.js and audiobook.js).
|
|
rehState shape (defined ~line 64):
|
|
|
|
lines:[], cast:{}, lineIndex:0, clips:[], voices:[], backend:'',
|
|
playing:false, repeat:false, savedId:null, synthCache:Map, staleLines:Set,
|
|
synthCancelled:false, synthRunning:false, skipDescriptions:true, narratorVoice:'',
|
|
practiceStart:null, practiceEnd:null, bulkMode:false, bulkSel:Set, bulkAnchor:null,
|
|
showHidden:false, plus mic-recording fields (recStream, recAudioCtx, mediaRec, recChunks, ...)
|
|
2. Cast phase (phase 2)
|
|
Per-character cards (renderCastList() → #reh-cast-list, card/list view applyCastView): library-linked characters render via _charCardHtml() reused from the shared character library, wrapped in .reh-cast-card-libcard; unmatched speakers (including a synthetic Narrator entry, key REH_NARRATOR_KEY) get a fallback header/card.
|
|
"I play this" mic toggle: .reh-me-check checkbox per card, sets rehState.cast[sp].voice = 'me' (handled via _castApplyToLines/change listener ~line 1518); when voice==='me' the character is recorded live instead of TTS-synthesized.
|
|
Narrator handling: _ensureNarrator() keeps rehState.cast[REH_NARRATOR_KEY] bidirectionally synced with rehState.narratorVoice (a dedicated cast row, not a separate UI section anymore).
|
|
Voice-matching tools: #reh-matchlib-btn ("Match local" — LLM picks best existing library voice, no TTS engine needed), #reh-matchonline-btn ("Match online" — searches fish.audio, imports voices), #reh-autodesign-btn ("Design all" — generates new voices via Voice Design engine); all driven by rehResearchCast()/rehWriteCharacterNote()/_pickVoiceAvatar() and progress bar #reh-autodesign-progress.
|
|
Bulk/filter tools: search/filter/sort toolbar (#reh-cast-search, gender/lang filters, #reh-cast-sort, #reh-cast-sort-dir), _castSortFilter(); per-card actions include voice audition (_rehAudition, _rehPreviewLocal, _rehPreviewCastLine), assign local voice (_rehAssignLocal), online search results (_rehSearchOnline, _rehUseSearchResult, _rehImportFishCandidate), "develop" a character bio (_castDevelop), and delete (_castDeleteCharacter).
|
|
Buttons: Back (#reh-back-1-btn), Save to library (#reh-cast-save-btn), Open rehearsal (#reh-start-btn → showPhase(3)).
|
|
3. Stage phase (phase 3)
|
|
Rendering: buildScriptPage() builds #reh-script-lines from rehState.lines; each line has type ∈ act | scene | action | pagebreak | transition | direction | dialog (confirmed switch in buildScriptPage), each rendered with its own block class (.reh-act, .reh-scene, .reh-action-block, .reh-block-pagebreak, .reh-transition, .reh-direction, .reh-block for dialog).
|
|
Per-dialog-line controls: play/pause avatar button (_rehCharAvatarHtml, .reh-line-play-avatar), speaker name/color, TTS-vs-Me badge, emotion picker (openEmoPicker/selectEmotion/closeEmoPicker), inline text edit pencil (startInlineEdit, .reh-edit-btn), per-line note (.reh-note-btn/.reh-note-ta), synth-status dot and re-synth button (_markSynthDot, _showReSynthBtn).
|
|
Practice range markers: gutter button sets practiceStart/practiceEnd (updatePracticeRange, clearPracticeRange, setPracticeRangeFromSelection), shown in transport bar (#reh-practice-info).
|
|
Bulk edit mode: setBulkMode(), .reh-bulk-check per line, toolbar #reh-bulk-bar (select all/clear/set range/ignore/unignore/hide/delete/show-hidden/done) backed by _toggleBulkSel, _bulkApply, _bulkDelete, _reindexLineState.
|
|
Page-mode / A4 rendering: applyPageMode(), cyclePageMode(), paginateScript(), #reh-page-mode-btn (scroll / A4 pages / PDF-style pages), font size controls (rehApplyStageFont, #reh-font-inc/dec).
|
|
Character sidebar strip: renderCastStrip() → #reh-cast-strip, collapsible (rehToggleCast, #reh-cast-toggle), with its own search/sort (#reh-cast-side-search, #reh-cast-side-sort).
|
|
Top toolbar (director's console, #reh-transport-bar): transport (prev/play/stop/next/loop), progress label; action buttons — Synth all (#reh-tb-synth-all → synthAll()), Audiobook (#reh-tb-audiobook → audiobookExport() in audiobook.js), Character sheets (#reh-charsheets-btn → csForRehearser() in character-sheets.js), Re-synth stale (#reh-tb-resynth-stale), Skip-desc toggle, Bulk-edit toggle (#reh-bulk-toggle), Edit script/title (#reh-edit-script-btn → openScriptEditorModal), Save (#reh-tb-save), Export .reh/.fountain buttons, font size, page-mode toggle, Train mode (#reh-train-open-btn), Exit.
|
|
Also on stage: TTS now-playing bar (#reh-tts-status-bar), "my turn" mic recording overlay (#reh-rec-overlay, startRehMic/rehRenderMeter), and the Train Mode panel (#reh-train-panel, see §7).
|
|
4. Summary phase (phase 4)
|
|
renderSummary() → #reh-summary-list: lists all recorded/synthesized clips from the session.
|
|
Actions: Save to library, export .reh/.fountain/.fdx/.osf, Resume (back to stage), New script (reset to Library).
|
|
5. Audio pipeline
|
|
Per-line synth: synthOneLine(idx) (single line, on-demand) and synthAll() (batch pre-synth all TTS-eligible lines), both call fetchTtsPreviewBlob(voice, text, 'wav', instruct, backend) and cache results in rehState.synthCache (Map keyed by line index); stale entries (tone changed after synth) tracked in rehState.staleLines and surfaced via a re-synth button.
|
|
Playback: startPlay/pausePlay/stopPlay/playNextLine() drive line-by-line playback; preDecodeBlob/playPreDecoded/rehDecodedBuffers give zero-latency playback via pre-decoded PCM; computeWordTimings drives text highlighting.
|
|
Narrator confirmed: dialog lines always use their cast voice; non-dialog lines (action/scene/direction/transition/act, "any line with text") are read by rehState.narratorVoice — but only if a narrator voice is actually set. If no narrator voice is set, skipDescriptions (default true) skips them entirely during playback and synth (playNextLine/synthAll both check narratorVoice before narrating). Narrator is modeled as a regular cast entry keyed by sentinel REH_NARRATOR_KEY, kept in sync with rehState.narratorVoice via _ensureNarrator().
|
|
6. Persistence
|
|
saveToLibrary() builds a record via rehCurrentRecord() (title, reconstructed script text via linesToScriptText(), cast, per-line emotions/notes/ignored/hidden maps, backend, narratorVoice, lineIndex, clips) and calls rehDbPut/rehDbAdd (update vs insert based on rehState.savedId); also writes voice choices back into the shared character roster via castWriteBack().
|
|
rehDbGetAll/rehDbGetById/rehDbAdd/rehDbPut/rehDbDelete wrap REST calls to /api/rehearsals (routes/rehearsals_db.py), backed by SQLite table rehearsals in core/database.py (columns: id, title, script, cast, emotions, notes, ignored, hidden_lines, backend, narrator_voice, line_index, created, updated — cast/emotions/notes/ignored/hidden stored as JSON TEXT). _rehSanitize/clipsToJson/clipsFromJson handle audio blob (de)serialization for clips.
|
|
7. Other distinct features
|
|
Train mode (trainState object, ~line 4233): a practice mode separate from Stage — buildTrainSeq() builds cue/my-line turn sequence, trainGoTo() navigates turns, plays the other actor's cue (_trainPlayCueLines), records the user's line, transcribes via _trainTranscribe (speech-to-text), and compares expected vs actual text (_trainCompare/_trainLcs/_trainNormWords) with a score shown in #reh-train-result. Entered/exited via enterTrainMode()/exitTrainMode().
|
|
IMSDb/Scripts.com browser: modal #reh-imsdb-modal lets users browse ~1300 movie scripts or paste a URL, fetched server-side and parsed like any pasted script.
|
|
Import/Export panel (#reh-impex-panel): supports import from .txt/.md/.pdf/.fountain/.fdx/.osf/.reh and URL; export to .reh (full session incl. audio clips), .fountain, .fdx (Final Draft XML), .txt, .md, .osf (Open Screenplay Format) via exportFountain/exportFDX/exportOSF/exportAsTxt/exportAsMd/exportToFile.
|
|
Character sheets integration (character-sheets.js's csForRehearser) generates RPG-style actor sheets from the script's cast.
|
|
Audiobook export integration (audiobook.js's audiobookExport) synthesizes the whole script and packages it as chapterized MP3s; audiobook.js can also open a rehearsal directly via window.rehLoadRecord.
|
|
Now I have a complete picture. Here is the feature inventory.
|
|
|
|
Read Aloud — Feature Inventory
|
|
1. Source import
|
|
Paste text: #reader-paste-text textarea + #reader-paste-load button → readerImportPastedText() → readerLoadText(text). Sets readerState.docText, readerState.mode = 'text'.
|
|
File import (drag/drop or #reader-file-input): readerImportFile(file). .txt/.md → readerLoadText; .pdf → two-stage: readerLoadPdfSkeleton(file) (fast raster via pdf.js, no text layer yet, shows #reader-extract-banner) then explicit readerRunExtraction() → readerExtractPdfText(loaded) (runs getTextContent(), optional heading OCR via #reader-ocr-headings → readerOcrPageHeading/Tesseract.js, builds sentences).
|
|
Sentence/paragraph building: readerBuildSentences(words), readerMarkParagraphBreaks(pageWords) (sets .para/.paraStart), readerGroupUnits(base, mode) groups into synthesis chunks per chunkMode (sentence/paragraph/page, UI: #reader-chunk-mode).
|
|
Key readerState fields: mode ('pdf'|'text'), pdfDoc, pages[], sentences[] (synth units), baseSentences[] (raw), docText, fileBlob, chunkMode, _seq (import generation guard), savedId/savedAudioIdx/sourceUploaded (library linkage).
|
|
Document viewport: #reader-doc, zoom (readerApplyZoom, #reader-zoom-*), lazy raster (readerSetupLazyRaster, IntersectionObserver, readerEvictCanvases), in-doc search (readerSearchApply, #reader-search*).
|
|
Library persistence (separate from casting): readerSaveLibrary(), readerOpenLibraryDoc(id), READER_API = '/api/reader/docs' — stores meta/source/per-unit audio server-side; resume position via readerSaveResume/readerLoadResume (localStorage reader-resume).
|
|
2. Casting audiobook (speaker attribution)
|
|
Entry point: #reader-audiobook-btn → audiobookOpenCastView() (static/js/audiobook.js), which restores an in-memory/localStorage/server draft or starts fresh via audiobookCastView(...) (the live UI, #reader-audiobook-panel).
|
|
Scope text: audiobookScopeText() reads readerScopeIndices()/readerState.sentences (selection, PDF page-range, or whole doc) and builds page-mark offsets (_audiobook.pageMarks).
|
|
Pass 1 — initial cast: audiobookCast(overrideUrl, overrideModel, resume). Splits scope text into ~3000-char chunks (splitTextIntoChunks), for each chunk calls attributeChunk() → audiobookAttributeStream() (SSE /api/attribute-dialogue/stream, shows live LLM "thinking") falling back to blocking POST /api/attribute-dialogue; on failure retries by splitting the chunk in half; falls back to audiobookSplitByQuotes() (regex-only) if LLM unusable. Runs deterministic audiobookResolveUnknowns() per chunk (colon-rule / inquit heuristics), builds roster[]. Autosaves via _abSaveDraft every chunk / _abStartDraftAutosave.
|
|
Cleanup pass (end of audiobookCast, not LLM-based): _audiobookFixOrphanedQuoteMarks() → _audiobookMergeAdjacentSameSpeaker() (also _audiobookDedupNearbyDuplicates exists) — merges consecutive same-speaker segments, fixes stray quote boundaries.
|
|
Pass 2 — "Verify"/recast unknowns: audiobookRecastUnknown(overrideUrl, overrideModel, options) — re-resolves mechanical cases, then re-sends only Unknown-speaker (and optionally, via options.includeNarrator, all Narrator) lines in small context groups (audiobookRecastGroups/audiobookRecastContext) to the LLM, canonicalizing returned names against saved character-library aliases (clGetAllByTagOrBook, clSplitIdentityTokens).
|
|
State: _audiobook = { running, cancel, segments, roster, lastText, pageMarks, rehId, completedChunks, completedTotal, bookId } (global, window._audiobook).
|
|
Persists cast to Rehearsal library via audiobookSaveAsRehearsal({silent:true}) and hands off to Script Rehearser: audiobookShowPreview() → audiobookOpenCurrentInRehearser() / (legacy path) audiobookBuildScript(segments) + audiobookOpenInRehearser(script, title, dialogueEmotions) (fills #reh-script-text, calls navTo('s-rehearser'), clicks #reh-parse-btn).
|
|
3. Cast Characters (character sheet generation) — character-sheets.js
|
|
Distinct from dialogue casting: generates rich profiles, not speaker attribution. Entry csForReader(opts) (button "Cast from Reader" in #reader-charsheets-panel's empty state) — builds csReaderText() from readerScopeIndices(), seeds from existing library records (csSeedSheet, clGetAllByTagOrBook), attaches dialogue-line counts from _audiobook.segments (csAttachLineCounts), then calls csGenerate(text, cacheKey, knownRoster, {pageHost, seedSheets, lineCounts}) which streams passage-by-passage extraction (csGenerateStream) and merges results (csMerge, csConsolidateAliasDuplicates).
|
|
opts.fresh triggers a full recast discarding existing sheets.
|
|
csForReaderSelective(selectedNames) regenerates only picked characters (targeted evidence windows via csEvidenceWindowText/csParagraphBlocks).
|
|
Also csForRehearser() — same mechanism sourced from the Rehearser script instead (not in scope here).
|
|
Persists via csSaveToLibrary(book, sheets) → clUpsertMany (character library, shared with casting/Rehearser).
|
|
Renders in #reader-charsheets-panel via csShow(sheets, title, sourceText, book, hostEl); each sheet card built by csCardHtml; auto-generates external prompts (SillyTavern card / image-gen turnaround) via csAutoGenerateExternalPrompts as a background pass.
|
|
4. Cast / roster review UI (inside audiobookCastView, audiobook.js ~1313-3100)
|
|
Live feed (.ab-cv-feed inside #reader-audiobook-panel) shows attributed narration/dialogue rows as they stream in, with page-break dividers (view.pagemark).
|
|
Sidebar roster: renderRoster() — list of characters with line counts, colors, search filter, sort (count/alpha), persisted sort in localStorage ttsvc_ab_side_sort.
|
|
Per-row: click a speaker name to open a reassignment popup; _abOpenAliasPopup(name, anchorEl) lets you add an alias (writes via clUpsert(book, {name, aliases})) or merge two roster entries — _abMergeCharacters(fromName, intoName) relabels every matching segment, then _abPatchScatteredRows/_abRedrawSegmentsChunked repaints only affected rows.
|
|
_abOpenRecastSelectPopup(existingChars) / audiobookRecastSelectedCharacters(selectedNames) — pick specific characters to re-verify.
|
|
Export/handoff buttons inside the panel: recast unknowns ("Verify"), open/continue casting, export cast as Markdown (audiobookExportCastMd() → _abCastMarkdown(data)), and "Review & cast" → audiobookShowPreview().
|
|
5. TTS playback in Read Aloud itself (separate from audiobook generation)
|
|
Yes — Read Aloud has its own single-voice sentence-by-sentence playback with word highlighting, independent of the multi-speaker audiobook pipeline:
|
|
|
|
Transport bar #reader-transport (#reader-play, #reader-stop, #reader-prev/#reader-next, #reader-progress-fill).
|
|
readerPlay()/readerPause()/readerPlayCurrent() — fetches/caches per-unit audio (readerGetBuffer, fetchTtsPreviewBlob), computes word timings via computeWordTimings (reused from rehearser.js), and highlights the active word (readerHighlightWord) plus auto-scroll (readerEnsureVisible).
|
|
Uses readerCtx() (Web Audio API), readerComputeGain for RMS loudness normalization (#reader-normalize), readerState.speed (#reader-speed slider) for playback-rate control, readerGenParams() for seed/temperature/native-speed.
|
|
Batch pre-synthesis ("Synthesise ahead"): readerSynthAll()/readerSynthIndices(targets), scoped via readerScopeIndices() (selection / page range / all), shown in #reader-synthbar / #reader-synth-prog.
|
|
Per-unit status coloring (pending/synthesising/ready/reading) via readerSetStatus/readerPaintStatus, legend .reader-legend.
|
|
6. Other distinct features
|
|
Range selection: click-to-select start/end sentence (readerSetSelecting, readerPickSentence, #reader-select-toggle), used to scope both synthesis and casting.
|
|
MP3 export: readerExport(mode) — #reader-export-mode (page/sentence), #reader-export-btn; synthesizes any missing scoped units first and blocks export if any fail.
|
|
Document search: readerSearchApply/readerJumpToSearchHit (#reader-search*), independent of casting.
|
|
Voice-consistency ("chunk mode") re-grouping: readerRechunk(mode) — clears cached audio when granularity changes.
|
|
Backend/voice pickers + tuning: #reader-backend-select, #reader-voice-select (VoicePicker), #reader-instruct, seed/temperature/native-speed inputs, readerUpdateBackendHint() (warns about voice drift on non-style-aware backends).
|
|
Reader library (#reader-library-card/#reader-lib-list, readerRenderLibrary) — separate saved-books browser with cover-image drag/drop upload, delete-with-confirm overlay; distinct from the Character Library and Rehearsal Library.
|
|
View switching within the section: window.showReaderView(view) toggles reader-main-view / reader-audiobook-panel ('cast') / reader-charsheets-panel ('chars') / reader-library-card ('library'), driving refreshWorkflowCrumbs. Also window.readerJumpToPage(pageNum) for cross-linking from other sections (e.g. jump to a PDF page from a character sheet's evidence citation). |