mARTin-B78/tts-voice-creator-clone-and-design-2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3fffae9015
tts-voice-creator-clone-and.../docs/ARCHITECTURE.md

9.6 KiB
Raw Blame History

Architecture & optimization notes

Current shape

  • Backend: FastAPI, modular — server.py (app + middleware) → routes/*.py (admin, tts, library, sources, stt, conversation, docker, settings) → core/*.py (audio, voice, config, validation, tts_helpers, …).
  • Provider status: TTS/STT discovery responses expose configured independently from available/health, with only bounded probe endpoint, HTTP status, and sanitized error details returned to the UI.
  • Frontend: feature code is split into manifest-driven lazy modules in static/js/*.js, loaded in order by static/loader.js. Sections are HTML fragments in static/sections/*.html fetched at runtime. Legacy modules still communicate through compatibility globals (window._voices, $, escHtml, toast, saveMeta, loadVoiceLibrary, …), while extracted public APIs are published under window.TTVC.

Already optimised

  • GZip responses (server.py) — ~75% smaller text transfer.
  • Static caching middleware (immutable for versioned JS/CSS).
  • Native virtualization: content-visibility:auto on long-list rows (voice library, online cards, cast cards).
  • Lazy images (loading="lazy" decoding="async") across all big lists.
  • Rehearser memory cap: decoded PCM is kept to a sliding window (REH_DECODE_WINDOW) around the playhead.
  • AudioContext instances are closed (iOS limit).
  • PWA: manifest.webmanifest + network-first sw.js (served from root for / scope), installable, offline shell.
  • Concurrency pool (runPool in utils.js) for bulk network ops.
  • Smoke tests: Playwright (tests/, desktop + iPhone profiles).

Build tooling (opt-in) — DONE

npm run minify → scripts/minify.mjs concatenates the feature modules in loader order into static/dist/main.min.js and minifies whitespace/syntax only (identifiers kept — inline HTML onclick="navTo(...)" and cross-file refs rely on global names). loader.js loads this single bundle when window.APP_USE_BUNDLE === true (or ?bundle=1), else falls back to per-file loading — so editing static/js/* stays live by default. Verified end-to-end by tests/bundle.spec.js. Rebuild the bundle after JS edits if you ship with it enabled.

Staged migration plan (the two big refactors — do incrementally, behind the smoke tests)

1. Decouple globals → ES modules

The global scope + monkey-patching is the root cause of the recurring "wiring/timing" bugs (controls bound before the section mounts, etc.). Migrate one file at a time:

  1. Add "type": "module" loading path; keep a compat shim that re-exposes the few cross-module symbols on window during transition.
  2. Convert leaf modules first (pure Rehearser/Audiobook helpers are now privately scoped), exporting explicitly and importing where used.
  3. Replace the renderVoiceList monkey-patch with a normal function + an onRender hook.
  4. Switch loader.js to a single <script type="module"> entry that imports the graph; esbuild --bundle then produces one minified, tree-shaken file.

2. Split rehearser.js (was 196 KB) — RESPONSIBILITY SPLIT COMPLETE

Done incrementally as global-scope ordered files (no ES-module change needed) — each loaded before rehearser.js in loader.js + added to the bundle list in scripts/minify.mjs, verified by tests/rehearser.spec.js (parse → cast) and the smoke suite:

  • ✅ rehearser-parse.js — parseScript, detectCharacters (pure; only call-time SPEAKER_COLORS).
  • ✅ rehearser-cast.js — renderCastList, sort/filter, online voice picker, and character research.
  • ✅ rehearser-stage.js — stage font/cast sidebar controls and cast-strip rendering.
  • ✅ rehearser-playback.js — WebAudio playback, transport state, bounded decoded-audio prefetch, and persistent line-audio cache/synthesis controls.
  • ✅ rehearser-io.js — rehearsal serialization, library save/load, autosave, session/file export, and screenplay/PDF imports.
  • ✅ rehearser-recording.js — microphone recording overlay, level meter, and recorded-line controls.
  • ✅ rehearser-training-ui.js — training-mode state, cue playback, transcription, recording, and phase controls.
  • ✅ rehearser-voice-picker.js — voice preview state, local matching, fish.audio search/import, and candidate assignment; DOM event registration remains in rehearser-cast.js.
  • ✅ rehearser-cast.js — cast view, full cast-card rendering, sort/filter state handlers, and delegated cast toolbar wiring.
  • ✅ rehearser-voice-design.js — character development, backend selection, and backend-refresh synchronization.
  • ✅ audiobook-segmentation.js — quote segmentation, conservative turn-taking, and dialogue-piece validation.
  • ✅ audiobook-parse.js — dialogue detection, dehyphenation, and speaker-heuristic constants.
  • ✅ audiobook-cast.js — cast Markdown export, legacy page-stamping migration, deterministic speaker/sentence/duplicate/alias normalization helpers, and the live cast-view renderer.
  • ✅ audiobook-pipeline.js — LLM casting, unknown-speaker recast, automatic repair, and cast-view orchestration.
  • ✅ audiobook-preview-export.js — editable cast preview, Rehearser handoff, chapter-aware export, and export browsing.
  • ✅ Stateful Audiobook modules use private scopes with explicit legacy exports; their public APIs are namespace-tested.
  • ✅ Rehearser playback uses a private scope with explicit transport/cache compatibility exports and a namespace-tested API.
  • ✅ Rehearser I/O uses a private scope with explicit serialization/library/import compatibility exports.
  • ✅ Rehearser recording uses a private scope with explicit microphone/overlay compatibility exports.
  • ✅ Rehearser core uses a private scope with explicit state, pagination, and cast-management compatibility exports.
  • ✅ Audiobook core uses a private scope with explicit reader-side state and handoff compatibility exports.
  • ✅ Reader uses a private scope with explicit document, playback, and search compatibility exports.
  • ✅ Rehearser library persistence uses a private scope with explicit SQLite/IndexedDB compatibility exports.
  • ✅ Audiobook network transport uses a private scope with explicit timeout/stream compatibility exports.
  • ✅ Rehearser stage controls use a private scope with explicit scaling and cast-sidebar compatibility exports.
  • ✅ Rehearser voice-picker uses a private scope with explicit preview, search, and assignment compatibility exports.
  • ✅ Rehearser voice-design uses a private scope with explicit development and backend-refresh compatibility exports.
  • ✅ Rehearser cast rendering uses a private scope with explicit view/filter/sort compatibility exports.
  • ✅ Studio navigation and DOM-borrowing coordination use a private scope with explicit compatibility exports.
  • ✅ Character library persistence and rendering use a private scope with explicit identity/image compatibility exports.
  • ✅ Settings state and backend configuration use a private scope with live compatibility accessors and a namespace API.
  • ✅ TTS preview and performance-history helpers use a private scope with explicit preview/history compatibility exports.
  • ✅ Voice-clone audio, recording, and design controls use a private scope with explicit compatibility exports.
  • ✅ Voice-source discovery and import helpers use a private scope with explicit source-list/import compatibility exports.
  • ✅ TTS routing configuration and routing-log UI use a private scope with live route/sound compatibility accessors.
  • ✅ Engine browser, custom-card, and local-container helpers use a private scope with an explicit namespace API.
  • ✅ Combined library production rendering and Reader/Rehearser handoffs use a private scope with an explicit namespace API.
  • ✅ static/css/resource-diagnostics.css — isolated Performance/resource diagnostic panel styles.
  • ✅ static/css/background-jobs.css — isolated background-job history and retry-panel styles.
  • ✅ static/css/voice-design.css — isolated Voice Design form, samples, and preview-match styles.
  • ✅ static/css/rehearser.css — isolated Rehearser modal, cast, stage, training, bookshelf, and script-view styles.
  • ✅ static/css/audiobook.css — isolated Reader/Audiobook controls, casting feed, page layout, character overlays, and review styles.
  • ✅ rehearser-train.js — transcript normalization, LCS comparison, and word-level training feedback; microphone/state wiring remains in rehearser.js.
  • ✅ rehearser-library.js — SQLite persistence helpers and IndexedDB migration; library rendering remains ordered with the main module because it owns the phase UI.

Pattern (proven): extract a cohesive block whose only cross-file references resolve at call time (event handlers), not at module load. Top-level const rehState and top-level addEventListener registrations must stay ordered — extract those last and load them after their dependencies. Re-run npx playwright test after each extraction.

Remaining recommendations

  • Complete the eventual ES-module/global-compatibility migration one leaf at a time; current behavior intentionally keeps legacy globals available while the window.TTVC surface is adopted.
  • Complete the eventual ES-module/global-compatibility migration one leaf at a time; current behavior intentionally keeps legacy globals available while the window.TTVC surface is adopted.
  • Replace remaining silent .catch(() => {}) with logErr(context, e) (helper added in utils.js).
  • Full a11y sweep (icon-button aria-labels, focus traps, keyboard nav in createSearchablePicker) — run the web-design-guidelines review.
  • If ever exposed beyond localhost: add an auth token + rate limiting (SSRF is already guarded in core/validation.py).