tts-voice-creator-clone-and-design-2/CHANGELOG.md

Changelog

All notable changes to TTS Voice Creator — Clone and Design are documented here.
Follows Keep a Changelog · versioned with Semantic Versioning.

---

Unreleased

---

[1.6.0] — 2026-06-03

Added

• Script Rehearser — Cast overhaul: Card / List view toggle; sort & filter (name, gender, language, line count, tag); character-card-game styling (large portrait, name, description line, action row); per-character online voice picker (audition the match, browse alternatives, pick from your library, or search fish.audio inline); "Hear a line" button that synthesizes a representative one-liner from the character's own dialogue in their assigned voice.
• AI character notes — Match local / Match online / Design all now research the play and drop a per-character note (description, gender, speaking style).
• Rehearser import auto-save — uploading a script (PDF/text/FDX/Fountain) saves it to the Library immediately.
• Internationalization (i18n) — interface language picker with German translation of the UI chrome; English is the source language (static/js/i18n.js, extend via I18N_DICT).
• Progressive Web App — installable with offline app shell (manifest.webmanifest + network-first service worker), iOS web-app meta and safe-area support.
• Test suite — Playwright smoke + functional tests (desktop and iPhone/WebKit profiles): app load, sections, clone tabs, PWA, rehearser parse→cast, bundle, i18n.
• Build tooling — opt-in single minified bundle (npm run minify → static/dist/main.min.js, loaded when ?bundle=1); architecture & migration notes in docs/ARCHITECTURE.md.

Changed

• Accessibility → WCAG 2.1 AA — accessible names on all controls, AA text/badge/button contrast, keyboard-focusable scroll regions (audited with axe-core; 40+ violations → a handful of edge cases).
• Performance / mobile stability — GZip responses; content-visibility virtualization for long lists; lazy-loaded images; Rehearser caps decoded-PCM memory to a sliding window (fixes iPhone crashes); leaked AudioContext closed; bounded-concurrency bulk operations.
• Clone a Voice — reworked GUI — integrated tab strip (Microphone · Upload · URL/YouTube), clearer sections, scroll-to + obvious "transcribing…" feedback, sample sentence keeps the typed name across language switches.
• Fish-Speech tone — per-line tones now reach OpenAudio S2 via inline [tag] markers in the text (the instruct field is ignored by S2).
• fish.audio import — de-duplicates voices already in the library and diversifies matches so different characters don't all get the same fallback voice.
• Get Voices Online — tabbed, integrated source switcher; the scrape box lives only under "Direct sources".
• Voice library — editable Voice ID (rename), complete country/accent list (decoupled from language), always-visible Select all toggle, redesigned bulk-delete confirmation modal.

Fixed

• Screenplay parser — title-page text, numbered scene headings (A1 EXT. … EVENINGA1), OMITTED/CONTINUED markers and dated page slugs are no longer detected as characters.
• Narrator & all voice pickers now list the full voice library (lazy-loaded if needed).
• Cast list controls wire reliably regardless of when the section mounts; role names no longer truncate; avatars enlarged.

---

[1.5.0] — 2026-06-01

Added

• Fish-Speech TTS backend — clones a voice's saved reference WAV (consistent identity) and honours inline emotion markers like (angry), (whispering), (excited) per line. The only backend that is both WAV-anchored and style-aware; the Rehearser prefers it when available. Configurable via FISHSPEECH_URL.
• Fish.audio Voice Library browser (Get Voices Online) — search/filter the ~2M public voices at api.fish.audio, preview samples, and one-click Import (MP3 → WAV + reference transcript) → an instantly clonable voice.
• Cast tab redesigned as character cards — big avatar, name, language, gender, tags, voice picker, voice-design prompt, "Character soul · LLM brief" with Develop (LLM), and per-character Ignore / Hide / Delete.
• Bulk-edit lines on the Stage — a Select mode adds per-line checkboxes: Ignore, Hide, Delete, Un-ignore, Show-hidden.
• Designed voices — display name is the character name, the script becomes a tag, and an auto-picked gender/type avatar icon replaces the language flag.
• Clone a Voice — name-first flow — name first (drops into the read-aloud sentence), live voice-ID, auto-transcribe after trim, auto-save when ready, and a File / URL / Microphone source picker.

Changed

• IMSDb scraper — resolves the real script via each title's detail-page "Read Script" link instead of guessing a slug.
• Rehearser default backend — voice_clone (then fishspeech) for consistent identity; the tone-warning explains the trade-off both ways.
• Try It Out — the cramped voice/backend row is now a clean responsive layout.

Fixed

• Narrator was silent — narratorVoice now stays in sync with the narrator cast row.
• About → Changelog was empty — CHANGELOG.md is now shipped in the image and resolved resiliently.

---

[1.4.0] — 2026-06-01

Added

• IMSDb browser — list / cover view toggle — switch between poster-grid and compact list view; preference persisted in localStorage.
• IMSDb browser — local catalogue cache — catalogue is cached in localStorage for 6 h (matching server cache), making reopening the browser instant.
• IMSDb browser — title in fallback — script title shown on each gradient poster card while the real poster loads.
• IMSDb browser — loading spinner — animated spinner while the catalogue fetches.
• IMSDb browse button on Import / Export tab — the "Browse IMSDb" button is now also available on the Import / Export panel; modal moved to global scope.
• Auto-design — detailed progress panel — each character shows an expandable card during voice design: gender chip, language, voice ID, age, and the full LLM-generated character description with a live spinner.
• Auto-design — script title as voice tag — designed voices receive the script title as their tag value so they're easy to filter/find.
• Auto-design — LLM endpoint datalist — the LLM endpoint field is now backed by a <datalist> auto-populated from all configured Language Models engines, plus hardcoded defaults (Ollama, vLLM, LM Studio, llama-swap, LiteLLM).
• Stage — synthesis progress — the synth bar is now more prominent (gradient fill, spinner, sticky), each synthesising line pulses with a blue glow, and the page auto-scrolls to the active line.
• Stage — tone warning banner — when a non-style-aware backend (voice_clone, streaming, NVIDIA) is selected and tone is set on lines, a dismissable amber warning banner names the backend and suggests a style-aware alternative.
• Bulk-edit tools — new sticky toolbar in My Voices: select any number of voices with checkboxes, then: Set tag, Hide, Unhide, Rate, or Delete in one action.
• Rehearser — Voice Design default — the TTS backend picker in Cast now defaults to voice_design (style-aware) instead of voice_clone, so tone selections work out of the box.

Changed

• Stage — edit button moved to right gutter — the pencil (edit text) button is now stacked with the note button in the right-side gutter of each dialog block, keeping the block header clean.
• Tone / instruct order — when an emotion is set on a line, the instruction now leads with a directive ("Speak in a <emotion> manner. <voice profile>") so the model prioritises the tone over the base identity description.
• My Voices — hidden voices in sub-tabs — fixed: Cloned, Designed, and Favorites tabs now respect the "Disabled" checkbox filter; hidden voices no longer appear unless explicitly requested.

Fixed

• IMSDb covers showing as flat lines — replaced aspect-ratio on a flex child (unreliable in all major browsers) with the padding-bottom: 150% wrapper trick, guaranteeing a correct 2:3 poster ratio.
• Rehearser TTS backend "No backend available" — refreshRehBackends now triggers the global backend probe if _ttsBackends is empty, and registers a _ttsRefreshHook so the select stays in sync with the Engines page.

---

[1.3.0] — 2026-05-31

Added

• Live mic monitor in Clone a Voice — level-meter and scrolling oscilloscope waveform in the Microphone card. "Check level" / "Stop monitor" buttons, mic gain slider. Recording uses raw mic constraints (no echo-cancel / AGC).
• STT engine picker in Clone → Transcript — pick any configured STT backend when auto-transcribing, bypassing an unavailable Whisper.
• Better sample texts — all 8 languages rewritten to ~38 words / ~15 s, phonetically rich, proper Unicode diacritics.

Fixed

• Empty "Read aloud" field — sample text now reliably populates on load and when navigating to the Clone section.
• Recording quality — MediaRecorder requests 256 kbps in Clone and STT→TTS.
• OGG file import — explicit extension list in accept=.

---

1.2.0 — 2026-05-29

Added

• Remember last section on reload — the active section (and Settings / Engines sub-page) is persisted in localStorage. A hard-reload (Ctrl+Shift+R) now returns to the same page instead of always jumping to My Voices.
• Conversation: live speech preview — while recording, the active Whisper STT backend transcribes accumulated audio every 2.5 s and shows the result in the text input field in real time. The input is pre-populated with this live guess before the final Whisper result arrives. Also tries the browser's Web Speech API first (works on HTTPS / localhost) for even faster results.
• Conversation: Voice Activity Detection (VAD) — recording now auto-stops after 1.5 s of silence detected via the Web Audio AnalyserNode RMS level. A "Sending in X.Xs" countdown appears in the status bar so the timing is visible. An Auto-stop toggle in the input bar lets users disable VAD and revert to click-to-stop. A thin audio-level bar below the status line shows microphone volume in real time during recording.
• Conversation: hands-free mode — after the agent finishes speaking, the microphone restarts automatically. A Hands-free toggle (on by default) disables this; clicking the mic button manually always cancels any pending auto-restart.
• scripts/release.py — automates version bump + CHANGELOG promotion. python scripts/release.py --patch|--minor|--major [--dry-run] renames [Unreleased] to the new version, updates compare links, writes VERSION, commits, and creates an annotated git tag in one command.
• Git pre-commit hook (scripts/hooks/pre-commit) — warns (does not block) when .py/.js/.css/.html files are staged but CHANGELOG.md or VERSION are not. Run bash scripts/install-hooks.sh after cloning.
• scripts/install-hooks.sh — one-liner to install the hook after a fresh clone: bash scripts/install-hooks.sh.

Changed

• Config and logs are now bind-mounted local folders — replaced the opaque named Docker volume with ./config/ and ./logs/ host directories. portainer-stack.yml updated with absolute host paths.
• Server writes a rotating log file — RotatingFileHandler writes INFO-level and above to ./logs/app.log (rotates at 5 MB, 3 backups).

Performance

• Skeleton loading view — index.html shows an animated shimmer placeholder immediately on first paint; fades out once JS finishes loading.
• Self-hosted WaveSurfer and MDI icon font — removed render-blocking CDN requests; assets now served locally from static/vendor/.
• Parallel JS module loading — restructured loader.js into 4 ordered batches; round-trips reduced from 17 to 6, 9 files fetched simultaneously.
• Version-based JS/CSS cache busting — versioned assets served with max-age=31536000, immutable; bumping version invalidates the cache.

Fixed

• LLM returned empty response (Qwen3 thinking mode) — conversation turn now falls back to reasoning_content for think-only responses; error message hints to add /no-think to the system prompt.
• Engine settings lost after container recreate — container names and URL overrides now persisted as server settings (engine_container_names, engine_local_urls); restored from server on first page load.
• Text-input turns returned 422 — changed audio form field to Optional[UploadFile] = None so text-only turns don't require audio.
• Conversation input bar hidden when mic unavailable — warning box moved inside conv-chat-window so it never pushes the input bar off-screen.
• Browser caches old section HTML — loader.js appends ?v=<timestamp> to every section fetch.
• Various import errors and container restart issues fixed.

---

1.1.0 — 2026-05-29

Security

• Fixed path traversal in /api/browse-dirs — Added a _BROWSE_BLOCKED blocklist (/proc, /sys, /dev, /run, /boot). Requests for paths under these directories now return HTTP 403 instead of listing kernel/system files.
• Hardened yt-dlp output path — After a YouTube download completes, the resolved output path is verified to be inside TEMP_DIR via .relative_to(). A file written outside the temp directory is rejected with an SSE error event and never registered.
• Removed CORS wildcard on /api/proxy-audio — Access-Control-Allow-Origin: * was unnecessary (all callers are same-origin) and exposed proxied audio to arbitrary cross-origin requests. Header removed.
• Temp file registry now enforces a TTL — _registry changed to dict[str, tuple[Path, float]]. _registry_gc() evicts entries older than TEMP_FILE_TTL_SECONDS (default 2 h, configurable via env var) and unlinks their files, preventing unbounded disk growth on long-running instances.

Performance

• Settings and routing rules cached in memory — _load_settings() and _load_tts_routes() previously read from disk on every API request (55+ calls per TTS synthesis). Both now use mtime-checked in-memory caches that invalidate automatically on write, eliminating redundant file I/O.

Added

• Version number — VERSION file at repo root; read by core/constants.__version__ and surfaced via GET /api/version. Displayed as v1.1.0 in Settings → About.
• Text input in Conversation Playground — a pill-shaped text field and send button (→) sit left of the mic button. Pressing Enter or → sends text directly through the LLM → TTS pipeline, skipping STT entirely. Makes the playground fully usable without a microphone (HTTP context, no mic permission, remote access). The backend /api/conversation/turn now accepts an optional text form field; when set, the STT step is skipped and the STT latency row shows —.
• Container name field on all engine cards — every TTS and STT engine card (Docker stack cards and static "Other Local" cards) now always shows the Docker container name input row. Previously absent/not-installed cards hid it; now it is always visible so the container can be pre-configured before starting.
• Connect / Disconnect toggle — the Connect button now shows "Disconnect" (green, check-network icon) when already connected and toggles back on click. State persists in localStorage.
• Auto-apply on Connect — a successful connection probe automatically saves the URL to Settings and makes the backend available in TTS/STT dropdown menus immediately, without requiring a separate "Use as TTS/STT" click.

Changed

• Connect button redesigned — moved out of the URL input row into a dedicated dc-controls-row. Restyled as a solid blue primary CTA (was a small teal outline button). Shows a spinner icon while probing.

• "Use as TTS / STT" button — larger padding, bolder teal border, chevron icon, tooltip explaining it sets the URL in Settings. Gains .active highlight once applied.

• Unified controls row on every engine card — consistent left-to-right order: [Connect/Disconnect] [Stop | Start | Restart] [Use as →]. Docker action buttons hidden until a container name is entered; Use-as button right-aligned.

• initStaticDockerManagement — rebuilt to use the same dc-controls-row structure as the dynamic Docker stack cards. The existing .llm-local-ping button is moved from inside the URL row into the controls row at initialisation time.

• Backend refactor — server.py (5 560 lines → 43 lines) — all logic extracted into single-responsibility modules:

  Package	Module	Responsibility
  core/	constants.py	Boot-time env defaults, path constants, version, log buffer
  	registry.py	TTL-based temp file registry
  	validation.py	URL validation, SSRF guard, path safety
  	docker_client.py	Raw Unix-socket Docker HTTP client
  	config.py	Settings load/save/normalize, backend URL resolution
  	routing.py	TTS route rules load/save/resolve, language detection
  	audio.py	Audio conversion, normalisation, auto-trim scoring
  	voice.py	Voice metadata, backup management, benchmark helpers
  	presets.py	Voice Design preset load/save, virtual voice resolution
  	tts_helpers.py	TTS request helpers, streaming, per-backend logic
  routes/	admin.py	Index, favicon, browse-dirs, robots, version
  	settings.py	/api/settings, routing rules, logs, design presets
  	library.py	All voice CRUD, upload, save, normalize, export/import
  	stt.py	/api/transcribe*, /api/stt-backends
  	sources.py	Voice scraping, proxy-audio, yt-dlp download
  	docker.py	/api/local-containers/*, /api/probe-url
  	tts.py	TTS preview, streaming, voice design, /v1/*, backends
  	conversation.py	Refine-text, effects, export/import, speak, MCP, conversation

  Dockerfile updated with COPY core/ core/ and COPY routes/ routes/. docker-compose.yml updated with ./core:/app/core:ro and ./routes:/app/routes:ro.

• Frontend refactor — app.js (8 744 lines → 16 modules) — split into static/js/ with loader.js loading them sequentially in dependency order:

  Module	Lines	Responsibility
  utils.js	364	Core helpers: $, toast, escHtml, theme, language/flag, picker, tabs
  voice-inspector.js	397	3-pane voice workbench
  voice-sources.js	277	External voice source scraping UI
  integrations.js	211	Code snippet generation (SillyTavern, Open WebUI, HA, curl, MCP)
  routing.js	542	TTS routing rules editor
  settings.js	385	loadSettings, applyAndSaveSettings, settings panel
  voice-clone.js	774	WaveSurfer, drop zone, mic recording, trim, voice design
  voice-library.js	2654	Full voice library: list, CRUD, benchmark, normalize
  tts-preview.js	528	TTS preview, fetchTtsPreviewBlob
  benchmark.js	218	Performance + batch benchmark
  stt.js	287	STT→TTS playground, refreshSttBackends
  init.js	49	App bootstrap
  engines.js	625	ElevenLabs browser, custom engine cards, Docker management
  ai-backends.js	520	AI backend cards, LLM snippets, initStaticDockerManagement
  generation.js	393	WAV merge, chunked TTS, history, playlist, audio effects
  conversation.js	520	Conversation playground, LLM refinement, import, About

Fixed

• chrome://flags/… URL unreadable in mic-blocked warning — the global code { background: var(--panel) } rule caused the URL text to render as white-on-light-grey inside the red warning box. Fixed with inline styles (background: rgba(0,0,0,.35); color: #fff) on the <code> element, plus a Copy button so users don't need to manually select invisible text.

---

1.0.0 — 2026-05-28

Initial feature-complete release.

Added

• Voice library — clone voices from audio samples; design voices from text descriptions using instruction-based synthesis; benchmark synthesis speed (RTF); normalize loudness; export/import voice packages as ZIP bundles.
• TTS backends — Qwen3 TTS (Voice Clone, Voice Design, Custom Voice, Streaming), NVIDIA Magpie / Zeroshot / Flow, Kokoro FastAPI, VibeVoice, XTTS v2, ElevenLabs.
• STT backends — OpenAI Whisper (port 8010), faster-whisper-server, whisper.cpp, Groq Whisper (cloud, free tier), NVIDIA Parakeet ASR. Real transcription probe in health check (not just TCP reachability).
• App Routing — per-app / per-voice / per-language TTS routing rules with automatic language detection and optional before/after sound effects.
• Conversation Playground — full STT → LLM → TTS pipeline with real-time SSE streaming, latency stats panel (STT / LLM TTFT / LLM total / TTS / Total), turn history, system prompt, and insecure-context warning.
• Engines section — LLM / STT / TTS sub-pages; Docker container management (Start / Stop / Restart via Docker socket); custom engine cards; ElevenLabs voice library browser.
• Performance Benchmark — single-voice and batch benchmark with RTF tracking, sparkline trend, and persistent history.
• Audio effects — reverb, chorus, delay, compressor, gain, pitch shift (via pedalboard).
• Chunked TTS + generation history — long-text synthesis split into chunks, per-chunk playback, playlist export as WAV.
• MCP server — built-in JSON-RPC 2.0 endpoint at /mcp; tools: speak, transcribe, list_captures, list_profiles.
• LLM refinement & persona rewriting — clean up STT transcripts or rewrite responses with a chosen persona via any OpenAI-compatible LLM endpoint.
• Connect Apps — ready-made config snippets for SillyTavern, Open WebUI, Home Assistant, curl, and MCP (claude mcp add one-liner).
• Voice sources — scrape voice assets from Aiartes, Freesound, GitHub, and Google Drive; YouTube download via yt-dlp; quick import directly to library.
• OpenAI-compatible proxy — /v1/audio/speech and /v1/audio/transcriptions for drop-in use with Open WebUI, SillyTavern, and Home Assistant.
• Settings — sub-pages: General, Connections, Playback, Captures, Payloads, Storage, API Keys, Logs, About.
• Voice Design presets — saved persona templates for instruction-based synthesis; virtual vd_… voices usable from external apps without exporting WAV files.
• Multilingual support — language/flag pickers, per-language preview texts, LANG_FLAG_DEFAULT mapping for 16 languages.
• Tags, ratings, and metadata — per-voice tags with autocomplete, star ratings, gender label, country flag.
• Dark/light theme — toggle with persistence in localStorage.
• Docker socket integration — Start/Stop/Restart Docker containers from the UI via raw Unix socket HTTP; container health visible in engine cards.

---