mARTin-B78/tts-voice-creator-clone-and-design-2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1af02f8d1b
tts-voice-creator-clone-and.../CHANGELOG.md
mARTin-B78 1af02f8d1b Update CHANGELOG for chunked TTS fix 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 03:58:31 +02:00

24 KiB
Raw Blame History

Changelog

All notable changes to TTS Voice Creator — Clone and Design are documented here.
Follows Keep a Changelog · versioned with Semantic Versioning.

Unreleased

Fixed

  • Chunked TTS — "Failed to fetch" on long text: splitTextIntoChunks only split on sentence terminators (.!?), so newline-delimited text (e.g. German bullet lists or care-plan notes) was never split — the full page was sent as one request, causing a TCP timeout that the browser surfaced as "Failed to fetch". Fixed by processing each line individually before applying the sentence regex. Also moved generation.js from deferred batch E into the main feature batch so generateChunkedTts is always defined before the user can click Generate.

[1.6.0] — 2026-06-03

Added

  • Script Rehearser — Cast overhaul: Card / List view toggle; sort & filter (name, gender, language, line count, tag); character-card-game styling (large portrait, name, description line, action row); per-character online voice picker (audition the match, browse alternatives, pick from your library, or search fish.audio inline); "Hear a line" button that synthesizes a representative one-liner from the character's own dialogue in their assigned voice.
  • AI character notes — Match local / Match online / Design all now research the play and drop a per-character note (description, gender, speaking style).
  • Rehearser import auto-save — uploading a script (PDF/text/FDX/Fountain) saves it to the Library immediately.
  • Internationalization (i18n) — interface language picker with German translation of the UI chrome; English is the source language (static/js/i18n.js, extend via I18N_DICT).
  • Progressive Web App — installable with offline app shell (manifest.webmanifest + network-first service worker), iOS web-app meta and safe-area support.
  • Test suite — Playwright smoke + functional tests (desktop and iPhone/WebKit profiles): app load, sections, clone tabs, PWA, rehearser parse→cast, bundle, i18n.
  • Build tooling — opt-in single minified bundle (npm run minify → static/dist/main.min.js, loaded when ?bundle=1); architecture & migration notes in docs/ARCHITECTURE.md.

Changed

  • Accessibility → WCAG 2.1 AA — accessible names on all controls, AA text/badge/button contrast, keyboard-focusable scroll regions (audited with axe-core; 40+ violations → a handful of edge cases).
  • Performance / mobile stability — GZip responses; content-visibility virtualization for long lists; lazy-loaded images; Rehearser caps decoded-PCM memory to a sliding window (fixes iPhone crashes); leaked AudioContext closed; bounded-concurrency bulk operations.
  • Clone a Voice — reworked GUI — integrated tab strip (Microphone · Upload · URL/YouTube), clearer sections, scroll-to + obvious "transcribing…" feedback, sample sentence keeps the typed name across language switches.
  • Fish-Speech tone — per-line tones now reach OpenAudio S2 via inline [tag] markers in the text (the instruct field is ignored by S2).
  • fish.audio import — de-duplicates voices already in the library and diversifies matches so different characters don't all get the same fallback voice.
  • Get Voices Online — tabbed, integrated source switcher; the scrape box lives only under "Direct sources".
  • Voice library — editable Voice ID (rename), complete country/accent list (decoupled from language), always-visible Select all toggle, redesigned bulk-delete confirmation modal.

Fixed

  • Screenplay parser — title-page text, numbered scene headings (A1 EXT. … EVENINGA1), OMITTED/CONTINUED markers and dated page slugs are no longer detected as characters.
  • Narrator & all voice pickers now list the full voice library (lazy-loaded if needed).
  • Cast list controls wire reliably regardless of when the section mounts; role names no longer truncate; avatars enlarged.

[1.5.0] — 2026-06-01

Added

  • Fish-Speech TTS backend — clones a voice's saved reference WAV (consistent identity) and honours inline emotion markers like (angry), (whispering), (excited) per line. The only backend that is both WAV-anchored and style-aware; the Rehearser prefers it when available. Configurable via FISHSPEECH_URL.
  • Fish.audio Voice Library browser (Get Voices Online) — search/filter the ~2M public voices at api.fish.audio, preview samples, and one-click Import (MP3 → WAV + reference transcript) → an instantly clonable voice.
  • Cast tab redesigned as character cards — big avatar, name, language, gender, tags, voice picker, voice-design prompt, "Character soul · LLM brief" with Develop (LLM), and per-character Ignore / Hide / Delete.
  • Bulk-edit lines on the Stage — a Select mode adds per-line checkboxes: Ignore, Hide, Delete, Un-ignore, Show-hidden.
  • Designed voices — display name is the character name, the script becomes a tag, and an auto-picked gender/type avatar icon replaces the language flag.
  • Clone a Voice — name-first flow — name first (drops into the read-aloud sentence), live voice-ID, auto-transcribe after trim, auto-save when ready, and a File / URL / Microphone source picker.

Changed

  • IMSDb scraper — resolves the real script via each title's detail-page "Read Script" link instead of guessing a slug.
  • Rehearser default backend — voice_clone (then fishspeech) for consistent identity; the tone-warning explains the trade-off both ways.
  • Try It Out — the cramped voice/backend row is now a clean responsive layout.

Fixed

  • Narrator was silent — narratorVoice now stays in sync with the narrator cast row.
  • About → Changelog was empty — CHANGELOG.md is now shipped in the image and resolved resiliently.

[1.4.0] — 2026-06-01

Added

  • IMSDb browser — list / cover view toggle — switch between poster-grid and compact list view; preference persisted in localStorage.
  • IMSDb browser — local catalogue cache — catalogue is cached in localStorage for 6 h (matching server cache), making reopening the browser instant.
  • IMSDb browser — title in fallback — script title shown on each gradient poster card while the real poster loads.
  • IMSDb browser — loading spinner — animated spinner while the catalogue fetches.
  • IMSDb browse button on Import / Export tab — the "Browse IMSDb" button is now also available on the Import / Export panel; modal moved to global scope.
  • Auto-design — detailed progress panel — each character shows an expandable card during voice design: gender chip, language, voice ID, age, and the full LLM-generated character description with a live spinner.
  • Auto-design — script title as voice tag — designed voices receive the script title as their tag value so they're easy to filter/find.
  • Auto-design — LLM endpoint datalist — the LLM endpoint field is now backed by a <datalist> auto-populated from all configured Language Models engines, plus hardcoded defaults (Ollama, vLLM, LM Studio, llama-swap, LiteLLM).
  • Stage — synthesis progress — the synth bar is now more prominent (gradient fill, spinner, sticky), each synthesising line pulses with a blue glow, and the page auto-scrolls to the active line.
  • Stage — tone warning banner — when a non-style-aware backend (voice_clone, streaming, NVIDIA) is selected and tone is set on lines, a dismissable amber warning banner names the backend and suggests a style-aware alternative.
  • Bulk-edit tools — new sticky toolbar in My Voices: select any number of voices with checkboxes, then: Set tag, Hide, Unhide, Rate, or Delete in one action.
  • Rehearser — Voice Design default — the TTS backend picker in Cast now defaults to voice_design (style-aware) instead of voice_clone, so tone selections work out of the box.

Changed

  • Stage — edit button moved to right gutter — the pencil (edit text) button is now stacked with the note button in the right-side gutter of each dialog block, keeping the block header clean.
  • Tone / instruct order — when an emotion is set on a line, the instruction now leads with a directive ("Speak in a <emotion> manner. <voice profile>") so the model prioritises the tone over the base identity description.
  • My Voices — hidden voices in sub-tabs — fixed: Cloned, Designed, and Favorites tabs now respect the "Disabled" checkbox filter; hidden voices no longer appear unless explicitly requested.

Fixed

  • IMSDb covers showing as flat lines — replaced aspect-ratio on a flex child (unreliable in all major browsers) with the padding-bottom: 150% wrapper trick, guaranteeing a correct 2:3 poster ratio.
  • Rehearser TTS backend "No backend available" — refreshRehBackends now triggers the global backend probe if _ttsBackends is empty, and registers a _ttsRefreshHook so the select stays in sync with the Engines page.

[1.3.0] — 2026-05-31

Added

  • Live mic monitor in Clone a Voice — level-meter and scrolling oscilloscope waveform in the Microphone card. "Check level" / "Stop monitor" buttons, mic gain slider. Recording uses raw mic constraints (no echo-cancel / AGC).
  • STT engine picker in Clone → Transcript — pick any configured STT backend when auto-transcribing, bypassing an unavailable Whisper.
  • Better sample texts — all 8 languages rewritten to ~38 words / ~15 s, phonetically rich, proper Unicode diacritics.

Fixed

  • Empty "Read aloud" field — sample text now reliably populates on load and when navigating to the Clone section.
  • Recording quality — MediaRecorder requests 256 kbps in Clone and STT→TTS.
  • OGG file import — explicit extension list in accept=.

1.2.0 — 2026-05-29

Added

  • Remember last section on reload — the active section (and Settings / Engines sub-page) is persisted in localStorage. A hard-reload (Ctrl+Shift+R) now returns to the same page instead of always jumping to My Voices.
  • Conversation: live speech preview — while recording, the active Whisper STT backend transcribes accumulated audio every 2.5 s and shows the result in the text input field in real time. The input is pre-populated with this live guess before the final Whisper result arrives. Also tries the browser's Web Speech API first (works on HTTPS / localhost) for even faster results.
  • Conversation: Voice Activity Detection (VAD) — recording now auto-stops after 1.5 s of silence detected via the Web Audio AnalyserNode RMS level. A "Sending in X.Xs" countdown appears in the status bar so the timing is visible. An Auto-stop toggle in the input bar lets users disable VAD and revert to click-to-stop. A thin audio-level bar below the status line shows microphone volume in real time during recording.
  • Conversation: hands-free mode — after the agent finishes speaking, the microphone restarts automatically. A Hands-free toggle (on by default) disables this; clicking the mic button manually always cancels any pending auto-restart.
  • scripts/release.py — automates version bump + CHANGELOG promotion. python scripts/release.py --patch|--minor|--major [--dry-run] renames [Unreleased] to the new version, updates compare links, writes VERSION, commits, and creates an annotated git tag in one command.
  • Git pre-commit hook (scripts/hooks/pre-commit) — warns (does not block) when .py/.js/.css/.html files are staged but CHANGELOG.md or VERSION are not. Run bash scripts/install-hooks.sh after cloning.
  • scripts/install-hooks.sh — one-liner to install the hook after a fresh clone: bash scripts/install-hooks.sh.

Changed

  • Config and logs are now bind-mounted local folders — replaced the opaque named Docker volume with ./config/ and ./logs/ host directories. portainer-stack.yml updated with absolute host paths.
  • Server writes a rotating log file — RotatingFileHandler writes INFO-level and above to ./logs/app.log (rotates at 5 MB, 3 backups).

Performance

  • Skeleton loading view — index.html shows an animated shimmer placeholder immediately on first paint; fades out once JS finishes loading.
  • Self-hosted WaveSurfer and MDI icon font — removed render-blocking CDN requests; assets now served locally from static/vendor/.
  • Parallel JS module loading — restructured loader.js into 4 ordered batches; round-trips reduced from 17 to 6, 9 files fetched simultaneously.
  • Version-based JS/CSS cache busting — versioned assets served with max-age=31536000, immutable; bumping version invalidates the cache.

Fixed

  • LLM returned empty response (Qwen3 thinking mode) — conversation turn now falls back to reasoning_content for think-only responses; error message hints to add /no-think to the system prompt.
  • Engine settings lost after container recreate — container names and URL overrides now persisted as server settings (engine_container_names, engine_local_urls); restored from server on first page load.
  • Text-input turns returned 422 — changed audio form field to Optional[UploadFile] = None so text-only turns don't require audio.
  • Conversation input bar hidden when mic unavailable — warning box moved inside conv-chat-window so it never pushes the input bar off-screen.
  • Browser caches old section HTML — loader.js appends ?v=<timestamp> to every section fetch.
  • Various import errors and container restart issues fixed.

1.1.0 — 2026-05-29

Security

  • Fixed path traversal in /api/browse-dirs — Added a _BROWSE_BLOCKED blocklist (/proc, /sys, /dev, /run, /boot). Requests for paths under these directories now return HTTP 403 instead of listing kernel/system files.
  • Hardened yt-dlp output path — After a YouTube download completes, the resolved output path is verified to be inside TEMP_DIR via .relative_to(). A file written outside the temp directory is rejected with an SSE error event and never registered.
  • Removed CORS wildcard on /api/proxy-audio — Access-Control-Allow-Origin: * was unnecessary (all callers are same-origin) and exposed proxied audio to arbitrary cross-origin requests. Header removed.
  • Temp file registry now enforces a TTL — _registry changed to dict[str, tuple[Path, float]]. _registry_gc() evicts entries older than TEMP_FILE_TTL_SECONDS (default 2 h, configurable via env var) and unlinks their files, preventing unbounded disk growth on long-running instances.

Performance

  • Settings and routing rules cached in memory — _load_settings() and _load_tts_routes() previously read from disk on every API request (55+ calls per TTS synthesis). Both now use mtime-checked in-memory caches that invalidate automatically on write, eliminating redundant file I/O.

Added

  • Version number — VERSION file at repo root; read by core/constants.__version__ and surfaced via GET /api/version. Displayed as v1.1.0 in Settings → About.
  • Text input in Conversation Playground — a pill-shaped text field and send button (→) sit left of the mic button. Pressing Enter or → sends text directly through the LLM → TTS pipeline, skipping STT entirely. Makes the playground fully usable without a microphone (HTTP context, no mic permission, remote access). The backend /api/conversation/turn now accepts an optional text form field; when set, the STT step is skipped and the STT latency row shows —.
  • Container name field on all engine cards — every TTS and STT engine card (Docker stack cards and static "Other Local" cards) now always shows the Docker container name input row. Previously absent/not-installed cards hid it; now it is always visible so the container can be pre-configured before starting.
  • Connect / Disconnect toggle — the Connect button now shows "Disconnect" (green, check-network icon) when already connected and toggles back on click. State persists in localStorage.
  • Auto-apply on Connect — a successful connection probe automatically saves the URL to Settings and makes the backend available in TTS/STT dropdown menus immediately, without requiring a separate "Use as TTS/STT" click.

Changed

  • Connect button redesigned — moved out of the URL input row into a dedicated dc-controls-row. Restyled as a solid blue primary CTA (was a small teal outline button). Shows a spinner icon while probing.

  • "Use as TTS / STT" button — larger padding, bolder teal border, chevron icon, tooltip explaining it sets the URL in Settings. Gains .active highlight once applied.

  • Unified controls row on every engine card — consistent left-to-right order: [Connect/Disconnect] [Stop | Start | Restart] [Use as →]. Docker action buttons hidden until a container name is entered; Use-as button right-aligned.

  • initStaticDockerManagement — rebuilt to use the same dc-controls-row structure as the dynamic Docker stack cards. The existing .llm-local-ping button is moved from inside the URL row into the controls row at initialisation time.

  • Backend refactor — server.py (5 560 lines → 43 lines) — all logic extracted into single-responsibility modules:

    Package Module Responsibility
    core/ constants.py Boot-time env defaults, path constants, version, log buffer
    registry.py TTL-based temp file registry
    validation.py URL validation, SSRF guard, path safety
    docker_client.py Raw Unix-socket Docker HTTP client
    config.py Settings load/save/normalize, backend URL resolution
    routing.py TTS route rules load/save/resolve, language detection
    audio.py Audio conversion, normalisation, auto-trim scoring
    voice.py Voice metadata, backup management, benchmark helpers
    presets.py Voice Design preset load/save, virtual voice resolution
    tts_helpers.py TTS request helpers, streaming, per-backend logic
    routes/ admin.py Index, favicon, browse-dirs, robots, version
    settings.py /api/settings, routing rules, logs, design presets
    library.py All voice CRUD, upload, save, normalize, export/import
    stt.py /api/transcribe*, /api/stt-backends
    sources.py Voice scraping, proxy-audio, yt-dlp download
    docker.py /api/local-containers/*, /api/probe-url
    tts.py TTS preview, streaming, voice design, /v1/*, backends
    conversation.py Refine-text, effects, export/import, speak, MCP, conversation

    Dockerfile updated with COPY core/ core/ and COPY routes/ routes/. docker-compose.yml updated with ./core:/app/core:ro and ./routes:/app/routes:ro.

  • Frontend refactor — app.js (8 744 lines → 16 modules) — split into static/js/ with loader.js loading them sequentially in dependency order:

    Module Lines Responsibility
    utils.js 364 Core helpers: $, toast, escHtml, theme, language/flag, picker, tabs
    voice-inspector.js 397 3-pane voice workbench
    voice-sources.js 277 External voice source scraping UI
    integrations.js 211 Code snippet generation (SillyTavern, Open WebUI, HA, curl, MCP)
    routing.js 542 TTS routing rules editor
    settings.js 385 loadSettings, applyAndSaveSettings, settings panel
    voice-clone.js 774 WaveSurfer, drop zone, mic recording, trim, voice design
    voice-library.js 2654 Full voice library: list, CRUD, benchmark, normalize
    tts-preview.js 528 TTS preview, fetchTtsPreviewBlob
    benchmark.js 218 Performance + batch benchmark
    stt.js 287 STT→TTS playground, refreshSttBackends
    init.js 49 App bootstrap
    engines.js 625 ElevenLabs browser, custom engine cards, Docker management
    ai-backends.js 520 AI backend cards, LLM snippets, initStaticDockerManagement
    generation.js 393 WAV merge, chunked TTS, history, playlist, audio effects
    conversation.js 520 Conversation playground, LLM refinement, import, About

Fixed

  • chrome://flags/… URL unreadable in mic-blocked warning — the global code { background: var(--panel) } rule caused the URL text to render as white-on-light-grey inside the red warning box. Fixed with inline styles (background: rgba(0,0,0,.35); color: #fff) on the <code> element, plus a Copy button so users don't need to manually select invisible text.

1.0.0 — 2026-05-28

Initial feature-complete release.

Added

  • Voice library — clone voices from audio samples; design voices from text descriptions using instruction-based synthesis; benchmark synthesis speed (RTF); normalize loudness; export/import voice packages as ZIP bundles.
  • TTS backends — Qwen3 TTS (Voice Clone, Voice Design, Custom Voice, Streaming), NVIDIA Magpie / Zeroshot / Flow, Kokoro FastAPI, VibeVoice, XTTS v2, ElevenLabs.
  • STT backends — OpenAI Whisper (port 8010), faster-whisper-server, whisper.cpp, Groq Whisper (cloud, free tier), NVIDIA Parakeet ASR. Real transcription probe in health check (not just TCP reachability).
  • App Routing — per-app / per-voice / per-language TTS routing rules with automatic language detection and optional before/after sound effects.
  • Conversation Playground — full STT → LLM → TTS pipeline with real-time SSE streaming, latency stats panel (STT / LLM TTFT / LLM total / TTS / Total), turn history, system prompt, and insecure-context warning.
  • Engines section — LLM / STT / TTS sub-pages; Docker container management (Start / Stop / Restart via Docker socket); custom engine cards; ElevenLabs voice library browser.
  • Performance Benchmark — single-voice and batch benchmark with RTF tracking, sparkline trend, and persistent history.
  • Audio effects — reverb, chorus, delay, compressor, gain, pitch shift (via pedalboard).
  • Chunked TTS + generation history — long-text synthesis split into chunks, per-chunk playback, playlist export as WAV.
  • MCP server — built-in JSON-RPC 2.0 endpoint at /mcp; tools: speak, transcribe, list_captures, list_profiles.
  • LLM refinement & persona rewriting — clean up STT transcripts or rewrite responses with a chosen persona via any OpenAI-compatible LLM endpoint.
  • Connect Apps — ready-made config snippets for SillyTavern, Open WebUI, Home Assistant, curl, and MCP (claude mcp add one-liner).
  • Voice sources — scrape voice assets from Aiartes, Freesound, GitHub, and Google Drive; YouTube download via yt-dlp; quick import directly to library.
  • OpenAI-compatible proxy — /v1/audio/speech and /v1/audio/transcriptions for drop-in use with Open WebUI, SillyTavern, and Home Assistant.
  • Settings — sub-pages: General, Connections, Playback, Captures, Payloads, Storage, API Keys, Logs, About.
  • Voice Design presets — saved persona templates for instruction-based synthesis; virtual vd_… voices usable from external apps without exporting WAV files.
  • Multilingual support — language/flag pickers, per-language preview texts, LANG_FLAG_DEFAULT mapping for 16 languages.
  • Tags, ratings, and metadata — per-voice tags with autocomplete, star ratings, gender label, country flag.
  • Dark/light theme — toggle with persistence in localStorage.
  • Docker socket integration — Start/Stop/Restart Docker containers from the UI via raw Unix socket HTTP; container health visible in engine cards.