From 6808db3c5990682d2888acf809b98e6ece14d59e Mon Sep 17 00:00:00 2001 From: mARTin-B78 Date: Wed, 27 May 2026 21:05:51 +0200 Subject: [PATCH] Rename LLMs to Engines with LLM/STT/TTS sub-page nav, collapsible integration cards, equal-width card grids - Rename "LLMs" section to "Engines" with brain icon - Add three sub-pages (Language Models / Speech to Text / Text to Speech) following the same nav-tree pattern as Settings and My Voices - Rewrite s-llms.html: three s-engines-page divs, docker container grids (dc-grid-tts, dc-grid-stt), VibeVoice card in TTS section, static cloud API cards per category - Add navEnginesCat() and applyEnginesPage() to nav.js; engines tree open/close in showSection() - Remove obsolete initLlmCatTabs IIFE; fix dc-refresh-btn from ID to class-based querySelectorAll - Make integration cards in Connect Apps collapsible (collapsed by default) with favicon/icon prepended to h3 - Unify all Engines card grids to minmax(380px, 1fr) so local, docker, and cloud cards are the same width - Add s-engines-page CSS (display:none / is-active:flex) Co-Authored-By: Claude Sonnet 4.6 --- README.md | 747 +++++++++++++-------------- server.py | 397 ++++++++++++++- static/app.js | 393 ++++++++++----- static/index.html | 11 +- static/nav.js | 29 ++ static/sections/s-connect.html | 30 +- static/sections/s-llms.html | 860 +++++++++++++++++--------------- static/sections/s-settings.html | 85 ++++ static/sections/s-tryout.html | 7 +- static/style.css | 28 +- 10 files changed, 1633 insertions(+), 954 deletions(-) diff --git a/README.md b/README.md index ee957b9..7e85ec4 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ -# TTS Voice Creator - Clone and Design +# TTS Voice Creator — Clone & Design -A browser-based app for managing Qwen3 TTS voice clones and Voice Design prompts. It runs in Docker, opens in any browser, and gives you one place to curate the voice library, crop/reference audio, normalize loudness, write reusable voice design prompts, and copy integration snippets for apps such as SillyTavern, Open WebUI, and Home Assistant. +A browser-based voice management studio running in Docker. Clone real voices, design new ones from text descriptions, manage your voice library, and connect any app or AI agent via REST or MCP. --- @@ -14,14 +14,18 @@ A browser-based app for managing Qwen3 TTS voice clones and Voice Design prompts [ 24 kHz · mono · 16-bit PCM WAV + reference.txt ] ↓ [ Named voice saved to disk ] - ↓ - [ TTS preview / Library management ] + ↓ ↓ + [ TTS preview / Library ] [ /speak · /mcp · /v1 proxy ] ``` -Two workflows in one tool: +**Core workflows:** -- **Voice cloning** — record or import a real voice, trim a clean segment, auto-transcribe, normalize loudness, and save it as a TTS reference voice. -- **Voice design** — describe a voice in plain language and let the Qwen3-TTS VoiceDesign model synthesize one from scratch. Designed voices can be exported to the clone library or used live as virtual voices through this app's `/v1` proxy. +- **Voice cloning** — record or import audio, trim a clean segment, auto-transcribe, normalize loudness, and save as a TTS reference voice. +- **Voice design** — describe a voice in plain language; Qwen3-TTS VoiceDesign synthesises one from scratch. Export to the clone library or use live as a virtual voice. +- **Character persona** — attach a personality description to any voice; a local LLM rewrites incoming text in that character's style before synthesis. +- **Speak API** — `POST /speak` for any script, app, or agent, with per-client voice bindings and optional persona rewrite. +- **Native MCP server** — built-in at `POST /mcp`; exposes `speak`, `transcribe`, `list_captures`, and `list_profiles` as MCP tools for Claude Code, Cursor, or any MCP-aware agent. +- **Global hotkey daemon** — run `hotkey_daemon.py` on the host for push-to-talk dictation that types transcribed text into any focused window. --- @@ -30,11 +34,11 @@ Two workflows in one tool: | Requirement | Notes | |---|---| | Docker Engine + Docker Compose v2 | [docs.docker.com/get-docker](https://docs.docker.com/get-docker/) | -| A Docker network shared with your TTS/STT containers | `docker network create tts_net` | +| A Docker network shared with your TTS/STT containers | `docker network create dgx_net` | | A Whisper-compatible transcription server | See [Recommended backends](#recommended-backends) | | A TTS server | See [Recommended backends](#recommended-backends) | -| ARM64 (aarch64) or x86-64 | Image is based on `python:3.11-slim-bookworm` — works on both | -| Internet access during build | pip packages; WaveSurfer.js is loaded from CDN at runtime | +| ARM64 (aarch64) or x86-64 | Image based on `python:3.11-slim-bookworm` | +| Internet access during build | pip packages; WaveSurfer.js loaded from CDN at runtime | --- @@ -43,438 +47,352 @@ Two workflows in one tool: ### 1 — Clone the repository ```bash -git clone https://github.com/mARTin-B78/TTS-Voice-Creator.git tts-voice-creator-clone-and-design -cd tts-voice-creator-clone-and-design +git clone https://github.com/mARTin-B78/tts-voice-creator-clone-and-design-2.git +cd tts-voice-creator-clone-and-design-2 ``` -### 2 — Create a Docker network (if you don't have one already) - -All containers that need to talk to each other (TTS Voice Creator - Clone and Design, your TTS server, your Whisper server) should share a network: +### 2 — Create or reuse a Docker network ```bash -docker network create tts_net +docker network create dgx_net # skip if you already have one ``` -If you already have a shared network, note its name — you will use it in the next step. +### 3 — Configure `docker-compose.yml` -### 3 — Configure volume paths - -Edit `docker-compose.yml` (or `portainer-stack.yml` for Portainer) and set the host path where your voices are stored: +Set the host path for voices and the network name: ```yaml volumes: - /your/voices/directory:/voices:rw -``` - -This directory is shared with the TTS server — voices saved here become immediately available after the TTS server is restarted. - -Also update the network name if yours differs from the default: - -```yaml networks: - tts_net: + dgx_net: external: true ``` -### 4 — Build the image +### 4 — Build and start ```bash -docker build -t tts-voice-creator:latest . +docker compose up -d --build ``` -This step is required before the first run, and whenever you pull updates. - -### 5a — Run with Docker Compose - -```bash -docker compose up -d -``` - -### 5b — Deploy via Portainer - -> Use **`portainer-stack.yml`**, not `docker-compose.yml`. -> The Portainer file references the pre-built image and has no `build:` section, which avoids the "no such file: Dockerfile" error that occurs because Portainer stores stacks in its own internal directory. - -1. Build the image on the host (step 4 above) -2. Portainer → **Stacks** → **Add stack** -3. Enter a stack name (e.g. `tts-voice-creator`) -4. Select **Web editor** -5. Paste the contents of [`portainer-stack.yml`](portainer-stack.yml) -6. Click **Deploy the stack** - -### 6 — Open the UI +### 5 — Open the UI ``` -http://:7860 +http://:7890 ``` -### 7 — Configure API endpoints +### 6 — Configure API endpoints -Click the **⚙ gear icon** (top right) and fill in: +**Settings → Connections** — fill in at minimum: -| Field | Example | Notes | -|---|---|---| -| Whisper API URL | `http://whisper:8000` | Use container name if on the same network | -| Whisper API key | *(optional)* | Only needed for cloud APIs | -| TTS API URL | `http://tts-server:8000` | | -| TTS API key | *(optional)* | | -| TTS backend | `Qwen3-TTS / OpenAI` | Select to match your server | -| Voice Design URL | `http://host.docker.internal:8021` | Optional; required for prompt-based Voice Design. For Qwen-style dialogue/timbre reuse, point this at the 1.7B VoiceDesign backend. | -| Voice Design API key | *(leave empty)* | Falls back to TTS key if blank | -| Voice scan directory | `/voices` | Where the library tab looks for voices | -| Active voices directory | `/voices/active_voices` | Where newly saved and TTS-visible voices land | +| Field | Example | +|---|---| +| Voice Clone / Base URL | `http://host.docker.internal:8020` | +| Voice Design URL | `http://host.docker.internal:8021` | +| Whisper URL | `http://host.docker.internal:8010` | +| LLM base URL | `http://localhost:11434/v1` *(for persona rewrite)* | -Settings are persisted to a named Docker volume and survive container restarts. +Settings persist to a named Docker volume across restarts. --- ## Recommended Backends -### Speech-to-Text (transcription) +### Speech-to-Text -The app sends audio to a Whisper-compatible REST endpoint (`POST /v1/audio/transcriptions`). - -| Option | Type | Recommendation | -|---|---|---| -| **[faster-whisper-server](https://github.com/fedirz/faster-whisper-server)** | Local, free | Best local option. Runs `large-v3` efficiently on GPU or CPU. OpenAI-compatible API. | -| **[Whisper.cpp server](https://github.com/ggerganov/whisper.cpp)** | Local, free | Lightweight, works on any hardware with no Python. Good for CPU-only setups. | -| **[OpenAI Whisper API](https://platform.openai.com/docs/guides/speech-to-text)** | Cloud, paid | Highest accuracy, no local GPU needed. Set URL to `https://api.openai.com` and add your API key. | -| **[Groq Whisper API](https://console.groq.com/docs/speech-text)** | Cloud, free tier | Very fast inference, generous free tier. OpenAI-compatible endpoint. | - -**Recommended for local use:** `faster-whisper-server` with `large-v3` on a GPU. Example Docker run: - -```bash -docker run -d --gpus all --network tts_net --name whisper \ - -p 8000:8000 \ - fedirz/faster-whisper-server:latest-cuda -``` - -Then set the Whisper URL in settings to `http://whisper:8000`. - ---- +| Option | Notes | +|---|---| +| **[faster-whisper-server](https://github.com/fedirz/faster-whisper-server)** | Best local option. GPU-accelerated, OpenAI-compatible. | +| **[Whisper.cpp server](https://github.com/ggerganov/whisper.cpp)** | CPU-only, no Python. | +| **[Groq Whisper API](https://console.groq.com/docs/speech-text)** | Fast cloud, generous free tier. | +| **NVIDIA Parakeet** | `http://host.docker.internal:8092` — 200× realtime on GPU. | ### Text-to-Speech -The app supports any server that exposes a compatible REST endpoint. - -| Option | Type | Recommendation | +| Option | Backend ID | Notes | |---|---|---| -| **[Qwen3-TTS](https://github.com/QwenLM/Qwen3-TTS)** | Local, free | State-of-the-art multilingual TTS with voice cloning and voice design. Primary recommendation. | -| **[faster-qwen3-tts](https://github.com/mARTin-B78/TTS-Voice-Creator)** | Local wrapper | Docker wrapper for Qwen3-TTS with an OpenAI-compatible API and automatic voice loading from the configured active voices directory. | -| **[LocalAI](https://github.com/mudler/LocalAI)** | Local, free | Supports many TTS backends (Piper, Bark, XTTS, etc.) behind a unified API. Select "LocalAI" in settings. | -| **[Pocket-TTS](https://github.com/ai-joe-git/pocket-tts-server)** | Local, free | Lightweight XTTS-based server. Select "Pocket-TTS" in settings. | -| **[OpenAI TTS API](https://platform.openai.com/docs/guides/text-to-speech)** | Cloud, paid | High quality, no local hardware needed. Set URL to `https://api.openai.com` and add your API key. | -| **NVIDIA Magpie + Parakeet** | Local, free | OpenAI-compatible NVIDIA speech stack. Use Magpie Multilingual on `http://host.docker.internal:8091` for fixed-speaker TTS and Parakeet on `http://host.docker.internal:8092` for STT, or route both through `8090`. NVIDIA Magpie Zeroshot/Flow clone NIMs can be configured separately, usually on `http://host.docker.internal:8093`. | -| **[ElevenLabs](https://elevenlabs.io/docs/api-reference)** | Cloud, freemium | Best-in-class voice quality and cloning. OpenAI-compatible endpoint available. | - -**Recommended for local use:** Qwen3-TTS is the best open-source multilingual TTS model as of 2025. It produces very natural speech, supports 16+ languages, and has built-in zero-shot voice cloning and voice design. +| **[faster-qwen3-tts](https://github.com/mARTin-B78/TTS-Voice-Creator)** | `voice_clone` | Recommended. Docker wrapper for Qwen3-TTS with auto voice loading. | +| Qwen3-TTS Voice Design | `voice_design` | Prompt-based voice synthesis. Needs `Qwen3-TTS-12Hz-1.7B-VoiceDesign`. | +| CustomVoice | `customvoice` | Style-controlled synthesis with pre-configured speakers. | +| Streaming | `streaming` | Progressive low-latency WAV playback. | +| **[Kokoro FastAPI](https://github.com/remsky/Kokoro-FastAPI)** | `kokoro` | OpenAI-compatible. 82M model, ~300 MB RAM, CPU-friendly. | +| **VibeVoice** | `vibevoice` | Simple `POST /tts` service. Default: `http://192.168.178.8:8027`. | +| **NVIDIA Magpie TTS** | `nvidia_magpie` | Fixed-speaker GPU TTS. `http://host.docker.internal:8091`. | +| **NVIDIA Magpie Zeroshot** | `nvidia_zeroshot` | WAV-prompted zero-shot clone NIM. | +| **NVIDIA Magpie Flow** | `nvidia_flow` | WAV + transcript high-fidelity clone NIM. | +| **[ElevenLabs](https://elevenlabs.io/)** | — | Cloud; browse the shared library in the **Get Voices Online** tab. | --- -### Voice Cloning vs Voice Design +## Application Sections -This tool supports two fundamentally different ways to create a new TTS voice: +### Voices — Clone library -#### Option A — Voice Cloning - -You provide a real recording of a voice. The TTS model uses it as a reference to reproduce that speaker's characteristics. - -**What makes a good reference recording:** - -- **Duration:** 5–20 seconds (10–15 s is ideal) -- **Content:** Natural, expressive speech — not just counting or reciting the alphabet -- **Audio quality:** No background music, reverb, noise, or multiple speakers -- **Emotion:** Should match the emotion you want the TTS to reproduce (calm for a narrator, energetic for an announcer, etc.) -- **Format:** The app converts anything to 24 kHz mono 16-bit PCM WAV automatically -- **Transcript:** The reference text (what was spoken) is critical — Qwen3-TTS uses it for alignment - -**Good sources for reference audio:** - -- Record yourself or a voice actor directly in the app (mic tab) -- Extract a clean segment from a podcast, audiobook, or interview using the YouTube/URL downloader -- Any audio or video file: WAV, MP3, OGG, FLAC, M4A, MP4, MKV, WEBM - -#### Option B — Voice Design - -Describe the voice you want in plain language. The Qwen3-TTS VoiceDesign model synthesises a new voice from your description — no recording required. - -**Requires:** A server running the `Qwen3-TTS-12Hz-1.7B-VoiceDesign` model or a compatible alias (typically a separate model from the main TTS model). The container environment variable `VOICE_DESIGN_MODEL` controls the model name sent to the VoiceDesign backend. - -**How to write good voice descriptions:** - -Describe: age, gender, pitch, pace, accent, emotion, and speaking style. Be specific. - -``` -# Good descriptions: -"A middle-aged British woman with a calm, authoritative tone and clear diction. - Slightly formal, measured speech rate." - -"Young male voice, mid-20s, enthusiastic and upbeat. American accent, slightly - fast pace, warm and friendly." - -"Elderly male narrator with a deep, resonant voice. Slow and deliberate, - with gravitas. No discernible accent." - -# Bad (too vague): -"A nice voice" -"Female voice" -``` - ---- - -## Application Tabs - -### 1 Voice Clone - -The first tab is the main voice library. It contains the old Source & Trim and -Name & Save workflow inside an `Add new voice` panel, so you no longer need to -move back and forth between separate tabs. +Manage WAV reference voices. Crop and normalize audio, auto-transcribe, set metadata (language, gender, rating, note), and attach a **character persona** for LLM-based text rewriting. | Feature | Details | |---|---| -| Add new voice | Opens drag and drop, YouTube/audio URL, microphone, waveform trim, transcript, and save controls | -| Library editing | Rename by editing the name directly; the app renames the WAV and sidecar files | -| Language | Formerly shown as Flag; language and region are stored in metadata | -| Type / Length / dB | Shows audio type, duration, and stored loudness values | -| Pencil optimizer | Opens waveform crop, transcript, dB controls, save crop, save volume, and undo crop | -| Active toggle | Moves complete voice packages between `active_voices` and `hidden_voices` | -| Copy active voices | Copies active voice IDs as a comma-separated list | +| Add voice | Drag-and-drop, URL/YouTube import, microphone recording | +| Waveform editor | Crop to clean segment, normalize loudness to target dBFS | +| Auto-transcribe | Sends audio to the configured STT backend | +| Character persona | Plain-language personality prompt; LLM rewrites text before synthesis | +| Active / hidden | Toggle moves voice between `active_voices` and `hidden_voices` | +| Sync / Calculate dB / Benchmark | Bulk tools for library health | -Cropping is reversible: the app keeps an original copy before replacing the -active voice file. +### Clone a Voice -### 2 Voice Design +Step-by-step wizard: import audio → trim → transcribe → save to library. -Voice Design is for prompt-based voices. Samples are listed in a collapsible -table. `Preview` generates audio from a sample, and `Use` fills the description, -sample text, and language fields. +### Design a Voice -| Feature | Details | -|---|---| -| Voice Design samples | Example prompts for acoustic attribute control, age control, gradual control, human-likeness, background information, and timbre reuse | -| Reference transcript | Shared transcript field used by samples and presets | -| Generate voice | Calls the VoiceDesign backend and plays the generated audio | -| Export to Voice Clone Library | Saves the generated WAV, transcript, name, language, and gender into the clone library | -| Prompt library | Stores reusable Voice Design presets | +Describe a voice in natural language; Qwen3-TTS VoiceDesign generates it from scratch. Export to the clone library or use as a live `vd_...` virtual voice. -### 3 Routing +Includes curated sample prompts: acoustic attribute control, age control, timbre reuse, multi-speaker dialogue. -The Routing tab lets this app act as an OpenAI-compatible TTS router. External -apps can send a simple voice name such as `default` to this app's `/v1` proxy, -and routing rules can map it to a real Qwen3-TTS voice by app and detected -language. +### Studio — Get Voices Online -Example: +Browse public voice clip sources, scrape and preview direct audio files, import from URLs, and browse the ElevenLabs shared voice library. -| App | Input voice | Language | Output voice | -|---|---|---|---| -| Open WebUI | `default` | `EN` | `EN_F_Anna` | -| Open WebUI | `default` | `DE` | `DE_M_Max` | +### Try It Out — TTS Playground -### 4 Use In Apps +Generate speech from any backend and voice. Features: -This tab shows ready-to-copy examples for SillyTavern, Open WebUI, Home -Assistant, and OpenAI-compatible clients. +- Backend and voice selector with live availability check +- Style instruction field (emotion, pacing, tone) +- **Apply character persona** toggle — LLM rewrites input text through the voice's personality before synthesis +- Chunked generation for long texts (split at sentence boundaries) +- Audio effects: reverb, compressor, chorus, pitch shift +- Generation history (last 20 clips, in-memory) +- Playlist with drag-to-reorder and merged WAV export -Exported clone voices should usually point external apps directly at the Qwen3 -TTS server. Virtual VoiceDesign voices should point external apps at this app's -`/v1` proxy. +Also includes an **STT → TTS workspace**: record or upload speech → transcribe → LLM-refine → re-synthesize with any voice. + +### Performance — Benchmark + +Run latency and quality benchmarks across backends. Batch mode tests every active voice against a configurable sentence set. Results (RTSL, duration, per-voice) are shown in a sortable table and saved to voice metadata. + +### App Routing + +Route incoming OpenAI-compatible TTS requests by app name, voice, and detected language. Rules map generic voice names (`default`, `assistant`) to specific library voices. Before/after sounds and per-route backend selection are supported. + +### Connect Your Apps + +Ready-to-copy integration snippets for SillyTavern, Open WebUI, Home Assistant, generic curl, virtual VoiceDesign voices, streaming TTS, and the native MCP server. + +### Settings + +Organized into sub-pages: **Connections** (all backend URLs including VibeVoice, NVIDIA stack), **STT**, **Generation** (extra TTS params per backend), **LLM** (URL for persona rewrite and text refinement), **Voice** (scan directory, normalization targets), **Output**, **Captures**, **General**, **About** (MCP connection commands, `/speak` examples, hotkey daemon setup). --- -## Command Line Usage +## Character Persona -Everything needed for headless operation is available through `ttsvc_cli.py`. -It talks to the running app over HTTP, so it works locally or against a remote -creator instance. - -```bash -./ttsvc_cli.py --help -./ttsvc_cli.py settings show -./ttsvc_cli.py settings set --tts-url http://host.docker.internal:8020 --tts-backend openai -./ttsvc_cli.py voices list --all -./ttsvc_cli.py voices enable EN_F_Anna -./ttsvc_cli.py routes openwebui-defaults --en EN_F_Anna --de DE_M_Max -./ttsvc_cli.py routes list -./ttsvc_cli.py tts voices -./ttsvc_cli.py tts speak --voice default --text "Hallo, das ist ein Test." --out test.wav -./ttsvc_cli.py create-voice --file sample.wav --voice-id EN_F_NewVoice --transcript "Exact words spoken." -``` - -Use `--base-url http://HOST:7860` when the app is not running on localhost. -Each command and subcommand has its own `--help` output. - ---- - -## Guided Installer - -For non-command-line users, start the graphical installer: - -```bash -python3 guided_installer.py -``` - -Or double-click one of these launchers: - -- Linux: `start-guided-installer.sh` -- macOS: `start-guided-installer.command` -- Windows: `start-guided-installer.bat` - -The installer checks Docker, Docker Compose, optional Portainer, this app, and -the Qwen3-TTS clone/design/custom containers. It explains what each component is -for, lets the user choose ports and model locations, generates a Docker Compose -file, and can run the selected Docker actions with visible logs. - -Additional publishing assets: - -- [Forum playbook](docs/forum-playbook.md) -- [Tutorial video script](docs/tutorial-video-script.md) - ---- - -## API compatibility - -### TTS endpoints - -| Backend | Setting | Endpoint | Request body | -|---|---|---|---| -| Qwen3-TTS / OpenAI | `openai` | `POST /v1/audio/speech` | `{"model":"tts-1","input":"...","voice":"EN_F_Anna","response_format":"wav"}` | -| LocalAI | `localai` | `POST /tts` | `{"input":"...","model":"EN_F_Anna","response_format":"wav"}` | -| Pocket-TTS | `pocket` | `POST /v1/audio/speech` | `{"input":"...","voice":"EN_F_Anna","response_format":"wav"}` | -| NVIDIA Magpie Multilingual | `nvidia_magpie` | `POST /v1/audio/speech` | `{"model":"tts-1","input":"...","voice":"sofia","response_format":"wav"}` | -| NVIDIA Magpie Zeroshot | `nvidia_zeroshot` | `POST /v1/audio/synthesize` | Multipart form with `language`, `text`, and selected library WAV as `audio_prompt` | -| NVIDIA Magpie Flow | `nvidia_flow` | `POST /v1/audio/synthesize` | Multipart form with `language`, `text`, selected library WAV as `audio_prompt`, and saved reference transcript as `audio_prompt_transcript` | - -### Voice Design endpoint - -``` -POST /v1/audio/speech -{ - "model": "Qwen3-TTS-12Hz-1.7B-VoiceDesign", - "input": "sample text to speak", - "instruct": "describe the voice here", - "language": "Auto", - "response_format": "wav" -} -``` - -For Qwen-style multi-speaker VoiceDesign, send the complete dialogue in `input` -and the speaker timbre map in `instruct`, for example: +Each voice can have a personality description stored in its `.meta.json` sidecar: ```json -{ - "model": "Qwen3-TTS-12Hz-1.7B-VoiceDesign", - "input": "Lucas:Hi there.\nMia:Hello back.", - "instruct": "\"Lucas\": \"Male, 17 years old, tenor range\"\n\"Mia\": \"Female, 16 years old, mezzo-soprano range\"", - "language": "English", - "response_format": "wav" -} +{ "persona": "Gruff Scottish engineer, dry wit, technical vocabulary, measured pace." } ``` -### Whisper endpoint +When persona rewrite is active, a local LLM at the configured LLM URL rephrases the input text in that character's voice before TTS synthesis. This runs through the same OpenAI-compatible endpoint used for text refinement. -``` -POST /v1/audio/transcriptions -Content-Type: multipart/form-data +**Activate persona rewrite:** +- In the playground: check **Apply character persona** before generating. +- Via API: pass `"apply_persona": true` in the request body to `/api/tts-preview` or `/speak`. +- Via MCP: pass `"apply_persona": true` to the `speak` tool. -file= -model=large-v3 -response_format=text -``` - -For NVIDIA Parakeet, set the Whisper/STT URL to `http://host.docker.internal:8092` for the direct ASR container, or `http://host.docker.internal:8090` for the speech router. For NVIDIA voice cloning, set the NVIDIA Zeroshot/Flow NIM URL to the deployed Speech NIM endpoint, commonly `http://host.docker.internal:8093`; the app sends saved library WAVs as `audio_prompt`, and Flow also sends the saved `.reference.txt` transcript. - -All endpoints accept an optional `Authorization: Bearer ` header, configurable in Settings. +Edit a voice's persona in the voice inspector (**Voices** tab → select voice → **Character persona** section). --- -## Timbre Control Support +## Speak API -There are two different modes: - -- `faster-qwen3-tts` uses saved voice/reference audio IDs. It supports timbre reuse through cloned or exported voices, but it does not normally use prompt fields such as gender, age, pitch, texture, or emotion. -- `faster-qwen3-tts-voicedesign` is the prompt-based path. It can use acoustic descriptions and timbre-style prompts, including multi-speaker/timbre reuse style descriptions, but those voices are generated live from the prompt unless you export the result. - -So: timbre descriptions belong in the Voice Design container. Stable reusable -voice IDs belong in the normal TTS container after export. - ---- - -## Use In Apps - -### Exported Clone Voices - -For SillyTavern, Open WebUI, Home Assistant, or any OpenAI-compatible client: - -```text -Base URL: http://YOUR_TTS_HOST:8020/v1 -Model: qwen -Voice: one of the active voice IDs -``` - -After adding or changing voices, restart the TTS container so it rereads the -active voice folder. - -### Virtual VoiceDesign Voices Without Export - -This app exposes an OpenAI-compatible proxy: - -```text -Base URL: http://YOUR_APP_HOST:7860/v1 -Model: qwen3-tts-voice-creator -Voice: vd_ -``` - -Virtual voices call the Voice Design backend live for each generation. This is -useful when you want to keep editing prompts without exporting WAV files yet. It -is slower than using exported clone voices and requires the VoiceDesign backend -URL to be configured. - -Example request: +`POST /speak` — generate audio from any script or app without needing routing rules. ```bash -curl http://localhost:7860/v1/audio/speech \ +# Bind a client ID to a voice (persisted) +curl -X PUT http://localhost:7890/speak/bindings/my-script \ -H "Content-Type: application/json" \ - -d '{ - "model": "qwen3-tts-voice-creator", - "voice": "vd_WarmNarrator", - "input": "This line is generated through a saved Voice Design preset." - }' \ + -d '{"voice": "EN_F_Anna"}' + +# Generate speech — voice from binding, optional persona rewrite +curl -X POST http://localhost:7890/speak \ + -H "Content-Type: application/json" \ + -H "X-Voice-Creator-Client-Id: my-script" \ + -d '{"text": "Hello world", "apply_persona": true}' \ + --output speech.wav + +# Or specify voice directly +curl -X POST http://localhost:7890/speak \ + -H "Content-Type: application/json" \ + -d '{"text": "Hello world", "voice": "EN_F_Anna", "backend": "voice_clone"}' \ --output speech.wav ``` +**Voice resolution order:** explicit `voice` param → `X-Voice-Creator-Client-Id` binding → `captures_default_voice` in Settings. + +**Binding endpoints:** + +| Method | Path | Description | +|---|---|---| +| `GET` | `/speak/bindings` | List all per-client voice bindings | +| `PUT` | `/speak/bindings/{client_id}` | Set or update a binding | +| `DELETE` | `/speak/bindings/{client_id}` | Remove a binding | + --- -## Voice file format +## Native MCP Server -Qwen3-TTS (and most modern neural TTS systems) expect reference audio in a specific format: +The app ships a built-in MCP server at `/mcp` (JSON-RPC 2.0, Streamable HTTP transport). No external Python script or extra packages required. + +**Tools:** + +| Tool | Arguments | Returns | +|---|---|---| +| `speak` | `text`, `voice?`, `apply_persona?` | Base64 WAV audio + confirmation text | +| `transcribe` | `audio_base64` | Transcribed text | +| `list_captures` | — | 20 most recent output WAV files | +| `list_profiles` | — | All voice profiles with id, lang, persona, enabled | + +**Connect Claude Code:** + +```bash +claude mcp add voice-creator \ + --transport http \ + --url http://localhost:7890/mcp \ + --header "X-Voice-Creator-Client-Id: claude-code" +``` + +**Any HTTP MCP client (Cursor, Windsurf, VS Code, etc.):** + +```json +{ + "mcpServers": { + "voice-creator": { + "url": "http://localhost:7890/mcp", + "headers": { "X-Voice-Creator-Client-Id": "my-agent" } + } + } +} +``` + +The `X-Voice-Creator-Client-Id` header binds this client to a voice via `PUT /speak/bindings/{id}`. Per-client bindings persist in Settings. + +--- + +## Global Hotkey Daemon + +`hotkey_daemon.py` is a standalone host-side script (runs outside Docker) for push-to-talk dictation. Hold **Ctrl+Shift+Space** to record, release to transcribe and type the result into any focused window. + +```bash +# Install dependencies (host Python, not Docker) +pip install pynput sounddevice soundfile pyperclip requests + +# Linux: install xdotool for direct key injection (falls back to clipboard paste) +sudo apt install xdotool + +# Run +python hotkey_daemon.py --server http://localhost:7890 +``` + +| Option | Default | Notes | +|---|---|---| +| `--server` | `http://localhost:7890` | Voice Creator URL | +| `--backend` | `configured` | STT backend to use | + +**Platform notes:** +- **Linux X11:** uses `xdotool type` for direct injection; clipboard fallback on Wayland. +- **macOS:** clipboard paste via Cmd+V; grant Accessibility + Microphone in System Preferences. +- **Windows:** clipboard paste via Ctrl+V. + +--- + +## OpenAI-Compatible Proxy + +This app exposes a `/v1` proxy that any OpenAI-compatible client can use: + +``` +Base URL: http://YOUR_HOST:7890/v1 +API key: dummy (any non-empty value) +Model: tts-1 +Voice: EN_F_Anna (or any active voice ID, or vd_ for virtual voices) +``` + +Routing rules (App Routing tab) run on every request through this proxy. + +--- + +## API Reference + +### TTS endpoints + +| Endpoint | Description | +|---|---| +| `POST /v1/audio/speech` | OpenAI-compatible TTS with routing | +| `POST /speak` | Direct TTS with per-client voice binding | +| `POST /api/tts-preview` | Preview any backend/voice combination | +| `POST /api/tts-preview` with `apply_persona:true` | Preview with LLM persona rewrite | +| `GET /v1/audio/voices` | List active voice IDs | +| `GET /v1/models` | List TTS models | + +### STT endpoints + +| Endpoint | Description | +|---|---| +| `POST /v1/audio/transcriptions` | OpenAI-compatible transcription | +| `POST /api/transcribe-bytes` | Multipart upload → transcription (used by hotkey daemon) | + +### MCP endpoint + +| Endpoint | Description | +|---|---| +| `POST /mcp` | JSON-RPC 2.0 handler (`initialize`, `tools/list`, `tools/call`) | +| `GET /mcp` | SSE keep-alive stream | + +### Voice library + +| Endpoint | Description | +|---|---| +| `GET /api/voices` | List all voices with metadata | +| `POST /api/save` | Save a voice from a temp file | +| `POST /api/update-voice-meta` | Update note, rating, flag, gender, persona | +| `POST /api/voices/calculate-db` | Normalize dBFS for all active voices | +| `POST /api/voices/benchmark` | Run latency/quality benchmarks | +| `GET /api/tts-backends` | List all backends with availability and capabilities | + +### /speak bindings + +| Endpoint | Description | +|---|---| +| `GET /speak/bindings` | List all per-client voice bindings | +| `PUT /speak/bindings/{id}` | Set a per-client voice binding | +| `DELETE /speak/bindings/{id}` | Remove a binding | + +--- + +## Voice File Format | Property | Value | |---|---| | Format | WAV (PCM, uncompressed) | -| Sample rate | **24 000 Hz** | -| Channels | **Mono** | -| Bit depth | **16-bit** | -| Duration | **5–20 seconds** | -| Companion file | `.reference.txt` — verbatim transcript of the spoken audio | +| Sample rate | 24 000 Hz | +| Channels | Mono | +| Bit depth | 16-bit | +| Duration | 5–20 seconds | +| Companion file | `.reference.txt` — verbatim transcript | -The app converts all source audio to this specification automatically using `pydub` + `ffmpeg`. +The app converts all source audio automatically using `pydub` + `ffmpeg`. -### Voice metadata sidecar files - -Each saved voice can have optional sidecar files stored alongside the WAV: +### Sidecar files | File | Contents | |---|---| -| `.reference.txt` | Transcript of the reference audio (required for voice cloning) | -| `.meta.json` | Note, star rating, flag, gender, enabled state | +| `.reference.txt` | Reference transcript (required for cloning) | +| `.meta.json` | Note, rating, flag, gender, enabled, **persona**, loudness, benchmark | | `.jpg` / `.png` / `.webp` | Profile picture shown in the library | --- -## Naming convention - -Voice IDs have no forced format. The suggested convention is: +## Naming Convention ``` {LANG}_{GENDER}_{Name}.wav @@ -483,47 +401,9 @@ Voice IDs have no forced format. The suggested convention is: Examples: `EN_F_Anna.wav`, `DE_M_Christoph.wav`, `ZH_F_Mei.wav` Language codes: `EN DE ZH FR ES JA KO IT PT RU AR PL NL SV TR HI` -Gender codes: `F` (female) · `M` (male) · `N` (neutral / non-binary) +Gender codes: `F` (female) · `M` (male) · `N` (neutral) -Custom IDs like `narrator_calm` or `my-voice-v2` are equally valid — the naming helper in the UI is optional. - ---- - -## Volume mounts - -| Host path | Container path | Purpose | -|---|---|---| -| `${VOICE_HOST_DIR:-./voices}` | `/voices` | Shared voice collection (read-write) | -| Named volume `tts_voice_creator_config` | `/home/app/.config/tts-voice-creator` | Persisted settings and Voice Design presets | - -The TTS server should mount and scan the `active_voices` subfolder. The `hidden_voices` subfolder is for voices kept in the library but removed from Qwen3-TTS model discovery. - ---- - -## Reloading voices into your TTS server - -Most local TTS servers load voices on startup. After saving a new voice, restart the TTS container: - -```bash -docker restart -``` - -The default public Compose file does not mount the Docker socket. Restart the -TTS container from Docker, Portainer, or your host scripts. - ---- - -## Public Repository Safety - -The repository ignores private runtime data: - -- `.env` files and local settings. -- `.claude/`, editor workspaces, logs, and caches. -- Voice folders and audio/video files. -- Per-voice transcript and metadata sidecars. - -Do not commit real API keys, private voice samples, speaker folders, or -host-specific paths. +Custom IDs like `narrator_calm` are equally valid. --- @@ -531,33 +411,82 @@ host-specific paths. ``` Browser (any device on your local network) - └── port 7860 ──► FastAPI + uvicorn (server.py) - ├── pydub + ffmpeg audio conversion & trimming - ├── yt-dlp YouTube / URL audio extraction - ├── Whisper API ──────► transcription - ├── TTS API ──────► preview & voice list - └── VoiceDesign API ───► AI voice generation + └── port 7890 ──► FastAPI + uvicorn (server.py) + ├── /v1 OpenAI-compatible TTS/STT proxy + routing + ├── /speak Direct TTS with per-client bindings + ├── /mcp Native MCP server (JSON-RPC 2.0) + ├── /api/... Voice library, settings, benchmarks + ├── pydub+ffmpeg audio conversion & trimming + ├── yt-dlp YouTube / URL audio extraction + ├── Whisper API transcription (multiple backends) + ├── TTS backends voice_clone · voice_design · customvoice + │ streaming · kokoro · vibevoice + │ nvidia_magpie · nvidia_zeroshot · nvidia_flow + └── LLM API persona rewrite, text refinement -Frontend (single HTML file, no build step) - ├── WaveSurfer.js 7 interactive waveform + region trimmer - └── MediaRecorder API browser microphone recording +Host machine (optional) + └── hotkey_daemon.py global Ctrl+Shift+Space → record → transcribe → type + +Frontend (static HTML + Vanilla JS, no build step) + ├── WaveSurfer.js 7 interactive waveform + region trimmer + ├── MediaRecorder API browser microphone recording + └── MDI v7.4.47 icons ``` --- +## CLI + +`ttsvc_cli.py` talks to the running app over HTTP: + +```bash +./ttsvc_cli.py settings show +./ttsvc_cli.py settings set --tts-url http://host.docker.internal:8020 +./ttsvc_cli.py voices list --all +./ttsvc_cli.py voices enable EN_F_Anna +./ttsvc_cli.py routes openwebui-defaults --en EN_F_Anna --de DE_M_Max +./ttsvc_cli.py tts speak --voice EN_F_Anna --text "Hello world" --out test.wav +./ttsvc_cli.py create-voice --file sample.wav --voice-id EN_F_Anna --transcript "Exact words." +``` + +Use `--base-url http://HOST:7890` for remote instances. + +--- + +## Guided Installer + +```bash +python3 guided_installer.py +# or: start-guided-installer.sh / .command / .bat +``` + +Checks Docker, configures containers, generates Compose files, and runs deployment with visible logs. + +--- + +## Volume Mounts + +| Host path | Container path | Purpose | +|---|---|---| +| `${VOICE_HOST_DIR:-./voices}` | `/voices` | Voice collection (read-write) | +| Named volume `tts-voice-creator-clone-and-design-2` | `/home/app/.config/tts-voice-creator` | Settings and presets | +| `./static` | `/app/static` | Frontend (live-reload without rebuild) | +| `./server.py` | `/app/server.py` | Backend (live-reload without rebuild) | + +--- + ## Troubleshooting -**WaveSurfer / UI doesn't load** — the container needs internet access to fetch WaveSurfer.js from `unpkg.com`. If your network is air-gapped, download the two JS files and serve them from `static/`. +**"Cannot connect to TTS/STT API"** — ensure both containers share the same Docker network; use container hostnames, not `localhost`. -**"Cannot connect to Whisper/TTS API"** — check that: -1. The API server container is running -2. Both containers are on the same Docker network -3. You're using the container name (not `localhost`) as the URL inside Docker — e.g. `http://whisper:8000`, not `http://localhost:8000` +**Microphone not working** — browsers block `getUserMedia` on non-`localhost` origins without HTTPS. Use a reverse proxy with TLS for remote access. -**Microphone not working** — browsers block `getUserMedia` on non-`localhost` origins without HTTPS. Set up a reverse proxy with a TLS certificate (e.g. Nginx + Let's Encrypt or Caddy) if accessing from another device. +**Persona rewrite not working** — check the LLM URL in **Settings → LLM server**. The endpoint must be OpenAI-compatible (e.g. Ollama at `http://localhost:11434/v1`). -**Generated voice sounds wrong** — for cloning, check the reference audio quality: no background noise, single speaker, natural speech, and a correct transcript. For voice design, make the description more specific — add age, pitch, pace, and accent details. +**MCP tools not appearing** — run `claude mcp list` and confirm `voice-creator` is shown. Restart Claude Code after adding. -**Voice not appearing in TTS server after save** — restart the TTS container. Most servers only scan the voices directory at startup. +**Hotkey daemon not typing** — on Linux, install `xdotool` for X11; on Wayland it falls back to clipboard paste (Ctrl+V). On macOS, grant Accessibility permission. -**Transcription produces garbled text** — try a larger Whisper model (`large-v3` recommended), or manually type the transcript before saving. +**Voice not appearing after save** — restart the TTS container (`docker restart `). Most engines only scan the voice directory on startup. + +**Transcription garbled** — use `large-v3` Whisper model; manually correct the transcript before saving. diff --git a/server.py b/server.py index 04b01cf..965ffe0 100644 --- a/server.py +++ b/server.py @@ -2,6 +2,7 @@ from __future__ import annotations import asyncio +import base64 import glob import ipaddress import io @@ -50,6 +51,7 @@ _FASTER_WHISPER_DEFAULT = os.environ.get("FASTER_WHISPER_URL", "http://host.dock _WHISPER_CPP_DEFAULT = os.environ.get("WHISPER_CPP_URL", "http://host.docker.internal:8080") _GROQ_STT_ENDPOINT = "https://api.groq.com/openai/v1" _KOKORO_DEFAULT = os.environ.get("KOKORO_URL", "http://host.docker.internal:8880/v1") +_VIBEVOICE_DEFAULT = os.environ.get("VIBEVOICE_URL", "http://192.168.178.8:8027") _TTS_CONTAINER = os.environ.get("TTS_CONTAINER_NAME", "faster-qwen3-tts") _TTS_CONTAINERS_RAW = os.environ.get("TTS_CONTAINER_NAMES", "") # comma-separated override _VOICE_DESIGN_MODEL = os.environ.get("VOICE_DESIGN_MODEL", "Qwen3-TTS-12Hz-1.7B-VoiceDesign") @@ -313,7 +315,7 @@ _SETTINGS_KEYS = { "output_dir", "voices_scan_dir", "voice_design_url", "customvoice_url", "nvidia_router_url", "nvidia_tts_url", "nvidia_asr_url", "nvidia_clone_url", "nvidia_zeroshot_url", "nvidia_flow_url", - "faster_whisper_url", "whisper_cpp_url", "groq_api_key", "kokoro_url", + "faster_whisper_url", "whisper_cpp_url", "groq_api_key", "kokoro_url", "vibevoice_url", "whisper_api_key", "tts_api_key", "voice_design_api_key", "elevenlabs_api_key", "tts_stability_enabled", "tts_extra_params", "tts_extra_params_by_backend", # Captures settings @@ -321,6 +323,8 @@ _SETTINGS_KEYS = { "auto_refine", "refine_model", "refine_fillers", "refine_repetitions", "refine_corrections", "refine_punctuation", "captures_default_voice", + "client_voice_bindings", + "llm_url", } @@ -335,6 +339,7 @@ _TTS_STABILITY_BY_BACKEND_DEFAULT = { "nvidia_zeroshot": {}, "nvidia_flow": {}, "kokoro": {}, + "vibevoice": {}, } _TTS_PAYLOAD_CORE_KEYS = {"model", "input", "voice", "response_format", "instruct", "language"} @@ -437,9 +442,12 @@ def _clean_preview_backend(value: str) -> str: "kokoro_fastapi": "kokoro", "kokoro_tts": "kokoro", "kokoro_local": "kokoro", + "vibevoice_service": "vibevoice", + "vibe_voice": "vibevoice", + "vibetts": "vibevoice", } key = aliases.get(key, key) - return key if key in {"voice_clone", "streaming", "customvoice", "voice_design", "nvidia_magpie", "nvidia_zeroshot", "nvidia_flow", "kokoro"} else "voice_clone" + return key if key in {"voice_clone", "streaming", "customvoice", "voice_design", "nvidia_magpie", "nvidia_zeroshot", "nvidia_flow", "kokoro", "vibevoice"} else "voice_clone" def _preview_backend_base_url(settings: dict, backend: str) -> str: @@ -458,6 +466,8 @@ def _preview_backend_base_url(settings: dict, backend: str) -> str: return settings.get("nvidia_flow_url") or settings.get("nvidia_clone_url") or settings.get("nvidia_router_url") or _NVIDIA_FLOW_DEFAULT if backend == "kokoro": return settings.get("kokoro_url") or _KOKORO_DEFAULT + if backend == "vibevoice": + return settings.get("vibevoice_url") or _VIBEVOICE_DEFAULT return settings.get("tts_url") or _TTS_DEFAULT @@ -496,6 +506,7 @@ def _load_settings() -> dict: "whisper_cpp_url": _WHISPER_CPP_DEFAULT, "groq_api_key": "", "kokoro_url": _KOKORO_DEFAULT, + "vibevoice_url": _VIBEVOICE_DEFAULT, "whisper_api_key": "", "tts_api_key": "", "voice_design_api_key": "", @@ -512,6 +523,8 @@ def _load_settings() -> dict: "refine_corrections": True, "refine_punctuation": True, "captures_default_voice": "", + "client_voice_bindings": {}, + "llm_url": "http://localhost:11434/v1", } if CONFIG_FILE.exists(): try: @@ -1973,6 +1986,38 @@ async def transcribe(request: Request): raise HTTPException(502, f"STT error ({backend}): {e}") +@app.post("/api/transcribe-bytes") +async def transcribe_bytes( + file: UploadFile = File(...), + backend: str = Form("configured"), +): + """Accept raw audio upload and return transcription directly (used by hotkey daemon).""" + suffix = Path(file.filename or "audio.wav").suffix.lower() or ".wav" + if suffix not in _AUDIO_EXTS: + raise HTTPException(400, "Unsupported audio type") + tmp = TEMP_DIR / f"{uuid.uuid4().hex}_daemon{suffix}" + wav_tmp = tmp + try: + with tmp.open("wb") as f: + _copy_limited(file.file, f, _MAX_UPLOAD_BYTES) + if suffix != ".wav": + wav_tmp = _to_wav_24k(tmp) + settings = _load_settings() + stt_backend = _clean_stt_backend(backend) + text, used_backend = await asyncio.to_thread(_transcribe_audio, wav_tmp, settings, stt_backend) + return {"text": text, "backend": used_backend} + except HTTPException: + raise + except Exception as e: + raise HTTPException(502, f"STT error: {e}") + finally: + for p in {tmp, wav_tmp}: + try: + p.unlink(missing_ok=True) + except Exception: + pass + + # ── Save voice ──────────────────────────────────────────────────────────────── @app.post("/api/save") @@ -3146,6 +3191,7 @@ def _backend_display_name(backend: str, url: str) -> str: "nvidia_zeroshot":"NVIDIA Magpie Zeroshot Clone", "nvidia_flow": "NVIDIA Magpie Flow Clone", "kokoro": "Kokoro FastAPI (82M)", + "vibevoice": "VibeVoice TTS", } port = _backend_port_label(url) return f"{port} {names.get(backend, backend)}" if port else names.get(backend, backend) @@ -3217,6 +3263,14 @@ def _backend_capabilities(backend: str) -> dict: "uses_wav": False, "style_aware": False, "true_streaming": False, "speed": "~0.1× CPU", "latency": "0.2–0.5 s", "quality": "High (82M)", "ram": "300 MB CPU", }, + "vibevoice": { + "purpose": "VibeVoice TTS service. Simple text-in, audio-out REST endpoint.", + "identity": "Single built-in voice; no WAV cloning or voice selection.", + "style": "Text only — no voice ID or style parameters.", + "best_for": "Lightweight local TTS on port 8027. Minimal setup, fast response.", + "uses_wav": False, "style_aware": False, "true_streaming": False, + "speed": "fast", "latency": "0.2–1 s", "quality": "High", "ram": "varies", + }, } return caps.get(_clean_preview_backend(backend), {}) @@ -3234,7 +3288,7 @@ def _backend_health(url: str) -> bool: def _backend_available(backend: str, voices: list, health: bool) -> bool: - if _clean_preview_backend(backend) in {"nvidia_zeroshot", "nvidia_flow", "kokoro"}: + if _clean_preview_backend(backend) in {"nvidia_zeroshot", "nvidia_flow", "kokoro", "vibevoice"}: return health return bool(voices) or health @@ -3243,7 +3297,7 @@ def _backend_available(backend: str, voices: list, health: bool) -> bool: async def tts_backends(): settings = _load_settings() items = [] - for backend in ("voice_clone", "voice_design", "customvoice", "streaming", "kokoro", "nvidia_magpie", "nvidia_zeroshot", "nvidia_flow"): + for backend in ("voice_clone", "voice_design", "customvoice", "streaming", "kokoro", "vibevoice", "nvidia_magpie", "nvidia_zeroshot", "nvidia_flow"): url = _validate_http_url(_preview_backend_base_url(settings, backend), allow_private=True).rstrip("/") voices = _fetch_backend_voices(settings, backend) health = _backend_health(url) @@ -3675,6 +3729,17 @@ def _nvidia_clone_request_audio(text: str, voice: str, settings: dict, mode: str return audio, media_type +def _vibevoice_request_audio(text: str, settings: dict) -> tuple[bytes, str]: + url = _preview_backend_base_url(settings, "vibevoice").rstrip("/") + resp = requests.post(f"{url}/tts", json={"text": text}, timeout=60) + resp.raise_for_status() + audio = resp.content + if not audio: + raise RuntimeError("VibeVoice backend returned empty audio") + media_type = resp.headers.get("content-type", "audio/wav").split(";", 1)[0] or "audio/wav" + return audio, media_type + + def _preview_request_audio(text: str, voice: str, settings: dict, instruct: str = "", backend: str = "voice_clone") -> tuple[bytes, str]: backend = _clean_preview_backend(backend) if backend == "streaming": @@ -3719,6 +3784,8 @@ def _preview_request_audio(text: str, voice: str, settings: dict, instruct: str backend_override="openai", extra_backend="kokoro", ) + if backend == "vibevoice": + return _vibevoice_request_audio(text, settings) return _tts_request_audio(text, voice, settings, instruct) @@ -3886,6 +3953,18 @@ async def tts_preview(request: Request): backend = _clean_preview_backend(data.get("backend", "voice_clone")) settings = _load_settings() + if data.get("apply_persona"): + scan_dir = Path(settings.get("voices_scan_dir", _VOICES_DIR_DEFAULT)) + wav = _find_voice_audio(voice, scan_dir) + if wav: + persona = _load_meta(wav).get("persona", "") + if persona: + llm_url = (settings.get("llm_url") or "http://localhost:11434/v1").rstrip("/") + try: + text = await asyncio.to_thread(_rewrite_with_persona_sync, text, persona, llm_url) + except Exception as e: + raise HTTPException(502, f"Persona rewrite failed: {e}") + try: audio, media_type = await asyncio.to_thread(_preview_request_audio, text, voice, settings, instruct, backend) except Exception as e: @@ -4533,6 +4612,36 @@ async def refine_text(request: Request): # ── LLM persona rewrite ─────────────────────────────────────────────────────── +def _rewrite_with_persona_sync(text: str, persona: str, llm_url: str, model: str = "") -> str: + """Inline synchronous persona rewrite; raises RuntimeError on failure.""" + system = ( + f"Rephrase the user's text as if spoken by this character: {persona}\n" + "Keep the same meaning but adapt vocabulary, tone, and style to the character. " + "Return ONLY the rephrased text — no quotes, no explanation." + ) + payload: dict = { + "messages": [ + {"role": "system", "content": system}, + {"role": "user", "content": text}, + ], + "temperature": 0.3, + "max_tokens": 512, + } + if model: + payload["model"] = model + resp = requests.post( + f"{llm_url.rstrip('/')}/chat/completions", + json=payload, + headers={"Authorization": "Bearer no-key"}, + timeout=60, + ) + resp.raise_for_status() + result = resp.json()["choices"][0]["message"]["content"].strip() + if result.startswith('"') and result.endswith('"'): + result = result[1:-1].strip() + return result + + @app.post("/api/rewrite-with-persona") async def rewrite_with_persona(request: Request): """Rewrite user text in a voice persona's character using a local LLM.""" @@ -4760,6 +4869,286 @@ async def voices_import(file: UploadFile = File(...)): return {"ok": True, "imported": imported} +# ── /speak REST endpoint ────────────────────────────────────────────────────── + +def _resolve_speak_voice(settings: dict, client_id: str, explicit_voice: str) -> str: + if explicit_voice: + return explicit_voice + bindings: dict = settings.get("client_voice_bindings") or {} + if client_id and client_id in bindings: + return bindings[client_id] + return settings.get("captures_default_voice") or "" + + +@app.post("/speak") +async def speak(request: Request): + data = await request.json() + text: str = str(data.get("text") or "").strip() + if not text: + raise HTTPException(400, "text is required") + + explicit_voice: str = str(data.get("voice") or data.get("profile_id") or data.get("profile") or "").strip() + backend: str = _clean_preview_backend(str(data.get("backend") or "voice_clone")) + apply_persona: bool = bool(data.get("apply_persona") or data.get("personality")) + client_id: str = request.headers.get("X-Voice-Creator-Client-Id", "").strip() + + settings = _load_settings() + voice = _resolve_speak_voice(settings, client_id, explicit_voice) + if not voice: + raise HTTPException(400, "No voice specified and no default voice configured") + + if apply_persona: + scan_dir = Path(settings.get("voices_scan_dir", _VOICES_DIR_DEFAULT)) + wav = _find_voice_audio(voice, scan_dir) + if wav: + persona = _load_meta(wav).get("persona", "") + if persona: + llm_url = (settings.get("llm_url") or "http://localhost:11434/v1").rstrip("/") + try: + text = await asyncio.to_thread(_rewrite_with_persona_sync, text, persona, llm_url) + except Exception as e: + raise HTTPException(502, f"Persona rewrite failed: {e}") + + instruct = str(data.get("instruct") or "") + try: + audio, media_type = await asyncio.to_thread( + _preview_request_audio, text, voice, settings, instruct, backend + ) + except Exception as e: + raise HTTPException(502, f"TTS error: {e}") + + return Response(content=audio, media_type=media_type) + + +@app.get("/speak/bindings") +async def get_speak_bindings(): + s = _load_settings() + return {"bindings": s.get("client_voice_bindings") or {}} + + +@app.put("/speak/bindings/{client_id}") +async def put_speak_binding(client_id: str, request: Request): + data = await request.json() + voice = str(data.get("voice") or "").strip() + if not voice: + raise HTTPException(400, "voice is required") + s = _load_settings() + bindings: dict = dict(s.get("client_voice_bindings") or {}) + bindings[client_id] = voice + s["client_voice_bindings"] = bindings + _save_settings(s) + return {"ok": True, "client_id": client_id, "voice": voice} + + +@app.delete("/speak/bindings/{client_id}") +async def delete_speak_binding(client_id: str): + s = _load_settings() + bindings: dict = dict(s.get("client_voice_bindings") or {}) + if client_id not in bindings: + raise HTTPException(404, f"No binding for client '{client_id}'") + del bindings[client_id] + s["client_voice_bindings"] = bindings + _save_settings(s) + return {"ok": True, "client_id": client_id} + + +# ── MCP JSON-RPC 2.0 server ─────────────────────────────────────────────────── + +_MCP_SERVER_INFO = {"name": "tts-voice-creator", "version": "1.0.0"} + +_MCP_TOOLS = [ + { + "name": "speak", + "description": "Generate speech audio from text using a cloned voice. Returns a data URI with the WAV audio.", + "inputSchema": { + "type": "object", + "properties": { + "text": {"type": "string", "description": "Text to synthesize"}, + "voice": {"type": "string", "description": "Voice ID (optional — uses per-client binding or default if omitted)"}, + "apply_persona": {"type": "boolean", "description": "Rewrite text through the voice's persona before synthesis"}, + }, + "required": ["text"], + }, + }, + { + "name": "transcribe", + "description": "Transcribe base64-encoded WAV audio to text.", + "inputSchema": { + "type": "object", + "properties": { + "audio_base64": {"type": "string", "description": "Base64-encoded audio bytes (WAV preferred)"}, + }, + "required": ["audio_base64"], + }, + }, + { + "name": "list_captures", + "description": "List the 20 most recently generated audio files.", + "inputSchema": {"type": "object", "properties": {}}, + }, + { + "name": "list_profiles", + "description": "List all available voice profiles with their language, persona, and enabled state.", + "inputSchema": {"type": "object", "properties": {}}, + }, +] + + +async def _mcp_tool_speak(args: dict, client_id: str) -> dict: + text = str(args.get("text") or "").strip() + if not text: + raise ValueError("text is required") + explicit_voice = str(args.get("voice") or "").strip() + apply_persona = bool(args.get("apply_persona", False)) + settings = _load_settings() + voice = _resolve_speak_voice(settings, client_id, explicit_voice) + if not voice: + raise ValueError("No voice specified and no default voice configured") + if apply_persona: + scan_dir = Path(settings.get("voices_scan_dir", _VOICES_DIR_DEFAULT)) + wav = _find_voice_audio(voice, scan_dir) + if wav: + persona = _load_meta(wav).get("persona", "") + if persona: + llm_url = (settings.get("llm_url") or "http://localhost:11434/v1").rstrip("/") + text = await asyncio.to_thread(_rewrite_with_persona_sync, text, persona, llm_url) + audio, _media_type = await asyncio.to_thread( + _preview_request_audio, text, voice, settings, "", "voice_clone" + ) + audio_b64 = base64.b64encode(audio).decode() + return { + "content": [ + {"type": "text", "text": f"Generated {len(audio)} bytes of speech audio for voice '{voice}'."}, + {"type": "resource", "resource": {"uri": f"data:audio/wav;base64,{audio_b64}", "mimeType": "audio/wav"}}, + ] + } + + +async def _mcp_tool_transcribe(args: dict) -> dict: + raw = args.get("audio_base64") or "" + try: + audio_bytes = base64.b64decode(raw) + except Exception: + raise ValueError("audio_base64 is not valid base64") + tmp = TEMP_DIR / f"{uuid.uuid4().hex}_mcp_transcribe.wav" + tmp.write_bytes(audio_bytes) + settings = _load_settings() + try: + text, used_backend = await asyncio.to_thread(_transcribe_audio, tmp, settings, "configured") + finally: + try: + tmp.unlink(missing_ok=True) + except Exception: + pass + return {"content": [{"type": "text", "text": text}], "backend": used_backend} + + +async def _mcp_tool_list_captures() -> dict: + settings = _load_settings() + out_dir = _active_voices_dir(settings) + if not out_dir.exists(): + return {"content": [{"type": "text", "text": "[]"}]} + files = sorted( + (p for p in out_dir.rglob("*.wav") if not _is_internal_voice_file(p)), + key=lambda p: p.stat().st_mtime, + reverse=True, + )[:20] + result = [{"name": p.stem, "path": str(p), "mtime": p.stat().st_mtime} for p in files] + return {"content": [{"type": "text", "text": json.dumps(result)}]} + + +async def _mcp_tool_list_profiles() -> dict: + settings = _load_settings() + scan_dir = Path(settings.get("voices_scan_dir", _VOICES_DIR_DEFAULT)) + profiles = [] + seen: set[str] = set() + for audio in sorted(_voice_audio_files(scan_dir), key=lambda p: p.stem.lower()): + if audio.stem in seen: + continue + seen.add(audio.stem) + meta = _load_meta(audio) + parts = audio.stem.split("_", 2) + lang = parts[0].upper() if parts else "" + profiles.append({ + "id": audio.stem, + "name": audio.stem, + "lang": lang, + "persona": meta.get("persona", ""), + "enabled": meta.get("enabled", True), + }) + return {"content": [{"type": "text", "text": json.dumps(profiles)}]} + + +def _mcp_error_response(code: int, message: str, rpc_id) -> Response: + body = {"jsonrpc": "2.0", "error": {"code": code, "message": message}, "id": rpc_id} + return Response(content=json.dumps(body), media_type="application/json") + + +@app.post("/mcp") +async def mcp_jsonrpc(request: Request): + try: + body = await request.json() + except Exception: + return _mcp_error_response(-32700, "Parse error", None) + + rpc_id = body.get("id") + method = body.get("method", "") + params = body.get("params") or {} + client_id = request.headers.get("X-Voice-Creator-Client-Id", "").strip() + + try: + if method == "initialize": + result = { + "protocolVersion": "2024-11-05", + "serverInfo": _MCP_SERVER_INFO, + "capabilities": {"tools": {}}, + } + elif method == "notifications/initialized": + return Response(status_code=204) + elif method == "tools/list": + result = {"tools": _MCP_TOOLS} + elif method == "tools/call": + tool_name = str(params.get("name") or "") + tool_args = params.get("arguments") or {} + if tool_name == "speak": + result = await _mcp_tool_speak(tool_args, client_id) + elif tool_name == "transcribe": + result = await _mcp_tool_transcribe(tool_args) + elif tool_name == "list_captures": + result = await _mcp_tool_list_captures() + elif tool_name == "list_profiles": + result = await _mcp_tool_list_profiles() + else: + return _mcp_error_response(-32601, f"Unknown tool: {tool_name}", rpc_id) + else: + return _mcp_error_response(-32601, f"Method not found: {method}", rpc_id) + except ValueError as e: + return _mcp_error_response(-32602, str(e), rpc_id) + except Exception as e: + logger.exception("MCP tool error in method %s", method) + return _mcp_error_response(-32000, f"Server error: {e}", rpc_id) + + return Response( + content=json.dumps({"jsonrpc": "2.0", "result": result, "id": rpc_id}), + media_type="application/json", + ) + + +@app.get("/mcp") +async def mcp_sse(request: Request): + """SSE keep-alive stream (satisfies MCP spec GET /mcp requirement).""" + async def _keepalive() -> AsyncGenerator[str, None]: + yield ": mcp-sse-ready\n\n" + while True: + await asyncio.sleep(15) + yield ": ping\n\n" + return StreamingResponse( + _keepalive(), + media_type="text/event-stream", + headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"}, + ) + + # ── Static ──────────────────────────────────────────────────────────────────── app.mount("/static", StaticFiles(directory=str(STATIC_DIR)), name="static") diff --git a/static/app.js b/static/app.js index 0133fd5..86793e4 100644 --- a/static/app.js +++ b/static/app.js @@ -1092,6 +1092,7 @@ function renderIntegrationSnippets() { const v1 = getTtsV1Url(); const streamV1 = getTtsStreamV1Url(); const proxyV1 = getCreatorV1Url(); + const proxyBase = proxyV1.replace(/\/v1$/, ''); updateCreatorUrlHints(); const voice = integrationVoiceExample(); const voices = integrationVoiceList(); @@ -1189,58 +1190,41 @@ curl -s "${proxyV1}/audio/speech" \\ -d '{"model":"tts-1","voice":"${vdVoice}","input":"This line is generated through the VoiceDesign container.","response_format":"wav"}' \\ --output voicedesign-virtual.wav`; - if ($('snippet-mcp-server')) $('snippet-mcp-server').textContent = -`#!/usr/bin/env python3 -"""MCP server — exposes TTS Voice Creator as tools for AI agents. -Install: pip install mcp httpx -Run: python3 tts_mcp_server.py -""" -import base64, httpx -from mcp.server.fastmcp import FastMCP - -BASE = "${proxyV1}" -mcp = FastMCP("tts-voice-creator") - -@mcp.tool() -async def list_voices() -> list: - """Return all active TTS voice IDs.""" - async with httpx.AsyncClient() as c: - r = await c.get(f"{BASE}/models") - return [m["id"] for m in r.json().get("data", [])] - -@mcp.tool() -async def speak(text: str, voice: str = "${voice}", format: str = "mp3") -> str: - """Synthesize text to speech. Returns base64-encoded audio bytes.""" - async with httpx.AsyncClient(timeout=60) as c: - r = await c.post(f"{BASE}/audio/speech", - json={"model": "tts-1", "voice": voice, - "input": text, "response_format": format}) - r.raise_for_status() - return base64.b64encode(r.content).decode() - -@mcp.tool() -async def transcribe(audio_b64: str, language: str = "") -> str: - """Transcribe base64-encoded audio to text via the configured STT endpoint.""" - audio = base64.b64decode(audio_b64) - async with httpx.AsyncClient(timeout=60) as c: - files = {"file": ("audio.wav", audio, "audio/wav")} - data = {"language": language} if language else {} - r = await c.post(f"{BASE}/audio/transcriptions", files=files, data=data) - r.raise_for_status() - return r.json().get("text", "") - -if __name__ == "__main__": - mcp.run()`; + if ($('snippet-mcp-claude-cmd')) $('snippet-mcp-claude-cmd').textContent = +`claude mcp add voice-creator \\ + --transport http \\ + --url ${proxyBase}/mcp \\ + --header "X-Voice-Creator-Client-Id: claude-code"`; if ($('snippet-mcp-claude-config')) $('snippet-mcp-claude-config').textContent = `{ "mcpServers": { - "tts-voice-creator": { - "command": "python3", - "args": ["/path/to/tts_mcp_server.py"] + "voice-creator": { + "url": "${proxyBase}/mcp", + "headers": { "X-Voice-Creator-Client-Id": "my-agent" } } } }`; + + if ($('snippet-speak')) $('snippet-speak').textContent = +`# Bind your client ID to a voice once (persisted in Settings) +curl -X PUT ${proxyBase}/speak/bindings/my-script \\ + -H "Content-Type: application/json" \\ + -d '{"voice":"${voice}"}' + +# Generate speech — voice from binding, persona rewrite optional +curl -X POST ${proxyBase}/speak \\ + -H "Content-Type: application/json" \\ + -H "X-Voice-Creator-Client-Id: my-script" \\ + -d '{"text":"Hello world","apply_persona":true}' \\ + --output speech.wav + +# Or specify voice directly +curl -X POST ${proxyBase}/speak \\ + -H "Content-Type: application/json" \\ + -d '{"text":"Hello world","voice":"${voice}"}' \\ + --output speech.wav`; + } document.querySelectorAll('.copy-snippet').forEach(btn => btn.addEventListener('click', async () => { const el = $(btn.dataset.snippet); @@ -1974,6 +1958,8 @@ async function loadSettings() { $('s-whisper-cpp-url').value = s.whisper_cpp_url || ''; $('s-groq-api-key').value = s.groq_api_key || ''; $('s-kokoro-url').value = s.kokoro_url || ''; + $('s-vibevoice-url').value = s.vibevoice_url || ''; + const llmUrlEl = $('s-llm-url'); if (llmUrlEl) llmUrlEl.value = s.llm_url || ''; _appSettings = s; $('s-tts-stream-url').value = s.tts_stream_url || ''; $('s-customvoice-url').value = s.customvoice_url || 'http://host.docker.internal:8022'; @@ -1994,7 +1980,9 @@ async function loadSettings() { $('s-tts-extra-nvidia-magpie').value = JSON.stringify(byBackend.nvidia_magpie || {}, null, 2); $('s-tts-extra-nvidia-zeroshot').value = JSON.stringify(byBackend.nvidia_zeroshot || {}, null, 2); $('s-tts-extra-nvidia-flow').value = JSON.stringify(byBackend.nvidia_flow || {}, null, 2); - $('s-tts-extra-kokoro').value = JSON.stringify(byBackend.kokoro || {}, null, 2); + $('s-tts-extra-kokoro').value = JSON.stringify(byBackend.kokoro || {}, null, 2); + const vibevoiceExtraEl = $('s-tts-extra-vibevoice'); + if (vibevoiceExtraEl) vibevoiceExtraEl.value = JSON.stringify(byBackend.vibevoice || {}, null, 2); $('s-voice-design-url').value = s.voice_design_url || 'http://host.docker.internal:8021'; $('s-vd-key').value = s.voice_design_api_key || ''; $('s-voices-scan-dir').value = s.voices_scan_dir || ''; @@ -2058,6 +2046,7 @@ document.addEventListener('click', async e => { if (!e.target.closest('.s-save-b ['nvidia_zeroshot', 's-tts-extra-nvidia-zeroshot', 'NVIDIA Zeroshot'], ['nvidia_flow', 's-tts-extra-nvidia-flow', 'NVIDIA Flow'], ['kokoro', 's-tts-extra-kokoro', 'Kokoro'], + ['vibevoice', 's-tts-extra-vibevoice', 'VibeVoice'], ]; try { for (const [key, id, label] of paramFields) { @@ -2076,6 +2065,8 @@ document.addEventListener('click', async e => { if (!e.target.closest('.s-save-b groq_api_key: $('s-groq-api-key').value, tts_url: $('s-tts-url').value, kokoro_url: $('s-kokoro-url').value, + vibevoice_url: $('s-vibevoice-url').value, + llm_url: $('s-llm-url')?.value || '', tts_stream_url: $('s-tts-stream-url').value, customvoice_url: $('s-customvoice-url').value, nvidia_router_url: $('s-nvidia-router-url').value, @@ -2111,6 +2102,7 @@ document.addEventListener('click', async e => { if (!e.target.closest('.s-save-b _appSettings.nvidia_zeroshot_url = $('s-nvidia-zeroshot-url').value; _appSettings.nvidia_flow_url = $('s-nvidia-flow-url').value; _appSettings.voice_design_url = $('s-voice-design-url').value; + _appSettings.vibevoice_url = $('s-vibevoice-url').value; _appSettings.tts_stream_mode = $('s-tts-stream-mode').value; _ttsStreamHealth = null; await refreshTtsBackendAvailability($('tts-backend-select')?.value || ''); @@ -5778,23 +5770,24 @@ async function createTtsStreamUrl(voice, text, instruct = '') { const data = await r.json(); return data.url; } -async function fetchTtsPreviewBlob(voice, text, responseFormat = 'wav', instruct = '', backend = 'voice_clone') { - const r = await fetch('/api/tts-preview', {method:'POST',headers:{'Content-Type':'application/json'}, - body:JSON.stringify({text,voice,response_format:responseFormat,instruct,backend})}); +async function fetchTtsPreviewBlob(voice, text, responseFormat = 'wav', instruct = '', backend = 'voice_clone', applyPersona = false) { + const body = {text, voice, response_format: responseFormat, instruct, backend}; + if (applyPersona) body.apply_persona = true; + const r = await fetch('/api/tts-preview', {method:'POST', headers:{'Content-Type':'application/json'}, body: JSON.stringify(body)}); if (!r.ok) { const e=await r.json().catch(()=>({})); throw new Error(e.detail || r.statusText); } return await r.blob(); } -async function createTtsAudioSource(voice, text, backend = 'voice_clone', modeOverride = 'settings', instruct = '') { +async function createTtsAudioSource(voice, text, backend = 'voice_clone', modeOverride = 'settings', instruct = '', applyPersona = false) { const mode = effectiveTtsPlaybackMode(modeOverride); if (backend !== 'streaming' || mode === 'buffered') { - const blob = await fetchTtsPreviewBlob(voice, text, 'wav', instruct, backend); + const blob = await fetchTtsPreviewBlob(voice, text, 'wav', instruct, backend, applyPersona); return {url: URL.createObjectURL(blob), blob, streaming:false, label:'buffered'}; } try { return {url: await createTtsStreamUrl(voice, text, instruct), blob:null, streaming:true, label:'streaming'}; } catch (e) { if (mode === 'streaming') throw e; - const blob = await fetchTtsPreviewBlob(voice, text, 'wav', instruct, backend); + const blob = await fetchTtsPreviewBlob(voice, text, 'wav', instruct, backend, applyPersona); return {url: URL.createObjectURL(blob), blob, streaming:false, label:'buffered'}; } } @@ -5813,6 +5806,7 @@ function _onPreviewGenerated(source, voice, text, backend) { $('preview-btn').addEventListener('click', async () => { const voice=$('tts-voice-select').value, backend=$('tts-backend-select').value, text=$('preview-text-area').value.trim(), instruct=$('preview-style-instruction').value.trim(); + const applyPersona = $('preview-persona-toggle')?.checked || false; if(!backend) { toast('No available TTS backend','error'); return; } if(!voice) { toast('Select a TTS voice','error'); return; } if(!text) { toast('Enter preview text','error'); return; } @@ -5824,7 +5818,7 @@ $('preview-btn').addEventListener('click', async () => { const useChunked = $('preview-chunked-toggle')?.checked && text.length > 200 && typeof generateChunkedTts === 'function'; const source = useChunked ? await generateChunkedTts(voice, text, backend, instruct) - : await createTtsAudioSource(voice, text, backend, $('preview-playback-mode').value, instruct); + : await createTtsAudioSource(voice, text, backend, $('preview-playback-mode').value, instruct, applyPersona); previewBlob = source.blob; window._previewVoice = voice; window._previewBackend = backend; window._previewText = text; audio.src = source.url; @@ -6834,44 +6828,36 @@ loadSettings().then(() => { })(); loadVoiceLibrary().then(renderIntegrationSnippets).catch(e => status('Voice library load failed: ' + e.message)); -// ── AI Backends section: category tabs ──────────────────────────────────── - -(function initLlmCatTabs() { - const tabs = document.querySelectorAll('.llm-cat-tab[data-llm-cat]'); - const secs = document.querySelectorAll('.llm-section[data-llm-section]'); - if (!tabs.length) return; - - function showCat(cat) { - tabs.forEach(t => t.classList.toggle('active', t.dataset.llmCat === cat)); - secs.forEach(s => { s.hidden = s.dataset.llmSection !== cat; }); - if (cat === 'local') loadLocalContainers(); - } - - tabs.forEach(t => t.addEventListener('click', () => showCat(t.dataset.llmCat))); - // Load containers immediately since Local is the default active tab - loadLocalContainers(); -})(); +// ── Engines section: load containers on startup ──────────────────────────── +loadLocalContainers(); // ── Local Docker container management ───────────────────────────────────── async function loadLocalContainers() { - const grid = $('dc-grid'); - if (!grid) return; - grid.innerHTML = '
Checking container status…
'; + const gridTts = $('dc-grid-tts'); + const gridStt = $('dc-grid-stt'); + if (!gridTts && !gridStt) return; + const loading = '
Checking container status…
'; + if (gridTts) gridTts.innerHTML = loading; + if (gridStt) gridStt.innerHTML = loading; try { const r = await fetch('/api/local-containers'); const d = await r.json(); renderLocalContainers(d.containers || []); } catch (e) { - grid.innerHTML = `
Could not reach server: ${escHtml(e.message)}
`; + const err = `
Could not reach server: ${escHtml(e.message)}
`; + if (gridTts) gridTts.innerHTML = err; + if (gridStt) gridStt.innerHTML = err; } } function renderLocalContainers(containers) { - const grid = $('dc-grid'); - if (!grid) return; + const gridTts = $('dc-grid-tts'); + const gridStt = $('dc-grid-stt'); + if (!gridTts && !gridStt) return; if (!containers.length) { - grid.innerHTML = '
No containers defined.
'; + if (gridTts) gridTts.innerHTML = ''; + if (gridStt) gridStt.innerHTML = ''; return; } @@ -6900,71 +6886,93 @@ function renderLocalContainers(containers) { }; const roleIcon = { tts: '', stt: '', 'stt+tts': '', llm: '' }; - grid.innerHTML = containers.map(c => { + function buildCardHtml(c) { const st = c.status || 'not_found'; - const dotCls = st === 'running' ? 'dc-dot dc-running' - : st === 'exited' || st === 'stopped' ? 'dc-dot dc-stopped' - : 'dc-dot dc-absent'; - const stLabel = st === 'running' ? 'Running' - : st === 'exited' ? 'Stopped' - : st === 'stopped' ? 'Stopped' - : st === 'not_found' ? 'Not installed' - : st; + const running = st === 'running'; + const stopped = st === 'exited' || st === 'stopped'; + const absent = st === 'not_found'; + const stLabel = running ? 'Running' : stopped ? 'Stopped' : 'Not installed'; + const stCls = running ? 'llm-compat-running' : stopped ? 'llm-compat-stopped' : ''; + const cardCls = running ? ' llm-local-card-running' : stopped ? ' llm-local-card-offline' : ''; const roleBadge = ROLE_LABEL[c.role] || c.role || ''; - const portBadge = c.port ? `:${c.port}` : ''; - const installed = st !== 'not_found'; + const portStr = c.port ? ` :${c.port}` : ''; + const installed = !absent; const icon = DC_ICONS[c.name] || roleIcon[c.role] || '📦'; const metricChips = (DC_METRICS[c.name] || []) - .map(([em, txt]) => `${em} ${escHtml(txt)}`).join(''); - const metricsHtml = metricChips ? `
${metricChips}
` : ''; - + .map(([em, txt]) => `${em} ${escHtml(txt)}`).join(''); + const metricsHtml = metricChips ? `
${metricChips}
` : ''; const n = escHtml(c.name); const actions = installed - ? (st === 'running' - ? ` - ` - : ` - `) + ? (running + ? ` + ` + : ` + `) : (c.repo - ? `View on GitHub ` + ? ` View on GitHub` : ''); - - return `
-
- ${icon} - - ${escHtml(c.label || c.name)} - - ${roleBadge ? `${escHtml(roleBadge)}` : ''} - ${portBadge} - + const portNum = c.port || ''; + const snippetCode = c.role === 'tts' + ? `curl -s "http://localhost:${portNum}/v1/audio/speech" \\\n -H "Authorization: Bearer dummy" \\\n -H "Content-Type: application/json" \\\n -d '{"model":"tts-1","voice":"default","input":"Hello world","response_format":"wav"}' \\\n --output test.wav` + : `curl -s "http://localhost:${portNum}/v1/audio/transcriptions" \\\n -F "file=@audio.wav" \\\n -F "model=whisper-1"`; + const snippetCopy = escHtml(snippetCode); + const snippetHtml = portNum ? ` +
+
+ API test + +
+ 
${escHtml(snippetCode)}
+
` : ''; + return `
+
+ ${icon} + ${escHtml(c.label || c.name)} + ${roleBadge ? `${escHtml(roleBadge)}${escHtml(portStr)}` : ''} + ${escHtml(stLabel)}
-
${escHtml(stLabel)}
${metricsHtml} - ${c.description ? `

${escHtml(c.description)}

` : ''} -
${actions}
+ ${c.description ? `

${escHtml(c.description)}

` : ''} +
${actions}
${snippetHtml}
`; - }).join(''); + } - grid.querySelectorAll('.dc-btn[data-dc-action]').forEach(btn => { - btn.addEventListener('click', async () => { - const action = btn.dataset.dcAction; - const name = btn.dataset.dcName; - btn.disabled = true; - btn.textContent = action === 'start' ? 'Starting…' : 'Stopping…'; - try { - const r = await fetch(`/api/local-containers/${encodeURIComponent(name)}/${action}`, { method: 'POST' }); - const d = await r.json(); - if (!d.ok) toast(d.error || `${action} failed`, 'error'); - } catch (e) { - toast(`${action} failed: ${e.message}`, 'error'); - } - await loadLocalContainers(); + const isTts = c => c.role === 'tts'; + const isStt = c => c.role === 'stt' || c.role === 'stt+tts'; + + if (gridTts) gridTts.innerHTML = containers.filter(isTts).map(buildCardHtml).join(''); + if (gridStt) gridStt.innerHTML = containers.filter(isStt).map(buildCardHtml).join(''); + + [gridTts, gridStt].forEach(grid => { + if (!grid) return; + grid.querySelectorAll('.dc-btn[data-dc-action]').forEach(btn => { + btn.addEventListener('click', async () => { + const action = btn.dataset.dcAction; + const name = btn.dataset.dcName; + btn.disabled = true; + btn.textContent = action === 'start' ? 'Starting…' : 'Stopping…'; + try { + const r = await fetch(`/api/local-containers/${encodeURIComponent(name)}/${action}`, { method: 'POST' }); + const d = await r.json(); + if (!d.ok) toast(d.error || `${action} failed`, 'error'); + } catch (e) { + toast(`${action} failed: ${e.message}`, 'error'); + } + await loadLocalContainers(); + }); }); + grid.querySelectorAll('.dc-copy-btn').forEach(btn => { + btn.addEventListener('click', async () => { + const text = (btn.dataset.copy || '').replace(/ /g, '\n').replace(/"/g, '"').replace(/&/g, '&'); + await copyText(text); + toast('Copied', 'success'); + }); + }); + initLlmSnippets(grid); }); } -$('dc-refresh-btn')?.addEventListener('click', loadLocalContainers); +document.querySelectorAll('.dc-refresh-btn').forEach(b => b.addEventListener('click', loadLocalContainers)); // ── AI Backends section: copy, API keys, local service connect ───────────── @@ -7118,8 +7126,70 @@ $('dc-refresh-btn')?.addEventListener('click', loadLocalContainers); toast('Kokoro FastAPI URL saved → Settings. It now appears as "Kokoro FastAPI (82M)" in the TTS backend dropdown.', 'success'); }); + $('llm-use-vibevoice-tts')?.addEventListener('click', () => { + const url = document.querySelector('[data-llm-local-key="vibevoice"]')?.value.trim() + || 'http://192.168.178.8:8027'; + applyAndSaveSettings({ vibevoice_url: url }); + toast('VibeVoice URL saved → Settings. Pick "VibeVoice" in Try It Out → Backend or a routing rule.', 'success'); + }); + + const LLM_USE_MAP = { + 'llm-use-ollama-llm': { key: 'ollama', fallback: 'http://localhost:11434/v1' }, + 'llm-use-vllm-llm': { key: 'vllm', fallback: 'http://localhost:8000/v1' }, + 'llm-use-lmstudio-llm': { key: 'lmstudio', fallback: 'http://localhost:1234/v1' }, + 'llm-use-llamacpp-llm': { key: 'llamacpp', fallback: 'http://localhost:8080/v1' }, + }; + Object.entries(LLM_USE_MAP).forEach(([id, { key, fallback }]) => { + $(id)?.addEventListener('click', () => { + const url = document.querySelector(`[data-llm-local-key="${key}"]`)?.value.trim() || fallback; + applyAndSaveSettings({ llm_url: url }); + toast(`LLM URL set to ${url} → Settings. Used for persona rewrite and transcription refinement.`, 'success'); + }); + }); + })(); +// ── LLM snippet collapse ─────────────────────────────────────────────────── + +function initLlmSnippets(root) { + (root || document).querySelectorAll('.llm-local-snippet:not([data-snip-init])').forEach(snip => { + snip.dataset.snipInit = '1'; + const bar = snip.querySelector('.llm-snippet-bar'); + const pre = snip.querySelector('pre'); + const copyBtn = snip.querySelector('.llm-copy-btn'); + if (!bar || !pre) return; + + const labelEl = bar.querySelector('span:not(.llm-snip-chev)'); + const origText = labelEl ? labelEl.textContent.trim() : 'snippet'; + const cardName = snip.closest('.llm-local-card')?.querySelector('.llm-local-name')?.textContent?.trim() || ''; + const key = 'snip-' + (cardName + '-' + origText).toLowerCase().replace(/[^a-z0-9]+/g, '-').slice(0, 52); + + const chev = document.createElement('span'); + chev.className = 'llm-snip-chev'; + if (labelEl) bar.insertBefore(chev, labelEl); + else bar.prepend(chev); + + const apply = (open) => { + pre.hidden = !open; + if (copyBtn) copyBtn.hidden = !open; + if (labelEl) labelEl.textContent = open ? origText : 'Show code snippet'; + chev.textContent = open ? '▾' : '▸'; + }; + + const saved = localStorage.getItem(key); + apply(saved === '1'); // default: collapsed + + bar.addEventListener('click', e => { + if (copyBtn && (e.target === copyBtn || copyBtn.contains(e.target))) return; + const nowOpen = pre.hidden; + apply(nowOpen); + try { localStorage.setItem(key, nowOpen ? '1' : ''); } catch {} + }); + }); +} + +initLlmSnippets(document); + // ── Collapsible cards ────────────────────────────────────────────────────── (function initCollapsibleCards() { @@ -7178,6 +7248,77 @@ $('dc-refresh-btn')?.addEventListener('click', loadLocalContainers); }); })(); +// ── Collapsible integration cards (collapsed by default) ─────────────────── + +(function initIntegrationCards() { + const LS_KEY = 'icard-collapse-v1'; + const load = () => { try { return JSON.parse(localStorage.getItem(LS_KEY) || '{}'); } catch { return {}; } }; + const save = (k, v) => { const s = load(); s[k] = v; try { localStorage.setItem(LS_KEY, JSON.stringify(s)); } catch {} }; + const slug = t => t.trim().toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, '').slice(0, 48); + + document.querySelectorAll('.integration-card').forEach(card => { + const h3 = card.querySelector('h3'); + if (!h3 || card.dataset.icardInit) return; + card.dataset.icardInit = '1'; + + const key = 'icard-' + slug(h3.textContent); + const open = load()[key] === true; // default: closed + + // Prepend logo — favicon img or MDI icon (skip if h3 already has a leading .mdi span) + const hasIcon = h3.firstElementChild && h3.firstElementChild.classList.contains('mdi'); + const faviconUrl = card.dataset.favicon; + const iconClass = card.dataset.icon; + if (!hasIcon) { + if (faviconUrl) { + const img = document.createElement('img'); + img.className = 'icard-logo'; + img.src = faviconUrl; + img.alt = ''; + img.onerror = () => { img.style.display = 'none'; }; + h3.prepend(img); + } else if (iconClass) { + const span = document.createElement('span'); + span.className = 'icard-icon ' + iconClass; + h3.prepend(span); + } + } else if (faviconUrl) { + // Has existing .mdi span but also has a favicon — add favicon before the mdi span + const img = document.createElement('img'); + img.className = 'icard-logo'; + img.src = faviconUrl; + img.alt = ''; + img.onerror = () => { img.style.display = 'none'; }; + h3.prepend(img); + } + + // Append chevron + const chev = document.createElement('span'); + chev.className = 'icard-chev'; + chev.setAttribute('aria-hidden', 'true'); + h3.appendChild(chev); + + // Wrap all siblings after h3 in a body div + const body = document.createElement('div'); + body.className = 'icard-body'; + let sib = h3.nextElementSibling; + while (sib) { const nx = sib.nextElementSibling; body.appendChild(sib); sib = nx; } + card.appendChild(body); + + const apply = (isOpen) => { + body.hidden = !isOpen; + card.classList.toggle('icard-closed', !isOpen); + }; + + apply(open); + + h3.addEventListener('click', () => { + const nowOpen = body.hidden; + apply(nowOpen); + save(key, nowOpen); + }); + }); +})(); + // ── WAV merge utility (chunked TTS + playlist export) ────────────────────── async function mergeWavBlobs(blobs) { diff --git a/static/index.html b/static/index.html index 41122b0..a81cabe 100644 --- a/static/index.html +++ b/static/index.html @@ -70,8 +70,15 @@
Setup
-
- LLMs + +
App Routing diff --git a/static/nav.js b/static/nav.js index 0512282..faadad3 100644 --- a/static/nav.js +++ b/static/nav.js @@ -60,11 +60,22 @@ if (settingsTree) settingsTree.classList.toggle('open', sectionId === 's-settings'); if (settingsChevron) settingsChevron.classList.toggle('open', sectionId === 's-settings'); + // Engines nav tree + var enginesTree = document.getElementById('nav-engines-tree'); + var enginesChevron = document.getElementById('nav-engines-chevron'); + if (enginesTree) enginesTree.classList.toggle('open', sectionId === 's-llms'); + if (enginesChevron) enginesChevron.classList.toggle('open', sectionId === 's-llms'); + // When entering settings, show the active sub-page (default: connections) if (sectionId === 's-settings') { applySettingsPage(window._settingsSidebarCat || 'connections'); } + // When entering engines, show the active sub-page (default: llm) + if (sectionId === 's-llms') { + applyEnginesPage(window._enginesSidebarCat || 'llm'); + } + var main = document.getElementById('main-content'); if (main) main.scrollTop = 0; } @@ -118,6 +129,24 @@ if (cat === 'about' && typeof renderSettingsAbout === 'function') renderSettingsAbout(); }; + // ── Engines tree sub-navigation ─────────────────────────────────────────── + function applyEnginesPage(cat) { + document.querySelectorAll('.s-engines-page').forEach(function (p) { + p.classList.toggle('is-active', p.dataset.page === cat); + }); + document.querySelectorAll('[data-engines-cat]').forEach(function (el) { + el.classList.toggle('is-active', el.dataset.enginesCat === cat); + }); + } + + window._enginesSidebarCat = 'llm'; + + window.navEnginesCat = function (cat) { + window._enginesSidebarCat = cat; + navTo('s-llms'); // showSection will call applyEnginesPage(cat) + if ((cat === 'tts' || cat === 'stt') && typeof loadLocalContainers === 'function') loadLocalContainers(); + }; + // Show default section on load showSection('s-voices'); runSideEffects('library'); diff --git a/static/sections/s-connect.html b/static/sections/s-connect.html index d3a3b8d..d5e5f3a 100644 --- a/static/sections/s-connect.html +++ b/static/sections/s-connect.html @@ -18,47 +18,53 @@
-
+

SillyTavern

Use an OpenAI-compatible TTS provider. Paste one active voice into the voice field, or paste the comma-separated list where SillyTavern accepts custom voices.

-
+

Open WebUI

Enable TTS in Open WebUI under Settings → Audio. Set API base URL to this app's proxy and pick any active voice name. STT also works via the same proxy.

-
+

Home Assistant

Use this as a REST example for automations or scripts that call the TTS backend. Save the returned audio somewhere Home Assistant can play from.

-
+

Generic curl test

Quick terminal test for the voice list and speech endpoint after restarting the TTS container.

-
+

VoiceDesign virtual voices

Use saved Voice Design prompt presets without exporting WAVs. Point the external app at this creator app as an OpenAI-compatible TTS proxy and select a vd_... voice.

-
-

MCP — AI agents

-

Expose TTS and STT as tools for Claude Code, Cursor, or any MCP-compatible agent. Copy the Python MCP server below, install it with pip install mcp httpx, then point your agent config at it.

-
+
+

MCP — Native built-in server

+

The app ships a built-in MCP server at /mcp (JSON-RPC 2.0, Streamable HTTP). No external script or extra packages needed. Tools: speak, transcribe, list_captures, list_profiles.

+
- - + +
-
+
+

/speak — direct REST

+

POST text to /speak from any script, agent, or app. Voice resolves from explicit param → per-client binding → default voice. Optional persona LLM rewrite.

+
+ +
+

Streaming TTS

Use this when the target app can play audio progressively. For routed streaming, keep response format WAV and avoid before/after route sounds, otherwise the proxy must buffer before playback.

diff --git a/static/sections/s-llms.html b/static/sections/s-llms.html index bb695b9..442d966 100644 --- a/static/sections/s-llms.html +++ b/static/sections/s-llms.html @@ -1,41 +1,23 @@
- +
-

AI Backends

-

Connect local or cloud services for speech recognition, synthesis, and text generation.

+

Engines

+

Local and cloud services for language generation, speech recognition, and speech synthesis.

- -
- - -
+ +
- -
-

Local AI Services

-

Run everything on your own hardware — no API key, no rate limits, no data leaves your machine.

+

Language Models

+

Used for persona rewriting and transcription refinement. Click Use as LLM to apply a URL to this app’s LLM setting.

- 100% Local + Local first
- -
-
-

Active Docker Stack

-

Start, stop, or install TTS and ASR containers running on this machine.

-
- -
-
-
Checking container status…
-
- - -

Language Models (LLM)

+

Local

@@ -44,12 +26,21 @@ Ollama OpenAI-compat
+
+ ~40–80 tok/s GPU + <1 s TTFT + Any HF/GGUF model + GPU / CPU +

Easiest local LLM setup. Pull any model with one command. Runs Llama, Mistral, Qwen, Phi, Gemma and many more.

URL
+
+ +
Quick start @@ -67,12 +58,21 @@ ollama pull llama3.3 vLLM Detected in stack
+
+ ~200+ tok/s GPU + <0.5 s TTFT + PagedAttention + GPU required +

High-throughput GPU inference. Already in your Docker stack. Serve any HuggingFace model with PagedAttention.

URL
+
+ +
Add a model to your stack @@ -90,12 +90,21 @@ ollama pull llama3.3 LM Studio OpenAI-compat
+
+ ~20–60 tok/s + <2 s TTFT + GGUF format + GPU / CPU +

GUI app for running quantized models. Built-in model browser, chat UI, and a local server with OpenAI API.

URL
+
+ +
Enable in LM Studio
Developer tab → Start server → Port 1234
@@ -110,12 +119,21 @@ Check "Enable CORS" for browser access
llama.cpp OpenAI-compat
+
+ ~5–60 tok/s + <1 s TTFT + GGUF format + CPU native +

Lightweight GGUF model server. Minimal RAM usage. Runs on CPU or GPU. Great for embedding in Docker.

URL
+
+ +
Docker one-liner @@ -130,373 +148,7 @@ Check "Enable CORS" for browser access
- -

Speech Recognition (STT)

-
- -
-
- ️ - faster-whisper-server - OpenAI-compat -
-
- ~70× RT GPU - 0.5–2 s - large-v3 - 1.5 GB VRAM -
-

Drop-in local replacement for the Whisper API. GPU-accelerated via CTranslate2. OpenAI-compatible endpoint.

-
- URL - - -
-
- -
-
-
- docker-compose snippet - -
- 
services:
-  whisper:
-    image: fedirz/faster-whisper-server:latest-cuda
-    ports: ["8000:8000"]
-    environment:
-      - WHISPER__MODEL=large-v3
-    deploy:
-      resources:
-        reservations:
-          devices: [{driver: nvidia, count: 1, capabilities: [gpu]}]
-
- github.com/fedirz/faster-whisper-server -
- -
-
- 🔉 - whisper.cpp - HTTP server -
-
- ~8–15× RT CPU - 1–5 s - large-v3 Q5 - ~1 GB RAM -
-

Minimal C++ Whisper with a built-in HTTP server. CPU or Metal/CUDA. Low memory, fast on consumer hardware.

-
- URL - - -
-
- -
-
-
- Build & run - -
- 
git clone https://github.com/ggml-org/whisper.cpp
-cd whisper.cpp && cmake -B build && cmake --build build -j
-./build/bin/whisper-server \
-  -m models/ggml-large-v3.bin --port 8080
-
- github.com/ggml-org/whisper.cpp -
- -
- - -

Text-to-Speech (local TTS)

-
- -
-
- - Piper TTS - Fast · offline -
-
- ~1× CPU realtime - <50 ms - Good (VITS) - ~50 MB RAM -
-

Lightning-fast offline TTS. Runs on CPU in real time. 50+ language voices available. Uses Wyoming protocol (port 10200) — not directly OpenAI-compatible.

-
- URL - - -
-
-
- Docker - -
- 
docker run -p 10200:10200 \
-  rhasspy/wyoming-piper \
-  --voice en_US-lessac-medium
-
- github.com/rhasspy/piper -
- -
-
- 🌸 - Kokoro FastAPI - OpenAI-compat TTS -
-
- ~0.1× CPU RTF - ~200 ms - High (82M) - 300 MB CPU -
-

Kokoro-82M running behind an OpenAI-compatible TTS endpoint. 11 built-in voices (af_bella, bf_emma, am_adam…). Drop-in replacement for OpenAI’s TTS API.

-
- URL - - -
-
- -
-
-
- Docker - -
- 
docker run -p 8880:8880 \
-  ghcr.io/remsky/kokoro-fastapi-cpu:v0.2.2
-# GPU:
-docker run -p 8880:8880 --gpus all \
-  ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.2
-
- github.com/remsky/Kokoro-FastAPI -
- -
-
- 🗣️ - XTTS v2 - Voice cloning -
-
- ~0.5× GPU RTF - 1–3 s - High (WAV clone) - 3–4 GB VRAM -
-

Coqui XTTS — multilingual voice cloning from a 6-second sample. 17 languages. Compatible with this app’s voice library.

-
- URL - - -
-
-
- Docker - -
- 
docker run -p 8020:80 --gpus all \
-  -v /voices:/voices \
-  daswer123/xtts-api-server:latest
-
- xtts-api-server -
- -
- -
-
-
- Adding a local service to this app -

All OpenAI-compatible services work the same way: open App Routing in the sidebar, add a new backend entry with the local URL, leave the API key field empty, and set it as the default backend for the voice type you want.

-
-
- -
- - - +
+ + +
+ +
+
+

Speech Recognition

+

Your Docker stack containers appear at the top. Local runners and cloud APIs below.

+
+
+ + 100% Local +
+
+ +
+ +

Other local STT

+
+ +
+
+ ️ + faster-whisper-server + OpenAI-compat +
+
+ ~70× RT GPU + 0.5–2 s + large-v3 + 1.5 GB VRAM +
+

Drop-in local replacement for the Whisper API. GPU-accelerated via CTranslate2. OpenAI-compatible endpoint.

+
+ URL + + +
+
+ +
+
+
+ docker-compose snippet + +
+ 
services:
+  whisper:
+    image: fedirz/faster-whisper-server:latest-cuda
+    ports: ["8000:8000"]
+    environment:
+      - WHISPER__MODEL=large-v3
+    deploy:
+      resources:
+        reservations:
+          devices: [{driver: nvidia, count: 1, capabilities: [gpu]}]
+
+ github.com/fedirz/faster-whisper-server +
+ +
+
+ 🔉 + whisper.cpp + HTTP server +
+
+ ~8–15× RT CPU + 1–5 s + large-v3 Q5 + ~1 GB RAM +
+

Minimal C++ Whisper with a built-in HTTP server. CPU or Metal/CUDA. Low memory, fast on consumer hardware.

+
+ URL + + +
+
+ +
+
+
+ Build & run + +
+ 
git clone https://github.com/ggml-org/whisper.cpp
+cd whisper.cpp && cmake -B build && cmake --build build -j
+./build/bin/whisper-server \
+  -m models/ggml-large-v3.bin --port 8080
+
+ github.com/ggml-org/whisper.cpp +
+ +
+ +

Cloud APIs Free tiers

+
+ +
+
+ +
+
Groq Whisper
+
whisper-large-v3-turbo · OpenAI-compatible
+
+ Free +
+
+ 2 000 req / day + Fastest cloud STT + Cloud · 0 VRAM +
+
+ + + Get key +
+
+ Endpoint + https://api.groq.com/openai/v1 +
+
+ whisper-large-v3-turbo + whisper-large-v3 + distil-whisper-large-v3-en +
+
Save key in Settings → Groq API key to enable Groq Whisper in the STT dropdown.
+
+ +
+
+ +
+
HuggingFace Inference
+
Serverless Whisper models
+
+ Free +
+
+ ~1 000 req / day + Slower cold starts + Many model variants +
+
+ + + Get key +
+
+ Endpoint + https://api-inference.huggingface.co/models/openai/whisper-large-v3 +
+
+ +
+
+ +
+
AssemblyAI
+
High-accuracy transcription + speaker diarization
+
+ Free +
+
+ 100 h lifetime + Speaker labels + Auto-chapters +
+
+ + + Get key +
+
+ Endpoint + https://api.assemblyai.com/v2/transcript +
+
+ +
+ +
+ + +
+ +
+
+

Text to Speech

+

Your Docker stack containers appear at the top. Click Use as TTS to apply a URL to this app’s backend settings.

+
+
+ + 100% Local +
+
+ +
+ +

Other local TTS

+
+ +
+
+ + Piper TTS + Fast · offline +
+
+ ~1× CPU realtime + <50 ms + Good (VITS) + ~50 MB RAM +
+

Lightning-fast offline TTS. Runs on CPU in real time. 50+ language voices available. Uses Wyoming protocol (port 10200) — not directly OpenAI-compatible.

+
+ URL + + +
+
+
+ Docker + +
+ 
docker run -p 10200:10200 \
+  rhasspy/wyoming-piper \
+  --voice en_US-lessac-medium
+
+ github.com/rhasspy/piper +
+ +
+
+ 🌸 + Kokoro FastAPI + OpenAI-compat TTS +
+
+ ~0.1× CPU RTF + ~200 ms + High (82M) + 300 MB CPU +
+

Kokoro-82M running behind an OpenAI-compatible TTS endpoint. 11 built-in voices (af_bella, bf_emma, am_adam…). Drop-in replacement for OpenAI’s TTS API.

+
+ URL + + +
+
+ +
+
+
+ Docker + +
+ 
docker run -p 8880:8880 \
+  ghcr.io/remsky/kokoro-fastapi-cpu:v0.2.2
+# GPU:
+docker run -p 8880:8880 --gpus all \
+  ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.2
+
+ github.com/remsky/Kokoro-FastAPI +
+ +
+
+ 🗣️ + XTTS v2 + Voice cloning +
+
+ ~0.5× GPU RTF + 1–3 s + High (WAV clone) + 3–4 GB VRAM +
+

Coqui XTTS — multilingual voice cloning from a 6-second sample. 17 languages. Compatible with this app’s voice library.

+
+ URL + + +
+
+
+ Docker + +
+ 
docker run -p 8020:80 --gpus all \
+  -v /voices:/voices \
+  daswer123/xtts-api-server:latest
+
+ xtts-api-server +
+ +
+
+ + VibeVoice + Simple REST TTS +
+
+ Lightweight + Low latency + No auth + LAN service +
+

Lightweight local TTS service with a minimal REST API — POST text, get audio. No API key, no model field, no OpenAI wrapper needed. Set the URL in Settings and pick VibeVoice as the backend in Try It Out or a routing rule.

+
+ URL + + +
+
+ +
+
+
+ Direct REST test + +
+ 
curl -X POST http://192.168.178.8:8027/tts \
+  -H "Content-Type: application/json" \
+  -d '{"text":"Hello from VibeVoice"}' \
+  --output vibevoice-test.wav
+
+
+ +
+ +

Cloud APIs Free tiers

+
+ +
+
+ +
+
ElevenLabs
+
High-quality voice cloning & synthesis
+
+ Free +
+
+ 10 000 chars / month + 2 500 char / request max + Voice cloning supported +
+
+ + + Get key +
+
+ Endpoint + https://api.elevenlabs.io/v1/text-to-speech +
+
+ +
+
+ +
+
Fish Audio
+
Voice cloning & multilingual TTS
+
+ Free +
+
+ 1 h audio / month + 100 req / min + 30+ languages +
+
+ + + Get key +
+
+ Endpoint + https://api.fish.audio/v1/tts +
+
+ +
+
+ +
+
Kokoro TTS
+
82M model · HuggingFace Spaces demo
+
+ Demo +
+
+ Free web demo + <$1 per 1M chars (paid) + High naturalness +
+
+ Use the HF Spaces web demo for quick tests, or run Kokoro locally via Docker for production use. +
+ Open Kokoro HF Space +
+
+ +
+ +
+
+
+ Adding a local service to this app +

All OpenAI-compatible services work the same way: open App Routing in the sidebar, add a new backend entry with the local URL, leave the API key field empty, and set it as the default backend for the voice type you want.

+
+
+ +
diff --git a/static/sections/s-settings.html b/static/sections/s-settings.html index 1512e20..8b407bb 100644 --- a/static/sections/s-settings.html +++ b/static/sections/s-settings.html @@ -72,6 +72,11 @@ OpenAI-compatible TTS. Built-in voices: af_bella, bf_emma, am_adam… ~300 MB RAM.
+
+ + + Simple text-in audio-out TTS. POST /tts with {"text":"..."}. +
@@ -235,6 +240,20 @@
+
+
+ LLM server + Base URL for persona rewrite, text refinement, and other local LLM features +
+
+
+ + + OpenAI-compatible endpoint. Used for persona rewrites, voice design, and auto-refine. +
+
+
+
LLM refinement defaults @@ -358,6 +377,11 @@ Extra JSON fields for Kokoro. Usually empty — voice is selected from the dropdown.
+
+ + + Extra JSON fields for VibeVoice. Usually empty — service only requires text. +
@@ -492,6 +516,67 @@
+ +
+
+ MCP Server + Expose voice tools to Claude Code, Cursor, and other MCP-aware agents +
+
+
+ + http://localhost:7890/mcp + Streamable HTTP (JSON-RPC 2.0). Tools: speak, transcribe, list_captures, list_profiles. +
+
+ + claude mcp add voice-creator --transport http --url http://localhost:7890/mcp --header "X-Voice-Creator-Client-Id: claude-code" +
+
+ + 
{"mcpServers":{"voice-creator":{"url":"http://localhost:7890/mcp","headers":{"X-Voice-Creator-Client-Id":"my-agent"}}}}
+
+
+
+ +
+
+ /speak REST endpoint + Generate audio from any app or script without routing rules +
+
+
+ + 
curl -X POST http://localhost:7890/speak \
+  -H "Content-Type: application/json" \
+  -H "X-Voice-Creator-Client-Id: my-script" \
+  -d '{"text":"Hello world","voice":"EN_F_Anna"}' \
+  --output speech.wav
+
+
+ + 
curl -X PUT http://localhost:7890/speak/bindings/my-script \
+  -H "Content-Type: application/json" \
+  -d '{"voice":"EN_F_Anna"}'
+
+
+
+ +
+
+ Global hotkey daemon + Push-to-talk transcription that types into any focused window +
+
+
+ + 
pip install pynput sounddevice soundfile pyperclip requests
+python hotkey_daemon.py --server http://localhost:7890
+ Hold Ctrl+Shift+Space to record, release to transcribe and type. Linux: install xdotool for direct key injection. +
+
+
+
diff --git a/static/sections/s-tryout.html b/static/sections/s-tryout.html index d7c4e8b..2a84403 100644 --- a/static/sections/s-tryout.html +++ b/static/sections/s-tryout.html @@ -66,7 +66,12 @@ This is sent as instruct. Voice Clone/Base and Streaming are fastest; CustomVoice and Voice Design are style-aware.
-
+
+