TTS Voice Creator — Clone & Design

A browser-based voice management studio running in Docker. Clone real voices, design new ones from text descriptions, manage your voice library, and connect any app or AI agent via REST or MCP.

---

User Manual

For a step-by-step app tutorial, see docs/USER_MANUAL.md.

---

What it does

[ Drop file / YouTube URL / Record mic ]   or   [ Describe voice in text ]
                     ↓                                       ↓
      [ Interactive waveform trimmer ]          [ Qwen3-TTS VoiceDesign API ]
                     ↓                                       ↓
            [ 24 kHz · mono · 16-bit PCM WAV  +  reference.txt ]
                                   ↓
                     [ Named voice saved to disk ]
                         ↓                  ↓
              [ TTS preview / Library ]   [ /speak · /mcp · /v1 proxy ]

Core workflows:

• Voice cloning — record or import audio, trim a clean segment, auto-transcribe, normalize loudness, and save as a TTS reference voice.
• Voice design — describe a voice in plain language; Qwen3-TTS VoiceDesign synthesises one from scratch. Export to the clone library or use live as a virtual voice.
• Character persona — attach a personality description to any voice; a local LLM rewrites incoming text in that character's style before synthesis.
• Speak API — POST /speak for any script, app, or agent, with per-client voice bindings and optional persona rewrite.
• Native MCP server — built-in at POST /mcp; exposes speak, transcribe, list_captures, and list_profiles as MCP tools for Claude Code, Cursor, or any MCP-aware agent.
• Global hotkey daemon — run hotkey_daemon.py on the host for push-to-talk dictation that types transcribed text into any focused window.

---

Requirements

Requirement	Notes
Docker Engine + Docker Compose v2	docs.docker.com/get-docker
A Docker network shared with your TTS/STT containers	docker network create dgx_net
A Whisper-compatible transcription server	See Recommended backends
A TTS server	See Recommended backends
ARM64 (aarch64) or x86-64	Image based on python:3.11-slim-bookworm
Internet access during build	pip packages; WaveSurfer.js loaded from CDN at runtime

---

Installation & Setup

1 — Clone the repository

git clone https://github.com/mARTin-B78/tts-voice-creator-clone-and-design-2.git
cd tts-voice-creator-clone-and-design-2

2 — Create or reuse a Docker network

docker network create dgx_net   # skip if you already have one

3 — Configure docker-compose.yml

Set the host path for voices and the network name:

volumes:
  - /your/voices/directory:/voices:rw
networks:
  dgx_net:
    external: true

4 — Build and start

docker compose up -d --build

5 — Open the UI

http://<host-ip>:7890

6 — Configure API endpoints

Settings → Connections — fill in at minimum:

Field	Example
Voice Clone / Base URL	http://host.docker.internal:8020
Voice Design URL	http://host.docker.internal:8021
Whisper URL	http://host.docker.internal:8010
LLM base URL	http://localhost:11434/v1 (for persona rewrite)

Settings persist to a named Docker volume across restarts.

---

Recommended Backends

Speech-to-Text

Option	Notes
faster-whisper-server	Best local option. GPU-accelerated, OpenAI-compatible.
Whisper.cpp server	CPU-only, no Python.
Groq Whisper API	Fast cloud, generous free tier.
NVIDIA Parakeet	http://host.docker.internal:8092 — 200× realtime on GPU.

Text-to-Speech

Option	Backend ID	Notes
faster-qwen3-tts	voice_clone	Recommended. Docker wrapper for Qwen3-TTS with auto voice loading.
Qwen3-TTS Voice Design	voice_design	Prompt-based voice synthesis. Needs Qwen3-TTS-12Hz-1.7B-VoiceDesign.
CustomVoice	customvoice	Style-controlled synthesis with pre-configured speakers.
Streaming	streaming	Progressive low-latency WAV playback.
Kokoro FastAPI	kokoro	OpenAI-compatible. 82M model, ~300 MB RAM, CPU-friendly.
VibeVoice	vibevoice	Simple POST /tts service. Default: http://192.168.178.8:8027.
NVIDIA Magpie TTS	nvidia_magpie	Fixed-speaker GPU TTS. http://host.docker.internal:8091.
NVIDIA Magpie Zeroshot	nvidia_zeroshot	WAV-prompted zero-shot clone NIM.
NVIDIA Magpie Flow	nvidia_flow	WAV + transcript high-fidelity clone NIM.
ElevenLabs	—	Cloud; browse the shared library in the Get Voices Online tab.

---

Application Sections

Voices — Clone library

Manage WAV reference voices. Crop and normalize audio, auto-transcribe, set metadata (language, gender, rating, note), and attach a character persona for LLM-based text rewriting.

Feature	Details
Add voice	Drag-and-drop, URL/YouTube import, microphone recording
Waveform editor	Crop to clean segment, normalize loudness to target dBFS
Auto-transcribe	Sends audio to the configured STT backend
Character persona	Plain-language personality prompt; LLM rewrites text before synthesis
Active / hidden	Toggle moves voice between active_voices and hidden_voices
Sync / Calculate dB / Benchmark	Bulk tools for library health

Clone a Voice

Step-by-step wizard: import audio → trim → transcribe → save to library.

Design a Voice

Describe a voice in natural language; Qwen3-TTS VoiceDesign generates it from scratch. Export to the clone library or use as a live vd_... virtual voice.

Includes curated sample prompts: acoustic attribute control, age control, timbre reuse, multi-speaker dialogue.

Studio — Get Voices Online

Browse public voice clip sources, scrape and preview direct audio files, import from URLs, and browse the ElevenLabs shared voice library.

Try It Out — TTS Playground

Generate speech from any backend and voice. Features:

• Backend and voice selector with live availability check
• Style instruction field (emotion, pacing, tone)
• Apply character persona toggle — LLM rewrites input text through the voice's personality before synthesis
• Chunked generation for long texts (split at sentence boundaries)
• Audio effects: reverb, compressor, chorus, pitch shift
• Generation history (last 20 clips, in-memory)
• Playlist with drag-to-reorder and merged WAV export

Also includes an STT → TTS workspace: record or upload speech → transcribe → LLM-refine → re-synthesize with any voice.

Performance — Benchmark

Run latency and quality benchmarks across backends. Batch mode tests every active voice against a configurable sentence set. Results (RTSL, duration, per-voice) are shown in a sortable table and saved to voice metadata.

App Routing

Route incoming OpenAI-compatible TTS requests by app name, voice, and detected language. Rules map generic voice names (default, assistant) to specific library voices. Before/after sounds and per-route backend selection are supported.

Connect Your Apps

Ready-to-copy integration snippets for SillyTavern, Open WebUI, Home Assistant, generic curl, virtual VoiceDesign voices, streaming TTS, and the native MCP server.

Settings

Organized into sub-pages: Connections (all backend URLs including VibeVoice, NVIDIA stack), STT, Generation (extra TTS params per backend), LLM (URL for persona rewrite and text refinement), Voice (scan directory, normalization targets), Output, Captures, General, About (MCP connection commands, /speak examples, hotkey daemon setup).

---

Character Persona

Each voice can have a personality description stored in its .meta.json sidecar:

{ "persona": "Gruff Scottish engineer, dry wit, technical vocabulary, measured pace." }

When persona rewrite is active, a local LLM at the configured LLM URL rephrases the input text in that character's voice before TTS synthesis. This runs through the same OpenAI-compatible endpoint used for text refinement.

Activate persona rewrite:

• In the playground: check Apply character persona before generating.
• Via API: pass "apply_persona": true in the request body to /api/tts-preview or /speak.
• Via MCP: pass "apply_persona": true to the speak tool.

Edit a voice's persona in the voice inspector (Voices tab → select voice → Character persona section).

---

Speak API

POST /speak — generate audio from any script or app without needing routing rules.

# Bind a client ID to a voice (persisted)
curl -X PUT http://localhost:7890/speak/bindings/my-script \
  -H "Content-Type: application/json" \
  -d '{"voice": "EN_F_Anna"}'

# Generate speech — voice from binding, optional persona rewrite
curl -X POST http://localhost:7890/speak \
  -H "Content-Type: application/json" \
  -H "X-Voice-Creator-Client-Id: my-script" \
  -d '{"text": "Hello world", "apply_persona": true}' \
  --output speech.wav

# Or specify voice directly
curl -X POST http://localhost:7890/speak \
  -H "Content-Type: application/json" \
  -d '{"text": "Hello world", "voice": "EN_F_Anna", "backend": "voice_clone"}' \
  --output speech.wav

Voice resolution order: explicit voice param → X-Voice-Creator-Client-Id binding → captures_default_voice in Settings.

Binding endpoints:

Method	Path	Description
GET	/speak/bindings	List all per-client voice bindings
PUT	/speak/bindings/{client_id}	Set or update a binding
DELETE	/speak/bindings/{client_id}	Remove a binding

---

Native MCP Server

The app ships a built-in MCP server at /mcp (JSON-RPC 2.0, Streamable HTTP transport). No external Python script or extra packages required.

Tools:

Tool	Arguments	Returns
speak	text, voice?, apply_persona?	Base64 WAV audio + confirmation text
transcribe	audio_base64	Transcribed text
list_captures	—	20 most recent output WAV files
list_profiles	—	All voice profiles with id, lang, persona, enabled

Connect Claude Code:

claude mcp add voice-creator \
  --transport http \
  --url http://localhost:7890/mcp \
  --header "X-Voice-Creator-Client-Id: claude-code"

Any HTTP MCP client (Cursor, Windsurf, VS Code, etc.):

{
  "mcpServers": {
    "voice-creator": {
      "url": "http://localhost:7890/mcp",
      "headers": { "X-Voice-Creator-Client-Id": "my-agent" }
    }
  }
}

The X-Voice-Creator-Client-Id header binds this client to a voice via PUT /speak/bindings/{id}. Per-client bindings persist in Settings.

---

Global Hotkey Daemon

hotkey_daemon.py is a standalone host-side script (runs outside Docker) for push-to-talk dictation. Hold Ctrl+Shift+Space to record, release to transcribe and type the result into any focused window.

# Install dependencies (host Python, not Docker)
pip install pynput sounddevice soundfile pyperclip requests

# Linux: install xdotool for direct key injection (falls back to clipboard paste)
sudo apt install xdotool

# Run
python hotkey_daemon.py --server http://localhost:7890

Option	Default	Notes
--server	http://localhost:7890	Voice Creator URL
--backend	configured	STT backend to use

Platform notes:

• Linux X11: uses xdotool type for direct injection; clipboard fallback on Wayland.
• macOS: clipboard paste via Cmd+V; grant Accessibility + Microphone in System Preferences.
• Windows: clipboard paste via Ctrl+V.

---

OpenAI-Compatible Proxy

This app exposes a /v1 proxy that any OpenAI-compatible client can use:

Base URL: http://YOUR_HOST:7890/v1
API key:  dummy   (any non-empty value)
Model:    tts-1
Voice:    EN_F_Anna   (or any active voice ID, or vd_<PresetName> for virtual voices)

Routing rules (App Routing tab) run on every request through this proxy.

---

API Reference

TTS endpoints

Endpoint	Description
POST /v1/audio/speech	OpenAI-compatible TTS with routing
POST /speak	Direct TTS with per-client voice binding
POST /api/tts-preview	Preview any backend/voice combination
POST /api/tts-preview with apply_persona:true	Preview with LLM persona rewrite
GET /v1/audio/voices	List active voice IDs
GET /v1/models	List TTS models

STT endpoints

Endpoint	Description
POST /v1/audio/transcriptions	OpenAI-compatible transcription
POST /api/transcribe-bytes	Multipart upload → transcription (used by hotkey daemon)

MCP endpoint

Endpoint	Description
POST /mcp	JSON-RPC 2.0 handler (initialize, tools/list, tools/call)
GET /mcp	SSE keep-alive stream

Voice library

Endpoint	Description
GET /api/voices	List all voices with metadata
POST /api/save	Save a voice from a temp file
POST /api/update-voice-meta	Update note, rating, flag, gender, persona
POST /api/voices/calculate-db	Normalize dBFS for all active voices
POST /api/voices/benchmark	Run latency/quality benchmarks
GET /api/tts-backends	List all backends with availability and capabilities

/speak bindings

Endpoint	Description
GET /speak/bindings	List all per-client voice bindings
PUT /speak/bindings/{id}	Set a per-client voice binding
DELETE /speak/bindings/{id}	Remove a binding

---

Voice File Format

Property	Value
Format	WAV (PCM, uncompressed)
Sample rate	24 000 Hz
Channels	Mono
Bit depth	16-bit
Duration	5–20 seconds
Companion file	<id>.reference.txt — verbatim transcript

The app converts all source audio automatically using pydub + ffmpeg.

Sidecar files

File	Contents
<id>.reference.txt	Reference transcript (required for cloning)
<id>.meta.json	Note, rating, flag, gender, enabled, persona, loudness, benchmark
<id>.jpg / .png / .webp	Profile picture shown in the library

---

Naming Convention

{LANG}_{GENDER}_{Name}.wav

Examples: EN_F_Anna.wav, DE_M_Christoph.wav, ZH_F_Mei.wav

Language codes: EN DE ZH FR ES JA KO IT PT RU AR PL NL SV TR HI
Gender codes: F (female) · M (male) · N (neutral)

Custom IDs like narrator_calm are equally valid.

---

Architecture

Browser (any device on your local network)
  └── port 7890 ──► FastAPI + uvicorn  (server.py)
                         ├── /v1          OpenAI-compatible TTS/STT proxy + routing
                         ├── /speak       Direct TTS with per-client bindings
                         ├── /mcp         Native MCP server (JSON-RPC 2.0)
                         ├── /api/...     Voice library, settings, benchmarks
                         ├── pydub+ffmpeg audio conversion & trimming
                         ├── yt-dlp       YouTube / URL audio extraction
                         ├── Whisper API  transcription (multiple backends)
                         ├── TTS backends voice_clone · voice_design · customvoice
                         │               streaming · kokoro · vibevoice
                         │               nvidia_magpie · nvidia_zeroshot · nvidia_flow
                         └── LLM API      persona rewrite, text refinement

Host machine (optional)
  └── hotkey_daemon.py   global Ctrl+Shift+Space → record → transcribe → type

Frontend (static HTML + Vanilla JS, no build step)
  ├── WaveSurfer.js 7    interactive waveform + region trimmer
  ├── MediaRecorder API  browser microphone recording
  └── MDI v7.4.47        icons

---

CLI

ttsvc_cli.py talks to the running app over HTTP:

./ttsvc_cli.py settings show
./ttsvc_cli.py settings set --tts-url http://host.docker.internal:8020
./ttsvc_cli.py voices list --all
./ttsvc_cli.py voices enable EN_F_Anna
./ttsvc_cli.py routes openwebui-defaults --en EN_F_Anna --de DE_M_Max
./ttsvc_cli.py tts speak --voice EN_F_Anna --text "Hello world" --out test.wav
./ttsvc_cli.py create-voice --file sample.wav --voice-id EN_F_Anna --transcript "Exact words."

Use --base-url http://HOST:7890 for remote instances.

---

Guided Installer

python3 guided_installer.py
# or: start-guided-installer.sh  /  .command  /  .bat

Checks Docker, configures containers, generates Compose files, and runs deployment with visible logs.

---

Volume Mounts

Host path	Container path	Purpose
${VOICE_HOST_DIR:-./voices}	/voices	Voice collection (read-write)
Named volume tts-voice-creator-clone-and-design-2	/home/app/.config/tts-voice-creator	Settings and presets
./static	/app/static	Frontend (live-reload without rebuild)
./server.py	/app/server.py	Backend (live-reload without rebuild)

---

Troubleshooting

"Cannot connect to TTS/STT API" — ensure both containers share the same Docker network; use container hostnames, not localhost.

Microphone not working — browsers block getUserMedia on non-localhost origins without HTTPS. Use a reverse proxy with TLS for remote access.

Persona rewrite not working — check the LLM URL in Settings → LLM server. The endpoint must be OpenAI-compatible (e.g. Ollama at http://localhost:11434/v1).

MCP tools not appearing — run claude mcp list and confirm voice-creator is shown. Restart Claude Code after adding.

Hotkey daemon not typing — on Linux, install xdotool for X11; on Wayland it falls back to clipboard paste (Ctrl+V). On macOS, grant Accessibility permission.

Voice not appearing after save — restart the TTS container (docker restart <name>). Most engines only scan the voice directory on startup.

Transcription garbled — use large-v3 Whisper model; manually correct the transcript before saving.