TTS Voice Creator - Clone and Design

A browser-based app for managing Qwen3 TTS voice clones and Voice Design prompts. It runs in Docker, opens in any browser, and gives you one place to curate the voice library, crop/reference audio, normalize loudness, write reusable voice design prompts, and copy integration snippets for apps such as SillyTavern, Open WebUI, and Home Assistant.

---

What it does

[ Drop file / YouTube URL / Record mic ]   or   [ Describe voice in text ]
                     ↓                                       ↓
      [ Interactive waveform trimmer ]          [ Qwen3-TTS VoiceDesign API ]
                     ↓                                       ↓
            [ 24 kHz · mono · 16-bit PCM WAV  +  reference.txt ]
                                   ↓
                     [ Named voice saved to disk ]
                                   ↓
                     [ TTS preview / Library management ]

Two workflows in one tool:

• Voice cloning — record or import a real voice, trim a clean segment, auto-transcribe, normalize loudness, and save it as a TTS reference voice.
• Voice design — describe a voice in plain language and let the Qwen3-TTS VoiceDesign model synthesize one from scratch. Designed voices can be exported to the clone library or used live as virtual voices through this app's /v1 proxy.

---

Requirements

Requirement	Notes
Docker Engine + Docker Compose v2	docs.docker.com/get-docker
A Docker network shared with your TTS/STT containers	docker network create tts_net
A Whisper-compatible transcription server	See Recommended backends
A TTS server	See Recommended backends
ARM64 (aarch64) or x86-64	Image is based on python:3.11-slim-bookworm — works on both
Internet access during build	pip packages; WaveSurfer.js is loaded from CDN at runtime

---

Installation & Setup

1 — Clone the repository

git clone https://github.com/mARTin-B78/TTS-Voice-Creator.git tts-voice-creator-clone-and-design
cd tts-voice-creator-clone-and-design

2 — Create a Docker network (if you don't have one already)

All containers that need to talk to each other (TTS Voice Creator - Clone and Design, your TTS server, your Whisper server) should share a network:

docker network create tts_net

If you already have a shared network, note its name — you will use it in the next step.

3 — Configure volume paths

Edit docker-compose.yml (or portainer-stack.yml for Portainer) and set the host path where your voices are stored:

volumes:
  - /your/voices/directory:/voices:rw

This directory is shared with the TTS server — voices saved here become immediately available after the TTS server is restarted.

Also update the network name if yours differs from the default:

networks:
  tts_net:
    external: true

4 — Build the image

docker build -t tts-voice-creator:latest .

This step is required before the first run, and whenever you pull updates.

5a — Run with Docker Compose

docker compose up -d

5b — Deploy via Portainer

Use portainer-stack.yml, not docker-compose.yml.
The Portainer file references the pre-built image and has no build: section, which avoids the "no such file: Dockerfile" error that occurs because Portainer stores stacks in its own internal directory.

1. Build the image on the host (step 4 above)
2. Portainer → Stacks → Add stack
3. Enter a stack name (e.g. tts-voice-creator)
4. Select Web editor
5. Paste the contents of portainer-stack.yml
6. Click Deploy the stack

6 — Open the UI

http://<host-ip>:7860

7 — Configure API endpoints

Click the ⚙ gear icon (top right) and fill in:

Field	Example	Notes
Whisper API URL	http://whisper:8000	Use container name if on the same network
Whisper API key	(optional)	Only needed for cloud APIs
TTS API URL	http://tts-server:8000
TTS API key	(optional)
TTS backend	Qwen3-TTS / OpenAI	Select to match your server
Voice Design URL	http://host.docker.internal:8021	Optional; required for prompt-based Voice Design. For Qwen-style dialogue/timbre reuse, point this at the 1.7B VoiceDesign backend.
Voice Design API key	(leave empty)	Falls back to TTS key if blank
Voice scan directory	/voices	Where the library tab looks for voices
Active voices directory	/voices/active_voices	Where newly saved and TTS-visible voices land

Settings are persisted to a named Docker volume and survive container restarts.

---

Recommended Backends

Speech-to-Text (transcription)

The app sends audio to a Whisper-compatible REST endpoint (POST /v1/audio/transcriptions).

Option	Type	Recommendation
faster-whisper-server	Local, free	Best local option. Runs large-v3 efficiently on GPU or CPU. OpenAI-compatible API.
Whisper.cpp server	Local, free	Lightweight, works on any hardware with no Python. Good for CPU-only setups.
OpenAI Whisper API	Cloud, paid	Highest accuracy, no local GPU needed. Set URL to https://api.openai.com and add your API key.
Groq Whisper API	Cloud, free tier	Very fast inference, generous free tier. OpenAI-compatible endpoint.

Recommended for local use: faster-whisper-server with large-v3 on a GPU. Example Docker run:

docker run -d --gpus all --network tts_net --name whisper \
  -p 8000:8000 \
  fedirz/faster-whisper-server:latest-cuda

Then set the Whisper URL in settings to http://whisper:8000.

---

Text-to-Speech

The app supports any server that exposes a compatible REST endpoint.

Option	Type	Recommendation
Qwen3-TTS	Local, free	State-of-the-art multilingual TTS with voice cloning and voice design. Primary recommendation.
faster-qwen3-tts	Local wrapper	Docker wrapper for Qwen3-TTS with an OpenAI-compatible API and automatic voice loading from the configured active voices directory.
LocalAI	Local, free	Supports many TTS backends (Piper, Bark, XTTS, etc.) behind a unified API. Select "LocalAI" in settings.
Pocket-TTS	Local, free	Lightweight XTTS-based server. Select "Pocket-TTS" in settings.
OpenAI TTS API	Cloud, paid	High quality, no local hardware needed. Set URL to https://api.openai.com and add your API key.
NVIDIA Magpie + Parakeet	Local, free	OpenAI-compatible NVIDIA speech stack. Use Magpie Multilingual on http://host.docker.internal:8091 for fixed-speaker TTS and Parakeet on http://host.docker.internal:8092 for STT, or route both through 8090. NVIDIA Magpie Zeroshot/Flow clone NIMs can be configured separately, usually on http://host.docker.internal:8093.
ElevenLabs	Cloud, freemium	Best-in-class voice quality and cloning. OpenAI-compatible endpoint available.

Recommended for local use: Qwen3-TTS is the best open-source multilingual TTS model as of 2025. It produces very natural speech, supports 16+ languages, and has built-in zero-shot voice cloning and voice design.

---

Voice Cloning vs Voice Design

This tool supports two fundamentally different ways to create a new TTS voice:

Option A — Voice Cloning

You provide a real recording of a voice. The TTS model uses it as a reference to reproduce that speaker's characteristics.

What makes a good reference recording:

• Duration: 5–20 seconds (10–15 s is ideal)
• Content: Natural, expressive speech — not just counting or reciting the alphabet
• Audio quality: No background music, reverb, noise, or multiple speakers
• Emotion: Should match the emotion you want the TTS to reproduce (calm for a narrator, energetic for an announcer, etc.)
• Format: The app converts anything to 24 kHz mono 16-bit PCM WAV automatically
• Transcript: The reference text (what was spoken) is critical — Qwen3-TTS uses it for alignment

Good sources for reference audio:

• Record yourself or a voice actor directly in the app (mic tab)
• Extract a clean segment from a podcast, audiobook, or interview using the YouTube/URL downloader
• Any audio or video file: WAV, MP3, OGG, FLAC, M4A, MP4, MKV, WEBM

Option B — Voice Design

Describe the voice you want in plain language. The Qwen3-TTS VoiceDesign model synthesises a new voice from your description — no recording required.

Requires: A server running the Qwen3-TTS-12Hz-1.7B-VoiceDesign model or a compatible alias (typically a separate model from the main TTS model). The container environment variable VOICE_DESIGN_MODEL controls the model name sent to the VoiceDesign backend.

How to write good voice descriptions:

Describe: age, gender, pitch, pace, accent, emotion, and speaking style. Be specific.

# Good descriptions:
"A middle-aged British woman with a calm, authoritative tone and clear diction.
 Slightly formal, measured speech rate."

"Young male voice, mid-20s, enthusiastic and upbeat. American accent, slightly
 fast pace, warm and friendly."

"Elderly male narrator with a deep, resonant voice. Slow and deliberate,
 with gravitas. No discernible accent."

# Bad (too vague):
"A nice voice"
"Female voice"

---

Application Tabs

1 Voice Clone

The first tab is the main voice library. It contains the old Source & Trim and Name & Save workflow inside an Add new voice panel, so you no longer need to move back and forth between separate tabs.

Feature	Details
Add new voice	Opens drag and drop, YouTube/audio URL, microphone, waveform trim, transcript, and save controls
Library editing	Rename by editing the name directly; the app renames the WAV and sidecar files
Language	Formerly shown as Flag; language and region are stored in metadata
Type / Length / dB	Shows audio type, duration, and stored loudness values
Pencil optimizer	Opens waveform crop, transcript, dB controls, save crop, save volume, and undo crop
Active toggle	Moves complete voice packages between active_voices and hidden_voices
Copy active voices	Copies active voice IDs as a comma-separated list

Cropping is reversible: the app keeps an original copy before replacing the active voice file.

2 Voice Design

Voice Design is for prompt-based voices. Samples are listed in a collapsible table. Preview generates audio from a sample, and Use fills the description, sample text, and language fields.

Feature	Details
Voice Design samples	Example prompts for acoustic attribute control, age control, gradual control, human-likeness, background information, and timbre reuse
Reference transcript	Shared transcript field used by samples and presets
Generate voice	Calls the VoiceDesign backend and plays the generated audio
Export to Voice Clone Library	Saves the generated WAV, transcript, name, language, and gender into the clone library
Prompt library	Stores reusable Voice Design presets

3 Routing

The Routing tab lets this app act as an OpenAI-compatible TTS router. External apps can send a simple voice name such as default to this app's /v1 proxy, and routing rules can map it to a real Qwen3-TTS voice by app and detected language.

Example:

App	Input voice	Language	Output voice
Open WebUI	default	EN	EN_F_Anna
Open WebUI	default	DE	DE_M_Max

4 Use In Apps

This tab shows ready-to-copy examples for SillyTavern, Open WebUI, Home Assistant, and OpenAI-compatible clients.

Exported clone voices should usually point external apps directly at the Qwen3 TTS server. Virtual VoiceDesign voices should point external apps at this app's /v1 proxy.

---

Command Line Usage

Everything needed for headless operation is available through ttsvc_cli.py. It talks to the running app over HTTP, so it works locally or against a remote creator instance.

./ttsvc_cli.py --help
./ttsvc_cli.py settings show
./ttsvc_cli.py settings set --tts-url http://host.docker.internal:8020 --tts-backend openai
./ttsvc_cli.py voices list --all
./ttsvc_cli.py voices enable EN_F_Anna
./ttsvc_cli.py routes openwebui-defaults --en EN_F_Anna --de DE_M_Max
./ttsvc_cli.py routes list
./ttsvc_cli.py tts voices
./ttsvc_cli.py tts speak --voice default --text "Hallo, das ist ein Test." --out test.wav
./ttsvc_cli.py create-voice --file sample.wav --voice-id EN_F_NewVoice --transcript "Exact words spoken."

Use --base-url http://HOST:7860 when the app is not running on localhost. Each command and subcommand has its own --help output.

---

Guided Installer

For non-command-line users, start the graphical installer:

python3 guided_installer.py

Or double-click one of these launchers:

• Linux: start-guided-installer.sh
• macOS: start-guided-installer.command
• Windows: start-guided-installer.bat

The installer checks Docker, Docker Compose, optional Portainer, this app, and the Qwen3-TTS clone/design/custom containers. It explains what each component is for, lets the user choose ports and model locations, generates a Docker Compose file, and can run the selected Docker actions with visible logs.

Additional publishing assets:

• Forum playbook
• Tutorial video script

---

API compatibility

TTS endpoints

Backend	Setting	Endpoint	Request body
Qwen3-TTS / OpenAI	openai	POST /v1/audio/speech	{"model":"tts-1","input":"...","voice":"EN_F_Anna","response_format":"wav"}
LocalAI	localai	POST /tts	{"input":"...","model":"EN_F_Anna","response_format":"wav"}
Pocket-TTS	pocket	POST /v1/audio/speech	{"input":"...","voice":"EN_F_Anna","response_format":"wav"}
NVIDIA Magpie Multilingual	nvidia_magpie	POST /v1/audio/speech	{"model":"tts-1","input":"...","voice":"sofia","response_format":"wav"}
NVIDIA Magpie Zeroshot	nvidia_zeroshot	POST /v1/audio/synthesize	Multipart form with language, text, and selected library WAV as audio_prompt
NVIDIA Magpie Flow	nvidia_flow	POST /v1/audio/synthesize	Multipart form with language, text, selected library WAV as audio_prompt, and saved reference transcript as audio_prompt_transcript

Voice Design endpoint

POST /v1/audio/speech
{
  "model":    "Qwen3-TTS-12Hz-1.7B-VoiceDesign",
  "input":    "sample text to speak",
  "instruct": "describe the voice here",
  "language": "Auto",
  "response_format": "wav"
}

For Qwen-style multi-speaker VoiceDesign, send the complete dialogue in input and the speaker timbre map in instruct, for example:

{
  "model": "Qwen3-TTS-12Hz-1.7B-VoiceDesign",
  "input": "Lucas:Hi there.\nMia:Hello back.",
  "instruct": "\"Lucas\": \"Male, 17 years old, tenor range\"\n\"Mia\": \"Female, 16 years old, mezzo-soprano range\"",
  "language": "English",
  "response_format": "wav"
}

Whisper endpoint

POST /v1/audio/transcriptions
Content-Type: multipart/form-data

file=<audio.wav>
model=large-v3
response_format=text

For NVIDIA Parakeet, set the Whisper/STT URL to http://host.docker.internal:8092 for the direct ASR container, or http://host.docker.internal:8090 for the speech router. For NVIDIA voice cloning, set the NVIDIA Zeroshot/Flow NIM URL to the deployed Speech NIM endpoint, commonly http://host.docker.internal:8093; the app sends saved library WAVs as audio_prompt, and Flow also sends the saved .reference.txt transcript.

All endpoints accept an optional Authorization: Bearer <key> header, configurable in Settings.

---

Timbre Control Support

There are two different modes:

• faster-qwen3-tts uses saved voice/reference audio IDs. It supports timbre reuse through cloned or exported voices, but it does not normally use prompt fields such as gender, age, pitch, texture, or emotion.
• faster-qwen3-tts-voicedesign is the prompt-based path. It can use acoustic descriptions and timbre-style prompts, including multi-speaker/timbre reuse style descriptions, but those voices are generated live from the prompt unless you export the result.

So: timbre descriptions belong in the Voice Design container. Stable reusable voice IDs belong in the normal TTS container after export.

---

Use In Apps

Exported Clone Voices

For SillyTavern, Open WebUI, Home Assistant, or any OpenAI-compatible client:

Base URL: http://YOUR_TTS_HOST:8020/v1
Model:    qwen
Voice:    one of the active voice IDs

After adding or changing voices, restart the TTS container so it rereads the active voice folder.

Virtual VoiceDesign Voices Without Export

This app exposes an OpenAI-compatible proxy:

Base URL: http://YOUR_APP_HOST:7860/v1
Model:    qwen3-tts-voice-creator
Voice:    vd_<PresetName>

Virtual voices call the Voice Design backend live for each generation. This is useful when you want to keep editing prompts without exporting WAV files yet. It is slower than using exported clone voices and requires the VoiceDesign backend URL to be configured.

Example request:

curl http://localhost:7860/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-tts-voice-creator",
    "voice": "vd_WarmNarrator",
    "input": "This line is generated through a saved Voice Design preset."
  }' \
  --output speech.wav

---

Voice file format

Qwen3-TTS (and most modern neural TTS systems) expect reference audio in a specific format:

Property	Value
Format	WAV (PCM, uncompressed)
Sample rate	24 000 Hz
Channels	Mono
Bit depth	16-bit
Duration	5–20 seconds
Companion file	<voice_id>.reference.txt — verbatim transcript of the spoken audio

The app converts all source audio to this specification automatically using pydub + ffmpeg.

Voice metadata sidecar files

Each saved voice can have optional sidecar files stored alongside the WAV:

File	Contents
<id>.reference.txt	Transcript of the reference audio (required for voice cloning)
<id>.meta.json	Note, star rating, flag, gender, enabled state
<id>.jpg / .png / .webp	Profile picture shown in the library

---

Naming convention

Voice IDs have no forced format. The suggested convention is:

{LANG}_{GENDER}_{Name}.wav

Examples: EN_F_Anna.wav, DE_M_Christoph.wav, ZH_F_Mei.wav

Language codes: EN DE ZH FR ES JA KO IT PT RU AR PL NL SV TR HI
Gender codes: F (female) · M (male) · N (neutral / non-binary)

Custom IDs like narrator_calm or my-voice-v2 are equally valid — the naming helper in the UI is optional.

---

Volume mounts

Host path	Container path	Purpose
${VOICE_HOST_DIR:-./voices}	/voices	Shared voice collection (read-write)
Named volume tts_voice_creator_config	/home/app/.config/tts-voice-creator	Persisted settings and Voice Design presets

The TTS server should mount and scan the active_voices subfolder. The hidden_voices subfolder is for voices kept in the library but removed from Qwen3-TTS model discovery.

---

Reloading voices into your TTS server

Most local TTS servers load voices on startup. After saving a new voice, restart the TTS container:

docker restart <your-tts-container-name>

The default public Compose file does not mount the Docker socket. Restart the TTS container from Docker, Portainer, or your host scripts.

---

Public Repository Safety

The repository ignores private runtime data:

• .env files and local settings.
• .claude/, editor workspaces, logs, and caches.
• Voice folders and audio/video files.
• Per-voice transcript and metadata sidecars.

Do not commit real API keys, private voice samples, speaker folders, or host-specific paths.

---

Architecture

Browser (any device on your local network)
  └── port 7860 ──► FastAPI + uvicorn  (server.py)
                         ├── pydub + ffmpeg        audio conversion & trimming
                         ├── yt-dlp                YouTube / URL audio extraction
                         ├── Whisper API  ──────►  transcription
                         ├── TTS API      ──────►  preview & voice list
                         └── VoiceDesign API ───►  AI voice generation

Frontend (single HTML file, no build step)
  ├── WaveSurfer.js 7   interactive waveform + region trimmer
  └── MediaRecorder API browser microphone recording

---

Troubleshooting

WaveSurfer / UI doesn't load — the container needs internet access to fetch WaveSurfer.js from unpkg.com. If your network is air-gapped, download the two JS files and serve them from static/.

"Cannot connect to Whisper/TTS API" — check that:

1. The API server container is running
2. Both containers are on the same Docker network
3. You're using the container name (not localhost) as the URL inside Docker — e.g. http://whisper:8000, not http://localhost:8000

Microphone not working — browsers block getUserMedia on non-localhost origins without HTTPS. Set up a reverse proxy with a TLS certificate (e.g. Nginx + Let's Encrypt or Caddy) if accessing from another device.

Generated voice sounds wrong — for cloning, check the reference audio quality: no background noise, single speaker, natural speech, and a correct transcript. For voice design, make the description more specific — add age, pitch, pace, and accent details.

Voice not appearing in TTS server after save — restart the TTS container. Most servers only scan the voices directory at startup.

Transcription produces garbled text — try a larger Whisper model (large-v3 recommended), or manually type the transcript before saving.