Local and cloud services for language generation, speech recognition, and speech synthesis.
Pick the Active Language Model below — it is used by default for every LLM task (persona rewriting, transcription refinement, conversation, and rehearser character analysis). The cards below let you connect engines and apply a URL with Use as LLM.
Default endpoint & model for all LLM tasks. Per-task dropdowns can still override it.
Easiest local LLM setup. Pull any model with one command. Runs Llama, Mistral, Qwen, Phi, Gemma and many more.
curl https://ollama.ai/install.sh | sh ollama pull llama3.3
High-throughput GPU inference. Already in your Docker stack. Serve any HuggingFace model with PagedAttention.
--model Qwen/Qwen3-8B \ --served-model-name qwen3-8b
GUI app for running quantized models. Built-in model browser, chat UI, and a local server with OpenAI API.
Developer tab → Start server → Port 1234 Check "Enable CORS" for browser access
Lightweight GGUF model server. Minimal RAM usage. Runs on CPU or GPU. Great for embedding in Docker.
docker run -p 8080:8080 ghcr.io/ggml-org/llama.cpp:server \ -hf QuantFactory/Meta-Llama-3-8B-GGUF \ -hff Meta-Llama-3-8B.Q4_K_M.gguf
Proxy that loads/unloads llama.cpp (and other) models on demand and exposes them all on one OpenAI endpoint. Pick any served model below.
docker run -p 28080:8080 \ -v ./config.yaml:/app/config.yaml \ ghcr.io/mostlygeek/llama-swap:cuda
Gateway that exposes local and cloud models behind one OpenAI-compatible endpoint, with routing, fallbacks, and budgets. Pick any configured model below.
docker run -p 14000:4000 \ -v ./config.yaml:/app/config.yaml \ ghcr.io/berriai/litellm:main-latest \ --config /app/config.yaml
https://api.groq.com/openai/v1
https://openrouter.ai/api/v1
https://generativelanguage.googleapis.com/v1beta/openai/
gemini-2.5-flashhttps://api.mistral.ai/v1
Your Docker stack containers appear at the top. Local runners and cloud APIs below.
Drop-in local replacement for the Whisper API. GPU-accelerated via CTranslate2. OpenAI-compatible endpoint.
services:
whisper:
image: fedirz/faster-whisper-server:latest-cuda
ports: ["8000:8000"]
environment:
- WHISPER__MODEL=large-v3
deploy:
resources:
reservations:
devices: [{driver: nvidia, count: 1, capabilities: [gpu]}]
Minimal C++ Whisper with a built-in HTTP server. CPU or Metal/CUDA. Low memory, fast on consumer hardware.
git clone https://github.com/ggml-org/whisper.cpp cd whisper.cpp && cmake -B build && cmake --build build -j ./build/bin/whisper-server \ -m models/ggml-large-v3.bin --port 8085
https://api.groq.com/openai/v1
https://api-inference.huggingface.co/models/openai/whisper-large-v3
https://api.assemblyai.com/v2/transcript
Docker-detected engines and local presets are listed together here. Click Use as TTS to apply a URL to this app’s backend settings.
Clones a saved WAV voice and honours inline emotion markers like (angry), (whispering), (excited) per line — consistent speaker identity with real tone control. The only backend that is both WAV-anchored and style-aware.
cd fishaudio/s1 docker compose up -d # S2-Pro API on :38080
Lightning-fast offline TTS. Runs on CPU in real time. 50+ language voices available. Uses Wyoming protocol (port 10200) — not directly OpenAI-compatible.
docker run -p 10200:10200 \ rhasspy/wyoming-piper \ --voice en_US-lessac-medium
Kokoro-82M running behind an OpenAI-compatible TTS endpoint. 11 built-in voices (af_bella, bf_emma, am_adam…). Drop-in replacement for OpenAI’s TTS API.
docker run -p 8880:8880 \ ghcr.io/remsky/kokoro-fastapi-cpu:v0.2.2 # GPU: docker run -p 8880:8880 --gpus all \ ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.2
Coqui XTTS — multilingual voice cloning from a 6-second sample. 17 languages. Compatible with this app’s voice library.
docker run -p 8020:80 --gpus all \ -v /voices:/voices \ daswer123/xtts-api-server:latest
Lightweight local TTS service with a minimal REST API — POST text, get audio. No API key, no model field, no OpenAI wrapper needed. Set the URL in Settings and pick VibeVoice as the backend in Try It Out or a routing rule.
curl -X POST http://192.168.178.8:8027/tts \
-H "Content-Type: application/json" \
-d '{"text":"Hello from VibeVoice"}' \
--output vibevoice-test.wav
https://api.elevenlabs.io/v1/text-to-speech
https://api.fish.audio/v1/tts
All OpenAI-compatible services work the same way: open App Routing in the sidebar, add a new backend entry with the local URL, leave the API key field empty, and set it as the default backend for the voice type you want.