Local and cloud services for language generation, speech recognition, and speech synthesis.
Used for persona rewriting and transcription refinement. Click Use as LLM to apply a URL to this app’s LLM setting.
Easiest local LLM setup. Pull any model with one command. Runs Llama, Mistral, Qwen, Phi, Gemma and many more.
curl https://ollama.ai/install.sh | sh ollama pull llama3.3
High-throughput GPU inference. Already in your Docker stack. Serve any HuggingFace model with PagedAttention.
--model Qwen/Qwen3-8B \ --served-model-name qwen3-8b
GUI app for running quantized models. Built-in model browser, chat UI, and a local server with OpenAI API.
Developer tab → Start server → Port 1234 Check "Enable CORS" for browser access
Lightweight GGUF model server. Minimal RAM usage. Runs on CPU or GPU. Great for embedding in Docker.
docker run -p 8080:8080 ghcr.io/ggml-org/llama.cpp:server \ -hf QuantFactory/Meta-Llama-3-8B-GGUF \ -hff Meta-Llama-3-8B.Q4_K_M.gguf
https://api.groq.com/openai/v1
https://openrouter.ai/api/v1
https://generativelanguage.googleapis.com/v1beta/openai/
gemini-2.5-flashhttps://api.mistral.ai/v1
Your Docker stack containers appear at the top. Local runners and cloud APIs below.
Drop-in local replacement for the Whisper API. GPU-accelerated via CTranslate2. OpenAI-compatible endpoint.
services:
whisper:
image: fedirz/faster-whisper-server:latest-cuda
ports: ["8000:8000"]
environment:
- WHISPER__MODEL=large-v3
deploy:
resources:
reservations:
devices: [{driver: nvidia, count: 1, capabilities: [gpu]}]
Minimal C++ Whisper with a built-in HTTP server. CPU or Metal/CUDA. Low memory, fast on consumer hardware.
git clone https://github.com/ggml-org/whisper.cpp cd whisper.cpp && cmake -B build && cmake --build build -j ./build/bin/whisper-server \ -m models/ggml-large-v3.bin --port 8080
https://api.groq.com/openai/v1
https://api-inference.huggingface.co/models/openai/whisper-large-v3
https://api.assemblyai.com/v2/transcript
Your Docker stack containers appear at the top. Click Use as TTS to apply a URL to this app’s backend settings.
Lightning-fast offline TTS. Runs on CPU in real time. 50+ language voices available. Uses Wyoming protocol (port 10200) — not directly OpenAI-compatible.
docker run -p 10200:10200 \ rhasspy/wyoming-piper \ --voice en_US-lessac-medium
Kokoro-82M running behind an OpenAI-compatible TTS endpoint. 11 built-in voices (af_bella, bf_emma, am_adam…). Drop-in replacement for OpenAI’s TTS API.
docker run -p 8880:8880 \ ghcr.io/remsky/kokoro-fastapi-cpu:v0.2.2 # GPU: docker run -p 8880:8880 --gpus all \ ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.2
Coqui XTTS — multilingual voice cloning from a 6-second sample. 17 languages. Compatible with this app’s voice library.
docker run -p 8020:80 --gpus all \ -v /voices:/voices \ daswer123/xtts-api-server:latest
Lightweight local TTS service with a minimal REST API — POST text, get audio. No API key, no model field, no OpenAI wrapper needed. Set the URL in Settings and pick VibeVoice as the backend in Try It Out or a routing rule.
curl -X POST http://192.168.178.8:8027/tts \
-H "Content-Type: application/json" \
-d '{"text":"Hello from VibeVoice"}' \
--output vibevoice-test.wav
https://api.elevenlabs.io/v1/text-to-speech
https://api.fish.audio/v1/tts
All OpenAI-compatible services work the same way: open App Routing in the sidebar, add a new backend entry with the local URL, leave the API key field empty, and set it as the default backend for the voice type you want.