## Engines

Local and cloud services for language generation, speech recognition, and speech synthesis.

### Language Models

Used for persona rewriting and transcription refinement. Click Use as LLM to apply a URL to this app’s LLM setting.

Local first

#### Local

Ollama OpenAI-compat
~40–80 tok/s GPU <1 s TTFT Any HF/GGUF model GPU / CPU

Easiest local LLM setup. Pull any model with one command. Runs Llama, Mistral, Qwen, Phi, Gemma and many more.

URL
Quick start

    curl https://ollama.ai/install.sh | sh
    ollama pull llama3.3

ollama.ai
vLLM Detected in stack
~200+ tok/s GPU <0.5 s TTFT PagedAttention GPU required

High-throughput GPU inference. Already in your Docker stack. Serve any HuggingFace model with PagedAttention.

URL
Add a model to your stack

    --model Qwen/Qwen3-8B \
    --served-model-name qwen3-8b

docs.vllm.ai
🎛️ LM Studio OpenAI-compat
~20–60 tok/s <2 s TTFT GGUF format GPU / CPU

GUI app for running quantized models. Built-in model browser, chat UI, and a local server with OpenAI API.

URL
Enable in LM Studio

    Developer tab → Start server → Port 1234
    Check "Enable CORS" for browser access

lmstudio.ai
🔷 llama.cpp OpenAI-compat
~5–60 tok/s <1 s TTFT GGUF format CPU native

Lightweight GGUF model server. Minimal RAM usage. Runs on CPU or GPU. Great for embedding in Docker.

URL
Docker one-liner

    docker run -p 8080:8080 ghcr.io/ggml-org/llama.cpp:server \
      -hf QuantFactory/Meta-Llama-3-8B-GGUF \
      -hff Meta-Llama-3-8B.Q4_K_M.gguf

github.com/ggml-org/llama.cpp

#### Cloud APIs Free tiers

Groq
Ultra-fast inference · OpenAI-compatible
Free
30 000 tokens / min 14 400 req / day Lowest latency
API key Get key
Endpoint `https://api.groq.com/openai/v1`
llama-3.3-70b-versatile qwen-qwq-32b deepseek-r1-distill-llama-70b
OpenRouter
50+ free models · OpenAI-compatible
Free
20 req / min 200 req / day (free models) Single API for all models
API key Get key
Endpoint `https://openrouter.ai/api/v1`
qwen/qwen3-235b-a22b:free deepseek/deepseek-r1-0528:free mistralai/mistral-7b-instruct:free
Google Gemini
Gemini 2.5 Flash · Generous free tier
Free
250 000 tokens / min 15 req / min free 1M context window
API key Get key
Endpoint `https://generativelanguage.googleapis.com/v1beta/openai/`
Uses OpenAI-compat wrapper — use model `gemini-2.5-flash`
Mistral AI
OpenAI-compatible · EU-based
Free
1B tokens / month 2 req / min free GDPR-compliant
API key Get key
Endpoint `https://api.mistral.ai/v1`
mistral-small-latest mistral-large-latest

### Speech Recognition

Your Docker stack containers appear at the top. Local runners and cloud APIs below.

100% Local

#### Other local STT

️ faster-whisper-server OpenAI-compat
~70× RT GPU 0.5–2 s large-v3 1.5 GB VRAM

Drop-in local replacement for the Whisper API. GPU-accelerated via CTranslate2. OpenAI-compatible endpoint.

URL
docker-compose snippet

    services:
      whisper:
        image: fedirz/faster-whisper-server:latest-cuda
        ports: ["8000:8000"]
        environment:
          - WHISPER__MODEL=large-v3
        deploy:
          resources:
            reservations:
              devices: [{driver: nvidia, count: 1, capabilities: [gpu]}]

github.com/fedirz/faster-whisper-server
🔉 whisper.cpp HTTP server
~8–15× RT CPU 1–5 s large-v3 Q5 ~1 GB RAM

Minimal C++ Whisper with a built-in HTTP server. CPU or Metal/CUDA. Low memory, fast on consumer hardware.

URL
Build & run

    git clone https://github.com/ggml-org/whisper.cpp
    cd whisper.cpp && cmake -B build && cmake --build build -j
    ./build/bin/whisper-server \
      -m models/ggml-large-v3.bin --port 8080

github.com/ggml-org/whisper.cpp

#### Cloud APIs Free tiers

Groq Whisper
whisper-large-v3-turbo · OpenAI-compatible
Free
2 000 req / day Fastest cloud STT Cloud · 0 VRAM
API key Get key
Endpoint `https://api.groq.com/openai/v1`
whisper-large-v3-turbo whisper-large-v3 distil-whisper-large-v3-en
Save key in Settings → Groq API key to enable Groq Whisper in the STT dropdown.
HuggingFace Inference
Serverless Whisper models
Free
~1 000 req / day Slower cold starts Many model variants
API key Get key
Endpoint `https://api-inference.huggingface.co/models/openai/whisper-large-v3`
AssemblyAI
High-accuracy transcription + speaker diarization
Free
100 h lifetime Speaker labels Auto-chapters
API key Get key
Endpoint `https://api.assemblyai.com/v2/transcript`

### Text to Speech

Your Docker stack containers appear at the top. Click Use as TTS to apply a URL to this app’s backend settings.

100% Local

#### Other local TTS

Piper TTS Fast · offline
~1× CPU realtime <50 ms Good (VITS) ~50 MB RAM

Lightning-fast offline TTS. Runs on CPU in real time. 50+ language voices available. Uses Wyoming protocol (port 10200) — not directly OpenAI-compatible.

URL
Docker

    docker run -p 10200:10200 \
      rhasspy/wyoming-piper \
      --voice en_US-lessac-medium

github.com/rhasspy/piper
🌸 Kokoro FastAPI OpenAI-compat TTS
~0.1× CPU RTF ~200 ms High (82M) 300 MB CPU

Kokoro-82M running behind an OpenAI-compatible TTS endpoint. 11 built-in voices (af_bella, bf_emma, am_adam…). Drop-in replacement for OpenAI’s TTS API.

URL
Docker

    docker run -p 8880:8880 \
      ghcr.io/remsky/kokoro-fastapi-cpu:v0.2.2
    # GPU:
    docker run -p 8880:8880 --gpus all \
      ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.2

github.com/remsky/Kokoro-FastAPI
🗣️ XTTS v2 Voice cloning
~0.5× GPU RTF 1–3 s High (WAV clone) 3–4 GB VRAM

Coqui XTTS — multilingual voice cloning from a 6-second sample. 17 languages. Compatible with this app’s voice library.

URL
Docker

    docker run -p 8020:80 --gpus all \
      -v /voices:/voices \
      daswer123/xtts-api-server:latest

xtts-api-server
VibeVoice Simple REST TTS
Lightweight Low latency No auth LAN service

Lightweight local TTS service with a minimal REST API — POST text, get audio. No API key, no model field, no OpenAI wrapper needed. Set the URL in Settings and pick VibeVoice as the backend in Try It Out or a routing rule.

URL
Direct REST test

    curl -X POST http://192.168.178.8:8027/tts \
      -H "Content-Type: application/json" \
      -d '{"text":"Hello from VibeVoice"}' \
      --output vibevoice-test.wav

github.com/Mekopa/VibeVoice

#### Cloud APIs Free tiers

ElevenLabs
High-quality voice cloning & synthesis
Free
10 000 chars / month 2 500 char / request max Voice cloning supported
API key Get key
Endpoint `https://api.elevenlabs.io/v1/text-to-speech`
Fish Audio
Voice cloning & multilingual TTS
Free
1 h audio / month 100 req / min 30+ languages
API key Get key
Endpoint `https://api.fish.audio/v1/tts`
Kokoro TTS
82M model · HuggingFace Spaces demo
Demo
Free web demo <$1 per 1M chars (paid) High naturalness
Use the HF Spaces web demo for quick tests, or run Kokoro locally via Docker for production use.
Open Kokoro HF Space
Adding a local service to this app

All OpenAI-compatible services work the same way: open App Routing in the sidebar, add a new backend entry with the local URL, leave the API key field empty, and set it as the default backend for the voice type you want.

### Add Custom Engine

Name
Role
URL
Docker container name (optional — enables Stop / Start / Restart)
Description (optional)