tts-voice-creator-clone-and.../static/sections/s-llms.html

878 lines
53 KiB
HTML
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<div class="section-head">
<span class="section-icon"><span class="mdi mdi-brain"></span></span>
<div class="section-title">
<h2>Engines</h2>
<p>Local and cloud services for language generation, speech recognition, and speech synthesis.</p>
</div>
</div>
<!-- ── Language Models sub-page ──────────────────────────── -->
<div class="s-engines-page" data-page="llm">
<div class="llm-sec-header">
<div>
<h3 class="llm-sec-title">Language Models</h3>
<p class="llm-sec-note">Pick the <strong>Active Language Model</strong> below — it is used by default for every LLM task (persona rewriting, transcription refinement, conversation, and rehearser character analysis). The cards below let you connect engines and apply a URL with <strong>Use as LLM</strong>.</p>
</div>
<div style="display:flex;gap:8px;align-items:center">
<button class="btn-secondary" type="button" onclick="openAddEngineDialog('llm')"><span class="mdi mdi-plus"></span> Add Custom</button>
<span class="llm-free-badge llm-free-local">Local first</span>
</div>
</div>
<!-- ── Active Language Model (global picker) ──────────────── -->
<div class="llm-active-panel card">
<div class="llm-active-head">
<span class="llm-active-icon"><span class="mdi mdi-star-four-points"></span></span>
<div>
<h4 class="llm-active-title">Active Language Model</h4>
<p class="llm-active-sub">Default endpoint &amp; model for all LLM tasks. Per-task dropdowns can still override it.</p>
</div>
</div>
<div class="llm-active-controls">
<label class="llm-active-lbl"><span class="mdi mdi-server-network"></span> Endpoint</label>
<input id="llm-active-url" type="text" class="llm-active-url-inp" list="llm-active-url-suggestions"
placeholder="http://host:port/v1" spellcheck="false" autocomplete="off">
<datalist id="llm-active-url-suggestions">
<option value="http://host.docker.internal:11434/v1">Ollama — :11434</option>
<option value="http://host.docker.internal:8000/v1">vLLM — :8000</option>
<option value="http://host.docker.internal:1234/v1">LM Studio — :1234</option>
<option value="http://host.docker.internal:8080/v1">llama.cpp — :8080</option>
<option value="http://host.docker.internal:28080/v1">llama-swap — :28080</option>
<option value="http://host.docker.internal:14000/v1">LiteLLM — :14000</option>
</datalist>
<label class="llm-active-lbl"><span class="mdi mdi-brain"></span> Model</label>
<select id="llm-active-model" aria-label="Active language model"><option value="">— fetch models —</option></select>
<button id="llm-active-refresh" class="btn-secondary" type="button" title="Fetch models"><span class="mdi mdi-refresh"></span></button>
<span id="llm-active-status" class="llm-active-status"></span>
</div>
</div>
<div id="dc-grid-llm" class="dc-grid" style="margin-bottom:14px"></div>
<h4 class="llm-local-cat">Local</h4>
<div class="llm-local-grid">
<div class="llm-local-card">
<div class="llm-local-head">
<span class="llm-local-icon"><span class="mdi mdi-head-dots-horizontal-outline"></span></span>
<span class="llm-local-name">Ollama</span>
<span class="llm-local-compat">OpenAI-compat</span>
</div>
<div class="llm-local-metrics">
<span class="llm-metric-chip"><span class="mdi mdi-lightning-bolt"></span> ~4080 tok/s GPU</span>
<span class="llm-metric-chip"><span class="mdi mdi-clock-outline"></span> &lt;1 s TTFT</span>
<span class="llm-metric-chip"><span class="mdi mdi-star-circle-outline"></span> Any HF/GGUF model</span>
<span class="llm-metric-chip"><span class="mdi mdi-memory"></span> GPU / CPU</span>
</div>
<p class="llm-local-desc">Easiest local LLM setup. Pull any model with one command. Runs Llama, Mistral, Qwen, Phi, Gemma and many more.</p>
<div class="llm-local-url">
<span class="llm-local-url-label">URL</span>
<input class="llm-local-url-inp" type="text" placeholder="http://localhost:11434/v1" data-llm-local-key="ollama" data-llm-local-default="http://localhost:11434/v1" spellcheck="false">
<button class="llm-local-ping" data-ping-key="ollama" title="Test connection">Connect</button>
</div>
<div class="llm-local-actions">
<button class="llm-use-btn" id="llm-use-ollama-llm" type="button"><span class="mdi mdi-content-copy"></span> Use as LLM</button>
</div>
<div class="llm-local-snippet">
<div class="llm-snippet-bar">
<span>Quick start</span>
<button class="llm-copy-btn" data-copy="curl https://ollama.ai/install.sh | sh&#10;ollama pull llama3.3">Copy</button>
</div>
<pre>curl https://ollama.ai/install.sh | sh
ollama pull llama3.3</pre>
</div>
<a class="llm-local-link" href="https://ollama.ai" target="_blank" rel="noopener">ollama.ai <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-local-card llm-local-card-running">
<div class="llm-local-head">
<span class="llm-local-icon"><span class="mdi mdi-lightning-bolt"></span></span>
<span class="llm-local-name">vLLM</span>
<span class="llm-local-compat llm-compat-detected">Detected in stack</span>
</div>
<div class="llm-local-metrics">
<span class="llm-metric-chip"><span class="mdi mdi-lightning-bolt"></span> ~200+ tok/s GPU</span>
<span class="llm-metric-chip"><span class="mdi mdi-clock-outline"></span> &lt;0.5 s TTFT</span>
<span class="llm-metric-chip"><span class="mdi mdi-star-circle-outline"></span> PagedAttention</span>
<span class="llm-metric-chip"><span class="mdi mdi-memory"></span> GPU required</span>
</div>
<p class="llm-local-desc">High-throughput GPU inference. Already in your Docker stack. Serve any HuggingFace model with PagedAttention.</p>
<div class="llm-local-url">
<span class="llm-local-url-label">URL</span>
<input class="llm-local-url-inp" type="text" placeholder="http://localhost:8000/v1" data-llm-local-key="vllm" data-llm-local-default="http://localhost:8000/v1" spellcheck="false">
<button class="llm-local-ping" data-ping-key="vllm" title="Test connection">Connect</button>
</div>
<div class="llm-local-actions">
<button class="llm-use-btn" id="llm-use-vllm-llm" type="button"><span class="mdi mdi-content-copy"></span> Use as LLM</button>
</div>
<div class="llm-local-snippet">
<div class="llm-snippet-bar">
<span>Add a model to your stack</span>
<button class="llm-copy-btn" data-copy="--model Qwen/Qwen3-8B --served-model-name qwen3-8b">Copy</button>
</div>
<pre>--model Qwen/Qwen3-8B \
--served-model-name qwen3-8b</pre>
</div>
<a class="llm-local-link" href="https://docs.vllm.ai" target="_blank" rel="noopener">docs.vllm.ai <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-local-card">
<div class="llm-local-head">
<span class="llm-local-icon">🎛️</span>
<span class="llm-local-name">LM Studio</span>
<span class="llm-local-compat">OpenAI-compat</span>
</div>
<div class="llm-local-metrics">
<span class="llm-metric-chip"><span class="mdi mdi-lightning-bolt"></span> ~2060 tok/s</span>
<span class="llm-metric-chip"><span class="mdi mdi-clock-outline"></span> &lt;2 s TTFT</span>
<span class="llm-metric-chip"><span class="mdi mdi-star-circle-outline"></span> GGUF format</span>
<span class="llm-metric-chip"><span class="mdi mdi-memory"></span> GPU / CPU</span>
</div>
<p class="llm-local-desc">GUI app for running quantized models. Built-in model browser, chat UI, and a local server with OpenAI API.</p>
<div class="llm-local-url">
<span class="llm-local-url-label">URL</span>
<input class="llm-local-url-inp" type="text" placeholder="http://localhost:1234/v1" data-llm-local-key="lmstudio" data-llm-local-default="http://localhost:1234/v1" spellcheck="false">
<button class="llm-local-ping" data-ping-key="lmstudio" title="Test connection">Connect</button>
</div>
<div class="llm-local-actions">
<button class="llm-use-btn" id="llm-use-lmstudio-llm" type="button"><span class="mdi mdi-content-copy"></span> Use as LLM</button>
</div>
<div class="llm-local-snippet">
<div class="llm-snippet-bar"><span>Enable in LM Studio</span></div>
<pre>Developer tab → Start server → Port 1234
Check "Enable CORS" for browser access</pre>
</div>
<a class="llm-local-link" href="https://lmstudio.ai" target="_blank" rel="noopener">lmstudio.ai <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-local-card">
<div class="llm-local-head">
<span class="llm-local-icon">🔷</span>
<span class="llm-local-name">llama.cpp</span>
<span class="llm-local-compat">OpenAI-compat</span>
</div>
<div class="llm-local-metrics">
<span class="llm-metric-chip"><span class="mdi mdi-lightning-bolt"></span> ~560 tok/s</span>
<span class="llm-metric-chip"><span class="mdi mdi-clock-outline"></span> &lt;1 s TTFT</span>
<span class="llm-metric-chip"><span class="mdi mdi-star-circle-outline"></span> GGUF format</span>
<span class="llm-metric-chip"><span class="mdi mdi-memory"></span> CPU native</span>
</div>
<p class="llm-local-desc">Lightweight GGUF model server. Minimal RAM usage. Runs on CPU or GPU. Great for embedding in Docker.</p>
<div class="llm-local-url">
<span class="llm-local-url-label">URL</span>
<input class="llm-local-url-inp" type="text" placeholder="http://localhost:8080/v1" data-llm-local-key="llamacpp" data-llm-local-default="http://localhost:8080/v1" spellcheck="false">
<button class="llm-local-ping" data-ping-key="llamacpp" title="Test connection">Connect</button>
</div>
<div class="llm-local-actions">
<button class="llm-use-btn" id="llm-use-llamacpp-llm" type="button"><span class="mdi mdi-content-copy"></span> Use as LLM</button>
</div>
<div class="llm-local-snippet">
<div class="llm-snippet-bar">
<span>Docker one-liner</span>
<button class="llm-copy-btn" data-copy="docker run -p 8080:8080 ghcr.io/ggml-org/llama.cpp:server -hf QuantFactory/Meta-Llama-3-8B-GGUF -hff Meta-Llama-3-8B.Q4_K_M.gguf">Copy</button>
</div>
<pre>docker run -p 8080:8080 ghcr.io/ggml-org/llama.cpp:server \
-hf QuantFactory/Meta-Llama-3-8B-GGUF \
-hff Meta-Llama-3-8B.Q4_K_M.gguf</pre>
</div>
<a class="llm-local-link" href="https://github.com/ggml-org/llama.cpp" target="_blank" rel="noopener">github.com/ggml-org/llama.cpp <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-local-card">
<div class="llm-local-head">
<span class="llm-local-icon"><span class="mdi mdi-swap-horizontal"></span></span>
<span class="llm-local-name">llama-swap</span>
<span class="llm-local-compat">OpenAI-compat</span>
</div>
<div class="llm-local-metrics">
<span class="llm-metric-chip"><span class="mdi mdi-lightning-bolt"></span> Hot model swap</span>
<span class="llm-metric-chip"><span class="mdi mdi-clock-outline"></span> On-demand load</span>
<span class="llm-metric-chip"><span class="mdi mdi-star-circle-outline"></span> Many models, 1 port</span>
<span class="llm-metric-chip"><span class="mdi mdi-memory"></span> GPU / CPU</span>
</div>
<p class="llm-local-desc">Proxy that loads/unloads llama.cpp (and other) models on demand and exposes them all on one OpenAI endpoint. Pick any served model below.</p>
<div class="llm-local-url">
<span class="llm-local-url-label">URL</span>
<input class="llm-local-url-inp" type="text" placeholder="http://host.docker.internal:28080/v1" data-llm-local-key="llama-swap" data-llm-local-default="http://host.docker.internal:28080/v1" spellcheck="false">
<button class="llm-local-ping" data-ping-key="llama-swap" title="Test connection">Connect</button>
</div>
<div class="llm-local-actions">
<button class="llm-use-btn" id="llm-use-llama-swap-llm" type="button"><span class="mdi mdi-content-copy"></span> Use as LLM</button>
</div>
<div class="llm-local-snippet">
<div class="llm-snippet-bar">
<span>Docker one-liner</span>
<button class="llm-copy-btn" data-copy="docker run -p 28080:8080 -v ./config.yaml:/app/config.yaml ghcr.io/mostlygeek/llama-swap:cuda">Copy</button>
</div>
<pre>docker run -p 28080:8080 \
-v ./config.yaml:/app/config.yaml \
ghcr.io/mostlygeek/llama-swap:cuda</pre>
</div>
<a class="llm-local-link" href="https://github.com/mostlygeek/llama-swap" target="_blank" rel="noopener">github.com/mostlygeek/llama-swap <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-local-card">
<div class="llm-local-head">
<span class="llm-local-icon"><span class="mdi mdi-call-split"></span></span>
<span class="llm-local-name">LiteLLM</span>
<span class="llm-local-compat">OpenAI-compat</span>
</div>
<div class="llm-local-metrics">
<span class="llm-metric-chip"><span class="mdi mdi-lightning-bolt"></span> 100+ providers</span>
<span class="llm-metric-chip"><span class="mdi mdi-clock-outline"></span> Unified API</span>
<span class="llm-metric-chip"><span class="mdi mdi-star-circle-outline"></span> Routing &amp; fallback</span>
<span class="llm-metric-chip"><span class="mdi mdi-memory"></span> Proxy / gateway</span>
</div>
<p class="llm-local-desc">Gateway that exposes local and cloud models behind one OpenAI-compatible endpoint, with routing, fallbacks, and budgets. Pick any configured model below.</p>
<div class="llm-local-url">
<span class="llm-local-url-label">URL</span>
<input class="llm-local-url-inp" type="text" placeholder="http://host.docker.internal:14000/v1" data-llm-local-key="litellm" data-llm-local-default="http://host.docker.internal:14000/v1" spellcheck="false">
<button class="llm-local-ping" data-ping-key="litellm" title="Test connection">Connect</button>
</div>
<div class="llm-local-actions">
<button class="llm-use-btn" id="llm-use-litellm-llm" type="button"><span class="mdi mdi-content-copy"></span> Use as LLM</button>
</div>
<div class="llm-local-snippet">
<div class="llm-snippet-bar">
<span>Docker one-liner</span>
<button class="llm-copy-btn" data-copy="docker run -p 14000:4000 -v ./config.yaml:/app/config.yaml ghcr.io/berriai/litellm:main-latest --config /app/config.yaml">Copy</button>
</div>
<pre>docker run -p 14000:4000 \
-v ./config.yaml:/app/config.yaml \
ghcr.io/berriai/litellm:main-latest \
--config /app/config.yaml</pre>
</div>
<a class="llm-local-link" href="https://docs.litellm.ai" target="_blank" rel="noopener">docs.litellm.ai <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
</div>
<h4 class="llm-local-cat" style="margin-top:28px;margin-bottom:12px"><span class="mdi mdi-earth"></span> Cloud APIs <span class="llm-free-badge" style="margin-left:8px;vertical-align:middle">Free tiers</span></h4>
<div class="llm-service-grid">
<div class="llm-card llm-card-highlight">
<div class="llm-card-head">
<span class="llm-card-icon"><span class="mdi mdi-lightning-bolt"></span></span>
<div>
<div class="llm-card-name">Groq</div>
<div class="llm-card-sub">Ultra-fast inference &middot; OpenAI-compatible</div>
</div>
<span class="llm-tier-badge llm-tier-free">Free</span>
</div>
<div class="llm-card-stats">
<span>30 000 tokens / min</span>
<span>14 400 req / day</span>
<span>Lowest latency</span>
</div>
<div class="llm-field-row">
<label class="llm-label">API key</label>
<input type="password" class="llm-input" placeholder="gsk_&hellip;" data-llm-key="groq_llm">
<a class="llm-link" href="https://console.groq.com" target="_blank" rel="noopener">Get key <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-endpoint">
<span class="llm-endpoint-label">Endpoint</span>
<code>https://api.groq.com/openai/v1</code>
</div>
<div class="llm-models">
<span class="llm-model-tag">llama-3.3-70b-versatile</span>
<span class="llm-model-tag">qwen-qwq-32b</span>
<span class="llm-model-tag">deepseek-r1-distill-llama-70b</span>
</div>
</div>
<div class="llm-card">
<div class="llm-card-head">
<span class="llm-card-icon"><span class="mdi mdi-earth"></span></span>
<div>
<div class="llm-card-name">OpenRouter</div>
<div class="llm-card-sub">50+ free models &middot; OpenAI-compatible</div>
</div>
<span class="llm-tier-badge llm-tier-free">Free</span>
</div>
<div class="llm-card-stats">
<span>20 req / min</span>
<span>200 req / day (free models)</span>
<span>Single API for all models</span>
</div>
<div class="llm-field-row">
<label class="llm-label">API key</label>
<input type="password" class="llm-input" placeholder="sk-or-&hellip;" data-llm-key="openrouter">
<a class="llm-link" href="https://openrouter.ai" target="_blank" rel="noopener">Get key <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-endpoint">
<span class="llm-endpoint-label">Endpoint</span>
<code>https://openrouter.ai/api/v1</code>
</div>
<div class="llm-models">
<span class="llm-model-tag">qwen/qwen3-235b-a22b:free</span>
<span class="llm-model-tag">deepseek/deepseek-r1-0528:free</span>
<span class="llm-model-tag">mistralai/mistral-7b-instruct:free</span>
</div>
</div>
<div class="llm-card">
<div class="llm-card-head">
<span class="llm-card-icon"><span class="mdi mdi-diamond-outline"></span></span>
<div>
<div class="llm-card-name">Google Gemini</div>
<div class="llm-card-sub">Gemini 2.5 Flash &middot; Generous free tier</div>
</div>
<span class="llm-tier-badge llm-tier-free">Free</span>
</div>
<div class="llm-card-stats">
<span>250 000 tokens / min</span>
<span>15 req / min free</span>
<span>1M context window</span>
</div>
<div class="llm-field-row">
<label class="llm-label">API key</label>
<input type="password" class="llm-input" placeholder="AIza&hellip;" data-llm-key="gemini">
<a class="llm-link" href="https://ai.google.dev" target="_blank" rel="noopener">Get key <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-endpoint">
<span class="llm-endpoint-label">Endpoint</span>
<code>https://generativelanguage.googleapis.com/v1beta/openai/</code>
</div>
<div class="llm-info-note">Uses OpenAI-compat wrapper &mdash; use model <code>gemini-2.5-flash</code></div>
</div>
<div class="llm-card">
<div class="llm-card-head">
<span class="llm-card-icon"><span class="mdi mdi-alpha-m-circle-outline"></span></span>
<div>
<div class="llm-card-name">Mistral AI</div>
<div class="llm-card-sub">OpenAI-compatible &middot; EU-based</div>
</div>
<span class="llm-tier-badge llm-tier-free">Free</span>
</div>
<div class="llm-card-stats">
<span>1B tokens / month</span>
<span>2 req / min free</span>
<span>GDPR-compliant</span>
</div>
<div class="llm-field-row">
<label class="llm-label">API key</label>
<input type="password" class="llm-input" placeholder="Mistral key&hellip;" data-llm-key="mistral">
<a class="llm-link" href="https://mistral.ai" target="_blank" rel="noopener">Get key <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-endpoint">
<span class="llm-endpoint-label">Endpoint</span>
<code>https://api.mistral.ai/v1</code>
</div>
<div class="llm-models">
<span class="llm-model-tag">mistral-small-latest</span>
<span class="llm-model-tag">mistral-large-latest</span>
</div>
</div>
</div>
</div><!-- /llm -->
<!-- ── Speech to Text sub-page ───────────────────────────── -->
<div class="s-engines-page" data-page="stt">
<div class="llm-sec-header">
<div>
<h3 class="llm-sec-title">Speech Recognition</h3>
<p class="llm-sec-note">Your Docker stack containers appear at the top. Local runners and cloud APIs below.</p>
</div>
<div style="display:flex;gap:8px;align-items:center">
<button class="btn-secondary" type="button" onclick="openAddEngineDialog('stt')"><span class="mdi mdi-plus"></span> Add Custom</button>
<button class="btn-secondary dc-refresh-btn" type="button"><span class="mdi mdi-refresh"></span> Refresh</button>
<span class="llm-free-badge llm-free-local">100% Local</span>
</div>
</div>
<div id="dc-grid-stt" class="dc-grid" style="margin-bottom:14px"></div>
<!-- Inline STT test panel -->
<div class="stt-test-panel" id="stt-test-panel">
<div class="stt-test-row">
<span class="stt-test-label">Quick test</span>
<select id="stt-test-backend" class="stt-test-select"></select>
<button id="stt-test-mic-btn" class="stt-test-mic" title="Hold to record, click again to stop">
<span class="mdi mdi-microphone"></span>
</button>
<span id="stt-test-status" class="stt-test-status">Ready</span>
</div>
<div id="stt-test-result" class="stt-test-result" style="display:none"></div>
</div>
<h4 class="llm-local-cat">Other local STT</h4>
<div class="llm-local-grid">
<div class="llm-local-card">
<div class="llm-local-head">
<span class="llm-local-icon"><span class="mdi mdi-microphone-variant"></span></span>
<span class="llm-local-name">faster-whisper-server</span>
<span class="llm-local-compat">OpenAI-compat</span>
</div>
<div class="llm-local-metrics">
<span class="llm-metric-chip"><span class="mdi mdi-lightning-bolt"></span> ~70&times; RT GPU</span>
<span class="llm-metric-chip"><span class="mdi mdi-clock-outline"></span> 0.52 s</span>
<span class="llm-metric-chip"><span class="mdi mdi-star-circle-outline"></span> large-v3</span>
<span class="llm-metric-chip"><span class="mdi mdi-memory"></span> 1.5 GB VRAM</span>
</div>
<p class="llm-local-desc">Drop-in local replacement for the Whisper API. GPU-accelerated via CTranslate2. OpenAI-compatible endpoint.</p>
<div class="llm-local-url">
<span class="llm-local-url-label">URL</span>
<input class="llm-local-url-inp" type="text" placeholder="http://localhost:8000" data-llm-local-key="faster-whisper" data-llm-local-default="http://localhost:8000" spellcheck="false">
<button class="llm-local-ping" data-ping-key="faster-whisper" title="Test connection">Connect</button>
</div>
<div class="llm-local-actions">
<button class="llm-use-btn" id="llm-use-faster-whisper-stt" type="button" title="Copy URL to Settings → faster-whisper-server URL and set as active STT"><span class="mdi mdi-content-copy"></span> Use as STT</button>
</div>
<div class="llm-local-snippet">
<div class="llm-snippet-bar">
<span>docker-compose snippet</span>
<button class="llm-copy-btn" data-copy="services:&#10; whisper:&#10; image: fedirz/faster-whisper-server:latest-cuda&#10; ports: [&quot;8000:8000&quot;]&#10; environment:&#10; - WHISPER__MODEL=large-v3&#10; deploy:&#10; resources:&#10; reservations:&#10; devices: [{driver: nvidia, count: 1, capabilities: [gpu]}]">Copy</button>
</div>
<pre>services:
whisper:
image: fedirz/faster-whisper-server:latest-cuda
ports: ["8000:8000"]
environment:
- WHISPER__MODEL=large-v3
deploy:
resources:
reservations:
devices: [{driver: nvidia, count: 1, capabilities: [gpu]}]</pre>
</div>
<a class="llm-local-link" href="https://github.com/fedirz/faster-whisper-server" target="_blank" rel="noopener">github.com/fedirz/faster-whisper-server <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-local-card">
<div class="llm-local-head">
<span class="llm-local-icon">🔉</span>
<span class="llm-local-name">whisper.cpp</span>
<span class="llm-local-compat">HTTP server</span>
</div>
<div class="llm-local-metrics">
<span class="llm-metric-chip"><span class="mdi mdi-lightning-bolt"></span> ~815&times; RT CPU</span>
<span class="llm-metric-chip"><span class="mdi mdi-clock-outline"></span> 15 s</span>
<span class="llm-metric-chip"><span class="mdi mdi-star-circle-outline"></span> large-v3 Q5</span>
<span class="llm-metric-chip"><span class="mdi mdi-memory"></span> ~1 GB RAM</span>
</div>
<p class="llm-local-desc">Minimal C++ Whisper with a built-in HTTP server. CPU or Metal/CUDA. Low memory, fast on consumer hardware.</p>
<div class="llm-local-url">
<span class="llm-local-url-label">URL</span>
<input class="llm-local-url-inp" type="text" placeholder="http://localhost:8085" data-llm-local-key="whisper-cpp" data-llm-local-default="http://localhost:8085" spellcheck="false">
<button class="llm-local-ping" data-ping-key="whisper-cpp" title="Test connection">Connect</button>
</div>
<div class="llm-local-actions">
<button class="llm-use-btn" id="llm-use-whisper-cpp-stt" type="button" title="Copy URL to Settings → whisper.cpp URL and set as active STT"><span class="mdi mdi-content-copy"></span> Use as STT</button>
</div>
<div class="llm-local-snippet">
<div class="llm-snippet-bar">
<span>Build &amp; run</span>
<button class="llm-copy-btn" data-copy="git clone https://github.com/ggml-org/whisper.cpp&#10;cd whisper.cpp && cmake -B build && cmake --build build -j&#10;./build/bin/whisper-server -m models/ggml-large-v3.bin --port 8085">Copy</button>
</div>
<pre>git clone https://github.com/ggml-org/whisper.cpp
cd whisper.cpp && cmake -B build && cmake --build build -j
./build/bin/whisper-server \
-m models/ggml-large-v3.bin --port 8085</pre>
</div>
<a class="llm-local-link" href="https://github.com/ggml-org/whisper.cpp" target="_blank" rel="noopener">github.com/ggml-org/whisper.cpp <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
</div>
<h4 class="llm-local-cat" style="margin-top:28px;margin-bottom:12px"><span class="mdi mdi-earth"></span> Cloud APIs <span class="llm-free-badge" style="margin-left:8px;vertical-align:middle">Free tiers</span></h4>
<div class="llm-service-grid">
<div class="llm-card llm-card-highlight">
<div class="llm-card-head">
<span class="llm-card-icon"><span class="mdi mdi-lightning-bolt"></span></span>
<div>
<div class="llm-card-name">Groq Whisper</div>
<div class="llm-card-sub">whisper-large-v3-turbo &middot; OpenAI-compatible</div>
</div>
<span class="llm-tier-badge llm-tier-free">Free</span>
</div>
<div class="llm-card-stats">
<span>2 000 req / day</span>
<span><span class="mdi mdi-lightning-bolt"></span> Fastest cloud STT</span>
<span><span class="mdi mdi-memory"></span> Cloud &middot; 0 VRAM</span>
</div>
<div class="llm-field-row">
<label class="llm-label">API key</label>
<input type="password" class="llm-input" placeholder="gsk_&hellip;" data-llm-key="groq_stt">
<a class="llm-link" href="https://console.groq.com" target="_blank" rel="noopener">Get key <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-endpoint">
<span class="llm-endpoint-label">Endpoint</span>
<code>https://api.groq.com/openai/v1</code>
</div>
<div class="llm-models">
<span class="llm-model-tag">whisper-large-v3-turbo</span>
<span class="llm-model-tag">whisper-large-v3</span>
<span class="llm-model-tag">distil-whisper-large-v3-en</span>
</div>
<div class="llm-info-note">Save key in <strong>Settings &rarr; Groq API key</strong> to enable <em>Groq Whisper</em> in the STT dropdown.</div>
</div>
<div class="llm-card">
<div class="llm-card-head">
<span class="llm-card-icon"><span class="mdi mdi-emoticon-happy-outline"></span></span>
<div>
<div class="llm-card-name">HuggingFace Inference</div>
<div class="llm-card-sub">Serverless Whisper models</div>
</div>
<span class="llm-tier-badge llm-tier-free">Free</span>
</div>
<div class="llm-card-stats">
<span>~1 000 req / day</span>
<span>Slower cold starts</span>
<span>Many model variants</span>
</div>
<div class="llm-field-row">
<label class="llm-label">API key</label>
<input type="password" class="llm-input" placeholder="hf_&hellip;" data-llm-key="hf_stt">
<a class="llm-link" href="https://huggingface.co/settings/tokens" target="_blank" rel="noopener">Get key <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-endpoint">
<span class="llm-endpoint-label">Endpoint</span>
<code>https://api-inference.huggingface.co/models/openai/whisper-large-v3</code>
</div>
</div>
<div class="llm-card">
<div class="llm-card-head">
<span class="llm-card-icon"><span class="mdi mdi-content-copy"></span></span>
<div>
<div class="llm-card-name">AssemblyAI</div>
<div class="llm-card-sub">High-accuracy transcription + speaker diarization</div>
</div>
<span class="llm-tier-badge llm-tier-free">Free</span>
</div>
<div class="llm-card-stats">
<span>100 h lifetime</span>
<span>Speaker labels</span>
<span>Auto-chapters</span>
</div>
<div class="llm-field-row">
<label class="llm-label">API key</label>
<input type="password" class="llm-input" placeholder="AssemblyAI key&hellip;" data-llm-key="assemblyai">
<a class="llm-link" href="https://www.assemblyai.com" target="_blank" rel="noopener">Get key <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-endpoint">
<span class="llm-endpoint-label">Endpoint</span>
<code>https://api.assemblyai.com/v2/transcript</code>
</div>
</div>
</div>
</div><!-- /stt -->
<!-- ── Text to Speech sub-page ───────────────────────────── -->
<div class="s-engines-page" data-page="tts">
<div class="llm-sec-header">
<div>
<h3 class="llm-sec-title">Text to Speech</h3>
<p class="llm-sec-note">Docker-detected engines and local presets are listed together here. Click <strong>Use as TTS</strong> to apply a URL to this app&rsquo;s backend settings.</p>
</div>
<div style="display:flex;gap:8px;align-items:center">
<button class="btn-secondary" type="button" onclick="openAddEngineDialog('tts')"><span class="mdi mdi-plus"></span> Add Custom</button>
<button class="btn-secondary dc-refresh-btn" type="button"><span class="mdi mdi-refresh"></span> Refresh</button>
<span class="llm-free-badge llm-free-local">100% Local</span>
</div>
</div>
<div id="dc-grid-tts" class="dc-grid tts-local-engines-grid"></div>
<div class="llm-local-grid tts-local-engines-grid">
<div class="llm-local-card">
<div class="llm-local-head">
<span class="llm-local-icon">🐟</span>
<span class="llm-local-name">Fish-Speech (S2-Pro)</span>
<span class="llm-local-compat">Clone + Emotion</span>
</div>
<div class="llm-local-metrics">
<span class="llm-metric-chip"><span class="mdi mdi-lightning-bolt"></span> ~0.4&times; GPU</span>
<span class="llm-metric-chip"><span class="mdi mdi-clock-outline"></span> 1&ndash;4 s</span>
<span class="llm-metric-chip"><span class="mdi mdi-star-circle-outline"></span> Premium</span>
<span class="llm-metric-chip"><span class="mdi mdi-memory"></span> ~4 GB VRAM</span>
</div>
<p class="llm-local-desc">Clones a saved WAV voice <strong>and</strong> honours inline emotion markers like <code>(angry)</code>, <code>(whispering)</code>, <code>(excited)</code> per line &mdash; consistent speaker identity with real tone control. The only backend that is both WAV-anchored and style-aware.</p>
<div class="llm-local-url">
<span class="llm-local-url-label">URL</span>
<input class="llm-local-url-inp" type="text" placeholder="http://localhost:38080" data-llm-local-key="fishspeech" data-llm-local-default="http://host.docker.internal:38080" spellcheck="false">
<button class="llm-local-ping" data-ping-key="fishspeech" title="Test connection">Connect</button>
</div>
<div class="llm-local-actions">
<button class="llm-use-btn" id="llm-use-fishspeech-tts" type="button"><span class="mdi mdi-content-copy"></span> Use as TTS</button>
</div>
<div class="llm-local-snippet">
<div class="llm-snippet-bar">
<span>Docker</span>
<button class="llm-copy-btn" data-copy="cd /home/sparky/Docker/fishaudio/s1 && docker compose up -d">Copy</button>
</div>
<pre>cd fishaudio/s1
docker compose up -d # S2-Pro API on :38080</pre>
</div>
<a class="llm-local-link" href="https://github.com/fishaudio/fish-speech" target="_blank" rel="noopener">github.com/fishaudio/fish-speech <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-local-card">
<div class="llm-local-head">
<span class="llm-local-icon"><span class="mdi mdi-speaker-outline"></span></span>
<span class="llm-local-name">Piper TTS</span>
<span class="llm-local-compat">Fast &middot; offline</span>
</div>
<div class="llm-local-metrics">
<span class="llm-metric-chip"><span class="mdi mdi-lightning-bolt"></span> ~1× CPU realtime</span>
<span class="llm-metric-chip"><span class="mdi mdi-clock-outline"></span> &lt;50 ms</span>
<span class="llm-metric-chip"><span class="mdi mdi-star-circle-outline"></span> Good (VITS)</span>
<span class="llm-metric-chip"><span class="mdi mdi-memory"></span> ~50 MB RAM</span>
</div>
<p class="llm-local-desc">Lightning-fast offline TTS. Runs on CPU in real time. 50+ language voices available. Uses Wyoming protocol (port 10200) &mdash; not directly OpenAI-compatible.</p>
<div class="llm-local-url">
<span class="llm-local-url-label">URL</span>
<input class="llm-local-url-inp" type="text" placeholder="localhost:10200" data-llm-local-key="piper" data-llm-local-default="localhost:10200" spellcheck="false">
<button class="llm-local-ping" data-ping-key="piper" title="Test connection">Connect</button>
</div>
<div class="llm-local-actions">
<button class="llm-use-btn" id="llm-use-piper-tts" type="button"><span class="mdi mdi-content-copy"></span> Use as TTS</button>
</div>
<div class="llm-local-snippet">
<div class="llm-snippet-bar">
<span>Docker</span>
<button class="llm-copy-btn" data-copy="docker run -p 10200:10200 rhasspy/wyoming-piper --voice en_US-lessac-medium">Copy</button>
</div>
<pre>docker run -p 10200:10200 \
rhasspy/wyoming-piper \
--voice en_US-lessac-medium</pre>
</div>
<a class="llm-local-link" href="https://github.com/rhasspy/piper" target="_blank" rel="noopener">github.com/rhasspy/piper <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-local-card">
<div class="llm-local-head">
<span class="llm-local-icon">🌸</span>
<span class="llm-local-name">Kokoro FastAPI</span>
<span class="llm-local-compat">OpenAI-compat TTS</span>
</div>
<div class="llm-local-metrics">
<span class="llm-metric-chip"><span class="mdi mdi-lightning-bolt"></span> ~0.1&times; CPU RTF</span>
<span class="llm-metric-chip"><span class="mdi mdi-clock-outline"></span> ~200 ms</span>
<span class="llm-metric-chip"><span class="mdi mdi-star-circle-outline"></span> High (82M)</span>
<span class="llm-metric-chip"><span class="mdi mdi-memory"></span> 300 MB CPU</span>
</div>
<p class="llm-local-desc">Kokoro-82M running behind an OpenAI-compatible TTS endpoint. 11 built-in voices (af_bella, bf_emma, am_adam&hellip;). Drop-in replacement for OpenAI&rsquo;s TTS API.</p>
<div class="llm-local-url">
<span class="llm-local-url-label">URL</span>
<input class="llm-local-url-inp" type="text" placeholder="http://localhost:8880/v1" data-llm-local-key="kokoro" data-llm-local-default="http://localhost:8880/v1" spellcheck="false">
<button class="llm-local-ping" data-ping-key="kokoro" title="Test connection">Connect</button>
</div>
<div class="llm-local-actions">
<button class="llm-use-btn" id="llm-use-kokoro-tts" type="button" title="Copy URL to Settings → Kokoro FastAPI URL and enable Kokoro in Try It Out"><span class="mdi mdi-content-copy"></span> Use as TTS</button>
</div>
<div class="llm-local-snippet">
<div class="llm-snippet-bar">
<span>Docker</span>
<button class="llm-copy-btn" data-copy="docker run -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-cpu:v0.2.2">Copy</button>
</div>
<pre>docker run -p 8880:8880 \
ghcr.io/remsky/kokoro-fastapi-cpu:v0.2.2
# GPU:
docker run -p 8880:8880 --gpus all \
ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.2</pre>
</div>
<a class="llm-local-link" href="https://github.com/remsky/Kokoro-FastAPI" target="_blank" rel="noopener">github.com/remsky/Kokoro-FastAPI <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-local-card">
<div class="llm-local-head">
<span class="llm-local-icon">🗣️</span>
<span class="llm-local-name">XTTS v2</span>
<span class="llm-local-compat">Voice cloning</span>
</div>
<div class="llm-local-metrics">
<span class="llm-metric-chip"><span class="mdi mdi-lightning-bolt"></span> ~0.5× GPU RTF</span>
<span class="llm-metric-chip"><span class="mdi mdi-clock-outline"></span> 13 s</span>
<span class="llm-metric-chip"><span class="mdi mdi-star-circle-outline"></span> High (WAV clone)</span>
<span class="llm-metric-chip"><span class="mdi mdi-memory"></span> ~5 GB</span>
</div>
<p class="llm-local-desc">Coqui XTTS &mdash; multilingual voice cloning from a 6-second sample. 17 languages. Compatible with this app&rsquo;s voice library.</p>
<div class="llm-local-url">
<span class="llm-local-url-label">URL</span>
<input class="llm-local-url-inp" type="text" placeholder="http://localhost:8020/tts_to_audio" data-llm-local-key="xtts" data-llm-local-default="http://localhost:8020/tts_to_audio" spellcheck="false">
<button class="llm-local-ping" data-ping-key="xtts" title="Test connection">Connect</button>
</div>
<div class="llm-local-actions">
<button class="llm-use-btn" id="llm-use-xtts-tts" type="button"><span class="mdi mdi-content-copy"></span> Use as TTS</button>
</div>
<div class="llm-local-snippet">
<div class="llm-snippet-bar">
<span>Docker</span>
<button class="llm-copy-btn" data-copy="docker run -p 8020:80 --gpus all -v /voices:/voices daswer123/xtts-api-server:latest">Copy</button>
</div>
<pre>docker run -p 8020:80 --gpus all \
-v /voices:/voices \
daswer123/xtts-api-server:latest</pre>
</div>
<a class="llm-local-link" href="https://github.com/daswer123/xtts-api-server" target="_blank" rel="noopener">xtts-api-server <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-local-card">
<div class="llm-local-head">
<span class="llm-local-icon"><span class="mdi mdi-microphone-variant"></span></span>
<span class="llm-local-name">VibeVoice</span>
<span class="llm-local-compat">Simple REST TTS</span>
</div>
<div class="llm-local-metrics">
<span class="llm-metric-chip"><span class="mdi mdi-lightning-bolt"></span> Lightweight</span>
<span class="llm-metric-chip"><span class="mdi mdi-clock-outline"></span> Low latency</span>
<span class="llm-metric-chip"><span class="mdi mdi-star-circle-outline"></span> No auth</span>
<span class="llm-metric-chip"><span class="mdi mdi-lan-connect"></span> LAN service</span>
</div>
<p class="llm-local-desc">Lightweight local TTS service with a minimal REST API &mdash; POST text, get audio. No API key, no model field, no OpenAI wrapper needed. Set the URL in Settings and pick <em>VibeVoice</em> as the backend in Try It Out or a routing rule.</p>
<div class="llm-local-url">
<span class="llm-local-url-label">URL</span>
<input class="llm-local-url-inp" type="text" placeholder="http://192.168.178.8:8027" data-llm-local-key="vibevoice" data-llm-local-default="http://192.168.178.8:8027" spellcheck="false">
<button class="llm-local-ping" data-ping-key="vibevoice" title="Test connection">Connect</button>
</div>
<div class="llm-local-actions">
<button class="llm-use-btn" id="llm-use-vibevoice-tts" type="button" title="Copy URL to Settings → VibeVoice URL and enable VibeVoice backend"><span class="mdi mdi-content-copy"></span> Use as TTS</button>
</div>
<div class="llm-local-snippet">
<div class="llm-snippet-bar">
<span>Direct REST test</span>
<button class="llm-copy-btn" data-copy="curl -X POST http://192.168.178.8:8027/tts \&#10; -H &quot;Content-Type: application/json&quot; \&#10; -d '{&quot;text&quot;:&quot;Hello from VibeVoice&quot;}' \&#10; --output vibevoice-test.wav">Copy</button>
</div>
<pre>curl -X POST http://192.168.178.8:8027/tts \
-H "Content-Type: application/json" \
-d '{"text":"Hello from VibeVoice"}' \
--output vibevoice-test.wav</pre>
</div>
<a class="llm-local-link" href="https://github.com/Mekopa/VibeVoice" target="_blank" rel="noopener">github.com/Mekopa/VibeVoice <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
</div>
<h4 class="llm-local-cat" style="margin-top:28px;margin-bottom:12px"><span class="mdi mdi-earth"></span> Cloud APIs <span class="llm-free-badge" style="margin-left:8px;vertical-align:middle">Free tiers</span></h4>
<div class="llm-service-grid">
<div class="llm-card llm-card-highlight">
<div class="llm-card-head">
<span class="llm-card-icon"><span class="mdi mdi-waveform"></span></span>
<div>
<div class="llm-card-name">ElevenLabs</div>
<div class="llm-card-sub">High-quality voice cloning &amp; synthesis</div>
</div>
<span class="llm-tier-badge llm-tier-free">Free</span>
</div>
<div class="llm-card-stats">
<span>10 000 chars / month</span>
<span>2 500 char / request max</span>
<span>Voice cloning supported</span>
</div>
<div class="llm-field-row">
<label class="llm-label">API key</label>
<input type="password" class="llm-input" placeholder="xi-api-key&hellip;" data-llm-key="elevenlabs">
<a class="llm-link" href="https://elevenlabs.io" target="_blank" rel="noopener">Get key <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-endpoint">
<span class="llm-endpoint-label">Endpoint</span>
<code>https://api.elevenlabs.io/v1/text-to-speech</code>
</div>
</div>
<div class="llm-card">
<div class="llm-card-head">
<span class="llm-card-icon"><span class="mdi mdi-fish"></span></span>
<div>
<div class="llm-card-name">Fish Audio</div>
<div class="llm-card-sub">Voice cloning &amp; multilingual TTS</div>
</div>
<span class="llm-tier-badge llm-tier-free">Free</span>
</div>
<div class="llm-card-stats">
<span>1 h audio / month</span>
<span>100 req / min</span>
<span>30+ languages</span>
</div>
<div class="llm-field-row">
<label class="llm-label">API key</label>
<input type="password" class="llm-input" placeholder="Fish Audio key&hellip;" data-llm-key="fish_audio">
<a class="llm-link" href="https://fish.audio" target="_blank" rel="noopener">Get key <span class="mdi mdi-open-in-new link-icon"></span></a>
</div>
<div class="llm-endpoint">
<span class="llm-endpoint-label">Endpoint</span>
<code>https://api.fish.audio/v1/tts</code>
</div>
</div>
<div class="llm-card">
<div class="llm-card-head">
<span class="llm-card-icon"><span class="mdi mdi-music-note"></span></span>
<div>
<div class="llm-card-name">Kokoro TTS</div>
<div class="llm-card-sub">82M model &middot; HuggingFace Spaces demo</div>
</div>
<span class="llm-tier-badge llm-tier-demo">Demo</span>
</div>
<div class="llm-card-stats">
<span>Free web demo</span>
<span>&lt;$1 per 1M chars (paid)</span>
<span>High naturalness</span>
</div>
<div class="llm-info-box">
Use the HF Spaces web demo for quick tests, or run Kokoro locally via Docker for production use.
<br>
<a class="llm-link-inline" href="https://huggingface.co/spaces/hexgrad/Kokoro-TTS" target="_blank" rel="noopener"><span class="mdi mdi-arrow-right"></span> Open Kokoro HF Space</a>
</div>
</div>
</div>
<div class="llm-local-howto">
<div class="llm-howto-icon"><span class="mdi mdi-lightbulb-outline"></span></div>
<div class="llm-howto-body">
<strong>Adding a local service to this app</strong>
<p>All OpenAI-compatible services work the same way: open <em>App Routing</em> in the sidebar, add a new backend entry with the local URL, leave the API key field empty, and set it as the default backend for the voice type you want.</p>
</div>
</div>
</div><!-- /tts -->
<!-- ── Add Custom Engine dialog ─────────────────────────────── -->
<dialog id="add-engine-dlg" class="add-engine-dlg">
<div class="add-engine-form">
<h3 class="add-engine-title"><span class="mdi mdi-plus-circle-outline"></span> Add Custom Engine</h3>
<div class="add-engine-field">
<label for="aed-name">Name</label>
<input id="aed-name" type="text" placeholder="My TTS Server" spellcheck="false">
</div>
<div class="add-engine-field">
<label for="aed-role">Role</label>
<select id="aed-role">
<option value="tts">Text to Speech (TTS)</option>
<option value="stt">Speech to Text (STT)</option>
<option value="llm">Language Model (LLM)</option>
</select>
</div>
<div class="add-engine-field">
<label for="aed-url">URL</label>
<input id="aed-url" type="text" placeholder="http://192.168.1.100:8080" spellcheck="false">
</div>
<div class="add-engine-field">
<label for="aed-container">Docker container name <span class="add-engine-opt">(optional — enables Stop / Start / Restart)</span></label>
<input id="aed-container" type="text" placeholder="my-tts-container" spellcheck="false">
</div>
<div class="add-engine-field">
<label for="aed-desc">Description <span class="add-engine-opt">(optional)</span></label>
<input id="aed-desc" type="text" placeholder="Short description">
</div>
<div class="add-engine-actions">
<button type="button" class="btn-secondary" id="aed-cancel">Cancel</button>
<button type="button" class="btn-primary" id="aed-save">Add Card</button>
</div>
</div>
</dialog>