fix(engines): resolve LiteLLM card's own key server-side instead of a stale global one

Connect and "Use as LLM" on the LiteLLM Gateway card now resolve that
card's own saved key (engine_api_keys[name]) server-side in
routes/docker.py, ahead of the old global llm_api_key fallback. The
browser can no longer reliably forward a saved key at all -- it's
masked to "__configured__" in every /api/settings response the moment
it's saved -- so relying on a frontend-forwarded header alone left
Connect silently using a stale global key after LiteLLM's own key was
rotated, surfacing a misleading /api/version 404 instead of the real
401.

"Use as LLM" now goes through a dedicated /use-as-llm endpoint that
saves the exact credential Connect just verified (llm_url normalized
to .../v1, llm_api_key copied from the card), rather than a
separately-typed value that could drift again.

Also documents per-user provider-key storage as an explicit item for
the deferred multi-user/identity planning pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

CHANGELOG.md

@@ -5,6 +5,16 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
 
 ---
 
+## [1.28.2] — 2026-09-14
+
+### Fixed
+- **LiteLLM Gateway engine card connected using a stale global LLM key instead of its own saved key, after LiteLLM's key was rotated on a different port.** `/api/probe-url` only ever knew a card's *role* (`llm`), not which specific card it was, so it had no way to look up that card's own saved key — it fell back to the old global `llm_api_key`, got a 401 from `/v1/models`, then fell through `/api/tags` and `/api/version` (both genuinely 404 for LiteLLM) and reported the last of those as the failure, masking the real cause. Connect now passes the card's name through, and the backend resolves that card's own key from `engine_api_keys` before falling back to the global key. "Use as LLM" now goes through a dedicated endpoint that saves the same key Connect just verified (not a separately-typed, easily-stale value) alongside the normalized `.../v1` URL.
+
+### Docs
+- Documented per-user provider-API-key storage as an explicit open item under the deferred multi-user/identity planning pass (`architecture/multi-tenant-and-identity.md`, `TODO.md`) — today's single global, gitignored, 0600 `settings.json` is adequate for one operator but has no per-user secret separation.
+
+---
+
 ## [1.28.1] — 2026-09-14
 
 ### Added

TODO.md

@@ -151,6 +151,11 @@ planning pass — this is a multi-week, cross-repo effort.
   core/database.py, move voices/output/reader_library/audio-cache to per-user
   paths, replace the single global settings.json with per-user settings, replace
   the single shared external_api_key with per-user API keys.
+  - [ ] Includes provider API keys (llm_api_key, groq_api_key, elevenlabs_api_key,
+    engine_api_keys, etc.) — today one value per provider for the whole install
+    (gitignored + 0600 settings.json, never echoed to the browser once saved).
+    Each user needs their own key per provider, scoped under their identity-
+    service user id; see architecture/multi-tenant-and-identity.md §1.
 - [ ] Replace the global TTS/STT/image concurrency semaphores (routes/tts.py,
   routes/stt.py, routes/conversation.py) with per-user fair-share admission
   control.

VERSION

@@ -1 +1 @@
-1.28.1
+1.28.2

architecture/multi-tenant-and-identity.md

@@ -50,6 +50,18 @@ Everything hangs off one process-wide `CONFIG_DIR`
   global, unscoped.
 - `background_jobs` (`core/database.py`) has no owner column either —
   `background_job_list()` returns every job process-wide.
+- **Provider API keys** (`llm_api_key`, `groq_api_key`, `elevenlabs_api_key`,
+  `engine_api_keys` etc.) live in the same global `settings.json`, one value
+  per provider for the whole install — not per user. Today that's protected
+  by gitignore + file mode `0600` and never sent back to the browser once
+  saved (masked as `"__configured__"` in every `/api/settings` response,
+  `routes/settings.py`). That's adequate for one trusted operator; it is
+  **not** a multi-user secret store — there's no way for two accounts to
+  each hold their own key for the same provider, and no per-key access
+  control. When this gets planned: each user's provider keys should be
+  scoped under their identity-service user id, not global settings keys,
+  with the same "never echo a saved value to the browser" rule carried
+  forward.
 
 ### 2. The browser UI has no authentication at all
 

routes/docker.py

@@ -7,10 +7,10 @@ from pathlib import Path
 from urllib.parse import quote
 
 import requests
-from fastapi import APIRouter, Header, HTTPException
+from fastapi import APIRouter, Header, HTTPException, Request
 
 from core.docker_client import _docker_available, _docker_get_json, _docker_post
-from core.config import _load_settings
+from core.config import _load_settings, _save_settings, _normalize_settings
 from core.validation import _normalize_service_url
 from core.provider_health import probe_provider
 
@@ -124,7 +124,7 @@ async def restart_local_container(name: str):
 
 
 @router.get("/api/probe-url")
-async def probe_url(url: str, type: str = "", provider_api_key: str = Header("", alias="X-Provider-API-Key")):
+async def probe_url(url: str, type: str = "", name: str = "", provider_api_key: str = Header("", alias="X-Provider-API-Key")):
     """Server-side API probe — checks service-specific endpoints and validates JSON responses."""
     base = _normalize_service_url(url).rstrip("/")
     for _suffix in ("/v1", "/api/v1"):
@@ -132,13 +132,23 @@ async def probe_url(url: str, type: str = "", provider_api_key: str = Header("",
             base = base[: -len(_suffix)]
             break
     settings = _load_settings()
-    key_to_use = {
+    # Key precedence:
+    #  1. A key supplied explicitly by the browser for this probe (e.g. the
+    #     user just typed/edited it and it isn't saved yet).
+    #  2. The key saved for this specific engine card (settings.engine_api_keys[name]).
+    #     Card values are masked to "__configured__" before ever reaching the
+    #     browser, so on a normal page load the browser has no real secret to
+    #     forward — the backend must resolve it itself from `name`.
+    #  3. The old global llm_api_key, which may belong to a different
+    #     gateway after that card's own key was rotated.
+    card_key = (settings.get("engine_api_keys") or {}).get(name) if name else None
+    key_to_use = provider_api_key or card_key or {
         "anthropic": settings.get("llm_api_key"),
         "llm": settings.get("llm_api_key"),
         "openai": settings.get("llm_api_key"),
         "stt": settings.get("groq_api_key") or settings.get("whisper_api_key"),
         "tts": settings.get("tts_api_key"),
-    }.get(type) or provider_api_key or "sk-dummy-key"
+    }.get(type) or "sk-dummy-key"
     # Anthropic doesn't speak the OpenAI-style Bearer auth every other card
     # here uses — it needs x-api-key + an anthropic-version header, so the
     # generic Bearer probe below would 401 even with a perfectly valid key.
@@ -165,6 +175,40 @@ async def probe_url(url: str, type: str = "", provider_api_key: str = Header("",
     result = probe_provider(base, checks, headers=hdrs, timeout=5)
     return {key: value for key, value in result.items() if key != 'payload'}
 
+
+@router.post("/api/local-containers/{name}/use-as-llm")
+async def use_as_llm(name: str, request: Request, provider_api_key: str = Header("", alias="X-Provider-API-Key")):
+    """Make an engine card the active LLM, saving the same credential Connect
+    just verified — not a possibly-stale global llm_api_key.
+
+    The URL always comes from the request body. The key comes from (in
+    order): an explicit header (a freshly typed, not-yet-saved key), then
+    this card's own saved engine_api_keys[name] — the browser cannot supply
+    that value itself once saved, since it is masked before ever reaching
+    the browser.
+    """
+    body = await request.json()
+    url = _normalize_service_url(str(body.get("url") or "")).strip()
+    if not url:
+        raise HTTPException(400, "URL is required")
+
+    s = _load_settings()
+    card_key = (s.get("engine_api_keys") or {}).get(name)
+    key_to_use = provider_api_key or card_key or ""
+
+    s["llm_url"] = url
+    if key_to_use:
+        s["llm_api_key"] = key_to_use
+        if provider_api_key:
+            # Keep this card's own saved key in sync with what was just typed.
+            engine_keys = dict(s.get("engine_api_keys") or {})
+            engine_keys[name] = provider_api_key
+            s["engine_api_keys"] = engine_keys
+    s = _normalize_settings(s)
+    _save_settings(s)
+    return {"ok": True, "llm_url": url, "key_set": bool(key_to_use)}
+
+
 @router.get("/api/engine-models")
 async def engine_models(url: str, type: str = "", provider_api_key: str = Header("", alias="X-Provider-API-Key")):
     """Return model IDs from an OpenAI-style engine URL without exposing host-only URLs to the browser."""

static/js/ai-backends.js

@@ -138,11 +138,17 @@
     return raw.replace(/^(https?:\/\/)0\.0\.0\.0([\/:])/, '$1host.docker.internal$2');
   }
 
-  async function probeUrl(rawUrl, type = '', apiKey = '') {
+  async function probeUrl(rawUrl, type = '', apiKey = '', name = '') {
     const url = normalizeProbeUrl(rawUrl);
     const params = { url };
     if (type) params.type = type;
-    const r = await fetch('/api/probe-url?' + new URLSearchParams(params));
+    // `name` identifies which engine card this is (e.g. "litellm") so the
+    // backend can resolve that card's own saved key when the browser has
+    // none to forward — engine_api_keys values are masked to the browser
+    // once saved, so a fresh page load never has the real secret in hand.
+    if (name) params.name = name;
+    const headers = apiKey ? { 'X-Provider-API-Key': apiKey } : {};
+    const r = await fetch('/api/probe-url?' + new URLSearchParams(params), { headers });
     return r.json();
   }
 
@@ -184,7 +190,9 @@
       btn.textContent = 'Connecting…';
       try {
         const type = cardType(card);
-        const d = await probeUrl(rawUrl, type);
+        const apiKey = typeof window.engineApiKeyValue === 'function'
+          ? window.engineApiKeyValue(key) : '';
+        const d = await probeUrl(rawUrl, type, apiKey);
         applyCardState(card, key, d.ok, !d.ok);
         if (d.ok) {
           toast(`✓ ${type.toUpperCase() || 'Service'} reachable — ${d.endpoint}`, 'success');
@@ -776,7 +784,9 @@ initLlmSnippets(document);
       connectBtn.innerHTML = '<span class="mdi mdi-loading mdi-spin"></span> Connecting…';
       try {
         const type = cardType(card);
-        const d    = await probeUrl(rawUrl, type);
+        const apiKey = typeof window.engineApiKeyValue === 'function'
+          ? window.engineApiKeyValue(key) : '';
+        const d    = await probeUrl(rawUrl, type, apiKey);
         setConnected(d.ok);
         localStorage.setItem('llm-local-con-' + key, d.ok ? '1' : '0');
         if (d.ok) {

static/js/engines.js

@@ -747,21 +747,17 @@ function renderLocalContainers(containers) {
         try {
           if (window.probeUrl) {
             const type = card ? cardType(card) : '';
-            const d    = await window.probeUrl(rawUrl, type, apiKey);
+            const d    = await window.probeUrl(rawUrl, type, apiKey, key);
             if (d.ok) {
               setConnected(btn, card, useBtn, true);
               localStorage.setItem('dc-con-' + key, '1');
-              // Auto-apply URL to settings so it appears in dropdowns immediately
-              if (useBtn && window.applyAndSaveSettings) {
-                const settingKey = useBtn.dataset.dcUseKey;
-                const url = inp?.value.trim() || inp?.placeholder || '';
-                if (settingKey && url) {
-                  const patch = { [settingKey]: url };
-                  await window.applyAndSaveSettings(patch);
-                  toast(`✓ Connected and set as active ${useBtn.textContent.replace('Use as ', '').trim()}`, 'success');
-                } else {
-                  toast('✓ Service reachable', 'success');
-                }
+              // Auto-apply URL to settings so it appears in dropdowns immediately.
+              // Reuses the "Use as" button's own click handler (rather than
+              // duplicating its logic) so LLM cards go through the same
+              // credential-aware /use-as-llm endpoint Connect just verified.
+              if (useBtn) {
+                useBtn.click();
+                toast(`✓ Connected and set as active ${useBtn.textContent.replace('Use as ', '').trim()}`, 'success');
               } else {
                 toast('✓ Service reachable', 'success');
               }
@@ -793,11 +789,28 @@ function renderLocalContainers(containers) {
         const url = inp?.value.trim() || inp?.placeholder || '';
         const apiKey = apikeyInp?.value.trim() || '';
         if (!url) { toast('Enter a URL first', 'error'); return; }
+        if (settingKey === 'llm_url') {
+          // Goes through a dedicated endpoint (not applyAndSaveSettings) so the
+          // backend can fall back to this card's own saved key — the browser
+          // only has a real key in hand here if the user just typed one.
+          try {
+            const headers = { 'Content-Type': 'application/json' };
+            if (apiKey) headers['X-Provider-API-Key'] = apiKey;
+            const r = await fetch(`/api/local-containers/${encodeURIComponent(urlKey)}/use-as-llm`, {
+              method: 'POST', headers, body: JSON.stringify({ url }),
+            });
+            const d = await r.json();
+            if (!r.ok || !d.ok) { toast(d.error || 'Failed to set active LLM', 'error'); return; }
+            if (_appSettings) { _appSettings.llm_url = url; if (d.key_set) _appSettings.llm_api_key = '__configured__'; }
+            if (window.syncActiveLlmPanel) window.syncActiveLlmPanel(url, apiKey);
+            btn.classList.add('active');
+            toast(`✓ ${btn.textContent.trim()} — now available in the backend dropdown`, 'success');
+          } catch (e) { toast('Failed to set active LLM: ' + e.message, 'error'); }
+          return;
+        }
         if (window.applyAndSaveSettings) {
           const patch = { [settingKey]: url };
-          if (apiKey && settingKey === 'llm_url') patch.llm_api_key = apiKey;
           await window.applyAndSaveSettings(patch);
-          if (settingKey === 'llm_url' && window.syncActiveLlmPanel) window.syncActiveLlmPanel(url, apiKey);
           if (settingKey === 'vision_ocr_url' && window.syncActiveOcrPanel) window.syncActiveOcrPanel(url);
           btn.classList.add('active');
           toast(`✓ ${btn.textContent.trim()} — now available in the backend dropdown`, 'success');