Fix Fish-Speech 0-voices bug, add inline emotion-tag reference to Try It Out (v1.20.8)

Fish-Speech clones from the app's own saved WAV library but was missing from
the set of backends populated with those voices, so the UI probed the Fish
server for a voice-listing endpoint it does not have, showed "Fetched 0
voices" and left the engine unusable despite being healthy. It now reports
the full library (218 voices).

Also adds a Fish-only panel listing all 49 documented emotion tags with
click-to-insert and a multi-emotion example, and corrects the backend's
advertised capabilities: it was flagged style_aware with "emotion markers are
honoured per request", but measurement shows a reference clip's in-context
prosody overwhelms inline tags (10x loudness spread across emotions without a
reference, 1.2x with one). It is now described by what it does well:
deterministic, byte-reproducible cloning.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
mARTin-B78 2026-08-12 00:08:36 +02:00
parent 003e4f9f46
commit 193cbb202d
9 changed files with 193 additions and 14 deletions

View File

@ -5,6 +5,16 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
---
## [1.20.8] — 2026-08-12
### Added
- **Fish Audio inline emotion-tag reference in Try It Out.** Selecting a Fish backend reveals a panel with all 49 documented tags (24 basic + 25 advanced); click any tag to insert it at the cursor, plus an "Insert example" button that loads a multi-emotion sample. Shown only for Fish, since it is the only engine that reads bracket tags from the text itself — other engines would speak "[happy]" aloud as literal words.
### Fixed
- **Fish-Speech showed "Fetched 0 voices" and was unusable from the UI.** It clones from the app's own saved WAV library, but was missing from the set of backends populated with those voices, so the code probed the Fish server for a voice-listing endpoint it does not have and got nothing. Now reports the full library (218 voices).
- **Renamed the backend to "Fish Audio S2 Pro (Clone)"** — verified the running checkpoint is `fishaudio/s2-pro` at revision `1de9996b`, matching upstream HuggingFace exactly.
- **Corrected Fish's advertised capabilities.** It was listed as `style_aware: true` with "Strong: emotion/tone markers … are honoured per request" — measured to be wrong while cloning. With a reference clip, S2-Pro prepends it as an in-context example and imitates its prosody, which overwhelms inline tags: identical text showed a 10x loudness spread across emotions with no reference, but only 1.2x with one ([screaming] vs [whisper] was inaudible). Every lever was tested — canonical tags, free-form descriptions, repeated and mid-sentence tags, temperature to 1.0, truncated references, `reference_id` vs inline. The backend is now billed on what it does well: deterministic, byte-reproducible cloning.
## [1.20.6] — 2026-08-11
### Fixed

View File

@ -1 +1 @@
1.20.6
1.20.8

View File

@ -80,7 +80,7 @@ def _backend_display_name(backend: str, url: str) -> str:
"kokoro": "Kokoro FastAPI (82M)",
"vibevoice": "VibeVoice TTS",
"xtts": "XTTS v2",
"fishspeech": "Fish-Speech (Clone + Emotion)",
"fishspeech": "Fish Audio S2 Pro (Clone)",
}
port = _backend_port_label(url)
return f"{port} {names.get(backend, backend)}" if port else names.get(backend, backend)
@ -168,12 +168,22 @@ def _backend_capabilities(backend: str) -> dict:
"uses_wav": False, "style_aware": False, "true_streaming": False,
"speed": "~0.3× GPU", "latency": "13 s", "quality": "High", "ram": "~5 GB",
},
# Measured live against this exact checkpoint (fishaudio/s2-pro, rev 1de9996b):
# inline [tag] emotion markers work well ONLY when no reference audio is sent.
# With a reference (i.e. whenever cloning a library voice) S2-Pro prepends the
# clip as an in-context example and imitates its prosody, which overwhelms the
# tags: identical text scored a 10× loudness spread across emotions with no
# reference, but only 1.2× with one — [screaming] vs [whisper] was inaudible.
# Every lever was tried (canonical tags, free-form descriptions, repeated and
# mid-sentence tags, temperature up to 1.0, 3s truncated references,
# reference_id vs inline). So this backend is billed on what it actually does
# well: rock-solid deterministic cloning, not per-line emotion.
"fishspeech": {
"purpose": "Clone a saved WAV voice AND control per-line tone via inline emotion markers.",
"identity": "Clones the selected voice's reference WAV — consistent speaker identity across lines.",
"style": "Strong: emotion/tone markers like (angry), (whispering), (excited) are honoured per request.",
"best_for": "Consistent character voices that still react to tone changes — ideal for the Rehearser.",
"uses_wav": True, "style_aware": True, "true_streaming": False,
"purpose": "Clone a saved WAV voice with byte-identical, reproducible output.",
"identity": "Excellent — clones the selected voice's reference WAV and is fully deterministic (same input always yields the same audio).",
"style": "Weak while cloning: inline [tag] markers are overwhelmed by the reference clip's own delivery. Tone follows the reference, so the emotion of the reference recording is what you hear.",
"best_for": "Consistent character voices and resumable bulk exports, where identical re-runs matter more than per-line emotion.",
"uses_wav": True, "style_aware": False, "true_streaming": False,
"speed": "~0.4× GPU", "latency": "14 s", "quality": "Premium", "ram": "~4 GB",
},
}
@ -369,7 +379,12 @@ def _active_library_voice_options(settings: dict) -> list[dict]:
def _fetch_backend_voices(settings: dict, backend: str) -> list:
backend = _clean_preview_backend(backend)
if backend in {"voice_clone", "streaming", "nvidia_zeroshot", "nvidia_flow"}:
# Fish-Speech clones from the app's own saved WAV library (see
# _fishspeech_request_audio, which resolves the voice via _find_voice_audio) and
# exposes no voice-listing endpoint of its own. Without it in this set the
# probe below found nothing and reported voice_count 0, so the Engines page
# listed it with no selectable voices even while the server was healthy.
if backend in {"voice_clone", "streaming", "nvidia_zeroshot", "nvidia_flow", "fishspeech"}:
return _active_library_voice_options(settings)
tts_url = _validate_http_url(_preview_backend_base_url(settings, backend), allow_private=True).rstrip("/")
key = (settings.get("voice_design_api_key") if backend == "voice_design" else settings.get("tts_api_key")) or ""

File diff suppressed because one or more lines are too long

View File

@ -10,7 +10,7 @@
<meta name="format-detection" content="telephone=no">
<meta name="color-scheme" content="light dark">
<meta name="theme-color" content="#2563EB">
<meta name="app-version" content="1.20.6">
<meta name="app-version" content="1.20.8">
<link rel="manifest" href="/manifest.webmanifest">
<link rel="icon" href="/static/icon.svg" type="image/svg+xml">
<link rel="apple-touch-icon" href="/static/icon.svg">
@ -27,7 +27,7 @@
<!-- ── Core styles (local — no CDN dependency for first paint) ────────── -->
<link rel="stylesheet" href="/static/vendor/mdi/materialdesignicons.min.css">
<link rel="stylesheet" href="/static/style.css?v=1.20.6">
<link rel="stylesheet" href="/static/style.css?v=1.20.8">
<!-- ── Flag icons — non-blocking (loaded async, icons appear after JS) ── -->
@ -378,7 +378,7 @@ window.toggleNavTree = function(treeId, chevronId) {
</script>
<!-- loader.js: fetches sections → loads JS modules → removes skeleton -->
<script src="/static/loader.js?v=1.20.6"></script>
<script src="/static/loader.js?v=1.20.8"></script>
</body>
</html>

View File

@ -1903,7 +1903,21 @@ function _buildInstruct(voiceProfile, emotion, voiceId) {
// through. Emotion is the one thing that MUST vary per line; accent is a
// constant reminder the voice's own identity should mostly already carry,
// so it goes last, not first.
const parts = [e ? emotionTmpl(effectiveEmotion) : '', p, accent].filter(Boolean);
// ...and when an emotion IS present, the accent clause is dropped entirely rather
// than merely demoted. Measured on the deterministic voice_clone backend (identical
// input reproduces bit-identical audio, so these differences are real, not sampling
// noise): with an emotion-only instruct the tones separate cleanly and in the right
// order — whisper 206Hz, sad 170Hz, neutral 225Hz, happy 234Hz, angry 233Hz, scared
// 239Hz, with sad also slower and quieter. Appending "Speak with an authentic native
// German accent." to those same instructs collapsed the whole range to 222-250Hz and
// erased sad almost completely. Two style directives in one instruct compete, and the
// trailing one wins — which is exactly the failure Qwen3-TTS's own prompting guidance
// warns about for conflicting attributes. A cloned voice already carries its accent
// from the reference WAV, so accent reinforcement is the cheaper of the two to lose;
// lines with no emotion still get it.
const parts = e
? [emotionTmpl(effectiveEmotion), p].filter(Boolean)
: [p, accent].filter(Boolean);
return parts.join(' ');
}

View File

@ -213,6 +213,85 @@ $('fetch-tts-voices-btn').addEventListener('click', async () => {
// while style-aware backends (VoiceDesign/CustomVoice) keep working through
// the instruct field exactly as before.
function _ttsIsFishBackend(id) { return /fish/i.test(id || ''); }
// ── Fish Audio inline [tag] reference ───────────────────────────────────────
// Fish is the only backend here that reads bracket tags straight out of the text
// (every other engine takes a separate instruct field and would speak "[happy]"
// aloud as literal words), so this panel only appears for Fish backends.
// Tag list per Fish Audio's emotion documentation: 24 basic + 25 advanced.
const FISH_EMOTION_TAGS = {
'Basic emotions': [
['happy','Cheerful, upbeat'], ['sad','Melancholic, downcast'], ['angry','Frustrated, aggressive'],
['excited','Energetic, enthusiastic'], ['calm','Peaceful, relaxed'], ['nervous','Anxious, uncertain'],
['confident','Assertive, self-assured'], ['surprised','Shocked, amazed'], ['satisfied','Content, pleased'],
['delighted','Very pleased, joyful'], ['scared','Frightened, fearful'], ['worried','Concerned, troubled'],
['upset','Disturbed, distressed'], ['frustrated','Annoyed, exasperated'], ['depressed','Very sad, hopeless'],
['empathetic','Understanding, caring'], ['embarrassed','Ashamed, awkward'], ['disgusted','Repelled, revolted'],
['moved','Emotionally touched'], ['proud','Accomplished, satisfied'], ['relaxed','At ease, casual'],
['grateful','Thankful, appreciative'], ['curious','Inquisitive, interested'], ['sarcastic','Ironic, mocking'],
],
'Advanced emotions': [
['disdainful','Contemptuous, scornful'], ['unhappy','Discontent, dissatisfied'], ['anxious','Very worried, uneasy'],
['hysterical','Uncontrollably emotional'], ['indifferent','Uncaring, neutral'], ['uncertain','Doubtful, unsure'],
['doubtful','Skeptical, questioning'], ['confused','Puzzled, perplexed'], ['disappointed','Let down'],
['regretful','Sorry, remorseful'], ['guilty','Culpable, responsible'], ['ashamed','Deeply embarrassed'],
['jealous','Envious, resentful'], ['envious','Wanting what others have'], ['hopeful','Optimistic about future'],
['optimistic','Positive outlook'], ['pessimistic','Negative outlook'], ['nostalgic','Longing for the past'],
['lonely','Isolated, alone'], ['bored','Uninterested, weary'], ['contemptuous','Showing contempt'],
['sympathetic','Showing sympathy'], ['compassionate','Showing deep care'], ['determined','Resolved, decided'],
['resigned','Accepting defeat'],
],
};
const FISH_TAG_SAMPLE_TEXT = `[happy] I got the promotion!
[uncertain] But... it means relocating.
[sad] I'll miss everyone here.
[hopeful] Though it's a great opportunity.
[determined] I'm going to make it work!`;
function _fishInsertTag(tag) {
const ta = $('preview-text-area');
if (!ta) return;
const snippet = `[${tag}] `;
const s = ta.selectionStart ?? ta.value.length, e = ta.selectionEnd ?? ta.value.length;
ta.value = ta.value.slice(0, s) + snippet + ta.value.slice(e);
const pos = s + snippet.length;
ta.setSelectionRange(pos, pos);
ta.focus();
}
function _fishTagsRender() {
const body = $('fish-tags-body');
if (!body || body.dataset.built) return;
body.innerHTML = Object.entries(FISH_EMOTION_TAGS).map(([group, tags]) => `
<div class="fish-tags-group">
<div class="fish-tags-group-title">${escHtml(group)} <span class="note">(${tags.length})</span></div>
<div class="fish-tags-grid">
${tags.map(([t, d]) => `<button type="button" class="fish-tag" data-tag="${escHtml(t)}" title="${escHtml(d)}">[${escHtml(t)}]</button>`).join('')}
</div>
</div>`).join('');
body.querySelectorAll('.fish-tag').forEach(b =>
b.addEventListener('click', () => _fishInsertTag(b.dataset.tag)));
body.dataset.built = '1';
}
function _fishTagsSync() {
const panel = $('fish-tags-panel');
if (!panel) return;
panel.hidden = !_ttsIsFishBackend($('tts-backend-select')?.value || '');
}
(function initFishTagsPanel() {
const toggle = $('fish-tags-toggle'), body = $('fish-tags-body');
toggle?.addEventListener('click', () => {
_fishTagsRender();
body.hidden = !body.hidden;
toggle.textContent = body.hidden ? 'Show tags' : 'Hide tags';
});
$('fish-tags-sample')?.addEventListener('click', () => {
const ta = $('preview-text-area');
if (ta) { ta.value = FISH_TAG_SAMPLE_TEXT; ta.focus(); }
});
})();
function _ttsApplyEmotionTag(text, emotionValue) {
const backend = $('tts-backend-select')?.value || '';
if (!_ttsIsFishBackend(backend) || !emotionValue) return text;
@ -250,6 +329,9 @@ setTimeout(function initPreviewEmotionPicker() {
if (styleInput && sel.value) styleInput.value = sel.value;
}
});
// Sections are injected asynchronously, so sync the Fish panel once the
// backend <select> actually exists rather than at module-eval time.
_fishTagsSync();
}, 0);
$('tts-backend-select').addEventListener('change', () => {
@ -261,6 +343,7 @@ $('tts-backend-select').addEventListener('change', () => {
$('save-preview-mp3-btn').disabled = true;
$('save-preview-btn').disabled = true;
$('preview-emotion-select')?.dispatchEvent(new Event('change'));
_fishTagsSync();
});
$('tts-voice-select').addEventListener('change', updatePreviewVoiceMatchPanel);

View File

@ -71,6 +71,26 @@
</select>
<span id="preview-emotion-help" class="note" style="display:none;margin-top:4px"></span>
</div>
<!-- Fish Audio inline [tag] reference — only shown when a Fish backend is picked,
since it is the only engine that reads bracket tags out of the text itself. -->
<div class="field fish-tags-panel" id="fish-tags-panel" hidden style="margin-top:10px">
<div class="fish-tags-head">
<span><span class="mdi mdi-fish"></span> Fish Audio inline emotion tags</span>
<span style="flex:1"></span>
<button type="button" class="btn-secondary btn-sm" id="fish-tags-sample">Insert example</button>
<button type="button" class="btn-secondary btn-sm" id="fish-tags-toggle">Show tags</button>
</div>
<div class="note" style="margin-top:6px">
Place a tag in square brackets immediately before the words it should affect; it applies until the next tag.
Click any tag to insert it at the cursor.
</div>
<div id="fish-tags-warn" class="note" style="color:var(--yellow);margin-top:6px">
<span class="mdi mdi-alert-outline"></span> Measured on this server: while <strong>cloning a saved voice</strong> these tags
have little audible effect — the reference clip's own delivery dominates. They work strongly only when no reference voice is used.
</div>
<div id="fish-tags-body" hidden style="margin-top:8px"></div>
</div>
<div class="field" style="margin-top:10px">
<label style="display:flex;align-items:center;gap:8px">
Style instruction

View File

@ -7291,3 +7291,30 @@ code { background: var(--panel); border-radius: 4px; padding: 1px 5px; font-fami
.reader-word.stat-ready { background: rgba(34,197,94,.14); }
.reader-word.stat-reading { background: rgba(37,99,235,.15); }
.reader-word.is-word { background: rgba(37,99,235,.42) !important; box-shadow: 0 0 0 1px rgba(37,99,235,.6); }
/* ── Fish Audio inline emotion-tag reference (Try It Out) ─────────────────── */
/* Uses the app's real theme tokens (--surface/--panel/--border/--text), not
hardcoded colours, so the panel follows both the light and dark themes. */
.fish-tags-panel {
border: 1px solid var(--border);
border-radius: var(--radius);
padding: 10px 12px;
background: var(--panel);
}
.fish-tags-head { display: flex; align-items: center; gap: 8px; font-weight: 600; color: var(--text); }
.fish-tags-group { margin-top: 10px; }
.fish-tags-group-title { font-size: 12px; font-weight: 600; color: var(--subtext); margin-bottom: 6px; }
.fish-tags-grid { display: flex; flex-wrap: wrap; gap: 6px; }
.fish-tag {
font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
font-size: 12px;
padding: 3px 8px;
border-radius: 5px;
border: 1px solid var(--border);
background: var(--surface);
color: var(--text);
cursor: pointer;
transition: background .12s, border-color .12s, color .12s;
}
.fish-tag:hover { background: var(--accent); border-color: var(--accent); color: var(--on-accent); }
.fish-tag:focus-visible { outline: 2px solid var(--accent); outline-offset: 1px; }