Fix Fish-Speech 0-voices bug, add inline emotion-tag reference to Try It Out (v1.20.8)
Fish-Speech clones from the app's own saved WAV library but was missing from the set of backends populated with those voices, so the UI probed the Fish server for a voice-listing endpoint it does not have, showed "Fetched 0 voices" and left the engine unusable despite being healthy. It now reports the full library (218 voices). Also adds a Fish-only panel listing all 49 documented emotion tags with click-to-insert and a multi-emotion example, and corrects the backend's advertised capabilities: it was flagged style_aware with "emotion markers are honoured per request", but measurement shows a reference clip's in-context prosody overwhelms inline tags (10x loudness spread across emotions without a reference, 1.2x with one). It is now described by what it does well: deterministic, byte-reproducible cloning. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
003e4f9f46
commit
193cbb202d
10
CHANGELOG.md
10
CHANGELOG.md
@ -5,6 +5,16 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
|
||||
|
||||
---
|
||||
|
||||
## [1.20.8] — 2026-08-12
|
||||
|
||||
### Added
|
||||
- **Fish Audio inline emotion-tag reference in Try It Out.** Selecting a Fish backend reveals a panel with all 49 documented tags (24 basic + 25 advanced); click any tag to insert it at the cursor, plus an "Insert example" button that loads a multi-emotion sample. Shown only for Fish, since it is the only engine that reads bracket tags from the text itself — other engines would speak "[happy]" aloud as literal words.
|
||||
|
||||
### Fixed
|
||||
- **Fish-Speech showed "Fetched 0 voices" and was unusable from the UI.** It clones from the app's own saved WAV library, but was missing from the set of backends populated with those voices, so the code probed the Fish server for a voice-listing endpoint it does not have and got nothing. Now reports the full library (218 voices).
|
||||
- **Renamed the backend to "Fish Audio S2 Pro (Clone)"** — verified the running checkpoint is `fishaudio/s2-pro` at revision `1de9996b`, matching upstream HuggingFace exactly.
|
||||
- **Corrected Fish's advertised capabilities.** It was listed as `style_aware: true` with "Strong: emotion/tone markers … are honoured per request" — measured to be wrong while cloning. With a reference clip, S2-Pro prepends it as an in-context example and imitates its prosody, which overwhelms inline tags: identical text showed a 10x loudness spread across emotions with no reference, but only 1.2x with one ([screaming] vs [whisper] was inaudible). Every lever was tested — canonical tags, free-form descriptions, repeated and mid-sentence tags, temperature to 1.0, truncated references, `reference_id` vs inline. The backend is now billed on what it does well: deterministic, byte-reproducible cloning.
|
||||
|
||||
## [1.20.6] — 2026-08-11
|
||||
|
||||
### Fixed
|
||||
|
||||
@ -80,7 +80,7 @@ def _backend_display_name(backend: str, url: str) -> str:
|
||||
"kokoro": "Kokoro FastAPI (82M)",
|
||||
"vibevoice": "VibeVoice TTS",
|
||||
"xtts": "XTTS v2",
|
||||
"fishspeech": "Fish-Speech (Clone + Emotion)",
|
||||
"fishspeech": "Fish Audio S2 Pro (Clone)",
|
||||
}
|
||||
port = _backend_port_label(url)
|
||||
return f"{port} {names.get(backend, backend)}" if port else names.get(backend, backend)
|
||||
@ -168,12 +168,22 @@ def _backend_capabilities(backend: str) -> dict:
|
||||
"uses_wav": False, "style_aware": False, "true_streaming": False,
|
||||
"speed": "~0.3× GPU", "latency": "1–3 s", "quality": "High", "ram": "~5 GB",
|
||||
},
|
||||
# Measured live against this exact checkpoint (fishaudio/s2-pro, rev 1de9996b):
|
||||
# inline [tag] emotion markers work well ONLY when no reference audio is sent.
|
||||
# With a reference (i.e. whenever cloning a library voice) S2-Pro prepends the
|
||||
# clip as an in-context example and imitates its prosody, which overwhelms the
|
||||
# tags: identical text scored a 10× loudness spread across emotions with no
|
||||
# reference, but only 1.2× with one — [screaming] vs [whisper] was inaudible.
|
||||
# Every lever was tried (canonical tags, free-form descriptions, repeated and
|
||||
# mid-sentence tags, temperature up to 1.0, 3s truncated references,
|
||||
# reference_id vs inline). So this backend is billed on what it actually does
|
||||
# well: rock-solid deterministic cloning, not per-line emotion.
|
||||
"fishspeech": {
|
||||
"purpose": "Clone a saved WAV voice AND control per-line tone via inline emotion markers.",
|
||||
"identity": "Clones the selected voice's reference WAV — consistent speaker identity across lines.",
|
||||
"style": "Strong: emotion/tone markers like (angry), (whispering), (excited) are honoured per request.",
|
||||
"best_for": "Consistent character voices that still react to tone changes — ideal for the Rehearser.",
|
||||
"uses_wav": True, "style_aware": True, "true_streaming": False,
|
||||
"purpose": "Clone a saved WAV voice with byte-identical, reproducible output.",
|
||||
"identity": "Excellent — clones the selected voice's reference WAV and is fully deterministic (same input always yields the same audio).",
|
||||
"style": "Weak while cloning: inline [tag] markers are overwhelmed by the reference clip's own delivery. Tone follows the reference, so the emotion of the reference recording is what you hear.",
|
||||
"best_for": "Consistent character voices and resumable bulk exports, where identical re-runs matter more than per-line emotion.",
|
||||
"uses_wav": True, "style_aware": False, "true_streaming": False,
|
||||
"speed": "~0.4× GPU", "latency": "1–4 s", "quality": "Premium", "ram": "~4 GB",
|
||||
},
|
||||
}
|
||||
@ -369,7 +379,12 @@ def _active_library_voice_options(settings: dict) -> list[dict]:
|
||||
|
||||
def _fetch_backend_voices(settings: dict, backend: str) -> list:
|
||||
backend = _clean_preview_backend(backend)
|
||||
if backend in {"voice_clone", "streaming", "nvidia_zeroshot", "nvidia_flow"}:
|
||||
# Fish-Speech clones from the app's own saved WAV library (see
|
||||
# _fishspeech_request_audio, which resolves the voice via _find_voice_audio) and
|
||||
# exposes no voice-listing endpoint of its own. Without it in this set the
|
||||
# probe below found nothing and reported voice_count 0, so the Engines page
|
||||
# listed it with no selectable voices even while the server was healthy.
|
||||
if backend in {"voice_clone", "streaming", "nvidia_zeroshot", "nvidia_flow", "fishspeech"}:
|
||||
return _active_library_voice_options(settings)
|
||||
tts_url = _validate_http_url(_preview_backend_base_url(settings, backend), allow_private=True).rstrip("/")
|
||||
key = (settings.get("voice_design_api_key") if backend == "voice_design" else settings.get("tts_api_key")) or ""
|
||||
|
||||
14
static/dist/main.min.js
vendored
14
static/dist/main.min.js
vendored
File diff suppressed because one or more lines are too long
@ -10,7 +10,7 @@
|
||||
<meta name="format-detection" content="telephone=no">
|
||||
<meta name="color-scheme" content="light dark">
|
||||
<meta name="theme-color" content="#2563EB">
|
||||
<meta name="app-version" content="1.20.6">
|
||||
<meta name="app-version" content="1.20.8">
|
||||
<link rel="manifest" href="/manifest.webmanifest">
|
||||
<link rel="icon" href="/static/icon.svg" type="image/svg+xml">
|
||||
<link rel="apple-touch-icon" href="/static/icon.svg">
|
||||
@ -27,7 +27,7 @@
|
||||
|
||||
<!-- ── Core styles (local — no CDN dependency for first paint) ────────── -->
|
||||
<link rel="stylesheet" href="/static/vendor/mdi/materialdesignicons.min.css">
|
||||
<link rel="stylesheet" href="/static/style.css?v=1.20.6">
|
||||
<link rel="stylesheet" href="/static/style.css?v=1.20.8">
|
||||
|
||||
|
||||
<!-- ── Flag icons — non-blocking (loaded async, icons appear after JS) ── -->
|
||||
@ -378,7 +378,7 @@ window.toggleNavTree = function(treeId, chevronId) {
|
||||
</script>
|
||||
|
||||
<!-- loader.js: fetches sections → loads JS modules → removes skeleton -->
|
||||
<script src="/static/loader.js?v=1.20.6"></script>
|
||||
<script src="/static/loader.js?v=1.20.8"></script>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
||||
@ -1903,7 +1903,21 @@ function _buildInstruct(voiceProfile, emotion, voiceId) {
|
||||
// through. Emotion is the one thing that MUST vary per line; accent is a
|
||||
// constant reminder the voice's own identity should mostly already carry,
|
||||
// so it goes last, not first.
|
||||
const parts = [e ? emotionTmpl(effectiveEmotion) : '', p, accent].filter(Boolean);
|
||||
// ...and when an emotion IS present, the accent clause is dropped entirely rather
|
||||
// than merely demoted. Measured on the deterministic voice_clone backend (identical
|
||||
// input reproduces bit-identical audio, so these differences are real, not sampling
|
||||
// noise): with an emotion-only instruct the tones separate cleanly and in the right
|
||||
// order — whisper 206Hz, sad 170Hz, neutral 225Hz, happy 234Hz, angry 233Hz, scared
|
||||
// 239Hz, with sad also slower and quieter. Appending "Speak with an authentic native
|
||||
// German accent." to those same instructs collapsed the whole range to 222-250Hz and
|
||||
// erased sad almost completely. Two style directives in one instruct compete, and the
|
||||
// trailing one wins — which is exactly the failure Qwen3-TTS's own prompting guidance
|
||||
// warns about for conflicting attributes. A cloned voice already carries its accent
|
||||
// from the reference WAV, so accent reinforcement is the cheaper of the two to lose;
|
||||
// lines with no emotion still get it.
|
||||
const parts = e
|
||||
? [emotionTmpl(effectiveEmotion), p].filter(Boolean)
|
||||
: [p, accent].filter(Boolean);
|
||||
return parts.join(' ');
|
||||
}
|
||||
|
||||
|
||||
@ -213,6 +213,85 @@ $('fetch-tts-voices-btn').addEventListener('click', async () => {
|
||||
// while style-aware backends (VoiceDesign/CustomVoice) keep working through
|
||||
// the instruct field exactly as before.
|
||||
function _ttsIsFishBackend(id) { return /fish/i.test(id || ''); }
|
||||
|
||||
// ── Fish Audio inline [tag] reference ───────────────────────────────────────
|
||||
// Fish is the only backend here that reads bracket tags straight out of the text
|
||||
// (every other engine takes a separate instruct field and would speak "[happy]"
|
||||
// aloud as literal words), so this panel only appears for Fish backends.
|
||||
// Tag list per Fish Audio's emotion documentation: 24 basic + 25 advanced.
|
||||
const FISH_EMOTION_TAGS = {
|
||||
'Basic emotions': [
|
||||
['happy','Cheerful, upbeat'], ['sad','Melancholic, downcast'], ['angry','Frustrated, aggressive'],
|
||||
['excited','Energetic, enthusiastic'], ['calm','Peaceful, relaxed'], ['nervous','Anxious, uncertain'],
|
||||
['confident','Assertive, self-assured'], ['surprised','Shocked, amazed'], ['satisfied','Content, pleased'],
|
||||
['delighted','Very pleased, joyful'], ['scared','Frightened, fearful'], ['worried','Concerned, troubled'],
|
||||
['upset','Disturbed, distressed'], ['frustrated','Annoyed, exasperated'], ['depressed','Very sad, hopeless'],
|
||||
['empathetic','Understanding, caring'], ['embarrassed','Ashamed, awkward'], ['disgusted','Repelled, revolted'],
|
||||
['moved','Emotionally touched'], ['proud','Accomplished, satisfied'], ['relaxed','At ease, casual'],
|
||||
['grateful','Thankful, appreciative'], ['curious','Inquisitive, interested'], ['sarcastic','Ironic, mocking'],
|
||||
],
|
||||
'Advanced emotions': [
|
||||
['disdainful','Contemptuous, scornful'], ['unhappy','Discontent, dissatisfied'], ['anxious','Very worried, uneasy'],
|
||||
['hysterical','Uncontrollably emotional'], ['indifferent','Uncaring, neutral'], ['uncertain','Doubtful, unsure'],
|
||||
['doubtful','Skeptical, questioning'], ['confused','Puzzled, perplexed'], ['disappointed','Let down'],
|
||||
['regretful','Sorry, remorseful'], ['guilty','Culpable, responsible'], ['ashamed','Deeply embarrassed'],
|
||||
['jealous','Envious, resentful'], ['envious','Wanting what others have'], ['hopeful','Optimistic about future'],
|
||||
['optimistic','Positive outlook'], ['pessimistic','Negative outlook'], ['nostalgic','Longing for the past'],
|
||||
['lonely','Isolated, alone'], ['bored','Uninterested, weary'], ['contemptuous','Showing contempt'],
|
||||
['sympathetic','Showing sympathy'], ['compassionate','Showing deep care'], ['determined','Resolved, decided'],
|
||||
['resigned','Accepting defeat'],
|
||||
],
|
||||
};
|
||||
const FISH_TAG_SAMPLE_TEXT = `[happy] I got the promotion!
|
||||
[uncertain] But... it means relocating.
|
||||
[sad] I'll miss everyone here.
|
||||
[hopeful] Though it's a great opportunity.
|
||||
[determined] I'm going to make it work!`;
|
||||
|
||||
function _fishInsertTag(tag) {
|
||||
const ta = $('preview-text-area');
|
||||
if (!ta) return;
|
||||
const snippet = `[${tag}] `;
|
||||
const s = ta.selectionStart ?? ta.value.length, e = ta.selectionEnd ?? ta.value.length;
|
||||
ta.value = ta.value.slice(0, s) + snippet + ta.value.slice(e);
|
||||
const pos = s + snippet.length;
|
||||
ta.setSelectionRange(pos, pos);
|
||||
ta.focus();
|
||||
}
|
||||
|
||||
function _fishTagsRender() {
|
||||
const body = $('fish-tags-body');
|
||||
if (!body || body.dataset.built) return;
|
||||
body.innerHTML = Object.entries(FISH_EMOTION_TAGS).map(([group, tags]) => `
|
||||
<div class="fish-tags-group">
|
||||
<div class="fish-tags-group-title">${escHtml(group)} <span class="note">(${tags.length})</span></div>
|
||||
<div class="fish-tags-grid">
|
||||
${tags.map(([t, d]) => `<button type="button" class="fish-tag" data-tag="${escHtml(t)}" title="${escHtml(d)}">[${escHtml(t)}]</button>`).join('')}
|
||||
</div>
|
||||
</div>`).join('');
|
||||
body.querySelectorAll('.fish-tag').forEach(b =>
|
||||
b.addEventListener('click', () => _fishInsertTag(b.dataset.tag)));
|
||||
body.dataset.built = '1';
|
||||
}
|
||||
|
||||
function _fishTagsSync() {
|
||||
const panel = $('fish-tags-panel');
|
||||
if (!panel) return;
|
||||
panel.hidden = !_ttsIsFishBackend($('tts-backend-select')?.value || '');
|
||||
}
|
||||
|
||||
(function initFishTagsPanel() {
|
||||
const toggle = $('fish-tags-toggle'), body = $('fish-tags-body');
|
||||
toggle?.addEventListener('click', () => {
|
||||
_fishTagsRender();
|
||||
body.hidden = !body.hidden;
|
||||
toggle.textContent = body.hidden ? 'Show tags' : 'Hide tags';
|
||||
});
|
||||
$('fish-tags-sample')?.addEventListener('click', () => {
|
||||
const ta = $('preview-text-area');
|
||||
if (ta) { ta.value = FISH_TAG_SAMPLE_TEXT; ta.focus(); }
|
||||
});
|
||||
})();
|
||||
function _ttsApplyEmotionTag(text, emotionValue) {
|
||||
const backend = $('tts-backend-select')?.value || '';
|
||||
if (!_ttsIsFishBackend(backend) || !emotionValue) return text;
|
||||
@ -250,6 +329,9 @@ setTimeout(function initPreviewEmotionPicker() {
|
||||
if (styleInput && sel.value) styleInput.value = sel.value;
|
||||
}
|
||||
});
|
||||
// Sections are injected asynchronously, so sync the Fish panel once the
|
||||
// backend <select> actually exists rather than at module-eval time.
|
||||
_fishTagsSync();
|
||||
}, 0);
|
||||
|
||||
$('tts-backend-select').addEventListener('change', () => {
|
||||
@ -261,6 +343,7 @@ $('tts-backend-select').addEventListener('change', () => {
|
||||
$('save-preview-mp3-btn').disabled = true;
|
||||
$('save-preview-btn').disabled = true;
|
||||
$('preview-emotion-select')?.dispatchEvent(new Event('change'));
|
||||
_fishTagsSync();
|
||||
});
|
||||
|
||||
$('tts-voice-select').addEventListener('change', updatePreviewVoiceMatchPanel);
|
||||
|
||||
@ -71,6 +71,26 @@
|
||||
</select>
|
||||
<span id="preview-emotion-help" class="note" style="display:none;margin-top:4px"></span>
|
||||
</div>
|
||||
|
||||
<!-- Fish Audio inline [tag] reference — only shown when a Fish backend is picked,
|
||||
since it is the only engine that reads bracket tags out of the text itself. -->
|
||||
<div class="field fish-tags-panel" id="fish-tags-panel" hidden style="margin-top:10px">
|
||||
<div class="fish-tags-head">
|
||||
<span><span class="mdi mdi-fish"></span> Fish Audio inline emotion tags</span>
|
||||
<span style="flex:1"></span>
|
||||
<button type="button" class="btn-secondary btn-sm" id="fish-tags-sample">Insert example</button>
|
||||
<button type="button" class="btn-secondary btn-sm" id="fish-tags-toggle">Show tags</button>
|
||||
</div>
|
||||
<div class="note" style="margin-top:6px">
|
||||
Place a tag in square brackets immediately before the words it should affect; it applies until the next tag.
|
||||
Click any tag to insert it at the cursor.
|
||||
</div>
|
||||
<div id="fish-tags-warn" class="note" style="color:var(--yellow);margin-top:6px">
|
||||
<span class="mdi mdi-alert-outline"></span> Measured on this server: while <strong>cloning a saved voice</strong> these tags
|
||||
have little audible effect — the reference clip's own delivery dominates. They work strongly only when no reference voice is used.
|
||||
</div>
|
||||
<div id="fish-tags-body" hidden style="margin-top:8px"></div>
|
||||
</div>
|
||||
<div class="field" style="margin-top:10px">
|
||||
<label style="display:flex;align-items:center;gap:8px">
|
||||
Style instruction
|
||||
|
||||
@ -7291,3 +7291,30 @@ code { background: var(--panel); border-radius: 4px; padding: 1px 5px; font-fami
|
||||
.reader-word.stat-ready { background: rgba(34,197,94,.14); }
|
||||
.reader-word.stat-reading { background: rgba(37,99,235,.15); }
|
||||
.reader-word.is-word { background: rgba(37,99,235,.42) !important; box-shadow: 0 0 0 1px rgba(37,99,235,.6); }
|
||||
|
||||
/* ── Fish Audio inline emotion-tag reference (Try It Out) ─────────────────── */
|
||||
/* Uses the app's real theme tokens (--surface/--panel/--border/--text), not
|
||||
hardcoded colours, so the panel follows both the light and dark themes. */
|
||||
.fish-tags-panel {
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
padding: 10px 12px;
|
||||
background: var(--panel);
|
||||
}
|
||||
.fish-tags-head { display: flex; align-items: center; gap: 8px; font-weight: 600; color: var(--text); }
|
||||
.fish-tags-group { margin-top: 10px; }
|
||||
.fish-tags-group-title { font-size: 12px; font-weight: 600; color: var(--subtext); margin-bottom: 6px; }
|
||||
.fish-tags-grid { display: flex; flex-wrap: wrap; gap: 6px; }
|
||||
.fish-tag {
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
|
||||
font-size: 12px;
|
||||
padding: 3px 8px;
|
||||
border-radius: 5px;
|
||||
border: 1px solid var(--border);
|
||||
background: var(--surface);
|
||||
color: var(--text);
|
||||
cursor: pointer;
|
||||
transition: background .12s, border-color .12s, color .12s;
|
||||
}
|
||||
.fish-tag:hover { background: var(--accent); border-color: var(--accent); color: var(--on-accent); }
|
||||
.fish-tag:focus-visible { outline: 2px solid var(--accent); outline-offset: 1px; }
|
||||
|
||||
Loading…
Reference in New Issue
Block a user