feat: add OCR engine profiles
This commit is contained in:
parent
2caa417401
commit
60ef436ce1
@ -8,6 +8,9 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
|
||||
## [Unreleased]
|
||||
|
||||
### Changed
|
||||
- **OCR engine category.** Engines now includes local OCR/document-parser profiles for KDL Frontier Parser nano and
|
||||
Baidu Unlimited-OCR, with custom OCR profiles, connection probes, page-image API guidance, and future Magazine
|
||||
workflow integration documented in the TODO roadmap.
|
||||
- **Initial audiobook casts preserve dialogue boundaries.** The final cast
|
||||
normalization now merges adjacent narration only; same-speaker dialogue
|
||||
rows remain separate so Run 3 segmentation errors stay visible and do not
|
||||
|
||||
30
TODO.md
30
TODO.md
@ -283,6 +283,36 @@ codebase; benefit is relative user/reliability impact.
|
||||
- [ ] **P2 / M–L — Medium–high benefit — Enrich character-form UX.** Add richer inline form editing and per-form
|
||||
bulk-generation controls while preserving one character ID and one voice.
|
||||
|
||||
## Magazine reader roadmap
|
||||
|
||||
Magazines are a separate Studio document type: they use a narrator and layout-aware article extraction, not character
|
||||
casting. The original PDF remains the visual source of truth while approved articles become linked, reflowable HTML.
|
||||
|
||||
### P0 — Magazine foundation
|
||||
|
||||
- [ ] Add a Magazine project type beside Books and Theater Plays.
|
||||
- [ ] Add PDF/JPG/PNG import with page previews and safe local storage.
|
||||
- [x] Add OCR-engine selection from Engines > OCR & Document Parsing, with page-image input and structured output.
|
||||
- [ ] Detect contents/index pages and propose article titles, authors, and page ranges.
|
||||
- [ ] Build a review screen where users approve or correct article boundaries before TTS.
|
||||
- [ ] Preserve source coordinates for every extracted block, image, caption, and page reference.
|
||||
|
||||
### P1 — Magazine listening experience
|
||||
|
||||
- [ ] Generate clean article HTML with headings, captions, figures, sidebars, and page links.
|
||||
- [ ] Generate narrator audio per article with resume/progress and a play-all queue.
|
||||
- [ ] Synchronize paragraph/sentence highlighting with playback and keep PDF and article views linked.
|
||||
- [ ] Show article images or page crops during playback, with optional caption reading.
|
||||
- [ ] Add confidence warnings, extraction diffs, and manual text correction before audio generation.
|
||||
|
||||
### P2 — Advanced magazine intelligence
|
||||
|
||||
- [ ] Improve cross-page continuation detection and automatic sidebar/advertisement exclusion.
|
||||
- [ ] Add image descriptions, chart/table handling, and optional spoken figure summaries.
|
||||
- [ ] Add word-level karaoke timing and EPUB 3 Media Overlay export.
|
||||
- [ ] Add mobile/driving mode with large controls, automatic article advance, and hands-free resume.
|
||||
- [ ] Add OCR-engine benchmark comparisons and per-page fallback selection.
|
||||
|
||||
Three full regression runs of *Der Sturm* against the hand-corrected reference.
|
||||
Everything below is measured, not estimated. Artefacts live in
|
||||
`config/regression_snapshots/der_sturm/` (`*_manual.json` are the human
|
||||
|
||||
@ -43,6 +43,8 @@ _LOCAL_CONTAINER_DEFS: list[dict] = [
|
||||
{"name": "voicebox", "label": "Voicebox", "role": "llm", "port": 17493, "stack": "voicebox", "image": "local", "repo": "", "description": "Companion voice app/control plane. Not used as a normal TTS engine unless wrapped as an API."},
|
||||
{"name": "stt-whisperx-blackwell", "label": "WhisperX · faster-whisper (Blackwell GPU)", "role": "stt", "port": 8003, "stack": "whisperx-blackwell", "image": "local", "repo": "https://github.com/m-bain/whisperX", "description": "WhisperX transcription/alignment service using faster-whisper on the Blackwell GPU. Use as custom STT if its API is OpenAI-compatible or /transcribe."},
|
||||
{"name": "xtts-dgx-spark", "label": "XTTS v2", "role": "tts", "port": 8005, "stack": "xtts_v2", "image": "local", "repo": "https://github.com/daswer123/xtts-api-server", "description": "XTTS v2 OpenAI-compatible speaker endpoint."},
|
||||
{"name": "ocr-kdl-frontier-parser-nano", "label": "KDL Frontier Parser nano", "role": "ocr", "port": 8099, "stack": "", "image": "local", "repo": "https://huggingface.co/KDLAI/KDL-Frontier-Parser-nano", "description": "Local 1.2B document parser for layout detection, OCR, tables, figures, and reading order."},
|
||||
{"name": "ocr-baidu-unlimited-ocr", "label": "Baidu Unlimited-OCR", "role": "ocr", "port": 8098, "stack": "", "image": "local", "repo": "https://github.com/baidu/Unlimited-OCR", "description": "Local long-document OCR and layout parser with single-image and multi-page/PDF workflows."},
|
||||
]
|
||||
|
||||
|
||||
@ -153,6 +155,8 @@ async def probe_url(url: str, type: str = "", provider_api_key: str = Header("",
|
||||
checks = [("/health", None), ("/v1/models", "data"), ("/v1/audio/transcriptions", None)]
|
||||
elif type == "tts":
|
||||
checks = [("/health", None), ("/v1/health", None), ("/v1/audio/voices", None), ("/speakers", None), ("/voices", None)]
|
||||
elif type == "ocr":
|
||||
checks = [("/health", None), ("/v1/models", "data"), ("/v1/chat/completions", None), ("/parse", None), ("/ocr", None)]
|
||||
elif type == "comfyui":
|
||||
checks = [("/system_stats", "system")]
|
||||
else:
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta name="format-detection" content="telephone=no">
|
||||
<meta name="color-scheme" content="light dark">
|
||||
<meta name="theme-color" content="#2563EB">
|
||||
<meta name="app-version" content="1.25.1">
|
||||
<meta name="app-version" content="1.25.2">
|
||||
<link rel="manifest" href="/manifest.webmanifest">
|
||||
<link rel="icon" href="/static/icon.svg" type="image/svg+xml">
|
||||
<link rel="apple-touch-icon" href="/static/icon.svg">
|
||||
@ -27,12 +27,12 @@
|
||||
|
||||
<!-- ── Core styles (local — no CDN dependency for first paint) ────────── -->
|
||||
<link rel="stylesheet" href="/static/vendor/mdi/materialdesignicons.min.css">
|
||||
<link rel="stylesheet" href="/static/style.css?v=1.25.1">
|
||||
<link rel="stylesheet" href="/static/css/resource-diagnostics.css?v=1.25.1">
|
||||
<link rel="stylesheet" href="/static/css/background-jobs.css?v=1.25.1">
|
||||
<link rel="stylesheet" href="/static/css/voice-design.css?v=1.25.1">
|
||||
<link rel="stylesheet" href="/static/css/rehearser.css?v=1.25.1">
|
||||
<link rel="stylesheet" href="/static/css/audiobook.css?v=1.25.1">
|
||||
<link rel="stylesheet" href="/static/style.css?v=1.25.2">
|
||||
<link rel="stylesheet" href="/static/css/resource-diagnostics.css?v=1.25.2">
|
||||
<link rel="stylesheet" href="/static/css/background-jobs.css?v=1.25.2">
|
||||
<link rel="stylesheet" href="/static/css/voice-design.css?v=1.25.2">
|
||||
<link rel="stylesheet" href="/static/css/rehearser.css?v=1.25.2">
|
||||
<link rel="stylesheet" href="/static/css/audiobook.css?v=1.25.2">
|
||||
|
||||
|
||||
<!-- ── Flag icons — non-blocking (loaded async, icons appear after JS) ── -->
|
||||
@ -220,6 +220,7 @@
|
||||
<div class="nav-tree-item" data-engines-cat="stt" onclick="navEnginesCat('stt')"><span class="mdi mdi-ear-hearing"></span> Speech to Text</div>
|
||||
<div class="nav-tree-item" data-engines-cat="tts" onclick="navEnginesCat('tts')"><span class="mdi mdi-account-voice"></span> Text to Speech</div>
|
||||
<div class="nav-tree-item" data-engines-cat="image" onclick="navEnginesCat('image')"><span class="mdi mdi-image-outline"></span> Image Generation</div>
|
||||
<div class="nav-tree-item" data-engines-cat="ocr" onclick="navEnginesCat('ocr')"><span class="mdi mdi-file-document-multiple-outline"></span> OCR & Document Parsing</div>
|
||||
</div>
|
||||
<div class="nav-tree-head nav-subhead" id="nav-integrations-head" onclick="toggleNavTree('nav-integrations-tree','nav-integrations-chevron')">
|
||||
<span class="nav-icon"><span class="mdi mdi-transit-connection-variant"></span></span>
|
||||
@ -397,7 +398,7 @@ window.toggleNavTree = function(treeId, chevronId) {
|
||||
</script>
|
||||
|
||||
<!-- loader.js: fetches sections → loads JS modules → removes skeleton -->
|
||||
<script src="/static/loader.js?v=1.25.1"></script>
|
||||
<script src="/static/loader.js?v=1.25.2"></script>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
||||
@ -392,11 +392,13 @@ async function loadLocalContainers() {
|
||||
const gridTts = $('dc-grid-tts');
|
||||
const gridStt = $('dc-grid-stt');
|
||||
const gridLlm = $('dc-grid-llm');
|
||||
if (!gridTts && !gridStt && !gridLlm) return;
|
||||
const gridOcr = $('dc-grid-ocr');
|
||||
if (!gridTts && !gridStt && !gridLlm && !gridOcr) return;
|
||||
const skel = _dcSkel(2);
|
||||
if (gridTts && !gridTts.querySelector('.llm-local-card')) gridTts.innerHTML = skel;
|
||||
if (gridStt && !gridStt.querySelector('.llm-local-card')) gridStt.innerHTML = skel;
|
||||
if (gridLlm && !gridLlm.querySelector('.llm-local-card')) gridLlm.innerHTML = skel;
|
||||
if (gridOcr && !gridOcr.querySelector('.llm-local-card')) gridOcr.innerHTML = skel;
|
||||
const custom = loadCustomEngineCards();
|
||||
try {
|
||||
const d = await fetchLocalContainersWithRetry();
|
||||
@ -417,6 +419,7 @@ async function loadLocalContainers() {
|
||||
if (gridTts) gridTts.innerHTML = err;
|
||||
if (gridStt) gridStt.innerHTML = err;
|
||||
if (gridLlm) gridLlm.innerHTML = err;
|
||||
if (gridOcr) gridOcr.innerHTML = err;
|
||||
document.querySelectorAll('.dc-retry-inline').forEach(btn => btn.addEventListener('click', loadLocalContainers));
|
||||
}
|
||||
}
|
||||
@ -426,9 +429,10 @@ function renderLocalContainers(containers) {
|
||||
const gridTts = $('dc-grid-tts');
|
||||
const gridStt = $('dc-grid-stt');
|
||||
const gridLlm = $('dc-grid-llm');
|
||||
if (!gridTts && !gridStt && !gridLlm) return;
|
||||
const gridOcr = $('dc-grid-ocr');
|
||||
if (!gridTts && !gridStt && !gridLlm && !gridOcr) return;
|
||||
|
||||
const ROLE_LABEL = { tts: 'TTS', stt: 'STT', 'stt+tts': 'STT · TTS', llm: 'LLM' };
|
||||
const ROLE_LABEL = { tts: 'TTS', stt: 'STT', 'stt+tts': 'STT · TTS', llm: 'LLM', ocr: 'OCR' };
|
||||
const DC_ICONS = {
|
||||
'tts-faster-qwen3-tts-voiceclone': '<span class="mdi mdi-speaker-outline"></span>',
|
||||
'tts-faster-qwen3-tts-voicedesign': '<span class="mdi mdi-auto-fix"></span>',
|
||||
@ -448,6 +452,8 @@ function renderLocalContainers(containers) {
|
||||
'nemotron-asr-streaming': '<span class="mdi mdi-waveform"></span>',
|
||||
'vibevoice-msft-tts': '<span class="mdi mdi-account-voice"></span>',
|
||||
'xtts-dgx-spark': '<span class="mdi mdi-account-music-outline"></span>',
|
||||
'ocr-kdl-frontier-parser-nano': '<span class="mdi mdi-file-document-outline"></span>',
|
||||
'ocr-baidu-unlimited-ocr': '<span class="mdi mdi-file-document-multiple-outline"></span>',
|
||||
'litellm': '<span class="mdi mdi-router-network"></span>',
|
||||
'llama.cpp': '<span class="mdi mdi-chip"></span>',
|
||||
'ollama': '<span class="mdi mdi-circle-outline"></span>',
|
||||
@ -473,8 +479,10 @@ function renderLocalContainers(containers) {
|
||||
'llama.cpp': [[_t(),'local'],[_l(),'model dependent'],[_q(),'OpenAI API'],[_m(),'model dependent']],
|
||||
'ollama': [[_t(),'local'],[_l(),'model dependent'],[_q(),'OpenAI API'],[_m(),'model dependent']],
|
||||
'llama-swap': [[_t(),'swap'],[_l(),'warm after load'],[_q(),'OpenAI API'],[_m(),'model dependent']],
|
||||
'ocr-kdl-frontier-parser-nano': [[_t(),'local'],[_l(),'layout-aware'],[_q(),'1.2B parser'],[_m(),'model dependent']],
|
||||
'ocr-baidu-unlimited-ocr': [[_t(),'local'],[_l(),'multi-page'],[_q(),'long-horizon'],[_m(),'model dependent']],
|
||||
};
|
||||
const roleIcon = { tts: '<span class="mdi mdi-speaker-outline"></span>', stt: '<span class="mdi mdi-microphone-variant"></span>', 'stt+tts': '<span class="mdi mdi-swap-horizontal"></span>', llm: '<span class="mdi mdi-robot-outline"></span>' };
|
||||
const roleIcon = { tts: '<span class="mdi mdi-speaker-outline"></span>', stt: '<span class="mdi mdi-microphone-variant"></span>', 'stt+tts': '<span class="mdi mdi-swap-horizontal"></span>', llm: '<span class="mdi mdi-robot-outline"></span>', ocr: '<span class="mdi mdi-file-document-outline"></span>' };
|
||||
|
||||
function buildCardHtml(c) {
|
||||
const isCustom = !!c.isCustom;
|
||||
@ -577,6 +585,10 @@ function renderLocalContainers(containers) {
|
||||
snippetCode = `curl -s "${snippetBaseUrl}/v1/audio/speech" \\\n -H "Authorization: Bearer dummy" \\\n -H "Content-Type: application/json" \\\n -d '{"model":"tts-1","voice":"default","input":"Hello world","response_format":"wav"}' \\\n --output test.wav`;
|
||||
} else if (c.role === 'stt') {
|
||||
snippetCode = `curl -s "${snippetBaseUrl}/v1/audio/transcriptions" \\\n -F "file=@audio.wav" \\\n -F "model=whisper-1"`;
|
||||
} else if (c.role === 'ocr') {
|
||||
snippetCode = `curl -s "${snippetBaseUrl}/v1/chat/completions" \\
|
||||
-H "Content-Type: application/json" \\
|
||||
-d '{"model":"${c.name}","messages":[{"role":"user","content":[{"type":"text","text":"\\nText Recognition:"},{"type":"image_url","image_url":{"url":"data:image/png;base64,..."}}]}]}'`;
|
||||
} else {
|
||||
snippetCode = `curl -s "${snippetBaseUrl}/v1/chat/completions" \\\n -H "Authorization: Bearer dummy" \\\n -H "Content-Type: application/json" \\\n -d '{"model":"default","messages":[{"role":"user","content":"Hello!"}]}'`;
|
||||
}
|
||||
@ -605,12 +617,14 @@ function renderLocalContainers(containers) {
|
||||
const isTts = c => c.role === 'tts';
|
||||
const isStt = c => c.role === 'stt' || c.role === 'stt+tts';
|
||||
const isLlm = c => c.role === 'llm';
|
||||
const isOcr = c => c.role === 'ocr';
|
||||
|
||||
if (gridTts) gridTts.innerHTML = containers.filter(isTts).map(buildCardHtml).join('');
|
||||
if (gridStt) gridStt.innerHTML = containers.filter(isStt).map(buildCardHtml).join('');
|
||||
if (gridLlm) gridLlm.innerHTML = containers.filter(isLlm).map(buildCardHtml).join('');
|
||||
if (gridOcr) gridOcr.innerHTML = containers.filter(isOcr).map(buildCardHtml).join('');
|
||||
|
||||
[gridTts, gridStt, gridLlm].forEach(grid => {
|
||||
[gridTts, gridStt, gridLlm, gridOcr].forEach(grid => {
|
||||
if (!grid) return;
|
||||
|
||||
// Stop / Start / Restart docker containers
|
||||
|
||||
@ -43,6 +43,18 @@
|
||||
<p class="llm-sec-note">Used to generate character profile pictures. Pick the active provider below and add its API key.</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="engines-head-context header-cat-block" data-engines-cat="ocr">
|
||||
<div class="engines-context-copy">
|
||||
<h3 class="llm-sec-title">OCR & Document Parsing</h3>
|
||||
<p class="llm-sec-note">Configure local OCR and layout-aware document parsers for scanned PDFs, images, and future Magazine projects.</p>
|
||||
</div>
|
||||
<div class="engines-context-actions">
|
||||
<button class="btn-secondary" type="button" onclick="openAddEngineDialog('ocr')"><span class="mdi mdi-plus"></span> Add Custom</button>
|
||||
<button class="btn-secondary dc-refresh-btn" type="button"><span class="mdi mdi-refresh"></span> Refresh</button>
|
||||
<span class="llm-free-badge llm-free-local">100% Local</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- ── Language Models sub-page ──────────────────────────── -->
|
||||
@ -809,6 +821,7 @@
|
||||
<option value="tts">Text to Speech (TTS)</option>
|
||||
<option value="stt">Speech to Text (STT)</option>
|
||||
<option value="llm">Language Model (LLM)</option>
|
||||
<option value="ocr">OCR & Document Parser</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="add-engine-field">
|
||||
|
||||
Loading…
Reference in New Issue
Block a user