Commit Graph

59 Commits

Author SHA1 Message Date
48af2b3ccb feat(magazine): detect continuation pages and suggest titles for untitled articles
"Page N" entries come from review stand-ins created after Approve saved only
the visible page's regions. New core/magazine_untitled.py classifies untitled
entries (continuation, article head, advert, picture page, contents, junk),
proposes merges into the previous article with confidence and evidence
(optional LLM check in the ambiguous band), and suggests titles
(deterministic, optional validated and cached LLM titles). Exposed through
derived GET /articles fields, GET /untitled, quality report, undoable
merge-into-previous / accept-title routes, POST /titles/suggest and
scripts/magazine_untitled.py (dry run by default, --apply backs up first).
Merges mark audio_stale; no audio is deleted; real titles are never
overwritten.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 14:36:29 +02:00
7eabdf0e43 feat(characters): distinct deterministic face profiles per character
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-03 14:25:56 +02:00
1f812d2faf fix(characters): merge profiles only on real names, keep other cast names, reasoning and foreign quotes out of a character
Real-library audit (270 records) traced the data damage to:
- identity matching on ANY shared token ("Zwerg", "der Zwerg", "Mann") with
  "longer text wins" merging (Arthag b02 = Kolon's species/relationships), and
- csRosterCanonicalName renaming a sheet to the first cast name among its
  aliases, which saved 16 whole-sheet copies (Zwerg/Kolon = Arthag b03,
  Fremder = Mann, Alchimist = Promos ...).

Fixes:
- static/js/character-identity.js + core/character_records.py: generic labels
  are never identity evidence; csFindMergeKey, csConsolidateAliasDuplicates,
  csRosterCanonicalName, clSameIdentity use it; own name always wins; clUpsert
  writes to the exact-name record before any alias scan; no seed rename onto a
  name that has its own record.
- _charsheets_parse: in cast-target mode drop other cast members' names from
  aliases / first / last name (Kolon: "Sharraz Garthai"), strip reasoning
  leakage ("Warte:", "Der Text sagt ...") from every text field.
- _character_scene_evidence: only a quote naming the character is used,
  otherwise its own mannerisms (16 panels showed another character's scene).
- _character_age_category: Junge/Mädchen/boy/girl, number words, Sommer/
  Winter ages, evidence clause, age-tagged quotes; "15 Jahre älter" is no age.
- scripts/audit_character_records.py (read-only) and
  scripts/clean_character_records.py (dry run by default, --apply backs up).

Tests: tests/test_character_record_hygiene.py, tests/character-identity.test.mjs.
Docs: architecture/character-extraction.md, Manual, CHANGELOG.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 14:09:53 +02:00
04f791e1f5 feat(characters): translate character info to English before image prompts
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-03 12:01:42 +02:00
e60126721d feat(magazine): record/replay cache for vision responses in OCR comparisons
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-03 11:09:17 +02:00
9e5ea609ce feat(magazine): word-level timing via forced-aligner sidecar with player word highlight
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-03 00:37:40 +02:00
788f186b9c fix(magazine): use sentence timings of interview renders for player and EPUB
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-03 00:09:45 +02:00
80ec22b752 fix(magazine): deleting a picture removes both of its records; say clearly while a turn is being read
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 18:35:20 +02:00
28d6439604 fix(magazine): unbreakable text can no longer collapse the review; endless repeated tokens are OCR noise
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 13:13:40 +02:00
130fd273c6 fix(magazine): read a turned page through a ladder (turned, original with carried boxes, tesseract)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 12:31:44 +02:00
6fdf9fbc38 feat(magazine): turn a page scanned sideways or upside down and read it again
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 12:13:13 +02:00
9ad881ada8 feat(magazine): manage OPDS catalogues in the app, env-based login, result paging
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 11:03:03 +02:00
268ed391a3 feat(magazine): EPUB 3 Media Overlay export with measured sentence timings; tolerate missing mutagen
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:33:08 +02:00
4a20a12f66 feat(magazine): show sidebars in the reading view as labelled, never-narrated asides
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:26:27 +02:00
0f4b28ba14 feat(magazine): link a tail to its head from 'continued from page N'
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:20:35 +02:00
0005ae0ebd feat(magazine): import a magazine from PNG/JPEG page images
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:19:14 +02:00
0d3178f733 docs: general-purpose OCR porting brief for letters, papers and forms
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:10:59 +02:00
6faaa349d5 docs: OCR porting brief; test: current prompt version fixture, known phone-width issue
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 21:38:28 +02:00
35e0a64665 feat: OCR benchmark harness and first candidate results (docling, PaddleOCR)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 17:55:48 +02:00
c3c5c55f17 feat(magazine): find and import issues from the Internet Archive and OPDS catalogues
Also reuses the existing magFmtClock for the contents list (the duplicate
tripped the frontend collision gate).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 17:35:08 +02:00
df4bfc934a docs: deep-dive UI/UX and feature-parity comparison with VoiceStudio
Grounded in VoiceStudio's own published screenshots (voice cloning, voice
design, dubbing, model management) and this app's actual served nav/routes
-- no live browser render was available this session, stated explicitly.
Identifies real gaps (video dubbing, live dictation, telephony
integrations, offline translation, model-pack UI, command palette, a
curated voice-design preset gallery) and two low-risk ideas worth adopting
(preset gallery, compute-device indicator).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 13:46:14 +02:00
fd43710caa docs: evaluate and compare VoiceStudio (debpalash/VoiceStudio)
Studied the repo (README, license notice, engine catalog, MCP/API docs) and
the GitHub API record, and probed the user's already-running instance at
192.168.178.8:3900 read-only (/health only). No integration attempted.
Written comparison at architecture/voicestudio-comparison.md; TODO items
1-3 marked done.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 13:23:16 +02:00
faa3c16a91 fix(magazine): exclude advertisement pages from magazine-comparison statistics
A page either side marks as a full-page ad is skipped by that side's quality
run, so comparing its near-empty boxes against the same page freshly OCR'd
on the other side (ad-marking is a one-time manual step) was a review-state
difference, not a pipeline one, skewing whole-issue numbers. Verified: text
similarity 0.837->0.870, box precision 0.62->0.66 on the real re-import.

Also records the clean from-scratch re-ingest results: the ingest-time
duplicate-check ordering fix is proven in isolation (0 additional duplicates
found afterward, vs 13 before the fix), and the auto-chained Auto-fix ran
end to end.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 13:11:00 +02:00
6f2dfca89a docs(magazine): record D-E re-run results with both G fixes applied
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 09:18:11 +02:00
1fb5528112 fix(magazine): drop hallucinated vision-OCR refusal text tiled over unreadable images
TODO A-E marked done for this pass; comparison report saved as evidence.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-28 23:26:59 +02:00
91caa6cf90 feat(magazine): fuse paragraphs across columns/pages, article-aware acronyms, German/English spellcheck
- fuse_paragraph_blocks joins a paragraph split by a column or page break into one
  narration block (reader, podcast source and MP3), keeping source regions for highlighting
- language decided once per article; per-article acronym glossary (LLM classifies unknown
  acronyms from context), letter-spelled acronym lists, extended DE/EN database
- core/spelling: dictionary findings (compounds, inflections, line-break hyphens, other-language
  words), marked in review and listed per page/article; guarded LLM agent fixes; quality run
  now includes the spelling pass
- keep suspended compound hyphens (Programmier- und) when merging regions / building layouts

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-28 08:09:30 +02:00
48c546fbc8 fix(magazine): region-type picker and page-quality buttons work; stronger quality audit
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
- Export the region-type picker handlers and magRunPageQuality to the page (they threw
  "not defined", so the picker never opened and Find missed text / Verify OCR did nothing)
- Add a unit test that fails when an inline mag* handler is not exported
- Quality audit: pixel-level clipping check on all four box edges, duplicate-article
  detection, detected full-page advertisements skipped and shown green (no manual button)
- Driving mode text is karaoke-style (whole article, current sentence and word highlighted)

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-26 23:07:19 +02:00
9904459913 feat(magazine): whole-magazine quality check, page stages, ad pages, driving-mode player
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
- Quality run: per-page missed-text recovery, guarded OCR re-read and a pixel audit
  (uncovered ink, overlapping boxes, clipped box edges); report saved as quality_report.json
- Thumbnail grid: quality-stage colours, flag badge, live per-page indicator for every
  batch step (OCR, detection, figure fixing, quality run, audio)
- Pages can be approved as advertisements (not narrated, skipped by quality run)
- Driving mode: +/-15 s, seek bar, playback speed, text and image toggles
- Plus the previously uncommitted 1.53.5 magazine/OCR work in the same tree

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-26 19:29:43 +02:00
b1c0197652 Release v1.53.2
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
2026-09-24 07:44:16 +02:00
f24a83eb75 fix(magazine): screen invented picture/chart/table regions against page ink
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 05:03:57 +02:00
1edc941dbf feat(magazine): live region boxes in the import preview; interrupted imports no longer spin for ever
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
The preview carries each page's regions (percent of the page, type, reading
order) for the reading pass and the labelling pass, drawn over the page image
with a legend. A server restart that killed the import left it "running" for
ever; the status route now reports it as failed. v1.42.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 01:09:12 +02:00
703052d1d2 fix(magazine): adding a region crashed after a byline split; undo could roll back later changes
original_text is now a Block field, so paths that rebuild blocks keep it
instead of crashing. Undo snapshots are sealed with a digest of the saved
state and refused once anything else changed; actions without their own undo
point clear the Undo button. v1.41.1.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 00:50:33 +02:00
2d5d5da777 feat(characters): short framing-first portrait prompt; the sheet follows what the finished portrait shows
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
Portrait backdrop measured 0/4 -> 4/4 plain on the failing character, no
change on the one that already passed. After the portrait passes, the vision
model describes hair, facial hair and eyes, and the turnaround and story
prompts are rebuilt with it -- a character with no hair in his sheet got a
different haircut on the sheet and failed the same-person check 3/3.
Costume cut and colours restored for armed characters. v1.41.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 20:09:45 +02:00
fb56f4a6d7 fix(characters): short framing-first turnaround prompt; drop passing moods, gestures and scars from images
Measured with the image check: full-body turnarounds 0/5 -> 10/10 (Karyla),
12/12 on Karyla, Darrag and Eolan after the face fixes. The 4-step model
loses the tail of a ~3000-char prompt. Scars are left out of image prompts:
any scar wording, even "fully healed", rendered as fresh gashes. v1.40.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 17:38:03 +02:00
f155839300 feat(magazine): select many boxes and act once, undo instead of confirm, fold labels into new pictures
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
Bulk delete/type for a click- or rectangle-selected set; every destructive
region change returns an undo token (one level, refused once something newer
happened); text inside a region that becomes a picture is folded into it.
TODO: model-based layout detection (PP-DocLayout, LayoutReader) with licence
notes. v1.39.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 16:33:01 +02:00
99f2f2260e feat(magazine): edit, split, merge and re-OCR a region from its row, with keyboard control
Split at the cursor and merge-with-next keep article membership right; typed
letters in the editor never fire shortcuts; focus returns to the same row
after a redraw. v1.38.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 12:11:14 +02:00
b7aa823455 feat(magazine): clean up boxes - text in pictures, OCR noise, duplicates, run-on author lines
Rules sized on a real 204-page issue before writing them; each re-types
(not read aloud, reversible), none deletes. A run-on author line's sentence
becomes its own paragraph instead of being glued to "the next paragraph",
which was right in only 1 of 5 real cases. Runs on import and re-OCR, and on
demand with a layout backup. v1.37.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 12:06:33 +02:00
baab49dbdf feat(magazine): set a page's reading order by hand (type a position or click in order)
A hand-ordered page is locked: Fix reading order, type changes and newly drawn
boxes leave it alone until Automatic order. v1.36.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 11:58:37 +02:00
fd1de80f83 feat(magazine): column reading order, region type picker and delete per row, fix reading order
Columns are found from where column-width text starts, not from the whitespace
between them (loose OCR boxes shrank one gutter to 4-6 pt and a pull-quote
bridged it, collapsing page 38 into one interleaved column). Headlines get
their own band, pull-quotes are read at the end of the page. Measured over a
204-page issue: 89 sentence continuations kept vs 85.

Each review row gets a type picker (text kinds, page number, source, ad,
image) and its own delete button; text<->image conversions keep the region
in its article. Fix reading order re-reads existing issues without OCR and
leaves hand-ordered pages alone. The shared confirm dialog is now an
alertdialog for screen readers. v1.35.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 11:38:33 +02:00
981341de3f feat(characters): check every render, retry with a new seed, and take the sheet's face from the profile
The job checks the portrait crop, then the turnaround, story panel and
same-person against the sheet, re-renders on failure (up to 3 attempts) and
keeps the attempt with the fewest faults. ReActor transfers the profile's face
onto the turnaround and the largest face in the story panel. If the checker is
down, retries stop and the pair is marked unverified. The outcome is saved
with the pair and shown on the character page.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 10:32:47 +02:00
ac8a6c7a24 feat(characters): automatic vision check for generated character images
The model only reports observations; expectations come from the sheet and the
comparison is in code. Fail-safe: an unreachable checker or unreadable answer
is a failure. Calibrated 14/14 on panels with known defects using the already
loaded LLM; Ling 3.0 Flash VL (free) scored 8/14 with 6 unreadable answers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 10:23:42 +02:00
227c42b85d docs: record the reverted per-view turnaround experiment
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 09:33:52 +02:00
fb4cdbba0c feat(characters): turnaround drawn from the portrait, Dark Eye medieval setting, object-only prop tiles
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-20 06:19:56 +02:00
4ab345d1c1 fix(audiobook): warm the model before span cast/verify; keep tests off the real LLM
A cold-booting model server would otherwise time out the first passage into tag guessing.
Documents that the custom attribution prompt applies to the classic engine only.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-19 21:08:01 +02:00
ded70c3853 docs(audiobook): record the rejected scene-cast experiment
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-19 20:54:22 +02:00
f34266c0d2 feat(audiobook): calibrate span-cast confidence on pass agreement
Pooled over three hand-corrected novels (~54k spoken words): agreed 82% right, disputed 48%,
Unknown ~0%, unverified ~76%. The old heuristic bands claimed 16% for lines that were 62% right
and tag/role signals did not discriminate in span casts; classic casts keep the old model.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-19 20:41:25 +02:00
37e2622d4e feat(audiobook): advisory verify and span-native recast for span casts
Verify / recast now run span-natively (no text re-emission): a second differently-windowed
pass marks lines agreed/disputed and records the other answer, without changing any speaker
(~9 min vs ~49 min). Measured on the real pipeline, overruling disputed lines with a reasoning
adjudication made them worse (66.5% -> 61.4%), so that step was removed; agreed lines were 83%
right vs 48.5% for disputed ones, which is the signal used for confidence and review.
Auto-repair no longer runs after a span cast; segments carry a spanId.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-19 20:17:38 +02:00
5877aa2edc fix(characters): persist base-appearance image pairs, deterministic prompts, form-aware review; add forms panel
- fix: base-appearance matched pair lost its concept image + metadata on reload; visual save
  wrote base concept art to the record root; one save path for base and forms now
- fix: extraction no longer lets the LLM author image_prompt (source of avatar/concept drift);
  avatar-only generation uses the deterministic prompt and flags an existing pair
- fix: prompt writes and Physical/Clothing edits go to the active form; review endpoint checks
  each form's own pair; fingerprint parity with the browser incl. visual_* overrides
- feat: appearance forms panel (add/rename/delete/show, per-form stale flag, generate one/all)

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-19 15:27:21 +02:00
e7cb92c572 feat(audiobook): span-mode casting - deterministic quote spans, LLM only names speakers
Split the book into narration/dialogue spans from its own quotation marks and ask
the LLM only who speaks each dialogue span (POST /api/attribute-spans), so text can
no longer be fused, dropped or duplicated and a novel takes ~10 min instead of ~4 h.
Book 1 vs the hand-corrected reference: every-word-right 87.0% -> 92.3%, type
agreement 93.6% -> 97.4%; Books 2/3 (not tuned on) 90.8% / 89.9%.

- static/js/audiobook-spans.js (pure, unit-tested) + audiobook-cast-spans.js
- core/attribution_spans.py, route, tests; scripts/score_attribution.py
- default on; localStorage ab-span-mode=0 restores the classic engine
- architecture/audiobook-attribution.md, Manual, CHANGELOG 1.32.0, TODO

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-19 08:30:26 +02:00
1a2ea421c4 fix(engines): resolve LiteLLM card's own key server-side instead of a stale global one
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
Connect and "Use as LLM" on the LiteLLM Gateway card now resolve that
card's own saved key (engine_api_keys[name]) server-side in
routes/docker.py, ahead of the old global llm_api_key fallback. The
browser can no longer reliably forward a saved key at all -- it's
masked to "__configured__" in every /api/settings response the moment
it's saved -- so relying on a frontend-forwarded header alone left
Connect silently using a stale global key after LiteLLM's own key was
rotated, surfacing a misleading /api/version 404 instead of the real
401.

"Use as LLM" now goes through a dedicated /use-as-llm endpoint that
saves the exact credential Connect just verified (llm_url normalized
to .../v1, llm_api_key copied from the card), rather than a
separately-typed value that could drift again.

Also documents per-user provider-key storage as an explicit item for
the deferred multi-user/identity planning pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-14 13:58:54 +02:00