Commit Graph

450 Commits

Author SHA1 Message Date
a280ddacde fix(characters): do not flag a character reusing its own picture across books
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-19 22:31:17 +02:00
4ab345d1c1 fix(audiobook): warm the model before span cast/verify; keep tests off the real LLM
A cold-booting model server would otherwise time out the first passage into tag guessing.
Documents that the custom attribution prompt applies to the classic engine only.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-19 21:08:01 +02:00
75e452fdfc fix(audiobook): route the Verify menu action to the advisory second pass
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-19 20:55:39 +02:00
ded70c3853 docs(audiobook): record the rejected scene-cast experiment
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-19 20:54:22 +02:00
f34266c0d2 feat(audiobook): calibrate span-cast confidence on pass agreement
Pooled over three hand-corrected novels (~54k spoken words): agreed 82% right, disputed 48%,
Unknown ~0%, unverified ~76%. The old heuristic bands claimed 16% for lines that were 62% right
and tag/role signals did not discriminate in span casts; classic casts keep the old model.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-19 20:41:25 +02:00
37e2622d4e feat(audiobook): advisory verify and span-native recast for span casts
Verify / recast now run span-natively (no text re-emission): a second differently-windowed
pass marks lines agreed/disputed and records the other answer, without changing any speaker
(~9 min vs ~49 min). Measured on the real pipeline, overruling disputed lines with a reasoning
adjudication made them worse (66.5% -> 61.4%), so that step was removed; agreed lines were 83%
right vs 48.5% for disputed ones, which is the signal used for confidence and review.
Auto-repair no longer runs after a span cast; segments carry a spanId.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-19 20:17:38 +02:00
a920f67008 test(characters): mock persistence in the forms spec so it cannot write to the real library
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-19 15:29:34 +02:00
5877aa2edc fix(characters): persist base-appearance image pairs, deterministic prompts, form-aware review; add forms panel
- fix: base-appearance matched pair lost its concept image + metadata on reload; visual save
  wrote base concept art to the record root; one save path for base and forms now
- fix: extraction no longer lets the LLM author image_prompt (source of avatar/concept drift);
  avatar-only generation uses the deterministic prompt and flags an existing pair
- fix: prompt writes and Physical/Clothing edits go to the active form; review endpoint checks
  each form's own pair; fingerprint parity with the browser incl. visual_* overrides
- feat: appearance forms panel (add/rename/delete/show, per-form stale flag, generate one/all)

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-19 15:27:21 +02:00
e7cb92c572 feat(audiobook): span-mode casting - deterministic quote spans, LLM only names speakers
Split the book into narration/dialogue spans from its own quotation marks and ask
the LLM only who speaks each dialogue span (POST /api/attribute-spans), so text can
no longer be fused, dropped or duplicated and a novel takes ~10 min instead of ~4 h.
Book 1 vs the hand-corrected reference: every-word-right 87.0% -> 92.3%, type
agreement 93.6% -> 97.4%; Books 2/3 (not tuned on) 90.8% / 89.9%.

- static/js/audiobook-spans.js (pure, unit-tested) + audiobook-cast-spans.js
- core/attribution_spans.py, route, tests; scripts/score_attribution.py
- default on; localStorage ab-span-mode=0 restores the classic engine
- architecture/audiobook-attribution.md, Manual, CHANGELOG 1.32.0, TODO

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-19 08:30:26 +02:00
e3d6cb2cca feat(magazine): fix Edit boxes, add spread view, searchable narrator picker and Prepare audio
- fix: Edit boxes and page-nav controls were dead clicks (handlers never exported
  to root); resize from any edge/corner instead of only bottom-right
- feat: two-page spread in review (page images above, OCR lists beneath)
- feat: searchable voice picker for narrator and podcast hosts
- feat: Prepare audio synthesises the whole article ahead of playback; playback
  also stays three sentences ahead
- fix: sync index.html cache-busting version with VERSION (was stuck at 1.26.12)

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-18 23:10:16 +02:00
2dd5cea7b7 feat(magazine): let review screen boxes be moved, resized, deleted, and re-OCRed in place
Some checks failed
Test and SonarQube / quality (push) Has been cancelled
Adds "Edit boxes" mode to article review: drag an already-detected
region to move it, its corner handle to resize it, or press its x to
delete it -- for correcting a box the layout engine got wrong, not one
it missed entirely ("Draw it" still covers that case). Moving or
resizing opens a confirm panel; "Save" keeps the region's existing
text at its new position, and for text regions "Save & re-OCR" reads
the corrected area again and replaces the text.

New core/magazine.py update_region()/delete_region() and PATCH/DELETE
/api/magazine/{doc}/pages/{page}/regions/{block_id}. An image region
is re-cropped from the page render whenever its bbox changes, since
its saved file is exactly that crop.

Live-verified against the real ingested c't KI Wissen 2026 magazine:
add -> move -> move+re-OCR (correctly kept old text when the corrected
area had nothing readable) -> delete -> confirmed gone, repeat delete
404s.

275 tests passing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-15 12:40:07 +02:00
53624d470b fix(docker): remove undefined-variable reconnect that broke all container status/start/stop/restart
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
_docker_get_json/_docker_post called sock.connect(sock_path) a second
time after _docker_connection() had already connected the socket --
sock_path was never defined in either function, so every call raised
NameError. Surfaced in the UI as every engine card reporting "Not
installed" and the Active OCR Engine model dropdown reporting "Not
Found" regardless of what was actually running, found while diagnosing
a user report of that dropdown. Docker socket API calls (list/status/
start/stop/restart) were affected for every engine card, not just OCR.

Added tests/test_docker_client.py, which did not exist before, to
catch this class of regression going forward.

Also records a request to make the magazine ingestion "watching it
work" panel draw detected region boxes live over the page image as
each is classified, instead of today's static page-image-plus-text-
list (TODO.md; no code change for that yet).

254 tests passing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-14 18:14:22 +02:00
531cb50cae docs: list docling, PaddleOCR, DeepSeek-OCR, Qwen3-VL-30B-A3B for OCR-engine evaluation
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
User-requested candidates to install as Docker containers alongside the
existing engines and add through the Engines tab, to be benchmarked
against the two already wired (baidu/Unlimited-OCR, KDL-Frontier-Parser-
nano) for both page-text transcription and open-ended figure/chart
description. No code change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-14 17:57:31 +02:00
c81ae28da2 feat(magazine): add figure/chart/table description generation and spoken narration
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
Adds a "Describe" action per figure in article review: sends the cropped
image to the configured vision engine (with fallback) for a type-aware
one- or two-sentence description -- a chart gets a prompt about what it
measures and its trend, a table about headings and what stands out, etc.
Editable before approval. A new "Read figure descriptions" toggle in the
reading controls speaks these where they occur; purely additive, off by
default, and a figure with no description behaves exactly as before.

New core/magazine.py describe_visual_region()/_chain() reuse the same
reliability guards as page OCR (repetition-loop detection, sentence-
boundary truncation). New routes: POST .../images/{id}/describe
(background job), PATCH .../images/{id} (manual edit), descriptions=1 on
the article-text endpoint.

Live-verified end to end against the ingested c't KI Wissen 2026
magazine. Also found and documented a real, config-dependent limitation:
the currently-configured primary vision engine is transcription-only and
returns empty content for any descriptive prompt (confirmed with three
different prompts) -- not a code defect, but description generation
needs a genuinely caption-capable engine reachable as the fallback.

251 tests passing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-14 17:39:57 +02:00
1a2ea421c4 fix(engines): resolve LiteLLM card's own key server-side instead of a stale global one
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
Connect and "Use as LLM" on the LiteLLM Gateway card now resolve that
card's own saved key (engine_api_keys[name]) server-side in
routes/docker.py, ahead of the old global llm_api_key fallback. The
browser can no longer reliably forward a saved key at all -- it's
masked to "__configured__" in every /api/settings response the moment
it's saved -- so relying on a frontend-forwarded header alone left
Connect silently using a stale global key after LiteLLM's own key was
rotated, surfacing a misleading /api/version 404 instead of the real
401.

"Use as LLM" now goes through a dedicated /use-as-llm endpoint that
saves the exact credential Connect just verified (llm_url normalized
to .../v1, llm_api_key copied from the card), rather than a
separately-typed value that could drift again.

Also documents per-user provider-key storage as an explicit item for
the deferred multi-user/identity planning pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-14 13:58:54 +02:00
4c6f41c831 docs: record work-order priority (Magazine, Character, Audiobook)
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
No code change. User set explicit priority for upcoming feature work:
Magazine reader roadmap first, then the character-identity system,
then audiobook speaker attribution. Recorded in TODO.md so it survives
a context reset. Deployment and the deferred multi-user initiative
stay explicitly parked, not part of this order.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-14 12:58:01 +02:00
4249382ac6 feat(deploy): stand up correspondence-nuc as a second instance
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
The NFS blocker from TODO.md's deployment entry is resolved (export
set up by the user on the DGX); picks up from there and completes the
deployment end to end.

- Mounted the DGX's TTS_Voices export read-write on the NUC at
  /mnt/dgx-tts-voices, persisted in /etc/fstab -- confirmed writable
  and confirmed the NUC instance sees the DGX's real voice library
  (344 voices), not a separate copy.
- Deployed by rsyncing the working tree from the DGX (excluding .git,
  config/, logs/ and other local-only paths) and building the image
  directly on the NUC, rather than pulling ghcr.io/martin-b78/... --
  that registry copy turned out to be both several versions stale and
  private with no credentials configured on the NUC, and the source
  was already right there to build from.
- Found and fixed a real gap while wiring it up: vision_ocr_url and
  vision_ocr_fallback_url default to host.docker.internal, which
  resolves to whichever host the container is running on -- correct
  on the DGX itself, silently wrong on the NUC (would have pointed at
  the NUC's own nonexistent OCR engine). Pinned both explicitly to the
  DGX's LAN IP via environment overrides.
- The container crash-looped on first start (sqlite3.OperationalError:
  unable to open database file) because docker compose had created
  config/ and logs/ root-owned under this session's root SSH access to
  the NUC, while the image's app user is uid 1000 -- chown -R 1000:1000
  fixed it.
- New docker-compose.nuc.yml, tracked here as the reviewable reference
  for that host (docker-compose.yml stays the DGX's own -- different
  build/network/socket-proxy shape). Deployed to the NUC as its own
  docker-compose.yml, since that host has no .git checkout to diff
  against, only the rsynced tree.

Verified: container stable (no restart loop), HTTP 200, settings
confirm all four engine URLs correctly resolve to the DGX, voice count
matches the DGX exactly, restart policy is unless-stopped.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-14 10:06:55 +02:00
c91aaabe3a fix(mobile): retry failed script loads; fix 8 phone-width layout bugs
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
Two reports from the same mobile session tied together: a voice-library
crash ("fmtDbfs is not defined") and a save button reporting "No audio
ready" despite a file being correctly loaded and trimmed. Root cause
was the same in both -- the module loader fetched each JS file once
with no retry, so a single dropped request on a flaky connection left
that module's functions/variables undefined while every other module
kept executing anyway, producing confusing symptoms nowhere near the
real cause.

loader.js: _load() now retries up to 3 times with backoff before
rejecting. If a module still can't load after that, the app says so
plainly via a toast (or a plain DOM banner if even utils.js failed to
load) instead of silently revealing a partially-broken page. Wired into
both the core-module fallback path and ensureFeatureModules' lazy path.

Loading every section of the app at a 375px viewport and checking for
elements running past the viewport edge (all now 0) found eight real,
independent layout bugs, each with a distinct root cause:

- .section-head's mobile wrap rule targeted .section-header, a class
  nothing in the app uses -- dead since it was written. Every section's
  icon/title/action row (e.g. "+ New voice") ran off-screen unreachable.
- The synth-mode toggle and Clone-a-Voice source tabs' icon-only mobile
  rules matched a <span> wrapper the markup didn't have. Added it, plus
  aria-labels so hiding the text doesn't erase the accessible name.
- voice-design.css loads after style.css and re-declares
  .design-sample-grid unscoped -- style.css's mobile override always
  lost the cascade regardless of viewport. Moved the override to the
  file that actually wins.
- Three separate minmax(380-440px,1fr) engine-card grids couldn't fit
  one column under ~460px. Collapsed to 1fr with min-width:0 on the
  items -- the column fix alone wasn't enough, since a grid item's
  content-based min-width is still a floor even inside a 1fr track.
- The routing test row (450px+ minimum) and the conversation input bar
  (a single non-wrapping flex row for text field + mic + gate + three
  toggles) now wrap under 767px.
- Try It Out's EQ sliders shrank to fit beside their toggle instead of
  ever wrapping below it, overflowing their own squeezed container.
  Forced onto their own full-width line.

Verified with a Playwright sweep of all 16 sections (0 overflowing
elements, 0 console errors) plus the existing 225-test suite and the
frontend syntax gate.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-13 17:09:24 +02:00
a78dac98fa fix(magazine): catch vision OCR decoding loops before they become page text
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
Confirmed live: re-ingesting c't KI Wissen 2026 with a local Qwen VL
engine, page 18's chart transcription degenerated into "das Produkt,
das Produkt, das Produkt, ..." repeated hundreds of times, filling the
whole response budget -- and was accepted as real region text, which
would have been narrated verbatim.

The codebase already had two narrow signatures for specific observed
failures (a formatting-instruction echo, a fabricated LaTeX loop), both
scoped to the plain-text fallback path only. Added a general detector
(_looks_like_repetition_loop): any short phrase containing a real word,
repeated 8+ times immediately back-to-back. Deliberately excludes
patterns with no letters -- a dot-leader contents line, a repeated
number sequence -- which are legitimate and must not be flagged.

Applied at both call sites: vision_ocr_page (the labelled-region path
most engines use) and _whole_page_region (the plain-text fallback).
Treated as a failed call in both, so vision_ocr_page_chain's existing
fallback (secondary engine, then tesseract) gets a real chance instead
of garbage being accepted as the page's transcription.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-13 10:09:32 +02:00
b458306ab9 fix(magazine): recover sparse charts, stabilize adjacent-figure snapping
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
Real failures found by re-inspecting a fresh re-ingest of c't KI Wissen
2026 after the previous figure-boundary-snapping release: a bar chart
was still truncated to roughly a third of its real height, and a
chart's top edge cut into an unrelated paragraph above it.

Root causes, each independently confirmed against the real page render
before fixing:

- A bar chart's bars/gridlines/axis labels rarely reach the "solid
  ink" density a photo does (measured 0.25-0.5 vs the 0.55 threshold),
  so the existing pass anchored only on the one fully-solid element
  (the zero-line) and never grew further. Added a second, looser pass
  that grows an anchor -- or the seed itself -- through any real mark
  at all, fenced by the same blocked cells.

- Anchor selection ("which solid component does the seed overlap
  most") was seed-size-dependent: a small seed touching only the
  zero-line anchored there, but re-running the SAME function on the
  box it had just produced -- now large enough to also touch a dense
  axis-label strip -- picked that instead and shrank an already-good
  box back down. Fixed by dilating the solid mask and labelling once,
  BEFORE any seed is considered, so which components count as one
  figure is a property of the page alone.

- A merged bounding box can enclose real text sitting between two
  components even though neither component's own cells touched it (a
  rectangle union, not the components' actual shape). Added
  _trim_blocked_edges() to pull each edge back to the nearest text
  wall found searching outward from the box's own centre.

- Two adjacent figures fencing each other's growth made each one's
  result depend on the other's CURRENT box -- for two dark photos
  sharing a caption strip, this never settled, swinging by over 100pt
  call after call. snap_visual_bboxes() now re-seeds every visual on a
  page from every other's last result for a bounded number of rounds,
  deciding same-graphic ambiguity once up front (fenced against the
  original seeds, before any iteration) so two disputed seeds can't
  mutually fence each other into a tidy-looking split instead of being
  recognised as the same disputed claim.

- A result is now only accepted once it reproduces itself when fed
  back through the same rules (IoU-checked), and the minimum
  seed/graphic overlap for repositioning a badly-offset box was
  lowered slightly -- it was narrowly rejecting some real cases.

Verified against the full 204-page/147-figure document: idempotency
holds except for one confirmed non-convergent pair, which now lands on
the same deterministic answer every run instead of continuing to
drift.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-12 21:21:04 +02:00
96b0b9dd71 fix(magazine): finish and harden the live ingest preview
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
Resumes an in-progress, uncommitted change from a previous session: the
live "page image + recognised text" preview shown during ingestion was
reworked to render formatted HTML (styled headings, a figure
placeholder) instead of a plain-text dump, but was left with a
duplication bug, dead code, inline styles, and no tests.

- Fixed duplication: a figure block's own transcribed text (usually its
  on-screen UI copy) was rendered as an ordinary paragraph in addition
  to the new figure placeholder box. Figure/chart/table/code blocks are
  now represented only by the placeholder.
- Furniture/sidebar/pullquote/ad text is shown muted instead of being
  dropped with no replacement, and is never promoted to a heading
  regardless of font size.
- Factored the formatting logic out of the progress closure into a
  standalone, unit-tested _render_ingest_preview_html().
- Replaced dead getattr(bbox, "y0", ...) calls (bbox is always a plain
  [x, y, w, h] list) with direct indexing, moved statistics/logging
  imports to module level, switched manual character replacement to
  html.escape(), and bounded the preview by an accumulated line budget
  instead of a blind slice that could truncate mid-tag.
- Replaced inline style="..." with CSS classes matching the panel's
  existing dark console treatment.
- The frontend now always uses innerHTML (every text node is escaped
  server-side), dropping the startsWith('<') content-type sniffing.
- Marked the preview panel aria-hidden -- decorative, and would
  otherwise churn noisily for screen reader users on top of the
  adjacent progress bar.

Verified against the real c't KI Wissen 2026 layout: a figure's raw
OCR text no longer appears twice, a real headline renders as <h3>, a
footer/pull-quote render muted, and all text is correctly escaped.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-12 15:24:16 +02:00
8e74060394 docs: scope magazine contents-view redesign and legal source browsing
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
No code change. Added to TODO.md:

- Magazine contents view (#s-magazine): logged three candidate layouts
  as open options rather than picking one -- three-column page/text/
  figures, synthesis-state-and-inline-play (the recommended quick win
  under any layout), and a flipbook with content columns underneath.
  Needs a decision before implementation.

- Browsing free PDF magazine sources: the two sites given as examples
  (freemagazines.top, pdf-magazines.org) redistribute current commercial
  titles without licence, so no scraper was built against them. Scoped
  instead against OPDS catalogues, archive.org's public-domain
  periodicals, publisher-run free issues, and open-access journals.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-12 10:25:12 +02:00
55f147d7a7 fix(magazine): derive figure boundaries from the page render
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
A vision OCR engine grounds a figure far less precisely than it
transcribes one. Over all 138 figure regions of a 204-page issue the
transcribed contents were right while the boxes were routinely offset,
undersized, or sitting on the neighbouring text column -- page 23's card
grid was reported at x 0.24-0.54 where the graphic runs 0.41-0.95.

snap_visual_bboxes() reduces the page render to an ink-coverage grid,
seeds a connected-component search with the engine's box, and re-joins
tiles a gutter apart so a card grid comes back as one figure. Text
blocks are excluded from the mask, which is what stops a figure
swallowing the column beside it. A candidate must be solidly inked or
framed on nearly all of its rows and columns (admitting a white-interior
screenshot), may not shrink below 30% of the seed, and reverts when two
figures land on the same graphic; anything else keeps the box it had.

Runs before the figure crop in both OCR paths, so the images shown
during playback are framed correctly too. Replaces refine_picture_
bounds(), which only recognised coloured panels and had no text fence.

Also adds "Fix figure boundaries" in review and POST
/api/magazine/{id}/figures/refine behind it, so an issue ingested before
this pass can be corrected without re-running OCR. Verified live against
c't KI Wissen 2026: 78 of 138 boxes corrected, the rest declined.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 10:17:00 +02:00
a4b7c18a2f Fix magazine dropzone upload button and finalize side-by-side layout 2026-09-10 17:44:37 +02:00
c1a2d6dc74 Release v1.26.14: Studio-style side-by-side Magazine upload layout with Durchsuchen button
Some checks failed
Test and SonarQube / quality (push) Has been cancelled
2026-09-10 14:45:00 +02:00
30e01a5cec Release v1.26.13: Fix drag-and-drop file opening in tab & improve magazine dropzone reliability 2026-09-10 14:31:16 +02:00
11142842c2 Release v1.26.12: Fix page-wide drag-and-drop interception & magOnShow navigation hook
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
2026-09-10 13:22:26 +02:00
f5f887bb1d Release v1.26.11: Enhanced layout-aware OCR element recognition, grounding parsing, and preview thumbnails
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
2026-09-10 13:08:53 +02:00
29dc913ed1 Release v1.26.10
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
2026-09-10 12:50:33 +02:00
e6211f1333 Release v1.26.9
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
2026-09-10 12:45:42 +02:00
9b1e29bb79 feat: automate semantic magazine layout
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
2026-09-09 17:48:56 +02:00
f8af18e8f7 fix: release current magazine UI assets
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
2026-09-09 17:40:25 +02:00
31b13a4e16 feat: improve magazine review navigation
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
2026-09-09 17:36:27 +02:00
c1d86ec13d fix: return All magazines to library
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
2026-09-09 17:14:27 +02:00
272d4872bd fix: use native Unlimited-OCR layout regions
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
2026-09-09 13:27:14 +02:00
b43bacc275 chore: add local maintenance utilities
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
2026-09-09 07:53:13 +02:00
037f3dd74d fix: stabilize OCR and character sheet casting
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
2026-09-09 07:51:59 +02:00
ff827a74bd fix: recover character sheet requests from gateway 405
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
2026-09-09 06:49:52 +02:00
b7aade39db Release v1.26.1
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
2026-09-08 23:39:44 +02:00
c7b473910a docs: record public multi-user readiness as a deferred cross-app initiative
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
Three research passes into what multi-user hosting would require found the
app has zero multi-tenancy today: one global settings/database/voice
library, no browser-UI authentication at all, global (not per-user)
concurrency limits, and no consent/audit mechanism for voice cloning.

The user wants a single shared, interconnected login/profile system across
all their self-hosted apps rather than a per-app login, extending the
GoTrue+Postgres+PostgREST stack OurFlashcards already runs on the NUC — so
this is recorded as a deferred, cross-repo initiative rather than
implemented now. One policy decision was made and is captured for future
implementation: voice cloning will require a consent attestation with an
audit trail once accounts exist.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-07 22:33:01 +02:00
948c8a453e docs: track NUC deployment as a deferred TODO, blocked on DGX sudo access
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
Recorded the exact NFS setup needed on the DGX (export the voice
library read-write, scoped to the NUC's IP) so the app's voice list
never drifts from what the DGX's TTS engine actually has on disk, plus
the accepted cross-host limitation on voice-change auto-restart.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-07 21:38:17 +02:00
1a06a8bf8c Release v1.26.0
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
2026-09-07 19:05:10 +02:00
85080f4b82 feat: add live ingest preview, region icons/ownership badges, manual region drawing to magazine review
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
Ingestion now streams each page's image and recognised text into a
console-styled preview panel (Baidu Unlimited-OCR demo style) as pages
finish, instead of a bare progress bar -- raw text is written to a
per-job ingest_preview.json file since the background-job result field
strips text-like keys for data minimization.

Split-screen review boxes and their matching list rows now show a
centred icon for the region's kind and, when a region on a shared page
belongs to a different article, a badge naming that article, so a
claimed region is never mistaken for unclaimed text.

Added a "Draw it" mode: click-drag a box directly on the page image to
add a region the layout engine missed, as typed/OCRed text or as a new
figure (add_manual_region, POST /api/magazine/{doc_id}/pages/{page}/regions).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-07 18:52:21 +02:00
5adba6b4c0 feat: split-screen article review with numbered regions, per-page/whole-magazine "OCR again", and a real column-interleaving fix it found
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
Split-screen review: the rendered page (left) with a numbered box per
detected block, the extracted text (right) with matching numbers.
Clicking either side highlights the other. Checkboxes moved from a flat
list into this layout without changing their semantics (still gate
what gets narrated).

"OCR again": reocr_page() forces one page through the configured
vision engine again, replacing only that page's blocks -- deliberately
does NOT call build_articles(), so an already-approved article is not
silently re-grouped. A magazine-level option re-reads every page and
re-runs detection (with a confirmation, since that does reset article
status). extract_pages() gained force_vision_ocr to skip its quality
gate for this case.

Building the numbered-box review immediately did what it was built to
do -- surfaced a real bug on live production data (real magazine, real
scan, not a synthetic fixture): a photo caption at 49.9% of the page
width sat in the gutter between two genuinely separate columns and
merged them into one, so every block on the page came back column=0
and reading order fell back to pure top-to-bottom, interleaving the
two columns' sentences. Fixed two compounding causes: column boundary
detection now excludes blocks much wider than the page's median width,
and detect_columns' gutter threshold (3.5%) was itself narrower than
some real magazine gutters (~2.1% measured) -- lowered to 1.5%.

10 new/updated tests, including one built directly from the real
measured page-17 block coordinates that exposed the bug.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-07 14:00:34 +02:00
e81fa64448 fix: magazine state persistence, misfiled in book library, confidence badge layout, docker-socket-proxy crash loop
Magazine "forgets" the open document (real bug, confirmed live):
- A full page reload or the browser's own Back button tears down all
  in-memory JS state; magInit() always defaulted to the empty library
  view with no way back to what was open. Persists the last-opened
  magazine id in localStorage and auto-reopens it on load, falling back
  to the library only if that document no longer exists.
- The actual root cause of the reported duplicate: readerRenderLibrary()
  (Studio/Reader "Source") and libraryRenderBooks() (Library > Books)
  both listed EVERY document from the shared reader_library store with
  no kind filter, so a magazine (kind: "magazine") was already showing
  up there disguised as a plain book. The magazine never actually
  vanished -- it was misfiled in a different list, which is what led to
  the user re-uploading it and getting a real duplicate under a
  different kind. Both listings now exclude kind: "magazine".

Confidence badge layout (Studio Characters review):
- The badge (radiobox is already positioned after the speaker name in
  DOM order) rendered on its own line far below the name because both
  .ab-cv-spk and .ab-cv-txt were `display: block`, forcing a line break
  regardless of markup order. Wrapped the name + badge + check button in
  a shared flex row (.ab-cv-spk-row) so the badge now sits directly
  beside the name/after "NARRATOR" as requested, without needing to
  touch the two existing incremental-patch code paths (they insert
  relative to the same elements, now just nested one level deeper).

docker-socket-proxy crash loop (found while restarting for the magazine
Python changes to take effect -- unrelated to this session's earlier
work, landed in an earlier commit during the interruption):
- read_only: true left the entrypoint with nowhere to write its own
  haproxy.cfg ("Read-only file system"), crash-looping the container and
  leaving the app without Docker access. Removed read_only for this one
  service -- cap_drop: ALL, no-new-privileges, and the :ro socket mount
  already provide the real hardening.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-07 09:39:20 +02:00
9942f920bc docs: changelog entry for the dropzone fix
Some checks are pending
Test and SonarQube / quality (push) Waiting to run
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-07 00:12:09 +02:00
3c210df8a4 fix: magazine dropzone opened the file in the browser instead of accepting the drop
Two real bugs, both confirmed live:

1. dragover's preventDefault() was gated on dataTransfer.types including
   'Files' -- during a real OS file drag, Chromium does not reliably
   populate that list synchronously on every dragover event, so the
   condition failed often enough that preventDefault was silently
   skipped. Per the HTML5 DnD spec, an element only becomes a valid drop
   target once dragover (and, in some browsers, dragenter) has been
   cancelled -- without that, the browser falls through to its native
   "open the dropped file" behaviour, which is exactly what was
   reported. Removed the gate entirely; dragenter is now also handled,
   not just dragover.

2. No page-wide safety net existed, so a drop landing even slightly
   outside the dropzone element bubbled to the browser's default action
   uncontested. Added a window-level dragover/drop preventDefault
   installed once regardless of which section is active.

This also explains the "no hover highlight" symptom: the same
gated-preventDefault bug meant dragover never reliably reached the
class-toggle code either.

Verified live: dragenter/dragover/drop each report defaultPrevented,
the highlight class appears on dragenter and clears on drop, the
dropped file populates the hidden input, and a drop dispatched directly
on <body> (simulating one that misses the zone) is also intercepted by
the new window-level guard.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-07 00:11:54 +02:00
fbb0de8e2c docs: changelog entry for the magazine dropzone
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-07 00:03:22 +02:00
5b0c8e0bf5 feat: add drag-and-drop dropzone to the magazine upload form
Replaces the plain <input type="file"> with the same reader-dropzone
component the Book/Source screen already uses (reused CSS, no new
styles needed beyond grid/layout adjustments) -- drag a PDF onto the
zone or click "Choose file", both paths route through the same hidden
input magCreate() already reads via a DataTransfer handoff, so no other
logic needed to change. Shows the chosen/dropped filename in place of
the default hint text, and resets it after a successful add.

Verified live: click-to-choose and a simulated real 'drop' DragEvent
both populate the file input correctly, dragover highlighting toggles,
no console errors from the app itself.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-07 00:03:01 +02:00
813e06c914 feat: complete magazine P1 (images, play-all, karaoke, driving mode, sidebar exclusion) + OCR engines page + second OCR engine
Magazine P1:
- Images shown automatically during playback (assign_images_to_articles
  links each figure to the article whose column it falls in; the client
  shows the last image reached in reading order as playback advances).
- "Play all articles" queues every approved article and advances
  automatically at the end of each rather than stopping.
- Word-level highlighting, explicitly documented as an ESTIMATE (no
  forced-alignment/timestamp source exists): each word's share of the
  sentence's real measured audio duration is proportional to its
  character length. A checkbox turns it off; sentence highlighting
  (measured, not estimated) is unaffected.
- Driving mode: full-screen overlay, two oversized controls, reuses
  whatever is already playing/queued.
- Sidebars/pull-out boxes excluded from narration automatically
  (mark_sidebars: an anomalously narrow column vs. the page's regular
  grid). Fixed a real defect found while building this: a full-width
  headline was left in the column-width analysis and could bridge two
  genuinely separate columns into one, hiding a real sidebar column
  entirely -- caught by a failing test before the fix.

Second OCR engine + fallback ladder:
- KDL Frontier Parser nano wired in via vision_ocr_page_chain (ordered
  engine list, first usable result wins, tesseract last resort).
  Confirmed live: ~3x faster than Baidu (~12s vs ~40s/page) but returns
  plain text with NO coordinate markers, unlike Baidu's labeled regions.
  _whole_page_region treats the whole response as one ambiguous
  full-page block (gated by ocr_quality_score) rather than discarding
  usable text just because it lacks layout structure.

OCR engines page + a real UI bug:
- The OCR & Document Parsing page had no .s-engines-page[data-page="ocr"]
  markup at all -- renderLocalContainers already supported an OCR grid
  generically, but $('dc-grid-ocr') was always null, so the card grid
  silently never rendered. Added an Active OCR Engine panel mirroring
  Active Language Model exactly, plus the Local grid -- both configured
  engines now appear as ready-made cards, no "Add Custom" dialog needed.
- Fixed the audiobook pipeline's "Previous/Next" nav bar bleeding into
  every .section-head in the document (confirmed live on the Engines
  page: a "Cast Audiobook" button and "STEP 1/8" hint that made no sense
  there). Changed from a blocklist (naming each unrelated section as
  noticed) to an allowlist of the pipeline's three actual destinations.

Verified live end-to-end: play-all queue building and manual advance,
word-span markup, driving-mode toggle, OCR page showing both engines as
local cards, zero stray nav bars on the Engines page, no console errors.

20 new backend tests (156 total in the magazine module).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-06 23:39:34 +02:00