feat(magazine): mark hand-edited text, show a word diff against the recognised text, restore it
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
0f4b28ba14
commit
30b80fcbf7
@ -7,6 +7,7 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi
|
||||
|
||||
## [Unreleased]
|
||||
### Added
|
||||
- **Magazine review: extraction diff.** A region whose text you edited by hand is marked *Edited*, with *Show changes* (word-level comparison against the text as first recognised; removals struck through and marked "−", additions underlined and marked "+") and *Restore recognised text*. The first reading is kept in a new `ocr_text` field across repeated edits; re-reading the region or typing the original back clears it.
|
||||
- **"Continued from page N" now joins a feature's tail to its head.** Until now only the forward "continued on page N" reference linked pages. A tail that says "Fortsetzung von Seite 10" / "Continued from page 10" (found even when set as small furniture or sidebar text, on lines up to 160 characters) is attached to the single article on that page. Skipped when a contents entry anchors the tail, when the page is shared by several articles, or when the marker points forward or at an empty page. Articles gain a `continued_from` list.
|
||||
- **Magazine import from PNG/JPEG page images.** Choose or drop several scans or phone photos; they are checked by content (not by name), ordered by filename with natural numbering, turned into one scan-style PDF (page size from the image's own resolution, EXIF rotation applied, 72 dpi treated as unknown) and ingested like any scanned issue. Limits: 300 images, 25 MB and 60 megapixels each; a bad image rejects the whole upload and leaves nothing behind; an existing source is never replaced. Verified on real pages: OCR through the image path scored like the original PDF (F1 0.85 / 0.78).
|
||||
- `architecture/ocr-porting-prompt-general.md`: the OCR porting brief rewritten for general documents (letters, papers, forms, invoices). Adds privacy, pre-processing of skewed/photographed scans, document-type handling, field-level accuracy and handwriting, and tags every claim as measured on magazines, a general principle, or untested for these document types.
|
||||
|
||||
20
Manual.md
20
Manual.md
@ -338,6 +338,26 @@ Open an article and expand **Pronunciation & spelling** under the player.
|
||||
**Mobile / accessibility:** the lists are ordinary buttons with labels ("Replace X with Y"); marks are underlined, not colour-only.
|
||||
**Privacy:** the agent sends only the flagged word and its surrounding sentence to the LLM you configured in Settings; without an LLM nothing leaves the machine.
|
||||
|
||||
### Editing recognised text, and seeing what you changed
|
||||
|
||||
**Purpose.** Fix what OCR got wrong and always be able to see, and undo, what you changed.
|
||||
|
||||
**Steps.** In Review, click a region's text (or its pencil) and press *Save text*. A region you edited shows an
|
||||
**Edited** mark. *Show changes* opens a word-by-word comparison with the text as first recognised: words removed are
|
||||
struck through and marked with a minus, words added are underlined and marked with a plus (never colour alone).
|
||||
*Restore recognised text* puts the original back. Typing the original back yourself also clears the mark.
|
||||
|
||||
**Expected result.** The first recognised text is kept through any number of edits; reading the region again (*Re-OCR*)
|
||||
makes the new reading the original. Edits are reversible with Undo as before.
|
||||
|
||||
**Errors.** Empty text is refused (delete the region instead). Very long texts (over about 1,500 words) are compared as
|
||||
one removed block and one added block rather than word by word.
|
||||
|
||||
**Mobile / accessibility.** The buttons are real buttons with an expanded/collapsed state; the comparison is a labelled
|
||||
group that explains the minus/plus convention to a screen reader.
|
||||
|
||||
**Privacy.** The recognised text stays on the server with the magazine; nothing is sent elsewhere.
|
||||
|
||||
### Find an issue online
|
||||
|
||||
**Purpose.** Pull a magazine issue into the app without leaving for a browser and uploading by hand.
|
||||
|
||||
4
TODO.md
4
TODO.md
@ -688,8 +688,8 @@ casting. The original PDF remains the visual source of truth while approved arti
|
||||
*(sentence-level; clicking a sentence or switching to the original-page view jumps to its source bbox)*
|
||||
- [x] Show article images or page crops during playback, with optional caption reading.
|
||||
*(images shown, matched to reading position; caption text is not separately spoken)*
|
||||
- [~] Add confidence warnings, extraction diffs, and manual text correction before audio generation.
|
||||
*(confidence badges + block-level boundary editing done; no extraction diff view or free-text correction)*
|
||||
- [x] Add confidence warnings, extraction diffs, and manual text correction before audio generation.
|
||||
*(confidence badges, boundary editing, per-region text editing, and (2026-09-29) an Edited mark with a word diff against the recognised text and one-click restore)*
|
||||
- [ ] **Redesign the magazine contents view** (`#s-magazine`, the article list for one issue). Today it is a flat
|
||||
list of title + page range + badge + Review button, which shows nothing of the issue itself: no page, no
|
||||
opening text, no figure, no sign of which articles already have audio. Requested directions, none decided:
|
||||
|
||||
@ -126,6 +126,9 @@ class Block:
|
||||
# (clean_up_regions). Part of the model so every path that rebuilds blocks keeps it -- and does
|
||||
# not crash on it (drawing a region did, "unexpected keyword argument 'original_text'").
|
||||
original_text: str = ""
|
||||
# The text as recognised, kept when the reader first edits it by hand (set_region_text), so the
|
||||
# review can show what changed and put it back. Empty = never edited.
|
||||
ocr_text: str = ""
|
||||
# box_manual: the reviewer placed or resized this box, so snap_text_boxes never moves it.
|
||||
# box_snapped: snap_text_boxes fitted it to its text. Both are model fields for the same
|
||||
# reason as original_text: every path that rebuilds blocks with Block(**b) must accept them.
|
||||
@ -134,7 +137,7 @@ class Block:
|
||||
|
||||
def to_json(self) -> dict[str, Any]:
|
||||
data = asdict(self)
|
||||
for optional in ("original_text", "box_manual", "box_snapped"):
|
||||
for optional in ("original_text", "ocr_text", "box_manual", "box_snapped"):
|
||||
if not data[optional]:
|
||||
del data[optional]
|
||||
return data
|
||||
@ -4484,6 +4487,7 @@ def update_region(doc_dir: Path, layout: dict[str, Any], page_no: int, block_id:
|
||||
if same_top:
|
||||
new_text = prefer_drop_cap_repair(new_text, old_text)
|
||||
block["text"] = new_text
|
||||
block.pop("ocr_text", None) # freshly read: the new reading is the original
|
||||
blocks[block_idx] = block
|
||||
layout["blocks"] = blocks
|
||||
return layout, new_text
|
||||
@ -4774,8 +4778,18 @@ def set_region_text(layout: dict[str, Any], page_no: int, region_id: str, text:
|
||||
if not any(b["id"] == region_id and b["page"] == page_no for b in layout.get("blocks", [])):
|
||||
raise ValueError("Region not found on this page")
|
||||
layout = dict(layout)
|
||||
layout["blocks"] = [dict(b, text=text, confidence=1.0) if b["id"] == region_id else b
|
||||
for b in layout.get("blocks", [])]
|
||||
|
||||
def edited(b: dict[str, Any]) -> dict[str, Any]:
|
||||
before = str(b.get("text") or "")
|
||||
first_read = b.get("ocr_text") or before
|
||||
out = dict(b, text=text, confidence=1.0)
|
||||
if text == first_read:
|
||||
out.pop("ocr_text", None) # typed back to the original: no longer an edit
|
||||
else:
|
||||
out["ocr_text"] = first_read
|
||||
return out
|
||||
|
||||
layout["blocks"] = [edited(b) if b["id"] == region_id else b for b in layout.get("blocks", [])]
|
||||
return layout
|
||||
|
||||
|
||||
|
||||
@ -1719,7 +1719,8 @@ function magReviewPageColumn(page, wrapId, interactive) {
|
||||
(interactive ? ' onclick="event.preventDefault();event.stopPropagation();magRowEdit(\'' + b.id + '\')" title="Click to edit"' : '') + '>' +
|
||||
(owner ? '<span class="mag-block-owner">Article ' + owner.index + ': ' + esc(owner.title) + '</span> ' : '') +
|
||||
(b.continues_from_previous_page ? '<span class="mag-block-owner">Continued from previous page</span> ' : '') +
|
||||
'<span data-text-of="' + esc(b.id) + '">' + magSpellMarked(b.id, b.text) + '</span></span>' +
|
||||
'<span data-text-of="' + esc(b.id) + '">' + magSpellMarked(b.id, b.text) + '</span>' +
|
||||
(b.ocr_text ? magEditedChip(b.id, interactive) : '') + '</span>' +
|
||||
(interactive ? magRowActions(b.id, i + 1, i + 1 < regions.length) : '') + '</label>';
|
||||
});
|
||||
if (!regions.length) listHtml += '<p class="muted">Nothing detected on this page.</p>';
|
||||
@ -2755,6 +2756,80 @@ async function magReloadReviewNow(message) {
|
||||
if (message) say(message, 'ok');
|
||||
}
|
||||
|
||||
// Word-level difference between the recognised text and the reader's edit. Whitespace-separated words,
|
||||
// longest-common-subsequence; returns escaped HTML with <del> (only in the original) and <ins> (only in
|
||||
// the edit). Bounded: two very long texts are shown whole rather than freezing the page.
|
||||
const MAG_DIFF_MAX_WORDS = 1500;
|
||||
function magWordDiff(before, after) {
|
||||
const a = String(before || '').split(/\s+/).filter(Boolean);
|
||||
const b = String(after || '').split(/\s+/).filter(Boolean);
|
||||
if (a.length > MAG_DIFF_MAX_WORDS || b.length > MAG_DIFF_MAX_WORDS) {
|
||||
return '<del>' + esc(a.join(' ')) + '</del> <ins>' + esc(b.join(' ')) + '</ins>';
|
||||
}
|
||||
const n = a.length, m = b.length, w = m + 1;
|
||||
const t = new Uint16Array((n + 1) * w);
|
||||
for (let i = n - 1; i >= 0; i--) {
|
||||
for (let j = m - 1; j >= 0; j--) {
|
||||
t[i * w + j] = a[i] === b[j] ? t[(i + 1) * w + j + 1] + 1 : Math.max(t[(i + 1) * w + j], t[i * w + j + 1]);
|
||||
}
|
||||
}
|
||||
const out = [];
|
||||
let i = 0, j = 0;
|
||||
const push = (kind, word) => {
|
||||
const last = out[out.length - 1];
|
||||
if (last && last.kind === kind) last.words.push(word); else out.push({ kind, words: [word] });
|
||||
};
|
||||
while (i < n && j < m) {
|
||||
if (a[i] === b[j]) { push('same', a[i]); i++; j++; }
|
||||
else if (t[(i + 1) * w + j] >= t[i * w + j + 1]) { push('del', a[i]); i++; }
|
||||
else { push('ins', b[j]); j++; }
|
||||
}
|
||||
while (i < n) push('del', a[i++]);
|
||||
while (j < m) push('ins', b[j++]);
|
||||
return out.map((g) => {
|
||||
const text = esc(g.words.join(' '));
|
||||
return g.kind === 'del' ? '<del>' + text + '</del>' : g.kind === 'ins' ? '<ins>' + text + '</ins>' : text;
|
||||
}).join(' ');
|
||||
}
|
||||
|
||||
// Marks a region the reader edited by hand, with a way to see the change and to put the recognised text back.
|
||||
// Inside a <label> row, so every click must stop the tick box from toggling.
|
||||
function magEditedChip(id, interactive) {
|
||||
const stop = 'event.preventDefault();event.stopPropagation();';
|
||||
return ' <span class="mag-edited"><span class="mdi mdi-pencil-outline" aria-hidden="true"></span> Edited' +
|
||||
(interactive
|
||||
? ' <button type="button" class="btn-sm" aria-expanded="false" onclick="' + stop + 'magRowDiff(\'' + esc(id) + '\', this)">Show changes</button>' +
|
||||
' <button type="button" class="btn-sm" onclick="' + stop + 'magRowRestore(\'' + esc(id) + '\')">Restore recognised text</button>'
|
||||
: '') + '</span>';
|
||||
}
|
||||
|
||||
function magRowDiff(id, btn) {
|
||||
const row = document.querySelector('#mag-review [data-block-row="' + CSS.escape(id) + '"]');
|
||||
const block = (state.layout?.blocks || []).find((b) => b.id === id);
|
||||
if (!row || !block?.ocr_text) return;
|
||||
let box = row.querySelector('.mag-diff');
|
||||
if (box) { box.remove(); btn?.setAttribute('aria-expanded', 'false'); if (btn) btn.textContent = 'Show changes'; return; }
|
||||
box = document.createElement('div');
|
||||
box.className = 'mag-diff';
|
||||
box.setAttribute('role', 'group');
|
||||
box.setAttribute('aria-label', 'Changes from the recognised text: removed words struck through and marked minus, added words underlined and marked plus');
|
||||
box.innerHTML = magWordDiff(block.ocr_text, block.text);
|
||||
row.querySelector('.mag-block-text').after(box);
|
||||
btn?.setAttribute('aria-expanded', 'true');
|
||||
if (btn) btn.textContent = 'Hide changes';
|
||||
}
|
||||
|
||||
async function magRowRestore(id) {
|
||||
const block = (state.layout?.blocks || []).find((b) => b.id === id);
|
||||
if (!block?.ocr_text) return;
|
||||
try {
|
||||
await api(magRegionUrl(id), { method: 'PATCH', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ text: block.ocr_text }) });
|
||||
} catch (err) {
|
||||
return say('Could not restore the recognised text: ' + err.message, 'err');
|
||||
}
|
||||
magReloadReview('Recognised text restored');
|
||||
}
|
||||
|
||||
async function magRowEdit(id) {
|
||||
if (state.reloadPending) await state.reloadPending;
|
||||
const block = (state.layout?.blocks || []).find((b) => b.id === id);
|
||||
@ -5290,6 +5365,9 @@ root.magRunPageQuality = magRunPageQuality;
|
||||
root.magFixReadingOrder = magFixReadingOrder;
|
||||
root.magCleanUpBoxes = magCleanUpBoxes;
|
||||
root.magRowEdit = magRowEdit;
|
||||
root.magRowDiff = magRowDiff;
|
||||
root.magRowRestore = magRowRestore;
|
||||
root.magWordDiff = magWordDiff;
|
||||
root.magSelectToggle = magSelectToggle;
|
||||
root.magSelectNone = magSelectNone;
|
||||
root.magBulk = magBulk;
|
||||
|
||||
@ -5558,3 +5558,12 @@ details.mag-report-group summary { cursor: pointer; margin-bottom: 6px; }
|
||||
.mag-find-main { display: flex; flex-direction: column; gap: 2px; min-width: 0; }
|
||||
.mag-find-summary { color: var(--subtext); font-size: 13px; overflow-wrap: anywhere; }
|
||||
@media (max-width: 600px) { .mag-find-item { flex-direction: column; align-items: stretch; } }
|
||||
|
||||
/* Review: text edited by hand, and its difference from the recognised text */
|
||||
.mag-edited { display: inline-flex; align-items: center; gap: 6px; margin-left: 8px; font-size: 12px; color: var(--subtext); }
|
||||
.mag-diff { margin: 6px 0 2px 28px; padding: 8px 10px; border: 1px dashed var(--border); border-radius: var(--radius);
|
||||
background: var(--surface); line-height: 1.6; overflow-wrap: anywhere; }
|
||||
.mag-diff del { text-decoration: line-through; color: var(--red, #b91c1c); }
|
||||
.mag-diff ins { text-decoration: underline; color: var(--green); }
|
||||
.mag-diff del::before { content: "\2212\00a0"; }
|
||||
.mag-diff ins::before { content: "+\00a0"; }
|
||||
|
||||
@ -1443,3 +1443,65 @@ test('mixing a PDF with images, or choosing another type, is refused before anyt
|
||||
await page.waitForTimeout(300);
|
||||
expect(created).toBe(0);
|
||||
});
|
||||
|
||||
test.describe('Edited text in review', () => {
|
||||
async function openReviewWithEditedBlock(page) {
|
||||
await mockBackend(page);
|
||||
const edited = { ...LAYOUT, blocks: LAYOUT.blocks.map(b => b.id === 'b3'
|
||||
? { ...b, ocr_text: 'Battery technolgy is entering a new phase. It will take years to mature.' } : b) };
|
||||
await page.route(`**/api/magazine/${DOC_ID}/layout*`, route => route.fulfill({ json: edited }));
|
||||
await page.goto('/', { waitUntil: 'domcontentloaded' });
|
||||
await page.waitForSelector('#app-skeleton', { state: 'detached', timeout: 15_000 });
|
||||
await navTo(page, 's-magazine');
|
||||
await page.locator('#mag-library-list .mag-card').first().click();
|
||||
await page.locator('#mag-contents-view-toggle').click();
|
||||
await page.locator('.mag-card-wrap').filter({ hasText: 'Page 3' }).getByRole('button', { name: /^Review/ }).click();
|
||||
await expect(page.locator('[data-block-row="b3"]')).toBeVisible();
|
||||
}
|
||||
|
||||
test('a hand-edited region is marked and shows a word diff against the recognised text', async ({ page }) => {
|
||||
await openReviewWithEditedBlock(page);
|
||||
const row = page.locator('[data-block-row="b3"]');
|
||||
await expect(row).toContainText('Edited');
|
||||
await expect(page.locator('[data-block-row="b1"]')).not.toContainText('Edited');
|
||||
const toggle = row.locator('.mag-edited button').first();
|
||||
await expect(toggle).toHaveText('Show changes');
|
||||
await toggle.click();
|
||||
await expect(toggle).toHaveAttribute('aria-expanded', 'true');
|
||||
await expect(row.locator('.mag-diff del')).toHaveText('technolgy');
|
||||
await expect(row.locator('.mag-diff ins')).toHaveText('technology');
|
||||
await row.getByRole('button', { name: 'Hide changes' }).click();
|
||||
await expect(row.locator('.mag-diff')).toHaveCount(0);
|
||||
});
|
||||
|
||||
test('restoring puts the recognised text back through the region route', async ({ page }) => {
|
||||
await openReviewWithEditedBlock(page);
|
||||
const patches = [];
|
||||
await page.route(`**/api/magazine/${DOC_ID}/pages/3/regions/b3`, route => {
|
||||
patches.push(route.request().postDataJSON());
|
||||
route.fulfill({ json: { ok: true } });
|
||||
});
|
||||
await page.locator('[data-block-row="b3"]').getByRole('button', { name: 'Restore recognised text' }).click();
|
||||
await expect.poll(() => patches.length).toBe(1);
|
||||
expect(patches[0]).toEqual({ text: 'Battery technolgy is entering a new phase. It will take years to mature.' });
|
||||
});
|
||||
|
||||
test('the word diff handles insertions, deletions, escaping and oversize input', async ({ page }) => {
|
||||
await mockBackend(page);
|
||||
await page.goto('/', { waitUntil: 'domcontentloaded' });
|
||||
await page.waitForSelector('#app-skeleton', { state: 'detached', timeout: 15_000 });
|
||||
await navTo(page, 's-magazine');
|
||||
const r = await page.evaluate(() => ({
|
||||
same: window.magWordDiff('a b c', 'a b c'),
|
||||
ins: window.magWordDiff('a c', 'a b c'),
|
||||
del: window.magWordDiff('a b c', 'a c'),
|
||||
html: window.magWordDiff('x <b>y</b>', 'x y'),
|
||||
big: window.magWordDiff('w '.repeat(2000), 'v '.repeat(2000)).startsWith('<del>'),
|
||||
}));
|
||||
expect(r.same).toBe('a b c');
|
||||
expect(r.ins).toBe('a <ins>b</ins> c');
|
||||
expect(r.del).toBe('a <del>b</del> c');
|
||||
expect(r.html).not.toContain('<b>');
|
||||
expect(r.big).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
@ -1071,6 +1071,40 @@ class QuickCorrectionTests(unittest.TestCase):
|
||||
with self.assertRaises(ValueError):
|
||||
set_region_text(self.layout(), 1, "c", " ")
|
||||
|
||||
def get(self, layout, bid):
|
||||
return next(b for b in layout["blocks"] if b["id"] == bid)
|
||||
|
||||
def test_the_first_edit_keeps_the_recognised_text_and_later_edits_do_not_overwrite_it(self):
|
||||
from core.magazine import set_region_text
|
||||
once = set_region_text(self.layout(), 1, "c", "Erster Satz. Zweiter Absatz.")
|
||||
self.assertEqual(self.get(once, "c")["ocr_text"], "Erster Satz. Zweiter Absatz beginnt.")
|
||||
twice = set_region_text(once, 1, "c", "Ganz anders.")
|
||||
self.assertEqual(self.get(twice, "c")["ocr_text"], "Erster Satz. Zweiter Absatz beginnt.")
|
||||
|
||||
def test_typing_the_original_back_clears_the_edit_mark(self):
|
||||
from core.magazine import set_region_text
|
||||
edited = set_region_text(self.layout(), 1, "c", "Changed.")
|
||||
back = set_region_text(edited, 1, "c", "Erster Satz. Zweiter Absatz beginnt.")
|
||||
self.assertNotIn("ocr_text", self.get(back, "c"))
|
||||
|
||||
def test_saving_unchanged_text_marks_nothing(self):
|
||||
from core.magazine import set_region_text
|
||||
same = set_region_text(self.layout(), 1, "c", "Erster Satz. Zweiter Absatz beginnt.")
|
||||
self.assertNotIn("ocr_text", self.get(same, "c"))
|
||||
|
||||
def test_other_regions_and_the_input_layout_are_untouched(self):
|
||||
from core.magazine import set_region_text
|
||||
layout = self.layout()
|
||||
out = set_region_text(layout, 1, "c", "Changed.")
|
||||
self.assertNotIn("ocr_text", self.get(out, "a"))
|
||||
self.assertNotIn("ocr_text", self.get(layout, "c"))
|
||||
|
||||
def test_ocr_text_survives_a_block_round_trip(self):
|
||||
from core.magazine import Block
|
||||
b = Block(id="x", page=1, bbox=[0, 0, 1, 1], type="body", text="t", ocr_text="old")
|
||||
self.assertEqual(b.to_json()["ocr_text"], "old")
|
||||
self.assertNotIn("ocr_text", Block(id="y", page=1, bbox=[0, 0, 1, 1], type="body", text="t").to_json())
|
||||
|
||||
def test_merging_closes_a_hyphenated_word_and_covers_both_boxes(self):
|
||||
from core.magazine import merge_with_next
|
||||
out, removed = merge_with_next(self.layout(), 1, "a")
|
||||
|
||||
Loading…
Reference in New Issue
Block a user