docs(audiobook): record the rejected scene-cast experiment
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
f34266c0d2
commit
ded70c3853
5
TODO.md
5
TODO.md
@ -345,8 +345,9 @@ Still open (each needs a real decision or new evidence, not more prompt tuning):
|
||||
until the book itself ties it to a name.
|
||||
- [ ] Look-alike main characters in one scene (Marcian/Zerwas/Lancorian) remain the biggest real error
|
||||
class (~1/5 of spoken words). Tried and rejected: reasoning mode (25× slower, no gain), larger/smaller
|
||||
windows, tag-based overrides, reasoning adjudication of disputes (made disputed lines worse). Untried:
|
||||
an explicit scene-cast constraint (characters present in the scene) built from the text.
|
||||
windows, tag-based overrides, reasoning adjudication of disputes (made disputed lines worse), an explicit
|
||||
scene-cast hint (names mentioned in the passage: +0.3 points, noise). What remains is mostly model
|
||||
capacity plus reference noise; the useful lever left is human review guided by the agreed/disputed flags.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@ -132,7 +132,8 @@ lines) but compared against a biased baseline; on the real pipeline it made the
|
||||
so the adjudication step (and the `candidates` request field) was removed. The only reliable signal is
|
||||
agreement itself: agreed lines are right 82 % of the time, contested first answers 67 %.
|
||||
Sampling matters: windows of ~1 200 characters were clearly worse (−4 points) and ~5 000 about equal to
|
||||
the default, so the default stays.
|
||||
the default, so the default stays. A scene-cast hint (which known characters are named in the passage or
|
||||
just before it) gave +0.3 points — noise — and was not adopted.
|
||||
|
||||
Request additions: `attribute_only` `[ids]` — the other spans are context; unrequested lines are never
|
||||
reported missing.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user