docs(audiobook): record the rejected scene-cast experiment
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
f34266c0d2
commit
ded70c3853
5
TODO.md
5
TODO.md
@ -345,8 +345,9 @@ Still open (each needs a real decision or new evidence, not more prompt tuning):
|
|||||||
until the book itself ties it to a name.
|
until the book itself ties it to a name.
|
||||||
- [ ] Look-alike main characters in one scene (Marcian/Zerwas/Lancorian) remain the biggest real error
|
- [ ] Look-alike main characters in one scene (Marcian/Zerwas/Lancorian) remain the biggest real error
|
||||||
class (~1/5 of spoken words). Tried and rejected: reasoning mode (25× slower, no gain), larger/smaller
|
class (~1/5 of spoken words). Tried and rejected: reasoning mode (25× slower, no gain), larger/smaller
|
||||||
windows, tag-based overrides, reasoning adjudication of disputes (made disputed lines worse). Untried:
|
windows, tag-based overrides, reasoning adjudication of disputes (made disputed lines worse), an explicit
|
||||||
an explicit scene-cast constraint (characters present in the scene) built from the text.
|
scene-cast hint (names mentioned in the passage: +0.3 points, noise). What remains is mostly model
|
||||||
|
capacity plus reference noise; the useful lever left is human review guided by the agreed/disputed flags.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@ -132,7 +132,8 @@ lines) but compared against a biased baseline; on the real pipeline it made the
|
|||||||
so the adjudication step (and the `candidates` request field) was removed. The only reliable signal is
|
so the adjudication step (and the `candidates` request field) was removed. The only reliable signal is
|
||||||
agreement itself: agreed lines are right 82 % of the time, contested first answers 67 %.
|
agreement itself: agreed lines are right 82 % of the time, contested first answers 67 %.
|
||||||
Sampling matters: windows of ~1 200 characters were clearly worse (−4 points) and ~5 000 about equal to
|
Sampling matters: windows of ~1 200 characters were clearly worse (−4 points) and ~5 000 about equal to
|
||||||
the default, so the default stays.
|
the default, so the default stays. A scene-cast hint (which known characters are named in the passage or
|
||||||
|
just before it) gave +0.3 points — noise — and was not adopted.
|
||||||
|
|
||||||
Request additions: `attribute_only` `[ids]` — the other spans are context; unrequested lines are never
|
Request additions: `attribute_only` `[ids]` — the other spans are context; unrequested lines are never
|
||||||
reported missing.
|
reported missing.
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user