diff --git a/CHANGELOG.md b/CHANGELOG.md index 24fb7ae..12657c3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,8 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) · versioned wi --- ## [Unreleased] +### Changed +- **Magazine interviews: journalist/guest voices.** Interview labels such as `c't:` (also `c’t`, `ct`, OCR `Cc't`), `Redaktion`, `Interviewer`, `Frage` are the interviewing side and are read in the narrator voice; every other recurring label is a guest and gets the second voice. Unlabelled blocks continue the current speaker's turn; headings, decks and bylines stay narrator. Detection stays strict (label starts a block and recurs at least twice). Audio-timing records now store each segment's voice and speaker. Tests: `tests/test_magazine.py` (`InterviewJournalistTests`). --- diff --git a/Manual.md b/Manual.md index 561f661..6c3c1c0 100644 --- a/Manual.md +++ b/Manual.md @@ -407,7 +407,7 @@ turned into a podcast discussion between two or three voices you pick. **Layout and views:** driving mode is a top bar, a content area and a controls dock. The dock (position slider, time and playback speed, then previous, back 15 s, play/pause, forward 15 s, next) is always completely visible at the bottom and is never covered or pushed off screen, on phones from 320x568, tablets and in landscape. The top bar has *Share*, a *menu* button that shows the current view, *Index* and *Exit*. The menu opens a flyout in four groups. **View** (radio options with icon and name): **Picture** (the whole picture, no text), **Text** (the whole text, lyrics-style, no picture) and **Split** (picture plus three lines, the default; in landscape side by side). **Show** (switches, label left and switch right; on = thumb right with a tick, off = thumb left with a cross): *Show text* and *Show pictures* hide just that part in the views that contain it (so with Split you can drop either), and *Minimize text and picture*. **Playback:** *Play next article automatically* (see below). **Theme:** one control with *Light*, *Auto* and *Dark*; the chosen one carries a tick. Each control is at least 44 px high and the flyout fits phones from 320 px wide. Keyboard: Enter or Arrow Down opens the menu; Up/Down/Left/Right move over every control, Home/End jump to the first/last, Space or Enter switches or selects, Escape or a tap outside closes it and returns focus to the menu button. Choosing a view closes the flyout; switches and the theme keep it open so you see the effect. **Automatic next article:** when an article ends it is marked finished and the next article that has audio starts (articles without audio are skipped, the speed stays); title, picture, text and the index follow and a screen reader hears "Now playing: ..." once. After the last article it stops and shows *End of magazine* (it never starts over). Switch *Play next article automatically* off to stop after each article. An article whose audio cannot be loaded is skipped with a short message. Older saved views are converted (Lyrics to Text, Compact to Split). **Index:** the *Index* button opens a list of all articles of the magazine with their length (articles without audio are dimmed and marked *no audio*). The article that is playing is highlighted; tap it or press Enter on another to jump there while driving mode keeps playing. Every row is its own item, separated from the next by a thin divider line and generous space above and below (the playing row is highlighted instead), so long two- or three-line titles do not run into each other. Escape or the close button closes the list (Tab stays inside it while open); the controls below stay visible and usable by touch. Lengths come from the already generated audio, nothing is recomputed. Each row also has a round **play button** (Play for unplayed, *Resume* with a circular-arrow icon for partly played, *Play again* with the replay icon for finished articles; it starts that article in driving mode and continues from the saved position), the author as *by Name* in smaller muted text under the title (omitted when unknown, long names are shortened), and a status icon on the right (empty circle: not played, dashed circle: partly played, check: finished) that also has the text *Not played / Partly played / Finished-Played* for screen readers and updates live while you listen. **Speed** is the button after the time next to the slider. **Picture zoom:** double-tap or double-click, pinch, mouse wheel, the + and - keys, or the zoom in / zoom out / reset buttons on the picture; drag (or arrow keys) to move a zoomed picture; Escape or the reset button returns to full view. **Lyrics:** the spoken sentence is large and kept in the middle, earlier and later ones are dimmed; tap or press Enter on a sentence to play from it; scrolling by hand stops the automatic following and shows *Back to current*. With reduced motion switched on in your device the text jumps instead of gliding. **Minimize** (in the menu) reduces the area to the title and the current line so the dock keeps minimal text; choose it again to expand. Your view and the minimized state are remembered in this browser only. Titles wrap to two lines (with an ellipsis beyond that), picture captions to two lines. **Theme (night driving):** the *Theme* control at the end of the menu offers **Auto** (follows the system, default), **Light** and **Dark** (night). Dark is a night palette for the whole driving overlay (top bar, menu, index and rows, text lines, slider, dock, zoom buttons, the *Tap to play* button, toasts and the share box): true black background, text at about 85% white, a muted blue accent, pictures slightly dimmed, and the browser's address bar colour follows while driving mode is open and is restored when you leave. Text keeps at least 4.5:1 contrast and buttons/borders 3:1, also for rows without audio (shown dimmed with a dashed play button); the listening states stay distinguishable by icon, not only by colour, and the focus ring and spoken-word highlight stay clearly visible. Auto switches live when your phone changes between day and night mode. The choice is independent of the rest of the app and remembered in this browser only (if storage is blocked it applies until you leave the page). Nothing glides or fades in extra for the theme, so reduced-motion settings are unaffected. **How much text (Split view):** the menu section *Text in Split view* sets the amount, remembered on this device. **Sentence** (default) shows only the sentence being read, large; **Context** shows the previous sentence (small), the current one (large) and the next (small); **Paragraph** shows the whole paragraph the sentence belongs to with the current sentence bold and underlined and the spoken word marked. The text frame always stays between the picture and the controls: the current text shrinks from 1.5 rem down to 1 rem (it follows your device's text size and browser zoom) until it fits, the picture gives way before the text is squeezed, and whatever still does not fit scrolls inside its frame while following the spoken word (scroll by hand to look around; it follows again at the next sentence). In landscape the picture and the text sit side by side. The *Text* view always shows the whole text. Tap or press Enter on a dimmed line (Context, Paragraph) to jump to it; the text is not announced sentence by sentence. **Context lines:** with *Context*, driving mode shows the previous sentence (small), the current one (large, the spoken word marked) and the next (small) in a box of fixed height, so nothing moves around it. Tap or press Enter on the previous or next line to jump to that sentence. The text is deliberately not announced by screen readers sentence by sentence; the play/pause button and the position slider stay the way to control it. With reduced motion switched on in your device, lines change without animation. On a phone (375 px wide) the sentences wrap and a very long current sentence scrolls inside its box. Turn the text off with *Show text* in the menu to see only the picture. **Lyrics** in the older wording is now the **Text** view. - **Text and picture follow the voice:** for audio generated since sentence times were recorded, the highlighted sentence (playlist and driving mode) and the picture shown come from the exact time each sentence was spoken, so they stay right after the slider, the 15-second buttons, a tap on a sentence, the lock screen or a resume. Captions are shown but are not in the MP3, so they are never highlighted. The highlight lights up a sentence just before its first word is heard (each sentence starts with a short silence, which is not counted). Audio from before sentence times were recorded, and audio whose text was edited afterwards, has no usable record and is followed by an estimate from sentence length (less exact); regenerate the audio to get exact sync. Interview-voice audio is covered like any other. + **Text and picture follow the voice:** for audio generated since sentence times were recorded, the highlighted sentence (playlist and driving mode) and the picture shown come from the exact time each sentence was spoken, so they stay right after the slider, the 15-second buttons, a tap on a sentence, the lock screen or a resume. Captions are shown but are not in the MP3, so they are never highlighted. The highlight lights up a sentence just before its first word is heard (each sentence starts with a short silence, which is not counted). Audio from before sentence times were recorded, and audio whose text was edited afterwards, has no usable record and is followed by an estimate from sentence length (less exact); regenerate the audio to get exact sync. Interview-voice audio is covered like any other. In interviews, the interviewer (`c't:`, `Redaktion:`, `Interviewer:`, `Frage:`) is read in the narrator voice and each guest in the second voice; paragraphs without a label continue the previous speaker, and the labels themselves are not spoken. **Word by word:** the first time an article is played, the app asks the forced-aligner service (the separate `tts-aligner` container, see below) to measure where each word starts and ends and stores that beside the audio. Playback never waits for it: until it is ready (usually well under a minute for a magazine article) the marked word is estimated from word length as before, and it switches to the measured word on its own. The marked word is the last one that has started, so it stays lit through a pause; the *Lyrics* view now marks the spoken word too (as do *Compact* and the playlist text). If the service is not running, nothing changes for you and the app tries again later (at most every 10 minutes per article). Prerequisites: `docker compose up -d tts-aligner` (first start downloads about 1.2 GB of model weights into the `aligner-models` volume; after that it needs no internet). `ALIGNER_URL` empty turns word timing off. Privacy: text and audio go only to your own container on the internal Docker network. Accessibility: the marking is visual only and adds no announcements; with reduced motion nothing animates. API: `POST /api/magazine/{doc}/articles/{id}/align-words` starts it explicitly, `GET` shows progress. **Share a link:** the *Share* button in the toolbar (the whole magazine), the share button in the playlist header and the one at the top left of driving mode (the current article) share a link of the form `/#s-magazine?doc=&article=
&mode=drive`. Where the browser has a share sheet (phones, and any browser on https) it opens; otherwise the link is copied and a confirmation is announced (on a plain-http address a box with the link is shown to copy by hand). Opening the link goes straight to that article in driving mode with audio, pictures and text; if the browser refuses to start audio without a tap, a large *Tap to play* button appears. A link for the whole magazine opens at its first unfinished article. **Privacy and access:** the link contains only ids, no password or key. It is just an address: whoever opens it needs to reach this app the same way as for any other page (this app has no user accounts; access is controlled by your network or reverse proxy), and an unknown or removed article simply shows a message. Only articles that already have audio can be shared. A link whose parameters are not in the expected form is ignored. diff --git a/architecture/magazine-ingestion.md b/architecture/magazine-ingestion.md index d12a352..902cec6 100644 --- a/architecture/magazine-ingestion.md +++ b/architecture/magazine-ingestion.md @@ -963,3 +963,7 @@ gets `audio_stale: true` with `audio_stale_reason`. Audio and timing files are * `static/js/magazine.js` and is not fixed here. Until it is, new stand-ins can appear. * Reading order across a page that holds two articles relies on the existing per-page order. A wrong page order gives a wrong *previous entry*. + +## Interview voice slots + +`interview_tts_segments` returns `{speaker, text, slot}` per block (slot 0 = narrator voice, 1 = second voice). A label must start a block and recur at least twice. Labels in `INTERVIEW_JOURNALIST_LABELS` (`c't`, `ct`, `redaktion`, `interviewer`, `frage`; apostrophes normalised, OCR `Cc't` folded) get slot 0; all other recurring labels get slot 1. With no journalist label, exactly two recurring labels are required and the first seen gets slot 0. Unlabelled blocks continue the current speaker; title/heading/deck/byline/figure_description reset to narrator. `write_audio_timings` records `voice` and `speaker` per segment. diff --git a/core/magazine.py b/core/magazine.py index d3f66a1..9bc4b80 100644 --- a/core/magazine.py +++ b/core/magazine.py @@ -5651,20 +5651,31 @@ def split_sentences_for_tts(text: str) -> list[str]: _INTERVIEW_LABEL_RE = re.compile( - r"^\s*([A-ZÄÖÜ][A-Za-zÄÖÜäöüß.'’\-]+(?:\s+[A-ZÄÖÜ][A-Za-zÄÖÜäöüß.'’\-]+){0,3})\s*:\s+(.+)$", + r"^\s*((?:[A-ZÄÖÜ][A-Za-zÄÖÜäöüß.'’\-]+(?:\s+[A-ZÄÖÜ][A-Za-zÄÖÜäöüß.'’\-]+){0,3})|[Cc]{1,2}['’‘`´]t|ct)\s*:\s+(.+)$", re.S, ) +# Labels of the interviewing side (casefolded, apostrophes normalised to "'"). Spoken in the narrator voice; +# every other recurring label is a guest and gets the second voice. +INTERVIEW_JOURNALIST_LABELS = frozenset({"c't", "ct", "redaktion", "interviewer", "frage"}) +_NON_TURN_TYPES = frozenset({"title", "heading", "deck", "byline", "figure_description"}) -def interview_tts_segments(blocks: Sequence[dict[str, Any]]) -> list[dict[str, str | None]]: - """Return article text segments with speaker labels for a two-person interview. +def _label_key(label: str) -> str: + key = re.sub(r"[’‘`´]", "'", label.strip().casefold()) + return "c't" if key == "cc't" else key # OCR doubles the drop-cap: "Cc't:" - Detection is intentionally strict: exactly two recurring labels must begin - paragraph blocks. Colons in ordinary prose therefore do not suddenly - switch voices. Speaker labels themselves are omitted from speech because - the voice change already conveys who is talking. + +def interview_tts_segments(blocks: Sequence[dict[str, Any]]) -> list[dict[str, Any]]: + """Return article text segments with speaker labels and voice slots for an interview. + + Each segment: ``speaker`` (label or None), ``text`` (label stripped), ``slot`` (0 narrator voice, + 1 second voice). Detection is strict so colons in prose never switch voices: a label must start a + block and recur at least twice, and there must be a journalist label (``INTERVIEW_JOURNALIST_LABELS``) + or two distinct recurring labels. Journalist -> slot 0, every other label -> slot 1; with no journalist + label the first label seen gets slot 0. An unlabelled block continues the current speaker's turn; + titles/headings/decks/bylines and everything before the first label stay narrator. """ - parsed: list[tuple[str | None, str]] = [] + parsed: list[tuple[str | None, str, str, dict[str, Any]]] = [] # key, body, full text, block counts: dict[str, int] = {} labels: dict[str, str] = {} for block in blocks: @@ -5673,22 +5684,32 @@ def interview_tts_segments(blocks: Sequence[dict[str, Any]]) -> list[dict[str, s continue match = _INTERVIEW_LABEL_RE.match(text) if not match: - parsed.append((None, text)) + parsed.append((None, text, text, block)) continue label, body = match.group(1).strip(), match.group(2).strip() - key = label.casefold() + key = _label_key(label) labels.setdefault(key, label) counts[key] = counts.get(key, 0) + 1 - parsed.append((key, body)) + parsed.append((key, body, text, block)) recurring = [key for key, count in counts.items() if count >= 2] - if len(recurring) != 2: - return [{"speaker": None, "text": text} for _, text in parsed] - allowed = set(recurring) - return [ - {"speaker": labels[speaker] if speaker in allowed else None, "text": text} - for speaker, text in parsed - ] + journalist = [k for k in recurring if k in INTERVIEW_JOURNALIST_LABELS] + if not journalist and len(recurring) != 2: + return [{"speaker": None, "text": full, "slot": 0} for _, _, full, _ in parsed] + slot_of = {k: (0 if k in journalist else 1) for k in recurring} if journalist else {k: i for i, k in enumerate(recurring)} + out: list[dict[str, Any]] = [] + speaker: str | None = None + slot = 0 + for key, body, full, block in parsed: + if key in slot_of: + speaker, slot = labels[key], slot_of[key] + out.append({"speaker": speaker, "text": body, "slot": slot}) + elif block.get("type") in _NON_TURN_TYPES: + speaker, slot = None, 0 + out.append({"speaker": None, "text": full, "slot": 0}) + else: + out.append({"speaker": speaker, "text": full, "slot": slot}) + return out def spoken_chars_for(blocks: Sequence[dict[str, Any]], timings: dict[str, Any] | None) -> list[int] | None: @@ -5728,18 +5749,12 @@ def synthesize_article_audio(doc_dir: Path, article: dict[str, Any], layout: dic {"speaker": None, "text": b["text"], "spoken": b.get("spoken_text")} for b in blocks if (b.get("text") or "").strip() ] - speakers = [] - for segment in segments: - speaker = segment.get("speaker") - if speaker and speaker not in speakers: - speakers.append(speaker) - voice_for = {speaker: (voice if i == 0 else interview_voice) - for i, speaker in enumerate(speakers[:2])} utterances: list[tuple[str, str]] = [] utterance_segment: list[int] = [] # which segment each utterance came from, for the timing record for n, segment in enumerate(segments): spoken = segment.get("spoken") or prepare_text_for_tts(str(segment["text"]), language, glossary) - segment_voice = voice_for.get(segment.get("speaker"), voice) + segment_voice = interview_voice if segment.get("slot") == 1 else voice + segment["voice"] = segment_voice for sentence in split_sentences_for_tts(spoken): utterances.append((sentence, segment_voice)) utterance_segment.append(n) @@ -5849,7 +5864,9 @@ def write_audio_timings(mp3_path: Path, segments: Sequence[dict[str, Any]], utte record = { "version": 1, "interview": bool(interview), "duration_ms": int(spans[-1][1]) if spans else 0, "segments": [{"chars": len(str(seg.get("text") or "")), "times": times, - **({"onsets": on} if onsets else {})} + **({"onsets": on} if onsets else {}), + **({"voice": seg["voice"]} if seg.get("voice") else {}), + **({"speaker": seg["speaker"]} if seg.get("speaker") else {})} for seg, times, on in zip(segments, per_segment, per_onset)], } try: diff --git a/tests/test_magazine.py b/tests/test_magazine.py index 61f036d..a073a62 100644 --- a/tests/test_magazine.py +++ b/tests/test_magazine.py @@ -819,6 +819,69 @@ class InterviewVoiceTests(unittest.TestCase): self.assertTrue(all(s["speaker"] is None for s in interview_tts_segments(blocks))) +class InterviewJournalistTests(unittest.TestCase): + @staticmethod + def _ct_blocks(label="c't"): + return [ + {"text": "Interview mit Michael Gerlich", "type": "heading"}, + {"text": "Einleitung zum Gespräch.", "type": "body"}, + {"text": f"{label}: Was ist kognitives Offloading?"}, + {"text": "Michael Gerlich: Wir lagern Denken an KI aus."}, + {"text": "Ein zweiter Absatz der Antwort ohne Label."}, + {"text": f"{label}: Ist das gefährlich?"}, + {"text": "Michael Gerlich: Teilweise."}, + ] + + def test_ct_variants_map_journalist_to_narrator_and_guest_to_second_voice(self): + for label in ("c't", "c’t", "ct", "C't", "Cc't"): + with self.subTest(label=label): + segs = interview_tts_segments(self._ct_blocks(label)) + self.assertEqual([s["slot"] for s in segs], [0, 0, 0, 1, 1, 0, 1]) + self.assertEqual(segs[2]["text"], "Was ist kognitives Offloading?") + self.assertEqual(segs[4]["speaker"], "Michael Gerlich") + self.assertEqual(segs[4]["text"], "Ein zweiter Absatz der Antwort ohne Label.") + self.assertIsNone(segs[0]["speaker"]) + + def test_multiple_guests_all_use_second_voice(self): + blocks = [{"text": "c't: Frage eins?"}, {"text": "Anna Meyer: A."}, {"text": "c't: Frage zwei?"}, + {"text": "Anna Meyer: B."}, {"text": "c't: Frage drei?"}, {"text": "Tom Schulz: C."}, + {"text": "c't: Frage vier?"}, {"text": "Tom Schulz: D."}] + self.assertEqual([s["slot"] for s in interview_tts_segments(blocks)], [0, 1, 0, 1, 0, 1, 0, 1]) + + def test_two_plain_labels_keep_first_appearance_rule(self): + blocks = [{"text": "Zed Alpha: Eins."}, {"text": "Bob Beta: Zwei."}, {"text": "Zed Alpha: Drei."}, + {"text": "Bob Beta: Vier."}] + self.assertEqual([s["slot"] for s in interview_tts_segments(blocks)], [0, 1, 0, 1]) + + def test_single_colon_labels_are_not_speakers_and_keep_their_text(self): + blocks = [{"text": "Hinweis: Das gilt nur für Version 2."}, {"text": "Fazit: Es lohnt sich."}, + {"text": "Hinweis: Noch einmal."}] + segs = interview_tts_segments(blocks) + self.assertTrue(all(s["speaker"] is None and s["slot"] == 0 for s in segs)) + self.assertEqual(segs[0]["text"], "Hinweis: Das gilt nur für Version 2.") + + def test_one_journalist_label_alone_is_not_enough(self): + segs = interview_tts_segments([{"text": "c't: Einmal."}, {"text": "Text."}]) + self.assertTrue(all(s["speaker"] is None for s in segs)) + + def test_timings_record_voice_and_speaker_and_stay_backward_compatible(self): + import tempfile + import json + from pathlib import Path + from core.magazine import write_audio_timings + segs = interview_tts_segments(self._ct_blocks()) + for s in segs: + s["voice"] = "B" if s["slot"] else "A" + with tempfile.TemporaryDirectory() as d: + mp3 = Path(d) / "x.mp3" + write_audio_timings(mp3, segs, list(range(len(segs))), [(i, i + 1) for i in range(len(segs))], interview=True) + rec = json.loads((Path(d) / "x.timings.json").read_text()) + self.assertEqual(rec["version"], 1) + self.assertEqual({s["voice"] for s in rec["segments"]}, {"A", "B"}) + self.assertTrue(all("chars" in s and "times" in s for s in rec["segments"])) + self.assertEqual(rec["segments"][3]["speaker"], "Michael Gerlich") + + class OcrQualityScoreTests(unittest.TestCase): """Grounded in real output from both OCR paths on a scanned c't magazine.