overlay: fix AT-SPI caret freeze; fuse routing match + live LLM into HUD; Release 1.6.0

Fix a desktop-session freeze (forced logout/reboot) caused by the overlay's
AT-SPI caret tracker: it subscribed to the high-frequency object:text-caret-moved
signal and made synchronous, blocking AT-SPI reads from inside the event handler,
re-entering the a11y dispatcher and getting stormed by the app's own xdotool
typing until GNOME stopped responding. Now track focus changes only and read the
caret rectangle lazily, once, when the overlay shows — never on the hot path.

Fuse voice-routing feedback into the overlay instead of a desktop notification:
show the matched preset's emoji, name, and spoken keyword on a banner, narrate
the phase (Transcribing -> Rewriting), and stream the LLM rewrite into the bubble
token-by-token. Redundant per-dictation notifications are suppressed when the
overlay is present (errors still notify); headless/overlay-off is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
mARTin-B78 2026-06-07 16:58:47 +02:00
parent 6a9c1fc95a
commit ae01472b77
9 changed files with 337 additions and 62 deletions

View File

@ -9,6 +9,31 @@ The version is defined in [`blitztext/__init__.py`](blitztext/__init__.py).
## [Unreleased] ## [Unreleased]
## [1.6.0] - 2026-06-07
### Fixed
- **Session freeze when the overlay's caret tracking was active** (could lock up
the whole GNOME/X11 desktop, forcing a logout/reboot). The AT-SPI caret tracker
subscribed to the high-frequency `object:text-caret-moved` signal and made
**synchronous, blocking AT-SPI reads from inside the event handler** — which
re-enters the accessibility dispatcher and is stormed by the app's *own*
`xdotool` typing (one event per character), congesting the a11y bus until the
desktop stopped responding. It now tracks **focus changes only** and reads the
caret rectangle lazily (once, when the overlay shows), never from inside an
event dispatch.
### Changed
- **Matched preset is fused into the overlay instead of a desktop notification**:
when voice routing picks a preset, the overlay shows its emoji icon, name, and
the spoken keyword on a banner, and narrates the phase ("Transcribing…" →
"Rewriting…"). With the overlay on, the redundant per-dictation notifications
are suppressed (errors still notify); headless/overlay-off keeps notifications.
### Added
- **Live LLM rewrite in the overlay**: rewrite presets now stream the model's
output into the bubble token-by-token, so you watch it write. The delivered
text is still the complete result, typed once the rewrite finishes.
## [1.5.1] - 2026-06-07 ## [1.5.1] - 2026-06-07
### Added ### Added

View File

@ -6,4 +6,4 @@ counterpart to the macOS Blitztext menu bar app: it runs natively on the host
(not in a container) so it can type into any application via xdotool. (not in a container) so it can type into any application via xdotool.
""" """
__version__ = "1.5.1" __version__ = "1.6.0"

View File

@ -6,8 +6,10 @@ through a chain of decreasing precision:
1. AT-SPI caret the real text insertion point, when the focused app exposes 1. AT-SPI caret the real text insertion point, when the focused app exposes
it over accessibility (native GTK/Qt apps do; many terminals / Electron / it over accessibility (native GTK/Qt apps do; many terminals / Electron /
web views do not). Tracked passively via the GLib main loop, so reads are web views do not). We track only the *focused* object via a11y events (cheap,
instant and never block. low-frequency) and read its caret rectangle lazily, once, when the overlay
shows never from inside an event dispatch, since synchronous AT-SPI reads
on the hot path can wedge the accessibility bus and freeze the session.
2. Mouse pointer `xdotool getmouselocation`. Always available on X11; a good 2. Mouse pointer `xdotool getmouselocation`. Always available on X11; a good
proxy since the pointer is usually near where you're typing. proxy since the pointer is usually near where you're typing.
3. Window / screen top-centre of the target window, else screen bottom-centre. 3. Window / screen top-centre of the target window, else screen bottom-centre.
@ -40,25 +42,39 @@ class Anchor:
# Tier 1: AT-SPI caret tracking (best-effort, passive) # Tier 1: AT-SPI caret tracking (best-effort, passive)
# --------------------------------------------------------------------------- # # --------------------------------------------------------------------------- #
class _CaretTracker: class _CaretTracker:
"""Passively follow the focused editable text's caret via AT-SPI events. """Remember the most recently focused text object; read its caret lazily.
We never poll the a11y tree on the hot path (that can be slow and can block *Why so cautious.* AT-SPI queries (``get_character_extents`` and friends) are
on unresponsive apps). Instead we subscribe to caret-moved / focus events and **synchronous, blocking D-Bus round-trips into the target application**. The
cache the last caret rectangle; `rect()` just returns the cached value if it earlier design subscribed to the high-frequency ``object:text-caret-moved``
is fresh enough to still be meaningful. signal and ran those blocking reads from *inside* the event handler. Two ways
that wedges a whole GNOME/X11 session:
Calling a synchronous AT-SPI method from within an AT-SPI event dispatch
re-enters the a11y dispatcher and can deadlock the accessibility bus.
``text-caret-moved`` fires once *per character* and delivering text is
exactly what this app does, typing via ``xdotool`` into the focused
field. So a single dictation became a storm of blocking round-trips on
the GTK main loop, congesting the a11y bus until the desktop froze.
So we now subscribe to **focus changes only** (rare, and never emitted by our
own synthetic typing), cache just the focused accessible, and do the one
blocking extents read **on demand** in :meth:`rect` called once, when the
overlay shows, outside any event dispatch. Worst case is a slightly delayed
overlay placement, never a frozen session.
""" """
STALE_SECONDS = 30.0 # ignore a cached caret older than this STALE_SECONDS = 30.0 # ignore a focus older than this
def __init__(self) -> None: def __init__(self) -> None:
self._ok = False self._ok = False
self._listener = None self._listener = None
self._rect: tuple[int, int, int, int] | None = None # x, y, w, h (screen) self._focused = None # last focused accessible (read lazily)
self._stamp = 0.0 self._stamp = 0.0
self._Atspi = None self._Atspi = None
def start(self) -> bool: def start(self) -> bool:
"""Register AT-SPI listeners on the (already running) GLib main loop. """Register the AT-SPI focus listener on the running GLib main loop.
Safe to call when accessibility is disabled it just returns False and Safe to call when accessibility is disabled it just returns False and
the anchor logic skips this tier from then on. the anchor logic skips this tier from then on.
@ -74,10 +90,10 @@ class _CaretTracker:
# init() is idempotent; returns 0/1. Connects to the a11y registry. # init() is idempotent; returns 0/1. Connects to the a11y registry.
Atspi.init() Atspi.init()
self._Atspi = Atspi self._Atspi = Atspi
self._listener = Atspi.EventListener.new(self._on_event) self._listener = Atspi.EventListener.new(self._on_focus)
# Caret moves give us the live position; focus changes let us grab the # Focus changes only. Deliberately NOT "object:text-caret-moved": that
# caret of a freshly-focused field even before it moves. # firehose (one event per typed character, including our own output)
self._listener.register("object:text-caret-moved") # plus synchronous reads is what could freeze the session.
self._listener.register("object:state-changed:focused") self._listener.register("object:state-changed:focused")
self._ok = True self._ok = True
log("[overlay] AT-SPI caret tracking active") log("[overlay] AT-SPI caret tracking active")
@ -90,26 +106,23 @@ class _CaretTracker:
def stop(self) -> None: def stop(self) -> None:
try: try:
if self._listener is not None: if self._listener is not None:
self._listener.deregister("object:text-caret-moved")
self._listener.deregister("object:state-changed:focused") self._listener.deregister("object:state-changed:focused")
except Exception: # noqa: BLE001 except Exception: # noqa: BLE001
pass pass
self._listener = None self._listener = None
self._focused = None
self._ok = False self._ok = False
def _on_event(self, event) -> None: def _on_focus(self, event) -> None:
# Runs on the GLib main thread (same loop GTK uses). Keep it cheap and # Runs on the GLib main thread (same loop GTK uses). Do the *minimum*:
# never raise — an exception here would bubble into the a11y dispatcher. # stash the focused accessible and stamp it. Crucially, make NO synchronous
# AT-SPI calls here — that would re-enter the a11y dispatcher and risk
# deadlocking the bus. The blocking extents read happens later, in rect().
try: try:
if event.type.startswith("object:state-changed:focused") and not event.detail1: if not event.detail1:
return # a *de*focus event — nothing to read return # a *de*focus event — nothing to track
source = event.source self._focused = event.source
if source is None: self._stamp = time.time()
return
rect = self._caret_rect(source)
if rect is not None:
self._rect = rect
self._stamp = time.time()
except Exception: # noqa: BLE001 except Exception: # noqa: BLE001
pass pass
@ -142,11 +155,17 @@ class _CaretTracker:
return None return None
def rect(self) -> tuple[int, int, int, int] | None: def rect(self) -> tuple[int, int, int, int] | None:
if not self._ok or self._rect is None: # Called once when the overlay shows (not on the a11y hot path), so the
# single blocking extents read here is safe: at worst it briefly delays
# the overlay, it cannot storm or re-enter the bus.
if not self._ok or self._focused is None:
return None return None
if time.time() - self._stamp > self.STALE_SECONDS: if time.time() - self._stamp > self.STALE_SECONDS:
return None return None
return self._rect try:
return self._caret_rect(self._focused)
except Exception: # noqa: BLE001 - focused app may be gone/unresponsive
return None
# --------------------------------------------------------------------------- # # --------------------------------------------------------------------------- #

View File

@ -37,17 +37,25 @@ class Daemon:
def __init__(self, cfg: Config, status_cb: StatusCallback | None = None, def __init__(self, cfg: Config, status_cb: StatusCallback | None = None,
level_cb: Callable[[float], None] | None = None, level_cb: Callable[[float], None] | None = None,
text_cb: Callable[[str], None] | None = None, text_cb: Callable[[str], None] | None = None,
countdown_cb: Callable[[float | None, float], None] | None = None): countdown_cb: Callable[[float | None, float], None] | None = None,
routing_cb: Callable[[str, str, str | None], None] | None = None):
self.cfg = cfg self.cfg = cfg
self.status_cb = status_cb self.status_cb = status_cb
# Optional UI feedback hooks for the on-screen overlay. The daemon stays # Optional UI feedback hooks for the on-screen overlay. The daemon stays
# UI-agnostic: these are no-ops in headless mode. level_cb gets the live # UI-agnostic: these are no-ops in headless mode. level_cb gets the live
# mic level (0..1); text_cb gets the running transcript while streaming; # mic level (0..1); text_cb gets the running transcript while streaming
# countdown_cb(seconds_left, window) drives the silence auto-stop ring # (and the live LLM rewrite); countdown_cb(seconds_left, window) drives the
# (seconds_left=None while you're speaking, so the ring clears). # silence auto-stop ring (seconds_left=None while you're speaking, so the
# ring clears); routing_cb(icon, preset_name, keyword) fires when voice
# routing picks a preset, so the overlay can show it.
self.level_cb = level_cb self.level_cb = level_cb
self.text_cb = text_cb self.text_cb = text_cb
self.countdown_cb = countdown_cb self.countdown_cb = countdown_cb
self.routing_cb = routing_cb
# An overlay consumes routing_cb, and it narrates every phase on-screen, so
# the redundant desktop notifications are fused into it (only errors still
# pop a bubble). Headless / overlay-off keeps the notifications.
self._overlay = routing_cb is not None
self._ov_meter = None self._ov_meter = None
self._ov_text_final = "" self._ov_text_final = ""
self._lock = threading.Lock() self._lock = threading.Lock()
@ -99,14 +107,20 @@ class Daemon:
notify(title, body, urgency=urgency, enabled=self.cfg.notify) notify(title, body, urgency=urgency, enabled=self.cfg.notify)
def _dnotify(self, title: str, body: str = "", urgency: str = "normal") -> None: def _dnotify(self, title: str, body: str = "", urgency: str = "normal") -> None:
"""Per-dictation notification — suppressed for hands-free (wakeword) sessions.""" """Per-dictation notification — suppressed for hands-free (wakeword)
if not self._session_silent: sessions, and (except for errors) when an overlay is narrating on-screen."""
self._notify(title, body, urgency=urgency) if self._session_silent:
return
if self._overlay and urgency != "critical":
return # fused into the on-screen overlay instead of a desktop bubble
self._notify(title, body, urgency=urgency)
def _rnotify(self, title: str, body: str = "", urgency: str = "normal") -> None: def _rnotify(self, title: str, body: str = "", urgency: str = "normal") -> None:
"""Routing feedback — which preset/keyword a voice command matched. Shown """Routing feedback — which preset/keyword a voice command matched. When an
even for hands-free sessions (it has its own toggle) so you can always see overlay is present this is shown there (via routing_cb) instead of a
what you triggered. Only fires on a real match, so it never spams silence.""" notification; otherwise it pops a bubble. Only fires on a real match."""
if self._overlay:
return # shown on the overlay banner instead
notify(title, body, urgency=urgency, enabled=self.cfg.notify_routing) notify(title, body, urgency=urgency, enabled=self.cfg.notify_routing)
def _emit(self, state: str, workflow: str | None = None, message: str = "") -> None: def _emit(self, state: str, workflow: str | None = None, message: str = "") -> None:
@ -456,6 +470,13 @@ class Daemon:
icon = (getattr(target, "icon", "") or "🎙") if target else "🎙" icon = (getattr(target, "icon", "") or "🎙") if target else "🎙"
via = f"{res.keyword}" if res.keyword else "no keyword → default" via = f"{res.keyword}" if res.keyword else "no keyword → default"
self._emit("busy", label, f"{label} ({via})") self._emit("busy", label, f"{label} ({via})")
# Fuse the match onto the overlay (icon + preset + keyword); falls
# back to a desktop notification only when there's no overlay.
if self.routing_cb:
try:
self.routing_cb(icon, label, res.keyword)
except Exception: # noqa: BLE001 - UI must not break the engine
pass
self._rnotify(f"{icon} {label}", f"matched: {via}") self._rnotify(f"{icon} {label}", f"matched: {via}")
log(f"→ routed to {label} (matched: {via})") log(f"→ routed to {label} (matched: {via})")
else: else:
@ -468,6 +489,17 @@ class Daemon:
return return
self._emit("busy", label, "Rewriting…") self._emit("busy", label, "Rewriting…")
self._dnotify(f"{label}", "Rewriting…") self._dnotify(f"{label}", "Rewriting…")
# Stream the rewrite into the overlay so you watch the model write
# (the bubble updates token-by-token). The delivered text is still
# the complete result, typed once the rewrite finishes.
on_token = None
if self.text_cb:
acc: list[str] = []
def on_token(delta: str, _acc=acc) -> None:
_acc.append(delta)
self.text_cb("".join(_acc))
try: try:
text = llm.chat( text = llm.chat(
self.cfg.active_llm, self.cfg.active_llm,
@ -476,6 +508,7 @@ class Daemon:
model=target.model or None, model=target.model or None,
temperature=target.temperature, temperature=target.temperature,
timeout=self.cfg.timeout, timeout=self.cfg.timeout,
on_token=on_token,
) )
except LLMError as exc: except LLMError as exc:
self._emit("error", label, str(exc)) self._emit("error", label, str(exc))

View File

@ -108,6 +108,9 @@ class App:
cfg, status_cb=self._status_cb, cfg, status_cb=self._status_cb,
level_cb=self._on_level, text_cb=self._on_text, level_cb=self._on_level, text_cb=self._on_text,
countdown_cb=self._on_countdown, countdown_cb=self._on_countdown,
# Only consume routing on the overlay when there's an overlay to show
# it on; otherwise the daemon keeps the desktop notification.
routing_cb=self._on_routing if self.overlay is not None else None,
) )
_install_css() _install_css()
@ -276,6 +279,10 @@ class App:
if self.overlay is not None: if self.overlay is not None:
self.overlay.set_countdown(remaining, total) self.overlay.set_countdown(remaining, total)
def _on_routing(self, icon: str, name: str, keyword: str | None) -> None:
if self.overlay is not None:
self.overlay.set_preset(icon, name, keyword)
def _overlay_status(self, state: str, message: str) -> None: def _overlay_status(self, state: str, message: str) -> None:
"""Translate engine phases into overlay show/update/hide (GTK thread).""" """Translate engine phases into overlay show/update/hide (GTK thread)."""
ov = self.overlay ov = self.overlay
@ -287,7 +294,10 @@ class App:
ov.show(state, getattr(self.daemon, "_target_window", None)) ov.show(state, getattr(self.daemon, "_target_window", None))
elif state == "busy": elif state == "busy":
self._ov_state = state self._ov_state = state
ov.set_state("busy", message) # The routing detail ("→ Nicer email (matched: …)") is shown on the
# preset banner, not as a phase chip — keep the chip a clean phase word.
phase = "Transcribing…" if message.startswith("") else message
ov.set_state("busy", phase)
elif state == "done": elif state == "done":
# Non-streaming: the 'done' message carries the final text. Streaming # Non-streaming: the 'done' message carries the final text. Streaming
# already showed it live, so don't overwrite with "Streaming stopped". # already showed it live, so don't overwrite with "Streaming stopped".

View File

@ -13,6 +13,7 @@ import os
import urllib.error import urllib.error
import urllib.request import urllib.request
from dataclasses import dataclass from dataclasses import dataclass
from typing import Callable
from .stt import reachable from .stt import reachable
@ -48,18 +49,28 @@ def chat(
model: str | None = None, model: str | None = None,
temperature: float | None = None, temperature: float | None = None,
timeout: int = 45, timeout: int = 45,
on_token: Callable[[str], None] | None = None,
) -> str: ) -> str:
"""Run a chat completion and return the full text.
When ``on_token`` is given, the request is streamed and each content delta is
handed to the callback as it arrives (so a UI can show the model writing in
real time). The callback is best-effort it never affects the return value,
which is always the complete, stripped response.
"""
stream = on_token is not None
api_key = engine.api_key api_key = engine.api_key
payload = json.dumps( body_obj = {
{ "model": model or engine.model,
"model": model or engine.model, "temperature": engine.temperature if temperature is None else temperature,
"temperature": engine.temperature if temperature is None else temperature, "messages": [
"messages": [ {"role": "system", "content": system_prompt},
{"role": "system", "content": system_prompt}, {"role": "user", "content": user_text},
{"role": "user", "content": user_text}, ],
], }
} if stream:
).encode("utf-8") body_obj["stream"] = True
payload = json.dumps(body_obj).encode("utf-8")
headers = {"Content-Type": "application/json"} headers = {"Content-Type": "application/json"}
if api_key: if api_key:
@ -70,19 +81,46 @@ def chat(
) )
try: try:
with urllib.request.urlopen(req, timeout=timeout) as resp: with urllib.request.urlopen(req, timeout=timeout) as resp:
body = json.loads(resp.read().decode("utf-8")) if stream:
content = _read_stream(resp, on_token)
else:
body = json.loads(resp.read().decode("utf-8"))
content = body["choices"][0]["message"]["content"]
except urllib.error.HTTPError as exc: except urllib.error.HTTPError as exc:
detail = exc.read().decode("utf-8", "replace")[:300] detail = exc.read().decode("utf-8", "replace")[:300]
raise LLMError(f"HTTP {exc.code}: {detail}") from exc raise LLMError(f"HTTP {exc.code}: {detail}") from exc
except urllib.error.URLError as exc: except urllib.error.URLError as exc:
raise LLMError(f"Connection failed: {exc.reason}") from exc raise LLMError(f"Connection failed: {exc.reason}") from exc
try:
content = body["choices"][0]["message"]["content"]
except (KeyError, IndexError, TypeError) as exc: except (KeyError, IndexError, TypeError) as exc:
raise LLMError(f"Unexpected response: {str(body)[:300]}") from exc raise LLMError(f"Unexpected response: {exc}") from exc
content = (content or "").strip() content = (content or "").strip()
if not content: if not content:
raise LLMError("Empty response from model.") raise LLMError("Empty response from model.")
return content return content
def _read_stream(resp, on_token: Callable[[str], None]) -> str:
"""Parse an OpenAI-style SSE stream, returning the accumulated content and
feeding each delta to ``on_token``. Tolerant of keep-alive blanks and the
trailing ``[DONE]`` sentinel."""
parts: list[str] = []
for raw in resp:
line = raw.decode("utf-8", "replace").strip()
if not line or not line.startswith("data:"):
continue
data = line[len("data:"):].strip()
if data == "[DONE]":
break
try:
obj = json.loads(data)
delta = obj["choices"][0]["delta"].get("content")
except (ValueError, KeyError, IndexError, TypeError):
continue
if delta:
parts.append(delta)
try:
on_token(delta)
except Exception: # noqa: BLE001 - UI hiccups must not break delivery
pass
return "".join(parts)

View File

@ -40,6 +40,7 @@ _TAIL_W = 20
_TAIL_H = 11 _TAIL_H = 11
_GAP = 12 # clearance between the tail tip and the anchor _GAP = 12 # clearance between the tail tip and the anchor
_BARS = 30 # waveform bar count _BARS = 30 # waveform bar count
_PRESET_H = 22 # matched-preset header row (emoji + name + keyword)
_MIN_TEXT_H = 0 _MIN_TEXT_H = 0
_MAX_TEXT_H = 120 _MAX_TEXT_H = 120
@ -62,6 +63,12 @@ class Overlay:
self._state = "recording" self._state = "recording"
self._text = "" self._text = ""
self._phase_label = "Listening…" self._phase_label = "Listening…"
# Matched-preset banner (fused in from voice routing instead of a separate
# desktop notification): the preset's emoji, its name, and the spoken
# keyword that selected it.
self._preset_icon = ""
self._preset_name = ""
self._keyword = ""
self._anchor: caret.Anchor | None = None self._anchor: caret.Anchor | None = None
self._tail_up = False # tail on top edge (bubble below anchor)? self._tail_up = False # tail on top edge (bubble below anchor)?
self._tail_x = _WIDTH // 2 # tail tip, window-local x self._tail_x = _WIDTH // 2 # tail tip, window-local x
@ -118,6 +125,11 @@ class Overlay:
def set_text(self, text: str) -> None: def set_text(self, text: str) -> None:
GLib.idle_add(self._set_text, text or "") GLib.idle_add(self._set_text, text or "")
def set_preset(self, icon: str, name: str, keyword: str | None) -> None:
"""Show the matched voice-routing preset on the overlay (emoji + name +
the spoken keyword), in place of a separate desktop notification."""
GLib.idle_add(self._set_preset, icon or "", name or "", keyword or "")
def set_countdown(self, remaining: float | None, total: float) -> None: def set_countdown(self, remaining: float | None, total: float) -> None:
"""Silence auto-stop progress: ``remaining`` seconds until it fires over """Silence auto-stop progress: ``remaining`` seconds until it fires over
a ``total``-second window, or ``None`` while you're still speaking.""" a ``total``-second window, or ``None`` while you're still speaking."""
@ -138,6 +150,9 @@ class Overlay:
self._state = state self._state = state
self._phase_label = _PHASES.get(state, ((1, 1, 1), ""))[1] self._phase_label = _PHASES.get(state, ((1, 1, 1), ""))[1]
self._text = "" self._text = ""
self._preset_icon = ""
self._preset_name = ""
self._keyword = ""
self._levels = deque([0.0] * _BARS, maxlen=_BARS) self._levels = deque([0.0] * _BARS, maxlen=_BARS)
self._disp = [0.0] * _BARS self._disp = [0.0] * _BARS
self._cd_deadline = None self._cd_deadline = None
@ -163,6 +178,16 @@ class Overlay:
self._area.queue_draw() self._area.queue_draw()
return False return False
def _set_preset(self, icon: str, name: str, keyword: str) -> bool:
if (icon, name, keyword) == (self._preset_icon, self._preset_name, self._keyword):
return False
self._preset_icon = icon
self._preset_name = name
self._keyword = keyword
self._relayout()
self._area.queue_draw()
return False
def _set_countdown(self, remaining: float | None, total: float) -> bool: def _set_countdown(self, remaining: float | None, total: float) -> bool:
if remaining is None: if remaining is None:
self._cd_deadline = None self._cd_deadline = None
@ -173,7 +198,13 @@ class Overlay:
def _set_state(self, state: str, message: str) -> bool: def _set_state(self, state: str, message: str) -> bool:
self._state = state self._state = state
self._phase_label = _PHASES.get(state, ((1, 1, 1), message[:40]))[1] or message[:40] if state == "busy":
# Honour a clean phase word from the caller ("Transcribing…",
# "Rewriting…") so the overlay narrates what's happening; fall back to
# the canned label otherwise.
self._phase_label = (message.strip() or _PHASES["busy"][1])[:28]
else:
self._phase_label = _PHASES.get(state, ((1, 1, 1), message[:40]))[1] or message[:40]
if state not in ("recording", "streaming"): if state not in ("recording", "streaming"):
# The countdown only makes sense while listening; drop it as soon as # The countdown only makes sense while listening; drop it as soon as
# we move on to transcribing / done / idle so the ring doesn't linger. # we move on to transcribing / done / idle so the ring doesn't linger.
@ -255,8 +286,10 @@ class Overlay:
def _relayout(self) -> None: def _relayout(self) -> None:
text_h = self._text_height() text_h = self._text_height()
preset_h = _PRESET_H if self._preset_name else 0
gap_preset = 8 if preset_h else 0
gap_text = 8 if text_h else 0 gap_text = 8 if text_h else 0
body_h = _HEADER_H + gap_text + text_h + 2 * _PAD body_h = _HEADER_H + gap_preset + preset_h + gap_text + text_h + 2 * _PAD
self._height = body_h + _TAIL_H self._height = body_h + _TAIL_H
geo = self._monitor_geo() geo = self._monitor_geo()
@ -314,15 +347,56 @@ class Overlay:
wf_w = w - _PAD - wf_x wf_w = w - _PAD - wf_x
self._draw_wave(cr, wf_x, body_top + _PAD, wf_w, _HEADER_H) self._draw_wave(cr, wf_x, body_top + _PAD, wf_w, _HEADER_H)
# Phase label (top-right, small) when there's room and no text yet. # Phase label by the waveform (top-right) — only when there's no preset
if self._phase_label and not self._text: # banner below (which carries the phase instead) and no text yet.
if self._phase_label and not self._text and not self._preset_name:
self._draw_label(cr, w - _PAD, body_top + _PAD + 12, self._phase_label) self._draw_label(cr, w - _PAD, body_top + _PAD + 12, self._phase_label)
# Recognised text below the header row. y = body_top + _PAD + _HEADER_H
# Matched-preset banner: emoji + name (left), live phase chip (right).
if self._preset_name:
y += 8
self._draw_preset(cr, _PAD, y, w - 2 * _PAD)
y += _PRESET_H
# Recognised / rewritten text below.
if self._text: if self._text:
self._draw_text(cr, _PAD, body_top + _PAD + _HEADER_H + 8, w - 2 * _PAD) y += 8
self._draw_text(cr, _PAD, y, w - 2 * _PAD)
return False return False
def _draw_preset(self, cr, x, y, w) -> None:
"""The matched voice-routing preset, fused onto the overlay in place of a
desktop notification: emoji + name (and the spoken keyword) on the left,
the current phase ("Transcribing…", "Rewriting…") on the right."""
cy = y + _PRESET_H / 2
cursor = x
if self._preset_icon:
ic = self._win.create_pango_layout(self._preset_icon)
ic.set_font_description(Pango.FontDescription("Sans 13"))
iw, ih = ic.get_pixel_size()
cr.set_source_rgba(1, 1, 1, 0.95)
cr.move_to(cursor, cy - ih / 2)
PangoCairo.show_layout(cr, ic)
cursor += iw + 7
name = GLib.markup_escape_text(self._preset_name)
if self._keyword:
kw = GLib.markup_escape_text(self._keyword)
markup = f'<b>{name}</b> <span alpha="55%">“{kw}”</span>'
else:
markup = f"<b>{name}</b>"
layout = self._win.create_pango_layout("")
layout.set_markup(markup, -1)
layout.set_font_description(Pango.FontDescription("Sans 10"))
layout.set_ellipsize(Pango.EllipsizeMode.END)
avail = (x + w) - cursor - 96 # leave room for the phase chip
layout.set_width(max(40, avail) * Pango.SCALE)
_nw, nh = layout.get_pixel_size()
cr.set_source_rgba(0.95, 0.96, 0.99, 0.98)
cr.move_to(cursor, cy - nh / 2)
PangoCairo.show_layout(cr, layout)
if self._phase_label:
self._draw_label(cr, x + w, cy, self._phase_label)
def _bubble_path(self, cr, x, y, w, h) -> None: def _bubble_path(self, cr, x, y, w, h) -> None:
r = _RADIUS r = _RADIUS
cr.new_sub_path() cr.new_sub_path()

View File

@ -108,6 +108,41 @@ def test_matched_preset_announced_even_hands_free(monkeypatch):
assert shown == [], "no announcement when 'Announce matched preset' is off" assert shown == [], "no announcement when 'Announce matched preset' is off"
def test_overlay_fuses_notifications(monkeypatch):
"""With an overlay (routing_cb wired), the on-screen bubble narrates every
phase, so informational desktop notifications are suppressed but errors
still pop a bubble, and the routing match is shown on the overlay, not a
notification."""
calls = []
monkeypatch.setattr(daemon_mod, "notify", lambda *a, **k: calls.append((a, k)))
monkeypatch.setattr(daemon_mod, "detect_recorder", lambda pref="auto": "pw-record")
d = Daemon(Config(), routing_cb=lambda *a: None)
assert d._overlay is True
d._session_silent = False
d._dnotify("⌛ Nicer email", "Rewriting…") # informational
assert calls == [], "informational notifications fuse into the overlay"
d._dnotify("Error", "boom", urgency="critical") # errors still notify
assert len(calls) == 1, "errors must still pop a desktop bubble"
calls.clear()
d.cfg.notify_routing = True
d._rnotify("⚡ Nicer email", "matched: “nicer email”")
assert calls == [], "the match is shown on the overlay banner, not as a notification"
def test_no_overlay_keeps_notifications(monkeypatch):
"""Headless / overlay-off: notifications remain the only feedback."""
calls = []
monkeypatch.setattr(daemon_mod, "notify", lambda *a, **k: calls.append((a, k)))
d = _make_daemon(monkeypatch) # no routing_cb
assert d._overlay is False
d._session_silent = False
d._dnotify("⌛ Nicer email", "Rewriting…")
assert len(calls) == 1, "without an overlay, informational notifications still show"
def test_wakeword_while_busy_does_not_notify(monkeypatch): def test_wakeword_while_busy_does_not_notify(monkeypatch):
"""The away-from-keyboard "Busy" storm: a detection arriving while the """The away-from-keyboard "Busy" storm: a detection arriving while the
previous clip is still being processed must be ignored silently.""" previous clip is still being processed must be ignored silently."""

View File

@ -0,0 +1,41 @@
"""Streaming the LLM rewrite into the overlay (so you watch the model write).
`llm.chat(..., on_token=cb)` must parse an OpenAI-style SSE stream, hand each
content delta to the callback, and still return the complete text.
"""
import io
import blitztext.llm as llm
def test_read_stream_accumulates_and_calls_back():
sse = (
'data: {"choices":[{"delta":{"content":"Hel"}}]}\n'
"\n" # keep-alive blank line
'data: {"choices":[{"delta":{"content":"lo"}}]}\n'
'data: {"choices":[{"delta":{}}]}\n' # role-only / empty delta
": comment line\n" # SSE comment, ignored
"data: [DONE]\n"
'data: {"choices":[{"delta":{"content":"X"}}]}\n' # after DONE -> ignored
)
tokens = []
out = llm._read_stream(io.BytesIO(sse.encode("utf-8")), tokens.append)
assert out == "Hello"
assert tokens == ["Hel", "lo"]
def test_read_stream_survives_callback_errors():
sse = (
'data: {"choices":[{"delta":{"content":"a"}}]}\n'
'data: {"choices":[{"delta":{"content":"b"}}]}\n'
"data: [DONE]\n"
)
def boom(_delta):
raise RuntimeError("UI exploded")
# A failing UI callback must not break accumulation / delivery.
out = llm._read_stream(io.BytesIO(sse.encode("utf-8")), boom)
assert out == "ab"