Compare commits

...

98 Commits
v1.4.0 ... main

Author SHA1 Message Date
Hermes Agent
f678d7d20f security: add path validation for user-configured sound files
Some checks failed
CI / Test Linux app (push) Has been cancelled
Adds validate_sound_path() that checks file type, allowed directories, and known audio extensions. Updates play() to use validation before passing paths to audio players.
2026-06-15 07:38:05 +00:00
8f056e8cf0 feat: cancel button during transcription and rewriting (v2.03.41)
- overlay: show × button in busy state (transcribing/rewriting), not only
  while recording — updates hit-region, draw call, and label layout
- daemon: cancel_dictation() now handles _busy via threading.Event
  (_abort_event); _process() clears the event at start, checks after STT
  returns and after LLM rewrite completes, skipping delivery if set
- llm: chat() and _read_stream() accept abort_event; streaming loop breaks
  immediately when the event is set so cancellation is near-instant

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 17:57:41 +02:00
4915884316 chore: ignore raw screenshot source folders
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 17:46:26 +02:00
e053719c59 docs: screenshots, README and MANUAL overhaul (v2.03.40)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 17:26:19 +02:00
4b8b9db43e docs: add screenshots to README and MANUAL, rename screenshot files
- Renamed 21 raw screenshots to meaningful filenames under Screenshots/
- Updated README.md and linux/README.md: added organized screenshot
  sections (Main panel & overlay, Settings pages, Setup wizard) using
  new filenames with click-to-enlarge links
- Updated MANUAL.md: rewrote intro to reflect new sidebar navigation
  (Presets · General · Keyboard · Wakeword · STT Engines · LLM Engines ·
  Benchmark — STT · Benchmark — Wakeword · Log · Manual · About);
  renamed section headers to match; added screenshot at each section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 17:22:34 +02:00
a9c08a54c8 fix: eliminate remaining horizontal scrollbars, split kw rows, fix WW benchmark layout (v2.03.39)
Scrollbar root causes:
- infobox max_width_chars 72→58 (72 chars × 9px/char > 677px content area)
- engine selector combos (stt/llm/ww) lacked CellRendererText ellipsize;
  long engine names drove combo natural width; add _ellipsize_combo() helper
  (ellipsize=END, max-width-chars=28) applied to _combo() and all 3 engine combos
- _STT_TYPES/_LLM_TYPES/_DEVICE_OPTIONS/_COMPUTE_OPTIONS labels shortened to ≤27 chars

UX:
- _kw_shortcut_row: split into two ListBox rows (keywords / shortcut)
- Benchmark WW: move Run button to top of results pane, paned 390→340px

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 17:01:22 +02:00
125bd17713 fix: entry natural width causes scrollbars in STT Engines and Wakeword (v2.03.38)
Gtk.Entry reports its placeholder text width as natural width. Without
set_width_chars(1), entries can't shrink below that, so pages with long
placeholders (URL fields, keyword rows) ended up wider than the dialog.

Added set_width_chars(1) to _entry(), _url_field(), ModelPicker,
_kw_shortcut_row, and _sound_field. Also added set_max_width_chars(50)
to the stt_result wrapping label for the same reason.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 15:28:44 +02:00
0500b24900 fix: horizontal scrollbars and empty STT Engines page in settings (v2.03.37)
- Guard _refresh_status() with hasattr checks so opening STT Engines
  before LLM Engines page is built no longer crashes the builder silently
- ww_status label: add max_width_chars(30) + ellipsize END so long model
  lists don't widen the Wakeword page
- Benchmark STT sel_sw: NEVER→AUTOMATIC horizontal policy so wide engine
  names scroll internally instead of propagating to the dialog
- _combo()/_type_combo(): set_size_request(10,-1) so ComboBoxText widgets
  (e.g. long microphone names) can shrink below natural width

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 15:16:49 +02:00
8181ad75f9 feat(2.03.36): split STT Engines / LLM Engines into separate sidebar pages
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 14:54:19 +02:00
79d6020fc6 fix(2.03.35): settings pages no longer widen the dialog
_switch_row description labels had set_line_wrap(True) but no
set_max_width_chars, so GTK computed their natural width as the full
un-wrapped text (~700px for 87-char descriptions). With NEVER horizontal
policy on the page ScrolledWindow this propagated to the dialog, making
Keyboard/Wakeword/Engines/Benchmark pages 1000–1360px wide.

Fix: add set_max_width_chars(50) to description labels, and change the
page SW horizontal policy from NEVER to AUTOMATIC as a safety net.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 14:52:28 +02:00
1f3bbf2668 fix(2.03.34): stop settings window growing wide/tall between pages
Stack.set_homogeneous(True) was requesting the max natural size of all
children (incl. wide Benchmark TreeViews) and forcing the dialog to 1000+px
wide for every page. Reverted to False so the window stays at 860×700 and
pages scroll if taller than the viewport.

Also removed the NEVER/NEVER ScrolledWindow policy on both benchmark pages —
it was propagating natural TreeView width up to the Stack, amplifying the
homogeneous bug.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 14:40:42 +02:00
5cd31c8385 fix(2.03.33): stable window size between pages, fix benchmark-ww controls clipped
- Stack.set_homogeneous(True) keeps dialog height constant when switching pages
- Benchmark WW paned position raised 260→390 so all TTS config, engine
  checkboxes, wakeword/samples/run fields are visible without scrolling
- Both benchmark pages disable their page-level ScrolledWindow so the
  Paned fills the viewport rather than growing to natural height

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 12:56:39 +02:00
af34360caa feat(2.03.32): replace Notebook with sidebar+Stack, split Input and Benchmark pages
Settings dialog navigation redesigned — Gtk.Notebook replaced with a 170px
left sidebar (section headers + flat nav buttons) + Gtk.Stack content area
(860×700 default). Lazy-loading preserved.

Input tab split into Keyboard (hotkeys, quality gate, audio cues) and
Wakeword (engine config, cancel/send words, wakeword sound cues).

Benchmark tab split into Benchmark — STT and Benchmark — Wakeword.
All field names and collect logic unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 12:46:33 +02:00
d45ab4e91d feat(2.03.32): replace Notebook with sidebar+Stack, split Input and Benchmark pages
- Replace Gtk.Notebook tab bar with a 170px left sidebar (flat buttons,
  section headers) + Gtk.Stack navigation; dialog widened to 860×700
- Split "Input" page into "Keyboard" (keys, quality gate, audio cues)
  and "Wakeword" (enable switch, mic level, test, engine CRUD, sound cues)
- Split "Benchmark" page into "Benchmark — STT" and "Benchmark — Wakeword"
- Remove _page() helper and bt-nb CSS; add .bt-sidebar-active + .sidebar CSS
- Lazy-loading preserved via _pending_pages dict + _show_page/_build_page
- Meter start guard kept in both _build_keyboard and _build_wakeword

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 12:42:08 +02:00
ca1f408876 feat: first-run setup wizard for new users (v2.03.31)
Paged GTK dialog guides through trigger method, keyboard shortcuts,
wakeword server, STT engine, and optional LLM setup. Shows automatically
on first launch, re-openable via Settings → "Setup Wizard…". Sets
setup_complete in config so it doesn't reappear.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 12:14:06 +02:00
5aafb572ed feat: dedicated wakeword models for cancel and send actions (v2.03.30)
WakewordActionListener opens a second Wyoming connection during active
wakeword recording, listening for the configured cancel/send models. When
either fires it immediately calls cancel_dictation() or finish_dictation()
without any silence timer or Whisper pass. Settings UI adds Cancel model
and Send model pickers to the wakeword config card, populated from the
same server model list as the trigger model.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 12:07:03 +02:00
a77b15ccaf feat: real-time cancel keyword detection during wakeword recording (v2.03.29)
CancelWatcher accumulates raw PCM from the VAD LevelMeter (via new
on_chunk callback) and runs a fast beam_size=1 transcription check every
~0.6s. When a cancel keyword is detected it immediately calls
cancel_dictation() without waiting for the silence timer to expire or
a full transcription to complete.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 11:47:10 +02:00
1c276e767b fix: block scroll-wheel changes on all ComboBoxText dropdowns (v2.03.28)
Hovering over a combo and scrolling could silently change the selection.
All ComboBoxText widgets now return True from their scroll-event handler,
swallowing the event before GTK's default handler can act on it.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 11:38:48 +02:00
016ffe0390 feat: overlay × cancel button + wakeword shortcut fields inline (v2.03.27)
- Overlay HUD shows a × button in the top-right corner during recording/
  streaming; clicking it cancels dictation. The button area is the only
  clickable region — rest stays fully click-through.
- Wakeword tab "Cancel words" and "Send words" rows now include an inline
  keyboard shortcut entry + Set button, so key_cancel / key_send can be
  configured right next to the spoken-word equivalents.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 04:00:23 +02:00
834cc20553 fix: cancel recording works in wakeword mode + tray Cancel item (v2.03.26)
- Cancel hotkey now fires even when wakeword triggered the recording
  (ModifierScheme state was "idle" so the key was silently ignored)
- Tray menu: "✕ Cancel recording" — always visible, enabled while recording,
  grayed out at idle; works for both wakeword and manual recordings

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 03:44:14 +02:00
599eb08c5a feat: smart Save — inline status, no restart for safe changes (v2.03.25)
Save now diffs old vs new config. Safe changes (language, sounds, LLM,
keywords, overlay) show "✓ Applied" in the header for 4s. Restart-required
changes (STT engine, hotkeys, mic, wakeword) show "⚠ restart needed for: …"
and highlight Save & Restart. No modal popup on Save.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 03:29:27 +02:00
9c812a960e fix: Input level meter works without visiting General first (v2.03.24)
_start_meter() was only called from _build_general() and referenced
self.mic_level unconditionally. Now _build_input() also starts the meter
when it's not already running, and both level bars are updated via hasattr.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 02:48:55 +02:00
8301cbb457 fix: wakeword pane actually resizable — wrap controls in ScrolledWindow (v2.03.23)
The controls pane now has shrink=True and a ScrolledWindow wrapper, so
dragging the divider upward collapses the controls and expands the table.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 02:26:32 +02:00
012a069ad0 feat: wakeword benchmark — draggable split pane for results table (v2.03.22)
Controls (TTS config, engine checkboxes, run button) are in the top pane;
the results table is in the bottom pane. Drag the divider to see more rows.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 01:57:31 +02:00
8b26bc3379 feat: wakeword results table — sortable columns + CSV export (v2.03.21)
- All columns sortable by clicking header; numeric cols sort numerically
- "Copy as CSV" → clipboard; "Save CSV…" → file chooser

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 01:47:47 +02:00
6f133771e7 fix: icon vertical centering + wakeword results table (v2.03.20)
- Section header icons were pushed down by bt-section CSS margin-top;
  now only applied to the row container, not the image widget
- Wakeword benchmark shows a full TreeView table: per-voice Detected/Total/
  Recall%/False-fires/Time with colour coding, plus aggregate row per engine

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 01:25:14 +02:00
66e431f50a feat: wakeword engine checkboxes + wakeword model selector in benchmark (v2.03.19)
- Engine checkboxes let you pick which wakeword servers to include in the run
- Wakeword combo selects which model/phrase to test; leave empty for each
  engine's own, pick a specific one (e.g. okay_computer) to override all
- Also fixes: TTS ⟳ no longer fills model combo with Kokoro voice names

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 01:13:54 +02:00
b20aa31261 feat: TTS model fix + per-engine wakeword benchmark results (v2.03.18)
- TTS ⟳ no longer floods model combo with voice names (Kokoro exposes
  voices as /models entries — detected and skipped automatically)
- Wakeword benchmark now shows per-engine Recall/False-fires/time results;
  fixes progress callback signature mismatch that crashed multi-engine runs
- Wakeword fetch feedback: ⟳ shows "Connecting…" then model count/names
- Wakeword Quickstart expanded to ports 10400–10403 + hey_jarvis/alexa
- Wakeword info box added explaining model directory setup

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 00:59:28 +02:00
7a9dc5a2f7 feat: MP3/OGG/FLAC sound support + browse dialog with auto-preview (v2.03.16)
Sound fields accept WAV/MP3/OGG/FLAC/M4A/AAC/AIFF/Opus. Browse dialog
auto-plays each file on selection so you can preview before confirming.
sound.py falls back to ffplay/gst-play-1.0 for formats not supported
by pw-play/paplay.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 00:37:07 +02:00
571dae7cc9 feat: full CRUD for wakeword engine presets (v2.03.15)
Add/Quickstart/Reload/Delete buttons + Name field mirror the STT engines UI.
Four quickstart templates for common wyoming-openwakeword setups.
Migration: existing wakeword_uri/model auto-promoted to first preset.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 00:32:17 +02:00
e831d7f1e6 fix: remove Internal engine section header in Engines tab (v2.03.14)
Device/Compute rows already only show for local engines; the titled section
break was redundant and visually separated fields that belong together.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 00:24:42 +02:00
34f75844c1 feat: Save buttons in header bar, remove Close button, center section icons (v2.03.13)
HeaderBar replaces bottom button row — Save and Save & Restart appear in the
title bar on the right, X button closes. Section header icons vertically
centered using SMALL_TOOLBAR size and valign=CENTER.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 00:22:53 +02:00
ccf3801545 feat: icons on all settings tabs and section headers (v2.03.12)
GTK symbolic icons on every tab label and every section/card header.
Also fixes the resize grip landing in the tab bar instead of the bottom-right corner.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 00:07:41 +02:00
efa44c8872 feat: resize grip indicator on settings window (v2.03.11)
Draws a classic dotted SE-corner grip overlaid on the bottom-right of the
notebook so users know the settings dialog is resizable.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 23:50:20 +02:00
c292ff2242 fix: probe Prometheus /metrics at server root, not under /v1 (v2.03.10)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 23:30:33 +02:00
c3cf123afe feat: server RAM via Prometheus /metrics in benchmark (v2.03.09)
Probe remote engines' /metrics for process_resident_memory_bytes or
container_memory_rss; show actual server-side MB in RAM column.
Falls back to "server" when the endpoint is not exposed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 23:24:40 +02:00
86632e1918 fix: hide Internal Engine section for remote/streaming engines (v2.03.08)
Removes whitespace gap between STT config card and device/precision card
by hiding the latter when a Server or Realtime engine type is selected.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 23:06:32 +02:00
d5e512a812 feat: per-engine timeout field, WhisperX default 120s (v2.03.07)
- STTEngine gains timeout: int = 30 field; used as HTTP timeout per engine
- Engines tab shows "Timeout (s)" field; blank = 30s default
- WhisperX quickstart template pre-sets 120s (diarization is slow)
- config load/save: only written when != 30 to keep config clean
- RAM column header tooltip explains "server" for Docker/remote engines

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 22:48:53 +02:00
2326f20a28 fix: numeric sort for Time, Accuracy, RAM columns (v2.03.06)
String sort put "14.85" before "2.47". Now uses a custom comparator
that strips % and non-numeric markers before comparing as float.
Non-numeric values ("—", "server") sort to the bottom.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 22:37:17 +02:00
05ccfbdeb4 fix: RAM column shows "server" for remote engines (v2.03.05)
Remote engines run models server-side so local RSS never changes.
Show "server" instead of "—" to make the reason clear.
Local engines show measured MB; local already-loaded shows "—".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 22:34:41 +02:00
d40721dd3d fix: reduce transcription timeout to 30s (v2.03.04)
300s was too long; nobody waits 5 min for a result.
30s gives slow remote servers a fair window while still failing fast.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 22:25:17 +02:00
79f3b5cdab fix: API probe URLs, 5min timeout for slow engines like WhisperX (v2.03.03)
- Add _api_base() to strip endpoint-specific path suffixes before probing
  /models, /metadata, /info — fixes /transcribe/models 404 spam when URL
  ends with a custom path like /v1/transcribe
- Increase default transcribe() timeout 60s → 300s — WhisperX with speaker
  diarization (pyannote) takes 2-5 min and was always timing out

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 22:23:13 +02:00
766002a478 fix: WhisperX URL — add template, tooltip clarifying /transcribe path (v2.03.02)
- Add "WhisperX server" quickstart template pre-filled with /transcribe endpoint
- Add tooltip to STT URL field explaining that non-standard servers (WhisperX)
  should use the full endpoint as the URL, not /v1
- _url_field_lb accepts optional tooltip= kwarg

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 22:16:21 +02:00
d9603c98bf feat: RAM usage column in benchmark, docs update (v2.03.01)
- benchmark.py: measure RSS delta via /proc/self/status before/after each
  transcription; add ram_mb field to BenchRow
- gtksettings.py: add RAM (MB) column to results table (index 8); tooltip
  column shifted to index 10
- CHANGELOG.md: full history from v2.02.00 through v2.03.01
- README.md: benchmark description updated to mention RAM column
- MANUAL.md: benchmark result columns as table; Log tab documents
  level filter dropdown

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 22:11:38 +02:00
f5f65ffba9 fix: stability, WhisperX 404, log levels (v2.03.00)
- Stability: remove blocking TCP socket call from _collect() — was
  freezing the GTK main thread on Save when wakeword server unreachable;
  now checks async in background and logs result
- Thread safety: fix _ww_load() reading GTK widget from background thread;
  capture URI on main thread before spawning
- WhisperX 404: _transcribe_remote now detects non-standard paths
  (anything other than /v1) and uses the URL as the full endpoint,
  so http://host/transcribe works without /audio/transcriptions appended
- Log levels: logbuffer stores (ts, level, msg) tuples; log() accepts
  level= (DEBUG/INFO/WARNING/ERROR); Log tab gets a Level dropdown
  (Verbose/Info/Warning/Error) that filters displayed entries live

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 22:01:14 +02:00
6fb41d5a9b feat: wakeword server preset dropdown in Input tab (v2.02.03)
Adds a "Server preset" combo in the Hands-free wakeword card that lists
all configured wakeword engines by name. Selecting one auto-fills the
URI and model fields and re-probes the connection. Selection persists
via wakeword_active in config.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 21:01:57 +02:00
48014ae662 fix: license in md style, benchmark pane minimum height (v2.02.02)
- About tab: render License with _md_panel instead of _text_panel
- Benchmark pane: set_size_request 320px min, shrink=False on both sides
  so the engine list and results table are always visible

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 20:55:57 +02:00
9b95c632fa feat: show last benchmark time+accuracy on selected STT engine (v2.02.01)
After running a benchmark, each engine's best result (time, accuracy)
is persisted to config and shown as a small info line in the Engines tab
when that engine is selected. Updates live as the benchmark runs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 20:42:22 +02:00
2df5be3212 feat: language metadata in benchmark — Lang column, filter support (v2.02.00)
- stt.py: add ModelMeta dataclass, fmt_languages(), list_models_meta(),
  detect_remote_device(); refactor list_models() to delegate
- benchmark.py: add languages field to BenchRow; fetch via _get_langs()
  with URL-level caching using list_models_meta()
- gtksettings.py: show language labels per engine in checkbox list;
  add language codes to search filter; add Lang column to results table

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 20:02:12 +02:00
51ab6d5aff ui: larger, more visible tab labels (v2.01.03)
- Tab font-size 12px → 14px (matches body text)
- Inactive tabs: muted foreground color so they're clearly readable
  but visually distinct from the active tab
- Active tab: bold + blue (#1a73e8), slightly more padding (8px 18px)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 19:34:48 +02:00
8d79c84a8f chore: adopt x.xx.xx versioning schema (v2.01.02)
Schema: MAJOR.FEATURE.FIX
  MAJOR  — breaking changes / major redesign
  FEATURE — two-digit, new user-visible features (00-99)
  FIX     — two-digit, bug fixes within a feature release (00-99)

Renamed 2.1.2 → 2.01.02 to start the new format.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 19:27:13 +02:00
81b79299ea fix: _run_bench crashes when Engines tab was never opened (v2.1.2)
_stt_commit() accesses self.stt_name which only exists after the Engines
tab is lazily built. Guard with hasattr so running a benchmark directly
from the Benchmark tab no longer throws AttributeError.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 19:21:56 +02:00
0eafb59d82 benchmark: resizable engine/results pane; auto-save WAV+ref paths (v2.1.1)
- Engine checklist and results table now split by a Gtk.Paned (vertical)
  so the user can drag the divider to give more room to either panel
- WAV and reference .txt paths are written to disk (save()) the moment a
  file is picked via the file chooser, without needing to click Save;
  also saved on Run if they changed since the last disk write

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 19:18:34 +02:00
e39b253906 benchmark: engine checkbox selector + status dots; manual/about/templates (v2.0.9)
- Benchmark tab: scrollable engine checklist above Run with live
  reachability dots (green/red), name+model+URL filter, All/None buttons;
  _run_bench respects selection and shows clear message when nothing ticked
- About tab: changelog now rendered via _md_panel (markdown headers, bold,
  lists) instead of plain monospace _text_panel
- Manual tab: graceful fallback with clickable GitHub link when MANUAL.md
  is not installed; build-deb.sh now copies MANUAL.md from repo root so
  /opt/blitztext/MANUAL.md exists in future installs
- STT quickstart templates expanded: Speaches docker, whisper.cpp server,
  NVIDIA NIM/Parakeet, five built-in local model sizes (tiny→large-v3)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 19:06:08 +02:00
a6d960bfbf benchmark: sortable columns (v2.0.8)
Wrap ListStore in TreeModelSort and set sort_column_id on every column
so clicking any header sorts ascending/descending.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 18:36:27 +02:00
5458fdc1eb engines: Test button inline with result + selectable; benchmark: persist WAV/ref paths (v2.0.7)
- Engines tab: move Test button out of the toolbar, place it in a row
  directly beside the result label so button and output are co-located;
  result label is now selectable so error text can be copied
- Benchmark tab: add bench_wav/bench_ref/bench_expand_models to Config;
  file pickers restore last-used paths on open; any change (file-set,
  toggle, or Run) writes directly to cfg so paths survive without Save;
  [benchmark] section written to config.toml on Save

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 18:29:01 +02:00
1c16cfeb0f benchmark: remote GPU detection, URL column, test-all-models (v2.0.6)
- stt.detect_remote_device(): probes /info (faster-whisper-server) then
  /metadata (NVIDIA NIM) to detect CUDA vs CPU; cached per unique URL
- BenchRow gains url field; Device column now shows "CUDA" for GPU remotes
  instead of the generic "remote"
- Benchmark table gains URL column (scheme stripped, max 180px wide)
- "Test all models per engine" checkbox: fetches list_models() for each
  remote engine and expands to one row per model when checked
- benchmark.run() gains expand_models parameter

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 18:19:42 +02:00
bc6e2a6e54 benchmark: deduplicate engines, friendly error messages, tooltip for full error (v2.0.5)
- _run_bench: deduplicate STT engines by name before running; show a
  warning in the summary line listing which names were skipped
- _bench_add_row: replace raw HTTP error strings with human-readable
  reasons ("Wrong model name", "Server offline", "Timed out", etc.);
  full raw error stored in hidden column 7 shown as row tooltip on hover
- bench_store: added 8th column (tooltip text); tree.set_tooltip_column(7)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 18:12:09 +02:00
e01b130e5f build: auto-copy .deb to ~ with correct 644 permissions
install -m644 guarantees world-readable regardless of umask, so
`sudo apt install ~/blitztext_*.deb` works without a manual chmod.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 17:33:08 +02:00
4d59853da6 benchmark: add 'Best for' column, rename GPU→CUDA (v2.0.4)
- BenchRow gains `best_for` field: "Short clips" / "Short / medium" /
  "Long / batch" / "Streaming" — derived from engine type and model name
- Device now shows "CUDA" instead of "GPU" for clarity
- Benchmark table gains a "Best for" column between Device and Time(s)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 17:28:55 +02:00
0ff0f604c2 stt: omit model field when empty (fixes Riva 400); settings: always-visible scrollbar (v2.0.3)
- stt._transcribe_remote: no longer falls back to "whisper-1" when
  engine.model is empty — omits the field entirely so Riva/NIM uses its
  default model instead of rejecting the request with HTTP 400
- gtksettings._page: switch vertical scroll policy to ALWAYS and disable
  overlay scrolling so the scrollbar is permanently visible, making
  it obvious when a tab has more content below the visible area

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 17:00:58 +02:00
101bb1e796 engines: detect Riva/NIM model from /metadata, show placeholder on empty fetch
- stt.list_models(): fall back to /metadata (NVIDIA Riva/NIM) when /models
  returns nothing — extracts shortName and strips the version tag suffix
- Settings: show "type model name manually" placeholder when fetch returns
  empty and no model is currently set

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 16:49:58 +02:00
06167a1e6d build: chmod 644 the .deb so apt's _apt user can read it
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 16:07:51 +02:00
6534284c56 bump version to 2.0.2
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 16:04:38 +02:00
4f68eda501 fix: suppress numpy Pylance false positive in wakeword_bench.py
numpy is a runtime dep bundled in the .deb venv but not in the dev
.venv, so Pylance can't resolve it at edit time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 16:02:50 +02:00
9022b17186 fix: Pylance errors in gtksettings.py + pyrightconfig venv path
- str() wrap float config values passed to _entry() (min_speech_seconds,
  silence_rms, wakeword_silence_seconds) — type error + runtime safety
- Guard m.group() calls after re.match() with 'if m:' — m can be None
- Fix _bench_done: guard 'acc' None check alongside 'fastest'
- Fix wakeword_bench.run() call: pass engines as first positional arg,
  remove invalid wakeword_model/wakeword_uri kwargs
- gi.repository import: add type: ignore[import-untyped] to suppress
  Pylance false positives from missing C-extension stubs
- pyrightconfig.json: point venv to linux/.venv so numpy resolves;
  add reportMissingModuleSource=none for stub-less system packages

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 16:02:07 +02:00
aea24400af fix: auto-paste long/multiline output, O(n²) overlay accumulation (v2.0.1)
Root cause of session freeze confirmed: a 15 000-char code block was typed
character-by-character via xdotool at 12ms/char = ~3 min, flooding the X11
per-client event buffer until the entire session froze.

- paste.py: any text >300 chars or containing newlines auto-upgrades to
  clipboard paste (instant Ctrl+V) regardless of configured output mode.
  xdotool type is kept only for short single-line text where it matters.
- daemon.py: replace on_token "".join(acc) accumulation (O(n²) for long
  code blocks) with a sliding deque that shows only the last 400 chars in
  the overlay — Pango no longer re-lays out a growing 15 KB string on each
  incoming token.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 15:56:44 +02:00
73264af2be fix SyntaxWarning: escape backslash in SIGQUIT docstring
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 15:38:39 +02:00
99340b0ff3 bump version to 2.0.0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 15:35:28 +02:00
0d1311ba18 stability: fix GTK thread safety, wakeword hang, freeze diagnostic (v2.0.0)
- daemon: GLib.timeout_add (thinking animation) was called from a background
  thread — NOT thread-safe in PyGObject/GTK3 and the prime suspect for the
  session-level freezes. Fixed via idle_add so timeout_add runs on main thread.
- daemon: add SIGQUIT handler — kill -QUIT <pid> dumps all thread stacks to
  the log so the freeze location can be identified without rebooting.
- daemon: VAD hard-timeout safety net — if LevelMeter fails to open the mic
  (device busy because wakeword pw-record holds it), dictation now auto-stops
  after 30s instead of hanging forever. Wakeword sessions also capped at 60s.
- overlay: coalesce set_level() calls the same way set_text() is coalesced,
  so mic level spikes from 3 simultaneous pw-record streams can't flood the
  GTK main loop either.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 15:29:45 +02:00
59118364fb bump version to 1.9.9
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 15:10:22 +02:00
0875fe7ac0 llm: catch all exceptions, validate response, show transcription preview (v1.9.9)
- llm.py: catch OSError/TimeoutError (socket.timeout is NOT a URLError subclass)
  and bare Exception so no path kills the background thread silently
- daemon.py: catch unexpected exceptions from llm.chat and surface them as
  overlay errors instead of silent thread death
- daemon.py: reject LLM responses that are >80 % whitespace (cold-start
  model returning spaces/blank lines gets discarded, not typed)
- daemon.py: show transcribed text in overlay immediately before thinking
  animation so user can see what was heard while LLM processes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 15:02:45 +02:00
749ef78383 dictation: spoken newlines + LLM thinking animation (v1.9.8)
- Add expand_spoken_punctuation() in quality.py: converts spoken phrases
  "new line" / "neue Zeile" → \n and "new paragraph" / "neuer Absatz" → \n\n
  before text is typed; xdotool/wtype handle \n as Return
- Daemon: pulse " Thinking..." animation in the overlay while waiting for
  the first LLM token (400ms interval), replaced automatically when streaming
  starts so the user sees activity during cold-start latency

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 14:43:15 +02:00
946d344090 presets: per-preset LLM engine selector + fix streaming freeze (v1.9.7)
- Add llm_engine field to Workflow (config.py load/save)
- Daemon picks the named engine per-preset, falls back to active engine
- Settings Presets tab: replace free-text model field with engine dropdown
  populated from configured engines; "(active engine)" = default
- Fix LLM token streaming crash: coalesce GLib.idle_add calls so only
  one flush is ever queued (prevents GTK main loop flooding)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 14:39:59 +02:00
9e291437bd stability + settings: fix AT-SPI session freeze, markdown Manual, bigger emojis (v1.9.6)
caret: change default overlay_anchor from "caret" to "pointer" — AT-SPI's
synchronous D-Bus extents read on the GTK main loop could freeze the entire
X11 session when the focused app was slow to respond. Pointer anchor is a
reliable fallback (the mouse is almost always near where you're typing) and
avoids AT-SPI entirely by default.

caret: wrap the AT-SPI rect() read in a daemon thread with a hard 300ms
timeout — so even if overlay_anchor="caret" is configured manually, a frozen
or unresponsive target app can never stall the GTK main loop.

settings: replace the Manual tab's plain TextView with a styled markdown
renderer (_md_panel): # h1/h2/h3 headings, **bold**, *italic*, `inline code`,
> blockquotes, --- rules, bullet/numbered lists, and | tables |.

settings: infobox background changed from theme-adaptive grey (unreadable on
dark themes) to a fixed blue tint (rgba 66,133,244 @ 9%) with a matching blue
border — readable in both light and dark themes.

settings: active notebook tab now shows bold text with a #1a73e8 blue colour
and extra padding, making the current tab obviously distinct.

settings: emoji picker buttons enlarged to 20 px / 38×38 px minimum (grid)
and 18 px / 34×34 px (category bar) via bt-emoji-btn / bt-emoji-cat-btn CSS
classes — previously rendered at the default button label size (~11 px).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 14:20:05 +02:00
48be824a50 settings: emoji search, Manual tab fix, readable infobox (v1.9.5)
- Emoji picker: SearchEntry at top filters all categories via unicodedata.name()
  in real time; category view hides while searching, restores on clear
- Manual tab: add pkg_dir/MANUAL.md to _app_paths() search list so it works in
  both venv and deb installs (MANUAL.md deployed alongside the package)
- bt-infobox CSS: replace @theme_selected_bg_color (saturated blue) with a
  neutral 5% mix of fg/bg so banner text is always readable

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 13:09:42 +02:00
2fe6632269 settings: complete UI redesign — card layout, section titles, scrollable tabs (v1.9.4)
All settings tabs now use a card-based GTK3 layout:
- Related fields grouped into Gtk.ListBox cards with rounded borders (bt-card /
  boxed-list CSS) and bold all-caps section headers via _card_section helper
- CSS provider injected at init: .bt-section, .bt-card, .bt-infobox styles
- Each notebook page wraps content in a ScrolledWindow (740×700 px dialog)
- _labeled and _switch_row accept both Gtk.Box and Gtk.ListBox parents
- New helpers: _section_title, _card_section, _lb_add, _url_field_lb,
  _icon_field_lb, _key_field_lb
- Engines toolbars: creation buttons (+ Add/Stream/Quickstart) left-aligned,
  action buttons (Delete/Test/⟳) right-aligned via pack_end
- Presets: Identity / Trigger / Behaviour / Prompt cards
- Input: Input mode & keys / Quality gate / Wakeword / Sound cues cards
- General: Microphone / Output & language / Notifications / Startup cards
- Section names cleaned up (no more "WW -" prefix, cleaner device label)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 12:18:50 +02:00
dd617f625e settings: info buttons, Manual tab, engine quickstart templates, human-readable type names (v1.9.3)
- Every _labeled and _switch_row field now gets a clickable ⓘ info button
  that opens a plain-language help popover — targeted at non-technical users
- New Manual tab in Settings renders MANUAL.md directly inside the dialog
- STT and LLM toolbars each get a "Quickstart ▾" button: a menu of common
  providers (OpenAI, Groq, OpenRouter, Ollama, LM Studio, vLLM, llama-swap,
  faster-whisper-server, NVIDIA Riva) that pre-fills the engine form in one click
- Engine type combos now show human-readable labels ("Internal — faster-whisper",
  "LAN server — runs on your machine", "GPU (CUDA)", "int8 — fast, less memory")
  while storing the same internal key values (no config migration needed)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 11:48:02 +02:00
d2c6d352a5 presets: replace emoji picker with categorized WhatsApp-style grid
Replaces the 60-emoji flat list with 8 standard Unicode categories
(Smileys, People, Animals, Food, Travel, Activities, Objects, Symbols),
each with 30–100 emojis. The picker now has a category tab bar at the top
(emoji icons, highlighted active tab) and a scrollable emoji grid below —
matching the layout used in WhatsApp and other messaging apps.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 11:33:30 +02:00
cc589f254b presets: add emoji picker to the Icon field (v1.9.2)
Adds a 😀 button next to the Icon (emoji) entry in Settings → Presets.
Clicking it opens a GTK popover with 60 common emojis in a scrollable
flow grid; selecting one writes it into the field and closes the picker.
The entry still accepts direct keyboard input as before.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 10:32:11 +02:00
7cbd828781 gtksettings: fix NameError in ww_load error lambda (Python 3.12 scoping)
In Python 3.12 the 'except ... as e' variable is deleted after the except
block exits, so a plain lambda that closes over 'e' raises NameError when
GLib fires it later. Bind e as a default argument at lambda creation time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 21:21:43 +02:00
b8fde76d7e install: fix glob-in-redirect for .pth file; resolve site-packages via Python
Globs don't expand in redirect targets. Use the venv Python to resolve the
exact site-packages path instead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 20:55:32 +02:00
836287d4f4 install: add .pth for /usr/lib/python3/dist-packages (fixes gi import in venv)
On Debian/Ubuntu, python3-gi is installed into the non-versioned
/usr/lib/python3/dist-packages path which venvs exclude even with
--system-site-packages. A debian-system-packages.pth file adds it to
sys.path so `import gi` works from the venv.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 20:49:54 +02:00
f877a76550 install: use Python entry-point script instead of python -m for launcher
'python -m blitztext' adds cwd to sys.path, so running blitztext from inside
a directory that contains a blitztext/ folder shadows the installed package.
A proper shebang entry-point script sets sys.path[0] to the script's own
directory (venv/bin), making the launcher cwd-independent.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 20:41:30 +02:00
2303dd8f78 install: fix apt package array (quoted string caused single-arg error)
Each apt package must be a separate array element; passing them as one quoted
string caused apt-get to fail with "unable to locate package <all names>".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 20:30:50 +02:00
15f8c1d9f7 install: auto-install python3-gi and other missing apt deps
Check for python3-gi (PyGObject), xdotool, libnotify-bin, and pipewire-bin
before creating the venv; if any are missing, run sudo apt-get install
automatically so the install works out of the box on a fresh system.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 20:24:41 +02:00
0ce4f8e7ac install: venv at ~/.local/lib/blitztext with --copies; source-independent after install
Moves the venv out of the source tree to ~/.local/lib/blitztext so it works on
CIFS/SMB shares (--copies avoids the lib64 symlink that CIFS rejects). Copies
the blitztext package into site-packages so the cloned source folder can be
deleted immediately after running install.sh. Adds a ~/.local/bin/blitztext
launcher and installs hicolor icons.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 20:17:07 +02:00
b0b7a8c95e gtksettings: align Input tab labels to uniform 175px width; autostart: add 12s delay
All rows in the Input tab (labeled, key_field, url_field, sound_field) now share
a single LW=175 label width, eliminating the jagged left edge on input fields.
Added X-GNOME-Autostart-Delay=12 to the .desktop entry so gnome-shell, AT-SPI,
and the input stack are fully initialised before Blitztext connects — fixes the
session crash on first login after installation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 20:00:33 +02:00
a472ce62ba settings: lazy-build tabs (instant open); move connection dots beside the field; single-instance Settings dialog; Release 1.9.1
- gtksettings: build each notebook tab on first view instead of all up front, so
  the dialog opens instantly (was ~1.3s building Input/Benchmark file-choosers);
  _collect() force-builds unvisited tabs before saving so no field is missed.
- gtksettings: connection dot now sits left of the URL entry (like the Engines
  tab) instead of at the far right.
- gtkui: open_settings raises the existing dialog instead of opening a second.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 19:30:08 +02:00
27cf81de46 gtksettings: connection dots for the Wakeword (Wyoming) + TTS URLs; rename "Wyoming URI" to "Wakeword engine"; Release 1.9.0
Green/red/grey reachability dot next to the wakeword and TTS endpoint fields,
matching the existing STT/LLM engine dots. Lightweight background TCP probe,
refreshed on open, on reload (⟳), and on focus-out — never blocks dialog build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 19:15:29 +02:00
0d4ad0920d gtkui: force GIO unix volume monitor so file choosers don't hang on dbus; Settings + panel open again; Release 1.8.1
On headless/minimal desktops the gvfs org.gtk.vfs.UDisks2VolumeMonitor dbus
service often fails to activate; each Gtk.FileChooserButton then blocked ~25s on
a StartServiceByName timeout while realizing, so the Settings dialog never
appeared and the stalled main loop froze the control panel too. Set
GIO_USE_VOLUME_MONITOR=unix in run_gui() before any window is realized.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 14:30:48 +02:00
28020fdeda routing: send-by-voice keyword (types + presses Enter); wakeword benchmark via TTS; Release 1.8.0
- Send by voice: a configured edge-anchored phrase (e.g. "computer send") is
  stripped and the rest is delivered AND submitted with Enter. Off by default;
  [routing] send_keywords + Settings → Input.
- Wakeword benchmark (Settings → Benchmark): synthesize the wake phrase in
  random voices via any OpenAI-compatible TTS server, stream to
  wyoming-openwakeword, report recall / false-fires / per-voice breakdown.
  New [tts] config block.
- Tests: test_voice_send.py, test_wakeword_bench.py.
- Ignore agent workspace folder jules/.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 13:17:33 +02:00
1c64f19bce overlay: drive waveform + silence countdown via pw-record (fix PipeWire); app reports as "Blitztext" not __main__.py; Release 1.7.1
The live waveform and silence auto-stop countdown were driven by a level
meter that was the last user of sounddevice/PortAudio, which hangs opening
the default input on PipeWire systems — so both stayed blank on the hotkey
and wakeword paths alike. Rewrite LevelMeter to stream raw PCM from the same
recorder as the WAV path (pw-record/parecord/arecord) and RMS it; identical
API, scaling, and ~10 Hz cadence. Also fixes the Settings mic-level preview.

Set GLib prgname/application name to "Blitztext" before any window is
realized (and add StartupWMClass to the .desktop) so the taskbar and GNOME's
"… is not responding" dialog show the app name instead of "__main__.py",
without touching the `python -m blitztext` entry point.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 08:39:37 +02:00
1567202e1a routing: spoken cancel keyword to discard a clip; Release 1.7.0
Add a configurable voice cancel: saying "abbrechen" (or "cancel") at the start
or end of a clip discards the whole dictation — it is never routed onward,
rewritten, or typed. The rescue for accidentally triggered (e.g. wakeword)
recordings. Matched the same edge-anchored, ASR-tolerant way as routing keywords
via routing.is_cancel(), so the word buried mid-sentence won't trip it; checked
in Daemon._process right after transcription, before routing/rewrite/delivery.
Configurable via [routing] cancel_keywords (default ["abbrechen", "cancel"];
empty disables) and Settings -> Mic/Cues -> "Cancel words". The overlay briefly
shows "Abgebrochen". Docs (both READMEs) and CHANGELOG updated; tests cover the
matcher, the config round-trip, and the discard/deliver pipeline branches.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:09:07 +02:00
ae01472b77 overlay: fix AT-SPI caret freeze; fuse routing match + live LLM into HUD; Release 1.6.0
Fix a desktop-session freeze (forced logout/reboot) caused by the overlay's
AT-SPI caret tracker: it subscribed to the high-frequency object:text-caret-moved
signal and made synchronous, blocking AT-SPI reads from inside the event handler,
re-entering the a11y dispatcher and getting stormed by the app's own xdotool
typing until GNOME stopped responding. Now track focus changes only and read the
caret rectangle lazily, once, when the overlay shows — never on the hot path.

Fuse voice-routing feedback into the overlay instead of a desktop notification:
show the matched preset's emoji, name, and spoken keyword on a banner, narrate
the phase (Transcribing -> Rewriting), and stream the LLM rewrite into the bubble
token-by-token. Redundant per-dictation notifications are suppressed when the
overlay is present (errors still notify); headless/overlay-off is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 16:58:47 +02:00
6a9c1fc95a overlay: silence auto-stop countdown ring around the mic; Release 1.5.1
A full circle wraps the mic glyph and drains clockwise as the trailing-
silence timer runs out, recolouring cyan→amber→red and emptying exactly
as auto-stop fires. The daemon emits the countdown from the same VAD loop
that decides auto-stop, so the ring stays in sync; the overlay drains it
against its own clock for smooth motion and fades it in/out so word gaps
don't flicker it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 15:41:17 +02:00
d55cdafcc1 on-screen dictation overlay at the cursor; Release 1.5.0
Show a translucent, click-through bubble the moment recording starts (by
hotkey or wakeword): a pulsing microphone, a live waveform of the mic
level, and the recognised text (word-by-word when streaming, otherwise a
brief final-result confirmation). The tail points at the text caret via
AT-SPI accessibility, falling back to the mouse pointer, then a screen
corner. Also gives hands-free wakeword sessions visible feedback, whose
notifications are suppressed by design. X11 only.

- overlay.py: GTK override-redirect HUD (mic + waveform + bubble), drawn
  with Cairo; thread-safe, marshalled onto the GTK loop.
- caret.py: best-effort anchor (AT-SPI caret -> pointer -> window/corner).
- daemon: optional level_cb/text_cb hooks; reuses the VAD level meter for
  non-streaming, a dedicated meter for streaming. Stays UI-agnostic.
- gtkui: instantiate the overlay, drive show/update/hide from status.
- General settings: "Visual overlay" toggle; config overlay_enabled /
  overlay_anchor (default on, "caret").
- Docs: CHANGELOG 1.5.0, version bump, README + MANUAL.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 13:13:41 +02:00
d00064b6cd routing by preset name; right-aligned General switches; About copyright
- Spoken presets: route() now also matches a preset's name as an implicit
  keyword, so presets with no configured keywords (Nicer email, Calm down,
  Add emojis) are triggerable by voice — say the name. Explicit keywords still
  win; names are added to STT hotwords too. Tests added.
- General tab: switches moved to the far right of each row with an inline
  grey description (new _switch_row helper) so each toggle is self-explanatory.
- About tab: add "Copyright: 2026 mARTin Bierschenk - Design".
- MANUAL + CHANGELOG updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:48:22 +02:00
59 changed files with 7467 additions and 628 deletions

8
.gitignore vendored
View File

@ -30,3 +30,11 @@ Secrets.swift
# Tooling
node_modules/
# Agent workspace
jules/
# Raw screenshot source folders (published screenshots stay in Screenshots/)
Screenshots/Settings - new/
Screenshots/Settings-old/
Screenshots/Welcome Setup/

244
MANUAL.md
View File

@ -1,10 +1,11 @@
# Blitztext — User Manual
A reference for every setting in the Blitztext **Settings** window, tab by tab.
A reference for every setting in the Blitztext **Settings** window, page by page.
Open Settings from the system-tray menu (**Settings…**) or the control panel. The
window has seven tabs — **Presets · Engines · Input · General · Benchmark · Log ·
About** — and three buttons along the bottom.
sidebar lists all pages: **Presets · General · Keyboard · Wakeword · STT Engines ·
LLM Engines · Benchmark — STT · Benchmark — Wakeword · Log · Manual · About**.
Three buttons run along the top: **Save**, **Save & Restart**, and **✕ Close**.
> **Where settings are stored:** `~/.config/blitztext/config.toml`
> (or `$XDG_CONFIG_HOME/blitztext/config.toml`). You can edit that file directly;
@ -20,7 +21,11 @@ About** — and three buttons along the bottom.
---
## Presets tab
## Presets
<p align="center">
<a href="Screenshots/settings-presets.png"><img src="Screenshots/settings-presets.png" alt="Presets page" width="100%"></a>
</p>
Presets are your dictation **actions**. Each one either types what you say, or
rewrites it through the language model first (e.g. into a polished email). Trigger
@ -35,7 +40,7 @@ Use the dropdown at the top to pick a preset to edit, **+ Add** to create one, o
| **Name** | `name` | Short name for the action, shown in the main panel. |
| **Icon (emoji)** | `icon` | Emoji shown next to this preset in the "matched preset" notification — give each a distinct one to tell them apart at a glance. Default `⚡`. |
| **Description** | `description` | One line explaining what the preset does (shown in the panel). |
| **Keywords (comma)** | `keywords` | Spoken trigger words, comma-separated. Say one at the **start or end** of your speech to select this preset (fuzzy-matched, e.g. `nicer email, bessere email`). |
| **Keywords (comma)** | `keywords` | Spoken trigger words, comma-separated. Say one at the **start or end** of your speech to select this preset (fuzzy-matched, e.g. `nicer email, bessere email`). The preset's **name is always an implicit trigger**, so it works by voice even with no keywords here; add keywords for alternate/foreign-language phrasings. |
| **Hotkey (optional)** | `hotkey` | A direct keyboard shortcut for this preset. Click **Set** and press the combo, or type it (e.g. `<ctrl>+<alt>+e`). Leave blank for keyword-only. |
| **Mode** | `mode` | `transcribe` types your words as-is · `rewrite` sends them to the language model first · `stream` shows live text from a realtime STT engine. |
| **LLM model (opt.)** | `model` | Override the language model for *this preset only*. Blank = use the active LLM engine's model. |
@ -44,54 +49,35 @@ Use the dropdown at the top to pick a preset to edit, **+ Add** to create one, o
---
## Engines tab
## General
Engines do the work: the **speech-to-text (STT)** engine turns your voice into
text; the **language model (LLM)** rewrites it. Each engine can run locally or on
a server you specify. A **green dot** means it's reachable, **red** means offline.
The currently selected engine in each dropdown is the **active** one.
<p align="center">
<a href="Screenshots/settings-general.png"><img src="Screenshots/settings-general.png" alt="General page" width="100%"></a>
</p>
### Speech-to-text engine
Buttons: **+ Add** (cloud/OpenAI-style), **+ Stream** (realtime Riva/NIM),
**Delete**, **Test** (records 4 s and transcribes), **Refresh** (re-check status).
Each engine maps to a `[[stt_engine]]` entry; the active one is `[stt] active`.
Microphone, text delivery, language, notifications, the on-screen overlay, and
autostart.
| Setting | TOML key | Description |
|---|---|---|
| **Name** | `name` | A label for this engine (e.g. "faster-whisper GPU"). |
| **Type** | `type` | `local` (in-process faster-whisper) · `openai` (any OpenAI-compatible `/v1` STT server) · `riva_realtime` (live streaming engine). |
| **URL** | `url` | Server endpoint. Example: `http://localhost:8010/v1` · realtime: `http://localhost:8006/v1`. Ignored for `local`. |
| **Model** | `model` | Model name. For `local`: `tiny`/`base`/`small`/`medium`/`large-v3` or a path. For remote: blank = server default, or pick from the searchable list fetched from the URL. |
| **API key env** | `api_key_env` | *Name of the environment variable* holding the API key (e.g. `GROQ_API_KEY`). Optional. |
**Local engine (faster-whisper) — device & precision** (global, `[whisper]`):
| Setting | TOML key | Description |
|---|---|---|
| **Device** | `device` | `auto` (try CUDA, fall back to CPU) · `cpu` · `cuda`. |
| **Compute type** | `compute_type` | `auto` · `int8` · `float16` · `int8_float16`. Lower precision is faster and uses less memory. |
### Language model (rewrite)
Buttons: **+ Add**, **Delete**, **Refresh**. Each maps to a `[[llm_engine]]`
entry; the active one is `[llm] active`.
| Setting | TOML key | Description |
|---|---|---|
| **Name** | `name` | A label for this LLM (e.g. "Local Qwen"). |
| **Type** | `type` | `local` (a server on this machine) or `cloud`. |
| **Base URL** | `url` | OpenAI-compatible endpoint, e.g. `http://localhost:28080/v1` or `https://api.openai.com/v1`. |
| **Model** | `model` | The model to use; pick from the list once the URL is set. |
| **API key env** | `api_key_env` | Environment-variable name holding the key (e.g. `OPENAI_API_KEY`). Blank for local servers. |
| **Temperature** | `temperature` | Default creativity for rewrites (e.g. `0.3`). Presets can override this. |
| **Microphone** | `mic` | Which input device Blitztext records from. |
| **Input level** | — | Live level bar (read-only); should move when you speak. |
| **Output** | `output` | `type` types the text key-by-key · `paste` copies it and presses Ctrl+V (faster for long text). |
| **Language hint** | `language` | Spoken-language code (`de`, `en`, …). Blank = auto-detect. |
| **Notifications** | `notify` | Show desktop notifications for recording/transcription status and errors (manual sessions). |
| **Announce matched preset** | `notify_routing` | After a voice command, pop a notification showing which preset (and spoken keyword) matched — shown **even for hands-free** sessions, with the preset's emoji. Only fires on a real match. |
| **Visual overlay** | `overlay_enabled` | Show a translucent bubble at the cursor while you dictate — a pulsing **microphone**, a **live waveform** of your mic level, and the **recognised text** (word-by-word with a streaming engine, or the final result as a brief confirmation). The tail points at where the text lands, and it gives **hands-free** sessions visible feedback. Click-through; never takes focus. *(X11 only.)* |
| **Launch on login** | *(autostart file)* | Start Blitztext automatically when you log in (writes a desktop autostart entry, not `config.toml`). |
---
## Input tab
## Keyboard
Controls **how you start and stop** dictating, the noise filter, hands-free
wakeword, and audio cues.
<p align="center">
<a href="Screenshots/settings-keyboard.png"><img src="Screenshots/settings-keyboard.png" alt="Keyboard page" width="100%"></a>
</p>
Controls **how you start and stop** dictating with the keyboard, the noise filter, and audio cues.
### Input mode & keys
@ -120,7 +106,26 @@ in the `[quality]` section.
| **Reject hallucinations** | `reject_hallucinations` | Drop STT "ghost" outputs like *"Thank you."* / *"Bye."* that Whisper invents from silence. |
| **Strip trailing punctuation** | `strip_trailing_punctuation` | Remove ending periods from delivered text — handy for code insertion. |
### Hands-free (Wakeword)
### Audio cues (manual dictation)
These control the chimes for **manual** (keyboard/hotkey) dictation only. The
hands-free wakeword sounds are **separate and independent** (see Wakeword page).
| Setting | TOML key | Description |
|---|---|---|
| **Play audio cues** | `[sounds] enabled` | On/off for the **manual** start/stop chimes below. Does **not** affect the wakeword sounds. |
| **Play before** | `[sounds] before` | Chime when recording **starts** (manual dictation). Empty = built-in system sound. |
| **Play after** | `[sounds] after` | Chime when recording **stops** (paste, paste+Enter, or auto-stop on silence). Empty = built-in system sound. |
> Each sound row has ▶ (preview) and ⌫ (clear).
---
## Wakeword
<p align="center">
<a href="Screenshots/settings-wakeword.png"><img src="Screenshots/settings-wakeword.png" alt="Wakeword page" width="100%"></a>
</p>
Start dictation with a spoken keyword via an external
[Wyoming](https://github.com/rhasspy/wyoming) openWakeWord server. Maps to the
@ -143,20 +148,7 @@ Start dictation with a spoken keyword via an external
> You can also pause/resume detection from the tray ("Pause wakeword"), which
> toggles the `/tmp/wake_muted` flag.
### Audio cues (manual dictation)
These control the chimes for **manual** (keyboard/hotkey) dictation only. The
hands-free wakeword sounds above are **separate and independent**.
| Setting | TOML key | Description |
|---|---|---|
| **Play audio cues** | `[sounds] enabled` | On/off for the **manual** start/stop chimes below. Does **not** affect the wakeword sounds above. |
| **Play before** | `[sounds] before` | Chime when recording **starts** (manual dictation). Empty = built-in system sound. |
| **Play after** | `[sounds] after` | Chime when recording **stops** (paste, paste+Enter, or auto-stop on silence). Empty = built-in system sound. |
> Each sound row has ▶ (preview) and ⌫ (clear).
>
> **The two pairs differ by trigger *and* by empty-behaviour:**
> **The two sound pairs differ by trigger *and* by empty-behaviour:**
>
> | | Plays on | Used for | When empty |
> |---|---|---|---|
@ -165,26 +157,69 @@ hands-free wakeword sounds above are **separate and independent**.
---
## General tab
## STT Engines
Microphone, text delivery, language, notifications, and autostart.
<p align="center">
<a href="Screenshots/settings-stt-engines.png"><img src="Screenshots/settings-stt-engines.png" alt="STT Engines page" width="100%"></a>
</p>
The **speech-to-text** engine turns your voice into text. Each engine can run
locally or on a server. A **green dot** means it's reachable, **red** means
offline. The active engine is the one selected in the top dropdown.
Buttons: **+ Add** (batch/cloud/OpenAI-style), **+ Stream** (realtime Riva/NIM),
**Delete**, **Test** (records 4 s and transcribes), **Refresh** (re-check status).
Each engine maps to a `[[stt_engine]]` entry; the active one is `[stt] active`.
| Setting | TOML key | Description |
|---|---|---|
| **Microphone** | `mic` | Which input device Blitztext records from. |
| **Input level** | — | Live level bar (read-only); should move when you speak. |
| **Output** | `output` | `type` types the text key-by-key · `paste` copies it and presses Ctrl+V (faster for long text). |
| **Language hint** | `language` | Spoken-language code (`de`, `en`, …). Blank = auto-detect. |
| **Notifications** | `notify` | Show desktop notifications for recording/transcription status and errors (manual sessions). |
| **Announce matched preset** | `notify_routing` | After a voice command, pop a notification showing which preset (and spoken keyword) matched — shown **even for hands-free** sessions, with the preset's emoji. Only fires on a real match. |
| **Launch on login** | *(autostart file)* | Start Blitztext automatically when you log in (writes a desktop autostart entry, not `config.toml`). |
| **Name** | `name` | A label for this engine (e.g. "faster-whisper GPU"). |
| **Type** | `type` | `local` (in-process faster-whisper) · `openai` (any OpenAI-compatible `/v1` STT server) · `riva_realtime` (live streaming engine). |
| **URL** | `url` | Server endpoint. Example: `http://localhost:8010/v1` · realtime: `http://localhost:8006/v1`. Ignored for `local`. |
| **Model** | `model` | Model name. For `local`: `tiny`/`base`/`small`/`medium`/`large-v3` or a path. For remote: blank = server default, or pick from the searchable list fetched from the URL. |
| **API key env** | `api_key_env` | *Name of the environment variable* holding the API key (e.g. `GROQ_API_KEY`). Optional. |
**Local engine (faster-whisper) — device & precision** (global, `[whisper]`):
| Setting | TOML key | Description |
|---|---|---|
| **Device** | `device` | `auto` (try CUDA, fall back to CPU) · `cpu` · `cuda`. |
| **Compute type** | `compute_type` | `auto` · `int8` · `float16` · `int8_float16`. Lower precision is faster and uses less memory. |
---
## Benchmark tab
## LLM Engines
<p align="center">
<a href="Screenshots/settings-llm-engines.png"><img src="Screenshots/settings-llm-engines.png" alt="LLM Engines page" width="100%"></a>
</p>
The **language model** rewrites your dictated text (e.g. into a polished email).
Each engine can be a local LAN server or a cloud service. The active engine is
the one selected in the top dropdown.
Buttons: **+ Add**, **Delete**, **Refresh**. Each maps to a `[[llm_engine]]`
entry; the active one is `[llm] active`.
| Setting | TOML key | Description |
|---|---|---|
| **Name** | `name` | A label for this LLM (e.g. "Local Qwen"). |
| **Type** | `type` | `local` (a server on this machine) or `cloud`. |
| **Base URL** | `url` | OpenAI-compatible endpoint, e.g. `http://localhost:28080/v1` or `https://api.openai.com/v1`. |
| **Model** | `model` | The model to use; pick from the list once the URL is set. |
| **API key env** | `api_key_env` | Environment-variable name holding the key (e.g. `OPENAI_API_KEY`). Blank for local servers. |
| **Temperature** | `temperature` | Default creativity for rewrites (e.g. `0.3`). Presets can override this. |
---
## Benchmark — STT
<p align="center">
<a href="Screenshots/settings-benchmark-stt.png"><img src="Screenshots/settings-benchmark-stt.png" alt="Benchmark — STT page" width="100%"></a>
</p>
Compare your STT engines for **speed and accuracy** on the same clip. Add an
engine preset (Engines tab) for each model you want to compare. No persistent
engine in the STT Engines page for each model you want to compare. No persistent
settings — it's a one-off tool.
1. **Audio (.wav)** — a recording to transcribe.
@ -192,27 +227,77 @@ settings — it's a one-off tool.
a matching `*.txt` / `*.reference.txt` sits next to the WAV.)
3. **Run benchmark** — fills the table with one row per engine.
Result columns: **Engine · Model · Device · Time (s) · Accuracy · Output**. A
summary line names the **fastest** and **most accurate** engine.
Result columns:
| Column | Description |
|---|---|
| **Engine** | Engine preset name |
| **URL** | Server address (blank for local) |
| **Model** | Model name used |
| **Device** | `CPU`, `CUDA`, or `remote` |
| **Best for** | `Short clips` · `Short / medium` · `Long / batch` · `Streaming` |
| **Lang** | Supported languages from the server's `/v1/models` (`—` if unknown) |
| **Time (s)** | Wall-clock seconds for this transcription |
| **Accuracy** | `1 WER` × 100 %. 100 % = word-perfect, case-sensitive |
| **RAM (MB)** | RSS increase while the engine ran. Captures model load cost on first run. Remote engines show `—`. |
| **Output** | Transcribed text (hover for full error on failure) |
A summary line names the **fastest** and **most accurate** engine. Click any column header to sort.
---
## Log tab
## Benchmark — Wakeword
<p align="center">
<a href="Screenshots/settings-benchmark-wakeword.png"><img src="Screenshots/settings-benchmark-wakeword.png" alt="Benchmark — Wakeword page" width="100%"></a>
</p>
Stress-test your wakeword detection by generating speech with a TTS server and
checking whether the wake word fires correctly. Reports **recall** (how often it
fires when it should) and **false-fire rate** (how often it fires on non-wake
speech) across multiple synthetic voices.
---
## Log
<p align="center">
<a href="Screenshots/settings-log.png"><img src="Screenshots/settings-log.png" alt="Log page" width="100%"></a>
</p>
A live activity log — useful to watch a model load/download or to diagnose a
problem (recording, transcription, routing, and wakeword events all appear here).
Press **Copy** to put the log on the clipboard when reporting an issue. No
settings.
| Control | Description |
|---|---|
| **Level** dropdown | Filter by severity: **Verbose** (all), **Info** (default), **Warning**, **Error**. Switch to Warning or Error to cut noise when troubleshooting. |
| **Copy** | Put the log on the clipboard when reporting an issue. |
| **Clear** | Discard all current log entries. |
| **Auto-scroll** | Keep the view scrolled to the latest entry. |
---
## About tab
## Manual
<p align="center">
<a href="Screenshots/settings-manual.png"><img src="Screenshots/settings-manual.png" alt="Manual page" width="100%"></a>
</p>
Inline copy of this manual, readable without leaving the app.
---
## About
<p align="center">
<a href="Screenshots/settings-about.png"><img src="Screenshots/settings-about.png" alt="About page" width="100%"></a>
</p>
Read-only information:
- **Version** and a link to the source repository
(`github.com/mARTin-B78/blitztext-app-linux`).
- **License: MIT**.
- **License: MIT** · **Copyright: 2026 mARTin Bierschenk - Design**.
- Sub-tabs with the full **Changelog** and **License** text.
---
@ -239,3 +324,8 @@ A few behaviours live in `config.toml` without a dedicated tab control:
no keyword matches), and `threshold` (`0``1` fuzzy-match strictness).
- **`timeout`** — network timeout (seconds) for remote STT/LLM requests.
- **`type_delay_ms`** — delay between simulated keystrokes in `type` output mode.
- **`overlay_anchor`** — where the overlay's tail points: `caret` (best-effort —
follows the focused app's text caret via AT-SPI accessibility, falling back to
the pointer), `pointer` (always the mouse pointer), or `corner` (a fixed screen
corner; also the automatic fallback on Wayland or when the cursor can't be
located). Paired with the **Visual overlay** toggle above.

145
README.md
View File

@ -4,13 +4,17 @@
Blitztext is a native Linux dictation tool that captures your voice, transcribes it locally with [faster-whisper](https://github.com/SYSTRAN/faster-whisper), optionally rewrites the text through an LLM, and types the result directly into whatever application has focus. Think macOS Dictation, but open-source, extensible, and designed for power users who want full control over their speech-to-text pipeline.
> **Status:** Experimental open-source Linux/X11 desktop app (v1.4.0).
> **Status:** Experimental open-source Linux/X11 desktop app (v1.7.0).
> No hosted backend — bring your own models and endpoints.
<p align="center">
<img src="Screenshots/panel.png" alt="Blitztext control panel" width="380">
<img src="Screenshots/main-panel.png" alt="Blitztext control panel" width="360">
</p>
<p align="center">
<img src="Screenshots/overlay-listening.png" alt="On-screen overlay while listening" width="360">
&nbsp;&nbsp;
<img src="Screenshots/tray-menu.png" alt="Blitztext system-tray menu" width="300">
<img src="Screenshots/overlay-result.png" alt="On-screen overlay showing transcription result" width="360">
</p>
📖 **[User manual](MANUAL.md)** — every setting in every tab, explained.
@ -55,55 +59,134 @@ Stream: hotkey → mic PCM chunks → Riva/NIM WebSocket → live words typed
- **Fully local STT.** Batch transcription via `faster-whisper` never leaves your machine. No cloud account needed for basic dictation.
- **Pluggable engines.** Configure multiple STT and LLM backends as named presets — local `faster-whisper`, remote OpenAI-compatible batch endpoints, Riva/NIM realtime WebSocket servers, and any OpenAI-compatible chat API (OpenAI, vLLM, llama-swap, Ollama, LM Studio, Groq, OpenRouter).
- **Voice-keyword routing.** One hotkey, multiple workflows. Say "nicer email" at the start or end of your speech and the email-rewrite preset activates automatically (fuzzy-matched, ASR-tolerant).
- **Spoken cancel.** Say "abbrechen" (or "cancel") at the start or end of a clip and the whole dictation is discarded — never routed, rewritten, or typed. The rescue for an accidentally triggered (e.g. wakeword) recording. Configurable in Settings; empty list disables it.
- **Quality gate.** Silent clips, too-short recordings, and Whisper hallucinations ("Thank you.", "Untertitel…") are caught and rejected before they reach your text field.
- **Realtime streaming.** Connect a Riva/NIM realtime STT server and see stable words typed live as you speak.
- **On-screen overlay at the cursor.** The moment you start dictating — by hotkey *or* wakeword — a translucent bubble pops up at the cursor with a pulsing microphone, a live waveform of your mic level, and the recognised text. When a voice keyword routes to a preset it shows that preset's icon, name, and the matched keyword on a banner (instead of a desktop notification), and streams the LLM rewrite into the bubble token-by-token so you watch it write. Its tail points at the text caret (via accessibility) and finally gives hands-free wakeword sessions visible feedback. Click-through, never steals focus; toggle in Settings → General.
- **Built-in benchmarking.** Compare all your configured STT engines against a reference WAV + transcript to find the fastest and most accurate.
---
## Screenshots
Everything is configured in the GTK **Settings** window — every tab has tooltips
and screen-reader (ATK) support. Click any image to open it full size.
Everything is configured in the GTK **Settings** window — the sidebar gives quick
access to every page. All controls have tooltips and screen-reader (ATK) support.
Click any image to open it full size.
### Main panel & overlay
<p align="center">
<a href="Screenshots/settings-presets.png"><img src="Screenshots/settings-presets.png" alt="Presets settings tab" width="100%"></a><br>
<em><b>Presets</b> — your dictation actions. Each preset is either a plain transcription or an LLM rewrite, and carries its own spoken keyword(s) for voice routing, an optional global hotkey, and a custom rewrite prompt.</em>
<a href="Screenshots/main-panel.png"><img src="Screenshots/main-panel.png" alt="Blitztext main panel" width="46%"></a>
&nbsp;
<a href="Screenshots/overlay-listening.png"><img src="Screenshots/overlay-listening.png" alt="Overlay — listening" width="46%"></a>
</p>
<p align="center">
<em><b>Left:</b> Control panel listing all presets with icons, descriptions, and hotkeys.</em>
&nbsp;&nbsp;
<em><b>Right:</b> On-screen overlay showing the live waveform while listening.</em>
</p>
<p align="center">
<a href="Screenshots/settings-engines.png"><img src="Screenshots/settings-engines.png" alt="Engines settings tab" width="100%"></a><br>
<em><b>Engines</b> — your speech-to-text and language-model back-ends, local or remote. Add and rename engines, watch live online/offline status, and pick models from a searchable list fetched straight from the endpoint.</em>
<a href="Screenshots/overlay-result.png"><img src="Screenshots/overlay-result.png" alt="Overlay — transcription result" width="46%"></a>
</p>
<p align="center">
<em>Overlay after transcription — preset name and recognised text appear at the cursor.</em>
</p>
### Settings — General & Input
<p align="center">
<a href="Screenshots/settings-presets.png"><img src="Screenshots/settings-presets.png" alt="Presets page" width="48%"></a>
&nbsp;
<a href="Screenshots/settings-general.png"><img src="Screenshots/settings-general.png" alt="General page" width="48%"></a>
</p>
<p align="center">
<em><b>Presets</b> — dictation actions with keywords, hotkeys, LLM mode, and custom prompts.</em>
&nbsp;&nbsp;
<em><b>General</b> — microphone, output mode, language hint, notifications, overlay, autostart.</em>
</p>
<p align="center">
<a href="Screenshots/settings-input.png"><img src="Screenshots/settings-input.png" alt="Input settings tab" width="100%"></a><br>
<em><b>Input</b> — how you start and stop dictation: the modifier-key scheme (Ctrl+Win / Ctrl / Alt / Esc) or custom hotkeys, plus the silence-based auto-stop (VAD), the quality gate, and audio cues.</em>
<a href="Screenshots/settings-keyboard.png"><img src="Screenshots/settings-keyboard.png" alt="Keyboard page" width="48%"></a>
&nbsp;
<a href="Screenshots/settings-wakeword.png"><img src="Screenshots/settings-wakeword.png" alt="Wakeword page" width="48%"></a>
</p>
<p align="center">
<em><b>Keyboard</b> — modifier-key scheme or direct hotkeys, quality gate, audio cues.</em>
&nbsp;&nbsp;
<em><b>Wakeword</b> — hands-free dictation via a Wyoming/openWakeWord server, with live level meter and model picker.</em>
</p>
### Settings — Engines
<p align="center">
<a href="Screenshots/settings-stt-engines.png"><img src="Screenshots/settings-stt-engines.png" alt="STT Engines page" width="48%"></a>
&nbsp;
<a href="Screenshots/settings-llm-engines.png"><img src="Screenshots/settings-llm-engines.png" alt="LLM Engines page" width="48%"></a>
</p>
<p align="center">
<em><b>STT Engines</b> — speech-to-text back-ends (local faster-whisper, OpenAI-compatible server, or Riva realtime), with green/red status dot and Test button.</em>
&nbsp;&nbsp;
<em><b>LLM Engines</b> — language-model back-ends for text rewriting (LAN server or cloud service).</em>
</p>
### Settings — Benchmark
<p align="center">
<a href="Screenshots/settings-benchmark-stt.png"><img src="Screenshots/settings-benchmark-stt.png" alt="Benchmark — STT page" width="48%"></a>
&nbsp;
<a href="Screenshots/settings-benchmark-wakeword.png"><img src="Screenshots/settings-benchmark-wakeword.png" alt="Benchmark — Wakeword page" width="48%"></a>
</p>
<p align="center">
<em><b>Benchmark — STT</b> — compare every configured engine against a reference WAV + transcript; results table shows speed, accuracy, device, and language support.</em>
&nbsp;&nbsp;
<em><b>Benchmark — Wakeword</b> — stress-test wakeword detection using a TTS server to synthesise wake phrases in random voices, reporting recall and false-fire rates.</em>
</p>
### Settings — Log & About
<p align="center">
<a href="Screenshots/settings-log.png"><img src="Screenshots/settings-log.png" alt="Log page" width="48%"></a>
&nbsp;
<a href="Screenshots/settings-about.png"><img src="Screenshots/settings-about.png" alt="About page" width="48%"></a>
</p>
<p align="center">
<em><b>Log</b> — live activity log for recording, transcription, routing, and wakeword events.</em>
&nbsp;&nbsp;
<em><b>About</b> — version, source link, inline changelog, and licence.</em>
</p>
### Setup Wizard
The first-run wizard guides you through the essentials in a few steps.
<p align="center">
<a href="Screenshots/setup-welcome.png"><img src="Screenshots/setup-welcome.png" alt="Setup — Welcome" width="32%"></a>
&nbsp;
<a href="Screenshots/setup-trigger.png"><img src="Screenshots/setup-trigger.png" alt="Setup — Trigger mode" width="32%"></a>
&nbsp;
<a href="Screenshots/setup-shortcuts.png"><img src="Screenshots/setup-shortcuts.png" alt="Setup — Keyboard shortcuts" width="32%"></a>
</p>
<p align="center">
<em>Welcome &nbsp;·&nbsp; Choose trigger mode (keyboard / wakeword / both) &nbsp;·&nbsp; Set keyboard shortcuts</em>
</p>
<p align="center">
<a href="Screenshots/wakeword.png"><img src="Screenshots/wakeword.png" alt="Wakeword (hands-free) settings" width="100%"></a><br>
<em><b>Wakeword (hands-free)</b> — point Blitztext at a Wyoming/openWakeWord server, choose a wake model, and test the connection live so a spoken keyword starts dictation with no keys at all.</em>
<a href="Screenshots/setup-voice.png"><img src="Screenshots/setup-voice.png" alt="Setup — Voice activation" width="32%"></a>
&nbsp;
<a href="Screenshots/setup-stt.png"><img src="Screenshots/setup-stt.png" alt="Setup — STT engine" width="32%"></a>
&nbsp;
<a href="Screenshots/setup-ai.png"><img src="Screenshots/setup-ai.png" alt="Setup — AI rewriting" width="32%"></a>
</p>
<p align="center">
<em>Voice activation (wakeword server) &nbsp;·&nbsp; Choose STT engine &nbsp;·&nbsp; Optional AI text rewriting</em>
</p>
<p align="center">
<a href="Screenshots/settings-general.png"><img src="Screenshots/settings-general.png" alt="General settings tab" width="100%"></a><br>
<em><b>General</b> — core preferences: microphone with a live level meter, output mode (type vs. paste), language hint, type delay, and autostart on login.</em>
<a href="Screenshots/setup-done.png"><img src="Screenshots/setup-done.png" alt="Setup — All done" width="32%"></a>
</p>
<p align="center">
<a href="Screenshots/settings-benchmark.png"><img src="Screenshots/settings-benchmark.png" alt="Benchmark settings tab" width="100%"></a><br>
<em><b>Benchmark</b> — compare every configured STT engine against a reference WAV + transcript to find the fastest and most accurate, with a Device column (CPU / GPU / remote).</em>
</p>
<p align="center">
<a href="Screenshots/settings-log.png"><img src="Screenshots/settings-log.png" alt="Log settings tab" width="100%"></a><br>
<em><b>Log</b> — the in-app log buffer: a live view of recording, transcription, routing, and wakeword events for quick troubleshooting.</em>
</p>
<p align="center">
<a href="Screenshots/settings-about.png"><img src="Screenshots/settings-about.png" alt="About settings tab" width="100%"></a><br>
<em><b>About</b> — version, source link, changelog, and licence.</em>
<em>Summary screen — ready to dictate.</em>
</p>
---
@ -240,6 +323,10 @@ Now try with voice routing:
1. Press `Ctrl+Alt+Space` → say **"nicer email** hey john can you send me the report"
2. Press `Ctrl` — Blitztext detects the keyword, runs the "Nicer email" rewrite, and types a polished email.
### 4b. Cancel by voice
Started a recording by accident (or changed your mind)? Just say **"abbrechen"** (or **"cancel"**) at the start or end of what you say. The whole clip is discarded — nothing is transcribed onward, routed, rewritten, or typed, and the overlay briefly shows *✗ Abgebrochen*. This is especially handy with the hands-free wakeword, where a stray trigger could otherwise type ambient speech. Tune the words under **Settings → Mic/Cues → "Cancel words"** (or `[routing] cancel_keywords`); clear the list to switch it off.
### 5. Explore Settings
Click the ⚙️ gear icon in the panel header, or right-click the tray → **Settings…**
@ -291,6 +378,7 @@ With the default `modifiers` input mode:
| `Ctrl` | Stop → transcribe → type |
| `Alt` | Stop → transcribe → type → press Enter |
| `Esc` | Cancel (discard recording) |
| say *"abbrechen"* / *"cancel"* | Cancel by voice — discard the clip (works hands-free too) |
---
@ -413,6 +501,7 @@ systemctl --user enable --now blitztext
- **Wayland support** requires `wtype` or `ydotool`. Wayland security prevents global window focus manipulation, so text is delivered to whatever window is active when delivery occurs.
- **No automated tests yet.** Contributions welcome (routing, quality gate, config parsing are all highly testable).
- **Realtime streaming** requires a compatible Riva/NIM server.
- **The on-screen overlay is X11-only** (it positions a window at the cursor and reads the pointer/caret); on Wayland it falls back to a fixed screen corner. Caret-accurate anchoring further needs the focused app to expose its text caret over AT-SPI accessibility — otherwise it follows the mouse pointer.
- **Local STT speed** depends on your hardware, Whisper model size, and CTranslate2 build (CPU `int8` by default).
- This is experimental software provided as-is.

BIN
Screenshots/main-panel.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 54 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 158 KiB

After

Width:  |  Height:  |  Size: 202 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 127 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 134 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 56 KiB

After

Width:  |  Height:  |  Size: 118 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 145 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 102 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 60 KiB

After

Width:  |  Height:  |  Size: 106 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 219 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 78 KiB

After

Width:  |  Height:  |  Size: 118 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 111 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 164 KiB

BIN
Screenshots/setup-ai.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 54 KiB

BIN
Screenshots/setup-done.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 49 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 52 KiB

BIN
Screenshots/setup-stt.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 54 KiB

BIN
Screenshots/setup-voice.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 52 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

View File

@ -9,6 +9,685 @@ The version is defined in [`blitztext/__init__.py`](blitztext/__init__.py).
## [Unreleased]
## [2.03.41] - 2026-06-10
### Added
- **Cancel button shown during transcription and rewriting.** The `×` button in
the top-right corner of the overlay is now visible and clickable in the
`busy` state (Transcribing… / Rewriting…), not only while recording.
Clicking it during transcription discards the result once the STT call
returns. Clicking it during a rewrite breaks out of the LLM stream
immediately — the partial text is discarded and nothing is typed.
## [2.03.40] - 2026-06-10
### Changed
- **Documentation overhaul with screenshots.** Renamed 21 raw screenshots to
meaningful filenames (`main-panel.png`, `overlay-listening.png`,
`settings-presets.png`, `settings-general.png`, `settings-keyboard.png`,
`settings-wakeword.png`, `settings-stt-engines.png`,
`settings-llm-engines.png`, `settings-benchmark-stt.png`,
`settings-benchmark-wakeword.png`, `settings-log.png`,
`settings-manual.png`, `settings-about.png`, and 7 setup-wizard screens).
- **README.md / linux/README.md** updated with organized screenshot sections
(Main panel & overlay, Settings — General & Input, Settings — Engines,
Settings — Benchmark, Settings — Log & About, Setup wizard) using
click-to-enlarge image links.
- **MANUAL.md** rewritten to match the new sidebar navigation: intro updated;
section headers renamed (Engines tab → STT Engines / LLM Engines; Input tab
→ Keyboard + Wakeword; Benchmark tab → Benchmark — STT / Benchmark —
Wakeword); screenshot added at the top of every section including Manual and
About pages.
## [2.03.39] - 2026-06-10
### Fixed
- **Remaining horizontal scrollbars eliminated.** Root causes: (1) infobox
`max_width_chars` was 72 — at typical system font sizes (9 px/char) this
produced a natural width wider than the ~677 px content area; reduced to 58.
(2) Engine-selector combos (`stt_combo`, `llm_combo`, `ww_combo`) and the
`_combo()` helper had no constraint on CellRendererText width — long engine
names or ALSA device names drove the combo's natural width to 300500 px.
Fixed by adding `_ellipsize_combo()` (sets `ellipsize=END` and
`max-width-chars=28` on the internal CellRendererText) to all combos.
(3) `_STT_TYPES`, `_LLM_TYPES`, `_DEVICE_OPTIONS`, `_COMPUTE_OPTIONS`
labels were 4052 characters; shortened to ≤27 chars.
### Changed
- **Wakeword Cancel/Send word rows split into two rows.** Keywords and
keyboard shortcut are now on separate lines inside the card, avoiding the
cramped single-row layout.
- **Benchmark — Wakeword: "Run wakeword benchmark" button moved** from the
bottom of the settings pane to the top of the results pane; pane divider
adjusted from 390 → 340 px. The button is now always visible without
scrolling and sits logically above the results it produces.
## [2.03.38] - 2026-06-10
### Fixed
- **STT Engines and Wakeword no longer have horizontal scrollbars.** Root
cause: `Gtk.Entry` widgets compute natural width from placeholder text
(e.g. `"http://localhost:8010/v1 · realtime: http://localhost:8006/v1"`
≈ 500 px). Without `set_width_chars(1)` the entry cannot shrink below its
natural width even when placed in an expanding container. Added
`set_width_chars(1)` to all entry-creating helpers: `_entry()`, `_url_field()`,
`ModelPicker`, `_kw_shortcut_row`, and `_sound_field`. Also added
`set_max_width_chars(50)` to the `stt_result` wrapping label.
## [2.03.37] - 2026-06-10
### Fixed
- **STT Engines page no longer appears empty.** `_refresh_status()` called
`_stt_commit()` / `_llm_commit()` and accessed `stt_dot` / `llm_dot`
unconditionally; if the STT page was opened before the LLM page was built
(lazy), the builder crashed silently with `AttributeError`. Added `hasattr`
guards so each section is only committed / updated when its widgets exist.
- **Wakeword page no longer causes horizontal scrollbar.** The `ww_status`
label (showing model list like "7 models loaded: okay_nabu, hey_jarvis…")
had no width limit and expanded the page. Added `set_max_width_chars(30)`
and `set_ellipsize(END)`.
- **Benchmark STT engine list no longer causes horizontal scrollbar.**
`sel_sw` used `NEVER` horizontal policy, propagating long engine-name labels
(~800 px) up through the paned. Changed to `AUTOMATIC` so content scrolls
internally.
- **General page and LLM Engines no longer cause horizontal scrollbar.**
`_combo()` and `_type_combo()` lacked `set_size_request(10, -1)`, so
ComboBoxText widgets (e.g. long microphone device names) could not shrink
below their natural width. Added the size request to both helpers.
## [2.03.36] - 2026-06-10
### Changed
- **STT Engines split from LLM Engines.** The single "STT Engines" sidebar
page is now two pages — "STT Engines" (speech-to-text configuration) and
"LLM Engines" (language model / rewrite configuration) — each with its
own infobox. The underlying `_stt_section()` and `_llm_section()` methods
are unchanged.
## [2.03.35] - 2026-06-10
### Fixed
- **Settings pages no longer widen the dialog.** `_switch_row` description
labels had `set_line_wrap(True)` but no `set_max_width_chars`, so GTK
computed their natural width as the full un-wrapped text (87 chars × ~8 px
= ~700 px). With `NEVER` horizontal policy on the page `ScrolledWindow`,
that propagated directly to the dialog width, making Keyboard, Wakeword,
STT Engines, and Benchmark pages ~10001360 px wide. Fixed by adding
`set_max_width_chars(50)` (≈ 375 px) to description labels, and changed the
page `ScrolledWindow` horizontal policy from `NEVER` to `AUTOMATIC` as a
safety net for any other wide widget.
## [2.03.34] - 2026-06-10
### Fixed
- **Settings window no longer grows wide or tall when switching pages.**
`Stack.set_homogeneous(True)` was causing the Stack to request the maximum
natural size of all its children (both width and height), so pages like
Benchmark — STT (wide TreeView) inflated the dialog to 1000+ px wide for
every other page. Reverted to `False`; the dialog now stays at its default
860 × 700 and each page scrolls if its content is taller than the window.
Also removed the erroneous `NEVER/NEVER` ScrolledWindow policy on the two
benchmark pages that was propagating natural TreeView width to the Stack.
## [2.03.33] - 2026-06-10
### Fixed
- **Settings window no longer resizes when switching pages.** Added
`Gtk.Stack.set_homogeneous(True)` so the dialog always allocates the
maximum page height, preventing the window from growing or shrinking
as pages are visited.
- **Benchmark — Wakeword: "Engines to test" section no longer hidden.**
The paned divider position was raised from 260 to 390 px so all TTS
config fields, the engine checkboxes, wakeword model selector, sample
count, and run button are fully visible without scrolling. Both
benchmark pages also disable the page-level `ScrolledWindow` so the
paned correctly fills the viewport height rather than expanding
past it.
## [2.03.32] - 2026-06-10
### Changed
- **Settings dialog navigation redesigned.** Replaced the `Gtk.Notebook` tab bar
with a 170 px left sidebar (`Gtk.ListBox` of flat buttons with section headers)
and a `Gtk.Stack` for the content area. The dialog is now 860 × 700 px by default.
Lazy-loading is preserved: each page is built only on first visit.
- **Input page split into Keyboard and Wakeword.** The former "Input" tab is now
two separate pages — "Keyboard" (input mode, hotkeys, quality gate, audio cues)
and "Wakeword" (enable switch, mic level, test button, silence timeout, cancel/send
words, engine preset selector, engine config card, wakeword sound cues). Either
page may be visited first; the mic level-meter starts on whichever is opened.
- **Benchmark page split into Benchmark — STT and Benchmark — Wakeword.** The
single "Benchmark" tab is now two dedicated pages. All field names and collect
logic are unchanged.
## [2.03.31] - 2026-06-10
### Added
- **First-run setup wizard.** A paged GTK dialog (`setup_wizard.py`) guides
new users through five steps: trigger method (keyboard / wakeword / both),
keyboard shortcut assignment (with live key capture), wakeword server
configuration (with connection test), speech-to-text engine selection
(local Whisper model size or remote API), and optional AI rewriting (LLM
endpoint + model). Navigation has Back, Next, and Skip buttons. The wizard
shows automatically on first launch (before the daemon starts) and can be
reopened via the "Setup Wizard…" button in the Settings header bar.
Completing the wizard sets `setup_complete = true` in the config so it
does not reappear.
## [2.03.30] - 2026-06-10
### Added
- **Dedicated wakeword models for cancel and send.** Two new optional fields in
the wakeword settings card — "Cancel model" and "Send model" — let you assign
a specific wakeword model (e.g. a custom "stop" or "send it" ONNX model) to
each action. When configured, a `WakewordActionListener` opens a second
Wyoming connection during recording and fires the action the instant the model
triggers — no Whisper pass, no silence timer. The Whisper-based cancel watcher
from v2.03.29 remains as a fallback when no cancel wakeword model is set.
Model dropdowns are populated from the same server fetch as the trigger model.
## [2.03.29] - 2026-06-10
### Changed
- **Cancel keywords now fire immediately during wakeword recording.** A
real-time `_CancelWatcher` accumulates raw PCM from the VAD level-meter,
then every ~0.6 s of new audio runs a fast `beam_size=1` local transcription
pass to check for cancel keywords. When one is found it calls
`cancel_dictation()` instantly — no waiting for the silence timer or a full
transcription of the whole clip. Falls back to the existing post-transcription
check if no local transcriber is loaded or no cancel keywords are configured.
## [2.03.28] - 2026-06-10
### Fixed
- **Dropdowns no longer change on accidental scroll.** All `ComboBoxText`
widgets in the settings dialog now swallow scroll events so hovering over
a combo and scrolling doesn't silently change the selected value.
## [2.03.27] - 2026-06-10
### Added
- **Overlay × cancel button.** A small × button appears in the top-right corner
of the on-screen waveform HUD while recording or streaming. Clicking it
cancels the current dictation. The rest of the overlay remains fully
click-through; only the button area receives pointer events.
- **Cancel/Send keyboard shortcuts in Wakeword tab.** The "Cancel words" and
"Send words" rows in the Input → Wakeword section now include an inline
shortcut entry + "Set" button, so you can configure `key_cancel` /
`key_send` right next to the spoken-word equivalents without visiting the
keyboard-mode card.
## [2.03.26] - 2026-06-10
### Fixed
- **Cancel key now works during wakeword-triggered recording.** Previously the
`ModifierScheme` state machine stayed in "idle" when the wakeword fired
(it bypasses the key-press path), so the cancel hotkey was silently ignored.
It now checks `daemon.is_recording` as a fallback so it fires regardless of
how recording started.
### Added
- **"✕ Cancel recording" in the tray menu.** Always visible; grayed out when
idle, enabled as soon as recording starts (wakeword or manual). The primary
escape hatch when the wakeword fires on audiobook / TV audio and spoken
cancel words can't be heard over the background audio.
## [2.03.25] - 2026-06-10
### Changed
- **Smart Save — no restart popup anymore.** "Save" now diffs the changed
settings against what requires a daemon restart. If only safe settings
changed (language, sounds, LLM prompt, keywords, overlay, …) it shows
"✓ Settings applied" inline in the header bar for 4 s and closes the
dialog. If restart-required fields changed (STT engine, hotkeys,
microphone, wakeword server) it shows "⚠ Saved — restart needed for: …"
and highlights "Save & Restart" so you can act on it. No modal popups.
## [2.03.24] - 2026-06-10
### Fixed
- **Input level meter works without visiting General first.** The level meter
was only started inside `_build_general()` and referenced `mic_level`
unconditionally. If Input was opened first the meter never started. Now
`_build_input()` also starts the meter when it isn't running yet, and both
level bars (`mic_level` in General and `ww_mic_level` in Input) are updated
defensively via `hasattr` so either tab can be visited in any order.
## [2.03.23] - 2026-06-10
### Fixed
- **Wakeword results table actually resizable.** The controls pane is now
wrapped in a ScrolledWindow with `shrink=True`, so dragging the divider
upward collapses the controls and expands the table freely.
## [2.03.22] - 2026-06-10
### Changed
- **Wakeword benchmark uses a split pane.** The TTS config / engine selector
controls sit in the top pane; the results table sits in the bottom pane.
Drag the divider to give the table as much vertical space as needed.
## [2.03.21] - 2026-06-10
### Added
- **Wakeword results table: sortable columns.** Click any column header to sort
ascending/descending. Numeric columns (Detected, Total, Recall %, False fires,
Time) sort numerically.
- **Wakeword results table: CSV export.** "Copy as CSV" copies the table to the
clipboard; "Save CSV…" opens a file chooser to write a `.csv` file.
## [2.03.20] - 2026-06-10
### Added
- **Wakeword benchmark results table.** Results are now shown in a TreeView
with one row per engine per voice: Engine | Wakeword | Voice | Detected |
Total | Recall % | False fires | Time. Rows are colour-coded green/orange/red
by recall. An aggregate "ALL (N voices)" row is appended per engine.
### Fixed
- **Section header icons now vertically centred with the headline text.**
The `.bt-section` CSS class was inadvertently applied to the icon widget,
giving it a 14 px top margin and pushing it down. The image no longer
receives that class; a `set_pixel_size(14)` pin ensures consistent sizing.
## [2.03.19] - 2026-06-10
### Added
- **Wakeword engine checkboxes in benchmark.** A row of checkboxes above the
"Run wakeword benchmark" button lets you pick which engines to include.
All are checked by default.
- **Wakeword model selector in benchmark.** A "Wakeword" combo lets you
override which wakeword phrase (model) to test. Leave empty for the default
(each engine uses its own configured model). Pick a specific model (e.g.
`okay_computer`) to test that phrase on every selected engine.
## [2.03.18] - 2026-06-10
### Fixed
- **TTS model dropdown no longer floods with voice names.** Servers like Kokoro
expose each voice as a `/models` entry. The ⟳ button now detects this case
and skips filling the model combo, prompting the user to type the model id
manually (e.g. `kokoro`). The status line shows "type model id manually" as
a hint.
### Changed
- **Wakeword benchmark runs across all engines and shows per-engine results.**
Previously a callback signature mismatch caused the benchmark to crash when
more than one engine was configured. Now progress shows `[1/3] engine name`,
and the results panel lists Recall / False fires / time per engine.
## [2.03.17] - 2026-06-10
### Added
- **Wakeword model fetch feedback.** The ⟳ button now shows a status line while
connecting; after loading it reports how many models were found (with their
names) or "Unreachable" if the server is down.
- **Wakeword Quickstart covers all four ports.** The Quickstart menu now lists
presets for ports 1040010403, plus `hey_jarvis` and `alexa` variants.
- **Wakeword info box.** An info banner explains how wyoming-openwakeword works,
where to put model files, and lists the common built-in models.
## [2.03.16] - 2026-06-10
### Added
- **MP3/OGG/FLAC support for sound cues.** The sound file picker now accepts
WAV, MP3, OGG, FLAC, M4A, AAC, AIFF, and Opus. Playback uses `ffplay` or
`gst-play-1.0` as a universal fallback when the native `pw-play`/`paplay`
can't handle the format.
- **Browse dialog with auto-preview.** The 📁 browse button opens a
`FileChooserDialog`; selecting a file auto-plays it so you can hear it before
confirming. The ▶ play button still works on the current selection.
## [2.03.15] - 2026-06-10
### Added
- **Wakeword engine CRUD.** The wakeword server section now has the same full
management UI as STT engines: a named-preset selector combo, + Add, Quickstart
(with 4 common wyoming-openwakeword templates), ⟳ reload, and Delete. Existing
users are migrated: their `wakeword_uri` / `wakeword_model` become the first
preset automatically.
## [2.03.14] - 2026-06-10
### Fixed
- **Engines tab.** Removed the "Internal engine — device & precision" section
header. The Device and Compute type fields already only appear when a local
engine type is selected; the separate header was redundant.
## [2.03.13] - 2026-06-10
### Changed
- **Settings header bar.** Save and Save & Restart moved from the bottom button
bar into the title bar (GTK HeaderBar). The X button closes without saving.
Bottom button row removed.
- **Section icon alignment.** Icons in section headers are now vertically
centred with the label text (`SMALL_TOOLBAR` size, `valign=CENTER`).
## [2.03.12] - 2026-06-10
### Added
- **Icons in settings.** All tab labels (Presets, Engines, Input, General,
Benchmark, Log, Manual, About) and every section header inside each tab now
show a small GTK symbolic icon, making the layout easier to scan.
### Fixed
- **Resize grip position.** The grip indicator now appears correctly at the
bottom-right corner below the notebook, not misplaced in the tab bar.
## [2.03.11] - 2026-06-09
### Added
- **Resize grip indicator.** A dotted SE-corner grip is drawn over the
bottom-right of the settings window so users discover it is resizable.
## [2.03.10] - 2026-06-09
### Fixed
- **Server RAM probe.** Prometheus `/metrics` is almost always at the server
root (`http://host:port/metrics`), not under `/v1`. Now tries the root URL
first before falling back to the API base path.
## [2.03.09] - 2026-06-09
### Added
- **Server RAM in benchmark.** For remote/Docker STT engines the benchmark now
probes the server's Prometheus `/metrics` endpoint for
`process_resident_memory_bytes` (standard Python/Go exporter) or
`container_memory_rss` (cAdvisor) and shows the server-side RSS in MB in the
RAM column. Falls back to `server` when the endpoint is not exposed.
## [2.03.08] - 2026-06-09
### Fixed
- **Engines tab layout.** The "Internal engine — device & precision" section is
now hidden when a remote (Server) or streaming engine type is selected —
removing the confusing whitespace gap and irrelevant device controls for
non-local engines.
## [2.03.01] - 2026-06-09
### Added
- **RAM usage column in benchmark.** The results table now shows a **RAM (MB)**
column — the increase in process RSS while the transcription ran. For local
models this captures the memory cost of loading the model on first use; for
remote engines it shows `—` (work happens server-side). Values are measured via
`/proc/self/status` (VmRSS), so they reflect actual resident memory, not
virtual address space.
## [2.03.00] - 2026-06-09
### Fixed
- **"Not responding" / system instability on Save.** `_collect()` was calling
`socket.create_connection()` *synchronously* on the GTK main thread when
wakeword is enabled — freezing the UI for up to 1.5 s (longer if DNS is slow).
The check is now done on a daemon thread and the result is logged instead of
blocking the save path.
- **GTK thread-safety crash in wakeword model load.** `_ww_load()` read
`self.ww_uri.get_text()` from inside a background thread — unsafe. The URI is
now captured on the main thread before the thread is spawned.
- **HTTP 404 with WhisperX and other non-standard endpoints.** The remote
transcription call always appended `/audio/transcriptions` to the base URL, but
services like WhisperX use `/transcribe` as the full path. The URL path is now
inspected: if it is anything other than empty / `/v1` / `/v1.0`, the URL is
used as the complete endpoint with nothing appended — so
`http://host:8081/transcribe` works out of the box.
- **Log levels.** `logbuffer` now stores `(timestamp, level, message)` tuples and
accepts a `level=` keyword (`DEBUG` / `INFO` / `WARNING` / `ERROR`). The Log
tab gains a **Level** dropdown (Verbose · Info · Warning · Error) that filters
the displayed entries live. Wakeword and socket errors are now tagged
`WARNING`; library records are forwarded at their native level.
### Added
- **Wakeword server preset dropdown** (Input → Hands-free wakeword). A
**Server preset** combo lists all configured wakeword server engines by name.
Picking one auto-fills the URI and model fields and re-probes reachability.
The selection is persisted as `wakeword_active` in config.
## [2.02.03] - 2026-06-09
### Added
- Wakeword server preset dropdown in Input tab.
## [2.02.02] - 2026-06-09
### Fixed
- License tab now renders with markdown styling.
- Benchmark pane minimum height (320 px, `shrink=False`) prevents the engine
list or results table from collapsing to zero when the window is small.
## [2.02.01] - 2026-06-09
### Added
- Last benchmark time and accuracy shown on the selected STT engine in the
Engines tab. Persisted to config so it survives restarts.
## [2.02.00] - 2026-06-09
### Added
- **Language metadata in benchmark.** The engine checkbox list shows supported
language codes next to each engine (fetched async). Filter box searches by
language code. Results table has a **Lang** column. Data comes from the
`/v1/models` `language` field (faster-whisper-server) or NVIDIA NIM `/metadata`.
## [1.9.5] - 2026-06-09
### Added
- **Emoji picker search.** A search field at the top of the emoji picker filters
all categories in real time using Unicode character names (e.g. "fire", "dog",
"heart"). Typing hides the category bar and shows matching results; clearing
restores the category view.
### Fixed
- **Manual tab now shows content.** `MANUAL.md` is copied next to the package
module so the Manual tab finds it in both venv and deb installs.
- **Info banner no longer bright blue.** The `.bt-infobox` background now uses
a neutral 5 % tint of the foreground colour instead of the theme accent
colour, so text stays readable on any theme.
## [1.9.4] - 2026-06-09
### Changed
- **Settings UI completely redesigned.** All six settings tabs (Presets, Engines,
Input, General, Input, General) now use a card-based layout following GTK3 best
practices: related fields are grouped inside visually distinct cards with bold
section titles. CSS is injected at start-up to give cards a consistent rounded
border (`boxed-list` + `bt-card`) and a styled info banner at the top of each
tab.
- **Dialog is larger (740×700 px) and every tab scrolls.** The notebook pages
now wrap their content in a `Gtk.ScrolledWindow` so no fields are ever clipped,
even on small screens.
- **Engines toolbar reorganised.** Creation actions (+ Add, + Stream, Quickstart)
are left-aligned; destructive/status actions (Delete, Test, ⟳) are
right-aligned via `pack_end`, making the bar scannable at a glance.
- **Section titles replace plain separators.** The old `Gtk.Separator` +
unstyled `Gtk.Label` pattern is gone; every section now has a small, dimmed,
bold all-caps header rendered with markup.
- **Cleaner section names.** "WW - Wakeword (Hands-free)" → "Hands-free
wakeword"; "Audio cues (manual dictation)" → "Audio cues (keyboard / hotkey
dictation)"; "Local engine … device & precision" → "Internal engine — device &
precision".
## [1.9.3] - 2026-06-09
### Added
- **ⓘ info buttons on every settings field.** Each field in every tab now has a
small information icon that opens a plain-language help popover when clicked —
so non-technical users can understand what each setting does without hovering
or reading the manual.
- **Manual tab in Settings.** A new "Manual" tab shows the full `MANUAL.md`
reference doc directly inside the Settings window.
- **Quickstart templates for engines.** A "Quickstart ▾" button in the STT and
LLM engine toolbars opens a menu of common services (OpenAI, Groq, OpenRouter,
Ollama, LM Studio, vLLM, llama-swap, faster-whisper-server, NVIDIA Riva) and
pre-fills the form — one click to configure a provider.
### Changed
- **Engine type names are now human-readable.** STT types now read "Internal —
faster-whisper, runs inside the app", "Server — OpenAI-compatible API (LAN or
cloud)", and "Realtime — NVIDIA Riva / NIM streaming" instead of the raw
identifiers. LLM types read "LAN server — runs on your machine or local
network" and "Cloud service — OpenAI, Groq, OpenRouter, …".
- **Device selector now shows "GPU (CUDA)" instead of "cuda"**, and compute
types have plain-language descriptions (e.g. "int8 — fast, less memory").
## [1.9.2] - 2026-06-09
### Added
- **Emoji picker for preset icons.** The "Icon (emoji)" field in Settings →
Presets now has a 😀 button that opens a scrollable emoji grid (60 common
emojis across six categories). Click any emoji to insert it — or keep typing
directly into the field as before.
## [1.9.1] - 2026-06-08
### Changed
- **Settings opens instantly.** Each tab's contents are now built the first time
you view it instead of all up front, so the dialog no longer pauses ~1.3s
constructing the file-choosers in the Input/Benchmark tabs. Saving force-builds
any tab you didn't visit first, so no field is ever missed.
- **Connection dots moved beside their field.** The Wakeword and TTS reachability
dots now sit just left of the URL entry (matching the Engines tab) instead of
at the far right of the row.
### Fixed
- **Settings could be opened more than once.** Choosing Settings while it's
already open now raises the existing window instead of stacking a second copy.
## [1.9.0] - 2026-06-08
### Added
- **Connection indicators** for remote endpoints. The **Wakeword engine** field
(Input tab — renamed from "Wyoming URI" to read more generally) and the **TTS
URL** field (Benchmark tab) now show a coloured dot: green when the server is
reachable, red when it's configured but unreachable, grey when blank — mirroring
the STT/LLM engine dots. It's a lightweight background TCP probe, refreshed when
the dialog opens, when you press ⟳, and when you leave the field.
## [1.8.1] - 2026-06-08
### Fixed
- **Settings dialog and control panel wouldn't open on some desktops.** When the
gvfs `org.gtk.vfs.UDisks2VolumeMonitor` dbus service fails to activate (common
on headless or minimal sessions), every `Gtk.FileChooserButton` blocked ~25s on
a `StartServiceByName` timeout while realizing — so the Settings dialog never
finished appearing, and the stalled GTK main loop froze the panel too. Blitztext
now selects GIO's native `/proc/mounts` volume monitor
(`GIO_USE_VOLUME_MONITOR=unix`) before any window is realized, so file choosers
open instantly with no dbus dependency.
## [1.8.0] - 2026-06-08
### Added
- **Send by voice**: say a distinctive phrase like **"computer send"** at the
start or end of a clip and the word is stripped, then the rest is typed **and
submitted with Enter** — the spoken equivalent of "stop + paste + Enter".
Mainly for hands-free use, where you can't press a key. Configure under
Settings → Input → "Send words", or `[routing] send_keywords`. Off by default;
because it presses Enter, use a multi-word phrase (e.g. your wakeword + "send")
so a sentence that merely ends in "send" doesn't submit by accident. Matched
the same edge-anchored, ASR-tolerant way as routing/cancel keywords.
- **Wakeword benchmark** (Settings → Benchmark): stress-test hands-free
detection. It synthesizes short sentences with your wake phrase spoken in
random voices (plus pure-filler utterances with none), streams them to your
wyoming-openwakeword server, and reports **recall** (how reliably it fires),
**false fires**, and a **per-voice** breakdown. Speech comes from any
OpenAI-compatible TTS server (Kokoro-FastAPI, XTTS, OpenAI, …): set its URL,
optional API-key env var, model, and voices under the new `[tts]` config / the
Benchmark tab, and use **Connect** to test it (it auto-fills the voice list
when the server exposes one).
## [1.7.1] - 2026-06-08
### Fixed
- **Overlay waveform and silence countdown ring never appeared** on systems
where PortAudio/`sounddevice` can't open the default input — notably PipeWire
boxes, where opening an input stream simply hangs. Both the live waveform and
the auto-stop countdown are driven by a single level meter, which was the only
part of the app still using `sounddevice` (everything else records via
`pw-record`). The meter now streams raw PCM from the **same system recorder as
the WAV recorder** (`pw-record`/`parecord`/`arecord`) and computes the level
itself, so it works wherever recording works — on both the hotkey and
hands-free (wakeword) paths, plus the mic-level preview in Settings. No more
PortAudio dependency for metering.
- **App reported itself as "`__main__.py`"** in the taskbar and in GNOME's
"… is not responding" dialog. Launched via `python -m blitztext`, GTK's default
program name is `argv[0]`'s basename. It now sets `prgname`/application name to
**Blitztext** before any window is realized (and the desktop file gains
`StartupWMClass=blitztext` for the .desktop match + icon), without touching the
`-m blitztext` entry point.
## [1.7.0] - 2026-06-07
### Added
- **Spoken cancel keyword**: say a word like **"abbrechen"** (or "cancel") at the
start or end of a clip and the whole dictation is **discarded** — it is never
routed, rewritten, or typed anywhere. Mainly rescues an accidentally triggered
(e.g. wakeword) recording. Configure under Settings → Mic/Cues → "Cancel words",
or `[routing] cancel_keywords` (default `["abbrechen", "cancel"]`; empty list
disables it). Matched the same edge-anchored, ASR-tolerant way as routing
keywords, so the word buried mid-sentence won't trip it.
## [1.6.0] - 2026-06-07
### Fixed
- **Session freeze when the overlay's caret tracking was active** (could lock up
the whole GNOME/X11 desktop, forcing a logout/reboot). The AT-SPI caret tracker
subscribed to the high-frequency `object:text-caret-moved` signal and made
**synchronous, blocking AT-SPI reads from inside the event handler** — which
re-enters the accessibility dispatcher and is stormed by the app's *own*
`xdotool` typing (one event per character), congesting the a11y bus until the
desktop stopped responding. It now tracks **focus changes only** and reads the
caret rectangle lazily (once, when the overlay shows), never from inside an
event dispatch.
### Changed
- **Matched preset is fused into the overlay instead of a desktop notification**:
when voice routing picks a preset, the overlay shows its emoji icon, name, and
the spoken keyword on a banner, and narrates the phase ("Transcribing…" →
"Rewriting…"). With the overlay on, the redundant per-dictation notifications
are suppressed (errors still notify); headless/overlay-off keeps notifications.
### Added
- **Live LLM rewrite in the overlay**: rewrite presets now stream the model's
output into the bubble token-by-token, so you watch it write. The delivered
text is still the complete result, typed once the rewrite finishes.
## [1.5.1] - 2026-06-07
### Added
- **Silence auto-stop countdown ring** on the dictation overlay: when you stop
speaking, a full circle wrapping the microphone glyph drains clockwise as the
trailing-silence timer runs out, recolouring from calm cyan to an urgent red
and emptying exactly as the recording auto-stops. It spans the configured
"Silence to stop (s)" window (`[wakeword] silence_seconds`), fades back in/out
as you pause and resume, and so finally makes the hands-free auto-stop visible
instead of a silent surprise.
## [1.5.0] - 2026-06-07
### Added
- **On-screen dictation overlay** (Settings → General → "Visual overlay", or
`[general] overlay_enabled`, default on): the moment recording starts — by
hotkey **or** wakeword — a translucent bubble appears at the cursor showing a
pulsing **microphone**, a **live waveform** of your mic level, and the
**recognised text** (word-by-word with a realtime streaming STT engine, or the
final result as a brief confirmation otherwise). Its tail points at where the
text will land: it follows the **text caret** when the focused app exposes it
over accessibility (AT-SPI), otherwise the **mouse pointer**, otherwise a
screen corner — tune via `[general] overlay_anchor = "caret" | "pointer" |
"corner"`. The window is click-through and never takes focus, and it finally
gives **hands-free wakeword sessions** visible feedback (their notifications
are suppressed by design). X11 only; falls back to a corner where the cursor
can't be located.
### Changed
- **Presets are speakable by name**: voice routing now matches a preset's *name*
as an implicit keyword, so a preset works by voice even with no keywords
configured (e.g. just say "nicer email …"). Explicit keywords still take
precedence, and preset names also bias the STT for better recognition.
- **General settings switches** moved to the far right of each row, each with an
inline description so it's clear what the toggle does without hovering.
- **About**: added a "Copyright: 2026 mARTin Bierschenk - Design" line.
## [1.4.0] - 2026-06-07
### Added
@ -176,7 +855,9 @@ into that field.
AppIndicator typelibs and GNOME `ubuntu-appindicators` extension are already
present on the target host).
[Unreleased]: https://github.com/mARTin-B78/blitztext-app-linux/compare/v1.1.0...HEAD
[Unreleased]: https://github.com/mARTin-B78/blitztext-app-linux/compare/v1.5.1...HEAD
[1.5.1]: https://github.com/mARTin-B78/blitztext-app-linux/compare/v1.5.0...v1.5.1
[1.5.0]: https://github.com/mARTin-B78/blitztext-app-linux/compare/v1.4.0...v1.5.0
[1.1.0]: https://github.com/mARTin-B78/blitztext-app-linux/compare/v1.0.1...v1.1.0
[1.0.1]: https://github.com/mARTin-B78/blitztext-app-linux/compare/v1.0.0...v1.0.1
[1.0.0]: https://github.com/mARTin-B78/blitztext-app-linux/releases/tag/v1.0.0

View File

@ -13,9 +13,13 @@ experiment along those lines was moved out to
the optional rewrite step calls out to an LLM.
<p align="center">
<img src="../Screenshots/panel.png" alt="Blitztext control panel" width="380">
<img src="../Screenshots/main-panel.png" alt="Blitztext control panel" width="360">
</p>
<p align="center">
<img src="../Screenshots/overlay-listening.png" alt="On-screen overlay while listening" width="360">
&nbsp;&nbsp;
<img src="../Screenshots/tray-menu.png" alt="Blitztext system-tray menu" width="300">
<img src="../Screenshots/overlay-result.png" alt="On-screen overlay showing transcription result" width="360">
</p>
## Inspiration
@ -39,49 +43,112 @@ Each normal hotkey **toggles**: press to start recording, press again to stop
then it transcribes, optionally rewrites, and types the result where your cursor
is. Streaming workflows type stable words live while you speak.
**Cancel by voice:** say *"abbrechen"* (or *"cancel"*) at the start or end of a
clip and the whole dictation is discarded — never routed, rewritten, or typed.
It's the rescue for an accidentally triggered (e.g. wakeword) recording. Set the
words under `[routing] cancel_keywords` (default `["abbrechen", "cancel"]`; an
empty list turns it off).
**Send by voice:** say a distinctive phrase like *"computer send"* at the start
or end of a clip and the word is stripped, then the rest is typed **and submitted
with Enter** — the spoken equivalent of "stop + paste + Enter", ideal hands-free.
Off by default; set the phrases under `[routing] send_keywords` (use a multi-word
phrase so a sentence merely ending in "send" doesn't submit by accident).
While you dictate, an optional **on-screen overlay** (Settings → General →
"Visual overlay", default on) shows a translucent bubble at the cursor with a
pulsing microphone, a live waveform of your mic level, and the recognised text —
word-by-word in streaming mode, or the final result as a brief confirmation. Its
tail points at the text caret (via AT-SPI accessibility) or the mouse pointer; it
is click-through, never steals focus, and also gives hands-free wakeword sessions
visible feedback. Tune the anchor with `[general] overlay_anchor`. X11 only.
## Screenshots
Everything is configured in the **Settings** window — every tab has tooltips and
screen-reader (ATK) support. Click any image to open it full size.
Everything is configured in the **Settings** window — the sidebar gives quick
access to every page. Click any image to open it full size.
### Main panel & overlay
<p align="center">
<a href="../Screenshots/settings-presets.png"><img src="../Screenshots/settings-presets.png" alt="Presets settings tab" width="100%"></a><br>
<em><b>Presets</b> — your dictation actions. Each preset is either a plain transcription or an LLM rewrite, and carries its own spoken keyword(s) for voice routing, an optional global hotkey, and a custom rewrite prompt.</em>
<a href="../Screenshots/main-panel.png"><img src="../Screenshots/main-panel.png" alt="Blitztext main panel" width="46%"></a>
&nbsp;
<a href="../Screenshots/overlay-listening.png"><img src="../Screenshots/overlay-listening.png" alt="Overlay — listening" width="46%"></a>
</p>
<p align="center">
<em><b>Left:</b> Control panel listing all presets with icons, descriptions, and hotkeys.</em>
&nbsp;&nbsp;
<em><b>Right:</b> On-screen overlay showing the live waveform while listening.</em>
</p>
<p align="center">
<a href="../Screenshots/settings-engines.png"><img src="../Screenshots/settings-engines.png" alt="Engines settings tab" width="100%"></a><br>
<em><b>Engines</b> — your speech-to-text and language-model back-ends, local or remote. Add and rename engines, watch live online/offline status, and pick models from a searchable list fetched straight from the endpoint.</em>
<a href="../Screenshots/overlay-result.png"><img src="../Screenshots/overlay-result.png" alt="Overlay — transcription result" width="46%"></a>
</p>
<p align="center">
<em>Overlay after transcription — preset name and recognised text appear at the cursor.</em>
</p>
### Settings — General & Input
<p align="center">
<a href="../Screenshots/settings-presets.png"><img src="../Screenshots/settings-presets.png" alt="Presets page" width="48%"></a>
&nbsp;
<a href="../Screenshots/settings-general.png"><img src="../Screenshots/settings-general.png" alt="General page" width="48%"></a>
</p>
<p align="center">
<em><b>Presets</b> — dictation actions with keywords, hotkeys, LLM mode, and custom prompts.</em>
&nbsp;&nbsp;
<em><b>General</b> — microphone, output mode, language hint, notifications, overlay, autostart.</em>
</p>
<p align="center">
<a href="../Screenshots/settings-input.png"><img src="../Screenshots/settings-input.png" alt="Input settings tab" width="100%"></a><br>
<em><b>Input</b> — how you start and stop dictation: the modifier-key scheme (Ctrl+Win / Ctrl / Alt / Esc) or custom hotkeys, plus the silence-based auto-stop (VAD), the quality gate, and audio cues.</em>
<a href="../Screenshots/settings-keyboard.png"><img src="../Screenshots/settings-keyboard.png" alt="Keyboard page" width="48%"></a>
&nbsp;
<a href="../Screenshots/settings-wakeword.png"><img src="../Screenshots/settings-wakeword.png" alt="Wakeword page" width="48%"></a>
</p>
<p align="center">
<em><b>Keyboard</b> — modifier-key scheme or direct hotkeys, quality gate, audio cues.</em>
&nbsp;&nbsp;
<em><b>Wakeword</b> — hands-free dictation via a Wyoming/openWakeWord server, with live level meter and model picker.</em>
</p>
<p align="center">
<a href="../Screenshots/wakeword.png"><img src="../Screenshots/wakeword.png" alt="Wakeword (hands-free) settings" width="100%"></a><br>
<em><b>Wakeword (hands-free)</b> — point Blitztext at a Wyoming/openWakeWord server, choose a wake model, and test the connection live so a spoken keyword starts dictation with no keys at all.</em>
</p>
### Settings — Engines
<p align="center">
<a href="../Screenshots/settings-general.png"><img src="../Screenshots/settings-general.png" alt="General settings tab" width="100%"></a><br>
<em><b>General</b> — core preferences: microphone with a live level meter, output mode (type vs. paste), language hint, type delay, and autostart on login.</em>
<a href="../Screenshots/settings-stt-engines.png"><img src="../Screenshots/settings-stt-engines.png" alt="STT Engines page" width="48%"></a>
&nbsp;
<a href="../Screenshots/settings-llm-engines.png"><img src="../Screenshots/settings-llm-engines.png" alt="LLM Engines page" width="48%"></a>
</p>
<p align="center">
<em><b>STT Engines</b> — speech-to-text back-ends (local faster-whisper, OpenAI-compatible server, or Riva realtime), with green/red status dot and Test button.</em>
&nbsp;&nbsp;
<em><b>LLM Engines</b> — language-model back-ends for text rewriting (LAN server or cloud service).</em>
</p>
<p align="center">
<a href="../Screenshots/settings-benchmark.png"><img src="../Screenshots/settings-benchmark.png" alt="Benchmark settings tab" width="100%"></a><br>
<em><b>Benchmark</b> — compare every configured STT engine against a reference WAV + transcript to find the fastest and most accurate, with a Device column (CPU / GPU / remote).</em>
</p>
### Settings — Benchmark
<p align="center">
<a href="../Screenshots/settings-log.png"><img src="../Screenshots/settings-log.png" alt="Log settings tab" width="100%"></a><br>
<em><b>Log</b> — the in-app log buffer: a live view of recording, transcription, routing, and wakeword events for quick troubleshooting.</em>
<a href="../Screenshots/settings-benchmark-stt.png"><img src="../Screenshots/settings-benchmark-stt.png" alt="Benchmark — STT page" width="48%"></a>
&nbsp;
<a href="../Screenshots/settings-benchmark-wakeword.png"><img src="../Screenshots/settings-benchmark-wakeword.png" alt="Benchmark — Wakeword page" width="48%"></a>
</p>
<p align="center">
<em><b>Benchmark — STT</b> — compare engines against a reference WAV + transcript; table shows speed, accuracy, device, and language support.</em>
&nbsp;&nbsp;
<em><b>Benchmark — Wakeword</b> — stress-test wakeword detection via a TTS server, reporting recall and false-fire rates per voice.</em>
</p>
### Settings — Log & About
<p align="center">
<a href="../Screenshots/settings-about.png"><img src="../Screenshots/settings-about.png" alt="About settings tab" width="100%"></a><br>
<em><b>About</b> — version, source link, changelog, and licence.</em>
<a href="../Screenshots/settings-log.png"><img src="../Screenshots/settings-log.png" alt="Log page" width="48%"></a>
&nbsp;
<a href="../Screenshots/settings-about.png"><img src="../Screenshots/settings-about.png" alt="About page" width="48%"></a>
</p>
<p align="center">
<em><b>Log</b> — live activity log for recording, transcription, routing, and wakeword events.</em>
&nbsp;&nbsp;
<em><b>About</b> — version, source link, inline changelog, and licence.</em>
</p>
## Requirements

View File

@ -6,4 +6,4 @@ counterpart to the macOS Blitztext menu bar app: it runs natively on the host
(not in a container) so it can type into any application via xdotool.
"""
__version__ = "1.4.0"
__version__ = "2.03.41"

View File

@ -1,43 +1,39 @@
"""Audio helpers: enumerate input devices and a live input-level meter.
Mic enumeration uses pactl (PipeWire/PulseAudio source names, which pw-record
and parecord accept via --target/-d). The level meter uses sounddevice to read
the chosen input and report a 0..1 level to a callback.
and parecord accept via --target/-d). The level meter shells out to the same
system recorder the app uses (pw-record/parecord/arecord), reading raw PCM from
its stdout and reporting a 0..1 level to a callback no Python audio binding,
so it works wherever the recorder does (PortAudio/sounddevice can't open the
default input on some PipeWire systems).
"""
from __future__ import annotations
import contextlib
import os
import shutil
import subprocess
import sys
import threading
from .recorder import detect_recorder
@contextlib.contextmanager
def _quiet_c_stderr():
"""Silence chatter written directly to fd 2 by C libraries.
# Raw-PCM (s16le, 16 kHz mono) variants of the recorders, streamed to stdout so
# we can RMS each chunk directly. Mirrors recorder.py's WAV commands but emits
# headerless PCM. pw-record/parecord default to stdout; arecord uses "-t raw".
_METER_ARGV: dict[str, list[str]] = {
"pw-record": ["pw-record", "--rate=16000", "--channels=1", "--format=s16", "-"],
"parecord": ["parecord", "--rate=16000", "--channels=1", "--format=s16le"],
"arecord": ["arecord", "-q", "-f", "S16_LE", "-r", "16000", "-c", "1", "-t", "raw"],
}
PortAudio/ALSA print harmless thread-teardown noise
("pthread_join ... failed", "PaUnixThread_Terminate ... failed") straight to
the underlying stderr file descriptor, which Python-level redirection can't
catch. We briefly point fd 2 at /dev/null around the offending call.
"""
try:
stderr_fd = sys.stderr.fileno()
except (AttributeError, ValueError, OSError):
yield # No real stderr fd (already captured/redirected) — nothing to do.
return
saved_fd = os.dup(stderr_fd)
devnull_fd = os.open(os.devnull, os.O_WRONLY)
try:
os.dup2(devnull_fd, stderr_fd)
yield
finally:
os.dup2(saved_fd, stderr_fd)
os.close(devnull_fd)
os.close(saved_fd)
# How to point each recorder at a specific pactl/pipewire source (mirrors
# recorder._DEVICE_FLAG; arecord uses ALSA names, so it stays on the default).
_DEVICE_FLAG: dict[str, list[str]] = {
"pw-record": ["--target"],
"parecord": ["-d"],
"arecord": [],
}
_CHUNK_BYTES = 3200 # 100 ms of 16 kHz, 16-bit, mono → ~10 Hz level updates
def list_mics() -> list[tuple[str, str]]:
@ -68,58 +64,79 @@ def list_mics() -> list[tuple[str, str]]:
class LevelMeter:
"""Open the given input device and call `on_level(0..1)` periodically."""
"""Stream mic audio via a system recorder and call `on_level(0..1)` ~10x/s.
def __init__(self, device: str = "", on_level=None):
self.device = device or None
Uses pw-record/parecord/arecord (the same recorders as the WAV recorder)
rather than a Python audio binding, so it works on PipeWire boxes where
PortAudio can't open the default input. Best-effort: ``start()`` returns
False if no recorder is available or the device can't be opened.
"""
def __init__(self, device: str = "", on_level=None, recorder: str = "auto", on_chunk=None):
self.device = device or ""
self.on_level = on_level
self._stream = None
self._lock = threading.Lock()
self.on_chunk = on_chunk # optional: called with raw s16le PCM bytes each chunk
self._recorder = recorder
self._proc: subprocess.Popen | None = None
self._thread: threading.Thread | None = None
self._stop = threading.Event()
def _argv(self, recorder: str) -> list[str]:
argv = list(_METER_ARGV[recorder])
flag = _DEVICE_FLAG.get(recorder, [])
if self.device and flag:
argv += flag + [self.device]
return argv
def start(self) -> bool:
import numpy as np
import sounddevice as sd
try:
recorder = detect_recorder(self._recorder)
except RuntimeError:
return False
if recorder not in _METER_ARGV:
return False
try:
self._proc = subprocess.Popen(
self._argv(recorder), stdout=subprocess.PIPE, stderr=subprocess.DEVNULL
)
except OSError:
self._proc = None
return False
self._stop.clear()
self._thread = threading.Thread(target=self._loop, daemon=True, name="LevelMeter")
self._thread.start()
return True
def _cb(indata, _frames, _time, _status):
level = float(np.sqrt(np.mean(np.square(indata)))) if indata.size else 0.0
def _loop(self) -> None:
import numpy as np
proc = self._proc
if proc is None or proc.stdout is None:
return
try:
while not self._stop.is_set() and proc.poll() is None:
chunk = proc.stdout.read(_CHUNK_BYTES)
if not chunk:
break
if self.on_chunk:
self.on_chunk(chunk)
samples = np.frombuffer(chunk, dtype=np.int16).astype(np.float32) / 32768.0
level = float(np.sqrt(np.mean(np.square(samples)))) if samples.size else 0.0
if self.on_level:
# Scale RMS (typically small) into a usable 0..1 range.
self.on_level(min(1.0, level * 12.0))
try:
with _quiet_c_stderr():
self._stream = sd.InputStream(
samplerate=16000, channels=1, dtype="float32",
blocksize=1600, device=self._resolve_device(), callback=_cb,
)
self._stream.start()
return True
except Exception: # noqa: BLE001 - device may be busy/unavailable
self._stream = None
return False
def _resolve_device(self):
# sounddevice wants an index/name it knows; pactl names rarely match, so
# fall back to the default input when the name isn't resolvable.
if not self.device:
return None
try:
import sounddevice as sd
for i, d in enumerate(sd.query_devices()):
if d["max_input_channels"] > 0 and self.device in d["name"]:
return i
except Exception: # noqa: BLE001
except Exception: # noqa: BLE001 - metering is eye-candy; never crash the app
pass
return None
def stop(self) -> None:
with self._lock:
if self._stream is not None:
self._stop.set()
proc, self._proc = self._proc, None
if proc is not None and proc.poll() is None:
proc.terminate()
try:
# PortAudio/ALSA spews thread-teardown noise to fd 2 here.
with _quiet_c_stderr():
self._stream.stop()
self._stream.close()
finally:
self._stream = None
proc.wait(timeout=1.0)
except subprocess.TimeoutExpired:
proc.kill()
thread, self._thread = self._thread, None
if thread is not None:
thread.join(timeout=1.0)

View File

@ -50,6 +50,7 @@ def set_enabled(enabled: bool) -> None:
f"Exec={_exec_command()}\n"
"Icon=blitztext\n"
"Terminal=false\n"
"X-GNOME-Autostart-enabled=true\n",
"X-GNOME-Autostart-enabled=true\n"
"X-GNOME-Autostart-Delay=12\n",
encoding="utf-8",
)

View File

@ -7,6 +7,7 @@ Benchmark tab to find the fastest and most accurate engine/model.
from __future__ import annotations
import dataclasses
import re
from dataclasses import dataclass
from pathlib import Path
@ -15,16 +16,33 @@ from . import stt
from .routing import normalize
def _rss_mb() -> float:
"""Current process RSS in MB via /proc/self/status (Linux only)."""
try:
with open("/proc/self/status") as fh:
for line in fh:
if line.startswith("VmRSS:"):
return int(line.split()[1]) / 1024.0 # kB → MB
except OSError:
pass
return 0.0
@dataclass
class BenchRow:
engine: str
url: str # base URL of the engine (empty for local)
model: str
device: str # "CPU" | "GPU" | "remote"
device: str # "CPU" | "CUDA" | "remote"
best_for: str # "Short clips" | "Short / medium" | "Long / batch" | "Streaming"
languages: list[str] # ISO 639-1 codes from /v1/models, empty if unknown
ok: bool
seconds: float
wer: float
accuracy: float # percent, max(0, 1-wer)*100
text: str
ram_mb: float = 0.0 # local RSS delta in MB
srv_ram_mb: float | None = None # server /metrics RSS in MB, None = not available
error: str = ""
@ -58,32 +76,103 @@ def wer(reference: str, hypothesis: str, *, case_sensitive: bool = False) -> flo
return _edit_distance(ref, hyp) / len(ref)
def _engine_device(engine, transcriber) -> str:
def _engine_device(engine, transcriber, _cache: dict) -> str:
if engine.is_local:
return "GPU" if getattr(transcriber, "device", "cpu") == "cuda" else "CPU"
return "remote"
return "CUDA" if getattr(transcriber, "device", "cpu") == "cuda" else "CPU"
url = engine.url
if url not in _cache:
_cache[url] = stt.detect_remote_device(url)
return _cache[url]
def _engine_best_for(engine) -> str:
if engine.type == "riva_realtime":
return "Streaming"
model = (engine.model or "").lower()
name = engine.name.lower()
if any(x in name for x in ("stream", "realtime", "real-time", "live")):
return "Streaming"
if engine.is_local:
if any(x in model for x in ("tiny", "base")):
return "Short clips"
if any(x in model for x in ("large",)):
return "Long / batch"
return "Short / medium"
# Remote endpoint
if any(x in name for x in ("large", "batch")):
return "Long / batch"
return "Short / medium"
def run(engines, wav_path: Path, reference: str, *, language: str = "",
case_sensitive: bool = True, get_local_transcriber=None, progress=None) -> list[BenchRow]:
case_sensitive: bool = True, get_local_transcriber=None, progress=None,
expand_models: bool = False) -> list[BenchRow]:
"""Benchmark each engine; calls progress(row) as each finishes.
expand_models=True fetches available models for each remote engine and
runs one benchmark row per model instead of just the configured one.
Accuracy is case-sensitive by default so wrong capitalisation counts.
"""
rows: list[BenchRow] = []
# Build the run list, optionally expanding remote engines by their models
run_list: list = []
for e in engines:
if expand_models and not e.is_local and not e.is_streaming:
models = stt.list_models(e.url, e.api_key_env)
if len(models) > 1:
for m in models:
run_list.append(dataclasses.replace(e, model=m,
name=f"{e.name} [{m}]"))
continue
run_list.append(e)
device_cache: dict = {}
meta_cache: dict = {} # url → list[ModelMeta]
server_ram_cache: dict = {} # url → float | None (MB from /metrics)
def _get_langs(e) -> list[str]:
if e.is_local:
return []
url = e.url
if url not in meta_cache:
meta_cache[url] = stt.list_models_meta(url, e.api_key_env)
for m in meta_cache[url]:
if not e.model or m.id == e.model or m.id.endswith("/" + e.model):
return m.languages
return meta_cache[url][0].languages if meta_cache[url] else []
rows: list[BenchRow] = []
for e in run_list:
tr = get_local_transcriber(e) if (e.is_local and get_local_transcriber) else None
# Snapshot server RAM before (if Prometheus metrics available)
if not e.is_local and e.url not in server_ram_cache:
server_ram_cache[e.url] = None # sentinel — probe once per URL
srv_before: float | None = None
if not e.is_local:
srv_before = stt.probe_server_ram_mb(e.url)
rss_before = _rss_mb()
res = stt.benchmark(e, wav_path, language=language, local_transcriber=tr)
rss_after = _rss_mb()
ram_delta = max(0.0, rss_after - rss_before)
srv_ram: float | None = None
if not e.is_local:
srv_after = stt.probe_server_ram_mb(e.url)
if srv_before is not None and srv_after is not None:
srv_ram = srv_after # report current RSS, not delta (server may not unload)
w = wer(reference, res.text, case_sensitive=case_sensitive) if res.ok else 1.0
row = BenchRow(
engine=e.name,
url=e.url,
model=e.model or ("local" if e.is_local else "(default)"),
device=_engine_device(e, tr),
device=_engine_device(e, tr, device_cache),
best_for=_engine_best_for(e),
languages=_get_langs(e),
ok=res.ok,
seconds=res.seconds,
wer=w,
accuracy=max(0.0, 1.0 - w) * 100.0,
text=res.text,
ram_mb=ram_delta,
srv_ram_mb=srv_ram,
error=res.error,
)
rows.append(row)

284
linux/blitztext/caret.py Normal file
View File

@ -0,0 +1,284 @@
"""Best-effort screen anchor for the overlay: *where* to point the bubble's tip.
The overlay wants to sit at "the cursor where the text will land". On X11 there
is no portable way to read the text caret of an arbitrary app, so we degrade
through a chain of decreasing precision:
1. Mouse pointer `xdotool getmouselocation`. Default; always available on X11;
a reliable proxy since the pointer is usually near where you're typing.
2. AT-SPI caret the real text insertion point (opt-in via overlay_anchor=caret).
Only native GTK/Qt apps expose it; many terminals / Electron / web views do
not. The blocking D-Bus read runs in a thread with a 300ms hard timeout so it
can never stall the GTK main loop and freeze the session.
3. Window / screen top-centre of the target window, else screen bottom-centre.
Everything here is defensive: any failure falls through to the next tier, and
the whole module is optional if AT-SPI isn't running you simply get the
pointer anchor. Returns a `(x, y, region)` anchor in **root/screen** pixels,
where `region` is the caret/line box so the overlay can avoid covering it.
"""
from __future__ import annotations
import shutil
import subprocess
import threading
import time
from dataclasses import dataclass
from .logbuffer import log
@dataclass
class Anchor:
x: int # root-relative x to point the tail tip at
y: int # root-relative y (top of the caret/line box)
height: int = 0 # caret/line height, so the bubble can clear the line
source: str = "" # "caret" | "pointer" | "window" | "corner" — for logging
# --------------------------------------------------------------------------- #
# Tier 1: AT-SPI caret tracking (best-effort, passive)
# --------------------------------------------------------------------------- #
class _CaretTracker:
"""Remember the most recently focused text object; read its caret lazily.
*Why so cautious.* AT-SPI queries (``get_character_extents`` and friends) are
**synchronous, blocking D-Bus round-trips into the target application**. The
earlier design subscribed to the high-frequency ``object:text-caret-moved``
signal and ran those blocking reads from *inside* the event handler. Two ways
that wedges a whole GNOME/X11 session:
Calling a synchronous AT-SPI method from within an AT-SPI event dispatch
re-enters the a11y dispatcher and can deadlock the accessibility bus.
``text-caret-moved`` fires once *per character* and delivering text is
exactly what this app does, typing via ``xdotool`` into the focused
field. So a single dictation became a storm of blocking round-trips on
the GTK main loop, congesting the a11y bus until the desktop froze.
So we now subscribe to **focus changes only** (rare, and never emitted by our
own synthetic typing), cache just the focused accessible, and do the one
blocking extents read **on demand** in :meth:`rect` called once, when the
overlay shows, outside any event dispatch. Worst case is a slightly delayed
overlay placement, never a frozen session.
"""
STALE_SECONDS = 30.0 # ignore a focus older than this
def __init__(self) -> None:
self._ok = False
self._listener = None
self._focused = None # last focused accessible (read lazily)
self._stamp = 0.0
self._Atspi = None
def start(self) -> bool:
"""Register the AT-SPI focus listener on the running GLib main loop.
Safe to call when accessibility is disabled it just returns False and
the anchor logic skips this tier from then on.
"""
try:
import gi
gi.require_version("Atspi", "2.0")
from gi.repository import Atspi # noqa: N813
except (ImportError, ValueError) as exc:
log(f"[overlay] AT-SPI unavailable, caret anchor disabled: {exc}")
return False
try:
# init() is idempotent; returns 0/1. Connects to the a11y registry.
Atspi.init()
self._Atspi = Atspi
self._listener = Atspi.EventListener.new(self._on_focus)
# Focus changes only. Deliberately NOT "object:text-caret-moved": that
# firehose (one event per typed character, including our own output)
# plus synchronous reads is what could freeze the session.
self._listener.register("object:state-changed:focused")
self._ok = True
log("[overlay] AT-SPI caret tracking active")
return True
except Exception as exc: # noqa: BLE001 - a11y bus may be down/locked
log(f"[overlay] AT-SPI init failed, caret anchor disabled: {exc}")
self._ok = False
return False
def stop(self) -> None:
try:
if self._listener is not None:
self._listener.deregister("object:state-changed:focused")
except Exception: # noqa: BLE001
pass
self._listener = None
self._focused = None
self._ok = False
def _on_focus(self, event) -> None:
# Runs on the GLib main thread (same loop GTK uses). Do the *minimum*:
# stash the focused accessible and stamp it. Crucially, make NO synchronous
# AT-SPI calls here — that would re-enter the a11y dispatcher and risk
# deadlocking the bus. The blocking extents read happens later, in rect().
try:
if not event.detail1:
return # a *de*focus event — nothing to track
self._focused = event.source
self._stamp = time.time()
except Exception: # noqa: BLE001
pass
def _caret_rect(self, acc) -> tuple[int, int, int, int] | None:
Atspi = self._Atspi
text = None
try:
text = acc.get_text_iface()
except Exception: # noqa: BLE001
text = None
if text is None:
return None
try:
offset = text.get_caret_offset()
# Extents of the character at the caret, in absolute screen coords.
ext = text.get_character_extents(offset, Atspi.CoordType.SCREEN)
x, y, w, h = ext.x, ext.y, ext.width, ext.height
if w == 0 and h == 0:
# End-of-line / empty field: fall back to the component box so we
# at least anchor on the right widget.
comp = acc.get_component_iface()
if comp is None:
return None
cext = comp.get_extents(Atspi.CoordType.SCREEN)
return int(cext.x), int(cext.y), 2, int(cext.height) or 18
if x < 0 or y < 0:
return None
return int(x), int(y), int(w) or 2, int(h) or 18
except Exception: # noqa: BLE001
return None
def rect(self) -> tuple[int, int, int, int] | None:
# Called once when the overlay shows (not on the a11y hot path).
# We run the blocking D-Bus extents read in a worker thread and join it
# with a hard 300ms deadline — so even a frozen or slow target app can
# never stall the GTK main loop long enough to freeze the session.
if not self._ok or self._focused is None:
return None
if time.time() - self._stamp > self.STALE_SECONDS:
return None
result: list[tuple[int, int, int, int] | None] = [None]
focused = self._focused
def _read() -> None:
try:
result[0] = self._caret_rect(focused)
except Exception: # noqa: BLE001
pass
t = threading.Thread(target=_read, daemon=True)
t.start()
t.join(timeout=0.3)
return result[0]
# --------------------------------------------------------------------------- #
# Tier 2 + 3: pointer and window fallbacks
# --------------------------------------------------------------------------- #
def _pointer() -> tuple[int, int] | None:
if not shutil.which("xdotool"):
return None
try:
out = subprocess.run(
["xdotool", "getmouselocation", "--shell"],
capture_output=True, text=True, check=True, timeout=0.5,
).stdout
except (OSError, subprocess.SubprocessError):
return None
vals: dict[str, str] = {}
for line in out.splitlines():
if "=" in line:
k, _, v = line.partition("=")
vals[k.strip()] = v.strip()
try:
return int(vals["X"]), int(vals["Y"])
except (KeyError, ValueError):
return None
def _window_box(window_id: str | None) -> tuple[int, int, int] | None:
"""Top-centre of the target window: (x, y, height_hint)."""
if not window_id or not shutil.which("xdotool"):
return None
try:
out = subprocess.run(
["xdotool", "getwindowgeometry", "--shell", window_id],
capture_output=True, text=True, check=True, timeout=0.5,
).stdout
except (OSError, subprocess.SubprocessError):
return None
vals: dict[str, str] = {}
for line in out.splitlines():
if "=" in line:
k, _, v = line.partition("=")
vals[k.strip()] = v.strip()
try:
x, y = int(vals["X"]), int(vals["Y"])
w = int(vals["WIDTH"])
return x + w // 2, y + 40, 0
except (KeyError, ValueError):
return None
def _screen_bottom_centre() -> tuple[int, int]:
try:
import gi
gi.require_version("Gdk", "3.0")
from gi.repository import Gdk
disp = Gdk.Display.get_default()
mon = disp.get_primary_monitor() or disp.get_monitor(0)
geo = mon.get_geometry()
return geo.x + geo.width // 2, geo.y + geo.height - 140
except Exception: # noqa: BLE001
return 960, 800 # last-ditch constant; better than crashing
_tracker: _CaretTracker | None = None
def start_tracking() -> None:
"""Begin passive caret tracking (call once, from the GTK main thread)."""
global _tracker
if _tracker is None:
_tracker = _CaretTracker()
_tracker.start()
def stop_tracking() -> None:
global _tracker
if _tracker is not None:
_tracker.stop()
_tracker = None
def resolve(anchor_mode: str, window_id: str | None) -> Anchor:
"""Resolve the on-screen anchor for the overlay, honouring `anchor_mode`.
anchor_mode:
"caret" -> caret pointer window corner
"pointer" -> pointer window corner
"corner" -> window corner
Always returns an Anchor (never None) so callers don't special-case failure.
"""
if anchor_mode == "caret" and _tracker is not None:
r = _tracker.rect()
if r is not None:
x, y, _w, h = r
return Anchor(x=x, y=y, height=h or 18, source="caret")
if anchor_mode in ("caret", "pointer"):
p = _pointer()
if p is not None:
return Anchor(x=p[0], y=p[1], height=0, source="pointer")
wb = _window_box(window_id)
if wb is not None:
return Anchor(x=wb[0], y=wb[1], height=0, source="window")
x, y = _screen_bottom_centre()
return Anchor(x=x, y=y, height=0, source="corner")

View File

@ -25,11 +25,22 @@ class Workflow:
# Optional per-workflow overrides of the [rewrite] defaults.
model: str | None = None
temperature: float | None = None
# Which LLM engine to use for this preset's rewrite step.
# "" (empty) = use whichever engine is currently active in the Engines tab.
llm_engine: str = ""
# Cosmetic, used by the GUI.
description: str = ""
icon: str = ""
@dataclass
class WakewordEngine:
"""A wyoming-openwakeword endpoint to compare in the wakeword benchmark."""
name: str = ""
uri: str = "tcp://127.0.0.1:10400"
model: str = "okay_computer"
@dataclass
class Config:
# general
@ -40,6 +51,9 @@ class Config:
notify: bool = True
notify_routing: bool = True # announce which preset/keyword a voice command matched
language: str = "de" # whisper hint; "" = autodetect
# on-screen overlay (mic + live waveform + recognised-text bubble)
overlay_enabled: bool = True
overlay_anchor: str = "pointer" # pointer | caret (AT-SPI, may freeze) | corner
# input scheme
input_mode: str = "modifiers" # "modifiers" (Ctrl+Win/Ctrl/Alt/Esc) | "hotkeys" (combos)
push_to_talk: bool = False
@ -72,6 +86,16 @@ class Config:
routing_hotkey: str = "<ctrl>+<alt>+<space>"
routing_default: str = "" # preset name used when no keyword matches; "" = first
routing_threshold: float = 0.82
# Spoken abort: if one of these words is heard at the start/end of a clip, the
# dictation is discarded — never transcribed onward, routed, rewritten, or
# typed. Empty list = disabled. Mainly for accidental wakeword triggers.
cancel_keywords: list[str] = field(default_factory=lambda: ["abbrechen", "cancel"])
# Spoken send: if one of these is heard at the start/end of a clip, it is
# stripped and the rest is delivered *and submitted with Enter* — the spoken
# equivalent of "stop + paste + Enter". Empty = disabled. Because it presses
# Enter, prefer a distinctive multi-word phrase (e.g. "computer send") so a
# sentence that merely ends in "send" doesn't submit by accident.
send_keywords: list[str] = field(default_factory=list)
# speech-to-text engines (presets)
stt_engines: list[STTEngine] = field(default_factory=list)
stt_active: str = ""
@ -80,11 +104,31 @@ class Config:
llm_active: str = ""
# wakeword
wakeword_enabled: bool = False
wakeword_active: str = "" # name of selected WakewordEngine preset, "" = custom
wakeword_uri: str = "tcp://127.0.0.1:10400"
wakeword_model: str = "okay_computer"
wakeword_sound_detected: str = "" # WAV played when the wakeword fires (speak now)
wakeword_sound_done: str = "" # WAV played when the command is captured
wakeword_silence_seconds: float = 2.0 # auto-stop after this much trailing silence
wakeword_cancel_model: str = "" # wakeword model that cancels an in-progress recording
wakeword_send_model: str = "" # wakeword model that finishes + sends (Enter) a recording
setup_complete: bool = False # True once the first-run wizard has been completed
# Text-to-speech for the wakeword benchmark — its own OpenAI-compatible
# endpoint (Kokoro, XTTS, OpenAI, …): base URL incl. /v1, an optional bearer
# key env var, a model id, and the voices to cycle through.
tts_url: str = ""
tts_api_key_env: str = ""
tts_model: str = ""
tts_voices: list[str] = field(
default_factory=lambda: ["alloy", "echo", "fable", "onyx", "nova", "shimmer"])
# Wakeword engines to compare in the benchmark (each a wyoming-openwakeword
# endpoint). Empty → the benchmark falls back to the live [wakeword] above.
wakeword_engines: list[WakewordEngine] = field(default_factory=list)
# STT benchmark: last-used WAV / reference transcript paths and options
bench_wav: str = ""
bench_ref: str = ""
bench_expand_models: bool = False
bench_last: dict = field(default_factory=dict) # {engine_name: {seconds, accuracy, ok}}
# workflows
workflows: list[Workflow] = field(default_factory=list)
@ -126,6 +170,8 @@ class Config:
out: list[str] = []
for w in self.workflows:
out.extend(w.keywords)
if w.name and w.name not in out:
out.append(w.name) # names are implicit triggers — bias STT for them too
return out
@property
@ -149,6 +195,7 @@ def load(path: Path = CONFIG_PATH) -> Config:
q = data.get("quality", {})
snd = data.get("sounds", {})
ww = data.get("wakeword", {})
tts = data.get("tts", {})
cfg = Config(
recorder=g.get("recorder", "auto"),
@ -156,8 +203,11 @@ def load(path: Path = CONFIG_PATH) -> Config:
output=g.get("output", "type"),
type_delay_ms=int(g.get("type_delay_ms", 4)),
notify=bool(g.get("notify", True)),
setup_complete=bool(g.get("setup_complete", False)),
notify_routing=bool(g.get("notify_routing", True)),
language=g.get("language", "de"),
overlay_enabled=bool(g.get("overlay_enabled", True)),
overlay_anchor=g.get("overlay_anchor", "pointer"),
model=w.get("model", "small"),
device=w.get("device", "auto"),
compute_type=w.get("compute_type", "auto"),
@ -171,6 +221,8 @@ def load(path: Path = CONFIG_PATH) -> Config:
routing_hotkey=rt.get("hotkey", "<ctrl>+<alt>+<space>"),
routing_default=rt.get("default", ""),
routing_threshold=float(rt.get("threshold", 0.82)),
cancel_keywords=list(rt.get("cancel_keywords", ["abbrechen", "cancel"])),
send_keywords=list(rt.get("send_keywords", [])),
input_mode=inp.get("mode", "modifiers"),
push_to_talk=bool(inp.get("push_to_talk", False)),
key_start=inp.get("start", "<ctrl>+<cmd>"),
@ -185,11 +237,22 @@ def load(path: Path = CONFIG_PATH) -> Config:
sound_before=snd.get("before", ""),
sound_after=snd.get("after", ""),
wakeword_enabled=bool(ww.get("enabled", False)),
wakeword_active=ww.get("active", ""),
wakeword_uri=ww.get("uri", "tcp://127.0.0.1:10400"),
wakeword_model=ww.get("model", "okay_computer"),
wakeword_sound_detected=ww.get("sound_detected", ""),
wakeword_sound_done=ww.get("sound_done", ""),
wakeword_silence_seconds=float(ww.get("silence_seconds", 2.0)),
wakeword_cancel_model=ww.get("cancel_model", ""),
wakeword_send_model=ww.get("send_model", ""),
tts_url=tts.get("url", "").rstrip("/"),
tts_api_key_env=tts.get("api_key_env", ""),
tts_model=tts.get("model", ""),
tts_voices=list(tts.get("voices", ["alloy", "echo", "fable", "onyx", "nova", "shimmer"])),
bench_wav=data.get("benchmark", {}).get("wav", ""),
bench_ref=data.get("benchmark", {}).get("ref", ""),
bench_expand_models=bool(data.get("benchmark", {}).get("expand_models", False)),
bench_last=dict(data.get("benchmark", {}).get("last", {})),
)
for entry in data.get("workflow", []):
@ -202,6 +265,7 @@ def load(path: Path = CONFIG_PATH) -> Config:
keywords=list(entry.get("keywords", [])),
model=entry.get("model"),
temperature=entry.get("temperature"),
llm_engine=entry.get("llm_engine", ""),
description=entry.get("description", ""),
icon=entry.get("icon", ""),
)
@ -218,11 +282,30 @@ def load(path: Path = CONFIG_PATH) -> Config:
url=e.get("url", "").rstrip("/"),
model=e.get("model", ""),
api_key_env=e.get("api_key_env", ""),
timeout=int(e.get("timeout", 30)),
)
for e in data.get("stt_engine", [])
] or [STTEngine("Local faster-whisper", "local", model=cfg.model)]
cfg.stt_active = data.get("stt", {}).get("active", cfg.stt_engines[0].name)
# Wakeword engines for the benchmark (optional; benchmark falls back to the
# live [wakeword] config when none are listed).
cfg.wakeword_engines = [
WakewordEngine(
name=e.get("name", ""),
uri=e.get("uri", "tcp://127.0.0.1:10400"),
model=e.get("model", "okay_computer"),
)
for e in data.get("wakeword_engine", [])
]
# Migration: ensure at least one engine exists so the CRUD UI always has a row.
if not cfg.wakeword_engines:
cfg.wakeword_engines = [WakewordEngine(
name=cfg.wakeword_active or "Local wyoming-openwakeword",
uri=cfg.wakeword_uri,
model=cfg.wakeword_model,
)]
# LLM engines (default: synthesized from the legacy [rewrite] block).
cfg.llm_engines = [
LLMEngine(
@ -254,8 +337,11 @@ def save(cfg: Config, path: Path = CONFIG_PATH) -> None:
"output": cfg.output,
"type_delay_ms": cfg.type_delay_ms,
"notify": cfg.notify,
"setup_complete": cfg.setup_complete,
"notify_routing": cfg.notify_routing,
"language": cfg.language,
"overlay_enabled": cfg.overlay_enabled,
"overlay_anchor": cfg.overlay_anchor,
},
"whisper": {
"model": cfg.model,
@ -283,6 +369,8 @@ def save(cfg: Config, path: Path = CONFIG_PATH) -> None:
"hotkey": cfg.routing_hotkey,
"default": cfg.routing_default,
"threshold": cfg.routing_threshold,
"cancel_keywords": cfg.cancel_keywords,
"send_keywords": cfg.send_keywords,
},
"quality": {
"min_speech_seconds": cfg.min_speech_seconds,
@ -297,18 +385,37 @@ def save(cfg: Config, path: Path = CONFIG_PATH) -> None:
},
"wakeword": {
"enabled": cfg.wakeword_enabled,
"active": cfg.wakeword_active,
"uri": cfg.wakeword_uri,
"model": cfg.wakeword_model,
"sound_detected": cfg.wakeword_sound_detected,
"sound_done": cfg.wakeword_sound_done,
"silence_seconds": cfg.wakeword_silence_seconds,
"cancel_model": cfg.wakeword_cancel_model,
"send_model": cfg.wakeword_send_model,
},
"tts": {
"url": cfg.tts_url,
"api_key_env": cfg.tts_api_key_env,
"model": cfg.tts_model,
"voices": cfg.tts_voices,
},
"benchmark": {
"wav": cfg.bench_wav,
"ref": cfg.bench_ref,
"expand_models": cfg.bench_expand_models,
"last": cfg.bench_last,
},
"wakeword_engine": [
{"name": e.name, "uri": e.uri, "model": e.model} for e in cfg.wakeword_engines
],
"stt": {"active": cfg.stt_active},
"stt_engine": [
{k: v for k, v in {
"name": e.name, "type": e.type, "url": e.url,
"model": e.model, "api_key_env": e.api_key_env,
}.items() if v or k in ("name", "type")}
"timeout": e.timeout if e.timeout != 30 else None,
}.items() if v is not None and (v or k in ("name", "type"))}
for e in cfg.stt_engines
],
"llm": {"active": cfg.llm_active},
@ -329,6 +436,8 @@ def save(cfg: Config, path: Path = CONFIG_PATH) -> None:
entry["model"] = wf.model
if wf.temperature is not None:
entry["temperature"] = wf.temperature
if wf.llm_engine:
entry["llm_engine"] = wf.llm_engine
if wf.description:
entry["description"] = wf.description
if wf.icon and wf.icon != "":
@ -360,6 +469,8 @@ type_delay_ms = 12 # per-keystroke delay for xdotool type (raise if chars
notify = true # desktop notifications for each phase
notify_routing = true # announce which preset/keyword a voice command matched (shown even hands-free)
language = "de" # Whisper language hint; "" = autodetect
overlay_enabled = true # on-screen mic + live waveform + recognised-text bubble at the cursor
overlay_anchor = "pointer" # pointer (near the mouse — safe) | caret (AT-SPI, may freeze) | corner
[input]
# How you start/stop dictation.
@ -418,6 +529,16 @@ enabled = true
hotkey = "<ctrl>+<alt>+<space>"
default = "Transcribe" # preset used when no keyword is recognised
threshold = 0.82 # 0..1 fuzzy-match strictness (higher = stricter)
# Say one of these at the start or end of a clip to DISCARD it — nothing is
# routed, rewritten, or typed. Handy when a wakeword fires by accident. Pick
# words you won't naturally end a real dictation with. Empty list = off.
cancel_keywords = ["abbrechen", "cancel"]
# Say one of these at the start or end of a clip to SEND it: the word is stripped
# and the rest is delivered AND submitted with Enter (spoken "stop+paste+Enter").
# Because it presses Enter, use a distinctive multi-word phrase (e.g. your
# wakeword + "send") so a sentence that just ends in "send" won't submit. Off by
# default; empty list = off.
send_keywords = []
[wakeword]
# Hands-free dictation using an external wyoming-openwakeword server.
@ -434,6 +555,30 @@ sound_done = ""
# Auto-stop the recording this many seconds after you stop speaking (silence).
silence_seconds = 2.0
[tts]
# Text-to-speech for the WAKEWORD BENCHMARK only (Settings → Benchmark). Point it
# at any OpenAI-compatible TTS server's /audio/speech (Kokoro-FastAPI, XTTS-v2,
# OpenAI, …). url = base incl. /v1; api_key_env = env var holding a bearer key
# (leave empty for no-auth local servers); model = the TTS model id; voices = the
# names your endpoint serves. Leave url/model empty to disable the benchmark.
url = ""
api_key_env = ""
model = ""
voices = ["alloy", "echo", "fable", "onyx", "nova", "shimmer"]
# Wakeword engines to COMPARE in the benchmark — add one block per
# wyoming-openwakeword endpoint/model you want side by side. Leave this out to
# just benchmark the live [wakeword] engine above.
# [[wakeword_engine]]
# name = "openWakeWord · computer"
# uri = "tcp://127.0.0.1:10400"
# model = "computer"
#
# [[wakeword_engine]]
# name = "microWakeWord · hey_jarvis"
# uri = "tcp://127.0.0.1:10500"
# model = "hey_jarvis"
# ----------------------------------------------------------------------------
# Speech-to-text engines (presets). The active one is used for transcription.
# type = "local" -> in-process faster-whisper (uses [whisper] above)

View File

@ -7,8 +7,10 @@ routing: one hotkey records, then the spoken keyword selects the preset.
from __future__ import annotations
import signal
import sys
import threading
import traceback
from typing import Callable
from . import llm, quality, stt
@ -19,18 +21,141 @@ from .notify import notify
from .paste import active_window_id, deliver
from .streaming import RivaRealtimeStreamer
from .recorder import Recording, detect_recorder
from .routing import route
from .routing import is_cancel, match_send, route
from .transcribe import Transcriber
# status_cb(state, workflow_name, message)
# state in {"loading", "idle", "recording", "streaming", "busy", "done", "error"}
StatusCallback = Callable[[str, str | None, str], None]
# A trailing pause shorter than this never arms the auto-stop countdown, so the
# overlay ring doesn't flicker in the gaps between words. The visible countdown
# therefore spans (silence_seconds - this) and begins full once you genuinely
# stop speaking.
_VAD_COUNTDOWN_GRACE = 0.35
# Cancel-watcher: accumulate this much audio before the first check, then
# re-check every time this many new bytes arrive. 3200 bytes = 100 ms at
# 16 kHz s16le mono. 0.8 s min avoids false positives on the very first chunk;
# 0.6 s poll keeps latency low without hammering the transcriber.
_CANCEL_MIN_BYTES = int(0.8 * 16000 * 2) # 25600
_CANCEL_POLL_BYTES = int(0.6 * 16000 * 2) # 19200
def _pcm_to_wav(path: str, pcm: bytes) -> None:
"""Write raw s16le 16 kHz mono PCM bytes as a minimal RIFF WAV."""
import struct
data_len = len(pcm)
with open(path, "wb") as f:
f.write(b"RIFF")
f.write(struct.pack("<I", 36 + data_len))
f.write(b"WAVE")
f.write(b"fmt ")
f.write(struct.pack("<IHHIIHH", 16, 1, 1, 16000, 32000, 2, 16))
f.write(b"data")
f.write(struct.pack("<I", data_len))
f.write(pcm)
class _CancelWatcher:
"""Listens to in-progress VAD audio and triggers cancel if a keyword is heard.
PCM chunks (raw s16le 16 kHz mono) are fed via :meth:`feed` from the VAD
level-meter thread. Every _CANCEL_POLL_BYTES of *new* audio (after an
initial _CANCEL_MIN_BYTES warm-up), a fast beam_size=1 transcription of the
accumulated buffer runs in a background thread. If a cancel keyword is
found, the supplied ``on_cancel`` callback fires once and the watcher stops.
"""
def __init__(self, transcriber, cancel_keywords, language, threshold, on_cancel):
self._transcriber = transcriber
self._keywords = cancel_keywords
self._language = language
self._threshold = threshold
self._on_cancel = on_cancel
self._buf = bytearray()
self._new_bytes = 0
self._active = True
self._lock = threading.Lock()
self._checking = False # prevents overlapping check threads
def feed(self, chunk: bytes) -> None:
if not self._active:
return
with self._lock:
if not self._active:
return
self._buf.extend(chunk)
self._new_bytes += len(chunk)
ready = (len(self._buf) >= _CANCEL_MIN_BYTES
and self._new_bytes >= _CANCEL_POLL_BYTES
and not self._checking)
if ready:
self._new_bytes = 0
self._checking = True
snapshot = bytes(self._buf)
if ready:
threading.Thread(target=self._check, args=(snapshot,), daemon=True,
name="CancelWatcher").start()
def stop(self) -> None:
with self._lock:
self._active = False
def _check(self, audio: bytes) -> None:
import os
import tempfile
from pathlib import Path
fd, tmp = tempfile.mkstemp(prefix="bt-cw-", suffix=".wav")
try:
os.close(fd)
_pcm_to_wav(tmp, audio)
text = self._transcriber.transcribe(
Path(tmp), language=self._language, beam_size=1)
with self._lock:
if not self._active:
return
kw = is_cancel(text, self._keywords, threshold=self._threshold)
if kw:
with self._lock:
if not self._active:
return
self._active = False
log(f'[cancel-watcher] “{kw}” heard live — cancelling immediately.')
self._on_cancel()
except Exception: # noqa: BLE001 — watcher must never crash the daemon
pass
finally:
Path(tmp).unlink(missing_ok=True)
with self._lock:
self._checking = False
class Daemon:
def __init__(self, cfg: Config, status_cb: StatusCallback | None = None):
def __init__(self, cfg: Config, status_cb: StatusCallback | None = None,
level_cb: Callable[[float], None] | None = None,
text_cb: Callable[[str], None] | None = None,
countdown_cb: Callable[[float | None, float], None] | None = None,
routing_cb: Callable[[str, str, str | None], None] | None = None):
self.cfg = cfg
self.status_cb = status_cb
# Optional UI feedback hooks for the on-screen overlay. The daemon stays
# UI-agnostic: these are no-ops in headless mode. level_cb gets the live
# mic level (0..1); text_cb gets the running transcript while streaming
# (and the live LLM rewrite); countdown_cb(seconds_left, window) drives the
# silence auto-stop ring (seconds_left=None while you're speaking, so the
# ring clears); routing_cb(icon, preset_name, keyword) fires when voice
# routing picks a preset, so the overlay can show it.
self.level_cb = level_cb
self.text_cb = text_cb
self.countdown_cb = countdown_cb
self.routing_cb = routing_cb
# An overlay consumes routing_cb, and it narrates every phase on-screen, so
# the redundant desktop notifications are fused into it (only errors still
# pop a bubble). Headless / overlay-off keeps the notifications.
self._overlay = routing_cb is not None
self._ov_meter = None
self._ov_text_final = ""
self._lock = threading.Lock()
self._recording: Recording | None = None
self._streaming: RivaRealtimeStreamer | None = None
@ -38,6 +163,7 @@ class Daemon:
self._active_workflow: Workflow | None = None
self._target_window: str | None = None
self._busy = False
self._abort_event = threading.Event()
self._prepared = False
self._listener = None
# Synthetic preset used by the voice-routing hotkey.
@ -80,14 +206,20 @@ class Daemon:
notify(title, body, urgency=urgency, enabled=self.cfg.notify)
def _dnotify(self, title: str, body: str = "", urgency: str = "normal") -> None:
"""Per-dictation notification — suppressed for hands-free (wakeword) sessions."""
if not self._session_silent:
"""Per-dictation notification — suppressed for hands-free (wakeword)
sessions, and (except for errors) when an overlay is narrating on-screen."""
if self._session_silent:
return
if self._overlay and urgency != "critical":
return # fused into the on-screen overlay instead of a desktop bubble
self._notify(title, body, urgency=urgency)
def _rnotify(self, title: str, body: str = "", urgency: str = "normal") -> None:
"""Routing feedback — which preset/keyword a voice command matched. Shown
even for hands-free sessions (it has its own toggle) so you can always see
what you triggered. Only fires on a real match, so it never spams silence."""
"""Routing feedback — which preset/keyword a voice command matched. When an
overlay is present this is shown there (via routing_cb) instead of a
notification; otherwise it pops a bubble. Only fires on a real match."""
if self._overlay:
return # shown on the overlay banner instead
notify(title, body, urgency=urgency, enabled=self.cfg.notify_routing)
def _emit(self, state: str, workflow: str | None = None, message: str = "") -> None:
@ -118,9 +250,30 @@ class Daemon:
log(f"Using remote STT '{engine.name}' — no local model to load")
self._prepared = True
self._init_wakeword()
self._install_freeze_diagnostic()
log("Ready.")
self._emit("idle", None, "Ready")
def _install_freeze_diagnostic(self) -> None:
"""Register SIGQUIT (Ctrl+\\ or kill -QUIT) to dump all thread stacks.
When the system appears frozen, run:
kill -QUIT $(pgrep -f blitztext)
and the full thread dump appears in the Blitztext log window.
"""
def _dump(_sig, _frame):
lines = ["\n=== FREEZE DIAGNOSTIC — all thread stacks ==="]
for tid, frame in sys._current_frames().items():
name = next((t.name for t in threading.enumerate() if t.ident == tid), str(tid))
lines.append(f"\n-- Thread: {name} (id={tid}) --")
lines.extend(traceback.format_stack(frame))
lines.append("=== END FREEZE DIAGNOSTIC ===")
log("\n".join(lines))
try:
signal.signal(signal.SIGQUIT, _dump)
except (OSError, ValueError):
pass # not available on all platforms
@property
def ready(self) -> bool:
return getattr(self, "_prepared", False)
@ -140,21 +293,110 @@ class Daemon:
silence = max(0.5, self.cfg.wakeword_silence_seconds)
def on_level(level):
# Feed the overlay waveform (the VAD meter is already capturing, so
# we reuse its level rather than opening a second stream).
if self.level_cb:
self.level_cb(level)
now = time.time()
if level > 0.05:
self._vad_last_speech = now
elif now - self._vad_started_at > 2.0 and now - self._vad_last_speech > silence:
if getattr(self, "is_recording", False):
if self.countdown_cb:
self.countdown_cb(None, silence) # speaking — no countdown
return
quiet = now - self._vad_last_speech
armed = now - self._vad_started_at > 2.0
if armed and quiet > _VAD_COUNTDOWN_GRACE:
# Mirror the auto-stop window into the overlay ring: full when
# you fall quiet, empty exactly as it fires.
if self.countdown_cb:
self.countdown_cb(silence - quiet, silence - _VAD_COUNTDOWN_GRACE)
if quiet > silence and getattr(self, "is_recording", False):
GLib.idle_add(lambda: self.finish_dictation(send_enter=False))
self._vad_stop()
elif self.countdown_cb:
self.countdown_cb(None, silence)
self._vad_meter = audio.LevelMeter(self.cfg.mic, on_level=on_level)
self._vad_meter.start()
# Wakeword-based action listener: if cancel/send wakeword models are
# configured, open a second Wyoming connection during recording so a
# dedicated wakeword ("stop", "send it") fires instantly — no Whisper pass.
self._action_listener = None
if self.cfg.wakeword_enabled and (
self.cfg.wakeword_cancel_model or self.cfg.wakeword_send_model):
from .wakeword import WakewordActionListener
cbs: dict = {}
if self.cfg.wakeword_cancel_model:
cbs[self.cfg.wakeword_cancel_model] = lambda: GLib.idle_add(
self.cancel_dictation)
if self.cfg.wakeword_send_model:
cbs[self.cfg.wakeword_send_model] = lambda: GLib.idle_add(
lambda: self.finish_dictation(send_enter=True))
self._action_listener = WakewordActionListener(
uri=self.cfg.wakeword_uri, model_callbacks=cbs, mic=self.cfg.mic)
self._action_listener.start()
# Whisper-based cancel watcher: fallback when no cancel wakeword model is
# set, or as belt-and-suspenders for the spoken cancel keyword list.
on_chunk = None
use_whisper_watcher = (self.cfg.cancel_keywords
and getattr(self, "transcriber", None) is not None
and not self.cfg.wakeword_cancel_model)
if use_whisper_watcher:
self._cancel_watcher = _CancelWatcher(
self.transcriber,
self.cfg.cancel_keywords,
self.cfg.language,
self.cfg.routing_threshold,
on_cancel=lambda: GLib.idle_add(self.cancel_dictation),
)
on_chunk = self._cancel_watcher.feed
else:
self._cancel_watcher = None
self._vad_meter = audio.LevelMeter(self.cfg.mic, on_level=on_level,
recorder=self.recorder_name, on_chunk=on_chunk)
ok = self._vad_meter.start()
# Safety net: if the LevelMeter fails to open the mic (e.g. device busy
# because the wakeword listener already holds a pw-record stream), the
# on_level callback never fires and dictation hangs forever. Add a hard
# 30-second timeout so the session always terminates.
_MAX_WAKEWORD_SECONDS = 30
if not ok:
log("[vad] LevelMeter failed to start — scheduling 30s hard timeout")
GLib.timeout_add(_MAX_WAKEWORD_SECONDS * 1000,
lambda: self.finish_dictation(send_enter=False) or False)
else:
# Even when the meter works, cap wakeword sessions at 60s.
GLib.timeout_add(60_000,
lambda: self.is_recording and self.finish_dictation(send_enter=False) or False)
def _vad_stop(self) -> None:
if getattr(self, '_vad_meter', None) is not None:
self._vad_meter.stop()
self._vad_meter = None
if getattr(self, "_cancel_watcher", None) is not None:
self._cancel_watcher.stop()
self._cancel_watcher = None
if getattr(self, "_action_listener", None) is not None:
self._action_listener.stop()
self._action_listener = None
def _ov_meter_start(self) -> None:
"""A level meter purely to drive the overlay waveform in streaming mode.
Non-streaming recordings reuse the VAD meter instead; this only runs when
an overlay is attached and we have no other level source. Best-effort:
LevelMeter.start() fails quietly if the device is busy."""
if not self.level_cb:
return
from . import audio
self._ov_meter = audio.LevelMeter(self.cfg.mic, on_level=self.level_cb, recorder=self.recorder_name)
self._ov_meter.start()
def _ov_meter_stop(self) -> None:
if getattr(self, "_ov_meter", None) is not None:
self._ov_meter.stop()
self._ov_meter = None
def _play_sound(self, sound_name: str) -> None:
if not self.cfg.sounds_enabled:
@ -204,6 +446,7 @@ class Daemon:
self._notify("Streaming unavailable", "Select a riva_realtime STT engine.", "critical")
return
self._stream_segment_text = ""
self._ov_text_final = ""
streamer = RivaRealtimeStreamer(
engine,
device=self.cfg.mic,
@ -222,7 +465,9 @@ class Daemon:
self._dnotify(f"{wf.name}", "Live transcript…")
try:
streamer.start()
self._ov_meter_start()
except Exception as exc: # noqa: BLE001
self._ov_meter_stop()
with self._lock:
if self._streaming is streamer:
self._streaming = None
@ -253,6 +498,7 @@ class Daemon:
self._busy = True
if streamer is not None:
streamer.stop()
self._ov_meter_stop()
if send_enter:
from .paste import press_enter
press_enter(win)
@ -275,19 +521,31 @@ class Daemon:
self._active_workflow = None
self._stream_segment_text = ""
rec = None
else:
busy = False
elif self._recording is not None:
streamer = None
if self._recording is None:
return
rec = self._recording
self._recording = None
self._active_workflow = None
busy = False
elif self._busy:
streamer = None
rec = None
busy = True
else:
return
if streamer is not None:
streamer.stop()
self._ov_meter_stop()
self._emit("idle", None, "Cancelled")
self._notify("Cancelled", "Streaming stopped.", "low")
self._play_sound("device-removed")
return
if busy:
self._abort_event.set()
self._emit("idle", None, "Cancelled")
self._play_sound("device-removed")
return
rec.discard()
self._emit("idle", None, "Cancelled")
self._notify("Cancelled", "Recording discarded.", "low")
@ -323,6 +581,14 @@ class Daemon:
return text[:cut + 1] if cut >= 0 else ""
def _on_stream_text(self, text: str, final: bool) -> None:
# Mirror the live hypothesis into the overlay bubble (the full current
# guess, which is more responsive than the delivered stable prefix).
if self.text_cb:
disp_seg = quality.clean(text, strip_trailing_punctuation=False)
running = (self._ov_text_final + " " + disp_seg).strip()
self.text_cb(running)
if final:
self._ov_text_final = running
stable = self._stable_stream_text(text, final)
if not stable:
return
@ -343,6 +609,7 @@ class Daemon:
self._stream_segment_text = "" if final else stable
def _on_stream_error(self, label: str, exc: Exception) -> None:
self._ov_meter_stop()
with self._lock:
self._streaming = None
self._active_workflow = None
@ -353,6 +620,7 @@ class Daemon:
# -- worker ---------------------------------------------------------------
def _process(self, audio_path, workflow: Workflow, window_id, send_enter: bool = False) -> None:
self._abort_event.clear()
label = workflow.name
try:
# Quality gate: drop silent / too-short clips before we even transcribe.
@ -375,13 +643,37 @@ class Daemon:
local_transcriber=self.transcriber,
timeout=self.cfg.timeout,
)
if self._abort_event.is_set():
log(f"{label}: cancelled during transcription.")
return
text = quality.clean(text, strip_trailing_punctuation=self.cfg.strip_trailing_punctuation)
text = quality.expand_spoken_punctuation(text)
if not text or (self.cfg.reject_hallucinations and quality.is_hallucination(text, duration)):
self._emit("idle", label, "No speech detected")
log("Nothing heard — no speech detected.")
return
# Spoken abort: a configured cancel word heard at an edge discards the
# whole clip — nothing is routed, rewritten, or typed. This is the
# rescue for an accidentally triggered (e.g. wakeword) dictation.
cancel_kw = is_cancel(text, self.cfg.cancel_keywords, threshold=self.cfg.routing_threshold)
if cancel_kw:
log(f"✗ Discarded by voice keyword “{cancel_kw}”.")
if self.text_cb:
self.text_cb("✗ Abgebrochen")
self._emit("idle", label, "Cancelled")
self._dnotify("Abgebrochen", f"{cancel_kw}“ gehört — verworfen.", "low")
return
# Spoken send: a configured word at an edge ("computer send") is
# stripped, and the rest is delivered AND submitted with Enter — the
# spoken equivalent of stop+paste+Enter. Mainly for hands-free use.
send_kw, text = match_send(text, self.cfg.send_keywords, threshold=self.cfg.routing_threshold)
if send_kw:
send_enter = True
log(f"⏎ Send keyword “{send_kw}” — delivering and pressing Enter.")
# Voice routing: pick the preset from a spoken keyword, strip it.
if workflow.mode == "route":
res = route(text, self.cfg.workflows, threshold=self.cfg.routing_threshold)
@ -391,6 +683,13 @@ class Daemon:
icon = (getattr(target, "icon", "") or "🎙") if target else "🎙"
via = f"{res.keyword}" if res.keyword else "no keyword → default"
self._emit("busy", label, f"{label} ({via})")
# Fuse the match onto the overlay (icon + preset + keyword); falls
# back to a desktop notification only when there's no overlay.
if self.routing_cb:
try:
self.routing_cb(icon, label, res.keyword)
except Exception: # noqa: BLE001 - UI must not break the engine
pass
self._rnotify(f"{icon} {label}", f"matched: {via}")
log(f"→ routed to {label} (matched: {via})")
else:
@ -403,19 +702,98 @@ class Daemon:
return
self._emit("busy", label, "Rewriting…")
self._dnotify(f"{label}", "Rewriting…")
# Show the transcribed text immediately so the user sees what was heard.
if self.text_cb:
self.text_cb(f"📝 {text}")
# Stream the rewrite into the overlay so you watch the model write
# (the bubble updates token-by-token). The delivered text is still
# the complete result, typed once the rewrite finishes.
on_token = None
if self.text_cb:
# Use a deque to stream the last ~400 chars to the overlay so
# Pango never has to lay out a 15 000-char code block on each
# token, and "".join() stays O(window) not O(total).
from collections import deque
_window: deque[str] = deque()
_window_len: list[int] = [0]
_OVERLAY_CHARS = 400
_thinking_frames = ["⏳ Thinking.", "⏳ Thinking..", "⏳ Thinking...", "⏳ Thinking"]
_thinking_state: list[int] = [0]
_first_token: list[bool] = [False]
def _pulse_thinking(_s=_thinking_state, _f=_first_token) -> bool:
if _f[0]:
return False
self.text_cb(_thinking_frames[_s[0] % len(_thinking_frames)])
_s[0] += 1
return True
try:
from gi.repository import GLib as _GLib
# idle_add ensures timeout_add runs on the GTK main thread —
# calling timeout_add from a background thread is not safe.
_GLib.idle_add(lambda: _GLib.timeout_add(400, _pulse_thinking) and False)
except Exception: # noqa: BLE001
pass
def on_token(delta: str,
_w=_window, _wl=_window_len, _f=_first_token) -> None:
_f[0] = True
_w.append(delta)
_wl[0] += len(delta)
# Trim old tokens from the front to stay within the window.
while _wl[0] > _OVERLAY_CHARS and len(_w) > 1:
_wl[0] -= len(_w[0])
_w.popleft()
self.text_cb("".join(_w))
# Use the preset's pinned engine when set, else the active one.
engine_name = getattr(target, "llm_engine", "") or ""
llm_engine = (
next((e for e in self.cfg.llm_engines if e.name == engine_name), None)
or self.cfg.active_llm
)
try:
text = llm.chat(
self.cfg.active_llm,
llm_engine,
target.prompt,
text,
model=target.model or None,
temperature=target.temperature,
timeout=self.cfg.timeout,
on_token=on_token,
abort_event=self._abort_event,
)
except LLMError as exc:
self._emit("error", label, str(exc))
self._dnotify("Rewrite failed", str(exc), "critical")
log(f"ERROR ({label} rewrite): {exc}")
if self.text_cb:
self.text_cb(f"{exc}")
return
except Exception as exc: # noqa: BLE001 - guard against any uncaught error
msg = f"LLM error: {exc}"
self._emit("error", label, msg)
log(f"ERROR ({label} rewrite unexpected): {exc}")
if self.text_cb:
self.text_cb(f"{msg}")
return
if self._abort_event.is_set():
log(f"{label}: cancelled during rewrite.")
return
# Sanity-check: reject responses that are >80 % whitespace —
# a model that's cold-starting or misconfigured sometimes streams
# spaces or blank lines instead of real output.
non_ws = sum(1 for c in text if not c.isspace())
if non_ws < max(1, len(text) * 0.20):
msg = "LLM returned mostly whitespace — discarded"
self._emit("error", label, msg)
log(f"ERROR ({label}): {msg} (len={len(text)})")
if self.text_cb:
self.text_cb(f"{msg}")
return
if not text:

File diff suppressed because it is too large Load Diff

View File

@ -85,13 +85,47 @@ class App:
self.cfg = cfg
self.tray_mode = tray_mode
self.tray = None
self.daemon = Daemon(cfg, status_cb=self._status_cb)
self._rows: dict[str, dict] = {}
self._active: str | None = None
# On-screen dictation HUD (mic + waveform + recognised-text bubble).
self.overlay = None
self._ov_state = "idle"
self._ov_streaming = False
if cfg.overlay_enabled:
try:
from .overlay import Overlay
self.overlay = Overlay(
anchor_mode=cfg.overlay_anchor,
on_cancel=lambda: threading.Thread(
target=self.daemon.cancel_dictation, daemon=True).start(),
)
if cfg.overlay_anchor == "caret":
from . import caret
caret.start_tracking()
except Exception: # noqa: BLE001 - overlay is optional eye-candy
self.overlay = None
self.daemon = Daemon(
cfg, status_cb=self._status_cb,
level_cb=self._on_level, text_cb=self._on_text,
countdown_cb=self._on_countdown,
# Only consume routing on the overlay when there's an overlay to show
# it on; otherwise the daemon keeps the desktop notification.
routing_cb=self._on_routing if self.overlay is not None else None,
)
_install_css()
self._build_window()
# Show the first-run wizard for fresh installs before the daemon starts.
if not getattr(cfg, "setup_complete", False):
from .setup_wizard import SetupWizard
wiz = SetupWizard(cfg, parent=None)
wiz.run()
threading.Thread(target=self._startup, daemon=True).start()
# -- window ---------------------------------------------------------------
@ -242,6 +276,55 @@ class App:
return
threading.Thread(target=lambda: self.daemon.toggle(wf), daemon=True).start()
# -- overlay feedback -----------------------------------------------------
def _on_level(self, level: float) -> None:
if self.overlay is not None:
self.overlay.set_level(level)
def _on_text(self, text: str) -> None:
if self.overlay is not None:
self.overlay.set_text(text)
def _on_countdown(self, remaining: float | None, total: float) -> None:
if self.overlay is not None:
self.overlay.set_countdown(remaining, total)
def _on_routing(self, icon: str, name: str, keyword: str | None) -> None:
if self.overlay is not None:
self.overlay.set_preset(icon, name, keyword)
def _overlay_status(self, state: str, message: str) -> None:
"""Translate engine phases into overlay show/update/hide (GTK thread)."""
ov = self.overlay
if ov is None:
return
if state in ("recording", "streaming"):
self._ov_streaming = state == "streaming"
self._ov_state = state
ov.show(state, getattr(self.daemon, "_target_window", None))
elif state == "busy":
self._ov_state = state
# The routing detail ("→ Nicer email (matched: …)") is shown on the
# preset banner, not as a phase chip — keep the chip a clean phase word.
phase = "Transcribing…" if message.startswith("") else message
ov.set_state("busy", phase)
elif state == "done":
# Non-streaming: the 'done' message carries the final text. Streaming
# already showed it live, so don't overwrite with "Streaming stopped".
if not self._ov_streaming and message:
ov.set_text(message)
self._ov_state = state
ov.set_state("done", message)
elif state == "error":
self._ov_state = state
ov.set_state("error", message)
elif state == "idle":
# Only when a session was actually live — and never clip a 'done'
# linger (idle is emitted right after done in the worker's finally).
if self._ov_state in ("recording", "streaming", "busy"):
ov.set_state("idle", message)
self._ov_state = "idle"
# -- status (marshalled to GTK thread) ------------------------------------
def _status_cb(self, state: str, workflow: str | None, message: str) -> None:
GLib.idle_add(self._apply_status, state, workflow, message)
@ -257,6 +340,8 @@ class App:
self._set_dot(colors.get(state, "#7b818b"))
self.status_lbl.set_text(labels.get(state, message))
self._overlay_status(state, message)
if self.tray is not None:
self.tray.update_status(state, labels.get(state, message))
@ -291,9 +376,18 @@ class App:
# -- panel / settings / lifecycle -----------------------------------------
def open_settings(self) -> None:
# Single instance: if Settings is already open, bring it to the front
# instead of spawning a second dialog (the tray menu bypasses the
# dialog's own modality, so it could otherwise be opened repeatedly).
existing = getattr(self, "_settings", None)
if existing is not None:
existing.dlg.present()
return
from .gtksettings import SettingsDialog
SettingsDialog(self.win, self.cfg, daemon=self.daemon).run_dialog()
self._settings = SettingsDialog(self.win, self.cfg, daemon=self.daemon)
self._settings.dlg.connect("destroy", lambda *_: setattr(self, "_settings", None))
self._settings.run_dialog()
def show_panel(self) -> None:
self.win.show_all()
@ -312,6 +406,11 @@ class App:
def quit_all(self) -> None:
try:
self.daemon.stop_input()
if self.overlay is not None:
self.overlay.destroy()
from . import caret
caret.stop_tracking()
finally:
Gtk.main_quit()
@ -328,6 +427,21 @@ class App:
def run_gui(tray_mode: bool = False) -> int:
# Use GIO's native /proc/mounts volume monitor instead of the gvfs/udisks2
# one. On headless or minimal desktops the `org.gtk.vfs.UDisks2VolumeMonitor`
# dbus service often fails to activate, and every Gtk.FileChooserButton then
# blocks ~25s on a StartServiceByName timeout while realizing — which freezes
# the settings dialog (and, via the stalled main loop, the panel) so neither
# ever appears. The unix monitor needs no dbus and opens choosers instantly.
os.environ.setdefault("GIO_USE_VOLUME_MONITOR", "unix")
# Identify to the window manager as "blitztext" rather than the Python entry
# point's filename. Launched via `python -m blitztext`, GTK's default program
# name is argv[0]'s basename ("__main__.py"), which is what shows in the
# taskbar and in GNOME's "… is not responding" dialog. Setting it here (before
# any window is realized) gives every window the app's real name + .desktop
# match, without touching the `-m blitztext` entry point.
GLib.set_prgname("blitztext")
GLib.set_application_name("Blitztext")
cfg = load()
App(cfg, tray_mode=tray_mode).run()
return 0

View File

@ -74,18 +74,21 @@ class ModifierScheme:
return
self._pressed.add(token)
# Cancel works while arming/armed AND when the daemon is recording via
# wakeword (state stays "idle" in the scheme because wakeword bypasses
# the key-press path entirely).
if self._is(token, self.cancel) and (
self._state != "idle" or self.daemon.is_recording):
self._state = "idle"
self.daemon.cancel_dictation()
return
if self._state == "idle":
if self.start.issubset(self._pressed):
self._state = "arming"
self.daemon.start_dictation()
return
# Cancel works while arming or armed.
if self._is(token, self.cancel):
self._state = "idle"
self.daemon.cancel_dictation()
return
if self._state == "armed":
if self._is(token, self.send):
self._state = "idle"

View File

@ -13,6 +13,7 @@ import os
import urllib.error
import urllib.request
from dataclasses import dataclass
from typing import Callable
from .stt import reachable
@ -48,10 +49,19 @@ def chat(
model: str | None = None,
temperature: float | None = None,
timeout: int = 45,
on_token: Callable[[str], None] | None = None,
abort_event=None,
) -> str:
"""Run a chat completion and return the full text.
When ``on_token`` is given, the request is streamed and each content delta is
handed to the callback as it arrives (so a UI can show the model writing in
real time). The callback is best-effort it never affects the return value,
which is always the complete, stripped response.
"""
stream = on_token is not None
api_key = engine.api_key
payload = json.dumps(
{
body_obj = {
"model": model or engine.model,
"temperature": engine.temperature if temperature is None else temperature,
"messages": [
@ -59,7 +69,9 @@ def chat(
{"role": "user", "content": user_text},
],
}
).encode("utf-8")
if stream:
body_obj["stream"] = True
payload = json.dumps(body_obj).encode("utf-8")
headers = {"Content-Type": "application/json"}
if api_key:
@ -70,19 +82,55 @@ def chat(
)
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
if stream:
content = _read_stream(resp, on_token, abort_event)
else:
body = json.loads(resp.read().decode("utf-8"))
content = body["choices"][0]["message"]["content"]
except urllib.error.HTTPError as exc:
detail = exc.read().decode("utf-8", "replace")[:300]
raise LLMError(f"HTTP {exc.code}: {detail}") from exc
except urllib.error.URLError as exc:
raise LLMError(f"Connection failed: {exc.reason}") from exc
try:
content = body["choices"][0]["message"]["content"]
except (KeyError, IndexError, TypeError) as exc:
raise LLMError(f"Unexpected response: {str(body)[:300]}") from exc
raise LLMError(f"Unexpected response: {exc}") from exc
except (TimeoutError, OSError) as exc:
# socket.timeout (subclass of OSError / TimeoutError) fires when the server
# stops sending data mid-stream. Not wrapped in URLError — must be caught
# separately or it would propagate uncaught and kill the background thread.
raise LLMError(f"Request timed out or connection lost: {exc}") from exc
except Exception as exc: # noqa: BLE001
raise LLMError(f"Unexpected error: {exc}") from exc
content = (content or "").strip()
if not content:
raise LLMError("Empty response from model.")
return content
def _read_stream(resp, on_token: Callable[[str], None], abort_event=None) -> str:
"""Parse an OpenAI-style SSE stream, returning the accumulated content and
feeding each delta to ``on_token``. Tolerant of keep-alive blanks and the
trailing ``[DONE]`` sentinel."""
parts: list[str] = []
for raw in resp:
if abort_event and abort_event.is_set():
break
line = raw.decode("utf-8", "replace").strip()
if not line or not line.startswith("data:"):
continue
data = line[len("data:"):].strip()
if data == "[DONE]":
break
try:
obj = json.loads(data)
delta = obj["choices"][0]["delta"].get("content")
except (ValueError, KeyError, IndexError, TypeError):
continue
if delta:
parts.append(delta)
try:
on_token(delta)
except Exception: # noqa: BLE001 - UI hiccups must not break delivery
pass
return "".join(parts)

View File

@ -3,6 +3,9 @@
Our own messages go through log(); library logs (faster-whisper, huggingface_hub)
are captured via a logging handler so model download/load progress is visible
instead of an opaque "Loading…".
Each entry stores (timestamp_str, level, message) so the UI can filter by level.
Levels: DEBUG, INFO, WARNING, ERROR (default: INFO)
"""
from __future__ import annotations
@ -13,32 +16,53 @@ import threading
import time
from collections import deque
_LINES: deque[str] = deque(maxlen=2000)
# Each entry: (time_str, level, message)
_ENTRIES: deque[tuple[str, str, str]] = deque(maxlen=2000)
_LOCK = threading.Lock()
_LEVEL_ORDER = {"DEBUG": 0, "INFO": 1, "WARNING": 2, "ERROR": 3}
def log(msg: str, *, echo: bool = True) -> None:
line = f"{time.strftime('%H:%M:%S')} {msg}"
def log(msg: str, *, echo: bool = True, level: str = "INFO") -> None:
level = level.upper()
ts = time.strftime("%H:%M:%S")
with _LOCK:
_LINES.append(line)
_ENTRIES.append((ts, level, msg))
if echo:
print(line, file=sys.stderr, flush=True)
print(f"{ts} [{level}] {msg}", file=sys.stderr, flush=True)
def lines() -> list[str]:
def lines(min_level: str = "DEBUG") -> list[str]:
"""Return formatted lines at or above min_level."""
threshold = _LEVEL_ORDER.get(min_level.upper(), 0)
with _LOCK:
return list(_LINES)
entries = list(_ENTRIES)
result = []
for ts, lvl, msg in entries:
if _LEVEL_ORDER.get(lvl, 1) >= threshold:
prefix = f"[{lvl}] " if lvl not in ("INFO",) else ""
result.append(f"{ts} {prefix}{msg}")
return result
def clear() -> None:
with _LOCK:
_LINES.clear()
_ENTRIES.clear()
class _BufferHandler(logging.Handler):
_PY_TO_LEVEL = {
logging.DEBUG: "DEBUG",
logging.INFO: "INFO",
logging.WARNING: "WARNING",
logging.ERROR: "ERROR",
logging.CRITICAL:"ERROR",
}
def emit(self, record: logging.LogRecord) -> None:
try:
log(self.format(record), echo=False)
lvl = self._PY_TO_LEVEL.get(record.levelno, "INFO")
log(self.format(record), echo=False, level=lvl)
except Exception: # noqa: BLE001 - never let logging break the app
pass
@ -57,4 +81,4 @@ def install_logging() -> None:
for name in ("faster_whisper", "huggingface_hub", "blitztext"):
lg = logging.getLogger(name)
lg.addHandler(handler)
lg.setLevel(logging.INFO)
lg.setLevel(logging.DEBUG)

602
linux/blitztext/overlay.py Normal file
View File

@ -0,0 +1,602 @@
"""On-screen dictation HUD: a translucent bubble at the cursor.
Shows, while you dictate:
a microphone glyph that pulses red as it listens,
a live waveform driven by the real mic level,
the recognised text (word-by-word in streaming mode, or the final result as
a brief confirmation in record-then-transcribe mode),
with a little tail whose tip points at the cursor where the text will land
(see :mod:`blitztext.caret` for how that anchor is resolved).
It is a click-through, focus-free override-redirect window so it never steals
input from the field you're dictating into. All public methods are safe to call
from worker threads they marshal onto the GTK main loop via ``GLib.idle_add``.
Pure feedback: nothing here touches recording, transcription, or delivery, and
the whole feature is gated by ``cfg.overlay_enabled`` in the caller.
"""
from __future__ import annotations
import math
import time
from collections import deque
import cairo
import gi
gi.require_version("Gtk", "3.0")
gi.require_version("Gdk", "3.0")
gi.require_version("PangoCairo", "1.0")
from gi.repository import Gdk, GLib, Gtk, Pango, PangoCairo # noqa: E402
from . import caret # noqa: E402
# Layout constants (logical px).
_WIDTH = 360
_PAD = 16
_HEADER_H = 40 # mic + waveform row
_RADIUS = 16
_TAIL_W = 20
_TAIL_H = 11
_GAP = 12 # clearance between the tail tip and the anchor
_BARS = 30 # waveform bar count
_PRESET_H = 22 # matched-preset header row (emoji + name + keyword)
_MIN_TEXT_H = 0
_MAX_TEXT_H = 120
_FPS_MS = 33 # ~30 fps animation tick
# Phase → (mic colour, label). Recording/streaming pulse; others are steady.
_PHASES = {
"recording": ((1.0, 0.27, 0.23), "Listening…"),
"streaming": ((1.0, 0.27, 0.23), "Listening…"),
"busy": ((1.0, 0.74, 0.16), "Transcribing…"),
"done": ((0.30, 0.80, 0.36), ""),
"error": ((1.0, 0.35, 0.35), "Error"),
}
_CANCEL_BTN_R = 11 # hit-radius of the × button (px)
class Overlay:
def __init__(self, anchor_mode: str = "caret", on_cancel=None) -> None:
self.anchor_mode = anchor_mode
self._on_cancel_cb = on_cancel # callable() → cancels current recording
self._visible = False
self._state = "recording"
self._text = ""
self._phase_label = "Listening…"
# Matched-preset banner (fused in from voice routing instead of a separate
# desktop notification): the preset's emoji, its name, and the spoken
# keyword that selected it.
self._preset_icon = ""
self._preset_name = ""
self._keyword = ""
self._anchor: caret.Anchor | None = None
self._tail_up = False # tail on top edge (bubble below anchor)?
self._tail_x = _WIDTH // 2 # tail tip, window-local x
self._height = _HEADER_H + 2 * _PAD + _TAIL_H
self._levels: deque[float] = deque([0.0] * _BARS, maxlen=_BARS)
self._disp = [0.0] * _BARS # eased bar heights for smooth motion
self._pulse = 0.0
# Silence auto-stop countdown ring (wraps the mic): a deadline the ring
# drains toward, the window it spans, and an eased opacity so it fades in
# when you fall quiet and out the moment you speak again.
self._cd_deadline: float | None = None
self._cd_total = 1.0
self._cd_frac = 0.0
self._cd_alpha = 0.0
self._tick_id: int | None = None
self._hide_id: int | None = None
self._t0 = time.time()
# Coalescing text updates: background LLM streaming can fire dozens of
# set_text() calls per second. We buffer the latest text and only ever
# have ONE idle_add pending — so the GTK main loop is never flooded.
self._pending_text: str = ""
self._text_flush_queued: bool = False
self._pending_level: float = 0.0
self._level_flush_queued: bool = False
self._cancel_btn_rect = (0, 0, 0, 0) # (x, y, w, h) in window coords
self._win = Gtk.Window(type=Gtk.WindowType.POPUP)
self._win.set_app_paintable(True)
self._win.set_resizable(False)
self._win.set_skip_taskbar_hint(True)
self._win.set_skip_pager_hint(True)
self._win.set_accept_focus(False)
self._win.set_focus_on_map(False)
self._win.set_keep_above(True)
screen = self._win.get_screen()
visual = screen.get_rgba_visual() if screen else None
if visual is not None:
self._win.set_visual(visual)
self._area = Gtk.DrawingArea()
if visual is not None:
self._area.set_visual(visual)
self._area.connect("draw", self._on_draw)
self._win.add(self._area)
self._win.connect("realize", self._on_realize)
self._win.add_events(Gdk.EventMask.BUTTON_PRESS_MASK)
self._win.connect("button-press-event", self._on_click)
self._win.set_default_size(_WIDTH, self._height)
# -- click-through (all except the × button) ------------------------------
def _on_realize(self, _w) -> None:
self._update_input_region()
def _update_input_region(self) -> None:
gdkwin = self._win.get_window()
if gdkwin is None:
return
if self._on_cancel_cb and self._state in ("recording", "streaming", "busy"):
x, y, w, h = self._cancel_btn_rect
r = cairo.Region(cairo.RectangleInt(int(x), int(y), int(w), int(h)))
else:
r = cairo.Region() # empty → fully click-through
gdkwin.input_shape_combine_region(r, 0, 0)
def _on_click(self, _win, event) -> bool:
if self._on_cancel_cb is None:
return False
x, y, w, h = self._cancel_btn_rect
if x <= event.x <= x + w and y <= event.y <= y + h:
self._on_cancel_cb()
return True
# -- thread-safe public API ----------------------------------------------
def show(self, state: str, window_id: str | None) -> None:
GLib.idle_add(self._show, state, window_id)
def set_level(self, level: float) -> None:
self._pending_level = float(level)
if not self._level_flush_queued:
self._level_flush_queued = True
GLib.idle_add(self._flush_level)
def _flush_level(self) -> bool:
self._level_flush_queued = False
self._set_level(self._pending_level)
return False
def set_text(self, text: str) -> None:
self._pending_text = text or ""
# Only schedule a flush if none is already queued; this collapses a burst
# of token callbacks (e.g. 50/s from LLM streaming) into a single GTK
# redraw, preventing main-loop flooding and session freezes.
if not self._text_flush_queued:
self._text_flush_queued = True
GLib.idle_add(self._flush_text)
def _flush_text(self) -> bool:
self._text_flush_queued = False
self._set_text(self._pending_text)
return False
def set_preset(self, icon: str, name: str, keyword: str | None) -> None:
"""Show the matched voice-routing preset on the overlay (emoji + name +
the spoken keyword), in place of a separate desktop notification."""
GLib.idle_add(self._set_preset, icon or "", name or "", keyword or "")
def set_countdown(self, remaining: float | None, total: float) -> None:
"""Silence auto-stop progress: ``remaining`` seconds until it fires over
a ``total``-second window, or ``None`` while you're still speaking."""
GLib.idle_add(self._set_countdown, remaining, total)
def set_state(self, state: str, message: str = "") -> None:
GLib.idle_add(self._set_state, state, message)
def hide(self) -> None:
GLib.idle_add(self._hide)
def destroy(self) -> None:
GLib.idle_add(self._destroy)
# -- main-thread handlers -------------------------------------------------
def _show(self, state: str, window_id: str | None) -> bool:
self._cancel_hide()
self._state = state
self._phase_label = _PHASES.get(state, ((1, 1, 1), ""))[1]
self._text = ""
self._preset_icon = ""
self._preset_name = ""
self._keyword = ""
self._levels = deque([0.0] * _BARS, maxlen=_BARS)
self._disp = [0.0] * _BARS
self._cd_deadline = None
self._cd_frac = 0.0
self._cd_alpha = 0.0
self._anchor = caret.resolve(self.anchor_mode, window_id)
self._relayout()
self._visible = True
self._win.show_all()
if self._tick_id is None:
self._tick_id = GLib.timeout_add(_FPS_MS, self._tick)
return False
def _set_level(self, level: float) -> bool:
self._levels.append(max(0.0, min(1.0, level)))
return False
def _set_text(self, text: str) -> bool:
if text == self._text:
return False
self._text = text
self._relayout()
self._area.queue_draw()
return False
def _set_preset(self, icon: str, name: str, keyword: str) -> bool:
if (icon, name, keyword) == (self._preset_icon, self._preset_name, self._keyword):
return False
self._preset_icon = icon
self._preset_name = name
self._keyword = keyword
self._relayout()
self._area.queue_draw()
return False
def _set_countdown(self, remaining: float | None, total: float) -> bool:
if remaining is None:
self._cd_deadline = None
else:
self._cd_total = max(0.1, total)
self._cd_deadline = time.time() + max(0.0, remaining)
return False
def _set_state(self, state: str, message: str) -> bool:
self._state = state
if state == "busy":
# Honour a clean phase word from the caller ("Transcribing…",
# "Rewriting…") so the overlay narrates what's happening; fall back to
# the canned label otherwise.
self._phase_label = (message.strip() or _PHASES["busy"][1])[:28]
else:
self._phase_label = _PHASES.get(state, ((1, 1, 1), message[:40]))[1] or message[:40]
if state not in ("recording", "streaming"):
# The countdown only makes sense while listening; drop it as soon as
# we move on to transcribing / done / idle so the ring doesn't linger.
self._cd_deadline = None
self._update_input_region() # enable/disable × hit area
if state in ("recording", "streaming", "busy"):
self._cancel_hide()
elif state in ("done", "idle", "error"):
# Linger on the final text, then fade out. Errors stay a touch longer.
delay = 2500 if state == "error" else (1600 if self._text else 700)
self._schedule_hide(delay)
self._area.queue_draw()
return False
def _hide(self) -> bool:
self._visible = False
self._cancel_hide()
if self._tick_id is not None:
GLib.source_remove(self._tick_id)
self._tick_id = None
self._win.hide()
return False
def _destroy(self) -> bool:
self._hide()
self._win.destroy()
return False
# -- hide scheduling ------------------------------------------------------
def _schedule_hide(self, delay_ms: int) -> None:
self._cancel_hide()
self._hide_id = GLib.timeout_add(delay_ms, self._hide)
def _cancel_hide(self) -> None:
if self._hide_id is not None:
GLib.source_remove(self._hide_id)
self._hide_id = None
# -- animation ------------------------------------------------------------
def _tick(self) -> bool:
if not self._visible:
self._tick_id = None
return False
# Ease displayed bars toward the latest levels for fluid motion even
# though real levels arrive at ~10 Hz.
targets = list(self._levels)
for i, t in enumerate(targets):
self._disp[i] += (t - self._disp[i]) * 0.35
self._pulse = (math.sin((time.time() - self._t0) * 5.0) + 1.0) * 0.5
# Drain the silence ring against its own clock so it stays smooth between
# the ~10 Hz level samples. Fade in gently (so word gaps don't flash it)
# and out a touch faster.
if self._cd_deadline is not None:
self._cd_frac = max(0.0, min(1.0, (self._cd_deadline - time.time()) / self._cd_total))
self._cd_alpha += (1.0 - self._cd_alpha) * 0.15
else:
self._cd_alpha += (0.0 - self._cd_alpha) * 0.30
self._area.queue_draw()
return True
# -- geometry -------------------------------------------------------------
def _monitor_geo(self):
disp = Gdk.Display.get_default()
if self._anchor is not None:
mon = disp.get_monitor_at_point(self._anchor.x, self._anchor.y)
else:
mon = disp.get_primary_monitor() or disp.get_monitor(0)
return mon.get_geometry()
def _text_height(self) -> int:
if not self._text:
return _MIN_TEXT_H
layout = self._win.create_pango_layout(self._text)
layout.set_width((_WIDTH - 2 * _PAD) * Pango.SCALE)
layout.set_wrap(Pango.WrapMode.WORD_CHAR)
font = Pango.FontDescription("Sans 11")
layout.set_font_description(font)
_w, h = layout.get_pixel_size()
return min(_MAX_TEXT_H, max(_MIN_TEXT_H, h))
def _relayout(self) -> None:
text_h = self._text_height()
preset_h = _PRESET_H if self._preset_name else 0
gap_preset = 8 if preset_h else 0
gap_text = 8 if text_h else 0
body_h = _HEADER_H + gap_preset + preset_h + gap_text + text_h + 2 * _PAD
self._height = body_h + _TAIL_H
geo = self._monitor_geo()
a = self._anchor
ax = a.x if a else geo.x + geo.width // 2
ay = a.y if a else geo.y + geo.height - 140
ah = a.height if a else 0
# Prefer placing the bubble above the anchor; flip below if it won't fit.
above_y = ay - _GAP - self._height
if above_y >= geo.y + 4:
self._tail_up = False
win_y = above_y
else:
self._tail_up = True
win_y = ay + ah + _GAP
tip_x = max(geo.x + _RADIUS + _TAIL_W, min(ax, geo.x + geo.width - _RADIUS - _TAIL_W))
win_x = tip_x - _WIDTH // 2
win_x = max(geo.x + 6, min(win_x, geo.x + geo.width - _WIDTH - 6))
self._tail_x = tip_x - win_x
self._win.resize(_WIDTH, self._height)
self._win.move(int(win_x), int(win_y))
self._area.set_size_request(_WIDTH, self._height)
# X button: top-right corner of the bubble body.
_body_top = _TAIL_H if self._tail_up else 0
btn_d = _CANCEL_BTN_R * 2
self._cancel_btn_rect = (
_WIDTH - _PAD - btn_d, _body_top + _PAD // 2, btn_d, btn_d)
self._update_input_region()
# -- drawing --------------------------------------------------------------
def _on_draw(self, _area, cr) -> bool:
# Start fully transparent.
cr.set_operator(cairo.OPERATOR_SOURCE)
cr.set_source_rgba(0, 0, 0, 0)
cr.paint()
cr.set_operator(cairo.OPERATOR_OVER)
w = _WIDTH
body_top = _TAIL_H if self._tail_up else 0
body_h = self._height - _TAIL_H
body_bottom = body_top + body_h
# Bubble + tail as one path, so the fill/stroke wrap the tail cleanly.
self._bubble_path(cr, 0, body_top, w, body_h)
cr.set_source_rgba(0.10, 0.11, 0.14, 0.94)
cr.fill_preserve()
cr.set_source_rgba(1, 1, 1, 0.08)
cr.set_line_width(1.0)
cr.stroke()
cx = _PAD + 14
cy = body_top + _PAD + 12
self._draw_mic(cr, cx, cy)
self._draw_countdown(cr, cx, cy)
# Waveform fills the space right of the mic across the header row.
wf_x = cx + 26
wf_w = w - _PAD - wf_x
self._draw_wave(cr, wf_x, body_top + _PAD, wf_w, _HEADER_H)
# × cancel button (top-right corner, recording/streaming/busy).
if self._on_cancel_cb and self._state in ("recording", "streaming", "busy"):
self._draw_cancel_btn(cr, body_top)
# Phase label by the waveform — only when there's no preset banner and no text.
# Shift left to leave room for the × button.
label_right = (w - _PAD - _CANCEL_BTN_R * 2 - 6
if self._on_cancel_cb and self._state in ("recording", "streaming", "busy")
else w - _PAD)
if self._phase_label and not self._text and not self._preset_name:
self._draw_label(cr, label_right, body_top + _PAD + 12, self._phase_label)
y = body_top + _PAD + _HEADER_H
# Matched-preset banner: emoji + name (left), live phase chip (right).
if self._preset_name:
y += 8
self._draw_preset(cr, _PAD, y, w - 2 * _PAD)
y += _PRESET_H
# Recognised / rewritten text below.
if self._text:
y += 8
self._draw_text(cr, _PAD, y, w - 2 * _PAD)
return False
def _draw_preset(self, cr, x, y, w) -> None:
"""The matched voice-routing preset, fused onto the overlay in place of a
desktop notification: emoji + name (and the spoken keyword) on the left,
the current phase ("Transcribing…", "Rewriting…") on the right."""
cy = y + _PRESET_H / 2
cursor = x
if self._preset_icon:
ic = self._win.create_pango_layout(self._preset_icon)
ic.set_font_description(Pango.FontDescription("Sans 13"))
iw, ih = ic.get_pixel_size()
cr.set_source_rgba(1, 1, 1, 0.95)
cr.move_to(cursor, cy - ih / 2)
PangoCairo.show_layout(cr, ic)
cursor += iw + 7
name = GLib.markup_escape_text(self._preset_name)
if self._keyword:
kw = GLib.markup_escape_text(self._keyword)
markup = f'<b>{name}</b> <span alpha="55%">“{kw}”</span>'
else:
markup = f"<b>{name}</b>"
layout = self._win.create_pango_layout("")
layout.set_markup(markup, -1)
layout.set_font_description(Pango.FontDescription("Sans 10"))
layout.set_ellipsize(Pango.EllipsizeMode.END)
avail = (x + w) - cursor - 96 # leave room for the phase chip
layout.set_width(max(40, avail) * Pango.SCALE)
_nw, nh = layout.get_pixel_size()
cr.set_source_rgba(0.95, 0.96, 0.99, 0.98)
cr.move_to(cursor, cy - nh / 2)
PangoCairo.show_layout(cr, layout)
if self._phase_label:
self._draw_label(cr, x + w, cy, self._phase_label)
def _bubble_path(self, cr, x, y, w, h) -> None:
r = _RADIUS
cr.new_sub_path()
cr.arc(x + w - r, y + r, r, -math.pi / 2, 0)
cr.arc(x + w - r, y + h - r, r, 0, math.pi / 2)
cr.arc(x + r, y + h - r, r, math.pi / 2, math.pi)
cr.arc(x + r, y + r, r, math.pi, 1.5 * math.pi)
cr.close_path()
# Tail: a small triangle on the top or bottom edge at self._tail_x.
tx = max(r + _TAIL_W, min(self._tail_x, w - r - _TAIL_W))
if self._tail_up:
cr.move_to(tx - _TAIL_W / 2, y)
cr.line_to(tx, y - _TAIL_H)
cr.line_to(tx + _TAIL_W / 2, y)
else:
cr.move_to(tx - _TAIL_W / 2, y + h)
cr.line_to(tx, y + h + _TAIL_H)
cr.line_to(tx + _TAIL_W / 2, y + h)
cr.close_path()
def _draw_mic(self, cr, cx, cy) -> None:
colour, _ = _PHASES.get(self._state, ((1, 1, 1), ""))
pulsing = self._state in ("recording", "streaming")
# Soft pulsing halo while listening — but yield to the countdown ring as
# it takes over (you've gone quiet, so the pulse fades out under it).
if pulsing:
rad = 12 + self._pulse * 6
cr.set_source_rgba(*colour, 0.18 * (1 - self._pulse * 0.6) * (1 - self._cd_alpha))
cr.arc(cx, cy, rad, 0, 2 * math.pi)
cr.fill()
cr.set_source_rgba(*colour, 1.0)
cr.set_line_width(2.0)
# Capsule head.
head_w, head_top, head_bot = 9.0, cy - 11, cy + 1
cr.arc(cx, head_top + head_w / 2, head_w / 2, math.pi, 2 * math.pi)
cr.arc(cx, head_bot - head_w / 2, head_w / 2, 0, math.pi)
cr.close_path()
cr.fill()
# Stand arc + post + base.
cr.set_source_rgba(*colour, 0.95)
cr.arc(cx, cy, 8, math.radians(25), math.radians(155))
cr.stroke()
cr.move_to(cx, cy + 8)
cr.line_to(cx, cy + 12)
cr.stroke()
cr.move_to(cx - 5, cy + 12)
cr.line_to(cx + 5, cy + 12)
cr.stroke()
def _draw_countdown(self, cr, cx, cy) -> None:
"""Silence auto-stop ring wrapping the mic: a full circle that drains
clockwise as the trailing-silence timer runs out, recolouring from calm
cyan to an urgent red just before it fires."""
a = self._cd_alpha
if a <= 0.01:
return
r, lw = 16.0, 2.6
cr.set_line_width(lw)
# Faint full track so the drained part of the ring stays legible.
cr.set_source_rgba(1, 1, 1, 0.10 * a)
cr.arc(cx, cy, r, 0, 2 * math.pi)
cr.stroke()
frac = self._cd_frac
if frac <= 0.0:
return
spent = 1.0 - frac
red = 0.30 + spent * 0.70
grn = 0.80 - spent * 0.45
blu = 0.90 - spent * 0.60
cr.set_source_rgba(red, grn, blu, 0.95 * a)
cr.set_line_cap(cairo.LINE_CAP_ROUND)
start = -math.pi / 2 # 12 o'clock
cr.arc(cx, cy, r, start, start + frac * 2 * math.pi)
cr.stroke()
cr.set_line_cap(cairo.LINE_CAP_BUTT)
def _draw_wave(self, cr, x, y, w, h) -> None:
gap = 2.0
bw = max(1.5, (w - gap * (_BARS - 1)) / _BARS)
mid = y + h / 2
listening = self._state in ("recording", "streaming")
for i in range(_BARS):
v = self._disp[i] if i < len(self._disp) else 0.0
# Idle baseline shimmer so the meter never looks frozen.
if listening and v < 0.04:
v = 0.04 + 0.03 * math.sin((self._t0 - time.time()) * 4 + i * 0.5)
bh = max(2.0, v * (h - 2))
bx = x + i * (bw + gap)
alpha = 0.85 if listening else 0.4
cr.set_source_rgba(0.42, 0.62, 1.0, alpha)
self._round_rect(cr, bx, mid - bh / 2, bw, bh, min(bw / 2, 2))
cr.fill()
def _draw_label(self, cr, right_x, cy, text) -> None:
layout = self._win.create_pango_layout(text)
layout.set_font_description(Pango.FontDescription("Sans 9"))
tw, th = layout.get_pixel_size()
cr.set_source_rgba(1, 1, 1, 0.55)
cr.move_to(right_x - tw, cy - th / 2)
PangoCairo.show_layout(cr, layout)
def _draw_cancel_btn(self, cr, body_top) -> None:
x, y, w, h = self._cancel_btn_rect
cx_btn = x + w / 2
cy_btn = y + h / 2
cr.set_source_rgba(1, 1, 1, 0.15)
cr.arc(cx_btn, cy_btn, _CANCEL_BTN_R, 0, 2 * math.pi)
cr.fill()
arm = _CANCEL_BTN_R * 0.45
cr.set_source_rgba(1, 1, 1, 0.80)
cr.set_line_width(1.8)
cr.set_line_cap(cairo.LINE_CAP_ROUND)
cr.move_to(cx_btn - arm, cy_btn - arm)
cr.line_to(cx_btn + arm, cy_btn + arm)
cr.stroke()
cr.move_to(cx_btn + arm, cy_btn - arm)
cr.line_to(cx_btn - arm, cy_btn + arm)
cr.stroke()
cr.set_line_cap(cairo.LINE_CAP_BUTT)
def _draw_text(self, cr, x, y, w) -> None:
layout = self._win.create_pango_layout(self._text)
layout.set_width(w * Pango.SCALE)
layout.set_wrap(Pango.WrapMode.WORD_CHAR)
layout.set_ellipsize(Pango.EllipsizeMode.END)
layout.set_height(_MAX_TEXT_H * Pango.SCALE)
layout.set_font_description(Pango.FontDescription("Sans 11"))
cr.set_source_rgba(0.95, 0.96, 0.99, 0.97)
cr.move_to(x, y)
PangoCairo.show_layout(cr, layout)
@staticmethod
def _round_rect(cr, x, y, w, h, r) -> None:
r = min(r, w / 2, h / 2)
cr.new_sub_path()
cr.arc(x + w - r, y + r, r, -math.pi / 2, 0)
cr.arc(x + w - r, y + h - r, r, 0, math.pi / 2)
cr.arc(x + r, y + h - r, r, math.pi / 2, math.pi)
cr.arc(x + r, y + r, r, math.pi, 1.5 * math.pi)
cr.close_path()

View File

@ -41,6 +41,13 @@ def _focus(window_id: str | None) -> None:
time.sleep(0.05)
# Above this character count, or when the text contains newlines, xdotool type
# sends thousands of synchronous X11 round-trips and can flood the X11 server's
# per-client event buffer until the whole session freezes. Auto-upgrade to a
# single clipboard paste instead, which is instantaneous.
_TYPE_THRESHOLD = 300
def deliver(text: str, *, mode: str = "type", window_id: str | None = None, type_delay_ms: int = 4) -> None:
if not text:
return
@ -55,6 +62,15 @@ def deliver(text: str, *, mode: str = "type", window_id: str | None = None, type
# Give the user time to release the hotkey modifiers before we synthesize input.
time.sleep(0.12)
# Long or multi-line text: force clipboard paste regardless of configured mode.
# xdotool type at 12ms/char for a 15 000-char code block takes ~3 minutes and
# sends so many synchronous X11 events that the server's per-client buffer
# overflows, freezing the entire X11 session.
if mode == "type" and (len(text) > _TYPE_THRESHOLD or "\n" in text):
if _set_clipboard(text):
mode = "paste"
# If clipboard isn't available we fall through to xdotool type as before.
if mode == "paste" and _set_clipboard(text):
if wayland:
if shutil.which("wtype"):

View File

@ -71,3 +71,23 @@ def clean(text: str, *, strip_trailing_punctuation: bool = False) -> str:
if strip_trailing_punctuation:
text = text.rstrip(" .,!?;:")
return text
# Spoken phrases that should become newlines/paragraphs in the typed output.
# Matched case-insensitively, whole-phrase (surrounded by word boundaries or
# start/end of string). German and English variants are both covered.
_NEWLINE_PHRASES: list[tuple[re.Pattern[str], str]] = [
# Paragraph break (blank line) — must come before single-newline patterns
(re.compile(r"(?i)\b(new paragraph|neuer absatz|neues absatz|absatz)\b"), "\n\n"),
# Single line break
(re.compile(r"(?i)\b(new line|neue zeile|zeilenumbruch|line break)\b"), "\n"),
]
def expand_spoken_punctuation(text: str) -> str:
"""Replace spoken newline/paragraph commands with actual control characters."""
for pattern, replacement in _NEWLINE_PHRASES:
text = pattern.sub(replacement, text)
# Collapse leading/trailing whitespace per line but preserve intentional newlines
lines = [ln.strip() for ln in text.split("\n")]
return "\n".join(lines).strip()

View File

@ -85,7 +85,13 @@ def route(
best = None # (score, span_len, position, preset_name, keyword)
for preset in presets:
for keyword in getattr(preset, "keywords", None) or []:
# The preset's own name is always an implicit trigger, so a preset is
# speakable by name even when it has no keywords configured.
candidates = list(getattr(preset, "keywords", None) or [])
name = getattr(preset, "name", "")
if name:
candidates.append(name)
for keyword in candidates:
kw_tokens = normalize(keyword)
m = _match_window(tokens, kw_tokens, threshold)
if m is None:
@ -104,6 +110,55 @@ def route(
return RouteResult(name, cleaned, keyword, position, score)
def is_cancel(transcript: str, cancel_keywords, *, threshold: float = DEFAULT_THRESHOLD) -> str | None:
"""Return the cancel keyword that matches an edge of the transcript, else None.
Lets a spoken word like "abbrechen" abort an (often accidentally triggered)
dictation before it is routed, rewritten, or delivered. Matched the same
edge-anchored, ASR-tolerant way as routing keywords, so the word appearing
deep inside a sentence won't trigger it — only at the start or end.
"""
if not cancel_keywords:
return None
tokens = normalize(transcript)
if not tokens:
return None
for kw in cancel_keywords:
kw_tokens = normalize(kw)
if kw_tokens and _match_window(tokens, kw_tokens, threshold) is not None:
return kw
return None
def match_send(transcript: str, send_keywords, *, threshold: float = DEFAULT_THRESHOLD):
"""Detect a spoken 'send' keyword at an edge; return (keyword, cleaned_text).
Like is_cancel, but the keyword is *stripped* and the remaining text is meant
to be delivered and submitted with Enter the spoken equivalent of
stop+paste+Enter. Returns (None, transcript) when nothing matches. Matched the
same edge-anchored, ASR-tolerant way as routing keywords, so the word deep
inside a sentence won't trigger it — only at the start or end.
"""
if not send_keywords:
return None, transcript
tokens = normalize(transcript)
if not tokens:
return None, transcript
best = None # (score, span, position, keyword)
for kw in send_keywords:
kw_tokens = normalize(kw)
m = _match_window(tokens, kw_tokens, threshold)
if m is None:
continue
position, score, span = m
if best is None or (score, span) > (best[0], best[1]):
best = (score, span, position, kw)
if best is None:
return None, transcript
_score, span, position, kw = best
return kw, _strip_span(transcript, span, position)
def _strip_span(transcript: str, span_words: int, position: str) -> str:
"""Remove the matched keyword from the given edge of the original transcript.

View File

@ -0,0 +1,844 @@
"""First-run setup wizard — guides new users through the essential settings.
Shows automatically when no config file exists yet (fresh install).
Can also be reopened from Settings "Setup Wizard…".
Flow:
Welcome Trigger method [Keyboard] [Wakeword] STT LLM Done
Pages in brackets are shown conditionally depending on the trigger choice.
"""
from __future__ import annotations
import threading
from pathlib import Path
import gi
gi.require_version("Gtk", "3.0")
gi.require_version("Gdk", "3.0")
from gi.repository import Gdk, GLib, Gtk, Pango # noqa: E402
# ---------------------------------------------------------------------------
# Tiny style helpers
# ---------------------------------------------------------------------------
def _h1(text: str) -> Gtk.Label:
lbl = Gtk.Label(label=text, xalign=0.0)
lbl.set_line_wrap(True)
attrs = Pango.AttrList()
attrs.insert(Pango.attr_weight_new(Pango.Weight.BOLD))
attrs.insert(Pango.attr_scale_new(1.45))
lbl.set_attributes(attrs)
return lbl
def _sub(text: str) -> Gtk.Label:
lbl = Gtk.Label(label=text, xalign=0.0)
lbl.set_line_wrap(True)
lbl.set_max_width_chars(64)
lbl.get_style_context().add_class("dim-label")
return lbl
def _section(text: str) -> Gtk.Label:
lbl = Gtk.Label(xalign=0.0)
lbl.set_markup(f"<b><small>{GLib.markup_escape_text(text.upper())}</small></b>")
lbl.set_margin_top(12)
lbl.get_style_context().add_class("dim-label")
return lbl
def _page_box() -> Gtk.Box:
box = Gtk.Box(orientation=Gtk.Orientation.VERTICAL, spacing=10)
box.set_margin_top(28); box.set_margin_bottom(8)
box.set_margin_start(32); box.set_margin_end(32)
return box
def _option_card(icon: str, title: str, desc: str) -> tuple[Gtk.RadioButton, Gtk.Box]:
"""A large selectable card with an icon, bold title, and grey description."""
card = Gtk.Box(spacing=14)
card.set_margin_top(4); card.set_margin_bottom(4)
card.get_style_context().add_class("card")
icon_lbl = Gtk.Label(label=icon)
attrs = Pango.AttrList()
attrs.insert(Pango.attr_scale_new(2.0))
icon_lbl.set_attributes(attrs)
icon_lbl.set_margin_start(14)
card.pack_start(icon_lbl, False, False, 0)
text_box = Gtk.Box(orientation=Gtk.Orientation.VERTICAL, spacing=2)
text_box.set_margin_top(12); text_box.set_margin_bottom(12)
title_lbl = Gtk.Label(label=title, xalign=0.0)
attrs2 = Pango.AttrList()
attrs2.insert(Pango.attr_weight_new(Pango.Weight.BOLD))
title_lbl.set_attributes(attrs2)
text_box.pack_start(title_lbl, False, False, 0)
desc_lbl = Gtk.Label(label=desc, xalign=0.0)
desc_lbl.get_style_context().add_class("dim-label")
desc_lbl.set_line_wrap(True)
desc_lbl.set_max_width_chars(55)
text_box.pack_start(desc_lbl, False, False, 0)
card.pack_start(text_box, True, True, 0)
# Invisible RadioButton — the whole card is the clickable area.
rb = Gtk.RadioButton()
rb.set_margin_end(14)
card.pack_start(rb, False, False, 0)
return rb, card
def _keyval_token(keyval: int) -> str | None:
from gi.repository import Gdk as _Gdk
name = _Gdk.keyval_name(keyval) or ""
low = name.lower()
for mod in ("control", "ctrl"):
if low.startswith(mod):
return "ctrl"
for mod in ("alt", "meta"):
if low.startswith(mod):
return "alt"
if low.startswith("super") or low.startswith("hyper") or low.startswith("win"):
return "cmd"
if low.startswith("shift"):
return "shift"
if low in ("escape", "esc"):
return "esc"
if low in ("return", "enter"):
return "enter"
if low == "space":
return "space"
if len(name) == 1 and name.isalpha():
return name.lower()
return None
def _format_combo(tokens: list[str]) -> str:
if not tokens:
return ""
order = ["ctrl", "alt", "shift", "cmd"]
mods = [t for t in order if t in tokens]
rest = [t for t in tokens if t not in order]
parts = mods + rest
return "+".join(f"<{p}>" for p in parts)
# ---------------------------------------------------------------------------
# Wizard
# ---------------------------------------------------------------------------
class SetupWizard:
"""Paged first-run setup dialog."""
# Page names in fixed order; "keyboard" and "wakeword" are conditional.
_ALL_PAGES = ("welcome", "trigger", "keyboard", "wakeword", "stt", "llm", "done")
def __init__(self, cfg, parent: Gtk.Window | None = None):
self.cfg = cfg
self._parent = parent
# Wizard state collected as user moves through pages.
self._trigger = "keyboard" # "keyboard" | "wakeword" | "both"
self._kb_start = cfg.key_start
self._kb_stop = cfg.key_stop
self._kb_send = cfg.key_send
self._kb_cancel = cfg.key_cancel
self._ww_uri = cfg.wakeword_uri
self._ww_model = cfg.wakeword_model
self._stt_local = True # True = faster-whisper, False = remote API
self._stt_size = "small"
self._stt_url = ""
self._stt_key = ""
self._llm_enabled = bool(cfg.base_url and cfg.base_url != "https://api.openai.com/v1")
self._llm_url = cfg.base_url
self._llm_model = cfg.rewrite_model
self._llm_key = cfg.api_key_env
# Key-binding capture state
self._bind_entry: Gtk.Entry | None = None
self._bind_pressed: list[str] = []
self._build()
# ------------------------------------------------------------------
# Dialog shell
# ------------------------------------------------------------------
def _build(self) -> None:
self.dlg = Gtk.Dialog(title="Blitztext Setup")
if self._parent:
self.dlg.set_transient_for(self._parent)
self.dlg.set_modal(True)
self.dlg.set_default_size(580, 520)
self.dlg.set_resizable(False)
self.dlg.get_action_area().hide()
# Key capture
self.dlg.add_events(Gdk.EventMask.KEY_PRESS_MASK | Gdk.EventMask.KEY_RELEASE_MASK)
self.dlg.connect("key-press-event", self._on_key_press)
self.dlg.connect("key-release-event", self._on_key_release)
content = self.dlg.get_content_area()
content.set_spacing(0)
# Stack
self.stack = Gtk.Stack()
self.stack.set_transition_type(Gtk.StackTransitionType.SLIDE_LEFT_RIGHT)
self.stack.set_transition_duration(180)
content.pack_start(self.stack, True, True, 0)
self._pg: dict[str, Gtk.Widget] = {}
builders = {
"welcome": self._build_welcome,
"trigger": self._build_trigger,
"keyboard": self._build_keyboard,
"wakeword": self._build_wakeword,
"stt": self._build_stt,
"llm": self._build_llm,
"done": self._build_done,
}
for name in self._ALL_PAGES:
w = builders[name]()
self.stack.add_named(w, name)
self._pg[name] = w
# Nav bar
sep = Gtk.Separator(orientation=Gtk.Orientation.HORIZONTAL)
content.pack_start(sep, False, False, 0)
nav = Gtk.Box(spacing=8)
nav.set_margin_top(10); nav.set_margin_bottom(10)
nav.set_margin_start(20); nav.set_margin_end(20)
self._btn_back = Gtk.Button(label="← Back")
self._btn_back.connect("clicked", lambda _b: self._go(-1))
nav.pack_start(self._btn_back, False, False, 0)
self._step_lbl = Gtk.Label()
self._step_lbl.get_style_context().add_class("dim-label")
nav.pack_start(self._step_lbl, True, True, 0)
self._btn_skip = Gtk.Button(label="Skip")
self._btn_skip.get_style_context().add_class("flat")
self._btn_skip.connect("clicked", lambda _b: self._go(+1, skip=True))
nav.pack_start(self._btn_skip, False, False, 0)
self._btn_next = Gtk.Button(label="Next →")
self._btn_next.get_style_context().add_class("suggested-action")
self._btn_next.connect("clicked", lambda _b: self._go(+1))
nav.pack_start(self._btn_next, False, False, 0)
content.pack_start(nav, False, False, 0)
self._page_order: list[str] = []
self._idx = 0
self._refresh_page_order()
self._show(0)
# ------------------------------------------------------------------
# Navigation
# ------------------------------------------------------------------
def _refresh_page_order(self) -> None:
pages = ["welcome", "trigger"]
if self._trigger in ("keyboard", "both"):
pages.append("keyboard")
if self._trigger in ("wakeword", "both"):
pages.append("wakeword")
pages += ["stt", "llm", "done"]
self._page_order = pages
def _show(self, idx: int) -> None:
self._idx = max(0, min(idx, len(self._page_order) - 1))
name = self._page_order[self._idx]
self.stack.set_visible_child_name(name)
is_first = self._idx == 0
is_last = self._idx == len(self._page_order) - 1
is_welcome = name == "welcome"
is_done = name == "done"
self._btn_back.set_sensitive(not is_first)
self._btn_skip.set_visible(not is_welcome and not is_done)
self._btn_next.set_label("Start dictating →" if is_last else "Next →")
# Step counter (exclude welcome and done from the count)
core_pages = [p for p in self._page_order if p not in ("welcome", "done")]
if name in core_pages:
step = core_pages.index(name) + 1
self._step_lbl.set_text(f"Step {step} of {len(core_pages)}")
else:
self._step_lbl.set_text("")
# When leaving "trigger", re-evaluate page order.
if name == "trigger":
self._read_trigger()
self._refresh_page_order()
def _go(self, direction: int, skip: bool = False) -> None:
name = self._page_order[self._idx]
if not skip:
self._commit_page(name)
if direction > 0 and self._idx >= len(self._page_order) - 1:
self._finish()
return
self._show(self._idx + direction)
def _finish(self) -> None:
self._apply_to_cfg()
self.dlg.response(Gtk.ResponseType.OK)
# ------------------------------------------------------------------
# Page builders
# ------------------------------------------------------------------
def _build_welcome(self) -> Gtk.Widget:
box = _page_box()
box.set_valign(Gtk.Align.CENTER)
logo = Gtk.Label(label="🎙")
attrs = Pango.AttrList()
attrs.insert(Pango.attr_scale_new(4.0))
logo.set_attributes(attrs)
logo.set_margin_bottom(12)
box.pack_start(logo, False, False, 0)
box.pack_start(_h1("Welcome to Blitztext"), False, False, 0)
box.pack_start(_sub(
"Blitztext lets you speak and have your words typed anywhere on screen — "
"with optional AI polishing.\n\n"
"This short wizard sets up the basics. "
"You can change everything later in Settings."
), False, False, 0)
return box
def _build_trigger(self) -> Gtk.Widget:
box = _page_box()
box.pack_start(_h1("How do you want to trigger recording?"), False, False, 0)
box.pack_start(_sub("You can change this later in Settings → Input."), False, False, 0)
self._rb_kb, card_kb = _option_card("", "Keyboard shortcut",
"Press a key combination to start and stop recording. "
"Great for desktop use.")
self._rb_ww, card_ww = _option_card("🎙", "Voice wakeword",
"Say a wake phrase like \"okay computer\" to start hands-free. "
"Needs a wakeword server.")
self._rb_both, card_both = _option_card("", "Both",
"Use a keyboard shortcut AND a voice wakeword — whichever is handy.")
# Group the radio buttons
self._rb_ww.join_group(self._rb_kb)
self._rb_both.join_group(self._rb_kb)
self._rb_kb.set_active(True)
for rb, card in ((self._rb_kb, card_kb), (self._rb_ww, card_ww),
(self._rb_both, card_both)):
# Make the card clickable by forwarding clicks to the radio button
ebox = Gtk.EventBox()
ebox.add(card)
ebox.connect("button-press-event",
lambda _e, _ev, r=rb: r.set_active(True))
box.pack_start(ebox, False, False, 0)
return box
def _build_keyboard(self) -> Gtk.Widget:
box = _page_box()
box.pack_start(_h1("Set up your keyboard shortcuts"), False, False, 0)
box.pack_start(_sub(
'Click "Set" and press a key combination to capture it. '
"The defaults shown here work well for most users."
), False, False, 0)
grid = Gtk.Grid(column_spacing=8, row_spacing=8)
grid.set_margin_top(16)
rows = [
("Start recording", self._kb_start, "_kb_start"),
("Stop and paste", self._kb_stop, "_kb_stop"),
("Stop, paste + Enter", self._kb_send, "_kb_send"),
("Cancel", self._kb_cancel, "_kb_cancel"),
]
self._kb_entries: dict[str, Gtk.Entry] = {}
for i, (label, value, attr) in enumerate(rows):
lbl = Gtk.Label(label=label, xalign=0.0)
lbl.set_size_request(180, -1)
grid.attach(lbl, 0, i, 1, 1)
entry = Gtk.Entry()
entry.set_text(value)
entry.set_size_request(140, -1)
entry.set_editable(False)
self._kb_entries[attr] = entry
grid.attach(entry, 1, i, 1, 1)
btn = Gtk.Button(label="Set")
btn.connect("clicked", lambda _b, e=entry: self._bind_key(e))
grid.attach(btn, 2, i, 1, 1)
box.pack_start(grid, False, False, 0)
note = Gtk.Label(xalign=0.0)
note.set_markup(
'<span size="small" alpha="75%">'
"Tip: use modifier keys (Ctrl, Alt, Win/Cmd) rather than letters "
"so shortcuts don't interfere with typing."
"</span>")
note.set_line_wrap(True)
note.set_margin_top(8)
box.pack_start(note, False, False, 0)
return box
def _build_wakeword(self) -> Gtk.Widget:
box = _page_box()
box.pack_start(_h1("Set up voice activation"), False, False, 0)
box.pack_start(_sub(
"Blitztext listens for your wake phrase through a local wakeword server. "
"The server runs on your machine — nothing is sent to the cloud."
), False, False, 0)
grid = Gtk.Grid(column_spacing=8, row_spacing=10)
grid.set_margin_top(16)
# Server URL
lbl_uri = Gtk.Label(label="Server URL", xalign=0.0)
lbl_uri.set_size_request(120, -1)
self._wiz_ww_uri = Gtk.Entry()
self._wiz_ww_uri.set_text(self._ww_uri)
self._wiz_ww_uri.set_hexpand(True)
self._wiz_ww_uri.set_placeholder_text("tcp://127.0.0.1:10400")
grid.attach(lbl_uri, 0, 0, 1, 1)
grid.attach(self._wiz_ww_uri, 1, 0, 1, 1)
# Model
lbl_model = Gtk.Label(label="Wake phrase", xalign=0.0)
self._wiz_ww_model = Gtk.Entry()
self._wiz_ww_model.set_text(self._ww_model)
self._wiz_ww_model.set_placeholder_text("okay_computer")
grid.attach(lbl_model, 0, 1, 1, 1)
grid.attach(self._wiz_ww_model, 1, 1, 1, 1)
box.pack_start(grid, False, False, 0)
# Test row
test_row = Gtk.Box(spacing=10)
test_row.set_margin_top(8)
self._wiz_ww_test_btn = Gtk.Button(label="Test connection")
self._wiz_ww_test_btn.connect("clicked", self._ww_test)
self._wiz_ww_test_lbl = Gtk.Label(label="", xalign=0.0)
test_row.pack_start(self._wiz_ww_test_btn, False, False, 0)
test_row.pack_start(self._wiz_ww_test_lbl, False, False, 0)
box.pack_start(test_row, False, False, 0)
# Setup hint (collapsible-ish — just always visible for simplicity)
hint = Gtk.Label(xalign=0.0)
hint.set_markup(
'<span size="small" alpha="75%">'
"Don't have a wakeword server yet? Run one with Docker:\n"
"<tt>docker run -it -p 10400:10400 homeassistant/wyoming-openwakeword</tt>"
"</span>")
hint.set_line_wrap(True)
hint.set_selectable(True)
hint.set_margin_top(12)
box.pack_start(hint, False, False, 0)
return box
def _build_stt(self) -> Gtk.Widget:
box = _page_box()
box.pack_start(_h1("Choose your speech-to-text engine"), False, False, 0)
box.pack_start(_sub(
"Blitztext transcribes your speech locally using Whisper, "
"or you can connect to a remote OpenAI-compatible API."
), False, False, 0)
# Local vs remote toggle
self._rb_local, card_local = _option_card(
"💻", "Local (faster-whisper)",
"Runs on your machine — private, no API key needed. "
"Choose the model size below.")
self._rb_remote, card_remote = _option_card(
"", "Remote API",
"Send audio to an OpenAI-compatible endpoint "
"(OpenAI, local Whisper server, etc.).")
self._rb_remote.join_group(self._rb_local)
self._rb_local.set_active(True)
for rb, card in ((self._rb_local, card_local), (self._rb_remote, card_remote)):
ebox = Gtk.EventBox()
ebox.add(card)
ebox.connect("button-press-event",
lambda _e, _ev, r=rb: (r.set_active(True),
self._stt_toggle()))
box.pack_start(ebox, False, False, 0)
rb.connect("toggled", lambda _r: self._stt_toggle())
# Local options
self._stt_local_box = Gtk.Box(orientation=Gtk.Orientation.VERTICAL, spacing=6)
self._stt_local_box.set_margin_top(8)
size_lbl = Gtk.Label(label="Model size", xalign=0.0)
size_lbl.get_style_context().add_class("dim-label")
self._stt_size_rb: dict[str, Gtk.RadioButton] = {}
sizes = [
("tiny", "Tiny — fastest, basic accuracy"),
("base", "Base — fast, decent accuracy"),
("small", "Small — balanced ✓ recommended"),
("medium", "Medium — better accuracy, slower"),
("large-v3", "Large — best accuracy, most memory"),
]
size_box = Gtk.Box(orientation=Gtk.Orientation.VERTICAL, spacing=3)
prev = None
for key, desc in sizes:
rb = Gtk.RadioButton(label=desc)
if prev:
rb.join_group(prev)
rb.set_active(key == self._stt_size)
rb.connect("toggled", lambda r, k=key: r.get_active() and
setattr(self, "_stt_size", k))
self._stt_size_rb[key] = rb
size_box.pack_start(rb, False, False, 0)
prev = rb
self._stt_local_box.pack_start(size_lbl, False, False, 0)
self._stt_local_box.pack_start(size_box, False, False, 0)
box.pack_start(self._stt_local_box, False, False, 0)
# Remote options
self._stt_remote_box = Gtk.Box(orientation=Gtk.Orientation.VERTICAL, spacing=6)
self._stt_remote_box.set_margin_top(8)
rgrid = Gtk.Grid(column_spacing=8, row_spacing=6)
rlbl_url = Gtk.Label(label="API URL", xalign=0.0); rlbl_url.set_size_request(100, -1)
self._stt_url_entry = Gtk.Entry(); self._stt_url_entry.set_hexpand(True)
self._stt_url_entry.set_text(self._stt_url or "http://localhost:8010/v1")
self._stt_url_entry.set_placeholder_text("http://localhost:8010/v1")
rlbl_key = Gtk.Label(label="API key env", xalign=0.0)
self._stt_key_entry = Gtk.Entry(); self._stt_key_entry.set_hexpand(True)
self._stt_key_entry.set_text(self._stt_key)
self._stt_key_entry.set_placeholder_text("OPENAI_API_KEY (leave empty if not needed)")
rgrid.attach(rlbl_url, 0, 0, 1, 1); rgrid.attach(self._stt_url_entry, 1, 0, 1, 1)
rgrid.attach(rlbl_key, 0, 1, 1, 1); rgrid.attach(self._stt_key_entry, 1, 1, 1, 1)
self._stt_remote_box.pack_start(rgrid, False, False, 0)
self._stt_remote_box.set_no_show_all(True)
box.pack_start(self._stt_remote_box, False, False, 0)
first_run_note = Gtk.Label(xalign=0.0)
first_run_note.set_markup(
'<span size="small" alpha="65%">'
"The local model is downloaded the first time you use it (~500 MB for Small). "
"Subsequent starts are instant."
"</span>")
first_run_note.set_line_wrap(True)
first_run_note.set_margin_top(6)
box.pack_start(first_run_note, False, False, 0)
return box
def _build_llm(self) -> Gtk.Widget:
box = _page_box()
box.pack_start(_h1("AI text processing (optional)"), False, False, 0)
box.pack_start(_sub(
"Blitztext can send your transcript to an AI model that rewrites it — "
"cleaning up speech artifacts, adjusting tone, or reformatting it."
), False, False, 0)
enable_row = Gtk.Box(spacing=10)
enable_row.set_margin_top(12)
self._llm_switch = Gtk.Switch()
self._llm_switch.set_active(self._llm_enabled)
self._llm_switch.set_valign(Gtk.Align.CENTER)
enable_lbl = Gtk.Label(label="Enable AI rewriting", xalign=0.0)
enable_row.pack_start(self._llm_switch, False, False, 0)
enable_row.pack_start(enable_lbl, False, False, 0)
box.pack_start(enable_row, False, False, 0)
self._llm_detail = Gtk.Box(orientation=Gtk.Orientation.VERTICAL, spacing=6)
self._llm_detail.set_margin_top(10)
lgrid = Gtk.Grid(column_spacing=8, row_spacing=8)
llbl_url = Gtk.Label(label="API URL", xalign=0.0); llbl_url.set_size_request(120, -1)
self._llm_url_entry = Gtk.Entry(); self._llm_url_entry.set_hexpand(True)
self._llm_url_entry.set_text(self._llm_url)
self._llm_url_entry.set_placeholder_text("https://api.openai.com/v1")
llbl_model = Gtk.Label(label="Model", xalign=0.0)
self._llm_model_entry = Gtk.Entry(); self._llm_model_entry.set_hexpand(True)
self._llm_model_entry.set_text(self._llm_model)
self._llm_model_entry.set_placeholder_text("gpt-4o-mini")
llbl_key = Gtk.Label(label="API key env", xalign=0.0)
self._llm_key_entry = Gtk.Entry(); self._llm_key_entry.set_hexpand(True)
self._llm_key_entry.set_text(self._llm_key)
self._llm_key_entry.set_placeholder_text("OPENAI_API_KEY")
lgrid.attach(llbl_url, 0, 0, 1, 1); lgrid.attach(self._llm_url_entry, 1, 0, 1, 1)
lgrid.attach(llbl_model, 0, 1, 1, 1); lgrid.attach(self._llm_model_entry, 1, 1, 1, 1)
lgrid.attach(llbl_key, 0, 2, 1, 1); lgrid.attach(self._llm_key_entry, 1, 2, 1, 1)
self._llm_detail.pack_start(lgrid, False, False, 0)
llm_note = Gtk.Label(xalign=0.0)
llm_note.set_markup(
'<span size="small" alpha="65%">'
"Works with OpenAI, or a local model via Ollama / LM Studio / vLLM. "
"You can fine-tune prompts per workflow in Settings → Workflows."
"</span>")
llm_note.set_line_wrap(True)
self._llm_detail.pack_start(llm_note, False, False, 0)
box.pack_start(self._llm_detail, False, False, 0)
self._llm_switch.connect("notify::active", lambda s, _p: self._llm_toggle())
self._llm_toggle()
return box
def _build_done(self) -> Gtk.Widget:
box = _page_box()
box.set_valign(Gtk.Align.CENTER)
lbl_done = Gtk.Label(label="")
attrs = Pango.AttrList()
attrs.insert(Pango.attr_scale_new(3.5))
lbl_done.set_attributes(attrs)
lbl_done.set_margin_bottom(12)
box.pack_start(lbl_done, False, False, 0)
box.pack_start(_h1("You're all set!"), False, False, 0)
self._done_summary = Gtk.Label(xalign=0.0)
self._done_summary.set_line_wrap(True)
self._done_summary.set_max_width_chars(60)
self._done_summary.get_style_context().add_class("dim-label")
box.pack_start(self._done_summary, False, False, 0)
tip = Gtk.Label(xalign=0.0)
tip.set_markup(
'\n<span size="small" alpha="70%">'
"You can always open <b>Settings</b> from the system tray to add workflows, "
"tune STT quality, or configure more shortcuts."
"</span>")
tip.set_line_wrap(True)
box.pack_start(tip, False, False, 0)
return box
# ------------------------------------------------------------------
# Page commit helpers
# ------------------------------------------------------------------
def _read_trigger(self) -> None:
if self._rb_ww.get_active():
self._trigger = "wakeword"
elif self._rb_both.get_active():
self._trigger = "both"
else:
self._trigger = "keyboard"
def _commit_page(self, name: str) -> None:
if name == "trigger":
self._read_trigger()
self._refresh_page_order()
elif name == "keyboard":
for attr, entry in self._kb_entries.items():
setattr(self, attr, entry.get_text().strip())
elif name == "wakeword":
self._ww_uri = self._wiz_ww_uri.get_text().strip()
self._ww_model = self._wiz_ww_model.get_text().strip()
elif name == "stt":
self._stt_local = self._rb_local.get_active()
if not self._stt_local:
self._stt_url = self._stt_url_entry.get_text().strip()
self._stt_key = self._stt_key_entry.get_text().strip()
elif name == "llm":
self._llm_enabled = self._llm_switch.get_active()
self._llm_url = self._llm_url_entry.get_text().strip()
self._llm_model = self._llm_model_entry.get_text().strip()
self._llm_key = self._llm_key_entry.get_text().strip()
self._update_done_summary()
def _update_done_summary(self) -> None:
lines: list[str] = []
if self._trigger in ("keyboard", "both"):
lines.append(f"⌨ Keyboard — start: {self._kb_start}, cancel: {self._kb_cancel}")
if self._trigger in ("wakeword", "both"):
lines.append(f"🎙 Wakeword — {self._ww_model!r} @ {self._ww_uri}")
if self._stt_local:
lines.append(f"🤖 Local Whisper ({self._stt_size})")
else:
lines.append(f"☁ Remote STT — {self._stt_url}")
if self._llm_enabled:
lines.append(f"✨ AI rewriting — {self._llm_model} @ {self._llm_url}")
else:
lines.append("✨ AI rewriting — disabled")
if hasattr(self, "_done_summary"):
self._done_summary.set_text("\n".join(lines))
# ------------------------------------------------------------------
# Apply to config
# ------------------------------------------------------------------
def _apply_to_cfg(self) -> None:
from .config import save
from .stt import STTEngine
cfg = self.cfg
# Trigger / keyboard mode
if self._trigger in ("keyboard", "both"):
cfg.input_mode = "modifiers"
cfg.key_start = self._kb_start
cfg.key_stop = self._kb_stop
cfg.key_send = self._kb_send
cfg.key_cancel = self._kb_cancel
# Wakeword
cfg.wakeword_enabled = self._trigger in ("wakeword", "both")
if cfg.wakeword_enabled:
cfg.wakeword_uri = self._ww_uri
cfg.wakeword_model = self._ww_model
from .config import WakewordEngine
# Update or create the first engine preset.
if cfg.wakeword_engines:
cfg.wakeword_engines[0].uri = self._ww_uri
cfg.wakeword_engines[0].model = self._ww_model
else:
cfg.wakeword_engines = [WakewordEngine(
name="Default", uri=self._ww_uri, model=self._ww_model)]
cfg.wakeword_active = cfg.wakeword_engines[0].name
# STT
if self._stt_local:
local_eng = STTEngine(name="Local Whisper", type="local",
model=self._stt_size)
# Replace or add.
locals_ = [e for e in cfg.stt_engines if e.is_local]
if locals_:
idx = cfg.stt_engines.index(locals_[0])
cfg.stt_engines[idx] = local_eng
else:
cfg.stt_engines.insert(0, local_eng)
cfg.stt_active = local_eng.name
cfg.model = self._stt_size
else:
remote_eng = STTEngine(name="Remote STT", type="openai",
url=self._stt_url, api_key_env=self._stt_key)
remotes = [e for e in cfg.stt_engines if not e.is_local]
if remotes:
idx = cfg.stt_engines.index(remotes[0])
cfg.stt_engines[idx] = remote_eng
else:
cfg.stt_engines.append(remote_eng)
cfg.stt_active = remote_eng.name
# LLM
if self._llm_enabled:
cfg.base_url = self._llm_url
cfg.rewrite_model = self._llm_model
cfg.api_key_env = self._llm_key
from .llm import LLMEngine
llm_eng = LLMEngine("Default", self._llm_url,
self._llm_model, self._llm_key)
if cfg.llm_engines:
cfg.llm_engines[0] = llm_eng
else:
cfg.llm_engines = [llm_eng]
cfg.llm_active = llm_eng.name
# Mark setup as complete so the wizard doesn't auto-show again.
cfg.setup_complete = True
save(cfg)
# ------------------------------------------------------------------
# UI toggle helpers
# ------------------------------------------------------------------
def _stt_toggle(self) -> None:
local = self._rb_local.get_active()
self._stt_local_box.set_visible(local)
self._stt_remote_box.set_visible(not local)
def _llm_toggle(self) -> None:
on = self._llm_switch.get_active()
self._llm_detail.set_sensitive(on)
# ------------------------------------------------------------------
# Wakeword connection test
# ------------------------------------------------------------------
def _ww_test(self, _btn) -> None:
uri = self._wiz_ww_uri.get_text().strip()
self._wiz_ww_test_lbl.set_text("Testing…")
self._wiz_ww_test_btn.set_sensitive(False)
def _probe():
import socket as _socket
from urllib.parse import urlparse
p = urlparse(uri)
host = p.hostname or "127.0.0.1"
port = p.port or 10400
try:
with _socket.create_connection((host, port), timeout=3.0):
ok = True
except OSError:
ok = False
def _update():
self._wiz_ww_test_btn.set_sensitive(True)
if ok:
self._wiz_ww_test_lbl.set_markup(
'<span foreground="#2a7d2a">✓ Connected</span>')
else:
self._wiz_ww_test_lbl.set_markup(
'<span foreground="#cc3333">✗ Could not connect — is the server running?</span>')
GLib.idle_add(_update)
threading.Thread(target=_probe, daemon=True).start()
# ------------------------------------------------------------------
# Key-binding capture
# ------------------------------------------------------------------
def _bind_key(self, entry: Gtk.Entry) -> None:
self._bind_entry = entry
self._bind_pressed = []
entry.set_text("")
entry.set_placeholder_text("press the key combination…")
def _on_key_press(self, _w, event) -> bool:
if self._bind_entry is None:
return False
tok = _keyval_token(event.keyval)
if tok and tok not in self._bind_pressed:
self._bind_pressed.append(tok)
return True
def _on_key_release(self, _w, event) -> bool:
if self._bind_entry is None:
return False
combo = _format_combo(self._bind_pressed)
if combo:
self._bind_entry.set_text(combo)
self._bind_entry.set_placeholder_text("")
self._bind_entry = None
self._bind_pressed = []
return True
# ------------------------------------------------------------------
# Public entry point
# ------------------------------------------------------------------
def run(self) -> None:
self.dlg.show_all()
# Hide remote STT box initially (local is default)
self._stt_remote_box.hide()
self.dlg.run()
self.dlg.destroy()
# ---------------------------------------------------------------------------
# Convenience: show if this is a fresh install
# ---------------------------------------------------------------------------
def maybe_show(cfg, parent: Gtk.Window | None = None) -> None:
"""Show the wizard if setup has never been completed."""
if not getattr(cfg, "setup_complete", False):
wiz = SetupWizard(cfg, parent=parent)
wiz.run()

130
linux/blitztext/sound.py Normal file → Executable file
View File

@ -1,38 +1,140 @@
"""Play short audio cues (a user WAV, or a built-in system sound) without blocking."""
"""Play short audio cues (a user WAV, or a built-in system sound) without blocking.
Path safety: all user-configured sound file paths are validated before use
to prevent path-traversal, device-file access, and symlink attacks.
"""
from __future__ import annotations
import os
import shutil
import stat
import subprocess
from pathlib import Path
_FREEDESKTOP = "/usr/share/sounds/freedesktop/stereo/{name}.oga"
_PLAYERS = ("pw-play", "paplay", "aplay")
# (player, extra_args, wav_only)
# Ordered: native PipeWire/PulseAudio first, then ffplay/gst as universal fallback.
_PLAYERS: list[tuple[str, list[str], bool]] = [
("pw-play", [], False),
("paplay", [], False),
("aplay", [], True), # WAV only
("ffplay", ["-nodisp", "-autoexit", "-loglevel", "quiet"], False),
("gst-play-1.0", [], False),
]
_NATIVE_EXTS = {".wav", ".oga", ".ogg", ".flac"}
# Allowed audio file extensions for user-configured sound paths.
_ALLOWED_AUDIO_EXTS = {".wav", ".mp3", ".ogg", ".oga", ".flac",
".m4a", ".aac", ".aif", ".aiff", ".opus"}
# Directories that user-configured sound paths are allowed to resolve into.
_ALLOWED_PARENTS: tuple[str, ...] = (
str(Path.home()),
"/usr/share/sounds",
"/opt/blitztext",
)
def play(path: str = "", *, fallback: str | None = None) -> None:
"""Play `path` (a WAV/OGA file); if unset/missing, play the freedesktop
`fallback` system sound. Returns immediately (fire-and-forget)."""
def validate_sound_path(path: str) -> str | None:
"""Validate a user-configured sound file path.
Returns the resolved absolute path if the file is safe to play,
or ``None`` if the path is empty, does not exist, or fails any
security check.
Checks performed:
1. Empty / whitespace-only paths are rejected.
2. ``~user`` is expanded via ``expanduser``.
3. The path is resolved to an absolute canonical path (``realpath``),
which eliminates symlinks and ``..`` components.
4. The resolved path must be a regular file (not a device, FIFO,
directory, or socket).
5. The resolved path must reside under one of the allowed parent
directories (user home, system sounds, or the Blitztext install
prefix).
6. The file extension must be a known audio format.
"""
if not path or not path.strip():
return None
expanded = os.path.expanduser(path.strip())
if not expanded:
return None
try:
resolved = Path(expanded).resolve(strict=False)
except (OSError, RuntimeError, ValueError):
return None
# Must exist and be a regular file.
if not resolved.exists():
return None
try:
mode = resolved.stat().st_mode
except OSError:
return None
if not stat.S_ISREG(mode):
return None
# Must be under an allowed parent directory.
resolved_str = str(resolved)
allowed = False
for parent in _ALLOWED_PARENTS:
try:
common = os.path.commonpath([resolved_str, parent])
if common == parent:
allowed = True
break
except ValueError:
continue
if not allowed:
return None
# Extension must be a known audio format.
ext = resolved.suffix.lower()
if ext not in _ALLOWED_AUDIO_EXTS:
return None
return resolved_str
def play(path: str = "", *, fallback: str | None = None) -> "subprocess.Popen | None":
"""Play `path` (WAV/MP3/OGG/FLAC/…); fallback to a freedesktop system sound.
User-supplied paths are validated via :func:`validate_sound_path` before
being passed to any audio player. Invalid or unsafe paths are silently
ignored (the fallback sound is still attempted).
Returns the Popen object so callers can terminate a preview, or None.
"""
target = ""
if path:
expanded = os.path.expanduser(path)
if os.path.exists(expanded):
target = expanded
safe = validate_sound_path(path)
if safe is not None:
target = safe
if not target and fallback:
fd = _FREEDESKTOP.format(name=fallback)
if os.path.exists(fd):
target = fd
if not target:
return
return None
# aplay only handles WAV; pw-play/paplay handle WAV + OGA, so try them first.
for player in _PLAYERS:
ext = os.path.splitext(target)[1].lower()
for player, extra, wav_only in _PLAYERS:
if not shutil.which(player):
continue
if player == "aplay" and not target.lower().endswith(".wav"):
if wav_only and ext != ".wav":
continue
if player == "paplay" and ext not in _NATIVE_EXTS:
continue
try:
subprocess.Popen([player, target], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
return
proc = subprocess.Popen(
[player] + extra + [target],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
)
return proc
except OSError:
continue
return None

View File

@ -27,6 +27,7 @@ class STTEngine:
url: str = "" # base URL incl. /v1 for remote, e.g. http://localhost:8010/v1
model: str = "" # remote model id, or local whisper size override
api_key_env: str = "" # env var holding a bearer key (optional)
timeout: int = 30 # HTTP timeout in seconds; raise for slow servers (WhisperX, etc.)
@property
def is_local(self) -> bool:
@ -61,31 +62,169 @@ def status(engine: STTEngine, timeout: float = 2.0) -> bool:
return reachable(engine.url, timeout)
@dataclass
class ModelMeta:
"""Model id plus optional metadata (languages, etc.) from the server."""
id: str
languages: list[str] = field(default_factory=list)
def fmt_languages(langs: list[str]) -> str:
"""Compact display string for a language list, e.g. 'en, de, fr +45'."""
if not langs:
return ""
if len(langs) >= 50:
return f"multilingual ({len(langs)})"
if len(langs) > 5:
return f"{', '.join(langs[:5])} +{len(langs) - 5}"
return ", ".join(langs)
def _api_base(url: str) -> str:
"""Strip endpoint-specific path suffixes so metadata/model probes hit the right root.
http://host:8081/v1/transcribe http://host:8081/v1
http://host:8081/transcribe http://host:8081
http://host:8081/v1 http://host:8081/v1 (unchanged)
"""
from urllib.parse import urlparse, urlunparse
p = urlparse(url.rstrip("/"))
path = p.path.rstrip("/")
v1_idx = path.find("/v1")
if v1_idx >= 0:
path = path[:v1_idx + 3] # keep up to and including /v1
elif path not in ("", "/"):
path = "" # strip unknown custom suffix entirely
return urlunparse((p.scheme, p.netloc, path, "", "", ""))
def list_models(base_url: str, api_key_env: str = "", timeout: float = 5.0) -> list[str]:
"""Fetch model ids from an OpenAI-compatible (or Ollama-style) /models endpoint."""
"""Fetch model ids from an OpenAI-compatible, Ollama-style, or Riva/NIM /models endpoint."""
return [m.id for m in list_models_meta(base_url, api_key_env, timeout)]
def list_models_meta(base_url: str, api_key_env: str = "", timeout: float = 5.0) -> list[ModelMeta]:
"""Like list_models() but returns ModelMeta with language info when available."""
import os
if not base_url:
return []
url = base_url.rstrip("/") + "/models"
headers = {}
base = _api_base(base_url)
headers: dict[str, str] = {}
key = os.environ.get(api_key_env) if api_key_env else None
if key:
headers["Authorization"] = f"Bearer {key}"
def _get(url: str) -> dict | None:
try:
req = urllib.request.Request(url, headers=headers)
with urllib.request.urlopen(req, timeout=timeout) as resp:
data = json.loads(resp.read().decode("utf-8"))
return json.loads(resp.read().decode("utf-8"))
except (urllib.error.URLError, json.JSONDecodeError, OSError):
return None
# 1. Standard OpenAI /models — faster-whisper-server also returns "language"
data = _get(base + "/models")
if isinstance(data, dict):
items = data.get("data")
if isinstance(items, list):
result = [ModelMeta(id=m["id"], languages=m.get("language") or [])
for m in items if isinstance(m, dict) and m.get("id")]
if result:
return result
items = data.get("models")
if isinstance(items, list): # Ollama shape
return [ModelMeta(id=m.get("name") or m.get("model", ""))
for m in items if m.get("name") or m.get("model")]
# 2. NVIDIA Riva / NIM
data = _get(base + "/metadata")
if isinstance(data, dict):
for info in data.get("modelInfo") or []:
name = info.get("shortName") or info.get("modelUrl") or ""
if name:
return [ModelMeta(id=name.split(":")[0])]
return []
items = data.get("data") if isinstance(data, dict) else None
if isinstance(items, list): # OpenAI shape: {"data":[{"id":...}]}
return [m["id"] for m in items if isinstance(m, dict) and m.get("id")]
items = data.get("models") if isinstance(data, dict) else None
if isinstance(items, list): # Ollama shape: {"models":[{"name"/"model":...}]}
return [m.get("name") or m.get("model") for m in items if (m.get("name") or m.get("model"))]
return []
def probe_server_ram_mb(base_url: str, timeout: float = 3.0) -> float | None:
"""Probe a remote server's Prometheus /metrics endpoint for RSS memory.
Returns RSS in MB if the server exposes Prometheus metrics with either
``process_resident_memory_bytes`` (standard Python/Go exporter) or
``container_memory_rss`` (cAdvisor). Returns None if not available.
Tries the server root first (http://host:port/metrics) then the api base
path (http://host:port/v1/metrics), because Prometheus endpoints are almost
always mounted at root even when the API lives under /v1.
"""
if not base_url:
return None
p = urlparse(base_url.rstrip("/"))
root = f"{p.scheme}://{p.netloc}"
candidates = [root + "/metrics"]
api_base_metrics = _api_base(base_url) + "/metrics"
if api_base_metrics != candidates[0]:
candidates.append(api_base_metrics)
def _parse(text: str) -> float | None:
for line in text.splitlines():
line = line.strip()
if line.startswith("#"):
continue
for metric in ("process_resident_memory_bytes", "container_memory_rss"):
if line.startswith(metric):
parts = line.split()
if len(parts) >= 2:
try:
return float(parts[-1]) / (1024 * 1024) # bytes → MB
except ValueError:
pass
return None
for url in candidates:
try:
req = urllib.request.Request(url)
with urllib.request.urlopen(req, timeout=timeout) as resp:
result = _parse(resp.read().decode("utf-8", "replace"))
if result is not None:
return result
except Exception:
continue
return None
def detect_remote_device(base_url: str, timeout: float = 3.0) -> str:
"""Best-effort GPU/CPU detection for a remote STT server.
Tries faster-whisper-server's /info endpoint (returns {"device":"cuda",...}),
then NVIDIA NIM /metadata (GPU-only service). Falls back to "remote".
"""
if not base_url:
return "remote"
base = _api_base(base_url)
try:
req = urllib.request.Request(base + "/info")
with urllib.request.urlopen(req, timeout=timeout) as resp:
data = json.loads(resp.read().decode("utf-8"))
if isinstance(data, dict):
dev = str(data.get("device") or data.get("compute_type") or "")
if "cuda" in dev.lower():
return "CUDA"
if dev:
return dev.upper()[:16]
except Exception:
pass
try:
req = urllib.request.Request(base + "/metadata")
with urllib.request.urlopen(req, timeout=timeout) as resp:
data = json.loads(resp.read().decode("utf-8"))
if isinstance(data, dict) and data.get("modelInfo"):
return "CUDA" # NVIDIA NIM is always GPU
except Exception:
pass
return "remote"
def _host_port(url: str) -> tuple[str | None, int]:
@ -105,7 +244,7 @@ def transcribe(
language: str = "",
hotwords: str = "",
local_transcriber=None,
timeout: int = 60,
timeout: int = 30,
) -> str:
if engine.is_local:
if local_transcriber is None:
@ -113,15 +252,24 @@ def transcribe(
return local_transcriber.transcribe(audio_path, language=language, hotwords=hotwords)
if engine.is_streaming:
raise STTError("Streaming STT engines are live-only. Use a workflow with mode = \"stream\".")
return _transcribe_remote(engine, audio_path, language=language, prompt=hotwords, timeout=timeout)
return _transcribe_remote(engine, audio_path, language=language, prompt=hotwords,
timeout=engine.timeout if engine.timeout > 0 else timeout)
def _transcribe_remote(engine: STTEngine, audio_path: Path, *, language: str, prompt: str, timeout: int) -> str:
import os
base = engine.url.rstrip("/")
# If path is non-standard (not empty / /v1 / /v1.0), treat the full URL as
# the endpoint — supports WhisperX (/transcribe) and other custom paths.
_path = urlparse(base).path.rstrip("/")
if _path in ("", "/v1", "/v1.0"):
endpoint = base + "/audio/transcriptions"
fields = {"model": engine.model or "whisper-1", "response_format": "json"}
else:
endpoint = base
fields: dict[str, str] = {"response_format": "json"}
if engine.model:
fields["model"] = engine.model
if language:
fields["language"] = language
if prompt:

View File

@ -54,8 +54,11 @@ class Transcriber:
log(f"{dev} unavailable ({exc}); trying next device")
raise RuntimeError(f"Failed to load Whisper model '{model}': {last_err}")
def transcribe(self, audio_path: Path, language: str = "", hotwords: str = "") -> str:
kwargs = dict(language=language or None, beam_size=self.beam_size, vad_filter=True)
def transcribe(self, audio_path: Path, language: str = "", hotwords: str = "",
beam_size: int | None = None) -> str:
kwargs = dict(language=language or None,
beam_size=beam_size if beam_size is not None else self.beam_size,
vad_filter=True)
if hotwords:
# Bias recognition toward the routing keywords so they transcribe
# reliably. Older faster-whisper builds lack `hotwords`; fall back.

View File

@ -88,6 +88,15 @@ class Tray:
menu.append(Gtk.SeparatorMenuItem())
# Cancel recording — always visible, only sensitive while recording.
# Primary escape hatch when wakeword fires on audiobook/TV audio.
self.cancel_item = Gtk.MenuItem(label="✕ Cancel recording")
self.cancel_item.set_sensitive(False)
self.cancel_item.connect(
"activate", lambda _i: self.app.daemon.cancel_dictation())
menu.append(self.cancel_item)
menu.append(Gtk.SeparatorMenuItem())
# Hands-free wakeword: a reversible pause toggle. Without this, a stale
# /tmp/wake_muted flag would silently disable detection with no way back.
if getattr(self.app.cfg, "wakeword_enabled", False):
@ -120,6 +129,8 @@ class Tray:
def update_status(self, state: str, message: str) -> None:
self.indicator.set_icon_full(ICONS.get(state, ICONS["idle"]), state)
self.status_item.set_label(f"{message or state.title()}")
if hasattr(self, "cancel_item"):
self.cancel_item.set_sensitive(state in ("recording", "armed", "busy"))
def pump(self) -> None:
"""Service pending GLib/GTK events; called from tkinter's loop."""

View File

@ -35,7 +35,7 @@ def set_muted(muted: bool) -> None:
elif os.path.exists(MUTE_FILE):
os.remove(MUTE_FILE)
except OSError as e: # noqa: BLE001 - mute is best-effort, never crash
logbuffer.log(f"[wakeword] Could not update mute flag: {e}")
logbuffer.log(f"[wakeword] Could not update mute flag: {e}", level="WARNING")
class WakewordListener:
@ -67,7 +67,7 @@ class WakewordListener:
try:
self._stream()
except Exception as e:
logbuffer.log(f"[wakeword] Connection error: {e}")
logbuffer.log(f"[wakeword] Connection error: {e}", level="WARNING")
time.sleep(3) # Retry backoff
def _stream(self):
@ -166,3 +166,133 @@ class WakewordListener:
logbuffer.log(f"[wakeword] Detected '{self.model}'!")
self._cooldown_until = time.time() + 3.0 # 3s cooldown
self.on_detect()
class WakewordActionListener:
"""Listens for multiple wakeword models simultaneously and calls per-model callbacks.
Used during active wakeword recording so that dedicated "cancel" and "send"
wakeword phrases trigger :meth:`~blitztext.daemon.Daemon.cancel_dictation` or
:meth:`~blitztext.daemon.Daemon.finish_dictation` immediately much faster
than waiting for Whisper to transcribe the whole clip.
``model_callbacks`` is a ``{model_name: callable}`` dict; only the models
present in the dict are requested from the server. No cooldown is applied
because the listener is torn down immediately after the first action fires.
"""
def __init__(self, uri: str, model_callbacks: dict, mic: str):
self.uri = uri
self.model_callbacks = dict(model_callbacks) # {name: callable}
self.mic = mic
self._stop_event = threading.Event()
self._thread: threading.Thread | None = None
def start(self) -> None:
if not self.model_callbacks:
return
self._stop_event.clear()
self._thread = threading.Thread(
target=self._run, daemon=True, name="WakewordActionListener")
self._thread.start()
def stop(self) -> None:
self._stop_event.set()
if self._thread:
self._thread.join(timeout=2.0)
self._thread = None
def _run(self) -> None:
while not self._stop_event.is_set():
try:
self._stream()
except Exception as e:
logbuffer.log(f"[wakeword-action] Connection error: {e}", level="WARNING")
time.sleep(2)
def _stream(self) -> None:
parsed = urlparse(self.uri)
host = parsed.hostname or "127.0.0.1"
port = parsed.port or 10400
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.settimeout(5.0)
sock.connect((host, port))
logbuffer.log(f"[wakeword-action] Connected — listening for {list(self.model_callbacks)}")
detect_msg = {"type": "detect", "data": {"names": list(self.model_callbacks)}}
sock.sendall((json.dumps(detect_msg) + "\n").encode("utf-8"))
audio_start = {"type": "audio-start",
"data": {"rate": 16000, "width": 2, "channels": 1}}
sock.sendall((json.dumps(audio_start) + "\n").encode("utf-8"))
read_active = True
def read_loop() -> None:
try:
sock.settimeout(1.0)
while read_active and not self._stop_event.is_set():
try:
line = b""
while not line.endswith(b"\n"):
byte = sock.recv(1)
if not byte:
return
line += byte
if not line:
return
msg = json.loads(line.decode("utf-8"))
if msg.get("type") == "detection":
name = msg.get("data", {}).get("name", "")
cb = self.model_callbacks.get(name)
if cb is None:
# Try partial match — some servers omit the lang suffix
for k, v in self.model_callbacks.items():
if name.startswith(k) or k.startswith(name):
cb = v
break
if cb:
logbuffer.log(
f"[wakeword-action] '{name}' detected — firing action")
self._stop_event.set() # one-shot: stop after first fire
cb()
payload_len = msg.get("payload_length", 0)
if payload_len > 0:
remaining = payload_len
while remaining > 0:
chunk = sock.recv(min(remaining, 4096))
if not chunk:
break
remaining -= len(chunk)
except socket.timeout:
pass
except Exception:
pass
reader = threading.Thread(target=read_loop, daemon=True)
reader.start()
cmd = ["pw-record", "--rate=16000", "--channels=1", "--format=s16", "-"]
if self.mic:
cmd.extend(["--target", self.mic])
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)
try:
while not self._stop_event.is_set() and proc.poll() is None:
chunk = proc.stdout.read(3200)
if not chunk:
break
header = {"type": "audio-chunk",
"data": {"rate": 16000, "width": 2, "channels": 1},
"payload_length": len(chunk)}
sock.sendall((json.dumps(header) + "\n").encode("utf-8"))
sock.sendall(chunk)
finally:
read_active = False
proc.terminate()
try:
proc.wait(timeout=1.0)
except subprocess.TimeoutExpired:
proc.kill()
reader.join(timeout=1.0)

View File

@ -0,0 +1,379 @@
"""Benchmark the wakeword detector with synthetic speech.
Generates short utterances random filler text with the wakeword spoken in it,
plus pure-filler utterances with no wakeword synthesizes each in a random voice
via the *same* (OpenAI-compatible) endpoint configured for speech-to-text, then
streams the audio to the running wyoming-openwakeword server and counts what it
detects. Reports:
recall share of wakeword utterances that fired at least one detection
false fires detections during pure-filler utterances (should be zero)
per-voice recall, so you can see which voices your model handles
TTS reuses the STT engine on purpose: on a typical NIM/OpenAI-compatible setup
the same server answers ``/audio/speech``, so there's nothing extra to configure
beyond a TTS model id and a voice list.
"""
from __future__ import annotations
import io
import json
import os
import random
import socket
import time
import urllib.error
import urllib.request
import wave
from dataclasses import dataclass, field
from urllib.parse import urlparse
# Bilingual filler so the synthesized speech is sentence-like (the detector sees
# realistic context around the wakeword, not just the bare phrase). Unknown
# languages fall back to English text — the voice still speaks it in its accent.
_FILLERS = {
"en": [
"the weather today is unusually calm and bright",
"remind me to call the office before noon",
"i think the train leaves around half past nine",
"could you put the report on my desk later",
"we should grab a coffee once this is done",
"the package was delivered to the wrong address again",
],
"de": [
"das wetter ist heute ungewöhnlich ruhig und klar",
"erinnere mich daran das büro vor mittag anzurufen",
"ich glaube der zug fährt gegen halb zehn",
"könntest du den bericht später auf meinen tisch legen",
"wir sollten einen kaffee trinken wenn das erledigt ist",
"das paket wurde wieder an die falsche adresse geliefert",
],
}
# OpenAI's stock voices; a sensible default for any OpenAI-compatible TTS. Local
# servers (Kokoro, openedai-speech, …) expose their own names — override in the
# config / Settings to match what your endpoint actually serves.
DEFAULT_VOICES = ["alloy", "echo", "fable", "onyx", "nova", "shimmer"]
_TARGET_RATE = 16000 # wyoming-openwakeword expects 16 kHz mono s16le
_CHUNK_BYTES = 3200 # 100 ms per audio-chunk, matching the live listener
@dataclass
class Utterance:
text: str
has_wakeword: bool
voice: str
detections: int = 0
ok: bool = False # synthesized + streamed without error
error: str = ""
@dataclass
class BenchResult:
utterances: list[Utterance] = field(default_factory=list)
seconds: float = 0.0
@property
def wake(self) -> list[Utterance]:
return [u for u in self.utterances if u.has_wakeword]
@property
def filler(self) -> list[Utterance]:
return [u for u in self.utterances if not u.has_wakeword]
@property
def detected(self) -> int:
return sum(1 for u in self.wake if u.ok and u.detections > 0)
@property
def expected(self) -> int:
return sum(1 for u in self.wake if u.ok)
@property
def recall(self) -> float:
return self.detected / self.expected if self.expected else 0.0
@property
def false_fires(self) -> int:
return sum(u.detections for u in self.filler if u.ok)
def recall_by_voice(self) -> dict[str, tuple[int, int]]:
"""voice -> (detected, expected) over wakeword utterances."""
out: dict[str, list[int]] = {}
for u in self.wake:
if not u.ok:
continue
d = out.setdefault(u.voice, [0, 0])
d[1] += 1
if u.detections > 0:
d[0] += 1
return {v: (d[0], d[1]) for v, d in out.items()}
def wakeword_phrase(model: str) -> str:
"""Turn a wakeword model id into the phrase to speak (best effort).
"okay_computer" -> "okay computer", "hey_jarvis" -> "hey jarvis". Strips a
trailing version/format suffix like "_v0.1" or ".tflite".
"""
name = model.rsplit("/", 1)[-1]
for ext in (".tflite", ".onnx"):
if name.endswith(ext):
name = name[: -len(ext)]
name = name.replace("_", " ").replace("-", " ")
# Drop a trailing token that is just a version like "v0.1".
parts = [p for p in name.split() if not (p.startswith("v") and any(c.isdigit() for c in p))]
return " ".join(parts).strip() or name.strip()
def _filler_pool(language: str) -> list[str]:
lang = (language or "").lower()
if lang.startswith("de"):
return _FILLERS["de"]
return _FILLERS["en"]
def build_utterances(phrase: str, count: int, language: str, *,
filler_count: int | None = None, voices=None,
rng: random.Random | None = None) -> list[Utterance]:
"""Build `count` wakeword utterances + `filler_count` pure-filler ones.
Each wakeword utterance embeds `phrase` at the start, middle, or end of a
random filler sentence; voices are assigned round-robin-ish at random so the
set covers every configured voice. Deterministic when `rng` is seeded.
"""
rng = rng or random.Random()
voices = list(voices or DEFAULT_VOICES) or DEFAULT_VOICES
pool = _filler_pool(language)
if filler_count is None:
filler_count = max(3, count // 3)
def a_voice() -> str:
return rng.choice(voices)
out: list[Utterance] = []
for _ in range(count):
filler = rng.choice(pool)
where = rng.choice(("start", "end", "mid"))
if where == "start":
text = f"{phrase}, {filler}"
elif where == "end":
text = f"{filler}, {phrase}"
else:
words = filler.split()
cut = len(words) // 2
text = " ".join(words[:cut] + [phrase] + words[cut:])
out.append(Utterance(text=text, has_wakeword=True, voice=a_voice()))
for _ in range(filler_count):
out.append(Utterance(text=rng.choice(pool), has_wakeword=False, voice=a_voice()))
rng.shuffle(out)
return out
# --- TTS via an OpenAI-compatible /audio/speech endpoint ---------------------
def _auth_headers(api_key_env: str) -> dict:
key = os.environ.get(api_key_env or "", "")
return {"Authorization": f"Bearer {key}"} if key else {}
def synthesize(tts_url: str, text: str, *, model: str, voice: str,
api_key_env: str = "", timeout: float = 30.0) -> bytes:
"""Return 16 kHz mono s16le PCM for `text` from an OpenAI-compatible TTS.
`tts_url` is the base incl. /v1 (e.g. http://localhost:8880/v1). Asks for WAV
and resamples whatever rate/-channels come back down to 16 kHz mono.
"""
base = (tts_url or "").rstrip("/")
if not base:
raise RuntimeError("No TTS URL configured (Settings → Benchmark → TTS URL).")
payload = json.dumps({
"model": model, "input": text, "voice": voice, "response_format": "wav",
}).encode("utf-8")
headers = {"Content-Type": "application/json", **_auth_headers(api_key_env)}
req = urllib.request.Request(base + "/audio/speech", data=payload, headers=headers, method="POST")
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
raw = resp.read()
except urllib.error.HTTPError as exc:
detail = exc.read().decode("utf-8", "replace")[:200]
raise RuntimeError(f"TTS HTTP {exc.code}: {detail}") from exc
except (urllib.error.URLError, OSError) as exc:
raise RuntimeError(f"TTS request failed: {exc}") from exc
return _wav_to_pcm16k(raw)
def list_voices(tts_url: str, *, api_key_env: str = "", timeout: float = 8.0) -> list[str]:
"""Best-effort voice discovery (Kokoro/XTTS expose /audio/voices or /voices).
Returns [] if the server has no such endpoint voices are then entered by
hand. Accepts the common shapes: ["a", ], {"voices": }, {"data": [{id}]}.
"""
base = (tts_url or "").rstrip("/")
if not base:
return []
for path in ("/audio/voices", "/voices"):
try:
req = urllib.request.Request(base + path, headers=_auth_headers(api_key_env))
with urllib.request.urlopen(req, timeout=timeout) as resp:
data = json.loads(resp.read())
except Exception: # noqa: BLE001 - endpoint optional/non-standard
continue
items = data.get("voices") or data.get("data") or [] if isinstance(data, dict) else data
out: list[str] = []
for it in items or []:
if isinstance(it, str):
out.append(it)
elif isinstance(it, dict):
name = it.get("id") or it.get("name") or it.get("voice")
if name:
out.append(str(name))
if out:
return out
return []
def probe(tts_url: str, *, model: str, voice: str, api_key_env: str = "",
timeout: float = 20.0) -> tuple[bool, str]:
"""Connectivity check: synthesize one short phrase. Returns (ok, message)."""
try:
pcm = synthesize(tts_url, "connection test", model=model, voice=voice,
api_key_env=api_key_env, timeout=timeout)
except Exception as exc: # noqa: BLE001 - surface the reason to the user
return False, str(exc)
return True, f"Connected — “{voice}” returned {len(pcm) / 2 / _TARGET_RATE:.1f}s of audio."
def _wav_to_pcm16k(wav_bytes: bytes) -> bytes:
"""Decode a WAV blob to 16 kHz mono signed-16 PCM (linear resample)."""
import numpy as np # type: ignore[import-untyped]
with wave.open(io.BytesIO(wav_bytes), "rb") as w:
ch, width, rate, n = w.getnchannels(), w.getsampwidth(), w.getframerate(), w.getnframes()
frames = w.readframes(n)
if width != 2:
raise RuntimeError(f"Unexpected TTS sample width {width*8}-bit (need 16-bit WAV).")
a = np.frombuffer(frames, dtype=np.int16).astype(np.float32)
if ch > 1:
a = a.reshape(-1, ch).mean(axis=1)
if rate != _TARGET_RATE and a.size:
new_len = int(round(a.size * _TARGET_RATE / rate))
if new_len > 0:
xp = np.linspace(0.0, 1.0, num=a.size, endpoint=False)
x = np.linspace(0.0, 1.0, num=new_len, endpoint=False)
a = np.interp(x, xp, a)
return np.clip(a, -32768, 32767).astype("<i2").tobytes()
# --- wyoming-openwakeword detection ------------------------------------------
def count_detections(uri: str, model: str, pcm: bytes, *, settle: float = 1.5,
timeout: float = 15.0) -> int:
"""Stream `pcm` (16 kHz mono s16le) to wyoming-openwakeword; count detections.
Sends a fresh detect/audio-start//audio-stop session, then drains detection
events until the server falls quiet for `settle` seconds (or `timeout`).
"""
parsed = urlparse(uri)
host, port = parsed.hostname or "127.0.0.1", parsed.port or 10400
deadline = time.time() + timeout
detections = 0
with socket.create_connection((host, port), timeout=5.0) as sock:
_send(sock, {"type": "detect", "data": {"names": [model]}})
_send(sock, {"type": "audio-start", "data": {"rate": 16000, "width": 2, "channels": 1}})
for i in range(0, len(pcm), _CHUNK_BYTES):
chunk = pcm[i:i + _CHUNK_BYTES]
_send(sock, {"type": "audio-chunk",
"data": {"rate": 16000, "width": 2, "channels": 1},
"payload_length": len(chunk)}, chunk)
_send(sock, {"type": "audio-stop", "data": {}})
sock.settimeout(settle)
buf = b""
while time.time() < deadline:
try:
data = sock.recv(4096)
except socket.timeout:
break # server quiet for `settle`s → done
if not data:
break
buf += data
buf, n = _drain_detections(buf)
detections += n
return detections
def _send(sock: socket.socket, msg: dict, payload: bytes = b"") -> None:
sock.sendall((json.dumps(msg) + "\n").encode("utf-8"))
if payload:
sock.sendall(payload)
def _drain_detections(buf: bytes) -> tuple[bytes, int]:
"""Parse whole newline-framed messages from buf; return (rest, detections).
Consumes each message's binary payload too, so payload bytes are never
mistaken for the next header line.
"""
found = 0
while b"\n" in buf:
line, rest = buf.split(b"\n", 1)
try:
msg = json.loads(line.decode("utf-8"))
except (ValueError, UnicodeDecodeError):
return rest, found
plen = msg.get("payload_length", 0) or 0
if len(rest) < plen:
return buf, found # payload not fully arrived yet; wait for more
rest = rest[plen:]
if msg.get("type") == "detection":
found += 1
buf = rest
return buf, found
@dataclass
class EngineRun:
"""One wakeword engine's benchmark result."""
name: str
uri: str
model: str
result: BenchResult
def run(engines, *, tts_url: str, tts_model: str, tts_api_key_env: str = "", voices=None,
language: str = "", count: int = 12, seed: int | None = None,
progress=None) -> list[EngineRun]:
"""Benchmark each wakeword engine against the same synthesized speech.
`engines` is a list of objects with ``.name`` / ``.uri`` / ``.model`` (e.g.
``config.WakewordEngine``). Each engine is scored on `count` utterances of
its own wake phrase (+ filler). Audio is synthesized once per (text, voice)
and reused across engines that share a phrase, so adding engines is cheap.
Calls ``progress(engine_idx, engine_total, engine_name, done, total, u)``.
"""
cache: dict[tuple[str, str], bytes] = {}
runs: list[EngineRun] = []
n_eng = len(engines)
for ei, eng in enumerate(engines, 1):
phrase = wakeword_phrase(eng.model)
utterances = build_utterances(phrase, count, language, voices=voices, rng=random.Random(seed))
res = BenchResult(utterances=utterances)
total = len(utterances)
t0 = time.time()
for ui, u in enumerate(utterances, 1):
try:
key = (u.text, u.voice)
pcm = cache.get(key)
if pcm is None:
pcm = synthesize(tts_url, u.text, model=tts_model, voice=u.voice,
api_key_env=tts_api_key_env)
cache[key] = pcm
u.detections = count_detections(eng.uri, eng.model, pcm)
u.ok = True
except Exception as exc: # noqa: BLE001 - record per-utterance, keep going
u.error = str(exc)
if progress:
progress(ei, n_eng, eng.name or eng.model, ui, total, u)
res.seconds = time.time() - t0
runs.append(EngineRun(eng.name or eng.model, eng.uri, eng.model, res))
return runs

View File

@ -1,49 +1,87 @@
#!/usr/bin/env bash
# Set up Blitztext for Linux: create a venv, install deps, check host tools,
# and write the default config. Run from the linux/ directory.
# Install Blitztext for Linux (no root needed).
#
# Creates a self-contained venv at ~/.local/lib/blitztext (outside any CIFS
# share, using --copies so no symlinks are needed), copies the package into it,
# and writes a launcher to ~/.local/bin/blitztext. The source folder can be
# deleted after this script finishes.
set -euo pipefail
cd "$(dirname "$0")"
VENV=".venv"
APPDIR="$HOME/.local/lib/blitztext"
VENV="$APPDIR/venv"
BIN="$HOME/.local/bin/blitztext"
echo "==> Checking host tools"
need_pkg=()
command -v xdotool >/dev/null || need_pkg+=("xdotool")
command -v notify-send >/dev/null || need_pkg+=("libnotify-bin")
# python3-gi (PyGObject) must come from apt — it can't be pip-installed.
python3 -c "import gi" 2>/dev/null || need_pkg+=(python3-gi python3-gi-cairo gir1.2-gtk-3.0 gir1.2-appindicator3-0.1)
command -v xdotool >/dev/null || need_pkg+=(xdotool)
command -v notify-send >/dev/null || need_pkg+=(libnotify-bin)
if ! command -v pw-record >/dev/null && ! command -v arecord >/dev/null && ! command -v parecord >/dev/null; then
need_pkg+=("pipewire-bin (or alsa-utils)")
need_pkg+=(pipewire-bin)
fi
if ((${#need_pkg[@]})); then
echo " Missing host tools: ${need_pkg[*]}"
echo " On Ubuntu/Debian: sudo apt install xdotool libnotify-bin pipewire-bin"
echo " (continuing — install them before running the daemon)"
if (( ${#need_pkg[@]} )); then
echo " Missing system packages — installing now (requires sudo):"
echo " sudo apt install ${need_pkg[*]}"
sudo apt-get install -y "${need_pkg[@]}"
fi
# --system-site-packages lets the venv see an apt-installed PyGObject (python3-gi)
# for the optional system-tray mode; harmless if it's not installed.
echo "==> Creating venv at $VENV"
python3 -m venv --system-site-packages "$VENV"
mkdir -p "$APPDIR"
# --copies avoids lib64->lib symlinks that fail on CIFS/SMB shares.
# --system-site-packages lets the venv see apt-installed python3-gi for the tray.
python3 -m venv --copies --system-site-packages "$VENV"
"$VENV/bin/pip" install --upgrade pip -q
"$VENV/bin/pip" install -r requirements.txt
"$VENV/bin/pip" install -q -r requirements.txt
echo "==> Installing blitztext package"
# Copy the package into site-packages so this source folder can be deleted.
cp -r blitztext "$VENV/lib/python"*/site-packages/
# On Debian/Ubuntu, python3-gi lives in /usr/lib/python3/dist-packages — the
# generic (non-versioned) path that venvs don't include even with
# --system-site-packages. A .pth file adds it to sys.path at startup.
SITE_PKG="$("$VENV/bin/python" -c "import site; print(site.getsitepackages()[0])")"
echo "/usr/lib/python3/dist-packages" > "$SITE_PKG/debian-system-packages.pth"
echo "==> Writing default config (if absent)"
"$VENV/bin/python" -m blitztext config-path
echo "==> Installing launcher → $BIN"
mkdir -p "$(dirname "$BIN")"
# Use a Python entry-point script (not `python -m`) so the current working
# directory is never added to sys.path and can't shadow the installed package.
cat > "$BIN" <<EOF
#!$VENV/bin/python
import sys
from blitztext.blitztext import main
sys.exit(main())
EOF
chmod +x "$BIN"
# Copy icons to hicolor so the app shows up properly in the tray / launcher.
ICON_DIR="$HOME/.local/share/icons/hicolor"
PKG_DIR="$(cd "$(dirname "$0")" && pwd)"
for s in 32 48 64 128 256; do
SRC="$PKG_DIR/packaging/blitztext_${s}.png"
if [ -f "$SRC" ]; then
DEST="$ICON_DIR/${s}x${s}/apps/blitztext.png"
mkdir -p "$(dirname "$DEST")"
cp "$SRC" "$DEST"
fi
done
gtk-update-icon-cache -qf "$ICON_DIR" 2>/dev/null || true
cat <<EOF
Done. To start it:
Done! Blitztext is installed at $APPDIR
$VENV/bin/python -m blitztext tray # system tray (needs: sudo apt install python3-gi)
$VENV/bin/python -m blitztext gui # control-panel window
$VENV/bin/python -m blitztext run # headless, hotkeys only
Start it:
blitztext tray # system tray
blitztext gui # control-panel window
Edit your config (hotkeys, Whisper model, rewrite endpoint) at:
(Make sure ~/.local/bin is in your PATH — it is by default on Ubuntu.)
\$($VENV/bin/python -m blitztext config-path)
For the rewrite workflows, export your key first, e.g.:
export OPENAI_API_KEY=sk-...
To run it in the background on login, see blitztext.service in this folder.
You can now delete the source folder if you cloned it just for installation.
EOF

View File

@ -6,6 +6,7 @@ Comment=Linux voice dictation with local, remote, and realtime STT engines
Exec=blitztext tray
Icon=blitztext
Terminal=false
StartupWMClass=blitztext
Categories=Utility;AudioVideo;Accessibility;
Keywords=dictation;speech;voice;transcription;whisper;stt;riva;nim;
StartupNotify=false

View File

@ -44,6 +44,9 @@ rm -rf "$VENV/lib/python"*/site-packages/pip "$VENV/lib/python"*/site-packages/p
"$VENV"/bin/pip* "$VENV/lib/python"*/site-packages/setuptools* 2>/dev/null || true
cp "$LINUX_DIR/README.md" "$LINUX_DIR/CHANGELOG.md" "$APPDIR/"
# MANUAL.md lives at repo root (one level above linux/)
REPO_MANUAL="$(dirname "$LINUX_DIR")/MANUAL.md"
[ -f "$REPO_MANUAL" ] && cp "$REPO_MANUAL" "$APPDIR/MANUAL.md"
# 2) Launcher, desktop entry, icon, docs ------------------------------------
install -Dm755 /dev/stdin "$ROOT/usr/bin/$PKG" <<'EOF'
@ -104,4 +107,9 @@ chmod 755 "$ROOT/DEBIAN/postrm"
mkdir -p "$OUT_DIR"
DEB="$OUT_DIR/${PKG}_${VER}_${ARCH}.deb"
fakeroot dpkg-deb --build --root-owner-group "$ROOT" "$DEB" >/dev/null
chmod 644 "$DEB"
# Also copy to ~ so `sudo apt install ~/blitztext_*.deb` works without a separate cp
DEST="$HOME/$(basename "$DEB")"
install -m644 "$DEB" "$DEST"
echo "==> Built: $DEB ($(du -h "$DEB" | cut -f1))"
echo "==> Copied: $DEST"

View File

@ -108,6 +108,41 @@ def test_matched_preset_announced_even_hands_free(monkeypatch):
assert shown == [], "no announcement when 'Announce matched preset' is off"
def test_overlay_fuses_notifications(monkeypatch):
"""With an overlay (routing_cb wired), the on-screen bubble narrates every
phase, so informational desktop notifications are suppressed but errors
still pop a bubble, and the routing match is shown on the overlay, not a
notification."""
calls = []
monkeypatch.setattr(daemon_mod, "notify", lambda *a, **k: calls.append((a, k)))
monkeypatch.setattr(daemon_mod, "detect_recorder", lambda pref="auto": "pw-record")
d = Daemon(Config(), routing_cb=lambda *a: None)
assert d._overlay is True
d._session_silent = False
d._dnotify("⌛ Nicer email", "Rewriting…") # informational
assert calls == [], "informational notifications fuse into the overlay"
d._dnotify("Error", "boom", urgency="critical") # errors still notify
assert len(calls) == 1, "errors must still pop a desktop bubble"
calls.clear()
d.cfg.notify_routing = True
d._rnotify("⚡ Nicer email", "matched: “nicer email”")
assert calls == [], "the match is shown on the overlay banner, not as a notification"
def test_no_overlay_keeps_notifications(monkeypatch):
"""Headless / overlay-off: notifications remain the only feedback."""
calls = []
monkeypatch.setattr(daemon_mod, "notify", lambda *a, **k: calls.append((a, k)))
d = _make_daemon(monkeypatch) # no routing_cb
assert d._overlay is False
d._session_silent = False
d._dnotify("⌛ Nicer email", "Rewriting…")
assert len(calls) == 1, "without an overlay, informational notifications still show"
def test_wakeword_while_busy_does_not_notify(monkeypatch):
"""The away-from-keyboard "Busy" storm: a detection arriving while the
previous clip is still being processed must be ignored silently."""

View File

@ -0,0 +1,41 @@
"""Streaming the LLM rewrite into the overlay (so you watch the model write).
`llm.chat(..., on_token=cb)` must parse an OpenAI-style SSE stream, hand each
content delta to the callback, and still return the complete text.
"""
import io
import blitztext.llm as llm
def test_read_stream_accumulates_and_calls_back():
sse = (
'data: {"choices":[{"delta":{"content":"Hel"}}]}\n'
"\n" # keep-alive blank line
'data: {"choices":[{"delta":{"content":"lo"}}]}\n'
'data: {"choices":[{"delta":{}}]}\n' # role-only / empty delta
": comment line\n" # SSE comment, ignored
"data: [DONE]\n"
'data: {"choices":[{"delta":{"content":"X"}}]}\n' # after DONE -> ignored
)
tokens = []
out = llm._read_stream(io.BytesIO(sse.encode("utf-8")), tokens.append)
assert out == "Hello"
assert tokens == ["Hel", "lo"]
def test_read_stream_survives_callback_errors():
sse = (
'data: {"choices":[{"delta":{"content":"a"}}]}\n'
'data: {"choices":[{"delta":{"content":"b"}}]}\n'
"data: [DONE]\n"
)
def boom(_delta):
raise RuntimeError("UI exploded")
# A failing UI callback must not break accumulation / delivery.
out = llm._read_stream(io.BytesIO(sse.encode("utf-8")), boom)
assert out == "ab"

View File

@ -51,3 +51,23 @@ def test_strip_span():
# 'nicer e-mail' normalizes to 3 tokens
res = _strip_span("Nicer e-mail can you send me the report", 3, "start")
assert res == "can you send me the report"
def test_route_matches_preset_name_without_keywords():
"""A preset with no keywords is still speakable by its name."""
presets = [
DummyPreset("Transcribe", []),
DummyPreset("Nicer email", []), # no keywords configured
]
res = route("nicer email please call me back tomorrow", presets)
assert res.preset_name == "Nicer email"
assert res.text == "please call me back tomorrow"
def test_explicit_keyword_still_wins_over_name():
presets = [
DummyPreset("Add emojis", ["emoji"]),
]
res = route("emoji this is great", presets)
assert res.preset_name == "Add emojis"
assert res.text == "this is great"

View File

@ -0,0 +1,79 @@
"""Spoken abort: a cancel word discards a clip before it is acted on.
Rescues accidentally triggered (e.g. wakeword) dictations the transcript is
recognised but nothing is routed, rewritten, or typed.
"""
import blitztext.daemon as dm
from blitztext.config import Config, Workflow
from blitztext.daemon import Daemon
from blitztext.routing import is_cancel
# -- matcher --------------------------------------------------------------------
def test_is_cancel_matches_edges_only():
kws = ["abbrechen", "cancel"]
assert is_cancel("abbrechen", kws) == "abbrechen" # whole utterance
assert is_cancel("ok das ganze bitte abbrechen", kws) == "abbrechen" # end edge
assert is_cancel("cancel this please", kws) == "cancel" # start edge
assert is_cancel("abbrechn", ["abbrechen"]) == "abbrechen" # ASR drift, fuzzy
# Word buried mid-sentence must NOT cancel a legitimate dictation.
assert is_cancel("ich will den vorgang abbrechen weil es spaet ist", ["abbrechen"]) is None
assert is_cancel("hallo welt", ["abbrechen"]) is None
assert is_cancel("abbrechen", []) is None # disabled
assert is_cancel("", ["abbrechen"]) is None
def test_cancel_keywords_round_trip(tmp_path):
from blitztext.config import load, save
p = tmp_path / "config.toml"
cfg = load(p)
assert cfg.cancel_keywords == ["abbrechen", "cancel"] # shipped default
cfg.cancel_keywords = ["nein doch nicht", "scrap that"]
save(cfg, p)
assert load(p).cancel_keywords == ["nein doch nicht", "scrap that"]
# -- pipeline -------------------------------------------------------------------
def _wire_clean_pipeline(monkeypatch):
monkeypatch.setattr(dm, "detect_recorder", lambda pref="auto": "pw-record")
monkeypatch.setattr(dm, "notify", lambda *a, **k: None)
monkeypatch.setattr(dm.quality, "analyze_wav", lambda p: (2.0, 0.5))
monkeypatch.setattr(dm.quality, "too_quiet", lambda *a, **k: False)
monkeypatch.setattr(dm.quality, "clean", lambda t, **k: t)
monkeypatch.setattr(dm.quality, "is_hallucination", lambda *a, **k: False)
delivered = []
monkeypatch.setattr(dm, "deliver", lambda *a, **k: delivered.append((a, k)))
return delivered
def test_process_discards_when_cancel_spoken(monkeypatch, tmp_path):
delivered = _wire_clean_pipeline(monkeypatch)
monkeypatch.setattr(dm.stt, "transcribe", lambda *a, **k: "ok das ganze bitte abbrechen")
d = Daemon(Config())
d._prepared = True
d.cfg.cancel_keywords = ["abbrechen", "cancel"]
audio = tmp_path / "clip.wav"
audio.write_bytes(b"x")
d._process(audio, Workflow(name="Transcribe", hotkey="", mode="transcribe"), None)
assert delivered == [], "a voice-cancelled clip must never be typed"
assert not audio.exists(), "the temp clip is still cleaned up afterwards"
assert d._busy is False
def test_process_delivers_without_cancel_word(monkeypatch, tmp_path):
"""Control: the same path with no cancel word still types normally."""
delivered = _wire_clean_pipeline(monkeypatch)
monkeypatch.setattr(dm.stt, "transcribe", lambda *a, **k: "hallo welt")
d = Daemon(Config())
d._prepared = True
d.cfg.cancel_keywords = ["abbrechen", "cancel"]
audio = tmp_path / "clip.wav"
audio.write_bytes(b"x")
d._process(audio, Workflow(name="Transcribe", hotkey="", mode="transcribe"), None)
assert len(delivered) == 1, "a normal clip must still be delivered"
assert delivered[0][0][0] == "hallo welt"

View File

@ -0,0 +1,83 @@
"""Spoken send: a configured word delivers the clip AND presses Enter.
The spoken equivalent of stop+paste+Enter mainly hands-free. The keyword is
stripped from the transcript before the rest is delivered and submitted.
"""
import blitztext.daemon as dm
import blitztext.paste as paste
from blitztext.config import Config, Workflow
from blitztext.daemon import Daemon
from blitztext.routing import match_send
# -- matcher --------------------------------------------------------------------
def test_match_send_strips_edges_only():
kws = ["computer send", "computer abschicken"]
assert match_send("computer send", kws)[0] == "computer send" # whole utterance
kw, text = match_send("hey team the build is green computer send", kws)
assert kw == "computer send" and text == "hey team the build is green" # end edge, stripped
kw, text = match_send("computer send hey team the build is green", kws)
assert kw == "computer send" and text == "hey team the build is green" # start edge, stripped
# A bare word inside a sentence must NOT submit — the phrase is distinctive.
assert match_send("please send me the report tomorrow", kws)[0] is None
assert match_send("computer send", [])[0] is None # disabled
assert match_send("", kws)[0] is None
# -- pipeline -------------------------------------------------------------------
def _wire_clean_pipeline(monkeypatch):
monkeypatch.setattr(dm, "detect_recorder", lambda pref="auto": "pw-record")
monkeypatch.setattr(dm, "notify", lambda *a, **k: None)
monkeypatch.setattr(dm.quality, "analyze_wav", lambda p: (2.0, 0.5))
monkeypatch.setattr(dm.quality, "too_quiet", lambda *a, **k: False)
monkeypatch.setattr(dm.quality, "clean", lambda t, **k: t)
monkeypatch.setattr(dm.quality, "is_hallucination", lambda *a, **k: False)
delivered = []
monkeypatch.setattr(dm, "deliver", lambda *a, **k: delivered.append((a, k)))
enters = []
monkeypatch.setattr(paste, "press_enter", lambda win=None: enters.append(win))
return delivered, enters
def test_process_sends_with_enter_when_send_spoken(monkeypatch, tmp_path):
delivered, enters = _wire_clean_pipeline(monkeypatch)
monkeypatch.setattr(dm.stt, "transcribe", lambda *a, **k: "die nachricht ist fertig computer send")
d = Daemon(Config())
d._prepared = True
d.cfg.send_keywords = ["computer send"]
d.cfg.cancel_keywords = []
audio = tmp_path / "clip.wav"; audio.write_bytes(b"x")
d._process(audio, Workflow(name="Transcribe", hotkey="", mode="transcribe"), None)
assert len(delivered) == 1
assert delivered[0][0][0] == "die nachricht ist fertig" # keyword stripped
assert len(enters) == 1, "send keyword must press Enter"
def test_process_no_enter_without_send_word(monkeypatch, tmp_path):
"""Control: the same path without a send word delivers but never hits Enter."""
delivered, enters = _wire_clean_pipeline(monkeypatch)
monkeypatch.setattr(dm.stt, "transcribe", lambda *a, **k: "die nachricht ist fertig")
d = Daemon(Config())
d._prepared = True
d.cfg.send_keywords = ["computer send"]
d.cfg.cancel_keywords = []
audio = tmp_path / "clip.wav"; audio.write_bytes(b"x")
d._process(audio, Workflow(name="Transcribe", hotkey="", mode="transcribe"), None)
assert len(delivered) == 1
assert delivered[0][0][0] == "die nachricht ist fertig"
assert enters == [], "no send word → no Enter"
def test_send_keywords_round_trip(tmp_path):
from blitztext.config import load, save
p = tmp_path / "config.toml"
cfg = load(p)
assert cfg.send_keywords == [] # shipped default: off
cfg.send_keywords = ["computer send", "computer abschicken"]
save(cfg, p)
assert load(p).send_keywords == ["computer send", "computer abschicken"]

View File

@ -0,0 +1,68 @@
"""Wakeword benchmark: deterministic parts (script, audio decode, framing, scoring).
The network parts (TTS /audio/speech, streaming to wyoming-openwakeword) need
live servers and are exercised manually from Settings Benchmark; here we cover
everything that can be checked offline.
"""
import io
import json
import random
import wave
import numpy as np
from blitztext import wakeword_bench as wb
def test_wakeword_phrase():
assert wb.wakeword_phrase("okay_computer") == "okay computer"
assert wb.wakeword_phrase("hey_jarvis") == "hey jarvis"
assert wb.wakeword_phrase("models/okay_nabu.tflite") == "okay nabu"
assert wb.wakeword_phrase("alexa_v0.1") == "alexa"
def test_build_utterances_deterministic_and_covers_voices():
a = wb.build_utterances("computer", 8, "de", voices=["nova", "onyx"], rng=random.Random(7))
b = wb.build_utterances("computer", 8, "de", voices=["nova", "onyx"], rng=random.Random(7))
key = lambda us: [(u.text, u.has_wakeword, u.voice) for u in us]
assert key(a) == key(b) # seeded → reproducible
assert sum(u.has_wakeword for u in a) == 8
assert all(u.voice in ("nova", "onyx") for u in a)
assert all("computer" in u.text for u in a if u.has_wakeword)
assert any(not u.has_wakeword for u in a) # filler added for false-fire check
def test_wav_to_pcm_resamples_to_16k_mono():
buf = io.BytesIO()
with wave.open(buf, "wb") as w:
w.setnchannels(2); w.setsampwidth(2); w.setframerate(48000)
n = 48000 # 1.0 s stereo @ 48 kHz
tone = (np.sin(2 * np.pi * 440 * np.linspace(0, 1, n, endpoint=False)) * 8000).astype("<i2")
w.writeframes(np.repeat(tone, 2).tobytes())
pcm = wb._wav_to_pcm16k(buf.getvalue())
assert len(pcm) == 16000 * 2 # 1.0 s of 16 kHz mono s16le
def test_drain_detections_handles_payloads():
det = (json.dumps({"type": "detection", "data": {}}) + "\n").encode()
chunk = (json.dumps({"type": "audio-chunk", "payload_length": 3}) + "\n").encode() + b"abc"
rest, n = wb._drain_detections(chunk + det)
assert n == 1 and rest == b"" # payload skipped, detection counted
held = (json.dumps({"type": "x", "payload_length": 10}) + "\n").encode() + b"ab"
rest2, n2 = wb._drain_detections(held)
assert n2 == 0 and rest2 == held # incomplete payload held, not misparsed
def test_bench_result_metrics():
r = wb.BenchResult(utterances=[
wb.Utterance("a", True, "nova", detections=1, ok=True),
wb.Utterance("b", True, "nova", detections=0, ok=True),
wb.Utterance("c", True, "onyx", detections=2, ok=True),
wb.Utterance("d", False, "onyx", detections=1, ok=True), # false fire
wb.Utterance("e", True, "x", ok=False, error="boom"), # failed → excluded
])
assert r.expected == 3 and r.detected == 2
assert abs(r.recall - 2 / 3) < 1e-9
assert r.false_fires == 1
assert r.recall_by_voice() == {"nova": (1, 2), "onyx": (1, 1)}

View File

@ -1,5 +1,8 @@
{
"extraPaths": [
"/usr/lib/python3/dist-packages"
]
],
"venvPath": "linux",
"venv": ".venv",
"reportMissingModuleSource": "none"
}