Compare commits
101 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f678d7d20f | ||
| 8f056e8cf0 | |||
| 4915884316 | |||
| e053719c59 | |||
| 4b8b9db43e | |||
| a9c08a54c8 | |||
| 125bd17713 | |||
| 0500b24900 | |||
| 8181ad75f9 | |||
| 79d6020fc6 | |||
| 1f3bbf2668 | |||
| 5cd31c8385 | |||
| af34360caa | |||
| d45ab4e91d | |||
| ca1f408876 | |||
| 5aafb572ed | |||
| a77b15ccaf | |||
| 1c276e767b | |||
| 016ffe0390 | |||
| 834cc20553 | |||
| 599eb08c5a | |||
| 9c812a960e | |||
| 8301cbb457 | |||
| 012a069ad0 | |||
| 8b26bc3379 | |||
| 6f133771e7 | |||
| 66e431f50a | |||
| b20aa31261 | |||
| 7a9dc5a2f7 | |||
| 571dae7cc9 | |||
| e831d7f1e6 | |||
| 34f75844c1 | |||
| ccf3801545 | |||
| efa44c8872 | |||
| c292ff2242 | |||
| c3cf123afe | |||
| 86632e1918 | |||
| d5e512a812 | |||
| 2326f20a28 | |||
| 05ccfbdeb4 | |||
| d40721dd3d | |||
| 79f3b5cdab | |||
| 766002a478 | |||
| d9603c98bf | |||
| f5f65ffba9 | |||
| 6fb41d5a9b | |||
| 48014ae662 | |||
| 9b95c632fa | |||
| 2df5be3212 | |||
| 51ab6d5aff | |||
| 8d79c84a8f | |||
| 81b79299ea | |||
| 0eafb59d82 | |||
| e39b253906 | |||
| a6d960bfbf | |||
| 5458fdc1eb | |||
| 1c16cfeb0f | |||
| bc6e2a6e54 | |||
| e01b130e5f | |||
| 4d59853da6 | |||
| 0ff0f604c2 | |||
| 101bb1e796 | |||
| 06167a1e6d | |||
| 6534284c56 | |||
| 4f68eda501 | |||
| 9022b17186 | |||
| aea24400af | |||
| 73264af2be | |||
| 99340b0ff3 | |||
| 0d1311ba18 | |||
| 59118364fb | |||
| 0875fe7ac0 | |||
| 749ef78383 | |||
| 946d344090 | |||
| 9e291437bd | |||
| 48be824a50 | |||
| 2fe6632269 | |||
| dd617f625e | |||
| d2c6d352a5 | |||
| cc589f254b | |||
| 7cbd828781 | |||
| b8fde76d7e | |||
| 836287d4f4 | |||
| f877a76550 | |||
| 2303dd8f78 | |||
| 15f8c1d9f7 | |||
| 0ce4f8e7ac | |||
| b0b7a8c95e | |||
| a472ce62ba | |||
| 27cf81de46 | |||
| 0d4ad0920d | |||
| 28020fdeda | |||
| 1c64f19bce | |||
| 1567202e1a | |||
| ae01472b77 | |||
| 6a9c1fc95a | |||
| d55cdafcc1 | |||
| d00064b6cd | |||
| 8cfc006296 | |||
| 227e4407c4 | |||
| ae12eea861 |
8
.gitignore
vendored
@ -30,3 +30,11 @@ Secrets.swift
|
||||
|
||||
# Tooling
|
||||
node_modules/
|
||||
|
||||
# Agent workspace
|
||||
jules/
|
||||
|
||||
# Raw screenshot source folders (published screenshots stay in Screenshots/)
|
||||
Screenshots/Settings - new/
|
||||
Screenshots/Settings-old/
|
||||
Screenshots/Welcome Setup/
|
||||
|
||||
244
MANUAL.md
@ -1,10 +1,11 @@
|
||||
# Blitztext — User Manual
|
||||
|
||||
A reference for every setting in the Blitztext **Settings** window, tab by tab.
|
||||
A reference for every setting in the Blitztext **Settings** window, page by page.
|
||||
|
||||
Open Settings from the system-tray menu (**Settings…**) or the control panel. The
|
||||
window has seven tabs — **Presets · Engines · Input · General · Benchmark · Log ·
|
||||
About** — and three buttons along the bottom.
|
||||
sidebar lists all pages: **Presets · General · Keyboard · Wakeword · STT Engines ·
|
||||
LLM Engines · Benchmark — STT · Benchmark — Wakeword · Log · Manual · About**.
|
||||
Three buttons run along the top: **Save**, **Save & Restart**, and **✕ Close**.
|
||||
|
||||
> **Where settings are stored:** `~/.config/blitztext/config.toml`
|
||||
> (or `$XDG_CONFIG_HOME/blitztext/config.toml`). You can edit that file directly;
|
||||
@ -20,7 +21,11 @@ About** — and three buttons along the bottom.
|
||||
|
||||
---
|
||||
|
||||
## Presets tab
|
||||
## Presets
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-presets.png"><img src="Screenshots/settings-presets.png" alt="Presets page" width="100%"></a>
|
||||
</p>
|
||||
|
||||
Presets are your dictation **actions**. Each one either types what you say, or
|
||||
rewrites it through the language model first (e.g. into a polished email). Trigger
|
||||
@ -33,8 +38,9 @@ Use the dropdown at the top to pick a preset to edit, **+ Add** to create one, o
|
||||
| Setting | TOML key | Description |
|
||||
|---|---|---|
|
||||
| **Name** | `name` | Short name for the action, shown in the main panel. |
|
||||
| **Icon (emoji)** | `icon` | Emoji shown next to this preset in the "matched preset" notification — give each a distinct one to tell them apart at a glance. Default `⚡`. |
|
||||
| **Description** | `description` | One line explaining what the preset does (shown in the panel). |
|
||||
| **Keywords (comma)** | `keywords` | Spoken trigger words, comma-separated. Say one at the **start or end** of your speech to select this preset (fuzzy-matched, e.g. `nicer email, bessere email`). |
|
||||
| **Keywords (comma)** | `keywords` | Spoken trigger words, comma-separated. Say one at the **start or end** of your speech to select this preset (fuzzy-matched, e.g. `nicer email, bessere email`). The preset's **name is always an implicit trigger**, so it works by voice even with no keywords here; add keywords for alternate/foreign-language phrasings. |
|
||||
| **Hotkey (optional)** | `hotkey` | A direct keyboard shortcut for this preset. Click **Set** and press the combo, or type it (e.g. `<ctrl>+<alt>+e`). Leave blank for keyword-only. |
|
||||
| **Mode** | `mode` | `transcribe` types your words as-is · `rewrite` sends them to the language model first · `stream` shows live text from a realtime STT engine. |
|
||||
| **LLM model (opt.)** | `model` | Override the language model for *this preset only*. Blank = use the active LLM engine's model. |
|
||||
@ -43,54 +49,35 @@ Use the dropdown at the top to pick a preset to edit, **+ Add** to create one, o
|
||||
|
||||
---
|
||||
|
||||
## Engines tab
|
||||
## General
|
||||
|
||||
Engines do the work: the **speech-to-text (STT)** engine turns your voice into
|
||||
text; the **language model (LLM)** rewrites it. Each engine can run locally or on
|
||||
a server you specify. A **green dot** means it's reachable, **red** means offline.
|
||||
The currently selected engine in each dropdown is the **active** one.
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-general.png"><img src="Screenshots/settings-general.png" alt="General page" width="100%"></a>
|
||||
</p>
|
||||
|
||||
### Speech-to-text engine
|
||||
|
||||
Buttons: **+ Add** (cloud/OpenAI-style), **+ Stream** (realtime Riva/NIM),
|
||||
**Delete**, **Test** (records 4 s and transcribes), **Refresh** (re-check status).
|
||||
Each engine maps to a `[[stt_engine]]` entry; the active one is `[stt] active`.
|
||||
Microphone, text delivery, language, notifications, the on-screen overlay, and
|
||||
autostart.
|
||||
|
||||
| Setting | TOML key | Description |
|
||||
|---|---|---|
|
||||
| **Name** | `name` | A label for this engine (e.g. "faster-whisper GPU"). |
|
||||
| **Type** | `type` | `local` (in-process faster-whisper) · `openai` (any OpenAI-compatible `/v1` STT server) · `riva_realtime` (live streaming engine). |
|
||||
| **URL** | `url` | Server endpoint. Example: `http://localhost:8010/v1` · realtime: `http://localhost:8006/v1`. Ignored for `local`. |
|
||||
| **Model** | `model` | Model name. For `local`: `tiny`/`base`/`small`/`medium`/`large-v3` or a path. For remote: blank = server default, or pick from the searchable list fetched from the URL. |
|
||||
| **API key env** | `api_key_env` | *Name of the environment variable* holding the API key (e.g. `GROQ_API_KEY`). Optional. |
|
||||
|
||||
**Local engine (faster-whisper) — device & precision** (global, `[whisper]`):
|
||||
|
||||
| Setting | TOML key | Description |
|
||||
|---|---|---|
|
||||
| **Device** | `device` | `auto` (try CUDA, fall back to CPU) · `cpu` · `cuda`. |
|
||||
| **Compute type** | `compute_type` | `auto` · `int8` · `float16` · `int8_float16`. Lower precision is faster and uses less memory. |
|
||||
|
||||
### Language model (rewrite)
|
||||
|
||||
Buttons: **+ Add**, **Delete**, **Refresh**. Each maps to a `[[llm_engine]]`
|
||||
entry; the active one is `[llm] active`.
|
||||
|
||||
| Setting | TOML key | Description |
|
||||
|---|---|---|
|
||||
| **Name** | `name` | A label for this LLM (e.g. "Local Qwen"). |
|
||||
| **Type** | `type` | `local` (a server on this machine) or `cloud`. |
|
||||
| **Base URL** | `url` | OpenAI-compatible endpoint, e.g. `http://localhost:28080/v1` or `https://api.openai.com/v1`. |
|
||||
| **Model** | `model` | The model to use; pick from the list once the URL is set. |
|
||||
| **API key env** | `api_key_env` | Environment-variable name holding the key (e.g. `OPENAI_API_KEY`). Blank for local servers. |
|
||||
| **Temperature** | `temperature` | Default creativity for rewrites (e.g. `0.3`). Presets can override this. |
|
||||
| **Microphone** | `mic` | Which input device Blitztext records from. |
|
||||
| **Input level** | — | Live level bar (read-only); should move when you speak. |
|
||||
| **Output** | `output` | `type` types the text key-by-key · `paste` copies it and presses Ctrl+V (faster for long text). |
|
||||
| **Language hint** | `language` | Spoken-language code (`de`, `en`, …). Blank = auto-detect. |
|
||||
| **Notifications** | `notify` | Show desktop notifications for recording/transcription status and errors (manual sessions). |
|
||||
| **Announce matched preset** | `notify_routing` | After a voice command, pop a notification showing which preset (and spoken keyword) matched — shown **even for hands-free** sessions, with the preset's emoji. Only fires on a real match. |
|
||||
| **Visual overlay** | `overlay_enabled` | Show a translucent bubble at the cursor while you dictate — a pulsing **microphone**, a **live waveform** of your mic level, and the **recognised text** (word-by-word with a streaming engine, or the final result as a brief confirmation). The tail points at where the text lands, and it gives **hands-free** sessions visible feedback. Click-through; never takes focus. *(X11 only.)* |
|
||||
| **Launch on login** | *(autostart file)* | Start Blitztext automatically when you log in (writes a desktop autostart entry, not `config.toml`). |
|
||||
|
||||
---
|
||||
|
||||
## Input tab
|
||||
## Keyboard
|
||||
|
||||
Controls **how you start and stop** dictating, the noise filter, hands-free
|
||||
wakeword, and audio cues.
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-keyboard.png"><img src="Screenshots/settings-keyboard.png" alt="Keyboard page" width="100%"></a>
|
||||
</p>
|
||||
|
||||
Controls **how you start and stop** dictating with the keyboard, the noise filter, and audio cues.
|
||||
|
||||
### Input mode & keys
|
||||
|
||||
@ -119,7 +106,26 @@ in the `[quality]` section.
|
||||
| **Reject hallucinations** | `reject_hallucinations` | Drop STT "ghost" outputs like *"Thank you."* / *"Bye."* that Whisper invents from silence. |
|
||||
| **Strip trailing punctuation** | `strip_trailing_punctuation` | Remove ending periods from delivered text — handy for code insertion. |
|
||||
|
||||
### Hands-free (Wakeword)
|
||||
### Audio cues (manual dictation)
|
||||
|
||||
These control the chimes for **manual** (keyboard/hotkey) dictation only. The
|
||||
hands-free wakeword sounds are **separate and independent** (see Wakeword page).
|
||||
|
||||
| Setting | TOML key | Description |
|
||||
|---|---|---|
|
||||
| **Play audio cues** | `[sounds] enabled` | On/off for the **manual** start/stop chimes below. Does **not** affect the wakeword sounds. |
|
||||
| **Play before** | `[sounds] before` | Chime when recording **starts** (manual dictation). Empty = built-in system sound. |
|
||||
| **Play after** | `[sounds] after` | Chime when recording **stops** (paste, paste+Enter, or auto-stop on silence). Empty = built-in system sound. |
|
||||
|
||||
> Each sound row has ▶ (preview) and ⌫ (clear).
|
||||
|
||||
---
|
||||
|
||||
## Wakeword
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-wakeword.png"><img src="Screenshots/settings-wakeword.png" alt="Wakeword page" width="100%"></a>
|
||||
</p>
|
||||
|
||||
Start dictation with a spoken keyword via an external
|
||||
[Wyoming](https://github.com/rhasspy/wyoming) openWakeWord server. Maps to the
|
||||
@ -142,20 +148,7 @@ Start dictation with a spoken keyword via an external
|
||||
> You can also pause/resume detection from the tray ("Pause wakeword"), which
|
||||
> toggles the `/tmp/wake_muted` flag.
|
||||
|
||||
### Audio cues (manual dictation)
|
||||
|
||||
These control the chimes for **manual** (keyboard/hotkey) dictation only. The
|
||||
hands-free wakeword sounds above are **separate and independent**.
|
||||
|
||||
| Setting | TOML key | Description |
|
||||
|---|---|---|
|
||||
| **Play audio cues** | `[sounds] enabled` | On/off for the **manual** start/stop chimes below. Does **not** affect the wakeword sounds above. |
|
||||
| **Play before** | `[sounds] before` | Chime when recording **starts** (manual dictation). Empty = built-in system sound. |
|
||||
| **Play after** | `[sounds] after` | Chime when recording **stops** (paste, paste+Enter, or auto-stop on silence). Empty = built-in system sound. |
|
||||
|
||||
> Each sound row has ▶ (preview) and ⌫ (clear).
|
||||
>
|
||||
> **The two pairs differ by trigger *and* by empty-behaviour:**
|
||||
> **The two sound pairs differ by trigger *and* by empty-behaviour:**
|
||||
>
|
||||
> | | Plays on | Used for | When empty |
|
||||
> |---|---|---|---|
|
||||
@ -164,25 +157,69 @@ hands-free wakeword sounds above are **separate and independent**.
|
||||
|
||||
---
|
||||
|
||||
## General tab
|
||||
## STT Engines
|
||||
|
||||
Microphone, text delivery, language, notifications, and autostart.
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-stt-engines.png"><img src="Screenshots/settings-stt-engines.png" alt="STT Engines page" width="100%"></a>
|
||||
</p>
|
||||
|
||||
The **speech-to-text** engine turns your voice into text. Each engine can run
|
||||
locally or on a server. A **green dot** means it's reachable, **red** means
|
||||
offline. The active engine is the one selected in the top dropdown.
|
||||
|
||||
Buttons: **+ Add** (batch/cloud/OpenAI-style), **+ Stream** (realtime Riva/NIM),
|
||||
**Delete**, **Test** (records 4 s and transcribes), **Refresh** (re-check status).
|
||||
Each engine maps to a `[[stt_engine]]` entry; the active one is `[stt] active`.
|
||||
|
||||
| Setting | TOML key | Description |
|
||||
|---|---|---|
|
||||
| **Microphone** | `mic` | Which input device Blitztext records from. |
|
||||
| **Input level** | — | Live level bar (read-only); should move when you speak. |
|
||||
| **Output** | `output` | `type` types the text key-by-key · `paste` copies it and presses Ctrl+V (faster for long text). |
|
||||
| **Language hint** | `language` | Spoken-language code (`de`, `en`, …). Blank = auto-detect. |
|
||||
| **Notifications** | `notify` | Show desktop notifications for recording/transcription status and errors (manual sessions). |
|
||||
| **Launch on login** | *(autostart file)* | Start Blitztext automatically when you log in (writes a desktop autostart entry, not `config.toml`). |
|
||||
| **Name** | `name` | A label for this engine (e.g. "faster-whisper GPU"). |
|
||||
| **Type** | `type` | `local` (in-process faster-whisper) · `openai` (any OpenAI-compatible `/v1` STT server) · `riva_realtime` (live streaming engine). |
|
||||
| **URL** | `url` | Server endpoint. Example: `http://localhost:8010/v1` · realtime: `http://localhost:8006/v1`. Ignored for `local`. |
|
||||
| **Model** | `model` | Model name. For `local`: `tiny`/`base`/`small`/`medium`/`large-v3` or a path. For remote: blank = server default, or pick from the searchable list fetched from the URL. |
|
||||
| **API key env** | `api_key_env` | *Name of the environment variable* holding the API key (e.g. `GROQ_API_KEY`). Optional. |
|
||||
|
||||
**Local engine (faster-whisper) — device & precision** (global, `[whisper]`):
|
||||
|
||||
| Setting | TOML key | Description |
|
||||
|---|---|---|
|
||||
| **Device** | `device` | `auto` (try CUDA, fall back to CPU) · `cpu` · `cuda`. |
|
||||
| **Compute type** | `compute_type` | `auto` · `int8` · `float16` · `int8_float16`. Lower precision is faster and uses less memory. |
|
||||
|
||||
---
|
||||
|
||||
## Benchmark tab
|
||||
## LLM Engines
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-llm-engines.png"><img src="Screenshots/settings-llm-engines.png" alt="LLM Engines page" width="100%"></a>
|
||||
</p>
|
||||
|
||||
The **language model** rewrites your dictated text (e.g. into a polished email).
|
||||
Each engine can be a local LAN server or a cloud service. The active engine is
|
||||
the one selected in the top dropdown.
|
||||
|
||||
Buttons: **+ Add**, **Delete**, **Refresh**. Each maps to a `[[llm_engine]]`
|
||||
entry; the active one is `[llm] active`.
|
||||
|
||||
| Setting | TOML key | Description |
|
||||
|---|---|---|
|
||||
| **Name** | `name` | A label for this LLM (e.g. "Local Qwen"). |
|
||||
| **Type** | `type` | `local` (a server on this machine) or `cloud`. |
|
||||
| **Base URL** | `url` | OpenAI-compatible endpoint, e.g. `http://localhost:28080/v1` or `https://api.openai.com/v1`. |
|
||||
| **Model** | `model` | The model to use; pick from the list once the URL is set. |
|
||||
| **API key env** | `api_key_env` | Environment-variable name holding the key (e.g. `OPENAI_API_KEY`). Blank for local servers. |
|
||||
| **Temperature** | `temperature` | Default creativity for rewrites (e.g. `0.3`). Presets can override this. |
|
||||
|
||||
---
|
||||
|
||||
## Benchmark — STT
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-benchmark-stt.png"><img src="Screenshots/settings-benchmark-stt.png" alt="Benchmark — STT page" width="100%"></a>
|
||||
</p>
|
||||
|
||||
Compare your STT engines for **speed and accuracy** on the same clip. Add an
|
||||
engine preset (Engines tab) for each model you want to compare. No persistent
|
||||
engine in the STT Engines page for each model you want to compare. No persistent
|
||||
settings — it's a one-off tool.
|
||||
|
||||
1. **Audio (.wav)** — a recording to transcribe.
|
||||
@ -190,27 +227,77 @@ settings — it's a one-off tool.
|
||||
a matching `*.txt` / `*.reference.txt` sits next to the WAV.)
|
||||
3. **Run benchmark** — fills the table with one row per engine.
|
||||
|
||||
Result columns: **Engine · Model · Device · Time (s) · Accuracy · Output**. A
|
||||
summary line names the **fastest** and **most accurate** engine.
|
||||
Result columns:
|
||||
|
||||
| Column | Description |
|
||||
|---|---|
|
||||
| **Engine** | Engine preset name |
|
||||
| **URL** | Server address (blank for local) |
|
||||
| **Model** | Model name used |
|
||||
| **Device** | `CPU`, `CUDA`, or `remote` |
|
||||
| **Best for** | `Short clips` · `Short / medium` · `Long / batch` · `Streaming` |
|
||||
| **Lang** | Supported languages from the server's `/v1/models` (`—` if unknown) |
|
||||
| **Time (s)** | Wall-clock seconds for this transcription |
|
||||
| **Accuracy** | `1 − WER` × 100 %. 100 % = word-perfect, case-sensitive |
|
||||
| **RAM (MB)** | RSS increase while the engine ran. Captures model load cost on first run. Remote engines show `—`. |
|
||||
| **Output** | Transcribed text (hover for full error on failure) |
|
||||
|
||||
A summary line names the **fastest** and **most accurate** engine. Click any column header to sort.
|
||||
|
||||
---
|
||||
|
||||
## Log tab
|
||||
## Benchmark — Wakeword
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-benchmark-wakeword.png"><img src="Screenshots/settings-benchmark-wakeword.png" alt="Benchmark — Wakeword page" width="100%"></a>
|
||||
</p>
|
||||
|
||||
Stress-test your wakeword detection by generating speech with a TTS server and
|
||||
checking whether the wake word fires correctly. Reports **recall** (how often it
|
||||
fires when it should) and **false-fire rate** (how often it fires on non-wake
|
||||
speech) across multiple synthetic voices.
|
||||
|
||||
---
|
||||
|
||||
## Log
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-log.png"><img src="Screenshots/settings-log.png" alt="Log page" width="100%"></a>
|
||||
</p>
|
||||
|
||||
A live activity log — useful to watch a model load/download or to diagnose a
|
||||
problem (recording, transcription, routing, and wakeword events all appear here).
|
||||
Press **Copy** to put the log on the clipboard when reporting an issue. No
|
||||
settings.
|
||||
|
||||
| Control | Description |
|
||||
|---|---|
|
||||
| **Level** dropdown | Filter by severity: **Verbose** (all), **Info** (default), **Warning**, **Error**. Switch to Warning or Error to cut noise when troubleshooting. |
|
||||
| **Copy** | Put the log on the clipboard when reporting an issue. |
|
||||
| **Clear** | Discard all current log entries. |
|
||||
| **Auto-scroll** | Keep the view scrolled to the latest entry. |
|
||||
|
||||
---
|
||||
|
||||
## About tab
|
||||
## Manual
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-manual.png"><img src="Screenshots/settings-manual.png" alt="Manual page" width="100%"></a>
|
||||
</p>
|
||||
|
||||
Inline copy of this manual, readable without leaving the app.
|
||||
|
||||
---
|
||||
|
||||
## About
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-about.png"><img src="Screenshots/settings-about.png" alt="About page" width="100%"></a>
|
||||
</p>
|
||||
|
||||
Read-only information:
|
||||
|
||||
- **Version** and a link to the source repository
|
||||
(`github.com/mARTin-B78/blitztext-app-linux`).
|
||||
- **License: MIT**.
|
||||
- **License: MIT** · **Copyright: 2026 mARTin Bierschenk - Design**.
|
||||
- Sub-tabs with the full **Changelog** and **License** text.
|
||||
|
||||
---
|
||||
@ -237,3 +324,8 @@ A few behaviours live in `config.toml` without a dedicated tab control:
|
||||
no keyword matches), and `threshold` (`0`–`1` fuzzy-match strictness).
|
||||
- **`timeout`** — network timeout (seconds) for remote STT/LLM requests.
|
||||
- **`type_delay_ms`** — delay between simulated keystrokes in `type` output mode.
|
||||
- **`overlay_anchor`** — where the overlay's tail points: `caret` (best-effort —
|
||||
follows the focused app's text caret via AT-SPI accessibility, falling back to
|
||||
the pointer), `pointer` (always the mouse pointer), or `corner` (a fixed screen
|
||||
corner; also the automatic fallback on Wayland or when the cursor can't be
|
||||
located). Paired with the **Visual overlay** toggle above.
|
||||
|
||||
145
README.md
@ -4,13 +4,17 @@
|
||||
|
||||
Blitztext is a native Linux dictation tool that captures your voice, transcribes it locally with [faster-whisper](https://github.com/SYSTRAN/faster-whisper), optionally rewrites the text through an LLM, and types the result directly into whatever application has focus. Think macOS Dictation, but open-source, extensible, and designed for power users who want full control over their speech-to-text pipeline.
|
||||
|
||||
> **Status:** Experimental open-source Linux/X11 desktop app (v1.3.0).
|
||||
> **Status:** Experimental open-source Linux/X11 desktop app (v1.7.0).
|
||||
> No hosted backend — bring your own models and endpoints.
|
||||
|
||||
<p align="center">
|
||||
<img src="Screenshots/panel.png" alt="Blitztext control panel" width="380">
|
||||
<img src="Screenshots/main-panel.png" alt="Blitztext control panel" width="360">
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<img src="Screenshots/overlay-listening.png" alt="On-screen overlay while listening" width="360">
|
||||
|
||||
<img src="Screenshots/tray-menu.png" alt="Blitztext system-tray menu" width="300">
|
||||
<img src="Screenshots/overlay-result.png" alt="On-screen overlay showing transcription result" width="360">
|
||||
</p>
|
||||
|
||||
📖 **[User manual](MANUAL.md)** — every setting in every tab, explained.
|
||||
@ -55,55 +59,134 @@ Stream: hotkey → mic PCM chunks → Riva/NIM WebSocket → live words typed
|
||||
- **Fully local STT.** Batch transcription via `faster-whisper` never leaves your machine. No cloud account needed for basic dictation.
|
||||
- **Pluggable engines.** Configure multiple STT and LLM backends as named presets — local `faster-whisper`, remote OpenAI-compatible batch endpoints, Riva/NIM realtime WebSocket servers, and any OpenAI-compatible chat API (OpenAI, vLLM, llama-swap, Ollama, LM Studio, Groq, OpenRouter).
|
||||
- **Voice-keyword routing.** One hotkey, multiple workflows. Say "nicer email" at the start or end of your speech and the email-rewrite preset activates automatically (fuzzy-matched, ASR-tolerant).
|
||||
- **Spoken cancel.** Say "abbrechen" (or "cancel") at the start or end of a clip and the whole dictation is discarded — never routed, rewritten, or typed. The rescue for an accidentally triggered (e.g. wakeword) recording. Configurable in Settings; empty list disables it.
|
||||
- **Quality gate.** Silent clips, too-short recordings, and Whisper hallucinations ("Thank you.", "Untertitel…") are caught and rejected before they reach your text field.
|
||||
- **Realtime streaming.** Connect a Riva/NIM realtime STT server and see stable words typed live as you speak.
|
||||
- **On-screen overlay at the cursor.** The moment you start dictating — by hotkey *or* wakeword — a translucent bubble pops up at the cursor with a pulsing microphone, a live waveform of your mic level, and the recognised text. When a voice keyword routes to a preset it shows that preset's icon, name, and the matched keyword on a banner (instead of a desktop notification), and streams the LLM rewrite into the bubble token-by-token so you watch it write. Its tail points at the text caret (via accessibility) and finally gives hands-free wakeword sessions visible feedback. Click-through, never steals focus; toggle in Settings → General.
|
||||
- **Built-in benchmarking.** Compare all your configured STT engines against a reference WAV + transcript to find the fastest and most accurate.
|
||||
|
||||
---
|
||||
|
||||
## Screenshots
|
||||
|
||||
Everything is configured in the GTK **Settings** window — every tab has tooltips
|
||||
and screen-reader (ATK) support. Click any image to open it full size.
|
||||
Everything is configured in the GTK **Settings** window — the sidebar gives quick
|
||||
access to every page. All controls have tooltips and screen-reader (ATK) support.
|
||||
Click any image to open it full size.
|
||||
|
||||
### Main panel & overlay
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-presets.png"><img src="Screenshots/settings-presets.png" alt="Presets settings tab" width="100%"></a><br>
|
||||
<em><b>Presets</b> — your dictation actions. Each preset is either a plain transcription or an LLM rewrite, and carries its own spoken keyword(s) for voice routing, an optional global hotkey, and a custom rewrite prompt.</em>
|
||||
<a href="Screenshots/main-panel.png"><img src="Screenshots/main-panel.png" alt="Blitztext main panel" width="46%"></a>
|
||||
|
||||
<a href="Screenshots/overlay-listening.png"><img src="Screenshots/overlay-listening.png" alt="Overlay — listening" width="46%"></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<em><b>Left:</b> Control panel listing all presets with icons, descriptions, and hotkeys.</em>
|
||||
|
||||
<em><b>Right:</b> On-screen overlay showing the live waveform while listening.</em>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-engines.png"><img src="Screenshots/settings-engines.png" alt="Engines settings tab" width="100%"></a><br>
|
||||
<em><b>Engines</b> — your speech-to-text and language-model back-ends, local or remote. Add and rename engines, watch live online/offline status, and pick models from a searchable list fetched straight from the endpoint.</em>
|
||||
<a href="Screenshots/overlay-result.png"><img src="Screenshots/overlay-result.png" alt="Overlay — transcription result" width="46%"></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<em>Overlay after transcription — preset name and recognised text appear at the cursor.</em>
|
||||
</p>
|
||||
|
||||
### Settings — General & Input
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-presets.png"><img src="Screenshots/settings-presets.png" alt="Presets page" width="48%"></a>
|
||||
|
||||
<a href="Screenshots/settings-general.png"><img src="Screenshots/settings-general.png" alt="General page" width="48%"></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<em><b>Presets</b> — dictation actions with keywords, hotkeys, LLM mode, and custom prompts.</em>
|
||||
|
||||
<em><b>General</b> — microphone, output mode, language hint, notifications, overlay, autostart.</em>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-input.png"><img src="Screenshots/settings-input.png" alt="Input settings tab" width="100%"></a><br>
|
||||
<em><b>Input</b> — how you start and stop dictation: the modifier-key scheme (Ctrl+Win / Ctrl / Alt / Esc) or custom hotkeys, plus the silence-based auto-stop (VAD), the quality gate, and audio cues.</em>
|
||||
<a href="Screenshots/settings-keyboard.png"><img src="Screenshots/settings-keyboard.png" alt="Keyboard page" width="48%"></a>
|
||||
|
||||
<a href="Screenshots/settings-wakeword.png"><img src="Screenshots/settings-wakeword.png" alt="Wakeword page" width="48%"></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<em><b>Keyboard</b> — modifier-key scheme or direct hotkeys, quality gate, audio cues.</em>
|
||||
|
||||
<em><b>Wakeword</b> — hands-free dictation via a Wyoming/openWakeWord server, with live level meter and model picker.</em>
|
||||
</p>
|
||||
|
||||
### Settings — Engines
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-stt-engines.png"><img src="Screenshots/settings-stt-engines.png" alt="STT Engines page" width="48%"></a>
|
||||
|
||||
<a href="Screenshots/settings-llm-engines.png"><img src="Screenshots/settings-llm-engines.png" alt="LLM Engines page" width="48%"></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<em><b>STT Engines</b> — speech-to-text back-ends (local faster-whisper, OpenAI-compatible server, or Riva realtime), with green/red status dot and Test button.</em>
|
||||
|
||||
<em><b>LLM Engines</b> — language-model back-ends for text rewriting (LAN server or cloud service).</em>
|
||||
</p>
|
||||
|
||||
### Settings — Benchmark
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-benchmark-stt.png"><img src="Screenshots/settings-benchmark-stt.png" alt="Benchmark — STT page" width="48%"></a>
|
||||
|
||||
<a href="Screenshots/settings-benchmark-wakeword.png"><img src="Screenshots/settings-benchmark-wakeword.png" alt="Benchmark — Wakeword page" width="48%"></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<em><b>Benchmark — STT</b> — compare every configured engine against a reference WAV + transcript; results table shows speed, accuracy, device, and language support.</em>
|
||||
|
||||
<em><b>Benchmark — Wakeword</b> — stress-test wakeword detection using a TTS server to synthesise wake phrases in random voices, reporting recall and false-fire rates.</em>
|
||||
</p>
|
||||
|
||||
### Settings — Log & About
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-log.png"><img src="Screenshots/settings-log.png" alt="Log page" width="48%"></a>
|
||||
|
||||
<a href="Screenshots/settings-about.png"><img src="Screenshots/settings-about.png" alt="About page" width="48%"></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<em><b>Log</b> — live activity log for recording, transcription, routing, and wakeword events.</em>
|
||||
|
||||
<em><b>About</b> — version, source link, inline changelog, and licence.</em>
|
||||
</p>
|
||||
|
||||
### Setup Wizard
|
||||
|
||||
The first-run wizard guides you through the essentials in a few steps.
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/setup-welcome.png"><img src="Screenshots/setup-welcome.png" alt="Setup — Welcome" width="32%"></a>
|
||||
|
||||
<a href="Screenshots/setup-trigger.png"><img src="Screenshots/setup-trigger.png" alt="Setup — Trigger mode" width="32%"></a>
|
||||
|
||||
<a href="Screenshots/setup-shortcuts.png"><img src="Screenshots/setup-shortcuts.png" alt="Setup — Keyboard shortcuts" width="32%"></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<em>Welcome · Choose trigger mode (keyboard / wakeword / both) · Set keyboard shortcuts</em>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/wakeword.png"><img src="Screenshots/wakeword.png" alt="Wakeword (hands-free) settings" width="100%"></a><br>
|
||||
<em><b>Wakeword (hands-free)</b> — point Blitztext at a Wyoming/openWakeWord server, choose a wake model, and test the connection live so a spoken keyword starts dictation with no keys at all.</em>
|
||||
<a href="Screenshots/setup-voice.png"><img src="Screenshots/setup-voice.png" alt="Setup — Voice activation" width="32%"></a>
|
||||
|
||||
<a href="Screenshots/setup-stt.png"><img src="Screenshots/setup-stt.png" alt="Setup — STT engine" width="32%"></a>
|
||||
|
||||
<a href="Screenshots/setup-ai.png"><img src="Screenshots/setup-ai.png" alt="Setup — AI rewriting" width="32%"></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<em>Voice activation (wakeword server) · Choose STT engine · Optional AI text rewriting</em>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-general.png"><img src="Screenshots/settings-general.png" alt="General settings tab" width="100%"></a><br>
|
||||
<em><b>General</b> — core preferences: microphone with a live level meter, output mode (type vs. paste), language hint, type delay, and autostart on login.</em>
|
||||
<a href="Screenshots/setup-done.png"><img src="Screenshots/setup-done.png" alt="Setup — All done" width="32%"></a>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-benchmark.png"><img src="Screenshots/settings-benchmark.png" alt="Benchmark settings tab" width="100%"></a><br>
|
||||
<em><b>Benchmark</b> — compare every configured STT engine against a reference WAV + transcript to find the fastest and most accurate, with a Device column (CPU / GPU / remote).</em>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-log.png"><img src="Screenshots/settings-log.png" alt="Log settings tab" width="100%"></a><br>
|
||||
<em><b>Log</b> — the in-app log buffer: a live view of recording, transcription, routing, and wakeword events for quick troubleshooting.</em>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="Screenshots/settings-about.png"><img src="Screenshots/settings-about.png" alt="About settings tab" width="100%"></a><br>
|
||||
<em><b>About</b> — version, source link, changelog, and licence.</em>
|
||||
<em>Summary screen — ready to dictate.</em>
|
||||
</p>
|
||||
|
||||
---
|
||||
@ -240,6 +323,10 @@ Now try with voice routing:
|
||||
1. Press `Ctrl+Alt+Space` → say **"nicer email** hey john can you send me the report"
|
||||
2. Press `Ctrl` — Blitztext detects the keyword, runs the "Nicer email" rewrite, and types a polished email.
|
||||
|
||||
### 4b. Cancel by voice
|
||||
|
||||
Started a recording by accident (or changed your mind)? Just say **"abbrechen"** (or **"cancel"**) at the start or end of what you say. The whole clip is discarded — nothing is transcribed onward, routed, rewritten, or typed, and the overlay briefly shows *✗ Abgebrochen*. This is especially handy with the hands-free wakeword, where a stray trigger could otherwise type ambient speech. Tune the words under **Settings → Mic/Cues → "Cancel words"** (or `[routing] cancel_keywords`); clear the list to switch it off.
|
||||
|
||||
### 5. Explore Settings
|
||||
|
||||
Click the ⚙️ gear icon in the panel header, or right-click the tray → **Settings…**
|
||||
@ -291,6 +378,7 @@ With the default `modifiers` input mode:
|
||||
| `Ctrl` | Stop → transcribe → type |
|
||||
| `Alt` | Stop → transcribe → type → press Enter |
|
||||
| `Esc` | Cancel (discard recording) |
|
||||
| say *"abbrechen"* / *"cancel"* | Cancel by voice — discard the clip (works hands-free too) |
|
||||
|
||||
---
|
||||
|
||||
@ -413,6 +501,7 @@ systemctl --user enable --now blitztext
|
||||
- **Wayland support** requires `wtype` or `ydotool`. Wayland security prevents global window focus manipulation, so text is delivered to whatever window is active when delivery occurs.
|
||||
- **No automated tests yet.** Contributions welcome (routing, quality gate, config parsing are all highly testable).
|
||||
- **Realtime streaming** requires a compatible Riva/NIM server.
|
||||
- **The on-screen overlay is X11-only** (it positions a window at the cursor and reads the pointer/caret); on Wayland it falls back to a fixed screen corner. Caret-accurate anchoring further needs the focused app to expose its text caret over AT-SPI accessibility — otherwise it follows the mouse pointer.
|
||||
- **Local STT speed** depends on your hardware, Whisper model size, and CTranslate2 build (CPU `int8` by default).
|
||||
- This is experimental software provided as-is.
|
||||
|
||||
|
||||
BIN
Screenshots/main-panel.png
Normal file
|
After Width: | Height: | Size: 54 KiB |
BIN
Screenshots/overlay-listening.png
Normal file
|
After Width: | Height: | Size: 8.2 KiB |
BIN
Screenshots/overlay-result.png
Normal file
|
After Width: | Height: | Size: 15 KiB |
|
Before Width: | Height: | Size: 158 KiB After Width: | Height: | Size: 202 KiB |
BIN
Screenshots/settings-benchmark-stt.png
Normal file
|
After Width: | Height: | Size: 127 KiB |
BIN
Screenshots/settings-benchmark-wakeword.png
Normal file
|
After Width: | Height: | Size: 134 KiB |
|
Before Width: | Height: | Size: 56 KiB After Width: | Height: | Size: 118 KiB |
BIN
Screenshots/settings-keyboard.png
Normal file
|
After Width: | Height: | Size: 145 KiB |
BIN
Screenshots/settings-llm-engines.png
Normal file
|
After Width: | Height: | Size: 102 KiB |
|
Before Width: | Height: | Size: 60 KiB After Width: | Height: | Size: 106 KiB |
BIN
Screenshots/settings-manual.png
Normal file
|
After Width: | Height: | Size: 219 KiB |
|
Before Width: | Height: | Size: 78 KiB After Width: | Height: | Size: 118 KiB |
BIN
Screenshots/settings-stt-engines.png
Normal file
|
After Width: | Height: | Size: 111 KiB |
BIN
Screenshots/settings-wakeword.png
Normal file
|
After Width: | Height: | Size: 164 KiB |
BIN
Screenshots/setup-ai.png
Normal file
|
After Width: | Height: | Size: 54 KiB |
BIN
Screenshots/setup-done.png
Normal file
|
After Width: | Height: | Size: 49 KiB |
BIN
Screenshots/setup-shortcuts.png
Normal file
|
After Width: | Height: | Size: 52 KiB |
BIN
Screenshots/setup-stt.png
Normal file
|
After Width: | Height: | Size: 71 KiB |
BIN
Screenshots/setup-trigger.png
Normal file
|
After Width: | Height: | Size: 54 KiB |
BIN
Screenshots/setup-voice.png
Normal file
|
After Width: | Height: | Size: 52 KiB |
BIN
Screenshots/setup-welcome.png
Normal file
|
After Width: | Height: | Size: 38 KiB |
@ -9,6 +9,704 @@ The version is defined in [`blitztext/__init__.py`](blitztext/__init__.py).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [2.03.41] - 2026-06-10
|
||||
|
||||
### Added
|
||||
- **Cancel button shown during transcription and rewriting.** The `×` button in
|
||||
the top-right corner of the overlay is now visible and clickable in the
|
||||
`busy` state (Transcribing… / Rewriting…), not only while recording.
|
||||
Clicking it during transcription discards the result once the STT call
|
||||
returns. Clicking it during a rewrite breaks out of the LLM stream
|
||||
immediately — the partial text is discarded and nothing is typed.
|
||||
|
||||
## [2.03.40] - 2026-06-10
|
||||
|
||||
### Changed
|
||||
- **Documentation overhaul with screenshots.** Renamed 21 raw screenshots to
|
||||
meaningful filenames (`main-panel.png`, `overlay-listening.png`,
|
||||
`settings-presets.png`, `settings-general.png`, `settings-keyboard.png`,
|
||||
`settings-wakeword.png`, `settings-stt-engines.png`,
|
||||
`settings-llm-engines.png`, `settings-benchmark-stt.png`,
|
||||
`settings-benchmark-wakeword.png`, `settings-log.png`,
|
||||
`settings-manual.png`, `settings-about.png`, and 7 setup-wizard screens).
|
||||
- **README.md / linux/README.md** updated with organized screenshot sections
|
||||
(Main panel & overlay, Settings — General & Input, Settings — Engines,
|
||||
Settings — Benchmark, Settings — Log & About, Setup wizard) using
|
||||
click-to-enlarge image links.
|
||||
- **MANUAL.md** rewritten to match the new sidebar navigation: intro updated;
|
||||
section headers renamed (Engines tab → STT Engines / LLM Engines; Input tab
|
||||
→ Keyboard + Wakeword; Benchmark tab → Benchmark — STT / Benchmark —
|
||||
Wakeword); screenshot added at the top of every section including Manual and
|
||||
About pages.
|
||||
|
||||
## [2.03.39] - 2026-06-10
|
||||
|
||||
### Fixed
|
||||
- **Remaining horizontal scrollbars eliminated.** Root causes: (1) infobox
|
||||
`max_width_chars` was 72 — at typical system font sizes (9 px/char) this
|
||||
produced a natural width wider than the ~677 px content area; reduced to 58.
|
||||
(2) Engine-selector combos (`stt_combo`, `llm_combo`, `ww_combo`) and the
|
||||
`_combo()` helper had no constraint on CellRendererText width — long engine
|
||||
names or ALSA device names drove the combo's natural width to 300–500 px.
|
||||
Fixed by adding `_ellipsize_combo()` (sets `ellipsize=END` and
|
||||
`max-width-chars=28` on the internal CellRendererText) to all combos.
|
||||
(3) `_STT_TYPES`, `_LLM_TYPES`, `_DEVICE_OPTIONS`, `_COMPUTE_OPTIONS`
|
||||
labels were 40–52 characters; shortened to ≤27 chars.
|
||||
|
||||
### Changed
|
||||
- **Wakeword Cancel/Send word rows split into two rows.** Keywords and
|
||||
keyboard shortcut are now on separate lines inside the card, avoiding the
|
||||
cramped single-row layout.
|
||||
- **Benchmark — Wakeword: "Run wakeword benchmark" button moved** from the
|
||||
bottom of the settings pane to the top of the results pane; pane divider
|
||||
adjusted from 390 → 340 px. The button is now always visible without
|
||||
scrolling and sits logically above the results it produces.
|
||||
|
||||
## [2.03.38] - 2026-06-10
|
||||
|
||||
### Fixed
|
||||
- **STT Engines and Wakeword no longer have horizontal scrollbars.** Root
|
||||
cause: `Gtk.Entry` widgets compute natural width from placeholder text
|
||||
(e.g. `"http://localhost:8010/v1 · realtime: http://localhost:8006/v1"`
|
||||
≈ 500 px). Without `set_width_chars(1)` the entry cannot shrink below its
|
||||
natural width even when placed in an expanding container. Added
|
||||
`set_width_chars(1)` to all entry-creating helpers: `_entry()`, `_url_field()`,
|
||||
`ModelPicker`, `_kw_shortcut_row`, and `_sound_field`. Also added
|
||||
`set_max_width_chars(50)` to the `stt_result` wrapping label.
|
||||
|
||||
## [2.03.37] - 2026-06-10
|
||||
|
||||
### Fixed
|
||||
- **STT Engines page no longer appears empty.** `_refresh_status()` called
|
||||
`_stt_commit()` / `_llm_commit()` and accessed `stt_dot` / `llm_dot`
|
||||
unconditionally; if the STT page was opened before the LLM page was built
|
||||
(lazy), the builder crashed silently with `AttributeError`. Added `hasattr`
|
||||
guards so each section is only committed / updated when its widgets exist.
|
||||
- **Wakeword page no longer causes horizontal scrollbar.** The `ww_status`
|
||||
label (showing model list like "7 models loaded: okay_nabu, hey_jarvis…")
|
||||
had no width limit and expanded the page. Added `set_max_width_chars(30)`
|
||||
and `set_ellipsize(END)`.
|
||||
- **Benchmark STT engine list no longer causes horizontal scrollbar.**
|
||||
`sel_sw` used `NEVER` horizontal policy, propagating long engine-name labels
|
||||
(~800 px) up through the paned. Changed to `AUTOMATIC` so content scrolls
|
||||
internally.
|
||||
- **General page and LLM Engines no longer cause horizontal scrollbar.**
|
||||
`_combo()` and `_type_combo()` lacked `set_size_request(10, -1)`, so
|
||||
ComboBoxText widgets (e.g. long microphone device names) could not shrink
|
||||
below their natural width. Added the size request to both helpers.
|
||||
|
||||
## [2.03.36] - 2026-06-10
|
||||
|
||||
### Changed
|
||||
- **STT Engines split from LLM Engines.** The single "STT Engines" sidebar
|
||||
page is now two pages — "STT Engines" (speech-to-text configuration) and
|
||||
"LLM Engines" (language model / rewrite configuration) — each with its
|
||||
own infobox. The underlying `_stt_section()` and `_llm_section()` methods
|
||||
are unchanged.
|
||||
|
||||
## [2.03.35] - 2026-06-10
|
||||
|
||||
### Fixed
|
||||
- **Settings pages no longer widen the dialog.** `_switch_row` description
|
||||
labels had `set_line_wrap(True)` but no `set_max_width_chars`, so GTK
|
||||
computed their natural width as the full un-wrapped text (87 chars × ~8 px
|
||||
= ~700 px). With `NEVER` horizontal policy on the page `ScrolledWindow`,
|
||||
that propagated directly to the dialog width, making Keyboard, Wakeword,
|
||||
STT Engines, and Benchmark pages ~1000–1360 px wide. Fixed by adding
|
||||
`set_max_width_chars(50)` (≈ 375 px) to description labels, and changed the
|
||||
page `ScrolledWindow` horizontal policy from `NEVER` to `AUTOMATIC` as a
|
||||
safety net for any other wide widget.
|
||||
|
||||
## [2.03.34] - 2026-06-10
|
||||
|
||||
### Fixed
|
||||
- **Settings window no longer grows wide or tall when switching pages.**
|
||||
`Stack.set_homogeneous(True)` was causing the Stack to request the maximum
|
||||
natural size of all its children (both width and height), so pages like
|
||||
Benchmark — STT (wide TreeView) inflated the dialog to 1000+ px wide for
|
||||
every other page. Reverted to `False`; the dialog now stays at its default
|
||||
860 × 700 and each page scrolls if its content is taller than the window.
|
||||
Also removed the erroneous `NEVER/NEVER` ScrolledWindow policy on the two
|
||||
benchmark pages that was propagating natural TreeView width to the Stack.
|
||||
|
||||
## [2.03.33] - 2026-06-10
|
||||
|
||||
### Fixed
|
||||
- **Settings window no longer resizes when switching pages.** Added
|
||||
`Gtk.Stack.set_homogeneous(True)` so the dialog always allocates the
|
||||
maximum page height, preventing the window from growing or shrinking
|
||||
as pages are visited.
|
||||
- **Benchmark — Wakeword: "Engines to test" section no longer hidden.**
|
||||
The paned divider position was raised from 260 to 390 px so all TTS
|
||||
config fields, the engine checkboxes, wakeword model selector, sample
|
||||
count, and run button are fully visible without scrolling. Both
|
||||
benchmark pages also disable the page-level `ScrolledWindow` so the
|
||||
paned correctly fills the viewport height rather than expanding
|
||||
past it.
|
||||
|
||||
## [2.03.32] - 2026-06-10
|
||||
|
||||
### Changed
|
||||
- **Settings dialog navigation redesigned.** Replaced the `Gtk.Notebook` tab bar
|
||||
with a 170 px left sidebar (`Gtk.ListBox` of flat buttons with section headers)
|
||||
and a `Gtk.Stack` for the content area. The dialog is now 860 × 700 px by default.
|
||||
Lazy-loading is preserved: each page is built only on first visit.
|
||||
- **Input page split into Keyboard and Wakeword.** The former "Input" tab is now
|
||||
two separate pages — "Keyboard" (input mode, hotkeys, quality gate, audio cues)
|
||||
and "Wakeword" (enable switch, mic level, test button, silence timeout, cancel/send
|
||||
words, engine preset selector, engine config card, wakeword sound cues). Either
|
||||
page may be visited first; the mic level-meter starts on whichever is opened.
|
||||
- **Benchmark page split into Benchmark — STT and Benchmark — Wakeword.** The
|
||||
single "Benchmark" tab is now two dedicated pages. All field names and collect
|
||||
logic are unchanged.
|
||||
## [2.03.31] - 2026-06-10
|
||||
|
||||
### Added
|
||||
- **First-run setup wizard.** A paged GTK dialog (`setup_wizard.py`) guides
|
||||
new users through five steps: trigger method (keyboard / wakeword / both),
|
||||
keyboard shortcut assignment (with live key capture), wakeword server
|
||||
configuration (with connection test), speech-to-text engine selection
|
||||
(local Whisper model size or remote API), and optional AI rewriting (LLM
|
||||
endpoint + model). Navigation has Back, Next, and Skip buttons. The wizard
|
||||
shows automatically on first launch (before the daemon starts) and can be
|
||||
reopened via the "Setup Wizard…" button in the Settings header bar.
|
||||
Completing the wizard sets `setup_complete = true` in the config so it
|
||||
does not reappear.
|
||||
|
||||
## [2.03.30] - 2026-06-10
|
||||
|
||||
### Added
|
||||
- **Dedicated wakeword models for cancel and send.** Two new optional fields in
|
||||
the wakeword settings card — "Cancel model" and "Send model" — let you assign
|
||||
a specific wakeword model (e.g. a custom "stop" or "send it" ONNX model) to
|
||||
each action. When configured, a `WakewordActionListener` opens a second
|
||||
Wyoming connection during recording and fires the action the instant the model
|
||||
triggers — no Whisper pass, no silence timer. The Whisper-based cancel watcher
|
||||
from v2.03.29 remains as a fallback when no cancel wakeword model is set.
|
||||
Model dropdowns are populated from the same server fetch as the trigger model.
|
||||
|
||||
## [2.03.29] - 2026-06-10
|
||||
|
||||
### Changed
|
||||
- **Cancel keywords now fire immediately during wakeword recording.** A
|
||||
real-time `_CancelWatcher` accumulates raw PCM from the VAD level-meter,
|
||||
then every ~0.6 s of new audio runs a fast `beam_size=1` local transcription
|
||||
pass to check for cancel keywords. When one is found it calls
|
||||
`cancel_dictation()` instantly — no waiting for the silence timer or a full
|
||||
transcription of the whole clip. Falls back to the existing post-transcription
|
||||
check if no local transcriber is loaded or no cancel keywords are configured.
|
||||
|
||||
## [2.03.28] - 2026-06-10
|
||||
|
||||
### Fixed
|
||||
- **Dropdowns no longer change on accidental scroll.** All `ComboBoxText`
|
||||
widgets in the settings dialog now swallow scroll events so hovering over
|
||||
a combo and scrolling doesn't silently change the selected value.
|
||||
|
||||
## [2.03.27] - 2026-06-10
|
||||
|
||||
### Added
|
||||
- **Overlay × cancel button.** A small × button appears in the top-right corner
|
||||
of the on-screen waveform HUD while recording or streaming. Clicking it
|
||||
cancels the current dictation. The rest of the overlay remains fully
|
||||
click-through; only the button area receives pointer events.
|
||||
- **Cancel/Send keyboard shortcuts in Wakeword tab.** The "Cancel words" and
|
||||
"Send words" rows in the Input → Wakeword section now include an inline
|
||||
shortcut entry + "Set" button, so you can configure `key_cancel` /
|
||||
`key_send` right next to the spoken-word equivalents without visiting the
|
||||
keyboard-mode card.
|
||||
|
||||
## [2.03.26] - 2026-06-10
|
||||
|
||||
### Fixed
|
||||
- **Cancel key now works during wakeword-triggered recording.** Previously the
|
||||
`ModifierScheme` state machine stayed in "idle" when the wakeword fired
|
||||
(it bypasses the key-press path), so the cancel hotkey was silently ignored.
|
||||
It now checks `daemon.is_recording` as a fallback so it fires regardless of
|
||||
how recording started.
|
||||
|
||||
### Added
|
||||
- **"✕ Cancel recording" in the tray menu.** Always visible; grayed out when
|
||||
idle, enabled as soon as recording starts (wakeword or manual). The primary
|
||||
escape hatch when the wakeword fires on audiobook / TV audio and spoken
|
||||
cancel words can't be heard over the background audio.
|
||||
|
||||
## [2.03.25] - 2026-06-10
|
||||
|
||||
### Changed
|
||||
- **Smart Save — no restart popup anymore.** "Save" now diffs the changed
|
||||
settings against what requires a daemon restart. If only safe settings
|
||||
changed (language, sounds, LLM prompt, keywords, overlay, …) it shows
|
||||
"✓ Settings applied" inline in the header bar for 4 s and closes the
|
||||
dialog. If restart-required fields changed (STT engine, hotkeys,
|
||||
microphone, wakeword server) it shows "⚠ Saved — restart needed for: …"
|
||||
and highlights "Save & Restart" so you can act on it. No modal popups.
|
||||
|
||||
## [2.03.24] - 2026-06-10
|
||||
|
||||
### Fixed
|
||||
- **Input level meter works without visiting General first.** The level meter
|
||||
was only started inside `_build_general()` and referenced `mic_level`
|
||||
unconditionally. If Input was opened first the meter never started. Now
|
||||
`_build_input()` also starts the meter when it isn't running yet, and both
|
||||
level bars (`mic_level` in General and `ww_mic_level` in Input) are updated
|
||||
defensively via `hasattr` so either tab can be visited in any order.
|
||||
|
||||
## [2.03.23] - 2026-06-10
|
||||
|
||||
### Fixed
|
||||
- **Wakeword results table actually resizable.** The controls pane is now
|
||||
wrapped in a ScrolledWindow with `shrink=True`, so dragging the divider
|
||||
upward collapses the controls and expands the table freely.
|
||||
|
||||
## [2.03.22] - 2026-06-10
|
||||
|
||||
### Changed
|
||||
- **Wakeword benchmark uses a split pane.** The TTS config / engine selector
|
||||
controls sit in the top pane; the results table sits in the bottom pane.
|
||||
Drag the divider to give the table as much vertical space as needed.
|
||||
|
||||
## [2.03.21] - 2026-06-10
|
||||
|
||||
### Added
|
||||
- **Wakeword results table: sortable columns.** Click any column header to sort
|
||||
ascending/descending. Numeric columns (Detected, Total, Recall %, False fires,
|
||||
Time) sort numerically.
|
||||
- **Wakeword results table: CSV export.** "Copy as CSV" copies the table to the
|
||||
clipboard; "Save CSV…" opens a file chooser to write a `.csv` file.
|
||||
|
||||
## [2.03.20] - 2026-06-10
|
||||
|
||||
### Added
|
||||
- **Wakeword benchmark results table.** Results are now shown in a TreeView
|
||||
with one row per engine per voice: Engine | Wakeword | Voice | Detected |
|
||||
Total | Recall % | False fires | Time. Rows are colour-coded green/orange/red
|
||||
by recall. An aggregate "ALL (N voices)" row is appended per engine.
|
||||
|
||||
### Fixed
|
||||
- **Section header icons now vertically centred with the headline text.**
|
||||
The `.bt-section` CSS class was inadvertently applied to the icon widget,
|
||||
giving it a 14 px top margin and pushing it down. The image no longer
|
||||
receives that class; a `set_pixel_size(14)` pin ensures consistent sizing.
|
||||
|
||||
## [2.03.19] - 2026-06-10
|
||||
|
||||
### Added
|
||||
- **Wakeword engine checkboxes in benchmark.** A row of checkboxes above the
|
||||
"Run wakeword benchmark" button lets you pick which engines to include.
|
||||
All are checked by default.
|
||||
- **Wakeword model selector in benchmark.** A "Wakeword" combo lets you
|
||||
override which wakeword phrase (model) to test. Leave empty for the default
|
||||
(each engine uses its own configured model). Pick a specific model (e.g.
|
||||
`okay_computer`) to test that phrase on every selected engine.
|
||||
|
||||
## [2.03.18] - 2026-06-10
|
||||
|
||||
### Fixed
|
||||
- **TTS model dropdown no longer floods with voice names.** Servers like Kokoro
|
||||
expose each voice as a `/models` entry. The ⟳ button now detects this case
|
||||
and skips filling the model combo, prompting the user to type the model id
|
||||
manually (e.g. `kokoro`). The status line shows "type model id manually" as
|
||||
a hint.
|
||||
|
||||
### Changed
|
||||
- **Wakeword benchmark runs across all engines and shows per-engine results.**
|
||||
Previously a callback signature mismatch caused the benchmark to crash when
|
||||
more than one engine was configured. Now progress shows `[1/3] engine name`,
|
||||
and the results panel lists Recall / False fires / time per engine.
|
||||
|
||||
## [2.03.17] - 2026-06-10
|
||||
|
||||
### Added
|
||||
- **Wakeword model fetch feedback.** The ⟳ button now shows a status line while
|
||||
connecting; after loading it reports how many models were found (with their
|
||||
names) or "Unreachable" if the server is down.
|
||||
- **Wakeword Quickstart covers all four ports.** The Quickstart menu now lists
|
||||
presets for ports 10400–10403, plus `hey_jarvis` and `alexa` variants.
|
||||
- **Wakeword info box.** An info banner explains how wyoming-openwakeword works,
|
||||
where to put model files, and lists the common built-in models.
|
||||
|
||||
## [2.03.16] - 2026-06-10
|
||||
|
||||
### Added
|
||||
- **MP3/OGG/FLAC support for sound cues.** The sound file picker now accepts
|
||||
WAV, MP3, OGG, FLAC, M4A, AAC, AIFF, and Opus. Playback uses `ffplay` or
|
||||
`gst-play-1.0` as a universal fallback when the native `pw-play`/`paplay`
|
||||
can't handle the format.
|
||||
- **Browse dialog with auto-preview.** The 📁 browse button opens a
|
||||
`FileChooserDialog`; selecting a file auto-plays it so you can hear it before
|
||||
confirming. The ▶ play button still works on the current selection.
|
||||
|
||||
## [2.03.15] - 2026-06-10
|
||||
|
||||
### Added
|
||||
- **Wakeword engine CRUD.** The wakeword server section now has the same full
|
||||
management UI as STT engines: a named-preset selector combo, + Add, Quickstart
|
||||
(with 4 common wyoming-openwakeword templates), ⟳ reload, and Delete. Existing
|
||||
users are migrated: their `wakeword_uri` / `wakeword_model` become the first
|
||||
preset automatically.
|
||||
|
||||
## [2.03.14] - 2026-06-10
|
||||
|
||||
### Fixed
|
||||
- **Engines tab.** Removed the "Internal engine — device & precision" section
|
||||
header. The Device and Compute type fields already only appear when a local
|
||||
engine type is selected; the separate header was redundant.
|
||||
|
||||
## [2.03.13] - 2026-06-10
|
||||
|
||||
### Changed
|
||||
- **Settings header bar.** Save and Save & Restart moved from the bottom button
|
||||
bar into the title bar (GTK HeaderBar). The X button closes without saving.
|
||||
Bottom button row removed.
|
||||
- **Section icon alignment.** Icons in section headers are now vertically
|
||||
centred with the label text (`SMALL_TOOLBAR` size, `valign=CENTER`).
|
||||
|
||||
## [2.03.12] - 2026-06-10
|
||||
|
||||
### Added
|
||||
- **Icons in settings.** All tab labels (Presets, Engines, Input, General,
|
||||
Benchmark, Log, Manual, About) and every section header inside each tab now
|
||||
show a small GTK symbolic icon, making the layout easier to scan.
|
||||
|
||||
### Fixed
|
||||
- **Resize grip position.** The grip indicator now appears correctly at the
|
||||
bottom-right corner below the notebook, not misplaced in the tab bar.
|
||||
|
||||
## [2.03.11] - 2026-06-09
|
||||
|
||||
### Added
|
||||
- **Resize grip indicator.** A dotted SE-corner grip is drawn over the
|
||||
bottom-right of the settings window so users discover it is resizable.
|
||||
|
||||
## [2.03.10] - 2026-06-09
|
||||
|
||||
### Fixed
|
||||
- **Server RAM probe.** Prometheus `/metrics` is almost always at the server
|
||||
root (`http://host:port/metrics`), not under `/v1`. Now tries the root URL
|
||||
first before falling back to the API base path.
|
||||
|
||||
## [2.03.09] - 2026-06-09
|
||||
|
||||
### Added
|
||||
- **Server RAM in benchmark.** For remote/Docker STT engines the benchmark now
|
||||
probes the server's Prometheus `/metrics` endpoint for
|
||||
`process_resident_memory_bytes` (standard Python/Go exporter) or
|
||||
`container_memory_rss` (cAdvisor) and shows the server-side RSS in MB in the
|
||||
RAM column. Falls back to `server` when the endpoint is not exposed.
|
||||
|
||||
## [2.03.08] - 2026-06-09
|
||||
|
||||
### Fixed
|
||||
- **Engines tab layout.** The "Internal engine — device & precision" section is
|
||||
now hidden when a remote (Server) or streaming engine type is selected —
|
||||
removing the confusing whitespace gap and irrelevant device controls for
|
||||
non-local engines.
|
||||
|
||||
## [2.03.01] - 2026-06-09
|
||||
|
||||
### Added
|
||||
- **RAM usage column in benchmark.** The results table now shows a **RAM (MB)**
|
||||
column — the increase in process RSS while the transcription ran. For local
|
||||
models this captures the memory cost of loading the model on first use; for
|
||||
remote engines it shows `—` (work happens server-side). Values are measured via
|
||||
`/proc/self/status` (VmRSS), so they reflect actual resident memory, not
|
||||
virtual address space.
|
||||
|
||||
## [2.03.00] - 2026-06-09
|
||||
|
||||
### Fixed
|
||||
- **"Not responding" / system instability on Save.** `_collect()` was calling
|
||||
`socket.create_connection()` *synchronously* on the GTK main thread when
|
||||
wakeword is enabled — freezing the UI for up to 1.5 s (longer if DNS is slow).
|
||||
The check is now done on a daemon thread and the result is logged instead of
|
||||
blocking the save path.
|
||||
- **GTK thread-safety crash in wakeword model load.** `_ww_load()` read
|
||||
`self.ww_uri.get_text()` from inside a background thread — unsafe. The URI is
|
||||
now captured on the main thread before the thread is spawned.
|
||||
- **HTTP 404 with WhisperX and other non-standard endpoints.** The remote
|
||||
transcription call always appended `/audio/transcriptions` to the base URL, but
|
||||
services like WhisperX use `/transcribe` as the full path. The URL path is now
|
||||
inspected: if it is anything other than empty / `/v1` / `/v1.0`, the URL is
|
||||
used as the complete endpoint with nothing appended — so
|
||||
`http://host:8081/transcribe` works out of the box.
|
||||
- **Log levels.** `logbuffer` now stores `(timestamp, level, message)` tuples and
|
||||
accepts a `level=` keyword (`DEBUG` / `INFO` / `WARNING` / `ERROR`). The Log
|
||||
tab gains a **Level** dropdown (Verbose · Info · Warning · Error) that filters
|
||||
the displayed entries live. Wakeword and socket errors are now tagged
|
||||
`WARNING`; library records are forwarded at their native level.
|
||||
|
||||
### Added
|
||||
- **Wakeword server preset dropdown** (Input → Hands-free wakeword). A
|
||||
**Server preset** combo lists all configured wakeword server engines by name.
|
||||
Picking one auto-fills the URI and model fields and re-probes reachability.
|
||||
The selection is persisted as `wakeword_active` in config.
|
||||
|
||||
## [2.02.03] - 2026-06-09
|
||||
|
||||
### Added
|
||||
- Wakeword server preset dropdown in Input tab.
|
||||
|
||||
## [2.02.02] - 2026-06-09
|
||||
|
||||
### Fixed
|
||||
- License tab now renders with markdown styling.
|
||||
- Benchmark pane minimum height (320 px, `shrink=False`) prevents the engine
|
||||
list or results table from collapsing to zero when the window is small.
|
||||
|
||||
## [2.02.01] - 2026-06-09
|
||||
|
||||
### Added
|
||||
- Last benchmark time and accuracy shown on the selected STT engine in the
|
||||
Engines tab. Persisted to config so it survives restarts.
|
||||
|
||||
## [2.02.00] - 2026-06-09
|
||||
|
||||
### Added
|
||||
- **Language metadata in benchmark.** The engine checkbox list shows supported
|
||||
language codes next to each engine (fetched async). Filter box searches by
|
||||
language code. Results table has a **Lang** column. Data comes from the
|
||||
`/v1/models` `language` field (faster-whisper-server) or NVIDIA NIM `/metadata`.
|
||||
|
||||
## [1.9.5] - 2026-06-09
|
||||
|
||||
### Added
|
||||
- **Emoji picker search.** A search field at the top of the emoji picker filters
|
||||
all categories in real time using Unicode character names (e.g. "fire", "dog",
|
||||
"heart"). Typing hides the category bar and shows matching results; clearing
|
||||
restores the category view.
|
||||
|
||||
### Fixed
|
||||
- **Manual tab now shows content.** `MANUAL.md` is copied next to the package
|
||||
module so the Manual tab finds it in both venv and deb installs.
|
||||
- **Info banner no longer bright blue.** The `.bt-infobox` background now uses
|
||||
a neutral 5 % tint of the foreground colour instead of the theme accent
|
||||
colour, so text stays readable on any theme.
|
||||
|
||||
## [1.9.4] - 2026-06-09
|
||||
|
||||
### Changed
|
||||
- **Settings UI completely redesigned.** All six settings tabs (Presets, Engines,
|
||||
Input, General, Input, General) now use a card-based layout following GTK3 best
|
||||
practices: related fields are grouped inside visually distinct cards with bold
|
||||
section titles. CSS is injected at start-up to give cards a consistent rounded
|
||||
border (`boxed-list` + `bt-card`) and a styled info banner at the top of each
|
||||
tab.
|
||||
- **Dialog is larger (740×700 px) and every tab scrolls.** The notebook pages
|
||||
now wrap their content in a `Gtk.ScrolledWindow` so no fields are ever clipped,
|
||||
even on small screens.
|
||||
- **Engines toolbar reorganised.** Creation actions (+ Add, + Stream, Quickstart)
|
||||
are left-aligned; destructive/status actions (Delete, Test, ⟳) are
|
||||
right-aligned via `pack_end`, making the bar scannable at a glance.
|
||||
- **Section titles replace plain separators.** The old `Gtk.Separator` +
|
||||
unstyled `Gtk.Label` pattern is gone; every section now has a small, dimmed,
|
||||
bold all-caps header rendered with markup.
|
||||
- **Cleaner section names.** "WW - Wakeword (Hands-free)" → "Hands-free
|
||||
wakeword"; "Audio cues (manual dictation)" → "Audio cues (keyboard / hotkey
|
||||
dictation)"; "Local engine … device & precision" → "Internal engine — device &
|
||||
precision".
|
||||
|
||||
## [1.9.3] - 2026-06-09
|
||||
|
||||
### Added
|
||||
- **ⓘ info buttons on every settings field.** Each field in every tab now has a
|
||||
small information icon that opens a plain-language help popover when clicked —
|
||||
so non-technical users can understand what each setting does without hovering
|
||||
or reading the manual.
|
||||
- **Manual tab in Settings.** A new "Manual" tab shows the full `MANUAL.md`
|
||||
reference doc directly inside the Settings window.
|
||||
- **Quickstart templates for engines.** A "Quickstart ▾" button in the STT and
|
||||
LLM engine toolbars opens a menu of common services (OpenAI, Groq, OpenRouter,
|
||||
Ollama, LM Studio, vLLM, llama-swap, faster-whisper-server, NVIDIA Riva) and
|
||||
pre-fills the form — one click to configure a provider.
|
||||
|
||||
### Changed
|
||||
- **Engine type names are now human-readable.** STT types now read "Internal —
|
||||
faster-whisper, runs inside the app", "Server — OpenAI-compatible API (LAN or
|
||||
cloud)", and "Realtime — NVIDIA Riva / NIM streaming" instead of the raw
|
||||
identifiers. LLM types read "LAN server — runs on your machine or local
|
||||
network" and "Cloud service — OpenAI, Groq, OpenRouter, …".
|
||||
- **Device selector now shows "GPU (CUDA)" instead of "cuda"**, and compute
|
||||
types have plain-language descriptions (e.g. "int8 — fast, less memory").
|
||||
|
||||
## [1.9.2] - 2026-06-09
|
||||
|
||||
### Added
|
||||
- **Emoji picker for preset icons.** The "Icon (emoji)" field in Settings →
|
||||
Presets now has a 😀 button that opens a scrollable emoji grid (60 common
|
||||
emojis across six categories). Click any emoji to insert it — or keep typing
|
||||
directly into the field as before.
|
||||
|
||||
## [1.9.1] - 2026-06-08
|
||||
|
||||
### Changed
|
||||
- **Settings opens instantly.** Each tab's contents are now built the first time
|
||||
you view it instead of all up front, so the dialog no longer pauses ~1.3s
|
||||
constructing the file-choosers in the Input/Benchmark tabs. Saving force-builds
|
||||
any tab you didn't visit first, so no field is ever missed.
|
||||
- **Connection dots moved beside their field.** The Wakeword and TTS reachability
|
||||
dots now sit just left of the URL entry (matching the Engines tab) instead of
|
||||
at the far right of the row.
|
||||
|
||||
### Fixed
|
||||
- **Settings could be opened more than once.** Choosing Settings while it's
|
||||
already open now raises the existing window instead of stacking a second copy.
|
||||
|
||||
## [1.9.0] - 2026-06-08
|
||||
|
||||
### Added
|
||||
- **Connection indicators** for remote endpoints. The **Wakeword engine** field
|
||||
(Input tab — renamed from "Wyoming URI" to read more generally) and the **TTS
|
||||
URL** field (Benchmark tab) now show a coloured dot: green when the server is
|
||||
reachable, red when it's configured but unreachable, grey when blank — mirroring
|
||||
the STT/LLM engine dots. It's a lightweight background TCP probe, refreshed when
|
||||
the dialog opens, when you press ⟳, and when you leave the field.
|
||||
|
||||
## [1.8.1] - 2026-06-08
|
||||
|
||||
### Fixed
|
||||
- **Settings dialog and control panel wouldn't open on some desktops.** When the
|
||||
gvfs `org.gtk.vfs.UDisks2VolumeMonitor` dbus service fails to activate (common
|
||||
on headless or minimal sessions), every `Gtk.FileChooserButton` blocked ~25s on
|
||||
a `StartServiceByName` timeout while realizing — so the Settings dialog never
|
||||
finished appearing, and the stalled GTK main loop froze the panel too. Blitztext
|
||||
now selects GIO's native `/proc/mounts` volume monitor
|
||||
(`GIO_USE_VOLUME_MONITOR=unix`) before any window is realized, so file choosers
|
||||
open instantly with no dbus dependency.
|
||||
|
||||
## [1.8.0] - 2026-06-08
|
||||
|
||||
### Added
|
||||
- **Send by voice**: say a distinctive phrase like **"computer send"** at the
|
||||
start or end of a clip and the word is stripped, then the rest is typed **and
|
||||
submitted with Enter** — the spoken equivalent of "stop + paste + Enter".
|
||||
Mainly for hands-free use, where you can't press a key. Configure under
|
||||
Settings → Input → "Send words", or `[routing] send_keywords`. Off by default;
|
||||
because it presses Enter, use a multi-word phrase (e.g. your wakeword + "send")
|
||||
so a sentence that merely ends in "send" doesn't submit by accident. Matched
|
||||
the same edge-anchored, ASR-tolerant way as routing/cancel keywords.
|
||||
- **Wakeword benchmark** (Settings → Benchmark): stress-test hands-free
|
||||
detection. It synthesizes short sentences with your wake phrase spoken in
|
||||
random voices (plus pure-filler utterances with none), streams them to your
|
||||
wyoming-openwakeword server, and reports **recall** (how reliably it fires),
|
||||
**false fires**, and a **per-voice** breakdown. Speech comes from any
|
||||
OpenAI-compatible TTS server (Kokoro-FastAPI, XTTS, OpenAI, …): set its URL,
|
||||
optional API-key env var, model, and voices under the new `[tts]` config / the
|
||||
Benchmark tab, and use **Connect** to test it (it auto-fills the voice list
|
||||
when the server exposes one).
|
||||
|
||||
## [1.7.1] - 2026-06-08
|
||||
|
||||
### Fixed
|
||||
- **Overlay waveform and silence countdown ring never appeared** on systems
|
||||
where PortAudio/`sounddevice` can't open the default input — notably PipeWire
|
||||
boxes, where opening an input stream simply hangs. Both the live waveform and
|
||||
the auto-stop countdown are driven by a single level meter, which was the only
|
||||
part of the app still using `sounddevice` (everything else records via
|
||||
`pw-record`). The meter now streams raw PCM from the **same system recorder as
|
||||
the WAV recorder** (`pw-record`/`parecord`/`arecord`) and computes the level
|
||||
itself, so it works wherever recording works — on both the hotkey and
|
||||
hands-free (wakeword) paths, plus the mic-level preview in Settings. No more
|
||||
PortAudio dependency for metering.
|
||||
- **App reported itself as "`__main__.py`"** in the taskbar and in GNOME's
|
||||
"… is not responding" dialog. Launched via `python -m blitztext`, GTK's default
|
||||
program name is `argv[0]`'s basename. It now sets `prgname`/application name to
|
||||
**Blitztext** before any window is realized (and the desktop file gains
|
||||
`StartupWMClass=blitztext` for the .desktop match + icon), without touching the
|
||||
`-m blitztext` entry point.
|
||||
|
||||
## [1.7.0] - 2026-06-07
|
||||
|
||||
### Added
|
||||
- **Spoken cancel keyword**: say a word like **"abbrechen"** (or "cancel") at the
|
||||
start or end of a clip and the whole dictation is **discarded** — it is never
|
||||
routed, rewritten, or typed anywhere. Mainly rescues an accidentally triggered
|
||||
(e.g. wakeword) recording. Configure under Settings → Mic/Cues → "Cancel words",
|
||||
or `[routing] cancel_keywords` (default `["abbrechen", "cancel"]`; empty list
|
||||
disables it). Matched the same edge-anchored, ASR-tolerant way as routing
|
||||
keywords, so the word buried mid-sentence won't trip it.
|
||||
|
||||
## [1.6.0] - 2026-06-07
|
||||
|
||||
### Fixed
|
||||
- **Session freeze when the overlay's caret tracking was active** (could lock up
|
||||
the whole GNOME/X11 desktop, forcing a logout/reboot). The AT-SPI caret tracker
|
||||
subscribed to the high-frequency `object:text-caret-moved` signal and made
|
||||
**synchronous, blocking AT-SPI reads from inside the event handler** — which
|
||||
re-enters the accessibility dispatcher and is stormed by the app's *own*
|
||||
`xdotool` typing (one event per character), congesting the a11y bus until the
|
||||
desktop stopped responding. It now tracks **focus changes only** and reads the
|
||||
caret rectangle lazily (once, when the overlay shows), never from inside an
|
||||
event dispatch.
|
||||
|
||||
### Changed
|
||||
- **Matched preset is fused into the overlay instead of a desktop notification**:
|
||||
when voice routing picks a preset, the overlay shows its emoji icon, name, and
|
||||
the spoken keyword on a banner, and narrates the phase ("Transcribing…" →
|
||||
"Rewriting…"). With the overlay on, the redundant per-dictation notifications
|
||||
are suppressed (errors still notify); headless/overlay-off keeps notifications.
|
||||
|
||||
### Added
|
||||
- **Live LLM rewrite in the overlay**: rewrite presets now stream the model's
|
||||
output into the bubble token-by-token, so you watch it write. The delivered
|
||||
text is still the complete result, typed once the rewrite finishes.
|
||||
|
||||
## [1.5.1] - 2026-06-07
|
||||
|
||||
### Added
|
||||
- **Silence auto-stop countdown ring** on the dictation overlay: when you stop
|
||||
speaking, a full circle wrapping the microphone glyph drains clockwise as the
|
||||
trailing-silence timer runs out, recolouring from calm cyan to an urgent red
|
||||
and emptying exactly as the recording auto-stops. It spans the configured
|
||||
"Silence to stop (s)" window (`[wakeword] silence_seconds`), fades back in/out
|
||||
as you pause and resume, and so finally makes the hands-free auto-stop visible
|
||||
instead of a silent surprise.
|
||||
|
||||
## [1.5.0] - 2026-06-07
|
||||
|
||||
### Added
|
||||
- **On-screen dictation overlay** (Settings → General → "Visual overlay", or
|
||||
`[general] overlay_enabled`, default on): the moment recording starts — by
|
||||
hotkey **or** wakeword — a translucent bubble appears at the cursor showing a
|
||||
pulsing **microphone**, a **live waveform** of your mic level, and the
|
||||
**recognised text** (word-by-word with a realtime streaming STT engine, or the
|
||||
final result as a brief confirmation otherwise). Its tail points at where the
|
||||
text will land: it follows the **text caret** when the focused app exposes it
|
||||
over accessibility (AT-SPI), otherwise the **mouse pointer**, otherwise a
|
||||
screen corner — tune via `[general] overlay_anchor = "caret" | "pointer" |
|
||||
"corner"`. The window is click-through and never takes focus, and it finally
|
||||
gives **hands-free wakeword sessions** visible feedback (their notifications
|
||||
are suppressed by design). X11 only; falls back to a corner where the cursor
|
||||
can't be located.
|
||||
|
||||
### Changed
|
||||
- **Presets are speakable by name**: voice routing now matches a preset's *name*
|
||||
as an implicit keyword, so a preset works by voice even with no keywords
|
||||
configured (e.g. just say "nicer email …"). Explicit keywords still take
|
||||
precedence, and preset names also bias the STT for better recognition.
|
||||
- **General settings switches** moved to the far right of each row, each with an
|
||||
inline description so it's clear what the toggle does without hovering.
|
||||
- **About**: added a "Copyright: 2026 mARTin Bierschenk - Design" line.
|
||||
|
||||
## [1.4.0] - 2026-06-07
|
||||
|
||||
### Added
|
||||
- **"Announce matched preset" notification** (Settings → General, or
|
||||
`[general] notify_routing`, default on): after a voice command, a notification
|
||||
shows which preset and spoken keyword matched — **shown even for hands-free
|
||||
wakeword sessions**, so you can see what you triggered. It only fires on a real
|
||||
match, so it never spams when nothing is said.
|
||||
- **Per-preset emoji icon** (Presets → "Icon (emoji)"): give each preset a
|
||||
distinct emoji, shown in the matched-preset notification so you can tell at a
|
||||
glance which fired.
|
||||
|
||||
### Fixed
|
||||
- **Voice-routing default went to a rewrite**: when no `[routing] default` preset
|
||||
is set, the no-keyword fallback used the *first* preset — which, if that happened
|
||||
to be an LLM rewrite (e.g. "Improve text"), sent every unrouted wakeword command
|
||||
to the language model (and failed when the LLM was down). The fallback now
|
||||
prefers a `transcribe` preset, so the default action is plain transcription.
|
||||
|
||||
## [1.3.0] - 2026-06-07
|
||||
|
||||
### Added
|
||||
@ -157,7 +855,9 @@ into that field.
|
||||
AppIndicator typelibs and GNOME `ubuntu-appindicators` extension are already
|
||||
present on the target host).
|
||||
|
||||
[Unreleased]: https://github.com/mARTin-B78/blitztext-app-linux/compare/v1.1.0...HEAD
|
||||
[Unreleased]: https://github.com/mARTin-B78/blitztext-app-linux/compare/v1.5.1...HEAD
|
||||
[1.5.1]: https://github.com/mARTin-B78/blitztext-app-linux/compare/v1.5.0...v1.5.1
|
||||
[1.5.0]: https://github.com/mARTin-B78/blitztext-app-linux/compare/v1.4.0...v1.5.0
|
||||
[1.1.0]: https://github.com/mARTin-B78/blitztext-app-linux/compare/v1.0.1...v1.1.0
|
||||
[1.0.1]: https://github.com/mARTin-B78/blitztext-app-linux/compare/v1.0.0...v1.0.1
|
||||
[1.0.0]: https://github.com/mARTin-B78/blitztext-app-linux/releases/tag/v1.0.0
|
||||
|
||||
115
linux/README.md
@ -13,9 +13,13 @@ experiment along those lines was moved out to
|
||||
the optional rewrite step calls out to an LLM.
|
||||
|
||||
<p align="center">
|
||||
<img src="../Screenshots/panel.png" alt="Blitztext control panel" width="380">
|
||||
<img src="../Screenshots/main-panel.png" alt="Blitztext control panel" width="360">
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<img src="../Screenshots/overlay-listening.png" alt="On-screen overlay while listening" width="360">
|
||||
|
||||
<img src="../Screenshots/tray-menu.png" alt="Blitztext system-tray menu" width="300">
|
||||
<img src="../Screenshots/overlay-result.png" alt="On-screen overlay showing transcription result" width="360">
|
||||
</p>
|
||||
|
||||
## Inspiration
|
||||
@ -39,49 +43,112 @@ Each normal hotkey **toggles**: press to start recording, press again to stop
|
||||
then it transcribes, optionally rewrites, and types the result where your cursor
|
||||
is. Streaming workflows type stable words live while you speak.
|
||||
|
||||
**Cancel by voice:** say *"abbrechen"* (or *"cancel"*) at the start or end of a
|
||||
clip and the whole dictation is discarded — never routed, rewritten, or typed.
|
||||
It's the rescue for an accidentally triggered (e.g. wakeword) recording. Set the
|
||||
words under `[routing] cancel_keywords` (default `["abbrechen", "cancel"]`; an
|
||||
empty list turns it off).
|
||||
|
||||
**Send by voice:** say a distinctive phrase like *"computer send"* at the start
|
||||
or end of a clip and the word is stripped, then the rest is typed **and submitted
|
||||
with Enter** — the spoken equivalent of "stop + paste + Enter", ideal hands-free.
|
||||
Off by default; set the phrases under `[routing] send_keywords` (use a multi-word
|
||||
phrase so a sentence merely ending in "send" doesn't submit by accident).
|
||||
|
||||
While you dictate, an optional **on-screen overlay** (Settings → General →
|
||||
"Visual overlay", default on) shows a translucent bubble at the cursor with a
|
||||
pulsing microphone, a live waveform of your mic level, and the recognised text —
|
||||
word-by-word in streaming mode, or the final result as a brief confirmation. Its
|
||||
tail points at the text caret (via AT-SPI accessibility) or the mouse pointer; it
|
||||
is click-through, never steals focus, and also gives hands-free wakeword sessions
|
||||
visible feedback. Tune the anchor with `[general] overlay_anchor`. X11 only.
|
||||
|
||||
## Screenshots
|
||||
|
||||
Everything is configured in the **Settings** window — every tab has tooltips and
|
||||
screen-reader (ATK) support. Click any image to open it full size.
|
||||
Everything is configured in the **Settings** window — the sidebar gives quick
|
||||
access to every page. Click any image to open it full size.
|
||||
|
||||
### Main panel & overlay
|
||||
|
||||
<p align="center">
|
||||
<a href="../Screenshots/settings-presets.png"><img src="../Screenshots/settings-presets.png" alt="Presets settings tab" width="100%"></a><br>
|
||||
<em><b>Presets</b> — your dictation actions. Each preset is either a plain transcription or an LLM rewrite, and carries its own spoken keyword(s) for voice routing, an optional global hotkey, and a custom rewrite prompt.</em>
|
||||
<a href="../Screenshots/main-panel.png"><img src="../Screenshots/main-panel.png" alt="Blitztext main panel" width="46%"></a>
|
||||
|
||||
<a href="../Screenshots/overlay-listening.png"><img src="../Screenshots/overlay-listening.png" alt="Overlay — listening" width="46%"></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<em><b>Left:</b> Control panel listing all presets with icons, descriptions, and hotkeys.</em>
|
||||
|
||||
<em><b>Right:</b> On-screen overlay showing the live waveform while listening.</em>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="../Screenshots/settings-engines.png"><img src="../Screenshots/settings-engines.png" alt="Engines settings tab" width="100%"></a><br>
|
||||
<em><b>Engines</b> — your speech-to-text and language-model back-ends, local or remote. Add and rename engines, watch live online/offline status, and pick models from a searchable list fetched straight from the endpoint.</em>
|
||||
<a href="../Screenshots/overlay-result.png"><img src="../Screenshots/overlay-result.png" alt="Overlay — transcription result" width="46%"></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<em>Overlay after transcription — preset name and recognised text appear at the cursor.</em>
|
||||
</p>
|
||||
|
||||
### Settings — General & Input
|
||||
|
||||
<p align="center">
|
||||
<a href="../Screenshots/settings-presets.png"><img src="../Screenshots/settings-presets.png" alt="Presets page" width="48%"></a>
|
||||
|
||||
<a href="../Screenshots/settings-general.png"><img src="../Screenshots/settings-general.png" alt="General page" width="48%"></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<em><b>Presets</b> — dictation actions with keywords, hotkeys, LLM mode, and custom prompts.</em>
|
||||
|
||||
<em><b>General</b> — microphone, output mode, language hint, notifications, overlay, autostart.</em>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="../Screenshots/settings-input.png"><img src="../Screenshots/settings-input.png" alt="Input settings tab" width="100%"></a><br>
|
||||
<em><b>Input</b> — how you start and stop dictation: the modifier-key scheme (Ctrl+Win / Ctrl / Alt / Esc) or custom hotkeys, plus the silence-based auto-stop (VAD), the quality gate, and audio cues.</em>
|
||||
<a href="../Screenshots/settings-keyboard.png"><img src="../Screenshots/settings-keyboard.png" alt="Keyboard page" width="48%"></a>
|
||||
|
||||
<a href="../Screenshots/settings-wakeword.png"><img src="../Screenshots/settings-wakeword.png" alt="Wakeword page" width="48%"></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<em><b>Keyboard</b> — modifier-key scheme or direct hotkeys, quality gate, audio cues.</em>
|
||||
|
||||
<em><b>Wakeword</b> — hands-free dictation via a Wyoming/openWakeWord server, with live level meter and model picker.</em>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="../Screenshots/wakeword.png"><img src="../Screenshots/wakeword.png" alt="Wakeword (hands-free) settings" width="100%"></a><br>
|
||||
<em><b>Wakeword (hands-free)</b> — point Blitztext at a Wyoming/openWakeWord server, choose a wake model, and test the connection live so a spoken keyword starts dictation with no keys at all.</em>
|
||||
</p>
|
||||
### Settings — Engines
|
||||
|
||||
<p align="center">
|
||||
<a href="../Screenshots/settings-general.png"><img src="../Screenshots/settings-general.png" alt="General settings tab" width="100%"></a><br>
|
||||
<em><b>General</b> — core preferences: microphone with a live level meter, output mode (type vs. paste), language hint, type delay, and autostart on login.</em>
|
||||
<a href="../Screenshots/settings-stt-engines.png"><img src="../Screenshots/settings-stt-engines.png" alt="STT Engines page" width="48%"></a>
|
||||
|
||||
<a href="../Screenshots/settings-llm-engines.png"><img src="../Screenshots/settings-llm-engines.png" alt="LLM Engines page" width="48%"></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<em><b>STT Engines</b> — speech-to-text back-ends (local faster-whisper, OpenAI-compatible server, or Riva realtime), with green/red status dot and Test button.</em>
|
||||
|
||||
<em><b>LLM Engines</b> — language-model back-ends for text rewriting (LAN server or cloud service).</em>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="../Screenshots/settings-benchmark.png"><img src="../Screenshots/settings-benchmark.png" alt="Benchmark settings tab" width="100%"></a><br>
|
||||
<em><b>Benchmark</b> — compare every configured STT engine against a reference WAV + transcript to find the fastest and most accurate, with a Device column (CPU / GPU / remote).</em>
|
||||
</p>
|
||||
### Settings — Benchmark
|
||||
|
||||
<p align="center">
|
||||
<a href="../Screenshots/settings-log.png"><img src="../Screenshots/settings-log.png" alt="Log settings tab" width="100%"></a><br>
|
||||
<em><b>Log</b> — the in-app log buffer: a live view of recording, transcription, routing, and wakeword events for quick troubleshooting.</em>
|
||||
<a href="../Screenshots/settings-benchmark-stt.png"><img src="../Screenshots/settings-benchmark-stt.png" alt="Benchmark — STT page" width="48%"></a>
|
||||
|
||||
<a href="../Screenshots/settings-benchmark-wakeword.png"><img src="../Screenshots/settings-benchmark-wakeword.png" alt="Benchmark — Wakeword page" width="48%"></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<em><b>Benchmark — STT</b> — compare engines against a reference WAV + transcript; table shows speed, accuracy, device, and language support.</em>
|
||||
|
||||
<em><b>Benchmark — Wakeword</b> — stress-test wakeword detection via a TTS server, reporting recall and false-fire rates per voice.</em>
|
||||
</p>
|
||||
|
||||
### Settings — Log & About
|
||||
|
||||
<p align="center">
|
||||
<a href="../Screenshots/settings-about.png"><img src="../Screenshots/settings-about.png" alt="About settings tab" width="100%"></a><br>
|
||||
<em><b>About</b> — version, source link, changelog, and licence.</em>
|
||||
<a href="../Screenshots/settings-log.png"><img src="../Screenshots/settings-log.png" alt="Log page" width="48%"></a>
|
||||
|
||||
<a href="../Screenshots/settings-about.png"><img src="../Screenshots/settings-about.png" alt="About page" width="48%"></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<em><b>Log</b> — live activity log for recording, transcription, routing, and wakeword events.</em>
|
||||
|
||||
<em><b>About</b> — version, source link, inline changelog, and licence.</em>
|
||||
</p>
|
||||
|
||||
## Requirements
|
||||
|
||||
@ -6,4 +6,4 @@ counterpart to the macOS Blitztext menu bar app: it runs natively on the host
|
||||
(not in a container) so it can type into any application via xdotool.
|
||||
"""
|
||||
|
||||
__version__ = "1.3.0"
|
||||
__version__ = "2.03.41"
|
||||
|
||||
@ -1,43 +1,39 @@
|
||||
"""Audio helpers: enumerate input devices and a live input-level meter.
|
||||
|
||||
Mic enumeration uses pactl (PipeWire/PulseAudio source names, which pw-record
|
||||
and parecord accept via --target/-d). The level meter uses sounddevice to read
|
||||
the chosen input and report a 0..1 level to a callback.
|
||||
and parecord accept via --target/-d). The level meter shells out to the same
|
||||
system recorder the app uses (pw-record/parecord/arecord), reading raw PCM from
|
||||
its stdout and reporting a 0..1 level to a callback — no Python audio binding,
|
||||
so it works wherever the recorder does (PortAudio/sounddevice can't open the
|
||||
default input on some PipeWire systems).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
|
||||
from .recorder import detect_recorder
|
||||
|
||||
@contextlib.contextmanager
|
||||
def _quiet_c_stderr():
|
||||
"""Silence chatter written directly to fd 2 by C libraries.
|
||||
# Raw-PCM (s16le, 16 kHz mono) variants of the recorders, streamed to stdout so
|
||||
# we can RMS each chunk directly. Mirrors recorder.py's WAV commands but emits
|
||||
# headerless PCM. pw-record/parecord default to stdout; arecord uses "-t raw".
|
||||
_METER_ARGV: dict[str, list[str]] = {
|
||||
"pw-record": ["pw-record", "--rate=16000", "--channels=1", "--format=s16", "-"],
|
||||
"parecord": ["parecord", "--rate=16000", "--channels=1", "--format=s16le"],
|
||||
"arecord": ["arecord", "-q", "-f", "S16_LE", "-r", "16000", "-c", "1", "-t", "raw"],
|
||||
}
|
||||
|
||||
PortAudio/ALSA print harmless thread-teardown noise
|
||||
("pthread_join ... failed", "PaUnixThread_Terminate ... failed") straight to
|
||||
the underlying stderr file descriptor, which Python-level redirection can't
|
||||
catch. We briefly point fd 2 at /dev/null around the offending call.
|
||||
"""
|
||||
try:
|
||||
stderr_fd = sys.stderr.fileno()
|
||||
except (AttributeError, ValueError, OSError):
|
||||
yield # No real stderr fd (already captured/redirected) — nothing to do.
|
||||
return
|
||||
saved_fd = os.dup(stderr_fd)
|
||||
devnull_fd = os.open(os.devnull, os.O_WRONLY)
|
||||
try:
|
||||
os.dup2(devnull_fd, stderr_fd)
|
||||
yield
|
||||
finally:
|
||||
os.dup2(saved_fd, stderr_fd)
|
||||
os.close(devnull_fd)
|
||||
os.close(saved_fd)
|
||||
# How to point each recorder at a specific pactl/pipewire source (mirrors
|
||||
# recorder._DEVICE_FLAG; arecord uses ALSA names, so it stays on the default).
|
||||
_DEVICE_FLAG: dict[str, list[str]] = {
|
||||
"pw-record": ["--target"],
|
||||
"parecord": ["-d"],
|
||||
"arecord": [],
|
||||
}
|
||||
|
||||
_CHUNK_BYTES = 3200 # 100 ms of 16 kHz, 16-bit, mono → ~10 Hz level updates
|
||||
|
||||
|
||||
def list_mics() -> list[tuple[str, str]]:
|
||||
@ -68,58 +64,79 @@ def list_mics() -> list[tuple[str, str]]:
|
||||
|
||||
|
||||
class LevelMeter:
|
||||
"""Open the given input device and call `on_level(0..1)` periodically."""
|
||||
"""Stream mic audio via a system recorder and call `on_level(0..1)` ~10x/s.
|
||||
|
||||
def __init__(self, device: str = "", on_level=None):
|
||||
self.device = device or None
|
||||
Uses pw-record/parecord/arecord (the same recorders as the WAV recorder)
|
||||
rather than a Python audio binding, so it works on PipeWire boxes where
|
||||
PortAudio can't open the default input. Best-effort: ``start()`` returns
|
||||
False if no recorder is available or the device can't be opened.
|
||||
"""
|
||||
|
||||
def __init__(self, device: str = "", on_level=None, recorder: str = "auto", on_chunk=None):
|
||||
self.device = device or ""
|
||||
self.on_level = on_level
|
||||
self._stream = None
|
||||
self._lock = threading.Lock()
|
||||
self.on_chunk = on_chunk # optional: called with raw s16le PCM bytes each chunk
|
||||
self._recorder = recorder
|
||||
self._proc: subprocess.Popen | None = None
|
||||
self._thread: threading.Thread | None = None
|
||||
self._stop = threading.Event()
|
||||
|
||||
def _argv(self, recorder: str) -> list[str]:
|
||||
argv = list(_METER_ARGV[recorder])
|
||||
flag = _DEVICE_FLAG.get(recorder, [])
|
||||
if self.device and flag:
|
||||
argv += flag + [self.device]
|
||||
return argv
|
||||
|
||||
def start(self) -> bool:
|
||||
import numpy as np
|
||||
import sounddevice as sd
|
||||
try:
|
||||
recorder = detect_recorder(self._recorder)
|
||||
except RuntimeError:
|
||||
return False
|
||||
if recorder not in _METER_ARGV:
|
||||
return False
|
||||
try:
|
||||
self._proc = subprocess.Popen(
|
||||
self._argv(recorder), stdout=subprocess.PIPE, stderr=subprocess.DEVNULL
|
||||
)
|
||||
except OSError:
|
||||
self._proc = None
|
||||
return False
|
||||
self._stop.clear()
|
||||
self._thread = threading.Thread(target=self._loop, daemon=True, name="LevelMeter")
|
||||
self._thread.start()
|
||||
return True
|
||||
|
||||
def _cb(indata, _frames, _time, _status):
|
||||
level = float(np.sqrt(np.mean(np.square(indata)))) if indata.size else 0.0
|
||||
def _loop(self) -> None:
|
||||
import numpy as np
|
||||
|
||||
proc = self._proc
|
||||
if proc is None or proc.stdout is None:
|
||||
return
|
||||
try:
|
||||
while not self._stop.is_set() and proc.poll() is None:
|
||||
chunk = proc.stdout.read(_CHUNK_BYTES)
|
||||
if not chunk:
|
||||
break
|
||||
if self.on_chunk:
|
||||
self.on_chunk(chunk)
|
||||
samples = np.frombuffer(chunk, dtype=np.int16).astype(np.float32) / 32768.0
|
||||
level = float(np.sqrt(np.mean(np.square(samples)))) if samples.size else 0.0
|
||||
if self.on_level:
|
||||
# Scale RMS (typically small) into a usable 0..1 range.
|
||||
self.on_level(min(1.0, level * 12.0))
|
||||
|
||||
try:
|
||||
with _quiet_c_stderr():
|
||||
self._stream = sd.InputStream(
|
||||
samplerate=16000, channels=1, dtype="float32",
|
||||
blocksize=1600, device=self._resolve_device(), callback=_cb,
|
||||
)
|
||||
self._stream.start()
|
||||
return True
|
||||
except Exception: # noqa: BLE001 - device may be busy/unavailable
|
||||
self._stream = None
|
||||
return False
|
||||
|
||||
def _resolve_device(self):
|
||||
# sounddevice wants an index/name it knows; pactl names rarely match, so
|
||||
# fall back to the default input when the name isn't resolvable.
|
||||
if not self.device:
|
||||
return None
|
||||
try:
|
||||
import sounddevice as sd
|
||||
|
||||
for i, d in enumerate(sd.query_devices()):
|
||||
if d["max_input_channels"] > 0 and self.device in d["name"]:
|
||||
return i
|
||||
except Exception: # noqa: BLE001
|
||||
except Exception: # noqa: BLE001 - metering is eye-candy; never crash the app
|
||||
pass
|
||||
return None
|
||||
|
||||
def stop(self) -> None:
|
||||
with self._lock:
|
||||
if self._stream is not None:
|
||||
self._stop.set()
|
||||
proc, self._proc = self._proc, None
|
||||
if proc is not None and proc.poll() is None:
|
||||
proc.terminate()
|
||||
try:
|
||||
# PortAudio/ALSA spews thread-teardown noise to fd 2 here.
|
||||
with _quiet_c_stderr():
|
||||
self._stream.stop()
|
||||
self._stream.close()
|
||||
finally:
|
||||
self._stream = None
|
||||
proc.wait(timeout=1.0)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
thread, self._thread = self._thread, None
|
||||
if thread is not None:
|
||||
thread.join(timeout=1.0)
|
||||
|
||||
@ -50,6 +50,7 @@ def set_enabled(enabled: bool) -> None:
|
||||
f"Exec={_exec_command()}\n"
|
||||
"Icon=blitztext\n"
|
||||
"Terminal=false\n"
|
||||
"X-GNOME-Autostart-enabled=true\n",
|
||||
"X-GNOME-Autostart-enabled=true\n"
|
||||
"X-GNOME-Autostart-Delay=12\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
@ -7,6 +7,7 @@ Benchmark tab to find the fastest and most accurate engine/model.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
@ -15,16 +16,33 @@ from . import stt
|
||||
from .routing import normalize
|
||||
|
||||
|
||||
def _rss_mb() -> float:
|
||||
"""Current process RSS in MB via /proc/self/status (Linux only)."""
|
||||
try:
|
||||
with open("/proc/self/status") as fh:
|
||||
for line in fh:
|
||||
if line.startswith("VmRSS:"):
|
||||
return int(line.split()[1]) / 1024.0 # kB → MB
|
||||
except OSError:
|
||||
pass
|
||||
return 0.0
|
||||
|
||||
|
||||
@dataclass
|
||||
class BenchRow:
|
||||
engine: str
|
||||
url: str # base URL of the engine (empty for local)
|
||||
model: str
|
||||
device: str # "CPU" | "GPU" | "remote"
|
||||
device: str # "CPU" | "CUDA" | "remote"
|
||||
best_for: str # "Short clips" | "Short / medium" | "Long / batch" | "Streaming"
|
||||
languages: list[str] # ISO 639-1 codes from /v1/models, empty if unknown
|
||||
ok: bool
|
||||
seconds: float
|
||||
wer: float
|
||||
accuracy: float # percent, max(0, 1-wer)*100
|
||||
text: str
|
||||
ram_mb: float = 0.0 # local RSS delta in MB
|
||||
srv_ram_mb: float | None = None # server /metrics RSS in MB, None = not available
|
||||
error: str = ""
|
||||
|
||||
|
||||
@ -58,32 +76,103 @@ def wer(reference: str, hypothesis: str, *, case_sensitive: bool = False) -> flo
|
||||
return _edit_distance(ref, hyp) / len(ref)
|
||||
|
||||
|
||||
def _engine_device(engine, transcriber) -> str:
|
||||
def _engine_device(engine, transcriber, _cache: dict) -> str:
|
||||
if engine.is_local:
|
||||
return "GPU" if getattr(transcriber, "device", "cpu") == "cuda" else "CPU"
|
||||
return "remote"
|
||||
return "CUDA" if getattr(transcriber, "device", "cpu") == "cuda" else "CPU"
|
||||
url = engine.url
|
||||
if url not in _cache:
|
||||
_cache[url] = stt.detect_remote_device(url)
|
||||
return _cache[url]
|
||||
|
||||
|
||||
def _engine_best_for(engine) -> str:
|
||||
if engine.type == "riva_realtime":
|
||||
return "Streaming"
|
||||
model = (engine.model or "").lower()
|
||||
name = engine.name.lower()
|
||||
if any(x in name for x in ("stream", "realtime", "real-time", "live")):
|
||||
return "Streaming"
|
||||
if engine.is_local:
|
||||
if any(x in model for x in ("tiny", "base")):
|
||||
return "Short clips"
|
||||
if any(x in model for x in ("large",)):
|
||||
return "Long / batch"
|
||||
return "Short / medium"
|
||||
# Remote endpoint
|
||||
if any(x in name for x in ("large", "batch")):
|
||||
return "Long / batch"
|
||||
return "Short / medium"
|
||||
|
||||
|
||||
def run(engines, wav_path: Path, reference: str, *, language: str = "",
|
||||
case_sensitive: bool = True, get_local_transcriber=None, progress=None) -> list[BenchRow]:
|
||||
case_sensitive: bool = True, get_local_transcriber=None, progress=None,
|
||||
expand_models: bool = False) -> list[BenchRow]:
|
||||
"""Benchmark each engine; calls progress(row) as each finishes.
|
||||
|
||||
expand_models=True fetches available models for each remote engine and
|
||||
runs one benchmark row per model instead of just the configured one.
|
||||
Accuracy is case-sensitive by default so wrong capitalisation counts.
|
||||
"""
|
||||
rows: list[BenchRow] = []
|
||||
# Build the run list, optionally expanding remote engines by their models
|
||||
run_list: list = []
|
||||
for e in engines:
|
||||
if expand_models and not e.is_local and not e.is_streaming:
|
||||
models = stt.list_models(e.url, e.api_key_env)
|
||||
if len(models) > 1:
|
||||
for m in models:
|
||||
run_list.append(dataclasses.replace(e, model=m,
|
||||
name=f"{e.name} [{m}]"))
|
||||
continue
|
||||
run_list.append(e)
|
||||
|
||||
device_cache: dict = {}
|
||||
meta_cache: dict = {} # url → list[ModelMeta]
|
||||
server_ram_cache: dict = {} # url → float | None (MB from /metrics)
|
||||
|
||||
def _get_langs(e) -> list[str]:
|
||||
if e.is_local:
|
||||
return []
|
||||
url = e.url
|
||||
if url not in meta_cache:
|
||||
meta_cache[url] = stt.list_models_meta(url, e.api_key_env)
|
||||
for m in meta_cache[url]:
|
||||
if not e.model or m.id == e.model or m.id.endswith("/" + e.model):
|
||||
return m.languages
|
||||
return meta_cache[url][0].languages if meta_cache[url] else []
|
||||
|
||||
rows: list[BenchRow] = []
|
||||
for e in run_list:
|
||||
tr = get_local_transcriber(e) if (e.is_local and get_local_transcriber) else None
|
||||
# Snapshot server RAM before (if Prometheus metrics available)
|
||||
if not e.is_local and e.url not in server_ram_cache:
|
||||
server_ram_cache[e.url] = None # sentinel — probe once per URL
|
||||
srv_before: float | None = None
|
||||
if not e.is_local:
|
||||
srv_before = stt.probe_server_ram_mb(e.url)
|
||||
rss_before = _rss_mb()
|
||||
res = stt.benchmark(e, wav_path, language=language, local_transcriber=tr)
|
||||
rss_after = _rss_mb()
|
||||
ram_delta = max(0.0, rss_after - rss_before)
|
||||
srv_ram: float | None = None
|
||||
if not e.is_local:
|
||||
srv_after = stt.probe_server_ram_mb(e.url)
|
||||
if srv_before is not None and srv_after is not None:
|
||||
srv_ram = srv_after # report current RSS, not delta (server may not unload)
|
||||
w = wer(reference, res.text, case_sensitive=case_sensitive) if res.ok else 1.0
|
||||
row = BenchRow(
|
||||
engine=e.name,
|
||||
url=e.url,
|
||||
model=e.model or ("local" if e.is_local else "(default)"),
|
||||
device=_engine_device(e, tr),
|
||||
device=_engine_device(e, tr, device_cache),
|
||||
best_for=_engine_best_for(e),
|
||||
languages=_get_langs(e),
|
||||
ok=res.ok,
|
||||
seconds=res.seconds,
|
||||
wer=w,
|
||||
accuracy=max(0.0, 1.0 - w) * 100.0,
|
||||
text=res.text,
|
||||
ram_mb=ram_delta,
|
||||
srv_ram_mb=srv_ram,
|
||||
error=res.error,
|
||||
)
|
||||
rows.append(row)
|
||||
|
||||
284
linux/blitztext/caret.py
Normal file
@ -0,0 +1,284 @@
|
||||
"""Best-effort screen anchor for the overlay: *where* to point the bubble's tip.
|
||||
|
||||
The overlay wants to sit at "the cursor where the text will land". On X11 there
|
||||
is no portable way to read the text caret of an arbitrary app, so we degrade
|
||||
through a chain of decreasing precision:
|
||||
|
||||
1. Mouse pointer — `xdotool getmouselocation`. Default; always available on X11;
|
||||
a reliable proxy since the pointer is usually near where you're typing.
|
||||
2. AT-SPI caret — the real text insertion point (opt-in via overlay_anchor=caret).
|
||||
Only native GTK/Qt apps expose it; many terminals / Electron / web views do
|
||||
not. The blocking D-Bus read runs in a thread with a 300ms hard timeout so it
|
||||
can never stall the GTK main loop and freeze the session.
|
||||
3. Window / screen — top-centre of the target window, else screen bottom-centre.
|
||||
|
||||
Everything here is defensive: any failure falls through to the next tier, and
|
||||
the whole module is optional — if AT-SPI isn't running you simply get the
|
||||
pointer anchor. Returns a `(x, y, region)` anchor in **root/screen** pixels,
|
||||
where `region` is the caret/line box so the overlay can avoid covering it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
|
||||
from .logbuffer import log
|
||||
|
||||
|
||||
@dataclass
|
||||
class Anchor:
|
||||
x: int # root-relative x to point the tail tip at
|
||||
y: int # root-relative y (top of the caret/line box)
|
||||
height: int = 0 # caret/line height, so the bubble can clear the line
|
||||
source: str = "" # "caret" | "pointer" | "window" | "corner" — for logging
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Tier 1: AT-SPI caret tracking (best-effort, passive)
|
||||
# --------------------------------------------------------------------------- #
|
||||
class _CaretTracker:
|
||||
"""Remember the most recently focused text object; read its caret lazily.
|
||||
|
||||
*Why so cautious.* AT-SPI queries (``get_character_extents`` and friends) are
|
||||
**synchronous, blocking D-Bus round-trips into the target application**. The
|
||||
earlier design subscribed to the high-frequency ``object:text-caret-moved``
|
||||
signal and ran those blocking reads from *inside* the event handler. Two ways
|
||||
that wedges a whole GNOME/X11 session:
|
||||
|
||||
• Calling a synchronous AT-SPI method from within an AT-SPI event dispatch
|
||||
re-enters the a11y dispatcher and can deadlock the accessibility bus.
|
||||
• ``text-caret-moved`` fires once *per character* — and delivering text is
|
||||
exactly what this app does, typing via ``xdotool`` into the focused
|
||||
field. So a single dictation became a storm of blocking round-trips on
|
||||
the GTK main loop, congesting the a11y bus until the desktop froze.
|
||||
|
||||
So we now subscribe to **focus changes only** (rare, and never emitted by our
|
||||
own synthetic typing), cache just the focused accessible, and do the one
|
||||
blocking extents read **on demand** in :meth:`rect` — called once, when the
|
||||
overlay shows, outside any event dispatch. Worst case is a slightly delayed
|
||||
overlay placement, never a frozen session.
|
||||
"""
|
||||
|
||||
STALE_SECONDS = 30.0 # ignore a focus older than this
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._ok = False
|
||||
self._listener = None
|
||||
self._focused = None # last focused accessible (read lazily)
|
||||
self._stamp = 0.0
|
||||
self._Atspi = None
|
||||
|
||||
def start(self) -> bool:
|
||||
"""Register the AT-SPI focus listener on the running GLib main loop.
|
||||
|
||||
Safe to call when accessibility is disabled — it just returns False and
|
||||
the anchor logic skips this tier from then on.
|
||||
"""
|
||||
try:
|
||||
import gi
|
||||
gi.require_version("Atspi", "2.0")
|
||||
from gi.repository import Atspi # noqa: N813
|
||||
except (ImportError, ValueError) as exc:
|
||||
log(f"[overlay] AT-SPI unavailable, caret anchor disabled: {exc}")
|
||||
return False
|
||||
try:
|
||||
# init() is idempotent; returns 0/1. Connects to the a11y registry.
|
||||
Atspi.init()
|
||||
self._Atspi = Atspi
|
||||
self._listener = Atspi.EventListener.new(self._on_focus)
|
||||
# Focus changes only. Deliberately NOT "object:text-caret-moved": that
|
||||
# firehose (one event per typed character, including our own output)
|
||||
# plus synchronous reads is what could freeze the session.
|
||||
self._listener.register("object:state-changed:focused")
|
||||
self._ok = True
|
||||
log("[overlay] AT-SPI caret tracking active")
|
||||
return True
|
||||
except Exception as exc: # noqa: BLE001 - a11y bus may be down/locked
|
||||
log(f"[overlay] AT-SPI init failed, caret anchor disabled: {exc}")
|
||||
self._ok = False
|
||||
return False
|
||||
|
||||
def stop(self) -> None:
|
||||
try:
|
||||
if self._listener is not None:
|
||||
self._listener.deregister("object:state-changed:focused")
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
self._listener = None
|
||||
self._focused = None
|
||||
self._ok = False
|
||||
|
||||
def _on_focus(self, event) -> None:
|
||||
# Runs on the GLib main thread (same loop GTK uses). Do the *minimum*:
|
||||
# stash the focused accessible and stamp it. Crucially, make NO synchronous
|
||||
# AT-SPI calls here — that would re-enter the a11y dispatcher and risk
|
||||
# deadlocking the bus. The blocking extents read happens later, in rect().
|
||||
try:
|
||||
if not event.detail1:
|
||||
return # a *de*focus event — nothing to track
|
||||
self._focused = event.source
|
||||
self._stamp = time.time()
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
|
||||
def _caret_rect(self, acc) -> tuple[int, int, int, int] | None:
|
||||
Atspi = self._Atspi
|
||||
text = None
|
||||
try:
|
||||
text = acc.get_text_iface()
|
||||
except Exception: # noqa: BLE001
|
||||
text = None
|
||||
if text is None:
|
||||
return None
|
||||
try:
|
||||
offset = text.get_caret_offset()
|
||||
# Extents of the character at the caret, in absolute screen coords.
|
||||
ext = text.get_character_extents(offset, Atspi.CoordType.SCREEN)
|
||||
x, y, w, h = ext.x, ext.y, ext.width, ext.height
|
||||
if w == 0 and h == 0:
|
||||
# End-of-line / empty field: fall back to the component box so we
|
||||
# at least anchor on the right widget.
|
||||
comp = acc.get_component_iface()
|
||||
if comp is None:
|
||||
return None
|
||||
cext = comp.get_extents(Atspi.CoordType.SCREEN)
|
||||
return int(cext.x), int(cext.y), 2, int(cext.height) or 18
|
||||
if x < 0 or y < 0:
|
||||
return None
|
||||
return int(x), int(y), int(w) or 2, int(h) or 18
|
||||
except Exception: # noqa: BLE001
|
||||
return None
|
||||
|
||||
def rect(self) -> tuple[int, int, int, int] | None:
|
||||
# Called once when the overlay shows (not on the a11y hot path).
|
||||
# We run the blocking D-Bus extents read in a worker thread and join it
|
||||
# with a hard 300ms deadline — so even a frozen or slow target app can
|
||||
# never stall the GTK main loop long enough to freeze the session.
|
||||
if not self._ok or self._focused is None:
|
||||
return None
|
||||
if time.time() - self._stamp > self.STALE_SECONDS:
|
||||
return None
|
||||
result: list[tuple[int, int, int, int] | None] = [None]
|
||||
focused = self._focused
|
||||
|
||||
def _read() -> None:
|
||||
try:
|
||||
result[0] = self._caret_rect(focused)
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
|
||||
t = threading.Thread(target=_read, daemon=True)
|
||||
t.start()
|
||||
t.join(timeout=0.3)
|
||||
return result[0]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Tier 2 + 3: pointer and window fallbacks
|
||||
# --------------------------------------------------------------------------- #
|
||||
def _pointer() -> tuple[int, int] | None:
|
||||
if not shutil.which("xdotool"):
|
||||
return None
|
||||
try:
|
||||
out = subprocess.run(
|
||||
["xdotool", "getmouselocation", "--shell"],
|
||||
capture_output=True, text=True, check=True, timeout=0.5,
|
||||
).stdout
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
return None
|
||||
vals: dict[str, str] = {}
|
||||
for line in out.splitlines():
|
||||
if "=" in line:
|
||||
k, _, v = line.partition("=")
|
||||
vals[k.strip()] = v.strip()
|
||||
try:
|
||||
return int(vals["X"]), int(vals["Y"])
|
||||
except (KeyError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def _window_box(window_id: str | None) -> tuple[int, int, int] | None:
|
||||
"""Top-centre of the target window: (x, y, height_hint)."""
|
||||
if not window_id or not shutil.which("xdotool"):
|
||||
return None
|
||||
try:
|
||||
out = subprocess.run(
|
||||
["xdotool", "getwindowgeometry", "--shell", window_id],
|
||||
capture_output=True, text=True, check=True, timeout=0.5,
|
||||
).stdout
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
return None
|
||||
vals: dict[str, str] = {}
|
||||
for line in out.splitlines():
|
||||
if "=" in line:
|
||||
k, _, v = line.partition("=")
|
||||
vals[k.strip()] = v.strip()
|
||||
try:
|
||||
x, y = int(vals["X"]), int(vals["Y"])
|
||||
w = int(vals["WIDTH"])
|
||||
return x + w // 2, y + 40, 0
|
||||
except (KeyError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def _screen_bottom_centre() -> tuple[int, int]:
|
||||
try:
|
||||
import gi
|
||||
gi.require_version("Gdk", "3.0")
|
||||
from gi.repository import Gdk
|
||||
disp = Gdk.Display.get_default()
|
||||
mon = disp.get_primary_monitor() or disp.get_monitor(0)
|
||||
geo = mon.get_geometry()
|
||||
return geo.x + geo.width // 2, geo.y + geo.height - 140
|
||||
except Exception: # noqa: BLE001
|
||||
return 960, 800 # last-ditch constant; better than crashing
|
||||
|
||||
|
||||
_tracker: _CaretTracker | None = None
|
||||
|
||||
|
||||
def start_tracking() -> None:
|
||||
"""Begin passive caret tracking (call once, from the GTK main thread)."""
|
||||
global _tracker
|
||||
if _tracker is None:
|
||||
_tracker = _CaretTracker()
|
||||
_tracker.start()
|
||||
|
||||
|
||||
def stop_tracking() -> None:
|
||||
global _tracker
|
||||
if _tracker is not None:
|
||||
_tracker.stop()
|
||||
_tracker = None
|
||||
|
||||
|
||||
def resolve(anchor_mode: str, window_id: str | None) -> Anchor:
|
||||
"""Resolve the on-screen anchor for the overlay, honouring `anchor_mode`.
|
||||
|
||||
anchor_mode:
|
||||
"caret" -> caret → pointer → window → corner
|
||||
"pointer" -> pointer → window → corner
|
||||
"corner" -> window → corner
|
||||
Always returns an Anchor (never None) so callers don't special-case failure.
|
||||
"""
|
||||
if anchor_mode == "caret" and _tracker is not None:
|
||||
r = _tracker.rect()
|
||||
if r is not None:
|
||||
x, y, _w, h = r
|
||||
return Anchor(x=x, y=y, height=h or 18, source="caret")
|
||||
|
||||
if anchor_mode in ("caret", "pointer"):
|
||||
p = _pointer()
|
||||
if p is not None:
|
||||
return Anchor(x=p[0], y=p[1], height=0, source="pointer")
|
||||
|
||||
wb = _window_box(window_id)
|
||||
if wb is not None:
|
||||
return Anchor(x=wb[0], y=wb[1], height=0, source="window")
|
||||
|
||||
x, y = _screen_bottom_centre()
|
||||
return Anchor(x=x, y=y, height=0, source="corner")
|
||||
@ -25,11 +25,22 @@ class Workflow:
|
||||
# Optional per-workflow overrides of the [rewrite] defaults.
|
||||
model: str | None = None
|
||||
temperature: float | None = None
|
||||
# Which LLM engine to use for this preset's rewrite step.
|
||||
# "" (empty) = use whichever engine is currently active in the Engines tab.
|
||||
llm_engine: str = ""
|
||||
# Cosmetic, used by the GUI.
|
||||
description: str = ""
|
||||
icon: str = "⚡"
|
||||
|
||||
|
||||
@dataclass
|
||||
class WakewordEngine:
|
||||
"""A wyoming-openwakeword endpoint to compare in the wakeword benchmark."""
|
||||
name: str = ""
|
||||
uri: str = "tcp://127.0.0.1:10400"
|
||||
model: str = "okay_computer"
|
||||
|
||||
|
||||
@dataclass
|
||||
class Config:
|
||||
# general
|
||||
@ -38,7 +49,11 @@ class Config:
|
||||
output: str = "type" # type | paste
|
||||
type_delay_ms: int = 12
|
||||
notify: bool = True
|
||||
notify_routing: bool = True # announce which preset/keyword a voice command matched
|
||||
language: str = "de" # whisper hint; "" = autodetect
|
||||
# on-screen overlay (mic + live waveform + recognised-text bubble)
|
||||
overlay_enabled: bool = True
|
||||
overlay_anchor: str = "pointer" # pointer | caret (AT-SPI, may freeze) | corner
|
||||
# input scheme
|
||||
input_mode: str = "modifiers" # "modifiers" (Ctrl+Win/Ctrl/Alt/Esc) | "hotkeys" (combos)
|
||||
push_to_talk: bool = False
|
||||
@ -71,6 +86,16 @@ class Config:
|
||||
routing_hotkey: str = "<ctrl>+<alt>+<space>"
|
||||
routing_default: str = "" # preset name used when no keyword matches; "" = first
|
||||
routing_threshold: float = 0.82
|
||||
# Spoken abort: if one of these words is heard at the start/end of a clip, the
|
||||
# dictation is discarded — never transcribed onward, routed, rewritten, or
|
||||
# typed. Empty list = disabled. Mainly for accidental wakeword triggers.
|
||||
cancel_keywords: list[str] = field(default_factory=lambda: ["abbrechen", "cancel"])
|
||||
# Spoken send: if one of these is heard at the start/end of a clip, it is
|
||||
# stripped and the rest is delivered *and submitted with Enter* — the spoken
|
||||
# equivalent of "stop + paste + Enter". Empty = disabled. Because it presses
|
||||
# Enter, prefer a distinctive multi-word phrase (e.g. "computer send") so a
|
||||
# sentence that merely ends in "send" doesn't submit by accident.
|
||||
send_keywords: list[str] = field(default_factory=list)
|
||||
# speech-to-text engines (presets)
|
||||
stt_engines: list[STTEngine] = field(default_factory=list)
|
||||
stt_active: str = ""
|
||||
@ -79,11 +104,31 @@ class Config:
|
||||
llm_active: str = ""
|
||||
# wakeword
|
||||
wakeword_enabled: bool = False
|
||||
wakeword_active: str = "" # name of selected WakewordEngine preset, "" = custom
|
||||
wakeword_uri: str = "tcp://127.0.0.1:10400"
|
||||
wakeword_model: str = "okay_computer"
|
||||
wakeword_sound_detected: str = "" # WAV played when the wakeword fires (speak now)
|
||||
wakeword_sound_done: str = "" # WAV played when the command is captured
|
||||
wakeword_silence_seconds: float = 2.0 # auto-stop after this much trailing silence
|
||||
wakeword_cancel_model: str = "" # wakeword model that cancels an in-progress recording
|
||||
wakeword_send_model: str = "" # wakeword model that finishes + sends (Enter) a recording
|
||||
setup_complete: bool = False # True once the first-run wizard has been completed
|
||||
# Text-to-speech for the wakeword benchmark — its own OpenAI-compatible
|
||||
# endpoint (Kokoro, XTTS, OpenAI, …): base URL incl. /v1, an optional bearer
|
||||
# key env var, a model id, and the voices to cycle through.
|
||||
tts_url: str = ""
|
||||
tts_api_key_env: str = ""
|
||||
tts_model: str = ""
|
||||
tts_voices: list[str] = field(
|
||||
default_factory=lambda: ["alloy", "echo", "fable", "onyx", "nova", "shimmer"])
|
||||
# Wakeword engines to compare in the benchmark (each a wyoming-openwakeword
|
||||
# endpoint). Empty → the benchmark falls back to the live [wakeword] above.
|
||||
wakeword_engines: list[WakewordEngine] = field(default_factory=list)
|
||||
# STT benchmark: last-used WAV / reference transcript paths and options
|
||||
bench_wav: str = ""
|
||||
bench_ref: str = ""
|
||||
bench_expand_models: bool = False
|
||||
bench_last: dict = field(default_factory=dict) # {engine_name: {seconds, accuracy, ok}}
|
||||
# workflows
|
||||
workflows: list[Workflow] = field(default_factory=list)
|
||||
|
||||
@ -110,13 +155,23 @@ class Config:
|
||||
|
||||
@property
|
||||
def default_preset(self) -> "Workflow | None":
|
||||
return self.preset_by_name(self.routing_default) or (self.workflows[0] if self.workflows else None)
|
||||
named = self.preset_by_name(self.routing_default)
|
||||
if named:
|
||||
return named
|
||||
if not self.workflows:
|
||||
return None
|
||||
# No explicit default configured: prefer a plain transcribe preset over
|
||||
# whatever happens to be first, so the no-keyword fallback never silently
|
||||
# routes to an LLM rewrite.
|
||||
return next((w for w in self.workflows if w.mode == "transcribe"), self.workflows[0])
|
||||
|
||||
@property
|
||||
def all_keywords(self) -> list[str]:
|
||||
out: list[str] = []
|
||||
for w in self.workflows:
|
||||
out.extend(w.keywords)
|
||||
if w.name and w.name not in out:
|
||||
out.append(w.name) # names are implicit triggers — bias STT for them too
|
||||
return out
|
||||
|
||||
@property
|
||||
@ -140,6 +195,7 @@ def load(path: Path = CONFIG_PATH) -> Config:
|
||||
q = data.get("quality", {})
|
||||
snd = data.get("sounds", {})
|
||||
ww = data.get("wakeword", {})
|
||||
tts = data.get("tts", {})
|
||||
|
||||
cfg = Config(
|
||||
recorder=g.get("recorder", "auto"),
|
||||
@ -147,7 +203,11 @@ def load(path: Path = CONFIG_PATH) -> Config:
|
||||
output=g.get("output", "type"),
|
||||
type_delay_ms=int(g.get("type_delay_ms", 4)),
|
||||
notify=bool(g.get("notify", True)),
|
||||
setup_complete=bool(g.get("setup_complete", False)),
|
||||
notify_routing=bool(g.get("notify_routing", True)),
|
||||
language=g.get("language", "de"),
|
||||
overlay_enabled=bool(g.get("overlay_enabled", True)),
|
||||
overlay_anchor=g.get("overlay_anchor", "pointer"),
|
||||
model=w.get("model", "small"),
|
||||
device=w.get("device", "auto"),
|
||||
compute_type=w.get("compute_type", "auto"),
|
||||
@ -161,6 +221,8 @@ def load(path: Path = CONFIG_PATH) -> Config:
|
||||
routing_hotkey=rt.get("hotkey", "<ctrl>+<alt>+<space>"),
|
||||
routing_default=rt.get("default", ""),
|
||||
routing_threshold=float(rt.get("threshold", 0.82)),
|
||||
cancel_keywords=list(rt.get("cancel_keywords", ["abbrechen", "cancel"])),
|
||||
send_keywords=list(rt.get("send_keywords", [])),
|
||||
input_mode=inp.get("mode", "modifiers"),
|
||||
push_to_talk=bool(inp.get("push_to_talk", False)),
|
||||
key_start=inp.get("start", "<ctrl>+<cmd>"),
|
||||
@ -175,11 +237,22 @@ def load(path: Path = CONFIG_PATH) -> Config:
|
||||
sound_before=snd.get("before", ""),
|
||||
sound_after=snd.get("after", ""),
|
||||
wakeword_enabled=bool(ww.get("enabled", False)),
|
||||
wakeword_active=ww.get("active", ""),
|
||||
wakeword_uri=ww.get("uri", "tcp://127.0.0.1:10400"),
|
||||
wakeword_model=ww.get("model", "okay_computer"),
|
||||
wakeword_sound_detected=ww.get("sound_detected", ""),
|
||||
wakeword_sound_done=ww.get("sound_done", ""),
|
||||
wakeword_silence_seconds=float(ww.get("silence_seconds", 2.0)),
|
||||
wakeword_cancel_model=ww.get("cancel_model", ""),
|
||||
wakeword_send_model=ww.get("send_model", ""),
|
||||
tts_url=tts.get("url", "").rstrip("/"),
|
||||
tts_api_key_env=tts.get("api_key_env", ""),
|
||||
tts_model=tts.get("model", ""),
|
||||
tts_voices=list(tts.get("voices", ["alloy", "echo", "fable", "onyx", "nova", "shimmer"])),
|
||||
bench_wav=data.get("benchmark", {}).get("wav", ""),
|
||||
bench_ref=data.get("benchmark", {}).get("ref", ""),
|
||||
bench_expand_models=bool(data.get("benchmark", {}).get("expand_models", False)),
|
||||
bench_last=dict(data.get("benchmark", {}).get("last", {})),
|
||||
)
|
||||
|
||||
for entry in data.get("workflow", []):
|
||||
@ -192,6 +265,7 @@ def load(path: Path = CONFIG_PATH) -> Config:
|
||||
keywords=list(entry.get("keywords", [])),
|
||||
model=entry.get("model"),
|
||||
temperature=entry.get("temperature"),
|
||||
llm_engine=entry.get("llm_engine", ""),
|
||||
description=entry.get("description", ""),
|
||||
icon=entry.get("icon", "⚡"),
|
||||
)
|
||||
@ -208,11 +282,30 @@ def load(path: Path = CONFIG_PATH) -> Config:
|
||||
url=e.get("url", "").rstrip("/"),
|
||||
model=e.get("model", ""),
|
||||
api_key_env=e.get("api_key_env", ""),
|
||||
timeout=int(e.get("timeout", 30)),
|
||||
)
|
||||
for e in data.get("stt_engine", [])
|
||||
] or [STTEngine("Local faster-whisper", "local", model=cfg.model)]
|
||||
cfg.stt_active = data.get("stt", {}).get("active", cfg.stt_engines[0].name)
|
||||
|
||||
# Wakeword engines for the benchmark (optional; benchmark falls back to the
|
||||
# live [wakeword] config when none are listed).
|
||||
cfg.wakeword_engines = [
|
||||
WakewordEngine(
|
||||
name=e.get("name", ""),
|
||||
uri=e.get("uri", "tcp://127.0.0.1:10400"),
|
||||
model=e.get("model", "okay_computer"),
|
||||
)
|
||||
for e in data.get("wakeword_engine", [])
|
||||
]
|
||||
# Migration: ensure at least one engine exists so the CRUD UI always has a row.
|
||||
if not cfg.wakeword_engines:
|
||||
cfg.wakeword_engines = [WakewordEngine(
|
||||
name=cfg.wakeword_active or "Local wyoming-openwakeword",
|
||||
uri=cfg.wakeword_uri,
|
||||
model=cfg.wakeword_model,
|
||||
)]
|
||||
|
||||
# LLM engines (default: synthesized from the legacy [rewrite] block).
|
||||
cfg.llm_engines = [
|
||||
LLMEngine(
|
||||
@ -244,7 +337,11 @@ def save(cfg: Config, path: Path = CONFIG_PATH) -> None:
|
||||
"output": cfg.output,
|
||||
"type_delay_ms": cfg.type_delay_ms,
|
||||
"notify": cfg.notify,
|
||||
"setup_complete": cfg.setup_complete,
|
||||
"notify_routing": cfg.notify_routing,
|
||||
"language": cfg.language,
|
||||
"overlay_enabled": cfg.overlay_enabled,
|
||||
"overlay_anchor": cfg.overlay_anchor,
|
||||
},
|
||||
"whisper": {
|
||||
"model": cfg.model,
|
||||
@ -272,6 +369,8 @@ def save(cfg: Config, path: Path = CONFIG_PATH) -> None:
|
||||
"hotkey": cfg.routing_hotkey,
|
||||
"default": cfg.routing_default,
|
||||
"threshold": cfg.routing_threshold,
|
||||
"cancel_keywords": cfg.cancel_keywords,
|
||||
"send_keywords": cfg.send_keywords,
|
||||
},
|
||||
"quality": {
|
||||
"min_speech_seconds": cfg.min_speech_seconds,
|
||||
@ -286,18 +385,37 @@ def save(cfg: Config, path: Path = CONFIG_PATH) -> None:
|
||||
},
|
||||
"wakeword": {
|
||||
"enabled": cfg.wakeword_enabled,
|
||||
"active": cfg.wakeword_active,
|
||||
"uri": cfg.wakeword_uri,
|
||||
"model": cfg.wakeword_model,
|
||||
"sound_detected": cfg.wakeword_sound_detected,
|
||||
"sound_done": cfg.wakeword_sound_done,
|
||||
"silence_seconds": cfg.wakeword_silence_seconds,
|
||||
"cancel_model": cfg.wakeword_cancel_model,
|
||||
"send_model": cfg.wakeword_send_model,
|
||||
},
|
||||
"tts": {
|
||||
"url": cfg.tts_url,
|
||||
"api_key_env": cfg.tts_api_key_env,
|
||||
"model": cfg.tts_model,
|
||||
"voices": cfg.tts_voices,
|
||||
},
|
||||
"benchmark": {
|
||||
"wav": cfg.bench_wav,
|
||||
"ref": cfg.bench_ref,
|
||||
"expand_models": cfg.bench_expand_models,
|
||||
"last": cfg.bench_last,
|
||||
},
|
||||
"wakeword_engine": [
|
||||
{"name": e.name, "uri": e.uri, "model": e.model} for e in cfg.wakeword_engines
|
||||
],
|
||||
"stt": {"active": cfg.stt_active},
|
||||
"stt_engine": [
|
||||
{k: v for k, v in {
|
||||
"name": e.name, "type": e.type, "url": e.url,
|
||||
"model": e.model, "api_key_env": e.api_key_env,
|
||||
}.items() if v or k in ("name", "type")}
|
||||
"timeout": e.timeout if e.timeout != 30 else None,
|
||||
}.items() if v is not None and (v or k in ("name", "type"))}
|
||||
for e in cfg.stt_engines
|
||||
],
|
||||
"llm": {"active": cfg.llm_active},
|
||||
@ -318,6 +436,8 @@ def save(cfg: Config, path: Path = CONFIG_PATH) -> None:
|
||||
entry["model"] = wf.model
|
||||
if wf.temperature is not None:
|
||||
entry["temperature"] = wf.temperature
|
||||
if wf.llm_engine:
|
||||
entry["llm_engine"] = wf.llm_engine
|
||||
if wf.description:
|
||||
entry["description"] = wf.description
|
||||
if wf.icon and wf.icon != "⚡":
|
||||
@ -347,7 +467,10 @@ recorder = "auto" # auto | pw-record | parecord | arecord
|
||||
output = "type" # "type" = xdotool types it; "paste" = clipboard + Ctrl+V
|
||||
type_delay_ms = 12 # per-keystroke delay for xdotool type (raise if chars drop)
|
||||
notify = true # desktop notifications for each phase
|
||||
notify_routing = true # announce which preset/keyword a voice command matched (shown even hands-free)
|
||||
language = "de" # Whisper language hint; "" = autodetect
|
||||
overlay_enabled = true # on-screen mic + live waveform + recognised-text bubble at the cursor
|
||||
overlay_anchor = "pointer" # pointer (near the mouse — safe) | caret (AT-SPI, may freeze) | corner
|
||||
|
||||
[input]
|
||||
# How you start/stop dictation.
|
||||
@ -406,6 +529,16 @@ enabled = true
|
||||
hotkey = "<ctrl>+<alt>+<space>"
|
||||
default = "Transcribe" # preset used when no keyword is recognised
|
||||
threshold = 0.82 # 0..1 fuzzy-match strictness (higher = stricter)
|
||||
# Say one of these at the start or end of a clip to DISCARD it — nothing is
|
||||
# routed, rewritten, or typed. Handy when a wakeword fires by accident. Pick
|
||||
# words you won't naturally end a real dictation with. Empty list = off.
|
||||
cancel_keywords = ["abbrechen", "cancel"]
|
||||
# Say one of these at the start or end of a clip to SEND it: the word is stripped
|
||||
# and the rest is delivered AND submitted with Enter (spoken "stop+paste+Enter").
|
||||
# Because it presses Enter, use a distinctive multi-word phrase (e.g. your
|
||||
# wakeword + "send") so a sentence that just ends in "send" won't submit. Off by
|
||||
# default; empty list = off.
|
||||
send_keywords = []
|
||||
|
||||
[wakeword]
|
||||
# Hands-free dictation using an external wyoming-openwakeword server.
|
||||
@ -422,6 +555,30 @@ sound_done = ""
|
||||
# Auto-stop the recording this many seconds after you stop speaking (silence).
|
||||
silence_seconds = 2.0
|
||||
|
||||
[tts]
|
||||
# Text-to-speech for the WAKEWORD BENCHMARK only (Settings → Benchmark). Point it
|
||||
# at any OpenAI-compatible TTS server's /audio/speech (Kokoro-FastAPI, XTTS-v2,
|
||||
# OpenAI, …). url = base incl. /v1; api_key_env = env var holding a bearer key
|
||||
# (leave empty for no-auth local servers); model = the TTS model id; voices = the
|
||||
# names your endpoint serves. Leave url/model empty to disable the benchmark.
|
||||
url = ""
|
||||
api_key_env = ""
|
||||
model = ""
|
||||
voices = ["alloy", "echo", "fable", "onyx", "nova", "shimmer"]
|
||||
|
||||
# Wakeword engines to COMPARE in the benchmark — add one block per
|
||||
# wyoming-openwakeword endpoint/model you want side by side. Leave this out to
|
||||
# just benchmark the live [wakeword] engine above.
|
||||
# [[wakeword_engine]]
|
||||
# name = "openWakeWord · computer"
|
||||
# uri = "tcp://127.0.0.1:10400"
|
||||
# model = "computer"
|
||||
#
|
||||
# [[wakeword_engine]]
|
||||
# name = "microWakeWord · hey_jarvis"
|
||||
# uri = "tcp://127.0.0.1:10500"
|
||||
# model = "hey_jarvis"
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Speech-to-text engines (presets). The active one is used for transcription.
|
||||
# type = "local" -> in-process faster-whisper (uses [whisper] above)
|
||||
|
||||
@ -7,8 +7,10 @@ routing: one hotkey records, then the spoken keyword selects the preset.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import signal
|
||||
import sys
|
||||
import threading
|
||||
import traceback
|
||||
from typing import Callable
|
||||
|
||||
from . import llm, quality, stt
|
||||
@ -19,18 +21,141 @@ from .notify import notify
|
||||
from .paste import active_window_id, deliver
|
||||
from .streaming import RivaRealtimeStreamer
|
||||
from .recorder import Recording, detect_recorder
|
||||
from .routing import route
|
||||
from .routing import is_cancel, match_send, route
|
||||
from .transcribe import Transcriber
|
||||
|
||||
# status_cb(state, workflow_name, message)
|
||||
# state in {"loading", "idle", "recording", "streaming", "busy", "done", "error"}
|
||||
StatusCallback = Callable[[str, str | None, str], None]
|
||||
|
||||
# A trailing pause shorter than this never arms the auto-stop countdown, so the
|
||||
# overlay ring doesn't flicker in the gaps between words. The visible countdown
|
||||
# therefore spans (silence_seconds - this) and begins full once you genuinely
|
||||
# stop speaking.
|
||||
_VAD_COUNTDOWN_GRACE = 0.35
|
||||
|
||||
# Cancel-watcher: accumulate this much audio before the first check, then
|
||||
# re-check every time this many new bytes arrive. 3200 bytes = 100 ms at
|
||||
# 16 kHz s16le mono. 0.8 s min avoids false positives on the very first chunk;
|
||||
# 0.6 s poll keeps latency low without hammering the transcriber.
|
||||
_CANCEL_MIN_BYTES = int(0.8 * 16000 * 2) # 25600
|
||||
_CANCEL_POLL_BYTES = int(0.6 * 16000 * 2) # 19200
|
||||
|
||||
|
||||
def _pcm_to_wav(path: str, pcm: bytes) -> None:
|
||||
"""Write raw s16le 16 kHz mono PCM bytes as a minimal RIFF WAV."""
|
||||
import struct
|
||||
data_len = len(pcm)
|
||||
with open(path, "wb") as f:
|
||||
f.write(b"RIFF")
|
||||
f.write(struct.pack("<I", 36 + data_len))
|
||||
f.write(b"WAVE")
|
||||
f.write(b"fmt ")
|
||||
f.write(struct.pack("<IHHIIHH", 16, 1, 1, 16000, 32000, 2, 16))
|
||||
f.write(b"data")
|
||||
f.write(struct.pack("<I", data_len))
|
||||
f.write(pcm)
|
||||
|
||||
|
||||
class _CancelWatcher:
|
||||
"""Listens to in-progress VAD audio and triggers cancel if a keyword is heard.
|
||||
|
||||
PCM chunks (raw s16le 16 kHz mono) are fed via :meth:`feed` from the VAD
|
||||
level-meter thread. Every _CANCEL_POLL_BYTES of *new* audio (after an
|
||||
initial _CANCEL_MIN_BYTES warm-up), a fast beam_size=1 transcription of the
|
||||
accumulated buffer runs in a background thread. If a cancel keyword is
|
||||
found, the supplied ``on_cancel`` callback fires once and the watcher stops.
|
||||
"""
|
||||
|
||||
def __init__(self, transcriber, cancel_keywords, language, threshold, on_cancel):
|
||||
self._transcriber = transcriber
|
||||
self._keywords = cancel_keywords
|
||||
self._language = language
|
||||
self._threshold = threshold
|
||||
self._on_cancel = on_cancel
|
||||
self._buf = bytearray()
|
||||
self._new_bytes = 0
|
||||
self._active = True
|
||||
self._lock = threading.Lock()
|
||||
self._checking = False # prevents overlapping check threads
|
||||
|
||||
def feed(self, chunk: bytes) -> None:
|
||||
if not self._active:
|
||||
return
|
||||
with self._lock:
|
||||
if not self._active:
|
||||
return
|
||||
self._buf.extend(chunk)
|
||||
self._new_bytes += len(chunk)
|
||||
ready = (len(self._buf) >= _CANCEL_MIN_BYTES
|
||||
and self._new_bytes >= _CANCEL_POLL_BYTES
|
||||
and not self._checking)
|
||||
if ready:
|
||||
self._new_bytes = 0
|
||||
self._checking = True
|
||||
snapshot = bytes(self._buf)
|
||||
if ready:
|
||||
threading.Thread(target=self._check, args=(snapshot,), daemon=True,
|
||||
name="CancelWatcher").start()
|
||||
|
||||
def stop(self) -> None:
|
||||
with self._lock:
|
||||
self._active = False
|
||||
|
||||
def _check(self, audio: bytes) -> None:
|
||||
import os
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
fd, tmp = tempfile.mkstemp(prefix="bt-cw-", suffix=".wav")
|
||||
try:
|
||||
os.close(fd)
|
||||
_pcm_to_wav(tmp, audio)
|
||||
text = self._transcriber.transcribe(
|
||||
Path(tmp), language=self._language, beam_size=1)
|
||||
with self._lock:
|
||||
if not self._active:
|
||||
return
|
||||
kw = is_cancel(text, self._keywords, threshold=self._threshold)
|
||||
if kw:
|
||||
with self._lock:
|
||||
if not self._active:
|
||||
return
|
||||
self._active = False
|
||||
log(f'[cancel-watcher] “{kw}” heard live — cancelling immediately.')
|
||||
self._on_cancel()
|
||||
except Exception: # noqa: BLE001 — watcher must never crash the daemon
|
||||
pass
|
||||
finally:
|
||||
Path(tmp).unlink(missing_ok=True)
|
||||
with self._lock:
|
||||
self._checking = False
|
||||
|
||||
|
||||
class Daemon:
|
||||
def __init__(self, cfg: Config, status_cb: StatusCallback | None = None):
|
||||
def __init__(self, cfg: Config, status_cb: StatusCallback | None = None,
|
||||
level_cb: Callable[[float], None] | None = None,
|
||||
text_cb: Callable[[str], None] | None = None,
|
||||
countdown_cb: Callable[[float | None, float], None] | None = None,
|
||||
routing_cb: Callable[[str, str, str | None], None] | None = None):
|
||||
self.cfg = cfg
|
||||
self.status_cb = status_cb
|
||||
# Optional UI feedback hooks for the on-screen overlay. The daemon stays
|
||||
# UI-agnostic: these are no-ops in headless mode. level_cb gets the live
|
||||
# mic level (0..1); text_cb gets the running transcript while streaming
|
||||
# (and the live LLM rewrite); countdown_cb(seconds_left, window) drives the
|
||||
# silence auto-stop ring (seconds_left=None while you're speaking, so the
|
||||
# ring clears); routing_cb(icon, preset_name, keyword) fires when voice
|
||||
# routing picks a preset, so the overlay can show it.
|
||||
self.level_cb = level_cb
|
||||
self.text_cb = text_cb
|
||||
self.countdown_cb = countdown_cb
|
||||
self.routing_cb = routing_cb
|
||||
# An overlay consumes routing_cb, and it narrates every phase on-screen, so
|
||||
# the redundant desktop notifications are fused into it (only errors still
|
||||
# pop a bubble). Headless / overlay-off keeps the notifications.
|
||||
self._overlay = routing_cb is not None
|
||||
self._ov_meter = None
|
||||
self._ov_text_final = ""
|
||||
self._lock = threading.Lock()
|
||||
self._recording: Recording | None = None
|
||||
self._streaming: RivaRealtimeStreamer | None = None
|
||||
@ -38,6 +163,7 @@ class Daemon:
|
||||
self._active_workflow: Workflow | None = None
|
||||
self._target_window: str | None = None
|
||||
self._busy = False
|
||||
self._abort_event = threading.Event()
|
||||
self._prepared = False
|
||||
self._listener = None
|
||||
# Synthetic preset used by the voice-routing hotkey.
|
||||
@ -80,10 +206,22 @@ class Daemon:
|
||||
notify(title, body, urgency=urgency, enabled=self.cfg.notify)
|
||||
|
||||
def _dnotify(self, title: str, body: str = "", urgency: str = "normal") -> None:
|
||||
"""Per-dictation notification — suppressed for hands-free (wakeword) sessions."""
|
||||
if not self._session_silent:
|
||||
"""Per-dictation notification — suppressed for hands-free (wakeword)
|
||||
sessions, and (except for errors) when an overlay is narrating on-screen."""
|
||||
if self._session_silent:
|
||||
return
|
||||
if self._overlay and urgency != "critical":
|
||||
return # fused into the on-screen overlay instead of a desktop bubble
|
||||
self._notify(title, body, urgency=urgency)
|
||||
|
||||
def _rnotify(self, title: str, body: str = "", urgency: str = "normal") -> None:
|
||||
"""Routing feedback — which preset/keyword a voice command matched. When an
|
||||
overlay is present this is shown there (via routing_cb) instead of a
|
||||
notification; otherwise it pops a bubble. Only fires on a real match."""
|
||||
if self._overlay:
|
||||
return # shown on the overlay banner instead
|
||||
notify(title, body, urgency=urgency, enabled=self.cfg.notify_routing)
|
||||
|
||||
def _emit(self, state: str, workflow: str | None = None, message: str = "") -> None:
|
||||
if self.status_cb:
|
||||
try:
|
||||
@ -112,9 +250,30 @@ class Daemon:
|
||||
log(f"Using remote STT '{engine.name}' — no local model to load")
|
||||
self._prepared = True
|
||||
self._init_wakeword()
|
||||
self._install_freeze_diagnostic()
|
||||
log("Ready.")
|
||||
self._emit("idle", None, "Ready")
|
||||
|
||||
def _install_freeze_diagnostic(self) -> None:
|
||||
"""Register SIGQUIT (Ctrl+\\ or kill -QUIT) to dump all thread stacks.
|
||||
|
||||
When the system appears frozen, run:
|
||||
kill -QUIT $(pgrep -f blitztext)
|
||||
and the full thread dump appears in the Blitztext log window.
|
||||
"""
|
||||
def _dump(_sig, _frame):
|
||||
lines = ["\n=== FREEZE DIAGNOSTIC — all thread stacks ==="]
|
||||
for tid, frame in sys._current_frames().items():
|
||||
name = next((t.name for t in threading.enumerate() if t.ident == tid), str(tid))
|
||||
lines.append(f"\n-- Thread: {name} (id={tid}) --")
|
||||
lines.extend(traceback.format_stack(frame))
|
||||
lines.append("=== END FREEZE DIAGNOSTIC ===")
|
||||
log("\n".join(lines))
|
||||
try:
|
||||
signal.signal(signal.SIGQUIT, _dump)
|
||||
except (OSError, ValueError):
|
||||
pass # not available on all platforms
|
||||
|
||||
@property
|
||||
def ready(self) -> bool:
|
||||
return getattr(self, "_prepared", False)
|
||||
@ -134,21 +293,110 @@ class Daemon:
|
||||
|
||||
silence = max(0.5, self.cfg.wakeword_silence_seconds)
|
||||
def on_level(level):
|
||||
# Feed the overlay waveform (the VAD meter is already capturing, so
|
||||
# we reuse its level rather than opening a second stream).
|
||||
if self.level_cb:
|
||||
self.level_cb(level)
|
||||
now = time.time()
|
||||
if level > 0.05:
|
||||
self._vad_last_speech = now
|
||||
elif now - self._vad_started_at > 2.0 and now - self._vad_last_speech > silence:
|
||||
if getattr(self, "is_recording", False):
|
||||
if self.countdown_cb:
|
||||
self.countdown_cb(None, silence) # speaking — no countdown
|
||||
return
|
||||
quiet = now - self._vad_last_speech
|
||||
armed = now - self._vad_started_at > 2.0
|
||||
if armed and quiet > _VAD_COUNTDOWN_GRACE:
|
||||
# Mirror the auto-stop window into the overlay ring: full when
|
||||
# you fall quiet, empty exactly as it fires.
|
||||
if self.countdown_cb:
|
||||
self.countdown_cb(silence - quiet, silence - _VAD_COUNTDOWN_GRACE)
|
||||
if quiet > silence and getattr(self, "is_recording", False):
|
||||
GLib.idle_add(lambda: self.finish_dictation(send_enter=False))
|
||||
self._vad_stop()
|
||||
elif self.countdown_cb:
|
||||
self.countdown_cb(None, silence)
|
||||
|
||||
self._vad_meter = audio.LevelMeter(self.cfg.mic, on_level=on_level)
|
||||
self._vad_meter.start()
|
||||
# Wakeword-based action listener: if cancel/send wakeword models are
|
||||
# configured, open a second Wyoming connection during recording so a
|
||||
# dedicated wakeword ("stop", "send it") fires instantly — no Whisper pass.
|
||||
self._action_listener = None
|
||||
if self.cfg.wakeword_enabled and (
|
||||
self.cfg.wakeword_cancel_model or self.cfg.wakeword_send_model):
|
||||
from .wakeword import WakewordActionListener
|
||||
cbs: dict = {}
|
||||
if self.cfg.wakeword_cancel_model:
|
||||
cbs[self.cfg.wakeword_cancel_model] = lambda: GLib.idle_add(
|
||||
self.cancel_dictation)
|
||||
if self.cfg.wakeword_send_model:
|
||||
cbs[self.cfg.wakeword_send_model] = lambda: GLib.idle_add(
|
||||
lambda: self.finish_dictation(send_enter=True))
|
||||
self._action_listener = WakewordActionListener(
|
||||
uri=self.cfg.wakeword_uri, model_callbacks=cbs, mic=self.cfg.mic)
|
||||
self._action_listener.start()
|
||||
|
||||
# Whisper-based cancel watcher: fallback when no cancel wakeword model is
|
||||
# set, or as belt-and-suspenders for the spoken cancel keyword list.
|
||||
on_chunk = None
|
||||
use_whisper_watcher = (self.cfg.cancel_keywords
|
||||
and getattr(self, "transcriber", None) is not None
|
||||
and not self.cfg.wakeword_cancel_model)
|
||||
if use_whisper_watcher:
|
||||
self._cancel_watcher = _CancelWatcher(
|
||||
self.transcriber,
|
||||
self.cfg.cancel_keywords,
|
||||
self.cfg.language,
|
||||
self.cfg.routing_threshold,
|
||||
on_cancel=lambda: GLib.idle_add(self.cancel_dictation),
|
||||
)
|
||||
on_chunk = self._cancel_watcher.feed
|
||||
else:
|
||||
self._cancel_watcher = None
|
||||
|
||||
self._vad_meter = audio.LevelMeter(self.cfg.mic, on_level=on_level,
|
||||
recorder=self.recorder_name, on_chunk=on_chunk)
|
||||
ok = self._vad_meter.start()
|
||||
|
||||
# Safety net: if the LevelMeter fails to open the mic (e.g. device busy
|
||||
# because the wakeword listener already holds a pw-record stream), the
|
||||
# on_level callback never fires and dictation hangs forever. Add a hard
|
||||
# 30-second timeout so the session always terminates.
|
||||
_MAX_WAKEWORD_SECONDS = 30
|
||||
if not ok:
|
||||
log("[vad] LevelMeter failed to start — scheduling 30s hard timeout")
|
||||
GLib.timeout_add(_MAX_WAKEWORD_SECONDS * 1000,
|
||||
lambda: self.finish_dictation(send_enter=False) or False)
|
||||
else:
|
||||
# Even when the meter works, cap wakeword sessions at 60s.
|
||||
GLib.timeout_add(60_000,
|
||||
lambda: self.is_recording and self.finish_dictation(send_enter=False) or False)
|
||||
|
||||
def _vad_stop(self) -> None:
|
||||
if getattr(self, '_vad_meter', None) is not None:
|
||||
self._vad_meter.stop()
|
||||
self._vad_meter = None
|
||||
if getattr(self, "_cancel_watcher", None) is not None:
|
||||
self._cancel_watcher.stop()
|
||||
self._cancel_watcher = None
|
||||
if getattr(self, "_action_listener", None) is not None:
|
||||
self._action_listener.stop()
|
||||
self._action_listener = None
|
||||
|
||||
def _ov_meter_start(self) -> None:
|
||||
"""A level meter purely to drive the overlay waveform in streaming mode.
|
||||
|
||||
Non-streaming recordings reuse the VAD meter instead; this only runs when
|
||||
an overlay is attached and we have no other level source. Best-effort:
|
||||
LevelMeter.start() fails quietly if the device is busy."""
|
||||
if not self.level_cb:
|
||||
return
|
||||
from . import audio
|
||||
self._ov_meter = audio.LevelMeter(self.cfg.mic, on_level=self.level_cb, recorder=self.recorder_name)
|
||||
self._ov_meter.start()
|
||||
|
||||
def _ov_meter_stop(self) -> None:
|
||||
if getattr(self, "_ov_meter", None) is not None:
|
||||
self._ov_meter.stop()
|
||||
self._ov_meter = None
|
||||
|
||||
def _play_sound(self, sound_name: str) -> None:
|
||||
if not self.cfg.sounds_enabled:
|
||||
@ -198,6 +446,7 @@ class Daemon:
|
||||
self._notify("Streaming unavailable", "Select a riva_realtime STT engine.", "critical")
|
||||
return
|
||||
self._stream_segment_text = ""
|
||||
self._ov_text_final = ""
|
||||
streamer = RivaRealtimeStreamer(
|
||||
engine,
|
||||
device=self.cfg.mic,
|
||||
@ -216,7 +465,9 @@ class Daemon:
|
||||
self._dnotify(f"● {wf.name}", "Live transcript…")
|
||||
try:
|
||||
streamer.start()
|
||||
self._ov_meter_start()
|
||||
except Exception as exc: # noqa: BLE001
|
||||
self._ov_meter_stop()
|
||||
with self._lock:
|
||||
if self._streaming is streamer:
|
||||
self._streaming = None
|
||||
@ -247,6 +498,7 @@ class Daemon:
|
||||
self._busy = True
|
||||
if streamer is not None:
|
||||
streamer.stop()
|
||||
self._ov_meter_stop()
|
||||
if send_enter:
|
||||
from .paste import press_enter
|
||||
press_enter(win)
|
||||
@ -269,19 +521,31 @@ class Daemon:
|
||||
self._active_workflow = None
|
||||
self._stream_segment_text = ""
|
||||
rec = None
|
||||
else:
|
||||
busy = False
|
||||
elif self._recording is not None:
|
||||
streamer = None
|
||||
if self._recording is None:
|
||||
return
|
||||
rec = self._recording
|
||||
self._recording = None
|
||||
self._active_workflow = None
|
||||
busy = False
|
||||
elif self._busy:
|
||||
streamer = None
|
||||
rec = None
|
||||
busy = True
|
||||
else:
|
||||
return
|
||||
if streamer is not None:
|
||||
streamer.stop()
|
||||
self._ov_meter_stop()
|
||||
self._emit("idle", None, "Cancelled")
|
||||
self._notify("Cancelled", "Streaming stopped.", "low")
|
||||
self._play_sound("device-removed")
|
||||
return
|
||||
if busy:
|
||||
self._abort_event.set()
|
||||
self._emit("idle", None, "Cancelled")
|
||||
self._play_sound("device-removed")
|
||||
return
|
||||
rec.discard()
|
||||
self._emit("idle", None, "Cancelled")
|
||||
self._notify("Cancelled", "Recording discarded.", "low")
|
||||
@ -317,6 +581,14 @@ class Daemon:
|
||||
return text[:cut + 1] if cut >= 0 else ""
|
||||
|
||||
def _on_stream_text(self, text: str, final: bool) -> None:
|
||||
# Mirror the live hypothesis into the overlay bubble (the full current
|
||||
# guess, which is more responsive than the delivered stable prefix).
|
||||
if self.text_cb:
|
||||
disp_seg = quality.clean(text, strip_trailing_punctuation=False)
|
||||
running = (self._ov_text_final + " " + disp_seg).strip()
|
||||
self.text_cb(running)
|
||||
if final:
|
||||
self._ov_text_final = running
|
||||
stable = self._stable_stream_text(text, final)
|
||||
if not stable:
|
||||
return
|
||||
@ -337,6 +609,7 @@ class Daemon:
|
||||
self._stream_segment_text = "" if final else stable
|
||||
|
||||
def _on_stream_error(self, label: str, exc: Exception) -> None:
|
||||
self._ov_meter_stop()
|
||||
with self._lock:
|
||||
self._streaming = None
|
||||
self._active_workflow = None
|
||||
@ -347,6 +620,7 @@ class Daemon:
|
||||
|
||||
# -- worker ---------------------------------------------------------------
|
||||
def _process(self, audio_path, workflow: Workflow, window_id, send_enter: bool = False) -> None:
|
||||
self._abort_event.clear()
|
||||
label = workflow.name
|
||||
try:
|
||||
# Quality gate: drop silent / too-short clips before we even transcribe.
|
||||
@ -369,22 +643,55 @@ class Daemon:
|
||||
local_transcriber=self.transcriber,
|
||||
timeout=self.cfg.timeout,
|
||||
)
|
||||
if self._abort_event.is_set():
|
||||
log(f"✗ {label}: cancelled during transcription.")
|
||||
return
|
||||
|
||||
text = quality.clean(text, strip_trailing_punctuation=self.cfg.strip_trailing_punctuation)
|
||||
text = quality.expand_spoken_punctuation(text)
|
||||
if not text or (self.cfg.reject_hallucinations and quality.is_hallucination(text, duration)):
|
||||
self._emit("idle", label, "No speech detected")
|
||||
log("Nothing heard — no speech detected.")
|
||||
return
|
||||
|
||||
# Spoken abort: a configured cancel word heard at an edge discards the
|
||||
# whole clip — nothing is routed, rewritten, or typed. This is the
|
||||
# rescue for an accidentally triggered (e.g. wakeword) dictation.
|
||||
cancel_kw = is_cancel(text, self.cfg.cancel_keywords, threshold=self.cfg.routing_threshold)
|
||||
if cancel_kw:
|
||||
log(f"✗ Discarded by voice keyword “{cancel_kw}”.")
|
||||
if self.text_cb:
|
||||
self.text_cb("✗ Abgebrochen")
|
||||
self._emit("idle", label, "Cancelled")
|
||||
self._dnotify("Abgebrochen", f"„{cancel_kw}“ gehört — verworfen.", "low")
|
||||
return
|
||||
|
||||
# Spoken send: a configured word at an edge ("computer send") is
|
||||
# stripped, and the rest is delivered AND submitted with Enter — the
|
||||
# spoken equivalent of stop+paste+Enter. Mainly for hands-free use.
|
||||
send_kw, text = match_send(text, self.cfg.send_keywords, threshold=self.cfg.routing_threshold)
|
||||
if send_kw:
|
||||
send_enter = True
|
||||
log(f"⏎ Send keyword “{send_kw}” — delivering and pressing Enter.")
|
||||
|
||||
# Voice routing: pick the preset from a spoken keyword, strip it.
|
||||
if workflow.mode == "route":
|
||||
res = route(text, self.cfg.workflows, threshold=self.cfg.routing_threshold)
|
||||
target = self.cfg.preset_by_name(res.preset_name) or self.cfg.default_preset
|
||||
text = res.text
|
||||
label = target.name if target else "Transcribe"
|
||||
via = f"“{res.keyword}”" if res.keyword else "default"
|
||||
icon = (getattr(target, "icon", "") or "🎙") if target else "🎙"
|
||||
via = f"“{res.keyword}”" if res.keyword else "no keyword → default"
|
||||
self._emit("busy", label, f"→ {label} ({via})")
|
||||
self._dnotify(f"🎙 {label}", f"matched: {via}")
|
||||
# Fuse the match onto the overlay (icon + preset + keyword); falls
|
||||
# back to a desktop notification only when there's no overlay.
|
||||
if self.routing_cb:
|
||||
try:
|
||||
self.routing_cb(icon, label, res.keyword)
|
||||
except Exception: # noqa: BLE001 - UI must not break the engine
|
||||
pass
|
||||
self._rnotify(f"{icon} {label}", f"matched: {via}")
|
||||
log(f"→ routed to {label} (matched: {via})")
|
||||
else:
|
||||
target = workflow
|
||||
|
||||
@ -395,19 +702,98 @@ class Daemon:
|
||||
return
|
||||
self._emit("busy", label, "Rewriting…")
|
||||
self._dnotify(f"⌛ {label}", "Rewriting…")
|
||||
# Show the transcribed text immediately so the user sees what was heard.
|
||||
if self.text_cb:
|
||||
self.text_cb(f"📝 {text}")
|
||||
# Stream the rewrite into the overlay so you watch the model write
|
||||
# (the bubble updates token-by-token). The delivered text is still
|
||||
# the complete result, typed once the rewrite finishes.
|
||||
on_token = None
|
||||
if self.text_cb:
|
||||
# Use a deque to stream the last ~400 chars to the overlay so
|
||||
# Pango never has to lay out a 15 000-char code block on each
|
||||
# token, and "".join() stays O(window) not O(total).
|
||||
from collections import deque
|
||||
_window: deque[str] = deque()
|
||||
_window_len: list[int] = [0]
|
||||
_OVERLAY_CHARS = 400
|
||||
|
||||
_thinking_frames = ["⏳ Thinking.", "⏳ Thinking..", "⏳ Thinking...", "⏳ Thinking"]
|
||||
_thinking_state: list[int] = [0]
|
||||
_first_token: list[bool] = [False]
|
||||
|
||||
def _pulse_thinking(_s=_thinking_state, _f=_first_token) -> bool:
|
||||
if _f[0]:
|
||||
return False
|
||||
self.text_cb(_thinking_frames[_s[0] % len(_thinking_frames)])
|
||||
_s[0] += 1
|
||||
return True
|
||||
|
||||
try:
|
||||
from gi.repository import GLib as _GLib
|
||||
# idle_add ensures timeout_add runs on the GTK main thread —
|
||||
# calling timeout_add from a background thread is not safe.
|
||||
_GLib.idle_add(lambda: _GLib.timeout_add(400, _pulse_thinking) and False)
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
|
||||
def on_token(delta: str,
|
||||
_w=_window, _wl=_window_len, _f=_first_token) -> None:
|
||||
_f[0] = True
|
||||
_w.append(delta)
|
||||
_wl[0] += len(delta)
|
||||
# Trim old tokens from the front to stay within the window.
|
||||
while _wl[0] > _OVERLAY_CHARS and len(_w) > 1:
|
||||
_wl[0] -= len(_w[0])
|
||||
_w.popleft()
|
||||
self.text_cb("".join(_w))
|
||||
|
||||
# Use the preset's pinned engine when set, else the active one.
|
||||
engine_name = getattr(target, "llm_engine", "") or ""
|
||||
llm_engine = (
|
||||
next((e for e in self.cfg.llm_engines if e.name == engine_name), None)
|
||||
or self.cfg.active_llm
|
||||
)
|
||||
try:
|
||||
text = llm.chat(
|
||||
self.cfg.active_llm,
|
||||
llm_engine,
|
||||
target.prompt,
|
||||
text,
|
||||
model=target.model or None,
|
||||
temperature=target.temperature,
|
||||
timeout=self.cfg.timeout,
|
||||
on_token=on_token,
|
||||
abort_event=self._abort_event,
|
||||
)
|
||||
except LLMError as exc:
|
||||
self._emit("error", label, str(exc))
|
||||
self._dnotify("Rewrite failed", str(exc), "critical")
|
||||
log(f"ERROR ({label} rewrite): {exc}")
|
||||
if self.text_cb:
|
||||
self.text_cb(f"✗ {exc}")
|
||||
return
|
||||
except Exception as exc: # noqa: BLE001 - guard against any uncaught error
|
||||
msg = f"LLM error: {exc}"
|
||||
self._emit("error", label, msg)
|
||||
log(f"ERROR ({label} rewrite unexpected): {exc}")
|
||||
if self.text_cb:
|
||||
self.text_cb(f"✗ {msg}")
|
||||
return
|
||||
|
||||
if self._abort_event.is_set():
|
||||
log(f"✗ {label}: cancelled during rewrite.")
|
||||
return
|
||||
|
||||
# Sanity-check: reject responses that are >80 % whitespace —
|
||||
# a model that's cold-starting or misconfigured sometimes streams
|
||||
# spaces or blank lines instead of real output.
|
||||
non_ws = sum(1 for c in text if not c.isspace())
|
||||
if non_ws < max(1, len(text) * 0.20):
|
||||
msg = "LLM returned mostly whitespace — discarded"
|
||||
self._emit("error", label, msg)
|
||||
log(f"ERROR ({label}): {msg} (len={len(text)})")
|
||||
if self.text_cb:
|
||||
self.text_cb(f"✗ {msg}")
|
||||
return
|
||||
|
||||
if not text:
|
||||
|
||||
@ -85,13 +85,47 @@ class App:
|
||||
self.cfg = cfg
|
||||
self.tray_mode = tray_mode
|
||||
self.tray = None
|
||||
self.daemon = Daemon(cfg, status_cb=self._status_cb)
|
||||
self._rows: dict[str, dict] = {}
|
||||
self._active: str | None = None
|
||||
|
||||
# On-screen dictation HUD (mic + waveform + recognised-text bubble).
|
||||
self.overlay = None
|
||||
self._ov_state = "idle"
|
||||
self._ov_streaming = False
|
||||
if cfg.overlay_enabled:
|
||||
try:
|
||||
from .overlay import Overlay
|
||||
|
||||
self.overlay = Overlay(
|
||||
anchor_mode=cfg.overlay_anchor,
|
||||
on_cancel=lambda: threading.Thread(
|
||||
target=self.daemon.cancel_dictation, daemon=True).start(),
|
||||
)
|
||||
if cfg.overlay_anchor == "caret":
|
||||
from . import caret
|
||||
|
||||
caret.start_tracking()
|
||||
except Exception: # noqa: BLE001 - overlay is optional eye-candy
|
||||
self.overlay = None
|
||||
|
||||
self.daemon = Daemon(
|
||||
cfg, status_cb=self._status_cb,
|
||||
level_cb=self._on_level, text_cb=self._on_text,
|
||||
countdown_cb=self._on_countdown,
|
||||
# Only consume routing on the overlay when there's an overlay to show
|
||||
# it on; otherwise the daemon keeps the desktop notification.
|
||||
routing_cb=self._on_routing if self.overlay is not None else None,
|
||||
)
|
||||
|
||||
_install_css()
|
||||
self._build_window()
|
||||
|
||||
# Show the first-run wizard for fresh installs before the daemon starts.
|
||||
if not getattr(cfg, "setup_complete", False):
|
||||
from .setup_wizard import SetupWizard
|
||||
wiz = SetupWizard(cfg, parent=None)
|
||||
wiz.run()
|
||||
|
||||
threading.Thread(target=self._startup, daemon=True).start()
|
||||
|
||||
# -- window ---------------------------------------------------------------
|
||||
@ -242,6 +276,55 @@ class App:
|
||||
return
|
||||
threading.Thread(target=lambda: self.daemon.toggle(wf), daemon=True).start()
|
||||
|
||||
# -- overlay feedback -----------------------------------------------------
|
||||
def _on_level(self, level: float) -> None:
|
||||
if self.overlay is not None:
|
||||
self.overlay.set_level(level)
|
||||
|
||||
def _on_text(self, text: str) -> None:
|
||||
if self.overlay is not None:
|
||||
self.overlay.set_text(text)
|
||||
|
||||
def _on_countdown(self, remaining: float | None, total: float) -> None:
|
||||
if self.overlay is not None:
|
||||
self.overlay.set_countdown(remaining, total)
|
||||
|
||||
def _on_routing(self, icon: str, name: str, keyword: str | None) -> None:
|
||||
if self.overlay is not None:
|
||||
self.overlay.set_preset(icon, name, keyword)
|
||||
|
||||
def _overlay_status(self, state: str, message: str) -> None:
|
||||
"""Translate engine phases into overlay show/update/hide (GTK thread)."""
|
||||
ov = self.overlay
|
||||
if ov is None:
|
||||
return
|
||||
if state in ("recording", "streaming"):
|
||||
self._ov_streaming = state == "streaming"
|
||||
self._ov_state = state
|
||||
ov.show(state, getattr(self.daemon, "_target_window", None))
|
||||
elif state == "busy":
|
||||
self._ov_state = state
|
||||
# The routing detail ("→ Nicer email (matched: …)") is shown on the
|
||||
# preset banner, not as a phase chip — keep the chip a clean phase word.
|
||||
phase = "Transcribing…" if message.startswith("→") else message
|
||||
ov.set_state("busy", phase)
|
||||
elif state == "done":
|
||||
# Non-streaming: the 'done' message carries the final text. Streaming
|
||||
# already showed it live, so don't overwrite with "Streaming stopped".
|
||||
if not self._ov_streaming and message:
|
||||
ov.set_text(message)
|
||||
self._ov_state = state
|
||||
ov.set_state("done", message)
|
||||
elif state == "error":
|
||||
self._ov_state = state
|
||||
ov.set_state("error", message)
|
||||
elif state == "idle":
|
||||
# Only when a session was actually live — and never clip a 'done'
|
||||
# linger (idle is emitted right after done in the worker's finally).
|
||||
if self._ov_state in ("recording", "streaming", "busy"):
|
||||
ov.set_state("idle", message)
|
||||
self._ov_state = "idle"
|
||||
|
||||
# -- status (marshalled to GTK thread) ------------------------------------
|
||||
def _status_cb(self, state: str, workflow: str | None, message: str) -> None:
|
||||
GLib.idle_add(self._apply_status, state, workflow, message)
|
||||
@ -257,6 +340,8 @@ class App:
|
||||
self._set_dot(colors.get(state, "#7b818b"))
|
||||
self.status_lbl.set_text(labels.get(state, message))
|
||||
|
||||
self._overlay_status(state, message)
|
||||
|
||||
if self.tray is not None:
|
||||
self.tray.update_status(state, labels.get(state, message))
|
||||
|
||||
@ -291,9 +376,18 @@ class App:
|
||||
|
||||
# -- panel / settings / lifecycle -----------------------------------------
|
||||
def open_settings(self) -> None:
|
||||
# Single instance: if Settings is already open, bring it to the front
|
||||
# instead of spawning a second dialog (the tray menu bypasses the
|
||||
# dialog's own modality, so it could otherwise be opened repeatedly).
|
||||
existing = getattr(self, "_settings", None)
|
||||
if existing is not None:
|
||||
existing.dlg.present()
|
||||
return
|
||||
from .gtksettings import SettingsDialog
|
||||
|
||||
SettingsDialog(self.win, self.cfg, daemon=self.daemon).run_dialog()
|
||||
self._settings = SettingsDialog(self.win, self.cfg, daemon=self.daemon)
|
||||
self._settings.dlg.connect("destroy", lambda *_: setattr(self, "_settings", None))
|
||||
self._settings.run_dialog()
|
||||
|
||||
def show_panel(self) -> None:
|
||||
self.win.show_all()
|
||||
@ -312,6 +406,11 @@ class App:
|
||||
def quit_all(self) -> None:
|
||||
try:
|
||||
self.daemon.stop_input()
|
||||
if self.overlay is not None:
|
||||
self.overlay.destroy()
|
||||
from . import caret
|
||||
|
||||
caret.stop_tracking()
|
||||
finally:
|
||||
Gtk.main_quit()
|
||||
|
||||
@ -328,6 +427,21 @@ class App:
|
||||
|
||||
|
||||
def run_gui(tray_mode: bool = False) -> int:
|
||||
# Use GIO's native /proc/mounts volume monitor instead of the gvfs/udisks2
|
||||
# one. On headless or minimal desktops the `org.gtk.vfs.UDisks2VolumeMonitor`
|
||||
# dbus service often fails to activate, and every Gtk.FileChooserButton then
|
||||
# blocks ~25s on a StartServiceByName timeout while realizing — which freezes
|
||||
# the settings dialog (and, via the stalled main loop, the panel) so neither
|
||||
# ever appears. The unix monitor needs no dbus and opens choosers instantly.
|
||||
os.environ.setdefault("GIO_USE_VOLUME_MONITOR", "unix")
|
||||
# Identify to the window manager as "blitztext" rather than the Python entry
|
||||
# point's filename. Launched via `python -m blitztext`, GTK's default program
|
||||
# name is argv[0]'s basename ("__main__.py"), which is what shows in the
|
||||
# taskbar and in GNOME's "… is not responding" dialog. Setting it here (before
|
||||
# any window is realized) gives every window the app's real name + .desktop
|
||||
# match, without touching the `-m blitztext` entry point.
|
||||
GLib.set_prgname("blitztext")
|
||||
GLib.set_application_name("Blitztext")
|
||||
cfg = load()
|
||||
App(cfg, tray_mode=tray_mode).run()
|
||||
return 0
|
||||
|
||||
@ -74,18 +74,21 @@ class ModifierScheme:
|
||||
return
|
||||
self._pressed.add(token)
|
||||
|
||||
# Cancel works while arming/armed AND when the daemon is recording via
|
||||
# wakeword (state stays "idle" in the scheme because wakeword bypasses
|
||||
# the key-press path entirely).
|
||||
if self._is(token, self.cancel) and (
|
||||
self._state != "idle" or self.daemon.is_recording):
|
||||
self._state = "idle"
|
||||
self.daemon.cancel_dictation()
|
||||
return
|
||||
|
||||
if self._state == "idle":
|
||||
if self.start.issubset(self._pressed):
|
||||
self._state = "arming"
|
||||
self.daemon.start_dictation()
|
||||
return
|
||||
|
||||
# Cancel works while arming or armed.
|
||||
if self._is(token, self.cancel):
|
||||
self._state = "idle"
|
||||
self.daemon.cancel_dictation()
|
||||
return
|
||||
|
||||
if self._state == "armed":
|
||||
if self._is(token, self.send):
|
||||
self._state = "idle"
|
||||
|
||||
@ -13,6 +13,7 @@ import os
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from dataclasses import dataclass
|
||||
from typing import Callable
|
||||
|
||||
from .stt import reachable
|
||||
|
||||
@ -48,10 +49,19 @@ def chat(
|
||||
model: str | None = None,
|
||||
temperature: float | None = None,
|
||||
timeout: int = 45,
|
||||
on_token: Callable[[str], None] | None = None,
|
||||
abort_event=None,
|
||||
) -> str:
|
||||
"""Run a chat completion and return the full text.
|
||||
|
||||
When ``on_token`` is given, the request is streamed and each content delta is
|
||||
handed to the callback as it arrives (so a UI can show the model writing in
|
||||
real time). The callback is best-effort — it never affects the return value,
|
||||
which is always the complete, stripped response.
|
||||
"""
|
||||
stream = on_token is not None
|
||||
api_key = engine.api_key
|
||||
payload = json.dumps(
|
||||
{
|
||||
body_obj = {
|
||||
"model": model or engine.model,
|
||||
"temperature": engine.temperature if temperature is None else temperature,
|
||||
"messages": [
|
||||
@ -59,7 +69,9 @@ def chat(
|
||||
{"role": "user", "content": user_text},
|
||||
],
|
||||
}
|
||||
).encode("utf-8")
|
||||
if stream:
|
||||
body_obj["stream"] = True
|
||||
payload = json.dumps(body_obj).encode("utf-8")
|
||||
|
||||
headers = {"Content-Type": "application/json"}
|
||||
if api_key:
|
||||
@ -70,19 +82,55 @@ def chat(
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
if stream:
|
||||
content = _read_stream(resp, on_token, abort_event)
|
||||
else:
|
||||
body = json.loads(resp.read().decode("utf-8"))
|
||||
content = body["choices"][0]["message"]["content"]
|
||||
except urllib.error.HTTPError as exc:
|
||||
detail = exc.read().decode("utf-8", "replace")[:300]
|
||||
raise LLMError(f"HTTP {exc.code}: {detail}") from exc
|
||||
except urllib.error.URLError as exc:
|
||||
raise LLMError(f"Connection failed: {exc.reason}") from exc
|
||||
|
||||
try:
|
||||
content = body["choices"][0]["message"]["content"]
|
||||
except (KeyError, IndexError, TypeError) as exc:
|
||||
raise LLMError(f"Unexpected response: {str(body)[:300]}") from exc
|
||||
raise LLMError(f"Unexpected response: {exc}") from exc
|
||||
except (TimeoutError, OSError) as exc:
|
||||
# socket.timeout (subclass of OSError / TimeoutError) fires when the server
|
||||
# stops sending data mid-stream. Not wrapped in URLError — must be caught
|
||||
# separately or it would propagate uncaught and kill the background thread.
|
||||
raise LLMError(f"Request timed out or connection lost: {exc}") from exc
|
||||
except Exception as exc: # noqa: BLE001
|
||||
raise LLMError(f"Unexpected error: {exc}") from exc
|
||||
|
||||
content = (content or "").strip()
|
||||
if not content:
|
||||
raise LLMError("Empty response from model.")
|
||||
return content
|
||||
|
||||
|
||||
def _read_stream(resp, on_token: Callable[[str], None], abort_event=None) -> str:
|
||||
"""Parse an OpenAI-style SSE stream, returning the accumulated content and
|
||||
feeding each delta to ``on_token``. Tolerant of keep-alive blanks and the
|
||||
trailing ``[DONE]`` sentinel."""
|
||||
parts: list[str] = []
|
||||
for raw in resp:
|
||||
if abort_event and abort_event.is_set():
|
||||
break
|
||||
line = raw.decode("utf-8", "replace").strip()
|
||||
if not line or not line.startswith("data:"):
|
||||
continue
|
||||
data = line[len("data:"):].strip()
|
||||
if data == "[DONE]":
|
||||
break
|
||||
try:
|
||||
obj = json.loads(data)
|
||||
delta = obj["choices"][0]["delta"].get("content")
|
||||
except (ValueError, KeyError, IndexError, TypeError):
|
||||
continue
|
||||
if delta:
|
||||
parts.append(delta)
|
||||
try:
|
||||
on_token(delta)
|
||||
except Exception: # noqa: BLE001 - UI hiccups must not break delivery
|
||||
pass
|
||||
return "".join(parts)
|
||||
|
||||
@ -3,6 +3,9 @@
|
||||
Our own messages go through log(); library logs (faster-whisper, huggingface_hub)
|
||||
are captured via a logging handler so model download/load progress is visible
|
||||
instead of an opaque "Loading…".
|
||||
|
||||
Each entry stores (timestamp_str, level, message) so the UI can filter by level.
|
||||
Levels: DEBUG, INFO, WARNING, ERROR (default: INFO)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@ -13,32 +16,53 @@ import threading
|
||||
import time
|
||||
from collections import deque
|
||||
|
||||
_LINES: deque[str] = deque(maxlen=2000)
|
||||
# Each entry: (time_str, level, message)
|
||||
_ENTRIES: deque[tuple[str, str, str]] = deque(maxlen=2000)
|
||||
_LOCK = threading.Lock()
|
||||
|
||||
_LEVEL_ORDER = {"DEBUG": 0, "INFO": 1, "WARNING": 2, "ERROR": 3}
|
||||
|
||||
def log(msg: str, *, echo: bool = True) -> None:
|
||||
line = f"{time.strftime('%H:%M:%S')} {msg}"
|
||||
|
||||
def log(msg: str, *, echo: bool = True, level: str = "INFO") -> None:
|
||||
level = level.upper()
|
||||
ts = time.strftime("%H:%M:%S")
|
||||
with _LOCK:
|
||||
_LINES.append(line)
|
||||
_ENTRIES.append((ts, level, msg))
|
||||
if echo:
|
||||
print(line, file=sys.stderr, flush=True)
|
||||
print(f"{ts} [{level}] {msg}", file=sys.stderr, flush=True)
|
||||
|
||||
|
||||
def lines() -> list[str]:
|
||||
def lines(min_level: str = "DEBUG") -> list[str]:
|
||||
"""Return formatted lines at or above min_level."""
|
||||
threshold = _LEVEL_ORDER.get(min_level.upper(), 0)
|
||||
with _LOCK:
|
||||
return list(_LINES)
|
||||
entries = list(_ENTRIES)
|
||||
result = []
|
||||
for ts, lvl, msg in entries:
|
||||
if _LEVEL_ORDER.get(lvl, 1) >= threshold:
|
||||
prefix = f"[{lvl}] " if lvl not in ("INFO",) else ""
|
||||
result.append(f"{ts} {prefix}{msg}")
|
||||
return result
|
||||
|
||||
|
||||
def clear() -> None:
|
||||
with _LOCK:
|
||||
_LINES.clear()
|
||||
_ENTRIES.clear()
|
||||
|
||||
|
||||
class _BufferHandler(logging.Handler):
|
||||
_PY_TO_LEVEL = {
|
||||
logging.DEBUG: "DEBUG",
|
||||
logging.INFO: "INFO",
|
||||
logging.WARNING: "WARNING",
|
||||
logging.ERROR: "ERROR",
|
||||
logging.CRITICAL:"ERROR",
|
||||
}
|
||||
|
||||
def emit(self, record: logging.LogRecord) -> None:
|
||||
try:
|
||||
log(self.format(record), echo=False)
|
||||
lvl = self._PY_TO_LEVEL.get(record.levelno, "INFO")
|
||||
log(self.format(record), echo=False, level=lvl)
|
||||
except Exception: # noqa: BLE001 - never let logging break the app
|
||||
pass
|
||||
|
||||
@ -57,4 +81,4 @@ def install_logging() -> None:
|
||||
for name in ("faster_whisper", "huggingface_hub", "blitztext"):
|
||||
lg = logging.getLogger(name)
|
||||
lg.addHandler(handler)
|
||||
lg.setLevel(logging.INFO)
|
||||
lg.setLevel(logging.DEBUG)
|
||||
|
||||
602
linux/blitztext/overlay.py
Normal file
@ -0,0 +1,602 @@
|
||||
"""On-screen dictation HUD: a translucent bubble at the cursor.
|
||||
|
||||
Shows, while you dictate:
|
||||
• a microphone glyph that pulses red as it listens,
|
||||
• a live waveform driven by the real mic level,
|
||||
• the recognised text (word-by-word in streaming mode, or the final result as
|
||||
a brief confirmation in record-then-transcribe mode),
|
||||
with a little tail whose tip points at the cursor where the text will land
|
||||
(see :mod:`blitztext.caret` for how that anchor is resolved).
|
||||
|
||||
It is a click-through, focus-free override-redirect window so it never steals
|
||||
input from the field you're dictating into. All public methods are safe to call
|
||||
from worker threads — they marshal onto the GTK main loop via ``GLib.idle_add``.
|
||||
Pure feedback: nothing here touches recording, transcription, or delivery, and
|
||||
the whole feature is gated by ``cfg.overlay_enabled`` in the caller.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
import time
|
||||
from collections import deque
|
||||
|
||||
import cairo
|
||||
import gi
|
||||
|
||||
gi.require_version("Gtk", "3.0")
|
||||
gi.require_version("Gdk", "3.0")
|
||||
gi.require_version("PangoCairo", "1.0")
|
||||
from gi.repository import Gdk, GLib, Gtk, Pango, PangoCairo # noqa: E402
|
||||
|
||||
from . import caret # noqa: E402
|
||||
|
||||
# Layout constants (logical px).
|
||||
_WIDTH = 360
|
||||
_PAD = 16
|
||||
_HEADER_H = 40 # mic + waveform row
|
||||
_RADIUS = 16
|
||||
_TAIL_W = 20
|
||||
_TAIL_H = 11
|
||||
_GAP = 12 # clearance between the tail tip and the anchor
|
||||
_BARS = 30 # waveform bar count
|
||||
_PRESET_H = 22 # matched-preset header row (emoji + name + keyword)
|
||||
_MIN_TEXT_H = 0
|
||||
_MAX_TEXT_H = 120
|
||||
|
||||
_FPS_MS = 33 # ~30 fps animation tick
|
||||
|
||||
# Phase → (mic colour, label). Recording/streaming pulse; others are steady.
|
||||
_PHASES = {
|
||||
"recording": ((1.0, 0.27, 0.23), "Listening…"),
|
||||
"streaming": ((1.0, 0.27, 0.23), "Listening…"),
|
||||
"busy": ((1.0, 0.74, 0.16), "Transcribing…"),
|
||||
"done": ((0.30, 0.80, 0.36), ""),
|
||||
"error": ((1.0, 0.35, 0.35), "Error"),
|
||||
}
|
||||
|
||||
|
||||
_CANCEL_BTN_R = 11 # hit-radius of the × button (px)
|
||||
|
||||
class Overlay:
|
||||
def __init__(self, anchor_mode: str = "caret", on_cancel=None) -> None:
|
||||
self.anchor_mode = anchor_mode
|
||||
self._on_cancel_cb = on_cancel # callable() → cancels current recording
|
||||
self._visible = False
|
||||
self._state = "recording"
|
||||
self._text = ""
|
||||
self._phase_label = "Listening…"
|
||||
# Matched-preset banner (fused in from voice routing instead of a separate
|
||||
# desktop notification): the preset's emoji, its name, and the spoken
|
||||
# keyword that selected it.
|
||||
self._preset_icon = ""
|
||||
self._preset_name = ""
|
||||
self._keyword = ""
|
||||
self._anchor: caret.Anchor | None = None
|
||||
self._tail_up = False # tail on top edge (bubble below anchor)?
|
||||
self._tail_x = _WIDTH // 2 # tail tip, window-local x
|
||||
self._height = _HEADER_H + 2 * _PAD + _TAIL_H
|
||||
self._levels: deque[float] = deque([0.0] * _BARS, maxlen=_BARS)
|
||||
self._disp = [0.0] * _BARS # eased bar heights for smooth motion
|
||||
self._pulse = 0.0
|
||||
# Silence auto-stop countdown ring (wraps the mic): a deadline the ring
|
||||
# drains toward, the window it spans, and an eased opacity so it fades in
|
||||
# when you fall quiet and out the moment you speak again.
|
||||
self._cd_deadline: float | None = None
|
||||
self._cd_total = 1.0
|
||||
self._cd_frac = 0.0
|
||||
self._cd_alpha = 0.0
|
||||
self._tick_id: int | None = None
|
||||
self._hide_id: int | None = None
|
||||
self._t0 = time.time()
|
||||
# Coalescing text updates: background LLM streaming can fire dozens of
|
||||
# set_text() calls per second. We buffer the latest text and only ever
|
||||
# have ONE idle_add pending — so the GTK main loop is never flooded.
|
||||
self._pending_text: str = ""
|
||||
self._text_flush_queued: bool = False
|
||||
self._pending_level: float = 0.0
|
||||
self._level_flush_queued: bool = False
|
||||
|
||||
self._cancel_btn_rect = (0, 0, 0, 0) # (x, y, w, h) in window coords
|
||||
|
||||
self._win = Gtk.Window(type=Gtk.WindowType.POPUP)
|
||||
self._win.set_app_paintable(True)
|
||||
self._win.set_resizable(False)
|
||||
self._win.set_skip_taskbar_hint(True)
|
||||
self._win.set_skip_pager_hint(True)
|
||||
self._win.set_accept_focus(False)
|
||||
self._win.set_focus_on_map(False)
|
||||
self._win.set_keep_above(True)
|
||||
screen = self._win.get_screen()
|
||||
visual = screen.get_rgba_visual() if screen else None
|
||||
if visual is not None:
|
||||
self._win.set_visual(visual)
|
||||
self._area = Gtk.DrawingArea()
|
||||
if visual is not None:
|
||||
self._area.set_visual(visual)
|
||||
self._area.connect("draw", self._on_draw)
|
||||
self._win.add(self._area)
|
||||
self._win.connect("realize", self._on_realize)
|
||||
self._win.add_events(Gdk.EventMask.BUTTON_PRESS_MASK)
|
||||
self._win.connect("button-press-event", self._on_click)
|
||||
self._win.set_default_size(_WIDTH, self._height)
|
||||
|
||||
# -- click-through (all except the × button) ------------------------------
|
||||
def _on_realize(self, _w) -> None:
|
||||
self._update_input_region()
|
||||
|
||||
def _update_input_region(self) -> None:
|
||||
gdkwin = self._win.get_window()
|
||||
if gdkwin is None:
|
||||
return
|
||||
if self._on_cancel_cb and self._state in ("recording", "streaming", "busy"):
|
||||
x, y, w, h = self._cancel_btn_rect
|
||||
r = cairo.Region(cairo.RectangleInt(int(x), int(y), int(w), int(h)))
|
||||
else:
|
||||
r = cairo.Region() # empty → fully click-through
|
||||
gdkwin.input_shape_combine_region(r, 0, 0)
|
||||
|
||||
def _on_click(self, _win, event) -> bool:
|
||||
if self._on_cancel_cb is None:
|
||||
return False
|
||||
x, y, w, h = self._cancel_btn_rect
|
||||
if x <= event.x <= x + w and y <= event.y <= y + h:
|
||||
self._on_cancel_cb()
|
||||
return True
|
||||
|
||||
# -- thread-safe public API ----------------------------------------------
|
||||
def show(self, state: str, window_id: str | None) -> None:
|
||||
GLib.idle_add(self._show, state, window_id)
|
||||
|
||||
def set_level(self, level: float) -> None:
|
||||
self._pending_level = float(level)
|
||||
if not self._level_flush_queued:
|
||||
self._level_flush_queued = True
|
||||
GLib.idle_add(self._flush_level)
|
||||
|
||||
def _flush_level(self) -> bool:
|
||||
self._level_flush_queued = False
|
||||
self._set_level(self._pending_level)
|
||||
return False
|
||||
|
||||
def set_text(self, text: str) -> None:
|
||||
self._pending_text = text or ""
|
||||
# Only schedule a flush if none is already queued; this collapses a burst
|
||||
# of token callbacks (e.g. 50/s from LLM streaming) into a single GTK
|
||||
# redraw, preventing main-loop flooding and session freezes.
|
||||
if not self._text_flush_queued:
|
||||
self._text_flush_queued = True
|
||||
GLib.idle_add(self._flush_text)
|
||||
|
||||
def _flush_text(self) -> bool:
|
||||
self._text_flush_queued = False
|
||||
self._set_text(self._pending_text)
|
||||
return False
|
||||
|
||||
def set_preset(self, icon: str, name: str, keyword: str | None) -> None:
|
||||
"""Show the matched voice-routing preset on the overlay (emoji + name +
|
||||
the spoken keyword), in place of a separate desktop notification."""
|
||||
GLib.idle_add(self._set_preset, icon or "", name or "", keyword or "")
|
||||
|
||||
def set_countdown(self, remaining: float | None, total: float) -> None:
|
||||
"""Silence auto-stop progress: ``remaining`` seconds until it fires over
|
||||
a ``total``-second window, or ``None`` while you're still speaking."""
|
||||
GLib.idle_add(self._set_countdown, remaining, total)
|
||||
|
||||
def set_state(self, state: str, message: str = "") -> None:
|
||||
GLib.idle_add(self._set_state, state, message)
|
||||
|
||||
def hide(self) -> None:
|
||||
GLib.idle_add(self._hide)
|
||||
|
||||
def destroy(self) -> None:
|
||||
GLib.idle_add(self._destroy)
|
||||
|
||||
# -- main-thread handlers -------------------------------------------------
|
||||
def _show(self, state: str, window_id: str | None) -> bool:
|
||||
self._cancel_hide()
|
||||
self._state = state
|
||||
self._phase_label = _PHASES.get(state, ((1, 1, 1), ""))[1]
|
||||
self._text = ""
|
||||
self._preset_icon = ""
|
||||
self._preset_name = ""
|
||||
self._keyword = ""
|
||||
self._levels = deque([0.0] * _BARS, maxlen=_BARS)
|
||||
self._disp = [0.0] * _BARS
|
||||
self._cd_deadline = None
|
||||
self._cd_frac = 0.0
|
||||
self._cd_alpha = 0.0
|
||||
self._anchor = caret.resolve(self.anchor_mode, window_id)
|
||||
self._relayout()
|
||||
self._visible = True
|
||||
self._win.show_all()
|
||||
if self._tick_id is None:
|
||||
self._tick_id = GLib.timeout_add(_FPS_MS, self._tick)
|
||||
return False
|
||||
|
||||
def _set_level(self, level: float) -> bool:
|
||||
self._levels.append(max(0.0, min(1.0, level)))
|
||||
return False
|
||||
|
||||
def _set_text(self, text: str) -> bool:
|
||||
if text == self._text:
|
||||
return False
|
||||
self._text = text
|
||||
self._relayout()
|
||||
self._area.queue_draw()
|
||||
return False
|
||||
|
||||
def _set_preset(self, icon: str, name: str, keyword: str) -> bool:
|
||||
if (icon, name, keyword) == (self._preset_icon, self._preset_name, self._keyword):
|
||||
return False
|
||||
self._preset_icon = icon
|
||||
self._preset_name = name
|
||||
self._keyword = keyword
|
||||
self._relayout()
|
||||
self._area.queue_draw()
|
||||
return False
|
||||
|
||||
def _set_countdown(self, remaining: float | None, total: float) -> bool:
|
||||
if remaining is None:
|
||||
self._cd_deadline = None
|
||||
else:
|
||||
self._cd_total = max(0.1, total)
|
||||
self._cd_deadline = time.time() + max(0.0, remaining)
|
||||
return False
|
||||
|
||||
def _set_state(self, state: str, message: str) -> bool:
|
||||
self._state = state
|
||||
if state == "busy":
|
||||
# Honour a clean phase word from the caller ("Transcribing…",
|
||||
# "Rewriting…") so the overlay narrates what's happening; fall back to
|
||||
# the canned label otherwise.
|
||||
self._phase_label = (message.strip() or _PHASES["busy"][1])[:28]
|
||||
else:
|
||||
self._phase_label = _PHASES.get(state, ((1, 1, 1), message[:40]))[1] or message[:40]
|
||||
if state not in ("recording", "streaming"):
|
||||
# The countdown only makes sense while listening; drop it as soon as
|
||||
# we move on to transcribing / done / idle so the ring doesn't linger.
|
||||
self._cd_deadline = None
|
||||
self._update_input_region() # enable/disable × hit area
|
||||
if state in ("recording", "streaming", "busy"):
|
||||
self._cancel_hide()
|
||||
elif state in ("done", "idle", "error"):
|
||||
# Linger on the final text, then fade out. Errors stay a touch longer.
|
||||
delay = 2500 if state == "error" else (1600 if self._text else 700)
|
||||
self._schedule_hide(delay)
|
||||
self._area.queue_draw()
|
||||
return False
|
||||
|
||||
def _hide(self) -> bool:
|
||||
self._visible = False
|
||||
self._cancel_hide()
|
||||
if self._tick_id is not None:
|
||||
GLib.source_remove(self._tick_id)
|
||||
self._tick_id = None
|
||||
self._win.hide()
|
||||
return False
|
||||
|
||||
def _destroy(self) -> bool:
|
||||
self._hide()
|
||||
self._win.destroy()
|
||||
return False
|
||||
|
||||
# -- hide scheduling ------------------------------------------------------
|
||||
def _schedule_hide(self, delay_ms: int) -> None:
|
||||
self._cancel_hide()
|
||||
self._hide_id = GLib.timeout_add(delay_ms, self._hide)
|
||||
|
||||
def _cancel_hide(self) -> None:
|
||||
if self._hide_id is not None:
|
||||
GLib.source_remove(self._hide_id)
|
||||
self._hide_id = None
|
||||
|
||||
# -- animation ------------------------------------------------------------
|
||||
def _tick(self) -> bool:
|
||||
if not self._visible:
|
||||
self._tick_id = None
|
||||
return False
|
||||
# Ease displayed bars toward the latest levels for fluid motion even
|
||||
# though real levels arrive at ~10 Hz.
|
||||
targets = list(self._levels)
|
||||
for i, t in enumerate(targets):
|
||||
self._disp[i] += (t - self._disp[i]) * 0.35
|
||||
self._pulse = (math.sin((time.time() - self._t0) * 5.0) + 1.0) * 0.5
|
||||
# Drain the silence ring against its own clock so it stays smooth between
|
||||
# the ~10 Hz level samples. Fade in gently (so word gaps don't flash it)
|
||||
# and out a touch faster.
|
||||
if self._cd_deadline is not None:
|
||||
self._cd_frac = max(0.0, min(1.0, (self._cd_deadline - time.time()) / self._cd_total))
|
||||
self._cd_alpha += (1.0 - self._cd_alpha) * 0.15
|
||||
else:
|
||||
self._cd_alpha += (0.0 - self._cd_alpha) * 0.30
|
||||
self._area.queue_draw()
|
||||
return True
|
||||
|
||||
# -- geometry -------------------------------------------------------------
|
||||
def _monitor_geo(self):
|
||||
disp = Gdk.Display.get_default()
|
||||
if self._anchor is not None:
|
||||
mon = disp.get_monitor_at_point(self._anchor.x, self._anchor.y)
|
||||
else:
|
||||
mon = disp.get_primary_monitor() or disp.get_monitor(0)
|
||||
return mon.get_geometry()
|
||||
|
||||
def _text_height(self) -> int:
|
||||
if not self._text:
|
||||
return _MIN_TEXT_H
|
||||
layout = self._win.create_pango_layout(self._text)
|
||||
layout.set_width((_WIDTH - 2 * _PAD) * Pango.SCALE)
|
||||
layout.set_wrap(Pango.WrapMode.WORD_CHAR)
|
||||
font = Pango.FontDescription("Sans 11")
|
||||
layout.set_font_description(font)
|
||||
_w, h = layout.get_pixel_size()
|
||||
return min(_MAX_TEXT_H, max(_MIN_TEXT_H, h))
|
||||
|
||||
def _relayout(self) -> None:
|
||||
text_h = self._text_height()
|
||||
preset_h = _PRESET_H if self._preset_name else 0
|
||||
gap_preset = 8 if preset_h else 0
|
||||
gap_text = 8 if text_h else 0
|
||||
body_h = _HEADER_H + gap_preset + preset_h + gap_text + text_h + 2 * _PAD
|
||||
self._height = body_h + _TAIL_H
|
||||
|
||||
geo = self._monitor_geo()
|
||||
a = self._anchor
|
||||
ax = a.x if a else geo.x + geo.width // 2
|
||||
ay = a.y if a else geo.y + geo.height - 140
|
||||
ah = a.height if a else 0
|
||||
|
||||
# Prefer placing the bubble above the anchor; flip below if it won't fit.
|
||||
above_y = ay - _GAP - self._height
|
||||
if above_y >= geo.y + 4:
|
||||
self._tail_up = False
|
||||
win_y = above_y
|
||||
else:
|
||||
self._tail_up = True
|
||||
win_y = ay + ah + _GAP
|
||||
|
||||
tip_x = max(geo.x + _RADIUS + _TAIL_W, min(ax, geo.x + geo.width - _RADIUS - _TAIL_W))
|
||||
win_x = tip_x - _WIDTH // 2
|
||||
win_x = max(geo.x + 6, min(win_x, geo.x + geo.width - _WIDTH - 6))
|
||||
self._tail_x = tip_x - win_x
|
||||
|
||||
self._win.resize(_WIDTH, self._height)
|
||||
self._win.move(int(win_x), int(win_y))
|
||||
self._area.set_size_request(_WIDTH, self._height)
|
||||
|
||||
# X button: top-right corner of the bubble body.
|
||||
_body_top = _TAIL_H if self._tail_up else 0
|
||||
btn_d = _CANCEL_BTN_R * 2
|
||||
self._cancel_btn_rect = (
|
||||
_WIDTH - _PAD - btn_d, _body_top + _PAD // 2, btn_d, btn_d)
|
||||
self._update_input_region()
|
||||
|
||||
# -- drawing --------------------------------------------------------------
|
||||
def _on_draw(self, _area, cr) -> bool:
|
||||
# Start fully transparent.
|
||||
cr.set_operator(cairo.OPERATOR_SOURCE)
|
||||
cr.set_source_rgba(0, 0, 0, 0)
|
||||
cr.paint()
|
||||
cr.set_operator(cairo.OPERATOR_OVER)
|
||||
|
||||
w = _WIDTH
|
||||
body_top = _TAIL_H if self._tail_up else 0
|
||||
body_h = self._height - _TAIL_H
|
||||
body_bottom = body_top + body_h
|
||||
|
||||
# Bubble + tail as one path, so the fill/stroke wrap the tail cleanly.
|
||||
self._bubble_path(cr, 0, body_top, w, body_h)
|
||||
cr.set_source_rgba(0.10, 0.11, 0.14, 0.94)
|
||||
cr.fill_preserve()
|
||||
cr.set_source_rgba(1, 1, 1, 0.08)
|
||||
cr.set_line_width(1.0)
|
||||
cr.stroke()
|
||||
|
||||
cx = _PAD + 14
|
||||
cy = body_top + _PAD + 12
|
||||
self._draw_mic(cr, cx, cy)
|
||||
self._draw_countdown(cr, cx, cy)
|
||||
|
||||
# Waveform fills the space right of the mic across the header row.
|
||||
wf_x = cx + 26
|
||||
wf_w = w - _PAD - wf_x
|
||||
self._draw_wave(cr, wf_x, body_top + _PAD, wf_w, _HEADER_H)
|
||||
|
||||
# × cancel button (top-right corner, recording/streaming/busy).
|
||||
if self._on_cancel_cb and self._state in ("recording", "streaming", "busy"):
|
||||
self._draw_cancel_btn(cr, body_top)
|
||||
|
||||
# Phase label by the waveform — only when there's no preset banner and no text.
|
||||
# Shift left to leave room for the × button.
|
||||
label_right = (w - _PAD - _CANCEL_BTN_R * 2 - 6
|
||||
if self._on_cancel_cb and self._state in ("recording", "streaming", "busy")
|
||||
else w - _PAD)
|
||||
if self._phase_label and not self._text and not self._preset_name:
|
||||
self._draw_label(cr, label_right, body_top + _PAD + 12, self._phase_label)
|
||||
|
||||
y = body_top + _PAD + _HEADER_H
|
||||
# Matched-preset banner: emoji + name (left), live phase chip (right).
|
||||
if self._preset_name:
|
||||
y += 8
|
||||
self._draw_preset(cr, _PAD, y, w - 2 * _PAD)
|
||||
y += _PRESET_H
|
||||
# Recognised / rewritten text below.
|
||||
if self._text:
|
||||
y += 8
|
||||
self._draw_text(cr, _PAD, y, w - 2 * _PAD)
|
||||
return False
|
||||
|
||||
def _draw_preset(self, cr, x, y, w) -> None:
|
||||
"""The matched voice-routing preset, fused onto the overlay in place of a
|
||||
desktop notification: emoji + name (and the spoken keyword) on the left,
|
||||
the current phase ("Transcribing…", "Rewriting…") on the right."""
|
||||
cy = y + _PRESET_H / 2
|
||||
cursor = x
|
||||
if self._preset_icon:
|
||||
ic = self._win.create_pango_layout(self._preset_icon)
|
||||
ic.set_font_description(Pango.FontDescription("Sans 13"))
|
||||
iw, ih = ic.get_pixel_size()
|
||||
cr.set_source_rgba(1, 1, 1, 0.95)
|
||||
cr.move_to(cursor, cy - ih / 2)
|
||||
PangoCairo.show_layout(cr, ic)
|
||||
cursor += iw + 7
|
||||
name = GLib.markup_escape_text(self._preset_name)
|
||||
if self._keyword:
|
||||
kw = GLib.markup_escape_text(self._keyword)
|
||||
markup = f'<b>{name}</b> <span alpha="55%">“{kw}”</span>'
|
||||
else:
|
||||
markup = f"<b>{name}</b>"
|
||||
layout = self._win.create_pango_layout("")
|
||||
layout.set_markup(markup, -1)
|
||||
layout.set_font_description(Pango.FontDescription("Sans 10"))
|
||||
layout.set_ellipsize(Pango.EllipsizeMode.END)
|
||||
avail = (x + w) - cursor - 96 # leave room for the phase chip
|
||||
layout.set_width(max(40, avail) * Pango.SCALE)
|
||||
_nw, nh = layout.get_pixel_size()
|
||||
cr.set_source_rgba(0.95, 0.96, 0.99, 0.98)
|
||||
cr.move_to(cursor, cy - nh / 2)
|
||||
PangoCairo.show_layout(cr, layout)
|
||||
if self._phase_label:
|
||||
self._draw_label(cr, x + w, cy, self._phase_label)
|
||||
|
||||
def _bubble_path(self, cr, x, y, w, h) -> None:
|
||||
r = _RADIUS
|
||||
cr.new_sub_path()
|
||||
cr.arc(x + w - r, y + r, r, -math.pi / 2, 0)
|
||||
cr.arc(x + w - r, y + h - r, r, 0, math.pi / 2)
|
||||
cr.arc(x + r, y + h - r, r, math.pi / 2, math.pi)
|
||||
cr.arc(x + r, y + r, r, math.pi, 1.5 * math.pi)
|
||||
cr.close_path()
|
||||
# Tail: a small triangle on the top or bottom edge at self._tail_x.
|
||||
tx = max(r + _TAIL_W, min(self._tail_x, w - r - _TAIL_W))
|
||||
if self._tail_up:
|
||||
cr.move_to(tx - _TAIL_W / 2, y)
|
||||
cr.line_to(tx, y - _TAIL_H)
|
||||
cr.line_to(tx + _TAIL_W / 2, y)
|
||||
else:
|
||||
cr.move_to(tx - _TAIL_W / 2, y + h)
|
||||
cr.line_to(tx, y + h + _TAIL_H)
|
||||
cr.line_to(tx + _TAIL_W / 2, y + h)
|
||||
cr.close_path()
|
||||
|
||||
def _draw_mic(self, cr, cx, cy) -> None:
|
||||
colour, _ = _PHASES.get(self._state, ((1, 1, 1), ""))
|
||||
pulsing = self._state in ("recording", "streaming")
|
||||
# Soft pulsing halo while listening — but yield to the countdown ring as
|
||||
# it takes over (you've gone quiet, so the pulse fades out under it).
|
||||
if pulsing:
|
||||
rad = 12 + self._pulse * 6
|
||||
cr.set_source_rgba(*colour, 0.18 * (1 - self._pulse * 0.6) * (1 - self._cd_alpha))
|
||||
cr.arc(cx, cy, rad, 0, 2 * math.pi)
|
||||
cr.fill()
|
||||
cr.set_source_rgba(*colour, 1.0)
|
||||
cr.set_line_width(2.0)
|
||||
# Capsule head.
|
||||
head_w, head_top, head_bot = 9.0, cy - 11, cy + 1
|
||||
cr.arc(cx, head_top + head_w / 2, head_w / 2, math.pi, 2 * math.pi)
|
||||
cr.arc(cx, head_bot - head_w / 2, head_w / 2, 0, math.pi)
|
||||
cr.close_path()
|
||||
cr.fill()
|
||||
# Stand arc + post + base.
|
||||
cr.set_source_rgba(*colour, 0.95)
|
||||
cr.arc(cx, cy, 8, math.radians(25), math.radians(155))
|
||||
cr.stroke()
|
||||
cr.move_to(cx, cy + 8)
|
||||
cr.line_to(cx, cy + 12)
|
||||
cr.stroke()
|
||||
cr.move_to(cx - 5, cy + 12)
|
||||
cr.line_to(cx + 5, cy + 12)
|
||||
cr.stroke()
|
||||
|
||||
def _draw_countdown(self, cr, cx, cy) -> None:
|
||||
"""Silence auto-stop ring wrapping the mic: a full circle that drains
|
||||
clockwise as the trailing-silence timer runs out, recolouring from calm
|
||||
cyan to an urgent red just before it fires."""
|
||||
a = self._cd_alpha
|
||||
if a <= 0.01:
|
||||
return
|
||||
r, lw = 16.0, 2.6
|
||||
cr.set_line_width(lw)
|
||||
# Faint full track so the drained part of the ring stays legible.
|
||||
cr.set_source_rgba(1, 1, 1, 0.10 * a)
|
||||
cr.arc(cx, cy, r, 0, 2 * math.pi)
|
||||
cr.stroke()
|
||||
frac = self._cd_frac
|
||||
if frac <= 0.0:
|
||||
return
|
||||
spent = 1.0 - frac
|
||||
red = 0.30 + spent * 0.70
|
||||
grn = 0.80 - spent * 0.45
|
||||
blu = 0.90 - spent * 0.60
|
||||
cr.set_source_rgba(red, grn, blu, 0.95 * a)
|
||||
cr.set_line_cap(cairo.LINE_CAP_ROUND)
|
||||
start = -math.pi / 2 # 12 o'clock
|
||||
cr.arc(cx, cy, r, start, start + frac * 2 * math.pi)
|
||||
cr.stroke()
|
||||
cr.set_line_cap(cairo.LINE_CAP_BUTT)
|
||||
|
||||
def _draw_wave(self, cr, x, y, w, h) -> None:
|
||||
gap = 2.0
|
||||
bw = max(1.5, (w - gap * (_BARS - 1)) / _BARS)
|
||||
mid = y + h / 2
|
||||
listening = self._state in ("recording", "streaming")
|
||||
for i in range(_BARS):
|
||||
v = self._disp[i] if i < len(self._disp) else 0.0
|
||||
# Idle baseline shimmer so the meter never looks frozen.
|
||||
if listening and v < 0.04:
|
||||
v = 0.04 + 0.03 * math.sin((self._t0 - time.time()) * 4 + i * 0.5)
|
||||
bh = max(2.0, v * (h - 2))
|
||||
bx = x + i * (bw + gap)
|
||||
alpha = 0.85 if listening else 0.4
|
||||
cr.set_source_rgba(0.42, 0.62, 1.0, alpha)
|
||||
self._round_rect(cr, bx, mid - bh / 2, bw, bh, min(bw / 2, 2))
|
||||
cr.fill()
|
||||
|
||||
def _draw_label(self, cr, right_x, cy, text) -> None:
|
||||
layout = self._win.create_pango_layout(text)
|
||||
layout.set_font_description(Pango.FontDescription("Sans 9"))
|
||||
tw, th = layout.get_pixel_size()
|
||||
cr.set_source_rgba(1, 1, 1, 0.55)
|
||||
cr.move_to(right_x - tw, cy - th / 2)
|
||||
PangoCairo.show_layout(cr, layout)
|
||||
|
||||
def _draw_cancel_btn(self, cr, body_top) -> None:
|
||||
x, y, w, h = self._cancel_btn_rect
|
||||
cx_btn = x + w / 2
|
||||
cy_btn = y + h / 2
|
||||
cr.set_source_rgba(1, 1, 1, 0.15)
|
||||
cr.arc(cx_btn, cy_btn, _CANCEL_BTN_R, 0, 2 * math.pi)
|
||||
cr.fill()
|
||||
arm = _CANCEL_BTN_R * 0.45
|
||||
cr.set_source_rgba(1, 1, 1, 0.80)
|
||||
cr.set_line_width(1.8)
|
||||
cr.set_line_cap(cairo.LINE_CAP_ROUND)
|
||||
cr.move_to(cx_btn - arm, cy_btn - arm)
|
||||
cr.line_to(cx_btn + arm, cy_btn + arm)
|
||||
cr.stroke()
|
||||
cr.move_to(cx_btn + arm, cy_btn - arm)
|
||||
cr.line_to(cx_btn - arm, cy_btn + arm)
|
||||
cr.stroke()
|
||||
cr.set_line_cap(cairo.LINE_CAP_BUTT)
|
||||
|
||||
def _draw_text(self, cr, x, y, w) -> None:
|
||||
layout = self._win.create_pango_layout(self._text)
|
||||
layout.set_width(w * Pango.SCALE)
|
||||
layout.set_wrap(Pango.WrapMode.WORD_CHAR)
|
||||
layout.set_ellipsize(Pango.EllipsizeMode.END)
|
||||
layout.set_height(_MAX_TEXT_H * Pango.SCALE)
|
||||
layout.set_font_description(Pango.FontDescription("Sans 11"))
|
||||
cr.set_source_rgba(0.95, 0.96, 0.99, 0.97)
|
||||
cr.move_to(x, y)
|
||||
PangoCairo.show_layout(cr, layout)
|
||||
|
||||
@staticmethod
|
||||
def _round_rect(cr, x, y, w, h, r) -> None:
|
||||
r = min(r, w / 2, h / 2)
|
||||
cr.new_sub_path()
|
||||
cr.arc(x + w - r, y + r, r, -math.pi / 2, 0)
|
||||
cr.arc(x + w - r, y + h - r, r, 0, math.pi / 2)
|
||||
cr.arc(x + r, y + h - r, r, math.pi / 2, math.pi)
|
||||
cr.arc(x + r, y + r, r, math.pi, 1.5 * math.pi)
|
||||
cr.close_path()
|
||||
@ -41,6 +41,13 @@ def _focus(window_id: str | None) -> None:
|
||||
time.sleep(0.05)
|
||||
|
||||
|
||||
# Above this character count, or when the text contains newlines, xdotool type
|
||||
# sends thousands of synchronous X11 round-trips and can flood the X11 server's
|
||||
# per-client event buffer until the whole session freezes. Auto-upgrade to a
|
||||
# single clipboard paste instead, which is instantaneous.
|
||||
_TYPE_THRESHOLD = 300
|
||||
|
||||
|
||||
def deliver(text: str, *, mode: str = "type", window_id: str | None = None, type_delay_ms: int = 4) -> None:
|
||||
if not text:
|
||||
return
|
||||
@ -55,6 +62,15 @@ def deliver(text: str, *, mode: str = "type", window_id: str | None = None, type
|
||||
# Give the user time to release the hotkey modifiers before we synthesize input.
|
||||
time.sleep(0.12)
|
||||
|
||||
# Long or multi-line text: force clipboard paste regardless of configured mode.
|
||||
# xdotool type at 12ms/char for a 15 000-char code block takes ~3 minutes and
|
||||
# sends so many synchronous X11 events that the server's per-client buffer
|
||||
# overflows, freezing the entire X11 session.
|
||||
if mode == "type" and (len(text) > _TYPE_THRESHOLD or "\n" in text):
|
||||
if _set_clipboard(text):
|
||||
mode = "paste"
|
||||
# If clipboard isn't available we fall through to xdotool type as before.
|
||||
|
||||
if mode == "paste" and _set_clipboard(text):
|
||||
if wayland:
|
||||
if shutil.which("wtype"):
|
||||
|
||||
@ -71,3 +71,23 @@ def clean(text: str, *, strip_trailing_punctuation: bool = False) -> str:
|
||||
if strip_trailing_punctuation:
|
||||
text = text.rstrip(" .,!?;:")
|
||||
return text
|
||||
|
||||
|
||||
# Spoken phrases that should become newlines/paragraphs in the typed output.
|
||||
# Matched case-insensitively, whole-phrase (surrounded by word boundaries or
|
||||
# start/end of string). German and English variants are both covered.
|
||||
_NEWLINE_PHRASES: list[tuple[re.Pattern[str], str]] = [
|
||||
# Paragraph break (blank line) — must come before single-newline patterns
|
||||
(re.compile(r"(?i)\b(new paragraph|neuer absatz|neues absatz|absatz)\b"), "\n\n"),
|
||||
# Single line break
|
||||
(re.compile(r"(?i)\b(new line|neue zeile|zeilenumbruch|line break)\b"), "\n"),
|
||||
]
|
||||
|
||||
|
||||
def expand_spoken_punctuation(text: str) -> str:
|
||||
"""Replace spoken newline/paragraph commands with actual control characters."""
|
||||
for pattern, replacement in _NEWLINE_PHRASES:
|
||||
text = pattern.sub(replacement, text)
|
||||
# Collapse leading/trailing whitespace per line but preserve intentional newlines
|
||||
lines = [ln.strip() for ln in text.split("\n")]
|
||||
return "\n".join(lines).strip()
|
||||
|
||||
@ -85,7 +85,13 @@ def route(
|
||||
|
||||
best = None # (score, span_len, position, preset_name, keyword)
|
||||
for preset in presets:
|
||||
for keyword in getattr(preset, "keywords", None) or []:
|
||||
# The preset's own name is always an implicit trigger, so a preset is
|
||||
# speakable by name even when it has no keywords configured.
|
||||
candidates = list(getattr(preset, "keywords", None) or [])
|
||||
name = getattr(preset, "name", "")
|
||||
if name:
|
||||
candidates.append(name)
|
||||
for keyword in candidates:
|
||||
kw_tokens = normalize(keyword)
|
||||
m = _match_window(tokens, kw_tokens, threshold)
|
||||
if m is None:
|
||||
@ -104,6 +110,55 @@ def route(
|
||||
return RouteResult(name, cleaned, keyword, position, score)
|
||||
|
||||
|
||||
def is_cancel(transcript: str, cancel_keywords, *, threshold: float = DEFAULT_THRESHOLD) -> str | None:
|
||||
"""Return the cancel keyword that matches an edge of the transcript, else None.
|
||||
|
||||
Lets a spoken word like "abbrechen" abort an (often accidentally triggered)
|
||||
dictation before it is routed, rewritten, or delivered. Matched the same
|
||||
edge-anchored, ASR-tolerant way as routing keywords, so the word appearing
|
||||
deep inside a sentence won't trigger it — only at the start or end.
|
||||
"""
|
||||
if not cancel_keywords:
|
||||
return None
|
||||
tokens = normalize(transcript)
|
||||
if not tokens:
|
||||
return None
|
||||
for kw in cancel_keywords:
|
||||
kw_tokens = normalize(kw)
|
||||
if kw_tokens and _match_window(tokens, kw_tokens, threshold) is not None:
|
||||
return kw
|
||||
return None
|
||||
|
||||
|
||||
def match_send(transcript: str, send_keywords, *, threshold: float = DEFAULT_THRESHOLD):
|
||||
"""Detect a spoken 'send' keyword at an edge; return (keyword, cleaned_text).
|
||||
|
||||
Like is_cancel, but the keyword is *stripped* and the remaining text is meant
|
||||
to be delivered and submitted with Enter — the spoken equivalent of
|
||||
stop+paste+Enter. Returns (None, transcript) when nothing matches. Matched the
|
||||
same edge-anchored, ASR-tolerant way as routing keywords, so the word deep
|
||||
inside a sentence won't trigger it — only at the start or end.
|
||||
"""
|
||||
if not send_keywords:
|
||||
return None, transcript
|
||||
tokens = normalize(transcript)
|
||||
if not tokens:
|
||||
return None, transcript
|
||||
best = None # (score, span, position, keyword)
|
||||
for kw in send_keywords:
|
||||
kw_tokens = normalize(kw)
|
||||
m = _match_window(tokens, kw_tokens, threshold)
|
||||
if m is None:
|
||||
continue
|
||||
position, score, span = m
|
||||
if best is None or (score, span) > (best[0], best[1]):
|
||||
best = (score, span, position, kw)
|
||||
if best is None:
|
||||
return None, transcript
|
||||
_score, span, position, kw = best
|
||||
return kw, _strip_span(transcript, span, position)
|
||||
|
||||
|
||||
def _strip_span(transcript: str, span_words: int, position: str) -> str:
|
||||
"""Remove the matched keyword from the given edge of the original transcript.
|
||||
|
||||
|
||||
844
linux/blitztext/setup_wizard.py
Normal file
@ -0,0 +1,844 @@
|
||||
"""First-run setup wizard — guides new users through the essential settings.
|
||||
|
||||
Shows automatically when no config file exists yet (fresh install).
|
||||
Can also be reopened from Settings → "Setup Wizard…".
|
||||
|
||||
Flow:
|
||||
Welcome → Trigger method → [Keyboard] → [Wakeword] → STT → LLM → Done
|
||||
|
||||
Pages in brackets are shown conditionally depending on the trigger choice.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
import gi
|
||||
|
||||
gi.require_version("Gtk", "3.0")
|
||||
gi.require_version("Gdk", "3.0")
|
||||
from gi.repository import Gdk, GLib, Gtk, Pango # noqa: E402
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tiny style helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _h1(text: str) -> Gtk.Label:
|
||||
lbl = Gtk.Label(label=text, xalign=0.0)
|
||||
lbl.set_line_wrap(True)
|
||||
attrs = Pango.AttrList()
|
||||
attrs.insert(Pango.attr_weight_new(Pango.Weight.BOLD))
|
||||
attrs.insert(Pango.attr_scale_new(1.45))
|
||||
lbl.set_attributes(attrs)
|
||||
return lbl
|
||||
|
||||
|
||||
def _sub(text: str) -> Gtk.Label:
|
||||
lbl = Gtk.Label(label=text, xalign=0.0)
|
||||
lbl.set_line_wrap(True)
|
||||
lbl.set_max_width_chars(64)
|
||||
lbl.get_style_context().add_class("dim-label")
|
||||
return lbl
|
||||
|
||||
|
||||
def _section(text: str) -> Gtk.Label:
|
||||
lbl = Gtk.Label(xalign=0.0)
|
||||
lbl.set_markup(f"<b><small>{GLib.markup_escape_text(text.upper())}</small></b>")
|
||||
lbl.set_margin_top(12)
|
||||
lbl.get_style_context().add_class("dim-label")
|
||||
return lbl
|
||||
|
||||
|
||||
def _page_box() -> Gtk.Box:
|
||||
box = Gtk.Box(orientation=Gtk.Orientation.VERTICAL, spacing=10)
|
||||
box.set_margin_top(28); box.set_margin_bottom(8)
|
||||
box.set_margin_start(32); box.set_margin_end(32)
|
||||
return box
|
||||
|
||||
|
||||
def _option_card(icon: str, title: str, desc: str) -> tuple[Gtk.RadioButton, Gtk.Box]:
|
||||
"""A large selectable card with an icon, bold title, and grey description."""
|
||||
card = Gtk.Box(spacing=14)
|
||||
card.set_margin_top(4); card.set_margin_bottom(4)
|
||||
card.get_style_context().add_class("card")
|
||||
|
||||
icon_lbl = Gtk.Label(label=icon)
|
||||
attrs = Pango.AttrList()
|
||||
attrs.insert(Pango.attr_scale_new(2.0))
|
||||
icon_lbl.set_attributes(attrs)
|
||||
icon_lbl.set_margin_start(14)
|
||||
card.pack_start(icon_lbl, False, False, 0)
|
||||
|
||||
text_box = Gtk.Box(orientation=Gtk.Orientation.VERTICAL, spacing=2)
|
||||
text_box.set_margin_top(12); text_box.set_margin_bottom(12)
|
||||
title_lbl = Gtk.Label(label=title, xalign=0.0)
|
||||
attrs2 = Pango.AttrList()
|
||||
attrs2.insert(Pango.attr_weight_new(Pango.Weight.BOLD))
|
||||
title_lbl.set_attributes(attrs2)
|
||||
text_box.pack_start(title_lbl, False, False, 0)
|
||||
desc_lbl = Gtk.Label(label=desc, xalign=0.0)
|
||||
desc_lbl.get_style_context().add_class("dim-label")
|
||||
desc_lbl.set_line_wrap(True)
|
||||
desc_lbl.set_max_width_chars(55)
|
||||
text_box.pack_start(desc_lbl, False, False, 0)
|
||||
card.pack_start(text_box, True, True, 0)
|
||||
|
||||
# Invisible RadioButton — the whole card is the clickable area.
|
||||
rb = Gtk.RadioButton()
|
||||
rb.set_margin_end(14)
|
||||
card.pack_start(rb, False, False, 0)
|
||||
return rb, card
|
||||
|
||||
|
||||
def _keyval_token(keyval: int) -> str | None:
|
||||
from gi.repository import Gdk as _Gdk
|
||||
name = _Gdk.keyval_name(keyval) or ""
|
||||
low = name.lower()
|
||||
for mod in ("control", "ctrl"):
|
||||
if low.startswith(mod):
|
||||
return "ctrl"
|
||||
for mod in ("alt", "meta"):
|
||||
if low.startswith(mod):
|
||||
return "alt"
|
||||
if low.startswith("super") or low.startswith("hyper") or low.startswith("win"):
|
||||
return "cmd"
|
||||
if low.startswith("shift"):
|
||||
return "shift"
|
||||
if low in ("escape", "esc"):
|
||||
return "esc"
|
||||
if low in ("return", "enter"):
|
||||
return "enter"
|
||||
if low == "space":
|
||||
return "space"
|
||||
if len(name) == 1 and name.isalpha():
|
||||
return name.lower()
|
||||
return None
|
||||
|
||||
|
||||
def _format_combo(tokens: list[str]) -> str:
|
||||
if not tokens:
|
||||
return ""
|
||||
order = ["ctrl", "alt", "shift", "cmd"]
|
||||
mods = [t for t in order if t in tokens]
|
||||
rest = [t for t in tokens if t not in order]
|
||||
parts = mods + rest
|
||||
return "+".join(f"<{p}>" for p in parts)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Wizard
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class SetupWizard:
|
||||
"""Paged first-run setup dialog."""
|
||||
|
||||
# Page names in fixed order; "keyboard" and "wakeword" are conditional.
|
||||
_ALL_PAGES = ("welcome", "trigger", "keyboard", "wakeword", "stt", "llm", "done")
|
||||
|
||||
def __init__(self, cfg, parent: Gtk.Window | None = None):
|
||||
self.cfg = cfg
|
||||
self._parent = parent
|
||||
|
||||
# Wizard state collected as user moves through pages.
|
||||
self._trigger = "keyboard" # "keyboard" | "wakeword" | "both"
|
||||
self._kb_start = cfg.key_start
|
||||
self._kb_stop = cfg.key_stop
|
||||
self._kb_send = cfg.key_send
|
||||
self._kb_cancel = cfg.key_cancel
|
||||
self._ww_uri = cfg.wakeword_uri
|
||||
self._ww_model = cfg.wakeword_model
|
||||
self._stt_local = True # True = faster-whisper, False = remote API
|
||||
self._stt_size = "small"
|
||||
self._stt_url = ""
|
||||
self._stt_key = ""
|
||||
self._llm_enabled = bool(cfg.base_url and cfg.base_url != "https://api.openai.com/v1")
|
||||
self._llm_url = cfg.base_url
|
||||
self._llm_model = cfg.rewrite_model
|
||||
self._llm_key = cfg.api_key_env
|
||||
|
||||
# Key-binding capture state
|
||||
self._bind_entry: Gtk.Entry | None = None
|
||||
self._bind_pressed: list[str] = []
|
||||
|
||||
self._build()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Dialog shell
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _build(self) -> None:
|
||||
self.dlg = Gtk.Dialog(title="Blitztext Setup")
|
||||
if self._parent:
|
||||
self.dlg.set_transient_for(self._parent)
|
||||
self.dlg.set_modal(True)
|
||||
self.dlg.set_default_size(580, 520)
|
||||
self.dlg.set_resizable(False)
|
||||
self.dlg.get_action_area().hide()
|
||||
|
||||
# Key capture
|
||||
self.dlg.add_events(Gdk.EventMask.KEY_PRESS_MASK | Gdk.EventMask.KEY_RELEASE_MASK)
|
||||
self.dlg.connect("key-press-event", self._on_key_press)
|
||||
self.dlg.connect("key-release-event", self._on_key_release)
|
||||
|
||||
content = self.dlg.get_content_area()
|
||||
content.set_spacing(0)
|
||||
|
||||
# Stack
|
||||
self.stack = Gtk.Stack()
|
||||
self.stack.set_transition_type(Gtk.StackTransitionType.SLIDE_LEFT_RIGHT)
|
||||
self.stack.set_transition_duration(180)
|
||||
content.pack_start(self.stack, True, True, 0)
|
||||
|
||||
self._pg: dict[str, Gtk.Widget] = {}
|
||||
builders = {
|
||||
"welcome": self._build_welcome,
|
||||
"trigger": self._build_trigger,
|
||||
"keyboard": self._build_keyboard,
|
||||
"wakeword": self._build_wakeword,
|
||||
"stt": self._build_stt,
|
||||
"llm": self._build_llm,
|
||||
"done": self._build_done,
|
||||
}
|
||||
for name in self._ALL_PAGES:
|
||||
w = builders[name]()
|
||||
self.stack.add_named(w, name)
|
||||
self._pg[name] = w
|
||||
|
||||
# Nav bar
|
||||
sep = Gtk.Separator(orientation=Gtk.Orientation.HORIZONTAL)
|
||||
content.pack_start(sep, False, False, 0)
|
||||
|
||||
nav = Gtk.Box(spacing=8)
|
||||
nav.set_margin_top(10); nav.set_margin_bottom(10)
|
||||
nav.set_margin_start(20); nav.set_margin_end(20)
|
||||
|
||||
self._btn_back = Gtk.Button(label="← Back")
|
||||
self._btn_back.connect("clicked", lambda _b: self._go(-1))
|
||||
nav.pack_start(self._btn_back, False, False, 0)
|
||||
|
||||
self._step_lbl = Gtk.Label()
|
||||
self._step_lbl.get_style_context().add_class("dim-label")
|
||||
nav.pack_start(self._step_lbl, True, True, 0)
|
||||
|
||||
self._btn_skip = Gtk.Button(label="Skip")
|
||||
self._btn_skip.get_style_context().add_class("flat")
|
||||
self._btn_skip.connect("clicked", lambda _b: self._go(+1, skip=True))
|
||||
nav.pack_start(self._btn_skip, False, False, 0)
|
||||
|
||||
self._btn_next = Gtk.Button(label="Next →")
|
||||
self._btn_next.get_style_context().add_class("suggested-action")
|
||||
self._btn_next.connect("clicked", lambda _b: self._go(+1))
|
||||
nav.pack_start(self._btn_next, False, False, 0)
|
||||
|
||||
content.pack_start(nav, False, False, 0)
|
||||
|
||||
self._page_order: list[str] = []
|
||||
self._idx = 0
|
||||
self._refresh_page_order()
|
||||
self._show(0)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Navigation
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _refresh_page_order(self) -> None:
|
||||
pages = ["welcome", "trigger"]
|
||||
if self._trigger in ("keyboard", "both"):
|
||||
pages.append("keyboard")
|
||||
if self._trigger in ("wakeword", "both"):
|
||||
pages.append("wakeword")
|
||||
pages += ["stt", "llm", "done"]
|
||||
self._page_order = pages
|
||||
|
||||
def _show(self, idx: int) -> None:
|
||||
self._idx = max(0, min(idx, len(self._page_order) - 1))
|
||||
name = self._page_order[self._idx]
|
||||
self.stack.set_visible_child_name(name)
|
||||
|
||||
is_first = self._idx == 0
|
||||
is_last = self._idx == len(self._page_order) - 1
|
||||
is_welcome = name == "welcome"
|
||||
is_done = name == "done"
|
||||
|
||||
self._btn_back.set_sensitive(not is_first)
|
||||
self._btn_skip.set_visible(not is_welcome and not is_done)
|
||||
self._btn_next.set_label("Start dictating →" if is_last else "Next →")
|
||||
|
||||
# Step counter (exclude welcome and done from the count)
|
||||
core_pages = [p for p in self._page_order if p not in ("welcome", "done")]
|
||||
if name in core_pages:
|
||||
step = core_pages.index(name) + 1
|
||||
self._step_lbl.set_text(f"Step {step} of {len(core_pages)}")
|
||||
else:
|
||||
self._step_lbl.set_text("")
|
||||
|
||||
# When leaving "trigger", re-evaluate page order.
|
||||
if name == "trigger":
|
||||
self._read_trigger()
|
||||
self._refresh_page_order()
|
||||
|
||||
def _go(self, direction: int, skip: bool = False) -> None:
|
||||
name = self._page_order[self._idx]
|
||||
if not skip:
|
||||
self._commit_page(name)
|
||||
if direction > 0 and self._idx >= len(self._page_order) - 1:
|
||||
self._finish()
|
||||
return
|
||||
self._show(self._idx + direction)
|
||||
|
||||
def _finish(self) -> None:
|
||||
self._apply_to_cfg()
|
||||
self.dlg.response(Gtk.ResponseType.OK)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Page builders
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _build_welcome(self) -> Gtk.Widget:
|
||||
box = _page_box()
|
||||
box.set_valign(Gtk.Align.CENTER)
|
||||
|
||||
logo = Gtk.Label(label="🎙")
|
||||
attrs = Pango.AttrList()
|
||||
attrs.insert(Pango.attr_scale_new(4.0))
|
||||
logo.set_attributes(attrs)
|
||||
logo.set_margin_bottom(12)
|
||||
box.pack_start(logo, False, False, 0)
|
||||
|
||||
box.pack_start(_h1("Welcome to Blitztext"), False, False, 0)
|
||||
box.pack_start(_sub(
|
||||
"Blitztext lets you speak and have your words typed anywhere on screen — "
|
||||
"with optional AI polishing.\n\n"
|
||||
"This short wizard sets up the basics. "
|
||||
"You can change everything later in Settings."
|
||||
), False, False, 0)
|
||||
return box
|
||||
|
||||
def _build_trigger(self) -> Gtk.Widget:
|
||||
box = _page_box()
|
||||
box.pack_start(_h1("How do you want to trigger recording?"), False, False, 0)
|
||||
box.pack_start(_sub("You can change this later in Settings → Input."), False, False, 0)
|
||||
|
||||
self._rb_kb, card_kb = _option_card("⌨", "Keyboard shortcut",
|
||||
"Press a key combination to start and stop recording. "
|
||||
"Great for desktop use.")
|
||||
self._rb_ww, card_ww = _option_card("🎙", "Voice wakeword",
|
||||
"Say a wake phrase like \"okay computer\" to start hands-free. "
|
||||
"Needs a wakeword server.")
|
||||
self._rb_both, card_both = _option_card("✨", "Both",
|
||||
"Use a keyboard shortcut AND a voice wakeword — whichever is handy.")
|
||||
|
||||
# Group the radio buttons
|
||||
self._rb_ww.join_group(self._rb_kb)
|
||||
self._rb_both.join_group(self._rb_kb)
|
||||
self._rb_kb.set_active(True)
|
||||
|
||||
for rb, card in ((self._rb_kb, card_kb), (self._rb_ww, card_ww),
|
||||
(self._rb_both, card_both)):
|
||||
# Make the card clickable by forwarding clicks to the radio button
|
||||
ebox = Gtk.EventBox()
|
||||
ebox.add(card)
|
||||
ebox.connect("button-press-event",
|
||||
lambda _e, _ev, r=rb: r.set_active(True))
|
||||
box.pack_start(ebox, False, False, 0)
|
||||
|
||||
return box
|
||||
|
||||
def _build_keyboard(self) -> Gtk.Widget:
|
||||
box = _page_box()
|
||||
box.pack_start(_h1("Set up your keyboard shortcuts"), False, False, 0)
|
||||
box.pack_start(_sub(
|
||||
'Click "Set" and press a key combination to capture it. '
|
||||
"The defaults shown here work well for most users."
|
||||
), False, False, 0)
|
||||
|
||||
grid = Gtk.Grid(column_spacing=8, row_spacing=8)
|
||||
grid.set_margin_top(16)
|
||||
|
||||
rows = [
|
||||
("Start recording", self._kb_start, "_kb_start"),
|
||||
("Stop and paste", self._kb_stop, "_kb_stop"),
|
||||
("Stop, paste + Enter", self._kb_send, "_kb_send"),
|
||||
("Cancel", self._kb_cancel, "_kb_cancel"),
|
||||
]
|
||||
self._kb_entries: dict[str, Gtk.Entry] = {}
|
||||
|
||||
for i, (label, value, attr) in enumerate(rows):
|
||||
lbl = Gtk.Label(label=label, xalign=0.0)
|
||||
lbl.set_size_request(180, -1)
|
||||
grid.attach(lbl, 0, i, 1, 1)
|
||||
|
||||
entry = Gtk.Entry()
|
||||
entry.set_text(value)
|
||||
entry.set_size_request(140, -1)
|
||||
entry.set_editable(False)
|
||||
self._kb_entries[attr] = entry
|
||||
grid.attach(entry, 1, i, 1, 1)
|
||||
|
||||
btn = Gtk.Button(label="Set")
|
||||
btn.connect("clicked", lambda _b, e=entry: self._bind_key(e))
|
||||
grid.attach(btn, 2, i, 1, 1)
|
||||
|
||||
box.pack_start(grid, False, False, 0)
|
||||
|
||||
note = Gtk.Label(xalign=0.0)
|
||||
note.set_markup(
|
||||
'<span size="small" alpha="75%">'
|
||||
"Tip: use modifier keys (Ctrl, Alt, Win/Cmd) rather than letters "
|
||||
"so shortcuts don't interfere with typing."
|
||||
"</span>")
|
||||
note.set_line_wrap(True)
|
||||
note.set_margin_top(8)
|
||||
box.pack_start(note, False, False, 0)
|
||||
return box
|
||||
|
||||
def _build_wakeword(self) -> Gtk.Widget:
|
||||
box = _page_box()
|
||||
box.pack_start(_h1("Set up voice activation"), False, False, 0)
|
||||
box.pack_start(_sub(
|
||||
"Blitztext listens for your wake phrase through a local wakeword server. "
|
||||
"The server runs on your machine — nothing is sent to the cloud."
|
||||
), False, False, 0)
|
||||
|
||||
grid = Gtk.Grid(column_spacing=8, row_spacing=10)
|
||||
grid.set_margin_top(16)
|
||||
|
||||
# Server URL
|
||||
lbl_uri = Gtk.Label(label="Server URL", xalign=0.0)
|
||||
lbl_uri.set_size_request(120, -1)
|
||||
self._wiz_ww_uri = Gtk.Entry()
|
||||
self._wiz_ww_uri.set_text(self._ww_uri)
|
||||
self._wiz_ww_uri.set_hexpand(True)
|
||||
self._wiz_ww_uri.set_placeholder_text("tcp://127.0.0.1:10400")
|
||||
grid.attach(lbl_uri, 0, 0, 1, 1)
|
||||
grid.attach(self._wiz_ww_uri, 1, 0, 1, 1)
|
||||
|
||||
# Model
|
||||
lbl_model = Gtk.Label(label="Wake phrase", xalign=0.0)
|
||||
self._wiz_ww_model = Gtk.Entry()
|
||||
self._wiz_ww_model.set_text(self._ww_model)
|
||||
self._wiz_ww_model.set_placeholder_text("okay_computer")
|
||||
grid.attach(lbl_model, 0, 1, 1, 1)
|
||||
grid.attach(self._wiz_ww_model, 1, 1, 1, 1)
|
||||
|
||||
box.pack_start(grid, False, False, 0)
|
||||
|
||||
# Test row
|
||||
test_row = Gtk.Box(spacing=10)
|
||||
test_row.set_margin_top(8)
|
||||
self._wiz_ww_test_btn = Gtk.Button(label="Test connection")
|
||||
self._wiz_ww_test_btn.connect("clicked", self._ww_test)
|
||||
self._wiz_ww_test_lbl = Gtk.Label(label="", xalign=0.0)
|
||||
test_row.pack_start(self._wiz_ww_test_btn, False, False, 0)
|
||||
test_row.pack_start(self._wiz_ww_test_lbl, False, False, 0)
|
||||
box.pack_start(test_row, False, False, 0)
|
||||
|
||||
# Setup hint (collapsible-ish — just always visible for simplicity)
|
||||
hint = Gtk.Label(xalign=0.0)
|
||||
hint.set_markup(
|
||||
'<span size="small" alpha="75%">'
|
||||
"Don't have a wakeword server yet? Run one with Docker:\n"
|
||||
"<tt>docker run -it -p 10400:10400 homeassistant/wyoming-openwakeword</tt>"
|
||||
"</span>")
|
||||
hint.set_line_wrap(True)
|
||||
hint.set_selectable(True)
|
||||
hint.set_margin_top(12)
|
||||
box.pack_start(hint, False, False, 0)
|
||||
return box
|
||||
|
||||
def _build_stt(self) -> Gtk.Widget:
|
||||
box = _page_box()
|
||||
box.pack_start(_h1("Choose your speech-to-text engine"), False, False, 0)
|
||||
box.pack_start(_sub(
|
||||
"Blitztext transcribes your speech locally using Whisper, "
|
||||
"or you can connect to a remote OpenAI-compatible API."
|
||||
), False, False, 0)
|
||||
|
||||
# Local vs remote toggle
|
||||
self._rb_local, card_local = _option_card(
|
||||
"💻", "Local (faster-whisper)",
|
||||
"Runs on your machine — private, no API key needed. "
|
||||
"Choose the model size below.")
|
||||
self._rb_remote, card_remote = _option_card(
|
||||
"☁", "Remote API",
|
||||
"Send audio to an OpenAI-compatible endpoint "
|
||||
"(OpenAI, local Whisper server, etc.).")
|
||||
self._rb_remote.join_group(self._rb_local)
|
||||
self._rb_local.set_active(True)
|
||||
|
||||
for rb, card in ((self._rb_local, card_local), (self._rb_remote, card_remote)):
|
||||
ebox = Gtk.EventBox()
|
||||
ebox.add(card)
|
||||
ebox.connect("button-press-event",
|
||||
lambda _e, _ev, r=rb: (r.set_active(True),
|
||||
self._stt_toggle()))
|
||||
box.pack_start(ebox, False, False, 0)
|
||||
rb.connect("toggled", lambda _r: self._stt_toggle())
|
||||
|
||||
# Local options
|
||||
self._stt_local_box = Gtk.Box(orientation=Gtk.Orientation.VERTICAL, spacing=6)
|
||||
self._stt_local_box.set_margin_top(8)
|
||||
size_lbl = Gtk.Label(label="Model size", xalign=0.0)
|
||||
size_lbl.get_style_context().add_class("dim-label")
|
||||
|
||||
self._stt_size_rb: dict[str, Gtk.RadioButton] = {}
|
||||
sizes = [
|
||||
("tiny", "Tiny — fastest, basic accuracy"),
|
||||
("base", "Base — fast, decent accuracy"),
|
||||
("small", "Small — balanced ✓ recommended"),
|
||||
("medium", "Medium — better accuracy, slower"),
|
||||
("large-v3", "Large — best accuracy, most memory"),
|
||||
]
|
||||
size_box = Gtk.Box(orientation=Gtk.Orientation.VERTICAL, spacing=3)
|
||||
prev = None
|
||||
for key, desc in sizes:
|
||||
rb = Gtk.RadioButton(label=desc)
|
||||
if prev:
|
||||
rb.join_group(prev)
|
||||
rb.set_active(key == self._stt_size)
|
||||
rb.connect("toggled", lambda r, k=key: r.get_active() and
|
||||
setattr(self, "_stt_size", k))
|
||||
self._stt_size_rb[key] = rb
|
||||
size_box.pack_start(rb, False, False, 0)
|
||||
prev = rb
|
||||
self._stt_local_box.pack_start(size_lbl, False, False, 0)
|
||||
self._stt_local_box.pack_start(size_box, False, False, 0)
|
||||
box.pack_start(self._stt_local_box, False, False, 0)
|
||||
|
||||
# Remote options
|
||||
self._stt_remote_box = Gtk.Box(orientation=Gtk.Orientation.VERTICAL, spacing=6)
|
||||
self._stt_remote_box.set_margin_top(8)
|
||||
rgrid = Gtk.Grid(column_spacing=8, row_spacing=6)
|
||||
rlbl_url = Gtk.Label(label="API URL", xalign=0.0); rlbl_url.set_size_request(100, -1)
|
||||
self._stt_url_entry = Gtk.Entry(); self._stt_url_entry.set_hexpand(True)
|
||||
self._stt_url_entry.set_text(self._stt_url or "http://localhost:8010/v1")
|
||||
self._stt_url_entry.set_placeholder_text("http://localhost:8010/v1")
|
||||
rlbl_key = Gtk.Label(label="API key env", xalign=0.0)
|
||||
self._stt_key_entry = Gtk.Entry(); self._stt_key_entry.set_hexpand(True)
|
||||
self._stt_key_entry.set_text(self._stt_key)
|
||||
self._stt_key_entry.set_placeholder_text("OPENAI_API_KEY (leave empty if not needed)")
|
||||
rgrid.attach(rlbl_url, 0, 0, 1, 1); rgrid.attach(self._stt_url_entry, 1, 0, 1, 1)
|
||||
rgrid.attach(rlbl_key, 0, 1, 1, 1); rgrid.attach(self._stt_key_entry, 1, 1, 1, 1)
|
||||
self._stt_remote_box.pack_start(rgrid, False, False, 0)
|
||||
self._stt_remote_box.set_no_show_all(True)
|
||||
box.pack_start(self._stt_remote_box, False, False, 0)
|
||||
|
||||
first_run_note = Gtk.Label(xalign=0.0)
|
||||
first_run_note.set_markup(
|
||||
'<span size="small" alpha="65%">'
|
||||
"The local model is downloaded the first time you use it (~500 MB for Small). "
|
||||
"Subsequent starts are instant."
|
||||
"</span>")
|
||||
first_run_note.set_line_wrap(True)
|
||||
first_run_note.set_margin_top(6)
|
||||
box.pack_start(first_run_note, False, False, 0)
|
||||
return box
|
||||
|
||||
def _build_llm(self) -> Gtk.Widget:
|
||||
box = _page_box()
|
||||
box.pack_start(_h1("AI text processing (optional)"), False, False, 0)
|
||||
box.pack_start(_sub(
|
||||
"Blitztext can send your transcript to an AI model that rewrites it — "
|
||||
"cleaning up speech artifacts, adjusting tone, or reformatting it."
|
||||
), False, False, 0)
|
||||
|
||||
enable_row = Gtk.Box(spacing=10)
|
||||
enable_row.set_margin_top(12)
|
||||
self._llm_switch = Gtk.Switch()
|
||||
self._llm_switch.set_active(self._llm_enabled)
|
||||
self._llm_switch.set_valign(Gtk.Align.CENTER)
|
||||
enable_lbl = Gtk.Label(label="Enable AI rewriting", xalign=0.0)
|
||||
enable_row.pack_start(self._llm_switch, False, False, 0)
|
||||
enable_row.pack_start(enable_lbl, False, False, 0)
|
||||
box.pack_start(enable_row, False, False, 0)
|
||||
|
||||
self._llm_detail = Gtk.Box(orientation=Gtk.Orientation.VERTICAL, spacing=6)
|
||||
self._llm_detail.set_margin_top(10)
|
||||
|
||||
lgrid = Gtk.Grid(column_spacing=8, row_spacing=8)
|
||||
llbl_url = Gtk.Label(label="API URL", xalign=0.0); llbl_url.set_size_request(120, -1)
|
||||
self._llm_url_entry = Gtk.Entry(); self._llm_url_entry.set_hexpand(True)
|
||||
self._llm_url_entry.set_text(self._llm_url)
|
||||
self._llm_url_entry.set_placeholder_text("https://api.openai.com/v1")
|
||||
|
||||
llbl_model = Gtk.Label(label="Model", xalign=0.0)
|
||||
self._llm_model_entry = Gtk.Entry(); self._llm_model_entry.set_hexpand(True)
|
||||
self._llm_model_entry.set_text(self._llm_model)
|
||||
self._llm_model_entry.set_placeholder_text("gpt-4o-mini")
|
||||
|
||||
llbl_key = Gtk.Label(label="API key env", xalign=0.0)
|
||||
self._llm_key_entry = Gtk.Entry(); self._llm_key_entry.set_hexpand(True)
|
||||
self._llm_key_entry.set_text(self._llm_key)
|
||||
self._llm_key_entry.set_placeholder_text("OPENAI_API_KEY")
|
||||
|
||||
lgrid.attach(llbl_url, 0, 0, 1, 1); lgrid.attach(self._llm_url_entry, 1, 0, 1, 1)
|
||||
lgrid.attach(llbl_model, 0, 1, 1, 1); lgrid.attach(self._llm_model_entry, 1, 1, 1, 1)
|
||||
lgrid.attach(llbl_key, 0, 2, 1, 1); lgrid.attach(self._llm_key_entry, 1, 2, 1, 1)
|
||||
self._llm_detail.pack_start(lgrid, False, False, 0)
|
||||
|
||||
llm_note = Gtk.Label(xalign=0.0)
|
||||
llm_note.set_markup(
|
||||
'<span size="small" alpha="65%">'
|
||||
"Works with OpenAI, or a local model via Ollama / LM Studio / vLLM. "
|
||||
"You can fine-tune prompts per workflow in Settings → Workflows."
|
||||
"</span>")
|
||||
llm_note.set_line_wrap(True)
|
||||
self._llm_detail.pack_start(llm_note, False, False, 0)
|
||||
|
||||
box.pack_start(self._llm_detail, False, False, 0)
|
||||
|
||||
self._llm_switch.connect("notify::active", lambda s, _p: self._llm_toggle())
|
||||
self._llm_toggle()
|
||||
return box
|
||||
|
||||
def _build_done(self) -> Gtk.Widget:
|
||||
box = _page_box()
|
||||
box.set_valign(Gtk.Align.CENTER)
|
||||
|
||||
lbl_done = Gtk.Label(label="✅")
|
||||
attrs = Pango.AttrList()
|
||||
attrs.insert(Pango.attr_scale_new(3.5))
|
||||
lbl_done.set_attributes(attrs)
|
||||
lbl_done.set_margin_bottom(12)
|
||||
box.pack_start(lbl_done, False, False, 0)
|
||||
|
||||
box.pack_start(_h1("You're all set!"), False, False, 0)
|
||||
self._done_summary = Gtk.Label(xalign=0.0)
|
||||
self._done_summary.set_line_wrap(True)
|
||||
self._done_summary.set_max_width_chars(60)
|
||||
self._done_summary.get_style_context().add_class("dim-label")
|
||||
box.pack_start(self._done_summary, False, False, 0)
|
||||
|
||||
tip = Gtk.Label(xalign=0.0)
|
||||
tip.set_markup(
|
||||
'\n<span size="small" alpha="70%">'
|
||||
"You can always open <b>Settings</b> from the system tray to add workflows, "
|
||||
"tune STT quality, or configure more shortcuts."
|
||||
"</span>")
|
||||
tip.set_line_wrap(True)
|
||||
box.pack_start(tip, False, False, 0)
|
||||
return box
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Page commit helpers
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _read_trigger(self) -> None:
|
||||
if self._rb_ww.get_active():
|
||||
self._trigger = "wakeword"
|
||||
elif self._rb_both.get_active():
|
||||
self._trigger = "both"
|
||||
else:
|
||||
self._trigger = "keyboard"
|
||||
|
||||
def _commit_page(self, name: str) -> None:
|
||||
if name == "trigger":
|
||||
self._read_trigger()
|
||||
self._refresh_page_order()
|
||||
elif name == "keyboard":
|
||||
for attr, entry in self._kb_entries.items():
|
||||
setattr(self, attr, entry.get_text().strip())
|
||||
elif name == "wakeword":
|
||||
self._ww_uri = self._wiz_ww_uri.get_text().strip()
|
||||
self._ww_model = self._wiz_ww_model.get_text().strip()
|
||||
elif name == "stt":
|
||||
self._stt_local = self._rb_local.get_active()
|
||||
if not self._stt_local:
|
||||
self._stt_url = self._stt_url_entry.get_text().strip()
|
||||
self._stt_key = self._stt_key_entry.get_text().strip()
|
||||
elif name == "llm":
|
||||
self._llm_enabled = self._llm_switch.get_active()
|
||||
self._llm_url = self._llm_url_entry.get_text().strip()
|
||||
self._llm_model = self._llm_model_entry.get_text().strip()
|
||||
self._llm_key = self._llm_key_entry.get_text().strip()
|
||||
self._update_done_summary()
|
||||
|
||||
def _update_done_summary(self) -> None:
|
||||
lines: list[str] = []
|
||||
if self._trigger in ("keyboard", "both"):
|
||||
lines.append(f"⌨ Keyboard — start: {self._kb_start}, cancel: {self._kb_cancel}")
|
||||
if self._trigger in ("wakeword", "both"):
|
||||
lines.append(f"🎙 Wakeword — {self._ww_model!r} @ {self._ww_uri}")
|
||||
if self._stt_local:
|
||||
lines.append(f"🤖 Local Whisper ({self._stt_size})")
|
||||
else:
|
||||
lines.append(f"☁ Remote STT — {self._stt_url}")
|
||||
if self._llm_enabled:
|
||||
lines.append(f"✨ AI rewriting — {self._llm_model} @ {self._llm_url}")
|
||||
else:
|
||||
lines.append("✨ AI rewriting — disabled")
|
||||
if hasattr(self, "_done_summary"):
|
||||
self._done_summary.set_text("\n".join(lines))
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Apply to config
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _apply_to_cfg(self) -> None:
|
||||
from .config import save
|
||||
from .stt import STTEngine
|
||||
cfg = self.cfg
|
||||
|
||||
# Trigger / keyboard mode
|
||||
if self._trigger in ("keyboard", "both"):
|
||||
cfg.input_mode = "modifiers"
|
||||
cfg.key_start = self._kb_start
|
||||
cfg.key_stop = self._kb_stop
|
||||
cfg.key_send = self._kb_send
|
||||
cfg.key_cancel = self._kb_cancel
|
||||
|
||||
# Wakeword
|
||||
cfg.wakeword_enabled = self._trigger in ("wakeword", "both")
|
||||
if cfg.wakeword_enabled:
|
||||
cfg.wakeword_uri = self._ww_uri
|
||||
cfg.wakeword_model = self._ww_model
|
||||
from .config import WakewordEngine
|
||||
# Update or create the first engine preset.
|
||||
if cfg.wakeword_engines:
|
||||
cfg.wakeword_engines[0].uri = self._ww_uri
|
||||
cfg.wakeword_engines[0].model = self._ww_model
|
||||
else:
|
||||
cfg.wakeword_engines = [WakewordEngine(
|
||||
name="Default", uri=self._ww_uri, model=self._ww_model)]
|
||||
cfg.wakeword_active = cfg.wakeword_engines[0].name
|
||||
|
||||
# STT
|
||||
if self._stt_local:
|
||||
local_eng = STTEngine(name="Local Whisper", type="local",
|
||||
model=self._stt_size)
|
||||
# Replace or add.
|
||||
locals_ = [e for e in cfg.stt_engines if e.is_local]
|
||||
if locals_:
|
||||
idx = cfg.stt_engines.index(locals_[0])
|
||||
cfg.stt_engines[idx] = local_eng
|
||||
else:
|
||||
cfg.stt_engines.insert(0, local_eng)
|
||||
cfg.stt_active = local_eng.name
|
||||
cfg.model = self._stt_size
|
||||
else:
|
||||
remote_eng = STTEngine(name="Remote STT", type="openai",
|
||||
url=self._stt_url, api_key_env=self._stt_key)
|
||||
remotes = [e for e in cfg.stt_engines if not e.is_local]
|
||||
if remotes:
|
||||
idx = cfg.stt_engines.index(remotes[0])
|
||||
cfg.stt_engines[idx] = remote_eng
|
||||
else:
|
||||
cfg.stt_engines.append(remote_eng)
|
||||
cfg.stt_active = remote_eng.name
|
||||
|
||||
# LLM
|
||||
if self._llm_enabled:
|
||||
cfg.base_url = self._llm_url
|
||||
cfg.rewrite_model = self._llm_model
|
||||
cfg.api_key_env = self._llm_key
|
||||
from .llm import LLMEngine
|
||||
llm_eng = LLMEngine("Default", self._llm_url,
|
||||
self._llm_model, self._llm_key)
|
||||
if cfg.llm_engines:
|
||||
cfg.llm_engines[0] = llm_eng
|
||||
else:
|
||||
cfg.llm_engines = [llm_eng]
|
||||
cfg.llm_active = llm_eng.name
|
||||
|
||||
# Mark setup as complete so the wizard doesn't auto-show again.
|
||||
cfg.setup_complete = True
|
||||
save(cfg)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# UI toggle helpers
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _stt_toggle(self) -> None:
|
||||
local = self._rb_local.get_active()
|
||||
self._stt_local_box.set_visible(local)
|
||||
self._stt_remote_box.set_visible(not local)
|
||||
|
||||
def _llm_toggle(self) -> None:
|
||||
on = self._llm_switch.get_active()
|
||||
self._llm_detail.set_sensitive(on)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Wakeword connection test
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _ww_test(self, _btn) -> None:
|
||||
uri = self._wiz_ww_uri.get_text().strip()
|
||||
self._wiz_ww_test_lbl.set_text("Testing…")
|
||||
self._wiz_ww_test_btn.set_sensitive(False)
|
||||
|
||||
def _probe():
|
||||
import socket as _socket
|
||||
from urllib.parse import urlparse
|
||||
p = urlparse(uri)
|
||||
host = p.hostname or "127.0.0.1"
|
||||
port = p.port or 10400
|
||||
try:
|
||||
with _socket.create_connection((host, port), timeout=3.0):
|
||||
ok = True
|
||||
except OSError:
|
||||
ok = False
|
||||
|
||||
def _update():
|
||||
self._wiz_ww_test_btn.set_sensitive(True)
|
||||
if ok:
|
||||
self._wiz_ww_test_lbl.set_markup(
|
||||
'<span foreground="#2a7d2a">✓ Connected</span>')
|
||||
else:
|
||||
self._wiz_ww_test_lbl.set_markup(
|
||||
'<span foreground="#cc3333">✗ Could not connect — is the server running?</span>')
|
||||
GLib.idle_add(_update)
|
||||
|
||||
threading.Thread(target=_probe, daemon=True).start()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Key-binding capture
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _bind_key(self, entry: Gtk.Entry) -> None:
|
||||
self._bind_entry = entry
|
||||
self._bind_pressed = []
|
||||
entry.set_text("")
|
||||
entry.set_placeholder_text("press the key combination…")
|
||||
|
||||
def _on_key_press(self, _w, event) -> bool:
|
||||
if self._bind_entry is None:
|
||||
return False
|
||||
tok = _keyval_token(event.keyval)
|
||||
if tok and tok not in self._bind_pressed:
|
||||
self._bind_pressed.append(tok)
|
||||
return True
|
||||
|
||||
def _on_key_release(self, _w, event) -> bool:
|
||||
if self._bind_entry is None:
|
||||
return False
|
||||
combo = _format_combo(self._bind_pressed)
|
||||
if combo:
|
||||
self._bind_entry.set_text(combo)
|
||||
self._bind_entry.set_placeholder_text("")
|
||||
self._bind_entry = None
|
||||
self._bind_pressed = []
|
||||
return True
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Public entry point
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def run(self) -> None:
|
||||
self.dlg.show_all()
|
||||
# Hide remote STT box initially (local is default)
|
||||
self._stt_remote_box.hide()
|
||||
self.dlg.run()
|
||||
self.dlg.destroy()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Convenience: show if this is a fresh install
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def maybe_show(cfg, parent: Gtk.Window | None = None) -> None:
|
||||
"""Show the wizard if setup has never been completed."""
|
||||
if not getattr(cfg, "setup_complete", False):
|
||||
wiz = SetupWizard(cfg, parent=parent)
|
||||
wiz.run()
|
||||
130
linux/blitztext/sound.py
Normal file → Executable file
@ -1,38 +1,140 @@
|
||||
"""Play short audio cues (a user WAV, or a built-in system sound) without blocking."""
|
||||
"""Play short audio cues (a user WAV, or a built-in system sound) without blocking.
|
||||
|
||||
Path safety: all user-configured sound file paths are validated before use
|
||||
to prevent path-traversal, device-file access, and symlink attacks.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import stat
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
_FREEDESKTOP = "/usr/share/sounds/freedesktop/stereo/{name}.oga"
|
||||
_PLAYERS = ("pw-play", "paplay", "aplay")
|
||||
|
||||
# (player, extra_args, wav_only)
|
||||
# Ordered: native PipeWire/PulseAudio first, then ffplay/gst as universal fallback.
|
||||
_PLAYERS: list[tuple[str, list[str], bool]] = [
|
||||
("pw-play", [], False),
|
||||
("paplay", [], False),
|
||||
("aplay", [], True), # WAV only
|
||||
("ffplay", ["-nodisp", "-autoexit", "-loglevel", "quiet"], False),
|
||||
("gst-play-1.0", [], False),
|
||||
]
|
||||
_NATIVE_EXTS = {".wav", ".oga", ".ogg", ".flac"}
|
||||
|
||||
# Allowed audio file extensions for user-configured sound paths.
|
||||
_ALLOWED_AUDIO_EXTS = {".wav", ".mp3", ".ogg", ".oga", ".flac",
|
||||
".m4a", ".aac", ".aif", ".aiff", ".opus"}
|
||||
|
||||
# Directories that user-configured sound paths are allowed to resolve into.
|
||||
_ALLOWED_PARENTS: tuple[str, ...] = (
|
||||
str(Path.home()),
|
||||
"/usr/share/sounds",
|
||||
"/opt/blitztext",
|
||||
)
|
||||
|
||||
|
||||
def play(path: str = "", *, fallback: str | None = None) -> None:
|
||||
"""Play `path` (a WAV/OGA file); if unset/missing, play the freedesktop
|
||||
`fallback` system sound. Returns immediately (fire-and-forget)."""
|
||||
def validate_sound_path(path: str) -> str | None:
|
||||
"""Validate a user-configured sound file path.
|
||||
|
||||
Returns the resolved absolute path if the file is safe to play,
|
||||
or ``None`` if the path is empty, does not exist, or fails any
|
||||
security check.
|
||||
|
||||
Checks performed:
|
||||
1. Empty / whitespace-only paths are rejected.
|
||||
2. ``~user`` is expanded via ``expanduser``.
|
||||
3. The path is resolved to an absolute canonical path (``realpath``),
|
||||
which eliminates symlinks and ``..`` components.
|
||||
4. The resolved path must be a regular file (not a device, FIFO,
|
||||
directory, or socket).
|
||||
5. The resolved path must reside under one of the allowed parent
|
||||
directories (user home, system sounds, or the Blitztext install
|
||||
prefix).
|
||||
6. The file extension must be a known audio format.
|
||||
"""
|
||||
if not path or not path.strip():
|
||||
return None
|
||||
|
||||
expanded = os.path.expanduser(path.strip())
|
||||
if not expanded:
|
||||
return None
|
||||
|
||||
try:
|
||||
resolved = Path(expanded).resolve(strict=False)
|
||||
except (OSError, RuntimeError, ValueError):
|
||||
return None
|
||||
|
||||
# Must exist and be a regular file.
|
||||
if not resolved.exists():
|
||||
return None
|
||||
try:
|
||||
mode = resolved.stat().st_mode
|
||||
except OSError:
|
||||
return None
|
||||
if not stat.S_ISREG(mode):
|
||||
return None
|
||||
|
||||
# Must be under an allowed parent directory.
|
||||
resolved_str = str(resolved)
|
||||
allowed = False
|
||||
for parent in _ALLOWED_PARENTS:
|
||||
try:
|
||||
common = os.path.commonpath([resolved_str, parent])
|
||||
if common == parent:
|
||||
allowed = True
|
||||
break
|
||||
except ValueError:
|
||||
continue
|
||||
if not allowed:
|
||||
return None
|
||||
|
||||
# Extension must be a known audio format.
|
||||
ext = resolved.suffix.lower()
|
||||
if ext not in _ALLOWED_AUDIO_EXTS:
|
||||
return None
|
||||
|
||||
return resolved_str
|
||||
|
||||
|
||||
def play(path: str = "", *, fallback: str | None = None) -> "subprocess.Popen | None":
|
||||
"""Play `path` (WAV/MP3/OGG/FLAC/…); fallback to a freedesktop system sound.
|
||||
|
||||
User-supplied paths are validated via :func:`validate_sound_path` before
|
||||
being passed to any audio player. Invalid or unsafe paths are silently
|
||||
ignored (the fallback sound is still attempted).
|
||||
|
||||
Returns the Popen object so callers can terminate a preview, or None.
|
||||
"""
|
||||
target = ""
|
||||
if path:
|
||||
expanded = os.path.expanduser(path)
|
||||
if os.path.exists(expanded):
|
||||
target = expanded
|
||||
safe = validate_sound_path(path)
|
||||
if safe is not None:
|
||||
target = safe
|
||||
if not target and fallback:
|
||||
fd = _FREEDESKTOP.format(name=fallback)
|
||||
if os.path.exists(fd):
|
||||
target = fd
|
||||
if not target:
|
||||
return
|
||||
return None
|
||||
|
||||
# aplay only handles WAV; pw-play/paplay handle WAV + OGA, so try them first.
|
||||
for player in _PLAYERS:
|
||||
ext = os.path.splitext(target)[1].lower()
|
||||
for player, extra, wav_only in _PLAYERS:
|
||||
if not shutil.which(player):
|
||||
continue
|
||||
if player == "aplay" and not target.lower().endswith(".wav"):
|
||||
if wav_only and ext != ".wav":
|
||||
continue
|
||||
if player == "paplay" and ext not in _NATIVE_EXTS:
|
||||
continue
|
||||
try:
|
||||
subprocess.Popen([player, target], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
return
|
||||
proc = subprocess.Popen(
|
||||
[player] + extra + [target],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
|
||||
)
|
||||
return proc
|
||||
except OSError:
|
||||
continue
|
||||
return None
|
||||
@ -27,6 +27,7 @@ class STTEngine:
|
||||
url: str = "" # base URL incl. /v1 for remote, e.g. http://localhost:8010/v1
|
||||
model: str = "" # remote model id, or local whisper size override
|
||||
api_key_env: str = "" # env var holding a bearer key (optional)
|
||||
timeout: int = 30 # HTTP timeout in seconds; raise for slow servers (WhisperX, etc.)
|
||||
|
||||
@property
|
||||
def is_local(self) -> bool:
|
||||
@ -61,31 +62,169 @@ def status(engine: STTEngine, timeout: float = 2.0) -> bool:
|
||||
return reachable(engine.url, timeout)
|
||||
|
||||
|
||||
@dataclass
|
||||
class ModelMeta:
|
||||
"""Model id plus optional metadata (languages, etc.) from the server."""
|
||||
id: str
|
||||
languages: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
def fmt_languages(langs: list[str]) -> str:
|
||||
"""Compact display string for a language list, e.g. 'en, de, fr +45'."""
|
||||
if not langs:
|
||||
return "—"
|
||||
if len(langs) >= 50:
|
||||
return f"multilingual ({len(langs)})"
|
||||
if len(langs) > 5:
|
||||
return f"{', '.join(langs[:5])} +{len(langs) - 5}"
|
||||
return ", ".join(langs)
|
||||
|
||||
|
||||
def _api_base(url: str) -> str:
|
||||
"""Strip endpoint-specific path suffixes so metadata/model probes hit the right root.
|
||||
|
||||
http://host:8081/v1/transcribe → http://host:8081/v1
|
||||
http://host:8081/transcribe → http://host:8081
|
||||
http://host:8081/v1 → http://host:8081/v1 (unchanged)
|
||||
"""
|
||||
from urllib.parse import urlparse, urlunparse
|
||||
p = urlparse(url.rstrip("/"))
|
||||
path = p.path.rstrip("/")
|
||||
v1_idx = path.find("/v1")
|
||||
if v1_idx >= 0:
|
||||
path = path[:v1_idx + 3] # keep up to and including /v1
|
||||
elif path not in ("", "/"):
|
||||
path = "" # strip unknown custom suffix entirely
|
||||
return urlunparse((p.scheme, p.netloc, path, "", "", ""))
|
||||
|
||||
|
||||
def list_models(base_url: str, api_key_env: str = "", timeout: float = 5.0) -> list[str]:
|
||||
"""Fetch model ids from an OpenAI-compatible (or Ollama-style) /models endpoint."""
|
||||
"""Fetch model ids from an OpenAI-compatible, Ollama-style, or Riva/NIM /models endpoint."""
|
||||
return [m.id for m in list_models_meta(base_url, api_key_env, timeout)]
|
||||
|
||||
|
||||
def list_models_meta(base_url: str, api_key_env: str = "", timeout: float = 5.0) -> list[ModelMeta]:
|
||||
"""Like list_models() but returns ModelMeta with language info when available."""
|
||||
import os
|
||||
|
||||
if not base_url:
|
||||
return []
|
||||
url = base_url.rstrip("/") + "/models"
|
||||
headers = {}
|
||||
base = _api_base(base_url)
|
||||
headers: dict[str, str] = {}
|
||||
key = os.environ.get(api_key_env) if api_key_env else None
|
||||
if key:
|
||||
headers["Authorization"] = f"Bearer {key}"
|
||||
|
||||
def _get(url: str) -> dict | None:
|
||||
try:
|
||||
req = urllib.request.Request(url, headers=headers)
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
data = json.loads(resp.read().decode("utf-8"))
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
except (urllib.error.URLError, json.JSONDecodeError, OSError):
|
||||
return None
|
||||
|
||||
# 1. Standard OpenAI /models — faster-whisper-server also returns "language"
|
||||
data = _get(base + "/models")
|
||||
if isinstance(data, dict):
|
||||
items = data.get("data")
|
||||
if isinstance(items, list):
|
||||
result = [ModelMeta(id=m["id"], languages=m.get("language") or [])
|
||||
for m in items if isinstance(m, dict) and m.get("id")]
|
||||
if result:
|
||||
return result
|
||||
items = data.get("models")
|
||||
if isinstance(items, list): # Ollama shape
|
||||
return [ModelMeta(id=m.get("name") or m.get("model", ""))
|
||||
for m in items if m.get("name") or m.get("model")]
|
||||
|
||||
# 2. NVIDIA Riva / NIM
|
||||
data = _get(base + "/metadata")
|
||||
if isinstance(data, dict):
|
||||
for info in data.get("modelInfo") or []:
|
||||
name = info.get("shortName") or info.get("modelUrl") or ""
|
||||
if name:
|
||||
return [ModelMeta(id=name.split(":")[0])]
|
||||
|
||||
return []
|
||||
|
||||
items = data.get("data") if isinstance(data, dict) else None
|
||||
if isinstance(items, list): # OpenAI shape: {"data":[{"id":...}]}
|
||||
return [m["id"] for m in items if isinstance(m, dict) and m.get("id")]
|
||||
items = data.get("models") if isinstance(data, dict) else None
|
||||
if isinstance(items, list): # Ollama shape: {"models":[{"name"/"model":...}]}
|
||||
return [m.get("name") or m.get("model") for m in items if (m.get("name") or m.get("model"))]
|
||||
return []
|
||||
|
||||
def probe_server_ram_mb(base_url: str, timeout: float = 3.0) -> float | None:
|
||||
"""Probe a remote server's Prometheus /metrics endpoint for RSS memory.
|
||||
|
||||
Returns RSS in MB if the server exposes Prometheus metrics with either
|
||||
``process_resident_memory_bytes`` (standard Python/Go exporter) or
|
||||
``container_memory_rss`` (cAdvisor). Returns None if not available.
|
||||
|
||||
Tries the server root first (http://host:port/metrics) then the api base
|
||||
path (http://host:port/v1/metrics), because Prometheus endpoints are almost
|
||||
always mounted at root even when the API lives under /v1.
|
||||
"""
|
||||
if not base_url:
|
||||
return None
|
||||
p = urlparse(base_url.rstrip("/"))
|
||||
root = f"{p.scheme}://{p.netloc}"
|
||||
candidates = [root + "/metrics"]
|
||||
api_base_metrics = _api_base(base_url) + "/metrics"
|
||||
if api_base_metrics != candidates[0]:
|
||||
candidates.append(api_base_metrics)
|
||||
|
||||
def _parse(text: str) -> float | None:
|
||||
for line in text.splitlines():
|
||||
line = line.strip()
|
||||
if line.startswith("#"):
|
||||
continue
|
||||
for metric in ("process_resident_memory_bytes", "container_memory_rss"):
|
||||
if line.startswith(metric):
|
||||
parts = line.split()
|
||||
if len(parts) >= 2:
|
||||
try:
|
||||
return float(parts[-1]) / (1024 * 1024) # bytes → MB
|
||||
except ValueError:
|
||||
pass
|
||||
return None
|
||||
|
||||
for url in candidates:
|
||||
try:
|
||||
req = urllib.request.Request(url)
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
result = _parse(resp.read().decode("utf-8", "replace"))
|
||||
if result is not None:
|
||||
return result
|
||||
except Exception:
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
def detect_remote_device(base_url: str, timeout: float = 3.0) -> str:
|
||||
"""Best-effort GPU/CPU detection for a remote STT server.
|
||||
|
||||
Tries faster-whisper-server's /info endpoint (returns {"device":"cuda",...}),
|
||||
then NVIDIA NIM /metadata (GPU-only service). Falls back to "remote".
|
||||
"""
|
||||
if not base_url:
|
||||
return "remote"
|
||||
base = _api_base(base_url)
|
||||
try:
|
||||
req = urllib.request.Request(base + "/info")
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
data = json.loads(resp.read().decode("utf-8"))
|
||||
if isinstance(data, dict):
|
||||
dev = str(data.get("device") or data.get("compute_type") or "")
|
||||
if "cuda" in dev.lower():
|
||||
return "CUDA"
|
||||
if dev:
|
||||
return dev.upper()[:16]
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
req = urllib.request.Request(base + "/metadata")
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
data = json.loads(resp.read().decode("utf-8"))
|
||||
if isinstance(data, dict) and data.get("modelInfo"):
|
||||
return "CUDA" # NVIDIA NIM is always GPU
|
||||
except Exception:
|
||||
pass
|
||||
return "remote"
|
||||
|
||||
|
||||
def _host_port(url: str) -> tuple[str | None, int]:
|
||||
@ -105,7 +244,7 @@ def transcribe(
|
||||
language: str = "",
|
||||
hotwords: str = "",
|
||||
local_transcriber=None,
|
||||
timeout: int = 60,
|
||||
timeout: int = 30,
|
||||
) -> str:
|
||||
if engine.is_local:
|
||||
if local_transcriber is None:
|
||||
@ -113,15 +252,24 @@ def transcribe(
|
||||
return local_transcriber.transcribe(audio_path, language=language, hotwords=hotwords)
|
||||
if engine.is_streaming:
|
||||
raise STTError("Streaming STT engines are live-only. Use a workflow with mode = \"stream\".")
|
||||
return _transcribe_remote(engine, audio_path, language=language, prompt=hotwords, timeout=timeout)
|
||||
return _transcribe_remote(engine, audio_path, language=language, prompt=hotwords,
|
||||
timeout=engine.timeout if engine.timeout > 0 else timeout)
|
||||
|
||||
|
||||
def _transcribe_remote(engine: STTEngine, audio_path: Path, *, language: str, prompt: str, timeout: int) -> str:
|
||||
import os
|
||||
|
||||
base = engine.url.rstrip("/")
|
||||
# If path is non-standard (not empty / /v1 / /v1.0), treat the full URL as
|
||||
# the endpoint — supports WhisperX (/transcribe) and other custom paths.
|
||||
_path = urlparse(base).path.rstrip("/")
|
||||
if _path in ("", "/v1", "/v1.0"):
|
||||
endpoint = base + "/audio/transcriptions"
|
||||
fields = {"model": engine.model or "whisper-1", "response_format": "json"}
|
||||
else:
|
||||
endpoint = base
|
||||
fields: dict[str, str] = {"response_format": "json"}
|
||||
if engine.model:
|
||||
fields["model"] = engine.model
|
||||
if language:
|
||||
fields["language"] = language
|
||||
if prompt:
|
||||
|
||||
@ -54,8 +54,11 @@ class Transcriber:
|
||||
log(f"{dev} unavailable ({exc}); trying next device")
|
||||
raise RuntimeError(f"Failed to load Whisper model '{model}': {last_err}")
|
||||
|
||||
def transcribe(self, audio_path: Path, language: str = "", hotwords: str = "") -> str:
|
||||
kwargs = dict(language=language or None, beam_size=self.beam_size, vad_filter=True)
|
||||
def transcribe(self, audio_path: Path, language: str = "", hotwords: str = "",
|
||||
beam_size: int | None = None) -> str:
|
||||
kwargs = dict(language=language or None,
|
||||
beam_size=beam_size if beam_size is not None else self.beam_size,
|
||||
vad_filter=True)
|
||||
if hotwords:
|
||||
# Bias recognition toward the routing keywords so they transcribe
|
||||
# reliably. Older faster-whisper builds lack `hotwords`; fall back.
|
||||
|
||||
@ -88,6 +88,15 @@ class Tray:
|
||||
|
||||
menu.append(Gtk.SeparatorMenuItem())
|
||||
|
||||
# Cancel recording — always visible, only sensitive while recording.
|
||||
# Primary escape hatch when wakeword fires on audiobook/TV audio.
|
||||
self.cancel_item = Gtk.MenuItem(label="✕ Cancel recording")
|
||||
self.cancel_item.set_sensitive(False)
|
||||
self.cancel_item.connect(
|
||||
"activate", lambda _i: self.app.daemon.cancel_dictation())
|
||||
menu.append(self.cancel_item)
|
||||
menu.append(Gtk.SeparatorMenuItem())
|
||||
|
||||
# Hands-free wakeword: a reversible pause toggle. Without this, a stale
|
||||
# /tmp/wake_muted flag would silently disable detection with no way back.
|
||||
if getattr(self.app.cfg, "wakeword_enabled", False):
|
||||
@ -120,6 +129,8 @@ class Tray:
|
||||
def update_status(self, state: str, message: str) -> None:
|
||||
self.indicator.set_icon_full(ICONS.get(state, ICONS["idle"]), state)
|
||||
self.status_item.set_label(f"● {message or state.title()}")
|
||||
if hasattr(self, "cancel_item"):
|
||||
self.cancel_item.set_sensitive(state in ("recording", "armed", "busy"))
|
||||
|
||||
def pump(self) -> None:
|
||||
"""Service pending GLib/GTK events; called from tkinter's loop."""
|
||||
|
||||
@ -35,7 +35,7 @@ def set_muted(muted: bool) -> None:
|
||||
elif os.path.exists(MUTE_FILE):
|
||||
os.remove(MUTE_FILE)
|
||||
except OSError as e: # noqa: BLE001 - mute is best-effort, never crash
|
||||
logbuffer.log(f"[wakeword] Could not update mute flag: {e}")
|
||||
logbuffer.log(f"[wakeword] Could not update mute flag: {e}", level="WARNING")
|
||||
|
||||
|
||||
class WakewordListener:
|
||||
@ -67,7 +67,7 @@ class WakewordListener:
|
||||
try:
|
||||
self._stream()
|
||||
except Exception as e:
|
||||
logbuffer.log(f"[wakeword] Connection error: {e}")
|
||||
logbuffer.log(f"[wakeword] Connection error: {e}", level="WARNING")
|
||||
time.sleep(3) # Retry backoff
|
||||
|
||||
def _stream(self):
|
||||
@ -166,3 +166,133 @@ class WakewordListener:
|
||||
logbuffer.log(f"[wakeword] Detected '{self.model}'!")
|
||||
self._cooldown_until = time.time() + 3.0 # 3s cooldown
|
||||
self.on_detect()
|
||||
|
||||
|
||||
class WakewordActionListener:
|
||||
"""Listens for multiple wakeword models simultaneously and calls per-model callbacks.
|
||||
|
||||
Used during active wakeword recording so that dedicated "cancel" and "send"
|
||||
wakeword phrases trigger :meth:`~blitztext.daemon.Daemon.cancel_dictation` or
|
||||
:meth:`~blitztext.daemon.Daemon.finish_dictation` immediately — much faster
|
||||
than waiting for Whisper to transcribe the whole clip.
|
||||
|
||||
``model_callbacks`` is a ``{model_name: callable}`` dict; only the models
|
||||
present in the dict are requested from the server. No cooldown is applied
|
||||
because the listener is torn down immediately after the first action fires.
|
||||
"""
|
||||
|
||||
def __init__(self, uri: str, model_callbacks: dict, mic: str):
|
||||
self.uri = uri
|
||||
self.model_callbacks = dict(model_callbacks) # {name: callable}
|
||||
self.mic = mic
|
||||
self._stop_event = threading.Event()
|
||||
self._thread: threading.Thread | None = None
|
||||
|
||||
def start(self) -> None:
|
||||
if not self.model_callbacks:
|
||||
return
|
||||
self._stop_event.clear()
|
||||
self._thread = threading.Thread(
|
||||
target=self._run, daemon=True, name="WakewordActionListener")
|
||||
self._thread.start()
|
||||
|
||||
def stop(self) -> None:
|
||||
self._stop_event.set()
|
||||
if self._thread:
|
||||
self._thread.join(timeout=2.0)
|
||||
self._thread = None
|
||||
|
||||
def _run(self) -> None:
|
||||
while not self._stop_event.is_set():
|
||||
try:
|
||||
self._stream()
|
||||
except Exception as e:
|
||||
logbuffer.log(f"[wakeword-action] Connection error: {e}", level="WARNING")
|
||||
time.sleep(2)
|
||||
|
||||
def _stream(self) -> None:
|
||||
parsed = urlparse(self.uri)
|
||||
host = parsed.hostname or "127.0.0.1"
|
||||
port = parsed.port or 10400
|
||||
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
|
||||
sock.settimeout(5.0)
|
||||
sock.connect((host, port))
|
||||
logbuffer.log(f"[wakeword-action] Connected — listening for {list(self.model_callbacks)}")
|
||||
|
||||
detect_msg = {"type": "detect", "data": {"names": list(self.model_callbacks)}}
|
||||
sock.sendall((json.dumps(detect_msg) + "\n").encode("utf-8"))
|
||||
|
||||
audio_start = {"type": "audio-start",
|
||||
"data": {"rate": 16000, "width": 2, "channels": 1}}
|
||||
sock.sendall((json.dumps(audio_start) + "\n").encode("utf-8"))
|
||||
|
||||
read_active = True
|
||||
|
||||
def read_loop() -> None:
|
||||
try:
|
||||
sock.settimeout(1.0)
|
||||
while read_active and not self._stop_event.is_set():
|
||||
try:
|
||||
line = b""
|
||||
while not line.endswith(b"\n"):
|
||||
byte = sock.recv(1)
|
||||
if not byte:
|
||||
return
|
||||
line += byte
|
||||
if not line:
|
||||
return
|
||||
msg = json.loads(line.decode("utf-8"))
|
||||
if msg.get("type") == "detection":
|
||||
name = msg.get("data", {}).get("name", "")
|
||||
cb = self.model_callbacks.get(name)
|
||||
if cb is None:
|
||||
# Try partial match — some servers omit the lang suffix
|
||||
for k, v in self.model_callbacks.items():
|
||||
if name.startswith(k) or k.startswith(name):
|
||||
cb = v
|
||||
break
|
||||
if cb:
|
||||
logbuffer.log(
|
||||
f"[wakeword-action] '{name}' detected — firing action")
|
||||
self._stop_event.set() # one-shot: stop after first fire
|
||||
cb()
|
||||
payload_len = msg.get("payload_length", 0)
|
||||
if payload_len > 0:
|
||||
remaining = payload_len
|
||||
while remaining > 0:
|
||||
chunk = sock.recv(min(remaining, 4096))
|
||||
if not chunk:
|
||||
break
|
||||
remaining -= len(chunk)
|
||||
except socket.timeout:
|
||||
pass
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
reader = threading.Thread(target=read_loop, daemon=True)
|
||||
reader.start()
|
||||
|
||||
cmd = ["pw-record", "--rate=16000", "--channels=1", "--format=s16", "-"]
|
||||
if self.mic:
|
||||
cmd.extend(["--target", self.mic])
|
||||
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)
|
||||
|
||||
try:
|
||||
while not self._stop_event.is_set() and proc.poll() is None:
|
||||
chunk = proc.stdout.read(3200)
|
||||
if not chunk:
|
||||
break
|
||||
header = {"type": "audio-chunk",
|
||||
"data": {"rate": 16000, "width": 2, "channels": 1},
|
||||
"payload_length": len(chunk)}
|
||||
sock.sendall((json.dumps(header) + "\n").encode("utf-8"))
|
||||
sock.sendall(chunk)
|
||||
finally:
|
||||
read_active = False
|
||||
proc.terminate()
|
||||
try:
|
||||
proc.wait(timeout=1.0)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
reader.join(timeout=1.0)
|
||||
|
||||
379
linux/blitztext/wakeword_bench.py
Normal file
@ -0,0 +1,379 @@
|
||||
"""Benchmark the wakeword detector with synthetic speech.
|
||||
|
||||
Generates short utterances — random filler text with the wakeword spoken in it,
|
||||
plus pure-filler utterances with no wakeword — synthesizes each in a random voice
|
||||
via the *same* (OpenAI-compatible) endpoint configured for speech-to-text, then
|
||||
streams the audio to the running wyoming-openwakeword server and counts what it
|
||||
detects. Reports:
|
||||
|
||||
• recall — share of wakeword utterances that fired at least one detection
|
||||
• false fires — detections during pure-filler utterances (should be zero)
|
||||
• per-voice recall, so you can see which voices your model handles
|
||||
|
||||
TTS reuses the STT engine on purpose: on a typical NIM/OpenAI-compatible setup
|
||||
the same server answers ``/audio/speech``, so there's nothing extra to configure
|
||||
beyond a TTS model id and a voice list.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import random
|
||||
import socket
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import wave
|
||||
from dataclasses import dataclass, field
|
||||
from urllib.parse import urlparse
|
||||
|
||||
# Bilingual filler so the synthesized speech is sentence-like (the detector sees
|
||||
# realistic context around the wakeword, not just the bare phrase). Unknown
|
||||
# languages fall back to English text — the voice still speaks it in its accent.
|
||||
_FILLERS = {
|
||||
"en": [
|
||||
"the weather today is unusually calm and bright",
|
||||
"remind me to call the office before noon",
|
||||
"i think the train leaves around half past nine",
|
||||
"could you put the report on my desk later",
|
||||
"we should grab a coffee once this is done",
|
||||
"the package was delivered to the wrong address again",
|
||||
],
|
||||
"de": [
|
||||
"das wetter ist heute ungewöhnlich ruhig und klar",
|
||||
"erinnere mich daran das büro vor mittag anzurufen",
|
||||
"ich glaube der zug fährt gegen halb zehn",
|
||||
"könntest du den bericht später auf meinen tisch legen",
|
||||
"wir sollten einen kaffee trinken wenn das erledigt ist",
|
||||
"das paket wurde wieder an die falsche adresse geliefert",
|
||||
],
|
||||
}
|
||||
|
||||
# OpenAI's stock voices; a sensible default for any OpenAI-compatible TTS. Local
|
||||
# servers (Kokoro, openedai-speech, …) expose their own names — override in the
|
||||
# config / Settings to match what your endpoint actually serves.
|
||||
DEFAULT_VOICES = ["alloy", "echo", "fable", "onyx", "nova", "shimmer"]
|
||||
|
||||
_TARGET_RATE = 16000 # wyoming-openwakeword expects 16 kHz mono s16le
|
||||
_CHUNK_BYTES = 3200 # 100 ms per audio-chunk, matching the live listener
|
||||
|
||||
|
||||
@dataclass
|
||||
class Utterance:
|
||||
text: str
|
||||
has_wakeword: bool
|
||||
voice: str
|
||||
detections: int = 0
|
||||
ok: bool = False # synthesized + streamed without error
|
||||
error: str = ""
|
||||
|
||||
|
||||
@dataclass
|
||||
class BenchResult:
|
||||
utterances: list[Utterance] = field(default_factory=list)
|
||||
seconds: float = 0.0
|
||||
|
||||
@property
|
||||
def wake(self) -> list[Utterance]:
|
||||
return [u for u in self.utterances if u.has_wakeword]
|
||||
|
||||
@property
|
||||
def filler(self) -> list[Utterance]:
|
||||
return [u for u in self.utterances if not u.has_wakeword]
|
||||
|
||||
@property
|
||||
def detected(self) -> int:
|
||||
return sum(1 for u in self.wake if u.ok and u.detections > 0)
|
||||
|
||||
@property
|
||||
def expected(self) -> int:
|
||||
return sum(1 for u in self.wake if u.ok)
|
||||
|
||||
@property
|
||||
def recall(self) -> float:
|
||||
return self.detected / self.expected if self.expected else 0.0
|
||||
|
||||
@property
|
||||
def false_fires(self) -> int:
|
||||
return sum(u.detections for u in self.filler if u.ok)
|
||||
|
||||
def recall_by_voice(self) -> dict[str, tuple[int, int]]:
|
||||
"""voice -> (detected, expected) over wakeword utterances."""
|
||||
out: dict[str, list[int]] = {}
|
||||
for u in self.wake:
|
||||
if not u.ok:
|
||||
continue
|
||||
d = out.setdefault(u.voice, [0, 0])
|
||||
d[1] += 1
|
||||
if u.detections > 0:
|
||||
d[0] += 1
|
||||
return {v: (d[0], d[1]) for v, d in out.items()}
|
||||
|
||||
|
||||
def wakeword_phrase(model: str) -> str:
|
||||
"""Turn a wakeword model id into the phrase to speak (best effort).
|
||||
|
||||
"okay_computer" -> "okay computer", "hey_jarvis" -> "hey jarvis". Strips a
|
||||
trailing version/format suffix like "_v0.1" or ".tflite".
|
||||
"""
|
||||
name = model.rsplit("/", 1)[-1]
|
||||
for ext in (".tflite", ".onnx"):
|
||||
if name.endswith(ext):
|
||||
name = name[: -len(ext)]
|
||||
name = name.replace("_", " ").replace("-", " ")
|
||||
# Drop a trailing token that is just a version like "v0.1".
|
||||
parts = [p for p in name.split() if not (p.startswith("v") and any(c.isdigit() for c in p))]
|
||||
return " ".join(parts).strip() or name.strip()
|
||||
|
||||
|
||||
def _filler_pool(language: str) -> list[str]:
|
||||
lang = (language or "").lower()
|
||||
if lang.startswith("de"):
|
||||
return _FILLERS["de"]
|
||||
return _FILLERS["en"]
|
||||
|
||||
|
||||
def build_utterances(phrase: str, count: int, language: str, *,
|
||||
filler_count: int | None = None, voices=None,
|
||||
rng: random.Random | None = None) -> list[Utterance]:
|
||||
"""Build `count` wakeword utterances + `filler_count` pure-filler ones.
|
||||
|
||||
Each wakeword utterance embeds `phrase` at the start, middle, or end of a
|
||||
random filler sentence; voices are assigned round-robin-ish at random so the
|
||||
set covers every configured voice. Deterministic when `rng` is seeded.
|
||||
"""
|
||||
rng = rng or random.Random()
|
||||
voices = list(voices or DEFAULT_VOICES) or DEFAULT_VOICES
|
||||
pool = _filler_pool(language)
|
||||
if filler_count is None:
|
||||
filler_count = max(3, count // 3)
|
||||
|
||||
def a_voice() -> str:
|
||||
return rng.choice(voices)
|
||||
|
||||
out: list[Utterance] = []
|
||||
for _ in range(count):
|
||||
filler = rng.choice(pool)
|
||||
where = rng.choice(("start", "end", "mid"))
|
||||
if where == "start":
|
||||
text = f"{phrase}, {filler}"
|
||||
elif where == "end":
|
||||
text = f"{filler}, {phrase}"
|
||||
else:
|
||||
words = filler.split()
|
||||
cut = len(words) // 2
|
||||
text = " ".join(words[:cut] + [phrase] + words[cut:])
|
||||
out.append(Utterance(text=text, has_wakeword=True, voice=a_voice()))
|
||||
for _ in range(filler_count):
|
||||
out.append(Utterance(text=rng.choice(pool), has_wakeword=False, voice=a_voice()))
|
||||
rng.shuffle(out)
|
||||
return out
|
||||
|
||||
|
||||
# --- TTS via an OpenAI-compatible /audio/speech endpoint ---------------------
|
||||
def _auth_headers(api_key_env: str) -> dict:
|
||||
key = os.environ.get(api_key_env or "", "")
|
||||
return {"Authorization": f"Bearer {key}"} if key else {}
|
||||
|
||||
|
||||
def synthesize(tts_url: str, text: str, *, model: str, voice: str,
|
||||
api_key_env: str = "", timeout: float = 30.0) -> bytes:
|
||||
"""Return 16 kHz mono s16le PCM for `text` from an OpenAI-compatible TTS.
|
||||
|
||||
`tts_url` is the base incl. /v1 (e.g. http://localhost:8880/v1). Asks for WAV
|
||||
and resamples whatever rate/-channels come back down to 16 kHz mono.
|
||||
"""
|
||||
base = (tts_url or "").rstrip("/")
|
||||
if not base:
|
||||
raise RuntimeError("No TTS URL configured (Settings → Benchmark → TTS URL).")
|
||||
payload = json.dumps({
|
||||
"model": model, "input": text, "voice": voice, "response_format": "wav",
|
||||
}).encode("utf-8")
|
||||
headers = {"Content-Type": "application/json", **_auth_headers(api_key_env)}
|
||||
req = urllib.request.Request(base + "/audio/speech", data=payload, headers=headers, method="POST")
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
raw = resp.read()
|
||||
except urllib.error.HTTPError as exc:
|
||||
detail = exc.read().decode("utf-8", "replace")[:200]
|
||||
raise RuntimeError(f"TTS HTTP {exc.code}: {detail}") from exc
|
||||
except (urllib.error.URLError, OSError) as exc:
|
||||
raise RuntimeError(f"TTS request failed: {exc}") from exc
|
||||
return _wav_to_pcm16k(raw)
|
||||
|
||||
|
||||
def list_voices(tts_url: str, *, api_key_env: str = "", timeout: float = 8.0) -> list[str]:
|
||||
"""Best-effort voice discovery (Kokoro/XTTS expose /audio/voices or /voices).
|
||||
|
||||
Returns [] if the server has no such endpoint — voices are then entered by
|
||||
hand. Accepts the common shapes: ["a", …], {"voices": …}, {"data": [{id}…]}.
|
||||
"""
|
||||
base = (tts_url or "").rstrip("/")
|
||||
if not base:
|
||||
return []
|
||||
for path in ("/audio/voices", "/voices"):
|
||||
try:
|
||||
req = urllib.request.Request(base + path, headers=_auth_headers(api_key_env))
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
data = json.loads(resp.read())
|
||||
except Exception: # noqa: BLE001 - endpoint optional/non-standard
|
||||
continue
|
||||
items = data.get("voices") or data.get("data") or [] if isinstance(data, dict) else data
|
||||
out: list[str] = []
|
||||
for it in items or []:
|
||||
if isinstance(it, str):
|
||||
out.append(it)
|
||||
elif isinstance(it, dict):
|
||||
name = it.get("id") or it.get("name") or it.get("voice")
|
||||
if name:
|
||||
out.append(str(name))
|
||||
if out:
|
||||
return out
|
||||
return []
|
||||
|
||||
|
||||
def probe(tts_url: str, *, model: str, voice: str, api_key_env: str = "",
|
||||
timeout: float = 20.0) -> tuple[bool, str]:
|
||||
"""Connectivity check: synthesize one short phrase. Returns (ok, message)."""
|
||||
try:
|
||||
pcm = synthesize(tts_url, "connection test", model=model, voice=voice,
|
||||
api_key_env=api_key_env, timeout=timeout)
|
||||
except Exception as exc: # noqa: BLE001 - surface the reason to the user
|
||||
return False, str(exc)
|
||||
return True, f"Connected — “{voice}” returned {len(pcm) / 2 / _TARGET_RATE:.1f}s of audio."
|
||||
|
||||
|
||||
def _wav_to_pcm16k(wav_bytes: bytes) -> bytes:
|
||||
"""Decode a WAV blob to 16 kHz mono signed-16 PCM (linear resample)."""
|
||||
import numpy as np # type: ignore[import-untyped]
|
||||
|
||||
with wave.open(io.BytesIO(wav_bytes), "rb") as w:
|
||||
ch, width, rate, n = w.getnchannels(), w.getsampwidth(), w.getframerate(), w.getnframes()
|
||||
frames = w.readframes(n)
|
||||
if width != 2:
|
||||
raise RuntimeError(f"Unexpected TTS sample width {width*8}-bit (need 16-bit WAV).")
|
||||
a = np.frombuffer(frames, dtype=np.int16).astype(np.float32)
|
||||
if ch > 1:
|
||||
a = a.reshape(-1, ch).mean(axis=1)
|
||||
if rate != _TARGET_RATE and a.size:
|
||||
new_len = int(round(a.size * _TARGET_RATE / rate))
|
||||
if new_len > 0:
|
||||
xp = np.linspace(0.0, 1.0, num=a.size, endpoint=False)
|
||||
x = np.linspace(0.0, 1.0, num=new_len, endpoint=False)
|
||||
a = np.interp(x, xp, a)
|
||||
return np.clip(a, -32768, 32767).astype("<i2").tobytes()
|
||||
|
||||
|
||||
# --- wyoming-openwakeword detection ------------------------------------------
|
||||
def count_detections(uri: str, model: str, pcm: bytes, *, settle: float = 1.5,
|
||||
timeout: float = 15.0) -> int:
|
||||
"""Stream `pcm` (16 kHz mono s16le) to wyoming-openwakeword; count detections.
|
||||
|
||||
Sends a fresh detect/audio-start/…/audio-stop session, then drains detection
|
||||
events until the server falls quiet for `settle` seconds (or `timeout`).
|
||||
"""
|
||||
parsed = urlparse(uri)
|
||||
host, port = parsed.hostname or "127.0.0.1", parsed.port or 10400
|
||||
deadline = time.time() + timeout
|
||||
detections = 0
|
||||
with socket.create_connection((host, port), timeout=5.0) as sock:
|
||||
_send(sock, {"type": "detect", "data": {"names": [model]}})
|
||||
_send(sock, {"type": "audio-start", "data": {"rate": 16000, "width": 2, "channels": 1}})
|
||||
for i in range(0, len(pcm), _CHUNK_BYTES):
|
||||
chunk = pcm[i:i + _CHUNK_BYTES]
|
||||
_send(sock, {"type": "audio-chunk",
|
||||
"data": {"rate": 16000, "width": 2, "channels": 1},
|
||||
"payload_length": len(chunk)}, chunk)
|
||||
_send(sock, {"type": "audio-stop", "data": {}})
|
||||
sock.settimeout(settle)
|
||||
buf = b""
|
||||
while time.time() < deadline:
|
||||
try:
|
||||
data = sock.recv(4096)
|
||||
except socket.timeout:
|
||||
break # server quiet for `settle`s → done
|
||||
if not data:
|
||||
break
|
||||
buf += data
|
||||
buf, n = _drain_detections(buf)
|
||||
detections += n
|
||||
return detections
|
||||
|
||||
|
||||
def _send(sock: socket.socket, msg: dict, payload: bytes = b"") -> None:
|
||||
sock.sendall((json.dumps(msg) + "\n").encode("utf-8"))
|
||||
if payload:
|
||||
sock.sendall(payload)
|
||||
|
||||
|
||||
def _drain_detections(buf: bytes) -> tuple[bytes, int]:
|
||||
"""Parse whole newline-framed messages from buf; return (rest, detections).
|
||||
|
||||
Consumes each message's binary payload too, so payload bytes are never
|
||||
mistaken for the next header line.
|
||||
"""
|
||||
found = 0
|
||||
while b"\n" in buf:
|
||||
line, rest = buf.split(b"\n", 1)
|
||||
try:
|
||||
msg = json.loads(line.decode("utf-8"))
|
||||
except (ValueError, UnicodeDecodeError):
|
||||
return rest, found
|
||||
plen = msg.get("payload_length", 0) or 0
|
||||
if len(rest) < plen:
|
||||
return buf, found # payload not fully arrived yet; wait for more
|
||||
rest = rest[plen:]
|
||||
if msg.get("type") == "detection":
|
||||
found += 1
|
||||
buf = rest
|
||||
return buf, found
|
||||
|
||||
|
||||
@dataclass
|
||||
class EngineRun:
|
||||
"""One wakeword engine's benchmark result."""
|
||||
name: str
|
||||
uri: str
|
||||
model: str
|
||||
result: BenchResult
|
||||
|
||||
|
||||
def run(engines, *, tts_url: str, tts_model: str, tts_api_key_env: str = "", voices=None,
|
||||
language: str = "", count: int = 12, seed: int | None = None,
|
||||
progress=None) -> list[EngineRun]:
|
||||
"""Benchmark each wakeword engine against the same synthesized speech.
|
||||
|
||||
`engines` is a list of objects with ``.name`` / ``.uri`` / ``.model`` (e.g.
|
||||
``config.WakewordEngine``). Each engine is scored on `count` utterances of
|
||||
its own wake phrase (+ filler). Audio is synthesized once per (text, voice)
|
||||
and reused across engines that share a phrase, so adding engines is cheap.
|
||||
Calls ``progress(engine_idx, engine_total, engine_name, done, total, u)``.
|
||||
"""
|
||||
cache: dict[tuple[str, str], bytes] = {}
|
||||
runs: list[EngineRun] = []
|
||||
n_eng = len(engines)
|
||||
for ei, eng in enumerate(engines, 1):
|
||||
phrase = wakeword_phrase(eng.model)
|
||||
utterances = build_utterances(phrase, count, language, voices=voices, rng=random.Random(seed))
|
||||
res = BenchResult(utterances=utterances)
|
||||
total = len(utterances)
|
||||
t0 = time.time()
|
||||
for ui, u in enumerate(utterances, 1):
|
||||
try:
|
||||
key = (u.text, u.voice)
|
||||
pcm = cache.get(key)
|
||||
if pcm is None:
|
||||
pcm = synthesize(tts_url, u.text, model=tts_model, voice=u.voice,
|
||||
api_key_env=tts_api_key_env)
|
||||
cache[key] = pcm
|
||||
u.detections = count_detections(eng.uri, eng.model, pcm)
|
||||
u.ok = True
|
||||
except Exception as exc: # noqa: BLE001 - record per-utterance, keep going
|
||||
u.error = str(exc)
|
||||
if progress:
|
||||
progress(ei, n_eng, eng.name or eng.model, ui, total, u)
|
||||
res.seconds = time.time() - t0
|
||||
runs.append(EngineRun(eng.name or eng.model, eng.uri, eng.model, res))
|
||||
return runs
|
||||
@ -1,49 +1,87 @@
|
||||
#!/usr/bin/env bash
|
||||
# Set up Blitztext for Linux: create a venv, install deps, check host tools,
|
||||
# and write the default config. Run from the linux/ directory.
|
||||
# Install Blitztext for Linux (no root needed).
|
||||
#
|
||||
# Creates a self-contained venv at ~/.local/lib/blitztext (outside any CIFS
|
||||
# share, using --copies so no symlinks are needed), copies the package into it,
|
||||
# and writes a launcher to ~/.local/bin/blitztext. The source folder can be
|
||||
# deleted after this script finishes.
|
||||
set -euo pipefail
|
||||
|
||||
cd "$(dirname "$0")"
|
||||
VENV=".venv"
|
||||
APPDIR="$HOME/.local/lib/blitztext"
|
||||
VENV="$APPDIR/venv"
|
||||
BIN="$HOME/.local/bin/blitztext"
|
||||
|
||||
echo "==> Checking host tools"
|
||||
need_pkg=()
|
||||
command -v xdotool >/dev/null || need_pkg+=("xdotool")
|
||||
command -v notify-send >/dev/null || need_pkg+=("libnotify-bin")
|
||||
# python3-gi (PyGObject) must come from apt — it can't be pip-installed.
|
||||
python3 -c "import gi" 2>/dev/null || need_pkg+=(python3-gi python3-gi-cairo gir1.2-gtk-3.0 gir1.2-appindicator3-0.1)
|
||||
command -v xdotool >/dev/null || need_pkg+=(xdotool)
|
||||
command -v notify-send >/dev/null || need_pkg+=(libnotify-bin)
|
||||
if ! command -v pw-record >/dev/null && ! command -v arecord >/dev/null && ! command -v parecord >/dev/null; then
|
||||
need_pkg+=("pipewire-bin (or alsa-utils)")
|
||||
need_pkg+=(pipewire-bin)
|
||||
fi
|
||||
if ((${#need_pkg[@]})); then
|
||||
echo " Missing host tools: ${need_pkg[*]}"
|
||||
echo " On Ubuntu/Debian: sudo apt install xdotool libnotify-bin pipewire-bin"
|
||||
echo " (continuing — install them before running the daemon)"
|
||||
if (( ${#need_pkg[@]} )); then
|
||||
echo " Missing system packages — installing now (requires sudo):"
|
||||
echo " sudo apt install ${need_pkg[*]}"
|
||||
sudo apt-get install -y "${need_pkg[@]}"
|
||||
fi
|
||||
|
||||
# --system-site-packages lets the venv see an apt-installed PyGObject (python3-gi)
|
||||
# for the optional system-tray mode; harmless if it's not installed.
|
||||
echo "==> Creating venv at $VENV"
|
||||
python3 -m venv --system-site-packages "$VENV"
|
||||
mkdir -p "$APPDIR"
|
||||
# --copies avoids lib64->lib symlinks that fail on CIFS/SMB shares.
|
||||
# --system-site-packages lets the venv see apt-installed python3-gi for the tray.
|
||||
python3 -m venv --copies --system-site-packages "$VENV"
|
||||
"$VENV/bin/pip" install --upgrade pip -q
|
||||
"$VENV/bin/pip" install -r requirements.txt
|
||||
"$VENV/bin/pip" install -q -r requirements.txt
|
||||
|
||||
echo "==> Installing blitztext package"
|
||||
# Copy the package into site-packages so this source folder can be deleted.
|
||||
cp -r blitztext "$VENV/lib/python"*/site-packages/
|
||||
|
||||
# On Debian/Ubuntu, python3-gi lives in /usr/lib/python3/dist-packages — the
|
||||
# generic (non-versioned) path that venvs don't include even with
|
||||
# --system-site-packages. A .pth file adds it to sys.path at startup.
|
||||
SITE_PKG="$("$VENV/bin/python" -c "import site; print(site.getsitepackages()[0])")"
|
||||
echo "/usr/lib/python3/dist-packages" > "$SITE_PKG/debian-system-packages.pth"
|
||||
|
||||
echo "==> Writing default config (if absent)"
|
||||
"$VENV/bin/python" -m blitztext config-path
|
||||
|
||||
echo "==> Installing launcher → $BIN"
|
||||
mkdir -p "$(dirname "$BIN")"
|
||||
# Use a Python entry-point script (not `python -m`) so the current working
|
||||
# directory is never added to sys.path and can't shadow the installed package.
|
||||
cat > "$BIN" <<EOF
|
||||
#!$VENV/bin/python
|
||||
import sys
|
||||
from blitztext.blitztext import main
|
||||
sys.exit(main())
|
||||
EOF
|
||||
chmod +x "$BIN"
|
||||
|
||||
# Copy icons to hicolor so the app shows up properly in the tray / launcher.
|
||||
ICON_DIR="$HOME/.local/share/icons/hicolor"
|
||||
PKG_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
for s in 32 48 64 128 256; do
|
||||
SRC="$PKG_DIR/packaging/blitztext_${s}.png"
|
||||
if [ -f "$SRC" ]; then
|
||||
DEST="$ICON_DIR/${s}x${s}/apps/blitztext.png"
|
||||
mkdir -p "$(dirname "$DEST")"
|
||||
cp "$SRC" "$DEST"
|
||||
fi
|
||||
done
|
||||
gtk-update-icon-cache -qf "$ICON_DIR" 2>/dev/null || true
|
||||
|
||||
cat <<EOF
|
||||
|
||||
Done. To start it:
|
||||
Done! Blitztext is installed at $APPDIR
|
||||
|
||||
$VENV/bin/python -m blitztext tray # system tray (needs: sudo apt install python3-gi)
|
||||
$VENV/bin/python -m blitztext gui # control-panel window
|
||||
$VENV/bin/python -m blitztext run # headless, hotkeys only
|
||||
Start it:
|
||||
blitztext tray # system tray
|
||||
blitztext gui # control-panel window
|
||||
|
||||
Edit your config (hotkeys, Whisper model, rewrite endpoint) at:
|
||||
(Make sure ~/.local/bin is in your PATH — it is by default on Ubuntu.)
|
||||
|
||||
\$($VENV/bin/python -m blitztext config-path)
|
||||
|
||||
For the rewrite workflows, export your key first, e.g.:
|
||||
|
||||
export OPENAI_API_KEY=sk-...
|
||||
|
||||
To run it in the background on login, see blitztext.service in this folder.
|
||||
You can now delete the source folder if you cloned it just for installation.
|
||||
EOF
|
||||
|
||||
@ -6,6 +6,7 @@ Comment=Linux voice dictation with local, remote, and realtime STT engines
|
||||
Exec=blitztext tray
|
||||
Icon=blitztext
|
||||
Terminal=false
|
||||
StartupWMClass=blitztext
|
||||
Categories=Utility;AudioVideo;Accessibility;
|
||||
Keywords=dictation;speech;voice;transcription;whisper;stt;riva;nim;
|
||||
StartupNotify=false
|
||||
|
||||
@ -44,6 +44,9 @@ rm -rf "$VENV/lib/python"*/site-packages/pip "$VENV/lib/python"*/site-packages/p
|
||||
"$VENV"/bin/pip* "$VENV/lib/python"*/site-packages/setuptools* 2>/dev/null || true
|
||||
|
||||
cp "$LINUX_DIR/README.md" "$LINUX_DIR/CHANGELOG.md" "$APPDIR/"
|
||||
# MANUAL.md lives at repo root (one level above linux/)
|
||||
REPO_MANUAL="$(dirname "$LINUX_DIR")/MANUAL.md"
|
||||
[ -f "$REPO_MANUAL" ] && cp "$REPO_MANUAL" "$APPDIR/MANUAL.md"
|
||||
|
||||
# 2) Launcher, desktop entry, icon, docs ------------------------------------
|
||||
install -Dm755 /dev/stdin "$ROOT/usr/bin/$PKG" <<'EOF'
|
||||
@ -104,4 +107,9 @@ chmod 755 "$ROOT/DEBIAN/postrm"
|
||||
mkdir -p "$OUT_DIR"
|
||||
DEB="$OUT_DIR/${PKG}_${VER}_${ARCH}.deb"
|
||||
fakeroot dpkg-deb --build --root-owner-group "$ROOT" "$DEB" >/dev/null
|
||||
chmod 644 "$DEB"
|
||||
# Also copy to ~ so `sudo apt install ~/blitztext_*.deb` works without a separate cp
|
||||
DEST="$HOME/$(basename "$DEB")"
|
||||
install -m644 "$DEB" "$DEST"
|
||||
echo "==> Built: $DEB ($(du -h "$DEB" | cut -f1))"
|
||||
echo "==> Copied: $DEST"
|
||||
|
||||
@ -68,3 +68,26 @@ def test_properties():
|
||||
]
|
||||
assert cfg.preset_by_name("WF2").name == "WF2"
|
||||
assert cfg.preset_by_name("NonExistent") is None
|
||||
|
||||
|
||||
def test_default_preset_prefers_transcribe_when_unset():
|
||||
"""With no routing default, fall back to a transcribe preset, not whatever
|
||||
happens to be first (which could be an LLM rewrite)."""
|
||||
cfg = Config()
|
||||
cfg.routing_default = ""
|
||||
cfg.workflows = [
|
||||
Workflow(name="Improve text", hotkey="", mode="rewrite"),
|
||||
Workflow(name="Transcribe", hotkey="", mode="transcribe"),
|
||||
]
|
||||
dp = cfg.default_preset
|
||||
assert dp is not None and dp.mode == "transcribe" and dp.name == "Transcribe"
|
||||
|
||||
|
||||
def test_default_preset_honours_explicit_name():
|
||||
cfg = Config()
|
||||
cfg.workflows = [
|
||||
Workflow(name="Improve text", hotkey="", mode="rewrite"),
|
||||
Workflow(name="Transcribe", hotkey="", mode="transcribe"),
|
||||
]
|
||||
cfg.routing_default = "Improve text"
|
||||
assert cfg.default_preset.name == "Improve text"
|
||||
|
||||
@ -85,6 +85,64 @@ def test_wakeword_cues_independent_of_master_switch(monkeypatch):
|
||||
assert plays == [], "an unset wakeword cue should be silent, not fall back to a system sound"
|
||||
|
||||
|
||||
def test_matched_preset_announced_even_hands_free(monkeypatch):
|
||||
"""You should see which preset/keyword you triggered, even via the wakeword.
|
||||
The routing announcement is gated by notify_routing, NOT by _session_silent."""
|
||||
shown = []
|
||||
|
||||
def fake_notify(*a, enabled=True, **k):
|
||||
if enabled:
|
||||
shown.append(a)
|
||||
|
||||
monkeypatch.setattr(daemon_mod, "notify", fake_notify)
|
||||
d = _make_daemon(monkeypatch)
|
||||
d._session_silent = True # hands-free session
|
||||
|
||||
d.cfg.notify_routing = True
|
||||
d._rnotify("⚡ Nicer email", "matched: “nicer email”")
|
||||
assert shown, "the matched preset must be announced even for hands-free sessions"
|
||||
|
||||
shown.clear()
|
||||
d.cfg.notify_routing = False
|
||||
d._rnotify("⚡ Nicer email", "matched: “nicer email”")
|
||||
assert shown == [], "no announcement when 'Announce matched preset' is off"
|
||||
|
||||
|
||||
def test_overlay_fuses_notifications(monkeypatch):
|
||||
"""With an overlay (routing_cb wired), the on-screen bubble narrates every
|
||||
phase, so informational desktop notifications are suppressed — but errors
|
||||
still pop a bubble, and the routing match is shown on the overlay, not a
|
||||
notification."""
|
||||
calls = []
|
||||
monkeypatch.setattr(daemon_mod, "notify", lambda *a, **k: calls.append((a, k)))
|
||||
monkeypatch.setattr(daemon_mod, "detect_recorder", lambda pref="auto": "pw-record")
|
||||
d = Daemon(Config(), routing_cb=lambda *a: None)
|
||||
assert d._overlay is True
|
||||
d._session_silent = False
|
||||
|
||||
d._dnotify("⌛ Nicer email", "Rewriting…") # informational
|
||||
assert calls == [], "informational notifications fuse into the overlay"
|
||||
|
||||
d._dnotify("Error", "boom", urgency="critical") # errors still notify
|
||||
assert len(calls) == 1, "errors must still pop a desktop bubble"
|
||||
|
||||
calls.clear()
|
||||
d.cfg.notify_routing = True
|
||||
d._rnotify("⚡ Nicer email", "matched: “nicer email”")
|
||||
assert calls == [], "the match is shown on the overlay banner, not as a notification"
|
||||
|
||||
|
||||
def test_no_overlay_keeps_notifications(monkeypatch):
|
||||
"""Headless / overlay-off: notifications remain the only feedback."""
|
||||
calls = []
|
||||
monkeypatch.setattr(daemon_mod, "notify", lambda *a, **k: calls.append((a, k)))
|
||||
d = _make_daemon(monkeypatch) # no routing_cb
|
||||
assert d._overlay is False
|
||||
d._session_silent = False
|
||||
d._dnotify("⌛ Nicer email", "Rewriting…")
|
||||
assert len(calls) == 1, "without an overlay, informational notifications still show"
|
||||
|
||||
|
||||
def test_wakeword_while_busy_does_not_notify(monkeypatch):
|
||||
"""The away-from-keyboard "Busy" storm: a detection arriving while the
|
||||
previous clip is still being processed must be ignored silently."""
|
||||
|
||||
41
linux/tests/test_llm_stream.py
Normal file
@ -0,0 +1,41 @@
|
||||
"""Streaming the LLM rewrite into the overlay (so you watch the model write).
|
||||
|
||||
`llm.chat(..., on_token=cb)` must parse an OpenAI-style SSE stream, hand each
|
||||
content delta to the callback, and still return the complete text.
|
||||
"""
|
||||
|
||||
import io
|
||||
|
||||
import blitztext.llm as llm
|
||||
|
||||
|
||||
def test_read_stream_accumulates_and_calls_back():
|
||||
sse = (
|
||||
'data: {"choices":[{"delta":{"content":"Hel"}}]}\n'
|
||||
"\n" # keep-alive blank line
|
||||
'data: {"choices":[{"delta":{"content":"lo"}}]}\n'
|
||||
'data: {"choices":[{"delta":{}}]}\n' # role-only / empty delta
|
||||
": comment line\n" # SSE comment, ignored
|
||||
"data: [DONE]\n"
|
||||
'data: {"choices":[{"delta":{"content":"X"}}]}\n' # after DONE -> ignored
|
||||
)
|
||||
tokens = []
|
||||
out = llm._read_stream(io.BytesIO(sse.encode("utf-8")), tokens.append)
|
||||
|
||||
assert out == "Hello"
|
||||
assert tokens == ["Hel", "lo"]
|
||||
|
||||
|
||||
def test_read_stream_survives_callback_errors():
|
||||
sse = (
|
||||
'data: {"choices":[{"delta":{"content":"a"}}]}\n'
|
||||
'data: {"choices":[{"delta":{"content":"b"}}]}\n'
|
||||
"data: [DONE]\n"
|
||||
)
|
||||
|
||||
def boom(_delta):
|
||||
raise RuntimeError("UI exploded")
|
||||
|
||||
# A failing UI callback must not break accumulation / delivery.
|
||||
out = llm._read_stream(io.BytesIO(sse.encode("utf-8")), boom)
|
||||
assert out == "ab"
|
||||
@ -51,3 +51,23 @@ def test_strip_span():
|
||||
# 'nicer e-mail' normalizes to 3 tokens
|
||||
res = _strip_span("Nicer e-mail can you send me the report", 3, "start")
|
||||
assert res == "can you send me the report"
|
||||
|
||||
|
||||
def test_route_matches_preset_name_without_keywords():
|
||||
"""A preset with no keywords is still speakable by its name."""
|
||||
presets = [
|
||||
DummyPreset("Transcribe", []),
|
||||
DummyPreset("Nicer email", []), # no keywords configured
|
||||
]
|
||||
res = route("nicer email please call me back tomorrow", presets)
|
||||
assert res.preset_name == "Nicer email"
|
||||
assert res.text == "please call me back tomorrow"
|
||||
|
||||
|
||||
def test_explicit_keyword_still_wins_over_name():
|
||||
presets = [
|
||||
DummyPreset("Add emojis", ["emoji"]),
|
||||
]
|
||||
res = route("emoji this is great", presets)
|
||||
assert res.preset_name == "Add emojis"
|
||||
assert res.text == "this is great"
|
||||
|
||||
79
linux/tests/test_voice_cancel.py
Normal file
@ -0,0 +1,79 @@
|
||||
"""Spoken abort: a cancel word discards a clip before it is acted on.
|
||||
|
||||
Rescues accidentally triggered (e.g. wakeword) dictations — the transcript is
|
||||
recognised but nothing is routed, rewritten, or typed.
|
||||
"""
|
||||
|
||||
import blitztext.daemon as dm
|
||||
from blitztext.config import Config, Workflow
|
||||
from blitztext.daemon import Daemon
|
||||
from blitztext.routing import is_cancel
|
||||
|
||||
|
||||
# -- matcher --------------------------------------------------------------------
|
||||
def test_is_cancel_matches_edges_only():
|
||||
kws = ["abbrechen", "cancel"]
|
||||
assert is_cancel("abbrechen", kws) == "abbrechen" # whole utterance
|
||||
assert is_cancel("ok das ganze bitte abbrechen", kws) == "abbrechen" # end edge
|
||||
assert is_cancel("cancel this please", kws) == "cancel" # start edge
|
||||
assert is_cancel("abbrechn", ["abbrechen"]) == "abbrechen" # ASR drift, fuzzy
|
||||
# Word buried mid-sentence must NOT cancel a legitimate dictation.
|
||||
assert is_cancel("ich will den vorgang abbrechen weil es spaet ist", ["abbrechen"]) is None
|
||||
assert is_cancel("hallo welt", ["abbrechen"]) is None
|
||||
assert is_cancel("abbrechen", []) is None # disabled
|
||||
assert is_cancel("", ["abbrechen"]) is None
|
||||
|
||||
|
||||
def test_cancel_keywords_round_trip(tmp_path):
|
||||
from blitztext.config import load, save
|
||||
p = tmp_path / "config.toml"
|
||||
cfg = load(p)
|
||||
assert cfg.cancel_keywords == ["abbrechen", "cancel"] # shipped default
|
||||
cfg.cancel_keywords = ["nein doch nicht", "scrap that"]
|
||||
save(cfg, p)
|
||||
assert load(p).cancel_keywords == ["nein doch nicht", "scrap that"]
|
||||
|
||||
|
||||
# -- pipeline -------------------------------------------------------------------
|
||||
def _wire_clean_pipeline(monkeypatch):
|
||||
monkeypatch.setattr(dm, "detect_recorder", lambda pref="auto": "pw-record")
|
||||
monkeypatch.setattr(dm, "notify", lambda *a, **k: None)
|
||||
monkeypatch.setattr(dm.quality, "analyze_wav", lambda p: (2.0, 0.5))
|
||||
monkeypatch.setattr(dm.quality, "too_quiet", lambda *a, **k: False)
|
||||
monkeypatch.setattr(dm.quality, "clean", lambda t, **k: t)
|
||||
monkeypatch.setattr(dm.quality, "is_hallucination", lambda *a, **k: False)
|
||||
delivered = []
|
||||
monkeypatch.setattr(dm, "deliver", lambda *a, **k: delivered.append((a, k)))
|
||||
return delivered
|
||||
|
||||
|
||||
def test_process_discards_when_cancel_spoken(monkeypatch, tmp_path):
|
||||
delivered = _wire_clean_pipeline(monkeypatch)
|
||||
monkeypatch.setattr(dm.stt, "transcribe", lambda *a, **k: "ok das ganze bitte abbrechen")
|
||||
d = Daemon(Config())
|
||||
d._prepared = True
|
||||
d.cfg.cancel_keywords = ["abbrechen", "cancel"]
|
||||
|
||||
audio = tmp_path / "clip.wav"
|
||||
audio.write_bytes(b"x")
|
||||
d._process(audio, Workflow(name="Transcribe", hotkey="", mode="transcribe"), None)
|
||||
|
||||
assert delivered == [], "a voice-cancelled clip must never be typed"
|
||||
assert not audio.exists(), "the temp clip is still cleaned up afterwards"
|
||||
assert d._busy is False
|
||||
|
||||
|
||||
def test_process_delivers_without_cancel_word(monkeypatch, tmp_path):
|
||||
"""Control: the same path with no cancel word still types normally."""
|
||||
delivered = _wire_clean_pipeline(monkeypatch)
|
||||
monkeypatch.setattr(dm.stt, "transcribe", lambda *a, **k: "hallo welt")
|
||||
d = Daemon(Config())
|
||||
d._prepared = True
|
||||
d.cfg.cancel_keywords = ["abbrechen", "cancel"]
|
||||
|
||||
audio = tmp_path / "clip.wav"
|
||||
audio.write_bytes(b"x")
|
||||
d._process(audio, Workflow(name="Transcribe", hotkey="", mode="transcribe"), None)
|
||||
|
||||
assert len(delivered) == 1, "a normal clip must still be delivered"
|
||||
assert delivered[0][0][0] == "hallo welt"
|
||||
83
linux/tests/test_voice_send.py
Normal file
@ -0,0 +1,83 @@
|
||||
"""Spoken send: a configured word delivers the clip AND presses Enter.
|
||||
|
||||
The spoken equivalent of stop+paste+Enter — mainly hands-free. The keyword is
|
||||
stripped from the transcript before the rest is delivered and submitted.
|
||||
"""
|
||||
|
||||
import blitztext.daemon as dm
|
||||
import blitztext.paste as paste
|
||||
from blitztext.config import Config, Workflow
|
||||
from blitztext.daemon import Daemon
|
||||
from blitztext.routing import match_send
|
||||
|
||||
|
||||
# -- matcher --------------------------------------------------------------------
|
||||
def test_match_send_strips_edges_only():
|
||||
kws = ["computer send", "computer abschicken"]
|
||||
assert match_send("computer send", kws)[0] == "computer send" # whole utterance
|
||||
kw, text = match_send("hey team the build is green computer send", kws)
|
||||
assert kw == "computer send" and text == "hey team the build is green" # end edge, stripped
|
||||
kw, text = match_send("computer send hey team the build is green", kws)
|
||||
assert kw == "computer send" and text == "hey team the build is green" # start edge, stripped
|
||||
# A bare word inside a sentence must NOT submit — the phrase is distinctive.
|
||||
assert match_send("please send me the report tomorrow", kws)[0] is None
|
||||
assert match_send("computer send", [])[0] is None # disabled
|
||||
assert match_send("", kws)[0] is None
|
||||
|
||||
|
||||
# -- pipeline -------------------------------------------------------------------
|
||||
def _wire_clean_pipeline(monkeypatch):
|
||||
monkeypatch.setattr(dm, "detect_recorder", lambda pref="auto": "pw-record")
|
||||
monkeypatch.setattr(dm, "notify", lambda *a, **k: None)
|
||||
monkeypatch.setattr(dm.quality, "analyze_wav", lambda p: (2.0, 0.5))
|
||||
monkeypatch.setattr(dm.quality, "too_quiet", lambda *a, **k: False)
|
||||
monkeypatch.setattr(dm.quality, "clean", lambda t, **k: t)
|
||||
monkeypatch.setattr(dm.quality, "is_hallucination", lambda *a, **k: False)
|
||||
delivered = []
|
||||
monkeypatch.setattr(dm, "deliver", lambda *a, **k: delivered.append((a, k)))
|
||||
enters = []
|
||||
monkeypatch.setattr(paste, "press_enter", lambda win=None: enters.append(win))
|
||||
return delivered, enters
|
||||
|
||||
|
||||
def test_process_sends_with_enter_when_send_spoken(monkeypatch, tmp_path):
|
||||
delivered, enters = _wire_clean_pipeline(monkeypatch)
|
||||
monkeypatch.setattr(dm.stt, "transcribe", lambda *a, **k: "die nachricht ist fertig computer send")
|
||||
d = Daemon(Config())
|
||||
d._prepared = True
|
||||
d.cfg.send_keywords = ["computer send"]
|
||||
d.cfg.cancel_keywords = []
|
||||
|
||||
audio = tmp_path / "clip.wav"; audio.write_bytes(b"x")
|
||||
d._process(audio, Workflow(name="Transcribe", hotkey="", mode="transcribe"), None)
|
||||
|
||||
assert len(delivered) == 1
|
||||
assert delivered[0][0][0] == "die nachricht ist fertig" # keyword stripped
|
||||
assert len(enters) == 1, "send keyword must press Enter"
|
||||
|
||||
|
||||
def test_process_no_enter_without_send_word(monkeypatch, tmp_path):
|
||||
"""Control: the same path without a send word delivers but never hits Enter."""
|
||||
delivered, enters = _wire_clean_pipeline(monkeypatch)
|
||||
monkeypatch.setattr(dm.stt, "transcribe", lambda *a, **k: "die nachricht ist fertig")
|
||||
d = Daemon(Config())
|
||||
d._prepared = True
|
||||
d.cfg.send_keywords = ["computer send"]
|
||||
d.cfg.cancel_keywords = []
|
||||
|
||||
audio = tmp_path / "clip.wav"; audio.write_bytes(b"x")
|
||||
d._process(audio, Workflow(name="Transcribe", hotkey="", mode="transcribe"), None)
|
||||
|
||||
assert len(delivered) == 1
|
||||
assert delivered[0][0][0] == "die nachricht ist fertig"
|
||||
assert enters == [], "no send word → no Enter"
|
||||
|
||||
|
||||
def test_send_keywords_round_trip(tmp_path):
|
||||
from blitztext.config import load, save
|
||||
p = tmp_path / "config.toml"
|
||||
cfg = load(p)
|
||||
assert cfg.send_keywords == [] # shipped default: off
|
||||
cfg.send_keywords = ["computer send", "computer abschicken"]
|
||||
save(cfg, p)
|
||||
assert load(p).send_keywords == ["computer send", "computer abschicken"]
|
||||
68
linux/tests/test_wakeword_bench.py
Normal file
@ -0,0 +1,68 @@
|
||||
"""Wakeword benchmark: deterministic parts (script, audio decode, framing, scoring).
|
||||
|
||||
The network parts (TTS /audio/speech, streaming to wyoming-openwakeword) need
|
||||
live servers and are exercised manually from Settings → Benchmark; here we cover
|
||||
everything that can be checked offline.
|
||||
"""
|
||||
|
||||
import io
|
||||
import json
|
||||
import random
|
||||
import wave
|
||||
|
||||
import numpy as np
|
||||
|
||||
from blitztext import wakeword_bench as wb
|
||||
|
||||
|
||||
def test_wakeword_phrase():
|
||||
assert wb.wakeword_phrase("okay_computer") == "okay computer"
|
||||
assert wb.wakeword_phrase("hey_jarvis") == "hey jarvis"
|
||||
assert wb.wakeword_phrase("models/okay_nabu.tflite") == "okay nabu"
|
||||
assert wb.wakeword_phrase("alexa_v0.1") == "alexa"
|
||||
|
||||
|
||||
def test_build_utterances_deterministic_and_covers_voices():
|
||||
a = wb.build_utterances("computer", 8, "de", voices=["nova", "onyx"], rng=random.Random(7))
|
||||
b = wb.build_utterances("computer", 8, "de", voices=["nova", "onyx"], rng=random.Random(7))
|
||||
key = lambda us: [(u.text, u.has_wakeword, u.voice) for u in us]
|
||||
assert key(a) == key(b) # seeded → reproducible
|
||||
assert sum(u.has_wakeword for u in a) == 8
|
||||
assert all(u.voice in ("nova", "onyx") for u in a)
|
||||
assert all("computer" in u.text for u in a if u.has_wakeword)
|
||||
assert any(not u.has_wakeword for u in a) # filler added for false-fire check
|
||||
|
||||
|
||||
def test_wav_to_pcm_resamples_to_16k_mono():
|
||||
buf = io.BytesIO()
|
||||
with wave.open(buf, "wb") as w:
|
||||
w.setnchannels(2); w.setsampwidth(2); w.setframerate(48000)
|
||||
n = 48000 # 1.0 s stereo @ 48 kHz
|
||||
tone = (np.sin(2 * np.pi * 440 * np.linspace(0, 1, n, endpoint=False)) * 8000).astype("<i2")
|
||||
w.writeframes(np.repeat(tone, 2).tobytes())
|
||||
pcm = wb._wav_to_pcm16k(buf.getvalue())
|
||||
assert len(pcm) == 16000 * 2 # 1.0 s of 16 kHz mono s16le
|
||||
|
||||
|
||||
def test_drain_detections_handles_payloads():
|
||||
det = (json.dumps({"type": "detection", "data": {}}) + "\n").encode()
|
||||
chunk = (json.dumps({"type": "audio-chunk", "payload_length": 3}) + "\n").encode() + b"abc"
|
||||
rest, n = wb._drain_detections(chunk + det)
|
||||
assert n == 1 and rest == b"" # payload skipped, detection counted
|
||||
held = (json.dumps({"type": "x", "payload_length": 10}) + "\n").encode() + b"ab"
|
||||
rest2, n2 = wb._drain_detections(held)
|
||||
assert n2 == 0 and rest2 == held # incomplete payload held, not misparsed
|
||||
|
||||
|
||||
def test_bench_result_metrics():
|
||||
r = wb.BenchResult(utterances=[
|
||||
wb.Utterance("a", True, "nova", detections=1, ok=True),
|
||||
wb.Utterance("b", True, "nova", detections=0, ok=True),
|
||||
wb.Utterance("c", True, "onyx", detections=2, ok=True),
|
||||
wb.Utterance("d", False, "onyx", detections=1, ok=True), # false fire
|
||||
wb.Utterance("e", True, "x", ok=False, error="boom"), # failed → excluded
|
||||
])
|
||||
assert r.expected == 3 and r.detected == 2
|
||||
assert abs(r.recall - 2 / 3) < 1e-9
|
||||
assert r.false_fires == 1
|
||||
assert r.recall_by_voice() == {"nova": (1, 2), "onyx": (1, 1)}
|
||||
@ -1,5 +1,8 @@
|
||||
{
|
||||
"extraPaths": [
|
||||
"/usr/lib/python3/dist-packages"
|
||||
]
|
||||
],
|
||||
"venvPath": "linux",
|
||||
"venv": ".venv",
|
||||
"reportMissingModuleSource": "none"
|
||||
}
|
||||
|
||||