yunchan/d3ro-voice
1
0
Fork 0
Code Issues Pull requests Projects Releases 3 Packages Wiki Activity Actions

feat(keybinding): several shortcuts per action, mouse buttons, searchable picker

Browse source 
Shortcuts were defined in four places that drifted apart: per-action IPC channel
pairs, a hand-written VK table in the service, a second one in the renderer, and
three copies of the keycap styling. Adding an action meant editing all of them,
so two shortcuts stayed hardcoded in bootstrap and one had no settings entry at
all.

packages/core/src/keybinding.ts is now the single source for the binding type,
the selectable key catalog, the action catalog, normalization, validation,
conflict detection, display labels, search and deserialization. Main, preload
and renderer all read from it; nothing redefines keys or rules locally.

- Each action holds a list of bindings instead of one. AppConfig's four
  *Shortcut fields collapse into a single keyBindings map, migrated on launch.
- Mouse buttons can be bound. Left click is refused, right/middle need a
  modifier, side buttons are free. uiohook cannot swallow events, so the
  original click still fires and the UI says so.
- Keys can be picked from a grouped dropdown with a search box, not only by
  recording a keypress.
- HOTKEY's 14 channels become KEYBINDING's 9, taking the action as a parameter,
  so actions no longer multiply channels. The history and command popups moved
  out of bootstrap into ordinary actions.
- displayLabel is gone; labels derive from the binding and follow the app
  language and platform.

Fixes found on the way:
- Double-press hands-free was unreachable: lookup returned only the first
  matching action, and dictation shares its default binding.
- Reserved-combination checks compared joined key names, so a different modifier
  order let Ctrl+C through.
- Disabling shortcuts released every global registration in the process,
  including the popup ones, and never restored them.
- Enabling shortcuts after starting disabled left nothing registered.
- The dashboard stored the caption event payload instead of the state in it.
This commit is contained in:
Yun Chan 2026-09-21 13:41:47 +09:00
parent 0ca9e242fa
commit 4ad1ae6ed4
49 changed files with 5901 additions and 1792 deletions
Download patch file Download diff file Expand all files Collapse all files

11
docs/map/10-feature-catalog.md
View file

@ -14,8 +14,8 @@ Status quick-reference: `[x]` done+verified · `[~]` partial/unverified · `[ ]`
| ID | Feature | D | W | M | B | Anchors / notes |
|---|---|---|---|---|---|---|
| CAP-01 | Push-to-talk dictation (hold/release) | [x] | [-] | [x] | [-] | Desktop `VoiceModeService`; mobile RecordScreen via app CTA/notification action (no global hotkey) |
| CAP-02 | Hands-free toggle dictation | [x] | [-] | [x] | [-] | Desktop double-press; mobile toggle |
| CAP-01 | Push-to-talk dictation (hold/release) | [x] | [-] | [x] | [-] | Desktop `VoiceModeService`; the trigger is now the rebindable `dictation` action of CAP-16 (several bindings per action, keyboard or mouse) rather than a single stored shortcut. The pipeline itself is unchanged and tested; the rewritten entry layer was confirmed in the 2026-09-21 manual run (CAP-16). Mobile RecordScreen via app CTA/notification action (no global hotkey) |
| CAP-02 | Hands-free toggle dictation | [x] | [-] | [x] | [-] | Desktop double-press shares the `dictation` binding and is split by the action's `doublePress` flag (`KeyBindingService.ts:680`). This path was **dead in shipped builds**: the previous lookup returned only the first matching action, so with both actions on the same binding double-press never reached hands-free. Fixed and confirmed in the 2026-09-21 manual run (CAP-16); core tests cover the contract side (same binding is not a conflict, `keybinding.test.ts:499`/`:734`). `KeyBindingService` still has no unit test of its own (GAP-KEY-01 evidence). Mobile toggle |
| CAP-03 | Live partial transcript while recording | [x] | [ ] | [ ] | [-] | Desktop `voice:partialTranscript` + recording-tip; producer added in `1.3.0` (`VoiceModeService._runPartial` → `LocalSTTService.transcribePartial`, 1.5 s cadence / 7.5 s window, never inserted). The row was `[x]` before any producer existed. |
| CAP-04 | Recording waveform + level meter | [x] | [x] | [x] | [-] | Desktop 9-bar cos distribution; mobile audio level; the recording-tip popup bundle and its on-disk assets are verified by `scripts/ci/verify-desktop-renderer-bundles.mjs` |
| CAP-05 | Device/mic selection | [x] | [ ] | [~] | [-] | Desktop config; mobile uses system default |
@ -29,6 +29,7 @@ Status quick-reference: `[x]` done+verified · `[~]` partial/unverified · `[ ]`
| CAP-13 | Live captions overlay | [x] | [-] | [-] | [-] | Desktop `CaptionService` + caption-overlay popup; popup assets verified in the packaged build (`scripts/ci/verify-desktop-renderer-bundles.mjs`); GAP-INFRA-05 |
| CAP-14 | Recording persistence / crash recovery | [x] | [ ] | [x] | [-] | Desktop WAV persist; mobile durable queue + process-kill WAV recovery |
| CAP-15 | Android foreground recording service | [-] | [-] | [x] | [-] | Mobile API 34 FGS + persistent notification (SSOT R-005 GREEN) |
| CAP-16 | Rebindable global key bindings (keyboard + mouse) | [x] | [-] | [-] | [-] | Contract SSOT `packages/core/src/keybinding.ts`: `KEY_CATALOG` (10 groups, `:615`), `KEYBINDING_ACTIONS` (6 actions, `:719`), `validateBinding` (`:953`), `detectBindingConflicts` (`:1016`). Multiple bindings per action persist as one `AppConfig.keyBindings` map (`packages/core/src/types.ts:459`), replacing the four singular `*Shortcut` fields; `ConfigService` migrates legacy values once (`ConfigService.ts:142`). `KeyBindingService` hooks keyboard **and** mouse via uiohook (`KeyBindingService.ts:387`) — MB1 is not bindable, MB2/MB3 need a modifier, MB4/MB5 are free, and no mouse button can be suppressed, so the original click still fires (warning surfaced in the UI). Selection is either key-recording or a searchable grouped dropdown (`KeyBindingPicker.tsx:536`). `history-popup`/`command-popup` were hardcoded in `bootstrap.ts` and are now rebindable actions (`bootstrap.ts:159`). **Verified 2026-09-21 on Windows by a manual run** (`%APPDATA%/d3ro-voice/logs/main.log`, 12:53–13:06): `ConfigService` migrated the four legacy shortcuts with the user's non-default values preserved exactly, `KeyBindingService` loaded 6 bindings for 6 actions and started the uiohook keyboard **and** mouse hook with zero boot errors, and keyboard plus mouse (MB4/MB5) bindings were exercised through the UI. A `Loaded 7 key binding(s) … for 6 action(s)` line later in the same session shows multi-binding working end to end. The migrated map was read back from `d3ro-voice-config.json`: legacy `*Shortcut` fields gone, no `displayLabel` left. Contract evidence: `packages/core` 117 tests GREEN, no renderer type errors in the key-binding files. **Still open:** `KeyBindingService` has no unit test of its own, macOS/Linux mouse behavior is unconfirmed (`11` GAP-KEY-02), and `command` still falls back to the dictation pipeline (GAP-KEY-03). W/M `[-]`: no OS-level global binding surface exists there (browser sandbox; mobile has no global hotkey, see CAP-01). B `[-]`: device-local setting, nothing server-side. See `11` GAP-KEY-02/03 (open), GAP-KEY-01 (`[x]`), and `11` §7 CONSTRAINT-I18N-01. |
---
@ -158,9 +159,9 @@ Status quick-reference: `[x]` done+verified · `[~]` partial/unverified · `[ ]`
| ID | Feature | D | W | M | B | Anchors / notes |
|---|---|---|---|---|---|---|
| SHELL-01 | Settings / preferences | [x] | [~] | [x] | [x] | Desktop tabbed modal; web theme/i18n; mobile `SettingsScreen` |
| SHELL-01 | Settings / preferences | [x] | [~] | [x] | [x] | Desktop tabbed modal; the General tab hosts the whole key-binding editor (CAP-16: global on/off switch + one `KeyBindingField` per action, grouped voice/window — `SettingsModal.tsx:239`), which is also the first settings entry point the `command` action ever had; web theme/i18n; mobile `SettingsScreen` |
| SHELL-02 | Theme system (6 themes) | [x] | [x] | [x] | [-] | `theme.ts` SSOT |
| SHELL-03 | i18n (12 locales) | [x] | [x] | [x] | [-] | `@d3ro/i18n`; ko/en fully translated, others partial |
| SHELL-03 | i18n (12 locales) | [x] | [x] | [x] | [-] | `@d3ro/i18n`; ko/en fully translated, others partial. Measured 2026-09-21: `ko` 1716 keys / `en` 1709 / the other ten 327 each, so ~1,380 keys fall back for non-English locales — tracked as `11` GAP-I18N-01 |
| SHELL-04 | Onboarding / first-run | [x] | [ ] | [x] | [-] | Desktop model bootstrap; mobile audience/theme/locale |
| SHELL-05 | Accessibility / reduced motion | [~] | [~] | [~] | [-] | Desktop reduced-motion honored; mobile a11y rows pending |
| SHELL-06 | System tray / background | [x] | [-] | [-] | [-] | Desktop tray |
@ -203,7 +204,7 @@ Status quick-reference: `[x]` done+verified · `[~]` partial/unverified · `[ ]`
| Surface | `[x]` | `[~]` | `[ ]` | Notable strength | Notable weakness |
|---|---|---|---|---|---|
| Desktop | ~40 | 3 | ~8 | Local AI pipeline, meetings, RAG, conversation, hotkeys | Ads stubs, no team admin, no email account |
| Desktop | ~40 | 3 | ~8 | Local AI pipeline, meetings, RAG, conversation, key bindings | Ads stubs, no team admin, no email account |
| Web | ~22 | 6 | ~14 | Server-shared data UX, billing, meetings, teams | No local AI, limited knowledge upload/search |
| Mobile | ~40 | 12 | ~18 | Cloud + native recording, portability, admin, IAP/ads | External store/console gates, a11y, deep E2E pending |
| Backend | ~45 | 6 | ~4 | RLS, Edge functions, billing, fail-closed AI | Payple webhook signature, some external provider keys |