A rebindable screen-ocr action (default Ctrl+Alt+O) freezes the monitor under the cursor and
covers it with a focusable overlay. Text is selected by flow drag, rectangle drag, double/triple
click, Shift+click or Ctrl+A, then copied, saved as PNG or searchable PDF, translated,
summarized, read aloud, opened as a link or searched.
- Selection, text assembly and the PDF document are pure functions in @d3ro/core/screen-ocr,
shared by the main process and the overlay.
- Recognition uses Windows.Media.Ocr through powershell.exe -EncodedCommand with 2x upscaling;
the capture reaches disk only as a temp PNG that is removed right after recognition.
- Capture uses GDI BitBlt for the target monitor (39 ms) because desktopCapturer captured all
three monitors and took 2.9 s when DXGI duplication timed out.
- The accelerator is reserved only where the feature is available, so macOS does not swallow
the key for a feature it cannot run (fails closed with a message).
Map: CAP-17 [~], GAP-OCR-01 (live hotkey journey not yet driven in the running app),
GAP-OCR-02 (macOS engine, mixed-language accuracy).
The lock desktop swallows keyup, so a hold-to-talk recording never ended,
the first shortcut after unlocking was dropped as auto-repeat, and Win from
Win+L made the next suggestion accept wait out the full 1.5s modifier timeout.
On powerMonitor lock-screen/suspend the chord machine now emits released for
emitted triggers and clears its state, and modifier tracking forgets held keys.
Typo auto-correct (INPUT-19)
- TypoCorrectionService + core typo-correction: 2-set Korean layout
mistypes both ways, always-wrong spellings, near-misses of the user's
frequent words, optional model check re-validated in core; applied by
backspace N + paste and cancelled on new keys or window change.
Prediction quality (INPUT-20/21)
- gemma models are sent raw with the user's unfinished sentence pre-filled
into the model turn, so the model truly continues the text (mid-word
joins and spacing exact). Other models keep the instruction prompt.
- Context: text before the caret (opening lines and a fact list of names,
dates, amounts outside the 400-char window), messages just sent in the
same chat window (memory only, 30 min), personal-graph follow-ups and
IDF-ranked related sentences with Korean stemming.
- Guards: first sentence only, restated-prefix stripping (also fixes the
instruction path), repeat drop, no invented dates/names/numbers
(one retry), plain -다 register, no emoji unless the user uses them,
search-box keywords. Fill loop mixes instruction prompts for variety.
- Opt-in real-model probe: tests/integration/suggestion-quality.ollama.test.ts
Suggestions now actually show up
- Terminal (Claude Code): caret trust from the input line, not the whole
screen (status line below the box blocked every keystroke); Korean
composition counts as typing at the end; a grey placeholder filling the
empty box at once is not user text.
- Chrome: UIA focus flicker while typing no longer resets "edited".
- KakaoTalk/RichEdit: trailing paragraph mark ignored for edit position
and stripped so accepted text keeps its separating space.
- While the overlay is visible the suggestion shortcuts are captured so
Ctrl+Alt+Enter no longer also types a newline; released when hidden.
- nut-js preloaded after boot (first accept waited 4.8 s).