fix(desktop): resolve Ctrl+Alt+Enter accept drops and duplicate typo LLM checks
This commit is contained in:
parent
2a62f2496a
commit
02d0d0c6c1
15 changed files with 238 additions and 19 deletions
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
> Status: ACTIVE
|
||||
> Last full audit: 2026-09-13
|
||||
> Last update: 2026-10-31 — **v1.15.1 릴리스 + 터미널·xterm.js 환경 한글 1글자씩 씹힘 근본 해결(`INPUT-04`, `INPUT-19`, `GAP-INPUT-08`, `GAP-INPUT-11`).** 터미널(`Agent Switchboard.exe` `xterm-helper-textarea`, `WindowsTerminal.exe`, `Kitedesk.exe`, `pwsh.exe`, `cmd.exe` 등)에서 한글 연속 타이핑 시 글자가 한 글자씩 씹히던 4가지 원인을 전수 제거했다: (1) `InputTelemetryService._handleKeyDown`에서 터미널 표면일 때 매 키다운마다 호출하던 크로스-프로세스 `isForegroundKoreanIme`(`SendMessageTimeoutW(WM_IME_CONTROL)`)가 Windows TSF/DefIME 조합 상태를 흔들던 문제를 제거하고 유휴 스냅샷 시점 1회 동기화(`TerminalKeystrokeTracker.syncKoreanIme`)로 전환, (2) `sidecar/uia_bridge.py`에서 `xterm-helper-textarea` 조회 시 Chromium 렌더러 메인 스레드를 최대 1,500ms 블로킹하고 `comp=True` 고착을 유발하던 `GetTopLevelControl()`·`TextPattern.GetSelection()`·`ExpandToEnclosingUnit(Character)`·`TextEditPattern.GetActiveComposition()` 호출을 건너뛰는 고속 경로 추가 및 `_WAKE_TTL_SECONDS` 60s→86,400s 연장, (3) 일반 한글 단어(`"근데 "`, `"터미널 "` 등) 입력 중 공백 직후 40ms 경계 스냅샷·680ms 중복 타이머·800ms 타이핑 중 폴링 억제(`_shouldRunFastTerminalBoundarySnapshot`, `_sampleWhileTyping`), (4) `TypoCorrectionService`에서 터미널의 공백 전 완성형 한글 음절(`[가-힣]$`)에 대해 `commitComposition`(`VK_SPACE`+`VK_BACK`) 주입을 원천 차단. 직전 같은 날 `v1.15.0`: `CommandsPage.tsx` 다단계 명령 파이프라인 복수 선택·순서 조정 UI 및 단일 패스 합성 실행(`AI-05`, `AI-07`).
|
||||
> Last update: 2026-10-31 — **v1.15.2 릴리스 + `Ctrl+Alt+Enter` 제안 수락 불발 근본 해결 + 터미널·xterm.js 한글 씹힘 및 UIA·LLM 캐시 최적화(`INPUT-04`, `INPUT-06`, `INPUT-19`, `GAP-INPUT-08`, `GAP-INPUT-11`).** (1) **`Ctrl+Alt+Enter` 제안 수락 근본 해결:** ① `classifyKeyStroke`가 수정자 VK(`0x10..0x12`, `0x5b..0x5c`, `0xa0..0xa5`)보다 수정자 플래그를 먼저 검사해 `Ctrl`/`Alt`를 누르는 순간 `'shortcut'`(`key=shortcut+C+A`)으로 분류되어 `TerminalKeystrokeTracker` 버퍼가 `prefixLen=0`으로 초기화되고 `Enter`를 누르기 전에 제안 창이 닫히던 버그 수정, ② `ChordStateMachine.keyUp`에서 `Ctrl` 또는 `RightAlt`를 먼저 놓았을 때 `command`(`Ctrl+RightAlt`)만 해제하고 `break`/`return`으로 빠져나가 `suggestion-accept`(`Ctrl+Alt+Enter`)가 `_isDown`에 영구 고착되어 이후 입력이 auto-repeat으로 무시되던 버그 수정, ③ `KeyBindingService`에 `UiohookKey.NumpadEnter` 매핑 및 OS `globalShortcut` 캡처 25ms 폴백 트리거 추가, ④ `"생성 중..."` 또는 첫 후보 스트리밍 중 `Ctrl+Alt+Enter`를 누르면 무시하지 않고 첫 후보 완성(최대 2.5s) 즉시 수락(`_waitForFirstCandidate`)하도록 개선, ⑤ 터미널(`terminalMode` / `isTerminalApp`) 수락 시 클립보드 `Ctrl+V` 경합 대신 `KEYEVENTF_UNICODE` 직접 주입 및 백그라운드 채우기 루프 즉시 중단. (2) **UIA·Ollama 최적화:** 비타이핑 키 및 짧은 입력에서 780ms 중복 `_settleTimer` 스냅샷을 취소하고, `TypoCorrectionService.propose`가 이미 검사 완료된 구간(`_isRegionChecked`)에 대해 336토큰 오타 LLM 호출을 중복 실행해 제안 KV 캐시를 밀어내던 문제를 제거했다. (3) 직전 `v1.15.1`: 터미널 `WM_IME_CONTROL` 키다운 호출 제거 및 `uia_bridge.py` `xterm-helper-textarea` 고속 경로 적용.
|
||||
>
|
||||
> Previous update: 2026-10-30 — **백로그 26건 일괄 해소 (`GAP-INPUT-03`, `GAP-LLM-01`, `GAP-UX-01/02/03/05/06/07`, `GAP-I18N-01`, `GAP-REL-11/14/15`, `GAP-INFRA-01/02/04/06`, `GAP-OPS-01`, `GAP-QA-02`, `GAP-KEY-01/03`, `GAP-ADS-02`, `GAP-ID-01/02`, `GAP-TEAM-02`, `GAP-SHELL-01`, `GAP-PUSH-03`, `GAP-MTG-01`, `DOC-01`) + 입력 누락 검증 인프라 및 서비스 결함 재현(`INPUT-04/19`, `INFRA-20`, `GAP-INPUT-11`).** (1) **백로그 26건 해소:** CJK 무공백 어절 집계(`countWords`), 12개 번역 대상 언어 설정(`targetLanguage`), 12개 로케일 2,716개 키 100% 동기화(`sync-i18n-locales.mjs`), 데스크톱 strict 타입체크 0 에러(`tsconfig.node.json` + `tsconfig.web.json`), Forgejo 릴리스 태그 푸시 순서·SHA 고정(`GAP-REL-14`), 포터블 워크플로 매니페스트 검증(`GAP-REL-11`), 네이티브 ABI 스위처(`rebuild:app`/`rebuild:test`), Supabase 마이그레이션 정합성 검사기(`check:migrations`), NAS 배포 런북(`nas-deploy-runbook.md`), 오프라인 격리 테스트 하네스(`GAP-QA-02`), `KeyBindingService` 단위 테스트·`command` 전용 액션 핸들러, 데스크톱 광고 보상 쿼터 연동(`grantRewardQuota`), 회의 종료 후 메모 추가·노트패드 동기화, 이력 SQLite 커서 페이지네이션·인라인 편집·전체 내보내기, 회의 요약 `[MM:SS]` 타임스탬프 인용·클릭 시크 연동(데스크톱·웹·모바일), `resolveEntitlement` 정본 통일, 통합 검색(`UnifiedSearchService`) 취소·세그먼트 검색·페이로드 경량화, 웹 팀 초대 수락(`/app/accept-invite`), 관리자 고객지원 콘솔(`/support`), Web Push 서비스워커(`sw.js`)·iOS APNs 토큰 등록을 완료했다. (2) **입력 누락 검증 인프라(`GAP-INPUT-11`):** `typing-race-proof.mjs`가 체크아웃을 바꾸지 않고 3개 서비스만 HEAD `f49acf0`와 수정본으로 바꿔 같은 현재 의존성·시험에서 비교했다: HEAD 1/17, 수정본 17/17. 3개 문자열 손상 사례와 13개 스냅샷·메타데이터·조회 병합 계약 실패를 구분했고, 정상 교정 대조군은 양쪽 모두 통과했다. `typing-fixture`(scan-code·IME·CPU 부하)와 `typing-runtime`(별도 로컬 프로필·실제 제품 번들)을 준비했으며, 실제 GUI·IME·CPU·모델 cold/warm 시험은 아직 미실행이다. 증거: `test-results/typing-load/race-proof-2026-10-30T00-50-32-284Z/report.json`; 실행 명령은 `02` §3, 검증 경계는 `04` §6과 `11` GAP-INPUT-11.
|
||||
>
|
||||
|
|
|
|||
|
|
@ -84,7 +84,7 @@ Singleton + `EventEmitter` pattern (`getXService()` accessors).
|
|||
| Service / file | Purpose |
|
||||
|---|---|
|
||||
| `InputTelemetryService` | Global input telemetry and context observer: keystroke/click/scroll counters, mouse travel (Manhattan sum of per-axis deltas), active time, per-hour-per-app buckets flushed every 5 s; typed-text learning via UIA snapshot diffs. **Since 2026-10-26**, the UIA context observer runs by default whenever `typoAutoCorrectEnabled !== false` (or `suggestionEnabled === true`), while SQLite `input_activity` persistence only runs when `inputTelemetryEnabled === true` (opt-in). **Since 2026-10-27**, includes `TerminalKeystrokeTracker` (`@d3ro/core/input-intelligence`) + `isForegroundKoreanIme` (`utils/win32-foreground.ts`) to reconstruct the active terminal/Vim line (`reconcileWithScreen` for UIA screen buffers like Windows Terminal/pwsh/cmd, `reconcileWithXtermValue` for `xterm.js` `<textarea class="xterm-helper-textarea">` controls in Agent Switchboard/VS Code/Cursor where `.value` stays populated until Enter/Ctrl+C, and `getSnapshot` for canvas/remote terminals with `len=0` like Kitedesk/WezTerm/Alacritty), preserving trailing spaces stripped by grid terminals and tracking Vim modes (`insert`, `normal`, `command`). Programmatic keys (`_programmaticKeys`) no longer advance `_lastKeyAt`, and `Enter`-triggered chat submissions that clear or shrink the input field rebase focus text immediately so cleared/toolbar text does not trigger post-send suggestions. **Since 2026-10-29**, tracks `_typedInCurrentFocus` and `_lastTypedHwnd` and forces immediate foreground sampling on the first keystroke after a mouse click (`_rebaseFocusText`) or window switch, so typing `pc` (`ㅔㅊ`) immediately after clicking into a search box or new field sets `editedSinceFocus = true` on the very first snapshot. It extends the telemetry store with Flow Radar (hourly activity density + chars + edit stability; a potential-focus score, **not** a real session), Edit Friction (chars/backspaces; edits per 100 chars), App Quality (per-app suggestion total/accepted/accept rate/average latency), Privacy Receipt (actual row counts and retention), and Smart Exclusion evidence. Raw key events, key codes and content streams are never retained; separately, opt-in learned text may be retained in `typing_samples` and `personal_phrases` under the stated receipt policy. |
|
||||
| `TypoCorrectionService` | Built-in automatic Han/Eng layout conversion and typo auto-correction (`INPUT-19`). Runs on each `typed-context`: Araso-style layout conversions (`source: 'layout'`) fire after 25 ms on word boundaries (Space/punctuation, 40 ms quick snapshot) or 135 ms at token tail (150 ms quick snapshot, even when Chromium/Discord keeps `isComposing=true`, committing IME composition first only when the trailing character is Hangul/jamo), replacing 2-set Han/Eng layout errors, mixed English+Korean words (`githubdp`→`github에`, `PDFvkdlf`→`PDF파일`, `ㅖㅇㄹ파일`→`PDF파일`), lingering Shift on double consonants (`djEJs`→`어떤`), reverse Hangul-to-English (`ㅎ샤ㅗㅕㅠ에`→`github에`, `ㅑㄹ ㅑ 쇼ㅔㄷ ㅗ디ㅣㅐ 소두 ㅑㅅ 초뭏ㄷㄴ ㅑㅜ 10 ㄴㄷ채ㅜㅇㄴ`→`if I type hello then it changes in 10 seconds`), lookahead English clauses (`새 ㅠㄷ `→`to be `, `새 애 this `→`to do this `), and partially-converted tokens (`안녕gktpdy`→`안녕하세요`). **In terminals and Vim (`terminalMode: true`)**, layout-only conversion stays active while spelling/model checks are suppressed: converts CLI commands (`퍄ㅡ `→`vim `, `ㅜ퍄ㅡ `→`nvim `, `ㅎ샤 `→`git `, `ㅣㄴ `→`ls `, `채ㅜㄴㅅ `→`const `), Vim Insert-mode typing (stripping leading `i/a/o/I/A/O/s/S` mode-entry keys when present in raw keystroke buffers), and Vim Command-line ex-commands (`:ㅈㅂ`→`:wq`, `:ㅈㅂ!`→`:wq!`, `:ㅈ`→`:w`, `:ㅂ`→`:q`, `:ㅂ!`→`:q!`, `:ㅌ`→`:x`, `:ㅍ네`→`:vsp`, `:네`→`:sp`, `:ㅜㅐㅗ`→`:noh`, preserving `:`), while blocking all conversions in Vim Normal mode (`vimMode === 'normal'`). **Since 2026-10-28**, on Windows, `TextInsertService.replaceBeforeCaret` (`method === 'keyboard'`) executes an atomic single-syscall `sendWin32UnicodeReplacement` (`utils/win32-unicode-input.ts`) packing `(optional VK_SPACE) + N × VK_BACK + M × KEYEVENTF_UNICODE` into one `SendInput` call (zero JS sleep, zero clipboard access), and `TypoCorrectionService` delta-rebases replacement plans (`rebaseTypoReplacementPlan`) when the user appends characters while an async snapshot or LLM check is in flight. Both Stage 1 (deterministic layout/spelling) and Stage 2 (tail-anchored `trailingWordOnly: true` Gemma check) run seamlessly even when `SuggestionService` is enabled. |
|
||||
| `TypoCorrectionService` | Built-in automatic Han/Eng layout conversion and typo auto-correction (`INPUT-19`). Runs on each `typed-context`: Araso-style layout conversions (`source: 'layout'`) fire after 25 ms on word boundaries (Space/punctuation, 40 ms quick snapshot) or 135 ms at token tail (150 ms quick snapshot, even when Chromium/Discord keeps `isComposing=true`, committing IME composition first only when the trailing character is Hangul/jamo), replacing 2-set Han/Eng layout errors, mixed English+Korean words (`githubdp`→`github에`, `PDFvkdlf`→`PDF파일`, `ㅖㅇㄹ파일`→`PDF파일`), lingering Shift on double consonants (`djEJs`→`어떤`), reverse Hangul-to-English (`ㅎ샤ㅗㅕㅠ에`→`github에`, `ㅑㄹ ㅑ 쇼ㅔㄷ ㅗ디ㅣㅐ 소두 ㅑㅅ 초뭏ㄷㄴ ㅑㅜ 10 ㄴㄷ채ㅜㅇㄴ`→`if I type hello then it changes in 10 seconds`), lookahead English clauses (`새 ㅠㄷ `→`to be `, `새 애 this `→`to do this `), and partially-converted tokens (`안녕gktpdy`→`안녕하세요`). **In terminals and Vim (`terminalMode: true`)**, layout-only conversion stays active while spelling/model checks are suppressed: converts CLI commands (`퍄ㅡ `→`vim `, `ㅜ퍄ㅡ `→`nvim `, `ㅎ샤 `→`git `, `ㅣㄴ `→`ls `, `채ㅜㄴㅅ `→`const `), Vim Insert-mode typing (stripping leading `i/a/o/I/A/O/s/S` mode-entry keys when present in raw keystroke buffers), and Vim Command-line ex-commands (`:ㅈㅂ`→`:wq`, `:ㅈㅂ!`→`:wq!`, `:ㅈ`→`:w`, `:ㅂ`→`:q`, `:ㅂ!`→`:q!`, `:ㅌ`→`:x`, `:ㅍ네`→`:vsp`, `:네`→`:sp`, `:ㅜㅐㅗ`→`:noh`, preserving `:`), while blocking all conversions in Vim Normal mode (`vimMode === 'normal'`). **Since 2026-10-28**, on Windows, `TextInsertService.replaceBeforeCaret` (`method === 'keyboard'`) executes an atomic single-syscall `sendWin32UnicodeReplacement` (`utils/win32-unicode-input.ts`) packing `(optional VK_SPACE) + N × VK_BACK + M × KEYEVENTF_UNICODE` into one `SendInput` call (zero JS sleep, zero clipboard access), and `TypoCorrectionService` delta-rebases replacement plans (`rebaseTypoReplacementPlan`) when the user appends characters while an async snapshot or LLM check is in flight. **Since 2026-10-31 (`v1.15.2`)**, `propose()` checks `!this._isRegionChecked(region)` before calling `_askModel`, avoiding duplicate 336-token LLM checks when `handleTypingContext` already verified the same region. Both Stage 1 (deterministic layout/spelling) and Stage 2 (tail-anchored `trailingWordOnly: true` Gemma check) run seamlessly even when `SuggestionService` is enabled. |
|
||||
| `UiaContextService` | Client for the sidecar UIA bridge (`GET /uia/focus`): focused text, caret rect/offset, `IsPassword`, IME composition state and `hasSelection`. The sidecar derives `hasSelection` only by comparing the TextPattern selection range Start/End endpoints: it does not call `GetText` on that range or add a separate selected-text payload (the existing focused-field text can still include a selection). Fail-closed on password/unavailable; transient failures back off for 1.5/3/6/12/30 s and permanent failures for 5 min. |
|
||||
| `SuggestionService` | Next-sentence ghost text: 600 ms debounce, then `buildSuggestionPrompt` → `LocalLLMService.streamGenerate` with a caller-owned `AbortSignal` → candidate ranking and accept (via `TextInsertService`). Presentation-active includes candidates plus `generating`, `warmingUp` and `partialText`; clear/dismiss aborts the active request, invalidates its generation token, clears TTL state, then emits `cleared`/hide so a spinner-only popup closes. A successful final candidate payload resets `generating=false` and `partialText=null`. App DNA extends phrase ranking with `appName` context and a 1.75× same-app bonus; Memory Decay uses a 30-day half-life plus frequency. Why This Suggestion exposes only `local-model`/`local-memory` provenance and continuation/related/phrase/appPhrase **counts** to the overlay, never raw evidence text. Instant Recall uses local-memory provenance without another model-budget spend only when the model is unavailable, raises a non-cancellation error, truly times out, or returns empty; it never publishes after dismiss, new typing, token/context mismatch or staleness. Its runaway limits are a 5 s minimum interval, 6 requests/min default (hard-config maximum 12), 3 candidates, 64 output tokens, 12-character regeneration growth and an 8 s timeout; app startup does not warm the model and suggestion requests use `keep_alive: 2m`. |
|
||||
| `global-input-hook.ts` | Ref-counted owner of the single process-wide `uiohook` hook so `KeyBindingService` and telemetry can both attach without one stopping the other. |
|
||||
|
|
|
|||
|
|
@ -170,7 +170,7 @@ end-to-end behaviour has **not been verified by typing in a real app** (`11` GAP
|
|||
| INPUT-03 | Weekly input insights | [~] | [-] | [-] | [-] | `INPUT_TELEMETRY.getSummary` aggregates `input_activity` into totals, daily series, top hours and top apps; rendered in Settings → Input and as a dashboard card. Daily average mouse travel is converted px → m using the display scale factor. |
|
||||
| INPUT-04 | Typed-text learning (UIA, password-excluded) | [~] | [-] | [-] | [-] | Text is read from the focused field via `GET /uia/focus` (sidecar UIA bridge) and diffed longest-common-prefix/suffix, so **IME-committed Hangul/kana is counted correctly** — keycodes cannot reconstruct CJK text. UIA sends `hasSelection` only, derived by TextPattern range Start/End comparison without calling `GetText` on the selection range or adding a selected-text payload; the existing focused-field text can still include a selection. A non-collapsed selection immediately clears suggestions as `selection-active`. `IsPassword` is checked before any read (fail-closed); IME composition suppresses both stats and suggestions. **2026-09-24:** text equal to the control name (empty-field placeholder, e.g. "메시지 입력") is treated as empty (`withoutPlaceholderText`); a suggestion requires `editedSinceFocus` + typing within `recentTypingWindowMs` (8 s), otherwise `not-typing` — clicking into a pre-filled field no longer triggers suggestions. **2026-10-29:** `_typedInCurrentFocus` and `_lastTypedHwnd` track whether a typing keystroke occurred after the latest click/focus rebase (`_rebaseFocusText`), and `_sampleForeground` forces immediate foreground sampling on that first keystroke, so typing immediately after clicking into a search box or new window sets `editedSinceFocus = true` on the very first snapshot. **2026-10-24:** typed deltas are no longer saved per snapshot (that stored mid-word fragments every 0.4–0.8 s). They are buffered per field and saved as one baseline→latest diff on 2 s idle, sentence-ending punctuation, 150-char growth, field clear (send), focus change or stop. Typed samples under 4 non-space chars, URL/query-like lines, trailing incomplete jamo and PUA/lone-surrogate chars (emoji truncated to 16 bits upstream, root cause unconfirmed) are dropped. Rows already in the DB are pruned on start by the same rule (`isLearnableTypedText` in `_pruneUnlearnableCorpus`, typed source only; dry count on the author machine 2026-10-24: 1,235/2,664 samples, 285/1,209 phrases). Expressive jamo runs (ㅋㅋ, ㅠㅠ, ㅇㅇ) and space-less questions are kept. Mid-word fragments of 4+ chars cannot be detected and remain until the 30-day retention. **2026-10-30..31: UIA reads are single-flight and discarded after input/focus/lifecycle changes, stale capture times, or changed native GetLastInputInfo tokens; contexts carry inputRevision/nativeInputTick for replacement-time validation (`InputTelemetryService`, `win32-input-activity`). Standalone modifier keydowns/key-repeats (`Ctrl`/`Alt`/`Shift`/`Win`) and `Ctrl+Alt` suggestion navigation chords are ignored in `_handleKeyDown` without advancing `_inputRevision`/`_lastKeyAt` or scheduling UIA snapshots (`classifyKeyStroke` checks modifier VKs before modifier flags), and `_settleTimer` (780 ms) is skipped on non-typing keys and auto-cancelled when the 380 ms `_textTimer` snapshot cannot trigger a suggestion (`prefixLen < minPrefixChars`). See GAP-INPUT-11.** |
|
||||
| INPUT-05 | Personal phrase corpus (typed + voice) | [~] | [-] | [-] | [-] | Sentence-level phrases from typed text and from voice history (`HistoryService.create` feeds `recordExternalText`), ranked by frequency/recency as prompt hints; users can delete individual phrases. **2026-09-24:** typed text from `LEARNING_EXCLUDED_APPS` (terminals, code editors, Agent Switchboard) is never learned, and every phrase must pass `isLearnablePhrase` (no box/block/geometric glyphs, ≥60 % letters). `_pruneUnlearnableCorpus` re-applies both rules to existing phrases/samples/edges on start and every retention cycle (first run on the author machine removed 69 phrases / 91 samples of terminal status lines and agent chats). **2026-10-25:** Agent Switchboard moved from `LEARNING_EXCLUDED_APPS` to `APP_SCOPED_LEARNING_APPS` — its text is kept only as `typing_samples` (with app name), never in shared phrases/graph, and is used only for suggestions inside that same app (`findContinuationHints(..., { appName })` feeds `continuationHints`). It is the user's most-typed app; the 2026-09-24 exclusion was about dev instructions leaking into KakaoTalk suggestions, which scoping prevents. |
|
||||
| INPUT-06 | Next-sentence suggestion (ghost text) | [~] | [-] | [-] | [-] | `SuggestionService` + `buildSuggestionPrompt` (`llm-prompts.ts` SSOT, instruction stays in the system prompt). The 2026-09-22 guard contract is 600 ms debounce, ≥5 s between requests, 6 requests/min by default (hard-config maximum 12), 3 candidates, 64 output tokens, 12-character growth before regeneration, 8 s request timeout and `keep_alive: 2m`; boot warmup is removed. Each request has its own cancellation signal. Presentation-active includes candidates, `generating`, `warmingUp` and `partialText`; clear/dismiss aborts, invalidates the generation token, clears TTL state and emits `cleared`/hide, and a final success resets `generating=false`/`partialText=null`. Focused evidence for the lifecycle and Windows-child-process change: five test files / 80 tests passed; desktop typecheck/lint, Python `py_compile`, and `git diff --check` exited 0 (core 131-test pass was independently verified earlier). Status remains `[~]`: this is not app-restart, GUI overlay, or real automatic-typing evidence. **2026-09-24 (supersedes the numbers above):** `keep_alive: 10m`, warm-up on boot, on Ollama reconnect and whenever the model is presumed cold (cold reload measured 11.4 s > 8 s timeout); a session asks for 1 candidate, then a sequential fill loop appends one unique candidate at a time up to `maxCandidatesTotal` 12 (avoid-list in the prompt, stops after 2 empty fills); only the first request consumes rate/daily budget; stale aborts refund the budget; any further typing ends the session (`matchesSessionPrefix`, IME last-syllable tolerant); prompt forbids assistant-style questions. Terminals (`TERMINAL_APPS`) never get suggestions. **2026-10-24 accept fix:** `waitForModifiersReleased` now trusts OS `GetAsyncKeyState` (`utils/win32-key-state.ts`) over the uiohook-tracked set. A missed keyup made acceptance wait the full 1.5 s and then paste Ctrl+V while modifiers were still considered held; in the logged case the field length stayed 15 after "inserted" (6 chars). New suggestions are also suppressed while an accept is in flight, so the overlay no longer reappears during the wait. **2026-10-24~25 experiment, reverted:** an n-gram-first / confidence-gated / short-candidate rework raised offline next-word precision (3%→33%) but made suggestions rare, and the user judged the app much worse. Suggestion behaviour is back to v1.10.1. Kept: suggestions that copy on-screen text are rejected (`copiesContext`), no suggestions while typing an address/path (`isAddressInput`, skip reason `address-input`). Lessons and data: `docs/evidence/2026-10-24-suggestion-accuracy-research.md` §9; replay harness `tests/eval/`. **2026-10-25 additive upgrades (frequency unchanged):** (1) `suggestion-accept-word` (default Ctrl+Alt+Space) inserts only the first word and keeps the rest on screen (`splitFirstWord`; partial accepts are not learned as full sentences). (2) a confident personal n-gram candidate (`PersonalNgramIndex`, strict `PERSONAL_NGRAM_SETTINGS`) is shown instantly as candidate #1 while the model generates; model candidates are appended after it, and it survives an empty/failed/timed-out model run (no "no suggestions" notice then). Agent Switchboard text never enters this shared n-gram. Hotkey accept now force-releases held Ctrl/Alt/Shift after 200 ms (`prepareModifiersForPaste`). **2026-10-28 extreme personalization & n-gram upgrades (frequency unchanged):** `PersonalNgramModel` now weights repeated `personalPhrases` (`count >= 2`) and falls back to a Korean stemmed-word bigram index (`_stemBigram` via `stemTerm`, e.g. matching `"배포를 완료했습니다"` when typing `"배포가 "`), and `SuggestionService._fillOne` reranks newly generated LLM candidates by `scoreCandidatePersonalization` (overlap with personal graph `continuations`/`followUps`/`related`/`phrases` + register match) while preserving the instant n-gram slot at index 0. **2026-10-31 (`v1.15.2`) terminal accept & fill-stop fix:** when accepting a suggestion in `terminalMode === true` (`xterm-helper-textarea`, `WindowsTerminal`, `Kitedesk`, etc.), `SuggestionService._insertAccepted` uses `'keyboard'` (`sendWin32UnicodeText` / `sendWin32UnicodeReplacement` via `KEYEVENTF_UNICODE`) instead of `'clipboard'` (`Ctrl+V`) so stale OS clipboard contents (e.g., copied URLs) can never be pasted into the terminal on delayed paste processing, syncs `TerminalKeystrokeTracker` via `applyProgrammaticReplacement`, and stops `_fillRemainingCandidates` immediately when `accept()` is triggered (`suggestion-typo-correction.test.ts` 8/8 PASS). |
|
||||
| INPUT-06 | Next-sentence suggestion (ghost text) | [~] | [-] | [-] | [-] | `SuggestionService` + `buildSuggestionPrompt` (`llm-prompts.ts` SSOT, instruction stays in the system prompt). The 2026-09-22 guard contract is 600 ms debounce, ≥5 s between requests, 6 requests/min by default (hard-config maximum 12), 3 candidates, 64 output tokens, 12-character growth before regeneration, 8 s request timeout and `keep_alive: 2m`; boot warmup is removed. Each request has its own cancellation signal. Presentation-active includes candidates, `generating`, `warmingUp` and `partialText`; clear/dismiss aborts, invalidates the generation token, clears TTL state and emits `cleared`/hide, and a final success resets `generating=false`/`partialText=null`. Focused evidence for the lifecycle and Windows-child-process change: five test files / 80 tests passed; desktop typecheck/lint, Python `py_compile`, and `git diff --check` exited 0 (core 131-test pass was independently verified earlier). Status remains `[~]`: this is not app-restart, GUI overlay, or real automatic-typing evidence. **2026-09-24 (supersedes the numbers above):** `keep_alive: 10m`, warm-up on boot, on Ollama reconnect and whenever the model is presumed cold (cold reload measured 11.4 s > 8 s timeout); a session asks for 1 candidate, then a sequential fill loop appends one unique candidate at a time up to `maxCandidatesTotal` 12 (avoid-list in the prompt, stops after 2 empty fills); only the first request consumes rate/daily budget; stale aborts refund the budget; any further typing ends the session (`matchesSessionPrefix`, IME last-syllable tolerant); prompt forbids assistant-style questions. Terminals (`TERMINAL_APPS`) never get suggestions. **2026-10-24 accept fix:** `waitForModifiersReleased` now trusts OS `GetAsyncKeyState` (`utils/win32-key-state.ts`) over the uiohook-tracked set. A missed keyup made acceptance wait the full 1.5 s and then paste Ctrl+V while modifiers were still considered held; in the logged case the field length stayed 15 after "inserted" (6 chars). New suggestions are also suppressed while an accept is in flight, so the overlay no longer reappears during the wait. **2026-10-24~25 experiment, reverted:** an n-gram-first / confidence-gated / short-candidate rework raised offline next-word precision (3%→33%) but made suggestions rare, and the user judged the app much worse. Suggestion behaviour is back to v1.10.1. Kept: suggestions that copy on-screen text are rejected (`copiesContext`), no suggestions while typing an address/path (`isAddressInput`, skip reason `address-input`). Lessons and data: `docs/evidence/2026-10-24-suggestion-accuracy-research.md` §9; replay harness `tests/eval/`. **2026-10-25 additive upgrades (frequency unchanged):** (1) `suggestion-accept-word` (default Ctrl+Alt+Space) inserts only the first word and keeps the rest on screen (`splitFirstWord`; partial accepts are not learned as full sentences). (2) a confident personal n-gram candidate (`PersonalNgramIndex`, strict `PERSONAL_NGRAM_SETTINGS`) is shown instantly as candidate #1 while the model generates; model candidates are appended after it, and it survives an empty/failed/timed-out model run (no "no suggestions" notice then). Agent Switchboard text never enters this shared n-gram. Hotkey accept now force-releases held Ctrl/Alt/Shift after 200 ms (`prepareModifiersForPaste`). **2026-10-28 extreme personalization & n-gram upgrades (frequency unchanged):** `PersonalNgramModel` now weights repeated `personalPhrases` (`count >= 2`) and falls back to a Korean stemmed-word bigram index (`_stemBigram` via `stemTerm`, e.g. matching `"배포를 완료했습니다"` when typing `"배포가 "`), and `SuggestionService._fillOne` reranks newly generated LLM candidates by `scoreCandidatePersonalization` (overlap with personal graph `continuations`/`followUps`/`related`/`phrases` + register match) while preserving the instant n-gram slot at index 0. **2026-10-31 (`v1.15.2`) `Ctrl+Alt+Enter` reliability & terminal accept fix:** (1) `ChordStateMachine.keyUp` releases all held bindings sharing a released modifier (`Ctrl`/`RightAlt`) so `suggestion-accept` never stays stuck in `_isDown` after `command` (`Ctrl+RightAlt`), (2) `KeyBindingService` maps `UiohookKey.NumpadEnter` and adds a 25 ms `globalShortcut` fallback trigger while `isOverlayOrCaptureActive` is true, (3) `SuggestionService.accept()` waits up to 2.5 s (`_waitForFirstCandidate`) when invoked while candidate #1 is still generating/pending, and (4) in `terminalMode` / `isTerminalApp`, `_insertAccepted` uses `'keyboard'` (`KEYEVENTF_UNICODE`) instead of `'clipboard'` (`Ctrl+V`), syncs `TerminalKeystrokeTracker` via `applyProgrammaticReplacement`, and stops `_fillRemainingCandidates` immediately (`suggestion-accept-hold.test.ts` 4/4, `KeyBindingService.test.ts` 12/12, `keybinding-runtime.test.ts` 29/29 PASS). |
|
||||
| INPUT-07 | Caret-anchored suggestion overlay | [~] | [-] | [-] | [-] | `suggestion-overlay` popup placed by `anchorFloatingPanel` (caret → element → cursor fallback, flip above when the caret is near the bottom, clamped to the work area). Non-focusable; click-through unless `suggestionOverlayInteractive`. While actually visible it continues periodic UIA validation after 5 s and revalidates 120 ms after mouse-up; unavailable/non-editable focus or a non-collapsed selection aborts and hides it. X first hides the renderer panel, then main IPC directly hides `BrowserWindow` and dismisses the service, so late tokened results cannot revive it. Accept/next/prev/dismiss are four global key bindings (`suggestion-accept`/`next`/`prev`/`dismiss`, default `Ctrl+Alt+→/↓/↑/←`), and the overlay has a mouse close button. Up to three candidates are shown in a scrollable list with a warm-up/generating spinner. **2026-09-24:** one page of 3 fixed-height (2-line) numbered items, footer = range/progress + source + key guide derived from the live bindings (`buildSuggestionKeyHints`, shared modifier shown once). Keys: Ctrl+Alt+↑/↓ move (the page follows the active item; at the last item `next` waits while filling, otherwise wraps; the 1.6.0 Left/Right page actions were removed as unreachable before a page filled and colliding with Intel display rotation), Ctrl+Alt+Enter accept, plain Esc closes while visible (overlay-scoped, not a binding), Ctrl+Alt+Backspace secondary dismiss; old default bindings migrate at tuning revision 5. Streaming updates never reposition/re-present the window (`decideSuggestionOverlayAction`); placement uses `anchorKind` — caret: below the line, element: outside the element (below → above → right → left → inner bottom-right). Navigation re-arms the visible TTL. Window 460×208. |
|
||||
| INPUT-08 | Per-app exclusions & consent controls | [~] | [-] | [-] | [-] | `inputExcludedApps` (executable names, case-insensitive) blocks both collection context and suggestions; telemetry master switch, pause, text-learning toggle and "delete collected data" all live in Settings → Input. 30-day retention prune runs on start. |
|
||||
| INPUT-09 | Flow Radar | [~] | [-] | [-] | [-] | `rankFlowWindows` ranks hourly aggregate activity density, character volume and edit stability into potential-focus time windows. It is not a real-session detector or session record. |
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
Loading…
Add table
Add a link
Reference in a new issue