release: ship v1.5.0 with on-device writing suggestions
Adds next-sentence suggestions while typing, weekly input insights and a personal phrase memory to the desktop app, and fixes custom instructions so they process the text instead of inserting the instruction's own wording. Local model requests are now bounded and individually cancellable. Bumps the product version to 1.5.0 (Android/iOS build 1050000), refreshes the landing and web download links, and records the new INPUT feature rows and the open verification gaps in the infrastructure map.
This commit is contained in:
parent
99f06c253c
commit
5c11ee2fde
104 changed files with 14410 additions and 174 deletions
|
|
@ -37,7 +37,7 @@ Status quick-reference: `[x]` done+verified · `[~]` partial/unverified · `[ ]`
|
|||
|
||||
| ID | Feature | D | W | M | B | Anchors / notes |
|
||||
|---|---|---|---|---|---|---|
|
||||
| AI-01 | Local LLM (Ollama) | [x] | [-] | [ ] | [-] | Desktop bundled Ollama |
|
||||
| AI-01 | Local LLM (Ollama) | [~] | [-] | [ ] | [-] | Desktop bundled Ollama. 2026-09-22 guard contract: no boot warmup; suggestion `keep_alive: 2m`; request-owned cancellation; generate/stream 2048 tokens / 120 s and chat 512 / 60 s; `done` frame required with incomplete-stream cleanup; deduplicated Ollama spawn/poll plus lifecycle disposal. GAP-LLM-03 has targeted evidence (6 test files / 69 passed / 0 failed; changed code/tests ESLint and diff check exit 0). Raw Ollama confirms cold timeout cleanup, a 553 ms `num_predict=1`/`keep_alive='2m'` response, and after the about-119.9 s expiry a 19:48:59 +09:00 `/api/ps` HTTP 200 / 45.8 ms `{models:[]}` observation with no intervening generate/unload/kill/retry; `/api/version` was HTTP 200 / 7.3 ms / `0.32.13`. This is raw API expiry/unload evidence, not app-restart or GUI/runtime proof (GAP-LLM-04). |
|
||||
| AI-02 | Cloud LLM (Claude/OpenAI) | [x] | [x] | [x] | [x] | Desktop `PremiumLLMService`; web/mobile via `llm-proxy`; .NET `LlmProxyService` |
|
||||
| AI-03 | Auto Polish (cleanup/filler removal) | [x] | [~] | [~] | [x] | Desktop built-in; web/mobile via commands. Desktop Auto Polish is the plain `refine` action (`llm-prompts.ts:14`), not a custom instruction, so it was **not** affected by the 2026-09-21 instruction-prompt fix (AI-05); regression cases now pin `refine`/`summarize`/`grammar`/`expand` (`VoiceModeService.test.ts:435`, `llm-prompts.test.ts:153`) |
|
||||
| AI-04 | Translate / summarize / rephrase | [~] | [x] | [x] | [x] | Built-in instructions. **Desktop has two paths and only one of them worked.** The plain-action path (Settings → `defaultLLMAction`, `SettingsModal.tsx:653`) reads `BASE_SYSTEM_PROMPTS` directly and was always correct. The built-in *instruction* presets (`CustomInstructionService.ts:26/35/44/53/62`) ran through the custom-instruction path and inserted the instruction's own wording instead of the result — see AI-05. Fixed in `9c2b4d4` (2026-09-21), **not verified in a running app** (`11` GAP-LLM-02). Translate still always targets English: `AppConfig` has no target-language key and neither `language` (UI locale) nor `sttLanguage` (source language) can stand in (`llm-prompts.ts:37-52`, `11` GAP-LLM-01) |
|
||||
|
|
@ -155,6 +155,36 @@ Status quick-reference: `[x]` done+verified · `[~]` partial/unverified · `[ ]`
|
|||
|
||||
---
|
||||
|
||||
## INPUT — Input Intelligence (typing telemetry & next-sentence suggestions)
|
||||
|
||||
Everything here is **desktop-only** (Windows today) and **opt-in**: `inputTelemetryEnabled` and
|
||||
`inputLearnTypedText` default to `false`. D=desktop, W=web, M=mobile, B=backend.
|
||||
Status is `[~]` on desktop for one shared reason: the code and its 48 unit cases exist, but the
|
||||
end-to-end behaviour has **not been verified by typing in a real app** (`11` GAP-INPUT-01).
|
||||
|
||||
| ID | Feature | D | W | M | B | Anchors / notes |
|
||||
|---|---|---|---|---|---|---|
|
||||
| INPUT-01 | Keyboard/mouse telemetry capture (opt-in) | [~] | [-] | [-] | [-] | `InputTelemetryService` — keystroke/click/scroll counters, mouse travel as the Manhattan sum of per-axis pixel deltas, active time, per-hour×app buckets flushed every 5 s. Key **contents** are never stored (ActivityWatch `aw-watcher-input` data-minimisation policy, adopted deliberately — see `11` §7). Hook ownership is ref-counted so `KeyBindingService` keeps working (`global-input-hook.ts`). |
|
||||
| INPUT-02 | Foreground-app attribution | [~] | [-] | [-] | [-] | `utils/win32-foreground.ts` via `koffi` FFI (title/pid/exe/bounds), sampled at most 1×/s. `get-windows` was rejected: it needs an install script this repo does not run. |
|
||||
| INPUT-03 | Weekly input insights | [~] | [-] | [-] | [-] | `INPUT_TELEMETRY.getSummary` aggregates `input_activity` into totals, daily series, top hours and top apps; rendered in Settings → Input and as a dashboard card. Daily average mouse travel is converted px → m using the display scale factor. |
|
||||
| INPUT-04 | Typed-text learning (UIA, password-excluded) | [~] | [-] | [-] | [-] | Text is read from the focused field via `GET /uia/focus` (sidecar UIA bridge) and diffed longest-common-prefix/suffix, so **IME-committed Hangul/kana is counted correctly** — keycodes cannot reconstruct CJK text. UIA sends `hasSelection` only, derived by TextPattern range Start/End comparison without calling `GetText` on the selection range or adding a selected-text payload; the existing focused-field text can still include a selection. A non-collapsed selection immediately clears suggestions as `selection-active`. `IsPassword` is checked before any read (fail-closed); IME composition suppresses both stats and suggestions. |
|
||||
| INPUT-05 | Personal phrase corpus (typed + voice) | [~] | [-] | [-] | [-] | Sentence-level phrases from typed text and from voice history (`HistoryService.create` feeds `recordExternalText`), ranked by frequency/recency as prompt hints; users can delete individual phrases. |
|
||||
| INPUT-06 | Next-sentence suggestion (ghost text) | [~] | [-] | [-] | [-] | `SuggestionService` + `buildSuggestionPrompt` (`llm-prompts.ts` SSOT, instruction stays in the system prompt). The 2026-09-22 guard contract is 600 ms debounce, ≥5 s between requests, 6 requests/min by default (hard-config maximum 12), 3 candidates, 64 output tokens, 12-character growth before regeneration, 8 s request timeout and `keep_alive: 2m`; boot warmup is removed. Each request has its own cancellation signal. Presentation-active includes candidates, `generating`, `warmingUp` and `partialText`; clear/dismiss aborts, invalidates the generation token, clears TTL state and emits `cleared`/hide, and a final success resets `generating=false`/`partialText=null`. Focused evidence for the lifecycle and Windows-child-process change: five test files / 80 tests passed; desktop typecheck/lint, Python `py_compile`, and `git diff --check` exited 0 (core 131-test pass was independently verified earlier). Status remains `[~]`: this is not app-restart, GUI overlay, or real automatic-typing evidence. |
|
||||
| INPUT-07 | Caret-anchored suggestion overlay | [~] | [-] | [-] | [-] | `suggestion-overlay` popup placed by `anchorFloatingPanel` (caret → element → cursor fallback, flip above when the caret is near the bottom, clamped to the work area). Non-focusable; click-through unless `suggestionOverlayInteractive`. While actually visible it continues periodic UIA validation after 5 s and revalidates 120 ms after mouse-up; unavailable/non-editable focus or a non-collapsed selection aborts and hides it. X first hides the renderer panel, then main IPC directly hides `BrowserWindow` and dismisses the service, so late tokened results cannot revive it. Accept/next/prev/dismiss are four global key bindings (`suggestion-accept`/`next`/`prev`/`dismiss`, default `Ctrl+Alt+→/↓/↑/←`), and the overlay has a mouse close button. Up to three candidates are shown in a scrollable list with a warm-up/generating spinner. |
|
||||
| INPUT-08 | Per-app exclusions & consent controls | [~] | [-] | [-] | [-] | `inputExcludedApps` (executable names, case-insensitive) blocks both collection context and suggestions; telemetry master switch, pause, text-learning toggle and "delete collected data" all live in Settings → Input. 30-day retention prune runs on start. |
|
||||
| INPUT-09 | Flow Radar | [~] | [-] | [-] | [-] | `rankFlowWindows` ranks hourly aggregate activity density, character volume and edit stability into potential-focus time windows. It is not a real-session detector or session record. |
|
||||
| INPUT-10 | Edit Friction | [~] | [-] | [-] | [-] | `calculateFrictionInsight` derives friction from char/backspace quantities and reports edits per 100 chars; it does not infer sentiment or productivity. |
|
||||
| INPUT-11 | App DNA | [~] | [-] | [-] | [-] | Personal phrase ranking passes `appName` context and applies a 1.75× same-app ranking bonus (`selectPhraseHints`); learned text remains opt-in. |
|
||||
| INPUT-12 | App Quality | [~] | [-] | [-] | [-] | `InputTelemetryService.getSummary` aggregates per-app suggestion total, accepted count, accept rate and average latency from local suggestion history. |
|
||||
| INPUT-13 | Privacy Receipt | [~] | [-] | [-] | [-] | Local-only receipt shows actual row counts. Raw individual key events/key codes/content stream are not retained; opt-in learned text can reside in `typing_samples`/`personal_phrases`. `input_activity`, `typing_samples` and `suggestions` retain 30 days; `personal_phrases` has no age-based automatic expiry and is removed by individual deletion, delete-all, or consent withdrawal. Receipt read/delete storage failures are IPC errors. |
|
||||
| INPUT-14 | Smart Exclusion | [~] | [-] | [-] | [-] | Password fields add no evidence. Only the current app may receive a one-click recommendation after ≥4 observations, readable=0 and problematic ratio ≥75%; it never auto-excludes. |
|
||||
| INPUT-15 | Why This Suggestion | [~] | [-] | [-] | [-] | Overlay exposes only local-model/local-memory provenance and continuation/related/phrase/appPhrase counts; raw evidence text is not shown. |
|
||||
| INPUT-16 | Memory Decay | [~] | [-] | [-] | [-] | Phrase ranking combines a 30-day half-life, frequency and the App DNA 1.75× same-app bonus. |
|
||||
| INPUT-17 | Instant Recall | [~] | [-] | [-] | [-] | Local-memory fallback (`provenance=local-memory`) occurs only for unavailable model, non-cancellation error, actual timeout or empty model result; it spends no additional model budget and is blocked on dismiss, new typing, token/context mismatch or staleness. |
|
||||
| INPUT-18 | Shortcut Safety Audit | [~] | [-] | [-] | [-] | Settings UI applies core `auditKeyBindingMap` to surface invalid/conflict issues while hold/double-press exceptions remain unchanged. |
|
||||
|
||||
---
|
||||
|
||||
## SHELL — Platform Shell, Settings, Onboarding, Support
|
||||
|
||||
| ID | Feature | D | W | M | B | Anchors / notes |
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue