d3ro-voice/docs/map/10-feature-catalog.md
Yun Chan 3aa68f466c fix(meeting): keep polish from inventing markers or dropping qualifiers
The local model follows the polish instructions only most of the time: in
repeated runs it added a [00:00] or [화자 1] marker that was never in the
audio, and it turned "사실 거의 근간" into "사실 근간". Both change what the
meeting record says, so they are now enforced in code instead of hoped for
in the prompt: markers absent from the original chunk are stripped, and
meaning qualifiers the speaker said are restored from that chunk.

The instructions themselves were re-measured on the model: a fixed removal
list, an explicit qualifier example and a marker ban keep the filler cleanup
without touching ordinary words, and the same wording is used by the app,
the tests and the probe. Unknown-word repair still depends on the model.
2026-11-06 23:24:30 +09:00

88 KiB
Raw Blame History

10 — Feature Catalog (Feature Map)

The canonical feature map. One row = one user-facing capability. Status is per surface. Legend in 00-index.md §2. Surfaces: D = desktop (apps/desktop), W = web (apps/web), M = mobile (apps/mobile-rn), B = backend (Supabase/.NET).

How to use an ID: cite it in commits, plans, and backlog. Example: "CAP-03 is [~] on mobile (external OAuth pending)".

Status quick-reference: [x] done+verified · [~] partial/unverified · [ ] planned/absent · [!] blocked externally · [-] N/A.


CAP — Capture & Transcribe

ID Feature D W M B Anchors / notes
CAP-01 Push-to-talk dictation (hold/release) [x] [-] [x] [-] Desktop VoiceModeService; the trigger is now the rebindable dictation action of CAP-16 (several bindings per action, keyboard or mouse) rather than a single stored shortcut. The pipeline itself is unchanged and tested; the rewritten entry layer was confirmed in the 2026-09-21 manual run (CAP-16). Mobile RecordScreen via app CTA/notification action (no global hotkey)
CAP-02 Hands-free toggle dictation [x] [-] [x] [-] Desktop double-press shares the dictation binding and is split by the action's doublePress flag (KeyBindingService.ts:680). Fixed and confirmed in the 2026-09-21 manual run (CAP-16); unit-tested in KeyBindingService.test.ts (11 cases, 2026-10-30, GAP-KEY-01 [x]) and keybinding.test.ts. Mobile toggle
CAP-03 Live partial transcript while recording [x] [ ] [ ] [-] Desktop voice:partialTranscript + recording-tip; producer added in 1.3.0 (VoiceModeService._runPartial → LocalSTTService.transcribePartial, 1.5 s cadence / 7.5 s window, never inserted). The row was [x] before any producer existed.
CAP-04 Recording waveform + level meter [x] [x] [x] [-] Desktop 9-bar cos distribution; mobile audio level; the recording-tip popup bundle and its on-disk assets are verified by scripts/ci/verify-desktop-renderer-bundles.mjs
CAP-05 Device/mic selection [x] [ ] [~] [-] Desktop config; mobile uses system default
CAP-06 System/loopback audio capture [x] [-] [ ] [-] Desktop only (caption source); mobile policy-limited
CAP-07 Local Whisper STT [x] [-] [x] [-] Desktop ships the faster-whisper sidecar (resources/sidecar, built by sidecar:build, verified by scripts/ci/verify-sidecar-bundle.mjs), warms it up at app start, and connects over IPv4 loopback; mobile on-device Whisper (supported devices) 2026-10-30: sidecar/cpu_budget.py applies the same bounded worker policy to primary/auxiliary Whisper models and CPU fallback; budget and constructor regression tests pass. 2026-11-01: rebuilt sidecar startup/health passed and runtime 1.16.0 is published; actual high-load typing remains open (GAP-INPUT-11).
CAP-08 Cloud STT (multi-provider) [x] [x] [x] [x] Desktop 6 providers + D3RO Cloud; web/mobile via stt-proxy; .NET internal gateway
CAP-09 STT auto-fallback + fail-closed [x] [x] [x] [x] STTManager; SSOT R-021/R-022 GREEN
CAP-10 STT model download/management UI [x] [-] [~] [-] Desktop model manager + onboarding; mobile bundled model
CAP-11 File transcription (audio/video) [x] [ ] [x] [~] Desktop ffmpeg chunking; mobile import picker; web deferred
CAP-12 Audio import from other apps (share intent) [-] [-] [x] [-] Mobile Android ACTION_SEND/ACTION_VIEW (SSOT R-016 GREEN)
CAP-13 Live captions overlay [x] [-] [-] [-] Desktop CaptionService + caption-overlay popup; popup assets verified in the packaged build (scripts/ci/verify-desktop-renderer-bundles.mjs); GAP-INFRA-05 2026-09-24 (unreleased, 1.7.0 candidate): fixed 6 s batches replaced by streaming (StreamingCaptionTrack + core caption-streaming.ts): the uncommitted buffer is re-recognised every 1 s (greedy partial) and sent as caption:delta {text, stable} where stable is the LocalAgreement-2 prefix; 0.7 s of silence finalises the line with a full pass; unbroken audio over 12 s commits segments ending before the last 1.5 s using Whisper segment times; idle audio is trimmed to 1.5 s. Captions always use the local engine; auto language is pinned after the first final. Finished lines are refined by the local LLM (buildCaptionRefinePrompt, acceptCaptionRefinement rejects >35 % change) and replaced via caption:segmentUpdated; toggle captionRefineEnabled (Settings). Overlay: draggable handle with remembered position, waiting notice until the first caption. Separate caption model captionSttModelId (null = dictation model): the sidecar keeps one aux model next to the primary (/load slot:"aux", /transcribe model_id, 409 model_not_loaded → reload + one retry); runtime minimum raised to 1.7.0.
CAP-14 Recording persistence / crash recovery [x] [ ] [x] [-] Desktop WAV persist; mobile durable queue + process-kill WAV recovery
CAP-15 Android foreground recording service [-] [-] [x] [-] Mobile API 34 FGS + persistent notification (SSOT R-005 GREEN)
CAP-16 Rebindable global key bindings (keyboard + mouse) [x] [-] [-] [-] Contract SSOT packages/core/src/keybinding.ts: KEY_CATALOG (10 groups), KEYBINDING_ACTIONS (6 actions), validateBinding, detectBindingConflicts. Multiple bindings per action persist as one AppConfig.keyBindings map (packages/core/src/types.ts), replacing the four singular *Shortcut fields; ConfigService migrates legacy values once. KeyBindingService hooks keyboard and mouse via uiohook — MB1 is not bindable, MB2/MB3 need a modifier, MB4/MB5 are free. Verified 2026-09-21 on Windows by a manual run and 2026-10-30 unit-tested in KeyBindingService.test.ts (11 tests, GAP-KEY-01 [x]), with dedicated command action toggle/hold + forced voice-command matching in VoiceModeService.ts / voice-intent.ts (voice-intent.test.ts 19 tests, GAP-KEY-03 [x]). Still open: macOS/Linux mouse behavior (11 GAP-KEY-02). W/M [-]: no OS-level global binding surface exists there. B [-]: device-local setting.

AI — AI Processing

ID Feature D W M B Anchors / notes
AI-01 Local LLM (Ollama) [~] [-] [ ] [-] Desktop bundled Ollama. 2026-09-22 guard contract: no boot warmup; suggestion keep_alive: 2m; request-owned cancellation; generate/stream 2048 tokens / 120 s and chat 512 / 60 s; done frame required with incomplete-stream cleanup; deduplicated Ollama spawn/poll plus lifecycle disposal. GAP-LLM-03 has targeted evidence (6 test files / 69 passed / 0 failed; changed code/tests ESLint and diff check exit 0). Raw Ollama confirms cold timeout cleanup, a 553 ms num_predict=1/keep_alive='2m' response, and after the about-119.9 s expiry a 19:48:59 +09:00 /api/ps HTTP 200 / 45.8 ms {models:[]} observation with no intervening generate/unload/kill/retry; /api/version was HTTP 200 / 7.3 ms / 0.32.13. This is raw API expiry/unload evidence, not app-restart or GUI/runtime proof (GAP-LLM-04). 2026-10-11 residency policy (supersedes 2m / later 30m): single LOCAL_LLM_KEEP_ALIVE='10m' for every local request (services/llm/model-residency.ts), no boot warmup (first keystroke warms via SuggestionService._keepWarm), keep-warm refresh only when <4 min left, and keep_alive: 0 unload of this app's resident models on quit (LocalLLMService.unloadResidentModels). Reason: gemma4:e4b holds ~11.6 GB RAM (5.9 GB pinned host buffer, no mmap on Windows CUDA) — boot warmup took free RAM 7.9 GB→32 MB within 20 s (GAP-PERF-01). 2026-10-30: loopback Ollama generate/stream/chat use max(1,min(4,available CPUs-2)) inference workers via inference-cpu-budget.ts; remote hosts retain their defaults. This is worker headroom, not a total CPU cap; CPU-only throughput and actual typing latency need measurement (GAP-INPUT-11).
AI-02 Cloud LLM (Claude/OpenAI) [x] [x] [x] [x] Desktop PremiumLLMService; web/mobile via llm-proxy; .NET LlmProxyService
AI-03 Auto Polish (cleanup/filler removal) [x] [~] [~] [x] Desktop built-in; web/mobile via commands. Desktop Auto Polish combines deterministic zero-latency speech disfluency cleanup (cleanDictatedText in @d3ro/core/dictation-cleaner, wired into VoiceModeService._transcribe) with the plain refine LLM action (llm-prompts.ts:28) and MeetingModeService.polishTranscript: strips spoken fillers (어, 음, 그 쓸데없는 while keeping quoted 어라든가/아라든가), mid-sentence self-corrections (검색 아니 검색이 아니라 타이핑을→타이핑을), redundancies (쓸데없는 의미없는→불필요한), colloquial suffixes (예를 들자면은→예를 들어, 보면은→보면), and contextual STT homophones (스시가 영어로 변경되지→PC가 영어로 변경되지, 그 길을 쓸 이유가→그 기능을 쓸 이유가) while strictly preserving the speaker's original tone/register (반말/해요체/합쇼체). Regression cases pin refine/summarize/grammar/expand (VoiceModeService.test.ts:435, llm-prompts.test.ts:153, dictation-cleaner.test.ts).
AI-04 Translate / summarize / rephrase [~] [x] [x] [x] Built-in instructions. Plain-action path (SettingsModal.tsx → defaultLLMAction) and built-in instruction presets (CustomInstructionService.ts) both use llm-prompts.ts. 2026-10-30 (GAP-LLM-01 [x]): added AppConfig.targetLanguage (default 'English'), 12-language selector in SettingsModal.tsx, and resolveTargetLanguage(explicit) reading ConfigService (llm-prompts.test.ts 43/43 PASS). [~] on desktop only until interactive GUI confirmation of custom-instruction insertion (11 GAP-LLM-02).
AI-05 Custom instructions (user commands) [~] [x] [x] [x] Web commands; mobile CommandsScreen — both go through Edge Functions and are unaffected. Desktop: the custom-instruction path never worked in any shipped release. The instruction was passed in the text argument of processText(text, action, targetLanguage, customPrompt) with the system-prompt argument left empty; BASE_SYSTEM_PROMPTS has no custom key, so resolution fell back to refine silently and the model polished the instruction it was handed — the transcript never reached it. Separately, only {{text}} was substituted and none of the built-ins use it ({{targetLanguage}}, {{userPrompt}}, or no placeholder), so the substitution was a no-op from the day it was written. Introduced fea923d (2026-04-05); present v0.1.0-alpha..v1.4.0; not a regression. Three entry points were affected: commands-page activation (CommandsPage.tsx:102-117), command-popup selection (bootstrap.ts:377-381), voice keyword match (VoiceModeService.ts:681-686). Fixed in 9c2b4d4 (2026-09-21) — llm-prompts.ts is now the SSOT for placeholder substitution and argument placement (renderInstructionPrompt:78, buildInstructionInvocation:101, resolveSystemPrompt:118). 2026-10-29: added a 6th built-in preset builtin-clean-speech (구어·추임새·중복 정리, CustomInstructionService.ts) and voice command triggers (정리해줘, 문장 정리해줘, 깔끔하게 정리해줘, 추임새 빼줘, 중복 정리해줘, clean this up in voice-command-policy.ts) that remove speech disfluencies, self-corrections, and redundancies while keeping the speaker's original tone (반말/존댓말). 2026-10-31: upgraded CommandsPage.tsx, tools-model.ts, VoiceModeService._processWithInstruction, and @d3ro/core/instruction-template to support multi-command ordered pipelines (parseActiveInstructionIds / composeInstructionPipelinePrompt, checkbox multi-select with 1단계·2단계 badges and ↑/↓ reordering, e.g., 1단계: 구어·추임새·중복 정리 → 2단계: 격식체 변환). The earlier "Red Team RT-03 verified" claim did not catch this and its cited evidence file (red_team_log.md) is not in the repo. [~] because unit tests pass (llm-prompts.test.ts 42, llm-handlers.test.ts 7, VoiceModeService.test.ts 22, ChainService.test.ts 5, CustomInstructionService.test.ts 12) but there is no running-app verification (11 GAP-LLM-02). See 11 GAP-LLM-01/02 Sync (2026-09-27): desktop user commands (not the desktop presets) mirror Supabase custom_instructions both ways, deletions included; server presets (builtin_key) are not copied to desktop. Presets (2026-09-27): prompt edits of the four desktop presets that have a phone counterpart (translate/summarize/formal/explain-code) update the server preset row through sync_set_builtin_instruction_prompt_v1 (NULL restores the default; {{targetLanguage}} is sent as English because both sides translate to English); the phone executes that prompt. 자유 프롬프트 and 구어·추임새·중복 정리 stay local
AI-06 Voice keyword commands [~] [-] [ ] [-] Desktop VoiceCommandService + command popup. Keyword matching itself works (VoiceModeService.ts:681-686), but execution went through the broken custom-instruction path (AI-05), and a second, separate defect made the shortcut skip LLM processing entirely whenever defaultLLMAction === 'none' — which is exactly the value the commands UI and the command popup write when no command is active (CommandsPage.tsx:116, bootstrap.ts:385), so an explicitly named instruction was nullified by the default setting. Both fixed in 9c2b4d4 (VoiceModeService.ts:779/:785); not verified in a running app (11 GAP-LLM-02)
AI-07 LLM Chains (multi-step pipelines) [~] [ ] [ ] [-] Desktop ChainService. The argument placement was already correct here, but chain steps never substituted placeholders, so {{targetLanguage}} / {{userPrompt}} reached the model verbatim as the system prompt. Now shares the same resolution function as the other two paths (ChainService.ts:196). Fixed in 9c2b4d4 (2026-09-21); 5 unit tests GREEN, but tests/red/chain.usecase.test.ts could not execute (11 GAP-INFRA-06) and there is no running-app verification (11 GAP-LLM-02)
AI-08 Screen/context capture for prompts [x] [-] [ ] [-] Desktop ScreenContextService
AI-09 Streaming responses [x] [x] [x] [x] SSE/NDJSON streaming
AI-10 Dictation templates (voice form fill) [x] [ ] [x] [~] Desktop DictationTemplateService; mobile TemplatesScreen Sync (2026-09-27): desktop custom dictation and meeting-document templates keep their id on the server through sync_upsert_user_template_v1/sync_delete_user_template_v1, which reuse the revision-checked mobile RPCs; builtins exist on both sides and are not copied.

MEM — Memory & Knowledge

ID Feature D W M B Anchors / notes
MEM-01 History list + search [x] [x] [x] [x] Desktop SQLite (HistoryService.list/getAll with SQLite mode, isFavorite, tag, search, and beforeCreatedAt cursor pagination — 2026-10-30 GAP-UX-02 [x]); web/mobile Supabase
MEM-02 History detail + edit [x] [x] [x] [x] Desktop HistoryService.update + history:update IPC + inline edit in HistoryEntryCard.tsx (2026-10-30 GAP-UX-02 [x]); Mobile HistoryDetailScreen
MEM-03 History favorites [~] [x] [x] [x] Desktop had no favorites until 2026-09-27 (the old [x] was wrong): history.is_favorite column, HistoryService.setFavorite + history:setFavorite IPC, star toggle in HistoryEntryCard; the value is the same Supabase history.is_favorite mobile filters on, so it syncs both ways (apps/desktop/tests/main/sync/SyncEngine.test.ts (20) + apps/desktop/tests/integration/cross-device-sync.supabase.test.ts (5, local Supabase stack)). [~] until a GUI run toggles it and mobile shows it
MEM-04 History export / share [x] [~] [x] [x] Desktop MemoService.exportMarkdown exports both tagged and untagged history with filter support and interactive save dialog (2026-10-30 GAP-UX-02 [x]); web limited; mobile share sheet
MEM-05 History audio playback [~] [x] [x] [x] Signed URLs on web/mobile. Desktop had no playback at all (the old [x] was wrong); since 2026-09-27 the history card plays this device's recording from the file (history:getAudio → bytes → Blob) and a phone/web recording through a 5-minute signed URL of its audio_files row. Desktop recordings upload to the same audio bucket contract ({userId}/desktop/..., audio_files, ≤50 MiB, Settings > Cloud toggle cloudSyncAudio) so the phone plays them too (tests/main/sync/SyncExtensions.test.ts (9) + integration case "지식 문서·설정·녹음" against the local Supabase stack). [~] until a GUI run plays both kinds
MEM-06 Dictionary (custom vocabulary) [x] [x] [x] [x] All surfaces CRUD; Desktop Red Team RT-02 & RT-18 fuzzed/verified
MEM-07 Dictionary import/export [x] [x] [x] [-] Desktop dictionary:import/export JSON+CSV (file dialogs, DictionaryService); web serializeDictionary/importDictionaryFile + header buttons; mobile CSV/TXT export + CSV/JSON/TXT import via data-portability
MEM-08 Memos (tags over history) [x] [ ] [x] [x] Desktop MemoService; mobile MemosScreen; web none. Desktop tags sync with mobile since 2026-09-27: pushed through the mobile RPCs (mobile_add/remove_memo_tag_v1) and reconciled by natural key historyId + normalized tag because the server owns tag ids and renames rows (services/sync/memo-tag-sync.ts). Tag normalisation (trim, collapse spaces, lower-case) is now identical on both sides
MEM-09 Knowledge base / RAG add+index [x] [x] [x] [x] Desktop local RAG (DEF-008 infinite chunking loop resolved, RT-08 verified); web/mobile cloud RAG Sync (2026-09-27): desktop documents mirror knowledge_documents + knowledge_chunks as source text; each surface embeds with its own model (desktop nomic 768-d, server 1536-d), the server index is requested through embed-chunks. Documents synced from the phone are stored without a file and re-indexed from their stored chunks (RAGService.applyRemoteDocument). Chunk text is now kept even when local embedding fails
MEM-10 Semantic search over knowledge [x] [x] [x] [x] Web KnowledgeSearch calls search-knowledge (was mislabeled deferred); mobile + Edge search-knowledge
MEM-11 Knowledge file upload [x] [x] [x] [x] Web .txt/.md picker + newline-aware chunking + embed-chunks; desktop txt/md/pdf/docx; mobile file picker Sync (2026-09-27): desktop documents mirror knowledge_documents + knowledge_chunks as source text; each surface embeds with its own model (desktop nomic 768-d, server 1536-d), the server index is requested through embed-chunks. Documents synced from the phone are stored without a file and re-indexed from their stored chunks (RAGService.applyRemoteDocument). Chunk text is now kept even when local embedding fails
MEM-12 Voice actions (OS automation) [x] [x] [x] [x] Desktop VoiceActionService; web ActionRunner (simulated); mobile ActionsScreen
MEM-13 Cross-surface data sync [~] [x] [x] [x] Desktop rewritten 2026-09-27 (services/sync/SyncEngine.ts): persistent outbox (offline/restart safe), server-clock keyset cursors per user DB with pagination, pending local edits never overwritten by pull, deletions both ways through sync_tombstones, per-row failure isolation, Realtime that actually connects (ws transport). Covers history (+title/favorite), dictionary, meetings, meeting memos/documents, memo tags, custom commands, dictation + meeting-doc templates; plus device registration. Evidence: apps/desktop/tests/main/sync/SyncEngine.test.ts (20) + apps/desktop/tests/integration/cross-device-sync.supabase.test.ts (5, local Supabase stack). [~] until migration 20260927000034 is deployed to production and a desktop build is exercised against a phone (GAP-SYNC-05); knowledge/audio/settings still local (GAP-SYNC-06) 2026-09-27 second batch: knowledge base, recordings (upload + remote playback) and the mobile user_settings fields that mean the same thing on both sides (ko/en locale, system/light/dark theme, auto-polish on/off, active user command) now sync too; remote setting changes reach the open window (app:dataChanged, config:changed). Production DB is at migration 0035 and all Edge Functions are redeployed from the repo
MEM-14 Memo tag search [x] [ ] [x] [x]
MEM-15 Unified record + meeting search [~] [ ] [x] [-] Shared @d3ro/core/types result contract. 2026-10-30 (GAP-UX-05): Mobile (unified-search-service.ts, UnifiedSearchScreen.tsx) now supports AbortSignal/AbortController in-flight cancellation, transcripts segment table search (searchTranscriptSegments), and bounded payload projection (boundPayloadText), verified by unified-search-service.test.ts + 320dp dark/411dp light runtime journeys ([x]). Desktop (UnifiedSearchService.ts) searches history, meetings, meetingMemos, and timed transcript segments (UnifiedSearchService.test.ts 12/12 PASS; [~] pending Electron Ctrl+K manual check).

MTG — Meetings

ID Feature D W M B Anchors / notes
MTG-01 Meeting recording + live transcript [x] [~] [x] [x] Desktop MeetingModeService; web realtime view; mobile Sync (2026-09-27): the phone renders transcripts segments in preference to edited_transcript, so desktop edits, auto-polish and diarization never reached it; the desktop now rewrites the meeting's segments from its [MM:SS] [speaker] text lines on every transcript change (upsert by meeting_id,segment_index, trailing segments trimmed; parser SSOT @d3ro/core/meeting-transcript, tests/main/sync/SyncTranscriptsPresets.test.ts (4) + local-stack integration case "전사 구간·프리셋")
MTG-02 Meeting creation (title/attendees/language/template) [x] [x] [x] [x] SSOT H-006 GREEN
MTG-03 Timestamped memos during & after meeting [x] [x] [x] [x] 2026-10-30 (GAP-UX-01 [x]): MeetingModeService.addMemo(content, meetingId?) supports ended meetings and syncs MeetingNotesTab notepad notes via CloudSyncService.pushOne('meeting_memos', id) (desktop-ux-gaps.test.ts 13/13, meeting.usecase.test.ts 23/23 PASS).
MTG-04 AI summary generation [x] [x] [~] [x] Desktop MeetingSummaryService generates [MM:SS] timestamp citations (parseCitationTimestampMs, extractTranscriptCitations) with clickable jump-to-segment badges in MarkdownRenderer/MeetingSummaryTab (2026-10-30 GAP-UX-03 [x]); web generate-document-button; mobile via Edge
MTG-05 Document generation (minutes/report/idea-note/mindmap) [x] [x] [x] [x] Edge generate-meeting-document; SSOT F-011 GREEN
MTG-06 Document edit (Markdown) [x] [x] [x] [x] Desktop editor; web document-editor
MTG-07 Export PDF/DOCX/TXT/Markdown [x] [~] [x] [x] Desktop ExportMenu; web markdown; mobile print/DOCX chooser (SSOT data portability GREEN)
MTG-08 Speaker diarization [~] [ ] [ ] [ ] Desktop phase-15.5 (LLM estimate + pyannote prep); mobile SSOT H-014 pending
MTG-09 Audio seek ↔ transcript timestamp [x] [x] [x] [-] 2026-10-30 (GAP-MTG-01 [x]): Desktop (TranscriptTab.tsx / EditableSegment.tsx), Web (meeting-audio-player.tsx ↔ live-transcript-list.tsx), and Mobile (meetings-service.ts, MeetingDetailScreen.tsx) parse [MM:SS] / timestamp_ms segments, seek audio on badge tap, and highlight the active transcript segment (__tests__/meetings.test.ts PASS).
MTG-10 Meeting list search/filter/sort [~] [ ] [~] [x] Basic lists; advanced filters pending
MTG-11 Meeting AI chat over transcript [x] [ ] [ ] [x] Desktop MeetingChatPanel + MEETING_CHAT
MTG-12 Content reporting for generated docs [-] [ ] [x] [x] content-report Edge + generation receipts
MTG-13 Existing audio/video import into a meeting [x] [ ] [~] [x] 2026-11-01: Desktop meetingMode:importRecording reuses FileTranscriptionService without duplicate history, saves canonical timestamped transcript + normalized WAV, then opens existing minutes generation. Mobile MeetingsScreen creates the workspace then routes meetingId + importRequest to the existing durable Record meeting pipeline. 2026-11-02: installed Windows 1.16.0 passed the actual Electron picker→timestamped transcript→local summary/action-item journey, so desktop is [x]. Fold6 .3 import entry/editor rendered, but authenticated device file-to-minutes E2E remains (GAP-MTG-03). Evidence.
MTG-14 Learning-informed transcript correction (user dictionary + typed vocabulary) [~] [ ] [ ] [ ] 2026-11-05 (root cause + rebuild of the transcript-correction seam): the learned corpus was only wired to input suggestions. Meeting/live captions ran STT with a hardcoded 다음은 한국어 대화입니다. prompt (no dictionary, no learned terms), the refine prompt had no glossary, the refine result updated only CaptionService's own copy (meetings and the renderer kept the raw line), and the persisted rawTranscript was a plain join of raw segments. Now: core SSOT @d3ro/core/transcript-correction (glossary build, dictionary-variant replacement, learned-identifier correction with a unique-best rule, LLM reference block) + desktop TranscriptGlossaryService (dictionary + personal_phrases, 15 s cache invalidated on dictionary writes, fail-closed) are wired into CaptionService (hint on every recognition, deterministic correction per final line, glossary-aware refine, segment-updated event), MeetingRecordingSession/MeetingModeService (refined lines reach the meeting transcript and renderer via meetingMode:segmentUpdated; auto-polish gets the glossary block), FileTranscriptionService (per-chunk hint + correction) and VoiceModeService (learned-term pass). 2026-11-05 (v1.16.3 correction policy): the first cut also rewrote ordinary Hangul words by 1-jamo distance. Measured against the real local corpus (users/_local/d3ro.db, 3 stored meeting transcripts) it changed 286 of 3,005 words (하면 to 라면, 이제 to 이게, 매일 to 메일, 정보 to 정도). Deterministic correction is now limited to dictionary pronunciations and typed Latin identifiers (canonical casing preserved); Korean terms reach recognition through the STT hint and the polish prompt only. Re-measured on the same corpus: 0 of 3,005. Real-engine check (sidecar large-v3-turbo, Korean TTS fixture containing the user term): baseline 하람로그에 to hinted transcript haramlog에, and a dictionary pronunciation entry still maps 디에로 to D3RO. Hangul-layout typing artifacts (dkssudgktpdy) are dropped from the hint. 2026-11-06 (v1.16.4 AI correction grounded in learning): the polish pass no longer runs as a generic rewrite. Deterministic dictionary/identifier correction runs before and after the model, and the model receives a learned term list plus the user expressions with explicit rules (fix a misheard name/product/in-house term to the learned spelling; never swap ordinary nouns or endings on similarity alone). Re-measured on the real corpus: 0 of 3,005 deterministic changes, and the generated term block now shows canonical identifiers (HaramLog, Android, AI) with Hangul-layout artifacts removed from terms, hints and style examples. Dictation output is re-checked against the learned vocabulary after the LLM finishes. Tests: core transcript-correction.test.ts (17), desktop transcript-glossary.test.ts (10), red meeting-llm usecase (+1), polish-prompt.test.ts (4). Real-model probe (2026-11-06, local gemma4:e4b with think:false): with the production prompt and the real learned corpus, the misheard 하람록에 was rewritten to HaramLog에 and the dictionary pronunciation 디에로 to D3RO (2 word changes, no collateral edits); a natural Korean sentence without a learned term came back with 0 word substitutions (only a spacing merge); filler and self-correction were removed. Trap: without think:false (or /no_think) this model spends the whole output budget on its thinking block and returns empty content, which the chunk guard rejects. 2026-11-06 (v1.16.5 polish guards): the polish prompt was re-tuned by A/B on the local model and two invariants moved from prompt to code, because the model broke them intermittently: (a) invented [MM:SS] / [화자 N] markers are stripped from the saved transcript (stripInventedMarkers), (b) meaning qualifiers the speaker said (거의/대부분/아직 …) are restored from the original chunk (restoreProtectedQualifiers). Measured on the reporter transcript: fillers gone, comma fragments joined, 유치역학 → 유체역학 repaired, 거의 kept after the guard even when the model dropped it. Unknown-word repair itself is model-dependent (2/3 then 1/2 across prompt sets) and becomes deterministic only once the term is in the dictionary or has been typed. Tests: core hardening suite (15), polish-prompt (7), red meeting-llm (+2). Live meeting with real jargon still open (GAP-MTG-04).

CV — Conversation

ID Feature D W M B Anchors / notes
CV-01 Local duplex voice conversation (STT→LLM→TTS) [x] [ ] [x] [-] Desktop VoiceConversationService; mobile Talk
CV-02 Realtime voice (OpenAI gpt-realtime, Premium) [x] [ ] [ ] [x] Desktop useRealtimeConversation + realtime-token Edge
CV-03 Text AI chat [x] [x] [x] [x] Desktop chat, web chat-panel, mobile TalkScreen
CV-04 TTS playback + controls [x] [ ] [x] [-] Desktop SAPI/say; mobile Android TTS
CV-05 Voice selection / backend selection [x] [ ] [~] [-] Desktop settings conversationBackend

ACC — Accounts, Sync, Devices, Portability

ID Feature D W M B Anchors / notes
ACC-01 Email sign-up/login [ ] [x] [x] [x] Desktop uses separate online-auth; SSOT A-001..A-005 [ ] rows are mobile checklist granularity
ACC-02 OAuth Google [~] [x] [~] [x] Mobile full consent→callback pending (external)
ACC-03 OAuth GitHub / Apple [~] [~] [~] [~] GitHub/Apple provider secrets pending
ACC-04 Password reset / recovery deep link [ ] [ ] [x] [x] Mobile ForgotPassword/UpdatePassword
ACC-05 Account profile / identity management [~] [~] [x] [x] Desktop LicenseTab; mobile AccountScreen
ACC-06 Logout + local sensitive purge [x] [x] [x] [x] Mobile central purge GREEN
ACC-07 Account deletion (server cascade + local purge) [ ] [ ] [~] [x] account-delete Edge; mobile device E2E pending. 2026-11-04 backend retention proof: isolated local Supabase verified audio-purge authorization and both direct and pg_cron/Vault dispatch; the queued DB row, deleted audio_files row, and private Storage object were removed together. Production deployment/schedule remains GAP-OPS-04.
ACC-08 Cloud sync (per-user data) [~] [x] [x] [x] Desktop CloudSyncService + services/sync/. The earlier [x] hid shipped defects fixed on 2026-09-27 (GAP-SYNC-01/02): first pull after sign-in fetched nothing, only the first meeting's memos/documents were pushed, deletions never propagated, team meetings were pulled into the personal DB and their team_id cleared on re-push, Realtime never connected. Anonymous local-mode data is now imported into the first account that signs in (db/index.ts importLocalModeData). Production deploy pending (GAP-SYNC-05)
ACC-09 Device registration + revocation [~] [~] [x] [x] Mobile DevicesScreen. Desktop registers itself since 2026-09-27 (services/sync/device-registration.ts, platform windows/macos, installation id in AppConfig.deviceInstallationId); revoking it from the phone signs the desktop out (Realtime on devices + 5-min heartbeat), signing out on desktop unregisters it. Web creates/checks in a stable platform='web' device only when the user opts into browser notifications, refuses to resurrect a revoked row, and revalidates an existing local Push subscription through the register RPC before showing it connected (web-device.ts, web-push-session.ts, web-push.spec.ts). An isolated throwaway local-browser run produced exactly one active web device and one device-bound token; no web device-management UI or production proof. Mobile parser accepts all five platforms and names them on the card. Local-stack desktop evidence exists; production pending GAP-SYNC-05
ACC-10 Offline queue + retry [~] [ ] [x] [x] Mobile durable queue
ACC-11 Data export/import (portability) [~] [ ] [x] [x] Mobile canonical JSON E2E GREEN; desktop has export files. 2026-09-27 backend fix: export_account_portability serialised whole rows, so columns added after the v1 archive (meeting creation metadata, meeting-document template/idempotency) made any account with meetings fail the exact-key check in both the server restore and the mobile parser; the export now projects onto v1 keys (migration 20260927000034, integration test asserts memo/meeting keys)
ACC-12 Notification / push [ ] [~] [~] [x] Backend transports for FCM + Web Push (VAPID) + APNs (.p8) + outbox cron drain implemented. 2026-11-04 (GAP-PUSH-03 [~]): Web production sidebar calls the explicit user-gesture permission → stable web device → subscribeWebPush → register_push_registration path. Opt-out records disabled intent before bounded parallel server/local cleanup; a stale local subscription is re-registered through the server RPC before it may appear connected, and logout never waits on cleanup (web-push.spec.ts 17 contract tests). The standalone image copies public/sw.js, and missing build-time NEXT_PUBLIC_VAPID_PUBLIC_KEY fails closed. Isolated local evidence reached active device + token + pending outbox + completed drain + push_deliveries.status='delivered'; this is provider acceptance, not browser receipt. The final production-standalone browser attempt timed out at login navigation before the dashboard, so production permission/subscription/render/receipt and deployed cron/provider configuration remain open (EXT-FIREBASE-01 for Android).

TEAM — Teams & Admin

ID Feature D W M B Anchors / notes
TEAM-01 Team create / rename / delete [-] [x] [x] [x] Desktop N/A
TEAM-02 Invite by email / link + accept deep link [-] [x] [x] [x] team-invite/team-accept Edge Functions; mobile deep link; 2026-10-30 (GAP-TEAM-02 [x]): /app/accept-invite?token=... browser flow + landing /accept-invite/ web CTA.
TEAM-03 Members + roles (admin/member/viewer) + leave [-] [x] [x] [x]
TEAM-04 Team meetings/docs sharing + RLS isolation [-] [x] [~] [x] Cross-user isolation tested
TEAM-05 Team comments / activity feed [-] [x] [x] [x] team_activities migration + create_team_activity RPC + realtime; web ActivityFeed, mobile TeamDetail activity card (2026-09-13)
TEAM-06 Admin back office (users/subs/models/usage/audit) [-] [-] [x] [x] apps/admin + mobile AdminScreen
TEAM-07 Role-based destructive action confirm + audit [-] [x] [x] [x] SSOT T-006..T-008 GREEN
TEAM-08 Desktop admin surface [ ] [-] [-] [-] None; N/A by design

MON — Monetization & Ads

ID Feature D W M B Anchors / notes
MON-01 Tier gating (Free/Pro/Pro+/Team/Enterprise) [x] [x] [x] [x] LicenseService, entitlement provider
MON-02 Usage quotas (daily_usage) [x] [x] [x] [x] packages/core/src/plan-catalog.ts is the quota SSOT, including knowledge_embedding; Deno consumers use the generated core-contract.generated.ts copy guarded by npm run contract:check.
MON-03 Desktop offline license (Ed25519) [x] [-] [-] [x] crypto-license + admin issuer
MON-04 Web checkout (Stripe) — 삭제됨 2026-09-26 [-] [-] [-] [-] 사용자 결정: 결제는 웹 Payple(KRW) + 모바일 Google Play만. stripe-checkout/stripe-portal/stripe-webhook 함수, 웹 checkout-button/portal-button, 데스크톱 CheckoutModal·payment-handlers·payment:* IPC 삭제. 과거 provider='stripe' 구독 행은 읽기 전용 표시만(마이그레이션 불변). 운영 정리는 GAP-BILL-03
MON-05 Web checkout (Payple) [-] [x] [~] [~] Payple checkout/manage/renew/webhook; webhook signature pending
MON-06 Paywall / upgrade prompts [x] [x] [x] [x] UpgradePromptModal, ProPaywallScreen
MON-07 Mobile IAP purchase + restore [-] [-] [~] [x] iap-verify + billing-context; live store E2E blocked
MON-08 Billing catalog / pricing display [x] [x] [x] [x] Server catalog SSOT; hardcoded prices removed
MON-09 Free-tier banner ads [~] [-] [x] [x] Desktop adapters fail-closed; mobile AdMob test GREEN, prod serving blocked
MON-10 Rewarded ads → quota credits [x] [-] [x] [x] 2026-10-30 (GAP-ADS-02 [x]): Desktop AdMediationEngine.claimReward verifies adapter reportRewardCompletion, impression, duration, cooldown, and nonce uniqueness before calling LicenseService.grantRewardQuota (AdMediationEngine.test.ts 23/23 PASS); mobile SSV GREEN
MON-11 Ad mediation engine + settlement [~] [-] [~] [x] Engine + settlement built. DirectHouseSponsorAdapter is now a real configurable REST adapter (bid/impression/click/reward via endpointUrl, fail-closed when unconfigured, unit-tested). Other 9 networks remain UnavailableAdAdapter stubs pending official SDKs.
MON-12 Subscription management (portal/store) [-] [x] [x] [x] Payple manage / Play manage (Stripe portal 삭제 2026-09-26)

INPUT — Input Intelligence (typing telemetry & next-sentence suggestions)

Everything here is desktop-only (Windows today) and opt-in: inputTelemetryEnabled and inputLearnTypedText default to false. D=desktop, W=web, M=mobile, B=backend. Status is [~] on desktop for one shared reason: the code and its 48 unit cases exist, but the end-to-end behaviour has not been verified by typing in a real app (11 GAP-INPUT-01).

ID Feature D W M B Anchors / notes
INPUT-01 Keyboard/mouse telemetry capture (opt-in) [~] [-] [-] [-] InputTelemetryService — keystroke/click/scroll counters, mouse travel as the Manhattan sum of per-axis pixel deltas, active time, per-hour×app buckets flushed every 5 s. Key contents are never stored (ActivityWatch aw-watcher-input data-minimisation policy, adopted deliberately — see 11 §7). Hook ownership is ref-counted so KeyBindingService keeps working (global-input-hook.ts).
INPUT-02 Foreground-app attribution [~] [-] [-] [-] utils/win32-foreground.ts via koffi FFI (title/pid/exe/bounds), sampled at most 1×/s. get-windows was rejected: it needs an install script this repo does not run.
INPUT-03 Weekly input insights [~] [-] [-] [-] INPUT_TELEMETRY.getSummary aggregates input_activity into totals, daily series, top hours and top apps; rendered in Settings → Input and as a dashboard card. Daily average mouse travel is converted px → m using the display scale factor.
INPUT-04 Typed-text learning (UIA, password-excluded) [~] [-] [-] [-] Text is read from the focused field via GET /uia/focus (sidecar UIA bridge) and diffed longest-common-prefix/suffix, so IME-committed Hangul/kana is counted correctly — keycodes cannot reconstruct CJK text. UIA sends hasSelection only, derived by TextPattern range Start/End comparison without calling GetText on the selection range or adding a selected-text payload; the existing focused-field text can still include a selection. A non-collapsed selection immediately clears suggestions as selection-active. IsPassword is checked before any read (fail-closed); IME composition suppresses both stats and suggestions. 2026-09-24: text equal to the control name (empty-field placeholder, e.g. "메시지 입력") is treated as empty (withoutPlaceholderText); a suggestion requires editedSinceFocus + typing within recentTypingWindowMs (8 s), otherwise not-typing — clicking into a pre-filled field no longer triggers suggestions. 2026-10-29: _typedInCurrentFocus and _lastTypedHwnd track whether a typing keystroke occurred after the latest click/focus rebase (_rebaseFocusText), and _sampleForeground forces immediate foreground sampling on that first keystroke, so typing immediately after clicking into a search box or new window sets editedSinceFocus = true on the very first snapshot. 2026-10-24: typed deltas are no longer saved per snapshot (that stored mid-word fragments every 0.4–0.8 s). They are buffered per field and saved as one baseline→latest diff on 2 s idle, sentence-ending punctuation, 150-char growth, field clear (send), focus change or stop. Typed samples under 4 non-space chars, URL/query-like lines, trailing incomplete jamo and PUA/lone-surrogate chars (emoji truncated to 16 bits upstream, root cause unconfirmed) are dropped. Rows already in the DB are pruned on start by the same rule (isLearnableTypedText in _pruneUnlearnableCorpus, typed source only; dry count on the author machine 2026-10-24: 1,235/2,664 samples, 285/1,209 phrases). Expressive jamo runs (ㅋㅋ, ㅠㅠ, ㅇㅇ) and space-less questions are kept. Mid-word fragments of 4+ chars cannot be detected and remain until the 30-day retention. 2026-10-30..31: UIA reads are single-flight and discarded after input/focus/lifecycle changes, stale capture times, or changed native GetLastInputInfo tokens; contexts carry inputRevision/nativeInputTick for replacement-time validation (InputTelemetryService, win32-input-activity). Standalone modifier keydowns/key-repeats (Ctrl/Alt/Shift/Win) and Ctrl+Alt suggestion navigation chords are ignored in _handleKeyDown without advancing _inputRevision/_lastKeyAt or scheduling UIA snapshots (classifyKeyStroke checks modifier VKs before modifier flags), and _settleTimer (780 ms) is skipped on non-typing keys and auto-cancelled when the 380 ms _textTimer snapshot cannot trigger a suggestion (prefixLen < minPrefixChars). See GAP-INPUT-11.
INPUT-05 Personal phrase corpus (typed + voice) [~] [-] [-] [-] Sentence-level phrases from typed text and from voice history (HistoryService.create feeds recordExternalText), ranked by frequency/recency as prompt hints; users can delete individual phrases. 2026-09-24: typed text from LEARNING_EXCLUDED_APPS (terminals, code editors, Agent Switchboard) is never learned, and every phrase must pass isLearnablePhrase (no box/block/geometric glyphs, ≥60 % letters). _pruneUnlearnableCorpus re-applies both rules to existing phrases/samples/edges on start and every retention cycle (first run on the author machine removed 69 phrases / 91 samples of terminal status lines and agent chats). 2026-10-25: Agent Switchboard moved from LEARNING_EXCLUDED_APPS to APP_SCOPED_LEARNING_APPS — its text is kept only as typing_samples (with app name), never in shared phrases/graph, and is used only for suggestions inside that same app (findContinuationHints(..., { appName }) feeds continuationHints). It is the user's most-typed app; the 2026-09-24 exclusion was about dev instructions leaking into KakaoTalk suggestions, which scoping prevents.
INPUT-06 Next-sentence suggestion (ghost text) [~] [-] [-] [-] SuggestionService + buildSuggestionPrompt (llm-prompts.ts SSOT, instruction stays in the system prompt). The 2026-09-22 guard contract is 600 ms debounce, ≥5 s between requests, 6 requests/min by default (hard-config maximum 12), 3 candidates, 64 output tokens, 12-character growth before regeneration, 8 s request timeout and keep_alive: 2m; boot warmup is removed. Each request has its own cancellation signal. Presentation-active includes candidates, generating, warmingUp and partialText; clear/dismiss aborts, invalidates the generation token, clears TTL state and emits cleared/hide, and a final success resets generating=false/partialText=null. Focused evidence for the lifecycle and Windows-child-process change: five test files / 80 tests passed; desktop typecheck/lint, Python py_compile, and git diff --check exited 0 (core 131-test pass was independently verified earlier). Status remains [~]: this is not app-restart, GUI overlay, or real automatic-typing evidence. 2026-09-24 (supersedes the numbers above): keep_alive: 10m, warm-up on boot, on Ollama reconnect and whenever the model is presumed cold (cold reload measured 11.4 s > 8 s timeout); a session asks for 1 candidate, then a sequential fill loop appends one unique candidate at a time up to maxCandidatesTotal 12 (avoid-list in the prompt, stops after 2 empty fills); only the first request consumes rate/daily budget; stale aborts refund the budget; any further typing ends the session (matchesSessionPrefix, IME last-syllable tolerant); prompt forbids assistant-style questions. Terminals (TERMINAL_APPS) never get suggestions. 2026-10-24 accept fix: waitForModifiersReleased now trusts OS GetAsyncKeyState (utils/win32-key-state.ts) over the uiohook-tracked set. A missed keyup made acceptance wait the full 1.5 s and then paste Ctrl+V while modifiers were still considered held; in the logged case the field length stayed 15 after "inserted" (6 chars). New suggestions are also suppressed while an accept is in flight, so the overlay no longer reappears during the wait. 2026-10-24~25 experiment, reverted: an n-gram-first / confidence-gated / short-candidate rework raised offline next-word precision (3%→33%) but made suggestions rare, and the user judged the app much worse. Suggestion behaviour is back to v1.10.1. Kept: suggestions that copy on-screen text are rejected (copiesContext), no suggestions while typing an address/path (isAddressInput, skip reason address-input). Lessons and data: docs/evidence/2026-10-24-suggestion-accuracy-research.md §9; replay harness tests/eval/. 2026-10-25 additive upgrades (frequency unchanged): (1) suggestion-accept-word (default Ctrl+Alt+Space) inserts only the first word and keeps the rest on screen (splitFirstWord; partial accepts are not learned as full sentences). (2) a confident personal n-gram candidate (PersonalNgramIndex, strict PERSONAL_NGRAM_SETTINGS) is shown instantly as candidate #1 while the model generates; model candidates are appended after it, and it survives an empty/failed/timed-out model run (no "no suggestions" notice then). Agent Switchboard text never enters this shared n-gram. Hotkey accept now force-releases held Ctrl/Alt/Shift after 200 ms (prepareModifiersForPaste). 2026-10-28 extreme personalization & n-gram upgrades (frequency unchanged): PersonalNgramModel now weights repeated personalPhrases (count >= 2) and falls back to a Korean stemmed-word bigram index (_stemBigram via stemTerm, e.g. matching "배포를 완료했습니다" when typing "배포가 "), and SuggestionService._fillOne reranks newly generated LLM candidates by scoreCandidatePersonalization (overlap with personal graph continuations/followUps/related/phrases + register match) while preserving the instant n-gram slot at index 0. 2026-10-31 (v1.15.2) Ctrl+Alt+Enter reliability & terminal accept fix: (1) ChordStateMachine.keyUp releases all held bindings sharing a released modifier (Ctrl/RightAlt) so suggestion-accept never stays stuck in _isDown after command (Ctrl+RightAlt), (2) KeyBindingService maps UiohookKey.NumpadEnter and adds a 25 ms globalShortcut fallback trigger while isOverlayOrCaptureActive is true, (3) SuggestionService.accept() waits up to 2.5 s (_waitForFirstCandidate) when invoked while candidate #1 is still generating/pending, and (4) in terminalMode / isTerminalApp, _insertAccepted uses 'keyboard' (KEYEVENTF_UNICODE) instead of 'clipboard' (Ctrl+V), syncs TerminalKeystrokeTracker via applyProgrammaticReplacement, and stops _fillRemainingCandidates immediately (suggestion-accept-hold.test.ts 4/4, KeyBindingService.test.ts 12/12, keybinding-runtime.test.ts 29/29 PASS).
INPUT-07 Caret-anchored suggestion overlay [~] [-] [-] [-] suggestion-overlay popup placed by anchorFloatingPanel (caret → element → cursor fallback, flip above when the caret is near the bottom, clamped to the work area). Non-focusable; click-through unless suggestionOverlayInteractive. While actually visible it continues periodic UIA validation after 5 s and revalidates 120 ms after mouse-up; unavailable/non-editable focus or a non-collapsed selection aborts and hides it. X first hides the renderer panel, then main IPC directly hides BrowserWindow and dismisses the service, so late tokened results cannot revive it. Accept/next/prev/dismiss are four global key bindings (suggestion-accept/next/prev/dismiss, default Ctrl+Alt+→/↓/↑/←), and the overlay has a mouse close button. Up to three candidates are shown in a scrollable list with a warm-up/generating spinner. 2026-09-24: one page of 3 fixed-height (2-line) numbered items, footer = range/progress + source + key guide derived from the live bindings (buildSuggestionKeyHints, shared modifier shown once). Keys: Ctrl+Alt+↑/↓ move (the page follows the active item; at the last item next waits while filling, otherwise wraps; the 1.6.0 Left/Right page actions were removed as unreachable before a page filled and colliding with Intel display rotation), Ctrl+Alt+Enter accept, plain Esc closes while visible (overlay-scoped, not a binding), Ctrl+Alt+Backspace secondary dismiss; old default bindings migrate at tuning revision 5. Streaming updates never reposition/re-present the window (decideSuggestionOverlayAction); placement uses anchorKind — caret: below the line, element: outside the element (below → above → right → left → inner bottom-right). Navigation re-arms the visible TTL. Window 460×208.
INPUT-08 Per-app exclusions & consent controls [~] [-] [-] [-] inputExcludedApps (executable names, case-insensitive) blocks both collection context and suggestions; telemetry master switch, pause, text-learning toggle and "delete collected data" all live in Settings → Input. 30-day retention prune runs on start.
INPUT-09 Flow Radar [~] [-] [-] [-] rankFlowWindows ranks hourly aggregate activity density, character volume and edit stability into potential-focus time windows. It is not a real-session detector or session record.
INPUT-10 Edit Friction [~] [-] [-] [-] calculateFrictionInsight derives friction from char/backspace quantities and reports edits per 100 chars; it does not infer sentiment or productivity.
INPUT-11 App DNA [~] [-] [-] [-] Personal phrase ranking passes appName context and applies a 1.75× same-app ranking bonus (selectPhraseHints); learned text remains opt-in.
INPUT-12 App Quality [~] [-] [-] [-] InputTelemetryService.getSummary aggregates per-app suggestion total, accepted count, accept rate and average latency from local suggestion history.
INPUT-13 Privacy Receipt [~] [-] [-] [-] Local-only receipt shows actual row counts. Raw individual key events/key codes/content stream are not retained; opt-in learned text can reside in typing_samples/personal_phrases. input_activity, typing_samples and suggestions retain 30 days; personal_phrases has no age-based automatic expiry and is removed by individual deletion, delete-all, or consent withdrawal. Receipt read/delete storage failures are IPC errors.
INPUT-14 Smart Exclusion [~] [-] [-] [-] Password fields add no evidence. Only the current app may receive a one-click recommendation after ≥4 observations, readable=0 and problematic ratio ≥75%; it never auto-excludes.
INPUT-15 Why This Suggestion [~] [-] [-] [-] Overlay exposes only local-model/local-memory provenance and continuation/related/phrase/appPhrase counts; raw evidence text is not shown.
INPUT-16 Memory Decay [~] [-] [-] [-] Phrase ranking combines a 30-day half-life, frequency and the App DNA 1.75× same-app bonus.
INPUT-17 Instant Recall [~] [-] [-] [-] Local-memory fallback (provenance=local-memory) occurs only for unavailable model, non-cancellation error, actual timeout or empty model result; it spends no additional model budget and is blocked on dismiss, new typing, token/context mismatch or staleness.
INPUT-18 Shortcut Safety Audit [~] [-] [-] [-] Settings UI applies core auditKeyBindingMap to surface invalid/conflict issues while hold/double-press exceptions remain unchanged.
INPUT-19 Typo auto-correct while typing (built-in automatic Han/Eng conversion) [~] [-] [-] [-] 2026-10-03; upgraded 2026-10-26..31 (2-Stage Hybrid Auto-Correction: Araso-caliber instant layout engine + Smart Background Gemma Pass + Atomic Single-Syscall SendInput + Delta Rebasing + Terminal & Vim & xterm.js zero-swallow IME protection + live Windows GUI E2E verified). TypoCorrectionService (main, singleton + EventEmitter) runs on each typed-context before SuggestionService. Active by default out of the box: InputTelemetryService starts the UIA context observer whenever typoAutoCorrectEnabled !== false (even when inputTelemetryEnabled is false), and skips persisting input_activity rows to SQLite unless inputTelemetryEnabled === true; in Settings → Input (InputConsentPanel.tsx), the typoAutoCorrect switch works independently of telemetry consent. 2-Stage Hybrid Auto-Correction (2026-10-27..28): bootstrap.ts calls suggestion.ensureWarmForTyping(context) on every typed-context (keeping gemma4:e4b warm even when suggestionEnabled is off) and runs the full 2-stage typo.handleTypingContext(context) in-place: Stage 1 (Instant <1 ms deterministic rule engine at 25 ms after word boundary / 650 ms at token tail) immediately converts known Han/Eng layout errors and dictionary misspellings, while Stage 2 (Background Gemma Pass at 350 ms after word boundary for suspicious tokens or 600 ms idle for general sentences, anchored to the trailing 2 words via trailingWordOnly: true) queries the warm local LLM (gemma4:e4b) with 2-set keyboard hints (buildModelTypoHints in [한영 자판 변환 참고]) and validates cross-script layout+typo fixes (isPlausibleTypoFix + validateModelCorrections, e.g. ㅗ디ㅐ→hello, ㅡㅁ가둥→markdown, 네갸ㅜㅎ에서→spring에서, dkssudgktpd→안녕하세요, 회으에서→회의에서, 아ㄴ녀ㅇ→안녕) before handing off to SuggestionService via 'settled'. Ultra-fast atomic replacement & delta-rebasing (2026-10-28): TextInsertService.replaceBeforeCaret (method === 'keyboard') executes a single atomic SendInput call via sendWin32UnicodeReplacement (utils/win32-unicode-input.ts) packing (optional VK_SPACE) + N × VK_BACK + M × KEYEVENTF_UNICODE with 0 ms JS sleep (native batch ordering only; concurrent input still requires burst deferral), and TypoCorrectionService uses rebaseTypoReplacementPlan (packages/core/src/typo-correction.ts) to adjust deleteCount and append any newly typed suffix characters (<= 12 chars) when the user keeps typing during the 40 ms UIA verification or LLM check. Terminal, Vim & xterm.js support (2026-10-27..31): TypoCorrectionService keeps layoutOnly: true active inside TERMINAL_APPS (WindowsTerminal, wt, pwsh, cmd, Kitedesk, wezterm, alacritty, kitty, ghostty, putty, MobaXterm, nvim-qt, neovide, gvim, etc.) and developer/xterm.js surfaces (isTerminalOrDeveloperSurface, isXtermHelperControl) while keeping spelling/model corrections off. 2026-10-31 terminal character-swallow fix (v1.15.1): removed per-keydown isForegroundKoreanIme (SendMessageTimeoutW(WM_IME_CONTROL)) from InputTelemetryService._handleKeyDown (syncing IME once at idle snapshot via TerminalKeystrokeTracker.syncKoreanIme), added an xterm-helper-textarea fast path in sidecar/uia_bridge.py that skips GetTopLevelControl() and TextPattern / TextEditPattern.GetActiveComposition() calls (which stalled Chromium's renderer up to 1,500 ms and stuck comp=True), suppressed 40 ms boundary snapshots on normal Korean words (_shouldRunFastTerminalBoundarySnapshot) and 800 ms active-typing polls on terminals, and blocked TypoCorrectionService from touching trailing composed Hangul syllables (/[가-힣]$/u) in terminals before Space is pressed. TerminalKeystrokeTracker (packages/core/src/terminal-prompt.ts) + extractTerminalActiveContext + isForegroundKoreanIme (utils/win32-foreground.ts, checking IMC_GETCONVERSIONMODE IME_CMODE_NATIVE on Windows TSF/Chromium) reconstruct the active line across (a) UIA screen-buffer terminals (WindowsTerminal/pwsh/cmd, restoring trailing spaces stripped by grid buffers and reading the bottom viewport via GetVisibleRanges() / [-MAX_TEXT_CHARS:] and ExpandToEnclosingUnit(..., waitTime=0) in sidecar/uia_bridge.py), (b) xterm.js hidden <textarea class="xterm-helper-textarea" aria-label="Terminal input"> controls in Agent Switchboard, VS Code, Cursor, Hyper, and Tabby (reconcileWithXtermValue, preventing the infinite re-correction loop caused by xterm.js keeping .value until Enter/Ctrl+C), and (c) canvas/remote terminals with len=0 (Kitedesk/WezTerm/Alacritty), tracking Vim modes (insert, normal, command). In Vim Insert mode, converts EN↔KO (dkssudgktpdy →안녕하세요 , ㅗ디ㅣㅐ →hello ) and strips leading mode-entry keys (i/I/a/A/o/O/s/S) from raw keystroke buffers; in Vim Command-line mode, converts Hangul ex-commands (:ㅈㅂ→:wq, :ㅈㅂ!→:wq!, :ㅈ→:w, :ㅂ→:q, :ㅂ!→:q!, :ㅌ→:x, :ㅍ네→:vsp, :네→:sp, :ㅜㅐㅗ→:noh) while preserving :; in Vim Normal mode (vimMode === 'normal'), suppresses all conversions so normal-mode motions (hjkl, dd, ciw) are never touched; in shell prompts, converts CLI/code tokens (퍄ㅡ →vim , ㅜ퍄ㅡ →nvim , ㅎ샤 →git , ㅣㄴ →ls , 채ㅜㄴㅅ →const ). Layout conversions fire after 25 ms idle on word boundaries (Space/punctuation, with a 140 ms quick snapshot timer), 135 ms idle in Vim command mode (:ㅈㅂ), or 650 ms idle at token tail (with a 680 ms quick snapshot timer so compound words like some→sometime/somebody are never split mid-word), work even when Chromium/Discord reports isComposing=true (passing commitComposition: true only when the character before the caret is a composed Hangul syllable /[가-힣]$/u, never on single compatibility jamo [ㄱ-ㅎㅏ-ㅣ]), ignore synthetic KEYEVENTF_UNICODE events (keycode=0) while restricting _programmaticKeys to synthetic Backspace/Space/Ctrl+V, preserve keys typed during async replacement via baseLength in TerminalKeystrokeTracker.applyReplacement, retain _lastCheckedRegion on successful _apply (and only reset on cancellation/failure) while programmatic keys no longer advance _lastKeyAt, guard _watchRevert against false-positive revert blacklisting across new sentences/cleared fields (and never blacklist loose-jamo strings), preserve caretReliable=true after space-terminated replacements (isEditAtTextEnd strips trailing whitespace), and reset UiaContextService._failureStreak on successful UIA snapshots (BACKOFF_STEPS_MS capped at 2.5 s instead of sticky 30 s). Layer 1 (packages/core/src/typo-correction.ts + packages/core/src/araso-layout.ts): 2-set Korean layout mistypes both ways (dkssudgktpdy→안녕하세요, ㅗ디ㅣㅐ→hello, 재깅→world, ㅗ디ㅣㅐ 재깅→hello world, ㅗ디ㅣㅐ재깅→hello world, 내ㅡㄷ샤ㅡㄷ/some샤ㅡㄷ→sometime, ㅎ샤ㅗㅕㅠ→github, ㅎ샤ㅗㅕㅠ에→github에, ㅠㅕㅎ가→bug가, ㄴ가→s가, ㄴ은→s은, 메ㅔ을→app을, ㅖㅇㄹ파일→PDF파일, ㅁㅔㅑ키→API키), Hangul-typed English contractions & fuzzy 1-edit typos (ㅊ무ㅅ/ㅊ무'ㅅ/ㅊ무ㅜ'ㅅ→can't, ㅊ무'ㅅ는→can't는, 애ㅜ'ㅅ→don't, ㅑ'ㅡ/ㅑㅡ→I'm, ㅑㅅ'ㄴ→it's, ㅗ디ㅐ/ㅗ디ㅣㅐㅐ→hello, ㅔㅐㄴ냐ㅠㅣ→possible, 세갸ㅜㅎ→spring, ㅡㅁ가애주→markdown, help[→help, my self→myself), standalone sentence-initial ㅑ →I (ㅑ ㅊ무'ㅅ →I can't ), Araso batchim-agreement & rare-stem Hangul-typed acronyms (츤를→CMS를, 먀를→AI를, 틔을→XML을, 뗘를→EU를, 넴를→SPA를, 젳를→VPC를, 눈를→SNS를 vs 눈을 kept, 운를→DNS를 vs 운이 kept, 긴를→RLS를, 쓰은→TM은 vs 쓰는 kept, 꾸은→RN은 vs 꾸는 kept), 2-letter loose-jamo English words & acronyms (ㅑㄹ→if, ㅑㅅ→it, ㅑㅜ→in, ㅑㄴ→is, ㅐㄹ→of, ㅐㅜ→on, ㅣㄴ→ls, ㅎ샤→git, ㅔㅊ/ㅖㅊ→PC, ㅐㄴ/ㅒㄴ→OS, ㅔㄱ/ㅖㄱ→PR, ㅊㅑ→CI, ㅍㅡ→VM, ㅔㅡ→PM, ㅔㅐ→PO, ㅗㄱ→HR, ㅑㄱ→IR, ㅇㅡ→DM, ㅡㅇ→MD, ㅔㅇ→PD, ㅍㅔ→VP, ㅠㅡ→BM, ㅆㅍ→TV, ㅑㅖ→IP, plus Korean-prefix + loose-jamo compounds 내ㅔㅊ→내PC, 회사ㅔㅊ에서→회사PC에서, 한글로ㅔㅊ라고→한글로PC라고, 윈도우ㅐㄴ→윈도우OS, 새ㅁㅔㅑ를→새API를), contextual ㅑ→I / ㅁ→a and 새→to / 애→do / 무→an / 뭉→and / 랙→for / 좨→who / 양→did / 햇→got / 둥→end + lookahead phrase agreement (새 ㅠㄷ →to be , 새 솓 →to the , 새 애 this →to do this , 랙 솓 →for the , 뭉 솓 →and the , I want 새 애 this→I want to do this, into english 새 애→into english to do while preserving 오늘 새 API를 만들었어 and 이거 commit 해) plus 소두→then / 조두→when / 견→run / 재가→work / 쟈소→with / 소드→them / 소무→than / 미내→also / 소됴→they / 생묘→today / 갸홋→right, ~1,500+ common & extended English stems (EXTENDED_ENGLISH_STEMS) with inflectional (-s/-es/-ed/-ing/-ly/-er/-est) and derivational morphology (matchesInflectedOrDerivedEnglish: -tion/-sion/-ation/-ment/-ness/-able/-ible/-ful/-less/-ive/-ous/-al/-ic/-ize and prefixes un-/re-/pre-/non-/sub-/over-/under-/auto-/multi-/micro-/inter-, e.g. ㅏㅕㅠㄷ구ㄷㅅㄷㄴ→kubernetes, ㅡㅑㅊ개낸ㅅ→microsoft), partially-converted token recomposition (안녕gktpdy→안녕하세요, 안녀d→안녕, github에tj→github에서, hellㅐ→hello, hello재깅→hello world), mixed English/Acronym/CamelCase + Korean words/particles with batchim agreement (githubdp→github에, PDFvkdlf→PDF파일, Reactzjavhsjsxm→React컴포넌트, VSCodetjfwjd→VSCode설정, dhsmfdmsgoogledptj→오늘은google에서, 1:1ansdml→1:1문의, TMdms→TM은 vs TMsms→쓰는), lingering Shift on double consonants (djEJs→어떤, dlTDj→있어, qkRNjTdj→바꿨어), chat jamo (ㅋㅋ/ㅎㅎ/ㅠㅠ/ㅜㅜ), number+counter conversion (3tl→3시), sentence-level Korean context propagation (Stay hungry, stay foolishsms wkqtmdml akfdlek.→Stay hungry, stay foolish는 잡스의 말이다., dlrj go→이거 해), URL/path/email run protection (isUrlOrPathRun), always-wrong spellings (됬→됐, ㄹ께→ㄹ게, 몇일→며칠, teh→the, dont→don't …), and 1-edit near-misses of the user's frequent English words. Layer 2: warm suggestion model check (buildTypoCheckPrompt + buildModelTypoHints, re-validated by validateModelCorrections + isPlausibleTypoFix). Evidence: packages/core/__tests__/typo-correction.test.ts 53 tests, terminal-prompt.test.ts 13 tests, suggestion-continuation.test.ts 26 tests, desktop TypoCorrectionService.test.ts 29 tests, input-flow-services.test.ts 14 tests, suggestion-typo-correction.test.ts 7 tests, plus live Windows GUI E2E tests (E2ETestBox.exe 8/8 + neovide.exe Terminal & Vim 6/6 on WinSta0\Default) (GAP-INPUT-08). 2026-10-30 load-safety fix: validate keyboard revision, native input token and foreground again when the insertion queue drains; register synthetic keys only immediately before injection; skip automatic Windows corrections without atomic input support. Partial/unknown SendInput results throw without replay. Automated regressions pass; reported missing Hangul under real CPU/model load is not yet reproduced/verified (GAP-INPUT-11). Deterministic A/B evidence (2026-10-30): Only the three changed services come from HEAD f49acf0; dependencies/test doubles/tests stay current in both runs. HEAD 1/17 vs current 17/17 comprises three independent delayed-queue text-corruption cases, 13 snapshot/metadata/coalescing contracts, and one unchanged-input control that passes in both. This is not a full historical-release A/B. scripts/verification/typing-race-proof.mjs; test-results/typing-load/race-proof-2026-10-30T00-50-32-284Z/report.json. 2026-11-02: installed 1.16.0 native Korean IME passed 7/7 under bounded 100% CPU and 7/7 with direct gemma4:e4b cold-load overlap. The Latin-layout control reproduced next-key/backspace interleaving, so boundary snapshots now wait 140 ms and short-idle partial chains are deferred; after a 650 ms pause the current trailing correction region (up to the core 120-character region) is replaced together. Desktop 2,773 tests/build PASS. Updated-app Latin GUI and physical/Chromium/terminal proof remain open (GAP-INPUT-11). Evidence.
INPUT-20 Context-aware suggestion prompt & graph retrieval [~] [-] [-] [-] 2026-10-03; upgraded 2026-10-28. Prompt (buildSuggestionPrompt) splits the prefix with core writing-context.ts into earlier text / last sentence / unfinished current sentence and states the mode (finish this sentence vs. write the next one), language and Korean register (formal/polite/plain/casual); adds the text after the caret (no overlap). Graph (PersonalGraphService): indexText now incrementally creates shares_terms edges against recent existing nodes (not only within the same call) and links consecutive short messages within the same app (_lastIndexedByScope, 10-min window) via follows edges so single-sentence chat messages form a sequence chain; retrieveContext accepts recentWrites (from RecentWritesBuffer) to recover followUps even after Enter clears a chat box, augments exact substring continuations (continuationsFrom) with particle-invariant stemmed continuations (stemContinuationsFrom, e.g. "오늘 회의록을 정리해서" → "내일 오전까지 공유드리겠습니다" from "회의록 정리해서 내일 오전까지 공유드리겠습니다"), and expands 2-hop graph paths (rankTwoHopGraphTargets across shares_terms→follows and follows→follows). Evidence: personal-graph-context.test.ts (real SQLite) 7/7, packages/core/__tests__/personal-graph.test.ts 19/19, packages/core/__tests__/personal-ngram.test.ts 9/9.
INPUT-21 Prefill continuation + document/conversation context for suggestions [~] [-] [-] [-] 2026-10-03. For models whose chat template we know (gemma 4 <|turn>, gemma 2/3 <start_of_turn>; core chat-template.ts), the suggestion request is sent raw with the user's own unfinished sentence (or last sentence) pre-filled into the model turn, so the model literally continues the text: mid-word joins (회의실+에서) and spacing come out exact (extractContinuation keeps joint attach/separate; trailing space is not pre-filled — it destabilised output — and an attach after a finished word is rejected). Ollama's chat-API prefill was measured and rejected (drops the leading space, closes the turn: 검토해 보시→부탁드립니다.). Other models keep the instruction prompt. Context added to both paths: whole text before the caret (TypingContext.documentBefore, ≤6000 chars; null for document-like/terminal/password) → opening lines outside the 1,200-char window and a fact list (names+titles, attendee lines, dates, times, amounts, versions, acronyms, quoted terms — analyzeDocumentContext); same-window messages the user just sent (chat boxes empty on send — RecentWritesBuffer, memory only, 30 min, cleared when input observation stops, terminals excluded); search-box detection (keywords, not sentences); plain -다 register split from casual -어; no emoji unless the user uses them. Output guards: first sentence/line only, stream cut as soon as it ends, restated-prefix stripping (also fixes the instruction path, where CI는 통과했고… was inserted after CI는), repeat-of-written-sentence drop, and unsupported-fact rejection (findUnsupportedFacts: dates/numbers/names+titles/lone Latin identifiers absent from document, window title, recent messages and personal hints) with one retry for the first candidate. Fill loop: temperature 1.0, prefill only (the 2026-10-17 change removed the alternating instruction prompt — see INPUT-22). Evidence: real gemma4:e4b probe tests/integration/suggestion-quality.ollama.test.ts (15 scenarios, opt-in D3RO_SUGGESTION_PROBE=1) — first-candidate warnings 0/15 vs 1/15 for the old path (old path invented a launch date instead of the 11월 15일 stated 1,200 chars earlier), mean ≈0.68 s both; core suggestion-continuation.test.ts 25, llm-prompts.test.ts +7. Not yet verified by typing in a real app (GAP-INPUT-09).
INPUT-22 Typo fixes proposed inside the suggestion popup + snappier suggestions [~] [-] [-] [-] 2026-10-17. While suggestions are on, typos are no longer replaced silently: TypoCorrectionService.propose() builds a plan without applying it and SuggestionService shows it in the popup as a 오타 고침 row (del struck-through original → ins fix) with the corrected tail prefixed to every candidate; the model continues from the corrected text. Accept = backspace the typo span + paste fix + candidate in one replaceBeforeCaret (if the caret char is still IME-composing, a real Space commits it first and is deleted too); if the text before the caret no longer ends with the typo span, only the candidate is inserted. Silent auto-correct (INPUT-19) still runs when suggestions are off. New core rule detectJamoNoise: a lone stray jamo inside/before Hangul syllables (그ㅓ런, ㅇ다른, 가쓴ㄷ거) or a lone jamo word mid-sentence is removed; ㅋㅋ/ㅠㅠ/trailing ㅎ/first-word ㅇ are kept. Model typo check runs before the first candidate only when hasSuspiciousTypo (stray jamo, rare syllable); otherwise once in the background after the first candidate (thorough, ≥2 words) and the row is attached late. Proposals also work in learning-excluded apps (agent windows) but not terminals/excluded apps. Responsiveness: minIntervalMs 5000→1200, per-minute cap 6→20 (config migration rev 7, IPC clamp 30), composing idle ×1.5→×1.2, settle snapshot fires right after the required idle (≈780 ms instead of 1 s), accept relaxes the interval so the next pause after Space suggests again, visible TTL 20→45 s. Fill loop no longer mixes instruction prompts (instruction-path gemma answered the user: "설명해 주실 수 있을까요?"). Round 2 (same day, live test: "panel should appear the moment I pause; it still closes mid-generation / flashes"): 15 of 18 first requests returned an empty model output — a prefill ending mid-sentence with no trailing space (…잠시 쉬면) makes gemma close its turn at once; each empty result closed the panel and the next snapshot regenerated the same text ~1 s later (flash loop). Fixes: empty raw output → one retry with a trailing space appended to the prefill (ModelRequest.spaced, output re-parsed with a leading space so the joint is separate; probe: 4/4 sentences continue; fills reuse the spaced variant); a prefix whose first request failed is not regenerated until the text changes (_failedPrefix); a lone trailing jamo is removed with the following space so the text still ends "word + space"; snapshot debounce 700→380 ms and a "생성 중" placeholder (presenceDelayMs 350, self-clears after presenceMaxMs 3 s or when typing resumes, only when a request will follow — warm model, budget, not dismissed, not a failed prefix) shown before generation starts; it is not counted as "visible" by the policy. Tests: suggestion-presence.test.ts 5. Recent same-window writes are observed even while IME-composing (Korean chat boxes are almost always composing, so sent messages were never remembered). Diagnosis (installed v1.9.0 log + history DB, 1,122 suggestions / 11 accepted): v1.9.0 predates INPUT-19/21 and cleared a visible popup once typedRecently expired 8 s after the last key — ≈ the time the 12-candidate fill takes ("closes by itself when all candidates are done"). Evidence: real gemma4:e4b — 안ㄴ뇽하세여.?→안녕하세요.? (1.1 s), 회으에서→회의에서, 괜찬으세요→괜찮으세요, ㅇ다른/그ㅓ런 (0.8 s); desktop suggestion-typo-correction.test.ts 7, TypoCorrectionService.test.ts +4, core typo-correction.test.ts +4; desktop 2556 pass (2 known caption-env failures). Real-app check pending (GAP-INPUT-10).

SHELL — Platform Shell, Settings, Onboarding, Support

ID Feature D W M B Anchors / notes
SHELL-01 Settings / preferences [x] [~] [x] [x] 2026-09-28 개편: 데스크톱 설정은 모달이 아니라 메인 창 전체 화면 라우트(settings) — 왼쪽 세로 탭 8개 + 본문 components/settings/SettingsRows(구획 h3 + 라벨·설명/컨트롤 행). 일반 탭에 키 바인딩 편집기(CAP-16). 이전 기록: Desktop tabbed modal; the General tab hosts the whole key-binding editor (CAP-16: global on/off switch + one KeyBindingField per action, grouped voice/window — SettingsModal.tsx:239), which is also the first settings entry point the command action ever had; web theme/i18n; mobile SettingsScreen
SHELL-02 Theme system (6 themes) [x] [x] [x] [-] theme.ts SSOT
SHELL-03 i18n (12 locales) [x] [~] [~] [-] @d3ro/i18n; all 12 catalogs (ko, en, ja, zh, zh-TW, es, fr, de, pt, ru, vi, th) are synchronized to 2,742 keys and guarded by i18n-locale-parity.test.ts (15/15 PASS), including the updater dialog keys added on 2026-11-04. Catalog parity is not surface exposure: web has no locale selector and mobile currently exposes ko/en only (GAP-I18N-03, SSOT O-012), so those surfaces remain [~].
SHELL-04 Onboarding / first-run [x] [ ] [~] [-] Desktop model bootstrap (onboarding.llmModelSize aligned to 4.0GB gemma4:e4b, GAP-UX-07 [x]); mobile audience/theme/locale. 2026-11-01: 신규 사용자 목적지를 persistence 전에 준비하도록 수정하고 Fold6에서 .2 fresh auth/signup/invite/small-screen 4/4 PASS (fold6-20261101-fix1); latest .3 first-run auth and retry fail at onboarding visibility (fold6-20261101-mtg13). Exact .3 APK now passes isolated API 34 emulator fresh auth 37 s / signup 50 s with restart persistence; personal Fold6 data was preserved and its separate failures remain open.
SHELL-05 Accessibility / reduced motion [~] [~] [~] [-] Desktop reduced-motion honored. Mobile source labels/roles and 48dp tests plus 2026-11-04 inspected-route UIAutomator/font-scale/rotation evidence are partial GREEN; the AdMob SDK descendant exposed one unlabeled clickable WebView. Manual TalkBack/focus order, reduced-motion/app-wide coverage, and iOS remain (evidence).
SHELL-06 System tray / background [x] [-] [-] [-] Desktop tray
SHELL-07 Auto-launch on login [x] [-] [-] [-] Desktop only
SHELL-08 Auto-update [x] [-] [!] [-] Desktop electron-updater; Forgejo canonical feed; channels + mandatory/major-vs-delta policy (release/update-policy.json); mobile store updates 2026-10-24 UI: updates are now visible in the app, not only through native dialogs. Settings → About opens with an Updates section (components/settings/UpdateSection.tsx): status, progress bar, last check, and Check / Download / Restart-to-install buttons. A title-bar chip (components/shell/UpdateChip.tsx) appears only while an update is available, downloading or ready, and opens that tab. IPC `appUpdate:getStatus
SHELL-09 Support / diagnostics [x] [ ] [ ] [x] Desktop SupportModal (support-handlers.ts strict-typed, GAP-REL-15 [x]); admin /support fail-closed content_reports & support console (GAP-SHELL-01 [x]).
SHELL-10 Download center / releases [-] [x] [-] [x] Web /download, admin /releases, Forgejo
SHELL-11 Landing site / legal pages [-] [-] [-] [-] site/ — the only landing/download/legal/invite/assetlinks surface (Wave 3, 2026-09-26: copies in api-server wwwroot and apps/web removed; 404.html; hero loops a copy of the app recording capsule). Prices/URLs from packages/core plan-catalog/web-urls — see design.md. 2026-10-28 designpaca audit: flat header (Header.tsx blur removed), WCAG AA contrast verified across all 14 routes (ko..vi, privacy, terms, delete-account, accept-invite), legal.css explicit font-weight: 500/600 (no >=700 fallback), accept-invite polite Korean register (해요체) and --faint/--accent AA contrast.
SHELL-12 Notifications (in-app / desktop) [x] [ ] [x] [x] Desktop events; mobile push
SHELL-13 Desktop/mobile information architecture (2026-09-28) [x] [-] [x] [-] 1급 홈·기록·회의 + 2급 도구. Desktop: renderer/navigation/app-navigation.tsx, AppSidebar, status bar removed. Mobile: visible Home/Records/Meetings/More tabs, hidden Record route, floating three-action Record FAB, More tool/account grouping; full Jest and the canonical 320dp dark/411dp light runtime journeys are GREEN (11 GAP-UX-04). DashScreen/WorksHubScreen were removed; WorksHub-only local filter/pinning was intentionally retired while every destination remains in tabs/More and user content is untouched. Decision SSOT: root design.md.
SHELL-14 Home ("오늘") [x] [-] [x] [-] Desktop pages/HomePage.tsx: recording key, starts, blockers, summary, recent 6/4, file transcription, bounded usage. Mobile HomeScreen: today summary, recent records 5/meetings 3, unified-search and plan entry; canonical 320dp dark and 411dp light runtime journeys GREEN.
SHELL-15 Design system v4 "조용한 계기" [x] [x] [~] [-] packages/ui/src/theme-raw.ts raw-value SSOT (--d3-text-inverse mapped to accent.ink); flat surfaces, three text levels with AA contrast, accent.solid/ink, InstrumentPanel/SegmentControl aligned with v4. 2026-10-28 designpaca audit: apps/web, apps/admin, apps/desktop/src/renderer (main 16 routes/tabs + 6 popups), site/ (14 routes), and apps/mobile-rn audited and fixed (npm run check:design 0 violations; Playwright headless Chromium 36/36 routes PASS for contrast, <11px floor, <=600 weight cap, zero blur, single h1, labeled controls, and zero horizontal overflow across 320–1440px). packages/ui-native mirrors dark/light semantics and exposes 48dp control/numeral/motion tokens. 2026-11-04 Android follow-up: inspected app-owned Home/Login actions were labeled and >=48dp; API 34 Login was reachable at font scale 2.0 in portrait/landscape with scroll. Mobile stays [~] because this is automated route-level evidence, while physical TalkBack/focus order, an SDK-owned unlabeled AdMob WebView descendant, app-wide coverage, and iOS remain (evidence).

INFRA — Build, CI, Release, Quality

ID Feature Status Anchors
INFRA-01 Monorepo + npm workspaces [x] package.json; unused turbo.json and pnpm-workspace.yaml removed in Wave 4
INFRA-14 Unified entitlement resolver (@d3ro/core/entitlement) [x] EntitlementSnapshot + resolveEntitlement map Supabase/desktop-license/.NET sources to one contract; adopted across desktop (LicenseService), web (apps/web/src/lib/entitlement.ts), and mobile (apps/mobile-rn/src/lib/entitlement-context.tsx) (11 GAP-ID-01/02 [x]).
INFRA-02 Shared packages [x] packages/*
INFRA-03 Desktop build + signed packaging [x] electron-builder.yml, GitLab package-windows/macos
INFRA-04 Mobile CI (debug/E2E/release) [~] .forgejo/workflows/ci.yml, .gitlab-ci.yml, root test:mobile → mobile test:ci (jest --runInBand); production AAB remains external
INFRA-05 .NET API tests [x] apps/api-server.Tests
INFRA-06 Edge function tests (Deno) [x] server/supabase/functions/**/*.test.ts (root tests/ was not an Edge test home and was removed 2026-11-04)
INFRA-07 E2E desktop (playwright) [x] apps/desktop/tests/e2e (5 protected journeys + shared support/electron-app.ts fixture), apps/desktop/playwright.config.ts. 2026-11-04: red-team cycle specs and 4 stale feature specs deleted, duplicated launch code consolidated. Policy: docs/tdd-red/06-repo-application.md.
INFRA-08 Web tests (vitest + playwright) [x] apps/web/tests (vitest pure-logic contracts, now part of root npm test); apps/web/e2e (playwright; smoke.spec.ts owns the unauthenticated surface contract — fail-closed redirects + /download handoff). Web declares its own @playwright/test/vitest devDeps instead of using the desktop workspace's hoisted binaries.
INFRA-09 E2E mobile (Maestro + instrumentation) [~] .maestro/, androidTest; emulator API 35 gate in CI. 2026-11-04 current-source dual-ABI E2E APK verifier + state-preserving adb install -r/cold launch passed on Fold6 Android 16 and API 34 AVD; physical Notifications status/privacy/recent flow passed 1/1 without a permission command, and automated UIA/font-scale/rotation checks are recorded. Manual TalkBack, permission transition/real FCM receipt, full journey, iOS, and production AAB remain (evidence).
INFRA-10 Secret scanning / release boundaries [x] scripts/ci/check-no-hardcoded-secrets.mjs, verify-mobile-release-*.mjs
INFRA-11 Docker + NAS deploy [x] docker-compose.nas.yml, scripts/deploy-nas.*
INFRA-12 Cloudflare edge + tunnel [x] server/cloudflare-worker, Cloudflare Tunnel kd-nas
INFRA-13 Site deploy (Cloudflare Pages) [x] .forgejo/workflows/deploy-site.yml; the unused GitHub Pages workflow was removed in Wave 3
INFRA-17 Test hygiene gate (TDD policy) [x] scripts/ci/check-test-hygiene.mjs (npm run test:hygiene + :test), wired into .forgejo/workflows/ci.yml quality and .gitlab-ci.yml validate. Rules: .only, reason-less .skip(), trivial assertions, isVisible()-gated assertions, machine paths, waitForTimeout, .e2e.test.ts, e2e/ 밖의 *.spec.ts, assertion-free files. Applied policy + deletion/consolidation log: docs/tdd-red/06-repo-application.md.
INFRA-15 Update & release system [x] Canonical Forgejo feed + channels/policy (release/update-policy.json, src/main/update-policy.ts), canonical publisher scripts/ci/publish-forgejo-release.mjs, legacy GitLab mirror; npm run release:metadata:test. v1.1.0 was published to Forgejo on 2026-09-15; product version moved to 1.2.0 as a forward-fix with CI-only publication, a same-version re-release guard, and download centers that link the feed instead of repository paths. 1.3.0 (2026-09-18) carries the local-STT fixes; Windows publication still needs the CI signing secrets (11 GAP-REL-02).
INFRA-16 Desktop STT engine packaging [x] apps/desktop/scripts/setup-sidecar.mjs + build-sidecar.mjs, electron-builder.yml extraResources (sidecar-dist/sidecar → resources/sidecar, resources/ffmpeg), and scripts/ci/verify-sidecar-bundle.mjs run in package-windows/package-macos before electron-builder. Verified on the real bundle: sidecar.exe + _internal including faster_whisper/assets/silero_vad_v6.onnx, plus a packaged-engine transcription round-trip on GPU.
INFRA-17 서명 없는 배포 채널 (portable + Scoop) [x] scripts/ci/build-portable.mjs (95MiB 7z 분할 볼륨 + Scoop 매니페스트), scripts/ci/publish-portable-release.mjs, scripts/local/install-d3ro-voice.ps1, bucket/ 버킷, .forgejo/workflows/portable.yml; 7z 분할 볼륨(Scoop, 162MiB) + zip 분할 부품(수동 설치, 243MiB, 7-Zip 불필요); updater feed와 분리. 2026-09-18 portable-1.3.1 게시 + 실제 설치 검증.
INFRA-18 로컬 런타임 온디맨드 설치 [x] RuntimeProvisioner(부품 다운로드 + SHA-256 검증 + tar 해제, %APPDATA%/d3ro-voice/runtime), POST runtime:ensure / runtime:progress IPC, 설정 > STT 상태/내려받기 UI. 설치본에서 엔진/ffmpeg를 분리해 189MB → 90.6MiB, 업데이트 피드 게시 복구. 검증은 전부 디스크에 기록된 파일 기준이며(부품 크기·해시 → 결합본 크기·해시), 부품 다운로드는 최대 3회 재시도한다. 2026-09-18 실제 feed 통합 검증(엔진 94.4MiB/18초, ffmpeg 21.7MiB/5초).
INFRA-19 네이티브 ABI + updater 설정 게이트 [x] scripts/ci/verify-native-abi.mjs는 패키징된 better_sqlite3.node를 host Node가 로드하면 잘못된 ABI로 실패시키고, 대상별 실제 uiohook-napi/prebuilds/<platform>-<arch>/uiohook-napi.node도 필수로 검사한다. --self-test가 N-API prebuild 누락/존재 fixture를 고정하며 상대·절대 --dir 모두 같은 package를 검증한다. fix-native-abi.mjs는 better-sqlite3 주입만 담당한다. GitLab desktop jobs와 Forgejo release/portable scripts가 검증기를 실행하고, installer/portable build는 resources/app-update.yml도 필수로 확인한다. 2026-09-18의 잘못된 Node ABI 즉시 종료와 updater 설정 누락 사고를 함께 방지한다.
INFRA-20 Desktop typing race and load verification [~] typing-race-proof.mjs deterministic baseline 2/17 vs current 17/17. 2026-11-02: installed 1.16.0 native Korean IME passed 7/7 under bounded 100% CPU and 7/7 with direct gemma4:e4b cold-load overlap. The Latin-layout control reproduced next-key/backspace interleaving; source now defers bursts for 140 ms and batches the current trailing region after a pause, with desktop 2,773 tests/build GREEN. The fix is published in Windows v1.16.1; installed-app Latin GUI, physical keyboard and Chromium/terminal runs remain. See evidence, GAP-INPUT-11.

Coverage summary (by surface)

Surface [x] [~] [ ] Notable strength Notable weakness
Desktop ~40 6 ~8 Local AI pipeline, meetings, RAG, conversation, key bindings Ads stubs, no team admin, no email account; custom-instruction/chain LLM path (AI-04..07) fixed 2026-09-21 but unverified in a running app; cross-device sync rewritten 2026-09-27, production migration deploy pending (GAP-SYNC-05)
Web ~22 6 ~14 Server-shared data UX, billing, meetings, teams No local AI, limited knowledge upload/search
Mobile ~40 12 ~18 Cloud + native recording, portability, admin, IAP/ads External store/console gates, a11y, deep E2E pending
Backend ~45 6 ~4 RLS, Edge functions, billing, fail-closed AI Payple webhook signature, some external provider keys