diff --git a/CHANGELOG.md b/CHANGELOG.md index 3b3495c..9af01b1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Additional Whisper model management UI - Plugin system for custom pipelines +## [1.13.0] - 2026-10-29 + +> Published from an annotated tag. Installer and update metadata are served by the +> canonical Forgejo feed; no binaries are committed to this repository. + +### Added +- **Graph-relational extreme personalization & sentence suggestion upgrades (그래프 관계성 기반 극한 개인화 및 문장 제시 고도화).** + - Chains consecutive short messages within the same app (`_lastIndexedByScope`, 10-minute session window) via `follows` edges and restores post-send chat context from `recentWrites` when Enter clears the input box. + - Adds particle-invariant stemmed continuations (`stemContinuationsFrom`) so phrases like `"회의록 정리해서 내일 오전까지 공유드리겠습니다"` continue seamlessly when you type `"오늘 회의록을 정리해서"`, alongside 2-hop graph path scoring (`rankTwoHopGraphTargets` across `shares_terms → follows` and `follows → follows`). + - Weights frequent `personalPhrases` (`count >= 2`) and adds a Korean stemmed-word bigram fallback (`_stemBigram`) in `PersonalNgramModel`, while reranking generated LLM candidates by `scoreCandidatePersonalization` (graph overlap + Korean register match) without reducing suggestion frequency. +- **Stronger English detection via derivational morphology & phrase lookahead (파생 형태론 및 전방 구문 호응 기반 영어 감지 강화).** + - Expands English stem lexicon (`EXTENDED_ENGLISH_STEMS`: `kubernetes`, `microsoft`, `supabase`, `postgres`, `typescript`, `refactor`, `endpoint`, `workflow`, `dashboard`, etc.) with productive derivational prefixes (`un-`, `re-`, `pre-`, `non-`, `sub-`, `over-`, `under-`, `auto-`, `multi-`, `micro-`, `inter-`) and suffixes (`-tion`, `-sion`, `-ation`, `-ment`, `-ness`, `-able`, `-ible`, `-ful`, `-less`, `-ive`, `-ous`, `-al`, `-ic`, `-ize`, `-or`, `-ist`, `-ism`). + - Detects clause-initial and post-Korean Hangul-typed English function words via lookahead (`"새 ㅠㄷ "` → `"to be "`, `"새 솓 "` → `"to the "`, `"새 애 this "` → `"to do this "`, `"랙 솓 "` → `"for the "`, `"뭉 솓 "` → `"and the "`) while keeping Korean determiners intact (`"오늘 새 API를 만들었어 "`). + +### Changed +- **Zero-sleep single-syscall atomic replacement & in-flight delta rebasing (단일 `SendInput` 원자적 치환 및 타이핑 중 델타 리베이싱).** + - Packs `(optional VK_SPACE) + N × VK_BACK + M × KEYEVENTF_UNICODE` into a single Windows `SendInput` call (`sendWin32UnicodeReplacement`, `< 0.5 ms`, `0 ms` sleep) so physical keystrokes never interleave with auto-correction. + - Rebases replacement plans (`rebaseTypoReplacementPlan`) when you keep typing during the 40 ms UIA verification or background LLM check, preserving newly typed suffix characters (`<= 12` chars) in one atomic swap. + ## [1.12.0] - 2026-10-28 > Published from an annotated tag. Installer and update metadata are served by the diff --git a/apps/desktop/src/main/bootstrap.ts b/apps/desktop/src/main/bootstrap.ts index 4ab0f67..4a1f0a4 100644 --- a/apps/desktop/src/main/bootstrap.ts +++ b/apps/desktop/src/main/bootstrap.ts @@ -249,12 +249,13 @@ async function initInputIntelligence(): Promise { typo.on('corrected', () => { if (suggestion.isPresentationActive) suggestion.dismiss('stale') }) - // 인플레이스 자동 교정(handleTypingContext)은 타이핑과 절대 경합하지 않도록 - // 0.1ms 결정적 한영·영문오타 엔진(layoutOnly: true, 커서 직전 어절만 교체)만 즉시 실행하고, - // 문장 단위 로컬 LLM(Gemma) 오타 교정은 키보드를 건드리지 않는 제안 팝업(propose)에서만 제시한다. + // 인플레이스 자동 교정(handleTypingContext)은 커서 직전 꼬리 어절만 원자적(단일 Win32 SendInput)으로 교체한다: + // 1단계(25ms 단어 경계 / 650ms 어절 끝): 결정적 한영·영문오타(Araso) 즉시 전환 + // 2단계(350ms 수상한 어절 경계 / 600ms 유휴): 철자 사전·개인 어휘·자모 잡음 및 로컬 LLM(Gemma) 꼬리 오타 보정 + // 이미 지나간 문장 앞쪽 오타는 키보드를 건드리지 않는 제안 팝업(propose)에서 제시한다. suggestion.setTypoProposer({ propose: (context, signal, options) => typo.propose(context, signal, options) }) telemetry.on('typed-context', (context) => { - const verdict = typo.handleTypingContext(context, { layoutOnly: true, trailingWordOnly: true }) + const verdict = typo.handleTypingContext(context, { trailingWordOnly: true }) if (verdict === 'pass') suggestion.handleTypingContext(context) }) telemetry.on('state-changed', (state) => diff --git a/apps/desktop/src/main/services/PersonalGraphService.ts b/apps/desktop/src/main/services/PersonalGraphService.ts index 4c821a5..ad084e8 100644 --- a/apps/desktop/src/main/services/PersonalGraphService.ts +++ b/apps/desktop/src/main/services/PersonalGraphService.ts @@ -16,8 +16,10 @@ import { continuationsFrom, extractTerms, jaccard, + rankTwoHopGraphTargets, scoreRelatedNodes, splitSentences, + stemContinuationsFrom, type GraphContext, type PersonalGraphQuery, type PersonalGraphStats, @@ -40,13 +42,20 @@ export type { PersonalGraphQuery, PersonalGraphStats } const MAINTENANCE_NODE_LIMIT = 300 /** 관계 조회에서 최근 노드를 훑는 수. */ const TERM_SCAN_LIMIT = 400 +/** 같은 앱에서 단문 메시지를 연속으로 보낼 때 follows 엣지로 잇는 최대 세션 간격 (10분). */ +const SESSION_FOLLOWS_WINDOW_MS = 10 * 60_000 +/** 새 문장을 넣을 때 즉시 shares_terms 엣지를 연결할 최근 노드 수. */ +const INCREMENTAL_SHARES_SCAN_LIMIT = 40 class PersonalGraphService { + /** 앱별 마지막 학습 문장 — 카카오톡·슬랙 등 단문 메시지 사이의 follows 사슬을 잇는다. */ + private readonly _lastIndexedByScope = new Map() + /** * 텍스트를 그래프에 반영한다. * - * 문장마다 노드를 upsert 하고, 연속한 문장 쌍을 follows 엣지로, 용어를 충분히 - * 공유하는 쌍을 shares_terms 엣지로 남긴다. + * 문장마다 노드를 upsert 하고, 연속한 문장 쌍(같은 앱의 직전 메시지 포함)을 follows 엣지로, + * 용어를 충분히 공유하는 쌍을 shares_terms 엣지로 남긴다. */ indexText( text: string, @@ -56,9 +65,17 @@ class PersonalGraphService { if (sentences.length === 0) return const at = meta.at ?? Date.now() + const scopeKey = meta.appName?.trim().toLowerCase() || null try { const db = getDatabase() + const recentExisting = db + .select({ phrase: personalPhrases.phrase, terms: personalPhrases.terms }) + .from(personalPhrases) + .orderBy(desc(personalPhrases.lastUsedAt)) + .limit(INCREMENTAL_SHARES_SCAN_LIMIT) + .all() + for (const sentence of sentences) { const terms = extractTerms(sentence) db.insert(personalPhrases) @@ -82,9 +99,36 @@ class PersonalGraphService { } }) .run() + + if (terms.length > 0) { + for (const existing of recentExisting) { + if (existing.phrase === sentence) continue + const existingTerms = parseTerms(existing.terms) + if (existingTerms.length === 0) continue + if (jaccard(terms, existingTerms) >= SHARES_TERMS_MIN_SIMILARITY) { + this._upsertEdge(existing.phrase, sentence, 'shares_terms') + } + } + } } - // follows 엣지 + // 같은 앱에서 직전 세션(10분 이내)에 쓴 문장과 첫 문장을 follows 로 연결 + if (scopeKey) { + const previousInScope = this._lastIndexedByScope.get(scopeKey) + if ( + previousInScope && + previousInScope.phrase !== sentences[0] && + Math.abs(at - previousInScope.at) <= SESSION_FOLLOWS_WINDOW_MS + ) { + this._upsertEdge(previousInScope.phrase, sentences[0], 'follows') + } + this._lastIndexedByScope.set(scopeKey, { + phrase: sentences[sentences.length - 1], + at + }) + } + + // follows 엣지 (같은 텍스트 내 인접 문장) for (const edge of buildSequenceEdges(sentences)) { this._upsertEdge(edge.from, edge.to, 'follows') } @@ -135,7 +179,11 @@ class PersonalGraphService { * 3) related — 지금 문장·직전 문장·앞 문맥의 용어를 IDF 로 가중해 고른 관련 문장과, * 그 문장들 뒤에 이어 쓴 문장(follows 1단계 확장). */ - retrieveContext(prefix: string, limit = 4, options: { appName?: string | null } = {}): GraphContext { + retrieveContext( + prefix: string, + limit = 4, + options: { appName?: string | null; recentWrites?: readonly string[] } = {} + ): GraphContext { const empty: GraphContext = { continuations: [], related: [], followUps: [] } const normalizedPrefix = prefix.replace(/\s+/gu, ' ').trim() if (normalizedPrefix.length < 2) return empty @@ -143,19 +191,26 @@ class PersonalGraphService { try { const db = getDatabase() const context = analyzeWritingContext(prefix) + const recentWriteSentences = + options.recentWrites && options.recentWrites.length > 0 + ? splitSentences(options.recentWrites[options.recentWrites.length - 1] ?? '') + : [] + const anchorLastSentence = + context.lastSentence || + (recentWriteSentences.length > 0 ? recentWriteSentences[recentWriteSentences.length - 1] : '') const inPrefix = (text: string): boolean => normalizedPrefix.includes(text) - // (1) 직전 문장 뒤에 이어 쓴 문장 - const followUps = this._followUpsAfter(context.lastSentence, context.currentSentence ? 2 : 3, inPrefix) + // (1) 직전 문장(또는 방금 보낸 채팅 메시지) 뒤에 이어 쓴 문장 + const followUps = this._followUpsAfter(anchorLastSentence, context.currentSentence ? 2 : 3, inPrefix) // (2) 꼬리 이어쓰기 — 지금 문장 안에서만 찾는다 (앞 문장 끝과 섞이지 않게) const tailSource = context.currentSentence || context.lastSentence const continuations = this._continuationsForTail(tailSource) - // (3) 관련 문장 — IDF 가중 용어 겹침 + follows 1단계 확장 + // (3) 관련 문장 — IDF 가중 용어 겹침 + 1-hop follows + 2-hop (shares_terms -> follows) 경로 확장 const queryTerms = buildQueryTerms([ { text: context.currentSentence, weight: 2 }, - { text: context.lastSentence, weight: 1.5 }, + { text: anchorLastSentence, weight: 1.5 }, { text: context.earlierText.slice(-200), weight: 1 } ]) const recentNodes = db @@ -185,22 +240,66 @@ class PersonalGraphService { if (related.length >= limit || related.includes(text) || followUps.includes(text) || inPrefix(text)) return related.push(text) } - // 관련 문장의 다음 문장이 관련 문장 자체보다 "이어질 말" 에 가깝다 — 상위 앵커 2개만 확장한다. - const anchors = scored.slice(0, 2).map((item) => item.text) - const hops = anchors.length + // 관련 문장의 다음 문장이 관련 문장 자체보다 "이어질 말" 에 가깝다 — 상위 앵커 확장 + 2-hop 경로 탐색 + const anchors = scored.slice(0, 3).map((item) => item.text) + const firstHopEdges = anchors.length ? db .select() .from(phraseEdges) - .where(and(inArray(phraseEdges.fromPhrase, anchors), eq(phraseEdges.kind, 'follows'))) + .where(or(inArray(phraseEdges.fromPhrase, anchors), inArray(phraseEdges.toPhrase, anchors))) .orderBy(desc(phraseEdges.weight)) - .limit(6) + .limit(12) .all() : [] + const directFollowHops = firstHopEdges.filter( + (edge) => edge.kind === 'follows' && anchors.includes(edge.fromPhrase) + ) + const normalizedFirstHop = firstHopEdges.flatMap((edge) => { + if (edge.kind === 'follows' && anchors.includes(edge.fromPhrase)) { + return [{ from: edge.fromPhrase, to: edge.toPhrase, kind: 'follows' as const, weight: edge.weight }] + } + if (edge.kind === 'shares_terms') { + if (anchors.includes(edge.fromPhrase)) { + return [{ from: edge.fromPhrase, to: edge.toPhrase, kind: 'shares_terms' as const, weight: edge.weight }] + } + if (anchors.includes(edge.toPhrase)) { + return [{ from: edge.toPhrase, to: edge.fromPhrase, kind: 'shares_terms' as const, weight: edge.weight }] + } + } + return [] + }) + const midNodes = [...new Set(normalizedFirstHop.map((edge) => edge.to))] + const secondHopFollows = midNodes.length + ? db + .select() + .from(phraseEdges) + .where(and(inArray(phraseEdges.fromPhrase, midNodes), eq(phraseEdges.kind, 'follows'))) + .orderBy(desc(phraseEdges.weight)) + .limit(12) + .all() + .map((edge) => ({ + from: edge.fromPhrase, + to: edge.toPhrase, + kind: 'follows' as const, + weight: edge.weight + })) + : [] + const twoHopTargets = rankTwoHopGraphTargets({ + firstHopEdges: normalizedFirstHop, + secondHopFollowEdges: secondHopFollows, + queryTerms, + exclude: (text) => inPrefix(text) || followUps.includes(text), + limit: 2 + }) + for (const item of scored) { push(item.text) - const hop = hops.find((edge) => edge.fromPhrase === item.text) + const hop = directFollowHops.find((edge) => edge.fromPhrase === item.text) if (hop) push(hop.toPhrase) } + for (const target of twoHopTargets) { + push(target) + } return { continuations, related, followUps } } catch (error) { logger.warn( @@ -305,7 +404,16 @@ class PersonalGraphService { ) if (found.length > 0) return found } - return [] + + // 정확한 문자열 꼬리가 어긋나도("회의록을 정리해서" vs "회의록 정리해서") 어간 열이 일치하면 이어쓰기를 뽑는다. + const recentPhrases = db + .select({ phrase: personalPhrases.phrase }) + .from(personalPhrases) + .orderBy(desc(personalPhrases.count), desc(personalPhrases.lastUsedAt)) + .limit(120) + .all() + .map((row) => row.phrase) + return stemContinuationsFrom(recentPhrases, normalized, 3) } /** 용어 공유 엣지를 백필한다 (기존 노드들 사이 관계 보강). */ @@ -469,6 +577,7 @@ class PersonalGraphService { clearAll(): void { try { + this._lastIndexedByScope.clear() const db = getDatabase() db.delete(phraseEdges).run() logger.info('개인 그래프 엣지 전체 삭제') diff --git a/apps/desktop/src/main/services/SuggestionService.ts b/apps/desktop/src/main/services/SuggestionService.ts index 9ed2a18..8e4e036 100644 --- a/apps/desktop/src/main/services/SuggestionService.ts +++ b/apps/desktop/src/main/services/SuggestionService.ts @@ -43,7 +43,7 @@ import { type SuggestionStep } from '@d3ro/core/input-intelligence' import { renderRawChatPrompt, rawChatStopSequences } from '@d3ro/core/chat-template' -import { buildQueryTerms, splitSentences } from '@d3ro/core/personal-graph' +import { buildQueryTerms, scoreCandidatePersonalization, splitSentences } from '@d3ro/core/personal-graph' import { copiesContext } from '@d3ro/core/suggestion-confidence' import { splitFirstWord } from '@d3ro/core/suggestion-text' import { predictPersonalContinuation } from '@d3ro/core/personal-ngram' @@ -1028,14 +1028,22 @@ export class SuggestionService extends EventEmitter { private _collectMemoryHints(prefix: string, appName: string | null): LocalMemoryContext { const telemetry = getInputTelemetryService() - const graphContext = getPersonalGraphService().retrieveContext(prefix, 4, { appName }) + const recentWrites = telemetry + .listRecentWrites({ + windowHandle: this._lastContext?.windowHandle ?? null, + appName, + windowTitle: this._lastContext?.windowTitle ?? null + }) + .map((entry) => entry.text) + const graphContext = getPersonalGraphService().retrieveContext(prefix, 4, { appName, recentWrites }) const phrases = telemetry.listPhrases(120) // 지금 글과 용어가 겹치는 문구만 — 문맥과 무관한 "자주 쓰는 말" 은 프롬프트를 흐린다. const writing = analyzeWritingContext(prefix) + const lastRecentWrite = recentWrites.length > 0 ? recentWrites[recentWrites.length - 1] ?? '' : '' const relevanceTerms = new Set( buildQueryTerms([ { text: writing.currentSentence, weight: 2 }, - { text: writing.lastSentence, weight: 1 } + { text: writing.lastSentence || lastRecentWrite, weight: 1 } ]).keys() ) const phraseHints = selectPhraseHints(phrases, prefix, 5, { appName, relevanceTerms }) @@ -1759,6 +1767,32 @@ export class SuggestionService extends EventEmitter { if (token !== this._generationToken || this._isDuplicateCandidate(candidate.text)) return false this._candidates.push({ ...candidate, rank: this._candidates.length }) + if (this._activeIndex === 0 && this._candidates.length >= 2) { + const personalContext = { + prefix, + continuationHints: memory.continuationHints, + followUpHints: memory.followUpHints, + relatedHints: memory.relatedHints, + phraseHints: memory.phraseHints + } + const newScore = scoreCandidatePersonalization(candidate.text, personalContext) + if (newScore >= 1.5) { + const personalItems = this._candidates.filter((item) => item.source === 'personal') + const modelItems = this._candidates + .filter((item) => item.source !== 'personal') + .map((item, idx) => ({ + item, + idx, + score: scoreCandidatePersonalization(item.text, personalContext) + })) + .sort((a, b) => (Math.abs(b.score - a.score) >= 0.8 ? b.score - a.score : a.idx - b.idx)) + .map((entry) => entry.item) + this._candidates = [...personalItems, ...modelItems].map((item, index) => ({ + ...item, + rank: index + })) + } + } logger.debug(`제안 후보 추가 ${this._candidates.length}/${SUGGESTION_DEFAULTS.maxCandidatesTotal}`) this._armVisibleTtl() this.emit('updated', this.getState()) diff --git a/apps/desktop/src/main/services/TextInsertService.ts b/apps/desktop/src/main/services/TextInsertService.ts index e014f9d..4e21daf 100644 --- a/apps/desktop/src/main/services/TextInsertService.ts +++ b/apps/desktop/src/main/services/TextInsertService.ts @@ -104,6 +104,24 @@ export class TextInsertService extends EventEmitter { options: { commitComposition?: boolean } = {} ): Promise { const run = this._tail.then(async () => { + if (method === 'keyboard' && this._keyboard.replaceBeforeCaret) { + const start = performance.now() + const handled = await this._keyboard.replaceBeforeCaret(deleteCount, text, options) + if (handled) { + this.emit('insert-started', { text, method: 'keyboard' }) + const result: InsertResult = { + success: true, + method: 'keyboard', + textLength: text.length, + durationMs: performance.now() - start + } + this.emit('insert-completed', { result }) + logger.info( + `Text replaced atomically (-${deleteCount} / +${text.length} chars, ${Math.round(result.durationMs)}ms, keyboard)` + ) + return result + } + } try { let count = deleteCount // 커서 앞 글자가 IME 조합 중이면 먼저 확정한다 (Space → 남은 공백까지 지운다). diff --git a/apps/desktop/src/main/services/TypoCorrectionService.ts b/apps/desktop/src/main/services/TypoCorrectionService.ts index b70a11b..db05dd2 100644 --- a/apps/desktop/src/main/services/TypoCorrectionService.ts +++ b/apps/desktop/src/main/services/TypoCorrectionService.ts @@ -32,6 +32,7 @@ import { mergeTypoCorrections, parseModelTypoPairs, planTypoReplacement, + rebaseTypoReplacementPlan, tokenizeWords, typoRegionStart, validateModelCorrections, @@ -123,7 +124,7 @@ const REVERT_LENGTH_TOLERANCE = 12 export class TypoCorrectionService extends EventEmitter { private _applying = false - private _check: { abort: AbortController; prefix: string; context: TypingContext } | null = null + private _check: { abort: AbortController; prefix: string; context: TypingContext; latestKeyAt: number } | null = null /** 마지막으로 결정적·모델 검사를 마친 구간(교정 전·후 모두 포함) — 같은 구간을 되풀이해 묻거나 연쇄 재교정하지 않는다. */ private _lastCheckedRegion = '' private _lastPostRegion = '' @@ -170,12 +171,29 @@ export class TypoCorrectionService extends EventEmitter { this._watchRevert(context) if (this._applying) return 'correcting' - // 진행 중 모델 검사 — 같은 접두면 계속 기다리고, 달라졌으면(더 쳤다) 버린다. + // 진행 중 모델 검사 — 같은 접두이거나 짧은 공백·구두점/완결 어절 꼬리만 덧붙었으면 리베이스 대기로 유지하고, + // 한창 글자를 치는 중이면(유휴 부족·조합 중) 검사를 버리고 타이핑을 방해하지 않는다. if (this._check) { if (this._check.prefix === context.prefix) { this._check.context = context + this._check.latestKeyAt = this._deps.lastKeyAt() return 'checking' } + if (context.prefix.startsWith(this._check.prefix)) { + const delta = context.prefix.slice(this._check.prefix.length) + const canRebaseDelta = + delta.length > 0 && + delta.length <= 6 && + (/^[ \t.,!?;:]{1,4}$/u.test(delta) || + (!context.isComposing && + context.idleMs >= LAYOUT_BOUNDARY_MIN_IDLE_MS && + /[ \t.,!?;:]$/u.test(delta))) + if (canRebaseDelta) { + this._check.context = context + this._check.latestKeyAt = this._deps.lastKeyAt() + return 'checking' + } + } this._cancelCheck() } @@ -296,7 +314,7 @@ export class TypoCorrectionService extends EventEmitter { this._markRegionChecked(region) const abort = new AbortController() - this._check = { abort, prefix, context } + this._check = { abort, prefix, context, latestKeyAt: keyAtDecision } void this._runModelCheck(model, context, abort, keyAtDecision, ignoredWords, vocabulary.known) return 'checking' } @@ -438,11 +456,20 @@ export class TypoCorrectionService extends EventEmitter { const tailCorrections = mergeTypoCorrections(fromModel, detectJamoNoise(prefix, { ignoredWords })).filter( (c) => prefix.length - c.start <= MAX_MODEL_DELETE_CHARS ) - const plan = planTypoReplacement(prefix, tailCorrections) - if (!plan || plan.deleteCount > MAX_MODEL_DELETE_CHARS) return - // 검사하는 사이 사용자가 더 쳤으면 접두가 달라져 위에서 취소됐다. 여기서 한 번 더 키를 본다. + const basePlan = planTypoReplacement(prefix, tailCorrections) + if (!basePlan) return + const activeCheck = this._check?.abort === abort ? this._check : null + const targetContext = activeCheck?.context ?? context + const targetKeyAt = activeCheck?.latestKeyAt ?? keyAtDecision + const rebasedPlan = rebaseTypoReplacementPlan( + basePlan, + prefix, + targetContext.prefix, + MAX_MODEL_DELETE_CHARS + ) + if (!rebasedPlan) return this._check = null - corrected = await this._apply(plan, context, keyAtDecision) + corrected = await this._apply(rebasedPlan, targetContext, targetKeyAt) } catch (error) { if (!abort.signal.aborted) { logger.debug(`모델 오타 검사 실패: ${error instanceof Error ? error.message : String(error)}`) diff --git a/apps/desktop/src/main/services/suggestion/PersonalNgramIndex.ts b/apps/desktop/src/main/services/suggestion/PersonalNgramIndex.ts index 90b5776..1d699df 100644 --- a/apps/desktop/src/main/services/suggestion/PersonalNgramIndex.ts +++ b/apps/desktop/src/main/services/suggestion/PersonalNgramIndex.ts @@ -9,7 +9,7 @@ import { PersonalNgramModel } from '@d3ro/core/personal-ngram' import { getDatabase } from '../../db' -import { typingSamples } from '../../db/schema' +import { personalPhrases, typingSamples } from '../../db/schema' import { getLogger } from '../LoggerService' const logger = getLogger('PersonalNgramIndex') @@ -37,7 +37,21 @@ export class PersonalNgramIndex { const startedAt = this._now() try { const model = new PersonalNgramModel() - for (const row of db.select({ text: typingSamples.text }).from(typingSamples).all()) model.addText(row.text) + for (const row of db.select({ text: typingSamples.text }).from(typingSamples).all()) { + model.addText(row.text) + } + try { + for (const row of db + .select({ phrase: personalPhrases.phrase, count: personalPhrases.count }) + .from(personalPhrases) + .all()) { + if (row.count >= 2 && typeof row.phrase === 'string') { + model.addText(row.phrase, Math.min(row.count - 1, 3)) + } + } + } catch { + // personal_phrases 테이블이 없는 테스트 목(mock) DB 에서는 typing_samples 만으로 진행한다. + } this._model = model this._builtFor = db logger.info(`개인 n-gram 생성: 표본 ${model.textCount}개 (${this._now() - startedAt}ms)`) diff --git a/apps/desktop/src/main/services/text-insert/nut-keyboard-port.ts b/apps/desktop/src/main/services/text-insert/nut-keyboard-port.ts index 3d7ebe2..5539415 100644 --- a/apps/desktop/src/main/services/text-insert/nut-keyboard-port.ts +++ b/apps/desktop/src/main/services/text-insert/nut-keyboard-port.ts @@ -4,7 +4,7 @@ import type { Key as NutKey } from '@nut-tree-fork/nut-js' import { getLogger } from '../LoggerService' -import { sendWin32UnicodeText } from '../../utils/win32-unicode-input' +import { sendWin32UnicodeReplacement, sendWin32UnicodeText } from '../../utils/win32-unicode-input' import { D3ROError, ErrorCode } from '@d3ro/core/errors' const logger = getLogger('TextInsertService') @@ -22,6 +22,15 @@ export interface KeyboardPort { * 글자가 아니라 낱자가 지워져(여 → ㅇ) 지울 개수가 어긋난다. 호출자는 남은 공백까지 지운다 (선택). */ commitComposition?(): Promise + /** + * (선택적 IME 조합 확정 Space) + 커서 앞 백스페이스 + 텍스트 타이핑을 단일 원자적(atomic) 호출로 수행한다. + * 지원하고 성공했으면 true, 폴백이 필요하면 false 를 돌려준다. + */ + replaceBeforeCaret?( + deleteCount: number, + text: string, + options?: { commitComposition?: boolean } + ): Promise /** 네이티브 키 모듈을 미리 불러 둔다 — 첫 삽입이 로딩을 기다리지 않게 (선택) */ preload?(): Promise } @@ -112,6 +121,21 @@ export class NutKeyboardPort implements KeyboardPort { await nut.native.click(nut.Key.Space) } + async replaceBeforeCaret( + deleteCount: number, + text: string, + options: { commitComposition?: boolean } = {} + ): Promise { + if ( + this._platform === 'win32' && + this._load === loadNutModule && + sendWin32UnicodeReplacement(deleteCount, text, options) + ) { + return true + } + return false + } + /** 백스페이스를 count 번 — 키마다 네이티브 keyTap 한 번이다. */ async backspace(count: number): Promise { if (count <= 0) return diff --git a/apps/desktop/src/main/utils/win32-unicode-input.ts b/apps/desktop/src/main/utils/win32-unicode-input.ts index 858bcbb..c46616b 100644 --- a/apps/desktop/src/main/utils/win32-unicode-input.ts +++ b/apps/desktop/src/main/utils/win32-unicode-input.ts @@ -18,6 +18,8 @@ const logger = getLogger('win32-unicode-input') const INPUT_KEYBOARD = 1 const KEYEVENTF_KEYUP = 0x0002 const KEYEVENTF_UNICODE = 0x0004 +const VK_BACK = 0x08 +const VK_SPACE = 0x20 type SendInputFn = (cInputs: number, pInputs: Buffer, cbSize: number) => number @@ -80,6 +82,34 @@ function ensureLoaded(): SendInputFn | null { } } +function writeVirtualKeyPair(buffer: Buffer, pairIndex: number, vk: number): void { + const downBase = pairIndex * 2 * inputStructSize + buffer.writeUInt32LE(INPUT_KEYBOARD, downBase) + buffer.writeUInt16LE(vk, downBase + unionOffset) + buffer.writeUInt16LE(0, downBase + unionOffset + 2) + buffer.writeUInt32LE(0, downBase + unionOffset + 4) + + const upBase = (pairIndex * 2 + 1) * inputStructSize + buffer.writeUInt32LE(INPUT_KEYBOARD, upBase) + buffer.writeUInt16LE(vk, upBase + unionOffset) + buffer.writeUInt16LE(0, upBase + unionOffset + 2) + buffer.writeUInt32LE(KEYEVENTF_KEYUP, upBase + unionOffset + 4) +} + +function writeUnicodeKeyPair(buffer: Buffer, pairIndex: number, codeUnit: number): void { + const downBase = pairIndex * 2 * inputStructSize + buffer.writeUInt32LE(INPUT_KEYBOARD, downBase) + buffer.writeUInt16LE(0, downBase + unionOffset) + buffer.writeUInt16LE(codeUnit, downBase + unionOffset + 2) + buffer.writeUInt32LE(KEYEVENTF_UNICODE, downBase + unionOffset + 4) + + const upBase = (pairIndex * 2 + 1) * inputStructSize + buffer.writeUInt32LE(INPUT_KEYBOARD, upBase) + buffer.writeUInt16LE(0, upBase + unionOffset) + buffer.writeUInt16LE(codeUnit, upBase + unionOffset + 2) + buffer.writeUInt32LE(KEYEVENTF_UNICODE | KEYEVENTF_KEYUP, upBase + unionOffset + 4) +} + /** * UTF-16 문자열을 Windows SendInput(KEYEVENTF_UNICODE) 호출 한 번으로 입력 큐에 넣는다. * 성공 시 true, 지원하지 않거나 실패 시 false 를 반환한다. @@ -95,18 +125,7 @@ export function sendWin32UnicodeText(text: string): boolean { const buffer = Buffer.alloc(totalEvents * inputStructSize) for (let i = 0; i < codeUnits; i += 1) { - const code = text.charCodeAt(i) - const downBase = i * 2 * inputStructSize - buffer.writeUInt32LE(INPUT_KEYBOARD, downBase) - buffer.writeUInt16LE(0, downBase + unionOffset) - buffer.writeUInt16LE(code, downBase + unionOffset + 2) - buffer.writeUInt32LE(KEYEVENTF_UNICODE, downBase + unionOffset + 4) - - const upBase = (i * 2 + 1) * inputStructSize - buffer.writeUInt32LE(INPUT_KEYBOARD, upBase) - buffer.writeUInt16LE(0, upBase + unionOffset) - buffer.writeUInt16LE(code, upBase + unionOffset + 2) - buffer.writeUInt32LE(KEYEVENTF_UNICODE | KEYEVENTF_KEYUP, upBase + unionOffset + 4) + writeUnicodeKeyPair(buffer, i, text.charCodeAt(i)) } const sent = sendInput(totalEvents, buffer, inputStructSize) @@ -116,3 +135,54 @@ export function sendWin32UnicodeText(text: string): boolean { return false } } + +/** + * (선택적 IME 조합 확정 Space) + 커서 앞 백스페이스 N번 + UTF-16 문자열 입력을 + * 단 한 번의 Windows SendInput 호출로 원자적(atomic)으로 주입한다. + * + * Win32 SendInput 규약상 단일 호출 안의 INPUT 배열 사이에 사용자의 물리 키 입력이 끼어들 수 없으므로, + * 빠른 타이핑 도중에도 지우기와 다시 쓰기가 경쟁하지 않고 < 0.5ms 안에 끝난다. + */ +export function sendWin32UnicodeReplacement( + deleteCount: number, + text: string, + options: { commitComposition?: boolean } = {} +): boolean { + const safeDeleteCount = Math.max(0, Math.trunc(deleteCount)) + if (safeDeleteCount === 0 && !text) return true + const sendInput = ensureLoaded() + if (!sendInput || inputStructSize <= 0) return false + + try { + const commitSpacePairs = options.commitComposition && safeDeleteCount > 0 ? 1 : 0 + const backspacePairs = safeDeleteCount + commitSpacePairs + const textPairs = text.length + const totalPairs = commitSpacePairs + backspacePairs + textPairs + const totalEvents = totalPairs * 2 + if (totalEvents === 0) return true + + const buffer = Buffer.alloc(totalEvents * inputStructSize) + let pairCursor = 0 + + if (commitSpacePairs > 0) { + writeVirtualKeyPair(buffer, pairCursor, VK_SPACE) + pairCursor += 1 + } + for (let i = 0; i < backspacePairs; i += 1) { + writeVirtualKeyPair(buffer, pairCursor, VK_BACK) + pairCursor += 1 + } + for (let i = 0; i < textPairs; i += 1) { + writeUnicodeKeyPair(buffer, pairCursor, text.charCodeAt(i)) + pairCursor += 1 + } + + const sent = sendInput(totalEvents, buffer, inputStructSize) + return sent === totalEvents + } catch (error) { + logger.warn( + `SendInput(atomic replacement) 실행 실패: ${error instanceof Error ? error.message : String(error)}` + ) + return false + } +} diff --git a/apps/desktop/tests/main/services/TypoCorrectionService.test.ts b/apps/desktop/tests/main/services/TypoCorrectionService.test.ts index db9734d..d7ed6ce 100644 --- a/apps/desktop/tests/main/services/TypoCorrectionService.test.ts +++ b/apps/desktop/tests/main/services/TypoCorrectionService.test.ts @@ -141,6 +141,47 @@ describe('TypoCorrectionService', () => { expect(fake.replaceBeforeCaret).toHaveBeenCalledWith('ㅡㅁ가둥 '.length, 'markdown ') }) + it('모델 검사 도중 사용자가 구두점·짧은 완결 어절을 덧붙이면 취소하지 않고 새 접두 위로 리베이스해 교정한다', async () => { + let release: (() => void) | null = null + const generate = vi.fn(async function* () { + await new Promise((resolve) => { + release = resolve + }) + yield 'ㅡㅁ가둥 => markdown' + }) + const fake = deps({ warmModel: () => 'gemma4:e4b', generate }) + const service = new TypoCorrectionService(fake.deps) + + expect( + service.handleTypingContext( + typingContext({ + prefix: 'ㅡㅁ가둥', + isComposing: false, + idleMs: 650, + appName: 'Antigravity.exe' + }) + ) + ).toBe('checking') + + // 모델이 응답하기 전에 사용자가 공백을 쳤지만 유휴(40ms) 조건을 만족한 경우 리베이스 대기 유지 + fake.state.lastKeyAt = 1005 + expect( + service.handleTypingContext( + typingContext({ + prefix: 'ㅡㅁ가둥 ', + isComposing: false, + idleMs: 40, + appName: 'Antigravity.exe' + }) + ) + ).toBe('checking') + + release?.() + await flush() + await flush() + expect(fake.replaceBeforeCaret).toHaveBeenCalledWith('ㅡㅁ가둥 '.length, 'markdown ') + }) + it('설정으로 끌 수 있다', () => { config.typoAutoCorrectEnabled = false const fake = deps() diff --git a/apps/desktop/tests/main/services/personal-graph-context.test.ts b/apps/desktop/tests/main/services/personal-graph-context.test.ts index b1ae890..7362a55 100644 --- a/apps/desktop/tests/main/services/personal-graph-context.test.ts +++ b/apps/desktop/tests/main/services/personal-graph-context.test.ts @@ -74,4 +74,30 @@ describe('PersonalGraphService.retrieveContext', () => { expect(context.related).not.toContain('배포 로그 확인했습니다') expect(context.followUps).not.toContain('배포 로그 확인했습니다') }) + + it('같은 앱에서 단문 메시지를 연속으로 보내도 세션 follows 사슬을 잇고 recentWrites 로 복원한다', () => { + const graph = getPersonalGraphService() + const now = Date.now() + graph.indexText('배포 모두 끝났습니다', { source: 'typed', appName: 'Slack.exe', at: now }) + graph.indexText('모니터링 대시보드 확인 부탁드립니다', { + source: 'typed', + appName: 'Slack.exe', + at: now + 15_000 + }) + + // 채팅창에서 엔터로 전송해 입력칸이 비워진 뒤 다음 메시지를 치기 시작한 상태 + const context = graph.retrieveContext('지금 ', 4, { + appName: 'Slack.exe', + recentWrites: ['배포 모두 끝났습니다'] + }) + expect(context.followUps).toContain('모니터링 대시보드 확인 부탁드립니다') + }) + + it('조사가 달라 정확한 부분 문자열이 어긋나도 어간(stem) 연속 일치로 이어쓰기를 찾는다', () => { + const graph = getPersonalGraphService() + graph.indexText('회의록 정리해서 내일 오전까지 공유드리겠습니다', { source: 'typed' }) + + const context = graph.retrieveContext('오늘 회의록을 정리해서') + expect(context.continuations).toEqual(['내일 오전까지 공유드리겠습니다']) + }) }) diff --git a/docs/map/00-index.md b/docs/map/00-index.md index d449d5a..a289193 100644 --- a/docs/map/00-index.md +++ b/docs/map/00-index.md @@ -2,7 +2,7 @@ > Status: ACTIVE > Last full audit: 2026-09-13 -> Last update: 2026-10-28 — **v1.12.0 릴리스 + 한영 축약형·약어·오타 자동 교정 고도화(`INPUT-19`) + 전 표면 디자인파카(`designpaca`) 순회 감사(`SHELL-11`, `SHELL-15`).** (1) `@d3ro/core/typo-correction` 및 `TypoCorrectionService`를 확장해 영어 축약형·축약형+조사(`ㅊ무ㅅ`/`ㅊ무'ㅅ`/`ㅊ무ㅜ'ㅅ`→`can't`, `ㅊ무'ㅅ는`→`can't는`, `애ㅜ'ㅅ`→`don't`, `ㅑ'ㅡ`/`ㅑㅡ`→`I'm`, `재ㅜ'ㅅ`→`won't`), 문두 단독 `ㅑ `→`I `, 영어절 단음절 기능어(`새 애`→`to do`, `뭉`→`and`, `랙`→`for`, `소됴`→`they`, `생묘`→`today`), 일반 Araso 받침-조사 불일치 및 희소 어간 약어(`츤를`/`츤는`→`CMS를`/`CMS는`, `눈를`→`SNS를`, `운를`→`DNS를`, `긴를`→`RLS를`, `틔을`→`XML을`, `먀를`→`AI를`, `뗘를`→`EU를`, `쓰은`→`TM은`, `꾸은`→`RN은`, `띠이`→`EL이`, `꺼을`→`RJ을`, `ㅏㅅㅌ를`→`KTX를`, `ㅎㅅㅌ를`→`GTX를`), 1-편집 오타(`ㅗ디ㅐ`→`hello`, `ㅔㅐㄴ냐ㅠㅣ`→`possible`, `세갸ㅜㅎ`→`spring`, `ㅡㅁ가애주`→`markdown`, `help[`→`help`, `my self`→`myself`), 커서 꼬리 연쇄 구간(`MAX_FAST_DELETE_CHARS = 22`) 한정 고속 치환으로 타이핑 경합을 완전히 해소했다. (2) `packages/ui`, `apps/web`, `apps/admin`, `apps/desktop/src/renderer`(메인·팝업 6종), `site/`, `apps/mobile-rn` 6개 표면을 전수 순회해 `design.md` 계약 위반(WCAG AA 대비, `<11px` 미세 글자, `>=700` 굵기, 양수 자간, `uppercase`, 단독 `monospace`, `backdrop-filter` 블러, `h1` 위계 및 접근성 라벨)을 모두 해소했다 (`npm run check:design` 0건). +> Last update: 2026-10-29 — **v1.13.0 릴리스 + 그래프 관계성 기반 극한 개인화(`INPUT-06`, `INPUT-20`) + 한영 형태론·전방 구문 감지 및 단일-시스콜 초고속 인플레이스 수정(`INPUT-19`, `INPUT-22`) + 전 표면 디자인파카(`designpaca`) 순회 감사(`SHELL-11`, `SHELL-15`).** (1) **그래프 관계성 & 극한 개인화:** `@d3ro/core/personal-graph` 및 `PersonalGraphService`에 앱별 연속 단문 메시지 간 세션 `follows` 체이닝(`_lastIndexedByScope`), 엔터 전송 직후 비워진 채팅창 문맥 복원(`recentWrites`), 증분 `shares_terms` 엣지 연결, 조사 차이 무관 어간 연속 이어쓰기(`stemContinuationsFrom`), 2-홉 그래프 경로 랭킹(`rankTwoHopGraphTargets`), 개인 그래프·어투 일치 기반 LLM 후보 재정렬(`scoreCandidatePersonalization`), 어간 바이그램 폴백 및 빈도 가중 개인 n-gram(`PersonalNgramModel`)을 추가했다(제안 노출 빈도는 그대로 유지). (2) **영어 감지 강화 & 초고속 타이핑 중 무경합 교정:** `@d3ro/core/typo-correction`에 확장 영어 어간(`EXTENDED_ENGLISH_STEMS`, `kubernetes`/`microsoft` 등)과 생산적 접두·접미 파생 형태론(`matchesInflectedOrDerivedEnglish`), 전방 영어 구문 호응(`새 ㅠㄷ `→`to be `, `새 애 this `→`to do this `, `랙 솓 `→`for the ` vs `오늘 새 API를 만들었어` 보존), 영어 축약형·약어(`ㅊ무ㅅ`/`ㅊ무'ㅅ`/`ㅊ무ㅜ'ㅅ`→`can't`, `츤를`→`CMS를`, `눈를`→`SNS를` 등)를 추가하고, `sendWin32UnicodeReplacement`(`utils/win32-unicode-input.ts`)로 `(선택 Space) + N × VK_BACK + M × KEYEVENTF_UNICODE`를 **단일 `SendInput` 시스템콜(`< 0.5 ms`, 대기 `0 ms`)**로 원자적 주입하며, 타이핑 중 추가 입력된 꼬리 글자를 보존하는 델타 리베이싱(`rebaseTypoReplacementPlan`)을 구현했다. (3) `packages/ui`, `apps/web`, `apps/admin`, `apps/desktop/src/renderer`(메인·팝업 6종), `site/`, `apps/mobile-rn` 6개 표면을 전수 순회해 `design.md` 계약 위반을 모두 해소했다 (`npm run check:design` 0건). > > Previous update: 2026-10-27 — **v1.11.0 릴리스 + Araso 방식 한영 자동 변환·터미널/Vim/xterm.js 지원 완성 (`INPUT-19`, `GAP-INPUT-08`).** `nyttastudio/araso`를 분석해 `@d3ro/core/araso-layout` 및 `@d3ro/core/typo-correction`을 대폭 업그레이드하고 터미널·Vim·xterm.js(`Agent Switchboard`)까지 확장했다: 영어·약어·CamelCase+한국어 조사/단어 혼합어 분리(`githubdp`→`github에`, `PDFvkdlf`/`ㅖㅇㄹ파일`→`PDF파일`, `Reactzjavhsjsxm`→`React컴포넌트`, `dhsmfdmsgoogledptj`→`오늘은google에서`, `1:1ansdml`→`1:1문의`, 2글자 약어 받침 호응 `TMdms`→`TM은` vs `TMsms`→`쓰는`), 쌍자음 잔여 Shift 정규화(`djEJs`→`어떤`, `qkRNjTdj`→`바꿨어`), 한글 자판 상태 영어/혼합어 역변환(`ㅗ디ㅣㅐ 재깅`→`hello world`, `내ㅡㄷ샤ㅡㄷ`→`sometime`, `ㅎ샤ㅗㅕㅠ에`→`github에`, `ㅠㅕㅎ가`→`bug가`, 알파벳+조사 `ㄴ가`→`s가`, `ㄴ은`→`s은`), 문장 문맥 한영 전환(`Stay hungry, stay foolishsms wkqtmdml akfdlek.`), URL·경로·이메일 보호(`isUrlOrPathRun`), 터미널·Vim 명령(`:ㅈㅂ`→`:wq`, `:ㅂ!`→`:q!`, `퍄ㅡ `→`vim `, Vim 노멀 모드 교정 억제), `SendInput(KEYEVENTF_UNICODE)` 직접 유니코드 타이핑(클립보드 무접촉), 단어 경계(`25ms`)와 어절 내부(`650ms`) 유휴 분리 및 완성형 음절(`[가-힣]`) 한정 `commitComposition`으로 고속 타이핑 시 글자 삼킴·유령 공백·앞글자 중복을 완전히 해소했다. 통계 수집 동의(`inputTelemetryEnabled`)가 꺼져 있어도 기본 기능(`typoAutoCorrectEnabled: true`)으로 즉시 동작한다. > @@ -13,7 +13,7 @@ > Previous update: 2026-09-22 — **Gemma/Ollama 폭주 방어 계약을 기록했다.** 19:14:12 부팅 워밍업이 `keep_alive: 30m`으로 `gemma4:e4b`를 19:44:12까지 VRAM 3,226,342,521 bytes / context 4096으로 강제 상주시킨 것이 관측됐으며, 같은 시점 Windows GPU Engine PID 표본에는 Ollama의 활성 compute가 없었다. 즉 당시 상태는 무한 추론이 아니라 강제 residency였다. 19:11:17~19:11:58의 자동 제안 연속 생성은 기존 900 ms·12/min·5 candidates·128 tokens·한 글자 재생성 정책이 허용한 burst였다. 현재 구현 계약은 부팅 warmup 제거, `keep_alive: 2m`, 제안 600 ms / 최소 5 s 간격 / 기본 6회·hard max 12회 per min / 3 candidates / 64 tokens / 12-char growth / 8 s timeout, 그리고 요청별 취소·상한·종료 정리다. 독립 표적 검증은 6 test files / 69 tests passed / 0 failed, 변경 코드·테스트 ESLint와 `git diff --check`도 exit 0이다. Raw Ollama에서는 cold bounded 요청이 client hard timeout 15.044 s에 취소된 뒤 `/api/ps`가 비었고 `/api/version`은 80 ms에 회복했다. 명시 warmup은 HTTP 200 / 16.639 s, 후속 warm 요청은 body `options.num_predict=1`, `keep_alive='2m'`로 553 ms HTTP 200 / `done:true` / `eval_count:1` / `done_reason:length`였고 `/api/ps` expiry는 약 119.9 s였다. **19:48:59 +09:00에는 새 generate/unload/kill/retry 없이 충분히 지난 뒤 한 번의 `/api/ps`가 HTTP 200 / 45.8 ms / `{models:[]}`였고 `/api/version`은 HTTP 200 / 7.3 ms / `0.32.13`이었다.** 이는 raw API 수준의 expiry 뒤 unload 확인일 뿐 앱 재시작·GUI·실제 타이핑 증거는 아니므로 상태는 `[~]`로 유지한다 (`11` GAP-LLM-04, GAP-INPUT-06). > > Previous update: 2026-09-21 — **Input intelligence (입력 레메트리 + 다음 문장 제안) 신규**. 데스크톱에 입력 수집기(`InputTelemetryService`), UIA 컨텍스트 브리지(사이드카 `GET /uia/focus`), 제안 서비스(`SuggestionService`), 어렛 커 오버레이, 설정 > 입력 탭(동의·정책·주간 인사이트·개인 문구)을 추가했다. 카탈로그에 `INPUT-01`~`INPUT-08`(전부 데스크톱 `[~]` — 유닛 48건은 GREEN 이지만 **실앱 타이핑 검증 전**), 백로그에 GAP-INPUT-01~05 + GAP-LLM-03, §7 에 CONSTRAINT-INPUT-01(키 내용 미저장 — ActivityWatch 정책 채택)을 기록. 설계 근거는 조사 기반이다: 어렛은 `GetGUIThreadInfo` 가 아니라 UIA `TextPattern.GetSelection`(Chromium 은 `TextPattern2` 미구현), 타이핑 스트는 키코드 복원이 아니라 UIA 스냅샷 diff(한/일 IME 대응), 디바운스/토큰 한도는 인라인 컴플리션 실측값(Continue 350 / Tabby 250 / twinny 300 ms, 출력 64~256 토큰). 의존성: `koffi` 3.3.1(포그라운드 창 FFI), 사이드카 `uiautomation` 2.0.29 + `comtypes`. 데스크톱 유닛 총계 1409(+49), Electron ABI 실행에서 신규 실패 0건. 당시의 24.7초/4.9초 지연 설명과 `keep_alive: 30m`·부팅 워밍업 처방은 **현재 상태가 아닌 과거 가설/완화 이력**이며, 최신 운영 결론은 위 2026-09-22 항목과 `11` GAP-LLM-04를 따른다. 직전: LLM instruction-prompt fix (`9c2b4d4`): the custom-instruction path inserted the instruction's own wording instead of the processed result and had **never worked in any shipped release** (`v0.1.0-alpha`..`v1.4.0`, introduced `fea923d` 2026-04-05, not a regression). `llm-prompts.ts` is now the SSOT for prompt resolution and placeholder substitution, shared by `VoiceModeService` / `ChainService` / `LLM.PROCESS`. AI-04/05/06/07 are demoted to `[~]` on desktop — fixed with unit tests, but **not verified in a running app** and the four related `tests/red/*.usecase.test.ts` could not execute (`better-sqlite3` ABI). New: GAP-LLM-01 (no target-language setting), GAP-LLM-02 (this fix unverified); GAP-INFRA-06 amended (the ABI masks verification, not just dev-env switching cost); GAP-I18N-01 amended (`popup.error.default` missing in 10 locales). Earlier the same day: CAP-16 (desktop key bindings rebuilt on one `@d3ro/core/keybinding` SSOT — multiple bindings per action, mouse buttons, `HOTKEY` → `KEYBINDING` IPC group), verified on Windows by a manual run, so CAP-16 and CAP-02 are `[x]` and GAP-KEY-01 is closed. Still open: GAP-KEY-02/03, GAP-QA-02, GAP-I18N-01/02, GAP-INFRA-06, GAP-LLM-01/02, GAP-INPUT-01~05; `11` §7 holds accepted design constraints (things deliberately kept, not gaps) -> Scope: entire monorepo `D:/workspace/D3ROVoice` at product version `1.12.0` (`release/product-version.json`, released 2026-10-28) +> Scope: entire monorepo `D:/workspace/D3ROVoice` at product version `1.13.0` (`release/product-version.json`, released 2026-10-29) > Purpose: let any agent (or human) answer two questions in under a minute: > 1. **What infrastructure exists?** (build, CI, services, APIs, data, packages, deploy) > 2. **How far is each feature developed?** (per surface, with file anchors and status) diff --git a/docs/map/03-shared-packages.md b/docs/map/03-shared-packages.md index 7dca238..29241d6 100644 --- a/docs/map/03-shared-packages.md +++ b/docs/map/03-shared-packages.md @@ -18,7 +18,8 @@ The canonical place for types and cross-surface logic. Both desktop and web/mobi | IPC channels | `./ipc-channels` | `IPC_CHANNELS` object + `IPCChannel` union. **SSOT** for every desktop IPC channel (VOICE, AUDIO, STT, TTS, LLM, KEYBINDING, CONFIG, HISTORY, UNIFIED_SEARCH, DICTIONARY, WINDOW, SYSTEM, STATS, MEMO, VOICE_COMMAND, CONTEXT, CHAIN, CAPTION, FILE_TRANSCRIPTION, MEETING_SUMMARY, DICTATION_TEMPLATE, VOICE_CONVERSATION, RAG, VOICE_ACTION, MEETING_MODE, MEETING_DOC_TEMPLATE, MEETING_CHAT, LICENSE, CLOUD_SYNC, INSTRUCTION, SYSTEM_AUDIO, POPUP_RESULT, POPUP_HISTORY, POPUP_COMMAND, POPUP_CAPTION, VOICE_PARTIAL, CLIPBOARD, APP, ONLINE_AUTH, ADS, SUPPORT, INPUT_TELEMETRY, SUGGESTION, POPUP_SUGGESTION) | | Key bindings | `./keybinding` | **SSOT** for every global shortcut in the app: `KeyBinding` (`device`/`code`/`ctrl`/`alt`/`shift`/`meta`), `KEY_CATALOG` (10 selectable groups incl. mouse), `KEYBINDING_ACTIONS` (9 rebindable actions incl. the three suggestion actions), `bindingKey`/`normalizeBinding`/`validateBinding`/`detectBindingConflicts`/`formatBindingSegments`/`searchKeyCatalog`/`parseBindingMap`. Persisted as `AppConfig.keyBindings`. i18n keys are exposed as plain `string` so core stays independent of `@d3ro/i18n`; consumers narrow at the boundary (`asTranslationKey`) and a contract test guards the keys — accepted constraint, `11` §7 CONSTRAINT-I18N-01. Tests: `__tests__/keybinding*.test.ts` via `vitest.config.ts` (`npm run test --workspace=@d3ro/core`), 117 cases as of 2026-09-21 | | Input intelligence | `./input-intelligence` | **SSOT** for the input-telemetry and next-sentence-suggestion domain (added 2026-09-21): key classification (`classifyKeyStroke`), text metrics (`countWords`/`countSentences`/`endsSentence`/`textBeforeCaret`), `computeTypedDelta` (UIA snapshot diff — the IME-safe way to count typed text), `decideSuggestion` + `isAppExcluded` (when to request / skip / clear), `parseSuggestionCandidates`/`sanitizeSuggestionLine` (prompt-leak and prefix-echo defence), `anchorFloatingPanel` (caret-anchored overlay placement), `mergeActivityBucket`/`summarizeActivity`/`pixelsToMeters`, `extractPhrases`/`selectPhraseHints`, `INPUT_TELEMETRY_DEFAULTS`/`SUGGESTION_DEFAULTS` | -| Typo correction & Araso layout | `./typo-correction`, `./araso-layout` | **SSOT** for built-in automatic Han/Eng conversion and typo auto-correction (upgraded 2026-10-26): 2-set Korean/English layout conversion (`engToHangul`/`hangulToEng`), lingering Shift normalization (`normalizeLingeringShift`/`lingersAfterKoreanShift`), mixed-word splitting with Korean particle batchim agreement (`splitMixedWord`/`particleAgrees`), reverse Hangul-to-English/mixed restoration, URL/path/email run protection (`isUrlOrPathRun`), deterministic spelling & jamo-noise rules (`detectTypos`/`detectJamoNoise`), and LLM typo validation (`validateModelCorrections`). | +| Typo correction & Araso layout | `./typo-correction`, `./araso-layout` | **SSOT** for built-in automatic Han/Eng conversion and typo auto-correction (upgraded 2026-10-28): 2-set Korean/English layout conversion (`engToHangul`/`hangulToEng`), productive English derivational prefixes/suffixes (`matchesInflectedOrDerivedEnglish`, `EXTENDED_ENGLISH_STEMS`), phrase-level lookahead English detection (`TO_FOLLOWING_ENGLISH_WORDS`, `resolveTokenAsEnglishPreview`), lingering Shift normalization (`normalizeLingeringShift`/`lingersAfterKoreanShift`), mixed-word splitting with Korean particle batchim agreement (`splitMixedWord`/`particleAgrees`), reverse Hangul-to-English/mixed restoration, URL/path/email run protection (`isUrlOrPathRun`), deterministic spelling & jamo-noise rules (`detectTypos`/`detectJamoNoise`), delta-rebasing for in-flight typing (`rebaseTypoReplacementPlan`), and LLM typo validation (`validateModelCorrections`). | +| Personal graph & n-gram | `./personal-graph`, `./personal-ngram` | **SSOT** for relational personalization and n-gram continuation (upgraded 2026-10-28): exact & particle-invariant stemmed continuations (`continuationsFrom`/`stemContinuationsFrom`), IDF-weighted related node scoring (`scoreRelatedNodes`), 2-hop graph path ranking (`rankTwoHopGraphTargets` across `shares_terms→follows` and `follows→follows`), personalized candidate reranking (`scoreCandidatePersonalization`), and frequency-weighted n-gram indexing with Korean stemmed-word bigram fallback (`PersonalNgramModel`). | | Constants | `./constants` | Shared constants | | Crypto license | `./utils/crypto-license` | Ed25519 license sign/verify (used by admin issuer + desktop verifier) | | PII | `pii-redactor`, `secure-memory` | Redaction + secure memory helpers | diff --git a/docs/map/04-desktop-app.md b/docs/map/04-desktop-app.md index a8f0eef..5be6e1a 100644 --- a/docs/map/04-desktop-app.md +++ b/docs/map/04-desktop-app.md @@ -84,7 +84,7 @@ Singleton + `EventEmitter` pattern (`getXService()` accessors). | Service / file | Purpose | |---|---| | `InputTelemetryService` | Global input telemetry and context observer: keystroke/click/scroll counters, mouse travel (Manhattan sum of per-axis deltas), active time, per-hour-per-app buckets flushed every 5 s; typed-text learning via UIA snapshot diffs. **Since 2026-10-26**, the UIA context observer runs by default whenever `typoAutoCorrectEnabled !== false` (or `suggestionEnabled === true`), while SQLite `input_activity` persistence only runs when `inputTelemetryEnabled === true` (opt-in). **Since 2026-10-27**, includes `TerminalKeystrokeTracker` (`@d3ro/core/input-intelligence`) + `isForegroundKoreanIme` (`utils/win32-foreground.ts`) to reconstruct the active terminal/Vim line (`reconcileWithScreen` for UIA screen buffers like Windows Terminal/pwsh/cmd, `reconcileWithXtermValue` for `xterm.js` `