fix(desktop): add precise local OCR reading
This commit is contained in:
parent
59cdddbecb
commit
25fd3ac004
30 changed files with 626 additions and 15 deletions
|
|
@ -2,7 +2,9 @@
|
|||
|
||||
> Status: ACTIVE
|
||||
> Last full audit: 2026-09-13
|
||||
> Last update: 2026-11-20 — **데스크톱 `v1.18.4` 자동 업데이트 피드 복구.** 기존 설치본은 정상 확인했으나 공개 피드가 `1.17.1`에 멈춰 있었다. 미게시 1.18.x의 크기 초과·과거 release 재귀 포함·만료된 Forgejo token을 해결하고 `1.18.4`(`f77c86d`)를 게시했다. Windows full-only 설치본 `104,691,365 bytes`, 공개 signed manifest와 Release 다운로드 HTTP 206을 확인했다. 실제 설치 앱의 제안 화면은 사용자 수동 확인으로 남는다(`11` GAP-REL-17).
|
||||
> Last update: 2026-11-21 — **데스크톱 `1.17.1 → 1.18.4` 실제 자동 업데이트 완료.** 제안 발견·다운로드·Ed25519 서명 검증·재시작 설치·기존 DB 열기를 설치 앱 로그로 확인했고, 실행 파일 ProductVersion `1.18.4.0`과 실제 화면 OCR 사용을 확인했다(`11` GAP-REL-17, INFRA-15 완료). 사용자가 보고한 OCR 기호·정확도 문제는 실제 Windows 엔진으로 재현했다. URL·이메일 토큰 사이의 부당한 공백은 소스에서 수정했고, GLM-OCR 선택 영역 정밀 읽기도 추가했다(51 focused tests·실제 서비스/API·빌드 UI 검증). 실제 설치 GUI 확인과 드문 한글·공백 정확도 개선은 `11` GAP-OCR-02에 남는다.
|
||||
>
|
||||
> Previous update: 2026-11-20 — **데스크톱 `v1.18.4` 자동 업데이트 피드 복구.** 기존 설치본은 정상 확인했으나 공개 피드가 `1.17.1`에 멈춰 있었다. 미게시 1.18.x의 크기 초과·과거 release 재귀 포함·만료된 Forgejo token을 해결하고 `1.18.4`(`f77c86d`)를 게시했다. Windows full-only 설치본 `104,691,365 bytes`, 공개 signed manifest와 Release 다운로드 HTTP 206을 확인했다.
|
||||
>
|
||||
> Previous update: 2026-11-14 — **전 표면 프로덕션 `v1.17.1` 배포 완결.** 데스크톱(`v1.17.1`, 커밋 `6c6875b`, 서명 매니페스트 `1.17.1/latest.yml`·`.sig` 및 `latest/latest.yml`, 런타임 `runtime-1.17.1`/`runtime-latest`, 포터블 `portable-1.17.1`/`portable-latest` 모두 Forgejo HTTP `200`, 태그 `v1.17.1` canonical/mirror 푸시; Human Release #339 ~91MB 바이너리 첨부만 Cloudflare HTTP 524/`ECONNRESET` `GAP-REL-16` 예외), Supabase 운영 DB(`db push --remote`, 전체 74개 마이그레이션 일치) 및 Edge Functions 24종(`--import-map deno.json --use-api`) 재배포, 랜딩 사이트(`site/` → Cloudflare Pages `d3ro`, `https://d3ro.chanpaca.net/release-identity.json` = `1.17.1`/`6c6875b` & `assetlinks.json` 검증 GREEN), Cloudflare Worker(`d3ro-voice-api` `--keep-vars`, Version ID `fe6d9532-0052-48f5-b62c-45c86f6aaaa5`), NAS Docker 스택(`d3ro_voice_api` `v1.17.1` `/health` `200`, `d3ro_voice_admin` `:3001`/`d3ro-admin.chanpaca.net/login` `200`, `d3ro_voice_web` `:3002/app/login`·`d3ro.chanpaca.net/app/login` `200`) 반영 완료.
|
||||
>
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
# 02 — Repository & Infrastructure Map
|
||||
|
||||
> Surface: whole repo
|
||||
> Last updated: 2026-11-20 — `v1.18.4` 데스크톱 업데이트 피드 복구: Electron 43 패키지의 100 MiB 업로드 한도 초과를 패키지 중복·미사용 플랫폼 바이너리 제거와 Windows full-only 최대 압축으로 해소하고 실제 설치본·서명 매니페스트를 검증
|
||||
> Last updated: 2026-11-21 — 화면 OCR에 선택 영역 전용 GLM-OCR 정밀 읽기 추가: 설치본에는 모델을 포함하지 않고 명시적 첫 요청 시 다운로드하며, loopback 전용 이미지 처리·2분 상주·출력/시간 제한을 적용. `v1.18.4` full-only 패키지·피드 계약은 변경하지 않음
|
||||
> Source of truth for: layout, workspaces, build/test commands, CI/CD, Docker, deploy, scripts, resources
|
||||
|
||||
---
|
||||
|
|
@ -267,6 +267,7 @@ Version sync is enforced by `scripts/ci/sync-version.mjs` and `verify-release-me
|
|||
- `resources/ffmpeg/` — optional bundled ffmpeg (CI or manual); `getFfmpegPath()` also resolves the `@ffmpeg-installer/ffmpeg` binary from `app.asar.unpacked`.
|
||||
- `apps/desktop/sidecar-dist/` — PyInstaller sidecar bundle consumed by `extraResources` (gitignored; built by `npm --prefix apps/desktop run sidecar:build`).
|
||||
- `resources/icons/` — empty; electron-builder falls back to `build/icon.ico|png`.
|
||||
- GLM-OCR precise screen reading (`apps/desktop/src/main/services/screen-ocr/ollama-precision-engine.ts`) uses the existing Ollama server and downloads `glm-ocr:latest` (about 2.7 GB) only on the user's first explicit precise-reading request. The model is **not bundled in the installer**. Only a selected crop is sent to a loopback server, remote Ollama URLs are rejected, the model stays loaded for 2 minutes, and output length/timeouts are bounded. No Paddle/PyTorch runtime was added to the product.
|
||||
- Desktop tests: `apps/desktop/tests/` — `unit/`, `main/`, `red/`, `shared/`, `integration/`, `eval/` run under Vitest; `e2e/` holds Playwright specs plus the shared `e2e/support/electron-app.ts` fixture (per-test temp user-data dir). Artifacts in `apps/desktop/test-results/`.
|
||||
- Test taxonomy + hygiene gate: `docs/tdd-red/06-repo-application.md`. `npm run test:hygiene` (+ `:test`) rejects `.only`, reason-less `.skip()`, trivial assertions, `isVisible()`-gated assertions, machine-specific paths, `waitForTimeout`, and `.e2e.test.ts` names; wired into CI `quality`.
|
||||
- Test-system gates (TDD 2.0 Phases 0–5, 2026-11-08..13):
|
||||
|
|
|
|||
|
|
@ -62,7 +62,7 @@ Singleton + `EventEmitter` pattern (`getXService()` accessors).
|
|||
| `VoiceCommandService` | Keyword → command rule matching |
|
||||
| `ChainService` | Multi-step LLM pipelines (LLMChain). Each step resolves its instruction through `llm-prompts.ts` (`ChainService.ts:196`); before that, chain steps sent placeholders through unsubstituted |
|
||||
| `ScreenContextService` | Active-window + selected-text context (on Windows uses `sendWin32CopyChord()` via `koffi` `SendInput` without loading `nut-js`) |
|
||||
| `ScreenOcrService` | Screen text capture (CAP-17): owns one capture session — capture → overlay → async recognition → overlay requests (copy text/image, save PNG/searchable PDF, translate, summarize, read aloud, open link/search, language switch) → close. All OS access is behind ports (`services/screen-ocr/electron-adapters.ts`); engine port `OcrEnginePort` has one adapter, `windows-ocr-engine.ts` (Windows.Media.Ocr). Assembled in `src/main/screen-ocr-wiring.ts` (key binding, overlay IPC with sender check, `Escape` capture while open except during a save dialog). Events: `opened`, `closed`. |
|
||||
| `ScreenOcrService` | Screen text capture (CAP-17): owns one capture session — capture → overlay → async recognition → overlay requests (copy text/image, save PNG/searchable PDF, **local GLM-OCR precise reading**, translate, summarize, read aloud, open link/search, language switch) → close. All OS access is behind ports (`services/screen-ocr/electron-adapters.ts`); base engine port `OcrEnginePort` has one adapter, `windows-ocr-engine.ts` (Windows.Media.Ocr). `selectionText` uses word boxes to join tightly spaced URL/email/punctuation tokens without collapsing ordinary word spacing; it never guesses replacement glyphs. The separate `PrecisionOcrEnginePort` (`screen-ocr/ollama-precision-engine.ts`) sends only a selected crop to native `/api/generate` on a **loopback** Ollama server (remote URLs rejected), downloads `glm-ocr:latest` only after the explicit precise action, keeps it for 2m, bounds output tokens, rejects empty/truncated output, and preserves the original on failure. `precision-output.ts` removes only a single generated outer code fence when neither the body nor source contains fences. Assembled in `src/main/screen-ocr-wiring.ts` (key binding, overlay IPC with sender check, `Escape` capture while open except during a save dialog). Events: `opened`, `closed`. |
|
||||
|
||||
### Phase 10+ features
|
||||
| Service | Purpose |
|
||||
|
|
@ -105,7 +105,7 @@ The existing `input-telemetry-handlers` and `suggestion-handlers` IPC extensions
|
|||
| `CloudSyncService` | Supabase auth + lifecycle: per-user DB switching, local-mode import on first sign-in, Realtime (`ws` transport) + 5-min heartbeat, device check-in, debounced flush/pull triggers. The sync itself lives in `services/sync/` |
|
||||
| `services/sync/` | `SyncEngine` (backfill once per user DB → push outbox → pull by server-clock keyset cursor → tombstones → memo-tag reconcile), `sync-outbox` (`sync_outbox`/`sync_state` tables), `sync-adapters` (history, dictionary, meetings, meeting memos/documents, custom commands, templates), `memo-tag-sync`, `supabase-sync-remote`, `device-registration`, `realtime-transport`, `settings-sync` (mobile `user_settings` ↔ language/theme/defaultLLMAction/activeInstructionId), `audio-sync` (recording upload to the `audio` bucket, remote cleanup on delete), knowledge adapter (source-text chunks), `transcript-sync` (meeting segments from the desktop transcript), `builtin-instruction-sync` (preset prompt edits ↔ server preset rows). Services record changes with `getCloudSyncService().pushOne/pushDelete(entity, id)`; pulled rows are applied without re-queuing |
|
||||
| `CloudSTTService` | Thin cloud STT wrapper over `D3ROCloudDriver` |
|
||||
| `UpdateService` | electron-updater (canonical Forgejo feed, channels, mandatory/full-vs-delta policy, staged rollout). Native consent/restart dialogs resolve `update.dialog.*` through a fresh `getPopupTranslator()` at each prompt, so a runtime locale change is reflected without rebuilding the prompter; unit tests cover required/optional paths and a ko→en switch on the same prompter. `update-dialog-preview.ts` exposes only the explicit `--preview-update-dialogs` manual-QA entry described above. The current-source package was built and its native ABI verified; packaged dialog appearance/interaction remains unverified until that executable is run on the interactive desktop. |
|
||||
| `UpdateService` | electron-updater (canonical Forgejo feed, channels, mandatory/full-vs-delta policy, staged rollout). Native consent/restart dialogs resolve `update.dialog.*` through a fresh `getPopupTranslator()` at each prompt, so a runtime locale change is reflected without rebuilding the prompter; unit tests cover required/optional paths and a ko→en switch on the same prompter. `update-dialog-preview.ts` exposes only the explicit `--preview-update-dialogs` manual-QA entry described above. The installed 1.17.1 app found, downloaded, signature-verified and installed 1.18.4 on 2026-11-21, then opened its existing DB and ran the real OCR overlay. The optional consent/restart journey is verified; the four explicit preview-dialog variants are not an exhaustive visual test. |
|
||||
| `AutoLaunchService` | OS login-item auto-start |
|
||||
| `LoggerService` | electron-log wrapper + category loggers |
|
||||
| upgrade / billing | No in-app checkout. `license-handlers.ts` `LICENSE.OPEN_BILLING` opens `billingUrl({ tier })` (web Payple); tier returns via `license:tierChanged`. Stripe `payment-handlers.ts`·`CheckoutModal`·`payment:*` IPC removed 2026-09-26 |
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ Status quick-reference: `[x]` done+verified · `[~]` partial/unverified · `[ ]`
|
|||
| CAP-14 | Recording persistence / crash recovery | [x] | [ ] | [x] | [-] | Desktop WAV persist; mobile durable queue + process-kill WAV recovery |
|
||||
| CAP-15 | Android foreground recording service | [-] | [-] | [x] | [-] | Mobile API 34 FGS + persistent notification (SSOT R-005 GREEN) |
|
||||
| CAP-16 | Rebindable global key bindings (keyboard + mouse) | [x] | [-] | [-] | [-] | Contract SSOT `packages/core/src/keybinding.ts`: `KEY_CATALOG` (10 groups), `KEYBINDING_ACTIONS` (6 actions), `validateBinding`, `detectBindingConflicts`. Multiple bindings per action persist as one `AppConfig.keyBindings` map (`packages/core/src/types.ts`), replacing the four singular `*Shortcut` fields; `ConfigService` migrates legacy values once. `KeyBindingService` hooks keyboard **and** mouse via uiohook — MB1 is not bindable, MB2/MB3 need a modifier, MB4/MB5 are free. **Verified 2026-09-21 on Windows by a manual run** and **2026-10-30 unit-tested** in `KeyBindingService.test.ts` (11 tests, GAP-KEY-01 `[x]`), with dedicated `command` action toggle/hold + forced voice-command matching in `VoiceModeService.ts` / `voice-intent.ts` (`voice-intent.test.ts` 19 tests, GAP-KEY-03 `[x]`). **Still open:** macOS/Linux mouse behavior (`11` GAP-KEY-02). W/M `[-]`: no OS-level global binding surface exists there. B `[-]`: device-local setting. |
|
||||
| CAP-17 | Screen text capture (OCR overlay) | [~] | [-] | [-] | [-] | Desktop/Windows. The rebindable `screen-ocr` action (default `Ctrl+Alt+O`, CAP-16) freezes the monitor under the cursor and covers it with the `screen-ocr` popup; text is selected by flow drag (start on a word), rectangle drag (start on empty space or in Area mode), double-click word, triple-click line, Shift+click extend, `Ctrl+A`. Actions: copy (multi-line / one line), copy image, save PNG, **searchable PDF** (image above a real text layer), translate and summarize through `LlmGateway`, read aloud (`TTSPlaybackService`), open link / web search, recognition language switch (`AppConfig.screenOcrLanguage`). Pure selection/text/PDF logic `packages/core/src/screen-ocr.ts`; service `apps/desktop/src/main/services/ScreenOcrService.ts`; engine `services/screen-ocr/windows-ocr-engine.ts` (Windows.Media.Ocr via `powershell.exe -EncodedCommand`, 2x upscale); capture `utils/win32-screen-capture.ts` (GDI BitBlt, 39 ms vs 2.9 s for `desktopCapturer` on a 3-monitor PC). The capture touches disk only as one temp PNG that is deleted right after recognition. **Verified 2026-11-20:** core 34 + desktop 41 tests, real-engine probe (Korean+English screen, 0.64 s), headless overlay run (flow/rect selection, copy, translate, PDF request, release on close) and a printed PDF whose text layer is searchable. **Not yet verified:** the live hotkey journey in the running app (focus, mixed-DPI multi-monitor) — `GAP-OCR-01`. macOS has no engine and fails closed with a message — `GAP-OCR-02`. |
|
||||
| CAP-17 | Screen text capture (OCR overlay) | [~] | [-] | [-] | [-] | Desktop/Windows. The rebindable `screen-ocr` action (default `Ctrl+Alt+O`, CAP-16) freezes the monitor under the cursor and covers it with the `screen-ocr` popup; text is selected by flow drag (start on a word), rectangle drag (start on empty space or in Area mode), double-click word, triple-click line, Shift+click extend, `Ctrl+A`. Actions: copy (multi-line / one line), copy image, save PNG, **searchable PDF** (image above a real text layer), translate and summarize through `LlmGateway`, read aloud (`TTSPlaybackService`), open link / web search, recognition language switch (`AppConfig.screenOcrLanguage`). Pure selection/text/PDF logic `packages/core/src/screen-ocr.ts`; service `apps/desktop/src/main/services/ScreenOcrService.ts`; engine `services/screen-ocr/windows-ocr-engine.ts` (Windows.Media.Ocr via `powershell.exe -EncodedCommand`, 2x upscale); capture `utils/win32-screen-capture.ts` (GDI BitBlt, 39 ms vs 2.9 s for `desktopCapturer` on a 3-monitor PC). The capture touches disk only as one temp PNG that is deleted right after recognition. **Verified 2026-11-20:** core 34 + desktop 39 tests, real-engine probe (Korean+English screen, 0.64 s), headless overlay run (flow/rect selection, copy, translate, PDF request, release on close) and a printed PDF whose text layer is searchable. **2026-11-21:** installed 1.18.4 ran the real OCR overlay. Pixel-aware joining has 37 core tests (full core 666); GLM precise mode has 51 focused tests. A local GLM-OCR precise action now reads only the selected crop, downloads the model on explicit request, keeps the original on failure, and rejects remote Ollama URLs and truncated/repeated output. Real model comparison: Windows 32.29% overall CER, PP-OCRv5 14.35%, PaddleOCR-VL crop 2.99%, GLM crop 5.08% (GLM has rare Korean/format errors but exact code content apart from generated fences and was 3.74s vs Paddle 20.89s mean). The real service preserved all 1,215 non-whitespace characters and 403 symbols in the code fixture, with three whitespace-only differences and no temp files left. Built-overlay selection→crop→result→copy passes. **Not yet verified:** installed-app precise button, mixed-DPI multi-monitor focus/copy/PDF, rare Korean/whitespace parity — `GAP-OCR-01` and `GAP-OCR-02`. macOS has no engine and fails closed with a message — `GAP-OCR-02`. |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -234,7 +234,7 @@ end-to-end behaviour has **not been verified by typing in a real app** (`11` GAP
|
|||
| INFRA-12 | Cloudflare edge + tunnel | [x] | `server/cloudflare-worker`, Cloudflare Tunnel `kd-nas` |
|
||||
| INFRA-13 | Site deploy (Cloudflare Pages) | [x] | `.forgejo/workflows/deploy-site.yml`; the unused GitHub Pages workflow was removed in Wave 3 |
|
||||
| INFRA-17 | Test hygiene & TDD v2.0 gates | [x] | `scripts/ci/check-test-hygiene.mjs` (`test:hygiene`), `check-protected-tests.mjs` (`check:protected-tests`), `check-test-quality.mjs` (`test:quality`), `check-i18n-contract.mjs` (`check:i18n`), `check-ds-contract.mjs` (`check:ds-contract`), `run-affected-tests.mjs` (`test:affected`), `verify-desktop-installer-smoke.mjs` (`release:installer:smoke`), and `@d3ro/core/ports` (`Clock`/`Random` deterministic ports) — all equipped with `--self-test` and wired into `verify:all`, `.forgejo/workflows/ci.yml`, `release-desktop.mjs`, and `.forgejo/workflows/release.yml`. Applied policy: `docs/tdd-red/06-repo-application.md`, `07-revamp-plan.md`. |
|
||||
| INFRA-15 | Update & release system | [~] | Canonical Forgejo feed + channels/policy, publisher `scripts/ci/publish-forgejo-release.mjs`; `npm run release:metadata:test`. 2026-11-20 `1.18.4`(`f77c86d`) Windows full-only 설치본 `104,691,365 bytes`를 게시하고 공개 피드·signed manifest·Release HTTP 206을 확인했다. 과거 release 재귀 포함과 source map·미사용 플랫폼 binary를 차단한다. 실제 1.17.1 설치본의 업데이트 제안·설치 GUI는 `11` GAP-REL-17의 남은 게이트다. |
|
||||
| INFRA-15 | Update & release system | [x] | Canonical Forgejo feed + channels/policy, publisher `scripts/ci/publish-forgejo-release.mjs`; `npm run release:metadata:test`. `1.18.4`(`f77c86d`) Windows full-only 설치본 `104,691,365 bytes`를 게시했다. 2026-11-21 실제 1.17.1 설치본에서 제안 발견·다운로드·서명 검증·재시작 설치·기존 DB 열기를 확인했고 ProductVersion `1.18.4.0`과 실제 화면 OCR 사용을 검증했다. 과거 release 재귀 포함과 source map·미사용 플랫폼 binary를 차단한다. |
|
||||
| INFRA-16 | Desktop STT engine packaging | [x] | `apps/desktop/scripts/setup-sidecar.mjs` + `build-sidecar.mjs`, `electron-builder.yml` `extraResources` (`sidecar-dist/sidecar` → `resources/sidecar`, `resources/ffmpeg`), and `scripts/ci/verify-sidecar-bundle.mjs` run in `package-windows`/`package-macos` before electron-builder. Verified on the real bundle: `sidecar.exe` + `_internal` including `faster_whisper/assets/silero_vad_v6.onnx`, plus a packaged-engine transcription round-trip on GPU. |
|
||||
| INFRA-17 | 서명 없는 배포 채널 (portable + Scoop) | [x] | `scripts/ci/build-portable.mjs` (95MiB 7z 분할 볼륨 + Scoop 매니페스트), `scripts/ci/publish-portable-release.mjs`, `scripts/local/install-d3ro-voice.ps1`, `bucket/` 버킷, `.forgejo/workflows/portable.yml`; 7z 분할 볼륨(Scoop, 162MiB) + zip 분할 부품(수동 설치, 243MiB, 7-Zip 불필요); updater feed와 분리. 2026-09-18 `portable-1.3.1` 게시 + 실제 설치 검증. |
|
||||
| INFRA-18 | 로컬 런타임 온디맨드 설치 | [x] | `RuntimeProvisioner`(부품 다운로드 + SHA-256 검증 + tar 해제, `%APPDATA%/d3ro-voice/runtime`), `POST runtime:ensure` / `runtime:progress` IPC, 설정 > STT 상태/내려받기 UI. 설치본에서 엔진/ffmpeg를 분리해 189MB → 90.6MiB, 업데이트 피드 게시 복구. 검증은 전부 디스크에 기록된 파일 기준이며(부품 크기·해시 → 결합본 크기·해시), 부품 다운로드는 최대 3회 재시도한다. 2026-09-18 실제 feed 통합 검증(엔진 94.4MiB/18초, ffmpeg 21.7MiB/5초). |
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ Legend: `[ ]` open · `[~]` in progress · `[!]` blocked externally · `[x]` res
|
|||
| ID | Area | Gap | Evidence | Suggested next step |
|
||||
|---|---|---|---|---|
|
||||
| GAP-OCR-01 | Desktop | Screen text capture (CAP-17) has not been driven end-to-end in the running app: global hotkey → overlay takes focus → drag select → copy/PDF on a real desktop, including a mixed-DPI multi-monitor setup. Unit, real-engine and headless-overlay checks pass. | `apps/desktop/src/main/screen-ocr-wiring.ts`, `apps/desktop/src/main/windows/screen-ocr-overlay.ts`, `apps/desktop/tests/main/services/screen-ocr-service.test.ts` | Run the installed build: press `Ctrl+Alt+O` on each monitor, confirm the frozen image lines up with the real screen, Esc/Ctrl+C work without a first click, and a saved PDF is searchable. Then flip CAP-17 to `[x]`. |
|
||||
| GAP-OCR-02 | Desktop | Screen text capture is Windows-only and uses a single recognizer language per pass. macOS has no engine (fails closed with a message); with only the Korean Windows OCR pack installed, small English UI text is sometimes misread. | `apps/desktop/src/main/services/screen-ocr/windows-ocr-engine.ts` (`OcrEnginePort`) | Add a macOS Vision adapter behind `OcrEnginePort`; consider a second pass with another installed language for mixed-language screens, or a bundled ONNX recognizer downloaded at runtime. |
|
||||
| GAP-OCR-02 | Desktop | The default screen OCR remains Windows-only; macOS has no engine (fails closed). Korean Windows OCR misread symbols and English/code, while generative alternatives can still alter rare Hangul, punctuation and whitespace. The installed-app precise-reading journey and mixed real-screen evaluation are not verified. | `apps/desktop/src/main/services/screen-ocr/{windows-ocr-engine,ollama-precision-engine,precision-output}.ts`, `apps/desktop/src/main/services/ScreenOcrService.ts`, `packages/core/src/screen-ocr.ts` | `[~]` 2026-11-21: GLM-OCR precise reading is implemented for selected crops with loopback-only privacy, explicit model download, bounded output, original-text fallback and 51 focused tests. Official model/GGUF comparisons included Windows, PP-OCRv5 and PaddleOCR-VL 1.6; GLM was selected for code fidelity and crop latency, not the best overall prose CER. The real service preserved all 1,215 non-whitespace characters and 403 symbols in a code fixture but had three whitespace differences. **Next:** verify the precise button in an installed build and test held-out real screens; add macOS Vision and evaluate rare Hangul/whitespace separately. Do not claim universal exact OCR or ship Paddle/PyTorch dependencies based on synthetic scores alone. |
|
||||
| GAP-QA-01 | Quality | Extreme Red Team: headful end-to-end bug hunting across real desktop Electron, Web Next.js, and CI pipelines. | `red_team_log.md`, `apps/desktop/tests/e2e`, `apps/web/e2e/smoke.spec.ts` | `[x]` 2026-09-15: 18 scenarios executed, 14 defects caught and 100% resolved (infinite chunking loop DEF-008, IPC signature mismatch DEF-004, markdown editor typing rollback DEF-006, Web RSC Link serialization DEF-012, secret scanner lookahead DEF-013, etc.). All 18 scenarios GREEN with zero regressions. **2026-11-04 rework:** the red-team cycle specs were one-off bug hunts (machine screenshot paths, fixed sleeps, conditional assertions) and were replaced by 5 behavior-scoped protected journeys + a shared Electron fixture; 4 stale/duplicate feature specs, the orphaned root `tests/e2e`, and 7 ad-hoc Playwright runner scripts were deleted. Duplicates merged. New `npm run test:hygiene` gate blocks the patterns that made them meaningless. Log: `docs/tdd-red/06-repo-application.md`. |
|
||||
| GAP-REL-01 | Release | Official release publication to Forgejo and active public download center deployment. | `scripts/ci/publish-forgejo-release.mjs`, `apps/web/src/app/download/page.tsx`, `site/src/sections/Download.tsx`, `apps/web/e2e/smoke.spec.ts` | `[~]` 2026-09-15: v1.1.0 release assets (`D3RO-Voice-Setup-1.1.0-x64.exe`, `.blockmap`, `latest.yml`, `update-policy.json`) published to canonical Forgejo registry and release hub. 2026-09-16: the published 1.1.0 installer carries no Authenticode signature, so it does not satisfy the release policy; product version moved to `1.2.0` and publication must come from CI with the signing gate GREEN. Download centers in `apps/web` (`/download`) and `site` (`#download`) link the canonical Forgejo feed. 2026-09-19: `1.3.7` (overlay fix) is published to the canonical updater feed (`latest.yml` = 1.3.7, 90.6MiB); the CI signing gate that blocks tag-driven publication is still unresolved, so this went out through the local updater path (GAP-REL-06). 2026-09-23: `1.5.0` published the same way (updater feed `latest.yml` = 1.5.0, installer 91.2MiB / 95,612,108 bytes, sha512 matches the local build and the feed serves it anonymously); the runtime and portable channels were republished first. **2026-11-14 (`v1.17.1`):** Desktop `1.17.1` (`6c6875b`, `D3RO-Voice-Setup-1.17.1-x64.exe` `95,294,301` bytes + Ed25519 signed `latest.yml`/`.sig`), `runtime-1.17.1`/`runtime-latest`, `portable-1.17.1`/`portable-latest`, 및 Landing `site/` (`https://d3ro.chanpaca.net/release-identity.json` = `1.17.1`/`6c6875b`)가 프로덕션에 게시됐다(단 Authenticode SmartScreen 평판 `GAP-REL-02`/`06` 및 Release #339 ~91MB 첨부 `GAP-REL-16` 예외는 유지). |
|
||||
| GAP-REL-08 | Release | 다운로드 센터가 **존재하지 않는 설치 파일**을 가리켰다. `apps/web/src/lib/desktop-release.ts`와 `site/src/release.ts`의 `DESKTOP_VERSION`이 `1.2.0`에 멈춰 있어 설치 URL이 `D3RO-Voice-Setup-1.2.0-x64.exe`였고, 그 경로는 피드에서 404다(실측: 1.2.0=404, 1.3.7=206). `version:sync`가 이 두 표면을 덮지 않아 계속 어긋났다. | `scripts/ci/sync-version.mjs`, `apps/web/src/lib/desktop-release.ts`, `site/src/release.ts` | `[x]` 2026-09-19: 두 다운로드 계약 파일을 `sync-version.mjs` 대상에 추가해 버전·릴리스일이 SSOT에서 자동 반영되도록 하고, 현재 값(1.3.7 / 2026-09-19)으로 정정했다. `version:check`·typecheck·site 빌드 GREEN. |
|
||||
|
|
@ -72,7 +72,7 @@ Legend: `[ ]` open · `[~]` in progress · `[!]` blocked externally · `[x]` res
|
|||
| GAP-MTG-03 | Meetings | Existing recordings could become plain transcription history, but not a meeting with linked audio and minutes. | desktop `MeetingModeService`, `meeting/imported-recording.ts`, `meeting-mode-handlers.ts`; mobile `MeetingsScreen` → `RecordScreen`; MTG-13 | `[~]` 2026-11-02: installed Windows 1.16.0 passed native picker→timestamped transcript→local summary/action item with a real WAV (`2026-11-02-verification.md`). Desktop GUI gate is complete; authenticated Android picker→upload→transcript→minutes on Fold6 remains. |
|
||||
| GAP-MTG-04 | Meetings | Learned typing vocabulary never reached meeting transcription: STT used a hardcoded Korean hint, refine had no glossary, refined lines stopped inside `CaptionService`, and the stored transcript was raw segments. | `packages/core/src/transcript-correction.ts`, `apps/desktop/src/main/services/TranscriptGlossaryService.ts`, `CaptionService.ts`, `meeting/MeetingRecordingSession.ts`, `MeetingModeService.ts`, `FileTranscriptionService.ts`, `VoiceModeService.ts`; MTG-14 | `[~]` 2026-11-05: one glossary SSOT wired into caption, meeting, file transcription and dictation; refined lines now propagate to the meeting transcript and renderer (`meetingMode:segmentUpdated`). **Direct measurement against the real local database** (`users/_local/d3ro.db`: 1,952 learned phrases / 2,522 typing samples / 3 meeting transcripts) exposed that the first cut also rewrote ordinary Korean (286 of 3,005 words: 하면→라면, 이제→이게, 매일→메일, 정보→정도), and v1.16.3 restricts deterministic correction to dictionary pronunciations plus typed Latin identifiers — re-measured **0 of 3,005**. Real-engine check with the local sidecar (`large-v3-turbo`): a Korean TTS fixture containing the user term transcribes as 하람로그에 with no hint and haramlog에 with the learned hint; a dictionary pronunciation entry maps 디에로 → D3RO. Hangul-layout artifacts (dkssudgktpdy) are excluded from the hint. Evidence: `packages/core/__tests__/transcript-correction.test.ts` 16/16, `apps/desktop/tests/main/services/transcript-glossary.test.ts` 10/10, core 589/589, desktop offline suite, workspace typecheck, touched-file ESLint, `test:hygiene`. **Remaining:** a live meeting with real names/jargon by voice, and a packaged-install check of the new IPC event. |
|
||||
| GAP-REL-16 | Release | Forgejo release-page asset attachment for the ~91 MB installer timed out through Cloudflare (HTTP 524), although package registry upload and signed manifest succeeded. | `scripts/ci/publish-forgejo-release.mjs`, `scripts/ci/lib/immutable-package-guard.mjs` | `[x]` 2026-11-20: `v1.18.4`(`f77c86d`)의 `104,691,365-byte` 설치본·`latest.yml`·`.sig` Release 첨부가 모두 성공하고 공개 Release 설치본 Range GET도 HTTP 206이다. 과거 1.16.2~1.17.1의 524/ECONNRESET 이력과 달리 현재 경로는 완료했다. 증거: `docs/evidence/2026-11-17-backlog-operations.md`. |
|
||||
| GAP-REL-17 | Release | 설치된 1.17.1은 자동 업데이트를 정상 확인하지만 공개 피드도 1.17.1이라 1.18.x 제안이 나오지 않았다. 미게시 1.18.1·1.18.2 설치본이 각각 120,821,996·120,821,907 bytes(115.2 MiB)라 100 MiB 업로드 게이트에서 중단됐고, 버전 태그와 원격 피드가 어긋났다. | `apps/desktop/electron-builder.yml`, `scripts/ci/{build-desktop-installer,release-desktop,publish-forgejo-release}.mjs`, `docs/evidence/2026-11-17-backlog-operations.md` | `[~]` 2026-11-20: `1.18.4`(`f77c86d`)는 `!release/**`·명시적 `out/**/*`, source map·미사용 플랫폼 바이너리 제외로 `104,691,365 bytes`가 됐다. Windows는 `.blockmap` 대신 전체 설치본을 받는다(`fullInstallVersionGap=0`). 빌드·native DB·smoke(3 valid/10 negative)·공개 피드·signed manifest·Release 자산 모두 GREEN. `.env` Forgejo token 401은 남아 있고 이번 게시는 유효한 기존 Git credential을 프로세스 메모리에서만 사용했다. **남은 것:** 1.17.1 실제 앱의 설정→정보→업데이트 제안·다운로드·재시작 GUI 확인. |
|
||||
| GAP-REL-17 | Release | 설치된 1.17.1은 자동 업데이트를 정상 확인하지만 공개 피드도 1.17.1이라 1.18.x 제안이 나오지 않았다. 미게시 1.18.1·1.18.2 설치본이 각각 120,821,996·120,821,907 bytes(115.2 MiB)라 100 MiB 업로드 게이트에서 중단됐고, 버전 태그와 원격 피드가 어긋났다. | `apps/desktop/electron-builder.yml`, `scripts/ci/{build-desktop-installer,release-desktop,publish-forgejo-release}.mjs`, `docs/evidence/2026-11-17-backlog-operations.md` | `[x]` 2026-11-21: `1.18.4`(`f77c86d`, `104,691,365 bytes`)의 제안 발견·전체 설치본 다운로드·Ed25519 서명 검증·`quitAndInstall`·재시작·기존 DB 열기를 실제 설치 앱 로그로 확인했다. ProductVersion `1.18.4.0`, 이후 최신 버전 확인 및 실제 화면 OCR 사용도 검증했다. Windows full-only와 packaging exclusions는 유지한다. `.env` Forgejo token 401은 별도 교체 대상이며 이번 게시는 유효한 기존 Git credential을 프로세스 메모리에서만 사용했다. |
|
||||
| GAP-INFRA-01 | Build | `apps/mobile-rn` is outside npm workspaces, so root `typecheck`/`test`/`lint` skip it. `typecheck:mobile`/`lint:mobile`/`test:mobile`/`verify:all` root scripts added 2026-09-13 (`package.json`), but membership/CI integration is still open. | `package.json`, `.forgejo/workflows/ci.yml`, `.gitlab-ci.yml` | `[x]` 2026-10-30: React Native Metro/Gradle 네이티브 오토링킹 격리를 위해 `apps/mobile-rn`은 독립 `node_modules`로 유지하고, 루트 `verify:all` 및 `.forgejo/workflows/ci.yml`·`.gitlab-ci.yml`의 `mobile-quality` 잡이 모바일 `lint`·`typecheck`·`test:ci`를 필수 게이트로 실행하도록 정본화했다. **2026-11-04:** root `test:mobile`도 watch-capable 기본 Jest가 아니라 mobile `test:ci`(`jest --runInBand`)를 호출하도록 고정했다. |
|
||||
| GAP-MOB-01 | Legacy | `apps/mobile` Expo skeleton duplicated auth/data code and got version-sync edits. | SSOT G-002 | `[x]` 2026-09-13: deleted `apps/mobile`, removed from `sync-version.mjs` and `package-lock.json`; `version:check` GREEN. |
|
||||
| GAP-INFRA-02 | CI | GitLab admin NAS deploy job is disabled; admin deploy is manual/GHCR. | `.gitlab-ci.yml`, `docs/deployment/nas-deploy-runbook.md` | `[x]` 2026-10-30: 임시 CI DinD 데몬만 바꾸던 구 잡 대신 `docker-compose.nas.yml` 기반의 정본 배포·헬스체크·롤백 절차를 `docs/deployment/nas-deploy-runbook.md`로 문서화하고 `.gitlab-ci.yml` 및 `02-infrastructure.md`에 연결했다. |
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue