feat(desktop): screen text capture (OCR) overlay with selectable text

A rebindable screen-ocr action (default Ctrl+Alt+O) freezes the monitor under the cursor and
covers it with a focusable overlay. Text is selected by flow drag, rectangle drag, double/triple
click, Shift+click or Ctrl+A, then copied, saved as PNG or searchable PDF, translated,
summarized, read aloud, opened as a link or searched.

- Selection, text assembly and the PDF document are pure functions in @d3ro/core/screen-ocr,
  shared by the main process and the overlay.
- Recognition uses Windows.Media.Ocr through powershell.exe -EncodedCommand with 2x upscaling;
  the capture reaches disk only as a temp PNG that is removed right after recognition.
- Capture uses GDI BitBlt for the target monitor (39 ms) because desktopCapturer captured all
  three monitors and took 2.9 s when DXGI duplication timed out.
- The accelerator is reserved only where the feature is available, so macOS does not swallow
  the key for a feature it cannot run (fails closed with a message).

Map: CAP-17 [~], GAP-OCR-01 (live hotkey journey not yet driven in the running app),
GAP-OCR-02 (macOS engine, mixed-language accuracy).
This commit is contained in:
Yun Chan 2026-11-20 15:16:37 +09:00
parent 6791e29f27
commit 03cd8bf890
41 changed files with 4561 additions and 17 deletions

View file

@ -2500,5 +2500,46 @@
"web.push.unconfigured": "Notifications are not configured on the server.",
"web.push.blocked": "Notifications are blocked. Allow them in this site's browser settings.",
"web.push.revoked": "This browser was removed from the account. Clear this site's stored data before adding it again.",
"web.push.error": "Notifications could not be changed. Check the browser permission and try again."
"web.push.error": "Notifications could not be changed. Check the browser permission and try again.",
"keybinding.action.screenOcr": "Screen text capture (OCR)",
"keybinding.action.screenOcr.desc": "Captures the screen so you can drag to select the text on it.",
"screenOcr.dialog.savePdf": "Save as PDF",
"screenOcr.dialog.saveImage": "Save as image",
"screenOcr.errorCapture": "Could not capture the screen.",
"screenOcr.modeText": "Text",
"screenOcr.modeRect": "Area",
"screenOcr.copy": "Copy",
"screenOcr.copyLine": "Copy as one line",
"screenOcr.copyImage": "Copy image",
"screenOcr.saveImage": "Save image",
"screenOcr.savePdf": "Save PDF",
"screenOcr.translate": "Translate",
"screenOcr.summarize": "Summarize",
"screenOcr.speak": "Read aloud",
"screenOcr.stopSpeak": "Stop reading",
"screenOcr.openLink": "Open link",
"screenOcr.search": "Search the web",
"screenOcr.selectAll": "Select all",
"screenOcr.close": "Close",
"screenOcr.language": "Language",
"screenOcr.languageAuto": "Auto",
"screenOcr.statusRecognizing": "Reading text…",
"screenOcr.statusReady": "{{count}} words · drag to select",
"screenOcr.statusEmpty": "No text found",
"screenOcr.statusSelected": "{{count}} characters selected",
"screenOcr.hint": "Drag to select · double-click a word · Ctrl+A all · Ctrl+C copy · Esc close",
"screenOcr.toastCopied": "Copied",
"screenOcr.toastImageCopied": "Image copied",
"screenOcr.toastSaved": "Saved",
"screenOcr.working": "Working…",
"screenOcr.resultTranslate": "Translation",
"screenOcr.resultSummary": "Summary",
"screenOcr.errorRecognition": "Could not read the text",
"screenOcr.errorUnavailable": "Screen text capture is not available on this PC. Install a language pack with optical character recognition in Windows Settings.",
"screenOcr.errorExport": "Could not save or copy",
"screenOcr.errorLlm": "No response from the AI. Check your model settings.",
"screenOcr.errorTts": "Could not read aloud",
"screenOcr.errorLink": "Could not open the link",
"screenOcr.errorInvalid": "This capture is already closed. Capture again.",
"screenOcr.errorEmpty": "No text selected"
}