# 04 — Desktop App (Electron) Map > Surface: `apps/desktop` > Stack: Electron 33 + React 19 + MUI 7 + Vite (electron-vite) + better-sqlite3/drizzle + uiohook-napi + nut-js > Source root: `apps/desktop/src` (`main/`, `preload/`, `renderer/`) --- ## 1. Process architecture | Layer | Path | Contents | |---|---|---| | Main | `src/main/` | Services, IPC handlers, windows, bootstrap/lifecycle, DB | | Preload | `src/preload/` | `index.ts` exposes `window.electronAPI`; `popup.ts` exposes `window.popupAPI` | | Renderer | `src/renderer/` | React app: `AppLayout` + 7 pages + modals + 5 vanilla popups | **Main entry** `src/main/index.ts`: sets app name/AppUserModelId, disables GPU acceleration, EPIPE/uncaught handlers, registers `d3ro-voice://` deep-link protocol (Supabase OAuth implicit + PKCE), single-instance lock, then `bootstrap()` + `setupLifecycle()`. The explicit manual-QA argument `--preview-update-dialogs` is isolated before protocol registration/bootstrap: it bypasses the single-instance lock, invokes the real update-dialog prompter for four consent/restart variants, touches no feed/download/install/settings state, then quits. This path is unit-tested, but its packaged native dialogs still require an external-terminal visual run. **Bootstrap** `src/main/bootstrap.ts`: ordered `BootstrapStep[]` — logger, config, **database (critical)**, license, create-windows (critical), tray, **ipc-handlers (critical)**, custom-instructions, voice-commands, sound-effects, auto-launch, popup-preload, key-bindings, voice-mode, stt-warmup, llm-polling, meeting-summary-wiring, meeting-mode, cloud-sync, auto-update. Wires VoiceMode events to sound + history persistence, and subscribes to `KeyBindingService` `triggered` for the `history-popup` / `command-popup` actions (`bootstrap.ts:159`) — those two were hardcoded accelerators before and are now rebindable like everything else. --- ## 2. Main services (`src/main/services/`) Singleton + `EventEmitter` pattern (`getXService()` accessors). ### Core voice pipeline | Service | Purpose | |---|---| | `VoiceModeService` | Orchestrator: 9-state `RecognitionState` + 4-state `AudioState`, dual-condition flush, action queue. Events: session-started/completed/cancelled, transcription-update, audio-level, recognition/audio-state-changed, premium-llm-fallback, error | | `AudioCaptureService` | Mic PCM16 16kHz mono (bundled SoX on Windows, node-record-lpcm16 elsewhere). Spawns hidden (`windowsHide`); a missing SoX fails with the exact fix command | | `LocalSTTService` | faster-whisper Python sidecar manager (state machine, dual-flush, model download/cancel, background warm-up, live partial transcription). Connects over IPv4 loopback (`getSidecarBaseUrl`) and fails fast with an actionable message when the bundled engine or virtualenv is missing | | `KeyBindingService` | uiohook-napi global hooking for **keyboard and mouse**, driven by the `@d3ro/core/keybinding` contract: 6 rebindable actions (dictation, hands-free, command, caption, history-popup, command-popup), several bindings per action, structural reserved-combo checks. Events: `triggered` (in-process payload carries `actionId`, `type` (`pressed`/`released`), `isDoublePress`, `holdMode`, `timestamp`; the renderer-facing `keybinding:triggered` event is the narrower `KeyBindingTriggeredEvent`, `keybinding.ts:1092`), `changed`, `error`. `globalShortcut` is used only to mute the macOS system beep, and only for accelerators it registered itself. Mouse events cannot be suppressed by uiohook, so a bound button also performs its native action | | `TextInsertService` | Clipboard save→set→Ctrl+V→restore via nut-js | | `SoundEffectService` | Preloaded WAV feedback (start/stop/error/cancel/chime) | ### STT engine layer (`services/stt/`) | File | Purpose | |---|---| | `STTManager` | Dispatcher across local + 6 cloud providers, auto-fallback (events provider-changed, config-changed, fallback-to-local). `transcribePartial`/`warmUpLocal` route to the local engine only | | `types.ts` | `ISTTDriver` contract | | `audio-utils.ts` | `pcmToWav`, `createProbeWav` | | `drivers/OpenAI|Groq|Deepgram|AssemblyAI|Google|Custom|D3ROCloud` | Provider drivers; `D3ROCloudDriver` uses Supabase access token | ### LLM layer | Service | Purpose | |---|---| | `LocalLLMService` | Ollama REST (models, pull w/ progress, server start, NDJSON streaming). The 2026-09-22 runaway guard gives each request its own `AbortController` (external caller signals are relayed and detached on completion), requires `done` before an NDJSON stream succeeds, and clears incomplete streams. Generation/stream requests are bounded to 2048 tokens / 120 s; chat is bounded to 512 / 60 s. Ollama server spawn and polling are deduplicated, and lifecycle dispose terminates owned work. | | `PremiumLLMService` | Claude via Supabase `llm-proxy`, local fallback | | `OnlineLLMService` | JWT-authenticated .NET backend client | | `llm-prompts.ts` | **SSOT for prompt resolution, placeholder substitution, and argument placement.** `resolveSystemPrompt` (`:118`) maps an `LLMAction` to its base prompt and handles `custom` explicitly instead of dropping silently to `refine`. `renderInstructionPrompt` (`:78`) substitutes `{{text}}` / `{{userPrompt}}` / `{{targetLanguage}}` and **warns by name** for any placeholder left standing rather than letting it reach the model. `buildInstructionInvocation` (`:101`) decides where an instruction goes in `processText(text, action, targetLanguage, customPrompt)`: the instruction becomes the **system prompt** and the transcript the **text**, except for instructions that spell out `{{text}}`, which keep the old meaning for backward compatibility. `resolveTargetLanguage` (`:50`) is the one place translate targets are decided (still `English`, see `11` GAP-LLM-01). **All three LLM entry paths call the same functions** — `VoiceModeService` (`:838`), `ChainService` (`:196`), and the `LLM.PROCESS` IPC handler (`llm-handlers.ts:96`) — so no caller re-implements the rules | ### Memory & knowledge | Service | Purpose | |---|---| | `HistoryService` | SQLite history CRUD/search/stats | | `UnifiedSearchService` | Read-only SQLite search across history title/content and meeting title/transcript; merges newest-first results without changing either schema | | `DictionaryService` | Custom vocabulary CRUD/search + cloud sync hooks + JSON/CSV import/export (`dictionary:import`/`export`, save/open dialogs) | | `MemoService` | Memo tags over history (`memo_tags`) | | `RAGService` | Local RAG: `nomic-embed-text` embeddings, cosine search over `rag_chunks` | | `CustomInstructionService` | User LLM commands (5 built-ins) | | `VoiceCommandService` | Keyword → command rule matching | | `ChainService` | Multi-step LLM pipelines (LLMChain). Each step resolves its instruction through `llm-prompts.ts` (`ChainService.ts:196`); before that, chain steps sent placeholders through unsubstituted | | `ScreenContextService` | Active-window + selected-text context | ### Phase 10+ features | Service | Purpose | |---|---| | `CaptionService` | Live captions from system/loopback audio; caption overlay (events segment, segment-updated, state-changed, session-saved, error). Since 2026-11-05 every recognition call carries the learned-vocabulary STT hint, each final line passes the deterministic glossary correction, and the LLM refine prompt receives the user-term/style block | | `TranscriptGlossaryService` | Learned-vocabulary gateway for transcription (`@d3ro/core/transcript-correction`): merges user dictionary words + pronunciations and personal-graph phrases into a 15 s cached glossary, and exposes the STT `initial_prompt` hint, deterministic word correction (dictionary variants + 1-jamo learned-term match that never touches words the user actually wrote), and the LLM 교열 reference block. Sources are injectable; DB failures fall back to an empty glossary so transcription never blocks | | `FileTranscriptionService` | Audio/video file → ffmpeg → 30s chunks → STT merge (events progress, complete, error, state-changed). Shared media-format SSOT is `file-transcription/media-formats.ts`; meeting import can skip duplicate history persistence and retain the normalized WAV | | `MeetingSummaryService` | Post-caption LLM summary | | `DictationTemplateService` | Field-by-field voice form filling | | `VoiceConversationService` | STT→LLM→TTS loop, 10-turn memory. Local conversations are single-flight and pass their request signal through to local LLM chat, so a voice cancel aborts its own active request rather than only changing UI/watchdog state. | | `TTSPlaybackService` | Platform TTS (macOS `say`, Windows SAPI), sentence queue | | `VoiceActionService` | Voice → LLM JSON action plan → OS execution (dangerous blocked) | ### Phase 12–15 | Service | Purpose | |---|---| | `MeetingModeService` | Meeting recording and existing-media import: live/file transcript, normalized meeting audio, timestamp memos, doc generation/export, diarization. `meeting/imported-recording.ts` maps file-transcription segments to the canonical meeting transcript. Account revision checks before/after WAV copy and after transcription reject even A→B→A changes; unsuccessful imports remove only their own generated audio. Since 2026-11-05 refined caption lines reach the meeting transcript (`meetingMode:segmentUpdated`) instead of the raw text, and auto-polish receives the learned-vocabulary block | | `MeetingDocTemplateService` | Meeting-doc templates (built-ins + CRUD) | ### Input intelligence (2026-09-21; Local Flow Intelligence extension 2026-09-22) | Service / file | Purpose | |---|---| | `InputTelemetryService` | Global input telemetry and context observer: keystroke/click/scroll counters, mouse travel (Manhattan sum of per-axis deltas), active time, per-hour-per-app buckets flushed every 5 s; typed-text learning via UIA snapshot diffs. **Since 2026-10-26**, the UIA context observer runs by default whenever `typoAutoCorrectEnabled !== false` (or `suggestionEnabled === true`), while SQLite `input_activity` persistence only runs when `inputTelemetryEnabled === true` (opt-in). **Since 2026-10-27**, includes `TerminalKeystrokeTracker` (`@d3ro/core/input-intelligence`) + `isForegroundKoreanIme` (`utils/win32-foreground.ts`) to reconstruct the active terminal/Vim line (`reconcileWithScreen` for UIA screen buffers like Windows Terminal/pwsh/cmd, `reconcileWithXtermValue` for `xterm.js` `