ae7efb6acf
Popup pages loaded their scripts as classic <script src> tags, which the
renderer build never bundles, so an installed app rendered only the static
markup: the recording tip stayed at 0:00 with no wave bars and live captions
showed nothing.
- declare popup scripts as modules so the build emits them, and fail
packaging when a renderer page references an asset that was never produced
- hold popup IPC until the renderer has loaded and re-assert visibility on
every show, so a popup hidden once still appears next time
- surface popup renderer console and load failures in the main log
💘 Generated with Crush
Assisted-by: Crush:deepseek-v4.1-flash
14 KiB
04 — Desktop App (Electron) Map
Surface:
apps/desktopStack: Electron 33 + React 19 + MUI 7 + Vite (electron-vite) + better-sqlite3/drizzle + uiohook-napi + nut-js Source root:apps/desktop/src(main/,preload/,renderer/)
1. Process architecture
| Layer | Path | Contents |
|---|---|---|
| Main | src/main/ |
Services, IPC handlers, windows, bootstrap/lifecycle, DB |
| Preload | src/preload/ |
index.ts exposes window.electronAPI; popup.ts exposes window.popupAPI |
| Renderer | src/renderer/ |
React app: AppLayout + 7 pages + modals + 5 vanilla popups |
Main entry src/main/index.ts: sets app name/AppUserModelId, disables GPU acceleration, EPIPE/uncaught handlers, registers d3ro-voice:// deep-link protocol (Supabase OAuth implicit + PKCE), single-instance lock, then bootstrap() + setupLifecycle().
Bootstrap src/main/bootstrap.ts: ordered BootstrapStep[] — logger, config, database (critical), license, create-windows (critical), tray, ipc-handlers (critical), custom-instructions, voice-commands, sound-effects, auto-launch, popup-preload, hotkey, voice-mode, llm-polling, meeting-summary-wiring, meeting-mode, cloud-sync, auto-update. Wires VoiceMode events to sound + history persistence.
2. Main services (src/main/services/)
Singleton + EventEmitter pattern (getXService() accessors).
Core voice pipeline
| Service | Purpose |
|---|---|
VoiceModeService |
Orchestrator: 9-state RecognitionState + 4-state AudioState, dual-condition flush, action queue. Events: session-started/completed/cancelled, transcription-update, audio-level, recognition/audio-state-changed, premium-llm-fallback, error |
AudioCaptureService |
Mic PCM16 16kHz mono (bundled SoX on Windows, node-record-lpcm16 elsewhere). Spawns hidden (windowsHide); a missing SoX fails with the exact fix command |
LocalSTTService |
faster-whisper Python sidecar manager (state machine, dual-flush, model download/cancel, background warm-up, live partial transcription). Connects over IPv4 loopback (getSidecarBaseUrl) and fails fast with an actionable message when the bundled engine or virtualenv is missing |
HotkeyService |
uiohook-napi global hooking (dictation/hands-free/command/caption). Events: hotkey-pressed/released, double-press, error |
TextInsertService |
Clipboard save→set→Ctrl+V→restore via nut-js |
SoundEffectService |
Preloaded WAV feedback (start/stop/error/cancel/chime) |
STT engine layer (services/stt/)
| File | Purpose |
|---|---|
STTManager |
Dispatcher across local + 6 cloud providers, auto-fallback (events provider-changed, config-changed, fallback-to-local). transcribePartial/warmUpLocal route to the local engine only |
types.ts |
ISTTDriver contract |
audio-utils.ts |
pcmToWav, createProbeWav |
| `drivers/OpenAI | Groq |
LLM layer
| Service | Purpose |
|---|---|
LocalLLMService |
Ollama REST (models, pull w/ progress, server start, NDJSON streaming) |
PremiumLLMService |
Claude via Supabase llm-proxy, local fallback |
OnlineLLMService |
JWT-authenticated .NET backend client |
llm-prompts.ts |
resolveSystemPrompt SSOT for action prompts |
Memory & knowledge
| Service | Purpose |
|---|---|
HistoryService |
SQLite history CRUD/search/stats |
DictionaryService |
Custom vocabulary CRUD/search + cloud sync hooks + JSON/CSV import/export (dictionary:import/export, save/open dialogs) |
MemoService |
Memo tags over history (memo_tags) |
RAGService |
Local RAG: nomic-embed-text embeddings, cosine search over rag_chunks |
CustomInstructionService |
User LLM commands (5 built-ins) |
VoiceCommandService |
Keyword → command rule matching |
ChainService |
Multi-step LLM pipelines (LLMChain) |
ScreenContextService |
Active-window + selected-text context |
Phase 10+ features
| Service | Purpose |
|---|---|
CaptionService |
Live captions from system/loopback audio; caption overlay (events segment, state-changed, session-saved, error) |
FileTranscriptionService |
Audio/video file → ffmpeg → 30s chunks → STT merge (events progress, complete, error, state-changed) |
MeetingSummaryService |
Post-caption LLM summary |
DictationTemplateService |
Field-by-field voice form filling |
VoiceConversationService |
STT→LLM→TTS loop, 10-turn memory |
TTSPlaybackService |
Platform TTS (macOS say, Windows SAPI), sentence queue |
VoiceActionService |
Voice → LLM JSON action plan → OS execution (dangerous blocked) |
Phase 12–15
| Service | Purpose |
|---|---|
MeetingModeService |
Meeting recording: live transcript, timestamp memos, doc generation/export, diarization |
MeetingDocTemplateService |
Meeting-doc templates (built-ins + CRUD) |
Account / infra / monetization
| Service | Purpose |
|---|---|
ConfigService |
electron-store AppConfig (configGet/Set, defaults) |
LicenseService |
Freemium tiers, quotas (daily_usage), activation, upgrade prompts |
CloudSyncService |
Supabase sync, per-user DB switching, history/dictionary/meeting mirror |
CloudSTTService |
Thin cloud STT wrapper over D3ROCloudDriver |
UpdateService |
electron-updater (canonical Forgejo feed, channels, mandatory/full-vs-delta policy, staged rollout, restart dialog) |
AutoLaunchService |
OS login-item auto-start |
LoggerService |
electron-log wrapper + category loggers |
checkout/payment |
payment-handlers.ts — authenticated Edge-only Stripe/Payple checkout + server readback |
Ads (services/ads/)
| File | Purpose |
|---|---|
AdMediationEngine |
Multi-ad mediation + header bidding |
AdSettlementService |
Revenue settlement, withholding, payout ledger |
BaseAdAdapter / UnavailableAdAdapter |
Adapter contract + fail-closed base |
DirectHouseSponsorAdapter |
Real configurable adapter: bids/reports against an operator HTTPS endpointUrl (AdNetworkConfig.endpointUrl), validates creatives, fail-closed (adapter_not_configured) when unconfigured |
| 9 placeholder adapters (AppLovin, Carbon, EthicalAds, GoogleAdManager, InMobi, Mintegral, Playwire, PubMatic, Unity) | Extend UnavailableAdAdapter — registered, no live bids (provider_not_integrated) |
3. IPC layer
Registry: src/main/ipc/index.ts calls 29 registerXHandlers() in fixed order. Channel SSOT: packages/core/src/ipc-channels.ts.
| Handler | Channel group(s) |
|---|---|
ads-handlers |
ADS |
audio-handlers |
AUDIO |
caption-handlers |
CAPTION + SYSTEM_AUDIO |
chain-handlers |
CHAIN |
cloud-sync-handlers |
CLOUD_SYNC |
config-handlers |
CONFIG |
context-handlers |
CONTEXT |
dictionary-handlers |
DICTIONARY |
file-transcription-handlers |
FILE_TRANSCRIPTION |
history-handlers |
HISTORY + stats:getSummary |
hotkey-handlers |
HOTKEY |
instruction-handlers |
INSTRUCTION |
license-handlers |
LICENSE |
llm-handlers |
LLM + llm:premium:* + ONLINE_AUTH |
meeting-doc-template-handlers |
MEETING_DOC_TEMPLATE |
meeting-mode-handlers |
MEETING_MODE + MEETING_CHAT |
meeting-summary-handlers |
MEETING_SUMMARY |
memo-handlers |
MEMO |
payment-handlers |
PAYMENT |
rag-handlers |
RAG |
stt-handlers |
STT |
support-handlers |
SUPPORT |
system-handlers |
SYSTEM |
template-handlers |
DICTATION_TEMPLATE |
voice-action-handlers |
VOICE_ACTION |
voice-command-handlers |
VOICE_COMMAND |
voice-conversation-handlers |
VOICE_CONVERSATION |
voice-handlers |
VOICE |
window-handlers |
WINDOW + SYSTEM.OPEN_EXTERNAL |
Preload exposes window.electronAPI with 33 namespaces: platform, audio, config, voice, stt, hotkey, llm (incl. premium), history, dictionary, stats, window, system, instruction, app, memo, voiceCommand, context, chain, caption, license, fileTranscription, meetingSummary, dictationTemplate, rag, voiceAction, voiceConversation, meetingMode, meetingChat, meetingDocTemplate, cloudSync, onlineAuth, ads, support, payment. Envelope: IPCResult<T> (success/error); app.onDataChanged is the global refresh channel.
4. Windows & popups
windows/WindowManager.ts creates 6 windows: main (borderless, custom TitleBar; macOS hiddenInset), recording-tip, result-popup, history-popup, command-popup, caption-overlay. Injects popup theme CSS + i18n strings; 2-phase resize. windows/TrayManager.ts — tray icon + menu + double-click show.
Popup invariants (each shipped broken once — do not regress):
- 팝업 HTML의 스크립트는 반드시
<script type="module">로 선언한다. Vite는 모듈 스크립트만 번들에 포함하므로 classic<script src="./script.js">는 dev에서만 로드되고 패키징 산출물에서는 파일 자체가 사라진다(오버레이가 정적 HTML로 멈춘 원인).scripts/ci/verify-desktop-renderer-bundles.mjs가 빌드 HTML이 참조하는 모든 로컬 asset의 존재를 검사한다. - 렌더러 로드 전의
webContents.send는 조용히 버려진다. 팝업 전송은sendToPopupWindow를 쓰고, 이 함수가did-finish-load까지 메시지를 보관했다가 전달한다.attachPopupLifecycle이 로드 상태 추적·테마 주입·팝업 렌더러 진단 로그를 한 곳에서 묶는다. - 팝업 표시는
presentPopup으로 통일한다(showInactive+ topmost 재선언 +moveTop+webContents.invalidate). 한 번hide()된 팝업이 두 번째 표시에서 z-order/repaint를 잃어 보이지 않던 문제를 막는다.
Vanilla popups (src/renderer/popups/):
| Popup | Purpose |
|---|---|
recording-tip |
9-bar waveform indicator, partial transcript |
result-popup |
Transcription result + copy, auto-close with hover pause |
history-popup |
Recent transcriptions; ↑↓/Enter/1-9/ESC |
command-popup |
Command selection (Ctrl+Shift+C) |
caption-overlay |
Live caption overlay (font/opacity/maxLines) |
5. Renderer IA
Routing is state-based in AppLayout.tsx (Route union + NAV_ITEMS), no react-router.
| Page | Route | Feature |
|---|---|---|
DashboardPage |
dashboard | Voice cockpit: hero, bento tiles, multi-engine hub (STT/LLM), telemetry, recent history, file drop |
HistoryPage |
history | History & memory timeline; search, tag filter, pagination, export/delete |
DictionaryPage |
dictionary | Custom vocabulary editor |
CommandsPage |
commands | Custom instructions + voice keyword rules + LLM chains + dictation templates |
VoiceConversationPage |
conversation | Duplex voice assistant (local pipeline vs OpenAI Realtime) |
KnowledgeBasePage |
knowledge | Local RAG: add/index docs, semantic query, reindex/remove |
MeetingModePage |
meeting | Meeting studio: live transcript, memos, doc generation/export, diarization |
Modals/components: SettingsModal (tabs General/Audio/STT/LLM/License/Cloud/About), LicenseModal, LicenseTab, CloudSyncSection, OnboardingModal, UpgradePromptModal, ProBadge, TemplateSection, FileDropZone, HotkeyRecordModal, OllamaGuideModal, CodexOAuthGuideModal, TitleBar, StatusBar, meeting components (9), voice-conversation, payment (CheckoutModal, checkout-flow.ts), support (SupportModal), ads (AdBanner, RewardedQuotaModal), shared cards.
Hooks: useRealtimeConversation (OpenAI Realtime WebRTC), useLicenseState, useProFeature.
DB schema (src/main/db/schema.ts, drizzle SQLite): history, dictionary, stats, memo_tags, daily_usage, rag_documents, rag_chunks, meeting_sessions, meeting_memos, meeting_documents.
6. Desktop status summary
- Core dictation/LLM/history pipeline: implemented + tested (~590 desktop tests; vitest + playwright).
- Cross-platform packaging: Windows NSIS (signed,
forceCodeSigning), macOS DMG/ZIP arm64 (ad-hoc signing); auto-update via canonical Forgejo feed with update policy (release/update-policy.json). - Local-first AI (SoX + faster-whisper sidecar + bundled Ollama) and cloud paths both present.
- Local STT is packaged (
1.3.0):electron-builder.ymlextraResourcescopiessidecar-dist/sidecar→resources/sidecarandresources/ffmpeg→resources/ffmpeg;scripts/ci/verify-sidecar-bundle.mjsgates packaging. Build locally withnpm --prefix apps/desktop run sidecar:setup && npm --prefix apps/desktop run sidecar:build. The sidecar stays in console mode sostdout/stderrreach the app log (UTF-8, line-buffered); a packaged sidecar must exist or startup fails loudly instead of silently falling back to a system Python. - All local engine URLs (
LocalSTTService,LocalLLMService,RAGService,OnlineLLMService,STTManager) pass throughsrc/main/utils/loopback.ts, which rewriteslocalhostto127.0.0.1, because some Windows hosts resolvelocalhostto IPv6 only and local engines bind IPv4. - Meeting intelligence, RAG, voice conversation (local + Realtime), captions, file transcription: implemented.
- Ad mediation:
DirectHouseSponsorAdapterperforms real configurable REST bids; the other 9 adapters remain fail-closed stubs pending official SDKs (see11-gap-backlog.mdGAP-ADS-01/02). - Tier resolution now routes through
@d3ro/core/entitlement(resolveEntitlement,normalizeEntitlementTier);useLicenseState.isProincludespro_plus. - No
TODO/FIXMEmarkers found insrc(grep clean).src/main/types/is an empty directory.
7. Key file anchors
| Thing | Path |
|---|---|
| App entry / deep links | src/main/index.ts |
| Bootstrap order | src/main/bootstrap.ts |
| IPC registry | src/main/ipc/index.ts |
| IPC channel SSOT | packages/core/src/ipc-channels.ts |
| Preload API | src/preload/index.ts |
| Windows | src/main/windows/WindowManager.ts |
| Voice orchestrator | src/main/services/VoiceModeService.ts |
| DB schema | src/main/db/schema.ts |
| Renderer shell / routes | src/renderer/components/AppLayout.tsx |
| Update feed SSOT | src/main/update-feed.ts |
| Update policy SSOT | release/update-policy.json + src/main/update-policy.ts |
| Path/loopback resolution | src/main/utils/paths.ts, src/main/utils/loopback.ts |
| Sidecar source / packaging | sidecar/main.py, scripts/setup-sidecar.mjs, scripts/build-sidecar.mjs, scripts/ci/verify-sidecar-bundle.mjs |