Phase 12~13 전체 구현: Pro+ 피처 6종 + 음성 대화 + RAG + OS 자동화

Phase 12:
- FileTranscriptionService: ffmpeg PCM 변환 + 30초 청크 순차 STT
- MeetingSummaryService: 자막 세션 → LLM 자동 요약 + DB summaryText
- DictationTemplateService: 필드별 음성 입력 상태 머신 + 프리셋 3개

Phase 13.1:
- VoiceConversationService: STT→Ollama /api/chat→TTS 대화 루프 (10턴)
- TTSPlaybackService: Windows SAPI 문장 단위 큐 재생
- LocalLLMService.chatStream: Ollama /api/chat 스트리밍

Phase 13.2:
- RAGService: Ollama 임베딩 + SQLite 벡터 + 코사인 유사도 검색
- KnowledgeBasePage: 문서 관리 + 질문/답변 UI
- PDF 파서: zlib FlateDecode 해제 + BT/ET 텍스트 추출

Phase 13.3:
- VoiceActionService: LLM JSON 액션 플랜 생성 + 실행
- 프리셋 6개 (크롬/메모장/탐색기/볼륨), 위험 명령 차단

공통: IPC ~70채널, 에러코드 780-878, i18n 100+키
버그픽스: 라이선스 로컬 키 우선, i18n featureLabel, DOM 중첩
This commit is contained in:
Yun Chan 2026-04-05 23:52:14 +09:00
parent a31f96bbb8
commit eb83682269
38 changed files with 5678 additions and 19 deletions

View file

@ -77,11 +77,20 @@ npm run typecheck # tsc --noEmit
6. **녹음 UI**: 9개 웨이브 바, cos 분포 가중치, 100ms 애니메이션 6. **녹음 UI**: 9개 웨이브 바, cos 분포 가중치, 100ms 애니메이션
## 현재 상태 ## 현재 상태
Phase: 11 완료 (Phase 1~11 전체 완료) Phase: 13 전체 완료 + 랜딩 페이지 완성 (Phase 1~13 + site/)
마지막 작업: LicenseService + Feature Gate + LicenseModal + ProBadge + UpgradePrompt + 일일 쿼터 + LemonSqueezy API 마지막 작업: 랜딩 페이지 디자인 통합 + 10개 국어 i18n
다음 작업: Phase 12 랜딩 페이지 & 마케팅 또는 빌드/배포 안정화 다음 작업: 빌드/배포 안정화, E2E 테스트, 또는 새 피처
차단 이슈: @nut-tree-fork/nut-js 포크 사용 차단 이슈: @nut-tree-fork/nut-js 포크 사용
### 랜딩 페이지 (site/)
- Vite + React 19 + TypeScript + Tailwind CSS
- 4가지 디자인 레퍼런스 통합 (KKG 3종 + Meteorological → D3RO 인더스트리얼)
- SSOT: tokens.ts + tailwind.config.js 토큰 중앙 관리
- 재사용 컴포넌트 8개: Badge/Container/Crosshair/DataPoint/GlowButton/InstrumentCard/Led/SectionHeader
- CSS 이펙트: CRT scanline, crosshair, noise texture, glow button (WebGL 없음)
- i18n 10개 국어: en/ko/ja/zh/es/fr/de/pt/ru/vi (Context+Hook, LanguageSwitcher, CJK line-height 보정)
- GitHub Pages 자동 배포: .github/workflows/deploy-site.yml
### DS 컴포넌트 현황 (src/renderer/components/ds/) ### DS 컴포넌트 현황 (src/renderer/components/ds/)
- CrtDisplay.tsx: WebGL CRT 셰이더 (파형+스캔라인+비네팅+글리치) ✅ - CrtDisplay.tsx: WebGL CRT 셰이더 (파형+스캔라인+비네팅+글리치) ✅
- InstrumentPanel.tsx: 메탈 섀시 컨테이너 (각인 텍스트, 노이즈) ✅ - InstrumentPanel.tsx: 메탈 섀시 컨테이너 (각인 텍스트, 노이즈) ✅
@ -98,6 +107,47 @@ Phase: 11 완료 (Phase 1~11 전체 완료)
- d3roShadow: 10종 그림자 (chassis/card/inset/button/dialog/tooltip 등) - d3roShadow: 10종 그림자 (chassis/card/inset/button/dialog/tooltip 등)
- d3roRadius: 7종 반경 (outer 24px ~ pill 999px) - d3roRadius: 7종 반경 (outer 24px ~ pill 999px)
### Phase 13.2 구현 내용 (로컬 RAG)
- **RAGService**: Ollama nomic-embed-text 임베딩, SQLite에 JSON 벡터 저장, 코사인 유사도 검색
- **문서 관리**: .txt/.md/.pdf/.docx 지원, 500자 청크 분할 (50자 오버랩), 비동기 인덱싱
- **벡터 검색**: 쿼리 임베딩 → 전체 청크 코사인 유사도 → topK 결과 → LLM 컨텍스트 주입
- **KnowledgeBasePage**: 전용 UI (문서 목록, 인덱싱 진행률, 질문 입력, 답변+참조 소스 표시)
- **AppLayout**: 'knowledge' 라우트 + RAG 네비 항목
- **DB 테이블**: rag_documents + rag_chunks (Phase 13.2 신규)
- **IPC**: RAG 네임스페이스 9채널
- **ErrorCode**: 870-874 (DocumentNotFound, IndexingFailed, EmbeddingFailed, QueryFailed, UnsupportedFormat)
### Phase 13.3 구현 내용 (OS 자동화)
- **VoiceActionService**: 음성→LLM이 JSON 액션 플랜 생성→실행, 프리셋 6개 (크롬/메모장/탐색기/볼륨)
- **액션 타입**: open_app, open_url, open_file, keyboard_shortcut, type_text, system_command
- **안전장치**: 위험 명령어 블랙리스트 (rm, del, format, shutdown 등), safe=false 시 차단
- **IPC**: VOICE_ACTION 네임스페이스 9채널
- **ErrorCode**: 875-878 (PlanFailed, ExecutionFailed, Blocked, InvalidPlan)
### Phase 13.1 구현 내용 (음성 대화 모드)
- **VoiceConversationService**: STT→LLM→TTS 대화 루프 오케스트레이션, 세션 관리, 대화 히스토리 최근 10턴
- **TTSPlaybackService**: Windows SAPI (PowerShell System.Speech) 기반 로컬 TTS, 문장 단위 큐 재생
- **LocalLLMService.chatStream**: Ollama /api/chat 스트리밍, 대화 히스토리 messages 배열 전달
- **VoiceConversationPage**: 전용 대화 UI (채팅 메시지 목록, 녹음/정지/취소 버튼, 텍스트 입력)
- **AppLayout**: 'conversation' 라우트 + TALK 네비게이션 항목 추가
- **IPC**: VOICE_CONVERSATION 네임스페이스 15채널 (startSession, sendMessage, assistantDelta 등)
- **Preload**: voiceConversation API 네임스페이스 (17메서드/이벤트)
- **ErrorCode**: 795-798 (ConversationSessionAlreadyActive, TTSFailed, LLMFailed 등)
- **i18n**: conversation.* 번역 키 추가 (ko/en)
### Phase 12 구현 내용 (Pro+ 피처)
- **FileTranscriptionService**: ffmpeg(fluent-ffmpeg + @ffmpeg-installer/ffmpeg)로 미디어→PCM 변환, 30초 청크 분할 STT, 진행률 이벤트, 히스토리 자동 저장 (mode: 'file-transcription')
- **MeetingSummaryService**: CaptionService session-saved 이벤트 → Ollama 자동 요약, 마크다운 파싱 (요약/결정/할일), history.summaryText 컬럼 저장, 마크다운 내보내기
- **DictationTemplateService**: 템플릿 CRUD (electron-store), 세션 상태 머신 (idle→field-prompting→field-recording→completing), 프리셋 3개 (이메일/회의록/보고서), 출력 포맷 mustache 치환
- **FileDropZone UI**: DashboardPage 드래그앤드롭 존, 진행률/결과/에러 상태 표시
- **HistoryEntryCard 확장**: 요약 배지(Chip), 확장 뷰(요약/결정/할일), 요약 생성/내보내기 버튼
- **TemplateSection UI**: CommandsPage 하단 섹션, 템플릿 목록/생성/편집/삭제/세션 시작
- **DB 마이그레이션**: history 테이블 summary_text 컬럼 추가 (ALTER TABLE), mode enum에 'file-transcription' 추가
- **IPC**: FILE_TRANSCRIPTION(6채널) + MEETING_SUMMARY(5채널) + DICTATION_TEMPLATE(11채널) = 22개 신규 채널
- **Preload**: fileTranscription/meetingSummary/dictationTemplate 3개 API 네임스페이스
- **ErrorCode**: 780-794 범위 (FFmpegFailed, InvalidFormat, ChunkFailed, SummaryGenerationFailed, TemplateNotFound 등)
- **i18n**: fileTranscription.*/meetingSummary.*/template.* 번역 키 추가 (ko/en)
### Phase 11 구현 내용 (수익화 기반) ### Phase 11 구현 내용 (수익화 기반)
- **LicenseService**: Free/Pro/Pro+ 3단계 티어, electron-store 대신 별도 JSON 파일, 오프라인 우선 설계 - **LicenseService**: Free/Pro/Pro+ 3단계 티어, electron-store 대신 별도 JSON 파일, 오프라인 우선 설계
- **Feature Gating**: Feature enum (16개 기능), 티어별 접근 매핑, 쿼터 한도 (Free: 받아쓰기 20회/일, LLM 10회/일) - **Feature Gating**: Feature enum (16개 기능), 티어별 접근 매핑, 쿼터 한도 (Free: 받아쓰기 20회/일, LLM 10회/일)
@ -254,8 +304,10 @@ Phase: 11 완료 (Phase 1~11 전체 완료)
- Phase 9: 품질 보강 + UX 개선 (온보딩, 마이크 테스트, WAV 저장, 커맨드 팝업, Ollama 안내) - Phase 9: 품질 보강 + UX 개선 (온보딩, 마이크 테스트, WAV 저장, 커맨드 팝업, Ollama 안내)
- Phase 10: 킬러 피처 — Speakly를 넘어서 (실시간 자막, 스크린 컨텍스트, 음성 메모장, 멀티 LLM 체인, 음성 단축키) - Phase 10: 킬러 피처 — Speakly를 넘어서 (실시간 자막, 스크린 컨텍스트, 음성 메모장, 멀티 LLM 체인, 음성 단축키)
- Phase 11: 수익화 기반 — LicenseService + Feature Gating + Freemium UI - Phase 11: 수익화 기반 — LicenseService + Feature Gating + Freemium UI
- Phase 12: Pro 피처 — 파일 전사, 회의록 자동 요약, 딕테이션 템플릿 - Phase 12: Pro 피처 — 파일 전사, 회의록 자동 요약, 딕테이션 템플릿 ✅
- Phase 13: Pro+ 프리미엄 — 음성 대화 모드, 로컬 RAG, OS 자동화 - Phase 13.1: Pro+ 프리미엄 — 음성 대화 모드 (STT→LLM→TTS 루프) ✅
- Phase 13.2: Pro+ 프리미엄 — 로컬 RAG (문서 임베딩+검색) ✅
- Phase 13.3: Pro+ 프리미엄 — OS 자동화 (음성→JSON 액션) ✅
## 설계 문서 (구현 시 반드시 참조) ## 설계 문서 (구현 시 반드시 참조)
@docs/design/00-master-architecture.md @docs/design/00-master-architecture.md

387
package-lock.json generated
View file

@ -13,6 +13,7 @@
"@electron-toolkit/utils": "^4.0.0", "@electron-toolkit/utils": "^4.0.0",
"@emotion/react": "^11.14.0", "@emotion/react": "^11.14.0",
"@emotion/styled": "^11.14.0", "@emotion/styled": "^11.14.0",
"@ffmpeg-installer/ffmpeg": "^1.1.0",
"@mui/icons-material": "^7.0.0", "@mui/icons-material": "^7.0.0",
"@mui/material": "^7.0.0", "@mui/material": "^7.0.0",
"@nut-tree-fork/nut-js": "^4.2.6", "@nut-tree-fork/nut-js": "^4.2.6",
@ -22,8 +23,10 @@
"electron-audio-loopback": "^1.0.6", "electron-audio-loopback": "^1.0.6",
"electron-log": "^5.2.0", "electron-log": "^5.2.0",
"electron-store": "^10.0.0", "electron-store": "^10.0.0",
"fluent-ffmpeg": "^2.1.3",
"nanoid": "^5.1.7", "nanoid": "^5.1.7",
"node-record-lpcm16": "^1.0.1", "node-record-lpcm16": "^1.0.1",
"pdf-parse": "^2.4.5",
"react": "^19.0.0", "react": "^19.0.0",
"react-dom": "^19.0.0", "react-dom": "^19.0.0",
"uiohook-napi": "^1.5.5" "uiohook-napi": "^1.5.5"
@ -31,6 +34,7 @@
"devDependencies": { "devDependencies": {
"@electron-toolkit/tsconfig": "^1.0.1", "@electron-toolkit/tsconfig": "^1.0.1",
"@types/better-sqlite3": "^7.6.13", "@types/better-sqlite3": "^7.6.13",
"@types/fluent-ffmpeg": "^2.1.28",
"@types/node": "^22.13.0", "@types/node": "^22.13.0",
"@types/react": "^19.0.0", "@types/react": "^19.0.0",
"@types/react-dom": "^19.0.0", "@types/react-dom": "^19.0.0",
@ -1574,6 +1578,132 @@
"node": "^12.22.0 || ^14.17.0 || >=16.0.0" "node": "^12.22.0 || ^14.17.0 || >=16.0.0"
} }
}, },
"node_modules/@ffmpeg-installer/darwin-arm64": {
"version": "4.1.5",
"resolved": "https://registry.npmjs.org/@ffmpeg-installer/darwin-arm64/-/darwin-arm64-4.1.5.tgz",
"integrity": "sha512-hYqTiP63mXz7wSQfuqfFwfLOfwwFChUedeCVKkBtl/cliaTM7/ePI9bVzfZ2c+dWu3TqCwLDRWNSJ5pqZl8otA==",
"cpu": [
"arm64"
],
"hasInstallScript": true,
"license": "https://git.ffmpeg.org/gitweb/ffmpeg.git/blob_plain/HEAD:/LICENSE.md",
"optional": true,
"os": [
"darwin"
]
},
"node_modules/@ffmpeg-installer/darwin-x64": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@ffmpeg-installer/darwin-x64/-/darwin-x64-4.1.0.tgz",
"integrity": "sha512-Z4EyG3cIFjdhlY8wI9aLUXuH8nVt7E9SlMVZtWvSPnm2sm37/yC2CwjUzyCQbJbySnef1tQwGG2Sx+uWhd9IAw==",
"cpu": [
"x64"
],
"hasInstallScript": true,
"license": "LGPL-2.1",
"optional": true,
"os": [
"darwin"
]
},
"node_modules/@ffmpeg-installer/ffmpeg": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/@ffmpeg-installer/ffmpeg/-/ffmpeg-1.1.0.tgz",
"integrity": "sha512-Uq4rmwkdGxIa9A6Bd/VqqYbT7zqh1GrT5/rFwCwKM70b42W5gIjWeVETq6SdcL0zXqDtY081Ws/iJWhr1+xvQg==",
"license": "LGPL-2.1",
"optionalDependencies": {
"@ffmpeg-installer/darwin-arm64": "4.1.5",
"@ffmpeg-installer/darwin-x64": "4.1.0",
"@ffmpeg-installer/linux-arm": "4.1.3",
"@ffmpeg-installer/linux-arm64": "4.1.4",
"@ffmpeg-installer/linux-ia32": "4.1.0",
"@ffmpeg-installer/linux-x64": "4.1.0",
"@ffmpeg-installer/win32-ia32": "4.1.0",
"@ffmpeg-installer/win32-x64": "4.1.0"
}
},
"node_modules/@ffmpeg-installer/linux-arm": {
"version": "4.1.3",
"resolved": "https://registry.npmjs.org/@ffmpeg-installer/linux-arm/-/linux-arm-4.1.3.tgz",
"integrity": "sha512-NDf5V6l8AfzZ8WzUGZ5mV8O/xMzRag2ETR6+TlGIsMHp81agx51cqpPItXPib/nAZYmo55Bl2L6/WOMI3A5YRg==",
"cpu": [
"arm"
],
"hasInstallScript": true,
"license": "GPLv3",
"optional": true,
"os": [
"linux"
]
},
"node_modules/@ffmpeg-installer/linux-arm64": {
"version": "4.1.4",
"resolved": "https://registry.npmjs.org/@ffmpeg-installer/linux-arm64/-/linux-arm64-4.1.4.tgz",
"integrity": "sha512-dljEqAOD0oIM6O6DxBW9US/FkvqvQwgJ2lGHOwHDDwu/pX8+V0YsDL1xqHbj1DMX/+nP9rxw7G7gcUvGspSoKg==",
"cpu": [
"arm64"
],
"hasInstallScript": true,
"license": "GPLv3",
"optional": true,
"os": [
"linux"
]
},
"node_modules/@ffmpeg-installer/linux-ia32": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@ffmpeg-installer/linux-ia32/-/linux-ia32-4.1.0.tgz",
"integrity": "sha512-0LWyFQnPf+Ij9GQGD034hS6A90URNu9HCtQ5cTqo5MxOEc7Rd8gLXrJvn++UmxhU0J5RyRE9KRYstdCVUjkNOQ==",
"cpu": [
"ia32"
],
"hasInstallScript": true,
"license": "GPLv3",
"optional": true,
"os": [
"linux"
]
},
"node_modules/@ffmpeg-installer/linux-x64": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@ffmpeg-installer/linux-x64/-/linux-x64-4.1.0.tgz",
"integrity": "sha512-Y5BWhGLU/WpQjOArNIgXD3z5mxxdV8c41C+U15nsE5yF8tVcdCGet5zPs5Zy3Ta6bU7haGpIzryutqCGQA/W8A==",
"cpu": [
"x64"
],
"hasInstallScript": true,
"license": "GPLv3",
"optional": true,
"os": [
"linux"
]
},
"node_modules/@ffmpeg-installer/win32-ia32": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@ffmpeg-installer/win32-ia32/-/win32-ia32-4.1.0.tgz",
"integrity": "sha512-FV2D7RlaZv/lrtdhaQ4oETwoFUsUjlUiasiZLDxhEUPdNDWcH1OU9K1xTvqz+OXLdsmYelUDuBS/zkMOTtlUAw==",
"cpu": [
"ia32"
],
"license": "GPLv3",
"optional": true,
"os": [
"win32"
]
},
"node_modules/@ffmpeg-installer/win32-x64": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@ffmpeg-installer/win32-x64/-/win32-x64-4.1.0.tgz",
"integrity": "sha512-Drt5u2vzDnIONf4ZEkKtFlbvwj6rI3kxw1Ck9fpudmtgaZIHD4ucsWB2lCZBXRxJgXR+2IMSti+4rtM4C4rXgg==",
"cpu": [
"x64"
],
"license": "GPLv3",
"optional": true,
"os": [
"win32"
]
},
"node_modules/@humanwhocodes/config-array": { "node_modules/@humanwhocodes/config-array": {
"version": "0.13.0", "version": "0.13.0",
"resolved": "https://registry.npmjs.org/@humanwhocodes/config-array/-/config-array-0.13.0.tgz", "resolved": "https://registry.npmjs.org/@humanwhocodes/config-array/-/config-array-0.13.0.tgz",
@ -2535,6 +2665,190 @@
} }
} }
}, },
"node_modules/@napi-rs/canvas": {
"version": "0.1.80",
"resolved": "https://registry.npmjs.org/@napi-rs/canvas/-/canvas-0.1.80.tgz",
"integrity": "sha512-DxuT1ClnIPts1kQx8FBmkk4BQDTfI5kIzywAaMjQSXfNnra5UFU9PwurXrl+Je3bJ6BGsp/zmshVVFbCmyI+ww==",
"license": "MIT",
"workspaces": [
"e2e/*"
],
"engines": {
"node": ">= 10"
},
"optionalDependencies": {
"@napi-rs/canvas-android-arm64": "0.1.80",
"@napi-rs/canvas-darwin-arm64": "0.1.80",
"@napi-rs/canvas-darwin-x64": "0.1.80",
"@napi-rs/canvas-linux-arm-gnueabihf": "0.1.80",
"@napi-rs/canvas-linux-arm64-gnu": "0.1.80",
"@napi-rs/canvas-linux-arm64-musl": "0.1.80",
"@napi-rs/canvas-linux-riscv64-gnu": "0.1.80",
"@napi-rs/canvas-linux-x64-gnu": "0.1.80",
"@napi-rs/canvas-linux-x64-musl": "0.1.80",
"@napi-rs/canvas-win32-x64-msvc": "0.1.80"
}
},
"node_modules/@napi-rs/canvas-android-arm64": {
"version": "0.1.80",
"resolved": "https://registry.npmjs.org/@napi-rs/canvas-android-arm64/-/canvas-android-arm64-0.1.80.tgz",
"integrity": "sha512-sk7xhN/MoXeuExlggf91pNziBxLPVUqF2CAVnB57KLG/pz7+U5TKG8eXdc3pm0d7Od0WreB6ZKLj37sX9muGOQ==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@napi-rs/canvas-darwin-arm64": {
"version": "0.1.80",
"resolved": "https://registry.npmjs.org/@napi-rs/canvas-darwin-arm64/-/canvas-darwin-arm64-0.1.80.tgz",
"integrity": "sha512-O64APRTXRUiAz0P8gErkfEr3lipLJgM6pjATwavZ22ebhjYl/SUbpgM0xcWPQBNMP1n29afAC/Us5PX1vg+JNQ==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@napi-rs/canvas-darwin-x64": {
"version": "0.1.80",
"resolved": "https://registry.npmjs.org/@napi-rs/canvas-darwin-x64/-/canvas-darwin-x64-0.1.80.tgz",
"integrity": "sha512-FqqSU7qFce0Cp3pwnTjVkKjjOtxMqRe6lmINxpIZYaZNnVI0H5FtsaraZJ36SiTHNjZlUB69/HhxNDT1Aaa9vA==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@napi-rs/canvas-linux-arm-gnueabihf": {
"version": "0.1.80",
"resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-arm-gnueabihf/-/canvas-linux-arm-gnueabihf-0.1.80.tgz",
"integrity": "sha512-eyWz0ddBDQc7/JbAtY4OtZ5SpK8tR4JsCYEZjCE3dI8pqoWUC8oMwYSBGCYfsx2w47cQgQCgMVRVTFiiO38hHQ==",
"cpu": [
"arm"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@napi-rs/canvas-linux-arm64-gnu": {
"version": "0.1.80",
"resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-arm64-gnu/-/canvas-linux-arm64-gnu-0.1.80.tgz",
"integrity": "sha512-qwA63t8A86bnxhuA/GwOkK3jvb+XTQaTiVML0vAWoHyoZYTjNs7BzoOONDgTnNtr8/yHrq64XXzUoLqDzU+Uuw==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@napi-rs/canvas-linux-arm64-musl": {
"version": "0.1.80",
"resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-arm64-musl/-/canvas-linux-arm64-musl-0.1.80.tgz",
"integrity": "sha512-1XbCOz/ymhj24lFaIXtWnwv/6eFHXDrjP0jYkc6iHQ9q8oXKzUX1Lc6bu+wuGiLhGh2GS/2JlfORC5ZcXimRcg==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@napi-rs/canvas-linux-riscv64-gnu": {
"version": "0.1.80",
"resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-riscv64-gnu/-/canvas-linux-riscv64-gnu-0.1.80.tgz",
"integrity": "sha512-XTzR125w5ZMs0lJcxRlS1K3P5RaZ9RmUsPtd1uGt+EfDyYMu4c6SEROYsxyatbbu/2+lPe7MPHOO/0a0x7L/gw==",
"cpu": [
"riscv64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@napi-rs/canvas-linux-x64-gnu": {
"version": "0.1.80",
"resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-x64-gnu/-/canvas-linux-x64-gnu-0.1.80.tgz",
"integrity": "sha512-BeXAmhKg1kX3UCrJsYbdQd3hIMDH/K6HnP/pG2LuITaXhXBiNdh//TVVVVCBbJzVQaV5gK/4ZOCMrQW9mvuTqA==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@napi-rs/canvas-linux-x64-musl": {
"version": "0.1.80",
"resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-x64-musl/-/canvas-linux-x64-musl-0.1.80.tgz",
"integrity": "sha512-x0XvZWdHbkgdgucJsRxprX/4o4sEed7qo9rCQA9ugiS9qE2QvP0RIiEugtZhfLH3cyI+jIRFJHV4Fuz+1BHHMg==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@napi-rs/canvas-win32-x64-msvc": {
"version": "0.1.80",
"resolved": "https://registry.npmjs.org/@napi-rs/canvas-win32-x64-msvc/-/canvas-win32-x64-msvc-0.1.80.tgz",
"integrity": "sha512-Z8jPsM6df5V8B1HrCHB05+bDiCxjE9QA//3YrkKIdVDEwn5RKaqOxCJDRJkl48cJbylcrJbW4HxZbTte8juuPg==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@nodelib/fs.scandir": { "node_modules/@nodelib/fs.scandir": {
"version": "2.1.5", "version": "2.1.5",
"resolved": "https://registry.npmjs.org/@nodelib/fs.scandir/-/fs.scandir-2.1.5.tgz", "resolved": "https://registry.npmjs.org/@nodelib/fs.scandir/-/fs.scandir-2.1.5.tgz",
@ -3254,6 +3568,16 @@
"dev": true, "dev": true,
"license": "MIT" "license": "MIT"
}, },
"node_modules/@types/fluent-ffmpeg": {
"version": "2.1.28",
"resolved": "https://registry.npmjs.org/@types/fluent-ffmpeg/-/fluent-ffmpeg-2.1.28.tgz",
"integrity": "sha512-5ovxsDwBcPfJ+eYs1I/ZpcYCnkce7pvH9AHSvrZllAp1ZPpTRDZAFjF3TRFbukxSgIYTTNYePbS0rKUmaxVbXw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/node": "*"
}
},
"node_modules/@types/fs-extra": { "node_modules/@types/fs-extra": {
"version": "9.0.13", "version": "9.0.13",
"resolved": "https://registry.npmjs.org/@types/fs-extra/-/fs-extra-9.0.13.tgz", "resolved": "https://registry.npmjs.org/@types/fs-extra/-/fs-extra-9.0.13.tgz",
@ -6606,6 +6930,37 @@
"dev": true, "dev": true,
"license": "ISC" "license": "ISC"
}, },
"node_modules/fluent-ffmpeg": {
"version": "2.1.3",
"resolved": "https://registry.npmjs.org/fluent-ffmpeg/-/fluent-ffmpeg-2.1.3.tgz",
"integrity": "sha512-Be3narBNt2s6bsaqP6Jzq91heDgOEaDCJAXcE3qcma/EJBSy5FB4cvO31XBInuAuKBx8Kptf8dkhjK0IOru39Q==",
"deprecated": "Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.",
"license": "MIT",
"dependencies": {
"async": "^0.2.9",
"which": "^1.1.1"
},
"engines": {
"node": ">=18"
}
},
"node_modules/fluent-ffmpeg/node_modules/async": {
"version": "0.2.10",
"resolved": "https://registry.npmjs.org/async/-/async-0.2.10.tgz",
"integrity": "sha512-eAkdoKxU6/LkKDBzLpT+t6Ff5EtfSF4wx1WfJiPEEV7WNLnDaRXk0oVysiEPm262roaachGexwUv94WhSgN5TQ=="
},
"node_modules/fluent-ffmpeg/node_modules/which": {
"version": "1.3.1",
"resolved": "https://registry.npmjs.org/which/-/which-1.3.1.tgz",
"integrity": "sha512-HxJdYWq1MTIQbJ3nw0cqssHoTNU267KlrDuGZ1WYlxDStUtKUhOaJmh112/TZmHxxUfuJqPXSOm7tDyas0OSIQ==",
"license": "ISC",
"dependencies": {
"isexe": "^2.0.0"
},
"bin": {
"which": "bin/which"
}
},
"node_modules/follow-redirects": { "node_modules/follow-redirects": {
"version": "1.15.11", "version": "1.15.11",
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.11.tgz", "resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.11.tgz",
@ -8660,6 +9015,38 @@
"node": ">= 14.16" "node": ">= 14.16"
} }
}, },
"node_modules/pdf-parse": {
"version": "2.4.5",
"resolved": "https://registry.npmjs.org/pdf-parse/-/pdf-parse-2.4.5.tgz",
"integrity": "sha512-mHU89HGh7v+4u2ubfnevJ03lmPgQ5WU4CxAVmTSh/sxVTEDYd1er/dKS/A6vg77NX47KTEoihq8jZBLr8Cxuwg==",
"license": "Apache-2.0",
"dependencies": {
"@napi-rs/canvas": "0.1.80",
"pdfjs-dist": "5.4.296"
},
"bin": {
"pdf-parse": "bin/cli.mjs"
},
"engines": {
"node": ">=20.16.0 <21 || >=22.3.0"
},
"funding": {
"type": "github",
"url": "https://github.com/sponsors/mehmet-kozan"
}
},
"node_modules/pdfjs-dist": {
"version": "5.4.296",
"resolved": "https://registry.npmjs.org/pdfjs-dist/-/pdfjs-dist-5.4.296.tgz",
"integrity": "sha512-DlOzet0HO7OEnmUmB6wWGJrrdvbyJKftI1bhMitK7O2N8W2gc757yyYBbINy9IDafXAV9wmKr9t7xsTaNKRG5Q==",
"license": "Apache-2.0",
"engines": {
"node": ">=20.16.0 || >=22.3.0"
},
"optionalDependencies": {
"@napi-rs/canvas": "^0.1.80"
}
},
"node_modules/pe-library": { "node_modules/pe-library": {
"version": "0.4.1", "version": "0.4.1",
"resolved": "https://registry.npmjs.org/pe-library/-/pe-library-0.4.1.tgz", "resolved": "https://registry.npmjs.org/pe-library/-/pe-library-0.4.1.tgz",

View file

@ -27,6 +27,7 @@
"devDependencies": { "devDependencies": {
"@electron-toolkit/tsconfig": "^1.0.1", "@electron-toolkit/tsconfig": "^1.0.1",
"@types/better-sqlite3": "^7.6.13", "@types/better-sqlite3": "^7.6.13",
"@types/fluent-ffmpeg": "^2.1.28",
"@types/node": "^22.13.0", "@types/node": "^22.13.0",
"@types/react": "^19.0.0", "@types/react": "^19.0.0",
"@types/react-dom": "^19.0.0", "@types/react-dom": "^19.0.0",
@ -48,6 +49,7 @@
"@electron-toolkit/utils": "^4.0.0", "@electron-toolkit/utils": "^4.0.0",
"@emotion/react": "^11.14.0", "@emotion/react": "^11.14.0",
"@emotion/styled": "^11.14.0", "@emotion/styled": "^11.14.0",
"@ffmpeg-installer/ffmpeg": "^1.1.0",
"@mui/icons-material": "^7.0.0", "@mui/icons-material": "^7.0.0",
"@mui/material": "^7.0.0", "@mui/material": "^7.0.0",
"@nut-tree-fork/nut-js": "^4.2.6", "@nut-tree-fork/nut-js": "^4.2.6",
@ -57,8 +59,10 @@
"electron-audio-loopback": "^1.0.6", "electron-audio-loopback": "^1.0.6",
"electron-log": "^5.2.0", "electron-log": "^5.2.0",
"electron-store": "^10.0.0", "electron-store": "^10.0.0",
"fluent-ffmpeg": "^2.1.3",
"nanoid": "^5.1.7", "nanoid": "^5.1.7",
"node-record-lpcm16": "^1.0.1", "node-record-lpcm16": "^1.0.1",
"pdf-parse": "^2.4.5",
"react": "^19.0.0", "react": "^19.0.0",
"react-dom": "^19.0.0", "react-dom": "^19.0.0",
"uiohook-napi": "^1.5.5" "uiohook-napi": "^1.5.5"

View file

@ -56,7 +56,8 @@ export async function bootstrap(): Promise<void> {
{ name: 'popup-preload', critical: false, fn: initPopupWindows }, { name: 'popup-preload', critical: false, fn: initPopupWindows },
{ name: 'hotkey', critical: false, fn: initHotkey }, { name: 'hotkey', critical: false, fn: initHotkey },
{ name: 'voice-mode', critical: false, fn: initVoiceMode }, { name: 'voice-mode', critical: false, fn: initVoiceMode },
{ name: 'llm-polling', critical: false, fn: initLLMPolling } { name: 'llm-polling', critical: false, fn: initLLMPolling },
{ name: 'meeting-summary-wiring', critical: false, fn: initMeetingSummaryWiring },
] ]
for (const step of steps) { for (const step of steps) {
@ -264,6 +265,23 @@ async function initLLMPolling(): Promise<void> {
llm.startPolling() llm.startPolling()
} }
async function initMeetingSummaryWiring(): Promise<void> {
try {
const { getCaptionService } = await import('./services/CaptionService')
const { getMeetingSummaryService } = await import('./services/MeetingSummaryService')
const captionService = getCaptionService()
const summaryService = getMeetingSummaryService()
captionService.on('session-saved', (summary: { sessionId: string }) => {
summaryService.onCaptionSessionSaved(summary).catch((err) => {
logger.warn('Auto meeting summary failed:', err)
})
})
} catch (err) {
logger.warn('Meeting summary wiring failed:', err)
}
}
// ── HistoryPopup IPC 연동 ──────────────────────────── // ── HistoryPopup IPC 연동 ────────────────────────────
function setupHistoryPopupIPC(): void { function setupHistoryPopupIPC(): void {

View file

@ -105,6 +105,42 @@ export function initDatabase(): BetterSQLite3Database<typeof schema> {
CREATE INDEX IF NOT EXISTS idx_daily_usage_date ON daily_usage(date); CREATE INDEX IF NOT EXISTS idx_daily_usage_date ON daily_usage(date);
`) `)
// Phase 13.2: RAG 테이블 생성
sqlite.exec(`
CREATE TABLE IF NOT EXISTS rag_documents (
id TEXT PRIMARY KEY,
file_name TEXT NOT NULL,
file_path TEXT NOT NULL,
file_type TEXT NOT NULL,
chunk_count INTEGER NOT NULL DEFAULT 0,
indexed INTEGER NOT NULL DEFAULT 0,
indexed_at INTEGER,
added_at INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_rag_documents_added_at ON rag_documents(added_at);
CREATE TABLE IF NOT EXISTS rag_chunks (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL,
content TEXT NOT NULL,
embedding TEXT NOT NULL,
chunk_index INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_rag_chunks_document_id ON rag_chunks(document_id);
`)
// Phase 12.2: history 테이블에 summary_text 컬럼 마이그레이션
try {
const columns = sqlite.pragma('table_info(history)') as Array<{ name: string }>
const hasSummaryText = columns.some((c) => c.name === 'summary_text')
if (!hasSummaryText) {
sqlite.exec('ALTER TABLE history ADD COLUMN summary_text TEXT')
logger.info('Migrated: added summary_text column to history')
}
} catch (err) {
logger.warn('summary_text migration check failed:', err)
}
db = drizzle(sqlite, { schema }) db = drizzle(sqlite, { schema })
logger.info('Database initialized') logger.info('Database initialized')

View file

@ -13,7 +13,7 @@ export const history = sqliteTable(
focusedApp: text('focused_app'), focusedApp: text('focused_app'),
focusedAppName: text('focused_app_name'), focusedAppName: text('focused_app_name'),
focusedAppWindowTitle: text('focused_app_window_title'), focusedAppWindowTitle: text('focused_app_window_title'),
mode: text('mode', { enum: ['dictation', 'translate', 'command', 'caption'] }) mode: text('mode', { enum: ['dictation', 'translate', 'command', 'caption', 'file-transcription'] })
.notNull() .notNull()
.default('dictation'), .default('dictation'),
status: text('status', { enum: ['completed', 'cancelled', 'error'] }) status: text('status', { enum: ['completed', 'cancelled', 'error'] })
@ -31,7 +31,9 @@ export const history = sqliteTable(
llmLatencyMs: integer('llm_latency_ms'), llmLatencyMs: integer('llm_latency_ms'),
createdAt: integer('created_at').notNull(), createdAt: integer('created_at').notNull(),
updatedAt: integer('updated_at').notNull(), updatedAt: integer('updated_at').notNull(),
appVersion: text('app_version').notNull().default('1.0.0') appVersion: text('app_version').notNull().default('1.0.0'),
/** Phase 12.2: 회의록 자동 요약 텍스트 (마크다운) */
summaryText: text('summary_text'),
}, },
(table) => [ (table) => [
index('idx_history_created_at').on(table.createdAt), index('idx_history_created_at').on(table.createdAt),
@ -112,6 +114,44 @@ export const dailyUsage = sqliteTable(
export type DailyUsageRow = typeof dailyUsage.$inferSelect export type DailyUsageRow = typeof dailyUsage.$inferSelect
export type NewDailyUsageRow = typeof dailyUsage.$inferInsert export type NewDailyUsageRow = typeof dailyUsage.$inferInsert
// ── rag_documents (Phase 13.2) ──────────────────────────
export const ragDocuments = sqliteTable(
'rag_documents',
{
id: text('id').primaryKey(),
fileName: text('file_name').notNull(),
filePath: text('file_path').notNull(),
fileType: text('file_type', { enum: ['txt', 'md', 'pdf', 'docx'] }).notNull(),
chunkCount: integer('chunk_count').notNull().default(0),
indexed: integer('indexed', { mode: 'boolean' }).notNull().default(false),
indexedAt: integer('indexed_at'),
addedAt: integer('added_at').notNull(),
},
(table) => [
index('idx_rag_documents_added_at').on(table.addedAt),
]
)
export const ragChunks = sqliteTable(
'rag_chunks',
{
id: text('id').primaryKey(),
documentId: text('document_id').notNull(),
content: text('content').notNull(),
/** 임베딩 벡터 (JSON 직렬화 float[]) */
embedding: text('embedding').notNull(),
chunkIndex: integer('chunk_index').notNull(),
},
(table) => [
index('idx_rag_chunks_document_id').on(table.documentId),
]
)
export type RAGDocumentRow = typeof ragDocuments.$inferSelect
export type NewRAGDocumentRow = typeof ragDocuments.$inferInsert
export type RAGChunkRow = typeof ragChunks.$inferSelect
export type NewRAGChunkRow = typeof ragChunks.$inferInsert
// ── 타입 추출 ──────────────────────────────────────────── // ── 타입 추출 ────────────────────────────────────────────
export type History = typeof history.$inferSelect export type History = typeof history.$inferSelect
export type NewHistory = typeof history.$inferInsert export type NewHistory = typeof history.$inferInsert

View file

@ -0,0 +1,69 @@
// src/main/ipc/file-transcription-handlers.ts
// Phase 12.1: 파일 전사 IPC 핸들러
import { ipcMain, dialog } from 'electron'
import { IPC_CHANNELS } from '@shared/ipc-channels'
import { ErrorCode, ipcSuccess, ipcError } from '@shared/errors'
import { getFileTranscriptionService } from '../services/FileTranscriptionService'
import { getLogger } from '../services/LoggerService'
import type { FileTranscriptionStartParams } from '@shared/types'
const logger = getLogger('file-transcription-handlers')
export function registerFileTranscriptionHandlers(): void {
ipcMain.handle(
IPC_CHANNELS.FILE_TRANSCRIPTION.START,
async (_event, params: FileTranscriptionStartParams) => {
try {
let filePath = params.filePath
// filePath가 없으면 파일 선택 다이얼로그
if (!filePath) {
const result = await dialog.showOpenDialog({
properties: ['openFile'],
filters: [
{
name: 'Audio/Video',
extensions: [
'mp3', 'wav', 'm4a', 'ogg', 'flac', 'wma', 'aac',
'mp4', 'mkv', 'webm', 'avi', 'mov',
],
},
],
})
if (result.canceled || result.filePaths.length === 0) {
return ipcError(ErrorCode.FileTranscriptionCancelled, 'File selection cancelled')
}
filePath = result.filePaths[0]
}
const service = getFileTranscriptionService()
const resultData = await service.startTranscription(filePath, params.language)
return ipcSuccess(resultData)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
logger.error('File transcription failed:', msg)
return ipcError(ErrorCode.FileTranscriptionChunkFailed, msg)
}
},
)
ipcMain.handle(IPC_CHANNELS.FILE_TRANSCRIPTION.CANCEL, async () => {
try {
getFileTranscriptionService().cancel()
return ipcSuccess(undefined)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.FileTranscriptionCancelled, msg)
}
})
ipcMain.handle(IPC_CHANNELS.FILE_TRANSCRIPTION.GET_STATE, async () => {
try {
return ipcSuccess(getFileTranscriptionService().getStateInfo())
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.UnknownError, msg)
}
})
}

View file

@ -17,6 +17,12 @@ import { registerContextHandlers } from './context-handlers'
import { registerCaptionHandlers } from './caption-handlers' import { registerCaptionHandlers } from './caption-handlers'
import { registerChainHandlers } from './chain-handlers' import { registerChainHandlers } from './chain-handlers'
import { registerLicenseHandlers } from './license-handlers' import { registerLicenseHandlers } from './license-handlers'
import { registerFileTranscriptionHandlers } from './file-transcription-handlers'
import { registerMeetingSummaryHandlers } from './meeting-summary-handlers'
import { registerTemplateHandlers } from './template-handlers'
import { registerVoiceConversationHandlers } from './voice-conversation-handlers'
import { registerRAGHandlers } from './rag-handlers'
import { registerVoiceActionHandlers } from './voice-action-handlers'
import { getLogger } from '../services/LoggerService' import { getLogger } from '../services/LoggerService'
const logger = getLogger('ipc') const logger = getLogger('ipc')
@ -39,5 +45,11 @@ export function registerAllIpcHandlers(): void {
registerCaptionHandlers() registerCaptionHandlers()
registerChainHandlers() registerChainHandlers()
registerLicenseHandlers() registerLicenseHandlers()
registerFileTranscriptionHandlers()
registerMeetingSummaryHandlers()
registerTemplateHandlers()
registerVoiceConversationHandlers()
registerRAGHandlers()
registerVoiceActionHandlers()
logger.info('All IPC handlers registered') logger.info('All IPC handlers registered')
} }

View file

@ -0,0 +1,54 @@
// src/main/ipc/meeting-summary-handlers.ts
// Phase 12.2: 회의록 요약 IPC 핸들러
import { ipcMain } from 'electron'
import { IPC_CHANNELS } from '@shared/ipc-channels'
import { ErrorCode, ipcSuccess, ipcError } from '@shared/errors'
import { getMeetingSummaryService } from '../services/MeetingSummaryService'
import { getLogger } from '../services/LoggerService'
import type { MeetingSummarizeParams, MeetingSummaryGetParams, MeetingSummaryExportParams } from '@shared/types'
const logger = getLogger('meeting-summary-handlers')
export function registerMeetingSummaryHandlers(): void {
ipcMain.handle(
IPC_CHANNELS.MEETING_SUMMARY.SUMMARIZE,
async (_event, params: MeetingSummarizeParams) => {
try {
const result = await getMeetingSummaryService().summarize(params.historyId)
return ipcSuccess(result)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
logger.error('Meeting summary failed:', msg)
return ipcError(ErrorCode.MeetingSummaryGenerationFailed, msg)
}
},
)
ipcMain.handle(
IPC_CHANNELS.MEETING_SUMMARY.GET_SUMMARY,
async (_event, params: MeetingSummaryGetParams) => {
try {
const result = getMeetingSummaryService().getSummary(params.historyId)
return ipcSuccess(result)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.MeetingSummaryGenerationFailed, msg)
}
},
)
ipcMain.handle(
IPC_CHANNELS.MEETING_SUMMARY.EXPORT_MARKDOWN,
async (_event, params: MeetingSummaryExportParams) => {
try {
const filePath = await getMeetingSummaryService().exportMarkdown(params.historyId)
return ipcSuccess(filePath)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
logger.error('Meeting summary export failed:', msg)
return ipcError(ErrorCode.MeetingSummaryExportFailed, msg)
}
},
)
}

View file

@ -0,0 +1,84 @@
// src/main/ipc/rag-handlers.ts
// Phase 13.2: 로컬 RAG IPC 핸들러
import { ipcMain, dialog } from 'electron'
import { IPC_CHANNELS } from '@shared/ipc-channels'
import { ErrorCode, ipcSuccess, ipcError } from '@shared/errors'
import { getRAGService } from '../services/RAGService'
import { getLogger } from '../services/LoggerService'
import type { RAGQueryParams, RAGRemoveDocumentParams } from '@shared/types'
const logger = getLogger('rag-handlers')
export function registerRAGHandlers(): void {
ipcMain.handle(IPC_CHANNELS.RAG.ADD_DOCUMENT, async (_event, params: { filePath?: string }) => {
try {
let filePath = params?.filePath
if (!filePath) {
const result = await dialog.showOpenDialog({
properties: ['openFile'],
filters: [{ name: 'Documents', extensions: ['txt', 'md', 'pdf', 'docx'] }],
})
if (result.canceled || result.filePaths.length === 0) {
return ipcError(ErrorCode.RAGDocumentNotFound, 'File selection cancelled')
}
filePath = result.filePaths[0]
}
const doc = await getRAGService().addDocument(filePath)
return ipcSuccess(doc)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
logger.error('RAG add document failed:', msg)
return ipcError(ErrorCode.RAGIndexingFailed, msg)
}
})
ipcMain.handle(IPC_CHANNELS.RAG.REMOVE_DOCUMENT, async (_event, params: RAGRemoveDocumentParams) => {
try {
getRAGService().removeDocument(params.documentId)
return ipcSuccess(undefined)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.RAGDocumentNotFound, msg)
}
})
ipcMain.handle(IPC_CHANNELS.RAG.GET_DOCUMENTS, async () => {
try {
return ipcSuccess(getRAGService().getDocuments())
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.UnknownError, msg)
}
})
ipcMain.handle(IPC_CHANNELS.RAG.QUERY, async (_event, params: RAGQueryParams) => {
try {
const result = await getRAGService().query(params.query, params.topK)
return ipcSuccess(result)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
logger.error('RAG query failed:', msg)
return ipcError(ErrorCode.RAGQueryFailed, msg)
}
})
ipcMain.handle(IPC_CHANNELS.RAG.GET_STATE, async () => {
try {
return ipcSuccess(getRAGService().getStateInfo())
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.UnknownError, msg)
}
})
ipcMain.handle(IPC_CHANNELS.RAG.REINDEX, async (_event, params: { documentId: string }) => {
try {
await getRAGService().reindex(params.documentId)
return ipcSuccess(undefined)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.RAGIndexingFailed, msg)
}
})
}

View file

@ -0,0 +1,128 @@
// src/main/ipc/template-handlers.ts
// Phase 12.3: 딕테이션 템플릿 IPC 핸들러
import { ipcMain } from 'electron'
import { IPC_CHANNELS } from '@shared/ipc-channels'
import { ErrorCode, ipcSuccess, ipcError } from '@shared/errors'
import { getDictationTemplateService } from '../services/DictationTemplateService'
import { getLogger } from '../services/LoggerService'
import type {
CreateTemplateParams,
UpdateTemplateParams,
DeleteTemplateParams,
StartTemplateSessionParams,
SetFieldValueParams,
} from '@shared/types'
const logger = getLogger('template-handlers')
export function registerTemplateHandlers(): void {
ipcMain.handle(IPC_CHANNELS.DICTATION_TEMPLATE.GET_ALL, async () => {
try {
return ipcSuccess(getDictationTemplateService().getAll())
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.UnknownError, msg)
}
})
ipcMain.handle(
IPC_CHANNELS.DICTATION_TEMPLATE.CREATE,
async (_event, params: CreateTemplateParams) => {
try {
// 라이센스 체크
try {
const { getLicenseService } = await import('../services/LicenseService')
const { Feature } = await import('@shared/types')
const license = getLicenseService()
const access = license.canUse(Feature.DICTATION_TEMPLATE)
if (!access.allowed) {
license.promptUpgrade(Feature.DICTATION_TEMPLATE, 'tier_required')
return ipcError(ErrorCode.FeatureNotAvailable, 'Pro+ required')
}
} catch {
// LicenseService 미초기화 시 허용
}
const result = getDictationTemplateService().create(params)
return ipcSuccess(result)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
logger.error('Template create failed:', msg)
return ipcError(ErrorCode.TemplateInvalidFormat, msg)
}
},
)
ipcMain.handle(
IPC_CHANNELS.DICTATION_TEMPLATE.UPDATE,
async (_event, params: UpdateTemplateParams) => {
try {
const result = getDictationTemplateService().update(params)
return ipcSuccess(result)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.TemplateNotFound, msg)
}
},
)
ipcMain.handle(
IPC_CHANNELS.DICTATION_TEMPLATE.DELETE,
async (_event, params: DeleteTemplateParams) => {
try {
getDictationTemplateService().delete(params.id)
return ipcSuccess(undefined)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.TemplateNotFound, msg)
}
},
)
ipcMain.handle(
IPC_CHANNELS.DICTATION_TEMPLATE.START_SESSION,
async (_event, params: StartTemplateSessionParams) => {
try {
getDictationTemplateService().startSession(params.templateId)
return ipcSuccess(undefined)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
logger.error('Template session start failed:', msg)
return ipcError(ErrorCode.TemplateSessionAlreadyActive, msg)
}
},
)
ipcMain.handle(IPC_CHANNELS.DICTATION_TEMPLATE.CANCEL_SESSION, async () => {
try {
getDictationTemplateService().cancelSession()
return ipcSuccess(undefined)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.TemplateSessionNotActive, msg)
}
})
ipcMain.handle(IPC_CHANNELS.DICTATION_TEMPLATE.GET_SESSION_STATE, async () => {
try {
return ipcSuccess(getDictationTemplateService().getSessionState())
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.UnknownError, msg)
}
})
ipcMain.handle(
IPC_CHANNELS.DICTATION_TEMPLATE.SET_FIELD_VALUE,
async (_event, params: SetFieldValueParams) => {
try {
getDictationTemplateService().setFieldValue(params.fieldId, params.value)
return ipcSuccess(undefined)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.TemplateFieldRecordingFailed, msg)
}
},
)
}

View file

@ -0,0 +1,74 @@
// src/main/ipc/voice-action-handlers.ts
// Phase 13.3: OS 자동화 IPC 핸들러
import { ipcMain } from 'electron'
import { IPC_CHANNELS } from '@shared/ipc-channels'
import { ErrorCode, ipcSuccess, ipcError } from '@shared/errors'
import { getVoiceActionService } from '../services/VoiceActionService'
import { getLogger } from '../services/LoggerService'
import type { VoiceActionExecuteParams } from '@shared/types'
const logger = getLogger('voice-action-handlers')
export function registerVoiceActionHandlers(): void {
ipcMain.handle(
IPC_CHANNELS.VOICE_ACTION.EXECUTE,
async (_event, params: VoiceActionExecuteParams) => {
try {
await getVoiceActionService().execute(params.text)
return ipcSuccess(undefined)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
logger.error('Voice action failed:', msg)
return ipcError(ErrorCode.VoiceActionExecutionFailed, msg)
}
},
)
ipcMain.handle(IPC_CHANNELS.VOICE_ACTION.GET_PRESETS, async () => {
try {
return ipcSuccess(getVoiceActionService().getPresets())
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.UnknownError, msg)
}
})
ipcMain.handle(IPC_CHANNELS.VOICE_ACTION.GET_HISTORY, async () => {
try {
return ipcSuccess(getVoiceActionService().getHistory())
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.UnknownError, msg)
}
})
ipcMain.handle(IPC_CHANNELS.VOICE_ACTION.CLEAR_HISTORY, async () => {
try {
getVoiceActionService().clearHistory()
return ipcSuccess(undefined)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.UnknownError, msg)
}
})
ipcMain.handle(IPC_CHANNELS.VOICE_ACTION.SET_ENABLED, async (_event, params: { enabled: boolean }) => {
try {
getVoiceActionService().setEnabled(params.enabled)
return ipcSuccess(undefined)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.UnknownError, msg)
}
})
ipcMain.handle(IPC_CHANNELS.VOICE_ACTION.IS_ENABLED, async () => {
try {
return ipcSuccess(getVoiceActionService().isEnabled)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.UnknownError, msg)
}
})
}

View file

@ -0,0 +1,97 @@
// src/main/ipc/voice-conversation-handlers.ts
// Phase 13.1: 음성 대화 모드 IPC 핸들러
import { ipcMain } from 'electron'
import { IPC_CHANNELS } from '@shared/ipc-channels'
import { ErrorCode, ipcSuccess, ipcError } from '@shared/errors'
import { getVoiceConversationService } from '../services/VoiceConversationService'
import { getLogger } from '../services/LoggerService'
import type { ConversationSendParams } from '@shared/types'
const logger = getLogger('voice-conversation-handlers')
export function registerVoiceConversationHandlers(): void {
ipcMain.handle(IPC_CHANNELS.VOICE_CONVERSATION.START_SESSION, async () => {
try {
await getVoiceConversationService().startSession()
return ipcSuccess(undefined)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
logger.error('Conversation start failed:', msg)
return ipcError(ErrorCode.ConversationSessionAlreadyActive, msg)
}
})
ipcMain.handle(IPC_CHANNELS.VOICE_CONVERSATION.STOP_SESSION, async () => {
try {
getVoiceConversationService().stopSession()
return ipcSuccess(undefined)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.ConversationNoActiveSession, msg)
}
})
ipcMain.handle(
IPC_CHANNELS.VOICE_CONVERSATION.SEND_MESSAGE,
async (_event, params: ConversationSendParams) => {
try {
await getVoiceConversationService().sendTextMessage(params.text)
return ipcSuccess(undefined)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
logger.error('Conversation send failed:', msg)
return ipcError(ErrorCode.ConversationLLMFailed, msg)
}
},
)
ipcMain.handle(IPC_CHANNELS.VOICE_CONVERSATION.GET_STATE, async () => {
try {
return ipcSuccess(getVoiceConversationService().getSessionInfo())
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.UnknownError, msg)
}
})
ipcMain.handle(IPC_CHANNELS.VOICE_CONVERSATION.GET_HISTORY, async () => {
try {
return ipcSuccess(getVoiceConversationService().getHistory())
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.UnknownError, msg)
}
})
ipcMain.handle(IPC_CHANNELS.VOICE_CONVERSATION.CLEAR_HISTORY, async () => {
try {
getVoiceConversationService().clearHistory()
return ipcSuccess(undefined)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.UnknownError, msg)
}
})
ipcMain.handle(IPC_CHANNELS.VOICE_CONVERSATION.CANCEL_RESPONSE, async () => {
try {
getVoiceConversationService().cancelResponse()
return ipcSuccess(undefined)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.UnknownError, msg)
}
})
// finishListening — 렌더러에서 녹음 종료 버튼 클릭 시
ipcMain.handle('voiceConversation:finishListening', async () => {
try {
await getVoiceConversationService().finishListening()
return ipcSuccess(undefined)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
return ipcError(ErrorCode.ConversationLLMFailed, msg)
}
})
}

View file

@ -0,0 +1,307 @@
// src/main/services/DictationTemplateService.ts
// Phase 12.3: 딕테이션 템플릿 서비스
// 템플릿 CRUD + 세션 상태 머신 (필드별 음성 입력)
import { EventEmitter } from 'events'
import { nanoid } from 'nanoid'
import Store from 'electron-store'
import { getLogger } from './LoggerService'
import { getMainWindow } from '../windows/WindowManager'
import { IPC_CHANNELS } from '@shared/ipc-channels'
import { D3ROError, ErrorCode } from '@shared/errors'
import type {
DictationTemplate,
TemplateField,
TemplateSessionState,
TemplateSessionInfo,
TemplateFieldCompletedEvent,
TemplateSessionCompletedEvent,
CreateTemplateParams,
UpdateTemplateParams,
} from '@shared/types'
const logger = getLogger('DictationTemplateService')
// ── 프리셋 템플릿 ──
const BUILTIN_TEMPLATES: DictationTemplate[] = [
{
id: 'builtin-email',
name: 'Email',
description: 'Email template with recipient, subject, and body',
fields: [
{ id: 'recipient', name: 'recipient', label: 'Recipient', promptText: 'Who is this email for?', required: true, maxDurationSec: 15 },
{ id: 'subject', name: 'subject', label: 'Subject', promptText: 'What is the subject?', required: true, maxDurationSec: 15 },
{ id: 'body', name: 'body', label: 'Body', promptText: 'Please dictate the email body.', required: true, maxDurationSec: 120 },
],
outputFormat: 'To: {{recipient}}\nSubject: {{subject}}\n\n{{body}}',
isBuiltin: true,
createdAt: Date.now(),
updatedAt: Date.now(),
},
{
id: 'builtin-meeting-notes',
name: 'Meeting Notes',
description: 'Meeting notes template',
fields: [
{ id: 'title', name: 'title', label: 'Title', promptText: 'What is the meeting title?', required: true, maxDurationSec: 15 },
{ id: 'attendees', name: 'attendees', label: 'Attendees', promptText: 'Who attended?', required: false, maxDurationSec: 30 },
{ id: 'agenda', name: 'agenda', label: 'Agenda', promptText: 'What was discussed?', required: true, maxDurationSec: 120 },
{ id: 'decisions', name: 'decisions', label: 'Decisions', promptText: 'What decisions were made?', required: false, maxDurationSec: 60 },
],
outputFormat: '# {{title}}\n\nAttendees: {{attendees}}\n\n## Agenda\n{{agenda}}\n\n## Decisions\n{{decisions}}',
isBuiltin: true,
createdAt: Date.now(),
updatedAt: Date.now(),
},
{
id: 'builtin-report',
name: 'Report',
description: 'Simple report template',
fields: [
{ id: 'title', name: 'title', label: 'Title', promptText: 'Report title?', required: true, maxDurationSec: 15 },
{ id: 'summary', name: 'summary', label: 'Summary', promptText: 'Summarize the key points.', required: true, maxDurationSec: 60 },
{ id: 'details', name: 'details', label: 'Details', promptText: 'Provide the details.', required: true, maxDurationSec: 180 },
],
outputFormat: '# {{title}}\n\n## Summary\n{{summary}}\n\n## Details\n{{details}}',
isBuiltin: true,
createdAt: Date.now(),
updatedAt: Date.now(),
},
]
interface TemplateStoreSchema {
templates: DictationTemplate[]
}
class DictationTemplateService extends EventEmitter {
private _store: Store<TemplateStoreSchema>
private _session: TemplateSessionInfo | null = null
constructor() {
super()
this._store = new Store<TemplateStoreSchema>({
name: 'dictation-templates',
defaults: {
templates: [...BUILTIN_TEMPLATES],
},
})
// 프리셋이 없으면 추가
this._ensureBuiltins()
}
// ── CRUD ──
getAll(): DictationTemplate[] {
return this._store.get('templates', [])
}
getById(id: string): DictationTemplate | null {
const templates = this.getAll()
return templates.find((t) => t.id === id) ?? null
}
create(params: CreateTemplateParams): DictationTemplate {
const template: DictationTemplate = {
id: nanoid(),
name: params.name,
description: params.description,
fields: params.fields,
outputFormat: params.outputFormat,
isBuiltin: false,
createdAt: Date.now(),
updatedAt: Date.now(),
}
const templates = this.getAll()
templates.push(template)
this._store.set('templates', templates)
return template
}
update(params: UpdateTemplateParams): DictationTemplate {
const templates = this.getAll()
const idx = templates.findIndex((t) => t.id === params.id)
if (idx === -1) {
throw new D3ROError(ErrorCode.TemplateNotFound, `Template not found: ${params.id}`)
}
const existing = templates[idx]
const updated: DictationTemplate = {
...existing,
...(params.name !== undefined && { name: params.name }),
...(params.description !== undefined && { description: params.description }),
...(params.fields !== undefined && { fields: params.fields }),
...(params.outputFormat !== undefined && { outputFormat: params.outputFormat }),
updatedAt: Date.now(),
}
templates[idx] = updated
this._store.set('templates', templates)
return updated
}
delete(id: string): void {
const templates = this.getAll()
const template = templates.find((t) => t.id === id)
if (!template) {
throw new D3ROError(ErrorCode.TemplateNotFound, `Template not found: ${id}`)
}
if (template.isBuiltin) {
throw new D3ROError(ErrorCode.TemplateInvalidFormat, 'Cannot delete builtin template')
}
this._store.set(
'templates',
templates.filter((t) => t.id !== id),
)
}
// ── 세션 관리 ──
getSessionState(): TemplateSessionInfo | null {
return this._session
}
startSession(templateId: string): void {
if (this._session) {
throw new D3ROError(ErrorCode.TemplateSessionAlreadyActive, 'Template session already active')
}
const template = this.getById(templateId)
if (!template) {
throw new D3ROError(ErrorCode.TemplateNotFound, `Template not found: ${templateId}`)
}
if (template.fields.length === 0) {
throw new D3ROError(ErrorCode.TemplateInvalidFormat, 'Template has no fields')
}
this._session = {
templateId,
templateName: template.name,
state: 'field-prompting',
currentFieldIndex: 0,
totalFields: template.fields.length,
currentField: template.fields[0],
fieldValues: {},
}
this._emitSessionState()
logger.info(`Template session started: ${template.name}`)
}
setFieldValue(fieldId: string, value: string): void {
if (!this._session) {
throw new D3ROError(ErrorCode.TemplateSessionNotActive, 'No active template session')
}
const template = this.getById(this._session.templateId)
if (!template) {
throw new D3ROError(ErrorCode.TemplateNotFound, 'Session template not found')
}
this._session.fieldValues[fieldId] = value
const currentField = this._session.currentField
const nextIndex = this._session.currentFieldIndex + 1
const nextField = nextIndex < template.fields.length ? template.fields[nextIndex] : null
// 필드 완료 이벤트
const fieldEvent: TemplateFieldCompletedEvent = {
fieldId,
fieldName: currentField?.name ?? fieldId,
value,
nextField,
}
this._sendToRenderer(IPC_CHANNELS.DICTATION_TEMPLATE.FIELD_COMPLETED, fieldEvent)
this.emit('field-completed', fieldEvent)
if (nextField) {
// 다음 필드로 이동
this._session.currentFieldIndex = nextIndex
this._session.currentField = nextField
this._session.state = 'field-prompting'
this._emitSessionState()
} else {
// 모든 필드 완료 → 출력 생성
this._completeSession(template)
}
}
cancelSession(): void {
if (!this._session) return
logger.info('Template session cancelled')
this._session = null
this._emitSessionState()
}
private _completeSession(template: DictationTemplate): void {
if (!this._session) return
this._session.state = 'completing'
this._emitSessionState()
// 출력 포맷 적용 (mustache-like 치환)
let output = template.outputFormat
for (const [key, value] of Object.entries(this._session.fieldValues)) {
output = output.replace(new RegExp(`\\{\\{${key}\\}\\}`, 'g'), value)
}
// 미입력 필드 자리표시자 제거
output = output.replace(/\{\{[^}]+\}\}/g, '')
const completedEvent: TemplateSessionCompletedEvent = {
templateId: template.id,
outputText: output.trim(),
fieldValues: { ...this._session.fieldValues },
}
this._sendToRenderer(IPC_CHANNELS.DICTATION_TEMPLATE.SESSION_COMPLETED, completedEvent)
this.emit('session-completed', completedEvent)
this._session = null
logger.info(`Template session completed: ${template.name}`)
}
private _emitSessionState(): void {
this._sendToRenderer(IPC_CHANNELS.DICTATION_TEMPLATE.SESSION_STATE_CHANGED, this._session)
this.emit('session-state-changed', this._session)
}
private _sendToRenderer(channel: string, data: unknown): void {
try {
const mainWindow = getMainWindow()
if (mainWindow && !mainWindow.isDestroyed()) {
mainWindow.webContents.send(channel, data)
}
} catch {
// 윈도우 없으면 무시
}
}
private _ensureBuiltins(): void {
const templates = this.getAll()
for (const builtin of BUILTIN_TEMPLATES) {
if (!templates.find((t) => t.id === builtin.id)) {
templates.push(builtin)
}
}
this._store.set('templates', templates)
}
dispose(): void {
this.cancelSession()
this.removeAllListeners()
}
}
// ── 싱글톤 ──
let instance: DictationTemplateService | null = null
export function getDictationTemplateService(): DictationTemplateService {
if (!instance) {
instance = new DictationTemplateService()
}
return instance
}

View file

@ -0,0 +1,407 @@
// src/main/services/FileTranscriptionService.ts
// Phase 12.1: 파일 전사 서비스
// 오디오/비디오 파일 → ffmpeg PCM 변환 → 30초 청크 순차 STT → 병합
import { EventEmitter } from 'events'
import path from 'path'
import fs from 'fs'
import { nanoid } from 'nanoid'
import { app } from 'electron'
import { getLogger } from './LoggerService'
import { getLocalSTTService } from './LocalSTTService'
import { getHistoryService } from './HistoryService'
import { configGet } from './ConfigService'
import { getFfmpegPath } from '../utils/paths'
import { getMainWindow } from '../windows/WindowManager'
import { IPC_CHANNELS } from '@shared/ipc-channels'
import { D3ROError, ErrorCode } from '@shared/errors'
import type {
FileTranscriptionState,
FileTranscriptionProgress,
FileTranscriptionResult,
FileTranscriptionSegment,
FileTranscriptionStateInfo,
} from '@shared/types'
const logger = getLogger('FileTranscriptionService')
/** 청크 길이 (초) */
const CHUNK_DURATION_SEC = 30
/** 최대 파일 크기 (2GB) */
const MAX_FILE_SIZE_BYTES = 2 * 1024 * 1024 * 1024
/** initialPrompt 컨텍스트 윈도우 (자) */
const CONTEXT_WINDOW_SIZE = 300
/** 지원 확장자 */
const SUPPORTED_EXTENSIONS = new Set([
'.mp3', '.wav', '.m4a', '.ogg', '.flac', '.wma', '.aac',
'.mp4', '.mkv', '.webm', '.avi', '.mov',
])
class FileTranscriptionService extends EventEmitter {
private _state: FileTranscriptionState = 'idle'
private _jobId: string | null = null
private _cancelled = false
private _progress: FileTranscriptionProgress | null = null
private _tempDir: string | null = null
get state(): FileTranscriptionState {
return this._state
}
getStateInfo(): FileTranscriptionStateInfo {
return {
state: this._state,
jobId: this._jobId,
progress: this._progress,
}
}
async startTranscription(filePath: string, language?: string): Promise<FileTranscriptionResult> {
if (this._state !== 'idle') {
throw new D3ROError(ErrorCode.FileTranscriptionChunkFailed, 'Transcription already in progress')
}
// 파일 검증
const ext = path.extname(filePath).toLowerCase()
if (!SUPPORTED_EXTENSIONS.has(ext)) {
throw new D3ROError(ErrorCode.FileTranscriptionInvalidFormat, `Unsupported format: ${ext}`)
}
const stat = fs.statSync(filePath)
if (stat.size > MAX_FILE_SIZE_BYTES) {
throw new D3ROError(ErrorCode.FileTranscriptionFileTooLarge, 'File exceeds 2GB limit')
}
// 라이센스 체크
try {
const { getLicenseService } = await import('./LicenseService')
const { Feature } = await import('@shared/types')
const license = getLicenseService()
const access = license.canUse(Feature.FILE_TRANSCRIPTION)
if (!access.allowed) {
license.promptUpgrade(
Feature.FILE_TRANSCRIPTION,
access.reason === 'quota_exceeded' ? 'quota_exceeded' : 'tier_required',
)
throw new D3ROError(ErrorCode.FeatureNotAvailable, 'Pro+ required for file transcription')
}
} catch (err) {
if (err instanceof D3ROError) throw err
}
this._jobId = nanoid()
this._cancelled = false
this._tempDir = path.join(app.getPath('temp'), `d3ro-ft-${this._jobId}`)
fs.mkdirSync(this._tempDir, { recursive: true })
const startTime = Date.now()
const fileName = path.basename(filePath)
try {
// Phase 1: ffmpeg 변환 → PCM WAV
this._setState('converting')
const wavPath = path.join(this._tempDir, 'audio.wav')
await this._convertToWav(filePath, wavPath)
if (this._cancelled) {
throw new D3ROError(ErrorCode.FileTranscriptionCancelled, 'Transcription cancelled')
}
// 오디오 길이 확인
const totalDurationSec = await this._probeDuration(wavPath)
const totalChunks = Math.ceil(totalDurationSec / CHUNK_DURATION_SEC)
// Phase 2: 청크별 STT
this._setState('transcribing')
const allSegments: FileTranscriptionSegment[] = []
const allTexts: string[] = []
let previousContext = ''
for (let i = 0; i < totalChunks; i++) {
if (this._cancelled) {
throw new D3ROError(ErrorCode.FileTranscriptionCancelled, 'Transcription cancelled')
}
const startSec = i * CHUNK_DURATION_SEC
const chunkBuffer = await this._extractChunk(wavPath, startSec, CHUNK_DURATION_SEC)
const sttService = getLocalSTTService()
const lang = language ?? (configGet('sttLanguage') as string | undefined) ?? 'auto'
const result = await sttService.transcribe(chunkBuffer, {
language: lang,
initialPrompt: previousContext || undefined,
vadFilter: true,
})
if (result.text && result.text.trim().length > 0) {
allTexts.push(result.text.trim())
previousContext = result.text.trim().slice(-CONTEXT_WINDOW_SIZE)
for (const seg of result.segments) {
allSegments.push({
text: seg.text,
start: seg.start + startSec,
end: seg.end + startSec,
confidence: seg.confidence,
})
}
}
this._progress = {
jobId: this._jobId!,
currentChunk: i + 1,
totalChunks,
percent: Math.round(((i + 1) / totalChunks) * 100),
currentText: result.text?.trim() ?? '',
}
this._sendToRenderer(IPC_CHANNELS.FILE_TRANSCRIPTION.PROGRESS, this._progress)
this.emit('progress', this._progress)
}
const fullText = allTexts.join(' ')
const processingTimeMs = Date.now() - startTime
const resultData: FileTranscriptionResult = {
jobId: this._jobId!,
filePath,
fileName,
fullText,
segments: allSegments,
totalDurationSec,
processingTimeMs,
}
// 히스토리에 저장
try {
const wordCount = fullText.split(/\s+/).filter((w) => w.length > 0).length
getHistoryService().create({
originalText: fullText,
polishedText: null,
focusedApp: null,
focusedAppName: null,
focusedAppWindowTitle: null,
mode: 'file-transcription',
status: 'completed',
errorCode: null,
audioLocalPath: filePath,
duration: totalDurationSec,
detectedLanguage: null,
micDevice: null,
wordCount,
sttModel: configGet('sttModelId') as string | null,
llmModel: null,
sttLatencyMs: processingTimeMs,
llmLatencyMs: null,
appVersion: app.getVersion(),
})
} catch (err) {
logger.warn('Failed to save file transcription to history:', err)
}
this._setState('completed')
this._sendToRenderer(IPC_CHANNELS.FILE_TRANSCRIPTION.COMPLETE, resultData)
this.emit('complete', resultData)
return resultData
} catch (err) {
if (err instanceof D3ROError && err.code === ErrorCode.FileTranscriptionCancelled) {
this._setState('idle')
throw err
}
this._setState('error')
const errorMsg = err instanceof Error ? err.message : String(err)
this._sendToRenderer(IPC_CHANNELS.FILE_TRANSCRIPTION.ERROR, { message: errorMsg })
this.emit('error', err)
throw err instanceof D3ROError
? err
: new D3ROError(ErrorCode.FileTranscriptionChunkFailed, errorMsg)
} finally {
this._cleanup()
// 완료/에러 후 idle로 복귀
setTimeout(() => {
this._state = 'idle'
this._jobId = null
this._progress = null
}, 1000)
}
}
cancel(): void {
if (this._state !== 'idle') {
this._cancelled = true
logger.info(`File transcription cancelled: ${this._jobId}`)
}
}
private _setState(state: FileTranscriptionState): void {
this._state = state
this.emit('state-changed', state)
}
private _sendToRenderer(channel: string, data: unknown): void {
try {
const mainWindow = getMainWindow()
if (mainWindow && !mainWindow.isDestroyed()) {
mainWindow.webContents.send(channel, data)
}
} catch {
// 윈도우 없으면 무시
}
}
/**
* ffmpeg로 PCM16 16kHz mono WAV로
*/
private _convertToWav(inputPath: string, outputPath: string): Promise<void> {
return new Promise((resolve, reject) => {
const { spawn } = require('child_process') as typeof import('child_process')
const ffmpegPath = getFfmpegPath()
const args = [
'-i', inputPath,
'-ar', '16000',
'-ac', '1',
'-sample_fmt', 's16',
'-y',
outputPath,
]
logger.info(`ffmpeg convert: ${ffmpegPath} ${args.join(' ')}`)
const proc = spawn(ffmpegPath, args, { stdio: ['pipe', 'pipe', 'pipe'] })
let stderr = ''
proc.stderr?.on('data', (data: Buffer) => {
stderr += data.toString()
})
proc.on('close', (code: number) => {
if (code === 0) {
resolve()
} else {
logger.error(`ffmpeg failed (code ${code}):`, stderr.slice(-500))
reject(new D3ROError(ErrorCode.FileTranscriptionFFmpegFailed, `ffmpeg exited with code ${code}`))
}
})
proc.on('error', (err: Error) => {
reject(new D3ROError(ErrorCode.FileTranscriptionFFmpegFailed, `ffmpeg error: ${err.message}`))
})
})
}
/**
* ffprobe(ffmpeg) ()
*/
private _probeDuration(wavPath: string): Promise<number> {
return new Promise((resolve, reject) => {
const { spawn } = require('child_process') as typeof import('child_process')
const ffmpegPath = getFfmpegPath()
// ffprobe는 보통 ffmpeg과 같은 디렉토리에 있으나,
// @ffmpeg-installer는 ffmpeg만 제공 → -i로 duration 추출
const args = [
'-i', wavPath,
'-f', 'null',
'-',
]
const proc = spawn(ffmpegPath, args, { stdio: ['pipe', 'pipe', 'pipe'] })
let stderr = ''
proc.stderr?.on('data', (data: Buffer) => {
stderr += data.toString()
})
proc.on('close', () => {
// "Duration: HH:MM:SS.ms" 패턴 파싱
const match = stderr.match(/Duration:\s*(\d+):(\d+):(\d+)\.(\d+)/)
if (match) {
const hours = parseInt(match[1], 10)
const minutes = parseInt(match[2], 10)
const seconds = parseInt(match[3], 10)
const ms = parseInt(match[4], 10) / 100
resolve(hours * 3600 + minutes * 60 + seconds + ms)
} else {
// WAV 파일 크기로 폴백 추정 (16kHz 16bit mono = 32000 bytes/sec)
try {
const stat = fs.statSync(wavPath)
const headerSize = 44
const bytesPerSec = 16000 * 2 * 1
resolve(Math.max(0, (stat.size - headerSize) / bytesPerSec))
} catch {
reject(new D3ROError(ErrorCode.FileTranscriptionFFmpegFailed, 'Cannot determine audio duration'))
}
}
})
})
}
/**
* WAV PCM16 Buffer로
*/
private _extractChunk(wavPath: string, startSec: number, durationSec: number): Promise<Buffer> {
return new Promise((resolve, reject) => {
const { spawn } = require('child_process') as typeof import('child_process')
const ffmpegPath = getFfmpegPath()
const args = [
'-ss', String(startSec),
'-t', String(durationSec),
'-i', wavPath,
'-ar', '16000',
'-ac', '1',
'-f', 's16le',
'-acodec', 'pcm_s16le',
'pipe:1',
]
const proc = spawn(ffmpegPath, args, { stdio: ['pipe', 'pipe', 'pipe'] })
const chunks: Buffer[] = []
proc.stdout?.on('data', (data: Buffer) => {
chunks.push(data)
})
proc.on('close', (code: number) => {
if (code === 0 || chunks.length > 0) {
resolve(Buffer.concat(chunks))
} else {
reject(new D3ROError(ErrorCode.FileTranscriptionChunkFailed, `Chunk extraction failed at ${startSec}s`))
}
})
proc.on('error', (err: Error) => {
reject(new D3ROError(ErrorCode.FileTranscriptionFFmpegFailed, `ffmpeg chunk error: ${err.message}`))
})
})
}
private _cleanup(): void {
if (this._tempDir && fs.existsSync(this._tempDir)) {
try {
fs.rmSync(this._tempDir, { recursive: true, force: true })
} catch (err) {
logger.warn('Failed to cleanup temp dir:', err)
}
this._tempDir = null
}
}
dispose(): void {
this.cancel()
this._cleanup()
this.removeAllListeners()
}
}
// ── 싱글톤 ──
let instance: FileTranscriptionService | null = null
export function getFileTranscriptionService(): FileTranscriptionService {
if (!instance) {
instance = new FileTranscriptionService()
}
return instance
}

View file

@ -348,15 +348,32 @@ class LicenseService extends EventEmitter {
return { success: false, tier: 'free', message: 'License key is empty' } return { success: false, tier: 'free', message: 'License key is empty' }
} }
// 1. LemonSqueezy API 시도 // 1. 로컬 키 검증 먼저 (개발/테스트용 D3RO-PRO-*, D3RO-PLUS-* 패턴)
const localTier = this._validateKeyLocally(trimmedKey)
if (localTier) {
logger.info(`Local key validated: tier=${localTier}`)
const now = Date.now()
this._info = {
...this._info,
tier: localTier,
licenseKey: trimmedKey,
activatedAt: now,
lastVerifiedAt: now,
offlineGraceUntil: now + OFFLINE_GRACE_PERIOD_MS,
}
this._persistLicenseInfo()
this._sendToRenderer(IPC_CHANNELS.LICENSE.TIER_CHANGED, this._info)
return { success: true, tier: localTier, message: `Activated ${localTier} license (local)` }
}
// 2. LemonSqueezy API 시도
const apiResult = await this._activateViaLemonSqueezy(trimmedKey) const apiResult = await this._activateViaLemonSqueezy(trimmedKey)
if (apiResult) { if (apiResult) {
return apiResult return apiResult
} }
// 2. API 실패 시 로컬 키 검증 폴백 (개발/테스트용) // 3. 둘 다 실패
logger.info('LemonSqueezy API unreachable, trying local key validation') const tier = null as LicenseTier | null
const tier = this._validateKeyLocally(trimmedKey)
if (!tier) { if (!tier) {
return { success: false, tier: 'free', message: 'Invalid license key' } return { success: false, tier: 'free', message: 'Invalid license key' }
} }
@ -506,6 +523,27 @@ class LicenseService extends EventEmitter {
pro: false, pro: false,
proPlus: true, proPlus: true,
}, },
{
feature: Feature.DICTATION_TEMPLATE,
featureLabel: 'license.feature.dictationTemplate',
free: false,
pro: false,
proPlus: true,
},
{
feature: Feature.LOCAL_RAG,
featureLabel: 'license.feature.localRag',
free: false,
pro: false,
proPlus: true,
},
{
feature: Feature.OS_AUTOMATION,
featureLabel: 'license.feature.osAutomation',
free: false,
pro: false,
proPlus: true,
},
] ]
} }

View file

@ -378,6 +378,81 @@ class LocalLLMService extends EventEmitter {
return this._available return this._available
} }
/**
* Ollama /api/chat .
* messages .
* yield, return.
*/
async *chatStream(
messages: Array<{ role: string; content: string }>,
options?: { model?: string; temperature?: number },
): AsyncGenerator<string, string> {
if (!this._available) {
throw new D3ROError(ErrorCode.LLMServerUnreachable, 'Ollama server not available')
}
const serverUrl = configGet('ollamaServerUrl')
const model = options?.model ?? configGet('llmModelId') ?? 'qwen3:4b'
this._abortController = new AbortController()
this._state = LLMState.Generating
try {
const response = await fetch(`${serverUrl}/api/chat`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
model,
messages,
stream: true,
options: {
temperature: options?.temperature ?? 0.7,
},
}),
signal: this._abortController.signal,
})
if (!response.ok || !response.body) {
throw new D3ROError(ErrorCode.LLMProcessingFailed, `Chat API error: ${response.status}`)
}
const reader = response.body.getReader()
const decoder = new TextDecoder()
let buffer = ''
let accumulated = ''
while (true) {
const { done, value } = await reader.read()
if (done) break
buffer += decoder.decode(value, { stream: true })
const lines = buffer.split('\n')
buffer = lines.pop() ?? ''
for (const line of lines) {
if (!line.trim()) continue
try {
const chunk = JSON.parse(line) as { message?: { content: string }; done: boolean }
if (chunk.message?.content) {
accumulated += chunk.message.content
yield chunk.message.content
}
if (chunk.done) {
return accumulated
}
} catch {
// 불완전 JSON 무시
}
}
}
return accumulated
} finally {
this._state = this._available ? LLMState.Available : LLMState.Unavailable
this._abortController = null
}
}
dispose(): void { dispose(): void {
this._disposed = true this._disposed = true
this.stopPolling() this.stopPolling()

View file

@ -0,0 +1,257 @@
// src/main/services/MeetingSummaryService.ts
// Phase 12.2: 회의록 자동 요약 서비스
// CaptionService 세션 종료 후 LLM으로 요약 생성
import { EventEmitter } from 'events'
import path from 'path'
import fs from 'fs'
import { app, dialog } from 'electron'
import { eq } from 'drizzle-orm'
import { getLogger } from './LoggerService'
import { getLocalLLMService } from './LocalLLMService'
import { getDatabase } from '../db'
import { history } from '../db/schema'
import { getMainWindow } from '../windows/WindowManager'
import { IPC_CHANNELS } from '@shared/ipc-channels'
import { D3ROError, ErrorCode } from '@shared/errors'
import type { MeetingSummaryResult, MeetingSummaryProgress } from '@shared/types'
const logger = getLogger('MeetingSummaryService')
const MEETING_SUMMARY_PROMPT = `다음은 회의 전사록입니다. 아래 형식으로 정리해주세요:
##
(3 )
##
- ( 1)
- ( 2)
##
- [ ] ( 1)
- [ ] ( 2)
. "요약할 내용이 충분하지 않습니다" .`
class MeetingSummaryService extends EventEmitter {
/**
* LLM으로
*/
async summarize(historyId: string): Promise<MeetingSummaryResult> {
// 라이센스 체크
try {
const { getLicenseService } = await import('./LicenseService')
const { Feature } = await import('@shared/types')
const license = getLicenseService()
const access = license.canUse(Feature.MEETING_SUMMARY)
if (!access.allowed) {
license.promptUpgrade(
Feature.MEETING_SUMMARY,
access.reason === 'quota_exceeded' ? 'quota_exceeded' : 'tier_required',
)
throw new D3ROError(ErrorCode.FeatureNotAvailable, 'Pro+ required for meeting summary')
}
} catch (err) {
if (err instanceof D3ROError) throw err
}
// 히스토리 항목 조회
const db = getDatabase()
const rows = db.select().from(history).where(eq(history.id, historyId)).all()
if (rows.length === 0) {
throw new D3ROError(ErrorCode.HistoryNotFound, `History not found: ${historyId}`)
}
const entry = rows[0]
const transcript = entry.originalText
if (!transcript || transcript.trim().length === 0) {
throw new D3ROError(ErrorCode.MeetingSummaryNoTranscript, 'No transcript text to summarize')
}
// 진행 상태 알림
this._sendProgress(historyId, 'generating')
try {
// LLM 요약 생성
const llmService = getLocalLLMService()
const result = await llmService.generate(transcript, {
systemPrompt: MEETING_SUMMARY_PROMPT,
temperature: 0.3,
})
const rawMarkdown = result.text.trim()
const parsed = this._parseMarkdown(rawMarkdown)
const summaryResult: MeetingSummaryResult = {
historyId,
summary: parsed.summary,
decisions: parsed.decisions,
actionItems: parsed.actionItems,
rawMarkdown,
generatedAt: Date.now(),
}
// DB에 요약 저장
db.update(history)
.set({ summaryText: rawMarkdown, updatedAt: Date.now() })
.where(eq(history.id, historyId))
.run()
this._sendProgress(historyId, 'done')
this._sendToRenderer(IPC_CHANNELS.MEETING_SUMMARY.SUMMARY_READY, summaryResult)
this.emit('summary-ready', summaryResult)
return summaryResult
} catch (err) {
this._sendProgress(historyId, 'error')
if (err instanceof D3ROError) throw err
const msg = err instanceof Error ? err.message : String(err)
throw new D3ROError(ErrorCode.MeetingSummaryGenerationFailed, msg)
}
}
/**
*
*/
getSummary(historyId: string): MeetingSummaryResult | null {
const db = getDatabase()
const rows = db.select().from(history).where(eq(history.id, historyId)).all()
if (rows.length === 0 || !rows[0].summaryText) {
return null
}
const entry = rows[0]
const parsed = this._parseMarkdown(entry.summaryText!)
return {
historyId,
summary: parsed.summary,
decisions: parsed.decisions,
actionItems: parsed.actionItems,
rawMarkdown: entry.summaryText!,
generatedAt: entry.updatedAt,
}
}
/**
*
*/
async exportMarkdown(historyId: string): Promise<string> {
const summary = this.getSummary(historyId)
if (!summary) {
throw new D3ROError(ErrorCode.MeetingSummaryExportFailed, 'No summary to export')
}
const db = getDatabase()
const rows = db.select().from(history).where(eq(history.id, historyId)).all()
const entry = rows[0]
const date = new Date(entry.createdAt)
const dateStr = date.toISOString().slice(0, 10)
const defaultName = `meeting-summary-${dateStr}.md`
const result = await dialog.showSaveDialog({
defaultPath: path.join(app.getPath('documents'), defaultName),
filters: [{ name: 'Markdown', extensions: ['md'] }],
})
if (result.canceled || !result.filePath) {
throw new D3ROError(ErrorCode.MeetingSummaryExportFailed, 'Export cancelled')
}
const content = `# Meeting Summary — ${dateStr}\n\n${summary.rawMarkdown}\n\n---\n\n## Full Transcript\n\n${entry.originalText}\n`
fs.writeFileSync(result.filePath, content, 'utf-8')
return result.filePath
}
/**
* CaptionService ()
*/
async onCaptionSessionSaved(sessionSummary: { sessionId: string }): Promise<void> {
// caption 모드 히스토리에서 해당 세션 찾기
const db = getDatabase()
const rows = db
.select()
.from(history)
.where(eq(history.id, sessionSummary.sessionId))
.all()
if (rows.length === 0) {
logger.warn(`Caption session not found in history: ${sessionSummary.sessionId}`)
return
}
const entry = rows[0]
if (!entry.originalText || entry.originalText.trim().length < 50) {
logger.info('Caption session too short for summary, skipping')
return
}
try {
await this.summarize(entry.id)
logger.info(`Auto-summary generated for caption session: ${entry.id}`)
} catch (err) {
logger.warn('Auto-summary failed:', err)
}
}
private _parseMarkdown(markdown: string): {
summary: string
decisions: string[]
actionItems: string[]
} {
const sections = markdown.split(/^## /m)
let summary = ''
const decisions: string[] = []
const actionItems: string[] = []
for (const section of sections) {
const lines = section.trim().split('\n')
const heading = lines[0]?.trim().toLowerCase() ?? ''
const body = lines.slice(1).join('\n').trim()
if (heading.includes('요약') || heading.includes('summary')) {
summary = body
} else if (heading.includes('결정') || heading.includes('decision')) {
const items = body.split('\n').filter((l) => l.trim().startsWith('-'))
decisions.push(...items.map((l) => l.replace(/^-\s*/, '').trim()))
} else if (heading.includes('할 일') || heading.includes('action')) {
const items = body.split('\n').filter((l) => l.trim().startsWith('-'))
actionItems.push(
...items.map((l) => l.replace(/^-\s*\[[ x]?\]\s*/, '').trim()),
)
}
}
return { summary, decisions, actionItems }
}
private _sendProgress(historyId: string, status: MeetingSummaryProgress['status']): void {
this._sendToRenderer(IPC_CHANNELS.MEETING_SUMMARY.SUMMARY_PROGRESS, { historyId, status })
}
private _sendToRenderer(channel: string, data: unknown): void {
try {
const mainWindow = getMainWindow()
if (mainWindow && !mainWindow.isDestroyed()) {
mainWindow.webContents.send(channel, data)
}
} catch {
// 윈도우 없으면 무시
}
}
dispose(): void {
this.removeAllListeners()
}
}
// ── 싱글톤 ──
let instance: MeetingSummaryService | null = null
export function getMeetingSummaryService(): MeetingSummaryService {
if (!instance) {
instance = new MeetingSummaryService()
}
return instance
}

View file

@ -0,0 +1,536 @@
// src/main/services/RAGService.ts
// Phase 13.2: 로컬 RAG — 문서 임베딩 + 코사인 유사도 검색 + LLM 컨텍스트 주입
// Ollama nomic-embed-text 모델, SQLite에 JSON 직렬화 벡터 저장.
import { EventEmitter } from 'events'
import path from 'path'
import fs from 'fs'
import { nanoid } from 'nanoid'
import { eq } from 'drizzle-orm'
import { getLogger } from './LoggerService'
import { getLocalLLMService } from './LocalLLMService'
import { configGet } from './ConfigService'
import { getDatabase } from '../db'
import { ragDocuments, ragChunks } from '../db/schema'
import { getMainWindow } from '../windows/WindowManager'
import { IPC_CHANNELS } from '@shared/ipc-channels'
import { D3ROError, ErrorCode } from '@shared/errors'
import type {
RAGDocument,
RAGQueryResult,
RAGState,
RAGStateInfo,
RAGIndexProgress,
} from '@shared/types'
const logger = getLogger('RAGService')
/** 임베딩 모델 */
const EMBED_MODEL = 'nomic-embed-text'
/** 청크 크기 (자) */
const CHUNK_SIZE = 500
/** 청크 오버랩 (자) */
const CHUNK_OVERLAP = 50
/** 검색 기본 topK */
const DEFAULT_TOP_K = 5
class RAGService extends EventEmitter {
private _state: RAGState = 'idle'
get state(): RAGState {
return this._state
}
getStateInfo(): RAGStateInfo {
const db = getDatabase()
const docs = db.select().from(ragDocuments).all()
const chunks = db.select().from(ragChunks).all()
return {
state: this._state,
documentCount: docs.length,
totalChunks: chunks.length,
}
}
getDocuments(): RAGDocument[] {
const db = getDatabase()
const rows = db.select().from(ragDocuments).all()
return rows.map((r) => ({
id: r.id,
fileName: r.fileName,
filePath: r.filePath,
fileType: r.fileType as RAGDocument['fileType'],
chunkCount: r.chunkCount,
indexed: r.indexed,
indexedAt: r.indexedAt,
addedAt: r.addedAt,
}))
}
/**
* + ( DB )
*/
async addDocument(filePath: string): Promise<RAGDocument> {
// 라이센스 체크
try {
const { getLicenseService } = await import('./LicenseService')
const { Feature } = await import('@shared/types')
const license = getLicenseService()
const access = license.canUse(Feature.LOCAL_RAG)
if (!access.allowed) {
license.promptUpgrade(Feature.LOCAL_RAG, 'tier_required')
throw new D3ROError(ErrorCode.FeatureNotAvailable, 'Pro+ required for Local RAG')
}
} catch (err) {
if (err instanceof D3ROError) throw err
}
const ext = path.extname(filePath).toLowerCase()
const supportedTypes: Record<string, RAGDocument['fileType']> = {
'.txt': 'txt',
'.md': 'md',
'.pdf': 'pdf',
'.docx': 'docx',
}
const fileType = supportedTypes[ext]
if (!fileType) {
throw new D3ROError(ErrorCode.RAGUnsupportedFormat, `Unsupported format: ${ext}. Supported: .txt, .md, .pdf, .docx`)
}
const fileName = path.basename(filePath)
const docId = nanoid()
// 텍스트 추출
let content: string
try {
content = await this._extractText(filePath, fileType)
} catch (err) {
if (err instanceof D3ROError) throw err
throw new D3ROError(ErrorCode.RAGIndexingFailed, `Text extraction failed: ${err instanceof Error ? err.message : String(err)}`)
}
if (!content || content.trim().length < 20) {
throw new D3ROError(ErrorCode.RAGIndexingFailed, 'No readable text content found in document')
}
logger.info(`RAG text extracted: ${fileName} (${content.length} chars)`)
// 청킹
const chunks = this._chunkText(content)
if (chunks.length === 0) {
throw new D3ROError(ErrorCode.RAGIndexingFailed, 'Document produced no valid text chunks')
}
// DB에 문서 레코드 삽입
const db = getDatabase()
db.insert(ragDocuments).values({
id: docId,
fileName,
filePath,
fileType,
chunkCount: chunks.length,
indexed: false,
indexedAt: null,
addedAt: Date.now(),
}).run()
// 비동기 인덱싱 (임베딩 생성)
this._indexDocument(docId, fileName, chunks).catch((err) => {
logger.error(`Indexing failed for ${fileName}:`, err)
})
return {
id: docId,
fileName,
filePath,
fileType,
chunkCount: chunks.length,
indexed: false,
indexedAt: null,
addedAt: Date.now(),
}
}
/**
* ( )
*/
removeDocument(documentId: string): void {
const db = getDatabase()
db.delete(ragChunks).where(eq(ragChunks.documentId, documentId)).run()
db.delete(ragDocuments).where(eq(ragDocuments.id, documentId)).run()
logger.info(`RAG document removed: ${documentId}`)
}
/**
*
*/
async reindex(documentId: string): Promise<void> {
const db = getDatabase()
const rows = db.select().from(ragDocuments).where(eq(ragDocuments.id, documentId)).all()
if (rows.length === 0) {
throw new D3ROError(ErrorCode.RAGDocumentNotFound, 'Document not found')
}
const doc = rows[0]
// 기존 청크 삭제
db.delete(ragChunks).where(eq(ragChunks.documentId, documentId)).run()
// 텍스트 재추출 + 재인덱싱
const content = await this._extractText(doc.filePath, doc.fileType as RAGDocument['fileType'])
const chunks = this._chunkText(content)
db.update(ragDocuments)
.set({ chunkCount: chunks.length, indexed: false })
.where(eq(ragDocuments.id, documentId))
.run()
await this._indexDocument(documentId, doc.fileName, chunks)
}
/**
* + LLM
*/
async query(queryText: string, topK: number = DEFAULT_TOP_K): Promise<RAGQueryResult> {
this._state = 'querying'
try {
// 쿼리 임베딩
const queryEmbedding = await this._embed(queryText)
// 모든 청크에서 코사인 유사도 계산
const db = getDatabase()
const allChunks = db.select().from(ragChunks).all()
const allDocs = db.select().from(ragDocuments).all()
const docMap = new Map(allDocs.map((d) => [d.id, d.fileName]))
const scored = allChunks.map((chunk) => {
const embedding = JSON.parse(chunk.embedding) as number[]
const similarity = this._cosineSimilarity(queryEmbedding, embedding)
return {
documentId: chunk.documentId,
fileName: docMap.get(chunk.documentId) ?? 'unknown',
content: chunk.content,
similarity,
}
})
// 상위 topK
scored.sort((a, b) => b.similarity - a.similarity)
const topResults = scored.slice(0, topK)
// LLM에 컨텍스트 주입
const context = topResults
.map((r, i) => `[${i + 1}] (${r.fileName})\n${r.content}`)
.join('\n\n')
const systemPrompt = `You are a helpful assistant. Answer the user's question based on the following documents. If the documents don't contain relevant information, say so. Respond in the same language as the question.
Documents:
${context}`
const llmService = getLocalLLMService()
const result = await llmService.generate(queryText, { systemPrompt })
return {
query: queryText,
results: topResults,
answer: result.text.trim(),
}
} finally {
this._state = 'idle'
}
}
// ── 내부 메서드 ──
private async _indexDocument(docId: string, fileName: string, chunks: string[]): Promise<void> {
this._state = 'indexing'
const db = getDatabase()
logger.info(`RAG indexing started: ${fileName} (${chunks.length} chunks)`)
// 초기 진행률 즉시 전송
this._sendToRenderer(IPC_CHANNELS.RAG.INDEX_PROGRESS, {
documentId: docId,
fileName,
currentChunk: 0,
totalChunks: chunks.length,
percent: 0,
} as RAGIndexProgress)
let successCount = 0
for (let i = 0; i < chunks.length; i++) {
try {
const embedding = await this._embed(chunks[i])
db.insert(ragChunks).values({
id: nanoid(),
documentId: docId,
content: chunks[i],
embedding: JSON.stringify(embedding),
chunkIndex: i,
}).run()
successCount++
} catch (err) {
logger.warn(`RAG embedding failed for chunk ${i}/${chunks.length} of ${fileName}:`, err)
// 개별 청크 실패는 건너뛰고 계속 진행
}
const progress: RAGIndexProgress = {
documentId: docId,
fileName,
currentChunk: i + 1,
totalChunks: chunks.length,
percent: Math.round(((i + 1) / chunks.length) * 100),
}
this._sendToRenderer(IPC_CHANNELS.RAG.INDEX_PROGRESS, progress)
// 이벤트 루프 양보 (UI 블로킹 방지)
await new Promise((r) => setTimeout(r, 10))
}
// 인덱싱 완료 표시
db.update(ragDocuments)
.set({ indexed: true, indexedAt: Date.now(), chunkCount: successCount })
.where(eq(ragDocuments.id, docId))
.run()
this._state = 'idle'
this._sendToRenderer(IPC_CHANNELS.RAG.INDEX_COMPLETE, { documentId: docId, fileName })
logger.info(`RAG indexing complete: ${fileName} (${successCount}/${chunks.length} chunks embedded)`)
}
/**
* Ollama /api/embed
*/
private async _embed(text: string): Promise<number[]> {
const serverUrl = configGet('ollamaServerUrl')
try {
const response = await fetch(`${serverUrl}/api/embed`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
model: EMBED_MODEL,
input: text,
}),
signal: AbortSignal.timeout(30000),
})
if (!response.ok) {
throw new D3ROError(ErrorCode.RAGEmbeddingFailed, `Embed API error: ${response.status}`)
}
const data = (await response.json()) as { embeddings: number[][] }
if (!data.embeddings || data.embeddings.length === 0) {
throw new D3ROError(ErrorCode.RAGEmbeddingFailed, 'No embeddings returned')
}
return data.embeddings[0]
} catch (err) {
if (err instanceof D3ROError) throw err
throw new D3ROError(
ErrorCode.RAGEmbeddingFailed,
`Embedding failed: ${err instanceof Error ? err.message : String(err)}`,
)
}
}
private _chunkText(text: string): string[] {
// 텍스트 크기 제한 (500KB 초과 시 잘라냄)
const MAX_TEXT_LENGTH = 500000
const safeText = text.length > MAX_TEXT_LENGTH ? text.slice(0, MAX_TEXT_LENGTH) : text
const chunks: string[] = []
let start = 0
const MAX_CHUNKS = 2000
while (start < safeText.length && chunks.length < MAX_CHUNKS) {
const end = Math.min(start + CHUNK_SIZE, safeText.length)
const chunk = safeText.slice(start, end).trim()
if (chunk.length > 10) {
chunks.push(chunk)
}
start = end - CHUNK_OVERLAP
if (start >= safeText.length) break
}
return chunks
}
private async _extractText(filePath: string, fileType: string): Promise<string> {
const { promises: fsp } = await import('fs')
if (fileType === 'txt' || fileType === 'md') {
return fsp.readFile(filePath, 'utf-8')
}
if (fileType === 'pdf') {
try {
const buffer = await fsp.readFile(filePath)
const text = this._extractPdfText(buffer)
if (text.trim().length < 10) {
throw new Error('No readable text found in PDF')
}
return text.trim()
} catch (err) {
logger.error('PDF parsing failed:', err)
throw new D3ROError(ErrorCode.RAGUnsupportedFormat, 'PDF parsing failed. Ensure the PDF contains readable text.')
}
}
if (fileType === 'docx') {
try {
const buffer = await fsp.readFile(filePath)
// DOCX는 ZIP 내 XML — w:t 태그에서 텍스트 추출
const raw = buffer.toString('utf-8')
const matches = raw.match(/<w:t[^>]*>([^<]*)<\/w:t>/g)
if (matches) {
const text = matches
.map((m) => m.replace(/<[^>]+>/g, ''))
.join(' ')
return text.trim()
}
// XML 태그 제거 폴백
return raw.replace(/<[^>]+>/g, ' ').replace(/\s+/g, ' ').trim().slice(0, 500000)
} catch {
throw new D3ROError(ErrorCode.RAGUnsupportedFormat, 'DOCX parsing failed')
}
}
throw new D3ROError(ErrorCode.RAGUnsupportedFormat, `Unsupported: ${fileType}`)
}
/**
* PDF ( JS + zlib).
* FlateDecode BT...ET Tj/TJ .
*/
private _extractPdfText(buffer: Buffer): string {
const zlib = require('zlib') as typeof import('zlib')
const raw = buffer.toString('binary')
const textParts: string[] = []
// 스트림 블록 추출 — 바이너리 오프셋 기반
const streamMarker = 'stream\r\n'
const streamMarker2 = 'stream\n'
const endMarker = 'endstream'
let pos = 0
while (pos < raw.length) {
let streamStart = raw.indexOf(streamMarker, pos)
let offset = streamMarker.length
if (streamStart === -1) {
streamStart = raw.indexOf(streamMarker2, pos)
offset = streamMarker2.length
}
if (streamStart === -1) break
const dataStart = streamStart + offset
const streamEnd = raw.indexOf(endMarker, dataStart)
if (streamEnd === -1) break
const streamData = Buffer.from(raw.slice(dataStart, streamEnd), 'binary')
pos = streamEnd + endMarker.length
// FlateDecode 해제 시도
let decoded: string
try {
const inflated = zlib.inflateSync(streamData)
decoded = inflated.toString('binary')
} catch {
// 압축 안 된 스트림
decoded = streamData.toString('binary')
}
// BT...ET 블록 파싱
this._extractTextFromStream(decoded, textParts)
}
// PDF 이스케이프 디코딩 + 정리
let text = textParts.join(' ')
text = text
.replace(/\\n/g, '\n')
.replace(/\\r/g, '\r')
.replace(/\\t/g, '\t')
.replace(/\\\(/g, '(')
.replace(/\\\)/g, ')')
.replace(/\\\\/g, '\\')
.replace(/[^\x20-\x7E\u00A0-\u00FF\u3000-\u9FFF\uAC00-\uD7AF\n]/g, ' ')
.replace(/\s+/g, ' ')
return text.trim()
}
private _extractTextFromStream(content: string, textParts: string[]): void {
const btRegex = /BT([\s\S]*?)ET/g
let btMatch: RegExpExecArray | null = null
while ((btMatch = btRegex.exec(content)) !== null) {
const block = btMatch[1]
// Tj: (text) Tj
const tjRegex = /\(([^)]*)\)\s*Tj/g
let tjMatch: RegExpExecArray | null = null
while ((tjMatch = tjRegex.exec(block)) !== null) {
if (tjMatch[1].trim()) textParts.push(tjMatch[1])
}
// TJ: [(text) num (text)] TJ
const tjArrayRegex = /\[(.*?)\]\s*TJ/g
let tjArrayMatch: RegExpExecArray | null = null
while ((tjArrayMatch = tjArrayRegex.exec(block)) !== null) {
const items = tjArrayMatch[1]
const itemRegex = /\(([^)]*)\)/g
let itemMatch: RegExpExecArray | null = null
while ((itemMatch = itemRegex.exec(items)) !== null) {
if (itemMatch[1].trim()) textParts.push(itemMatch[1])
}
}
// ' 연산자: (text) '
const quoteRegex = /\(([^)]*)\)\s*'/g
let quoteMatch: RegExpExecArray | null = null
while ((quoteMatch = quoteRegex.exec(block)) !== null) {
if (quoteMatch[1].trim()) textParts.push(quoteMatch[1])
}
}
}
private _cosineSimilarity(a: number[], b: number[]): number {
if (a.length !== b.length) return 0
let dotProduct = 0
let normA = 0
let normB = 0
for (let i = 0; i < a.length; i++) {
dotProduct += a[i] * b[i]
normA += a[i] * a[i]
normB += b[i] * b[i]
}
const denom = Math.sqrt(normA) * Math.sqrt(normB)
return denom === 0 ? 0 : dotProduct / denom
}
private _sendToRenderer(channel: string, data: unknown): void {
try {
const mainWindow = getMainWindow()
if (mainWindow && !mainWindow.isDestroyed()) {
mainWindow.webContents.send(channel, data)
}
} catch {
// ignore
}
}
dispose(): void {
this.removeAllListeners()
}
}
// ── 싱글톤 ──
let instance: RAGService | null = null
export function getRAGService(): RAGService {
if (!instance) {
instance = new RAGService()
}
return instance
}

View file

@ -0,0 +1,152 @@
// src/main/services/TTSPlaybackService.ts
// Phase 13.1: TTS 재생 서비스
// Windows SAPI (PowerShell) 기반 로컬 TTS.
// 온라인 불필요, 완전 로컬. 문장 단위 큐 재생.
import { EventEmitter } from 'events'
import { spawn, type ChildProcess } from 'child_process'
import { getLogger } from './LoggerService'
import { configGet } from './ConfigService'
import { D3ROError, ErrorCode } from '@shared/errors'
const logger = getLogger('TTSPlaybackService')
class TTSPlaybackService extends EventEmitter {
private _speaking = false
private _queue: string[] = []
private _currentProcess: ChildProcess | null = null
private _cancelled = false
get isSpeaking(): boolean {
return this._speaking
}
/**
* (Windows SAPI).
* .
*/
async speak(text: string): Promise<void> {
if (!text.trim()) return
this._queue.push(text.trim())
if (!this._speaking) {
await this._processQueue()
}
}
/**
* .
* LLM .
*/
async speakSentences(sentences: string[]): Promise<void> {
for (const sentence of sentences) {
if (this._cancelled) break
this._queue.push(sentence.trim())
}
if (!this._speaking) {
await this._processQueue()
}
}
/**
* + .
*/
stop(): void {
this._cancelled = true
this._queue = []
if (this._currentProcess) {
this._currentProcess.kill()
this._currentProcess = null
}
this._speaking = false
this.emit('stopped')
}
private async _processQueue(): Promise<void> {
this._speaking = true
this._cancelled = false
this.emit('started')
while (this._queue.length > 0 && !this._cancelled) {
const text = this._queue.shift()!
try {
await this._speakOne(text)
} catch (err) {
logger.warn('TTS playback failed for segment:', err)
}
}
this._speaking = false
if (!this._cancelled) {
this.emit('finished')
}
}
/**
* PowerShell SAPI로 .
*/
private _speakOne(text: string): Promise<void> {
return new Promise((resolve, reject) => {
// 텍스트를 PowerShell 안전 문자열로 이스케이프
const escaped = text
.replace(/'/g, "''")
.replace(/\n/g, ' ')
.replace(/\r/g, '')
const rate = this._getRate()
const script = `
Add-Type -AssemblyName System.Speech
$synth = New-Object System.Speech.Synthesis.SpeechSynthesizer
$synth.Rate = ${rate}
$synth.Speak('${escaped}')
$synth.Dispose()
`
this._currentProcess = spawn('powershell', [
'-NoProfile',
'-NonInteractive',
'-Command',
script,
], { stdio: 'pipe' })
this._currentProcess.on('close', (code) => {
this._currentProcess = null
if (code === 0 || this._cancelled) {
resolve()
} else {
reject(new D3ROError(ErrorCode.ConversationTTSFailed, `TTS exited with code ${code}`))
}
})
this._currentProcess.on('error', (err) => {
this._currentProcess = null
reject(new D3ROError(ErrorCode.ConversationTTSFailed, `TTS error: ${err.message}`))
})
})
}
/**
* SAPI Rate: -10( ) ~ 10( ), 0
*/
private _getRate(): number {
const speed = configGet('ttsSpeed') as number | undefined
if (!speed || speed === 1.0) return 0
// 0.5 → -5, 1.0 → 0, 2.0 → 5
return Math.round((speed - 1.0) * 5)
}
dispose(): void {
this.stop()
this.removeAllListeners()
}
}
// ── 싱글톤 ──
let instance: TTSPlaybackService | null = null
export function getTTSPlaybackService(): TTSPlaybackService {
if (!instance) {
instance = new TTSPlaybackService()
}
return instance
}

View file

@ -0,0 +1,376 @@
// src/main/services/VoiceActionService.ts
// Phase 13.3: OS 자동화 — 음성 → LLM이 JSON 액션 플랜 생성 → 실행
// 사전 정의 명령 + LLM 자유 해석. 위험 액션은 차단.
import { EventEmitter } from 'events'
import { exec } from 'child_process'
import { shell } from 'electron'
import { nanoid } from 'nanoid'
import { getLogger } from './LoggerService'
import { getLocalLLMService } from './LocalLLMService'
import { configGet } from './ConfigService'
import { getMainWindow } from '../windows/WindowManager'
import { IPC_CHANNELS } from '@shared/ipc-channels'
import { D3ROError, ErrorCode } from '@shared/errors'
import type {
VoiceActionPlan,
VoiceActionPreset,
VoiceActionHistoryEntry,
VoiceActionPlannedEvent,
VoiceActionExecutedEvent,
VoiceActionErrorEvent,
} from '@shared/types'
const logger = getLogger('VoiceActionService')
/** 위험 명령어 블랙리스트 (실행 차단) */
const BLOCKED_COMMANDS = [
'rm ', 'del ', 'format ', 'rmdir', 'rd ',
'shutdown', 'restart', 'taskkill',
'reg delete', 'reg add',
'net user', 'net localgroup',
]
/** LLM에 보낼 시스템 프롬프트 */
const ACTION_SYSTEM_PROMPT = `You are an OS automation assistant. Parse the user's voice command and output a JSON action plan.
Output EXACTLY one JSON object (no markdown, no explanation):
{
"action": "open_app" | "open_url" | "open_file" | "keyboard_shortcut" | "type_text" | "system_command",
"target": "<the target value>",
"description": "<brief description of what this does>",
"safe": true | false
}
Rules:
- "open_app": target = app name (e.g., "notepad", "chrome", "code")
- "open_url": target = full URL (e.g., "https://google.com")
- "open_file": target = file path
- "keyboard_shortcut": target = key combo (e.g., "ctrl+c", "alt+tab")
- "type_text": target = text to type
- "system_command": target = shell command
- Set safe=false for destructive operations (delete, format, shutdown, etc.)
If the command is unclear, output: {"action": "type_text", "target": "", "description": "Could not parse command", "safe": true}`
/** 프리셋 명령어 (LLM 없이 바로 실행) */
const PRESETS: VoiceActionPreset[] = [
{
keywords: ['크롬 열어', '크롬', 'chrome', 'open chrome'],
action: { action: 'open_app', target: 'chrome', description: 'Open Chrome browser', safe: true },
},
{
keywords: ['메모장 열어', '메모장', 'notepad', 'open notepad'],
action: { action: 'open_app', target: 'notepad', description: 'Open Notepad', safe: true },
},
{
keywords: ['탐색기 열어', '탐색기', 'explorer', 'open explorer'],
action: { action: 'open_app', target: 'explorer', description: 'Open File Explorer', safe: true },
},
{
keywords: ['볼륨 올려', '소리 올려', 'volume up'],
action: { action: 'keyboard_shortcut', target: 'volumeup', description: 'Volume up', safe: true },
},
{
keywords: ['볼륨 내려', '소리 내려', 'volume down'],
action: { action: 'keyboard_shortcut', target: 'volumedown', description: 'Volume down', safe: true },
},
{
keywords: ['음소거', 'mute'],
action: { action: 'keyboard_shortcut', target: 'volumemute', description: 'Toggle mute', safe: true },
},
]
class VoiceActionService extends EventEmitter {
private _history: VoiceActionHistoryEntry[] = []
private _enabled = false
get isEnabled(): boolean {
return this._enabled
}
setEnabled(enabled: boolean): void {
this._enabled = enabled
}
getPresets(): VoiceActionPreset[] {
return PRESETS
}
getHistory(): VoiceActionHistoryEntry[] {
return [...this._history].reverse()
}
clearHistory(): void {
this._history = []
}
/**
*
*/
async execute(text: string): Promise<void> {
// 라이센스 체크
try {
const { getLicenseService } = await import('./LicenseService')
const { Feature } = await import('@shared/types')
const license = getLicenseService()
const access = license.canUse(Feature.OS_AUTOMATION)
if (!access.allowed) {
license.promptUpgrade(Feature.OS_AUTOMATION, 'tier_required')
throw new D3ROError(ErrorCode.FeatureNotAvailable, 'Pro+ required for OS automation')
}
} catch (err) {
if (err instanceof D3ROError) throw err
}
// 1. 프리셋 매칭
const preset = this._matchPreset(text)
if (preset) {
await this._executePlan(preset.action, text)
return
}
// 2. LLM으로 액션 플랜 생성
try {
const llmService = getLocalLLMService()
const result = await llmService.generate(text, {
systemPrompt: ACTION_SYSTEM_PROMPT,
temperature: 0.1,
})
const plan = this._parsePlan(result.text)
if (!plan) {
this._emitError(text, 'Failed to parse action plan from LLM response')
return
}
// 안전장치
if (!plan.safe) {
const entry: VoiceActionHistoryEntry = {
id: nanoid(),
userText: text,
plan,
executed: false,
timestamp: Date.now(),
}
this._history.push(entry)
this._sendToRenderer(IPC_CHANNELS.VOICE_ACTION.ACTION_PLANNED, { plan, userText: text })
logger.warn(`Voice action blocked (unsafe): ${plan.description}`)
return
}
await this._executePlan(plan, text)
} catch (err) {
this._emitError(text, err instanceof Error ? err.message : String(err))
}
}
private _matchPreset(text: string): VoiceActionPreset | null {
const lower = text.toLowerCase().trim()
for (const preset of PRESETS) {
for (const keyword of preset.keywords) {
if (lower.includes(keyword.toLowerCase())) {
return preset
}
}
}
return null
}
private _parsePlan(llmOutput: string): VoiceActionPlan | null {
try {
// JSON 블록 추출
const jsonMatch = llmOutput.match(/\{[\s\S]*\}/)
if (!jsonMatch) return null
const parsed = JSON.parse(jsonMatch[0]) as Record<string, unknown>
if (!parsed.action || !parsed.target) return null
return {
action: parsed.action as VoiceActionPlan['action'],
target: String(parsed.target),
description: String(parsed.description ?? ''),
safe: parsed.safe !== false,
}
} catch {
return null
}
}
private async _executePlan(plan: VoiceActionPlan, userText: string): Promise<void> {
// 차단된 명령어 체크
if (plan.action === 'system_command') {
const targetLower = plan.target.toLowerCase()
for (const blocked of BLOCKED_COMMANDS) {
if (targetLower.includes(blocked)) {
plan.safe = false
this._sendToRenderer(IPC_CHANNELS.VOICE_ACTION.ACTION_PLANNED, { plan, userText })
logger.warn(`Voice action blocked: ${plan.target}`)
return
}
}
}
this._sendToRenderer(IPC_CHANNELS.VOICE_ACTION.ACTION_PLANNED, { plan, userText } as VoiceActionPlannedEvent)
try {
switch (plan.action) {
case 'open_app':
await this._openApp(plan.target)
break
case 'open_url':
await shell.openExternal(plan.target)
break
case 'open_file':
await shell.openPath(plan.target)
break
case 'keyboard_shortcut':
await this._simulateKeyboard(plan.target)
break
case 'type_text':
// @nut-tree 사용 (lazy import)
await this._typeText(plan.target)
break
case 'system_command':
await this._runCommand(plan.target)
break
}
const entry: VoiceActionHistoryEntry = {
id: nanoid(),
userText,
plan,
executed: true,
timestamp: Date.now(),
}
this._history.push(entry)
if (this._history.length > 50) this._history.shift()
const event: VoiceActionExecutedEvent = { plan, success: true }
this._sendToRenderer(IPC_CHANNELS.VOICE_ACTION.ACTION_EXECUTED, event)
this.emit('action-executed', event)
logger.info(`Voice action executed: ${plan.action}${plan.target}`)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
this._emitError(userText, msg)
throw new D3ROError(ErrorCode.VoiceActionExecutionFailed, msg)
}
}
private _openApp(appName: string): Promise<void> {
return new Promise((resolve, reject) => {
const cmd = `start "" "${appName}"`
exec(cmd, { shell: 'cmd.exe' }, (err) => {
if (err) reject(err)
else resolve()
})
})
}
private _simulateKeyboard(combo: string): Promise<void> {
return new Promise((resolve, reject) => {
// PowerShell SendKeys 사용
const keys = combo.toLowerCase()
let sendKeysStr = ''
if (keys === 'volumeup') {
// nircmd 대안: PowerShell로 볼륨 조절
const script = `
$wshell = New-Object -ComObject WScript.Shell
$wshell.SendKeys([char]175)
`
exec(`powershell -NoProfile -Command "${script}"`, (err) => {
if (err) reject(err)
else resolve()
})
return
}
if (keys === 'volumedown') {
const script = `
$wshell = New-Object -ComObject WScript.Shell
$wshell.SendKeys([char]174)
`
exec(`powershell -NoProfile -Command "${script}"`, (err) => {
if (err) reject(err)
else resolve()
})
return
}
if (keys === 'volumemute') {
const script = `
$wshell = New-Object -ComObject WScript.Shell
$wshell.SendKeys([char]173)
`
exec(`powershell -NoProfile -Command "${script}"`, (err) => {
if (err) reject(err)
else resolve()
})
return
}
// 일반 키보드 단축키 (ctrl+c 등)
if (keys.includes('ctrl')) sendKeysStr += '^'
if (keys.includes('alt')) sendKeysStr += '%'
if (keys.includes('shift')) sendKeysStr += '+'
const key = keys.replace(/ctrl\+|alt\+|shift\+/g, '').trim()
sendKeysStr += key
const script = `
$wshell = New-Object -ComObject WScript.Shell
$wshell.SendKeys('${sendKeysStr}')
`
exec(`powershell -NoProfile -Command "${script}"`, (err) => {
if (err) reject(err)
else resolve()
})
})
}
private async _typeText(text: string): Promise<void> {
try {
const { getTextInsertService } = await import('./TextInsertService')
await getTextInsertService().insertText(text)
} catch (err) {
throw new D3ROError(ErrorCode.VoiceActionExecutionFailed, `Type text failed: ${err instanceof Error ? err.message : String(err)}`)
}
}
private _runCommand(command: string): Promise<void> {
return new Promise((resolve, reject) => {
exec(command, { timeout: 10000 }, (err) => {
if (err) reject(err)
else resolve()
})
})
}
private _emitError(userText: string, message: string): void {
const event: VoiceActionErrorEvent = { message, userText }
this._sendToRenderer(IPC_CHANNELS.VOICE_ACTION.ACTION_ERROR, event)
this.emit('error', event)
}
private _sendToRenderer(channel: string, data: unknown): void {
try {
const mainWindow = getMainWindow()
if (mainWindow && !mainWindow.isDestroyed()) {
mainWindow.webContents.send(channel, data)
}
} catch { /* ignore */ }
}
dispose(): void {
this.removeAllListeners()
}
}
// ── 싱글톤 ──
let instance: VoiceActionService | null = null
export function getVoiceActionService(): VoiceActionService {
if (!instance) {
instance = new VoiceActionService()
}
return instance
}

View file

@ -0,0 +1,376 @@
// src/main/services/VoiceConversationService.ts
// Phase 13.1: 음성 대화 모드 — STT → LLM(chat) → TTS 루프
// 싱글톤 + EventEmitter. 대화 히스토리 최근 10턴 유지.
import { EventEmitter } from 'events'
import { nanoid } from 'nanoid'
import { getLogger } from './LoggerService'
import { getLocalLLMService } from './LocalLLMService'
import { getLocalSTTService } from './LocalSTTService'
import { getAudioCaptureService } from './AudioCaptureService'
import { getTTSPlaybackService } from './TTSPlaybackService'
import { configGet } from './ConfigService'
import { getMainWindow } from '../windows/WindowManager'
import { IPC_CHANNELS } from '@shared/ipc-channels'
import { D3ROError, ErrorCode } from '@shared/errors'
import type {
ConversationState,
ConversationMessage,
ConversationSessionInfo,
ConversationAssistantDelta,
ConversationAssistantMessage,
ConversationError,
} from '@shared/types'
const logger = getLogger('VoiceConversationService')
/** 대화 히스토리 최대 턴 수 (user+assistant 쌍) */
const MAX_HISTORY_TURNS = 10
/** 시스템 프롬프트 */
const SYSTEM_PROMPT = `You are D3RO, a helpful local AI voice assistant. Respond concisely and naturally, as if having a spoken conversation. Keep answers brief (2-3 sentences) unless the user asks for detail. Respond in the same language the user speaks.`
class VoiceConversationService extends EventEmitter {
private _state: ConversationState = 'idle'
private _messages: ConversationMessage[] = []
private _isActive = false
private _audioBuffers: Buffer[] = []
private _audioListenerBound = false
get state(): ConversationState {
return this._state
}
get isActive(): boolean {
return this._isActive
}
getSessionInfo(): ConversationSessionInfo {
return {
state: this._state,
messages: [...this._messages],
isActive: this._isActive,
}
}
getHistory(): ConversationMessage[] {
return [...this._messages]
}
/**
* . listening .
*/
async startSession(): Promise<void> {
if (this._isActive) {
throw new D3ROError(ErrorCode.ConversationSessionAlreadyActive, 'Conversation session already active')
}
// 라이센스 체크
try {
const { getLicenseService } = await import('./LicenseService')
const { Feature } = await import('@shared/types')
const license = getLicenseService()
const access = license.canUse(Feature.VOICE_CONVERSATION)
if (!access.allowed) {
license.promptUpgrade(Feature.VOICE_CONVERSATION, 'tier_required')
throw new D3ROError(ErrorCode.FeatureNotAvailable, 'Pro+ required for voice conversation')
}
} catch (err) {
if (err instanceof D3ROError) throw err
}
this._isActive = true
this._setState('listening')
this._startListening()
logger.info('Voice conversation session started')
}
/**
* .
*/
stopSession(): void {
if (!this._isActive) return
this._stopListening()
getTTSPlaybackService().stop()
getLocalLLMService().cancelGeneration()
this._isActive = false
this._setState('idle')
logger.info('Voice conversation session stopped')
}
/**
* ( ).
*/
async sendTextMessage(text: string): Promise<void> {
if (!this._isActive) {
throw new D3ROError(ErrorCode.ConversationNoActiveSession, 'No active conversation session')
}
await this._processUserMessage(text)
}
/**
* LLM TTS .
*/
cancelResponse(): void {
getLocalLLMService().cancelGeneration()
getTTSPlaybackService().stop()
if (this._isActive) {
this._setState('listening')
this._startListening()
}
}
/**
* .
*/
clearHistory(): void {
this._messages = []
this._sendToRenderer(IPC_CHANNELS.VOICE_CONVERSATION.STATE_CHANGED, this.getSessionInfo())
}
// ── 내부 로직 ──
private _setState(state: ConversationState): void {
this._state = state
this._sendToRenderer(IPC_CHANNELS.VOICE_CONVERSATION.STATE_CHANGED, this.getSessionInfo())
this.emit('state-changed', state)
}
private _startListening(): void {
this._audioBuffers = []
const audioService = getAudioCaptureService()
if (!this._audioListenerBound) {
audioService.on('audio-data', this._onAudioData)
this._audioListenerBound = true
}
audioService.start().catch((err) => {
logger.error('Failed to start audio capture for conversation:', err)
this._emitError('stt', 'Failed to start microphone')
})
}
private _stopListening(): void {
const audioService = getAudioCaptureService()
if (this._audioListenerBound) {
audioService.off('audio-data', this._onAudioData)
this._audioListenerBound = false
}
audioService.stop().catch(() => { /* ignore */ })
this._audioBuffers = []
}
private _onAudioData = (payload: { buffer: Buffer }): void => {
if (this._state !== 'listening') return
this._audioBuffers.push(payload.buffer)
}
/**
* (UI에서 stop ).
* STT로 LLM .
*/
async finishListening(): Promise<void> {
if (this._state !== 'listening' || this._audioBuffers.length === 0) return
this._stopListening()
this._setState('thinking')
const audioBuffer = Buffer.concat(this._audioBuffers)
this._audioBuffers = []
// 최소 오디오 길이 체크 (500ms @ 16kHz 16bit mono)
const minBytes = 16000 * 2 * 0.5
if (audioBuffer.length < minBytes) {
this._setState('listening')
this._startListening()
return
}
try {
// STT
const sttService = getLocalSTTService()
const language = (configGet('sttLanguage') as string | undefined) ?? 'auto'
const result = await sttService.transcribe(audioBuffer, { language, vadFilter: true })
if (!result.text || result.text.trim().length === 0) {
this._setState('listening')
this._startListening()
return
}
await this._processUserMessage(result.text.trim())
} catch (err) {
logger.error('STT failed in conversation:', err)
this._emitError('stt', err instanceof Error ? err.message : 'STT failed')
this._setState('listening')
this._startListening()
}
}
private async _processUserMessage(text: string): Promise<void> {
// 사용자 메시지 추가
const userMsg: ConversationMessage = {
id: nanoid(),
role: 'user',
content: text,
timestamp: Date.now(),
}
this._messages.push(userMsg)
this._trimHistory()
this._sendToRenderer(IPC_CHANNELS.VOICE_CONVERSATION.USER_MESSAGE, userMsg)
this._setState('thinking')
try {
// Ollama /api/chat 호출 (스트리밍)
const llmService = getLocalLLMService()
const chatMessages = this._buildChatMessages()
const assistantMsgId = nanoid()
let accumulated = ''
const ttsSentences: string[] = []
let sentenceBuffer = ''
const generator = llmService.chatStream(chatMessages)
for await (const token of generator) {
accumulated += token
// 렌더러에 델타 전송
const delta: ConversationAssistantDelta = {
messageId: assistantMsgId,
delta: token,
accumulated,
}
this._sendToRenderer(IPC_CHANNELS.VOICE_CONVERSATION.ASSISTANT_DELTA, delta)
// 문장 단위 TTS 큐잉
sentenceBuffer += token
const sentenceEnd = sentenceBuffer.match(/[.!?。!?]\s*/g)
if (sentenceEnd) {
const lastEnd = sentenceBuffer.lastIndexOf(sentenceEnd[sentenceEnd.length - 1])
const completeSentence = sentenceBuffer.slice(
0,
lastEnd + sentenceEnd[sentenceEnd.length - 1].length,
)
sentenceBuffer = sentenceBuffer.slice(
lastEnd + sentenceEnd[sentenceEnd.length - 1].length,
)
if (completeSentence.trim()) {
ttsSentences.push(completeSentence.trim())
}
}
}
// 남은 텍스트도 TTS 큐에 추가
if (sentenceBuffer.trim()) {
ttsSentences.push(sentenceBuffer.trim())
}
// 어시스턴트 메시지 저장
const assistantMsg: ConversationMessage = {
id: assistantMsgId,
role: 'assistant',
content: accumulated,
timestamp: Date.now(),
}
this._messages.push(assistantMsg)
this._trimHistory()
const completeEvent: ConversationAssistantMessage = {
messageId: assistantMsgId,
content: accumulated,
}
this._sendToRenderer(IPC_CHANNELS.VOICE_CONVERSATION.ASSISTANT_MESSAGE, completeEvent)
// TTS 재생
if (ttsSentences.length > 0) {
this._setState('speaking')
this._sendToRenderer(IPC_CHANNELS.VOICE_CONVERSATION.TTS_STARTED, {})
const ttsService = getTTSPlaybackService()
await ttsService.speakSentences(ttsSentences)
this._sendToRenderer(IPC_CHANNELS.VOICE_CONVERSATION.TTS_FINISHED, {})
}
// 재생 완료 → 다시 listening
if (this._isActive) {
this._setState('listening')
this._startListening()
}
} catch (err) {
logger.error('LLM chat failed in conversation:', err)
this._emitError('llm', err instanceof Error ? err.message : 'LLM failed')
if (this._isActive) {
this._setState('listening')
this._startListening()
}
}
}
private _buildChatMessages(): Array<{ role: string; content: string }> {
const chatMsgs: Array<{ role: string; content: string }> = [
{ role: 'system', content: SYSTEM_PROMPT },
]
for (const msg of this._messages) {
if (msg.role === 'user' || msg.role === 'assistant') {
chatMsgs.push({ role: msg.role, content: msg.content })
}
}
return chatMsgs
}
private _trimHistory(): void {
// user+assistant 쌍 기준으로 최근 MAX_HISTORY_TURNS개만 유지
const pairs: ConversationMessage[] = []
let turnCount = 0
for (let i = this._messages.length - 1; i >= 0; i--) {
pairs.unshift(this._messages[i])
if (this._messages[i].role === 'user') {
turnCount++
if (turnCount >= MAX_HISTORY_TURNS) break
}
}
this._messages = pairs
}
private _emitError(phase: ConversationError['phase'], message: string): void {
const error: ConversationError = { message, phase }
this._sendToRenderer(IPC_CHANNELS.VOICE_CONVERSATION.ERROR, error)
this.emit('error', error)
}
private _sendToRenderer(channel: string, data: unknown): void {
try {
const mainWindow = getMainWindow()
if (mainWindow && !mainWindow.isDestroyed()) {
mainWindow.webContents.send(channel, data)
}
} catch {
// 윈도우 없으면 무시
}
}
dispose(): void {
this.stopSession()
this.removeAllListeners()
}
}
// ── 싱글톤 ──
let instance: VoiceConversationService | null = null
export function getVoiceConversationService(): VoiceConversationService {
if (!instance) {
instance = new VoiceConversationService()
}
return instance
}

View file

@ -90,6 +90,29 @@ export function getSoundPath(filename: string): string {
return path.join(app.getAppPath(), 'resources', 'sounds', filename) return path.join(app.getAppPath(), 'resources', 'sounds', filename)
} }
/**
* ffmpeg .
* - dev: @ffmpeg-installer/ffmpeg의 node_modules
* - production: extraResources로
*/
export function getFfmpegPath(): string {
if (isPackaged()) {
const bundled = path.join(process.resourcesPath, 'ffmpeg', 'ffmpeg.exe')
if (existsSync(bundled)) {
return bundled
}
}
// dev: @ffmpeg-installer/ffmpeg에서 제공하는 경로
try {
// eslint-disable-next-line @typescript-eslint/no-require-imports
const installer = require('@ffmpeg-installer/ffmpeg')
return installer.path as string
} catch {
return 'ffmpeg'
}
}
/** /**
* (DB, ). * (DB, ).
*/ */

View file

@ -95,6 +95,47 @@ import type {
UsageQuota, UsageQuota,
UpgradePromptEvent, UpgradePromptEvent,
TierComparison, TierComparison,
// Phase 12
FileTranscriptionStartParams,
// Phase 13.1
ConversationSessionInfo,
ConversationMessage,
ConversationSendParams,
ConversationAssistantDelta,
ConversationAssistantMessage,
ConversationError,
// Phase 13.2
RAGDocument,
RAGQueryParams,
RAGQueryResult,
RAGStateInfo,
RAGIndexProgress,
RAGAddDocumentParams,
RAGRemoveDocumentParams,
// Phase 13.3
VoiceActionExecuteParams,
VoiceActionPreset,
VoiceActionHistoryEntry,
VoiceActionPlannedEvent,
VoiceActionExecutedEvent,
VoiceActionErrorEvent,
FileTranscriptionProgress,
FileTranscriptionResult,
FileTranscriptionStateInfo,
MeetingSummarizeParams,
MeetingSummaryResult,
MeetingSummaryGetParams,
MeetingSummaryExportParams,
MeetingSummaryProgress,
DictationTemplate,
CreateTemplateParams,
UpdateTemplateParams,
DeleteTemplateParams,
StartTemplateSessionParams,
SetFieldValueParams,
TemplateSessionInfo,
TemplateFieldCompletedEvent,
TemplateSessionCompletedEvent,
} from '@shared/types' } from '@shared/types'
import { Feature } from '@shared/types' import { Feature } from '@shared/types'
import type { IPCResult } from '@shared/errors' import type { IPCResult } from '@shared/errors'
@ -382,6 +423,136 @@ const electronAPI = {
onTierChanged: (cb: (e: LicenseInfo) => void): Unsubscribe => onTierChanged: (cb: (e: LicenseInfo) => void): Unsubscribe =>
on(IPC_CHANNELS.LICENSE.TIER_CHANGED, cb), on(IPC_CHANNELS.LICENSE.TIER_CHANGED, cb),
}, },
// ── File Transcription (Phase 12.1) ────────────────────
fileTranscription: {
start: (params: FileTranscriptionStartParams) =>
invoke<FileTranscriptionResult>(IPC_CHANNELS.FILE_TRANSCRIPTION.START, params),
cancel: () =>
invoke<void>(IPC_CHANNELS.FILE_TRANSCRIPTION.CANCEL),
getState: () =>
invoke<FileTranscriptionStateInfo>(IPC_CHANNELS.FILE_TRANSCRIPTION.GET_STATE),
onProgress: (cb: (data: FileTranscriptionProgress) => void): Unsubscribe =>
on(IPC_CHANNELS.FILE_TRANSCRIPTION.PROGRESS, cb),
onComplete: (cb: (data: FileTranscriptionResult) => void): Unsubscribe =>
on(IPC_CHANNELS.FILE_TRANSCRIPTION.COMPLETE, cb),
onError: (cb: (data: { message: string }) => void): Unsubscribe =>
on(IPC_CHANNELS.FILE_TRANSCRIPTION.ERROR, cb),
},
// ── Meeting Summary (Phase 12.2) ─────────────────────
meetingSummary: {
summarize: (params: MeetingSummarizeParams) =>
invoke<MeetingSummaryResult>(IPC_CHANNELS.MEETING_SUMMARY.SUMMARIZE, params),
getSummary: (params: MeetingSummaryGetParams) =>
invoke<MeetingSummaryResult | null>(IPC_CHANNELS.MEETING_SUMMARY.GET_SUMMARY, params),
exportMarkdown: (params: MeetingSummaryExportParams) =>
invoke<string>(IPC_CHANNELS.MEETING_SUMMARY.EXPORT_MARKDOWN, params),
onSummaryReady: (cb: (data: MeetingSummaryResult) => void): Unsubscribe =>
on(IPC_CHANNELS.MEETING_SUMMARY.SUMMARY_READY, cb),
onProgress: (cb: (data: MeetingSummaryProgress) => void): Unsubscribe =>
on(IPC_CHANNELS.MEETING_SUMMARY.SUMMARY_PROGRESS, cb),
},
// ── Dictation Templates (Phase 12.3) ─────────────────
dictationTemplate: {
getAll: () =>
invoke<DictationTemplate[]>(IPC_CHANNELS.DICTATION_TEMPLATE.GET_ALL),
create: (params: CreateTemplateParams) =>
invoke<DictationTemplate>(IPC_CHANNELS.DICTATION_TEMPLATE.CREATE, params),
update: (params: UpdateTemplateParams) =>
invoke<DictationTemplate>(IPC_CHANNELS.DICTATION_TEMPLATE.UPDATE, params),
delete: (params: DeleteTemplateParams) =>
invoke<void>(IPC_CHANNELS.DICTATION_TEMPLATE.DELETE, params),
startSession: (params: StartTemplateSessionParams) =>
invoke<void>(IPC_CHANNELS.DICTATION_TEMPLATE.START_SESSION, params),
cancelSession: () =>
invoke<void>(IPC_CHANNELS.DICTATION_TEMPLATE.CANCEL_SESSION),
getSessionState: () =>
invoke<TemplateSessionInfo | null>(IPC_CHANNELS.DICTATION_TEMPLATE.GET_SESSION_STATE),
setFieldValue: (params: SetFieldValueParams) =>
invoke<void>(IPC_CHANNELS.DICTATION_TEMPLATE.SET_FIELD_VALUE, params),
onSessionStateChanged: (cb: (data: TemplateSessionInfo) => void): Unsubscribe =>
on(IPC_CHANNELS.DICTATION_TEMPLATE.SESSION_STATE_CHANGED, cb),
onFieldCompleted: (cb: (data: TemplateFieldCompletedEvent) => void): Unsubscribe =>
on(IPC_CHANNELS.DICTATION_TEMPLATE.FIELD_COMPLETED, cb),
onSessionCompleted: (cb: (data: TemplateSessionCompletedEvent) => void): Unsubscribe =>
on(IPC_CHANNELS.DICTATION_TEMPLATE.SESSION_COMPLETED, cb),
},
// ── RAG (Phase 13.2) ──────────────────────────────────
rag: {
addDocument: (params?: RAGAddDocumentParams) =>
invoke<RAGDocument>(IPC_CHANNELS.RAG.ADD_DOCUMENT, params ?? {}),
removeDocument: (params: RAGRemoveDocumentParams) =>
invoke<void>(IPC_CHANNELS.RAG.REMOVE_DOCUMENT, params),
getDocuments: () =>
invoke<RAGDocument[]>(IPC_CHANNELS.RAG.GET_DOCUMENTS),
query: (params: RAGQueryParams) =>
invoke<RAGQueryResult>(IPC_CHANNELS.RAG.QUERY, params),
getState: () =>
invoke<RAGStateInfo>(IPC_CHANNELS.RAG.GET_STATE),
reindex: (documentId: string) =>
invoke<void>(IPC_CHANNELS.RAG.REINDEX, { documentId }),
onIndexProgress: (cb: (data: RAGIndexProgress) => void): Unsubscribe =>
on(IPC_CHANNELS.RAG.INDEX_PROGRESS, cb),
onIndexComplete: (cb: (data: { documentId: string; fileName: string }) => void): Unsubscribe =>
on(IPC_CHANNELS.RAG.INDEX_COMPLETE, cb),
},
// ── Voice Action (Phase 13.3) ────────────────────────
voiceAction: {
execute: (params: VoiceActionExecuteParams) =>
invoke<void>(IPC_CHANNELS.VOICE_ACTION.EXECUTE, params),
getPresets: () =>
invoke<VoiceActionPreset[]>(IPC_CHANNELS.VOICE_ACTION.GET_PRESETS),
getHistory: () =>
invoke<VoiceActionHistoryEntry[]>(IPC_CHANNELS.VOICE_ACTION.GET_HISTORY),
clearHistory: () =>
invoke<void>(IPC_CHANNELS.VOICE_ACTION.CLEAR_HISTORY),
setEnabled: (enabled: boolean) =>
invoke<void>(IPC_CHANNELS.VOICE_ACTION.SET_ENABLED, { enabled }),
isEnabled: () =>
invoke<boolean>(IPC_CHANNELS.VOICE_ACTION.IS_ENABLED),
onActionPlanned: (cb: (data: VoiceActionPlannedEvent) => void): Unsubscribe =>
on(IPC_CHANNELS.VOICE_ACTION.ACTION_PLANNED, cb),
onActionExecuted: (cb: (data: VoiceActionExecutedEvent) => void): Unsubscribe =>
on(IPC_CHANNELS.VOICE_ACTION.ACTION_EXECUTED, cb),
onActionError: (cb: (data: VoiceActionErrorEvent) => void): Unsubscribe =>
on(IPC_CHANNELS.VOICE_ACTION.ACTION_ERROR, cb),
},
// ── Voice Conversation (Phase 13.1) ───────────────────
voiceConversation: {
startSession: () =>
invoke<void>(IPC_CHANNELS.VOICE_CONVERSATION.START_SESSION),
stopSession: () =>
invoke<void>(IPC_CHANNELS.VOICE_CONVERSATION.STOP_SESSION),
sendMessage: (params: ConversationSendParams) =>
invoke<void>(IPC_CHANNELS.VOICE_CONVERSATION.SEND_MESSAGE, params),
getState: () =>
invoke<ConversationSessionInfo>(IPC_CHANNELS.VOICE_CONVERSATION.GET_STATE),
getHistory: () =>
invoke<ConversationMessage[]>(IPC_CHANNELS.VOICE_CONVERSATION.GET_HISTORY),
clearHistory: () =>
invoke<void>(IPC_CHANNELS.VOICE_CONVERSATION.CLEAR_HISTORY),
cancelResponse: () =>
invoke<void>(IPC_CHANNELS.VOICE_CONVERSATION.CANCEL_RESPONSE),
finishListening: () =>
invoke<void>('voiceConversation:finishListening'),
onStateChanged: (cb: (data: ConversationSessionInfo) => void): Unsubscribe =>
on(IPC_CHANNELS.VOICE_CONVERSATION.STATE_CHANGED, cb),
onUserMessage: (cb: (data: ConversationMessage) => void): Unsubscribe =>
on(IPC_CHANNELS.VOICE_CONVERSATION.USER_MESSAGE, cb),
onAssistantDelta: (cb: (data: ConversationAssistantDelta) => void): Unsubscribe =>
on(IPC_CHANNELS.VOICE_CONVERSATION.ASSISTANT_DELTA, cb),
onAssistantMessage: (cb: (data: ConversationAssistantMessage) => void): Unsubscribe =>
on(IPC_CHANNELS.VOICE_CONVERSATION.ASSISTANT_MESSAGE, cb),
onTTSStarted: (cb: () => void): Unsubscribe =>
on(IPC_CHANNELS.VOICE_CONVERSATION.TTS_STARTED, cb),
onTTSFinished: (cb: () => void): Unsubscribe =>
on(IPC_CHANNELS.VOICE_CONVERSATION.TTS_FINISHED, cb),
onError: (cb: (data: ConversationError) => void): Unsubscribe =>
on(IPC_CHANNELS.VOICE_CONVERSATION.ERROR, cb),
},
} as const } as const
contextBridge.exposeInMainWorld('electronAPI', electronAPI) contextBridge.exposeInMainWorld('electronAPI', electronAPI)

View file

@ -7,12 +7,16 @@ import DashboardIcon from '@mui/icons-material/Dashboard'
import HistoryIcon from '@mui/icons-material/History' import HistoryIcon from '@mui/icons-material/History'
import MenuBookIcon from '@mui/icons-material/MenuBook' import MenuBookIcon from '@mui/icons-material/MenuBook'
import ExtensionIcon from '@mui/icons-material/Extension' import ExtensionIcon from '@mui/icons-material/Extension'
import RecordVoiceOverIcon from '@mui/icons-material/RecordVoiceOver'
import AutoStoriesIcon from '@mui/icons-material/AutoStories'
import SettingsIcon from '@mui/icons-material/Settings' import SettingsIcon from '@mui/icons-material/Settings'
import { Led } from './ds' import { Led } from './ds'
import { DashboardPage } from '../pages/DashboardPage' import { DashboardPage } from '../pages/DashboardPage'
import { HistoryPage } from '../pages/HistoryPage' import { HistoryPage } from '../pages/HistoryPage'
import { DictionaryPage } from '../pages/DictionaryPage' import { DictionaryPage } from '../pages/DictionaryPage'
import { CommandsPage } from '../pages/CommandsPage' import { CommandsPage } from '../pages/CommandsPage'
import { VoiceConversationPage } from '../pages/VoiceConversationPage'
import { KnowledgeBasePage } from '../pages/KnowledgeBasePage'
import { SettingsModal } from './SettingsModal' import { SettingsModal } from './SettingsModal'
import { LicenseModal } from './LicenseModal' import { LicenseModal } from './LicenseModal'
import { OnboardingModal } from './OnboardingModal' import { OnboardingModal } from './OnboardingModal'
@ -22,7 +26,7 @@ import { useI18n } from '../i18n'
import type { TranslationKey } from '../i18n' import type { TranslationKey } from '../i18n'
import type { LicenseTier } from '@shared/types' import type { LicenseTier } from '@shared/types'
type Route = 'dashboard' | 'history' | 'dictionary' | 'commands' type Route = 'dashboard' | 'history' | 'dictionary' | 'commands' | 'conversation' | 'knowledge'
interface NavItem { interface NavItem {
route: Route route: Route
@ -36,6 +40,8 @@ const NAV_ITEMS: NavItem[] = [
{ route: 'history', labelKey: 'nav.history', abbr: 'HIST', icon: <HistoryIcon sx={{ fontSize: 20 }} /> }, { route: 'history', labelKey: 'nav.history', abbr: 'HIST', icon: <HistoryIcon sx={{ fontSize: 20 }} /> },
{ route: 'dictionary', labelKey: 'nav.dictionary', abbr: 'DICT', icon: <MenuBookIcon sx={{ fontSize: 20 }} /> }, { route: 'dictionary', labelKey: 'nav.dictionary', abbr: 'DICT', icon: <MenuBookIcon sx={{ fontSize: 20 }} /> },
{ route: 'commands', labelKey: 'nav.commands', abbr: 'CMD', icon: <ExtensionIcon sx={{ fontSize: 20 }} /> }, { route: 'commands', labelKey: 'nav.commands', abbr: 'CMD', icon: <ExtensionIcon sx={{ fontSize: 20 }} /> },
{ route: 'conversation', labelKey: 'nav.conversation', abbr: 'TALK', icon: <RecordVoiceOverIcon sx={{ fontSize: 20 }} /> },
{ route: 'knowledge', labelKey: 'nav.knowledge', abbr: 'RAG', icon: <AutoStoriesIcon sx={{ fontSize: 20 }} /> },
] ]
function tierToLedColor(tier: LicenseTier): 'amber' | 'green' { function tierToLedColor(tier: LicenseTier): 'amber' | 'green' {
@ -217,6 +223,8 @@ export function AppLayout(): React.ReactElement {
{currentRoute === 'history' && <HistoryPage />} {currentRoute === 'history' && <HistoryPage />}
{currentRoute === 'dictionary' && <DictionaryPage />} {currentRoute === 'dictionary' && <DictionaryPage />}
{currentRoute === 'commands' && <CommandsPage />} {currentRoute === 'commands' && <CommandsPage />}
{currentRoute === 'conversation' && <VoiceConversationPage />}
{currentRoute === 'knowledge' && <KnowledgeBasePage />}
</Box> </Box>
</Box> </Box>
<StatusBar /> <StatusBar />

View file

@ -0,0 +1,272 @@
// src/renderer/components/FileDropZone.tsx
// Phase 12.1: 파일 전사 드래그앤드롭 UI
import { useState, useEffect, useCallback, useRef } from 'react'
import { Box, LinearProgress, IconButton, Tooltip } from '@mui/material'
import ContentCopyIcon from '@mui/icons-material/ContentCopy'
import CloseIcon from '@mui/icons-material/Close'
import UploadFileIcon from '@mui/icons-material/UploadFile'
import { MetalCard, PhosphorText, Led } from './ds'
import { d3roPalette, d3roTypo } from '../theme'
import { useI18n } from '../i18n'
import type {
FileTranscriptionProgress,
FileTranscriptionResult,
FileTranscriptionState,
} from '@shared/types'
const SUPPORTED_EXTENSIONS = [
'.mp3', '.wav', '.m4a', '.ogg', '.flac', '.wma', '.aac',
'.mp4', '.mkv', '.webm', '.avi', '.mov',
]
export function FileDropZone(): React.ReactElement {
const { t } = useI18n()
const [dragging, setDragging] = useState(false)
const [state, setState] = useState<FileTranscriptionState>('idle')
const [progress, setProgress] = useState<FileTranscriptionProgress | null>(null)
const [result, setResult] = useState<FileTranscriptionResult | null>(null)
const [error, setError] = useState<string | null>(null)
const [copied, setCopied] = useState(false)
const dropRef = useRef<HTMLDivElement>(null)
useEffect(() => {
const unsubProgress = window.electronAPI.fileTranscription.onProgress((data) => {
setProgress(data)
setState('transcribing')
})
const unsubComplete = window.electronAPI.fileTranscription.onComplete((data) => {
setResult(data)
setState('completed')
setProgress(null)
})
const unsubError = window.electronAPI.fileTranscription.onError((data) => {
setError(data.message)
setState('error')
setProgress(null)
})
return () => {
unsubProgress()
unsubComplete()
unsubError()
}
}, [])
const handleDrop = useCallback(async (e: React.DragEvent) => {
e.preventDefault()
setDragging(false)
const file = e.dataTransfer.files[0]
if (!file) return
const ext = '.' + file.name.split('.').pop()?.toLowerCase()
if (!SUPPORTED_EXTENSIONS.includes(ext)) {
setError(t('fileTranscription.error.invalidFormat'))
setState('error')
return
}
setState('converting')
setError(null)
setResult(null)
const filePath = (file as unknown as { path: string }).path
const resp = await window.electronAPI.fileTranscription.start({ filePath })
if (!resp.success) {
setError(resp.error.message)
setState('error')
}
}, [t])
const handleBrowse = useCallback(async () => {
setState('converting')
setError(null)
setResult(null)
const resp = await window.electronAPI.fileTranscription.start({ filePath: '' })
if (!resp.success) {
if (resp.error.message.includes('cancelled')) {
setState('idle')
} else {
setError(resp.error.message)
setState('error')
}
}
}, [])
const handleCancel = useCallback(async () => {
await window.electronAPI.fileTranscription.cancel()
setState('idle')
setProgress(null)
}, [])
const handleCopy = useCallback(() => {
if (result?.fullText) {
navigator.clipboard.writeText(result.fullText)
setCopied(true)
setTimeout(() => setCopied(false), 2000)
}
}, [result])
const handleReset = useCallback(() => {
setState('idle')
setResult(null)
setError(null)
setProgress(null)
}, [])
// ── idle: 드래그 존 ──
if (state === 'idle') {
return (
<MetalCard>
<Box
ref={dropRef}
onDragOver={(e) => { e.preventDefault(); setDragging(true) }}
onDragLeave={() => setDragging(false)}
onDrop={handleDrop}
onClick={handleBrowse}
sx={{
p: 4,
textAlign: 'center',
border: `2px dashed ${dragging ? d3roPalette.accent.amber : d3roPalette.border.subtle}`,
borderRadius: '8px',
cursor: 'pointer',
transition: 'border-color 0.2s',
'&:hover': { borderColor: d3roPalette.accent.amber },
}}
>
<UploadFileIcon sx={{ fontSize: 40, color: d3roPalette.text.inactive, mb: 1 }} />
<PhosphorText variant="body" sx={{ color: d3roPalette.text.secondary }}>
{t('fileTranscription.dropZone')}
</PhosphorText>
<PhosphorText variant="dim" sx={{ mt: 0.5 }}>
{t('fileTranscription.dropZoneHint')}
</PhosphorText>
</Box>
</MetalCard>
)
}
// ── converting / transcribing: 진행률 ──
if (state === 'converting' || state === 'transcribing') {
return (
<MetalCard>
<Box sx={{ p: 3 }}>
<Box sx={{ display: 'flex', alignItems: 'center', justifyContent: 'space-between', mb: 2 }}>
<Box sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
<Led color="amber" pulse />
<PhosphorText variant="body">
{state === 'converting'
? t('fileTranscription.converting')
: t('fileTranscription.processing')}
</PhosphorText>
</Box>
<PhysicalButton size="small" onClick={handleCancel}>
{t('fileTranscription.cancel')}
</PhysicalButton>
</Box>
{progress && (
<>
<LinearProgress
variant="determinate"
value={progress.percent}
sx={{
mb: 1,
height: 6,
borderRadius: 3,
bgcolor: d3roPalette.bg.inset,
'& .MuiLinearProgress-bar': { bgcolor: d3roPalette.accent.amber },
}}
/>
<PhosphorText variant="dim">
{t('fileTranscription.progress', {
current: String(progress.currentChunk),
total: String(progress.totalChunks),
})}
</PhosphorText>
{progress.currentText && (
<PhosphorText variant="compact" sx={{ mt: 1, opacity: 0.7, fontStyle: 'italic' }}>
{progress.currentText.slice(0, 100)}...
</PhosphorText>
)}
</>
)}
</Box>
</MetalCard>
)
}
// ── completed: 결과 ──
if (state === 'completed' && result) {
return (
<MetalCard>
<Box sx={{ p: 3 }}>
<Box sx={{ display: 'flex', alignItems: 'center', justifyContent: 'space-between', mb: 2 }}>
<Box sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
<Led color="green" />
<PhosphorText variant="body">
{t('fileTranscription.complete')}
</PhosphorText>
<PhosphorText variant="dim">
({result.fileName})
</PhosphorText>
</Box>
<Box sx={{ display: 'flex', gap: 0.5 }}>
<Tooltip title={copied ? 'Copied!' : t('fileTranscription.copyAll')}>
<IconButton size="small" onClick={handleCopy}>
<ContentCopyIcon sx={{ fontSize: 16, color: copied ? d3roPalette.accent.amber : d3roPalette.text.inactive }} />
</IconButton>
</Tooltip>
<IconButton size="small" onClick={handleReset}>
<CloseIcon sx={{ fontSize: 16, color: d3roPalette.text.inactive }} />
</IconButton>
</Box>
</Box>
<Box
sx={{
maxHeight: 200,
overflow: 'auto',
p: 2,
bgcolor: d3roPalette.bg.inset,
borderRadius: '6px',
fontSize: d3roTypo.compact.size,
lineHeight: d3roTypo.compact.line,
color: d3roPalette.text.primary,
whiteSpace: 'pre-wrap',
}}
>
{result.fullText}
</Box>
<PhosphorText variant="dim" sx={{ mt: 1 }}>
{Math.round(result.totalDurationSec)}s audio / {Math.round(result.processingTimeMs / 1000)}s processing
</PhosphorText>
</Box>
</MetalCard>
)
}
// ── error ──
if (state === 'error') {
return (
<MetalCard>
<Box sx={{ p: 3 }}>
<Box sx={{ display: 'flex', alignItems: 'center', gap: 1, mb: 1 }}>
<Led color="red" />
<PhosphorText variant="body" sx={{ color: d3roPalette.tag.red }}>
{error ?? t('fileTranscription.error.unknown')}
</PhosphorText>
</Box>
<PhysicalButton size="small" onClick={handleReset}>
{t('fileTranscription.retry')}
</PhysicalButton>
</Box>
</MetalCard>
)
}
return <></>
}

View file

@ -263,7 +263,7 @@ export function LicenseTab(): React.ReactElement {
<tbody> <tbody>
{comparison.map((row) => ( {comparison.map((row) => (
<tr key={row.feature}> <tr key={row.feature}>
<td>{t(`license.feature.${row.feature}`)}</td> <td>{t(row.featureLabel as Parameters<typeof t>[0])}</td>
<td><TierCell value={row.free} /></td> <td><TierCell value={row.free} /></td>
<td><TierCell value={row.pro} /></td> <td><TierCell value={row.pro} /></td>
<td><TierCell value={row.proPlus} /></td> <td><TierCell value={row.proPlus} /></td>

View file

@ -0,0 +1,294 @@
// src/renderer/components/TemplateSection.tsx
// Phase 12.3: 딕테이션 템플릿 관리 UI (CommandsPage 내 섹션)
import { useState, useEffect, useCallback } from 'react'
import { Box, Button, IconButton, Dialog, DialogTitle, DialogContent, DialogActions, TextField, Tooltip } from '@mui/material'
import AddIcon from '@mui/icons-material/Add'
import DeleteIcon from '@mui/icons-material/Delete'
import EditIcon from '@mui/icons-material/Edit'
import PlayArrowIcon from '@mui/icons-material/PlayArrow'
import { MetalCard, PhosphorText, Led, PhysicalButton } from './ds'
import { PageHeader, EmptyStateCard } from './shared'
import { d3roPalette, d3roFontMono, d3roTypo, d3roRadius } from '../theme'
import { useI18n } from '../i18n'
import type { DictationTemplate, TemplateField, TemplateSessionInfo } from '@shared/types'
export function TemplateSection(): React.ReactElement {
const { t } = useI18n()
const [templates, setTemplates] = useState<DictationTemplate[]>([])
const [loading, setLoading] = useState(true)
const [dialogOpen, setDialogOpen] = useState(false)
const [editId, setEditId] = useState<string | null>(null)
const [formName, setFormName] = useState('')
const [formDesc, setFormDesc] = useState('')
const [formOutput, setFormOutput] = useState('')
const [formFields, setFormFields] = useState<TemplateField[]>([])
const [session, setSession] = useState<TemplateSessionInfo | null>(null)
const loadData = useCallback(async () => {
setLoading(true)
const result = await window.electronAPI.dictationTemplate.getAll()
if (result.success) setTemplates(result.data)
const sessionResult = await window.electronAPI.dictationTemplate.getSessionState()
if (sessionResult.success) setSession(sessionResult.data)
setLoading(false)
}, [])
useEffect(() => { loadData() }, [loadData])
useEffect(() => {
const unsub = window.electronAPI.dictationTemplate.onSessionStateChanged((data) => {
setSession(data)
})
const unsubComplete = window.electronAPI.dictationTemplate.onSessionCompleted(() => {
setSession(null)
loadData()
})
return () => { unsub(); unsubComplete() }
}, [loadData])
const openCreate = () => {
setEditId(null)
setFormName('')
setFormDesc('')
setFormOutput('{{field1}}')
setFormFields([{ id: 'field1', name: 'field1', label: 'Field 1', promptText: '', required: true, maxDurationSec: 30 }])
setDialogOpen(true)
}
const openEdit = (template: DictationTemplate) => {
setEditId(template.id)
setFormName(template.name)
setFormDesc(template.description)
setFormOutput(template.outputFormat)
setFormFields([...template.fields])
setDialogOpen(true)
}
const handleSave = async () => {
if (editId) {
await window.electronAPI.dictationTemplate.update({
id: editId,
name: formName.trim(),
description: formDesc.trim(),
fields: formFields,
outputFormat: formOutput,
})
} else {
await window.electronAPI.dictationTemplate.create({
name: formName.trim(),
description: formDesc.trim(),
fields: formFields,
outputFormat: formOutput,
})
}
setDialogOpen(false)
loadData()
}
const handleDelete = async (id: string) => {
await window.electronAPI.dictationTemplate.delete({ id })
loadData()
}
const handleStartSession = async (templateId: string) => {
await window.electronAPI.dictationTemplate.startSession({ templateId })
}
const handleCancelSession = async () => {
await window.electronAPI.dictationTemplate.cancelSession()
setSession(null)
}
const addField = () => {
const idx = formFields.length + 1
setFormFields([...formFields, {
id: `field${idx}`,
name: `field${idx}`,
label: `Field ${idx}`,
promptText: '',
required: true,
maxDurationSec: 30,
}])
}
const updateField = (index: number, updates: Partial<TemplateField>) => {
const updated = [...formFields]
updated[index] = { ...updated[index], ...updates }
setFormFields(updated)
}
const removeField = (index: number) => {
setFormFields(formFields.filter((_, i) => i !== index))
}
return (
<Box sx={{ mt: 5 }}>
<PageHeader
title={t('template.title').toUpperCase()}
action={
<PhysicalButton size="small" onClick={openCreate}>
<AddIcon sx={{ fontSize: 14, mr: 0.5 }} /> {t('template.create')}
</PhysicalButton>
}
/>
{/* 활성 세션 표시 */}
{session && (
<MetalCard sx={{ mb: 2, border: `1px solid ${d3roPalette.accent.amber}` }}>
<Box sx={{ display: 'flex', alignItems: 'center', justifyContent: 'space-between' }}>
<Box sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
<Led color="amber" pulse />
<PhosphorText variant="body">
{session.templateName}: {session.currentField?.label ?? '...'}
</PhosphorText>
<PhosphorText variant="dim">
({session.currentFieldIndex + 1}/{session.totalFields})
</PhosphorText>
</Box>
<PhysicalButton size="small" onClick={handleCancelSession}>
{t('template.cancelSession')}
</PhysicalButton>
</Box>
</MetalCard>
)}
{/* 템플릿 목록 */}
{templates.length === 0 && !loading ? (
<EmptyStateCard message={t('template.empty')} />
) : (
<Box sx={{ display: 'flex', flexDirection: 'column', gap: 1 }}>
{templates.map((tmpl) => (
<MetalCard key={tmpl.id}>
<Box sx={{ display: 'flex', alignItems: 'center', gap: 2 }}>
<Box sx={{ flex: 1 }}>
<PhosphorText variant="body" sx={{ fontWeight: d3roTypo.body.weight }}>
{tmpl.name}
{tmpl.isBuiltin && (
<Box component="span" sx={{ ml: 1, fontSize: d3roTypo.micro.size, color: d3roPalette.text.dimLabel }}>
PRESET
</Box>
)}
</PhosphorText>
<PhosphorText variant="dim" sx={{ mt: 0.25 }}>
{tmpl.fields.length} {t('template.fields').toLowerCase()} {tmpl.description}
</PhosphorText>
</Box>
<Box sx={{ display: 'flex', gap: 0.5 }}>
<Tooltip title={t('template.startSession')}>
<IconButton
size="small"
onClick={() => handleStartSession(tmpl.id)}
disabled={!!session}
sx={{ color: d3roPalette.accent.amber, '&:hover': { opacity: 0.8 } }}
>
<PlayArrowIcon sx={{ fontSize: 18 }} />
</IconButton>
</Tooltip>
<IconButton
size="small"
onClick={() => openEdit(tmpl)}
sx={{ color: d3roPalette.text.inactive, '&:hover': { color: d3roPalette.accent.amber } }}
>
<EditIcon sx={{ fontSize: 16 }} />
</IconButton>
{!tmpl.isBuiltin && (
<IconButton
size="small"
onClick={() => handleDelete(tmpl.id)}
sx={{ color: d3roPalette.text.inactive, '&:hover': { color: d3roPalette.tag.red } }}
>
<DeleteIcon sx={{ fontSize: 16 }} />
</IconButton>
)}
</Box>
</Box>
</MetalCard>
))}
</Box>
)}
{/* 편집 다이얼로그 */}
<Dialog open={dialogOpen} onClose={() => setDialogOpen(false)} maxWidth="sm" fullWidth>
<DialogTitle>{editId ? t('template.edit') : t('template.create')}</DialogTitle>
<DialogContent sx={{ display: 'flex', flexDirection: 'column', gap: 2, pt: '8px !important' }}>
<TextField
label={t('template.name')}
value={formName}
onChange={(e) => setFormName(e.target.value)}
size="small"
fullWidth
/>
<TextField
label={t('template.description')}
value={formDesc}
onChange={(e) => setFormDesc(e.target.value)}
size="small"
fullWidth
/>
<PhosphorText variant="label" sx={{ mt: 1, color: d3roPalette.text.dimLabel }}>
{t('template.fields').toUpperCase()}
</PhosphorText>
{formFields.map((field, idx) => (
<Box key={idx} sx={{ display: 'flex', gap: 1, alignItems: 'center' }}>
<TextField
label={t('template.fieldName')}
value={field.name}
onChange={(e) => updateField(idx, { name: e.target.value, id: e.target.value })}
size="small"
sx={{ flex: 1 }}
/>
<TextField
label={t('template.fieldLabel')}
value={field.label}
onChange={(e) => updateField(idx, { label: e.target.value })}
size="small"
sx={{ flex: 1 }}
/>
<TextField
label={t('template.fieldPrompt')}
value={field.promptText}
onChange={(e) => updateField(idx, { promptText: e.target.value })}
size="small"
sx={{ flex: 2 }}
/>
<IconButton
size="small"
onClick={() => removeField(idx)}
disabled={formFields.length <= 1}
sx={{ color: d3roPalette.text.inactive }}
>
<DeleteIcon sx={{ fontSize: 16 }} />
</IconButton>
</Box>
))}
<Button onClick={addField} startIcon={<AddIcon />} size="small" sx={{ alignSelf: 'flex-start' }}>
{t('template.addField')}
</Button>
<TextField
label={t('template.outputFormat')}
value={formOutput}
onChange={(e) => setFormOutput(e.target.value)}
size="small"
fullWidth
multiline
rows={3}
helperText="Use {{fieldName}} for placeholders"
/>
</DialogContent>
<DialogActions sx={{ px: 3, pb: 2 }}>
<Button onClick={() => setDialogOpen(false)} sx={{ color: d3roPalette.text.inactive }}>
{t('common.cancel')}
</Button>
<Button onClick={handleSave} variant="contained" disabled={!formName.trim() || formFields.length === 0}>
{t('common.save')}
</Button>
</DialogActions>
</Dialog>
</Box>
)
}

View file

@ -8,11 +8,13 @@ import ContentCopyIcon from '@mui/icons-material/ContentCopy'
import DeleteIcon from '@mui/icons-material/Delete' import DeleteIcon from '@mui/icons-material/Delete'
import LocalOfferIcon from '@mui/icons-material/LocalOffer' import LocalOfferIcon from '@mui/icons-material/LocalOffer'
import CloseIcon from '@mui/icons-material/Close' import CloseIcon from '@mui/icons-material/Close'
import SummarizeIcon from '@mui/icons-material/Summarize'
import ExpandMoreIcon from '@mui/icons-material/ExpandMore'
import { MetalCard, Led } from '../ds' import { MetalCard, Led } from '../ds'
import { d3roPalette, d3roFontMono, d3roTypo, d3roRadius } from '../../theme' import { d3roPalette, d3roFontMono, d3roTypo, d3roRadius } from '../../theme'
import { useI18n } from '../../i18n' import { useI18n } from '../../i18n'
import { formatDuration } from '../../utils/formatters' import { formatDuration } from '../../utils/formatters'
import type { HistoryEntry, MemoTag } from '@shared/types' import type { HistoryEntry, MemoTag, MeetingSummaryResult } from '@shared/types'
interface HistoryEntryCardProps { interface HistoryEntryCardProps {
entry: HistoryEntry entry: HistoryEntry
@ -28,6 +30,10 @@ export function HistoryEntryCard({ entry, onCopy, onDelete, showTags = false, on
const [tags, setTags] = useState<MemoTag[]>([]) const [tags, setTags] = useState<MemoTag[]>([])
const [tagInput, setTagInput] = useState('') const [tagInput, setTagInput] = useState('')
const [showTagInput, setShowTagInput] = useState(false) const [showTagInput, setShowTagInput] = useState(false)
const [summaryExpanded, setSummaryExpanded] = useState(false)
const [summary, setSummary] = useState<MeetingSummaryResult | null>(null)
const [summaryLoading, setSummaryLoading] = useState(false)
const hasSummary = !!entry.summaryText
const loadTags = useCallback(async () => { const loadTags = useCallback(async () => {
if (!showTags) return if (!showTags) return
@ -60,6 +66,35 @@ export function HistoryEntryCard({ entry, onCopy, onDelete, showTags = false, on
if (e.key === 'Escape') { setShowTagInput(false); setTagInput('') } if (e.key === 'Escape') { setShowTagInput(false); setTagInput('') }
}, [handleAddTag]) }, [handleAddTag])
const handleToggleSummary = useCallback(async () => {
if (summaryExpanded) {
setSummaryExpanded(false)
return
}
setSummaryExpanded(true)
if (!summary) {
setSummaryLoading(true)
const result = await window.electronAPI.meetingSummary.getSummary({ historyId: entry.id })
if (result.success && result.data) {
setSummary(result.data)
}
setSummaryLoading(false)
}
}, [summaryExpanded, summary, entry.id])
const handleGenerateSummary = useCallback(async () => {
setSummaryLoading(true)
const result = await window.electronAPI.meetingSummary.summarize({ historyId: entry.id })
if (result.success) {
setSummary(result.data)
}
setSummaryLoading(false)
}, [entry.id])
const handleExportSummary = useCallback(async () => {
await window.electronAPI.meetingSummary.exportMarkdown({ historyId: entry.id })
}, [entry.id])
return ( return (
<MetalCard> <MetalCard>
<Box sx={{ display: 'flex', alignItems: 'flex-start', gap: 2 }}> <Box sx={{ display: 'flex', alignItems: 'flex-start', gap: 2 }}>
@ -94,6 +129,24 @@ export function HistoryEntryCard({ entry, onCopy, onDelete, showTags = false, on
<span>{formatDuration(entry.duration)}</span> <span>{formatDuration(entry.duration)}</span>
{entry.detectedLanguage && <span>{entry.detectedLanguage.toUpperCase()}</span>} {entry.detectedLanguage && <span>{entry.detectedLanguage.toUpperCase()}</span>}
<span>{entry.mode.toUpperCase()}</span> <span>{entry.mode.toUpperCase()}</span>
{hasSummary && (
<Chip
icon={<SummarizeIcon sx={{ fontSize: '12px !important' }} />}
label={t('meetingSummary.title')}
size="small"
onClick={handleToggleSummary}
sx={{
height: 18,
fontFamily: d3roFontMono,
fontSize: d3roTypo.micro.size,
bgcolor: d3roPalette.tag.greenBg,
color: d3roPalette.tag.green,
borderRadius: d3roRadius.small,
cursor: 'pointer',
'& .MuiChip-icon': { color: d3roPalette.tag.green },
}}
/>
)}
</Box> </Box>
{/* 태그 영역 */} {/* 태그 영역 */}
{showTags && ( {showTags && (
@ -180,6 +233,82 @@ export function HistoryEntryCard({ entry, onCopy, onDelete, showTags = false, on
)} )}
</Box> </Box>
</Box> </Box>
{/* Phase 12.2: 회의록 요약 확장 뷰 */}
{summaryExpanded && (
<Box sx={{ mt: 2, pt: 2, borderTop: `1px solid ${d3roPalette.border.subtle}` }}>
{summaryLoading ? (
<PhosphorText variant="dim">{t('meetingSummary.generating')}</PhosphorText>
) : summary ? (
<Box sx={{ fontSize: d3roTypo.compact.size, color: d3roPalette.text.secondary, lineHeight: 1.6 }}>
{summary.summary && (
<Box sx={{ mb: 1.5 }}>
<PhosphorText variant="label" sx={{ color: d3roPalette.text.dimLabel, mb: 0.5, display: 'block' }}>
{t('meetingSummary.summary').toUpperCase()}
</PhosphorText>
<Box sx={{ whiteSpace: 'pre-wrap' }}>{summary.summary}</Box>
</Box>
)}
{summary.decisions.length > 0 && (
<Box sx={{ mb: 1.5 }}>
<PhosphorText variant="label" sx={{ color: d3roPalette.text.dimLabel, mb: 0.5, display: 'block' }}>
{t('meetingSummary.decisions').toUpperCase()}
</PhosphorText>
{summary.decisions.map((d, i) => (
<Box key={i} sx={{ pl: 1.5 }}> {d}</Box>
))}
</Box>
)}
{summary.actionItems.length > 0 && (
<Box sx={{ mb: 1 }}>
<PhosphorText variant="label" sx={{ color: d3roPalette.text.dimLabel, mb: 0.5, display: 'block' }}>
{t('meetingSummary.actionItems').toUpperCase()}
</PhosphorText>
{summary.actionItems.map((a, i) => (
<Box key={i} sx={{ pl: 1.5 }}> {a}</Box>
))}
</Box>
)}
<Box sx={{ display: 'flex', gap: 1, mt: 1 }}>
<Tooltip title={t('meetingSummary.exportMarkdown')} arrow>
<IconButton size="small" onClick={handleExportSummary} sx={{ color: d3roPalette.text.inactive, '&:hover': { color: d3roPalette.accent.amber } }}>
<SummarizeIcon sx={{ fontSize: 14 }} />
</IconButton>
</Tooltip>
</Box>
</Box>
) : (
<Box sx={{ textAlign: 'center' }}>
<PhosphorText variant="dim" sx={{ mb: 1 }}>{t('meetingSummary.noSummary')}</PhosphorText>
{(entry.mode === 'caption' || entry.mode === 'file-transcription') && (
<Chip
label={t('meetingSummary.generate')}
size="small"
onClick={handleGenerateSummary}
sx={{
fontFamily: d3roFontMono,
fontSize: d3roTypo.micro.size,
bgcolor: d3roPalette.accent.amber,
color: d3roPalette.bg.chassis,
cursor: 'pointer',
'&:hover': { opacity: 0.85 },
}}
/>
)}
</Box>
)}
</Box>
)}
{/* 요약 토글 버튼 (caption/file-transcription 모드만) */}
{(entry.mode === 'caption' || entry.mode === 'file-transcription') && !summaryExpanded && !hasSummary && (
<Box
sx={{ mt: 1, textAlign: 'center', cursor: 'pointer', color: d3roPalette.text.muted, '&:hover': { color: d3roPalette.accent.amber } }}
onClick={handleToggleSummary}
>
<ExpandMoreIcon sx={{ fontSize: 16 }} />
</Box>
)}
</MetalCard> </MetalCard>
) )
} }

View file

@ -300,13 +300,28 @@
"license.feature.voice_memo": "Voice Memo", "license.feature.voice_memo": "Voice Memo",
"license.feature.history_unlimited": "Unlimited History", "license.feature.history_unlimited": "Unlimited History",
"license.feature.history_export": "History Export", "license.feature.history_export": "History Export",
"license.feature.custom_instruction_create": "Custom Instruction Create", "license.feature.custom_instruction_create": "Custom Instruction",
"license.feature.file_transcription": "File Transcription", "license.feature.file_transcription": "File Transcription",
"license.feature.voice_conversation": "Voice Conversation", "license.feature.voice_conversation": "Voice Conversation",
"license.feature.dictation_template": "Dictation Template", "license.feature.dictation_template": "Dictation Template",
"license.feature.meeting_summary": "Meeting Summary", "license.feature.meeting_summary": "Meeting Summary",
"license.feature.local_rag": "Local RAG", "license.feature.local_rag": "Local RAG",
"license.feature.os_automation": "OS Automation", "license.feature.os_automation": "OS Automation",
"license.feature.llmProcess": "LLM Processing",
"license.feature.liveCaption": "Live Caption",
"license.feature.screenContext": "Screen Context",
"license.feature.voiceCommand": "Voice Commands",
"license.feature.llmChain": "LLM Chain",
"license.feature.voiceMemo": "Voice Memo",
"license.feature.historyUnlimited": "Unlimited History",
"license.feature.historyExport": "History Export",
"license.feature.customInstruction": "Custom Instruction",
"license.feature.fileTranscription": "File Transcription",
"license.feature.voiceConversation": "Voice Conversation",
"license.feature.dictationTemplate": "Dictation Template",
"license.feature.meetingSummary": "Meeting Summary",
"license.feature.localRag": "Local RAG",
"license.feature.osAutomation": "OS Automation",
"license.pro.required": "PRO Required", "license.pro.required": "PRO Required",
"license.proPlus.required": "PRO+ Required", "license.proPlus.required": "PRO+ Required",
"license.included": "Included", "license.included": "Included",
@ -327,5 +342,80 @@
"license.perDay": "/day", "license.perDay": "/day",
"license.unlimited": "Unlimited", "license.unlimited": "Unlimited",
"license.locked": "Locked", "license.locked": "Locked",
"license.nav": "License" "license.nav": "License",
"fileTranscription.title": "File Transcription",
"fileTranscription.dropZone": "Drop audio/video file here",
"fileTranscription.dropZoneHint": "Supports MP3, WAV, M4A, MP4, MKV, WEBM",
"fileTranscription.converting": "Converting...",
"fileTranscription.processing": "Transcribing...",
"fileTranscription.progress": "Chunk {{current}} / {{total}}",
"fileTranscription.complete": "Transcription complete",
"fileTranscription.copyAll": "Copy All",
"fileTranscription.cancel": "Cancel",
"fileTranscription.retry": "Retry",
"fileTranscription.error.invalidFormat": "Unsupported file format",
"fileTranscription.error.unknown": "Unknown error",
"meetingSummary.title": "Meeting Summary",
"meetingSummary.generating": "Generating summary...",
"meetingSummary.summary": "Summary",
"meetingSummary.decisions": "Key Decisions",
"meetingSummary.actionItems": "Action Items",
"meetingSummary.exportMarkdown": "Export as Markdown",
"meetingSummary.noSummary": "No summary available",
"meetingSummary.generate": "Generate Summary",
"template.title": "Dictation Templates",
"template.create": "New Template",
"template.edit": "Edit Template",
"template.delete": "Delete",
"template.name": "Template Name",
"template.description": "Description",
"template.fields": "Fields",
"template.addField": "Add Field",
"template.fieldName": "Field Name",
"template.fieldLabel": "Label",
"template.fieldPrompt": "Voice Prompt",
"template.outputFormat": "Output Format",
"template.startSession": "Start",
"template.cancelSession": "Cancel Session",
"template.empty": "No templates",
"nav.conversation": "Talk",
"conversation.title": "Voice Conversation",
"conversation.idle": "Idle",
"conversation.listening": "Listening",
"conversation.thinking": "Thinking",
"conversation.speaking": "Speaking",
"conversation.empty": "Talk to D3RO",
"conversation.emptyHint": "Press the mic button or type a message",
"conversation.inputPlaceholder": "Type a message...",
"conversation.end": "End",
"conversation.clearHistory": "Clear history",
"nav.knowledge": "Knowledge",
"rag.title": "Knowledge Base",
"rag.addDocument": "Add Document",
"rag.documents": "Documents",
"rag.noDocuments": "No documents",
"rag.chunks": "chunks",
"rag.indexed": "Indexed",
"rag.pending": "Pending",
"rag.indexing": "Indexing",
"rag.reindex": "Reindex",
"rag.askQuestion": "Ask a Question",
"rag.queryPlaceholder": "Ask about your documents...",
"rag.searching": "Searching...",
"rag.sources": "Sources",
"rag.adding": "Adding...",
"rag.parsing": "Parsing document...",
"voiceAction.title": "Voice Actions",
"voiceAction.execute": "Execute",
"voiceAction.presets": "Preset Commands",
"voiceAction.history": "Action History",
"voiceAction.noHistory": "No action history",
"voiceAction.blocked": "Blocked (unsafe)",
"voiceAction.executed": "Executed"
} }

View file

@ -307,6 +307,21 @@
"license.feature.meeting_summary": "회의 요약", "license.feature.meeting_summary": "회의 요약",
"license.feature.local_rag": "로컬 RAG", "license.feature.local_rag": "로컬 RAG",
"license.feature.os_automation": "OS 자동화", "license.feature.os_automation": "OS 자동화",
"license.feature.llmProcess": "LLM 처리",
"license.feature.liveCaption": "실시간 자막",
"license.feature.screenContext": "화면 컨텍스트",
"license.feature.voiceCommand": "음성 명령어",
"license.feature.llmChain": "LLM 체인",
"license.feature.voiceMemo": "음성 메모",
"license.feature.historyUnlimited": "무제한 히스토리",
"license.feature.historyExport": "히스토리 내보내기",
"license.feature.customInstruction": "커스텀 명령어 생성",
"license.feature.fileTranscription": "파일 전사",
"license.feature.voiceConversation": "음성 대화",
"license.feature.dictationTemplate": "받아쓰기 템플릿",
"license.feature.meetingSummary": "회의 요약",
"license.feature.localRag": "로컬 RAG",
"license.feature.osAutomation": "OS 자동화",
"license.pro.required": "PRO 필요", "license.pro.required": "PRO 필요",
"license.proPlus.required": "PRO+ 필요", "license.proPlus.required": "PRO+ 필요",
"license.included": "포함", "license.included": "포함",
@ -327,5 +342,80 @@
"license.perDay": "회/일", "license.perDay": "회/일",
"license.unlimited": "무제한", "license.unlimited": "무제한",
"license.locked": "잠금", "license.locked": "잠금",
"license.nav": "라이선스" "license.nav": "라이선스",
"fileTranscription.title": "파일 전사",
"fileTranscription.dropZone": "오디오/비디오 파일을 여기에 드래그하세요",
"fileTranscription.dropZoneHint": "MP3, WAV, M4A, MP4, MKV, WEBM 지원",
"fileTranscription.converting": "변환 중...",
"fileTranscription.processing": "전사 중...",
"fileTranscription.progress": "청크 {{current}} / {{total}}",
"fileTranscription.complete": "전사 완료",
"fileTranscription.copyAll": "전체 복사",
"fileTranscription.cancel": "취소",
"fileTranscription.retry": "다시 시도",
"fileTranscription.error.invalidFormat": "지원하지 않는 파일 형식입니다",
"fileTranscription.error.unknown": "알 수 없는 오류",
"meetingSummary.title": "회의록 요약",
"meetingSummary.generating": "요약 생성 중...",
"meetingSummary.summary": "요약",
"meetingSummary.decisions": "핵심 결정사항",
"meetingSummary.actionItems": "할 일 목록",
"meetingSummary.exportMarkdown": "마크다운 내보내기",
"meetingSummary.noSummary": "요약이 없습니다",
"meetingSummary.generate": "요약 생성",
"template.title": "딕테이션 템플릿",
"template.create": "새 템플릿",
"template.edit": "템플릿 편집",
"template.delete": "삭제",
"template.name": "템플릿 이름",
"template.description": "설명",
"template.fields": "필드",
"template.addField": "필드 추가",
"template.fieldName": "필드명",
"template.fieldLabel": "라벨",
"template.fieldPrompt": "음성 안내",
"template.outputFormat": "출력 포맷",
"template.startSession": "시작",
"template.cancelSession": "세션 취소",
"template.empty": "템플릿이 없습니다",
"nav.conversation": "대화",
"conversation.title": "음성 대화",
"conversation.idle": "대기",
"conversation.listening": "듣는 중",
"conversation.thinking": "생각 중",
"conversation.speaking": "말하는 중",
"conversation.empty": "D3RO에게 말을 걸어보세요",
"conversation.emptyHint": "마이크 버튼을 누르거나 텍스트를 입력하세요",
"conversation.inputPlaceholder": "메시지 입력...",
"conversation.end": "종료",
"conversation.clearHistory": "대화 초기화",
"nav.knowledge": "지식 베이스",
"rag.title": "지식 베이스",
"rag.addDocument": "문서 추가",
"rag.documents": "문서",
"rag.noDocuments": "문서가 없습니다",
"rag.chunks": "청크",
"rag.indexed": "인덱싱 완료",
"rag.pending": "대기 중",
"rag.indexing": "인덱싱 중",
"rag.reindex": "재인덱싱",
"rag.askQuestion": "질문하기",
"rag.queryPlaceholder": "문서에 대해 질문하세요...",
"rag.searching": "검색 중...",
"rag.sources": "참조 문서",
"rag.adding": "추가 중...",
"rag.parsing": "문서 분석 중...",
"voiceAction.title": "음성 액션",
"voiceAction.execute": "실행",
"voiceAction.presets": "프리셋 명령",
"voiceAction.history": "실행 이력",
"voiceAction.noHistory": "실행 이력이 없습니다",
"voiceAction.blocked": "차단됨 (위험 명령)",
"voiceAction.executed": "실행됨"
} }

View file

@ -14,6 +14,7 @@ import { MetalCard, PhosphorText, Led, ScreenPanel } from '../components/ds'
import { PageHeader, EmptyStateCard } from '../components/shared' import { PageHeader, EmptyStateCard } from '../components/shared'
import { d3roPalette, d3roFontMono, d3roTypo, d3roRadius } from '../theme' import { d3roPalette, d3roFontMono, d3roTypo, d3roRadius } from '../theme'
import { useI18n } from '../i18n' import { useI18n } from '../i18n'
import { TemplateSection } from '../components/TemplateSection'
import type { VoiceCommandRule, VoiceCommandKeyword, KeywordMatchMode, LLMChain, ChainStep } from '@shared/types' import type { VoiceCommandRule, VoiceCommandKeyword, KeywordMatchMode, LLMChain, ChainStep } from '@shared/types'
interface CustomInstruction { interface CustomInstruction {
@ -574,6 +575,9 @@ function ChainSection({ instructions }: { instructions: CustomInstruction[] }):
<Button onClick={handleSave} variant="contained" disabled={!formName.trim() || formSteps.length === 0}>{t('common.save')}</Button> <Button onClick={handleSave} variant="contained" disabled={!formName.trim() || formSteps.length === 0}>{t('common.save')}</Button>
</DialogActions> </DialogActions>
</Dialog> </Dialog>
{/* ── Phase 12.3: 딕테이션 템플릿 ── */}
<TemplateSection />
</Box> </Box>
) )
} }

View file

@ -9,6 +9,7 @@ import { EmptyStateCard, HistoryEntryCard } from '../components/shared'
import { d3roPalette, d3roTypo } from '../theme' import { d3roPalette, d3roTypo } from '../theme'
import { useI18n } from '../i18n' import { useI18n } from '../i18n'
import { formatRecordingTime, formatRecordingTimeUnit, formatNumber, getDateKey } from '../utils/formatters' import { formatRecordingTime, formatRecordingTimeUnit, formatNumber, getDateKey } from '../utils/formatters'
import { FileDropZone } from '../components/FileDropZone'
import type { StatsSummary, HistoryEntry, HotkeyBinding, CaptionState, LicenseTier, UsageQuota } from '@shared/types' import type { StatsSummary, HistoryEntry, HotkeyBinding, CaptionState, LicenseTier, UsageQuota } from '@shared/types'
// ── 메인 컴포넌트 ───────────────────────────────────── // ── 메인 컴포넌트 ─────────────────────────────────────
@ -349,6 +350,14 @@ export function DashboardPage(): React.ReactElement {
</CrtDisplay> </CrtDisplay>
</Box> </Box>
{/* ── 3.5 파일 전사 (Phase 12.1) ──────────────── */}
<Box sx={{ mt: 3 }}>
<PhosphorText variant="label" sx={{ mb: 1.5, display: 'block', color: d3roPalette.text.inactive }}>
{t('fileTranscription.title').toUpperCase()}
</PhosphorText>
<FileDropZone />
</Box>
{/* ── 4. 최근 히스토리 ──────────────────────── */} {/* ── 4. 최근 히스토리 ──────────────────────── */}
<Box sx={{ mt: 4 }}> <Box sx={{ mt: 4 }}>
<PhosphorText variant="label" sx={{ mb: 2, display: 'block', color: d3roPalette.text.inactive }}> <PhosphorText variant="label" sx={{ mb: 2, display: 'block', color: d3roPalette.text.inactive }}>

View file

@ -0,0 +1,262 @@
// src/renderer/pages/KnowledgeBasePage.tsx
// Phase 13.2: 로컬 RAG Knowledge Base UI
// 문서 관리 + 질문 입력 + 답변 표시
import { useState, useEffect, useCallback } from 'react'
import { Box, TextField, IconButton, Tooltip, LinearProgress } from '@mui/material'
import AddIcon from '@mui/icons-material/Add'
import DeleteIcon from '@mui/icons-material/Delete'
import RefreshIcon from '@mui/icons-material/Refresh'
import SearchIcon from '@mui/icons-material/Search'
import SendIcon from '@mui/icons-material/Send'
import DescriptionIcon from '@mui/icons-material/Description'
import { MetalCard, PhosphorText, Led, PhysicalButton, ScreenPanel } from '../components/ds'
import { PageHeader, EmptyStateCard } from '../components/shared'
import { d3roPalette, d3roFontMono, d3roTypo } from '../theme'
import { useI18n } from '../i18n'
import type { RAGDocument, RAGQueryResult, RAGIndexProgress } from '@shared/types'
export function KnowledgeBasePage(): React.ReactElement {
const { t } = useI18n()
const [documents, setDocuments] = useState<RAGDocument[]>([])
const [loading, setLoading] = useState(true)
const [query, setQuery] = useState('')
const [querying, setQuerying] = useState(false)
const [result, setResult] = useState<RAGQueryResult | null>(null)
const [indexProgress, setIndexProgress] = useState<RAGIndexProgress | null>(null)
const [adding, setAdding] = useState(false)
const [addError, setAddError] = useState<string | null>(null)
const loadDocuments = useCallback(async () => {
setLoading(true)
const resp = await window.electronAPI.rag.getDocuments()
if (resp.success) setDocuments(resp.data)
setLoading(false)
}, [])
useEffect(() => { loadDocuments() }, [loadDocuments])
useEffect(() => {
const unsubProgress = window.electronAPI.rag.onIndexProgress((data) => {
setIndexProgress(data)
})
const unsubComplete = window.electronAPI.rag.onIndexComplete(() => {
setIndexProgress(null)
loadDocuments()
})
return () => { unsubProgress(); unsubComplete() }
}, [loadDocuments])
const handleAddDocument = useCallback(async () => {
setAdding(true)
setAddError(null)
const resp = await window.electronAPI.rag.addDocument()
setAdding(false)
if (resp.success) {
loadDocuments()
} else {
if (!resp.error.message.includes('cancelled')) {
setAddError(resp.error.message)
}
}
}, [loadDocuments])
const handleRemoveDocument = useCallback(async (docId: string) => {
await window.electronAPI.rag.removeDocument({ documentId: docId })
loadDocuments()
}, [loadDocuments])
const handleReindex = useCallback(async (docId: string) => {
await window.electronAPI.rag.reindex(docId)
}, [])
const handleQuery = useCallback(async () => {
if (!query.trim()) return
setQuerying(true)
setResult(null)
const resp = await window.electronAPI.rag.query({ query: query.trim() })
if (resp.success) {
setResult(resp.data)
}
setQuerying(false)
}, [query])
const handleQueryKeyDown = useCallback((e: React.KeyboardEvent) => {
if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault()
handleQuery()
}
}, [handleQuery])
return (
<Box sx={{ maxWidth: 900, mx: 'auto', p: 4, pb: 8 }}>
<PageHeader
title={t('rag.title').toUpperCase()}
action={
<PhysicalButton size="small" onClick={handleAddDocument} disabled={adding}>
<AddIcon sx={{ fontSize: 14, mr: 0.5 }} /> {adding ? t('rag.adding') : t('rag.addDocument')}
</PhysicalButton>
}
/>
{/* 인덱싱 진행률 */}
{indexProgress && (
<MetalCard sx={{ mb: 2 }}>
<Box sx={{ display: 'flex', alignItems: 'center', gap: 1, mb: 1 }}>
<Led color="amber" pulse />
<PhosphorText variant="body">
{t('rag.indexing')}: {indexProgress.fileName}
</PhosphorText>
</Box>
<LinearProgress
variant="determinate"
value={indexProgress.percent}
sx={{
height: 4,
borderRadius: 2,
bgcolor: d3roPalette.bg.inset,
'& .MuiLinearProgress-bar': { bgcolor: d3roPalette.accent.amber },
}}
/>
<PhosphorText variant="dim" sx={{ mt: 0.5 }}>
{indexProgress.currentChunk}/{indexProgress.totalChunks}
</PhosphorText>
</MetalCard>
)}
{/* 에러 표시 */}
{addError && (
<MetalCard sx={{ mb: 2, border: `1px solid ${d3roPalette.tag.red}` }}>
<Box sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
<Led color="red" />
<PhosphorText variant="body" sx={{ color: d3roPalette.tag.red }}>
{addError}
</PhosphorText>
</Box>
</MetalCard>
)}
{/* 추가 중 인디케이터 */}
{adding && !indexProgress && (
<MetalCard sx={{ mb: 2 }}>
<Box sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
<Led color="amber" pulse />
<PhosphorText variant="body">{t('rag.parsing')}</PhosphorText>
</Box>
</MetalCard>
)}
{/* 문서 목록 */}
<PhosphorText variant="label" sx={{ mb: 1.5, display: 'block', color: d3roPalette.text.inactive }}>
{t('rag.documents').toUpperCase()} ({documents.length})
</PhosphorText>
{documents.length === 0 && !loading ? (
<EmptyStateCard message={t('rag.noDocuments')} />
) : (
<Box sx={{ display: 'flex', flexDirection: 'column', gap: 1, mb: 3 }}>
{documents.map((doc) => (
<MetalCard key={doc.id}>
<Box sx={{ display: 'flex', alignItems: 'center', gap: 2 }}>
<DescriptionIcon sx={{ fontSize: 20, color: d3roPalette.text.inactive }} />
<Box sx={{ flex: 1 }}>
<PhosphorText variant="body">{doc.fileName}</PhosphorText>
<Box sx={{ display: 'flex', alignItems: 'center', gap: 0.5, mt: 0.25 }}>
<PhosphorText variant="dim">
{doc.fileType.toUpperCase()} · {doc.chunkCount} {t('rag.chunks')} ·
</PhosphorText>
<Led color={doc.indexed ? 'green' : 'amber'} size={6} />
<PhosphorText variant="dim">
{doc.indexed ? t('rag.indexed') : t('rag.pending')}
</PhosphorText>
</Box>
</Box>
<Tooltip title={t('rag.reindex')}>
<IconButton size="small" onClick={() => handleReindex(doc.id)}
sx={{ color: d3roPalette.text.inactive, '&:hover': { color: d3roPalette.accent.amber } }}>
<RefreshIcon sx={{ fontSize: 16 }} />
</IconButton>
</Tooltip>
<Tooltip title={t('common.delete')}>
<IconButton size="small" onClick={() => handleRemoveDocument(doc.id)}
sx={{ color: d3roPalette.text.inactive, '&:hover': { color: d3roPalette.tag.red } }}>
<DeleteIcon sx={{ fontSize: 16 }} />
</IconButton>
</Tooltip>
</Box>
</MetalCard>
))}
</Box>
)}
{/* 질문 입력 */}
<PhosphorText variant="label" sx={{ mb: 1.5, display: 'block', color: d3roPalette.text.inactive }}>
{t('rag.askQuestion').toUpperCase()}
</PhosphorText>
<MetalCard>
<Box sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
<SearchIcon sx={{ fontSize: 20, color: d3roPalette.text.inactive }} />
<TextField
value={query}
onChange={(e) => setQuery(e.target.value)}
onKeyDown={handleQueryKeyDown}
placeholder={t('rag.queryPlaceholder')}
size="small"
fullWidth
sx={{
'& .MuiOutlinedInput-root': {
fontFamily: d3roFontMono,
fontSize: d3roTypo.compact.size,
bgcolor: d3roPalette.bg.inset,
'& fieldset': { borderColor: d3roPalette.border.subtle },
'&:hover fieldset': { borderColor: d3roPalette.accent.amber },
},
'& .MuiOutlinedInput-input': { color: d3roPalette.text.primary },
}}
/>
<IconButton onClick={handleQuery} disabled={!query.trim() || querying}
sx={{ color: query.trim() ? d3roPalette.accent.amber : d3roPalette.text.inactive }}>
<SendIcon sx={{ fontSize: 20 }} />
</IconButton>
</Box>
</MetalCard>
{/* 답변 */}
{querying && (
<Box sx={{ mt: 2, textAlign: 'center' }}>
<Led color="amber" pulse />
<PhosphorText variant="dim" sx={{ ml: 1 }}>{t('rag.searching')}</PhosphorText>
</Box>
)}
{result && (
<Box sx={{ mt: 2 }}>
<ScreenPanel sx={{ p: 3 }}>
<PhosphorText variant="body" sx={{ whiteSpace: 'pre-wrap', lineHeight: 1.8 }}>
{result.answer}
</PhosphorText>
</ScreenPanel>
{result.results.length > 0 && (
<Box sx={{ mt: 2 }}>
<PhosphorText variant="label" sx={{ color: d3roPalette.text.dimLabel, mb: 1, display: 'block' }}>
{t('rag.sources').toUpperCase()}
</PhosphorText>
{result.results.slice(0, 3).map((r, i) => (
<MetalCard key={i} sx={{ mb: 1 }}>
<PhosphorText variant="dim">
[{i + 1}] {r.fileName} ({Math.round(r.similarity * 100)}%)
</PhosphorText>
<PhosphorText variant="compact" sx={{ mt: 0.5, opacity: 0.7 }}>
{r.content.slice(0, 150)}...
</PhosphorText>
</MetalCard>
))}
</Box>
)}
</Box>
)}
</Box>
)
}

View file

@ -0,0 +1,291 @@
// src/renderer/pages/VoiceConversationPage.tsx
// Phase 13.1: 음성 대화 모드 UI
// STT→LLM→TTS 대화 루프. 채팅 메시지 목록 + 녹음 버튼.
import { useState, useEffect, useCallback, useRef } from 'react'
import { Box, IconButton, TextField, Tooltip } from '@mui/material'
import MicIcon from '@mui/icons-material/Mic'
import StopIcon from '@mui/icons-material/Stop'
import SendIcon from '@mui/icons-material/Send'
import DeleteSweepIcon from '@mui/icons-material/DeleteSweep'
import CancelIcon from '@mui/icons-material/Cancel'
import { MetalCard, PhosphorText, Led, PhysicalButton, ScreenPanel, InstrumentPanel } from '../components/ds'
import { PageHeader } from '../components/shared'
import { d3roPalette, d3roFontMono, d3roTypo } from '../theme'
import { useI18n } from '../i18n'
import type {
ConversationState,
ConversationMessage,
ConversationAssistantDelta,
} from '@shared/types'
export function VoiceConversationPage(): React.ReactElement {
const { t } = useI18n()
const [state, setState] = useState<ConversationState>('idle')
const [messages, setMessages] = useState<ConversationMessage[]>([])
const [isActive, setIsActive] = useState(false)
const [streamingText, setStreamingText] = useState('')
const [streamingMsgId, setStreamingMsgId] = useState<string | null>(null)
const [textInput, setTextInput] = useState('')
const messagesEndRef = useRef<HTMLDivElement>(null)
const scrollToBottom = useCallback(() => {
messagesEndRef.current?.scrollIntoView({ behavior: 'smooth' })
}, [])
// IPC 이벤트 구독
useEffect(() => {
const unsubState = window.electronAPI.voiceConversation.onStateChanged((info) => {
setState(info.state)
setMessages(info.messages)
setIsActive(info.isActive)
})
const unsubUser = window.electronAPI.voiceConversation.onUserMessage((msg) => {
setMessages((prev) => [...prev, msg])
setStreamingText('')
setStreamingMsgId(null)
setTimeout(scrollToBottom, 50)
})
const unsubDelta = window.electronAPI.voiceConversation.onAssistantDelta((data: ConversationAssistantDelta) => {
setStreamingMsgId(data.messageId)
setStreamingText(data.accumulated)
setTimeout(scrollToBottom, 50)
})
const unsubComplete = window.electronAPI.voiceConversation.onAssistantMessage((msg) => {
setMessages((prev) => [...prev, { id: msg.messageId, role: 'assistant', content: msg.content, timestamp: Date.now() }])
setStreamingText('')
setStreamingMsgId(null)
setTimeout(scrollToBottom, 50)
})
const unsubError = window.electronAPI.voiceConversation.onError(() => {
// 에러 시 자동 복구 (서비스에서 listening으로 전환)
})
// 초기 상태 로드
window.electronAPI.voiceConversation.getState().then((r) => {
if (r.success) {
setState(r.data.state)
setMessages(r.data.messages)
setIsActive(r.data.isActive)
}
})
return () => {
unsubState()
unsubUser()
unsubDelta()
unsubComplete()
unsubError()
}
}, [scrollToBottom])
const handleStartSession = useCallback(async () => {
await window.electronAPI.voiceConversation.startSession()
}, [])
const handleStopSession = useCallback(async () => {
await window.electronAPI.voiceConversation.stopSession()
}, [])
const handleFinishListening = useCallback(async () => {
await window.electronAPI.voiceConversation.finishListening()
}, [])
const handleSendText = useCallback(async () => {
if (!textInput.trim()) return
const text = textInput.trim()
setTextInput('')
if (!isActive) {
await window.electronAPI.voiceConversation.startSession()
}
await window.electronAPI.voiceConversation.sendMessage({ text })
}, [textInput, isActive])
const handleClearHistory = useCallback(async () => {
await window.electronAPI.voiceConversation.clearHistory()
setMessages([])
}, [])
const handleCancelResponse = useCallback(async () => {
await window.electronAPI.voiceConversation.cancelResponse()
}, [])
const handleTextKeyDown = useCallback((e: React.KeyboardEvent) => {
if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault()
handleSendText()
}
}, [handleSendText])
const stateLabel = {
idle: t('conversation.idle'),
listening: t('conversation.listening'),
thinking: t('conversation.thinking'),
speaking: t('conversation.speaking'),
}
const stateLedColor = {
idle: 'amber' as const,
listening: 'red' as const,
thinking: 'amber' as const,
speaking: 'green' as const,
}
return (
<Box sx={{ maxWidth: 800, mx: 'auto', p: 4, pb: 8, display: 'flex', flexDirection: 'column', height: '100%' }}>
<PageHeader
title={t('conversation.title').toUpperCase()}
action={
<Box sx={{ display: 'flex', gap: 1, alignItems: 'center' }}>
<Led color={stateLedColor[state]} pulse={state === 'listening' || state === 'thinking'} size={8} />
<PhosphorText variant="label" sx={{ color: d3roPalette.text.dimLabel }}>
{stateLabel[state].toUpperCase()}
</PhosphorText>
{messages.length > 0 && (
<Tooltip title={t('conversation.clearHistory')}>
<IconButton size="small" onClick={handleClearHistory} sx={{ color: d3roPalette.text.inactive }}>
<DeleteSweepIcon sx={{ fontSize: 18 }} />
</IconButton>
</Tooltip>
)}
</Box>
}
/>
{/* 메시지 목록 */}
<Box
sx={{
flex: 1,
overflow: 'auto',
mt: 2,
mb: 2,
display: 'flex',
flexDirection: 'column',
gap: 1.5,
}}
>
{messages.length === 0 && !streamingText && (
<Box sx={{ textAlign: 'center', mt: 8 }}>
<PhosphorText variant="heading" sx={{ color: d3roPalette.text.inactive, mb: 1 }}>
{t('conversation.empty')}
</PhosphorText>
<PhosphorText variant="dim">
{t('conversation.emptyHint')}
</PhosphorText>
</Box>
)}
{messages.map((msg) => (
<Box
key={msg.id}
sx={{
display: 'flex',
justifyContent: msg.role === 'user' ? 'flex-end' : 'flex-start',
}}
>
<MetalCard
sx={{
maxWidth: '75%',
...(msg.role === 'user' && {
bgcolor: d3roPalette.accent.amber,
'& *': { color: `${d3roPalette.bg.chassis} !important` },
}),
}}
>
<PhosphorText
variant="compact"
sx={{
whiteSpace: 'pre-wrap',
lineHeight: 1.6,
}}
>
{msg.content}
</PhosphorText>
</MetalCard>
</Box>
))}
{/* 스트리밍 중인 어시스턴트 메시지 */}
{streamingText && streamingMsgId && (
<Box sx={{ display: 'flex', justifyContent: 'flex-start' }}>
<MetalCard sx={{ maxWidth: '75%' }}>
<PhosphorText variant="compact" sx={{ whiteSpace: 'pre-wrap', lineHeight: 1.6 }}>
{streamingText}
<Box component="span" sx={{ animation: 'blink 1s infinite', color: d3roPalette.accent.amber }}>
{'▌'}
</Box>
</PhosphorText>
</MetalCard>
</Box>
)}
<div ref={messagesEndRef} />
</Box>
{/* 하단 컨트롤 바 */}
<MetalCard>
<Box sx={{ display: 'flex', alignItems: 'center', gap: 1.5 }}>
{/* 녹음 버튼 */}
{!isActive ? (
<PhysicalButton onClick={handleStartSession} sx={{ minWidth: 48, px: 2 }}>
<MicIcon sx={{ fontSize: 20 }} />
</PhysicalButton>
) : state === 'listening' ? (
<PhysicalButton selected onClick={handleFinishListening} sx={{ minWidth: 48, px: 2 }}>
<StopIcon sx={{ fontSize: 20 }} />
</PhysicalButton>
) : state === 'thinking' || state === 'speaking' ? (
<PhysicalButton onClick={handleCancelResponse} sx={{ minWidth: 48, px: 2 }}>
<CancelIcon sx={{ fontSize: 20 }} />
</PhysicalButton>
) : (
<PhysicalButton onClick={handleFinishListening} sx={{ minWidth: 48, px: 2 }}>
<MicIcon sx={{ fontSize: 20 }} />
</PhysicalButton>
)}
{/* 텍스트 입력 */}
<TextField
value={textInput}
onChange={(e) => setTextInput(e.target.value)}
onKeyDown={handleTextKeyDown}
placeholder={t('conversation.inputPlaceholder')}
size="small"
fullWidth
sx={{
'& .MuiOutlinedInput-root': {
fontFamily: d3roFontMono,
fontSize: d3roTypo.compact.size,
bgcolor: d3roPalette.bg.inset,
'& fieldset': { borderColor: d3roPalette.border.subtle },
'&:hover fieldset': { borderColor: d3roPalette.accent.amber },
'&.Mui-focused fieldset': { borderColor: d3roPalette.accent.amber },
},
'& .MuiOutlinedInput-input': {
color: d3roPalette.text.primary,
},
}}
/>
{/* 전송 버튼 */}
<IconButton
onClick={handleSendText}
disabled={!textInput.trim()}
sx={{
color: textInput.trim() ? d3roPalette.accent.amber : d3roPalette.text.inactive,
}}
>
<SendIcon sx={{ fontSize: 20 }} />
</IconButton>
{/* 세션 종료 */}
{isActive && (
<PhysicalButton onClick={handleStopSession} sx={{ minWidth: 48, px: 1 }}>
<PhosphorText variant="micro">{t('conversation.end')}</PhosphorText>
</PhysicalButton>
)}
</Box>
</MetalCard>
</Box>
)
}

View file

@ -113,6 +113,44 @@ export enum ErrorCode {
CaptionAlreadyActive = 771, CaptionAlreadyActive = 771,
CaptionSTTFailed = 772, CaptionSTTFailed = 772,
// === Phase 12: File Transcription (780-784) ===
FileTranscriptionFFmpegFailed = 780,
FileTranscriptionInvalidFormat = 781,
FileTranscriptionChunkFailed = 782,
FileTranscriptionCancelled = 783,
FileTranscriptionFileTooLarge = 784,
// === Phase 12: Meeting Summary (785-789) ===
MeetingSummaryGenerationFailed = 785,
MeetingSummaryNoTranscript = 786,
MeetingSummaryExportFailed = 787,
// === Phase 12: Dictation Template (790-794) ===
TemplateNotFound = 790,
TemplateSessionAlreadyActive = 791,
TemplateSessionNotActive = 792,
TemplateFieldRecordingFailed = 793,
TemplateInvalidFormat = 794,
// === Phase 13: Voice Conversation (795-799) ===
ConversationSessionAlreadyActive = 795,
ConversationNoActiveSession = 796,
ConversationTTSFailed = 797,
ConversationLLMFailed = 798,
// === Phase 13: Local RAG (870-874) ===
RAGDocumentNotFound = 870,
RAGIndexingFailed = 871,
RAGEmbeddingFailed = 872,
RAGQueryFailed = 873,
RAGUnsupportedFormat = 874,
// === Phase 13: Voice Action (875-879) ===
VoiceActionPlanFailed = 875,
VoiceActionExecutionFailed = 876,
VoiceActionBlocked = 877,
VoiceActionInvalidPlan = 878,
// === Config (800-849) === // === Config (800-849) ===
ConfigReadFailed = 800, ConfigReadFailed = 800,
ConfigWriteFailed = 801, ConfigWriteFailed = 801,

View file

@ -221,6 +221,90 @@ export const IPC_CHANNELS = {
STOP_SYSTEM_AUDIO: 'caption:stopSystemAudio', STOP_SYSTEM_AUDIO: 'caption:stopSystemAudio',
}, },
// ── Phase 12: File Transcription (12.1) ──
FILE_TRANSCRIPTION: {
START: 'fileTranscription:start',
CANCEL: 'fileTranscription:cancel',
GET_STATE: 'fileTranscription:getState',
// Main → Renderer events
PROGRESS: 'fileTranscription:progress',
COMPLETE: 'fileTranscription:complete',
ERROR: 'fileTranscription:error',
},
// ── Phase 12: Meeting Summary (12.2) ──
MEETING_SUMMARY: {
SUMMARIZE: 'meetingSummary:summarize',
GET_SUMMARY: 'meetingSummary:getSummary',
EXPORT_MARKDOWN: 'meetingSummary:exportMarkdown',
// Main → Renderer events
SUMMARY_READY: 'meetingSummary:summaryReady',
SUMMARY_PROGRESS: 'meetingSummary:progress',
},
// ── Phase 12: Dictation Templates (12.3) ──
DICTATION_TEMPLATE: {
GET_ALL: 'dictationTemplate:getAll',
CREATE: 'dictationTemplate:create',
UPDATE: 'dictationTemplate:update',
DELETE: 'dictationTemplate:delete',
START_SESSION: 'dictationTemplate:startSession',
CANCEL_SESSION: 'dictationTemplate:cancelSession',
GET_SESSION_STATE: 'dictationTemplate:getSessionState',
SET_FIELD_VALUE: 'dictationTemplate:setFieldValue',
// Main → Renderer events
SESSION_STATE_CHANGED: 'dictationTemplate:sessionStateChanged',
FIELD_COMPLETED: 'dictationTemplate:fieldCompleted',
SESSION_COMPLETED: 'dictationTemplate:sessionCompleted',
},
// ── Phase 13: Voice Conversation (13.1) ──
VOICE_CONVERSATION: {
START_SESSION: 'voiceConversation:startSession',
STOP_SESSION: 'voiceConversation:stopSession',
SEND_MESSAGE: 'voiceConversation:sendMessage',
GET_STATE: 'voiceConversation:getState',
GET_HISTORY: 'voiceConversation:getHistory',
CLEAR_HISTORY: 'voiceConversation:clearHistory',
CANCEL_RESPONSE: 'voiceConversation:cancelResponse',
// Main → Renderer events
STATE_CHANGED: 'voiceConversation:stateChanged',
USER_MESSAGE: 'voiceConversation:userMessage',
ASSISTANT_DELTA: 'voiceConversation:assistantDelta',
ASSISTANT_MESSAGE: 'voiceConversation:assistantMessage',
TTS_STARTED: 'voiceConversation:ttsStarted',
TTS_FINISHED: 'voiceConversation:ttsFinished',
ERROR: 'voiceConversation:error',
},
// ── Phase 13: Local RAG (13.2) ──
RAG: {
ADD_DOCUMENT: 'rag:addDocument',
REMOVE_DOCUMENT: 'rag:removeDocument',
GET_DOCUMENTS: 'rag:getDocuments',
QUERY: 'rag:query',
GET_STATE: 'rag:getState',
REINDEX: 'rag:reindex',
// Main → Renderer events
INDEX_PROGRESS: 'rag:indexProgress',
INDEX_COMPLETE: 'rag:indexComplete',
QUERY_RESULT: 'rag:queryResult',
},
// ── Phase 13: Voice Action / OS Automation (13.3) ──
VOICE_ACTION: {
EXECUTE: 'voiceAction:execute',
GET_PRESETS: 'voiceAction:getPresets',
GET_HISTORY: 'voiceAction:getHistory',
CLEAR_HISTORY: 'voiceAction:clearHistory',
SET_ENABLED: 'voiceAction:setEnabled',
IS_ENABLED: 'voiceAction:isEnabled',
// Main → Renderer events
ACTION_PLANNED: 'voiceAction:actionPlanned',
ACTION_EXECUTED: 'voiceAction:actionExecuted',
ACTION_ERROR: 'voiceAction:actionError',
},
// ── Phase 11: License & Monetization ── // ── Phase 11: License & Monetization ──
LICENSE: { LICENSE: {
GET_INFO: 'license:getInfo', GET_INFO: 'license:getInfo',

View file

@ -432,7 +432,7 @@ export interface HistoryEntry {
focusedApp: string | null focusedApp: string | null
focusedAppName: string | null focusedAppName: string | null
focusedAppWindowTitle: string | null focusedAppWindowTitle: string | null
mode: 'dictation' | 'translate' | 'command' mode: 'dictation' | 'translate' | 'command' | 'caption' | 'file-transcription'
status: 'completed' | 'cancelled' | 'error' status: 'completed' | 'cancelled' | 'error'
errorCode: string | null errorCode: string | null
audioLocalPath: string | null audioLocalPath: string | null
@ -447,6 +447,8 @@ export interface HistoryEntry {
createdAt: number createdAt: number
updatedAt: number updatedAt: number
appVersion: string appVersion: string
/** Phase 12.2: 회의록 요약 마크다운 */
summaryText: string | null
} }
export interface HistoryQueryParams { export interface HistoryQueryParams {
@ -947,3 +949,316 @@ export interface TierComparison {
pro: boolean | string pro: boolean | string
proPlus: boolean | string proPlus: boolean | string
} }
// ============================================================
// Phase 12: File Transcription (12.1)
// ============================================================
export type FileTranscriptionState = 'idle' | 'converting' | 'transcribing' | 'completed' | 'error'
export interface FileTranscriptionStartParams {
filePath: string
language?: string
}
export interface FileTranscriptionProgress {
jobId: string
currentChunk: number
totalChunks: number
percent: number
currentText: string
}
export interface FileTranscriptionSegment {
text: string
start: number
end: number
confidence: number
}
export interface FileTranscriptionResult {
jobId: string
filePath: string
fileName: string
fullText: string
segments: FileTranscriptionSegment[]
totalDurationSec: number
processingTimeMs: number
}
export interface FileTranscriptionStateInfo {
state: FileTranscriptionState
jobId: string | null
progress: FileTranscriptionProgress | null
}
// ============================================================
// Phase 12: Meeting Summary (12.2)
// ============================================================
export interface MeetingSummaryResult {
historyId: string
summary: string
decisions: string[]
actionItems: string[]
rawMarkdown: string
generatedAt: number
}
export interface MeetingSummarizeParams {
historyId: string
}
export interface MeetingSummaryGetParams {
historyId: string
}
export interface MeetingSummaryExportParams {
historyId: string
}
export interface MeetingSummaryProgress {
historyId: string
status: 'generating' | 'done' | 'error'
}
// ============================================================
// Phase 12: Dictation Templates (12.3)
// ============================================================
export interface TemplateField {
id: string
name: string
label: string
promptText: string
required: boolean
maxDurationSec: number
}
export interface DictationTemplate {
id: string
name: string
description: string
fields: TemplateField[]
outputFormat: string
isBuiltin: boolean
createdAt: number
updatedAt: number
}
export interface CreateTemplateParams {
name: string
description: string
fields: TemplateField[]
outputFormat: string
}
export interface UpdateTemplateParams {
id: string
name?: string
description?: string
fields?: TemplateField[]
outputFormat?: string
}
export interface DeleteTemplateParams {
id: string
}
export type TemplateSessionState =
| 'idle'
| 'field-prompting'
| 'field-recording'
| 'completing'
export interface TemplateSessionInfo {
templateId: string
templateName: string
state: TemplateSessionState
currentFieldIndex: number
totalFields: number
currentField: TemplateField | null
fieldValues: Record<string, string>
}
export interface StartTemplateSessionParams {
templateId: string
}
export interface SetFieldValueParams {
fieldId: string
value: string
}
export interface TemplateFieldCompletedEvent {
fieldId: string
fieldName: string
value: string
nextField: TemplateField | null
}
export interface TemplateSessionCompletedEvent {
templateId: string
outputText: string
fieldValues: Record<string, string>
}
// ============================================================
// Phase 13: Voice Conversation (13.1)
// ============================================================
export type ConversationState =
| 'idle'
| 'listening'
| 'thinking'
| 'speaking'
export type ConversationRole = 'user' | 'assistant' | 'system'
export interface ConversationMessage {
id: string
role: ConversationRole
content: string
timestamp: number
}
export interface ConversationSessionInfo {
state: ConversationState
messages: ConversationMessage[]
isActive: boolean
}
export interface ConversationSendParams {
text: string
}
export interface ConversationAssistantDelta {
messageId: string
delta: string
accumulated: string
}
export interface ConversationAssistantMessage {
messageId: string
content: string
}
export interface ConversationError {
message: string
phase: 'stt' | 'llm' | 'tts'
}
// ============================================================
// Phase 13: Local RAG (13.2)
// ============================================================
export interface RAGDocument {
id: string
fileName: string
filePath: string
fileType: 'txt' | 'md' | 'pdf' | 'docx'
chunkCount: number
indexed: boolean
indexedAt: number | null
addedAt: number
}
export interface RAGChunk {
id: string
documentId: string
content: string
embedding: number[]
chunkIndex: number
}
export interface RAGQueryParams {
query: string
topK?: number
}
export interface RAGQueryResult {
query: string
results: Array<{
documentId: string
fileName: string
content: string
similarity: number
}>
/** LLM 답변 (컨텍스트 주입 후) */
answer: string
}
export interface RAGAddDocumentParams {
filePath: string
}
export interface RAGRemoveDocumentParams {
documentId: string
}
export interface RAGIndexProgress {
documentId: string
fileName: string
currentChunk: number
totalChunks: number
percent: number
}
export type RAGState = 'idle' | 'indexing' | 'querying'
export interface RAGStateInfo {
state: RAGState
documentCount: number
totalChunks: number
}
// ============================================================
// Phase 13: Voice Action / OS Automation (13.3)
// ============================================================
export type VoiceActionType =
| 'open_app'
| 'open_url'
| 'open_file'
| 'keyboard_shortcut'
| 'type_text'
| 'system_command'
export interface VoiceActionPlan {
action: VoiceActionType
target: string
description: string
safe: boolean
}
export interface VoiceActionExecuteParams {
text: string
}
export interface VoiceActionPreset {
keywords: string[]
action: VoiceActionPlan
}
export interface VoiceActionHistoryEntry {
id: string
userText: string
plan: VoiceActionPlan
executed: boolean
timestamp: number
}
export interface VoiceActionPlannedEvent {
plan: VoiceActionPlan
userText: string
}
export interface VoiceActionExecutedEvent {
plan: VoiceActionPlan
success: boolean
}
export interface VoiceActionErrorEvent {
message: string
userText: string
}