diff --git a/CLAUDE.md b/CLAUDE.md index 9f91ea5..93bd17e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -77,11 +77,20 @@ npm run typecheck # tsc --noEmit 6. **녹음 UI**: 9개 웨이브 바, cos 분포 가중치, 100ms 애니메이션 ## 현재 상태 -Phase: 11 완료 (Phase 1~11 전체 완료) -마지막 작업: LicenseService + Feature Gate + LicenseModal + ProBadge + UpgradePrompt + 일일 쿼터 + LemonSqueezy API -다음 작업: Phase 12 랜딩 페이지 & 마케팅 또는 빌드/배포 안정화 +Phase: 13 전체 완료 + 랜딩 페이지 완성 (Phase 1~13 + site/) +마지막 작업: 랜딩 페이지 디자인 통합 + 10개 국어 i18n +다음 작업: 빌드/배포 안정화, E2E 테스트, 또는 새 피처 차단 이슈: @nut-tree-fork/nut-js 포크 사용 +### 랜딩 페이지 (site/) +- Vite + React 19 + TypeScript + Tailwind CSS +- 4가지 디자인 레퍼런스 통합 (KKG 3종 + Meteorological → D3RO 인더스트리얼) +- SSOT: tokens.ts + tailwind.config.js 토큰 중앙 관리 +- 재사용 컴포넌트 8개: Badge/Container/Crosshair/DataPoint/GlowButton/InstrumentCard/Led/SectionHeader +- CSS 이펙트: CRT scanline, crosshair, noise texture, glow button (WebGL 없음) +- i18n 10개 국어: en/ko/ja/zh/es/fr/de/pt/ru/vi (Context+Hook, LanguageSwitcher, CJK line-height 보정) +- GitHub Pages 자동 배포: .github/workflows/deploy-site.yml + ### DS 컴포넌트 현황 (src/renderer/components/ds/) - CrtDisplay.tsx: WebGL CRT 셰이더 (파형+스캔라인+비네팅+글리치) ✅ - InstrumentPanel.tsx: 메탈 섀시 컨테이너 (각인 텍스트, 노이즈) ✅ @@ -98,6 +107,47 @@ Phase: 11 완료 (Phase 1~11 전체 완료) - d3roShadow: 10종 그림자 (chassis/card/inset/button/dialog/tooltip 등) - d3roRadius: 7종 반경 (outer 24px ~ pill 999px) +### Phase 13.2 구현 내용 (로컬 RAG) +- **RAGService**: Ollama nomic-embed-text 임베딩, SQLite에 JSON 벡터 저장, 코사인 유사도 검색 +- **문서 관리**: .txt/.md/.pdf/.docx 지원, 500자 청크 분할 (50자 오버랩), 비동기 인덱싱 +- **벡터 검색**: 쿼리 임베딩 → 전체 청크 코사인 유사도 → topK 결과 → LLM 컨텍스트 주입 +- **KnowledgeBasePage**: 전용 UI (문서 목록, 인덱싱 진행률, 질문 입력, 답변+참조 소스 표시) +- **AppLayout**: 'knowledge' 라우트 + RAG 네비 항목 +- **DB 테이블**: rag_documents + rag_chunks (Phase 13.2 신규) +- **IPC**: RAG 네임스페이스 9채널 +- **ErrorCode**: 870-874 (DocumentNotFound, IndexingFailed, EmbeddingFailed, QueryFailed, UnsupportedFormat) + +### Phase 13.3 구현 내용 (OS 자동화) +- **VoiceActionService**: 음성→LLM이 JSON 액션 플랜 생성→실행, 프리셋 6개 (크롬/메모장/탐색기/볼륨) +- **액션 타입**: open_app, open_url, open_file, keyboard_shortcut, type_text, system_command +- **안전장치**: 위험 명령어 블랙리스트 (rm, del, format, shutdown 등), safe=false 시 차단 +- **IPC**: VOICE_ACTION 네임스페이스 9채널 +- **ErrorCode**: 875-878 (PlanFailed, ExecutionFailed, Blocked, InvalidPlan) + +### Phase 13.1 구현 내용 (음성 대화 모드) +- **VoiceConversationService**: STT→LLM→TTS 대화 루프 오케스트레이션, 세션 관리, 대화 히스토리 최근 10턴 +- **TTSPlaybackService**: Windows SAPI (PowerShell System.Speech) 기반 로컬 TTS, 문장 단위 큐 재생 +- **LocalLLMService.chatStream**: Ollama /api/chat 스트리밍, 대화 히스토리 messages 배열 전달 +- **VoiceConversationPage**: 전용 대화 UI (채팅 메시지 목록, 녹음/정지/취소 버튼, 텍스트 입력) +- **AppLayout**: 'conversation' 라우트 + TALK 네비게이션 항목 추가 +- **IPC**: VOICE_CONVERSATION 네임스페이스 15채널 (startSession, sendMessage, assistantDelta 등) +- **Preload**: voiceConversation API 네임스페이스 (17메서드/이벤트) +- **ErrorCode**: 795-798 (ConversationSessionAlreadyActive, TTSFailed, LLMFailed 등) +- **i18n**: conversation.* 번역 키 추가 (ko/en) + +### Phase 12 구현 내용 (Pro+ 피처) +- **FileTranscriptionService**: ffmpeg(fluent-ffmpeg + @ffmpeg-installer/ffmpeg)로 미디어→PCM 변환, 30초 청크 분할 STT, 진행률 이벤트, 히스토리 자동 저장 (mode: 'file-transcription') +- **MeetingSummaryService**: CaptionService session-saved 이벤트 → Ollama 자동 요약, 마크다운 파싱 (요약/결정/할일), history.summaryText 컬럼 저장, 마크다운 내보내기 +- **DictationTemplateService**: 템플릿 CRUD (electron-store), 세션 상태 머신 (idle→field-prompting→field-recording→completing), 프리셋 3개 (이메일/회의록/보고서), 출력 포맷 mustache 치환 +- **FileDropZone UI**: DashboardPage 드래그앤드롭 존, 진행률/결과/에러 상태 표시 +- **HistoryEntryCard 확장**: 요약 배지(Chip), 확장 뷰(요약/결정/할일), 요약 생성/내보내기 버튼 +- **TemplateSection UI**: CommandsPage 하단 섹션, 템플릿 목록/생성/편집/삭제/세션 시작 +- **DB 마이그레이션**: history 테이블 summary_text 컬럼 추가 (ALTER TABLE), mode enum에 'file-transcription' 추가 +- **IPC**: FILE_TRANSCRIPTION(6채널) + MEETING_SUMMARY(5채널) + DICTATION_TEMPLATE(11채널) = 22개 신규 채널 +- **Preload**: fileTranscription/meetingSummary/dictationTemplate 3개 API 네임스페이스 +- **ErrorCode**: 780-794 범위 (FFmpegFailed, InvalidFormat, ChunkFailed, SummaryGenerationFailed, TemplateNotFound 등) +- **i18n**: fileTranscription.*/meetingSummary.*/template.* 번역 키 추가 (ko/en) + ### Phase 11 구현 내용 (수익화 기반) - **LicenseService**: Free/Pro/Pro+ 3단계 티어, electron-store 대신 별도 JSON 파일, 오프라인 우선 설계 - **Feature Gating**: Feature enum (16개 기능), 티어별 접근 매핑, 쿼터 한도 (Free: 받아쓰기 20회/일, LLM 10회/일) @@ -254,8 +304,10 @@ Phase: 11 완료 (Phase 1~11 전체 완료) - Phase 9: 품질 보강 + UX 개선 (온보딩, 마이크 테스트, WAV 저장, 커맨드 팝업, Ollama 안내) - Phase 10: 킬러 피처 — Speakly를 넘어서 (실시간 자막, 스크린 컨텍스트, 음성 메모장, 멀티 LLM 체인, 음성 단축키) - Phase 11: 수익화 기반 — LicenseService + Feature Gating + Freemium UI -- Phase 12: Pro 피처 — 파일 전사, 회의록 자동 요약, 딕테이션 템플릿 -- Phase 13: Pro+ 프리미엄 — 음성 대화 모드, 로컬 RAG, OS 자동화 +- Phase 12: Pro 피처 — 파일 전사, 회의록 자동 요약, 딕테이션 템플릿 ✅ +- Phase 13.1: Pro+ 프리미엄 — 음성 대화 모드 (STT→LLM→TTS 루프) ✅ +- Phase 13.2: Pro+ 프리미엄 — 로컬 RAG (문서 임베딩+검색) ✅ +- Phase 13.3: Pro+ 프리미엄 — OS 자동화 (음성→JSON 액션) ✅ ## 설계 문서 (구현 시 반드시 참조) @docs/design/00-master-architecture.md diff --git a/package-lock.json b/package-lock.json index 1cbb6c0..5ba8188 100644 --- a/package-lock.json +++ b/package-lock.json @@ -13,6 +13,7 @@ "@electron-toolkit/utils": "^4.0.0", "@emotion/react": "^11.14.0", "@emotion/styled": "^11.14.0", + "@ffmpeg-installer/ffmpeg": "^1.1.0", "@mui/icons-material": "^7.0.0", "@mui/material": "^7.0.0", "@nut-tree-fork/nut-js": "^4.2.6", @@ -22,8 +23,10 @@ "electron-audio-loopback": "^1.0.6", "electron-log": "^5.2.0", "electron-store": "^10.0.0", + "fluent-ffmpeg": "^2.1.3", "nanoid": "^5.1.7", "node-record-lpcm16": "^1.0.1", + "pdf-parse": "^2.4.5", "react": "^19.0.0", "react-dom": "^19.0.0", "uiohook-napi": "^1.5.5" @@ -31,6 +34,7 @@ "devDependencies": { "@electron-toolkit/tsconfig": "^1.0.1", "@types/better-sqlite3": "^7.6.13", + "@types/fluent-ffmpeg": "^2.1.28", "@types/node": "^22.13.0", "@types/react": "^19.0.0", "@types/react-dom": "^19.0.0", @@ -1574,6 +1578,132 @@ "node": "^12.22.0 || ^14.17.0 || >=16.0.0" } }, + "node_modules/@ffmpeg-installer/darwin-arm64": { + "version": "4.1.5", + "resolved": "https://registry.npmjs.org/@ffmpeg-installer/darwin-arm64/-/darwin-arm64-4.1.5.tgz", + "integrity": "sha512-hYqTiP63mXz7wSQfuqfFwfLOfwwFChUedeCVKkBtl/cliaTM7/ePI9bVzfZ2c+dWu3TqCwLDRWNSJ5pqZl8otA==", + "cpu": [ + "arm64" + ], + "hasInstallScript": true, + "license": "https://git.ffmpeg.org/gitweb/ffmpeg.git/blob_plain/HEAD:/LICENSE.md", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@ffmpeg-installer/darwin-x64": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/@ffmpeg-installer/darwin-x64/-/darwin-x64-4.1.0.tgz", + "integrity": "sha512-Z4EyG3cIFjdhlY8wI9aLUXuH8nVt7E9SlMVZtWvSPnm2sm37/yC2CwjUzyCQbJbySnef1tQwGG2Sx+uWhd9IAw==", + "cpu": [ + "x64" + ], + "hasInstallScript": true, + "license": "LGPL-2.1", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@ffmpeg-installer/ffmpeg": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/@ffmpeg-installer/ffmpeg/-/ffmpeg-1.1.0.tgz", + "integrity": "sha512-Uq4rmwkdGxIa9A6Bd/VqqYbT7zqh1GrT5/rFwCwKM70b42W5gIjWeVETq6SdcL0zXqDtY081Ws/iJWhr1+xvQg==", + "license": "LGPL-2.1", + "optionalDependencies": { + "@ffmpeg-installer/darwin-arm64": "4.1.5", + "@ffmpeg-installer/darwin-x64": "4.1.0", + "@ffmpeg-installer/linux-arm": "4.1.3", + "@ffmpeg-installer/linux-arm64": "4.1.4", + "@ffmpeg-installer/linux-ia32": "4.1.0", + "@ffmpeg-installer/linux-x64": "4.1.0", + "@ffmpeg-installer/win32-ia32": "4.1.0", + "@ffmpeg-installer/win32-x64": "4.1.0" + } + }, + "node_modules/@ffmpeg-installer/linux-arm": { + "version": "4.1.3", + "resolved": "https://registry.npmjs.org/@ffmpeg-installer/linux-arm/-/linux-arm-4.1.3.tgz", + "integrity": "sha512-NDf5V6l8AfzZ8WzUGZ5mV8O/xMzRag2ETR6+TlGIsMHp81agx51cqpPItXPib/nAZYmo55Bl2L6/WOMI3A5YRg==", + "cpu": [ + "arm" + ], + "hasInstallScript": true, + "license": "GPLv3", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@ffmpeg-installer/linux-arm64": { + "version": "4.1.4", + "resolved": "https://registry.npmjs.org/@ffmpeg-installer/linux-arm64/-/linux-arm64-4.1.4.tgz", + "integrity": "sha512-dljEqAOD0oIM6O6DxBW9US/FkvqvQwgJ2lGHOwHDDwu/pX8+V0YsDL1xqHbj1DMX/+nP9rxw7G7gcUvGspSoKg==", + "cpu": [ + "arm64" + ], + "hasInstallScript": true, + "license": "GPLv3", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@ffmpeg-installer/linux-ia32": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/@ffmpeg-installer/linux-ia32/-/linux-ia32-4.1.0.tgz", + "integrity": "sha512-0LWyFQnPf+Ij9GQGD034hS6A90URNu9HCtQ5cTqo5MxOEc7Rd8gLXrJvn++UmxhU0J5RyRE9KRYstdCVUjkNOQ==", + "cpu": [ + "ia32" + ], + "hasInstallScript": true, + "license": "GPLv3", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@ffmpeg-installer/linux-x64": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/@ffmpeg-installer/linux-x64/-/linux-x64-4.1.0.tgz", + "integrity": "sha512-Y5BWhGLU/WpQjOArNIgXD3z5mxxdV8c41C+U15nsE5yF8tVcdCGet5zPs5Zy3Ta6bU7haGpIzryutqCGQA/W8A==", + "cpu": [ + "x64" + ], + "hasInstallScript": true, + "license": "GPLv3", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@ffmpeg-installer/win32-ia32": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/@ffmpeg-installer/win32-ia32/-/win32-ia32-4.1.0.tgz", + "integrity": "sha512-FV2D7RlaZv/lrtdhaQ4oETwoFUsUjlUiasiZLDxhEUPdNDWcH1OU9K1xTvqz+OXLdsmYelUDuBS/zkMOTtlUAw==", + "cpu": [ + "ia32" + ], + "license": "GPLv3", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@ffmpeg-installer/win32-x64": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/@ffmpeg-installer/win32-x64/-/win32-x64-4.1.0.tgz", + "integrity": "sha512-Drt5u2vzDnIONf4ZEkKtFlbvwj6rI3kxw1Ck9fpudmtgaZIHD4ucsWB2lCZBXRxJgXR+2IMSti+4rtM4C4rXgg==", + "cpu": [ + "x64" + ], + "license": "GPLv3", + "optional": true, + "os": [ + "win32" + ] + }, "node_modules/@humanwhocodes/config-array": { "version": "0.13.0", "resolved": "https://registry.npmjs.org/@humanwhocodes/config-array/-/config-array-0.13.0.tgz", @@ -2535,6 +2665,190 @@ } } }, + "node_modules/@napi-rs/canvas": { + "version": "0.1.80", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas/-/canvas-0.1.80.tgz", + "integrity": "sha512-DxuT1ClnIPts1kQx8FBmkk4BQDTfI5kIzywAaMjQSXfNnra5UFU9PwurXrl+Je3bJ6BGsp/zmshVVFbCmyI+ww==", + "license": "MIT", + "workspaces": [ + "e2e/*" + ], + "engines": { + "node": ">= 10" + }, + "optionalDependencies": { + "@napi-rs/canvas-android-arm64": "0.1.80", + "@napi-rs/canvas-darwin-arm64": "0.1.80", + "@napi-rs/canvas-darwin-x64": "0.1.80", + "@napi-rs/canvas-linux-arm-gnueabihf": "0.1.80", + "@napi-rs/canvas-linux-arm64-gnu": "0.1.80", + "@napi-rs/canvas-linux-arm64-musl": "0.1.80", + "@napi-rs/canvas-linux-riscv64-gnu": "0.1.80", + "@napi-rs/canvas-linux-x64-gnu": "0.1.80", + "@napi-rs/canvas-linux-x64-musl": "0.1.80", + "@napi-rs/canvas-win32-x64-msvc": "0.1.80" + } + }, + "node_modules/@napi-rs/canvas-android-arm64": { + "version": "0.1.80", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-android-arm64/-/canvas-android-arm64-0.1.80.tgz", + "integrity": "sha512-sk7xhN/MoXeuExlggf91pNziBxLPVUqF2CAVnB57KLG/pz7+U5TKG8eXdc3pm0d7Od0WreB6ZKLj37sX9muGOQ==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/canvas-darwin-arm64": { + "version": "0.1.80", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-darwin-arm64/-/canvas-darwin-arm64-0.1.80.tgz", + "integrity": "sha512-O64APRTXRUiAz0P8gErkfEr3lipLJgM6pjATwavZ22ebhjYl/SUbpgM0xcWPQBNMP1n29afAC/Us5PX1vg+JNQ==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/canvas-darwin-x64": { + "version": "0.1.80", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-darwin-x64/-/canvas-darwin-x64-0.1.80.tgz", + "integrity": "sha512-FqqSU7qFce0Cp3pwnTjVkKjjOtxMqRe6lmINxpIZYaZNnVI0H5FtsaraZJ36SiTHNjZlUB69/HhxNDT1Aaa9vA==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/canvas-linux-arm-gnueabihf": { + "version": "0.1.80", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-arm-gnueabihf/-/canvas-linux-arm-gnueabihf-0.1.80.tgz", + "integrity": "sha512-eyWz0ddBDQc7/JbAtY4OtZ5SpK8tR4JsCYEZjCE3dI8pqoWUC8oMwYSBGCYfsx2w47cQgQCgMVRVTFiiO38hHQ==", + "cpu": [ + "arm" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/canvas-linux-arm64-gnu": { + "version": "0.1.80", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-arm64-gnu/-/canvas-linux-arm64-gnu-0.1.80.tgz", + "integrity": "sha512-qwA63t8A86bnxhuA/GwOkK3jvb+XTQaTiVML0vAWoHyoZYTjNs7BzoOONDgTnNtr8/yHrq64XXzUoLqDzU+Uuw==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/canvas-linux-arm64-musl": { + "version": "0.1.80", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-arm64-musl/-/canvas-linux-arm64-musl-0.1.80.tgz", + "integrity": "sha512-1XbCOz/ymhj24lFaIXtWnwv/6eFHXDrjP0jYkc6iHQ9q8oXKzUX1Lc6bu+wuGiLhGh2GS/2JlfORC5ZcXimRcg==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/canvas-linux-riscv64-gnu": { + "version": "0.1.80", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-riscv64-gnu/-/canvas-linux-riscv64-gnu-0.1.80.tgz", + "integrity": "sha512-XTzR125w5ZMs0lJcxRlS1K3P5RaZ9RmUsPtd1uGt+EfDyYMu4c6SEROYsxyatbbu/2+lPe7MPHOO/0a0x7L/gw==", + "cpu": [ + "riscv64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/canvas-linux-x64-gnu": { + "version": "0.1.80", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-x64-gnu/-/canvas-linux-x64-gnu-0.1.80.tgz", + "integrity": "sha512-BeXAmhKg1kX3UCrJsYbdQd3hIMDH/K6HnP/pG2LuITaXhXBiNdh//TVVVVCBbJzVQaV5gK/4ZOCMrQW9mvuTqA==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/canvas-linux-x64-musl": { + "version": "0.1.80", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-x64-musl/-/canvas-linux-x64-musl-0.1.80.tgz", + "integrity": "sha512-x0XvZWdHbkgdgucJsRxprX/4o4sEed7qo9rCQA9ugiS9qE2QvP0RIiEugtZhfLH3cyI+jIRFJHV4Fuz+1BHHMg==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/canvas-win32-x64-msvc": { + "version": "0.1.80", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-win32-x64-msvc/-/canvas-win32-x64-msvc-0.1.80.tgz", + "integrity": "sha512-Z8jPsM6df5V8B1HrCHB05+bDiCxjE9QA//3YrkKIdVDEwn5RKaqOxCJDRJkl48cJbylcrJbW4HxZbTte8juuPg==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 10" + } + }, "node_modules/@nodelib/fs.scandir": { "version": "2.1.5", "resolved": "https://registry.npmjs.org/@nodelib/fs.scandir/-/fs.scandir-2.1.5.tgz", @@ -3254,6 +3568,16 @@ "dev": true, "license": "MIT" }, + "node_modules/@types/fluent-ffmpeg": { + "version": "2.1.28", + "resolved": "https://registry.npmjs.org/@types/fluent-ffmpeg/-/fluent-ffmpeg-2.1.28.tgz", + "integrity": "sha512-5ovxsDwBcPfJ+eYs1I/ZpcYCnkce7pvH9AHSvrZllAp1ZPpTRDZAFjF3TRFbukxSgIYTTNYePbS0rKUmaxVbXw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/node": "*" + } + }, "node_modules/@types/fs-extra": { "version": "9.0.13", "resolved": "https://registry.npmjs.org/@types/fs-extra/-/fs-extra-9.0.13.tgz", @@ -6606,6 +6930,37 @@ "dev": true, "license": "ISC" }, + "node_modules/fluent-ffmpeg": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/fluent-ffmpeg/-/fluent-ffmpeg-2.1.3.tgz", + "integrity": "sha512-Be3narBNt2s6bsaqP6Jzq91heDgOEaDCJAXcE3qcma/EJBSy5FB4cvO31XBInuAuKBx8Kptf8dkhjK0IOru39Q==", + "deprecated": "Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.", + "license": "MIT", + "dependencies": { + "async": "^0.2.9", + "which": "^1.1.1" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/fluent-ffmpeg/node_modules/async": { + "version": "0.2.10", + "resolved": "https://registry.npmjs.org/async/-/async-0.2.10.tgz", + "integrity": "sha512-eAkdoKxU6/LkKDBzLpT+t6Ff5EtfSF4wx1WfJiPEEV7WNLnDaRXk0oVysiEPm262roaachGexwUv94WhSgN5TQ==" + }, + "node_modules/fluent-ffmpeg/node_modules/which": { + "version": "1.3.1", + "resolved": "https://registry.npmjs.org/which/-/which-1.3.1.tgz", + "integrity": "sha512-HxJdYWq1MTIQbJ3nw0cqssHoTNU267KlrDuGZ1WYlxDStUtKUhOaJmh112/TZmHxxUfuJqPXSOm7tDyas0OSIQ==", + "license": "ISC", + "dependencies": { + "isexe": "^2.0.0" + }, + "bin": { + "which": "bin/which" + } + }, "node_modules/follow-redirects": { "version": "1.15.11", "resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.11.tgz", @@ -8660,6 +9015,38 @@ "node": ">= 14.16" } }, + "node_modules/pdf-parse": { + "version": "2.4.5", + "resolved": "https://registry.npmjs.org/pdf-parse/-/pdf-parse-2.4.5.tgz", + "integrity": "sha512-mHU89HGh7v+4u2ubfnevJ03lmPgQ5WU4CxAVmTSh/sxVTEDYd1er/dKS/A6vg77NX47KTEoihq8jZBLr8Cxuwg==", + "license": "Apache-2.0", + "dependencies": { + "@napi-rs/canvas": "0.1.80", + "pdfjs-dist": "5.4.296" + }, + "bin": { + "pdf-parse": "bin/cli.mjs" + }, + "engines": { + "node": ">=20.16.0 <21 || >=22.3.0" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/mehmet-kozan" + } + }, + "node_modules/pdfjs-dist": { + "version": "5.4.296", + "resolved": "https://registry.npmjs.org/pdfjs-dist/-/pdfjs-dist-5.4.296.tgz", + "integrity": "sha512-DlOzet0HO7OEnmUmB6wWGJrrdvbyJKftI1bhMitK7O2N8W2gc757yyYBbINy9IDafXAV9wmKr9t7xsTaNKRG5Q==", + "license": "Apache-2.0", + "engines": { + "node": ">=20.16.0 || >=22.3.0" + }, + "optionalDependencies": { + "@napi-rs/canvas": "^0.1.80" + } + }, "node_modules/pe-library": { "version": "0.4.1", "resolved": "https://registry.npmjs.org/pe-library/-/pe-library-0.4.1.tgz", diff --git a/package.json b/package.json index 9a8eda7..8ad7dd4 100644 --- a/package.json +++ b/package.json @@ -27,6 +27,7 @@ "devDependencies": { "@electron-toolkit/tsconfig": "^1.0.1", "@types/better-sqlite3": "^7.6.13", + "@types/fluent-ffmpeg": "^2.1.28", "@types/node": "^22.13.0", "@types/react": "^19.0.0", "@types/react-dom": "^19.0.0", @@ -48,6 +49,7 @@ "@electron-toolkit/utils": "^4.0.0", "@emotion/react": "^11.14.0", "@emotion/styled": "^11.14.0", + "@ffmpeg-installer/ffmpeg": "^1.1.0", "@mui/icons-material": "^7.0.0", "@mui/material": "^7.0.0", "@nut-tree-fork/nut-js": "^4.2.6", @@ -57,8 +59,10 @@ "electron-audio-loopback": "^1.0.6", "electron-log": "^5.2.0", "electron-store": "^10.0.0", + "fluent-ffmpeg": "^2.1.3", "nanoid": "^5.1.7", "node-record-lpcm16": "^1.0.1", + "pdf-parse": "^2.4.5", "react": "^19.0.0", "react-dom": "^19.0.0", "uiohook-napi": "^1.5.5" diff --git a/src/main/bootstrap.ts b/src/main/bootstrap.ts index f82d408..f09ba3e 100644 --- a/src/main/bootstrap.ts +++ b/src/main/bootstrap.ts @@ -56,7 +56,8 @@ export async function bootstrap(): Promise { { name: 'popup-preload', critical: false, fn: initPopupWindows }, { name: 'hotkey', critical: false, fn: initHotkey }, { name: 'voice-mode', critical: false, fn: initVoiceMode }, - { name: 'llm-polling', critical: false, fn: initLLMPolling } + { name: 'llm-polling', critical: false, fn: initLLMPolling }, + { name: 'meeting-summary-wiring', critical: false, fn: initMeetingSummaryWiring }, ] for (const step of steps) { @@ -264,6 +265,23 @@ async function initLLMPolling(): Promise { llm.startPolling() } +async function initMeetingSummaryWiring(): Promise { + try { + const { getCaptionService } = await import('./services/CaptionService') + const { getMeetingSummaryService } = await import('./services/MeetingSummaryService') + const captionService = getCaptionService() + const summaryService = getMeetingSummaryService() + + captionService.on('session-saved', (summary: { sessionId: string }) => { + summaryService.onCaptionSessionSaved(summary).catch((err) => { + logger.warn('Auto meeting summary failed:', err) + }) + }) + } catch (err) { + logger.warn('Meeting summary wiring failed:', err) + } +} + // ── HistoryPopup IPC 연동 ──────────────────────────── function setupHistoryPopupIPC(): void { diff --git a/src/main/db/index.ts b/src/main/db/index.ts index 520f20b..807ee18 100644 --- a/src/main/db/index.ts +++ b/src/main/db/index.ts @@ -105,6 +105,42 @@ export function initDatabase(): BetterSQLite3Database { CREATE INDEX IF NOT EXISTS idx_daily_usage_date ON daily_usage(date); `) + // Phase 13.2: RAG 테이블 생성 + sqlite.exec(` + CREATE TABLE IF NOT EXISTS rag_documents ( + id TEXT PRIMARY KEY, + file_name TEXT NOT NULL, + file_path TEXT NOT NULL, + file_type TEXT NOT NULL, + chunk_count INTEGER NOT NULL DEFAULT 0, + indexed INTEGER NOT NULL DEFAULT 0, + indexed_at INTEGER, + added_at INTEGER NOT NULL + ); + CREATE INDEX IF NOT EXISTS idx_rag_documents_added_at ON rag_documents(added_at); + + CREATE TABLE IF NOT EXISTS rag_chunks ( + id TEXT PRIMARY KEY, + document_id TEXT NOT NULL, + content TEXT NOT NULL, + embedding TEXT NOT NULL, + chunk_index INTEGER NOT NULL + ); + CREATE INDEX IF NOT EXISTS idx_rag_chunks_document_id ON rag_chunks(document_id); + `) + + // Phase 12.2: history 테이블에 summary_text 컬럼 마이그레이션 + try { + const columns = sqlite.pragma('table_info(history)') as Array<{ name: string }> + const hasSummaryText = columns.some((c) => c.name === 'summary_text') + if (!hasSummaryText) { + sqlite.exec('ALTER TABLE history ADD COLUMN summary_text TEXT') + logger.info('Migrated: added summary_text column to history') + } + } catch (err) { + logger.warn('summary_text migration check failed:', err) + } + db = drizzle(sqlite, { schema }) logger.info('Database initialized') diff --git a/src/main/db/schema.ts b/src/main/db/schema.ts index d5ae02a..83226fe 100644 --- a/src/main/db/schema.ts +++ b/src/main/db/schema.ts @@ -13,7 +13,7 @@ export const history = sqliteTable( focusedApp: text('focused_app'), focusedAppName: text('focused_app_name'), focusedAppWindowTitle: text('focused_app_window_title'), - mode: text('mode', { enum: ['dictation', 'translate', 'command', 'caption'] }) + mode: text('mode', { enum: ['dictation', 'translate', 'command', 'caption', 'file-transcription'] }) .notNull() .default('dictation'), status: text('status', { enum: ['completed', 'cancelled', 'error'] }) @@ -31,7 +31,9 @@ export const history = sqliteTable( llmLatencyMs: integer('llm_latency_ms'), createdAt: integer('created_at').notNull(), updatedAt: integer('updated_at').notNull(), - appVersion: text('app_version').notNull().default('1.0.0') + appVersion: text('app_version').notNull().default('1.0.0'), + /** Phase 12.2: 회의록 자동 요약 텍스트 (마크다운) */ + summaryText: text('summary_text'), }, (table) => [ index('idx_history_created_at').on(table.createdAt), @@ -112,6 +114,44 @@ export const dailyUsage = sqliteTable( export type DailyUsageRow = typeof dailyUsage.$inferSelect export type NewDailyUsageRow = typeof dailyUsage.$inferInsert +// ── rag_documents (Phase 13.2) ────────────────────────── +export const ragDocuments = sqliteTable( + 'rag_documents', + { + id: text('id').primaryKey(), + fileName: text('file_name').notNull(), + filePath: text('file_path').notNull(), + fileType: text('file_type', { enum: ['txt', 'md', 'pdf', 'docx'] }).notNull(), + chunkCount: integer('chunk_count').notNull().default(0), + indexed: integer('indexed', { mode: 'boolean' }).notNull().default(false), + indexedAt: integer('indexed_at'), + addedAt: integer('added_at').notNull(), + }, + (table) => [ + index('idx_rag_documents_added_at').on(table.addedAt), + ] +) + +export const ragChunks = sqliteTable( + 'rag_chunks', + { + id: text('id').primaryKey(), + documentId: text('document_id').notNull(), + content: text('content').notNull(), + /** 임베딩 벡터 (JSON 직렬화 float[]) */ + embedding: text('embedding').notNull(), + chunkIndex: integer('chunk_index').notNull(), + }, + (table) => [ + index('idx_rag_chunks_document_id').on(table.documentId), + ] +) + +export type RAGDocumentRow = typeof ragDocuments.$inferSelect +export type NewRAGDocumentRow = typeof ragDocuments.$inferInsert +export type RAGChunkRow = typeof ragChunks.$inferSelect +export type NewRAGChunkRow = typeof ragChunks.$inferInsert + // ── 타입 추출 ──────────────────────────────────────────── export type History = typeof history.$inferSelect export type NewHistory = typeof history.$inferInsert diff --git a/src/main/ipc/file-transcription-handlers.ts b/src/main/ipc/file-transcription-handlers.ts new file mode 100644 index 0000000..56fc8de --- /dev/null +++ b/src/main/ipc/file-transcription-handlers.ts @@ -0,0 +1,69 @@ +// src/main/ipc/file-transcription-handlers.ts +// Phase 12.1: 파일 전사 IPC 핸들러 + +import { ipcMain, dialog } from 'electron' +import { IPC_CHANNELS } from '@shared/ipc-channels' +import { ErrorCode, ipcSuccess, ipcError } from '@shared/errors' +import { getFileTranscriptionService } from '../services/FileTranscriptionService' +import { getLogger } from '../services/LoggerService' +import type { FileTranscriptionStartParams } from '@shared/types' + +const logger = getLogger('file-transcription-handlers') + +export function registerFileTranscriptionHandlers(): void { + ipcMain.handle( + IPC_CHANNELS.FILE_TRANSCRIPTION.START, + async (_event, params: FileTranscriptionStartParams) => { + try { + let filePath = params.filePath + + // filePath가 없으면 파일 선택 다이얼로그 + if (!filePath) { + const result = await dialog.showOpenDialog({ + properties: ['openFile'], + filters: [ + { + name: 'Audio/Video', + extensions: [ + 'mp3', 'wav', 'm4a', 'ogg', 'flac', 'wma', 'aac', + 'mp4', 'mkv', 'webm', 'avi', 'mov', + ], + }, + ], + }) + if (result.canceled || result.filePaths.length === 0) { + return ipcError(ErrorCode.FileTranscriptionCancelled, 'File selection cancelled') + } + filePath = result.filePaths[0] + } + + const service = getFileTranscriptionService() + const resultData = await service.startTranscription(filePath, params.language) + return ipcSuccess(resultData) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + logger.error('File transcription failed:', msg) + return ipcError(ErrorCode.FileTranscriptionChunkFailed, msg) + } + }, + ) + + ipcMain.handle(IPC_CHANNELS.FILE_TRANSCRIPTION.CANCEL, async () => { + try { + getFileTranscriptionService().cancel() + return ipcSuccess(undefined) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.FileTranscriptionCancelled, msg) + } + }) + + ipcMain.handle(IPC_CHANNELS.FILE_TRANSCRIPTION.GET_STATE, async () => { + try { + return ipcSuccess(getFileTranscriptionService().getStateInfo()) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.UnknownError, msg) + } + }) +} diff --git a/src/main/ipc/index.ts b/src/main/ipc/index.ts index 768d87d..3d6a316 100644 --- a/src/main/ipc/index.ts +++ b/src/main/ipc/index.ts @@ -17,6 +17,12 @@ import { registerContextHandlers } from './context-handlers' import { registerCaptionHandlers } from './caption-handlers' import { registerChainHandlers } from './chain-handlers' import { registerLicenseHandlers } from './license-handlers' +import { registerFileTranscriptionHandlers } from './file-transcription-handlers' +import { registerMeetingSummaryHandlers } from './meeting-summary-handlers' +import { registerTemplateHandlers } from './template-handlers' +import { registerVoiceConversationHandlers } from './voice-conversation-handlers' +import { registerRAGHandlers } from './rag-handlers' +import { registerVoiceActionHandlers } from './voice-action-handlers' import { getLogger } from '../services/LoggerService' const logger = getLogger('ipc') @@ -39,5 +45,11 @@ export function registerAllIpcHandlers(): void { registerCaptionHandlers() registerChainHandlers() registerLicenseHandlers() + registerFileTranscriptionHandlers() + registerMeetingSummaryHandlers() + registerTemplateHandlers() + registerVoiceConversationHandlers() + registerRAGHandlers() + registerVoiceActionHandlers() logger.info('All IPC handlers registered') } diff --git a/src/main/ipc/meeting-summary-handlers.ts b/src/main/ipc/meeting-summary-handlers.ts new file mode 100644 index 0000000..35f0113 --- /dev/null +++ b/src/main/ipc/meeting-summary-handlers.ts @@ -0,0 +1,54 @@ +// src/main/ipc/meeting-summary-handlers.ts +// Phase 12.2: 회의록 요약 IPC 핸들러 + +import { ipcMain } from 'electron' +import { IPC_CHANNELS } from '@shared/ipc-channels' +import { ErrorCode, ipcSuccess, ipcError } from '@shared/errors' +import { getMeetingSummaryService } from '../services/MeetingSummaryService' +import { getLogger } from '../services/LoggerService' +import type { MeetingSummarizeParams, MeetingSummaryGetParams, MeetingSummaryExportParams } from '@shared/types' + +const logger = getLogger('meeting-summary-handlers') + +export function registerMeetingSummaryHandlers(): void { + ipcMain.handle( + IPC_CHANNELS.MEETING_SUMMARY.SUMMARIZE, + async (_event, params: MeetingSummarizeParams) => { + try { + const result = await getMeetingSummaryService().summarize(params.historyId) + return ipcSuccess(result) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + logger.error('Meeting summary failed:', msg) + return ipcError(ErrorCode.MeetingSummaryGenerationFailed, msg) + } + }, + ) + + ipcMain.handle( + IPC_CHANNELS.MEETING_SUMMARY.GET_SUMMARY, + async (_event, params: MeetingSummaryGetParams) => { + try { + const result = getMeetingSummaryService().getSummary(params.historyId) + return ipcSuccess(result) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.MeetingSummaryGenerationFailed, msg) + } + }, + ) + + ipcMain.handle( + IPC_CHANNELS.MEETING_SUMMARY.EXPORT_MARKDOWN, + async (_event, params: MeetingSummaryExportParams) => { + try { + const filePath = await getMeetingSummaryService().exportMarkdown(params.historyId) + return ipcSuccess(filePath) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + logger.error('Meeting summary export failed:', msg) + return ipcError(ErrorCode.MeetingSummaryExportFailed, msg) + } + }, + ) +} diff --git a/src/main/ipc/rag-handlers.ts b/src/main/ipc/rag-handlers.ts new file mode 100644 index 0000000..2478ed1 --- /dev/null +++ b/src/main/ipc/rag-handlers.ts @@ -0,0 +1,84 @@ +// src/main/ipc/rag-handlers.ts +// Phase 13.2: 로컬 RAG IPC 핸들러 + +import { ipcMain, dialog } from 'electron' +import { IPC_CHANNELS } from '@shared/ipc-channels' +import { ErrorCode, ipcSuccess, ipcError } from '@shared/errors' +import { getRAGService } from '../services/RAGService' +import { getLogger } from '../services/LoggerService' +import type { RAGQueryParams, RAGRemoveDocumentParams } from '@shared/types' + +const logger = getLogger('rag-handlers') + +export function registerRAGHandlers(): void { + ipcMain.handle(IPC_CHANNELS.RAG.ADD_DOCUMENT, async (_event, params: { filePath?: string }) => { + try { + let filePath = params?.filePath + if (!filePath) { + const result = await dialog.showOpenDialog({ + properties: ['openFile'], + filters: [{ name: 'Documents', extensions: ['txt', 'md', 'pdf', 'docx'] }], + }) + if (result.canceled || result.filePaths.length === 0) { + return ipcError(ErrorCode.RAGDocumentNotFound, 'File selection cancelled') + } + filePath = result.filePaths[0] + } + const doc = await getRAGService().addDocument(filePath) + return ipcSuccess(doc) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + logger.error('RAG add document failed:', msg) + return ipcError(ErrorCode.RAGIndexingFailed, msg) + } + }) + + ipcMain.handle(IPC_CHANNELS.RAG.REMOVE_DOCUMENT, async (_event, params: RAGRemoveDocumentParams) => { + try { + getRAGService().removeDocument(params.documentId) + return ipcSuccess(undefined) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.RAGDocumentNotFound, msg) + } + }) + + ipcMain.handle(IPC_CHANNELS.RAG.GET_DOCUMENTS, async () => { + try { + return ipcSuccess(getRAGService().getDocuments()) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.UnknownError, msg) + } + }) + + ipcMain.handle(IPC_CHANNELS.RAG.QUERY, async (_event, params: RAGQueryParams) => { + try { + const result = await getRAGService().query(params.query, params.topK) + return ipcSuccess(result) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + logger.error('RAG query failed:', msg) + return ipcError(ErrorCode.RAGQueryFailed, msg) + } + }) + + ipcMain.handle(IPC_CHANNELS.RAG.GET_STATE, async () => { + try { + return ipcSuccess(getRAGService().getStateInfo()) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.UnknownError, msg) + } + }) + + ipcMain.handle(IPC_CHANNELS.RAG.REINDEX, async (_event, params: { documentId: string }) => { + try { + await getRAGService().reindex(params.documentId) + return ipcSuccess(undefined) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.RAGIndexingFailed, msg) + } + }) +} diff --git a/src/main/ipc/template-handlers.ts b/src/main/ipc/template-handlers.ts new file mode 100644 index 0000000..87a9132 --- /dev/null +++ b/src/main/ipc/template-handlers.ts @@ -0,0 +1,128 @@ +// src/main/ipc/template-handlers.ts +// Phase 12.3: 딕테이션 템플릿 IPC 핸들러 + +import { ipcMain } from 'electron' +import { IPC_CHANNELS } from '@shared/ipc-channels' +import { ErrorCode, ipcSuccess, ipcError } from '@shared/errors' +import { getDictationTemplateService } from '../services/DictationTemplateService' +import { getLogger } from '../services/LoggerService' +import type { + CreateTemplateParams, + UpdateTemplateParams, + DeleteTemplateParams, + StartTemplateSessionParams, + SetFieldValueParams, +} from '@shared/types' + +const logger = getLogger('template-handlers') + +export function registerTemplateHandlers(): void { + ipcMain.handle(IPC_CHANNELS.DICTATION_TEMPLATE.GET_ALL, async () => { + try { + return ipcSuccess(getDictationTemplateService().getAll()) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.UnknownError, msg) + } + }) + + ipcMain.handle( + IPC_CHANNELS.DICTATION_TEMPLATE.CREATE, + async (_event, params: CreateTemplateParams) => { + try { + // 라이센스 체크 + try { + const { getLicenseService } = await import('../services/LicenseService') + const { Feature } = await import('@shared/types') + const license = getLicenseService() + const access = license.canUse(Feature.DICTATION_TEMPLATE) + if (!access.allowed) { + license.promptUpgrade(Feature.DICTATION_TEMPLATE, 'tier_required') + return ipcError(ErrorCode.FeatureNotAvailable, 'Pro+ required') + } + } catch { + // LicenseService 미초기화 시 허용 + } + + const result = getDictationTemplateService().create(params) + return ipcSuccess(result) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + logger.error('Template create failed:', msg) + return ipcError(ErrorCode.TemplateInvalidFormat, msg) + } + }, + ) + + ipcMain.handle( + IPC_CHANNELS.DICTATION_TEMPLATE.UPDATE, + async (_event, params: UpdateTemplateParams) => { + try { + const result = getDictationTemplateService().update(params) + return ipcSuccess(result) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.TemplateNotFound, msg) + } + }, + ) + + ipcMain.handle( + IPC_CHANNELS.DICTATION_TEMPLATE.DELETE, + async (_event, params: DeleteTemplateParams) => { + try { + getDictationTemplateService().delete(params.id) + return ipcSuccess(undefined) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.TemplateNotFound, msg) + } + }, + ) + + ipcMain.handle( + IPC_CHANNELS.DICTATION_TEMPLATE.START_SESSION, + async (_event, params: StartTemplateSessionParams) => { + try { + getDictationTemplateService().startSession(params.templateId) + return ipcSuccess(undefined) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + logger.error('Template session start failed:', msg) + return ipcError(ErrorCode.TemplateSessionAlreadyActive, msg) + } + }, + ) + + ipcMain.handle(IPC_CHANNELS.DICTATION_TEMPLATE.CANCEL_SESSION, async () => { + try { + getDictationTemplateService().cancelSession() + return ipcSuccess(undefined) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.TemplateSessionNotActive, msg) + } + }) + + ipcMain.handle(IPC_CHANNELS.DICTATION_TEMPLATE.GET_SESSION_STATE, async () => { + try { + return ipcSuccess(getDictationTemplateService().getSessionState()) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.UnknownError, msg) + } + }) + + ipcMain.handle( + IPC_CHANNELS.DICTATION_TEMPLATE.SET_FIELD_VALUE, + async (_event, params: SetFieldValueParams) => { + try { + getDictationTemplateService().setFieldValue(params.fieldId, params.value) + return ipcSuccess(undefined) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.TemplateFieldRecordingFailed, msg) + } + }, + ) +} diff --git a/src/main/ipc/voice-action-handlers.ts b/src/main/ipc/voice-action-handlers.ts new file mode 100644 index 0000000..c966f64 --- /dev/null +++ b/src/main/ipc/voice-action-handlers.ts @@ -0,0 +1,74 @@ +// src/main/ipc/voice-action-handlers.ts +// Phase 13.3: OS 자동화 IPC 핸들러 + +import { ipcMain } from 'electron' +import { IPC_CHANNELS } from '@shared/ipc-channels' +import { ErrorCode, ipcSuccess, ipcError } from '@shared/errors' +import { getVoiceActionService } from '../services/VoiceActionService' +import { getLogger } from '../services/LoggerService' +import type { VoiceActionExecuteParams } from '@shared/types' + +const logger = getLogger('voice-action-handlers') + +export function registerVoiceActionHandlers(): void { + ipcMain.handle( + IPC_CHANNELS.VOICE_ACTION.EXECUTE, + async (_event, params: VoiceActionExecuteParams) => { + try { + await getVoiceActionService().execute(params.text) + return ipcSuccess(undefined) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + logger.error('Voice action failed:', msg) + return ipcError(ErrorCode.VoiceActionExecutionFailed, msg) + } + }, + ) + + ipcMain.handle(IPC_CHANNELS.VOICE_ACTION.GET_PRESETS, async () => { + try { + return ipcSuccess(getVoiceActionService().getPresets()) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.UnknownError, msg) + } + }) + + ipcMain.handle(IPC_CHANNELS.VOICE_ACTION.GET_HISTORY, async () => { + try { + return ipcSuccess(getVoiceActionService().getHistory()) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.UnknownError, msg) + } + }) + + ipcMain.handle(IPC_CHANNELS.VOICE_ACTION.CLEAR_HISTORY, async () => { + try { + getVoiceActionService().clearHistory() + return ipcSuccess(undefined) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.UnknownError, msg) + } + }) + + ipcMain.handle(IPC_CHANNELS.VOICE_ACTION.SET_ENABLED, async (_event, params: { enabled: boolean }) => { + try { + getVoiceActionService().setEnabled(params.enabled) + return ipcSuccess(undefined) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.UnknownError, msg) + } + }) + + ipcMain.handle(IPC_CHANNELS.VOICE_ACTION.IS_ENABLED, async () => { + try { + return ipcSuccess(getVoiceActionService().isEnabled) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.UnknownError, msg) + } + }) +} diff --git a/src/main/ipc/voice-conversation-handlers.ts b/src/main/ipc/voice-conversation-handlers.ts new file mode 100644 index 0000000..cfc9141 --- /dev/null +++ b/src/main/ipc/voice-conversation-handlers.ts @@ -0,0 +1,97 @@ +// src/main/ipc/voice-conversation-handlers.ts +// Phase 13.1: 음성 대화 모드 IPC 핸들러 + +import { ipcMain } from 'electron' +import { IPC_CHANNELS } from '@shared/ipc-channels' +import { ErrorCode, ipcSuccess, ipcError } from '@shared/errors' +import { getVoiceConversationService } from '../services/VoiceConversationService' +import { getLogger } from '../services/LoggerService' +import type { ConversationSendParams } from '@shared/types' + +const logger = getLogger('voice-conversation-handlers') + +export function registerVoiceConversationHandlers(): void { + ipcMain.handle(IPC_CHANNELS.VOICE_CONVERSATION.START_SESSION, async () => { + try { + await getVoiceConversationService().startSession() + return ipcSuccess(undefined) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + logger.error('Conversation start failed:', msg) + return ipcError(ErrorCode.ConversationSessionAlreadyActive, msg) + } + }) + + ipcMain.handle(IPC_CHANNELS.VOICE_CONVERSATION.STOP_SESSION, async () => { + try { + getVoiceConversationService().stopSession() + return ipcSuccess(undefined) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.ConversationNoActiveSession, msg) + } + }) + + ipcMain.handle( + IPC_CHANNELS.VOICE_CONVERSATION.SEND_MESSAGE, + async (_event, params: ConversationSendParams) => { + try { + await getVoiceConversationService().sendTextMessage(params.text) + return ipcSuccess(undefined) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + logger.error('Conversation send failed:', msg) + return ipcError(ErrorCode.ConversationLLMFailed, msg) + } + }, + ) + + ipcMain.handle(IPC_CHANNELS.VOICE_CONVERSATION.GET_STATE, async () => { + try { + return ipcSuccess(getVoiceConversationService().getSessionInfo()) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.UnknownError, msg) + } + }) + + ipcMain.handle(IPC_CHANNELS.VOICE_CONVERSATION.GET_HISTORY, async () => { + try { + return ipcSuccess(getVoiceConversationService().getHistory()) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.UnknownError, msg) + } + }) + + ipcMain.handle(IPC_CHANNELS.VOICE_CONVERSATION.CLEAR_HISTORY, async () => { + try { + getVoiceConversationService().clearHistory() + return ipcSuccess(undefined) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.UnknownError, msg) + } + }) + + ipcMain.handle(IPC_CHANNELS.VOICE_CONVERSATION.CANCEL_RESPONSE, async () => { + try { + getVoiceConversationService().cancelResponse() + return ipcSuccess(undefined) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.UnknownError, msg) + } + }) + + // finishListening — 렌더러에서 녹음 종료 버튼 클릭 시 + ipcMain.handle('voiceConversation:finishListening', async () => { + try { + await getVoiceConversationService().finishListening() + return ipcSuccess(undefined) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + return ipcError(ErrorCode.ConversationLLMFailed, msg) + } + }) +} diff --git a/src/main/services/DictationTemplateService.ts b/src/main/services/DictationTemplateService.ts new file mode 100644 index 0000000..90e80ef --- /dev/null +++ b/src/main/services/DictationTemplateService.ts @@ -0,0 +1,307 @@ +// src/main/services/DictationTemplateService.ts +// Phase 12.3: 딕테이션 템플릿 서비스 +// 템플릿 CRUD + 세션 상태 머신 (필드별 음성 입력) + +import { EventEmitter } from 'events' +import { nanoid } from 'nanoid' +import Store from 'electron-store' +import { getLogger } from './LoggerService' +import { getMainWindow } from '../windows/WindowManager' +import { IPC_CHANNELS } from '@shared/ipc-channels' +import { D3ROError, ErrorCode } from '@shared/errors' +import type { + DictationTemplate, + TemplateField, + TemplateSessionState, + TemplateSessionInfo, + TemplateFieldCompletedEvent, + TemplateSessionCompletedEvent, + CreateTemplateParams, + UpdateTemplateParams, +} from '@shared/types' + +const logger = getLogger('DictationTemplateService') + +// ── 프리셋 템플릿 ── +const BUILTIN_TEMPLATES: DictationTemplate[] = [ + { + id: 'builtin-email', + name: 'Email', + description: 'Email template with recipient, subject, and body', + fields: [ + { id: 'recipient', name: 'recipient', label: 'Recipient', promptText: 'Who is this email for?', required: true, maxDurationSec: 15 }, + { id: 'subject', name: 'subject', label: 'Subject', promptText: 'What is the subject?', required: true, maxDurationSec: 15 }, + { id: 'body', name: 'body', label: 'Body', promptText: 'Please dictate the email body.', required: true, maxDurationSec: 120 }, + ], + outputFormat: 'To: {{recipient}}\nSubject: {{subject}}\n\n{{body}}', + isBuiltin: true, + createdAt: Date.now(), + updatedAt: Date.now(), + }, + { + id: 'builtin-meeting-notes', + name: 'Meeting Notes', + description: 'Meeting notes template', + fields: [ + { id: 'title', name: 'title', label: 'Title', promptText: 'What is the meeting title?', required: true, maxDurationSec: 15 }, + { id: 'attendees', name: 'attendees', label: 'Attendees', promptText: 'Who attended?', required: false, maxDurationSec: 30 }, + { id: 'agenda', name: 'agenda', label: 'Agenda', promptText: 'What was discussed?', required: true, maxDurationSec: 120 }, + { id: 'decisions', name: 'decisions', label: 'Decisions', promptText: 'What decisions were made?', required: false, maxDurationSec: 60 }, + ], + outputFormat: '# {{title}}\n\nAttendees: {{attendees}}\n\n## Agenda\n{{agenda}}\n\n## Decisions\n{{decisions}}', + isBuiltin: true, + createdAt: Date.now(), + updatedAt: Date.now(), + }, + { + id: 'builtin-report', + name: 'Report', + description: 'Simple report template', + fields: [ + { id: 'title', name: 'title', label: 'Title', promptText: 'Report title?', required: true, maxDurationSec: 15 }, + { id: 'summary', name: 'summary', label: 'Summary', promptText: 'Summarize the key points.', required: true, maxDurationSec: 60 }, + { id: 'details', name: 'details', label: 'Details', promptText: 'Provide the details.', required: true, maxDurationSec: 180 }, + ], + outputFormat: '# {{title}}\n\n## Summary\n{{summary}}\n\n## Details\n{{details}}', + isBuiltin: true, + createdAt: Date.now(), + updatedAt: Date.now(), + }, +] + +interface TemplateStoreSchema { + templates: DictationTemplate[] +} + +class DictationTemplateService extends EventEmitter { + private _store: Store + private _session: TemplateSessionInfo | null = null + + constructor() { + super() + this._store = new Store({ + name: 'dictation-templates', + defaults: { + templates: [...BUILTIN_TEMPLATES], + }, + }) + + // 프리셋이 없으면 추가 + this._ensureBuiltins() + } + + // ── CRUD ── + + getAll(): DictationTemplate[] { + return this._store.get('templates', []) + } + + getById(id: string): DictationTemplate | null { + const templates = this.getAll() + return templates.find((t) => t.id === id) ?? null + } + + create(params: CreateTemplateParams): DictationTemplate { + const template: DictationTemplate = { + id: nanoid(), + name: params.name, + description: params.description, + fields: params.fields, + outputFormat: params.outputFormat, + isBuiltin: false, + createdAt: Date.now(), + updatedAt: Date.now(), + } + + const templates = this.getAll() + templates.push(template) + this._store.set('templates', templates) + + return template + } + + update(params: UpdateTemplateParams): DictationTemplate { + const templates = this.getAll() + const idx = templates.findIndex((t) => t.id === params.id) + if (idx === -1) { + throw new D3ROError(ErrorCode.TemplateNotFound, `Template not found: ${params.id}`) + } + + const existing = templates[idx] + const updated: DictationTemplate = { + ...existing, + ...(params.name !== undefined && { name: params.name }), + ...(params.description !== undefined && { description: params.description }), + ...(params.fields !== undefined && { fields: params.fields }), + ...(params.outputFormat !== undefined && { outputFormat: params.outputFormat }), + updatedAt: Date.now(), + } + + templates[idx] = updated + this._store.set('templates', templates) + return updated + } + + delete(id: string): void { + const templates = this.getAll() + const template = templates.find((t) => t.id === id) + if (!template) { + throw new D3ROError(ErrorCode.TemplateNotFound, `Template not found: ${id}`) + } + if (template.isBuiltin) { + throw new D3ROError(ErrorCode.TemplateInvalidFormat, 'Cannot delete builtin template') + } + + this._store.set( + 'templates', + templates.filter((t) => t.id !== id), + ) + } + + // ── 세션 관리 ── + + getSessionState(): TemplateSessionInfo | null { + return this._session + } + + startSession(templateId: string): void { + if (this._session) { + throw new D3ROError(ErrorCode.TemplateSessionAlreadyActive, 'Template session already active') + } + + const template = this.getById(templateId) + if (!template) { + throw new D3ROError(ErrorCode.TemplateNotFound, `Template not found: ${templateId}`) + } + + if (template.fields.length === 0) { + throw new D3ROError(ErrorCode.TemplateInvalidFormat, 'Template has no fields') + } + + this._session = { + templateId, + templateName: template.name, + state: 'field-prompting', + currentFieldIndex: 0, + totalFields: template.fields.length, + currentField: template.fields[0], + fieldValues: {}, + } + + this._emitSessionState() + logger.info(`Template session started: ${template.name}`) + } + + setFieldValue(fieldId: string, value: string): void { + if (!this._session) { + throw new D3ROError(ErrorCode.TemplateSessionNotActive, 'No active template session') + } + + const template = this.getById(this._session.templateId) + if (!template) { + throw new D3ROError(ErrorCode.TemplateNotFound, 'Session template not found') + } + + this._session.fieldValues[fieldId] = value + + const currentField = this._session.currentField + const nextIndex = this._session.currentFieldIndex + 1 + const nextField = nextIndex < template.fields.length ? template.fields[nextIndex] : null + + // 필드 완료 이벤트 + const fieldEvent: TemplateFieldCompletedEvent = { + fieldId, + fieldName: currentField?.name ?? fieldId, + value, + nextField, + } + this._sendToRenderer(IPC_CHANNELS.DICTATION_TEMPLATE.FIELD_COMPLETED, fieldEvent) + this.emit('field-completed', fieldEvent) + + if (nextField) { + // 다음 필드로 이동 + this._session.currentFieldIndex = nextIndex + this._session.currentField = nextField + this._session.state = 'field-prompting' + this._emitSessionState() + } else { + // 모든 필드 완료 → 출력 생성 + this._completeSession(template) + } + } + + cancelSession(): void { + if (!this._session) return + logger.info('Template session cancelled') + this._session = null + this._emitSessionState() + } + + private _completeSession(template: DictationTemplate): void { + if (!this._session) return + + this._session.state = 'completing' + this._emitSessionState() + + // 출력 포맷 적용 (mustache-like 치환) + let output = template.outputFormat + for (const [key, value] of Object.entries(this._session.fieldValues)) { + output = output.replace(new RegExp(`\\{\\{${key}\\}\\}`, 'g'), value) + } + // 미입력 필드 자리표시자 제거 + output = output.replace(/\{\{[^}]+\}\}/g, '') + + const completedEvent: TemplateSessionCompletedEvent = { + templateId: template.id, + outputText: output.trim(), + fieldValues: { ...this._session.fieldValues }, + } + + this._sendToRenderer(IPC_CHANNELS.DICTATION_TEMPLATE.SESSION_COMPLETED, completedEvent) + this.emit('session-completed', completedEvent) + + this._session = null + logger.info(`Template session completed: ${template.name}`) + } + + private _emitSessionState(): void { + this._sendToRenderer(IPC_CHANNELS.DICTATION_TEMPLATE.SESSION_STATE_CHANGED, this._session) + this.emit('session-state-changed', this._session) + } + + private _sendToRenderer(channel: string, data: unknown): void { + try { + const mainWindow = getMainWindow() + if (mainWindow && !mainWindow.isDestroyed()) { + mainWindow.webContents.send(channel, data) + } + } catch { + // 윈도우 없으면 무시 + } + } + + private _ensureBuiltins(): void { + const templates = this.getAll() + for (const builtin of BUILTIN_TEMPLATES) { + if (!templates.find((t) => t.id === builtin.id)) { + templates.push(builtin) + } + } + this._store.set('templates', templates) + } + + dispose(): void { + this.cancelSession() + this.removeAllListeners() + } +} + +// ── 싱글톤 ── +let instance: DictationTemplateService | null = null + +export function getDictationTemplateService(): DictationTemplateService { + if (!instance) { + instance = new DictationTemplateService() + } + return instance +} diff --git a/src/main/services/FileTranscriptionService.ts b/src/main/services/FileTranscriptionService.ts new file mode 100644 index 0000000..09f0631 --- /dev/null +++ b/src/main/services/FileTranscriptionService.ts @@ -0,0 +1,407 @@ +// src/main/services/FileTranscriptionService.ts +// Phase 12.1: 파일 전사 서비스 +// 오디오/비디오 파일 → ffmpeg PCM 변환 → 30초 청크 순차 STT → 병합 + +import { EventEmitter } from 'events' +import path from 'path' +import fs from 'fs' +import { nanoid } from 'nanoid' +import { app } from 'electron' +import { getLogger } from './LoggerService' +import { getLocalSTTService } from './LocalSTTService' +import { getHistoryService } from './HistoryService' +import { configGet } from './ConfigService' +import { getFfmpegPath } from '../utils/paths' +import { getMainWindow } from '../windows/WindowManager' +import { IPC_CHANNELS } from '@shared/ipc-channels' +import { D3ROError, ErrorCode } from '@shared/errors' +import type { + FileTranscriptionState, + FileTranscriptionProgress, + FileTranscriptionResult, + FileTranscriptionSegment, + FileTranscriptionStateInfo, +} from '@shared/types' + +const logger = getLogger('FileTranscriptionService') + +/** 청크 길이 (초) */ +const CHUNK_DURATION_SEC = 30 +/** 최대 파일 크기 (2GB) */ +const MAX_FILE_SIZE_BYTES = 2 * 1024 * 1024 * 1024 +/** initialPrompt 컨텍스트 윈도우 (자) */ +const CONTEXT_WINDOW_SIZE = 300 +/** 지원 확장자 */ +const SUPPORTED_EXTENSIONS = new Set([ + '.mp3', '.wav', '.m4a', '.ogg', '.flac', '.wma', '.aac', + '.mp4', '.mkv', '.webm', '.avi', '.mov', +]) + +class FileTranscriptionService extends EventEmitter { + private _state: FileTranscriptionState = 'idle' + private _jobId: string | null = null + private _cancelled = false + private _progress: FileTranscriptionProgress | null = null + private _tempDir: string | null = null + + get state(): FileTranscriptionState { + return this._state + } + + getStateInfo(): FileTranscriptionStateInfo { + return { + state: this._state, + jobId: this._jobId, + progress: this._progress, + } + } + + async startTranscription(filePath: string, language?: string): Promise { + if (this._state !== 'idle') { + throw new D3ROError(ErrorCode.FileTranscriptionChunkFailed, 'Transcription already in progress') + } + + // 파일 검증 + const ext = path.extname(filePath).toLowerCase() + if (!SUPPORTED_EXTENSIONS.has(ext)) { + throw new D3ROError(ErrorCode.FileTranscriptionInvalidFormat, `Unsupported format: ${ext}`) + } + + const stat = fs.statSync(filePath) + if (stat.size > MAX_FILE_SIZE_BYTES) { + throw new D3ROError(ErrorCode.FileTranscriptionFileTooLarge, 'File exceeds 2GB limit') + } + + // 라이센스 체크 + try { + const { getLicenseService } = await import('./LicenseService') + const { Feature } = await import('@shared/types') + const license = getLicenseService() + const access = license.canUse(Feature.FILE_TRANSCRIPTION) + if (!access.allowed) { + license.promptUpgrade( + Feature.FILE_TRANSCRIPTION, + access.reason === 'quota_exceeded' ? 'quota_exceeded' : 'tier_required', + ) + throw new D3ROError(ErrorCode.FeatureNotAvailable, 'Pro+ required for file transcription') + } + } catch (err) { + if (err instanceof D3ROError) throw err + } + + this._jobId = nanoid() + this._cancelled = false + this._tempDir = path.join(app.getPath('temp'), `d3ro-ft-${this._jobId}`) + fs.mkdirSync(this._tempDir, { recursive: true }) + + const startTime = Date.now() + const fileName = path.basename(filePath) + + try { + // Phase 1: ffmpeg 변환 → PCM WAV + this._setState('converting') + const wavPath = path.join(this._tempDir, 'audio.wav') + await this._convertToWav(filePath, wavPath) + + if (this._cancelled) { + throw new D3ROError(ErrorCode.FileTranscriptionCancelled, 'Transcription cancelled') + } + + // 오디오 길이 확인 + const totalDurationSec = await this._probeDuration(wavPath) + const totalChunks = Math.ceil(totalDurationSec / CHUNK_DURATION_SEC) + + // Phase 2: 청크별 STT + this._setState('transcribing') + const allSegments: FileTranscriptionSegment[] = [] + const allTexts: string[] = [] + let previousContext = '' + + for (let i = 0; i < totalChunks; i++) { + if (this._cancelled) { + throw new D3ROError(ErrorCode.FileTranscriptionCancelled, 'Transcription cancelled') + } + + const startSec = i * CHUNK_DURATION_SEC + const chunkBuffer = await this._extractChunk(wavPath, startSec, CHUNK_DURATION_SEC) + + const sttService = getLocalSTTService() + const lang = language ?? (configGet('sttLanguage') as string | undefined) ?? 'auto' + + const result = await sttService.transcribe(chunkBuffer, { + language: lang, + initialPrompt: previousContext || undefined, + vadFilter: true, + }) + + if (result.text && result.text.trim().length > 0) { + allTexts.push(result.text.trim()) + previousContext = result.text.trim().slice(-CONTEXT_WINDOW_SIZE) + + for (const seg of result.segments) { + allSegments.push({ + text: seg.text, + start: seg.start + startSec, + end: seg.end + startSec, + confidence: seg.confidence, + }) + } + } + + this._progress = { + jobId: this._jobId!, + currentChunk: i + 1, + totalChunks, + percent: Math.round(((i + 1) / totalChunks) * 100), + currentText: result.text?.trim() ?? '', + } + + this._sendToRenderer(IPC_CHANNELS.FILE_TRANSCRIPTION.PROGRESS, this._progress) + this.emit('progress', this._progress) + } + + const fullText = allTexts.join(' ') + const processingTimeMs = Date.now() - startTime + + const resultData: FileTranscriptionResult = { + jobId: this._jobId!, + filePath, + fileName, + fullText, + segments: allSegments, + totalDurationSec, + processingTimeMs, + } + + // 히스토리에 저장 + try { + const wordCount = fullText.split(/\s+/).filter((w) => w.length > 0).length + getHistoryService().create({ + originalText: fullText, + polishedText: null, + focusedApp: null, + focusedAppName: null, + focusedAppWindowTitle: null, + mode: 'file-transcription', + status: 'completed', + errorCode: null, + audioLocalPath: filePath, + duration: totalDurationSec, + detectedLanguage: null, + micDevice: null, + wordCount, + sttModel: configGet('sttModelId') as string | null, + llmModel: null, + sttLatencyMs: processingTimeMs, + llmLatencyMs: null, + appVersion: app.getVersion(), + }) + } catch (err) { + logger.warn('Failed to save file transcription to history:', err) + } + + this._setState('completed') + this._sendToRenderer(IPC_CHANNELS.FILE_TRANSCRIPTION.COMPLETE, resultData) + this.emit('complete', resultData) + + return resultData + } catch (err) { + if (err instanceof D3ROError && err.code === ErrorCode.FileTranscriptionCancelled) { + this._setState('idle') + throw err + } + this._setState('error') + const errorMsg = err instanceof Error ? err.message : String(err) + this._sendToRenderer(IPC_CHANNELS.FILE_TRANSCRIPTION.ERROR, { message: errorMsg }) + this.emit('error', err) + throw err instanceof D3ROError + ? err + : new D3ROError(ErrorCode.FileTranscriptionChunkFailed, errorMsg) + } finally { + this._cleanup() + // 완료/에러 후 idle로 복귀 + setTimeout(() => { + this._state = 'idle' + this._jobId = null + this._progress = null + }, 1000) + } + } + + cancel(): void { + if (this._state !== 'idle') { + this._cancelled = true + logger.info(`File transcription cancelled: ${this._jobId}`) + } + } + + private _setState(state: FileTranscriptionState): void { + this._state = state + this.emit('state-changed', state) + } + + private _sendToRenderer(channel: string, data: unknown): void { + try { + const mainWindow = getMainWindow() + if (mainWindow && !mainWindow.isDestroyed()) { + mainWindow.webContents.send(channel, data) + } + } catch { + // 윈도우 없으면 무시 + } + } + + /** + * ffmpeg로 미디어 파일을 PCM16 16kHz mono WAV로 변환 + */ + private _convertToWav(inputPath: string, outputPath: string): Promise { + return new Promise((resolve, reject) => { + const { spawn } = require('child_process') as typeof import('child_process') + const ffmpegPath = getFfmpegPath() + + const args = [ + '-i', inputPath, + '-ar', '16000', + '-ac', '1', + '-sample_fmt', 's16', + '-y', + outputPath, + ] + + logger.info(`ffmpeg convert: ${ffmpegPath} ${args.join(' ')}`) + + const proc = spawn(ffmpegPath, args, { stdio: ['pipe', 'pipe', 'pipe'] }) + let stderr = '' + + proc.stderr?.on('data', (data: Buffer) => { + stderr += data.toString() + }) + + proc.on('close', (code: number) => { + if (code === 0) { + resolve() + } else { + logger.error(`ffmpeg failed (code ${code}):`, stderr.slice(-500)) + reject(new D3ROError(ErrorCode.FileTranscriptionFFmpegFailed, `ffmpeg exited with code ${code}`)) + } + }) + + proc.on('error', (err: Error) => { + reject(new D3ROError(ErrorCode.FileTranscriptionFFmpegFailed, `ffmpeg error: ${err.message}`)) + }) + }) + } + + /** + * ffprobe(ffmpeg)로 오디오 길이 측정 (초) + */ + private _probeDuration(wavPath: string): Promise { + return new Promise((resolve, reject) => { + const { spawn } = require('child_process') as typeof import('child_process') + const ffmpegPath = getFfmpegPath() + // ffprobe는 보통 ffmpeg과 같은 디렉토리에 있으나, + // @ffmpeg-installer는 ffmpeg만 제공 → -i로 duration 추출 + const args = [ + '-i', wavPath, + '-f', 'null', + '-', + ] + + const proc = spawn(ffmpegPath, args, { stdio: ['pipe', 'pipe', 'pipe'] }) + let stderr = '' + + proc.stderr?.on('data', (data: Buffer) => { + stderr += data.toString() + }) + + proc.on('close', () => { + // "Duration: HH:MM:SS.ms" 패턴 파싱 + const match = stderr.match(/Duration:\s*(\d+):(\d+):(\d+)\.(\d+)/) + if (match) { + const hours = parseInt(match[1], 10) + const minutes = parseInt(match[2], 10) + const seconds = parseInt(match[3], 10) + const ms = parseInt(match[4], 10) / 100 + resolve(hours * 3600 + minutes * 60 + seconds + ms) + } else { + // WAV 파일 크기로 폴백 추정 (16kHz 16bit mono = 32000 bytes/sec) + try { + const stat = fs.statSync(wavPath) + const headerSize = 44 + const bytesPerSec = 16000 * 2 * 1 + resolve(Math.max(0, (stat.size - headerSize) / bytesPerSec)) + } catch { + reject(new D3ROError(ErrorCode.FileTranscriptionFFmpegFailed, 'Cannot determine audio duration')) + } + } + }) + }) + } + + /** + * WAV 파일에서 특정 구간을 PCM16 Buffer로 추출 + */ + private _extractChunk(wavPath: string, startSec: number, durationSec: number): Promise { + return new Promise((resolve, reject) => { + const { spawn } = require('child_process') as typeof import('child_process') + const ffmpegPath = getFfmpegPath() + + const args = [ + '-ss', String(startSec), + '-t', String(durationSec), + '-i', wavPath, + '-ar', '16000', + '-ac', '1', + '-f', 's16le', + '-acodec', 'pcm_s16le', + 'pipe:1', + ] + + const proc = spawn(ffmpegPath, args, { stdio: ['pipe', 'pipe', 'pipe'] }) + const chunks: Buffer[] = [] + + proc.stdout?.on('data', (data: Buffer) => { + chunks.push(data) + }) + + proc.on('close', (code: number) => { + if (code === 0 || chunks.length > 0) { + resolve(Buffer.concat(chunks)) + } else { + reject(new D3ROError(ErrorCode.FileTranscriptionChunkFailed, `Chunk extraction failed at ${startSec}s`)) + } + }) + + proc.on('error', (err: Error) => { + reject(new D3ROError(ErrorCode.FileTranscriptionFFmpegFailed, `ffmpeg chunk error: ${err.message}`)) + }) + }) + } + + private _cleanup(): void { + if (this._tempDir && fs.existsSync(this._tempDir)) { + try { + fs.rmSync(this._tempDir, { recursive: true, force: true }) + } catch (err) { + logger.warn('Failed to cleanup temp dir:', err) + } + this._tempDir = null + } + } + + dispose(): void { + this.cancel() + this._cleanup() + this.removeAllListeners() + } +} + +// ── 싱글톤 ── +let instance: FileTranscriptionService | null = null + +export function getFileTranscriptionService(): FileTranscriptionService { + if (!instance) { + instance = new FileTranscriptionService() + } + return instance +} diff --git a/src/main/services/LicenseService.ts b/src/main/services/LicenseService.ts index 3039a1d..39b50fe 100644 --- a/src/main/services/LicenseService.ts +++ b/src/main/services/LicenseService.ts @@ -348,15 +348,32 @@ class LicenseService extends EventEmitter { return { success: false, tier: 'free', message: 'License key is empty' } } - // 1. LemonSqueezy API 시도 + // 1. 로컬 키 검증 먼저 (개발/테스트용 D3RO-PRO-*, D3RO-PLUS-* 패턴) + const localTier = this._validateKeyLocally(trimmedKey) + if (localTier) { + logger.info(`Local key validated: tier=${localTier}`) + const now = Date.now() + this._info = { + ...this._info, + tier: localTier, + licenseKey: trimmedKey, + activatedAt: now, + lastVerifiedAt: now, + offlineGraceUntil: now + OFFLINE_GRACE_PERIOD_MS, + } + this._persistLicenseInfo() + this._sendToRenderer(IPC_CHANNELS.LICENSE.TIER_CHANGED, this._info) + return { success: true, tier: localTier, message: `Activated ${localTier} license (local)` } + } + + // 2. LemonSqueezy API 시도 const apiResult = await this._activateViaLemonSqueezy(trimmedKey) if (apiResult) { return apiResult } - // 2. API 실패 시 로컬 키 검증 폴백 (개발/테스트용) - logger.info('LemonSqueezy API unreachable, trying local key validation') - const tier = this._validateKeyLocally(trimmedKey) + // 3. 둘 다 실패 + const tier = null as LicenseTier | null if (!tier) { return { success: false, tier: 'free', message: 'Invalid license key' } } @@ -506,6 +523,27 @@ class LicenseService extends EventEmitter { pro: false, proPlus: true, }, + { + feature: Feature.DICTATION_TEMPLATE, + featureLabel: 'license.feature.dictationTemplate', + free: false, + pro: false, + proPlus: true, + }, + { + feature: Feature.LOCAL_RAG, + featureLabel: 'license.feature.localRag', + free: false, + pro: false, + proPlus: true, + }, + { + feature: Feature.OS_AUTOMATION, + featureLabel: 'license.feature.osAutomation', + free: false, + pro: false, + proPlus: true, + }, ] } diff --git a/src/main/services/LocalLLMService.ts b/src/main/services/LocalLLMService.ts index 896ded9..5f307d5 100644 --- a/src/main/services/LocalLLMService.ts +++ b/src/main/services/LocalLLMService.ts @@ -378,6 +378,81 @@ class LocalLLMService extends EventEmitter { return this._available } + /** + * Ollama /api/chat 스트리밍 대화. + * messages 배열로 대화 히스토리를 전달한다. + * 각 토큰마다 yield, 완료 시 전체 응답 텍스트를 return. + */ + async *chatStream( + messages: Array<{ role: string; content: string }>, + options?: { model?: string; temperature?: number }, + ): AsyncGenerator { + if (!this._available) { + throw new D3ROError(ErrorCode.LLMServerUnreachable, 'Ollama server not available') + } + + const serverUrl = configGet('ollamaServerUrl') + const model = options?.model ?? configGet('llmModelId') ?? 'qwen3:4b' + + this._abortController = new AbortController() + this._state = LLMState.Generating + + try { + const response = await fetch(`${serverUrl}/api/chat`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + model, + messages, + stream: true, + options: { + temperature: options?.temperature ?? 0.7, + }, + }), + signal: this._abortController.signal, + }) + + if (!response.ok || !response.body) { + throw new D3ROError(ErrorCode.LLMProcessingFailed, `Chat API error: ${response.status}`) + } + + const reader = response.body.getReader() + const decoder = new TextDecoder() + let buffer = '' + let accumulated = '' + + while (true) { + const { done, value } = await reader.read() + if (done) break + + buffer += decoder.decode(value, { stream: true }) + const lines = buffer.split('\n') + buffer = lines.pop() ?? '' + + for (const line of lines) { + if (!line.trim()) continue + try { + const chunk = JSON.parse(line) as { message?: { content: string }; done: boolean } + if (chunk.message?.content) { + accumulated += chunk.message.content + yield chunk.message.content + } + if (chunk.done) { + return accumulated + } + } catch { + // 불완전 JSON 무시 + } + } + } + + return accumulated + } finally { + this._state = this._available ? LLMState.Available : LLMState.Unavailable + this._abortController = null + } + } + dispose(): void { this._disposed = true this.stopPolling() diff --git a/src/main/services/MeetingSummaryService.ts b/src/main/services/MeetingSummaryService.ts new file mode 100644 index 0000000..286eafd --- /dev/null +++ b/src/main/services/MeetingSummaryService.ts @@ -0,0 +1,257 @@ +// src/main/services/MeetingSummaryService.ts +// Phase 12.2: 회의록 자동 요약 서비스 +// CaptionService 세션 종료 후 LLM으로 요약 생성 + +import { EventEmitter } from 'events' +import path from 'path' +import fs from 'fs' +import { app, dialog } from 'electron' +import { eq } from 'drizzle-orm' +import { getLogger } from './LoggerService' +import { getLocalLLMService } from './LocalLLMService' +import { getDatabase } from '../db' +import { history } from '../db/schema' +import { getMainWindow } from '../windows/WindowManager' +import { IPC_CHANNELS } from '@shared/ipc-channels' +import { D3ROError, ErrorCode } from '@shared/errors' +import type { MeetingSummaryResult, MeetingSummaryProgress } from '@shared/types' + +const logger = getLogger('MeetingSummaryService') + +const MEETING_SUMMARY_PROMPT = `다음은 회의 전사록입니다. 아래 형식으로 정리해주세요: + +## 요약 +(3줄 이내 핵심 요약) + +## 핵심 결정사항 +- (결정 1) +- (결정 2) + +## 할 일 목록 +- [ ] (할 일 1) +- [ ] (할 일 2) + +전사록 외의 내용을 추가하지 마세요. 전사록이 비어 있거나 내용이 부족하면 "요약할 내용이 충분하지 않습니다"라고만 출력하세요.` + +class MeetingSummaryService extends EventEmitter { + /** + * 히스토리 항목의 전사 텍스트를 LLM으로 요약 + */ + async summarize(historyId: string): Promise { + // 라이센스 체크 + try { + const { getLicenseService } = await import('./LicenseService') + const { Feature } = await import('@shared/types') + const license = getLicenseService() + const access = license.canUse(Feature.MEETING_SUMMARY) + if (!access.allowed) { + license.promptUpgrade( + Feature.MEETING_SUMMARY, + access.reason === 'quota_exceeded' ? 'quota_exceeded' : 'tier_required', + ) + throw new D3ROError(ErrorCode.FeatureNotAvailable, 'Pro+ required for meeting summary') + } + } catch (err) { + if (err instanceof D3ROError) throw err + } + + // 히스토리 항목 조회 + const db = getDatabase() + const rows = db.select().from(history).where(eq(history.id, historyId)).all() + if (rows.length === 0) { + throw new D3ROError(ErrorCode.HistoryNotFound, `History not found: ${historyId}`) + } + + const entry = rows[0] + const transcript = entry.originalText + if (!transcript || transcript.trim().length === 0) { + throw new D3ROError(ErrorCode.MeetingSummaryNoTranscript, 'No transcript text to summarize') + } + + // 진행 상태 알림 + this._sendProgress(historyId, 'generating') + + try { + // LLM 요약 생성 + const llmService = getLocalLLMService() + const result = await llmService.generate(transcript, { + systemPrompt: MEETING_SUMMARY_PROMPT, + temperature: 0.3, + }) + + const rawMarkdown = result.text.trim() + const parsed = this._parseMarkdown(rawMarkdown) + + const summaryResult: MeetingSummaryResult = { + historyId, + summary: parsed.summary, + decisions: parsed.decisions, + actionItems: parsed.actionItems, + rawMarkdown, + generatedAt: Date.now(), + } + + // DB에 요약 저장 + db.update(history) + .set({ summaryText: rawMarkdown, updatedAt: Date.now() }) + .where(eq(history.id, historyId)) + .run() + + this._sendProgress(historyId, 'done') + this._sendToRenderer(IPC_CHANNELS.MEETING_SUMMARY.SUMMARY_READY, summaryResult) + this.emit('summary-ready', summaryResult) + + return summaryResult + } catch (err) { + this._sendProgress(historyId, 'error') + if (err instanceof D3ROError) throw err + const msg = err instanceof Error ? err.message : String(err) + throw new D3ROError(ErrorCode.MeetingSummaryGenerationFailed, msg) + } + } + + /** + * 저장된 요약 조회 + */ + getSummary(historyId: string): MeetingSummaryResult | null { + const db = getDatabase() + const rows = db.select().from(history).where(eq(history.id, historyId)).all() + if (rows.length === 0 || !rows[0].summaryText) { + return null + } + + const entry = rows[0] + const parsed = this._parseMarkdown(entry.summaryText!) + return { + historyId, + summary: parsed.summary, + decisions: parsed.decisions, + actionItems: parsed.actionItems, + rawMarkdown: entry.summaryText!, + generatedAt: entry.updatedAt, + } + } + + /** + * 마크다운 파일로 내보내기 + */ + async exportMarkdown(historyId: string): Promise { + const summary = this.getSummary(historyId) + if (!summary) { + throw new D3ROError(ErrorCode.MeetingSummaryExportFailed, 'No summary to export') + } + + const db = getDatabase() + const rows = db.select().from(history).where(eq(history.id, historyId)).all() + const entry = rows[0] + + const date = new Date(entry.createdAt) + const dateStr = date.toISOString().slice(0, 10) + const defaultName = `meeting-summary-${dateStr}.md` + + const result = await dialog.showSaveDialog({ + defaultPath: path.join(app.getPath('documents'), defaultName), + filters: [{ name: 'Markdown', extensions: ['md'] }], + }) + + if (result.canceled || !result.filePath) { + throw new D3ROError(ErrorCode.MeetingSummaryExportFailed, 'Export cancelled') + } + + const content = `# Meeting Summary — ${dateStr}\n\n${summary.rawMarkdown}\n\n---\n\n## Full Transcript\n\n${entry.originalText}\n` + fs.writeFileSync(result.filePath, content, 'utf-8') + + return result.filePath + } + + /** + * CaptionService 세션 종료 시 자동 요약 (옵션) + */ + async onCaptionSessionSaved(sessionSummary: { sessionId: string }): Promise { + // caption 모드 히스토리에서 해당 세션 찾기 + const db = getDatabase() + const rows = db + .select() + .from(history) + .where(eq(history.id, sessionSummary.sessionId)) + .all() + + if (rows.length === 0) { + logger.warn(`Caption session not found in history: ${sessionSummary.sessionId}`) + return + } + + const entry = rows[0] + if (!entry.originalText || entry.originalText.trim().length < 50) { + logger.info('Caption session too short for summary, skipping') + return + } + + try { + await this.summarize(entry.id) + logger.info(`Auto-summary generated for caption session: ${entry.id}`) + } catch (err) { + logger.warn('Auto-summary failed:', err) + } + } + + private _parseMarkdown(markdown: string): { + summary: string + decisions: string[] + actionItems: string[] + } { + const sections = markdown.split(/^## /m) + let summary = '' + const decisions: string[] = [] + const actionItems: string[] = [] + + for (const section of sections) { + const lines = section.trim().split('\n') + const heading = lines[0]?.trim().toLowerCase() ?? '' + const body = lines.slice(1).join('\n').trim() + + if (heading.includes('요약') || heading.includes('summary')) { + summary = body + } else if (heading.includes('결정') || heading.includes('decision')) { + const items = body.split('\n').filter((l) => l.trim().startsWith('-')) + decisions.push(...items.map((l) => l.replace(/^-\s*/, '').trim())) + } else if (heading.includes('할 일') || heading.includes('action')) { + const items = body.split('\n').filter((l) => l.trim().startsWith('-')) + actionItems.push( + ...items.map((l) => l.replace(/^-\s*\[[ x]?\]\s*/, '').trim()), + ) + } + } + + return { summary, decisions, actionItems } + } + + private _sendProgress(historyId: string, status: MeetingSummaryProgress['status']): void { + this._sendToRenderer(IPC_CHANNELS.MEETING_SUMMARY.SUMMARY_PROGRESS, { historyId, status }) + } + + private _sendToRenderer(channel: string, data: unknown): void { + try { + const mainWindow = getMainWindow() + if (mainWindow && !mainWindow.isDestroyed()) { + mainWindow.webContents.send(channel, data) + } + } catch { + // 윈도우 없으면 무시 + } + } + + dispose(): void { + this.removeAllListeners() + } +} + +// ── 싱글톤 ── +let instance: MeetingSummaryService | null = null + +export function getMeetingSummaryService(): MeetingSummaryService { + if (!instance) { + instance = new MeetingSummaryService() + } + return instance +} diff --git a/src/main/services/RAGService.ts b/src/main/services/RAGService.ts new file mode 100644 index 0000000..7e42230 --- /dev/null +++ b/src/main/services/RAGService.ts @@ -0,0 +1,536 @@ +// src/main/services/RAGService.ts +// Phase 13.2: 로컬 RAG — 문서 임베딩 + 코사인 유사도 검색 + LLM 컨텍스트 주입 +// Ollama nomic-embed-text 모델, SQLite에 JSON 직렬화 벡터 저장. + +import { EventEmitter } from 'events' +import path from 'path' +import fs from 'fs' +import { nanoid } from 'nanoid' +import { eq } from 'drizzle-orm' +import { getLogger } from './LoggerService' +import { getLocalLLMService } from './LocalLLMService' +import { configGet } from './ConfigService' +import { getDatabase } from '../db' +import { ragDocuments, ragChunks } from '../db/schema' +import { getMainWindow } from '../windows/WindowManager' +import { IPC_CHANNELS } from '@shared/ipc-channels' +import { D3ROError, ErrorCode } from '@shared/errors' +import type { + RAGDocument, + RAGQueryResult, + RAGState, + RAGStateInfo, + RAGIndexProgress, +} from '@shared/types' + +const logger = getLogger('RAGService') + +/** 임베딩 모델 */ +const EMBED_MODEL = 'nomic-embed-text' +/** 청크 크기 (자) */ +const CHUNK_SIZE = 500 +/** 청크 오버랩 (자) */ +const CHUNK_OVERLAP = 50 +/** 검색 기본 topK */ +const DEFAULT_TOP_K = 5 + +class RAGService extends EventEmitter { + private _state: RAGState = 'idle' + + get state(): RAGState { + return this._state + } + + getStateInfo(): RAGStateInfo { + const db = getDatabase() + const docs = db.select().from(ragDocuments).all() + const chunks = db.select().from(ragChunks).all() + return { + state: this._state, + documentCount: docs.length, + totalChunks: chunks.length, + } + } + + getDocuments(): RAGDocument[] { + const db = getDatabase() + const rows = db.select().from(ragDocuments).all() + return rows.map((r) => ({ + id: r.id, + fileName: r.fileName, + filePath: r.filePath, + fileType: r.fileType as RAGDocument['fileType'], + chunkCount: r.chunkCount, + indexed: r.indexed, + indexedAt: r.indexedAt, + addedAt: r.addedAt, + })) + } + + /** + * 문서 추가 + 인덱싱 (청킹 → 임베딩 → DB 저장) + */ + async addDocument(filePath: string): Promise { + // 라이센스 체크 + try { + const { getLicenseService } = await import('./LicenseService') + const { Feature } = await import('@shared/types') + const license = getLicenseService() + const access = license.canUse(Feature.LOCAL_RAG) + if (!access.allowed) { + license.promptUpgrade(Feature.LOCAL_RAG, 'tier_required') + throw new D3ROError(ErrorCode.FeatureNotAvailable, 'Pro+ required for Local RAG') + } + } catch (err) { + if (err instanceof D3ROError) throw err + } + + const ext = path.extname(filePath).toLowerCase() + const supportedTypes: Record = { + '.txt': 'txt', + '.md': 'md', + '.pdf': 'pdf', + '.docx': 'docx', + } + + const fileType = supportedTypes[ext] + if (!fileType) { + throw new D3ROError(ErrorCode.RAGUnsupportedFormat, `Unsupported format: ${ext}. Supported: .txt, .md, .pdf, .docx`) + } + + const fileName = path.basename(filePath) + const docId = nanoid() + + // 텍스트 추출 + let content: string + try { + content = await this._extractText(filePath, fileType) + } catch (err) { + if (err instanceof D3ROError) throw err + throw new D3ROError(ErrorCode.RAGIndexingFailed, `Text extraction failed: ${err instanceof Error ? err.message : String(err)}`) + } + + if (!content || content.trim().length < 20) { + throw new D3ROError(ErrorCode.RAGIndexingFailed, 'No readable text content found in document') + } + + logger.info(`RAG text extracted: ${fileName} (${content.length} chars)`) + + // 청킹 + const chunks = this._chunkText(content) + if (chunks.length === 0) { + throw new D3ROError(ErrorCode.RAGIndexingFailed, 'Document produced no valid text chunks') + } + + // DB에 문서 레코드 삽입 + const db = getDatabase() + db.insert(ragDocuments).values({ + id: docId, + fileName, + filePath, + fileType, + chunkCount: chunks.length, + indexed: false, + indexedAt: null, + addedAt: Date.now(), + }).run() + + // 비동기 인덱싱 (임베딩 생성) + this._indexDocument(docId, fileName, chunks).catch((err) => { + logger.error(`Indexing failed for ${fileName}:`, err) + }) + + return { + id: docId, + fileName, + filePath, + fileType, + chunkCount: chunks.length, + indexed: false, + indexedAt: null, + addedAt: Date.now(), + } + } + + /** + * 문서 제거 (청크 포함) + */ + removeDocument(documentId: string): void { + const db = getDatabase() + db.delete(ragChunks).where(eq(ragChunks.documentId, documentId)).run() + db.delete(ragDocuments).where(eq(ragDocuments.id, documentId)).run() + logger.info(`RAG document removed: ${documentId}`) + } + + /** + * 문서 재인덱싱 + */ + async reindex(documentId: string): Promise { + const db = getDatabase() + const rows = db.select().from(ragDocuments).where(eq(ragDocuments.id, documentId)).all() + if (rows.length === 0) { + throw new D3ROError(ErrorCode.RAGDocumentNotFound, 'Document not found') + } + const doc = rows[0] + + // 기존 청크 삭제 + db.delete(ragChunks).where(eq(ragChunks.documentId, documentId)).run() + + // 텍스트 재추출 + 재인덱싱 + const content = await this._extractText(doc.filePath, doc.fileType as RAGDocument['fileType']) + const chunks = this._chunkText(content) + + db.update(ragDocuments) + .set({ chunkCount: chunks.length, indexed: false }) + .where(eq(ragDocuments.id, documentId)) + .run() + + await this._indexDocument(documentId, doc.fileName, chunks) + } + + /** + * 벡터 검색 + LLM 답변 생성 + */ + async query(queryText: string, topK: number = DEFAULT_TOP_K): Promise { + this._state = 'querying' + + try { + // 쿼리 임베딩 + const queryEmbedding = await this._embed(queryText) + + // 모든 청크에서 코사인 유사도 계산 + const db = getDatabase() + const allChunks = db.select().from(ragChunks).all() + const allDocs = db.select().from(ragDocuments).all() + const docMap = new Map(allDocs.map((d) => [d.id, d.fileName])) + + const scored = allChunks.map((chunk) => { + const embedding = JSON.parse(chunk.embedding) as number[] + const similarity = this._cosineSimilarity(queryEmbedding, embedding) + return { + documentId: chunk.documentId, + fileName: docMap.get(chunk.documentId) ?? 'unknown', + content: chunk.content, + similarity, + } + }) + + // 상위 topK + scored.sort((a, b) => b.similarity - a.similarity) + const topResults = scored.slice(0, topK) + + // LLM에 컨텍스트 주입 + const context = topResults + .map((r, i) => `[${i + 1}] (${r.fileName})\n${r.content}`) + .join('\n\n') + + const systemPrompt = `You are a helpful assistant. Answer the user's question based on the following documents. If the documents don't contain relevant information, say so. Respond in the same language as the question. + +Documents: +${context}` + + const llmService = getLocalLLMService() + const result = await llmService.generate(queryText, { systemPrompt }) + + return { + query: queryText, + results: topResults, + answer: result.text.trim(), + } + } finally { + this._state = 'idle' + } + } + + // ── 내부 메서드 ── + + private async _indexDocument(docId: string, fileName: string, chunks: string[]): Promise { + this._state = 'indexing' + const db = getDatabase() + + logger.info(`RAG indexing started: ${fileName} (${chunks.length} chunks)`) + + // 초기 진행률 즉시 전송 + this._sendToRenderer(IPC_CHANNELS.RAG.INDEX_PROGRESS, { + documentId: docId, + fileName, + currentChunk: 0, + totalChunks: chunks.length, + percent: 0, + } as RAGIndexProgress) + + let successCount = 0 + for (let i = 0; i < chunks.length; i++) { + try { + const embedding = await this._embed(chunks[i]) + + db.insert(ragChunks).values({ + id: nanoid(), + documentId: docId, + content: chunks[i], + embedding: JSON.stringify(embedding), + chunkIndex: i, + }).run() + + successCount++ + } catch (err) { + logger.warn(`RAG embedding failed for chunk ${i}/${chunks.length} of ${fileName}:`, err) + // 개별 청크 실패는 건너뛰고 계속 진행 + } + + const progress: RAGIndexProgress = { + documentId: docId, + fileName, + currentChunk: i + 1, + totalChunks: chunks.length, + percent: Math.round(((i + 1) / chunks.length) * 100), + } + this._sendToRenderer(IPC_CHANNELS.RAG.INDEX_PROGRESS, progress) + + // 이벤트 루프 양보 (UI 블로킹 방지) + await new Promise((r) => setTimeout(r, 10)) + } + + // 인덱싱 완료 표시 + db.update(ragDocuments) + .set({ indexed: true, indexedAt: Date.now(), chunkCount: successCount }) + .where(eq(ragDocuments.id, docId)) + .run() + + this._state = 'idle' + this._sendToRenderer(IPC_CHANNELS.RAG.INDEX_COMPLETE, { documentId: docId, fileName }) + logger.info(`RAG indexing complete: ${fileName} (${successCount}/${chunks.length} chunks embedded)`) + } + + /** + * Ollama /api/embed 엔드포인트로 텍스트 임베딩 + */ + private async _embed(text: string): Promise { + const serverUrl = configGet('ollamaServerUrl') + + try { + const response = await fetch(`${serverUrl}/api/embed`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + model: EMBED_MODEL, + input: text, + }), + signal: AbortSignal.timeout(30000), + }) + + if (!response.ok) { + throw new D3ROError(ErrorCode.RAGEmbeddingFailed, `Embed API error: ${response.status}`) + } + + const data = (await response.json()) as { embeddings: number[][] } + if (!data.embeddings || data.embeddings.length === 0) { + throw new D3ROError(ErrorCode.RAGEmbeddingFailed, 'No embeddings returned') + } + + return data.embeddings[0] + } catch (err) { + if (err instanceof D3ROError) throw err + throw new D3ROError( + ErrorCode.RAGEmbeddingFailed, + `Embedding failed: ${err instanceof Error ? err.message : String(err)}`, + ) + } + } + + private _chunkText(text: string): string[] { + // 텍스트 크기 제한 (500KB 초과 시 잘라냄) + const MAX_TEXT_LENGTH = 500000 + const safeText = text.length > MAX_TEXT_LENGTH ? text.slice(0, MAX_TEXT_LENGTH) : text + + const chunks: string[] = [] + let start = 0 + const MAX_CHUNKS = 2000 + while (start < safeText.length && chunks.length < MAX_CHUNKS) { + const end = Math.min(start + CHUNK_SIZE, safeText.length) + const chunk = safeText.slice(start, end).trim() + if (chunk.length > 10) { + chunks.push(chunk) + } + start = end - CHUNK_OVERLAP + if (start >= safeText.length) break + } + return chunks + } + + private async _extractText(filePath: string, fileType: string): Promise { + const { promises: fsp } = await import('fs') + + if (fileType === 'txt' || fileType === 'md') { + return fsp.readFile(filePath, 'utf-8') + } + + if (fileType === 'pdf') { + try { + const buffer = await fsp.readFile(filePath) + const text = this._extractPdfText(buffer) + if (text.trim().length < 10) { + throw new Error('No readable text found in PDF') + } + return text.trim() + } catch (err) { + logger.error('PDF parsing failed:', err) + throw new D3ROError(ErrorCode.RAGUnsupportedFormat, 'PDF parsing failed. Ensure the PDF contains readable text.') + } + } + + if (fileType === 'docx') { + try { + const buffer = await fsp.readFile(filePath) + // DOCX는 ZIP 내 XML — w:t 태그에서 텍스트 추출 + const raw = buffer.toString('utf-8') + const matches = raw.match(/]*>([^<]*)<\/w:t>/g) + if (matches) { + const text = matches + .map((m) => m.replace(/<[^>]+>/g, '')) + .join(' ') + return text.trim() + } + // XML 태그 제거 폴백 + return raw.replace(/<[^>]+>/g, ' ').replace(/\s+/g, ' ').trim().slice(0, 500000) + } catch { + throw new D3ROError(ErrorCode.RAGUnsupportedFormat, 'DOCX parsing failed') + } + } + + throw new D3ROError(ErrorCode.RAGUnsupportedFormat, `Unsupported: ${fileType}`) + } + + /** + * PDF 바이너리에서 텍스트 추출 (순수 JS + zlib). + * FlateDecode 압축 해제 후 BT...ET 블록 내 Tj/TJ 파싱. + */ + private _extractPdfText(buffer: Buffer): string { + const zlib = require('zlib') as typeof import('zlib') + const raw = buffer.toString('binary') + const textParts: string[] = [] + + // 스트림 블록 추출 — 바이너리 오프셋 기반 + const streamMarker = 'stream\r\n' + const streamMarker2 = 'stream\n' + const endMarker = 'endstream' + + let pos = 0 + while (pos < raw.length) { + let streamStart = raw.indexOf(streamMarker, pos) + let offset = streamMarker.length + if (streamStart === -1) { + streamStart = raw.indexOf(streamMarker2, pos) + offset = streamMarker2.length + } + if (streamStart === -1) break + + const dataStart = streamStart + offset + const streamEnd = raw.indexOf(endMarker, dataStart) + if (streamEnd === -1) break + + const streamData = Buffer.from(raw.slice(dataStart, streamEnd), 'binary') + pos = streamEnd + endMarker.length + + // FlateDecode 해제 시도 + let decoded: string + try { + const inflated = zlib.inflateSync(streamData) + decoded = inflated.toString('binary') + } catch { + // 압축 안 된 스트림 + decoded = streamData.toString('binary') + } + + // BT...ET 블록 파싱 + this._extractTextFromStream(decoded, textParts) + } + + // PDF 이스케이프 디코딩 + 정리 + let text = textParts.join(' ') + text = text + .replace(/\\n/g, '\n') + .replace(/\\r/g, '\r') + .replace(/\\t/g, '\t') + .replace(/\\\(/g, '(') + .replace(/\\\)/g, ')') + .replace(/\\\\/g, '\\') + .replace(/[^\x20-\x7E\u00A0-\u00FF\u3000-\u9FFF\uAC00-\uD7AF\n]/g, ' ') + .replace(/\s+/g, ' ') + + return text.trim() + } + + private _extractTextFromStream(content: string, textParts: string[]): void { + const btRegex = /BT([\s\S]*?)ET/g + let btMatch: RegExpExecArray | null = null + while ((btMatch = btRegex.exec(content)) !== null) { + const block = btMatch[1] + + // Tj: (text) Tj + const tjRegex = /\(([^)]*)\)\s*Tj/g + let tjMatch: RegExpExecArray | null = null + while ((tjMatch = tjRegex.exec(block)) !== null) { + if (tjMatch[1].trim()) textParts.push(tjMatch[1]) + } + + // TJ: [(text) num (text)] TJ + const tjArrayRegex = /\[(.*?)\]\s*TJ/g + let tjArrayMatch: RegExpExecArray | null = null + while ((tjArrayMatch = tjArrayRegex.exec(block)) !== null) { + const items = tjArrayMatch[1] + const itemRegex = /\(([^)]*)\)/g + let itemMatch: RegExpExecArray | null = null + while ((itemMatch = itemRegex.exec(items)) !== null) { + if (itemMatch[1].trim()) textParts.push(itemMatch[1]) + } + } + + // ' 연산자: (text) ' + const quoteRegex = /\(([^)]*)\)\s*'/g + let quoteMatch: RegExpExecArray | null = null + while ((quoteMatch = quoteRegex.exec(block)) !== null) { + if (quoteMatch[1].trim()) textParts.push(quoteMatch[1]) + } + } + } + + private _cosineSimilarity(a: number[], b: number[]): number { + if (a.length !== b.length) return 0 + let dotProduct = 0 + let normA = 0 + let normB = 0 + for (let i = 0; i < a.length; i++) { + dotProduct += a[i] * b[i] + normA += a[i] * a[i] + normB += b[i] * b[i] + } + const denom = Math.sqrt(normA) * Math.sqrt(normB) + return denom === 0 ? 0 : dotProduct / denom + } + + private _sendToRenderer(channel: string, data: unknown): void { + try { + const mainWindow = getMainWindow() + if (mainWindow && !mainWindow.isDestroyed()) { + mainWindow.webContents.send(channel, data) + } + } catch { + // ignore + } + } + + dispose(): void { + this.removeAllListeners() + } +} + +// ── 싱글톤 ── +let instance: RAGService | null = null + +export function getRAGService(): RAGService { + if (!instance) { + instance = new RAGService() + } + return instance +} diff --git a/src/main/services/TTSPlaybackService.ts b/src/main/services/TTSPlaybackService.ts new file mode 100644 index 0000000..7231eb3 --- /dev/null +++ b/src/main/services/TTSPlaybackService.ts @@ -0,0 +1,152 @@ +// src/main/services/TTSPlaybackService.ts +// Phase 13.1: TTS 재생 서비스 +// Windows SAPI (PowerShell) 기반 로컬 TTS. +// 온라인 불필요, 완전 로컬. 문장 단위 큐 재생. + +import { EventEmitter } from 'events' +import { spawn, type ChildProcess } from 'child_process' +import { getLogger } from './LoggerService' +import { configGet } from './ConfigService' +import { D3ROError, ErrorCode } from '@shared/errors' + +const logger = getLogger('TTSPlaybackService') + +class TTSPlaybackService extends EventEmitter { + private _speaking = false + private _queue: string[] = [] + private _currentProcess: ChildProcess | null = null + private _cancelled = false + + get isSpeaking(): boolean { + return this._speaking + } + + /** + * 텍스트를 음성으로 재생 (Windows SAPI). + * 큐에 추가되어 순차 재생된다. + */ + async speak(text: string): Promise { + if (!text.trim()) return + this._queue.push(text.trim()) + if (!this._speaking) { + await this._processQueue() + } + } + + /** + * 문장 배열을 순차 재생. + * LLM 스트리밍에서 문장 단위로 호출한다. + */ + async speakSentences(sentences: string[]): Promise { + for (const sentence of sentences) { + if (this._cancelled) break + this._queue.push(sentence.trim()) + } + if (!this._speaking) { + await this._processQueue() + } + } + + /** + * 재생 중단 + 큐 비우기. + */ + stop(): void { + this._cancelled = true + this._queue = [] + if (this._currentProcess) { + this._currentProcess.kill() + this._currentProcess = null + } + this._speaking = false + this.emit('stopped') + } + + private async _processQueue(): Promise { + this._speaking = true + this._cancelled = false + this.emit('started') + + while (this._queue.length > 0 && !this._cancelled) { + const text = this._queue.shift()! + try { + await this._speakOne(text) + } catch (err) { + logger.warn('TTS playback failed for segment:', err) + } + } + + this._speaking = false + if (!this._cancelled) { + this.emit('finished') + } + } + + /** + * PowerShell SAPI로 단일 텍스트 재생. + */ + private _speakOne(text: string): Promise { + return new Promise((resolve, reject) => { + // 텍스트를 PowerShell 안전 문자열로 이스케이프 + const escaped = text + .replace(/'/g, "''") + .replace(/\n/g, ' ') + .replace(/\r/g, '') + + const rate = this._getRate() + + const script = ` + Add-Type -AssemblyName System.Speech + $synth = New-Object System.Speech.Synthesis.SpeechSynthesizer + $synth.Rate = ${rate} + $synth.Speak('${escaped}') + $synth.Dispose() + ` + + this._currentProcess = spawn('powershell', [ + '-NoProfile', + '-NonInteractive', + '-Command', + script, + ], { stdio: 'pipe' }) + + this._currentProcess.on('close', (code) => { + this._currentProcess = null + if (code === 0 || this._cancelled) { + resolve() + } else { + reject(new D3ROError(ErrorCode.ConversationTTSFailed, `TTS exited with code ${code}`)) + } + }) + + this._currentProcess.on('error', (err) => { + this._currentProcess = null + reject(new D3ROError(ErrorCode.ConversationTTSFailed, `TTS error: ${err.message}`)) + }) + }) + } + + /** + * SAPI Rate: -10(매우 느림) ~ 10(매우 빠름), 기본 0 + */ + private _getRate(): number { + const speed = configGet('ttsSpeed') as number | undefined + if (!speed || speed === 1.0) return 0 + // 0.5 → -5, 1.0 → 0, 2.0 → 5 + return Math.round((speed - 1.0) * 5) + } + + dispose(): void { + this.stop() + this.removeAllListeners() + } +} + +// ── 싱글톤 ── +let instance: TTSPlaybackService | null = null + +export function getTTSPlaybackService(): TTSPlaybackService { + if (!instance) { + instance = new TTSPlaybackService() + } + return instance +} diff --git a/src/main/services/VoiceActionService.ts b/src/main/services/VoiceActionService.ts new file mode 100644 index 0000000..b0edbc2 --- /dev/null +++ b/src/main/services/VoiceActionService.ts @@ -0,0 +1,376 @@ +// src/main/services/VoiceActionService.ts +// Phase 13.3: OS 자동화 — 음성 → LLM이 JSON 액션 플랜 생성 → 실행 +// 사전 정의 명령 + LLM 자유 해석. 위험 액션은 차단. + +import { EventEmitter } from 'events' +import { exec } from 'child_process' +import { shell } from 'electron' +import { nanoid } from 'nanoid' +import { getLogger } from './LoggerService' +import { getLocalLLMService } from './LocalLLMService' +import { configGet } from './ConfigService' +import { getMainWindow } from '../windows/WindowManager' +import { IPC_CHANNELS } from '@shared/ipc-channels' +import { D3ROError, ErrorCode } from '@shared/errors' +import type { + VoiceActionPlan, + VoiceActionPreset, + VoiceActionHistoryEntry, + VoiceActionPlannedEvent, + VoiceActionExecutedEvent, + VoiceActionErrorEvent, +} from '@shared/types' + +const logger = getLogger('VoiceActionService') + +/** 위험 명령어 블랙리스트 (실행 차단) */ +const BLOCKED_COMMANDS = [ + 'rm ', 'del ', 'format ', 'rmdir', 'rd ', + 'shutdown', 'restart', 'taskkill', + 'reg delete', 'reg add', + 'net user', 'net localgroup', +] + +/** LLM에 보낼 시스템 프롬프트 */ +const ACTION_SYSTEM_PROMPT = `You are an OS automation assistant. Parse the user's voice command and output a JSON action plan. + +Output EXACTLY one JSON object (no markdown, no explanation): +{ + "action": "open_app" | "open_url" | "open_file" | "keyboard_shortcut" | "type_text" | "system_command", + "target": "", + "description": "", + "safe": true | false +} + +Rules: +- "open_app": target = app name (e.g., "notepad", "chrome", "code") +- "open_url": target = full URL (e.g., "https://google.com") +- "open_file": target = file path +- "keyboard_shortcut": target = key combo (e.g., "ctrl+c", "alt+tab") +- "type_text": target = text to type +- "system_command": target = shell command +- Set safe=false for destructive operations (delete, format, shutdown, etc.) + +If the command is unclear, output: {"action": "type_text", "target": "", "description": "Could not parse command", "safe": true}` + +/** 프리셋 명령어 (LLM 없이 바로 실행) */ +const PRESETS: VoiceActionPreset[] = [ + { + keywords: ['크롬 열어', '크롬', 'chrome', 'open chrome'], + action: { action: 'open_app', target: 'chrome', description: 'Open Chrome browser', safe: true }, + }, + { + keywords: ['메모장 열어', '메모장', 'notepad', 'open notepad'], + action: { action: 'open_app', target: 'notepad', description: 'Open Notepad', safe: true }, + }, + { + keywords: ['탐색기 열어', '탐색기', 'explorer', 'open explorer'], + action: { action: 'open_app', target: 'explorer', description: 'Open File Explorer', safe: true }, + }, + { + keywords: ['볼륨 올려', '소리 올려', 'volume up'], + action: { action: 'keyboard_shortcut', target: 'volumeup', description: 'Volume up', safe: true }, + }, + { + keywords: ['볼륨 내려', '소리 내려', 'volume down'], + action: { action: 'keyboard_shortcut', target: 'volumedown', description: 'Volume down', safe: true }, + }, + { + keywords: ['음소거', 'mute'], + action: { action: 'keyboard_shortcut', target: 'volumemute', description: 'Toggle mute', safe: true }, + }, +] + +class VoiceActionService extends EventEmitter { + private _history: VoiceActionHistoryEntry[] = [] + private _enabled = false + + get isEnabled(): boolean { + return this._enabled + } + + setEnabled(enabled: boolean): void { + this._enabled = enabled + } + + getPresets(): VoiceActionPreset[] { + return PRESETS + } + + getHistory(): VoiceActionHistoryEntry[] { + return [...this._history].reverse() + } + + clearHistory(): void { + this._history = [] + } + + /** + * 음성 텍스트로부터 액션 계획 → 실행 + */ + async execute(text: string): Promise { + // 라이센스 체크 + try { + const { getLicenseService } = await import('./LicenseService') + const { Feature } = await import('@shared/types') + const license = getLicenseService() + const access = license.canUse(Feature.OS_AUTOMATION) + if (!access.allowed) { + license.promptUpgrade(Feature.OS_AUTOMATION, 'tier_required') + throw new D3ROError(ErrorCode.FeatureNotAvailable, 'Pro+ required for OS automation') + } + } catch (err) { + if (err instanceof D3ROError) throw err + } + + // 1. 프리셋 매칭 + const preset = this._matchPreset(text) + if (preset) { + await this._executePlan(preset.action, text) + return + } + + // 2. LLM으로 액션 플랜 생성 + try { + const llmService = getLocalLLMService() + const result = await llmService.generate(text, { + systemPrompt: ACTION_SYSTEM_PROMPT, + temperature: 0.1, + }) + + const plan = this._parsePlan(result.text) + if (!plan) { + this._emitError(text, 'Failed to parse action plan from LLM response') + return + } + + // 안전장치 + if (!plan.safe) { + const entry: VoiceActionHistoryEntry = { + id: nanoid(), + userText: text, + plan, + executed: false, + timestamp: Date.now(), + } + this._history.push(entry) + this._sendToRenderer(IPC_CHANNELS.VOICE_ACTION.ACTION_PLANNED, { plan, userText: text }) + logger.warn(`Voice action blocked (unsafe): ${plan.description}`) + return + } + + await this._executePlan(plan, text) + } catch (err) { + this._emitError(text, err instanceof Error ? err.message : String(err)) + } + } + + private _matchPreset(text: string): VoiceActionPreset | null { + const lower = text.toLowerCase().trim() + for (const preset of PRESETS) { + for (const keyword of preset.keywords) { + if (lower.includes(keyword.toLowerCase())) { + return preset + } + } + } + return null + } + + private _parsePlan(llmOutput: string): VoiceActionPlan | null { + try { + // JSON 블록 추출 + const jsonMatch = llmOutput.match(/\{[\s\S]*\}/) + if (!jsonMatch) return null + + const parsed = JSON.parse(jsonMatch[0]) as Record + if (!parsed.action || !parsed.target) return null + + return { + action: parsed.action as VoiceActionPlan['action'], + target: String(parsed.target), + description: String(parsed.description ?? ''), + safe: parsed.safe !== false, + } + } catch { + return null + } + } + + private async _executePlan(plan: VoiceActionPlan, userText: string): Promise { + // 차단된 명령어 체크 + if (plan.action === 'system_command') { + const targetLower = plan.target.toLowerCase() + for (const blocked of BLOCKED_COMMANDS) { + if (targetLower.includes(blocked)) { + plan.safe = false + this._sendToRenderer(IPC_CHANNELS.VOICE_ACTION.ACTION_PLANNED, { plan, userText }) + logger.warn(`Voice action blocked: ${plan.target}`) + return + } + } + } + + this._sendToRenderer(IPC_CHANNELS.VOICE_ACTION.ACTION_PLANNED, { plan, userText } as VoiceActionPlannedEvent) + + try { + switch (plan.action) { + case 'open_app': + await this._openApp(plan.target) + break + case 'open_url': + await shell.openExternal(plan.target) + break + case 'open_file': + await shell.openPath(plan.target) + break + case 'keyboard_shortcut': + await this._simulateKeyboard(plan.target) + break + case 'type_text': + // @nut-tree 사용 (lazy import) + await this._typeText(plan.target) + break + case 'system_command': + await this._runCommand(plan.target) + break + } + + const entry: VoiceActionHistoryEntry = { + id: nanoid(), + userText, + plan, + executed: true, + timestamp: Date.now(), + } + this._history.push(entry) + if (this._history.length > 50) this._history.shift() + + const event: VoiceActionExecutedEvent = { plan, success: true } + this._sendToRenderer(IPC_CHANNELS.VOICE_ACTION.ACTION_EXECUTED, event) + this.emit('action-executed', event) + logger.info(`Voice action executed: ${plan.action} → ${plan.target}`) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + this._emitError(userText, msg) + throw new D3ROError(ErrorCode.VoiceActionExecutionFailed, msg) + } + } + + private _openApp(appName: string): Promise { + return new Promise((resolve, reject) => { + const cmd = `start "" "${appName}"` + exec(cmd, { shell: 'cmd.exe' }, (err) => { + if (err) reject(err) + else resolve() + }) + }) + } + + private _simulateKeyboard(combo: string): Promise { + return new Promise((resolve, reject) => { + // PowerShell SendKeys 사용 + const keys = combo.toLowerCase() + let sendKeysStr = '' + + if (keys === 'volumeup') { + // nircmd 대안: PowerShell로 볼륨 조절 + const script = ` + $wshell = New-Object -ComObject WScript.Shell + $wshell.SendKeys([char]175) + ` + exec(`powershell -NoProfile -Command "${script}"`, (err) => { + if (err) reject(err) + else resolve() + }) + return + } + if (keys === 'volumedown') { + const script = ` + $wshell = New-Object -ComObject WScript.Shell + $wshell.SendKeys([char]174) + ` + exec(`powershell -NoProfile -Command "${script}"`, (err) => { + if (err) reject(err) + else resolve() + }) + return + } + if (keys === 'volumemute') { + const script = ` + $wshell = New-Object -ComObject WScript.Shell + $wshell.SendKeys([char]173) + ` + exec(`powershell -NoProfile -Command "${script}"`, (err) => { + if (err) reject(err) + else resolve() + }) + return + } + + // 일반 키보드 단축키 (ctrl+c 등) + if (keys.includes('ctrl')) sendKeysStr += '^' + if (keys.includes('alt')) sendKeysStr += '%' + if (keys.includes('shift')) sendKeysStr += '+' + + const key = keys.replace(/ctrl\+|alt\+|shift\+/g, '').trim() + sendKeysStr += key + + const script = ` + $wshell = New-Object -ComObject WScript.Shell + $wshell.SendKeys('${sendKeysStr}') + ` + exec(`powershell -NoProfile -Command "${script}"`, (err) => { + if (err) reject(err) + else resolve() + }) + }) + } + + private async _typeText(text: string): Promise { + try { + const { getTextInsertService } = await import('./TextInsertService') + await getTextInsertService().insertText(text) + } catch (err) { + throw new D3ROError(ErrorCode.VoiceActionExecutionFailed, `Type text failed: ${err instanceof Error ? err.message : String(err)}`) + } + } + + private _runCommand(command: string): Promise { + return new Promise((resolve, reject) => { + exec(command, { timeout: 10000 }, (err) => { + if (err) reject(err) + else resolve() + }) + }) + } + + private _emitError(userText: string, message: string): void { + const event: VoiceActionErrorEvent = { message, userText } + this._sendToRenderer(IPC_CHANNELS.VOICE_ACTION.ACTION_ERROR, event) + this.emit('error', event) + } + + private _sendToRenderer(channel: string, data: unknown): void { + try { + const mainWindow = getMainWindow() + if (mainWindow && !mainWindow.isDestroyed()) { + mainWindow.webContents.send(channel, data) + } + } catch { /* ignore */ } + } + + dispose(): void { + this.removeAllListeners() + } +} + +// ── 싱글톤 ── +let instance: VoiceActionService | null = null + +export function getVoiceActionService(): VoiceActionService { + if (!instance) { + instance = new VoiceActionService() + } + return instance +} diff --git a/src/main/services/VoiceConversationService.ts b/src/main/services/VoiceConversationService.ts new file mode 100644 index 0000000..4ce9004 --- /dev/null +++ b/src/main/services/VoiceConversationService.ts @@ -0,0 +1,376 @@ +// src/main/services/VoiceConversationService.ts +// Phase 13.1: 음성 대화 모드 — STT → LLM(chat) → TTS 루프 +// 싱글톤 + EventEmitter. 대화 히스토리 최근 10턴 유지. + +import { EventEmitter } from 'events' +import { nanoid } from 'nanoid' +import { getLogger } from './LoggerService' +import { getLocalLLMService } from './LocalLLMService' +import { getLocalSTTService } from './LocalSTTService' +import { getAudioCaptureService } from './AudioCaptureService' +import { getTTSPlaybackService } from './TTSPlaybackService' +import { configGet } from './ConfigService' +import { getMainWindow } from '../windows/WindowManager' +import { IPC_CHANNELS } from '@shared/ipc-channels' +import { D3ROError, ErrorCode } from '@shared/errors' +import type { + ConversationState, + ConversationMessage, + ConversationSessionInfo, + ConversationAssistantDelta, + ConversationAssistantMessage, + ConversationError, +} from '@shared/types' + +const logger = getLogger('VoiceConversationService') + +/** 대화 히스토리 최대 턴 수 (user+assistant 쌍) */ +const MAX_HISTORY_TURNS = 10 +/** 시스템 프롬프트 */ +const SYSTEM_PROMPT = `You are D3RO, a helpful local AI voice assistant. Respond concisely and naturally, as if having a spoken conversation. Keep answers brief (2-3 sentences) unless the user asks for detail. Respond in the same language the user speaks.` + +class VoiceConversationService extends EventEmitter { + private _state: ConversationState = 'idle' + private _messages: ConversationMessage[] = [] + private _isActive = false + private _audioBuffers: Buffer[] = [] + private _audioListenerBound = false + + get state(): ConversationState { + return this._state + } + + get isActive(): boolean { + return this._isActive + } + + getSessionInfo(): ConversationSessionInfo { + return { + state: this._state, + messages: [...this._messages], + isActive: this._isActive, + } + } + + getHistory(): ConversationMessage[] { + return [...this._messages] + } + + /** + * 대화 세션 시작. 마이크 캡처를 시작하고 listening 상태로 진입. + */ + async startSession(): Promise { + if (this._isActive) { + throw new D3ROError(ErrorCode.ConversationSessionAlreadyActive, 'Conversation session already active') + } + + // 라이센스 체크 + try { + const { getLicenseService } = await import('./LicenseService') + const { Feature } = await import('@shared/types') + const license = getLicenseService() + const access = license.canUse(Feature.VOICE_CONVERSATION) + if (!access.allowed) { + license.promptUpgrade(Feature.VOICE_CONVERSATION, 'tier_required') + throw new D3ROError(ErrorCode.FeatureNotAvailable, 'Pro+ required for voice conversation') + } + } catch (err) { + if (err instanceof D3ROError) throw err + } + + this._isActive = true + this._setState('listening') + this._startListening() + logger.info('Voice conversation session started') + } + + /** + * 대화 세션 종료. + */ + stopSession(): void { + if (!this._isActive) return + + this._stopListening() + getTTSPlaybackService().stop() + getLocalLLMService().cancelGeneration() + + this._isActive = false + this._setState('idle') + logger.info('Voice conversation session stopped') + } + + /** + * 텍스트 메시지를 직접 전송 (키보드 입력). + */ + async sendTextMessage(text: string): Promise { + if (!this._isActive) { + throw new D3ROError(ErrorCode.ConversationNoActiveSession, 'No active conversation session') + } + + await this._processUserMessage(text) + } + + /** + * 현재 LLM 응답 또는 TTS 재생을 취소. + */ + cancelResponse(): void { + getLocalLLMService().cancelGeneration() + getTTSPlaybackService().stop() + if (this._isActive) { + this._setState('listening') + this._startListening() + } + } + + /** + * 대화 히스토리 초기화. + */ + clearHistory(): void { + this._messages = [] + this._sendToRenderer(IPC_CHANNELS.VOICE_CONVERSATION.STATE_CHANGED, this.getSessionInfo()) + } + + // ── 내부 로직 ── + + private _setState(state: ConversationState): void { + this._state = state + this._sendToRenderer(IPC_CHANNELS.VOICE_CONVERSATION.STATE_CHANGED, this.getSessionInfo()) + this.emit('state-changed', state) + } + + private _startListening(): void { + this._audioBuffers = [] + const audioService = getAudioCaptureService() + + if (!this._audioListenerBound) { + audioService.on('audio-data', this._onAudioData) + this._audioListenerBound = true + } + + audioService.start().catch((err) => { + logger.error('Failed to start audio capture for conversation:', err) + this._emitError('stt', 'Failed to start microphone') + }) + } + + private _stopListening(): void { + const audioService = getAudioCaptureService() + if (this._audioListenerBound) { + audioService.off('audio-data', this._onAudioData) + this._audioListenerBound = false + } + audioService.stop().catch(() => { /* ignore */ }) + this._audioBuffers = [] + } + + private _onAudioData = (payload: { buffer: Buffer }): void => { + if (this._state !== 'listening') return + this._audioBuffers.push(payload.buffer) + } + + /** + * 녹음 완료 (UI에서 stop 버튼 클릭 시 호출). + * 수집된 오디오를 STT로 전사 후 LLM 대화 진행. + */ + async finishListening(): Promise { + if (this._state !== 'listening' || this._audioBuffers.length === 0) return + + this._stopListening() + this._setState('thinking') + + const audioBuffer = Buffer.concat(this._audioBuffers) + this._audioBuffers = [] + + // 최소 오디오 길이 체크 (500ms @ 16kHz 16bit mono) + const minBytes = 16000 * 2 * 0.5 + if (audioBuffer.length < minBytes) { + this._setState('listening') + this._startListening() + return + } + + try { + // STT + const sttService = getLocalSTTService() + const language = (configGet('sttLanguage') as string | undefined) ?? 'auto' + const result = await sttService.transcribe(audioBuffer, { language, vadFilter: true }) + + if (!result.text || result.text.trim().length === 0) { + this._setState('listening') + this._startListening() + return + } + + await this._processUserMessage(result.text.trim()) + } catch (err) { + logger.error('STT failed in conversation:', err) + this._emitError('stt', err instanceof Error ? err.message : 'STT failed') + this._setState('listening') + this._startListening() + } + } + + private async _processUserMessage(text: string): Promise { + // 사용자 메시지 추가 + const userMsg: ConversationMessage = { + id: nanoid(), + role: 'user', + content: text, + timestamp: Date.now(), + } + this._messages.push(userMsg) + this._trimHistory() + + this._sendToRenderer(IPC_CHANNELS.VOICE_CONVERSATION.USER_MESSAGE, userMsg) + this._setState('thinking') + + try { + // Ollama /api/chat 호출 (스트리밍) + const llmService = getLocalLLMService() + const chatMessages = this._buildChatMessages() + + const assistantMsgId = nanoid() + let accumulated = '' + const ttsSentences: string[] = [] + let sentenceBuffer = '' + + const generator = llmService.chatStream(chatMessages) + + for await (const token of generator) { + accumulated += token + + // 렌더러에 델타 전송 + const delta: ConversationAssistantDelta = { + messageId: assistantMsgId, + delta: token, + accumulated, + } + this._sendToRenderer(IPC_CHANNELS.VOICE_CONVERSATION.ASSISTANT_DELTA, delta) + + // 문장 단위 TTS 큐잉 + sentenceBuffer += token + const sentenceEnd = sentenceBuffer.match(/[.!?。！？]\s*/g) + if (sentenceEnd) { + const lastEnd = sentenceBuffer.lastIndexOf(sentenceEnd[sentenceEnd.length - 1]) + const completeSentence = sentenceBuffer.slice( + 0, + lastEnd + sentenceEnd[sentenceEnd.length - 1].length, + ) + sentenceBuffer = sentenceBuffer.slice( + lastEnd + sentenceEnd[sentenceEnd.length - 1].length, + ) + if (completeSentence.trim()) { + ttsSentences.push(completeSentence.trim()) + } + } + } + + // 남은 텍스트도 TTS 큐에 추가 + if (sentenceBuffer.trim()) { + ttsSentences.push(sentenceBuffer.trim()) + } + + // 어시스턴트 메시지 저장 + const assistantMsg: ConversationMessage = { + id: assistantMsgId, + role: 'assistant', + content: accumulated, + timestamp: Date.now(), + } + this._messages.push(assistantMsg) + this._trimHistory() + + const completeEvent: ConversationAssistantMessage = { + messageId: assistantMsgId, + content: accumulated, + } + this._sendToRenderer(IPC_CHANNELS.VOICE_CONVERSATION.ASSISTANT_MESSAGE, completeEvent) + + // TTS 재생 + if (ttsSentences.length > 0) { + this._setState('speaking') + this._sendToRenderer(IPC_CHANNELS.VOICE_CONVERSATION.TTS_STARTED, {}) + + const ttsService = getTTSPlaybackService() + await ttsService.speakSentences(ttsSentences) + + this._sendToRenderer(IPC_CHANNELS.VOICE_CONVERSATION.TTS_FINISHED, {}) + } + + // 재생 완료 → 다시 listening + if (this._isActive) { + this._setState('listening') + this._startListening() + } + } catch (err) { + logger.error('LLM chat failed in conversation:', err) + this._emitError('llm', err instanceof Error ? err.message : 'LLM failed') + if (this._isActive) { + this._setState('listening') + this._startListening() + } + } + } + + private _buildChatMessages(): Array<{ role: string; content: string }> { + const chatMsgs: Array<{ role: string; content: string }> = [ + { role: 'system', content: SYSTEM_PROMPT }, + ] + + for (const msg of this._messages) { + if (msg.role === 'user' || msg.role === 'assistant') { + chatMsgs.push({ role: msg.role, content: msg.content }) + } + } + + return chatMsgs + } + + private _trimHistory(): void { + // user+assistant 쌍 기준으로 최근 MAX_HISTORY_TURNS개만 유지 + const pairs: ConversationMessage[] = [] + let turnCount = 0 + + for (let i = this._messages.length - 1; i >= 0; i--) { + pairs.unshift(this._messages[i]) + if (this._messages[i].role === 'user') { + turnCount++ + if (turnCount >= MAX_HISTORY_TURNS) break + } + } + + this._messages = pairs + } + + private _emitError(phase: ConversationError['phase'], message: string): void { + const error: ConversationError = { message, phase } + this._sendToRenderer(IPC_CHANNELS.VOICE_CONVERSATION.ERROR, error) + this.emit('error', error) + } + + private _sendToRenderer(channel: string, data: unknown): void { + try { + const mainWindow = getMainWindow() + if (mainWindow && !mainWindow.isDestroyed()) { + mainWindow.webContents.send(channel, data) + } + } catch { + // 윈도우 없으면 무시 + } + } + + dispose(): void { + this.stopSession() + this.removeAllListeners() + } +} + +// ── 싱글톤 ── +let instance: VoiceConversationService | null = null + +export function getVoiceConversationService(): VoiceConversationService { + if (!instance) { + instance = new VoiceConversationService() + } + return instance +} diff --git a/src/main/utils/paths.ts b/src/main/utils/paths.ts index 680bac8..5249b6d 100644 --- a/src/main/utils/paths.ts +++ b/src/main/utils/paths.ts @@ -90,6 +90,29 @@ export function getSoundPath(filename: string): string { return path.join(app.getAppPath(), 'resources', 'sounds', filename) } +/** + * ffmpeg 실행 파일 경로. + * - dev: @ffmpeg-installer/ffmpeg의 node_modules 경로 + * - production: extraResources로 번들된 경로 + */ +export function getFfmpegPath(): string { + if (isPackaged()) { + const bundled = path.join(process.resourcesPath, 'ffmpeg', 'ffmpeg.exe') + if (existsSync(bundled)) { + return bundled + } + } + + // dev: @ffmpeg-installer/ffmpeg에서 제공하는 경로 + try { + // eslint-disable-next-line @typescript-eslint/no-require-imports + const installer = require('@ffmpeg-installer/ffmpeg') + return installer.path as string + } catch { + return 'ffmpeg' + } +} + /** * 사용자 데이터 경로 (DB, 로그 등). */ diff --git a/src/preload/index.ts b/src/preload/index.ts index c202f08..e0fdf25 100644 --- a/src/preload/index.ts +++ b/src/preload/index.ts @@ -95,6 +95,47 @@ import type { UsageQuota, UpgradePromptEvent, TierComparison, + // Phase 12 + FileTranscriptionStartParams, + // Phase 13.1 + ConversationSessionInfo, + ConversationMessage, + ConversationSendParams, + ConversationAssistantDelta, + ConversationAssistantMessage, + ConversationError, + // Phase 13.2 + RAGDocument, + RAGQueryParams, + RAGQueryResult, + RAGStateInfo, + RAGIndexProgress, + RAGAddDocumentParams, + RAGRemoveDocumentParams, + // Phase 13.3 + VoiceActionExecuteParams, + VoiceActionPreset, + VoiceActionHistoryEntry, + VoiceActionPlannedEvent, + VoiceActionExecutedEvent, + VoiceActionErrorEvent, + FileTranscriptionProgress, + FileTranscriptionResult, + FileTranscriptionStateInfo, + MeetingSummarizeParams, + MeetingSummaryResult, + MeetingSummaryGetParams, + MeetingSummaryExportParams, + MeetingSummaryProgress, + DictationTemplate, + CreateTemplateParams, + UpdateTemplateParams, + DeleteTemplateParams, + StartTemplateSessionParams, + SetFieldValueParams, + TemplateSessionInfo, + TemplateFieldCompletedEvent, + TemplateSessionCompletedEvent, } from '@shared/types' import { Feature } from '@shared/types' import type { IPCResult } from '@shared/errors' @@ -382,6 +423,136 @@ const electronAPI = { onTierChanged: (cb: (e: LicenseInfo) => void): Unsubscribe => on(IPC_CHANNELS.LICENSE.TIER_CHANGED, cb), }, + // ── File Transcription (Phase 12.1) ──────────────────── + fileTranscription: { + start: (params: FileTranscriptionStartParams) => + invoke(IPC_CHANNELS.FILE_TRANSCRIPTION.START, params), + cancel: () => + invoke(IPC_CHANNELS.FILE_TRANSCRIPTION.CANCEL), + getState: () => + invoke(IPC_CHANNELS.FILE_TRANSCRIPTION.GET_STATE), + onProgress: (cb: (data: FileTranscriptionProgress) => void): Unsubscribe => + on(IPC_CHANNELS.FILE_TRANSCRIPTION.PROGRESS, cb), + onComplete: (cb: (data: FileTranscriptionResult) => void): Unsubscribe => + on(IPC_CHANNELS.FILE_TRANSCRIPTION.COMPLETE, cb), + onError: (cb: (data: { message: string }) => void): Unsubscribe => + on(IPC_CHANNELS.FILE_TRANSCRIPTION.ERROR, cb), + }, + + // ── Meeting Summary (Phase 12.2) ───────────────────── + meetingSummary: { + summarize: (params: MeetingSummarizeParams) => + invoke(IPC_CHANNELS.MEETING_SUMMARY.SUMMARIZE, params), + getSummary: (params: MeetingSummaryGetParams) => + invoke(IPC_CHANNELS.MEETING_SUMMARY.GET_SUMMARY, params), + exportMarkdown: (params: MeetingSummaryExportParams) => + invoke(IPC_CHANNELS.MEETING_SUMMARY.EXPORT_MARKDOWN, params), + onSummaryReady: (cb: (data: MeetingSummaryResult) => void): Unsubscribe => + on(IPC_CHANNELS.MEETING_SUMMARY.SUMMARY_READY, cb), + onProgress: (cb: (data: MeetingSummaryProgress) => void): Unsubscribe => + on(IPC_CHANNELS.MEETING_SUMMARY.SUMMARY_PROGRESS, cb), + }, + + // ── Dictation Templates (Phase 12.3) ───────────────── + dictationTemplate: { + getAll: () => + invoke(IPC_CHANNELS.DICTATION_TEMPLATE.GET_ALL), + create: (params: CreateTemplateParams) => + invoke(IPC_CHANNELS.DICTATION_TEMPLATE.CREATE, params), + update: (params: UpdateTemplateParams) => + invoke(IPC_CHANNELS.DICTATION_TEMPLATE.UPDATE, params), + delete: (params: DeleteTemplateParams) => + invoke(IPC_CHANNELS.DICTATION_TEMPLATE.DELETE, params), + startSession: (params: StartTemplateSessionParams) => + invoke(IPC_CHANNELS.DICTATION_TEMPLATE.START_SESSION, params), + cancelSession: () => + invoke(IPC_CHANNELS.DICTATION_TEMPLATE.CANCEL_SESSION), + getSessionState: () => + invoke(IPC_CHANNELS.DICTATION_TEMPLATE.GET_SESSION_STATE), + setFieldValue: (params: SetFieldValueParams) => + invoke(IPC_CHANNELS.DICTATION_TEMPLATE.SET_FIELD_VALUE, params), + onSessionStateChanged: (cb: (data: TemplateSessionInfo) => void): Unsubscribe => + on(IPC_CHANNELS.DICTATION_TEMPLATE.SESSION_STATE_CHANGED, cb), + onFieldCompleted: (cb: (data: TemplateFieldCompletedEvent) => void): Unsubscribe => + on(IPC_CHANNELS.DICTATION_TEMPLATE.FIELD_COMPLETED, cb), + onSessionCompleted: (cb: (data: TemplateSessionCompletedEvent) => void): Unsubscribe => + on(IPC_CHANNELS.DICTATION_TEMPLATE.SESSION_COMPLETED, cb), + }, + // ── RAG (Phase 13.2) ────────────────────────────────── + rag: { + addDocument: (params?: RAGAddDocumentParams) => + invoke(IPC_CHANNELS.RAG.ADD_DOCUMENT, params ?? {}), + removeDocument: (params: RAGRemoveDocumentParams) => + invoke(IPC_CHANNELS.RAG.REMOVE_DOCUMENT, params), + getDocuments: () => + invoke(IPC_CHANNELS.RAG.GET_DOCUMENTS), + query: (params: RAGQueryParams) => + invoke(IPC_CHANNELS.RAG.QUERY, params), + getState: () => + invoke(IPC_CHANNELS.RAG.GET_STATE), + reindex: (documentId: string) => + invoke(IPC_CHANNELS.RAG.REINDEX, { documentId }), + onIndexProgress: (cb: (data: RAGIndexProgress) => void): Unsubscribe => + on(IPC_CHANNELS.RAG.INDEX_PROGRESS, cb), + onIndexComplete: (cb: (data: { documentId: string; fileName: string }) => void): Unsubscribe => + on(IPC_CHANNELS.RAG.INDEX_COMPLETE, cb), + }, + + // ── Voice Action (Phase 13.3) ──────────────────────── + voiceAction: { + execute: (params: VoiceActionExecuteParams) => + invoke(IPC_CHANNELS.VOICE_ACTION.EXECUTE, params), + getPresets: () => + invoke(IPC_CHANNELS.VOICE_ACTION.GET_PRESETS), + getHistory: () => + invoke(IPC_CHANNELS.VOICE_ACTION.GET_HISTORY), + clearHistory: () => + invoke(IPC_CHANNELS.VOICE_ACTION.CLEAR_HISTORY), + setEnabled: (enabled: boolean) => + invoke(IPC_CHANNELS.VOICE_ACTION.SET_ENABLED, { enabled }), + isEnabled: () => + invoke(IPC_CHANNELS.VOICE_ACTION.IS_ENABLED), + onActionPlanned: (cb: (data: VoiceActionPlannedEvent) => void): Unsubscribe => + on(IPC_CHANNELS.VOICE_ACTION.ACTION_PLANNED, cb), + onActionExecuted: (cb: (data: VoiceActionExecutedEvent) => void): Unsubscribe => + on(IPC_CHANNELS.VOICE_ACTION.ACTION_EXECUTED, cb), + onActionError: (cb: (data: VoiceActionErrorEvent) => void): Unsubscribe => + on(IPC_CHANNELS.VOICE_ACTION.ACTION_ERROR, cb), + }, + + // ── Voice Conversation (Phase 13.1) ─────────────────── + voiceConversation: { + startSession: () => + invoke(IPC_CHANNELS.VOICE_CONVERSATION.START_SESSION), + stopSession: () => + invoke(IPC_CHANNELS.VOICE_CONVERSATION.STOP_SESSION), + sendMessage: (params: ConversationSendParams) => + invoke(IPC_CHANNELS.VOICE_CONVERSATION.SEND_MESSAGE, params), + getState: () => + invoke(IPC_CHANNELS.VOICE_CONVERSATION.GET_STATE), + getHistory: () => + invoke(IPC_CHANNELS.VOICE_CONVERSATION.GET_HISTORY), + clearHistory: () => + invoke(IPC_CHANNELS.VOICE_CONVERSATION.CLEAR_HISTORY), + cancelResponse: () => + invoke(IPC_CHANNELS.VOICE_CONVERSATION.CANCEL_RESPONSE), + finishListening: () => + invoke('voiceConversation:finishListening'), + onStateChanged: (cb: (data: ConversationSessionInfo) => void): Unsubscribe => + on(IPC_CHANNELS.VOICE_CONVERSATION.STATE_CHANGED, cb), + onUserMessage: (cb: (data: ConversationMessage) => void): Unsubscribe => + on(IPC_CHANNELS.VOICE_CONVERSATION.USER_MESSAGE, cb), + onAssistantDelta: (cb: (data: ConversationAssistantDelta) => void): Unsubscribe => + on(IPC_CHANNELS.VOICE_CONVERSATION.ASSISTANT_DELTA, cb), + onAssistantMessage: (cb: (data: ConversationAssistantMessage) => void): Unsubscribe => + on(IPC_CHANNELS.VOICE_CONVERSATION.ASSISTANT_MESSAGE, cb), + onTTSStarted: (cb: () => void): Unsubscribe => + on(IPC_CHANNELS.VOICE_CONVERSATION.TTS_STARTED, cb), + onTTSFinished: (cb: () => void): Unsubscribe => + on(IPC_CHANNELS.VOICE_CONVERSATION.TTS_FINISHED, cb), + onError: (cb: (data: ConversationError) => void): Unsubscribe => + on(IPC_CHANNELS.VOICE_CONVERSATION.ERROR, cb), + }, } as const contextBridge.exposeInMainWorld('electronAPI', electronAPI) diff --git a/src/renderer/components/AppLayout.tsx b/src/renderer/components/AppLayout.tsx index fe3b508..bc031ea 100644 --- a/src/renderer/components/AppLayout.tsx +++ b/src/renderer/components/AppLayout.tsx @@ -7,12 +7,16 @@ import DashboardIcon from '@mui/icons-material/Dashboard' import HistoryIcon from '@mui/icons-material/History' import MenuBookIcon from '@mui/icons-material/MenuBook' import ExtensionIcon from '@mui/icons-material/Extension' +import RecordVoiceOverIcon from '@mui/icons-material/RecordVoiceOver' +import AutoStoriesIcon from '@mui/icons-material/AutoStories' import SettingsIcon from '@mui/icons-material/Settings' import { Led } from './ds' import { DashboardPage } from '../pages/DashboardPage' import { HistoryPage } from '../pages/HistoryPage' import { DictionaryPage } from '../pages/DictionaryPage' import { CommandsPage } from '../pages/CommandsPage' +import { VoiceConversationPage } from '../pages/VoiceConversationPage' +import { KnowledgeBasePage } from '../pages/KnowledgeBasePage' import { SettingsModal } from './SettingsModal' import { LicenseModal } from './LicenseModal' import { OnboardingModal } from './OnboardingModal' @@ -22,7 +26,7 @@ import { useI18n } from '../i18n' import type { TranslationKey } from '../i18n' import type { LicenseTier } from '@shared/types' -type Route = 'dashboard' | 'history' | 'dictionary' | 'commands' +type Route = 'dashboard' | 'history' | 'dictionary' | 'commands' | 'conversation' | 'knowledge' interface NavItem { route: Route @@ -36,6 +40,8 @@ const NAV_ITEMS: NavItem[] = [ { route: 'history', labelKey: 'nav.history', abbr: 'HIST', icon: }, { route: 'dictionary', labelKey: 'nav.dictionary', abbr: 'DICT', icon: }, { route: 'commands', labelKey: 'nav.commands', abbr: 'CMD', icon: }, + { route: 'conversation', labelKey: 'nav.conversation', abbr: 'TALK', icon: }, + { route: 'knowledge', labelKey: 'nav.knowledge', abbr: 'RAG', icon: }, ] function tierToLedColor(tier: LicenseTier): 'amber' | 'green' { @@ -217,6 +223,8 @@ export function AppLayout(): React.ReactElement { {currentRoute === 'history' && } {currentRoute === 'dictionary' && } {currentRoute === 'commands' && } + {currentRoute === 'conversation' && } + {currentRoute === 'knowledge' && } diff --git a/src/renderer/components/FileDropZone.tsx b/src/renderer/components/FileDropZone.tsx new file mode 100644 index 0000000..02ac956 --- /dev/null +++ b/src/renderer/components/FileDropZone.tsx @@ -0,0 +1,272 @@ +// src/renderer/components/FileDropZone.tsx +// Phase 12.1: 파일 전사 드래그앤드롭 UI + +import { useState, useEffect, useCallback, useRef } from 'react' +import { Box, LinearProgress, IconButton, Tooltip } from '@mui/material' +import ContentCopyIcon from '@mui/icons-material/ContentCopy' +import CloseIcon from '@mui/icons-material/Close' +import UploadFileIcon from '@mui/icons-material/UploadFile' +import { MetalCard, PhosphorText, Led } from './ds' +import { d3roPalette, d3roTypo } from '../theme' +import { useI18n } from '../i18n' +import type { + FileTranscriptionProgress, + FileTranscriptionResult, + FileTranscriptionState, +} from '@shared/types' + +const SUPPORTED_EXTENSIONS = [ + '.mp3', '.wav', '.m4a', '.ogg', '.flac', '.wma', '.aac', + '.mp4', '.mkv', '.webm', '.avi', '.mov', +] + +export function FileDropZone(): React.ReactElement { + const { t } = useI18n() + const [dragging, setDragging] = useState(false) + const [state, setState] = useState('idle') + const [progress, setProgress] = useState(null) + const [result, setResult] = useState(null) + const [error, setError] = useState(null) + const [copied, setCopied] = useState(false) + const dropRef = useRef(null) + + useEffect(() => { + const unsubProgress = window.electronAPI.fileTranscription.onProgress((data) => { + setProgress(data) + setState('transcribing') + }) + const unsubComplete = window.electronAPI.fileTranscription.onComplete((data) => { + setResult(data) + setState('completed') + setProgress(null) + }) + const unsubError = window.electronAPI.fileTranscription.onError((data) => { + setError(data.message) + setState('error') + setProgress(null) + }) + + return () => { + unsubProgress() + unsubComplete() + unsubError() + } + }, []) + + const handleDrop = useCallback(async (e: React.DragEvent) => { + e.preventDefault() + setDragging(false) + + const file = e.dataTransfer.files[0] + if (!file) return + + const ext = '.' + file.name.split('.').pop()?.toLowerCase() + if (!SUPPORTED_EXTENSIONS.includes(ext)) { + setError(t('fileTranscription.error.invalidFormat')) + setState('error') + return + } + + setState('converting') + setError(null) + setResult(null) + + const filePath = (file as unknown as { path: string }).path + const resp = await window.electronAPI.fileTranscription.start({ filePath }) + if (!resp.success) { + setError(resp.error.message) + setState('error') + } + }, [t]) + + const handleBrowse = useCallback(async () => { + setState('converting') + setError(null) + setResult(null) + + const resp = await window.electronAPI.fileTranscription.start({ filePath: '' }) + if (!resp.success) { + if (resp.error.message.includes('cancelled')) { + setState('idle') + } else { + setError(resp.error.message) + setState('error') + } + } + }, []) + + const handleCancel = useCallback(async () => { + await window.electronAPI.fileTranscription.cancel() + setState('idle') + setProgress(null) + }, []) + + const handleCopy = useCallback(() => { + if (result?.fullText) { + navigator.clipboard.writeText(result.fullText) + setCopied(true) + setTimeout(() => setCopied(false), 2000) + } + }, [result]) + + const handleReset = useCallback(() => { + setState('idle') + setResult(null) + setError(null) + setProgress(null) + }, []) + + // ── idle: 드래그 존 ── + if (state === 'idle') { + return ( + + { e.preventDefault(); setDragging(true) }} + onDragLeave={() => setDragging(false)} + onDrop={handleDrop} + onClick={handleBrowse} + sx={{ + p: 4, + textAlign: 'center', + border: `2px dashed ${dragging ? d3roPalette.accent.amber : d3roPalette.border.subtle}`, + borderRadius: '8px', + cursor: 'pointer', + transition: 'border-color 0.2s', + '&:hover': { borderColor: d3roPalette.accent.amber }, + }} + > + + + {t('fileTranscription.dropZone')} + + + {t('fileTranscription.dropZoneHint')} + + + + ) + } + + // ── converting / transcribing: 진행률 ── + if (state === 'converting' || state === 'transcribing') { + return ( + + + + + + + {state === 'converting' + ? t('fileTranscription.converting') + : t('fileTranscription.processing')} + + + + {t('fileTranscription.cancel')} + + + + {progress && ( + <> + + + {t('fileTranscription.progress', { + current: String(progress.currentChunk), + total: String(progress.totalChunks), + })} + + {progress.currentText && ( + + {progress.currentText.slice(0, 100)}... + + )} + + )} + + + ) + } + + // ── completed: 결과 ── + if (state === 'completed' && result) { + return ( + + + + + + + {t('fileTranscription.complete')} + + + ({result.fileName}) + + + + + + + + + + + + + + + + {result.fullText} + + + + {Math.round(result.totalDurationSec)}s audio / {Math.round(result.processingTimeMs / 1000)}s processing + + + + ) + } + + // ── error ── + if (state === 'error') { + return ( + + + + + + {error ?? t('fileTranscription.error.unknown')} + + + + {t('fileTranscription.retry')} + + + + ) + } + + return <> +} diff --git a/src/renderer/components/LicenseTab.tsx b/src/renderer/components/LicenseTab.tsx index 9cb4cd7..16d89ce 100644 --- a/src/renderer/components/LicenseTab.tsx +++ b/src/renderer/components/LicenseTab.tsx @@ -263,7 +263,7 @@ export function LicenseTab(): React.ReactElement { {comparison.map((row) => ( - {t(`license.feature.${row.feature}`)} + {t(row.featureLabel as Parameters[0])} diff --git a/src/renderer/components/TemplateSection.tsx b/src/renderer/components/TemplateSection.tsx new file mode 100644 index 0000000..41c9db1 --- /dev/null +++ b/src/renderer/components/TemplateSection.tsx @@ -0,0 +1,294 @@ +// src/renderer/components/TemplateSection.tsx +// Phase 12.3: 딕테이션 템플릿 관리 UI (CommandsPage 내 섹션) + +import { useState, useEffect, useCallback } from 'react' +import { Box, Button, IconButton, Dialog, DialogTitle, DialogContent, DialogActions, TextField, Tooltip } from '@mui/material' +import AddIcon from '@mui/icons-material/Add' +import DeleteIcon from '@mui/icons-material/Delete' +import EditIcon from '@mui/icons-material/Edit' +import PlayArrowIcon from '@mui/icons-material/PlayArrow' +import { MetalCard, PhosphorText, Led, PhysicalButton } from './ds' +import { PageHeader, EmptyStateCard } from './shared' +import { d3roPalette, d3roFontMono, d3roTypo, d3roRadius } from '../theme' +import { useI18n } from '../i18n' +import type { DictationTemplate, TemplateField, TemplateSessionInfo } from '@shared/types' + +export function TemplateSection(): React.ReactElement { + const { t } = useI18n() + const [templates, setTemplates] = useState([]) + const [loading, setLoading] = useState(true) + const [dialogOpen, setDialogOpen] = useState(false) + const [editId, setEditId] = useState(null) + const [formName, setFormName] = useState('') + const [formDesc, setFormDesc] = useState('') + const [formOutput, setFormOutput] = useState('') + const [formFields, setFormFields] = useState([]) + const [session, setSession] = useState(null) + + const loadData = useCallback(async () => { + setLoading(true) + const result = await window.electronAPI.dictationTemplate.getAll() + if (result.success) setTemplates(result.data) + const sessionResult = await window.electronAPI.dictationTemplate.getSessionState() + if (sessionResult.success) setSession(sessionResult.data) + setLoading(false) + }, []) + + useEffect(() => { loadData() }, [loadData]) + + useEffect(() => { + const unsub = window.electronAPI.dictationTemplate.onSessionStateChanged((data) => { + setSession(data) + }) + const unsubComplete = window.electronAPI.dictationTemplate.onSessionCompleted(() => { + setSession(null) + loadData() + }) + return () => { unsub(); unsubComplete() } + }, [loadData]) + + const openCreate = () => { + setEditId(null) + setFormName('') + setFormDesc('') + setFormOutput('{{field1}}') + setFormFields([{ id: 'field1', name: 'field1', label: 'Field 1', promptText: '', required: true, maxDurationSec: 30 }]) + setDialogOpen(true) + } + + const openEdit = (template: DictationTemplate) => { + setEditId(template.id) + setFormName(template.name) + setFormDesc(template.description) + setFormOutput(template.outputFormat) + setFormFields([...template.fields]) + setDialogOpen(true) + } + + const handleSave = async () => { + if (editId) { + await window.electronAPI.dictationTemplate.update({ + id: editId, + name: formName.trim(), + description: formDesc.trim(), + fields: formFields, + outputFormat: formOutput, + }) + } else { + await window.electronAPI.dictationTemplate.create({ + name: formName.trim(), + description: formDesc.trim(), + fields: formFields, + outputFormat: formOutput, + }) + } + setDialogOpen(false) + loadData() + } + + const handleDelete = async (id: string) => { + await window.electronAPI.dictationTemplate.delete({ id }) + loadData() + } + + const handleStartSession = async (templateId: string) => { + await window.electronAPI.dictationTemplate.startSession({ templateId }) + } + + const handleCancelSession = async () => { + await window.electronAPI.dictationTemplate.cancelSession() + setSession(null) + } + + const addField = () => { + const idx = formFields.length + 1 + setFormFields([...formFields, { + id: `field${idx}`, + name: `field${idx}`, + label: `Field ${idx}`, + promptText: '', + required: true, + maxDurationSec: 30, + }]) + } + + const updateField = (index: number, updates: Partial) => { + const updated = [...formFields] + updated[index] = { ...updated[index], ...updates } + setFormFields(updated) + } + + const removeField = (index: number) => { + setFormFields(formFields.filter((_, i) => i !== index)) + } + + return ( + + + {t('template.create')} + + } + /> + + {/* 활성 세션 표시 */} + {session && ( + + + + + + {session.templateName}: {session.currentField?.label ?? '...'} + + + ({session.currentFieldIndex + 1}/{session.totalFields}) + + + + {t('template.cancelSession')} + + + + )} + + {/* 템플릿 목록 */} + {templates.length === 0 && !loading ? ( + + ) : ( + + {templates.map((tmpl) => ( + + + + + {tmpl.name} + {tmpl.isBuiltin && ( + + PRESET + + )} + + + {tmpl.fields.length} {t('template.fields').toLowerCase()} — {tmpl.description} + + + + + handleStartSession(tmpl.id)} + disabled={!!session} + sx={{ color: d3roPalette.accent.amber, '&:hover': { opacity: 0.8 } }} + > + + + + openEdit(tmpl)} + sx={{ color: d3roPalette.text.inactive, '&:hover': { color: d3roPalette.accent.amber } }} + > + + + {!tmpl.isBuiltin && ( + handleDelete(tmpl.id)} + sx={{ color: d3roPalette.text.inactive, '&:hover': { color: d3roPalette.tag.red } }} + > + + + )} + + + + ))} + + )} + + {/* 편집 다이얼로그 */} + setDialogOpen(false)} maxWidth="sm" fullWidth> + {editId ? t('template.edit') : t('template.create')} + + setFormName(e.target.value)} + size="small" + fullWidth + /> + setFormDesc(e.target.value)} + size="small" + fullWidth + /> + + + {t('template.fields').toUpperCase()} + + + {formFields.map((field, idx) => ( + + updateField(idx, { name: e.target.value, id: e.target.value })} + size="small" + sx={{ flex: 1 }} + /> + updateField(idx, { label: e.target.value })} + size="small" + sx={{ flex: 1 }} + /> + updateField(idx, { promptText: e.target.value })} + size="small" + sx={{ flex: 2 }} + /> + removeField(idx)} + disabled={formFields.length <= 1} + sx={{ color: d3roPalette.text.inactive }} + > + + + + ))} + + + + setFormOutput(e.target.value)} + size="small" + fullWidth + multiline + rows={3} + helperText="Use {{fieldName}} for placeholders" + /> + + + + + + + + ) +} diff --git a/src/renderer/components/shared/HistoryEntryCard.tsx b/src/renderer/components/shared/HistoryEntryCard.tsx index fc97c7f..78efa50 100644 --- a/src/renderer/components/shared/HistoryEntryCard.tsx +++ b/src/renderer/components/shared/HistoryEntryCard.tsx @@ -8,11 +8,13 @@ import ContentCopyIcon from '@mui/icons-material/ContentCopy' import DeleteIcon from '@mui/icons-material/Delete' import LocalOfferIcon from '@mui/icons-material/LocalOffer' import CloseIcon from '@mui/icons-material/Close' +import SummarizeIcon from '@mui/icons-material/Summarize' +import ExpandMoreIcon from '@mui/icons-material/ExpandMore' import { MetalCard, Led } from '../ds' import { d3roPalette, d3roFontMono, d3roTypo, d3roRadius } from '../../theme' import { useI18n } from '../../i18n' import { formatDuration } from '../../utils/formatters' -import type { HistoryEntry, MemoTag } from '@shared/types' +import type { HistoryEntry, MemoTag, MeetingSummaryResult } from '@shared/types' interface HistoryEntryCardProps { entry: HistoryEntry @@ -28,6 +30,10 @@ export function HistoryEntryCard({ entry, onCopy, onDelete, showTags = false, on const [tags, setTags] = useState([]) const [tagInput, setTagInput] = useState('') const [showTagInput, setShowTagInput] = useState(false) + const [summaryExpanded, setSummaryExpanded] = useState(false) + const [summary, setSummary] = useState(null) + const [summaryLoading, setSummaryLoading] = useState(false) + const hasSummary = !!entry.summaryText const loadTags = useCallback(async () => { if (!showTags) return @@ -60,6 +66,35 @@ export function HistoryEntryCard({ entry, onCopy, onDelete, showTags = false, on if (e.key === 'Escape') { setShowTagInput(false); setTagInput('') } }, [handleAddTag]) + const handleToggleSummary = useCallback(async () => { + if (summaryExpanded) { + setSummaryExpanded(false) + return + } + setSummaryExpanded(true) + if (!summary) { + setSummaryLoading(true) + const result = await window.electronAPI.meetingSummary.getSummary({ historyId: entry.id }) + if (result.success && result.data) { + setSummary(result.data) + } + setSummaryLoading(false) + } + }, [summaryExpanded, summary, entry.id]) + + const handleGenerateSummary = useCallback(async () => { + setSummaryLoading(true) + const result = await window.electronAPI.meetingSummary.summarize({ historyId: entry.id }) + if (result.success) { + setSummary(result.data) + } + setSummaryLoading(false) + }, [entry.id]) + + const handleExportSummary = useCallback(async () => { + await window.electronAPI.meetingSummary.exportMarkdown({ historyId: entry.id }) + }, [entry.id]) + return ( @@ -94,6 +129,24 @@ export function HistoryEntryCard({ entry, onCopy, onDelete, showTags = false, on {formatDuration(entry.duration)} {entry.detectedLanguage && {entry.detectedLanguage.toUpperCase()}} {entry.mode.toUpperCase()} + {hasSummary && ( + } + label={t('meetingSummary.title')} + size="small" + onClick={handleToggleSummary} + sx={{ + height: 18, + fontFamily: d3roFontMono, + fontSize: d3roTypo.micro.size, + bgcolor: d3roPalette.tag.greenBg, + color: d3roPalette.tag.green, + borderRadius: d3roRadius.small, + cursor: 'pointer', + '& .MuiChip-icon': { color: d3roPalette.tag.green }, + }} + /> + )} {/* 태그 영역 */} {showTags && ( @@ -180,6 +233,82 @@ export function HistoryEntryCard({ entry, onCopy, onDelete, showTags = false, on )} + + {/* Phase 12.2: 회의록 요약 확장 뷰 */} + {summaryExpanded && ( + + {summaryLoading ? ( + {t('meetingSummary.generating')} + ) : summary ? ( + + {summary.summary && ( + + + {t('meetingSummary.summary').toUpperCase()} + + {summary.summary} + + )} + {summary.decisions.length > 0 && ( + + + {t('meetingSummary.decisions').toUpperCase()} + + {summary.decisions.map((d, i) => ( + • {d} + ))} + + )} + {summary.actionItems.length > 0 && ( + + + {t('meetingSummary.actionItems').toUpperCase()} + + {summary.actionItems.map((a, i) => ( + ☐ {a} + ))} + + )} + + + + + + + + + ) : ( + + {t('meetingSummary.noSummary')} + {(entry.mode === 'caption' || entry.mode === 'file-transcription') && ( + + )} + + )} + + )} + + {/* 요약 토글 버튼 (caption/file-transcription 모드만) */} + {(entry.mode === 'caption' || entry.mode === 'file-transcription') && !summaryExpanded && !hasSummary && ( + + + + )} ) } diff --git a/src/renderer/i18n/en.json b/src/renderer/i18n/en.json index d3a734b..2763e87 100644 --- a/src/renderer/i18n/en.json +++ b/src/renderer/i18n/en.json @@ -300,13 +300,28 @@ "license.feature.voice_memo": "Voice Memo", "license.feature.history_unlimited": "Unlimited History", "license.feature.history_export": "History Export", - "license.feature.custom_instruction_create": "Custom Instruction Create", + "license.feature.custom_instruction_create": "Custom Instruction", "license.feature.file_transcription": "File Transcription", "license.feature.voice_conversation": "Voice Conversation", "license.feature.dictation_template": "Dictation Template", "license.feature.meeting_summary": "Meeting Summary", "license.feature.local_rag": "Local RAG", "license.feature.os_automation": "OS Automation", + "license.feature.llmProcess": "LLM Processing", + "license.feature.liveCaption": "Live Caption", + "license.feature.screenContext": "Screen Context", + "license.feature.voiceCommand": "Voice Commands", + "license.feature.llmChain": "LLM Chain", + "license.feature.voiceMemo": "Voice Memo", + "license.feature.historyUnlimited": "Unlimited History", + "license.feature.historyExport": "History Export", + "license.feature.customInstruction": "Custom Instruction", + "license.feature.fileTranscription": "File Transcription", + "license.feature.voiceConversation": "Voice Conversation", + "license.feature.dictationTemplate": "Dictation Template", + "license.feature.meetingSummary": "Meeting Summary", + "license.feature.localRag": "Local RAG", + "license.feature.osAutomation": "OS Automation", "license.pro.required": "PRO Required", "license.proPlus.required": "PRO+ Required", "license.included": "Included", @@ -327,5 +342,80 @@ "license.perDay": "/day", "license.unlimited": "Unlimited", "license.locked": "Locked", - "license.nav": "License" + "license.nav": "License", + + "fileTranscription.title": "File Transcription", + "fileTranscription.dropZone": "Drop audio/video file here", + "fileTranscription.dropZoneHint": "Supports MP3, WAV, M4A, MP4, MKV, WEBM", + "fileTranscription.converting": "Converting...", + "fileTranscription.processing": "Transcribing...", + "fileTranscription.progress": "Chunk {{current}} / {{total}}", + "fileTranscription.complete": "Transcription complete", + "fileTranscription.copyAll": "Copy All", + "fileTranscription.cancel": "Cancel", + "fileTranscription.retry": "Retry", + "fileTranscription.error.invalidFormat": "Unsupported file format", + "fileTranscription.error.unknown": "Unknown error", + + "meetingSummary.title": "Meeting Summary", + "meetingSummary.generating": "Generating summary...", + "meetingSummary.summary": "Summary", + "meetingSummary.decisions": "Key Decisions", + "meetingSummary.actionItems": "Action Items", + "meetingSummary.exportMarkdown": "Export as Markdown", + "meetingSummary.noSummary": "No summary available", + "meetingSummary.generate": "Generate Summary", + + "template.title": "Dictation Templates", + "template.create": "New Template", + "template.edit": "Edit Template", + "template.delete": "Delete", + "template.name": "Template Name", + "template.description": "Description", + "template.fields": "Fields", + "template.addField": "Add Field", + "template.fieldName": "Field Name", + "template.fieldLabel": "Label", + "template.fieldPrompt": "Voice Prompt", + "template.outputFormat": "Output Format", + "template.startSession": "Start", + "template.cancelSession": "Cancel Session", + "template.empty": "No templates", + + "nav.conversation": "Talk", + "conversation.title": "Voice Conversation", + "conversation.idle": "Idle", + "conversation.listening": "Listening", + "conversation.thinking": "Thinking", + "conversation.speaking": "Speaking", + "conversation.empty": "Talk to D3RO", + "conversation.emptyHint": "Press the mic button or type a message", + "conversation.inputPlaceholder": "Type a message...", + "conversation.end": "End", + "conversation.clearHistory": "Clear history", + + "nav.knowledge": "Knowledge", + "rag.title": "Knowledge Base", + "rag.addDocument": "Add Document", + "rag.documents": "Documents", + "rag.noDocuments": "No documents", + "rag.chunks": "chunks", + "rag.indexed": "Indexed", + "rag.pending": "Pending", + "rag.indexing": "Indexing", + "rag.reindex": "Reindex", + "rag.askQuestion": "Ask a Question", + "rag.queryPlaceholder": "Ask about your documents...", + "rag.searching": "Searching...", + "rag.sources": "Sources", + "rag.adding": "Adding...", + "rag.parsing": "Parsing document...", + + "voiceAction.title": "Voice Actions", + "voiceAction.execute": "Execute", + "voiceAction.presets": "Preset Commands", + "voiceAction.history": "Action History", + "voiceAction.noHistory": "No action history", + "voiceAction.blocked": "Blocked (unsafe)", + "voiceAction.executed": "Executed" } diff --git a/src/renderer/i18n/ko.json b/src/renderer/i18n/ko.json index c1085d0..1b8990e 100644 --- a/src/renderer/i18n/ko.json +++ b/src/renderer/i18n/ko.json @@ -307,6 +307,21 @@ "license.feature.meeting_summary": "회의 요약", "license.feature.local_rag": "로컬 RAG", "license.feature.os_automation": "OS 자동화", + "license.feature.llmProcess": "LLM 처리", + "license.feature.liveCaption": "실시간 자막", + "license.feature.screenContext": "화면 컨텍스트", + "license.feature.voiceCommand": "음성 명령어", + "license.feature.llmChain": "LLM 체인", + "license.feature.voiceMemo": "음성 메모", + "license.feature.historyUnlimited": "무제한 히스토리", + "license.feature.historyExport": "히스토리 내보내기", + "license.feature.customInstruction": "커스텀 명령어 생성", + "license.feature.fileTranscription": "파일 전사", + "license.feature.voiceConversation": "음성 대화", + "license.feature.dictationTemplate": "받아쓰기 템플릿", + "license.feature.meetingSummary": "회의 요약", + "license.feature.localRag": "로컬 RAG", + "license.feature.osAutomation": "OS 자동화", "license.pro.required": "PRO 필요", "license.proPlus.required": "PRO+ 필요", "license.included": "포함", @@ -327,5 +342,80 @@ "license.perDay": "회/일", "license.unlimited": "무제한", "license.locked": "잠금", - "license.nav": "라이선스" + "license.nav": "라이선스", + + "fileTranscription.title": "파일 전사", + "fileTranscription.dropZone": "오디오/비디오 파일을 여기에 드래그하세요", + "fileTranscription.dropZoneHint": "MP3, WAV, M4A, MP4, MKV, WEBM 지원", + "fileTranscription.converting": "변환 중...", + "fileTranscription.processing": "전사 중...", + "fileTranscription.progress": "청크 {{current}} / {{total}}", + "fileTranscription.complete": "전사 완료", + "fileTranscription.copyAll": "전체 복사", + "fileTranscription.cancel": "취소", + "fileTranscription.retry": "다시 시도", + "fileTranscription.error.invalidFormat": "지원하지 않는 파일 형식입니다", + "fileTranscription.error.unknown": "알 수 없는 오류", + + "meetingSummary.title": "회의록 요약", + "meetingSummary.generating": "요약 생성 중...", + "meetingSummary.summary": "요약", + "meetingSummary.decisions": "핵심 결정사항", + "meetingSummary.actionItems": "할 일 목록", + "meetingSummary.exportMarkdown": "마크다운 내보내기", + "meetingSummary.noSummary": "요약이 없습니다", + "meetingSummary.generate": "요약 생성", + + "template.title": "딕테이션 템플릿", + "template.create": "새 템플릿", + "template.edit": "템플릿 편집", + "template.delete": "삭제", + "template.name": "템플릿 이름", + "template.description": "설명", + "template.fields": "필드", + "template.addField": "필드 추가", + "template.fieldName": "필드명", + "template.fieldLabel": "라벨", + "template.fieldPrompt": "음성 안내", + "template.outputFormat": "출력 포맷", + "template.startSession": "시작", + "template.cancelSession": "세션 취소", + "template.empty": "템플릿이 없습니다", + + "nav.conversation": "대화", + "conversation.title": "음성 대화", + "conversation.idle": "대기", + "conversation.listening": "듣는 중", + "conversation.thinking": "생각 중", + "conversation.speaking": "말하는 중", + "conversation.empty": "D3RO에게 말을 걸어보세요", + "conversation.emptyHint": "마이크 버튼을 누르거나 텍스트를 입력하세요", + "conversation.inputPlaceholder": "메시지 입력...", + "conversation.end": "종료", + "conversation.clearHistory": "대화 초기화", + + "nav.knowledge": "지식 베이스", + "rag.title": "지식 베이스", + "rag.addDocument": "문서 추가", + "rag.documents": "문서", + "rag.noDocuments": "문서가 없습니다", + "rag.chunks": "청크", + "rag.indexed": "인덱싱 완료", + "rag.pending": "대기 중", + "rag.indexing": "인덱싱 중", + "rag.reindex": "재인덱싱", + "rag.askQuestion": "질문하기", + "rag.queryPlaceholder": "문서에 대해 질문하세요...", + "rag.searching": "검색 중...", + "rag.sources": "참조 문서", + "rag.adding": "추가 중...", + "rag.parsing": "문서 분석 중...", + + "voiceAction.title": "음성 액션", + "voiceAction.execute": "실행", + "voiceAction.presets": "프리셋 명령", + "voiceAction.history": "실행 이력", + "voiceAction.noHistory": "실행 이력이 없습니다", + "voiceAction.blocked": "차단됨 (위험 명령)", + "voiceAction.executed": "실행됨" } diff --git a/src/renderer/pages/CommandsPage.tsx b/src/renderer/pages/CommandsPage.tsx index bfea8b0..3c72583 100644 --- a/src/renderer/pages/CommandsPage.tsx +++ b/src/renderer/pages/CommandsPage.tsx @@ -14,6 +14,7 @@ import { MetalCard, PhosphorText, Led, ScreenPanel } from '../components/ds' import { PageHeader, EmptyStateCard } from '../components/shared' import { d3roPalette, d3roFontMono, d3roTypo, d3roRadius } from '../theme' import { useI18n } from '../i18n' +import { TemplateSection } from '../components/TemplateSection' import type { VoiceCommandRule, VoiceCommandKeyword, KeywordMatchMode, LLMChain, ChainStep } from '@shared/types' interface CustomInstruction { @@ -574,6 +575,9 @@ function ChainSection({ instructions }: { instructions: CustomInstruction[] }): + + {/* ── Phase 12.3: 딕테이션 템플릿 ── */} + ) } diff --git a/src/renderer/pages/DashboardPage.tsx b/src/renderer/pages/DashboardPage.tsx index 0fa1dd7..ee38e84 100644 --- a/src/renderer/pages/DashboardPage.tsx +++ b/src/renderer/pages/DashboardPage.tsx @@ -9,6 +9,7 @@ import { EmptyStateCard, HistoryEntryCard } from '../components/shared' import { d3roPalette, d3roTypo } from '../theme' import { useI18n } from '../i18n' import { formatRecordingTime, formatRecordingTimeUnit, formatNumber, getDateKey } from '../utils/formatters' +import { FileDropZone } from '../components/FileDropZone' import type { StatsSummary, HistoryEntry, HotkeyBinding, CaptionState, LicenseTier, UsageQuota } from '@shared/types' // ── 메인 컴포넌트 ───────────────────────────────────── @@ -349,6 +350,14 @@ export function DashboardPage(): React.ReactElement { + {/* ── 3.5 파일 전사 (Phase 12.1) ──────────────── */} + + + {t('fileTranscription.title').toUpperCase()} + + + + {/* ── 4. 최근 히스토리 ──────────────────────── */} diff --git a/src/renderer/pages/KnowledgeBasePage.tsx b/src/renderer/pages/KnowledgeBasePage.tsx new file mode 100644 index 0000000..9442492 --- /dev/null +++ b/src/renderer/pages/KnowledgeBasePage.tsx @@ -0,0 +1,262 @@ +// src/renderer/pages/KnowledgeBasePage.tsx +// Phase 13.2: 로컬 RAG Knowledge Base UI +// 문서 관리 + 질문 입력 + 답변 표시 + +import { useState, useEffect, useCallback } from 'react' +import { Box, TextField, IconButton, Tooltip, LinearProgress } from '@mui/material' +import AddIcon from '@mui/icons-material/Add' +import DeleteIcon from '@mui/icons-material/Delete' +import RefreshIcon from '@mui/icons-material/Refresh' +import SearchIcon from '@mui/icons-material/Search' +import SendIcon from '@mui/icons-material/Send' +import DescriptionIcon from '@mui/icons-material/Description' +import { MetalCard, PhosphorText, Led, PhysicalButton, ScreenPanel } from '../components/ds' +import { PageHeader, EmptyStateCard } from '../components/shared' +import { d3roPalette, d3roFontMono, d3roTypo } from '../theme' +import { useI18n } from '../i18n' +import type { RAGDocument, RAGQueryResult, RAGIndexProgress } from '@shared/types' + +export function KnowledgeBasePage(): React.ReactElement { + const { t } = useI18n() + const [documents, setDocuments] = useState([]) + const [loading, setLoading] = useState(true) + const [query, setQuery] = useState('') + const [querying, setQuerying] = useState(false) + const [result, setResult] = useState(null) + const [indexProgress, setIndexProgress] = useState(null) + const [adding, setAdding] = useState(false) + const [addError, setAddError] = useState(null) + + const loadDocuments = useCallback(async () => { + setLoading(true) + const resp = await window.electronAPI.rag.getDocuments() + if (resp.success) setDocuments(resp.data) + setLoading(false) + }, []) + + useEffect(() => { loadDocuments() }, [loadDocuments]) + + useEffect(() => { + const unsubProgress = window.electronAPI.rag.onIndexProgress((data) => { + setIndexProgress(data) + }) + const unsubComplete = window.electronAPI.rag.onIndexComplete(() => { + setIndexProgress(null) + loadDocuments() + }) + return () => { unsubProgress(); unsubComplete() } + }, [loadDocuments]) + + const handleAddDocument = useCallback(async () => { + setAdding(true) + setAddError(null) + const resp = await window.electronAPI.rag.addDocument() + setAdding(false) + if (resp.success) { + loadDocuments() + } else { + if (!resp.error.message.includes('cancelled')) { + setAddError(resp.error.message) + } + } + }, [loadDocuments]) + + const handleRemoveDocument = useCallback(async (docId: string) => { + await window.electronAPI.rag.removeDocument({ documentId: docId }) + loadDocuments() + }, [loadDocuments]) + + const handleReindex = useCallback(async (docId: string) => { + await window.electronAPI.rag.reindex(docId) + }, []) + + const handleQuery = useCallback(async () => { + if (!query.trim()) return + setQuerying(true) + setResult(null) + const resp = await window.electronAPI.rag.query({ query: query.trim() }) + if (resp.success) { + setResult(resp.data) + } + setQuerying(false) + }, [query]) + + const handleQueryKeyDown = useCallback((e: React.KeyboardEvent) => { + if (e.key === 'Enter' && !e.shiftKey) { + e.preventDefault() + handleQuery() + } + }, [handleQuery]) + + return ( + + + {adding ? t('rag.adding') : t('rag.addDocument')} + + } + /> + + {/* 인덱싱 진행률 */} + {indexProgress && ( + + + + + {t('rag.indexing')}: {indexProgress.fileName} + + + + + {indexProgress.currentChunk}/{indexProgress.totalChunks} + + + )} + + {/* 에러 표시 */} + {addError && ( + + + + + {addError} + + + + )} + + {/* 추가 중 인디케이터 */} + {adding && !indexProgress && ( + + + + {t('rag.parsing')} + + + )} + + {/* 문서 목록 */} + + {t('rag.documents').toUpperCase()} ({documents.length}) + + + {documents.length === 0 && !loading ? ( + + ) : ( + + {documents.map((doc) => ( + + + + + {doc.fileName} + + + {doc.fileType.toUpperCase()} · {doc.chunkCount} {t('rag.chunks')} · + + + + {doc.indexed ? t('rag.indexed') : t('rag.pending')} + + + + + handleReindex(doc.id)} + sx={{ color: d3roPalette.text.inactive, '&:hover': { color: d3roPalette.accent.amber } }}> + + + + + handleRemoveDocument(doc.id)} + sx={{ color: d3roPalette.text.inactive, '&:hover': { color: d3roPalette.tag.red } }}> + + + + + + ))} + + )} + + {/* 질문 입력 */} + + {t('rag.askQuestion').toUpperCase()} + + + + + + setQuery(e.target.value)} + onKeyDown={handleQueryKeyDown} + placeholder={t('rag.queryPlaceholder')} + size="small" + fullWidth + sx={{ + '& .MuiOutlinedInput-root': { + fontFamily: d3roFontMono, + fontSize: d3roTypo.compact.size, + bgcolor: d3roPalette.bg.inset, + '& fieldset': { borderColor: d3roPalette.border.subtle }, + '&:hover fieldset': { borderColor: d3roPalette.accent.amber }, + }, + '& .MuiOutlinedInput-input': { color: d3roPalette.text.primary }, + }} + /> + + + + + + + {/* 답변 */} + {querying && ( + + + {t('rag.searching')} + + )} + + {result && ( + + + + {result.answer} + + + + {result.results.length > 0 && ( + + + {t('rag.sources').toUpperCase()} + + {result.results.slice(0, 3).map((r, i) => ( + + + [{i + 1}] {r.fileName} ({Math.round(r.similarity * 100)}%) + + + {r.content.slice(0, 150)}... + + + ))} + + )} + + )} + + ) +} diff --git a/src/renderer/pages/VoiceConversationPage.tsx b/src/renderer/pages/VoiceConversationPage.tsx new file mode 100644 index 0000000..6dd1ad8 --- /dev/null +++ b/src/renderer/pages/VoiceConversationPage.tsx @@ -0,0 +1,291 @@ +// src/renderer/pages/VoiceConversationPage.tsx +// Phase 13.1: 음성 대화 모드 UI +// STT→LLM→TTS 대화 루프. 채팅 메시지 목록 + 녹음 버튼. + +import { useState, useEffect, useCallback, useRef } from 'react' +import { Box, IconButton, TextField, Tooltip } from '@mui/material' +import MicIcon from '@mui/icons-material/Mic' +import StopIcon from '@mui/icons-material/Stop' +import SendIcon from '@mui/icons-material/Send' +import DeleteSweepIcon from '@mui/icons-material/DeleteSweep' +import CancelIcon from '@mui/icons-material/Cancel' +import { MetalCard, PhosphorText, Led, PhysicalButton, ScreenPanel, InstrumentPanel } from '../components/ds' +import { PageHeader } from '../components/shared' +import { d3roPalette, d3roFontMono, d3roTypo } from '../theme' +import { useI18n } from '../i18n' +import type { + ConversationState, + ConversationMessage, + ConversationAssistantDelta, +} from '@shared/types' + +export function VoiceConversationPage(): React.ReactElement { + const { t } = useI18n() + const [state, setState] = useState('idle') + const [messages, setMessages] = useState([]) + const [isActive, setIsActive] = useState(false) + const [streamingText, setStreamingText] = useState('') + const [streamingMsgId, setStreamingMsgId] = useState(null) + const [textInput, setTextInput] = useState('') + const messagesEndRef = useRef(null) + + const scrollToBottom = useCallback(() => { + messagesEndRef.current?.scrollIntoView({ behavior: 'smooth' }) + }, []) + + // IPC 이벤트 구독 + useEffect(() => { + const unsubState = window.electronAPI.voiceConversation.onStateChanged((info) => { + setState(info.state) + setMessages(info.messages) + setIsActive(info.isActive) + }) + const unsubUser = window.electronAPI.voiceConversation.onUserMessage((msg) => { + setMessages((prev) => [...prev, msg]) + setStreamingText('') + setStreamingMsgId(null) + setTimeout(scrollToBottom, 50) + }) + const unsubDelta = window.electronAPI.voiceConversation.onAssistantDelta((data: ConversationAssistantDelta) => { + setStreamingMsgId(data.messageId) + setStreamingText(data.accumulated) + setTimeout(scrollToBottom, 50) + }) + const unsubComplete = window.electronAPI.voiceConversation.onAssistantMessage((msg) => { + setMessages((prev) => [...prev, { id: msg.messageId, role: 'assistant', content: msg.content, timestamp: Date.now() }]) + setStreamingText('') + setStreamingMsgId(null) + setTimeout(scrollToBottom, 50) + }) + const unsubError = window.electronAPI.voiceConversation.onError(() => { + // 에러 시 자동 복구 (서비스에서 listening으로 전환) + }) + + // 초기 상태 로드 + window.electronAPI.voiceConversation.getState().then((r) => { + if (r.success) { + setState(r.data.state) + setMessages(r.data.messages) + setIsActive(r.data.isActive) + } + }) + + return () => { + unsubState() + unsubUser() + unsubDelta() + unsubComplete() + unsubError() + } + }, [scrollToBottom]) + + const handleStartSession = useCallback(async () => { + await window.electronAPI.voiceConversation.startSession() + }, []) + + const handleStopSession = useCallback(async () => { + await window.electronAPI.voiceConversation.stopSession() + }, []) + + const handleFinishListening = useCallback(async () => { + await window.electronAPI.voiceConversation.finishListening() + }, []) + + const handleSendText = useCallback(async () => { + if (!textInput.trim()) return + const text = textInput.trim() + setTextInput('') + if (!isActive) { + await window.electronAPI.voiceConversation.startSession() + } + await window.electronAPI.voiceConversation.sendMessage({ text }) + }, [textInput, isActive]) + + const handleClearHistory = useCallback(async () => { + await window.electronAPI.voiceConversation.clearHistory() + setMessages([]) + }, []) + + const handleCancelResponse = useCallback(async () => { + await window.electronAPI.voiceConversation.cancelResponse() + }, []) + + const handleTextKeyDown = useCallback((e: React.KeyboardEvent) => { + if (e.key === 'Enter' && !e.shiftKey) { + e.preventDefault() + handleSendText() + } + }, [handleSendText]) + + const stateLabel = { + idle: t('conversation.idle'), + listening: t('conversation.listening'), + thinking: t('conversation.thinking'), + speaking: t('conversation.speaking'), + } + + const stateLedColor = { + idle: 'amber' as const, + listening: 'red' as const, + thinking: 'amber' as const, + speaking: 'green' as const, + } + + return ( + + + + + {stateLabel[state].toUpperCase()} + + {messages.length > 0 && ( + + + + + + )} + + } + /> + + {/* 메시지 목록 */} + + {messages.length === 0 && !streamingText && ( + + + {t('conversation.empty')} + + + {t('conversation.emptyHint')} + + + )} + + {messages.map((msg) => ( + + + + {msg.content} + + + + ))} + + {/* 스트리밍 중인 어시스턴트 메시지 */} + {streamingText && streamingMsgId && ( + + + + {streamingText} + + {'▌'} + + + + + )} + +
+ + + {/* 하단 컨트롤 바 */} + + + {/* 녹음 버튼 */} + {!isActive ? ( + + + + ) : state === 'listening' ? ( + + + + ) : state === 'thinking' || state === 'speaking' ? ( + + + + ) : ( + + + + )} + + {/* 텍스트 입력 */} + setTextInput(e.target.value)} + onKeyDown={handleTextKeyDown} + placeholder={t('conversation.inputPlaceholder')} + size="small" + fullWidth + sx={{ + '& .MuiOutlinedInput-root': { + fontFamily: d3roFontMono, + fontSize: d3roTypo.compact.size, + bgcolor: d3roPalette.bg.inset, + '& fieldset': { borderColor: d3roPalette.border.subtle }, + '&:hover fieldset': { borderColor: d3roPalette.accent.amber }, + '&.Mui-focused fieldset': { borderColor: d3roPalette.accent.amber }, + }, + '& .MuiOutlinedInput-input': { + color: d3roPalette.text.primary, + }, + }} + /> + + {/* 전송 버튼 */} + + + + + {/* 세션 종료 */} + {isActive && ( + + {t('conversation.end')} + + )} + + + + ) +} diff --git a/src/shared/errors.ts b/src/shared/errors.ts index c833b97..ab20cd9 100644 --- a/src/shared/errors.ts +++ b/src/shared/errors.ts @@ -113,6 +113,44 @@ export enum ErrorCode { CaptionAlreadyActive = 771, CaptionSTTFailed = 772, + // === Phase 12: File Transcription (780-784) === + FileTranscriptionFFmpegFailed = 780, + FileTranscriptionInvalidFormat = 781, + FileTranscriptionChunkFailed = 782, + FileTranscriptionCancelled = 783, + FileTranscriptionFileTooLarge = 784, + + // === Phase 12: Meeting Summary (785-789) === + MeetingSummaryGenerationFailed = 785, + MeetingSummaryNoTranscript = 786, + MeetingSummaryExportFailed = 787, + + // === Phase 12: Dictation Template (790-794) === + TemplateNotFound = 790, + TemplateSessionAlreadyActive = 791, + TemplateSessionNotActive = 792, + TemplateFieldRecordingFailed = 793, + TemplateInvalidFormat = 794, + + // === Phase 13: Voice Conversation (795-799) === + ConversationSessionAlreadyActive = 795, + ConversationNoActiveSession = 796, + ConversationTTSFailed = 797, + ConversationLLMFailed = 798, + + // === Phase 13: Local RAG (870-874) === + RAGDocumentNotFound = 870, + RAGIndexingFailed = 871, + RAGEmbeddingFailed = 872, + RAGQueryFailed = 873, + RAGUnsupportedFormat = 874, + + // === Phase 13: Voice Action (875-879) === + VoiceActionPlanFailed = 875, + VoiceActionExecutionFailed = 876, + VoiceActionBlocked = 877, + VoiceActionInvalidPlan = 878, + // === Config (800-849) === ConfigReadFailed = 800, ConfigWriteFailed = 801, diff --git a/src/shared/ipc-channels.ts b/src/shared/ipc-channels.ts index 42ffb71..04309de 100644 --- a/src/shared/ipc-channels.ts +++ b/src/shared/ipc-channels.ts @@ -221,6 +221,90 @@ export const IPC_CHANNELS = { STOP_SYSTEM_AUDIO: 'caption:stopSystemAudio', }, + // ── Phase 12: File Transcription (12.1) ── + FILE_TRANSCRIPTION: { + START: 'fileTranscription:start', + CANCEL: 'fileTranscription:cancel', + GET_STATE: 'fileTranscription:getState', + // Main → Renderer events + PROGRESS: 'fileTranscription:progress', + COMPLETE: 'fileTranscription:complete', + ERROR: 'fileTranscription:error', + }, + + // ── Phase 12: Meeting Summary (12.2) ── + MEETING_SUMMARY: { + SUMMARIZE: 'meetingSummary:summarize', + GET_SUMMARY: 'meetingSummary:getSummary', + EXPORT_MARKDOWN: 'meetingSummary:exportMarkdown', + // Main → Renderer events + SUMMARY_READY: 'meetingSummary:summaryReady', + SUMMARY_PROGRESS: 'meetingSummary:progress', + }, + + // ── Phase 12: Dictation Templates (12.3) ── + DICTATION_TEMPLATE: { + GET_ALL: 'dictationTemplate:getAll', + CREATE: 'dictationTemplate:create', + UPDATE: 'dictationTemplate:update', + DELETE: 'dictationTemplate:delete', + START_SESSION: 'dictationTemplate:startSession', + CANCEL_SESSION: 'dictationTemplate:cancelSession', + GET_SESSION_STATE: 'dictationTemplate:getSessionState', + SET_FIELD_VALUE: 'dictationTemplate:setFieldValue', + // Main → Renderer events + SESSION_STATE_CHANGED: 'dictationTemplate:sessionStateChanged', + FIELD_COMPLETED: 'dictationTemplate:fieldCompleted', + SESSION_COMPLETED: 'dictationTemplate:sessionCompleted', + }, + + // ── Phase 13: Voice Conversation (13.1) ── + VOICE_CONVERSATION: { + START_SESSION: 'voiceConversation:startSession', + STOP_SESSION: 'voiceConversation:stopSession', + SEND_MESSAGE: 'voiceConversation:sendMessage', + GET_STATE: 'voiceConversation:getState', + GET_HISTORY: 'voiceConversation:getHistory', + CLEAR_HISTORY: 'voiceConversation:clearHistory', + CANCEL_RESPONSE: 'voiceConversation:cancelResponse', + // Main → Renderer events + STATE_CHANGED: 'voiceConversation:stateChanged', + USER_MESSAGE: 'voiceConversation:userMessage', + ASSISTANT_DELTA: 'voiceConversation:assistantDelta', + ASSISTANT_MESSAGE: 'voiceConversation:assistantMessage', + TTS_STARTED: 'voiceConversation:ttsStarted', + TTS_FINISHED: 'voiceConversation:ttsFinished', + ERROR: 'voiceConversation:error', + }, + + // ── Phase 13: Local RAG (13.2) ── + RAG: { + ADD_DOCUMENT: 'rag:addDocument', + REMOVE_DOCUMENT: 'rag:removeDocument', + GET_DOCUMENTS: 'rag:getDocuments', + QUERY: 'rag:query', + GET_STATE: 'rag:getState', + REINDEX: 'rag:reindex', + // Main → Renderer events + INDEX_PROGRESS: 'rag:indexProgress', + INDEX_COMPLETE: 'rag:indexComplete', + QUERY_RESULT: 'rag:queryResult', + }, + + // ── Phase 13: Voice Action / OS Automation (13.3) ── + VOICE_ACTION: { + EXECUTE: 'voiceAction:execute', + GET_PRESETS: 'voiceAction:getPresets', + GET_HISTORY: 'voiceAction:getHistory', + CLEAR_HISTORY: 'voiceAction:clearHistory', + SET_ENABLED: 'voiceAction:setEnabled', + IS_ENABLED: 'voiceAction:isEnabled', + // Main → Renderer events + ACTION_PLANNED: 'voiceAction:actionPlanned', + ACTION_EXECUTED: 'voiceAction:actionExecuted', + ACTION_ERROR: 'voiceAction:actionError', + }, + // ── Phase 11: License & Monetization ── LICENSE: { GET_INFO: 'license:getInfo', diff --git a/src/shared/types.ts b/src/shared/types.ts index e0134b5..79e34c6 100644 --- a/src/shared/types.ts +++ b/src/shared/types.ts @@ -432,7 +432,7 @@ export interface HistoryEntry { focusedApp: string | null focusedAppName: string | null focusedAppWindowTitle: string | null - mode: 'dictation' | 'translate' | 'command' + mode: 'dictation' | 'translate' | 'command' | 'caption' | 'file-transcription' status: 'completed' | 'cancelled' | 'error' errorCode: string | null audioLocalPath: string | null @@ -447,6 +447,8 @@ export interface HistoryEntry { createdAt: number updatedAt: number appVersion: string + /** Phase 12.2: 회의록 요약 마크다운 */ + summaryText: string | null } export interface HistoryQueryParams { @@ -947,3 +949,316 @@ export interface TierComparison { pro: boolean | string proPlus: boolean | string } + +// ============================================================ +// Phase 12: File Transcription (12.1) +// ============================================================ + +export type FileTranscriptionState = 'idle' | 'converting' | 'transcribing' | 'completed' | 'error' + +export interface FileTranscriptionStartParams { + filePath: string + language?: string +} + +export interface FileTranscriptionProgress { + jobId: string + currentChunk: number + totalChunks: number + percent: number + currentText: string +} + +export interface FileTranscriptionSegment { + text: string + start: number + end: number + confidence: number +} + +export interface FileTranscriptionResult { + jobId: string + filePath: string + fileName: string + fullText: string + segments: FileTranscriptionSegment[] + totalDurationSec: number + processingTimeMs: number +} + +export interface FileTranscriptionStateInfo { + state: FileTranscriptionState + jobId: string | null + progress: FileTranscriptionProgress | null +} + +// ============================================================ +// Phase 12: Meeting Summary (12.2) +// ============================================================ + +export interface MeetingSummaryResult { + historyId: string + summary: string + decisions: string[] + actionItems: string[] + rawMarkdown: string + generatedAt: number +} + +export interface MeetingSummarizeParams { + historyId: string +} + +export interface MeetingSummaryGetParams { + historyId: string +} + +export interface MeetingSummaryExportParams { + historyId: string +} + +export interface MeetingSummaryProgress { + historyId: string + status: 'generating' | 'done' | 'error' +} + +// ============================================================ +// Phase 12: Dictation Templates (12.3) +// ============================================================ + +export interface TemplateField { + id: string + name: string + label: string + promptText: string + required: boolean + maxDurationSec: number +} + +export interface DictationTemplate { + id: string + name: string + description: string + fields: TemplateField[] + outputFormat: string + isBuiltin: boolean + createdAt: number + updatedAt: number +} + +export interface CreateTemplateParams { + name: string + description: string + fields: TemplateField[] + outputFormat: string +} + +export interface UpdateTemplateParams { + id: string + name?: string + description?: string + fields?: TemplateField[] + outputFormat?: string +} + +export interface DeleteTemplateParams { + id: string +} + +export type TemplateSessionState = + | 'idle' + | 'field-prompting' + | 'field-recording' + | 'completing' + +export interface TemplateSessionInfo { + templateId: string + templateName: string + state: TemplateSessionState + currentFieldIndex: number + totalFields: number + currentField: TemplateField | null + fieldValues: Record +} + +export interface StartTemplateSessionParams { + templateId: string +} + +export interface SetFieldValueParams { + fieldId: string + value: string +} + +export interface TemplateFieldCompletedEvent { + fieldId: string + fieldName: string + value: string + nextField: TemplateField | null +} + +export interface TemplateSessionCompletedEvent { + templateId: string + outputText: string + fieldValues: Record +} + +// ============================================================ +// Phase 13: Voice Conversation (13.1) +// ============================================================ + +export type ConversationState = + | 'idle' + | 'listening' + | 'thinking' + | 'speaking' + +export type ConversationRole = 'user' | 'assistant' | 'system' + +export interface ConversationMessage { + id: string + role: ConversationRole + content: string + timestamp: number +} + +export interface ConversationSessionInfo { + state: ConversationState + messages: ConversationMessage[] + isActive: boolean +} + +export interface ConversationSendParams { + text: string +} + +export interface ConversationAssistantDelta { + messageId: string + delta: string + accumulated: string +} + +export interface ConversationAssistantMessage { + messageId: string + content: string +} + +export interface ConversationError { + message: string + phase: 'stt' | 'llm' | 'tts' +} + +// ============================================================ +// Phase 13: Local RAG (13.2) +// ============================================================ + +export interface RAGDocument { + id: string + fileName: string + filePath: string + fileType: 'txt' | 'md' | 'pdf' | 'docx' + chunkCount: number + indexed: boolean + indexedAt: number | null + addedAt: number +} + +export interface RAGChunk { + id: string + documentId: string + content: string + embedding: number[] + chunkIndex: number +} + +export interface RAGQueryParams { + query: string + topK?: number +} + +export interface RAGQueryResult { + query: string + results: Array<{ + documentId: string + fileName: string + content: string + similarity: number + }> + /** LLM 답변 (컨텍스트 주입 후) */ + answer: string +} + +export interface RAGAddDocumentParams { + filePath: string +} + +export interface RAGRemoveDocumentParams { + documentId: string +} + +export interface RAGIndexProgress { + documentId: string + fileName: string + currentChunk: number + totalChunks: number + percent: number +} + +export type RAGState = 'idle' | 'indexing' | 'querying' + +export interface RAGStateInfo { + state: RAGState + documentCount: number + totalChunks: number +} + +// ============================================================ +// Phase 13: Voice Action / OS Automation (13.3) +// ============================================================ + +export type VoiceActionType = + | 'open_app' + | 'open_url' + | 'open_file' + | 'keyboard_shortcut' + | 'type_text' + | 'system_command' + +export interface VoiceActionPlan { + action: VoiceActionType + target: string + description: string + safe: boolean +} + +export interface VoiceActionExecuteParams { + text: string +} + +export interface VoiceActionPreset { + keywords: string[] + action: VoiceActionPlan +} + +export interface VoiceActionHistoryEntry { + id: string + userText: string + plan: VoiceActionPlan + executed: boolean + timestamp: number +} + +export interface VoiceActionPlannedEvent { + plan: VoiceActionPlan + userText: string +} + +export interface VoiceActionExecutedEvent { + plan: VoiceActionPlan + success: boolean +} + +export interface VoiceActionErrorEvent { + message: string + userText: string +}