2d585bfc29
Local dictation had never produced a transcript on an installed build. The engine itself was healthy; every connection to it was broken. Installed builds shipped no speech engine at all: the packaging config had no entry for the faster-whisper sidecar and no pipeline step built one, so the app always fell back to a system Python without the runtime. Development was broken too, because the sidecar and SoX paths were resolved against the Vite output directory instead of the app root, which also meant recording failed with a SoX ENOENT. On hosts where localhost resolves only to IPv6, every local request was refused outright, which silently disabled both local transcription and the local LLM. The sidecar is now built and bundled (including the Silero VAD data it needs), gated by a packaging check that fails when the engine or its data is missing. Paths are discovered from the app root and fail loudly when the engine is absent. Local engine URLs are normalized to the IPv4 loopback, decoding is tuned so repeated hallucinations cannot compound (the same transcript now takes about a fifth of the time), the engine is warmed up at startup, and holding the hotkey now shows the text forming live in the recording tip.
206 lines
17 KiB
Markdown
206 lines
17 KiB
Markdown
# 10 — Feature Catalog (Feature Map)
|
|
|
|
> The canonical feature map. One row = one user-facing capability.
|
|
> Status is per surface. Legend in [`00-index.md`](./00-index.md) §2.
|
|
> Surfaces: D = desktop (`apps/desktop`), W = web (`apps/web`), M = mobile (`apps/mobile-rn`), B = backend (Supabase/.NET).
|
|
|
|
**How to use an ID:** cite it in commits, plans, and backlog. Example: "CAP-03 is `[~]` on mobile (external OAuth pending)".
|
|
|
|
Status quick-reference: `[x]` done+verified · `[~]` partial/unverified · `[ ]` planned/absent · `[!]` blocked externally · `[-]` N/A.
|
|
|
|
---
|
|
|
|
## CAP — Capture & Transcribe
|
|
|
|
| ID | Feature | D | W | M | B | Anchors / notes |
|
|
|---|---|---|---|---|---|---|
|
|
| CAP-01 | Push-to-talk dictation (hold/release) | [x] | [-] | [x] | [-] | Desktop `VoiceModeService`; mobile RecordScreen via app CTA/notification action (no global hotkey) |
|
|
| CAP-02 | Hands-free toggle dictation | [x] | [-] | [x] | [-] | Desktop double-press; mobile toggle |
|
|
| CAP-03 | Live partial transcript while recording | [x] | [ ] | [ ] | [-] | Desktop `voice:partialTranscript` + recording-tip; producer added in `1.3.0` (`VoiceModeService._runPartial` → `LocalSTTService.transcribePartial`, 1.5 s cadence / 7.5 s window, never inserted). The row was `[x]` before any producer existed. |
|
|
| CAP-04 | Recording waveform + level meter | [x] | [x] | [x] | [-] | Desktop 9-bar cos distribution; mobile audio level |
|
|
| CAP-05 | Device/mic selection | [x] | [ ] | [~] | [-] | Desktop config; mobile uses system default |
|
|
| CAP-06 | System/loopback audio capture | [x] | [-] | [ ] | [-] | Desktop only (caption source); mobile policy-limited |
|
|
| CAP-07 | Local Whisper STT | [x] | [-] | [x] | [-] | Desktop ships the faster-whisper sidecar (`resources/sidecar`, built by `sidecar:build`, verified by `scripts/ci/verify-sidecar-bundle.mjs`), warms it up at app start, and connects over IPv4 loopback; mobile on-device Whisper (supported devices) |
|
|
| CAP-08 | Cloud STT (multi-provider) | [x] | [x] | [x] | [x] | Desktop 6 providers + D3RO Cloud; web/mobile via `stt-proxy`; .NET internal gateway |
|
|
| CAP-09 | STT auto-fallback + fail-closed | [x] | [x] | [x] | [x] | `STTManager`; SSOT R-021/R-022 GREEN |
|
|
| CAP-10 | STT model download/management UI | [x] | [-] | [~] | [-] | Desktop model manager + onboarding; mobile bundled model |
|
|
| CAP-11 | File transcription (audio/video) | [x] | [ ] | [x] | [~] | Desktop ffmpeg chunking; mobile import picker; web deferred |
|
|
| CAP-12 | Audio import from other apps (share intent) | [-] | [-] | [x] | [-] | Mobile Android `ACTION_SEND`/`ACTION_VIEW` (SSOT R-016 GREEN) |
|
|
| CAP-13 | Live captions overlay | [x] | [-] | [-] | [-] | Desktop `CaptionService` + caption-overlay popup |
|
|
| CAP-14 | Recording persistence / crash recovery | [x] | [ ] | [x] | [-] | Desktop WAV persist; mobile durable queue + process-kill WAV recovery |
|
|
| CAP-15 | Android foreground recording service | [-] | [-] | [x] | [-] | Mobile API 34 FGS + persistent notification (SSOT R-005 GREEN) |
|
|
|
|
---
|
|
|
|
## AI — AI Processing
|
|
|
|
| ID | Feature | D | W | M | B | Anchors / notes |
|
|
|---|---|---|---|---|---|---|
|
|
| AI-01 | Local LLM (Ollama) | [x] | [-] | [ ] | [-] | Desktop bundled Ollama |
|
|
| AI-02 | Cloud LLM (Claude/OpenAI) | [x] | [x] | [x] | [x] | Desktop `PremiumLLMService`; web/mobile via `llm-proxy`; .NET `LlmProxyService` |
|
|
| AI-03 | Auto Polish (cleanup/filler removal) | [x] | [~] | [~] | [x] | Desktop built-in; web/mobile via commands |
|
|
| AI-04 | Translate / summarize / rephrase | [x] | [x] | [x] | [x] | Built-in instructions |
|
|
| AI-05 | Custom instructions (user commands) | [x] | [x] | [x] | [x] | Desktop `CommandsPage` (Red Team RT-03 verified); web `commands`; mobile `CommandsScreen` |
|
|
| AI-06 | Voice keyword commands | [x] | [-] | [ ] | [-] | Desktop `VoiceCommandService` + command popup |
|
|
| AI-07 | LLM Chains (multi-step pipelines) | [x] | [ ] | [ ] | [-] | Desktop `ChainService` |
|
|
| AI-08 | Screen/context capture for prompts | [x] | [-] | [ ] | [-] | Desktop `ScreenContextService` |
|
|
| AI-09 | Streaming responses | [x] | [x] | [x] | [x] | SSE/NDJSON streaming |
|
|
| AI-10 | Dictation templates (voice form fill) | [x] | [ ] | [x] | [~] | Desktop `DictationTemplateService`; mobile `TemplatesScreen` |
|
|
|
|
---
|
|
|
|
## MEM — Memory & Knowledge
|
|
|
|
| ID | Feature | D | W | M | B | Anchors / notes |
|
|
|---|---|---|---|---|---|---|
|
|
| MEM-01 | History list + search | [x] | [x] | [x] | [x] | Desktop SQLite; web/mobile Supabase |
|
|
| MEM-02 | History detail + edit | [x] | [x] | [x] | [x] | Mobile `HistoryDetailScreen` |
|
|
| MEM-03 | History favorites | [x] | [x] | [x] | [x] | |
|
|
| MEM-04 | History export / share | [x] | [~] | [x] | [x] | Desktop export; web limited; mobile share sheet |
|
|
| MEM-05 | History audio playback | [x] | [x] | [x] | [x] | Signed URLs on web/mobile |
|
|
| MEM-06 | Dictionary (custom vocabulary) | [x] | [x] | [x] | [x] | All surfaces CRUD; Desktop Red Team RT-02 & RT-18 fuzzed/verified |
|
|
| MEM-07 | Dictionary import/export | [x] | [x] | [x] | [-] | Desktop `dictionary:import/export` JSON+CSV (file dialogs, `DictionaryService`); web `serializeDictionary`/`importDictionaryFile` + header buttons; mobile CSV/TXT export + CSV/JSON/TXT import via `data-portability` |
|
|
| MEM-08 | Memos (tags over history) | [x] | [ ] | [x] | [x] | Desktop `MemoService`; mobile `MemosScreen`; web none |
|
|
| MEM-09 | Knowledge base / RAG add+index | [x] | [x] | [x] | [x] | Desktop local RAG (DEF-008 infinite chunking loop resolved, RT-08 verified); web/mobile cloud RAG |
|
|
| MEM-10 | Semantic search over knowledge | [x] | [x] | [x] | [x] | Web `KnowledgeSearch` calls `search-knowledge` (was mislabeled deferred); mobile + Edge `search-knowledge` |
|
|
| MEM-11 | Knowledge file upload | [x] | [x] | [x] | [x] | Web `.txt`/`.md` picker + newline-aware chunking + `embed-chunks`; desktop txt/md/pdf/docx; mobile file picker |
|
|
| MEM-12 | Voice actions (OS automation) | [x] | [x] | [x] | [x] | Desktop `VoiceActionService`; web `ActionRunner` (simulated); mobile `ActionsScreen` |
|
|
| MEM-13 | Cross-surface data sync | [~] | [x] | [x] | [x] | Desktop Supabase sync (V2-4); web/mobile native |
|
|
| MEM-14 | Memo tag search | [x] | [ ] | [x] | [x] | |
|
|
|
|
---
|
|
|
|
## MTG — Meetings
|
|
|
|
| ID | Feature | D | W | M | B | Anchors / notes |
|
|
|---|---|---|---|---|---|---|
|
|
| MTG-01 | Meeting recording + live transcript | [x] | [~] | [x] | [x] | Desktop `MeetingModeService`; web realtime view; mobile |
|
|
| MTG-02 | Meeting creation (title/attendees/language/template) | [x] | [x] | [x] | [x] | SSOT H-006 GREEN |
|
|
| MTG-03 | Timestamped memos during meeting | [x] | [x] | [x] | [x] | |
|
|
| MTG-04 | AI summary generation | [x] | [x] | [~] | [x] | Desktop `MeetingSummaryService`; web `generate-document-button`; mobile via Edge |
|
|
| MTG-05 | Document generation (minutes/report/idea-note/mindmap) | [x] | [x] | [x] | [x] | Edge `generate-meeting-document`; SSOT F-011 GREEN |
|
|
| MTG-06 | Document edit (Markdown) | [x] | [x] | [x] | [x] | Desktop editor; web `document-editor` |
|
|
| MTG-07 | Export PDF/DOCX/TXT/Markdown | [x] | [~] | [x] | [x] | Desktop `ExportMenu`; web markdown; mobile print/DOCX chooser (SSOT data portability GREEN) |
|
|
| MTG-08 | Speaker diarization | [~] | [ ] | [ ] | [ ] | Desktop `phase-15.5` (LLM estimate + pyannote prep); mobile SSOT H-014 pending |
|
|
| MTG-09 | Audio seek ↔ transcript timestamp | [ ] | [ ] | [ ] | [ ] | SSOT H-011 pending |
|
|
| MTG-10 | Meeting list search/filter/sort | [~] | [ ] | [~] | [x] | Basic lists; advanced filters pending |
|
|
| MTG-11 | Meeting AI chat over transcript | [x] | [ ] | [ ] | [x] | Desktop `MeetingChatPanel` + `MEETING_CHAT` |
|
|
| MTG-12 | Content reporting for generated docs | [-] | [ ] | [x] | [x] | `content-report` Edge + generation receipts |
|
|
|
|
---
|
|
|
|
## CV — Conversation
|
|
|
|
| ID | Feature | D | W | M | B | Anchors / notes |
|
|
|---|---|---|---|---|---|---|
|
|
| CV-01 | Local duplex voice conversation (STT→LLM→TTS) | [x] | [ ] | [x] | [-] | Desktop `VoiceConversationService`; mobile Talk |
|
|
| CV-02 | Realtime voice (OpenAI gpt-realtime, Premium) | [x] | [ ] | [ ] | [x] | Desktop `useRealtimeConversation` + `realtime-token` Edge |
|
|
| CV-03 | Text AI chat | [x] | [x] | [x] | [x] | Desktop chat, web `chat-panel`, mobile `TalkScreen` |
|
|
| CV-04 | TTS playback + controls | [x] | [ ] | [x] | [-] | Desktop SAPI/`say`; mobile Android TTS |
|
|
| CV-05 | Voice selection / backend selection | [x] | [ ] | [~] | [-] | Desktop settings `conversationBackend` |
|
|
|
|
---
|
|
|
|
## ACC — Accounts, Sync, Devices, Portability
|
|
|
|
| ID | Feature | D | W | M | B | Anchors / notes |
|
|
|---|---|---|---|---|---|---|
|
|
| ACC-01 | Email sign-up/login | [ ] | [x] | [x] | [x] | Desktop uses separate online-auth; SSOT A-001..A-005 `[ ]` rows are mobile checklist granularity |
|
|
| ACC-02 | OAuth Google | [~] | [x] | [~] | [x] | Mobile full consent→callback pending (external) |
|
|
| ACC-03 | OAuth GitHub / Apple | [~] | [~] | [~] | [~] | GitHub/Apple provider secrets pending |
|
|
| ACC-04 | Password reset / recovery deep link | [ ] | [ ] | [x] | [x] | Mobile ForgotPassword/UpdatePassword |
|
|
| ACC-05 | Account profile / identity management | [~] | [~] | [x] | [x] | Desktop `LicenseTab`; mobile `AccountScreen` |
|
|
| ACC-06 | Logout + local sensitive purge | [x] | [x] | [x] | [x] | Mobile central purge GREEN |
|
|
| ACC-07 | Account deletion (server cascade + local purge) | [ ] | [ ] | [~] | [x] | `account-delete` Edge; mobile device E2E pending |
|
|
| ACC-08 | Cloud sync (per-user data) | [x] | [x] | [x] | [x] | Desktop `CloudSyncService`; SSOT D-* largely `[ ]` granular |
|
|
| ACC-09 | Device registration + revocation | [-] | [ ] | [x] | [x] | Mobile `DevicesScreen` |
|
|
| ACC-10 | Offline queue + retry | [~] | [ ] | [x] | [x] | Mobile durable queue |
|
|
| ACC-11 | Data export/import (portability) | [~] | [ ] | [x] | [x] | Mobile canonical JSON E2E GREEN; desktop has export files |
|
|
| ACC-12 | Notification / push | [ ] | [~] | [~] | [x] | Backend transports for FCM + Web Push (VAPID) + APNs (.p8) + outbox cron drain implemented. Web/mobile client registration for webpush/apns still pending; Android delivery needs FCM project. |
|
|
|
|
---
|
|
|
|
## TEAM — Teams & Admin
|
|
|
|
| ID | Feature | D | W | M | B | Anchors / notes |
|
|
|---|---|---|---|---|---|---|
|
|
| TEAM-01 | Team create / rename / delete | [-] | [x] | [x] | [x] | Desktop N/A |
|
|
| TEAM-02 | Invite by email / link + accept deep link | [-] | [x] | [x] | [x] | `team-invite`/`team-accept` |
|
|
| TEAM-03 | Members + roles (admin/member/viewer) + leave | [-] | [x] | [x] | [x] | |
|
|
| TEAM-04 | Team meetings/docs sharing + RLS isolation | [-] | [x] | [~] | [x] | Cross-user isolation tested |
|
|
| TEAM-05 | Team comments / activity feed | [-] | [x] | [x] | [x] | `team_activities` migration + `create_team_activity` RPC + realtime; web `ActivityFeed`, mobile TeamDetail activity card (2026-09-13) |
|
|
| TEAM-06 | Admin back office (users/subs/models/usage/audit) | [-] | [-] | [x] | [x] | `apps/admin` + mobile `AdminScreen` |
|
|
| TEAM-07 | Role-based destructive action confirm + audit | [-] | [x] | [x] | [x] | SSOT T-006..T-008 GREEN |
|
|
| TEAM-08 | Desktop admin surface | [ ] | [-] | [-] | [-] | None; N/A by design |
|
|
|
|
---
|
|
|
|
## MON — Monetization & Ads
|
|
|
|
| ID | Feature | D | W | M | B | Anchors / notes |
|
|
|---|---|---|---|---|---|---|
|
|
| MON-01 | Tier gating (Free/Pro/Pro+/Team/Enterprise) | [x] | [x] | [x] | [x] | `LicenseService`, entitlement provider |
|
|
| MON-02 | Usage quotas (daily_usage) | [x] | [x] | [x] | [x] | |
|
|
| MON-03 | Desktop offline license (Ed25519) | [x] | [-] | [-] | [x] | `crypto-license` + admin issuer |
|
|
| MON-04 | Web checkout (Stripe) | [-] | [x] | [~] | [x] | Stripe checkout/portal/webhook |
|
|
| MON-05 | Web checkout (Payple) | [-] | [x] | [~] | [~] | Payple checkout/manage/renew/webhook; webhook signature pending |
|
|
| MON-06 | Paywall / upgrade prompts | [x] | [x] | [x] | [x] | `UpgradePromptModal`, `ProPaywallScreen` |
|
|
| MON-07 | Mobile IAP purchase + restore | [-] | [-] | [~] | [x] | `iap-verify` + `billing-context`; live store E2E blocked |
|
|
| MON-08 | Billing catalog / pricing display | [x] | [x] | [x] | [x] | Server catalog SSOT; hardcoded prices removed |
|
|
| MON-09 | Free-tier banner ads | [~] | [-] | [x] | [x] | Desktop adapters fail-closed; mobile AdMob test GREEN, prod serving blocked |
|
|
| MON-10 | Rewarded ads → quota credits | [~] | [-] | [x] | [x] | Desktop `RewardedQuotaModal` (stub adapters); mobile SSV GREEN |
|
|
| MON-11 | Ad mediation engine + settlement | [~] | [-] | [~] | [x] | Engine + settlement built. `DirectHouseSponsorAdapter` is now a **real configurable REST adapter** (bid/impression/click/reward via `endpointUrl`, fail-closed when unconfigured, unit-tested). Other 9 networks remain `UnavailableAdAdapter` stubs pending official SDKs. |
|
|
| MON-12 | Subscription management (portal/store) | [-] | [x] | [x] | [x] | Stripe portal / Payple manage / Play manage |
|
|
|
|
---
|
|
|
|
## SHELL — Platform Shell, Settings, Onboarding, Support
|
|
|
|
| ID | Feature | D | W | M | B | Anchors / notes |
|
|
|---|---|---|---|---|---|---|
|
|
| SHELL-01 | Settings / preferences | [x] | [~] | [x] | [x] | Desktop tabbed modal; web theme/i18n; mobile `SettingsScreen` |
|
|
| SHELL-02 | Theme system (6 themes) | [x] | [x] | [x] | [-] | `theme.ts` SSOT |
|
|
| SHELL-03 | i18n (12 locales) | [x] | [x] | [x] | [-] | `@d3ro/i18n`; ko/en fully translated, others partial |
|
|
| SHELL-04 | Onboarding / first-run | [x] | [ ] | [x] | [-] | Desktop model bootstrap; mobile audience/theme/locale |
|
|
| SHELL-05 | Accessibility / reduced motion | [~] | [~] | [~] | [-] | Desktop reduced-motion honored; mobile a11y rows pending |
|
|
| SHELL-06 | System tray / background | [x] | [-] | [-] | [-] | Desktop tray |
|
|
| SHELL-07 | Auto-launch on login | [x] | [-] | [-] | [-] | Desktop only |
|
|
| SHELL-08 | Auto-update | [x] | [-] | [!] | [-] | Desktop electron-updater; Forgejo canonical feed; channels + mandatory/major-vs-delta policy (`release/update-policy.json`); mobile store updates |
|
|
| SHELL-09 | Support / diagnostics | [x] | [ ] | [ ] | [~] | Desktop `SupportModal`; admin `/support` stub |
|
|
| SHELL-10 | Download center / releases | [-] | [x] | [-] | [x] | Web `/download`, admin `/releases`, Forgejo |
|
|
| SHELL-11 | Landing site / legal pages | [-] | [-] | [-] | [-] | `site/` — privacy/terms/delete-account live |
|
|
| SHELL-12 | Notifications (in-app / desktop) | [x] | [ ] | [x] | [x] | Desktop events; mobile push |
|
|
|
|
---
|
|
|
|
## INFRA — Build, CI, Release, Quality
|
|
|
|
| ID | Feature | Status | Anchors |
|
|
|---|---|---|---|
|
|
| INFRA-01 | Monorepo + workspaces + turbo | [x] | `package.json`, `turbo.json` |
|
|
| INFRA-14 | Unified entitlement resolver (`@d3ro/core/entitlement`) | [x] | `EntitlementSnapshot` + `resolveEntitlement` map Supabase/desktop-license/.NET sources to one contract; desktop `isPro` fixed, `syncFromCloud` normalized; mobile/web adoption incremental (see `11` GAP-ID-02) |
|
|
| INFRA-02 | Shared packages | [x] | `packages/*` |
|
|
| INFRA-03 | Desktop build + signed packaging | [x] | `electron-builder.yml`, GitLab `package-windows/macos` |
|
|
| INFRA-04 | Mobile CI (debug/E2E/release) | [~] | `.github/workflows/ci.yml`, `.gitlab-ci.yml`; production AAB external |
|
|
| INFRA-05 | .NET API tests | [x] | `apps/api-server.Tests` |
|
|
| INFRA-06 | Edge function tests (Deno) | [x] | `server/supabase/functions`, `tests/` |
|
|
| INFRA-07 | E2E desktop (playwright) | [x] | `apps/desktop/tests`, `apps/desktop/playwright.config.ts` |
|
|
| INFRA-08 | E2E web (playwright) | [x] | `apps/web/e2e` |
|
|
| INFRA-09 | E2E mobile (Maestro + instrumentation) | [~] | `.maestro/`, `androidTest`; emulator API 35 gate in CI |
|
|
| INFRA-10 | Secret scanning / release boundaries | [x] | `scripts/ci/check-no-hardcoded-secrets.mjs`, `verify-mobile-release-*.mjs` |
|
|
| INFRA-11 | Docker + NAS deploy | [x] | `docker-compose.nas.yml`, `scripts/deploy-nas.*` |
|
|
| INFRA-12 | Cloudflare edge + tunnel | [x] | `server/cloudflare-worker`, Cloudflare Tunnel `kd-nas` |
|
|
| INFRA-13 | Site deploy (Cloudflare Pages + GitHub Pages) | [x] | `.forgejo/workflows/deploy-site.yml`, `.github/workflows/deploy-site.yml` |
|
|
| INFRA-15 | Update & release system | [x] | Canonical Forgejo feed + channels/policy (`release/update-policy.json`, `src/main/update-policy.ts`), canonical publisher `scripts/ci/publish-forgejo-release.mjs`, legacy GitLab mirror; `npm run release:metadata:test`. v1.1.0 was published to Forgejo on 2026-09-15; product version moved to `1.2.0` as a forward-fix with CI-only publication, a same-version re-release guard, and download centers that link the feed instead of repository paths. `1.3.0` (2026-09-18) carries the local-STT fixes; Windows publication still needs the CI signing secrets (`11` GAP-REL-02). |
|
|
| INFRA-16 | Desktop STT engine packaging | [x] | `apps/desktop/scripts/setup-sidecar.mjs` + `build-sidecar.mjs`, `electron-builder.yml` `extraResources` (`sidecar-dist/sidecar` → `resources/sidecar`, `resources/ffmpeg`), and `scripts/ci/verify-sidecar-bundle.mjs` run in `package-windows`/`package-macos` before electron-builder. Verified on the real bundle: `sidecar.exe` + `_internal` including `faster_whisper/assets/silero_vad_v6.onnx`, plus a packaged-engine transcription round-trip on GPU. |
|
|
|
|
---
|
|
|
|
## Coverage summary (by surface)
|
|
|
|
| Surface | `[x]` | `[~]` | `[ ]` | Notable strength | Notable weakness |
|
|
|---|---|---|---|---|---|
|
|
| Desktop | ~40 | 3 | ~8 | Local AI pipeline, meetings, RAG, conversation, hotkeys | Ads stubs, no team admin, no email account |
|
|
| Web | ~22 | 6 | ~14 | Server-shared data UX, billing, meetings, teams | No local AI, limited knowledge upload/search |
|
|
| Mobile | ~40 | 12 | ~18 | Cloud + native recording, portability, admin, IAP/ads | External store/console gates, a11y, deep E2E pending |
|
|
| Backend | ~45 | 6 | ~4 | RLS, Edge functions, billing, fail-closed AI | Payple webhook signature, some external provider keys |
|