d3ro-voice/docs/map/01-system-overview.md

01 — System Overview & Information Architecture

Surface: whole product Source of truth for: vision, platforms, IA, identity/data model, AI pipeline, tiers

---

1. Product in one line

A multi-platform AI voice assistant: press/hold or tap to speak, get a transcript, optionally run it through an LLM (cleanup, translate, summarize, execute a command), and keep the result in a searchable history that syncs across desktop, web, and mobile.

---

2. Surfaces (apps) and their roles

Surface	Path	Stack	Runtime model	Role
Desktop	apps/desktop	Electron 33 + React 19 + MUI 7 + Vite	Local-first (SoX, faster-whisper sidecar, Ollama, SQLite) with optional cloud sync	The flagship: global hotkey dictation, text insertion into other apps, meetings, captions, RAG, voice conversation, OS actions
Web	apps/web	Next.js 15 App Router + Supabase	Cloud	Browser console: record/STT, history, commands, meetings, knowledge, teams, chat, billing
Mobile	apps/mobile-rn	React Native 0.85 + React 19 (CLI, not Expo)	Cloud-first (Supabase + Edge Functions), on-device Whisper fallback	Product mobile app: recording/import, history, meetings, memos, templates, teams, Talk, admin, data portability, IAP + ads
API server	apps/api-server	ASP.NET Core 10 + EF Core + SQLite	Cloud (self-hosted/NAS)	LLM/STT proxy and admin back-office backend for the .NET identity side
Admin console	apps/admin	Next.js 16 + MUI	Cloud	Back office: users, subscriptions, models/STT providers, usage, audit log, releases, ads
Landing site	site/	Vite 6 + React 19 + Tailwind	Static	Marketing/download/legal pages, deployed to Cloudflare Pages + GitHub Pages
Edge gateway	server/cloudflare-worker	Cloudflare Worker (TS)	Edge	CORS + proxy to the NAS-hosted API origin
Backend data/functions	server/supabase	Postgres + Deno Edge Functions	Cloud	Canonical product data, auth, RLS, storage, AI proxies, billing, delivery

---

3. Information architecture (feature domains)

The product IA is stable across surfaces; each surface implements a subset.

D3RO Voice
├── Capture & Transcribe
│   ├── Dictation (hold/release, push-to-talk)
│   ├── Hands-free (toggle)
│   ├── File transcription (audio/video)
│   ├── Live captions (system audio)
│   └── Multiple STT engines (local Whisper, cloud providers)
├── AI Processing
│   ├── Local LLM (Ollama) / Cloud LLM (Claude, OpenAI)
│   ├── Auto Polish / cleanup / translate / summarize
│   ├── Custom instructions (user commands)
│   ├── Voice commands (keyword → command)
│   └── LLM Chains (multi-step pipelines)
├── Memory & Knowledge
│   ├── History (search, favorites, export)
│   ├── Dictionary (custom vocabulary)
│   ├── Memos (tags over history)
│   ├── Knowledge base (local RAG / cloud RAG)
│   └── Voice actions (OS automation)
├── Meetings
│   ├── Meeting recording + live transcript
│   ├── Timestamped memos
│   ├── AI summary + speaker diarization
│   ├── Document generation (minutes/report/idea-note/mindmap)
│   └── Export (PDF/DOCX/TXT/Markdown)
├── Conversation
│   ├── Local duplex conversation (STT→LLM→TTS)
│   └── Realtime conversation (OpenAI gpt-realtime, Premium)
├── Accounts & Sync
│   ├── Supabase auth (email + Google/GitHub/Apple OAuth)
│   ├── Cloud sync (per-user DB/rows)
│   ├── Devices (registration, revocation)
│   ├── Teams (members, invites, roles)
│   └── Data portability (export/import, account delete)
├── Monetization
│   ├── Tiers: Free / Pro / Pro+ / Team / Enterprise
│   ├── Desktop licenses (Ed25519, offline)
│   ├── Web billing (Stripe + Payple)
│   ├── Mobile IAP (Google Play / App Store)
│   └── Free-tier ads (AdMob rewarded + banner, mediation roster)
├── Platform Shell
│   ├── Settings / preferences / themes (6 themes)
│   ├── Onboarding
│   ├── Notifications / push
│   ├── Support & content reporting
│   └── Admin & audit

Legacy/other: voice keyword shortcuts, screen/context capture, auto-launch, system tray.

---

4. Identity & data model (multi-source, converging)

There are three identity/data systems in the repo. This is a known architectural tension (see 11-gap-backlog.md G-01).

System	Where	Stores	Status
Supabase Auth + Postgres	server/supabase	Canonical product users, profiles, subscriptions, history, meetings, teams, knowledge, push, billing, ads	Canonical SSOT for web + mobile
.NET API server	apps/api-server	Its own SQLite Users (JWT, roles), model/STT endpoints, usage/error logs, admin audit	Back-office + AI proxy; legacy SHA-256 users force-disabled at startup
Desktop local license	apps/desktop	Ed25519-signed offline license key, local SQLite DB per user (_local for anonymous)	Local-first tier gating + optional Supabase cloud sync

Data flow:

• Desktop: local SQLite (per-user file) ↔ optional Supabase sync (history/dictionary/meetings).
• Web/Mobile: Supabase directly (tables + RLS) and via Edge Functions.
• Admin: Next.js server routes → Supabase service role and/or .NET /api/admin/*.

---

5. AI pipeline

Local path (desktop): mic → SoX/native capture (PCM16 16kHz mono) → faster-whisper Python sidecar → optional Ollama LLM → SQLite history → clipboard/text insertion into the active app.

Cloud path (web/mobile/desktop online):

• STT → Supabase Edge Function stt-proxy (quota reservation, provider fallback) or .NET SttProxyService (internal gateway token only).
• LLM → Supabase llm-proxy / realtime-token (Claude/OpenAI) or .NET LlmProxyService.
• Meetings/documents → generate-meeting-document, embed-chunks, search-knowledge.

Realtime voice (Premium): OpenAI gpt-realtime-2.1 via ephemeral token from realtime-token, WebRTC in the desktop renderer with local-pipeline fallback.

STT providers supported by the desktop dispatcher (apps/desktop/src/main/services/stt): local Whisper, OpenAI, Groq, Deepgram, AssemblyAI, Google, Custom (OpenAI-compatible), and D3RO Cloud.

---

6. Monetization tiers

Tier	Notes
Free	Quotas on STT/LLM; free-tier ads (desktop/mobile).
Pro / Pro+	Paid subscriptions. Desktop: offline Ed25519 license. Web: Stripe/Payple. Mobile: Google Play Billing.
Team / Enterprise	Teams, shared meetings, admin roles.

Desktop license verification is Ed25519 (public key in release/desktop-license-public.pem); the private key was rotated out of the repo. Mobile release evidence uses a separate Ed25519 keypair.

---

7. Platform architecture diagram (text)

                    ┌──────────────────────────────────────────┐
                    │              Supabase (SSOT)             │
                    │  Postgres+RLS · Auth · Storage · Realtime │
                    │  ~27 Deno Edge Functions                 │
                    └───────────────┬──────────────────────────┘
             ┌──────────────┬───────┴────────┬───────────────┐
             │              │                │               │
        apps/web       apps/mobile-rn    apps/desktop    apps/admin
        (Next.js)      (React Native)    (Electron)      (Next.js)
             │              │                │               │
             └──────────────┴────────────────┘               │
                            │                                 │
                    ┌───────┴────────┐              ┌─────────┴─────────┐
                    │ Cloudflare      │              │ apps/api-server   │
                    │ Worker (edge)   │──────────────▶ .NET 10 + SQLite  │
                    └───────┬────────┘              │ + admin audit     │
                            │                        └───────────────────┘
                    Cloudflare Tunnel
                            │
                    ┌───────┴────────┐
                    │ D3RO NAS        │  docker-compose.nas.yml
                    │ API + Admin     │  d3ro.chanpaca.net / admin.chanpaca.net
                    └────────────────┘

---

8. Cross-cutting concerns

Concern	Implementation
Design system	packages/ui (web/desktop, MUI + tokens), packages/ui-native (mobile). Theme SSOT theme.ts (d3roPalette/d3roTypo/d3roShadow/d3roRadius). 6 themes.
i18n	packages/i18n, 12 locales, ko master, t() + type-safe keys.
IPC	packages/core/src/ipc-channels.ts is the channel SSOT; desktop preload exposes window.electronAPI (33 namespaces).
Errors	D3ROError + ErrorCode, IPCResult<T> envelope.
Crypto	Ed25519 license signing (packages/core/src/utils/crypto-license), HMAC admin sessions, PKCE on mobile.
Observability	LoggerService (electron-log) on desktop; ServerErrorLog/ApiUsageLog/SttUsageLog in .NET; admin audit log in Supabase.
Security posture	Fail-closed defaults: STT/LLM never return synthetic success; admin panels show explicit "unavailable" rather than sample data; secret scanning in CI.

---

9. Related deep documents

• Desktop architecture: docs/design/00-master-architecture.md … 09-history-popup.md
• Build-out history: docs/phases/phase-1.md … phase-15.5-speaker-diarization.md
• Multi-platform plan: docs/v2/00-v2-master-plan.md
• Mobile SSOT (authoritative checklist): docs/v3/MOBILE_APP_COMPLETION_SSOT.md
• Monetization: docs/monetization-plan.md
• Release process: docs/deployment/release-guide.md, docs/deployment/nas-deployment-guide.md