c3ddd36c6f
Some checks failed
deploy-site / deploy (push) Failing after 40s
Release notes for 1.1.0 were split between an Unreleased section and the version section, so the published notes would have omitted the update-feed and desktop changes. Everything shipping in this version now sits under one `## [1.1.0]` heading. `docs/map/` becomes the entry point for what infrastructure exists per platform and how far each feature is developed, with a documented update protocol so feature work and this map do not drift apart again. The release guide now states that installer binaries live in the update feed rather than the repository.
181 lines
10 KiB
Markdown
181 lines
10 KiB
Markdown
# 01 — System Overview & Information Architecture
|
|
|
|
> Surface: whole product
|
|
> Source of truth for: vision, platforms, IA, identity/data model, AI pipeline, tiers
|
|
|
|
---
|
|
|
|
## 1. Product in one line
|
|
|
|
A multi-platform AI voice assistant: press/hold or tap to speak, get a transcript, optionally run it through an LLM (cleanup, translate, summarize, execute a command), and keep the result in a searchable history that syncs across desktop, web, and mobile.
|
|
|
|
---
|
|
|
|
## 2. Surfaces (apps) and their roles
|
|
|
|
| Surface | Path | Stack | Runtime model | Role |
|
|
|---|---|---|---|---|
|
|
| Desktop | `apps/desktop` | Electron 33 + React 19 + MUI 7 + Vite | **Local-first** (SoX, faster-whisper sidecar, Ollama, SQLite) with optional cloud sync | The flagship: global hotkey dictation, text insertion into other apps, meetings, captions, RAG, voice conversation, OS actions |
|
|
| Web | `apps/web` | Next.js 15 App Router + Supabase | **Cloud** | Browser console: record/STT, history, commands, meetings, knowledge, teams, chat, billing |
|
|
| Mobile | `apps/mobile-rn` | React Native 0.85 + React 19 (CLI, not Expo) | **Cloud-first** (Supabase + Edge Functions), on-device Whisper fallback | Product mobile app: recording/import, history, meetings, memos, templates, teams, Talk, admin, data portability, IAP + ads |
|
|
| API server | `apps/api-server` | ASP.NET Core 10 + EF Core + SQLite | Cloud (self-hosted/NAS) | LLM/STT proxy and admin back-office backend for the .NET identity side |
|
|
| Admin console | `apps/admin` | Next.js 16 + MUI | Cloud | Back office: users, subscriptions, models/STT providers, usage, audit log, releases, ads |
|
|
| Landing site | `site/` | Vite 6 + React 19 + Tailwind | Static | Marketing/download/legal pages, deployed to Cloudflare Pages + GitHub Pages |
|
|
| Edge gateway | `server/cloudflare-worker` | Cloudflare Worker (TS) | Edge | CORS + proxy to the NAS-hosted API origin |
|
|
| Backend data/functions | `server/supabase` | Postgres + Deno Edge Functions | Cloud | Canonical product data, auth, RLS, storage, AI proxies, billing, delivery |
|
|
|
|
---
|
|
|
|
## 3. Information architecture (feature domains)
|
|
|
|
The product IA is stable across surfaces; each surface implements a subset.
|
|
|
|
```
|
|
D3RO Voice
|
|
├── Capture & Transcribe
|
|
│ ├── Dictation (hold/release, push-to-talk)
|
|
│ ├── Hands-free (toggle)
|
|
│ ├── File transcription (audio/video)
|
|
│ ├── Live captions (system audio)
|
|
│ └── Multiple STT engines (local Whisper, cloud providers)
|
|
├── AI Processing
|
|
│ ├── Local LLM (Ollama) / Cloud LLM (Claude, OpenAI)
|
|
│ ├── Auto Polish / cleanup / translate / summarize
|
|
│ ├── Custom instructions (user commands)
|
|
│ ├── Voice commands (keyword → command)
|
|
│ └── LLM Chains (multi-step pipelines)
|
|
├── Memory & Knowledge
|
|
│ ├── History (search, favorites, export)
|
|
│ ├── Dictionary (custom vocabulary)
|
|
│ ├── Memos (tags over history)
|
|
│ ├── Knowledge base (local RAG / cloud RAG)
|
|
│ └── Voice actions (OS automation)
|
|
├── Meetings
|
|
│ ├── Meeting recording + live transcript
|
|
│ ├── Timestamped memos
|
|
│ ├── AI summary + speaker diarization
|
|
│ ├── Document generation (minutes/report/idea-note/mindmap)
|
|
│ └── Export (PDF/DOCX/TXT/Markdown)
|
|
├── Conversation
|
|
│ ├── Local duplex conversation (STT→LLM→TTS)
|
|
│ └── Realtime conversation (OpenAI gpt-realtime, Premium)
|
|
├── Accounts & Sync
|
|
│ ├── Supabase auth (email + Google/GitHub/Apple OAuth)
|
|
│ ├── Cloud sync (per-user DB/rows)
|
|
│ ├── Devices (registration, revocation)
|
|
│ ├── Teams (members, invites, roles)
|
|
│ └── Data portability (export/import, account delete)
|
|
├── Monetization
|
|
│ ├── Tiers: Free / Pro / Pro+ / Team / Enterprise
|
|
│ ├── Desktop licenses (Ed25519, offline)
|
|
│ ├── Web billing (Stripe + Payple)
|
|
│ ├── Mobile IAP (Google Play / App Store)
|
|
│ └── Free-tier ads (AdMob rewarded + banner, mediation roster)
|
|
├── Platform Shell
|
|
│ ├── Settings / preferences / themes (6 themes)
|
|
│ ├── Onboarding
|
|
│ ├── Notifications / push
|
|
│ ├── Support & content reporting
|
|
│ └── Admin & audit
|
|
```
|
|
|
|
Legacy/other: voice keyword shortcuts, screen/context capture, auto-launch, system tray.
|
|
|
|
---
|
|
|
|
## 4. Identity & data model (multi-source, converging)
|
|
|
|
There are **three** identity/data systems in the repo. This is a known architectural tension (see `11-gap-backlog.md` G-01).
|
|
|
|
| System | Where | Stores | Status |
|
|
|---|---|---|---|
|
|
| Supabase Auth + Postgres | `server/supabase` | Canonical product users, profiles, subscriptions, history, meetings, teams, knowledge, push, billing, ads | **Canonical SSOT** for web + mobile |
|
|
| .NET API server | `apps/api-server` | Its own SQLite `Users` (JWT, roles), model/STT endpoints, usage/error logs, admin audit | Back-office + AI proxy; legacy SHA-256 users force-disabled at startup |
|
|
| Desktop local license | `apps/desktop` | Ed25519-signed offline license key, local SQLite DB per user (`_local` for anonymous) | Local-first tier gating + optional Supabase cloud sync |
|
|
|
|
Data flow:
|
|
- Desktop: local SQLite (per-user file) ↔ optional Supabase sync (history/dictionary/meetings).
|
|
- Web/Mobile: Supabase directly (tables + RLS) and via Edge Functions.
|
|
- Admin: Next.js server routes → Supabase service role and/or .NET `/api/admin/*`.
|
|
|
|
---
|
|
|
|
## 5. AI pipeline
|
|
|
|
**Local path (desktop):** mic → SoX/native capture (PCM16 16kHz mono) → faster-whisper Python sidecar → optional Ollama LLM → SQLite history → clipboard/text insertion into the active app.
|
|
|
|
**Cloud path (web/mobile/desktop online):**
|
|
- STT → Supabase Edge Function `stt-proxy` (quota reservation, provider fallback) or .NET `SttProxyService` (internal gateway token only).
|
|
- LLM → Supabase `llm-proxy` / `realtime-token` (Claude/OpenAI) or .NET `LlmProxyService`.
|
|
- Meetings/documents → `generate-meeting-document`, `embed-chunks`, `search-knowledge`.
|
|
|
|
**Realtime voice (Premium):** OpenAI `gpt-realtime-2.1` via ephemeral token from `realtime-token`, WebRTC in the desktop renderer with local-pipeline fallback.
|
|
|
|
STT providers supported by the desktop dispatcher (`apps/desktop/src/main/services/stt`): local Whisper, OpenAI, Groq, Deepgram, AssemblyAI, Google, Custom (OpenAI-compatible), and D3RO Cloud.
|
|
|
|
---
|
|
|
|
## 6. Monetization tiers
|
|
|
|
| Tier | Notes |
|
|
|---|---|
|
|
| Free | Quotas on STT/LLM; free-tier ads (desktop/mobile). |
|
|
| Pro / Pro+ | Paid subscriptions. Desktop: offline Ed25519 license. Web: Stripe/Payple. Mobile: Google Play Billing. |
|
|
| Team / Enterprise | Teams, shared meetings, admin roles. |
|
|
|
|
Desktop license verification is Ed25519 (public key in `release/desktop-license-public.pem`); the private key was rotated out of the repo. Mobile release evidence uses a separate Ed25519 keypair.
|
|
|
|
---
|
|
|
|
## 7. Platform architecture diagram (text)
|
|
|
|
```
|
|
┌──────────────────────────────────────────┐
|
|
│ Supabase (SSOT) │
|
|
│ Postgres+RLS · Auth · Storage · Realtime │
|
|
│ ~27 Deno Edge Functions │
|
|
└───────────────┬──────────────────────────┘
|
|
┌──────────────┬───────┴────────┬───────────────┐
|
|
│ │ │ │
|
|
apps/web apps/mobile-rn apps/desktop apps/admin
|
|
(Next.js) (React Native) (Electron) (Next.js)
|
|
│ │ │ │
|
|
└──────────────┴────────────────┘ │
|
|
│ │
|
|
┌───────┴────────┐ ┌─────────┴─────────┐
|
|
│ Cloudflare │ │ apps/api-server │
|
|
│ Worker (edge) │──────────────▶ .NET 10 + SQLite │
|
|
└───────┬────────┘ │ + admin audit │
|
|
│ └───────────────────┘
|
|
Cloudflare Tunnel
|
|
│
|
|
┌───────┴────────┐
|
|
│ D3RO NAS │ docker-compose.nas.yml
|
|
│ API + Admin │ d3ro.chanpaca.net / admin.chanpaca.net
|
|
└────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## 8. Cross-cutting concerns
|
|
|
|
| Concern | Implementation |
|
|
|---|---|
|
|
| Design system | `packages/ui` (web/desktop, MUI + tokens), `packages/ui-native` (mobile). Theme SSOT `theme.ts` (`d3roPalette`/`d3roTypo`/`d3roShadow`/`d3roRadius`). 6 themes. |
|
|
| i18n | `packages/i18n`, 12 locales, `ko` master, `t()` + type-safe keys. |
|
|
| IPC | `packages/core/src/ipc-channels.ts` is the channel SSOT; desktop preload exposes `window.electronAPI` (33 namespaces). |
|
|
| Errors | `D3ROError` + `ErrorCode`, `IPCResult<T>` envelope. |
|
|
| Crypto | Ed25519 license signing (`packages/core/src/utils/crypto-license`), HMAC admin sessions, PKCE on mobile. |
|
|
| Observability | `LoggerService` (electron-log) on desktop; `ServerErrorLog`/`ApiUsageLog`/`SttUsageLog` in .NET; admin audit log in Supabase. |
|
|
| Security posture | Fail-closed defaults: STT/LLM never return synthetic success; admin panels show explicit "unavailable" rather than sample data; secret scanning in CI. |
|
|
|
|
---
|
|
|
|
## 9. Related deep documents
|
|
|
|
- Desktop architecture: `docs/design/00-master-architecture.md` … `09-history-popup.md`
|
|
- Build-out history: `docs/phases/phase-1.md` … `phase-15.5-speaker-diarization.md`
|
|
- Multi-platform plan: `docs/v2/00-v2-master-plan.md`
|
|
- Mobile SSOT (authoritative checklist): `docs/v3/MOBILE_APP_COMPLETION_SSOT.md`
|
|
- Monetization: `docs/monetization-plan.md`
|
|
- Release process: `docs/deployment/release-guide.md`, `docs/deployment/nas-deployment-guide.md`
|