대시보드 폴드아웃/드릴다운 정리 + 페르소나 역린·misconduct 반응 + 게이트웨이 격리·RAG 비차단 수정
SSOT 대시보드:
- 한신대 기술분석 PDF(19쪽) 정합성 분석 + 이번 세션 발견 섹션 추가
- 섹션 폴드아웃(접기)·상단 목차(드릴다운)·모두 펼치기/접기 — 내용 보존, 레이아웃만 정리
페르소나 반응 강화('저항·반응 조절' 핵심 차별):
- PersonaCard.triggers(역린) 필드 + CCD 핵심상처 파생 역린 블록
- L0에 무례·모욕·조롱 시 현실적 동맹 균열 반응 지침
버그·성능 수정(라이브/E2E로 포착):
- 게이트웨이 페르소나 격리: --append-system-prompt를 --system-prompt(교체)로 + --exclude-dynamic-system-prompt-sections (내담자 캐릭터 붕괴·개발맥락 누출 차단)
- RAG: 임베더 동기 로드(약 7-13초)를 _warm_rag_caches 백그라운드 warm으로(세션 생성 블로킹 회귀 수정)
- voice TTS RMS 데드힌트 제거, init_state OpennessParams 파라미터객체화
- 한국어 PII(날짜·금액·주소) 마스킹 보강
- 레이아웃 시각 게이트: 폼 컨트롤 값 스크롤 오탐 제외(7/7)
검증: 백엔드 84/84, E2E 42(데스크톱 27·모바일 11·아바타 4), 시각 게이트 7/7
This commit is contained in:
Yun Chan
parent
cb2aebd76c
commit
085460b5e0
327 changed files with 31226 additions and 1829 deletions
187
docs/phase3/pilot-runbook.md
Normal file
187
docs/phase3/pilot-runbook.md
Normal file
|
|
@ -0,0 +1,187 @@
|
|||
# Phase 3 Education Pilot Runbook
|
||||
|
||||
## Purpose
|
||||
|
||||
Run a 20-person education pilot with auditable evidence that each included participant
|
||||
was consented, completed the required learning sessions, produced pre/post measures,
|
||||
and was included or excluded from KPI and export outputs correctly.
|
||||
|
||||
This runbook assumes Phase 2 role separation, session persistence, supervisor review,
|
||||
and safety event logging are already deployed. It does not authorize real participant
|
||||
collection until the privacy and consent checklist is signed.
|
||||
|
||||
## Entry Gates
|
||||
|
||||
- `G-P3-01` Product scope is locked as education/training, not diagnosis or treatment.
|
||||
- `G-P3-02` Current consent form, privacy notice, retention rule, withdrawal path, and
|
||||
third-party/inference transfer disclosure are approved by the project owner and legal
|
||||
reviewer.
|
||||
- `G-P3-03` The pilot uses pseudonymous `participant_id` values only. Names, emails,
|
||||
phone numbers, student numbers, and direct identifiers are not stored in evidence
|
||||
files.
|
||||
- `G-P3-04` Safety policy is active: self-harm scenarios remain excluded unless a later
|
||||
approved protocol explicitly adds them.
|
||||
- `G-P3-05` Instructor/admin access has RBAC and audit logging enabled before any pilot
|
||||
session starts.
|
||||
- `G-P3-06` Evidence root has been created outside the source tree and the checker runs:
|
||||
|
||||
```powershell
|
||||
$root="D:\vignette-evidence\phase3-pilot-YYYYMMDD"
|
||||
python scripts\check-phase3-artifacts.py --check --evidence-root $root
|
||||
```
|
||||
|
||||
The first run is expected to fail on missing files. The run output becomes the setup
|
||||
gap list.
|
||||
|
||||
## Participant Roster Gate
|
||||
|
||||
Target: 20 active, consented trainees.
|
||||
|
||||
Evidence file: `00-intake/pilot_roster.csv`
|
||||
|
||||
Required header:
|
||||
|
||||
```csv
|
||||
participant_id,cohort_id,consent_version,consent_signed_at,withdrawal_state,enrolled_at
|
||||
```
|
||||
|
||||
Allowed `withdrawal_state` values:
|
||||
|
||||
- `active`
|
||||
- `withdrawn_before_data_use`
|
||||
- `withdrawn_after_data_use`
|
||||
- `excluded_by_operator`
|
||||
|
||||
Rules:
|
||||
|
||||
- `participant_id` must be a pseudonymous value such as `P3-001`.
|
||||
- The roster must not include names, emails, phone numbers, student IDs, addresses, or
|
||||
date of birth.
|
||||
- A separate identity map may exist only in the approved restricted location owned by
|
||||
the data steward.
|
||||
- The pilot can start when 20 participants are `active` with a non-empty
|
||||
`consent_signed_at`.
|
||||
|
||||
Evidence file: `00-intake/consent_receipts.csv`
|
||||
|
||||
Required header:
|
||||
|
||||
```csv
|
||||
participant_id,consent_version,signed_at,signer_role,receipt_id
|
||||
```
|
||||
|
||||
Operator command:
|
||||
|
||||
```powershell
|
||||
python scripts\check-phase3-artifacts.py --check --evidence-root $root
|
||||
```
|
||||
|
||||
## Session Completion Gate
|
||||
|
||||
Minimum completion definition for dashboard Phase 3:
|
||||
|
||||
- Consent receipt exists before the first session.
|
||||
- Participant has at least two completed simulation sessions.
|
||||
- Each completed session has an end timestamp, non-zero learner turns, and supervisor
|
||||
review evidence.
|
||||
- Participant has both pre and post measures.
|
||||
- Participant has SUS responses or an explicit missing-response reason in the KPI
|
||||
report.
|
||||
|
||||
Evidence file: `01-sessions/session_completion.csv`
|
||||
|
||||
Required header:
|
||||
|
||||
```csv
|
||||
participant_id,session_id,persona_id,started_at,ended_at,completion_state,turns_count,supervisor_reviewed_at
|
||||
```
|
||||
|
||||
Allowed `completion_state` values:
|
||||
|
||||
- `completed`
|
||||
- `abandoned`
|
||||
- `operator_cancelled`
|
||||
- `excluded_from_analysis`
|
||||
|
||||
Operational cadence:
|
||||
|
||||
1. Before each session, confirm the participant remains `active`.
|
||||
2. After each session, export one row to `session_completion.csv`.
|
||||
3. Supervisor reviews the persisted session and records `supervisor_reviewed_at`.
|
||||
4. Any safety event or access override is linked in the KPI report, not copied into the
|
||||
roster.
|
||||
|
||||
Session evidence query placeholder:
|
||||
|
||||
```sql
|
||||
-- Run against the approved read-only reporting connection.
|
||||
-- Replace table/column names with the deployed schema names if they differ.
|
||||
SELECT
|
||||
s.learner_id AS participant_id,
|
||||
s.id AS session_id,
|
||||
s.persona_id,
|
||||
s.started_at,
|
||||
s.ended_at,
|
||||
CASE WHEN s.ended_at IS NOT NULL THEN 'completed' ELSE 'abandoned' END AS completion_state,
|
||||
COUNT(t.id) FILTER (WHERE t.speaker = 'counselor') AS turns_count,
|
||||
MAX(r.reviewed_at) AS supervisor_reviewed_at
|
||||
FROM app.sessions s
|
||||
LEFT JOIN app.turns t ON t.session_id = s.id
|
||||
LEFT JOIN app.session_reviews r ON r.session_id = s.id
|
||||
GROUP BY s.learner_id, s.id, s.persona_id, s.started_at, s.ended_at;
|
||||
```
|
||||
|
||||
If the deployed schema does not yet have `app.session_reviews`, the operator must
|
||||
attach a signed reviewer export and note the schema gap in `02-measures/kpi_report.json`.
|
||||
|
||||
## Measure Collection Gate
|
||||
|
||||
Evidence file: `02-measures/prepost_measures.csv`
|
||||
|
||||
Required header:
|
||||
|
||||
```csv
|
||||
participant_id,measure_name,timepoint,score,collected_at
|
||||
```
|
||||
|
||||
Allowed `timepoint` values:
|
||||
|
||||
- `pre`
|
||||
- `post`
|
||||
|
||||
Evidence file: `02-measures/sus_responses.csv`
|
||||
|
||||
Required header:
|
||||
|
||||
```csv
|
||||
participant_id,item,response,collected_at
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `measure_name` must be stable across pre and post rows.
|
||||
- SUS rows use item numbers and numeric responses only. Do not store free-text comments
|
||||
in `sus_responses.csv`.
|
||||
- Free-text participant feedback, if collected, must go through the privacy review path
|
||||
before being added to any export.
|
||||
|
||||
## Exit Gate
|
||||
|
||||
Phase 3 pilot completion evidence is sufficient when:
|
||||
|
||||
- `pilot_roster.csv` contains at least 20 active, consented participants.
|
||||
- At least 20 participants meet the minimum completion definition.
|
||||
- `kpi_report.json` is generated and reviewed.
|
||||
- `export_manifest.json` is generated, PII scan is pass or reviewed, and the export is
|
||||
approved.
|
||||
- `privacy_audit.md` is signed with no unresolved high severity item.
|
||||
- `withdrawal_log.csv` exists, even if it only contains the header.
|
||||
|
||||
Archive command:
|
||||
|
||||
```powershell
|
||||
python scripts\check-phase3-artifacts.py --check --evidence-root $root --output "$root\phase3_artifact_check.json"
|
||||
```
|
||||
|
||||
The checker writes only the report path given by `--output`; otherwise it is read-only.
|
||||
|
||||
Loading…
Add table
Add a link
Reference in a new issue