관리자 워크스페이스 접근 확장과 소유자 결정 7건 확정 반영

- 관리자(role=admin)가 학습자·교수자·관리자 워크스페이스를 모두 접근하도록
  can_access_role/require_role와 프론트 auth 헬퍼·Sidebar 내비를 정리하고
  admin 워크스페이스 내비 E2E를 추가.
- 소유자 결정 7건 전건 확정(2026-06-30)을 SSOT 대시보드·백로그에 반영하고
  결정 필요 7→0으로 동기화. SSOT drift 게이트 기대 카운트도 갱신.
- 확정된 H1 평가설계(κ≥0.70·ICC≥0.75·환각률≤0.03·t-검정 α=0.05·무작위 배정)를
  approved-export κ 게이트(checker·dataset_export·recursive export)와
  KPI 측정계획·export manifest 문서에 반영.

검증: npm run typecheck, npm run check:api-types, 백엔드 pytest 290 passed,
admin 내비 E2E 1 passed, 레이아웃 시각게이트 9/9, session-layout 4 passed,
SSOT drift 게이트 PASS.
This commit is contained in:
Yun Chan 2026-06-30 10:57:46 +09:00
parent 1274ba9ccc
commit 4e6b0045e3
17 changed files with 110 additions and 53 deletions

View file

@ -17,7 +17,7 @@ The export may be marked `approved_for_recursive_learning_seed` only when:
- Direct identifiers and raw identity mapping are absent from exported files.
- Free text is masked or transformed according to the manifest.
- PII scan is pass, or every finding has a reviewer disposition.
- IAA gates pass for gold exports: kappa >=0.60 and ICC >=0.75.
- IAA gates pass for gold exports: kappa >=0.70 and ICC >=0.75. (owner-confirmed 2026-06-30)
- Legal/data-steward review is complete.
If any gate is missing, set `export_status` to `technical_dry_run` or `blocked`, not

View file

@ -7,6 +7,14 @@ time. These metrics are pilot evidence, not claims of validated clinical effecti
Primary evidence file: `02-measures/kpi_report.json`
## Evaluation Design (owner-confirmed 2026-06-30)
- **표본·배정**: 20명을 실험군/통제군으로 **무작위 배정(randomized)**. 단회기 50분, 3척도(self-efficacy / skill proficiency / training satisfaction) pre-post.
- **통계검정**: 군 간 비교는 **독립표본 t-검정**, 유의수준 **α=0.05**. 정규성 위반 시 Mann-Whitney U로 대체 가능하나 기본은 t-검정.
- **신뢰도 목표**: 평가자 간 **κ≥0.70**(Cohen's kappa), 수치 평정 **ICC≥0.75**.
- **환각률 목표**: 상담 응답 환각률 **≤0.03**.
- 위 파라미터는 설계 확정값이며, 실제 효과성 판정·통계해석은 20명 파일럿 evidence 수집 후 수행한다(파일럿 게이트).
## Report Shape
```json
@ -60,10 +68,10 @@ Each metric object must include:
| Pilot completion | >=20 participants | Count active participants with consent, required sessions, pre/post, SUS, and supervisor review. | `pilot_roster.csv`, `session_completion.csv`, `prepost_measures.csv`, `sus_responses.csv` |
| Session completion | >=20 participants | Count participants with at least two `completed` sessions and non-empty `supervisor_reviewed_at`. | `session_completion.csv` |
| Embedding consistency | >=0.79 | Mean cosine similarity for persona invariant checks across masked session turns. Report model, embedding version, mean, p10, and count. | DB query export, `kpi_report.json` |
| Hallucination rate | <=0.05, target <=0.01 | Unsupported or contradicted factual claims divided by reviewed eligible evaluator/client-response claims. | supervisor review export, `kpi_report.json` |
| Hallucination rate | <=0.03 (owner 2026-06-30) | Unsupported or contradicted factual claims divided by reviewed eligible evaluator/client-response claims. | supervisor review export, `kpi_report.json` |
| Top-1 retrieval | >=0.80 | Top-1 evidence match rate for golden feedback or RAG evaluation queries. | golden evaluation run, `kpi_report.json` |
| SUS | >=80 | Standard 10-item SUS score transformed to a 0-100 score; report mean, median, and response count. | `sus_responses.csv` |
| Cohen kappa | >=0.60 | Agreement between human reviewers or human-vs-AI labels for categorical tags used in dataset export. | annotation export, `kpi_report.json` |
| Cohen kappa | >=0.70 (owner 2026-06-30) | Agreement between human reviewers or human-vs-AI labels for categorical tags used in dataset export. | annotation export, `kpi_report.json` |
| ICC | >=0.75 | Intraclass correlation for numeric ratings such as feedback score dimensions. | annotation export, `kpi_report.json` |
| Self-efficacy pre/post | Report and review | Paired pre/post change for the approved self-efficacy measure. Report effect direction, test used, p-value, and missingness. | `prepost_measures.csv` |