diff --git a/AGENTS.md b/AGENTS.md index 6625e59..449ed52 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,17 +7,25 @@ | 목적 | 문서 | |---|---| +| **문서 인덱스(먼저 여기서 시작)** | [`docs/README.md`](./docs/README.md) | +| **할 일(안된 것들 통합 TODO)** | [`docs/TODO.md`](./docs/TODO.md) | | 저장소 개요·빠른 시작 | [`README.md`](./README.md) | | **로컬 서버 띄우기·테스트** | [`docs/guides/local-development.md`](./docs/guides/local-development.md) | | 시스템 아키텍처·데이터 흐름 | [`docs/guides/architecture.md`](./docs/guides/architecture.md) | | 테스트·검증 실행 | [`docs/guides/testing.md`](./docs/guides/testing.md) | | 원천문서·갭 로드맵 | [`docs/guides/source-docs-and-gaps.md`](./docs/guides/source-docs-and-gaps.md) | | **SSOT 상태판** | [`docs/dev_dashboard.html`](./docs/dev_dashboard.html) | -| 백로그 | [`docs/ops/backlog-2026-06-26.md`](./docs/ops/backlog-2026-06-26.md) | +| 백로그(열린 항목만·얇은 현행본) | [`docs/ops/backlog-2026-06-26.md`](./docs/ops/backlog-2026-06-26.md) | | **서연 아바타 핸드오프** | [`docs/ops/handoff-avatar-seoyeon-2026-06-27.md`](./docs/ops/handoff-avatar-seoyeon-2026-06-27.md) | +| 냉동 보관소(읽지 않는다) | [`docs/archive/`](./docs/archive/README.md) | 작업 결과로 동작/구조가 바뀌면 해당 가이드와 SSOT 대시보드를 함께 갱신한다. +> **`docs/archive/`는 평상시 작업에서 읽지 않는다.** 완료·검증된 작업의 변경 로그, 실행이 끝난 구현/리팩터 +> 계획, 일회성 smoke·PoC·리서치 스냅샷, 완료된 재설계 프롬프트만 냉동 보관한 이력이다. 현재 상태의 근거로 +> 삼지 마라. 열린 작업은 얇은 현행 백로그와 SSOT 대시보드가 소유한다. 완료된 작업 기록이 다시 쌓이면 +> `docs/archive/`로 옮겨 활성 문서를 얇게 유지한다. + --- ## ⚠️ 0. 무조건 OS를 먼저 파악하고 시작한다 (최우선·필수) diff --git a/README.md b/README.md index a3e61a1..957cf90 100644 --- a/README.md +++ b/README.md @@ -121,8 +121,11 @@ Docker Desktop이 필요하다. RAG 모델 의존성까지 API 이미지에 넣 | 문서 | 용도 | |---|---| +| **`docs/README.md`** | **문서 인덱스(먼저 여기서 시작)** — docs 전체 통합 목차 | +| **`docs/TODO.md`** | **할 일(안된 것들) 통합** — 흩어진 열린 작업을 주제별로 모은 인덱스 | | **`docs/dev_dashboard.html`** | **SSOT(단일 진실 공급원)** — 상태·검증 증거·결정 필요·로드맵의 권위 기준 | -| `docs/ops/backlog-2026-06-26.md` | 운영 백로그(B1 코스메틱 · B2 환경제약 · B3 소유자결정 · B4 외부거버넌스). 대시보드와 일치 | +| `docs/ops/backlog-2026-06-26.md` | 운영 백로그 — **열린 항목만** 얇게 유지(B1~B4·Phase3 게이트). 대시보드와 일치 | +| `docs/archive/` | 냉동 보관소 — 완료 기록·실행된 계획·일회성 스냅샷. **평상시 읽지 않음**(이력용) | | `docs/guides/source-docs-and-gaps.md` | 원천문서·갭 로드맵 요약(대시보드 상태와 동기화) | | `docs/ops/source-docs-gap-analysis-2026-06-26.md` | 원천문서 갭 분석 상세 근거 | | `docs/guides/local-development.md` | 로컬 개발 환경 구축·실행 상세 가이드 | diff --git a/apps/api/app/config.py b/apps/api/app/config.py index 5fc65c7..554bb0b 100644 --- a/apps/api/app/config.py +++ b/apps/api/app/config.py @@ -108,6 +108,10 @@ class Settings(BaseSettings): default=0.0, validation_alias="SESSION_EVALUATION_TIMEOUT", ) + session_evaluation_recovery_limit: int = Field( + default=5, + validation_alias="SESSION_EVALUATION_RECOVERY_LIMIT", + ) session_digest_worker_enabled: bool = Field( default=False, validation_alias="SESSION_DIGEST_WORKER_ENABLED", diff --git a/apps/api/app/deps.py b/apps/api/app/deps.py index d2cd7a2..77b4a94 100644 --- a/apps/api/app/deps.py +++ b/apps/api/app/deps.py @@ -152,7 +152,7 @@ def require_admin_access(): async def _checker( principal: Annotated[Principal, Depends(get_current_principal)], ) -> Principal: - if not principal.admin_access: + if not principal.can_access_role(Role.ADMIN): raise HTTPException( status_code=status.HTTP_403_FORBIDDEN, detail="admin access required", diff --git a/apps/api/app/main.py b/apps/api/app/main.py index d77fc2f..97c5fff 100644 --- a/apps/api/app/main.py +++ b/apps/api/app/main.py @@ -58,9 +58,11 @@ async def lifespan(app: FastAPI): ) await engine_client.startup() await voice_service.startup() + session_routes.schedule_missing_session_evaluation_recovery() try: yield finally: + session_routes.cancel_missing_session_evaluation_recovery() await voice_service.shutdown() await engine_client.shutdown() try: diff --git a/apps/api/app/routes/eval.py b/apps/api/app/routes/eval.py index 532fcf4..40cf2d8 100644 --- a/apps/api/app/routes/eval.py +++ b/apps/api/app/routes/eval.py @@ -27,6 +27,7 @@ from .. import session_persistence from ..deps import Principal, Role, require_role from ..engine_client import EngineError, engine_client from ..runtime_policy import runtime_fallback_allowed +from ..session_evaluation_input import enriched_masked_turns from ..session_read_model import StageLabel, stage_label_or_none from ..services import evaluator from ..services.evaluator import SessionEvaluation, TurnEvaluation @@ -122,13 +123,7 @@ async def reevaluate_session( 엔진 장애는 503 으로 변환(평가는 비치명적이지만 트리거는 사용자 명시 요청이라 에러 노출). """ sess = await _load_session_or_404(session_id, principal) - masked = sess.masked_turns() - # 발화 seq 보강(deep 프롬프트 가독성 — store 가 seq 미포함이라 인덱스로 부여) - enriched: list[dict[str, Any]] = [] - for i, t in enumerate(masked): - item = dict(t) - item.setdefault("seq", i) - enriched.append(item) + enriched = enriched_masked_turns(sess.masked_turns()) # 누적 기법 코드 — DB 미가용이라 fast 결과가 없으면 빈 분포(deep LLM 정성 평가는 그대로 유효). technique_codes: list[str] = [] @@ -233,8 +228,25 @@ async def reevaluate_turn( engine=engine_client, audit_hook=session_persistence.record_llm_call_audit, ) - if result.error and result.error.startswith("engine_error"): - raise HTTPException(status.HTTP_503_SERVICE_UNAVAILABLE, detail=result.error) + result_payload = result.to_hook_dict() + turn_id = getattr(learner, "turn_id", None) + if not turn_id: + raise HTTPException( + status.HTTP_503_SERVICE_UNAVAILABLE, + detail="turn evaluation retry result was generated but the target turn has no durable id", + ) + saved = await session_persistence.replace_turn_evaluation( + turn_id=turn_id, + evaluation=result_payload, + ) + if not saved: + raise HTTPException( + status.HTTP_503_SERVICE_UNAVAILABLE, + detail="turn evaluation retry result was generated but could not be saved", + ) + learner.evaluation = result_payload + if result.error: + raise HTTPException(_session_evaluation_error_status(result.error), detail=result.error) return result diff --git a/apps/api/app/routes/sessions.py b/apps/api/app/routes/sessions.py index f20920f..2273229 100644 --- a/apps/api/app/routes/sessions.py +++ b/apps/api/app/routes/sessions.py @@ -26,6 +26,7 @@ from ..deps import CurrentPrincipal, Principal, Role from ..engine_client import EngineError, engine_client from ..persona_repository import get_catalog_persona from ..runtime_policy import require_runtime_fallback_allowed +from ..session_evaluation_input import enriched_masked_turns from ..services import ( evaluator, guardrail, @@ -42,6 +43,7 @@ from ..session_read_model import ( LearnerSessionsResponse, LearnerSessionSummary, LEARNER_VISIBLE_AI_ROLE, + MISSING_SESSION_EVALUATION_GRACE_SECONDS, ReviewCaseWorksheet, ReviewCaseWorksheetSaveRequest, ReviewWorksheetItem, @@ -70,6 +72,8 @@ from ..store import InProcSession, TurnRecord, store router = APIRouter(prefix="/sessions", tags=["sessions"]) logger = logging.getLogger(__name__) +_SESSION_EVALUATION_IN_FLIGHT: set[str] = set() +_SESSION_EVALUATION_RECOVERY_TASK: asyncio.Task[int] | None = None TheoryMode = Literal["humanistic", "cbt", "integrative"] EndStateValue = str | int | float | bool | None | dict[str, float] @@ -102,7 +106,14 @@ class LiveCoachRequest(BaseModel): class LiveCoachHistoryResponse(BaseModel): source: Literal["database", "runtime"] = "runtime" + quota: live_coach.LiveCoachQuota = Field( + default_factory=lambda: live_coach.LiveCoachQuota( + remaining=session_persistence.LIVE_COACH_INITIAL_CREDITS, + max=session_persistence.LIVE_COACH_MAX_CREDITS, + ) + ) events: list[live_coach.LiveCoachEvent] = Field(default_factory=list) + credit_events: list[live_coach.LiveCoachCreditEvent] = Field(default_factory=list) class CrisisResourceResponse(BaseModel): @@ -220,12 +231,20 @@ async def _retrieve_live_coach_grounding( body = chunk.body or chunk.behavior_cue or chunk.context_prefix or "" if not body: continue + meta = chunk.meta if isinstance(chunk.meta, dict) else {} + title = str(meta.get("source_title") or meta.get("title") or chunk.source_id or "Vignette KB").strip() + source_type = str(meta.get("source_type") or "").strip() + source_version = str(meta.get("source_version") or meta.get("version") or "").strip() + citation = str(meta.get("citation") or "").strip() out.append( live_coach.LiveCoachGrounding( source_id=chunk.source_id or f"kb:{chunk.chunk_id}", - title=chunk.source_id or "Vignette KB", + title=title or "Vignette KB", locator=chunk.heading_path, kb_kind=chunk.kb_kind, + source_type=source_type or None, + version=source_version or None, + citation=citation or None, summary=body[:500], ) ) @@ -643,8 +662,13 @@ async def _evaluate_stream_turn(ctx: orchestrator.TurnContext, final_reply: str) audit_hook=session_persistence.record_llm_call_audit, ) return await hook(ctx, final_reply) - except Exception: - return None + except Exception as exc: + logger.warning( + "turn fast-loop evaluation failed: session_id=%s", + ctx.session_id, + exc_info=True, + ) + return orchestrator.turn_evaluation_error_payload(ctx, exc) def _stream_result_from_done( @@ -678,11 +702,7 @@ async def _generate_and_save_session_evaluation(sess: InProcSession) -> None: return timeout_seconds = _session_evaluation_timeout_seconds() - enriched: list[dict[str, object]] = [] - for index, turn in enumerate(sess.masked_turns(), start=1): - item: dict[str, object] = dict(turn) - item["seq"] = index - enriched.append(item) + enriched = enriched_masked_turns(sess.masked_turns()) try: result = await asyncio.wait_for( @@ -753,6 +773,7 @@ async def _generate_and_save_session_evaluation(sess: InProcSession) -> None: def _observe_session_evaluation_task(task: asyncio.Task[None], session_id: str) -> None: + _SESSION_EVALUATION_IN_FLIGHT.discard(session_id) try: task.result() except asyncio.CancelledError: @@ -764,6 +785,13 @@ def _observe_session_evaluation_task(task: asyncio.Task[None], session_id: str) def _schedule_session_evaluation(sess: InProcSession) -> asyncio.Task[None] | None: if not sess.turns: return None + if sess.session_id in _SESSION_EVALUATION_IN_FLIGHT: + logger.info( + "session evaluation already scheduled: session_id=%s", + sess.session_id, + ) + return None + _SESSION_EVALUATION_IN_FLIGHT.add(sess.session_id) task = asyncio.create_task( _generate_and_save_session_evaluation(sess), name=f"session-evaluation:{sess.session_id}", @@ -774,6 +802,66 @@ def _schedule_session_evaluation(sess: InProcSession) -> asyncio.Task[None] | No return task +async def recover_missing_session_evaluations(*, limit: int | None = None) -> int: + recovery_limit = settings.session_evaluation_recovery_limit if limit is None else limit + if recovery_limit <= 0: + return 0 + stale_after_seconds = ( + _session_evaluation_timeout_seconds() + + MISSING_SESSION_EVALUATION_GRACE_SECONDS + ) + candidates, durable = await session_persistence.list_sessions_missing_session_evaluation( + older_than_seconds=stale_after_seconds, + limit=recovery_limit, + ) + if not durable: + logger.warning("session evaluation recovery skipped: durable store unavailable") + return 0 + scheduled = 0 + for sess in candidates: + if _schedule_session_evaluation(sess) is not None: + scheduled += 1 + if scheduled: + logger.info("session evaluation recovery scheduled %d session(s)", scheduled) + return scheduled + + +def _observe_session_evaluation_recovery_task(task: asyncio.Task[int]) -> None: + global _SESSION_EVALUATION_RECOVERY_TASK + if _SESSION_EVALUATION_RECOVERY_TASK is task: + _SESSION_EVALUATION_RECOVERY_TASK = None + try: + task.result() + except asyncio.CancelledError: + logger.warning("session evaluation recovery task cancelled") + except Exception: + logger.exception("session evaluation recovery task crashed") + + +def schedule_missing_session_evaluation_recovery() -> asyncio.Task[int] | None: + global _SESSION_EVALUATION_RECOVERY_TASK + if settings.session_evaluation_recovery_limit <= 0: + return None + if ( + _SESSION_EVALUATION_RECOVERY_TASK is not None + and not _SESSION_EVALUATION_RECOVERY_TASK.done() + ): + return _SESSION_EVALUATION_RECOVERY_TASK + task = asyncio.create_task( + recover_missing_session_evaluations(), + name="session-evaluation-recovery", + ) + _SESSION_EVALUATION_RECOVERY_TASK = task + task.add_done_callback(_observe_session_evaluation_recovery_task) + return task + + +def cancel_missing_session_evaluation_recovery() -> None: + task = _SESSION_EVALUATION_RECOVERY_TASK + if task is not None and not task.done(): + task.cancel() + + def _session_evaluation_timeout_seconds() -> float: configured = float(settings.session_evaluation_timeout or settings.engine_timeout) return max(configured, 1.0) @@ -792,12 +880,12 @@ async def _load_learner_sessions( include_turn_evaluation: bool = False, ) -> tuple[list[InProcSession], bool]: if include_turn_evaluation: - sessions, durable = await session_persistence.list_sessions( + sessions, durable = await session_persistence.list_recent_sessions( principal, include_turn_evaluation=True, ) else: - sessions, durable = await session_persistence.list_sessions(principal) + sessions, durable = await session_persistence.list_recent_sessions(principal) if not durable: require_runtime_fallback_allowed("session list") sessions = [ @@ -1254,9 +1342,16 @@ async def list_live_coach_history( principal = _ensure_learner(principal) await _load_session_or_404(session_id, principal) events, durable = await session_persistence.list_live_coach_events(session_id, principal) + quota, quota_durable = await session_persistence.get_live_coach_quota(session_id, principal) + credit_events, credit_durable = await session_persistence.list_live_coach_credit_events( + session_id, + principal, + ) return LiveCoachHistoryResponse( - source="database" if durable else "runtime", + source="database" if durable and quota_durable and credit_durable else "runtime", + quota=live_coach.LiveCoachQuota(**quota), events=[live_coach.LiveCoachEvent(**event) for event in events], + credit_events=[live_coach.LiveCoachCreditEvent(**event) for event in credit_events], ) @@ -1269,6 +1364,12 @@ async def live_coach_turn( """방금 완료된 턴에 대한 비차단 라이브 코칭을 반환한다.""" principal = _ensure_learner(principal) sess = await _load_session_or_404(session_id, principal) + quota, _ = await session_persistence.get_live_coach_quota(session_id, principal) + if int(quota.get("remaining", 0)) <= 0: + raise HTTPException( + status_code=status.HTTP_409_CONFLICT, + detail="live coach credit exhausted", + ) turn_seq = body.turn_seq or max(1, int(getattr(sess.state, "turn_seq", 1) or 1)) stage = _stage_label(sess.state.stage) grounding = await _retrieve_live_coach_grounding( @@ -1296,16 +1397,35 @@ async def live_coach_turn( grounding=grounding, audit_hook=session_persistence.record_llm_call_audit, ) - await session_persistence.save_live_coach_event( - session_id=sess.session_id, - learner_id=sess.learner_id, - turn_seq=turn_seq, - stage=stage, - learner_text=body.learner_text, - client_reply=body.client_reply, - suggestion=suggestion, + try: + _, coach_event_durable = await session_persistence.save_live_coach_event( + session_id=sess.session_id, + learner_id=sess.learner_id, + turn_seq=turn_seq, + stage=stage, + learner_text=body.learner_text, + client_reply=body.client_reply, + suggestion=suggestion, + ) + except session_persistence.LiveCoachCreditExhausted as exc: + raise HTTPException( + status_code=status.HTTP_409_CONFLICT, + detail="live coach credit exhausted", + ) from exc + quota_after, quota_durable = await session_persistence.get_live_coach_quota(session_id, principal) + credit_events, credit_durable = await session_persistence.list_live_coach_credit_events(session_id, principal) + turn_credit_events = [ + live_coach.LiveCoachCreditEvent(**event) + for event in credit_events + if int(event.get("turn_seq") or 0) == int(turn_seq) + ] + return suggestion.model_copy( + update={ + "persistence_source": "database" if coach_event_durable and quota_durable and credit_durable else "runtime", + "quota": live_coach.LiveCoachQuota(**quota_after), + "credit_events": turn_credit_events[-2:], + } ) - return suggestion @router.post("/{session_id}/stream") @@ -1398,6 +1518,7 @@ async def end_session( """End a learner-owned session and prepare carry-over state.""" principal = _ensure_learner(principal) sess = await _load_session_or_404(session_id, principal, allow_ended=True) + was_ended = bool(sess.ended) recall = _RECALL_CACHE.get(session_id) or memory.RecallContext() carry = memory.make_carry_over( @@ -1413,7 +1534,8 @@ async def end_session( await _end_persisted_session(sess, carry) _RECALL_CACHE.pop(session_id, None) _KB_CUES_CACHE.pop(session_id, None) - _schedule_session_evaluation(sess) + if not was_ended: + _schedule_session_evaluation(sess) return SessionEndResponse( session_id=session_id, diff --git a/apps/api/app/routes/teacher.py b/apps/api/app/routes/teacher.py index 5e5863c..50ccfb9 100644 --- a/apps/api/app/routes/teacher.py +++ b/apps/api/app/routes/teacher.py @@ -2,6 +2,7 @@ from __future__ import annotations +from datetime import datetime from typing import Annotated, Literal from fastapi import APIRouter, Depends, HTTPException, status @@ -10,7 +11,12 @@ from pydantic import BaseModel, Field from .. import session_persistence from ..deps import Principal, Role, require_role from ..runtime_policy import require_runtime_fallback_allowed -from ..session_read_model import StageLabel, stage_label +from ..session_read_model import ( + StageLabel, + learner_visible_turns, + missing_session_evaluation_record, + stage_label, +) from ..services import session_metrics from ..stage_contract import STAGE_LABEL_VALUES from ..store import InProcSession, store @@ -37,6 +43,10 @@ class TeacherSessionSummary(BaseModel): review_status: Literal["pending", "viewed", "closed"] = "pending" review_note: str | None = None reviewed_at: str | None = None + evaluation_status: Literal["pending", "ready", "error"] = "pending" + review_ready: bool = False + supervisor_state: Literal["기록 대기", "평가 대기", "평가 완료", "평가 실패"] = "기록 대기" + evaluation_error: str | None = None class TeacherSessionReviewStatusRequest(BaseModel): @@ -137,9 +147,14 @@ class TeacherDashboardResponse(BaseModel): message: str -def _learner_label(learner_id: str) -> str: +def _learner_label_from_id(learner_id: str) -> str: suffix = learner_id[-6:] if len(learner_id) > 6 else learner_id - return f"학습자 {suffix}" + return f"학습자 {suffix}" if suffix else "학습자" + + +def _session_learner_label(sess: InProcSession) -> str: + label = str(getattr(sess, "learner_label", "") or "").strip() + return label or _learner_label_from_id(sess.learner_id) def _growth_point(point: session_metrics.SessionGrowthPoint) -> TeacherGrowthPoint: @@ -163,9 +178,15 @@ def _build_learner_growth( limit: int | None = 12, point_limit: int | None = 6, ) -> list[TeacherLearnerGrowth]: + learner_labels = { + sess.learner_id: label + for sess in sessions + if (label := str(getattr(sess, "learner_label", "") or "").strip()) + } metrics = session_metrics.build_learner_growth( sessions, - learner_label=_learner_label, + learner_label=lambda learner_id: learner_labels.get(learner_id) + or _learner_label_from_id(learner_id), limit=limit, point_limit=point_limit, ) @@ -225,22 +246,68 @@ def _worksheet_review_status_value( return "pending" +def _evaluation_status_value(record: dict[str, object] | None) -> Literal["pending", "ready", "error"]: + value = str((record or {}).get("status") or "pending") + if value in {"ready", "error"}: + return value # type: ignore[return-value] + return "pending" + + +def _supervisor_state( + sess: InProcSession, + evaluation_record: dict[str, object] | None, + *, + has_visible_turns: bool | None = None, +) -> Literal["기록 대기", "평가 대기", "평가 완료", "평가 실패"]: + status = _evaluation_status_value(evaluation_record) + if status == "ready": + return "평가 완료" + if status == "error": + return "평가 실패" + if has_visible_turns is None: + has_visible_turns = bool(learner_visible_turns(sess)) + if has_visible_turns: + return "평가 대기" + return "기록 대기" + + +def _dashboard_evaluation_record( + sess: InProcSession, + evaluation_record: dict[str, object] | None, +) -> dict[str, object] | None: + visible_turns = learner_visible_turns(sess) + hidden_turns = len(visible_turns) != len(sess.turns) + if evaluation_record is not None: + return None if hidden_turns else evaluation_record + if hidden_turns: + return None + return missing_session_evaluation_record( + sess, + has_visible_turns=bool(visible_turns), + now_ts=datetime.now().timestamp(), + ) + + def _summary( sess: InProcSession, review_status: dict[str, object] | None = None, + evaluation_record: dict[str, object] | None = None, ) -> TeacherSessionSummary: - learner_turns = sum(1 for turn in sess.turns if turn.speaker == "counselor") - client_turns = sum(1 for turn in sess.turns if turn.speaker == "client") + visible_turns = learner_visible_turns(sess) + summary_evaluation_record = _dashboard_evaluation_record(sess, evaluation_record) + learner_turns = sum(1 for turn in visible_turns if turn.speaker == "counselor") + client_turns = sum(1 for turn in visible_turns if turn.speaker == "client") + evaluation_status = _evaluation_status_value(summary_evaluation_record) return TeacherSessionSummary( session_id=sess.session_id, learner_id=sess.learner_id, - learner_label=_learner_label(sess.learner_id), + learner_label=_session_learner_label(sess), persona_code=sess.persona_code, persona_name=sess.persona.display_name, session_no=sess.session_no, status="ended" if sess.ended else "active", stage=stage_label(sess.state.stage), - turn_count=len(sess.turns), + turn_count=len(visible_turns), learner_turn_count=learner_turns, client_turn_count=client_turns, started_at=session_metrics.iso_datetime(sess.created_at) or "", @@ -248,34 +315,53 @@ def _summary( review_status=_review_status_value(review_status), review_note=str(review_status.get("note") or "") if review_status else None, reviewed_at=str(review_status.get("reviewed_at") or "") if review_status else None, + evaluation_status=evaluation_status, + review_ready=evaluation_status == "ready", + supervisor_state=_supervisor_state( + sess, + summary_evaluation_record, + has_visible_turns=bool(visible_turns), + ), + evaluation_error=( + str(summary_evaluation_record.get("error") or "") if summary_evaluation_record else None + ), ) +def _needs_teacher_review(item: TeacherSessionSummary) -> bool: + if item.status != "ended" or item.review_status == "closed": + return False + return item.turn_count > 0 or item.evaluation_status in {"ready", "error"} + + @router.get("/dashboard", response_model=TeacherDashboardResponse) async def teacher_dashboard(principal: TeacherPrincipal) -> TeacherDashboardResponse: """Return teacher-visible dashboard data from real sessions only.""" - sessions, durable = await session_persistence.list_sessions( + sessions, durable = await session_persistence.list_all_sessions( principal, include_turn_evaluation=True, ) if not durable: require_runtime_fallback_allowed("teacher dashboard") sessions = sorted(store.list(), key=lambda sess: sess.created_at, reverse=True) + ended_session_ids = [sess.session_id for sess in sessions if sess.ended] review_statuses, _ = await session_persistence.list_session_review_statuses( - [sess.session_id for sess in sessions if sess.ended], + ended_session_ids, principal, ) + evaluation_records, evaluations_durable = await session_persistence.list_session_evaluations( + ended_session_ids, + principal, + ) + if not evaluations_durable: + require_runtime_fallback_allowed("session evaluation list") summaries = [ - _summary(sess, review_statuses.get(sess.session_id)) + _summary(sess, review_statuses.get(sess.session_id), evaluation_records.get(sess.session_id)) for sess in sessions ] - pending_reviews = [ - item - for item in summaries - if item.status == "ended" and item.review_status != "closed" - ] + pending_reviews = [item for item in summaries if _needs_teacher_review(item)] learners = {sess.learner_id for sess in sessions} - learner_growth = _build_learner_growth(sessions) + learner_growth = _build_learner_growth(sessions, limit=None) safety_alerts: list[TeacherSafetyAlert] = [] if durable: raw_alerts, alerts_durable = await session_persistence.list_safety_alerts(principal) @@ -334,7 +420,7 @@ async def learner_analysis( principal: TeacherPrincipal, ) -> TeacherLearnerAnalysisResponse: """Return one learner's full teacher-visible session timeline and trend.""" - sessions, durable = await session_persistence.list_sessions( + sessions, durable = await session_persistence.list_all_sessions( principal, include_turn_evaluation=True, ) @@ -350,20 +436,23 @@ async def learner_analysis( learner_sessions, key=lambda sess: (sess.created_at, sess.session_no, sess.session_id), ) + ended_session_ids = [sess.session_id for sess in ordered if sess.ended] review_statuses, _ = await session_persistence.list_session_review_statuses( - [sess.session_id for sess in ordered if sess.ended], + ended_session_ids, principal, ) + evaluation_records, evaluations_durable = await session_persistence.list_session_evaluations( + ended_session_ids, + principal, + ) + if not evaluations_durable: + require_runtime_fallback_allowed("session evaluation list") summaries = [ - _summary(sess, review_statuses.get(sess.session_id)) + _summary(sess, review_statuses.get(sess.session_id), evaluation_records.get(sess.session_id)) for sess in ordered ] growth = _build_learner_growth(ordered, limit=None, point_limit=None)[0] - pending_reviews = sum( - 1 - for item in summaries - if item.status == "ended" and item.review_status != "closed" - ) + pending_reviews = sum(1 for item in summaries if _needs_teacher_review(item)) closed_reviews = sum(1 for item in summaries if item.review_status == "closed") return TeacherLearnerAnalysisResponse( @@ -404,6 +493,18 @@ async def update_session_review_status( status_code=status.HTTP_409_CONFLICT, detail="active sessions cannot be closed as reviewed", ) + if sess.ended and request.status == "closed" and learner_visible_turns(sess): + evaluation_record, evaluation_durable = await session_persistence.load_session_evaluation( + session_id, + principal, + ) + if not evaluation_durable: + require_runtime_fallback_allowed("session evaluation") + if _evaluation_status_value(evaluation_record) != "ready": + raise HTTPException( + status_code=status.HTTP_409_CONFLICT, + detail="session evaluation must be ready before closing teacher review", + ) saved, _ = await session_persistence.save_session_review_status( session_id=session_id, reviewer_id=principal.user_id, diff --git a/apps/api/app/routes/voice.py b/apps/api/app/routes/voice.py index b230cc0..a6f3e7f 100644 --- a/apps/api/app/routes/voice.py +++ b/apps/api/app/routes/voice.py @@ -18,7 +18,7 @@ import hashlib import time from typing import Optional -from fastapi import APIRouter, WebSocket, WebSocketDisconnect +from fastapi import APIRouter, WebSocket, WebSocketDisconnect, HTTPException from fastapi.responses import JSONResponse from starlette.websockets import WebSocketState @@ -97,6 +97,13 @@ _PROVIDER_EVENT_TAXONOMY = { "noise": ("background_noise", "audio_quality"), "background_noise": ("background_noise", "audio_quality"), } + + +def _is_turn_persistence_unavailable(exc: Exception) -> bool: + if not isinstance(exc, HTTPException) or exc.status_code != 503: + return False + detail = str(exc.detail or "") + return "turn append" in detail and "persistence unavailable" in detail _PROVIDER_EVENT_TYPE_FIELDS = ("event_type", "type", "kind", "label") @@ -303,7 +310,18 @@ async def voice_ws(websocket: WebSocket) -> None: except WebSocketDisconnect: pass except Exception as e: - await _safe_send_json(websocket, {"type": "error", "detail": f"voice ws error: {e}"}) + if _is_turn_persistence_unavailable(e): + await _safe_send_json( + websocket, + { + "type": "error", + "code": "turn_persistence_unavailable", + "detail": "voice turn persistence unavailable; retry the utterance", + }, + ) + await _safe_send_json(websocket, {"type": "state", "state": "idle"}) + else: + await _safe_send_json(websocket, {"type": "error", "detail": f"voice ws error: {e}"}) finally: await _safe_close(websocket) @@ -609,6 +627,17 @@ async def _principal_from_websocket(websocket: WebSocket) -> Principal | None: ) +async def _practice_access_error(principal: Principal) -> str | None: + if ( + principal.profile_completed_at is None + and not await user_onboarding_complete(principal.user_id) + ): + return "onboarding_required" + if principal.consent_at is None and not await user_has_consent(principal.user_id): + return "consent_required" + return None + + async def _bind_session( websocket: WebSocket, principal: Principal, @@ -616,6 +645,9 @@ async def _bind_session( """Bind an existing session or create a dev-only voice session.""" qp = websocket.query_params explicit_preset = qp.get("preset") + access_error = await _practice_access_error(principal) + if access_error is not None: + return None, None, access_error, {} session_id = qp.get("session_id") if session_id: @@ -636,13 +668,6 @@ async def _bind_session( persona_code = qp.get("persona_code") if not persona_code: return None, None, "session_id or persona_code query required", {} - if ( - principal.profile_completed_at is None - and not await user_onboarding_complete(principal.user_id) - ): - return None, None, "onboarding_required", {} - if principal.consent_at is None and not await user_has_consent(principal.user_id): - return None, None, "consent_required", {} try: catalog_persona = await get_catalog_persona(persona_code) except Exception: diff --git a/apps/api/app/services/dataset_export.py b/apps/api/app/services/dataset_export.py index a17c301..478f116 100644 --- a/apps/api/app/services/dataset_export.py +++ b/apps/api/app/services/dataset_export.py @@ -111,6 +111,23 @@ def normalize_json_value(value: Any) -> Any: return value +def export_safe_supervisor_comments(value: Any) -> list[dict[str, Any]]: + comments = normalize_json_value(value or []) + if not isinstance(comments, list): + return [] + safe_comments: list[dict[str, Any]] = [] + for item in comments: + if not isinstance(item, Mapping): + continue + safe_item: dict[str, Any] = {} + for key in ("kind", "intent_deviation"): + if key in item: + safe_item[key] = json_safe(normalize_json_value(item[key])) + if safe_item: + safe_comments.append(safe_item) + return safe_comments + + def _redacted_sample(kind: str, value: str) -> str: if kind == "email" and "@" in value: return f"" @@ -180,7 +197,7 @@ def build_dataset_record( "techniques": json_safe(normalize_json_value(row.get("techniques") or [])), "client_states": json_safe(normalize_json_value(row.get("client_states") or [])), "feedback_scores": json_safe(normalize_json_value(row.get("feedback_scores") or [])), - "supervisor_comments": json_safe(normalize_json_value(row.get("supervisor_comments") or [])), + "supervisor_comments": export_safe_supervisor_comments(row.get("supervisor_comments") or []), "source_refs": { "session_started_at": json_safe(row.get("session_started_at") or row.get("started_at")), "export_manifest_id": export_manifest_id, @@ -393,6 +410,13 @@ def validate_manifest_gate(manifest: Mapping[str, Any]) -> None: errors.append("kappa must be >= 0.70") if (agreement.get("icc") or 0) < 0.75: errors.append("ICC must be >= 0.75") + selection_criteria = manifest.get("selection_criteria") or {} + if not isinstance(selection_criteria, Mapping) or selection_criteria.get("include_withdrawn") is not False: + errors.append("include_withdrawn must be false") + consent_scope = manifest.get("consent_scope") or {} + allowed_uses = consent_scope.get("allowed_uses") if isinstance(consent_scope, Mapping) else None + if not isinstance(allowed_uses, list) or "recursive_learning_seed" not in allowed_uses: + errors.append("recursive_learning_seed consent scope is required") for key in ("data_steward", "legal_or_privacy_reviewer", "technical_operator", "approved_at"): if not str(approvals.get(key) or "").strip(): errors.append(f"approval missing: {key}") diff --git a/apps/api/app/services/evaluator.py b/apps/api/app/services/evaluator.py index a28d8e5..dc6cd7d 100644 --- a/apps/api/app/services/evaluator.py +++ b/apps/api/app/services/evaluator.py @@ -52,6 +52,7 @@ from ..taxonomy import ( CommentKind, Technique, TechniqueCategory, + speaker_ko_label, ) from . import guardrail @@ -482,7 +483,7 @@ def build_fast_messages(ctx: "TurnContext", client_reply: str) -> list[EngineMes theory = _theory_mode(ctx) client_reply_masked = guardrail.mask_pii(client_reply).text_masked recent = "\n".join( - f"{('상담자' if t.get('speaker') == 'counselor' else '내담자')}: {t.get('text', '')}" + f"{speaker_ko_label(t.get('speaker'))}: {t.get('text', '')}" for t in (ctx.memory.recent_turns or [])[-4:] ) or "(직전 맥락 없음)" @@ -537,7 +538,7 @@ def build_deep_messages( ) -> list[EngineMessage]: """deep-loop 평가 프롬프트(전체 회기 + 코드 집계 분포 + 골든라벨 후보).""" transcript = "\n".join( - f"{t.get('seq', '')}{('상담자' if t.get('speaker') == 'counselor' else '내담자')}: {t.get('text', '')}" + f"{t.get('seq', '')}{speaker_ko_label(t.get('speaker'))}: {t.get('text', '')}" for t in masked_turns ) or "(축어록 없음)" dist_lines = ", ".join(f"{k}:{v}" for k, v in distribution.by_category.items()) or "(없음)" @@ -736,11 +737,11 @@ async def evaluate_turn( inference_geo=resp.inference_geo, latency_ms=latency_ms, ) - except EngineError as e: - base.error = f"engine_error: {e}" + except EngineError: + base.error = "engine_error" return base - except Exception as e: # 방어 — 어떤 예외도 상담 루프를 막지 않게 - base.error = f"eval_error: {e}" + except Exception: # 방어 — 어떤 예외도 상담 루프를 막지 않게 + base.error = "eval_error" return base payload = structured_payload_from_response(resp) @@ -751,8 +752,8 @@ async def evaluate_turn( result = _parse_fast(payload, turn_seq=st.turn_seq, stage=st.stage.value, theory=theory) _evaluator_cache_put(cache_key, result.model_dump()) return result - except Exception as e: # 파싱 방어 - base.error = f"parse_error: {e}" + except Exception: # 파싱 방어 + base.error = "parse_error" return base diff --git a/apps/api/app/services/live_coach.py b/apps/api/app/services/live_coach.py index 4c225ca..7730968 100644 --- a/apps/api/app/services/live_coach.py +++ b/apps/api/app/services/live_coach.py @@ -25,6 +25,7 @@ from ..contracts.engine_gateway import structured_payload_from_response from ..engine_client import EngineClient, EngineError, EngineMessage, GenerateRequest from ..paths import repo_root, repo_path from ..session_read_model import StageLabel, stage_label_or_none +from ..taxonomy import speaker_ko_label from . import guardrail if TYPE_CHECKING: @@ -33,6 +34,7 @@ if TYPE_CHECKING: Tone = Literal["pos", "warn", "neutral"] CoachStatus = Literal["ready", "degraded"] +CoachCreditEventType = Literal["use", "recharge"] CoachFocus = Literal[ "rapport", "exploration", @@ -71,6 +73,35 @@ class LiveCoachSuggestion(BaseModel): sources: list[LiveCoachSource] = Field(default_factory=list) safety_note: Optional[str] = None latency_ms: int = 0 + persistence_source: Literal["database", "runtime"] = "database" + quota: Optional["LiveCoachQuota"] = None + credit_events: list["LiveCoachCreditEvent"] = Field(default_factory=list) + + +class LiveCoachQuota(BaseModel): + """회기 중 즉시 코칭 사용 가능 횟수.""" + + remaining: int = Field(ge=0) + max: int = Field(ge=1) + + +class LiveCoachCreditEvent(BaseModel): + """코칭 기회 사용/충전 학습 기록.""" + + event_id: str + session_id: str + turn_seq: int + stage: StageLabel | None = None + event_type: CoachCreditEventType + delta: int + balance: int = Field(ge=0) + reason: str + created_at: str + + @field_validator("stage", mode="before") + @classmethod + def _normalize_stage(cls, value: object) -> StageLabel | None: + return stage_label_or_none(value) class LiveCoachEvent(BaseModel): @@ -313,6 +344,7 @@ def build_rag_index_payloads() -> list[dict[str, Any]]: "sensitivity": _rag_sensitivity(source, chunk, kb_kind), "meta": { "live_coaching_source": True, + "source_title": title, "source_type": str(chunk.get("source_type") or source.get("source_type") or ""), "source_version": version_label, "citation": chunk_citation, @@ -557,14 +589,33 @@ def _grounding_block(grounding: list[LiveCoachGrounding]) -> str: return "\n".join(lines) +def _mask_prompt_text(value: object) -> str: + return guardrail.mask_pii(str(value or "")).text_masked + + +def _mask_prompt_value(value: Any) -> Any: + if isinstance(value, str): + return _mask_prompt_text(value) + if isinstance(value, dict): + return { + _mask_prompt_text(key): _mask_prompt_value(child) + for key, child in value.items() + } + if isinstance(value, list): + return [_mask_prompt_value(child) for child in value] + if isinstance(value, tuple): + return [_mask_prompt_value(child) for child in value] + return value + + def _messages(item: LiveCoachInput, grounding: list[LiveCoachGrounding]) -> list[EngineMessage]: - learner_masked = guardrail.mask_pii(item.learner_text).text_masked - client_masked = guardrail.mask_pii(item.client_reply or "").text_masked + learner_masked = _mask_prompt_text(item.learner_text) + client_masked = _mask_prompt_text(item.client_reply or "") recent = "\n".join( - f"{'상담자' if t.get('speaker') == 'counselor' else '내담자'}: {t.get('text', '')}" + f"{speaker_ko_label(t.get('speaker'))}: {_mask_prompt_text(t.get('text', ''))}" for t in item.recent_turns[-6:] ) or "(최근 맥락 없음)" - evaluation = json.dumps(item.evaluation or {}, ensure_ascii=False)[:1200] + evaluation = json.dumps(_mask_prompt_value(item.evaluation or {}), ensure_ascii=False)[:1200] system = ( "당신은 심리상담 수련생에게 회기 중 즉시 피드백을 주는 라이브 코치다.\n" "목표는 지금 흐름을 끊지 않고 다음 상담자 발화 하나를 더 낫게 만드는 것이다.\n\n" @@ -669,6 +720,8 @@ __all__ = [ "LiveCoachEvent", "LiveCoachGrounding", "LiveCoachInput", + "LiveCoachCreditEvent", + "LiveCoachQuota", "LiveCoachSource", "LiveCoachSuggestion", "clear_local_source_pack_cache", diff --git a/apps/api/app/services/memory.py b/apps/api/app/services/memory.py index 34a73fa..1d0f117 100644 --- a/apps/api/app/services/memory.py +++ b/apps/api/app/services/memory.py @@ -23,6 +23,7 @@ from dataclasses import dataclass, field from typing import Any, Callable, Literal, Optional from .state_machine import SessionState +from ..taxonomy import speaker_ko_label _SESSION_DIGEST_EXCERPT_CHARS = 90 @@ -331,7 +332,7 @@ def build_compression_messages(job: CompressionJob) -> list[dict[str, str]]: pinned 사실 보존·정답 미포함 지시 포함. """ transcript = "\n".join( - f"{('상담자' if t.speaker == 'counselor' else '내담자')}: {t.text}" + f"{speaker_ko_label(t.speaker)}: {t.text}" for t in job.digest_input.masked_turns ) threads = "\n".join(f"- {t}" for t in job.digest_input.open_threads) or "(없음)" diff --git a/apps/api/app/services/orchestrator.py b/apps/api/app/services/orchestrator.py index 2e8aea3..c915374 100644 --- a/apps/api/app/services/orchestrator.py +++ b/apps/api/app/services/orchestrator.py @@ -50,6 +50,25 @@ EvalHook = Callable[["TurnContext", str], Awaitable[Optional[dict]]] LlmAuditHook = Callable[[dict[str, Any]], Awaitable[None]] +def turn_evaluation_error_payload(ctx: "TurnContext", error: BaseException | str) -> dict[str, Any]: + """Represent a non-fatal fast-loop evaluator failure without hiding it.""" + st = ctx.state_after or ctx.state_before + if isinstance(error, BaseException): + # Exception messages can contain raw learner/provider text. Keep review-facing + # evidence to the exception type; detailed trace stays in server logs. + detail = type(error).__name__ + else: + detail = str(error).strip() or "unknown turn evaluation error" + masked = guardrail.mask_pii(detail).text_masked.strip() or "unknown turn evaluation error" + return { + "loop": "fast", + "turn_seq": st.turn_seq, + "stage": st.stage.value, + "appropriateness": "neutral", + "error": masked, + } + + @dataclass(slots=True) class TurnContext: """한 턴 파이프라인을 관통하는 컨텍스트(가드레일/상태/페르소나 산출 집약).""" @@ -248,8 +267,9 @@ async def run_turn_generate( if eval_hook is not None: try: evaluation = await eval_hook(ctx, reply) - except Exception: - evaluation = None # 평가 실패가 상담 루프를 막지 않게(비치명적) + except Exception as exc: + # 평가 실패는 상담 루프를 막지 않되, 리뷰/대시보드에서 조용히 사라지지 않게 남긴다. + evaluation = turn_evaluation_error_payload(ctx, exc) return TurnResult( turn_seq=st.turn_seq, diff --git a/apps/api/app/services/session_metrics.py b/apps/api/app/services/session_metrics.py index c859bb7..b205ddd 100644 --- a/apps/api/app/services/session_metrics.py +++ b/apps/api/app/services/session_metrics.py @@ -77,6 +77,8 @@ def turn_eval(turn: Any) -> dict[str, Any] | None: def turn_score(ev: dict[str, Any]) -> float | None: + if str(ev.get("error") or "").strip(): + return None raw = str(ev.get("appropriateness") or "").strip().lower() return _APPROPRIATENESS_SCORE.get(raw) @@ -118,6 +120,8 @@ def turn_techniques(ev: dict[str, Any]) -> list[str]: def turn_feedback_note(ev: dict[str, Any]) -> str | None: + if str(ev.get("error") or "").strip(): + return None raw = ev.get("appropriateness_note") if raw is None: return None diff --git a/apps/api/app/services/state_machine.py b/apps/api/app/services/state_machine.py index 55546a1..2d83081 100644 --- a/apps/api/app/services/state_machine.py +++ b/apps/api/app/services/state_machine.py @@ -15,16 +15,9 @@ MASTERPLAN §0/§2.2 + MEMORY_KNOWLEDGE_PERSONA_DESIGN §1.1·P2: from __future__ import annotations from dataclasses import dataclass, field, replace -from enum import Enum from typing import Optional - -# ── 단계 (taxonomy.Stage 와 한글 값 동일, 서비스 내부 결정론 전이용) ──────── -class Stage(str, Enum): - RAPPORT = "라포" - EXPLORE = "탐색" - INTERVENE = "개입" - CLOSE = "정리" +from ..taxonomy import Stage # 단계 라벨 단일 정의 = taxonomy.Stage; 이 모듈은 전이 로직만 소유. # 단계별 기본 개방도(stage_base). 라포는 낮게 시작, 개입에서 가장 깊게 다룸. diff --git a/apps/api/app/session_evaluation_input.py b/apps/api/app/session_evaluation_input.py new file mode 100644 index 0000000..181f8bd --- /dev/null +++ b/apps/api/app/session_evaluation_input.py @@ -0,0 +1,15 @@ +"""Shared input shaping for session-level evaluation runs.""" + +from __future__ import annotations + +from collections.abc import Iterable, Mapping + + +def enriched_masked_turns(masked_turns: Iterable[Mapping[str, object]]) -> list[dict[str, object]]: + """Attach the same 1-based turn sequence to every session evaluation path.""" + enriched: list[dict[str, object]] = [] + for index, turn in enumerate(masked_turns, start=1): + item = dict(turn) + item["seq"] = index + enriched.append(item) + return enriched diff --git a/apps/api/app/session_persistence.py b/apps/api/app/session_persistence.py index 7b57030..2f290de 100644 --- a/apps/api/app/session_persistence.py +++ b/apps/api/app/session_persistence.py @@ -38,6 +38,14 @@ _APPROPRIATENESS_SCORE = { "neutral": 3.0, "pos": 5.0, } +LIVE_COACH_INITIAL_CREDITS = 3 +LIVE_COACH_MAX_CREDITS = 3 +LIVE_COACH_USE_REASON = "AI 코칭 힌트 사용" +LIVE_COACH_RECHARGE_REASON = "좋은 발화로 내담자 변화 신호 확인" + + +class LiveCoachCreditExhausted(RuntimeError): + """Raised when a learner tries to use live coaching without credits.""" def _coerce_error_message(error: BaseException | str) -> str: @@ -99,8 +107,8 @@ class SessionEvaluationWrite: source=source, scope=result.scope, stage=result.stage, - payload=result.to_dict(), - error=result.error, + payload=_mask_json_text_values(result.to_dict()), + error=_clean_masked_text(result.error), ) @classmethod @@ -122,7 +130,7 @@ class SessionEvaluationWrite: scope=scope, stage=stage, payload={}, - error=_coerce_error_message(error), + error=_clean_masked_text(_coerce_error_message(error)), ) def cache_record(self) -> dict[str, Any]: @@ -244,6 +252,90 @@ def _live_coach_event_from_row(row) -> dict[str, Any]: } +def _live_coach_remaining(value: int | float | None) -> int: + try: + parsed = int(value if value is not None else LIVE_COACH_INITIAL_CREDITS) + except (TypeError, ValueError): + parsed = LIVE_COACH_INITIAL_CREDITS + return max(0, min(LIVE_COACH_MAX_CREDITS, parsed)) + + +def _live_coach_quota_payload(remaining: int | float | None) -> dict[str, int]: + return {"remaining": _live_coach_remaining(remaining), "max": LIVE_COACH_MAX_CREDITS} + + +def _live_coach_delta_from_record(record: dict[str, Any]) -> int: + if record.get("credit_delta") is not None: + try: + return int(record["credit_delta"]) + except (TypeError, ValueError): + pass + return 1 if record.get("event_type") == "recharge" else -1 + + +def _live_coach_cached_remaining(session_id: str) -> int: + delta = sum( + _live_coach_delta_from_record(record) + for record in _LIVE_COACH_EVENT_CACHE.get(session_id, []) + ) + return _live_coach_remaining(LIVE_COACH_INITIAL_CREDITS + delta) + + +async def _live_coach_remaining_for_conn(conn: Any, session_id: str) -> int: + delta = await conn.fetchval( + """ + SELECT COALESCE(SUM(credit_delta), 0) + FROM app.live_coach_events + WHERE session_id = $1::uuid + """, + session_id, + ) + return _live_coach_remaining(LIVE_COACH_INITIAL_CREDITS + int(delta or 0)) + + +def _live_coach_credit_event_from_row(row) -> dict[str, Any]: + event_type = str(_row_value(row, "event_type") or "use") + if event_type not in {"use", "recharge"}: + event_type = "use" + delta = _row_value(row, "credit_delta") + if delta is None: + delta = 1 if event_type == "recharge" else -1 + balance = _row_value(row, "credit_balance") + return { + "event_id": str(row["id"]), + "session_id": str(row["session_id"]), + "turn_seq": int(row["turn_seq"] or 1), + "stage": str(row["stage"] or ""), + "event_type": event_type, + "delta": int(delta), + "balance": _live_coach_remaining(balance), + "reason": str(_row_value(row, "reason") or ( + LIVE_COACH_RECHARGE_REASON if event_type == "recharge" else LIVE_COACH_USE_REASON + )), + "created_at": _iso_dt(row["created_at"]), + } + + +def _live_coach_credit_event_from_record(record: dict[str, Any]) -> dict[str, Any]: + event_type = str(record.get("event_type") or "use") + if event_type not in {"use", "recharge"}: + event_type = "use" + delta = _live_coach_delta_from_record(record) + return { + "event_id": str(record.get("event_id") or uuid.uuid4()), + "session_id": str(record.get("session_id") or ""), + "turn_seq": int(record.get("turn_seq") or 1), + "stage": str(record.get("stage") or ""), + "event_type": event_type, + "delta": delta, + "balance": _live_coach_remaining(record.get("credit_balance")), + "reason": str(record.get("reason") or ( + LIVE_COACH_RECHARGE_REASON if event_type == "recharge" else LIVE_COACH_USE_REASON + )), + "created_at": str(record.get("created_at") or ""), + } + + def _live_coach_cache_record( *, session_id: str, @@ -252,6 +344,35 @@ def _live_coach_cache_record( learner_text: str, client_reply: str | None, suggestion: Any, + credit_balance: int | None = None, +) -> dict[str, Any]: + now = datetime.now(timezone.utc) + balance = _live_coach_remaining( + credit_balance if credit_balance is not None else _live_coach_cached_remaining(session_id) - 1 + ) + return { + "event_id": str(uuid.uuid4()), + "session_id": session_id, + "turn_seq": int(turn_seq), + "stage": stage, + "event_type": "use", + "credit_delta": -1, + "credit_balance": balance, + "reason": LIVE_COACH_USE_REASON, + "created_at": now.isoformat().replace("+00:00", "Z"), + "learner_text_excerpt": _masked_excerpt(learner_text), + "client_reply_excerpt": _masked_excerpt(client_reply), + "suggestion": _model_payload(suggestion), + } + + +def _live_coach_recharge_cache_record( + *, + session_id: str, + turn_seq: int, + stage: str, + credit_balance: int, + reason: str, ) -> dict[str, Any]: now = datetime.now(timezone.utc) return { @@ -259,13 +380,22 @@ def _live_coach_cache_record( "session_id": session_id, "turn_seq": int(turn_seq), "stage": stage, + "event_type": "recharge", + "credit_delta": 1, + "credit_balance": _live_coach_remaining(credit_balance), + "reason": reason, "created_at": now.isoformat().replace("+00:00", "Z"), - "learner_text_excerpt": _masked_excerpt(learner_text), - "client_reply_excerpt": _masked_excerpt(client_reply), - "suggestion": _model_payload(suggestion), + "learner_text_excerpt": None, + "client_reply_excerpt": None, + "suggestion": {}, } +def _live_coach_payload_from_record(record: dict[str, Any]) -> dict[str, Any]: + payload = dict(record.get("suggestion") or {}) + return payload + + def _row_value(row, key: str): try: return row[key] @@ -294,6 +424,26 @@ def _clean_text(value: Any) -> str | None: return text or None +def _clean_masked_text(value: Any) -> str | None: + text = _clean_text(value) + if text is None: + return None + masked = guardrail.mask_pii(text).text_masked.strip() + return masked or None + + +def _mask_json_text_values(value: Any) -> Any: + if isinstance(value, str): + return _clean_masked_text(value) or "" + if isinstance(value, dict): + return {key: _mask_json_text_values(child) for key, child in value.items()} + if isinstance(value, list): + return [_mask_json_text_values(child) for child in value] + if isinstance(value, tuple): + return [_mask_json_text_values(child) for child in value] + return value + + def _safe_float(value: Any) -> float | None: if isinstance(value, (int, float)): return float(value) @@ -354,7 +504,7 @@ def _evaluation_feedback_rows(evaluation: dict[str, Any] | None) -> list[dict[st add( "appropriateness", score=_APPROPRIATENESS_SCORE[appropriateness], - rationale=_clean_text(evaluation.get("appropriateness_note")), + rationale=_clean_masked_text(evaluation.get("appropriateness_note")), ) rapport = _safe_float(evaluation.get("rapport_signal")) @@ -365,19 +515,19 @@ def _evaluation_feedback_rows(evaluation: dict[str, Any] | None) -> list[dict[st if theory_mode: add("theory_mode", rationale=theory_mode) - error = _clean_text(evaluation.get("error")) + error = _clean_masked_text(evaluation.get("error")) if error: add("error", rationale=error) for tag in _dict_items(evaluation.get("techniques")): code = _clean_text(tag.get("code")) - rationale = _clean_text(tag.get("rationale")) + rationale = _clean_masked_text(tag.get("rationale")) if code and rationale: add(f"technique:{code}", rationale=rationale) for state in _dict_items(evaluation.get("client_state_read")): code = _clean_text(state.get("code")) - rationale = _clean_text(state.get("rationale")) + rationale = _clean_masked_text(state.get("rationale")) if code and rationale: add(f"client_state:{code}", rationale=rationale) @@ -425,8 +575,8 @@ def _evaluation_comment_rows(evaluation: dict[str, Any] | None) -> list[dict[str deviation = evaluation.get("intent_deviation") if not isinstance(deviation, dict): return [] - note = _clean_text(evaluation.get("appropriateness_note")) or "의도와 다른 부분" - return [{"kind": "critique", "text": note, "intent_deviation": deviation}] + note = _clean_masked_text(evaluation.get("appropriateness_note")) or "의도와 다른 부분" + return [{"kind": "critique", "text": note, "intent_deviation": _mask_json_text_values(deviation)}] def _evaluation_alternative_rows(evaluation: dict[str, Any] | None) -> list[dict[str, str | None]]: @@ -438,10 +588,10 @@ def _evaluation_alternative_rows(evaluation: dict[str, Any] | None) -> list[dict rows: list[dict[str, str | None]] = [] for item in alternatives: if isinstance(item, dict): - suggestion = _clean_text(item.get("suggestion") or item.get("text")) - rationale = _clean_text(item.get("rationale")) + suggestion = _clean_masked_text(item.get("suggestion") or item.get("text")) + rationale = _clean_masked_text(item.get("rationale")) else: - suggestion = _clean_text(item) + suggestion = _clean_masked_text(item) rationale = None if suggestion: rows.append({"suggestion": suggestion, "rationale": rationale}) @@ -652,12 +802,13 @@ def _state_from_row(row, card: PersonaCard) -> state_machine.SessionState: def _turn_from_row(row, evaluation: dict[str, Any] | None = None) -> TurnRecord: created_at = _ts(row["created_at"]) or time.time() + text_masked = _clean_text(row["text_masked"]) or _clean_masked_text(row["text"]) or "" return TurnRecord( turn_seq=int(row["seq"]), speaker=row["speaker"], stage=row["stage"], - text=row["text"] or row["text_masked"] or "", - text_masked=row["text_masked"] or row["text"] or "", + text=text_masked, + text_masked=text_masked, turn_id=str(_row_value(row, "id")) if _row_value(row, "id") is not None else None, created_at=created_at, llm_provider=_row_value(row, "llm_provider"), @@ -780,6 +931,31 @@ async def _persist_turn_evaluation(conn: Any, turn_id: str, evaluation: dict[str ) +async def _replace_turn_evaluation(conn: Any, turn_id: str, evaluation: dict[str, Any] | None) -> None: + await conn.execute("SELECT set_config('app.ai_context', '1', true)") + await conn.execute("SELECT set_config('app.current_ai_view', 'evaluator', true)") + await conn.execute("SELECT set_config('app.current_sens_max', '2', true)") + await conn.execute("DELETE FROM app.feedback_scores WHERE turn_id = $1::uuid", turn_id) + await conn.execute("DELETE FROM app.turn_technique WHERE turn_id = $1::uuid", turn_id) + await conn.execute("DELETE FROM app.turn_client_state WHERE turn_id = $1::uuid", turn_id) + await conn.execute("DELETE FROM app.supervisor_comment WHERE turn_id = $1::uuid", turn_id) + await conn.execute("DELETE FROM app.alternative_utterance WHERE turn_id = $1::uuid", turn_id) + await _persist_turn_evaluation(conn, turn_id, _mask_json_text_values(evaluation)) + + +async def replace_turn_evaluation(*, turn_id: str, evaluation: dict[str, Any] | None) -> bool: + if not turn_id or not isinstance(evaluation, dict): + return False + try: + get_pool() + async with acquire(ai_context=True, ai_view="evaluator") as conn: + await _replace_turn_evaluation(conn, turn_id, evaluation) + return True + except Exception: + require_runtime_fallback_allowed("turn evaluation replacement") + return False + + async def _load_turn_evaluations( conn: Any, turn_refs: list[tuple[str, int, str]], @@ -853,8 +1029,14 @@ async def _load_turn_evaluations( async def _hydrate_session_turn_evaluations(sess: InProcSession) -> None: + await _hydrate_sessions_turn_evaluations([sess]) + + +async def _hydrate_sessions_turn_evaluations(sessions: list[InProcSession]) -> None: + """여러 세션의 턴 평가를 evaluator-view 연결 1회로 배치 하이드레이트(세션당 N+1 제거).""" turn_refs = [ (turn.turn_id, turn.turn_seq, turn.stage) + for sess in sessions for turn in sess.turns if turn.turn_id is not None ] @@ -862,9 +1044,10 @@ async def _hydrate_session_turn_evaluations(sess: InProcSession) -> None: return async with acquire(ai_view="evaluator") as conn: evaluations = await _load_turn_evaluations(conn, turn_refs) - for turn in sess.turns: - if turn.turn_id and turn.turn_id in evaluations: - turn.evaluation = evaluations[turn.turn_id] + for sess in sessions: + for turn in sess.turns: + if turn.turn_id and turn.turn_id in evaluations: + turn.evaluation = evaluations[turn.turn_id] async def ensure_review_tables() -> None: @@ -946,6 +1129,47 @@ async def ensure_review_tables() -> None: ) """ ) + await conn.execute( + """ + ALTER TABLE app.feedback_scores ENABLE ROW LEVEL SECURITY; + ALTER TABLE app.turn_technique ENABLE ROW LEVEL SECURITY; + ALTER TABLE app.turn_client_state ENABLE ROW LEVEL SECURITY; + ALTER TABLE app.supervisor_comment ENABLE ROW LEVEL SECURITY; + ALTER TABLE app.alternative_utterance ENABLE ROW LEVEL SECURITY; + + DROP POLICY IF EXISTS p_feedback_delete ON app.feedback_scores; + DROP POLICY IF EXISTS p_turn_technique_delete ON app.turn_technique; + DROP POLICY IF EXISTS p_turn_client_state_delete ON app.turn_client_state; + DROP POLICY IF EXISTS p_supervisor_comment_delete ON app.supervisor_comment; + DROP POLICY IF EXISTS p_alternative_utterance_delete ON app.alternative_utterance; + + CREATE POLICY p_feedback_delete + ON app.feedback_scores FOR DELETE USING ( + app.is_ai_context() + OR app.current_role_name() IN ('admin','instructor') + ); + CREATE POLICY p_turn_technique_delete + ON app.turn_technique FOR DELETE USING ( + app.is_ai_context() + OR app.current_role_name() IN ('admin','instructor') + ); + CREATE POLICY p_turn_client_state_delete + ON app.turn_client_state FOR DELETE USING ( + app.is_ai_context() + OR app.current_role_name() IN ('admin','instructor') + ); + CREATE POLICY p_supervisor_comment_delete + ON app.supervisor_comment FOR DELETE USING ( + app.is_ai_context() + OR app.current_role_name() IN ('admin','instructor') + ); + CREATE POLICY p_alternative_utterance_delete + ON app.alternative_utterance FOR DELETE USING ( + app.is_ai_context() + OR app.current_role_name() IN ('admin','instructor') + ) + """ + ) await conn.execute( """ CREATE TABLE IF NOT EXISTS app.case_worksheet ( @@ -1013,13 +1237,45 @@ async def ensure_review_tables() -> None: session_id UUID NOT NULL REFERENCES app.sessions(id) ON DELETE CASCADE, turn_seq INT NOT NULL CHECK (turn_seq >= 1), stage TEXT NOT NULL, + event_type TEXT NOT NULL DEFAULT 'use', + credit_delta INT NOT NULL DEFAULT -1, + credit_balance INT, + reason TEXT, learner_text_excerpt TEXT, client_reply_excerpt TEXT, payload JSONB NOT NULL DEFAULT '{}'::jsonb, created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); + ALTER TABLE app.live_coach_events + ADD COLUMN IF NOT EXISTS event_type TEXT NOT NULL DEFAULT 'use'; + ALTER TABLE app.live_coach_events + ADD COLUMN IF NOT EXISTS credit_delta INT NOT NULL DEFAULT -1; + ALTER TABLE app.live_coach_events + ADD COLUMN IF NOT EXISTS credit_balance INT; + ALTER TABLE app.live_coach_events + ADD COLUMN IF NOT EXISTS reason TEXT; + UPDATE app.live_coach_events + SET event_type = 'use' + WHERE event_type IS NULL OR event_type NOT IN ('use', 'recharge'); + UPDATE app.live_coach_events + SET credit_delta = CASE WHEN event_type = 'recharge' THEN 1 ELSE -1 END + WHERE credit_delta IS NULL; + DO $$ + BEGIN + IF NOT EXISTS ( + SELECT 1 + FROM pg_constraint + WHERE conname = 'live_coach_events_event_type_check' + ) THEN + ALTER TABLE app.live_coach_events + ADD CONSTRAINT live_coach_events_event_type_check + CHECK (event_type IN ('use', 'recharge')); + END IF; + END $$; CREATE INDEX IF NOT EXISTS idx_live_coach_events_session_turn ON app.live_coach_events(session_id, turn_seq, created_at); + CREATE INDEX IF NOT EXISTS idx_live_coach_events_session_type + ON app.live_coach_events(session_id, event_type, created_at); ALTER TABLE app.live_coach_events ENABLE ROW LEVEL SECURITY; @@ -1270,6 +1526,8 @@ async def ensure_review_tables() -> None: async def save_session_evaluation(write: SessionEvaluationWrite) -> bool: + write.payload = _mask_json_text_values(write.payload) + write.error = _clean_masked_text(write.error) record = write.cache_record() if runtime_fallback_allowed(): _EVALUATION_CACHE[write.session_id] = record @@ -1342,6 +1600,150 @@ async def load_session_evaluation( return _EVALUATION_CACHE.get(session_id), False +async def list_session_evaluations( + session_ids: list[str], + principal: Principal, +) -> tuple[dict[str, dict[str, Any]], bool]: + if not session_ids: + return {}, True + try: + get_pool() + async with acquire( + role=principal.role.value, + user_id=principal.user_id, + cohort_ids=principal.cohort_ids, + ) as conn: + rows = await conn.fetch( + """ + SELECT session_id::text AS session_id, status, source, scope, stage, payload, error, updated_at + FROM app.session_evaluation + WHERE session_id = ANY($1::uuid[]) + """, + session_ids, + ) + records = { + str(row["session_id"]): { + "status": row["status"], + "source": row["source"], + "scope": row["scope"], + "stage": row["stage"], + "payload": dict(row["payload"] or {}), + "error": row["error"], + "updated_at": _ts(row["updated_at"]), + } + for row in rows + } + cached_used = False + if runtime_fallback_allowed(): + for session_id in session_ids: + if session_id not in records and session_id in _EVALUATION_CACHE: + records[session_id] = _EVALUATION_CACHE[session_id] + cached_used = True + return records, not cached_used + except Exception: + require_runtime_fallback_allowed("session evaluation") + return { + session_id: _EVALUATION_CACHE[session_id] + for session_id in session_ids + if session_id in _EVALUATION_CACHE + }, False + + +async def list_sessions_missing_session_evaluation( + *, + older_than_seconds: float, + limit: int, +) -> tuple[list[InProcSession], bool]: + if limit <= 0: + return [], True + try: + get_pool() + stale_seconds = max(float(older_than_seconds), 0.0) + async with acquire(ai_context=True, ai_view="evaluator") as conn: + rows = await conn.fetch( + """ + SELECT + s.id, s.runtime_case_id, s.case_id, s.learner_id, s.persona_code, + s.session_no, s.theory_mode, s.started_at, s.ended_at, + s.prev_rapport_credit, + COALESCE( + NULLIF(learner.display_name, ''), + NULLIF(learner.nickname, ''), + NULLIF(learner.email, ''), + s.learner_id::text + ) AS learner_label, + pc.persona_id AS card_persona_id, + pc.code AS card_code, + pc.version AS card_version, + pc.status AS card_status, + pc.display_name AS card_display_name, + pc.difficulty AS card_difficulty, + pc.theory_target AS card_theory_target, + pc.demographics AS card_demographics, + pc.presenting AS card_presenting, + pc.history AS card_history, + pc.big5 AS card_big5, + pc.resistance AS card_resistance, + pc.speech_style AS card_speech_style, + pc.affect_baseline AS card_affect_baseline, + pc.ccd AS card_ccd, + pc.dsm5_dimensional AS card_dsm5_dimensional, + pc.triggers AS card_triggers, + pc.source_provenance AS card_source_provenance, + pc.is_synthetic AS card_is_synthetic + FROM app.sessions s + LEFT JOIN app.session_evaluation se ON se.session_id = s.id + LEFT JOIN app.persona_card pc + ON pc.persona_id = s.persona_id + AND pc.version = s.persona_version + LEFT JOIN app.app_user learner ON learner.user_id = s.learner_id + WHERE s.ended_at IS NOT NULL + AND s.ended_at <= now() - ($1::double precision * interval '1 second') + AND se.session_id IS NULL + AND EXISTS ( + SELECT 1 + FROM app.turns t + WHERE t.session_id = s.id + AND 'client' = ANY(t.visible_to) + ) + ORDER BY s.ended_at ASC + LIMIT $2 + """, + stale_seconds, + max(1, int(limit)), + ) + sessions: list[InProcSession] = [] + for row in rows: + session_id = str(row["id"]) + state_row = await conn.fetchrow( + """ + SELECT stage, turn_seq, effective_openness, rapport_credit, resistance, + ideation_stage, turns_in_stage, affect_state + FROM app.session_state + WHERE session_id = $1::uuid + """, + session_id, + ) + turn_rows = await conn.fetch( + """ + SELECT id, seq, speaker, stage, text, text_masked, created_at, + llm_provider, model, tokens_in, tokens_out, cost_usd, + audio_ref, silence_ms, speech_rate, barge_in, provider_events, visible_to + FROM app.turns + WHERE session_id = $1::uuid + ORDER BY seq + """, + session_id, + ) + sess = _session_from_rows(row, state_row, turn_rows) + if sess is not None: + sessions.append(sess) + return sessions, True + except Exception: + require_runtime_fallback_allowed("missing session evaluation recovery") + return [], False + + async def save_case_worksheet( *, session_id: str, @@ -1411,6 +1813,9 @@ async def save_live_coach_event( client_reply: str | None, suggestion: Any, ) -> tuple[dict[str, Any] | None, bool]: + cached_remaining = _live_coach_cached_remaining(session_id) + if cached_remaining <= 0: + raise LiveCoachCreditExhausted("live coach credit exhausted") record = _live_coach_cache_record( session_id=session_id, turn_seq=turn_seq, @@ -1418,32 +1823,42 @@ async def save_live_coach_event( learner_text=learner_text, client_reply=client_reply, suggestion=suggestion, + credit_balance=cached_remaining - 1, ) - if runtime_fallback_allowed(): - _LIVE_COACH_EVENT_CACHE.setdefault(session_id, []).append(record) try: get_pool() async with acquire(role="learner", user_id=learner_id) as conn: + remaining_before = await _live_coach_remaining_for_conn(conn, session_id) + if remaining_before <= 0: + raise LiveCoachCreditExhausted("live coach credit exhausted") + remaining_after = _live_coach_remaining(remaining_before - 1) row = await conn.fetchrow( """ INSERT INTO app.live_coach_events ( - session_id, turn_seq, stage, learner_text_excerpt, + session_id, turn_seq, stage, event_type, credit_delta, + credit_balance, reason, learner_text_excerpt, client_reply_excerpt, payload, created_at ) - VALUES ($1::uuid, $2, $3, $4, $5, $6::jsonb, now()) - RETURNING id, session_id, turn_seq, stage, learner_text_excerpt, + VALUES ($1::uuid, $2, $3, 'use', -1, $4, $5, $6, $7, $8::jsonb, now()) + RETURNING id, session_id, turn_seq, stage, event_type, credit_delta, + credit_balance, reason, learner_text_excerpt, client_reply_excerpt, payload, created_at """, session_id, int(turn_seq), stage, + remaining_after, + LIVE_COACH_USE_REASON, record.get("learner_text_excerpt"), record.get("client_reply_excerpt"), _model_payload(suggestion), ) return (_live_coach_event_from_row(row) if row else None), True + except LiveCoachCreditExhausted: + raise except Exception: require_runtime_fallback_allowed("live coach event save") + _LIVE_COACH_EVENT_CACHE.setdefault(session_id, []).append(record) return record, False @@ -1464,6 +1879,7 @@ async def list_live_coach_events( client_reply_excerpt, payload, created_at FROM app.live_coach_events WHERE session_id = $1::uuid + AND event_type = 'use' ORDER BY created_at ASC, turn_seq ASC """, session_id, @@ -1471,7 +1887,127 @@ async def list_live_coach_events( return [_live_coach_event_from_row(row) for row in rows], True except Exception: require_runtime_fallback_allowed("live coach event list") - return [dict(item) for item in _LIVE_COACH_EVENT_CACHE.get(session_id, [])], False + return [ + dict(item) + for item in _LIVE_COACH_EVENT_CACHE.get(session_id, []) + if item.get("event_type", "use") == "use" + ], False + + +async def get_live_coach_quota( + session_id: str, + principal: Principal, +) -> tuple[dict[str, int], bool]: + try: + get_pool() + async with acquire( + role=principal.role.value, + user_id=principal.user_id, + cohort_ids=principal.cohort_ids, + ) as conn: + remaining = await _live_coach_remaining_for_conn(conn, session_id) + return _live_coach_quota_payload(remaining), True + except Exception: + require_runtime_fallback_allowed("live coach quota") + return _live_coach_quota_payload(_live_coach_cached_remaining(session_id)), False + + +async def list_live_coach_credit_events( + session_id: str, + principal: Principal, +) -> tuple[list[dict[str, Any]], bool]: + try: + get_pool() + async with acquire( + role=principal.role.value, + user_id=principal.user_id, + cohort_ids=principal.cohort_ids, + ) as conn: + rows = await conn.fetch( + """ + SELECT id, session_id, turn_seq, stage, event_type, credit_delta, + credit_balance, reason, created_at + FROM app.live_coach_events + WHERE session_id = $1::uuid + ORDER BY created_at ASC, turn_seq ASC + """, + session_id, + ) + return [_live_coach_credit_event_from_row(row) for row in rows], True + except Exception: + require_runtime_fallback_allowed("live coach credit event list") + return [ + _live_coach_credit_event_from_record(item) + for item in _LIVE_COACH_EVENT_CACHE.get(session_id, []) + ], False + + +async def record_live_coach_recharge( + *, + session_id: str, + learner_id: str, + turn_seq: int, + stage: str, + reason: str = LIVE_COACH_RECHARGE_REASON, +) -> tuple[dict[str, Any] | None, bool]: + try: + get_pool() + async with acquire(role="learner", user_id=learner_id) as conn: + existing = await conn.fetchval( + """ + SELECT id + FROM app.live_coach_events + WHERE session_id = $1::uuid + AND turn_seq = $2 + AND event_type = 'recharge' + LIMIT 1 + """, + session_id, + int(turn_seq), + ) + if existing: + return None, True + remaining_before = await _live_coach_remaining_for_conn(conn, session_id) + if remaining_before >= LIVE_COACH_MAX_CREDITS: + return None, True + remaining_after = _live_coach_remaining(remaining_before + 1) + row = await conn.fetchrow( + """ + INSERT INTO app.live_coach_events ( + session_id, turn_seq, stage, event_type, credit_delta, + credit_balance, reason, payload, created_at + ) + VALUES ($1::uuid, $2, $3, 'recharge', 1, $4, $5, '{}'::jsonb, now()) + RETURNING id, session_id, turn_seq, stage, event_type, credit_delta, + credit_balance, reason, created_at + """, + session_id, + int(turn_seq), + stage, + remaining_after, + reason, + ) + return (_live_coach_credit_event_from_row(row) if row else None), True + except Exception: + require_runtime_fallback_allowed("live coach recharge") + events = _LIVE_COACH_EVENT_CACHE.setdefault(session_id, []) + if any( + event.get("event_type") == "recharge" and int(event.get("turn_seq") or 0) == int(turn_seq) + for event in events + ): + return None, False + remaining_before = _live_coach_cached_remaining(session_id) + if remaining_before >= LIVE_COACH_MAX_CREDITS: + return None, False + record = _live_coach_recharge_cache_record( + session_id=session_id, + turn_seq=turn_seq, + stage=stage, + credit_balance=remaining_before + 1, + reason=reason, + ) + events.append(record) + return _live_coach_credit_event_from_record(record), False def _review_status_from_row(row: Any) -> dict[str, Any]: @@ -1883,6 +2419,7 @@ def _session_from_rows(row, state_row, turn_rows: Iterable) -> InProcSession | N turns=[_turn_from_row(turn_row) for turn_row in turn_rows], ended=ended_at is not None, prev_rapport_credit=float(row["prev_rapport_credit"] or 0.0), + learner_label=_clean_text(_row_value(row, "learner_label")), ) @@ -2063,6 +2600,12 @@ async def load_session( s.id, s.runtime_case_id, s.case_id, s.learner_id, s.persona_code, s.session_no, s.theory_mode, s.started_at, s.ended_at, s.prev_rapport_credit, + COALESCE( + NULLIF(learner.display_name, ''), + NULLIF(learner.nickname, ''), + NULLIF(learner.email, ''), + s.learner_id::text + ) AS learner_label, pc.persona_id AS card_persona_id, pc.code AS card_code, pc.version AS card_version, @@ -2086,6 +2629,7 @@ async def load_session( LEFT JOIN app.persona_card pc ON pc.persona_id = s.persona_id AND pc.version = s.persona_version + LEFT JOIN app.app_user learner ON learner.user_id = s.learner_id WHERE s.id = $1::uuid """, session_id, @@ -2490,15 +3034,50 @@ async def end_session(sess: InProcSession, carry: memory.CarryOver) -> bool: return False -async def list_sessions( +RECENT_SESSION_LIST_LIMIT = 100 + + +async def list_recent_sessions( principal: Principal, *, include_turn_evaluation: bool = False, +) -> tuple[list[InProcSession], bool]: + return await _list_sessions( + principal, + include_turn_evaluation=include_turn_evaluation, + session_limit=RECENT_SESSION_LIST_LIMIT, + audit_access="list_recent_sessions", + ) + + +async def list_all_sessions( + principal: Principal, + *, + include_turn_evaluation: bool = False, +) -> tuple[list[InProcSession], bool]: + return await _list_sessions( + principal, + include_turn_evaluation=include_turn_evaluation, + session_limit=None, + audit_access="list_all_sessions", + ) + + +async def _list_sessions( + principal: Principal, + *, + include_turn_evaluation: bool, + session_limit: int | None, + audit_access: str, ) -> tuple[list[InProcSession], bool]: try: get_pool() learner_filter = "WHERE s.learner_id = $1::uuid" if principal.role.value == "learner" else "" - query_args = [principal.user_id] if principal.role.value == "learner" else [] + query_args: list[object] = [principal.user_id] if principal.role.value == "learner" else [] + limit_clause = "" + if session_limit is not None: + query_args.append(max(1, int(session_limit))) + limit_clause = f"LIMIT ${len(query_args)}" async with acquire( role=principal.role.value, user_id=principal.user_id, @@ -2510,6 +3089,12 @@ async def list_sessions( s.id, s.runtime_case_id, s.case_id, s.learner_id, s.persona_code, s.session_no, s.theory_mode, s.started_at, s.ended_at, s.prev_rapport_credit, + COALESCE( + NULLIF(learner.display_name, ''), + NULLIF(learner.nickname, ''), + NULLIF(learner.email, ''), + s.learner_id::text + ) AS learner_label, pc.persona_id AS card_persona_id, pc.code AS card_code, pc.version AS card_version, @@ -2533,47 +3118,58 @@ async def list_sessions( LEFT JOIN app.persona_card pc ON pc.persona_id = s.persona_id AND pc.version = s.persona_version + LEFT JOIN app.app_user learner ON learner.user_id = s.learner_id {learner_filter} ORDER BY s.started_at DESC - LIMIT 100 + {limit_clause} """, *query_args, ) - sessions: list[InProcSession] = [] - for row in rows: - session_id = str(row["id"]) - state_row = await conn.fetchrow( + # N+1 제거: 세션별 state/turns fetch 루프(1+2N 왕복) 대신 id 집합으로 한 번씩 배치 조회. + session_ids = [str(row["id"]) for row in rows] + states_by_id: dict[str, Any] = {} + turns_by_id: dict[str, list] = {} + if session_ids: + state_rows = await conn.fetch( """ - SELECT stage, turn_seq, effective_openness, rapport_credit, resistance, + SELECT session_id, stage, turn_seq, effective_openness, rapport_credit, resistance, ideation_stage, turns_in_stage, affect_state FROM app.session_state - WHERE session_id = $1::uuid + WHERE session_id = ANY($1::uuid[]) """, - session_id, + session_ids, ) - turn_rows = await conn.fetch( + states_by_id = {str(state_row["session_id"]): state_row for state_row in state_rows} + turn_rows_all = await conn.fetch( """ - SELECT id, seq, speaker, stage, text, text_masked, created_at, + SELECT session_id, id, seq, speaker, stage, text, text_masked, created_at, llm_provider, model, tokens_in, tokens_out, cost_usd, audio_ref, silence_ms, speech_rate, barge_in, provider_events, visible_to FROM app.turns - WHERE session_id = $1::uuid - ORDER BY seq + WHERE session_id = ANY($1::uuid[]) + ORDER BY session_id, seq """, - session_id, + session_ids, + ) + for turn_row in turn_rows_all: + turns_by_id.setdefault(str(turn_row["session_id"]), []).append(turn_row) + sessions: list[InProcSession] = [] + for row in rows: + session_id = str(row["id"]) + sess = _session_from_rows( + row, states_by_id.get(session_id), turns_by_id.get(session_id, []) ) - sess = _session_from_rows(row, state_row, turn_rows) if sess is not None: - if include_turn_evaluation: - await _hydrate_session_turn_evaluations(sess) sessions.append(sess) + if include_turn_evaluation and sessions: + await _hydrate_sessions_turn_evaluations(sessions) await _record_session_read_audit( conn, principal, target_kind="session_list", target_id="sessions", detail={ - "access": "list_sessions", + "access": audit_access, "role": principal.role.value, "result_count": len(sessions), }, @@ -2602,9 +3198,16 @@ async def list_safety_alerts( SELECT se.id, se.session_id, se.trigger_type, se.ko_risk_level, se.escalated, se.detail, se.created_at, - s.learner_id, s.persona_code, s.session_no + s.learner_id, s.persona_code, s.session_no, + COALESCE( + NULLIF(learner.display_name, ''), + NULLIF(learner.nickname, ''), + NULLIF(learner.email, ''), + s.learner_id::text + ) AS learner_label FROM app.safety_events se LEFT JOIN app.sessions s ON s.id = se.session_id + LEFT JOIN app.app_user learner ON learner.user_id = s.learner_id WHERE se.escalated = TRUE ORDER BY se.created_at DESC LIMIT $1 @@ -2616,7 +3219,8 @@ async def list_safety_alerts( "id": str(row["id"]), "session_id": str(row["session_id"]), "learner_id": str(row["learner_id"] or ""), - "learner_label": _learner_label_from_id(str(row["learner_id"] or "")), + "learner_label": _clean_text(row["learner_label"]) + or _learner_label_from_id(str(row["learner_id"] or "")), "persona_code": str(row["persona_code"] or ""), "session_no": int(row["session_no"] or 0), "trigger_type": str(row["trigger_type"] or "crisis"), diff --git a/apps/api/app/session_read_model.py b/apps/api/app/session_read_model.py index 88d0b70..0c45d7d 100644 --- a/apps/api/app/session_read_model.py +++ b/apps/api/app/session_read_model.py @@ -11,12 +11,12 @@ import re from collections import Counter from dataclasses import dataclass from datetime import datetime -from typing import Literal, Optional, cast +from typing import Any, Literal, Optional, cast from pydantic import BaseModel, Field from .config import settings -from .services import session_metrics +from .services import guardrail, session_metrics from .stage_contract import ( ReviewPhaseKey, StageLabel, @@ -31,6 +31,10 @@ WorksheetItemSpec = tuple[str, str, list[str], WorksheetSpeaker | None] WorksheetSectionSpec = tuple[str, str, list[WorksheetItemSpec]] LEARNER_VISIBLE_AI_ROLE = "counselor" +MISSING_SESSION_EVALUATION_GRACE_SECONDS = 30.0 +MISSING_SESSION_EVALUATION_ERROR = ( + "회기말 평가가 제한 시간 이후에도 저장되지 않았습니다. AI 평가 재시도가 필요합니다." +) class LearnerSessionSummary(BaseModel): @@ -816,6 +820,46 @@ def _review_summary_from_evaluation( return prefix + (details if details else "아래 코칭 항목은 저장된 축어록과 평가 AI 결과를 기준으로 합니다.") +def _session_evaluation_timeout_seconds() -> float: + configured = float(settings.session_evaluation_timeout or settings.engine_timeout) + return max(1.0, configured) + + +def _missing_session_evaluation_record( + sess: InProcSession, + *, + has_visible_turns: bool, + now_ts: float, +) -> dict[str, object] | None: + if not sess.ended or not has_visible_turns or sess.ended_at is None: + return None + stale_after = _session_evaluation_timeout_seconds() + MISSING_SESSION_EVALUATION_GRACE_SECONDS + if now_ts - sess.ended_at < stale_after: + return None + return { + "status": "error", + "source": "read_model", + "scope": "session_end", + "stage": stage_label(sess.state.stage), + "payload": {"error": MISSING_SESSION_EVALUATION_ERROR}, + "error": MISSING_SESSION_EVALUATION_ERROR, + "updated_at": iso(now_ts), + } + + +def missing_session_evaluation_record( + sess: InProcSession, + *, + has_visible_turns: bool, + now_ts: float, +) -> dict[str, object] | None: + return _missing_session_evaluation_record( + sess, + has_visible_turns=has_visible_turns, + now_ts=now_ts, + ) + + def _next_line_from_evaluation(payload: dict[str, object]) -> str | None: alternatives = payload.get("alternative_utterances") if not isinstance(alternatives, list): @@ -1025,7 +1069,26 @@ def _evaluation_payload(record: dict[str, object] | None) -> dict[str, object]: if not record: return {} payload = record.get("payload") - return payload if isinstance(payload, dict) else {} + if not isinstance(payload, dict): + return {} + masked = _mask_payload_text_values(payload) + return masked if isinstance(masked, dict) else {} + + +def _mask_payload_text(value: object) -> str: + return guardrail.mask_pii(str(value or "")).text_masked + + +def _mask_payload_text_values(value: Any) -> Any: + if isinstance(value, str): + return _mask_payload_text(value) + if isinstance(value, dict): + return {str(key): _mask_payload_text_values(child) for key, child in value.items()} + if isinstance(value, list): + return [_mask_payload_text_values(child) for child in value] + if isinstance(value, tuple): + return [_mask_payload_text_values(child) for child in value] + return value _TECHNIQUE_KIND_BY_CATEGORY = { @@ -1091,6 +1154,17 @@ def _review_note_from_turn_eval( if not isinstance(ev, dict): return None quote = _review_quote_excerpt(learner_text) + error_text = str(ev.get("error") or "").strip() + if error_text: + return ReviewNote( + author="평가 AI", + tone="warn", + title="턴 평가 실패", + body=_review_note_body_markdown( + f"이 발화의 fast-loop 평가를 완료하지 못했습니다.\n\n사유: {error_text}" + ), + quote=quote, + ) dev = ev.get("intent_deviation") if isinstance(dev, dict): dimension = str(dev.get("dimension") or "").strip() @@ -1261,7 +1335,14 @@ def build_session_review(read_input: SessionReviewReadInput) -> SessionReviewRes if duration_seconds > 0: axis.append(_offset_label(duration_seconds)) + now_ts = read_input.now_ts or datetime.now().timestamp() evaluation_record = read_input.evaluation_record + if evaluation_record is None and not hidden_turns: + evaluation_record = _missing_session_evaluation_record( + sess, + has_visible_turns=bool(visible_turns), + now_ts=now_ts, + ) evaluation_payload = {} if hidden_turns else _evaluation_payload(evaluation_record) evaluation_status = ( "" if hidden_turns else str(evaluation_record.get("status") or "") if evaluation_record else "" diff --git a/apps/api/app/store.py b/apps/api/app/store.py index 2bca7d4..a18330b 100644 --- a/apps/api/app/store.py +++ b/apps/api/app/store.py @@ -68,11 +68,7 @@ class InProcSession: turns: list[TurnRecord] = field(default_factory=list) ended: bool = False prev_rapport_credit: float = 0.0 # carry-over delta 계산용 - - def recent_turns(self, k: int = 6, visible_to: str | None = None) -> list[dict[str, str]]: - """최근 K턴 버퍼(L6 직전 맥락). 마스킹본 사용.""" - turns = self.turns if visible_to is None else self.turns_visible_to(visible_to) - return [{"speaker": t.speaker, "text": t.text_masked} for t in turns[-k:]] + learner_label: str | None = None def turns_visible_to(self, role: str) -> list[TurnRecord]: return [turn for turn in self.turns if turn.is_visible_to(role)] @@ -81,6 +77,10 @@ class InProcSession: turns = self.turns if visible_to is None else self.turns_visible_to(visible_to) return [{"speaker": t.speaker, "text": t.text_masked} for t in turns] + def recent_turns(self, k: int = 6, visible_to: str | None = None) -> list[dict[str, str]]: + """최근 K턴 버퍼(L6 직전 맥락). 마스킹본은 masked_turns 단일 소유, 여기선 꼬리 K개만.""" + return self.masked_turns(visible_to)[-k:] + class SessionStore: """in-memory 세션 저장소. DB degraded 시 SoR 대용.""" diff --git a/apps/api/app/taxonomy.py b/apps/api/app/taxonomy.py index c3f349d..7242255 100644 --- a/apps/api/app/taxonomy.py +++ b/apps/api/app/taxonomy.py @@ -30,10 +30,11 @@ TAXONOMY_VERSION = "1.0.0" # 라벨 코드 불변 보장 버전. 새 라벨 추 # ════════════════════════════════════════════════════════════════════════════ -# 1. 회기 단계 (stage) — 결정론 상태머신이 소유 (MASTERPLAN §2.2) +# 1. 회기 단계 (stage) — 라벨 enum 단일 정의(SoT). 전이 로직은 상태머신 소유(MASTERPLAN §2.2). +# services.state_machine 이 이 Stage 를 re-export 한다(중복 정의 제거). # ════════════════════════════════════════════════════════════════════════════ class Stage(str, Enum): - """상담 회기 단계. FastAPI 백엔드가 결정론적으로 전이(LLM 아님).""" + """상담 회기 단계 라벨(SoT). 결정론 전이는 services.state_machine 이 소유(LLM 아님).""" RAPPORT = "라포" # 라포 형성 — 안전감·관계 구축, 비밀보장 구조화 EXPLORE = "탐색" # 호소문제·정서·인지·위험요인 탐색 @@ -51,6 +52,11 @@ class Speaker(str, Enum): CLIENT = "client" # 내담자(가상내담자 AI) +def speaker_ko_label(speaker: str | None) -> str: + """화자 코드 → 프롬프트용 한글 라벨(SoT). counselor→상담자, 그 외→내담자.""" + return "상담자" if speaker == Speaker.COUNSELOR.value else "내담자" + + # ════════════════════════════════════════════════════════════════════════════ # 3. 기법 군집 (TechniqueCategory) — 위계 상위층 # ════════════════════════════════════════════════════════════════════════════ @@ -273,6 +279,7 @@ __all__ = [ "TAXONOMY_VERSION", "Stage", "Speaker", + "speaker_ko_label", "TechniqueCategory", "Technique", "TECHNIQUE_CATEGORY", diff --git a/apps/api/app/test_auth_providers.py b/apps/api/app/test_auth_providers.py index 1db44ac..507c569 100644 --- a/apps/api/app/test_auth_providers.py +++ b/apps/api/app/test_auth_providers.py @@ -426,6 +426,22 @@ class AuthProviderScaffoldTest(unittest.IsolatedAsyncioTestCase): self.assertTrue(principal.can_access_role(Role.TEACHER)) self.assertTrue(principal.can_access_role(Role.ADMIN)) + async def test_admin_role_can_enter_admin_console_even_with_stale_access_flag(self) -> None: + principal = Principal( + user_id="00000000-0000-0000-0000-000000000604", + role=Role.ADMIN, + email="stale-admin@twentyoz.kr", + display_name="Stale Admin", + admin_access=False, + super_admin=False, + ) + + admin_checker = deps.require_admin_access() + admin_view = await admin_checker(principal) + + self.assertEqual(admin_view.role, Role.ADMIN) + self.assertTrue(principal.can_access_role(Role.ADMIN)) + async def test_admin_access_flag_without_admin_role_does_not_enter_learner_space(self) -> None: principal = Principal( user_id="00000000-0000-0000-0000-000000000603", diff --git a/apps/api/app/test_dataset_export.py b/apps/api/app/test_dataset_export.py index e597ba1..a9b403d 100644 --- a/apps/api/app/test_dataset_export.py +++ b/apps/api/app/test_dataset_export.py @@ -1,3 +1,5 @@ +import importlib.util +import sys import tempfile import unittest from pathlib import Path @@ -12,11 +14,34 @@ from app.services.dataset_export import ( intraclass_correlation, scan_for_pii, sha256_file, + validate_manifest_gate, write_jsonl, ) +REPO_ROOT = Path(__file__).resolve().parents[3] +EXPORT_SCRIPT_PATH = REPO_ROOT / "scripts" / "export-recursive-dataset.py" + + +def load_export_script(): + spec = importlib.util.spec_from_file_location("export_recursive_dataset", EXPORT_SCRIPT_PATH) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + sys.modules[spec.name] = module + spec.loader.exec_module(module) + return module + + class DatasetExportTests(unittest.TestCase): + def test_exporter_query_requires_consent_and_client_visible_turns(self) -> None: + exporter = load_export_script() + query = exporter.TURN_QUERY + + self.assertIn("u.consent_at IS NOT NULL", query) + self.assertIn("'client' = ANY(t.visible_to)", query) + self.assertNotRegex(query, r"\bt\.text\s*(?:,|AS\b)") + self.assertNotIn("sc.text", query) + def test_build_dataset_record_uses_masked_text_and_pseudonymous_keys(self) -> None: keys = ExportKeyMaps() record = build_dataset_record( @@ -45,6 +70,32 @@ class DatasetExportTests(unittest.TestCase): self.assertNotIn("22222222-2222-2222-2222-222222222222", blob) self.assertNotIn("raw text should never be exported", blob) + def test_build_dataset_record_drops_supervisor_comment_text(self) -> None: + record = build_dataset_record( + { + "session_id": "11111111-1111-1111-1111-111111111111", + "learner_id": "22222222-2222-2222-2222-222222222222", + "persona_code": "P1", + "stage": "rapport", + "speaker": "client", + "text_masked": "마스킹된 발화입니다.", + "supervisor_comments": [ + { + "kind": "note", + "text": "김서연 010-1234-5678 raw supervisor note", + "intent_deviation": {"severity": "low"}, + } + ], + }, + item_index=1, + export_manifest_id="phase3-rl-seed-test", + keys=ExportKeyMaps(), + ) + + self.assertEqual(record["supervisor_comments"], [{"kind": "note", "intent_deviation": {"severity": "low"}}]) + self.assertNotIn("김서연", str(record)) + self.assertNotIn("010-1234-5678", str(record)) + def test_pii_scan_detects_identifiers_without_raw_samples(self) -> None: findings = scan_for_pii( { @@ -89,6 +140,27 @@ class DatasetExportTests(unittest.TestCase): agreement={"kappa": 0.59, "icc": 0.74, "gold_status": "not_gold"}, ) + def test_manifest_gate_requires_withdrawal_and_recursive_consent_scope(self) -> None: + manifest = { + "export_status": APPROVED_EXPORT_STATUS, + "pii_scan": {"status": "pass"}, + "agreement": {"kappa": 0.70, "icc": 0.75}, + "selection_criteria": {"include_withdrawn": True}, + "consent_scope": {"allowed_uses": ["education_quality_review"]}, + "approvals": { + "data_steward": "steward", + "legal_or_privacy_reviewer": "privacy", + "technical_operator": "operator", + "approved_at": "2026-07-01T00:00:00Z", + }, + } + + with self.assertRaisesRegex( + ValueError, + "include_withdrawn must be false; recursive_learning_seed consent scope is required", + ): + validate_manifest_gate(manifest) + def test_jsonl_hash_manifest_dry_run(self) -> None: record = { "schema": "phase3_dataset_item_v1", diff --git a/apps/api/app/test_eval_routes.py b/apps/api/app/test_eval_routes.py index 73f4278..9e1cdd4 100644 --- a/apps/api/app/test_eval_routes.py +++ b/apps/api/app/test_eval_routes.py @@ -109,6 +109,38 @@ class ReevaluateSessionRouteTest(unittest.IsolatedAsyncioTestCase): self.assertEqual(evaluate_session.await_args.kwargs["theory_mode"], "cbt") + async def test_reevaluate_session_uses_one_based_masked_turn_seq(self) -> None: + session_id = "00000000-0000-0000-0000-00000000e222" + sess = _session_stub() + result = evaluator.SessionEvaluation( + session_id=session_id, + stage="정리", + scope="session_end", + turns_evaluated=2, + ) + + with ( + patch.object(eval_routes, "_load_session_or_404", AsyncMock(return_value=sess)), + patch.object( + eval_routes.evaluator, + "evaluate_session", + AsyncMock(return_value=result), + ) as evaluate_session, + patch.object( + eval_routes.session_persistence, + "save_session_evaluation", + AsyncMock(return_value=True), + ), + ): + await eval_routes.reevaluate_session( + session_id, + eval_routes.ReevaluateRequest(scope="session_end"), + _teacher_principal(), + ) + + masked_turns = evaluate_session.await_args.kwargs["masked_turns"] + self.assertEqual([turn["seq"] for turn in masked_turns], [1, 2]) + async def test_reevaluate_turn_preserves_session_theory_mode(self) -> None: session_id = "00000000-0000-0000-0000-00000000e222" sess = _session_stub(theory_mode="cbt", theory_target=["humanistic"], with_turns=True) @@ -121,6 +153,11 @@ class ReevaluateSessionRouteTest(unittest.IsolatedAsyncioTestCase): "evaluate_turn", AsyncMock(return_value=result), ) as evaluate_turn, + patch.object( + eval_routes.session_persistence, + "replace_turn_evaluation", + AsyncMock(return_value=True), + ) as replace_turn_evaluation, ): await eval_routes.reevaluate_turn( session_id, @@ -130,6 +167,119 @@ class ReevaluateSessionRouteTest(unittest.IsolatedAsyncioTestCase): ctx = evaluate_turn.await_args.args[0] self.assertEqual(ctx.theory_mode, "cbt") + replace_turn_evaluation.assert_awaited_once() + self.assertEqual( + replace_turn_evaluation.await_args.kwargs["turn_id"], + "00000000-0000-0000-0000-00000000a001", + ) + + async def test_reevaluate_turn_persists_result_for_review_hydration(self) -> None: + session_id = "00000000-0000-0000-0000-00000000e222" + sess = _session_stub(theory_mode="cbt", with_turns=True) + result = evaluator.TurnEvaluation( + turn_seq=1, + stage="정리", + theory_mode="cbt", + appropriateness="pos", + rapport_signal=0.4, + techniques=[ + evaluator.TechniqueTag( + code="empathy", + label_ko="공감", + category="relational", + rationale="감정을 반영했다.", + ) + ], + ) + + with ( + patch.object(eval_routes, "_load_session_or_404", AsyncMock(return_value=sess)), + patch.object( + eval_routes.evaluator, + "evaluate_turn", + AsyncMock(return_value=result), + ), + patch.object( + eval_routes.session_persistence, + "replace_turn_evaluation", + AsyncMock(return_value=True), + ) as replace_turn_evaluation, + ): + response = await eval_routes.reevaluate_turn( + session_id, + eval_routes.TurnReevaluateRequest(turn_seq=1), + _teacher_principal(), + ) + + self.assertEqual(response.appropriateness, "pos") + saved_payload = replace_turn_evaluation.await_args.kwargs["evaluation"] + self.assertEqual(saved_payload["turn_seq"], 1) + self.assertEqual(saved_payload["theory_mode"], "cbt") + self.assertEqual(saved_payload["rapport_signal"], 0.4) + self.assertEqual(saved_payload["techniques"][0]["label_ko"], "공감") + self.assertEqual(sess.turns[0].evaluation, saved_payload) + + async def test_reevaluate_turn_raises_when_result_cannot_be_saved(self) -> None: + session_id = "00000000-0000-0000-0000-00000000e222" + sess = _session_stub(with_turns=True) + result = evaluator.TurnEvaluation(turn_seq=1, stage="정리") + + with ( + patch.object(eval_routes, "_load_session_or_404", AsyncMock(return_value=sess)), + patch.object( + eval_routes.evaluator, + "evaluate_turn", + AsyncMock(return_value=result), + ), + patch.object( + eval_routes.session_persistence, + "replace_turn_evaluation", + AsyncMock(return_value=False), + ), + ): + with self.assertRaises(HTTPException) as raised: + await eval_routes.reevaluate_turn( + session_id, + eval_routes.TurnReevaluateRequest(turn_seq=1), + _teacher_principal(), + ) + + self.assertEqual(raised.exception.status_code, 503) + self.assertIn("could not be saved", raised.exception.detail) + + async def test_reevaluate_turn_saves_error_marker_then_raises(self) -> None: + session_id = "00000000-0000-0000-0000-00000000e222" + sess = _session_stub(with_turns=True) + result = evaluator.TurnEvaluation( + turn_seq=1, + stage="정리", + error="engine_error", + ) + + with ( + patch.object(eval_routes, "_load_session_or_404", AsyncMock(return_value=sess)), + patch.object( + eval_routes.evaluator, + "evaluate_turn", + AsyncMock(return_value=result), + ), + patch.object( + eval_routes.session_persistence, + "replace_turn_evaluation", + AsyncMock(return_value=True), + ) as replace_turn_evaluation, + ): + with self.assertRaises(HTTPException) as raised: + await eval_routes.reevaluate_turn( + session_id, + eval_routes.TurnReevaluateRequest(turn_seq=1), + _teacher_principal(), + ) + + self.assertEqual(raised.exception.status_code, 503) + self.assertEqual(raised.exception.detail, "engine_error") + saved_payload = replace_turn_evaluation.await_args.kwargs["evaluation"] + self.assertEqual(saved_payload["error"], "engine_error") async def test_get_session_evaluation_exposes_status_error_and_durable_flag(self) -> None: session_id = "00000000-0000-0000-0000-00000000e222" @@ -179,12 +329,14 @@ def _session_stub( turns = ( [ SimpleNamespace( + turn_id="00000000-0000-0000-0000-00000000a001", turn_seq=1, speaker="counselor", text="상담자 발화", text_masked="상담자 발화", ), SimpleNamespace( + turn_id="00000000-0000-0000-0000-00000000a002", turn_seq=2, speaker="client", text="내담자 응답", @@ -196,7 +348,10 @@ def _session_stub( ) def masked_turns(self): - return [{"speaker": "counselor", "text_masked": "상담자 발화"}] + return [ + {"speaker": "counselor", "text_masked": "상담자 발화"}, + {"speaker": "client", "text_masked": "내담자 응답"}, + ] return SessionStub() diff --git a/apps/api/app/test_evaluation_persistence.py b/apps/api/app/test_evaluation_persistence.py index bd86470..ca7cc2c 100644 --- a/apps/api/app/test_evaluation_persistence.py +++ b/apps/api/app/test_evaluation_persistence.py @@ -3,13 +3,14 @@ from __future__ import annotations from pathlib import Path +import json import unittest from unittest.mock import patch from .deps import Principal, Role from . import session_persistence from .routes import sessions -from .services import evaluator, session_metrics +from .services import evaluator, guardrail, session_metrics from .services import persona as persona_service from .services import state_machine from .store import InProcSession @@ -33,8 +34,26 @@ class FakeEvaluationConn: raise AssertionError(f"unexpected fetchval query: {query}") +class FakeMissingEvaluationConn: + def __init__(self) -> None: + self.fetches: list[tuple[str, tuple[object, ...]]] = [] + self.fetchrows: list[tuple[str, tuple[object, ...]]] = [] + + async def fetch(self, query: str, *args: object) -> list[dict[str, object]]: + self.fetches.append((query, args)) + if "FROM app.sessions s" in query: + return [{"id": "11111111-1111-1111-1111-111111111111"}] + if "FROM app.turns" in query: + return [] + raise AssertionError(f"unexpected fetch query: {query}") + + async def fetchrow(self, query: str, *args: object) -> dict[str, object]: + self.fetchrows.append((query, args)) + return {} + + class FakeAcquire: - def __init__(self, conn: FakeEvaluationConn) -> None: + def __init__(self, conn) -> None: self.conn = conn async def __aenter__(self) -> FakeEvaluationConn: @@ -44,7 +63,7 @@ class FakeAcquire: return None -class EvaluationPersistenceMappingTest(unittest.TestCase): +class EvaluationPersistenceMappingTest(unittest.IsolatedAsyncioTestCase): def test_session_metrics_prefers_rehydrated_technique_label_ko(self) -> None: ev = { "techniques": [ @@ -137,6 +156,91 @@ class EvaluationPersistenceMappingTest(unittest.TestCase): ], ) + def test_turn_evaluation_rows_mask_pii_before_db_persistence(self) -> None: + class FakeKoRecognizer: + def analyze(self, text: str): + spans = [] + for entity_type, value in ( + ("NAME", "보라별"), + ("ORG", "미래학교상담연구랩"), + ): + start = text.find(value) + if start >= 0: + spans.append(guardrail.PiiEntitySpan(entity_type, start, start + len(value))) + return spans + + guardrail.set_ko_pii_recognizer(FakeKoRecognizer()) + self.addCleanup(guardrail.set_ko_pii_recognizer, None) + evaluation = { + "appropriateness": "warn", + "appropriateness_note": "보라별에게 010-1234-5678을 되물었다.", + "techniques": [ + { + "code": "reflection", + "rationale": "미래학교상담연구랩 이야기를 바로 조언했다.", + } + ], + "client_state_read": [ + { + "code": "avoidance", + "rationale": "보라별이 기관 미래학교상담연구랩을 피했다.", + } + ], + "intent_deviation": { + "expected": "보라별의 감정을 확인", + "actual": "010-1234-5678 연락처를 재질문", + }, + "alternative_utterances": [ + { + "text": "보라별님, 미래학교상담연구랩 이야기는 잠시 미뤄도 괜찮아요.", + "rationale": "010-1234-5678 같은 연락처 재확인을 피함", + } + ], + } + + blob = json.dumps( + { + "feedback": session_persistence._evaluation_feedback_rows(evaluation), + "comments": session_persistence._evaluation_comment_rows(evaluation), + "alternatives": session_persistence._evaluation_alternative_rows(evaluation), + }, + ensure_ascii=False, + ) + + for raw in ("보라별", "미래학교상담연구랩", "010-1234-5678"): + self.assertNotIn(raw, blob) + for masked in ("[NAME]", "[ORG]", "[PHONE]"): + self.assertIn(masked, blob) + + def test_turn_from_legacy_row_remasks_raw_text_fallback(self) -> None: + class FakeKoRecognizer: + def analyze(self, text: str): + start = text.find("보라별") + if start < 0: + return [] + return [guardrail.PiiEntitySpan("NAME", start, start + len("보라별"))] + + guardrail.set_ko_pii_recognizer(FakeKoRecognizer()) + self.addCleanup(guardrail.set_ko_pii_recognizer, None) + turn = session_persistence._turn_from_row( + { + "id": "11111111-1111-1111-1111-111111111111", + "seq": 2, + "speaker": "client", + "stage": "explore", + "text": "보라별의 전화는 010-1234-5678입니다.", + "text_masked": "", + "created_at": None, + "visible_to": ("client", "evaluator"), + } + ) + + self.assertEqual(turn.text, turn.text_masked) + self.assertNotIn("보라별", turn.text) + self.assertNotIn("010-1234-5678", turn.text) + self.assertIn("[NAME]", turn.text) + self.assertIn("[PHONE]", turn.text) + def test_session_evaluation_write_from_result_preserves_payload_shape(self) -> None: result = evaluator.SessionEvaluation( session_id="session-1", @@ -159,6 +263,55 @@ class EvaluationPersistenceMappingTest(unittest.TestCase): self.assertNotIn("error", write.payload) self.assertIsNone(write.error) + def test_session_evaluation_write_masks_deep_payload_before_storage(self) -> None: + class FakeKoRecognizer: + def analyze(self, text: str): + spans = [] + for entity_type, value in ( + ("NAME", "보라별"), + ("ORG", "미래학교상담연구랩"), + ): + start = text.find(value) + if start >= 0: + spans.append(guardrail.PiiEntitySpan(entity_type, start, start + len(value))) + return spans + + guardrail.set_ko_pii_recognizer(FakeKoRecognizer()) + self.addCleanup(guardrail.set_ko_pii_recognizer, None) + result = evaluator.SessionEvaluation( + session_id="session-privacy", + stage="정리", + scope="session_end", + turns_evaluated=3, + strengths=["보라별의 감정을 반영했다."], + improvements=["010-1234-5678 같은 연락처 재확인은 피한다."], + intent_deviations=[ + evaluator.IntentDeviation( + dimension="pacing", + expected="미래학교상담연구랩 이야기를 기다린다", + actual="보라별에게 바로 조언했다", + severity="moderate", + ) + ], + supervisor_rationale="보라별의 호소를 요약했다.", + supervisor_critique="미래학교상담연구랩과 010-1234-5678을 반복했다.", + alternative_utterances=["보라별님, 지금 감정부터 천천히 볼까요?"], + error="보라별 평가 경고", + ) + + write = session_persistence.SessionEvaluationWrite.from_result( + session_id="session-privacy", + learner_id="learner-privacy", + result=result, + ) + blob = json.dumps({"payload": write.payload, "error": write.error}, ensure_ascii=False) + + self.assertEqual(write.status, "error") + for raw in ("보라별", "미래학교상담연구랩", "010-1234-5678"): + self.assertNotIn(raw, blob) + for masked in ("[NAME]", "[ORG]", "[PHONE]"): + self.assertIn(masked, blob) + def test_session_evaluation_write_from_error_preserves_fallback_shape(self) -> None: write = session_persistence.SessionEvaluationWrite.from_error( session_id="session-1", @@ -186,6 +339,65 @@ class EvaluationPersistenceMappingTest(unittest.TestCase): self.assertEqual(write.error, "TimeoutError") + def test_session_evaluation_write_from_error_masks_error_detail(self) -> None: + class FakeKoRecognizer: + def analyze(self, text: str): + start = text.find("보라별") + if start < 0: + return [] + return [guardrail.PiiEntitySpan("NAME", start, start + len("보라별"))] + + guardrail.set_ko_pii_recognizer(FakeKoRecognizer()) + self.addCleanup(guardrail.set_ko_pii_recognizer, None) + write = session_persistence.SessionEvaluationWrite.from_error( + session_id="session-1", + learner_id="learner-1", + scope="session_end", + stage="explore", + error="보라별 처리 중 010-1234-5678 오류", + ) + + self.assertNotIn("보라별", write.error or "") + self.assertNotIn("010-1234-5678", write.error or "") + self.assertIn("[NAME]", write.error or "") + self.assertIn("[PHONE]", write.error or "") + + async def test_save_session_evaluation_remasks_direct_write_payload(self) -> None: + class FakeKoRecognizer: + def analyze(self, text: str): + start = text.find("보라별") + if start < 0: + return [] + return [guardrail.PiiEntitySpan("NAME", start, start + len("보라별"))] + + guardrail.set_ko_pii_recognizer(FakeKoRecognizer()) + self.addCleanup(guardrail.set_ko_pii_recognizer, None) + conn = FakeEvaluationConn() + write = session_persistence.SessionEvaluationWrite( + session_id="11111111-1111-1111-1111-111111111111", + learner_id="22222222-2222-2222-2222-222222222222", + status="ready", + source="engine", + scope="session_end", + stage="정리", + payload={"strengths": ["보라별의 전화 010-1234-5678을 반복했다."]}, + error="보라별 direct write error", + ) + + with ( + patch.object(session_persistence, "get_pool", return_value=object()), + patch.object(session_persistence, "acquire", return_value=FakeAcquire(conn)), + ): + saved = await session_persistence.save_session_evaluation(write) + + self.assertTrue(saved) + _, args = conn.executed[0] + blob = json.dumps({"payload": args[5], "error": args[6]}, ensure_ascii=False) + self.assertNotIn("보라별", blob) + self.assertNotIn("010-1234-5678", blob) + self.assertIn("[NAME]", blob) + self.assertIn("[PHONE]", blob) + def test_rebuild_turn_evaluation_restores_review_shape(self) -> None: rebuilt = session_persistence._rebuild_turn_evaluations( [("11111111-1111-1111-1111-111111111111", 2, "탐색")], @@ -270,6 +482,11 @@ class EvaluationPersistenceMappingTest(unittest.TestCase): self.assertIn("ALTER TABLE app.turn_client_state ENABLE ROW LEVEL SECURITY", sql) self.assertIn("ALTER TABLE app.supervisor_comment ENABLE ROW LEVEL SECURITY", sql) self.assertIn("ALTER TABLE app.alternative_utterance ENABLE ROW LEVEL SECURITY", sql) + self.assertIn("CREATE POLICY p_feedback_delete", sql) + self.assertIn("CREATE POLICY p_turn_technique_delete", sql) + self.assertIn("CREATE POLICY p_turn_client_state_delete", sql) + self.assertIn("CREATE POLICY p_supervisor_comment_delete", sql) + self.assertIn("CREATE POLICY p_alternative_utterance_delete", sql) feedback_insert = sql.split("CREATE POLICY p_feedback_insert", 1)[1].split(");", 1)[0] self.assertNotIn("learner_id = app.current_uid()", feedback_insert) @@ -281,6 +498,32 @@ class EvaluationPersistenceMappingTest(unittest.TestCase): class EvaluationPersistenceIOTest(unittest.IsolatedAsyncioTestCase): + async def test_missing_session_evaluation_finder_uses_stale_ai_context_query(self) -> None: + conn = FakeMissingEvaluationConn() + sentinel = object() + + with ( + patch.object(session_persistence, "get_pool", return_value=object()), + patch.object(session_persistence, "acquire", return_value=FakeAcquire(conn)) as acquire, + patch.object(session_persistence, "_session_from_rows", return_value=sentinel), + ): + found, durable = await session_persistence.list_sessions_missing_session_evaluation( + older_than_seconds=150.0, + limit=3, + ) + + self.assertTrue(durable) + self.assertEqual(found, [sentinel]) + acquire.assert_called_once_with(ai_context=True, ai_view="evaluator") + self.assertEqual(conn.fetches[0][1], (150.0, 3)) + session_query = conn.fetches[0][0] + self.assertIn("s.ended_at IS NOT NULL", session_query) + self.assertIn("se.session_id IS NULL", session_query) + self.assertIn("'client' = ANY(t.visible_to)", session_query) + self.assertIn("ORDER BY s.ended_at ASC", session_query) + self.assertEqual(conn.fetchrows[0][1], ("11111111-1111-1111-1111-111111111111",)) + self.assertIn("FROM app.turns", conn.fetches[1][0]) + async def test_record_llm_call_audit_inserts_metadata_only(self) -> None: conn = FakeEvaluationConn() payload = { @@ -366,6 +609,44 @@ class EvaluationPersistenceIOTest(unittest.IsolatedAsyncioTestCase): self.assertIn("DELETE FROM app.alternative_utterance", executed_sql) self.assertIn("INSERT INTO app.alternative_utterance", executed_sql) + async def test_replace_turn_evaluation_clears_stale_normalized_rows_before_reinsert(self) -> None: + conn = FakeEvaluationConn() + evaluation = { + "loop": "fast", + "turn_seq": 3, + "stage": "탐색", + "appropriateness": "pos", + "techniques": [ + {"code": "reflection", "label_ko": "반영", "category": "relational"} + ], + "client_state_read": [{"code": "open", "label_ko": "개방"}], + "alternative_utterances": ["조금 더 머물러도 괜찮습니다."], + } + + with ( + patch.object(session_persistence, "get_pool", return_value=object()), + patch.object(session_persistence, "acquire", return_value=FakeAcquire(conn)) as acquire, + ): + ok = await session_persistence.replace_turn_evaluation( + turn_id="11111111-1111-1111-1111-111111111111", + evaluation=evaluation, + ) + + self.assertTrue(ok) + acquire.assert_called_once_with(ai_context=True, ai_view="evaluator") + executed_sql = "\n".join(query for query, _ in conn.executed) + self.assertLess( + executed_sql.index("DELETE FROM app.feedback_scores"), + executed_sql.index("INSERT INTO app.feedback_scores"), + ) + self.assertIn("DELETE FROM app.turn_technique", executed_sql) + self.assertIn("DELETE FROM app.turn_client_state", executed_sql) + self.assertIn("DELETE FROM app.supervisor_comment", executed_sql) + self.assertIn("DELETE FROM app.alternative_utterance", executed_sql) + self.assertIn("INSERT INTO app.turn_technique", executed_sql) + self.assertIn("INSERT INTO app.turn_client_state", executed_sql) + self.assertIn("INSERT INTO app.alternative_utterance", executed_sql) + async def test_route_loader_only_hydrates_when_requested(self) -> None: principal = Principal( user_id="00000000-0000-0000-0000-000000000101", diff --git a/apps/api/app/test_learner_dashboard.py b/apps/api/app/test_learner_dashboard.py index 5164eae..3df34ba 100644 --- a/apps/api/app/test_learner_dashboard.py +++ b/apps/api/app/test_learner_dashboard.py @@ -113,9 +113,9 @@ class LearnerDashboardTest(unittest.IsolatedAsyncioTestCase): with ( patch.object( sessions.session_persistence, - "list_sessions", + "list_recent_sessions", AsyncMock(return_value=(owned_sessions, True)), - ) as list_sessions, + ) as list_recent_sessions, patch.object( sessions.session_persistence, "load_session_evaluation", @@ -129,7 +129,7 @@ class LearnerDashboardTest(unittest.IsolatedAsyncioTestCase): ): response = await sessions.learner_dashboard(principal) - list_sessions.assert_awaited_once_with(principal, include_turn_evaluation=True) + list_recent_sessions.assert_awaited_once_with(principal, include_turn_evaluation=True) list_session_archives.assert_awaited_once_with( [sess.session_id for sess in owned_sessions], principal, @@ -155,6 +155,59 @@ class LearnerDashboardTest(unittest.IsolatedAsyncioTestCase): self.assertEqual(len(response.recent_feedback), 3) self.assertIn("근거 기반 피드백", response.recent_feedback[0].note) + async def test_dashboard_excludes_failed_fast_loop_evaluations_from_growth_and_feedback(self) -> None: + principal = _principal() + good_session = _session( + session_id="00000000-0000-0000-0000-00000000b211", + session_no=1, + persona_code=P1.code, + score="pos", + rapport=0.5, + technique="reflection", + created_at=1_000.0, + ) + failed_session = _session( + session_id="00000000-0000-0000-0000-00000000b212", + session_no=2, + persona_code=P1.code, + score="neutral", + rapport=0.2, + technique="failed evaluator", + created_at=2_000.0, + ) + failed_eval = failed_session.turns[0].evaluation + assert failed_eval is not None + failed_eval["error"] = "engine_error" + failed_eval["appropriateness_note"] = "이 문장은 최근 피드백에 노출되면 안 됩니다." + failed_eval["techniques"] = [] + + with ( + patch.object( + sessions.session_persistence, + "list_recent_sessions", + AsyncMock(return_value=([good_session, failed_session], True)), + ), + patch.object( + sessions.session_persistence, + "load_session_evaluation", + AsyncMock(return_value=({"status": "ready"}, True)), + ), + patch.object( + sessions.session_persistence, + "list_session_archives", + AsyncMock(return_value=({}, True)), + ), + ): + response = await sessions.learner_dashboard(principal) + + self.assertEqual(response.growth.evaluated_sessions, 1) + self.assertEqual(response.growth.first_score, 1.0) + self.assertEqual(response.growth.latest_score, 1.0) + self.assertEqual(response.growth.avg_score, 1.0) + self.assertEqual(len(response.recent_feedback), 1) + self.assertIn("reflection 근거 기반 피드백", response.recent_feedback[0].note) + self.assertNotIn("노출되면 안 됩니다", response.model_dump_json()) + async def test_session_list_marks_archived_rows_from_learner_archive_state(self) -> None: principal = _principal() owned_sessions = [ @@ -190,9 +243,9 @@ class LearnerDashboardTest(unittest.IsolatedAsyncioTestCase): with ( patch.object( sessions.session_persistence, - "list_sessions", + "list_recent_sessions", AsyncMock(return_value=(owned_sessions, True)), - ) as list_sessions, + ) as list_recent_sessions, patch.object( sessions.session_persistence, "list_session_archives", @@ -202,7 +255,7 @@ class LearnerDashboardTest(unittest.IsolatedAsyncioTestCase): ): response = await sessions.list_learner_sessions(principal) - list_sessions.assert_awaited_once_with(principal) + list_recent_sessions.assert_awaited_once_with(principal) list_session_archives.assert_awaited_once_with( [sess.session_id for sess in owned_sessions], principal, @@ -263,7 +316,7 @@ class LearnerDashboardTest(unittest.IsolatedAsyncioTestCase): with ( patch.object( sessions.session_persistence, - "list_sessions", + "list_recent_sessions", AsyncMock(return_value=(owned_sessions, True)), ), patch.object( diff --git a/apps/api/app/test_live_coach_privacy.py b/apps/api/app/test_live_coach_privacy.py new file mode 100644 index 0000000..b82a311 --- /dev/null +++ b/apps/api/app/test_live_coach_privacy.py @@ -0,0 +1,103 @@ +import unittest + +from . import turn_runtime +from .services import guardrail, live_coach +from .services import state_machine + + +class LiveCoachPromptPrivacyTest(unittest.TestCase): + def tearDown(self) -> None: + guardrail.set_ko_pii_recognizer(None) + + def test_messages_remask_recent_turns_and_evaluation_payload(self) -> None: + class FakeKoRecognizer: + def analyze(self, text: str): + spans = [] + for entity_type, value in ( + ("NAME", "보라별"), + ("ORG", "미래학교상담연구랩"), + ): + start = text.find(value) + if start >= 0: + spans.append(guardrail.PiiEntitySpan(entity_type, start, start + len(value))) + return spans + + guardrail.set_ko_pii_recognizer(FakeKoRecognizer()) + item = live_coach.LiveCoachInput( + session_id="privacy-live-coach", + turn_seq=3, + stage="exploration", + effective_openness=0.42, + theory_mode="humanistic", + persona_code="P1", + persona_name="서연", + learner_text="별명은 보라별이고 기관은 미래학교상담연구랩입니다.", + client_reply="전화는 010-1234-5678입니다.", + recent_turns=[ + { + "speaker": "counselor", + "text": "보라별이 미래학교상담연구랩에서 힘들다고 했죠.", + }, + { + "speaker": "client", + "text": "010-1234-5678로 연락하지 말아주세요.", + }, + ], + evaluation={ + "appropriateness": "warn", + "appropriateness_note": "보라별 연락처 010-1234-5678을 되물었다.", + "nested": {"rationale": "기관 미래학교상담연구랩 언급"}, + }, + ) + + messages = live_coach._messages(item, []) + user_prompt = messages[1].content + + for raw in ("보라별", "미래학교상담연구랩", "010-1234-5678"): + self.assertNotIn(raw, user_prompt) + for masked in ("[NAME]", "[ORG]", "[PHONE]"): + self.assertIn(masked, user_prompt) + + +class LiveCoachQuotaRechargeTest(unittest.TestCase): + def test_recharge_requires_positive_score_and_client_change(self) -> None: + before = state_machine.SessionState( + stage=state_machine.Stage.RAPPORT, + turn_seq=2, + effective_openness=0.15, + ) + after = state_machine.SessionState( + stage=state_machine.Stage.RAPPORT, + turn_seq=3, + effective_openness=0.19, + ) + + should_recharge, reason = turn_runtime.should_recharge_live_coach_credit( + {"appropriateness": "pos", "rapport_signal": 0.45}, + before, + after, + ) + self.assertTrue(should_recharge) + self.assertIn("개방도", reason) + + should_recharge, _ = turn_runtime.should_recharge_live_coach_credit( + {"appropriateness": "pos", "rapport_signal": 0.45}, + before, + state_machine.SessionState( + stage=state_machine.Stage.RAPPORT, + turn_seq=3, + effective_openness=0.16, + ), + ) + self.assertFalse(should_recharge) + + should_recharge, _ = turn_runtime.should_recharge_live_coach_credit( + {"appropriateness": "neutral", "rapport_signal": 0.8}, + before, + after, + ) + self.assertFalse(should_recharge) + + +if __name__ == "__main__": + unittest.main() diff --git a/apps/api/app/test_notifications.py b/apps/api/app/test_notifications.py index fa0f668..83257b6 100644 --- a/apps/api/app/test_notifications.py +++ b/apps/api/app/test_notifications.py @@ -228,6 +228,71 @@ class NotificationTriggerTest(unittest.IsolatedAsyncioTestCase): logs.output, ) + async def test_missing_session_evaluation_recovery_schedules_stale_sessions(self) -> None: + sess = _ended_session() + + with ( + patched_settings(session_evaluation_timeout=10.0), + patch.object( + sessions.session_persistence, + "list_sessions_missing_session_evaluation", + AsyncMock(return_value=([sess], True)), + ) as list_missing, + patch.object( + sessions, + "_schedule_session_evaluation", + return_value=object(), + ) as schedule_evaluation, + ): + scheduled = await sessions.recover_missing_session_evaluations(limit=2) + + self.assertEqual(scheduled, 1) + list_missing.assert_awaited_once_with(older_than_seconds=40.0, limit=2) + schedule_evaluation.assert_called_once_with(sess) + + async def test_missing_session_evaluation_recovery_requires_durable_store(self) -> None: + sess = _ended_session() + + with ( + patch.object( + sessions.session_persistence, + "list_sessions_missing_session_evaluation", + AsyncMock(return_value=([sess], False)), + ), + patch.object(sessions, "_schedule_session_evaluation") as schedule_evaluation, + ): + scheduled = await sessions.recover_missing_session_evaluations(limit=2) + + self.assertEqual(scheduled, 0) + schedule_evaluation.assert_not_called() + + async def test_session_evaluation_scheduler_deduplicates_in_flight_session(self) -> None: + sess = _ended_session() + gate = asyncio.Event() + sessions._SESSION_EVALUATION_IN_FLIGHT.clear() + + async def wait_until_released(_sess: InProcSession) -> None: + await gate.wait() + + try: + with patch.object( + sessions, + "_generate_and_save_session_evaluation", + wait_until_released, + ): + task = sessions._schedule_session_evaluation(sess) + self.assertIsNotNone(task) + self.assertIsNone(sessions._schedule_session_evaluation(sess)) + self.assertIn(sess.session_id, sessions._SESSION_EVALUATION_IN_FLIGHT) + gate.set() + assert task is not None + await task + await asyncio.sleep(0) + finally: + sessions._SESSION_EVALUATION_IN_FLIGHT.clear() + + self.assertNotIn(sess.session_id, sessions._SESSION_EVALUATION_IN_FLIGHT) + if __name__ == "__main__": unittest.main() diff --git a/apps/api/app/test_phase3_artifact_checker.py b/apps/api/app/test_phase3_artifact_checker.py index f8dc028..86d66fd 100644 --- a/apps/api/app/test_phase3_artifact_checker.py +++ b/apps/api/app/test_phase3_artifact_checker.py @@ -26,6 +26,33 @@ def sha256(path: Path) -> str: return hashlib.sha256(path.read_bytes()).hexdigest() +def valid_dataset_item() -> dict[str, object]: + return { + "schema": "phase3_dataset_item_v1", + "item_id": "DI-000001", + "participant_key": "PX-0001", + "session_key": "SX-0001", + "turn_key": "TX-000001", + "persona_id": "P1", + "stage": "rapport", + "speaker": "client", + "text_masked": "요즘 [NAME] 관련 고민이 있습니다.", + "techniques": [], + "client_states": [], + "feedback_scores": [], + "supervisor_comments": [], + "source_refs": { + "session_started_at": "2026-06-27T00:00:00Z", + "export_manifest_id": "phase3-fixture", + }, + "privacy": { + "direct_identifiers_removed": True, + "pii_scan_status": "pass", + "consent_scope": "recursive_learning_seed", + }, + } + + class Phase3ArtifactCheckerTests(unittest.TestCase): def make_root(self) -> Path: tmp = tempfile.TemporaryDirectory() @@ -66,7 +93,10 @@ class Phase3ArtifactCheckerTests(unittest.TestCase): root / "04-privacy" / "privacy_audit.md", "# Privacy audit\n\nLegal/privacy reviewer: reviewer@example.invalid\n", ) - write_text(root / "03-export" / "anonymized_dataset.jsonl", "{}\n") + write_text( + root / "03-export" / "anonymized_dataset.jsonl", + json.dumps(valid_dataset_item(), ensure_ascii=False, sort_keys=True) + "\n", + ) metrics = { name: { @@ -236,6 +266,33 @@ class Phase3ArtifactCheckerTests(unittest.TestCase): self.assertTrue(any("sha256 mismatch" in error for error in report.errors), report.errors) + def test_approved_manifest_rejects_malformed_dataset_jsonl(self) -> None: + root = self.make_root() + dataset_path = root / "03-export" / "anonymized_dataset.jsonl" + write_text(dataset_path, "{}\n") + manifest_path = root / "03-export" / "export_manifest.json" + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + manifest["files"][0]["sha256"] = sha256(dataset_path) + manifest_path.write_text(json.dumps(manifest), encoding="utf-8") + + report = checker.validate(root, max_scan_rows=100) + + self.assertTrue(any("dataset item missing keys" in error for error in report.errors), report.errors) + + def test_dry_run_manifest_rejects_malformed_dataset_jsonl(self) -> None: + root = self.make_root() + dataset_path = root / "03-export" / "anonymized_dataset.jsonl" + write_text(dataset_path, "{}\n") + manifest_path = root / "03-export" / "export_manifest.json" + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + manifest["export_status"] = "technical_dry_run" + manifest["files"][0]["sha256"] = sha256(dataset_path) + manifest_path.write_text(json.dumps(manifest), encoding="utf-8") + + report = checker.validate(root, max_scan_rows=100) + + self.assertTrue(any("dataset item missing keys" in error for error in report.errors), report.errors) + if __name__ == "__main__": unittest.main() diff --git a/apps/api/app/test_rbac_idor.py b/apps/api/app/test_rbac_idor.py index 70366bb..cc9c361 100644 --- a/apps/api/app/test_rbac_idor.py +++ b/apps/api/app/test_rbac_idor.py @@ -542,7 +542,7 @@ class LearnerSessionIdorTest(unittest.IsolatedAsyncioTestCase): with ( patch.object( sessions.session_persistence, - "list_sessions", + "list_recent_sessions", AsyncMock(return_value=([], False)), ), patch.object( @@ -560,7 +560,7 @@ class LearnerSessionIdorTest(unittest.IsolatedAsyncioTestCase): class TeacherAdminAuditTest(unittest.IsolatedAsyncioTestCase): - async def test_teacher_list_sessions_inserts_read_audit_log(self) -> None: + async def test_teacher_list_recent_sessions_inserts_read_audit_log(self) -> None: teacher = _principal( user_id="00000000-0000-0000-0000-000000000505", role=Role.TEACHER, @@ -571,9 +571,9 @@ class TeacherAdminAuditTest(unittest.IsolatedAsyncioTestCase): session_id=session_id, learner_id="00000000-0000-0000-0000-000000000606", ) + # 배치 조회(N+1 제거): main sessions → session_state 배치 → turns 배치 (fetch 3회, fetchrow 0회). conn = _FakeConn( - fetchrow_results=[None], - fetch_results=[[row], []], + fetch_results=[[row], [], []], ) acquire_calls: list[dict[str, Any]] = [] @@ -586,7 +586,7 @@ class TeacherAdminAuditTest(unittest.IsolatedAsyncioTestCase): patch.object(session_persistence, "acquire", fake_acquire), patch.object(session_persistence, "_session_from_rows", return_value=sess), ): - found, durable = await session_persistence.list_sessions(teacher) + found, durable = await session_persistence.list_recent_sessions(teacher) self.assertTrue(durable) self.assertEqual(found, [sess]) @@ -598,7 +598,7 @@ class TeacherAdminAuditTest(unittest.IsolatedAsyncioTestCase): self.assertEqual(args[1], "read_session") self.assertEqual(args[2], "session_list") self.assertEqual(args[3], "sessions") - self.assertEqual(args[4]["access"], "list_sessions") + self.assertEqual(args[4]["access"], "list_recent_sessions") self.assertEqual(args[4]["role"], "teacher") self.assertEqual(args[4]["result_count"], 1) diff --git a/apps/api/app/test_runtime_policy.py b/apps/api/app/test_runtime_policy.py index e8ec056..a00661d 100644 --- a/apps/api/app/test_runtime_policy.py +++ b/apps/api/app/test_runtime_policy.py @@ -78,7 +78,7 @@ class RuntimeFallbackPolicyTest(unittest.IsolatedAsyncioTestCase): ) with environment("staging"): with self.assertRaises(HTTPException) as caught: - await session_persistence.list_sessions(principal) + await session_persistence.list_recent_sessions(principal) self.assertEqual(caught.exception.status_code, 503) self.assertIn("runtime fallback is disabled in staging", caught.exception.detail) diff --git a/apps/api/app/test_session_turn_persistence.py b/apps/api/app/test_session_turn_persistence.py index e176451..53edc3f 100644 --- a/apps/api/app/test_session_turn_persistence.py +++ b/apps/api/app/test_session_turn_persistence.py @@ -16,7 +16,7 @@ from .deps import Principal, Role from .engine_client import EngineError from .routes import sessions from .routes import voice as voice_routes -from .services import live_coach, memory, orchestrator, persona as persona_service, state_machine +from .services import guardrail, live_coach, memory, orchestrator, persona as persona_service, rag, state_machine from .services.voice import TTSChunk, TranscriptResult, VoicePreset from .store import InProcSession, TurnRecord, store @@ -29,6 +29,17 @@ async def _decoded_stream_packets(stream_engine, req): yield packet +class _AsyncConnContext: + def __init__(self, conn): + self.conn = conn + + async def __aenter__(self): + return self.conn + + async def __aexit__(self, exc_type, exc, tb): + return False + + def _principal() -> Principal: return Principal( user_id="00000000-0000-0000-0000-000000000101", @@ -83,6 +94,22 @@ class SessionTurnPersistenceTest(unittest.IsolatedAsyncioTestCase): sessions._RECALL_CACHE.clear() sessions._KB_CUES_CACHE.clear() + async def test_end_session_does_not_reschedule_evaluation_for_already_ended_session(self) -> None: + principal = _principal() + sess = _session(principal) + sess.ended = True + sess.ended_at = 1_000.0 + + with ( + patch.object(sessions, "_load_session_or_404", AsyncMock(return_value=sess)), + patch.object(sessions, "_end_persisted_session", AsyncMock(return_value=None)), + patch.object(sessions, "_schedule_session_evaluation") as schedule_session_evaluation, + ): + response = await sessions.end_session(sess.session_id, principal) + + self.assertEqual(response.session_id, sess.session_id) + schedule_session_evaluation.assert_not_called() + async def test_append_turn_writes_provider_events_to_db(self) -> None: class FakeConn: def __init__(self) -> None: @@ -161,6 +188,80 @@ class SessionTurnPersistenceTest(unittest.IsolatedAsyncioTestCase): self.assertEqual(caught.exception.status_code, 503) self.assertEqual(sess.turns, []) + async def test_live_coach_turn_marks_runtime_persistence_source(self) -> None: + principal = _principal() + sess = _session(principal) + sess.turns.extend( + [ + TurnRecord( + turn_seq=1, + speaker="counselor", + stage="explore", + text="요즘 가장 크게 남는 마음은 무엇인가요?", + text_masked="요즘 가장 크게 남는 마음은 무엇인가요?", + ), + TurnRecord( + turn_seq=2, + speaker="client", + stage="explore", + text="잘 모르겠지만 계속 답답해요.", + text_masked="잘 모르겠지만 계속 답답해요.", + ), + ] + ) + suggestion = live_coach.LiveCoachSuggestion( + status="ready", + tone="pos", + focus="emotion", + title="감정 반영이 선명합니다", + message="방금 응답의 정서를 먼저 붙잡았습니다.", + ) + + with ( + patch.object( + sessions, + "_retrieve_live_coach_grounding", + AsyncMock(return_value=[]), + ), + patch.object( + sessions.live_coach, + "generate_live_coaching", + AsyncMock(return_value=suggestion), + ), + patch.object( + sessions.session_persistence, + "get_live_coach_quota", + AsyncMock( + side_effect=[ + ({"remaining": 3, "max": 3}, True), + ({"remaining": 2, "max": 3}, True), + ] + ), + ), + patch.object( + sessions.session_persistence, + "save_live_coach_event", + AsyncMock(return_value=({"event_id": "cached-live-coach"}, False)), + ), + patch.object( + sessions.session_persistence, + "list_live_coach_credit_events", + AsyncMock(return_value=([], True)), + ), + ): + response = await sessions.live_coach_turn( + sess.session_id, + sessions.LiveCoachRequest( + learner_text="요즘 가장 크게 남는 마음은 무엇인가요?", + client_reply="잘 모르겠지만 계속 답답해요.", + turn_seq=1, + ), + principal, + ) + + self.assertEqual(response.persistence_source, "runtime") + self.assertEqual(response.quota.remaining, 2) + async def test_generate_turn_persists_client_engine_telemetry(self) -> None: principal = _principal() sess = _session(principal) @@ -462,6 +563,70 @@ class SessionTurnPersistenceTest(unittest.IsolatedAsyncioTestCase): self.assertIn("조금 말해볼게요", learner_turn.evaluation["appropriateness_note"]) self.assertIsNone(client_turn.evaluation) + async def test_stream_turn_surfaces_fast_loop_evaluation_failure_on_review(self) -> None: + principal = _principal() + sess = _session(principal) + + async def successful_stream(ctx, engine, **kwargs): + assert ctx.state_after is not None + yield orchestrator.StreamEvent("token", {"text": "조금 말해볼게요."}) + yield orchestrator.StreamEvent( + "done", + { + "session_id": ctx.session_id, + "stage": ctx.state_after.stage.value, + "effective_openness": ctx.state_after.effective_openness, + "turn_seq": ctx.state_after.turn_seq, + "safety_flagged": False, + "llm_provider": "claude_cli", + "model": "gateway-default", + "tokens_in": 9, + "tokens_out": 10, + "cost_usd": 0.001, + }, + ) + + async def failing_eval_hook(ctx, client_reply): + raise RuntimeError("김서연 평가 timeout 010-1234-5678") + + with patch.object(sessions.orchestrator, "run_turn_stream", successful_stream), patch.object( + sessions.evaluator, + "make_eval_hook", + return_value=failing_eval_hook, + ): + response = await sessions.stream_turn( + sess.session_id, + sessions.TurnRequest(text="스트림 평가 실패 발화"), + principal, + ) + await _consume_event_source(response) + + self.assertEqual(len(sess.turns), 2) + learner_turn = sess.turns[0] + self.assertIsNotNone(learner_turn.evaluation) + assert learner_turn.evaluation is not None + self.assertEqual(learner_turn.evaluation["loop"], "fast") + self.assertEqual(learner_turn.evaluation["appropriateness"], "neutral") + self.assertEqual(learner_turn.evaluation["error"], "RuntimeError") + self.assertNotIn("김서연", learner_turn.evaluation["error"]) + self.assertNotIn("010-1234-5678", learner_turn.evaluation["error"]) + + review = sessions.build_session_review( + sessions.SessionReviewReadInput( + session=sess, + evaluation_record=None, + evaluation_durable=False, + ) + ) + review_turn = review.turns[0] + self.assertIsNotNone(review_turn.note) + assert review_turn.note is not None + self.assertEqual(review_turn.note.title, "턴 평가 실패") + self.assertIn("fast-loop 평가를 완료하지 못했습니다", review_turn.note.body) + self.assertIn("RuntimeError", review_turn.note.body) + self.assertNotIn("김서연", review_turn.note.body) + self.assertNotIn("010-1234-5678", review_turn.note.body) + async def test_live_coach_degrades_to_rule_based_suggestion_when_engine_fails(self) -> None: principal = _principal() sess = _session(principal) @@ -507,12 +672,58 @@ class SessionTurnPersistenceTest(unittest.IsolatedAsyncioTestCase): self.assertEqual(len(history.events), 1) self.assertEqual(history.events[0].turn_seq, 1) self.assertEqual(history.events[0].stage, "라포") + self.assertEqual(history.events[0].suggestion.status, "degraded") self.assertEqual(history.events[0].suggestion.title, response.title) self.assertIn("학교", history.events[0].learner_text_excerpt or "") session_persistence._LIVE_COACH_EVENT_CACHE[sess.session_id][0]["stage"] = "unknown-stage" legacy_history = await sessions.list_live_coach_history(sess.session_id, principal) self.assertIsNone(legacy_history.events[0].stage) + async def test_live_coach_rag_grounding_preserves_source_pack_metadata(self) -> None: + retrieval = rag.RetrievalResult( + chunks=[ + rag.RetrievedChunk( + chunk_id=17, + score=0.91, + kb_kind="supervisor_pattern", + heading_path="risk/protective-factors", + context_prefix="공식 자살위험 평가 지침 / 보호요인 / 2026-06-15", + body="최근성, 보호요인, 안전계획을 확인한다.", + meta={ + "source_title": "공식 자살위험 평가 지침", + "source_type": "official_guideline_summary", + "source_version": "2026-06-15", + "citation": "허가된 공식 지침 요약", + }, + source_id="official_suicide_risk_guidelines", + ) + ], + policy_name="evaluator:k=supervisor_pattern:s<=2", + top1_score=0.91, + latency_ms=12, + query_text="정리 humanistic 자살 위험 최근성 보호요인", + ) + + with ( + patch.object(sessions.db, "acquire", return_value=_AsyncConnContext(object())), + patch.object(sessions.rag, "retrieve_eval_grounding", AsyncMock(return_value=retrieval)), + patch.object(sessions.rag, "log_retrieval", AsyncMock()), + ): + grounding = await sessions._retrieve_live_coach_grounding( + learner_text="자살 위험 최근성과 보호요인을 확인했어요.", + client_reply="그래도 오늘은 친구에게 연락할 수 있어요.", + stage="정리", + theory_mode="humanistic", + ) + + self.assertEqual(len(grounding), 1) + source = grounding[0] + self.assertEqual(source.source_id, "official_suicide_risk_guidelines") + self.assertEqual(source.title, "공식 자살위험 평가 지침") + self.assertEqual(source.source_type, "official_guideline_summary") + self.assertEqual(source.version, "2026-06-15") + self.assertEqual(source.citation, "허가된 공식 지침 요약") + async def test_live_coach_persistence_failure_is_not_swallowed(self) -> None: principal = _principal() sess = _session(principal) @@ -1306,6 +1517,101 @@ class SessionTurnPersistenceTest(unittest.IsolatedAsyncioTestCase): self.assertIn("주호소", response.teacherReview.worksheetNote) self.assertEqual(response.teacherReview.worksheetReviewedAt, "2026-06-27T10:05:00Z") + async def test_review_remasks_legacy_session_evaluation_payload(self) -> None: + class FakeKoRecognizer: + def analyze(self, text: str): + spans = [] + for entity_type, value in ( + ("NAME", "보라별"), + ("ORG", "미래학교상담연구랩"), + ): + start = text.find(value) + if start >= 0: + spans.append(guardrail.PiiEntitySpan(entity_type, start, start + len(value))) + return spans + + guardrail.set_ko_pii_recognizer(FakeKoRecognizer()) + self.addCleanup(guardrail.set_ko_pii_recognizer, None) + principal = _principal() + sess = _session(principal) + sess.ended = True + sess.ended_at = sess.created_at + 180 + sess.turns.extend( + [ + TurnRecord( + turn_seq=1, + speaker="counselor", + stage=sess.state.stage.value, + text="감정을 먼저 확인해보겠습니다.", + text_masked="감정을 먼저 확인해보겠습니다.", + created_at=sess.created_at + 1, + ), + TurnRecord( + turn_seq=2, + speaker="client", + stage=sess.state.stage.value, + text="연락처 이야기는 부담스러워요.", + text_masked="연락처 이야기는 부담스러워요.", + created_at=sess.created_at + 2, + ), + ] + ) + raw_record = { + "status": "ready", + "source": "engine", + "scope": "session_end", + "stage": "정리", + "payload": { + "loop": "deep", + "session_id": sess.session_id, + "stage": "정리", + "scope": "session_end", + "turns_evaluated": 2, + "distribution": {}, + "strengths": ["보라별의 감정을 반영했다."], + "improvements": ["미래학교상담연구랩과 010-1234-5678 재확인을 줄인다."], + "intent_deviations": [ + { + "dimension": "pacing", + "expected": "보라별의 감정 확인", + "actual": "010-1234-5678 연락처 재질문", + "severity": "moderate", + } + ], + "supervisor_rationale": "보라별의 호소를 요약했다.", + "supervisor_critique": "미래학교상담연구랩 언급이 반복됐다.", + "alternative_utterances": ["보라별님, 지금 감정부터 천천히 볼까요?"], + }, + "error": None, + } + + with ( + patch.object( + sessions.session_persistence, + "load_session", + AsyncMock(return_value=sess), + ), + patch.object( + sessions.session_persistence, + "load_case_worksheet", + AsyncMock(return_value=(None, False)), + ), + patch.object( + sessions.session_persistence, + "load_session_evaluation", + AsyncMock(return_value=(raw_record, True)), + ), + ): + response = await sessions.get_session_review(sess.session_id, principal) + + blob = response.model_dump_json() + for raw in ("보라별", "미래학교상담연구랩", "010-1234-5678"): + self.assertNotIn(raw, blob) + for masked in ("[NAME]", "[ORG]", "[PHONE]"): + self.assertIn(masked, blob) + self.assertTrue(response.reviewReady) + self.assertEqual(response.supervisorState, "평가 완료") + async def test_review_surfaces_session_evaluation_error_record(self) -> None: principal = _principal() sess = _session(principal) @@ -1366,6 +1672,53 @@ class SessionTurnPersistenceTest(unittest.IsolatedAsyncioTestCase): self.assertEqual(response.goodMoments, []) self.assertEqual(response.growthPoints, []) + def test_review_marks_stale_missing_session_evaluation_as_failed(self) -> None: + principal = _principal() + sess = _session(principal) + sess.ended = True + sess.ended_at = sess.created_at + 180 + sess.turns.extend( + [ + TurnRecord( + turn_seq=1, + speaker="counselor", + stage=sess.state.stage.value, + text="많이 지쳐 보였어요. 지금 제일 버거운 마음이 뭔가요?", + text_masked="많이 지쳐 보였어요. 지금 제일 버거운 마음이 뭔가요?", + created_at=sess.created_at + 1, + ), + TurnRecord( + turn_seq=2, + speaker="client", + stage=sess.state.stage.value, + text="그냥 아무것도 하고 싶지 않아요.", + text_masked="그냥 아무것도 하고 싶지 않아요.", + created_at=sess.created_at + 2, + ), + ] + ) + previous_timeout = settings.session_evaluation_timeout + settings.session_evaluation_timeout = 10.0 + try: + response = sessions.build_session_review( + sessions.SessionReviewReadInput( + session=sess, + evaluation_record=None, + evaluation_durable=True, + now_ts=(sess.ended_at or sess.created_at) + 41.0, + ) + ) + finally: + settings.session_evaluation_timeout = previous_timeout + + self.assertFalse(response.reviewReady) + self.assertTrue(response.degraded) + self.assertEqual(response.supervisorState, "평가 실패") + self.assertIn("AI 평가 재시도가 필요합니다", response.summary) + self.assertEqual(response.rubric, []) + self.assertEqual(response.goodMoments, []) + self.assertEqual(response.growthPoints, []) + async def test_learner_can_save_case_formulation_worksheet(self) -> None: principal = _principal() sess = _session(principal) diff --git a/apps/api/app/test_teacher_dashboard.py b/apps/api/app/test_teacher_dashboard.py index 14babe7..08cf767 100644 --- a/apps/api/app/test_teacher_dashboard.py +++ b/apps/api/app/test_teacher_dashboard.py @@ -10,11 +10,37 @@ from fastapi import HTTPException from .config import settings from .deps import Principal, Role from .routes import teacher +from .session_read_model import MISSING_SESSION_EVALUATION_ERROR from .services import state_machine from .services.persona import P1 from .store import InProcSession, TurnRecord +class _ListSessionsConn: + def __init__(self) -> None: + self.query = "" + self.args: tuple[object, ...] = () + + async def fetch(self, query: str, *args: object) -> list[object]: + self.query = query + self.args = args + return [] + + async def execute(self, _query: str, *args: object) -> None: + return None + + +class _AcquireCtx: + def __init__(self, conn: _ListSessionsConn) -> None: + self.conn = conn + + async def __aenter__(self) -> _ListSessionsConn: + return self.conn + + async def __aexit__(self, exc_type, exc, tb) -> bool: + return False + + def _principal() -> Principal: return Principal( user_id="00000000-0000-0000-0000-000000000901", @@ -31,6 +57,8 @@ def _session( rapport: float, technique: str, created_at: float, + learner_label: str | None = None, + with_turns: bool = True, ) -> InProcSession: state = state_machine.init_state(params=P1.openness_params()) state.stage = state_machine.Stage.EXPLORE @@ -46,6 +74,7 @@ def _session( created_at=created_at, ended_at=created_at + 600, ended=True, + learner_label=learner_label, turns=[ TurnRecord( turn_seq=1, @@ -66,11 +95,51 @@ def _session( text="내담자 응답", text_masked="내담자 응답", ), - ], + ] + if with_turns + else [], ) class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase): + async def test_list_all_sessions_has_no_recent_limit_for_teacher_summaries(self) -> None: + principal = _principal() + conn = _ListSessionsConn() + + with ( + patch.object(teacher.session_persistence, "get_pool"), + patch.object( + teacher.session_persistence, + "acquire", + return_value=_AcquireCtx(conn), + ), + ): + sessions, durable = await teacher.session_persistence.list_all_sessions(principal) + + self.assertEqual(sessions, []) + self.assertTrue(durable) + self.assertNotIn("LIMIT", conn.query) + self.assertEqual(conn.args, ()) + + async def test_list_recent_sessions_keeps_recent_limit_for_general_lists(self) -> None: + principal = _principal() + conn = _ListSessionsConn() + + with ( + patch.object(teacher.session_persistence, "get_pool"), + patch.object( + teacher.session_persistence, + "acquire", + return_value=_AcquireCtx(conn), + ), + ): + sessions, durable = await teacher.session_persistence.list_recent_sessions(principal) + + self.assertEqual(sessions, []) + self.assertTrue(durable) + self.assertIn("LIMIT $1", conn.query) + self.assertEqual(conn.args, (100,)) + async def test_dashboard_returns_learner_growth_from_turn_evaluations(self) -> None: learner_id = "00000000-0000-0000-0000-000000000111" sessions = [ @@ -78,6 +147,7 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase): session_id="00000000-0000-0000-0000-00000000a111", session_no=1, learner_id=learner_id, + learner_label="김민수", score="neutral", rapport=0.1, technique="reflection", @@ -87,6 +157,7 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase): session_id="00000000-0000-0000-0000-00000000a112", session_no=2, learner_id=learner_id, + learner_label="김민수", score="pos", rapport=0.5, technique="reflection", @@ -98,9 +169,9 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase): with ( patch.object( teacher.session_persistence, - "list_sessions", + "list_all_sessions", AsyncMock(return_value=(sessions, True)), - ) as list_sessions, + ) as list_all_sessions, patch.object( teacher.session_persistence, "list_safety_alerts", @@ -111,13 +182,19 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase): "list_session_review_statuses", AsyncMock(return_value=({}, True)), ), + patch.object( + teacher.session_persistence, + "list_session_evaluations", + AsyncMock(return_value=({}, True)), + ), ): response = await teacher.teacher_dashboard(principal) - list_sessions.assert_awaited_once_with(principal, include_turn_evaluation=True) + list_all_sessions.assert_awaited_once_with(principal, include_turn_evaluation=True) self.assertEqual(response.total_learners, 1) self.assertEqual(len(response.learner_growth), 1) growth = response.learner_growth[0] + self.assertEqual(growth.learner_label, "김민수") self.assertEqual(growth.sessions, 2) self.assertEqual(growth.ended_sessions, 2) self.assertEqual(growth.first_score, 0.5) @@ -126,8 +203,56 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase): self.assertEqual(growth.trend, "up") self.assertEqual(growth.top_techniques, ["reflection"]) self.assertEqual([point.session_no for point in growth.points], [1, 2]) + self.assertEqual(response.recent_sessions[0].learner_label, "김민수") self.assertEqual(len(response.pending_reviews), 2) + async def test_dashboard_does_not_truncate_student_analysis_list(self) -> None: + sessions = [ + _session( + session_id=f"many-learners-session-{idx:02d}", + session_no=1, + learner_id=f"many-learner-{idx:02d}", + learner_label=f"학생 {idx:02d}", + score="pos", + rapport=0.2, + technique="reflection", + created_at=1_000.0 + idx, + ) + for idx in range(13) + ] + principal = _principal() + + with ( + patch.object( + teacher.session_persistence, + "list_all_sessions", + AsyncMock(return_value=(sessions, True)), + ), + patch.object( + teacher.session_persistence, + "list_safety_alerts", + AsyncMock(return_value=([], True)), + ), + patch.object( + teacher.session_persistence, + "list_session_review_statuses", + AsyncMock(return_value=({}, True)), + ), + patch.object( + teacher.session_persistence, + "list_session_evaluations", + AsyncMock(return_value=({}, True)), + ), + ): + response = await teacher.teacher_dashboard(principal) + + self.assertEqual(response.total_learners, 13) + self.assertEqual(len(response.learner_growth), 13) + self.assertEqual( + {item.learner_id for item in response.learner_growth}, + {f"many-learner-{idx:02d}" for idx in range(13)}, + ) + async def test_dashboard_excludes_closed_session_reviews_from_pending_queue(self) -> None: learner_id = "00000000-0000-0000-0000-000000000111" open_session = _session( @@ -153,7 +278,7 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase): with ( patch.object( teacher.session_persistence, - "list_sessions", + "list_all_sessions", AsyncMock(return_value=([open_session, closed_session], True)), ), patch.object( @@ -178,6 +303,11 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase): ) ), ), + patch.object( + teacher.session_persistence, + "list_session_evaluations", + AsyncMock(return_value=({}, True)), + ), ): response = await teacher.teacher_dashboard(principal) @@ -187,6 +317,159 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase): ) self.assertEqual(closed_summary.review_status, "closed") + async def test_dashboard_excludes_empty_ended_sessions_from_pending_queue(self) -> None: + principal = _principal() + empty_session = _session( + session_id="00000000-0000-0000-0000-00000000b411", + session_no=1, + learner_id="00000000-0000-0000-0000-000000000333", + learner_label="하린", + score="neutral", + rapport=0.1, + technique="reflection", + created_at=1_000.0, + with_turns=False, + ) + reviewable_session = _session( + session_id="00000000-0000-0000-0000-00000000b412", + session_no=2, + learner_id="00000000-0000-0000-0000-000000000333", + learner_label="하린", + score="pos", + rapport=0.5, + technique="reflection", + created_at=2_000.0, + ) + + with ( + patch.object( + teacher.session_persistence, + "list_all_sessions", + AsyncMock(return_value=([empty_session, reviewable_session], True)), + ), + patch.object( + teacher.session_persistence, + "list_safety_alerts", + AsyncMock(return_value=([], True)), + ), + patch.object( + teacher.session_persistence, + "list_session_review_statuses", + AsyncMock(return_value=({}, True)), + ), + patch.object( + teacher.session_persistence, + "list_session_evaluations", + AsyncMock(return_value=({}, True)), + ), + ): + response = await teacher.teacher_dashboard(principal) + + self.assertEqual([item.session_id for item in response.pending_reviews], [reviewable_session.session_id]) + empty_summary = next( + item for item in response.recent_sessions if item.session_id == empty_session.session_id + ) + self.assertEqual(empty_summary.turn_count, 0) + self.assertEqual(empty_summary.supervisor_state, "기록 대기") + self.assertFalse(empty_summary.review_ready) + + async def test_dashboard_surfaces_session_evaluation_failure(self) -> None: + principal = _principal() + sess = _session( + session_id="00000000-0000-0000-0000-00000000b311", + session_no=6, + learner_id="00000000-0000-0000-0000-000000000333", + learner_label="하린", + score="neutral", + rapport=0.1, + technique="reflection", + created_at=1_000.0, + ) + evaluation_record = { + "status": "error", + "source": "engine", + "scope": "session_end", + "stage": "정리", + "payload": {}, + "error": "session evaluation timeout after 45s", + "updated_at": "2026-07-01T00:00:00Z", + } + + with ( + patch.object( + teacher.session_persistence, + "list_all_sessions", + AsyncMock(return_value=([sess], True)), + ), + patch.object( + teacher.session_persistence, + "list_safety_alerts", + AsyncMock(return_value=([], True)), + ), + patch.object( + teacher.session_persistence, + "list_session_review_statuses", + AsyncMock(return_value=({}, True)), + ), + patch.object( + teacher.session_persistence, + "list_session_evaluations", + AsyncMock(return_value=({sess.session_id: evaluation_record}, True)), + ), + ): + response = await teacher.teacher_dashboard(principal) + + failed = response.pending_reviews[0] + self.assertEqual(failed.learner_label, "하린") + self.assertEqual(failed.evaluation_status, "error") + self.assertFalse(failed.review_ready) + self.assertEqual(failed.supervisor_state, "평가 실패") + self.assertEqual(failed.evaluation_error, "session evaluation timeout after 45s") + + async def test_dashboard_marks_stale_missing_session_evaluation_as_failed(self) -> None: + principal = _principal() + sess = _session( + session_id="00000000-0000-0000-0000-00000000b411", + session_no=7, + learner_id="00000000-0000-0000-0000-000000000333", + learner_label="하린", + score="neutral", + rapport=0.1, + technique="reflection", + created_at=1_000.0, + ) + + with ( + patch.object( + teacher.session_persistence, + "list_all_sessions", + AsyncMock(return_value=([sess], True)), + ), + patch.object( + teacher.session_persistence, + "list_safety_alerts", + AsyncMock(return_value=([], True)), + ), + patch.object( + teacher.session_persistence, + "list_session_review_statuses", + AsyncMock(return_value=({}, True)), + ), + patch.object( + teacher.session_persistence, + "list_session_evaluations", + AsyncMock(return_value=({}, True)), + ), + ): + response = await teacher.teacher_dashboard(principal) + + failed = response.pending_reviews[0] + self.assertEqual(failed.learner_label, "하린") + self.assertEqual(failed.evaluation_status, "error") + self.assertFalse(failed.review_ready) + self.assertEqual(failed.supervisor_state, "평가 실패") + self.assertEqual(failed.evaluation_error, MISSING_SESSION_EVALUATION_ERROR) + async def test_dashboard_fails_closed_when_safety_alerts_are_not_durable_outside_dev(self) -> None: principal = _principal() sess = _session( @@ -204,7 +487,7 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase): with ( patch.object( teacher.session_persistence, - "list_sessions", + "list_all_sessions", AsyncMock(return_value=([sess], True)), ), patch.object( @@ -217,6 +500,11 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase): "list_session_review_statuses", AsyncMock(return_value=({}, True)), ), + patch.object( + teacher.session_persistence, + "list_session_evaluations", + AsyncMock(return_value=({}, True)), + ), ): with self.assertRaises(HTTPException) as raised: await teacher.teacher_dashboard(principal) @@ -244,6 +532,11 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase): "load_session", AsyncMock(return_value=sess), ), + patch.object( + teacher.session_persistence, + "load_session_evaluation", + AsyncMock(return_value=({"status": "ready"}, True)), + ), patch.object( teacher.session_persistence, "save_session_review_status", @@ -286,6 +579,46 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase): self.assertIn("보호요인", response.worksheet_note) self.assertEqual(response.worksheet_reviewed_at, "2026-06-27T10:05:00Z") + async def test_teacher_cannot_close_review_before_ai_session_evaluation_ready(self) -> None: + principal = _principal() + sess = _session( + session_id="00000000-0000-0000-0000-00000000c211", + session_no=2, + learner_id="00000000-0000-0000-0000-000000000222", + score="pos", + rapport=0.5, + technique="reflection", + created_at=3_000.0, + ) + + with ( + patch.object( + teacher.session_persistence, + "load_session", + AsyncMock(return_value=sess), + ), + patch.object( + teacher.session_persistence, + "load_session_evaluation", + AsyncMock(return_value=({"status": "error", "error": "timeout"}, True)), + ), + patch.object( + teacher.session_persistence, + "save_session_review_status", + AsyncMock(), + ) as save_status, + ): + with self.assertRaises(HTTPException) as raised: + await teacher.update_session_review_status( + sess.session_id, + teacher.TeacherSessionReviewStatusRequest(status="closed", note="완료"), + principal, + ) + + self.assertEqual(raised.exception.status_code, 409) + self.assertIn("session evaluation must be ready", raised.exception.detail) + save_status.assert_not_awaited() + async def test_learner_analysis_returns_full_ordered_history(self) -> None: learner_id = "00000000-0000-0000-0000-000000000111" other_learner_id = "00000000-0000-0000-0000-000000000222" @@ -294,6 +627,7 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase): session_id=f"analysis-session-{idx}", session_no=idx, learner_id=learner_id, + learner_label="이서연", score="pos" if idx >= 5 else "neutral", rapport=0.1 * idx, technique="reflection" if idx % 2 else "open question", @@ -319,9 +653,9 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase): with ( patch.object( teacher.session_persistence, - "list_sessions", + "list_all_sessions", AsyncMock(return_value=(sessions, True)), - ) as list_sessions, + ) as list_all_sessions, patch.object( teacher.session_persistence, "list_session_review_statuses", @@ -339,17 +673,25 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase): ) ), ), + patch.object( + teacher.session_persistence, + "list_session_evaluations", + AsyncMock(return_value=({}, True)), + ), ): response = await teacher.learner_analysis(learner_id, principal) - list_sessions.assert_awaited_once_with(principal, include_turn_evaluation=True) + list_all_sessions.assert_awaited_once_with(principal, include_turn_evaluation=True) self.assertEqual(response.source, "database") self.assertEqual(response.learner_id, learner_id) + self.assertEqual(response.learner_label, "이서연") + self.assertEqual(response.summary.learner_label, "이서연") self.assertEqual(response.total_sessions, 7) self.assertEqual(response.ended_sessions, 7) self.assertEqual(response.pending_reviews, 6) self.assertEqual(response.closed_reviews, 1) self.assertEqual([session.session_no for session in response.sessions], [1, 2, 3, 4, 5, 6, 7]) + self.assertTrue(all(session.learner_label == "이서연" for session in response.sessions)) self.assertEqual([point.session_no for point in response.points], [1, 2, 3, 4, 5, 6, 7]) self.assertEqual(response.sessions[1].review_status, "closed") explore = next(item for item in response.stage_breakdown if item.stage == "탐색") @@ -360,7 +702,7 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase): principal = _principal() with patch.object( teacher.session_persistence, - "list_sessions", + "list_all_sessions", AsyncMock(return_value=([], True)), ): with self.assertRaises(HTTPException) as raised: diff --git a/apps/api/app/test_voice_ws.py b/apps/api/app/test_voice_ws.py index 45d1952..03deb2f 100644 --- a/apps/api/app/test_voice_ws.py +++ b/apps/api/app/test_voice_ws.py @@ -7,6 +7,8 @@ import unittest from types import SimpleNamespace from unittest.mock import AsyncMock, patch +from fastapi import HTTPException + from .deps import Principal, Role from .persona_repository import PersonaVoiceMap from .routes import voice as voice_routes @@ -17,13 +19,20 @@ SESSION_ID = "voice-ws-contract-session" VOICE_PRESET = VoicePreset(preset="neutral", openai_voice="sage") -def _principal(role: Role = Role.LEARNER) -> Principal: +def _principal( + role: Role = Role.LEARNER, + *, + consent_at: float | None = 1.0, + profile_completed_at: float | None = 1.0, +) -> Principal: return Principal( user_id="00000000-0000-0000-0000-000000000201", role=role, cohort_ids=[], email=f"voice-ws-{role.value}@hs.ac.kr", display_name="Voice WS Contract", + consent_at=consent_at if role == Role.LEARNER else None, + profile_completed_at=profile_completed_at if role == Role.LEARNER else None, ) @@ -318,6 +327,55 @@ class VoiceWebSocketContractTest(unittest.IsolatedAsyncioTestCase): self.assertEqual(kwargs["voice_preset"], VOICE_PRESET) self.assertEqual(kwargs["learner_text"], "I need help practicing.") + async def test_turn_persistence_failure_uses_structured_voice_error(self) -> None: + websocket = FakeWebSocket( + [ + _control({"type": "text_turn", "text": "I need help practicing."}), + _control({"type": "close"}), + ] + ) + run_turn = AsyncMock( + side_effect=HTTPException( + status_code=503, + detail=( + "voice session turn append persistence unavailable; " + "runtime fallback is disabled in prod" + ), + ) + ) + + with patch.object( + voice_routes, + "_principal_from_websocket", + AsyncMock(return_value=_principal()), + ), patch.object( + voice_routes, + "_bind_session", + AsyncMock(return_value=self._bind_result()), + ), patch.object( + voice_routes.voice_service, + "is_available", + return_value=True, + ), patch.object( + voice_routes, + "_run_turn_and_speak", + run_turn, + ): + await voice_routes.voice_ws(websocket) # type: ignore[arg-type] + + self.assertEqual(websocket.close_codes, [1000]) + self.assertEqual(websocket.sent_json[0].get("type"), "ready") + self.assertEqual(websocket.sent_json[0].get("state"), "idle") + self.assertEqual( + websocket.sent_json[-2], + { + "type": "error", + "code": "turn_persistence_unavailable", + "detail": "voice turn persistence unavailable; retry the utterance", + }, + ) + self.assertEqual(websocket.sent_json[-1], {"type": "state", "state": "idle"}) + async def test_stt_result_waits_for_final_transcript_before_running_turn(self) -> None: websocket = FakeWebSocket( [ @@ -606,6 +664,60 @@ class VoiceWebSocketContractTest(unittest.IsolatedAsyncioTestCase): )) get_voice_map.assert_awaited_once_with(SESSION_ID) + async def test_bind_session_rejects_existing_session_without_onboarding(self) -> None: + websocket = FakeWebSocket() + websocket.query_params = {"session_id": SESSION_ID} + load_voice_session = AsyncMock() + + with patch.object( + voice_routes, + "user_onboarding_complete", + AsyncMock(return_value=False), + ), patch.object( + voice_routes, + "user_has_consent", + AsyncMock(return_value=True), + ), patch.object( + voice_routes, + "_load_voice_session", + load_voice_session, + ): + session_id, voice, err, meta = await voice_routes._bind_session( + websocket, + _principal(profile_completed_at=None), + ) + + self.assertIsNone(session_id) + self.assertIsNone(voice) + self.assertEqual(err, "onboarding_required") + self.assertEqual(meta, {}) + load_voice_session.assert_not_awaited() + + async def test_bind_session_rejects_existing_session_without_consent(self) -> None: + websocket = FakeWebSocket() + websocket.query_params = {"session_id": SESSION_ID} + load_voice_session = AsyncMock() + + with patch.object( + voice_routes, + "user_has_consent", + AsyncMock(return_value=False), + ), patch.object( + voice_routes, + "_load_voice_session", + load_voice_session, + ): + session_id, voice, err, meta = await voice_routes._bind_session( + websocket, + _principal(consent_at=None), + ) + + self.assertIsNone(session_id) + self.assertIsNone(voice) + self.assertEqual(err, "consent_required") + self.assertEqual(meta, {}) + load_voice_session.assert_not_awaited() + async def test_bind_session_explicit_preset_overrides_db_voice_map(self) -> None: websocket = FakeWebSocket() websocket.query_params = {"session_id": SESSION_ID, "preset": "soft-young-fem"} diff --git a/apps/api/app/turn_runtime.py b/apps/api/app/turn_runtime.py index 4a57e11..9aba28e 100644 --- a/apps/api/app/turn_runtime.py +++ b/apps/api/app/turn_runtime.py @@ -19,6 +19,9 @@ from .store import InProcSession, TurnRecord, store logger = logging.getLogger(__name__) +_LIVE_COACH_RECHARGE_MIN_RAPPORT = 0.35 +_LIVE_COACH_RECHARGE_MIN_OPENNESS_GAIN = 0.02 + class SessionAccessError(str, Enum): NOT_FOUND = "not_found" @@ -179,6 +182,63 @@ async def record_safety_event( require_runtime_fallback_allowed("safety event") +def should_recharge_live_coach_credit( + evaluation: dict | None, + before: state_machine.SessionState, + after: state_machine.SessionState, +) -> tuple[bool, str]: + """Good-score recharge gate based on stored evaluator/state-machine evidence.""" + if not isinstance(evaluation, dict): + return False, "" + if evaluation.get("appropriateness") != "pos": + return False, "" + try: + rapport = float(evaluation.get("rapport_signal") or 0) + except (TypeError, ValueError): + rapport = 0.0 + if rapport < _LIVE_COACH_RECHARGE_MIN_RAPPORT: + return False, "" + + openness_gain = float(after.effective_openness or 0) - float(before.effective_openness or 0) + stage_changed = after.stage != before.stage + if not stage_changed and openness_gain < _LIVE_COACH_RECHARGE_MIN_OPENNESS_GAIN: + return False, "" + if stage_changed: + return True, "좋은 발화로 내담자 단계가 열려 코칭 기회 1개를 충전했습니다." + return True, "좋은 발화 뒤 내담자 개방도가 올라 코칭 기회 1개를 충전했습니다." + + +async def maybe_recharge_live_coach_credit( + sess: InProcSession, + ctx: orchestrator.TurnContext, + result: orchestrator.TurnResult, +) -> None: + """Record one live-coach recharge when a strong learner turn changes client state.""" + assert ctx.state_after is not None + should_recharge, reason = should_recharge_live_coach_credit( + result.evaluation, + ctx.state_before, + result.state_after, + ) + if not should_recharge: + return + try: + await session_persistence.record_live_coach_recharge( + session_id=sess.session_id, + learner_id=sess.learner_id, + turn_seq=result.turn_seq, + stage=stage_label(result.state_after.stage), + reason=reason, + ) + except Exception: + logger.exception( + "live coach recharge persistence failed: session_id=%s turn_seq=%s", + sess.session_id, + result.turn_seq, + ) + require_runtime_fallback_allowed("live coach recharge") + + async def finalize_completed_turn( sess: InProcSession, ctx: orchestrator.TurnContext, @@ -195,6 +255,7 @@ async def finalize_completed_turn( context_prefix=context_prefix, counselor_turn=counselor_turn, ) + await maybe_recharge_live_coach_credit(sess, ctx, result) await record_safety_event(sess, ctx, result) @@ -203,8 +264,10 @@ __all__ = [ "append_completed_turn", "finalize_completed_turn", "load_owned_session", + "maybe_recharge_live_coach_credit", "record_safety_event", "record_completed_turn", + "should_recharge_live_coach_credit", "stage_label", "update_session_state", ] diff --git a/apps/web/e2e/README.md b/apps/web/e2e/README.md index 1c8ac02..36dcd99 100644 --- a/apps/web/e2e/README.md +++ b/apps/web/e2e/README.md @@ -1,8 +1,10 @@ # Playwright E2E -These tests exercise the app through the Vite `/api` proxy and a running local -FastAPI server. They do not replace `/auth/me`, `/personas`, sessions, admin, or -review endpoints with Playwright route fixtures. +This directory contains both full-stack E2E tests that exercise the app through +the Vite `/api` proxy and a running local FastAPI server, and focused UI +regression tests that pin states with Playwright route fixtures. Treat a test as +DB-backed/real-API evidence only when the spec does not fulfill the endpoint +being verified and asserts the actual API response or persisted read model. Required local services: @@ -22,6 +24,14 @@ PLAYWRIGHT_BASE_URL=http://localhost:5173 npm run e2e VITE_API_BASE=http://127.0.0.1:8000 npm run e2e ``` +Feature evidence map: + +- `session-persistence.spec.ts` is the primary DB-backed evidence for session runtime flows. It covers browser `openSessionStream()` persistence, Korean PII masking through the browser stream into DB-backed detail/review payloads, crisis learner-only safety persistence without a client AI reply, AI tutor live coach history, source-pack metadata round-trip, voice metadata persistence, learner worksheet/review persistence, session-end evaluation storage, explicit teacher session/turn reevaluation, and manual teacher UI evaluation retry from a real failed row into durable DB state. The AI tutor history test must see `status=ready` and `latency_ms>0`; degraded fallback must not pass as normal engine-backed coaching. +- `kb-source-packs.spec.ts` is DB-backed source-pack sync evidence. It checks admin-only sync and source-scoped evaluator RAG lookup for the licensed source packs; evaluator retrieval 503 is a failure, not a skipped proof. +- `session-review.spec.ts` is route-fixture UI regression evidence for review states, including delayed `평가 대기` to `평가 완료` polling, `평가 실패`, `AI 평가 재시도`, and pre/post input validation states. It does not prove that the evaluator wrote a DB row unless paired with `session-persistence.spec.ts`. +- `teacher.spec.ts` mixes DB-backed teacher console paths with route-fixture queue/readability checks. Use the individual test body before citing it as persisted evidence. +- `session-mvp.spec.ts` is mostly route-fixture UI regression evidence. The AI tutor tests mock normal coaching, stale quota refresh, quota exhaustion over an old card, degraded fallback, history load failure, and runtime persistence source to prove the UI does not show replacement coaching, missing history, stale credits, or temporary storage as a silent success; the voice tests mock the browser microphone/WebSocket to prove both a final transcript followed by `turn_persistence_unavailable` and a voice `conversation_stopped` crisis reply remain visible as failure/safety states instead of successful client turns. These are not DB-backed engine success evidence. + Public Google OAuth `/turn` smoke: ```sh diff --git a/apps/web/e2e/admin.spec.ts b/apps/web/e2e/admin.spec.ts index 71e15c8..a5dacba 100644 --- a/apps/web/e2e/admin.spec.ts +++ b/apps/web/e2e/admin.spec.ts @@ -137,6 +137,7 @@ interface AdminTicketsResponse { open_count: number; high_priority_count: number; stale_count: number; + by_category: Record; }; } @@ -146,6 +147,134 @@ async function expectResponseOk(response: APIResponse | Response) { } } +async function mockApprovedAdminWithoutOnboarding(page: Page) { + const seenAdminEndpoints = new Set(); + const json = (body: unknown) => JSON.stringify(body); + + await page.route("**/api/**", async (route) => { + const request = route.request(); + const url = new URL(request.url()); + const method = request.method(); + const path = url.pathname; + const fulfillJson = (body: unknown, status = 200) => + route.fulfill({ + status, + contentType: "application/json", + body: json(body), + }); + + if (method === "GET" && path.endsWith("/auth/me")) { + await fulfillJson({ + user_id: "stale-admin", + email: "stale-admin@twentyoz.kr", + display_name: "Stale Admin", + role: "admin", + admin_access: false, + super_admin: false, + account_status: "approved", + approval_required: false, + cohort_ids: [], + consent_at: null, + onboarding_completed_at: null, + nickname: "", + self_introduction: "", + avatar_url: "", + }); + return; + } + + if (method === "GET" && path.endsWith("/admin/health")) { + seenAdminEndpoints.add("health"); + await fulfillJson({ + status: "degraded", + environment: "dev", + engine_mode: "openai", + services: [], + }); + return; + } + + if (method === "GET" && path.endsWith("/admin/users")) { + seenAdminEndpoints.add("users"); + await fulfillJson({ + source: "database", + durable: true, + users: [], + }); + return; + } + + if (method === "GET" && path.endsWith("/admin/usage")) { + seenAdminEndpoints.add("usage"); + await fulfillJson({ + source: "database", + durable: true, + window_days: 7, + total_turns: 0, + metered_turns: 0, + tokens_in: 0, + tokens_out: 0, + cost_usd: 0, + budget: { + limit_usd: 0, + used_ratio: 0, + remaining_usd: null, + status: "disabled", + }, + evaluator_cache: { + enabled: false, + entries: 0, + hits: 0, + misses: 0, + stores: 0, + evictions: 0, + requests: 0, + hit_rate: 0, + }, + by_provider: [], + daily_cost: [], + }); + return; + } + + if (method === "GET" && path.endsWith("/admin/uptime")) { + seenAdminEndpoints.add("uptime"); + await fulfillJson({ + source: "database", + durable: true, + window_hours: 24, + sample_count: 0, + ok_ratio: 0, + degraded_events: 0, + down_events: 0, + last_down_at: null, + }); + return; + } + + if (method === "GET" && path.endsWith("/admin/tickets")) { + seenAdminEndpoints.add("tickets"); + await fulfillJson({ + source: "database", + durable: true, + tickets: [], + summary: { + total: 0, + open_count: 0, + high_priority_count: 0, + stale_count: 0, + by_category: {}, + }, + }); + return; + } + + await fulfillJson({ detail: `unmocked API request: ${method} ${path}` }, 404); + }); + + return seenAdminEndpoints; +} + async function signInAsAdmin(page: Page) { let res: APIResponse | null = null; for (let attempt = 0; attempt < 3; attempt += 1) { @@ -451,6 +580,21 @@ async function expectVisibleButtonsFit(page: Page, selector: string, context: st expect(clippedButtons, `${context}: ${JSON.stringify(clippedButtons)}`).toEqual([]); } +test.describe("admin route guards", () => { + test("allows an approved admin without onboarding to open the admin console", async ({ page }) => { + const seenAdminEndpoints = await mockApprovedAdminWithoutOnboarding(page); + + await page.goto("/admin"); + + await expect(page).toHaveURL(/\/admin$/); + await expect(page.getByRole("heading", { name: "현재 서비스 상태" })).toBeVisible(); + await expect(page.locator(".ad-status")).toContainText("개발"); + await expect + .poll(() => Array.from(seenAdminEndpoints).sort()) + .toEqual(["health", "tickets", "uptime", "usage", "users"]); + }); +}); + test.describe("admin route", () => { test.beforeEach(async ({ page }) => { await useRealApi(page); diff --git a/apps/web/e2e/kb-source-packs.spec.ts b/apps/web/e2e/kb-source-packs.spec.ts new file mode 100644 index 0000000..51ef670 --- /dev/null +++ b/apps/web/e2e/kb-source-packs.spec.ts @@ -0,0 +1,117 @@ +import { expect, test, type APIResponse, type Page } from "@playwright/test"; +import { completeOnboarding } from "./support"; + +interface HealthResponse { + db?: boolean; +} + +interface SourcePackSyncItem { + source_id: string; + doc_id: number | null; + chunks_indexed: number; + skipped_unchanged: boolean; + embedded: boolean; + degraded: boolean; +} + +interface SourcePackSyncResponse { + sources_upserted: number; + chunks_indexed: number; + skipped_unchanged: number; + embedded: boolean; + degraded: boolean; + items: SourcePackSyncItem[]; +} + +interface EvalGroundingResponse { + chunks: Array<{ + source_id?: string | null; + }>; + policy: string; +} + +async function expectResponseOk(response: APIResponse) { + if (!response.ok()) { + expect(response.ok(), await response.text()).toBeTruthy(); + } +} + +async function signInForSourcePackSync(page: Page, role: "admin" | "learner") { + const response = await page.request.post("/api/auth/dev-login", { + data: { + email: role === "admin" ? "source-pack-admin@twentyoz.kr" : "source-pack-learner@hs.ac.kr", + role, + display_name: role === "admin" ? "Source Pack Admin" : "Source Pack Learner", + }, + }); + await expectResponseOk(response); + await completeOnboarding(page, { + legal_name: role === "admin" ? "Source Pack Admin" : "Source Pack Learner", + affiliation: "한신대학교", + department: role === "admin" ? "운영" : "상담심리학과", + grade_level: role === "admin" ? "관리자" : "3학년", + phone: role === "admin" ? "010-3333-3333" : "010-4444-4444", + contact_address: "경기도 오산시 한신대학교", + }); +} + +test.describe("live coach source packs", () => { + test("syncs licensed source packs into DB-backed evaluator RAG with admin-only access @single-run", async ({ + page, + }) => { + test.setTimeout(180_000); + + const healthResponse = await page.request.get("/api/health"); + await expectResponseOk(healthResponse); + const health = (await healthResponse.json()) as HealthResponse; + test.skip(!health.db, "source pack sync requires DB"); + + const anonymousSync = await page.request.post("/api/kb/live-coach/source-packs/sync"); + expect(anonymousSync.status()).toBe(401); + + await signInForSourcePackSync(page, "learner"); + const learnerSync = await page.request.post("/api/kb/live-coach/source-packs/sync"); + expect(learnerSync.status()).toBe(403); + + await signInForSourcePackSync(page, "admin"); + const syncResponse = await page.request.post("/api/kb/live-coach/source-packs/sync"); + expect(syncResponse.status(), await syncResponse.text()).toBe(202); + const sync = (await syncResponse.json()) as SourcePackSyncResponse; + const sourceIds = new Set(sync.items.map((item) => item.source_id)); + + expect(sync.sources_upserted).toBeGreaterThanOrEqual(4); + expect(sync.items.length).toBeGreaterThanOrEqual(4); + expect(sourceIds).toContain("workbook_0615_case_conceptualization"); + expect(sourceIds).toContain("dsm5tr_case_formulation"); + expect(sourceIds).toContain("official_counseling_guideline_seed"); + expect(sourceIds).toContain("official_suicide_risk_guidelines"); + expect(sync.chunks_indexed).toBeGreaterThanOrEqual(0); + expect(sync.skipped_unchanged).toBeGreaterThanOrEqual(0); + expect(typeof sync.embedded).toBe("boolean"); + expect(typeof sync.degraded).toBe("boolean"); + for (const item of sync.items) { + expect(typeof item.source_id).toBe("string"); + expect(item.source_id.length).toBeGreaterThan(0); + expect(item.doc_id === null || typeof item.doc_id === "number").toBe(true); + expect(item.chunks_indexed).toBeGreaterThanOrEqual(0); + expect(typeof item.skipped_unchanged).toBe("boolean"); + expect(typeof item.embedded).toBe("boolean"); + expect(typeof item.degraded).toBe("boolean"); + expect(item.skipped_unchanged || item.chunks_indexed > 0).toBe(true); + } + + const groundingResponse = await page.request.post("/api/kb/eval-grounding", { + data: { + query: "안전계획 보호요인 위기전화", + k: 3, + rerank: false, + source_id: ["official_suicide_risk_guidelines"], + }, + }); + expect(groundingResponse.status(), await groundingResponse.text()).toBe(200); + const grounding = (await groundingResponse.json()) as EvalGroundingResponse; + expect(grounding.policy).toMatch(/^evaluator:/); + expect(grounding.chunks.length).toBeGreaterThan(0); + expect(grounding.chunks.every((chunk) => chunk.source_id === "official_suicide_risk_guidelines")).toBe(true); + }); +}); diff --git a/apps/web/e2e/layout-visual-gate.spec.ts b/apps/web/e2e/layout-visual-gate.spec.ts index a2c9668..96ecbf5 100644 --- a/apps/web/e2e/layout-visual-gate.spec.ts +++ b/apps/web/e2e/layout-visual-gate.spec.ts @@ -19,7 +19,7 @@ import { /** * Strict visual layout gate. * - * The layout-redesign handoff (docs/ops/layout-redesign-handoff-2026-06-26.md) + * The layout-redesign handoff (docs/archive/ops/layout-redesign-handoff-2026-06-26.md) * required a human-style visual acceptance pass across the redesigned screens at * the suggested breakpoints. This spec hardens that pass into an automated gate: * every redesigned screen is rendered at every required width, asserted free of @@ -607,8 +607,14 @@ test.describe("layout visual gate @single-run", () => { await expect(page.locator('[data-learner-analysis-panel="true"]')).toBeVisible({ timeout: 15_000, }); - await page.getByRole("tab", { name: /^전체 회기/ }).click(); - await expect(page.locator('[data-learner-session-row="true"]')).toHaveCount(5); + await expect(page.locator('[data-learner-persona-group="true"]')).toHaveCount(2); + const firstPersona = page.locator('[data-learner-persona-group="true"]').first(); + const expandButton = firstPersona.getByRole("button", { name: /회기 (펼치기|접기)/ }); + if ((await expandButton.getAttribute("aria-expanded")) !== "true") { + await expandButton.click(); + } + await expect(page.locator('[data-learner-persona-sessions="true"]')).toBeVisible(); + await expect(page.locator('[data-learner-session-row="true"]').first()).toBeVisible(); }); }); diff --git a/apps/web/e2e/session-mvp.spec.ts b/apps/web/e2e/session-mvp.spec.ts index c03fb7f..a7378cd 100644 --- a/apps/web/e2e/session-mvp.spec.ts +++ b/apps/web/e2e/session-mvp.spec.ts @@ -4,6 +4,18 @@ const sessionId = "33333333-3333-4333-8333-333333333333"; const learnerText = "요즘 많이 힘들었겠어요. 어떤 마음이 가장 크게 남아 있나요?"; const clientReply = "괜찮아요. 천천히 말해볼게요."; +declare global { + interface Window { + __voiceErrorFixture?: { + releaseError: () => void; + sent: string[]; + }; + __voiceCrisisFixture?: { + sent: string[]; + }; + } +} + function deferred() { let resolve!: () => void; const promise = new Promise((next) => { @@ -16,11 +28,19 @@ interface RouteMvpOptions { endStatus?: number; endBody?: unknown; crisisStream?: boolean; + liveCoachStatus?: "ready" | "degraded"; + liveCoachHistoryStatus?: number; + liveCoachHistorySource?: "database" | "runtime"; + liveCoachPersistenceSource?: "database" | "runtime"; + liveCoachHistoryQuotas?: Array<{ remaining: number; max: number }>; + liveCoachSuggestionQuota?: { remaining: number; max: number }; } async function routeMvpApi(page: Page, options: RouteMvpOptions = {}) { const sessionStartRequests: unknown[] = []; const liveCoachRequests: unknown[] = []; + let liveCoachHistoryRequests = 0; + let deliveredCoachSuggestion: Record | null = null; const streamSeen = deferred(); const streamGate = deferred(); @@ -62,6 +82,14 @@ async function routeMvpApi(page: Page, options: RouteMvpOptions = {}) { }); }); + await page.route("**/api/voice/health", async (route) => { + await route.fulfill({ + status: 200, + contentType: "application/json", + body: JSON.stringify({ available: true, reason: null }), + }); + }); + await page.route("**/api/personas", async (route) => { await route.fulfill({ status: 200, @@ -198,38 +226,76 @@ async function routeMvpApi(page: Page, options: RouteMvpOptions = {}) { await page.route(`**/api/sessions/${sessionId}/live-coach`, async (route) => { const request = route.request(); if (request.method() === "GET") { + const quota = + options.liveCoachHistoryQuotas?.[ + Math.min(liveCoachHistoryRequests, options.liveCoachHistoryQuotas.length - 1) + ] ?? { remaining: 3, max: 3 }; + liveCoachHistoryRequests += 1; + if (options.liveCoachHistoryStatus && options.liveCoachHistoryStatus >= 400) { + await route.fulfill({ + status: options.liveCoachHistoryStatus, + contentType: "application/json", + body: JSON.stringify({ detail: "live coach history unavailable" }), + }); + return; + } await route.fulfill({ status: 200, contentType: "application/json", - body: JSON.stringify({ source: "runtime", events: [] }), + body: JSON.stringify({ + source: options.liveCoachHistorySource ?? "database", + quota, + credit_events: [], + events: deliveredCoachSuggestion + ? [ + { + event_id: "fixture-coach-1", + session_id: sessionId, + turn_seq: 1, + stage: "탐색", + created_at: "2026-06-26T00:00:03.000Z", + learner_text_excerpt: learnerText, + client_reply_excerpt: clientReply, + suggestion: deliveredCoachSuggestion, + }, + ] + : [], + }), }); return; } liveCoachRequests.push(request.postDataJSON()); + deliveredCoachSuggestion = { + status: options.liveCoachStatus ?? "ready", + tone: "pos", + focus: "reflection", + title: "감정 반영이 선명합니다", + message: "학습자가 내담자의 감정을 평가하지 않고 먼저 되짚었습니다.", + next_utterance: "그 마음이 가장 크게 올라온 장면을 조금 더 들려줄 수 있을까요?", + rationale: + options.liveCoachStatus === "degraded" + ? "AI 코칭 엔진 응답 대신 워크북 규칙과 현재 턴 신호로 만든 대체 판단입니다." + : "방금 발화는 정서 반영과 개방 질문을 함께 포함합니다.", + sources: [ + { + source_id: "live_coaching_workbook_0615", + title: "0615 사례개념화 워크북", + locator: "reflection", + kb_kind: "source_pack", + version: "2026-06-15", + citation: "허가된 요약 근거", + }, + ], + safety_note: null, + latency_ms: options.liveCoachStatus === "degraded" ? 0 : 12, + persistence_source: options.liveCoachPersistenceSource ?? "database", + quota: options.liveCoachSuggestionQuota ?? { remaining: 2, max: 3 }, + credit_events: [], + }; await route.fulfill({ status: 200, contentType: "application/json", - body: JSON.stringify({ - status: "ready", - tone: "pos", - focus: "reflection", - title: "감정 반영이 선명합니다", - message: "학습자가 내담자의 감정을 평가하지 않고 먼저 되짚었습니다.", - next_utterance: "그 마음이 가장 크게 올라온 장면을 조금 더 들려줄 수 있을까요?", - rationale: "방금 발화는 정서 반영과 개방 질문을 함께 포함합니다.", - sources: [ - { - source_id: "live_coaching_workbook_0615", - title: "0615 사례개념화 워크북", - locator: "reflection", - kb_kind: "source_pack", - version: "2026-06-15", - citation: "허가된 요약 근거", - }, - ], - safety_note: null, - latency_ms: 12, - }), + body: JSON.stringify(deliveredCoachSuggestion), }); }); @@ -290,7 +356,277 @@ async function routeMvpApi(page: Page, options: RouteMvpOptions = {}) { }); }); - return { sessionStartRequests, liveCoachRequests, streamGate, streamSeen }; + return { + sessionStartRequests, + liveCoachRequests, + streamGate, + streamSeen, + get liveCoachHistoryRequests() { + return liveCoachHistoryRequests; + }, + }; +} + +async function installVoicePersistenceFailureFixture(page: Page, transcriptText: string) { + await page.addInitScript((text) => { + const fixture = { + sent: [] as string[], + socket: null as FixtureWebSocket | null, + releaseError() { + const socket = fixture.socket; + if (!socket || socket.readyState >= FixtureWebSocket.CLOSING) return; + socket.emitJson({ + type: "error", + code: "turn_persistence_unavailable", + detail: "voice turn persistence unavailable; retry the utterance", + }); + socket.close(1000); + }, + }; + window.__voiceErrorFixture = fixture; + + const fakeTrack = { + kind: "audio", + readyState: "live", + stop() { + this.readyState = "ended"; + }, + }; + const fakeStream = { + active: true, + getTracks: () => [fakeTrack], + getAudioTracks: () => [fakeTrack], + }; + Object.defineProperty(navigator, "mediaDevices", { + configurable: true, + value: { getUserMedia: async () => fakeStream }, + }); + Object.defineProperty(window, "AudioContext", { configurable: true, value: undefined }); + + class FakeMediaRecorder extends EventTarget { + static isTypeSupported() { + return true; + } + + state = "inactive"; + mimeType = "audio/webm"; + ondataavailable: ((event: Event & { data: Blob }) => void) | null = null; + onstop: ((event: Event) => void) | null = null; + + constructor(_stream: unknown, options?: { mimeType?: string }) { + super(); + this.mimeType = options?.mimeType ?? "audio/webm"; + } + + start() { + this.state = "recording"; + } + + stop() { + if (this.state === "inactive") return; + this.state = "inactive"; + const event = new Event("stop"); + this.onstop?.(event); + this.dispatchEvent(event); + } + } + Object.defineProperty(window, "MediaRecorder", { + configurable: true, + value: FakeMediaRecorder, + }); + + class FixtureWebSocket { + static CONNECTING = 0; + static OPEN = 1; + static CLOSING = 2; + static CLOSED = 3; + + readyState = FixtureWebSocket.CONNECTING; + binaryType: BinaryType = "blob"; + onopen: ((event: Event) => void) | null = null; + onmessage: ((event: MessageEvent) => void) | null = null; + onerror: ((event: Event) => void) | null = null; + onclose: ((event: CloseEvent) => void) | null = null; + + constructor(_url: string | URL) { + fixture.socket = this; + window.setTimeout(() => { + if (this.readyState !== FixtureWebSocket.CONNECTING) return; + this.readyState = FixtureWebSocket.OPEN; + this.onopen?.(new Event("open")); + this.emitJson({ type: "ready", state: "idle" }); + }, 0); + } + + send(data: string | ArrayBufferLike | Blob | ArrayBufferView) { + if (typeof data !== "string") return; + fixture.sent.push(data); + let payload: { type?: string } = {}; + try { + payload = JSON.parse(data) as { type?: string }; + } catch { + return; + } + if (payload.type === "audio_end") { + window.setTimeout(() => { + this.emitJson({ type: "state", state: "thinking" }); + this.emitJson({ type: "transcript", text, final: true }); + }, 0); + } + } + + close(code = 1000) { + if (this.readyState === FixtureWebSocket.CLOSED) return; + this.readyState = FixtureWebSocket.CLOSED; + const event = new Event("close") as CloseEvent; + Object.defineProperty(event, "code", { value: code }); + this.onclose?.(event); + } + + emitJson(payload: unknown) { + if (this.readyState === FixtureWebSocket.CLOSED) return; + const event = new MessageEvent("message", { data: JSON.stringify(payload) }); + this.onmessage?.(event); + } + } + Object.defineProperty(window, "WebSocket", { + configurable: true, + value: FixtureWebSocket as unknown as typeof WebSocket, + }); + }, transcriptText); +} + +async function installVoiceCrisisFixture(page: Page, transcriptText: string) { + await page.addInitScript((text) => { + const crisisResource = { + title: "자살예방상담전화 109", + number: "109", + message: "지금은 연습을 멈추고 실제 안전 확인이 먼저입니다.", + }; + const fixture = { + sent: [] as string[], + socket: null as FixtureWebSocket | null, + }; + window.__voiceCrisisFixture = fixture; + + const fakeTrack = { + kind: "audio", + readyState: "live", + stop() { + this.readyState = "ended"; + }, + }; + const fakeStream = { + active: true, + getTracks: () => [fakeTrack], + getAudioTracks: () => [fakeTrack], + }; + Object.defineProperty(navigator, "mediaDevices", { + configurable: true, + value: { getUserMedia: async () => fakeStream }, + }); + Object.defineProperty(window, "AudioContext", { configurable: true, value: undefined }); + + class FakeMediaRecorder extends EventTarget { + static isTypeSupported() { + return true; + } + + state = "inactive"; + mimeType = "audio/webm"; + ondataavailable: ((event: Event & { data: Blob }) => void) | null = null; + onstop: ((event: Event) => void) | null = null; + + constructor(_stream: unknown, options?: { mimeType?: string }) { + super(); + this.mimeType = options?.mimeType ?? "audio/webm"; + } + + start() { + this.state = "recording"; + } + + stop() { + if (this.state === "inactive") return; + this.state = "inactive"; + const event = new Event("stop"); + this.onstop?.(event); + this.dispatchEvent(event); + } + } + Object.defineProperty(window, "MediaRecorder", { + configurable: true, + value: FakeMediaRecorder, + }); + + class FixtureWebSocket { + static CONNECTING = 0; + static OPEN = 1; + static CLOSING = 2; + static CLOSED = 3; + + readyState = FixtureWebSocket.CONNECTING; + binaryType: BinaryType = "blob"; + onopen: ((event: Event) => void) | null = null; + onmessage: ((event: MessageEvent) => void) | null = null; + onerror: ((event: Event) => void) | null = null; + onclose: ((event: CloseEvent) => void) | null = null; + + constructor(_url: string | URL) { + fixture.socket = this; + window.setTimeout(() => { + if (this.readyState !== FixtureWebSocket.CONNECTING) return; + this.readyState = FixtureWebSocket.OPEN; + this.onopen?.(new Event("open")); + this.emitJson({ type: "ready", state: "idle" }); + }, 0); + } + + send(data: string | ArrayBufferLike | Blob | ArrayBufferView) { + if (typeof data !== "string") return; + fixture.sent.push(data); + let payload: { type?: string } = {}; + try { + payload = JSON.parse(data) as { type?: string }; + } catch { + return; + } + if (payload.type === "audio_end") { + window.setTimeout(() => { + this.emitJson({ type: "state", state: "thinking" }); + this.emitJson({ type: "transcript", text, final: true }); + this.emitJson({ + type: "reply", + text: "", + safety_flagged: true, + crisis_resource: crisisResource, + conversation_stopped: true, + turn_seq: 1, + }); + this.emitJson({ type: "state", state: "idle" }); + }, 0); + } + } + + close(code = 1000) { + if (this.readyState === FixtureWebSocket.CLOSED) return; + this.readyState = FixtureWebSocket.CLOSED; + const event = new Event("close") as CloseEvent; + Object.defineProperty(event, "code", { value: code }); + this.onclose?.(event); + } + + emitJson(payload: unknown) { + if (this.readyState === FixtureWebSocket.CLOSED) return; + const event = new MessageEvent("message", { data: JSON.stringify(payload) }); + this.onmessage?.(event); + } + } + Object.defineProperty(window, "WebSocket", { + configurable: true, + value: FixtureWebSocket as unknown as typeof WebSocket, + }); + }, transcriptText); } test.describe("P1 MVP core loop", () => { @@ -354,6 +690,14 @@ test.describe("P1 MVP core loop", () => { await expect( page.getByText("학습자가 내담자의 감정을 평가하지 않고 먼저 되짚었습니다."), ).toBeVisible(); + await page.locator(".sx-coach-card").getByRole("button", { name: "근거 보기" }).click(); + const evidenceDialog = page.locator(".sx-coach-modal [role='dialog']"); + await expect(evidenceDialog).toBeVisible(); + await expect(evidenceDialog).toContainText("0615 사례개념화 워크북"); + await expect(evidenceDialog).toContainText("reflection · source_pack · 2026-06-15"); + await expect(evidenceDialog).toContainText("허가된 요약 근거"); + await evidenceDialog.getByRole("button", { name: "닫기" }).click(); + await expect(evidenceDialog).toHaveCount(0); expect(api.liveCoachRequests).toHaveLength(1); expect(api.liveCoachRequests[0]).toMatchObject({ learner_text: learnerText, @@ -362,6 +706,209 @@ test.describe("P1 MVP core loop", () => { }); }); + test("refreshes stale empty AI tutor quota before blocking coaching", async ({ page }) => { + const api = await routeMvpApi(page, { + liveCoachHistoryQuotas: [ + { remaining: 0, max: 3 }, + { remaining: 1, max: 3 }, + { remaining: 0, max: 3 }, + ], + liveCoachSuggestionQuota: { remaining: 0, max: 3 }, + }); + + await page.goto("/learn/session/P1"); + await page.getByRole("button", { name: "회기 시작" }).click(); + await page.getByRole("button", { name: "코칭" }).click(); + await page.getByLabel("학습자 발화 입력").fill(learnerText); + await page.getByRole("button", { name: "보내기" }).click(); + + await api.streamSeen.promise; + api.streamGate.resolve(); + + await expect(page.locator(".sx-coach-card").getByText("감정 반영이 선명합니다")).toBeVisible(); + expect(api.liveCoachRequests).toHaveLength(1); + await expect.poll(() => api.liveCoachHistoryRequests).toBeGreaterThanOrEqual(2); + }); + + test("shows refreshed AI tutor quota exhaustion over a stale coaching card", async ({ + page, + }) => { + const api = await routeMvpApi(page, { + liveCoachHistoryQuotas: [ + { remaining: 0, max: 3 }, + { remaining: 0, max: 3 }, + ], + liveCoachSuggestionQuota: { remaining: 0, max: 3 }, + }); + + await page.goto("/learn/session/P1"); + await page.getByRole("button", { name: "회기 시작" }).click(); + await page.getByRole("button", { name: "코칭" }).click(); + await page.getByLabel("학습자 발화 입력").fill(learnerText); + await page.getByRole("button", { name: "보내기" }).click(); + + await api.streamSeen.promise; + api.streamGate.resolve(); + + const coachCard = page.locator(".sx-coach-card"); + await expect(coachCard.getByText("감정 반영이 선명합니다")).toBeVisible(); + expect(api.liveCoachRequests).toHaveLength(1); + + await page.getByLabel("학습자 발화 입력").fill("그때 마음을 더 자세히 말해줘도 괜찮아요."); + await page.getByRole("button", { name: "보내기" }).click(); + + await expect(coachCard.getByText("코칭 기회를 모두 사용했습니다.")).toBeVisible(); + await expect(coachCard.getByText("감정 반영이 선명합니다")).toHaveCount(0); + expect(api.liveCoachRequests).toHaveLength(1); + }); + + test("surfaces AI tutor history load failure instead of an empty history", async ({ page }) => { + const api = await routeMvpApi(page, { liveCoachHistoryStatus: 503 }); + + await page.goto("/learn/session/P1"); + await page.getByRole("button", { name: "회기 시작" }).click(); + await page.getByRole("button", { name: "코칭" }).click(); + await page.getByLabel("학습자 발화 입력").fill(learnerText); + await page.getByRole("button", { name: "보내기" }).click(); + + await api.streamSeen.promise; + api.streamGate.resolve(); + + const coachCard = page.locator(".sx-coach-card"); + await expect(coachCard.getByText("감정 반영이 선명합니다")).toBeVisible(); + await coachCard.getByRole("button", { name: "이력 보기" }).click(); + + const historyDialog = page.locator(".sx-coach-history [role='dialog']"); + await expect(historyDialog).toBeVisible(); + await expect(historyDialog.getByRole("alert")).toContainText("코칭 이력을 불러오지 못했습니다."); + await expect(historyDialog).not.toContainText("아직 이 턴에 저장된 코칭 이력이 없습니다."); + expect(api.liveCoachRequests).toHaveLength(1); + }); + + test("labels runtime AI tutor persistence as temporary history", async ({ page }) => { + const api = await routeMvpApi(page, { + liveCoachHistorySource: "runtime", + liveCoachPersistenceSource: "runtime", + }); + + await page.goto("/learn/session/P1"); + await page.getByRole("button", { name: "회기 시작" }).click(); + await page.getByRole("button", { name: "코칭" }).click(); + await page.getByLabel("학습자 발화 입력").fill(learnerText); + await page.getByRole("button", { name: "보내기" }).click(); + + await api.streamSeen.promise; + api.streamGate.resolve(); + + const coachCard = page.locator(".sx-coach-card"); + await expect(coachCard.getByText("감정 반영이 선명합니다")).toBeVisible(); + await expect(coachCard).toContainText("임시 이력"); + await expect(coachCard).toContainText("DB에 확정 저장되지 않아"); + + await coachCard.getByRole("button", { name: "이력 보기" }).click(); + const historyDialog = page.locator(".sx-coach-history [role='dialog']"); + await expect(historyDialog).toContainText("임시 저장소 기준"); + expect(api.liveCoachRequests).toHaveLength(1); + }); + + test("marks pending voice transcript as failed when turn persistence fails @single-run", async ({ + page, + }) => { + const failedVoiceText = "요즘 잠을 잘 못 자요."; + await routeMvpApi(page); + await installVoicePersistenceFailureFixture(page, failedVoiceText); + + await page.goto("/learn/session/P1"); + await page.getByRole("button", { name: "회기 시작" }).click(); + const micButton = page.getByRole("button", { name: "마이크 켜기" }); + await expect(micButton).toBeEnabled(); + await micButton.click(); + await page.getByRole("button", { name: "발화 보내기" }).click(); + + const learnerBubble = page.locator(".sx-utt.is-learner").filter({ hasText: failedVoiceText }); + await expect(learnerBubble).toBeVisible(); + await expect(learnerBubble).toHaveClass(/is-partial/); + await expect( + page.locator(".sx-utt.is-thinking").filter({ hasText: "답변을 준비 중입니다." }), + ).toBeVisible(); + + await page.evaluate(() => window.__voiceErrorFixture?.releaseError()); + + await expect(learnerBubble).toBeVisible(); + await expect(learnerBubble).toHaveClass(/is-failed/); + await expect(learnerBubble).not.toHaveClass(/is-partial/); + await expect(learnerBubble.getByText("저장 실패 · 다시 시도하세요.")).toBeVisible(); + await expect(page.locator(".sx-utt.is-thinking")).toHaveCount(0); + await expect(page.getByRole("alert")).toContainText("음성 발화를 저장하지 못했습니다."); + await expect(page.getByLabel("학습자 발화 입력")).toBeEnabled(); + await expect(page.locator(".sx-mic-block__l")).toContainText("마이크 오류"); + }); + + test("keeps crisis safety gate visible for a voice conversation stop", async ({ page }) => { + const crisisText = "죽고 싶다는 생각이 자꾸 들어요."; + const api = await routeMvpApi(page); + await installVoiceCrisisFixture(page, crisisText); + + await page.goto("/learn/session/P1"); + await page.getByRole("button", { name: "회기 시작" }).click(); + await page.getByRole("button", { name: "코칭" }).click(); + const micButton = page.getByRole("button", { name: "마이크 켜기" }); + await expect(micButton).toBeEnabled(); + await micButton.click(); + await page.getByRole("button", { name: "발화 보내기" }).click(); + + const learnerBubble = page.locator(".sx-utt.is-learner").filter({ hasText: crisisText }); + await expect(learnerBubble).toBeVisible(); + await expect(learnerBubble).not.toHaveClass(/is-partial/); + await expect(learnerBubble).not.toHaveClass(/is-failed/); + await expect(page.locator(".sx-crisis-resource").getByText("자살예방상담전화 109")).toBeVisible(); + await expect(page.getByRole("link", { name: "109" })).toBeVisible(); + await expect(page.getByText("내담자 응답 없음")).toHaveCount(0); + await expect(page.locator(".sx-utt.is-client")).toHaveCount(0); + await expect(page.getByLabel("학습자 발화 입력")).toBeDisabled(); + await expect(page.getByRole("button", { name: "마이크 켜기" })).toBeDisabled(); + + const sent = await page.evaluate(() => window.__voiceCrisisFixture?.sent ?? []); + expect(sent.some((payload) => payload.includes("\"audio_end\""))).toBe(true); + expect(sent.some((payload) => payload.includes("\"close\""))).toBe(true); + expect(api.liveCoachRequests).toHaveLength(0); + }); + + test("labels degraded AI tutor fallback as replacement coaching", async ({ page }) => { + const api = await routeMvpApi(page, { liveCoachStatus: "degraded" }); + + await page.goto("/learn/session/P1"); + await page.getByRole("button", { name: "회기 시작" }).click(); + await page.getByRole("button", { name: "코칭" }).click(); + await page.getByLabel("학습자 발화 입력").fill(learnerText); + await page.getByRole("button", { name: "보내기" }).click(); + + await api.streamSeen.promise; + api.streamGate.resolve(); + + const coachCard = page.locator(".sx-coach-card"); + await expect(coachCard).toContainText("대체 코칭"); + await expect(coachCard).toContainText("AI 응답 대체"); + await expect(coachCard).toContainText("AI 코칭 엔진 응답 대신"); + expect(api.liveCoachRequests).toHaveLength(1); + + await coachCard.getByRole("button", { name: "근거 보기" }).click(); + const evidenceDialog = page.locator(".sx-coach-modal [role='dialog']"); + await expect(evidenceDialog).toBeVisible(); + await expect(evidenceDialog).toContainText("AI 코칭 엔진 응답 대신"); + await expect(evidenceDialog).toContainText("0615 사례개념화 워크북"); + await evidenceDialog.getByRole("button", { name: "닫기" }).click(); + await expect(evidenceDialog).toHaveCount(0); + + const coachMark = page.locator(".sx-utt__coach-mark.is-degraded"); + await expect(coachMark).toBeVisible(); + await expect(coachMark).toHaveAttribute("title", /AI 응답 대체/); + await coachMark.click(); + const historyDialog = page.locator(".sx-coach-history [role='dialog']"); + await expect(historyDialog).toContainText("AI 응답 대체"); + await expect(historyDialog).toContainText("0615 사례개념화 워크북"); + }); + test("keeps crisis safety gate visible instead of showing empty client reply @single-run", async ({ page }) => { const api = await routeMvpApi(page, { crisisStream: true }); diff --git a/apps/web/e2e/session-persistence.spec.ts b/apps/web/e2e/session-persistence.spec.ts index 68a1000..57aa1cf 100644 --- a/apps/web/e2e/session-persistence.spec.ts +++ b/apps/web/e2e/session-persistence.spec.ts @@ -1,5 +1,11 @@ import { expect, test, type Page } from "@playwright/test"; -import { fetchAvailablePersona, signInAsLearner, signInAsTeacher } from "./support"; +import { + fetchAvailablePersona, + signInAsAdmin, + signInAsLearner, + signInAsTeacher, + withGlobalEngineConfigLock, +} from "./support"; interface SessionStartResponse { session_id: string; @@ -20,16 +26,35 @@ interface TeacherSafetyAlert { interface TeacherDashboardResponse { source: string; safety_alerts: TeacherSafetyAlert[]; + pending_reviews?: TeacherSessionSummary[]; + recent_sessions?: TeacherSessionSummary[]; +} + +interface TeacherSessionSummary { + session_id: string; + turn_count: number; + evaluation_status?: "pending" | "ready" | "error"; + review_ready?: boolean; + supervisor_state?: string; } interface ReviewTurn { speaker: string; text: string; + techniques?: Array<{ + kind?: string; + label: string; + }>; nonverbal?: Array<{ kind: string; label: string; detail: string; }>; + note?: { + tone?: string; + title: string; + body: string; + } | null; } interface SessionReviewResponse { @@ -55,12 +80,24 @@ interface SessionReviewResponse { } | null; } +interface SessionDetailResponse { + session_id: string; + theory_mode: string; + status: "active" | "ended"; + turns?: Array<{ + speaker: "learner" | "client"; + text: string; + }>; +} + interface LiveCoachEvent { turn_seq: number; learner_text_excerpt?: string | null; suggestion: { + status?: "ready" | "degraded"; title: string; message: string; + latency_ms?: number; sources?: Array<{ source_id: string; title: string; @@ -111,6 +148,12 @@ interface EvaluationSummaryResponse { deep?: Record | null; } +interface AdminEngineConfigResponse { + engine_mode: string; + engine_url: string; + model: string; +} + interface PrepostMeasureItem { measure_name: string; timepoint: string; @@ -420,6 +463,37 @@ async function createSessionWithTurn(page: Page) { return { sessionId: started.session_id, expectedMinTurns }; } +async function createActiveSessionWithTurn(page: Page) { + const persona = await fetchAvailablePersona(page); + const startedResponse = await page.request.post("/api/sessions", { + data: { + persona_code: persona.code, + theory_mode: "humanistic", + }, + }); + await expectResponseOk(startedResponse); + + const started = (await startedResponse.json()) as SessionStartResponse; + expect(started.degraded).toBe(false); + + let turnResponse = await page.request.post(`/api/sessions/${started.session_id}/turn`, { + data: { + text: "제가 이해한 바로는 그 일이 꽤 오래 마음에 남아 있었던 것 같습니다.", + }, + }); + for (let attempt = 0; attempt < 2 && !turnResponse.ok(); attempt += 1) { + await page.waitForTimeout(1000); + turnResponse = await page.request.post(`/api/sessions/${started.session_id}/turn`, { + data: { + text: "그 장면을 떠올릴 때 몸이나 마음에서 먼저 느껴지는 반응이 있나요?", + }, + }); + } + await expectResponseOk(turnResponse); + + return started.session_id; +} + async function createEndedSessionWithoutTurn(page: Page) { const persona = await fetchAvailablePersona(page); const startedResponse = await page.request.post("/api/sessions", { @@ -456,6 +530,49 @@ async function evaluationState(page: Page, sessionId: string) { } test.describe("session persistence", () => { + test("persists selected CBT theory mode from session UI into DB-backed detail @single-run", async ({ + page, + }) => { + test.setTimeout(60_000); + + const healthResponse = await page.request.get("/api/health"); + await expectResponseOk(healthResponse); + const health = (await healthResponse.json()) as HealthResponse; + test.skip(!health.db, "theory mode persistence requires DB"); + + await signInAsLearner(page); + const persona = await fetchAvailablePersona(page, 1); + + await page.goto(`/learn/session/${persona.code}`); + const cbtButton = page.locator(".sx-theory").getByRole("button", { name: /CBT/ }); + await expect(cbtButton).toBeVisible(); + await cbtButton.click(); + await expect(cbtButton).toHaveAttribute("aria-pressed", "true"); + + const startResponsePromise = page.waitForResponse((response) => { + const url = new URL(response.url()); + return response.request().method() === "POST" && url.pathname.endsWith("/api/sessions"); + }); + await page.getByRole("button", { name: "회기 시작" }).click(); + const startResponse = await startResponsePromise; + await expectResponseOk(startResponse); + expect(startResponse.request().postDataJSON()).toMatchObject({ + persona_code: persona.code, + theory_mode: "cbt", + }); + + await expect(page).toHaveURL(/\/learn\/session\/[0-9a-f-]+$/i); + const sessionId = new URL(page.url()).pathname.split("/").at(-1); + expect(sessionId).toMatch(/^[0-9a-f-]+$/i); + + const detailResponse = await page.request.get(`/api/sessions/${sessionId}`); + await expectResponseOk(detailResponse); + const detail = (await detailResponse.json()) as SessionDetailResponse; + expect(detail.session_id).toBe(sessionId); + expect(detail.theory_mode).toBe("cbt"); + expect(detail.status).toBe("active"); + }); + test("persists the browser SSE stream into DB-backed review @single-run", async ({ page }) => { test.setTimeout(120_000); @@ -511,6 +628,77 @@ test.describe("session persistence", () => { ).toBe(true); }); + test("masks Korean PII through the browser stream into DB-backed review @single-run", async ({ + page, + }) => { + test.setTimeout(120_000); + + const healthResponse = await page.request.get("/api/health"); + await expectResponseOk(healthResponse); + const health = (await healthResponse.json()) as HealthResponse; + test.skip(!health.db || !health.engine, "PII stream masking requires DB and engine"); + + await signInAsLearner(page); + const persona = await fetchAvailablePersona(page, 1); + const piiText = + "제 이름은 김서연이고 한신대학교 상담센터에서 010-1234-5678로 연락을 받았어요."; + const forbiddenRaw = ["김서연", "한신대학교", "상담센터", "010-1234-5678"]; + + await page.goto(`/learn/session/${persona.code}`); + await page.getByRole("button", { name: "회기 시작" }).click(); + await expect(page.locator(".sx-page.sx-page--active")).toBeVisible({ timeout: 15_000 }); + await expect(page).toHaveURL(/\/learn\/session\/[0-9a-f-]+$/i); + const sessionId = new URL(page.url()).pathname.split("/").at(-1); + expect(sessionId).toMatch(/^[0-9a-f-]+$/i); + + const streamResponsePromise = page.waitForResponse((response) => { + const url = new URL(response.url()); + return ( + response.request().method() === "POST" && + url.pathname.endsWith(`/api/sessions/${sessionId}/stream`) + ); + }); + + const input = page.getByLabel("학습자 발화 입력"); + await input.fill(piiText); + await page.getByRole("button", { name: "보내기" }).click(); + + const streamResponse = await streamResponsePromise; + await expectResponseOk(streamResponse); + await expect(page.locator(".sx-utt.is-client.is-thinking")).toHaveCount(0, { + timeout: 90_000, + }); + const clientUtterance = page.locator(".sx-utt.is-client").first(); + await expect(clientUtterance).toBeVisible(); + await expect(clientUtterance).not.toContainText("답변을 준비 중입니다."); + await expect(input).toBeEnabled({ timeout: 90_000 }); + + const detailResponse = await page.request.get(`/api/sessions/${sessionId}`); + await expectResponseOk(detailResponse); + const detail = (await detailResponse.json()) as SessionDetailResponse; + expect(detail.session_id).toBe(sessionId); + const detailBlob = JSON.stringify(detail); + for (const raw of forbiddenRaw) { + expect(detailBlob).not.toContain(raw); + } + const detailLearnerTurn = detail.turns?.find((turn) => turn.speaker === "learner"); + expect(detailLearnerTurn?.text).toContain("[NAME]"); + expect(detailLearnerTurn?.text).toContain("[ORG]"); + expect(detailLearnerTurn?.text).toContain("[PHONE]"); + + const reviewResponse = await page.request.get(`/api/sessions/${sessionId}/review`); + await expectResponseOk(reviewResponse); + const review = (await reviewResponse.json()) as SessionReviewResponse; + const reviewBlob = JSON.stringify(review); + for (const raw of forbiddenRaw) { + expect(reviewBlob).not.toContain(raw); + } + const reviewLearnerTurn = review.turns.find((turn) => turn.speaker === "learner"); + expect(reviewLearnerTurn?.text).toContain("[NAME]"); + expect(reviewLearnerTurn?.text).toContain("[ORG]"); + expect(reviewLearnerTurn?.text).toContain("[PHONE]"); + }); + test("persists crisis safety event into DB-backed teacher dashboard @single-run", async ({ page, }) => { @@ -552,6 +740,19 @@ test.describe("session persistence", () => { await expect(page.getByRole("link", { name: "109" })).toBeVisible(); await expect(input).toBeDisabled(); + const reviewResponse = await page.request.get(`/api/sessions/${sessionId}/review`); + await expectResponseOk(reviewResponse); + const review = (await reviewResponse.json()) as SessionReviewResponse; + expect(review.session_id).toBe(sessionId); + expect( + review.turns.some((turn) => turn.speaker === "learner" && turn.text === crisisText), + "the crisis utterance itself should be persisted for supervisor review", + ).toBe(true); + expect( + review.turns.some((turn) => turn.speaker === "client" && turn.text.trim().length > 0), + "real crisis escalation must stop before a client AI reply is stored", + ).toBe(false); + await signInAsTeacher(page); await expect .poll( @@ -574,7 +775,9 @@ test.describe("session persistence", () => { await page.goto("/teach"); await expect(page.getByText("109 안전 확인 큐")).toBeVisible({ timeout: 15_000 }); - await expect(page.locator(".pf-alert").filter({ hasText: sessionId ?? "" })).toContainText("109"); + await expect(page.locator(".pf-alert").filter({ hasText: sessionId ?? "" })).toContainText("109", { + timeout: 30_000, + }); }); test("persists AI tutor coaching history through reload @single-run", async ({ page }) => { @@ -627,6 +830,8 @@ test.describe("session persistence", () => { const coachResponse = await coachResponsePromise; await expectResponseOk(coachResponse); const coachSuggestion = (await coachResponse.json()) as LiveCoachEvent["suggestion"]; + expect(coachSuggestion.status).toBe("ready"); + expect(coachSuggestion.latency_ms ?? 0).toBeGreaterThan(0); const workbookSource = coachSuggestion.sources?.find( (source) => source.source_id === "workbook_0615_case_conceptualization", ); @@ -656,6 +861,8 @@ test.describe("session persistence", () => { event.suggestion.title === coachSuggestion.title, ); expect(persistedCoachEvent, "DB-backed live coach history should include the delivered coaching event").toBeTruthy(); + expect(persistedCoachEvent?.suggestion.status).toBe("ready"); + expect(persistedCoachEvent?.suggestion.latency_ms ?? 0).toBeGreaterThan(0); expect( persistedCoachEvent?.suggestion.sources?.some( (source) => @@ -1106,6 +1313,19 @@ test.describe("session persistence", () => { expect(review.supervisorState).toBe("평가 완료"); expect(review.summary ?? "").toContain("평가 AI"); + const dashboardResponse = await page.request.get("/api/teacher/dashboard"); + await expectResponseOk(dashboardResponse); + const dashboard = (await dashboardResponse.json()) as TeacherDashboardResponse; + expect(dashboard.source).toBe("database"); + const dashboardSession = (dashboard.recent_sessions ?? []).find( + (session) => session.session_id === sessionId, + ); + expect(dashboardSession, "teacher dashboard should expose the same persisted session").toBeTruthy(); + expect(dashboardSession?.turn_count ?? 0).toBeGreaterThanOrEqual(expectedMinTurns); + expect(dashboardSession?.evaluation_status).toBe("ready"); + expect(dashboardSession?.review_ready).toBe(true); + expect(dashboardSession?.supervisor_state).toBe("평가 완료"); + await page.goto(`/teach/session/${sessionId}/review`); await expect(page.locator(".sr-head__stats")).toContainText("평가 완료", { timeout: 15_000, @@ -1113,4 +1333,206 @@ test.describe("session persistence", () => { await expect(page.locator('[aria-label="리뷰 생성 상태"]')).toContainText("준비됨"); await expect(page.getByRole("button", { name: "AI 평가 재시도" })).toHaveCount(0); }); + + test("persists explicit teacher session reevaluation into durable DB state @single-run", async ({ + page, + }) => { + test.setTimeout(180_000); + + const healthResponse = await page.request.get("/api/health"); + await expectResponseOk(healthResponse); + const health = (await healthResponse.json()) as HealthResponse; + test.skip(!health.db || !health.engine, "teacher session reevaluation requires DB and engine"); + + await signInAsLearner(page); + const sessionId = await createActiveSessionWithTurn(page); + + await signInAsTeacher(page); + const reevaluateResponse = await page.request.post(`/api/eval/sessions/${sessionId}/reevaluate`, { + data: { scope: "stage_transition" }, + }); + await expectResponseOk(reevaluateResponse); + const reevaluation = (await reevaluateResponse.json()) as { + scope?: string; + error?: string | null; + turns_evaluated?: number; + }; + expect(reevaluation.scope).toBe("stage_transition"); + expect(reevaluation.error ?? "").toBe(""); + expect(reevaluation.turns_evaluated ?? 0).toBeGreaterThan(0); + + const evaluationResponse = await page.request.get(`/api/eval/sessions/${sessionId}/evaluation`); + await expectResponseOk(evaluationResponse); + const evaluation = (await evaluationResponse.json()) as EvaluationSummaryResponse; + expect(evaluation.status).toBe("ready"); + expect(evaluation.durable).toBe(true); + expect(evaluation.deep?.scope).toBe("stage_transition"); + expect(evaluation.deep?.turns_evaluated).toBe(reevaluation.turns_evaluated); + + const reviewResponse = await page.request.get(`/api/sessions/${sessionId}/review`); + await expectResponseOk(reviewResponse); + const review = (await reviewResponse.json()) as SessionReviewResponse; + expect(review.reviewReady).toBe(true); + expect(review.supervisorState).toBe("평가 완료"); + }); + + test("persists explicit teacher turn reevaluation into DB-backed review state @single-run", async ({ + page, + }) => { + test.setTimeout(120_000); + + const healthResponse = await page.request.get("/api/health"); + await expectResponseOk(healthResponse); + const health = (await healthResponse.json()) as HealthResponse; + test.skip(!health.db || !health.engine, "teacher turn reevaluation requires DB and engine"); + + await signInAsLearner(page); + const sessionId = await createActiveSessionWithTurn(page); + + await signInAsTeacher(page); + const turnReevaluationResponse = await page.request.post(`/api/eval/sessions/${sessionId}/turn`, { + data: { turn_seq: 1 }, + }); + await expectResponseOk(turnReevaluationResponse); + const turnReevaluation = (await turnReevaluationResponse.json()) as { + error?: string | null; + techniques?: Array<{ label_ko?: string; code?: string }>; + }; + expect(turnReevaluation.error ?? "").toBe(""); + const expectedLabels = (turnReevaluation.techniques ?? []) + .map((technique) => technique.label_ko || technique.code || "") + .filter((label) => label.length > 0); + test.skip(expectedLabels.length === 0, "evaluator returned no technique labels to verify in review"); + + const reviewResponse = await page.request.get(`/api/sessions/${sessionId}/review`); + await expectResponseOk(reviewResponse); + const review = (await reviewResponse.json()) as SessionReviewResponse; + const learnerTurn = review.turns.find((turn) => turn.speaker === "learner"); + expect(learnerTurn, "review should still expose the original learner turn").toBeTruthy(); + const reviewLabels = (learnerTurn?.techniques ?? []).map((technique) => technique.label); + for (const label of expectedLabels) { + expect(reviewLabels).toContain(label); + } + }); + + test("runs manual AI evaluation retry from teacher review UI into durable DB state @single-run", async ({ + page, + }) => { + test.setTimeout(240_000); + + const healthResponse = await page.request.get("/api/health"); + await expectResponseOk(healthResponse); + const health = (await healthResponse.json()) as HealthResponse; + test.skip(!health.db || !health.engine, "manual evaluation retry requires DB and engine"); + + await signInAsLearner(page); + const sessionId = await createActiveSessionWithTurn(page); + const endedResponse = await page.request.post(`/api/sessions/${sessionId}/end`); + await expectResponseOk(endedResponse); + + await withGlobalEngineConfigLock("manual-evaluation-retry-ui", async () => { + await signInAsAdmin(page); + const engineResponse = await page.request.get("/api/admin/engine-config"); + await expectResponseOk(engineResponse); + const currentEngine = (await engineResponse.json()) as AdminEngineConfigResponse; + + try { + const brokenEnginePatch = await page.request.patch("/api/admin/engine-config", { + data: { + engine_mode: currentEngine.engine_mode, + engine_url: "http://127.0.0.1:9", + model: currentEngine.model, + }, + }); + await expectResponseOk(brokenEnginePatch); + + await signInAsTeacher(page); + const failedReevaluationResponse = await page.request.post( + `/api/eval/sessions/${sessionId}/reevaluate`, + { data: { scope: "session_end" } }, + ); + expect( + failedReevaluationResponse.ok(), + "broken engine should create a failed durable evaluation seed", + ).toBe(false); + + await signInAsAdmin(page); + const restoreResponse = await page.request.patch("/api/admin/engine-config", { + data: { + engine_mode: currentEngine.engine_mode, + engine_url: currentEngine.engine_url, + model: currentEngine.model, + }, + }); + await expectResponseOk(restoreResponse); + + await signInAsTeacher(page); + await page.goto(`/teach/session/${sessionId}/review`); + await expect(page.getByText("평가 실패")).toBeVisible({ timeout: 15_000 }); + const retryButton = page.getByRole("button", { name: "AI 평가 재시도" }); + await expect(retryButton).toBeVisible(); + + const reevaluateResponsePromise = page.waitForResponse((response) => { + const url = new URL(response.url()); + return ( + response.request().method() === "POST" && + url.pathname.endsWith(`/eval/sessions/${sessionId}/reevaluate`) + ); + }); + await retryButton.click(); + const reevaluateResponse = await reevaluateResponsePromise; + await expectResponseOk(reevaluateResponse); + const reevaluation = (await reevaluateResponse.json()) as { + scope?: string; + error?: string | null; + turns_evaluated?: number; + }; + expect(reevaluation.scope).toBe("session_end"); + expect(reevaluation.error ?? "").toBe(""); + expect(reevaluation.turns_evaluated ?? 0).toBeGreaterThan(0); + + await expect(page.locator(".sr-head__stats")).toContainText("평가 완료", { + timeout: 30_000, + }); + await expect(page.locator('[aria-label="리뷰 생성 상태"]')).toContainText("준비됨"); + await expect(page.getByRole("button", { name: "AI 평가 재시도" })).toHaveCount(0); + + const evaluationResponse = await page.request.get(`/api/eval/sessions/${sessionId}/evaluation`); + await expectResponseOk(evaluationResponse); + const evaluation = (await evaluationResponse.json()) as EvaluationSummaryResponse; + expect(evaluation.status).toBe("ready"); + expect(evaluation.durable).toBe(true); + expect(evaluation.deep?.scope).toBe("session_end"); + expect(evaluation.deep?.turns_evaluated).toBe(reevaluation.turns_evaluated); + + const reviewResponse = await page.request.get(`/api/sessions/${sessionId}/review`); + await expectResponseOk(reviewResponse); + const review = (await reviewResponse.json()) as SessionReviewResponse; + expect(review.reviewReady).toBe(true); + expect(review.supervisorState).toBe("평가 완료"); + + const dashboardResponse = await page.request.get("/api/teacher/dashboard"); + await expectResponseOk(dashboardResponse); + const dashboard = (await dashboardResponse.json()) as TeacherDashboardResponse; + expect(dashboard.source).toBe("database"); + const dashboardSession = [...(dashboard.pending_reviews ?? []), ...(dashboard.recent_sessions ?? [])].find( + (session) => session.session_id === sessionId, + ); + expect(dashboardSession, "teacher dashboard should expose the retried session").toBeTruthy(); + expect(dashboardSession?.evaluation_status).toBe("ready"); + expect(dashboardSession?.review_ready).toBe(true); + expect(dashboardSession?.supervisor_state).toBe("평가 완료"); + } finally { + await signInAsAdmin(page); + const restoreResponse = await page.request.patch("/api/admin/engine-config", { + data: { + engine_mode: currentEngine.engine_mode, + engine_url: currentEngine.engine_url, + model: currentEngine.model, + }, + }); + await expectResponseOk(restoreResponse); + } + }); + }); }); diff --git a/apps/web/e2e/session-review.spec.ts b/apps/web/e2e/session-review.spec.ts index dea966e..fe7bbd9 100644 --- a/apps/web/e2e/session-review.spec.ts +++ b/apps/web/e2e/session-review.spec.ts @@ -242,6 +242,80 @@ test.describe("session review", () => { expect(reviewAttempts).toBeGreaterThanOrEqual(2); }); + test("keeps polling long-running session evaluation until the ready review arrives", async ({ + page, + }) => { + await routeSupervisorCapableLearner(page); + await routePrepostMeasures(page); + + const sessionId = "review-delayed-ready"; + let reviewAttempts = 0; + const readyAtAttempt = 10; + await page.route(`**/api/sessions/${sessionId}/review`, async (route) => { + reviewAttempts += 1; + const ready = { + ...filledReviewResponse(sessionId), + summary: "평가 AI가 늦게 완료된 deep-loop 결과를 반영했습니다.", + }; + const pending = { + ...ready, + supervisorState: "평가 대기", + summary: "평가 AI가 긴 회기 축어록을 분석 중입니다.", + rubric: [], + goodMoments: [], + growthPoints: [], + degraded: true, + reviewReady: false, + }; + await route.fulfill({ + status: 200, + contentType: "application/json", + body: JSON.stringify(reviewAttempts >= readyAtAttempt ? ready : pending), + }); + }); + + await page.goto(`/learn/session/${sessionId}/review`); + + await expect(page.getByText("평가 AI가 늦게 완료된 deep-loop 결과를 반영했습니다.")).toBeVisible({ + timeout: 20_000, + }); + await expect(page.getByText("평가 완료")).toBeVisible(); + expect(reviewAttempts).toBeGreaterThanOrEqual(readyAtAttempt); + }); + + test("does not offer manual AI retry while session evaluation is still pending", async ({ + page, + }) => { + await routeSupervisorCapableLearner(page); + await routePrepostMeasures(page); + + const sessionId = "review-pending-no-manual-retry"; + await page.route(`**/api/sessions/${sessionId}/review`, async (route) => { + const pending = { + ...filledReviewResponse(sessionId), + supervisorState: "평가 대기", + summary: "평가 AI가 저장된 축어록을 분석 중입니다.", + rubric: [], + goodMoments: [], + growthPoints: [], + degraded: true, + reviewReady: false, + }; + await route.fulfill({ + status: 200, + contentType: "application/json", + body: JSON.stringify(pending), + }); + }); + + await page.goto(`/teach/session/${sessionId}/review`); + + await expect(page.getByText("평가 대기", { exact: true })).toBeVisible(); + await expect(page.getByText("평가 AI가 저장된 축어록을 분석 중입니다.")).toBeVisible(); + await expect(page.getByRole("button", { name: "AI 평가 재시도" })).toHaveCount(0); + await expect(page.getByRole("button", { name: "검토 완료" })).toBeDisabled(); + }); + test("lets supervisor retry a failed AI session evaluation", async ({ page }) => { await routeSupervisorCapableLearner(page); await routePrepostMeasures(page); @@ -303,11 +377,13 @@ test.describe("session review", () => { await page.goto(`/teach/session/${sessionId}/review`); await expect(page.getByText("평가 실패")).toBeVisible(); await expect(page.getByRole("button", { name: "AI 평가 재시도" })).toBeVisible(); + await expect(page.getByRole("button", { name: "검토 완료" })).toBeDisabled(); await page.getByRole("button", { name: "AI 평가 재시도" }).click(); await expect(page.getByText("평가 완료")).toBeVisible(); await expect(page.getByText("평가 실패")).toHaveCount(0); + await expect(page.getByRole("button", { name: "검토 완료" })).toBeEnabled(); expect(reevaluateRequests).toBe(1); }); @@ -483,5 +559,13 @@ test.describe("session review", () => { await expect(page.getByText("2/3 쌍")).toBeVisible(); await expect(page.locator(".sr-prepost__actions")).toContainText("저장된 값 기준"); + + const prepostCard = page.locator(".sr-card--prepost"); + await prepostInputs.nth(0).fill(""); + await expect(prepostInputs.nth(0)).toHaveAttribute("aria-invalid", "true"); + await expect(prepostCard.locator(".sr-prepost__actions")).toContainText("저장되지 않은 입력 있음"); + await expect(prepostCard.locator(".sr-prepost__actions")).not.toContainText("저장된 값 기준"); + await expect(prepostCard.getByRole("alert")).toContainText("기존 값을 비우려면 새 점수를 입력해 주세요."); + await expect(prepostCard.getByRole("button", { name: "점수 저장" })).toBeDisabled(); }); }); diff --git a/apps/web/e2e/support.ts b/apps/web/e2e/support.ts index 720410d..3b61551 100644 --- a/apps/web/e2e/support.ts +++ b/apps/web/e2e/support.ts @@ -10,8 +10,9 @@ export interface E2EPersona { } export async function useRealApi(_page: Page) { - // Intentionally empty. E2E should exercise the local API through the Vite - // proxy instead of replacing app data with browser-side route fixtures. + // No-op marker for tests that should use the configured Vite /api proxy unless + // a spec explicitly installs route fixtures for a focused UI or error state. + // This helper alone is not proof that every test in the file is fixture-free. } function sleep(ms: number) { @@ -98,6 +99,25 @@ export async function signInAsTeacher(page: Page) { }); } +export async function signInAsAdmin(page: Page) { + const res = await page.request.post("/api/auth/dev-login", { + data: { + email: "admin@twentyoz.kr", + role: "admin", + display_name: "E2E Admin", + }, + }); + expect(res.ok(), await res.text()).toBeTruthy(); + await completeOnboarding(page, { + legal_name: "E2E Admin", + affiliation: "한신대학교", + department: "운영", + grade_level: "관리자", + phone: "010-2222-2222", + contact_address: "경기도 오산시 한신대학교", + }); +} + export async function completeOnboarding( page: Page, profile: { diff --git a/apps/web/e2e/teacher.spec.ts b/apps/web/e2e/teacher.spec.ts index 7c43009..82c666f 100644 --- a/apps/web/e2e/teacher.spec.ts +++ b/apps/web/e2e/teacher.spec.ts @@ -660,7 +660,21 @@ test.describe("teacher console", () => { const analysisPanel = page.locator('[data-learner-analysis-panel="true"]'); await expect(analysisPanel).toBeVisible(); + await expect(analysisPanel.getByRole("tab", { name: /^페르소나별 회기/ })).toHaveAttribute( + "aria-selected", + "true", + ); + await expect(analysisPanel.locator('[data-learner-persona-group="true"]')).toHaveCount(2); await expect(analysisPanel.locator('[data-learner-session-row="true"]')).toHaveCount(0); + const personaGroup = analysisPanel + .locator('[data-learner-persona-group="true"]') + .filter({ hasText: "민재" }); + await personaGroup.getByRole("button", { name: /P2 민재 회기 펼치기/ }).click(); + await expect(analysisPanel.locator('[data-learner-persona-sessions="true"]')).toBeVisible(); + await expect( + analysisPanel.locator('[data-learner-persona-sessions="true"]').getByText(`${learnerB}-session-2`), + ).toBeVisible(); + await expect(analysisPanel.locator('[data-learner-session-row="true"]')).toHaveCount(1); await analysisPanel.getByRole("tab", { name: /^전체 회기/ }).click(); await expect(analysisPanel.locator('[data-learner-session-row="true"]')).toHaveCount(2); await expect(analysisPanel.getByText(`${learnerB}-session-2`)).toBeVisible(); @@ -766,6 +780,89 @@ test.describe("teacher console", () => { await expectNoHorizontalOverflow(page); }); + test("surfaces failed AI session evaluation in the teacher pending queue", async ({ + page, + }) => { + const sessionId = "teacher-failed-ai-evaluation-session"; + await page.route("**/api/teacher/dashboard", (route) => + route.fulfill({ + status: 200, + contentType: "application/json", + body: JSON.stringify({ + source: "database", + cohort_label: "E2E cohort", + total_learners: 1, + active_sessions: 0, + ended_sessions: 1, + learner_growth: [], + safety_alerts: [], + pending_reviews: [ + { + session_id: sessionId, + learner_id: "00000000-0000-0000-0000-000000000116", + learner_label: "하린", + persona_code: "P6", + persona_name: "하린", + session_no: 6, + status: "ended", + stage: "정리", + turn_count: 18, + learner_turn_count: 9, + client_turn_count: 9, + started_at: "2026-06-30T05:00:00Z", + ended_at: "2026-06-30T06:42:33Z", + review_status: "pending", + review_note: null, + reviewed_at: null, + evaluation_status: "error", + review_ready: false, + supervisor_state: "평가 실패", + evaluation_error: "session evaluation timeout after 45s", + }, + ], + recent_sessions: [], + message: "교수자 검토 대기 회기가 있습니다.", + }), + }), + ); + await page.route("**/api/personas/review", (route) => + route.fulfill({ + status: 200, + contentType: "application/json", + body: "[]", + }), + ); + await page.route("**/api/auth/me", (route) => + route.fulfill({ + status: 200, + contentType: "application/json", + body: JSON.stringify({ + user_id: "00000000-0000-0000-0000-000000000901", + email: "teacher@hs.ac.kr", + role: "teacher", + display_name: "E2E Teacher", + admin_access: false, + super_admin: false, + account_status: "approved", + approval_required: false, + cohort_ids: [], + consent_at: 1782820000, + onboarding_completed_at: 1782820001, + nickname: "E2E Teacher", + self_introduction: "", + avatar_url: "", + }), + }), + ); + + await page.goto("/teach"); + + const row = page.getByRole("button", { name: /하린 P6 회기 상세 검토/ }); + await expect(row).toBeVisible(); + await expect(row).toContainText("평가 실패"); + await expect(row).toContainText("검토 대기"); + }); + test("opens recent ended and active sessions from the teacher history @single-run", async ({ page, }) => { diff --git a/apps/web/e2e/voice.spec.ts b/apps/web/e2e/voice.spec.ts index 2811b42..add62ae 100644 --- a/apps/web/e2e/voice.spec.ts +++ b/apps/web/e2e/voice.spec.ts @@ -122,4 +122,30 @@ test.describe("voice websocket auth boundary", () => { }); expect(messages.some((message) => message.type === "degraded")).toBeFalsy(); }); + + test("rejects an existing voice session after consent withdrawal @single-run", async ({ + page, + }, testInfo) => { + await signInLearner(page, testInfo, "withdrawn"); + const persona = await fetchAvailablePersona(page); + const start = await page.request.post("/api/sessions", { + data: { persona_code: persona.code, theory_mode: "humanistic" }, + }); + expect(start.ok(), await start.text()).toBeTruthy(); + const { session_id } = (await start.json()) as { session_id: string }; + + const withdrawn = await page.request.delete("/api/auth/consent"); + expect(withdrawn.ok(), await withdrawn.text()).toBeTruthy(); + await page.goto("/login"); + + const result = await openVoiceSocket( + page, + `/api/voice/ws?session_id=${encodeURIComponent(session_id)}`, + ); + const messages = parsedMessages(result); + + expect(result.code).toBe(1008); + expect(messages).toContainEqual({ type: "error", detail: "consent_required" }); + expect(messages.some((message) => message.type === "degraded")).toBeFalsy(); + }); }); diff --git a/apps/web/src/App.tsx b/apps/web/src/App.tsx index a536600..446ceb6 100644 --- a/apps/web/src/App.tsx +++ b/apps/web/src/App.tsx @@ -17,7 +17,7 @@ import type { ReactNode } from "react"; import { BrowserRouter, Navigate, Route, Routes, useLocation } from "react-router-dom"; -import { AuthProvider, canAccessRole, useAuth, roleHomePath, type Role } from "./lib/auth"; +import { AuthProvider, canAccessRole, useAuth, roleHomePath, type AuthUser, type Role } from "./lib/auth"; import Login from "./pages/Login"; import Onboarding from "./pages/Onboarding"; @@ -74,6 +74,17 @@ function RequireAuth({ return <>{children}; } +function isAdminWorkspacePath(path: string) { + return path === "/admin" || path.startsWith("/admin/"); +} + +function approvedHomePath(user: AuthUser) { + if (user.onboardingCompletedAt == null) { + return canAccessRole(user, "admin") ? "/admin" : "/onboarding"; + } + return roleHomePath(user.role); +} + function OnboardingGate({ children }: { children: ReactNode }) { const { user, loading } = useAuth(); const location = useLocation(); @@ -82,6 +93,9 @@ function OnboardingGate({ children }: { children: ReactNode }) { if (loading) return ; if (path === "/pending") return <>{children}; if (user && user.onboardingCompletedAt == null && path !== "/onboarding") { + if (isAdminWorkspacePath(path) && canAccessRole(user, "admin")) { + return <>{children}; + } return ; } if (user && user.onboardingCompletedAt != null && (path === "/login" || path === "/onboarding")) { @@ -100,12 +114,7 @@ function PendingApprovalGate({ children }: { children: ReactNode }) { return ; } if (user && user.accountStatus === "approved" && path === "/pending") { - return ( - - ); + return ; } return <>{children}; } @@ -118,7 +127,7 @@ function RootRedirect() { return ; } if (user && user.onboardingCompletedAt == null) { - return ; + return ; } return ; } diff --git a/apps/web/src/lib/api.gen.ts b/apps/web/src/lib/api.gen.ts index 95f62e9..c9f1b97 100644 --- a/apps/web/src/lib/api.gen.ts +++ b/apps/web/src/lib/api.gen.ts @@ -2439,6 +2439,33 @@ export interface components { source_note: string; terms: components["schemas"]["LegalDocument"]; }; + /** + * LiveCoachCreditEvent + * @description 코칭 기회 사용/충전 학습 기록. + */ + LiveCoachCreditEvent: { + /** Balance */ + balance: number; + /** Created At */ + created_at: string; + /** Delta */ + delta: number; + /** Event Id */ + event_id: string; + /** + * Event Type + * @enum {string} + */ + event_type: "use" | "recharge"; + /** Reason */ + reason: string; + /** Session Id */ + session_id: string; + /** Stage */ + stage?: ("라포" | "탐색" | "개입" | "정리") | null; + /** Turn Seq */ + turn_seq: number; + }; /** * LiveCoachEvent * @description 회기 중 실제로 전달된 라이브 코칭 이력. @@ -2462,8 +2489,11 @@ export interface components { }; /** LiveCoachHistoryResponse */ LiveCoachHistoryResponse: { + /** Credit Events */ + credit_events?: components["schemas"]["LiveCoachCreditEvent"][]; /** Events */ events?: components["schemas"]["LiveCoachEvent"][]; + quota?: components["schemas"]["LiveCoachQuota"]; /** * Source * @default runtime @@ -2471,6 +2501,16 @@ export interface components { */ source: "database" | "runtime"; }; + /** + * LiveCoachQuota + * @description 회기 중 즉시 코칭 사용 가능 횟수. + */ + LiveCoachQuota: { + /** Max */ + max: number; + /** Remaining */ + remaining: number; + }; /** LiveCoachRequest */ LiveCoachRequest: { /** Client Reply */ @@ -2544,6 +2584,8 @@ export interface components { * @description 프론트가 그대로 표시하는 턴 직후 코칭 카드. */ LiveCoachSuggestion: { + /** Credit Events */ + credit_events?: components["schemas"]["LiveCoachCreditEvent"][]; /** * Focus * @default exploration @@ -2559,6 +2601,13 @@ export interface components { message: string; /** Next Utterance */ next_utterance?: string | null; + /** + * Persistence Source + * @default database + * @enum {string} + */ + persistence_source: "database" | "runtime"; + quota?: components["schemas"]["LiveCoachQuota"] | null; /** Rationale */ rationale?: string | null; /** Safety Note */ @@ -3784,6 +3833,14 @@ export interface components { client_turn_count: number; /** Ended At */ ended_at?: string | null; + /** Evaluation Error */ + evaluation_error?: string | null; + /** + * Evaluation Status + * @default pending + * @enum {string} + */ + evaluation_status: "pending" | "ready" | "error"; /** Learner Id */ learner_id: string; /** Learner Label */ @@ -3796,6 +3853,11 @@ export interface components { persona_name: string; /** Review Note */ review_note?: string | null; + /** + * Review Ready + * @default false + */ + review_ready: boolean; /** * Review Status * @default pending @@ -3817,6 +3879,12 @@ export interface components { started_at: string; /** Status */ status: string; + /** + * Supervisor State + * @default 기록 대기 + * @enum {string} + */ + supervisor_state: "기록 대기" | "평가 대기" | "평가 완료" | "평가 실패"; /** Turn Count */ turn_count: number; }; diff --git a/apps/web/src/lib/api.ts b/apps/web/src/lib/api.ts index a2d5d1a..10a47fb 100644 --- a/apps/web/src/lib/api.ts +++ b/apps/web/src/lib/api.ts @@ -201,8 +201,10 @@ export type SessionStartResponse = ApiSchema<"SessionStartResponse">; /** POST /sessions/{id}/turn — sessions.py TurnResponse */ export type TurnResponse = ApiSchema<"TurnResponse">; export type LiveCoachRequest = ApiSchema<"LiveCoachRequest">; +export type LiveCoachCreditEvent = ApiSchema<"LiveCoachCreditEvent">; export type LiveCoachEvent = ApiSchema<"LiveCoachEvent">; export type LiveCoachHistoryResponse = ApiSchema<"LiveCoachHistoryResponse">; +export type LiveCoachQuota = ApiSchema<"LiveCoachQuota">; export type LiveCoachSuggestion = ApiSchema<"LiveCoachSuggestion">; export type LiveCoachSource = ApiSchema<"LiveCoachSource">; export type ReevaluateRequest = ApiSchema<"ReevaluateRequest">; diff --git a/apps/web/src/pages/Professor.tsx b/apps/web/src/pages/Professor.tsx index f375653..3ce71bb 100644 --- a/apps/web/src/pages/Professor.tsx +++ b/apps/web/src/pages/Professor.tsx @@ -21,7 +21,19 @@ import "./professor.css"; type LoadState = "loading" | "ready" | "error"; type ProfessorView = "console" | "analysis"; type LearnerSort = "latest" | "sessions" | "score" | "delta"; -type LearnerAnalysisTab = "trend" | "sessions" | "stages"; +type LearnerAnalysisTab = "personas" | "trend" | "sessions" | "stages"; +type PersonaSessionGroup = { + key: string; + personaCode: string; + personaName: string; + sessions: TeacherSessionSummary[]; + latestAt: string | null; + endedSessions: number; + pendingReviews: number; + closedReviews: number; + avgScore: number | null; + avgRapport: number | null; +}; function formatDateTime(value: string | null): string { if (!value) return "-"; @@ -51,6 +63,24 @@ function formatRapport(value: number | null | undefined): string { return `${Math.round(((value + 1) / 2) * 100)}%`; } +function average(values: Array): number | null { + const numbers = values.filter( + (value): value is number => typeof value === "number" && !Number.isNaN(value), + ); + if (numbers.length === 0) return null; + return numbers.reduce((sum, value) => sum + value, 0) / numbers.length; +} + +function timeValue(value: string | null | undefined): number { + if (!value) return 0; + const time = new Date(value).getTime(); + return Number.isNaN(time) ? 0 : time; +} + +function domToken(value: string): string { + return value.replace(/[^A-Za-z0-9_-]+/g, "-") || "unknown"; +} + function trendLabel(value: string): string { if (value === "up") return "상승"; if (value === "down") return "하락"; @@ -74,6 +104,16 @@ function sessionReviewTone(status: string | null | undefined): "accent" | "neutr return "neutral"; } +function sessionSupervisorTone(status: string | null | undefined): "accent" | "neutral" | "warn" { + if (status === "평가 완료") return "accent"; + if (status === "평가 실패" || status === "평가 대기") return "warn"; + return "neutral"; +} + +function sessionSupervisorLabel(session: TeacherSessionSummary): string { + return session.supervisor_state ?? (session.review_ready ? "평가 완료" : "기록 대기"); +} + function personaReviewStatusLabel(status: PersonaReviewStatus): string { if (status === "review") return "검수 대기"; if (status === "draft") return "수정 대기"; @@ -557,6 +597,9 @@ export default function Professor({ view = "console" }: { view?: ProfessorView }
{formatDateTime(session.ended_at ?? null)} + + {sessionSupervisorLabel(session)} + {sessionReviewStatusLabel(session.review_status)} @@ -773,6 +816,7 @@ export default function Professor({ view = "console" }: { view?: ProfessorView } {session.status === "ended" ? ( + {sessionSupervisorLabel(session)} ·{" "} {sessionReviewStatusLabel(session.review_status)} ) : null} @@ -1033,13 +1077,80 @@ function LearnerAnalysisPanel({ analysis: TeacherLearnerAnalysisResponse; onOpenSession: (sessionId: string) => void; }) { - const [activeTab, setActiveTab] = useState("trend"); + const [activeTab, setActiveTab] = useState("personas"); + const [expandedPersonaKey, setExpandedPersonaKey] = useState(null); const points = analysis.points ?? []; const sessions = analysis.sessions ?? []; - const pointBySession = new Map(points.map((point) => [point.session_id, point])); + const pointBySession = useMemo( + () => new Map(points.map((point) => [point.session_id, point])), + [points], + ); const stageBreakdown = analysis.stage_breakdown ?? []; const summary = analysis.summary; + const personaGroups = useMemo(() => { + const groups = new Map(); + sessions.forEach((session) => { + const personaCode = session.persona_code || "페르소나"; + const personaName = session.persona_name || personaCode; + const key = personaCode || personaName; + const group = + groups.get(key) ?? + { + key, + personaCode, + personaName, + sessions: [], + latestAt: null, + endedSessions: 0, + pendingReviews: 0, + closedReviews: 0, + avgScore: null, + avgRapport: null, + }; + group.sessions.push(session); + if (session.status === "ended") group.endedSessions += 1; + if (session.review_status === "closed") { + group.closedReviews += 1; + } else { + group.pendingReviews += 1; + } + const sessionTime = timeValue(session.ended_at ?? session.started_at); + if (sessionTime >= timeValue(group.latestAt)) { + group.latestAt = session.ended_at ?? session.started_at; + } + groups.set(key, group); + }); + + return Array.from(groups.values()) + .map((group) => { + const orderedSessions = [...group.sessions].sort( + (left, right) => + left.session_no - right.session_no || + timeValue(left.started_at) - timeValue(right.started_at), + ); + return { + ...group, + sessions: orderedSessions, + avgScore: average( + orderedSessions.map((session) => pointBySession.get(session.session_id)?.score), + ), + avgRapport: average( + orderedSessions.map((session) => pointBySession.get(session.session_id)?.rapport), + ), + }; + }) + .sort( + (left, right) => + timeValue(right.latestAt) - timeValue(left.latestAt) || + right.sessions.length - left.sessions.length, + ); + }, [pointBySession, sessions]); + useEffect(() => { + setActiveTab("personas"); + setExpandedPersonaKey(null); + }, [analysis.learner_id]); const tabs: Array<{ key: LearnerAnalysisTab; label: string; count: string }> = [ + { key: "personas", label: "페르소나별 회기", count: `${personaGroups.length}종` }, { key: "trend", label: "추이", count: `${points.length}점` }, { key: "sessions", label: "전체 회기", count: `${sessions.length}건` }, { key: "stages", label: "단계 분석", count: `${stageBreakdown.length}단계` }, @@ -1098,6 +1209,94 @@ function LearnerAnalysisPanel({ ))}
+ {activeTab === "personas" ? ( +
+
+ 페르소나별 회기 + + {personaGroups.length}종 · {sessions.length}건 + +
+ {personaGroups.length > 0 ? ( +
+ {personaGroups.map((group) => { + const isExpanded = expandedPersonaKey === group.key; + const panelId = `persona-sessions-${domToken(group.key)}`; + return ( +
+ + {isExpanded ? ( +
+ {group.sessions.map((session) => ( + onOpenSession(session.session_id)} + /> + ))} +
+ ) : null} +
+ ); + })} +
+ ) : ( + + )} +
+ ) : null} + {activeTab === "trend" ? (
@@ -1197,6 +1396,12 @@ function LearnerSessionRow({ 라포 {formatRapport(point?.rapport)} + + AI 평가 + + {sessionSupervisorLabel(session)} + + 검토 diff --git a/apps/web/src/pages/Session.tsx b/apps/web/src/pages/Session.tsx index fec8b27..2ce9597 100644 --- a/apps/web/src/pages/Session.tsx +++ b/apps/web/src/pages/Session.tsx @@ -31,7 +31,10 @@ import { personaApi, sessionApi, type CrisisResource, + type LiveCoachCreditEvent, type LiveCoachEvent, + type LiveCoachHistoryResponse, + type LiveCoachQuota, type LiveCoachSuggestion, type PersonaSummary, type SessionDetailResponse, @@ -70,6 +73,7 @@ type VoiceServerState = "idle" | "listening" | "thinking" | "speaking"; interface VoiceEvent { type?: string; + code?: string; state?: VoiceServerState; text?: string; final?: boolean; @@ -83,6 +87,22 @@ interface VoiceEvent { conversation_stopped?: boolean; } +const VOICE_TURN_SAVE_FAILED = + "음성 발화를 저장하지 못했습니다. 방금 말한 내용을 확인한 뒤 다시 시도해 주세요."; +const VOICE_CONNECTION_SAVE_FAILED = + "음성 연결이 종료되어 발화를 저장하지 못했습니다. 방금 말한 내용을 확인한 뒤 다시 시도해 주세요."; + +function userFacingVoiceError(payload: VoiceEvent): string { + const detail = payload.detail ?? ""; + if ( + payload.code === "turn_persistence_unavailable" || + (detail.includes("turn append") && detail.includes("persistence unavailable")) + ) { + return VOICE_TURN_SAVE_FAILED; + } + return detail || "음성 처리 중 오류가 발생했습니다."; +} + type AudioContextWindow = Window & { webkitAudioContext?: typeof AudioContext }; type VoiceCaptureMode = "audio-worklet" | "media-recorder"; @@ -137,6 +157,8 @@ interface Utterance { at: number; /** 진행 중(스트리밍/타이핑) 발화 여부 */ partial?: boolean; + /** 음성 WebSocket이 reply 전에 실패해 DB 턴으로 확정되지 않은 발화 */ + failed?: boolean; } interface PhaseInfo { @@ -164,6 +186,7 @@ const THEORY_MODE_OPTIONS: { value: TheoryMode; label: string; detail: string }[ { value: "cbt", label: "CBT", detail: "생각·행동" }, { value: "integrative", label: "통합", detail: "혼합 접근" }, ]; +const DEFAULT_COACH_QUOTA: LiveCoachQuota = { remaining: 3, max: 3 }; const VOICE_WORKLET_MODULE_URL = "/worklets/voice-capture-worklet.js"; const VOICE_WORKLET_PROCESSOR = "voice-capture-processor"; @@ -839,8 +862,15 @@ export default function Session() { const [coachError, setCoachError] = useState(null); const [coachEvidenceOpen, setCoachEvidenceOpen] = useState(false); const [coachHistory, setCoachHistory] = useState([]); + const [coachQuota, setCoachQuota] = useState(DEFAULT_COACH_QUOTA); + const [coachCreditEvents, setCoachCreditEvents] = useState([]); + const [coachCreditPulse, setCoachCreditPulse] = useState(null); const [coachHistoryOpen, setCoachHistoryOpen] = useState(false); const [coachHistoryTurnSeq, setCoachHistoryTurnSeq] = useState(null); + const [coachHistoryLoading, setCoachHistoryLoading] = useState(false); + const [coachHistoryError, setCoachHistoryError] = useState(null); + const [coachPersistenceSource, setCoachPersistenceSource] = useState<"database" | "runtime" | null>(null); + const [coachSyncWarning, setCoachSyncWarning] = useState(null); // ── 경과 타이머 ── const [elapsed, setElapsed] = useState(0); @@ -860,6 +890,23 @@ export default function Session() { const pendingVoiceLearnerTextRef = useRef(""); const coachEvidenceCloseRef = useRef(null); const coachHistoryCloseRef = useRef(null); + const coachCreditSeenRef = useRef>(new Set()); + const coachCreditPulseTimerRef = useRef(null); + + const failPendingVoiceLearnerTurn = useCallback(() => { + const pendingId = pendingVoiceLearnerIdRef.current; + pendingVoiceLearnerIdRef.current = null; + pendingVoiceLearnerTextRef.current = ""; + if (pendingId == null) return false; + setUtterances((prev) => + prev.map((utterance) => + utterance.id === pendingId + ? { ...utterance, partial: false, failed: true, turnSeq: undefined } + : utterance, + ), + ); + return true; + }, []); const ensureVoiceAudioContext = useCallback(() => { const existing = audioContextRef.current; @@ -911,6 +958,12 @@ export default function Session() { setCoachError(null); setCoachEvidenceOpen(false); setCoachHistory([]); + setCoachQuota(DEFAULT_COACH_QUOTA); + setCoachCreditEvents([]); + setCoachCreditPulse(null); + setCoachPersistenceSource(null); + setCoachSyncWarning(null); + coachCreditSeenRef.current = new Set(); setCoachHistoryOpen(false); setCoachHistoryTurnSeq(null); }, [navigate, routeIsSessionId, routeParam]); @@ -1001,6 +1054,48 @@ export default function Session() { setSignalSeq((seq) => [...seq.slice(-4), tone]); }, []); + const showCoachCreditPulse = useCallback( + (event: LiveCoachCreditEvent) => { + setCoachCreditPulse(event); + if (coachCreditPulseTimerRef.current) { + window.clearTimeout(coachCreditPulseTimerRef.current); + } + coachCreditPulseTimerRef.current = window.setTimeout(() => { + setCoachCreditPulse(null); + coachCreditPulseTimerRef.current = null; + }, 2600); + pushSignal( + event.event_type === "recharge" ? "pos" : "neutral", + event.event_type === "recharge" ? "코칭 기회 충전" : "코칭 기회 사용", + ); + }, + [pushSignal], + ); + + const applyCoachCreditEvents = useCallback( + (events: LiveCoachCreditEvent[], animate: boolean) => { + setCoachCreditEvents(events); + const fresh: LiveCoachCreditEvent[] = []; + for (const event of events) { + if (coachCreditSeenRef.current.has(event.event_id)) continue; + coachCreditSeenRef.current.add(event.event_id); + fresh.push(event); + } + if (animate && fresh.length > 0) { + showCoachCreditPulse(fresh[fresh.length - 1]); + } + }, + [showCoachCreditPulse], + ); + + useEffect(() => { + return () => { + if (coachCreditPulseTimerRef.current) { + window.clearTimeout(coachCreditPulseTimerRef.current); + } + }; + }, []); + const applyCrisisGate = useCallback((resource?: CrisisResource | null) => { const fallback: CrisisResource = { title: "자살예방상담전화 109", @@ -1019,29 +1114,61 @@ export default function Session() { }, [pushSignal]); const refreshCoachHistory = useCallback( - async (sessionId = liveSessionId) => { + async ( + sessionId = liveSessionId, + options: { animateCredits?: boolean; surfaceErrors?: boolean; warnOnFailure?: boolean } = {}, + ): Promise => { + if (options.surfaceErrors) { + setCoachHistoryLoading(true); + setCoachHistoryError(null); + } if (!sessionId) { setCoachHistory([]); - return; + setCoachQuota(DEFAULT_COACH_QUOTA); + setCoachCreditEvents([]); + setCoachHistoryError(null); + setCoachPersistenceSource(null); + setCoachSyncWarning(null); + if (options.surfaceErrors) setCoachHistoryLoading(false); + return null; } try { const history = await sessionApi.liveCoachHistory(sessionId); setCoachHistory(history.events ?? []); + setCoachQuota(history.quota ?? DEFAULT_COACH_QUOTA); + setCoachPersistenceSource(history.source); + setCoachSyncWarning( + history.source === "runtime" + ? "코칭 이력이 DB에 확정 저장되지 않아 임시 이력으로 표시됩니다." + : null, + ); + setCoachHistoryError(null); + applyCoachCreditEvents(history.credit_events ?? [], !!options.animateCredits); + return history; } catch { - /* 코칭 이력 조회 실패는 회기 진행을 막지 않는다. */ + if (options.surfaceErrors) { + setCoachHistoryError("코칭 이력을 불러오지 못했습니다. 잠시 뒤 다시 열어 주세요."); + } + if (options.warnOnFailure) { + setCoachSyncWarning("코칭은 표시됐지만 이력 동기화를 확인하지 못했습니다. 이력 보기에서 다시 확인해 주세요."); + } + return null; + } finally { + if (options.surfaceErrors) setCoachHistoryLoading(false); } }, - [liveSessionId], + [applyCoachCreditEvents, liveSessionId], ); const openCoachHistory = useCallback( (turnSeq: number | null = null) => { setCoachHistoryTurnSeq(turnSeq); + setCoachHistoryError(null); setCoachEvidenceOpen(false); setCoachHistoryOpen(true); - void refreshCoachHistory(); + void refreshCoachHistory(liveSessionId, { surfaceErrors: true }); }, - [refreshCoachHistory], + [liveSessionId, refreshCoachHistory], ); const requestLiveCoach = useCallback( @@ -1055,6 +1182,25 @@ export default function Session() { turnSeq?: number; }) => { if (!liveSessionId || feedbackMode !== "coached") return; + let availableCoachCredits = coachQuota.remaining ?? 0; + if (availableCoachCredits <= 0) { + setCoachError(null); + const refreshed = await refreshCoachHistory(liveSessionId, { + animateCredits: true, + warnOnFailure: true, + }); + if (!refreshed) { + setCoachError("코칭 기회를 확인하지 못했습니다. 잠시 뒤 다시 시도해 주세요."); + pushSignal("warn", "코칭 기회 확인 실패"); + return; + } + availableCoachCredits = refreshed.quota?.remaining ?? 0; + } + if (availableCoachCredits <= 0) { + setCoachError("코칭 기회를 모두 사용했습니다. 좋은 발화로 내담자의 변화 신호가 생기면 1개씩 다시 충전됩니다."); + pushSignal("warn", "코칭 기회 소진"); + return; + } setCoachLoading(true); setCoachError(null); try { @@ -1064,16 +1210,41 @@ export default function Session() { turn_seq: turnSeq, }); setCoachSuggestion(suggestion); + setCoachPersistenceSource(suggestion.persistence_source ?? null); + setCoachSyncWarning( + suggestion.persistence_source === "runtime" + ? "코칭 이력이 DB에 확정 저장되지 않아 임시 이력으로 표시됩니다." + : null, + ); + if (suggestion.quota) { + setCoachQuota(suggestion.quota); + } + if (suggestion.credit_events?.length) { + applyCoachCreditEvents(suggestion.credit_events, true); + } pushSignal(suggestion.tone, suggestion.title); - void refreshCoachHistory(); - } catch { - setCoachError("코칭 근거를 불러오지 못했습니다. 다음 턴에서 다시 시도합니다."); - pushSignal("warn", "코칭 연결 실패"); + void refreshCoachHistory(liveSessionId, { animateCredits: false, warnOnFailure: true }); + } catch (err) { + if (err instanceof ApiError && err.status === 409) { + setCoachQuota((current) => ({ ...current, remaining: 0 })); + setCoachError("코칭 기회를 모두 사용했습니다. 좋은 발화가 실제 변화로 이어지면 다시 충전됩니다."); + pushSignal("warn", "코칭 기회 소진"); + } else { + setCoachError("코칭 근거를 불러오지 못했습니다. 다음 턴에서 다시 시도합니다."); + pushSignal("warn", "코칭 연결 실패"); + } } finally { setCoachLoading(false); } }, - [feedbackMode, liveSessionId, pushSignal, refreshCoachHistory], + [ + applyCoachCreditEvents, + coachQuota.remaining, + feedbackMode, + liveSessionId, + pushSignal, + refreshCoachHistory, + ], ); useEffect(() => { @@ -1125,6 +1296,13 @@ export default function Session() { setCoachSuggestion(null); setCoachError(null); setCoachEvidenceOpen(false); + setCoachHistory([]); + setCoachQuota(DEFAULT_COACH_QUOTA); + setCoachCreditEvents([]); + setCoachCreditPulse(null); + setCoachPersistenceSource(null); + setCoachSyncWarning(null); + coachCreditSeenRef.current = new Set(); setCoachHistoryOpen(false); setCoachHistoryTurnSeq(null); setClientReplyPending(false); @@ -1171,6 +1349,12 @@ export default function Session() { setCoachError(null); setCoachEvidenceOpen(false); setCoachHistory([]); + setCoachQuota(DEFAULT_COACH_QUOTA); + setCoachCreditEvents([]); + setCoachCreditPulse(null); + setCoachPersistenceSource(null); + setCoachSyncWarning(null); + coachCreditSeenRef.current = new Set(); setCoachHistoryOpen(false); setCoachHistoryTurnSeq(null); if (res.degraded) { @@ -1684,6 +1868,7 @@ export default function Session() { } if (payload.type === "transcript" && payload.final && payload.text) { + failPendingVoiceLearnerTurn(); const id = nextId(); pendingVoiceLearnerIdRef.current = id; pendingVoiceLearnerTextRef.current = payload.text; @@ -1712,7 +1897,9 @@ export default function Session() { if (pendingId != null) { setUtterances((prev) => prev.map((u) => - u.id === pendingId ? { ...u, partial: false, turnSeq: payload.turn_seq } : u, + u.id === pendingId + ? { ...u, partial: false, failed: false, turnSeq: payload.turn_seq } + : u, ), ); pendingVoiceLearnerIdRef.current = null; @@ -1729,6 +1916,9 @@ export default function Session() { turnSeq: payload.turn_seq, }); } + if (conversationStopped) { + closeVoiceSocket(); + } return; } @@ -1739,9 +1929,11 @@ export default function Session() { if (payload.type === "degraded") { setClientReplyPending(false); + const hadPendingLearner = failPendingVoiceLearnerTurn(); const fallbackReason = "음성 설정이 완료되지 않아 지금은 텍스트 입력으로 진행합니다."; const reason = payload.reason && !payload.reason.includes("OPENAI") ? payload.reason : fallbackReason; + if (hadPendingLearner) setTurnError(fallbackReason); shutdownVoice("degraded", reason); pushSignal("warn", "음성 기능 미설정"); return; @@ -1749,24 +1941,24 @@ export default function Session() { if (payload.type === "error") { setClientReplyPending(false); - if (pendingVoiceLearnerIdRef.current != null && payload.detail?.includes("engine unavailable")) { - const pendingId = pendingVoiceLearnerIdRef.current; - setUtterances((prev) => prev.filter((u) => u.id !== pendingId)); - pendingVoiceLearnerIdRef.current = null; - pendingVoiceLearnerTextRef.current = ""; - } - shutdownVoice("error", payload.detail || "음성 처리 중 오류가 발생했습니다."); + failPendingVoiceLearnerTurn(); + const detail = userFacingVoiceError(payload); + setTurnError(detail); + shutdownVoice("error", detail); pushSignal("warn", "음성 오류"); } }; ws.onerror = () => { setClientReplyPending(false); - shutdownVoice("error", "음성 연결을 확인하지 못했습니다."); + failPendingVoiceLearnerTurn(); + setTurnError(VOICE_CONNECTION_SAVE_FAILED); + shutdownVoice("error", VOICE_CONNECTION_SAVE_FAILED); pushSignal("warn", "음성 연결 실패"); }; ws.onclose = () => { + const hadPendingLearner = failPendingVoiceLearnerTurn(); const capture = voiceCaptureRef.current; voiceCaptureRef.current = null; capture?.abort(); @@ -1776,6 +1968,13 @@ export default function Session() { } stopMicStream(); setMicOn(false); + setClientReplyPending(false); + if (hadPendingLearner) { + setTurnError(VOICE_CONNECTION_SAVE_FAILED); + setVoiceStatus("error"); + setVoiceDetail(VOICE_CONNECTION_SAVE_FAILED); + return; + } setVoiceStatus((current) => (current === "degraded" || current === "error" ? current : "idle")); setVoiceDetail((current) => { if ( @@ -1794,6 +1993,7 @@ export default function Session() { closeVoiceSocket, elapsed, ensureVoiceAudioContext, + failPendingVoiceLearnerTurn, liveSessionId, paused, sessionEnded, @@ -2019,7 +2219,7 @@ export default function Session() { const elapsedLabel = formatElapsed(elapsed); const recommendedSessionSeconds = 30 * 60; const remainingLabel = formatElapsed(Math.max(0, recommendedSessionSeconds - elapsed)); - const turnCount = utterances.filter((utterance) => !utterance.partial).length; + const turnCount = utterances.filter((utterance) => !utterance.partial && !utterance.failed).length; const latestClientUtterance = [...utterances] .reverse() .find((utterance) => utterance.speaker === "client" && utterance.text.trim()); @@ -2051,10 +2251,25 @@ export default function Session() { : null; const coachSources = coachSuggestion?.sources ?? []; const coachTone = coachSuggestion?.tone ?? "neutral"; + const coachQuotaMax = Math.max(1, coachQuota.max ?? DEFAULT_COACH_QUOTA.max); + const coachQuotaRemaining = Math.max( + 0, + Math.min(coachQuotaMax, coachQuota.remaining ?? DEFAULT_COACH_QUOTA.remaining), + ); + const coachQuotaSlots = Array.from({ length: coachQuotaMax }, (_, index) => index); + const coachCreditPulseText = + coachCreditPulse == null + ? null + : coachCreditPulse.event_type === "recharge" + ? `+1 충전 · ${coachCreditPulse.balance}/${coachQuotaMax}` + : `-1 사용 · ${coachCreditPulse.balance}/${coachQuotaMax}`; + const coachIsDegraded = coachSuggestion?.status === "degraded"; + const coachDegradedNote = + "AI 코칭 엔진 응답 대신 워크북 규칙과 현재 턴 신호로 만든 대체 제안입니다."; const coachStatusText = coachLoading ? "코치가 근거를 확인 중" - : coachSuggestion?.status === "degraded" - ? "규칙 기반 코칭" + : coachIsDegraded + ? "AI 응답 대체" : coachSuggestion ? "근거 확인 완료" : "턴 완료 후 개입"; @@ -2525,7 +2740,8 @@ export default function Session() { className={ "sx-utt " + (u.speaker === "client" ? "is-client" : "is-learner") + - (u.partial ? " is-partial" : "") + (u.partial ? " is-partial" : "") + + (u.failed ? " is-failed" : "") } >
@@ -2539,6 +2755,11 @@ export default function Session() { {u.text} {u.partial ? : null}
+ {u.failed ? ( + + 저장 실패 · 다시 시도하세요. + + ) : null} {turnCoachEvents.length && u.turnSeq ? (
- ) : coachError ? ( -

{coachError}

+ ) : coachQuotaRemaining <= 0 ? ( +

코칭 기회를 모두 사용했습니다. 좋은 발화로 내담자가 열리면 다시 1개가 충전됩니다.

) : (

코칭 모드에서는 방금 발화의 강점과 조정점을 근거와 함께 바로 짚습니다.

)} @@ -2928,8 +3196,17 @@ export default function Session() { @@ -2992,6 +3269,24 @@ export default function Session() {

회기 중 실제로 받은 코칭과 근거를 시간순으로 확인합니다.

+ {coachPersistenceSource === "runtime" ? ( +
+ 현재 코칭 이력은 임시 저장소 기준입니다. +
+ ) : null} + {coachCreditEvents.length > 0 ? ( +
+ {coachCreditEvents.slice(-4).map((event) => ( + + {event.event_type === "recharge" ? "+1 충전" : "-1 사용"} ·{" "} + {event.balance}/{coachQuotaMax} + + ))} +
+ ) : null} -
현재

텍스트 /turn·/stream·음성 경로에서 fast-loop 턴 평가를 app.feedback_scores, app.turn_technique, app.turn_client_state, app.supervisor_comment, app.alternative_utterance로 정규화 적재 후 리뷰 조회에서 hydrate한다. 이번 패스에서 app/services/live_coach.py, data/kb/live_coaching_workbook_0615.json, data/kb/live_coaching_sources/*.json, POST /sessions/{id}/live-coach, GET /sessions/{id}/live-coach, POST /kb/live-coach/source-packs/sync, app.live_coach_events를 추가했다. 세션 UI는 코칭 모드에서 AI 코치 아바타 말풍선으로 즉시 개입하고, "근거 보기" 모달과 학습자 발화 우측 C 마커를 통해 받은 코칭 이력을 채팅 위 스크롤 오버레이로 보여준다. 저장 payload는 PII 마스킹 excerpt + 코칭 구조화 JSON이며, 공식 자료는 허가된 source pack의 version/citation/summary로 넣고 장문 원문·공식 문항을 재현하지 않는다. 같은 source pack은 app.services.source_pack_sync를 통해 evaluator 전용 RAG에 증분 색인하며, hash 변경 시 최신 active document version+1로 새 문서를 만든다. 74차에서는 로컬 source pack 캐시의 key/lifetime/무효화 경계를 명시해 live turn은 API 프로세스 snapshot을 재사용하고, 관리자 sync/CLI만 refresh=True로 repo 파일을 다시 읽게 했다. 학습자 SessionReview 데스크톱은 요약/흐름, 축어록, 평가 rail의 3열 workbench와 하단 워크시트로 재배치했고, 교수자 검토 화면과 모바일/태블릿은 기존 반응형 규칙을 유지한다.

검증

python -m compileall app -q, 과거 pytest app/test_live_coach_sources.py app/test_orchestrator_masking.py app/test_session_turn_persistence.py -q 27 passed, source sync focused py -3.11 -X utf8 -m pytest -p no:cacheprovider app/test_live_coach_sources.py -q 9 passed, H2 related py -3.11 -X utf8 -m pytest -p no:cacheprovider app/test_live_coach_sources.py app/test_orchestrator_masking.py app/test_session_turn_persistence.py -q 44 passed, py -3.11 -X utf8 scripts\sync-persona-sources.py --help, py -3.11 -X utf8 -m py_compile focused files, npm run generate:api-types, npm run check:api-types, npm run typecheck, npm run build, npm run e2e -- e2e/layout-visual-gate.spec.ts e2e/session-layout.spec.ts 15 passed. 최신 workbench 검증: npm run typecheck, npm run build, npx playwright test e2e/session-review.spec.ts --project=chromium-desktop --workers=1 3 passed, npx playwright test e2e/layout-visual-gate.spec.ts --project=chromium-single-run --workers=1 9 passed, 04-session-review-desktop.png 직접 확인. 로컬 DB smoke: source pack sync 4 sources/15 chunks embedded, 재호출 skipped_unchanged=4, official_suicide_risk_guidelines source filter 검색 성공.

잔여

임상팀 골든셋·라이브 코칭 루브릭 검수, 공식 지침/논문 source pack 추가와 임상 검수 상태 운영정책 보강이 남아 있다.

+
현재

텍스트 /turn·/stream·음성 경로에서 fast-loop 턴 평가를 app.feedback_scores, app.turn_technique, app.turn_client_state, app.supervisor_comment, app.alternative_utterance로 정규화 적재 후 리뷰 조회에서 hydrate한다. 평가 훅 예외와 evaluator error dict는 정상 neutral 점수로 둔갑하지 않게 evaluation.error로 남기고, 리뷰에는 턴 평가 실패 노트로 표면화하며, 성장 점수·최근 피드백 집계에서는 제외한다. 이번 패스에서 app/services/live_coach.py, data/kb/live_coaching_workbook_0615.json, data/kb/live_coaching_sources/*.json, POST /sessions/{id}/live-coach, GET /sessions/{id}/live-coach, POST /kb/live-coach/source-packs/sync, app.live_coach_events를 추가했다. 세션 UI는 코칭 모드에서 AI 코치 아바타 말풍선으로 즉시 개입하고, "근거 보기" 모달과 학습자 발화 우측 C 마커를 통해 받은 코칭 이력을 채팅 위 스크롤 오버레이로 보여준다. 저장 payload는 PII 마스킹 excerpt + 코칭 구조화 JSON이며, 공식 자료는 허가된 source pack의 version/citation/summary로 넣고 장문 원문·공식 문항을 재현하지 않는다. 같은 source pack은 app.services.source_pack_sync를 통해 evaluator 전용 RAG에 증분 색인하며, hash 변경 시 최신 active document version+1로 새 문서를 만든다. 74차에서는 로컬 source pack 캐시의 key/lifetime/무효화 경계를 명시해 live turn은 API 프로세스 snapshot을 재사용하고, 관리자 sync/CLI만 refresh=True로 repo 파일을 다시 읽게 했다. 학습자 SessionReview 데스크톱은 요약/흐름, 축어록, 평가 rail의 3열 workbench와 하단 워크시트로 재배치했고, 교수자 검토 화면과 모바일/태블릿은 기존 반응형 규칙을 유지한다.

검증

python -m compileall app -q, 과거 pytest app/test_live_coach_sources.py app/test_orchestrator_masking.py app/test_session_turn_persistence.py -q 27 passed, source sync focused py -3.11 -X utf8 -m pytest -p no:cacheprovider app/test_live_coach_sources.py -q 9 passed, H2 related py -3.11 -X utf8 -m pytest -p no:cacheprovider app/test_live_coach_sources.py app/test_orchestrator_masking.py app/test_session_turn_persistence.py -q 44 passed, fast-loop failure focused py -3.11 -X utf8 -m pytest -p no:cacheprovider app/test_session_turn_persistence.py -k "fast_loop_evaluation" -q 2 passed, failed metric exclusion focused py -3.11 -X utf8 -m pytest -p no:cacheprovider app/test_learner_dashboard.py -k "failed_fast_loop" -q 1 passed, py -3.11 -X utf8 scripts\sync-persona-sources.py --help, py -3.11 -X utf8 -m py_compile focused files, npm run generate:api-types, npm run check:api-types, npm run typecheck, npm run build, npm run e2e -- e2e/layout-visual-gate.spec.ts e2e/session-layout.spec.ts 15 passed. 최신 workbench 검증: npm run typecheck, npm run build, npx playwright test e2e/session-review.spec.ts --project=chromium-desktop --workers=1 3 passed, npx playwright test e2e/layout-visual-gate.spec.ts --project=chromium-single-run --workers=1 9 passed, 04-session-review-desktop.png 직접 확인. 로컬 DB smoke: source pack sync 4 sources/15 chunks embedded, 재호출 skipped_unchanged=4, official_suicide_risk_guidelines source filter 검색 성공.

잔여

임상팀 골든셋·라이브 코칭 루브릭 검수, 공식 지침/논문 source pack 추가와 임상 검수 상태 운영정책 보강이 남아 있다.

@@ -730,7 +743,7 @@
-
현재

X1: scripts/export-recursive-dataset.py와 app.services.dataset_export로 masked-text JSONL dry-run, PII scan, κ/ICC 계산, approved export 게이트를 구현했다. 기본은 technical_dry_run이고 ds.* write는 --write-dataset 명시 시에만 수행한다. X2: app.turns의 provider/model/tokens/cost를 최근 7일 기준으로 집계하는 GET /admin/usage를 추가했고, 운영 콘솔 /admin에 누적 비용·입출력 토큰·계량 커버리지·provider/model breakdown, 일별 daily_cost 추이, ADMIN_USAGE_BUDGET_USD 기반 예산 경고(ok/warn/exceeded)를 표시한다. EVALUATOR_FAST_MODEL/EVALUATOR_DEEP_MODEL 설정 시 fast/deep 평가 호출만 해당 모델 override로 gateway에 전달하고, 비워두면 기존 default 라우팅을 유지한다. fast/deep evaluator structured 결과는 canonical request SHA-256 키의 인메모리 semantic cache로 재사용한다. 캐시는 EVALUATOR_SEMANTIC_CACHE_ENABLED, EVALUATOR_SEMANTIC_CACHE_TTL_SECONDS, EVALUATOR_SEMANTIC_CACHE_MAX_ENTRIES로 제한하며, 원문 prompt·completion은 저장하지 않고 성공 파싱 결과만 캐시한다. /admin/usage는 evaluator cache enabled/entries/hits/misses/stores/evictions/requests/hit_rate도 함께 반환하고, 관리자 AI 비용 카드가 hit-rate와 일별 비용/턴 추이를 표시한다. app.services.usage_report와 scripts/report-ai-usage.py는 같은 usage JSON에서 provider/model별 cost share, token share, cost/turn, cost/1k tokens, budget/cache warning을 산출한다. DB 미가용 dev에서는 runtime store로 fallback하지만 prod에서는 fail-closed한다. L1: 소유자 결정으로 Node.js 우선 교체 가능성 방향을 채택했다. docs/decisions/backend-node-transition.md에 FastAPI 유지 사유와 strangler 전환 원칙을 기록했고, app/contracts/engine_gateway.py를 추가해 Python client와 현재 gateway가 같은 요청/응답/SSE 계약을 공유한다. 이어서 EngineClient.stream_packets()와 gateway GenerateResponse 반환으로 서비스 레이어의 wire-format 의존을 줄였다. engine_gateway_contract.v1.json golden fixture와 engine_gateway_schema.v1.json schema artifact로 generate request/response와 stream token/done/error packet을 고정했고, scripts/check-engine-gateway-contract.mjs가 같은 artifact를 Node.js에서 Python import 없이 검증한다. 78차에서는 gateway-default 기본 라우팅 sentinel과 모델 override 정규화를 같은 계약 모듈로 올려 FastAPI client, Python gateway, stream audit/done fallback, 관리자 기본 config가 같은 의미를 공유하게 했다. 79차에서는 structured_payload_from_response()가 structured field 우선, fenced JSON, embedded JSON object fallback을 소유해 evaluator/live-coach/persona draft의 legacy generate response 해석을 같은 계약 경계로 모았다. 80차에서는 app/persona_generation_contract.py가 persona draft schema, prompt bundle, payload extraction, generated draft coercion을 소유하게 했고, 81차/89차에서는 evaluator/live-coach가 structured payload 계약 helper를 직접 호출하며 SessionEvaluationWrite.from_result()/from_error()가 session evaluation write packet 생성을 소유하게 했다. app/session_read_model.py는 세션 목록·대시보드·상세·리뷰·공유 payload의 DTO와 builder를 route에서 분리했고, app/persona_read_model.py는 페르소나 catalog/review/draft/source/evidence DTO와 mapper를 route에서 분리했다. 각 route는 auth/RLS DB read/persistence/RAG/LLM side effect를 유지한다.

권고

X1 approved export·골든셋 승격은 steward/legal 승인, reviewer disposition, IAA 게이트(κ≥0.6/ICC≥0.75) 이후만 진행한다. X2 후속은 자동 차단·한도 enforcement 정책이다. L1의 남은 게이트는 신청서/저작권 등재 문서에 FastAPI 유지 사유와 Node 전환 계획을 반영하는 외부 거버넌스 증거다. H3 항목형 목록 저작 UI와 프롬프트 미리보기 de-JSON은 60~61차에서 처리됐다. 실제 운영 말뭉치 기반 H4 평가는 외부 데이터·reviewer 정답 라벨·법무/guardian evidence 전까지 gate로 남기고, 내부 후보였던 합성 fixture 확장과 운영 평가 입력/report schema 준비는 62차에서 technical_dry_run으로 완료했고, 실제 운영 평가는 여전히 gate다.

+
현재

X1: scripts/export-recursive-dataset.py와 app.services.dataset_export로 masked-text JSONL dry-run, PII scan, κ/ICC 계산, approved export 게이트를 구현했다. exporter는 동의가 남아 있고 learner-visible인 text_masked 턴만 선별하며, supervisor comment raw text는 JSONL에 넣지 않는다. artifact checker는 approved/dry-run JSONL의 필수 key·row count·privacy·PII shape를 검사한다. 기본은 technical_dry_run이고 ds.* write는 --write-dataset 명시 시에만 수행한다. X2: app.turns의 provider/model/tokens/cost를 최근 7일 기준으로 집계하는 GET /admin/usage를 추가했고, 운영 콘솔 /admin에 누적 비용·입출력 토큰·계량 커버리지·provider/model breakdown, 일별 daily_cost 추이, ADMIN_USAGE_BUDGET_USD 기반 예산 경고(ok/warn/exceeded)를 표시한다. EVALUATOR_FAST_MODEL/EVALUATOR_DEEP_MODEL 설정 시 fast/deep 평가 호출만 해당 모델 override로 gateway에 전달하고, 비워두면 기존 default 라우팅을 유지한다. fast/deep evaluator structured 결과는 canonical request SHA-256 키의 인메모리 semantic cache로 재사용한다. 캐시는 EVALUATOR_SEMANTIC_CACHE_ENABLED, EVALUATOR_SEMANTIC_CACHE_TTL_SECONDS, EVALUATOR_SEMANTIC_CACHE_MAX_ENTRIES로 제한하며, 원문 prompt·completion은 저장하지 않고 성공 파싱 결과만 캐시한다. /admin/usage는 evaluator cache enabled/entries/hits/misses/stores/evictions/requests/hit_rate도 함께 반환하고, 관리자 AI 비용 카드가 hit-rate와 일별 비용/턴 추이를 표시한다. app.services.usage_report와 scripts/report-ai-usage.py는 같은 usage JSON에서 provider/model별 cost share, token share, cost/turn, cost/1k tokens, budget/cache warning을 산출한다. DB 미가용 dev에서는 runtime store로 fallback하지만 prod에서는 fail-closed한다. L1: 소유자 결정으로 Node.js 우선 교체 가능성 방향을 채택했다. docs/decisions/backend-node-transition.md에 FastAPI 유지 사유와 strangler 전환 원칙을 기록했고, app/contracts/engine_gateway.py를 추가해 Python client와 현재 gateway가 같은 요청/응답/SSE 계약을 공유한다. 이어서 EngineClient.stream_packets()와 gateway GenerateResponse 반환으로 서비스 레이어의 wire-format 의존을 줄였다. engine_gateway_contract.v1.json golden fixture와 engine_gateway_schema.v1.json schema artifact로 generate request/response와 stream token/done/error packet을 고정했고, scripts/check-engine-gateway-contract.mjs가 같은 artifact를 Node.js에서 Python import 없이 검증한다. 78차에서는 gateway-default 기본 라우팅 sentinel과 모델 override 정규화를 같은 계약 모듈로 올려 FastAPI client, Python gateway, stream audit/done fallback, 관리자 기본 config가 같은 의미를 공유하게 했다. 79차에서는 structured_payload_from_response()가 structured field 우선, fenced JSON, embedded JSON object fallback을 소유해 evaluator/live-coach/persona draft의 legacy generate response 해석을 같은 계약 경계로 모았다. 80차에서는 app/persona_generation_contract.py가 persona draft schema, prompt bundle, payload extraction, generated draft coercion을 소유하게 했고, 81차/89차에서는 evaluator/live-coach가 structured payload 계약 helper를 직접 호출하며 SessionEvaluationWrite.from_result()/from_error()가 session evaluation write packet 생성을 소유하게 했다. app/session_read_model.py는 세션 목록·대시보드·상세·리뷰·공유 payload의 DTO와 builder를 route에서 분리했고, app/persona_read_model.py는 페르소나 catalog/review/draft/source/evidence DTO와 mapper를 route에서 분리했다. 각 route는 auth/RLS DB read/persistence/RAG/LLM side effect를 유지한다.

권고

X1 approved export·골든셋 승격은 steward/legal 승인, reviewer disposition, IAA 게이트(κ≥0.70/ICC≥0.75) 이후만 진행한다. X2 후속은 자동 차단·한도 enforcement 정책이다. L1의 남은 게이트는 신청서/저작권 등재 문서에 FastAPI 유지 사유와 Node 전환 계획을 반영하는 외부 거버넌스 증거다. H3 항목형 목록 저작 UI와 프롬프트 미리보기 de-JSON은 60~61차에서 처리됐다. 실제 운영 말뭉치 기반 H4 평가는 외부 데이터·reviewer 정답 라벨·법무/guardian evidence 전까지 gate로 남기고, 내부 후보였던 합성 fixture 확장과 운영 평가 입력/report schema 준비는 62차에서 technical_dry_run으로 완료했고, 실제 운영 평가는 여전히 gate다.

@@ -770,7 +783,7 @@
-
현재

LogHook/log_hook(caller 0) 타입·파라미터·분기·export 제거 · 게이트웨이/엔진클라 tier dead-contract 제거(write-only, gateway는 model만 사용). 잔여(RMS 힌트·Live2D 고아 자산·파라미터객체 init_state/prepare_turn/upsert)는 docs/ops/refactor-rag-patches-2026-06-26.{md,json}에 스테이징(단계 적용).

참고

일괄 적용 대신 라이브 스택 보호 위해 그룹별 적용+pytest 검증.

+
현재

LogHook/log_hook(caller 0) 타입·파라미터·분기·export 제거 · 게이트웨이/엔진클라 tier dead-contract 제거(write-only, gateway는 model만 사용). 잔여(RMS 힌트·Live2D 고아 자산·파라미터객체 init_state/prepare_turn/upsert)는 docs/archive/ops/refactor-rag-patches-2026-06-26.{md,json}에 스테이징(단계 적용).

참고

일괄 적용 대신 라이브 스택 보호 위해 그룹별 적용+pytest 검증.

@@ -858,7 +871,7 @@
검증

node scripts/check-engine-gateway-contract.mjs --json, py -3.11 -X utf8 -B -m pytest -p no:cacheprovider engine_gateway/test_gateway_model.py -q 27 passed, py -3.11 -X utf8 -B -m pytest -p no:cacheprovider engine_gateway/test_gateway_model.py app/test_evaluation_persistence.py app/test_evaluator_model_routing.py app/test_session_turn_persistence.py app/test_live_coach_sources.py app/test_persona_generation_contract.py app/test_persona_review.py -q 120 passed, npm run check:api-types, npm run typecheck.

DONE
상주 claude -p 엔진풀 실증

session_id가 살아 있는 gateway session을 재사용하고 missing session은 ephemeral로 닫히는 구조를 회귀화했고, live 게이트웨이(9099)로 실측까지 마쳤다. probe-engine-gateway.py가 단일 session_id로 2회 reused stream을 돌려 TTFT 1667–4199ms, cost 누적 0.068→0.123(turns 1→2), engine=claude_p, model=opus-4-8을 측정했다.

산출물

engine_gateway/test_gateway_model.py, scripts/probe-engine-gateway.py, live probe JSON

검증

session reuse 7 tests OK; live probe 2 reused streams, TTFT/cost 실측, session_id 재사용 확인

-
DONE
마스킹 게이트 + LLM call audit

current turn, recall, pinned fact, recent turns가 Presidio/가명처리 후에만 engine request로 들어가게 막았다. 한국어 이름/기관 로컬 휴리스틱 1차와 16-case summary-only 합성 fixture 평가 harness/input-report schema를 추가했고, fast evaluator prompt와 client turn text_masked도 마스킹본을 쓰게 보강했다. 66차에서는 자연 발화형 이름 라벨·자기소개 케이스를 추가해 라벨 단어만 마스킹하고 실명을 남기던 구멍을 막았다. 70차에서는 선택형 ko recognizer adapter를 mask_pii() 경계에 추가하고, adapter 성공/실패 모두 regex fallback이 마지막 안전망으로 남는지 회귀화했다. 외부 LLM 호출은 audit.llm_call_log에 provider/model/token/cost/inference_geo/latency metadata만 남기고 prompt/completion 본문은 저장하지 않는다.

산출물

app/services/guardrail.py, app/services/pii_masking_eval.py, data/privacy/pii-masking-ko-fixtures.json, data/privacy/pii-masking-eval-input.schema.json, data/privacy/pii-masking-eval-report.schema.json, scripts/evaluate-pii-masking.py, app/test_pii_masking_eval.py, app/test_orchestrator_masking.py, app/test_evaluation_persistence.py, app/test_session_turn_persistence.py, session_persistence.record_llm_call_audit

검증

phone/email/RRN 및 한국어 NAME/ORG raw 값이 generate/stream/evaluator payload에 없음, adapter fake span과 fallback phone 마스킹 경로 검증, adapter 실패 시 regex fallback 유지, 감사 payload 본문 미포함, fixture 16/16 pass, 최신 보정 회귀 59 passed, 현재 전체 API 178 기준 유지, 로컬 live /turn smoke 후 audit.llm_call_log 8→11

+
DONE
마스킹 게이트 + LLM call audit

current turn, recall, pinned fact, recent turns와 live coach recent/evaluation prompt, turn-evaluation persistence rationale/comment/alternative, legacy DB row text fallback, session-end deep evaluation payload/read-model이 Presidio/가명처리 후에만 engine request/DB 저장·복원·리뷰 응답 경계로 들어가게 막았다. 한국어 이름/기관 로컬 휴리스틱 1차와 16-case summary-only 합성 fixture 평가 harness/input-report schema를 추가했고, fast evaluator prompt와 client turn text_masked도 마스킹본을 쓰게 보강했다. 66차에서는 자연 발화형 이름 라벨·자기소개 케이스를 추가해 라벨 단어만 마스킹하고 실명을 남기던 구멍을 막았다. 70차에서는 선택형 ko recognizer adapter를 mask_pii() 경계에 추가하고, adapter 성공/실패 모두 regex fallback이 마지막 안전망으로 남는지 회귀화했다. 외부 LLM 호출은 audit.llm_call_log에 provider/model/token/cost/inference_geo/latency metadata만 남기고 prompt/completion 본문은 저장하지 않는다.

산출물

app/services/guardrail.py, app/services/pii_masking_eval.py, data/privacy/pii-masking-ko-fixtures.json, data/privacy/pii-masking-eval-input.schema.json, data/privacy/pii-masking-eval-report.schema.json, scripts/evaluate-pii-masking.py, app/test_pii_masking_eval.py, app/test_orchestrator_masking.py, app/test_evaluation_persistence.py, app/test_session_turn_persistence.py, app/test_live_coach_privacy.py, session_persistence.record_llm_call_audit

검증

phone/email/RRN 및 한국어 NAME/ORG raw 값이 generate/stream/evaluator payload에 없음, adapter fake span과 fallback phone 마스킹 경로 검증, adapter 실패 시 regex fallback 유지, 감사 payload 본문 미포함, fixture 16/16 pass, 최신 보정 회귀 59 passed, H4 prompt/persistence privacy 50 passed, 현재 전체 API 178 기준 유지, 로컬 live /turn smoke 후 audit.llm_call_log 8→11

DONE
P1 상담 완주 플로우

로컬 DB-backed 런타임에서 P1 선택, SSE 상담, 종료, 회기말 리뷰 피드백을 한 번에 통과시켰다. 실제 운영 로그인 proof는 P0 public OAuth blocker와 분리해 남긴다.

산출물

e2e/session-mvp.spec.ts, turn telemetry 저장 경로

검증

npx playwright test e2e/session-mvp.spec.ts --project=chromium-single-run --workers=1 1 passed.

DONE
저항 엔진 정량 검증

공감 발화와 조언점프 발화 5턴 세트를 상태머신 회귀 테스트와 실제 API+DB smoke로 비교했다. live smoke는 dev-login→onboarding→P1 세션 2개→각 5턴을 수행하고 app.session_state, app.turns, app.turn_client_state를 직접 조회했다. 공감 세션은 3턴부터 탐색으로 열리고 openness가 0.171까지 상승했다. 조언점프 세션은 라포와 openness 0.0을 유지했다.

산출물

app/test_state_machine_resistance.py, scripts/smoke-resistance-openness-db.py, docs/ops/resistance-openness-db-smoke-2026-06-28.json

검증

pytest app/test_state_machine_resistance.py app/test_evaluation_persistence.py -q 9 passed; live DB smoke PASS.

DONE
음성 왕복 1턴 PoC

Session UI 마이크 버튼 경로에서 synthetic getUserMedia/MediaRecorder로 실제 /voice/ws를 열고 audio_start, binary chunk, audio_end, transcript, reply, TTS 재생 상태까지 검증했다. 추가로 로컬 live API에서 실제 STT/TTS provider를 통과해 TTS 바이너리를 수신했다. 물리 마이크, 공개 WSS, 50분 양방향 실측은 별도 운영 smoke로 남아 있다.

산출물

apps/web/e2e/voice-success.spec.ts, controlled OpenAI/engine providers, Web Audio browser voice UI probe, live /voice/ws smoke

검증

npx playwright test e2e/voice-success.spec.ts --project=chromium-single-run 2 passed; live P2 WS smoke binary 66 chunks/266,496 bytes

@@ -881,7 +894,7 @@
LOCAL DONE
voice gateway WebSocket

/voice/ws backend contract를 fake WebSocket으로 회귀화했고, Session UI는 AudioWorklet-first PCM16 capture와 MediaRecorder fallback으로 한 턴 synthetic audio를 통과했다. 백엔드는 format=pcm raw chunk를 STT 업로드 직전 WAV로 감싼다. 로컬 API에서는 실제 OpenAI STT/TTS 경로 live WS smoke도 통과했다.

산출물

app/test_voice_ws.py, apps/web/public/worklets/voice-capture-worklet.js, apps/web/e2e/voice-success.spec.ts, audio_start/binary/audio_end/text_turn/stt_result/ping contract

검증

app/test_voice_ws.py app/test_voice_service.py 27 passed; voice-success chromium-single-run 2 passed; live P2 WS smoke transcript 52자, reply 105자, TTS binary 66 chunks/266,496 bytes.

LOCAL DONE
Streaming STT + EOT

provider-neutral EOT helper와 route-level stt_result control을 연결했다. final transcript와 silence threshold가 ready일 때만 턴을 실행하고, pending이면 state=listening으로 돌아간다. 이 행은 로컬 route 계약 완료 범위다.

산출물

assess_end_of_turn, EndOfTurnDecision, stt_result control, EOT silence threshold

검증

app/test_voice_ws.py app/test_voice_service.py 27 passed; final pending, empty transcript, silence threshold, route-level ready/pending 케이스

DONE
OpenAI TTS voice preset 추상화

P1/P2/P3 persona voice preset, invalid preset fallback, speed clamp, instructions payload 계약을 회귀화했고, app.persona_voice_map을 live /voice/ws TTS 선택에 연결했다. 명시 preset은 DB map보다 우선하고, OpenAI가 아닌 provider row는 기존 fallback으로 안전하게 흡수한다.

산출물

resolve_voice, resolve_voice_from_map, build_tts_payload, get_session_voice_map, seed voice map row, dev runtime schema guard

검증

pytest app/test_runtime_policy.py app/test_persona_review.py app/test_voice_service.py app/test_voice_ws.py -q 75 passed; full API 178 passed + gateway 11 passed.

-
POC DONE
P1 서연 무참조 synthetic voice 샘플

Higgs v3 로컬 런타임에서 실존 reference voice 없이 synthetic seed를 먼저 만들고, 그 seed만 재참조해 우울·피로·불안·라포·회복감 5개 변주를 생성했다. 개발용 VIGNETTE_VOICE_POC_SAMPLE_TTS 플래그로 /voice/ws P1 sample TTS provider에도 연결했다.

산출물

docs/voice-art/bimec-persona-voice-poc-2026-06-27.md, docs/voice-art/p1-seoyeon-higgs-v3-20260627/manifest.json, apps/api/app/services/voice.py

검증

mp3/wav 6개(seed+5) 생성, 24kHz, 7.72~11.68초. live /voice/ws text_turn smoke: provider p1-sample-poc, binary 20 chunks/78,573 bytes. 샘플 provider는 TTS override만 담당하고 마이크/STT는 OpenAI 키가 필요하다. 프로덕션 탑재는 모델 라이선스·권리정책 결정 전 금지.

+
POC DONE
P1 서연 무참조 synthetic voice 샘플

Higgs v3 로컬 런타임에서 실존 reference voice 없이 synthetic seed를 먼저 만들고, 그 seed만 재참조해 우울·피로·불안·라포·회복감 5개 변주를 생성했다. 개발용 VIGNETTE_VOICE_POC_SAMPLE_TTS 플래그로 /voice/ws P1 sample TTS provider에도 연결했다.

산출물

docs/archive/voice-art/bimec-persona-voice-poc-2026-06-27.md, docs/voice-art/p1-seoyeon-higgs-v3-20260627/manifest.json, apps/api/app/services/voice.py

검증

mp3/wav 6개(seed+5) 생성, 24kHz, 7.72~11.68초. live /voice/ws text_turn smoke: provider p1-sample-poc, binary 20 chunks/78,573 bytes. 샘플 provider는 TTS override만 담당하고 마이크/STT는 OpenAI 키가 필요하다. 프로덕션 탑재는 모델 라이선스·권리정책 결정 전 금지.

LOCAL DONE
barge-in / paralinguistic logging

음성 learner turn에 audio_ref, silence_ms, speech_rate, barge_in을 nullable 메타로 저장하고, 회기 리뷰에서 침묵·발화 속도·끼어듦·음성 입력 비언어 칩으로 표시한다. 2차로 provider_events JSONB 보존 슬롯을 추가해 control/STT provider 이벤트를 allowlist·size limit 후 저장하고, 3차로 내부 taxonomy event_type/category를 붙인다. 4차로 인증된 회기 리뷰에는 한숨·울음·웃음·호흡·운율·배경소음 같은 정규화 이벤트만 label/detail 칩으로 제한 노출한다. 실제 Session 마이크 UI는 browser voice activity/trailing silence 메타를 audio_end에 싣고, readiness는 음성 메타 5개 컬럼 전체를 확인한다. raw transcript/text/provider/source/raw type은 응답에서 제외하고, 공개 공유 카드는 축어록과 provider raw를 계속 싣지 않는다.

산출물

app.turns.provider_events, TurnRecord.provider_events, voice route 이벤트 sanitizing/taxonomy, ReviewNonverbalEvent, SessionReview 칩 UI, browser voice metadata capture, voice metadata readiness guard

검증

python -X utf8 -m pytest -p no:cacheprovider app/test_runtime_policy.py app/test_session_turn_persistence.py app/test_voice_ws.py app/test_voice_service.py -q 77 passed; provider_events DB insert placeholder/args, WS raw key drop, STT+control event merge, internal taxonomy, client turn 미혼입, review 파생 칩 raw provider 미노출, 침묵 칩 중복 방지, 5-column readiness 검증. PLAYWRIGHT_PORT=5197 npx playwright test e2e/voice-success.spec.ts --project=chromium-single-run --workers=1 2 passed, PLAYWRIGHT_PORT=5199 npx playwright test e2e/session-persistence.spec.ts --project=chromium-single-run --workers=1 5 passed, npm run typecheck.

GATE
공개·실기기 음성 장시간 실측

남은 범위는 실제 Deepgram interim/final WSS, 물리 마이크, 공개 WSS, 50분 양방향 장시간 실측, provider 기반 한숨·울음 감지다. 로컬 OpenAI STT/TTS smoke와 synthetic mic E2E는 이 실측을 대체하지 않는다.

산출물

public WSS smoke log, physical mic capture log, 50분 soak evidence, provider paralinguistic event sample

검증

실제 공개/실기기 환경에서만 완료 가능. 현재는 B2 음성 캐스케이드 live gate로 추적한다.

DECIDED
s2s 2차 PoC 판단

소유자 결정(2026-06-30): s2s 2차 PoC 유지. 캐스케이드 1차를 운영 기준으로 두고 voice-s2s-poc.md 비교 PoC를 계속 연다. 최종 채택/폐기는 비교 리포트 이후 재판단한다. 채택 여부 자체는 확정됐고, 비교 PoC 구현·실측이 후속 build로 남는다.

산출물

docs/decisions/voice-s2s-poc.md

검증

transcript, safety, audit, latency, failure, privacy 기준 명시. 실제 비교 실측은 후속.

@@ -891,9 +904,9 @@ @@ -901,13 +914,13 @@

운영 / 검증 상세

실제 운영 도메인에 반영됐는지, 서버가 하나로 정리됐는지, 완료라고 말할 증거가 있는지 확인하는 탭이다.

웹: 200API: recovered OKCompose smoke: OK
DONE
운영 웹 asset 확인

커스텀 도메인의 로그인 페이지는 계속 정상 응답한다. 2026-06-30 재확인 기준 로그인 HTML HEAD가 200이다.

산출물

https://vignette.chanpaca.net/login

검증

curl.exe -I https://vignette.chanpaca.net/login?dev_dashboard_redteam=20260630-recovered → 200

-
DONE
공개/API 런타임 health

2026-06-30 라이브 재확인에서 public API 터널이 한때 Cloudflare 530/error code 1033을 반환했다. 원인은 Docker Desktop 중단으로 vignette-dev-db가 꺼져 prod API가 DB 127.0.0.1:55432에 연결하지 못한 상태였다. Docker Desktop/DB 재기동 후 start-public-runtime.ps1를 다시 실행했고, 로컬 prod API와 public API가 모두 복구됐다.

산출물

http://127.0.0.1:8001/health OK, https://api-vignette.chanpaca.net/health 200, https://api-vignette.chanpaca.net/personas 401, watch-public-runtime.ps1 -CheckOnly healthy

검증

environment:"prod", db:true, engine:true, engine_mode:"claude_cli". 비인증 보호 경계는 401로 복구.

+
DONE
공개/API 런타임 health

2026-06-30과 2026-07-01 17:26에 Docker/DB 중단으로 public API Cloudflare 530이 관측됐지만 현재 실측 기준으로 복구됐다. 2026-07-01 재확인에서 https://api-vignette.chanpaca.net/health는 status=ok, db=true, engine=true이고, https://vignette.chanpaca.net/login은 200, 비로그인 GET /personas는 401을 반환한다. 현재 남은 항목은 public OAuth 실제 계정 /turn smoke와 재부팅 후 watchdog 실측이다. 로컬 dirty worktree 변경은 아직 배포 전이다.

산출물

https://api-vignette.chanpaca.net/health OK, https://vignette.chanpaca.net/login 200, unauth /personas 401

DONE
Tailscale Tailnet 런타임

로컬 web/api/engine을 Tailscale Serve로 묶어 같은 Tailnet PC·모바일에서 HTTPS로 접근한다. Vite proxy와 API 포트가 어긋나지 않도록 dev-up.ps1에서 VITE_API_PROXY_TARGET을 ApiPort와 동기화한다. Tailnet dev-origin에서는 공개 OAuth callback으로 세션이 섞이지 않게 dev-login을 기본 검증 경로로 둔다.

산출물

https://alpaca-home.taile93291.ts.net, scripts/start-tailscale-runtime.ps1, VITE_ALLOWED_HOSTS, vite.config.ts, auth.py

검증

/login 200, /api/auth/config 200, redirect_uri=https://api-vignette.chanpaca.net/auth/callback, dev_login_enabled:true. Vite host-header block 재현 안 됨.

DONE
Docker compose 패키징 smoke

더미 운영 env 기준 fresh compose 기동을 통과했다. API 컨테이너 import 실패, 구현 없는 RAG 사이드카, 로컬 apps/api/.env 이미지 혼입, startup owner-DDL, 빈 admin_engine_config 행 때문에 prod가 죽던 경로를 수정했다. 루트 계산은 app.paths.repo_root()로 모았고, backend requirements는 exact pin으로 고정했다.

운영 주의

패키징은 OK지만 실제 이식 전에는 배포지 infra/.env에 APP_DB_PASSWORD, OAuth client id/secret, OPENAI_API_KEY, SESSION_SECRET 등 소유자 비밀값을 채워야 한다. data/kb live-coaching source pack은 commit 또는 같은 경로 provision이 필요하다. 현재 로컬 env의 ENGINE_MODE=claude_p와 prod 금지 샘플 TTS 플래그는 운영값으로 쓰면 안 된다.

검증

docker compose -f infra/docker-compose.yml config --quiet, API/Web image build, API image .env* 미포함, prod import smoke, fresh docker compose up -d --build → /api/health 200, preflight OK requirements_pinned/live_coach_kb/env_file.

BLOCKED
공개 vnet.18ka.net 런타임

vnet.18ka.net과 api-vnet.18ka.net을 정식 공개 도메인으로 허용하도록 코드 경로는 보강했다.

산출물

FRONTEND_ORIGIN_MAP, 공개 host별 API 매핑, scripts/start-public-runtime.ps1 vnet ingress 옵션

블로커

현재 DNS 미해결, Cloudflare credential의 18ka.net zone 권한 불일치, Google OAuth redirect URI 등록 필요.

DONE
서버 정리 상태

운영 API와 로컬/Tailnet 테스트 API를 구분해 유지한다.

산출물

8001 prod API, 8000 local API, 8010 Tailnet API, 5173 web, 9099 engine, 20241 cloudflared

검증

Get-NetTCPConnection 대상 포트 확인

GATE
재부팅 후 런타임 재기동

engine gateway, prod API, cloudflared를 health-check하고 복구하는 watchdog과 Scheduled Task installer는 준비됐다. 아직 DNS가 없는 api-vnet.18ka.net은 기본 watchdog 검사에서 제외해 불필요한 restart loop를 막고, 실제 개통 후 -AdditionalPublicHealthUrls로 명시 추가한다. 남은 것은 실제 Windows 재부팅 후 자동 복구와 authenticated public /turn smoke 증거다.

산출물

watch-public-runtime.ps1, install-public-runtime-task.ps1, docs/ops/public-runtime-watchdog.md

검증

PowerShell parser OK. 2026-06-30 복구 후 powershell -NoProfile -ExecutionPolicy Bypass -File scripts\watch-public-runtime.ps1 -CheckOnly는 healthy: engine, api, web-preview, cloudflared, public-api. 실제 Windows 재부팅은 아직 미실행.

-
DONE
레이아웃 감사팀 / 전면 레이아웃 개선

서브에이전트 전담으로 1차 overflow/clipping 수리 후 2차 전면 재설계를 진행했고, 2026-06-28 라운드에서 빈 회기리뷰까지 포함해 7개 권장 너비 시각 수용을 닫았다. 세션, 학습자 홈/리뷰, 교수 콘솔, 관리자, 설정 화면을 작업 화면 기준으로 재배치했고, 과한 빈칸·튀어나감·버튼/텍스트 잘림·모바일 테이블 사용성을 줄였다. 당시 엄격한 시각 게이트 + 화면별 적대적 시각 재검수에서 critical/major 0, 회귀 0을 받아 DONE으로 닫았다.

산출물

docs/ops/layout-redesign-handoff-2026-06-26.md, page-by-page redesign patches, apps/web/e2e/layout-visual-gate.spec.ts

검증

npm run typecheck OK, 최신 엄격 시각 게이트 9 passed, layout focused E2E desktop/mobile 54 passed, 2026-06-28 적대적 시각 재검수 7/7 accept·회귀 0.

+
DONE
레이아웃 감사팀 / 전면 레이아웃 개선

서브에이전트 전담으로 1차 overflow/clipping 수리 후 2차 전면 재설계를 진행했고, 2026-06-28 라운드에서 빈 회기리뷰까지 포함해 7개 권장 너비 시각 수용을 닫았다. 세션, 학습자 홈/리뷰, 교수 콘솔, 관리자, 설정 화면을 작업 화면 기준으로 재배치했고, 과한 빈칸·튀어나감·버튼/텍스트 잘림·모바일 테이블 사용성을 줄였다. 당시 엄격한 시각 게이트 + 화면별 적대적 시각 재검수에서 critical/major 0, 회귀 0을 받아 DONE으로 닫았다.

산출물

docs/archive/ops/layout-redesign-handoff-2026-06-26.md, page-by-page redesign patches, apps/web/e2e/layout-visual-gate.spec.ts

검증

npm run typecheck OK, 최신 엄격 시각 게이트 9 passed, layout focused E2E desktop/mobile 54 passed, 2026-06-28 적대적 시각 재검수 7/7 accept·회귀 0.

DONE
엄격한 레이아웃 시각 게이트 구축 · 병렬 수정 · 적대적 재검수

핸드오프가 요구한 7개 권장 너비(390/720/861/900/1024/1280/1440) 시각 수용을 자동 게이트로 고정했다. 현재는 빈 회기리뷰 전용 화면까지 9개 화면을 각 너비에서 렌더링해 가로 overflow 0, 컨트롤 하드클립/텍스트클립 0, 다크 테마 캡처를 강제하고 화면당 7장 총 63장 풀페이지 스크린샷을 남긴다. 2026-06-28 당시 화면별 1에이전트 병렬 시각 리뷰로 49장 직접 판독 → 6개 병렬 수정 에이전트(파일 비중첩)가 결함 수정 → 화면별 적대적 재검수로 해소·무회귀를 확정했다.

산출물

apps/web/e2e/layout-visual-gate.spec.ts, node_modules/.tmp/layout-gate/*.png 63장, 2× workflow(review/verify)·6× fix agent

검증

최신 게이트 9 passed(재스크린샷). 해소된 핵심 결함: 학습자홈 1280/1440 3열 가운데 컬럼 붕괴(critical → 2열 분기 상향 + word-break:keep-all), 교수 'API 404' raw 배너 제거, 리뷰 빈상태 위계 역전과 1280 sparse third-column, 관리자 스켈레톤, 설정 와이드 좌측 데드존 제거, 세션 모바일 44px 터치타깃. 2026-06-28 적대적 재검수 7/7 accept, regression 0.

DONE
레이아웃 cosmetic minor 폴리시 일괄 처리

적대적 재검수 잔여 cosmetic minor를 5개 병렬 폴리시 에이전트로 처리했다. 학습자홈 헬퍼문구 고아 글자, 세션 1024 일시정지 아이콘 정리 + 보내기 버튼 대비 강화, 설정 2x2 칩 행 높이 균일, 교수 검토 큐 카드 갭 제거, 관리자 2열 높이 동기화를 적용했다.

산출물

page-by-page polish patches(LearnerHome/Session/Settings/Professor/Admin)

검증

npm run typecheck OK, 당시 cosmetic 시각 게이트 7 passed, 최신 통합 시각 게이트 9 passed, 레이아웃 E2E desktop/mobile 54 passed, session-layout 8/8 — 회귀 0.

DONE
회기 아카이브 저장/복원 API

/learn/history의 보관됨을 실제 학습자별 저장 상태로 연결했다. 종료 회기는 POST /sessions/{id}/archive/restore로 보관·복원하고, app.session_archive_state는 보기 상태만 저장한다. 진행 중 회기는 보관하지 못하며, 보관은 transcript/review/share/audit evidence를 삭제하지 않는다.

산출물

SessionArchiveResponse, LearnerSessionSummary.archived, LearnerDashboardOverview.archived_sessions, session_archive_state RLS, LearnerHome 보관/복원 버튼

검증

python -B -m pytest app/ -q 178 passed, python -B -m pytest engine_gateway/ -q 11 passed, npm run check:api-types, npm run typecheck, npm run build, npx playwright test e2e/learner.spec.ts --project=chromium-desktop --workers=1 6 passed.

@@ -919,11 +932,11 @@ 범위명령 / 확인 대상결과 Web typechecknpm run typecheckPassed - Full Playwright E2E baselinenpx playwright test --list현재 수집 기준 166 tests / 18 files. 최신 변경은 focused gate 행으로 별도 검증한다. + Full Playwright E2E baselinenpx playwright test --list현재 수집 기준 194 tests / 19 files. 최신 변경은 focused gate 행으로 별도 검증한다. API typegen SSOTnpm run check:api-typesPassed; FastAPI OpenAPI → src/lib/api.gen.ts stale check Contract SSOT aggregate DTOpy -3.11 -X utf8 -m pytest -p no:cacheprovider app/test_evaluation_persistence.py app/test_evaluator_model_routing.py app/test_teacher_dashboard.py app/test_rbac_idor.py app/test_session_turn_persistence.py -q / npm run check:api-types / npm run typecheck / npm run build50 backend passed; learner sessions, session review/worksheet, teacher dashboard, session start/detail DTOs use generated ApiSchema aliases with UI fallback. Stage responses are OpenAPI enum unions, including review phase key/label, reached phase, evaluation summary/trigger responses, and teacher session/growth stage. .github/workflows/api-contract.yml runs npm run check:api-types on API/Web contract changes. SEO/GEO share cardspytest app/test_session_share.py app/test_session_turn_persistence.py -q / npm run generate:api-types / npm run typecheck21 passed; session share creates hashed-token public unfurl payload without raw transcript, revoked token returns 404, OpenAPI generated share DTOs, review screen share button typechecks. Static robots.txt/sitemap.xml/llms.txt added. - Backend pytest baselineC:\Users\encep\AppData\Local\Programs\Python\Python311\python.exe -X utf8 -m pytest -p no:cacheprovider app/ -q2026-07-01 현재 309 passed, 1 warning. + Backend pytest baselineC:\Users\encep\AppData\Local\Programs\Python\Python311\python.exe -X utf8 -m pytest -p no:cacheprovider app/ -q이번 패스에서 전체 app/ 재실행은 하지 않았다. 현재 collect-only 349 tests, 최신 focused는 teacher dashboard 11 passed + live coach route persistence source 1 passed. X2 evaluator routing/cache/cost trendpython -B -m pytest -p no:cacheprovider app/test_usage_report.py app/test_evaluator_model_routing.py app/test_runtime_policy.py app/test_admin_ops.py -q / scripts/report-ai-usage.py sample CLI / prior web gates42 backend passed + CLI sample OK; fast/deep evaluator model override, semantic cache hit/miss, admin usage budget/fail-closed, cache hit-rate observability, daily cost trend, and provider/model cost report schema covered. Prior API typecheck/build/admin E2E evidence remains valid because the API response shape was not changed. H3 raw source isolationpython -B -m pytest -p no:cacheprovider app/test_persona_review.py app/test_live_coach_sources.py -q38 passed; persona source registration records raw hash-only artifacts separately from sanitized evaluator-only RAG chunks, and rag.index_document() rejects sensitivity=3/raw-marker chunks before DB access. H3 persona itemized authoring UInpm run typecheck / npm run build / npm run check:api-types / npx playwright test e2e/teacher.spec.ts --project=chromium-single-run --workers=1 / npx playwright test e2e/layout-visual-gate.spec.ts --project=chromium-single-run --workers=1 / npx playwright test e2e/session-layout.spec.ts --project=chromium-desktop --project=chromium-mobile --workers=1Typecheck/build/API type drift check passed; teacher PersonaStudio focused 8 passed including structured list payload and labeled prompt preview without raw JSON; visual gate 9 passed with the prompt tab opened; session-layout desktop/mobile 8 passed. @@ -935,44 +948,45 @@ C1 case worksheetpy -3.11 -X utf8 -m pytest -p no:cacheprovider app/test_session_turn_persistence.py app/test_teacher_dashboard.py app/test_rbac_idor.py app/test_case_worksheet_rubric.py -q / scripts/check-case-worksheet-rubric.py --json / PLAYWRIGHT_PORT=5228 npx playwright test e2e/session-persistence.spec.ts --project=chromium-single-run --workers=1 --grep "case worksheet" / local API smoke40 passed for worksheet persistence/RBAC/teacher review plus rubric scaffold; session review exposes transcript-grounded caseWorksheet, learner-edited saved_by_learner worksheet persists through app.case_worksheet, teacher/admin teacherReview exposes manual worksheetStatus/worksheetNote/worksheetReviewedAt, and full browser+API+DB E2E now verifies learner save → teacher read-only review → 수정요청 persistence. case-worksheet-rubric@scaffold-2026-06-28 still validates 5 sections/28 items with scoring_enabled=false. M3 auth claim mappingpython -B -m pytest -p no:cacheprovider app/test_auth_providers.py -q30 passed; Google/SAML cohort maps, provider external_id, provider error reason, PKCE/state, HttpOnly cookie, SAML fixture ACS covered Auth managed-user upsert boundarypy_compile / py -3.11 -X utf8 -B -m pytest -p no:cacheprovider app/test_auth_providers.py app/test_admin_ops.py app/test_rbac_idor.py app/test_runtime_policy.py app/test_session_turn_persistence.py app/test_learner_dashboard.py app/test_teacher_dashboard.py -q / npm run check:api-typesManagedUserUpsertInput owns managed-user create/reactivate inputs; create_session(), admin user create, and direct auth regression calls no longer pass long keyword bags. ManagedUserMemoryInput remains DB→memory fallback/store sync, and ManagedUserPatch remains partial profile/admin update. Focused backend 103 passed; API type drift check passed. - CrisisGate 109python -m pytest app/test_session_turn_persistence.py app/test_voice_ws.py -q / PLAYWRIGHT_PORT=5228 npx playwright test e2e/session-persistence.spec.ts --project=chromium-single-run --workers=1 --grep "crisis safety"real crisis stops before engine, returns 109 resource, and DB insert uses ai_context=true. db.healthcheck() now requires app.safety_events; alert detail handles legacy double-encoded JSON string rows; full browser+API+DB E2E verifies safety_events reaches the DB-backed teacher safety queue with resource 109. - Professor review workflowpython -X utf8 -m pytest -p no:cacheprovider app/test_teacher_dashboard.py -q / PLAYWRIGHT_PORT=5215 npx playwright test e2e/teacher.spec.ts --project=chromium-single-run --workers=1 / PLAYWRIGHT_PORT=5217 npx playwright test e2e/layout-visual-gate.spec.ts --project=chromium-single-run --workers=1 / npm run check:api-types / npm run typecheck / npm run build6 backend passed + 9 E2E passed + strict visual gate 12 passed; /teach now keeps only triage queues, and /teach/analysis starts with a searchable one-row-per-learner table, right-side row expansion, and ?learner= detail tabs for full session timeline, unrestricted trend points, stage breakdown, and review counts. + CrisisGate 109python -m pytest app/test_session_turn_persistence.py app/test_voice_ws.py -q / PLAYWRIGHT_PORT=5228 npx playwright test e2e/session-persistence.spec.ts --project=chromium-single-run --workers=1 --grep "crisis safety" / PLAYWRIGHT_PORT=5244 npx playwright test e2e/session-persistence.spec.ts --project=chromium-single-run --workers=1 --grep "crisis safety event"real crisis stops before engine, returns 109 resource, and DB insert uses ai_context=true. db.healthcheck() now requires app.safety_events; alert detail handles legacy double-encoded JSON string rows; full browser+API+DB E2E verifies safety_events reaches the DB-backed teacher safety queue with resource 109. 최신 E2E는 위기 발화가 교수자 검토용 learner turn으로 저장되고 client AI reply turn은 저장되지 않는지도 확인한다. + Professor review workflowpython -X utf8 -m pytest -p no:cacheprovider app/test_teacher_dashboard.py app/test_rbac_idor.py::TeacherAdminAuditTest::test_teacher_list_recent_sessions_inserts_read_audit_log app/test_rbac_idor.py::TeacherAdminAuditTest::test_admin_load_session_inserts_read_audit_log -q / PLAYWRIGHT_BASE_URL=http://localhost:5210 npx playwright test e2e/teacher.spec.ts --project=chromium-single-run --grep "shows selected learner analysis" / PLAYWRIGHT_BASE_URL=http://localhost:5210 npx playwright test e2e/layout-visual-gate.spec.ts --project=chromium-single-run --grep "professor (student analysis overview|learner detail analysis)" / npm run check:api-types / npm run typecheck / npm run build10 teacher-dashboard backend passed + student-analysis E2E 1 passed + strict visual focused gate 2 passed; /teach keeps triage queues, and /teach/analysis starts with a searchable one-row-per-learner table using app_user display labels, right-side row expansion, and ?learner= detail. Teacher session summaries now expose AI supervisor_state/evaluation_error separately from manual review status, so failed AI evaluations remain visible in professor queues. The learner detail default tab is now persona-by-persona drilldown: each practiced persona summarizes its sessions, score, rapport, review count, and expands into that persona's session rows before the full timeline/stage tabs. + Professor student list capC:\Users\encep\AppData\Local\Programs\Python\Python311\python.exe -X utf8 -m pytest -p no:cacheprovider app/test_teacher_dashboard.py -q10 passed; teacher dashboard API now calls session_persistence.list_all_sessions() while learner/general lists call list_recent_sessions(), so the newest 100 sessions from one learner cannot hide the rest of the student analysis list. total_learners and learner_growth stay on the same full learner set. Learner/teacher growth metricspy -3.11 -X utf8 -B -m pytest -p no:cacheprovider app/test_evaluation_persistence.py app/test_evaluator_model_routing.py app/test_session_turn_persistence.py app/test_rbac_idor.py app/test_teacher_dashboard.py app/test_learner_dashboard.py -q59 backend passed; shared session_metrics now prefers rehydrated label_ko for technique labels while preserving label/name/id/code fallback. alternative_utterances string and text/suggestion dict inputs are pinned before any future evaluator adapter extraction. - Teacher review statuspytest app/test_teacher_dashboard.py app/test_rbac_idor.py app/test_learner_dashboard.py -q / npm run typecheck / npm run build / teacher.spec.ts12 backend passed + 4 E2E passed; app.session_review_status, teacher dashboard review_status/review_note/reviewed_at, worksheet worksheet_status/worksheet_note/worksheet_reviewed_at, and PUT /teacher/sessions/{session_id}/review-status are documented. 검토 큐 visible 회귀와 교수자 전용 review grid 영역도 테스트로 고정했다. + Teacher review statuspytest app/test_teacher_dashboard.py app/test_rbac_idor.py app/test_learner_dashboard.py -q / npm run typecheck / npm run build / teacher.spec.ts최신 focused teacher dashboard 11 passed. app.session_review_status, teacher dashboard review_status/review_note/reviewed_at, worksheet worksheet_status/worksheet_note/worksheet_reviewed_at, and PUT /teacher/sessions/{session_id}/review-status are documented. 검토 큐 visible 회귀와 교수자 전용 review grid 영역도 테스트로 고정했다. 124차에서는 턴이 0개인 종료 세션을 pending queue에서 제외해 reviewReady=false 세션이 검토 대기처럼 보이는 mismatch를 막았다. Persona authoring + generation contractpy -3.11 -X utf8 -B -m pytest -p no:cacheprovider app/test_persona_generation_contract.py app/test_persona_review.py -q39 passed; teacher/admin draft create/read/update/review submit, approved persona revision clone, code-family archive delete, learner blocked, approved-only catalog maintained. app/persona_generation_contract.py owns draft structured schema, prompt bundle id/version/hash, legacy GenerateResponse payload extraction, and generated draft defaults/coercion. - Repo source pack syncpy -3.11 -X utf8 -m pytest -p no:cacheprovider app/test_live_coach_sources.py -q / py -3.11 -X utf8 scripts\sync-persona-sources.py --help9 passed; dry-run reports next version without writes, apply bumps changed hash to latest+1, identical hash skips indexing, CLI exposes DB-backed dry-run/apply modes, and admin/CLI sync boundaries refresh the process-local live-coach source pack cache before comparing repo files. File-change regression proves stale cached content_hash becomes a new hash and next document version when refreshed. + Repo source pack syncpy -3.11 -X utf8 -m pytest -p no:cacheprovider app/test_live_coach_sources.py -q / PLAYWRIGHT_PORT=5255 npx playwright test e2e/kb-source-packs.spec.ts --project=chromium-single-run --workers=1 / py -3.11 -X utf8 scripts\sync-persona-sources.py --help9 backend passed + latest full browser/API/DB E2E 1 passed; dry-run reports next version without writes, apply bumps changed hash to latest+1, identical hash skips indexing, CLI exposes DB-backed dry-run/apply modes, and admin/CLI sync boundaries refresh the process-local live-coach source pack cache before comparing repo files. E2E verifies unauth 401, learner 403, admin 202, four licensed live-coach source ids, non-empty-or-unchanged indexed items, and source-scoped evaluator retrieval 200; retrieval 503 is now a failing test, not skipped evidence. P1 cost telemetryapp.test_session_turn_persistence.SessionTurnPersistenceTestPassed; generate/stream client AI turns store provider/model/tokens/cost Engine stream contractnode scripts/check-engine-gateway-contract.mjs --json / py -3.11 -X utf8 -B -m pytest -p no:cacheprovider engine_gateway/test_gateway_model.py -q / py -3.11 -X utf8 -B -m pytest -p no:cacheprovider engine_gateway/test_gateway_model.py app/test_evaluation_persistence.py app/test_evaluator_model_routing.py app/test_session_turn_persistence.py app/test_live_coach_sources.py app/test_persona_generation_contract.py app/test_persona_review.py -qNode conformance runner OK, gateway contract 27 passed, backend focused 120 passed; EngineClient.stream_packets() owns raw SSE decode, EngineGatewaySseLineDecoder parses token/done/error from the shared contract, keeps legacy token text compatibility, treats provider [DONE] as an ignored compatibility line rather than completion, rejects unknown gateway events, validates schema-backed engine_gateway_contract.v1.json, and gateway error event does not create saved turns. Node runner는 StreamPacket.oneOf, payload schema $ref, x-engine-gateway-sse-events, x-engine-gateway-default-model-sentinel에서 stream/default-routing 계약을 읽는다. gateway-default request sentinel과 response/done resolved model 분리, GatewayPromptParts current-turn prompt split, user 없는 generate/stream 400 경계, GenerateResponse structured payload fallback, persona draft generation contract, SessionEvaluationWrite.from_result()/from_error() packet factory는 계약 모듈과 Node runner/focused tests가 함께 검증한다. Engine session reuseengine_gateway.test_gateway_model27 tests OK; shared engine contract, JSON Schema + golden fixture validation, Node.js artifact conformance runner, GenerateResponse response validation, structured payload fallback parser, current-turn prompt split, direct evaluator/live-coach parser ownership, SSE frames/decoder, live session_id reuse, gateway-default 기본 라우팅 sentinel 정규화, missing-user 400 guard, ephemeral close fixed - P1/H4 masking gateapp/test_pii_masking_eval.py app/test_orchestrator_masking.py app/test_evaluation_persistence.py app/test_session_turn_persistence.py47 passed; 최신 보정 회귀 59 passed. phone/email/RRN 및 한국어 NAME/ORG raw 값이 generate/stream/evaluator payload와 client text_masked에 남지 않음. Optional ko recognizer fake span은 [NAME]/[ORG]로 마스킹되고 같은 문장의 phone은 후단 regex가 처리하며, adapter 실패 시에도 regex fallback이 유지된다. Synthetic ko fixture 16/16 pass, 자연 발화형 이름 라벨·자기소개와 negative control 포함, input/report schema validation, summary-only report(evidence_text_included=false), entity recall 1.0, forbidden substring removal 1.0, unexpected entity violations 0. + P1/H4 masking gateapp/test_pii_masking_eval.py app/test_orchestrator_masking.py app/test_evaluation_persistence.py app/test_session_turn_persistence.py / PLAYWRIGHT_PORT=5244 npx playwright test e2e/session-persistence.spec.ts --project=chromium-single-run --workers=1 --grep "Korean PII"47 passed; 최신 보정 회귀 59 passed + DB-backed browser PII E2E 1 passed. phone/email/RRN 및 한국어 NAME/ORG raw 값이 generate/stream/evaluator payload와 client text_masked에 남지 않음. Optional ko recognizer fake span은 [NAME]/[ORG]로 마스킹되고 같은 문장의 phone은 후단 regex가 처리하며, adapter 실패 시에도 regex fallback이 유지된다. Synthetic ko fixture 16/16 pass, 자연 발화형 이름 라벨·자기소개와 negative control 포함, input/report schema validation, summary-only report(evidence_text_included=false), entity recall 1.0, forbidden substring removal 1.0, unexpected entity violations 0. 최신 브라우저 stream E2E는 한국어 이름·기관·전화번호 발화가 DB-backed GET /sessions/{id} 상세와 /review 모두에서 raw 값 없이 [NAME]/[ORG]/[PHONE]으로 남는지 확인한다. P2a RBAC/audit/visibilityapp.test_rbac_idor11 tests OK; other learner 403, read_session audit, evaluator-only hidden, teacher session review read allowed, super_admin learner review principal is promoted to admin for evaluation/worksheet/review-status loaders, admin_access-only learner remains blocked, and learner worksheet write remains 403 - Admin access delegationpython -B -m pytest -p no:cacheprovider app/test_auth_providers.py app/test_runtime_policy.py app/test_rbac_idor.py app/test_teacher_dashboard.py app/test_learner_dashboard.py -q / npx playwright test e2e/admin.spec.ts --project=chromium-desktop --workers=1 / npm run check:api-types / npm run typecheck / npm run build70 backend focused passed + admin/auth/runtime 64 passed + admin E2E 8 passed. 실제 admin role은 학습자·교수자·관리자 공간을 모두 열고, admin_access만 받은 비관리자 계정은 관리자 콘솔 진입권만 얻는다. 관리자 sidebar는 운영 홈/사용자/권한/티켓과 교수자·학습자 작업 공간 링크를 함께 노출한다. + Admin access delegationC:\Users\encep\AppData\Local\Programs\Python\Python311\python.exe -X utf8 -m pytest -p no:cacheprovider app/test_admin_ops.py app/test_auth_providers.py::AuthProviderScaffoldTest::test_admin_role_can_enter_admin_console_even_with_stale_access_flag app/test_auth_providers.py::AuthProviderScaffoldTest::test_admin_role_can_enter_all_role_spaces_without_super_admin app/test_auth_providers.py::AuthProviderScaffoldTest::test_admin_access_flag_without_admin_role_does_not_enter_learner_space -q / npx playwright test e2e/admin.spec.ts --project=chromium-desktop --project=chromium-mobile --workers=1 --grep "approved admin without onboarding" / npm run typecheck13 backend focused passed + admin onboarding guard E2E 2 passed + web typecheck passed. 실제 admin role은 admin_access=false 세션이어도 can_access_role(Role.ADMIN)으로 관리자 API를 열고, 승인된 관리자 또는 admin_access 계정은 /admin*에서 학습자 온보딩으로 우회하지 않는다. 관리자 sidebar와 기존 workspace 전환 계약은 유지된다. Session read-model DB readinesspython scripts\check-deploy-preflight.py --skip-db --env-file infra\.env.example --allow-placeholder-secrets / https://api-vignette.chanpaca.net/health2026-06-29 prod 503 원인은 운영 DB의 app.turns.provider_events 컬럼 누락이었다. 운영 DB hotfix 후 /teacher/dashboard code path는 source=database로 복구됐다. 현재 db.healthcheck(), runtime table readiness, deploy preflight DB mode는 app.turns 음성 메타 컬럼 5개(audio_ref, silence_ms, speech_rate, barge_in, provider_events), app.session_review_status worksheet 컬럼, app.safety_events 필수 컬럼을 함께 검증한다. 2026-06-30에는 Docker Desktop/DB 중단으로 public API 530/error code 1033이 재발했지만, Docker Desktop/DB 재기동 뒤 public health가 environment=prod, db=true, engine=true로 복구됐다. Admin usage persistencepython -m pytest app/test_admin_ops.py app/test_runtime_policy.py -q / authenticated local public-API smoke26 passed; /admin/usage returns 200 with source=database, durable=true. Cloudflare blocked raw Python public smoke with 1010, so app-level HTTP was verified against 127.0.0.1:8001 using the same prod process. Synthetic health samplerrecord_admin_health_sample(), record-admin-health-sample.py, install-health-sampler-task.ps1Backend focused 31 passed. Python compile/help passed; PowerShell parser + -PrintOnly passed. Local one-shot appended 5 service rows to app.admin_health_event: status ok, engine_mode claude_cli. This remains sample history, not an SLA claim. Health retention/rollupadmin_health_daily_rollup, maintain-admin-health-events.py, /admin/uptimeBackend focused 36 passed. Python compile/help passed. Local dry-run returned rollup_event_count=0, prunable_event_count=0. Uptime summary now combines raw samples with daily rollups when raw rows have been pruned; detailed events stay raw-only. Ticket duplicate linkingsupport_ticket.fingerprint, parent_ticket_id, /admin/ticketsBackend focused 38 passed. API types, web typecheck, and build passed. Admin tickets E2E 2 passed after restarting local API 8000 on the new code. Duplicate handling is deterministic hint + admin manual parent link/unlink only; no automatic merge, escalation, issue, PR, or thread creation. User support ticket self-viewapp/test_user_support_tickets.py, app/test_admin_ops.py, app/test_runtime_policy.py, settings.spec.ts, admin.spec.ts --grep "operation tickets"Backend 29 passed; /users/support-tickets filters by current reporter_id and excludes ticket body/reporter PII. API types, typecheck, build passed. Settings E2E 7 passed; admin operation ticket E2E 1 passed. - Turn evaluation SSOPpytest app/test_session_turn_persistence.py app/test_evaluation_persistence.py app/test_voice_ws.py20 tests OK; submit/stream/voice fast-loop evaluation normalized mapping + evaluator-only raw RLS + turn insert id 확인 + Turn evaluation SSOPpytest app/test_session_turn_persistence.py app/test_evaluation_persistence.py app/test_voice_ws.py / py -3.11 -X utf8 -m pytest -p no:cacheprovider app/test_session_turn_persistence.py -k "fast_loop_evaluation" -q / py -3.11 -X utf8 -m pytest -p no:cacheprovider app/test_learner_dashboard.py -k "failed_fast_loop" -q / py -3.11 -X utf8 -m pytest -p no:cacheprovider app/test_eval_routes.py app/test_evaluation_persistence.py -q / PLAYWRIGHT_PORT=5246 npx playwright test e2e/session-persistence.spec.ts --project=chromium-single-run --workers=1 --grep "teacher turn reevaluation"20 tests OK baseline; submit/stream/voice fast-loop evaluation normalized mapping + evaluator-only raw RLS + turn insert id 확인. 추가 focused 2 passed + 1 passed는 stream 평가 훅 예외를 evaluation.error로 남겨 리뷰 턴 평가 실패 노트로 표면화하고, error fast-loop 결과를 성장 점수와 최근 피드백에서 제외해 neutral 점수처럼 집계하지 않는지 검증한다. 최신 backend 29 passed + DB-backed E2E 1 passed는 교수자 POST /eval/sessions/{id}/turn 결과가 응답 바디에만 머물지 않고 normalized evaluation row를 교체 저장해 /review learner turn technique으로 hydrate되는지 검증한다. Postgres RLS/audit smoke checkerVIGNETTE_RLS_AUDIT_DSN=…vignette_app… check-postgres-rls-audit.py --write-fixtureslive DSN 5 checks PASS. help-only safe default 검증은 과거 준비 단계이며 live proof로 superseded. OAuth/BFF cookie securitypython -B -m pytest -p no:cacheprovider app/test_auth_providers.py -q30 tests OK; PKCE/state, cookie-bound signed state restart recovery, state cookie mismatch reject, opaque HttpOnly/Secure cookie, no browser tokens, SAML AuthnRequest/dev-only fixture ACS, dev/Tailnet OAuth guard, public-origin dev-login closed - H4 consent gatepytest app/test_auth_providers.py app/test_session_turn_persistence.py app/test_persona_review.py app/test_voice_ws.py -q64 passed; learner consent accept/withdraw, non-learner/unaccepted reject, no-consent session start 403 consent_required, voice dev persona consent guard, frontend generated API contract updated - H1 pre/post evidence API + KPI exportapp/test_user_support_tickets.py, app/test_runtime_policy.py, app/test_admin_ops.py, app/test_phase3_kpi_export.py, app/test_phase3_artifact_checker.py, npm run check:api-types, npm run typecheck, e2e/session-review.spec.ts, e2e/session-persistence.spec.tsCurrent H1 broad 46 passed; KPI contract focused 10 passed. app.learner_prepost_measure stores learner-scoped aggregate pre/post scores for self-efficacy, skill proficiency, and training satisfaction. scripts/export-phase3-kpi.py emits pseudonymous prepost_measures.csv and KPI report scaffold with paired normalized deltas. phase3_kpi_contract.py owns metric names, required keys, and computed_prepost/design_pending status values used by exporter/checker/tests. Generated API contract exposes UserPrepostMeasure*. 학습자 SessionReview는 1~5 aggregate score 조회/저장 카드를 표시하고, DB-backed pre/post 저장·재조회 E2E까지 포함한 session-persistence 7 passed + visual gate 2 passed. + H4 consent gatepy -3.11 -X utf8 -m pytest -p no:cacheprovider app/test_voice_ws.py -q / PLAYWRIGHT_PORT=5231 npx playwright test e2e/voice.spec.ts --project=chromium-single-run --workers=1 --grep "consent withdrawal"16 backend passed + browser WS 1 passed; voice binding now checks onboarding/consent before both existing session_id and dev persona_code paths. 동의 철회 뒤 기존 voice session id 재접속도 consent_required로 닫히며, 같은 suite가 voice turn 저장 실패 구조화 error도 검증한다. + H1 pre/post evidence API + KPI exportapp/test_user_support_tickets.py, app/test_runtime_policy.py, app/test_admin_ops.py, app/test_phase3_kpi_export.py, app/test_phase3_artifact_checker.py, npm run check:api-types, npm run typecheck, e2e/session-review.spec.ts, e2e/session-persistence.spec.tsCurrent H1 broad 46 passed; KPI contract focused 10 passed. app.learner_prepost_measure stores learner-scoped aggregate pre/post scores for self-efficacy, skill proficiency, and training satisfaction. scripts/export-phase3-kpi.py emits pseudonymous prepost_measures.csv and KPI report scaffold with paired normalized deltas. phase3_kpi_contract.py owns metric names, required keys, and computed_prepost/design_pending status values used by exporter/checker/tests. Generated API contract exposes UserPrepostMeasure*. 학습자 SessionReview는 1~5 aggregate score 조회/저장 카드를 표시하고, 기존 저장 점수를 비우는 입력은 invalid로 표면화해 저장된 값 기준으로 오인하지 않게 한다. 최신 route-fixture pre/post focused E2E 1 passed, DB-backed pre/post 저장·재조회 E2E까지 포함한 session-persistence 7 passed + visual gate 2 passed. Persona review workflowapp.test_persona_review31 tests OK; approved-only catalog, teacher/admin queue, draft authoring, approved persona revision, archive audit, learner 403 - Voice WS contractapp.test_voice_ws13 tests OK; auth guard, audio_start PCM metadata, binary chunks, audio_end, text_turn, stt_result ready/pending, ping, max audio cap, TTS chunk envelope + Voice WS contractapp.test_voice_ws16 tests OK; auth guard, onboarding/consent guard for existing session binding, audio_start PCM metadata, binary chunks, audio_end, text_turn, stt_result ready/pending, ping, max audio cap, TTS chunk envelope, and structured turn_persistence_unavailable error payload Voice preset/EOTapp.test_voice_service14 tests OK; TTS payload, preset fallback, DB voice-map resolver, unsupported provider fallback, P1 sample TTS, EOT readiness Voice metadatapython -X utf8 -m pytest -p no:cacheprovider app/test_runtime_policy.py app/test_session_turn_persistence.py app/test_voice_ws.py app/test_voice_service.py -q + PLAYWRIGHT_PORT=5197 npx playwright test e2e/voice-success.spec.ts --project=chromium-single-run --workers=1 + PLAYWRIGHT_PORT=5205 npx playwright test e2e/session-persistence.spec.ts --project=chromium-single-run --workers=177 backend passed + voice-success 2 passed + latest session-persistence 7 passed; audio_ref/silence_ms/speech_rate/barge_in/provider_events persisted or readiness-checked on learner voice turn, Session 마이크 UI는 browser voice activity/trailing silence 메타를 audio_end에 전송, provider_events JSONB 보존·sanitizing·taxonomy·client turn 미혼입 검증, DB-backed review nonverbal은 timing/audio와 제한된 paralinguistic/prosody/audio_quality 칩을 중복 침묵 없이 노출 - Session end/evaluation recovery + AI tutor E2EPLAYWRIGHT_PORT=5205 npx playwright test e2e/session-persistence.spec.ts --project=chromium-single-run --workers=1 / PLAYWRIGHT_PORT=5203 npx playwright test e2e/session-review.spec.ts --project=chromium-desktop --project=chromium-mobile --workers=1 / python -X utf8 -m pytest -p no:cacheprovider app/ -qsession-persistence 7 passed가 실제 DB/API/엔진으로 세션 생성→턴→종료→background deep 평가 durable row→교수자 리뷰 평가 완료까지 검증한다. Session review desktop/mobile 16 passed; 평가 대기→ready 폴링, 평가 실패→교수자 AI 평가 재시도→완료, 재시도 503 실패 시 기존 실패 리뷰와 새 사유 유지, 감독자 리뷰 라우트 정규화를 검증했다. Backend app suite 309 passed; 회기말 평가 timeout/error 저장, background task crash 관찰, 저장 실패 표면화, /eval/.../evaluation의 status/error/durable 노출을 포함한다. - Browser SSE stream + AI tutor persistencePLAYWRIGHT_PORT=5205 npx playwright test e2e/session-persistence.spec.ts --project=chromium-single-run --workers=17 passed; 기존 direct /turn real API 저장 검증에 더해 실제 Session UI가 openSessionStream()으로 POST /sessions/{id}/stream을 호출하고, stream 완료 후 DB-backed /review가 learner/client 축어록을 반환하는지 확인한다. 추가로 코칭 모드에서 POST /live-coach 성공 뒤 DB-backed GET /live-coach 이력과 재로딩 후 학습자 발화의 C 마커가 유지되는지 확인하고, voice WS stt_result 및 실제 Session 마이크 UI 생성 비언어 메타가 DB-backed review nonverbal로 파생되는지도 고정한다. 같은 spec은 Phase 3 pre/post 저장과 세션 종료 background deep 평가의 durable DB row까지 검증한다. db/engine health가 false면 skip하고, stream/live-coach/evaluation 503/error는 실패로 취급한다. - C3 theory-mode reevaluationC:\Users\encep\AppData\Local\Programs\Python\Python311\python.exe -X utf8 -m pytest -p no:cacheprovider app/test_eval_routes.py -q / app/test_eval_routes.py app/test_notifications.py app/test_session_turn_persistence.pyeval route 5 passed, focused backend 39 passed. 수동 session 재평가는 persona 기본 theory_target보다 학습자가 선택한 session theory_mode를 우선하고, turn 재평가도 TurnContext.theory_mode에 같은 값을 전달한다. - Crisis safety gate UIPLAYWRIGHT_PORT=5193 npx playwright test e2e/session-mvp.spec.ts --project=chromium-single-run --workers=12 passed; mock SSE safety + done.conversation_stopped에서 안전 자원 109와 입력 disabled가 유지되고, 빈 client reply를 내담자 응답 없음으로 덮지 않으며 live-coach를 호출하지 않는지 검증한다. - Persona/session review UI E2EPLAYWRIGHT_PORT=5215 npx playwright test e2e/teacher.spec.ts --project=chromium-single-run --workers=19 passed; teacher approves pending persona, opens pending/recent review detail, reaches the separate student analysis page from the menu, searches the learner table, expands a learner row, and drills into the full learner timeline tab. - Voice UI synthetic E2Enpx playwright test e2e/voice-success.spec.ts --project=chromium-single-run2 passed; direct WS cascade plus Session mic button path with synthetic browser audio and Web Audio playback under blocked media-element autoplay + Session end/evaluation recovery + AI tutor E2EPLAYWRIGHT_PORT=5205 npx playwright test e2e/session-persistence.spec.ts --project=chromium-single-run --workers=1 / PLAYWRIGHT_PORT=5238 npx playwright test e2e/session-persistence.spec.ts --project=chromium-single-run --workers=1 --grep "finishes session end evaluation" / PLAYWRIGHT_PORT=5237 npx playwright test e2e/session-persistence.spec.ts --project=chromium-single-run --workers=1 --grep "explicit teacher session reevaluation" / PLAYWRIGHT_PORT=5245 npx playwright test e2e/session-persistence.spec.ts --project=chromium-single-run --workers=1 --grep "manual AI evaluation retry" / PLAYWRIGHT_PORT=5254 npx playwright test e2e/session-review.spec.ts --project=chromium-desktop --workers=1 --grep "manual AI retry|long-running session evaluation|retry a failed AI session evaluation|retry fails" / py -3.11 -X utf8 -m pytest -p no:cacheprovider app/test_notifications.py app/test_session_turn_persistence.py app/test_evaluation_persistence.py -k "session_evaluation or missing_session_evaluation or scheduled_session_evaluation" -qDB-backed session-persistence 7 passed가 실제 DB/API/엔진으로 세션 생성→턴→종료→background deep 평가 durable row→교수자 리뷰 평가 완료까지 검증한다. 추가 자동 평가 E2E 1 passed는 같은 persisted row가 /teacher/dashboard의 evaluation_status=ready, review_ready=true, supervisor_state=평가 완료로 반영되는지 확인한다. 명시 재평가 1 passed는 교수자 POST /eval/sessions/{id}/reevaluate가 durable app.session_evaluation row를 저장하고 /eval/.../evaluation과 /review가 평가 완료로 반영되는지 확인한다. 수동 UI 재시도 E2E 1 passed는 실제 실패 row를 만든 뒤 교수자 리뷰의 AI 평가 재시도 버튼 클릭이 real POST /eval/sessions/{id}/reevaluate → durable ready row → review/dashboard ready 상태로 이어지는지 확인한다. 최신 backend/UI focused 8 passed + 4 passed는 오래된 missing evaluation row가 교수자 대시보드에서도 평가 실패로 보이고, 이미 종료된 session /end 재호출이 평가를 재예약하지 않으며, 평가 ready 전에는 검토 완료로 큐에서 숨길 수 없음을 검증한다. Backend recovery focused 16 passed는 종료됐지만 evaluation row가 없는 오래된 DB 세션을 startup recovery가 evaluator AI context로 찾아 기존 session-end 평가를 재예약하고, 같은 session_id 중복 background 평가를 in-flight set으로 막는지 검증한다. 남은 후속은 다중 프로세스 durable claim/job table 수준의 재시도 소유권이다. + Browser SSE stream + AI tutor persistencePLAYWRIGHT_PORT=5241 npx playwright test e2e/session-persistence.spec.ts --project=chromium-single-run --workers=1 --grep "AI tutor coaching history" / PLAYWRIGHT_PORT=5251 npx playwright test e2e/session-mvp.spec.ts --project=chromium-desktop --workers=1 --grep "AI tutor|runtime AI tutor" / PLAYWRIGHT_PORT=5259 npx playwright test e2e/session-mvp.spec.ts --project=chromium-desktop --workers=1 --grep "AI tutor|stale empty AI tutor quota|quota exhaustion|degraded AI tutor|voice conversation stop"DB-backed focused 1 passed + mock UI focused 7 passed. 실제 Session UI 코칭 모드에서 POST /live-coach 응답과 DB-backed history payload가 status=ready, latency_ms>0인지 확인하고, 0615 source pack version/citation이 근거 보기 모달, DB-backed GET /live-coach 이력, reload 후 학습자 발화의 C 마커 history dialog까지 보존되는지 검증한다. Mock MVP는 source title/kind/version/citation 렌더, status=degraded replacement 표시, GET /live-coach 실패 alert, persistence_source/source=runtime 임시 이력 경고, stale empty quota 서버 재조회, quota exhaustion 우선 표시를 각각 고정한다. + C3 theory-mode reevaluationC:\Users\encep\AppData\Local\Programs\Python\Python311\python.exe -X utf8 -m pytest -p no:cacheprovider app/test_eval_routes.py -q / PLAYWRIGHT_PORT=5229 npx playwright test e2e/session-persistence.spec.ts --project=chromium-single-run --workers=1 --grep "selected CBT theory mode" / app/test_eval_routes.py app/test_notifications.py app/test_session_turn_persistence.pyeval route 5 passed, focused backend 39 passed, DB-backed browser E2E 1 passed. 실제 Session UI에서 선택한 CBT가 POST /sessions payload와 DB-backed GET /sessions/{id} 상세의 theory_mode=cbt로 보존된다. 수동 session 재평가는 persona 기본 theory_target보다 학습자가 선택한 session theory_mode를 우선하고, turn 재평가도 TurnContext.theory_mode에 같은 값을 전달한다. + Crisis safety gate UIPLAYWRIGHT_PORT=5193 npx playwright test e2e/session-mvp.spec.ts --project=chromium-single-run --workers=1 / PLAYWRIGHT_PORT=5259 npx playwright test e2e/session-mvp.spec.ts --project=chromium-desktop --workers=1 --grep "voice conversation stop"mock SSE safety + done.conversation_stopped와 mock voice reply.conversation_stopped에서 안전 자원 109와 입력 disabled가 유지되고, 빈 client reply를 내담자 응답 없음으로 덮지 않으며 live-coach를 호출하지 않는지 검증한다. 음성 위기 경로는 TTS 종료 이벤트 없이도 socket close를 확인한다. + Persona/session review UI E2EPLAYWRIGHT_PORT=5215 npx playwright test e2e/teacher.spec.ts --project=chromium-single-run --workers=19 passed mixed suite; real server-row rendering과 route-fixture UI 흐름을 함께 포함한다. 교수자가 pending persona를 승인하고, pending/recent review detail로 진입하며, 별도 student analysis 메뉴·학습자 검색·행 펼침·full learner timeline drilldown을 검증한다. + Voice UI synthetic E2Enpx playwright test e2e/voice-success.spec.ts --project=chromium-single-run / PLAYWRIGHT_PORT=5248 npx playwright test e2e/session-mvp.spec.ts --project=chromium-single-run --workers=1 --grep "pending voice transcript"voice-success 2 passed + pending transcript failure UI 1 passed; direct WS cascade, Session mic button path with synthetic browser audio/Web Audio playback, and final transcript followed by persistence failure becoming a visible 저장 실패 turn instead of a silent successful transcript Voice s2s decision memodocs/decisions/voice-s2s-poc.mdcriteria recorded; keep/drop decision remains owner DECIDE Hanshin data/SSO gatedocs/ops/hanshin-data-governance-gate.mdartifact created; written external evidence still required Resistance engineapp.test_state_machine_resistance + scripts/smoke-resistance-openness-db.py9 focused tests OK; live DB smoke PASS. Empathy P1 curve 0→0→0.095→0.133→0.171 and DB final stage=탐색. Advice-jump curve stayed 0 and DB final stage=라포. Python compilepython -m compileall app engine_gatewayPassed - Phase 3 artifact gatespython scripts\check-phase3-artifacts.py --help + pytest app/test_phase3_artifact_checker.py -qPassed; checker now enforces CSV enums, KPI metric required fields/status, approved export PII/agreement/consent/withdrawal/file-hash gates. Actual pilot evidence still external. + Phase 3 artifact gatespy -3.11 -X utf8 scripts\check-phase3-artifacts.py --help + py -3.11 -X utf8 -m pytest -p no:cacheprovider app/test_dataset_export.py app/test_phase3_artifact_checker.py -q16 passed; checker now enforces CSV enums, KPI metric required fields/status, approved export PII/agreement/consent/withdrawal/file-hash gates, and approved/dry-run dataset JSONL required keys, row count, privacy, and PII shape. Actual pilot evidence still external. Web buildnpm run buildPassed Pages production deploywrangler pages deploy dist --project-name vignette --branch main --commit-dirty=true2026-06-29 admin IA/readiness fix deploy from current working tree; preview https://c67ca04a.vignette-b1q.pages.dev Custom domain assetshttps://vignette.chanpaca.net/login?dev_dashboard_redteam=20260630-recovered2026-06-30 recheck: login HTML HEAD 200. Public API 복구 후에도 web custom domain은 정상이다. @@ -1001,7 +1015,7 @@ Session voice guard/view-modelnpm run typecheck / npm run build / npx playwright test e2e/session-mvp.spec.ts --project=chromium-single-run --workers=1 / npx playwright test e2e/voice-success.spec.ts --project=chromium-single-run --workers=1 / npx playwright test e2e/session-layout.spec.ts --project=chromium-desktop --project=chromium-mobile --workers=1typecheck/build passed; P1 text MVP 1 passed, voice cascade + Session mic UI 2 passed, session-layout desktop/mobile 8 passed. isVoiceStatusBusy() and sessionVoiceStatusView() own mic busy/disabled, aria label, transcript/response/status labels, and text-turn blocking without changing WebSocket/TTS lifecycle, DOM/CSS, API payload, or the recording click-to-send path. Session end + dark theme UXnpm run typecheck, npm run build, layout-visual-gate, session-layout, avatar-expression, settings, session-reviewtypecheck/build passed; Playwright 묶음 36 passed. P1 active session은 rasterArtSet="seoyeon-live2d-psd-v2"를 사용한다. sad 표정은 PSD 기반 눈물/우는 입/울상 눈썹 파츠를 렌더한다. Settings now treats API system preference as current initial theme instead of forcing light. Voice/session focusedvoice + voice-success + session-layout15 passed - Layout redesign handoffdocs/ops/layout-redesign-handoff-2026-06-26.mdsubagent scopes, files, verification, remaining visual review recorded + Layout redesign handoffdocs/archive/ops/layout-redesign-handoff-2026-06-26.mdsubagent scopes, files, verification, remaining visual review recorded Layout redesign focused E2Enpx playwright test e2e/learner.spec.ts e2e/session-layout.spec.ts e2e/session-review.spec.ts e2e/admin.spec.ts e2e/settings.spec.ts e2e/teacher.spec.ts --project=chromium-desktop --project=chromium-mobile54 passed after isolated Admin tablet flake rerun Strict layout visual gatePLAYWRIGHT_PORT=5217 npx playwright test e2e/layout-visual-gate.spec.ts --project=chromium-single-run --workers=112 passed; 12 screens × 7 widths(390/720/861/900/1024/1280/1440), horizontal overflow 0 + control clip 0, 84 full-page screenshots. 학생 분석 목록 테이블/상세 화면과 빈 회기리뷰 fixture 포함. Parallel visual quality reviewworkflow layout-visual-quality-review (7 agents)49 screenshots read; per-screen findings(critical 1, major 8+, minor) → fix verdicts @@ -1015,7 +1029,7 @@ Postgres RLS/audit smoke (live)VIGNETTE_RLS_AUDIT_DSN=…vignette_app… check-postgres-rls-audit.py --write-fixtures5 checks PASS: preflight, learner 세션/턴 격리, teacher cohort 스코프, visible_to evaluator-only 숨김, teacher/admin read audit insert (NOBYPASSRLS app role, fixtures 자동 정리) Turn cost telemetry (live DB)SELECT … FROM app.turns WHERE actor_kind='client_ai'13 client_ai turns 적재: llm_provider=claude_cli, model=gateway-default, cost_usd 실값(합 $1.22, 평균 $0.094), visible_to 배열. 토큰은 CLI 게이트웨이 특성상 0, cost는 산출 Cosmetic polish re-verify5 polish agents → typecheck + gate + layout E2E re-runtypecheck OK; current gate 9 passed; layout focused E2E 54 passed; session-layout 8/8; regression 0 - Dashboard layout alignmentnpm run typecheck / npm run build / layout-visual-gate / session-layout / session-review / learner+teacher+admin E2Ecurrent rerun: typecheck/build passed; layout gate 9 passed; session-layout+session-review chromium-desktop 6 passed. 이전 정렬 라운드: session-layout desktop/mobile 8 passed; learner+teacher+admin chromium-desktop 17 passed. 대표 캡처는 docs/ops/layout-research-2026-06-28/final-gate-*.png. + Dashboard layout alignmentnpm run typecheck / npm run build / layout-visual-gate / session-layout / session-review / learner+teacher+admin E2Ecurrent rerun: typecheck/build passed; layout gate 9 passed; session-layout+session-review chromium-desktop 6 passed. 이전 정렬 라운드: session-layout desktop/mobile 8 passed; learner+teacher+admin chromium-desktop 17 passed. 대표 캡처는 docs/archive/ops/layout-research-2026-06-28/final-gate-*.png. Learner session archivepython -B -m pytest app/ -q / python -B -m pytest engine_gateway/ -q / npm run check:api-types / npm run typecheck / npm run build / npx playwright test e2e/learner.spec.ts --project=chromium-desktop --workers=1Archive 당시 178 backend passed + 11 gateway passed; 최신 gateway contract gate는 27 passed. OpenAPI generated archive DTOs are current; learner history can archive/restore an ended session and keeps archived sessions out of the review queue while preserving records. @@ -1059,7 +1073,7 @@ powershell -NoProfile -ExecutionPolicy Bypass -File scripts\start-public-runtime 이번 세션 내러티브 (historical snapshot · 현재 숫자 아님)team lead review

- 이번 세션은 seed persona 결합 제거, 비개발 fail-closed 설정, compose OAuth 주입, admin 모바일 overflow 수정, persona 인증 경계 보강, 전체 Playwright 재통과, 공개 API prod 전환까지 완료했다. 이번 이어받기에서는 내담자 AI 생성 턴의 provider/model/token/cost 텔레메트리를 /turn, SSE stream, 음성 생성 경로에서 DB app.turns까지 적재하도록 연결했고, gateway stream done/error 계약을 구조화했으며, P1 상담 시작→텍스트 stream→종료→리뷰 피드백 UI E2E와 저항 엔진 정량 회귀 테스트를 추가했다. 이어서 PII 마스킹 게이트 우회 차단, RBAC/IDOR·감사 insert 경로, 음성 paralinguistic 메타 저장, 공개 런타임 watchdog 스크립트와 runbook을 보강했다. 추가로 상주 engine session reuse, SAML provider 스캐폴드, visible_to 런타임 필터, EOT/TTS payload 계약까지 회귀화했다. 백엔드 회귀 테스트는 84개로 재검증했고, 페르소나 승인/반려 API와 교수 콘솔 검수 UI proof를 추가했다. Postgres RLS/audit live smoke checker와 s2s 2차 PoC 판단 메모도 준비했지만, live DB 실행과 s2s 채택 결정은 별도 증거가 필요하다. 이후 로컬 5175 런타임 재기동, admin 사용자 관리 E2E 안정화, 학습자 홈 0회 기록 UX, active session 실제 SSE 스트림 전환, 텍스트/음성 실패 턴 미저장, 직접 URL persona 우회 차단, Live2D demo 자산 제거, AI 운영 fail-closed 보강까지 전체 Playwright로 확인했다. 2026-06-26 15:18 KST 기준 공개 API 터널을 복구한 뒤 전체 Playwright 72개가 desktop/mobile/single-run 모두 통과했다. Cloudflare Pages production 배포 2a58ca49도 완료했고, 커스텀 도메인이 새 asset을 서빙하는 것을 확인했다. 과거 Mao/Haru Live2D demo 자산은 repo와 번들에서 제거했고, /live2d/* 운영 경로는 404로 차단했다. 현재 런타임은 prod API 8001, dev API 8000/8010, web 5173, engine gateway 9099, cloudflared tunnel 1개를 의도적으로 유지한다. 이번 라운드에서는 레이아웃 작업을 서브에이전트 전담으로 분리해 세션, 학습자 홈/리뷰, 관리자/교수자, 설정 화면을 overflow 수리 수준이 아니라 실제 작업 화면 재설계 수준으로 갱신했다. 통합 레이아웃 E2E는 desktop/mobile 54개를 최종 통과했고, 자세한 인수인계는 docs/ops/layout-redesign-handoff-2026-06-26.md에 남겼다. 이번 이어받기에서는 핸드오프가 남긴 "실기기 시각 수용"을 엄격한 자동 게이트로 닫았다. 7개 권장 너비(390/720/861/900/1024/1280/1440)에서 초기 7개 재설계 화면의 가로 overflow 0·컨트롤 클립 0을 강제하는 layout-visual-gate.spec.ts(초기 7 passed/49장, 현재 빈 회기리뷰 포함 9 passed/63장)를 만들고, 화면별 1에이전트 병렬 시각 리뷰 → 6개 병렬 수정 에이전트 → 화면별 적대적 재검수까지 돌려 7/7 accept(critical/major 0, 회귀 0)를 확정했다. 학습자홈 1280 3열 가운데 컬럼 붕괴(critical)와 교수 'API 404' raw 배너를 포함한 결함을 모두 해소했고, 레이아웃 항목을 DONE으로 닫았다. 이번 로그인 복구 라운드에서는 dev/Tailnet OAuth direct start까지 local_oauth_unavailable로 막고, public OAuth state를 HttpOnly 쿠키에 묶인 서명 토큰으로 바꿔 API 재시작 후 콜백이 invalid_state로 실패하지 않게 했으며, callback 실패 reason/status 진단 로그와 화면 메시지 분기를 추가했다. 추가로 Google provider error를 access_denied/provider_error로 분리하고 로그인 화면에 reason code를 노출했다. 이어서 C1 사례개념화 워크시트는 리뷰 API·편집 UI·app.case_worksheet 저장/재조회 경로까지 2차로 연결했다. 공개 Google OAuth 실제 /turn 증명만 storageState 캡처 후 남아 있다. + 이번 세션은 seed persona 결합 제거, 비개발 fail-closed 설정, compose OAuth 주입, admin 모바일 overflow 수정, persona 인증 경계 보강, 전체 Playwright 재통과, 공개 API prod 전환까지 완료했다. 이번 이어받기에서는 내담자 AI 생성 턴의 provider/model/token/cost 텔레메트리를 /turn, SSE stream, 음성 생성 경로에서 DB app.turns까지 적재하도록 연결했고, gateway stream done/error 계약을 구조화했으며, P1 상담 시작→텍스트 stream→종료→리뷰 피드백 UI E2E와 저항 엔진 정량 회귀 테스트를 추가했다. 이어서 PII 마스킹 게이트 우회 차단, RBAC/IDOR·감사 insert 경로, 음성 paralinguistic 메타 저장, 공개 런타임 watchdog 스크립트와 runbook을 보강했다. 추가로 상주 engine session reuse, SAML provider 스캐폴드, visible_to 런타임 필터, EOT/TTS payload 계약까지 회귀화했다. 백엔드 회귀 테스트는 84개로 재검증했고, 페르소나 승인/반려 API와 교수 콘솔 검수 UI proof를 추가했다. Postgres RLS/audit live smoke checker와 s2s 2차 PoC 판단 메모도 준비했지만, live DB 실행과 s2s 채택 결정은 별도 증거가 필요하다. 이후 로컬 5175 런타임 재기동, admin 사용자 관리 E2E 안정화, 학습자 홈 0회 기록 UX, active session 실제 SSE 스트림 전환, 텍스트/음성 실패 턴 미저장, 직접 URL persona 우회 차단, Live2D demo 자산 제거, AI 운영 fail-closed 보강까지 전체 Playwright로 확인했다. 2026-06-26 15:18 KST 기준 공개 API 터널을 복구한 뒤 전체 Playwright 72개가 desktop/mobile/single-run 모두 통과했다. Cloudflare Pages production 배포 2a58ca49도 완료했고, 커스텀 도메인이 새 asset을 서빙하는 것을 확인했다. 과거 Mao/Haru Live2D demo 자산은 repo와 번들에서 제거했고, /live2d/* 운영 경로는 404로 차단했다. 현재 런타임은 prod API 8001, dev API 8000/8010, web 5173, engine gateway 9099, cloudflared tunnel 1개를 의도적으로 유지한다. 이번 라운드에서는 레이아웃 작업을 서브에이전트 전담으로 분리해 세션, 학습자 홈/리뷰, 관리자/교수자, 설정 화면을 overflow 수리 수준이 아니라 실제 작업 화면 재설계 수준으로 갱신했다. 통합 레이아웃 E2E는 desktop/mobile 54개를 최종 통과했고, 자세한 인수인계는 docs/archive/ops/layout-redesign-handoff-2026-06-26.md에 남겼다. 이번 이어받기에서는 핸드오프가 남긴 "실기기 시각 수용"을 엄격한 자동 게이트로 닫았다. 7개 권장 너비(390/720/861/900/1024/1280/1440)에서 초기 7개 재설계 화면의 가로 overflow 0·컨트롤 클립 0을 강제하는 layout-visual-gate.spec.ts(초기 7 passed/49장, 현재 빈 회기리뷰 포함 9 passed/63장)를 만들고, 화면별 1에이전트 병렬 시각 리뷰 → 6개 병렬 수정 에이전트 → 화면별 적대적 재검수까지 돌려 7/7 accept(critical/major 0, 회귀 0)를 확정했다. 학습자홈 1280 3열 가운데 컬럼 붕괴(critical)와 교수 'API 404' raw 배너를 포함한 결함을 모두 해소했고, 레이아웃 항목을 DONE으로 닫았다. 이번 로그인 복구 라운드에서는 dev/Tailnet OAuth direct start까지 local_oauth_unavailable로 막고, public OAuth state를 HttpOnly 쿠키에 묶인 서명 토큰으로 바꿔 API 재시작 후 콜백이 invalid_state로 실패하지 않게 했으며, callback 실패 reason/status 진단 로그와 화면 메시지 분기를 추가했다. 추가로 Google provider error를 access_denied/provider_error로 분리하고 로그인 화면에 reason code를 노출했다. 이어서 C1 사례개념화 워크시트는 리뷰 API·편집 UI·app.case_worksheet 저장/재조회 경로까지 2차로 연결했다. 공개 Google OAuth 실제 /turn 증명만 storageState 캡처 후 남아 있다.

diff --git a/docs/guides/architecture.md b/docs/guides/architecture.md index c762cff..6a6f65a 100644 --- a/docs/guides/architecture.md +++ b/docs/guides/architecture.md @@ -250,10 +250,15 @@ RBAC×AIView로 차단된다. 이 모듈은 평가 신호만 산출한다. content_hash 기반으로 증분 색인한다. `app.services.source_pack_sync`가 active `kb.document`의 hash/version을 먼저 읽고, hash 변경 시 새 document version을 최신+1로 계산한다. 기본 visibility는 evaluator 전용이고, C/D·diagnostic 자료는 sensitivity 2로 고정한다. -- 출력은 `LiveCoachSuggestion` 구조화 JSON(`tone/focus/title/message/next_utterance/rationale/sources`). - UI는 코칭 아바타 말풍선, 근거 모달, 발화별 코칭 이력 오버레이로 표시한다. -- 전달된 코칭은 `app.live_coach_events`에 저장된다. 원문 축어록을 중복 저장하지 않고 PII 마스킹된 짧은 - learner/client excerpt와 코칭 payload만 저장한다. DB 미가용 dev에서는 런타임 캐시로 폴백한다. +- 출력은 `LiveCoachSuggestion` 구조화 JSON(`tone/focus/title/message/next_utterance/rationale/sources/quota/credit_events`) + 형태가 아니라 `quota`와 `credit_events`를 포함한 짧은 카드 계약이다. UI는 코칭 아바타 말풍선, + 근거 모달, 발화별 코칭 이력 오버레이로 표시한다. +- 회기별 코칭 기회는 기본/최대 3개다. `POST /sessions/{id}/live-coach` 사용 시 `use/-1` 이벤트를 남기고, + 턴 평가가 `appropriateness=pos`, `rapport_signal>=0.35`, 그리고 단계 전환 또는 `effective_openness` + 상승을 동시에 만족하면 `recharge/+1` 이벤트로 1개를 재충전한다(최대 3개). 사용권 없음은 409로 막는다. +- 전달된 코칭과 충전/사용 기록은 `app.live_coach_events`에 저장된다. 원문 축어록을 중복 저장하지 않고 + PII 마스킹된 짧은 learner/client excerpt, 코칭 payload, `event_type/credit_delta/credit_balance/reason`만 + 저장한다. DB 미가용 dev에서는 런타임 캐시로 폴백한다. - DSM/공식 지침/논문은 사용 허가된 source pack으로 투입할 수 있다. 다만 코칭 프롬프트와 UI에는 장문 원문이나 공식 문항을 재현하지 않고, chunk summary + version/citation + 출처 식별자로 노출한다. @@ -336,9 +341,11 @@ session lifecycle을 유지하며, future Node read API는 이 read-model contra - `POST /sessions/{id}/stream` — SSE 경로(`run_turn_stream`). token/done/ping/error를 흘리고, done 시점에 학습자 발화 + 누적 내담자 응답을 영속화. `sse_heartbeat_seconds`마다 ping(Cloudflare 타임아웃 회피). - `POST /sessions/{id}/live-coach` — 방금 완료된 상담자 발화를 워크북 요약/RAG/evaluator fast-loop 신호와 대조해 - 라이브 코칭 카드 1개를 반환하고 `app.live_coach_events`에 저장한다. 저장 payload는 마스킹 excerpt와 코칭 구조화 JSON이다. + 라이브 코칭 카드 1개를 반환하고 `app.live_coach_events`에 `use/-1`로 저장한다. 저장 payload는 마스킹 + excerpt와 코칭 구조화 JSON이며, 잔여 기회가 없으면 409로 차단한다. - `GET /sessions/{id}/live-coach` — 현재 회기에서 학습자에게 실제 전달된 코칭 이력을 시간순으로 반환한다. - 세션 화면은 이 응답을 `turn_seq`별로 묶어 학습자 발화 우측 코칭 마커와 채팅 위 스크롤 오버레이에 표시한다. + 세션 화면은 이 응답을 `turn_seq`별로 묶어 학습자 발화 우측 코칭 마커와 채팅 위 스크롤 오버레이에 표시하고, + `quota`/`credit_events`로 사용권 점 표시와 사용·충전 애니메이션을 갱신한다. - `POST /kb/live-coach/source-packs/sync` — 관리자 전용. 로컬 라이브 코칭 source pack을 `kb.source` upsert 후 `kb.document/kb.chunk`로 색인한다. active hash가 같으면 skip하고, 다르면 최신 document version+1로 색인한다. 임베딩 모델이 없으면 BM25-only degraded 색인으로 진행한다. @@ -375,6 +382,8 @@ session lifecycle을 유지하며, future Node read API는 이 read-model contra - `GET /teacher/dashboard` — 담당 학습자 성장, 안전 알림, 종료 회기 검토 큐를 반환한다. 회기 요약에는 `review_status`, `review_note`, `reviewed_at`를 포함해 교수자가 이미 검토한 회기를 구분한다. + 학습자 표시는 `app.app_user.display_name`/`nickname`/`email`을 우선 사용하고, 사용자 row가 없을 때만 + 축약 learner id로 폴백한다. - `GET /teacher/learners/{learner_id}/analysis` — 담당 범위 안의 특정 학습자 전체 회기 타임라인을 오래된 순서로 반환한다. `summary`/`points`는 제한 없는 사용자별 추이를 담고, `stage_breakdown`은 라포·탐색·개입·정리 단계별 회기 수와 턴 수를 담는다. 담당 범위 밖 learner id는 404로 닫는다. @@ -384,7 +393,9 @@ session lifecycle을 유지하며, future Node read API는 이 read-model contra `/teach/analysis` 학생 분석은 별도 메뉴/라우트로 분리되어 전체 담당 학습자를 검색 가능한 1행 테이블로 먼저 보여주고, 우측 펼침으로 행 아래 미니 추이·기법·최근 기록을 확장한다. 학습자 이름 또는 상세 보기 버튼은 `?learner=` 상세 드릴다운으로 이동하며, 상세 화면은 - 추이·전체 회기·단계 분석 탭으로 선택 학습자의 전체 이력을 보여준다. `/teach/session/:sessionId/review` 화면은 같은 + 페르소나별 회기를 기본 탭으로 먼저 보여준다. 각 페르소나 행은 회기 수·평균 적절성·라포·검토 + 카운트를 요약하고 펼침으로 해당 페르소나의 회기 목록을 연다. 추이·전체 회기·단계 분석 탭은 같은 + 상세 화면에서 선택 학습자의 전체 이력을 보조한다. `/teach/session/:sessionId/review` 화면은 같은 `GET /sessions/{id}/review` 자료를 교수자 읽기 전용으로 표시하고, 검토 메모 저장은 위 teacher endpoint로 분리한다. ### 2.9.2 운영 메일 알림 — `app/services/notifications.py` diff --git a/docs/guides/source-docs-and-gaps.md b/docs/guides/source-docs-and-gaps.md index 83e1a4b..c17b562 100644 --- a/docs/guides/source-docs-and-gaps.md +++ b/docs/guides/source-docs-and-gaps.md @@ -23,7 +23,7 @@ | SSOT 대시보드 | `docs/dev_dashboard.html` (`#` "원천문서 갭 분석" 섹션) | 상태·우선순위의 권위 기준, 갭 요약 카드 | | 데이터 거버넌스 게이트 | `docs/ops/hanshin-data-governance-gate.md` | H4/M3 관련 거버넌스 | | 백로그 | `docs/ops/backlog-2026-06-26.md` | 실행 작업 목록 | -| 레이아웃 핸드오프 | `docs/ops/layout-redesign-handoff-2026-06-26.md` | UI 재설계 맥락 | +| 레이아웃 핸드오프(아카이브) | `docs/archive/ops/layout-redesign-handoff-2026-06-26.md` | UI 재설계 맥락(냉동 보관) | --- @@ -52,26 +52,26 @@ | ID | 갭 | 현재상태 | 권고 | 근거 | |---|---|---|---|---| | **C1** | 사례개념화·치료계획 산출물 저장형 구조 4차 구현·루브릭 외부화 scaffold·수동 교수자 검수 골격 완료·채점/정책 잔여 | `SessionReviewResponse.caseWorksheet`가 축어록 근거 기반 초안을 제공하고, 리뷰 화면에서 학습자가 편집한 워크시트를 `PUT /sessions/{id}/review/worksheet`로 `app.case_worksheet`에 저장한다. 이후 `GET /review`는 `saved_by_learner` 저장본을 자동 초안보다 우선 반환한다. 3차에서는 `data/rubrics/case-worksheet-rubric.json`, Draft 2020-12 schema, `app.services.case_worksheet_rubric`, `scripts/check-case-worksheet-rubric.py`를 추가해 임상팀 확정 루브릭을 코드 수정 없이 받을 scaffold를 만들었다. 현재 rubric은 `scaffold_only`/`scoring_enabled=false`이며 서버 생성 워크시트 5개 section/28개 item key와 일치하는지만 검증한다. 템플릿 key source는 `CASE_WORKSHEET_SECTION_SPECS`와 `case_worksheet_template_item_keys()`로 명시해 validator/CLI/test가 더미 턴 없이 같은 생성 spec을 참조한다. 4차에서는 기존 `app.session_review_status`에 `worksheet_status`/`worksheet_note`/`worksheet_reviewed_at`을 붙이고, `PUT /teacher/sessions/{id}/review-status`와 교수자 `SessionReview` 카드에서 승인·수정요청·반려 수동 판정을 저장/표시한다. 2026-07-01 E2E는 학습자 저장본이 교수자 read-only 검토 화면으로 복원되고 `수정요청` 검수 메모가 DB-backed `teacherReview`로 재조회되는지 검증한다. CCD는 숨은 정답키로 남고 저장본에 점수로 노출하지 않는다. | 임상팀 확정 루브릭 콘텐츠(level anchor, 가중치, 컷오프, 예시 답안), AI 채점 적용·calibration, 승인 후 잠금·재제출 정책, 재귀학습 데이터셋 approved 연계. | doc1·doc4·doc5 | -| **C2** | 위기개입 프로토콜·생명유지서약·에스컬레이션 1차 배선 완료·임상 고도화 필요 | 실제 자해·자살 신호는 LLM/엔진 호출 전 중단하고 109 리소스와 `conversation_stopped`를 REST/SSE/voice 응답에 싣는다. `crisis.risk_level`은 상태머신 `ideation_observed`로 전달하고, escalate 시 `app.safety_events` detail 적재 및 교수자 대시보드 안전 알림 큐로 연결한다. 2026-07-01 E2E는 실제 브라우저 위기 발화가 `app.safety_events`를 거쳐 DB-backed 교수자 안전 큐의 109 알림으로 표시되는지 검증한다. `db.healthcheck()`는 이제 `app.safety_events` schema를 요구하고, JSONB detail double-encoding legacy row도 안전하게 읽는다. | 남은 것은 임상팀 콘텐츠: 비밀보장 예외고지→단계적 탐색→생명유지서약 스크립트, 실시간 push/메일 알림 정책, 위기탐색 누락 시 회기리뷰 감점 루브릭. | doc1·doc5(핵심 시나리오)·doc4(IRB 전제) | -| **C3** | 이론모드 2차 배선·학습자 명시 선택 UI 완료·CBT 콘텐츠 잔여 | `TurnContext`/`prepare_turn`/sessions/voice/evaluator에 `theory_mode`가 전달되고, `build_turn_messages`도 인간중심·CBT·통합 이론 프레이밍을 엔진 메시지에 넣는다. 프론트는 `persona.theory_target` 기준 기본값을 잡되, 세션 시작 전 `humanistic`/`cbt`/`integrative` segmented control로 학습자가 명시 선택하고 `POST /sessions`의 `theory_mode`로 전송한다. | 임상팀이 확정한 CBT 단계 프롬프트 체인, 이론부합 채점 루브릭. 이번 UI는 임상 문안·루브릭·백엔드 enum을 확장하지 않았다. | doc4(humanistic+CBT 필수)·doc2/doc5 | +| **C2** | 위기개입 프로토콜·생명유지서약·에스컬레이션 1차 배선 완료·임상 고도화 필요 | 실제 자해·자살 신호는 LLM/엔진 호출 전 중단하고 109 리소스와 `conversation_stopped`를 REST/SSE/voice 응답에 싣는다. `crisis.risk_level`은 상태머신 `ideation_observed`로 전달하고, escalate 시 `app.safety_events` detail 적재 및 교수자 대시보드 안전 알림 큐로 연결한다. 2026-07-01 E2E는 실제 브라우저 위기 발화가 `app.safety_events`를 거쳐 DB-backed 교수자 안전 큐의 109 알림으로 표시되는지 검증한다. 추가 E2E는 같은 위기 발화가 교수자 검토 축어록에는 learner turn으로 남고 client AI 응답 turn은 저장되지 않는지 확인한다. `db.healthcheck()`는 이제 `app.safety_events` schema를 요구하고, JSONB detail double-encoding legacy row도 안전하게 읽는다. | 남은 것은 임상팀 콘텐츠: 비밀보장 예외고지→단계적 탐색→생명유지서약 스크립트, 실시간 push/메일 알림 정책, 위기탐색 누락 시 회기리뷰 감점 루브릭. | doc1·doc5(핵심 시나리오)·doc4(IRB 전제) | +| **C3** | 이론모드 2차 배선·학습자 명시 선택 UI 완료·CBT 콘텐츠 잔여 | `TurnContext`/`prepare_turn`/sessions/voice/evaluator에 `theory_mode`가 전달되고, `build_turn_messages`도 인간중심·CBT·통합 이론 프레이밍을 엔진 메시지에 넣는다. 프론트는 `persona.theory_target` 기준 기본값을 잡되, 세션 시작 전 `humanistic`/`cbt`/`integrative` segmented control로 학습자가 명시 선택하고 `POST /sessions`의 `theory_mode`로 전송한다. 2026-07-01 E2E는 실제 Session UI에서 `CBT`를 선택한 회기가 DB-backed `GET /sessions/{id}` 상세에서도 `theory_mode=cbt`로 복원되는지 검증한다. | 임상팀이 확정한 CBT 단계 프롬프트 체인, 이론부합 채점 루브릭. 이번 UI는 임상 문안·루브릭·백엔드 enum을 확장하지 않았다. | doc4(humanistic+CBT 필수)·doc2/doc5 | ### High (4) | ID | 갭 | 현재상태 | 권고 | 근거 | |---|---|---|---|---| -| **H1** | 계약 평가 KPI(자기효능감·기술숙련도·수련만족도 사전사후) 수집·입력·CSV/report 계산 2차 | `app.learner_prepost_measure`, 학습자 본인용 `GET/PUT /users/me/prepost-measures`, `SessionReview`의 파일럿 증거 원장 카드가 3척도 pre/post 1~5 aggregate evidence를 저장·조회한다. `app.services.phase3_kpi_export`와 `scripts/export-phase3-kpi.py`는 원장 row를 Phase 3 evidence root의 `02-measures/prepost_measures.csv`와 `02-measures/kpi_report.json` scaffold로 산출한다. participant id는 가명화하고, 3척도 paired normalized mean pre/post/delta, complete/missing pair를 계산한다. 2차에서는 `phase3_kpi_contract.py`가 KPI metric 이름·필수키·`computed_prepost`/`design_pending` status 값을 소유해 exporter/checker/test의 drift를 줄이고, 계산 가능한 pre/post evidence와 평가설계 전 미계산 KPI를 report 안에서 구분한다. | 공식 문항 확정, 실험/통제군 배정, 추이 시각화, 통계검정 종류/alpha/결측 처리, 실제 20명 evidence와 steward/legal/IAA 검수. 현재 API/UI/export는 공식 효과성·성적·수료 판정이 아니라 파일럿 evidence 계산이다. (κ/ICC·환각률은 doc4 미명시 → 평가설계 확정.) | doc4(20명 실험/통제군·단회기 50분·3척도 pre-post) | -| **H2** | 턴별 fast-loop + 라이브 코칭 1차 가동·학습자 리뷰 3열 workbench 완료·골든셋 잔여 | `make_eval_hook`이 submit/voice 생성 경로에 주입되고, stream은 `_evaluate_stream_turn`으로 fast-loop 평가를 붙인다. 결과는 `feedback_scores`, `alternative_utterance` 등 정규화 테이블에 적재·hydrate된다. 추가로 `app/services/live_coach.py`, `POST/GET /sessions/{id}/live-coach`, `POST /kb/live-coach/source-packs/sync`, `app.live_coach_events`, `data/kb/live_coaching_workbook_0615.json`, `data/kb/live_coaching_sources/*.json`을 연결해 워크북·DSM·공식 지침 요약 기반 코칭 아바타 말풍선·근거 모달·발화별 이력 오버레이와 RAG 증분 색인을 제공한다. `app.services.source_pack_sync`가 repo source pack의 active `content_hash`를 비교하고 변경 시 document version을 최신+1로 올린다. live turn은 프로세스 로컬 source pack snapshot을 재사용하지만, 관리자 sync/CLI는 `refresh=True`로 캐시를 비운 뒤 repo 파일을 다시 읽어 stale `content_hash` 비교를 막는다. 학습자 `SessionReview` 데스크톱은 요약/흐름, 축어록, 평가 rail의 3열 workbench와 하단 워크시트로 재배치했고, empty review는 2열 이하로 유지한다. 워크시트/pre-post/교수자 메모 입력과 발화 이동 버튼은 name/autocomplete/aria-label 및 공유 focus token으로 접근성 회귀 표면을 줄였다. | 원천 축어록 few-shot 골든셋 적재, 임상팀 확정 루브릭과 source pack 임상 검수 상태 운영정책 보강. | doc2·doc5(골드 포맷) | +| **H1** | 계약 평가 KPI(자기효능감·기술숙련도·수련만족도 사전사후) 수집·입력·CSV/report 계산 2차 | `app.learner_prepost_measure`, 학습자 본인용 `GET/PUT /users/me/prepost-measures`, `SessionReview`의 파일럿 증거 원장 카드가 3척도 pre/post 1~5 aggregate evidence를 저장·조회한다. 기존 저장 점수를 빈 칸으로 지우는 입력은 `저장된 값 기준`으로 오인하지 않고 invalid 상태와 저장 차단으로 표면화한다. `app.services.phase3_kpi_export`와 `scripts/export-phase3-kpi.py`는 원장 row를 Phase 3 evidence root의 `02-measures/prepost_measures.csv`와 `02-measures/kpi_report.json` scaffold로 산출한다. participant id는 가명화하고, 3척도 paired normalized mean pre/post/delta, complete/missing pair를 계산한다. 2차에서는 `phase3_kpi_contract.py`가 KPI metric 이름·필수키·`computed_prepost`/`design_pending` status 값을 소유해 exporter/checker/test의 drift를 줄이고, 계산 가능한 pre/post evidence와 평가설계 전 미계산 KPI를 report 안에서 구분한다. | 공식 문항 확정, 실험/통제군 배정, 추이 시각화, 통계검정 종류/alpha/결측 처리, 실제 20명 evidence와 steward/legal/IAA 검수. 현재 API/UI/export는 공식 효과성·성적·수료 판정이 아니라 파일럿 evidence 계산이다. (κ/ICC·환각률은 doc4 미명시 → 평가설계 확정.) | doc4(20명 실험/통제군·단회기 50분·3척도 pre-post) | +| **H2** | 턴별 fast-loop + 라이브 코칭 1차 가동·학습자 리뷰 3열 workbench 완료·골든셋 잔여 | `make_eval_hook`이 submit/voice 생성 경로에 주입되고, stream은 `_evaluate_stream_turn`으로 fast-loop 평가를 붙인다. 결과는 `feedback_scores`, `alternative_utterance` 등 정규화 테이블에 적재·hydrate된다. fast-loop 훅 예외와 evaluator error dict는 정상 neutral 점수로 둔갑하지 않게 `evaluation.error`로 남기고 리뷰의 `턴 평가 실패` 노트로 표면화하며, 학습자 성장 점수·최근 피드백 집계에서는 제외한다. 교수자 `POST /eval/sessions/{id}/turn` 재평가도 응답 바디에만 머물지 않고 같은 normalized evaluation row를 교체 저장하며, 저장 실패는 503으로 표면화한다. 추가로 `app/services/live_coach.py`, `POST/GET /sessions/{id}/live-coach`, `POST /kb/live-coach/source-packs/sync`, `app.live_coach_events`, `data/kb/live_coaching_workbook_0615.json`, `data/kb/live_coaching_sources/*.json`을 연결해 워크북·DSM·공식 지침 요약 기반 코칭 아바타 말풍선·근거 모달·발화별 이력 오버레이와 RAG 증분 색인을 제공한다. `POST /live-coach`는 `persistence_source`를 반환하고, `GET /live-coach` 실패나 runtime 이력은 빈 정상 이력이 아니라 alert/임시 저장 경고로 표면화한다. `app.services.source_pack_sync`가 repo source pack의 active `content_hash`를 비교하고 변경 시 document version을 최신+1로 올린다. live turn은 프로세스 로컬 source pack snapshot을 재사용하지만, 관리자 sync/CLI는 `refresh=True`로 캐시를 비운 뒤 repo 파일을 다시 읽어 stale `content_hash` 비교를 막는다. RAG 회수 근거의 `source_version`/`citation`/`source_type`도 `LiveCoachGrounding`과 UI source list까지 보존한다. 2026-07-01 E2E는 실제 DB/engine 코칭 응답의 0615 source metadata가 근거 모달·DB history·reload 후 `C` history dialog에 남는지, 단일 턴 교수자 재평가 라벨이 DB-backed 리뷰에 hydrate되는지, 그리고 관리자 source pack sync가 실제 evaluator RAG/source-scoped retrieval로 이어지는지 검증한다. 학습자 `SessionReview` 데스크톱은 요약/흐름, 축어록, 평가 rail의 3열 workbench와 하단 워크시트로 재배치했고, empty review는 2열 이하로 유지한다. 워크시트/pre-post/교수자 메모 입력과 발화 이동 버튼은 name/autocomplete/aria-label 및 공유 focus token으로 접근성 회귀 표면을 줄였다. | 원천 축어록 few-shot 골든셋 적재, 임상팀 확정 루브릭과 source pack 임상 검수 상태 운영정책 보강. | doc2·doc5(골드 포맷) | | **H3** | 임상팀 콘텐츠 입력 경로(페르소나 저작 CRUD) 2차 구현·원문 격리 정책 1차·항목형 목록 저작/프롬프트 검토 UI 완료·임상 검수 잔여 | draft 생성·조회·편집·검수요청 API와 교수 콘솔 JSON 초안 패널은 연결됐다. `persona_repository.py`는 in-code `SEED_PERSONAS`(P1~P3)와 `data/personas/P4.json`~`P7.json`을 `PersonaCard`로 합쳐 `materialize_seed_personas()`와 seed fallback catalog에 포함한다. `scripts/materialize-persona-seeds.py`는 같은 seed manifest를 dry-run 기본으로 보고하고, `--apply`에서만 DB pool을 초기화한 뒤 기존 idempotent DB materializer를 호출한다. `scripts/sync-persona-sources.py`는 DB-backed dry-run/apply runner로 repo-managed source pack의 `content_hash`/document version을 비교한다. RAG 기반 draft 생성은 source/chunk evidence와 함께 `persona-draft-rag@2026-06-28.1` prompt bundle id/version/hash를 engine metadata 및 draft `source_provenance`에 남긴다. `POST /personas/sources`는 raw 원문 hash-only 증거를 `kb.raw_source_artifact`에 따로 기록하고, sanitized 파생본만 evaluator-only RAG chunk로 색인한다. `rag.index_document()`는 `sensitivity=3` 또는 raw marker chunk를 DB 접근 전에 차단한다. `app/persona_read_model.py`는 catalog/review/draft/source/evidence DTO와 mapper를 route에서 분리해 OpenAPI schema 이름을 유지한다. PersonaStudio는 자동사고, 회기 시나리오, 말투 filler/verbal tic/nonverbal cue, 역린·금기 응답·금기어를 행 추가/삭제 UI로 편집하고, 저장 직전 빈 항목을 제거하되 기존 배열 schema를 유지한다. 프롬프트 탭은 raw JSON textarea 대신 L1 카드·인적 범주·임상 배경·말투·수치 파라미터·역린·회기 시나리오·추가 계약 섹션으로 같은 draft 데이터를 검토하게 한다. | 루브릭·이론 콘텐츠 외부화, P4~P7 포함 임상팀 최종 검수/서면 evidence 확보, 암호화 blob/vault 기반 원문 실저장. | doc3(R&R)·doc4(페르소나=전문가 산출물) | -| **H4** | PII 마스킹 한국어 로컬 휴리스틱 + optional ko recognizer adapter + 16-case fixture/schema 평가 harness + 온보딩·동의 게이트 잔여 | Presidio `language='en'` 고정 한계를 보완하기 위해 정규식 폴백에 한국어 날짜·금액·행정구역 주소와 함께 이름/성명 라벨, 성씨+이름+조사/호칭, 대학교·학과·병원·센터 등 기관 suffix 기반 로컬 휴리스틱 마스킹을 추가했다. 66차에서는 `제 이름은 김서연입니다`, `보호자 이름은 박민수입니다`, `저는 최하늘입니다` 같은 자연 발화형 이름 라벨·자기소개 패턴을 추가하고 `이름은 중요하지 않다` negative control로 오탐을 막았다. Presidio가 설치돼도 한국어 누락을 막기 위해 fallback을 후단에 한 번 더 태운다. 70차에서는 `guardrail.mask_pii()` 내부에 선택형 한국어 PII recognizer adapter 경계를 추가했다. adapter는 import-time hard dependency가 아니며 명시 등록 전에는 비활성이고, 실패해도 기존 regex fallback이 마지막 안전망으로 유지된다. fake adapter 테스트는 regex가 못 잡는 별명/기관 span을 `[NAME]`/`[ORG]`로 마스킹하고 같은 문장의 전화번호는 후단 regex가 `[PHONE]`으로 처리하는지, adapter 실패 시에도 fallback이 유지되는지 검증한다. 상담 생성(generate/stream), fast evaluator prompt, client turn `text_masked`에서 한국어 NAME/ORG raw 값이 남지 않도록 회귀화했다. `app.services.pii_masking_eval`, `data/privacy/pii-masking-ko-fixtures.json`, `scripts/evaluate-pii-masking.py`로 합성 fixture 16케이스를 NAME/ORG/PHONE/EMAIL/RRN/NUMID/DATE/MONEY/ADDR/negative-control 범위에서 entity recall·forbidden substring removal·unexpected entity violation으로 평가한다. `data/privacy/pii-masking-eval-input.schema.json`과 `data/privacy/pii-masking-eval-report.schema.json`은 source/category/severity metadata와 summary-only `technical_dry_run` 리포트 계약을 고정하며, 기본 CLI JSON은 `masked_text`/`forbidden_remaining` 원문 증거를 제외한다. `소속`/`안내`/`이름` 및 `진로는` 같은 NAME 오탐도 보정했다. 외부 LLM 호출은 상담 생성(generate/stream)·fast/deep 평가 직후 `audit.llm_call_log`에 provider/model/token/cost/inference_geo/latency만 적재하도록 연결했고, prompt/completion 본문은 저장하지 않는다. 로컬 dev-login 실제 `/turn` smoke에서 `audit.llm_call_log` 3행 증가를 확인했다. `app_user`에 이름·소속·학과·학년/직위·연락처·주소/수령지·닉네임·자기소개·아바타 URL·약관/개인정보 동의 버전 필드를 추가했고, 로그인 직후 `/onboarding` 완료 전에는 역할 홈과 learner 회기 시작을 막는다. 아바타 이미지는 `/users/me/avatar`에서 MIME/시그니처/3MB 제한 후 파일 저장소에 두고 URL만 보관한다. 온보딩 저장 시 learner `consent_at`도 함께 세팅하며 auth E2E에서 신규 계정 온보딩→아바타 업로드→학습자 홈 이동을 검증했다. | 실제 ko recognizer 모델/provider 선정, 운영 말뭉치 기반 오탐/미탐 평가, 미성년/guardian 및 법무 검토가 필요한 최종 서명 동의서·개인정보 처리방침·약관 evidence 확보. 공개 Google OAuth 실제 `/turn` proof는 별도 운영 게이트. | doc1/2/5(실명·날짜·미성년·자살시도 다수)·doc4(IRB·개인정보) | +| **H4** | PII 마스킹 한국어 로컬 휴리스틱 + optional ko recognizer adapter + 16-case fixture/schema 평가 harness + 온보딩·동의 게이트 보강 | Presidio `language='en'` 고정 한계를 보완하기 위해 정규식 폴백에 한국어 날짜·금액·행정구역 주소와 함께 이름/성명 라벨, 성씨+이름+조사/호칭, 대학교·학과·병원·센터 등 기관 suffix 기반 로컬 휴리스틱 마스킹을 추가했다. 66차에서는 `제 이름은 김서연입니다`, `보호자 이름은 박민수입니다`, `저는 최하늘입니다` 같은 자연 발화형 이름 라벨·자기소개 패턴을 추가하고 `이름은 중요하지 않다` negative control로 오탐을 막았다. Presidio가 설치돼도 한국어 누락을 막기 위해 fallback을 후단에 한 번 더 태운다. 70차에서는 `guardrail.mask_pii()` 내부에 선택형 한국어 PII recognizer adapter 경계를 추가했다. adapter는 import-time hard dependency가 아니며 명시 등록 전에는 비활성이고, 실패해도 기존 regex fallback이 마지막 안전망으로 유지된다. fake adapter 테스트는 regex가 못 잡는 별명/기관 span을 `[NAME]`/`[ORG]`로 마스킹하고 같은 문장의 전화번호는 후단 regex가 `[PHONE]`으로 처리하는지, adapter 실패 시에도 fallback이 유지되는지 검증한다. 상담 생성(generate/stream), fast evaluator prompt, client turn `text_masked`에서 한국어 NAME/ORG raw 값이 남지 않도록 회귀화했다. 이번 보강으로 live coach prompt도 recent turns와 fast-loop evaluation dict의 문자열 leaf를 서비스 내부에서 다시 `mask_pii()`에 통과시켜 legacy row/direct service call/fixture가 raw 값을 넣어도 engine user message에 이름·기관·전화번호가 남지 않게 했다. turn evaluation persistence도 feedback rationale, supervisor comment text/intent deviation, alternative utterance suggestion/rationale을 DB insert 전 마스킹한다. legacy DB row에 `text_masked`가 비어 있고 raw `text`만 남은 경우 `_turn_from_row()`가 raw fallback을 그대로 쓰지 않고 양쪽 필드를 재마스킹본으로 복원한다. session-end deep evaluation은 `SessionEvaluationWrite` 생성과 `save_session_evaluation()` 직전, 그리고 legacy DB payload read-model 투영 시점에 문자열 leaf를 재마스킹해 `strengths`/`improvements`/`supervisor_*`/`nextLine`으로 raw PII가 노출되지 않게 했다. `test_evaluation_persistence.py`는 `IsolatedAsyncioTestCase`로 전환해 async persistence 테스트가 실제 await되도록 고정했다. `app.services.pii_masking_eval`, `data/privacy/pii-masking-ko-fixtures.json`, `scripts/evaluate-pii-masking.py`로 합성 fixture 16케이스를 NAME/ORG/PHONE/EMAIL/RRN/NUMID/DATE/MONEY/ADDR/negative-control 범위에서 entity recall·forbidden substring removal·unexpected entity violation으로 평가한다. `data/privacy/pii-masking-eval-input.schema.json`과 `data/privacy/pii-masking-eval-report.schema.json`은 source/category/severity metadata와 summary-only `technical_dry_run` 리포트 계약을 고정하며, 기본 CLI JSON은 `masked_text`/`forbidden_remaining` 원문 증거를 제외한다. `소속`/`안내`/`이름` 및 `진로는` 같은 NAME 오탐도 보정했다. 외부 LLM 호출은 상담 생성(generate/stream)·fast/deep 평가 직후 `audit.llm_call_log`에 provider/model/token/cost/inference_geo/latency만 적재하도록 연결했고, prompt/completion 본문은 저장하지 않는다. 로컬 dev-login 실제 `/turn` smoke에서 `audit.llm_call_log` 3행 증가를 확인했다. `app_user`에 이름·소속·학과·학년/직위·연락처·주소/수령지·닉네임·자기소개·아바타 URL·약관/개인정보 동의 버전 필드를 추가했고, 로그인 직후 `/onboarding` 완료 전에는 역할 홈과 learner 회기 시작을 막는다. 아바타 이미지는 `/users/me/avatar`에서 MIME/시그니처/3MB 제한 후 파일 저장소에 두고 URL만 보관한다. 온보딩 저장 시 learner `consent_at`도 함께 세팅하며 auth E2E에서 신규 계정 온보딩→아바타 업로드→학습자 홈 이동을 검증했다. 2026-07-01 E2E는 동의 철회 뒤 이미 생성된 `session_id`로 `/api/voice/ws`에 재접속해도 `consent_required`로 닫히는지 실제 브라우저 WebSocket으로 검증한다. 추가 E2E는 실제 Session UI stream으로 한국어 이름·기관·전화번호 발화를 보내고, DB-backed 세션 상세와 리뷰 응답 모두 raw 값 없이 `[NAME]`/`[ORG]`/`[PHONE]`으로 마스킹되는지 확인한다. | 실제 ko recognizer 모델/provider 선정, 운영 말뭉치 기반 오탐/미탐 평가, 미성년/guardian 및 법무 검토가 필요한 최종 서명 동의서·개인정보 처리방침·약관 evidence 확보. 공개 Google OAuth 실제 `/turn` proof는 별도 운영 게이트. | doc1/2/5(실명·날짜·미성년·자살시도 다수)·doc4(IRB·개인정보) | ### Medium+ (6) — M1~M3, X1~X2, L1 | ID | 갭 | 현재상태 | 권고 | |---|---|---|---| -| **M1** | 비언어/준언어 임상 이벤트 캡처·태깅 4차 진행 | 1차에서 `audio_ref`/`silence_ms`/`speech_rate`/`barge_in`을 learner voice turn에 저장하고 리뷰 `nonverbal` 칩으로 파생했다. 2차에서는 `app.turns.provider_events JSONB`와 `TurnRecord.provider_events`를 추가해 WebSocket control/STT provider 이벤트를 allowlist·size limit 후 보존한다. 3차에서는 저장 전 sanitizer에서 내부 taxonomy `event_type`/`category`를 붙인다. 4차에서는 인증된 회기 리뷰 API가 `sigh`/`cry`/`laugh`/`breath`, prosody, background noise 계열만 한글 label/detail 칩으로 파생 노출한다. raw transcript/text payload, provider/source/raw type, 공개 공유 카드 노출은 제외한다. | 실제 provider 기반 한숨·울음·억양 감지 연결, 장시간 마이크/WSS 실측, 리뷰 칩을 역량 지표로 해석할지에 대한 정책. | +| **M1** | 비언어/준언어 임상 이벤트 캡처·태깅 4차 진행 | 1차에서 `audio_ref`/`silence_ms`/`speech_rate`/`barge_in`을 learner voice turn에 저장하고 리뷰 `nonverbal` 칩으로 파생했다. 2차에서는 `app.turns.provider_events JSONB`와 `TurnRecord.provider_events`를 추가해 WebSocket control/STT provider 이벤트를 allowlist·size limit 후 보존한다. 3차에서는 저장 전 sanitizer에서 내부 taxonomy `event_type`/`category`를 붙인다. 4차에서는 인증된 회기 리뷰 API가 `sigh`/`cry`/`laugh`/`breath`, prosody, background noise 계열만 한글 label/detail 칩으로 파생 노출한다. raw transcript/text payload, provider/source/raw type, 공개 공유 카드 노출은 제외한다. 추가 UI 회귀는 voice final transcript 뒤 턴 저장이 실패할 때 해당 임시 발화를 정상 턴으로 확정하지 않고 `저장 실패` 상태로 표시해 DB/review와 화면 transcript 불일치를 표면화한다. | 실제 provider 기반 한숨·울음·억양 감지 연결, 장시간 마이크/WSS 실측, 리뷰 칩을 역량 지표로 해석할지에 대한 정책. | | **M2** | 다회기 종단 케이스 아크·교차회기 사례개념화 11차 구동 | `(persona_id, learner_id)` 안정 `case_profile` upsert, 원자적 `session_no`, voice/REST/SSE recall cache 주입을 연결했다. 세션 종료 시 마스킹 축어록 기반 fallback `session_summary.digest`, `case_profile.case_digest`, `rapport_trajectory`, `alliance_level`을 갱신하고, 다음 회기 seed recall은 `case_digest`·직전 `session_summary`·client-visible non-contradicted `pinned_fact`를 함께 조립한다. 3차에서는 마스킹된 client-visible 발화에서 `[NAME]`/`[ORG]` identity와 명시적 상담 약속만 보수적으로 `pinned_fact`에 upsert한다. 4차에서는 pinned fact 삽입 또는 값 변경 시 `pinned_fact_history`에 append-only 이력을 남긴다. 5차에서는 명시적 상담 약속 철회/부정만 기존 non-locked `agreement:counseling` fact를 `contradicted`로 격리하고 history reason `contradiction`을 남긴다. 6차에서는 세션 종료 저장 성공 뒤 마스킹된 client-visible 내담자 발화만 `app.turn_embedding`에 BGE-M3 dense/sparse로 idempotent 색인한다. 7차에서는 REST submit/SSE stream의 턴 컨텍스트 조립 경계를 `_prepare_turn_context()`로 묶고, DB seed recall과 pinned fact가 다음 턴 EngineMessage L2/L4에 raw 이름 마스킹 상태로 주입되는 route-level 회귀를 추가했다. 8차에서는 `SessionDigestInput`/`SessionDigestResult`/`SessionSummaryWrite`로 종료 digest 입력·fallback 결과·DB write 인자 경계를 명시해 LLM worker 후보가 raw text, evaluator-only turn, 평가 payload, CCD, deterministic carry를 압축 prompt에 우회 주입하지 못하게 했다. 9차에서는 `DigestQualityAssessment`/`SessionDigestWorkerOutcome`로 local quality harness를 추가해 빈/짧은 digest, raw forbidden substring, 내부 평가·CCD·상태 marker, 잘못된 `S{session_no}:` prefix를 fallback 유지 대상으로 판정한다. 10차에서는 `session_digest_worker.py`가 `CompressionJob`을 Node-compatible `GenerateRequest`/`EngineMessage`로 변환하고, 주입형 engine/audit 호출 뒤 quality gate 통과 결과만 `session_summary.digest/compressed_by/token_count`와 `case_profile.case_digest`에 적용하는 one-shot 경계를 소유한다. 11차에서는 `scripts/run-session-digest-worker.py` dry-run/apply runner와 `SESSION_DIGEST_WORKER_ENABLED=false` 기본값의 세션 종료 background scheduler 골격을 붙였다. scheduler는 DB load/apply 때만 connection을 잡고 engine 호출은 transaction 밖에서 수행하며, `compressed_by IS NULL` loader/apply CAS로 재실행 race를 막는다. DB loader는 persisted fallback summary와 client-visible `text_masked` transcript만 재구성하며 raw `text`, evaluator-only turn, CCD, end_state는 prompt에 넣지 않는다. `digest_pending`은 CompressionJob 생성 여부를 알리는 비동기 압축 필요 신호로 유지한다. 같은 값 재확인은 history를 늘리지 않고, `locked` fact는 건드리지 않는다. 관계갈등·위기·임상 추론은 자동 pinning/모순 처리에서 제외한다. | 관계·임상 fact 승격 기준, 실 provider 장시간 운영, 임상 골든셋 품질평가, 재압축, 접수면접→다회기 연속성·자기개념 진화 실증. | | **M3** | SSO claim 매핑·식별자 안정성 1차 완료·운영 IdP 감사 미연결 | Google/SAML/dev-login이 `AUTH_EMAIL_COHORT_MAP`·`AUTH_DOMAIN_COHORT_MAP` 및 SAML cohort claim을 `cohort_ids`로 전달하고, `app_user.external_id`는 provider subject(`google:`/`saml:`/`dev:`) 기반으로 저장한다. 운영 SAML 서명검증, 기관 claim schema/test tenant, deprovisioning audit은 아직 없다. | 한신 IdP 확정 후 SAML 서명검증, claim→role/cohort/institution_user_id 매핑 표 실연동, role변경/삭제 audit, deprovisioning evidence. | -| **X1** | 재귀학습·데이터셋 export 파이프라인 1차 구현 | `scripts/export-recursive-dataset.py`와 `app.services.dataset_export`로 masked-text JSONL dry-run, PII scan, kappa/ICC 계산, approved export 게이트를 구현했다. 기본은 `technical_dry_run`이며 실제 승인 export·골든셋 승격은 데이터 steward/legal review와 IAA 통과가 필요하다. | 파일럿 evidence에서 reviewer disposition, steward/legal 승인, gold annotation 라운드 적재 후 `approved_for_recursive_learning_seed` 승격 검증. | +| **X1** | 재귀학습·데이터셋 export 파이프라인 2차 구현 | `scripts/export-recursive-dataset.py`와 `app.services.dataset_export`로 masked-text JSONL dry-run, PII scan, kappa/ICC 계산, approved export 게이트를 구현했다. exporter는 consent가 남아 있고 client-visible인 `text_masked` 턴만 고르며, supervisor comment raw text를 JSONL에 넣지 않는다. `scripts/check-phase3-artifacts.py`는 approved와 dry-run dataset JSONL의 required keys, row count, privacy, PII shape를 검사해 `{}` 한 줄 같은 false-positive를 막는다. 기본은 `technical_dry_run`이며 실제 승인 export·골든셋 승격은 데이터 steward/legal review와 IAA 통과가 필요하다. | 파일럿 evidence에서 reviewer disposition, steward/legal 승인, gold annotation 라운드 적재, withdrawal/consent roster 대조, `min_completed_sessions` 정책 반영 후 `approved_for_recursive_learning_seed` 승격 검증. | | **X2** | AI API 비용 관측·예산 경고·평가 저비용 라우팅·evaluator cache 관측·일별 비용 추이·모델별 비용 검증 리포트 2차 완료 | 턴별 provider/model/tokens/cost 저장 경로와 `GET /admin/usage`, 관리자 비용 대시보드를 연결했다. `ADMIN_USAGE_BUDGET_USD` 기준 예산 상태(ok/warn/exceeded)도 응답/UI에 표시한다. `EVALUATOR_FAST_MODEL`/`EVALUATOR_DEEP_MODEL` 설정 시 fast/deep 평가 호출만 해당 모델 override로 gateway에 전달하고, 비워두면 기존 gateway default 라우팅을 유지한다. fast/deep evaluator structured 결과는 canonical request SHA-256 기반 인메모리 semantic cache로 재사용하며, 원문 prompt·completion은 저장하지 않고 성공 파싱 결과만 TTL/entry 제한 안에서 캐시한다. `/admin/usage`와 관리자 비용 카드가 cache enabled/entries/hits/misses/stores/evictions/requests/hit_rate와 일별 `daily_cost` 추이를 노출한다. `app.services.usage_report`와 `scripts/report-ai-usage.py`는 같은 usage JSON에서 provider/model별 cost share, token share, cost/turn, cost/1k tokens, metered coverage, budget/cache warning을 산출한다. DB 미가용 dev는 runtime store fallback, prod는 fail-closed다. | 자동 차단·한도 enforcement 정책. | | **L1** | 기술스택 신청서-구현 불일치 및 단기일정 산출물 압박 | doc4 신청서 스택(Spring Boot 3/Node.js·TimescaleDB) vs 실제 FastAPI/Python 불일치, 20주 단기일정·9월 저작권 등재 압박. 소유자 결정으로 장기 교체 대상은 Node.js 우선, 현재 FastAPI 전면 재작성은 보류했다. 내부 전환 증거로 engine gateway 공유 계약, `EngineClient.stream_packets()` decode 경계, schema-backed golden fixture(`engine_gateway_contract.v1.json`/`engine_gateway_schema.v1.json`), Python import 없는 `scripts/check-engine-gateway-contract.mjs` Node.js conformance runner, `gateway-default` default-routing sentinel 정규화, `structured_payload_from_response()` 기반 structured/legacy JSON response parser, 브라우저-facing 세션 read-model 분리(`app/session_read_model.py`), 페르소나 DTO/mapper 분리(`app/persona_read_model.py`), 페르소나 draft generation 계약(`app/persona_generation_contract.py`), session evaluation write packet(`SessionEvaluationWrite`), stage 라벨/phase-key 정규화 SSOT(`app/stage_contract.py`)까지 고정했다. | 신청서/저작권 등재 문서에 FastAPI 유지 사유와 계약 우선 Node 전환 계획을 반영하는 외부 거버넌스 증거. 내부 후보였던 H3 항목형 목록 저작 UI와 프롬프트 미리보기 de-JSON은 10차에서 완료. | @@ -86,12 +86,12 @@ ### A. 즉시 착수 가능 (내부 코드, 외부 합의 불요) - **C1 4차 완료 + E2E 고정**: `caseWorksheet` 응답 구조, 리뷰 화면 편집 UI, `app.case_worksheet` 저장/재조회 경로, 외부 루브릭 schema/loader/validation scaffold, 교수자 수동 워크시트 검수 상태 저장/표시를 구현했고, 실제 브라우저/API/DB E2E가 learner-save→teacher-review→수정요청 재조회를 검증한다. 워크시트 템플릿 key source는 `CASE_WORKSHEET_SECTION_SPECS`/`case_worksheet_template_item_keys()`로 명시했다. 후속은 임상팀 확정 루브릭 콘텐츠, AI 채점 보정, 승인 후 잠금·재제출 정책. -- **C2 1차 완료 + E2E 고정**: 위기 신호는 엔진 전 중단, 109 안내, `safety_events` 적재, 교수자 알림 큐, `ideation_observed` 전달까지 배선했다. 실제 브라우저/API/DB E2E가 crisis stream→`app.safety_events`→교수자 안전 큐 109를 검증한다. 후속은 임상 스크립트·서약 문안·감점 루브릭. -- **C3 2차 완료**: `theory_mode`가 세션·평가·생성 프롬프트까지 흐르고, 학습자는 세션 시작 전 기존 3개 모드 중 하나를 명시 선택해 `POST /sessions`로 보낸다. 후속은 임상팀 CBT 체인·이론부합 루브릭. -- **H2 1차+라이브 코칭+리뷰 3열 workbench 완료**: `make_eval_hook`과 stream 평가가 턴 파이프라인에 붙고 정규화 테이블로 적재·복원된다. 대안발화도 `app.alternative_utterance`로 정규화한다. 라이브 코칭은 0615 워크북·DSM·공식 지침 요약/RAG 근거로 코칭 아바타 말풍선·근거 모달·발화별 이력 오버레이까지 연결했다. `POST /kb/live-coach/source-packs/sync`는 공용 source pack sync service를 통해 evaluator 전용 RAG에 증분 색인하고, hash 변경 시 document version을 최신+1로 올린다. 관리자 sync/CLI는 source pack manifest 생성 전 process-local cache를 refresh해 현재 repo JSON 기준으로 비교한다. 학습자 `SessionReview` 데스크톱은 요약/흐름, 축어록, 평가 rail의 3열 workbench와 하단 워크시트로 재배치했고, 교수자/모바일 레이아웃은 기존 규칙을 유지한다. 후속은 골든셋·임상팀 루브릭·source pack 임상 검수 상태 운영. +- **C2 1차 완료 + E2E 고정**: 위기 신호는 엔진 전 중단, 109 안내, `safety_events` 적재, 교수자 알림 큐, `ideation_observed` 전달까지 배선했다. 실제 브라우저/API/DB E2E가 crisis stream→`app.safety_events`→교수자 안전 큐 109를 검증한다. 추가로 같은 위기 발화가 교수자 검토용 learner turn으로 남고 client AI reply turn은 저장되지 않는지 고정했다. 후속은 임상 스크립트·서약 문안·감점 루브릭. +- **C3 2차 완료 + E2E 고정**: `theory_mode`가 세션·평가·생성 프롬프트까지 흐르고, 학습자는 세션 시작 전 기존 3개 모드 중 하나를 명시 선택해 `POST /sessions`로 보낸다. 실제 브라우저/API/DB E2E는 `CBT` 선택값이 세션 상세에 보존되는지 확인한다. 후속은 임상팀 CBT 체인·이론부합 루브릭. +- **H2 1차+라이브 코칭+리뷰 3열 workbench 완료**: `make_eval_hook`과 stream 평가가 턴 파이프라인에 붙고 정규화 테이블로 적재·복원된다. fast-loop 실패는 `evaluation.error`와 리뷰 노트로 드러내고 성장 점수/최근 피드백에서는 제외한다. 교수자 단일 턴 재평가는 normalized evaluation row를 교체 저장해 `/review`에 hydrate된다. 대안발화도 `app.alternative_utterance`로 정규화한다. 라이브 코칭은 0615 워크북·DSM·공식 지침 요약/RAG 근거로 코칭 아바타 말풍선·근거 모달·발화별 이력 오버레이까지 연결했다. `GET /live-coach` 실패와 runtime persistence source는 빈 정상 이력으로 숨기지 않고 alert/임시 저장 경고로 표시한다. `POST /kb/live-coach/source-packs/sync`는 공용 source pack sync service를 통해 evaluator 전용 RAG에 증분 색인하고, hash 변경 시 document version을 최신+1로 올린다. 관리자 sync/CLI는 source pack manifest 생성 전 process-local cache를 refresh해 현재 repo JSON 기준으로 비교한다. RAG 회수 근거의 source metadata 보존과 actual sync/source-scoped retrieval은 DB-backed E2E로 고정했다. 학습자 `SessionReview` 데스크톱은 요약/흐름, 축어록, 평가 rail의 3열 workbench와 하단 워크시트로 재배치했고, 교수자/모바일 레이아웃은 기존 규칙을 유지한다. 후속은 골든셋·임상팀 루브릭·source pack 임상 검수 상태 운영. - **H3 10차 완료**: 페르소나 저작 CRUD(draft→review), P4~P7 저장소 JSON 로드, RAG source 기반 draft generation, prompt bundle id/version/hash provenance와 seed/version materializer runner가 붙었다. seed runner는 dry-run/JSON manifest를 제공하고, `--apply`에서만 DB pool을 초기화해 기존 materializer를 호출한다. repo-managed source pack runner는 DB-backed dry-run/apply를 제공하며 active `content_hash`가 바뀐 문서만 최신 version+1로 sync한다. `kb.raw_source_artifact` hash-only 레코드와 `rag.index_document()` raw/sensitivity=3 fail-closed guard를 추가해 raw 원문이 `kb.chunk`/embedding/FTS로 들어가지 않게 했고, `app/persona_read_model.py`로 persona DTO/mapper 경계를 분리했다. PersonaStudio의 자동사고·회기 시나리오·말투 목록·역린/금기 목록은 행 추가/삭제 UI로 바꾸고 기존 배열 payload 계약을 유지한다. 이번 패스에서 프롬프트 미리보기 raw JSON textarea를 라벨형 검토 섹션으로 교체했다. 후속은 루브릭·이론 콘텐츠 외부화, 임상팀 최종 검수 evidence, 암호화 blob/vault 기반 원문 실저장. -- **H3 11차 / L1 계약 보강 완료**: `app/persona_generation_contract.py`가 페르소나 draft structured schema, prompt bundle id/version/hash, `GenerateResponse` payload extraction, generated draft defaults/coercion을 소유한다. `routes/personas.py`는 auth, RAG evidence, engine invocation, provenance assembly, HTTP error mapping을 유지한다. 이어서 evaluator/live-coach의 로컬 structured payload alias를 제거하고 `structured_payload_from_response()`를 직접 호출하게 했으며, `SessionEvaluationWrite`가 `app.session_evaluation` 저장 packet을 소유한다. 최신 L1 검증은 persona contract+review 39 passed, gateway contract 27 passed, backend focused 120 passed, Node conformance OK, `npm run check:api-types`, `npm run typecheck`. -- **H4(부분)·X1·X2**: 한국어 날짜/금액/주소 및 이름/기관 로컬 휴리스틱 마스킹, 16-case 합성 fixture 평가 harness와 input/report schema(summary-only `technical_dry_run` report), 자연 발화형 이름 라벨·자기소개 마스킹 보강, optional ko recognizer adapter 인터페이스와 fake/failure 회귀, 외부 LLM 호출 metadata-only `audit.llm_call_log` 적재 경로, 학습자 동의 수락/철회/회기 시작 하드게이트 골격은 완료했다. 실제 ko recognizer 모델/provider 선정, 운영 말뭉치 기반 평가, guardian/legal 서명 evidence는 후속이다. dry-run JSONL export·PII scan·IAA 계산 1차도 완료했고 `ds.*` write는 `--write-dataset` 명시 시에만 수행한다. X2 비용 관측·예산 경고, evaluator fast/deep 모델 override, evaluator semantic cache, 운영 hit-rate 관측, 일별 비용 추이, 모델별 비용 검증 리포트는 완료했고 자동 차단·한도 enforcement 정책은 후속. M1은 provider_events 보존 슬롯, 내부 taxonomy, 인증 리뷰용 제한 파생 칩까지 완료했고, M2는 보수적 identity/agreement pinned_fact 자동 실적재, append-only history, 명시적 상담 약속 철회 contradiction, episodic embedding writer, 다음 턴 EngineMessage 주입 회귀, digest contract/local quality harness, one-shot worker/runner/default-off scheduler/CAS 경계까지 완료했다. L1은 Node.js conformance runner, `gateway-default` default-routing sentinel 정규화, session/persona read-model 분리까지 완료했다. 실제 provider 기반 한숨·울음 감지는 후속. +- **H3 11차 / L1 계약 보강 완료**: `app/persona_generation_contract.py`가 페르소나 draft structured schema, prompt bundle id/version/hash, `GenerateResponse` payload extraction, generated draft defaults/coercion을 소유한다. `routes/personas.py`는 auth, RAG evidence, engine invocation, provenance assembly, HTTP error mapping을 유지한다. 이어서 evaluator/live-coach의 로컬 structured payload alias를 제거하고 `structured_payload_from_response()`를 직접 호출하게 했으며, `SessionEvaluationWrite`가 `app.session_evaluation` 저장 packet을 소유한다. 회기 종료 자동 평가와 교수자 수동 재평가의 masked turn `seq`도 `session_evaluation_input.enriched_masked_turns()`로 1-based 규칙을 공유한다. 최신 L1 검증은 persona contract+review 39 passed, gateway contract 27 passed, backend focused 120 passed, Node conformance OK, `npm run check:api-types`, `npm run typecheck`. +- **H4(부분)·X1·X2**: 한국어 날짜/금액/주소 및 이름/기관 로컬 휴리스틱 마스킹, 16-case 합성 fixture 평가 harness와 input/report schema(summary-only `technical_dry_run` report), 자연 발화형 이름 라벨·자기소개 마스킹 보강, optional ko recognizer adapter 인터페이스와 fake/failure 회귀, 외부 LLM 호출 metadata-only `audit.llm_call_log` 적재 경로, live coach recent/evaluation prompt 재마스킹, turn-evaluation persistence rationale/comment/alternative scrub, legacy DB row raw `text` fallback 재마스킹, session-end deep evaluation payload/read-model scrub, 실제 브라우저 stream→DB detail/review 한국어 PII 마스킹, 학습자 동의 수락/철회/회기 시작 하드게이트와 철회 후 기존 voice `session_id` 차단은 완료했다. 실제 ko recognizer 모델/provider 선정, 운영 말뭉치 기반 평가, guardian/legal 서명 evidence는 후속이다. X1 dry-run JSONL export·PII scan·IAA 계산, consent/client-visible 선별, supervisor comment raw 제거, approved/dry-run JSONL shape gate도 완료했고 `ds.*` write는 `--write-dataset` 명시 시에만 수행한다. X2 비용 관측·예산 경고, evaluator fast/deep 모델 override, evaluator semantic cache, 운영 hit-rate 관측, 일별 비용 추이, 모델별 비용 검증 리포트는 완료했고 자동 차단·한도 enforcement 정책은 후속. M1은 provider_events 보존 슬롯, 내부 taxonomy, 인증 리뷰용 제한 파생 칩까지 완료했고, M2는 보수적 identity/agreement pinned_fact 자동 실적재, append-only history, 명시적 상담 약속 철회 contradiction, episodic embedding writer, 다음 턴 EngineMessage 주입 회귀, digest contract/local quality harness, one-shot worker/runner/default-off scheduler/CAS 경계까지 완료했다. L1은 Node.js conformance runner, `gateway-default` default-routing sentinel 정규화, session/persona read-model 분리까지 완료했다. 실제 provider 기반 한숨·울음 감지는 후속. ### B. 소유자 결정 / 외부(임상팀·기관) 의존 @@ -155,7 +155,7 @@ npm run e2e # Playwright — web+api+DB 스택 필요 - OCR 잔재·오탈자 존재(doc1·doc5). 의미는 시각 보정했으나 일부 표현 불확실. - **doc4 κ/ICC·환각률은 신청서 본문 미명시** — 계약 확정 지표로 단정 금지. - doc4/doc3 행정 불일치(참여교수 1명 vs 2명, 서식 연도 '2025' 오기, 연구책임자 표기 불일치, 트웬티온스 성명 공란). -- '현재상태'는 grep/코드 사실 기반 적대적 비평 인용에서 시작했으나, **C1·C2·C3·H1 pre/post 원장·KPI export·metric status contract·H2·H3·M1 provider_events 보존/taxonomy/인증 리뷰 파생 칩·M2 case memory/pinned_fact/history/명시철회 contradiction/episodic embedding writer/다음턴 주입회귀/digest contract/local quality harness/one-shot worker+runner+default-off scheduler boundary·M3, H4 로컬 마스킹/optional ko adapter/fixture 평가/감사/온보딩 경로, X1 dry-run export, X2 비용 관측·evaluator routing/cache/hit-rate 관측·모델별 비용 리포트, L1 engine gateway contract/golden/schema/Node conformance runner/default-routing sentinel/structured payload parser/session read-model/persona read-model/persona generation contract/session evaluation write packet 분리는 구현 후 재검증 완료** 기준이다. H1은 파일럿 evidence 계산과 report 계약까지이며, 공식 효과성 판정·통계해석·20명 evidence는 완료로 보지 않는다. M2는 fallback digest, `digest_pending` 응답 계약, LLM 후보 local quality harness, accepted-only one-shot worker/runner/default-off scheduler 경계까지이며, 실 provider 장시간 운영·임상 골든셋 품질평가·재압축은 완료로 보지 않는다. 다만 H4의 실제 ko recognizer 모델/provider 선정과 운영 corpus 평가, guardian/legal evidence, 공개 OAuth `/turn` proof, M1 실제 provider 기반 한숨·울음 감지는 여전히 게이트로 남아 있다. L1 외부 거버넌스는 FastAPI 유지 사유와 Node 전환 계획의 제출/등재 증거가 남아 있다. +- '현재상태'는 grep/코드 사실 기반 적대적 비평 인용에서 시작했으나, **C1·C2·C3·H1 pre/post 원장·KPI export·metric status contract·H2·H3·M1 provider_events 보존/taxonomy/인증 리뷰 파생 칩·M2 case memory/pinned_fact/history/명시철회 contradiction/episodic embedding writer/다음턴 주입회귀/digest contract/local quality harness/one-shot worker+runner+default-off scheduler boundary·M3, H4 로컬 마스킹/optional ko adapter/fixture 평가/감사/온보딩 경로, X1 dry-run export 선별/JSONL shape gate, X2 비용 관측·evaluator routing/cache/hit-rate 관측·모델별 비용 리포트, L1 engine gateway contract/golden/schema/Node conformance runner/default-routing sentinel/structured payload parser/session read-model/persona read-model/persona generation contract/session evaluation write packet 분리는 구현 후 재검증 완료** 기준이다. H1은 파일럿 evidence 계산과 report 계약까지이며, 공식 효과성 판정·통계해석·20명 evidence는 완료로 보지 않는다. M2는 fallback digest, `digest_pending` 응답 계약, LLM 후보 local quality harness, accepted-only one-shot worker/runner/default-off scheduler 경계까지이며, 실 provider 장시간 운영·임상 골든셋 품질평가·재압축은 완료로 보지 않는다. 다만 H4의 실제 ko recognizer 모델/provider 선정과 운영 corpus 평가, guardian/legal evidence, 공개 OAuth `/turn` proof, M1 실제 provider 기반 한숨·울음 감지는 여전히 게이트로 남아 있다. L1 외부 거버넌스는 FastAPI 유지 사유와 Node 전환 계획의 제출/등재 증거가 남아 있다. --- diff --git a/docs/guides/testing.md b/docs/guides/testing.md index d76f1a8..1f4ef3e 100644 --- a/docs/guides/testing.md +++ b/docs/guides/testing.md @@ -15,12 +15,12 @@ Vignette 저장소의 모든 검증 수단(백엔드 단위 테스트, 웹 타 | 검증 | 작업 디렉터리 | 명령 | DB | API(8000) | 웹(5173) | 엔진GW(9099) | 브라우저 | 현재 통과 | |---|---|---|---|---|---|---|---|---| -| 백엔드 단위 테스트 | `apps/api` | `python -m pytest app/ -q` | 불필요 | 불필요 | 불필요 | 불필요 | 불필요 | 현재 309 pass | +| 백엔드 단위 테스트 | `apps/api` | `python -m pytest app/ -q` | 불필요 | 불필요 | 불필요 | 불필요 | 불필요 | 현재 collect-only 349 tests, 최신 focused pass | | 엔진 게이트웨이 테스트 | `apps/api` | `python -m pytest engine_gateway/ -q` | 불필요 | 불필요 | 불필요 | 불필요 | 불필요 | 현재 27 pass | | API 타입 생성 체크 | `apps/web` | `npm run check:api-types` | 불필요 | 불필요 | 불필요 | 불필요 | 불필요 | pass | | 웹 타입체크 | `apps/web` | `npm run typecheck` | 불필요 | 불필요 | 불필요 | 불필요 | 불필요 | pass | | 웹 빌드 | `apps/web` | `npm run build` | 불필요 | 불필요 | 불필요 | 불필요 | 불필요 | pass | -| Playwright E2E(전체) | `apps/web` | `npm run e2e` | **필요(+시드)** | **필요** | 자동기동 | 일부만 | **필요** | 166 tests / 18 files | +| Playwright E2E(전체) | `apps/web` | `npm run e2e` | **필요(+시드)** | **필요** | 자동기동 | 일부만 | **필요** | 194 tests / 19 files | 핵심 원칙: **단위 테스트(pytest)와 타입체크/빌드는 외부 서비스 없이 단독 실행된다.** **E2E만 풀스택(DB+API+웹+브라우저)을 요구한다.** 아래 각 절에서 근거와 절차를 설명한다. @@ -42,14 +42,14 @@ Vignette 저장소의 모든 검증 수단(백엔드 단위 테스트, 웹 타 ```sh # apps/api -python -m pytest app/ -q # 앱 단위 테스트 현재 309 pass +python -m pytest app/ -q # 앱 단위 테스트: 현재 collect-only 349 tests, 최신 focused pass python -m pytest engine_gateway/ -q # 게이트웨이 단위 테스트 현재 27 pass ``` 수집만 빠르게 확인하려면: ```sh -python -m pytest app/ --collect-only -q # 현재: "305 tests collected" +python -m pytest app/ --collect-only -q # 현재: "349 tests collected" python -m pytest engine_gateway/ --collect-only -q # 현재: "27 tests collected" ``` @@ -154,9 +154,11 @@ GitHub Actions `.github/workflows/api-contract.yml`은 API/Web 계약 관련 파 ### 3.1 의존성 — 왜 풀스택인가 -E2E는 Playwright route fixture로 앱 데이터를 대체하지 않고, **Vite `/api` 프록시를 통해 실제 -로컬 FastAPI를 호출한다**(`apps/web/e2e/README.md`). 특히 공용 헬퍼 -`apps/web/e2e/support.ts`의 `fetchAvailablePersonas()`는 +Playwright suite에는 **Vite `/api` 프록시를 통해 실제 로컬 FastAPI/DB를 호출하는 full-stack +E2E**와, 특정 UI/error 상태를 고정하는 **route fixture UI 회귀 테스트**가 함께 있다 +(`apps/web/e2e/README.md`). DB-backed/real-API 증거는 해당 spec이 검증 대상 endpoint를 +`page.route().fulfill()`로 대체하지 않고 실제 API 응답 또는 persisted read-model을 확인한 +경우로 한정한다. 특히 공용 헬퍼 `apps/web/e2e/support.ts`의 `fetchAvailablePersonas()`는 `source === "database" && !degraded` 페르소나만 사용 가능으로 간주한다. ```ts @@ -258,17 +260,43 @@ VITE_API_BASE=http://127.0.0.1:8000 npm run e2e # 프록시 대신 API ### 3.6 실측 테스트 개수 (현재) -`npx playwright test --list` 기준 **총 166 tests / 18 files**. +`npx playwright test --list` 기준 **총 194 tests / 19 files**. -- **병렬 시나리오**: 130 tests (desktop 65 + mobile 65) -- **`@single-run` 직렬 시나리오**: 36 tests (DB 영속화·세션 MVP·음성 성공경로·회기말 평가 저장·교수자 사용자별 분석 등) +- **병렬 시나리오**: 150 tests (desktop 75 + mobile 75) +- **`@single-run` 직렬 시나리오**: 44 tests (DB 영속화·세션 MVP·음성 성공경로·회기말 평가 저장·교수자 명시 재평가 저장·교수자 턴 재평가 저장·교수자 UI 평가 재시도·교수자 사용자별 분석·source pack sync·이론모드 저장·동의 철회 후 voice 차단·브라우저 stream PII 마스킹·음성 transcript 저장 실패 UI 표면화 등) - `e2e/voice-success.spec.ts`는 직접 `/voice/ws` 캐스케이드와 Session 마이크 UI를 함께 검증하며, 브라우저 `