세션 평가·라이브코치·교수자 분석 라운드 마감 + 문서 정리 + 코드품질 리팩터
- 누적 작업트리 커밋: 회기 평가 복구·durable 저장, 라이브 코치 이력/근거, 교수자 학생분석, 음성 비언어 메타, PII 마스킹, 운영 티켓/헬스 등 - 문서: 완료 기록 docs/archive/ 냉동 보관, docs/ 단일 인덱스(docs/README.md)+통합 TODO(docs/TODO.md)로 정리 - 리팩터(행위 보존): Stage enum SSOT(taxonomy 소유·state_machine re-export), store recent/masked_turns 중복 제거, speaker_ko_label 단일 헬퍼, _list_sessions N+1 제거(state/turns 배치 + 턴평가 하이드레이션 배치) - 검증: 백엔드 pytest 352 passed, _list_sessions E2E chromium-single-run 2 passed
This commit is contained in:
parent
7c41c3ce79
commit
778e8526d4
108 changed files with 6457 additions and 455 deletions
|
|
@ -108,6 +108,10 @@ class Settings(BaseSettings):
|
|||
default=0.0,
|
||||
validation_alias="SESSION_EVALUATION_TIMEOUT",
|
||||
)
|
||||
session_evaluation_recovery_limit: int = Field(
|
||||
default=5,
|
||||
validation_alias="SESSION_EVALUATION_RECOVERY_LIMIT",
|
||||
)
|
||||
session_digest_worker_enabled: bool = Field(
|
||||
default=False,
|
||||
validation_alias="SESSION_DIGEST_WORKER_ENABLED",
|
||||
|
|
|
|||
|
|
@ -152,7 +152,7 @@ def require_admin_access():
|
|||
async def _checker(
|
||||
principal: Annotated[Principal, Depends(get_current_principal)],
|
||||
) -> Principal:
|
||||
if not principal.admin_access:
|
||||
if not principal.can_access_role(Role.ADMIN):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_403_FORBIDDEN,
|
||||
detail="admin access required",
|
||||
|
|
|
|||
|
|
@ -58,9 +58,11 @@ async def lifespan(app: FastAPI):
|
|||
)
|
||||
await engine_client.startup()
|
||||
await voice_service.startup()
|
||||
session_routes.schedule_missing_session_evaluation_recovery()
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
session_routes.cancel_missing_session_evaluation_recovery()
|
||||
await voice_service.shutdown()
|
||||
await engine_client.shutdown()
|
||||
try:
|
||||
|
|
|
|||
|
|
@ -27,6 +27,7 @@ from .. import session_persistence
|
|||
from ..deps import Principal, Role, require_role
|
||||
from ..engine_client import EngineError, engine_client
|
||||
from ..runtime_policy import runtime_fallback_allowed
|
||||
from ..session_evaluation_input import enriched_masked_turns
|
||||
from ..session_read_model import StageLabel, stage_label_or_none
|
||||
from ..services import evaluator
|
||||
from ..services.evaluator import SessionEvaluation, TurnEvaluation
|
||||
|
|
@ -122,13 +123,7 @@ async def reevaluate_session(
|
|||
엔진 장애는 503 으로 변환(평가는 비치명적이지만 트리거는 사용자 명시 요청이라 에러 노출).
|
||||
"""
|
||||
sess = await _load_session_or_404(session_id, principal)
|
||||
masked = sess.masked_turns()
|
||||
# 발화 seq 보강(deep 프롬프트 가독성 — store 가 seq 미포함이라 인덱스로 부여)
|
||||
enriched: list[dict[str, Any]] = []
|
||||
for i, t in enumerate(masked):
|
||||
item = dict(t)
|
||||
item.setdefault("seq", i)
|
||||
enriched.append(item)
|
||||
enriched = enriched_masked_turns(sess.masked_turns())
|
||||
|
||||
# 누적 기법 코드 — DB 미가용이라 fast 결과가 없으면 빈 분포(deep LLM 정성 평가는 그대로 유효).
|
||||
technique_codes: list[str] = []
|
||||
|
|
@ -233,8 +228,25 @@ async def reevaluate_turn(
|
|||
engine=engine_client,
|
||||
audit_hook=session_persistence.record_llm_call_audit,
|
||||
)
|
||||
if result.error and result.error.startswith("engine_error"):
|
||||
raise HTTPException(status.HTTP_503_SERVICE_UNAVAILABLE, detail=result.error)
|
||||
result_payload = result.to_hook_dict()
|
||||
turn_id = getattr(learner, "turn_id", None)
|
||||
if not turn_id:
|
||||
raise HTTPException(
|
||||
status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||
detail="turn evaluation retry result was generated but the target turn has no durable id",
|
||||
)
|
||||
saved = await session_persistence.replace_turn_evaluation(
|
||||
turn_id=turn_id,
|
||||
evaluation=result_payload,
|
||||
)
|
||||
if not saved:
|
||||
raise HTTPException(
|
||||
status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||
detail="turn evaluation retry result was generated but could not be saved",
|
||||
)
|
||||
learner.evaluation = result_payload
|
||||
if result.error:
|
||||
raise HTTPException(_session_evaluation_error_status(result.error), detail=result.error)
|
||||
return result
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -26,6 +26,7 @@ from ..deps import CurrentPrincipal, Principal, Role
|
|||
from ..engine_client import EngineError, engine_client
|
||||
from ..persona_repository import get_catalog_persona
|
||||
from ..runtime_policy import require_runtime_fallback_allowed
|
||||
from ..session_evaluation_input import enriched_masked_turns
|
||||
from ..services import (
|
||||
evaluator,
|
||||
guardrail,
|
||||
|
|
@ -42,6 +43,7 @@ from ..session_read_model import (
|
|||
LearnerSessionsResponse,
|
||||
LearnerSessionSummary,
|
||||
LEARNER_VISIBLE_AI_ROLE,
|
||||
MISSING_SESSION_EVALUATION_GRACE_SECONDS,
|
||||
ReviewCaseWorksheet,
|
||||
ReviewCaseWorksheetSaveRequest,
|
||||
ReviewWorksheetItem,
|
||||
|
|
@ -70,6 +72,8 @@ from ..store import InProcSession, TurnRecord, store
|
|||
|
||||
router = APIRouter(prefix="/sessions", tags=["sessions"])
|
||||
logger = logging.getLogger(__name__)
|
||||
_SESSION_EVALUATION_IN_FLIGHT: set[str] = set()
|
||||
_SESSION_EVALUATION_RECOVERY_TASK: asyncio.Task[int] | None = None
|
||||
|
||||
TheoryMode = Literal["humanistic", "cbt", "integrative"]
|
||||
EndStateValue = str | int | float | bool | None | dict[str, float]
|
||||
|
|
@ -102,7 +106,14 @@ class LiveCoachRequest(BaseModel):
|
|||
|
||||
class LiveCoachHistoryResponse(BaseModel):
|
||||
source: Literal["database", "runtime"] = "runtime"
|
||||
quota: live_coach.LiveCoachQuota = Field(
|
||||
default_factory=lambda: live_coach.LiveCoachQuota(
|
||||
remaining=session_persistence.LIVE_COACH_INITIAL_CREDITS,
|
||||
max=session_persistence.LIVE_COACH_MAX_CREDITS,
|
||||
)
|
||||
)
|
||||
events: list[live_coach.LiveCoachEvent] = Field(default_factory=list)
|
||||
credit_events: list[live_coach.LiveCoachCreditEvent] = Field(default_factory=list)
|
||||
|
||||
|
||||
class CrisisResourceResponse(BaseModel):
|
||||
|
|
@ -220,12 +231,20 @@ async def _retrieve_live_coach_grounding(
|
|||
body = chunk.body or chunk.behavior_cue or chunk.context_prefix or ""
|
||||
if not body:
|
||||
continue
|
||||
meta = chunk.meta if isinstance(chunk.meta, dict) else {}
|
||||
title = str(meta.get("source_title") or meta.get("title") or chunk.source_id or "Vignette KB").strip()
|
||||
source_type = str(meta.get("source_type") or "").strip()
|
||||
source_version = str(meta.get("source_version") or meta.get("version") or "").strip()
|
||||
citation = str(meta.get("citation") or "").strip()
|
||||
out.append(
|
||||
live_coach.LiveCoachGrounding(
|
||||
source_id=chunk.source_id or f"kb:{chunk.chunk_id}",
|
||||
title=chunk.source_id or "Vignette KB",
|
||||
title=title or "Vignette KB",
|
||||
locator=chunk.heading_path,
|
||||
kb_kind=chunk.kb_kind,
|
||||
source_type=source_type or None,
|
||||
version=source_version or None,
|
||||
citation=citation or None,
|
||||
summary=body[:500],
|
||||
)
|
||||
)
|
||||
|
|
@ -643,8 +662,13 @@ async def _evaluate_stream_turn(ctx: orchestrator.TurnContext, final_reply: str)
|
|||
audit_hook=session_persistence.record_llm_call_audit,
|
||||
)
|
||||
return await hook(ctx, final_reply)
|
||||
except Exception:
|
||||
return None
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"turn fast-loop evaluation failed: session_id=%s",
|
||||
ctx.session_id,
|
||||
exc_info=True,
|
||||
)
|
||||
return orchestrator.turn_evaluation_error_payload(ctx, exc)
|
||||
|
||||
|
||||
def _stream_result_from_done(
|
||||
|
|
@ -678,11 +702,7 @@ async def _generate_and_save_session_evaluation(sess: InProcSession) -> None:
|
|||
return
|
||||
|
||||
timeout_seconds = _session_evaluation_timeout_seconds()
|
||||
enriched: list[dict[str, object]] = []
|
||||
for index, turn in enumerate(sess.masked_turns(), start=1):
|
||||
item: dict[str, object] = dict(turn)
|
||||
item["seq"] = index
|
||||
enriched.append(item)
|
||||
enriched = enriched_masked_turns(sess.masked_turns())
|
||||
|
||||
try:
|
||||
result = await asyncio.wait_for(
|
||||
|
|
@ -753,6 +773,7 @@ async def _generate_and_save_session_evaluation(sess: InProcSession) -> None:
|
|||
|
||||
|
||||
def _observe_session_evaluation_task(task: asyncio.Task[None], session_id: str) -> None:
|
||||
_SESSION_EVALUATION_IN_FLIGHT.discard(session_id)
|
||||
try:
|
||||
task.result()
|
||||
except asyncio.CancelledError:
|
||||
|
|
@ -764,6 +785,13 @@ def _observe_session_evaluation_task(task: asyncio.Task[None], session_id: str)
|
|||
def _schedule_session_evaluation(sess: InProcSession) -> asyncio.Task[None] | None:
|
||||
if not sess.turns:
|
||||
return None
|
||||
if sess.session_id in _SESSION_EVALUATION_IN_FLIGHT:
|
||||
logger.info(
|
||||
"session evaluation already scheduled: session_id=%s",
|
||||
sess.session_id,
|
||||
)
|
||||
return None
|
||||
_SESSION_EVALUATION_IN_FLIGHT.add(sess.session_id)
|
||||
task = asyncio.create_task(
|
||||
_generate_and_save_session_evaluation(sess),
|
||||
name=f"session-evaluation:{sess.session_id}",
|
||||
|
|
@ -774,6 +802,66 @@ def _schedule_session_evaluation(sess: InProcSession) -> asyncio.Task[None] | No
|
|||
return task
|
||||
|
||||
|
||||
async def recover_missing_session_evaluations(*, limit: int | None = None) -> int:
|
||||
recovery_limit = settings.session_evaluation_recovery_limit if limit is None else limit
|
||||
if recovery_limit <= 0:
|
||||
return 0
|
||||
stale_after_seconds = (
|
||||
_session_evaluation_timeout_seconds()
|
||||
+ MISSING_SESSION_EVALUATION_GRACE_SECONDS
|
||||
)
|
||||
candidates, durable = await session_persistence.list_sessions_missing_session_evaluation(
|
||||
older_than_seconds=stale_after_seconds,
|
||||
limit=recovery_limit,
|
||||
)
|
||||
if not durable:
|
||||
logger.warning("session evaluation recovery skipped: durable store unavailable")
|
||||
return 0
|
||||
scheduled = 0
|
||||
for sess in candidates:
|
||||
if _schedule_session_evaluation(sess) is not None:
|
||||
scheduled += 1
|
||||
if scheduled:
|
||||
logger.info("session evaluation recovery scheduled %d session(s)", scheduled)
|
||||
return scheduled
|
||||
|
||||
|
||||
def _observe_session_evaluation_recovery_task(task: asyncio.Task[int]) -> None:
|
||||
global _SESSION_EVALUATION_RECOVERY_TASK
|
||||
if _SESSION_EVALUATION_RECOVERY_TASK is task:
|
||||
_SESSION_EVALUATION_RECOVERY_TASK = None
|
||||
try:
|
||||
task.result()
|
||||
except asyncio.CancelledError:
|
||||
logger.warning("session evaluation recovery task cancelled")
|
||||
except Exception:
|
||||
logger.exception("session evaluation recovery task crashed")
|
||||
|
||||
|
||||
def schedule_missing_session_evaluation_recovery() -> asyncio.Task[int] | None:
|
||||
global _SESSION_EVALUATION_RECOVERY_TASK
|
||||
if settings.session_evaluation_recovery_limit <= 0:
|
||||
return None
|
||||
if (
|
||||
_SESSION_EVALUATION_RECOVERY_TASK is not None
|
||||
and not _SESSION_EVALUATION_RECOVERY_TASK.done()
|
||||
):
|
||||
return _SESSION_EVALUATION_RECOVERY_TASK
|
||||
task = asyncio.create_task(
|
||||
recover_missing_session_evaluations(),
|
||||
name="session-evaluation-recovery",
|
||||
)
|
||||
_SESSION_EVALUATION_RECOVERY_TASK = task
|
||||
task.add_done_callback(_observe_session_evaluation_recovery_task)
|
||||
return task
|
||||
|
||||
|
||||
def cancel_missing_session_evaluation_recovery() -> None:
|
||||
task = _SESSION_EVALUATION_RECOVERY_TASK
|
||||
if task is not None and not task.done():
|
||||
task.cancel()
|
||||
|
||||
|
||||
def _session_evaluation_timeout_seconds() -> float:
|
||||
configured = float(settings.session_evaluation_timeout or settings.engine_timeout)
|
||||
return max(configured, 1.0)
|
||||
|
|
@ -792,12 +880,12 @@ async def _load_learner_sessions(
|
|||
include_turn_evaluation: bool = False,
|
||||
) -> tuple[list[InProcSession], bool]:
|
||||
if include_turn_evaluation:
|
||||
sessions, durable = await session_persistence.list_sessions(
|
||||
sessions, durable = await session_persistence.list_recent_sessions(
|
||||
principal,
|
||||
include_turn_evaluation=True,
|
||||
)
|
||||
else:
|
||||
sessions, durable = await session_persistence.list_sessions(principal)
|
||||
sessions, durable = await session_persistence.list_recent_sessions(principal)
|
||||
if not durable:
|
||||
require_runtime_fallback_allowed("session list")
|
||||
sessions = [
|
||||
|
|
@ -1254,9 +1342,16 @@ async def list_live_coach_history(
|
|||
principal = _ensure_learner(principal)
|
||||
await _load_session_or_404(session_id, principal)
|
||||
events, durable = await session_persistence.list_live_coach_events(session_id, principal)
|
||||
quota, quota_durable = await session_persistence.get_live_coach_quota(session_id, principal)
|
||||
credit_events, credit_durable = await session_persistence.list_live_coach_credit_events(
|
||||
session_id,
|
||||
principal,
|
||||
)
|
||||
return LiveCoachHistoryResponse(
|
||||
source="database" if durable else "runtime",
|
||||
source="database" if durable and quota_durable and credit_durable else "runtime",
|
||||
quota=live_coach.LiveCoachQuota(**quota),
|
||||
events=[live_coach.LiveCoachEvent(**event) for event in events],
|
||||
credit_events=[live_coach.LiveCoachCreditEvent(**event) for event in credit_events],
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -1269,6 +1364,12 @@ async def live_coach_turn(
|
|||
"""방금 완료된 턴에 대한 비차단 라이브 코칭을 반환한다."""
|
||||
principal = _ensure_learner(principal)
|
||||
sess = await _load_session_or_404(session_id, principal)
|
||||
quota, _ = await session_persistence.get_live_coach_quota(session_id, principal)
|
||||
if int(quota.get("remaining", 0)) <= 0:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT,
|
||||
detail="live coach credit exhausted",
|
||||
)
|
||||
turn_seq = body.turn_seq or max(1, int(getattr(sess.state, "turn_seq", 1) or 1))
|
||||
stage = _stage_label(sess.state.stage)
|
||||
grounding = await _retrieve_live_coach_grounding(
|
||||
|
|
@ -1296,16 +1397,35 @@ async def live_coach_turn(
|
|||
grounding=grounding,
|
||||
audit_hook=session_persistence.record_llm_call_audit,
|
||||
)
|
||||
await session_persistence.save_live_coach_event(
|
||||
session_id=sess.session_id,
|
||||
learner_id=sess.learner_id,
|
||||
turn_seq=turn_seq,
|
||||
stage=stage,
|
||||
learner_text=body.learner_text,
|
||||
client_reply=body.client_reply,
|
||||
suggestion=suggestion,
|
||||
try:
|
||||
_, coach_event_durable = await session_persistence.save_live_coach_event(
|
||||
session_id=sess.session_id,
|
||||
learner_id=sess.learner_id,
|
||||
turn_seq=turn_seq,
|
||||
stage=stage,
|
||||
learner_text=body.learner_text,
|
||||
client_reply=body.client_reply,
|
||||
suggestion=suggestion,
|
||||
)
|
||||
except session_persistence.LiveCoachCreditExhausted as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT,
|
||||
detail="live coach credit exhausted",
|
||||
) from exc
|
||||
quota_after, quota_durable = await session_persistence.get_live_coach_quota(session_id, principal)
|
||||
credit_events, credit_durable = await session_persistence.list_live_coach_credit_events(session_id, principal)
|
||||
turn_credit_events = [
|
||||
live_coach.LiveCoachCreditEvent(**event)
|
||||
for event in credit_events
|
||||
if int(event.get("turn_seq") or 0) == int(turn_seq)
|
||||
]
|
||||
return suggestion.model_copy(
|
||||
update={
|
||||
"persistence_source": "database" if coach_event_durable and quota_durable and credit_durable else "runtime",
|
||||
"quota": live_coach.LiveCoachQuota(**quota_after),
|
||||
"credit_events": turn_credit_events[-2:],
|
||||
}
|
||||
)
|
||||
return suggestion
|
||||
|
||||
|
||||
@router.post("/{session_id}/stream")
|
||||
|
|
@ -1398,6 +1518,7 @@ async def end_session(
|
|||
"""End a learner-owned session and prepare carry-over state."""
|
||||
principal = _ensure_learner(principal)
|
||||
sess = await _load_session_or_404(session_id, principal, allow_ended=True)
|
||||
was_ended = bool(sess.ended)
|
||||
|
||||
recall = _RECALL_CACHE.get(session_id) or memory.RecallContext()
|
||||
carry = memory.make_carry_over(
|
||||
|
|
@ -1413,7 +1534,8 @@ async def end_session(
|
|||
await _end_persisted_session(sess, carry)
|
||||
_RECALL_CACHE.pop(session_id, None)
|
||||
_KB_CUES_CACHE.pop(session_id, None)
|
||||
_schedule_session_evaluation(sess)
|
||||
if not was_ended:
|
||||
_schedule_session_evaluation(sess)
|
||||
|
||||
return SessionEndResponse(
|
||||
session_id=session_id,
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@
|
|||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Annotated, Literal
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, status
|
||||
|
|
@ -10,7 +11,12 @@ from pydantic import BaseModel, Field
|
|||
from .. import session_persistence
|
||||
from ..deps import Principal, Role, require_role
|
||||
from ..runtime_policy import require_runtime_fallback_allowed
|
||||
from ..session_read_model import StageLabel, stage_label
|
||||
from ..session_read_model import (
|
||||
StageLabel,
|
||||
learner_visible_turns,
|
||||
missing_session_evaluation_record,
|
||||
stage_label,
|
||||
)
|
||||
from ..services import session_metrics
|
||||
from ..stage_contract import STAGE_LABEL_VALUES
|
||||
from ..store import InProcSession, store
|
||||
|
|
@ -37,6 +43,10 @@ class TeacherSessionSummary(BaseModel):
|
|||
review_status: Literal["pending", "viewed", "closed"] = "pending"
|
||||
review_note: str | None = None
|
||||
reviewed_at: str | None = None
|
||||
evaluation_status: Literal["pending", "ready", "error"] = "pending"
|
||||
review_ready: bool = False
|
||||
supervisor_state: Literal["기록 대기", "평가 대기", "평가 완료", "평가 실패"] = "기록 대기"
|
||||
evaluation_error: str | None = None
|
||||
|
||||
|
||||
class TeacherSessionReviewStatusRequest(BaseModel):
|
||||
|
|
@ -137,9 +147,14 @@ class TeacherDashboardResponse(BaseModel):
|
|||
message: str
|
||||
|
||||
|
||||
def _learner_label(learner_id: str) -> str:
|
||||
def _learner_label_from_id(learner_id: str) -> str:
|
||||
suffix = learner_id[-6:] if len(learner_id) > 6 else learner_id
|
||||
return f"학습자 {suffix}"
|
||||
return f"학습자 {suffix}" if suffix else "학습자"
|
||||
|
||||
|
||||
def _session_learner_label(sess: InProcSession) -> str:
|
||||
label = str(getattr(sess, "learner_label", "") or "").strip()
|
||||
return label or _learner_label_from_id(sess.learner_id)
|
||||
|
||||
|
||||
def _growth_point(point: session_metrics.SessionGrowthPoint) -> TeacherGrowthPoint:
|
||||
|
|
@ -163,9 +178,15 @@ def _build_learner_growth(
|
|||
limit: int | None = 12,
|
||||
point_limit: int | None = 6,
|
||||
) -> list[TeacherLearnerGrowth]:
|
||||
learner_labels = {
|
||||
sess.learner_id: label
|
||||
for sess in sessions
|
||||
if (label := str(getattr(sess, "learner_label", "") or "").strip())
|
||||
}
|
||||
metrics = session_metrics.build_learner_growth(
|
||||
sessions,
|
||||
learner_label=_learner_label,
|
||||
learner_label=lambda learner_id: learner_labels.get(learner_id)
|
||||
or _learner_label_from_id(learner_id),
|
||||
limit=limit,
|
||||
point_limit=point_limit,
|
||||
)
|
||||
|
|
@ -225,22 +246,68 @@ def _worksheet_review_status_value(
|
|||
return "pending"
|
||||
|
||||
|
||||
def _evaluation_status_value(record: dict[str, object] | None) -> Literal["pending", "ready", "error"]:
|
||||
value = str((record or {}).get("status") or "pending")
|
||||
if value in {"ready", "error"}:
|
||||
return value # type: ignore[return-value]
|
||||
return "pending"
|
||||
|
||||
|
||||
def _supervisor_state(
|
||||
sess: InProcSession,
|
||||
evaluation_record: dict[str, object] | None,
|
||||
*,
|
||||
has_visible_turns: bool | None = None,
|
||||
) -> Literal["기록 대기", "평가 대기", "평가 완료", "평가 실패"]:
|
||||
status = _evaluation_status_value(evaluation_record)
|
||||
if status == "ready":
|
||||
return "평가 완료"
|
||||
if status == "error":
|
||||
return "평가 실패"
|
||||
if has_visible_turns is None:
|
||||
has_visible_turns = bool(learner_visible_turns(sess))
|
||||
if has_visible_turns:
|
||||
return "평가 대기"
|
||||
return "기록 대기"
|
||||
|
||||
|
||||
def _dashboard_evaluation_record(
|
||||
sess: InProcSession,
|
||||
evaluation_record: dict[str, object] | None,
|
||||
) -> dict[str, object] | None:
|
||||
visible_turns = learner_visible_turns(sess)
|
||||
hidden_turns = len(visible_turns) != len(sess.turns)
|
||||
if evaluation_record is not None:
|
||||
return None if hidden_turns else evaluation_record
|
||||
if hidden_turns:
|
||||
return None
|
||||
return missing_session_evaluation_record(
|
||||
sess,
|
||||
has_visible_turns=bool(visible_turns),
|
||||
now_ts=datetime.now().timestamp(),
|
||||
)
|
||||
|
||||
|
||||
def _summary(
|
||||
sess: InProcSession,
|
||||
review_status: dict[str, object] | None = None,
|
||||
evaluation_record: dict[str, object] | None = None,
|
||||
) -> TeacherSessionSummary:
|
||||
learner_turns = sum(1 for turn in sess.turns if turn.speaker == "counselor")
|
||||
client_turns = sum(1 for turn in sess.turns if turn.speaker == "client")
|
||||
visible_turns = learner_visible_turns(sess)
|
||||
summary_evaluation_record = _dashboard_evaluation_record(sess, evaluation_record)
|
||||
learner_turns = sum(1 for turn in visible_turns if turn.speaker == "counselor")
|
||||
client_turns = sum(1 for turn in visible_turns if turn.speaker == "client")
|
||||
evaluation_status = _evaluation_status_value(summary_evaluation_record)
|
||||
return TeacherSessionSummary(
|
||||
session_id=sess.session_id,
|
||||
learner_id=sess.learner_id,
|
||||
learner_label=_learner_label(sess.learner_id),
|
||||
learner_label=_session_learner_label(sess),
|
||||
persona_code=sess.persona_code,
|
||||
persona_name=sess.persona.display_name,
|
||||
session_no=sess.session_no,
|
||||
status="ended" if sess.ended else "active",
|
||||
stage=stage_label(sess.state.stage),
|
||||
turn_count=len(sess.turns),
|
||||
turn_count=len(visible_turns),
|
||||
learner_turn_count=learner_turns,
|
||||
client_turn_count=client_turns,
|
||||
started_at=session_metrics.iso_datetime(sess.created_at) or "",
|
||||
|
|
@ -248,34 +315,53 @@ def _summary(
|
|||
review_status=_review_status_value(review_status),
|
||||
review_note=str(review_status.get("note") or "") if review_status else None,
|
||||
reviewed_at=str(review_status.get("reviewed_at") or "") if review_status else None,
|
||||
evaluation_status=evaluation_status,
|
||||
review_ready=evaluation_status == "ready",
|
||||
supervisor_state=_supervisor_state(
|
||||
sess,
|
||||
summary_evaluation_record,
|
||||
has_visible_turns=bool(visible_turns),
|
||||
),
|
||||
evaluation_error=(
|
||||
str(summary_evaluation_record.get("error") or "") if summary_evaluation_record else None
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _needs_teacher_review(item: TeacherSessionSummary) -> bool:
|
||||
if item.status != "ended" or item.review_status == "closed":
|
||||
return False
|
||||
return item.turn_count > 0 or item.evaluation_status in {"ready", "error"}
|
||||
|
||||
|
||||
@router.get("/dashboard", response_model=TeacherDashboardResponse)
|
||||
async def teacher_dashboard(principal: TeacherPrincipal) -> TeacherDashboardResponse:
|
||||
"""Return teacher-visible dashboard data from real sessions only."""
|
||||
sessions, durable = await session_persistence.list_sessions(
|
||||
sessions, durable = await session_persistence.list_all_sessions(
|
||||
principal,
|
||||
include_turn_evaluation=True,
|
||||
)
|
||||
if not durable:
|
||||
require_runtime_fallback_allowed("teacher dashboard")
|
||||
sessions = sorted(store.list(), key=lambda sess: sess.created_at, reverse=True)
|
||||
ended_session_ids = [sess.session_id for sess in sessions if sess.ended]
|
||||
review_statuses, _ = await session_persistence.list_session_review_statuses(
|
||||
[sess.session_id for sess in sessions if sess.ended],
|
||||
ended_session_ids,
|
||||
principal,
|
||||
)
|
||||
evaluation_records, evaluations_durable = await session_persistence.list_session_evaluations(
|
||||
ended_session_ids,
|
||||
principal,
|
||||
)
|
||||
if not evaluations_durable:
|
||||
require_runtime_fallback_allowed("session evaluation list")
|
||||
summaries = [
|
||||
_summary(sess, review_statuses.get(sess.session_id))
|
||||
_summary(sess, review_statuses.get(sess.session_id), evaluation_records.get(sess.session_id))
|
||||
for sess in sessions
|
||||
]
|
||||
pending_reviews = [
|
||||
item
|
||||
for item in summaries
|
||||
if item.status == "ended" and item.review_status != "closed"
|
||||
]
|
||||
pending_reviews = [item for item in summaries if _needs_teacher_review(item)]
|
||||
learners = {sess.learner_id for sess in sessions}
|
||||
learner_growth = _build_learner_growth(sessions)
|
||||
learner_growth = _build_learner_growth(sessions, limit=None)
|
||||
safety_alerts: list[TeacherSafetyAlert] = []
|
||||
if durable:
|
||||
raw_alerts, alerts_durable = await session_persistence.list_safety_alerts(principal)
|
||||
|
|
@ -334,7 +420,7 @@ async def learner_analysis(
|
|||
principal: TeacherPrincipal,
|
||||
) -> TeacherLearnerAnalysisResponse:
|
||||
"""Return one learner's full teacher-visible session timeline and trend."""
|
||||
sessions, durable = await session_persistence.list_sessions(
|
||||
sessions, durable = await session_persistence.list_all_sessions(
|
||||
principal,
|
||||
include_turn_evaluation=True,
|
||||
)
|
||||
|
|
@ -350,20 +436,23 @@ async def learner_analysis(
|
|||
learner_sessions,
|
||||
key=lambda sess: (sess.created_at, sess.session_no, sess.session_id),
|
||||
)
|
||||
ended_session_ids = [sess.session_id for sess in ordered if sess.ended]
|
||||
review_statuses, _ = await session_persistence.list_session_review_statuses(
|
||||
[sess.session_id for sess in ordered if sess.ended],
|
||||
ended_session_ids,
|
||||
principal,
|
||||
)
|
||||
evaluation_records, evaluations_durable = await session_persistence.list_session_evaluations(
|
||||
ended_session_ids,
|
||||
principal,
|
||||
)
|
||||
if not evaluations_durable:
|
||||
require_runtime_fallback_allowed("session evaluation list")
|
||||
summaries = [
|
||||
_summary(sess, review_statuses.get(sess.session_id))
|
||||
_summary(sess, review_statuses.get(sess.session_id), evaluation_records.get(sess.session_id))
|
||||
for sess in ordered
|
||||
]
|
||||
growth = _build_learner_growth(ordered, limit=None, point_limit=None)[0]
|
||||
pending_reviews = sum(
|
||||
1
|
||||
for item in summaries
|
||||
if item.status == "ended" and item.review_status != "closed"
|
||||
)
|
||||
pending_reviews = sum(1 for item in summaries if _needs_teacher_review(item))
|
||||
closed_reviews = sum(1 for item in summaries if item.review_status == "closed")
|
||||
|
||||
return TeacherLearnerAnalysisResponse(
|
||||
|
|
@ -404,6 +493,18 @@ async def update_session_review_status(
|
|||
status_code=status.HTTP_409_CONFLICT,
|
||||
detail="active sessions cannot be closed as reviewed",
|
||||
)
|
||||
if sess.ended and request.status == "closed" and learner_visible_turns(sess):
|
||||
evaluation_record, evaluation_durable = await session_persistence.load_session_evaluation(
|
||||
session_id,
|
||||
principal,
|
||||
)
|
||||
if not evaluation_durable:
|
||||
require_runtime_fallback_allowed("session evaluation")
|
||||
if _evaluation_status_value(evaluation_record) != "ready":
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT,
|
||||
detail="session evaluation must be ready before closing teacher review",
|
||||
)
|
||||
saved, _ = await session_persistence.save_session_review_status(
|
||||
session_id=session_id,
|
||||
reviewer_id=principal.user_id,
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ import hashlib
|
|||
import time
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import APIRouter, WebSocket, WebSocketDisconnect
|
||||
from fastapi import APIRouter, WebSocket, WebSocketDisconnect, HTTPException
|
||||
from fastapi.responses import JSONResponse
|
||||
from starlette.websockets import WebSocketState
|
||||
|
||||
|
|
@ -97,6 +97,13 @@ _PROVIDER_EVENT_TAXONOMY = {
|
|||
"noise": ("background_noise", "audio_quality"),
|
||||
"background_noise": ("background_noise", "audio_quality"),
|
||||
}
|
||||
|
||||
|
||||
def _is_turn_persistence_unavailable(exc: Exception) -> bool:
|
||||
if not isinstance(exc, HTTPException) or exc.status_code != 503:
|
||||
return False
|
||||
detail = str(exc.detail or "")
|
||||
return "turn append" in detail and "persistence unavailable" in detail
|
||||
_PROVIDER_EVENT_TYPE_FIELDS = ("event_type", "type", "kind", "label")
|
||||
|
||||
|
||||
|
|
@ -303,7 +310,18 @@ async def voice_ws(websocket: WebSocket) -> None:
|
|||
except WebSocketDisconnect:
|
||||
pass
|
||||
except Exception as e:
|
||||
await _safe_send_json(websocket, {"type": "error", "detail": f"voice ws error: {e}"})
|
||||
if _is_turn_persistence_unavailable(e):
|
||||
await _safe_send_json(
|
||||
websocket,
|
||||
{
|
||||
"type": "error",
|
||||
"code": "turn_persistence_unavailable",
|
||||
"detail": "voice turn persistence unavailable; retry the utterance",
|
||||
},
|
||||
)
|
||||
await _safe_send_json(websocket, {"type": "state", "state": "idle"})
|
||||
else:
|
||||
await _safe_send_json(websocket, {"type": "error", "detail": f"voice ws error: {e}"})
|
||||
finally:
|
||||
await _safe_close(websocket)
|
||||
|
||||
|
|
@ -609,6 +627,17 @@ async def _principal_from_websocket(websocket: WebSocket) -> Principal | None:
|
|||
)
|
||||
|
||||
|
||||
async def _practice_access_error(principal: Principal) -> str | None:
|
||||
if (
|
||||
principal.profile_completed_at is None
|
||||
and not await user_onboarding_complete(principal.user_id)
|
||||
):
|
||||
return "onboarding_required"
|
||||
if principal.consent_at is None and not await user_has_consent(principal.user_id):
|
||||
return "consent_required"
|
||||
return None
|
||||
|
||||
|
||||
async def _bind_session(
|
||||
websocket: WebSocket,
|
||||
principal: Principal,
|
||||
|
|
@ -616,6 +645,9 @@ async def _bind_session(
|
|||
"""Bind an existing session or create a dev-only voice session."""
|
||||
qp = websocket.query_params
|
||||
explicit_preset = qp.get("preset")
|
||||
access_error = await _practice_access_error(principal)
|
||||
if access_error is not None:
|
||||
return None, None, access_error, {}
|
||||
|
||||
session_id = qp.get("session_id")
|
||||
if session_id:
|
||||
|
|
@ -636,13 +668,6 @@ async def _bind_session(
|
|||
persona_code = qp.get("persona_code")
|
||||
if not persona_code:
|
||||
return None, None, "session_id or persona_code query required", {}
|
||||
if (
|
||||
principal.profile_completed_at is None
|
||||
and not await user_onboarding_complete(principal.user_id)
|
||||
):
|
||||
return None, None, "onboarding_required", {}
|
||||
if principal.consent_at is None and not await user_has_consent(principal.user_id):
|
||||
return None, None, "consent_required", {}
|
||||
try:
|
||||
catalog_persona = await get_catalog_persona(persona_code)
|
||||
except Exception:
|
||||
|
|
|
|||
|
|
@ -111,6 +111,23 @@ def normalize_json_value(value: Any) -> Any:
|
|||
return value
|
||||
|
||||
|
||||
def export_safe_supervisor_comments(value: Any) -> list[dict[str, Any]]:
|
||||
comments = normalize_json_value(value or [])
|
||||
if not isinstance(comments, list):
|
||||
return []
|
||||
safe_comments: list[dict[str, Any]] = []
|
||||
for item in comments:
|
||||
if not isinstance(item, Mapping):
|
||||
continue
|
||||
safe_item: dict[str, Any] = {}
|
||||
for key in ("kind", "intent_deviation"):
|
||||
if key in item:
|
||||
safe_item[key] = json_safe(normalize_json_value(item[key]))
|
||||
if safe_item:
|
||||
safe_comments.append(safe_item)
|
||||
return safe_comments
|
||||
|
||||
|
||||
def _redacted_sample(kind: str, value: str) -> str:
|
||||
if kind == "email" and "@" in value:
|
||||
return f"<email:{value.rsplit('@', 1)[1].lower()}>"
|
||||
|
|
@ -180,7 +197,7 @@ def build_dataset_record(
|
|||
"techniques": json_safe(normalize_json_value(row.get("techniques") or [])),
|
||||
"client_states": json_safe(normalize_json_value(row.get("client_states") or [])),
|
||||
"feedback_scores": json_safe(normalize_json_value(row.get("feedback_scores") or [])),
|
||||
"supervisor_comments": json_safe(normalize_json_value(row.get("supervisor_comments") or [])),
|
||||
"supervisor_comments": export_safe_supervisor_comments(row.get("supervisor_comments") or []),
|
||||
"source_refs": {
|
||||
"session_started_at": json_safe(row.get("session_started_at") or row.get("started_at")),
|
||||
"export_manifest_id": export_manifest_id,
|
||||
|
|
@ -393,6 +410,13 @@ def validate_manifest_gate(manifest: Mapping[str, Any]) -> None:
|
|||
errors.append("kappa must be >= 0.70")
|
||||
if (agreement.get("icc") or 0) < 0.75:
|
||||
errors.append("ICC must be >= 0.75")
|
||||
selection_criteria = manifest.get("selection_criteria") or {}
|
||||
if not isinstance(selection_criteria, Mapping) or selection_criteria.get("include_withdrawn") is not False:
|
||||
errors.append("include_withdrawn must be false")
|
||||
consent_scope = manifest.get("consent_scope") or {}
|
||||
allowed_uses = consent_scope.get("allowed_uses") if isinstance(consent_scope, Mapping) else None
|
||||
if not isinstance(allowed_uses, list) or "recursive_learning_seed" not in allowed_uses:
|
||||
errors.append("recursive_learning_seed consent scope is required")
|
||||
for key in ("data_steward", "legal_or_privacy_reviewer", "technical_operator", "approved_at"):
|
||||
if not str(approvals.get(key) or "").strip():
|
||||
errors.append(f"approval missing: {key}")
|
||||
|
|
|
|||
|
|
@ -52,6 +52,7 @@ from ..taxonomy import (
|
|||
CommentKind,
|
||||
Technique,
|
||||
TechniqueCategory,
|
||||
speaker_ko_label,
|
||||
)
|
||||
from . import guardrail
|
||||
|
||||
|
|
@ -482,7 +483,7 @@ def build_fast_messages(ctx: "TurnContext", client_reply: str) -> list[EngineMes
|
|||
theory = _theory_mode(ctx)
|
||||
client_reply_masked = guardrail.mask_pii(client_reply).text_masked
|
||||
recent = "\n".join(
|
||||
f"{('상담자' if t.get('speaker') == 'counselor' else '내담자')}: {t.get('text', '')}"
|
||||
f"{speaker_ko_label(t.get('speaker'))}: {t.get('text', '')}"
|
||||
for t in (ctx.memory.recent_turns or [])[-4:]
|
||||
) or "(직전 맥락 없음)"
|
||||
|
||||
|
|
@ -537,7 +538,7 @@ def build_deep_messages(
|
|||
) -> list[EngineMessage]:
|
||||
"""deep-loop 평가 프롬프트(전체 회기 + 코드 집계 분포 + 골든라벨 후보)."""
|
||||
transcript = "\n".join(
|
||||
f"{t.get('seq', '')}{('상담자' if t.get('speaker') == 'counselor' else '내담자')}: {t.get('text', '')}"
|
||||
f"{t.get('seq', '')}{speaker_ko_label(t.get('speaker'))}: {t.get('text', '')}"
|
||||
for t in masked_turns
|
||||
) or "(축어록 없음)"
|
||||
dist_lines = ", ".join(f"{k}:{v}" for k, v in distribution.by_category.items()) or "(없음)"
|
||||
|
|
@ -736,11 +737,11 @@ async def evaluate_turn(
|
|||
inference_geo=resp.inference_geo,
|
||||
latency_ms=latency_ms,
|
||||
)
|
||||
except EngineError as e:
|
||||
base.error = f"engine_error: {e}"
|
||||
except EngineError:
|
||||
base.error = "engine_error"
|
||||
return base
|
||||
except Exception as e: # 방어 — 어떤 예외도 상담 루프를 막지 않게
|
||||
base.error = f"eval_error: {e}"
|
||||
except Exception: # 방어 — 어떤 예외도 상담 루프를 막지 않게
|
||||
base.error = "eval_error"
|
||||
return base
|
||||
|
||||
payload = structured_payload_from_response(resp)
|
||||
|
|
@ -751,8 +752,8 @@ async def evaluate_turn(
|
|||
result = _parse_fast(payload, turn_seq=st.turn_seq, stage=st.stage.value, theory=theory)
|
||||
_evaluator_cache_put(cache_key, result.model_dump())
|
||||
return result
|
||||
except Exception as e: # 파싱 방어
|
||||
base.error = f"parse_error: {e}"
|
||||
except Exception: # 파싱 방어
|
||||
base.error = "parse_error"
|
||||
return base
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -25,6 +25,7 @@ from ..contracts.engine_gateway import structured_payload_from_response
|
|||
from ..engine_client import EngineClient, EngineError, EngineMessage, GenerateRequest
|
||||
from ..paths import repo_root, repo_path
|
||||
from ..session_read_model import StageLabel, stage_label_or_none
|
||||
from ..taxonomy import speaker_ko_label
|
||||
from . import guardrail
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
|
@ -33,6 +34,7 @@ if TYPE_CHECKING:
|
|||
|
||||
Tone = Literal["pos", "warn", "neutral"]
|
||||
CoachStatus = Literal["ready", "degraded"]
|
||||
CoachCreditEventType = Literal["use", "recharge"]
|
||||
CoachFocus = Literal[
|
||||
"rapport",
|
||||
"exploration",
|
||||
|
|
@ -71,6 +73,35 @@ class LiveCoachSuggestion(BaseModel):
|
|||
sources: list[LiveCoachSource] = Field(default_factory=list)
|
||||
safety_note: Optional[str] = None
|
||||
latency_ms: int = 0
|
||||
persistence_source: Literal["database", "runtime"] = "database"
|
||||
quota: Optional["LiveCoachQuota"] = None
|
||||
credit_events: list["LiveCoachCreditEvent"] = Field(default_factory=list)
|
||||
|
||||
|
||||
class LiveCoachQuota(BaseModel):
|
||||
"""회기 중 즉시 코칭 사용 가능 횟수."""
|
||||
|
||||
remaining: int = Field(ge=0)
|
||||
max: int = Field(ge=1)
|
||||
|
||||
|
||||
class LiveCoachCreditEvent(BaseModel):
|
||||
"""코칭 기회 사용/충전 학습 기록."""
|
||||
|
||||
event_id: str
|
||||
session_id: str
|
||||
turn_seq: int
|
||||
stage: StageLabel | None = None
|
||||
event_type: CoachCreditEventType
|
||||
delta: int
|
||||
balance: int = Field(ge=0)
|
||||
reason: str
|
||||
created_at: str
|
||||
|
||||
@field_validator("stage", mode="before")
|
||||
@classmethod
|
||||
def _normalize_stage(cls, value: object) -> StageLabel | None:
|
||||
return stage_label_or_none(value)
|
||||
|
||||
|
||||
class LiveCoachEvent(BaseModel):
|
||||
|
|
@ -313,6 +344,7 @@ def build_rag_index_payloads() -> list[dict[str, Any]]:
|
|||
"sensitivity": _rag_sensitivity(source, chunk, kb_kind),
|
||||
"meta": {
|
||||
"live_coaching_source": True,
|
||||
"source_title": title,
|
||||
"source_type": str(chunk.get("source_type") or source.get("source_type") or ""),
|
||||
"source_version": version_label,
|
||||
"citation": chunk_citation,
|
||||
|
|
@ -557,14 +589,33 @@ def _grounding_block(grounding: list[LiveCoachGrounding]) -> str:
|
|||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _mask_prompt_text(value: object) -> str:
|
||||
return guardrail.mask_pii(str(value or "")).text_masked
|
||||
|
||||
|
||||
def _mask_prompt_value(value: Any) -> Any:
|
||||
if isinstance(value, str):
|
||||
return _mask_prompt_text(value)
|
||||
if isinstance(value, dict):
|
||||
return {
|
||||
_mask_prompt_text(key): _mask_prompt_value(child)
|
||||
for key, child in value.items()
|
||||
}
|
||||
if isinstance(value, list):
|
||||
return [_mask_prompt_value(child) for child in value]
|
||||
if isinstance(value, tuple):
|
||||
return [_mask_prompt_value(child) for child in value]
|
||||
return value
|
||||
|
||||
|
||||
def _messages(item: LiveCoachInput, grounding: list[LiveCoachGrounding]) -> list[EngineMessage]:
|
||||
learner_masked = guardrail.mask_pii(item.learner_text).text_masked
|
||||
client_masked = guardrail.mask_pii(item.client_reply or "").text_masked
|
||||
learner_masked = _mask_prompt_text(item.learner_text)
|
||||
client_masked = _mask_prompt_text(item.client_reply or "")
|
||||
recent = "\n".join(
|
||||
f"{'상담자' if t.get('speaker') == 'counselor' else '내담자'}: {t.get('text', '')}"
|
||||
f"{speaker_ko_label(t.get('speaker'))}: {_mask_prompt_text(t.get('text', ''))}"
|
||||
for t in item.recent_turns[-6:]
|
||||
) or "(최근 맥락 없음)"
|
||||
evaluation = json.dumps(item.evaluation or {}, ensure_ascii=False)[:1200]
|
||||
evaluation = json.dumps(_mask_prompt_value(item.evaluation or {}), ensure_ascii=False)[:1200]
|
||||
system = (
|
||||
"당신은 심리상담 수련생에게 회기 중 즉시 피드백을 주는 라이브 코치다.\n"
|
||||
"목표는 지금 흐름을 끊지 않고 다음 상담자 발화 하나를 더 낫게 만드는 것이다.\n\n"
|
||||
|
|
@ -669,6 +720,8 @@ __all__ = [
|
|||
"LiveCoachEvent",
|
||||
"LiveCoachGrounding",
|
||||
"LiveCoachInput",
|
||||
"LiveCoachCreditEvent",
|
||||
"LiveCoachQuota",
|
||||
"LiveCoachSource",
|
||||
"LiveCoachSuggestion",
|
||||
"clear_local_source_pack_cache",
|
||||
|
|
|
|||
|
|
@ -23,6 +23,7 @@ from dataclasses import dataclass, field
|
|||
from typing import Any, Callable, Literal, Optional
|
||||
|
||||
from .state_machine import SessionState
|
||||
from ..taxonomy import speaker_ko_label
|
||||
|
||||
|
||||
_SESSION_DIGEST_EXCERPT_CHARS = 90
|
||||
|
|
@ -331,7 +332,7 @@ def build_compression_messages(job: CompressionJob) -> list[dict[str, str]]:
|
|||
pinned 사실 보존·정답 미포함 지시 포함.
|
||||
"""
|
||||
transcript = "\n".join(
|
||||
f"{('상담자' if t.speaker == 'counselor' else '내담자')}: {t.text}"
|
||||
f"{speaker_ko_label(t.speaker)}: {t.text}"
|
||||
for t in job.digest_input.masked_turns
|
||||
)
|
||||
threads = "\n".join(f"- {t}" for t in job.digest_input.open_threads) or "(없음)"
|
||||
|
|
|
|||
|
|
@ -50,6 +50,25 @@ EvalHook = Callable[["TurnContext", str], Awaitable[Optional[dict]]]
|
|||
LlmAuditHook = Callable[[dict[str, Any]], Awaitable[None]]
|
||||
|
||||
|
||||
def turn_evaluation_error_payload(ctx: "TurnContext", error: BaseException | str) -> dict[str, Any]:
|
||||
"""Represent a non-fatal fast-loop evaluator failure without hiding it."""
|
||||
st = ctx.state_after or ctx.state_before
|
||||
if isinstance(error, BaseException):
|
||||
# Exception messages can contain raw learner/provider text. Keep review-facing
|
||||
# evidence to the exception type; detailed trace stays in server logs.
|
||||
detail = type(error).__name__
|
||||
else:
|
||||
detail = str(error).strip() or "unknown turn evaluation error"
|
||||
masked = guardrail.mask_pii(detail).text_masked.strip() or "unknown turn evaluation error"
|
||||
return {
|
||||
"loop": "fast",
|
||||
"turn_seq": st.turn_seq,
|
||||
"stage": st.stage.value,
|
||||
"appropriateness": "neutral",
|
||||
"error": masked,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class TurnContext:
|
||||
"""한 턴 파이프라인을 관통하는 컨텍스트(가드레일/상태/페르소나 산출 집약)."""
|
||||
|
|
@ -248,8 +267,9 @@ async def run_turn_generate(
|
|||
if eval_hook is not None:
|
||||
try:
|
||||
evaluation = await eval_hook(ctx, reply)
|
||||
except Exception:
|
||||
evaluation = None # 평가 실패가 상담 루프를 막지 않게(비치명적)
|
||||
except Exception as exc:
|
||||
# 평가 실패는 상담 루프를 막지 않되, 리뷰/대시보드에서 조용히 사라지지 않게 남긴다.
|
||||
evaluation = turn_evaluation_error_payload(ctx, exc)
|
||||
|
||||
return TurnResult(
|
||||
turn_seq=st.turn_seq,
|
||||
|
|
|
|||
|
|
@ -77,6 +77,8 @@ def turn_eval(turn: Any) -> dict[str, Any] | None:
|
|||
|
||||
|
||||
def turn_score(ev: dict[str, Any]) -> float | None:
|
||||
if str(ev.get("error") or "").strip():
|
||||
return None
|
||||
raw = str(ev.get("appropriateness") or "").strip().lower()
|
||||
return _APPROPRIATENESS_SCORE.get(raw)
|
||||
|
||||
|
|
@ -118,6 +120,8 @@ def turn_techniques(ev: dict[str, Any]) -> list[str]:
|
|||
|
||||
|
||||
def turn_feedback_note(ev: dict[str, Any]) -> str | None:
|
||||
if str(ev.get("error") or "").strip():
|
||||
return None
|
||||
raw = ev.get("appropriateness_note")
|
||||
if raw is None:
|
||||
return None
|
||||
|
|
|
|||
|
|
@ -15,16 +15,9 @@ MASTERPLAN §0/§2.2 + MEMORY_KNOWLEDGE_PERSONA_DESIGN §1.1·P2:
|
|||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field, replace
|
||||
from enum import Enum
|
||||
from typing import Optional
|
||||
|
||||
|
||||
# ── 단계 (taxonomy.Stage 와 한글 값 동일, 서비스 내부 결정론 전이용) ────────
|
||||
class Stage(str, Enum):
|
||||
RAPPORT = "라포"
|
||||
EXPLORE = "탐색"
|
||||
INTERVENE = "개입"
|
||||
CLOSE = "정리"
|
||||
from ..taxonomy import Stage # 단계 라벨 단일 정의 = taxonomy.Stage; 이 모듈은 전이 로직만 소유.
|
||||
|
||||
|
||||
# 단계별 기본 개방도(stage_base). 라포는 낮게 시작, 개입에서 가장 깊게 다룸.
|
||||
|
|
|
|||
15
apps/api/app/session_evaluation_input.py
Normal file
15
apps/api/app/session_evaluation_input.py
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
"""Shared input shaping for session-level evaluation runs."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterable, Mapping
|
||||
|
||||
|
||||
def enriched_masked_turns(masked_turns: Iterable[Mapping[str, object]]) -> list[dict[str, object]]:
|
||||
"""Attach the same 1-based turn sequence to every session evaluation path."""
|
||||
enriched: list[dict[str, object]] = []
|
||||
for index, turn in enumerate(masked_turns, start=1):
|
||||
item = dict(turn)
|
||||
item["seq"] = index
|
||||
enriched.append(item)
|
||||
return enriched
|
||||
|
|
@ -38,6 +38,14 @@ _APPROPRIATENESS_SCORE = {
|
|||
"neutral": 3.0,
|
||||
"pos": 5.0,
|
||||
}
|
||||
LIVE_COACH_INITIAL_CREDITS = 3
|
||||
LIVE_COACH_MAX_CREDITS = 3
|
||||
LIVE_COACH_USE_REASON = "AI 코칭 힌트 사용"
|
||||
LIVE_COACH_RECHARGE_REASON = "좋은 발화로 내담자 변화 신호 확인"
|
||||
|
||||
|
||||
class LiveCoachCreditExhausted(RuntimeError):
|
||||
"""Raised when a learner tries to use live coaching without credits."""
|
||||
|
||||
|
||||
def _coerce_error_message(error: BaseException | str) -> str:
|
||||
|
|
@ -99,8 +107,8 @@ class SessionEvaluationWrite:
|
|||
source=source,
|
||||
scope=result.scope,
|
||||
stage=result.stage,
|
||||
payload=result.to_dict(),
|
||||
error=result.error,
|
||||
payload=_mask_json_text_values(result.to_dict()),
|
||||
error=_clean_masked_text(result.error),
|
||||
)
|
||||
|
||||
@classmethod
|
||||
|
|
@ -122,7 +130,7 @@ class SessionEvaluationWrite:
|
|||
scope=scope,
|
||||
stage=stage,
|
||||
payload={},
|
||||
error=_coerce_error_message(error),
|
||||
error=_clean_masked_text(_coerce_error_message(error)),
|
||||
)
|
||||
|
||||
def cache_record(self) -> dict[str, Any]:
|
||||
|
|
@ -244,6 +252,90 @@ def _live_coach_event_from_row(row) -> dict[str, Any]:
|
|||
}
|
||||
|
||||
|
||||
def _live_coach_remaining(value: int | float | None) -> int:
|
||||
try:
|
||||
parsed = int(value if value is not None else LIVE_COACH_INITIAL_CREDITS)
|
||||
except (TypeError, ValueError):
|
||||
parsed = LIVE_COACH_INITIAL_CREDITS
|
||||
return max(0, min(LIVE_COACH_MAX_CREDITS, parsed))
|
||||
|
||||
|
||||
def _live_coach_quota_payload(remaining: int | float | None) -> dict[str, int]:
|
||||
return {"remaining": _live_coach_remaining(remaining), "max": LIVE_COACH_MAX_CREDITS}
|
||||
|
||||
|
||||
def _live_coach_delta_from_record(record: dict[str, Any]) -> int:
|
||||
if record.get("credit_delta") is not None:
|
||||
try:
|
||||
return int(record["credit_delta"])
|
||||
except (TypeError, ValueError):
|
||||
pass
|
||||
return 1 if record.get("event_type") == "recharge" else -1
|
||||
|
||||
|
||||
def _live_coach_cached_remaining(session_id: str) -> int:
|
||||
delta = sum(
|
||||
_live_coach_delta_from_record(record)
|
||||
for record in _LIVE_COACH_EVENT_CACHE.get(session_id, [])
|
||||
)
|
||||
return _live_coach_remaining(LIVE_COACH_INITIAL_CREDITS + delta)
|
||||
|
||||
|
||||
async def _live_coach_remaining_for_conn(conn: Any, session_id: str) -> int:
|
||||
delta = await conn.fetchval(
|
||||
"""
|
||||
SELECT COALESCE(SUM(credit_delta), 0)
|
||||
FROM app.live_coach_events
|
||||
WHERE session_id = $1::uuid
|
||||
""",
|
||||
session_id,
|
||||
)
|
||||
return _live_coach_remaining(LIVE_COACH_INITIAL_CREDITS + int(delta or 0))
|
||||
|
||||
|
||||
def _live_coach_credit_event_from_row(row) -> dict[str, Any]:
|
||||
event_type = str(_row_value(row, "event_type") or "use")
|
||||
if event_type not in {"use", "recharge"}:
|
||||
event_type = "use"
|
||||
delta = _row_value(row, "credit_delta")
|
||||
if delta is None:
|
||||
delta = 1 if event_type == "recharge" else -1
|
||||
balance = _row_value(row, "credit_balance")
|
||||
return {
|
||||
"event_id": str(row["id"]),
|
||||
"session_id": str(row["session_id"]),
|
||||
"turn_seq": int(row["turn_seq"] or 1),
|
||||
"stage": str(row["stage"] or ""),
|
||||
"event_type": event_type,
|
||||
"delta": int(delta),
|
||||
"balance": _live_coach_remaining(balance),
|
||||
"reason": str(_row_value(row, "reason") or (
|
||||
LIVE_COACH_RECHARGE_REASON if event_type == "recharge" else LIVE_COACH_USE_REASON
|
||||
)),
|
||||
"created_at": _iso_dt(row["created_at"]),
|
||||
}
|
||||
|
||||
|
||||
def _live_coach_credit_event_from_record(record: dict[str, Any]) -> dict[str, Any]:
|
||||
event_type = str(record.get("event_type") or "use")
|
||||
if event_type not in {"use", "recharge"}:
|
||||
event_type = "use"
|
||||
delta = _live_coach_delta_from_record(record)
|
||||
return {
|
||||
"event_id": str(record.get("event_id") or uuid.uuid4()),
|
||||
"session_id": str(record.get("session_id") or ""),
|
||||
"turn_seq": int(record.get("turn_seq") or 1),
|
||||
"stage": str(record.get("stage") or ""),
|
||||
"event_type": event_type,
|
||||
"delta": delta,
|
||||
"balance": _live_coach_remaining(record.get("credit_balance")),
|
||||
"reason": str(record.get("reason") or (
|
||||
LIVE_COACH_RECHARGE_REASON if event_type == "recharge" else LIVE_COACH_USE_REASON
|
||||
)),
|
||||
"created_at": str(record.get("created_at") or ""),
|
||||
}
|
||||
|
||||
|
||||
def _live_coach_cache_record(
|
||||
*,
|
||||
session_id: str,
|
||||
|
|
@ -252,6 +344,35 @@ def _live_coach_cache_record(
|
|||
learner_text: str,
|
||||
client_reply: str | None,
|
||||
suggestion: Any,
|
||||
credit_balance: int | None = None,
|
||||
) -> dict[str, Any]:
|
||||
now = datetime.now(timezone.utc)
|
||||
balance = _live_coach_remaining(
|
||||
credit_balance if credit_balance is not None else _live_coach_cached_remaining(session_id) - 1
|
||||
)
|
||||
return {
|
||||
"event_id": str(uuid.uuid4()),
|
||||
"session_id": session_id,
|
||||
"turn_seq": int(turn_seq),
|
||||
"stage": stage,
|
||||
"event_type": "use",
|
||||
"credit_delta": -1,
|
||||
"credit_balance": balance,
|
||||
"reason": LIVE_COACH_USE_REASON,
|
||||
"created_at": now.isoformat().replace("+00:00", "Z"),
|
||||
"learner_text_excerpt": _masked_excerpt(learner_text),
|
||||
"client_reply_excerpt": _masked_excerpt(client_reply),
|
||||
"suggestion": _model_payload(suggestion),
|
||||
}
|
||||
|
||||
|
||||
def _live_coach_recharge_cache_record(
|
||||
*,
|
||||
session_id: str,
|
||||
turn_seq: int,
|
||||
stage: str,
|
||||
credit_balance: int,
|
||||
reason: str,
|
||||
) -> dict[str, Any]:
|
||||
now = datetime.now(timezone.utc)
|
||||
return {
|
||||
|
|
@ -259,13 +380,22 @@ def _live_coach_cache_record(
|
|||
"session_id": session_id,
|
||||
"turn_seq": int(turn_seq),
|
||||
"stage": stage,
|
||||
"event_type": "recharge",
|
||||
"credit_delta": 1,
|
||||
"credit_balance": _live_coach_remaining(credit_balance),
|
||||
"reason": reason,
|
||||
"created_at": now.isoformat().replace("+00:00", "Z"),
|
||||
"learner_text_excerpt": _masked_excerpt(learner_text),
|
||||
"client_reply_excerpt": _masked_excerpt(client_reply),
|
||||
"suggestion": _model_payload(suggestion),
|
||||
"learner_text_excerpt": None,
|
||||
"client_reply_excerpt": None,
|
||||
"suggestion": {},
|
||||
}
|
||||
|
||||
|
||||
def _live_coach_payload_from_record(record: dict[str, Any]) -> dict[str, Any]:
|
||||
payload = dict(record.get("suggestion") or {})
|
||||
return payload
|
||||
|
||||
|
||||
def _row_value(row, key: str):
|
||||
try:
|
||||
return row[key]
|
||||
|
|
@ -294,6 +424,26 @@ def _clean_text(value: Any) -> str | None:
|
|||
return text or None
|
||||
|
||||
|
||||
def _clean_masked_text(value: Any) -> str | None:
|
||||
text = _clean_text(value)
|
||||
if text is None:
|
||||
return None
|
||||
masked = guardrail.mask_pii(text).text_masked.strip()
|
||||
return masked or None
|
||||
|
||||
|
||||
def _mask_json_text_values(value: Any) -> Any:
|
||||
if isinstance(value, str):
|
||||
return _clean_masked_text(value) or ""
|
||||
if isinstance(value, dict):
|
||||
return {key: _mask_json_text_values(child) for key, child in value.items()}
|
||||
if isinstance(value, list):
|
||||
return [_mask_json_text_values(child) for child in value]
|
||||
if isinstance(value, tuple):
|
||||
return [_mask_json_text_values(child) for child in value]
|
||||
return value
|
||||
|
||||
|
||||
def _safe_float(value: Any) -> float | None:
|
||||
if isinstance(value, (int, float)):
|
||||
return float(value)
|
||||
|
|
@ -354,7 +504,7 @@ def _evaluation_feedback_rows(evaluation: dict[str, Any] | None) -> list[dict[st
|
|||
add(
|
||||
"appropriateness",
|
||||
score=_APPROPRIATENESS_SCORE[appropriateness],
|
||||
rationale=_clean_text(evaluation.get("appropriateness_note")),
|
||||
rationale=_clean_masked_text(evaluation.get("appropriateness_note")),
|
||||
)
|
||||
|
||||
rapport = _safe_float(evaluation.get("rapport_signal"))
|
||||
|
|
@ -365,19 +515,19 @@ def _evaluation_feedback_rows(evaluation: dict[str, Any] | None) -> list[dict[st
|
|||
if theory_mode:
|
||||
add("theory_mode", rationale=theory_mode)
|
||||
|
||||
error = _clean_text(evaluation.get("error"))
|
||||
error = _clean_masked_text(evaluation.get("error"))
|
||||
if error:
|
||||
add("error", rationale=error)
|
||||
|
||||
for tag in _dict_items(evaluation.get("techniques")):
|
||||
code = _clean_text(tag.get("code"))
|
||||
rationale = _clean_text(tag.get("rationale"))
|
||||
rationale = _clean_masked_text(tag.get("rationale"))
|
||||
if code and rationale:
|
||||
add(f"technique:{code}", rationale=rationale)
|
||||
|
||||
for state in _dict_items(evaluation.get("client_state_read")):
|
||||
code = _clean_text(state.get("code"))
|
||||
rationale = _clean_text(state.get("rationale"))
|
||||
rationale = _clean_masked_text(state.get("rationale"))
|
||||
if code and rationale:
|
||||
add(f"client_state:{code}", rationale=rationale)
|
||||
|
||||
|
|
@ -425,8 +575,8 @@ def _evaluation_comment_rows(evaluation: dict[str, Any] | None) -> list[dict[str
|
|||
deviation = evaluation.get("intent_deviation")
|
||||
if not isinstance(deviation, dict):
|
||||
return []
|
||||
note = _clean_text(evaluation.get("appropriateness_note")) or "의도와 다른 부분"
|
||||
return [{"kind": "critique", "text": note, "intent_deviation": deviation}]
|
||||
note = _clean_masked_text(evaluation.get("appropriateness_note")) or "의도와 다른 부분"
|
||||
return [{"kind": "critique", "text": note, "intent_deviation": _mask_json_text_values(deviation)}]
|
||||
|
||||
|
||||
def _evaluation_alternative_rows(evaluation: dict[str, Any] | None) -> list[dict[str, str | None]]:
|
||||
|
|
@ -438,10 +588,10 @@ def _evaluation_alternative_rows(evaluation: dict[str, Any] | None) -> list[dict
|
|||
rows: list[dict[str, str | None]] = []
|
||||
for item in alternatives:
|
||||
if isinstance(item, dict):
|
||||
suggestion = _clean_text(item.get("suggestion") or item.get("text"))
|
||||
rationale = _clean_text(item.get("rationale"))
|
||||
suggestion = _clean_masked_text(item.get("suggestion") or item.get("text"))
|
||||
rationale = _clean_masked_text(item.get("rationale"))
|
||||
else:
|
||||
suggestion = _clean_text(item)
|
||||
suggestion = _clean_masked_text(item)
|
||||
rationale = None
|
||||
if suggestion:
|
||||
rows.append({"suggestion": suggestion, "rationale": rationale})
|
||||
|
|
@ -652,12 +802,13 @@ def _state_from_row(row, card: PersonaCard) -> state_machine.SessionState:
|
|||
|
||||
def _turn_from_row(row, evaluation: dict[str, Any] | None = None) -> TurnRecord:
|
||||
created_at = _ts(row["created_at"]) or time.time()
|
||||
text_masked = _clean_text(row["text_masked"]) or _clean_masked_text(row["text"]) or ""
|
||||
return TurnRecord(
|
||||
turn_seq=int(row["seq"]),
|
||||
speaker=row["speaker"],
|
||||
stage=row["stage"],
|
||||
text=row["text"] or row["text_masked"] or "",
|
||||
text_masked=row["text_masked"] or row["text"] or "",
|
||||
text=text_masked,
|
||||
text_masked=text_masked,
|
||||
turn_id=str(_row_value(row, "id")) if _row_value(row, "id") is not None else None,
|
||||
created_at=created_at,
|
||||
llm_provider=_row_value(row, "llm_provider"),
|
||||
|
|
@ -780,6 +931,31 @@ async def _persist_turn_evaluation(conn: Any, turn_id: str, evaluation: dict[str
|
|||
)
|
||||
|
||||
|
||||
async def _replace_turn_evaluation(conn: Any, turn_id: str, evaluation: dict[str, Any] | None) -> None:
|
||||
await conn.execute("SELECT set_config('app.ai_context', '1', true)")
|
||||
await conn.execute("SELECT set_config('app.current_ai_view', 'evaluator', true)")
|
||||
await conn.execute("SELECT set_config('app.current_sens_max', '2', true)")
|
||||
await conn.execute("DELETE FROM app.feedback_scores WHERE turn_id = $1::uuid", turn_id)
|
||||
await conn.execute("DELETE FROM app.turn_technique WHERE turn_id = $1::uuid", turn_id)
|
||||
await conn.execute("DELETE FROM app.turn_client_state WHERE turn_id = $1::uuid", turn_id)
|
||||
await conn.execute("DELETE FROM app.supervisor_comment WHERE turn_id = $1::uuid", turn_id)
|
||||
await conn.execute("DELETE FROM app.alternative_utterance WHERE turn_id = $1::uuid", turn_id)
|
||||
await _persist_turn_evaluation(conn, turn_id, _mask_json_text_values(evaluation))
|
||||
|
||||
|
||||
async def replace_turn_evaluation(*, turn_id: str, evaluation: dict[str, Any] | None) -> bool:
|
||||
if not turn_id or not isinstance(evaluation, dict):
|
||||
return False
|
||||
try:
|
||||
get_pool()
|
||||
async with acquire(ai_context=True, ai_view="evaluator") as conn:
|
||||
await _replace_turn_evaluation(conn, turn_id, evaluation)
|
||||
return True
|
||||
except Exception:
|
||||
require_runtime_fallback_allowed("turn evaluation replacement")
|
||||
return False
|
||||
|
||||
|
||||
async def _load_turn_evaluations(
|
||||
conn: Any,
|
||||
turn_refs: list[tuple[str, int, str]],
|
||||
|
|
@ -853,8 +1029,14 @@ async def _load_turn_evaluations(
|
|||
|
||||
|
||||
async def _hydrate_session_turn_evaluations(sess: InProcSession) -> None:
|
||||
await _hydrate_sessions_turn_evaluations([sess])
|
||||
|
||||
|
||||
async def _hydrate_sessions_turn_evaluations(sessions: list[InProcSession]) -> None:
|
||||
"""여러 세션의 턴 평가를 evaluator-view 연결 1회로 배치 하이드레이트(세션당 N+1 제거)."""
|
||||
turn_refs = [
|
||||
(turn.turn_id, turn.turn_seq, turn.stage)
|
||||
for sess in sessions
|
||||
for turn in sess.turns
|
||||
if turn.turn_id is not None
|
||||
]
|
||||
|
|
@ -862,9 +1044,10 @@ async def _hydrate_session_turn_evaluations(sess: InProcSession) -> None:
|
|||
return
|
||||
async with acquire(ai_view="evaluator") as conn:
|
||||
evaluations = await _load_turn_evaluations(conn, turn_refs)
|
||||
for turn in sess.turns:
|
||||
if turn.turn_id and turn.turn_id in evaluations:
|
||||
turn.evaluation = evaluations[turn.turn_id]
|
||||
for sess in sessions:
|
||||
for turn in sess.turns:
|
||||
if turn.turn_id and turn.turn_id in evaluations:
|
||||
turn.evaluation = evaluations[turn.turn_id]
|
||||
|
||||
|
||||
async def ensure_review_tables() -> None:
|
||||
|
|
@ -946,6 +1129,47 @@ async def ensure_review_tables() -> None:
|
|||
)
|
||||
"""
|
||||
)
|
||||
await conn.execute(
|
||||
"""
|
||||
ALTER TABLE app.feedback_scores ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE app.turn_technique ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE app.turn_client_state ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE app.supervisor_comment ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE app.alternative_utterance ENABLE ROW LEVEL SECURITY;
|
||||
|
||||
DROP POLICY IF EXISTS p_feedback_delete ON app.feedback_scores;
|
||||
DROP POLICY IF EXISTS p_turn_technique_delete ON app.turn_technique;
|
||||
DROP POLICY IF EXISTS p_turn_client_state_delete ON app.turn_client_state;
|
||||
DROP POLICY IF EXISTS p_supervisor_comment_delete ON app.supervisor_comment;
|
||||
DROP POLICY IF EXISTS p_alternative_utterance_delete ON app.alternative_utterance;
|
||||
|
||||
CREATE POLICY p_feedback_delete
|
||||
ON app.feedback_scores FOR DELETE USING (
|
||||
app.is_ai_context()
|
||||
OR app.current_role_name() IN ('admin','instructor')
|
||||
);
|
||||
CREATE POLICY p_turn_technique_delete
|
||||
ON app.turn_technique FOR DELETE USING (
|
||||
app.is_ai_context()
|
||||
OR app.current_role_name() IN ('admin','instructor')
|
||||
);
|
||||
CREATE POLICY p_turn_client_state_delete
|
||||
ON app.turn_client_state FOR DELETE USING (
|
||||
app.is_ai_context()
|
||||
OR app.current_role_name() IN ('admin','instructor')
|
||||
);
|
||||
CREATE POLICY p_supervisor_comment_delete
|
||||
ON app.supervisor_comment FOR DELETE USING (
|
||||
app.is_ai_context()
|
||||
OR app.current_role_name() IN ('admin','instructor')
|
||||
);
|
||||
CREATE POLICY p_alternative_utterance_delete
|
||||
ON app.alternative_utterance FOR DELETE USING (
|
||||
app.is_ai_context()
|
||||
OR app.current_role_name() IN ('admin','instructor')
|
||||
)
|
||||
"""
|
||||
)
|
||||
await conn.execute(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS app.case_worksheet (
|
||||
|
|
@ -1013,13 +1237,45 @@ async def ensure_review_tables() -> None:
|
|||
session_id UUID NOT NULL REFERENCES app.sessions(id) ON DELETE CASCADE,
|
||||
turn_seq INT NOT NULL CHECK (turn_seq >= 1),
|
||||
stage TEXT NOT NULL,
|
||||
event_type TEXT NOT NULL DEFAULT 'use',
|
||||
credit_delta INT NOT NULL DEFAULT -1,
|
||||
credit_balance INT,
|
||||
reason TEXT,
|
||||
learner_text_excerpt TEXT,
|
||||
client_reply_excerpt TEXT,
|
||||
payload JSONB NOT NULL DEFAULT '{}'::jsonb,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
ALTER TABLE app.live_coach_events
|
||||
ADD COLUMN IF NOT EXISTS event_type TEXT NOT NULL DEFAULT 'use';
|
||||
ALTER TABLE app.live_coach_events
|
||||
ADD COLUMN IF NOT EXISTS credit_delta INT NOT NULL DEFAULT -1;
|
||||
ALTER TABLE app.live_coach_events
|
||||
ADD COLUMN IF NOT EXISTS credit_balance INT;
|
||||
ALTER TABLE app.live_coach_events
|
||||
ADD COLUMN IF NOT EXISTS reason TEXT;
|
||||
UPDATE app.live_coach_events
|
||||
SET event_type = 'use'
|
||||
WHERE event_type IS NULL OR event_type NOT IN ('use', 'recharge');
|
||||
UPDATE app.live_coach_events
|
||||
SET credit_delta = CASE WHEN event_type = 'recharge' THEN 1 ELSE -1 END
|
||||
WHERE credit_delta IS NULL;
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conname = 'live_coach_events_event_type_check'
|
||||
) THEN
|
||||
ALTER TABLE app.live_coach_events
|
||||
ADD CONSTRAINT live_coach_events_event_type_check
|
||||
CHECK (event_type IN ('use', 'recharge'));
|
||||
END IF;
|
||||
END $$;
|
||||
CREATE INDEX IF NOT EXISTS idx_live_coach_events_session_turn
|
||||
ON app.live_coach_events(session_id, turn_seq, created_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_live_coach_events_session_type
|
||||
ON app.live_coach_events(session_id, event_type, created_at);
|
||||
|
||||
ALTER TABLE app.live_coach_events ENABLE ROW LEVEL SECURITY;
|
||||
|
||||
|
|
@ -1270,6 +1526,8 @@ async def ensure_review_tables() -> None:
|
|||
|
||||
|
||||
async def save_session_evaluation(write: SessionEvaluationWrite) -> bool:
|
||||
write.payload = _mask_json_text_values(write.payload)
|
||||
write.error = _clean_masked_text(write.error)
|
||||
record = write.cache_record()
|
||||
if runtime_fallback_allowed():
|
||||
_EVALUATION_CACHE[write.session_id] = record
|
||||
|
|
@ -1342,6 +1600,150 @@ async def load_session_evaluation(
|
|||
return _EVALUATION_CACHE.get(session_id), False
|
||||
|
||||
|
||||
async def list_session_evaluations(
|
||||
session_ids: list[str],
|
||||
principal: Principal,
|
||||
) -> tuple[dict[str, dict[str, Any]], bool]:
|
||||
if not session_ids:
|
||||
return {}, True
|
||||
try:
|
||||
get_pool()
|
||||
async with acquire(
|
||||
role=principal.role.value,
|
||||
user_id=principal.user_id,
|
||||
cohort_ids=principal.cohort_ids,
|
||||
) as conn:
|
||||
rows = await conn.fetch(
|
||||
"""
|
||||
SELECT session_id::text AS session_id, status, source, scope, stage, payload, error, updated_at
|
||||
FROM app.session_evaluation
|
||||
WHERE session_id = ANY($1::uuid[])
|
||||
""",
|
||||
session_ids,
|
||||
)
|
||||
records = {
|
||||
str(row["session_id"]): {
|
||||
"status": row["status"],
|
||||
"source": row["source"],
|
||||
"scope": row["scope"],
|
||||
"stage": row["stage"],
|
||||
"payload": dict(row["payload"] or {}),
|
||||
"error": row["error"],
|
||||
"updated_at": _ts(row["updated_at"]),
|
||||
}
|
||||
for row in rows
|
||||
}
|
||||
cached_used = False
|
||||
if runtime_fallback_allowed():
|
||||
for session_id in session_ids:
|
||||
if session_id not in records and session_id in _EVALUATION_CACHE:
|
||||
records[session_id] = _EVALUATION_CACHE[session_id]
|
||||
cached_used = True
|
||||
return records, not cached_used
|
||||
except Exception:
|
||||
require_runtime_fallback_allowed("session evaluation")
|
||||
return {
|
||||
session_id: _EVALUATION_CACHE[session_id]
|
||||
for session_id in session_ids
|
||||
if session_id in _EVALUATION_CACHE
|
||||
}, False
|
||||
|
||||
|
||||
async def list_sessions_missing_session_evaluation(
|
||||
*,
|
||||
older_than_seconds: float,
|
||||
limit: int,
|
||||
) -> tuple[list[InProcSession], bool]:
|
||||
if limit <= 0:
|
||||
return [], True
|
||||
try:
|
||||
get_pool()
|
||||
stale_seconds = max(float(older_than_seconds), 0.0)
|
||||
async with acquire(ai_context=True, ai_view="evaluator") as conn:
|
||||
rows = await conn.fetch(
|
||||
"""
|
||||
SELECT
|
||||
s.id, s.runtime_case_id, s.case_id, s.learner_id, s.persona_code,
|
||||
s.session_no, s.theory_mode, s.started_at, s.ended_at,
|
||||
s.prev_rapport_credit,
|
||||
COALESCE(
|
||||
NULLIF(learner.display_name, ''),
|
||||
NULLIF(learner.nickname, ''),
|
||||
NULLIF(learner.email, ''),
|
||||
s.learner_id::text
|
||||
) AS learner_label,
|
||||
pc.persona_id AS card_persona_id,
|
||||
pc.code AS card_code,
|
||||
pc.version AS card_version,
|
||||
pc.status AS card_status,
|
||||
pc.display_name AS card_display_name,
|
||||
pc.difficulty AS card_difficulty,
|
||||
pc.theory_target AS card_theory_target,
|
||||
pc.demographics AS card_demographics,
|
||||
pc.presenting AS card_presenting,
|
||||
pc.history AS card_history,
|
||||
pc.big5 AS card_big5,
|
||||
pc.resistance AS card_resistance,
|
||||
pc.speech_style AS card_speech_style,
|
||||
pc.affect_baseline AS card_affect_baseline,
|
||||
pc.ccd AS card_ccd,
|
||||
pc.dsm5_dimensional AS card_dsm5_dimensional,
|
||||
pc.triggers AS card_triggers,
|
||||
pc.source_provenance AS card_source_provenance,
|
||||
pc.is_synthetic AS card_is_synthetic
|
||||
FROM app.sessions s
|
||||
LEFT JOIN app.session_evaluation se ON se.session_id = s.id
|
||||
LEFT JOIN app.persona_card pc
|
||||
ON pc.persona_id = s.persona_id
|
||||
AND pc.version = s.persona_version
|
||||
LEFT JOIN app.app_user learner ON learner.user_id = s.learner_id
|
||||
WHERE s.ended_at IS NOT NULL
|
||||
AND s.ended_at <= now() - ($1::double precision * interval '1 second')
|
||||
AND se.session_id IS NULL
|
||||
AND EXISTS (
|
||||
SELECT 1
|
||||
FROM app.turns t
|
||||
WHERE t.session_id = s.id
|
||||
AND 'client' = ANY(t.visible_to)
|
||||
)
|
||||
ORDER BY s.ended_at ASC
|
||||
LIMIT $2
|
||||
""",
|
||||
stale_seconds,
|
||||
max(1, int(limit)),
|
||||
)
|
||||
sessions: list[InProcSession] = []
|
||||
for row in rows:
|
||||
session_id = str(row["id"])
|
||||
state_row = await conn.fetchrow(
|
||||
"""
|
||||
SELECT stage, turn_seq, effective_openness, rapport_credit, resistance,
|
||||
ideation_stage, turns_in_stage, affect_state
|
||||
FROM app.session_state
|
||||
WHERE session_id = $1::uuid
|
||||
""",
|
||||
session_id,
|
||||
)
|
||||
turn_rows = await conn.fetch(
|
||||
"""
|
||||
SELECT id, seq, speaker, stage, text, text_masked, created_at,
|
||||
llm_provider, model, tokens_in, tokens_out, cost_usd,
|
||||
audio_ref, silence_ms, speech_rate, barge_in, provider_events, visible_to
|
||||
FROM app.turns
|
||||
WHERE session_id = $1::uuid
|
||||
ORDER BY seq
|
||||
""",
|
||||
session_id,
|
||||
)
|
||||
sess = _session_from_rows(row, state_row, turn_rows)
|
||||
if sess is not None:
|
||||
sessions.append(sess)
|
||||
return sessions, True
|
||||
except Exception:
|
||||
require_runtime_fallback_allowed("missing session evaluation recovery")
|
||||
return [], False
|
||||
|
||||
|
||||
async def save_case_worksheet(
|
||||
*,
|
||||
session_id: str,
|
||||
|
|
@ -1411,6 +1813,9 @@ async def save_live_coach_event(
|
|||
client_reply: str | None,
|
||||
suggestion: Any,
|
||||
) -> tuple[dict[str, Any] | None, bool]:
|
||||
cached_remaining = _live_coach_cached_remaining(session_id)
|
||||
if cached_remaining <= 0:
|
||||
raise LiveCoachCreditExhausted("live coach credit exhausted")
|
||||
record = _live_coach_cache_record(
|
||||
session_id=session_id,
|
||||
turn_seq=turn_seq,
|
||||
|
|
@ -1418,32 +1823,42 @@ async def save_live_coach_event(
|
|||
learner_text=learner_text,
|
||||
client_reply=client_reply,
|
||||
suggestion=suggestion,
|
||||
credit_balance=cached_remaining - 1,
|
||||
)
|
||||
if runtime_fallback_allowed():
|
||||
_LIVE_COACH_EVENT_CACHE.setdefault(session_id, []).append(record)
|
||||
try:
|
||||
get_pool()
|
||||
async with acquire(role="learner", user_id=learner_id) as conn:
|
||||
remaining_before = await _live_coach_remaining_for_conn(conn, session_id)
|
||||
if remaining_before <= 0:
|
||||
raise LiveCoachCreditExhausted("live coach credit exhausted")
|
||||
remaining_after = _live_coach_remaining(remaining_before - 1)
|
||||
row = await conn.fetchrow(
|
||||
"""
|
||||
INSERT INTO app.live_coach_events (
|
||||
session_id, turn_seq, stage, learner_text_excerpt,
|
||||
session_id, turn_seq, stage, event_type, credit_delta,
|
||||
credit_balance, reason, learner_text_excerpt,
|
||||
client_reply_excerpt, payload, created_at
|
||||
)
|
||||
VALUES ($1::uuid, $2, $3, $4, $5, $6::jsonb, now())
|
||||
RETURNING id, session_id, turn_seq, stage, learner_text_excerpt,
|
||||
VALUES ($1::uuid, $2, $3, 'use', -1, $4, $5, $6, $7, $8::jsonb, now())
|
||||
RETURNING id, session_id, turn_seq, stage, event_type, credit_delta,
|
||||
credit_balance, reason, learner_text_excerpt,
|
||||
client_reply_excerpt, payload, created_at
|
||||
""",
|
||||
session_id,
|
||||
int(turn_seq),
|
||||
stage,
|
||||
remaining_after,
|
||||
LIVE_COACH_USE_REASON,
|
||||
record.get("learner_text_excerpt"),
|
||||
record.get("client_reply_excerpt"),
|
||||
_model_payload(suggestion),
|
||||
)
|
||||
return (_live_coach_event_from_row(row) if row else None), True
|
||||
except LiveCoachCreditExhausted:
|
||||
raise
|
||||
except Exception:
|
||||
require_runtime_fallback_allowed("live coach event save")
|
||||
_LIVE_COACH_EVENT_CACHE.setdefault(session_id, []).append(record)
|
||||
return record, False
|
||||
|
||||
|
||||
|
|
@ -1464,6 +1879,7 @@ async def list_live_coach_events(
|
|||
client_reply_excerpt, payload, created_at
|
||||
FROM app.live_coach_events
|
||||
WHERE session_id = $1::uuid
|
||||
AND event_type = 'use'
|
||||
ORDER BY created_at ASC, turn_seq ASC
|
||||
""",
|
||||
session_id,
|
||||
|
|
@ -1471,7 +1887,127 @@ async def list_live_coach_events(
|
|||
return [_live_coach_event_from_row(row) for row in rows], True
|
||||
except Exception:
|
||||
require_runtime_fallback_allowed("live coach event list")
|
||||
return [dict(item) for item in _LIVE_COACH_EVENT_CACHE.get(session_id, [])], False
|
||||
return [
|
||||
dict(item)
|
||||
for item in _LIVE_COACH_EVENT_CACHE.get(session_id, [])
|
||||
if item.get("event_type", "use") == "use"
|
||||
], False
|
||||
|
||||
|
||||
async def get_live_coach_quota(
|
||||
session_id: str,
|
||||
principal: Principal,
|
||||
) -> tuple[dict[str, int], bool]:
|
||||
try:
|
||||
get_pool()
|
||||
async with acquire(
|
||||
role=principal.role.value,
|
||||
user_id=principal.user_id,
|
||||
cohort_ids=principal.cohort_ids,
|
||||
) as conn:
|
||||
remaining = await _live_coach_remaining_for_conn(conn, session_id)
|
||||
return _live_coach_quota_payload(remaining), True
|
||||
except Exception:
|
||||
require_runtime_fallback_allowed("live coach quota")
|
||||
return _live_coach_quota_payload(_live_coach_cached_remaining(session_id)), False
|
||||
|
||||
|
||||
async def list_live_coach_credit_events(
|
||||
session_id: str,
|
||||
principal: Principal,
|
||||
) -> tuple[list[dict[str, Any]], bool]:
|
||||
try:
|
||||
get_pool()
|
||||
async with acquire(
|
||||
role=principal.role.value,
|
||||
user_id=principal.user_id,
|
||||
cohort_ids=principal.cohort_ids,
|
||||
) as conn:
|
||||
rows = await conn.fetch(
|
||||
"""
|
||||
SELECT id, session_id, turn_seq, stage, event_type, credit_delta,
|
||||
credit_balance, reason, created_at
|
||||
FROM app.live_coach_events
|
||||
WHERE session_id = $1::uuid
|
||||
ORDER BY created_at ASC, turn_seq ASC
|
||||
""",
|
||||
session_id,
|
||||
)
|
||||
return [_live_coach_credit_event_from_row(row) for row in rows], True
|
||||
except Exception:
|
||||
require_runtime_fallback_allowed("live coach credit event list")
|
||||
return [
|
||||
_live_coach_credit_event_from_record(item)
|
||||
for item in _LIVE_COACH_EVENT_CACHE.get(session_id, [])
|
||||
], False
|
||||
|
||||
|
||||
async def record_live_coach_recharge(
|
||||
*,
|
||||
session_id: str,
|
||||
learner_id: str,
|
||||
turn_seq: int,
|
||||
stage: str,
|
||||
reason: str = LIVE_COACH_RECHARGE_REASON,
|
||||
) -> tuple[dict[str, Any] | None, bool]:
|
||||
try:
|
||||
get_pool()
|
||||
async with acquire(role="learner", user_id=learner_id) as conn:
|
||||
existing = await conn.fetchval(
|
||||
"""
|
||||
SELECT id
|
||||
FROM app.live_coach_events
|
||||
WHERE session_id = $1::uuid
|
||||
AND turn_seq = $2
|
||||
AND event_type = 'recharge'
|
||||
LIMIT 1
|
||||
""",
|
||||
session_id,
|
||||
int(turn_seq),
|
||||
)
|
||||
if existing:
|
||||
return None, True
|
||||
remaining_before = await _live_coach_remaining_for_conn(conn, session_id)
|
||||
if remaining_before >= LIVE_COACH_MAX_CREDITS:
|
||||
return None, True
|
||||
remaining_after = _live_coach_remaining(remaining_before + 1)
|
||||
row = await conn.fetchrow(
|
||||
"""
|
||||
INSERT INTO app.live_coach_events (
|
||||
session_id, turn_seq, stage, event_type, credit_delta,
|
||||
credit_balance, reason, payload, created_at
|
||||
)
|
||||
VALUES ($1::uuid, $2, $3, 'recharge', 1, $4, $5, '{}'::jsonb, now())
|
||||
RETURNING id, session_id, turn_seq, stage, event_type, credit_delta,
|
||||
credit_balance, reason, created_at
|
||||
""",
|
||||
session_id,
|
||||
int(turn_seq),
|
||||
stage,
|
||||
remaining_after,
|
||||
reason,
|
||||
)
|
||||
return (_live_coach_credit_event_from_row(row) if row else None), True
|
||||
except Exception:
|
||||
require_runtime_fallback_allowed("live coach recharge")
|
||||
events = _LIVE_COACH_EVENT_CACHE.setdefault(session_id, [])
|
||||
if any(
|
||||
event.get("event_type") == "recharge" and int(event.get("turn_seq") or 0) == int(turn_seq)
|
||||
for event in events
|
||||
):
|
||||
return None, False
|
||||
remaining_before = _live_coach_cached_remaining(session_id)
|
||||
if remaining_before >= LIVE_COACH_MAX_CREDITS:
|
||||
return None, False
|
||||
record = _live_coach_recharge_cache_record(
|
||||
session_id=session_id,
|
||||
turn_seq=turn_seq,
|
||||
stage=stage,
|
||||
credit_balance=remaining_before + 1,
|
||||
reason=reason,
|
||||
)
|
||||
events.append(record)
|
||||
return _live_coach_credit_event_from_record(record), False
|
||||
|
||||
|
||||
def _review_status_from_row(row: Any) -> dict[str, Any]:
|
||||
|
|
@ -1883,6 +2419,7 @@ def _session_from_rows(row, state_row, turn_rows: Iterable) -> InProcSession | N
|
|||
turns=[_turn_from_row(turn_row) for turn_row in turn_rows],
|
||||
ended=ended_at is not None,
|
||||
prev_rapport_credit=float(row["prev_rapport_credit"] or 0.0),
|
||||
learner_label=_clean_text(_row_value(row, "learner_label")),
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -2063,6 +2600,12 @@ async def load_session(
|
|||
s.id, s.runtime_case_id, s.case_id, s.learner_id, s.persona_code,
|
||||
s.session_no, s.theory_mode, s.started_at, s.ended_at,
|
||||
s.prev_rapport_credit,
|
||||
COALESCE(
|
||||
NULLIF(learner.display_name, ''),
|
||||
NULLIF(learner.nickname, ''),
|
||||
NULLIF(learner.email, ''),
|
||||
s.learner_id::text
|
||||
) AS learner_label,
|
||||
pc.persona_id AS card_persona_id,
|
||||
pc.code AS card_code,
|
||||
pc.version AS card_version,
|
||||
|
|
@ -2086,6 +2629,7 @@ async def load_session(
|
|||
LEFT JOIN app.persona_card pc
|
||||
ON pc.persona_id = s.persona_id
|
||||
AND pc.version = s.persona_version
|
||||
LEFT JOIN app.app_user learner ON learner.user_id = s.learner_id
|
||||
WHERE s.id = $1::uuid
|
||||
""",
|
||||
session_id,
|
||||
|
|
@ -2490,15 +3034,50 @@ async def end_session(sess: InProcSession, carry: memory.CarryOver) -> bool:
|
|||
return False
|
||||
|
||||
|
||||
async def list_sessions(
|
||||
RECENT_SESSION_LIST_LIMIT = 100
|
||||
|
||||
|
||||
async def list_recent_sessions(
|
||||
principal: Principal,
|
||||
*,
|
||||
include_turn_evaluation: bool = False,
|
||||
) -> tuple[list[InProcSession], bool]:
|
||||
return await _list_sessions(
|
||||
principal,
|
||||
include_turn_evaluation=include_turn_evaluation,
|
||||
session_limit=RECENT_SESSION_LIST_LIMIT,
|
||||
audit_access="list_recent_sessions",
|
||||
)
|
||||
|
||||
|
||||
async def list_all_sessions(
|
||||
principal: Principal,
|
||||
*,
|
||||
include_turn_evaluation: bool = False,
|
||||
) -> tuple[list[InProcSession], bool]:
|
||||
return await _list_sessions(
|
||||
principal,
|
||||
include_turn_evaluation=include_turn_evaluation,
|
||||
session_limit=None,
|
||||
audit_access="list_all_sessions",
|
||||
)
|
||||
|
||||
|
||||
async def _list_sessions(
|
||||
principal: Principal,
|
||||
*,
|
||||
include_turn_evaluation: bool,
|
||||
session_limit: int | None,
|
||||
audit_access: str,
|
||||
) -> tuple[list[InProcSession], bool]:
|
||||
try:
|
||||
get_pool()
|
||||
learner_filter = "WHERE s.learner_id = $1::uuid" if principal.role.value == "learner" else ""
|
||||
query_args = [principal.user_id] if principal.role.value == "learner" else []
|
||||
query_args: list[object] = [principal.user_id] if principal.role.value == "learner" else []
|
||||
limit_clause = ""
|
||||
if session_limit is not None:
|
||||
query_args.append(max(1, int(session_limit)))
|
||||
limit_clause = f"LIMIT ${len(query_args)}"
|
||||
async with acquire(
|
||||
role=principal.role.value,
|
||||
user_id=principal.user_id,
|
||||
|
|
@ -2510,6 +3089,12 @@ async def list_sessions(
|
|||
s.id, s.runtime_case_id, s.case_id, s.learner_id, s.persona_code,
|
||||
s.session_no, s.theory_mode, s.started_at, s.ended_at,
|
||||
s.prev_rapport_credit,
|
||||
COALESCE(
|
||||
NULLIF(learner.display_name, ''),
|
||||
NULLIF(learner.nickname, ''),
|
||||
NULLIF(learner.email, ''),
|
||||
s.learner_id::text
|
||||
) AS learner_label,
|
||||
pc.persona_id AS card_persona_id,
|
||||
pc.code AS card_code,
|
||||
pc.version AS card_version,
|
||||
|
|
@ -2533,47 +3118,58 @@ async def list_sessions(
|
|||
LEFT JOIN app.persona_card pc
|
||||
ON pc.persona_id = s.persona_id
|
||||
AND pc.version = s.persona_version
|
||||
LEFT JOIN app.app_user learner ON learner.user_id = s.learner_id
|
||||
{learner_filter}
|
||||
ORDER BY s.started_at DESC
|
||||
LIMIT 100
|
||||
{limit_clause}
|
||||
""",
|
||||
*query_args,
|
||||
)
|
||||
sessions: list[InProcSession] = []
|
||||
for row in rows:
|
||||
session_id = str(row["id"])
|
||||
state_row = await conn.fetchrow(
|
||||
# N+1 제거: 세션별 state/turns fetch 루프(1+2N 왕복) 대신 id 집합으로 한 번씩 배치 조회.
|
||||
session_ids = [str(row["id"]) for row in rows]
|
||||
states_by_id: dict[str, Any] = {}
|
||||
turns_by_id: dict[str, list] = {}
|
||||
if session_ids:
|
||||
state_rows = await conn.fetch(
|
||||
"""
|
||||
SELECT stage, turn_seq, effective_openness, rapport_credit, resistance,
|
||||
SELECT session_id, stage, turn_seq, effective_openness, rapport_credit, resistance,
|
||||
ideation_stage, turns_in_stage, affect_state
|
||||
FROM app.session_state
|
||||
WHERE session_id = $1::uuid
|
||||
WHERE session_id = ANY($1::uuid[])
|
||||
""",
|
||||
session_id,
|
||||
session_ids,
|
||||
)
|
||||
turn_rows = await conn.fetch(
|
||||
states_by_id = {str(state_row["session_id"]): state_row for state_row in state_rows}
|
||||
turn_rows_all = await conn.fetch(
|
||||
"""
|
||||
SELECT id, seq, speaker, stage, text, text_masked, created_at,
|
||||
SELECT session_id, id, seq, speaker, stage, text, text_masked, created_at,
|
||||
llm_provider, model, tokens_in, tokens_out, cost_usd,
|
||||
audio_ref, silence_ms, speech_rate, barge_in, provider_events, visible_to
|
||||
FROM app.turns
|
||||
WHERE session_id = $1::uuid
|
||||
ORDER BY seq
|
||||
WHERE session_id = ANY($1::uuid[])
|
||||
ORDER BY session_id, seq
|
||||
""",
|
||||
session_id,
|
||||
session_ids,
|
||||
)
|
||||
for turn_row in turn_rows_all:
|
||||
turns_by_id.setdefault(str(turn_row["session_id"]), []).append(turn_row)
|
||||
sessions: list[InProcSession] = []
|
||||
for row in rows:
|
||||
session_id = str(row["id"])
|
||||
sess = _session_from_rows(
|
||||
row, states_by_id.get(session_id), turns_by_id.get(session_id, [])
|
||||
)
|
||||
sess = _session_from_rows(row, state_row, turn_rows)
|
||||
if sess is not None:
|
||||
if include_turn_evaluation:
|
||||
await _hydrate_session_turn_evaluations(sess)
|
||||
sessions.append(sess)
|
||||
if include_turn_evaluation and sessions:
|
||||
await _hydrate_sessions_turn_evaluations(sessions)
|
||||
await _record_session_read_audit(
|
||||
conn,
|
||||
principal,
|
||||
target_kind="session_list",
|
||||
target_id="sessions",
|
||||
detail={
|
||||
"access": "list_sessions",
|
||||
"access": audit_access,
|
||||
"role": principal.role.value,
|
||||
"result_count": len(sessions),
|
||||
},
|
||||
|
|
@ -2602,9 +3198,16 @@ async def list_safety_alerts(
|
|||
SELECT
|
||||
se.id, se.session_id, se.trigger_type, se.ko_risk_level,
|
||||
se.escalated, se.detail, se.created_at,
|
||||
s.learner_id, s.persona_code, s.session_no
|
||||
s.learner_id, s.persona_code, s.session_no,
|
||||
COALESCE(
|
||||
NULLIF(learner.display_name, ''),
|
||||
NULLIF(learner.nickname, ''),
|
||||
NULLIF(learner.email, ''),
|
||||
s.learner_id::text
|
||||
) AS learner_label
|
||||
FROM app.safety_events se
|
||||
LEFT JOIN app.sessions s ON s.id = se.session_id
|
||||
LEFT JOIN app.app_user learner ON learner.user_id = s.learner_id
|
||||
WHERE se.escalated = TRUE
|
||||
ORDER BY se.created_at DESC
|
||||
LIMIT $1
|
||||
|
|
@ -2616,7 +3219,8 @@ async def list_safety_alerts(
|
|||
"id": str(row["id"]),
|
||||
"session_id": str(row["session_id"]),
|
||||
"learner_id": str(row["learner_id"] or ""),
|
||||
"learner_label": _learner_label_from_id(str(row["learner_id"] or "")),
|
||||
"learner_label": _clean_text(row["learner_label"])
|
||||
or _learner_label_from_id(str(row["learner_id"] or "")),
|
||||
"persona_code": str(row["persona_code"] or ""),
|
||||
"session_no": int(row["session_no"] or 0),
|
||||
"trigger_type": str(row["trigger_type"] or "crisis"),
|
||||
|
|
|
|||
|
|
@ -11,12 +11,12 @@ import re
|
|||
from collections import Counter
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from typing import Literal, Optional, cast
|
||||
from typing import Any, Literal, Optional, cast
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from .config import settings
|
||||
from .services import session_metrics
|
||||
from .services import guardrail, session_metrics
|
||||
from .stage_contract import (
|
||||
ReviewPhaseKey,
|
||||
StageLabel,
|
||||
|
|
@ -31,6 +31,10 @@ WorksheetItemSpec = tuple[str, str, list[str], WorksheetSpeaker | None]
|
|||
WorksheetSectionSpec = tuple[str, str, list[WorksheetItemSpec]]
|
||||
|
||||
LEARNER_VISIBLE_AI_ROLE = "counselor"
|
||||
MISSING_SESSION_EVALUATION_GRACE_SECONDS = 30.0
|
||||
MISSING_SESSION_EVALUATION_ERROR = (
|
||||
"회기말 평가가 제한 시간 이후에도 저장되지 않았습니다. AI 평가 재시도가 필요합니다."
|
||||
)
|
||||
|
||||
|
||||
class LearnerSessionSummary(BaseModel):
|
||||
|
|
@ -816,6 +820,46 @@ def _review_summary_from_evaluation(
|
|||
return prefix + (details if details else "아래 코칭 항목은 저장된 축어록과 평가 AI 결과를 기준으로 합니다.")
|
||||
|
||||
|
||||
def _session_evaluation_timeout_seconds() -> float:
|
||||
configured = float(settings.session_evaluation_timeout or settings.engine_timeout)
|
||||
return max(1.0, configured)
|
||||
|
||||
|
||||
def _missing_session_evaluation_record(
|
||||
sess: InProcSession,
|
||||
*,
|
||||
has_visible_turns: bool,
|
||||
now_ts: float,
|
||||
) -> dict[str, object] | None:
|
||||
if not sess.ended or not has_visible_turns or sess.ended_at is None:
|
||||
return None
|
||||
stale_after = _session_evaluation_timeout_seconds() + MISSING_SESSION_EVALUATION_GRACE_SECONDS
|
||||
if now_ts - sess.ended_at < stale_after:
|
||||
return None
|
||||
return {
|
||||
"status": "error",
|
||||
"source": "read_model",
|
||||
"scope": "session_end",
|
||||
"stage": stage_label(sess.state.stage),
|
||||
"payload": {"error": MISSING_SESSION_EVALUATION_ERROR},
|
||||
"error": MISSING_SESSION_EVALUATION_ERROR,
|
||||
"updated_at": iso(now_ts),
|
||||
}
|
||||
|
||||
|
||||
def missing_session_evaluation_record(
|
||||
sess: InProcSession,
|
||||
*,
|
||||
has_visible_turns: bool,
|
||||
now_ts: float,
|
||||
) -> dict[str, object] | None:
|
||||
return _missing_session_evaluation_record(
|
||||
sess,
|
||||
has_visible_turns=has_visible_turns,
|
||||
now_ts=now_ts,
|
||||
)
|
||||
|
||||
|
||||
def _next_line_from_evaluation(payload: dict[str, object]) -> str | None:
|
||||
alternatives = payload.get("alternative_utterances")
|
||||
if not isinstance(alternatives, list):
|
||||
|
|
@ -1025,7 +1069,26 @@ def _evaluation_payload(record: dict[str, object] | None) -> dict[str, object]:
|
|||
if not record:
|
||||
return {}
|
||||
payload = record.get("payload")
|
||||
return payload if isinstance(payload, dict) else {}
|
||||
if not isinstance(payload, dict):
|
||||
return {}
|
||||
masked = _mask_payload_text_values(payload)
|
||||
return masked if isinstance(masked, dict) else {}
|
||||
|
||||
|
||||
def _mask_payload_text(value: object) -> str:
|
||||
return guardrail.mask_pii(str(value or "")).text_masked
|
||||
|
||||
|
||||
def _mask_payload_text_values(value: Any) -> Any:
|
||||
if isinstance(value, str):
|
||||
return _mask_payload_text(value)
|
||||
if isinstance(value, dict):
|
||||
return {str(key): _mask_payload_text_values(child) for key, child in value.items()}
|
||||
if isinstance(value, list):
|
||||
return [_mask_payload_text_values(child) for child in value]
|
||||
if isinstance(value, tuple):
|
||||
return [_mask_payload_text_values(child) for child in value]
|
||||
return value
|
||||
|
||||
|
||||
_TECHNIQUE_KIND_BY_CATEGORY = {
|
||||
|
|
@ -1091,6 +1154,17 @@ def _review_note_from_turn_eval(
|
|||
if not isinstance(ev, dict):
|
||||
return None
|
||||
quote = _review_quote_excerpt(learner_text)
|
||||
error_text = str(ev.get("error") or "").strip()
|
||||
if error_text:
|
||||
return ReviewNote(
|
||||
author="평가 AI",
|
||||
tone="warn",
|
||||
title="턴 평가 실패",
|
||||
body=_review_note_body_markdown(
|
||||
f"이 발화의 fast-loop 평가를 완료하지 못했습니다.\n\n사유: {error_text}"
|
||||
),
|
||||
quote=quote,
|
||||
)
|
||||
dev = ev.get("intent_deviation")
|
||||
if isinstance(dev, dict):
|
||||
dimension = str(dev.get("dimension") or "").strip()
|
||||
|
|
@ -1261,7 +1335,14 @@ def build_session_review(read_input: SessionReviewReadInput) -> SessionReviewRes
|
|||
if duration_seconds > 0:
|
||||
axis.append(_offset_label(duration_seconds))
|
||||
|
||||
now_ts = read_input.now_ts or datetime.now().timestamp()
|
||||
evaluation_record = read_input.evaluation_record
|
||||
if evaluation_record is None and not hidden_turns:
|
||||
evaluation_record = _missing_session_evaluation_record(
|
||||
sess,
|
||||
has_visible_turns=bool(visible_turns),
|
||||
now_ts=now_ts,
|
||||
)
|
||||
evaluation_payload = {} if hidden_turns else _evaluation_payload(evaluation_record)
|
||||
evaluation_status = (
|
||||
"" if hidden_turns else str(evaluation_record.get("status") or "") if evaluation_record else ""
|
||||
|
|
|
|||
|
|
@ -68,11 +68,7 @@ class InProcSession:
|
|||
turns: list[TurnRecord] = field(default_factory=list)
|
||||
ended: bool = False
|
||||
prev_rapport_credit: float = 0.0 # carry-over delta 계산용
|
||||
|
||||
def recent_turns(self, k: int = 6, visible_to: str | None = None) -> list[dict[str, str]]:
|
||||
"""최근 K턴 버퍼(L6 직전 맥락). 마스킹본 사용."""
|
||||
turns = self.turns if visible_to is None else self.turns_visible_to(visible_to)
|
||||
return [{"speaker": t.speaker, "text": t.text_masked} for t in turns[-k:]]
|
||||
learner_label: str | None = None
|
||||
|
||||
def turns_visible_to(self, role: str) -> list[TurnRecord]:
|
||||
return [turn for turn in self.turns if turn.is_visible_to(role)]
|
||||
|
|
@ -81,6 +77,10 @@ class InProcSession:
|
|||
turns = self.turns if visible_to is None else self.turns_visible_to(visible_to)
|
||||
return [{"speaker": t.speaker, "text": t.text_masked} for t in turns]
|
||||
|
||||
def recent_turns(self, k: int = 6, visible_to: str | None = None) -> list[dict[str, str]]:
|
||||
"""최근 K턴 버퍼(L6 직전 맥락). 마스킹본은 masked_turns 단일 소유, 여기선 꼬리 K개만."""
|
||||
return self.masked_turns(visible_to)[-k:]
|
||||
|
||||
|
||||
class SessionStore:
|
||||
"""in-memory 세션 저장소. DB degraded 시 SoR 대용."""
|
||||
|
|
|
|||
|
|
@ -30,10 +30,11 @@ TAXONOMY_VERSION = "1.0.0" # 라벨 코드 불변 보장 버전. 새 라벨 추
|
|||
|
||||
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 1. 회기 단계 (stage) — 결정론 상태머신이 소유 (MASTERPLAN §2.2)
|
||||
# 1. 회기 단계 (stage) — 라벨 enum 단일 정의(SoT). 전이 로직은 상태머신 소유(MASTERPLAN §2.2).
|
||||
# services.state_machine 이 이 Stage 를 re-export 한다(중복 정의 제거).
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
class Stage(str, Enum):
|
||||
"""상담 회기 단계. FastAPI 백엔드가 결정론적으로 전이(LLM 아님)."""
|
||||
"""상담 회기 단계 라벨(SoT). 결정론 전이는 services.state_machine 이 소유(LLM 아님)."""
|
||||
|
||||
RAPPORT = "라포" # 라포 형성 — 안전감·관계 구축, 비밀보장 구조화
|
||||
EXPLORE = "탐색" # 호소문제·정서·인지·위험요인 탐색
|
||||
|
|
@ -51,6 +52,11 @@ class Speaker(str, Enum):
|
|||
CLIENT = "client" # 내담자(가상내담자 AI)
|
||||
|
||||
|
||||
def speaker_ko_label(speaker: str | None) -> str:
|
||||
"""화자 코드 → 프롬프트용 한글 라벨(SoT). counselor→상담자, 그 외→내담자."""
|
||||
return "상담자" if speaker == Speaker.COUNSELOR.value else "내담자"
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 3. 기법 군집 (TechniqueCategory) — 위계 상위층
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
|
|
@ -273,6 +279,7 @@ __all__ = [
|
|||
"TAXONOMY_VERSION",
|
||||
"Stage",
|
||||
"Speaker",
|
||||
"speaker_ko_label",
|
||||
"TechniqueCategory",
|
||||
"Technique",
|
||||
"TECHNIQUE_CATEGORY",
|
||||
|
|
|
|||
|
|
@ -426,6 +426,22 @@ class AuthProviderScaffoldTest(unittest.IsolatedAsyncioTestCase):
|
|||
self.assertTrue(principal.can_access_role(Role.TEACHER))
|
||||
self.assertTrue(principal.can_access_role(Role.ADMIN))
|
||||
|
||||
async def test_admin_role_can_enter_admin_console_even_with_stale_access_flag(self) -> None:
|
||||
principal = Principal(
|
||||
user_id="00000000-0000-0000-0000-000000000604",
|
||||
role=Role.ADMIN,
|
||||
email="stale-admin@twentyoz.kr",
|
||||
display_name="Stale Admin",
|
||||
admin_access=False,
|
||||
super_admin=False,
|
||||
)
|
||||
|
||||
admin_checker = deps.require_admin_access()
|
||||
admin_view = await admin_checker(principal)
|
||||
|
||||
self.assertEqual(admin_view.role, Role.ADMIN)
|
||||
self.assertTrue(principal.can_access_role(Role.ADMIN))
|
||||
|
||||
async def test_admin_access_flag_without_admin_role_does_not_enter_learner_space(self) -> None:
|
||||
principal = Principal(
|
||||
user_id="00000000-0000-0000-0000-000000000603",
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
import importlib.util
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
|
@ -12,11 +14,34 @@ from app.services.dataset_export import (
|
|||
intraclass_correlation,
|
||||
scan_for_pii,
|
||||
sha256_file,
|
||||
validate_manifest_gate,
|
||||
write_jsonl,
|
||||
)
|
||||
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[3]
|
||||
EXPORT_SCRIPT_PATH = REPO_ROOT / "scripts" / "export-recursive-dataset.py"
|
||||
|
||||
|
||||
def load_export_script():
|
||||
spec = importlib.util.spec_from_file_location("export_recursive_dataset", EXPORT_SCRIPT_PATH)
|
||||
assert spec is not None and spec.loader is not None
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[spec.name] = module
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
class DatasetExportTests(unittest.TestCase):
|
||||
def test_exporter_query_requires_consent_and_client_visible_turns(self) -> None:
|
||||
exporter = load_export_script()
|
||||
query = exporter.TURN_QUERY
|
||||
|
||||
self.assertIn("u.consent_at IS NOT NULL", query)
|
||||
self.assertIn("'client' = ANY(t.visible_to)", query)
|
||||
self.assertNotRegex(query, r"\bt\.text\s*(?:,|AS\b)")
|
||||
self.assertNotIn("sc.text", query)
|
||||
|
||||
def test_build_dataset_record_uses_masked_text_and_pseudonymous_keys(self) -> None:
|
||||
keys = ExportKeyMaps()
|
||||
record = build_dataset_record(
|
||||
|
|
@ -45,6 +70,32 @@ class DatasetExportTests(unittest.TestCase):
|
|||
self.assertNotIn("22222222-2222-2222-2222-222222222222", blob)
|
||||
self.assertNotIn("raw text should never be exported", blob)
|
||||
|
||||
def test_build_dataset_record_drops_supervisor_comment_text(self) -> None:
|
||||
record = build_dataset_record(
|
||||
{
|
||||
"session_id": "11111111-1111-1111-1111-111111111111",
|
||||
"learner_id": "22222222-2222-2222-2222-222222222222",
|
||||
"persona_code": "P1",
|
||||
"stage": "rapport",
|
||||
"speaker": "client",
|
||||
"text_masked": "마스킹된 발화입니다.",
|
||||
"supervisor_comments": [
|
||||
{
|
||||
"kind": "note",
|
||||
"text": "김서연 010-1234-5678 raw supervisor note",
|
||||
"intent_deviation": {"severity": "low"},
|
||||
}
|
||||
],
|
||||
},
|
||||
item_index=1,
|
||||
export_manifest_id="phase3-rl-seed-test",
|
||||
keys=ExportKeyMaps(),
|
||||
)
|
||||
|
||||
self.assertEqual(record["supervisor_comments"], [{"kind": "note", "intent_deviation": {"severity": "low"}}])
|
||||
self.assertNotIn("김서연", str(record))
|
||||
self.assertNotIn("010-1234-5678", str(record))
|
||||
|
||||
def test_pii_scan_detects_identifiers_without_raw_samples(self) -> None:
|
||||
findings = scan_for_pii(
|
||||
{
|
||||
|
|
@ -89,6 +140,27 @@ class DatasetExportTests(unittest.TestCase):
|
|||
agreement={"kappa": 0.59, "icc": 0.74, "gold_status": "not_gold"},
|
||||
)
|
||||
|
||||
def test_manifest_gate_requires_withdrawal_and_recursive_consent_scope(self) -> None:
|
||||
manifest = {
|
||||
"export_status": APPROVED_EXPORT_STATUS,
|
||||
"pii_scan": {"status": "pass"},
|
||||
"agreement": {"kappa": 0.70, "icc": 0.75},
|
||||
"selection_criteria": {"include_withdrawn": True},
|
||||
"consent_scope": {"allowed_uses": ["education_quality_review"]},
|
||||
"approvals": {
|
||||
"data_steward": "steward",
|
||||
"legal_or_privacy_reviewer": "privacy",
|
||||
"technical_operator": "operator",
|
||||
"approved_at": "2026-07-01T00:00:00Z",
|
||||
},
|
||||
}
|
||||
|
||||
with self.assertRaisesRegex(
|
||||
ValueError,
|
||||
"include_withdrawn must be false; recursive_learning_seed consent scope is required",
|
||||
):
|
||||
validate_manifest_gate(manifest)
|
||||
|
||||
def test_jsonl_hash_manifest_dry_run(self) -> None:
|
||||
record = {
|
||||
"schema": "phase3_dataset_item_v1",
|
||||
|
|
|
|||
|
|
@ -109,6 +109,38 @@ class ReevaluateSessionRouteTest(unittest.IsolatedAsyncioTestCase):
|
|||
|
||||
self.assertEqual(evaluate_session.await_args.kwargs["theory_mode"], "cbt")
|
||||
|
||||
async def test_reevaluate_session_uses_one_based_masked_turn_seq(self) -> None:
|
||||
session_id = "00000000-0000-0000-0000-00000000e222"
|
||||
sess = _session_stub()
|
||||
result = evaluator.SessionEvaluation(
|
||||
session_id=session_id,
|
||||
stage="정리",
|
||||
scope="session_end",
|
||||
turns_evaluated=2,
|
||||
)
|
||||
|
||||
with (
|
||||
patch.object(eval_routes, "_load_session_or_404", AsyncMock(return_value=sess)),
|
||||
patch.object(
|
||||
eval_routes.evaluator,
|
||||
"evaluate_session",
|
||||
AsyncMock(return_value=result),
|
||||
) as evaluate_session,
|
||||
patch.object(
|
||||
eval_routes.session_persistence,
|
||||
"save_session_evaluation",
|
||||
AsyncMock(return_value=True),
|
||||
),
|
||||
):
|
||||
await eval_routes.reevaluate_session(
|
||||
session_id,
|
||||
eval_routes.ReevaluateRequest(scope="session_end"),
|
||||
_teacher_principal(),
|
||||
)
|
||||
|
||||
masked_turns = evaluate_session.await_args.kwargs["masked_turns"]
|
||||
self.assertEqual([turn["seq"] for turn in masked_turns], [1, 2])
|
||||
|
||||
async def test_reevaluate_turn_preserves_session_theory_mode(self) -> None:
|
||||
session_id = "00000000-0000-0000-0000-00000000e222"
|
||||
sess = _session_stub(theory_mode="cbt", theory_target=["humanistic"], with_turns=True)
|
||||
|
|
@ -121,6 +153,11 @@ class ReevaluateSessionRouteTest(unittest.IsolatedAsyncioTestCase):
|
|||
"evaluate_turn",
|
||||
AsyncMock(return_value=result),
|
||||
) as evaluate_turn,
|
||||
patch.object(
|
||||
eval_routes.session_persistence,
|
||||
"replace_turn_evaluation",
|
||||
AsyncMock(return_value=True),
|
||||
) as replace_turn_evaluation,
|
||||
):
|
||||
await eval_routes.reevaluate_turn(
|
||||
session_id,
|
||||
|
|
@ -130,6 +167,119 @@ class ReevaluateSessionRouteTest(unittest.IsolatedAsyncioTestCase):
|
|||
|
||||
ctx = evaluate_turn.await_args.args[0]
|
||||
self.assertEqual(ctx.theory_mode, "cbt")
|
||||
replace_turn_evaluation.assert_awaited_once()
|
||||
self.assertEqual(
|
||||
replace_turn_evaluation.await_args.kwargs["turn_id"],
|
||||
"00000000-0000-0000-0000-00000000a001",
|
||||
)
|
||||
|
||||
async def test_reevaluate_turn_persists_result_for_review_hydration(self) -> None:
|
||||
session_id = "00000000-0000-0000-0000-00000000e222"
|
||||
sess = _session_stub(theory_mode="cbt", with_turns=True)
|
||||
result = evaluator.TurnEvaluation(
|
||||
turn_seq=1,
|
||||
stage="정리",
|
||||
theory_mode="cbt",
|
||||
appropriateness="pos",
|
||||
rapport_signal=0.4,
|
||||
techniques=[
|
||||
evaluator.TechniqueTag(
|
||||
code="empathy",
|
||||
label_ko="공감",
|
||||
category="relational",
|
||||
rationale="감정을 반영했다.",
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
with (
|
||||
patch.object(eval_routes, "_load_session_or_404", AsyncMock(return_value=sess)),
|
||||
patch.object(
|
||||
eval_routes.evaluator,
|
||||
"evaluate_turn",
|
||||
AsyncMock(return_value=result),
|
||||
),
|
||||
patch.object(
|
||||
eval_routes.session_persistence,
|
||||
"replace_turn_evaluation",
|
||||
AsyncMock(return_value=True),
|
||||
) as replace_turn_evaluation,
|
||||
):
|
||||
response = await eval_routes.reevaluate_turn(
|
||||
session_id,
|
||||
eval_routes.TurnReevaluateRequest(turn_seq=1),
|
||||
_teacher_principal(),
|
||||
)
|
||||
|
||||
self.assertEqual(response.appropriateness, "pos")
|
||||
saved_payload = replace_turn_evaluation.await_args.kwargs["evaluation"]
|
||||
self.assertEqual(saved_payload["turn_seq"], 1)
|
||||
self.assertEqual(saved_payload["theory_mode"], "cbt")
|
||||
self.assertEqual(saved_payload["rapport_signal"], 0.4)
|
||||
self.assertEqual(saved_payload["techniques"][0]["label_ko"], "공감")
|
||||
self.assertEqual(sess.turns[0].evaluation, saved_payload)
|
||||
|
||||
async def test_reevaluate_turn_raises_when_result_cannot_be_saved(self) -> None:
|
||||
session_id = "00000000-0000-0000-0000-00000000e222"
|
||||
sess = _session_stub(with_turns=True)
|
||||
result = evaluator.TurnEvaluation(turn_seq=1, stage="정리")
|
||||
|
||||
with (
|
||||
patch.object(eval_routes, "_load_session_or_404", AsyncMock(return_value=sess)),
|
||||
patch.object(
|
||||
eval_routes.evaluator,
|
||||
"evaluate_turn",
|
||||
AsyncMock(return_value=result),
|
||||
),
|
||||
patch.object(
|
||||
eval_routes.session_persistence,
|
||||
"replace_turn_evaluation",
|
||||
AsyncMock(return_value=False),
|
||||
),
|
||||
):
|
||||
with self.assertRaises(HTTPException) as raised:
|
||||
await eval_routes.reevaluate_turn(
|
||||
session_id,
|
||||
eval_routes.TurnReevaluateRequest(turn_seq=1),
|
||||
_teacher_principal(),
|
||||
)
|
||||
|
||||
self.assertEqual(raised.exception.status_code, 503)
|
||||
self.assertIn("could not be saved", raised.exception.detail)
|
||||
|
||||
async def test_reevaluate_turn_saves_error_marker_then_raises(self) -> None:
|
||||
session_id = "00000000-0000-0000-0000-00000000e222"
|
||||
sess = _session_stub(with_turns=True)
|
||||
result = evaluator.TurnEvaluation(
|
||||
turn_seq=1,
|
||||
stage="정리",
|
||||
error="engine_error",
|
||||
)
|
||||
|
||||
with (
|
||||
patch.object(eval_routes, "_load_session_or_404", AsyncMock(return_value=sess)),
|
||||
patch.object(
|
||||
eval_routes.evaluator,
|
||||
"evaluate_turn",
|
||||
AsyncMock(return_value=result),
|
||||
),
|
||||
patch.object(
|
||||
eval_routes.session_persistence,
|
||||
"replace_turn_evaluation",
|
||||
AsyncMock(return_value=True),
|
||||
) as replace_turn_evaluation,
|
||||
):
|
||||
with self.assertRaises(HTTPException) as raised:
|
||||
await eval_routes.reevaluate_turn(
|
||||
session_id,
|
||||
eval_routes.TurnReevaluateRequest(turn_seq=1),
|
||||
_teacher_principal(),
|
||||
)
|
||||
|
||||
self.assertEqual(raised.exception.status_code, 503)
|
||||
self.assertEqual(raised.exception.detail, "engine_error")
|
||||
saved_payload = replace_turn_evaluation.await_args.kwargs["evaluation"]
|
||||
self.assertEqual(saved_payload["error"], "engine_error")
|
||||
|
||||
async def test_get_session_evaluation_exposes_status_error_and_durable_flag(self) -> None:
|
||||
session_id = "00000000-0000-0000-0000-00000000e222"
|
||||
|
|
@ -179,12 +329,14 @@ def _session_stub(
|
|||
turns = (
|
||||
[
|
||||
SimpleNamespace(
|
||||
turn_id="00000000-0000-0000-0000-00000000a001",
|
||||
turn_seq=1,
|
||||
speaker="counselor",
|
||||
text="상담자 발화",
|
||||
text_masked="상담자 발화",
|
||||
),
|
||||
SimpleNamespace(
|
||||
turn_id="00000000-0000-0000-0000-00000000a002",
|
||||
turn_seq=2,
|
||||
speaker="client",
|
||||
text="내담자 응답",
|
||||
|
|
@ -196,7 +348,10 @@ def _session_stub(
|
|||
)
|
||||
|
||||
def masked_turns(self):
|
||||
return [{"speaker": "counselor", "text_masked": "상담자 발화"}]
|
||||
return [
|
||||
{"speaker": "counselor", "text_masked": "상담자 발화"},
|
||||
{"speaker": "client", "text_masked": "내담자 응답"},
|
||||
]
|
||||
|
||||
return SessionStub()
|
||||
|
||||
|
|
|
|||
|
|
@ -3,13 +3,14 @@
|
|||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import json
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from .deps import Principal, Role
|
||||
from . import session_persistence
|
||||
from .routes import sessions
|
||||
from .services import evaluator, session_metrics
|
||||
from .services import evaluator, guardrail, session_metrics
|
||||
from .services import persona as persona_service
|
||||
from .services import state_machine
|
||||
from .store import InProcSession
|
||||
|
|
@ -33,8 +34,26 @@ class FakeEvaluationConn:
|
|||
raise AssertionError(f"unexpected fetchval query: {query}")
|
||||
|
||||
|
||||
class FakeMissingEvaluationConn:
|
||||
def __init__(self) -> None:
|
||||
self.fetches: list[tuple[str, tuple[object, ...]]] = []
|
||||
self.fetchrows: list[tuple[str, tuple[object, ...]]] = []
|
||||
|
||||
async def fetch(self, query: str, *args: object) -> list[dict[str, object]]:
|
||||
self.fetches.append((query, args))
|
||||
if "FROM app.sessions s" in query:
|
||||
return [{"id": "11111111-1111-1111-1111-111111111111"}]
|
||||
if "FROM app.turns" in query:
|
||||
return []
|
||||
raise AssertionError(f"unexpected fetch query: {query}")
|
||||
|
||||
async def fetchrow(self, query: str, *args: object) -> dict[str, object]:
|
||||
self.fetchrows.append((query, args))
|
||||
return {}
|
||||
|
||||
|
||||
class FakeAcquire:
|
||||
def __init__(self, conn: FakeEvaluationConn) -> None:
|
||||
def __init__(self, conn) -> None:
|
||||
self.conn = conn
|
||||
|
||||
async def __aenter__(self) -> FakeEvaluationConn:
|
||||
|
|
@ -44,7 +63,7 @@ class FakeAcquire:
|
|||
return None
|
||||
|
||||
|
||||
class EvaluationPersistenceMappingTest(unittest.TestCase):
|
||||
class EvaluationPersistenceMappingTest(unittest.IsolatedAsyncioTestCase):
|
||||
def test_session_metrics_prefers_rehydrated_technique_label_ko(self) -> None:
|
||||
ev = {
|
||||
"techniques": [
|
||||
|
|
@ -137,6 +156,91 @@ class EvaluationPersistenceMappingTest(unittest.TestCase):
|
|||
],
|
||||
)
|
||||
|
||||
def test_turn_evaluation_rows_mask_pii_before_db_persistence(self) -> None:
|
||||
class FakeKoRecognizer:
|
||||
def analyze(self, text: str):
|
||||
spans = []
|
||||
for entity_type, value in (
|
||||
("NAME", "보라별"),
|
||||
("ORG", "미래학교상담연구랩"),
|
||||
):
|
||||
start = text.find(value)
|
||||
if start >= 0:
|
||||
spans.append(guardrail.PiiEntitySpan(entity_type, start, start + len(value)))
|
||||
return spans
|
||||
|
||||
guardrail.set_ko_pii_recognizer(FakeKoRecognizer())
|
||||
self.addCleanup(guardrail.set_ko_pii_recognizer, None)
|
||||
evaluation = {
|
||||
"appropriateness": "warn",
|
||||
"appropriateness_note": "보라별에게 010-1234-5678을 되물었다.",
|
||||
"techniques": [
|
||||
{
|
||||
"code": "reflection",
|
||||
"rationale": "미래학교상담연구랩 이야기를 바로 조언했다.",
|
||||
}
|
||||
],
|
||||
"client_state_read": [
|
||||
{
|
||||
"code": "avoidance",
|
||||
"rationale": "보라별이 기관 미래학교상담연구랩을 피했다.",
|
||||
}
|
||||
],
|
||||
"intent_deviation": {
|
||||
"expected": "보라별의 감정을 확인",
|
||||
"actual": "010-1234-5678 연락처를 재질문",
|
||||
},
|
||||
"alternative_utterances": [
|
||||
{
|
||||
"text": "보라별님, 미래학교상담연구랩 이야기는 잠시 미뤄도 괜찮아요.",
|
||||
"rationale": "010-1234-5678 같은 연락처 재확인을 피함",
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
blob = json.dumps(
|
||||
{
|
||||
"feedback": session_persistence._evaluation_feedback_rows(evaluation),
|
||||
"comments": session_persistence._evaluation_comment_rows(evaluation),
|
||||
"alternatives": session_persistence._evaluation_alternative_rows(evaluation),
|
||||
},
|
||||
ensure_ascii=False,
|
||||
)
|
||||
|
||||
for raw in ("보라별", "미래학교상담연구랩", "010-1234-5678"):
|
||||
self.assertNotIn(raw, blob)
|
||||
for masked in ("[NAME]", "[ORG]", "[PHONE]"):
|
||||
self.assertIn(masked, blob)
|
||||
|
||||
def test_turn_from_legacy_row_remasks_raw_text_fallback(self) -> None:
|
||||
class FakeKoRecognizer:
|
||||
def analyze(self, text: str):
|
||||
start = text.find("보라별")
|
||||
if start < 0:
|
||||
return []
|
||||
return [guardrail.PiiEntitySpan("NAME", start, start + len("보라별"))]
|
||||
|
||||
guardrail.set_ko_pii_recognizer(FakeKoRecognizer())
|
||||
self.addCleanup(guardrail.set_ko_pii_recognizer, None)
|
||||
turn = session_persistence._turn_from_row(
|
||||
{
|
||||
"id": "11111111-1111-1111-1111-111111111111",
|
||||
"seq": 2,
|
||||
"speaker": "client",
|
||||
"stage": "explore",
|
||||
"text": "보라별의 전화는 010-1234-5678입니다.",
|
||||
"text_masked": "",
|
||||
"created_at": None,
|
||||
"visible_to": ("client", "evaluator"),
|
||||
}
|
||||
)
|
||||
|
||||
self.assertEqual(turn.text, turn.text_masked)
|
||||
self.assertNotIn("보라별", turn.text)
|
||||
self.assertNotIn("010-1234-5678", turn.text)
|
||||
self.assertIn("[NAME]", turn.text)
|
||||
self.assertIn("[PHONE]", turn.text)
|
||||
|
||||
def test_session_evaluation_write_from_result_preserves_payload_shape(self) -> None:
|
||||
result = evaluator.SessionEvaluation(
|
||||
session_id="session-1",
|
||||
|
|
@ -159,6 +263,55 @@ class EvaluationPersistenceMappingTest(unittest.TestCase):
|
|||
self.assertNotIn("error", write.payload)
|
||||
self.assertIsNone(write.error)
|
||||
|
||||
def test_session_evaluation_write_masks_deep_payload_before_storage(self) -> None:
|
||||
class FakeKoRecognizer:
|
||||
def analyze(self, text: str):
|
||||
spans = []
|
||||
for entity_type, value in (
|
||||
("NAME", "보라별"),
|
||||
("ORG", "미래학교상담연구랩"),
|
||||
):
|
||||
start = text.find(value)
|
||||
if start >= 0:
|
||||
spans.append(guardrail.PiiEntitySpan(entity_type, start, start + len(value)))
|
||||
return spans
|
||||
|
||||
guardrail.set_ko_pii_recognizer(FakeKoRecognizer())
|
||||
self.addCleanup(guardrail.set_ko_pii_recognizer, None)
|
||||
result = evaluator.SessionEvaluation(
|
||||
session_id="session-privacy",
|
||||
stage="정리",
|
||||
scope="session_end",
|
||||
turns_evaluated=3,
|
||||
strengths=["보라별의 감정을 반영했다."],
|
||||
improvements=["010-1234-5678 같은 연락처 재확인은 피한다."],
|
||||
intent_deviations=[
|
||||
evaluator.IntentDeviation(
|
||||
dimension="pacing",
|
||||
expected="미래학교상담연구랩 이야기를 기다린다",
|
||||
actual="보라별에게 바로 조언했다",
|
||||
severity="moderate",
|
||||
)
|
||||
],
|
||||
supervisor_rationale="보라별의 호소를 요약했다.",
|
||||
supervisor_critique="미래학교상담연구랩과 010-1234-5678을 반복했다.",
|
||||
alternative_utterances=["보라별님, 지금 감정부터 천천히 볼까요?"],
|
||||
error="보라별 평가 경고",
|
||||
)
|
||||
|
||||
write = session_persistence.SessionEvaluationWrite.from_result(
|
||||
session_id="session-privacy",
|
||||
learner_id="learner-privacy",
|
||||
result=result,
|
||||
)
|
||||
blob = json.dumps({"payload": write.payload, "error": write.error}, ensure_ascii=False)
|
||||
|
||||
self.assertEqual(write.status, "error")
|
||||
for raw in ("보라별", "미래학교상담연구랩", "010-1234-5678"):
|
||||
self.assertNotIn(raw, blob)
|
||||
for masked in ("[NAME]", "[ORG]", "[PHONE]"):
|
||||
self.assertIn(masked, blob)
|
||||
|
||||
def test_session_evaluation_write_from_error_preserves_fallback_shape(self) -> None:
|
||||
write = session_persistence.SessionEvaluationWrite.from_error(
|
||||
session_id="session-1",
|
||||
|
|
@ -186,6 +339,65 @@ class EvaluationPersistenceMappingTest(unittest.TestCase):
|
|||
|
||||
self.assertEqual(write.error, "TimeoutError")
|
||||
|
||||
def test_session_evaluation_write_from_error_masks_error_detail(self) -> None:
|
||||
class FakeKoRecognizer:
|
||||
def analyze(self, text: str):
|
||||
start = text.find("보라별")
|
||||
if start < 0:
|
||||
return []
|
||||
return [guardrail.PiiEntitySpan("NAME", start, start + len("보라별"))]
|
||||
|
||||
guardrail.set_ko_pii_recognizer(FakeKoRecognizer())
|
||||
self.addCleanup(guardrail.set_ko_pii_recognizer, None)
|
||||
write = session_persistence.SessionEvaluationWrite.from_error(
|
||||
session_id="session-1",
|
||||
learner_id="learner-1",
|
||||
scope="session_end",
|
||||
stage="explore",
|
||||
error="보라별 처리 중 010-1234-5678 오류",
|
||||
)
|
||||
|
||||
self.assertNotIn("보라별", write.error or "")
|
||||
self.assertNotIn("010-1234-5678", write.error or "")
|
||||
self.assertIn("[NAME]", write.error or "")
|
||||
self.assertIn("[PHONE]", write.error or "")
|
||||
|
||||
async def test_save_session_evaluation_remasks_direct_write_payload(self) -> None:
|
||||
class FakeKoRecognizer:
|
||||
def analyze(self, text: str):
|
||||
start = text.find("보라별")
|
||||
if start < 0:
|
||||
return []
|
||||
return [guardrail.PiiEntitySpan("NAME", start, start + len("보라별"))]
|
||||
|
||||
guardrail.set_ko_pii_recognizer(FakeKoRecognizer())
|
||||
self.addCleanup(guardrail.set_ko_pii_recognizer, None)
|
||||
conn = FakeEvaluationConn()
|
||||
write = session_persistence.SessionEvaluationWrite(
|
||||
session_id="11111111-1111-1111-1111-111111111111",
|
||||
learner_id="22222222-2222-2222-2222-222222222222",
|
||||
status="ready",
|
||||
source="engine",
|
||||
scope="session_end",
|
||||
stage="정리",
|
||||
payload={"strengths": ["보라별의 전화 010-1234-5678을 반복했다."]},
|
||||
error="보라별 direct write error",
|
||||
)
|
||||
|
||||
with (
|
||||
patch.object(session_persistence, "get_pool", return_value=object()),
|
||||
patch.object(session_persistence, "acquire", return_value=FakeAcquire(conn)),
|
||||
):
|
||||
saved = await session_persistence.save_session_evaluation(write)
|
||||
|
||||
self.assertTrue(saved)
|
||||
_, args = conn.executed[0]
|
||||
blob = json.dumps({"payload": args[5], "error": args[6]}, ensure_ascii=False)
|
||||
self.assertNotIn("보라별", blob)
|
||||
self.assertNotIn("010-1234-5678", blob)
|
||||
self.assertIn("[NAME]", blob)
|
||||
self.assertIn("[PHONE]", blob)
|
||||
|
||||
def test_rebuild_turn_evaluation_restores_review_shape(self) -> None:
|
||||
rebuilt = session_persistence._rebuild_turn_evaluations(
|
||||
[("11111111-1111-1111-1111-111111111111", 2, "탐색")],
|
||||
|
|
@ -270,6 +482,11 @@ class EvaluationPersistenceMappingTest(unittest.TestCase):
|
|||
self.assertIn("ALTER TABLE app.turn_client_state ENABLE ROW LEVEL SECURITY", sql)
|
||||
self.assertIn("ALTER TABLE app.supervisor_comment ENABLE ROW LEVEL SECURITY", sql)
|
||||
self.assertIn("ALTER TABLE app.alternative_utterance ENABLE ROW LEVEL SECURITY", sql)
|
||||
self.assertIn("CREATE POLICY p_feedback_delete", sql)
|
||||
self.assertIn("CREATE POLICY p_turn_technique_delete", sql)
|
||||
self.assertIn("CREATE POLICY p_turn_client_state_delete", sql)
|
||||
self.assertIn("CREATE POLICY p_supervisor_comment_delete", sql)
|
||||
self.assertIn("CREATE POLICY p_alternative_utterance_delete", sql)
|
||||
feedback_insert = sql.split("CREATE POLICY p_feedback_insert", 1)[1].split(");", 1)[0]
|
||||
self.assertNotIn("learner_id = app.current_uid()", feedback_insert)
|
||||
|
||||
|
|
@ -281,6 +498,32 @@ class EvaluationPersistenceMappingTest(unittest.TestCase):
|
|||
|
||||
|
||||
class EvaluationPersistenceIOTest(unittest.IsolatedAsyncioTestCase):
|
||||
async def test_missing_session_evaluation_finder_uses_stale_ai_context_query(self) -> None:
|
||||
conn = FakeMissingEvaluationConn()
|
||||
sentinel = object()
|
||||
|
||||
with (
|
||||
patch.object(session_persistence, "get_pool", return_value=object()),
|
||||
patch.object(session_persistence, "acquire", return_value=FakeAcquire(conn)) as acquire,
|
||||
patch.object(session_persistence, "_session_from_rows", return_value=sentinel),
|
||||
):
|
||||
found, durable = await session_persistence.list_sessions_missing_session_evaluation(
|
||||
older_than_seconds=150.0,
|
||||
limit=3,
|
||||
)
|
||||
|
||||
self.assertTrue(durable)
|
||||
self.assertEqual(found, [sentinel])
|
||||
acquire.assert_called_once_with(ai_context=True, ai_view="evaluator")
|
||||
self.assertEqual(conn.fetches[0][1], (150.0, 3))
|
||||
session_query = conn.fetches[0][0]
|
||||
self.assertIn("s.ended_at IS NOT NULL", session_query)
|
||||
self.assertIn("se.session_id IS NULL", session_query)
|
||||
self.assertIn("'client' = ANY(t.visible_to)", session_query)
|
||||
self.assertIn("ORDER BY s.ended_at ASC", session_query)
|
||||
self.assertEqual(conn.fetchrows[0][1], ("11111111-1111-1111-1111-111111111111",))
|
||||
self.assertIn("FROM app.turns", conn.fetches[1][0])
|
||||
|
||||
async def test_record_llm_call_audit_inserts_metadata_only(self) -> None:
|
||||
conn = FakeEvaluationConn()
|
||||
payload = {
|
||||
|
|
@ -366,6 +609,44 @@ class EvaluationPersistenceIOTest(unittest.IsolatedAsyncioTestCase):
|
|||
self.assertIn("DELETE FROM app.alternative_utterance", executed_sql)
|
||||
self.assertIn("INSERT INTO app.alternative_utterance", executed_sql)
|
||||
|
||||
async def test_replace_turn_evaluation_clears_stale_normalized_rows_before_reinsert(self) -> None:
|
||||
conn = FakeEvaluationConn()
|
||||
evaluation = {
|
||||
"loop": "fast",
|
||||
"turn_seq": 3,
|
||||
"stage": "탐색",
|
||||
"appropriateness": "pos",
|
||||
"techniques": [
|
||||
{"code": "reflection", "label_ko": "반영", "category": "relational"}
|
||||
],
|
||||
"client_state_read": [{"code": "open", "label_ko": "개방"}],
|
||||
"alternative_utterances": ["조금 더 머물러도 괜찮습니다."],
|
||||
}
|
||||
|
||||
with (
|
||||
patch.object(session_persistence, "get_pool", return_value=object()),
|
||||
patch.object(session_persistence, "acquire", return_value=FakeAcquire(conn)) as acquire,
|
||||
):
|
||||
ok = await session_persistence.replace_turn_evaluation(
|
||||
turn_id="11111111-1111-1111-1111-111111111111",
|
||||
evaluation=evaluation,
|
||||
)
|
||||
|
||||
self.assertTrue(ok)
|
||||
acquire.assert_called_once_with(ai_context=True, ai_view="evaluator")
|
||||
executed_sql = "\n".join(query for query, _ in conn.executed)
|
||||
self.assertLess(
|
||||
executed_sql.index("DELETE FROM app.feedback_scores"),
|
||||
executed_sql.index("INSERT INTO app.feedback_scores"),
|
||||
)
|
||||
self.assertIn("DELETE FROM app.turn_technique", executed_sql)
|
||||
self.assertIn("DELETE FROM app.turn_client_state", executed_sql)
|
||||
self.assertIn("DELETE FROM app.supervisor_comment", executed_sql)
|
||||
self.assertIn("DELETE FROM app.alternative_utterance", executed_sql)
|
||||
self.assertIn("INSERT INTO app.turn_technique", executed_sql)
|
||||
self.assertIn("INSERT INTO app.turn_client_state", executed_sql)
|
||||
self.assertIn("INSERT INTO app.alternative_utterance", executed_sql)
|
||||
|
||||
async def test_route_loader_only_hydrates_when_requested(self) -> None:
|
||||
principal = Principal(
|
||||
user_id="00000000-0000-0000-0000-000000000101",
|
||||
|
|
|
|||
|
|
@ -113,9 +113,9 @@ class LearnerDashboardTest(unittest.IsolatedAsyncioTestCase):
|
|||
with (
|
||||
patch.object(
|
||||
sessions.session_persistence,
|
||||
"list_sessions",
|
||||
"list_recent_sessions",
|
||||
AsyncMock(return_value=(owned_sessions, True)),
|
||||
) as list_sessions,
|
||||
) as list_recent_sessions,
|
||||
patch.object(
|
||||
sessions.session_persistence,
|
||||
"load_session_evaluation",
|
||||
|
|
@ -129,7 +129,7 @@ class LearnerDashboardTest(unittest.IsolatedAsyncioTestCase):
|
|||
):
|
||||
response = await sessions.learner_dashboard(principal)
|
||||
|
||||
list_sessions.assert_awaited_once_with(principal, include_turn_evaluation=True)
|
||||
list_recent_sessions.assert_awaited_once_with(principal, include_turn_evaluation=True)
|
||||
list_session_archives.assert_awaited_once_with(
|
||||
[sess.session_id for sess in owned_sessions],
|
||||
principal,
|
||||
|
|
@ -155,6 +155,59 @@ class LearnerDashboardTest(unittest.IsolatedAsyncioTestCase):
|
|||
self.assertEqual(len(response.recent_feedback), 3)
|
||||
self.assertIn("근거 기반 피드백", response.recent_feedback[0].note)
|
||||
|
||||
async def test_dashboard_excludes_failed_fast_loop_evaluations_from_growth_and_feedback(self) -> None:
|
||||
principal = _principal()
|
||||
good_session = _session(
|
||||
session_id="00000000-0000-0000-0000-00000000b211",
|
||||
session_no=1,
|
||||
persona_code=P1.code,
|
||||
score="pos",
|
||||
rapport=0.5,
|
||||
technique="reflection",
|
||||
created_at=1_000.0,
|
||||
)
|
||||
failed_session = _session(
|
||||
session_id="00000000-0000-0000-0000-00000000b212",
|
||||
session_no=2,
|
||||
persona_code=P1.code,
|
||||
score="neutral",
|
||||
rapport=0.2,
|
||||
technique="failed evaluator",
|
||||
created_at=2_000.0,
|
||||
)
|
||||
failed_eval = failed_session.turns[0].evaluation
|
||||
assert failed_eval is not None
|
||||
failed_eval["error"] = "engine_error"
|
||||
failed_eval["appropriateness_note"] = "이 문장은 최근 피드백에 노출되면 안 됩니다."
|
||||
failed_eval["techniques"] = []
|
||||
|
||||
with (
|
||||
patch.object(
|
||||
sessions.session_persistence,
|
||||
"list_recent_sessions",
|
||||
AsyncMock(return_value=([good_session, failed_session], True)),
|
||||
),
|
||||
patch.object(
|
||||
sessions.session_persistence,
|
||||
"load_session_evaluation",
|
||||
AsyncMock(return_value=({"status": "ready"}, True)),
|
||||
),
|
||||
patch.object(
|
||||
sessions.session_persistence,
|
||||
"list_session_archives",
|
||||
AsyncMock(return_value=({}, True)),
|
||||
),
|
||||
):
|
||||
response = await sessions.learner_dashboard(principal)
|
||||
|
||||
self.assertEqual(response.growth.evaluated_sessions, 1)
|
||||
self.assertEqual(response.growth.first_score, 1.0)
|
||||
self.assertEqual(response.growth.latest_score, 1.0)
|
||||
self.assertEqual(response.growth.avg_score, 1.0)
|
||||
self.assertEqual(len(response.recent_feedback), 1)
|
||||
self.assertIn("reflection 근거 기반 피드백", response.recent_feedback[0].note)
|
||||
self.assertNotIn("노출되면 안 됩니다", response.model_dump_json())
|
||||
|
||||
async def test_session_list_marks_archived_rows_from_learner_archive_state(self) -> None:
|
||||
principal = _principal()
|
||||
owned_sessions = [
|
||||
|
|
@ -190,9 +243,9 @@ class LearnerDashboardTest(unittest.IsolatedAsyncioTestCase):
|
|||
with (
|
||||
patch.object(
|
||||
sessions.session_persistence,
|
||||
"list_sessions",
|
||||
"list_recent_sessions",
|
||||
AsyncMock(return_value=(owned_sessions, True)),
|
||||
) as list_sessions,
|
||||
) as list_recent_sessions,
|
||||
patch.object(
|
||||
sessions.session_persistence,
|
||||
"list_session_archives",
|
||||
|
|
@ -202,7 +255,7 @@ class LearnerDashboardTest(unittest.IsolatedAsyncioTestCase):
|
|||
):
|
||||
response = await sessions.list_learner_sessions(principal)
|
||||
|
||||
list_sessions.assert_awaited_once_with(principal)
|
||||
list_recent_sessions.assert_awaited_once_with(principal)
|
||||
list_session_archives.assert_awaited_once_with(
|
||||
[sess.session_id for sess in owned_sessions],
|
||||
principal,
|
||||
|
|
@ -263,7 +316,7 @@ class LearnerDashboardTest(unittest.IsolatedAsyncioTestCase):
|
|||
with (
|
||||
patch.object(
|
||||
sessions.session_persistence,
|
||||
"list_sessions",
|
||||
"list_recent_sessions",
|
||||
AsyncMock(return_value=(owned_sessions, True)),
|
||||
),
|
||||
patch.object(
|
||||
|
|
|
|||
103
apps/api/app/test_live_coach_privacy.py
Normal file
103
apps/api/app/test_live_coach_privacy.py
Normal file
|
|
@ -0,0 +1,103 @@
|
|||
import unittest
|
||||
|
||||
from . import turn_runtime
|
||||
from .services import guardrail, live_coach
|
||||
from .services import state_machine
|
||||
|
||||
|
||||
class LiveCoachPromptPrivacyTest(unittest.TestCase):
|
||||
def tearDown(self) -> None:
|
||||
guardrail.set_ko_pii_recognizer(None)
|
||||
|
||||
def test_messages_remask_recent_turns_and_evaluation_payload(self) -> None:
|
||||
class FakeKoRecognizer:
|
||||
def analyze(self, text: str):
|
||||
spans = []
|
||||
for entity_type, value in (
|
||||
("NAME", "보라별"),
|
||||
("ORG", "미래학교상담연구랩"),
|
||||
):
|
||||
start = text.find(value)
|
||||
if start >= 0:
|
||||
spans.append(guardrail.PiiEntitySpan(entity_type, start, start + len(value)))
|
||||
return spans
|
||||
|
||||
guardrail.set_ko_pii_recognizer(FakeKoRecognizer())
|
||||
item = live_coach.LiveCoachInput(
|
||||
session_id="privacy-live-coach",
|
||||
turn_seq=3,
|
||||
stage="exploration",
|
||||
effective_openness=0.42,
|
||||
theory_mode="humanistic",
|
||||
persona_code="P1",
|
||||
persona_name="서연",
|
||||
learner_text="별명은 보라별이고 기관은 미래학교상담연구랩입니다.",
|
||||
client_reply="전화는 010-1234-5678입니다.",
|
||||
recent_turns=[
|
||||
{
|
||||
"speaker": "counselor",
|
||||
"text": "보라별이 미래학교상담연구랩에서 힘들다고 했죠.",
|
||||
},
|
||||
{
|
||||
"speaker": "client",
|
||||
"text": "010-1234-5678로 연락하지 말아주세요.",
|
||||
},
|
||||
],
|
||||
evaluation={
|
||||
"appropriateness": "warn",
|
||||
"appropriateness_note": "보라별 연락처 010-1234-5678을 되물었다.",
|
||||
"nested": {"rationale": "기관 미래학교상담연구랩 언급"},
|
||||
},
|
||||
)
|
||||
|
||||
messages = live_coach._messages(item, [])
|
||||
user_prompt = messages[1].content
|
||||
|
||||
for raw in ("보라별", "미래학교상담연구랩", "010-1234-5678"):
|
||||
self.assertNotIn(raw, user_prompt)
|
||||
for masked in ("[NAME]", "[ORG]", "[PHONE]"):
|
||||
self.assertIn(masked, user_prompt)
|
||||
|
||||
|
||||
class LiveCoachQuotaRechargeTest(unittest.TestCase):
|
||||
def test_recharge_requires_positive_score_and_client_change(self) -> None:
|
||||
before = state_machine.SessionState(
|
||||
stage=state_machine.Stage.RAPPORT,
|
||||
turn_seq=2,
|
||||
effective_openness=0.15,
|
||||
)
|
||||
after = state_machine.SessionState(
|
||||
stage=state_machine.Stage.RAPPORT,
|
||||
turn_seq=3,
|
||||
effective_openness=0.19,
|
||||
)
|
||||
|
||||
should_recharge, reason = turn_runtime.should_recharge_live_coach_credit(
|
||||
{"appropriateness": "pos", "rapport_signal": 0.45},
|
||||
before,
|
||||
after,
|
||||
)
|
||||
self.assertTrue(should_recharge)
|
||||
self.assertIn("개방도", reason)
|
||||
|
||||
should_recharge, _ = turn_runtime.should_recharge_live_coach_credit(
|
||||
{"appropriateness": "pos", "rapport_signal": 0.45},
|
||||
before,
|
||||
state_machine.SessionState(
|
||||
stage=state_machine.Stage.RAPPORT,
|
||||
turn_seq=3,
|
||||
effective_openness=0.16,
|
||||
),
|
||||
)
|
||||
self.assertFalse(should_recharge)
|
||||
|
||||
should_recharge, _ = turn_runtime.should_recharge_live_coach_credit(
|
||||
{"appropriateness": "neutral", "rapport_signal": 0.8},
|
||||
before,
|
||||
after,
|
||||
)
|
||||
self.assertFalse(should_recharge)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -228,6 +228,71 @@ class NotificationTriggerTest(unittest.IsolatedAsyncioTestCase):
|
|||
logs.output,
|
||||
)
|
||||
|
||||
async def test_missing_session_evaluation_recovery_schedules_stale_sessions(self) -> None:
|
||||
sess = _ended_session()
|
||||
|
||||
with (
|
||||
patched_settings(session_evaluation_timeout=10.0),
|
||||
patch.object(
|
||||
sessions.session_persistence,
|
||||
"list_sessions_missing_session_evaluation",
|
||||
AsyncMock(return_value=([sess], True)),
|
||||
) as list_missing,
|
||||
patch.object(
|
||||
sessions,
|
||||
"_schedule_session_evaluation",
|
||||
return_value=object(),
|
||||
) as schedule_evaluation,
|
||||
):
|
||||
scheduled = await sessions.recover_missing_session_evaluations(limit=2)
|
||||
|
||||
self.assertEqual(scheduled, 1)
|
||||
list_missing.assert_awaited_once_with(older_than_seconds=40.0, limit=2)
|
||||
schedule_evaluation.assert_called_once_with(sess)
|
||||
|
||||
async def test_missing_session_evaluation_recovery_requires_durable_store(self) -> None:
|
||||
sess = _ended_session()
|
||||
|
||||
with (
|
||||
patch.object(
|
||||
sessions.session_persistence,
|
||||
"list_sessions_missing_session_evaluation",
|
||||
AsyncMock(return_value=([sess], False)),
|
||||
),
|
||||
patch.object(sessions, "_schedule_session_evaluation") as schedule_evaluation,
|
||||
):
|
||||
scheduled = await sessions.recover_missing_session_evaluations(limit=2)
|
||||
|
||||
self.assertEqual(scheduled, 0)
|
||||
schedule_evaluation.assert_not_called()
|
||||
|
||||
async def test_session_evaluation_scheduler_deduplicates_in_flight_session(self) -> None:
|
||||
sess = _ended_session()
|
||||
gate = asyncio.Event()
|
||||
sessions._SESSION_EVALUATION_IN_FLIGHT.clear()
|
||||
|
||||
async def wait_until_released(_sess: InProcSession) -> None:
|
||||
await gate.wait()
|
||||
|
||||
try:
|
||||
with patch.object(
|
||||
sessions,
|
||||
"_generate_and_save_session_evaluation",
|
||||
wait_until_released,
|
||||
):
|
||||
task = sessions._schedule_session_evaluation(sess)
|
||||
self.assertIsNotNone(task)
|
||||
self.assertIsNone(sessions._schedule_session_evaluation(sess))
|
||||
self.assertIn(sess.session_id, sessions._SESSION_EVALUATION_IN_FLIGHT)
|
||||
gate.set()
|
||||
assert task is not None
|
||||
await task
|
||||
await asyncio.sleep(0)
|
||||
finally:
|
||||
sessions._SESSION_EVALUATION_IN_FLIGHT.clear()
|
||||
|
||||
self.assertNotIn(sess.session_id, sessions._SESSION_EVALUATION_IN_FLIGHT)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
|
|||
|
|
@ -26,6 +26,33 @@ def sha256(path: Path) -> str:
|
|||
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def valid_dataset_item() -> dict[str, object]:
|
||||
return {
|
||||
"schema": "phase3_dataset_item_v1",
|
||||
"item_id": "DI-000001",
|
||||
"participant_key": "PX-0001",
|
||||
"session_key": "SX-0001",
|
||||
"turn_key": "TX-000001",
|
||||
"persona_id": "P1",
|
||||
"stage": "rapport",
|
||||
"speaker": "client",
|
||||
"text_masked": "요즘 [NAME] 관련 고민이 있습니다.",
|
||||
"techniques": [],
|
||||
"client_states": [],
|
||||
"feedback_scores": [],
|
||||
"supervisor_comments": [],
|
||||
"source_refs": {
|
||||
"session_started_at": "2026-06-27T00:00:00Z",
|
||||
"export_manifest_id": "phase3-fixture",
|
||||
},
|
||||
"privacy": {
|
||||
"direct_identifiers_removed": True,
|
||||
"pii_scan_status": "pass",
|
||||
"consent_scope": "recursive_learning_seed",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
class Phase3ArtifactCheckerTests(unittest.TestCase):
|
||||
def make_root(self) -> Path:
|
||||
tmp = tempfile.TemporaryDirectory()
|
||||
|
|
@ -66,7 +93,10 @@ class Phase3ArtifactCheckerTests(unittest.TestCase):
|
|||
root / "04-privacy" / "privacy_audit.md",
|
||||
"# Privacy audit\n\nLegal/privacy reviewer: reviewer@example.invalid\n",
|
||||
)
|
||||
write_text(root / "03-export" / "anonymized_dataset.jsonl", "{}\n")
|
||||
write_text(
|
||||
root / "03-export" / "anonymized_dataset.jsonl",
|
||||
json.dumps(valid_dataset_item(), ensure_ascii=False, sort_keys=True) + "\n",
|
||||
)
|
||||
|
||||
metrics = {
|
||||
name: {
|
||||
|
|
@ -236,6 +266,33 @@ class Phase3ArtifactCheckerTests(unittest.TestCase):
|
|||
|
||||
self.assertTrue(any("sha256 mismatch" in error for error in report.errors), report.errors)
|
||||
|
||||
def test_approved_manifest_rejects_malformed_dataset_jsonl(self) -> None:
|
||||
root = self.make_root()
|
||||
dataset_path = root / "03-export" / "anonymized_dataset.jsonl"
|
||||
write_text(dataset_path, "{}\n")
|
||||
manifest_path = root / "03-export" / "export_manifest.json"
|
||||
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
|
||||
manifest["files"][0]["sha256"] = sha256(dataset_path)
|
||||
manifest_path.write_text(json.dumps(manifest), encoding="utf-8")
|
||||
|
||||
report = checker.validate(root, max_scan_rows=100)
|
||||
|
||||
self.assertTrue(any("dataset item missing keys" in error for error in report.errors), report.errors)
|
||||
|
||||
def test_dry_run_manifest_rejects_malformed_dataset_jsonl(self) -> None:
|
||||
root = self.make_root()
|
||||
dataset_path = root / "03-export" / "anonymized_dataset.jsonl"
|
||||
write_text(dataset_path, "{}\n")
|
||||
manifest_path = root / "03-export" / "export_manifest.json"
|
||||
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
|
||||
manifest["export_status"] = "technical_dry_run"
|
||||
manifest["files"][0]["sha256"] = sha256(dataset_path)
|
||||
manifest_path.write_text(json.dumps(manifest), encoding="utf-8")
|
||||
|
||||
report = checker.validate(root, max_scan_rows=100)
|
||||
|
||||
self.assertTrue(any("dataset item missing keys" in error for error in report.errors), report.errors)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
|
|||
|
|
@ -542,7 +542,7 @@ class LearnerSessionIdorTest(unittest.IsolatedAsyncioTestCase):
|
|||
with (
|
||||
patch.object(
|
||||
sessions.session_persistence,
|
||||
"list_sessions",
|
||||
"list_recent_sessions",
|
||||
AsyncMock(return_value=([], False)),
|
||||
),
|
||||
patch.object(
|
||||
|
|
@ -560,7 +560,7 @@ class LearnerSessionIdorTest(unittest.IsolatedAsyncioTestCase):
|
|||
|
||||
|
||||
class TeacherAdminAuditTest(unittest.IsolatedAsyncioTestCase):
|
||||
async def test_teacher_list_sessions_inserts_read_audit_log(self) -> None:
|
||||
async def test_teacher_list_recent_sessions_inserts_read_audit_log(self) -> None:
|
||||
teacher = _principal(
|
||||
user_id="00000000-0000-0000-0000-000000000505",
|
||||
role=Role.TEACHER,
|
||||
|
|
@ -571,9 +571,9 @@ class TeacherAdminAuditTest(unittest.IsolatedAsyncioTestCase):
|
|||
session_id=session_id,
|
||||
learner_id="00000000-0000-0000-0000-000000000606",
|
||||
)
|
||||
# 배치 조회(N+1 제거): main sessions → session_state 배치 → turns 배치 (fetch 3회, fetchrow 0회).
|
||||
conn = _FakeConn(
|
||||
fetchrow_results=[None],
|
||||
fetch_results=[[row], []],
|
||||
fetch_results=[[row], [], []],
|
||||
)
|
||||
acquire_calls: list[dict[str, Any]] = []
|
||||
|
||||
|
|
@ -586,7 +586,7 @@ class TeacherAdminAuditTest(unittest.IsolatedAsyncioTestCase):
|
|||
patch.object(session_persistence, "acquire", fake_acquire),
|
||||
patch.object(session_persistence, "_session_from_rows", return_value=sess),
|
||||
):
|
||||
found, durable = await session_persistence.list_sessions(teacher)
|
||||
found, durable = await session_persistence.list_recent_sessions(teacher)
|
||||
|
||||
self.assertTrue(durable)
|
||||
self.assertEqual(found, [sess])
|
||||
|
|
@ -598,7 +598,7 @@ class TeacherAdminAuditTest(unittest.IsolatedAsyncioTestCase):
|
|||
self.assertEqual(args[1], "read_session")
|
||||
self.assertEqual(args[2], "session_list")
|
||||
self.assertEqual(args[3], "sessions")
|
||||
self.assertEqual(args[4]["access"], "list_sessions")
|
||||
self.assertEqual(args[4]["access"], "list_recent_sessions")
|
||||
self.assertEqual(args[4]["role"], "teacher")
|
||||
self.assertEqual(args[4]["result_count"], 1)
|
||||
|
||||
|
|
|
|||
|
|
@ -78,7 +78,7 @@ class RuntimeFallbackPolicyTest(unittest.IsolatedAsyncioTestCase):
|
|||
)
|
||||
with environment("staging"):
|
||||
with self.assertRaises(HTTPException) as caught:
|
||||
await session_persistence.list_sessions(principal)
|
||||
await session_persistence.list_recent_sessions(principal)
|
||||
|
||||
self.assertEqual(caught.exception.status_code, 503)
|
||||
self.assertIn("runtime fallback is disabled in staging", caught.exception.detail)
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@ from .deps import Principal, Role
|
|||
from .engine_client import EngineError
|
||||
from .routes import sessions
|
||||
from .routes import voice as voice_routes
|
||||
from .services import live_coach, memory, orchestrator, persona as persona_service, state_machine
|
||||
from .services import guardrail, live_coach, memory, orchestrator, persona as persona_service, rag, state_machine
|
||||
from .services.voice import TTSChunk, TranscriptResult, VoicePreset
|
||||
from .store import InProcSession, TurnRecord, store
|
||||
|
||||
|
|
@ -29,6 +29,17 @@ async def _decoded_stream_packets(stream_engine, req):
|
|||
yield packet
|
||||
|
||||
|
||||
class _AsyncConnContext:
|
||||
def __init__(self, conn):
|
||||
self.conn = conn
|
||||
|
||||
async def __aenter__(self):
|
||||
return self.conn
|
||||
|
||||
async def __aexit__(self, exc_type, exc, tb):
|
||||
return False
|
||||
|
||||
|
||||
def _principal() -> Principal:
|
||||
return Principal(
|
||||
user_id="00000000-0000-0000-0000-000000000101",
|
||||
|
|
@ -83,6 +94,22 @@ class SessionTurnPersistenceTest(unittest.IsolatedAsyncioTestCase):
|
|||
sessions._RECALL_CACHE.clear()
|
||||
sessions._KB_CUES_CACHE.clear()
|
||||
|
||||
async def test_end_session_does_not_reschedule_evaluation_for_already_ended_session(self) -> None:
|
||||
principal = _principal()
|
||||
sess = _session(principal)
|
||||
sess.ended = True
|
||||
sess.ended_at = 1_000.0
|
||||
|
||||
with (
|
||||
patch.object(sessions, "_load_session_or_404", AsyncMock(return_value=sess)),
|
||||
patch.object(sessions, "_end_persisted_session", AsyncMock(return_value=None)),
|
||||
patch.object(sessions, "_schedule_session_evaluation") as schedule_session_evaluation,
|
||||
):
|
||||
response = await sessions.end_session(sess.session_id, principal)
|
||||
|
||||
self.assertEqual(response.session_id, sess.session_id)
|
||||
schedule_session_evaluation.assert_not_called()
|
||||
|
||||
async def test_append_turn_writes_provider_events_to_db(self) -> None:
|
||||
class FakeConn:
|
||||
def __init__(self) -> None:
|
||||
|
|
@ -161,6 +188,80 @@ class SessionTurnPersistenceTest(unittest.IsolatedAsyncioTestCase):
|
|||
self.assertEqual(caught.exception.status_code, 503)
|
||||
self.assertEqual(sess.turns, [])
|
||||
|
||||
async def test_live_coach_turn_marks_runtime_persistence_source(self) -> None:
|
||||
principal = _principal()
|
||||
sess = _session(principal)
|
||||
sess.turns.extend(
|
||||
[
|
||||
TurnRecord(
|
||||
turn_seq=1,
|
||||
speaker="counselor",
|
||||
stage="explore",
|
||||
text="요즘 가장 크게 남는 마음은 무엇인가요?",
|
||||
text_masked="요즘 가장 크게 남는 마음은 무엇인가요?",
|
||||
),
|
||||
TurnRecord(
|
||||
turn_seq=2,
|
||||
speaker="client",
|
||||
stage="explore",
|
||||
text="잘 모르겠지만 계속 답답해요.",
|
||||
text_masked="잘 모르겠지만 계속 답답해요.",
|
||||
),
|
||||
]
|
||||
)
|
||||
suggestion = live_coach.LiveCoachSuggestion(
|
||||
status="ready",
|
||||
tone="pos",
|
||||
focus="emotion",
|
||||
title="감정 반영이 선명합니다",
|
||||
message="방금 응답의 정서를 먼저 붙잡았습니다.",
|
||||
)
|
||||
|
||||
with (
|
||||
patch.object(
|
||||
sessions,
|
||||
"_retrieve_live_coach_grounding",
|
||||
AsyncMock(return_value=[]),
|
||||
),
|
||||
patch.object(
|
||||
sessions.live_coach,
|
||||
"generate_live_coaching",
|
||||
AsyncMock(return_value=suggestion),
|
||||
),
|
||||
patch.object(
|
||||
sessions.session_persistence,
|
||||
"get_live_coach_quota",
|
||||
AsyncMock(
|
||||
side_effect=[
|
||||
({"remaining": 3, "max": 3}, True),
|
||||
({"remaining": 2, "max": 3}, True),
|
||||
]
|
||||
),
|
||||
),
|
||||
patch.object(
|
||||
sessions.session_persistence,
|
||||
"save_live_coach_event",
|
||||
AsyncMock(return_value=({"event_id": "cached-live-coach"}, False)),
|
||||
),
|
||||
patch.object(
|
||||
sessions.session_persistence,
|
||||
"list_live_coach_credit_events",
|
||||
AsyncMock(return_value=([], True)),
|
||||
),
|
||||
):
|
||||
response = await sessions.live_coach_turn(
|
||||
sess.session_id,
|
||||
sessions.LiveCoachRequest(
|
||||
learner_text="요즘 가장 크게 남는 마음은 무엇인가요?",
|
||||
client_reply="잘 모르겠지만 계속 답답해요.",
|
||||
turn_seq=1,
|
||||
),
|
||||
principal,
|
||||
)
|
||||
|
||||
self.assertEqual(response.persistence_source, "runtime")
|
||||
self.assertEqual(response.quota.remaining, 2)
|
||||
|
||||
async def test_generate_turn_persists_client_engine_telemetry(self) -> None:
|
||||
principal = _principal()
|
||||
sess = _session(principal)
|
||||
|
|
@ -462,6 +563,70 @@ class SessionTurnPersistenceTest(unittest.IsolatedAsyncioTestCase):
|
|||
self.assertIn("조금 말해볼게요", learner_turn.evaluation["appropriateness_note"])
|
||||
self.assertIsNone(client_turn.evaluation)
|
||||
|
||||
async def test_stream_turn_surfaces_fast_loop_evaluation_failure_on_review(self) -> None:
|
||||
principal = _principal()
|
||||
sess = _session(principal)
|
||||
|
||||
async def successful_stream(ctx, engine, **kwargs):
|
||||
assert ctx.state_after is not None
|
||||
yield orchestrator.StreamEvent("token", {"text": "조금 말해볼게요."})
|
||||
yield orchestrator.StreamEvent(
|
||||
"done",
|
||||
{
|
||||
"session_id": ctx.session_id,
|
||||
"stage": ctx.state_after.stage.value,
|
||||
"effective_openness": ctx.state_after.effective_openness,
|
||||
"turn_seq": ctx.state_after.turn_seq,
|
||||
"safety_flagged": False,
|
||||
"llm_provider": "claude_cli",
|
||||
"model": "gateway-default",
|
||||
"tokens_in": 9,
|
||||
"tokens_out": 10,
|
||||
"cost_usd": 0.001,
|
||||
},
|
||||
)
|
||||
|
||||
async def failing_eval_hook(ctx, client_reply):
|
||||
raise RuntimeError("김서연 평가 timeout 010-1234-5678")
|
||||
|
||||
with patch.object(sessions.orchestrator, "run_turn_stream", successful_stream), patch.object(
|
||||
sessions.evaluator,
|
||||
"make_eval_hook",
|
||||
return_value=failing_eval_hook,
|
||||
):
|
||||
response = await sessions.stream_turn(
|
||||
sess.session_id,
|
||||
sessions.TurnRequest(text="스트림 평가 실패 발화"),
|
||||
principal,
|
||||
)
|
||||
await _consume_event_source(response)
|
||||
|
||||
self.assertEqual(len(sess.turns), 2)
|
||||
learner_turn = sess.turns[0]
|
||||
self.assertIsNotNone(learner_turn.evaluation)
|
||||
assert learner_turn.evaluation is not None
|
||||
self.assertEqual(learner_turn.evaluation["loop"], "fast")
|
||||
self.assertEqual(learner_turn.evaluation["appropriateness"], "neutral")
|
||||
self.assertEqual(learner_turn.evaluation["error"], "RuntimeError")
|
||||
self.assertNotIn("김서연", learner_turn.evaluation["error"])
|
||||
self.assertNotIn("010-1234-5678", learner_turn.evaluation["error"])
|
||||
|
||||
review = sessions.build_session_review(
|
||||
sessions.SessionReviewReadInput(
|
||||
session=sess,
|
||||
evaluation_record=None,
|
||||
evaluation_durable=False,
|
||||
)
|
||||
)
|
||||
review_turn = review.turns[0]
|
||||
self.assertIsNotNone(review_turn.note)
|
||||
assert review_turn.note is not None
|
||||
self.assertEqual(review_turn.note.title, "턴 평가 실패")
|
||||
self.assertIn("fast-loop 평가를 완료하지 못했습니다", review_turn.note.body)
|
||||
self.assertIn("RuntimeError", review_turn.note.body)
|
||||
self.assertNotIn("김서연", review_turn.note.body)
|
||||
self.assertNotIn("010-1234-5678", review_turn.note.body)
|
||||
|
||||
async def test_live_coach_degrades_to_rule_based_suggestion_when_engine_fails(self) -> None:
|
||||
principal = _principal()
|
||||
sess = _session(principal)
|
||||
|
|
@ -507,12 +672,58 @@ class SessionTurnPersistenceTest(unittest.IsolatedAsyncioTestCase):
|
|||
self.assertEqual(len(history.events), 1)
|
||||
self.assertEqual(history.events[0].turn_seq, 1)
|
||||
self.assertEqual(history.events[0].stage, "라포")
|
||||
self.assertEqual(history.events[0].suggestion.status, "degraded")
|
||||
self.assertEqual(history.events[0].suggestion.title, response.title)
|
||||
self.assertIn("학교", history.events[0].learner_text_excerpt or "")
|
||||
session_persistence._LIVE_COACH_EVENT_CACHE[sess.session_id][0]["stage"] = "unknown-stage"
|
||||
legacy_history = await sessions.list_live_coach_history(sess.session_id, principal)
|
||||
self.assertIsNone(legacy_history.events[0].stage)
|
||||
|
||||
async def test_live_coach_rag_grounding_preserves_source_pack_metadata(self) -> None:
|
||||
retrieval = rag.RetrievalResult(
|
||||
chunks=[
|
||||
rag.RetrievedChunk(
|
||||
chunk_id=17,
|
||||
score=0.91,
|
||||
kb_kind="supervisor_pattern",
|
||||
heading_path="risk/protective-factors",
|
||||
context_prefix="공식 자살위험 평가 지침 / 보호요인 / 2026-06-15",
|
||||
body="최근성, 보호요인, 안전계획을 확인한다.",
|
||||
meta={
|
||||
"source_title": "공식 자살위험 평가 지침",
|
||||
"source_type": "official_guideline_summary",
|
||||
"source_version": "2026-06-15",
|
||||
"citation": "허가된 공식 지침 요약",
|
||||
},
|
||||
source_id="official_suicide_risk_guidelines",
|
||||
)
|
||||
],
|
||||
policy_name="evaluator:k=supervisor_pattern:s<=2",
|
||||
top1_score=0.91,
|
||||
latency_ms=12,
|
||||
query_text="정리 humanistic 자살 위험 최근성 보호요인",
|
||||
)
|
||||
|
||||
with (
|
||||
patch.object(sessions.db, "acquire", return_value=_AsyncConnContext(object())),
|
||||
patch.object(sessions.rag, "retrieve_eval_grounding", AsyncMock(return_value=retrieval)),
|
||||
patch.object(sessions.rag, "log_retrieval", AsyncMock()),
|
||||
):
|
||||
grounding = await sessions._retrieve_live_coach_grounding(
|
||||
learner_text="자살 위험 최근성과 보호요인을 확인했어요.",
|
||||
client_reply="그래도 오늘은 친구에게 연락할 수 있어요.",
|
||||
stage="정리",
|
||||
theory_mode="humanistic",
|
||||
)
|
||||
|
||||
self.assertEqual(len(grounding), 1)
|
||||
source = grounding[0]
|
||||
self.assertEqual(source.source_id, "official_suicide_risk_guidelines")
|
||||
self.assertEqual(source.title, "공식 자살위험 평가 지침")
|
||||
self.assertEqual(source.source_type, "official_guideline_summary")
|
||||
self.assertEqual(source.version, "2026-06-15")
|
||||
self.assertEqual(source.citation, "허가된 공식 지침 요약")
|
||||
|
||||
async def test_live_coach_persistence_failure_is_not_swallowed(self) -> None:
|
||||
principal = _principal()
|
||||
sess = _session(principal)
|
||||
|
|
@ -1306,6 +1517,101 @@ class SessionTurnPersistenceTest(unittest.IsolatedAsyncioTestCase):
|
|||
self.assertIn("주호소", response.teacherReview.worksheetNote)
|
||||
self.assertEqual(response.teacherReview.worksheetReviewedAt, "2026-06-27T10:05:00Z")
|
||||
|
||||
async def test_review_remasks_legacy_session_evaluation_payload(self) -> None:
|
||||
class FakeKoRecognizer:
|
||||
def analyze(self, text: str):
|
||||
spans = []
|
||||
for entity_type, value in (
|
||||
("NAME", "보라별"),
|
||||
("ORG", "미래학교상담연구랩"),
|
||||
):
|
||||
start = text.find(value)
|
||||
if start >= 0:
|
||||
spans.append(guardrail.PiiEntitySpan(entity_type, start, start + len(value)))
|
||||
return spans
|
||||
|
||||
guardrail.set_ko_pii_recognizer(FakeKoRecognizer())
|
||||
self.addCleanup(guardrail.set_ko_pii_recognizer, None)
|
||||
principal = _principal()
|
||||
sess = _session(principal)
|
||||
sess.ended = True
|
||||
sess.ended_at = sess.created_at + 180
|
||||
sess.turns.extend(
|
||||
[
|
||||
TurnRecord(
|
||||
turn_seq=1,
|
||||
speaker="counselor",
|
||||
stage=sess.state.stage.value,
|
||||
text="감정을 먼저 확인해보겠습니다.",
|
||||
text_masked="감정을 먼저 확인해보겠습니다.",
|
||||
created_at=sess.created_at + 1,
|
||||
),
|
||||
TurnRecord(
|
||||
turn_seq=2,
|
||||
speaker="client",
|
||||
stage=sess.state.stage.value,
|
||||
text="연락처 이야기는 부담스러워요.",
|
||||
text_masked="연락처 이야기는 부담스러워요.",
|
||||
created_at=sess.created_at + 2,
|
||||
),
|
||||
]
|
||||
)
|
||||
raw_record = {
|
||||
"status": "ready",
|
||||
"source": "engine",
|
||||
"scope": "session_end",
|
||||
"stage": "정리",
|
||||
"payload": {
|
||||
"loop": "deep",
|
||||
"session_id": sess.session_id,
|
||||
"stage": "정리",
|
||||
"scope": "session_end",
|
||||
"turns_evaluated": 2,
|
||||
"distribution": {},
|
||||
"strengths": ["보라별의 감정을 반영했다."],
|
||||
"improvements": ["미래학교상담연구랩과 010-1234-5678 재확인을 줄인다."],
|
||||
"intent_deviations": [
|
||||
{
|
||||
"dimension": "pacing",
|
||||
"expected": "보라별의 감정 확인",
|
||||
"actual": "010-1234-5678 연락처 재질문",
|
||||
"severity": "moderate",
|
||||
}
|
||||
],
|
||||
"supervisor_rationale": "보라별의 호소를 요약했다.",
|
||||
"supervisor_critique": "미래학교상담연구랩 언급이 반복됐다.",
|
||||
"alternative_utterances": ["보라별님, 지금 감정부터 천천히 볼까요?"],
|
||||
},
|
||||
"error": None,
|
||||
}
|
||||
|
||||
with (
|
||||
patch.object(
|
||||
sessions.session_persistence,
|
||||
"load_session",
|
||||
AsyncMock(return_value=sess),
|
||||
),
|
||||
patch.object(
|
||||
sessions.session_persistence,
|
||||
"load_case_worksheet",
|
||||
AsyncMock(return_value=(None, False)),
|
||||
),
|
||||
patch.object(
|
||||
sessions.session_persistence,
|
||||
"load_session_evaluation",
|
||||
AsyncMock(return_value=(raw_record, True)),
|
||||
),
|
||||
):
|
||||
response = await sessions.get_session_review(sess.session_id, principal)
|
||||
|
||||
blob = response.model_dump_json()
|
||||
for raw in ("보라별", "미래학교상담연구랩", "010-1234-5678"):
|
||||
self.assertNotIn(raw, blob)
|
||||
for masked in ("[NAME]", "[ORG]", "[PHONE]"):
|
||||
self.assertIn(masked, blob)
|
||||
self.assertTrue(response.reviewReady)
|
||||
self.assertEqual(response.supervisorState, "평가 완료")
|
||||
|
||||
async def test_review_surfaces_session_evaluation_error_record(self) -> None:
|
||||
principal = _principal()
|
||||
sess = _session(principal)
|
||||
|
|
@ -1366,6 +1672,53 @@ class SessionTurnPersistenceTest(unittest.IsolatedAsyncioTestCase):
|
|||
self.assertEqual(response.goodMoments, [])
|
||||
self.assertEqual(response.growthPoints, [])
|
||||
|
||||
def test_review_marks_stale_missing_session_evaluation_as_failed(self) -> None:
|
||||
principal = _principal()
|
||||
sess = _session(principal)
|
||||
sess.ended = True
|
||||
sess.ended_at = sess.created_at + 180
|
||||
sess.turns.extend(
|
||||
[
|
||||
TurnRecord(
|
||||
turn_seq=1,
|
||||
speaker="counselor",
|
||||
stage=sess.state.stage.value,
|
||||
text="많이 지쳐 보였어요. 지금 제일 버거운 마음이 뭔가요?",
|
||||
text_masked="많이 지쳐 보였어요. 지금 제일 버거운 마음이 뭔가요?",
|
||||
created_at=sess.created_at + 1,
|
||||
),
|
||||
TurnRecord(
|
||||
turn_seq=2,
|
||||
speaker="client",
|
||||
stage=sess.state.stage.value,
|
||||
text="그냥 아무것도 하고 싶지 않아요.",
|
||||
text_masked="그냥 아무것도 하고 싶지 않아요.",
|
||||
created_at=sess.created_at + 2,
|
||||
),
|
||||
]
|
||||
)
|
||||
previous_timeout = settings.session_evaluation_timeout
|
||||
settings.session_evaluation_timeout = 10.0
|
||||
try:
|
||||
response = sessions.build_session_review(
|
||||
sessions.SessionReviewReadInput(
|
||||
session=sess,
|
||||
evaluation_record=None,
|
||||
evaluation_durable=True,
|
||||
now_ts=(sess.ended_at or sess.created_at) + 41.0,
|
||||
)
|
||||
)
|
||||
finally:
|
||||
settings.session_evaluation_timeout = previous_timeout
|
||||
|
||||
self.assertFalse(response.reviewReady)
|
||||
self.assertTrue(response.degraded)
|
||||
self.assertEqual(response.supervisorState, "평가 실패")
|
||||
self.assertIn("AI 평가 재시도가 필요합니다", response.summary)
|
||||
self.assertEqual(response.rubric, [])
|
||||
self.assertEqual(response.goodMoments, [])
|
||||
self.assertEqual(response.growthPoints, [])
|
||||
|
||||
async def test_learner_can_save_case_formulation_worksheet(self) -> None:
|
||||
principal = _principal()
|
||||
sess = _session(principal)
|
||||
|
|
|
|||
|
|
@ -10,11 +10,37 @@ from fastapi import HTTPException
|
|||
from .config import settings
|
||||
from .deps import Principal, Role
|
||||
from .routes import teacher
|
||||
from .session_read_model import MISSING_SESSION_EVALUATION_ERROR
|
||||
from .services import state_machine
|
||||
from .services.persona import P1
|
||||
from .store import InProcSession, TurnRecord
|
||||
|
||||
|
||||
class _ListSessionsConn:
|
||||
def __init__(self) -> None:
|
||||
self.query = ""
|
||||
self.args: tuple[object, ...] = ()
|
||||
|
||||
async def fetch(self, query: str, *args: object) -> list[object]:
|
||||
self.query = query
|
||||
self.args = args
|
||||
return []
|
||||
|
||||
async def execute(self, _query: str, *args: object) -> None:
|
||||
return None
|
||||
|
||||
|
||||
class _AcquireCtx:
|
||||
def __init__(self, conn: _ListSessionsConn) -> None:
|
||||
self.conn = conn
|
||||
|
||||
async def __aenter__(self) -> _ListSessionsConn:
|
||||
return self.conn
|
||||
|
||||
async def __aexit__(self, exc_type, exc, tb) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def _principal() -> Principal:
|
||||
return Principal(
|
||||
user_id="00000000-0000-0000-0000-000000000901",
|
||||
|
|
@ -31,6 +57,8 @@ def _session(
|
|||
rapport: float,
|
||||
technique: str,
|
||||
created_at: float,
|
||||
learner_label: str | None = None,
|
||||
with_turns: bool = True,
|
||||
) -> InProcSession:
|
||||
state = state_machine.init_state(params=P1.openness_params())
|
||||
state.stage = state_machine.Stage.EXPLORE
|
||||
|
|
@ -46,6 +74,7 @@ def _session(
|
|||
created_at=created_at,
|
||||
ended_at=created_at + 600,
|
||||
ended=True,
|
||||
learner_label=learner_label,
|
||||
turns=[
|
||||
TurnRecord(
|
||||
turn_seq=1,
|
||||
|
|
@ -66,11 +95,51 @@ def _session(
|
|||
text="내담자 응답",
|
||||
text_masked="내담자 응답",
|
||||
),
|
||||
],
|
||||
]
|
||||
if with_turns
|
||||
else [],
|
||||
)
|
||||
|
||||
|
||||
class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase):
|
||||
async def test_list_all_sessions_has_no_recent_limit_for_teacher_summaries(self) -> None:
|
||||
principal = _principal()
|
||||
conn = _ListSessionsConn()
|
||||
|
||||
with (
|
||||
patch.object(teacher.session_persistence, "get_pool"),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"acquire",
|
||||
return_value=_AcquireCtx(conn),
|
||||
),
|
||||
):
|
||||
sessions, durable = await teacher.session_persistence.list_all_sessions(principal)
|
||||
|
||||
self.assertEqual(sessions, [])
|
||||
self.assertTrue(durable)
|
||||
self.assertNotIn("LIMIT", conn.query)
|
||||
self.assertEqual(conn.args, ())
|
||||
|
||||
async def test_list_recent_sessions_keeps_recent_limit_for_general_lists(self) -> None:
|
||||
principal = _principal()
|
||||
conn = _ListSessionsConn()
|
||||
|
||||
with (
|
||||
patch.object(teacher.session_persistence, "get_pool"),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"acquire",
|
||||
return_value=_AcquireCtx(conn),
|
||||
),
|
||||
):
|
||||
sessions, durable = await teacher.session_persistence.list_recent_sessions(principal)
|
||||
|
||||
self.assertEqual(sessions, [])
|
||||
self.assertTrue(durable)
|
||||
self.assertIn("LIMIT $1", conn.query)
|
||||
self.assertEqual(conn.args, (100,))
|
||||
|
||||
async def test_dashboard_returns_learner_growth_from_turn_evaluations(self) -> None:
|
||||
learner_id = "00000000-0000-0000-0000-000000000111"
|
||||
sessions = [
|
||||
|
|
@ -78,6 +147,7 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase):
|
|||
session_id="00000000-0000-0000-0000-00000000a111",
|
||||
session_no=1,
|
||||
learner_id=learner_id,
|
||||
learner_label="김민수",
|
||||
score="neutral",
|
||||
rapport=0.1,
|
||||
technique="reflection",
|
||||
|
|
@ -87,6 +157,7 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase):
|
|||
session_id="00000000-0000-0000-0000-00000000a112",
|
||||
session_no=2,
|
||||
learner_id=learner_id,
|
||||
learner_label="김민수",
|
||||
score="pos",
|
||||
rapport=0.5,
|
||||
technique="reflection",
|
||||
|
|
@ -98,9 +169,9 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase):
|
|||
with (
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_sessions",
|
||||
"list_all_sessions",
|
||||
AsyncMock(return_value=(sessions, True)),
|
||||
) as list_sessions,
|
||||
) as list_all_sessions,
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_safety_alerts",
|
||||
|
|
@ -111,13 +182,19 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase):
|
|||
"list_session_review_statuses",
|
||||
AsyncMock(return_value=({}, True)),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_session_evaluations",
|
||||
AsyncMock(return_value=({}, True)),
|
||||
),
|
||||
):
|
||||
response = await teacher.teacher_dashboard(principal)
|
||||
|
||||
list_sessions.assert_awaited_once_with(principal, include_turn_evaluation=True)
|
||||
list_all_sessions.assert_awaited_once_with(principal, include_turn_evaluation=True)
|
||||
self.assertEqual(response.total_learners, 1)
|
||||
self.assertEqual(len(response.learner_growth), 1)
|
||||
growth = response.learner_growth[0]
|
||||
self.assertEqual(growth.learner_label, "김민수")
|
||||
self.assertEqual(growth.sessions, 2)
|
||||
self.assertEqual(growth.ended_sessions, 2)
|
||||
self.assertEqual(growth.first_score, 0.5)
|
||||
|
|
@ -126,8 +203,56 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase):
|
|||
self.assertEqual(growth.trend, "up")
|
||||
self.assertEqual(growth.top_techniques, ["reflection"])
|
||||
self.assertEqual([point.session_no for point in growth.points], [1, 2])
|
||||
self.assertEqual(response.recent_sessions[0].learner_label, "김민수")
|
||||
self.assertEqual(len(response.pending_reviews), 2)
|
||||
|
||||
async def test_dashboard_does_not_truncate_student_analysis_list(self) -> None:
|
||||
sessions = [
|
||||
_session(
|
||||
session_id=f"many-learners-session-{idx:02d}",
|
||||
session_no=1,
|
||||
learner_id=f"many-learner-{idx:02d}",
|
||||
learner_label=f"학생 {idx:02d}",
|
||||
score="pos",
|
||||
rapport=0.2,
|
||||
technique="reflection",
|
||||
created_at=1_000.0 + idx,
|
||||
)
|
||||
for idx in range(13)
|
||||
]
|
||||
principal = _principal()
|
||||
|
||||
with (
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_all_sessions",
|
||||
AsyncMock(return_value=(sessions, True)),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_safety_alerts",
|
||||
AsyncMock(return_value=([], True)),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_session_review_statuses",
|
||||
AsyncMock(return_value=({}, True)),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_session_evaluations",
|
||||
AsyncMock(return_value=({}, True)),
|
||||
),
|
||||
):
|
||||
response = await teacher.teacher_dashboard(principal)
|
||||
|
||||
self.assertEqual(response.total_learners, 13)
|
||||
self.assertEqual(len(response.learner_growth), 13)
|
||||
self.assertEqual(
|
||||
{item.learner_id for item in response.learner_growth},
|
||||
{f"many-learner-{idx:02d}" for idx in range(13)},
|
||||
)
|
||||
|
||||
async def test_dashboard_excludes_closed_session_reviews_from_pending_queue(self) -> None:
|
||||
learner_id = "00000000-0000-0000-0000-000000000111"
|
||||
open_session = _session(
|
||||
|
|
@ -153,7 +278,7 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase):
|
|||
with (
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_sessions",
|
||||
"list_all_sessions",
|
||||
AsyncMock(return_value=([open_session, closed_session], True)),
|
||||
),
|
||||
patch.object(
|
||||
|
|
@ -178,6 +303,11 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase):
|
|||
)
|
||||
),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_session_evaluations",
|
||||
AsyncMock(return_value=({}, True)),
|
||||
),
|
||||
):
|
||||
response = await teacher.teacher_dashboard(principal)
|
||||
|
||||
|
|
@ -187,6 +317,159 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase):
|
|||
)
|
||||
self.assertEqual(closed_summary.review_status, "closed")
|
||||
|
||||
async def test_dashboard_excludes_empty_ended_sessions_from_pending_queue(self) -> None:
|
||||
principal = _principal()
|
||||
empty_session = _session(
|
||||
session_id="00000000-0000-0000-0000-00000000b411",
|
||||
session_no=1,
|
||||
learner_id="00000000-0000-0000-0000-000000000333",
|
||||
learner_label="하린",
|
||||
score="neutral",
|
||||
rapport=0.1,
|
||||
technique="reflection",
|
||||
created_at=1_000.0,
|
||||
with_turns=False,
|
||||
)
|
||||
reviewable_session = _session(
|
||||
session_id="00000000-0000-0000-0000-00000000b412",
|
||||
session_no=2,
|
||||
learner_id="00000000-0000-0000-0000-000000000333",
|
||||
learner_label="하린",
|
||||
score="pos",
|
||||
rapport=0.5,
|
||||
technique="reflection",
|
||||
created_at=2_000.0,
|
||||
)
|
||||
|
||||
with (
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_all_sessions",
|
||||
AsyncMock(return_value=([empty_session, reviewable_session], True)),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_safety_alerts",
|
||||
AsyncMock(return_value=([], True)),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_session_review_statuses",
|
||||
AsyncMock(return_value=({}, True)),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_session_evaluations",
|
||||
AsyncMock(return_value=({}, True)),
|
||||
),
|
||||
):
|
||||
response = await teacher.teacher_dashboard(principal)
|
||||
|
||||
self.assertEqual([item.session_id for item in response.pending_reviews], [reviewable_session.session_id])
|
||||
empty_summary = next(
|
||||
item for item in response.recent_sessions if item.session_id == empty_session.session_id
|
||||
)
|
||||
self.assertEqual(empty_summary.turn_count, 0)
|
||||
self.assertEqual(empty_summary.supervisor_state, "기록 대기")
|
||||
self.assertFalse(empty_summary.review_ready)
|
||||
|
||||
async def test_dashboard_surfaces_session_evaluation_failure(self) -> None:
|
||||
principal = _principal()
|
||||
sess = _session(
|
||||
session_id="00000000-0000-0000-0000-00000000b311",
|
||||
session_no=6,
|
||||
learner_id="00000000-0000-0000-0000-000000000333",
|
||||
learner_label="하린",
|
||||
score="neutral",
|
||||
rapport=0.1,
|
||||
technique="reflection",
|
||||
created_at=1_000.0,
|
||||
)
|
||||
evaluation_record = {
|
||||
"status": "error",
|
||||
"source": "engine",
|
||||
"scope": "session_end",
|
||||
"stage": "정리",
|
||||
"payload": {},
|
||||
"error": "session evaluation timeout after 45s",
|
||||
"updated_at": "2026-07-01T00:00:00Z",
|
||||
}
|
||||
|
||||
with (
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_all_sessions",
|
||||
AsyncMock(return_value=([sess], True)),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_safety_alerts",
|
||||
AsyncMock(return_value=([], True)),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_session_review_statuses",
|
||||
AsyncMock(return_value=({}, True)),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_session_evaluations",
|
||||
AsyncMock(return_value=({sess.session_id: evaluation_record}, True)),
|
||||
),
|
||||
):
|
||||
response = await teacher.teacher_dashboard(principal)
|
||||
|
||||
failed = response.pending_reviews[0]
|
||||
self.assertEqual(failed.learner_label, "하린")
|
||||
self.assertEqual(failed.evaluation_status, "error")
|
||||
self.assertFalse(failed.review_ready)
|
||||
self.assertEqual(failed.supervisor_state, "평가 실패")
|
||||
self.assertEqual(failed.evaluation_error, "session evaluation timeout after 45s")
|
||||
|
||||
async def test_dashboard_marks_stale_missing_session_evaluation_as_failed(self) -> None:
|
||||
principal = _principal()
|
||||
sess = _session(
|
||||
session_id="00000000-0000-0000-0000-00000000b411",
|
||||
session_no=7,
|
||||
learner_id="00000000-0000-0000-0000-000000000333",
|
||||
learner_label="하린",
|
||||
score="neutral",
|
||||
rapport=0.1,
|
||||
technique="reflection",
|
||||
created_at=1_000.0,
|
||||
)
|
||||
|
||||
with (
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_all_sessions",
|
||||
AsyncMock(return_value=([sess], True)),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_safety_alerts",
|
||||
AsyncMock(return_value=([], True)),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_session_review_statuses",
|
||||
AsyncMock(return_value=({}, True)),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_session_evaluations",
|
||||
AsyncMock(return_value=({}, True)),
|
||||
),
|
||||
):
|
||||
response = await teacher.teacher_dashboard(principal)
|
||||
|
||||
failed = response.pending_reviews[0]
|
||||
self.assertEqual(failed.learner_label, "하린")
|
||||
self.assertEqual(failed.evaluation_status, "error")
|
||||
self.assertFalse(failed.review_ready)
|
||||
self.assertEqual(failed.supervisor_state, "평가 실패")
|
||||
self.assertEqual(failed.evaluation_error, MISSING_SESSION_EVALUATION_ERROR)
|
||||
|
||||
async def test_dashboard_fails_closed_when_safety_alerts_are_not_durable_outside_dev(self) -> None:
|
||||
principal = _principal()
|
||||
sess = _session(
|
||||
|
|
@ -204,7 +487,7 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase):
|
|||
with (
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_sessions",
|
||||
"list_all_sessions",
|
||||
AsyncMock(return_value=([sess], True)),
|
||||
),
|
||||
patch.object(
|
||||
|
|
@ -217,6 +500,11 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase):
|
|||
"list_session_review_statuses",
|
||||
AsyncMock(return_value=({}, True)),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_session_evaluations",
|
||||
AsyncMock(return_value=({}, True)),
|
||||
),
|
||||
):
|
||||
with self.assertRaises(HTTPException) as raised:
|
||||
await teacher.teacher_dashboard(principal)
|
||||
|
|
@ -244,6 +532,11 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase):
|
|||
"load_session",
|
||||
AsyncMock(return_value=sess),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"load_session_evaluation",
|
||||
AsyncMock(return_value=({"status": "ready"}, True)),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"save_session_review_status",
|
||||
|
|
@ -286,6 +579,46 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase):
|
|||
self.assertIn("보호요인", response.worksheet_note)
|
||||
self.assertEqual(response.worksheet_reviewed_at, "2026-06-27T10:05:00Z")
|
||||
|
||||
async def test_teacher_cannot_close_review_before_ai_session_evaluation_ready(self) -> None:
|
||||
principal = _principal()
|
||||
sess = _session(
|
||||
session_id="00000000-0000-0000-0000-00000000c211",
|
||||
session_no=2,
|
||||
learner_id="00000000-0000-0000-0000-000000000222",
|
||||
score="pos",
|
||||
rapport=0.5,
|
||||
technique="reflection",
|
||||
created_at=3_000.0,
|
||||
)
|
||||
|
||||
with (
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"load_session",
|
||||
AsyncMock(return_value=sess),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"load_session_evaluation",
|
||||
AsyncMock(return_value=({"status": "error", "error": "timeout"}, True)),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"save_session_review_status",
|
||||
AsyncMock(),
|
||||
) as save_status,
|
||||
):
|
||||
with self.assertRaises(HTTPException) as raised:
|
||||
await teacher.update_session_review_status(
|
||||
sess.session_id,
|
||||
teacher.TeacherSessionReviewStatusRequest(status="closed", note="완료"),
|
||||
principal,
|
||||
)
|
||||
|
||||
self.assertEqual(raised.exception.status_code, 409)
|
||||
self.assertIn("session evaluation must be ready", raised.exception.detail)
|
||||
save_status.assert_not_awaited()
|
||||
|
||||
async def test_learner_analysis_returns_full_ordered_history(self) -> None:
|
||||
learner_id = "00000000-0000-0000-0000-000000000111"
|
||||
other_learner_id = "00000000-0000-0000-0000-000000000222"
|
||||
|
|
@ -294,6 +627,7 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase):
|
|||
session_id=f"analysis-session-{idx}",
|
||||
session_no=idx,
|
||||
learner_id=learner_id,
|
||||
learner_label="이서연",
|
||||
score="pos" if idx >= 5 else "neutral",
|
||||
rapport=0.1 * idx,
|
||||
technique="reflection" if idx % 2 else "open question",
|
||||
|
|
@ -319,9 +653,9 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase):
|
|||
with (
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_sessions",
|
||||
"list_all_sessions",
|
||||
AsyncMock(return_value=(sessions, True)),
|
||||
) as list_sessions,
|
||||
) as list_all_sessions,
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_session_review_statuses",
|
||||
|
|
@ -339,17 +673,25 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase):
|
|||
)
|
||||
),
|
||||
),
|
||||
patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_session_evaluations",
|
||||
AsyncMock(return_value=({}, True)),
|
||||
),
|
||||
):
|
||||
response = await teacher.learner_analysis(learner_id, principal)
|
||||
|
||||
list_sessions.assert_awaited_once_with(principal, include_turn_evaluation=True)
|
||||
list_all_sessions.assert_awaited_once_with(principal, include_turn_evaluation=True)
|
||||
self.assertEqual(response.source, "database")
|
||||
self.assertEqual(response.learner_id, learner_id)
|
||||
self.assertEqual(response.learner_label, "이서연")
|
||||
self.assertEqual(response.summary.learner_label, "이서연")
|
||||
self.assertEqual(response.total_sessions, 7)
|
||||
self.assertEqual(response.ended_sessions, 7)
|
||||
self.assertEqual(response.pending_reviews, 6)
|
||||
self.assertEqual(response.closed_reviews, 1)
|
||||
self.assertEqual([session.session_no for session in response.sessions], [1, 2, 3, 4, 5, 6, 7])
|
||||
self.assertTrue(all(session.learner_label == "이서연" for session in response.sessions))
|
||||
self.assertEqual([point.session_no for point in response.points], [1, 2, 3, 4, 5, 6, 7])
|
||||
self.assertEqual(response.sessions[1].review_status, "closed")
|
||||
explore = next(item for item in response.stage_breakdown if item.stage == "탐색")
|
||||
|
|
@ -360,7 +702,7 @@ class TeacherDashboardGrowthTest(unittest.IsolatedAsyncioTestCase):
|
|||
principal = _principal()
|
||||
with patch.object(
|
||||
teacher.session_persistence,
|
||||
"list_sessions",
|
||||
"list_all_sessions",
|
||||
AsyncMock(return_value=([], True)),
|
||||
):
|
||||
with self.assertRaises(HTTPException) as raised:
|
||||
|
|
|
|||
|
|
@ -7,6 +7,8 @@ import unittest
|
|||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
from fastapi import HTTPException
|
||||
|
||||
from .deps import Principal, Role
|
||||
from .persona_repository import PersonaVoiceMap
|
||||
from .routes import voice as voice_routes
|
||||
|
|
@ -17,13 +19,20 @@ SESSION_ID = "voice-ws-contract-session"
|
|||
VOICE_PRESET = VoicePreset(preset="neutral", openai_voice="sage")
|
||||
|
||||
|
||||
def _principal(role: Role = Role.LEARNER) -> Principal:
|
||||
def _principal(
|
||||
role: Role = Role.LEARNER,
|
||||
*,
|
||||
consent_at: float | None = 1.0,
|
||||
profile_completed_at: float | None = 1.0,
|
||||
) -> Principal:
|
||||
return Principal(
|
||||
user_id="00000000-0000-0000-0000-000000000201",
|
||||
role=role,
|
||||
cohort_ids=[],
|
||||
email=f"voice-ws-{role.value}@hs.ac.kr",
|
||||
display_name="Voice WS Contract",
|
||||
consent_at=consent_at if role == Role.LEARNER else None,
|
||||
profile_completed_at=profile_completed_at if role == Role.LEARNER else None,
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -318,6 +327,55 @@ class VoiceWebSocketContractTest(unittest.IsolatedAsyncioTestCase):
|
|||
self.assertEqual(kwargs["voice_preset"], VOICE_PRESET)
|
||||
self.assertEqual(kwargs["learner_text"], "I need help practicing.")
|
||||
|
||||
async def test_turn_persistence_failure_uses_structured_voice_error(self) -> None:
|
||||
websocket = FakeWebSocket(
|
||||
[
|
||||
_control({"type": "text_turn", "text": "I need help practicing."}),
|
||||
_control({"type": "close"}),
|
||||
]
|
||||
)
|
||||
run_turn = AsyncMock(
|
||||
side_effect=HTTPException(
|
||||
status_code=503,
|
||||
detail=(
|
||||
"voice session turn append persistence unavailable; "
|
||||
"runtime fallback is disabled in prod"
|
||||
),
|
||||
)
|
||||
)
|
||||
|
||||
with patch.object(
|
||||
voice_routes,
|
||||
"_principal_from_websocket",
|
||||
AsyncMock(return_value=_principal()),
|
||||
), patch.object(
|
||||
voice_routes,
|
||||
"_bind_session",
|
||||
AsyncMock(return_value=self._bind_result()),
|
||||
), patch.object(
|
||||
voice_routes.voice_service,
|
||||
"is_available",
|
||||
return_value=True,
|
||||
), patch.object(
|
||||
voice_routes,
|
||||
"_run_turn_and_speak",
|
||||
run_turn,
|
||||
):
|
||||
await voice_routes.voice_ws(websocket) # type: ignore[arg-type]
|
||||
|
||||
self.assertEqual(websocket.close_codes, [1000])
|
||||
self.assertEqual(websocket.sent_json[0].get("type"), "ready")
|
||||
self.assertEqual(websocket.sent_json[0].get("state"), "idle")
|
||||
self.assertEqual(
|
||||
websocket.sent_json[-2],
|
||||
{
|
||||
"type": "error",
|
||||
"code": "turn_persistence_unavailable",
|
||||
"detail": "voice turn persistence unavailable; retry the utterance",
|
||||
},
|
||||
)
|
||||
self.assertEqual(websocket.sent_json[-1], {"type": "state", "state": "idle"})
|
||||
|
||||
async def test_stt_result_waits_for_final_transcript_before_running_turn(self) -> None:
|
||||
websocket = FakeWebSocket(
|
||||
[
|
||||
|
|
@ -606,6 +664,60 @@ class VoiceWebSocketContractTest(unittest.IsolatedAsyncioTestCase):
|
|||
))
|
||||
get_voice_map.assert_awaited_once_with(SESSION_ID)
|
||||
|
||||
async def test_bind_session_rejects_existing_session_without_onboarding(self) -> None:
|
||||
websocket = FakeWebSocket()
|
||||
websocket.query_params = {"session_id": SESSION_ID}
|
||||
load_voice_session = AsyncMock()
|
||||
|
||||
with patch.object(
|
||||
voice_routes,
|
||||
"user_onboarding_complete",
|
||||
AsyncMock(return_value=False),
|
||||
), patch.object(
|
||||
voice_routes,
|
||||
"user_has_consent",
|
||||
AsyncMock(return_value=True),
|
||||
), patch.object(
|
||||
voice_routes,
|
||||
"_load_voice_session",
|
||||
load_voice_session,
|
||||
):
|
||||
session_id, voice, err, meta = await voice_routes._bind_session(
|
||||
websocket,
|
||||
_principal(profile_completed_at=None),
|
||||
)
|
||||
|
||||
self.assertIsNone(session_id)
|
||||
self.assertIsNone(voice)
|
||||
self.assertEqual(err, "onboarding_required")
|
||||
self.assertEqual(meta, {})
|
||||
load_voice_session.assert_not_awaited()
|
||||
|
||||
async def test_bind_session_rejects_existing_session_without_consent(self) -> None:
|
||||
websocket = FakeWebSocket()
|
||||
websocket.query_params = {"session_id": SESSION_ID}
|
||||
load_voice_session = AsyncMock()
|
||||
|
||||
with patch.object(
|
||||
voice_routes,
|
||||
"user_has_consent",
|
||||
AsyncMock(return_value=False),
|
||||
), patch.object(
|
||||
voice_routes,
|
||||
"_load_voice_session",
|
||||
load_voice_session,
|
||||
):
|
||||
session_id, voice, err, meta = await voice_routes._bind_session(
|
||||
websocket,
|
||||
_principal(consent_at=None),
|
||||
)
|
||||
|
||||
self.assertIsNone(session_id)
|
||||
self.assertIsNone(voice)
|
||||
self.assertEqual(err, "consent_required")
|
||||
self.assertEqual(meta, {})
|
||||
load_voice_session.assert_not_awaited()
|
||||
|
||||
async def test_bind_session_explicit_preset_overrides_db_voice_map(self) -> None:
|
||||
websocket = FakeWebSocket()
|
||||
websocket.query_params = {"session_id": SESSION_ID, "preset": "soft-young-fem"}
|
||||
|
|
|
|||
|
|
@ -19,6 +19,9 @@ from .store import InProcSession, TurnRecord, store
|
|||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_LIVE_COACH_RECHARGE_MIN_RAPPORT = 0.35
|
||||
_LIVE_COACH_RECHARGE_MIN_OPENNESS_GAIN = 0.02
|
||||
|
||||
|
||||
class SessionAccessError(str, Enum):
|
||||
NOT_FOUND = "not_found"
|
||||
|
|
@ -179,6 +182,63 @@ async def record_safety_event(
|
|||
require_runtime_fallback_allowed("safety event")
|
||||
|
||||
|
||||
def should_recharge_live_coach_credit(
|
||||
evaluation: dict | None,
|
||||
before: state_machine.SessionState,
|
||||
after: state_machine.SessionState,
|
||||
) -> tuple[bool, str]:
|
||||
"""Good-score recharge gate based on stored evaluator/state-machine evidence."""
|
||||
if not isinstance(evaluation, dict):
|
||||
return False, ""
|
||||
if evaluation.get("appropriateness") != "pos":
|
||||
return False, ""
|
||||
try:
|
||||
rapport = float(evaluation.get("rapport_signal") or 0)
|
||||
except (TypeError, ValueError):
|
||||
rapport = 0.0
|
||||
if rapport < _LIVE_COACH_RECHARGE_MIN_RAPPORT:
|
||||
return False, ""
|
||||
|
||||
openness_gain = float(after.effective_openness or 0) - float(before.effective_openness or 0)
|
||||
stage_changed = after.stage != before.stage
|
||||
if not stage_changed and openness_gain < _LIVE_COACH_RECHARGE_MIN_OPENNESS_GAIN:
|
||||
return False, ""
|
||||
if stage_changed:
|
||||
return True, "좋은 발화로 내담자 단계가 열려 코칭 기회 1개를 충전했습니다."
|
||||
return True, "좋은 발화 뒤 내담자 개방도가 올라 코칭 기회 1개를 충전했습니다."
|
||||
|
||||
|
||||
async def maybe_recharge_live_coach_credit(
|
||||
sess: InProcSession,
|
||||
ctx: orchestrator.TurnContext,
|
||||
result: orchestrator.TurnResult,
|
||||
) -> None:
|
||||
"""Record one live-coach recharge when a strong learner turn changes client state."""
|
||||
assert ctx.state_after is not None
|
||||
should_recharge, reason = should_recharge_live_coach_credit(
|
||||
result.evaluation,
|
||||
ctx.state_before,
|
||||
result.state_after,
|
||||
)
|
||||
if not should_recharge:
|
||||
return
|
||||
try:
|
||||
await session_persistence.record_live_coach_recharge(
|
||||
session_id=sess.session_id,
|
||||
learner_id=sess.learner_id,
|
||||
turn_seq=result.turn_seq,
|
||||
stage=stage_label(result.state_after.stage),
|
||||
reason=reason,
|
||||
)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"live coach recharge persistence failed: session_id=%s turn_seq=%s",
|
||||
sess.session_id,
|
||||
result.turn_seq,
|
||||
)
|
||||
require_runtime_fallback_allowed("live coach recharge")
|
||||
|
||||
|
||||
async def finalize_completed_turn(
|
||||
sess: InProcSession,
|
||||
ctx: orchestrator.TurnContext,
|
||||
|
|
@ -195,6 +255,7 @@ async def finalize_completed_turn(
|
|||
context_prefix=context_prefix,
|
||||
counselor_turn=counselor_turn,
|
||||
)
|
||||
await maybe_recharge_live_coach_credit(sess, ctx, result)
|
||||
await record_safety_event(sess, ctx, result)
|
||||
|
||||
|
||||
|
|
@ -203,8 +264,10 @@ __all__ = [
|
|||
"append_completed_turn",
|
||||
"finalize_completed_turn",
|
||||
"load_owned_session",
|
||||
"maybe_recharge_live_coach_credit",
|
||||
"record_safety_event",
|
||||
"record_completed_turn",
|
||||
"should_recharge_live_coach_credit",
|
||||
"stage_label",
|
||||
"update_session_state",
|
||||
]
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue