세션 평가·라이브코치·교수자 분석 라운드 마감 + 문서 정리 + 코드품질 리팩터

- 누적 작업트리 커밋: 회기 평가 복구·durable 저장, 라이브 코치 이력/근거, 교수자 학생분석, 음성 비언어 메타, PII 마스킹, 운영 티켓/헬스 등
- 문서: 완료 기록 docs/archive/ 냉동 보관, docs/ 단일 인덱스(docs/README.md)+통합 TODO(docs/TODO.md)로 정리
- 리팩터(행위 보존): Stage enum SSOT(taxonomy 소유·state_machine re-export), store recent/masked_turns 중복 제거, speaker_ko_label 단일 헬퍼, _list_sessions N+1 제거(state/turns 배치 + 턴평가 하이드레이션 배치)
- 검증: 백엔드 pytest 352 passed, _list_sessions E2E chromium-single-run 2 passed
This commit is contained in:
Yun Chan 2026-07-02 02:50:36 +09:00
parent 7c41c3ce79
commit 778e8526d4
108 changed files with 6457 additions and 455 deletions

View file

@ -4,6 +4,18 @@ const sessionId = "33333333-3333-4333-8333-333333333333";
const learnerText = "요즘 많이 힘들었겠어요. 어떤 마음이 가장 크게 남아 있나요?";
const clientReply = "괜찮아요. 천천히 말해볼게요.";
declare global {
interface Window {
__voiceErrorFixture?: {
releaseError: () => void;
sent: string[];
};
__voiceCrisisFixture?: {
sent: string[];
};
}
}
function deferred() {
let resolve!: () => void;
const promise = new Promise<void>((next) => {
@ -16,11 +28,19 @@ interface RouteMvpOptions {
endStatus?: number;
endBody?: unknown;
crisisStream?: boolean;
liveCoachStatus?: "ready" | "degraded";
liveCoachHistoryStatus?: number;
liveCoachHistorySource?: "database" | "runtime";
liveCoachPersistenceSource?: "database" | "runtime";
liveCoachHistoryQuotas?: Array<{ remaining: number; max: number }>;
liveCoachSuggestionQuota?: { remaining: number; max: number };
}
async function routeMvpApi(page: Page, options: RouteMvpOptions = {}) {
const sessionStartRequests: unknown[] = [];
const liveCoachRequests: unknown[] = [];
let liveCoachHistoryRequests = 0;
let deliveredCoachSuggestion: Record<string, unknown> | null = null;
const streamSeen = deferred();
const streamGate = deferred();
@ -62,6 +82,14 @@ async function routeMvpApi(page: Page, options: RouteMvpOptions = {}) {
});
});
await page.route("**/api/voice/health", async (route) => {
await route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({ available: true, reason: null }),
});
});
await page.route("**/api/personas", async (route) => {
await route.fulfill({
status: 200,
@ -198,38 +226,76 @@ async function routeMvpApi(page: Page, options: RouteMvpOptions = {}) {
await page.route(`**/api/sessions/${sessionId}/live-coach`, async (route) => {
const request = route.request();
if (request.method() === "GET") {
const quota =
options.liveCoachHistoryQuotas?.[
Math.min(liveCoachHistoryRequests, options.liveCoachHistoryQuotas.length - 1)
] ?? { remaining: 3, max: 3 };
liveCoachHistoryRequests += 1;
if (options.liveCoachHistoryStatus && options.liveCoachHistoryStatus >= 400) {
await route.fulfill({
status: options.liveCoachHistoryStatus,
contentType: "application/json",
body: JSON.stringify({ detail: "live coach history unavailable" }),
});
return;
}
await route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({ source: "runtime", events: [] }),
body: JSON.stringify({
source: options.liveCoachHistorySource ?? "database",
quota,
credit_events: [],
events: deliveredCoachSuggestion
? [
{
event_id: "fixture-coach-1",
session_id: sessionId,
turn_seq: 1,
stage: "탐색",
created_at: "2026-06-26T00:00:03.000Z",
learner_text_excerpt: learnerText,
client_reply_excerpt: clientReply,
suggestion: deliveredCoachSuggestion,
},
]
: [],
}),
});
return;
}
liveCoachRequests.push(request.postDataJSON());
deliveredCoachSuggestion = {
status: options.liveCoachStatus ?? "ready",
tone: "pos",
focus: "reflection",
title: "감정 반영이 선명합니다",
message: "학습자가 내담자의 감정을 평가하지 않고 먼저 되짚었습니다.",
next_utterance: "그 마음이 가장 크게 올라온 장면을 조금 더 들려줄 수 있을까요?",
rationale:
options.liveCoachStatus === "degraded"
? "AI 코칭 엔진 응답 대신 워크북 규칙과 현재 턴 신호로 만든 대체 판단입니다."
: "방금 발화는 정서 반영과 개방 질문을 함께 포함합니다.",
sources: [
{
source_id: "live_coaching_workbook_0615",
title: "0615 사례개념화 워크북",
locator: "reflection",
kb_kind: "source_pack",
version: "2026-06-15",
citation: "허가된 요약 근거",
},
],
safety_note: null,
latency_ms: options.liveCoachStatus === "degraded" ? 0 : 12,
persistence_source: options.liveCoachPersistenceSource ?? "database",
quota: options.liveCoachSuggestionQuota ?? { remaining: 2, max: 3 },
credit_events: [],
};
await route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({
status: "ready",
tone: "pos",
focus: "reflection",
title: "감정 반영이 선명합니다",
message: "학습자가 내담자의 감정을 평가하지 않고 먼저 되짚었습니다.",
next_utterance: "그 마음이 가장 크게 올라온 장면을 조금 더 들려줄 수 있을까요?",
rationale: "방금 발화는 정서 반영과 개방 질문을 함께 포함합니다.",
sources: [
{
source_id: "live_coaching_workbook_0615",
title: "0615 사례개념화 워크북",
locator: "reflection",
kb_kind: "source_pack",
version: "2026-06-15",
citation: "허가된 요약 근거",
},
],
safety_note: null,
latency_ms: 12,
}),
body: JSON.stringify(deliveredCoachSuggestion),
});
});
@ -290,7 +356,277 @@ async function routeMvpApi(page: Page, options: RouteMvpOptions = {}) {
});
});
return { sessionStartRequests, liveCoachRequests, streamGate, streamSeen };
return {
sessionStartRequests,
liveCoachRequests,
streamGate,
streamSeen,
get liveCoachHistoryRequests() {
return liveCoachHistoryRequests;
},
};
}
async function installVoicePersistenceFailureFixture(page: Page, transcriptText: string) {
await page.addInitScript((text) => {
const fixture = {
sent: [] as string[],
socket: null as FixtureWebSocket | null,
releaseError() {
const socket = fixture.socket;
if (!socket || socket.readyState >= FixtureWebSocket.CLOSING) return;
socket.emitJson({
type: "error",
code: "turn_persistence_unavailable",
detail: "voice turn persistence unavailable; retry the utterance",
});
socket.close(1000);
},
};
window.__voiceErrorFixture = fixture;
const fakeTrack = {
kind: "audio",
readyState: "live",
stop() {
this.readyState = "ended";
},
};
const fakeStream = {
active: true,
getTracks: () => [fakeTrack],
getAudioTracks: () => [fakeTrack],
};
Object.defineProperty(navigator, "mediaDevices", {
configurable: true,
value: { getUserMedia: async () => fakeStream },
});
Object.defineProperty(window, "AudioContext", { configurable: true, value: undefined });
class FakeMediaRecorder extends EventTarget {
static isTypeSupported() {
return true;
}
state = "inactive";
mimeType = "audio/webm";
ondataavailable: ((event: Event & { data: Blob }) => void) | null = null;
onstop: ((event: Event) => void) | null = null;
constructor(_stream: unknown, options?: { mimeType?: string }) {
super();
this.mimeType = options?.mimeType ?? "audio/webm";
}
start() {
this.state = "recording";
}
stop() {
if (this.state === "inactive") return;
this.state = "inactive";
const event = new Event("stop");
this.onstop?.(event);
this.dispatchEvent(event);
}
}
Object.defineProperty(window, "MediaRecorder", {
configurable: true,
value: FakeMediaRecorder,
});
class FixtureWebSocket {
static CONNECTING = 0;
static OPEN = 1;
static CLOSING = 2;
static CLOSED = 3;
readyState = FixtureWebSocket.CONNECTING;
binaryType: BinaryType = "blob";
onopen: ((event: Event) => void) | null = null;
onmessage: ((event: MessageEvent) => void) | null = null;
onerror: ((event: Event) => void) | null = null;
onclose: ((event: CloseEvent) => void) | null = null;
constructor(_url: string | URL) {
fixture.socket = this;
window.setTimeout(() => {
if (this.readyState !== FixtureWebSocket.CONNECTING) return;
this.readyState = FixtureWebSocket.OPEN;
this.onopen?.(new Event("open"));
this.emitJson({ type: "ready", state: "idle" });
}, 0);
}
send(data: string | ArrayBufferLike | Blob | ArrayBufferView) {
if (typeof data !== "string") return;
fixture.sent.push(data);
let payload: { type?: string } = {};
try {
payload = JSON.parse(data) as { type?: string };
} catch {
return;
}
if (payload.type === "audio_end") {
window.setTimeout(() => {
this.emitJson({ type: "state", state: "thinking" });
this.emitJson({ type: "transcript", text, final: true });
}, 0);
}
}
close(code = 1000) {
if (this.readyState === FixtureWebSocket.CLOSED) return;
this.readyState = FixtureWebSocket.CLOSED;
const event = new Event("close") as CloseEvent;
Object.defineProperty(event, "code", { value: code });
this.onclose?.(event);
}
emitJson(payload: unknown) {
if (this.readyState === FixtureWebSocket.CLOSED) return;
const event = new MessageEvent("message", { data: JSON.stringify(payload) });
this.onmessage?.(event);
}
}
Object.defineProperty(window, "WebSocket", {
configurable: true,
value: FixtureWebSocket as unknown as typeof WebSocket,
});
}, transcriptText);
}
async function installVoiceCrisisFixture(page: Page, transcriptText: string) {
await page.addInitScript((text) => {
const crisisResource = {
title: "자살예방상담전화 109",
number: "109",
message: "지금은 연습을 멈추고 실제 안전 확인이 먼저입니다.",
};
const fixture = {
sent: [] as string[],
socket: null as FixtureWebSocket | null,
};
window.__voiceCrisisFixture = fixture;
const fakeTrack = {
kind: "audio",
readyState: "live",
stop() {
this.readyState = "ended";
},
};
const fakeStream = {
active: true,
getTracks: () => [fakeTrack],
getAudioTracks: () => [fakeTrack],
};
Object.defineProperty(navigator, "mediaDevices", {
configurable: true,
value: { getUserMedia: async () => fakeStream },
});
Object.defineProperty(window, "AudioContext", { configurable: true, value: undefined });
class FakeMediaRecorder extends EventTarget {
static isTypeSupported() {
return true;
}
state = "inactive";
mimeType = "audio/webm";
ondataavailable: ((event: Event & { data: Blob }) => void) | null = null;
onstop: ((event: Event) => void) | null = null;
constructor(_stream: unknown, options?: { mimeType?: string }) {
super();
this.mimeType = options?.mimeType ?? "audio/webm";
}
start() {
this.state = "recording";
}
stop() {
if (this.state === "inactive") return;
this.state = "inactive";
const event = new Event("stop");
this.onstop?.(event);
this.dispatchEvent(event);
}
}
Object.defineProperty(window, "MediaRecorder", {
configurable: true,
value: FakeMediaRecorder,
});
class FixtureWebSocket {
static CONNECTING = 0;
static OPEN = 1;
static CLOSING = 2;
static CLOSED = 3;
readyState = FixtureWebSocket.CONNECTING;
binaryType: BinaryType = "blob";
onopen: ((event: Event) => void) | null = null;
onmessage: ((event: MessageEvent) => void) | null = null;
onerror: ((event: Event) => void) | null = null;
onclose: ((event: CloseEvent) => void) | null = null;
constructor(_url: string | URL) {
fixture.socket = this;
window.setTimeout(() => {
if (this.readyState !== FixtureWebSocket.CONNECTING) return;
this.readyState = FixtureWebSocket.OPEN;
this.onopen?.(new Event("open"));
this.emitJson({ type: "ready", state: "idle" });
}, 0);
}
send(data: string | ArrayBufferLike | Blob | ArrayBufferView) {
if (typeof data !== "string") return;
fixture.sent.push(data);
let payload: { type?: string } = {};
try {
payload = JSON.parse(data) as { type?: string };
} catch {
return;
}
if (payload.type === "audio_end") {
window.setTimeout(() => {
this.emitJson({ type: "state", state: "thinking" });
this.emitJson({ type: "transcript", text, final: true });
this.emitJson({
type: "reply",
text: "",
safety_flagged: true,
crisis_resource: crisisResource,
conversation_stopped: true,
turn_seq: 1,
});
this.emitJson({ type: "state", state: "idle" });
}, 0);
}
}
close(code = 1000) {
if (this.readyState === FixtureWebSocket.CLOSED) return;
this.readyState = FixtureWebSocket.CLOSED;
const event = new Event("close") as CloseEvent;
Object.defineProperty(event, "code", { value: code });
this.onclose?.(event);
}
emitJson(payload: unknown) {
if (this.readyState === FixtureWebSocket.CLOSED) return;
const event = new MessageEvent("message", { data: JSON.stringify(payload) });
this.onmessage?.(event);
}
}
Object.defineProperty(window, "WebSocket", {
configurable: true,
value: FixtureWebSocket as unknown as typeof WebSocket,
});
}, transcriptText);
}
test.describe("P1 MVP core loop", () => {
@ -354,6 +690,14 @@ test.describe("P1 MVP core loop", () => {
await expect(
page.getByText("학습자가 내담자의 감정을 평가하지 않고 먼저 되짚었습니다."),
).toBeVisible();
await page.locator(".sx-coach-card").getByRole("button", { name: "근거 보기" }).click();
const evidenceDialog = page.locator(".sx-coach-modal [role='dialog']");
await expect(evidenceDialog).toBeVisible();
await expect(evidenceDialog).toContainText("0615 사례개념화 워크북");
await expect(evidenceDialog).toContainText("reflection · source_pack · 2026-06-15");
await expect(evidenceDialog).toContainText("허가된 요약 근거");
await evidenceDialog.getByRole("button", { name: "닫기" }).click();
await expect(evidenceDialog).toHaveCount(0);
expect(api.liveCoachRequests).toHaveLength(1);
expect(api.liveCoachRequests[0]).toMatchObject({
learner_text: learnerText,
@ -362,6 +706,209 @@ test.describe("P1 MVP core loop", () => {
});
});
test("refreshes stale empty AI tutor quota before blocking coaching", async ({ page }) => {
const api = await routeMvpApi(page, {
liveCoachHistoryQuotas: [
{ remaining: 0, max: 3 },
{ remaining: 1, max: 3 },
{ remaining: 0, max: 3 },
],
liveCoachSuggestionQuota: { remaining: 0, max: 3 },
});
await page.goto("/learn/session/P1");
await page.getByRole("button", { name: "회기 시작" }).click();
await page.getByRole("button", { name: "코칭" }).click();
await page.getByLabel("학습자 발화 입력").fill(learnerText);
await page.getByRole("button", { name: "보내기" }).click();
await api.streamSeen.promise;
api.streamGate.resolve();
await expect(page.locator(".sx-coach-card").getByText("감정 반영이 선명합니다")).toBeVisible();
expect(api.liveCoachRequests).toHaveLength(1);
await expect.poll(() => api.liveCoachHistoryRequests).toBeGreaterThanOrEqual(2);
});
test("shows refreshed AI tutor quota exhaustion over a stale coaching card", async ({
page,
}) => {
const api = await routeMvpApi(page, {
liveCoachHistoryQuotas: [
{ remaining: 0, max: 3 },
{ remaining: 0, max: 3 },
],
liveCoachSuggestionQuota: { remaining: 0, max: 3 },
});
await page.goto("/learn/session/P1");
await page.getByRole("button", { name: "회기 시작" }).click();
await page.getByRole("button", { name: "코칭" }).click();
await page.getByLabel("학습자 발화 입력").fill(learnerText);
await page.getByRole("button", { name: "보내기" }).click();
await api.streamSeen.promise;
api.streamGate.resolve();
const coachCard = page.locator(".sx-coach-card");
await expect(coachCard.getByText("감정 반영이 선명합니다")).toBeVisible();
expect(api.liveCoachRequests).toHaveLength(1);
await page.getByLabel("학습자 발화 입력").fill("그때 마음을 더 자세히 말해줘도 괜찮아요.");
await page.getByRole("button", { name: "보내기" }).click();
await expect(coachCard.getByText("코칭 기회를 모두 사용했습니다.")).toBeVisible();
await expect(coachCard.getByText("감정 반영이 선명합니다")).toHaveCount(0);
expect(api.liveCoachRequests).toHaveLength(1);
});
test("surfaces AI tutor history load failure instead of an empty history", async ({ page }) => {
const api = await routeMvpApi(page, { liveCoachHistoryStatus: 503 });
await page.goto("/learn/session/P1");
await page.getByRole("button", { name: "회기 시작" }).click();
await page.getByRole("button", { name: "코칭" }).click();
await page.getByLabel("학습자 발화 입력").fill(learnerText);
await page.getByRole("button", { name: "보내기" }).click();
await api.streamSeen.promise;
api.streamGate.resolve();
const coachCard = page.locator(".sx-coach-card");
await expect(coachCard.getByText("감정 반영이 선명합니다")).toBeVisible();
await coachCard.getByRole("button", { name: "이력 보기" }).click();
const historyDialog = page.locator(".sx-coach-history [role='dialog']");
await expect(historyDialog).toBeVisible();
await expect(historyDialog.getByRole("alert")).toContainText("코칭 이력을 불러오지 못했습니다.");
await expect(historyDialog).not.toContainText("아직 이 턴에 저장된 코칭 이력이 없습니다.");
expect(api.liveCoachRequests).toHaveLength(1);
});
test("labels runtime AI tutor persistence as temporary history", async ({ page }) => {
const api = await routeMvpApi(page, {
liveCoachHistorySource: "runtime",
liveCoachPersistenceSource: "runtime",
});
await page.goto("/learn/session/P1");
await page.getByRole("button", { name: "회기 시작" }).click();
await page.getByRole("button", { name: "코칭" }).click();
await page.getByLabel("학습자 발화 입력").fill(learnerText);
await page.getByRole("button", { name: "보내기" }).click();
await api.streamSeen.promise;
api.streamGate.resolve();
const coachCard = page.locator(".sx-coach-card");
await expect(coachCard.getByText("감정 반영이 선명합니다")).toBeVisible();
await expect(coachCard).toContainText("임시 이력");
await expect(coachCard).toContainText("DB에 확정 저장되지 않아");
await coachCard.getByRole("button", { name: "이력 보기" }).click();
const historyDialog = page.locator(".sx-coach-history [role='dialog']");
await expect(historyDialog).toContainText("임시 저장소 기준");
expect(api.liveCoachRequests).toHaveLength(1);
});
test("marks pending voice transcript as failed when turn persistence fails @single-run", async ({
page,
}) => {
const failedVoiceText = "요즘 잠을 잘 못 자요.";
await routeMvpApi(page);
await installVoicePersistenceFailureFixture(page, failedVoiceText);
await page.goto("/learn/session/P1");
await page.getByRole("button", { name: "회기 시작" }).click();
const micButton = page.getByRole("button", { name: "마이크 켜기" });
await expect(micButton).toBeEnabled();
await micButton.click();
await page.getByRole("button", { name: "발화 보내기" }).click();
const learnerBubble = page.locator(".sx-utt.is-learner").filter({ hasText: failedVoiceText });
await expect(learnerBubble).toBeVisible();
await expect(learnerBubble).toHaveClass(/is-partial/);
await expect(
page.locator(".sx-utt.is-thinking").filter({ hasText: "답변을 준비 중입니다." }),
).toBeVisible();
await page.evaluate(() => window.__voiceErrorFixture?.releaseError());
await expect(learnerBubble).toBeVisible();
await expect(learnerBubble).toHaveClass(/is-failed/);
await expect(learnerBubble).not.toHaveClass(/is-partial/);
await expect(learnerBubble.getByText("저장 실패 · 다시 시도하세요.")).toBeVisible();
await expect(page.locator(".sx-utt.is-thinking")).toHaveCount(0);
await expect(page.getByRole("alert")).toContainText("음성 발화를 저장하지 못했습니다.");
await expect(page.getByLabel("학습자 발화 입력")).toBeEnabled();
await expect(page.locator(".sx-mic-block__l")).toContainText("마이크 오류");
});
test("keeps crisis safety gate visible for a voice conversation stop", async ({ page }) => {
const crisisText = "죽고 싶다는 생각이 자꾸 들어요.";
const api = await routeMvpApi(page);
await installVoiceCrisisFixture(page, crisisText);
await page.goto("/learn/session/P1");
await page.getByRole("button", { name: "회기 시작" }).click();
await page.getByRole("button", { name: "코칭" }).click();
const micButton = page.getByRole("button", { name: "마이크 켜기" });
await expect(micButton).toBeEnabled();
await micButton.click();
await page.getByRole("button", { name: "발화 보내기" }).click();
const learnerBubble = page.locator(".sx-utt.is-learner").filter({ hasText: crisisText });
await expect(learnerBubble).toBeVisible();
await expect(learnerBubble).not.toHaveClass(/is-partial/);
await expect(learnerBubble).not.toHaveClass(/is-failed/);
await expect(page.locator(".sx-crisis-resource").getByText("자살예방상담전화 109")).toBeVisible();
await expect(page.getByRole("link", { name: "109" })).toBeVisible();
await expect(page.getByText("내담자 응답 없음")).toHaveCount(0);
await expect(page.locator(".sx-utt.is-client")).toHaveCount(0);
await expect(page.getByLabel("학습자 발화 입력")).toBeDisabled();
await expect(page.getByRole("button", { name: "마이크 켜기" })).toBeDisabled();
const sent = await page.evaluate(() => window.__voiceCrisisFixture?.sent ?? []);
expect(sent.some((payload) => payload.includes("\"audio_end\""))).toBe(true);
expect(sent.some((payload) => payload.includes("\"close\""))).toBe(true);
expect(api.liveCoachRequests).toHaveLength(0);
});
test("labels degraded AI tutor fallback as replacement coaching", async ({ page }) => {
const api = await routeMvpApi(page, { liveCoachStatus: "degraded" });
await page.goto("/learn/session/P1");
await page.getByRole("button", { name: "회기 시작" }).click();
await page.getByRole("button", { name: "코칭" }).click();
await page.getByLabel("학습자 발화 입력").fill(learnerText);
await page.getByRole("button", { name: "보내기" }).click();
await api.streamSeen.promise;
api.streamGate.resolve();
const coachCard = page.locator(".sx-coach-card");
await expect(coachCard).toContainText("대체 코칭");
await expect(coachCard).toContainText("AI 응답 대체");
await expect(coachCard).toContainText("AI 코칭 엔진 응답 대신");
expect(api.liveCoachRequests).toHaveLength(1);
await coachCard.getByRole("button", { name: "근거 보기" }).click();
const evidenceDialog = page.locator(".sx-coach-modal [role='dialog']");
await expect(evidenceDialog).toBeVisible();
await expect(evidenceDialog).toContainText("AI 코칭 엔진 응답 대신");
await expect(evidenceDialog).toContainText("0615 사례개념화 워크북");
await evidenceDialog.getByRole("button", { name: "닫기" }).click();
await expect(evidenceDialog).toHaveCount(0);
const coachMark = page.locator(".sx-utt__coach-mark.is-degraded");
await expect(coachMark).toBeVisible();
await expect(coachMark).toHaveAttribute("title", /AI 응답 대체/);
await coachMark.click();
const historyDialog = page.locator(".sx-coach-history [role='dialog']");
await expect(historyDialog).toContainText("AI 응답 대체");
await expect(historyDialog).toContainText("0615 사례개념화 워크북");
});
test("keeps crisis safety gate visible instead of showing empty client reply @single-run", async ({ page }) => {
const api = await routeMvpApi(page, { crisisStream: true });