"""G7 텍스트·음성 독립 측정과 검증된 경우에만 적용하는 보정 융합.""" from __future__ import annotations from collections.abc import Iterable, Mapping from pathlib import Path from typing import Any from ..contracts.multimodal_alliance import ( AlignedVoiceTimeline, CalibratedAxisReadModel, FusionCalibration, ModalityAxisMeasurement, MultimodalBenchmarkPack, VoiceInteractionEvent, WordTimestamp, ) def align_voice_timeline( *, audio_duration_ms: int, words: Iterable[WordTimestamp | Mapping[str, Any]], events: Iterable[VoiceInteractionEvent | Mapping[str, Any]], ) -> AlignedVoiceTimeline: """이미 추출된 관찰 이벤트를 오디오 시계에 정렬하고 범위를 검증한다.""" validated_words = tuple(WordTimestamp.model_validate(item) for item in words) validated_events = tuple( VoiceInteractionEvent.model_validate(item) for item in events ) return AlignedVoiceTimeline( audio_duration_ms=audio_duration_ms, words=tuple( sorted( validated_words, key=lambda item: (item.start_ms, item.word_index), ) ), events=tuple( sorted( validated_events, key=lambda item: (item.start_ms, item.event_id), ) ), ) def build_calibrated_axis_read_model( *, text: ModalityAxisMeasurement, voice: ModalityAxisMeasurement, calibration: FusionCalibration, ) -> CalibratedAxisReadModel: if text.axis != voice.axis or text.axis != calibration.axis: raise ValueError("text, voice, and fusion calibration must target one axis") if text.status != "ready": return CalibratedAxisReadModel( axis=text.axis, status=text.status, value=None, uncertainty=1.0, modalities_used=(), measurement_ids=(text.measurement_id,), fusion_applied=False, counterevidence=("text_measurement_not_ready",), ) assert text.value is not None voice_ready = voice.status == "ready" and voice.value is not None gain_sufficient = ( calibration.incremental_gain >= calibration.minimum_incremental_gain ) if not voice_ready or not gain_sufficient: counterevidence: list[str] = [] if not voice_ready: counterevidence.append("voice_measurement_not_ready") if voice_ready and not gain_sufficient: counterevidence.append("voice_incremental_gain_not_demonstrated") return CalibratedAxisReadModel( axis=text.axis, status="ready", value=text.value, uncertainty=text.uncertainty, modalities_used=("text",), measurement_ids=(text.measurement_id,), fusion_applied=False, incremental_gain=calibration.incremental_gain, counterevidence=tuple(counterevidence), ) assert voice.value is not None fused = ( text.value * calibration.text_weight + voice.value * calibration.voice_weight ) uncertainty = min( 1.0, text.uncertainty * calibration.text_weight + voice.uncertainty * calibration.voice_weight, ) return CalibratedAxisReadModel( axis=text.axis, status="ready", value=fused, uncertainty=uncertainty, modalities_used=("text", "voice"), measurement_ids=(text.measurement_id, voice.measurement_id), fusion_applied=True, fusion_calibration_id=calibration.calibration_id, incremental_gain=calibration.incremental_gain, ) def load_multimodal_benchmark(path: str | Path) -> MultimodalBenchmarkPack: return MultimodalBenchmarkPack.model_validate_json( Path(path).read_text(encoding="utf-8") ) def evaluate_multimodal_benchmark(pack: MultimodalBenchmarkPack) -> dict[str, object]: cases: list[dict[str, object]] = [] correct = 0 text_absolute_errors: list[float] = [] calibrated_absolute_errors: list[float] = [] minimum_gain = min( case.calibration.minimum_incremental_gain for case in pack.cases ) for case in pack.cases: result = build_calibrated_axis_read_model( text=case.text_measurement, voice=case.voice_measurement, calibration=case.calibration, ) matched = result.fusion_applied == case.expected_fusion_applied correct += matched assert case.text_measurement.value is not None assert result.value is not None text_absolute_error = abs(case.text_measurement.value - case.target_value) calibrated_absolute_error = abs(result.value - case.target_value) text_absolute_errors.append(text_absolute_error) calibrated_absolute_errors.append(calibrated_absolute_error) cases.append( { "case_id": case.case_id, "matched": matched, "target_value": case.target_value, "text_absolute_error": text_absolute_error, "calibrated_absolute_error": calibrated_absolute_error, "result": result.model_dump(mode="json"), } ) text_only_accuracy = 1.0 - sum(text_absolute_errors) / len(text_absolute_errors) calibrated_multimodal_accuracy = ( 1.0 - sum(calibrated_absolute_errors) / len(calibrated_absolute_errors) ) measured_incremental_gain = calibrated_multimodal_accuracy - text_only_accuracy return { "schema_version": "vignette.multimodal-alliance-benchmark-report.v1", "data_classification": pack.data_classification, "clinical_claim_allowed": pack.clinical_claim_allowed, "fusion_decision_accuracy": correct / len(pack.cases), "voice_gain_benchmark": { "metric": "one_minus_mean_absolute_error", "observations": len(pack.cases), "text_only_accuracy": text_only_accuracy, "calibrated_multimodal_accuracy": calibrated_multimodal_accuracy, "measured_incremental_gain": measured_incremental_gain, "minimum_incremental_gain": minimum_gain, "voice_gain_demonstrated": measured_incremental_gain >= minimum_gain, }, "cases": cases, } __all__ = [ "align_voice_timeline", "build_calibrated_axis_read_model", "evaluate_multimodal_benchmark", "load_multimodal_benchmark", ]