qitos.evaluate.base

• 模块分组: qitos.evaluate
• 源码: qitos/evaluate/base.py

快速跳转

• 类
• 函数
• Class: EvaluationContext
• Class: EvaluationResult
• Class: EvaluationSuite
• Class: SuiteEvaluationResult
• Class: TrajectoryEvaluator
• Function: load_run_artifacts

Classes

Class: EvaluationContext(self, task: 'Task', run: 'Any' = None, run_dir: 'Optional[str]' = None, manifest: 'Dict[str, Any]' = <factory>, events: 'List[Dict[str, Any]]' = <factory>, steps: 'List[Dict[str, Any]]' = <factory>, extras: 'Dict[str, Any]' = <factory>) -> None

EvaluationContext(task: 'Task', run: 'Any' = None, run_dir: 'Optional[str]' = None, manifest: 'Dict[str, Any]' = , events: 'List[Dict[str, Any]]' = , steps: 'List[Dict[str, Any]]' = , extras: 'Dict[str, Any]' = )

Class: EvaluationResult(self, name: 'str', success: 'bool', score: 'float' = 0.0, reasons: 'List[str]' = <factory>, evidence: 'Dict[str, Any]' = <factory>, tags: 'List[str]' = <factory>, metadata: 'Dict[str, Any]' = <factory>) -> None

EvaluationResult(name: 'str', success: 'bool', score: 'float' = 0.0, reasons: 'List[str]' = , evidence: 'Dict[str, Any]' = , tags: 'List[str]' = , metadata: 'Dict[str, Any]' = )

Class: EvaluationSuite(self, evaluators: 'Optional[Iterable[TrajectoryEvaluator]]' = None, mode: 'str' = 'all')

Compose multiple evaluators into one judgement.

Class: SuiteEvaluationResult(self, success: 'bool', score: 'float', results: 'List[EvaluationResult]', metadata: 'Dict[str, Any]' = <factory>) -> None

SuiteEvaluationResult(success: 'bool', score: 'float', results: 'List[EvaluationResult]', metadata: 'Dict[str, Any]' = )

Class: TrajectoryEvaluator(self, /, *args, **kwargs)

Helper class that provides a standard way to create an ABC using

Functions

Function: load_run_artifacts(run_dir: 'str | Path') -> 'Dict[str, Any]'

Load manifest/events/steps from a run directory with tolerant parsing.

Source Index

• qitos/evaluate/base.py