Documentation
¶
Overview ¶
Package sentineltest provides test assertions for evaluation results. It lives in a separate package to avoid import cycles between the root sentinel package and the evalrun package.
Index ¶
- func Assert(t *testing.T, stats *evalrun.ResultStats, checks ...AssertCheck)
- type AssertCheck
- func BehaviorTriggerRate(minScore float64) AssertCheck
- func CognitivePhaseCompliance(minScore float64) AssertCheck
- func CommunicationMatch(minScore float64) AssertCheck
- func MaxCost(cost float64) AssertCheck
- func MaxLatency(ms int) AssertCheck
- func MinAverageScore(score float64) AssertCheck
- func MinPassRate(rate float64) AssertCheck
- func NoRegressions() AssertCheck
- func PersonaCoherence(minScore float64) AssertCheck
- func SkillProficiency(minScore float64) AssertCheck
- func TraitConsistency(minScore float64) AssertCheck
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Assert ¶
func Assert(t *testing.T, stats *evalrun.ResultStats, checks ...AssertCheck)
Assert runs all checks against the result stats and fails the test if any check fails.
Types ¶
type AssertCheck ¶
type AssertCheck func(stats *evalrun.ResultStats) error
AssertCheck is a function that validates run result statistics. It returns an error message if the check fails, nil if it passes.
func BehaviorTriggerRate ¶
func BehaviorTriggerRate(minScore float64) AssertCheck
BehaviorTriggerRate requires the behavior dimension score to be at least the given value.
func CognitivePhaseCompliance ¶
func CognitivePhaseCompliance(minScore float64) AssertCheck
CognitivePhaseCompliance requires the cognition dimension score to be at least the given value.
func CommunicationMatch ¶
func CommunicationMatch(minScore float64) AssertCheck
CommunicationMatch requires the communication dimension score to be at least the given value.
func MaxCost ¶
func MaxCost(cost float64) AssertCheck
MaxCost requires the total cost to be at most the given value.
func MaxLatency ¶
func MaxLatency(ms int) AssertCheck
MaxLatency requires the average latency to be at most the given value in ms.
func MinAverageScore ¶
func MinAverageScore(score float64) AssertCheck
MinAverageScore requires the average score to be at least the given value.
func MinPassRate ¶
func MinPassRate(rate float64) AssertCheck
MinPassRate requires the pass rate to be at least the given value.
func NoRegressions ¶
func NoRegressions() AssertCheck
NoRegressions checks that there are no failed cases.
func PersonaCoherence ¶
func PersonaCoherence(minScore float64) AssertCheck
PersonaCoherence requires the persona dimension score to be at least the given value.
func SkillProficiency ¶
func SkillProficiency(minScore float64) AssertCheck
SkillProficiency requires the skill dimension score to be at least the given value.
func TraitConsistency ¶
func TraitConsistency(minScore float64) AssertCheck
TraitConsistency requires the trait dimension score to be at least the given value.
Source Files