Documentation
¶
Overview ¶
Package validators provides validation mechanisms for AI model responses. It includes support for both value matching and LLM-based semantic equivalence validation using judge models.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrJudgeNotFound is returned when a judge configuration is not found. ErrJudgeNotFound = errors.New("judge not found") // ErrJudgeVariantNotFound is returned when a judge run variant is not found. ErrJudgeVariantNotFound = errors.New("judge run variant not found") )
var ErrUnsupportedResponseFormatValidation = errors.New("unsupported response format validation")
ErrUnsupportedResponseFormatValidation is returned when a validator cannot handle the response format.
Functions ¶
This section is empty.
Types ¶
type Factory ¶
type Factory struct {
// contains filtered or unexported fields
}
Factory creates and manages validator instances. It provides caching to improve performance.
func NewFactory ¶
func NewFactory(availableJudges []config.JudgeConfig) *Factory
NewFactory creates a new validator factory with the provided judge configurations.
func (*Factory) AssertExists ¶
func (f *Factory) AssertExists(judge config.JudgeSelector) error
AssertExists checks if a judge configuration exists for the given judge selector. Returns an error if the judge configuration does not exist.
func (*Factory) GetValidator ¶
GetValidator returns a validator for the given judge selector. If judge is enabled, returns a cached judge validator; otherwise returns a value match validator.
type ValidationResult ¶
type ValidationResult struct {
// IsCorrect indicates whether the validation passed.
IsCorrect bool
// Title provides a descriptive title for the validation type.
Title string
// Explanation provides an optional explanation of the validation result.
Explanation string
// Usage contains token usage statistics for the validation step when available.
Usage providers.Usage
}
ValidationResult contains the result of a validation check.
type Validator ¶
type Validator interface {
// IsCorrect checks if result matches expected value using the provided validation rules.
// The originalPrompt and expectedResponseFormat provide additional context for semantic validation.
// The logger parameter allows validators to emit structured log messages during validation.
IsCorrect(ctx context.Context, logger logging.Logger, rules config.ValidationRules, expected utils.ValueSet, actual providers.Result, originalPrompt string, expectedResponseFormat config.ResponseFormat) (ValidationResult, error)
// ToCanonical normalizes value for validation using the provided validation rules.
// For string values, applies string normalization rules (case, whitespace, etc.).
// For object values, recursively normalizes all string fields within the object structure.
ToCanonical(rules config.ValidationRules, value interface{}) interface{}
// GetName returns a descriptive user-friendly name for the validator.
GetName() string
// Close cleans up any resources used by the validator.
Close(ctx context.Context) error
}
Validator verifies AI model responses.
func NewJudgeValidator ¶
func NewJudgeValidator(ctx context.Context, judgeConfig *config.JudgeConfig, judgeRunVariant config.RunConfig, availableTools []config.ToolConfig) (Validator, error)
NewJudgeValidator creates a new semantic Validator with the given judge configuration and run variant. The judge provider will be initialized from the configuration and used to evaluate responses for semantic equivalence.
func NewValueMatchValidator ¶
func NewValueMatchValidator() Validator
NewValueMatchValidator returns a new Validator that checks results by exact string matching. The validator applies validation rules for case sensitivity and whitespace handling.
Source Files