evaluator

package
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 8, 2026 License: MIT Imports: 4 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Criteria

type Criteria struct {
	// contains filtered or unexported fields
}

Criteria evaluates the relevancy of LLM responses.

func (*Criteria) Run

func (r *Criteria) Run(ctx context.Context, message *blades.Message) (*Evaluation, error)

Run evaluates the LLM response against the configured criteria.

type Evaluation

type Evaluation struct {
	Pass     bool      `json:"pass" jsonschema:"Indicates whether the response satisfies the evaluation criteria."`
	Score    float64   `json:"score" jsonschema:"LLM-judged similarity to the expected response; score in [0,1], higher is better."`
	Feedback *Feedback `json:"feedback" jsonschema:"Structured feedback on the evaluation results."`
}

Evaluation represents the result of evaluating an LLM response.

type Evaluator

type Evaluator interface {
	Run(context.Context, *blades.Message) (*Evaluation, error)
}

Evaluator defines the interface for evaluating LLM responses.

func New

func New(name string, opts ...blades.AgentOption) (Evaluator, error)

New creates a new Criteria evaluator.

type Feedback

type Feedback struct {
	Summary     string   `json:"summary" jsonschema:"Short summary of evaluation results."`
	Details     string   `json:"details" jsonschema:"Detailed explanation of strengths, weaknesses, and reasoning."`
	Suggestions []string `json:"suggestions" jsonschema:"List of recommended improvements or fixes."`
}

Feedback provides structured feedback on the evaluation results.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL