eval

package

v0.4.0 Latest Latest Go to latest Published: Jul 17, 2026 License: MIT Imports: 4 Imported by: 0

Details

@sk-task ecosystem#T3.1: Eval package (contains, regex, llm_judge)

This section is empty.

This section is empty.

This section is empty.

type Contains struct {
	Value string
}

Contains checks if the output contains a substring.

func (c Contains) Eval(output string) Result

Eval implements the Tool interface.

type Criterion interface {
	Eval(output string) Result
}

Criterion defines the interface for eval criteria.

type LLMJudge struct {
	Prompt string
	LLM    llm.Client
}

LLMJudge uses an LLM to evaluate output quality.

func (j LLMJudge) Eval(output string) Result

Eval implements the Criterion interface.

type Regex struct {
	Pattern string
	// contains filtered or unexported fields
}

Regex checks if the output matches a regular expression.

func NewRegex(pattern string) (*Regex, error)

NewRegex creates a new Regex criterion, compiling the pattern.

func (r *Regex) Eval(output string) Result

Eval implements the Criterion interface.

type Result struct {
	Criterion       string
	Value           string
	Passed          bool
	Details         string
	TaskDescription string
}

Result represents the outcome of a single eval criterion evaluation.

func Run(criteria []Criterion, taskOutputs map[string]string) []Result

Run executes eval criteria against a map of task descriptions to outputs.