judge

package
v0.15.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 7, 2026 License: Apache-2.0 Imports: 10 Imported by: 0

Documentation

Overview

Package judge evaluates arbitrary subjects with a structured-output chat model. Subject validation and prompt construction remain owned by callers.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Config

type Config[T any] struct {
	Model chat.Model
	// ModelID identifies the explicitly selected judge model and revision.
	ModelID string
	// RubricID versions the scoring rules implemented by Prompt. Callers must
	// change it when those rules change; a function cannot reveal its identity.
	RubricID string
	Metric   eval.Metric
	Prompt   Prompt[T]
	Options  chat.Options
	// Threshold is optional. Without one, evaluation produces a score without
	// inventing a pass/fail decision.
	Threshold *eval.Score
	Samples   int
}

Config binds a portable metric and subject prompt to a structured-output model judge. Multiple samples use a deterministic median aggregation.

type Evaluator

type Evaluator[T any] struct {
	// contains filtered or unexported fields
}

Evaluator asks a chat model for normalized scores without teaching the eval kernel any domain vocabulary.

func NewEvaluator

func NewEvaluator[T any](config Config[T]) (*Evaluator[T], error)

NewEvaluator freezes metric identity, options, threshold, and sampling policy.

func (*Evaluator[T]) Evaluate

func (e *Evaluator[T]) Evaluate(ctx context.Context, subject T) (eval.Report, error)

type Prompt

type Prompt[T any] func(T) (chat.Message, error)

Prompt projects a domain subject into one valid judge message.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL