derive

package
v0.15.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 16, 2026 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Overview

Package derive builds an eval suite for a skill that has none, by recovering a spec from the bundle and drafting evals from it.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Deriver

type Deriver struct {
	// contains filtered or unexported fields
}

Deriver builds suites. Construct with New.

func New

func New(o Options) (*Deriver, error)

New validates o and returns a Deriver.

func (*Deriver) Contest added in v0.14.0

func (d *Deriver) Contest(ctx context.Context, ins []Input) (*Result, error)

Contest derives one suite covering every candidate. A suite derived from one candidate grades the others against that candidate's claims, which is the bias a contest exists to remove.

func (*Deriver) Derive

func (d *Deriver) Derive(ctx context.Context, in Input) (*Result, error)

Derive recovers a spec from the bundle, drafts an eval set, and validates it statically.

type Input

type Input struct {
	Skill  string
	Bundle []byte
	// Tier and Panel are used only to dry-run runner.BuildPlan, so a suite too
	// thin to plan is refused before it is pushed and an eval row created.
	Tier  string
	Panel runner.Panel
}

Input is one derivation request.

type Options

type Options struct {
	Gateway llm.Gateway
	Logger  func(format string, args ...any)
}

Options are the deriver's injected dependencies.

type Result

type Result struct {
	Archive []byte
	// Tasks is how many evals the suite holds. The per-task check results are
	// not carried any further: suite.Validate is pure, so every later reader
	// runs it again rather than trusting a copy.
	Tasks int
	Spec  *spec.SkillSpec
}

Result is a derived suite, ready to push to the registry.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL