eval/

directory
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 8, 2026 License: MIT

Directories

Path Synopsis
core
domain
Package domain holds the evaluator's vocabulary: problems, attempts, what the browser observed, and the scorecards built from them.
Package domain holds the evaluator's vocabulary: problems, attempts, what the browser observed, and the scorecards built from them.
port/resource
Package resource declares what the evaluator needs from the outside world: where problems come from, the browser, and where results are kept.
Package resource declares what the evaluator needs from the outside world: where problems come from, the browser, and where results are kept.
port/service
Package service declares the evaluator's use cases: what the command line asks for.
Package service declares the evaluator's use cases: what the command line asks for.
probe
Package probe decides what the UI probes found.
Package probe decides what the UI probes found.
report
Package report turns attempts into scorecards and summaries.
Package report turns attempts into scorecards and summaries.
run
Package run asks problems through the browser and records what happened.
Package run asks problems through the browser and records what happened.
score
Package score grades answers and builds the prompt.
Package score grades answers and builds the prompt.
uicheck
Package uicheck judges one attempt's observations against the passive UI checks U-01 to U-08.
Package uicheck judges one attempt's observations against the passive UI checks U-01 to U-08.
infrastructure
config
Package config loads and validates the evaluator's settings.
Package config loads and validates the evaluator's settings.
devtools
Package devtools drives the Web UI through Chrome DevTools MCP: it starts the server, calls its tools, and reads what they print.
Package devtools drives the Web UI through Chrome DevTools MCP: it starts the server, calls its tools, and reads what they print.
gaia
Package gaia provides the evaluator's problems: GAIA from local parquet files (downloaded by fetch.go), and a synthetic set for the self-test.
Package gaia provides the evaluator's problems: GAIA from local parquet files (downloaded by fetch.go), and a synthetic set for the self-test.
safety
Package safety holds the checks that keep data and secrets out of places they must not go.
Package safety holds the checks that keep data and secrets out of places they must not go.
store
Package store keeps a run's files on disk: its results one row at a time, its description, what the probes found, its summaries and its evidence.
Package store keeps a run's files on disk: its results one row at a time, its description, what the probes found, its summaries and its evidence.
synthetic
Package synthetic holds the evaluator's self-test set: thirteen invented problems (none of it is GAIA content), the answers the scripted agent gives to them, and the results a correct evaluator must find.
Package synthetic holds the evaluator's self-test set: thirteen invented problems (none of it is GAIA content), the answers the scripted agent gives to them, and the results a correct evaluator must find.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL