runner

Overview ¶

Command runner executes the Agent Eval pilot: a fixed task set (eval/tasks.yaml) against a deterministic fixture, driven by any OpenAI-compatible chat-completions endpoint with tool calling. It scores tasks mechanically (see eval/README.md) and prints a JSON report to stdout.

Configuration (environment):

EVAL_API_KEY   required; bearer token for the model endpoint
EVAL_MODEL     required; model name
EVAL_BASE_URL  optional; defaults to https://api.openai.com/v1