Documentation
¶
Overview ¶
evalrun runs one eval case at both tiers and reports (#652).
It is the thin part. Everything worth arguing with lives in internal/evals; this binary parses flags, decides an exit code, and prints something a person can read at 2am.
Exit codes:
0 the tools tier passed and the no-access baseline held 1 a check failed, or the baseline scored, or the harness errored 2 the report is indeterminate — some check observed nothing
2 is separate from 1 on purpose. A failing check says the agent did the wrong thing; an indeterminate one says the harness did not find out what the agent did, and the two send an operator to different places. Both are non-zero: an unanswered question is not a pass.
Click to show internal directories.
Click to hide internal directories.