judge

package

v0.1.0 Latest Latest Go to latest Published: Oct 28, 2026 License: MIT Imports: 5 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/StevenACoffman/agentic-dev-harness

Links

Open Source Insights

Documentation ¶

Overview ¶

Package judge is the deterministic rule-judge for scoring an output against a set of checks (SPEC-ADDITIONS §11 test-checks; ported from skillsaw's judge, itself a port of SkillOpt-Sleep's judges.py). hard = 1.0 iff every check passes, soft = passed/total. It is pure — values in, values out, no I/O and no model — so it carries the deterministic first line of behavioral scoring, with a model reserved only for what a rule cannot decide.

Index ¶

type Check
type Op
type Result
- func Score(output string, checks []Check) (Result, error)

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Check ¶

type Check struct {
	Op  Op     `json:"op"`
	Arg string `json:"arg"`
}

Check is one rule: an operator applied to an argument.

type Op ¶

type Op string

Op is a rule-check operator.

const (
	OpSectionPresent Op = "section_present" // a heading line contains arg
	OpRegex          Op = "regex"           // arg is an RE2 pattern that matches
	OpContains       Op = "contains"        // arg is a substring of the output
	OpMaxChars       Op = "max_chars"       // rune count <= atoi(arg)
	OpMinChars       Op = "min_chars"       // rune count >= atoi(arg)
	OpToolCalled     Op = "tool_called"     // output names the tool (contains arg)
)

Rule-check operators (a closed set).

type Result ¶

type Result struct {
	Hard float64  `json:"hard"` // 1.0 iff every check passes, else 0.0
	Soft float64  `json:"soft"` // passed / total
	Why  []string `json:"why"`  // one line per check: pass/fail + reason
}

Result is the outcome of scoring an output against a set of checks.

func Score ¶

func Score(output string, checks []Check) (Result, error)

Score evaluates output against checks. It rejects an empty check set with EINVALID rather than returning a vacuous perfect score: judging nothing is a caller error, not a pass.

Source Files ¶

View all Source files

judge.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL