Documentation
¶
Overview ¶
Package report aggregates per-case Results into a stable JSON shape + human-readable summary. Phase 2's baseline differ depends on this shape being stable — schema_version below is bumped explicitly.
Index ¶
Constants ¶
const SchemaVersion = "1"
SchemaVersion of the report JSON. Bump on breaking shape changes.
Variables ¶
This section is empty.
Functions ¶
func WriteSummary ¶
WriteSummary writes a short human-readable summary to w.
func WriteTerminalSummary ¶
WriteTerminalSummary writes a short, optionally-coloured summary of the report to w. Colour is enabled only when w is *os.File pointing at a tty AND the NO_COLOR env var is unset (per https://no-color.org/).
This sits alongside WriteSummary (the plain version used by tests and JSON-companion writers) — both write the same information; this one adds ANSI colour for human-facing CLI output.
Types ¶
type AssertionEntry ¶
type AssertionEntry struct {
Kind string `json:"kind"`
Pass bool `json:"pass"`
Message string `json:"message,omitempty"`
Diagnostics map[string]any `json:"diagnostics,omitempty"`
}
AssertionEntry mirrors evalcase.AssertionResult but with deterministic ordering of diagnostic keys for snapshot stability.
type CaseEntry ¶
type CaseEntry struct {
CaseID string `json:"case_id"`
Pass bool `json:"pass"`
StartedAt time.Time `json:"started_at"`
EndedAt time.Time `json:"ended_at"`
Assertions []AssertionEntry `json:"assertions"`
Counts map[string]int `json:"counts,omitempty"`
// CustomScores is copied from the corresponding runner.Result. It is
// caller-populated (Run/RunLive never set it) — see runner.Result's
// own doc comment.
CustomScores map[string]float64 `json:"custom_scores,omitempty"`
}
CaseEntry is the projected result for one case.
type Report ¶
type Report struct {
SchemaVersion string `json:"schema_version"`
GeneratedAt time.Time `json:"generated_at"`
RunID string `json:"run_id,omitempty"`
Mode string `json:"mode"` // "deterministic" | "full"
Summary Summary `json:"summary"`
Cases []*CaseEntry `json:"cases"`
}
Report is the top-level eval-run report. Stable JSON shape — Phase 2's baseline differ keys off field names exactly.