report

package
v0.30.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 31, 2026 License: MIT Imports: 8 Imported by: 0

Documentation

Overview

Package report aggregates per-case Results into a stable JSON shape + human-readable summary. Phase 2's baseline differ depends on this shape being stable — schema_version below is bumped explicitly.

Index

Constants

View Source
const SchemaVersion = "1"

SchemaVersion of the report JSON. Bump on breaking shape changes.

Variables

This section is empty.

Functions

func WriteJSON

func WriteJSON(w io.Writer, r *Report) error

WriteJSON writes the report as indented JSON to w.

func WriteSummary

func WriteSummary(w io.Writer, r *Report) error

WriteSummary writes a short human-readable summary to w.

func WriteTerminalSummary

func WriteTerminalSummary(w io.Writer, r *Report) error

WriteTerminalSummary writes a short, optionally-coloured summary of the report to w. Colour is enabled only when w is *os.File pointing at a tty AND the NO_COLOR env var is unset (per https://no-color.org/).

This sits alongside WriteSummary (the plain version used by tests and JSON-companion writers) — both write the same information; this one adds ANSI colour for human-facing CLI output.

Types

type AssertionEntry

type AssertionEntry struct {
	Kind        string         `json:"kind"`
	Pass        bool           `json:"pass"`
	Message     string         `json:"message,omitempty"`
	Diagnostics map[string]any `json:"diagnostics,omitempty"`
}

AssertionEntry mirrors evalcase.AssertionResult but with deterministic ordering of diagnostic keys for snapshot stability.

type CaseEntry

type CaseEntry struct {
	CaseID     string           `json:"case_id"`
	Pass       bool             `json:"pass"`
	StartedAt  time.Time        `json:"started_at"`
	EndedAt    time.Time        `json:"ended_at"`
	Assertions []AssertionEntry `json:"assertions"`
	Counts     map[string]int   `json:"counts,omitempty"`
	// CustomScores is copied from the corresponding runner.Result. It is
	// caller-populated (Run/RunLive never set it) — see runner.Result's
	// own doc comment.
	CustomScores map[string]float64 `json:"custom_scores,omitempty"`
}

CaseEntry is the projected result for one case.

type Report

type Report struct {
	SchemaVersion string       `json:"schema_version"`
	GeneratedAt   time.Time    `json:"generated_at"`
	RunID         string       `json:"run_id,omitempty"`
	Mode          string       `json:"mode"` // "deterministic" | "full"
	Summary       Summary      `json:"summary"`
	Cases         []*CaseEntry `json:"cases"`
}

Report is the top-level eval-run report. Stable JSON shape — Phase 2's baseline differ keys off field names exactly.

func Aggregate

func Aggregate(mode string, results []*runner.Result) *Report

Aggregate folds a slice of runner.Results into a Report.

type Summary

type Summary struct {
	Total  int `json:"total"`
	Passed int `json:"passed"`
	Failed int `json:"failed"`
}

Summary is the aggregate pass/fail across all cases.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL