output

package
v0.1.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 23, 2026 License: Apache-2.0 Imports: 8 Imported by: 0

Documentation

Overview

Package output generates structured JSON reports after each Errata run.

A report captures the complete execution snapshot: recipe configuration, prompt, per-model results (text, tokens, cost, latency, tool events, proposed writes), aggregate statistics, and optional selection outcome.

Reports are written to data/outputs/ as pretty-printed JSON files named {recipeName}_output_{hex}.json. They are intended as a fundamental primitive for users to interrogate past runs programmatically.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func SanitizeName 

func SanitizeName(name string) string

SanitizeName cleans a recipe name for use in a filename. Empty input returns "default".

func SaveSession 

func SaveSession(dir string, report *SessionReport) (string, error)

SaveSession writes the session report as pretty-printed JSON to dir/{filename}. Parent directories are created as needed. Returns the full path.

Types

type AggregateStats 

type AggregateStats struct {
	TotalCostUSD      float64 `json:"total_cost_usd"`
	TotalInputTokens  int64   `json:"total_input_tokens"`
	TotalOutputTokens int64   `json:"total_output_tokens"`
	ModelCount        int     `json:"model_count"`
	SuccessCount      int     `json:"success_count"`
	FastestModel      string  `json:"fastest_model,omitempty"`
	FastestMS         int64   `json:"fastest_ms,omitempty"`
}

AggregateStats summarizes the run across all models.

type Collector 

type Collector struct {
	// contains filtered or unexported fields
}

Collector accumulates per-model AgentEvents during a run. It is safe for concurrent use from multiple goroutines.

func NewCollector 

func NewCollector() *Collector

NewCollector returns a ready-to-use Collector.

func (*Collector) Events 

func (c *Collector) Events(modelID string) []EventEntry

Events returns a copy of the collected events for the given model.

func (*Collector) WrapOnEvent 

func (c *Collector) WrapOnEvent(
	original func(modelID string, event models.AgentEvent),
) func(modelID string, event models.AgentEvent)

WrapOnEvent returns an onEvent callback that records the event and then forwards it to the original onEvent function.

type ConstraintsSnapshot 

type ConstraintsSnapshot struct {
	MaxSteps int    `json:"max_steps,omitempty"`
	Timeout  string `json:"timeout,omitempty"`
}

ConstraintsSnapshot captures constraint settings.

type EventEntry 

type EventEntry struct {
	Type models.EventType `json:"type"`
	Data string           `json:"data"`
}

EventEntry captures a single tool event during execution.

type ModelParamsSnapshot 

type ModelParamsSnapshot struct {
	Temperature *float64 `json:"temperature,omitempty"`
	MaxTokens   *int     `json:"max_tokens,omitempty"`
	Seed        *int64   `json:"seed,omitempty"`
}

ModelParamsSnapshot captures sampling parameters.

type ModelResult 

type ModelResult struct {
	ModelID         string         `json:"model_id"`
	Text            string         `json:"text"`
	LatencyMS       int64          `json:"latency_ms"`
	InputTokens     int64          `json:"input_tokens"`
	OutputTokens    int64          `json:"output_tokens"`
	ReasoningTokens int64          `json:"reasoning_tokens,omitempty"`
	CostUSD         float64        `json:"cost_usd"`
	Error           string         `json:"error,omitempty"`
	StopReason      string         `json:"stop_reason,omitempty"`
	Steps           int            `json:"steps,omitempty"`
	ToolCalls       map[string]int `json:"tool_calls,omitempty"`
	ProposedWrites  []WriteEntry   `json:"proposed_writes,omitempty"`
	Events          []EventEntry   `json:"events"`
}

ModelResult is the per-model execution result.

type RecipeSnapshot 

type RecipeSnapshot struct {
	Name         string               `json:"name"`
	Models       []string             `json:"models,omitempty"`
	SystemPrompt string               `json:"system_prompt,omitempty"`
	Tools        []string             `json:"tools,omitempty"`
	Constraints  *ConstraintsSnapshot `json:"constraints,omitempty"`
	ModelParams  *ModelParamsSnapshot `json:"model_params,omitempty"`
}

RecipeSnapshot captures the active configuration at the time of the run.

type Report 

type Report struct {
	ID        string    `json:"id"`
	Timestamp time.Time `json:"timestamp"`
	SessionID string    `json:"session_id"`

	Recipe RecipeSnapshot `json:"recipe"`
	Prompt string         `json:"prompt"`

	Models    []ModelResult  `json:"models"`
	Aggregate AggregateStats `json:"aggregate"`

	Selection *SelectionOutcome `json:"selection,omitempty"`
}

Report is the complete execution snapshot written to data/outputs/.

func BuildReport 

func BuildReport(
	sessionID string,
	rec *recipe.Recipe,
	prompt string,
	responses []models.ModelResponse,
	collector *Collector,
	activeToolNames []string,
) *Report

BuildReport constructs a Report from the run results and collected events.

func (*Report) Filename 

func (r *Report) Filename() string

Filename returns the output file name: {sanitizedRecipeName}_output_{id}.json

type SelectionOutcome 

type SelectionOutcome struct {
	SelectedModel string    `json:"selected_model"`
	AppliedFiles  []string  `json:"applied_files,omitempty"`
	Rating        string    `json:"rating,omitempty"`
	Timestamp     time.Time `json:"timestamp"`
}

SelectionOutcome records the user's choice after the run.

type SessionReport 

type SessionReport struct {
	ID        string       `json:"id"`
	Timestamp time.Time    `json:"timestamp"`
	SessionID string       `json:"session_id"`
	Turns     []TurnReport `json:"turns"`
	Aggregate SessionStats `json:"aggregate"`
}

SessionReport aggregates all runs within a session into a single export.

func BuildSessionReport 

func BuildSessionReport(sessionID string, reports []*Report) *SessionReport

BuildSessionReport constructs a SessionReport from a slice of per-run Reports.

func LoadSession 

func LoadSession(path string) (*SessionReport, error)

LoadSession reads a session report JSON file at the given path.

func (*SessionReport) Filename 

func (r *SessionReport) Filename() string

Filename returns the output file name: session_output_{id}.json

type SessionStats 

type SessionStats struct {
	TurnCount         int     `json:"turn_count"`
	TotalCostUSD      float64 `json:"total_cost_usd"`
	TotalInputTokens  int64   `json:"total_input_tokens"`
	TotalOutputTokens int64   `json:"total_output_tokens"`
}

SessionStats summarizes the session across all turns.

type TurnReport 

type TurnReport struct {
	TurnIndex int               `json:"turn_index"`
	Timestamp time.Time         `json:"timestamp"`
	Prompt    string            `json:"prompt"`
	Recipe    RecipeSnapshot    `json:"recipe"`
	Models    []ModelResult     `json:"models"`
	Aggregate AggregateStats    `json:"aggregate"`
	Selection *SelectionOutcome `json:"selection,omitempty"`
}

TurnReport is one prompt+results within the session.

type WriteEntry 

type WriteEntry struct {
	Path    string `json:"path"`
	Content string `json:"content"`
	Delete  bool   `json:"delete,omitempty"`
}

WriteEntry captures one proposed file write.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.