grading

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 7, 2026 License: MIT Imports: 8 Imported by: 0

Documentation

Overview

Package grading provides parsing and scoring for agent output reports.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func FormatResult

func FormatResult(r *GradeResult) string

FormatResult returns a human-readable multi-line summary of the grade result.

func RequiredSections

func RequiredSections(mode string) []string

RequiredSections returns the list of required sections for a given mode.

Types

type AgentStats

type AgentStats struct {
	Agent                  string         `json:"agent"`
	TotalRuns              int            `json:"total_runs"`
	LatestGrade            string         `json:"latest_grade"`
	AvgScore               float64        `json:"avg_score"`
	AvgReportQuality       float64        `json:"avg_report_quality"`
	AvgIterationEfficiency float64        `json:"avg_iteration_efficiency"`
	GradeCounts            map[string]int `json:"grade_counts"`
}

AgentStats provides aggregate statistics for an agent.

type GradeHistory

type GradeHistory struct {
	Grades []*GradeResult `json:"grades"`
}

GradeHistory is the top-level container written to and read from the store file. It holds all recorded GradeResult values in append order.

type GradeOptions

type GradeOptions struct {
	// Agent is the name of the agent being graded.
	Agent string
	// Iterations is the number of model iterations consumed by the run.
	Iterations int
	// FileCount is the number of files in the target codebase.
	FileCount int
	// RunID is an optional identifier linking the grade to a specific run.
	RunID string
}

GradeOptions configures the grading calculation.

type GradeResult

type GradeResult struct {
	// Identification
	Agent     string    `json:"agent"`
	Timestamp time.Time `json:"timestamp"`
	RunID     string    `json:"run_id,omitempty"`

	// Overall grade
	Grade      string  `json:"grade"`
	TotalScore float64 `json:"total_score"`

	// Component scores (0-100)
	ReportQuality       float64 `json:"report_quality"`
	IterationEfficiency float64 `json:"iteration_efficiency"`

	// Manual review indicators
	FindingQualityReview bool `json:"finding_quality_review"`
	SkipDisciplineReview bool `json:"skip_discipline_review"`

	// Metadata
	Iterations    int    `json:"iterations"`
	FileCount     int    `json:"file_count"`
	FilesTouched  int    `json:"files_touched"`
	IssuesFixed   int    `json:"issues_fixed"`
	IssuesSkipped int    `json:"issues_skipped"`
	Mode          string `json:"mode"`

	// Breakdown details
	MissingSections []string `json:"missing_sections,omitempty"`
	Notes           []string `json:"notes,omitempty"`
}

GradeResult represents the computed grade for an agent run.

func ComputeGrade

func ComputeGrade(parsed *ParsedOutput, opts GradeOptions) *GradeResult

ComputeGrade calculates the automated grade for a parsed agent output. It scores report quality (10% of total) and iteration efficiency (15%), returning a GradeResult with a letter grade and component breakdowns. The remaining 75% requires manual review.

type ParsedIssue

type ParsedIssue struct {
	Title    string
	Severity string
	Category string
	File     string
	Line     int
}

ParsedIssue represents a single issue found or fixed.

type ParsedOutput

type ParsedOutput struct {
	// Report sections detected
	HasChangesSummary  bool
	HasAnalysisSummary bool
	HasFilesTouched    bool
	HasValidation      bool
	HasIssuesFixed     bool
	HasIssuesSkipped   bool
	HasFindings        bool

	// Extracted data
	FilesTouched     []string
	IssuesFixed      []ParsedIssue
	IssuesSkipped    []ParsedSkip
	Findings         []ParsedIssue
	ValidationPassed bool

	// Metadata
	Mode string // "edit" or "readonly"
}

ParsedOutput represents the structured data extracted from agent output.

type ParsedSkip

type ParsedSkip struct {
	Title  string
	File   string
	Reason string
}

ParsedSkip represents an issue that was skipped.

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser extracts structured data from agent markdown output.

func NewParser

func NewParser() *Parser

NewParser creates a new output parser.

func (*Parser) Parse

func (p *Parser) Parse(output string) *ParsedOutput

Parse extracts structured data from agent output markdown.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store persists grade results to a JSON file on disk. Use NewStore to create one at the default cache location or NewStoreAt to specify an explicit path.

func NewStore

func NewStore() (*Store, error)

NewStore creates a store at the default location (~/.cache/squad/grades.json).

func NewStoreAt

func NewStoreAt(path string) *Store

NewStoreAt creates a store at a specific path.

func (*Store) List

func (s *Store) List(agentName string, limit int) ([]*GradeResult, error)

List returns grades for a specific agent, sorted by timestamp (newest first).

func (*Store) ListAll

func (s *Store) ListAll(limit int) ([]*GradeResult, error)

ListAll returns all grades, sorted by timestamp (newest first).

func (*Store) Path

func (s *Store) Path() string

Path returns the store's file path.

func (*Store) Save

func (s *Store) Save(result *GradeResult) error

Save appends a grade result to the store.

func (*Store) Stats

func (s *Store) Stats(agentName string) (*AgentStats, error)

Stats returns aggregate statistics for an agent.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL