Documentation
¶
Overview ¶
Package render generates HTML and JSON reports from test execution results.
This package transforms RunResult data into formatted reports with:
- Provider × Region performance matrices
- Token usage and cost breakdowns
- Average latency calculations
- Success/error rate summaries
- Markdown-formatted conversation logs
Reports can be generated as standalone HTML files or JSON data exports.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Artifact ¶ added in v1.5.2
type Artifact struct {
Name string `json:"name"`
Description string `json:"description,omitempty"`
Path string `json:"path"`
}
Artifact is a hook-declared output surfaced in the report's per-run Artifacts section. Hooks may record any number (a session hook a captured workspace, a turn hook a per-turn dump, etc.). Path is relative to the report's output directory so the link resolves when the report is opened from disk.
type HTMLReportData ¶
type HTMLReportData struct {
Title string `json:"title"`
GeneratedAt time.Time `json:"generated_at"`
Summary ReportSummary `json:"summary"`
Results []engine.RunResult `json:"results"`
Providers []string `json:"providers"`
Regions []string `json:"regions"`
Scenarios []string `json:"scenarios"`
Matrix [][]MatrixCell `json:"matrix"` // Deprecated: kept for backwards compatibility
ScenarioGroups []ScenarioGroup `json:"scenario_groups"`
// ArtifactsByRun maps a run's RunID to artifacts a hook declared for it via a
// manifest at <outDir>/artifacts/<RunID>.json, rendered as per-run links. The
// report knows nothing about what hooks produce — it only surfaces what they
// declare. Empty when no manifest exists.
ArtifactsByRun map[string][]Artifact `json:"artifacts_by_run,omitempty"`
}
HTMLReportData contains all data needed for the HTML report
type MatrixCell ¶
type MatrixCell struct {
Provider string `json:"provider"`
Region string `json:"region"`
Scenarios int `json:"scenarios"`
Successful int `json:"successful"`
Errors int `json:"errors"`
Cost float64 `json:"cost"`
}
MatrixCell represents a cell in the provider x region matrix
type MediaStats ¶ added in v1.1.0
type MediaStats struct {
TotalImages int
TotalAudio int
TotalVideo int
MediaLoadSuccess int
MediaLoadErrors int
TotalMediaSize int64
}
MediaStats holds aggregate media statistics across multiple run results.
type ReportSummary ¶
type ReportSummary struct {
TotalRuns int `json:"total_runs"`
SuccessfulRuns int `json:"successful_runs"`
ErrorRuns int `json:"error_runs"`
TotalCost float64 `json:"total_cost"`
TotalTokens int `json:"total_tokens"`
AvgLatency string `json:"avg_latency"`
// Media statistics
TotalImages int `json:"total_images"`
TotalAudio int `json:"total_audio"`
TotalVideo int `json:"total_video"`
MediaLoadSuccess int `json:"media_load_success"`
MediaLoadErrors int `json:"media_load_errors"`
TotalMediaSize int64 `json:"total_media_size_bytes"`
}
ReportSummary contains aggregate statistics
type ScenarioGroup ¶
type ScenarioGroup struct {
ScenarioID string `json:"scenario_id"`
Providers []string `json:"providers"`
Regions []string `json:"regions"`
Matrix [][]MatrixCell `json:"matrix"`
Results []engine.RunResult `json:"results"`
// Labels surfaces stratification labels copied from the scenario manifest's
// metadata.labels. Picked from the first labeled result in the group; all
// repetitions of a scenario carry identical labels.
Labels map[string]string `json:"labels,omitempty"`
}
ScenarioGroup represents a scenario with its provider×region matrix
Source Files