Documentation
¶
Overview ¶
Package harness runs mock parity checks against Codog behavior.
Index ¶
Constants ¶
const ManifestSchemaVersion = reportschema.MockParityManifestSchemaV1
ManifestSchemaVersion is the stable schema identifier for mock parity manifests.
const ReportSchemaVersion = reportschema.MockParityReportSchemaV1
ReportSchemaVersion is the stable schema identifier for mock parity reports.
Variables ¶
This section is empty.
Functions ¶
func ValidateReport ¶
ValidateReport verifies that a mock parity report is internally consistent.
Types ¶
type CapabilityCoverage ¶
type CapabilityCoverage struct {
Capability string `json:"capability"`
Status string `json:"status"`
RequiredRefs []string `json:"required_refs"`
CoveredRefs []string `json:"covered_refs"`
Scenarios []string `json:"scenarios"`
}
CapabilityCoverage maps the harness scenarios back to the Claude-Code-style capability surfaces the project is trying to prove.
type CategoryReport ¶
type CategoryReport struct {
Category string `json:"category"`
OK bool `json:"ok"`
Passed int `json:"passed"`
Total int `json:"total"`
Scenarios []string `json:"scenarios"`
}
CategoryReport summarizes mock parity results for one behavioral category.
type Manifest ¶
type Manifest struct {
SchemaVersion string `json:"schema_version"`
ScenarioCount int `json:"scenario_count"`
Categories []ManifestCategory `json:"categories"`
CapabilityCoverage []CapabilityCoverage `json:"capability_coverage"`
Scenarios []ManifestScenario `json:"scenarios"`
}
Manifest lists the deterministic mock parity scenarios without running them.
func ScenarioManifest ¶
func ScenarioManifest() Manifest
ScenarioManifest returns the mock parity scenario contract without executing provider or tool loops.
type ManifestCategory ¶
type ManifestCategory struct {
Category string `json:"category"`
Count int `json:"count"`
Scenarios []string `json:"scenarios"`
}
ManifestCategory summarizes scenarios by behavioral category.
type ManifestScenario ¶
type ManifestScenario struct {
Name string `json:"name"`
Category string `json:"category"`
Description string `json:"description"`
ParityRefs []string `json:"parity_refs"`
}
ManifestScenario describes one mock parity scenario contract.
type Report ¶
type Report struct {
SchemaVersion string `json:"schema_version"`
OK bool `json:"ok"`
Passed int `json:"passed"`
Total int `json:"total"`
ScenarioCount int `json:"scenario_count"`
RequestCount int `json:"request_count"`
Coverage []CategoryReport `json:"coverage"`
CapabilityCoverage []CapabilityCoverage `json:"capability_coverage"`
Workspace string `json:"workspace"`
Output string `json:"output"`
Iterations int `json:"iterations"`
MessageCount int `json:"message_count"`
ToolCalls int `json:"tool_calls"`
UsageSummary usage.Summary `json:"usage_summary"`
EstimatedCost float64 `json:"estimated_cost"`
Scenarios []ScenarioReport `json:"scenarios"`
}
Report summarizes one deterministic mock parity harness run.
type ScenarioReport ¶
type ScenarioReport struct {
Name string `json:"name"`
Category string `json:"category"`
Description string `json:"description"`
ParityRefs []string `json:"parity_refs"`
OK bool `json:"ok"`
Workspace string `json:"workspace"`
Output string `json:"output,omitempty"`
Iterations int `json:"iterations,omitempty"`
RequestCount int `json:"request_count"`
MessageCount int `json:"message_count,omitempty"`
ToolCalls int `json:"tool_calls,omitempty"`
ToolUses []string `json:"tool_uses"`
ToolErrorCount int `json:"tool_error_count"`
FinalMessage string `json:"final_message"`
UsageSummary usage.Summary `json:"usage_summary"`
EstimatedCost float64 `json:"estimated_cost"`
RequestMessageCounts []int `json:"request_message_counts,omitempty"`
Compactions int `json:"compactions,omitempty"`
Error string `json:"error,omitempty"`
}
ScenarioReport records the outcome of one mock parity scenario.
Source Files