Documentation
¶
Overview ¶
Package campaign assembles Chatwright's evidence-backed campaign report from a completed (or budget-stopped) actor.Loop run: a Goal, the goal.CampaignState snapshot it produced, and the actor.LoopEvents the loop recorded along the way.
Report is designed as an exported, versioned contract — the seed of Chatwright's machine-readable run bundle — not an internal struct: it is meant to be marshalled to JSON, stored, and read by tooling that never links against this package. See Assemble.
Index ¶
Constants ¶
const ReportSchemaVersion = 1
ReportSchemaVersion is the current version of Report's JSON shape. Bump it whenever Report changes in a way a consumer must branch on, and never reinterpret an old SchemaVersion's fields under a new meaning.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AggregateUsage ¶
type AggregateUsage struct {
InputTokens int `json:"inputTokens"`
OutputTokens int `json:"outputTokens"`
Cost float64 `json:"cost,omitempty"`
// CallCount is the number of Provider.Propose calls the campaign made —
// i.e. the number of LoopEvents.
CallCount int `json:"callCount"`
}
AggregateUsage sums the actor.Usage of every LoopEvent a Report was assembled from.
type AssembleInput ¶
type AssembleInput struct {
Goal goal.Goal
Campaign goal.CampaignSnapshot
Events []actor.LoopEvent
// CallerFindings are findings this slice's mechanics cannot derive on
// their own — chiefly FindingVerifiedDefect: "the actor acted and the
// observed outcome was wrong" requires deterministic or DTQL evidence
// (a later slice) or some other judgement outside this package's
// mechanical rules. Assemble includes them verbatim, appended after its
// own derived findings — this is the caller's seam to plug verification
// in; Assemble does not validate or reclassify them.
CallerFindings []Finding
}
AssembleInput is everything Assemble needs to build a Report: the Goal a campaign ran, a goal.CampaignSnapshot of the state it reached, and every actor.LoopEvent the loop recorded while running it.
type Evidence ¶
type Evidence struct {
// ObservationSequences are observe.Observation.Sequence values.
ObservationSequences []int64 `json:"observationSequences,omitempty"`
// LoopEventIndexes are actor.LoopEvent.Index values.
LoopEventIndexes []int `json:"loopEventIndexes,omitempty"`
}
Evidence links a Finding back to the observations and loop events that ground it, so a developer can navigate from a claim to its proof. Both slices may be empty — e.g. a coverage-gap finding for a task that was never attempted has nothing to link to; that is precisely what makes it a gap.
type Finding ¶
type Finding struct {
Kind FindingKind `json:"kind"`
TaskID string `json:"taskId"`
Summary string `json:"summary"`
Evidence Evidence `json:"evidence"`
// Confidence distinguishes how the Finding was derived: "mechanical"
// for the deterministic rules Assemble applies itself, or a caller's
// own label (e.g. "dtql-verified") for AssembleInput.CallerFindings.
Confidence string `json:"confidence,omitempty"`
}
Finding is one reportable outcome of the campaign: a claim, classified, scoped to a task, and linked to the evidence that grounds it.
type FindingKind ¶
type FindingKind string
FindingKind classifies one Finding. This slice supports exactly the three kinds mechanical evidence (plus an explicit caller hook) can ground: see Assemble.
const ( // FindingVerifiedDefect: the actor acted and the observed outcome was // wrong — backed by deterministic or DTQL evidence, or (this slice) a // caller-supplied classification; mechanics alone cannot derive this // kind, see AssembleInput.CallerFindings. FindingVerifiedDefect FindingKind = "verified-defect" // history shows the actor's own proposals going stale or invalid — the // bot was never shown to be at fault. FindingAINavigationFailure FindingKind = "ai-navigation-failure" // FindingCoverageGap: a task the campaign never attempted, or never // concluded, before it stopped — a gap in evidence, not a claim about // the bot. FindingCoverageGap FindingKind = "coverage-gap" // FindingActorOvershoot: the actor kept acting — or, via the loop's // overshoot probe, was shown to want to keep acting — after its // task's machine-checkable completion criteria already held. // Attributed to the actor, never misfiled as a bot defect — see // deriveMechanicalEventFindings and // spec/ideas/evidence-defined-completion.md. FindingActorOvershoot FindingKind = "actor-overshoot" // FindingConstraintViolation: the actor proposed text that violated // its task's (or goal's) machine-checkable content rules — a // vocabulary allowlist, a deny-pattern or a custom predicate. The loop // blocked it before it ever reached the bot; this finding records // that it was attempted — see deriveMechanicalEventFindings and // spec/ideas/proposal-content-constraints.md. FindingConstraintViolation FindingKind = "constraint-violation" )
Finding kinds. See FindingKind.
type Report ¶
type Report struct {
// SchemaVersion is always ReportSchemaVersion for a Report this package
// produced; a consumer reading an older or newer value should not
// assume today's field meanings.
SchemaVersion int `json:"schemaVersion"`
GoalID string `json:"goalId"`
GoalTitle string `json:"goalTitle"`
// StopReason is the campaign's goal.StopReason (e.g. "goal-complete",
// "budget-steps"), carried as a plain string so Report never imports
// goal's Go type into its own JSON contract.
StopReason string `json:"stopReason"`
Steps int `json:"steps"`
Cost float64 `json:"cost,omitempty"`
Elapsed time.Duration `json:"elapsedNanoseconds"`
Tasks []TaskOutcome `json:"tasks"`
Findings []Finding `json:"findings"`
Usage AggregateUsage `json:"usage"`
}
Report is one campaign run's complete, evidence-linked outcome: versioned, JSON-serialisable, and portable — a consumer needs nothing but this value (plus, optionally, the trace/transcript a Finding's evidence points at) to understand what an actor attempted, what it found, and how sure the report is of each conclusion.
func Assemble ¶
func Assemble(in AssembleInput) Report
Assemble builds a Report from in. Task outcomes are read straight off in.Campaign/in.Goal. Findings are derived mechanically per task:
- a task that reached TaskFailed or TaskBlocked, whose recorded history for that task shows a stale/invalid proposal or an unresolvable action, becomes FindingAINavigationFailure — the campaign never showed the bot itself to be at fault, only that the actor could not complete the task;
- a task never attempted (still TaskPending when the campaign stopped), or attempted but never concluded (non-terminal despite at least one recorded event — e.g. a budget interrupted it mid-task), becomes FindingCoverageGap;
- a cleanly failed/blocked task (no stale/invalid history) produces no mechanical finding: these mechanics cannot tell a genuine product defect from a persona/constraint dead end, so a verified-defect claim for it must come through AssembleInput.CallerFindings instead.
A TaskCompleted task never produces a finding by itself; its success already lives in its TaskOutcome.
type TaskOutcome ¶
type TaskOutcome struct {
TaskID string `json:"taskId"`
Title string `json:"title,omitempty"`
SuccessCriteria string `json:"successCriteria,omitempty"`
// Status is the task's goal.TaskStatus (e.g. "pending", "completed"),
// carried as a plain string for the same reason as Report.StopReason.
Status string `json:"status"`
// Attempted is true once at least one actor.LoopEvent was recorded for
// this task.
Attempted bool `json:"attempted"`
// FailureCount mirrors goal.CampaignState.FailureCount for this task.
FailureCount int `json:"failureCount"`
}
TaskOutcome is one task's result within the campaign, evidence-grounded: Attempted reflects whether the loop actually recorded any LoopEvent for this task, not merely its terminal status.
Source Files