audit

package
v0.11.0-beta.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 23, 2026 License: Apache-2.0, MIT Imports: 16 Imported by: 0

Documentation

Overview

Package audit — completion auditor that verifies agent output.

After the agent claims "done," the auditor compares pre-task and post-task state to detect lazy/incomplete work:

  • Were files actually modified that were supposed to change?
  • Do plan items marked "complete" have corresponding file changes?
  • Do build + tests still pass?
  • Did the diff show the claimed behavior change?

The auditor can optionally spawn a read-only sub-agent with read-only file/search/git tools to cross-check claims against reality.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func WithTranscript

func WithTranscript(ctx context.Context, transcript string) context.Context

WithTranscript supplies actual session history as bounded evidence. It does not replace explicit completion claims or the stored filesystem baseline.

Types

type Auditor

type Auditor struct {
	WorkDir  string         // repo root
	SubAgent SubAgentRunner // optional — spawns read-only verification sub-agent
	// contains filtered or unexported fields
}

Auditor runs completion verification.

func (*Auditor) Audit

func (a *Auditor) Audit(ctx context.Context, pre *Snapshot, claims []Claim) *Report

Audit runs the full verification pass, comparing pre to post state against what the agent claimed.

func (*Auditor) AuditCaptured

func (a *Auditor) AuditCaptured(ctx context.Context, claims []Claim) (*Report, error)

func (*Auditor) CaptureBaseline

func (a *Auditor) CaptureBaseline(ctx context.Context) (*Snapshot, error)

CaptureBaseline stores a pre-task snapshot on this session's auditor. The client cannot substitute hashes or a different workspace in an audit RPC.

type CheckResult

type CheckResult struct {
	Language  string   `json:"language"`
	Kind      string   `json:"kind"`
	Directory string   `json:"directory"`
	Command   []string `json:"command,omitempty"`
	Status    string   `json:"status"`
	Output    string   `json:"output,omitempty"`
}

CheckResult records the actual command and scope behind a verification. An absent runner/script is unavailable, never evidence of a passing build.

type Claim

type Claim struct {
	Description string   `json:"description"` // e.g. "Added error handling to auth.go"
	Files       []string `json:"files"`       // files that should have changed
	TestsPass   bool     `json:"tests_pass"`  // agent claims tests pass
	BuildPass   bool     `json:"build_pass"`  // agent claims build passes
}

Claim is something the agent asserts it accomplished.

type Finding

type Finding struct {
	Severity string `json:"severity"` // "ok", "warn", "fail"
	Claim    string `json:"claim"`
	Expected string `json:"expected"`
	Actual   string `json:"actual"`
	Evidence string `json:"evidence"` // grep output, diff snippet, etc.
}

Finding is a single discrepancy the auditor discovered.

type Report

type Report struct {
	Passed   bool          `json:"passed"`
	Findings []Finding     `json:"findings"`
	PreSHA   string        `json:"pre_sha"`
	PostSHA  string        `json:"post_sha"`
	DiffStat string        `json:"diff_stat"`
	BuildOK  bool          `json:"build_ok"`
	TestsOK  bool          `json:"tests_ok"`
	Duration string        `json:"duration"`
	Checks   []CheckResult `json:"checks"`
}

Report is the full audit result.

func (*Report) ToRevisionPrompt

func (r *Report) ToRevisionPrompt() string

ToRevisionPrompt converts audit findings into a prompt the agent can use to fix its lazy/incomplete work. Returns empty string if all pass.

type Snapshot

type Snapshot struct {
	Time    time.Time         `json:"time"`
	GitSHA  string            `json:"git_sha"`
	GitDiff string            `json:"git_diff"` // "git diff --stat" output
	Files   map[string]string `json:"files"`    // path → sha256 (only tracked files in scope)
	Dirty   bool              `json:"dirty"`    // working tree had uncommitted changes
}

Snapshot captures repo state before/after a task.

func TakeSnapshot

func TakeSnapshot(ctx context.Context, workDir string) (*Snapshot, error)

TakeSnapshot captures the current state of the repo.

type SubAgentRunner

type SubAgentRunner interface {
	Run(ctx context.Context, prompt string) (string, error)
}

SubAgentRunner is the minimal interface for spawning a verification sub-agent. The wiring layer provides this (e.g., delegate to Hermes sub-agent or OpenAI).

type SubAgentRunnerAdapter

type SubAgentRunnerAdapter struct {
	Manager *subagent.Manager
	Model   string // model to use for audit sub-agent
	WorkDir string
}

SubAgentRunnerAdapter wraps a subagent.Manager so the completion auditor can spawn verification workers with enforced read-only tools.

func (*SubAgentRunnerAdapter) Run

func (a *SubAgentRunnerAdapter) Run(ctx context.Context, prompt string) (string, error)

Run spawns a read-only sub-agent with file, bounded search, and fixed git tools to semantically verify the agent's claims against the actual diff.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL