assay

package
v0.20.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 4, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Overview

Package assay implements Forge's AI pull-request review engine ("Assay").

Review runs a multi-pass review over a PR diff: a cheap Triage pass scopes which files warrant deeper inspection, then five deep passes (Logic, Security, Conventions, Tests-missing, Repo-specific) run in parallel and emit structured findings. Findings are then aggregated in order: deduplicated by a stable content hash, then (on a repeat review of the same PR) already-posted Nits are suppressed, then the remaining Nits are capped at the configured Nit budget.

Every finding is idempotent per PR head SHA: re-running Review against the same head produces the same hashes, and persistence is OR-IGNORE keyed on the hash, so a finding is recorded exactly once.

The engine never hard-codes a model identifier — all model selection flows through Config (ModelTier / TriageModel / ReviewModel). See config.go.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Config

type Config struct {
	// ModelTier is a semantic label describing the desired model strength
	// (e.g. "default", "fast"). It is recorded for observability and may be
	// surfaced to operators; the concrete model identifiers are always the
	// TriageModel/ReviewModel hints below, never a literal baked into the code.
	ModelTier string

	// TriageProvider and ReviewProvider are provider specs (the same syntax as
	// settings.providers, e.g. "claude", "gemini/gemini-2.5-pro"). Empty
	// defaults to the Claude provider. TriageProvider drives the cheap scoping
	// pass; ReviewProvider drives the five deep passes.
	TriageProvider string
	ReviewProvider string

	// TriageModel and ReviewModel are model hints (provider-specific model
	// identifiers) used for the triage pass and the deep passes respectively.
	// Empty means "let the provider pick its default model".
	TriageModel string
	ReviewModel string

	// NitCap caps the number of Nit-severity findings retained after
	// aggregation. Values <= 0 mean "no cap".
	NitCap int

	// ShadowMode, when true, signals that the review must not produce public
	// side effects: Review still writes findings to pr_findings, but the caller
	// must not post them. Defaults to the safe value (true) via FromAssayConfig.
	ShadowMode bool

	// MaxDiffBytes caps the size of the diff embedded in pass prompts. Values
	// <= 0 fall back to diff.MaxBytes.
	MaxDiffBytes int

	// SkipPaths are doublestar globs whose files are excluded from review
	// (their hunks are dropped before the diff reaches any pass).
	SkipPaths []string

	// AutoGeneratedPatterns are doublestar globs whose hunks are filtered out
	// before review (machine-generated snapshots, etc.). Empty uses
	// diff.AutoGeneratedPatterns.
	AutoGeneratedPatterns []string
	// contains filtered or unexported fields
}

Config is the resolved configuration the Assay review engine needs for a single review. It is deliberately decoupled from config.AssayConfig (the on-disk YAML shape) so the engine can be exercised in tests without loading real configuration. Build one from a config.AssayConfig via FromAssayConfig, or construct it directly in tests.

IMPORTANT: every model identifier consumed by the engine comes from ModelTier or the TriageModel/ReviewModel hints — the package NEVER hard-codes a concrete model ID. When a model hint is empty the underlying provider chooses its own default model.

func DefaultConfig

func DefaultConfig() Config

DefaultConfig returns a Config with conservative defaults: shadow mode on, no model hints (providers pick their own defaults), and the shared diff auto-generated filters.

func FromAssayConfig

func FromAssayConfig(ac config.AssayConfig) Config

FromAssayConfig builds an engine Config from the resolved on-disk Assay configuration. Unset fields inherit the DefaultConfig values so the engine always has a usable configuration.

func (Config) WithRunner

func (c Config) WithRunner(r PassRunner) Config

WithRunner returns a copy of the Config with the given pass runner installed. External callers use this to inject a custom runner; the package's own tests may set the unexported field directly.

type ExistingFinding

type ExistingFinding struct {
	Anchor string
	Body   string
}

ExistingFinding is the narrow shape suppressSimilarToExisting needs from a persisted finding: just the anchor and body. Defining it locally keeps the dedup helper decoupled from state.Finding (and from sql/db wiring in tests).

type Finding

type Finding struct {
	// File is the affected file path (b-side path from the diff).
	File string `json:"file"`
	// Anchor locates the finding within the file (e.g. "path/to/file.go:42").
	Anchor string `json:"anchor"`
	// Category is a short machine label for the kind of issue (defaults to the
	// emitting pass name when the model omits it).
	Category string `json:"category"`
	// Severity is the triaged seriousness of the finding.
	Severity Severity `json:"severity"`
	// Title is a one-line summary.
	Title string `json:"title"`
	// Body is the full explanation.
	Body string `json:"body"`
	// Evidence is an optional supporting snippet (diff lines, etc.).
	Evidence string `json:"evidence"`
	// SourcePass records which pass produced the finding.
	SourcePass string `json:"-"`
	// Hash is the dedup key: sha256(anchor + category + canonical(body)).
	// Computed by the engine, not the model.
	Hash string `json:"-"`
}

Finding is a single code-review observation emitted by a pass.

type PassOutput

type PassOutput struct {
	// Text is the model's textual response (parsed for JSON downstream).
	Text string
	// CostUSD is the estimated cost of the invocation.
	CostUSD float64
}

PassOutput is the raw result of a single model invocation.

type PassProvider

type PassProvider struct {
	// Pass is the pass identifier ("triage", "logic", "security", …).
	Pass string
	// Provider is the resolved provider (Kind/Cmd/Model) for the pass, derived
	// from the Config's per-tier provider/model hints.
	Provider provider.Provider
}

PassProvider names a single Assay pass and the resolved provider that runs it. doctor uses this to verify each pass's CLI binary (typically `claude`) is available before a review is attempted.

func PassProviders

func PassProviders(c Config) []PassProvider

PassProviders returns the resolved provider for every Assay pass — the cheap triage scoping pass plus the five deep finding passes — given a Config. The concrete provider for each pass comes entirely from the Config's tier hints (never a hard-coded model); an empty hint resolves to the Claude provider.

type PassReport

type PassReport struct {
	// Name is the pass identifier (e.g. "logic").
	Name string
	// Findings is the number of findings the pass contributed before
	// aggregation.
	Findings int
	// CostUSD is the model cost attributed to the pass.
	CostUSD float64
}

PassReport captures per-pass execution metadata for observability.

type PassRunner

type PassRunner func(ctx context.Context, pass, tier, prompt string) (PassOutput, error)

PassRunner invokes a model for one pass and returns its output. It is the engine's single seam to the AI backend: the default implementation spawns the Claude CLI via smith, while tests inject a deterministic stub. pass is the pass name, tier is the model tier, and prompt is the fully-built prompt.

type PostRequest

type PostRequest struct {
	// Anvil is the repository (anvil) name; keys the persisted rows.
	Anvil string
	// PRNumber is the pull request to comment on.
	PRNumber int
	// HeadSHA is the commit the inline comments are anchored to (commit_id).
	HeadSHA string
	// WorktreePath is the directory gh commands run in (the PR worktree). gh
	// resolves the {owner}/{repo} API placeholders from this checkout.
	WorktreePath string
	// SummaryLine is the one-line verdict shown above the severity table in the
	// top-level review comment. When empty (and there are no findings) the
	// summary review is skipped.
	SummaryLine string
	// Findings is the aggregated set to post (already deduped/suppressed/capped
	// by Review).
	Findings []Finding
}

PostRequest carries everything the posting layer needs to publish a review.

type PostResult

type PostResult struct {
	// SummaryPosted reports whether the top-level review comment was posted.
	SummaryPosted bool
	// Posted is the number of inline comments successfully posted.
	Posted int
	// Failed is the number of inline comments whose gh call failed (left with
	// posted=0 so they retry on the next head).
	Failed int
	// Resolved is the number of stale findings whose threads were resolved
	// after crossing the consecutive-miss threshold.
	Resolved int
}

PostResult summarizes a posting run.

type Poster

type Poster struct {
	// contains filtered or unexported fields
}

Poster publishes Assay findings to a GitHub PR: a top-level summary review, one inline comment per finding, and auto-resolution of threads for findings that have disappeared across consecutive reviews. Build one with NewPoster.

The zero value is not usable; gh and logf are populated by NewPoster. db and resolver may be nil — a nil db skips all persistence (so nothing is recorded or auto-resolved), a nil resolver skips thread resolution only.

func NewPoster

func NewPoster(db *state.DB, resolver ThreadResolver) *Poster

NewPoster builds a Poster wired to the real gh CLI. db persists posting state (gh_comment_id / gh_thread_id / consecutive_misses); resolver performs thread lookup and resolution. Either may be nil.

func (*Poster) Post

func (p *Poster) Post(ctx context.Context, cfg Config, req PostRequest) (*PostResult, error)

Post publishes req's findings to the PR. When cfg.ShadowMode is true it is a no-op (the engine must not produce public side effects in shadow mode) and returns a zero result.

Posting never aborts on a single gh failure: each inline comment is attempted independently, failures are logged, and the corresponding finding keeps posted=0 so it is retried on the next head SHA. The only returned error is a fatal one that prevents the whole run (currently none — Post always returns a result and nil unless a future fatal path is added).

type ReviewRequest

type ReviewRequest struct {
	// Anvil is the repository (anvil) name; used to key persisted rows.
	Anvil string
	// AnvilPath is the local checkout path, used to read repo-level context
	// (REVIEW.md, AGENTS.md, etc.). Optional — may be empty. When set and
	// RepoGuidance is empty, Review() reads <AnvilPath>/REVIEW.md into
	// RepoGuidance once per run so every pass sees the same trusted guidance.
	AnvilPath string
	// RepoGuidance carries the repository's REVIEW.md content (or any
	// equivalent trusted instructions) injected into every pass prompt at
	// highest priority. Optional — empty means the engine uses default
	// calibration only. Pre-populated by Review() from AnvilPath when unset.
	RepoGuidance string
	// PRNumber is the pull request number under review.
	PRNumber int
	// HeadSHA is the PR head commit OID. It is the idempotency key: findings are
	// recorded against this SHA and a repeat review of the same SHA is a no-op.
	HeadSHA string
	// Diff is the unified git diff to review.
	Diff string
	// BeadID is the originating bead, used for context/logging. Optional.
	BeadID string
	// Title is the PR/bead title, supplied to passes as scope context. Optional.
	Title string
	// Description is the PR/bead body, supplied as scope context. Optional.
	Description string
	// WorkDir is the directory the default Smith-based runner executes in
	// (typically the PR worktree). Ignored when a custom runner is installed.
	WorkDir string
}

ReviewRequest carries everything the engine needs to review one PR head.

type ReviewResult

type ReviewResult struct {
	// Findings is the final, aggregated set (deduped, suppressed, capped).
	Findings []Finding
	// HeadSHA echoes the reviewed head SHA.
	HeadSHA string
	// ShadowMode reports whether the review ran in shadow mode (no posting).
	ShadowMode bool
	// CostUSD is the total model cost across all passes.
	CostUSD float64
	// Duration is the wall-clock time spent in Review.
	Duration time.Duration
	// Passes holds per-pass metadata.
	Passes []PassReport
	// NitsCapped is the number of Nit findings dropped by the Nit cap.
	NitsCapped int
	// NitsSuppressed is the number of Nit findings dropped because they were
	// already posted on a prior review of this PR.
	NitsSuppressed int
	// PassErrors lists per-pass error strings (formatted "<pass>: <reason>")
	// for any deep pass that failed. A non-empty PassErrors with a non-nil
	// result means at least one pass produced findings; aggregation only
	// fails hard when every deep pass errored.
	PassErrors []string
}

ReviewResult is the outcome of a Review.

func Review

func Review(ctx context.Context, req ReviewRequest, db *state.DB, cfg Config) (*ReviewResult, error)

Review runs the multi-pass Assay review for req and returns the aggregated result. db may be nil to skip all persistence and posted-Nit suppression (useful for dry runs); when non-nil, findings are written to pr_findings (OR IGNORE, so the call is idempotent per HeadSHA).

A pass that cannot produce parseable JSON after one retry is surfaced as a run error and aborts the whole review.

type Severity

type Severity string

Severity classifies how seriously a finding should be treated.

const (
	// SeverityImportant marks a finding that should block or strongly influence
	// the merge decision. Important findings are never capped or suppressed.
	SeverityImportant Severity = "Important"
	// SeverityNit marks a low-priority style/polish finding. Nits are subject to
	// the Nit cap and to already-posted suppression on re-runs.
	SeverityNit Severity = "Nit"
	// SeverityPreExisting marks an issue that predates the diff under review.
	SeverityPreExisting Severity = "PreExisting"
)

type ThreadResolver

type ThreadResolver interface {
	// ThreadIDByBodyHeader returns the platform thread ID whose top comment
	// body contains header, or "" when none matches.
	ThreadIDByBodyHeader(ctx context.Context, worktreePath string, prNumber int, header string) (string, error)
	// ResolveThread marks the identified review thread as resolved.
	ResolveThread(ctx context.Context, worktreePath string, threadID string) error
}

ThreadResolver locates and resolves review threads on a PR. It is satisfied by *github.Provider (see ThreadIDByBodyHeader / ResolveThread); the posting layer depends on the interface, not the concrete provider, so it stays testable and free of an import cycle.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL