Help us shape the future of Go! Take the 2026 Go Developer Survey

cigate

package
v1.0.14 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 2, 2026 License: Apache-2.0 Imports: 26 Imported by: 0

Documentation

Overview

Package cigate is the local CI gate: the one definition of what "CI passes" means for a forge project, runnable on a laptop and from a workflow.

A scaffolded workflow is a thin wrapper that prepares the environment (checkout, toolchains, caches) and calls `task ci:checks`, `task test` and `task test:integration`. `task ci` runs all three locally, so "CI did not run" never has to mean "unverified". This package is what makes the first and last of those honest, with four rules:

  1. Every check runs, even after one fails, so one pass shows every failure. The verdict is a summary table, and the gate exits non-zero if any row failed.
  2. A check that could not run (a missing tool or prerequisite) is SKIPPED, never PASS, and fails the gate unless CI_SKIP names it. A silent skip is false confidence. Rows a project's own configuration or kind makes moot are N/A, which is not a skip and does not fail the gate.
  3. Tools run at the versions CI pins (tools.go), not whatever is on PATH.
  4. Under GitHub Actions the table is appended to $GITHUB_STEP_SUMMARY.

Checks run CONCURRENTLY (scheduler below), because a gate whose ~25 steps run one after another takes the sum of them — measured at a p50 of 35 min and up to 3 h on a shared machine. Three things bound the concurrency:

  • `after` dependencies: a check starts once the checks it names have finished (whatever their verdict — rule 1 still holds);
  • tree mutation: a check that writes tracked files (a formatter, an extra declared `mutates_tree`) runs alone, before every check that only reads the tree. verify-generated is not one: it regenerates a scratch copy;
  • host slots: a heavy check (a whole-module build, test run or render) holds one of the machine's heavy slots (pkg/hostslot) while it runs, so the gate never adds more heavy jobs than the host has room for, and a per-gate job cap bounds the rest.

Each check's output goes to its own log file; the console gets one start and one finish line per check, a failing check's last lines, and the table.

Index

Constants

View Source
const (
	GroupSetup       = "setup"
	GroupStatic      = "static"
	GroupKCL         = "kcl"
	GroupTest        = "test"
	GroupIntegration = "integration"
	GroupExtras      = "extras"
)

Groups forge's own checks carry, for CI_ONLY (`CI_ONLY=kcl`). A project's extra checks default to GroupExtras.

View Source
const BaseEnv = "CI_BASE"

BaseEnv names the ref a branch's changes are measured from, for a branch stacked on another (default: origin/<default branch>).

View Source
const CacheDirEnv = "FORGE_CI_CACHE_DIR"

CacheDirEnv relocates the verdict cache.

View Source
const FullEnv = "CI_FULL"

FullEnv forces the full unit suite on a branch.

View Source
const NoCacheEnv = "CI_NO_CACHE"

NoCacheEnv turns the verdict cache off for one run.

View Source
const OnlyEnv = "CI_ONLY"

OnlyEnv names the checks (or groups) a caller wants to run, comma-separated; everything else is left out of the run and the table.

View Source
const SkipEnv = "CI_SKIP"

SkipEnv names the checks a caller has decided not to run, comma-separated.

View Source
const UnitTests = "unit-tests"

UnitTests is the name of the unit-test row, which affected-package mode (affected.go) narrows on a branch.

Variables

This section is empty.

Functions

func DefaultJobs added in v0.1.46

func DefaultJobs() int

DefaultJobs is the per-gate cap on concurrently running light checks: half the CPUs, between 2 and 8. Heavy checks are bounded by host slots instead.

func GateEnv added in v0.1.46

func GateEnv(cfg *config.ProjectConfig) []string

GateEnv is ci.gate.env as KEY=VALUE entries, for the base environment of every check.

func GatePath added in v0.1.46

func GatePath(root string, cfg *config.ProjectConfig) []string

GatePath is ci.gate.path made absolute against root.

func HasIntegrationTests

func HasIntegrationTests(root string) bool

HasIntegrationTests reports whether any Go test file under root carries the `integration` build tag. The scaffolded workflow asks the same question in its `changes` job (git grep for the same constraint) to decide whether the integration-test job runs at all.

func ParseSkips

func ParseSkips(v string) []string

ParseSkips splits a CI_SKIP (or CI_ONLY) value into names.

func UnknownOnly added in v0.1.46

func UnknownOnly(only []string, all []Check) []string

UnknownOnly returns the CI_ONLY names that are neither a check nor a group.

func UnknownSkips

func UnknownSkips(skips []string, all []Check) []string

UnknownSkips returns the names in skips that no check carries. A typo in CI_SKIP would otherwise quietly skip nothing, or — worse — be read as the caller having accounted for a check they did not.

Types

type Affected added in v0.1.46

type Affected struct {
	// Full means run everything; Reason says why.
	Full   bool
	Reason string
	// Packages are the import paths to test (affected mode).
	Packages []string
	// Total is how many packages in the module have tests.
	Total int
	// BaseRef is the ref diffed against (origin/main); Base its merge-base
	// with HEAD, short.
	BaseRef, Base string
	// Changed are the changed files, project-relative.
	Changed []string
}

Affected is the unit lane's mode for one gate run.

func ComputeAffected added in v0.1.46

func ComputeAffected(ctx context.Context, in AffectedInput) Affected

ComputeAffected decides the unit lane's mode for the work tree at in.Root. It never fails: anything it cannot decide is a reason to run everything.

func (Affected) Line added in v0.1.46

func (a Affected) Line() string

Line is the one line the gate prints about the unit lane's mode.

type AffectedInput added in v0.1.46

type AffectedInput struct {
	Root string
	// Env is the environment go list runs with; nil is the process's.
	Env []string
	// Force, when non-empty, is the reason to run the full suite regardless
	// (CI_FULL=1, --full).
	Force string
	// BaseRef overrides origin/<default>: the ref a stacked branch diffs
	// against (CI_BASE / --base).
	BaseRef string
	// Ignore are ci.gate.affected_ignore's globs: files no Go test reads.
	Ignore []string
	// Git and GoList run the two tools; nil uses the real binaries. Tests
	// substitute.
	Git    func(ctx context.Context, dir string, args ...string) (string, error)
	GoList func(ctx context.Context, dir string, env []string) (io.Reader, error)
}

AffectedInput configures ComputeAffected.

type CacheInput added in v0.1.46

type CacheInput struct {
	// Root is the project root (a git work tree).
	Root string
	// Dir is the cache root ($FORGE_CI_CACHE_DIR); empty is
	// <user cache dir>/forge/ci-verdicts.
	Dir string
	// Forge identifies the judging forge binary.
	Forge string
	// Frontends are the project-relative frontend directories, whose
	// package.json `file:` / `link:` dependencies are outside inputs.
	Frontends []string
	// Env is the gate's base environment.
	Env []string
}

CacheInput is what OpenVerdictCache keys verdicts on, beyond the tree.

type Check

type Check struct {
	Name string
	Lane Lane
	// Group is the label CI_ONLY can select by, besides the name.
	Group string
	// After names checks that must finish (with any verdict) before this one
	// starts. Selecting this check with CI_ONLY selects them too.
	After []string
	// Heavy makes the check hold a host-wide heavy slot while it runs.
	Heavy bool
	// MutatesTree marks a check that writes tracked files: it runs alone,
	// ahead of every check that only reads the tree.
	MutatesTree bool
	// NoCache keeps the verdict cache from ever answering for this check:
	// its verdict depends on state outside the tree.
	NoCache bool

	// NotApplicable, when set, reports the row as N/A with this reason and
	// runs nothing.
	NotApplicable string
	// Done returns a non-empty note when the check's work is already done
	// (dependencies installed); the row is PASS and nothing runs.
	Done func() string
	// Prereq returns a non-empty reason when the check cannot run here. The
	// row is then SKIPPED, which fails the gate.
	Prereq func() string
	// Requires lists binaries that must resolve on the check's own PATH
	// (pinned tool dirs and ci.gate.path included); a missing one SKIPs it.
	Requires []string
	// Probe is a command that must succeed before the check runs; when it
	// fails the row is SKIPPED with ProbeReason.
	Probe       []string
	ProbeReason string
	// Tools are pinned tools the command needs. Each is resolved to its
	// pinned version before the command runs, and its directory is put first
	// on the command's PATH.
	Tools []Tool

	Argv []string
	// SlotArgv, when set, runs instead of Argv while the check holds a host
	// heavy slot of its own. A tool with its own machine-global mutex drops
	// it here, because the slot already bounds what the mutex protected
	// (golangci-lint: --allow-parallel-runners instead of
	// --allow-serial-runners). Without a slot of its own — the limit is off,
	// or the gate itself runs inside a parent's slot — Argv runs.
	SlotArgv []string
	// Dir is project-relative; empty is the project root.
	Dir string
	Env []string
	// SkipExitCode, when non-zero, is the exit status by which the command
	// says it could not run here: the row is SKIPPED with its last output
	// line as the reason.
	SkipExitCode int
	// Note annotates the row when the command runs: how it was narrowed
	// (affected-package mode) or anything else the reader needs.
	Note string

	// OnFail runs after the command failed, to undo what it left behind.
	OnFail func(out io.Writer)
}

Check is one row of the gate.

func ApplyAffected added in v0.1.46

func ApplyAffected(checks []Check, a Affected) []Check

ApplyAffected narrows the unit-tests row to a's packages. In full mode it only annotates the row.

func Only added in v0.1.46

func Only(checks []Check, only []string) []Check

Only keeps the checks CI_ONLY names — by name or by group — plus every check they depend on (`after`), transitively, in plan order. An empty only keeps everything.

func Plan

func Plan(in PlanInput) ([]Check, error)

Plan returns every check of every lane, in the order the table prints them. The order is not the execution order: the scheduler (Run) starts each check as soon as its dependencies, the tree and a host slot allow.

func Select

func Select(all []Check, lanes []Lane) []Check

Select keeps the checks of the named lanes, in plan order, dropping a check whose name an earlier selected lane already ran (the frontend-deps row sits in both the checks and test lanes, so `task ci` installs once).

type DurationHistory added in v0.1.46

type DurationHistory struct {
	// contains filtered or unexported fields
}

DurationHistory is how long each check took the last time its command ran in this repository (any worktree, any tree), so the next run can start the checks on the longest path first (executionOrder).

WHY. Host slots are granted in execution order, and without history that is plan order: forge's static checks, then the project's extras, then the test lanes. On control-plane that queued its two longest checks — the unit suite (~15 min) and the integration suite (~22 min under load) — behind ~25 slot- minutes of shorter checks, so the gate's wall time was the queue plus the longest check instead of roughly the longest check. Starting the longest path first is the classic list-scheduling answer, and the only input it needs is a duration per check.

A duration is a scheduling hint, not a verdict: it is recorded whatever the tree's state, it is never trusted for correctness, and a stale one only reorders when checks start.

func OpenDurationHistory added in v0.1.46

func OpenDurationHistory(ctx context.Context, root, dir string) *DurationHistory

OpenDurationHistory reads root's history from dir ("" is the verdict cache's default directory). It returns nil — plan order — when root is not a git work tree or there is no cache directory.

func (*DurationHistory) Expected added in v0.1.46

func (h *DurationHistory) Expected() map[string]time.Duration

Expected is the recorded duration of every check that has one; nil for a nil history.

func (*DurationHistory) Observe added in v0.1.46

func (h *DurationHistory) Observe(rep Report)

Observe records how long each check in rep took when its command ran to a verdict. A cached, skipped or N/A row says nothing about the check's duration, so it leaves the recorded one alone.

func (*DurationHistory) Save added in v0.1.46

func (h *DurationHistory) Save() error

Save writes the history atomically. Two gates finishing at once each write a whole file and the last one wins, which is all a hint needs.

type ExecFunc

type ExecFunc func(ctx context.Context, dir string, env []string, argv []string, out io.Writer) error

ExecFunc runs argv in dir with env, writing combined output to out.

type HeldSlot added in v0.1.46

type HeldSlot interface {
	// Env is what the check's command inherits, so the heavy commands it
	// runs in turn (`task test`'s `forge slot run`, verify-generated's
	// `forge generate`) run inside this slot instead of queueing for another.
	Env() []string
	// SetLabel names the check holding the slot, for waiters to see.
	SetLabel(label string)
	Release() error
}

HeldSlot is one host slot as the scheduler uses it (*hostslot.Slot is one).

type Lane

type Lane string

Lane is a group of checks a CI job runs together.

const (
	// LaneChecks is the static and build lane: the `checks` CI job.
	LaneChecks Lane = "checks"
	// LaneTest is the unit-test lane: `task test`, the `test` CI job.
	LaneTest Lane = "test"
	// LaneIntegration is `task test:integration`, the `integration-test` job.
	LaneIntegration Lane = "integration"
)

func Lanes added in v0.1.46

func Lanes(all []Check) []Lane

Lanes lists every lane the plan has, built-in ones first.

type Options

type Options struct {
	// Out receives the progress lines and the checks' failure tails. Defaults
	// to os.Stdout.
	Out io.Writer
	// Skip names the checks the caller chose not to run.
	Skip []string
	// Resolver pins tools. Nil means checks that declare Tools cannot run.
	Resolver *Resolver
	// Exec runs one command. Nil uses os/exec; tests substitute.
	Exec ExecFunc
	// Root is the project root.
	Root string
	// Groups wraps each check's output in GitHub Actions ::group:: lines,
	// printed whole when the check finishes so concurrent checks never
	// interleave.
	Groups bool
	// Env is the base environment; nil is os.Environ().
	Env []string
	// Path is put first on every check's PATH, ahead of the pinned tools'
	// directories: the project's own declared tool directories
	// (ci.gate.path), which it chose over forge's pins on purpose.
	Path []string
	// Jobs caps how many checks run at once. Zero is DefaultJobs().
	Jobs int
	// Slot takes a host heavy slot for a Heavy check. Nil runs heavy checks
	// without one.
	Slot SlotFunc
	// Verdicts answers for checks whose PASS on this exact tree is already
	// recorded, and records new ones. Nil disables the cache.
	Verdicts Verdicts
	// LogDir receives one <check>.log per check that runs. Empty keeps the
	// output in memory only (tests).
	LogDir string
	// Clock stamps durations; nil is time.Now (tests substitute).
	Clock func() time.Time
	// Expected is how long each check took when it last ran
	// (DurationHistory). The checks that only read the tree start longest
	// path first by it; nil keeps plan order.
	Expected map[string]time.Duration
}

Options configure a Run.

type PlanInput

type PlanInput struct {
	// Root is the project root.
	Root string
	// CI is the data the scaffolded ci.yml renders with: which lints, scanners
	// and jobs the project has. The gate reads the same struct so a toggle
	// means the same thing in both.
	CI templates.CIWorkflowData
	// Cfg supplies the project's extra checks and its derived features.
	Cfg *config.ProjectConfig
	// Forge is the argv that re-enters this forge binary.
	Forge []string
}

PlanInput is everything the plan is derived from.

type Report

type Report struct {
	Lane    string
	Results []Result
	// Head is the commit the gate judged ("" outside git); Dirty says the
	// working tree had uncommitted changes on top of it.
	Head  string
	Dirty bool
	// Wall is the gate's elapsed time.
	Wall time.Duration
}

Report is every row of one gate run.

func Run

func Run(ctx context.Context, lane string, checks []Check, opt Options) Report

Run executes checks and returns the report, rows in the order given. It never stops early: a failing check is a row, not an abort.

func (Report) Counts

func (r Report) Counts() Tally

Counts tallies the rows by outcome.

func (Report) Failed

func (r Report) Failed() bool

Failed reports whether the gate is red: any FAIL, or any SKIPPED the caller did not allow.

func (Report) RenderMarkdown

func (r Report) RenderMarkdown(w io.Writer)

RenderMarkdown is the same table for $GITHUB_STEP_SUMMARY.

func (Report) RenderText

func (r Report) RenderText(w io.Writer)

RenderText writes the summary table.

type Resolver

type Resolver struct {
	// CacheRoot is where pinned tools are installed.
	CacheRoot string
	// ProjectDir selects the Go toolchain tools are built with: the one `go`
	// picks for the project, so a linter is never built with a Go older than
	// the module it must lint.
	ProjectDir string
	// Note receives one line per install, so a first-run build is explained.
	Note io.Writer

	LookPath func(string) (string, error)
	// Probe reports the version of the binary at path, or "".
	Probe func(path string, t Tool) string
	// Install builds t into dir.
	Install func(ctx context.Context, t Tool, dir, toolchain string) error
	// Toolchain reports the Go release `go` selects for ProjectDir.
	Toolchain func(ctx context.Context) string
	// ModuleGo is the project's go directive ("1.27.0"), "" when unknown.
	ModuleGo string
	// BuiltWith reports the Go release a binary was built with ("go1.27.2").
	BuiltWith func(path string) string
	// contains filtered or unexported fields
}

Resolver turns a Tool into a directory to put first on PATH.

func NewResolver

func NewResolver(projectDir string, note io.Writer) *Resolver

NewResolver is the production resolver.

func (*Resolver) Ensure

func (r *Resolver) Ensure(ctx context.Context, t Tool) (string, error)

Ensure returns the directory to prepend to PATH so t resolves at its pin, or "" when the binary on PATH already is the pin.

type Result

type Result struct {
	Name     string
	Lane     Lane
	Status   Status
	Duration time.Duration
	Note     string
	// Allowed marks a SKIPPED row the caller asked for (CI_SKIP), which does
	// not fail the gate.
	Allowed bool
	// Cached is the short commit a reused PASS was recorded at; "" when the
	// check ran.
	Cached string
	// Log is the check's output file, project-relative when under the root.
	Log string
	// Tail is the end of a failed command's output, for the job summary.
	Tail string
}

Result is the outcome of one Check.

type SlotFunc added in v0.1.46

type SlotFunc func(ctx context.Context) (HeldSlot, error)

SlotFunc takes one host-wide heavy slot, waiting while every slot is held. An error means the gate was interrupted while waiting.

type Status

type Status string

Status is one row's verdict.

const (
	// Pass is a check that ran and succeeded.
	Pass Status = "PASS"
	// Fail is a check that ran and failed.
	Fail Status = "FAIL"
	// Skip is a check that did not run. It fails the gate unless Allowed.
	Skip Status = "SKIPPED"
	// NA is a check this project has nothing for. It is not a skip.
	NA Status = "N/A"
)

type Tally

type Tally struct{ Passed, Cached, Failed, Skipped, Allowed, NA int }

Tally is the rows of a report counted by outcome. Skipped counts only the SKIPPED rows that fail the gate; Allowed counts the ones CI_SKIP named.

type Tool

type Tool struct {
	Binary string
	// Module is the `go install` package path.
	Module string
	// BuildModule is the module path the binary's build info records, when
	// that differs from Module's directory (golangci-lint's /cmd/ suffix).
	BuildModule string
	// Version is the pin, with the leading "v".
	Version string
	// MatchGo requires the binary to have been built with a Go at least as
	// new as the project's go directive: golangci-lint cannot analyze Go newer
	// than the Go it was built with, so a PATH copy at the right VERSION but
	// built with an older Go is the wrong binary.
	MatchGo bool
	// VersionArgs and VersionRE read the version from the binary's own output
	// when its build info does not carry one (release builds record
	// "(devel)").
	VersionArgs []string
	VersionRE   *regexp.Regexp
}

Tool is a binary the gate runs at a pinned version. CI installs these at the same versions (the scaffolded ci.yml renders the same templates constants), so a developer's `task ci` and the workflow judge the same code with the same linters. A tool on PATH at another version is not used: the pin is installed into forge's tool cache and put ahead of it for that command only. The developer's GOBIN and shell PATH are never touched.

func BufTool added in v1.0.0

func BufTool(root string) Tool

BufTool is the buf the project at root pins (internal/toolpin): its go.mod's github.com/bufbuild/buf when it declares one, forge's pin otherwise. The gate holds PATH to it for verify-generated and buf-lint, so the gate, `forge generate` and the scaffolded ci.yml run one buf.

type VerdictCache added in v0.1.46

type VerdictCache struct {
	// contains filtered or unexported fields
}

VerdictCache is an open cache for one gate run. It implements Verdicts.

func OpenVerdictCache added in v0.1.46

func OpenVerdictCache(ctx context.Context, in CacheInput) (*VerdictCache, string)

OpenVerdictCache opens the cache for the tree at in.Root. It returns nil and the reason, in a few words, when this run cannot use one: not a git work tree, uncommitted changes, an outside input that is not a clean git tree.

func (*VerdictCache) Commit added in v0.1.46

func (v *VerdictCache) Commit() string

Commit is the short commit the cache keys this run on.

func (*VerdictCache) Flush added in v0.1.46

func (v *VerdictCache) Flush(ctx context.Context) (int, string)

Flush stores the PASSes recorded this run — but only if the tree is still clean and still the tree the run started on: a check that rewrote tracked files, or a commit made mid-run, would otherwise file verdicts under a tree they were not earned on. It reports how many it stored, and why none were when the tree moved.

func (*VerdictCache) Key added in v0.1.46

func (v *VerdictCache) Key(c Check) string

Key is the cache key of c on this tree.

func (*VerdictCache) Lookup added in v0.1.46

func (v *VerdictCache) Lookup(c Check) (string, bool)

Lookup implements Verdicts.

func (*VerdictCache) Record added in v0.1.46

func (v *VerdictCache) Record(c Check)

Record implements Verdicts. Nothing is written until Flush.

type Verdicts added in v0.1.46

type Verdicts interface {
	// Lookup reports a recorded PASS for c on the tree being judged, and the
	// short commit it was recorded at.
	Lookup(c Check) (commit string, ok bool)
	// Record remembers a PASS c earned in this run.
	Record(c Check)
}

Verdicts is the verdict cache as the scheduler sees it.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL