Documentation
¶
Overview ¶
Baseline tracking for perf gates (ADR-0019 PR 3).
AssertWithinBaseline reads testdata/snapshots/baselines.toml and fails the test if the measured wall-time exceeds the recorded baseline by more than tolerance_pct percent.
IMPORTANT: this helper does NOT auto-update the baseline on success. Per ADR-0019 D.6, the perf gate is fixed-anchor — explicit rebaseline via `task fixtures:rebaseline` is the only way to bump wall_ms. Auto-rebaseline would convert the gate into a perf ratchet that only loosens (math: at 25% tolerance, 4 merges of +20% compound to 2.07x slower without the gate firing — see ADR-0019 D.6 for the derivation).
Snapshot curation filter for the real-corpus fixture registry. See ADR-0019 section D.7 for the inclusion/exclusion spec.
The filter takes a source directory + language and produces a curated copy at a destination directory. Only source files (per lang.Registry extensions for the requested language) and a small set of project-marker / documentation files are kept; build outputs, SCM metadata, and grammar-bomb binaries are excluded.
Used by tools/fixtures-snapshot/ to bake external snapshots into testdata/snapshots/<id>/; the filter is also covered by package tests so the spec can't silently rot.
Package testfixtures is the real-corpus fixture registry (ADR-0019).
Tests pull fixtures by ID via Get(t, id), which materializes a SQLiteGraph and caches it per-process for re-use across tests in the same package. Cleanup is automatic via t.Cleanup.
PR 1 of ADR-0019 ships only the "mache-self" sentinel fixture which resolves to mache's own repo root at runtime via runtime.Caller. External snapshots (rosary, ley-line-open) are PR 2; baseline tracking is PR 3.
See docs/adr/0019-real-corpus-fixture-registry.md.
Index ¶
- func AssertWithinBaseline(t *testing.T, key, fixtureID string, measured time.Duration)
- func DumpProjection(db *sql.DB, fixtureRoot string) (string, error)
- func Get(t *testing.T, id string) *graph.SQLiteGraph
- func LoadSchema(id string) (*api.Topology, error)
- func ParseBaselinesFile(path string) (map[string][]Baseline, error)
- func RequireTier(t *testing.T, tier string)
- func ResolvePath(id string) (string, error)
- type Baseline
- type CurateOptions
- type CurateResult
- type Fixture
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func AssertWithinBaseline ¶
AssertWithinBaseline checks measured wall-time against the recorded baseline for the (key, fixtureID) pair in baselines.toml.
Behavior per ADR-0019 D.6:
- absent baseline: t.Fatalf with a recommended rebaseline command (the gate refuses to silently pass when no anchor exists)
- measured <= baseline * (1 + tolerance_pct/100): pass; baseline is NOT auto-updated (fixed-anchor policy)
- measured > ceiling: t.Fatalf with both numbers AND a recommended `task fixtures:rebaseline` command so an intentional regression can be explicitly anchored with an audit-trail justification
func DumpProjection ¶ added in v0.22.0
DumpProjection renders the projection db as one deterministic text: a `## <table>` header per section followed by its sorted, tab-separated rows, with fixtureRoot (and its symlink-resolved form) rewritten to $FIXTURE. Two builds of the same source with the same engine produce the same bytes; a differing byte is a projection change.
func Get ¶
func Get(t *testing.T, id string) *graph.SQLiteGraph
Get materializes a fixture and returns an open SQLiteGraph for the test. Caches per-process so repeated calls within one `go test` invocation reuse the same .db. Cleanup is automatic via t.Cleanup at test-binary exit.
On unknown fixture id, calls t.Fatalf — the test fails loudly rather than returning nil.
func LoadSchema ¶
LoadSchema loads the preset schema for a fixture from the public schema package, keeping the embedded preset registry as the single owner.
func ParseBaselinesFile ¶
ParseBaselinesFile parses one baselines.toml at the given path. Extracted (and exported) so tooling (tools/fixtures-rebaseline) and tests can exercise the parser against arbitrary files without touching the package-level cache.
func RequireTier ¶
RequireTier skips the test unless the tier's env-var gate is set.
Tiers:
- "medium" — always on; no env var required.
- "large" — requires MACHE_E2E_LARGE=1 (perf-gate / nightly only).
Unknown tiers fatal-fail rather than skipping silently.
func ResolvePath ¶
ResolvePath returns the on-disk source-tree path for a fixture. "$REPO_ROOT" sentinels expand to mache's own repo root. Other paths resolve relative to testdata/snapshots/.
Types ¶
type Baseline ¶
type Baseline struct {
Fixture string `toml:"fixture"`
WallMs int `toml:"wall_ms"`
TolerancePct int `toml:"tolerance_pct"`
AnchoredAt string `toml:"anchored_at"`
AnchoredAtDate string `toml:"anchored_at_date"`
LastIntentionalBump string `toml:"last_intentional_bump"`
}
Baseline is one entry in baselines.toml — the per-(key, fixture) fixed anchor + tolerance band that perf gates assert against.
`Key` is implicit: it's the table-array name in TOML (e.g. the key in [["find_smells:dead_code"]] is "find_smells:dead_code"). Multiple baselines per key allow one rule to have entries for several fixtures.
type CurateOptions ¶
type CurateOptions struct {
// Source is the absolute path to the upstream working tree.
Source string
// Dest is the absolute path where the curated tree is written.
// If Dest exists, it is removed first; callers should ensure they
// pass a path they own.
Dest string
// Language is the canonical name from lang.Registry (e.g. "go",
// "rust"). Only files with extensions matching this language's
// Extensions slice are copied as "source" files; project-marker and
// documentation files are copied via a per-language allowlist.
Language string
}
CurateOptions controls the snapshot filter.
type CurateResult ¶
CurateResult reports what the filter produced.
func Curate ¶
func Curate(opts CurateOptions) (CurateResult, error)
Curate walks Source and writes a filtered copy under Dest per ADR-0019 D.7.
Inclusion (any of):
- File extension matches lang.Registry[Language].Extensions
- File name is in the per-language project-marker allowlist (e.g. Cargo.toml + Cargo.lock for Rust, go.mod + go.sum for Go)
- File is documentation (README.md / CHANGELOG.md at any level, or *.md anywhere under a docs/ subtree)
Exclusion (overrides inclusion):
- Any path component matches the build-output / SCM / IDE deny set (target, node_modules, .git, .idea, .vscode, etc.)
- testdata/**/*.db and testdata/**/*.tar
- Files named parser.c anywhere (tree-sitter grammar bombs that are valid C source but not the kind of source we want in fixtures)
- .DS_Store
Returns the number of files and bytes copied. Errors from individual file copies are wrapped and returned immediately — partial state is left on disk for the caller to clean up.
type Fixture ¶
type Fixture struct {
ID string `toml:"id"`
Tier string `toml:"tier"` // "medium" | "large"
Language string `toml:"language"` // "go" | "rust" | "polyglot"
Source string `toml:"source"` // "self" for sentinel, else URL or repo path
SHA string `toml:"sha"` // pinned (empty for sentinel)
Path string `toml:"path"` // relative to testdata/snapshots/, OR "$REPO_ROOT" for sentinel
SchemaPreset string `toml:"schema_preset"` // "go" | "rust" | etc.
License string `toml:"license"` // "own" | "MIT" | etc.
}
Fixture is one manifest entry. See docs/adr/0019-real-corpus-fixture-registry.md section D.4 for the manifest format.
Source Files