Documentation
¶
Overview ¶
coverage.go is the FIXT-01 coverage guard: it DERIVES the coverage claim from three committed documents rather than validating a precomputed summary. See 01-07-PLAN.md's objective — "The guard derives the claim; it does not validate a summary".
An earlier draft of this guard read a stored `coverage[kind]` map from a record and compared each entry against a threshold. Nothing in that design required the claimed supplier to be locked, to have actually been measured, or to match its own raw measurement — a summary naming a repository that was never locked, or citing a count higher than that repository's observation, would have passed. That is rule `84d1gfpywd` in its subtler form: not a guard that checks nothing, but a guard that checks the wrong artifact.
CheckCoverage therefore takes all three already-decoded documents and reconstructs the claim from first principles. Loading — and the failure modes of loading — belong to the CALLER, not to this function: CheckCoverage has no I/O. (An earlier draft's must_haves attributed loading to it, which its own declared signature contradicted.)
Package corpora is the sole pin authority for the FIXT-01/FIXT-02 third-party measurement corpora: a strictly validated manifest naming every candidate repository's pinned commit, license and lock state, plus the collision-free out-of-tree destination path each entry fetches into.
This is deliberately a SEPARATE package from tools/bench/realcorpus, this repository's other pinned-corpus manifest, rather than an extension of it. Two of the four reasons recorded in 01-04-PLAN.md's objective are load-bearing here: realcorpus performs no network I/O and only reports a path a caller must already have fetched (the opposite of what a Taskfile-driven fetch target needs), and realcorpus deliberately carries a BSD-3-Clause entry while Validate below enforces a strictly narrower MIT/Apache-2.0 bar — sharing one type would force a policy-parameterised validator or silently widen that bar. A reader who lands here from realcorpus's package doc, or vice versa, should find this paragraph.
Every value this package reads from a manifest file is untrusted input: it is interpolated into a `git remote add` / `git fetch` shell invocation by the Taskfile fetch targets, so Validate applies a strict allowlist to every field that reaches a shell, never a blocklist.
record.go declares the two independent typed documents Plan 01-05 builds on top of manifest.go's Entry/Manifest: Observations (GENERATED, upserted by task corpora:measure, keyed by repo@sha, never hand-edited) and Selection (CURATED, hand-authored policy — thresholds, threshold rationale, the locked set, the rejected-candidate ledger and any synthetic-coverage declarations — which this package's own tooling never writes). See 01-05-PLAN.md's "Why the record is TWO files, not one": a merge of the two into one generator-owned file is code that can be wrong, whereas "the generator has no write path to this file" cannot be.
Index ¶
- Variables
- func ComputeThresholds(obs Observations, eligible []string) map[string]int64
- func CorpusRoot() (string, error)
- func ObservationKey(repo, sha string) string
- func SelectLockedSet(obs Observations, th map[string]int64, eligible []string) ([]string, error)
- func StripVolatile(m map[string]any) map[string]any
- func Validate(m Manifest) error
- type CoverageResult
- type Entry
- type KindCoverage
- type LanguageGroup
- type Manifest
- type Observation
- type Observations
- type RejectedCandidate
- type Selection
Constants ¶
This section is empty.
Variables ¶
var ErrDuplicateObservationKey = errors.New("corpora: duplicate observation key")
ErrDuplicateObservationKey is returned by NewObservations when two input entries share an ObservationKey.
var ErrInvalidRepo = errors.New("corpora: invalid repo")
ErrInvalidRepo is returned by Validate when an entry's Repo is not a strict single-slash org/name form drawn from [A-Za-z0-9._-]. Tests assert on this sentinel's identity rather than on message text.
var ErrInvalidSHA = errors.New("corpora: invalid sha")
ErrInvalidSHA is returned by Validate when an entry's SHA is not exactly 40 lowercase hex characters. Tests assert on this sentinel's identity (errors.Is) rather than on message text, so the check can be reworded without breaking callers.
var ErrNoQualifyingSubset = errors.New("corpora: no qualifying subset")
ErrNoQualifyingSubset is returned by SelectLockedSet, wrapped with the unsatisfiable kinds and languages, when no subset of eligible clears every threshold and gives every PriorityLanguages member a non-zero summed file count.
var PriorityLanguages = []LanguageGroup{ {Name: "go", Keys: []string{"go"}}, {Name: "java", Keys: []string{"java"}}, {Name: "csharp", Keys: []string{"csharp"}}, {Name: "python", Keys: []string{"python"}}, {Name: "tsjs", Keys: []string{"typescript", "javascript", "tsx"}}, }
PriorityLanguages is the five priority-4 language groups FIXT-01 requires the locked corpus set to cover, declared once so ComputeThresholds/SelectLockedSet and the prose renderer (tools/corpora /prose.go) never hand-restate the group boundaries.
Functions ¶
func ComputeThresholds ¶
func ComputeThresholds(obs Observations, eligible []string) map[string]int64
ComputeThresholds derives a per-kind minimum edge-count threshold from obs, sourced from ALL of eligible — the full measured universe, never the locked subset. Sourcing best from the locked set would make thresholds and selection each depend on the other, and multiple fixed points could exist; sourcing it from the full measured universe breaks the cycle. For each query.RankEdges kind, best is the highest Observation.EdgeCount for that kind across eligible; the returned threshold is min(max(2, best/2), best) using integer division. The outer min is what makes every threshold SATISFIABLE BY CONSTRUCTION: without it, a kind whose best is 1 would derive max(2, 0) = 2, a bar nothing measured can clear. With the clamp, best == 1 yields 1 and best == 0 yields 0, so the record can say plainly that a kind is arithmetically uncoverable rather than inventing a bar nothing can pass.
func CorpusRoot ¶
CorpusRoot resolves the base directory fetched corpora land under: the caller's explicit override verbatim when set and non-empty; otherwise the XDG cache-home variable joined with "codegraph/corpora" when that is set and non-empty; otherwise the caller's home directory joined with ".cache/codegraph/corpora". This formula is applied literally and identically on every operating system: it does NOT use the standard library's own cache-directory resolver, because that resolver returns a platform-native location on Darwin instead of the formula above, and no per-operating-system branch belongs in this function — the same no-branch discipline internal/agents/opencode.go's config-directory resolution already applies to XDG_CONFIG_HOME.
func ObservationKey ¶
ObservationKey joins repo and sha the same way Observations keys its map: repo, an at-sign, sha — matching the manifest's own repo@SHA identity form (D-09).
func SelectLockedSet ¶
SelectLockedSet enumerates subsets of eligible in increasing cardinality and returns the FIRST that satisfies every kind in th (each kind's threshold is compared against the MAX Observation.EdgeCount among the subset's observations — the single named supplier the coverage claim attributes each kind to) and gives every PriorityLanguages member a non-zero summed file count across the subset. Within a cardinality, subsets are ordered by total TrackedFiles ascending, then by their sorted repository-name list lexicographically, so the result is fully determined. Brute-force enumeration is correct and cheap here: D-17 bounds the candidate count at ten, so at most 1024 subsets are ever considered. Returns ErrNoQualifyingSubset, naming the unsatisfiable kinds and languages (evaluated across the FULL eligible set), when no subset qualifies.
func StripVolatile ¶
StripVolatile returns a NEW map with every volatile key (isVolatileKey) removed, recursing into nested objects so a volatile key nested under "index" is removed too. m itself is never mutated. pendingChanges deliberately SURVIVES the strip: it is a deterministic all-zero placeholder today (query.PendingChanges), so removing it would shrink the record's schema for no reproducibility benefit — every measurement keeps the identical shape whether or not sync tracking is ever wired up.
func Validate ¶
Validate rejects any manifest entry whose fields do not pass the strict allowlists above. This is the control that keeps a manifest-derived value safe to interpolate into the Taskfile fetch targets' shell commands: Validate runs before any value reaches a shell, and the Taskfile targets re-validate independently at the actual interpolation point as a second, last-line-of-defence check.
The MIT/Apache-2.0 licence bar enforced here is deliberately NARROWER than tools/bench/realcorpus's manifest, which carries a BSD-3-Clause entry (cockroachdb-pebble) for the PERF-01 benchmark corpus — that difference in policy is exactly why the two manifests are not merged into one type; see the package doc for the other three reasons.
Types ¶
type CoverageResult ¶
type CoverageResult struct {
// CheckedKinds is the number of kinds swept: always len(query.RankEdges),
// DERIVED from the ranked-kind set so there is no hand-maintained
// expected-count constant to go stale beside it.
CheckedKinds int
// CheckedCorpora is the number of corpora swept: always
// len(LockedEntries(m)), DERIVED from the manifest (the sole pin
// authority, D-09). A run over an empty locked set reports a failure,
// never a green pass.
CheckedCorpora int
// Kinds is a per-kind view keyed by every query.RankEdges kind.
Kinds map[string]KindCoverage
// Failures is a deterministic (sorted) list of every violation found.
Failures []string
}
CoverageResult is CheckCoverage's whole output.
func CheckCoverage ¶
func CheckCoverage(m Manifest, obs Observations, sel Selection) CoverageResult
CheckCoverage derives the coverage claim from m, obs and sel together and reports every failure. It performs NO I/O — the caller loads the three documents (corpora.Load, corpora.LoadObservations, corpora.LoadSelection) and fails with the path named when any is missing or malformed; this function works only on the decoded values.
Derivation order:
- derive the locked repo@sha identities from the manifest's locked entries (LockedEntries);
- require EXACT set equality with sel.LockedSet — in both directions, so a manifest that locks a repository the selection omits fails just as a selection that names an unlocked repository does;
- require every locked identity to have an observation in obs;
- compute, per query.RankEdges kind, the best count across those locked observations and which repository supplied it — ONLY locked identities contribute, so an unlocked (or rejected) candidate's observation can never supply a kind;
- apply sel's thresholds to the DERIVED counts (at-least comparison: a kind measuring exactly its threshold PASSES — this is the boundary convention Plan 01-06 recorded in the selection);
- require sel.MinEdgesPerKind to cover every query.RankEdges kind and to carry no kind outside it;
- require every PriorityLanguages group to have a non-zero summed file count across the locked observations;
- require syntheticKinds to be empty.
The positive-count discipline is preserved WITHOUT a hand-maintained constant. The wire oracle's ExpectedScenarioCount (test/wireoracle/ scenarios.go) has no derivable source, so a constant is the only way to assert its count positively; the locked-corpus count DOES derive — from the manifest, which D-09 makes the sole authority — so this guard derives CheckedKinds from query.RankEdges and CheckedCorpora from len(LockedEntries(m)) instead of restating a constant beside the manifest. A hand-maintained constant would be a second authority requiring an edit on every manifest change and capable of disagreeing with it. This is the deliberate deviation from the ExpectedScenarioCount pattern.
type Entry ¶
type Entry struct {
// Repo is the GitHub "org/name" slug — the sole allowed form is a
// single slash separating two [A-Za-z0-9._-] runs (repoPattern).
Repo string `json:"repo"`
// SHA is the full 40-character lowercase hex commit this entry is
// pinned to. Never a branch or tag name — see D-11's shallow-fetch
// discipline in 01-04-PLAN.md.
SHA string `json:"sha"`
// License is this entry's SPDX identifier at SHA, resolved live from
// the GitHub API licence endpoint — never a README badge, never
// recalled from memory. Must be MIT or Apache-2.0 (validLicenses).
License string `json:"license"`
// Language records the primary language this candidate is nominated
// to close a coverage gap for.
Language string `json:"language"`
// Locked is true once this entry has been promoted into the final
// measured-and-justified corpus set. False entries remain in the
// manifest — including candidates rejected before measurement,
// per D-09 — recorded rather than deleted.
Locked bool `json:"locked"`
// Note carries free-form provenance: for a locked entry, the
// coverage gap it closes; for an unlocked one, the reason it was
// not selected.
Note string `json:"note,omitempty"`
}
Entry is one candidate corpus: a repository this phase measures or has measured, whether or not it ends up locked into the final set.
func LockedEntries ¶
LockedEntries returns only the entries of m whose Locked flag is true, in manifest order.
func (Entry) Dir ¶
Dir returns e's destination directory under root: e's readable Slug, a separator, the first 8 hex characters of a SHA-256 digest over the canonical Repo string, an at-sign, and the pinned SHA. The digest makes the path collision-free by construction — "a-b/c" and "a/b-c" hash to different digests despite sharing a Slug — where a hand-reasoned "these characters cannot collide" argument over the slug alone would not be, and was the shape of bug this construction exists to avoid. Embedding SHA also means a pin bump changes the path outright, so a stale tree at a superseded pin can never be mistaken for the current one.
type KindCoverage ¶
type KindCoverage struct {
// Threshold is sel.MinEdgesPerKind[kind], the bar the selection sets.
Threshold int64
// Count is the DERIVED best measured edge count for kind across the
// locked observations. If no locked observation measured kind, Count
// is zero and Repo is empty.
Count int64
// Repo is the manifest entry whose observation supplied Count — the
// derived best supplier, never a stored field.
Repo string
}
KindCoverage is CheckCoverage's per-kind derived view: the threshold the selection imposes, the best measured count across the locked observations, and the repository that supplied that count. There is NO stored supplier field behind this — Repo is computed from the observations here, which is exactly why it cannot disagree with the evidence.
type LanguageGroup ¶
LanguageGroup names one FIXT-01 priority-4 coverage target and the concrete query.StatusResult.FilesByLanguage keys that count toward it. Four groups map to exactly one indexer language ID; TS/JS is the one exception, spanning THREE real per-file language IDs (internal/indexer/languages_typescript.go registers "typescript" and "javascript" as separate LanguageSpecs, and TSX source lands under "tsx" — see testdata/golden/behavioral_tsjs_test.go's identical tsjsLanguages grouping) because REQUIREMENTS.md counts them as one coverage target ("TS/JS").
type Manifest ¶
type Manifest struct {
// Note is a top-level provenance record: that this file is the sole
// pin authority for this phase's corpora, the date its SHAs and
// licences were resolved live, and a pointer to the OTHER pinned
// manifest in this repository (tools/bench/realcorpus) so a reader
// of one is led to the other.
Note string `json:"note,omitempty"`
// Corpora is every candidate entry, locked or not, in manifest
// order.
Corpora []Entry `json:"corpora"`
}
Manifest is the full corpora/manifest.json document: the sole pin authority every reader (the Taskfile fetch loop, the CI cache path, the coverage drift guard) consults, per D-09.
func Load ¶
Load reads path, decodes it as JSON into a Manifest, and Validates the result before returning it. A manifest with a duplicate repo, a malformed field, or an unknown license returns a non-nil error and a zero Manifest — never a partially populated one — so a caller can never observe a manifest that decoded but did not pass validation.
type Observation ¶
type Observation struct {
// Repo is the GitHub org/name slug, matching a corpora/manifest.json
// entry's Repo (manifest.go).
Repo string `json:"repo"`
// SHA is the pinned commit this observation measured, matching the
// manifest entry's SHA.
SHA string `json:"sha"`
// License is copied from the manifest entry at measurement time, so
// a reader of the observation alone can see the candidate's SPDX
// identifier without cross-referencing the manifest.
License string `json:"license"`
// Language is the manifest entry's nominated Language — the
// coverage gap this candidate was fetched to help close.
Language string `json:"language"`
// TrackedFiles is `git ls-files | wc -l` at the pinned SHA: the
// tracked-file count of the fetched tree. NEVER the GitHub API
// repository "size" field, which reports full-history packed size
// and does not describe what a shallow `--depth 1` fetch at one
// commit yields (01-RESEARCH.md Pitfall 6).
TrackedFiles int64 `json:"trackedFiles"`
// Status is `codegraph status --json --all-kinds`'s decoded output
// for this corpus, after query.DenseEdgesByKind and then
// StripVolatile: dense edgesByKind (every ranked kind with an
// explicit value, plus any unranked kind carrying a positive count),
// filesByLanguage, languages, the three top-level counts
// (fileCount/nodeCount/edgeCount), backend, version and index
// survive; every key StripVolatile names is gone.
Status map[string]any `json:"status"`
}
Observation is one candidate's measured evidence: everything task corpora:measure recorded about repo@sha from a single indexing run. It is GENERATED — never hand-edited — and lives only inside Observations.
func (Observation) EdgeCount ¶
func (o Observation) EdgeCount(kind string) int64
EdgeCount returns o's measured count for kind (a query.RankEdges member or an unranked kind), read from its stripped status.edgesByKind map. Absent kinds read as 0, matching "not measured" — callers that need to distinguish "measured zero" from "absent" should inspect o.Status directly.
func (Observation) LanguageFileCount ¶
func (o Observation) LanguageFileCount(g LanguageGroup) int64
LanguageFileCount sums o's measured file counts across g's constituent filesByLanguage keys.
type Observations ¶
type Observations struct {
SchemaVersion int `json:"schemaVersion"`
ManifestPath string `json:"manifestPath"`
Observations map[string]Observation `json:"observations"`
}
Observations is the full corpora/observations.json document: every candidate ever measured, keyed by ObservationKey. Generated and UPSERTED by task corpora:measure — entries within a run's scope are replaced, entries outside it are left untouched, and nothing is deleted without an explicit -prune. Never hand-edited, never fully reconstructed.
func LoadObservations ¶
func LoadObservations(path string) (Observations, error)
LoadObservations reads path, decodes it as JSON into an Observations, and returns a non-nil error naming path when the file is absent or malformed — never an empty document a downstream guard would then vacuously pass over.
func NewObservations ¶
func NewObservations(schemaVersion int, manifestPath string, obs []Observation) (Observations, error)
NewObservations builds an Observations from a slice, computing each entry's key via ObservationKey and returning ErrDuplicateObservationKey (naming the key) when two entries collide. A Go map literal cannot represent that collision on its own, so this constructor is what surfaces it for any caller assembling one Observation at a time — the shape both task corpora:measure's upsert loop and this package's own tests use.
type RejectedCandidate ¶
RejectedCandidate is one entry in Selection's rejected-candidate ledger (D-17): a measured candidate that did NOT make the locked set, recorded with why — never simply deleted from the record.
type Selection ¶
type Selection struct {
SchemaVersion int `json:"schemaVersion"`
MinEdgesPerKind map[string]int64 `json:"minEdgesPerKind"`
ThresholdRationale string `json:"thresholdRationale"`
LockedSet []string `json:"lockedSet"`
Rejected []RejectedCandidate `json:"rejected"`
SyntheticKinds []string `json:"syntheticKinds"`
}
Selection is the full corpora/selection.json document: hand-authored CURATED policy — per-kind thresholds, the threshold rationale, the locked set, the rejected-candidate ledger and any synthetic-coverage declarations. This package's own tooling never writes this file; a human authors it (Plan 01-06).
func LoadSelection ¶
LoadSelection reads path, decodes it as JSON into a Selection, and validates it before returning: a non-nil error naming path when the file is absent or malformed, and a validation error when ThresholdRationale is empty or whitespace-only (D-15 makes the recorded rationale mandatory — an unexplained threshold is invalid input, not a lint warning) or when any Rejected entry carries an empty Reason.
Source Files