corpora

Overview ¶

Command corpora is the single resolution path both the Taskfile fetch targets and (in a later plan) CI read for the corpora manifest, plus the measurement instrument Plan 01-05 builds: given -mode, it prints the resolved out-of-tree corpus root (root), a pre-validated JSON array of fetchable entries (entries), the sorted query.RankEdges members (kinds), the deterministic threshold/locked-set proposal derived from an observations file (select), or drives this repository's own indexer + query engine in-process to produce a measured Observation per corpus and upsert it into corpora/observations.json (measure).

Neither root nor entries ever emits a value that has not already passed internal/corpora's strict allowlist — the Taskfile's bash re-validates independently at the interpolation point anyway, but this program's output is never itself the unvalidated source of a manifest field.

measure mode has NO write path to corpora/selection.json: that file is hand-authored curated policy (thresholds, threshold rationale, the locked set, the rejected-candidate ledger), and this package's own tooling must never be able to overwrite it — see internal/corpora/record.go's package doc.

Package main — tools/corpora/prose.go declares renderMeasurementProse, the pure function that turns BOTH committed JSON documents (corpora/observations.json and corpora/selection.json) into the human-readable docs/CORPUS-MEASUREMENT.md. It DERIVES every count it renders from the observations — never reads a stored supplier field, never transcribes a number by hand — so the prose cannot drift from the evidence it cites.