Documentation
¶
Overview ¶
Package critic assembles the cold critic's grounding (SPEC-ADDITIONS §19.1): the repository-owned working set a critic reviews against — the arc's touched paths, its acceptance bar, the proof packet the builder left, and the context units routed for its labels and paths (§10). It withholds exactly one input, the builder's transcript; that isolation lives in the prompt renderer. Ground is a pure assembly; Load and ForStage are the thin I/O shells around it.
Index ¶
- Constants
- func AppendCoverage(path string, entry *CoverageEntry) error
- func AppendPrecision(path string, entry *PrecisionEntry) error
- func FindingKinds(findings []adh.Finding) []string
- func NoisyKinds(entries []PrecisionEntry, minSamples int, maxFPR float64) []adh.FindingKind
- func ParseFindings(reply string) ([]adh.Finding, []finding.Unexamined, error)
- func UnderCovered(entries []CoverageEntry, all []adh.FindingKind) []adh.FindingKind
- func VerdictKinds(v *Verdict) (confirmed, unconfirmed []string)
- type Adjudicated
- type CoverageEntry
- type Grounding
- type Inputs
- type PrecisionEntry
- type Reply
- type Verdict
- func (v *Verdict) BlockingKind() adh.FindingKind
- func (v *Verdict) Classes() []string
- func (v *Verdict) FailureNotes() []string
- func (v *Verdict) HasStructural() bool
- func (v *Verdict) LessonNotes() []string
- func (v *Verdict) RefusedNotes() []string
- func (v *Verdict) ReservationNotes() []string
- func (v *Verdict) ReturnsToExecution() bool
- func (v *Verdict) UncheckedNotes() []string
Constants ¶
const ( DefaultMinAdjudications = 5 DefaultMaxFalsePositiveRate = 0.5 )
Default thresholds for NoisyKinds: a kind is judged noisy only after at least DefaultMinAdjudications findings of that kind have been adjudicated (so a small sample cannot condemn it), and only when more than DefaultMaxFalsePositiveRate of them were unconfirmed.
const ConvergenceConstraint = "critic runs once per arc (§19.3); no revision loop"
ConvergenceConstraint records SPEC-ADDITIONS §19.3: the critic runs once per arc, so it cannot widen a change across review rounds. A bounded critic/author revision loop is a deliberate future addition, not a runtime gate today.
const CoverageFile = ".adh/critic-coverage.jsonl"
CoverageFile is the conventional repo path the critic coverage log lives under (§19/§10.3): an append-only record of which finding kinds each arc's critic surfaced, so the next critic can be steered toward the kinds it keeps missing — accretion applied to the critic lever, kept distinct from the evidence trail.
const MaxContextUnits = contextstore.DefaultWorkingSet
MaxContextUnits caps how many routed context units enter a critic's working set, matching the context command's working-set size.
const PrecisionFile = ".adh/critic-precision.jsonl"
PrecisionFile is the conventional repo path the critic precision log lives under (§19): an append-only record of each adjudication's confirmed and unconfirmed finding kinds. It is the precision mirror of the coverage log — where coverage steers the critic toward kinds it under-covers, precision surfaces the kinds it over-flags (a high false-positive rate), so the next critic is held to a higher bar on them.
Variables ¶
This section is empty.
Functions ¶
func AppendCoverage ¶
func AppendCoverage(path string, entry *CoverageEntry) error
AppendCoverage records one arc's surfaced kinds to the append-only log at path, creating the parent directory. Recording nothing (no kinds) is a no-op, so an arc with no findings does not pollute the history.
func AppendPrecision ¶
func AppendPrecision(path string, entry *PrecisionEntry) error
AppendPrecision records one adjudication's confirmed/unconfirmed kinds to the append-only log at path. An entry with nothing adjudicated (a clean review) is a no-op, so it does not pollute the history.
func FindingKinds ¶
FindingKinds returns the sorted distinct kinds a critic turn's findings named, for recording an arc's coverage.
func NoisyKinds ¶
func NoisyKinds(entries []PrecisionEntry, minSamples int, maxFPR float64) []adh.FindingKind
NoisyKinds returns the finding kinds whose adjudicated false-positive rate is too high to trust (§19): a kind adjudicated at least minSamples times whose unconfirmed share exceeds maxFPR. The next critic holds these to a higher bar. A kind below the sample floor is never condemned. Sorted, deterministic. Pure.
func ParseFindings ¶
ParseFindings decodes a critic turn's reply (§19.2): the findings it raises and the aspects it declares it did not examine.
Requires: nothing; any string is a valid input and an invalid one is an error. Ensures: EINVALID for a reply that does not parse, a finding with no summary or an unknown kind or class, or an unexamined entry missing either field. Pure.
**An empty findings list used to be "a clean review" and could equally be a critic that looked at nothing** — `evaluation` disposes of the arc on that silence, so the two reached Ops identically. `unexamined` is how a critic says which it was.
It is **testimony, not a derived fact**: a critic claiming it did not examine something is a statement about its own behaviour, unverifiable from outside, and worth what the critic is worth. That is why an ungrounded review — which the harness knows mechanically — is reported through relay.Outcome instead of being written here on the critic's behalf. skillet keeps the two as separate types deliberately, and collapsing them would let a mechanical skip read as a critic's own admission.
An invalid entry rejects the whole reply rather than being dropped. Dropping it is how a reply that says nothing passes for a reply that found nothing, which is the failure this field exists to end.
func UnderCovered ¶
func UnderCovered(entries []CoverageEntry, all []adh.FindingKind) []adh.FindingKind
UnderCovered returns the finding kinds the recorded critic history has covered least (§19): the kinds tied for the minimum surfaced-arc count across all entries, so an empty history returns every kind and a mature one returns the current blind spots. The next critic is steered to probe these. It is pure.
func VerdictKinds ¶
VerdictKinds splits a disposed verdict into the confirmed and unconfirmed finding kinds, one entry per finding (with multiplicity). Pure.
Verdict.Unchecked contributes to neither, per PrecisionEntry's doc.
Types ¶
type Adjudicated ¶
type Adjudicated struct {
Finding adh.Finding
Ran bool
Failed bool
// Unrunnable says why the artifact did not run, and is meaningful only when Ran
// is false.
//
// §19.2 records that an unchecked finding does not block, on the reasoning that
// Ran:false covers both *the registered tool is broken* — an honest refusal worth
// acting on — and *the critic named a tool that never existed*, which is noise,
// and that blocking on the second would let one bad critic wedge every arc.
//
// **The adjudicator already knows which**, and threw the answer away one call up:
// runDeclaredTool's `found` distinguishes them. Carrying it does not change the
// blocking rule — that decision stands — it makes the decision revisitable on
// evidence rather than on the guess that the two are inseparable.
Unrunnable adh.Unrunnable
// Measured is the value a meter produced, and HasMeasure whether one did.
//
// A bool rather than a sentinel: 0 is a legitimate measurement, and NaN would be a
// second vocabulary for "absent" beside the one this struct already has.
Measured float64
HasMeasure bool
// Regressed reports a measurement worse than the spec's baseline that still
// cleared the acceptance bar. It is the reservation §19.2 had no disposition for:
// the change is admissible and it cost something, and both facts are true.
Regressed bool
}
Adjudicated is a finding paired with the outcome of running its named artifact (§19.2): Ran is whether the artifact could be run at all, Failed whether it failed when it ran.
type CoverageEntry ¶
CoverageEntry is one arc's critic coverage: the finding kinds it surfaced.
func LoadCoverage ¶
func LoadCoverage(path string) ([]CoverageEntry, error)
LoadCoverage reads the coverage log at path. An absent file is no entries, not an error. A corrupt line is a hard error — the log's integrity is its value.
type Grounding ¶
type Grounding struct {
Paths []string // the arc's touched repository paths
Diff string // unified diff of the change under review (§19.1)
AcceptanceBar string // what the resolution's proof must show (§5.4)
Proof []proof.Artifact // the proof packet the builder left
Context []contextstore.Unit // units routed for the arc's labels/paths (§10)
Tools []toolreg.Tool // capabilities available to the stage (§13)
Coverage []string // finding kinds recent critics under-covered (§19)
Noisy []string // finding kinds recent critics over-flagged (§19)
// Denied is the inputs config says the critic must not receive (§19.4). It is
// not something the critic sees; it is what the renderer asserts about what it
// built. Carried here so the assertion reaches the renderer without a policy
// argument threaded through Emit and Request.
Denied []adh.CriticInput
// PriorGaps is what the previous critic declared it did not examine (§19.2), so
// this one can start where that one stopped. See Inputs.PriorGaps for why this
// does not contaminate a cold review.
PriorGaps []finding.Unexamined
}
Grounding is a stage's repository-owned working set (§10, §19.1): the context and capabilities it acts against. Strategy and Execution use the routed Context and available Tools; the critic additionally reviews the Diff against the Proof and AcceptanceBar. It never carries the builder's transcript.
func ForStage ¶
ForStage returns the working set a stage acts against and whether the arc is a critic routing gap. Every stage is grounded now (§10): Strategy and Execution load the routed context and available tools before acting, and the critic also reviews the diff against the proof and acceptance bar. A routing gap applies only to the critic (§19.1): an arc that declared a footprint (labels or touched paths) against a context store that *exists* — has units — yet routes nothing, with no proof, means the environment did not teach this review, so the critic would fall back on its own priors; callers surface it as exit 12 (§10). An empty or absent store, or an arc that declared no footprint, is simply ungrounded, not a gap.
func Ground ¶
Ground assembles the working set from repository state already read: it routes units by the arc's labels and touched paths, carries the proof packet's artifacts, and carries the shell-supplied inputs (bar + diff). It mutates nothing; a nil packet contributes no artifacts.
func Load ¶
Load assembles the critic's working set. in carries the shell-supplied bar and diff, passed through to Ground.
func (*Grounding) HasGrounding ¶
HasGrounding reports whether the environment taught the critic anything repository-specific: a proof packet to check or a routed context unit. The acceptance bar alone (generic proof-kind text) does not count.
type Inputs ¶
type Inputs struct {
AcceptanceBar string
Diff string
Tools []toolreg.Tool
Coverage []string
Noisy []string
// Denied is `[critic] deny` (§19.4), carried through to the Grounding so the
// renderer can check the view it builds against it. Policy rather than a
// grounding fact, and it travels here for the reason the type exists: the
// alternative is a sixth parameter on Request and a seventh on Emit, threaded
// through two packages to reach one assertion.
Denied []adh.CriticInput
// PriorGaps is what the previous critic declared it did not examine (§19.2).
//
// **This is the one input the cold split lets through, and the exception is
// deliberate.** §19.1 withholds exactly one thing, the builder's transcript, and
// that is what makes the critic cold. A previous critic's *coverage* is neither
// the builder's reasoning nor a conclusion about the change — it is a record of
// angles already taken, so feeding it forward steers a fresh critic toward
// unexamined ground. That is the opposite of the contamination the split prevents.
PriorGaps []finding.Unexamined
}
Inputs are the grounding facts the shell computes and hands to the pure core: the acceptance bar (the deployment's configured proof contract, §19.4) and the diff of the change under review (§19.1). Bundling them keeps the critic free of config and vcs — they arrive as data — and lets more inputs be added without churning every Ground/Load/ForStage signature.
type PrecisionEntry ¶
type PrecisionEntry struct {
Arc string `json:"arc"`
Confirmed []string `json:"confirmed,omitempty"`
Unconfirmed []string `json:"unconfirmed,omitempty"`
}
PrecisionEntry is one adjudication's outcome by finding kind: the kinds confirmed (the named artifact ran and failed — a real defect) and the kinds unconfirmed (surfaced but not reproduced — a false positive). Kinds are recorded with multiplicity, one per finding, so the false-positive rate counts findings.
**A finding whose artifact could not run appears in neither**, and that omission is the point rather than an oversight. It used to be recorded as unconfirmed — as a false positive — so a missing or unbuilt tool drove up its kind's rate and NoisyKinds condemned the kind, holding the next critic to a higher bar for it. A broken environment taught the harness to distrust real defects, and it compounded.
Leaving it out of *both* slices is deliberate: dropping it from the numerator alone would leave it in the denominator and understate the rate instead. A finding nobody checked is evidence about that kind in neither direction.
func LoadPrecision ¶
func LoadPrecision(path string) ([]PrecisionEntry, error)
LoadPrecision reads the precision log at path. An absent file is no entries, not an error; a corrupt line is a hard error.
type Reply ¶
type Reply struct {
Findings []adh.Finding
Resolution adh.Resolution
Text string
// Unexamined is what a critic declared it did not look at (§19.2). Advisory by
// construction: finding.Result.HasBlocking iterates diagnostics only, so a
// declared gap cannot change a verdict however Dispose is written.
Unexamined []finding.Unexamined
}
Reply is a relayed stage reply after validation (§19.2). Findings is set only for a critic reply; Resolution only when a strategy reply chose one; Text is what the stage records in its history. A Reply is produced only after the reply passed its stage's contract, so a malformed answer never advances an arc.
func ParseReply ¶
ParseReply validates a relayed reply for the arc's current stage and extracts what that stage needs (§19.2). An empty reply is always EINVALID. A critic reply must be findings JSON (ParseFindings). A strategy reply may begin with a "resolution: <word>" line that chooses the resolution (§12) — an unknown word is EINVALID, and no such line leaves it unset so the loop defaults it to a change. Every other stage carries prose. It is pure: it neither reads I/O nor mutates.
type Verdict ¶
type Verdict struct {
// Confirmed: the named artifact ran and failed. A real defect; blocks.
Confirmed []adh.Finding
// Unconfirmed: the artifact ran and passed. A lesson candidate (§11), and the
// only bucket that is evidence of a false positive.
Unconfirmed []adh.Finding
// Unchecked: the artifact could not run — missing, unbuilt, wrong commit.
//
// It does not block, and that is a decision rather than an oversight; see
// ReturnsToExecution. It is also not a lesson candidate: a lesson is drawn from a
// finding that was checked and did not reproduce, and this one was not checked.
Unchecked []adh.Finding
// Reservations are findings that passed adjudication with a measurement worse
// than baseline. They do not block — a regression inside the bar is information,
// and blocking on it would make Fail and Baseline the same threshold — but an arc
// that advances carrying one has cost something, and that used to be invisible.
Reservations []adh.Finding
// Refused is the subset of Unchecked whose artifact was a **registered** tool
// that could not start, as opposed to a finding naming nothing or naming a tool
// the repository never declared.
//
// The distinction is §19.2's stated blocker: the decision not to block on an
// unchecked finding rests on the two being inseparable, and they are not. Kept as
// a subset rather than a fourth bucket so the partition above stays a partition —
// every finding is still in exactly one of the three.
Refused []adh.Finding
}
Verdict is the disposition of a critic's findings after adjudication (§19.2).
Three buckets, because the input carries three states and collapsing two of them corrupted the harness's own calibration. Ran:false and Ran:true,Failed:false used to land together in Unconfirmed — "nobody checked this" and "this was checked and did not reproduce" reported as one answer.
That was not merely imprecise. Unconfirmed feeds PrecisionEntry, whose own doc calls it "surfaced but not reproduced — a false positive", and NoisyKinds then condemns any kind whose false-positive rate is too high: "the next critic holds these to a higher bar." So a missing or unbuilt artifact taught the harness to distrust an entire category of real defect, and the more broken the environment the more it distrusted.
A finding nobody could check is evidence about that kind in neither direction, which is the rule quotecheck.Unchecked and gate.VerdictUnchecked follow one repo over.
func Dispose ¶
func Dispose(results []Adjudicated) Verdict
Dispose classifies each adjudicated finding (§19.2).
Requires: nothing; an empty result set is a clean review. Ensures: the three buckets partition results — every finding lands in exactly one — and order within each is the order adjudicated. Refused is a subset of Unchecked and is not part of the partition. Pure; the caller runs the artifacts and records the effects.
The artifact not running is tested first because it is the only case where Failed carries no information: an adjudicator that could not run a check has nothing to report about whether it passed, and reading Failed there would be reading a zero value as an answer.
func (*Verdict) BlockingKind ¶
func (v *Verdict) BlockingKind() adh.FindingKind
BlockingKind is the kind of the first confirmed finding, which the Evaluation gate maps to an exit code. It is empty when nothing is confirmed.
func (*Verdict) Classes ¶
Classes returns the distinct finding kinds across the confirmed and unconfirmed findings, sorted — the classes a disposed arc contributes to the failure-record log, so a recurring one can be gated for promotion (§11, §19.2). Candidates count: an unconfirmed finding kept to detect recurrence is exactly what promotion gates on.
func (*Verdict) FailureNotes ¶
FailureNotes renders the confirmed findings as failure-registry entries (§4.1), each classed by kind so a recurring one groups under lesson.Distill.
func (*Verdict) HasStructural ¶
HasStructural reports whether any confirmed finding is structural (§19.2) — one that needs a design change, so the arc escalates to a human rather than spending rework cycles on an edit that cannot close it.
func (*Verdict) LessonNotes ¶
LessonNotes renders the unconfirmed findings as lesson candidates (§11): kept to detect a recurring class, never a blocker (§19.2).
Unchecked findings are not candidates. A lesson is drawn from a finding that was checked and did not reproduce; one nobody could check has taught nothing yet.
func (*Verdict) RefusedNotes ¶
RefusedNotes renders the findings a registered tool refused to check.
Separate from UncheckedNotes because the two ask different things of a reader: a finding naming nothing is the critic's problem, and a registered tool that will not start is the environment's.
func (*Verdict) ReservationNotes ¶
ReservationNotes renders the findings that passed while measuring worse than baseline, so an advancing arc can say what it cost.
func (*Verdict) ReturnsToExecution ¶
ReturnsToExecution reports whether the verdict blocks the arc: any confirmed finding is a deterministic Evaluation failure that returns the arc to Execution (§19.2).
**This one line is where "artifacts decide what blocks" is enforced**, so it is an invariant rather than an implementation detail: Confirmed requires an artifact that ran and failed, which is why a critic's assertion alone can never block an arc. Widening this to any other slice would let narration block, and that is a change to a stated rule (§19.2) rather than a change to a condition.
The rule is narrow on purpose and the other direction is the known gap: a critic that finds nothing leaves nothing to adjudicate, so silence advances the arc, bounded only by the NFR guards that are adjudicated whether or not the critic mentioned them.
**An unchecked finding does not block, and the decision is recorded rather than emergent.** `vac-gate`'s rule — "'cannot regrade' is not 'regraded'" — argues the honest refusal should fail the gate, and it is the right rule where the refusal is trustworthy. Here it is not yet: a finding's artifact comes from a model's reply, so Ran:false covers both "the tool is broken" and "the critic named a tool that never existed", and blocking on the second would let one bad critic wedge every arc.
The trigger for revisiting: the §13 tool registry can distinguish the two, so once adjudication resolves a finding's ref against it, a registered-but-unrunnable artifact becomes a refusal worth blocking on and an unregistered one stays noise. Until then the state is reported rather than acted on — visible, which is the part that was missing, since it used to be folded into Unconfirmed and counted as a false positive.
func (*Verdict) UncheckedNotes ¶
UncheckedNotes renders the findings whose artifact could not run, so a caller can report them. Same shape as the other two, because a reader comparing the three lists should not have to notice a formatting difference.