Documentation
¶
Overview ¶
Package calibration implements Phase 2 conformal threshold calibration for the local-offload harness. It reads the JSONL ledger produced by package ledger, derives per-task margin thresholds using Conformal Risk Control, and writes the resulting thresholds to a JSON file for the pipeline to consume.
Algorithm (Conformal Risk Control, margin formulation):
For each task, collect labeled calls: (margin, correct). correct = (Grounded != nil && *Grounded) || (EscalatedAgreed != nil && *EscalatedAgreed). Skip: no usable label, Margin==0, CacheHit. For a candidate threshold t (a margin cutoff), ACCEPT a call when margin >= t. empirical error among accepted = count(accepted && !correct) / count(accepted). Adjusted rate = (n_accepted * err(t) + 1) / (n_accepted + 1). Choose the LARGEST t such that the adjusted rate <= alpha. If no t qualifies, fall back to the most conservative (largest) observed margin. Tasks with < 100 labeled rows are omitted (pipeline uses its constant instead).
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Run ¶
func Run(ledgerPath string, defaultAlpha float64, alphas map[string]float64, outPath string) (thresholds map[string]float64, report string, err error)
Run reads the ledger at ledgerPath, computes per-task conformal margin thresholds, writes a pretty-printed JSON map[task]float64 to outPath, and returns the map plus a human-readable summary report.
alphas overrides the per-task target error rate; missing tasks fall back to defaultAlpha. Tasks with fewer than 100 usable labeled rows are omitted from the returned map (pipeline falls back to its hardcoded constant).
Types ¶
This section is empty.
Source Files