Documentation
¶
Overview ¶
Package goldensuite is the repeatable regression gate: one definition of "our numbers", run against the shipped binary, comparable across runs.
It combines the three measurements that run everywhere and stay stable:
- gen corpus (hand-authored labels, the target metrics)
- self-retrieval sweep (label-free, any repo, the repo authors' words)
- verified negatives (false confidence — invisible to the other two)
giteval is deliberately NOT included: its label quality depends on how deep the clone's history is, so it is a manual tool, not a gate.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Config ¶
type Config struct {
GenManifest string `json:"gen_manifest"`
Repos []RepoEntry `json:"repos"`
// SelfN is how many self-retrieval cases to sample per repo.
SelfN int `json:"self_n"`
// SkipIntent turns the intent ranker off for the whole run (A/B knob).
SkipIntent bool `json:"skip_intent,omitempty"`
}
Config declares what the suite covers. Paths are relative to the config file, so the checked-in definition works on any machine that has the repos.
type RepoResult ¶
type RepoResult struct {
Self *SelfResult `json:"self,omitempty"`
Neg *NegResult `json:"neg,omitempty"`
}
type Report ¶
type Report struct {
Generated time.Time `json:"generated"`
Gen *GenResult `json:"gen,omitempty"`
Repos map[string]RepoResult `json:"repos"`
}
Report is one full run, JSON-serializable so runs can be diffed.
type SelfResult ¶
type SelfResult struct {
N int `json:"n"`
Hit1 float64 `json:"hit1"`
Hit5 float64 `json:"hit5"`
Hit10 float64 `json:"hit10"`
Miss float64 `json:"miss"`
// Hit5ByName accepts any symbol with the same short name as the target.
// DECISION(2026-07): interface/impl pairs make exact-symbol scoring
// unfair — the docstring lives on the trait declaration while the code
// lives in the implementing class, and returning the impl IS the answer
// an agent needs. os-lib read 0.23 "never retrieved" purely because
// BasePathImpl.baseName was scored as a miss for BasePath.baseName.
// Strict stays primary (it is the honest lower bound); this is the
// upper bound, and a wide gap means decl/impl duplication, not failure.
Hit5ByName float64 `json:"hit5_by_name"`
SeedGap float64 `json:"seed_loss_rate"`
AvgMs int64 `json:"avg_ms"`
}
Click to show internal directories.
Click to hide internal directories.