Documentation
¶
Overview ¶
Package rubric scores a harness guiding artifact against adh's own five-dimension rubric (SPEC-ADDITIONS §18.2) — distinct from the external skillsaw 9-dimension rubric adh consumes at its gate. It adapts skillsaw's rubric *pattern*: a deterministic floor (DetScore) that assumes a perfect base for judge-only dimensions and docks only detectable defects, an explicit list of dimensions that still need a model, and Diagnose naming the weakest deterministic dimension.
Three of the five dimensions — failure-handling, actionable-specificity, and boundary-section — are the microsoft/SkillLens quality dimensions (arXiv:2605.23899), each with published tests and anti-examples. Their deterministic detectors are skillet/skilllens (FailureMechanisms, BlacklistSections), which skillsaw scores too, so the two tools cannot disagree about whether an artifact encodes a failure mode. adh's weights, its 0..1 Deterministic factor, and its five-dimension set stay here: they grade a harness artifact, not a SKILL.md, which is why these three are worth 60 here and 35 in skillsaw. actionable-specificity is NeedsJudge, so its base is a model's to supply, against SkillLens's stated definition.
Consuming skilllens means this package now parses Markdown (via skillet/markdown); it still runs no runtime-neutrality scan, which stays skillsaw-specific.
Index ¶
Constants ¶
const ( KeyOutcome = "outcome-clarity" KeyFailure = "failure-handling" KeySpecificity = "actionable-specificity" KeyBoundary = "boundary-section" KeyArchitecture = "architecture" )
Dimension keys.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type DimScore ¶
type DimScore struct {
Dimension
Deterministic float64 `json:"deterministic"`
Reason string `json:"reason"`
}
DimScore is a dimension's outcome: a 0..1 deterministic factor (1.0 = no detectable defect) and the reason.
type Dimension ¶
type Dimension struct {
Key string `json:"key"`
Name string `json:"name"`
Weight int `json:"weight"`
NeedsJudge bool `json:"needs_judge"`
}
Dimension is one rubric axis. NeedsJudge marks a dimension whose base quality is an irreducible textual judgment a model must supply.
type Report ¶
type Report struct {
Dims []DimScore `json:"dims"`
DetScore float64 `json:"det_score"`
NeedsJudge []string `json:"needs_judge"`
}
Report is the rubric outcome: per-dimension scores, the weighted deterministic total (judge dims assumed perfect), and which dimensions still need a model.
func Evaluate ¶
Evaluate scores doc. Judge dimensions assume a perfect base for the floor; deterministic dimensions dock detectable defects. DetScore is the weighted total in [0, 100].
Requires: doc is a whole artifact, with or without a leading YAML header. Ensures: only the markdown body is scored; a "---" header is discarded, not
measured. It is pure.
The split happens here rather than at the call sites so no caller can pass an unsplit document: a description reading "...specifically when validation error..." otherwise satisfies failure-handling, and a hedged one moves specificity, on the strength of metadata that instructs nobody. None of the five dimensions is about the header, so the block is dropped rather than returned.
Source Files