Documentation
¶
Overview ¶
Package benchagg aggregates the per-dimension accuracy results of the owned scanner benchmark dimensions (secrets, IaC/misconfiguration, DAST, CSPM, runtime host-CVE, SAST per-CWE) into one machine-readable report WITHOUT erasing per-dimension semantics. This is the #1040 aggregation contract: each dimension defines its own truth (what a true/false positive and negative mean for that dimension), so the aggregate keeps every dimension as its own row carrying its own metric semantics and never computes a single cross-dimension confusion matrix, which would conflate, say, a secrets true-positive with a CSPM one. The overall verdict is the CONJUNCTION of each dimension meeting its committed floors, not a merged matrix.
Index ¶
Constants ¶
const ContractSchemaVersion = "synapse-dimension-contracts-v1"
ContractSchemaVersion tags the serialized truth-contract manifest.
const ReportSchemaVersion = "synapse-bench-aggregate-v1"
ReportSchemaVersion tags the serialized aggregate so a checked-in artifact is never compared across an incompatible schema.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Contract ¶
type Contract struct {
// Positive states what a positive case is for this dimension (its own truth semantics, never another's).
Positive string `json:"positive"`
// Unit is the counted entity (finding, resource, observation).
Unit string `json:"unit"`
// UnknownSemantics states how the dimension treats an input it cannot assess. Every dimension must document
// this, because the EPIC's no-false-suppression guardrail forbids converting an unknown into a clean result.
// A dimension whose corpus is closed states that explicitly (there is no unknown state), rather than leaving
// it blank.
UnknownSemantics string `json:"unknown_semantics"`
// Provenance records how the labeled corpus is produced, so a reader can reproduce the numbers.
Provenance string `json:"provenance"`
// RecallFloor / PrecisionFloor are the committed ratchets the dimension's gate enforces (documented here so
// the truth contract and the gate are read together). A floor of 0 means that metric is not gated.
RecallFloor float64 `json:"recall_floor"`
PrecisionFloor float64 `json:"precision_floor"`
}
Contract is one dimension's versioned truth contract.
type ContractManifest ¶
type ContractManifest struct {
Schema string `json:"schema"`
Dimensions map[string]Contract `json:"dimensions"`
}
ContractManifest is the committed set of per-dimension truth contracts, keyed by dimension id.
func (ContractManifest) Validate ¶
func (m ContractManifest) Validate() error
Validate reports whether the manifest is internally well-formed: the current schema, at least one dimension, and every contract complete (a positive, a unit, documented Unknown semantics, provenance, and floors within [0,1]). It does not measure anything; it guards the committed artifact against an incomplete or malformed contract, so a dimension can never ship a truth contract that omits its Unknown/NotAssessed posture.
type DimensionResult ¶
type DimensionResult struct {
Semantics MetricSemantics
TP int
FP int
FN int
TN int
RecallFloor float64
PrecisionFloor float64
}
DimensionResult is one dimension's confusion matrix plus its committed floors. It carries its own semantics, so a caller reduces each owned dimension gate into this shape and the aggregate never mixes them.
func (DimensionResult) MeetsFloors ¶
func (d DimensionResult) MeetsFloors() bool
MeetsFloors reports whether this dimension's recall and precision are at or above its committed floors. A floor of 0 is vacuously met (the dimension does not gate that metric).
func (DimensionResult) Precision ¶
func (d DimensionResult) Precision() float64
Precision is TP / (TP + FP); 0 when nothing was flagged (no precision is defined, reported as 0).
func (DimensionResult) Recall ¶
func (d DimensionResult) Recall() float64
Recall is TP / (TP + FN); 0 when there are no positive cases (no recall is defined, reported as 0).
type DimensionRow ¶
type DimensionRow struct {
Dimension string `json:"dimension"`
Positive string `json:"positive"`
Unit string `json:"unit"`
TP int `json:"tp"`
FP int `json:"fp"`
FN int `json:"fn"`
TN int `json:"tn"`
Recall float64 `json:"recall"`
Precision float64 `json:"precision"`
RecallFloor float64 `json:"recall_floor"`
PrecisionFloor float64 `json:"precision_floor"`
FloorsMet bool `json:"floors_met"`
}
DimensionRow is one dimension's serialized row in the aggregate: its semantics, its matrix, its derived metrics, its floors, and whether it met them. Nothing here is merged across dimensions.
type MetricSemantics ¶
type MetricSemantics struct {
Dimension string `json:"dimension"`
Positive string `json:"positive"`
Unit string `json:"unit"`
}
MetricSemantics records what a dimension's confusion matrix MEANS, so the aggregate never applies one dimension's definition of a positive to another. Dimension is a stable id (e.g. "secrets", "cspm"); Positive states what a positive case is; Unit is the counted entity (finding, resource, observation).
type Report ¶
type Report struct {
Schema string `json:"schema"`
Dimensions []DimensionRow `json:"dimensions"`
AllFloorsMet bool `json:"all_floors_met"`
}
Report is the machine-readable aggregate: per-dimension rows (semantics preserved) and the overall all-floors-met conjunction. It carries no cross-dimension confusion matrix by design.
func Aggregate ¶
func Aggregate(results []DimensionResult) (Report, error)
Aggregate reduces per-dimension results into one report, preserving each dimension's semantics. It refuses a duplicate or empty dimension id (a machine-readable aggregate must address each dimension unambiguously) and sorts the rows by dimension id for a deterministic artifact. AllFloorsMet is the conjunction over dimensions.
func (Report) FailingDimensions ¶
FailingDimensions returns the ids of the dimensions that did not meet their floors, sorted, so a CI gate can name exactly which dimension regressed without collapsing the per-dimension detail.
func (Report) MarshalJSON ¶
MarshalJSON renders the aggregate as the machine-readable artifact.
Source Files