benchmark

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 26, 2026 License: Apache-2.0 Imports: 11 Imported by: 0

Documentation

Overview

Package benchmark reduces fixture-supplied benchmark observations into a deterministic, versioned report. It performs no workload execution or external I/O.

Index

Constants

View Source 
const (
	// AccuracyInputSchemaVersion / AccuracyReportSchemaVersion version the accuracy
	// document shapes independently of the throughput benchmark schemas above.
	AccuracyInputSchemaVersion  = "synapse-accuracy-input-v1"
	AccuracyReportSchemaVersion = "synapse-accuracy-report-v1"
)
View Source 
const (
	InputSchemaVersion  = "synapse-benchmark-input-v1"
	OutputSchemaVersion = "synapse-benchmark-report-v1"
)
View Source 
const MaxJSONBytes int64 = 8 << 20

MaxJSONBytes bounds one untrusted benchmark JSON document before decoding or publication.

Variables

This section is empty.

Functions

func CanonicalJSON 

func CanonicalJSON(value any) ([]byte, error)

CanonicalJSON encodes a value with encoding/json's deterministic struct and map ordering. Callers must canonicalize order-insensitive slices before calling it.

func EncodeAccuracyReport 

func EncodeAccuracyReport(w io.Writer, report AccuracyReport) error

EncodeAccuracyReport writes one stable, indented JSON document followed by a newline.

func EncodeReport 

func EncodeReport(w io.Writer, report Report) error

EncodeReport writes one stable, indented JSON document followed by a newline.

func SHA256Digest 

func SHA256Digest(data []byte) string

SHA256Digest returns a lower-case immutable SHA-256 digest with its algorithm prefix.

func StrictDecode 

func StrictDecode(reader io.Reader, destination any) error

StrictDecode validates one bounded JSON document, recursively rejects unknown object fields, and decodes it.

func ValidateJSONDocument 

func ValidateJSONDocument(reader io.Reader) error

ValidateJSONDocument applies bounded, UTF-8, depth, duplicate-key, and single-value checks.

func WriteCanonicalJSON 

func WriteCanonicalJSON(writer io.Writer, encoded []byte) error

WriteCanonicalJSON writes one canonical JSON value followed by a newline within the document limit.

Types

type APIFailoverObservation 

type APIFailoverObservation struct {
	DetectionMilliseconds int64 `json:"detection_milliseconds"`
	RecoveryMilliseconds  int64 `json:"recovery_milliseconds"`
	Recovered             bool  `json:"recovered"`
}

type APIFailoverSummary 

type APIFailoverSummary struct {
	Count            int64     `json:"count"`
	RecoveredCount   int64     `json:"recovered_count"`
	UnrecoveredCount int64     `json:"unrecovered_count"`
	Detection        Quantiles `json:"detection_milliseconds"`
	Recovery         Quantiles `json:"recovery_milliseconds"`
}

type AccuracyInput 

type AccuracyInput struct {
	SchemaVersion string                `json:"schema_version"`
	Observations  []AccuracyObservation `json:"observations"`
}

AccuracyInput is a set of observations to reduce.

func DecodeAccuracyInput 

func DecodeAccuracyInput(r io.Reader) (AccuracyInput, error)

DecodeAccuracyInput reads exactly one accuracy input document, rejecting unknown fields.

type AccuracyObservation 

type AccuracyObservation struct {
	Case     string   `json:"case"`
	Group    string   `json:"group,omitempty"`
	Expected []string `json:"expected"`
	Produced []string `json:"produced"`
}

AccuracyObservation is one evaluation case: the ground-truth detection keys and the keys the engine produced for the same input. Keys are opaque, already-normalized strings (the caller decides the vocabulary, e.g. "component|CVE-id"); matching here is exact set membership. Group buckets the case (e.g. by ecosystem) for a per-group ratchet; an empty Group counts only toward the overall totals.

type AccuracyReport 

type AccuracyReport struct {
	SchemaVersion string          `json:"schema_version"`
	Cases         int64           `json:"cases"`
	Overall       Metrics         `json:"overall"`
	Groups        []GroupAccuracy `json:"groups"`
}

AccuracyReport is the deterministic reduction of an AccuracyInput.

func EvaluateAccuracy 

func EvaluateAccuracy(input AccuracyInput) (AccuracyReport, error)

EvaluateAccuracy validates and reduces the observations. It makes no measurements.

type CorrectnessObservation 

type CorrectnessObservation struct {
	Name   string `json:"name"`
	Passed bool   `json:"passed"`
}

type CorrectnessSummary 

type CorrectnessSummary struct {
	Checks   int64    `json:"checks"`
	Failures int64    `json:"failures"`
	Failed   []string `json:"failed"`
}

type EvidenceGrowth 

type EvidenceGrowth struct {
	DatabaseBeforeBytes int64 `json:"database_before_bytes"`
	DatabaseAfterBytes  int64 `json:"database_after_bytes"`
	ObjectBeforeBytes   int64 `json:"object_before_bytes"`
	ObjectAfterBytes    int64 `json:"object_after_bytes"`
}

type EvidenceGrowthSummary 

type EvidenceGrowthSummary struct {
	DatabaseBeforeBytes int64 `json:"database_before_bytes"`
	DatabaseAfterBytes  int64 `json:"database_after_bytes"`
	DatabaseGrowthBytes int64 `json:"database_growth_bytes"`
	ObjectBeforeBytes   int64 `json:"object_before_bytes"`
	ObjectAfterBytes    int64 `json:"object_after_bytes"`
	ObjectGrowthBytes   int64 `json:"object_growth_bytes"`
}

type GroupAccuracy 

type GroupAccuracy struct {
	Group   string  `json:"group"`
	Cases   int64   `json:"cases"`
	Metrics Metrics `json:"metrics"`
}

GroupAccuracy is the reduction for one group, identified by its label.

type Input 

type Input struct {
	SchemaVersion string                   `json:"schema_version"`
	Metadata      Metadata                 `json:"metadata"`
	Window        Window                   `json:"window"`
	Requests      []RequestObservation     `json:"requests"`
	Queue         QueueObservation         `json:"queue"`
	Pool          PoolObservation          `json:"pool"`
	Evidence      EvidenceGrowth           `json:"evidence"`
	Migration     MigrationObservation     `json:"migration"`
	APIFailovers  []APIFailoverObservation `json:"api_failovers"`
	Correctness   []CorrectnessObservation `json:"correctness"`
}

Input is a fixture or runner-produced set of measured benchmark observations.

func DecodeInput 

func DecodeInput(r io.Reader) (Input, error)

DecodeInput accepts exactly one JSON input document and rejects unknown fields.

type Metadata 

type Metadata struct {
	Environment       string `json:"environment"`
	EnvironmentDigest string `json:"environment_digest"`
	Release           string `json:"release"`
	ReleaseDigest     string `json:"release_digest"`
	DataDigest        string `json:"data_digest"`
}

Metadata identifies exactly what was measured without deriving release or data state.

type Metrics 

type Metrics struct {
	TruePositives      int64   `json:"true_positives"`
	FalsePositives     int64   `json:"false_positives"`
	FalseNegatives     int64   `json:"false_negatives"`
	Precision          float64 `json:"precision"`
	Recall             float64 `json:"recall"`
	F1                 float64 `json:"f1"`
	FalseDiscoveryRate float64 `json:"false_discovery_rate"`
	FalseNegativeRate  float64 `json:"false_negative_rate"`
}

Metrics is the reduced confusion-matrix summary. Rates use these conventions so the no-data cases never produce a NaN that would make a ratchet undefined:

  • Precision = TP/(TP+FP); 1.0 when nothing was produced (no false alarms).
  • Recall = TP/(TP+FN); 1.0 when nothing was expected (nothing to miss).
  • F1 = 2PR/(P+R); 0 when P+R == 0.
  • FalseDiscoveryRate = FP/(TP+FP) = 1 - Precision; 0 when nothing was produced. This is deliberately the false-discovery rate, not the textbook false-positive rate FP/(FP+TN): detection has no bounded universe of true negatives to count, so FPR is not computable.
  • FalseNegativeRate = FN/(TP+FN) = 1 - Recall (the miss rate); 0 when nothing was expected.

type MigrationObservation 

type MigrationObservation struct {
	DurationMilliseconds int64 `json:"duration_milliseconds"`
}

type PoolObservation 

type PoolObservation struct {
	AcquisitionMilliseconds []int64 `json:"acquisition_milliseconds"`
	SaturationEvents        int64   `json:"saturation_events"`
}

type PoolSummary 

type PoolSummary struct {
	Acquisition      Quantiles `json:"acquisition_milliseconds"`
	SaturationEvents int64     `json:"saturation_events"`
}

type Quantiles 

type Quantiles struct {
	Count int64 `json:"count"`
	P50   int64 `json:"p50"`
	P95   int64 `json:"p95"`
	P99   int64 `json:"p99"`
}

Quantiles use the nearest-rank method: rank=ceil(p*n), with the first value at rank 1.

type QueueObservation 

type QueueObservation struct {
	DelayMilliseconds    []int64 `json:"delay_milliseconds"`
	RecoveryMilliseconds []int64 `json:"recovery_milliseconds"`
}

type QueueSummary 

type QueueSummary struct {
	Delay    Quantiles `json:"delay_milliseconds"`
	Recovery Quantiles `json:"recovery_milliseconds"`
}

type Report 

type Report struct {
	SchemaVersion string                `json:"schema_version"`
	Metadata      Metadata              `json:"metadata"`
	Throughput    Throughput            `json:"throughput"`
	Requests      RequestSummary        `json:"requests"`
	Queue         QueueSummary          `json:"queue"`
	Pool          PoolSummary           `json:"pool"`
	Evidence      EvidenceGrowthSummary `json:"evidence"`
	Migration     MigrationObservation  `json:"migration"`
	APIFailover   APIFailoverSummary    `json:"api_failover"`
	Correctness   CorrectnessSummary    `json:"correctness"`
}

Report is a deterministic reduction of Input. It never asserts an SLO or target.

func Evaluate 

func Evaluate(input Input) (Report, error)

Evaluate validates and reduces the supplied measurements without making any measurements itself.

type RequestObservation 

type RequestObservation struct {
	DurationMilliseconds int64 `json:"duration_milliseconds"`
	Succeeded            bool  `json:"succeeded"`
}

type RequestSummary 

type RequestSummary struct {
	Count     int64     `json:"count"`
	Successes int64     `json:"successes"`
	Failures  int64     `json:"failures"`
	Duration  Quantiles `json:"duration_milliseconds"`
}

type Throughput 

type Throughput struct {
	WindowMilliseconds int64   `json:"window_milliseconds"`
	RequestsPerSecond  float64 `json:"requests_per_second"`
}

type Window 

type Window struct {
	DurationMilliseconds int64 `json:"duration_milliseconds"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.