bench

package
v0.1.0-alpha.9 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 7, 2026 License: Apache-2.0 Imports: 15 Imported by: 0

Documentation

Overview

Package bench drives a reproducible agent workload against the mesh.

The chaos agent that ships with the repo is the realism story: a real model deciding what to call next. That makes it useless as an instrument, because two runs never issue the same requests and a difference between them cannot be attributed to the thing under test. This package is the opposite trade: it does exactly what it is told, so a difference between two runs is a difference in the mesh.

It reaches the mesh the way an agent does, through the sandbox boundary's Unix socket, opening each flow as a named HTTP CONNECT tunnel, so what it measures is what an agent would experience rather than what an operator with host access would.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Distribution

type Distribution struct {
	Count int     `json:"count"`
	Min   float64 `json:"min"`
	Mean  float64 `json:"mean"`
	P50   float64 `json:"p50"`
	P90   float64 `json:"p90"`
	P95   float64 `json:"p95"`
	P99   float64 `json:"p99"`
	Max   float64 `json:"max"`
	// StdDev is reported so a reader can see whether a mean is describing
	// anything, rather than averaging a bimodal distribution into fiction.
	StdDev float64 `json:"stddev"`
}

Distribution summarises a set of latencies in milliseconds. Percentiles are computed from the full sorted sample rather than from bucketed counts, so they are exact for the run rather than an interpolation.

type Options

type Options struct {
	// Socket is the sandbox boundary's Unix socket, which speaks HTTP CONNECT.
	// Empty measures the same workload without the boundary, which is the
	// baseline the mesh numbers are only meaningful against.
	Socket string

	// UnixTarget dials the target over this Unix socket instead of resolving
	// its host. It is how the baseline reaches the node's own API without a
	// relay in between: putting one there, socat or otherwise, would add a
	// userspace hop to the baseline and quietly flatter everything measured
	// against it.
	UnixTarget string

	// Target is the URL to request, e.g. http://mesh.sam.alt/v1/models.
	Target string

	// Method and Body are the request to issue. An empty Method means GET.
	Method string
	Body   []byte

	// Header carries anything the target needs, such as a content type.
	Header http.Header

	// Requests is how many to issue in total, after warmup.
	Requests int

	// Concurrency is how many are in flight at once.
	Concurrency int

	// Warmup requests are issued and discarded before measurement starts, so
	// first-call costs (DHT lookup, provider discovery, connection setup) do
	// not contaminate the steady-state distribution. They are reported
	// separately rather than thrown away.
	Warmup int

	// NewFlowPerRequest opens a fresh boundary flow for every request instead
	// of reusing connections. This separates the cost of admitting a flow from
	// the cost of carrying one, which are different questions.
	NewFlowPerRequest bool

	// Timeout bounds a single request.
	Timeout time.Duration
}

Options describe one measurement run. The zero value is not useful: at minimum a Target and a positive Requests count are required.

type Report

type Report struct {
	Target            string        `json:"target"`
	ThroughBoundary   bool          `json:"through_boundary"`
	NewFlowPerRequest bool          `json:"new_flow_per_request"`
	Concurrency       int           `json:"concurrency"`
	Requests          int           `json:"requests"`
	Warmup            int           `json:"warmup"`
	Succeeded         int           `json:"succeeded"`
	Failed            int           `json:"failed"`
	Elapsed           float64       `json:"elapsed_seconds"`
	Throughput        float64       `json:"requests_per_second"`
	TTFB              Distribution  `json:"ttfb_ms"`
	Total             Distribution  `json:"total_ms"`
	WarmupTTFB        *Distribution `json:"warmup_ttfb_ms,omitempty"`

	// Errors counts failures by their message, so a run that mostly failed
	// cannot be mistaken for a fast one.
	Errors map[string]int `json:"errors,omitempty"`
}

Report is the outcome of a run, in a shape that serialises to JSON for later analysis without needing the raw samples.

func Run

func Run(ctx context.Context, opts Options) (*Report, error)

Run issues the workload and returns its report.

type Sample

type Sample struct {
	// TTFB is the time until the response headers were available, which is
	// what an agent waiting on a first token actually experiences.
	TTFB time.Duration

	// Total additionally includes reading the body to completion.
	Total time.Duration

	// Status is the HTTP status, or 0 if the request never got one.
	Status int

	// Err is set when the request did not complete.
	Err error
}

Sample is one completed request.

type Series

type Series struct {
	Name   string
	Labels map[string]string
	Value  float64
}

Series is one exported sample: a metric name, its labels, and its value.

func (Series) Key

func (s Series) Key() string

Key renders a series as a stable, sorted identifier.

type Snapshot

type Snapshot struct {
	Source string    `json:"source"`
	Taken  time.Time `json:"taken"`
	Series []Series  `json:"-"`
}

Snapshot is everything one process exported at one instant.

func Scrape

func Scrape(ctx context.Context, url string) (*Snapshot, error)

Scrape reads one process's metrics endpoint.

func (*Snapshot) Flatten

func (s *Snapshot) Flatten() map[string]float64

Flatten renders the snapshot as canonical "name{k=v,...}" keys, which is what gets recorded in a result file. Keeping every series rather than a chosen few means a question nobody thought to ask before the run can still be answered from the record afterwards.

func (*Snapshot) Sum

func (s *Snapshot) Sum(name string) float64

Sum adds every sample of name, ignoring labels. It is how a counter split across label sets, such as flows by route, becomes one total.

func (*Snapshot) Value

func (s *Snapshot) Value(name string, match map[string]string) (float64, bool)

Value returns the first sample of name whose labels include match, and whether one existed. A missing series and a zero one mean different things: a counter that was never incremented is absent from the exposition entirely.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL