Documentation
¶
Overview ¶
Package bench is the funcd benchmark & sustainability harness (ADR-0040): it embeds the platform, drives the public data plane, and samples process memory to answer whether the platform is sustainable on the target box (RAM-bound, scale-to-zero). It compares the memory and file substrate (blob + JetStream bus); the store is memory in both.
Load generation + latency percentiles use fortio, RSS sampling uses gopsutil — both test-only libraries confined to this package (ADR-0051).
Index ¶
- func WriteFootprintReport(dir string, r FootprintReport) (mdPath, jsonPath string, err error)
- func WritePoolReport(dir string, reports []Report) (mdPath, jsonPath string, err error)
- func WritePyPoolReport(dir string, c PyPoolComparison) (mdPath, jsonPath string, err error)
- func WriteReport(dir string, reports []Report) (mdPath, jsonPath string, err error)
- type Backend
- type Config
- type ContainerdConfig
- type FootprintReport
- type Latency
- type PoolComparison
- type PyPoolComparison
- type Report
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func WriteFootprintReport ¶
func WriteFootprintReport(dir string, r FootprintReport) (mdPath, jsonPath string, err error)
WriteFootprintReport writes the containerd footprint lane (ADR-0052) to dir as footprint-report.{md,json}, returning their paths. Unlike report.md (process RSS, ADR-0040), this measures the container cgroup — the honest absolute footprint over the production path.
func WritePoolReport ¶
WritePoolReport writes the worker-pool comparison (ADR-0044) to dir as pool-report.{md,json}, returning their paths. Returns ("", "", nil) — no error, no files — when no report carries pooled numbers (pooled measurement skipped or failed). Separate from the substrate report so the density-vs-latency trade reads on its own.
func WritePyPoolReport ¶
func WritePyPoolReport(dir string, c PyPoolComparison) (mdPath, jsonPath string, err error)
WritePyPoolReport writes the Python pool comparison (ADR-0050) to dir as py-pool-report.{md,json}.
Types ¶
type Backend ¶
type Backend string
Backend selects the durable substrate compared by the bench. The store is memory in both (the file store = badger (ADR-0065), out of scope, ADR-0040); only the blob + JetStream bus move.
type Config ¶
type Config struct {
Backends []Backend // which substrates to run (default: memory, file)
Concurrency int // load workers
Duration time.Duration // load phase per backend
Density int // extra warm functions deployed for the density sweep
MemBudgetMB int // function-memory budget for the verdict (e.g. 16384)
TargetFns int // agents to size for (e.g. 100)
}
Config parameterizes a bench run.
type ContainerdConfig ¶
type ContainerdConfig struct {
Socket string // containerd socket to dial (ADR-0054 Manager-provided)
Namespace string // isolated containerd namespace for the lane's throwaway containers (ADR-0055 dedicated-box); "" ⇒ "funcd-bench"
Snapshotter string // overlayfs
CNIBinDir string // /opt/cni/bin
CNIConfDir string // funcd-written conflist dir
SubnetCIDR string // lateral bridge subnet
ImagePrefix string // "funcd/runtime-"; image = prefix + runtime + ":latest"
// ImageOverride maps a runtime to the registry ref the lane runs instead (ctrmanager.Config.ImageFor).
ImageOverride map[string]string
// ShimPath is the in-container shim cmdline token (the curated image's entrypoint —
// "/opt/funcd/shim.mjs", ADR-0032). The container processes are host-visible PIDs, so
// shimRSSMB(ShimPath) sums the SAME containers' RSS — a same-population cgroup-vs-RSS ratio.
ShimPath string
Density int // extra warm functions for the marginal-cgroup sweep
MemBudgetMB int // function-memory budget for the verdict
TargetFns int // agents to size for
Concurrency int // load workers
Duration time.Duration // load phase
}
ContainerdConfig parameterizes the containerd footprint lane (ADR-0052). It mirrors the composition root's containerd wiring (cmd/funcd executionOptions) plus the bench's own measurement inputs. The lane boots funcd over the real containerd/crun production path and measures each function container's cgroup-v2 memory.current — the true footprint.
type FootprintReport ¶
type FootprintReport struct {
Skipped bool `json:"skipped"`
SkipReason string `json:"skipReason,omitempty"`
PerFunctionCgroupMB float64 `json:"perFunctionCgroupMB"` // marginal cgroup memory.current / fn (the true footprint)
PerFunctionRSSMB float64 `json:"perFunctionRSSMB"` // marginal RSS of the SAME container processes (the baseline)
CgroupOverRSS float64 `json:"cgroupOverRSS"` // container cgroup ÷ RSS — how much process RSS under-counts
PlatformBaselineMB float64 `json:"platformBaselineMB"` // funcd control-plane RSS
MaxDensity int `json:"maxDensity"` // functions within MemBudgetMB, from the cgroup number (honest)
FitsTarget bool `json:"fitsTarget"` // TargetFns within MemBudgetMB? — NOT optimistic (cgroup-measured)
RPS float64 `json:"rps"`
Latency Latency `json:"latency"`
}
FootprintReport is the containerd lane's result (ADR-0052). Skipped is true (with a reason and a nil error from RunContainerd) on any host that lacks the production path — the lane never fails the run. When not skipped, every memory number is the container cgroup memory.current, the honest production footprint.
func RunContainerd ¶
func RunContainerd(ctx context.Context, cfg ContainerdConfig) (FootprintReport, error)
RunContainerd boots funcd over the real containerd/crun production path (ADR-0032 driver), deploys functions that run in real crun containers, and measures each container's cgroup-v2 memory.current — the honest production footprint (ADR-0052). It runs only on a Linux host (root) with a reachable containerd; on any missing precondition it returns FootprintReport{Skipped:true,…} with a nil error, never failing the run.
type Latency ¶
type Latency struct {
P50 time.Duration `json:"p50"`
P90 time.Duration `json:"p90"`
P99 time.Duration `json:"p99"`
P999 time.Duration `json:"p999"`
Max time.Duration `json:"max"`
}
Latency is the tail-latency distribution of a load phase.
type PoolComparison ¶
type PoolComparison struct {
K int `json:"k"`
Substrate Backend `json:"substrate"`
// density (stable, primary)
PerFunctionMB float64 `json:"perFunctionMB"` // marginal RSS of one more separate shim
PooledMBPerFn float64 `json:"pooledMBPerFunction"` // pool RSS / K
DensityGain float64 `json:"densityGain"` // perFunctionMB / pooledMBPerFn
// throughput (indicative) — both measured DIRECTLY on the shim port (no data plane)
SingleDirectRPS float64 `json:"singleDirectRPS"` // one single-tenant shim, direct — HIGH VARIANCE
SingleDirectP99 time.Duration `json:"singleDirectP99"` //
PooledRPS float64 `json:"pooledRPS"` // one handler in the pool, direct (stable)
PooledP99 time.Duration `json:"pooledP99"` //
ThroughputRetained float64 `json:"throughputRetained"` // pooledRPS / singleDirectRPS — indicative only
}
PoolComparison is the head-to-head the separate pool report renders (ADR-0044). Two axes, NOT of equal reliability:
- density (the primary, stable result) — K handlers in ONE worker_threads pool (pool RSS / K) vs K separate single-tenant shim processes (the marginal RSS of one more).
- throughput/latency (indicative only) — the same handler hit DIRECTLY on each shim's own port (no gateway/activator/data plane). The pooled figure is stable run-to-run; the single-tenant baseline is ONE short-lived single-process shim and swings widely (≈15–65k req/s on a dev box) — single-threaded throughput is far more sensitive to JIT/core/GC timing than the pool's long-lived worker threads. So the pair shows throughput is COMPARABLE within that noise, not a precise ratio, and it is NOT a clean "worker_threads hop" isolation (the pool also gains a 2-core HTTP/handler pipeline). Read density as the decisive number.
type PyPoolComparison ¶
type PyPoolComparison struct {
K int `json:"k"`
Python string `json:"python"`
SoloMBPerFn float64 `json:"soloMBPerFunction"` // one solo shim's RSS = a function's marginal cost
PooledMBPerFn float64 `json:"pooledMBPerFunction"` // pool host RSS / K
DensityGain float64 `json:"densityGain"` // soloMBPerFn / pooledMBPerFn
SoloRPS float64 `json:"soloRPS"`
SoloP99 time.Duration `json:"soloP99"`
PooledRPS float64 `json:"pooledRPS"`
PooledP99 time.Duration `json:"pooledP99"`
}
PyPoolComparison is the Python solo-vs-pooled head-to-head (ADR-0050) — the Python analog of the Node WritePoolReport comparison. K trivial handlers in one subinterpreter pool host (pool.py) vs one solo shim (shim.py), both hit DIRECTLY on their own port (no data plane), so the difference isolates the pool host. Process RSS, dev machine (ADR-0040 caveats apply).
func MeasurePythonPool ¶
func MeasurePythonPool(ctx context.Context, python, soloEntry, poolHostEntry string, k, concurrency int, duration time.Duration) (PyPoolComparison, error)
MeasurePythonPool runs the Python solo-vs-pooled comparison (ADR-0050). It launches the subinterpreter pool host (poolEntry) with k identical handlers, samples its RSS (→ MB/function, the density), and drives load at one handler; then — for a fair baseline — it launches ONE solo shim (shimEntry) hosting the same handler and drives load on the SAME direct path. The two run sequentially so neither contends for CPU. The python interpreter must be ≥3.14 (the pool host).
type Report ¶
type Report struct {
Backend Backend `json:"backend"`
RPS float64 `json:"rps"`
Latency Latency `json:"latency"`
PlatformBaselineMB float64 `json:"platformBaselineMB"`
PerWorkerMB float64 `json:"perWorkerMB"`
IdleMB float64 `json:"idleMB"` // shim RSS while the only fn is scaled-to-zero (≈0 ⇒ reclaim works)
ColdStart time.Duration `json:"coldStart"` // wake → first response (timed only after 0 replicas)
PerFunctionMB float64 `json:"perFunctionMB"` // marginal slope: (RSS_at_K − perWorker) / K
MaxDensity int `json:"maxDensity"` // functions within MemBudgetMB
FitsTarget bool `json:"fitsTarget"` // TargetFns within MemBudgetMB? (optimistic — process RSS < cgroup)
PooledPerFunctionMB float64 `json:"pooledPerFunctionMB"` // same K handlers in ONE worker_threads pool / K (ADR-0044); 0 if not measured
PooledRPS float64 `json:"pooledRPS"` // throughput at one handler IN the pool, hit DIRECTLY (the worker_threads path); 0 if not measured
PooledLatency Latency `json:"pooledLatency"` // tail latency at one pooled handler, direct — the structured-clone cost
SingleDirectRPS float64 `json:"singleDirectRPS"` // throughput of ONE single-tenant shim on the SAME direct path — the fair baseline for the pool
SingleDirectLatency Latency `json:"singleDirectLatency"` // tail latency of the single-tenant shim, same direct path
PooledK int `json:"pooledK"` // pool size the pooled numbers were measured at
}
Report is one substrate's sustainability numbers.