replay

package
v0.9.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 5, 2026 License: Apache-2.0 Imports: 31 Imported by: 0

Documentation

Overview

Package replay drives a trace of job arrivals through the real control plane, node wire, node runtime and simulated backend at compressed time, and reads what billet recorded about each job back out of the ledger.

It exists so a scheduling change can be asked what it does to a workload: under this week of arrivals, where did placement put each job, how long did jobs queue, and how did two policies differ. `make test` proves invariants on single leases and the end-to-end suite proves the wire and the runtime; neither runs a thousand jobs through the placer, because every other backend needs a guest per job.

THE ALLOCATOR, THE LISTENER, THE PLACER AND THE NODE ARE THE REAL ONES, or the harness would measure a model of billet rather than billet. What is scripted is GitHub, through internal/fakeactions, and what is steered is time: one Clock the allocator and the simulated provider read, moved only between events, so a replay is deterministic and a modelled hour costs no wall time.

IT IS A TEST-SIDE CONSUMER AND STAYS ONE. It needs a *testing.T, no production code imports it, and boundary_test.go proves that over the module's import graph, as the simulated backend and the fake Actions service prove theirs.

Index

Constants

View Source
const (
	DefaultOwner = "acme"
	RunnerGroup  = "billet-replay"
	Workflow     = "acme/replay/.github/workflows/ci.yml@refs/heads/main"

	// DefaultBoot is how long a minted runner takes to register with GitHub and
	// be handed a job, when a fleet does not say. It is the harness's model of
	// something billet does not decide, and the report names it as such.
	DefaultBoot = 30 * time.Second
)

The one organization, runner group and workflow allowlist every replayed tier is declared under. They are the scripted GitHub's answers to the runner-group policy check, which the real listener and the real node both make before a registration is minted; a trace's per-job repository and workflow ride on the messages and are not checked against them, exactly as GitHub does not.

View Source
const ResultSucceeded = "succeeded"

ResultSucceeded is the one completion result billet recognises as success, spelled as the scale-set wire spells it. A trace that names no result gets it.

Variables

View Source
var DefaultStart = time.Date(2026, time.January, 5, 9, 0, 0, 0, time.UTC)

DefaultStart is a Monday morning, so a trace reads like a working day.

Functions

This section is empty.

Types

type Arrival

type Arrival struct {
	// Seq numbers the arrivals from 1 in arrival order. It is the request id the
	// scripted GitHub offers the job under, so the ledger's request_id joins a
	// recorded row back to this line without any table of the harness's own.
	Seq int64 `json:"seq"`
	// At is when the job was queued.
	At time.Time `json:"at"`
	// Tier is the runs-on label, which is the scale set that will carry it.
	Tier string `json:"tier"`
	// Owner, Repository and WorkflowRef are what GitHub puts on the message.
	Owner       string `json:"owner"`
	Repository  string `json:"repository"`
	WorkflowRef string `json:"workflow"`
	RunID       int64  `json:"run_id"`
	// Duration is how long the job ran once a runner had it.
	Duration Duration `json:"duration"`
	// Result is GitHub's conclusion; empty means ResultSucceeded.
	Result string `json:"result"`
}

Arrival is one job arriving at GitHub's queue: what a trace is made of.

A ZERO IS FILLED IN OR REFUSED, NEVER REPLACED ON THE WIRE. The message the scripted service builds overlays these fields on a default, and an overlay cannot tell "zero" from "absent"; so Normalize writes the run id it will use and Validate refuses a job with no time, and what the file says is what is replayed.

type Clock

type Clock struct {
	// contains filtered or unexported fields
}

Clock is the one instant the allocator and the simulated provider read.

VIRTUAL, NOT A WALL OFFSET. The end-to-end suite's clock is time.Now plus an offset, which is right for a scenario that waits on real things and wants one duration to pass faster. A replay wants the opposite: two runs of one trace must date every lease identically, and an offset from a wall clock cannot. So this holds an instant that moves only when the harness moves it, and every event in one step is stamped with exactly the same time.

func NewClock

func NewClock(start time.Time) *Clock

NewClock starts a clock at an instant.

func (*Clock) Now

func (c *Clock) Now() time.Time

Now reports the instant. It is the func the allocator and provider hold.

func (*Clock) Step

func (c *Clock) Step(t time.Time)

Step moves the clock to t, or one nanosecond past the present if t is not later, so that every event the harness delivers is dated at an instant of its own.

THE LEDGER'S ORDER MUST BE ITS TIMESTAMPS' ORDER. Two events at one trace instant are delivered one after the other, and what the first did to the fleet is the state the second was placed against; dated identically, a reader sweeping the ledger cannot tell which came first, and a charge made before a same-instant release reads as made after it. A nanosecond keeps the order and moves nothing a report would notice.

type Duration

type Duration time.Duration

Duration is a time.Duration that reads and writes as Go spells one ("4m20s"), which is what the exporter script emits and what a person can read in a file.

func (Duration) MarshalJSON

func (d Duration) MarshalJSON() ([]byte, error)

MarshalJSON renders the duration as Go spells it.

func (*Duration) UnmarshalJSON

func (d *Duration) UnmarshalJSON(b []byte) error

UnmarshalJSON reads a Go duration string.

type Fleet

type Fleet struct {
	Hosts []Host
	Tiers []TierShape
	// MaxVCPU and MaxMemory are the deployment ceiling, server.max_vcpu and
	// server.max_memory.
	MaxVCPU   int
	MaxMemory config.ByteSize
	// Placement is server.placement; empty means pack, as it does in a config.
	Placement config.PlacementPolicy
	// Boot is the modelled time between a mint and the runner taking a job. Zero
	// means DefaultBoot.
	Boot time.Duration
}

Fleet is the deployment a trace is replayed against.

type Host

type Host struct {
	Name   string
	VCPU   int
	Memory config.ByteSize
	// Site is where the machine is, or empty for a fleet in one place.
	Site string
	// RatePerHour is what running this machine costs per hour, in dollars. The
	// harness's input, like the trace: an owned box's rate is not a fact billet
	// records. Zero means unpriced.
	RatePerHour float64
}

Host is one simulated machine in the fleet.

type Options

type Options struct {
	// Log receives billet's own diagnostics. Nil writes them to the test log.
	Log *slog.Logger
	// SettleWithin bounds, in wall time, how long billet may take to settle
	// after one event before the replay is declared stuck. Zero means a minute.
	SettleWithin time.Duration
}

Options vary a replay beyond its fleet and trace.

type Params

type Params struct {
	// Jobs is how many arrivals to generate.
	Jobs int
	// Tiers are the labels jobs are spread over, first the most common.
	Tiers []string
	// Repositories are the repositories jobs belong to, first the most common.
	Repositories []string
	// Start is the trace's first instant. A Monday at nine, by default.
	Start time.Time
}

Params shape a synthetic trace. Zero values take the defaults below.

type Percentiles

type Percentiles struct {
	Count         int
	P50, P95, Max time.Duration
}

Percentiles are the queue-wait figures the summary reports for one group.

type Record

type Record struct {
	Seq         int64  `json:"seq"`
	LeaseID     string `json:"lease_id"`
	Tier        string `json:"tier"`
	Owner       string `json:"owner"`
	Repository  string `json:"repository"`
	WorkflowRef string `json:"workflow"`
	Node        string `json:"node"`

	// Provider, InstanceType, VCPU, Memory and Site are what the lease was
	// charged for: the backend it ran on, the shape placement bought (empty for a
	// host-backed backend), the charged vCPU and memory, and the host's site.
	Provider     string          `json:"provider"`
	InstanceType string          `json:"instance_type,omitempty"`
	VCPU         int             `json:"vcpu"`
	Memory       config.ByteSize `json:"memory"`
	Site         string          `json:"site,omitempty"`
	// PriceMicrosPerHour is the rate the shape was charged at when the lease was
	// escrowed, in millionths of a dollar. Zero means no price was recorded,
	// which is what a host-backed lease has.
	PriceMicrosPerHour int64 `json:"price_micros_per_hour"`
	// Cost is the run's cost in dollars and CostSource says what rate produced
	// it: "ledger" when the recorded price was positive, "fleet-rate" when the
	// fleet's per-host rate was applied to the charged vCPU-hours, and "" when
	// neither was available, in which case Cost is zero and means unknown.
	Cost       float64 `json:"cost"`
	CostSource string  `json:"cost_source,omitempty"`
	// ImageCache, CacheGeneration and ActionsCache are what the node observed the
	// cache do, in the ledger's closed vocabularies; empty means nothing was
	// observed, which is what a simulated guest reports.
	ImageCache      string `json:"image_cache,omitempty"`
	CacheGeneration string `json:"cache_generation,omitempty"`
	ActionsCache    string `json:"actions_cache,omitempty"`

	Arrival time.Time `json:"arrival"`
	// ChargedFrom is when the lease was escrowed, which is when its shape began
	// to count against the host and the deployment: before the job was assigned,
	// while the lease was still the tier's discovery slot.
	ChargedFrom time.Time `json:"charged_from"`
	AssignedAt  time.Time `json:"assigned_at"`
	StartedAt   time.Time `json:"started_at"`
	FinishedAt  time.Time `json:"finished_at"`

	// QueueWait is from the trace's arrival to the ledger's recorded start, and
	// RunDuration from that start to the lease's archive.
	QueueWait   Duration `json:"queue_wait"`
	RunDuration Duration `json:"run_duration"`

	Conclusion string `json:"conclusion"`
	Result     string `json:"result"`
	Disruption string `json:"disruption,omitempty"`

	// Locality is DERIVED FROM PLACEMENT, not observed: an earlier job of the
	// same repository was placed on the same node. Beside the observed cache
	// columns because a simulated guest observes nothing, and locality is the
	// question a placement policy can be asked without one.
	Locality bool `json:"locality"`
}

Record is what the ledger recorded about one job, joined to the trace line that produced it.

EVERYTHING BUT Arrival, Repository, WorkflowRef AND Cost COMES FROM THE LEDGER. The arrival is the trace's fact and the join is on the request id billet recorded; the provider, the charged shape and its price, the site and what the cache did are the history row's own columns (migration 49), written from the lease rather than from any catalogue. Cost is the one derived figure and says where its rate came from.

func (*Record) FullName

func (rec *Record) FullName() string

FullName is the repository as GitHub names it, owner included, because two organizations can each own a repository of one name and they share nothing.

type Report

type Report struct {
	Fleet Fleet
	// Records are the jobs the ledger knows about, in sequence order.
	Records []Record
	// Missing are trace jobs with no ledger row: gaps in the recording, reported
	// as gaps rather than filled in.
	Missing []int64
	// Unstarted are recorded jobs the ledger never saw start.
	Unstarted []int64
	// Unfinished are recorded leases the ledger never archived: their charge has
	// no end, so the sweep charges it and never releases it, and a capacity
	// verdict over such a ledger is provisional rather than final.
	Unfinished []int64
	// EscrowRows counts history rows for leases that were never assigned a job:
	// the discovery slots released at shutdown. Reported so a reader can tell
	// them from a missing job.
	EscrowRows int

	// Violations are the overcommits the ledger's charges prove: a host or the
	// deployment charged more than it has, at some instant.
	Violations []string
	// contains filtered or unexported fields
}

Report is what one replay recorded.

func Run

func Run(t *testing.T, fleet Fleet, trace Trace, opts Options) *Report

Run replays a trace against a fleet and reports what the ledger recorded.

ONE EVENT AT A TIME, TO QUIESCENCE. Every arrival, start and completion is delivered alone, and the next is not delivered until every listener is parked on an empty queue with its escrow settled. Placement happens at escrow, so two events in flight at once would let goroutine order decide which host a job lands on, and a replay whose placements vary between runs cannot compare two policies. Determinism is the property; the seed only shapes a synthetic trace.

func (*Report) CacheOutcomes

func (r *Report) CacheOutcomes() map[string]int

CacheOutcomes counts the cache observations the ledger recorded, keyed as "image:<token>" and "actions:<token>". Nothing observed counts nothing.

func (*Report) CostByTier

func (r *Report) CostByTier() (map[string]float64, int)

CostByTier sums the priced records' cost per tier, and counts the records whose cost is unknown.

func (*Report) HostsUsed

func (r *Report) HostsUsed() int

HostsUsed is how many distinct hosts carried a job.

func (*Report) LocalityRate

func (r *Report) LocalityRate() float64

LocalityRate is the share of started jobs whose repository already had a job on the same node.

func (*Report) PeakDeploymentVCPU

func (r *Report) PeakDeploymentVCPU() int

PeakDeploymentVCPU is the most vCPU the whole fleet carried at one instant, escrow included: the number the deployment ceiling bounds. Not the sum of the hosts' peaks, which need not coincide.

func (*Report) PeakVCPUByNode

func (r *Report) PeakVCPUByNode() map[string]int

PeakVCPUByNode is the most vCPU each host carried at any instant, escrow included.

func (*Report) Placements

func (r *Report) Placements() map[int64]string

Placements is the node each recorded job ran on, by sequence.

func (*Report) QueueWaitBy

func (r *Report) QueueWaitBy(key func(*Record) string) map[string]Percentiles

QueueWaitBy groups the started records by a key and reports each group's queue-wait percentiles.

func (*Report) Summary

func (r *Report) Summary() string

Summary renders the report for a person: what was recorded, what was not, how long jobs queued, where they went, and what the records prove about capacity.

func (*Report) WriteJSONL

func (r *Report) WriteJSONL(w io.Writer) error

WriteJSONL writes one record per line.

type TierShape

type TierShape struct {
	Label  string
	VCPU   int
	Memory config.ByteSize
}

TierShape is one runs-on label and what a job on it is charged.

type Trace

type Trace struct {
	Arrivals []Arrival
}

Trace is a workload: arrivals in time order.

func LongTail

func LongTail(seed uint64, p Params) Trace

LongTail is a steady trickle of short jobs beside a few that run for hours, which is the shape that shows whether a placement policy strands capacity behind something that will not finish.

func MonorepoFanOut

func MonorepoFanOut(seed uint64, p Params) Trace

MonorepoFanOut is one repository whose every run fans out into many jobs that arrive within seconds of each other, a few runs an hour.

func MorningBurst

func MorningBurst(seed uint64, p Params) Trace

MorningBurst is a working day: seventy percent of the jobs land in the first hour as everyone pushes at once, the rest trickle over the next seven.

func ReadTrace

func ReadTrace(r io.Reader) (Trace, error)

ReadTrace reads a trace as JSON lines: one Arrival per line, blank lines ignored. The result is normalized and validated.

func (*Trace) LongestDuration

func (t *Trace) LongestDuration(tier string) time.Duration

LongestDuration is the longest job on one tier, or zero for a tier the trace never uses.

func (*Trace) Normalize

func (t *Trace) Normalize()

Normalize sorts the arrivals by time, numbers them from 1 and fills the defaults a generator or an exporter may leave empty.

SEQUENCE FOLLOWS TIME, ties broken by the order the arrivals were given in, so a trace read from a file and the same trace regenerated number their jobs the same way.

func (*Trace) Tiers

func (t *Trace) Tiers() []string

Tiers reports the distinct labels the trace uses, sorted.

func (*Trace) Validate

func (t *Trace) Validate() error

Validate refuses a trace the replay could not carry faithfully.

func (*Trace) Write

func (t *Trace) Write(w io.Writer) error

Write renders the trace as JSON lines, the shape ReadTrace reads.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL