Documentation
¶
Overview ¶
Package replay drives a trace of job arrivals through the real control plane, node wire, node runtime and simulated backend at compressed time, and reads what billet recorded about each job back out of the ledger.
It exists so a scheduling change can be asked what it does to a workload: under this week of arrivals, where did placement put each job, how long did jobs queue, and how did two policies differ. `make test` proves invariants on single leases and the end-to-end suite proves the wire and the runtime; neither runs a thousand jobs through the placer, because every other backend needs a guest per job.
THE ALLOCATOR, THE LISTENER, THE PLACER AND THE NODE ARE THE REAL ONES, or the harness would measure a model of billet rather than billet. What is scripted is GitHub, through internal/fakeactions, and what is steered is time: one Clock the allocator and the simulated provider read, moved only between events, so a replay is deterministic and a modelled hour costs no wall time.
IT IS A TEST-SIDE CONSUMER AND STAYS ONE. It needs a *testing.T, no production code imports it, and boundary_test.go proves that over the module's import graph, as the simulated backend and the fake Actions service prove theirs.
Index ¶
- Constants
- Variables
- type Arrival
- type Clock
- type Duration
- type Fleet
- type Host
- type Options
- type Params
- type Percentiles
- type Record
- type Report
- func (r *Report) CacheOutcomes() map[string]int
- func (r *Report) CostByTier() (map[string]float64, int)
- func (r *Report) HostsUsed() int
- func (r *Report) LocalityRate() float64
- func (r *Report) PeakDeploymentVCPU() int
- func (r *Report) PeakVCPUByNode() map[string]int
- func (r *Report) Placements() map[int64]string
- func (r *Report) QueueWaitBy(key func(*Record) string) map[string]Percentiles
- func (r *Report) Summary() string
- func (r *Report) WriteJSONL(w io.Writer) error
- type TierShape
- type Trace
Constants ¶
const ( DefaultOwner = "acme" RunnerGroup = "billet-replay" Workflow = "acme/replay/.github/workflows/ci.yml@refs/heads/main" // DefaultBoot is how long a minted runner takes to register with GitHub and // be handed a job, when a fleet does not say. It is the harness's model of // something billet does not decide, and the report names it as such. DefaultBoot = 30 * time.Second )
The one organization, runner group and workflow allowlist every replayed tier is declared under. They are the scripted GitHub's answers to the runner-group policy check, which the real listener and the real node both make before a registration is minted; a trace's per-job repository and workflow ride on the messages and are not checked against them, exactly as GitHub does not.
const ResultSucceeded = "succeeded"
ResultSucceeded is the one completion result billet recognises as success, spelled as the scale-set wire spells it. A trace that names no result gets it.
Variables ¶
DefaultStart is a Monday morning, so a trace reads like a working day.
Functions ¶
This section is empty.
Types ¶
type Arrival ¶
type Arrival struct {
// Seq numbers the arrivals from 1 in arrival order. It is the request id the
// scripted GitHub offers the job under, so the ledger's request_id joins a
// recorded row back to this line without any table of the harness's own.
Seq int64 `json:"seq"`
// At is when the job was queued.
At time.Time `json:"at"`
// Tier is the runs-on label, which is the scale set that will carry it.
Tier string `json:"tier"`
// Owner, Repository and WorkflowRef are what GitHub puts on the message.
Owner string `json:"owner"`
Repository string `json:"repository"`
WorkflowRef string `json:"workflow"`
RunID int64 `json:"run_id"`
// Duration is how long the job ran once a runner had it.
Duration Duration `json:"duration"`
// Result is GitHub's conclusion; empty means ResultSucceeded.
Result string `json:"result"`
}
Arrival is one job arriving at GitHub's queue: what a trace is made of.
A ZERO IS FILLED IN OR REFUSED, NEVER REPLACED ON THE WIRE. The message the scripted service builds overlays these fields on a default, and an overlay cannot tell "zero" from "absent"; so Normalize writes the run id it will use and Validate refuses a job with no time, and what the file says is what is replayed.
type Clock ¶
type Clock struct {
// contains filtered or unexported fields
}
Clock is the one instant the allocator and the simulated provider read.
VIRTUAL, NOT A WALL OFFSET. The end-to-end suite's clock is time.Now plus an offset, which is right for a scenario that waits on real things and wants one duration to pass faster. A replay wants the opposite: two runs of one trace must date every lease identically, and an offset from a wall clock cannot. So this holds an instant that moves only when the harness moves it, and every event in one step is stamped with exactly the same time.
func (*Clock) Step ¶
Step moves the clock to t, or one nanosecond past the present if t is not later, so that every event the harness delivers is dated at an instant of its own.
THE LEDGER'S ORDER MUST BE ITS TIMESTAMPS' ORDER. Two events at one trace instant are delivered one after the other, and what the first did to the fleet is the state the second was placed against; dated identically, a reader sweeping the ledger cannot tell which came first, and a charge made before a same-instant release reads as made after it. A nanosecond keeps the order and moves nothing a report would notice.
type Duration ¶
Duration is a time.Duration that reads and writes as Go spells one ("4m20s"), which is what the exporter script emits and what a person can read in a file.
func (Duration) MarshalJSON ¶
MarshalJSON renders the duration as Go spells it.
func (*Duration) UnmarshalJSON ¶
UnmarshalJSON reads a Go duration string.
type Fleet ¶
type Fleet struct {
Hosts []Host
Tiers []TierShape
// MaxVCPU and MaxMemory are the deployment ceiling, server.max_vcpu and
// server.max_memory.
MaxVCPU int
MaxMemory config.ByteSize
// Placement is server.placement; empty means pack, as it does in a config.
Placement config.PlacementPolicy
// Boot is the modelled time between a mint and the runner taking a job. Zero
// means DefaultBoot.
Boot time.Duration
}
Fleet is the deployment a trace is replayed against.
type Host ¶
type Host struct {
Name string
VCPU int
Memory config.ByteSize
// Site is where the machine is, or empty for a fleet in one place.
Site string
// RatePerHour is what running this machine costs per hour, in dollars. The
// harness's input, like the trace: an owned box's rate is not a fact billet
// records. Zero means unpriced.
RatePerHour float64
}
Host is one simulated machine in the fleet.
type Options ¶
type Options struct {
// Log receives billet's own diagnostics. Nil writes them to the test log.
Log *slog.Logger
// SettleWithin bounds, in wall time, how long billet may take to settle
// after one event before the replay is declared stuck. Zero means a minute.
SettleWithin time.Duration
}
Options vary a replay beyond its fleet and trace.
type Params ¶
type Params struct {
// Jobs is how many arrivals to generate.
Jobs int
// Tiers are the labels jobs are spread over, first the most common.
Tiers []string
// Repositories are the repositories jobs belong to, first the most common.
Repositories []string
// Start is the trace's first instant. A Monday at nine, by default.
Start time.Time
}
Params shape a synthetic trace. Zero values take the defaults below.
type Percentiles ¶
Percentiles are the queue-wait figures the summary reports for one group.
type Record ¶
type Record struct {
Seq int64 `json:"seq"`
LeaseID string `json:"lease_id"`
Tier string `json:"tier"`
Owner string `json:"owner"`
Repository string `json:"repository"`
WorkflowRef string `json:"workflow"`
Node string `json:"node"`
// Provider, InstanceType, VCPU, Memory and Site are what the lease was
// charged for: the backend it ran on, the shape placement bought (empty for a
// host-backed backend), the charged vCPU and memory, and the host's site.
Provider string `json:"provider"`
InstanceType string `json:"instance_type,omitempty"`
VCPU int `json:"vcpu"`
Memory config.ByteSize `json:"memory"`
Site string `json:"site,omitempty"`
// PriceMicrosPerHour is the rate the shape was charged at when the lease was
// escrowed, in millionths of a dollar. Zero means no price was recorded,
// which is what a host-backed lease has.
PriceMicrosPerHour int64 `json:"price_micros_per_hour"`
// Cost is the run's cost in dollars and CostSource says what rate produced
// it: "ledger" when the recorded price was positive, "fleet-rate" when the
// fleet's per-host rate was applied to the charged vCPU-hours, and "" when
// neither was available, in which case Cost is zero and means unknown.
Cost float64 `json:"cost"`
CostSource string `json:"cost_source,omitempty"`
// ImageCache, CacheGeneration and ActionsCache are what the node observed the
// cache do, in the ledger's closed vocabularies; empty means nothing was
// observed, which is what a simulated guest reports.
ImageCache string `json:"image_cache,omitempty"`
CacheGeneration string `json:"cache_generation,omitempty"`
ActionsCache string `json:"actions_cache,omitempty"`
Arrival time.Time `json:"arrival"`
// ChargedFrom is when the lease was escrowed, which is when its shape began
// to count against the host and the deployment: before the job was assigned,
// while the lease was still the tier's discovery slot.
ChargedFrom time.Time `json:"charged_from"`
AssignedAt time.Time `json:"assigned_at"`
StartedAt time.Time `json:"started_at"`
FinishedAt time.Time `json:"finished_at"`
// QueueWait is from the trace's arrival to the ledger's recorded start, and
// RunDuration from that start to the lease's archive.
QueueWait Duration `json:"queue_wait"`
RunDuration Duration `json:"run_duration"`
Conclusion string `json:"conclusion"`
Result string `json:"result"`
Disruption string `json:"disruption,omitempty"`
// Locality is DERIVED FROM PLACEMENT, not observed: an earlier job of the
// same repository was placed on the same node. Beside the observed cache
// columns because a simulated guest observes nothing, and locality is the
// question a placement policy can be asked without one.
Locality bool `json:"locality"`
}
Record is what the ledger recorded about one job, joined to the trace line that produced it.
EVERYTHING BUT Arrival, Repository, WorkflowRef AND Cost COMES FROM THE LEDGER. The arrival is the trace's fact and the join is on the request id billet recorded; the provider, the charged shape and its price, the site and what the cache did are the history row's own columns (migration 49), written from the lease rather than from any catalogue. Cost is the one derived figure and says where its rate came from.
type Report ¶
type Report struct {
Fleet Fleet
// Records are the jobs the ledger knows about, in sequence order.
Records []Record
// Missing are trace jobs with no ledger row: gaps in the recording, reported
// as gaps rather than filled in.
Missing []int64
// Unstarted are recorded jobs the ledger never saw start.
Unstarted []int64
// Unfinished are recorded leases the ledger never archived: their charge has
// no end, so the sweep charges it and never releases it, and a capacity
// verdict over such a ledger is provisional rather than final.
Unfinished []int64
// EscrowRows counts history rows for leases that were never assigned a job:
// the discovery slots released at shutdown. Reported so a reader can tell
// them from a missing job.
EscrowRows int
// Violations are the overcommits the ledger's charges prove: a host or the
// deployment charged more than it has, at some instant.
Violations []string
// contains filtered or unexported fields
}
Report is what one replay recorded.
func Run ¶
Run replays a trace against a fleet and reports what the ledger recorded.
ONE EVENT AT A TIME, TO QUIESCENCE. Every arrival, start and completion is delivered alone, and the next is not delivered until every listener is parked on an empty queue with its escrow settled. Placement happens at escrow, so two events in flight at once would let goroutine order decide which host a job lands on, and a replay whose placements vary between runs cannot compare two policies. Determinism is the property; the seed only shapes a synthetic trace.
func (*Report) CacheOutcomes ¶
CacheOutcomes counts the cache observations the ledger recorded, keyed as "image:<token>" and "actions:<token>". Nothing observed counts nothing.
func (*Report) CostByTier ¶
CostByTier sums the priced records' cost per tier, and counts the records whose cost is unknown.
func (*Report) LocalityRate ¶
LocalityRate is the share of started jobs whose repository already had a job on the same node.
func (*Report) PeakDeploymentVCPU ¶
PeakDeploymentVCPU is the most vCPU the whole fleet carried at one instant, escrow included: the number the deployment ceiling bounds. Not the sum of the hosts' peaks, which need not coincide.
func (*Report) PeakVCPUByNode ¶
PeakVCPUByNode is the most vCPU each host carried at any instant, escrow included.
func (*Report) Placements ¶
Placements is the node each recorded job ran on, by sequence.
func (*Report) QueueWaitBy ¶
func (r *Report) QueueWaitBy(key func(*Record) string) map[string]Percentiles
QueueWaitBy groups the started records by a key and reports each group's queue-wait percentiles.
type Trace ¶
type Trace struct {
Arrivals []Arrival
}
Trace is a workload: arrivals in time order.
func LongTail ¶
LongTail is a steady trickle of short jobs beside a few that run for hours, which is the shape that shows whether a placement policy strands capacity behind something that will not finish.
func MonorepoFanOut ¶
MonorepoFanOut is one repository whose every run fans out into many jobs that arrive within seconds of each other, a few runs an hour.
func MorningBurst ¶
MorningBurst is a working day: seventy percent of the jobs land in the first hour as everyone pushes at once, the rest trickle over the next seven.
func ReadTrace ¶
ReadTrace reads a trace as JSON lines: one Arrival per line, blank lines ignored. The result is normalized and validated.
func (*Trace) LongestDuration ¶
LongestDuration is the longest job on one tier, or zero for a tier the trace never uses.
func (*Trace) Normalize ¶
func (t *Trace) Normalize()
Normalize sorts the arrivals by time, numbers them from 1 and fills the defaults a generator or an exporter may leave empty.
SEQUENCE FOLLOWS TIME, ties broken by the order the arrivals were given in, so a trace read from a file and the same trace regenerated number their jobs the same way.