Documentation
¶
Overview ¶
Package resource defines Dispatch's resource model: what a job needs, what a worker has, and the admission control between them.
Leaf package ¶
resource may import only the root dispatch package, dispatch/id, and the standard library. job imports resource for Options.Resources, so any edge back to job — or to artifact, worker, engine, or store — is an import cycle. This is why the estimator's input is plain data ([]InputSize) rather than []artifact.Ref: the caller translates.
Units ¶
Every quantity is an int64 in a canonical unit, so accounting that adds and subtracts the same values thousands of times cannot drift:
cpu millicores 1 core = 1000 memory bytes disk bytes gpu milli-devices 1 device = 1000
Any other key is a custom resource with user-defined semantics, in the style of Ray's resource dict. Custom quantities are integers.
Index ¶
- Constants
- Variables
- func CheckSchedulable(requests, maxCapacity Set) error
- func EncodeCustomKeys(s Set) string
- func EncodeSet(s Set) ([]byte, error)
- func EncodeSetString(s Set) (*string, error)
- type CapacityConfig
- type Estimator
- type InputSize
- type Lease
- type LeaseInfo
- type Manager
- type ManagerOption
- type Reclaimer
- type Request
- type ResolveInput
- type ResourceFunc
- type Set
- func CPUs(n float64) Set
- func Custom(key string, n int64) Set
- func DecodeSet(b []byte) (Set, error)
- func DecodeSetString(s *string) (Set, error)
- func Detect(cfg CapacityConfig) Set
- func DiskBytes(n int64) Set
- func DiskGB(n int64) Set
- func GPUs(n float64) Set
- func MemoryBytes(n int64) Set
- func MemoryGB(n int64) Set
- func (s Set) Add(o Set) Set
- func (s Set) Clone() Set
- func (s Set) CustomKeys() []string
- func (s Set) Exceeds(capacity Set) []string
- func (s Set) Fits(capacity Set) bool
- func (s Set) IsZero() bool
- func (s Set) Keys() []string
- func (s Set) Max(o Set) Set
- func (s Set) Scale(f float64) Set
- func (s Set) Sub(o Set) Set
- type Spec
Constants ¶
const ( // DefaultCPUOvercommit is 1.0: no overcommit unless asked for. DefaultCPUOvercommit = 1.0 // DefaultMemoryFraction leaves 20% for the Go runtime, the OS page // cache, and everything else sharing the box. DefaultMemoryFraction = 0.8 )
Default capacity tuning.
const ( // CPU is measured in millicores. One core is 1000. CPU = "cpu" // Memory is measured in bytes. Memory = "memory" // Disk is measured in bytes. For a worker this is the staging cache // budget; for a job, the bytes its inputs and outputs need locally. Disk = "disk" // GPU is measured in milli-devices. One device is 1000, so a // fractional declaration is expressible. Kubernetes accepts only // whole devices, so track C rounds up at translation. GPU = "gpu" )
Canonical resource keys. Any key outside this set is a custom resource: an integer quantity with user-defined semantics.
const ( // SourceFunc is a job definition's or enqueue's ResourceFunc. SourceFunc = "func" // SourceEstimator is the installed Estimator. SourceEstimator = "estimator" )
The sources ResolveInput.OnError names.
const CustomKeySep = ","
CustomKeySep delimits the custom-resource key list produced by EncodeCustomKeys. The list is stored as a delimited string rather than an array so every store backend can express the containment test in its own idiom (a LIKE/GLOB pattern, a substring match, ...) without a schema translation.
const MilliScale = 1000
MilliScale is the multiplier for the milli-denominated keys.
Variables ¶
var ErrCapacityExceeded = errors.New("dispatch/resource: capacity exceeded")
ErrCapacityExceeded means an acquisition could not be satisfied: either the request is larger than total capacity and never can be, or every holder still held its share when the caller's context ended.
var ErrUnschedulable = errors.New("dispatch/resource: no worker can satisfy the requirement")
ErrUnschedulable means a requirement exceeds the largest known worker capacity, so no worker could ever run it. Returned at enqueue so the job fails on a developer's machine rather than pending forever.
Functions ¶
func CheckSchedulable ¶
CheckSchedulable rejects a requirement no worker could ever satisfy, with an error wrapping ErrUnschedulable that names each dimension that does not fit. An empty maxCapacity means capacity is unknown, which disables the check rather than rejecting everything.
Resolve runs it on the requirement it resolves. It is exported for a caller that already holds a resolved requirement, such as a dead letter replay carrying the failed job's Resources, and must check it against today's fleet without resolving it again.
func EncodeCustomKeys ¶
EncodeCustomKeys renders the non-canonical keys of s carrying a nonzero quantity as a delimited string with a leading and trailing separator, so a containment test can match on ",fpga," and never partially match ",fpga-large,". A Set with no custom keys — including a nil or zero Set — encodes to "".
func EncodeSet ¶
EncodeSet marshals a Set for a byte-oriented JSON column (e.g. Postgres JSONB). A zero Set — nil or every quantity zero — encodes to nil rather than "{}", so an undeclared job's row stays indistinguishable from one written before these columns existed: a genuine SQL NULL, not an empty JSON object.
func EncodeSetString ¶
EncodeSetString is EncodeSet for backends whose JSON column is a string-typed TEXT rather than a byte column — SQLite has no JSONB type, so its column is *string, where nil is what stores as SQL NULL. The encoding rules are identical to EncodeSet: a zero Set produces a nil pointer, never a pointer to "" or "{}".
Types ¶
type CapacityConfig ¶
type CapacityConfig struct {
// CPUOvercommit multiplies detected cores. CPU is compressible:
// exceeding it makes jobs slow, not dead, so overcommit is safe
// where memory overcommit is not.
CPUOvercommit float64
// MemoryFraction is the share of the detected limit to advertise.
MemoryFraction float64
// DiskBytes is the staging cache budget. Zero omits the key.
DiskBytes int64
// Explicit overrides detection per key, and is the only way to
// declare a custom resource.
Explicit Set
}
CapacityConfig controls how a worker's capacity is derived.
There is no MemoryOvercommit. Overcommitting memory is how a box gets into the OOM cascade this package exists to prevent, and a knob whose only outcome is an incident should not exist.
func DefaultCapacityConfig ¶
func DefaultCapacityConfig() CapacityConfig
DefaultCapacityConfig returns the conservative defaults.
type Estimator ¶
Estimator infers a requirement from historical measurement.
There is no built-in implementation. The interface exists so that a rollup over observed usage, or a learned predictor, can be slotted in with nothing else moving; until one is installed, requirements come from the declaration and the enqueue-time override alone.
type InputSize ¶
type InputSize struct {
// Name is the declared input slot name.
Name string `json:"name"`
// Bytes is the input's size.
Bytes int64 `json:"bytes"`
// Hash may be empty: the artifact plane fills content_hash
// opportunistically at first staging, not at registration.
Hash string `json:"hash,omitempty"`
}
InputSize describes one declared input at enqueue time.
It is plain data rather than an artifact.Ref because resource is a leaf package (see doc.go). The engine translates bindings into these, which also makes an Estimator testable with a struct literal.
type LeaseInfo ¶
type LeaseInfo struct {
Owner string `json:"owner"`
Held Set `json:"held"`
AcquiredAt time.Time `json:"acquired_at"`
}
LeaseInfo is a point-in-time view of one lease, for the capacity API.
type Manager ¶
type Manager interface {
// Acquire blocks until want fits, reclaiming where a Reclaimer is
// registered. It is bounded by ctx, so a blocked job can never
// outlive its deadline. A want larger than total capacity fails
// immediately: no release or reclamation could ever satisfy it, so
// waiting would only defer a certain error.
Acquire(ctx context.Context, owner string, want Set) (Lease, error)
// TryAcquire is the non-blocking form. It never reclaims, because a
// caller that cannot wait also cannot afford eviction I/O.
TryAcquire(owner string, want Set) (Lease, bool)
// Free is what is immediately available.
Free() Set
// Reclaimable is what registered reclaimers could free on top of Free.
Reclaimable() Set
// Capacity is the configured total.
Capacity() Set
// Leases is a snapshot of what is currently held.
Leases() []LeaseInfo
// RegisterReclaimer installs the reclaim policy for one key.
RegisterReclaimer(key string, r Reclaimer)
// Wake tells blocked acquirers that a reclaimer's holdings changed
// — that something now reclaimable was not a moment ago.
//
// The manager broadcasts when a lease is released and on nothing
// else, and it cannot see inside a reclaimer. An artifact cache
// whose last stager lets go of an entry has just made those bytes
// evictable without releasing anything, so the acquirer that asked
// for them, was told "everything is pinned", and went to sleep will
// sleep to its deadline unless it is told to look again.
Wake()
}
Manager admits work against a fixed capacity.
It generalizes the artifact cache's single-key disk budget to N keys: same mutex and condition variable, same context-bounded wait, same reclaim-then-wait loop — but keyed, so memory and CPU are accounted by the mechanism that already worked for disk rather than by a second one built alongside it.
func NewManager ¶
func NewManager(capacity Set, opts ...ManagerOption) Manager
NewManager builds a manager over a fixed capacity.
type ManagerOption ¶
type ManagerOption func(*manager)
ManagerOption configures a manager.
func WithClock ¶
func WithClock(now func() time.Time) ManagerOption
WithClock replaces the time source, for deterministic tests.
type Reclaimer ¶
type Reclaimer interface {
// Reclaim frees up to need units of key, returning how many it
// actually freed. Returning zero means nothing more is reclaimable.
//
// The units are returned to the manager by releasing the Lease that
// held them. The return value is only a "something changed, re-check"
// signal — the manager does not add it to the ledger, because those
// units are still recorded against a live lease until that lease is
// released, and counting them in both places would invent capacity.
//
// Reclaim is called with the manager's lock dropped, so releasing a
// lease from inside it is safe and is the expected implementation.
Reclaim(ctx context.Context, key string, need int64) (int64, error)
// Available reports how much could be freed without blocking.
Available(key string) int64
}
Reclaimer frees capacity for one key on the manager's behalf.
It exists because the resource dimensions are not symmetric: cached bytes on disk can be evicted to make room, while memory held by a running job cannot. Registering a reclaimer for a key turns blocking into "reclaim, then block only if that was not enough". A key with no reclaimer — memory, CPU — can only wait for a release.
A reclaimer holds a Manager lease for what it caches, and returns units by releasing that lease — it never credits the ledger directly. The artifact cache holds one lease per cached entry, so evicting an entry releases that entry's lease and the bytes come back through the same path a finished job's would.
type Request ¶
type Request struct {
JobName string
Queue string
Payload []byte
Inputs []InputSize
InputBytes int64
Declared Set
Attempt int
ScopeOrgID string
}
Request is everything an estimator may consider.
type ResolveInput ¶
type ResolveInput struct {
GlobalDefault Set
QueueDefault Set
Declared Set
Func ResourceFunc
Estimator Estimator
Override Set
DeclaredLimits Set
OverrideLimits Set
Class string
Request Request
// MaxCapacity is the largest single-worker capacity known to the
// engine. Empty disables the unschedulable check, which is correct
// for a single-process engine with no registered workers.
MaxCapacity Set
// OnError is called when Func or Estimator returns an error, with
// the source that failed (SourceFunc or SourceEstimator).
//
// Neither may fail an enqueue — that contract is unchanged, and the
// lower-precedence value stands. But a fallback with no signal is
// how a misconfigured estimator under-sizes every job in the fleet
// forever: the jobs enqueue, they run, they OOM, and nothing
// anywhere says the estimator was the reason. Callers should log it.
//
// Optional. Nil discards, which is the right default for a test that
// only cares about precedence. Called synchronously inside Resolve on
// the enqueue path: it must not block or panic, or it fails or stalls
// the enqueue that triggered it.
OnError func(source string, err error)
}
ResolveInput carries every source a requirement can come from, lowest precedence first.
type ResourceFunc ¶
ResourceFunc computes a requirement from the enqueue-time request. It runs once, in the enqueuing process, never on the scheduling path.
type Set ¶
Set is a resource vector. An absent key is zero, so a Set never needs to enumerate keys it does not constrain.
Set is a map, so it is not safe for concurrent mutation. Every method here returns a new Set rather than mutating the receiver, which makes a Set stored on a job or a lease safe to read from many goroutines.
func DecodeSet ¶
DecodeSet unmarshals a column produced by EncodeSet, treating both a NULL column (nil slice) and an empty one (zero-length slice) as "unset" rather than an error.
func DecodeSetString ¶
DecodeSetString is DecodeSet for the *string TEXT representation, treating both a NULL column (nil pointer) and an empty string as "unset".
func Detect ¶
func Detect(cfg CapacityConfig) Set
Detect derives a worker's capacity, with Explicit overriding any autodetected key.
func (Set) CustomKeys ¶
CustomKeys returns the non-canonical keys carrying a nonzero quantity, sorted. This is the set persisted on the job row and matched by containment at dequeue.
func (Set) Exceeds ¶
Exceeds returns the keys on which s does not fit capacity, sorted. It is what turns a failed admission into an error naming the dimension that did not fit rather than a bare "does not fit".
func (Set) Fits ¶
Fits reports whether every quantity in s is within capacity. An absent capacity key is zero, so demanding a resource the capacity does not list never fits.
func (Set) IsZero ¶
IsZero reports whether every quantity is zero. A zero Set means "no declared requirement", which is how every job behaves before this feature is configured.
func (Set) Max ¶
Max returns the per-key maximum, used to merge a floor into a resolved requirement.
type Spec ¶
type Spec struct {
// Requests is what admission accounts for and what a pod requests.
Requests Set `json:"requests,omitempty"`
// Limits is the enforcement ceiling. Memory defaults to Requests
// (guaranteed); CPU is left unset (burstable), because overrunning
// CPU makes a job slow while overrunning memory makes it dead.
Limits Set `json:"limits,omitempty"`
// Class is an opaque scheduling class for the isolation backend.
Class string `json:"class,omitempty"`
}
Spec is the resolved, immutable resource contract for one job. It is produced once at enqueue and written to the job row, so scheduling reads columns and never calls user code.
Spec is also the contract track C consumes to build a pod: Requests map to resource requests, Limits to limits, and Class to whatever scheduling class the isolation backend uses.
func Resolve ¶
func Resolve(ctx context.Context, in ResolveInput) (Spec, error)
Resolve collapses every source into one Spec.
Precedence is a per-key overlay, lowest first:
global default → queue default → declaration → func → estimator → override
Per-key rather than whole-set replacement, so an estimator that predicts only memory leaves a declared CPU value intact.
Neither the func nor the estimator may fail enqueue: a failure there is handed to OnError and the lower-precedence value stands. The one error Resolve does return is ErrUnschedulable, because a job no worker can run must fail loudly and immediately.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package resourcetest provides fakes for testing resource-aware code: a deterministic clock, a lease-based reclaimer, and a scripted estimator.
|
Package resourcetest provides fakes for testing resource-aware code: a deterministic clock, a lease-based reclaimer, and a scripted estimator. |