Documentation
¶
Overview ¶
Package capability declares the three contracts a backend can implement and the capabilities it negotiates through them.
Mercator brokers two materially different things and must never confuse them. A CapacityProvider allocates and holds machine capacity. A NodeRuntime executes successive workloads on capacity Mercator controls through an enrolled agent. An EphemeralExecutor runs one workload on a provider-native execution product Mercator does not control between workloads.
The lane a backend declares is what the connection sells. A CapacityProvider sells machines that outlive the workloads run on them, which is the reusable lane; an EphemeralExecutor sells one execution that holds nothing afterwards, which is the ephemeral one.
A lane is not a licence to place work. Whether a workload can actually run on one machine is a fact about that machine, established by the node runtime enrolled on it, and no per-connection declaration can state it: a node enrolls with the control plane rather than with the connection that rented the machine it runs on. That is why the offers Mercator places against are the enrolled nodes' own, and why a capacity connection publishes no placement candidate until a Rental lifecycle can provision a machine and bootstrap an agent onto it. See broker.Backend.ListOffers.
Index ¶
- Constants
- Variables
- func StampLane(declaration Declaration, offers []domain.OfferSnapshot) []domain.OfferSnapshot
- type AcceleratorFacts
- type Backend
- type CapacityCommand
- type CapacityObservation
- type CapacityOperation
- type CapacityProvider
- type CapacityQuery
- type CapacityReceipt
- type CapacityRef
- type CapacityState
- type CapacitySupport
- type Declaration
- type DiskFacts
- type Enrollment
- type EnrollmentRequest
- type EnvironmentBinding
- type EphemeralExecutor
- type EphemeralSupport
- type HostFacts
- type ImageLocality
- type LaunchWorkloadCommand
- type NodeBootstrap
- type NodeFacts
- type NodeRef
- type NodeRuntime
- type NodeSupport
- type OperationReceipt
- type OwnedCapacity
- type OwnershipQuery
- type PrepareArtifactCommand
- type PrepareImageCommand
- type ProvisionCommand
- type Reconciliation
- type SessionRenewal
- type StopWorkloadCommand
- type WorkloadObservation
- type WorkloadPhase
- type WorkloadRef
Constants ¶
const ( IdempotentProvisionOperationKey = "operation_key" IdempotentProvisionNone = "none" )
Provision idempotency mechanisms. They are not interchangeable: a provider that deduplicates on a caller-supplied key can be asked for the same machine twice and allocate one, and a provider that honors nothing can only be reconciled by listing what this connection owns.
Variables ¶
var ErrCapabilityUnsupported = fmt.Errorf("capability: operation unsupported by this backend")
ErrCapabilityUnsupported is returned when a caller invokes an operation the provider's CapacitySupport does not claim. It is a routing bug, never a runtime condition to retry, so providers surface it explicitly rather than silently succeeding.
var ErrCapacityIndeterminate = fmt.Errorf("capability: the outcome of this provision is unknown")
ErrCapacityIndeterminate is a provision whose outcome nobody knows: the command reached the provider and the answer did not come back, so a machine may be billing for this lease right now and may not.
It is a different answer from a failure and callers act on the difference. A failure allocated nothing and can be asked again; this cannot, because asking again is how one lost answer becomes two machines. What resolves it is reading what the connection owns, which is why ListOwned is what CapacitySupport refuses a provider without server-side idempotency for lacking.
Functions ¶
func StampLane ¶
func StampLane(declaration Declaration, offers []domain.OfferSnapshot) []domain.OfferSnapshot
StampLane marks every offer with the lane its backend actually serves and clears whatever Rental identity the adapter stated.
A Rental is Mercator's own lease record, so only Mercator mints one, and it mints it where it holds the machine: the offers that carry a Rental identity are the enrolled nodes' own, from the invitation that named the Rental, and they never pass through here. An adapter that filled the field in from its instance type or its contract id would publish a Rental Mercator does not hold on /v1/offers, and a Booking bound to it would let a second Run queue behind a lease that never existed. Clearing it unconditionally is why no lane needs checking here.
Types ¶
type AcceleratorFacts ¶
type AcceleratorFacts struct {
Established bool `json:"established"`
// Vendor is who makes the driver this host runs, stated only where there is
// one. A machine that looked and found no driver states the fact that it
// looked and nothing else.
Vendor string `json:"vendor,omitempty"`
DriverVersion string `json:"driver_version,omitempty"`
DriverCapability string `json:"driver_capability,omitempty"`
// Devices is the cards this host holds, in the same vocabulary a provider
// lists them in, because Placement counts both against one requirement.
Devices []domain.AcceleratorInventory `json:"devices,omitempty"`
}
AcceleratorFacts is what this node established about the cards under it and the driver that drives them, and separately whether it established anything at all.
The two are stated apart for the reason DiskFacts states them apart, and the cost of not stating them apart is larger here. Every field below is empty on a machine with no cards and empty on a machine whose agent never looked, so a reader with only the values cannot tell a CPU box from an unmeasured GPU box. Read as a measurement, the empty inventory strikes a real GPU machine out of every accelerator placement with RESOURCE_INSUFFICIENT, which says the fleet can never run this work on the strength of nobody having run nvidia-smi.
Established is the agent saying it looked, whatever it found. A machine that looked and found no NVIDIA driver has established that there is none, which is a refusal an operator can act on; a machine that never looked has established nothing, and NodeFacts.Established erases whatever it happened to carry so no reader downstream has two answers to choose between.
func (AcceleratorFacts) Attestations ¶
func (facts AcceleratorFacts) Attestations() map[domain.HostFact]bool
Attestations is what this node's accelerator report says in the vocabulary an offer publishes. A node that looked states the driver fact either way; a node that never looked states nothing, so Placement refuses it as a silence rather than as a machine with no driver.
func (AcceleratorFacts) Driver ¶
func (facts AcceleratorFacts) Driver() domain.AcceleratorDriver
Driver is the host half of the accelerator stack as an offer carries it.
type Backend ¶
type Backend any
Backend is one built connection's implementation. It satisfies at least one of the three contracts in this package; Declare reports which, and refuses combinations that would claim semantics the implementation cannot deliver.
type CapacityCommand ¶
type CapacityCommand struct {
CapacityRef
OperationKey string
RequestHash string
// Generation is the fencing token for this Rental's lifecycle cycle. A
// provider ignores commands stamped with a superseded generation.
Generation uint64
}
CapacityCommand mutates allocated capacity. OperationKey makes the command replayable: the same key must produce the same effect exactly once, and a repeat reports Duplicate rather than acting twice.
type CapacityObservation ¶
type CapacityObservation struct {
NativeRef string `json:"native_ref"`
State CapacityState `json:"state"`
ObservedAt time.Time `json:"observed_at"`
// StateSince is the provider's own account of when this machine entered the
// state it is reporting, and the zero time from a provider that does not date
// its transitions.
//
// It is the difference between measuring a machine and measuring a poll. A
// caller that has only ObservedAt knows the stage finished somewhere between
// this look and the last one, and writing that interval down as the stage's
// duration would publish the caller's own cadence as a property of the
// machine. Callers that cannot get this fact record a bound and say so rather
// than reporting the interval as a measurement.
StateSince time.Time `json:"state_since,omitzero"`
// Endpoint is where the machine can be reached, when the provider exposes
// one. It is provenance for operators, never a control channel: Mercator
// reaches nodes only through their outbound agent session.
Endpoint string `json:"endpoint,omitempty"`
// Interrupted reports provider-initiated reclamation of interruptible
// capacity, which is a different fact from a machine an operator stopped.
Interrupted bool `json:"interrupted,omitempty"`
NativeJSON string `json:"native_json,omitempty"`
}
type CapacityOperation ¶
type CapacityOperation string
CapacityOperation is one act a caller can ask of a CapacityProvider. It is named for the act rather than for the method that performs it, because a capability set negotiates a stop and a resume and a caller refused one is being told which promise the provider never made: StartCapacity is how a resume is performed and "start" is not a promise anybody negotiated.
const ( CapacityProvision CapacityOperation = "provision" CapacityObserve CapacityOperation = "observe" CapacityStop CapacityOperation = "stop" CapacityResume CapacityOperation = "resume" CapacityTerminate CapacityOperation = "terminate" CapacityListOwned CapacityOperation = "list_owned" )
type CapacityProvider ¶
type CapacityProvider interface {
// CapacitySupport reports what this provider can actually do. Callers
// negotiate against it instead of assuming stop, resume, persistent disk,
// spot behavior, or exact pricing.
CapacitySupport() CapacitySupport
// Verify performs a cheap credential and reachability check. It allocates
// nothing.
Verify(ctx context.Context) error
ListCapacity(ctx context.Context, query CapacityQuery) ([]domain.OfferSnapshot, error)
// ProvisionCapacity allocates fresh capacity for one Rental. It is named
// for the lease like every other act here, because a provider that also
// launches workloads has two things it could be asked to provision.
ProvisionCapacity(ctx context.Context, command ProvisionCommand) (CapacityReceipt, error)
ObserveCapacity(ctx context.Context, ref CapacityRef) (CapacityObservation, error)
// StartCapacity resumes stopped capacity. Providers without SupportsResume
// return ErrCapabilityUnsupported.
StartCapacity(ctx context.Context, command CapacityCommand) (CapacityReceipt, error)
// StopCapacity suspends capacity while retaining its identity and, where
// SupportsPersistentDisk holds, its disk. Providers without SupportsStop
// return ErrCapabilityUnsupported.
StopCapacity(ctx context.Context, command CapacityCommand) (CapacityReceipt, error)
TerminateCapacity(ctx context.Context, command CapacityCommand) (CapacityReceipt, error)
// ListOwnedCapacity enumerates capacity this connection allocated, so
// reconciliation can adopt or terminate resources the control plane lost
// track of.
ListOwnedCapacity(ctx context.Context, query OwnershipQuery) ([]OwnedCapacity, error)
}
CapacityProvider allocates and holds machine capacity. It knows nothing about workloads: what runs on the capacity is the NodeRuntime's business. Splitting the two is what lets Mercator keep a machine across Runs without the provider knowing, and lets a provider stop or resume a machine without the control plane guessing what that does to a running workload.
type CapacityQuery ¶
type CapacityQuery struct {
Resources domain.ResourceRequirements
}
CapacityQuery scopes a capacity listing to one requirement.
type CapacityReceipt ¶
type CapacityReceipt struct {
NativeRef string `json:"native_ref"`
State CapacityState `json:"state"`
AcceptedAt time.Time `json:"accepted_at"`
// Duplicate reports that this OperationKey had already been accepted, so
// the caller must not count the effect twice.
Duplicate bool `json:"duplicate"`
// Interruptible reports what the provider actually allocated, which may
// differ from what was requested.
Interruptible bool `json:"interruptible"`
Pricing domain.PriceModel `json:"pricing"`
}
type CapacityRef ¶
type CapacityRef struct {
ConnectionID string
RentalID string
// NativeRef is the provider's own identifier for the machine.
NativeRef string
// OwnershipToken proves the capacity belongs to this deployment, so a
// reconciler never acts on a machine it merely resembles.
OwnershipToken string
}
CapacityRef names allocated capacity well enough to observe it after a control-plane restart, without consulting any in-memory state.
type CapacityState ¶
type CapacityState string
CapacityState is the provider's own view of one machine's lifecycle.
const ( CapacityStateRequested CapacityState = "requested" CapacityStateStarting CapacityState = "starting" CapacityStateActive CapacityState = "active" CapacityStateStopping CapacityState = "stopping" CapacityStateStopped CapacityState = "stopped" CapacityStateTerminated CapacityState = "terminated" // CapacityStateUnknown is an honest answer, not a failure: the provider // answered without a state Mercator recognizes, or did not answer at all. CapacityStateUnknown CapacityState = "unknown" )
func (CapacityState) Terminal ¶
func (state CapacityState) Terminal() bool
Terminal reports whether no further provider-side transition is expected.
func (CapacityState) Valid ¶
func (state CapacityState) Valid() bool
type CapacitySupport ¶
type CapacitySupport struct {
// Stop and Resume report whether capacity can be suspended and brought
// back under the same identity. A provider with Stop but not Resume can
// only shed cost by destroying capacity.
Stop bool `json:"stop"`
Resume bool `json:"resume"`
// PersistentDisk reports whether a stopped machine keeps its disk, which
// is what makes a resumed machine still warm.
PersistentDisk bool `json:"persistent_disk"`
// Spot reports whether this provider can allocate interruptible capacity.
Spot bool `json:"spot"`
// ExactPricing reports whether the provider states a rate Mercator can
// bill against, rather than an estimate.
ExactPricing bool `json:"exact_pricing"`
// IdempotentProvision names the mechanism the provider honors for repeated
// provision commands: "operation_key" when the provider deduplicates on a
// caller-supplied key, "none" when the caller must reconcile by listing
// owned capacity.
IdempotentProvision string `json:"idempotent_provision"`
ListOwned bool `json:"list_owned"`
// ObserveAfterTerminate reports whether a terminated resource remains
// observable. When false, a not-found observation after terminate is
// confirmation rather than ambiguity.
ObserveAfterTerminate bool `json:"observe_after_terminate"`
}
CapacitySupport is one provider's negotiated capability set. Every field is a claim the scheduler and reconciler are entitled to rely on.
func (CapacitySupport) Claims ¶
func (support CapacitySupport) Claims(operation CapacityOperation) bool
Claims reports whether this negotiated set promises one operation.
Allocating, observing, and destroying capacity are the floor of the contract: a provider that cannot do all three cannot hold a machine for Mercator at all, so there is no field to negotiate them with. Suspending, resuming, and enumerating what this connection owns are promises a provider makes or does not, and a caller that sends one anyway is asking a provider to keep a promise it never made.
func (CapacitySupport) Validate ¶
func (support CapacitySupport) Validate() error
Validate refuses a negotiated set no provider could keep. Every promise here is one the scheduler and the reconciler act on without asking again, so a set that contradicts itself has to be refused where it enters Mercator rather than caught by whichever caller happens to read two of its fields together.
type Declaration ¶
type Declaration struct {
// Type is the adapter type string ("docker", "shadeform", …).
Type string `json:"type"`
Lane domain.ExecutionLane `json:"lane"`
// Capacity is present exactly when the backend implements CapacityProvider.
Capacity *CapacitySupport `json:"capacity,omitempty"`
// Node is present exactly when the backend implements NodeRuntime, which is
// a runtime bound to one host. It is absent for a provider adapter, because
// the runtime that executes on a rented machine enrolls with the control
// plane rather than with the connection that rented it, and absent for every
// one-shot connection, because nothing executes a second workload there.
Node *NodeSupport `json:"node,omitempty"`
// Ephemeral is present exactly when the backend implements
// EphemeralExecutor.
Ephemeral *EphemeralSupport `json:"ephemeral,omitempty"`
}
Declaration is what one backend claims it can do. It is checked against the interfaces the backend actually implements, so a lane is evidence rather than an assertion.
func Declare ¶
func Declare(adapterType string, backend Backend) (Declaration, error)
Declare derives a backend's Declaration from the contracts it implements.
Capacity is the reusable lane, because a machine a provider allocates and holds is exactly capacity that outlives the workload run on it. There is no second condition on it: a deployment-wide "does anything execute here" check was one this deployment could always answer yes to while owning no runtime on the rented machine at all, so it refused nothing and licensed a Rental identity for a machine Mercator had not allocated. The falsifiable claim is per machine, and the enrolled node runtime is what makes it, which is why a capacity connection publishes no placement candidate.
type DiskFacts ¶
type DiskFacts struct {
Known bool `json:"known"`
TotalBytes int64 `json:"total_bytes,omitempty"`
FreeBytes int64 `json:"free_bytes,omitempty"`
}
DiskFacts is the room on the filesystem this node's content lands on, and separately whether this node could measure it at all. The two are stated apart for the reason every other inventory here states them apart: a machine that could not look is not a machine with no room, and a report that could only say "zero" would make the two indistinguishable.
A node that cannot measure its disk keeps reporting everything else. Its liveness, its running containers, and the exit codes they produce are facts the fleet needs whatever anyone knows about the disk, and failing the whole report over this one measurement ends the node's session and, on a node with no session yet, its enrollment. What silence costs instead is placements: an offer states the room a machine established, so a node that established none advertises none and is struck out by the disk floor every workload carries, in the Booking Decision where an operator can read it.
Bytes stated beside Known false are bytes nobody established, and NodeFacts. Established erases them where the report arrives. Every reader downstream is therefore reading one answer rather than deciding between two.
type Enrollment ¶
type Enrollment struct {
NodeID string `json:"node_id"`
// SessionToken authenticates subsequent calls for this session only. It
// expires; the node renews it while its lease holds.
SessionToken string `json:"session_token"`
SessionExpires time.Time `json:"session_expires"`
// FencingToken increases on every enrollment. A command carrying a lower
// token is refused, which is what stops a partitioned old session from
// acting after a new one took over.
FencingToken uint64 `json:"fencing_token"`
// LeaseExpires is when the control plane stops believing this node is
// alive absent a heartbeat.
LeaseExpires time.Time `json:"lease_expires"`
// Duplicate reports that this node identity and generation were already
// enrolled, so the caller resumed rather than joined.
Duplicate bool `json:"duplicate"`
}
Enrollment is the authenticated session the control plane grants.
type EnrollmentRequest ¶
type EnrollmentRequest struct {
NodeID string `json:"node_id"`
RentalID string `json:"rental_id"`
Generation uint64 `json:"generation"`
// EnrollmentToken is the short-lived, single-use material the node
// received through its bootstrap.
EnrollmentToken string `json:"enrollment_token"`
// AgentVersion is the build actually running, which may differ from the
// version the bootstrap pinned.
AgentVersion string `json:"agent_version"`
Facts NodeFacts `json:"facts"`
}
EnrollmentRequest is what a node presents to join. Identity is immutable: the node does not choose its own ID, and it cannot claim a generation it was not provisioned for.
type EnvironmentBinding ¶
EnvironmentBinding is one environment variable delivered to a workload.
type EphemeralExecutor ¶
type EphemeralExecutor interface {
// EphemeralSupport reports what this executor can do, including the
// lifecycle facts that are weaker here than on reusable capacity.
EphemeralSupport() EphemeralSupport
Verify(ctx context.Context) error
ListOffers(ctx context.Context, req adapter.OfferRequest) ([]domain.OfferSnapshot, error)
Launch(ctx context.Context, req adapter.LaunchRequest) (adapter.LaunchReceipt, error)
Observe(ctx context.Context, req adapter.ObserveRequest) (adapter.ExternalObservation, error)
// Release removes only Mercator's execution from a pool it does not own.
Release(ctx context.Context, req adapter.ReleaseRequest) (adapter.ReleaseReceipt, error)
// Terminate destroys a one-shot resource Mercator created for this
// execution. Standing-pool executors return adapter.ErrTerminateUnsupported.
Terminate(ctx context.Context, req adapter.TerminateRequest) (adapter.TerminateReceipt, error)
ListOwned(ctx context.Context, req adapter.OwnershipQuery) ([]adapter.OwnedExternalObject, error)
}
EphemeralExecutor runs one workload on a provider-native execution product. Mercator does not control a host runtime between workloads here, so an ephemeral execution never becomes a Rental, never carries a Rental Schedule, and its locality is not observable from one Run to the next.
The methods are the existing provider seam, unchanged: what changes is that they now name one lane instead of standing for every backend Mercator has. A provider that also rents durable machines implements CapacityProvider instead, and its machines execute through a NodeRuntime.
type EphemeralSupport ¶
type EphemeralSupport struct {
// ReusableBetweenRuns is always false. It exists so the negotiated
// capability set answers the reuse question directly rather than by
// omission, and so a test can assert no executor ever claims otherwise.
ReusableBetweenRuns bool `json:"reusable_between_runs"`
// ObservableLocality reports whether Mercator can learn what content the
// execution environment already holds. Ephemeral products generally cannot
// tell it, which makes their locality unknown rather than cold.
ObservableLocality bool `json:"observable_locality"`
// CancelQueued reports whether an execution can be cancelled before it
// starts running.
CancelQueued bool `json:"cancel_queued"`
// ProviderTTL reports whether the provider reclaims the execution on its
// own, which bounds the cost of a lost response.
ProviderTTL bool `json:"provider_ttl"`
// IdempotentLaunch names the mechanism the provider honors for repeated
// launch commands: "operation_key", "launch_key", or "none".
IdempotentLaunch string `json:"idempotent_launch"`
ListOwned bool `json:"list_owned"`
// ExactPricing reports whether the provider states a billable rate.
ExactPricing bool `json:"exact_pricing"`
}
EphemeralSupport states the weaker lifecycle and locality guarantees of the ephemeral lane explicitly, so no caller has to infer them from silence.
type HostFacts ¶
type HostFacts struct {
OS string `json:"os"`
KernelVersion string `json:"kernel_version"`
Architecture string `json:"architecture"`
ContainerRuntime string `json:"container_runtime"`
RuntimeVersion string `json:"runtime_version"`
// AcceleratorToolkit is what the container runtime needs in order to hand a
// container the cards at all, which is the runtime's business rather than the
// driver's. It sits beside the driver facts rather than inside them because
// the three legs of the contract have three owners: the host provides the
// driver, the container runtime provides the passthrough, and the image
// provides the workload's own accelerator stack.
AcceleratorToolkit string `json:"accelerator_toolkit,omitempty"`
Accelerator AcceleratorFacts `json:"accelerator,omitzero"`
CPUMillis int64 `json:"cpu_millis"`
MemoryBytes int64 `json:"memory_bytes"`
Disk DiskFacts `json:"disk,omitzero"`
Network []domain.NetworkFact `json:"network,omitempty"`
}
HostFacts is the substrate the node runs on, which is separate from what a workload image carries. Mercator matches a workload's compatibility contract against these, and never installs a workload's accelerator stack onto the host.
type ImageLocality ¶
type ImageLocality struct {
// ManifestDigest is the digest this image was pulled by, which for a
// multi-platform image is the index digest and never the platform manifest
// underneath it. It is the identity a Run is pinned to. A tag is never
// image identity.
ManifestDigest string `json:"manifest_digest"`
// Platform is which build of that digest this machine holds. An index
// digest names one image per platform, so the digest alone cannot say
// whether what is here is what this host would run: a machine that fetched
// another platform's build holds the same name and none of the bytes.
Platform domain.Platform `json:"platform"`
// LayerDigests is the compressed layer blobs this node holds unpacked,
// named the way a registry names them. A container daemon cannot enumerate
// these: it discards the compressed form when it unpacks an image, which is
// why LayerDiffIDs sits beside this rather than instead of it.
LayerDigests []string `json:"layer_digests,omitempty"`
// LayerDiffIDs is the same unpacked content named by its uncompressed form.
// This is what a Docker daemon can actually report, and comparing it
// against a registry manifest is what a resolved manifest carrying both
// spaces makes possible. Both lists name layers this node has assembled
// into a mountable chain: content a runtime has fetched and not unpacked is
// content it cannot name a layer identity for, which is why an image that
// is here and not ready says so with ContentPresent instead.
LayerDiffIDs []string `json:"layer_diff_ids,omitempty"`
// ContentPresent reports that every byte of this image is on this machine,
// whether or not the chain a container starts on is assembled. Fetching and
// unpacking are separate acts and only the node can say which it has done:
// what is left on content already here is local work rather than a pull,
// and an operator sent after a network problem for bytes on the disk finds
// nothing to fix. A runtime that cannot establish it says so with State
// unknown rather than guessing in either direction.
ContentPresent bool `json:"content_present"`
// State is how much of this image the node established it has: hot when the
// whole chain is assembled and a container can start on it now, partial
// when some of it is here, cold when none of it is, and unknown when the
// runtime could not establish which.
State domain.LocalityState `json:"state"`
LastVerifiedAt time.Time `json:"last_verified_at"`
}
ImageLocality is exact OCI image presence on one node. Every field states what this machine HOLDS. What an image still needs is a subtraction between this and a resolved manifest, and only the control plane holds both: a node that answered it would be restating a manifest it has no way to read.
type LaunchWorkloadCommand ¶
type LaunchWorkloadCommand struct {
RunID string
AttemptID string
BookingID string
Workload domain.WorkloadSpec
// ManifestDigest pins exactly what runs, independent of any tag in the
// workload spec.
ManifestDigest string
Environment []EnvironmentBinding
// CacheMounts is the mutable caches to attach, as the workload declared
// them. Cache identity is the declared name plus compatibility key, shared
// across the deployment.
CacheMounts []domain.CacheMountRequirement
// It carries no Artifact mounts. A field naming the immutable copies to
// attach read-only was declared here, populated by nothing, and read by
// nothing: the Docker runtime builds mount arguments for the caches above and
// for no other content, so a verified replica in a node's replica store is
// not reachable from inside the container a Run executes in. Attaching it,
// and telling a workload which of its inputs are local, is
// https://github.com/benngarcia/mercator/issues/171. A declaration of the
// half that is missing reads like the half that is there.
MaxRuntimeSeconds int64
// contains filtered or unexported fields
}
type NodeBootstrap ¶
type NodeBootstrap struct {
ControlPlaneURL string `json:"control_plane_url"`
NodeID string `json:"node_id"`
RentalID string `json:"rental_id"`
Generation uint64 `json:"generation"`
// EnrollmentToken is short-lived and redeemable once, for this node
// identity only.
EnrollmentToken string `json:"enrollment_token"`
// AgentVersion pins the node agent build the machine should run.
AgentVersion string `json:"agent_version"`
}
NodeBootstrap is the material a fresh machine needs to reach the control plane and prove which Rental generation it is. It never carries a long-lived credential: the enrollment token is short-lived and single-purpose.
type NodeFacts ¶
type NodeFacts struct {
ObservedAt time.Time `json:"observed_at"`
Host HostFacts `json:"host"`
// Images is the exact OCI inventory the node holds.
Images []ImageLocality `json:"images,omitempty"`
// Artifacts is what this node established about the immutable Artifact
// copies on its disk: whether it enumerated them at all, and each copy it
// found worth what checking it against the catalog said it is worth. It is
// the same record the control plane keeps, because a second vocabulary for
// one answer is how the two drift: a node reporting a copy "verified" beside
// a separate "state" would leave the control plane deciding which of the two
// it believes. Object storage remains the authority either way.
//
// Whether anything enumerated is this node's own claim, never a derivation
// from the fact that it answered about its host. A runtime with no replica
// store has not looked for a copy and will never find one, so a control
// plane reading "the node reported, therefore it enumerated" would publish
// "I hold no copy" as a fact on behalf of every machine in the fleet, and a
// hard start bound would then strike those machines out for content nobody
// ever looked for. A node that does not enumerate says nothing here, and
// silence is priced rather than refused.
Artifacts domain.ArtifactInventory `json:"artifacts,omitzero"`
// Caches is the mutable, application-owned state on this node's disk:
// whether it enumerated at all, and each cache it found in the deployment
// that owns it. It is best-effort by construction, because the contents are
// the application's business and nothing here can be checked against
// anything. It is the record the control plane already keeps, for the same
// reason the Artifact inventory is: a second vocabulary for one answer is
// how the two drift.
Caches domain.CacheInventory `json:"caches,omitzero"`
}
NodeFacts is everything the node reports about itself. Each group has one authority: the node observes its own host and inventory, and nothing else does.
func (NodeFacts) Established ¶
Established is this report with every measurement it says it did not make erased. A report is a claim by a machine Mercator does not compile, and the contract lets one arrive whose halves disagree: an agent that carries a previous measurement forward while marking it unestablished, an older build, or a third-party runtime implementing NodeRuntime. Whichever half a reader happens to take then decides whether a placement is promised room nobody established, and this tree has two readers, the offer projection and the fleet listing. The control plane keeps the half the reporter stands behind, once, where the report crosses into it.
type NodeRef ¶
NodeRef names one enrolled node and the generation it is bound to. A ref whose generation no longer matches the Rental is stale by construction.
type NodeRuntime ¶
type NodeRuntime interface {
// NodeSupport reports what this runtime implementation can do. A Docker
// runtime and a future containerd runtime differ here, not in the contract.
NodeSupport() NodeSupport
// Enroll redeems a short-lived enrollment token for an authenticated
// session bound to one immutable node identity and Rental generation.
Enroll(ctx context.Context, request EnrollmentRequest) (Enrollment, error)
// Facts returns the node's latest reported host, accelerator, runtime,
// disk, network, and locality inventory. It is an observation, so callers
// treat its age as material.
Facts(ctx context.Context, ref NodeRef) (NodeFacts, error)
PrepareImage(ctx context.Context, command PrepareImageCommand) (OperationReceipt, error)
PrepareArtifact(ctx context.Context, command PrepareArtifactCommand) (OperationReceipt, error)
LaunchWorkload(ctx context.Context, command LaunchWorkloadCommand) (OperationReceipt, error)
ObserveWorkload(ctx context.Context, ref WorkloadRef) (WorkloadObservation, error)
StopWorkload(ctx context.Context, command StopWorkloadCommand) (OperationReceipt, error)
// Reconcile reports what the node actually holds after either side
// restarted or reconnected: which operations it has already applied and
// which workloads it is still running. It is how the control plane learns
// it must not launch again.
Reconcile(ctx context.Context, ref NodeRef) (Reconciliation, error)
}
NodeRuntime executes successive workloads on capacity Mercator controls through an enrolled agent. It is the only contract that can make capacity reusable: without a node runtime there is no host runtime to hand a second workload to.
Every command carries an operation identity and a fencing token. The runtime promises that one operation identity produces one effect however many times the control plane sends it, and that a command stamped with a superseded fencing token is refused rather than applied late.
type NodeSupport ¶
type NodeSupport struct {
// ContainerRuntime names the implementation ("docker"). It is provenance
// for operators; the control-plane contract does not branch on it.
ContainerRuntime string `json:"container_runtime"`
// ExactImageInventory reports whether the node can enumerate image and
// layer digests it holds, rather than only answering whether a reference
// is present. Without it, image locality is an estimate.
ExactImageInventory bool `json:"exact_image_inventory"`
// ArtifactReplicas reports whether the node stores and verifies immutable
// artifact replicas locally.
ArtifactReplicas bool `json:"artifact_replicas"`
// CacheMounts reports whether the node can hold mutable, named application
// caches across workloads.
CacheMounts bool `json:"cache_mounts"`
// Prewarm reports whether the node accepts preparation work for a workload
// it has not been asked to launch.
Prewarm bool `json:"prewarm"`
// GarbageCollection reports whether the node reclaims disk on its own
// within the bounds the control plane sets.
GarbageCollection bool `json:"garbage_collection"`
// MaxConcurrentWorkloads is how many workloads may execute at once. One
// means the node serializes, which is what makes a Rental Schedule a queue.
MaxConcurrentWorkloads int `json:"max_concurrent_workloads"`
}
NodeSupport is one runtime implementation's negotiated capability set.
type OperationReceipt ¶
type OperationReceipt struct {
OperationID string `json:"operation_id"`
AcceptedAt time.Time `json:"accepted_at"`
Duplicate bool `json:"duplicate"`
}
OperationReceipt acknowledges one node command. Duplicate is how a node says it already applied this operation identity, which is what makes retry after a lost response safe.
type OwnedCapacity ¶
type OwnedCapacity struct {
NativeRef string `json:"native_ref"`
ConnectionID string `json:"connection_id"`
RentalID string `json:"rental_id,omitempty"`
Generation uint64 `json:"generation,omitempty"`
OwnershipToken string `json:"ownership_token,omitempty"`
State CapacityState `json:"state"`
CreatedAt time.Time `json:"created_at"`
}
OwnedCapacity is one machine the provider says belongs to this connection. Reconciliation compares it against known Rentals to find orphans.
type OwnershipQuery ¶
type OwnershipQuery struct {
}
OwnershipQuery scopes an owned-capacity listing.
type PrepareArtifactCommand ¶
type PrepareArtifactCommand struct {
ArtifactID string
ContentDigest string
// Source is the durable object-store location this version lives at. It is
// the catalog's own name for the content, which is what makes it safe to
// write down: nothing can be read with it.
Source string
// SourceCredential is the read the control plane minted for this one fetch,
// which is what the node actually streams from. A node with none has no way
// to reach the object store at all, which is deliberate: the durable
// authority's own credential is never on a machine an operator rents.
SourceCredential domain.ArtifactRead
SizeBytes int64
// contains filtered or unexported fields
}
func (PrepareArtifactCommand) WithoutMaterial ¶
func (command PrepareArtifactCommand) WithoutMaterial() any
WithoutMaterial is this command as the durable record holds it. A presigned GET is a bearer credential written as a URL, so the location goes and the bound stays; Source above is the catalog's own name for the same content and is what a reader of the record uses to say which version this was.
type PrepareImageCommand ¶
type PrepareImageCommand struct {
ManifestDigest string
Platform domain.Platform
// Reference is the registry reference to pull from, pinned to
// ManifestDigest.
Reference string
// RegistryCredential is short-lived material scoped to this pull. It is
// never logged, never persisted on the node, never entered into an event, and
// never written into the control plane's own record of the command.
//
// It is zero for an image any anonymous reader can have, which is a real
// answer rather than a missing one: Mercator holds no account at that
// registry and the node presents none.
RegistryCredential domain.RegistryPull
// Unpack requests the image be made ready to run, not merely fetched.
Unpack bool
// contains filtered or unexported fields
}
func (PrepareImageCommand) WithoutMaterial ¶
func (command PrepareImageCommand) WithoutMaterial() any
WithoutMaterial is this command as the durable record holds it. The secret is dropped and the bound it was minted under is kept, so an operator reading the record can say which pull this machine was authorised to make and until when, and can present nothing.
type ProvisionCommand ¶
type ProvisionCommand struct {
ConnectionID string
OperationKey string
RequestHash string
// RentalID is the identity Mercator assigns before the provider answers,
// so an accepted-but-lost response is still reconcilable.
RentalID string
Generation uint64
OwnershipToken string
OfferSnapshotID string
NativeRef string
Resources domain.ResourceRequirements
// Bootstrap carries what the machine needs to enroll its node agent with
// the control plane. A CapacityProvider delivers it verbatim through
// whatever mechanism it has (startup script, user data, baked image
// configuration) and never interprets it.
Bootstrap NodeBootstrap
// Interruptible requests spot or preemptible capacity. Only honored when
// CapacitySupport.Spot holds.
Interruptible bool
// MaxLifetimeSeconds is a provider-side reclamation backstop for providers
// that support one, so lost capacity cannot bill forever.
MaxLifetimeSeconds int64
}
ProvisionCommand allocates fresh capacity for one Rental.
type Reconciliation ¶
type Reconciliation struct {
NodeID string `json:"node_id"`
// Generation and FencingToken let the control plane detect that the node
// it is talking to is not the one it thought.
Generation uint64 `json:"generation"`
FencingToken uint64 `json:"fencing_token"`
// AppliedOperationIDs is every operation the node has already applied and
// still remembers. Re-sending one of these is safe and returns Duplicate.
AppliedOperationIDs []string `json:"applied_operation_ids,omitempty"`
// Workloads is every workload the node currently knows about, running or
// recently exited but unacknowledged.
Workloads []WorkloadObservation `json:"workloads,omitempty"`
Facts NodeFacts `json:"facts"`
}
Reconciliation is what a node reports after reconnecting: enough for the control plane to decide, without guessing, whether a command it sent took effect.
type SessionRenewal ¶
type SessionRenewal struct {
NodeID string `json:"node_id"`
SessionToken string `json:"session_token"`
SessionExpires time.Time `json:"session_expires"`
FencingToken uint64 `json:"fencing_token"`
}
SessionRenewal is a fresh credential for a session that already exists. It is deliberately not an Enrollment. Enrolling redeems a single-use invitation and moves the fencing token, which supersedes whatever the node was doing; renewing spends nothing, moves nothing, and leaves the machine exactly as it was a moment before.
A node holds one of these because both credentials it ever has are short lived. The invitation is spent the moment it is redeemed, so an agent whose session lapsed has nothing left to present: it renews ahead of the lapse, or it stops being able to speak to the control plane at all while its machine goes on running.
It carries the fencing token so the agent can see that renewing did not move it. A renewal that came back with a different one would mean something superseded this node while it was asking, and the agent's memory of what it has already applied would no longer line up with the control plane's.
type StopWorkloadCommand ¶
type WorkloadObservation ¶
type WorkloadObservation struct {
RunID string `json:"run_id"`
AttemptID string `json:"attempt_id"`
Phase WorkloadPhase `json:"phase"`
// ObservedAt is when the node looked, and StartedAt below is what its runtime
// said about the container. Both are the node's own clock, which is the one
// thing a control plane cannot check: two moments off one foreign clock agree
// with each other whatever that clock reads.
ObservedAt time.Time `json:"observed_at"`
// ReceivedAt is when Mercator accepted this report, on Mercator's own clock. It
// is the only moment here that can be compared with anything the control plane
// knows: a start the node dates an hour ahead is an hour ahead of this, and a
// start latency is this report's start minus an acceptance the control plane
// stamped. It is written where a node's report enters the control plane and
// never by the node, which is why it carries no omitempty: a stored observation
// without one is a report that reached the registry by some other door.
ReceivedAt time.Time `json:"received_at"`
// ExitCode is meaningful only when Phase.Exited() holds.
ExitCode *int `json:"exit_code,omitempty"`
// OOMKilled and FailureReason are resource-level facts the node owns,
// distinct from anything the application reports about itself.
OOMKilled bool `json:"oom_killed,omitempty"`
FailureReason string `json:"failure_reason,omitempty"`
StartedAt *time.Time `json:"started_at,omitempty"`
FinishedAt *time.Time `json:"finished_at,omitempty"`
}
type WorkloadPhase ¶
type WorkloadPhase string
WorkloadPhase is the node's authority: what the container actually did. Application readiness and semantic success are the workload's authority and arrive separately.
const ( WorkloadPhasePreparing WorkloadPhase = "preparing" WorkloadPhaseCreated WorkloadPhase = "created" WorkloadPhaseRunning WorkloadPhase = "running" WorkloadPhaseExited WorkloadPhase = "exited" // WorkloadPhaseAbsent means the node has no record of this workload, which // after a launch command is materially different from an exit. WorkloadPhaseAbsent WorkloadPhase = "absent" )
func (WorkloadPhase) Exited ¶
func (phase WorkloadPhase) Exited() bool
Exited reports whether an exit code carries meaning for this phase.
type WorkloadRef ¶
WorkloadRef names one workload execution on one node.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package capacitytest is the bounded set of promises every CapacityProvider keeps, stated once and run against any backend that implements the contract.
|
Package capacitytest is the bounded set of promises every CapacityProvider keeps, stated once and run against any backend that implements the contract. |