ceph

package
v0.7.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 5, 2026 License: Apache-2.0 Imports: 21 Imported by: 0

Documentation

Overview

Package ceph reaches the RBD pools a site keeps.

billet drives Ceph through the `rbd` COMMAND rather than through librados, and that is a decision rather than an omission. The Go binding is cgo over librados, which would end the single static binary and the cross-build matrix in one move — the same reason `mattn/go-sqlite3` is banned here — and billet already treats Ceph the way it treats Docker and Tart: an external dependency an operator installs. What it costs is that every call is a process, so this package is for operations measured in tens per job, never per block.

The same client owns golden-image roots and cache generations. Cache publication is fenced and pointer-based; a failed commit leaves the old generation readable.

Index

Constants

View Source 
const (

	// CacheLockStaleAfter bounds recovery from a cache-index holder that dies.
	// Ten minutes spans twenty heartbeat intervals and is far past the ordinary
	// metadata critical sections. A holder older than this is still never broken
	// while its heartbeat moves, including a longer eviction pass.
	CacheLockStaleAfter = 10 * time.Minute
)
View Source 
const DefaultTimeout = 15 * time.Second

DefaultTimeout bounds one rbd invocation.

A CLUSTER THAT IS NOT THERE DOES NOT REFUSE, IT WAITS. librados retries a monitor it cannot reach for minutes, so an unreachable cluster with no bound here is `billet check` hanging with no output rather than telling an operator which pool it could not read.

View Source 
const GenerationLayout = "20060102150405"

GenerationLayout is how a published generation encodes when it was built.

UTC BY CONSTRUCTION, because the alternative is a timezone bug waiting for a daylight-saving boundary: `rbd snap ls` prints a local-time string with no offset, so two nodes in different zones would disagree about the age of the same snapshot. billet names its own generations, so the name is the reliable clock.

View Source 
const GenerationPrefix = "g"

GenerationPrefix marks a snapshot as one billet published.

View Source 
const GuestContractKey = "billet.guest_contract"

GuestContractKey records which host/guest protocol a generation speaks.

PER GENERATION because compatibility belongs to the immutable root filesystem, not to the mutable image head or to the billet binary inspecting it. A host upgrade reads this before it drains working services, so it can keep a compatible verified generation without downloading another multi-gigabyte image and can refuse an incompatible one before any runner is restarted against it.

View Source 
const HeartbeatInterval = 30 * time.Second

HeartbeatInterval is how often a holder proves it is still alive.

The counter is written to the lock image's metadata rather than into the lock itself, because an rbd lock's id IS the cookie and cannot be updated: refreshing it would mean removing and re-adding, which opens exactly the window this is meant to close.

View Source 
const HeartbeatObservation = 3 * HeartbeatInterval

HeartbeatObservation is how long a would-be breaker watches before concluding silence.

SEVERAL INTERVALS, because one is a race: a holder that beat just before the first observation and is due to beat just after the second would look silent. This only runs on the break path, which by definition is already looking at a lock older than StaleLockAfter, so ninety seconds more costs nothing and buys certainty.

View Source 
const KernelKey = "billet.kernel"

KernelKey records WHICH KERNEL FILE a generation is paired with.

PER GENERATION, FOR THE SAME REASON THE RUNNER VERSION IS. The kernel and the root filesystem are a matched pair -- a guest booted with a different kernel fails in the middle of somebody's job -- and two generations of one image can want different kernels. An operator pointing one config at both has no other way to know which.

THE FILE NAME, NOT THE VERSION, and that distinction is load-bearing. The reaper decides what to delete by comparing this against the names on disk, so a version matches nothing: files are called `vmlinux-6.1.155-ea1d42638d13`. And a version does not identify a kernel anyway -- two builds can produce the same version from different sources, and reaping on it would remove a kernel a generation is verified against.

View Source 
const LockImageName = ".publish-lock"

LockImageName is the dedicated image the publish lock is taken on.

A DEDICATED IMAGE RATHER THAN A LOCK ON THE GOLDEN IMAGE ITSELF, because mapping an image takes an automatic exclusive-lock on it — measured: the head carries an `auto <id>` locker while mapped — so locking the thing being written collides with the write.

THE SAME NAME THE BUILD SCRIPT USES, and that is the whole point. A Go import and a hand-run `build-guest-image.sh` write the same head image, so a lock either side takes alone excludes nothing. This is the interop surface between them and it is why the name is a constant rather than a parameter.

View Source 
const MaxImageBytes = 1 << 40

MaxImageBytes bounds what will be imported, at 1 TiB.

Present so the megabyte rounding cannot overflow, not because anything near it is plausible: the guest filesystem is four gigabytes.

View Source 
const RunnerVersionKey = "billet.runner_version"

RunnerVersionKey is where a published image records the actions/runner it carries.

ON THE IMAGE BECAUSE THE IMAGE IS THE ONLY THING THAT KNOWS. billet compiles a pinned runner version into its own binary, and that says what a build WOULD install rather than what the running fleet HAS. The two part company the moment a scheduled rebuild takes up a newer release — and the question anyone actually asks is about the fleet: GitHub stops sending jobs to a runner more than thirty days behind a release, and it is the guests that are behind or not, never the binary. The full key is this plus "." plus the generation.

View Source 
const StaleLockAfter = 6 * time.Hour

StaleLockAfter is when a held publish lock stops being believed.

BECAUSE A LEAKED LOCK IS OTHERWISE PERMANENT. An `rbd lock` is NOT a lease: it outlives the process that took it, and nothing reclaims it. A killed build, a systemd timeout or a power loss therefore leaves it held by a process that no longer exists — and since a refusal never breaks a lock, every later publish on every node refuses too, forever. The fleet then stops being rebuilt, and thirty days after a runner release it stops being sent jobs, which is precisely the outage this whole mechanism exists to prevent.

Six hours matches the script's bound, deliberately: the two must agree, or whichever is more patient would refuse a lock the other had already broken and retaken.

View Source 
const Verified = "verified"

Verified is the generation reference a tier can name to mean "the newest one that was proved to work with this node's guest contract".

A WORD RATHER THAN AN IMPLICIT DEFAULT. A bare image name stays refused: choosing a generation for somebody who did not choose one is how a job silently boots something nobody decided on. Naming `@verified` IS the decision — it says "whatever passed, most recently" — and a tier that wants one exact generation still pins it and gets it forever.

View Source 
const VerifiedKey = "billet.verified"

VerifiedKey records that a generation was proved to work. The full key is this plus "." plus the generation.

IN THE CLUSTER, BECAUSE PROMOTION IS A FLEET-WIDE FACT. The alternative is a config file naming a generation, edited on every node and restarted — which is why nothing was ever promoted in practice: the schedule published a verified image every week and the fleet went on booting whatever somebody last typed.

Variables

View Source 
var ErrCloneV1 = errors.New("this cluster would clone snapshots the old way, which makes a cache " +
	"generation undeletable while any job holds a clone of it")

ErrCloneV1 is returned when the cluster would clone a snapshot the old way.

IT IS A REFUSAL RATHER THAN A NOTE, and the reason is when the cost lands. On a clone-v1 cluster a snapshot must be PROTECTED before it can be cloned, and a protected snapshot with a live clone can be neither unprotected nor removed — so a cache generation that any running job holds a clone of is undeletable, and eviction is blocked by ordinary traffic rather than by anything wrong. Nothing in billet clones yet, which is exactly why this is the moment to say so: the fix is one command on an empty cluster, and the same fix after a fleet has been built on it is a full pool and a debugging session that starts nowhere near here.

View Source 
var ErrNoClient = errors.New("the ceph client commands are not on PATH")

ErrNoClient is returned when the ceph client commands are not installed.

View Source 
var ErrNoSuchImage = errors.New("the golden image or its snapshot is not in the image pool")

ErrNoSuchImage is returned when a golden image or its snapshot is not there.

A SENTINEL BECAUSE THE CALLER'S NEXT MOVE DIFFERS. A missing golden image is a deployment that has not published one — an operator action — while a cluster that could not be reached is a transient the node should keep retrying. Both arrive as `(2) No such file or directory` on stderr, so without this the launch path cannot tell "publish an image" from "the monitors are down".

View Source 
var ErrUnclassifiedRelease = errors.New("the cluster's minimum client release is not one billet " +
	"can place relative to mimic")

ErrUnclassifiedRelease is returned when the cluster's answer is not a release billet can place relative to mimic.

`unknown` is the case that matters and it is not hypothetical: it is the zero value of Ceph's release enum, and `osd set-require-min-compat-client unknown` is REFUSED — measured — so it is a value that can only arrive from Ceph itself, never from an operator who chose it. Which is exactly why it must not be read as "some release newer than mimic": it means the cluster has not been told, and a cluster that has not been told defaults to the old clone format.

Functions

func EffectiveCloneFormat 

func EffectiveCloneFormat(minCompatClient, configured string) (int, error)

EffectiveCloneFormat resolves what rbd will ACTUALLY do from the two settings that decide it.

THE FLOOR ALONE IS A PROXY, AND IT IS DEFEATED BY ONE CONFIG KEY. `auto` — the default — means "v2 if the cluster admits mimic or later", which is where the release rule applies. But `rbd_default_clone_format` can be set outright, and then it wins: measured, a cluster whose floor is mimic and whose clone format is forced to 1 refuses to clone an unprotected snapshot with `rbd: clone error: (22) Invalid argument`. Checking the floor and calling it "this cluster clones the new way" would have been a green preflight beside the exact failure the preflight exists to prevent.

func PlanKernelReap 

func PlanKernelReap(
	onDisk []string,
	needed map[string]bool,
	generations, unknown int,
	configured string,
) ([]string, error)

PlanKernelReap reports which kernel files no surviving generation needs.

A KERNEL AND ITS GENERATION ARE A MATCHED PAIR: a guest booted with a different kernel fails in the middle of somebody's job. Every pull installs one, so without this a node accumulates a kernel a week forever and nothing says so -- the disk simply fills, which on this project has already happened once for a different reason.

TAKES WHAT IS NEEDED RATHER THAN COMPUTING IT, so the decision is a pure function of two lists and can be tested without a cluster. The caller reads the needed set out of the generations' metadata.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client runs the ceph client commands against one site's pools.

TWO BINARIES, ONE PACKAGE. `rbd` addresses images and `ceph` answers for the cluster itself, and ceph-common ships both — so a host that has one has the other, and a missing binary is one diagnostic rather than two.

func New 

func New(cfg config.CephConfig, opts ...Option) (*Client, error)

New returns a client for the pools this configuration names.

IT RE-APPLIES config.CheckCeph, because this constructor is exported and cannot assume its configuration came through config.Load. Two of those rules are load bearing HERE and not only there. A pool name is half of the POSITIONAL `pool/image` specs billet builds, where rbd reads a leading dash as an option it does not recognise — measured, and NOT true of `-p`, which consumes whatever token follows it. And an identity of `admin` would hand this process a key that can delete the pools it is only meant to read.

func (*Client) AcquireWriter 

func (c *Client) AcquireWriter(
	ctx context.Context,
	key, holder string,
	ttl time.Duration,
) (storecontract.WriterLease, storecontract.FencingToken, error)

AcquireWriter issues a short-lived cache writer lease and a newer fencing token.

func (*Client) CheckReachable 

func (c *Client) CheckReachable(ctx context.Context) (Report, error)

CheckReachable proves this host can act on its ceph configuration.

THE SAME DISTINCTION checkEC2Credentials MAKES, one backend over. Config validation proves the block is coherent, and coherence is not what an operator running `billet check` is asking: a node whose keyring is missing, whose monitor is unreachable, or whose pool was never created validates perfectly and then fails on the first job of the day, with a librados error that names none of those.

IT IS A READ, and the caller must say so. Listing a pool proves the monitors answered, the keyring authenticated and the pool exists; it proves nothing about permission to CREATE, clone or remove an image, which is what a launch actually does.

func (*Client) Clone 

func (c *Client) Clone(
	ctx context.Context,
	key, generation string,
) (storecontract.Volume, error)

Clone maps a writable clone of a current or explicitly named generation.

func (*Client) CloneRoot 

func (c *Client) CloneRoot(
	ctx context.Context, image, name string, capacity config.ByteSize,
) (string, error)

func (*Client) Create 

func (c *Client) Create(
	ctx context.Context,
	key string,
	sizeBytes int64,
) (storecontract.Volume, error)

Create maps a new unformatted cache volume.

func (*Client) Current 

func (c *Client) Current(ctx context.Context, key string) (string, error)

Current reports the generation named by the cache pointer.

func (*Client) Discard 

func (c *Client) Discard(ctx context.Context, volume storecontract.Volume) error

Discard unmaps and removes a writable cache clone.

func (*Client) DiscardRoot 

func (c *Client) DiscardRoot(ctx context.Context, name string) error

DiscardRoot unmaps every mapping of a per-job clone and removes it.

IDEMPOTENT, because it runs on teardown paths that have already failed once. Neither underlying command is idempotent on its own: unmapping a spec nothing has mapped answers `(22) Invalid argument`, and removing an image that is not there answers `(2) No such file or directory` — both measured, and both mean "already done" here.

EVERY MAPPING, NOT ONE. `rbd device unmap <spec>` on an image mapped twice reports `mapped more than once, unmapping /dev/rbd1 only` and leaves the other in place — so this reads the mapping table and unmaps by DEVICE, which names exactly one. An unmapped device left behind pins the image, and the remove then fails for a reason that names neither.

func (*Client) Evict 

func (c *Client) Evict(ctx context.Context, olderThan time.Duration) error

Evict removes old, unreferenced cache images under the same lock as publication.

func (*Client) GenerationGone 

func (c *Client) GenerationGone(err error) bool

GenerationGone reports whether a CloneRoot failure means the generation is no longer there.

THE STORE ANSWERS QUESTIONS ABOUT ITS OWN ERRORS, because the provider may not import this package -- so a caller that needs to distinguish "that generation was deleted" from "the cluster is unreachable" asks rather than matching a sentinel it would have to import.

func (*Client) GenerationGuestContracts 

func (c *Client) GenerationGuestContracts(
	ctx context.Context,
	image string,
) (map[string]string, error)

GenerationGuestContracts is the recorded guest contract of each generation.

func (*Client) Generations 

func (c *Client) Generations(ctx context.Context, image string) ([]Generation, error)

Generations lists every generation billet published of an image.

func (*Client) GuestContract 

func (c *Client) GuestContract(
	ctx context.Context,
	image, generation string,
) (string, bool, error)

GuestContract reports which host/guest protocol a generation speaks.

func (*Client) Images 

func (c *Client) Images(ctx context.Context) ([]string, error)

Images lists the images in the pool billet publishes to.

NEEDED BECAUSE THE KERNEL DIRECTORY IS SHARED AND THE POOL IS NOT ONE IMAGE. Reaping was scoped to the image named on the command line while the kernels of every image sit in one directory, so reaping one image could delete a kernel another image's generations are paired with -- and the failure lands on the next job that boots the other image, with nothing connecting it to the reap.

func (*Client) ImportGeneration 

func (c *Client) ImportGeneration(
	ctx context.Context,
	image, rawPath, runnerVersion, kernel, guestContract string,
	now time.Time,
) (generation string, err error)

ImportGeneration writes a raw filesystem image into the head image and publishes it as an immutable generation.

THE SAME SEQUENCE THE BUILD SCRIPT PERFORMS, for the same reasons, because a generation published by a pull and one published by a hand-run build have to be indistinguishable afterwards: nothing downstream — promotion, verification, reaping, the clone path — should be able to tell where an image came from.

WHY IT WRITES THROUGH A MAPPED DEVICE RATHER THAN `rbd import`. `rbd import` CREATES an image; it cannot write into one that already exists. Generations are snapshots OF one head image, so a pull that imported to a fresh image would publish a generation of something else, which no tier could name and no clone could descend from.

func (*Client) Kernel 

func (c *Client) Kernel(ctx context.Context, image, generation string) (string, bool, error)

Kernel reports which kernel file a generation is paired with.

func (*Client) KernelFor 

func (c *Client) KernelFor(ctx context.Context, image, generation string) (string, bool, error)

KernelFor reports which kernel file a generation was paired with.

THE NAME THE PROVIDER'S INTERFACE ASKS FOR. Kernel() is the same lookup; this exists so the provider can declare the operation it needs without the store having to rename a method that other callers already use.

func (*Client) MarkVerified 

func (c *Client) MarkVerified(
	ctx context.Context,
	image, expectedContract string,
	at time.Time,
) (err error)

MarkVerified records that a generation booted, registered and ran a container. It excludes reaping and proves the generation still exists with the expected guest contract before publishing it.

func (*Client) Match 

func (c *Client) Match(
	ctx context.Context,
	exact string,
	restorePrefixes []string,
) (string, string, error)

Match finds the exact key or the newest pointer under the first restore prefix.

func (*Client) NeededKernels 

func (c *Client) NeededKernels(
	ctx context.Context,
	image string,
	generations []Generation,
) (map[string]bool, int, error)

NeededKernels reports every kernel file the given generations name, and how many of them name none.

THE UNKNOWN COUNT IS RETURNED SEPARATELY BECAUSE IT CANNOT BE INFERRED FROM THE SET. Two generations sharing a kernel collapse into one entry, so a caller cannot compare len(needed) against the number of generations to find out whether anything is unaccounted for -- and "unaccounted for" is the one fact that decides whether reaping is safe at all. A generation that names no kernel still boots one, and that file on disk is indistinguishable from an orphan.

func (*Client) NewestForContract 

func (c *Client) NewestForContract(
	ctx context.Context,
	image, contract string,
) (Generation, bool, error)

NewestForContract resolves an image to the newest live generation that records one exact host/guest protocol, whether or not that generation is verified yet. Upgrade compatibility checks use it to boot-verify an already imported image instead of requiring a redundant download.

func (*Client) NewestGeneration 

func (c *Client) NewestGeneration(ctx context.Context, image string) (Generation, bool, error)

NewestGeneration reports the most recently built generation of an image.

BY BUILD TIME, NOT BY LIST ORDER. `rbd snap ls` returns snapshots in creation order today, and relying on that would make this answer wrong the first time somebody removes and re-adds one, or the ordering changes — for a question whose wrong answer is "no rebuild is due".

func (*Client) NewestVerified 

func (c *Client) NewestVerified(ctx context.Context, image string) (Generation, bool, error)

NewestVerified resolves an image to the most recent generation that passed, without filtering by guest contract. Operator commands use this to describe the publication history; a node launch uses NewestVerifiedForContract instead.

BY BUILD TIME, NOT BY WHEN IT WAS VERIFIED. Re-verifying an older generation — the obvious thing to do while investigating a bad one — would otherwise promote it over a newer one that has been serving jobs for a week.

NOT FOUND IS NOT AN ERROR here, and the caller must not treat it as one: a fleet whose newest generations have all failed verification has nothing to resolve to, and the honest answer is to say so rather than to fall back to something unproven.

func (*Client) NewestVerifiedForContract 

func (c *Client) NewestVerifiedForContract(
	ctx context.Context,
	image, contract string,
) (Generation, bool, error)

NewestVerifiedForContract resolves an image to the newest verified generation that speaks one exact host/guest protocol.

CONTRACT-RELATIVE SO A ROLLING UPGRADE DOES NOT ADVANCE OLD NODES. A candidate binary may publish a newer verified generation while nodes on the prior binary are still serving jobs. Those nodes must keep resolving @verified to their newest compatible generation rather than booting a guest that will reject their metadata.

func (*Client) PublishCAS 

func (c *Client) PublishCAS(
	ctx context.Context,
	key, expected string,
	candidate storecontract.Candidate,
	lease storecontract.WriterLease,
	fence storecontract.FencingToken,
) error

PublishCAS atomically advances one cache pointer under the cluster-wide index lock.

func (*Client) Reap 

func (c *Client) Reap(
	ctx context.Context,
	image string,
	plan []Reapable,
	retention Retention,
) (removed []string, err error)

Reap removes the generations a plan does not keep, oldest first.

OLDEST FIRST, so that an interrupted reap has removed the least useful things rather than a random half.

func (*Client) RecordVerification 

func (c *Client) RecordVerification(
	ctx context.Context,
	image, generation, kernel, guestContract string,
	paired, allowUnpaired bool,
	at time.Time,
) (err error)

RecordVerification records a generation's kernel pairing and its verification together, under the publish lock, having proved the generation still exists.

THREE THINGS THAT WERE SEPARATE AND HAD TO STOP BEING.

The two writes were unordered against reaping. A reap landing between them leaves a generation VERIFIED BUT UNPAIRED -- every node takes it up through `@verified` and each boots it against its own kernel -- which is the exact state the write order was chosen to prevent. Holding the publish lock is what makes the pair indivisible with respect to the only thing that removes generations.

And neither write proved the generation was still there. Both validate the NAME and nothing else, so a reap completing first left both keys recreated for a snapshot that no longer exists. The existence check happens under the lock, so it cannot go stale between the check and the writes.

AN EMPTY KERNEL IS NOT ALWAYS BENIGN, which is why the caller says which case it is rather than this inferring it. "Already paired" needs no write; "this node's kernel is not one billet manages" means the generation would become @verified with nothing recorded -- and every node resolving that alias then boots it against whatever it happens to be configured with, which is the state this function exists to prevent. The second case is refused unless the caller has been told to allow it.

func (*Client) ReleaseWriter 

func (c *Client) ReleaseWriter(
	ctx context.Context,
	lease storecontract.WriterLease,
	fence storecontract.FencingToken,
) error

ReleaseWriter removes exactly the recorded writer it is handed, under the cache index lock so it cannot interleave with an acquisition.

func (*Client) RenewActive 

func (c *Client) RenewActive(
	ctx context.Context,
	volume storecontract.Volume,
	until time.Time,
) error

RenewActive extends the protection of a mounted cache clone.

func (*Client) ResolveGeneration 

func (c *Client) ResolveGeneration(ctx context.Context, image, guestContract string) (string, error)

CloneRoot clones a golden image's snapshot into the cache pool, maps it, and returns the host device the kernel client gave it.

NO `snap protect`, WHICH IS THE WHOLE POINT OF THE CLONE-V2 PRECONDITION. On a clone-v1 cluster the protect step is mandatory and a protected snapshot with a live clone can be neither unprotected nor removed, so a generation any running job holds would be undeletable. CheckReachable refuses such a cluster, which is what lets this function be three commands instead of five.

IT REMOVES THE CLONE IF THE MAP FAILS. A clone that exists and is not mapped is invisible to everything above — no jail carries its name, so the sweep never looks for it — and it holds pool space until an operator finds it by hand. ResolveGeneration turns a tier's image reference into one exact generation.

THE ONLY PLACE THAT KNOWS WHAT `@verified` MEANS, and it answers with a concrete reference so that everything downstream — the log line naming what a job booted, the clone, the metadata — is about a generation somebody can go and look at. A caller that passed the alias through would leave "which image did this job actually run?" unanswerable after the fact.

The alias is relative to guestContract so a rolling upgrade can publish a new generation without making older nodes boot a guest they cannot speak to.

AN EXPLICIT GENERATION IS RETURNED UNCHANGED, including one that was never verified: pinning is a decision, and second-guessing it would make a pinned tier mean something other than what it says.

func (*Client) RunnerVersion 

func (c *Client) RunnerVersion(ctx context.Context, image string) (string, bool, error)

RunnerVersion reports which actions/runner a specific generation carries.

THE GENERATION, NOT THE HEAD, AND THAT DISTINCTION IS THE WHOLE POINT. Generations are immutable and promotion is deliberate, so a fleet can sit on last month's generation while the head advances every week. Reading the head would report the newest BUILD as though it were the fleet — saying everything is current while the guests that jobs actually boot age past the deadline, which is the outage this is meant to catch, hidden by the thing meant to catch it.

A MISSING KEY IS NOT AN ERROR AND NOT A VERSION. A generation published before billet recorded this, or by hand, has nothing to say — and answering "" lets the caller tell that apart from an answer, which is the difference between "cannot tell" and "here is the number".

func (*Client) SetGuestContract 

func (c *Client) SetGuestContract(ctx context.Context, image, generation, contract string) error

SetGuestContract records which host/guest protocol a generation speaks.

func (*Client) SetKernel 

func (c *Client) SetKernel(ctx context.Context, image, generation, file string) error

SetKernel records which kernel FILE a generation is paired with.

func (*Client) SetRunnerVersion 

func (c *Client) SetRunnerVersion(ctx context.Context, image, generation, version string) error

SetRunnerVersion records which actions/runner a generation carries.

THE COUNTERPART TO RunnerVersion, KEYED THE SAME WAY. A single head-level key would describe the last thing published rather than what any job runs, and an alarm reading it reports the newest image as though it were the fleet — staying green right through the expiry it exists to catch.

WRITTEN AFTER THE SNAPSHOT EXISTS, by every caller. The value describes a generation, so recording it against one that was never published leaves a key nothing will ever read and nothing will ever clean up.

func (*Client) Snapshot 

func (c *Client) Snapshot(
	ctx context.Context,
	volume storecontract.Volume,
) (storecontract.Candidate, error)

Snapshot verifies a quiesced volume and turns it into an immutable candidate.

func (*Client) TakePublishLock 

func (c *Client) TakePublishLock(ctx context.Context, now time.Time) (*PublishLock, error)

TakePublishLock claims the right to write the golden image.

func (*Client) UnmarkVerified 

func (c *Client) UnmarkVerified(ctx context.Context, image string) (err error)

UnmarkVerified withdraws that claim, which is how a fleet is rolled back.

IDEMPOTENT, because the thing an operator does in a hurry should not fail for having already worked.

func (*Client) VerifiedGenerations 

func (c *Client) VerifiedGenerations(ctx context.Context, image string) (map[string]bool, error)

VerifiedGenerations is the set of generations that passed verification.

type Generation 

type Generation struct {
	Name string
	// Built is when billet made it, read from the name rather than from the
	// cluster's own timestamp.
	Built time.Time
}

Generation is one published, immutable version of a golden image.

func ParseGeneration 

func ParseGeneration(name string) (Generation, bool)

ParseGeneration reads the build time out of a generation's name.

A NAME THAT DOES NOT PARSE IS NOT A GENERATION billet made — a snapshot somebody created by hand, or an older convention — and it is reported as such rather than guessed at. Deciding "a rebuild is not due" on a snapshot of unknown age is how a fleet quietly stops being rebuilt.

type Option 

type Option func(*Client)

Option configures a Client.

func WithBinary 

func WithBinary(path string) Option

WithBinary names the rbd executable, skipping the PATH lookup.

func WithCephBinary 

func WithCephBinary(path string) Option

WithCephBinary names the ceph executable, skipping the PATH lookup.

func WithTimeout 

func WithTimeout(d time.Duration) Option

WithTimeout bounds one invocation.

type Pool 

type Pool struct {
	// Name and Purpose are what the config called it and what billet keeps there.
	Name    string
	Purpose string
	// Images is a count rather than a list: this goes on an operator's terminal,
	// and a site with a thousand cache volumes should not print a thousand lines.
	Images int
	// Size and MinSize are the replication the operator chose. Zero means the
	// cluster did not say, which is reported rather than guessed at.
	Size    int
	MinSize int
	// CloneFormat is rbd_default_clone_format AS THIS POOL SEES IT.
	//
	// PER POOL, because it can be set per pool: `rbd config pool set billet-cache
	// rbd_default_clone_format 1` leaves the image pool reporting `auto` while
	// clones in the cache pool fail with `(22) Invalid argument` — measured. Both
	// pools hold clones (root clones in one, cache generations in the other), so
	// reading one and calling it the cluster's answer is the same proxy mistake one
	// level down.
	CloneFormat string
	// CloneFormatSource is where that value came from — `pool` for an override on
	// the pool, `config` for one set cluster-wide — because it decides which
	// command removes it.
	CloneFormatSource string
}

Pool is one pool billet uses, and what the cluster says about it.

type PublishLock 

type PublishLock struct {
	// contains filtered or unexported fields
}

PublishLock is a held cluster-wide publish lock.

func (*PublishLock) Release 

func (l *PublishLock) Release(ctx context.Context) error

Release gives the publish lock back.

NOT AUTOMATIC, AND NOT A LEASE. Nothing reclaims this if the process dies; that is what StaleLockAfter is for. Callers defer it.

type Reapable 

type Reapable struct {
	Generation Generation
	// Verified records the verification state the plan was built from. Reap uses
	// it to distinguish a deliberately expired verified generation from one that
	// became verified after the plan was made.
	Verified bool
	// Contract is the guest contract recorded when the plan was built. A
	// compatibility boot may backfill it before Reap takes the publish lock.
	Contract string
	// Reason is why this generation is being kept, empty when it is not.
	Reason string
}

Reapable is one generation and why it is being removed, or kept.

func PlanReap 

func PlanReap(
	all []Generation,
	verified map[string]bool,
	contracts map[string]string,
	keep Retention,
) []Reapable

PlanReap decides which generations may be removed.

A PLAN RATHER THAN AN ACTION, and the same function answers `--dry-run` and the real thing. A preview computed by different code than the operation is a preview that eventually stops describing it — which for an irreversible command against a cluster is the one property worth guaranteeing.

WHAT IT DOES NOT CONSIDER: whether anything is currently booting a generation. Measured on the cluster — clone v2 removes a snapshot with a live child, returns 0, and the child stays usable — so a running job is never disturbed by this. Retention is about what might still be BOOTED, not about what is in use, which is why there is no liveness check here to get wrong.

type Report 

type Report struct {
	// User is the identity the invocations authenticated as, without the `client.`
	// prefix. It comes from the configuration rather than from the cluster — Ceph
	// does not echo it back — so it says who billet asked AS, not who answered.
	User string
	// MinCompatClient is the oldest client release the cluster admits. It is one
	// half of what decides the clone format; the other half is per pool, in
	// Pool.CloneFormat.
	MinCompatClient string
	// CloneV2 reports whether a snapshot can be cloned WITHOUT being protected —
	// and, the half that matters, removed while a clone of it is still live.
	CloneV2 bool
	// Pools describes each pool billet was pointed at, in the order it names them.
	Pools []Pool
}

Report is what a reachability check learned.

type Retention 

type Retention struct {
	// Keep is how many VERIFIED generations per guest contract to leave behind,
	// newest first.
	//
	// A COUNT RATHER THAN AN AGE, because the reason to keep more than the current
	// one is rollback: the newest may turn out bad after it has been promoted, and
	// `billet images unpromote` needs somewhere to land. How many days ago that
	// candidate was built is not the question — how many candidates there are is.
	Keep int
	// Pinned are generations a tier names explicitly. A config may pin one and
	// expect it to be there forever, and removing it would strand that tier.
	Pinned []string
}

Retention says which generations survive a reap.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.