mutationlifecycle

package
v0.2.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 15, 2026 License: MIT Imports: 16 Imported by: 0

Documentation

Overview

Package mutationlifecycle holds the obligations both resource families carry when an operation may have changed a database.

The two controllers share Job construction, framing, fingerprints and Leases, and used to state each obligation twice. Using the same components does not preserve the same guarantees: an obligation written twice drifts, and the drift is invisible because each copy reads correctly on its own. The database-realm Lease is where that has already cost something -- one family retried a failed release and the other dropped it -- so it is the first obligation to have one implementation.

Index

Constants

This section is empty.

Variables

View Source
var ErrIncompleteBinding = errors.New("lock binding is incomplete")

ErrIncompleteBinding is returned for a binding that cannot produce a release.

View Source
var ErrMayHaveDispatched = errors.New("a mutating claim that may already have dispatched is never dispatched again")

ErrMayHaveDispatched refuses to dispatch a mutating claim whose Job may already exist. Such a claim is an outcome nobody established, and a second create would start a second executor beside one that may be running SQL.

Functions

func CompleteRelease

func CompleteRelease(
	ctx context.Context,
	locks *targetlock.Locker,
	coordinationNamespace string,
	owner LockOwner,
	persist func(context.Context) error,
) error

CompleteRelease hands the realm back and clears the record only once the release succeeded. A failure leaves the record standing, so the next pass tries again instead of leaving every other claimant to wait out the Lease.

persist writes the cleared record. It is the caller's own status patch because the two families patch differently, and it runs only after the locker returned nil: clearing first would lose the obligation on a crash between the two, which is the whole reason the record exists.

func MayHaveDispatched

func MayHaveDispatched(state DispatchState) bool

MayHaveDispatched reports whether a Job may exist for this claim.

It decides whether a missing Job is created or is an outcome nobody can account for, which is the difference between running a migration twice and asking a person to go and look. Answering it from anything else -- a phase, a condition, whether a Job is there now -- reads correctly and is wrong on exactly the pass where the create committed and the response did not arrive.

func OwedRelease

OwedRelease turns a claim's lock binding into the durable record, refusing one that could not be acted on.

func VerdictFor

func VerdictFor(claim JobClaim) (JobVerdict, JobCause)

VerdictFor decides what a pass does about the Job it looked for.

The order is the order both controllers already used, and it matters: a Job whose UID is not the recorded one is judged before ownership, because a replaced Job may legitimately be owned by this resource and is still not the one the claim dispatched.

Types

type Admission

type Admission interface {
	Resolve(ctx context.Context, template *corev1.PodTemplateSpec) (*operatorv1alpha1.PodAdmissionSnapshot, error)
	Validate(snapshot *operatorv1alpha1.PodAdmissionSnapshot) error
	Digest(template *corev1.PodTemplateSpec) (string, error)
}

Admission resolves and checks a Pod template's admission envelope. It is podintent seen through the reader and options a reconciler holds; this package does not import podintent, because the Job builder that package judges depends on this one.

type Claim

type Claim struct {
	// Type names the operation, for the messages the driver writes.
	Type string
	// Mutating says the claim may change the database.
	Mutating bool
	// HoldsLock says the claim holds the database realm while it runs.
	HoldsLock bool
	// Dispatch is what the claim records about whether its Job may exist.
	Dispatch DispatchState
	// Snapshot is the claim's admission snapshot, and SnapshotRefreshed says
	// it was already resolved again once for a template that moved.
	Snapshot          *operatorv1alpha1.PodAdmissionSnapshot
	SnapshotRefreshed bool
}

Claim is what the driver reads about the claim at each step. A step that writes status changes what the family reports next.

type CreateAnswer added in v0.2.3

type CreateAnswer string

CreateAnswer is what a failed Create says about whether the API server stored the Job.

const (
	// AnswerUnknown: the Job may have been stored and the answer lost. A
	// timeout, a dropped connection, or a server error nothing narrows down.
	AnswerUnknown CreateAnswer = "unknown"
	// AnswerNameTaken: the API server refused the Create because something
	// already holds the name.
	AnswerNameTaken CreateAnswer = "name-taken"
	// AnswerRefused: the API server turned the request away before storing it.
	// Authentication, authorization, admission or validation refused it, or
	// priority and fairness shed it.
	AnswerRefused CreateAnswer = "refused"
)

func ClassifyCreate added in v0.2.3

func ClassifyCreate(err error) CreateAnswer

ClassifyCreate reads a failed Create's error for what it proves.

Only an answer the API server gives before it writes to storage is a refusal. An admission webhook it could not call, with failurePolicy Fail, is one: the 0.2.1 and 0.2.2 restart bursts replaced every manager, which removed every endpoint of the manager's own Job-write webhook while the stopping leader was dispatching. Every other server error, a timeout among them, may come after the write and stays unknown.

type DispatchStage

type DispatchStage string

DispatchStage is the step of the dispatch boundary that failed.

const (
	// StageCreate is the single Create a claim is permitted.
	StageCreate DispatchStage = "create"
	// StageConfirm is the read that says what the Create left behind.
	StageConfirm DispatchStage = "confirm"
	// StageIntent is the check that the Job found is the one described.
	StageIntent DispatchStage = "intent"
)

type DispatchState

type DispatchState struct {
	DispatchStarted bool
	JobUID          string
}

DispatchState is what a claim records about whether its Job may exist.

Either field alone is enough. DispatchStarted is written before the one permitted create and is never cleared for a mutating claim, so it is the statement that a create may have committed even when no response came back. A Job UID is only ever written after a create returned, so a claim carrying one has certainly dispatched. No state holds a UID without DispatchStarted.

type Disposition

type Disposition string

Disposition is what a claim owes when it stops: after something goes wrong with it, or, for DispositionAccounted, when its own result was read.

const (
	// DispositionAccounted retires the claim with the outcome its own result
	// established.
	DispositionAccounted Disposition = "accounted"
	// DispositionUnaccounted retires the claim with an outcome nobody
	// established. A mutating run may already have opened the database, and no
	// later pass can establish that it did not.
	DispositionUnaccounted Disposition = "unaccounted"
	// DispositionRetry dispatches again under a fresh attempt. Re-running a
	// read costs nothing, so a read-only claim takes this wherever it can.
	DispositionRetry Disposition = "retry"
	// DispositionRequeue keeps the claim and comes back. The next pass reads
	// the Job the boundary already recorded and decides from what it finds.
	DispositionRequeue Disposition = "requeue"
	// DispositionDiscard drops the claim. Nothing it did needs accounting for,
	// because a read-only run changed nothing and its result is about inputs
	// that have since moved.
	DispositionDiscard Disposition = "discard"
	// DispositionWithdraw takes back the mark of a claim whose one create the
	// API server refused before storing anything. No Job stands under the name
	// or ever did, so the claim is where a process that stopped between
	// spending its approval and marking leaves it, and the next pass makes the
	// create under the approval already spent.
	DispositionWithdraw Disposition = "withdraw"
)

func DispatchFailure

func DispatchFailure(stage DispatchStage, mutating bool, answer CreateAnswer) Disposition

DispatchFailure reports what a failure at the boundary leaves the claim owing.

A name already taken is, for a read-only claim, another attempt's Job, and retrying under a new name is safe. For a mutating claim it is the most dangerous answer there is: the Job standing there may be one this claim created on a pass whose answer was lost, so retrying would dispatch a second executor beside an executor that may be running SQL. A refused create is the one failure that proves no Job exists, so a mutating claim withdraws its mark instead of retiring.

func HarvestFailure

func HarvestFailure(fault HarvestFault, mutating bool) Disposition

HarvestFailure reports what a claim owes when its terminal Job's evidence cannot settle it.

The asymmetry is the whole of it, and it is the same one the dispatch boundary has. A read-only run changed nothing, so a result that cannot be used costs a second read; the claim is retried, or dropped where its inputs have moved and the answer would be about the wrong question. A mutating run may already have changed the database, and no amount of re-reading establishes that it did not -- so every fault retires it with an outcome nobody established, including the one that looks most like staleness.

type Driver

type Driver struct {
	// contains filtered or unexported fields
}

Driver runs the sequence. Its zero value runs Order.

func (Driver) Dispatch

func (d Driver) Dispatch(ctx context.Context, family Family) (reconcile.Result, error)

Dispatch runs one pass of a claim whose Job does not exist toward its one permitted create.

func (Driver) Swapped

func (d Driver) Swapped(a, b Step) Driver

Swapped returns a driver that runs a and b in each other's place. It exists for the tests that hold each boundary to its reason: a test that passes against the order and against the order with its boundary moved measures nothing about the boundary. Production code runs the zero Driver.

type Family

type Family interface {
	// Claim describes the claim as it stands now.
	Claim() Claim
	// Authorize re-reads the claim's authorizations, and refuses it -- by
	// retiring or failing it -- when any of them moved.
	Authorize(ctx context.Context) Outcome
	// AcquireLease takes or renews the realm Lease under the claim's epoch.
	AcquireLease(ctx context.Context) Outcome
	// Stage writes what the Job reads besides the Job.
	Stage(ctx context.Context) Outcome
	// Build returns the Job the claim names, or refuses the claim.
	Build(ctx context.Context) (*batchv1.Job, Outcome)
	// Admission resolves and checks the envelope of the claim's Pod.
	Admission() Admission
	// SaveSnapshot persists the admission snapshot resolved for the claim.
	SaveSnapshot(ctx context.Context, snapshot *operatorv1alpha1.PodAdmissionSnapshot) (reconcile.Result, error)
	// RefreshSnapshot drops a snapshot whose template moved and records that
	// it was resolved again once.
	RefreshSnapshot(ctx context.Context) (reconcile.Result, error)
	// Refuse ends a claim whose Job cannot be judged against its envelope.
	Refuse(ctx context.Context, refusal Refusal, failure error) (reconcile.Result, error)
	// Consume spends the approval the claim was authorized by, and refuses
	// the claim when that approval no longer holds.
	Consume(ctx context.Context) Outcome
	// Mark persists the dispatch marker, with whatever the family records at
	// the same boundary.
	Mark(ctx context.Context) error
	// Withdraw clears the dispatch marker after the API server refused the
	// claim's one create before storing it.
	Withdraw(ctx context.Context) error
	// Create creates the Job.
	Create(ctx context.Context, job *batchv1.Job) error
	// Confirm reads the created Job back from the API server.
	Confirm(ctx context.Context, job *batchv1.Job) (*batchv1.Job, error)
	// Intent holds a Job read back to the Job the claim builds.
	Intent(actual, expected *batchv1.Job) error
	// Record persists the confirmed Job's UID and reports the start.
	Record(ctx context.Context, job *batchv1.Job) (reconcile.Result, error)
	// Unaccounted retires a mutating claim whose Job may exist as an outcome
	// nobody established; job is the Job in hand, or nil.
	Unaccounted(ctx context.Context, job *batchv1.Job, failure error) (reconcile.Result, error)
	// Retry moves a read-only claim to a fresh attempt.
	Retry(ctx context.Context, failure error) (reconcile.Result, error)
}

Family is one resource family's side of the sequence.

type HarvestFault

type HarvestFault string

HarvestFault names what a terminal Job's evidence turned out to be.

const (
	// FaultInputsChanged means the claim's inputs moved while its Job ran, so
	// the result describes something the resource no longer asks for.
	FaultInputsChanged HarvestFault = "inputs-changed"
	// FaultPodMultiplicity means the Job ran more than one executor Pod, or
	// one whose intent could not be established. One result frame is one Pod's
	// account and cannot speak for the other.
	FaultPodMultiplicity HarvestFault = "pod-multiplicity"
	// FaultUnreadableResult means the run left no result the controller can
	// read, or the Job did not succeed.
	FaultUnreadableResult HarvestFault = "unreadable-result"
)

type JobCause

type JobCause string

JobCause is why the pass reached its verdict. It is separate from the verdict because the two controllers word their evidence for an operator, and collapsing three situations into one message would cost a reader the one fact that tells them where to look.

const (
	// CauseNone: the Job is the claim's own.
	CauseNone JobCause = ""
	// CauseMissing: nothing exists under the reserved name.
	CauseMissing JobCause = "missing"
	// CauseReplaced: something exists under the name and is not the Job the
	// claim recorded.
	CauseReplaced JobCause = "replaced"
	// CauseDisowned: the Job under the name is not owned by exactly this
	// resource.
	CauseDisowned JobCause = "disowned"
)

type JobClaim

type JobClaim struct {
	// Mutating says whether this claim may execute SQL. It is the whole of the
	// asymmetry below.
	Mutating bool
	// DispatchStarted and RecordedJobUID are the claim's own dispatch state.
	DispatchStarted bool
	RecordedJobUID  string
	// Found says whether a Job exists under the reserved name, and the rest
	// describes it.
	Found        bool
	FoundJobUID  string
	OwnedExactly bool
}

JobClaim is what a pass knows when it looks for the Job a claim reserved.

type JobVerdict

type JobVerdict string

JobVerdict is what the pass does next.

const (
	// VerdictDispatch: nothing was created and nothing may have been, so the
	// claim proceeds toward its one permitted create.
	VerdictDispatch JobVerdict = "dispatch"
	// VerdictUnaccounted: a mutating claim whose Job is missing, replaced or
	// disowned. What it did is a question for the database rather than for a
	// retry, and the operator never creates a second one.
	VerdictUnaccounted JobVerdict = "unaccounted"
	// VerdictRetry: the same situation for a read-only claim, where another
	// attempt costs a Job and proves the same thing.
	VerdictRetry JobVerdict = "retry"
	// VerdictAdopt: the Job under the reserved name is this claim's and the
	// claim has not recorded its UID, which is what a create whose response
	// never arrived leaves behind.
	VerdictAdopt JobVerdict = "adopt"
	// VerdictSupervise: the Job is this claim's and the claim names it.
	VerdictSupervise JobVerdict = "supervise"
)

type Lock

type Lock int

Lock says when a claim of an operation type holds the database realm Lease.

const (
	// LockNever: the operation never opens the database, or opens it only to
	// read something no other claimant's work can make wrong.
	LockNever Lock = iota
	// LockAlways: a claim of this type holds the realm from before its Job
	// exists until the claim is retired.
	LockAlways
	// LockWhileProving: a claim of this type holds the realm only while it
	// carries out a proof a pending observation owes, and then under the
	// proof's epoch rather than one of its own.
	LockWhileProving
)

type LockBinding

type LockBinding struct {
	CoordinationDigest   string
	OperationID          string
	LeaseEpoch           string
	LeaseDurationSeconds int32
}

LockBinding is the part of a claim that decides a release. Every field is required, and the completeness check is the point of the type: a release naming no epoch is refused outright by the locker, and one naming no duration cannot reproduce the request that acquired the Lease. A partial binding is not a weaker release -- it is no release at all, and the realm stays claimed until the Lease expires.

type LockOwner

type LockOwner interface {
	// UID identifies the holder the Lease was taken under.
	UID() types.UID
	// OwedLockRelease is the release this resource still owes, or nil.
	OwedLockRelease() *operatorv1alpha1.TargetLockReleaseStatus
	// SetOwedLockRelease records or clears it.
	SetOwedLockRelease(*operatorv1alpha1.TargetLockReleaseStatus)
}

LockOwner is a resource that can owe a database realm back. Both families implement it over their own status field.

type Operation

type Operation[P ~string] struct {
	// Known says the type is one this binary defines.
	Known bool
	// Mutating says a claim of this type may change the database. It is the
	// whole of the asymmetry the lifecycle is built around: a read-only claim
	// that cannot be accounted for is run again, a mutating one never is.
	Mutating bool
	// Lock says when a claim of this type holds the database realm.
	Lock Lock
	// ServesProof says this is the kind of work a pending observation is
	// carried out by: a read of the database, or the plan that read requires.
	ServesProof bool
	// Phase is the phase a claim of this type publishes while it runs.
	Phase P
	// Runner is the operation the runner is told to perform.
	Runner runner.Operation
}

Operation is what the mutation lifecycle knows about one operation type.

The zero value is what every type this binary does not define gets, and it answers no to every question. That is the right answer for a site that refuses on it and the wrong one for a site that acts on its negation: "not mutating" is not "read-only", because a claim of a type a newer operator wrote is neither as far as this binary can tell. ReadOnly exists for that.

func MigrationOperation

MigrationOperation describes one PtahMigration operation type.

func SchemaOperation

SchemaOperation describes one PtahSchema operation type.

func (Operation[P]) HoldsLock

func (o Operation[P]) HoldsLock(proofOutstanding bool) bool

HoldsLock reports whether a claim of this type holds the database realm, given whether a proof is outstanding on the resource that carries it.

func (Operation[P]) ReadOnly

func (o Operation[P]) ReadOnly() bool

ReadOnly reports whether a claim of this type is known not to change the database. It is false for an unknown type, which is not known either way.

type Outcome

type Outcome struct {
	// contains filtered or unexported fields
}

Outcome is what a step says about the rest of the pass.

func Proceed

func Proceed() Outcome

Proceed lets the pass go on to the next step.

func Stop

func Stop(result reconcile.Result, err error) Outcome

Stop ends the pass with this result.

type RealmClaim

type RealmClaim struct {
	Mutating         bool
	ServesProof      bool
	Locked           bool
	ProofOutstanding bool
}

RealmClaim describes the claim being retired and the proof it may be serving.

Mutating says the claim may change the database, which is the only kind that holds the realm in its own right. ServesProof says the claim is the sort a pending observation carries out -- a read of the database, or the plan that read requires. Locked says the claim recorded an epoch of its own.

type RealmOwner

type RealmOwner string

RealmOwner is the record that holds the database realm on a resource's behalf.

const (
	// OwnerNone means the resource holds nothing. A claim that never took the
	// database has nothing to hand back.
	OwnerNone RealmOwner = "none"
	// OwnerClaim means the active claim's own epoch holds it.
	OwnerClaim RealmOwner = "claim"
	// OwnerProof means a pending observation holds it. The claim running
	// beside it is carrying out that proof, so retiring the claim releases
	// nothing: the realm is owed to the proof until the proof is settled.
	OwnerProof RealmOwner = "proof"
)

func RealmHeldBy

func RealmHeldBy(claim RealmClaim) RealmOwner

RealmHeldBy reports which record a resource holds the database through, and therefore what retiring this claim owes.

The proof comes first on purpose. A claim carrying out a proof borrows the realm the proof owns, so releasing on its behalf would hand the database to the next claimant while the proof it was taken for is still owed -- which is the one thing the realm is held for.

type Refusal

type Refusal int

Refusal is why a claim's Job cannot be judged against its admission envelope.

const (
	// RefusalSnapshotInvalid: the persisted snapshot does not validate.
	RefusalSnapshotInvalid Refusal = iota + 1
	// RefusalTemplateUnreadable: the rebuilt Pod template cannot be digested.
	RefusalTemplateUnreadable
	// RefusalTemplateMoved: the rebuilt template differs from a snapshot that
	// was already resolved again for it once.
	RefusalTemplateMoved
	// RefusalSnapshotUnresolvable: the envelope cannot be resolved.
	RefusalSnapshotUnresolvable
)

type Retirement

type Retirement struct {
	// Claim is the claim being retired, as it stood before the write.
	Claim RealmClaim
	// Disposition is why it is going. DispositionDiscard drops a claim whose
	// result, if it has one, is not used: one that never dispatched, or a
	// read-only one. DispositionUnaccounted retires a mutating run whose
	// effect nobody established. DispositionAccounted retires a run whose own
	// result was read and recorded.
	Disposition Disposition
	// RecordHoldsRealm says the family keeps the database realm with the
	// record a mutating run leaves until a reading settles it, so the run
	// hands nothing back. A schema's pending observation does. A migration's
	// run record does not: its realm goes back once nothing the run
	// dispatched can still write, and the reading that settles the record
	// takes no Lease.
	RecordHoldsRealm bool
	// MayStillWrite says something the claim dispatched may still be
	// executing, so its Lease is not handed back. The family must keep enough
	// of the claim to renew that realm while the workload may still write. It
	// matters only for a mutating run the family does not keep the realm for.
	MayStillWrite bool
	// EndsProof says the same write removes the pending observation the claim
	// was carrying out, which releases the realm the proof held.
	EndsProof bool
}

Retirement is a claim at the moment the write that retires it is built.

func (Retirement) Releases

func (r Retirement) Releases() RealmOwner

Releases reports whose epoch the write that retires the claim owes back: the claim's own, the pending observation's, or none.

A claim carrying out a proof never releases the proof's realm on its own account, only when its write ends the proof. A mutating run that may have changed the database leaves the realm with whatever outlives it: the record the family keeps it for, or an executor that may still be writing, whose claim keeps the realm until the workload stops. Everything else that holds the realm in its own right hands it back.

type Step

type Step int

Step is one boundary of the sequence.

const (
	// StepAuthorize re-reads everything the claim was decided from, and
	// retires or fails the claim when any of it moved. It writes nothing
	// otherwise.
	StepAuthorize Step = iota + 1
	// StepLease takes or renews the database realm Lease under the claim's own
	// epoch, for a claim that holds the realm.
	StepLease
	// StepStage writes what the Job reads that is not the Job itself: a schema
	// Apply's plan projection. A family with nothing to stage proceeds.
	StepStage
	// StepSnapshot makes the Pod admission snapshot durable. The pass that
	// writes it ends there, so no Job carries a digest the claim does not hold.
	StepSnapshot
	// StepConsume spends the approval that authorized a mutating claim.
	StepConsume
	// StepMark records that a Job may exist, for a claim that holds the realm.
	StepMark
	// StepCreate is the one create the claim is permitted.
	StepCreate
	// StepConfirm reads the created Job back and holds it to the claim.
	StepConfirm
	// StepRecord persists the confirmed Job's UID.
	StepRecord
)

func Order

func Order() []Step

Order returns the sequence a zero Driver runs.

func (Step) String

func (s Step) String() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL