reconcile

type Builder ¶

type Builder struct {
	// contains filtered or unexported fields
}

Builder is the sole public construction entry point for a Loop. Use New(reconciler).With*(...).Build() to create a Loop.

Design: Builder fields are all unexported; Build() constructs a *Loop with private fields set from the Builder's configuration and returns it. A Loop with no configured Trigger never receives external work and cannot converge — Build() returns an error in that case.

Missing-required validation in Build:

• reconciler must be non-nil (validation.IsNilInterface guard).
• trigger must be provided (missing trigger → Loop never gets work).
• a FencedRepo requires a Leader: fencing is only meaningful with a leadership epoch source; WithFencedRepo without WithLeader would silently bind Epoch 0 (no fencing), so Build rejects it.

All other runtime fail-fasts (typed-nil Leader/FencedRepo, backoff inversion, reconcilerID charset) are preserved in preStartValidate (Loop.Start) — not duplicated in Build.

There is no WithClock option by design: control-plane scheduling (timers, startup probe, lease renew) uses a sealed real-only clock (see Loop.controlPlaneClock); Reconciler implementations source their own domain clock.

There is no WithLogger option by design either: the Loop logs through slog.Default(), which every production entry point seals to the redacting process-global sink (runtime/observability/logging). A per-loop raw logger would bypass that fail-closed redaction. (So the With* set is the faithful superset of every CONSUMER-configurable field — logger and the control-plane clock are deliberately framework-owned, not Builder inputs.)

ref: kubernetes-sigs/controller-runtime pkg/builder/controller.go (TypedBuilder.Build — fields unexported, fluent With* return *Builder, terminal Build validates and constructs the controller).

func New(reconciler Reconciler) *Builder

func (b *Builder) Build() (*Loop, error)

func (b *Builder) WithBackoff(base, max time.Duration) *Builder

func (b *Builder) WithConcurrency(n int) *Builder

func (b *Builder) WithFencedRepo(repo FencedRepository) *Builder

func (b *Builder) WithInterval(d time.Duration) *Builder

func (b *Builder) WithLeader(leader LeaderElector) *Builder

func (b *Builder) WithMetrics(m Metrics) *Builder

func (b *Builder) WithName(name string) *Builder

func (b *Builder) WithReconcilerID(id string) *Builder

func (b *Builder) WithRenewInterval(d time.Duration) *Builder

func (b *Builder) WithTrigger(trigger Trigger) *Builder

func (b *Builder) WithoutDefaultRequeue() *Builder

type FencedRepository ¶

type FencedRepository interface {
	ApplyFenced(ctx context.Context, entityID string, epoch uint64, mutation any) (accepted bool, err error)
}

FencedRepository is the consumer-implemented, epoch-aware persistence seam that closes the cross-replica correctness gap leader election alone cannot (see LeaderElector — "leader election is NOT fencing"). A consumer that performs device writes / command emission from inside Reconcile wires its repository as a FencedRepository on the Loop; the Loop then injects a per-Reconcile epoch-bound FencedWriter into ctx, and that writer is the reconciler's ONLY write surface.

ApplyFenced MUST implement a monotonic-epoch compare-and-swap: the resource row records the highest epoch it has ever seen, and the write is applied only when epoch >= row.last_epoch (and last_epoch advanced to epoch). It returns accepted=false (NOT an error) when epoch is stale (< highest-seen) so a zombie leader's late write is rejected structurally rather than corrupting state. This is Kleppmann (DDIA §8.4) monotonic fencing, NOT the kernel/outbox UUID identity-fencing (which only rejects "not equal to current"): a device write may land after several L1→L2→L3 handoffs and only "older than highest-seen" can reject an out-of-order late write.

A type-assertion failure on mutation is a programmer error (the wrong mutation type was passed to the writer). The implementation SHOULD return a non-nil error (transient or wrapped with reconcile.PermanentError to dead-letter) rather than panic, so the Loop can classify and log it correctly.

The implementation MUST forward ctx to all I/O operations. Ignoring ctx cancellation defeats the lost-lease interrupt (ADR §4.2): a lease loss cancels the ctx so the in-flight Reconcile terminates promptly; an implementation that ignores the ctx will proceed with a write after the lease is gone, which is the split-brain scenario fencing exists to prevent.

mutation is `any` deliberately. The reconciler reaches FencedWriter through a non-generic context value (Reconciler.Reconcile is frozen single-method and takes no extra param), so a generic FencedRepository[M] would have to be boxed to `any` at the ctx seam anyway — the type parameter buys nothing while making the Loop (which mints the writer without knowing M) impossible to keep monomorphic. The consumer's ApplyFenced does the single type-assert at its own write boundary. The fencing guarantee depends on the epoch being unforgeable and the CAS being applied — not on the payload being statically typed.

type FencedWriter ¶

type FencedWriter struct {
	// contains filtered or unexported fields
}

FencedWriter is the reconciler's intended write surface for one Reconcile call. It is a STRUCT with unexported fields and an unexported constructor (newFencedWriter), so a consumer in another package can RECEIVE one (from FencedWriterFrom) but can never COMPOSE one with an attacker-chosen epoch — reconcile.FencedWriter{epoch: 999} is a compile error outside this package. The epoch a FencedWriter carries is therefore always the Loop's live-lease value; forging it is unrepresentable in Go's type system (type-system Hard).

Honest scope (PR-A6 review C3): this seal closes "forge a writer with a chosen epoch", NOT "a consumer cannot emit an unfenced write at all". A consumer's own FencedRepository.ApplyFenced(ctx, id, epoch, mut) takes epoch as a caller param, so a direct call bypasses this writer — that vector is closed DOWNSTREAM by the RECONCILE-FENCED-WRITE-FUNNEL-01 ApplyFenced caller-allowlist (archtest, not the type system). See that archtest's godoc for the full three-vector grade.

func FencedWriterFrom(ctx context.Context) (FencedWriter, bool)

func (w FencedWriter) Epoch() uint64

func (w FencedWriter) Write(ctx context.Context, entityID string, mutation any) error

type LeaderElector ¶

type LeaderElector interface {
	// AcquireLease attempts to become the leader for reconcilerID. On success it
	// returns a LeaseToken whose ExpiresAt is now+LeaseDuration and whose Epoch is
	// the monotonic fencing token (incremented on every holder CHANGE, see
	// LeaseToken.Epoch). On contention (another holder owns a live lease) it
	// returns a non-nil error; the Loop treats any acquire error as fail-closed
	// (it does NOT dispatch) and retries after a backoff.
	AcquireLease(ctx context.Context, reconcilerID string) (LeaseToken, error)
	// ReleaseLease relinquishes the lease for a graceful handoff (the next
	// AcquireLease by any holder takes over immediately rather than waiting for
	// TTL expiry). Idempotent; a release of an already-lost lease is not an error.
	ReleaseLease(ctx context.Context, token LeaseToken) error
	// RenewLease extends the lease's TTL while keeping its Epoch unchanged. It
	// returns ErrReconcileLeaseLost (errors.Is-matchable) when the lease is no
	// longer owned by this holder (expired or taken over) — the Loop cancels its
	// lease-scoped ctx the instant this happens to interrupt the in-flight
	// Reconcile. Other (I/O) errors are transient and also drive a re-acquire.
	RenewLease(ctx context.Context, token LeaseToken) error
}

LeaderElector is the leader-election seam a multi-replica reconcile deployment wires into a Loop. The interface is declared in kernel/reconcile and implemented in adapters (adapters/redis SETNX+EXPIRE, adapters/postgres pg_try_advisory_lock) — the kernel-declares / adapter-implements layering that mirrors outbox.Emitter and persistence.CellTxManager.

CRITICAL — leader election is NOT fencing. client-go's own tools/leaderelection documents: "This implementation does not guarantee that only one client is acting as a leader (a.k.a. fencing)." STW GC pauses, clock skew, and renew/acquire races all leave a residual dual-execution window: an old leader paused mid-Reconcile, its lease expired and taken over, then it wakes and finishes its write. The Loop uses the lease only to NARROW that window (single dispatcher in steady state + lost-lease ctx cancel). Cross-replica correctness is the job of the monotonic LeaseToken.Epoch fed into a FencedWriter (see fenced.go) plus consumer idempotency — never the lease.

A nil LeaderElector on a Loop means single-process mode: the Loop is always the leader, Epoch is 0, and no fencing is applied (the single-replica deployment has no zombie-leader window to fence).

INVARIANT: RECONCILE-LEADER-INTERFACE-FROZEN-01 — the method set is frozen to exactly AcquireLease / ReleaseLease / RenewLease with the signatures below, and LeaseToken's field set is frozen (in particular the monotonic Epoch uint64). A reflect golden assertion in CI rejects any added method or changed field; it is a public-contract change that must be made together with the design ADR.

ref: kubernetes/client-go tools/leaderelection/leaderelection.go (lease/renew model + the explicit "not fencing" disclaimer).

type LeaseToken ¶

type LeaseToken struct {
	// ReconcilerID is the leader-election key (one lease per reconciler identity).
	ReconcilerID string
	// HolderID identifies the replica that holds this lease — minted once per
	// elector instance so two replicas competing for the same ReconcilerID are
	// distinguishable (renew ownership CAS, handoff epoch bump).
	HolderID string
	// Epoch is the monotonic fencing token. The lease store increments it on
	// every holder CHANGE (a successful AcquireLease that takes the lease from
	// free/expired/another-holder) and keeps it UNCHANGED across RenewLease. The
	// Loop binds it to a FencedWriter so a zombie leader's late write — carrying a
	// strictly lower Epoch — is rejected by the resource's write-path CAS. This is
	// monotonic (Kleppmann) fencing: reject ALL epoch < highest-seen, not the
	// outbox UUID identity-fencing (equal-or-reject), because a device write may
	// arrive after several L1→L2→L3 handoffs and only "older than highest" — not
	// "not equal to current" — can reject an out-of-order late write.
	Epoch uint64
	// AcquiredAt is when this holder acquired (or re-acquired) the lease.
	AcquiredAt time.Time
	// ExpiresAt is when the lease lapses absent a successful RenewLease.
	ExpiresAt time.Time
}

LeaseToken is the opaque, value-typed proof of leadership an adapter mints and the Loop threads back into ReleaseLease / RenewLease. It is NOT sealed-construction: adapters in different packages must build it, so its fields are exported. The frozen field set is enforced by RECONCILE-LEADER-INTERFACE-FROZEN-01.

type Loop ¶

type Loop struct {
	// contains filtered or unexported fields
}

Loop is the reconcile scheduling environment: it pulls Requests from a source, dispatches each to the Reconciler across a bounded worker pool (serializing per EntityID), and requeues per the returned Result. Its lifecycle skeleton (Start fast-return + startup probe, owner-ctx derivation, graceful Stop) is the scheduling loop's own control shell; the per-entity worker dispatch and requeue are reconcile-specific.

Construct via the Builder (reconcile.New(r).With*().Build()) in production. All configuration fields are unexported; the Builder is the sole public construction entry point. Loop config fields are private so that &reconcile.Loop{Field: ...} from outside the package is a compile error — the funnel is enforced by the type system (Hard upstream).

Defaulting is handled by a single applyDefaults() called once at the end of preStartValidate(), before the metric preflight, so start() sees all config fields at their final values.

Owner ctx (controller-runtime Runnable.Start semantics): Start receives the long-lived owner ctx and derives the workers' runCtx from it, so assembly shutdown (ownerCancel) drains the Loop even without an explicit Stop.

Start must return promptly (spawn pool + fast probe, then return) — it is usable directly as a cell.LifecycleHook.OnStart; Stop as OnStop.

ref: kubernetes-sigs/controller-runtime pkg/builder/controller.go

func (l *Loop) Start(ownerCtx context.Context) error

func (l *Loop) Stop(ctx context.Context) error

type Metrics ¶

type Metrics struct {
	// Total counts reconcile outcomes, labels {reconciler, result}.
	Total kernelmetrics.CounterVec
	// Duration observes reconcile wall-clock seconds, labels {reconciler}.
	Duration kernelmetrics.HistogramVec
	// InFlight tracks concurrently-running reconciles, labels {reconciler}.
	InFlight kernelmetrics.GaugeVec
	// Leader is 1 when this instance owns the reconcile lease, else 0,
	// labels {reconciler}. In the single-process Loop it is set to 1 at Start;
	// real lease gating is the leader-election PR's concern.
	Leader kernelmetrics.GaugeVec
}

Metrics holds the optional pre-bound instruments the Loop records to. A nil field disables that instrument (the Loop nil-checks before every record). Build via RegisterMetrics at the composition root and inject; the Loop preflights label sets at Start (same construction-time label preflight pattern).

func RegisterMetrics(p kernelmetrics.Provider) (Metrics, error)

type Reconciler ¶

type Reconciler interface {
	Reconcile(ctx context.Context, req Request) (Result, error)
}

Reconciler is the single method a consumer implements to converge ONE entity toward its desired state. The Loop calls it once per scheduled Request.

Contract:

• Return (Result{RequeueAfter: d>0}, nil) to re-observe this entity after d.
• Return (Result{}, nil) (RequeueAfter == 0) to re-observe at the default tick interval.
• Return (Result{}, err) with a transient err to have the Loop back off and retry (Result is ignored when err != nil).
• Return (Result{}, PermanentError(err)) when retrying cannot help; the Loop records a dead-letter metric and stops scheduling this entity until a fresh trigger re-observes it.

Idempotency contract (P1): Reconcile MUST be idempotent. The framework may invoke it multiple times for the same entity without an intervening state change: on periodic resync (Interval tick), on F5 dirty re-run after coalesced duplicate triggers, and on transient-error retry. This is the level-triggered convergence contract — the reconciler observes current state and drives toward desired state, regardless of how many times it is called.

Reconcile MUST honor ctx cancellation (the Loop cancels ctx on shutdown / StopTimeout).

INVARIANT: RECONCILE-INTERFACE-FROZEN-01 — the method set is frozen to exactly Reconcile(context.Context, Request) (Result, error). Adding a method, or changing this signature, trips an exact-set reflect assertion in CI; it is a public-contract change that must be made together with the design ADR.

ref: kubernetes-sigs/controller-runtime pkg/reconcile/reconcile.go (Reconciler)

type Request ¶

type Request struct {
	// EntityID identifies the single entity to reconcile. It is opaque to the
	// Loop and meaningful only to the Reconciler (typically a primary key in the
	// cell's own table).
	//
	// An empty EntityID is the "resync-all" sentinel emitted by TickerTrigger
	// (a context-free interval ticker has no entity to name): a Reconciler wired
	// to a TickerTrigger MUST treat req.EntityID == "" as a "re-observe every
	// entity you own" sweep, fanning out to its own entity set.
	//
	// Bounded-set contract (S1): EntityID MUST come from a bounded set (e.g.
	// primary keys of a cell-local table). The Loop retains a small per-entity
	// backoff counter in memory for every entity that has experienced at least one
	// transient error; the counter is released on success, permanent failure, or
	// Loop restart (mirroring controller-runtime workqueue). An unbounded or
	// high-cardinality EntityID space (e.g. user input strings) would grow this
	// in-memory map without bound. The dirty/processing maps obey the same
	// constraint: they hold at most one entry per in-flight entity and are cleared
	// on completion or Loop restart.
	EntityID string
}

Request is the minimal locator the Loop hands a Reconciler. Unlike controller-runtime's reconcile.Request (which carries a types.NamespacedName), a GoCell entity lives in a single cell-local table and is addressed by one opaque EntityID — there is no namespace dimension.

INVARIANT: RECONCILE-REQUEST-FIELDS-FROZEN-01 — the field set is frozen to exactly { EntityID string }. Adding a field (e.g. a Namespace or Priority) trips an exact-set reflect assertion in CI.

type Result ¶

type Result struct {
	// RequeueAfter, when > 0, asks the Loop to re-observe this entity after the
	// given duration. When 0, the entity is re-observed at the Loop's default
	// tick interval. It is ignored when Reconcile also returns a non-nil error
	// (the error path drives backoff instead).
	RequeueAfter time.Duration
}

Result is the scheduling hint a Reconciler returns to the Loop. Unlike controller-runtime's reconcile.Result { Requeue bool; RequeueAfter Duration }, GoCell carries only RequeueAfter: "requeue immediately / at the default tick" is expressed as RequeueAfter == 0, so the Requeue bool is redundant and dropped. There is no Priority dimension.

INVARIANT: RECONCILE-RESULT-FIELDS-FROZEN-01 — the field set is frozen to exactly { RequeueAfter time.Duration }. Re-introducing Requeue bool or a Priority int (controller-runtime / K8s-scheduler residue) trips an exact-set reflect assertion in CI.

type Trigger ¶

type Trigger interface {
	Start(ctx context.Context, queue chan<- Request) error
}

Trigger is the source of reconcile work: Start launches a producer that feeds Requests into the Loop's work queue until ctx is canceled. It is GoCell's minimal analog of controller-runtime's source.Source — the K8s informer / predicate machinery is dropped (GoCell has no control plane to watch), so the sink is a plain chan<- Request rather than a rate-limiting workqueue (deduplication / backoff live in the Loop, not the Trigger).

Contract:

• Start MUST be non-blocking: spawn the producer goroutine and return. It is invoked once by the Loop (or the Builder, PR-A7) with the Loop's internal queue. (Mirrors source.Source's "Start must be non-blocking".)
• The producer MUST honor ctx: exit promptly on ctx.Done(), and never block a queue send past cancellation (no goroutine leak on shutdown).
• The producer forwards with backpressure — it blocks on a full queue rather than dropping (level-triggered: a coalesced signal is re-observed on the next trigger / requeue, never silently lost).
• Start's error return is for genuine runtime startup failures. The two minimal triggers below validate their configuration at CONSTRUCTION (a misconfigured trigger is a programmer error, surfaced via panic — the clock funnel + panic taxonomy convention), so their Start never fails; the return is kept for source.Source parity and future triggers (e.g. one that dials a broker) that can fail at startup.

The Loop's internal WaitGroup does NOT track the producer goroutine started from a Trigger; it is solely the Trigger's responsibility to exit on ctx.Done(). A Trigger that leaks a goroutine after ctx cancellation cannot be detected by Loop.Stop's drain. The two built-in triggers (TickerTrigger, ChannelTrigger) and the reconciletest.FakeTrigger comply with this contract.

INVARIANT: RECONCILE-TRIGGER-INTERFACE-FROZEN-01 — the method set is frozen to exactly Start(context.Context, chan<- Request) error.

ref: kubernetes-sigs/controller-runtime pkg/source/source.go

func ChannelTrigger(in <-chan Request) Trigger

func TickerTrigger(clk clock.Clock, interval time.Duration) Trigger