Documentation
¶
Overview ¶
Package lease implements Berth's lease coordination semantics.
A lease is identified by a Key and persisted as a Record. Each Record tracks the current Holder, the TTL, the time the lease was first acquired, the time it was last renewed, and a monotonic FencingToken. The FencingToken is bumped on each fresh acquisition (after release or expiry) and stable across renewals by the same holder; clients must include it on every state-changing call so the server can reject stale writes from a holder that has lost the lease.
Manager ¶
Manager exposes the three operations that drive coordination:
- Manager.Acquire — create a lease for a new holder, renew an existing hold, or reclaim an expired one.
- Manager.Renew — extend the TTL of a held lease, validating the caller's fencing token.
- Manager.Release — voluntarily relinquish a held lease.
Storage ¶
Store is the persistence interface. Implementations must execute Get, Put, and Delete linearizably per Key. MemStore provides an in-memory implementation suitable for tests and single-process deployments. K8sLeaseStore persists records as coordination.k8s.io/v1 Lease objects in a coordination cluster. The sqlstore subpackage persists records in Postgres, MariaDB/MySQL, or SQLite using ACID database transactions.
TTL ¶
TTL expiry is enforced lazily on each Acquire/Renew. TTLEnforcer runs an optional background scan loop for hygiene and metrics; it does not gate correctness.
Index ¶
- Constants
- Variables
- func ValidateHolder(holder string) error
- func ValidateKey(key Key) error
- type AcquireResult
- type K8sLeaseStore
- type Key
- type Manager
- func (m *Manager) Acquire(ctx context.Context, key Key, holder string, ttl time.Duration) (AcquireResult, error)
- func (m *Manager) Ready(ctx context.Context) error
- func (m *Manager) Release(ctx context.Context, key Key, holder string, token int32) error
- func (m *Manager) Renew(ctx context.Context, key Key, holder string, token int32, ttl time.Duration) (AcquireResult, error)
- func (m *Manager) WithClock(fn func() time.Time) *Manager
- type MemStore
- type Record
- type Store
- type TTLEnforcer
Constants ¶
const MaxHolderBytes = 253
MaxHolderBytes is the maximum decoded holder size for acquisition and renewal. This conservative byte limit fits every supported backend's holder column.
Variables ¶
var ( // ErrNotFound indicates that no record exists for the requested key. ErrNotFound = errors.New("lease: not found") // ErrConflict indicates that the expected version did not match the // current record. The caller should re-read state and retry. ErrConflict = errors.New("lease: conflict") )
Errors returned by Store implementations.
Functions ¶
func ValidateHolder ¶ added in v0.4.1
ValidateHolder validates a live holder without changing its identity. Release deliberately accepts older oversized holders so they can be tombstoned.
func ValidateKey ¶ added in v0.3.1
ValidateKey reports whether key is well-formed. The namespace must be an RFC 1123 DNS label (lowercase alphanumerics and '-', at most 63 characters — notably, no dots), the name an RFC 1123 DNS subdomain, and the dot-joined pair at most 253 characters.
The namespace charset is a safety property, not a style choice: backends that encode the pair into one identifier use '.' as the separator, and a dot-free namespace makes that encoding injective — two distinct keys can never collide into one stored object, so no tenant can address another tenant's lease by crafting an ambiguous key. It also matches how Kubernetes names namespaces, so keys mirrored from cluster resources are always valid.
Types ¶
type AcquireResult ¶
type AcquireResult struct {
// Acquired is true when the caller now holds (or continues to hold) the
// lease. When false, Holder/FencingToken/ExpiresAt describe the entity
// that does hold it.
Acquired bool
// Holder is the identity of the current holder.
Holder string
// FencingToken is the current fencing token. The caller must include
// this token on every subsequent Renew or Release call.
FencingToken int32
// ExpiresAt is the time at which the current TTL elapses.
ExpiresAt time.Time
// AcquiredAt is when the current holder first acquired the lease.
AcquiredAt time.Time
}
AcquireResult reports the outcome of an Manager.Acquire or Manager.Renew call.
type K8sLeaseStore ¶
type K8sLeaseStore struct {
// contains filtered or unexported fields
}
K8sLeaseStore implements Store against coordination.k8s.io/v1.Lease objects in a single namespace of a coordination cluster.
Each Berth lease maps 1:1 to a Lease object; the FencingToken is persisted as Lease.spec.leaseTransitions and the record Version as an annotation. Storage-level CAS is double-guarded: Put checks the version annotation against the caller's expectation and the subsequent UPDATE carries metadata.resourceVersion from the same read, so the kube-apiserver returns 409 Conflict on any interleaved write (translated to ErrConflict).
All Berth leases across all tenants are pooled in a single coordination namespace; the K8s Lease.metadata.name encodes the Berth (tenant-ns, lease-name) pair as "<tenant-ns>.<lease-name>".
func NewK8sLeaseStore ¶
func NewK8sLeaseStore(client kubernetes.Interface, namespace string) (*K8sLeaseStore, error)
NewK8sLeaseStore returns a Store that persists leases into namespace of the cluster reachable via client.
func (*K8sLeaseStore) List ¶
func (s *K8sLeaseStore) List(ctx context.Context) ([]Record, error)
List implements Store. Only Lease objects bearing the Berth managed-by label are returned.
func (*K8sLeaseStore) Ping ¶ added in v0.2.0
func (s *K8sLeaseStore) Ping(ctx context.Context) error
Ping implements Store. It issues a single-item List to confirm the coordination apiserver is reachable without paying for a full lease enumeration (Limit: 1 is constant cost regardless of lease count).
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager orchestrates lease acquisition, renewal, and release using a pluggable Store backend. Manager enforces server-side TTL semantics: a lease is reclaimable as soon as its RenewedAt+TTL has elapsed.
func NewManager ¶
NewManager creates a Manager backed by the given Store. The returned Manager uses time.Now for TTL calculations.
func (*Manager) Acquire ¶
func (m *Manager) Acquire(ctx context.Context, key Key, holder string, ttl time.Duration) (AcquireResult, error)
Acquire attempts to acquire or renew the lease at key for holder. ttl must be positive.
Cases:
- No record exists: a new lease is created; FencingToken is 1.
- Same holder, not expired: lease is renewed in place, FencingToken preserved.
- Existing record is a tombstone or expired: lease is (re)claimed for holder, FencingToken is bumped past the key's high-water mark.
- Held by another, not expired: returns Acquired=false and the existing holder's identity, fencing token, and expiry.
func (*Manager) Ready ¶ added in v0.2.0
Ready reports whether the backing store is reachable via the store's constant-cost Store.Ping. It returns the store's error unmodified so a readiness probe can fail (503) and drain the pod during a store outage, distinct from an always-200 liveness check. Ping (not List) keeps the unauthenticated /readyz route from amplifying into a full backend scan.
func (*Manager) Release ¶
Release voluntarily releases a lease held by holder under fencing token. The record is tombstoned, not deleted: the fencing token is preserved as the key's high-water mark so the next acquisition issues a strictly greater token. The operation is idempotent: releasing an already-released or absent lease, or losing a race with another reclaimer, returns nil. Returns ErrConflict when the lease is currently held by a different identity or token.
func (*Manager) Renew ¶
func (m *Manager) Renew(ctx context.Context, key Key, holder string, token int32, ttl time.Duration) (AcquireResult, error)
Renew extends the TTL of a lease held by holder under fencing token. ttl must be positive. Returns Acquired=false (with no error) if the lease has been lost — different holder, mismatched token, expired, released, or absent. The store write is conditioned on the record version observed here, so a renewal that races a concurrent reclaim can never overwrite it: exactly one of the two writes succeeds.
type MemStore ¶
type MemStore struct {
// contains filtered or unexported fields
}
MemStore is an in-memory Store implementation. It is safe for concurrent use and intended for tests, single-process deployments, and bring-up of the lease layer ahead of a durable backend.
type Record ¶
type Record struct {
// Key uniquely identifies the lease.
Key Key
// Holder identifies the entity currently holding the lease. An empty
// Holder marks a tombstone: a released or garbage-collected lease whose
// record is retained so FencingToken and Version stay a per-key
// high-water mark. No live lease can have an empty Holder ([Manager]
// requires a non-empty holder on every operation), so the marker is
// unambiguous.
Holder string
// TTL is the duration after RenewedAt before the lease expires.
TTL time.Duration
// AcquiredAt is when the current holder first acquired the lease. It is
// preserved across renewals by the same holder.
AcquiredAt time.Time
// RenewedAt is when the holder last renewed (or first acquired) the lease.
RenewedAt time.Time
// FencingToken is a monotonically increasing identifier for the current
// holder. It is bumped on each fresh acquisition (after release or
// expiry) and stable across renewals by the same holder. Because release
// and garbage collection tombstone the record instead of deleting it,
// the token is monotonic across the entire life of the key: a newly
// issued token is strictly greater than every token ever issued for the
// key, so downstream systems can safely reject any token at or below
// the highest they have observed.
//
// The width is deliberately int32, a domain decision rather than a
// backend artifact. A 31-bit positive range allows ~2.1e9 holder
// transitions per key — decades of churn at any realistic failover rate,
// so the ceiling is unreachable in practice — while matching
// coordination.k8s.io/v1.Lease's spec.leaseTransitions field so the k8s
// [Store] backend persists the token without narrowing. [Manager] treats
// math.MaxInt32 as a hard ceiling and refuses to wrap (wrapping would
// reuse a token and break fencing safety) rather than overflow into a
// negative or duplicate value; the sql and mem backends inherit the same
// width and invariant.
FencingToken int32
// Version is the record's write counter and the [Store]'s sole
// compare-and-swap predicate. The store owns it: a create stores 1 and
// every subsequent successful Put stores the expected version plus one,
// ignoring the caller-supplied value. Because it changes on every write
// — renewals included — and no code path hard-deletes a record, a
// version value observed for a key can gate at most one successful
// write, ever. That property is what makes a stale renew lose to a
// concurrent reclaim and a garbage-collection sweep unable to touch a
// record written after its scan.
Version int64
}
Record is the persisted state of a single lease.
func (*Record) Tombstone ¶ added in v0.3.1
Tombstone reports whether the record marks a released or garbage-collected lease rather than a live holder. See Record.Holder.
type Store ¶
type Store interface {
// Ping reports backend reachability with a constant-cost probe (no full
// scan): a nil error means the backend answered. It exists so a readiness
// check can verify the store without the unbounded cost of List, keeping
// the unauthenticated /readyz route from amplifying into an expensive
// backend query. Implementations must respect ctx cancellation.
Ping(ctx context.Context) error
// Get returns the record for key. Returns [ErrNotFound] if absent.
Get(ctx context.Context, key Key) (*Record, error)
// List returns a snapshot of all records, tombstones included. Order is
// not specified.
List(ctx context.Context) ([]Record, error)
// Put atomically writes record. If expectedVersion is 0 the operation
// succeeds only when no record exists for record.Key and stores the
// record with Version 1; otherwise it succeeds only when the current
// record's Version equals expectedVersion, and stores the record with
// Version expectedVersion+1. The caller-supplied record.Version is
// ignored. Returns [ErrConflict] when the predicate does not hold.
Put(ctx context.Context, expectedVersion int64, record *Record) error
}
Store is the persistence interface for lease records. Implementations must guarantee linearizable execution of Get, Put, and List per Key.
Store deliberately has no delete operation. Berth retains every record for the life of the backing store's data: release and TTL garbage collection write tombstones (Record.Holder empty) so Record.FencingToken and Record.Version are never reset or reused for a key. Removing a record out of band (SQL DELETE, kubectl delete lease) forfeits that guarantee for the key.
type TTLEnforcer ¶
type TTLEnforcer struct {
// contains filtered or unexported fields
}
TTLEnforcer garbage-collects expired lease records in a Store. TTL expiry is also enforced lazily on Acquire/Renew, but a key that is never reacquired would otherwise keep its holder, TTL, and timing state in a durable backend forever; this loop tombstones that state. It rewrites only records whose TTL elapsed more than a grace window ago, using the same version CAS as Manager: the write is conditioned on the Record.Version captured during the scan, and versions are never reused for a key, so a sweep can never touch a lease that was written since the scan.
Records are tombstoned (Record.Holder cleared, Record.FencingToken preserved), never deleted — deletion would reset the key's token high-water mark and break downstream fencing (see Record.FencingToken).
The enforcer needs no cross-replica coordination: every API server replica may run its own loop, and the CAS-guarded write makes concurrent sweeps idempotent — the first wins, the rest observe ErrConflict and skip.
func NewTTLEnforcer ¶
func NewTTLEnforcer(store Store, interval, grace time.Duration) *TTLEnforcer
NewTTLEnforcer creates a TTLEnforcer that sweeps store every interval and tombstones records expired longer than grace ago. A non-positive interval or grace falls back to the package default.