store

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 28, 2026 License: AGPL-3.0 Imports: 11 Imported by: 0

Documentation

Overview

Package store is the dispatcher's resource store: the runtime kinds (Event, WorkItem, Task, Run, Agent, Effect, BudgetCounter, Approval, and the internal Lease, Debounce, Session and Delivery records) with get, list, watch, compare-and-swap updates, advisory leased locks, retention and the transcript blobs.

Config kinds live in the catalog snapshot, never here. The in-memory implementation is in store/memory, the file-backed one (memory plus a snapshot and journal) in store/filestore, and P3's shared one (memory plus a log of create-only objects in a bucket) in an object store; store/storetest is the conformance suite every implementation must pass.

Design: design/workstreams/dispatcher-runtime/LLD/store.md.

Index

Constants

View Source
const (
	FieldTaskPhase     = "status.phase"        // Task
	FieldTaskQueue     = "spec.queue"          // Task
	FieldTaskWorkItem  = "metadata.workItem"   // Task
	FieldRunTask       = "spec.task"           // Run
	FieldWorkItemLink  = "spec.links"          // WorkItem: LinkFieldValue(provider, ref), one per link
	FieldEffectIdemKey = "spec.idempotencyKey" // Effect
	FieldEffectTask    = "spec.task"           // Effect
)

Indexed fields, for ListOptions.Field.

View Source
const (
	// SchemaVersion is the schema this release writes.
	SchemaVersion = 1
	// ReadableSchema is the newest schema this release reads without losing data: SchemaVersion
	// or SchemaVersion+1.
	ReadableSchema = 1
)

The schema of the stored kinds: one number for every stored type (the v1alpha1 runtime kinds and the internal kinds), bumped by any change to a stored kind or its fields. A shared log (an object store, P3) carries the writer's SchemaVersion on every change and snapshot, so two releases can run on one log during a deploy without losing writes.

Expand, then write. A change to stored types ships in two releases:

  1. Release N adds the new kind or optional field to the Go types, and bumps ReadableSchema to N+1. It still writes N.
  2. Release N+1 bumps SchemaVersion to N+1.

So during a deploy, and after a rollback from N+1 to N, every instance can read every entry and keep the fields it rewrites. While ReadableSchema is pending (SchemaVersion+1), further changes accumulate into it, until a release that reads it ships. Then N+1 is frozen: the deployed expand release wouldn't know a later change and would drop its fields on rewrite, so a later change waits for SchemaVersion N+1 to ship and opens N+2. Tests can't know what shipped, so this is the release checklist's to check. A store that meets an entry past its ReadableSchema stops writing (ErrReadOnly); one whose newest snapshot is past it refuses to start.

cmd/factory's TestStoredSchema keeps these honest: a golden of every stored kind's JSON fields fails when a stored type changes without a pending ReadableSchema.

Design: P3 §Two releases on one log; store.md §Interface, and §Object store when present.

View Source
const (
	DefaultRetention            = 90 * 24 * time.Hour
	DefaultSweepInterval        = 10 * time.Minute
	DefaultWatchBuffer          = 256
	DefaultHistory              = 10_000
	DefaultTranscriptMaxBytes   = 20 << 20
	DefaultTranscriptTotalBytes = 2 << 30
)

Defaults.

View Source
const EnvironmentLogSuffix = ".environment-log"

EnvironmentLogSuffix ends an environment log's blob name. Run names never end with it.

Variables

View Source
var (
	// ErrNotFound: no object (or transcript) with that key.
	ErrNotFound = errors.New("not found")
	// ErrAlreadyExists: Create on a key that is taken.
	ErrAlreadyExists = errors.New("already exists")
	// ErrConflict: Update with a stale resourceVersion; re-read and retry.
	ErrConflict = errors.New("resourceVersion conflict")
	// ErrInvalid: a malformed request (missing kind or name, config kind, bad selector, cursor,
	// field or version, wrong output type).
	ErrInvalid = errors.New("invalid")
	// ErrWatchExpired: the watch fell behind (buffer full) or SinceVersion is older than the
	// history; re-list and watch again.
	ErrWatchExpired = errors.New("watch expired")
	// ErrLockLost: Renew on a lease that expired or was released.
	ErrLockLost = errors.New("lock lost")
	// ErrTooLarge: a transcript over the per-transcript cap.
	ErrTooLarge = errors.New("too large")
	// ErrReadOnly: the store has stopped writing. A shared log holds a change from a newer
	// release than this one can read (its schema is past ReadableSchema), so a write could
	// drop fields it doesn't know. Reads go on; the serve layer answers writes with 503.
	ErrReadOnly = errors.New("store stopped writing: the shared state has a newer schema than this release reads")
)

Errors. Every error a store returns wraps one of these (test with errors.Is); most come as an *Error carrying the operation and key.

Functions

func CreatedAt

func CreatedAt(obj Object) time.Time

CreatedAt returns obj's creation time (zero for kinds without one).

func DecodeCursor

func DecodeCursor(kind, s string) (string, error)

DecodeCursor returns the name after which kind's list continues ("" for an empty cursor). ErrInvalid for a malformed cursor or one from another kind's list.

func DeleteIfUnchanged added in v0.2.0

func DeleteIfUnchanged(ctx context.Context, s Store, k Key, resourceVersion string) error

DeleteIfUnchanged deletes k at resourceVersion when s is an UnchangedDeleter, and unconditionally otherwise.

func EncodeCursor

func EncodeCursor(kind, name string) string

EncodeCursor returns the opaque cursor that continues kind's list after name.

func EnvironmentLogBlob

func EnvironmentLogBlob(run string) string

EnvironmentLogBlob is the TranscriptStore name of run's environment log: it is kept beside the transcript, under the same caps and retention, and deleted with the Run.

func Expired

func Expired(r Retention, created, updated, now time.Time, retention time.Duration) bool

Expired reports whether an object under rule r, created and last written at the given times, is past retention at now.

func FieldValues

func FieldValues(obj Object) map[string][]string

FieldValues returns obj's indexed field values (several for a multi-valued field such as WorkItem links). Kinds without indexed fields return nil.

func FormatVersion

func FormatVersion(v uint64) string

FormatVersion renders a resourceVersion: the store's global monotonic counter, in decimal.

func HasResourceVersion

func HasResourceVersion(obj Object) bool

HasResourceVersion reports whether obj's kind carries a resourceVersion, i.e. whether Update is compare-and-swap for it.

func IndexedFields

func IndexedFields(kind string) []string

IndexedFields lists the fields kind can be listed by.

func Labels

func Labels(obj Object) map[string]string

Labels returns obj's labels (nil for kinds without labels). Don't modify them.

func LinkFieldValue

func LinkFieldValue(provider, ref string) string

LinkFieldValue is the FieldWorkItemLink value of the link {provider, ref}. Provider names have no "/", so the first "/" separates the two.

func ParseVersion

func ParseVersion(op string, k Key, s string) (uint64, error)

ParseVersion parses a resourceVersion. ErrInvalid when it isn't one.

func ResourceVersion

func ResourceVersion(obj Object) string

ResourceVersion returns obj's resourceVersion ("" for kinds without one).

func RunSweeper

func RunSweeper(ctx context.Context, s Sweeper, interval time.Duration, log *slog.Logger)

RunSweeper calls s.Sweep every interval (DefaultSweepInterval when 0) until ctx ends. Errors are logged, not returned: the next tick tries again.

func SetCreatedAt

func SetCreatedAt(obj Object, t time.Time) bool

SetCreatedAt sets obj's creation time; false for kinds without one.

func SetResourceVersion

func SetResourceVersion(obj Object, rv string) bool

SetResourceVersion sets obj's resourceVersion; false for kinds without one.

func SyncIfSupported added in v0.2.0

func SyncIfSupported(ctx context.Context, s Store) error

SyncIfSupported syncs s if it is a Syncer, and is a no-op for a store that has nothing to sync (memory, the file store).

func ValidateFields

func ValidateFields(kind string, fields map[string]string) error

ValidateFields checks that every field in fields is indexed for kind.

Types

type Error

type Error struct {
	// Op is the operation: get, list, create, update, delete, watch, lock, renew, transcript,
	// sweep, sync.
	Op string
	// Key is the object, when there is one (Kind only for list and watch; Name only for locks
	// and transcripts).
	Key Key
	// Detail says more, when there is more to say.
	Detail string
	Err    error
}

Error is a store error on one operation.

func Errorf

func Errorf(op string, k Key, err error, format string, args ...any) *Error

Errorf returns an *Error for op on k wrapping err, with a formatted detail.

func (*Error) Error

func (e *Error) Error() string

func (*Error) Unwrap

func (e *Error) Unwrap() error

type Key

type Key struct {
	Kind, Name string
}

Key names one stored object.

func CheckObject

func CheckObject(op string, obj Object) (Key, error)

CheckObject validates obj for a write by op and returns its key: a non-nil pointer with a kind (not a config kind) and a name.

func KeyOf

func KeyOf(obj Object) Key

KeyOf returns obj's key.

func (Key) String

func (k Key) String() string

type ListMeta

type ListMeta struct {
	// NextCursor continues the list; empty on the last page.
	NextCursor string
	// ResourceVersion is the store's version when the list was read: pass it as
	// WatchOptions.SinceVersion to watch from exactly where the list left off (resync).
	ResourceVersion string
}

ListMeta describes a List result.

type ListOptions

type ListOptions struct {
	// LabelSelector keeps the objects whose labels match; nil or empty matches all.
	LabelSelector *v1alpha1.LabelSelector
	// Field keeps the objects whose indexed field has the value (all entries must match). The
	// fields are per kind: see IndexedFields. An unknown field is ErrInvalid.
	Field map[string]string
	// Cursor is a previous page's ListMeta.NextCursor; empty starts at the beginning.
	Cursor string
	// Limit caps the page size; 0 means no limit.
	Limit int
}

ListOptions select and page a List.

type MetaObject

type MetaObject interface {
	Object
	GetObjectMeta() *v1alpha1.ObjectMeta
}

MetaObject is implemented by stored kinds outside v1alpha1 (the internal Lease, Debounce, Session and Delivery records) to give the store their ObjectMeta.

type Object

type Object = v1alpha1.Object

Object is any stored kind. The store reads its kind from TypeMeta and its name from GetName (metadata.name, or the id for Task and Event); ResourceVersion, CreatedAt and Labels reach its metadata (see meta.go).

type Options

type Options struct {
	// Clock sets CreatedAt and drives lock expiry and retention. Default clock.Real.
	Clock clock.Clock
	// Retention is how long Runs, transcripts, Events, Effects and BudgetCounters are kept
	// (and terminal Tasks after their last change). Default 90 days (`--retention`).
	Retention time.Duration
	// WatchBuffer is the default per-watcher buffer. Default 256.
	WatchBuffer int
	// History is how many changes are kept for WatchOptions.SinceVersion. Default 10 000.
	History int
	// TranscriptMaxBytes caps one transcript. Default 20 MB.
	TranscriptMaxBytes int64
	// TranscriptTotalBytes caps all transcripts. Default 2 GB.
	TranscriptTotalBytes int64
	// Logger logs evictions and sweeps. Default slog.Default().
	Logger *slog.Logger
}

Options configure a store implementation. Zero fields take the defaults.

func (Options) WithDefaults

func (o Options) WithDefaults() Options

WithDefaults returns o with its zero fields set to the defaults.

type Retention

type Retention int

Retention says how an object ages out.

const (
	// Keep: never swept (WorkItem, Agent, Approval, internal kinds).
	Keep Retention = iota
	// SinceCreated: swept Options.Retention after creation (Run, Event, Effect, BudgetCounter;
	// transcripts likewise).
	SinceCreated
	// SinceUpdated: swept Options.Retention after its last write (a Task in a terminal phase,
	// Succeeded, Failed or Cancelled: its last write is when it got there).
	SinceUpdated
)

Retention rules (LLD store.md §Retention).

func RetentionOf

func RetentionOf(obj Object) Retention

RetentionOf returns obj's retention rule. It depends on obj's state (a Task's phase), so a store evaluates it on every write.

type Store

type Store interface {
	// Get decodes the object at k into out, which must be a pointer to the kind's Go type.
	// ErrNotFound when there is none.
	Get(ctx context.Context, k Key, out Object) error
	// List decodes the objects of kind that match opts into out, a *[]T where T is the kind's
	// Go type, a pointer to it, or an interface it implements (Object). Objects come in name
	// order; ListMeta.NextCursor continues the list when opts.Limit cut it short.
	List(ctx context.Context, kind string, opts ListOptions, out any) (ListMeta, error)
	// Create stores a new object. It sets obj's ResourceVersion and CreatedAt (from the
	// store's clock) on the stored copy and on obj. ErrAlreadyExists when the key is taken.
	Create(ctx context.Context, obj Object) error
	// Update replaces an object if obj's ResourceVersion is the stored one (compare-and-swap),
	// else ErrConflict; ErrNotFound when there is none. It sets the new ResourceVersion on obj
	// and keeps the stored CreatedAt. Kinds without a resourceVersion field (Event) are
	// replaced unconditionally.
	Update(ctx context.Context, obj Object) error
	// Delete removes an object. ErrNotFound when there is none.
	Delete(ctx context.Context, k Key) error
	// Watch streams the changes to kind's objects that match opts, in version order.
	Watch(ctx context.Context, kind string, opts WatchOptions) (Watcher, error)
	// Lock takes the advisory lock name for ttl, waiting while another holder has it and it
	// hasn't expired. It returns ctx's error (wrapped) if ctx ends first. An expired lock is
	// reclaimable by anyone.
	Lock(ctx context.Context, name string, ttl time.Duration) (Unlock, error)
}

Store is the resource store.

type SweepResult

type SweepResult struct {
	// Objects is per kind.
	Objects     map[string]int
	Transcripts int
}

SweepResult counts what a sweep deleted.

type Sweeper

type Sweeper interface {
	Sweep(ctx context.Context) (SweepResult, error)
}

Sweeper deletes what is past retention (see Retention). Deletions are sent to watchers.

type Syncer added in v0.2.0

type Syncer interface {
	Sync(ctx context.Context) error
}

Syncer is a store that shares its state with other processes (an object store, P3): it may lag the shared state by up to its sync interval. Sync brings it up to the shared state's latest change. A caller whose decision rests on an empty answer (a lookup that found nothing, after which it creates or drops something) syncs once and asks again before acting: SyncIfSupported.

type TranscriptStore

type TranscriptStore interface {
	// PutTranscript stores (or replaces) run's transcript. ErrTooLarge over the per-transcript
	// cap.
	PutTranscript(ctx context.Context, run string, blob []byte) error
	// GetTranscript returns a copy of run's transcript. ErrNotFound when there is none.
	GetTranscript(ctx context.Context, run string) ([]byte, error)
}

TranscriptStore holds run transcripts as the compressed blobs the agent uploads (gzip JSON Lines), keyed by run name. Blobs are capped per transcript and in total; when the total cap is reached the oldest are evicted (logged). They are deleted with their Run and swept by age like Runs. A run's environment log is a second blob, named EnvironmentLogBlob(run).

type UnchangedDeleter added in v0.2.0

type UnchangedDeleter interface {
	DeleteIfUnchanged(ctx context.Context, k Key, resourceVersion string) error
}

UnchangedDeleter is a store that deletes an object only at the resourceVersion its caller read: ErrConflict when the object changed since (it was replaced, or another writer updated it), ErrNotFound when it is gone. The memory store, the file store and the object store implement it; the Store interface doesn't change. A caller whose delete must not remove an object recreated under the same name (a redelivered Delivery, P3 §Several dispatchers) uses DeleteIfUnchanged.

type Unlock

type Unlock interface {
	// Unlock releases the lock if this holder still has it. It is idempotent, and never
	// releases a lock someone else reclaimed after expiry.
	Unlock()
	// Renew extends the lease to now+ttl. ErrLockLost when the lease expired or was released.
	Renew(ctx context.Context, ttl time.Duration) error
}

Unlock is a held lock (the LLD's Unlock, with renewal).

type WatchEvent

type WatchEvent struct {
	Type    WatchEventType
	Object  Object
	Version string
}

WatchEvent is one change. Object is the watcher's own copy: the new state, or the last state for Deleted. Version is the change's resourceVersion (also set on Object when its kind has one).

type WatchEventType

type WatchEventType string

WatchEventType is what happened to an object.

const (
	Added    WatchEventType = "added"
	Modified WatchEventType = "modified"
	Deleted  WatchEventType = "deleted"
)

Watch event types.

type WatchOptions

type WatchOptions struct {
	// LabelSelector keeps the objects whose labels match. An object that stops matching is
	// sent as Deleted, one that starts matching as Added.
	LabelSelector *v1alpha1.LabelSelector
	// SinceVersion replays the changes after this version before the live ones. Empty starts
	// with the next change. A version older than the store's history is ErrWatchExpired.
	SinceVersion string
	// Buffer is the channel buffer; 0 is the store's default. A watcher whose buffer is full
	// when a change is sent is expired (its channel closed, Err() == ErrWatchExpired): writers
	// never wait for readers.
	Buffer int
}

WatchOptions select a Watch.

type Watcher

type Watcher interface {
	// Events is closed when the watch ends (ctx done, Stop, or expiry).
	Events() <-chan WatchEvent
	// Err is, once Events is closed: nil after Stop, the context's error, or ErrWatchExpired
	// (the consumer re-lists and starts a new watch).
	Err() error
	// Stop ends the watch. It is idempotent.
	Stop()
}

Watcher is a running watch.

Directories

Path Synopsis
Package filestore is the single-node durable store: the in-memory store (store/memory) is the engine, and every committed change is appended to an fsync'd JSON Lines journal before it is applied.
Package filestore is the single-node durable store: the in-memory store (store/memory) is the engine, and every committed change is appended to an fsync'd JSON Lines journal before it is applied.
Package journal is the encoding the durable stores share: P2's journal line (one committed change) and snapshot (the whole store at a version), the kinds they can load and the transcript blob names.
Package journal is the encoding the durable stores share: P2's journal line (one committed change) and snapshot (the whole store at a version), the kinds they can load and the transcript blob names.
Package memory is the in-memory store.Store: one map per kind behind a single RWMutex (PoC scale: thousands of objects), JSON deep copies on the way in and out, field indexes, watch fan-out with bounded buffers and a replay history, leased locks, retention and the transcript blobs.
Package memory is the in-memory store.Store: one map per kind behind a single RWMutex (PoC scale: thousands of objects), JSON deep copies on the way in and out, field indexes, watch fan-out with bounded buffers and a replay history, leased locks, retention and the transcript blobs.
Package objectstore is the store several dispatchers share: the in-memory store (store/memory) is the engine in each of them, and the bucket holds the authoritative history, an append-only log of create-only objects, one per change, plus periodic snapshots.
Package objectstore is the store several dispatchers share: the in-memory store (store/memory) is the engine in each of them, and the bucket holds the authoritative history, an append-only log of create-only objects, one per change, plus periodic snapshots.
Package storeaudit keeps secrets out of the durable stores' bytes.
Package storeaudit keeps secrets out of the durable stores' bytes.
Package storetest is the conformance suite for store implementations: the in-memory store passes it today and a Postgres store must pass it later.
Package storetest is the conformance suite for store implementations: the in-memory store passes it today and a Postgres store must pass it later.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL