store

package
v1.0.5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 14, 2026 License: MIT Imports: 22 Imported by: 0

Documentation

Overview

Package store is the daemon's durable desired-state record: one encrypted row per instance, written on Apply and removed on Delete. It is opt-in; when no store is wired the daemon is a stateless proxy as before.

Index

Constants

View Source
const (
	DefaultJobLimit = 100
	MaxJobLimit     = 1000
)

Job listing page-size bounds.

Variables

View Source
var ErrNotFound = errors.New("store: not found")

ErrNotFound is returned when no row matches a lookup (specs, jobs, or host secrets).

View Source
var ErrSecretsNeedKey = errors.New("secrets require an encryption key (-spec-key-file)")

ErrSecretsNeedKey is returned when a secret operation is attempted on a store that was opened without an encryption key (-spec-key-file).

View Source
var ErrSecretsUndecryptable = errors.New("store: secrets undecryptable (wrong or missing -spec-key-file)")

ErrSecretsUndecryptable marks a spec whose sealed secrets blob will not open under the loaded key: the daemon was started with the WRONG -spec-key-file (or, rarer, the ciphertext is corrupt — the two are indistinguishable at the GCM layer). Unlike ErrSpecCorrupt (permanently malformed plaintext) this is recoverable: a restart with the correct key file makes the row readable again, so callers (boot reconciliation) keep retrying rather than failing terminally.

View Source
var ErrSpecCorrupt = errors.New("store: spec row corrupt (malformed)")

ErrSpecCorrupt marks a permanently unreadable spec row: a JSON column is malformed, or the secrets blob decrypts cleanly but its plaintext is not valid JSON. It is distinct from transient store errors (context cancellation, SQLITE_BUSY) and from the definitive ErrNotFound, so callers (e.g. boot reconciliation) can stop retrying a row that will never become readable. A decrypt failure (wrong/missing key) is NOT covered here — that recoverable case is ErrSecretsUndecryptable.

Functions

func LoadKeyFile

func LoadKeyFile(path string) ([32]byte, error)

LoadKeyFile reads a 32-byte encryption key from path. The file may contain either the 32 raw bytes, or the base64 (std) encoding of 32 bytes. Trailing whitespace/newlines are ignored. Anything else is an error.

func NewBackupID

func NewBackupID() string

NewBackupID returns a sortable backup id: "bk_" + the jobs id scheme (time-prefixed hex + random suffix).

Types

type Backup

type Backup struct {
	ID       string
	Host     string
	Template string
	Slug     string
	State    BackupState
	Volumes  []BackupVolume
	Image    string // image ref at backup time; informational hint only
	Created  time.Time
	Finished time.Time // zero until complete/failed
}

Backup is one row of the backups table.

type BackupState

type BackupState string

BackupState is the lifecycle state of a backup row.

const (
	BackupCreating BackupState = "creating" // job in flight; not restorable
	BackupComplete BackupState = "complete" // all volumes exported + manifests recorded
	BackupFailed   BackupState = "failed"   // job failed or was interrupted; not restorable
)

type BackupStore

type BackupStore interface {
	// CreateBackup inserts a new row in state creating, stamping Created.
	// The caller supplies the ID (NewBackupID).
	CreateBackup(ctx context.Context, b Backup) error
	// CompleteBackup transitions creating → complete, recording the exported
	// volumes and Finished. CAS: returns false (no error) if the row is not
	// currently creating.
	CompleteBackup(ctx context.Context, id string, vols []BackupVolume) (bool, error)
	// FailBackup transitions creating → failed, setting Finished. CAS like
	// CompleteBackup.
	FailBackup(ctx context.Context, id string) (bool, error)
	GetBackup(ctx context.Context, id string) (Backup, error) // ErrNotFound when absent
	// ListBackups returns the instance's backups newest-first. limit <= 0 uses
	// DefaultJobLimit; clamped at MaxJobLimit.
	ListBackups(ctx context.Context, host, template, slug string, limit int) ([]Backup, error)
	// DeleteBackup removes the row; ErrNotFound when absent. Blob deletion is
	// the caller's job (instance.Service.DeleteBackup) — the store only holds
	// metadata.
	DeleteBackup(ctx context.Context, id string) error
}

BackupStore persists backup metadata. Implemented by *SQLite and *Memory.

type BackupVolume

type BackupVolume struct {
	Name      string          `json:"name"`
	SizeBytes int64           `json:"size_bytes"`
	Manifest  json.RawMessage `json:"manifest"`
}

BackupVolume records one exported volume: its full name (<template>-<slug>-<vol>), the tar's byte size, and the sha256 per-file manifest (the instance package's Manifest, serialized). Manifests live in the row — not the blob store — so restore verifies the artifact against metadata it does not have to trust.

type DB

DB is the full backend: spec store + job store + template store + backup store + closer. main holds one of these; instance.Service takes the Store view, the runner takes the JobStore view.

type Job

type Job struct {
	ID       string
	Kind     string
	Args     json.RawMessage // opaque to the store; handlers unmarshal their own shape
	State    JobState
	Steps    []JobStep
	ParentID string // "" if none
	Error    string
	Created  time.Time
	Started  time.Time // zero until claimed
	Finished time.Time // zero until done
}

Job is one row of the jobs table.

type JobFilter

type JobFilter struct {
	State    JobState
	Kind     string
	ParentID string
	Limit    int    // <=0 → DefaultJobLimit; values above MaxJobLimit are clamped
	Before   string // cursor: return jobs with id < Before
}

JobFilter narrows ListJobs. Empty fields match anything. Limit/Before paginate the result (ordered newest-first); Before is the id of the previous page's last row (a cursor), returning only rows older than it.

type JobState

type JobState string

JobState is the lifecycle state of a job row.

const (
	JobQueued      JobState = "queued"
	JobRunning     JobState = "running"
	JobReconciling JobState = "reconciling"
	JobSucceeded   JobState = "succeeded"
	JobFailed      JobState = "failed"
	JobCanceled    JobState = "canceled"
)

func (JobState) Active

func (s JobState) Active() bool

Active reports whether the job is in a non-terminal state — queued, running, or reconciling — i.e. work that may still mutate hosts. Guards that must not run concurrently with a migrate-class job should use this so a new non-terminal state cannot silently slip past them.

func (JobState) Terminal

func (s JobState) Terminal() bool

Terminal reports whether the job has reached a final state (succeeded, failed, or canceled).

type JobStep

type JobStep struct {
	TS     time.Time `json:"ts"`
	Step   string    `json:"step"`
	Detail string    `json:"detail,omitempty"`
	// Count is the total number of consecutive identical occurrences of this
	// step, materialized only when coalesced (>1). 0/omitted ⇒ a single
	// occurrence. AppendStep collapses consecutive identical (Step, Detail)
	// rows so a long-looping reconcile can't grow the array unboundedly. (#117)
	Count int `json:"count,omitempty"`
}

JobStep is one progress entry recorded by a handler.

type JobStore

type JobStore interface {
	// Enqueue inserts a new queued job, generating its ID. parentID is "" for
	// top-level jobs.
	Enqueue(ctx context.Context, kind string, args json.RawMessage, parentID string) (Job, error)
	// StartChild inserts a child job already in the running state (never queued),
	// owned by parentID. Because it is not queued, ClaimNext never claims it — the
	// caller (a parent job handler) drives it directly.
	StartChild(ctx context.Context, kind string, args json.RawMessage, parentID string) (Job, error)
	GetJob(ctx context.Context, id string) (Job, error) // ErrNotFound when absent
	ListJobs(ctx context.Context, f JobFilter) ([]Job, error)
	// ClaimNext atomically transitions the oldest queued job to running and
	// returns it. ok=false when there is nothing to claim.
	ClaimNext(ctx context.Context) (job Job, ok bool, err error)
	AppendStep(ctx context.Context, id string, step JobStep) error
	// Finish sets the terminal state, finished timestamp, and error (empty for
	// success). state must be JobSucceeded or JobFailed; passing any other value
	// is a programming error.
	Finish(ctx context.Context, id string, state JobState, errMsg string) error
	// FailRunning marks every job still in running as failed with reason; returns
	// the count. Called once at startup to reap crash-interrupted jobs.
	FailRunning(ctx context.Context, reason string) (int, error)
	// MarkReconciling moves every running job whose kind is in kinds to the
	// reconciling state (non-terminal); returns the count moved. Called once at
	// startup, before FailRunning, so reconcilable kinds are recovered rather than
	// failed. An empty kinds slice is a no-op returning 0.
	MarkReconciling(ctx context.Context, kinds []string) (int, error)
	// ResolveReconciling transitions a reconciling job to a terminal state
	// (succeeded or failed), setting finished + error. Compare-and-swap: it
	// affects only a row currently in reconciling, so it no-ops (returns false) if
	// an operator cancel already moved it. Passing any non-terminal state is a
	// programming error.
	ResolveReconciling(ctx context.Context, id string, state JobState, errMsg string) (bool, error)
	// CancelReconciling transitions a reconciling job to canceled, setting
	// finished. Compare-and-swap: affects only a row currently in reconciling,
	// returning false otherwise. Used by the cancel endpoint as the escape hatch.
	CancelReconciling(ctx context.Context, id string) (bool, error)
	// CancelQueued atomically transitions a still-queued job to canceled (setting
	// finished). Returns true if it transitioned; false if the job was not in the
	// queued state (already claimed, terminal, or absent).
	CancelQueued(ctx context.Context, id string) (bool, error)
	// PruneJobs deletes terminal (succeeded/failed) jobs finished before
	// olderThan, preserving parent/child integrity: a parent row is deleted only
	// when it has no surviving child. Returns the number of rows deleted.
	PruneJobs(ctx context.Context, olderThan time.Time) (int, error)
}

JobStore persists and dispenses jobs. Implemented by *SQLite and *Memory.

type KeyStore

type KeyStore struct {
	// contains filtered or unexported fields
}

KeyStore is an atomically-swappable holder for the 32-byte secret key. Safe for concurrent Load/Store; mirrors internal/auth.KeyStore so a SIGHUP reload in main takes effect on the next seal/open without a restart.

func NewKeyStore

func NewKeyStore(k [32]byte) *KeyStore

NewKeyStore returns a store seeded with k.

func (*KeyStore) Load

func (s *KeyStore) Load() [32]byte

Load returns the current key (zero value if never set).

func (*KeyStore) Store

func (s *KeyStore) Store(k [32]byte)

Store atomically replaces the live key.

type Memory

type Memory struct {
	PutErr    error
	DeleteErr error
	// contains filtered or unexported fields
}

Memory is an in-memory Store for tests. Secrets are kept in plaintext and timestamps are NOT stamped (unlike the SQLite store) — it is a test double, not a production backend. PutErr/DeleteErr, when non-nil, make the corresponding call fail, to exercise callers' fatal-failure paths. It also implements JobStore with an in-memory []Job slice, TemplateStore with an in-memory map, and BackupStore with an in-memory map.

func NewMemory

func NewMemory() *Memory

NewMemory returns an empty in-memory store.

func (*Memory) AppendStep

func (m *Memory) AppendStep(_ context.Context, id string, step JobStep) error

func (*Memory) CancelQueued

func (m *Memory) CancelQueued(_ context.Context, id string) (bool, error)

func (*Memory) CancelReconciling

func (m *Memory) CancelReconciling(_ context.Context, id string) (bool, error)

func (*Memory) ClaimNext

func (m *Memory) ClaimNext(_ context.Context) (Job, bool, error)

func (*Memory) CompleteBackup

func (m *Memory) CompleteBackup(_ context.Context, id string, vols []BackupVolume) (bool, error)

func (*Memory) CountTemplates

func (m *Memory) CountTemplates(_ context.Context) (int, error)

func (*Memory) CreateBackup

func (m *Memory) CreateBackup(_ context.Context, b Backup) error

func (*Memory) DeleteBackup

func (m *Memory) DeleteBackup(_ context.Context, id string) error

func (*Memory) DeleteHostSecret

func (m *Memory) DeleteHostSecret(_ context.Context, host, name string) error

func (*Memory) DeleteSpec

func (m *Memory) DeleteSpec(_ context.Context, host, template, slug string) error

func (*Memory) DeleteTemplate

func (m *Memory) DeleteTemplate(_ context.Context, id string) error

func (*Memory) Enqueue

func (m *Memory) Enqueue(_ context.Context, kind string, args json.RawMessage, parentID string) (Job, error)

func (*Memory) FailBackup

func (m *Memory) FailBackup(_ context.Context, id string) (bool, error)

func (*Memory) FailRunning

func (m *Memory) FailRunning(_ context.Context, reason string) (int, error)

func (*Memory) Finish

func (m *Memory) Finish(_ context.Context, id string, state JobState, errMsg string) error

func (*Memory) GetBackup

func (m *Memory) GetBackup(_ context.Context, id string) (Backup, error)

func (*Memory) GetHostSecret

func (m *Memory) GetHostSecret(_ context.Context, host, name string) ([]byte, error)

func (*Memory) GetJob

func (m *Memory) GetJob(_ context.Context, id string) (Job, error)

func (*Memory) GetSpec

func (m *Memory) GetSpec(_ context.Context, host, template, slug string) (Spec, error)

func (*Memory) GetTemplate

func (m *Memory) GetTemplate(_ context.Context, id string) (Template, error)

func (*Memory) ListBackups

func (m *Memory) ListBackups(_ context.Context, host, template, slug string, limit int) ([]Backup, error)

func (*Memory) ListJobs

func (m *Memory) ListJobs(_ context.Context, f JobFilter) ([]Job, error)

func (*Memory) ListSpecKeys

func (m *Memory) ListSpecKeys(_ context.Context, host string) ([]SpecKey, error)

func (*Memory) ListTemplates

func (m *Memory) ListTemplates(_ context.Context) ([]Template, error)

func (*Memory) MarkReconciling

func (m *Memory) MarkReconciling(_ context.Context, kinds []string) (int, error)

func (*Memory) PruneJobs

func (m *Memory) PruneJobs(_ context.Context, olderThan time.Time) (int, error)

func (*Memory) PutHostSecret

func (m *Memory) PutHostSecret(_ context.Context, host, name string, value []byte) error

func (*Memory) PutSpec

func (m *Memory) PutSpec(_ context.Context, s Spec) error

PutSpec inserts or replaces (upserts) the spec for (host, template, slug).

func (*Memory) PutTemplate

func (m *Memory) PutTemplate(_ context.Context, t Template) error

func (*Memory) ResolveReconciling

func (m *Memory) ResolveReconciling(_ context.Context, id string, state JobState, errMsg string) (bool, error)

func (*Memory) SecretsEnabled

func (m *Memory) SecretsEnabled() bool

SecretsEnabled reports whether this store can persist secrets. The in-memory double is keyed-by-default for tests, so it always returns true.

func (*Memory) StartChild

func (m *Memory) StartChild(_ context.Context, kind string, args json.RawMessage, parentID string) (Job, error)

type SQLite

type SQLite struct {
	// contains filtered or unexported fields
}

SQLite is the durable Store backed by a single SQLite file. Secrets are sealed with the key held in keys, read fresh on every Put/Get so a SIGHUP key swap takes effect immediately.

func OpenSQLite

func OpenSQLite(path string, keys *KeyStore) (*SQLite, error)

OpenSQLite opens (creating if needed) the SQLite file at path and ensures the schema exists. keys supplies the AES-256-GCM secret key.

func (*SQLite) AppendStep

func (s *SQLite) AppendStep(ctx context.Context, id string, step JobStep) error

func (*SQLite) CancelQueued

func (s *SQLite) CancelQueued(ctx context.Context, id string) (bool, error)

func (*SQLite) CancelReconciling

func (s *SQLite) CancelReconciling(ctx context.Context, id string) (bool, error)

func (*SQLite) ClaimNext

func (s *SQLite) ClaimNext(ctx context.Context) (Job, bool, error)

func (*SQLite) Close

func (s *SQLite) Close() error

Close releases the underlying database handle.

func (*SQLite) CompleteBackup

func (s *SQLite) CompleteBackup(ctx context.Context, id string, vols []BackupVolume) (bool, error)

func (*SQLite) CountTemplates

func (s *SQLite) CountTemplates(ctx context.Context) (int, error)

func (*SQLite) CreateBackup

func (s *SQLite) CreateBackup(ctx context.Context, b Backup) error

func (*SQLite) DeleteBackup

func (s *SQLite) DeleteBackup(ctx context.Context, id string) error

func (*SQLite) DeleteHostSecret

func (s *SQLite) DeleteHostSecret(ctx context.Context, host, name string) error

func (*SQLite) DeleteSpec

func (s *SQLite) DeleteSpec(ctx context.Context, host, template, slug string) error

func (*SQLite) DeleteTemplate

func (s *SQLite) DeleteTemplate(ctx context.Context, id string) error

func (*SQLite) Enqueue

func (s *SQLite) Enqueue(ctx context.Context, kind string, args json.RawMessage, parentID string) (Job, error)

func (*SQLite) FailBackup

func (s *SQLite) FailBackup(ctx context.Context, id string) (bool, error)

func (*SQLite) FailRunning

func (s *SQLite) FailRunning(ctx context.Context, reason string) (int, error)

func (*SQLite) Finish

func (s *SQLite) Finish(ctx context.Context, id string, state JobState, errMsg string) error

func (*SQLite) GetBackup

func (s *SQLite) GetBackup(ctx context.Context, id string) (Backup, error)

func (*SQLite) GetHostSecret

func (s *SQLite) GetHostSecret(ctx context.Context, host, name string) ([]byte, error)

func (*SQLite) GetJob

func (s *SQLite) GetJob(ctx context.Context, id string) (Job, error)

func (*SQLite) GetSpec

func (s *SQLite) GetSpec(ctx context.Context, host, template, slug string) (Spec, error)

func (*SQLite) GetTemplate

func (s *SQLite) GetTemplate(ctx context.Context, id string) (Template, error)

func (*SQLite) ListBackups

func (s *SQLite) ListBackups(ctx context.Context, host, template, slug string, limit int) ([]Backup, error)

func (*SQLite) ListJobs

func (s *SQLite) ListJobs(ctx context.Context, f JobFilter) ([]Job, error)

func (*SQLite) ListSpecKeys

func (s *SQLite) ListSpecKeys(ctx context.Context, host string) ([]SpecKey, error)

func (*SQLite) ListTemplates

func (s *SQLite) ListTemplates(ctx context.Context) ([]Template, error)

func (*SQLite) MarkReconciling

func (s *SQLite) MarkReconciling(ctx context.Context, kinds []string) (int, error)

func (*SQLite) PruneJobs

func (s *SQLite) PruneJobs(ctx context.Context, olderThan time.Time) (int, error)

func (*SQLite) PutHostSecret

func (s *SQLite) PutHostSecret(ctx context.Context, host, name string, value []byte) error

func (*SQLite) PutSpec

func (s *SQLite) PutSpec(ctx context.Context, sp Spec) error

func (*SQLite) PutTemplate

func (s *SQLite) PutTemplate(ctx context.Context, t Template) error

func (*SQLite) ResolveReconciling

func (s *SQLite) ResolveReconciling(ctx context.Context, id string, state JobState, errMsg string) (bool, error)

func (*SQLite) SecretsEnabled

func (s *SQLite) SecretsEnabled() bool

SecretsEnabled reports whether this store can persist secrets — true only when it was opened with an encryption key (-spec-key-file).

func (*SQLite) StartChild

func (s *SQLite) StartChild(ctx context.Context, kind string, args json.RawMessage, parentID string) (Job, error)

type Spec

type Spec struct {
	Host     string
	Template string
	Slug     string
	// Parameters is the instance's render parameters. NOTE: SQLite-backed
	// storage round-trips this through JSON, so numbers come back as float64
	// (e.g. an int 5432 becomes float64(5432)). Callers that re-render via
	// text/template are unaffected; callers must not type-assert .(int).
	Parameters map[string]any
	Secrets    map[string]string
	// Domains are the public hostnames the ingress layer routes to this
	// instance. Empty for non-web instances. Non-secret; stored in plaintext.
	Domains []string
	Created time.Time
	Updated time.Time
}

Spec is the desired state of one instance.

type SpecKey

type SpecKey struct {
	Template string
	Slug     string
}

SpecKey identifies one stored instance without exposing its secrets. Used by host-wide planning (evacuate) that only needs to know what is on a host.

type Store

type Store interface {
	// PutSpec inserts or replaces the spec for (s.Host, s.Template, s.Slug).
	PutSpec(ctx context.Context, s Spec) error
	GetSpec(ctx context.Context, host, template, slug string) (Spec, error)
	DeleteSpec(ctx context.Context, host, template, slug string) error
	// ListSpecKeys returns the (template, slug) of every spec on host, without
	// decrypting secrets. Empty slice (no error) when the host has none.
	ListSpecKeys(ctx context.Context, host string) ([]SpecKey, error)

	// PutHostSecret inserts or replaces the sealed value of a per-host secret,
	// keyed by (host, name). Implementations seal Value at rest.
	PutHostSecret(ctx context.Context, host, name string, value []byte) error
	// GetHostSecret returns the decrypted per-host secret value, or ErrNotFound.
	GetHostSecret(ctx context.Context, host, name string) ([]byte, error)
	// DeleteHostSecret removes a per-host secret; absent is not an error.
	DeleteHostSecret(ctx context.Context, host, name string) error

	// SecretsEnabled reports whether this store can persist secrets — true only
	// when it was opened with an encryption key. Callers use it to reject a
	// secret-bearing operation BEFORE mutating any host, so a key-less store does
	// not leave orphaned host state when the later PutSpec fails with
	// ErrSecretsNeedKey.
	SecretsEnabled() bool
}

Store persists instance specs. Implementations encrypt Secrets at rest and stamp Created (first write) and Updated (every write); the in-memory test double does neither.

type Template

type Template struct {
	Meta    render.Meta
	Body    string
	Origin  string // "seed" | "user"
	Created time.Time
	Updated time.Time
}

Template is an authored contract (render.Meta) plus its renderable body and provenance. The template id is Meta.ID.

func ParseSeeds

func ParseSeeds(fsys fs.FS) ([]Template, error)

ParseSeeds reads every *.yaml in fsys, parses each via render.ParseMeta, and returns them as Origin:"seed" templates. Used to seed an empty store at boot.

type TemplateStore

type TemplateStore interface {
	ListTemplates(ctx context.Context) ([]Template, error)
	GetTemplate(ctx context.Context, id string) (Template, error)
	PutTemplate(ctx context.Context, t Template) error
	DeleteTemplate(ctx context.Context, id string) error
	CountTemplates(ctx context.Context) (int, error)
}

TemplateStore persists deployable templates.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL