store

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 25, 2026 License: AGPL-3.0 Imports: 4 Imported by: 0

Documentation

Overview

Package store defines the storage interface for sqi-server and the domain model types shared across all components.

Design

Store is a composite interface that embeds one sub-interface per aggregate (Farm, Queue, Worker, Job, Step, Task, etc.). Callers declare the narrowest interface they need rather than depending on the full Store:

// Scheduler only needs tasks and workers.
func NewScheduler(tasks store.TaskStore, workers store.WorkerStore) *Scheduler

// REST handler only needs jobs and queues.
func NewJobHandler(jobs store.JobStore, queues store.QueueStore) *JobHandler

The SQLite-backed implementation implements Store in full. The in-memory fake also implements Store in full, used by unit tests that must not touch the filesystem.

JSON columns

Fields that are stored as JSON in SQLite (e.g. Worker.Tags, Task.Parameters) are typed as Go maps or slices. The SQLite implementation is responsible for marshaling and unmarshaling; callers work with native Go types throughout.

Errors

Methods return ErrNotFound when the requested row does not exist, and ErrConflict when a uniqueness constraint would be violated. All other errors are driver-level and should be treated as opaque.

Index

Constants

View Source
const (
	// DefaultLimit is the page size used when the caller does not specify one.
	DefaultLimit = 50
	// MaxLimit is the maximum page size accepted from callers. Requests above
	// this are clamped rather than rejected, matching the behavior of most
	// REST pagination conventions.
	MaxLimit = 1000
)

Variables

View Source
var ErrConflict = errors.New("store: conflict")

ErrConflict is returned when an insert or update would violate a uniqueness constraint (e.g. creating a farm with a name that already exists).

View Source
var ErrInvalidPagination = errors.New("store: invalid pagination parameters")

ErrInvalidPagination is returned when Pagination contains values that cannot be corrected by clamping (reserved for future use).

View Source
var ErrNotFound = errors.New("store: not found")

ErrNotFound is returned by Get* methods when the requested row does not exist. Use errors.Is to check:

job, err := store.GetJob(ctx, id)
if errors.Is(err, store.ErrNotFound) { ... }
View Source
var ErrUsageAtCapacity = errors.New("store: usage pool at capacity")

ErrUsageAtCapacity is returned by UsageClaimStore.TryClaimSlots when one or more required usage pools are saturated (active claims have reached MaxConcurrent). The task should remain ready for reassignment on the next scheduler tick.

Functions

This section is empty.

Types

type AttemptStatus

type AttemptStatus string

AttemptStatus is the terminal-or-active state of a single task execution.

const (
	// AttemptStatusRunning means the worker is actively executing the task.
	AttemptStatusRunning AttemptStatus = "running"
	// AttemptStatusSucceeded means the task exited cleanly.
	AttemptStatusSucceeded AttemptStatus = "succeeded"
	// AttemptStatusFailed means the task exited with a non-zero code or the
	// worker reported a fatal error.
	AttemptStatusFailed AttemptStatus = "failed"
	// AttemptStatusCanceled means the attempt was interrupted before completion.
	AttemptStatusCanceled AttemptStatus = "canceled"
)

type AuditEntry

type AuditEntry struct {
	ID         string
	EntityType string         // e.g. "job", "worker", "queue", "farm"
	EntityID   string         // ID of the affected entity
	Action     string         // e.g. "submitted", "canceled", "priority_changed"
	Actor      string         // authenticated identity; empty in no-auth deployments
	Details    map[string]any // action-specific context
	CreatedAt  time.Time
}

AuditEntry is a single append-only record in the audit log. The log captures state-changing actions performed by users or internal components so that operators can reconstruct what happened and who did it.

Details is a free-form map for action-specific context — before/after values, failure reasons, parameter diffs, etc. The interpretation of Details is defined by the code that writes the entry.

type AuditStore

type AuditStore interface {
	// AppendAuditEntry inserts a new audit log entry. The caller must populate
	// all fields including a unique ID and CreatedAt.
	AppendAuditEntry(ctx context.Context, entry AuditEntry) error

	// ListAuditEntries returns all audit entries for the given entity,
	// ordered by CreatedAt ascending. Pass empty strings to list all entries
	// (for administrative views).
	ListAuditEntries(ctx context.Context, entityType, entityID string) ([]AuditEntry, error)
}

AuditStore is the persistence interface for AuditEntry records. The audit log is append-only; there are no update or delete methods.

type DeletedJob

type DeletedJob struct {
	ID      string
	Name    string
	FarmID  string
	QueueID string
}

DeletedJob is the summary of a job removed by JobStore.DeleteTerminalJobsBefore, carrying the fields a WebSocket "removed" event needs.

type Farm

type Farm struct {
	ID                 string
	Name               string
	Description        string
	MaxConcurrentTasks int // 0 = unlimited; farm-level policy
	CreatedAt          time.Time
	UpdatedAt          time.Time
}

Farm is the top-level organizational grouping in sqi. A deployment typically has one farm, but large facilities may define multiple to represent distinct infrastructure pools, studios, or departments.

type FarmStore

type FarmStore interface {
	// CreateFarm inserts a new farm. The caller must populate all fields;
	// the implementation persists them as-is. Returns [ErrConflict] if a
	// farm with the same name already exists.
	CreateFarm(ctx context.Context, farm Farm) (Farm, error)

	// GetFarm returns the farm with the given ID, or [ErrNotFound].
	GetFarm(ctx context.Context, id string) (Farm, error)

	// ListFarms returns all farms ordered by name.
	ListFarms(ctx context.Context) ([]Farm, error)

	// UpdateFarm replaces the mutable fields (Name, Description,
	// MaxConcurrentTasks) of an existing farm and updates UpdatedAt.
	// Returns [ErrNotFound] if the farm does not exist, or [ErrConflict]
	// if the new name is already taken.
	UpdateFarm(ctx context.Context, farm Farm) (Farm, error)

	// DeleteFarm removes a farm by ID. Returns [ErrNotFound] if it does not
	// exist. The caller is responsible for ensuring no queues or workers
	// reference the farm before deleting it.
	DeleteFarm(ctx context.Context, id string) error
}

FarmStore is the persistence interface for Farm records.

type GPUInfo

type GPUInfo struct {
	Vendor string `json:"vendor,omitempty"`
	Model  string `json:"model,omitempty"`
	// VRAMMb is the VRAM capacity of each GPU device in mebibytes.
	// All devices are assumed to be identical (see type-level note above).
	VRAMMb int `json:"vram_mb,omitempty"`
	Count  int `json:"count,omitempty"`
}

GPUInfo describes the GPU(s) available on a worker host.

Phase 1 assumes a homogeneous GPU configuration: all GPUs on the host are the same model with the same VRAM. VRAMMb is the per-device VRAM capacity; Count is the number of identical devices. Mixed-GPU workers (e.g. a render card alongside a display adapter) are not modeled — workers with heterogeneous GPUs should report the lowest common VRAM to avoid over-scheduling. A []GPUDevice slice should replace this struct when heterogeneous per-device tracking is required.

type Job

type Job struct {
	ID             string
	FarmID         string
	QueueID        string
	Name           string
	Owner          string // human responsible for the job
	Submitter      string // authenticated identity that placed the job
	Priority       int    // higher = more urgent; default 50
	Status         JobStatus
	Project        string
	RawTemplate    string // verbatim OpenJD YAML or JSON as submitted
	TemplateFormat TemplateFormat
	// Parameters holds the fully-bound job-parameter values produced by
	// BindJobParameters at submission time, including applied defaults.
	// Nil or empty for jobs with no declared parameters or submitted before
	// the parameters-persistence migration (back-compat: scheduler falls back
	// to template defaults).
	Parameters  map[string]string
	CreatedAt   time.Time
	UpdatedAt   time.Time
	StartedAt   *time.Time
	CompletedAt *time.Time
}

Job is the top-level unit of work submitted to sqi-server. It holds the verbatim OpenJD template alongside derived metadata used by the scheduler.

type JobSortField

type JobSortField string

JobSortField is a column by which JobStore.ListJobs results can be ordered.

const (
	// JobSortByCreatedAt orders jobs by submission time (default).
	JobSortByCreatedAt JobSortField = "created_at"
	// JobSortByPriority orders jobs by priority (higher values first when SortDesc).
	JobSortByPriority JobSortField = "priority"
	// JobSortByStatus orders jobs alphabetically by status string.
	JobSortByStatus JobSortField = "status"
	// JobSortByUpdatedAt orders jobs by the time of the most recent change.
	JobSortByUpdatedAt JobSortField = "updated_at"
	// JobSortByName orders jobs alphabetically by name.
	JobSortByName JobSortField = "name"
)

type JobStatus

type JobStatus string

JobStatus is the lifecycle state of a job as a whole.

const (
	// JobStatusPending means the job has been submitted but no tasks are
	// running yet (e.g. waiting on step dependency resolution or queue capacity).
	JobStatusPending JobStatus = "pending"
	// JobStatusRunning means at least one task is currently assigned or running.
	JobStatusRunning JobStatus = "running"
	// JobStatusPaused means the job has been administratively paused; no new
	// task assignments will be made until it is resumed.
	JobStatusPaused JobStatus = "paused"
	// JobStatusCompleted means all tasks across all steps succeeded.
	JobStatusCompleted JobStatus = "completed"
	// JobStatusFailed means one or more tasks failed and the job cannot proceed.
	JobStatusFailed JobStatus = "failed"
	// JobStatusCanceled means the job was explicitly canceled by a user or API
	// call before it could complete.
	JobStatusCanceled JobStatus = "canceled"
)

type JobStore

type JobStore interface {
	// CreateJob inserts a new job with all fields populated by the caller.
	CreateJob(ctx context.Context, job Job) (Job, error)

	// GetJob returns the job with the given ID, or [ErrNotFound].
	GetJob(ctx context.Context, id string) (Job, error)

	// ListJobs returns a paginated, filtered, and sorted page of jobs matching
	// opts. Call [Pagination.Validate] on opts.Pagination before passing it to
	// ensure sensible defaults are applied.
	ListJobs(ctx context.Context, opts ListJobsOptions) (Page[Job], error)

	// UpdateJob replaces the mutable user-settable fields of an existing job
	// (farm_id, queue_id, name, owner, submitter, priority, project,
	// raw_template, template_format) and updates UpdatedAt.
	//
	// status, started_at, and completed_at are lifecycle columns and are
	// intentionally excluded — use [UpdateJobStatus] or [CancelJobStatus]
	// for those. The returned Job reflects the current DB state of all columns.
	//
	// Returns [ErrNotFound] if the job does not exist.
	UpdateJob(ctx context.Context, job Job) (Job, error)

	// UpdateJobStatus transitions a job to a new status and updates UpdatedAt.
	// If the new status is [JobStatusRunning] and StartedAt is nil, StartedAt
	// is set to the current time. Terminal statuses set CompletedAt.
	// Returns [ErrNotFound] if the job does not exist.
	UpdateJobStatus(ctx context.Context, id string, status JobStatus) error

	// CancelJobStatus transitions a job to [JobStatusCanceled] only when the
	// job is not already in a terminal state, preventing a race where a
	// concurrent scheduler transition to completed/failed would be overwritten.
	//
	//   - Non-terminal job → canceled, returns nil.
	//   - Already canceled → no-op, returns nil (idempotent).
	//   - Already completed or failed → returns [ErrConflict].
	//   - Job not found → returns [ErrNotFound].
	CancelJobStatus(ctx context.Context, id string) error

	// DemoteStalledJobs returns any job in [JobStatusRunning] that currently has
	// no task in [TaskStatusAssigned] or [TaskStatusRunning] — yet still has at
	// least one schedulable (ready or pending) task — back to [JobStatusPending],
	// stamping updated_at = now. It reconciles the [JobStatusRunning] invariant
	// ("at least one task is currently assigned or running") after the heartbeat
	// sweep or stale-assigned reaper returns a job's last in-flight task to the
	// ready queue with no worker to pick it up, leaving the job spuriously marked
	// running. Jobs whose tasks are all terminal are left untouched so the
	// completion logic can finalize them to completed/failed. StartedAt is
	// preserved (the job did start). Returns the IDs of the demoted jobs.
	DemoteStalledJobs(ctx context.Context, now time.Time) ([]string, error)

	// DeleteJob hard-deletes a job and every row that belongs to it, in one
	// transaction and FK-safe order: usage_claims (for the job's task attempts),
	// task_logs, task_attempts, tasks, steps, then the jobs row.
	// Returns [ErrNotFound] when the job does not exist. The audit_log is left
	// intact (it references entities by id, not by foreign key).
	DeleteJob(ctx context.Context, id string) error

	// DeleteTerminalJobsBefore hard-deletes terminal jobs whose completion time
	// (completed_at, falling back to updated_at when NULL) is before cutoff, and
	// returns a summary of each removed job. completed and canceled jobs are
	// always eligible; failed jobs are included only when includeFailed is true.
	// Active jobs are never removed. Each removed job's children are deleted via
	// the same cascade as [DeleteJob].
	DeleteTerminalJobsBefore(ctx context.Context, cutoff time.Time, includeFailed bool) ([]DeletedJob, error)
}

JobStore is the persistence interface for Job records.

type ListJobsOptions

type ListJobsOptions struct {
	// Filters
	FarmID  string
	QueueID string
	Status  JobStatus // empty = all statuses
	Owner   string
	Project string
	// Search is a case-insensitive substring matched against name, id, owner,
	// and project. Empty = no search filter.
	Search string

	// Ordering — zero values use JobSortByCreatedAt / SortAsc.
	SortBy  JobSortField
	SortDir SortDir

	// Pagination — call Pagination.Validate() before use.
	Pagination Pagination
}

ListJobsOptions filters and orders JobStore.ListJobs results. Zero values mean "no filter / use defaults".

type ListQueuesOptions

type ListQueuesOptions struct {
	// Filters
	FarmID string
	Paused *bool // nil = all, true = paused only, false = active only

	// Ordering — zero values use QueueSortByName / SortAsc.
	SortBy  QueueSortField
	SortDir SortDir

	// Pagination — call Pagination.Validate() before use.
	Pagination Pagination
}

ListQueuesOptions filters and orders QueueStore.ListQueues results. Zero values mean "no filter / use defaults".

type ListTasksOptions

type ListTasksOptions struct {
	// Filters
	JobID    string
	StepID   string
	Status   TaskStatus   // empty = all statuses (mutually exclusive with Statuses)
	Statuses []TaskStatus // IN-filter; takes precedence over Status when non-empty
	WorkerID string       // filter by assigned worker

	// Ordering — zero values use TaskSortByCreatedAt / SortAsc.
	SortBy  TaskSortField
	SortDir SortDir

	// Pagination — call Pagination.Validate() before use.
	Pagination Pagination
}

ListTasksOptions filters and orders TaskStore.ListTasks results. Zero values mean "no filter / use defaults".

type ListWorkersOptions

type ListWorkersOptions struct {
	// Filters
	FarmID          string
	QueueID         string
	ComputeLocation string
	Status          WorkerStatus // empty = all statuses
	// Search is a case-insensitive substring matched against name, hostname,
	// id, and compute_location. Empty = no search filter.
	Search string

	// IncludeUnaffiliated, when true and FarmID is non-empty, also returns
	// workers whose FarmID is empty (unaffiliated workers that accept tasks
	// from any farm). Used by the scheduler's pickWorker so that workers
	// started without an explicit farm configuration are still dispatched to.
	IncludeUnaffiliated bool

	// Ordering — zero values use WorkerSortByHostname / SortAsc.
	SortBy  WorkerSortField
	SortDir SortDir

	// Pagination — call Pagination.Validate() before use.
	Pagination Pagination
}

ListWorkersOptions filters and orders WorkerStore.ListWorkers results. Zero values mean "no filter / use defaults".

type LogStream

type LogStream string

LogStream identifies the output stream a log chunk was captured from.

const (
	// LogStreamStdout is the standard output stream.
	LogStreamStdout LogStream = "stdout"
	// LogStreamStderr is the standard error stream.
	LogStreamStderr LogStream = "stderr"
)

type Page

type Page[T any] struct {
	// Items holds the current page of results. It is never nil; an empty
	// page is represented as an empty slice.
	Items []T
	// Total is the total number of items matching the filter, regardless of
	// the current page window. Used by clients to compute page counts.
	Total int
	// Limit mirrors the effective limit used to produce this page.
	Limit int
	// Offset mirrors the effective offset used to produce this page.
	Offset int
}

Page is the generic result type returned by paginated list queries. T is the item type (e.g. Job, Task, Worker).

func (Page[T]) HasMore

func (p Page[T]) HasMore() bool

HasMore reports whether there are items beyond the current page window.

type Pagination

type Pagination struct {
	// Limit is the maximum number of items to return. 0 means use
	// [DefaultLimit]. Values above [MaxLimit] are clamped to [MaxLimit].
	Limit int
	// Offset is the zero-based index of the first item to return.
	Offset int
}

Pagination carries cursor-free (limit/offset) pagination parameters. Zero values are treated as "use defaults"; call Pagination.Validate before passing to a store method to ensure limits are applied.

func (*Pagination) Validate

func (p *Pagination) Validate() error

Validate clamps Limit into the range [1, MaxLimit], applying DefaultLimit when Limit is zero. Negative Offset is reset to zero. It returns an error only for values that cannot be silently corrected (currently none — all out-of-range values are clamped).

type Queue

type Queue struct {
	ID                 string
	FarmID             string
	Name               string
	Description        string
	Priority           int
	MaxConcurrentTasks int // 0 = unlimited
	Paused             bool
	CreatedAt          time.Time
	UpdatedAt          time.Time
}

Queue belongs to a Farm and is the container into which jobs are submitted. Scheduling policy (priority, concurrency limits, paused state) is configured at the queue level and may be overridden per job.

type QueueSortField

type QueueSortField string

QueueSortField is a column by which QueueStore.ListQueues results can be ordered.

const (
	// QueueSortByName orders queues alphabetically by name (default).
	QueueSortByName QueueSortField = "name"
	// QueueSortByPriority orders queues by priority (higher values first when
	// SortDesc).
	QueueSortByPriority QueueSortField = "priority"
	// QueueSortByCreatedAt orders queues by creation time.
	QueueSortByCreatedAt QueueSortField = "created_at"
)

type QueueStore

type QueueStore interface {
	// CreateQueue inserts a new queue. Returns [ErrConflict] if a queue with
	// the same name already exists within the same farm.
	CreateQueue(ctx context.Context, queue Queue) (Queue, error)

	// GetQueue returns the queue with the given ID, or [ErrNotFound].
	GetQueue(ctx context.Context, id string) (Queue, error)

	// ListQueues returns a paginated, filtered, and sorted page of queues
	// matching opts. Call [Pagination.Validate] on opts.Pagination before
	// passing it to ensure sensible defaults are applied.
	ListQueues(ctx context.Context, opts ListQueuesOptions) (Page[Queue], error)

	// UpdateQueue replaces the mutable fields of an existing queue and updates
	// UpdatedAt. Returns [ErrNotFound] or [ErrConflict] as appropriate.
	UpdateQueue(ctx context.Context, queue Queue) (Queue, error)

	// DeleteQueue removes a queue by ID. Returns [ErrNotFound] if it does not
	// exist. The caller is responsible for ensuring no jobs reference the
	// queue before deleting it.
	DeleteQueue(ctx context.Context, id string) error
}

QueueStore is the persistence interface for Queue records.

type SortDir

type SortDir string

SortDir is the direction of a sort: ascending or descending.

const (
	// SortAsc sorts results from smallest to largest (A→Z, oldest→newest).
	SortAsc SortDir = "asc"
	// SortDesc sorts results from largest to smallest (Z→A, newest→oldest).
	SortDesc SortDir = "desc"
)

func (SortDir) Reverse

func (d SortDir) Reverse() SortDir

Reverse returns the opposite SortDir.

type Step

type Step struct {
	ID        string
	JobID     string
	Name      string
	DependsOn []string // names of steps that must complete before this one
	StepOrder int      // position within the job for deterministic ordering
	Status    StepStatus

	// HostRequirements declares the worker capabilities required by this step.
	// Nil means any capable worker is eligible.
	// Populated from the OpenJD hostRequirements block at submission time.
	HostRequirements *StepHostRequirements

	// ComputeLocation constrains tasks to workers in the named compute location
	// (e.g. "onprem_linux", "cloud_aws_us_east"). Empty means any location is
	// eligible. Mirrors the "attr.worker.computelocation" attribute in
	// HostRequirements and is stored separately for fast SQL-level pre-filtering.
	ComputeLocation string

	CreatedAt time.Time
	UpdatedAt time.Time
}

Step is one stage within a Job. Steps may depend on other steps; a step's tasks are not scheduled until all its dependencies have reached StepStatusCompleted.

type StepAmountRequirement

type StepAmountRequirement struct {
	// Name is the capability name, e.g. "amount.worker.vcpu".
	Name string `json:"name"`
	// Min, if non-nil, is the minimum acceptable value as a decimal string.
	// The scheduler parses it with strconv.ParseFloat for comparison.
	Min *string `json:"min,omitempty"`
	// Max, if non-nil, is the maximum acceptable value as a decimal string.
	Max *string `json:"max,omitempty"`
}

StepAmountRequirement is a single quantifiable host capability requirement.

type StepAttributeRequirement

type StepAttributeRequirement struct {
	// Name is the capability name, e.g. "attr.worker.os.family".
	Name string `json:"name"`
	// AnyOf, if non-empty, requires the host attribute to equal at least one
	// of the listed values.  String comparison is case-insensitive for
	// well-known attributes (os.family, computelocation); exact for custom tags.
	AnyOf []string `json:"any_of,omitempty"`
	// AllOf, if non-empty, requires the host attribute to equal every listed
	// value.  Useful for multi-valued tag semantics (e.g. requiring two
	// software packages to both be installed).
	AllOf []string `json:"all_of,omitempty"`
}

StepAttributeRequirement is a single categorical host capability requirement.

type StepHostRequirements

type StepHostRequirements struct {
	// Amounts are quantifiable requirements such as CPUs, RAM, or GPU VRAM.
	// Usage pool requirements use the "amount.worker.usagepool." prefix and
	// are also listed in UsagePools for quick iteration.
	Amounts []StepAmountRequirement `json:"amounts,omitempty"`

	// Attributes are categorical requirements such as OS family or custom tags.
	// The "attr.worker.computelocation" entry, if present, is mirrored as the
	// enclosing [Step.ComputeLocation] field for SQL-level pre-filtering.
	Attributes []StepAttributeRequirement `json:"attributes,omitempty"`

	// UsagePools lists the names of [UsagePool] records that must have
	// remaining capacity before a task from this step can be assigned.
	// Derived from amounts named "amount.worker.usagepool.<name>" at
	// submission time.
	UsagePools []string `json:"usage_pools,omitempty"`
}

StepHostRequirements is the scheduler-facing representation of a step's hostRequirements block, normalised from the raw OpenJD model at submission time. The scheduler's matching logic reads these fields to determine whether a given worker can run a task.

Capability names follow the conventions established in the OpenJD spec and documented in [matcher.go]:

  • "attr.worker.os.family" → worker OS family ("linux", "windows", "darwin")
  • "attr.worker.os.version" → worker OS version string
  • "attr.worker.computelocation" → worker compute location name
  • "attr.worker.tag.<key>" → arbitrary worker tag value
  • "amount.worker.vcpu" → worker CPU count
  • "amount.worker.memory.mb" → worker RAM in MiB
  • "amount.worker.gpu.count" → worker GPU count
  • "amount.worker.gpu.memory.mb" → worker GPU VRAM in MiB
  • "amount.worker.usagepool.<name>" → named usage pool (capacity check)

type StepStatus

type StepStatus string

StepStatus is the lifecycle state of a step within a job.

const (
	// StepStatusPending means the step is waiting for its dependencies to complete.
	StepStatusPending StepStatus = "pending"
	// StepStatusReady means all dependencies have succeeded; tasks can be scheduled.
	StepStatusReady StepStatus = "ready"
	// StepStatusRunning means at least one task in this step is running.
	StepStatusRunning StepStatus = "running"
	// StepStatusCompleted means all tasks in this step succeeded.
	StepStatusCompleted StepStatus = "completed"
	// StepStatusFailed means one or more tasks failed.
	StepStatusFailed StepStatus = "failed"
	// StepStatusCanceled means the step was canceled, typically because the
	// parent job was canceled.
	StepStatusCanceled StepStatus = "canceled"
)

type StepStore

type StepStore interface {
	// CreateStep inserts a new step. The (JobID, Name) pair must be unique
	// within the job; returns [ErrConflict] if violated.
	CreateStep(ctx context.Context, step Step) (Step, error)

	// GetStep returns the step with the given ID, or [ErrNotFound].
	GetStep(ctx context.Context, id string) (Step, error)

	// ListSteps returns all steps for the given job, ordered by StepOrder
	// ascending.
	ListSteps(ctx context.Context, jobID string) ([]Step, error)

	// UpdateStepStatus transitions a step to a new status and updates
	// UpdatedAt. Returns [ErrNotFound] if the step does not exist.
	UpdateStepStatus(ctx context.Context, id string, status StepStatus) error
}

StepStore is the persistence interface for Step records.

type StorageLocation

type StorageLocation struct {
	ID          string
	Name        string
	Type        StorageLocationType
	Description string
	Roots       map[string]string // compute_location_name -> root path
	CreatedAt   time.Time
	UpdatedAt   time.Time
}

StorageLocation is a named, globally-registered storage root. Jobs reference locations by name with a relative path; workers resolve the name to a concrete path for their compute location at execution time.

Roots maps a compute location name (or "default") to the root path for that location. Example:

Roots = map[string]string{
    "default":         "/mnt/nas/shows",
    "windows_workers": `Z:\shows`,
    "cloud_aws":       "s3://studio-bucket/shows",
}

type StorageLocationStore

type StorageLocationStore interface {
	// CreateStorageLocation inserts a new location. Returns [ErrConflict] if
	// a location with the same name already exists.
	CreateStorageLocation(ctx context.Context, loc StorageLocation) (StorageLocation, error)

	// GetStorageLocation returns the location with the given ID, or [ErrNotFound].
	GetStorageLocation(ctx context.Context, id string) (StorageLocation, error)

	// GetStorageLocationByName returns the location with the given name, or
	// [ErrNotFound]. Used by the path resolver to look up locations by their
	// symbolic name at task dispatch time.
	GetStorageLocationByName(ctx context.Context, name string) (StorageLocation, error)

	// ListStorageLocations returns all locations ordered by name.
	ListStorageLocations(ctx context.Context) ([]StorageLocation, error)

	// UpdateStorageLocation replaces the mutable fields of an existing
	// location and updates UpdatedAt. Returns [ErrNotFound] or [ErrConflict].
	UpdateStorageLocation(ctx context.Context, loc StorageLocation) (StorageLocation, error)

	// DeleteStorageLocation removes a location by ID. Returns [ErrNotFound]
	// if it does not exist.
	DeleteStorageLocation(ctx context.Context, id string) error
}

StorageLocationStore is the persistence interface for StorageLocation records.

type StorageLocationType

type StorageLocationType string

StorageLocationType identifies the backing storage technology.

const (
	// StorageLocationTypeFilesystem is a POSIX or Windows filesystem path.
	StorageLocationTypeFilesystem StorageLocationType = "filesystem"
	// StorageLocationTypeS3 is an S3-compatible object store (AWS S3,
	// Backblaze B2, Cloudflare R2, MinIO, etc.).
	StorageLocationTypeS3 StorageLocationType = "s3"
)

type Store

Store is the top-level storage interface for sqi-server. It is composed of one sub-interface per aggregate so callers can depend on the narrowest slice they need.

Store embeds io.Closer; implementations must release any open database connections or file handles when Close is called.

type Task

type Task struct {
	ID               string
	JobID            string // denormalized from Step for query efficiency
	StepID           string
	Name             string
	Parameters       map[string]string // resolved parameter values for this task
	Status           TaskStatus
	AssignedWorkerID string     // empty when unassigned
	AssignedAt       *time.Time // nil when unassigned
	CreatedAt        time.Time
	UpdatedAt        time.Time

	// RequiredCores is the task's declared CPU reservation (OpenJD
	// amount.worker.vcpu min). Nil means undeclared — the scheduler treats the
	// cost as the running worker's full CPUCount (one such task per worker).
	RequiredCores *int
}

Task is the atomic unit of work — one process on one worker. Tasks are derived from an OpenJD step's parameter space expansion.

type TaskAttempt

type TaskAttempt struct {
	ID            string
	TaskID        string
	WorkerID      string
	SessionID     string // OpenJD session ID; may be empty for legacy workers
	AttemptNumber int    // 1-based; incremented on each retry
	Status        AttemptStatus
	ExitCode      *int // nil while running or if the process was signaled
	StartedAt     time.Time
	EndedAt       *time.Time // nil while running
	CreatedAt     time.Time
}

TaskAttempt records a single execution of a Task on a specific worker. A task may have multiple attempts if it is retried after failure.

SessionID is the OpenJD session identifier reported by the worker. Multiple task attempts that share the same session ID ran within the same OpenJD Session on the same worker — they shared a working directory and environment setup. See docs/architecture.md ("SQLite schema overview") for why sessions are not a first-class table.

type TaskAttemptStore

type TaskAttemptStore interface {
	// CreateTaskAttempt inserts a new attempt record. Called when the server
	// assigns a task to a worker and the worker acknowledges it.
	CreateTaskAttempt(ctx context.Context, attempt TaskAttempt) (TaskAttempt, error)

	// GetTaskAttempt returns the attempt with the given ID, or [ErrNotFound].
	GetTaskAttempt(ctx context.Context, id string) (TaskAttempt, error)

	// LatestTaskAttempt returns the attempt with the highest AttemptNumber for
	// the given task, or [ErrNotFound] if no attempts exist yet. Used by the
	// scheduler to determine the correct AttemptNumber when creating a retry.
	LatestTaskAttempt(ctx context.Context, taskID string) (TaskAttempt, error)

	// ListTaskAttempts returns all attempts for the given task, ordered by
	// AttemptNumber ascending.
	ListTaskAttempts(ctx context.Context, taskID string) ([]TaskAttempt, error)

	// UpdateTaskAttempt replaces the mutable fields of an existing attempt
	// (Status, ExitCode, EndedAt). Returns [ErrNotFound] if it does not exist.
	UpdateTaskAttempt(ctx context.Context, attempt TaskAttempt) (TaskAttempt, error)

	// TerminateWorkerAttempts marks all running [TaskAttempt] records for tasks
	// currently assigned to workerID as the given terminal status with the
	// supplied end time. Called by the heartbeat sweep before reclaiming tasks
	// from an offline worker so that each attempt has a closed EndedAt.
	// Returns the number of attempts updated.
	TerminateWorkerAttempts(ctx context.Context, workerID string, status AttemptStatus, endedAt time.Time) (int, error)

	// CancelJobAttempts marks all running [TaskAttempt] records for tasks
	// belonging to the given job as [AttemptStatusCanceled] with the supplied
	// end time. Should be called before [TaskStore.CancelJobTasks] so
	// that attempts are closed while the tasks still carry their assigned worker.
	// Returns the number of attempts updated.
	CancelJobAttempts(ctx context.Context, jobID string, endedAt time.Time) (int, error)
}

TaskAttemptStore is the persistence interface for TaskAttempt records.

type TaskLog

type TaskLog struct {
	ID        string
	TaskID    string
	AttemptID string

	// SeqNum is the worker-assigned monotonic sequence number for this chunk
	// within the attempt.  Starts at 1; callers must not assume it is
	// contiguous — gaps can appear if chunks are dropped or reordered.
	SeqNum int64

	// NATSSeq is the JetStream message sequence number from the SQI_LOGS
	// stream.  Used as a stable, dense offset cursor for the log-tail REST
	// endpoint.  Stored as int64 to match SQLite's INTEGER affinity; NATS
	// stream sequences are uint64 but will never approach int64 max in practice.
	NATSSeq int64

	// At is the worker's local timestamp when the chunk was captured.
	At time.Time

	// ReceivedAt is the server's clock time when the chunk was persisted.
	// Monotonically increasing within a server instance; useful for
	// detecting clock skew between worker and server.
	ReceivedAt time.Time

	// Stream is "stdout" or "stderr".
	Stream LogStream

	// Data is the raw log text.
	Data string
}

TaskLog is one chunk of output produced by a running task.

Workers publish log chunks to task.logs.<taskID> as output is produced. The server persists each chunk here with:

  • A monotonic worker-side sequence number (SeqNum) per attempt, starting at 1 and incrementing by 1. This allows the REST log-tail endpoint to return chunks in the order the worker produced them regardless of NATS delivery order.
  • A NATS stream sequence number (NATSSeq) used as the primary DB insertion order and for efficient cursor-based pagination.
  • A server-side receipt timestamp (ReceivedAt) for audit purposes.

type TaskLogStore

type TaskLogStore interface {
	// CreateTaskLog inserts a new log chunk. The caller must populate all
	// fields including a unique ID.
	CreateTaskLog(ctx context.Context, log TaskLog) (TaskLog, error)

	// ListTaskLogs returns log chunks for a task attempt in NATSSeq ascending
	// order.  Pass afterNATSSeq = 0 to start from the beginning; pass the
	// NATSSeq of the last received chunk to page forward.  Limit controls the
	// maximum number of chunks returned.
	//
	// This method is used by the REST log-tail endpoint and by the WebSocket
	// log-streaming handler.
	ListTaskLogs(ctx context.Context, attemptID string, afterNATSSeq int64, limit int) ([]TaskLog, error)
}

TaskLogStore is the persistence interface for TaskLog records.

type TaskSortField

type TaskSortField string

TaskSortField is a column by which TaskStore.ListTasks results can be ordered.

const (
	// TaskSortByCreatedAt orders tasks by creation time (default).
	TaskSortByCreatedAt TaskSortField = "created_at"
	// TaskSortByStatus orders tasks alphabetically by status string.
	TaskSortByStatus TaskSortField = "status"
	// TaskSortByUpdatedAt orders tasks by the time of the most recent change.
	TaskSortByUpdatedAt TaskSortField = "updated_at"
	// TaskSortByName orders tasks alphabetically by name.
	TaskSortByName TaskSortField = "name"
)

type TaskStatus

type TaskStatus string

TaskStatus is the lifecycle state of an individual task.

const (
	// TaskStatusPending means the task exists but its step's dependencies have
	// not yet been satisfied.
	TaskStatusPending TaskStatus = "pending"
	// TaskStatusReady means the task is eligible for assignment; its step's
	// dependencies have completed.
	TaskStatusReady TaskStatus = "ready"
	// TaskStatusAssigned means the task has been assigned to a worker but the
	// worker has not yet confirmed it is running.
	TaskStatusAssigned TaskStatus = "assigned"
	// TaskStatusRunning means the worker has confirmed the task is executing.
	TaskStatusRunning TaskStatus = "running"
	// TaskStatusSucceeded means the task completed successfully.
	TaskStatusSucceeded TaskStatus = "succeeded"
	// TaskStatusFailed means the task exited with a non-zero code or the
	// worker reported a fatal error.
	TaskStatusFailed TaskStatus = "failed"
	// TaskStatusCanceled means the task was explicitly canceled before it could
	// complete.
	TaskStatusCanceled TaskStatus = "canceled"
)

type TaskStore

type TaskStore interface {
	// CreateTask inserts a new task. The caller must populate all fields
	// including a unique ID.
	CreateTask(ctx context.Context, task Task) (Task, error)

	// GetTask returns the task with the given ID, or [ErrNotFound].
	GetTask(ctx context.Context, id string) (Task, error)

	// ListTasks returns a paginated, filtered, and sorted page of tasks
	// matching opts. Call [Pagination.Validate] on opts.Pagination before
	// passing it to ensure sensible defaults are applied.
	ListTasks(ctx context.Context, opts ListTasksOptions) (Page[Task], error)

	// UpdateTaskStatus transitions a task to a new status and updates
	// UpdatedAt. Returns [ErrNotFound] if the task does not exist.
	UpdateTaskStatus(ctx context.Context, id string, status TaskStatus) error

	// AssignTask atomically sets AssignedWorkerID, AssignedAt, and Status to
	// [TaskStatusAssigned] for the given task. Returns [ErrNotFound] if the
	// task does not exist.
	AssignTask(ctx context.Context, id, workerID string, assignedAt time.Time) error

	// ListReadyTasks returns up to limit tasks in [TaskStatusReady] that
	// belong to non-paused queues within the given farm, ordered by:
	//   1. job priority descending (higher values first),
	//   2. job submission time ascending (earlier jobs win ties),
	//   3. step order ascending (earlier steps in a job run before later ones),
	//   4. task creation time ascending (stable tiebreaker within a step).
	//
	// Used by the scheduler's assignment loop.
	ListReadyTasks(ctx context.Context, farmID string, limit int) ([]Task, error)

	// ReclaimWorkerTasks resets all tasks assigned to workerID that are still
	// in [TaskStatusAssigned] or [TaskStatusRunning] back to [TaskStatusReady]
	// so they can be reassigned by the scheduler. Called by the heartbeat
	// timeout sweep after a worker is marked offline.
	// Returns the number of tasks reclaimed.
	ReclaimWorkerTasks(ctx context.Context, workerID string) (int, error)

	// ReclaimStaleAssignedTasks returns tasks stuck in [TaskStatusAssigned] whose
	// assigned_at is older than cutoff to [TaskStatusReady], clearing
	// assigned_worker_id and assigned_at so the scheduler can reassign them.
	// Tasks in [TaskStatusRunning] are left untouched — only assignments that
	// never started are reclaimed (e.g. the assignment message expired from the
	// work stream before the worker pulled it). Returns the reclaimed tasks,
	// each still carrying its pre-reset assigned_worker_id, so the caller can
	// close their attempts and release usage-pool claims.
	ReclaimStaleAssignedTasks(ctx context.Context, cutoff time.Time) ([]Task, error)

	// CountActiveTasksInQueue returns the number of tasks for the given queue
	// that are currently in [TaskStatusAssigned] or [TaskStatusRunning] state.
	// Used by the scheduler's per-queue policy gate.
	CountActiveTasksInQueue(ctx context.Context, queueID string) (int, error)

	// CountActiveTasksInFarm returns the number of tasks across all queues in
	// the given farm that are currently in [TaskStatusAssigned] or
	// [TaskStatusRunning] state. Used by the scheduler's per-farm policy gate.
	CountActiveTasksInFarm(ctx context.Context, farmID string) (int, error)

	// CountReadyTasksByQueue returns the number of tasks in [TaskStatusReady]
	// state for each queue within the given farm, keyed by queue ID.
	// Queues with no ready tasks are omitted from the map.
	// Used by the scheduler to update the [SchedulerQueueDepth] Prometheus
	// gauge.
	CountReadyTasksByQueue(ctx context.Context, farmID string) (map[string]int, error)

	// CancelJobTasks transitions all non-terminal tasks for the given job to
	// [TaskStatusCanceled], clearing AssignedWorkerID and AssignedAt on each,
	// and returns the subset that were in [TaskStatusAssigned] or
	// [TaskStatusRunning] at the time of the call (with their AssignedWorkerID
	// intact) so the caller can publish NATS cancel signals to the appropriate
	// workers.
	//
	// The SELECT and UPDATE run inside a single database transaction so no
	// concurrent assignment can race between observation and cancellation.
	// Tasks already in a terminal state (succeeded, failed, canceled) are not
	// modified.
	CancelJobTasks(ctx context.Context, jobID string, now time.Time) ([]Task, error)

	// RetryTasks revives failed/canceled tasks so they can run again. It
	// transitions every task of jobID in [TaskStatusFailed] or
	// [TaskStatusCanceled] — or, when taskIDs is non-nil, only those of the
	// given IDs that are failed/canceled — back to [TaskStatusPending]. Any of
	// their enclosing steps that are currently in a terminal status are reset to
	// [StepStatusPending], and the job itself is reset to [JobStatusPending]
	// when it is currently terminal (failed/canceled); a non-terminal job is
	// left unchanged. All updates run in a single transaction.
	//
	// Resetting to pending (rather than ready) lets the caller re-run
	// [openjd.ResolveDependencies] to re-gate the revived tasks in dependency
	// order. Tasks not in a terminal-retryable state are not modified. Returns
	// the revived task rows (each with Status == pending), or an empty slice
	// when nothing matched.
	RetryTasks(ctx context.Context, jobID string, taskIDs []string, now time.Time) ([]Task, error)

	// TransitionStepPendingTasks transitions every task of the given step that is
	// currently in [TaskStatusPending] to status `to`, updates UpdatedAt, and
	// returns the affected task rows. It is used to promote a step's tasks to
	// [TaskStatusReady] once its dependencies resolve, and to cancel them when an
	// upstream dependency fails.
	//
	// The transition is applied as a single statement covering all matching rows
	// regardless of count, so it is not subject to the [MaxLimit] pagination
	// ceiling. Tasks not in pending state are not modified.
	TransitionStepPendingTasks(ctx context.Context, stepID string, to TaskStatus) ([]Task, error)

	// CountTasksByJob returns the number of tasks for the given job keyed by
	// status. Statuses with zero tasks are omitted from the returned map.
	// Used by the REST layer to include aggregate task counts in job responses.
	CountTasksByJob(ctx context.Context, jobID string) (map[TaskStatus]int, error)

	// CommittedCores returns the sum of CPU-core reservations held by the worker
	// across its assigned and running tasks: Σ COALESCE(required_cores,
	// fullMachineCost). Callers pass the worker's CPUCount as fullMachineCost so
	// an undeclared task (required_cores NULL) counts as the whole machine.
	CommittedCores(ctx context.Context, workerID string, fullMachineCost int) (int, error)

	// LeaseReadyTask atomically transitions a task from [TaskStatusReady] to
	// [TaskStatusAssigned], setting assigned_worker_id and assigned_at. It
	// returns true iff the task was still ready (exactly one row changed); a
	// false return means another worker leased it first. This is the race guard
	// for concurrent lease requests.
	LeaseReadyTask(ctx context.Context, taskID, workerID string, now time.Time) (bool, error)
}

TaskStore is the persistence interface for Task records.

type TemplateFormat

type TemplateFormat string

TemplateFormat identifies the serialization format of Job.RawTemplate.

const (
	// TemplateFormatYAML means the raw template was submitted as YAML.
	TemplateFormatYAML TemplateFormat = "yaml"
	// TemplateFormatJSON means the raw template was submitted as JSON.
	TemplateFormatJSON TemplateFormat = "json"
)

type UsageClaim

type UsageClaim struct {
	ID            string
	PoolID        string
	TaskAttemptID string
	// DB column: checked_out_at (historic name; kept to avoid a column-rename migration).
	ClaimedAt  time.Time
	ReleasedAt *time.Time // nil = currently active
}

UsageClaim records one active or completed claim of a UsagePool slot by a specific TaskAttempt. ReleasedAt is nil while the claim is active; it is set when the task attempt reaches a terminal state.

type UsageClaimStore

type UsageClaimStore interface {
	// CreateClaim inserts a new active claim for the given pool and task
	// attempt. The (TaskAttemptID, PoolID) pair must be unique; returns
	// [ErrConflict] if violated.
	CreateClaim(ctx context.Context, claim UsageClaim) (UsageClaim, error)

	// ReleaseClaim sets ReleasedAt on the claim with the given ID,
	// marking it as no longer active. Returns [ErrNotFound] if it does not
	// exist.
	ReleaseClaim(ctx context.Context, id string, releasedAt time.Time) error

	// ActiveClaimCount returns the number of claims for the given pool
	// where ReleasedAt IS NULL. Used by the scheduler's admission check before
	// assigning a task that requires the pool.
	ActiveClaimCount(ctx context.Context, poolID string) (int, error)

	// TryClaimSlots atomically verifies that every pool in claims has
	// remaining capacity and, if so, inserts a [UsageClaim] row for each.
	// The operation runs inside a single database transaction so the
	// count-check and insert are indivisible.
	//
	// Returns [ErrUsageAtCapacity] if any pool is saturated; no rows are
	// inserted in that case. Returns [ErrConflict] if any claim ID or
	// (task_attempt_id, pool_id) pair already exists.
	//
	// taskAttemptID must reference an existing [TaskAttempt] row. claimedAt
	// is the timestamp recorded on each new claim.
	TryClaimSlots(ctx context.Context, taskAttemptID string, claims []UsagePoolClaim, claimedAt time.Time) error

	// ReleaseAttemptClaims sets ReleasedAt on every active claim
	// (released_at IS NULL) for the given taskAttemptID. Called when a task
	// attempt reaches a terminal state so the usage pool slots are freed for
	// reassignment.
	//
	// Returns the number of claims released (0 is not an error when the
	// attempt held no claims).
	ReleaseAttemptClaims(ctx context.Context, taskAttemptID string, releasedAt time.Time) (int, error)

	// ReleaseJobClaims sets ReleasedAt on every active claim for any task
	// attempt belonging to a task in the given job. Called during job
	// cancellation to free all usage pool slots held by that job in a single
	// operation, rather than iterating through individual attempts.
	//
	// Returns the number of claims released (0 is not an error when the job
	// held no claims).
	ReleaseJobClaims(ctx context.Context, jobID string, releasedAt time.Time) (int, error)
}

UsageClaimStore is the persistence interface for UsageClaim records. It is separated from UsagePoolStore because the scheduler is the primary caller and only needs claim operations, not pool CRUD.

type UsagePool

type UsagePool struct {
	ID            string
	Name          string
	ServerHint    string // e.g. "10.0.0.50:5053"; empty = not configured
	MaxConcurrent int
	CreatedAt     time.Time
	UpdatedAt     time.Time
}

UsagePool represents a tracked named concurrency limit. sqi enforces concurrency limits itself by counting active UsageClaim records; it does not query any upstream server directly.

ServerHint is informational only — the address of a vendor's software server (FLEXlm/RLM), useful for diagnostics when the pool represents software slots. Leave empty for non-vendor dimensions.

type UsagePoolClaim

type UsagePoolClaim struct {
	// ClaimID is a caller-supplied UUID for the claim row to be created.
	ClaimID string
	// PoolID is the [UsagePool] to claim from.
	PoolID string
	// PoolName is used in error messages and debug logging only.
	PoolName string
	// MaxConcurrent is the pool's current limit. The implementation uses this
	// value inside the transaction to avoid a second pool lookup.
	MaxConcurrent int
}

UsagePoolClaim describes a single usage pool slot to be claimed atomically by UsageClaimStore.TryClaimSlots.

type UsagePoolStore

type UsagePoolStore interface {
	// CreateUsagePool inserts a new pool. Returns [ErrConflict] if a pool
	// with the same name already exists.
	CreateUsagePool(ctx context.Context, pool UsagePool) (UsagePool, error)

	// GetUsagePool returns the pool with the given ID, or [ErrNotFound].
	GetUsagePool(ctx context.Context, id string) (UsagePool, error)

	// ListUsagePools returns all pools ordered by name.
	ListUsagePools(ctx context.Context) ([]UsagePool, error)

	// ListUsagePoolUtilization returns all pools ordered by name, each paired with
	// its current number of active claims. It computes utilization in a
	// single query rather than one count per pool.
	ListUsagePoolUtilization(ctx context.Context) ([]UsagePoolUtilization, error)

	// UpdateUsagePool replaces the mutable fields of an existing pool and
	// updates UpdatedAt. Returns [ErrNotFound] or [ErrConflict].
	UpdateUsagePool(ctx context.Context, pool UsagePool) (UsagePool, error)

	// DeleteUsagePool removes a pool by ID. Returns [ErrNotFound] if it
	// does not exist.
	DeleteUsagePool(ctx context.Context, id string) error
}

UsagePoolStore is the persistence interface for UsagePool records.

type UsagePoolUtilization

type UsagePoolUtilization struct {
	UsagePool

	// InUse is the number of active claims (released_at IS NULL) for the pool.
	InUse int
}

UsagePoolUtilization pairs a UsagePool with its current live utilization — the number of active (unreleased) claims against it. It is returned by UsagePoolStore.ListUsagePoolUtilization so callers can render "in use vs capacity" without an N+1 fan-out of per-pool count queries.

type Worker

type Worker struct {
	ID      string
	FarmID  string
	QueueID string // empty = no queue affinity
	// Name is the worker's human-readable display label (the worker.name
	// config field, default the hostname). Distinguishes multiple workers
	// running on one host in the UI; may be empty for workers registered
	// before this field existed, in which case callers fall back to Hostname.
	Name            string
	Hostname        string
	IPAddress       string
	ComputeLocation string
	OS              string
	OSVersion       string
	// Version is the sqi-worker build version the worker self-reports at
	// registration (the worker binary's internal/version.Version). May be empty
	// for workers registered before this field existed.
	Version         string
	CPUCount        int
	RAMMb           int
	GPUInfo         GPUInfo
	Tags            map[string]string // arbitrary capability tags
	Status          WorkerStatus
	LastHeartbeatAt *time.Time
	RegisteredAt    time.Time
	UpdatedAt       time.Time
}

Worker represents a registered sqi-worker agent. Workers self-report their capabilities at registration; the server persists the reported values and uses them for task matching.

type WorkerSortField

type WorkerSortField string

WorkerSortField is a column by which WorkerStore.ListWorkers results can be ordered.

const (
	// WorkerSortByHostname orders workers alphabetically by hostname (default).
	WorkerSortByHostname WorkerSortField = "hostname"
	// WorkerSortByStatus orders workers alphabetically by status string.
	WorkerSortByStatus WorkerSortField = "status"
	// WorkerSortByRegisteredAt orders workers by registration time.
	WorkerSortByRegisteredAt WorkerSortField = "registered_at"
	// WorkerSortByLastHeartbeatAt orders workers by most recent heartbeat time.
	WorkerSortByLastHeartbeatAt WorkerSortField = "last_heartbeat_at"
)

type WorkerStatus

type WorkerStatus string

WorkerStatus is the operational state of a worker as known to the server.

const (
	// WorkerStatusOnline means the worker is connected and accepting tasks.
	WorkerStatusOnline WorkerStatus = "online"
	// WorkerStatusOffline means the worker has not sent a heartbeat within the
	// configured timeout and is presumed unreachable.
	WorkerStatusOffline WorkerStatus = "offline"
	// WorkerStatusDisabled means an operator has administratively paused the
	// worker; it will not receive new task assignments until re-enabled.
	WorkerStatusDisabled WorkerStatus = "disabled"
)

type WorkerStore

type WorkerStore interface {
	// RegisterWorker inserts or replaces the worker record for the given ID.
	// Called by the server when a worker sends its registration message.
	// If the worker ID already exists its record is updated in full.
	RegisterWorker(ctx context.Context, worker Worker) (Worker, error)

	// GetWorker returns the worker with the given ID, or [ErrNotFound].
	GetWorker(ctx context.Context, id string) (Worker, error)

	// ListWorkers returns a paginated, filtered, and sorted page of workers
	// matching opts. Call [Pagination.Validate] on opts.Pagination before
	// passing it to ensure sensible defaults are applied.
	ListWorkers(ctx context.Context, opts ListWorkersOptions) (Page[Worker], error)

	// UpdateWorker replaces the mutable capability fields of an existing
	// worker (everything except ID and RegisteredAt) and updates UpdatedAt.
	// Returns [ErrNotFound] if the worker does not exist.
	UpdateWorker(ctx context.Context, worker Worker) (Worker, error)

	// UpdateWorkerStatus sets the status of the worker and updates UpdatedAt.
	// Returns [ErrNotFound] if the worker does not exist.
	UpdateWorkerStatus(ctx context.Context, id string, status WorkerStatus) error

	// UpdateWorkerHeartbeat records the most recent heartbeat time for the
	// given worker. This is a hot path; implementations should use a single
	// UPDATE statement with no unnecessary reads.
	UpdateWorkerHeartbeat(ctx context.Context, id string, at time.Time) error

	// ListStaleWorkers returns workers whose last heartbeat is older than
	// before and whose status is [WorkerStatusOnline]. Used by the heartbeat
	// timeout sweep to find workers to mark offline.
	ListStaleWorkers(ctx context.Context, before time.Time) ([]Worker, error)

	// CountIdleWorkers returns the number of online workers in the given farm
	// that have no task currently in [TaskStatusAssigned] or
	// [TaskStatusRunning] state. An empty farmID matches all farms.
	// Used by the scheduler to update the [SchedulerIdleWorkers] Prometheus
	// gauge.
	CountIdleWorkers(ctx context.Context, farmID string) (int, error)

	// DeleteWorker hard-deletes the worker record with the given ID. Returns
	// [ErrNotFound] if no such worker exists. Task and task-attempt rows that
	// reference the worker by ID are left intact (the ID lives on as a
	// snapshot); callers are responsible for ensuring the worker has no
	// in-flight work before removing it.
	DeleteWorker(ctx context.Context, id string) error

	// DeleteOfflineWorkersBefore hard-deletes every worker in
	// [WorkerStatusOffline] whose LastHeartbeatAt is strictly before cutoff,
	// and returns the deleted records. Workers in any other status (including
	// administratively disabled) are never touched. Used by the scheduler's
	// offline-retention sweep to bound the growth of the worker table.
	DeleteOfflineWorkersBefore(ctx context.Context, cutoff time.Time) ([]Worker, error)
}

WorkerStore is the persistence interface for Worker records.

Directories

Path Synopsis
Package fake provides an in-memory implementation of store.Store for unit tests that must avoid touching the filesystem.
Package fake provides an in-memory implementation of store.Store for unit tests that must avoid touching the filesystem.
Package migrations embeds the SQL migration files used by goose to manage the sqi-server SQLite schema.
Package migrations embeds the SQL migration files used by goose to manage the sqi-server SQLite schema.
Package sqlite provides a SQLite-backed implementation of store.Store.
Package sqlite provides a SQLite-backed implementation of store.Store.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL