Documentation
¶
Overview ¶
Package store defines the storage interface for sqi-server and the domain model types shared across all components.
Design ¶
Store is a composite interface that embeds one sub-interface per aggregate (Farm, Queue, Worker, Job, Step, Task, etc.). Callers declare the narrowest interface they need rather than depending on the full Store:
// Scheduler only needs tasks and workers. func NewScheduler(tasks store.TaskStore, workers store.WorkerStore) *Scheduler // REST handler only needs jobs and queues. func NewJobHandler(jobs store.JobStore, queues store.QueueStore) *JobHandler
The SQLite-backed implementation implements Store in full. The in-memory fake also implements Store in full, used by unit tests that must not touch the filesystem.
JSON columns ¶
Fields that are stored as JSON in SQLite (e.g. Worker.Tags, Task.Parameters) are typed as Go maps or slices. The SQLite implementation is responsible for marshaling and unmarshaling; callers work with native Go types throughout.
Errors ¶
Methods return ErrNotFound when the requested row does not exist, and ErrConflict when a uniqueness constraint would be violated. All other errors are driver-level and should be treated as opaque.
Index ¶
- Constants
- Variables
- type AttemptStatus
- type AuditEntry
- type AuditStore
- type DeletedJob
- type Farm
- type FarmStore
- type GPUInfo
- type Job
- type JobSortField
- type JobStatus
- type JobStore
- type ListJobsOptions
- type ListQueuesOptions
- type ListTasksOptions
- type ListWorkersOptions
- type LogStream
- type Page
- type Pagination
- type Queue
- type QueueSortField
- type QueueStore
- type SortDir
- type Step
- type StepAmountRequirement
- type StepAttributeRequirement
- type StepHostRequirements
- type StepStatus
- type StepStore
- type StorageLocation
- type StorageLocationStore
- type StorageLocationType
- type Store
- type Task
- type TaskAttempt
- type TaskAttemptStore
- type TaskLog
- type TaskLogStore
- type TaskSortField
- type TaskStatus
- type TaskStore
- type TemplateFormat
- type UsageClaim
- type UsageClaimStore
- type UsagePool
- type UsagePoolClaim
- type UsagePoolStore
- type UsagePoolUtilization
- type Worker
- type WorkerSortField
- type WorkerStatus
- type WorkerStore
Constants ¶
const ( // DefaultLimit is the page size used when the caller does not specify one. DefaultLimit = 50 // MaxLimit is the maximum page size accepted from callers. Requests above // this are clamped rather than rejected, matching the behavior of most // REST pagination conventions. MaxLimit = 1000 )
Variables ¶
var ErrConflict = errors.New("store: conflict")
ErrConflict is returned when an insert or update would violate a uniqueness constraint (e.g. creating a farm with a name that already exists).
var ErrInvalidPagination = errors.New("store: invalid pagination parameters")
ErrInvalidPagination is returned when Pagination contains values that cannot be corrected by clamping (reserved for future use).
var ErrNotFound = errors.New("store: not found")
ErrNotFound is returned by Get* methods when the requested row does not exist. Use errors.Is to check:
job, err := store.GetJob(ctx, id)
if errors.Is(err, store.ErrNotFound) { ... }
var ErrUsageAtCapacity = errors.New("store: usage pool at capacity")
ErrUsageAtCapacity is returned by UsageClaimStore.TryClaimSlots when one or more required usage pools are saturated (active claims have reached MaxConcurrent). The task should remain ready for reassignment on the next scheduler tick.
Functions ¶
This section is empty.
Types ¶
type AttemptStatus ¶
type AttemptStatus string
AttemptStatus is the terminal-or-active state of a single task execution.
const ( // AttemptStatusRunning means the worker is actively executing the task. AttemptStatusRunning AttemptStatus = "running" // AttemptStatusSucceeded means the task exited cleanly. AttemptStatusSucceeded AttemptStatus = "succeeded" // AttemptStatusFailed means the task exited with a non-zero code or the // worker reported a fatal error. AttemptStatusFailed AttemptStatus = "failed" // AttemptStatusCanceled means the attempt was interrupted before completion. AttemptStatusCanceled AttemptStatus = "canceled" )
type AuditEntry ¶
type AuditEntry struct {
ID string
EntityType string // e.g. "job", "worker", "queue", "farm"
EntityID string // ID of the affected entity
Action string // e.g. "submitted", "canceled", "priority_changed"
Actor string // authenticated identity; empty in no-auth deployments
Details map[string]any // action-specific context
CreatedAt time.Time
}
AuditEntry is a single append-only record in the audit log. The log captures state-changing actions performed by users or internal components so that operators can reconstruct what happened and who did it.
Details is a free-form map for action-specific context — before/after values, failure reasons, parameter diffs, etc. The interpretation of Details is defined by the code that writes the entry.
type AuditStore ¶
type AuditStore interface {
// AppendAuditEntry inserts a new audit log entry. The caller must populate
// all fields including a unique ID and CreatedAt.
AppendAuditEntry(ctx context.Context, entry AuditEntry) error
// ListAuditEntries returns all audit entries for the given entity,
// ordered by CreatedAt ascending. Pass empty strings to list all entries
// (for administrative views).
ListAuditEntries(ctx context.Context, entityType, entityID string) ([]AuditEntry, error)
}
AuditStore is the persistence interface for AuditEntry records. The audit log is append-only; there are no update or delete methods.
type DeletedJob ¶
DeletedJob is the summary of a job removed by JobStore.DeleteTerminalJobsBefore, carrying the fields a WebSocket "removed" event needs.
type Farm ¶
type Farm struct {
ID string
Name string
Description string
MaxConcurrentTasks int // 0 = unlimited; farm-level policy
CreatedAt time.Time
UpdatedAt time.Time
}
Farm is the top-level organizational grouping in sqi. A deployment typically has one farm, but large facilities may define multiple to represent distinct infrastructure pools, studios, or departments.
type FarmStore ¶
type FarmStore interface {
// CreateFarm inserts a new farm. The caller must populate all fields;
// the implementation persists them as-is. Returns [ErrConflict] if a
// farm with the same name already exists.
CreateFarm(ctx context.Context, farm Farm) (Farm, error)
// GetFarm returns the farm with the given ID, or [ErrNotFound].
GetFarm(ctx context.Context, id string) (Farm, error)
// ListFarms returns all farms ordered by name.
ListFarms(ctx context.Context) ([]Farm, error)
// UpdateFarm replaces the mutable fields (Name, Description,
// MaxConcurrentTasks) of an existing farm and updates UpdatedAt.
// Returns [ErrNotFound] if the farm does not exist, or [ErrConflict]
// if the new name is already taken.
UpdateFarm(ctx context.Context, farm Farm) (Farm, error)
// DeleteFarm removes a farm by ID. Returns [ErrNotFound] if it does not
// exist. The caller is responsible for ensuring no queues or workers
// reference the farm before deleting it.
DeleteFarm(ctx context.Context, id string) error
}
FarmStore is the persistence interface for Farm records.
type GPUInfo ¶
type GPUInfo struct {
Vendor string `json:"vendor,omitempty"`
Model string `json:"model,omitempty"`
// VRAMMb is the VRAM capacity of each GPU device in mebibytes.
// All devices are assumed to be identical (see type-level note above).
VRAMMb int `json:"vram_mb,omitempty"`
Count int `json:"count,omitempty"`
}
GPUInfo describes the GPU(s) available on a worker host.
Phase 1 assumes a homogeneous GPU configuration: all GPUs on the host are the same model with the same VRAM. VRAMMb is the per-device VRAM capacity; Count is the number of identical devices. Mixed-GPU workers (e.g. a render card alongside a display adapter) are not modeled — workers with heterogeneous GPUs should report the lowest common VRAM to avoid over-scheduling. A []GPUDevice slice should replace this struct when heterogeneous per-device tracking is required.
type Job ¶
type Job struct {
ID string
FarmID string
QueueID string
Name string
Owner string // human responsible for the job
Submitter string // authenticated identity that placed the job
Priority int // higher = more urgent; default 50
Status JobStatus
Project string
RawTemplate string // verbatim OpenJD YAML or JSON as submitted
TemplateFormat TemplateFormat
// Parameters holds the fully-bound job-parameter values produced by
// BindJobParameters at submission time, including applied defaults.
// Nil or empty for jobs with no declared parameters or submitted before
// the parameters-persistence migration (back-compat: scheduler falls back
// to template defaults).
Parameters map[string]string
CreatedAt time.Time
UpdatedAt time.Time
StartedAt *time.Time
CompletedAt *time.Time
}
Job is the top-level unit of work submitted to sqi-server. It holds the verbatim OpenJD template alongside derived metadata used by the scheduler.
type JobSortField ¶
type JobSortField string
JobSortField is a column by which JobStore.ListJobs results can be ordered.
const ( // JobSortByCreatedAt orders jobs by submission time (default). JobSortByCreatedAt JobSortField = "created_at" // JobSortByPriority orders jobs by priority (higher values first when SortDesc). JobSortByPriority JobSortField = "priority" // JobSortByStatus orders jobs alphabetically by status string. JobSortByStatus JobSortField = "status" // JobSortByUpdatedAt orders jobs by the time of the most recent change. JobSortByUpdatedAt JobSortField = "updated_at" // JobSortByName orders jobs alphabetically by name. JobSortByName JobSortField = "name" )
type JobStatus ¶
type JobStatus string
JobStatus is the lifecycle state of a job as a whole.
const ( // JobStatusPending means the job has been submitted but no tasks are // running yet (e.g. waiting on step dependency resolution or queue capacity). JobStatusPending JobStatus = "pending" // JobStatusRunning means at least one task is currently assigned or running. JobStatusRunning JobStatus = "running" // JobStatusPaused means the job has been administratively paused; no new // task assignments will be made until it is resumed. JobStatusPaused JobStatus = "paused" // JobStatusCompleted means all tasks across all steps succeeded. JobStatusCompleted JobStatus = "completed" // JobStatusFailed means one or more tasks failed and the job cannot proceed. JobStatusFailed JobStatus = "failed" // JobStatusCanceled means the job was explicitly canceled by a user or API // call before it could complete. JobStatusCanceled JobStatus = "canceled" )
type JobStore ¶
type JobStore interface {
// CreateJob inserts a new job with all fields populated by the caller.
CreateJob(ctx context.Context, job Job) (Job, error)
// GetJob returns the job with the given ID, or [ErrNotFound].
GetJob(ctx context.Context, id string) (Job, error)
// ListJobs returns a paginated, filtered, and sorted page of jobs matching
// opts. Call [Pagination.Validate] on opts.Pagination before passing it to
// ensure sensible defaults are applied.
ListJobs(ctx context.Context, opts ListJobsOptions) (Page[Job], error)
// UpdateJob replaces the mutable user-settable fields of an existing job
// (farm_id, queue_id, name, owner, submitter, priority, project,
// raw_template, template_format) and updates UpdatedAt.
//
// status, started_at, and completed_at are lifecycle columns and are
// intentionally excluded — use [UpdateJobStatus] or [CancelJobStatus]
// for those. The returned Job reflects the current DB state of all columns.
//
// Returns [ErrNotFound] if the job does not exist.
UpdateJob(ctx context.Context, job Job) (Job, error)
// UpdateJobStatus transitions a job to a new status and updates UpdatedAt.
// If the new status is [JobStatusRunning] and StartedAt is nil, StartedAt
// is set to the current time. Terminal statuses set CompletedAt.
// Returns [ErrNotFound] if the job does not exist.
UpdateJobStatus(ctx context.Context, id string, status JobStatus) error
// CancelJobStatus transitions a job to [JobStatusCanceled] only when the
// job is not already in a terminal state, preventing a race where a
// concurrent scheduler transition to completed/failed would be overwritten.
//
// - Non-terminal job → canceled, returns nil.
// - Already canceled → no-op, returns nil (idempotent).
// - Already completed or failed → returns [ErrConflict].
// - Job not found → returns [ErrNotFound].
CancelJobStatus(ctx context.Context, id string) error
// DemoteStalledJobs returns any job in [JobStatusRunning] that currently has
// no task in [TaskStatusAssigned] or [TaskStatusRunning] — yet still has at
// least one schedulable (ready or pending) task — back to [JobStatusPending],
// stamping updated_at = now. It reconciles the [JobStatusRunning] invariant
// ("at least one task is currently assigned or running") after the heartbeat
// sweep or stale-assigned reaper returns a job's last in-flight task to the
// ready queue with no worker to pick it up, leaving the job spuriously marked
// running. Jobs whose tasks are all terminal are left untouched so the
// completion logic can finalize them to completed/failed. StartedAt is
// preserved (the job did start). Returns the IDs of the demoted jobs.
DemoteStalledJobs(ctx context.Context, now time.Time) ([]string, error)
// DeleteJob hard-deletes a job and every row that belongs to it, in one
// transaction and FK-safe order: usage_claims (for the job's task attempts),
// task_logs, task_attempts, tasks, steps, then the jobs row.
// Returns [ErrNotFound] when the job does not exist. The audit_log is left
// intact (it references entities by id, not by foreign key).
DeleteJob(ctx context.Context, id string) error
// DeleteTerminalJobsBefore hard-deletes terminal jobs whose completion time
// (completed_at, falling back to updated_at when NULL) is before cutoff, and
// returns a summary of each removed job. completed and canceled jobs are
// always eligible; failed jobs are included only when includeFailed is true.
// Active jobs are never removed. Each removed job's children are deleted via
// the same cascade as [DeleteJob].
DeleteTerminalJobsBefore(ctx context.Context, cutoff time.Time, includeFailed bool) ([]DeletedJob, error)
}
JobStore is the persistence interface for Job records.
type ListJobsOptions ¶
type ListJobsOptions struct {
// Filters
FarmID string
QueueID string
Status JobStatus // empty = all statuses
Owner string
Project string
// Search is a case-insensitive substring matched against name, id, owner,
// and project. Empty = no search filter.
Search string
// Ordering — zero values use JobSortByCreatedAt / SortAsc.
SortBy JobSortField
SortDir SortDir
// Pagination — call Pagination.Validate() before use.
Pagination Pagination
}
ListJobsOptions filters and orders JobStore.ListJobs results. Zero values mean "no filter / use defaults".
type ListQueuesOptions ¶
type ListQueuesOptions struct {
// Filters
FarmID string
Paused *bool // nil = all, true = paused only, false = active only
// Ordering — zero values use QueueSortByName / SortAsc.
SortBy QueueSortField
SortDir SortDir
// Pagination — call Pagination.Validate() before use.
Pagination Pagination
}
ListQueuesOptions filters and orders QueueStore.ListQueues results. Zero values mean "no filter / use defaults".
type ListTasksOptions ¶
type ListTasksOptions struct {
// Filters
JobID string
StepID string
Status TaskStatus // empty = all statuses (mutually exclusive with Statuses)
Statuses []TaskStatus // IN-filter; takes precedence over Status when non-empty
WorkerID string // filter by assigned worker
// Ordering — zero values use TaskSortByCreatedAt / SortAsc.
SortBy TaskSortField
SortDir SortDir
// Pagination — call Pagination.Validate() before use.
Pagination Pagination
}
ListTasksOptions filters and orders TaskStore.ListTasks results. Zero values mean "no filter / use defaults".
type ListWorkersOptions ¶
type ListWorkersOptions struct {
// Filters
FarmID string
QueueID string
ComputeLocation string
Status WorkerStatus // empty = all statuses
// Search is a case-insensitive substring matched against name, hostname,
// id, and compute_location. Empty = no search filter.
Search string
// IncludeUnaffiliated, when true and FarmID is non-empty, also returns
// workers whose FarmID is empty (unaffiliated workers that accept tasks
// from any farm). Used by the scheduler's pickWorker so that workers
// started without an explicit farm configuration are still dispatched to.
IncludeUnaffiliated bool
// Ordering — zero values use WorkerSortByHostname / SortAsc.
SortBy WorkerSortField
SortDir SortDir
// Pagination — call Pagination.Validate() before use.
Pagination Pagination
}
ListWorkersOptions filters and orders WorkerStore.ListWorkers results. Zero values mean "no filter / use defaults".
type LogStream ¶
type LogStream string
LogStream identifies the output stream a log chunk was captured from.
type Page ¶
type Page[T any] struct { // Items holds the current page of results. It is never nil; an empty // page is represented as an empty slice. Items []T // Total is the total number of items matching the filter, regardless of // the current page window. Used by clients to compute page counts. Total int // Limit mirrors the effective limit used to produce this page. Limit int // Offset mirrors the effective offset used to produce this page. Offset int }
Page is the generic result type returned by paginated list queries. T is the item type (e.g. Job, Task, Worker).
type Pagination ¶
type Pagination struct {
// Limit is the maximum number of items to return. 0 means use
// [DefaultLimit]. Values above [MaxLimit] are clamped to [MaxLimit].
Limit int
// Offset is the zero-based index of the first item to return.
Offset int
}
Pagination carries cursor-free (limit/offset) pagination parameters. Zero values are treated as "use defaults"; call Pagination.Validate before passing to a store method to ensure limits are applied.
func (*Pagination) Validate ¶
func (p *Pagination) Validate() error
Validate clamps Limit into the range [1, MaxLimit], applying DefaultLimit when Limit is zero. Negative Offset is reset to zero. It returns an error only for values that cannot be silently corrected (currently none — all out-of-range values are clamped).
type Queue ¶
type Queue struct {
ID string
FarmID string
Name string
Description string
Priority int
MaxConcurrentTasks int // 0 = unlimited
Paused bool
CreatedAt time.Time
UpdatedAt time.Time
}
Queue belongs to a Farm and is the container into which jobs are submitted. Scheduling policy (priority, concurrency limits, paused state) is configured at the queue level and may be overridden per job.
type QueueSortField ¶
type QueueSortField string
QueueSortField is a column by which QueueStore.ListQueues results can be ordered.
const ( // QueueSortByName orders queues alphabetically by name (default). QueueSortByName QueueSortField = "name" // QueueSortByPriority orders queues by priority (higher values first when // SortDesc). QueueSortByPriority QueueSortField = "priority" // QueueSortByCreatedAt orders queues by creation time. QueueSortByCreatedAt QueueSortField = "created_at" )
type QueueStore ¶
type QueueStore interface {
// CreateQueue inserts a new queue. Returns [ErrConflict] if a queue with
// the same name already exists within the same farm.
CreateQueue(ctx context.Context, queue Queue) (Queue, error)
// GetQueue returns the queue with the given ID, or [ErrNotFound].
GetQueue(ctx context.Context, id string) (Queue, error)
// ListQueues returns a paginated, filtered, and sorted page of queues
// matching opts. Call [Pagination.Validate] on opts.Pagination before
// passing it to ensure sensible defaults are applied.
ListQueues(ctx context.Context, opts ListQueuesOptions) (Page[Queue], error)
// UpdateQueue replaces the mutable fields of an existing queue and updates
// UpdatedAt. Returns [ErrNotFound] or [ErrConflict] as appropriate.
UpdateQueue(ctx context.Context, queue Queue) (Queue, error)
// DeleteQueue removes a queue by ID. Returns [ErrNotFound] if it does not
// exist. The caller is responsible for ensuring no jobs reference the
// queue before deleting it.
DeleteQueue(ctx context.Context, id string) error
}
QueueStore is the persistence interface for Queue records.
type Step ¶
type Step struct {
ID string
JobID string
Name string
DependsOn []string // names of steps that must complete before this one
StepOrder int // position within the job for deterministic ordering
Status StepStatus
// HostRequirements declares the worker capabilities required by this step.
// Nil means any capable worker is eligible.
// Populated from the OpenJD hostRequirements block at submission time.
HostRequirements *StepHostRequirements
// ComputeLocation constrains tasks to workers in the named compute location
// (e.g. "onprem_linux", "cloud_aws_us_east"). Empty means any location is
// eligible. Mirrors the "attr.worker.computelocation" attribute in
// HostRequirements and is stored separately for fast SQL-level pre-filtering.
ComputeLocation string
CreatedAt time.Time
UpdatedAt time.Time
}
Step is one stage within a Job. Steps may depend on other steps; a step's tasks are not scheduled until all its dependencies have reached StepStatusCompleted.
type StepAmountRequirement ¶
type StepAmountRequirement struct {
// Name is the capability name, e.g. "amount.worker.vcpu".
Name string `json:"name"`
// Min, if non-nil, is the minimum acceptable value as a decimal string.
// The scheduler parses it with strconv.ParseFloat for comparison.
Min *string `json:"min,omitempty"`
// Max, if non-nil, is the maximum acceptable value as a decimal string.
Max *string `json:"max,omitempty"`
}
StepAmountRequirement is a single quantifiable host capability requirement.
type StepAttributeRequirement ¶
type StepAttributeRequirement struct {
// Name is the capability name, e.g. "attr.worker.os.family".
Name string `json:"name"`
// AnyOf, if non-empty, requires the host attribute to equal at least one
// of the listed values. String comparison is case-insensitive for
// well-known attributes (os.family, computelocation); exact for custom tags.
AnyOf []string `json:"any_of,omitempty"`
// AllOf, if non-empty, requires the host attribute to equal every listed
// value. Useful for multi-valued tag semantics (e.g. requiring two
// software packages to both be installed).
AllOf []string `json:"all_of,omitempty"`
}
StepAttributeRequirement is a single categorical host capability requirement.
type StepHostRequirements ¶
type StepHostRequirements struct {
// Amounts are quantifiable requirements such as CPUs, RAM, or GPU VRAM.
// Usage pool requirements use the "amount.worker.usagepool." prefix and
// are also listed in UsagePools for quick iteration.
Amounts []StepAmountRequirement `json:"amounts,omitempty"`
// Attributes are categorical requirements such as OS family or custom tags.
// The "attr.worker.computelocation" entry, if present, is mirrored as the
// enclosing [Step.ComputeLocation] field for SQL-level pre-filtering.
Attributes []StepAttributeRequirement `json:"attributes,omitempty"`
// UsagePools lists the names of [UsagePool] records that must have
// remaining capacity before a task from this step can be assigned.
// Derived from amounts named "amount.worker.usagepool.<name>" at
// submission time.
UsagePools []string `json:"usage_pools,omitempty"`
}
StepHostRequirements is the scheduler-facing representation of a step's hostRequirements block, normalised from the raw OpenJD model at submission time. The scheduler's matching logic reads these fields to determine whether a given worker can run a task.
Capability names follow the conventions established in the OpenJD spec and documented in [matcher.go]:
- "attr.worker.os.family" → worker OS family ("linux", "windows", "darwin")
- "attr.worker.os.version" → worker OS version string
- "attr.worker.computelocation" → worker compute location name
- "attr.worker.tag.<key>" → arbitrary worker tag value
- "amount.worker.vcpu" → worker CPU count
- "amount.worker.memory.mb" → worker RAM in MiB
- "amount.worker.gpu.count" → worker GPU count
- "amount.worker.gpu.memory.mb" → worker GPU VRAM in MiB
- "amount.worker.usagepool.<name>" → named usage pool (capacity check)
type StepStatus ¶
type StepStatus string
StepStatus is the lifecycle state of a step within a job.
const ( // StepStatusPending means the step is waiting for its dependencies to complete. StepStatusPending StepStatus = "pending" // StepStatusReady means all dependencies have succeeded; tasks can be scheduled. StepStatusReady StepStatus = "ready" // StepStatusRunning means at least one task in this step is running. StepStatusRunning StepStatus = "running" // StepStatusCompleted means all tasks in this step succeeded. StepStatusCompleted StepStatus = "completed" // StepStatusFailed means one or more tasks failed. StepStatusFailed StepStatus = "failed" // StepStatusCanceled means the step was canceled, typically because the // parent job was canceled. StepStatusCanceled StepStatus = "canceled" )
type StepStore ¶
type StepStore interface {
// CreateStep inserts a new step. The (JobID, Name) pair must be unique
// within the job; returns [ErrConflict] if violated.
CreateStep(ctx context.Context, step Step) (Step, error)
// GetStep returns the step with the given ID, or [ErrNotFound].
GetStep(ctx context.Context, id string) (Step, error)
// ListSteps returns all steps for the given job, ordered by StepOrder
// ascending.
ListSteps(ctx context.Context, jobID string) ([]Step, error)
// UpdateStepStatus transitions a step to a new status and updates
// UpdatedAt. Returns [ErrNotFound] if the step does not exist.
UpdateStepStatus(ctx context.Context, id string, status StepStatus) error
}
StepStore is the persistence interface for Step records.
type StorageLocation ¶
type StorageLocation struct {
ID string
Name string
Type StorageLocationType
Description string
Roots map[string]string // compute_location_name -> root path
CreatedAt time.Time
UpdatedAt time.Time
}
StorageLocation is a named, globally-registered storage root. Jobs reference locations by name with a relative path; workers resolve the name to a concrete path for their compute location at execution time.
Roots maps a compute location name (or "default") to the root path for that location. Example:
Roots = map[string]string{
"default": "/mnt/nas/shows",
"windows_workers": `Z:\shows`,
"cloud_aws": "s3://studio-bucket/shows",
}
type StorageLocationStore ¶
type StorageLocationStore interface {
// CreateStorageLocation inserts a new location. Returns [ErrConflict] if
// a location with the same name already exists.
CreateStorageLocation(ctx context.Context, loc StorageLocation) (StorageLocation, error)
// GetStorageLocation returns the location with the given ID, or [ErrNotFound].
GetStorageLocation(ctx context.Context, id string) (StorageLocation, error)
// GetStorageLocationByName returns the location with the given name, or
// [ErrNotFound]. Used by the path resolver to look up locations by their
// symbolic name at task dispatch time.
GetStorageLocationByName(ctx context.Context, name string) (StorageLocation, error)
// ListStorageLocations returns all locations ordered by name.
ListStorageLocations(ctx context.Context) ([]StorageLocation, error)
// UpdateStorageLocation replaces the mutable fields of an existing
// location and updates UpdatedAt. Returns [ErrNotFound] or [ErrConflict].
UpdateStorageLocation(ctx context.Context, loc StorageLocation) (StorageLocation, error)
// DeleteStorageLocation removes a location by ID. Returns [ErrNotFound]
// if it does not exist.
DeleteStorageLocation(ctx context.Context, id string) error
}
StorageLocationStore is the persistence interface for StorageLocation records.
type StorageLocationType ¶
type StorageLocationType string
StorageLocationType identifies the backing storage technology.
const ( // StorageLocationTypeFilesystem is a POSIX or Windows filesystem path. StorageLocationTypeFilesystem StorageLocationType = "filesystem" // StorageLocationTypeS3 is an S3-compatible object store (AWS S3, // Backblaze B2, Cloudflare R2, MinIO, etc.). StorageLocationTypeS3 StorageLocationType = "s3" )
type Store ¶
type Store interface {
FarmStore
QueueStore
StorageLocationStore
UsagePoolStore
UsageClaimStore
WorkerStore
JobStore
StepStore
TaskStore
TaskAttemptStore
TaskLogStore
AuditStore
io.Closer
}
Store is the top-level storage interface for sqi-server. It is composed of one sub-interface per aggregate so callers can depend on the narrowest slice they need.
Store embeds io.Closer; implementations must release any open database connections or file handles when Close is called.
type Task ¶
type Task struct {
ID string
JobID string // denormalized from Step for query efficiency
StepID string
Name string
Parameters map[string]string // resolved parameter values for this task
Status TaskStatus
AssignedWorkerID string // empty when unassigned
AssignedAt *time.Time // nil when unassigned
CreatedAt time.Time
UpdatedAt time.Time
// RequiredCores is the task's declared CPU reservation (OpenJD
// amount.worker.vcpu min). Nil means undeclared — the scheduler treats the
// cost as the running worker's full CPUCount (one such task per worker).
RequiredCores *int
}
Task is the atomic unit of work — one process on one worker. Tasks are derived from an OpenJD step's parameter space expansion.
type TaskAttempt ¶
type TaskAttempt struct {
ID string
TaskID string
WorkerID string
SessionID string // OpenJD session ID; may be empty for legacy workers
AttemptNumber int // 1-based; incremented on each retry
Status AttemptStatus
ExitCode *int // nil while running or if the process was signaled
StartedAt time.Time
EndedAt *time.Time // nil while running
CreatedAt time.Time
}
TaskAttempt records a single execution of a Task on a specific worker. A task may have multiple attempts if it is retried after failure.
SessionID is the OpenJD session identifier reported by the worker. Multiple task attempts that share the same session ID ran within the same OpenJD Session on the same worker — they shared a working directory and environment setup. See docs/architecture.md ("SQLite schema overview") for why sessions are not a first-class table.
type TaskAttemptStore ¶
type TaskAttemptStore interface {
// CreateTaskAttempt inserts a new attempt record. Called when the server
// assigns a task to a worker and the worker acknowledges it.
CreateTaskAttempt(ctx context.Context, attempt TaskAttempt) (TaskAttempt, error)
// GetTaskAttempt returns the attempt with the given ID, or [ErrNotFound].
GetTaskAttempt(ctx context.Context, id string) (TaskAttempt, error)
// LatestTaskAttempt returns the attempt with the highest AttemptNumber for
// the given task, or [ErrNotFound] if no attempts exist yet. Used by the
// scheduler to determine the correct AttemptNumber when creating a retry.
LatestTaskAttempt(ctx context.Context, taskID string) (TaskAttempt, error)
// ListTaskAttempts returns all attempts for the given task, ordered by
// AttemptNumber ascending.
ListTaskAttempts(ctx context.Context, taskID string) ([]TaskAttempt, error)
// UpdateTaskAttempt replaces the mutable fields of an existing attempt
// (Status, ExitCode, EndedAt). Returns [ErrNotFound] if it does not exist.
UpdateTaskAttempt(ctx context.Context, attempt TaskAttempt) (TaskAttempt, error)
// TerminateWorkerAttempts marks all running [TaskAttempt] records for tasks
// currently assigned to workerID as the given terminal status with the
// supplied end time. Called by the heartbeat sweep before reclaiming tasks
// from an offline worker so that each attempt has a closed EndedAt.
// Returns the number of attempts updated.
TerminateWorkerAttempts(ctx context.Context, workerID string, status AttemptStatus, endedAt time.Time) (int, error)
// CancelJobAttempts marks all running [TaskAttempt] records for tasks
// belonging to the given job as [AttemptStatusCanceled] with the supplied
// end time. Should be called before [TaskStore.CancelJobTasks] so
// that attempts are closed while the tasks still carry their assigned worker.
// Returns the number of attempts updated.
CancelJobAttempts(ctx context.Context, jobID string, endedAt time.Time) (int, error)
}
TaskAttemptStore is the persistence interface for TaskAttempt records.
type TaskLog ¶
type TaskLog struct {
ID string
TaskID string
AttemptID string
// SeqNum is the worker-assigned monotonic sequence number for this chunk
// within the attempt. Starts at 1; callers must not assume it is
// contiguous — gaps can appear if chunks are dropped or reordered.
SeqNum int64
// NATSSeq is the JetStream message sequence number from the SQI_LOGS
// stream. Used as a stable, dense offset cursor for the log-tail REST
// endpoint. Stored as int64 to match SQLite's INTEGER affinity; NATS
// stream sequences are uint64 but will never approach int64 max in practice.
NATSSeq int64
// At is the worker's local timestamp when the chunk was captured.
At time.Time
// ReceivedAt is the server's clock time when the chunk was persisted.
// Monotonically increasing within a server instance; useful for
// detecting clock skew between worker and server.
ReceivedAt time.Time
// Stream is "stdout" or "stderr".
Stream LogStream
// Data is the raw log text.
Data string
}
TaskLog is one chunk of output produced by a running task.
Workers publish log chunks to task.logs.<taskID> as output is produced. The server persists each chunk here with:
- A monotonic worker-side sequence number (SeqNum) per attempt, starting at 1 and incrementing by 1. This allows the REST log-tail endpoint to return chunks in the order the worker produced them regardless of NATS delivery order.
- A NATS stream sequence number (NATSSeq) used as the primary DB insertion order and for efficient cursor-based pagination.
- A server-side receipt timestamp (ReceivedAt) for audit purposes.
type TaskLogStore ¶
type TaskLogStore interface {
// CreateTaskLog inserts a new log chunk. The caller must populate all
// fields including a unique ID.
CreateTaskLog(ctx context.Context, log TaskLog) (TaskLog, error)
// ListTaskLogs returns log chunks for a task attempt in NATSSeq ascending
// order. Pass afterNATSSeq = 0 to start from the beginning; pass the
// NATSSeq of the last received chunk to page forward. Limit controls the
// maximum number of chunks returned.
//
// This method is used by the REST log-tail endpoint and by the WebSocket
// log-streaming handler.
ListTaskLogs(ctx context.Context, attemptID string, afterNATSSeq int64, limit int) ([]TaskLog, error)
}
TaskLogStore is the persistence interface for TaskLog records.
type TaskSortField ¶
type TaskSortField string
TaskSortField is a column by which TaskStore.ListTasks results can be ordered.
const ( // TaskSortByCreatedAt orders tasks by creation time (default). TaskSortByCreatedAt TaskSortField = "created_at" // TaskSortByStatus orders tasks alphabetically by status string. TaskSortByStatus TaskSortField = "status" // TaskSortByUpdatedAt orders tasks by the time of the most recent change. TaskSortByUpdatedAt TaskSortField = "updated_at" // TaskSortByName orders tasks alphabetically by name. TaskSortByName TaskSortField = "name" )
type TaskStatus ¶
type TaskStatus string
TaskStatus is the lifecycle state of an individual task.
const ( // TaskStatusPending means the task exists but its step's dependencies have // not yet been satisfied. TaskStatusPending TaskStatus = "pending" // TaskStatusReady means the task is eligible for assignment; its step's // dependencies have completed. TaskStatusReady TaskStatus = "ready" // TaskStatusAssigned means the task has been assigned to a worker but the // worker has not yet confirmed it is running. TaskStatusAssigned TaskStatus = "assigned" // TaskStatusRunning means the worker has confirmed the task is executing. TaskStatusRunning TaskStatus = "running" // TaskStatusSucceeded means the task completed successfully. TaskStatusSucceeded TaskStatus = "succeeded" // TaskStatusFailed means the task exited with a non-zero code or the // worker reported a fatal error. TaskStatusFailed TaskStatus = "failed" // TaskStatusCanceled means the task was explicitly canceled before it could // complete. TaskStatusCanceled TaskStatus = "canceled" )
type TaskStore ¶
type TaskStore interface {
// CreateTask inserts a new task. The caller must populate all fields
// including a unique ID.
CreateTask(ctx context.Context, task Task) (Task, error)
// GetTask returns the task with the given ID, or [ErrNotFound].
GetTask(ctx context.Context, id string) (Task, error)
// ListTasks returns a paginated, filtered, and sorted page of tasks
// matching opts. Call [Pagination.Validate] on opts.Pagination before
// passing it to ensure sensible defaults are applied.
ListTasks(ctx context.Context, opts ListTasksOptions) (Page[Task], error)
// UpdateTaskStatus transitions a task to a new status and updates
// UpdatedAt. Returns [ErrNotFound] if the task does not exist.
UpdateTaskStatus(ctx context.Context, id string, status TaskStatus) error
// AssignTask atomically sets AssignedWorkerID, AssignedAt, and Status to
// [TaskStatusAssigned] for the given task. Returns [ErrNotFound] if the
// task does not exist.
AssignTask(ctx context.Context, id, workerID string, assignedAt time.Time) error
// ListReadyTasks returns up to limit tasks in [TaskStatusReady] that
// belong to non-paused queues within the given farm, ordered by:
// 1. job priority descending (higher values first),
// 2. job submission time ascending (earlier jobs win ties),
// 3. step order ascending (earlier steps in a job run before later ones),
// 4. task creation time ascending (stable tiebreaker within a step).
//
// Used by the scheduler's assignment loop.
ListReadyTasks(ctx context.Context, farmID string, limit int) ([]Task, error)
// ReclaimWorkerTasks resets all tasks assigned to workerID that are still
// in [TaskStatusAssigned] or [TaskStatusRunning] back to [TaskStatusReady]
// so they can be reassigned by the scheduler. Called by the heartbeat
// timeout sweep after a worker is marked offline.
// Returns the number of tasks reclaimed.
ReclaimWorkerTasks(ctx context.Context, workerID string) (int, error)
// ReclaimStaleAssignedTasks returns tasks stuck in [TaskStatusAssigned] whose
// assigned_at is older than cutoff to [TaskStatusReady], clearing
// assigned_worker_id and assigned_at so the scheduler can reassign them.
// Tasks in [TaskStatusRunning] are left untouched — only assignments that
// never started are reclaimed (e.g. the assignment message expired from the
// work stream before the worker pulled it). Returns the reclaimed tasks,
// each still carrying its pre-reset assigned_worker_id, so the caller can
// close their attempts and release usage-pool claims.
ReclaimStaleAssignedTasks(ctx context.Context, cutoff time.Time) ([]Task, error)
// CountActiveTasksInQueue returns the number of tasks for the given queue
// that are currently in [TaskStatusAssigned] or [TaskStatusRunning] state.
// Used by the scheduler's per-queue policy gate.
CountActiveTasksInQueue(ctx context.Context, queueID string) (int, error)
// CountActiveTasksInFarm returns the number of tasks across all queues in
// the given farm that are currently in [TaskStatusAssigned] or
// [TaskStatusRunning] state. Used by the scheduler's per-farm policy gate.
CountActiveTasksInFarm(ctx context.Context, farmID string) (int, error)
// CountReadyTasksByQueue returns the number of tasks in [TaskStatusReady]
// state for each queue within the given farm, keyed by queue ID.
// Queues with no ready tasks are omitted from the map.
// Used by the scheduler to update the [SchedulerQueueDepth] Prometheus
// gauge.
CountReadyTasksByQueue(ctx context.Context, farmID string) (map[string]int, error)
// CancelJobTasks transitions all non-terminal tasks for the given job to
// [TaskStatusCanceled], clearing AssignedWorkerID and AssignedAt on each,
// and returns the subset that were in [TaskStatusAssigned] or
// [TaskStatusRunning] at the time of the call (with their AssignedWorkerID
// intact) so the caller can publish NATS cancel signals to the appropriate
// workers.
//
// The SELECT and UPDATE run inside a single database transaction so no
// concurrent assignment can race between observation and cancellation.
// Tasks already in a terminal state (succeeded, failed, canceled) are not
// modified.
CancelJobTasks(ctx context.Context, jobID string, now time.Time) ([]Task, error)
// RetryTasks revives failed/canceled tasks so they can run again. It
// transitions every task of jobID in [TaskStatusFailed] or
// [TaskStatusCanceled] — or, when taskIDs is non-nil, only those of the
// given IDs that are failed/canceled — back to [TaskStatusPending]. Any of
// their enclosing steps that are currently in a terminal status are reset to
// [StepStatusPending], and the job itself is reset to [JobStatusPending]
// when it is currently terminal (failed/canceled); a non-terminal job is
// left unchanged. All updates run in a single transaction.
//
// Resetting to pending (rather than ready) lets the caller re-run
// [openjd.ResolveDependencies] to re-gate the revived tasks in dependency
// order. Tasks not in a terminal-retryable state are not modified. Returns
// the revived task rows (each with Status == pending), or an empty slice
// when nothing matched.
RetryTasks(ctx context.Context, jobID string, taskIDs []string, now time.Time) ([]Task, error)
// TransitionStepPendingTasks transitions every task of the given step that is
// currently in [TaskStatusPending] to status `to`, updates UpdatedAt, and
// returns the affected task rows. It is used to promote a step's tasks to
// [TaskStatusReady] once its dependencies resolve, and to cancel them when an
// upstream dependency fails.
//
// The transition is applied as a single statement covering all matching rows
// regardless of count, so it is not subject to the [MaxLimit] pagination
// ceiling. Tasks not in pending state are not modified.
TransitionStepPendingTasks(ctx context.Context, stepID string, to TaskStatus) ([]Task, error)
// CountTasksByJob returns the number of tasks for the given job keyed by
// status. Statuses with zero tasks are omitted from the returned map.
// Used by the REST layer to include aggregate task counts in job responses.
CountTasksByJob(ctx context.Context, jobID string) (map[TaskStatus]int, error)
// CommittedCores returns the sum of CPU-core reservations held by the worker
// across its assigned and running tasks: Σ COALESCE(required_cores,
// fullMachineCost). Callers pass the worker's CPUCount as fullMachineCost so
// an undeclared task (required_cores NULL) counts as the whole machine.
CommittedCores(ctx context.Context, workerID string, fullMachineCost int) (int, error)
// LeaseReadyTask atomically transitions a task from [TaskStatusReady] to
// [TaskStatusAssigned], setting assigned_worker_id and assigned_at. It
// returns true iff the task was still ready (exactly one row changed); a
// false return means another worker leased it first. This is the race guard
// for concurrent lease requests.
LeaseReadyTask(ctx context.Context, taskID, workerID string, now time.Time) (bool, error)
}
TaskStore is the persistence interface for Task records.
type TemplateFormat ¶
type TemplateFormat string
TemplateFormat identifies the serialization format of Job.RawTemplate.
const ( // TemplateFormatYAML means the raw template was submitted as YAML. TemplateFormatYAML TemplateFormat = "yaml" // TemplateFormatJSON means the raw template was submitted as JSON. TemplateFormatJSON TemplateFormat = "json" )
type UsageClaim ¶
type UsageClaim struct {
ID string
PoolID string
TaskAttemptID string
// DB column: checked_out_at (historic name; kept to avoid a column-rename migration).
ClaimedAt time.Time
ReleasedAt *time.Time // nil = currently active
}
UsageClaim records one active or completed claim of a UsagePool slot by a specific TaskAttempt. ReleasedAt is nil while the claim is active; it is set when the task attempt reaches a terminal state.
type UsageClaimStore ¶
type UsageClaimStore interface {
// CreateClaim inserts a new active claim for the given pool and task
// attempt. The (TaskAttemptID, PoolID) pair must be unique; returns
// [ErrConflict] if violated.
CreateClaim(ctx context.Context, claim UsageClaim) (UsageClaim, error)
// ReleaseClaim sets ReleasedAt on the claim with the given ID,
// marking it as no longer active. Returns [ErrNotFound] if it does not
// exist.
ReleaseClaim(ctx context.Context, id string, releasedAt time.Time) error
// ActiveClaimCount returns the number of claims for the given pool
// where ReleasedAt IS NULL. Used by the scheduler's admission check before
// assigning a task that requires the pool.
ActiveClaimCount(ctx context.Context, poolID string) (int, error)
// TryClaimSlots atomically verifies that every pool in claims has
// remaining capacity and, if so, inserts a [UsageClaim] row for each.
// The operation runs inside a single database transaction so the
// count-check and insert are indivisible.
//
// Returns [ErrUsageAtCapacity] if any pool is saturated; no rows are
// inserted in that case. Returns [ErrConflict] if any claim ID or
// (task_attempt_id, pool_id) pair already exists.
//
// taskAttemptID must reference an existing [TaskAttempt] row. claimedAt
// is the timestamp recorded on each new claim.
TryClaimSlots(ctx context.Context, taskAttemptID string, claims []UsagePoolClaim, claimedAt time.Time) error
// ReleaseAttemptClaims sets ReleasedAt on every active claim
// (released_at IS NULL) for the given taskAttemptID. Called when a task
// attempt reaches a terminal state so the usage pool slots are freed for
// reassignment.
//
// Returns the number of claims released (0 is not an error when the
// attempt held no claims).
ReleaseAttemptClaims(ctx context.Context, taskAttemptID string, releasedAt time.Time) (int, error)
// ReleaseJobClaims sets ReleasedAt on every active claim for any task
// attempt belonging to a task in the given job. Called during job
// cancellation to free all usage pool slots held by that job in a single
// operation, rather than iterating through individual attempts.
//
// Returns the number of claims released (0 is not an error when the job
// held no claims).
ReleaseJobClaims(ctx context.Context, jobID string, releasedAt time.Time) (int, error)
}
UsageClaimStore is the persistence interface for UsageClaim records. It is separated from UsagePoolStore because the scheduler is the primary caller and only needs claim operations, not pool CRUD.
type UsagePool ¶
type UsagePool struct {
ID string
Name string
ServerHint string // e.g. "10.0.0.50:5053"; empty = not configured
MaxConcurrent int
CreatedAt time.Time
UpdatedAt time.Time
}
UsagePool represents a tracked named concurrency limit. sqi enforces concurrency limits itself by counting active UsageClaim records; it does not query any upstream server directly.
ServerHint is informational only — the address of a vendor's software server (FLEXlm/RLM), useful for diagnostics when the pool represents software slots. Leave empty for non-vendor dimensions.
type UsagePoolClaim ¶
type UsagePoolClaim struct {
// ClaimID is a caller-supplied UUID for the claim row to be created.
ClaimID string
// PoolID is the [UsagePool] to claim from.
PoolID string
// PoolName is used in error messages and debug logging only.
PoolName string
// MaxConcurrent is the pool's current limit. The implementation uses this
// value inside the transaction to avoid a second pool lookup.
MaxConcurrent int
}
UsagePoolClaim describes a single usage pool slot to be claimed atomically by UsageClaimStore.TryClaimSlots.
type UsagePoolStore ¶
type UsagePoolStore interface {
// CreateUsagePool inserts a new pool. Returns [ErrConflict] if a pool
// with the same name already exists.
CreateUsagePool(ctx context.Context, pool UsagePool) (UsagePool, error)
// GetUsagePool returns the pool with the given ID, or [ErrNotFound].
GetUsagePool(ctx context.Context, id string) (UsagePool, error)
// ListUsagePools returns all pools ordered by name.
ListUsagePools(ctx context.Context) ([]UsagePool, error)
// ListUsagePoolUtilization returns all pools ordered by name, each paired with
// its current number of active claims. It computes utilization in a
// single query rather than one count per pool.
ListUsagePoolUtilization(ctx context.Context) ([]UsagePoolUtilization, error)
// UpdateUsagePool replaces the mutable fields of an existing pool and
// updates UpdatedAt. Returns [ErrNotFound] or [ErrConflict].
UpdateUsagePool(ctx context.Context, pool UsagePool) (UsagePool, error)
// DeleteUsagePool removes a pool by ID. Returns [ErrNotFound] if it
// does not exist.
DeleteUsagePool(ctx context.Context, id string) error
}
UsagePoolStore is the persistence interface for UsagePool records.
type UsagePoolUtilization ¶
type UsagePoolUtilization struct {
UsagePool
// InUse is the number of active claims (released_at IS NULL) for the pool.
InUse int
}
UsagePoolUtilization pairs a UsagePool with its current live utilization — the number of active (unreleased) claims against it. It is returned by UsagePoolStore.ListUsagePoolUtilization so callers can render "in use vs capacity" without an N+1 fan-out of per-pool count queries.
type Worker ¶
type Worker struct {
ID string
FarmID string
QueueID string // empty = no queue affinity
// Name is the worker's human-readable display label (the worker.name
// config field, default the hostname). Distinguishes multiple workers
// running on one host in the UI; may be empty for workers registered
// before this field existed, in which case callers fall back to Hostname.
Name string
Hostname string
IPAddress string
ComputeLocation string
OS string
OSVersion string
// Version is the sqi-worker build version the worker self-reports at
// registration (the worker binary's internal/version.Version). May be empty
// for workers registered before this field existed.
Version string
CPUCount int
RAMMb int
GPUInfo GPUInfo
Tags map[string]string // arbitrary capability tags
Status WorkerStatus
LastHeartbeatAt *time.Time
RegisteredAt time.Time
UpdatedAt time.Time
}
Worker represents a registered sqi-worker agent. Workers self-report their capabilities at registration; the server persists the reported values and uses them for task matching.
type WorkerSortField ¶
type WorkerSortField string
WorkerSortField is a column by which WorkerStore.ListWorkers results can be ordered.
const ( // WorkerSortByHostname orders workers alphabetically by hostname (default). WorkerSortByHostname WorkerSortField = "hostname" // WorkerSortByStatus orders workers alphabetically by status string. WorkerSortByStatus WorkerSortField = "status" // WorkerSortByRegisteredAt orders workers by registration time. WorkerSortByRegisteredAt WorkerSortField = "registered_at" // WorkerSortByLastHeartbeatAt orders workers by most recent heartbeat time. WorkerSortByLastHeartbeatAt WorkerSortField = "last_heartbeat_at" )
type WorkerStatus ¶
type WorkerStatus string
WorkerStatus is the operational state of a worker as known to the server.
const ( // WorkerStatusOnline means the worker is connected and accepting tasks. WorkerStatusOnline WorkerStatus = "online" // WorkerStatusOffline means the worker has not sent a heartbeat within the // configured timeout and is presumed unreachable. WorkerStatusOffline WorkerStatus = "offline" // WorkerStatusDisabled means an operator has administratively paused the // worker; it will not receive new task assignments until re-enabled. WorkerStatusDisabled WorkerStatus = "disabled" )
type WorkerStore ¶
type WorkerStore interface {
// RegisterWorker inserts or replaces the worker record for the given ID.
// Called by the server when a worker sends its registration message.
// If the worker ID already exists its record is updated in full.
RegisterWorker(ctx context.Context, worker Worker) (Worker, error)
// GetWorker returns the worker with the given ID, or [ErrNotFound].
GetWorker(ctx context.Context, id string) (Worker, error)
// ListWorkers returns a paginated, filtered, and sorted page of workers
// matching opts. Call [Pagination.Validate] on opts.Pagination before
// passing it to ensure sensible defaults are applied.
ListWorkers(ctx context.Context, opts ListWorkersOptions) (Page[Worker], error)
// UpdateWorker replaces the mutable capability fields of an existing
// worker (everything except ID and RegisteredAt) and updates UpdatedAt.
// Returns [ErrNotFound] if the worker does not exist.
UpdateWorker(ctx context.Context, worker Worker) (Worker, error)
// UpdateWorkerStatus sets the status of the worker and updates UpdatedAt.
// Returns [ErrNotFound] if the worker does not exist.
UpdateWorkerStatus(ctx context.Context, id string, status WorkerStatus) error
// UpdateWorkerHeartbeat records the most recent heartbeat time for the
// given worker. This is a hot path; implementations should use a single
// UPDATE statement with no unnecessary reads.
UpdateWorkerHeartbeat(ctx context.Context, id string, at time.Time) error
// ListStaleWorkers returns workers whose last heartbeat is older than
// before and whose status is [WorkerStatusOnline]. Used by the heartbeat
// timeout sweep to find workers to mark offline.
ListStaleWorkers(ctx context.Context, before time.Time) ([]Worker, error)
// CountIdleWorkers returns the number of online workers in the given farm
// that have no task currently in [TaskStatusAssigned] or
// [TaskStatusRunning] state. An empty farmID matches all farms.
// Used by the scheduler to update the [SchedulerIdleWorkers] Prometheus
// gauge.
CountIdleWorkers(ctx context.Context, farmID string) (int, error)
// DeleteWorker hard-deletes the worker record with the given ID. Returns
// [ErrNotFound] if no such worker exists. Task and task-attempt rows that
// reference the worker by ID are left intact (the ID lives on as a
// snapshot); callers are responsible for ensuring the worker has no
// in-flight work before removing it.
DeleteWorker(ctx context.Context, id string) error
// DeleteOfflineWorkersBefore hard-deletes every worker in
// [WorkerStatusOffline] whose LastHeartbeatAt is strictly before cutoff,
// and returns the deleted records. Workers in any other status (including
// administratively disabled) are never touched. Used by the scheduler's
// offline-retention sweep to bound the growth of the worker table.
DeleteOfflineWorkersBefore(ctx context.Context, cutoff time.Time) ([]Worker, error)
}
WorkerStore is the persistence interface for Worker records.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package fake provides an in-memory implementation of store.Store for unit tests that must avoid touching the filesystem.
|
Package fake provides an in-memory implementation of store.Store for unit tests that must avoid touching the filesystem. |
|
Package migrations embeds the SQL migration files used by goose to manage the sqi-server SQLite schema.
|
Package migrations embeds the SQL migration files used by goose to manage the sqi-server SQLite schema. |
|
Package sqlite provides a SQLite-backed implementation of store.Store.
|
Package sqlite provides a SQLite-backed implementation of store.Store. |