Help us shape the future of Go! Take the 2026 Go Developer Survey

server

package
v0.10.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 23, 2026 License: MIT Imports: 70 Imported by: 0

Documentation

Index

Constants

View Source
const (
	ErrClassAuth              = "auth"
	ErrClassQuota             = "quota"
	ErrClassCapacity          = "capacity"
	ErrClassUnsupportedFlavor = "unsupported_flavor"
	ErrClassTransient         = "transient"
	ErrClassUnknown           = "unknown"
)

Job error classes surfaced on the job queue and used by the UI banner / CLI. See migrations/000028_job_error_class.up.sql for the schema and 2026-05-05 incident for the motivating story.

View Source
const (
	UpgradeStatusUpToDate        = "up_to_date"
	UpgradeStatusNeedsUpgrade    = "needs_upgrade"
	UpgradeStatusUnsupportedArch = "unsupported_arch"
	UpgradeStatusUnknown         = "unknown"
)
View Source
const DefaultAssignTrigger int32 = 500 // 5 sec
View Source
const SupportedUpgradeArch = "linux/amd64"

SupportedUpgradeArch is the only arch Phase II auto-apply will target. Phase I (visibility) reports status for every arch but distinguishes `unsupported_arch` from `needs_upgrade` so the UI/CLI can hide the "redeploy this one" call-to-action where redeployment is gated on multi-arch binary distribution we haven't built yet.

Variables

This section is empty.

Functions

func CheckPassword

func CheckPassword(plaintext, hashed string) bool

func FetchWorkersForWatchdog

func FetchWorkersForWatchdog(ctx context.Context, db *sql.DB) ([]watchdog.WorkerInfo, error)

FetchWorkersForWatchdog reads workers and builds WorkerInfo list

func GateExit added in v0.8.1

func GateExit() func(int)

GateExit returns the current exit hook used by the SIGUSR1 graceful drain. Tests use this to swap in a recorder, then restore on cleanup.

func GateStdout added in v0.8.1

func GateStdout() io.Writer

GateStdout / SetGateStdout: same pattern for the stdout writer the gate uses to emit its drained-line JSON. Tests substitute a buffer.

func HttpServer

func HttpServer(cfg config.Config) (tls.Certificate, string, http.Handler, error)

func IsAdmin

func IsAdmin(ctx context.Context) bool

func LoadEmbeddedCertificates

func LoadEmbeddedCertificates() (tls.Certificate, string, error)

func LogoutHandler

func LogoutHandler(w http.ResponseWriter, r *http.Request)

LogoutHandler clears the session_token cookie to log out the user.

func NewLogin

func NewLogin(s *taskQueueServer) http.HandlerFunc

NewLogin returns an HTTP handler function to process login requests.

func Serve

func Serve(cfg config.Config, ctx context.Context, cancel context.CancelFunc) error

func SetGateExit added in v0.8.1

func SetGateExit(f func(int))

SetGateExit replaces the SIGUSR1 graceful-drain exit hook. Default is `os.Exit`. Tests must restore the previous value via t.Cleanup().

func SetGateStdout added in v0.8.1

func SetGateStdout(w io.Writer)

func StatsDBForTest added in v0.9.1

func StatsDBForTest() *sql.DB

StatsDBForTest returns the DB handle the live server is using.

func TriggerStuckDeleteCleanup added in v0.8.1

func TriggerStuckDeleteCleanup() error

TriggerStuckDeleteCleanup runs one synchronous tick of the stuck-delete janitor. The janitor's regular schedule (~5 min) is too slow for tests; this seam lets the integration test fire it on demand. Returns nil if the server hasn't started its janitor yet (the trigger is only armed inside startStuckDeleteCleanup).

func WorkerUpgradeStatus added in v0.8.1

func WorkerUpgradeStatus(workerArch, workerCommit, serverCommit string) string

WorkerUpgradeStatus computes the derived upgrade status from the worker's reported build identity and the server's own commit. Inputs:

  • workerArch: GOOS/GOARCH the worker reported, e.g. "linux/amd64". Empty string = pre-Phase-I worker (didn't report).
  • workerCommit: the worker's git SHA (full or short — compared as-is). Empty = pre-Phase-I worker.
  • serverCommit: the server's own git SHA. If empty (e.g. a `dev` build that lost its VCS stamp), every worker reports `unknown` — we can't make a comparison either side.

Returns one of the UpgradeStatus* constants.

Types

type Accumulator

type Accumulator struct {
	Count int32
	Sum   float64
	Min   float64
	Max   float64
}

In-memory aggregation types for step/workflow stats Accumulator holds aggregation stats for durations/counts.

type AuthenticatedUser

type AuthenticatedUser struct {
	UserID   int
	Username string
	IsAdmin  bool
}

func GetUserFromContext

func GetUserFromContext(ctx context.Context) *AuthenticatedUser

type Job

type Job struct {
	JobID        int32
	WorkerID     int32
	WorkerName   string
	ProviderID   int32
	ProviderName string
	Region       string
	Flavor       string
	// HasGPU is the recruited flavor's has_gpu signal (looked up from
	// the flavor catalog at job-creation time). The provider's Create
	// uses it to swap to the operator's `gpu_image` config when set.
	// false for CPU-only flavors and when the catalog row is missing.
	HasGPU bool
	// Per-recruiter cloud image overrides (NULL when the deploy came
	// from CreateWorker without a recruiter hint, or when the
	// recruiter row left both columns NULL). Format is
	// provider-specific. Precedence at provider Create:
	// gpu_image (if HasGPU) > image > scitq.yaml default.
	Image    *string
	GPUImage *string
	// Per-recruiter swapfile sizing override (see Recruiter.swap_proportion).
	// NULL = provider.Create falls back to cfg.Scitq.SwapProportion.
	// Carried through so the provider's cloud-init can bake the right
	// `-swap` value into the worker's install command.
	SwapProportion *float32
	// Per-recruiter extra block storage (see Recruiter.extra_storage_gb /
	// extra_storage_type). NULL / 0 = no extra volume. OVH only for now;
	// the Azure provider returns an error when ExtraStorageGB > 0.
	ExtraStorageGB   *int32
	ExtraStorageType *string
	Action           rune // "C", "D", "R"
	Retry            int
	Timeout          time.Duration
	FromRecruiter    bool
}

Job represents a task to be executed.

type ObjectiveDefinition added in v0.7.5

type ObjectiveDefinition struct {
	Formula   string `json:"formula"`
	Direction string `json:"direction"` // "maximize" or "minimize"
}

ObjectiveDefinition defines one objective in a multi-objective quality definition.

type ParamSpec

type ParamSpec struct {
	Name     string `json:"name"`
	Type     string `json:"type"`
	Required bool   `json:"required"`
	// Default and Choices accept whatever JSON type the schema carries —
	// boolean, integer, float, string, etc. The Python side
	// (ParamSpec.schema in python/src/scitq2/param.py) emits native
	// types, not stringified defaults, so a YAML `default: true` round-
	// trips as the JSON literal `true`. Trying to unmarshal that into
	// `string` aborted the whole UploadTemplate with "json: cannot
	// unmarshal bool into Go struct field". Neither field is consumed
	// downstream (UploadTemplate only validates Name/Type and then
	// stores the schema verbatim), so the loose `interface{}` typing
	// has no downside.
	Default interface{}   `json:"default"`
	Help    string        `json:"help"` // Or `json:"description"` if your field is named so
	Choices []interface{} `json:"choices,omitempty"`
}

type QualityDefinition added in v0.7.2

type QualityDefinition struct {
	Variables  map[string]string     `json:"variables"`            // name -> regex pattern (one capture group)
	Formula    string                `json:"formula,omitempty"`    // single-objective formula (backward compat)
	Objectives []ObjectiveDefinition `json:"objectives,omitempty"` // multi-objective: list of {formula, direction}
}

QualityDefinition is the JSON structure stored in step.quality_definition.

func ParseQualityDefinition added in v0.7.2

func ParseQualityDefinition(jsonStr string) (*QualityDefinition, error)

ParseQualityDefinition parses a JSON quality definition string.

type QualityResult added in v0.7.2

type QualityResult struct {
	Vars   map[string]float64 `json:"vars"`
	Score  float64            `json:"score"`            // primary score (first objective, or single formula)
	Scores []float64          `json:"scores,omitempty"` // all objective scores (multi-objective only)
}

QualityResult holds extracted variables and computed scores.

func ExtractQuality added in v0.7.2

func ExtractQuality(def *QualityDefinition, stdout, stderr string) (*QualityResult, error)

ExtractQuality runs quality variable regexes against log text and evaluates the formula. Each regex captures the LAST match (for iterative programs that output metrics repeatedly).

type StepAgg

type StepAgg struct {
	// Task status counters
	Waiting      int32
	Pending      int32
	Accepted     int32
	Running      int32
	Uploading    int32
	Succeeded    int32
	Failed       int32
	ReallyFailed int32          // tasks that have exhausted all retries and are failed
	Retrying     int32          // tasks currently being retried
	RetryingSet  map[int32]bool // set of task IDs currently retrying
	Total        int32

	// Accumulators for durations
	Download   Accumulator
	Upload     Accumulator
	SuccessRun Accumulator
	FailRun    Accumulator

	// RunningTasks maps taskID to start time
	RunningTasks map[int32]time.Time

	// Time bounds (epoch seconds)
	StartTime *int64
	EndTime   *int64
}

StepAgg aggregates statistics for a step within a workflow.

func NewStepAgg

func NewStepAgg() *StepAgg

NewStepAgg creates a new StepAgg with sensible defaults.

type StepAggSnapshot added in v0.9.1

type StepAggSnapshot struct {
	Waiting      int32
	Pending      int32
	Accepted     int32
	Running      int32
	Uploading    int32
	Succeeded    int32
	Failed       int32
	ReallyFailed int32
	Retrying     int32
}

StepAggSnapshot is a value-type copy of a StepAgg's status counters, exported so integration tests can compare the live aggregator state with a fresh Reconcile without holding the aggregator mutex. Excludes RunningTasks (timestamps drift at ns precision between delta path and SQL) and duration Accumulators (compared separately when the test cares) — this is the "primary counters" projection the property test's invariant is built on.

type StepCounterSnapshot added in v0.9.1

type StepCounterSnapshot struct {
	StepId            int32 `json:"stepId"`
	TotalTasks        int32 `json:"totalTasks"`
	WaitingTasks      int32 `json:"waitingTasks"`
	PendingTasks      int32 `json:"pendingTasks"`
	AcceptedTasks     int32 `json:"acceptedTasks"`
	RunningTasks      int32 `json:"runningTasks"`
	UploadingTasks    int32 `json:"uploadingTasks"`
	SuccessfulTasks   int32 `json:"successfulTasks"`
	FailedTasks       int32 `json:"failedTasks"`       // hidden retried parents
	ReallyFailedTasks int32 `json:"reallyFailedTasks"` // visible F only
	RetryingTasks     int32 `json:"retryingTasks"`
	// Accumulators — sent as flat min/max/avg for the UI's duration
	// column. UI already renders these; format matches today's
	// StepStats proto response.
	SuccessRun stepRunStats `json:"successRun"`
	FailedRun  stepRunStats `json:"failedRun"`
	RunningRun stepRunStats `json:"runningRun"`
	Download   stepRunStats `json:"download"`
	Upload     stepRunStats `json:"upload"`
	// RunningTaskStarts is the raw set of running tasks (task_id +
	// run_started_at as fractional epoch seconds). The client rebuilds
	// its local runningByStep map from this list on every snapshot so
	// its 1 s timer computes min/avg/max from an up-to-date set — R
	// transitions after mount are no longer stale.
	RunningTaskStarts []stepRunningTask `json:"runningTaskStarts"`
	StartTime         *int64            `json:"startTime,omitempty"`
	EndTime           *int64            `json:"endTime,omitempty"`
}

StepCounterSnapshot is the wire shape shipped over WS. Field names match the UI's step-row fields so the client can `Object.assign` and be done. Kept flat (not nested under a `counters:` map) to keep the payload small and the client-side reactivity trivial.

type StepStatsAgg

type StepStatsAgg struct {
	// contains filtered or unexported fields
}

StepStatsAgg holds in-memory aggregated statistics for steps in workflows.

func NewStepStatsAgg

func NewStepStatsAgg(db *sql.DB) (*StepStatsAgg, error)

NewStepStatsAgg creates a new StepStatsAgg with initialized internal maps.

func StatsAggForTest added in v0.9.1

func StatsAggForTest() *StepStatsAgg

StatsAggForTest returns the live server's aggregator or nil if no server has been started this process lifetime.

func (*StepStatsAgg) Adjust added in v0.7.5

func (a *StepStatsAgg) Adjust(workflowID, stepID int32, fn func(*StepAgg))

Adjust runs fn with the StepAgg for (workflowID, stepID) while holding the aggregator mutex. It lazily creates any missing map entries so call sites don't need to care about initialisation order. This is the canonical way to mutate counters from anywhere outside UpdateTaskStatus; using it eliminates the two classes of drift caused by unlocked direct map access:

  • races with concurrent writers (int32 increments are not atomic)
  • updates to a StepAgg that Reconcile has just orphaned by swapping data

func (*StepStatsAgg) EnsureStep

func (a *StepStatsAgg) EnsureStep(workflowID, stepID int32)

EnsureStep guarantees an entry exists for (workflowID, stepID).

func (*StepStatsAgg) FlushLoop added in v0.9.1

func (a *StepStatsAgg) FlushLoop(interval time.Duration, stop <-chan struct{})

FlushLoop drives FlushOnce on a fixed cadence. Caller wires stop. Also runs the "running heartbeat" every runningHeartbeatInterval so steps with active tasks re-emit their RunningRun stats and the UI sees a live wall-clock counter rather than a frozen last-transition value.

func (*StepStatsAgg) FlushOnce added in v0.9.1

func (a *StepStatsAgg) FlushOnce()

FlushOnce collects the current dirty set under the mutex, snapshots each dirty step's counters, clears the set, then emits one WS message per workflow OUTSIDE the mutex. Zero traffic when nothing is dirty.

func (*StepStatsAgg) MarkDirty added in v0.9.1

func (a *StepStatsAgg) MarkDirty(workflowID, stepID int32)

MarkDirty is the entry point for callers that mutated the aggregator under a self-held lock (e.g. UpdateTaskStatus, which acquires a.mu.Lock() itself and manipulates a.data[wid][sid].Waiting-- etc. directly). Once every such site is migrated to Adjust or TransitionTask, this can go private again.

func (*StepStatsAgg) MarkDirtyLocked added in v0.9.1

func (a *StepStatsAgg) MarkDirtyLocked(workflowID, stepID int32)

MarkDirtyLocked is the same as MarkDirty but assumes the caller already holds a.mu — used inside UpdateTaskStatus which holds the lock across its aggregator work.

func (*StepStatsAgg) Reconcile added in v0.7.2

func (a *StepStatsAgg) Reconcile(db *sql.DB)

Reconcile rebuilds the in-memory stats from the database, fixing any drift between cached counters and actual task states.

func (*StepStatsAgg) RemoveStep

func (a *StepStatsAgg) RemoveStep(stepID int32) int32

RemoveStep deletes a step entry by its stepID, scanning all workflows to locate it.

func (*StepStatsAgg) RemoveWorkflow

func (a *StepStatsAgg) RemoveWorkflow(workflowID int32)

func (*StepStatsAgg) SnapshotCounters added in v0.9.1

func (a *StepStatsAgg) SnapshotCounters() map[int32]map[int32]StepAggSnapshot

SnapshotCounters returns a deep-copied snapshot of every step's primary counters. Test uses this on both the live aggregator and a fresh NewStepStatsAgg(db) and compares them.

type TaskUpdateBroadcast

type TaskUpdateBroadcast struct {
	TaskId   int32
	Status   string
	WorkerId int32
}

type TransitionExtras added in v0.9.1

type TransitionExtras struct {
	WorkerID     *int32
	FailureClass *string
}

TransitionExtras carries additional columns the transition writes. Populated as call sites migrate; add fields as more transitions need them (e.g. failure_class on F, run_started_at on R).

type TransitionRequest added in v0.9.1

type TransitionRequest struct {
	TaskID int32
	// New status letter: 'P'|'W'|'A'|'C'|'D'|'O'|'R'|'U'|'V'|'S'|'F'.
	NewStatus string
	// Optional guard: only transition if the CURRENT status equals
	// this value. Empty = unconditional. Mirrors the "WHERE status='W'"
	// inline guards several existing sites already use.
	ExpectStatus string
	// Extras applied in the same UPDATE. Nil pointers = leave alone.
	// The struct is intentionally typed rather than a map so a new
	// column touched by a transition forces an API extension rather
	// than sneaking in via a stringly-typed side channel.
	Extras TransitionExtras
}

TransitionRequest is the input to TransitionTask.

type TransitionResult added in v0.9.1

type TransitionResult struct {
	// OldStatus is the row's status before the UPDATE. Empty when the
	// row didn't exist or the guard (ExpectStatus) didn't match.
	OldStatus string
	// WorkflowID / StepID scraped from the row via the RETURNING
	// clause. Used internally to bill the transition against the
	// right aggregator bucket.
	WorkflowID int32
	StepID     int32
	// Applied is true iff the UPDATE affected a row (guard matched,
	// row existed, row wasn't hidden).
	Applied bool
}

TransitionResult reports what happened.

Directories

Path Synopsis
Package metrics exposes scitq server state as Prometheus-format gauges + counters.
Package metrics exposes scitq server state as Prometheus-format gauges + counters.
Package notifications sends user-facing convenience notifications (workflow completion, ...) to configured channels.
Package notifications sends user-facing convenience notifications (workflow completion, ...) to configured channels.
run

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL