admin

package module
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 10, 2026 License: Apache-2.0 Imports: 18 Imported by: 0

Documentation

Overview

Package admin is the operations side of ubgo/workflow (list failed runs, retry, forget, clear a backlog, pause, list schedules, and the data a queue dashboard shows): a Go API, a JSON HTTP handler that is closed by default, and a threshold Monitor. It is its own module, needs nothing beyond the standard library and the core, and holds no state: every call goes to the client.

Operations over any client

New(client) returns an Admin over any workflow.Client. Each operation needs a capability and an optional core interface (Requirements):

runs, run, failures, queues  CapRunQuery       workflow.RunQuerier
discarded                    CapRunQuery and CapRunStates  workflow.RunQuerier
retry                        CapRetryRun       workflow.RunRetrier
delete                       CapDeleteRun      workflow.RunDeleter
cancel                       CapCancel         Client.Cancel
cancel_all                   CapCancelAll      workflow.BulkCanceller
pause                        CapPauseWorkflow  workflow.Pauser (CapPauseQueue for a queue target)
schedules                    CapScheduleList   workflow.ScheduleLister
schedule_pause               CapScheduleList and CapSchedulePause  workflow.ScheduleLister and workflow.SchedulePauser
schedule_trigger             CapScheduleList and CapScheduleTrigger  workflow.ScheduleLister and workflow.ScheduleTriggerer
force                        CapForceRun       workflow.RunForcer
retry_all                    CapRunQuery and CapRetryRun   workflow.RunQuerier and workflow.RunRetrier
delete_all                   CapRunQuery and CapDeleteRun  workflow.RunQuerier and workflow.RunDeleter
nodes                        none              workflow.NodeLister
health                       none              workflow.CheckHealth (every client)
reschedule                   none              workflow.RunRescheduler
cancel_step                  none              workflow.StepCanceller

Support reports which ones the client has and what it lacks, so a screen hides what cannot work; an unsupported call returns a *workflow.UnsupportedError naming "admin.<operation>" and the missing capabilities. QueueStats pages RunQuerier over queued runs to count each queue's depth and find its oldest run, reading at most WithMaxScan runs (DefaultMaxScan) and saying Truncated when there were more. On a client that implements workflow.WorkerScaler (CapWorkerScale) it adds each queue's worker count and running attempts on that client (QueueReport.Workers), the state a workflow.BalanceLoop acts on. Failures lists failed runs and Discarded, beside it, the runs a step dropped with workflow.Discard; a driver without CapRunStates reports a discarded run as failed or cancelled, so Discarded is refused there. CountRuns counts the runs a query matches, within the same max scan, and returns the cursor that starts each page, so a screen can show "1–50 of N" and numbered pages over a driver's opaque cursors. Runs, Failures, Discarded and CountRuns search by label: RunQuery.Labels (every key=value pair must match) and RunQuery.LabelPrefixes (a key's value starts with a prefix; the HTTP filter label=key=prefix*), which is refused with an UnsupportedError on a client without CapLabelPrefix.

RetryAll and DeleteAll act on every run a filter matches: they list the matches first (at most WithMaxBulk, DefaultMaxBulk, per call; Truncated says to call again) and then retry or delete each, counting runs that changed in between as Skipped and stopping on any other error. A BulkOptions.DryRun lists and changes nothing, returning the count and a sample of refs; PreviewCancelAll is the same for CancelAll. DeleteAll refuses an empty filter, so forgetting every finished run is said explicitly with its statuses.

Metrics reads run-end history for the charts from a MetricsSource wired with WithMetrics: counts per step by terminal status, zero-filled. MemoryMetrics is the in-process default, fed by registering its Observe with the run-finished hook; it keeps one-minute buckets for a day, drops repeated deliveries, and starts empty on a restart (Since says when its data begins). An app with a metrics store adapts it with MetricsSourceFunc.

Nodes lists the online nodes (any client implementing workflow.NodeLister) and Health returns the client's health report, which an admin screen shows; a probe should call workflow.CheckHealth on the app's own unauthenticated route. Nodes, Reschedule (move a delayed run) and CancelStep (withdraw one step) have no capability bit: whether they are offered is workflow.Supports, which a wrapper (workflow.Forward, failover) answers for the clients it wraps, and a client without them is refused at the call.

Schedule returns one schedule with its overlap policy and, on a client with CapScheduleHistory, its last fire and the run it started (ScheduleInfo.Last). TriggerSchedule runs a schedule now with its payload and labels; the overlap policy still applies, so with OverlapSkip and the previous run unfinished it returns an error matching workflow.ErrOverlap and enqueues nothing. ForceComplete and ForceFail end a run from outside (workflow.RunForcer); ForceFail's reason is bounded by MaxForceReason. A run's per-attempt errors (RunInfo.Errors, CapErrorHistory) come with it. Captured step output is read through an OutputSource the app wires with WithOutputs; the library keeps no output itself, and MemoryOutputs is an in-process, bounded store that is both the middleware.CaptureOutput sink and the source.

The HTTP handler is closed by default

NewHandler(admin, Auth[P]{Identify, Authorize}) serves the JSON API. P is the app's own principal type. With either function nil, every request, on every path, is refused with 403. Otherwise each request is identified first (401 before routing on any error or panic), then parsed, then authorized with a Request[P] saying who, which Action and which Resource (the run, the parsed filter or the pause target); any error or panic from Authorize refuses it with 403, so a policy that fails, fails closed. Writes need Content-Type: application/json, which a cross-site form cannot send. Payloads are left out of responses unless WithPayloads is given, and an error without a known core sentinel is logged (WithLogger) and answered with a bare 500, so a database error never reaches the browser. Run lists are paginated with the driver's opaque cursor (count=1 adds the total and every page's cursor); schedules with an offset cursor. Attempt errors are sent with each run; their stack traces only with WithStacks. Force-fail sends its reason to Authorize in Resource.Reason, so a policy can require one or log it. Bulk retry, delete and cancel send the dry-run flag to Authorize in Resource.DryRun, so a policy can let more people preview than act. A run's tenant, due time, progress, nodes and result are sent with it; its progress data and its result only with WithPayloads, since both are app data like a payload.

Web UI

The same handler serves a web UI at ui/ under its mount point (GET / and GET /ui redirect there): queues with depth, oldest wait and worker counts, and pause and resume of queues, workflows and everything; runs filtered by status, workflow, queue and label, with numbered pages and a page size; one run with its attempts, error and, only with WithPayloads, its payload; its timing (queue wait, duration), the error of each failed attempt (stacks with WithStacks), its captured output per attempt when WithOutputs is wired, and force complete and force fail; failed and discarded runs with retry and delete; schedules with their last fire and last run, run now, pause and resume, and a page per schedule with its recent runs; "all matching" retry, delete and cancel on a run list, each showing the server's dry-run count before the confirming click; a run's progress, tenant, due time, nodes and result, with reschedule for a delayed run and cancel step for a running one; charts of run ends per step by status over one, six or 24 hours, with totals and a table view, when WithMetrics is wired; and a system page with the health report and the online nodes. It is three files embedded in the package, written for it, with no third-party code and nothing fetched at run time. They are served only after Identify and Authorize (ActionUI), like every route, and each call the page makes is a request to the JSON API, authorized on its own action, so a read-only role sees the screens and is refused every button. Writes go through the API's JSON-only rule. A browser sends no Authorization header when it opens a page, so Identify must accept the app's session cookie for the UI to load. Pages carry a Content-Security-Policy that allows only those files and same-origin requests, and forbids framing. Buttons for an operation the client does not support are hidden (the capabilities route).

Alerts

NewMonitor polls QueueStats on the Admin's clock (WithClock) and calls an Alerter when a queue's depth or oldest wait goes over a Threshold, once on crossing and once when it resolves (WithRepeat re-sends while it stays over). Nothing runs until Monitor.Run; Check polls once. Which alerts are open is kept in memory, so run one Monitor per deployment.

The OpenTelemetry queue-depth gauge lives in the otel module; Admin.QueueDepths has the otel.QueueDepthFunc signature, so neither module imports the other.

Index

Constants

View Source
const (
	CodeUnauthorized = "unauthorized"           // 401: Identify failed
	CodeForbidden    = "forbidden"              // 403: no Auth configured, or Authorize refused
	CodeInvalid      = "invalid"                // 400: a malformed request or workflow.ErrInvalid
	CodeNotFound     = "not_found"              // 404: workflow.ErrNotFound, or no such route
	CodeUnsupported  = "unsupported"            // 501: workflow.ErrUnsupported
	CodeConflict     = "conflict"               // 409: workflow.ErrWrongState, workflow.ErrDuplicate or workflow.ErrOverlap
	CodeMediaType    = "unsupported_media_type" // 415: a write without Content-Type: application/json
	CodeInternal     = "internal"               // 500: anything else; the detail is logged, not sent
)

The error codes in an error response's "code" field.

View Source
const (
	DefaultSchedulesLimit = 100
	MaxSchedulesLimit     = 1000
)

DefaultSchedulesLimit and MaxSchedulesLimit bound a page of GET /schedules.

View Source
const (
	DefaultMetricsWindow = time.Hour
	DefaultMetricsPoints = 60
)

DefaultMetricsWindow is the span GET /metrics covers without from or window, and DefaultMetricsPoints how many steps it splits a span into without step.

View Source
const (
	DefaultMetricsBucket = time.Minute
	DefaultMetricsKeep   = 24 * time.Hour
	// DefaultMetricsDedupe is how many recent run ends MemoryMetrics
	// remembers to drop a repeated delivery (delivery is at least once).
	DefaultMetricsDedupe = 4096
)

The MemoryMetrics defaults.

View Source
const (
	DefaultOutputRuns   = 1000
	DefaultOutputPerRun = 20
	DefaultOutputBytes  = 64 << 10
)

The MemoryOutputs defaults.

View Source
const BulkSample = 20

BulkSample is how many matched refs a BulkResult carries, so a dry run shows which runs it would touch without sending thousands of refs.

View Source
const DefaultMaxBody = 64 << 10

DefaultMaxBody is the largest request body the handler reads.

View Source
const DefaultMaxBulk = 1000

DefaultMaxBulk bounds how many runs one RetryAll or DeleteAll call acts on, so one HTTP request does a bounded amount of work; a caller with more matches repeats the call (BulkResult.Truncated says so). It is a floor on request cost, not on what an operator can clear: each repeat takes the next runs, because the ones acted on no longer match.

View Source
const DefaultMaxScan = 10_000

DefaultMaxScan bounds how many queued runs QueueStats reads per call, so a queue of a million runs costs a bounded number of RunQuerier pages.

View Source
const DefaultMonitorInterval = 30 * time.Second

DefaultMonitorInterval is how often a Monitor polls when WithInterval is not given.

View Source
const MaxForceReason = 1024

MaxForceReason bounds the reason ForceFail stores on a run, so an operator's text cannot grow a run row without limit.

View Source
const MaxMetricsPoints = 1440

MaxMetricsPoints bounds how many points one Metrics call returns, so a chart request over a year at one-minute steps is refused rather than answered with half a million points.

Variables

Actions lists every action, so a test or a permission screen can walk them.

Operations lists every operation in a fixed order, the order Support reports them in.

Functions

func NewHandler

func NewHandler[P any](a *Admin, auth Auth[P], opts ...HandlerOption) http.Handler

NewHandler returns the admin JSON API over a. It is closed by default: with a nil Auth.Identify or Auth.Authorize every request, on every path, is refused with 403 before anything else happens. With both set, each request is identified first (401 on error, also before routing, so an anonymous caller learns nothing about the routes), then parsed, then authorized with its action and resource (403 on any error or panic), and only then run.

Writes (POST, DELETE) must send Content-Type: application/json, which a browser cannot send cross-site without a CORS preflight the handler never answers, so a forged form post from another site is refused (415). Mount it under a prefix with http.StripPrefix.

Routes (JSON in and out; times are RFC 3339):

GET    /capabilities      the operations the client supports, and what it lacks
GET    /runs              list and search runs; filters below; paginated
GET    /runs/{id}         one run (?scheduled=1 when id is a ScheduledID)
GET    /failures          failed runs; same filters but status
GET    /discarded         runs a step discarded; same filters but status
POST   /runs/{id}/retry   retry a failed run from its failed step
POST   /runs/{id}/cancel  cancel one run
DELETE /runs/{id}         forget a finished run
POST   /runs/cancel       cancel every run a filter body matches; returns the count; {"dry_run":true} counts only
POST   /pause             body {"workflow":..} | {"queue":..} | {"all":true}
POST   /resume            same body
GET    /pauses            the active pauses
GET    /schedules         schedules with next fire times; ?limit= and ?cursor=
GET    /queues            queue depth and oldest wait (workers and running when the driver scales workers)
POST   /schedules/{id}/pause   stop one schedule firing
POST   /schedules/{id}/resume  let it fire again
GET    /schedules/{id}         one schedule: next fire, overlap policy, last fire and its run
POST   /schedules/{id}/trigger run it now; returns the run's ref (409 when its overlap policy skips)
POST   /runs/{id}/complete     force-complete a run that is not terminal, or is failed
POST   /runs/{id}/fail         force-fail a run that is not terminal; body {"reason":".."}
GET    /runs/{id}/output       captured step output, oldest attempt first (needs WithOutputs, else 501)
POST   /runs/retry             retry every failed run a filter body matches; {"dry_run":true} previews
POST   /runs/delete            forget every finished run a filter body matches; same body
GET    /metrics                run ends per step by status (needs WithMetrics, else 501); ?from=&to= or ?window=, ?step=, ?workflow=
GET    /nodes                  the online nodes (workflow.NodeLister)
GET    /health                 the client's health report: live, ready and each check
POST   /runs/{id}/reschedule   move a delayed run; body {"at":"RFC 3339"}
POST   /runs/{id}/steps/{step}/cancel  withdraw one step of a run
GET    /ui/               the web UI (see below); GET / and GET /ui redirect to it

Run filters: workflow, queue, tenant (CapTenant), due_from and due_to (RunInfo.RunAt, CapDueQuery), status (repeat or comma-separated), label=key=value (repeat; every pair must match; a value ending in "*" is a prefix, needing CapLabelPrefix, else 501), from and to (enqueue time), limit and cursor (from the previous page's "next"). On /runs, /failures and /discarded, count=1 adds "total", "truncated" and "cursors" (the cursor that starts each page at this limit), read by Admin.CountRuns within the max scan, so a screen can show numbered pages over the driver's opaque cursors.

The web UI is three files embedded in this package (a page, a style sheet and a script; no third-party code, nothing fetched at run time), served behind the same gate as the API: Identify, then Authorize with ActionUI. Every call the UI makes is a request to the routes above and is authorized on its own. A browser sends no Authorization header when it loads a page, so Identify must accept the app's session cookie for the UI to open. Pages are sent with a Content-Security-Policy that allows only these files and same-origin requests.

func Requirements

func Requirements(op Operation) workflow.Caps

Requirements returns the capabilities op needs. OpPause reports both pause capabilities; a client with either one supports pausing the targets that capability covers.

func ValidateMetricsQuery

func ValidateMetricsQuery(q MetricsQuery) error

ValidateMetricsQuery checks q's shape: From before To, a positive Step, and at most MaxMetricsPoints points. Errors wrap workflow.ErrInvalid.

Types

type Action

type Action string

Action is what an HTTP request asks to do, as the Authorizer sees it. The names are stable, so an app can store them in its own permission tables.

const (
	ActionCapabilities  Action = "capabilities.read" // GET  /capabilities
	ActionListRuns      Action = "runs.list"         // GET  /runs
	ActionShowRun       Action = "runs.show"         // GET  /runs/{id}
	ActionListFailures  Action = "failures.list"     // GET  /failures
	ActionListDiscarded Action = "discarded.list"    // GET  /discarded
	ActionRetryRun      Action = "runs.retry"        // POST /runs/{id}/retry
	ActionCancelRun     Action = "runs.cancel"       // POST /runs/{id}/cancel
	ActionDeleteRun     Action = "runs.delete"       // DELETE /runs/{id}
	ActionCancelRuns    Action = "runs.cancel_all"   // POST /runs/cancel
	ActionPause         Action = "pauses.create"     // POST /pause
	ActionResume        Action = "pauses.delete"     // POST /resume
	ActionListPauses    Action = "pauses.list"       // GET  /pauses
	ActionListSchedules Action = "schedules.list"    // GET  /schedules
	ActionQueueStats    Action = "queues.list"       // GET  /queues
	// ActionPauseSchedule and ActionResumeSchedule pause and resume one
	// schedule by ID.
	ActionPauseSchedule  Action = "schedules.pause"  // POST /schedules/{id}/pause
	ActionResumeSchedule Action = "schedules.resume" // POST /schedules/{id}/resume
	// ActionShowSchedule reads one schedule with its last and next fire;
	// ActionTriggerSchedule runs it now.
	ActionShowSchedule    Action = "schedules.show"    // GET  /schedules/{id}
	ActionTriggerSchedule Action = "schedules.trigger" // POST /schedules/{id}/trigger
	// ActionForceComplete and ActionForceFail end a run from outside
	// (workflow.RunForcer).
	ActionForceComplete Action = "runs.force_complete" // POST /runs/{id}/complete
	ActionForceFail     Action = "runs.force_fail"     // POST /runs/{id}/fail
	// ActionShowOutput reads a run's captured step output (WithOutputs).
	ActionShowOutput Action = "runs.output" // GET /runs/{id}/output
	// ActionRetryRuns and ActionDeleteRuns retry or forget every run a
	// filter matches (Admin.RetryAll, Admin.DeleteAll), dry runs included.
	ActionRetryRuns  Action = "runs.retry_all"  // POST /runs/retry
	ActionDeleteRuns Action = "runs.delete_all" // POST /runs/delete
	// ActionMetrics reads run-end history for the charts (WithMetrics).
	ActionMetrics Action = "metrics.read" // GET /metrics
	// ActionListNodes lists the online nodes; ActionHealth reads the
	// client's health report.
	ActionListNodes Action = "nodes.list"  // GET /nodes
	ActionHealth    Action = "health.read" // GET /health
	// ActionRescheduleRun moves a delayed run; ActionCancelStep withdraws
	// one step of a run.
	ActionRescheduleRun Action = "runs.reschedule"  // POST /runs/{id}/reschedule
	ActionCancelStep    Action = "runs.cancel_step" // POST /runs/{id}/steps/{step}/cancel
	// ActionUI is loading the web UI's page and its files. The UI's data
	// comes from the routes above, each authorized with its own action,
	// so a caller allowed only ActionUI sees an empty screen.
	ActionUI Action = "ui.read" // GET /, GET /ui, GET /ui/{file}
)

The actions, one per route.

func (Action) ReadOnly

func (a Action) ReadOnly() bool

ReadOnly reports whether the action only reads. An Authorizer that gives some users a read-only view allows exactly these.

type Admin

type Admin struct {
	// contains filtered or unexported fields
}

Admin is the operations an admin screen or a CLI needs, over any workflow.Client. It holds no state of its own: every call goes to the client, so several Admins over one store agree.

Each operation checks the client's capabilities first and then its optional interface, and refuses with a *workflow.UnsupportedError (which matches workflow.ErrUnsupported) naming the operation and the missing capabilities when either is absent. The capability is checked because a wrapper implements every optional interface and refuses at the call when the client it wraps lacks it.

func New

func New(c workflow.Client, opts ...Option) *Admin

New returns an Admin over c. It never fails: what c cannot do is reported by Support and refused per call.

func (*Admin) Cancel

func (a *Admin) Cancel(ctx context.Context, ref workflow.RunRef) error

Cancel cancels one run, as Client.Cancel does.

func (*Admin) CancelAll

func (a *Admin) CancelAll(ctx context.Context, q workflow.RunQuery) (int, error)

CancelAll cancels every run matching q and returns how many; see workflow.BulkCanceller for which statuses an empty q.Statuses means.

func (*Admin) CancelStep

func (a *Admin) CancelStep(ctx context.Context, ref workflow.RunRef, step string) error

CancelStep withdraws one step of a run; see workflow.StepCanceller for what the run does next.

func (*Admin) Client

func (a *Admin) Client() workflow.Client

Client returns the client Admin works on.

func (*Admin) CountRuns

func (a *Admin) CountRuns(ctx context.Context, q workflow.RunQuery) (RunCount, error)

CountRuns pages through the runs q matches, at q's page size, and returns their count and each page's cursor. q.Cursor must be empty. It reads at most the max scan (WithMaxScan) of runs, so a screen pays a bounded cost; a driver that could count in one query is still asked page by page, because RunQuerier is the only portable way to ask.

func (*Admin) Delete

func (a *Admin) Delete(ctx context.Context, ref workflow.RunRef) error

Delete forgets a finished run.

func (*Admin) DeleteAll

func (a *Admin) DeleteAll(ctx context.Context, q workflow.RunQuery, opts BulkOptions) (BulkResult, error)

DeleteAll forgets every terminal run q matches, as Delete does one. The filter must name at least one field (workflow, queue, statuses, labels, label prefixes, from or to), so "delete everything" is never an empty request: an operator who means every finished run says statuses=succeeded,failed,cancelled. q.Statuses may list terminal statuses only; empty means every terminal status the client reports (discarded only with CapRunStates). It needs OpDeleteAll: RunQuerier and RunDeleter. Runs that changed between listing and deleting are Skipped; any other error stops the call.

func (*Admin) Discarded

func (a *Admin) Discarded(ctx context.Context, q workflow.RunQuery) (workflow.RunPage, error)

Discarded lists runs a step ended with workflow.Discard, matching q, beside Failures: the run's Error keeps the reason. q.Statuses must be empty. It needs CapRunStates: on a driver without it a discarded run reads failed or cancelled and shows up there instead.

func (*Admin) Failures

func (a *Admin) Failures(ctx context.Context, q workflow.RunQuery) (workflow.RunPage, error)

Failures lists failed runs matching q. q.Statuses must be empty: the operation sets it to StatusFailed, and a caller that asks for other statuses wants Runs. Discarded runs are not failures (a step chose to drop them) and are listed by Discarded.

func (*Admin) ForceComplete

func (a *Admin) ForceComplete(ctx context.Context, ref workflow.RunRef) error

ForceComplete ends a run as succeeded without running its remaining steps; see workflow.RunForcer. A run that is not terminal, or is failed, is accepted; another terminal run returns ErrWrongState.

func (*Admin) ForceFail

func (a *Admin) ForceFail(ctx context.Context, ref workflow.RunRef, reason string) error

ForceFail ends a run that is not terminal as failed, with reason in its error and no retry; see workflow.RunForcer. A reason longer than MaxForceReason bytes is refused with ErrInvalid.

func (*Admin) HasMetrics

func (a *Admin) HasMetrics() bool

HasMetrics reports whether a MetricsSource is wired (WithMetrics).

func (*Admin) HasOutputs

func (a *Admin) HasOutputs() bool

HasOutputs reports whether an OutputSource is wired (WithOutputs).

func (*Admin) Health

func (a *Admin) Health(ctx context.Context) workflow.HealthReport

Health is the client's health report (workflow.CheckHealth). It never fails: a client without HealthChecker is reported live and ready with a check saying it cannot tell. An admin screen shows it; a liveness or readiness probe should call workflow.CheckHealth on the app's own unauthenticated probe route instead of going through the admin API.

func (*Admin) Metrics

func (a *Admin) Metrics(ctx context.Context, q MetricsQuery) (MetricsSeries, error)

Metrics returns run ends over q from the wired MetricsSource. Without WithMetrics it returns an error matching workflow.ErrUnsupported.

func (*Admin) Nodes

func (a *Admin) Nodes(ctx context.Context) ([]workflow.NodeInfo, error)

Nodes lists the nodes that are online, sorted by ID; see workflow.NodeLister for when a node drops out.

func (*Admin) Pause

func (a *Admin) Pause(ctx context.Context, t workflow.PauseTarget) error

Pause holds the runs t matches without cancelling them; see workflow.Pauser.

func (*Admin) PauseSchedule

func (a *Admin) PauseSchedule(ctx context.Context, id workflow.ScheduleID) error

PauseSchedule stops the schedule id firing without removing it; see workflow.SchedulePauser. An empty id is refused with ErrInvalid.

func (*Admin) Paused

func (a *Admin) Paused(ctx context.Context) ([]workflow.PauseTarget, error)

Paused lists the active pauses.

func (*Admin) PreviewCancelAll

func (a *Admin) PreviewCancelAll(ctx context.Context, q workflow.RunQuery) (BulkResult, error)

PreviewCancelAll is CancelAll's dry run: it counts the runs CancelAll would cancel (workflow.CancelStatuses) and samples them, changing nothing. It needs OpCancelAll and OpRuns. The count is a moment's snapshot: runs that start before the real call are no longer cancelled by its default statuses.

func (*Admin) QueueDepths

func (a *Admin) QueueDepths(ctx context.Context) (map[string]int64, error)

QueueDepths returns QueueStats as queue name to depth. Its signature matches otel.QueueDepthFunc, so a method value plugs straight into the OpenTelemetry queue-depth gauge without either module importing the other.

func (*Admin) QueueStats

func (a *Admin) QueueStats(ctx context.Context) (QueueReport, error)

QueueStats counts the queued runs per queue and finds the oldest one in each, by paging RunQuerier over StatusQueued. It reads at most the max scan (WithMaxScan) of runs, so its cost is bounded; a driver that can count with one query is still asked page by page, because RunQuerier is the only portable way to ask.

Depth and the oldest run come from that scan on every driver, so they count the queue across every process sharing the store. On a client that implements workflow.WorkerScaler and declares CapWorkerScale, QueueLoads adds this client's own worker count and running attempts per queue (the state a workflow.BalanceLoop acts on); a QueueLoads error fails the call, since a report that silently drops the worker counts would read as "no workers".

func (*Admin) Reschedule

func (a *Admin) Reschedule(ctx context.Context, ref workflow.RunRef, at time.Time) (workflow.RunRef, error)

Reschedule moves a delayed run that has not started to at; see workflow.RunRescheduler. A zero at is refused here (the interface reads it as now, which an operator should say on purpose with the current time).

func (*Admin) Resume

func (a *Admin) Resume(ctx context.Context, t workflow.PauseTarget) error

Resume removes the pause on t; Resume(workflow.PauseAll()) removes every pause.

func (*Admin) ResumeSchedule

func (a *Admin) ResumeSchedule(ctx context.Context, id workflow.ScheduleID) error

ResumeSchedule lets a paused schedule fire again.

func (*Admin) Retry

func (a *Admin) Retry(ctx context.Context, ref workflow.RunRef) (workflow.RunRef, error)

Retry re-runs a failed run from the step that failed; see workflow.RunRetrier for what is kept.

func (*Admin) RetryAll

func (a *Admin) RetryAll(ctx context.Context, q workflow.RunQuery, opts BulkOptions) (BulkResult, error)

RetryAll retries every failed run q matches, from the step that failed, as Retry does one; see workflow.RunRetrier. q.Statuses must be empty or exactly [failed], since Retry refuses every other status; an empty filter retries every failed run, up to the bound. It needs OpRetryAll: RunQuerier to list and RunRetrier to act. The runs are listed first and then retried one by one, so a run that changes in between is counted in Skipped; any other error stops the call and is returned with the result so far.

func (*Admin) Run

func (a *Admin) Run(ctx context.Context, ref workflow.RunRef) (workflow.RunInfo, error)

Run returns one run. An unknown ref returns workflow.ErrNotFound. With WithRunQuerier, a run the querier does not hold is looked up on the client when the client can list runs.

func (*Admin) RunOutput

func (a *Admin) RunOutput(ctx context.Context, ref workflow.RunRef) ([]AttemptOutput, error)

RunOutput returns the captured output of the run ref's attempts, oldest first. Without WithOutputs it returns an error matching workflow.ErrUnsupported. A ref with only a ScheduledID has not run yet and has no output.

func (*Admin) Runs

Runs lists runs matching q, one page at a time, newest enqueue first. Pass the returned RunPage.Next as q.Cursor for the next page.

func (*Admin) Schedule

Schedule returns one live schedule by ID, from the same list Schedules reads, with its next fire time and, on a client with CapScheduleHistory, its last fire (ScheduleInfo.Last). An empty id is refused with ErrInvalid and an unknown one returns ErrNotFound.

func (*Admin) Schedules

func (a *Admin) Schedules(ctx context.Context) ([]workflow.ScheduleInfo, error)

Schedules lists the live schedules with their next fire time, sorted by workflow, then name.

func (*Admin) Support

func (a *Admin) Support() []OperationSupport

Support reports, for every operation in Operations order, whether the client supports it. A UI hides what is unsupported instead of showing a button that always fails.

func (*Admin) Supports

func (a *Admin) Supports(op Operation) bool

Supports reports whether the client supports op.

func (*Admin) TriggerSchedule

func (a *Admin) TriggerSchedule(ctx context.Context, id workflow.ScheduleID) (workflow.RunRef, error)

TriggerSchedule runs the schedule id now, with its payload and labels, and returns the run it enqueued; see workflow.ScheduleTriggerer. The schedule's overlap policy still applies: with OverlapSkip and its previous run unfinished it enqueues nothing and returns an error matching workflow.ErrOverlap. An empty id is refused with ErrInvalid.

type Alert

type Alert struct {
	Kind  AlertKind
	Queue string
	// Depth and Wait are the queue's values at the poll that raised the
	// alert (Wait is its OldestWait).
	Depth int
	Wait  time.Duration
	// Threshold is the one crossed.
	Threshold Threshold
	// Resolved is true on the alert that reports the queue back under the
	// threshold.
	Resolved bool
	// Truncated is copied from the QueueReport: Depth is a lower bound.
	Truncated bool
	// At is the poll time on the Admin's clock.
	At time.Time
}

Alert is one threshold crossing, or the end of one.

type AlertKind

type AlertKind string

AlertKind is which threshold an alert is about.

const (
	// AlertQueueDepth: more runs wait in a queue than Threshold.MaxDepth.
	AlertQueueDepth AlertKind = "queue_depth"
	// AlertQueueWait: the oldest queued run has waited longer than
	// Threshold.MaxWait.
	AlertQueueWait AlertKind = "queue_wait"
)

The alert kinds.

type Alerter

type Alerter interface {
	Alert(ctx context.Context, a Alert)
}

Alerter receives alerts. Send them to a chat channel, a pager or a log. Alert runs on the Monitor's goroutine, one call at a time; a slow Alerter delays the next poll, and a panic is recovered and reported to the error handler.

type AlerterFunc

type AlerterFunc func(ctx context.Context, a Alert)

AlerterFunc adapts a function to Alerter.

func (AlerterFunc) Alert

func (f AlerterFunc) Alert(ctx context.Context, a Alert)

Alert calls f.

type AttemptOutput

type AttemptOutput struct {
	Step    string
	Attempt int
	// Started and Finished bound the attempt, on the capturing clock.
	Started, Finished time.Time
	// Text is the captured output (middleware.FormatOutput's format).
	Text string
	// Truncated is true when Text was cut to the source's size bound.
	Truncated bool
	// Error is the attempt's error message; empty when it succeeded.
	Error string
}

AttemptOutput is what one step attempt wrote, as an admin screen shows it: the text middleware.CaptureOutput captured, how the attempt ended and when.

type Auth

type Auth[P any] struct {
	// Identify returns the caller of r, from a session cookie or a bearer
	// token. An error means anonymous: the request is refused with 401
	// before any of it is parsed.
	Identify func(r *http.Request) (P, error)
	// Authorize allows req by returning nil. Any error, or a panic, refuses
	// it with 403, so an Authorizer that fails, fails closed.
	Authorize func(ctx context.Context, req Request[P]) error
}

Auth is how the handler decides who may do what. Both functions are required: a handler with either one nil refuses every request, so the zero Auth is closed.

type BulkOptions

type BulkOptions struct {
	// DryRun lists what the call would act on and changes nothing.
	DryRun bool
	// Max lowers the Admin's bound (WithMaxBulk) for this call; zero, or a
	// value above that bound, keeps the bound.
	Max int
}

BulkOptions shape one bulk call.

type BulkResult

type BulkResult struct {
	// Matched is how many runs the filter matched, up to the bound.
	Matched int
	// Done is how many runs were retried or deleted; zero on a dry run.
	Done int
	// Skipped is how many matched runs were left alone because they
	// changed between the listing and the action: no longer in a state
	// the action accepts (workflow.ErrWrongState), gone
	// (workflow.ErrNotFound), or, for a retry, their idempotency key held
	// by another unfinished run (workflow.ErrDuplicate).
	Skipped int
	// Truncated is true when more runs match than the bound: repeat the
	// call for the rest.
	Truncated bool
	// Sample is the first BulkSample matched refs, newest first.
	Sample []workflow.RunRef
}

BulkResult is what a bulk call matched and did.

type HandlerOption

type HandlerOption func(*handlerConfig)

HandlerOption configures NewHandler.

func WithLogger

func WithLogger(l *slog.Logger) HandlerOption

WithLogger logs internal errors (the ones answered with a bare 500) and authorization panics. Nil, the default, discards them.

func WithPayloads

func WithPayloads() HandlerOption

WithPayloads includes each run's payload (base64) in run responses. By default it is left out, because payloads can carry personal data and an admin screen is seen by more people than the code that wrote them.

func WithStacks

func WithStacks() HandlerOption

WithStacks includes the stack trace of each attempt error (workflow.AttemptError.Stack) in run responses. By default it is left out: a stack names source files and functions, which an admin screen does not need to show to everyone who can read run errors.

type MemoryMetrics

type MemoryMetrics struct {
	// contains filtered or unexported fields
}

MemoryMetrics is an in-process, bounded MetricsSource: it counts run ends by status, per workflow, in fixed buckets, fed by the run-finished hook (register Observe with workflow.RunNotifier.OnRunFinished before Start). Each process counts the runs it was told about, so on a durable driver where every process receives every end it shows the fleet, and where ends are delivered to one process it shows that process; it starts empty on every restart (Series.Since says when). An app that needs history across restarts wires its own MetricsSource.

func NewMemoryMetrics

func NewMemoryMetrics(cfg MemoryMetricsConfig) *MemoryMetrics

NewMemoryMetrics returns an empty MemoryMetrics.

func (*MemoryMetrics) Observe

func (m *MemoryMetrics) Observe(_ context.Context, ev workflow.RunEvent)

Observe counts one run end. Its signature is workflow.RunFinishedFunc, so it registers with OnRunFinished directly. A non-terminal status is ignored; a repeat of a recent (run, status, finish time) is counted once.

func (*MemoryMetrics) Series

Series sums the buckets into q's steps. q.Step is rounded up to a multiple of the bucket, and From down to a bucket boundary, so every bucket falls in exactly one point.

type MemoryMetricsConfig

type MemoryMetricsConfig struct {
	// Bucket is the resolution counts are kept at (DefaultMetricsBucket).
	Bucket time.Duration
	// Keep is how long counts are kept (DefaultMetricsKeep).
	Keep time.Duration
	// Dedupe is how many recent (run, status) pairs are remembered so a
	// redelivered end is counted once (DefaultMetricsDedupe). A repeat
	// arriving after that many other ends is counted again.
	Dedupe int
	// Clock prunes old buckets and stamps an event without a FinishedAt;
	// nil means clock.Real.
	Clock clock.Clock
}

MemoryMetricsConfig configures NewMemoryMetrics. Zero or less in a duration or count keeps its default.

type MemoryOutputLimits

type MemoryOutputLimits struct {
	// Runs is how many runs' output is kept; past it the run whose output
	// arrived first is forgotten. Default DefaultOutputRuns.
	Runs int
	// PerRun is how many attempts are kept per run, the newest. Default
	// DefaultOutputPerRun.
	PerRun int
	// Bytes bounds one attempt's Text; past it the text is cut and marked
	// Truncated. Default DefaultOutputBytes.
	Bytes int
}

MemoryOutputLimits bounds a MemoryOutputs. Zero or less in a field keeps its default.

type MemoryOutputs

type MemoryOutputs struct {
	// contains filtered or unexported fields
}

MemoryOutputs keeps captured output in process memory, bounded by MemoryOutputLimits. It is both ends: a middleware.OutputSink for CaptureOutput and an OutputSource for WithOutputs. It sees only the attempts that ran in this process, so it suits one process or a development setup; a fleet stores output somewhere shared and writes its own OutputSource over it.

func NewMemoryOutputs

func NewMemoryOutputs(lim MemoryOutputLimits) *MemoryOutputs

NewMemoryOutputs returns an empty MemoryOutputs.

func (*MemoryOutputs) Deliver

func (m *MemoryOutputs) Deliver(_ context.Context, out middleware.Output) error

Deliver stores one attempt's output under its run. Output without StepInfo names no run and is refused.

func (*MemoryOutputs) RunOutput

func (m *MemoryOutputs) RunOutput(_ context.Context, run workflow.RunID) ([]AttemptOutput, error)

RunOutput returns a copy of the run's stored attempts, oldest first.

type MetricsPoint

type MetricsPoint struct {
	At        time.Time
	Succeeded int64
	Failed    int64
	Cancelled int64
	// Discarded is counted only from a driver with CapRunStates; on
	// others a discarded run arrives as failed or cancelled.
	Discarded int64
}

MetricsPoint is the run ends in [At, At+Step), by terminal status.

func (MetricsPoint) Total

func (p MetricsPoint) Total() int64

Total is every run end in the point.

type MetricsQuery

type MetricsQuery struct {
	From, To time.Time
	// Step is the width of one point. A source with a coarser resolution
	// rounds it up to a multiple of its own (MetricsSeries.Step says what
	// was used).
	Step time.Duration
	// Workflow narrows the counts to one workflow; empty counts all.
	Workflow string
}

MetricsQuery asks for run ends over [From, To) in steps of Step.

type MetricsSeries

type MetricsSeries struct {
	Step   time.Duration
	Points []MetricsPoint
	// Since is the earliest time the source has data for; points before
	// it read zero because nothing was recorded, not because nothing ran.
	// Zero when the source cannot say.
	Since time.Time
}

MetricsSeries is the answer to a MetricsQuery: one point per step from From to To, zero-filled where nothing ended, oldest first.

type MetricsSource

type MetricsSource interface {
	Series(ctx context.Context, q MetricsQuery) (MetricsSeries, error)
}

MetricsSource answers historical run-end counts for the charts screen. The library keeps no history of its own: an app with a metrics store (Prometheus, a warehouse table, an OpenTelemetry backend) adapts it with MetricsSourceFunc, and MemoryMetrics is the in-process default for an app that has none. Implementations must return one point per step over the query, zero-filled.

type MetricsSourceFunc

type MetricsSourceFunc func(ctx context.Context, q MetricsQuery) (MetricsSeries, error)

MetricsSourceFunc adapts a function to MetricsSource.

func (MetricsSourceFunc) Series

Series calls f.

type Monitor

type Monitor struct {
	// contains filtered or unexported fields
}

Monitor polls Admin.QueueStats on the Admin's clock and calls an Alerter when a queue crosses a Threshold, and again when it is back under. It keeps which alerts are open in memory only, so two Monitors on two nodes each alert; run one per deployment.

func NewMonitor

func NewMonitor(a *Admin, alerter Alerter, thresholds []Threshold, opts ...MonitorOption) (*Monitor, error)

NewMonitor returns a Monitor over a. It refuses, with workflow.ErrInvalid, a nil Alerter, no thresholds, a negative field, a threshold that checks nothing, an invalid queue name or two thresholds for one queue; and, with an UnsupportedError, a client QueueStats cannot work on.

func (*Monitor) Check

func (m *Monitor) Check(ctx context.Context) error

Check polls once and sends the alerts the poll calls for. Run calls it on every tick; tests and a cron job can call it directly.

func (*Monitor) Run

func (m *Monitor) Run(ctx context.Context) error

Run polls every interval until ctx ends, then returns ctx's error. A poll error goes to the error handler and the next poll still runs, so a database blip does not stop alerting. Nothing runs before Run is called.

type MonitorOption

type MonitorOption func(*Monitor)

MonitorOption configures NewMonitor.

func WithErrorHandler

func WithErrorHandler(fn func(error)) MonitorOption

WithErrorHandler receives poll errors and recovered Alerter panics. Nil, the default, drops them; Run keeps polling either way.

func WithInterval

func WithInterval(d time.Duration) MonitorOption

WithInterval sets how often Run polls. Zero or less keeps DefaultMonitorInterval.

func WithRepeat

func WithRepeat(d time.Duration) MonitorOption

WithRepeat re-sends an alert every d while its queue stays over the threshold. Zero, the default, alerts once on crossing and once on resolving.

type Operation

type Operation string

Operation names one thing Admin can do. The names are part of the JSON the capabilities route returns, so they never change once released.

const (
	OpRuns      Operation = "runs"       // list and search runs (RunQuerier)
	OpRun       Operation = "run"        // show one run (RunQuerier)
	OpFailures  Operation = "failures"   // list failed runs (RunQuerier)
	OpDiscarded Operation = "discarded"  // list discarded runs (RunQuerier, CapRunStates)
	OpQueues    Operation = "queues"     // queue depth and oldest wait (RunQuerier)
	OpRetry     Operation = "retry"      // retry a failed run from its failed step (RunRetrier)
	OpDelete    Operation = "delete"     // forget a finished run (RunDeleter)
	OpCancel    Operation = "cancel"     // cancel one run (Client.Cancel)
	OpCancelAll Operation = "cancel_all" // cancel every run matching a filter (BulkCanceller)
	OpPause     Operation = "pause"      // pause, resume and list pauses (Pauser)
	OpSchedules Operation = "schedules"  // list schedules with their next fire time (ScheduleLister)
	// OpSchedulePause pauses and resumes one schedule (SchedulePauser).
	OpSchedulePause Operation = "schedule_pause"
	// OpScheduleTrigger runs one schedule now, outside its cron
	// (ScheduleTriggerer).
	OpScheduleTrigger Operation = "schedule_trigger"
	// OpForce ends a run from outside: force-complete or force-fail
	// (RunForcer).
	OpForce Operation = "force"
	// OpRetryAll retries every failed run a filter matches (RunQuerier
	// and RunRetrier).
	OpRetryAll Operation = "retry_all"
	// OpDeleteAll forgets every finished run a filter matches (RunQuerier
	// and RunDeleter).
	OpDeleteAll Operation = "delete_all"
	// OpNodes lists the nodes that are online (NodeLister).
	OpNodes Operation = "nodes"
	// OpHealth reads the client's health report (workflow.CheckHealth);
	// every client supports it, one without HealthChecker saying it
	// cannot tell.
	OpHealth Operation = "health"
	// OpReschedule moves a delayed run to a new time (RunRescheduler).
	OpReschedule Operation = "reschedule"
	// OpCancelStep withdraws one step of a run (StepCanceller).
	OpCancelStep Operation = "cancel_step"
)

The operations. Each needs the capabilities and optional interface listed in Requirements.

type OperationSupport

type OperationSupport struct {
	Operation Operation
	Supported bool
	// Missing is the capabilities the client lacks for the operation;
	// zero when Supported. For OpPause it lists both pause capabilities
	// when the client has neither. It is zero for an unsupported
	// operation that needs no capability, only an interface; Reason then
	// says so.
	Missing workflow.Caps
	// Reason says why an operation that needs no capability is
	// unsupported (its interface is absent); empty otherwise.
	Reason string
}

OperationSupport says whether the client supports one operation, and which capabilities it lacks when it does not.

type Option

type Option func(*Admin)

Option configures New.

func WithClock

func WithClock(clk clock.Clock) Option

WithClock sets the clock QueueStats measures wait times with and the Monitor polls on. Nil means clock.Real.

func WithMaxBulk

func WithMaxBulk(n int) Option

WithMaxBulk bounds how many runs one RetryAll or DeleteAll call acts on. Zero or less keeps DefaultMaxBulk.

func WithMaxScan

func WithMaxScan(n int) Option

WithMaxScan bounds how many queued runs QueueStats reads per call; past it the stats are marked Truncated. Zero or less keeps DefaultMaxScan.

func WithMetrics

func WithMetrics(src MetricsSource) Option

WithMetrics lets Admin.Metrics, the metrics route and the UI's charts read run-end history from src. Without it they report that no metrics source is wired.

func WithOutputs

func WithOutputs(src OutputSource) Option

WithOutputs lets Admin.RunOutput, the run output route and the UI's run page read captured output from src. Without it they report that no output store is wired. Output can hold whatever a step logged, so wiring a source is the opt-in that shows it to admin users.

func WithRunQuerier added in v0.1.1

func WithRunQuerier(rq workflow.RunQuerier, caps workflow.Caps) Option

WithRunQuerier makes the read operations (runs, run, failures, discarded, queue stats, counts) read rq instead of the client, typically a run recorder's store (recorder.Store), whose history outlives the driver's own and covers a driver without a run query. caps is what rq's queries honour (recorder.QueryCaps for a recorder store); a query filter rq lacks is refused, as it would be by the client. Actions (retry, cancel, delete, force, the bulk ones and the listings they act on) still go to the client, which owns the runs. Run falls back to the client's own RunQuerier for a run rq does not hold, so a run the recorder never saw still opens. A nil rq keeps the client.

type OutputSource

type OutputSource interface {
	RunOutput(ctx context.Context, run workflow.RunID) ([]AttemptOutput, error)
}

OutputSource returns the captured output of one run's attempts, oldest first. The library keeps no output itself: CaptureOutput hands each attempt's output to a sink the app chooses, so where it is kept (a table, an object store, a log service) is the app's decision, and an OutputSource is how the admin reads it back. A run with no captured output returns an empty slice, not an error.

type OutputSourceFunc

type OutputSourceFunc func(ctx context.Context, run workflow.RunID) ([]AttemptOutput, error)

OutputSourceFunc adapts a function to OutputSource: the glue that maps another module's output store (one that does not import admin) to AttemptOutput.

func (OutputSourceFunc) RunOutput

func (f OutputSourceFunc) RunOutput(ctx context.Context, run workflow.RunID) ([]AttemptOutput, error)

RunOutput calls f.

type QueueReport

type QueueReport struct {
	Queues []QueueStat
	// Truncated is true when more than the max scan of queued runs exist:
	// every Depth is then a lower bound and the oldest run may be missed.
	Truncated bool
	// At is when the report was taken, on the Admin's clock.
	At time.Time
	// Workers is true when the client implements workflow.WorkerScaler
	// and declares CapWorkerScale: each QueueStat then carries Workers
	// and Running, and every queue the client serves is listed, even
	// with nothing queued.
	Workers bool
}

QueueReport is every queue with at least one queued run, sorted by name.

type QueueStat

type QueueStat struct {
	// Queue is the queue name.
	Queue string
	// Depth is how many runs wait for a worker (StatusQueued), including
	// runs held by a pause. Delayed runs (StatusScheduled) are not counted
	// until they are due.
	Depth int
	// OldestEnqueuedAt is the enqueue time of the oldest queued run.
	OldestEnqueuedAt time.Time
	// OldestWait is how long that run has waited, on the Admin's clock.
	OldestWait time.Duration
	// Workers and Running are this client's worker count for the queue
	// and how many attempts those workers run now. They are filled only
	// when QueueReport.Workers is true; see QueueStats.
	Workers int
	Running int
}

QueueStat is one queue's backlog at the moment QueueStats read it.

type Request

type Request[P any] struct {
	Who      P
	Action   Action
	Resource Resource
	// HTTP is the request, for headers an Authorizer needs (a tenant
	// header, the remote address). Its body has been read; do not read it.
	HTTP *http.Request
}

Request is one HTTP call as the Authorizer sees it: who, which action, which resource. P is the app's own principal type, whatever Identify returns.

type Resource

type Resource struct {
	// Run is the run for ActionShowRun, ActionRetryRun, ActionCancelRun,
	// ActionDeleteRun, ActionForceComplete, ActionForceFail,
	// ActionShowOutput, ActionRescheduleRun and ActionCancelStep.
	Run workflow.RunRef
	// Query is the filter for ActionListRuns, ActionListFailures,
	// ActionCancelRuns, ActionRetryRuns and ActionDeleteRuns, so an
	// Authorizer can scope a caller to its own workflows, queues or labels
	// (a tenant label, for example).
	Query *workflow.RunQuery
	// DryRun is true when a bulk action (ActionCancelRuns,
	// ActionRetryRuns, ActionDeleteRuns) only previews, so an Authorizer
	// can let more people preview than act.
	DryRun bool
	// Metrics is the query for ActionMetrics.
	Metrics *MetricsQuery
	// Step is the step for ActionCancelStep.
	Step string
	// At is the new time for ActionRescheduleRun.
	At time.Time
	// Pause is the target for ActionPause and ActionResume.
	Pause *workflow.PauseTarget
	// Schedule is the schedule for ActionPauseSchedule,
	// ActionResumeSchedule, ActionShowSchedule and ActionTriggerSchedule.
	Schedule workflow.ScheduleID
	// Reason is the operator's reason for ActionForceFail, so an
	// Authorizer can require one or write it to an audit log.
	Reason string
	// File is the UI file for ActionUI ("index.html", "app.js", "app.css").
	File string
}

Resource is what an action is about. The field the action uses is set; the others are zero.

type RunCount

type RunCount struct {
	// Total is how many runs match, up to the max scan.
	Total int
	// Truncated is true when more runs match than the max scan read:
	// Total is then a lower bound and the last pages are missing.
	Truncated bool
	// Cursors holds the cursor that starts each page of the query's
	// page size: Cursors[0] is "" (the first page), Cursors[i] starts
	// page i+1. Pass one as RunQuery.Cursor with the same filters and
	// Limit to open that page.
	Cursors []string
}

RunCount is how many runs match a query and where each page of it starts, for a screen that shows "1–50 of N" and numbered pages over a driver whose cursors are opaque.

type Threshold

type Threshold struct {
	// Queue is the queue it applies to. Empty is the default for every
	// queue that has no Threshold of its own.
	Queue string
	// MaxDepth alerts when Depth is above it.
	MaxDepth int
	// MaxWait alerts when OldestWait is above it.
	MaxWait time.Duration
}

Threshold is when a queue is in trouble: too deep, or its oldest run waiting too long. A zero field is not checked.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL