worker

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 21, 2026 License: AGPL-3.0 Imports: 32 Imported by: 0

Documentation

Overview

Package worker is Atlas's out-of-process job worker: the binary, run as a client of its own HTTP API, leasing jobs of named types and reporting what happened (ADR-0157 step 5, over the protocol of ADR-0007).

It exists so that a service task's work does not have to run on the engine's single writer. An in-process connector's outbound call is charged to the whole engine for its duration; a job this worker takes is charged to the worker.

**What it does not do, and why.** ADR-0157 says the standard worker is "the same binary running the *existing* handler code". That turned out not to be reachable for the connector kinds: every one of them resolves its configuration from the compiled process through a ProcessLookup over the local state store (`mail.Handler(store, lookup, registry)` and its siblings), which a process outside the engine has neither of. Relocating those needs the connector detail to travel with the job — its own slice. What this worker serves instead is the kind that has always been meant for an external worker: a **model-authored** job type, a `<zeebe:taskDefinition type>` whose work is the customer's own code.

The contract for that code is the one the script connector already established: the job arrives as JSON on stdin, and whatever JSON object the command writes to stdout becomes the variables the job completes with. A non-zero exit fails the job, carrying the command's stderr as the message an operator reads on the incident.

Index

Constants

View Source
const (
	DefaultLease   = 5 * time.Minute
	DefaultWait    = 30 * time.Second
	DefaultMaxJobs = 1
	DefaultRetry   = 5 * time.Second
)

Defaults, chosen so an unconfigured worker is safe rather than surprising: it takes one job at a time, holds it for five minutes, and parks for thirty seconds per poll.

Variables

This section is empty.

Functions

func KnownConnectorKinds

func KnownConnectorKinds() []string

KnownConnectorKinds are the kinds BuiltinConnectors implements, for the error a misspelling produces. A kind may serve several job types — script serves one per language — which is why a caller must not check this by counting handlers.

func RunADJob

func RunADJob(ctx context.Context, j Job, dialer ad.Dialer, secret ad.SecretResolver) (map[string]any, error)

RunADJob performs a resolved AD job with the caller's own dialer and secret store. It is exported for the same reason RunMailJob and the rest are: the environment is only the default place a worker's credentials come from, and a caller embedding this package can resolve references from a vault of its own and get the identical operation.

func RunEntraJob

func RunEntraJob(ctx context.Context, j Job, reg *entra.Registry) (map[string]any, error)

RunEntraJob performs a resolved Entra job through a registry the caller owns. It is exported for the same reason RunMailJob and RunSQLJob are: the environment is only the default place a worker's credentials come from, and a caller embedding this package can build a registry from a vault or a managed identity and get the identical call.

func RunMailJob

func RunMailJob(ctx context.Context, j Job, reg *mail.Registry) (map[string]any, error)

RunMailJob sends a resolved mail job through a registry the caller owns. It is exported because the environment is only the *default* place a worker's mail credentials come from: a caller embedding this package can build a registry from a vault, a file, or an instance profile and still get the identical send. It shares mail.Run with the in-process path, so no two of those can disagree about what a resolved mail task means — only about which credentials are in reach.

func RunSQLJob

func RunSQLJob(ctx context.Context, j Job, reg *sqldb.Registry) (map[string]any, error)

RunSQLJob executes a resolved SQL job through a registry the caller owns. It is exported for the same reason RunMailJob is: the environment is only the default place a worker's connection strings come from, and a caller embedding this package can build a registry from a vault or an instance profile and get the identical execution.

Types

type CmdExec

type CmdExec struct {
	Name string
	Args []string
	// Env is added to the process's own environment.
	Env []string
}

CmdExec runs an operating-system command per job: the job arrives as JSON on stdin, and the JSON object the command writes to stdout becomes the variables the job completes with (no output completes with none). A non-zero exit fails the job and carries the command's stderr, which is what an operator reads on the incident.

func (CmdExec) Run

func (c CmdExec) Run(ctx context.Context, j Job) (map[string]any, error)

Run implements Exec.

type ConnectorPayload

type ConnectorPayload struct {
	Kind   string         `json:"kind"`
	Fields map[string]any `json:"fields"`
}

ConnectorPayload is a resolved connector task: what to do, with no engine concepts in it.

type Connectors

type Connectors struct {
	Handlers map[string]Exec
	Names    []string
	// Unconfigured are kinds this worker was asked to serve and holds no
	// configuration for, so it does not subscribe to them. It is reported at startup
	// rather than swallowed: "mail is not being served here" is the answer to why a
	// mail task is waiting, and the worker's log is in the Workers console.
	Unconfigured []string
}

Connectors is what a worker was configured to serve: the handlers, keyed by the job type each answers, and the connector *names* this worker holds credentials for.

The names are reported to the engine on every poll, because they are the half of "can this connector be served" that only the worker knows — once a kind is offloaded the engine holds no credential for it and cannot read another process's environment. The Workers view subtracts them from what deployed models reference to show the names configured nowhere (ADR-0168).

func BuiltinConnectors

func BuiltinConnectors(env func(string) string, kinds ...string) (Connectors, error)

BuiltinConnectors returns handlers for the named connector kinds, keyed by the job type each serves. env looks up this worker's configuration; pass os.Getenv outside of tests.

An unknown kind name yields nothing, and the caller compares counts to catch it: a worker is configured from its own command line, so a name it does not implement is a mistake to report at startup rather than a queue to lease work from and then fail. A kind that *is* implemented but misconfigured returns an error here for the same reason — the operator is still watching at startup, and a per-job discovery would spend a retry budget learning what was knowable before the first poll.

type Exec

type Exec interface {
	Run(ctx context.Context, j Job) (map[string]any, error)
}

Exec does one job's work and returns the variables it completed with, or an error that fails the job.

type ExecFunc

type ExecFunc func(ctx context.Context, j Job) (map[string]any, error)

ExecFunc adapts a function to Exec.

func (ExecFunc) Run

func (f ExecFunc) Run(ctx context.Context, j Job) (map[string]any, error)

Run implements Exec.

type Job

type Job struct {
	JobKey             uint64         `json:"jobKey"`
	Type               string         `json:"type"`
	ProcessInstanceKey uint64         `json:"processInstanceKey"`
	ElementInstanceKey uint64         `json:"elementInstanceKey"`
	ProcessDefKey      uint64         `json:"processDefKey"`
	ElementID          string         `json:"elementId"`
	Retries            int32          `json:"retries"`
	LeaseToken         uint64         `json:"leaseToken"`
	Variables          map[string]any `json:"variables"`
	// Connector is a connector task the engine resolved into plain values, present
	// only for a kind the server no longer serves itself (ADR-0168). Nil for a plain
	// job-worker task, whose work is the customer's own command.
	Connector *ConnectorPayload `json:"connector,omitempty"`
}

Job is one leased job as the engine hands it over.

type Options

type Options struct {
	// Server is the base URL of the Atlas to work for, e.g. http://localhost:8080.
	Server string
	// ID names this worker. It rides on every lease and every report, and it is what
	// the Workers view shows — so give each deployment its own.
	ID string
	// Token, when set, is sent as a bearer credential on every request.
	Token string
	// Handlers is what this worker serves, keyed by job type. A worker with none is a
	// misconfiguration rather than a process that polls forever over nothing.
	Handlers map[string]Exec
	// Lease is how long the engine holds a job for this worker. It must comfortably
	// exceed how long the work takes: an elapsed lease puts the job back on offer, and
	// the report that finally arrives is fenced out.
	Lease time.Duration
	// Wait is how long a poll may block waiting for work before coming back empty.
	Wait time.Duration
	// Retry is how long to wait after a failed poll before trying again. A worker
	// must not exit because the server was restarting, or because the process using
	// its job type is not deployed yet — both are ordinary and both resolve.
	Retry time.Duration
	// MaxJobs is how many jobs one poll may lease. Keep it to what this worker can
	// actually run: leased work nobody is running is work nobody else can take either.
	MaxJobs int
	// Connectors are the connector names this worker holds credentials for, reported
	// to the engine on every poll. Only the worker knows them — once a kind is
	// offloaded the engine holds no credential for it — and the Workers view
	// subtracts them from what deployed models reference to show which names are
	// configured nowhere (ADR-0168). Empty for a worker that serves only plain job
	// types, which need no credential of their own.
	Connectors []string
	// HTTP is the client to use; nil means a default one bounded by Lease + Wait.
	HTTP *http.Client
}

Options configures a Worker.

type Worker

type Worker struct {
	// contains filtered or unexported fields
}

Worker leases jobs of the types it handles and reports what happened. Build one with New.

func New

func New(opts Options) *Worker

New builds a worker, filling in defaults.

func (*Worker) Run

func (w *Worker) Run(ctx context.Context) error

Run works until ctx is cancelled. Each type is polled by its own goroutine, so a slow queue never starves another.

func (*Worker) RunOnce

func (w *Worker) RunOnce(ctx context.Context) error

RunOnce polls every handled type once and works whatever it is given. It is what a test drives, and what a one-shot invocation ("drain the queue and exit") wants.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL