worker

package
v0.0.0-...-266a4be Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 15, 2026 License: MIT Imports: 54 Imported by: 0

Documentation

Index

Constants

View Source
const (
	PullPolicyAlways       = "Always"
	PullPolicyIfNotPresent = "IfNotPresent"
	PullPolicyNever        = "Never"
)

PullPolicyAlways, PullPolicyIfNotPresent, and PullPolicyNever are the accepted backend.docker.image_pull_policy values. They mirror the Kubernetes backend's image_pull_policy spelling so operators configure the same three values for either backend.

View Source
const (

	// Worker-observed setup phases reported as run client events. warp-server
	// ingests them as setup metrics, labeled with the run's current timeline
	// phase (oz_run_claimed while the container is still being prepared).
	// SetupEventContainerStart spans container creation through a successful
	// start call on the Docker backend, and the span from the end of init
	// (or scheduling) to the task container start on the Kubernetes backend.
	SetupEventImagePull      = "setup_worker_image_pull"
	SetupEventSidecarPrep    = "setup_worker_sidecar_prep"
	SetupEventContainerStart = "setup_worker_container_start"

	// Kubernetes-backend phases. SetupEventJobCreate is timed directly around
	// the Job create call; the other phases are derived from pod status by
	// kubernetesSetupPhaseTracker.
	SetupEventJobCreate    = "setup_worker_job_create"
	SetupEventPodSchedule  = "setup_worker_pod_schedule"
	SetupEventSetupCommand = "setup_worker_setup_command"
)
View Source
const (
	InitialReconnectDelay = 1 * time.Second
	MaxReconnectDelay     = 60 * time.Second
	ReconnectBackoffRate  = 2.0

	HeartbeatInterval      = 30 * time.Second
	PongWait               = 60 * time.Second
	WriteWait              = 10 * time.Second
	BackendShutdownTimeout = 10 * time.Second
)
View Source
const DispatchPayloadVersion = 1

DispatchPayloadVersion is the schema version of DispatchPayload. It is bumped when the payload contract changes in a way operator dispatch commands must be aware of.

Variables

This section is empty.

Functions

This section is empty.

Types

type Backend

type Backend interface {
	// ExecuteTask runs the agent for the given task parameters. Execution
	// failures are surfaced as ExecuteOutcomeError with an error that is (or
	// wraps) *TaskFailure.
	ExecuteTask(ctx context.Context, params *TaskParams) ExecuteResult
	// CancelTask requests cancellation after ExecuteTask returns. A nil error
	// means the request was accepted, not that asynchronous resources stopped.
	// Failed requests may be retried and must be execution-scoped and idempotent.
	CancelTask(ctx context.Context, params *CancelParams) error
	// PreservesTasksOnShutdown reports whether active task execution units can
	// safely outlive the worker process during shutdown.
	PreservesTasksOnShutdown() bool
	// Shutdown cleans up backend resources.
	Shutdown(ctx context.Context)
}

Backend defines the interface for task execution backends.

type CancelParams

type CancelParams struct {
	TaskID      string
	ExecutionID string
}

CancelParams carries the minimal, non-secret identifiers a backend needs to cancel a task. It deliberately excludes env/secrets so the worker need not retain secrets for the lifetime of a spawned task.

type CommandBackend

type CommandBackend struct {
	// contains filtered or unexported fields
}

CommandBackend hands task execution to an operator-configured command, which dispatches the task to a remote runtime over any transport. Execution is fire-and-forget: a successful dispatch returns ExecuteOutcomeSpawned, and the remote oz agent reports terminal state to warp-server itself by running oz harness-support report-shutdown.

func NewCommandBackend

func NewCommandBackend(ctx context.Context, config CommandBackendConfig) (*CommandBackend, error)

NewCommandBackend constructs a command backend, requiring a dispatch command.

func (*CommandBackend) CancelTask

func (b *CommandBackend) CancelTask(ctx context.Context, params *CancelParams) error

CancelTask best-effort cancels a dispatched task via the configured cancel command. It is a no-op when no cancel command is configured.

func (*CommandBackend) Capabilities

func (b *CommandBackend) Capabilities() backendCapabilities

func (*CommandBackend) ExecuteTask

func (b *CommandBackend) ExecuteTask(ctx context.Context, params *TaskParams) ExecuteResult

ExecuteTask dispatches the task by invoking the configured dispatch command with the JSON payload on stdin. It returns ExecuteOutcomeSpawned on success.

func (*CommandBackend) PreservesTasksOnShutdown

func (b *CommandBackend) PreservesTasksOnShutdown() bool

PreservesTasksOnShutdown reports true: dispatched tasks run on the operator's remote runtime, independent of this worker, so worker shutdown must not cancel them.

func (*CommandBackend) Shutdown

func (b *CommandBackend) Shutdown(ctx context.Context)

Shutdown has nothing to clean up; the backend owns no local resources.

type CommandBackendConfig

type CommandBackendConfig struct {
	// DispatchCommand is the shell command invoked (via /bin/sh -c) to dispatch
	// a task. It receives the DispatchPayload as JSON on stdin. Required.
	DispatchCommand string
	// CancelCommand, when set, is invoked (via /bin/sh -c) to best-effort cancel
	// a previously dispatched task.
	CancelCommand string
	// DispatchTimeout bounds the dispatch command's runtime. Zero uses defaultDispatchTimeout.
	DispatchTimeout time.Duration
	// Env contains extra environment variables exposed to the dispatch and
	// cancel commands. Task env/secrets are NOT placed here; they travel only in
	// the JSON payload on stdin.
	Env map[string]string
	// ServerRootURL and WorkerID are copied into the dispatch payload.
	ServerRootURL string
	WorkerID      string
}

CommandBackendConfig configures the command backend, which dispatches tasks to an operator-owned runtime over any transport by invoking a shell command.

type Config

type Config struct {
	APIKey        string
	WorkerID      string
	WebSocketURL  string
	ServerRootURL string
	LogLevel      string
	BackendType   string // "docker", "direct", or "kubernetes"
	// MaxConcurrentTasks caps how many tasks may execute locally at once
	// (0 means unlimited). A task's slot is released when the backend's
	// ExecuteTask returns, so for backends that spawn tasks fire-and-forget
	// (e.g. command), a slot is held only for the brief dispatch and the limit
	// effectively does not bound the number of remote tasks running at once.
	MaxConcurrentTasks int
	// OneShot makes the worker accept one task and exit after it reaches a
	// terminal outcome. Backend support is validated before initialization.
	OneShot bool
	// IdleOnComplete is passed to the oz CLI's --idle-on-complete flag for every task.
	// Empty string means use the oz CLI default (45m). Use "0s" to disable idle.
	IdleOnComplete string
	// SessionSharingServerURL, when non-empty, is forwarded to the oz CLI via --session-sharing-server-url.
	SessionSharingServerURL string
	CollectTaskLogs         bool

	// Backend-specific configs. Only the one matching BackendType should be set.
	Docker     *DockerBackendConfig
	Direct     *DirectBackendConfig
	Kubernetes *KubernetesBackendConfig
	Command    *CommandBackendConfig
}

type DirectBackend

type DirectBackend struct {
	// contains filtered or unexported fields
}

DirectBackend executes tasks directly on the host without Docker.

func NewDirectBackend

func NewDirectBackend(ctx context.Context, config DirectBackendConfig) (*DirectBackend, error)

NewDirectBackend creates a new direct backend, verifying the oz CLI is available.

func (*DirectBackend) CancelTask

func (b *DirectBackend) CancelTask(context.Context, *CancelParams) error

CancelTask is a no-op: cancelling the ExecuteTask context fully stops a direct-backend task.

func (*DirectBackend) Capabilities

func (b *DirectBackend) Capabilities() backendCapabilities

func (*DirectBackend) ExecuteTask

func (b *DirectBackend) ExecuteTask(ctx context.Context, params *TaskParams) ExecuteResult

ExecuteTask runs the agent directly on the host.

func (*DirectBackend) PreservesTasksOnShutdown

func (b *DirectBackend) PreservesTasksOnShutdown() bool

func (*DirectBackend) Shutdown

func (b *DirectBackend) Shutdown(ctx context.Context)

Shutdown cleans up any workspace directories left behind under the workspace root.

type DirectBackendConfig

type DirectBackendConfig struct {
	WorkspaceRoot   string
	TargetDir       string // If set, run all tasks in this directory instead of creating per-task workspaces.
	OzPath          string // Path to the oz CLI binary. If empty, looks up "oz" in PATH.
	SetupCommand    string
	TeardownCommand string
	NoCleanup       bool
	Env             map[string]string
}

DirectBackendConfig holds configuration specific to the direct (non-containerized) backend.

type DispatchPayload

type DispatchPayload struct {
	Version       int                  `json:"version"`
	RunID         string               `json:"run_id"`
	ExecutionID   string               `json:"execution_id"`
	ServerRootURL string               `json:"server_root_url"`
	WorkerID      string               `json:"worker_id"`
	DockerImage   string               `json:"docker_image"`
	BaseArgs      []string             `json:"base_args"`
	Env           map[string]string    `json:"env"`
	Sidecars      []types.SidecarMount `json:"sidecars"`
	Task          *types.Task          `json:"task"`
}

DispatchPayload is the stable, versioned JSON contract handed to an operator's dispatch command (on stdin) by the command backend. It contains everything a remote runtime needs to launch the oz agent for a task. Secrets (e.g. GitHub tokens) travel only inside Env here, never via the dispatch subprocess's own environment or argv.

func NewDispatchPayload

func NewDispatchPayload(params *TaskParams, serverRootURL, workerID string) *DispatchPayload

NewDispatchPayload builds a DispatchPayload from backend-agnostic TaskParams plus the worker-level server URL and worker ID. The TaskParams.EnvVars slice (KEY=VALUE entries) is converted into the Env map, splitting on the first '=' so values may themselves contain '='. Later entries win on duplicate keys.

type DockerBackend

type DockerBackend struct {
	// contains filtered or unexported fields
}

DockerBackend executes tasks in Docker containers.

func NewDockerBackend

func NewDockerBackend(ctx context.Context, config DockerBackendConfig) (*DockerBackend, error)

NewDockerBackend creates a new Docker backend, connecting to the Docker daemon.

func (*DockerBackend) CancelTask

func (b *DockerBackend) CancelTask(ctx context.Context, params *CancelParams) error

CancelTask removes the captured execution container, including when normal cleanup failed.

func (*DockerBackend) Capabilities

func (b *DockerBackend) Capabilities() backendCapabilities

func (*DockerBackend) ExecuteTask

func (b *DockerBackend) ExecuteTask(ctx context.Context, params *TaskParams) ExecuteResult

ExecuteTask runs the agent in a Docker container.

func (*DockerBackend) PreservesTasksOnShutdown

func (b *DockerBackend) PreservesTasksOnShutdown() bool

func (*DockerBackend) Shutdown

func (b *DockerBackend) Shutdown(ctx context.Context)

Shutdown closes the Docker client.

type DockerBackendConfig

type DockerBackendConfig struct {
	NoCleanup bool
	Volumes   []string
	Env       map[string]string
	// ImagePullPolicy controls how the main task image and every Warp/additional sidecar image
	// are resolved before use: PullPolicyAlways, PullPolicyIfNotPresent, or PullPolicyNever. An
	// empty value defaults to PullPolicyAlways, preserving the backend's original
	// unconditional-pull behavior for installations that don't set it.
	ImagePullPolicy string
	SidecarImage    string
}

DockerBackendConfig holds configuration specific to the Docker backend.

type ExecuteOutcome

type ExecuteOutcome int

ExecuteOutcome describes how a backend handled a task in ExecuteTask.

const (
	// ExecuteOutcomeError means the task did not run to completion: it failed
	// to start, failed while running, or was cancelled. ExecuteResult.Error
	// carries the details.
	ExecuteOutcomeError ExecuteOutcome = iota
	// ExecuteOutcomeCompleted means the task was started and the backend
	// waited for it to finish successfully.
	//
	// The worker holds a concurrency slot until the task completes. After
	// the task completes, the worker also sends a terminal completion
	// message to the server.
	ExecuteOutcomeCompleted
	// ExecuteOutcomeSpawned means the task was started but the backend did not
	// wait for completion.
	//
	// The worker treats the task as accepted-but-not-finalized and must NOT
	// send a terminal completion message. Because ExecuteTask returns at
	// hand-off, a spawned task holds its concurrency slot only for the
	// duration of the dispatch, so MaxConcurrentTasks does not bound the
	// number of spawned tasks running remotely.
	ExecuteOutcomeSpawned
)

type ExecuteResult

type ExecuteResult struct {
	Outcome ExecuteOutcome
	Error   error
}

ExecuteResult contains the outcome of a backend's ExecuteTask call.

Error is set only when Outcome is ExecuteOutcomeError.

type KubernetesBackend

type KubernetesBackend struct {
	// contains filtered or unexported fields
}

KubernetesBackend executes tasks in Kubernetes Jobs.

func NewKubernetesBackend

func NewKubernetesBackend(ctx context.Context, config KubernetesBackendConfig) (*KubernetesBackend, error)

NewKubernetesBackend creates a new Kubernetes backend and validates startup requirements.

func (*KubernetesBackend) CancelTask

func (b *KubernetesBackend) CancelTask(ctx context.Context, params *CancelParams) error

CancelTask deletes the task Job so a server-requested cancellation actually stops the sandbox. Cancelling the ExecuteTask context only stops the local watch: the Job is deliberately left in place on context cancellation so that worker shutdown never kills a healthy run, which means an explicit cancellation has to remove it here. Without this, a Job whose pod is still pending outlives the cancellation and boots an agent against a run the server has already finished. A Job that no longer exists is not an error.

func (*KubernetesBackend) Capabilities

func (b *KubernetesBackend) Capabilities() backendCapabilities

func (*KubernetesBackend) ExecuteTask

func (b *KubernetesBackend) ExecuteTask(ctx context.Context, params *TaskParams) (res ExecuteResult)

ExecuteTask runs the agent in a Kubernetes Job.

func (*KubernetesBackend) PreservesTasksOnShutdown

func (b *KubernetesBackend) PreservesTasksOnShutdown() bool

func (*KubernetesBackend) Shutdown

func (b *KubernetesBackend) Shutdown(ctx context.Context)

Shutdown intentionally does not delete task Jobs.

Kubernetes Jobs are the durable execution unit for this backend. During worker pod disruption (for example Karpenter node rotation), deleting Jobs here would kill otherwise healthy running sessions.

type KubernetesBackendConfig

type KubernetesBackendConfig struct {
	WorkerID              string
	Namespace             string
	Kubeconfig            string
	DefaultImage          string
	ImagePullPolicy       string
	UseImageVolumes       bool
	PreflightImage        string
	SidecarImage          string
	SetupCommand          string
	TeardownCommand       string
	NoCleanup             bool
	ExtraLabels           map[string]string
	ExtraAnnotations      map[string]string
	ActiveDeadlineSeconds *int64
	TTLSecondsAfterFinish *int32
	WorkspaceSizeLimit    *resource.Quantity
	UnschedulableTimeout  *time.Duration
	// CodingCLISidecars maps harness config name (e.g. "claude", "codex") to a custom
	// Docker image for the coding CLI sidecar. When non-empty for a task's harness,
	// the worker overrides or injects the sidecar at /mnt/{harness}-cli-sidecar.
	CodingCLISidecars map[string]string
	// TaskEnv contains runtime-only env overrides from CLI -e/--env. Declarative
	// task container env in file or Helm config should be set in PodTemplate.
	TaskEnv map[string]string
	// PodTemplate, when non-nil, is the declarative PodSpec template for task
	// Jobs. Worker-required fields are overlaid at execution time.
	PodTemplate *corev1.PodSpec
	// PreflightResources, when non-nil, overrides the default cpu/memory
	// requests and limits applied to startup preflight Job containers.
	// Defaults: requests 10m CPU / 16Mi memory; limits 100m CPU / 64Mi memory.
	PreflightResources *corev1.ResourceRequirements
}

KubernetesBackendConfig holds configuration specific to the Kubernetes backend.

type TaskFailure

type TaskFailure struct {
	// contains filtered or unexported fields
}

TaskFailure is the structured error backends return from ExecuteTask when task execution fails. Backends record only the facts they observe (metrics labels and the failing process's exit status); the worker adds lifecycle context backends cannot see (graceful shutdown) before reporting the failure to warp-server.

func (*TaskFailure) Error

func (e *TaskFailure) Error() string

func (*TaskFailure) Unwrap

func (e *TaskFailure) Unwrap() error

type TaskParams

type TaskParams struct {
	TaskID      string
	ExecutionID string
	Task        *types.Task

	// EnvVars contains pre-resolved common environment variables (TASK_ID, Git config,
	// assignment env vars). Backends append their own config-specific env vars.
	EnvVars []string

	// BaseArgs contains the base CLI arguments for the agent command
	// (agent run --share ... --task-id ... --server-root-url ... + augmented args).
	// Backends prepend their invocation prefix and append backend-specific flags.
	BaseArgs []string

	// DockerImage is the resolved Docker image name (with default fallback applied).
	// Backends that don't use Docker can ignore this.
	DockerImage string

	// Sidecars is the unified list of sidecar mounts for this task. The Warp agent
	// sidecar (mounted at /agent) is included as the first entry when present, followed
	// by any additional sidecars from the assignment. Backends that don't use sidecars
	// can ignore this.
	Sidecars []types.SidecarMount

	// InstanceShape, when non-nil, is the resolved compute size for this task.
	// Containerized backends apply it as CPU/memory limits (Docker) or resource
	// requests/limits (Kubernetes). Backends that cannot enforce a shape (direct) ignore it.
	InstanceShape *types.InstanceShape

	// SetupEvents reports worker-observed setup phase durations to warp-server.
	// It may be nil, and all of its methods are safe to call on a nil reporter.
	SetupEvents *setupEventReporter
	Logs        *tasklogs.Reporter
}

TaskParams contains pre-processed task parameters common to all backends. This provides a layer of abstraction between the wire-format TaskAssignmentMessage and the backend interface, so backends don't need to handle common concerns like resolving environment variables, choosing default images, or building base CLI args.

type Worker

type Worker struct {
	// contains filtered or unexported fields
}

func New

func New(ctx context.Context, config Config) (*Worker, error)

func (*Worker) Run

func (w *Worker) Run() error

Run drives the worker's processing loops until the server context is cancelled for graceful shutdown or a one-shot task finishes: - The WebSocket connection loop - Task acceptance and processing - Client state maintained by backends

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL