worker

package
v1.0.0-rc.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 23, 2026 License: Apache-2.0 Imports: 49 Imported by: 0

README

Worker

apptown-worker (cmd/apptown-worker) runs an AppTown's tasks on a machine that shares no files with it. This package is its server: an HTTPS API over mutual TLS that accepts only the client certificate naming its one installation, workspaces cloned from Git bundles, tasks run through an execution.TaskDriver (the worker's own Docker in production), and results returned as Git bundles of the commits a task made.

Before acting on a task the worker writes one fsynced record of the request (state/records/<key>.json): a restarted worker refuses the same key for a different request, and finds its tasks again through its driver. It knows nothing about projects, workflows or AppTown's database; AppTown's side is execution.WorkerTasks. workertest starts a worker in a test: in process, as a process on this machine driving its Docker (StartOnHost), or in its own Docker-in-Docker container.

Every task's, check's and chat's container mounts the worker's tool cache (tool_cache.go, Server.ToolCache, <state>/tool-cache/mise) writable at execution.ToolCacheMount, where AppTown points mise's download cache: one cache of release listings and archives per worker, never a tool's installation, which stays in the container.

It also runs chats (sessions.go, session_io.go): AppTown's own chat runtime (a console.Agent, pi under tmux in production) on the worker's Docker, with the chat's files under the state directory. A session's record is marked removed before its runtime is taken away and is kept, so a late start cannot bring an ended chat back; a removal succeeds only once Docker shows nothing of it. Stopping or rebuilding keeps the session; a chat is rebuilt only while it is stopped, by Docker's answer, so a new image, model or apptown CLI (a new content-named blob, so a new mount) reaches a running chat when it next stops. On a worker the chat runtime gives each session its own cache volume, removed with its runtime, the worker's tool cache (tool_cache.go) in its container, and the worker's chat limits on the chat's container, its daemon and a save's container; the daemon uses the worker's resolvers and the mirror AppTown sends in the session's spec.

On AppTown, RemoteAgent is that runtime for the chat service: the same interface, answered by the worker (SessionClient), so the service is unchanged. What AppTown mounts into a chat travels as a blob named by its content (the apptown binary, generated registry files); the chat's other files are sent when they change; the work tree is made once from a bundle of the change's branch; messages, interrupts and the terminal go through; the chat's record is copied back with the time of the last good copy, and a turn handed back in that copy is the moment to save. A save is a bundle of the chat's commits since the branch's tip, made by the chat's own git in a short-lived container of the chat's image with only its work tree mounted (Save); AppTown imports it through its checked import and publishes it (repositoryedit.SaveChat). Ending a chat stops it first, then makes its last save with what it left uncommitted (SaveLast, through the Ending hook), copies its record, and removes it. AppTown calls the worker; the worker never calls AppTown. On AppTown, console.Placed gives each chat the runtime its record places it on. An answer AppTown cannot get is execution.ErrWorkerUnavailable, never absence, and the chat shows as unavailable. The agent image is built by the worker from its own deploy/apptown-agent and named by that context's content (AgentImage), so an AppTown and a worker of different versions refuse rather than mix; GET /v1/agent-image reports its name and labels. Reach reports which registries the worker can reach.

A task's inputs (task_io.go) are blobs AppTown sent, checked against their bounds by the blobs' own sizes before anything is created (AppTown asks, POST /v1/inputs/check, before it uploads a source, so a refusal leaves no workspace), copied into the task's folder (state/task-inputs/<key>) and mounted read-only at their paths. Each workspace has a state folder (state/workspace-state/<key>) mounted writable at /apptown/state, beside the tree, so nothing in it is collected; it goes with the workspace. A task's declared result files are read once its container has exited, confined as a check's report is, and kept durably with the task (state/task-results/<key>) until it is released, so a restarted worker answers the same, and a later change to the tree does not reach them. A release build's declared images (task_images.go) are captured at the same moment, under the task's lock: each OCI layout in the state folder, a folder of regular files reached and holding no link, within its admitted size, is copied into state/task-results/<key>/images, and only the record written after every copy makes it visible. The index and blobs are served file by file (GET /v1/tasks/{key}/images/{name}/{file}) with their length. A file is opened under the task's lock, so after a release is decided a new open is gone (410); a read already open finishes.

A task's private Docker (private.go, from the private-Docker plan) lives from its start to its exit. Its record names its network, services and daemon before any exists; the network is made when it is ensured, the services (each with the worker's service limits, by default 1 CPU, 1 GiB and 256 processes, waited on until healthy) and the daemon (internal/dind: the task's own limits, its tree at the same path, its client state in /apptown/state/docker, no cache or socket of the worker's) when it first starts, and never again. A failed start removes what it made and is refused with the reason. When the task exits its daemon and services are stopped, confirmed by Docker, before it reads as exited and its results are captured; until then it reads as running. One running without them is reported absent (lost). A stop stops the whole invocation, the task and then its daemon and services, and fails unless Docker confirms them stopped; a workspace is not collected while any task in it has private Docker not confirmed stopped. Ensure, start, stop and release of a task are serialized by its lock. Release first writes its decision (state/released/<key>), so from then on, after a restart too, a request for the key is refused as stale; it then removes everything, and confirms only when nothing of the key is left; one interrupted is asked again. The worker's sweep stops a task whose admission has expired and removes its private Docker, with or without AppTown.

New work is refused while a disk it writes to is short of space (diskspace.Admit, admitNew): a new task is admitted once, before the first thing of it is written (its own workspace's upload, or else its ensure before its record), reading the state folder and the task driver's Docker data folder; the driver does not read it again (execution.DockerTasks.CallerAdmitsDisk). A disk filling after that is the accepted race. An image capture that would leave the state folder under its floor is recorded refused, not copied. GET /v1/disks reports both readings.

A worker measures its profile when it starts (profile.go): what docker info says of its Docker (OS, architecture, kernel, cgroup mode, version, storage driver), the private daemon's image (pulled first if absent): its tag, Docker's local id of it, its repository digest and the version its own dockerd --version reports, whether a privileged container runs (probed), and the operator's -exclusive declaration. GET /v1/profile reports it; AppTown records it with each attempt admitted with private Docker and sends it in the request. A new task's ensure, and its first start, measure Docker again and refuse private Docker admitted on another profile, or on none: the OS, architecture, kernel, Docker version, storage driver, cgroup mode or (for a request with a daemon) the daemon's image, digest or version changed. The daemon's version is measured again only when the image's id is not the one measured at start. Operating guide: design/operations/workers.md.

Documentation

Overview

Package worker is the execution side of a separated AppTown (design/operations/workers.md): a small program on a machine that shares no files with AppTown, which runs tasks on its own Docker daemon for one registered installation.

It knows nothing about projects, workflows or AppTown's database. The AppTown reaches it over mutual TLS (a client certificate naming the installation it is registered to) and moves source only as Git bundles: a task's source arrives as a bundle the worker clones, and its result leaves as a bundle of the commits it made.

Before it acts on a task, the worker writes one fsynced record of the request under its state directory. A restarted worker answers from those records and from Docker, so a repeated request finds the same task.

Index

Constants

View Source
const (

	// PreviewHostHeader carries the visitor's Host across the worker hop;
	// only AppTown sets it, over the worker's authenticated connection.
	PreviewHostHeader = "X-AppTown-Preview-Host"
	// PreviewTargetHeader carries the visitor's request target (path and
	// query, escaped as sent), so this server's router never cleans or
	// redirects a visitor's path.
	PreviewTargetHeader = "X-AppTown-Preview-Target"
	// PreviewSessionHeader and PreviewSeqHeader are AppTown session and
	// the number within it that a request changing a preview carries (see
	// previewSessions).
	PreviewSessionHeader = "X-AppTown-Preview-Session"
	PreviewSeqHeader     = "X-AppTown-Preview-Seq"
	// PreviewClaimHeader, "required", answers a request refused because the
	// worker's fence is lost: AppTown claims the worker again.
	PreviewClaimHeader = "X-AppTown-Preview-Claim"
)
View Source
const (
	ProtocolHeader = "X-AppTown-Worker-Protocol"
	// Protocol is what this build's AppTown speaks. 2: a task's observation
	// says whether Docker killed it at its memory limit (OOMKilled), and a
	// AppTown that settles scripts by it must not hear a worker that cannot
	// say, which would read every such kill as an ordinary exit.
	Protocol = 2
	// ProtocolMin and ProtocolMax are what this build's worker serves: 1
	// for one release, so a worker installed first serves its old AppTown.
	ProtocolMin = 1
	ProtocolMax = 2
)

The worker protocol says which AppTown installations a worker serves (design/operations/workers.md, "Versions"). An AppTown sends the number it speaks on every request; a worker serves a range of them. A change that breaks the older side raises AppTown's number and the worker's ProtocolMax, and the worker keeps the previous number for one release, so a worker installed before its AppTown (the demo's order) still serves the AppTown it replaces.

View Source
const MaxBundleBytes = 256 << 20

MaxBundleBytes bounds a source or result bundle in either direction.

View Source
const RecordAsOfFile = "record-as-of"

RecordAsOfFile, in a session's directory on AppTown, holds the time of the last good copy of the chat's record from its worker.

View Source
const SaveRef = "refs/apptown/save"

SaveRef is the ref a save's bundle names its tip by.

Variables

View Source
var (
	// ErrSessionRemoved: the worker removed the session, and will not start
	// it again.
	ErrSessionRemoved = errors.New("the worker removed this session")
	// ErrNoSession: the worker has no record of the session.
	ErrNoSession = errors.New("the worker has no such session")
	// ErrSessionBusy: the worker refused to rebuild a running chat.
	ErrSessionBusy = errors.New("the chat is busy")
)
View Source
var ErrClaimRequired = errors.New("the worker lost the order of its previews' requests, and must be claimed again")

ErrClaimRequired is the worker refusing a request because it lost the order of its previews' requests: AppTown must claim it again.

View Source
var ErrNotConfirmed = errors.New("not confirmed: the message may or may not have reached the chat")

ErrNotConfirmed: AppTown could not confirm a message reached the chat. It may or may not have; the person decides whether to send it again.

Functions

func AgentImage

func AgentImage(dir string) (string, error)

AgentImage is the name of the agent image built from the build context dir.

func BuildAgentImage

func BuildAgentImage(ctx context.Context, docker, dir string) (string, error)

BuildAgentImage builds the agent image from dir with docker, unless the worker's Docker has it. It answers the image's name.

func NextSession

func NextSession(file string, now func() time.Time) (int64, error)

NextSession is the next AppTown session number, kept in file: greater than every earlier one, and than AppTown's clock (now; nil is the real one), and written durably before it is answered. A counter that cannot be read or written is an error, and no session: two starts must never share one.

func RecordAsOf

func RecordAsOf(dir string) time.Time

RecordAsOf reads when a chat's record was last copied from its worker, from its session directory on AppTown; zero when never.

func SpeakProtocol

func SpeakProtocol(rt http.RoundTripper) http.RoundTripper

SpeakProtocol wraps an AppTown's transport to a worker so every request states the protocol this AppTown speaks.

func StripForwarding

func StripForwarding(h http.Header)

StripForwarding removes every forwarding header a request carries: the whole X-Forwarded-* family, Forwarded and X-Real-Ip. A proxy that sets its own forwarding headers first removes whatever a visitor sent, so an app behind it trusts only the proxy's.

func TLSConfig

func TLSConfig(cert tls.Certificate, clients *x509.CertPool) *tls.Config

TLSConfig is the worker's transport: TLS 1.3 only, and every client must present a certificate signed by clients. Which installation the certificate names is checked per request (Handler).

func ToolCacheSize

func ToolCacheSize(dir string) int64

ToolCacheSize is how many bytes the files under dir take, as far as they could be read.

func ToolCacheVolume

func ToolCacheVolume(dir string) string

ToolCacheVolume is the mount argument that presents dir, writable, at execution.ToolCacheMount.

Types

type AdmissionRefused

type AdmissionRefused struct{ Reason string }

AdmissionRefused is the worker's refusal of a preview its daemon cannot hold: the contract's answer, not the worker's unavailability.

func (*AdmissionRefused) Error

func (e *AdmissionRefused) Error() string

type CheckAnswer

type CheckAnswer struct {
	Termination execution.Termination `json:"termination"`
	ExitCode    int                   `json:"exitCode"`
	RunErr      string                `json:"runErr,omitempty"`
	Output      []byte                `json:"output,omitempty"`
	Report      []byte                `json:"report,omitempty"`
	ReportErr   string                `json:"reportErr,omitempty"`
}

CheckAnswer is what a check's run answered.

type CheckSpec

type CheckSpec struct {
	Image        string   `json:"image"`
	Argv         []string `json:"argv"`
	Env          []string `json:"env,omitempty"`
	NoEntrypoint bool     `json:"noEntrypoint,omitempty"`
	// Egress lets the container reach the network, where a scanner binding
	// asks for it; otherwise it has none.
	Egress bool `json:"egress,omitempty"`
	// Limits bound the check's container (and so a test run in it); a
	// request with none (an older AppTown) runs under the task defaults.
	Limits         execution.Limits `json:"limits,omitzero"`
	TimeoutSeconds int              `json:"timeoutSeconds"`
	Writable       bool             `json:"writable,omitempty"`
	Report         string           `json:"report,omitempty"`
	// History says a bundle follows the tree, to present read-only at
	// /history.
	History bool `json:"history,omitempty"`
	// ExportImage is an image on this worker (by ID) saved into the tree
	// before the check runs (an image scan).
	ExportImage string `json:"exportImage,omitempty"`
	// NoTree runs the command with nothing mounted, in the image's own
	// working directory (an exec-image check).
	NoTree bool `json:"noTree,omitempty"`
	// Files are generated files the check sees read-only (execution.File),
	// written beside its tree, in the check's folder.
	Files []execution.File `json:"files,omitempty"`
	// Services run beside the check on a network of its own
	// (execution.Request.Services), labelled as the check is, so the sweep
	// removes them with it.
	Services []deploy.ServiceSpec `json:"services,omitempty"`
}

CheckSpec is the wire form of one check container: the runner's request with no path of AppTown's in it.

type Checks

type Checks struct {
	URL    string
	Client *http.Client
	// Limits bound each check's container on the worker (AppTown's task
	// limits).
	Limits execution.Limits
}

Checks is AppTown's check executor on a worker (design/operations/workers.md, "Checks on the worker"): Run answers a check container's request as the local runner does, having run it on the worker. A request the worker cannot honour (a mount of this machine's, a host route, a local process) is refused, never run here.

func (Checks) Image

Image prepares an image on the worker (execution.ImageRequest): the tree a build's context is under goes with the request. A worker that does not answer is an error, never "no image".

func (Checks) Run

Run sends the staged tree and the spec, and reads the answer. The worker failing to answer is a run that did not happen (LaunchFailed), never a pass.

type DeliverRequest

type DeliverRequest struct {
	Text string `json:"text"`
}

DeliverRequest is a message for the chat.

type DockerImages

type DockerImages struct{ Binary string }

DockerImages answers whether the worker's Docker has an image.

func (DockerImages) Has

func (d DockerImages) Has(ctx context.Context, image string) (bool, error)

func (DockerImages) Labels

func (d DockerImages) Labels(ctx context.Context, image string) (map[string]string, error)

Labels answers an image's labels.

type PreviewImages

type PreviewImages struct {
	Parts map[string]string `json:"parts"`
	Init  map[string]string `json:"init,omitempty"`
}

PreviewImages are a release's images by part and by init step.

type PreviewSpec

type PreviewSpec struct {
	Name         string `json:"name"`
	StackID      string `json:"stackId"`
	Slug         string `json:"slug"`
	Incarnation  string `json:"incarnation"`
	ProbeSeconds int    `json:"probeSeconds"`
	PageCheck    bool   `json:"pageCheck,omitempty"`
	// VisitorHost is the host a visitor's request carries (BootRequest).
	VisitorHost string `json:"visitorHost,omitempty"`
	// Images, for a release's preview, are the release's images by digest
	// (<repository>@sha256:<hex>): the boot pulls them with the prepared
	// registry login and builds nothing. nil builds the tree, as a change's
	// preview does.
	Images *PreviewImages `json:"images,omitempty"`
}

PreviewSpec is what a preview boot is, besides its prepared value and tree.

type PreviewState

type PreviewState struct {
	Running     bool   `json:"running"`
	Incarnation string `json:"incarnation,omitempty"`
	ContainerID string `json:"containerId,omitempty"`
	// Image and Digest are what the app runs, for a boot's answer.
	Image  string `json:"image,omitempty"`
	Digest string `json:"digest,omitempty"`
	// BrowserValues are the values the app's page is given, read from the
	// container, for a state's answer: null when they cannot be read, and
	// an empty object when the container states there are none.
	BrowserValues map[string]string `json:"browserValues"`
	// Boot failure, for a boot request: the phase, the error and the log.
	Phase   string `json:"phase,omitempty"`
	Error   string `json:"error,omitempty"`
	LogTail string `json:"logTail,omitempty"`
	// Page is what the boot's front-page check saw, for a boot's answer,
	// passed or refused.
	Page *docker.PageWitness `json:"page,omitempty"`
}

PreviewState is a preview on the worker as it is now.

type Previews

type Previews struct {
	URL    string
	Client *http.Client
	// Session is AppTown session its requests belong to, a positive
	// number (NextSession); "" is this process's start.
	Session string
}

Previews is AppTown's side of previews on a worker: boot an incarnation there, read it, remove it, and forward visitors to it. Every request that changes a preview carries its AppTown session and a number within it, an ordered token; the worker refuses one older than the last it acted on for that preview, or than the latest claim (removing every preview), so a request delayed past a later one never undoes it.

func (Previews) Admit

func (p Previews) Admit(ctx context.Context, c *appcontract.Contract) error

Admit asks the worker whether its daemon can give each part and need of c the resources it declares; it does nothing there. A refusal is the worker's sentence.

func (Previews) Boot

func (p Previews) Boot(ctx context.Context, spec PreviewSpec, prepared docker.Prepared, tree string) (PreviewState, error)

Boot sends a preview's spec, prepared value and tree, and answers with what the worker has running, or the boot's own failure.

func (Previews) Forward

func (p Previews) Forward(name, publicScheme string) *httputil.ReverseProxy

Forward is a proxy of visitors' requests to the named preview on the worker: the visitor's Host and request target (path and query, escaped as sent) go through unchanged, in headers the worker's router never reads as its own path; the forwarding headers are AppTown's own (publicScheme, the visitor's address); and upgrades such as WebSockets pass through. publicScheme "" is the naked door a probe uses: no forwarding headers at all, as a request to the app's own port has none.

func (Previews) List

func (p Previews) List(ctx context.Context) ([]string, error)

List is the name of every preview on the worker.

func (Previews) Remove

func (p Previews) Remove(ctx context.Context, name string) error

Remove removes one preview's whole stack on the worker; "" removes every preview there.

func (Previews) RemoveImages

func (p Previews) RemoveImages(ctx context.Context, name string) error

RemoveImages asks the worker to remove the build images a stopped change's preview kept.

func (Previews) State

func (p Previews) State(ctx context.Context, name string) (PreviewState, error)

State is the preview as the worker has it now.

type Reach

type Reach struct {
	URL       string `json:"url"`
	Reachable bool   `json:"reachable"`
	Error     string `json:"error,omitempty"`
}

Reach is one address's answer: any HTTP response counts as reachable (a registry may answer 401 or 404 at its root); no response does not.

type ReachRequest

type ReachRequest struct {
	URLs []string `json:"urls"`
}

ReachRequest names the addresses to try.

type RecordFile

type RecordFile struct {
	Name     string    `json:"name"`
	Size     int64     `json:"size"`
	Modified time.Time `json:"modified"`
}

RecordFile is one of the chat's record files, its size now and when it last changed: AppTown keeps the time on its copy, so the record's quiet (a turn handed back) reads the same there.

type RemoteAgent

type RemoteAgent struct {
	Sessions SessionClient
	// Image is the agent image when a launch names none.
	Image string
	// Root is the directory holding AppTown's session directories
	// (<root>/<id>), where each chat's record is copied back to.
	Root string
	// Source writes a bundle of the change's branch for a session's work
	// tree to dest and names its branch and commit. It is asked once, before
	// the chat first starts; nil sends none.
	Source func(ctx context.Context, in console.Launch, dest string) (branch, commit string, err error)
	// CopyEvery is how often a started chat's record is copied back; zero
	// means two seconds.
	CopyEvery time.Duration
	// Ending, when set, makes the last save of an ending chat, after its
	// runtime has stopped and before it is removed (SaveLast answers that
	// save's bundle). An error keeps the chat: its end is asked again.
	Ending func(ctx context.Context, id string) error
	// HandedBack, when set, is called after a copy in which AppTown's
	// copy of the record shows the agent has handed back a turn it had not
	// shown before: the moment to save. It is a trigger, never proof that
	// the chat has stopped working.
	HandedBack func(id string)
	// Now is AppTown's clock, which stamps a good copy's time
	// (RecordAsOf). Nil is the real clock (clock.Or).
	Now func() time.Time
	// contains filtered or unexported fields
}

RemoteAgent is a chat runtime on a worker, for AppTown's chat service: the console.Agent the service calls, answered by a worker's sessions (design/operations/workers.md, "Chats on the worker"). The chat service is unchanged; only where the agent runs moves.

What AppTown would mount into a chat's container travels as blobs, each sent once and named by its content, so AppTown's `apptown` binary and the project's generated registry configuration reach the chat without any path on either machine. The chat's other files (settings, prompt, pictures) are sent when they change. The chat's record is copied back to the session's directory on AppTown, with the time of the last good copy (RecordAsOf), so the conversation reads as it does for a chat on AppTown. Whether a rebuild is safe is the worker's answer, never the copy's.

func (*RemoteAgent) Attach

func (a *RemoteAgent) Attach(ctx context.Context, id string, size console.Size) (console.Terminal, error)

func (*RemoteAgent) Close

func (a *RemoteAgent) Close()

Close stops copying records, and waits for any copy (and the save it may have started) to finish. Sessions on the worker are untouched.

func (*RemoteAgent) Deliver

func (a *RemoteAgent) Deliver(ctx context.Context, in console.Launch, text string) error

func (*RemoteAgent) End

func (a *RemoteAgent) End(ctx context.Context, id string) error

End ends a chat on the worker, in this order:

  1. its runtime stops (its turn, its terminal, anything it started, and its private Docker daemon), so nothing writes its work tree;
  2. its last save is made (Ending);
  3. its record is copied one last time;
  4. the session is removed for good.

It succeeds only once the worker shows nothing of the chat; a worker that cannot be reached, or a last save not confirmed, leaves the chat unended, and its end is asked again.

func (*RemoteAgent) Interrupt

func (a *RemoteAgent) Interrupt(ctx context.Context, id string) error

func (*RemoteAgent) Reap

func (a *RemoteAgent) Reap(ctx context.Context, alive map[string]bool) error

Reap resumes copying (and so saving after each turn) for AppTown's live chats, and ends sessions the worker holds that AppTown has no live chat for. A chat outlives an AppTown restart, and its turns go on being saved without anyone opening it. Copying does not wait on the worker's inventory: each copy retries until the worker answers, and stops once the worker says it has no such session.

func (*RemoteAgent) Remove

func (a *RemoteAgent) Remove(ctx context.Context, id string) error

Remove rebuilds: the worker takes the runtime away and keeps the session.

func (*RemoteAgent) Save

func (a *RemoteAgent) Save(ctx context.Context, id, base, dest string) (string, error)

Save writes a bundle of the chat's commits since base to dest, made with the chat's own tools on the worker, and answers the commit it holds; "" when there is nothing new. The bundle is the chat's and untrusted: import it with gitexec.Quarantine, naming SaveRef.

func (*RemoteAgent) SaveLast

func (a *RemoteAgent) SaveLast(ctx context.Context, id, base, dest string) (string, error)

SaveLast is Save for an ending chat, once it has stopped: what it left uncommitted is committed first, as its leftovers.

func (*RemoteAgent) Stale

func (a *RemoteAgent) Stale(ctx context.Context, in console.Launch) (bool, error)

Stale sends the launch as the session's spec and answers the worker's view: stale, and safe to rebuild by the worker's own record.

func (*RemoteAgent) Start

func (a *RemoteAgent) Start(ctx context.Context, in console.Launch) error

Start sends the chat's spec, then its files if they changed, and starts it on the worker; the chat's record is copied back from then on.

func (*RemoteAgent) Status

func (a *RemoteAgent) Status(ctx context.Context, id string) (console.Status, error)

Status is the worker's answer. A session the worker does not know, or has removed, has no runtime; an unanswered question is an error, which the chat service shows as unavailable.

func (*RemoteAgent) Stop

func (a *RemoteAgent) Stop(ctx context.Context, id string) error

type Server

type Server struct {
	// Installation is the only client identity the worker accepts: the
	// Common Name of the client certificate.
	Installation string
	// State holds records and workspaces; nothing else on the machine is
	// touched.
	State string
	// Tasks runs the tasks: the worker's own Docker daemon in production.
	Tasks execution.TaskDriver
	// Sessions runs chats (sessions.go): AppTown's own chat runtime on the
	// worker's Docker. Nil for a worker that runs none.
	Sessions console.Agent
	// Images answers whether a session's image is on the worker's Docker;
	// a session whose image is absent is refused with the reason. Nil
	// checks nothing.
	Images interface {
		Has(ctx context.Context, image string) (bool, error)
	}
	// Checks runs check containers (checks.go) on the worker's Docker. Nil
	// for a worker that runs none.
	Checks interface {
		Run(ctx context.Context, backend string, req execution.Request) execution.Outcome
	}
	// AgentImage is the agent image this worker built (agent_image.go),
	// reported with its labels, read through ImageLabels, at
	// GET /v1/agent-image. "" reports none.
	AgentImage  string
	ImageLabels interface {
		Labels(ctx context.Context, image string) (map[string]string, error)
	}

	// DindImage is the image of a task's private Docker daemon
	// (private.go); "" refuses a task that asks for one.
	DindImage string
	// Docker is the Docker private Docker runs on and the profile is
	// measured from (docker.CLI in production); nil is the docker CLI.
	Docker docker.Client
	// Profile is what the worker measured when it started
	// (MeasureProfile), reported at GET /v1/profile.
	Profile execution.WorkerProfile

	// ServiceLimits are each private service's limits (private.go); zero is
	// execution.DefaultServiceLimits. A check's services get Checks' own.
	ServiceLimits execution.Limits
	// StateDisk reads the filesystem holding a folder: the state folder's,
	// for admitting new work. Nil reads it (diskspace.Statfs).
	StateDisk func(path string) (diskspace.Space, error)
	// ToolCache is the folder mounted writable at execution.ToolCacheMount
	// in every task's and check's container (tool_cache.go); "" mounts none.
	ToolCache string
	// Previews is the Docker previews boot with (docker.CLI in production);
	// nil runs none.
	Previews docker.Client
	// Now is the worker's clock: what it stamps and judges expiries,
	// deadlines and leases by. Nil is the real clock (clock.Or).
	Now func() time.Time
	// contains filtered or unexported fields
}

Server serves one installation's tasks.

func (*Server) CheckOwnerLabel

func (s *Server) CheckOwnerLabel() string

CheckOwnerLabel names this worker on the check containers it starts: its installation and its state folder, which no other worker shares. The sweep removes only containers so labelled.

func (*Server) Handler

func (s *Server) Handler() http.Handler

Handler routes the worker's API. The caller wraps it in a TLS server that requires a client certificate; Handler checks the certificate names this worker's installation.

func (*Server) MeasureProfile

func (s *Server) MeasureProfile(ctx context.Context, exclusive bool) execution.WorkerProfile

MeasureProfile measures the worker's profile, with what the operator declares (exclusive). The private daemon's image is pulled when it is not on the worker, so its id, repository digest and daemon version are known. Whether a privileged container runs is probed by running one. What cannot be measured is "unknown", with the reason.

func (*Server) ReapSessions

func (s *Server) ReapSessions(ctx context.Context) error

ReapSessions is run when the worker starts: it finishes removing sessions whose removal did not complete, and removes private daemons left by sessions it has no live record of. A session that cannot be read is kept.

func (*Server) SweepChecks

func (s *Server) SweepChecks(ctx context.Context, dk docker.Client, all bool)

SweepChecks removes this worker's check containers and folders past their deadline, or every one when all is set (the worker's start: a check the worker was running when it stopped is discarded, never answered). A removal that fails is logged and tried again by the next sweep.

func (*Server) SweepPreviews

func (s *Server) SweepPreviews(ctx context.Context, starting bool)

SweepPreviews removes, from the previews no request holds, each whose removal failed and each whose boot never completed (a boot holds its preview's lock from start to answer, so one found without it and without the booted mark was interrupted). Starting, nothing boots yet: it also removes each preview that is not running, and the boot configurations the stop left in those it keeps.

func (*Server) SweepPrivate

func (s *Server) SweepPrivate(ctx context.Context)

SweepPrivate stops the tasks whose private Docker's admission has expired, and removes their private resources, whether or not AppTown is there. Their records stay, for AppTown to collect and release. A removal that fails is logged and tried again by the next sweep.

func (*Server) TrimBuildCache

func (s *Server) TrimBuildCache(ctx context.Context)

TrimBuildCache keeps this worker's build cache within AppTown's budget, as AppTown keeps its own (docker.TrimBuildCache): the previews and release builds it runs fill it, and nothing else trims it.

type SessionClient

type SessionClient struct {
	URL    string
	Client *http.Client
}

SessionClient is AppTown's side of a worker's sessions. Its client must present the installation's certificate.

An answer AppTown cannot get or read is execution.ErrWorkerUnavailable: what the worker holds is then unknown, never absent. Only the worker's own answer says a session is gone (SessionStatus.Gone) or unknown to it.

func (SessionClient) Deliver

func (c SessionClient) Deliver(ctx context.Context, id, text string) error

Deliver types a message into the chat. An error means it may or may not have arrived.

func (SessionClient) HasBlob

func (c SessionClient) HasBlob(ctx context.Context, sum string) (bool, error)

HasBlob: the worker has the blob.

func (SessionClient) Interrupt

func (c SessionClient) Interrupt(ctx context.Context, id string) error

Interrupt stops the chat's current turn.

func (SessionClient) List

List lists the worker's sessions, removed ones included.

func (SessionClient) Put

Put records the session on the worker, or replaces its spec.

func (SessionClient) PutBlob

func (c SessionClient) PutBlob(ctx context.Context, sum string, tar io.Reader) error

PutBlob sends a blob: a tar with one entry named "content", named by its SHA-256.

func (SessionClient) PutFiles

func (c SessionClient) PutFiles(ctx context.Context, id string, tar io.Reader) (SessionStatus, error)

PutFiles replaces the session's files with the tar.

func (SessionClient) PutSource

func (c SessionClient) PutSource(ctx context.Context, id, branch, commit string, bundle io.Reader) (SessionStatus, error)

PutSource sends the bundle the chat's work tree is made from.

func (SessionClient) Reach

func (c SessionClient) Reach(ctx context.Context, urls []string) ([]Reach, error)

Reach asks the worker which of the addresses it can reach.

func (SessionClient) Rebuild

func (c SessionClient) Rebuild(ctx context.Context, id string) (SessionStatus, error)

Rebuild removes the session's runtime; the next Start creates it anew from the session's spec. The session stays.

func (SessionClient) Record

func (c SessionClient) Record(ctx context.Context, id, name string, from int64) ([]byte, error)

Record reads a record file from an offset.

func (SessionClient) Records

func (c SessionClient) Records(ctx context.Context, id string) ([]RecordFile, error)

Records lists the chat's record files.

func (SessionClient) Remove

func (c SessionClient) Remove(ctx context.Context, id string) (SessionStatus, error)

Remove ends the session for good. It succeeds only once the worker shows the session's runtime is gone; the worker never starts it again.

func (SessionClient) Save

func (c SessionClient) Save(ctx context.Context, id, base, leftovers, dest string) (string, error)

Save asks for a bundle of the chat's commits since base, writes it to dest, and answers the commit it holds; "" when there is nothing new. Only 204 means nothing new: a reply without its commit is not an answer. A non-empty leftovers commits what is uncommitted under that message first.

func (SessionClient) Start

func (c SessionClient) Start(ctx context.Context, id string) (SessionStatus, error)

Start makes the session's container and private daemon run.

func (SessionClient) Status

func (c SessionClient) Status(ctx context.Context, id string) (SessionStatus, error)

Status reports the session as the worker sees it now.

func (SessionClient) Stop

Stop stops the session's runtime; the session stays.

func (SessionClient) Terminal

func (c SessionClient) Terminal(ctx context.Context, id string, size console.Size) (console.Terminal, error)

Terminal opens the chat's terminal on the worker.

type SessionMount

type SessionMount struct {
	Blob   string `json:"blob"`
	Target string `json:"target"`
}

SessionMount mounts a blob's content, read-only, at Target in the chat's container.

type SessionSpec

type SessionSpec struct {
	Image        string `json:"image"`
	Model        string `json:"model,omitempty"`
	AgentSession string `json:"agentSession"`
	Docker       bool   `json:"docker,omitempty"`
	// Mirror is the Docker Hub mirror the chat's private daemon and its
	// builder pull through ("" none); its resolvers are the worker's own.
	Mirror string         `json:"mirror,omitempty"`
	Env    []string       `json:"env,omitempty"`
	Mounts []SessionMount `json:"mounts,omitempty"`
}

SessionSpec is what AppTown asks a session to run: the chat's launch without host paths. The worker supplies the paths: a mount names content already sent to the worker (a blob), never a path on either machine.

type SessionStatus

type SessionStatus struct {
	ID      string `json:"id"`
	Known   bool   `json:"known"`
	Removed bool   `json:"removed,omitempty"`
	// Agent is the chat's container; Docker its private daemon.
	Agent struct {
		Exists      bool `json:"exists"`
		Running     bool `json:"running"`
		EverStarted bool `json:"everStarted"`
		// Attached: the worker holds the chat's terminal.
		Attached bool `json:"attached"`
	} `json:"agent"`
	Docker struct {
		Exists  bool `json:"exists"`
		Running bool `json:"running"`
	} `json:"docker"`
	// Script: a save's container exists on the chat's work tree.
	Script struct {
		Exists bool `json:"exists"`
	} `json:"script"`
	// Stale: the container no longer matches the session's spec and is
	// stopped, so a rebuild may make a new one. A running chat is never
	// stale here: it is rebuilt only once it has stopped.
	Stale bool `json:"stale,omitempty"`
	// FilesDigest identifies the chat's files as last sent.
	FilesDigest string `json:"filesDigest,omitempty"`
	// Source is the commit the chat's work tree was made from; empty until
	// AppTown sends it.
	Source string `json:"source,omitempty"`
}

SessionStatus is the worker's answer about a session. Known is false when the worker has no record of it. The runtime's state is read from Docker at the time of asking; a worker that cannot read it answers with an error, never with absence.

func (SessionStatus) Gone

func (st SessionStatus) Gone() bool

Gone: nothing of the session runs or remains on the worker's Docker.

func (SessionStatus) Stopped

func (st SessionStatus) Stopped() bool

Stopped: nothing of the chat runs: not its container, its private daemon, or a save's container.

type SessionSummary

type SessionSummary struct {
	ID      string    `json:"id"`
	Created time.Time `json:"created"`
	Removed bool      `json:"removed,omitempty"`
}

SessionSummary is one line of the worker's list of sessions.

type TaskRequest

type TaskRequest struct {
	Key       string              `json:"key"`
	Admission execution.Admission `json:"admission"`
	Image     string              `json:"image"`
	Argv      []string            `json:"argv"`
	Env       []string            `json:"env"`
	Workspace string              `json:"workspace,omitempty"`
	// Source is the commit the workspace was uploaded with.
	Source string `json:"source,omitempty"`
	// Request is the client's identity for what the task runs. The worker
	// holds a key to it; without one, to the request as sent.
	Request  string   `json:"request,omitempty"`
	ReadOnly bool     `json:"readOnly,omitempty"`
	Labels   []string `json:"labels,omitempty"`
	// Files are generated files the task sees read-only (execution.File),
	// kept in the task's own folder until it is released.
	Files []execution.File `json:"files,omitempty"`
	// Inputs name blobs sent to this worker, presented read-only under
	// /apptown/in; Results are files captured when the task exits (task_io.go).
	Inputs  []execution.Input  `json:"inputs,omitempty"`
	Results []execution.Result `json:"results,omitempty"`
	// Images are OCI image layouts a release build leaves in its
	// workspace's state, captured when it exits (task_images.go).
	Images []execution.ImageResult `json:"images,omitempty"`
	// Private is the task's own network, services and daemon (private.go).
	Private *execution.PrivateDocker `json:"private,omitempty"`
}

TaskRequest is the wire form of an ensured task: the contract's spec without host paths. Workspace names a workspace uploaded to this worker.

type VersionReport

type VersionReport struct {
	Version     string `json:"version"`
	ProtocolMin int    `json:"protocolMin"`
	ProtocolMax int    `json:"protocolMax"`
}

VersionReport is GET /v1/version.

Directories

Path Synopsis
Package workertest starts a apptown-worker in the test process, over real mutual TLS, for tests of AppTown's worker path.
Package workertest starts a apptown-worker in the test process, over real mutual TLS, for tests of AppTown's worker path.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL