Documentation
¶
Overview ¶
Package execution runs bounded processes. It knows nothing about projects, workflow states, approvals or persistence; callers interpret its results.
Index ¶
- Constants
- Variables
- func DockerConfigDir() (string, error)
- func LimitFlags(l Limits) []string
- func LogFlags() []string
- func ReadReport(workspace, path string) ([]byte, error)
- func StageFiles(dir string, files []File) ([]string, error)
- func StepContainer(stepID ids.ID, what string) string
- func StepContainerPrefix(stepID ids.ID) string
- func UniqueName(prefix string) string
- func ValidFiles(files []File) error
- func ValidImage(ref string) bool
- func ValidImages(images []ImageResult) error
- func ValidInputs(inputs []Input) error
- func ValidResults(results []Result) error
- func WorkerRefusal(resp *http.Response) error
- func WriteRefusal(w http.ResponseWriter, status int, err error)
- type Admission
- type AgentGit
- type AgentImage
- type Build
- type Capabilities
- type DockerCLI
- type DockerSelection
- type DockerTasks
- func (d DockerTasks) Capabilities() Capabilities
- func (d DockerTasks) DataDisk(ctx context.Context) (root string, space diskspace.Space, err error)
- func (d DockerTasks) Ensure(ctx context.Context, spec TaskSpec) (TaskObservation, error)
- func (d DockerTasks) Observe(ctx context.Context, ref TaskRef) (TaskObservation, error)
- func (d DockerTasks) Output(ctx context.Context, ref TaskRef, read OutputRead) (OutputPage, error)
- func (d DockerTasks) Release(ctx context.Context, ref TaskRef) error
- func (d DockerTasks) ReleaseWorkspace(context.Context, string) error
- func (d DockerTasks) Start(ctx context.Context, ref TaskRef) error
- func (d DockerTasks) Stop(ctx context.Context, ref TaskRef) error
- func (d DockerTasks) Wait(ctx context.Context, ref TaskRef) (int, error)
- type File
- type FileIdentity
- type History
- type ImageAnswer
- type ImageCapture
- type ImageFiles
- type ImageLayout
- type ImageRequest
- type ImageResult
- type Input
- type Limits
- type NetworkPolicy
- type Outcome
- type OutputPage
- type OutputRead
- type PrivateDocker
- type Refusal
- type RefusalCode
- type RegistryLogin
- type Request
- type Result
- type ResultFile
- type ResultState
- type Runner
- type TaskDriver
- type TaskObservation
- type TaskRef
- type TaskSource
- type TaskSpec
- type TaskState
- type Termination
- type WorkerAvailability
- type WorkerClient
- func (c WorkerClient) Call(ctx context.Context, method, path string, in, out any, limit int64, ...) error
- func (c WorkerClient) Do(ctx context.Context, method, path string, body io.Reader, header http.Header) (*http.Response, error)
- func (c WorkerClient) EnsureBlob(ctx context.Context, sum string, tar []byte) error
- func (c WorkerClient) Send(ctx context.Context, method, path string, body io.Reader, contentType string, ...) error
- type WorkerProfile
- type WorkerTaskRequest
- type WorkerTasks
- func (w WorkerTasks) AgentImage(ctx context.Context) (AgentImage, error)
- func (w WorkerTasks) Availability(ctx context.Context) (WorkerAvailability, error)
- func (w WorkerTasks) Capabilities() Capabilities
- func (w WorkerTasks) Collect(ctx context.Context, key, base, message, dest string) (string, error)
- func (w WorkerTasks) Disks(ctx context.Context) ([]diskspace.Reading, error)
- func (w WorkerTasks) Ensure(ctx context.Context, spec TaskSpec) (TaskObservation, error)
- func (w WorkerTasks) ImageFile(ctx context.Context, ref TaskRef, image, file string) (io.ReadCloser, int64, error)
- func (w WorkerTasks) ImageResults(ctx context.Context, ref TaskRef) (map[string]ImageCapture, error)
- func (w WorkerTasks) Observe(ctx context.Context, ref TaskRef) (TaskObservation, error)
- func (w WorkerTasks) Output(ctx context.Context, ref TaskRef, read OutputRead) (OutputPage, error)
- func (w WorkerTasks) Profile(ctx context.Context) (WorkerProfile, error)
- func (w WorkerTasks) Release(ctx context.Context, ref TaskRef) error
- func (w WorkerTasks) ReleaseWorkspace(ctx context.Context, key string) error
- func (w WorkerTasks) Results(ctx context.Context, ref TaskRef) (map[string]ResultFile, error)
- func (w WorkerTasks) Start(ctx context.Context, ref TaskRef) error
- func (w WorkerTasks) Stop(ctx context.Context, ref TaskRef) error
- func (w WorkerTasks) Wait(ctx context.Context, ref TaskRef) (int, error)
Constants ¶
const ( MaxFiles = 32 MaxFileBytes = 1 << 20 // all of a request's files together )
Bounds on what a request's files may be: they are small generated configuration, never a way to send a tree.
const ( MaxInputs = 256 MaxInputBytes = 64 << 20 // one input MaxInputTotalBytes = 512 << 20 // all of a request's inputs )
Bounds on a request's inputs: enough for an agent's steering, resources, attachments and AppTown CLI, never a way to send a tree.
const ( MaxResults = 16 MaxResultBytes = 64 << 20 // one result's Max MaxResultTotalBytes = 96 << 20 // all of a request's Max together )
Bounds on a request's result files.
const ( RefusedSymlink = "symlink" // a symbolic link at the path or on the way to it RefusedNotRegular = "not-regular" // a folder, device, socket or pipe RefusedEscape = "escape" // the path leaves the tree or state folder )
Reasons a result file is refused.
const ( Local = "local" Docker = "docker" )
const CheckLabel = dockerlabel.Class + "=check"
CheckLabel marks what checks leave on a machine, so its sweep finds it.
const InputsRoot = "/apptown/in"
InputsRoot is where a task's inputs are presented.
const MaxReport = 64 << 20
MaxReport bounds a report carried back: a larger one is not read, and is an error, never a truncated answer. The bound keeps a runaway tool from filling AppTown's memory; it is set above what a supported scanner writes for an ordinary image. Trivy's SARIF over a full Debian image (rust:1.84.1, every severity) is 27 MB, mostly rule descriptions, and it cannot be cut down by severity without hiding findings the gate judges: trivy filters on the vendor's label, AppTown on the CVSS score.
const ProfileWorkerVM = "worker-vm"
ProfileWorkerVM is the first worker profile for private Docker: a machine the installation runs its worker on, whose Docker runs privileged daemons, one per invocation.
const RefusalContentType = "application/vnd.flokkit.worker-refusal+json"
RefusalContentType marks a body as a Refusal. The client classifies only a body that carries it.
const RefusedNotFolder = "not-folder"
RefusedNotFolder: an image's path holds something other than a folder.
const StateRoot = "/apptown/state"
StateRoot is where a workspace's own writable state is presented: beside the tree, never in it, and kept across the tasks that use the workspace.
const ToolCacheMount = "/apptown-cache/mise-cache"
ToolCacheMount is where a container finds the shared download cache of the toolchains projects declare (mise's MISE_CACHE_DIR): AppTown's apptown-cache volume on its own host, a worker's tool cache on a worker. It holds what mise fetches (release listings, tool archives), never a tool's installation, which stays the container's own.
Variables ¶
var ( // ErrTaskConflict: a task already exists under the key for a different // request. Nothing was created, and the existing task was not touched. ErrTaskConflict = errors.New("a different task already exists under this key") // ErrUnsupported: the request needs something this driver does not // provide. Nothing was created. ErrUnsupported = errors.New("the task needs something this backend does not provide") // ErrExpired: the admission's expiry has passed. Nothing was created. ErrExpired = errors.New("the task's admission has expired") // ErrStartRefused: the backend refused to start the task, and says why // (its private Docker could not be made). Nothing of it runs. ErrStartRefused = errors.New("the backend refused to start the task") // ErrWorkspaceInUse: a shared workspace still has a task not released. // Nothing was removed. ErrWorkspaceInUse = errors.New("a task in this workspace is not released") )
var DefaultChatLimits = Limits{CPUs: 2, MemoryBytes: 4 << 30, PIDs: 4096}
DefaultChatLimits bound a chat's container on AppTown's own host, as a worker's -chat-* flags do on a worker.
var DefaultServiceLimits = Limits{CPUs: 1, MemoryBytes: 1 << 30, PIDs: 256}
DefaultServiceLimits are each service's CPU, memory (swap included) and process limits when its runner names none: a project's test database shares the machine with every other run on it, and must not starve them.
var DefaultTaskLimits = Limits{CPUs: 4, MemoryBytes: 8 << 30, PIDs: 4096}
DefaultTaskLimits bound a step's, check's or task's container when the operator names none (apptownd -task-cpus, -task-memory-mib, -task-pids): room for an ordinary build, never the whole host.
var ErrImageGone = errors.New("the task was released: its image files are gone")
ErrImageGone: the task was released, and the files of its images went with it. A read already open may finish; a new one is refused.
var ErrMissingBlob = errors.New("the worker lacks the blob")
ErrMissingBlob: the worker lacks a blob a request names (a worker restart removes blobs no chat uses). AppTown sends it again, once.
var ErrNoAgentImage = errors.New("the worker has no agent image")
ErrNoAgentImage: the worker has no agent image to report.
var ErrTaskUnobservable = errors.New("the task could not be observed")
ErrTaskUnobservable means the backend could not be asked. It is never absence.
ErrWorkerUnavailable: the worker could not be asked, or its answer could not be read. What it did is unknown; it is never absence or refusal.
var ErrWorkspaceGone = errors.New("the worker has no such workspace")
ErrWorkspaceGone: the worker answered that the workspace does not exist. What was in it is lost; it is never "no changes".
var ImageFilePattern = regexp.MustCompile(`^(index\.json|blobs/sha256/[a-f0-9]{64})$`)
ImageFilePattern is the files of a captured image AppTown reads: its index and its blobs. Nothing else in the layout is served.
Functions ¶
func DockerConfigDir ¶
DockerConfigDir is the docker CLI's configuration folder, as the CLI finds it: DOCKER_CONFIG, else .docker in the home folder. A home folder that cannot be named is an error, never a relative path.
func LimitFlags ¶
LimitFlags are a container's docker flags for l: --cpus, --memory with equal swap, --pids-limit; none for a zero field.
func LogFlags ¶
func LogFlags() []string
LogFlags bound a long-lived container's log on its host's disk: Docker's json-file driver, rotated at 10 MiB with three files kept, so at most about 30 MiB a container. Without them a container keeps its whole output for as long as it exists, which for a standing deployment is until the disk is full. json-file is named, not left to the daemon's default, because docker logs reads it and a daemon whose default driver takes other options would refuse these. Rotation discards a container's oldest output, so a task's own container, whose whole output is read by byte cursor (DockerTasks.Output), does not get them: it lives only until it is released, and new tasks are refused on a disk short of space.
func ReadReport ¶
ReadReport reads the report at path under workspace: a regular file inside it, at most MaxReport bytes. A symbolic link, or a path that leaves the workspace, is refused rather than followed.
func StageFiles ¶
StageFiles writes files into dir, a folder the caller owns and that does not exist yet, and answers the Docker mounts presenting each read-only at its path. A dir that already exists is used only if it holds exactly these files (a task ensured again): files a container may already have mounted are never rewritten, and other files there are ErrTaskConflict.
func StepContainer ¶
StepContainer names a container a workflow step runs: apptown-step-<id>- <what> (agent, script, review, scan, tests…), before UniqueName's suffix where the step runs more than one. Every container a step starts is named here, so StepContainerPrefix finds them all when the step is interrupted.
func StepContainerPrefix ¶
StepContainerPrefix begins the name of every container step stepID runs.
func UniqueName ¶
UniqueName is prefix with a random suffix: a container name no other run shares, even one started in the same instant (a fleet check starts one per project at once). A suffix from the clock repeats; this does not.
func ValidFiles ¶
ValidFiles says whether files may be presented to a container: each at an absolute, clean path of its own, outside the tree (/work) and history (/history), with permission bits only, within the bounds.
func ValidImage ¶
ValidImage reports whether ref may be given to Docker as an image.
func ValidImages ¶
func ValidImages(images []ImageResult) error
ValidImages says whether images may be declared: each named once, at its own clean path under /apptown/state/, with a positive admitted size.
func ValidInputs ¶
ValidInputs says whether inputs may be presented to a task: each a blob digest, at a clean path of its own under /apptown/in/ that is no other input's folder, with mode 0644 or 0755, within the bounds. Sizes are checked here for inputs carrying their content, and by the backend, which knows every blob's size, for all.
func ValidResults ¶
ValidResults says whether results may be declared: each named once, at a clean path of its own in the tree or the state folder, with a maximum size within the bounds.
func WorkerRefusal ¶
WorkerRefusal reads a worker's answer of 300 or over. A 5xx, or a body that cannot be read, is the worker unavailable whatever it says; a classified refusal, read whole, is its class; anything else, a body longer than the client reads among it, is a plain refusal, whatever its status.
func WriteRefusal ¶
func WriteRefusal(w http.ResponseWriter, status int, err error)
WriteRefusal answers a refusal: err's class as its code (none when err is of no class), err's text as its detail, with status.
Types ¶
type Admission ¶
type Admission struct {
Profile string
Limits Limits
Network NetworkPolicy
// Expires is when the task must no longer run. The caller stops it then;
// a driver refuses to create a task already expired.
Expires time.Time
}
Admission is carried by every request, whatever runs it: the profile that admitted it, its resource limits, its network policy and its expiry. A driver enforces what it can and refuses what it cannot.
type AgentGit ¶
type AgentGit struct {
Branch string
Base string `json:",omitempty"`
BaseCommit string `json:",omitempty"`
}
AgentGit is the repository a task is given at /work/.git: an ordinary clone of its source commit's history, on Branch, with Base, when set, at BaseCommit (an ancestor of the source), no remote, and AppTown as its committer. It is the task's own: the backend's collection reads the work tree with a Git directory of its own and never this one, so the task's commits reach AppTown only as the tree they leave, in one commit.
type AgentImage ¶
AgentImage is a worker's agent image: its content-derived name and the labels its build gave it (the subagents and extension versions a step's preparation reads).
type Build ¶
Build runs one image build on the Docker a DockerCLI runs: the build's own arguments, its context last. It is the one build rule's (dockerbuild.Daemon.Build): the daemon's own builder named, the extra certificates as the build's secret, AppTown's label.
type Capabilities ¶
type Capabilities struct {
PrivateDocker bool // the task can run its own containers
Terminal bool // an interactive terminal can attach
Persistence bool // the task's files survive it for capture
Inputs bool // TaskSpec.Inputs are presented under /apptown/in (inputs.go)
Results bool // TaskSpec.Results are captured when the task exits
Images bool // TaskSpec.Images are captured when the task exits, and served by file
}
Capabilities are what a backend claims beyond the lifecycle every driver provides. Admission refuses work that needs a capability not claimed; conformance tests run the capability groups a driver claims.
func (Capabilities) Unclaimed ¶
func (c Capabilities) Unclaimed(s TaskSpec) error
Unclaimed refuses a request that needs a capability c does not claim, with ErrUnsupported. Nothing is created.
type DockerCLI ¶
DockerCLI runs one docker command (the arguments after the binary) on a machine's Docker, with stdin when given, and answers with its output.
type DockerSelection ¶
type DockerSelection struct {
// Host is the daemon's address, as the docker CLI takes it.
Host string
// Context is the context Host was read from, empty when DOCKER_HOST
// named it.
Context string
// Refused is why no address could be pinned: the context could not be
// read, or it needs TLS files a bare address does not carry. A driver
// with a refused selection creates nothing and observes nothing until
// AppTown restarts with a selection that resolves.
Refused error
// TLSFromEnvironment is whether the CLI would take TLS settings from
// DOCKER_TLS_VERIFY, DOCKER_TLS and DOCKER_CERT_PATH for this daemon: it
// does for DOCKER_HOST and the default context, not for a named context,
// which carries its own. A call passes them on only then.
TLSFromEnvironment bool
}
DockerSelection is the Docker daemon an AppTown's tasks run on, resolved once at startup to the daemon's address. Every call a DockerTasks makes passes that address itself (docker --host), so no later change to a context, the saved current context or the environment can send a call to another daemon: a context is a name the CLI would otherwise look up again on each call.
func SelectDocker ¶
func SelectDocker() DockerSelection
SelectDocker resolves the daemon the docker CLI would use now: DOCKER_HOST; else DOCKER_CONTEXT; else the current context saved in the CLI's configuration (DockerConfigDir); else the default context. It only reads. A configuration that is there but cannot be read refuses the selection: what it names is not known, and the default daemon is not assumed.
func (DockerSelection) Backend ¶
func (s DockerSelection) Backend() string
Backend names the selection in task records: the daemon's address, so the same daemon reached through DOCKER_HOST or through a context is one backend. A refused selection names its context and says it is unresolved; it never matches a resolved one.
func (DockerSelection) OtherMachine ¶
func (s DockerSelection) OtherMachine() string
OtherMachine is the address of the machine the daemon runs on when that is not this one: the host of a tcp:// or ssh:// address, from DOCKER_HOST or a Docker context alike. "" for a local socket (and for a refused selection, which names no daemon). It is the one answer to "is Docker on this machine", and where published ports answer when it is not.
type DockerTasks ¶
type DockerTasks struct {
Binary string
// Now is the clock a task's admission expiry is judged on: the one that
// wrote it. Nil is the real clock.
Now func() time.Time
// Daemon pins the Docker daemon every call uses (SelectDocker). Its
// address is passed on each call, so nothing the CLI would look up again
// can move a call to another daemon. A zero Daemon follows the CLI's own
// selection (the worker's own daemon, tests).
Daemon DockerSelection
// Namespace keeps one installation's containers apart from another's on
// a shared Docker daemon: it is part of every container name.
Namespace string
// Files is the folder tasks' generated files are kept in, one folder per
// task until it is released. "" refuses a task with generated files.
Files string
// ToolVersions names the df image a disk read runs in, read once at
// each admission or read. nil: the disk cannot be read, so new tasks
// are refused.
ToolVersions toolversions.Source
// CallerAdmitsDisk says the caller reads this daemon's data disk before
// its own first effect of a task (a worker, before the task's workspace
// or record), so Ensure does not read it again. Otherwise Ensure admits
// it before anything of the task is written (admitDisk).
CallerAdmitsDisk bool
}
DockerTasks is the local Docker implementation of TaskDriver: a task is a container created under a name derived from its key, started separately, observed by container ID, and removed only when released. A caller that restarts finds it again by key or ID instead of starting another.
A container carries its request's digest as a label; one found under the key for a different request is refused, never adopted. Start starts only a container that has never run, so a repeated start cannot run a task again.
Admission limits map to --cpus, --memory and --pids-limit, the network policy to --network (none, or Docker's default bridge), and the expiry to a label plus a refusal to create an expired task. The wall-clock limit is the caller's to enforce while waiting.
func (DockerTasks) Capabilities ¶
func (d DockerTasks) Capabilities() Capabilities
func (DockerTasks) DataDisk ¶
DataDisk reads the filesystem holding the daemon's data folder (its root, as docker info names it), through a throwaway container (diskspace.ReadThroughDocker).
func (DockerTasks) Ensure ¶
func (d DockerTasks) Ensure(ctx context.Context, spec TaskSpec) (TaskObservation, error)
Ensure creates the task's container without starting it. If one already exists under the key's name (a retried create after a lost reply), it is returned instead: one key, one container.
func (DockerTasks) Observe ¶
func (d DockerTasks) Observe(ctx context.Context, ref TaskRef) (TaskObservation, error)
Observe reports a task by ID or key. Absence is reported only when Docker answered that there is no such container.
func (DockerTasks) Output ¶
func (d DockerTasks) Output(ctx context.Context, ref TaskRef, read OutputRead) (OutputPage, error)
Output reads the task's combined output from a byte cursor. Docker keeps the whole log until the container is removed, so nothing is truncated.
func (DockerTasks) ReleaseWorkspace ¶
func (d DockerTasks) ReleaseWorkspace(context.Context, string) error
filesPath is the folder a task's generated files are kept in. ReleaseWorkspace: local tasks mount AppTown's own folder; they share no workspace on the backend.
func (DockerTasks) Start ¶
func (d DockerTasks) Start(ctx context.Context, ref TaskRef) error
Start starts a task that has never run. Starting one that runs or has run does nothing: Docker would run an exited container's command again. (A start racing another start of the same task is serialized within this process; a start issued by another process that outlived its caller is not, and callers that cannot rule that out guard the command itself.)
type File ¶
type File struct {
Path string `json:"path"` // absolute and clean, in the container
Content []byte `json:"content"`
Mode fs.FileMode `json:"mode"` // permission bits only
}
File is a file AppTown generated for a container (a registry client's configuration; design/operations/workers.md): its bytes, presented read-only at Path. It travels beside a request's tree, never in it: whoever runs the container writes it into a folder of its own and mounts it, so nothing in a project's tree can choose where it is written.
type FileIdentity ¶
type FileIdentity struct {
Path string `json:"path"`
Mode fs.FileMode `json:"mode"`
SHA256 string `json:"sha256"`
}
FileIdentity is what identifies a generated file in a request's identity: its path, its mode and a digest of its bytes.
func IdentifyFiles ¶
func IdentifyFiles(files []File) []FileIdentity
IdentifyFiles is the ordered identity of files: a request whose files change, at the same paths or not, is another request.
type History ¶
History is a repository and the commits in it a check needs, with their full histories.
type ImageAnswer ¶
type ImageAnswer struct {
ID string `json:"id,omitempty"`
Labels map[string]string `json:"labels,omitempty"`
// NotBuilt: nothing was there, and the caller did not allow a build.
NotBuilt bool `json:"notBuilt,omitempty"`
Err string `json:"err,omitempty"`
}
ImageAnswer names the image by its ID, which is what a check then runs and what its evidence records; a tag is only a way to find it.
func InspectImage ¶
InspectImage answers whether ref is on the Docker: found, not found (the daemon says there is no such image), or an error (it could not be asked, or its answer could not be read). Only "not found" is a cache miss.
func PrepareImage ¶
func PrepareImage(ctx context.Context, docker DockerCLI, build Build, req ImageRequest) (answer ImageAnswer)
PrepareImage answers an ImageRequest on the Docker that docker runs, building through build. A failure is Err, never an empty answer that reads as "no image".
type ImageCapture ¶
type ImageCapture struct {
State ResultState `json:"state"`
Reason string `json:"reason,omitempty"` // refused only
Size int64 `json:"size,omitempty"` // present and too large
}
ImageCapture is what became of a declared image when its task exited: present (a folder of regular files within Max, Size bytes), absent, refused (Reason: a symbolic link at, on the way to or inside the folder, something not a folder at the path, something not a regular file in it, or a copy that would leave the backend's disk short of space, in words) or too large (Size). None is ever turned into another; and once the task is released its files are gone (ErrImageGone), which is none of these.
func (ImageCapture) Valid ¶
func (c ImageCapture) Valid() error
Valid says whether a captured image is one of the four states, whole.
type ImageFiles ¶
type ImageFiles interface {
ImageFile(ctx context.Context, ref TaskRef, image, file string) (io.ReadCloser, int64, error)
}
ImageFiles opens a captured image's files (WorkerTasks.ImageFile).
type ImageLayout ¶
type ImageLayout struct {
Files ImageFiles
Ref TaskRef
Image string
}
ImageLayout is one captured image of a task read file by file, as the AppTown's publisher reads an OCI layout (oci.Layout): every file is opened on the backend when it is read, and checked by the reader.
func (ImageLayout) Open ¶
func (l ImageLayout) Open(ctx context.Context, name string) (io.ReadCloser, error)
type ImageRequest ¶
type ImageRequest struct {
Tag string
Pull string
Tree string // the staged tree a build's context is under
Context string // the build context, relative to Tree
AllowBuild bool
Registry *RegistryLogin
// Anonymous isolates registry authentication without changing the selected daemon.
Anonymous bool
Timeout time.Duration
// Scratch is where a login's Docker configuration is made, for the one
// request: a folder its owner sweeps, so a request interrupted mid-pull
// leaves no credentials behind for long. "" is the system's temporary
// folder.
Scratch string
}
ImageRequest asks for an application image a check runs in or scans (design/operations/workers.md, "Checks on the worker"): the image Tag names, when it is already there; else Pull pulled, with Registry's login for that one pull; else built from Context under Tree, when the caller allows a build. The machine that prepares it is the one that runs the check.
type ImageResult ¶
type ImageResult struct {
Name string `json:"name"`
// Path is under /apptown/state/, never in the tree: the source cannot
// have put an image there.
Path string `json:"path"`
// Max is the admitted size, in bytes, of all the layout's files
// together; a larger capture is ResultTooLarge.
Max int64 `json:"max"`
}
ImageResult is an image a release build declares it will leave (design/operations/workers.md, "Releases built on the worker"): an OCI image layout, as `docker buildx build --output type=oci,dest=<Path>,tar=false` writes it, in the folder at Path under the workspace's state. The backend captures it after the task and everything beside it have stopped, the way it captures result files, and keeps it until the task is released. AppTown reads it file by file (ImageFilePattern) and trusts none of it: what it publishes is what the registry reads back.
type Input ¶
type Input struct {
Blob string `json:"blob"` // SHA-256 of the blob's tar, hex
Path string `json:"path"` // absolute and clean, under /apptown/in/
Mode fs.FileMode `json:"mode"` // 0644 or 0755
// contains filtered or unexported fields
}
Input is a file a task reads, read-only, at Path under /apptown/in/ (design/operations/workers.md, "Agent and review steps on the worker"). It is prepared on AppTown as bytes and sent to the backend once as a content-addressed blob: a tar holding one regular entry named "content", named by the SHA-256 of the tar (the worker's PUT /v1/blobs/{sum}, which chats use too). A request names the blob, never a path on either machine, so its identity covers the content.
func NewInput ¶
NewInput prepares content to be presented at path with mode, as an input that carries its blob for the driver to send.
type Limits ¶
Limits bound a task. Zero means the profile imposes no limit of that kind; it is recorded as such, never as "unknown".
type NetworkPolicy ¶
type NetworkPolicy string
NetworkPolicy names what a task may reach. "none" has no network; "default" is the backend's ordinary egress, which a profile qualifies.
const ( NetworkNone NetworkPolicy = "none" NetworkDefault NetworkPolicy = "default" )
type Outcome ¶
type Outcome struct {
Output []byte
RunErr error
TimedOut bool // compatibility for existing consumers; Termination is authoritative
Termination Termination
ExitCode int // -1 when no normal exit code is available
CleanupErr error
// Report is the requested report's bytes, when it was read; ReportErr
// says why it was not (missing, not a regular file, too large, or outside
// the workspace). A caller whose tool declares a report treats a missing
// one as an error, never as a clean answer.
Report []byte
ReportErr error
}
type OutputPage ¶
type OutputRead ¶
type PrivateDocker ¶
type PrivateDocker struct {
Daemon bool `json:"daemon,omitempty"`
Services []deploy.ServiceSpec `json:"services,omitempty"`
// Mirror is the Docker Hub mirror the daemon and its builder pull
// through ("" none). Resolvers are the backend's own.
Mirror string `json:"mirror,omitempty"`
// Profile is the worker's profile the task was admitted on, as the
// worker reported it. A worker refuses the request when what it
// measures now differs (WorkerProfile.Changed).
Profile *WorkerProfile `json:"profile,omitempty"`
}
PrivateDocker is what a task has of its own beside its container, from its start to its exit (design/operations/workers.md, "Private Docker on the worker"): a network, the services on it, and, when Daemon is set, a private Docker daemon the task reaches as dind. The services are resolved when the task is admitted and travel in its request, so a request sent again runs the same ones whatever the project or the catalog says later.
func (*PrivateDocker) Valid ¶
func (p *PrivateDocker) Valid(l Limits) error
Valid refuses what no backend creates: a service without an image or a valid name, two with one name, one named dind (the daemon's name on the network), and a limit left at zero, which is never "unlimited" here.
type Refusal ¶
type Refusal struct {
Code RefusalCode `json:"code,omitempty"`
Detail string `json:"detail"`
}
Refusal is the body of a classified worker refusal. Its HTTP status is the refusal's status as before.
type RefusalCode ¶
type RefusalCode string
RefusalCode is a refusal's class on the worker protocol.
const ( RefusalExpired RefusalCode = "expired" RefusalStartRefused RefusalCode = "start-refused" RefusalUnsupported RefusalCode = "unsupported" RefusalNoAgentImage RefusalCode = "no-agent-image" RefusalMissingBlob RefusalCode = "missing-blob" RefusalConflict RefusalCode = "conflict" RefusalDisk RefusalCode = "disk" )
type RegistryLogin ¶
type RegistryLogin struct{ Host, Username, Token string }
RegistryLogin is one registry's credentials, for one request.
type Request ¶
type Request struct {
Workspace string
WorkspaceRO bool
Mounts []string
Argv []string
Env []string
Timeout time.Duration
Image string
Name string
Network string
NoEntrypoint bool
Labels []string // key=value, supplied by the owner
ExtraHosts []string
OnLine func(string)
// Report names one file the command writes under its workspace, which
// the answer carries back (Outcome.Report): a caller never reads it off
// the disk itself, so a check reads the same whichever machine ran it.
Report string
// History is a repository presented read-only at /history: the commits
// a check reads beyond its tree (a landing's history scan).
History *History
// ExportImage names an image (by ID) saved to .apptown/image.tar in the
// workspace before the command runs: an image scan reads the image
// there, on the machine that has it.
ExportImage string
// Files are generated files presented read-only in the container
// (files.go), written beside the tree by whoever runs it: never in it.
Files []File
// FilesDir is where they are written: a folder the caller owns, that
// does not exist yet. "" is a folder the runner makes under its
// Scratch, and removes after the run.
FilesDir string
// Services run beside a Docker run, on a network of its own that the
// run joins, each reachable by its name (services.go): made healthy
// before the command starts and removed with it. The run's Network is
// then "" (the network reaches out) or "none" (it does not).
Services []deploy.ServiceSpec
// Limits, when set, bound this run in place of the runner's own.
Limits Limits
}
type Result ¶
type Result struct {
Name string `json:"name"`
Path string `json:"path"`
Max int64 `json:"max"` // bytes; a larger file is ResultTooLarge
// Fresh: any file at Path is removed before the task starts, so what is
// read after it exits was left by this task.
Fresh bool `json:"fresh,omitempty"`
}
Result is a file a task declares it will leave (a review's verdict, a transcript): the backend reads it when the task's container exits and keeps it with the task, so what AppTown reads is bound to the task that produced it. Path is in the tree (/work/...) or the workspace's state (/apptown/state/...).
type ResultFile ¶
type ResultFile struct {
State ResultState `json:"state"`
Content []byte `json:"content,omitempty"` // present only
Reason string `json:"reason,omitempty"` // refused only
Size int64 `json:"size,omitempty"` // present and too large
}
ResultFile is a captured result file.
func (ResultFile) Valid ¶
func (f ResultFile) Valid() error
Valid says whether a captured result is one of the four states, whole.
type ResultState ¶
type ResultState string
ResultState is what became of a declared result file when its task exited. None is ever turned into another: absent is not empty.
const ( ResultPresent ResultState = "present" // a regular file, within its maximum: its bytes ResultAbsent ResultState = "absent" // nothing at the path ResultRefused ResultState = "refused" // something the worker will not read there (Reason) ResultTooLarge ResultState = "too-large" // a regular file over its maximum (Size) )
type Runner ¶
type Runner struct {
DockerBinary string
// Scratch is where a request's generated files are written when it
// names no folder of its own: a folder the Docker daemon can mount from.
// "" is the system's temporary folder.
Scratch string
// Refused, when set, refuses every run with it: the project-process
// runner of an AppTown that runs no project process on its own host.
Refused error
// ServiceLimits are each of a request's services' limits (services.go);
// zero is DefaultServiceLimits.
ServiceLimits Limits
// Limits bound every Docker run this runner makes (LimitFlags); a zero
// field is not bounded. AppTown sets them to its task limits.
Limits Limits
}
A zero Runner uses the host's docker executable. A binary can be supplied by the composition layer or a test without changing package-global state.
func (Runner) PrepareImage ¶
func (r Runner) PrepareImage(ctx context.Context, req ImageRequest) ImageAnswer
PrepareImage answers an ImageRequest on this runner's Docker, which is this machine's (a worker's own).
type TaskDriver ¶
type TaskDriver interface {
Capabilities() Capabilities
Ensure(context.Context, TaskSpec) (TaskObservation, error)
Start(context.Context, TaskRef) error
Observe(context.Context, TaskRef) (TaskObservation, error)
// Wait blocks until the task exits and returns its exit code. Cancelling
// ctx stops waiting, never the task.
Wait(context.Context, TaskRef) (int, error)
Stop(context.Context, TaskRef) error
Output(context.Context, TaskRef, OutputRead) (OutputPage, error)
// Release removes the task once its result is collected. A task in a
// shared workspace (TaskSpec.SharedWorkspace) leaves the workspace.
Release(context.Context, TaskRef) error
// ReleaseWorkspace removes a shared workspace. It is refused
// (ErrWorkspaceInUse) while any task in it is not released, and a
// workspace already gone is released. A driver without shared
// workspaces returns ErrUnsupported.
ReleaseWorkspace(ctx context.Context, key string) error
}
TaskDriver runs bounded tasks for one backend (design/operations/workers.md, "How work travels" and "Recovery"). It is the smallest contract the first consumer needs; terminals, image builds, captures and snapshot reads arrive with the consumers that need them.
A task is named by its caller's operation key. Ensure is idempotent: the same key finds the task already created instead of making another, so a retry after a lost reply cannot run the work twice. Ensure does not start the task; the caller records the returned identity first, then starts it. A driver keeps no records of its own beyond what the backend holds.
type TaskObservation ¶
type TaskObservation struct {
ID string
State TaskState
ExitCode int
// OOMKilled: Docker killed the task at its memory limit. Its exit code
// is then not its answer (a killed child in a pipeline can leave 0).
OOMKilled bool `json:"oomKilled,omitempty"`
// Request is the request digest the task was created for, where the
// backend keeps it: a caller acting on a task found by its key alone
// checks it is its own.
Request string
}
type TaskRef ¶
TaskRef names a task to its driver: the provider identity the caller recorded, or, before one was recorded, the task's key.
type TaskSource ¶
TaskSource is one commit in a repository on AppTown's side.
type TaskSpec ¶
type TaskSpec struct {
Key string
Admission Admission
Image string
Argv []string
Env []string
// Workspace is a host directory mounted at /work. Mounts and ExtraHosts
// are for the local Docker profile's caches and host routing. A remote
// driver shares no files with AppTown and refuses all three.
Workspace string
WorkspaceRO bool
Mounts []string
ExtraHosts []string
Labels []string
// Source is the committed tree a remote task works on at /work, sent to
// the backend as a Git bundle. A local driver refuses it.
Source *TaskSource
// other tasks (the invocations of one agent step attempt) instead of
// owning one. The workspace is made from Source the first time and must
// hold that commit after. Releasing the task leaves the workspace;
// ReleaseWorkspace removes it. Empty: the task owns its workspace, which
// goes with it. A driver without shared workspaces refuses it.
SharedWorkspace string `json:",omitempty"`
// AgentGit, with Source, gives the task a Git repository of its own at
// /work/.git when its workspace is made (an agent or review step's).
// Nil: /work has no .git.
AgentGit *AgentGit `json:",omitempty"`
// Files are generated files presented read-only in the task's container
// (files.go). The driver keeps them in a folder the task owns, from
// before it is created until it is released, and never rewrites them.
Files []File
// Inputs are files the task reads under /apptown/in, sent to the backend
// as content-addressed blobs (inputs.go). Results are files it declares
// it will leave, captured by the backend when it exits. Both need a
// backend claiming them (Capabilities).
// Left out of a request's identity when there are none, so a request
// without them keeps the identity it had before they existed.
Inputs []Input `json:",omitempty"`
Results []Result `json:",omitempty"`
// Images are OCI image layouts a release build declares it will leave
// in its workspace's state (image_results.go), captured when it exits.
// A request declaring them carries no environment, inputs or services.
Images []ImageResult `json:",omitempty"`
// Private is the task's own network, services and Docker daemon
// (private.go), on a backend claiming PrivateDocker. Network is the
// Docker network such a backend joins the task to, which it names
// itself: WorkerTasks refuses it, as it refuses Mounts.
Private *PrivateDocker `json:",omitempty"`
Network string `json:",omitempty"`
// Request is the caller's identity for what the task runs, when paths in
// the spec vary between identical requests (a freshly staged copy).
// Empty: Digest computes it from the spec.
Request string
}
TaskSpec is one task: its operation key (the caller's identity for it), the admission it runs under, and what it runs.
type Termination ¶
type Termination string
const ( Exited Termination = "exited" TimedOut Termination = "timed_out" Canceled Termination = "canceled" LaunchFailed Termination = "launch_failed" // OutOfMemory: Docker killed the container at its memory limit. The // command's exit is not its answer. OutOfMemory Termination = "out_of_memory" )
type WorkerAvailability ¶
type WorkerAvailability uint8
WorkerAvailability describes a fresh protocol observation, not readiness of every tool or image on the worker. An unreadable answer is not unreachable.
const ( WorkerUnknown WorkerAvailability = iota WorkerAvailable )
type WorkerClient ¶
WorkerClient is AppTown's one transport to a worker: every request, a task's or a chat session's, is built, sent and refused here. Its client must present the installation's certificate. What each endpoint means stays with WorkerTasks and the worker's SessionClient.
func (WorkerClient) Call ¶
func (c WorkerClient) Call(ctx context.Context, method, path string, in, out any, limit int64, refuse func(*http.Response) error) error
Call sends in (none when nil) as JSON with method to path and decodes the answer, read to at most limit bytes, into out (when not nil). An answer of 300 or over is refuse's error; a broken answer is ErrWorkerUnavailable.
func (WorkerClient) Do ¶
func (c WorkerClient) Do(ctx context.Context, method, path string, body io.Reader, header http.Header) (*http.Response, error)
Do sends one request to path on the worker. A request that got no answer is ErrWorkerUnavailable: what the worker did is unknown.
func (WorkerClient) EnsureBlob ¶
EnsureBlob sends a blob (a tar with one entry named "content", named by its SHA-256) unless the worker already has it. Task inputs and a chat's mounts travel this way.
type WorkerProfile ¶
type WorkerProfile struct {
Name string `json:"name"`
OS string `json:"os"`
Architecture string `json:"architecture"`
Kernel string `json:"kernel"`
Cgroup string `json:"cgroup"`
Docker string `json:"docker"`
StorageDriver string `json:"storageDriver"`
// DindImage is the private daemon's image as the worker is configured
// with it (a tag, such as docker:27-dind), or "none".
DindImage string `json:"dindImage"`
// DindImageID is Docker's local id of that image on the worker: what
// the worker compares to notice the image changed under a request.
DindImageID string `json:"dindImageId"`
// DindDigest is the image's repository digest (RepoDigests), as pulled
// from its registry; "unknown: ..." for an image without one.
DindDigest string `json:"dindDigest"`
// DindVersion is the version the image's own dockerd reports.
DindVersion string `json:"dindVersion"`
// Privileged says whether a privileged container ran when probed.
Privileged string `json:"privileged"`
// Exclusive is declared by the operator (apptown-worker -exclusive): the
// machine is the installation's alone. It is not measured.
Exclusive bool `json:"exclusive"`
// Egress is never contained in this profile.
Egress string `json:"egress"`
Measured time.Time `json:"measured"`
}
WorkerProfile is what a worker measured of the Docker it runs private Docker on, when it started, and what its operator declared. A fact that could not be measured is "unknown", never a guess.
func (WorkerProfile) Changed ¶
func (p WorkerProfile) Changed(now WorkerProfile) string
Changed says how now differs from p in what private Docker rests on: the operating system, architecture and kernel, the Docker version, the storage driver, the cgroup mode and the private daemon's image (its id, repository digest and daemon version). The time of measurement does not matter. "" is the same.
type WorkerTaskRequest ¶
type WorkerTaskRequest struct {
Key string `json:"key"`
Admission Admission `json:"admission"`
Image string `json:"image"`
Argv []string `json:"argv"`
Env []string `json:"env"`
Workspace string `json:"workspace,omitempty"`
// Source is the commit the workspace was uploaded with.
Source string `json:"source,omitempty"`
// Request is the client's identity for what the task runs. The worker
// holds a key to it; without one, to the request as sent.
Request string `json:"request,omitempty"`
ReadOnly bool `json:"readOnly,omitempty"`
Labels []string `json:"labels,omitempty"`
// Files are generated files the task sees read-only, kept in the
// task's own folder until it is released.
Files []File `json:"files,omitempty"`
// Inputs name blobs sent to the worker, presented read-only under
// /apptown/in; Results are files captured when the task exits.
Inputs []Input `json:"inputs,omitempty"`
Results []Result `json:"results,omitempty"`
// Images are OCI image layouts a release build leaves in its
// workspace's state, captured when it exits.
Images []ImageResult `json:"images,omitempty"`
// Private is the task's own network, services and daemon.
Private *PrivateDocker `json:"private,omitempty"`
}
WorkerTaskRequest is the wire form of an ensured task: the contract's spec without host paths. AppTown sends it and the worker decodes it, so the two cannot disagree on a field. Workspace names a workspace uploaded to the worker.
type WorkerTasks ¶
type WorkerTasks struct {
URL string
Client *http.Client
// Now is the clock a task's admission expiry is judged on before it is
// sent: the one that wrote it. Nil is the real clock.
Now func() time.Time
// MaxResultBytes bounds a returned bundle; zero means 256 MiB.
MaxResultBytes int64
}
WorkerTasks is the TaskDriver for a apptown-worker on another machine (design/operations/workers.md, "How work travels"). The worker shares no files with AppTown: a task's source goes to it as a Git bundle of one commit, and its work comes back as a bundle of the commits it made (Collect). Host paths, mounts and host routing are refused.
The client must present the installation's certificate; the worker accepts no other.
func (WorkerTasks) AgentImage ¶
func (w WorkerTasks) AgentImage(ctx context.Context) (AgentImage, error)
AgentImage asks the worker for its agent image: the name its build context gives and the image's labels. A worker without one answers ErrNoAgentImage.
func (WorkerTasks) Availability ¶
func (w WorkerTasks) Availability(ctx context.Context) (WorkerAvailability, error)
Availability uses the existing authenticated profile endpoint. It does not pull or inspect an agent image, start work, or cache a startup profile as proof of current reachability. Errors are diagnostic; public views use the state.
func (WorkerTasks) Capabilities ¶
func (w WorkerTasks) Capabilities() Capabilities
A worker presents inputs, keeps result files (inputs.go), captures images (image_results.go) and runs private Docker (private.go).
func (WorkerTasks) Collect ¶
Collect asks the worker for the commits the task made after base, with anything it left uncommitted committed under message, and writes them as a bundle to dest. It returns the worker's tip, or "" when the task changed nothing. The bundle is untrusted: import it with gitexec.Quarantine.
func (WorkerTasks) Disks ¶
Disks is each filesystem the worker writes to, as it reads them now (GET /v1/disks): its state folder, and its tasks' Docker data.
func (WorkerTasks) Ensure ¶
func (w WorkerTasks) Ensure(ctx context.Context, spec TaskSpec) (TaskObservation, error)
Ensure sends the task's inputs and has the worker check them, then sends its source, if it has one, then asks the worker to create the task. Each is idempotent under the task's key.
func (WorkerTasks) ImageFile ¶
func (w WorkerTasks) ImageFile(ctx context.Context, ref TaskRef, image, file string) (io.ReadCloser, int64, error)
ImageFile opens one file of a captured image (ImageFilePattern) and answers its length as the worker states it. The reader fails, as a transport error, if fewer bytes arrive: a short read is never a short file. A released task's files are ErrImageGone. The caller checks every byte against its descriptor's digest; the worker's word is not trusted.
func (WorkerTasks) ImageResults ¶
func (w WorkerTasks) ImageResults(ctx context.Context, ref TaskRef) (map[string]ImageCapture, error)
ImageResults reads what the worker captured of each declared image when the task exited, by name, each in one of its four states. A task that has not exited, or whose images the worker could not capture, is an error, never absence; a released task's are ErrImageGone.
func (WorkerTasks) Observe ¶
func (w WorkerTasks) Observe(ctx context.Context, ref TaskRef) (TaskObservation, error)
Observe reports a task. An unreachable worker is ErrTaskUnobservable, never absence.
func (WorkerTasks) Output ¶
func (w WorkerTasks) Output(ctx context.Context, ref TaskRef, read OutputRead) (OutputPage, error)
func (WorkerTasks) Profile ¶
func (w WorkerTasks) Profile(ctx context.Context) (WorkerProfile, error)
Profile is the profile the worker measured when it started (WorkerProfile): what its private Docker rests on.
func (WorkerTasks) Release ¶
func (w WorkerTasks) Release(ctx context.Context, ref TaskRef) error
Release removes the task, its own workspace on the worker (not one it shares) and the worker's record of it.
func (WorkerTasks) ReleaseWorkspace ¶
func (w WorkerTasks) ReleaseWorkspace(ctx context.Context, key string) error
ReleaseWorkspace removes a shared workspace from the worker. The worker refuses while a task naming it is not released (ErrWorkspaceInUse).
func (WorkerTasks) Results ¶
func (w WorkerTasks) Results(ctx context.Context, ref TaskRef) (map[string]ResultFile, error)
Results reads the result files the worker captured when the task exited, by name, each in one of its four states. A task that has not exited, or whose results the worker could not capture, is an error, never absence.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package executionfake is an in-memory TaskDriver for tests of callers.
|
Package executionfake is an in-memory TaskDriver for tests of callers. |
|
Package executiontest is the conformance suite every TaskDriver passes.
|
Package executiontest is the conformance suite every TaskDriver passes. |