Documentation
¶
Overview ¶
Package shell runs pipeline-defined commands via `sh -c`, either on the host (HostRunner) or inside a container (DockerRunner, see docker.go).
Index ¶
- Constants
- func BuildEnv(ctx context.Context) map[string]string
- func BuildEnvNames() []string
- func CanceledError(ctx context.Context) error
- func CloseRunner(r Runner, label string)
- func FormatBytes(n int) string
- func HostEnv() []string
- func IgnoreTerminalStops()
- func IsExitError(err error) bool
- func IsReadOnly(ctx context.Context) bool
- func KillAll()
- func NewPrefixedStream(label string, dst io.Writer) (w io.Writer, flush func())
- func OwnerHost() string
- func OwnershipLabels() map[string]string
- func PrefixLines(label, s string) string
- func PrepareImages(ctx context.Context, images []string) error
- func ProcessGone(pid int) bool
- func ReadOnly(ctx context.Context) context.Context
- func ResolveMountPath(cwd string) (string, error)
- func Run(ctx context.Context, r Runner, script string) error
- func RunCapture(ctx context.Context, r Runner, script string) ([]byte, error)
- func RunCaptureFull(ctx context.Context, r Runner, script string) (stdout, stderr string, exitCode int, err error)
- func RunCaptureFullLimited(ctx context.Context, r Runner, script string, maxBytes int, spillDir string) (stdout, stderr string, exitCode int, err error)
- func RunCaptureFullLimitedStreamed(ctx context.Context, r Runner, script string, maxBytes int, spillDir string) (stdout, stderr string, exitCode int, err error)
- func RunShell(ctx context.Context, command, cwd string) error
- func RunShellCapture(ctx context.Context, command, cwd string) ([]byte, error)
- func RunShellCaptureFull(ctx context.Context, command, cwd string) (stdout, stderr string, exitCode int, err error)
- func SpillPointerMessage(totalBytes int, path string, preview []byte) string
- func Start(ctx context.Context, cmd *exec.Cmd) (release func(), err error)
- func SweepOrphanedContainers(ctx context.Context, dockerHost string)
- func SweepOrphanedContainersOn(ctx context.Context, client *dockerapi.Client)
- func ValidateDocker(ctx context.Context) error
- func WidenSettleBoundForTest(bound time.Duration) (restore func())
- func WithBuildMetadata(ctx context.Context, m BuildMetadata) context.Context
- func WrapIfCanceled(ctx context.Context, err error) error
- type BuildMetadata
- type Capture
- type Command
- type DockerRunner
- type ExitError
- type HostRunner
- type Output
- type RemoteInput
- type Result
- type Runner
- type RunnerSpec
- type Streams
Constants ¶
const SignalledExitCode = dockerapi.SignalledExitCode
SignalledExitCode is the status a command killed by a signal reports.
os/exec answers -1 for a locally signalled process, and the venue carries the same sentinel across the wire so the two are indistinguishable to everything downstream. Named because two callers now have to ASK the question — "did this command choose its status, or was it killed?" — and a bare -1 at a call site says neither.
const SpillMaxBytes = 10 << 20 // 10 MiB
SpillMaxBytes bounds how much of an overflowing stream spillWriter will write to disk — a disk-exhaustion guard against a runaway model-directed command, the same reason boundedWriter caps memory. A file that hits this cap gets the same trailing "... [truncated N bytes]" marker a dropped boundedWriter overflow gets, just applied to the file instead of memory.
const SpillPreviewBytes = 2000
SpillPreviewBytes is how much of a spilled stream's head is echoed inline in the pointer message, so the model has some immediate signal without having to open the file first. Exported so internal/agent's one-shot spill helper (for MCP/sub-agent/fix output, which already holds its full content as a string rather than streaming it) uses the same preview size as spillWriter's own streaming spill path — one pointer-message shape across every spilled-output site, not two that can drift apart.
Variables ¶
This section is empty.
Functions ¶
func BuildEnv ¶
BuildEnv is the metadata the context carries as name→value, or nil when it carries none. An empty field is unset rather than empty, the STEPS_WORKER convention, and a fresh map is returned because callers merge into it.
func BuildEnvNames ¶
func BuildEnvNames() []string
BuildEnvNames lists the variables build metadata sets.
func CanceledError ¶
CanceledError returns a non-nil error wrapping ctx.Err() when ctx is canceled, or nil otherwise. RunCaptureFull/RunCaptureFullLimited deliberately report even a signal-killed process as data (a normal nonzero exit), not a Go error, so their own err return can't be used to detect "this result may be incomplete/unreliable because ctx was canceled while the command ran." A caller that needs that distinction — to tell a shutdown-interrupted step apart from a step that genuinely failed on its own — calls CanceledError immediately after such a call returns, instead.
func CloseRunner ¶
CloseRunner is a best-effort cleanup helper for the deferred Close calls every NewRunner caller owes: a cleanup failure must never mask the original error already being returned, so it is only logged. Mirrors internal/workspace's CloseBuild/CloseSpace precedent.
func FormatBytes ¶
FormatBytes renders n as a human-readable size (bytes, KB, or MB with one decimal place) for the spill pointer message. Exported so internal/agent's one-shot spill helper renders the same units/precision SpillPointerMessage itself uses.
func HostEnv ¶
func HostEnv() []string
HostEnv returns the subset of the current process's environment allowed to reach a host-executed command (see hostEnvAllowlist), in os.Environ's "KEY=VALUE" form so it can be assigned directly to exec.Cmd.Env. Exported so internal/mcp's stdio transport can apply the same trust boundary to an mcp_servers: subprocess.
func IgnoreTerminalStops ¶ added in v0.2.1
func IgnoreTerminalStops()
IgnoreTerminalStops makes a tty read or write from a background group fail with EIO instead of stopping this process, and children inherit it across exec.
func IsExitError ¶
IsExitError reports whether err (however wrapped) stems from a process that started and then exited nonzero or was killed by a signal — as opposed to an infrastructure failure to run it at all. Callers use it to tell a task-level failure (a command's own nonzero exit) apart from an errored one so hooks dispatch correctly; a resource check's failure, by contrast, is left unwrapped so it classifies as errored.
"Started and then exited nonzero" is exactly what processStarted answers, including for a command a venue ran on another machine — the two questions were always the same one, and only os/exec's monopoly on the answer made them look different.
func IsReadOnly ¶
IsReadOnly reports whether ctx was marked by ReadOnly.
func KillAll ¶ added in v0.2.1
func KillAll()
KillAll SIGKILLs every session Start has running: the second ^C, which must not wait out a step's grace.
func NewPrefixedStream ¶
NewPrefixedStream is prefixedStream: a writer that stamps each line with "[label] " on its way to dst, and the flush that emits a trailing partial line. A venue streams a worker's output through this so a placed step's lines look like every other step's.
func OwnerHost ¶
func OwnerHost() string
OwnerHost is the hostname this process labels what it leaves on other machines with.
func OwnershipLabels ¶
OwnershipLabels are stamped on every container this tool starts.
They are how a container survives its creator. If this process is SIGKILLed, nothing runs Close, and the only thing left is a container with a name and no explanation; these let the NEXT run recognise it as ours and whose it was. See SweepOrphanedContainers.
func PrefixLines ¶
PrefixLines prepends "[label] " to every line of s (splitting on \n and rejoining, so a trailing partial line — s not ending in \n — is prefixed too without gaining a synthetic newline it didn't have). Used for output that's captured in full before being printed (runAssertedTask/runFixTask's post-hoc printTaskOutput), as opposed to prefixedWriter's live-streaming case. Returns s unchanged when label is "" or s is empty.
func PrepareImages ¶
PrepareImages makes sure every image the pipeline can execute in is present on the daemon, pulling the ones that aren't.
This exists because an implicit pull is charged to the wrong place. Docker pulls on first use, so without this the first command needing an uncached image pays for the download inside its own step: the progress output goes wherever that command's stderr goes — mixed into a resource check's parsed output, or into an agent's tool result — and the time counts against the step's timeout, so a big image on a cold daemon can burn a budget meant for the work. Pulling up front makes it startup cost, visible as such.
Present images are skipped by asking the daemon, which is a local call: a warm run adds a few milliseconds and no network at all. That check is also what keeps a locally-built image (one that exists in no registry) working — it is found, so nothing is pulled.
Progress goes to the terminal rather than being captured. It is startup output for the operator, not a command's result.
One client for the whole sweep rather than one per image: connecting is the expensive half of asking about an image that is already there.
func ProcessGone ¶
ProcessGone reports whether pid definitely names no process on this machine; a pid it cannot check is not gone.
func ReadOnly ¶
ReadOnly marks ctx as carrying a command that cannot modify the tree, so a venue can skip bringing the outputs home afterwards.
The fetch after every command is what keeps a placed step's assert: and its local tree honest, and for a task — one command, then the check — it costs one transfer. An agent is different in kind: a conversation issues dozens of file-tool calls, and a read_file that fetched the step's outputs each time would pay a full transfer, over a worker's tunnel or as a presigned PUT, to bring back a tree nothing had touched.
It is stated by the caller rather than inferred from the command, because only the caller knows: a shell string cannot be read for whether it writes, and guessing wrong in the permissive direction loses a model's edits.
func ResolveMountPath ¶
ResolveMountPath returns cwd as an absolute path with symlinks resolved, so the host path handed to the daemon as a bind mount matches the real filesystem location Docker Desktop (or the daemon) actually shares. Rejects a resolved path containing ':' — a bind mount is spelled `host:container`, so a path containing one would be silently misparsed into the wrong mount (or rejected with a confusing error) rather than failing clearly here.
Exported for a caller outside this package that mounts the same directory at the same resolved path a step's session container uses. The one that did was internal/agent's containerized CLI run, deleted by issue #100.
func Run ¶
Run streams both streams live with stdin attached unless ctx is headless (a daemon's stdin is the terminal that launched it, and a read of that from a background process group stops the whole daemon); any nonzero exit is an error.
func RunCapture ¶
RunCapture returns stdout while streaming stderr live, stdin attached as Run attaches it; any nonzero exit is an error. The captured output is logged at debug on both outcomes, so a failing check/out command's output is not lost to the terse "exit status N".
func RunCaptureFull ¶
func RunCaptureFull(ctx context.Context, r Runner, script string) (stdout, stderr string, exitCode int, err error)
RunCaptureFull captures both streams unbounded and silently, with no stdin, reporting a nonzero exit as data — for a when: guard, whose output is not user-facing on its own.
func RunCaptureFullLimited ¶
func RunCaptureFullLimited(ctx context.Context, r Runner, script string, maxBytes int, spillDir string) (stdout, stderr string, exitCode int, err error)
RunCaptureFullLimited is RunCaptureFull bounded by maxBytes and spillDir; see Command.Limit.
func RunCaptureFullLimitedStreamed ¶
func RunCaptureFullLimitedStreamed(ctx context.Context, r Runner, script string, maxBytes int, spillDir string) (stdout, stderr string, exitCode int, err error)
RunCaptureFullLimitedStreamed is RunCaptureFullLimited, also streaming both streams live: run_shell and custom tools, whose model-directed commands were otherwise invisible until the agent's final answer, and a task judged by assert:/fix:, which needs its output both live and in hand.
func RunShell ¶
RunShell runs command on the host via `sh -c command` with cwd as its working directory, streaming stdout/stderr live. Equivalent to Run over HostRunner{cwd: cwd}; kept as a package-level function for callers that never need containerized execution.
func RunShellCapture ¶
RunShellCapture runs command on the host via `sh -c command`, capturing stdout. Equivalent to RunCapture over HostRunner{cwd: cwd}.
func RunShellCaptureFull ¶
func RunShellCaptureFull(ctx context.Context, command, cwd string) (stdout, stderr string, exitCode int, err error)
RunShellCaptureFull runs command on the host via `sh -c command`, capturing stdout/stderr separately and reporting a normal nonzero exit as data. Equivalent to RunCaptureFull over HostRunner{cwd: cwd}.
func SpillPointerMessage ¶
SpillPointerMessage renders the message a spilled-to-file tool result returns to the model in place of raw content: the true size, the file it was saved to, and a preview of its head. Shared by spillWriter (streaming command output, via resultFromFile) and internal/agent's one-shot spillOrTruncate (MCP tool text/structured content, a sub-agent's final answer, a fix loop's failure output — all of which already hold their full content as a string rather than streaming it), so every spilled-output path produces byte-identical wording rather than each inventing its own.
func Start ¶ added in v0.2.1
Start isolates a headless child in its own session, so a /dev/tty open fails instead of SIGTTIN-stopping the daemon, and a cancel reaps grandchildren; interactive children stay foreground so their prompts work.
func SweepOrphanedContainers ¶
SweepOrphanedContainers removes containers left behind by a steps process that never got to clean up after itself.
Close removes a step's container on every path a program can control, including failures and Ctrl-C. SIGKILL is the one it cannot: the process stops between instructions, and the container keeps running. Before this, the only thing that eventually reclaimed it was the keepalive expiring a day later — so a box where runs get killed (an OOM, a CI executor reaped mid-build, a laptop closed) accumulated live containers holding memory for 24 hours each.
An orphan is identified by its owning process no longer existing. That is precise where an age heuristic is not: a legitimately long step can run for hours, and killing its container because it looked old would break a working build to tidy up after a broken one. Pid reuse only ever makes this conservative — a reused pid means the container is skipped and reclaimed by a later run instead.
Only containers labeled with THIS hostname are considered, since a pid is meaningful only on the machine that issued it.
dockerHost names the daemon to sweep; empty is this machine's own. A WORKER daemon is swept on exactly the same terms and needs no special case, because the labels already answer the question that matters: a container a placed step started carries the ORCHESTRATOR's hostname and the ORCHESTRATOR's pid, so the host filter keeps it to containers this machine is responsible for and the pid is checkable against this machine's process table — which is precisely what ownerHostname was written for. Without pointing it anywhere but here, a placed containerized step whose teardown could not reach the daemon left a container running on a machine nothing ever swept.
Entirely best-effort: every failure is logged and none is returned. Running the pipeline matters more than tidying up before it.
func SweepOrphanedContainersOn ¶
SweepOrphanedContainersOn is SweepOrphanedContainers through a client the caller already holds, for a daemon with no address here (a docker+ worker's, over ssh).
func ValidateDocker ¶
ValidateDocker fails fast when a pipeline configures image: but the daemon is not usable. Mirrors internal/workspace's Provider.Validate() precedent — check once at startup, before any step runs.
The daemon and nothing else. There used to be a docker BINARY half as well, because container execution spawned one; it does not, so a machine with a reachable daemon and no docker installed is now a perfectly good machine to run a containerized pipeline on.
Which daemon is dockerapi's question, not this one's, and it is not the same question as "is DOCKER_HOST set" — see internal/dockerapi/host.go.
func WidenSettleBoundForTest ¶
WidenSettleBoundForTest is that seam for a test outside this package whose fixture image must be caught dying at birth on a loaded daemon, where 300ms loses the bet and the step's command runs in the dying container instead. Not safe beside a parallel test; restore with the func it returns.
func WithBuildMetadata ¶
func WithBuildMetadata(ctx context.Context, m BuildMetadata) context.Context
WithBuildMetadata carries m to every runner command made under the context. A zero m strips any inherited metadata.
func WrapIfCanceled ¶
WrapIfCanceled is wrapIfCanceled: it makes err's chain satisfy errors.Is(_, ctx.Err()) when ctx was cancelled by the time the command finished.
A venue needs it for the same reason HostRunner does, and cannot infer it from the wire: a cancelled remote command reports a signal death, which is indistinguishable from a command that died on its own. Only the context knows which happened.
Types ¶
type BuildMetadata ¶
type BuildMetadata struct {
RunID, JobName, PipelineName, PipelineRevision, URL string
}
BuildMetadata is which build a command is part of, handed to every command as STEPS_RUN_ID, STEPS_JOB_NAME, STEPS_PIPELINE_NAME, STEPS_PIPELINE_REVISION and STEPS_URL — what Concourse calls build metadata.
Never hashed: a run id changes every run, so a step keyed on it would never hit. It travels on the context, never in a config field, so no merkle builder can see it. config refuses these names in env: (see its reservedEnvNames).
func MetadataFrom ¶ added in v0.2.0
func MetadataFrom(ctx context.Context) BuildMetadata
MetadataFrom is the metadata the context carries, zero when none.
type Capture ¶
type Capture struct {
// contains filtered or unexported fields
}
Capture accumulates one stream of a command's output under the same rules RunCaptureFullLimited applies: unbounded when maxBytes is zero or less, truncated with a trailing marker when spillDir is empty, spilled to a file under spillDir otherwise.
The writer stays on the orchestrator even when the command does not. A spill file names a path a later read_file resolves HERE, so writing it on the worker would hand a model a pointer to a file it cannot open.
func NewCapture ¶
NewCapture returns a Capture applying the policy maxBytes and spillDir describe.
type Command ¶
type Command struct {
// Script runs through `sh -c`.
Script string
// Stdin wires this process's stdin through. Off, the command reads
// /dev/null: a model-directed command inheriting an interactive stdin
// (cat with no args, a tool prompting for input) blocks until the step's
// timeout instead of getting EOF. An ssh:// worker cannot honour it — its
// stdin carries the step's env, never the operator's terminal.
Stdin bool
// Stream sends the named streams live to events.Stdout/Stderr, prefixed
// when WithLabel was used.
Stream Streams
// Capture keeps both streams in the Result. Limit caps each one while the
// command RUNS, not afterwards, so a runaway command never fully buffers
// before being discarded: zero is unbounded; past it the overflow is
// dropped behind a "... [truncated N bytes]" marker (internal/agent's
// truncateToolOutput format), or, with SpillDir set, written in full to a
// file under SpillDir and replaced by a pointer message — see spillWriter.
// The writer is always on this machine, wherever the command runs, so a
// spill file is a path a later read_file resolves here.
Capture bool
Limit int
SpillDir string
// ExitAsData reports a nonzero exit, even a signal-killed one, in
// Result.Code rather than as an error; only a command that never started
// (no container, an unreachable worker, a bad cwd) is an error then, with
// Result.Code -1. Off, any nonzero exit is an error whose chain satisfies
// IsExitError, and the Result still carries whatever was captured.
//
// A caller that must tell "incomplete because ctx was cancelled" from an
// ordinary exit code asks CanceledError afterwards: a killed process is
// data here, by design.
ExitAsData bool
}
Command is one command and what to do with it: four independent settings every Runner honours the same way, so the combinations live in the helpers below rather than in a method per combination on every runner.
type DockerRunner ¶
type DockerRunner struct {
Image string
// contains filtered or unexported fields
}
DockerRunner runs commands inside one container per step: the session is started lazily on the first command and torn down by Close.
It used to be a fresh throwaway container per command, which had three problems this shape fixes at once. State did not carry between calls — an agent's `pip install x` followed by `python y` as two run_shell calls simply did not work, and no amount of prompting reliably stops a model from trying. Every call paid full container-start latency, which an agent conversation pays dozens of times. And a container the caller lost track of was reaped only much later, because nothing on our side knew its name.
The working directory is bind-mounted at its own (resolved) host path and set as the container's workdir, so host-side readers (agent read_file/list_dir, workspace Capture) see the same files a containerized command wrote. No host environment variables are passed through — the container starts from the image's own env only.
func (DockerRunner) Close ¶
func (d DockerRunner) Close() error
Close tears down the step's container. Safe to call when no command ever ran (nothing was started, so there is nothing to remove) and safe to call twice.
func (DockerRunner) Exec ¶
Exec runs cmd in the step's container. See Runner.
Daemon-level refusals (a bad image) surface as the same 125 `docker run` reports, and a command the container could not run or find as its own 126/127 — like any other nonzero exit, data under ExitAsData. Only a failure to have a container at all is an error there.
func (DockerRunner) WithLabel ¶
func (d DockerRunner) WithLabel(label string) Runner
WithLabel returns a copy of d that prefixes its live-streamed output — see the Runner interface doc. The copy shares the same container session, so a labeled runner is the same step's container, not a second one.
type ExitError ¶
type ExitError struct {
// Command is the shell string that failed. Error masks its URL credentials.
Command string
// Venue names the machine, because a red build on a fleet that does not
// say which box sends the operator to look at the wrong one. Empty for a
// container on the daemon this process is already talking to, where the
// caller names the image instead.
Venue string
// Code is the exit status, or -1 for a signalled command — the same
// sentinel exitCodeOf reports for a local kill, so the two are
// indistinguishable to every caller downstream.
Code int
}
ExitError is a command that started and then exited nonzero (or was killed by a signal) somewhere os/exec cannot see it: on a worker at the far end of a venue, or in a container this process talks to over the engine API.
It exists because the whole pipeline decides "the step said no" from "the machinery broke" by asking IsExitError, and *exec.ExitError — the only answer there used to be — carries an *os.ProcessState, has no constructor, and so can only ever describe a child of THIS process. Neither a venue nor a container has one. Reporting either as a plain error would classify every red step as infrastructure, firing on_error where the pipeline author wrote on_failure.
The inverse matters just as much and is why a venue must NOT reach for this type when a command never ran at all: guard.go turns that case into "guard command could not run" and fails the step, where an ExitError would let an unreachable worker read as "the guard said false" and silently skip work.
type HostRunner ¶
type HostRunner struct {
// contains filtered or unexported fields
}
HostRunner runs commands directly on the host via `sh -c`, with cwd as its working directory.
func (HostRunner) Close ¶
func (h HostRunner) Close() error
Close removes a fenced runner's private TMPDIR; an unfenced one holds nothing open.
func (HostRunner) WithLabel ¶
func (h HostRunner) WithLabel(label string) Runner
WithLabel returns a copy of h that prefixes its live-streamed output — see the Runner interface doc.
type Output ¶
Output is where one running command's two streams land on this end, built from a Command so every runner — and a venue running the command on another machine — streams, captures and bounds it identically.
type RemoteInput ¶
type RemoteInput struct {
Digest string
// Holder is the worker URL, as the mapping was written.
Holder string
}
RemoteInput is where an input's bytes are when they are on a worker.
type Runner ¶
type Runner interface {
// Exec runs one command as cmd describes. Every runner answers the same
// matrix (see Command); the package-level Run* helpers name the
// combinations callers actually use.
Exec(ctx context.Context, cmd Command) (Result, error)
// WithLabel returns a copy of the runner that prefixes every line of
// whatever it streams live (see Command.Stream) with "[label] ". A zero-value/unset
// label (the default from NewRunner) streams unprefixed, byte-identical
// to before this method existed.
WithLabel(label string) Runner
// Close releases whatever the runner holds open — for a DockerRunner,
// the step's container. Every caller of NewRunner must call it (see
// CloseRunner), including on the failure path: it is what keeps a step
// from stranding a container. A HostRunner has nothing to release and
// returns nil. Closing a runner that never ran a command is valid, as is
// closing twice; a copy made by WithLabel shares one underlying session,
// so closing either closes both.
Close() error
}
Runner runs pipeline-defined commands, either on the host or inside a container, against the working directory it was constructed with (see NewRunner) — binding cwd at construction, rather than taking it on every call, means a runner reused across many calls (an agent conversation's repeated run_shell calls, a task's fix-loop re-runs) resolves/validates its working directory once, not once per call.
func NewRunner ¶
func NewRunner(spec RunnerSpec) (Runner, error)
NewRunner returns a DockerRunner scoped to spec, or a HostRunner when spec.Image is empty — the single decision point every shell-out caller (task, agent, resource) funnels through. For a DockerRunner, Cwd is resolved and validated once here (see ResolveMountPath). HostRunner never errors — cwd needs no resolution for host execution. The returned runner has no label (see WithLabel) until a caller that wants prefixed output opts in explicitly.
type RunnerSpec ¶
type RunnerSpec struct {
// Image runs commands in a container from this image; empty runs them on
// the host.
Image string
// Cwd is the working directory. For a container it is bind-mounted at its
// own resolved host path and set as the workdir; empty (only a resource
// check:) mounts nothing — and, with a Worker, sends and fetches nothing.
Cwd string
// MountPath is where Cwd's tree lives on the DAEMON's filesystem, when
// that is not this machine. Set it and Cwd is not resolved locally —
// EvalSymlinks would fail on a path that exists on a worker and not here.
//
// This is the seam a placed containerized step turns on: the tree was
// sent to the worker, and the bind mount has to name the worker's copy,
// because that is the filesystem the daemon resolves `-v` against.
MountPath string
// DockerHost is the daemon to run this step's containers on. Empty is
// this machine's; a venue sets it to a socket it forwards to a worker.
DockerHost string
// DockerDial reaches DockerHost through a connection the caller makes, for a socket with no address here (a docker+ worker's, over ssh); DockerHost then only names it in errors.
DockerDial func(ctx context.Context) (net.Conn, error)
// Volumes are daemon-side named volumes as `name:target`; with MountPath they REPLACE the tree bind, because a docker+ worker's tree lives in volumes and never at a path.
Volumes []dockerapi.Bind
// EnvValues are variables the CALLER supplies with their values rather
// than by name, for the ones that exist nowhere in this process's own
// environment. A venue's STEPS_WORKER is the case: it names the tag a
// placed command is running on, and Env's name-only forwarding cannot
// carry a variable nobody set here.
EnvValues map[string]string
// Env are the variable NAMES the pipeline's env: opted this command into,
// on top of hostEnvAllowlist. They resolve against the steps process's own
// environment; a name that isn't set is simply absent, which is what lets
// a pipeline name an optional variable without every operator having to
// define it.
Env []string
// User is the container's user:, passed straight to the daemon.
// Empty takes the platform default — see defaultContainerUser, which is
// where the Linux file-ownership problem is actually handled. Ignored for
// host execution, which always runs as whoever started steps.
User string
// Network is the container's network mode; "none" cuts the container off
// from the network entirely.
// Empty takes docker's own default. On a host command only "none" beside
// Sandbox means anything; config rejects every other host use at load time.
Network string
// Privileged runs the container privileged. Ignored
// for host execution, which config rejects at load time.
Privileged bool
// Sandbox fences a HOST command with the OS sandbox (see newHostFence); with it, Network "none" cuts the host command off too. Config refuses it beside Image and on a placed step.
Sandbox bool
// count. Zero leaves it unset.
CPUShares int
// MemoryBytes is the memory limit, in bytes. Zero leaves it unset.
// A container over it is OOM-killed, which surfaces as exit 137.
MemoryBytes int64
// Worker is the venue a step's tags: resolved to — a worker URL, already
// mapped from tag to machine by whoever parsed the CLI. Empty, the
// overwhelming default, runs on this machine and leaves the host/container
// decision below exactly as it was.
//
// Cwd stays a LOCAL path when this is set: it is the tree that gets sent
// and the tree the results come back into. Nothing above here has to know
// which machine ran the command.
Worker string
// WorkerTag is the tag that resolved to Worker, published to the command
// as STEPS_WORKER so a script can tell it is on a worker and which one.
// Empty for a step running on this machine, which is what makes the
// variable's absence meaningful.
WorkerTag string
// Fetch names the directories to bring back after each command, which for
// a task step are its declared outputs.
//
// Named rather than everything, because a step that consumed a large
// checkout should not ship it home again to prove it did not change. Empty
// with a Worker set means a command whose effects on the tree are not
// wanted — a guard, which reads the tree and answers a question about it.
Fetch []string
// FetchAll brings the WHOLE tree back after each command, for a command
// whose output is the directory itself rather than a declared subset — a
// resource's in:, which fills an empty directory and declares nothing.
// Meaningless without a Worker.
FetchAll bool
// Caches are the step's caches: (path relative to Cwd → cache key). A host or local container gets them in Cwd already; a docker+ worker mounts a volume of its own at each, so the cache stays on the machine that ran the step.
Caches map[string]string
// HoldAs is the artifact name a FetchAll tree kept on its worker (DeferFetch) is held under: the name the next step offers it by. Read only by the venue.
HoldAs string
// Keep leaves a worker's scratch behind, following --keep-workspace. Read
// only by the venue: on this machine the workspace decides its own fate,
// and it is the one that answered this.
Keep bool
// Subdir is where the command runs, relative to Cwd, for a step whose working directory is below the tree that has to travel. Only a placed agent with dir: sets it: the venue sends and fetches the whole step directory, so the outputs it declares still resolve, while the container works one level down. Empty runs in Cwd itself.
Subdir string
// ReusedWarm is a worker taken from its idle window (?idle=) with nothing proving it still alive. Read only by the venue: a first connection it refuses is treated as the machine having been reclaimed, so the step is re-placed rather than failed.
ReusedWarm bool
// DeferFetch keeps the declared outputs (or, with FetchAll, the whole
// tree) ON the worker after each command instead of bringing them home:
// the worker files them under their digests and reports the digests, and
// the bytes come home only when something on this machine reads them.
// Wrong for a step whose outputs are read here as part of the step
// itself — an assert: files:, a fix: agent's file tools — so the caller
// sets it only when nothing does. Read only by the venue.
DeferFetch bool
// RemoteInputs are inputs whose bytes are NOT under Cwd: a worker holds
// each, under the digest given, and the venue offers it to the step's
// worker by that digest: pulled from the holder into a temporary
// directory here and sent on, never into the step's own tree. Only meaningful with a Worker; the caller
// leaves this empty otherwise and materializes the input under Cwd.
RemoteInputs map[string]RemoteInput
}
RunnerSpec is everything that decides how one step's commands execute: the container to run them in (or the host, when Image is empty), the working directory, and the per-entity knobs a pipeline can set alongside image:.
A struct rather than a parameter list because these arrive together from one config entity (a resource_type, task, or agent, with a step's overrides already merged in by config.Resolve*), and every future knob of the same kind belongs here rather than at the end of a growing signature.
type Streams ¶
type Streams uint8
Streams names which of a command's output streams go out live.
const ( StreamStdout Streams = 1 << iota StreamStderr StreamBoth = StreamStdout | StreamStderr )
The streams a Command can stream live.
Source Files