runtime

package
v0.1.0-alpha.41 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 22, 2026 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Overview

Package runtime abstracts the container engine. The implementation is the embedded containerd (package containerd, ADR 0011).

Index

Constants

View Source
const (
	LabelManaged  = "armada.managed"
	LabelApp      = "armada.app"
	LabelRevision = "armada.revision"
	LabelReplica  = "armada.replica"
	LabelSpecHash = "armada.spec-hash"
	// LabelFenceAfter is how many seconds without the control plane stop
	// the container (ADR 0041); absent: never.
	LabelFenceAfter = "armada.fence-after"
)

Labels set on every container Armada manages.

View Source
const (
	StateCreated    = "created"
	StateRunning    = "running"
	StateRestarting = "restarting"
	StateExited     = "exited"
	StatePaused     = "paused"
	StateDead       = "dead"
	// StateStarting is a replica's container whose init containers or
	// sidecars the driver is still starting (ADR 0038).
	StateStarting = "starting"
)

Container states.

View Source
const (
	RoleInit    = "init"
	RoleSidecar = "sidecar"
)

Roles of the containers of a replica, in the LabelRole label.

View Source
const (
	LabelRole      = "armada.role"
	LabelContainer = "armada.container"
)

Labels on init containers and sidecars, besides the replica's own: their role and name. A container without LabelRole is a replica's own.

View Source
const (
	LogMaxSize  = 10 << 20
	LogMaxFiles = 3
	LogRate     = 1 << 20
	LogBurst    = 4 << 20
)

Log limits for container output: rotation by size, and a write rate per container (bytes per second, with a burst) past which output is dropped so one container cannot take a node's disk bandwidth.

View Source
const (
	MaxExecArgs   = 64
	MaxExecArgLen = 4096
	// MaxExecChunk is the most data one exec message carries.
	MaxExecChunk = 32 << 10
)

Exec limits (ADR 0015).

View Source
const LabelSystem = "armada.system"

LabelSystem marks Armada's own infrastructure containers (e.g. HAProxy).

Variables

View Source
var DroppedCaps = []string{"NET_RAW", "MKNOD", "AUDIT_WRITE"}

DroppedCaps are removed from the engine's default capability set for app containers.

View Source
var ErrNotFound = errors.New("not found")

ErrNotFound is returned when a container or image does not exist.

Functions

func CanDial

func CanDial(d Driver) bool

CanDial reports whether d dials from containers, like CanExec.

func CanExec

func CanExec(d Driver) bool

CanExec reports whether d runs commands in app containers, like CanRunSystem.

func CanMountVolumes

func CanMountVolumes(d Driver) bool

CanMountVolumes reports whether d mounts volumes into app containers.

func CanRunMembers

func CanRunMembers(d Driver) bool

CanRunMembers reports whether d runs init containers and sidecars.

func CanRunSystem

func CanRunSystem(d Driver) bool

CanRunSystem reports whether d runs system containers.

func CanSandbox

func CanSandbox(d Driver) bool

CanSandbox reports whether d keeps network sandboxes, like CanExec.

func CanStats

func CanStats(d Driver) bool

CanStats reports whether d samples what containers use, like CanRunSystem.

func CheckContainerSpec

func CheckContainerSpec(s ContainerSpec) error

CheckContainerSpec enforces the rules every driver applies before running an app container: read-only regular-file mounts, private DNS servers, and a port published only on a private address.

func CheckExecCommand

func CheckExecCommand(cmd []string) error

CheckExecCommand checks a command for ExecApp: at least one argument, at most MaxExecArgs of at most MaxExecArgLen bytes each, and no NUL bytes, which cannot reach a process's arguments. Errors never repeat the arguments, which may hold secret values.

func CheckFileMount

func CheckFileMount(f FileMount) error

CheckFileMount allows only absolute, clean paths and a regular host file, so a mount can never expose an engine socket or a host directory.

func CheckSandboxID

func CheckSandboxID(id string) error

CheckSandboxID refuses an ID made of anything but an app ID's characters: it names a file and network plugin state on the node.

func CheckSystemMount

func CheckSystemMount(m Mount) error

CheckSystemMount accepts absolute paths without characters that engines treat as separators.

func CheckVolumeMount

func CheckVolumeMount(v VolumeMount) error

CheckVolumeMount allows a directory (not a link) at an absolute, clean host path, mounted at a target the spec would accept for a volume.

func ContainsFold

func ContainsFold(line, needle []byte) bool

ContainsFold reports whether line contains needle, ignoring case; needle must already be lower case. ASCII needles are matched without allocating, so filtering a busy log stays cheap.

func HasBridge

func HasBridge(d Driver) bool

HasBridge reports whether d reports a bridge network, like CanRunSystem.

func ValidMemberName

func ValidMemberName(name string) bool

ValidMemberName reports whether name can name an init container or a sidecar: a valid spec name of at most 30 characters, other than the replica's own container's.

Types

type AgentStarter

type AgentStarter interface {
	AgentStart(ctx context.Context) error
}

AgentStarter is told when the agent starts, after Ping: it does what only the agent should, never a command like doctor that also pings the runtime. It is optional; an error is only logged.

type Bridge

type Bridge struct {
	// Gateway is the host's address on the bridge, reachable only from the
	// node itself and its containers.
	Gateway netip.Addr
	Subnet  netip.Prefix
}

Bridge describes the network app containers are attached to.

type BridgeInspector

type BridgeInspector interface {
	Bridge(ctx context.Context) (Bridge, error)
}

BridgeInspector reports the app container network. It is optional: internal ingress and DNS need it.

type Container

type Container struct {
	ID       string
	Name     string
	Image    string
	Labels   map[string]string
	State    string
	ExitCode int
	HostIP   string
	HostPort int
	// IP is the container's address on its bridge network, reachable from
	// the node for health checks; empty if unknown.
	IP        string
	StartedAt time.Time
	// Members are the replica's init containers and sidecars (ADR 0038),
	// listed with the replica's own container rather than on their own. An
	// init container or sidecar whose replica's container is gone is listed
	// by itself, with LabelRole set, to be removed.
	Members []Member
	// StartError says why the driver gave up starting a replica's init
	// containers or sidecars (one could not be created, for example); empty
	// otherwise. It names no values from the spec.
	StartError string
}

Container is the observed state of a container.

func (Container) FailedInit

func (c Container) FailedInit() (Member, bool)

FailedInit returns the init container of c that exited with another code than 0, if any.

func (Container) Member

func (c Container) Member(name string) (Member, bool)

Member returns the init container or sidecar of c called name.

func (Container) StoppedSidecar

func (c Container) StoppedSidecar() (Member, bool)

StoppedSidecar returns a sidecar of c that no longer runs, if any.

type ContainerDialer

type ContainerDialer interface {
	// DialContainer dials address, an IP and port, from the network
	// namespace of the running container id.
	DialContainer(ctx context.Context, id, network, address string) (net.Conn, error)
}

ContainerDialer opens connections from inside an app container's network namespace, so the connection reaches only what the app could reach itself (queue rules, ADR 0034). It is optional.

type ContainerSpec

type ContainerSpec struct {
	Name        string
	Image       string
	Entrypoint  []string
	Cmd         []string
	Env         []string
	Labels      map[string]string
	Port        int    // container TCP port to publish; 0 for none
	HostIP      string // address to publish on; must be a private IP
	MemoryBytes int64
	NanoCPUs    int64
	// CPUReserveMillis sets the CPU weight: under contention, containers
	// share CPU in proportion to what they reserve. 0 keeps the default.
	CPUReserveMillis int64
	PidsLimit        int64
	// Files are mounted read-only; the driver refuses sources that are not
	// regular files (e.g. a runtime socket).
	Files []FileMount
	// DNS overrides the container's nameservers (private IPs only); empty
	// keeps the engine default.
	DNS []string
	// DNSSearch are search domains put before the host's: the internal
	// zone of the app's space (spec.ValidSearchDomain), so its apps
	// resolve by name alone.
	DNSSearch []string
	// Volumes are mounted read-write (ADR 0023); only drivers for which
	// CanMountVolumes reports true accept them.
	Volumes []VolumeMount
	// Init containers run in order, each to completion, before Sidecars
	// and then the container itself start; all of them share its network
	// namespace (ADR 0038). Only drivers for which CanRunMembers reports
	// true accept them.
	Init     []MemberSpec
	Sidecars []MemberSpec
}

ContainerSpec describes a container to run. Security defaults (no-new-privileges, dropped capabilities, no privileged mode) are applied by the driver and cannot be loosened through this struct.

type Driver

type Driver interface {
	// Ping verifies that the engine is reachable and supported.
	Ping(ctx context.Context) error
	// Pull fetches image for this node's platform. It returns a
	// *PlatformError when the image has no variant for it.
	Pull(ctx context.Context, image string, auth *RegistryAuth) (Image, error)
	// Run creates and starts a container. One with init containers or
	// sidecars (ContainerSpec.Init) is created and returned at once, and
	// listed as StateStarting while the driver runs the init containers in
	// order and starts the sidecars, then starts it. When an init container
	// fails, it is left exited, never started, and Container.FailedInit
	// names the init container, whose output stays readable until Remove.
	// A container the driver stopped starting (it went away) is listed as
	// stopped.
	Run(ctx context.Context, spec ContainerSpec) (string, error)
	// Stop stops a container (ending its start if it is still starting),
	// then its sidecars, each with grace.
	Stop(ctx context.Context, id string, grace time.Duration) error
	// Remove removes a container with its init containers and sidecars.
	Remove(ctx context.Context, id string) error
	// List returns containers (running or not) carrying all given labels.
	List(ctx context.Context, labels map[string]string) ([]Container, error)
	Inspect(ctx context.Context, id string) (Container, error)
	Logs(ctx context.Context, id string, opts LogOptions) (LogStream, error)
}

Driver runs containers on the local node.

type ExecOptions

type ExecOptions struct {
	// Cmd is the command and its arguments (CheckExecCommand).
	Cmd []string
	// TTY allocates a terminal; output then arrives on Stdout only.
	TTY bool
	// Size is the terminal's initial size; Resize carries changes until
	// the command ends.
	Size   TermSize
	Resize <-chan TermSize
	// Stdin is the command's standard input; nil gives it none. EOF closes
	// the command's standard input.
	Stdin io.Reader
	// Stdout and Stderr receive the command's output.
	Stdout, Stderr io.Writer
}

ExecOptions describes a command to run in an app container (ADR 0015).

type Execer

type Execer interface {
	// ExecApp runs opts.Cmd in a running app container and returns its exit
	// code once it has ended and its output has been written. When ctx
	// ends while output may still be on its way, it returns ctx's error
	// (with the exit code, if known), so a cut output is never taken for
	// a whole one.
	ExecApp(ctx context.Context, id string, opts ExecOptions) (exitCode int, err error)
}

Execer runs commands in app containers (armada app exec, ADR 0015). The command gets the container's own security context: user, capabilities, no-new-privileges, seccomp, limits, and namespaces. It is optional.

type FileMount

type FileMount struct {
	Source string // absolute path of a regular file on the node
	Target string // absolute path in the container
}

FileMount bind-mounts a single host file read-only into a container.

type Image

type Image struct {
	// Ref is the digest-pinned reference ("repo@sha256:..."), or "" if the
	// engine reports no digest.
	Ref string
	// Platforms lists the image's platforms as "os/arch" (sorted, no
	// duplicates); empty if the engine cannot tell.
	Platforms []string
}

Image is a pulled image.

type ImagePruner

type ImagePruner interface {
	// PruneImages removes every image that no container uses, that keep
	// does not name (by reference or digest), and that was not pulled or
	// updated within grace. It returns the names of the removed images.
	PruneImages(ctx context.Context, keep []string, grace time.Duration) ([]string, error)
}

ImagePruner removes images nothing needs any more. It is optional.

type LogFrame

type LogFrame struct {
	Stream Stream
	Data   []byte
	// Time is when the line was written; zero when the runtime does not say.
	Time time.Time
}

LogFrame is one chunk of container output.

type LogOptions

type LogOptions struct {
	Follow bool
	Tail   int // 0 means all, negative none (with Follow: only new output)
	// Grep keeps only lines containing it, ignoring case (lower case; empty
	// for all); Tail counts matching lines.
	Grep string
}

LogOptions controls log retrieval.

type LogPruner

type LogPruner interface {
	PruneLogs(ctx context.Context, retention time.Duration, maxBytes int64) error
}

LogPruner removes log history older than retention, then the oldest until it fits maxBytes (0 means the runtime's default).

type LogQuery

type LogQuery struct {
	App       string
	Replica   string // empty for every replica
	Container string // empty for the app's own container
	// Since and Until bound the entries' times; a zero Until means now.
	Since, Until time.Time
	// Grep keeps entries containing it, ignoring case; never a pattern.
	Grep string
	// Limit keeps the newest entries; MaxBytes bounds what is read.
	Limit    int
	MaxBytes int64
}

LogQuery asks for a window of an app's output from the node's log history and live logs (ADR 0043).

type LogRecord

type LogRecord struct {
	Time      time.Time
	Stream    Stream
	Data      []byte
	Replica   string
	Container string
	// App and Revision are set by LogTailer.
	App, Revision string
}

LogRecord is one entry a log search found.

type LogSearcher

type LogSearcher interface {
	SearchLogs(ctx context.Context, q LogQuery) ([]LogRecord, string, error)
}

LogSearcher is implemented by runtimes that keep a log history (ADR 0043). Search returns the newest q.Limit matching entries, oldest first, and a notice when it stopped early.

type LogStream

type LogStream interface {
	Next() (LogFrame, error)
	io.Closer
}

LogStream yields log frames until io.EOF.

type LogTail

type LogTail struct {
	Records []LogRecord
	Commit  func() error
}

LogTail is app output a LogTailer read; Commit records it as sent, so the next tail starts after it.

type LogTailer

type LogTailer interface {
	TailLogs(ctx context.Context, target string, maxBytes int64) (LogTail, error)
}

LogTailer reads app containers' output written since the last committed tail, at most about maxBytes of it, also across restarts (ADR 0044). target names the destination: for a new one, containers that already exist start at the end of their output.

type Member

type Member struct {
	ID        string
	Name      string
	Role      string
	State     string
	ExitCode  int
	StartedAt time.Time
}

Member is the observed state of an init container or a sidecar.

type MemberSpec

type MemberSpec struct {
	// Name is unique within the replica and a valid container name
	// (spec.Container).
	Name             string
	Image            string
	Entrypoint       []string
	Cmd              []string
	Env              []string
	MemoryBytes      int64
	NanoCPUs         int64
	CPUReserveMillis int64
	PidsLimit        int64
	Files            []FileMount
	Volumes          []VolumeMount
}

MemberSpec is an init container or a sidecar of a replica (ADR 0038). It gets the same security defaults as the replica's own container, joins its network namespace and its host files, and mounts only the files and volumes it lists, which must be among the replica's.

type Mount

type Mount struct {
	Source   string
	Target   string
	ReadOnly bool
}

Mount is a bind mount from the host.

type NetworkReleaser

type NetworkReleaser interface {
	ReleaseNetwork(ctx context.Context, id string) error
}

NetworkReleaser frees the network of a stopped container that is kept only for its logs. Releasing twice is harmless. It is optional.

type NetworkSandboxer

type NetworkSandboxer interface {
	// EnsureSandbox creates the sandbox id, or returns the one that exists.
	EnsureSandbox(ctx context.Context, id string) (Sandbox, error)
	// RemoveSandbox releases the sandbox's address and removes it; one that
	// is gone is not an error.
	RemoveSandbox(ctx context.Context, id string) error
	// ListSandboxes returns the node's sandboxes, so that an agent that
	// restarts adopts or removes them.
	ListSandboxes(ctx context.Context) ([]Sandbox, error)
	// DialSandbox dials address, an IP and port, from the sandbox.
	DialSandbox(ctx context.Context, id, network, address string) (net.Conn, error)
}

NetworkSandboxer keeps network sandboxes, from which the agent dials what a job's containers could reach while the job has none (queue triggers, ADR 0040). It is optional.

type OutputActivity

type OutputActivity struct {
	// contains filtered or unexported fields
}

OutputActivity tells when a command's output last moved. Runtimes wrap the writers they copy a command's output to with Wrap and, once the command exited, keep copying while output moves (WaitQuiet): a child the command left in the background can hold the output open for as long as it lives, but output still on its way to a slow reader is never cut. The zero value is ready to use.

func (*OutputActivity) WaitQuiet

func (a *OutputActivity) WaitQuiet(ctx context.Context, done <-chan struct{}, grace time.Duration) error

WaitQuiet waits until done is closed, or until no output has moved for grace, counted from the call at the earliest. It returns ctx.Err() when ctx ends first, since output may then have been cut.

func (*OutputActivity) Wrap

func (a *OutputActivity) Wrap(w io.Writer) io.Writer

Wrap returns a writer that writes to w and counts as activity; nil stays nil.

type PlatformError

type PlatformError struct {
	Image string
	// Want is the node's platform, Have the image's, both as "os/arch".
	Want string
	Have []string
}

PlatformError reports an image without a variant for the node's platform.

func (*PlatformError) Error

func (e *PlatformError) Error() string

type RegistryAuth

type RegistryAuth struct {
	Username      string
	Password      string
	ServerAddress string
}

RegistryAuth holds credentials for a private registry. It prints without the password.

func (RegistryAuth) GoString

func (a RegistryAuth) GoString() string

GoString implements fmt.GoStringer without the password.

func (RegistryAuth) LogValue

func (a RegistryAuth) LogValue() slog.Value

LogValue implements slog.LogValuer without the password.

func (RegistryAuth) String

func (a RegistryAuth) String() string

String implements fmt.Stringer without the password.

type Sandbox

type Sandbox struct {
	// ID names the sandbox (a job's ID).
	ID string
	// IP is its address on the container bridge.
	IP string
}

Sandbox is a network namespace wired into the node's container network like a container's, with no container and no process in it (ADR 0040).

type Stats

type Stats struct {
	// CPU is the CPU time the container used since it started.
	CPU time.Duration
	// MemoryBytes is the memory in use, without page cache the kernel can
	// reclaim.
	MemoryBytes int64
	// EgressBytes is what the container sent over its network interface
	// since it started; 0 when unknown.
	EgressBytes int64
	// NetworkLost is true when the container's network namespace has no
	// interface left: it can reach nothing until it is recreated.
	NetworkLost bool
	// OOMKills counts the processes the kernel killed at the container's
	// memory limit since it started. It stays readable after the container
	// exits, until it is removed.
	OOMKills int64
}

Stats is what an app container uses (ADR 0016).

type StatsReader

type StatsReader interface {
	Stats(ctx context.Context, id string) (Stats, error)
}

StatsReader samples what app containers use (armada app top, ADR 0016). It is optional.

type Stream

type Stream int

Stream identifies stdout or stderr.

const (
	Stdout Stream = 1
	Stderr Stream = 2
)

Streams.

type SystemRunner

type SystemRunner interface {
	// RunSystem creates and starts a host-network container with all
	// capabilities dropped except CapAdd.
	RunSystem(ctx context.Context, spec SystemSpec) (string, error)
	// Exec runs cmd in a running container and returns combined output.
	Exec(ctx context.Context, id string, cmd []string) (output string, exitCode int, err error)
	// Signal sends a signal (e.g. "SIGUSR2") to the container's main process.
	Signal(ctx context.Context, id, signal string) error
}

SystemRunner runs and controls Armada's infrastructure containers. It is separate from Driver so the app code path can never reach host networking or bind mounts.

type SystemSpec

type SystemSpec struct {
	Name   string
	Image  string
	Cmd    []string
	Labels map[string]string
	Mounts []Mount
	// CapAdd lists the only capabilities granted; all others are dropped.
	CapAdd []string
	User   string
}

SystemSpec describes an Armada infrastructure container. Unlike ContainerSpec it allows host networking and bind mounts, so it must never be built from user input.

type TermSize

type TermSize struct {
	Rows, Cols uint16
}

TermSize is a terminal's size in character cells.

type VolumeMount

type VolumeMount struct {
	Source string // absolute path of a directory on the node
	Target string // absolute path in the container
	// Shared marks a mount other containers, on this node or others, use
	// too (an NFS export, ADR 0042): it keeps the label it was mounted
	// with instead of getting the container's.
	Shared   bool
	ReadOnly bool
}

VolumeMount is a persistent volume: a directory the agent created for the app on this node, mounted read-write without setuid or device files.

Directories

Path Synopsis
Package conformance tests that the container runtime behaves as Armada needs (ADR 0011): the security defaults, checked from inside containers, the container lifecycle, labels, publishing on a private address, logs, exec, stats, volumes, network sandboxes, and errors.
Package conformance tests that the container runtime behaves as Armada needs (ADR 0011): the security defaults, checked from inside containers, the container lifecycle, labels, publishing on a private address, logs, exec, stats, volumes, network sandboxes, and errors.
Package containerd implements runtime.Driver on the containerd instance that Armada embeds (ADR 0011).
Package containerd implements runtime.Driver on the containerd instance that Armada embeds (ADR 0011).
Package embedded runs containers on Armada's own containerd instance (ADR 0011).
Package embedded runs containers on Armada's own containerd instance (ADR 0011).
mkbundle command
Command mkbundle downloads the artifacts pinned in embedded.Lock, checks their SHA-256, and writes the runtime pack that release builds embed.
Command mkbundle downloads the artifacts pinned in embedded.Lock, checks their SHA-256, and writes the runtime pack that release builds embed.
Package fake provides an in-memory runtime.Driver for tests.
Package fake provides an in-memory runtime.Driver for tests.
Package imagecheck checks that an image's layers are what its config says they are before a runtime reuses an unpacked layer for it.
Package imagecheck checks that an image's layers are what its config says they are before a runtime reuses an unpacked layer for it.
Package supervisor is the container runtime of ADR 0046: images and container records kept by the agent itself, containers supervised by armada-shim.
Package supervisor is the container runtime of ADR 0046: images and container records kept by the agent itself, containers supervised by armada-shim.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL