Documentation
¶
Overview ¶
Package kubernetes runs each step of a workflow as a POD of its own.
The dynamic is section 2's, and it is what separates this engine from a monolithic worker: the pod starts with the step's EXACT image, runs a command, reports and dies. A dbt step starts the dbt image with 1Gi; the Go fetcher next to it starts a 10 MB image with 32Mi. Under a single image both would pay the larger of the two — in pull bytes, in reserved memory and in surface.
The client is written on the stdlib, with no client-go. The official library brings hundreds of dependencies and tens of MB for what here are four calls REST: create a pod, read its status, read its log, delete it. The same choice was already made for React (a vendored bundle) and for the CSS (standalone Tailwind): a large dependency's cost only pays for itself when a large fraction of it is used.
Index ¶
- func PodName(t execution.TaskExec) string
- type API
- type Client
- func (c *Client) CreatePod(ctx context.Context, p Pod) (Pod, error)
- func (c *Client) DeletePod(ctx context.Context, name string) error
- func (c *Client) LerPod(ctx context.Context, name string) (Pod, error)
- func (c *Client) Logs(ctx context.Context, name string, follow bool) (io.ReadCloser, error)
- func (c *Client) Namespace() string
- type Condition
- type Container
- type EnvSource
- type ErrOutsideCluster
- type Executor
- type Grace
- type KeyRef
- type Metadata
- type Options
- type PVCSource
- type Pod
- type PodSpec
- type PodStatus
- type RefLocal
- type Resources
- type StatusContainer
- type Var
- type VarSource
- type Volume
- type VolumeMount
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func PodName ¶ added in v0.7.0
PodName produces a valid and STABLE name for the same attempt.
STABLE matters: if the process dies between creating the pod and recording that, the next attempt finds the existing pod (409 AlreadyExists) instead of starting a second one running the same dbt in parallel with the first.
The hash suffix resolves the collision the 63-character cut would create between two long node names with a common prefix.
Types ¶
type API ¶
type API interface {
CreatePod(ctx context.Context, p Pod) (Pod, error)
LerPod(ctx context.Context, name string) (Pod, error)
Logs(ctx context.Context, name string, follow bool) (io.ReadCloser, error)
DeletePod(ctx context.Context, name string) error
}
API is what the executor needs from the server. An interface in the consumer: it is what makes it possible to test the pod's whole lifecycle against a fake server.
type Client ¶ added in v0.7.0
type Client struct {
// contains filtered or unexported fields
}
Cliente speaks to the API server.
func (*Client) CreatePod ¶ added in v0.7.0
CreatePod creates the pod and returns the name it was given.
type Condition ¶ added in v0.7.0
type Condition struct {
Type string `json:"type"`
Status string `json:"status"`
Reason string `json:"reason,omitempty"`
Message string `json:"message,omitempty"`
}
Condition carries PodScheduled, where the scheduler explains why it did not fit.
type Container ¶
type Container struct {
Name string `json:"name"`
Image string `json:"image"`
Command []string `json:"command,omitempty"`
Args []string `json:"args,omitempty"`
Env []Var `json:"env,omitempty"`
EnvFrom []EnvSource `json:"envFrom,omitempty"`
Resources *Resources `json:"resources,omitempty"`
WorkingDir string `json:"workingDir,omitempty"`
VolumeMounts []VolumeMount `json:"volumeMounts,omitempty"`
// TerminationMessagePath is where the step publishes its context, and the
// kubelet copies that file's contents into the pod's status when the
// container ends -- with success or not.
//
// Declared even though it is the default, because it is half of a contract:
// the other half is the BREVIS_OUTPUT the step is handed, and the two
// disagreeing means a step writes somewhere nobody reads. Written side by
// side, they cannot drift.
TerminationMessagePath string `json:"terminationMessagePath,omitempty"`
}
type ErrOutsideCluster ¶ added in v0.7.0
type ErrOutsideCluster struct{ Reason string }
ErrOutsideCluster is returned when there is no service account mounted.
func (ErrOutsideCluster) Error ¶ added in v0.7.0
func (e ErrOutsideCluster) Error() string
type Executor ¶
type Executor struct {
// Status polling interval. Polling rather than watching is deliberate: a
// watch needs reconnection, resync and handling of missed events to gain
// seconds on a task that lasts minutes.
Interval time.Duration
// contains filtered or unexported fields
}
Executor runs each step as a pod.
The cycle is always the same: create the pod, wait for it to leave Pending, follow the log while it runs, read the exit code and delete it. No state lives here beyond the in-flight pods -- if the process restarts, the pods keep running and the dispatcher finds them again by their deterministic name.
func NewExecutor ¶
func (*Executor) Execute ¶
func (e *Executor) Execute(ctx context.Context, t execution.TaskExec) (<-chan execution.Event, error)
Execute creates the pod and returns the event channel. The channel closes when the pod finishes -- the same shape as the local executor, so the runner cannot tell them apart.
type Options ¶ added in v0.7.0
type Options struct {
Namespace string
ServiceAccount string
PullSecrets []string
NodeSelector map[string]string
Tolerations []Grace
EnvFromSecrets []string
EnvFromConfigMaps []string
// CredencialPVC and CredencialPath mount a volume where the SDK keeps the
// credential it rotates between runs.
//
// With both set, EVERY step pod gets the volume and a BREVIS_CREDENTIAL_DIR
// env pointing at the mount. Without them nothing changes -- which is how
// the feature stays a shortcut rather than a requirement.
//
// The credential on the volume is encrypted; the key is an ordinary Secret,
// arriving through EnvFromSecrets. The engine neither sees it nor needs
// it.
CredentialPVC string
CredentialPath string
// AllowedSecrets are the Secrets a YAML may name in `secrets:`.
//
// It exists because `secrets:` inverts who chooses. EnvFromSecrets comes
// from the scheduler's environment: the INSTALLATION decides. `secrets:` is
// in the file, and the file is written by somebody else -- without this
// list, a workflow could mount any Secret in the namespace, including
// Brevis's own database secret, and run an arbitrary command holding it.
//
// Empty denies everything. Denying by default costs one variable in the
// installation; allowing by default costs the opposite, and the opposite is
// irreversible.
//
// The final division is this: the installation says WHICH secrets exist for
// workflows, the YAML says WHICH step receives each one.
AllowedSecrets []string
Labels map[string]string
Shell []string
// EsperaParaIniciar is how long a pod may go without starting before the
// step gives up. It exists because `Pending` is not an error to Kubernetes:
// a pod that fits on no node sits there forever, and without this limit the
// step waits along with it -- no log, no failure, no retry. It happened in
// dev with a CPU request larger than the pool's free capacity.
EsperaParaIniciar time.Duration
// KeepFailedPod leaves the pod around for inspection when a step fails.
// A successful one is always deleted: thousands of Completed pods clutter
// the namespace and say nothing Brevis's own history does not say better.
KeepFailedPod bool
}
Opcoes parameterises how pods are created. These are the INSTALLATION's decisions -- credentials, node pool, service account -- not the workflow author's: a pipeline YAML must not get to pick the service account it runs as.
type Pod ¶
type Pod struct {
APIVersion string `json:"apiVersion,omitempty"`
Kind string `json:"kind,omitempty"`
Metadata Metadata `json:"metadata"`
Spec PodSpec `json:"spec,omitempty"`
// A pointer because `omitempty` does not omit an empty struct: without it
// every created pod would send `"status":{}` to the server -- harmless, but
// noise in an object people read to debug.
Status *PodStatus `json:"status,omitempty"`
}
Pod is the subset of the object this engine uses. Writing the structs by hand instead of importing client-go's keeps the dependency tree small and makes it visible exactly what is sent to the API server.
func BuildPod ¶ added in v0.7.0
BuildPod translates a task into the object that goes to the API server.
A pure function: it takes a task and options and returns the object. That is what makes it possible to test the whole spec -- image, command, resources, labels -- with no cluster at all.
func (Pod) PublishedContext ¶ added in v0.8.0
Saida returns the container's exit code and whether it has finished. PublishedContext is what the step wrote to its termination message.
The engine already fetches this status to get the exit code, so the context arrives with it: no extra call, no volume, no port.
Read on FAILURE too, and deliberately: a step that published and then failed said something true up to that point, and the runner decides what to do with it. Reading only on success throws away the one clue a failing step left.
func (Pod) Reason ¶ added in v0.7.0
Motivo is the status's `reason` (DeadlineExceeded, OOMKilled, Evicted) — the difference between "the code failed" and "the cluster killed the process".
func (Pod) WaitReason ¶ added in v0.7.0
WaitReason explains why the container has not run yet.
It is the most useful piece of information when a step "does nothing": ImagePullBackOff and CreateContainerConfigError are configuration problems that, without this, would show up only as a pod sitting still until the timeout.
type PodSpec ¶
type PodSpec struct {
RestartPolicy string `json:"restartPolicy,omitempty"`
ServiceAccountName string `json:"serviceAccountName,omitempty"`
ImagePullSecrets []RefLocal `json:"imagePullSecrets,omitempty"`
NodeSelector map[string]string `json:"nodeSelector,omitempty"`
Tolerations []Grace `json:"tolerations,omitempty"`
ActiveDeadlineSeconds *int64 `json:"activeDeadlineSeconds,omitempty"`
Volumes []Volume `json:"volumes,omitempty"`
Containers []Container `json:"containers"`
}
type PodStatus ¶
type PodStatus struct {
Phase string `json:"phase,omitempty"`
Conditions []Condition `json:"conditions,omitempty"`
Reason string `json:"reason,omitempty"`
Message string `json:"message,omitempty"`
ContainerStatuses []StatusContainer `json:"containerStatuses,omitempty"`
}
type StatusContainer ¶
type StatusContainer struct {
Name string `json:"name"`
State struct {
Waiting *struct {
Reason string `json:"reason"`
Message string `json:"message"`
} `json:"waiting,omitempty"`
Running *struct {
StartedAt string `json:"startedAt"`
} `json:"running,omitempty"`
Terminated *struct {
ExitCode int `json:"exitCode"`
Reason string `json:"reason"`
Message string `json:"message"`
} `json:"terminated,omitempty"`
} `json:"state"`
}
type VarSource ¶ added in v0.7.0
type VarSource struct {
SecretKeyRef *KeyRef `json:"secretKeyRef,omitempty"`
}
FonteVar points a variable at a key of a Secret. The value never passes through the engine: the kubelet reads it when starting the container.
type Volume ¶
type Volume struct {
Name string `json:"name"`
PVC *PVCSource `json:"persistentVolumeClaim,omitempty"`
}
Volume is a PersistentVolumeClaim mounted into the pod.
PVC only, and not the union of everything Kubernetes accepts: the engine mounts a volume for one purpose -- keeping a rotated credential between runs -- and a field that exists for one purpose should not accept ten shapes.