taskflow

package
v0.8.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 16, 2026 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Overview

Package taskflow is an optional DAG runner for containerkit.

Hand the same *kit.Container and *kit.Compose values you already have to a Flow, plus ordinary Go functions. pkg/kit does not import this package; a program that never imports taskflow never downloads go-taskflow. The containerkit binary blank-imports it so a plan document with a flow field can run.

import "github.com/aptd3v/containerkit/pkg/taskflow"

New names the graph and holds an optional *kit.Client. Add steps, wire edges, then call Run once. Independent steps whose predecessors are done run concurrently. Run.Concurrency is the worker pool (0 means GOMAXPROCS, raised above the number of subflows).

cli may be nil when every step is Task or a Compose func that brings its own client. Container and Build need a client (ErrNoClient otherwise).

Flow and Subflow share the step constructors. Every one returns *Task.

  • Task(name, fn) — fn(ctx) error. Go only.
  • Build(name, ...EngineBuildOption) — Engine image build
  • Container(name, *kit.Container, *ContainerRun) — remove, create, start, wait
  • Compose(name, *kit.Compose, cmd) — Up, Down, … or a func
  • Subflow(name, instantiate) — nested graph; instantiate runs when the subflow starts
  • If(name, pred) — boolean branch (Then / Else). Go only.
  • Condition(name, pred) — indexed branch (Precede order). Go only.

Import builds the same graph from the JSON document the TSX Flow helper emits. Task, If, and Condition callbacks are not in that document: a TSX If is a boolean chosen when the module renders.

Container is a one-shot job. A non-zero exit fails the step. Restart policies (always, unless-stopped, on-failure) and AutoRemove are rejected. The step writes Name and the containerkit.* labels onto the container.

A failed or skipped step skips its successors. Always runs a step even when a predecessor failed. Retry(n) reruns a failing step n more times. Panics and a canceled context are not retried. If is skipped when a predecessor failed, and then neither branch runs. Condition still runs after a failed predecessor.

Dump writes DOT. List is docker ps -a for the flow label. Remove downs every Compose project the graph referenced, then deletes leftover labeled containers.

Index

Constants

View Source
const (
	// LabelFlow is the Docker label for the flow name.
	LabelFlow = "containerkit.flow"
	// LabelTask is the Docker label for the step name.
	LabelTask = "containerkit.task"
	// LabelDepends lists predecessor container names, sorted and comma-separated.
	LabelDepends = "containerkit.depends"
	// LabelPrecedes lists successor container names, sorted and comma-separated.
	LabelPrecedes = "containerkit.precedes"
)

Variables

View Source
var (
	// ErrNoClient is returned when a Container or Build step runs without a client.
	ErrNoClient = errors.New("docker client is nil")
	// ErrAlreadyRun is returned when Run is called a second time on the same Flow.
	ErrAlreadyRun = errors.New("already run")
)

Functions

func List

func List(ctx context.Context, cli *kit.Client, flowName string) (kit.ContainerListResult, error)

List returns containers stamped with LabelFlow equal to flowName, including stopped ones.

Types

type ComposeCmd added in v0.8.1

type ComposeCmd struct {
	// contains filtered or unexported fields
}

ComposeCmd runs one method of *kit.Compose. Construct it with Up, Down, and the other constructors in this file. The option slice is copied.

func Attach added in v0.8.1

func Attach(service string, opts ...kit.AttachOption) ComposeCmd

Attach attaches to a service container.

func Commit added in v0.8.1

func Commit(service string, opts ...kit.CommitOption) ComposeCmd

Commit creates an image from a service container.

func ComposeBuild added in v0.8.1

func ComposeBuild(opts ...kit.ComposeBuildOption) ComposeCmd

ComposeBuild builds images for the project.

func Copy added in v0.8.1

func Copy(opts ...kit.CopyOption) ComposeCmd

Copy copies files between the host and a service.

func Create added in v0.8.1

func Create(opts ...kit.ComposeCreateOption) ComposeCmd

Create creates containers for the project.

func Down added in v0.8.1

func Down(opts ...kit.DownOption) ComposeCmd

Down tears the project down.

func DryRun added in v0.8.1

func DryRun(v bool) ComposeCmd

DryRun toggles Compose dry-run mode.

func Events added in v0.8.1

func Events(opts ...kit.EventsOption) ComposeCmd

Events streams project events.

func Exec added in v0.8.1

func Exec(service string, opts ...kit.RunOption) ComposeCmd

Exec runs a command in a service container. A non-zero exit fails the step.

func Export added in v0.8.1

func Export(service string, opts ...kit.ExportOption) ComposeCmd

Export copies a service container's filesystem out.

func Images added in v0.8.1

func Images(opts ...kit.ImagesOption) ComposeCmd

Images lists images used by the project.

func Kill added in v0.8.1

func Kill(opts ...kit.KillOption) ComposeCmd

Kill signals project containers.

func Logs added in v0.8.1

func Logs(opts ...kit.LogsOption) ComposeCmd

Logs streams project logs. The consumer is discarded; progress follows the logger on the run context.

func MaxConcurrency added in v0.8.1

func MaxConcurrency(n int) ComposeCmd

MaxConcurrency sets the Compose runtime's parallel operation limit.

func Pause added in v0.8.1

func Pause(opts ...kit.PauseOption) ComposeCmd

Pause pauses the project's services.

func Port added in v0.8.1

func Port(service string, port uint16, opts ...kit.ComposePortOption) ComposeCmd

Port prints the public address of a service port.

func Ps added in v0.8.1

func Ps(opts ...kit.PsOption) ComposeCmd

Ps lists project containers.

func PublishOCI added in v0.8.1

func PublishOCI(repository string, opts ...kit.PublishOption) ComposeCmd

PublishOCI publishes the project as an OCI artifact.

func Pull added in v0.8.1

func Pull(opts ...kit.PullOption) ComposeCmd

Pull pulls images for the project.

func Push added in v0.8.1

func Push(opts ...kit.PushOption) ComposeCmd

Push pushes images for the project.

func Restart added in v0.8.1

func Restart(opts ...kit.ComposeRestartOption) ComposeCmd

Restart restarts the project's services.

func Rm added in v0.8.1

func Rm(opts ...kit.RemoveOption) ComposeCmd

Rm removes project containers.

func RunService added in v0.8.1

func RunService(service string, opts ...kit.RunOption) ComposeCmd

RunService starts a one-off service command. A non-zero exit fails the step.

func Scale added in v0.8.1

func Scale(opts ...kit.ScaleOption) ComposeCmd

Scale changes service replica counts.

func Start added in v0.8.1

func Start(opts ...kit.ComposeStartOption) ComposeCmd

Start starts the project's services.

func Stop added in v0.8.1

func Stop(opts ...kit.ComposeStopOption) ComposeCmd

Stop stops the project's services.

func Top added in v0.8.1

func Top(services ...string) ComposeCmd

Top lists processes in the named services.

func Unpause added in v0.8.1

func Unpause(opts ...kit.PauseOption) ComposeCmd

Unpause resumes the project's services.

func Up added in v0.8.1

func Up(opts ...kit.UpOption) ComposeCmd

Up brings the project up.

func Viz added in v0.8.1

func Viz(opts ...kit.VizOption) ComposeCmd

Viz renders the project graph.

func VolumeList added in v0.8.1

func VolumeList(opts ...kit.VolumesListOption) ComposeCmd

VolumeList lists volumes for the project.

func Wait added in v0.8.1

func Wait(opts ...kit.WaitOption) ComposeCmd

Wait waits for the project to exit.

func Watch added in v0.8.1

func Watch(opts ...kit.WatchOption) ComposeCmd

Watch rebuilds and syncs the project.

type ContainerRun

type ContainerRun struct {
	// Logs receives the container's stdout and stderr after it exits. Nil
	// means Run.Writer. Use io.Discard to silence one step.
	Logs io.Writer
}

ContainerRun is options for a Container step. Nil means defaults.

type Event

type Event struct {
	// Task is the step. Step names are not unique across subflows.
	Task *Task
	Name string
	Kind EventKind
	// Err is set for EventError and EventRetry.
	Err error
	// Attempt is the 1-based attempt that just failed. Set for EventRetry.
	Attempt  int
	Duration time.Duration
}

Event is one step lifecycle update. Watch may be called from several goroutines.

type EventKind

type EventKind string

EventKind is a step lifecycle signal sent to Run.Watch.

const (
	EventStart EventKind = "start"
	EventDone  EventKind = "done"
	EventSkip  EventKind = "skip"
	EventError EventKind = "error"
	// EventRetry is sent when an attempt failed and another follows.
	EventRetry EventKind = "retry"
)

type Flow

type Flow struct {
	// contains filtered or unexported fields
}

Flow is a named task graph. One Run per Flow.

func Import added in v0.8.1

func Import(data []byte, cli *kit.Client) (*Flow, error)

Import builds a Flow from a flow document or from a plan document that carries a flow field. cli may be nil until Run.

func New

func New(name string, cli *kit.Client) *Flow

New returns a Flow. name is the graph name and the flow label value. cli may be nil when the flow has only Task steps and Compose funcs.

func (*Flow) Build

func (f *Flow) Build(name string, opts ...kit.EngineBuildOption) *Task

Build adds an image build step. Options are the same ones BuildImage takes.

func (*Flow) Compose

func (f *Flow) Compose(name string, app *kit.Compose, cmd any) *Task

Compose adds a step that runs a compose command. cmd is a ComposeCmd (Up, Down, Logs, …) or a func(context.Context, *kit.Compose) error. A func(context.Context, *kit.Compose, kit.Composer) error is accepted too.

func (*Flow) Condition

func (f *Flow) Condition(name string, predict func(context.Context) uint) *Task

Condition adds an indexed branch. The returned index selects the successor in Precede order. A Condition runs even when a predecessor failed. An index with no successor fails the step instead of panicking.

func (*Flow) Container

func (f *Flow) Container(name string, ctr *kit.Container, opt *ContainerRun) *Task

Container adds a one-shot container step: remove by name, create, start, wait for exit. A non-zero exit fails the step. Restart policies (always, unless-stopped, on-failure) and AutoRemove are rejected.

func (*Flow) Dump

func (f *Flow) Dump(w io.Writer) error

Dump writes the graph in DOT format.

func (*Flow) If

func (f *Flow) If(name string, pred func(context.Context) bool) *Task

If adds a boolean branch. Then successors run when pred returns true, Else successors when it returns false. A missing branch is a no-op. An If is skipped when a predecessor failed, and then neither branch runs.

func (*Flow) List

func (f *Flow) List(ctx context.Context) (kit.ContainerListResult, error)

List returns containers stamped with this flow's name, including stopped ones.

func (*Flow) Name

func (f *Flow) Name() string

Name is the graph name passed to New.

func (*Flow) Remove

func (f *Flow) Remove(ctx context.Context) error

Remove downs every Compose project the graph referenced, then force-removes Container-step IDs and any leftover containers stamped with this flow's name.

func (*Flow) Run

func (f *Flow) Run(ctx context.Context, opt *Run) error

Run executes the graph. Nil opt uses defaults. The returned error joins every failed step's TaskError.

func (*Flow) Subflow

func (f *Flow) Subflow(name string, instantiate func(*Subflow)) *Task

Subflow adds a nested graph. instantiate runs once, when the subflow starts. Successors of the subflow are skipped when any step inside it failed.

func (*Flow) Task

func (f *Flow) Task(name string, fn func(context.Context) error) *Task

Task adds a function step.

type Priority

type Priority uint

Priority is scheduler queue order. Zero leaves go-taskflow's default.

const (
	PriorityHigh Priority = iota + 1
	PriorityNormal
	PriorityLow
)

type Run

type Run struct {
	// Concurrency is the worker count. Zero means GOMAXPROCS; the value is
	// raised above the number of known subflows.
	Concurrency uint
	// Writer receives build output, container output, and Compose output.
	// Nil means os.Stdout.
	Writer io.Writer
	// Profile receives flamegraph text when the graph finishes. Nil disables the profiler.
	Profile io.Writer
	// Trace receives Chrome Trace JSON when the graph finishes. Nil disables the tracer.
	Trace io.Writer
	// Watch is called from worker goroutines as each step starts, finishes,
	// retries, skips, or fails.
	Watch func(Event)
	// Log is called with step-scoped output (image build streams, container
	// logs). May be called from worker goroutines with partial writes.
	Log func(step string, p []byte)
}

Run is options for Flow.Run. Nil means defaults.

type Subflow

type Subflow struct {
	// contains filtered or unexported fields
}

Subflow is a nested graph. It has the same step constructors as Flow.

func (*Subflow) Build

func (s *Subflow) Build(name string, opts ...kit.EngineBuildOption) *Task

Build adds an image build step to the subflow.

func (*Subflow) Compose

func (s *Subflow) Compose(name string, app *kit.Compose, cmd any) *Task

Compose adds a Compose step to the subflow. See Flow.Compose.

func (*Subflow) Condition

func (s *Subflow) Condition(name string, predict func(context.Context) uint) *Task

Condition adds an indexed branch inside the subflow. See Flow.Condition.

func (*Subflow) Container

func (s *Subflow) Container(name string, ctr *kit.Container, opt *ContainerRun) *Task

Container adds a one-shot container step to the subflow. See Flow.Container.

func (*Subflow) If

func (s *Subflow) If(name string, pred func(context.Context) bool) *Task

If adds a boolean branch inside the subflow. See Flow.If.

func (*Subflow) Subflow

func (s *Subflow) Subflow(name string, instantiate func(*Subflow)) *Task

Subflow adds a nested graph inside this subflow.

func (*Subflow) Task

func (s *Subflow) Task(name string, fn func(context.Context) error) *Task

Task adds a function step to the subflow.

type Task

type Task struct {
	// contains filtered or unexported fields
}

Task is one node in a Flow or Subflow.

func (*Task) Always

func (t *Task) Always() *Task

Always runs t even when a predecessor failed or was skipped. Use it for teardown. The upstream failure still propagates: successors of t are skipped as if t had failed.

func (*Task) DependsOn

func (t *Task) DependsOn(tasks ...*Task) *Task

DependsOn makes t depend on tasks. It is the inverse of Precede.

func (*Task) Else

func (t *Task) Else(tasks ...*Task) *Task

Else adds successors that run when an If step returns false. On any other step it does nothing.

func (*Task) Err

func (t *Task) Err() error

Err is the step error after Run, if any. For a Subflow it reports only the subflow's own error; inner step errors are on the inner tasks and in the error returned by Run.

func (*Task) ExitCode

func (t *Task) ExitCode() int

ExitCode is the container wait status after a Container step runs.

func (*Task) ID

func (t *Task) ID() string

ID is the Engine container ID after a Container step runs.

func (*Task) Name

func (t *Task) Name() string

Name is the step name.

func (*Task) Precede

func (t *Task) Precede(tasks ...*Task) *Task

Precede makes tasks depend on t. For a Condition, the order is the branch index. For an If, Precede is the same as Then.

func (*Task) Priority

func (t *Task) Priority(p Priority) *Task

Priority sets scheduler queue order. Zero is ignored.

func (*Task) Retry

func (t *Task) Retry(n int) *Task

Retry reruns a failing step up to n more times before it counts as failed. Each rerun emits EventRetry. Panics and a canceled context are not retried.

func (*Task) Then

func (t *Task) Then(tasks ...*Task) *Task

Then adds successors that run when an If step returns true. On any other step it is the same as Precede.

type TaskError

type TaskError struct {
	Task string
	Err  error
}

TaskError is a named step failure. Unwrap returns Err.

func (*TaskError) Error

func (e *TaskError) Error() string

func (*TaskError) Unwrap

func (e *TaskError) Unwrap() error

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL