engine

package
v0.12.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 8, 2026 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Overview

Package engine implements the wave-scheduling execution engine for Shipwright's declarative workflow layer (design.md D-K, workflow-execution spec). It is the FIRST place in the fixed seven-stage validation pipeline (design.md D-H) where a manifest's compiled graph.Graph (stage 5), its resolved providers.Registry (stage 6), and interpolated with/input values (stage 7) are all in scope simultaneously — see WU7's apply-progress ("Closing the WU5/WU6 loop") for the exact boundary this package closes: a steps.<id>.output reference's Kind is not knowable until the producing step has actually run, and a with-field's Kind is not checkable against a provider's declared WithSchema until that provider is resolved. Both happen here, per step, as each wave executes.

Execute invokes the Layer 1 capability interfaces (pkg/shipwright. Builder/Tester/Artifactor/Deployer/Runner) DIRECTLY — never through Plan (the .dagger/ Layer 2 composition type). This is a deliberate architectural decision (design.md D-G): Plan's five-slot chain cannot express needs[]/DAG dependencies, so the workflow engine bypasses it entirely.

Scheduling (design.md D-K): Kahn's waves (internal/workflow/graph) are executed in order — a later wave never starts before every step of the current wave has settled. WITHIN a wave, steps are independent by construction (Kahn only admits a step into a wave once every entry in its needs[] has already completed in an earlier wave, and graph.validateDataReferences rejects any steps.<id>.output reference not also present in needs[]) — so this package now runs them concurrently, bounded by Options.MaxParallel (the "worker pool drops into later" seam design.md D-K named for exactly this). MaxParallel <= 1 — including the zero-value/unset default — preserves the historical behavior exactly: strict, one-at-a-time, manifest-declaration-order execution. See runWave's doc comment for the full concurrency contract: ordering determinism, failFast semantics, and thread-safety argument.

Approval gates (design.md D-M): spec.environments.<name>.approvals is parsed metadata only. This package contains NO blocking, queueing, or "wait for approval" logic anywhere — a manifest declaring approvals executes exactly as if it had none; manifest.ValidateExecutable rejects declared approvals before Execute runs, so this is a defense-in-depth invariant for a direct, unvalidated caller.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Closure

func Closure(g *graph.Graph, stepID string) (*graph.Graph, error)

Closure computes the needs-transitive closure of stepID over g: a new *graph.Graph containing stepID and every step it depends on, directly or transitively, with Waves filtered to preserve g's own topological order and per-wave manifest-declaration order (design.md D-N: "--step <id> retargeted to a manifest step id — executes the needs-transitive closure of <id> in topological order and stops — a subgraph run, computed by reachability over the graph already built").

This function exposes ONLY the underlying reachability computation. Wiring it to the CLI's --step flag is Phase 9's job (design.md D-N); this package deliberately stops at the computed *graph.Graph so Phase 9 can pass it straight to Execute unchanged.

Closure never re-runs cycle detection: g is assumed to already be the product of graph.Build, which never returns a Graph containing a cycle (an acyclic input has no cycle to reintroduce by filtering a subset of its own edges), so the reachability walk below is unconditionally termination-safe.

Types

type Config

type Config struct {
	// Steps is the manifest's ordered step list (declaration order is
	// significant — see this package's doc comment on wave ordering).
	Steps []manifest.Step
	// Graph is the compiled DAG (internal/workflow/graph.Build's output).
	Graph *graph.Graph
	// Registry resolves each step's uses.provider/uses.module to a
	// concrete Layer 1 capability instance (internal/workflow/providers).
	Registry *providers.Registry
	// Source is the workflow's default Directory input (spec.source),
	// bound to a step's Directory-typed input whenever that step
	// declares no explicit "input" field (design.md D-H schema
	// simplification note).
	Source *dagger.Directory
	// Variables resolves "${{ variables.<name> }}" references
	// (spec.variables).
	Variables map[string]string
	// Secrets resolves "${{ secrets.<name> }}" references
	// (spec.secrets), already bound to their *dagger.Secret handles —
	// never a plaintext string (design.md D-L).
	Secrets map[string]*dagger.Secret
	// Predicates is the actual runtime value for each "when" key this
	// run evaluates against (for example {"branch": "main"}). A step
	// declaring "when: {branch: [main]}" executes only when
	// Predicates["branch"] is present in that list (design.md D-L,
	// exact-match evaluation, never an expression).
	Predicates map[string]string
	// Options configures failure strategy, per-step timeout, and
	// per-step retry.
	Options Options
	// Now is the injectable clock seam for StepOutcome.Duration (design.md
	// D3); nil falls back to time.Now.
	Now func() time.Time
}

Config bundles everything Execute needs beyond the compiled Graph and resolved Registry to run a manifest end-to-end. Steps carries the manifest's own step declarations (capability, uses, with, input, when) — deliberately separate from Graph, whose Node type carries only id+needs (internal/workflow/graph's own package doc comment: "everything else about the step belongs to a later phase's concern" — this IS that later phase).

type Diagnostic

type Diagnostic struct {
	Severity DiagnosticSeverity
	Code     string
	Message  string
}

Diagnostic is one supplementary, non-fatal observation attached to a StepOutcome (design.md D5). Code gives tests a stable identity.

type DiagnosticSeverity

type DiagnosticSeverity int

DiagnosticSeverity classifies a Diagnostic's importance. No SeverityError exists: failure is already Err/StatusFailed (design.md D5).

const (
	SeverityInfo DiagnosticSeverity = iota
	SeverityWarning
)

func (DiagnosticSeverity) String

func (s DiagnosticSeverity) String() string

String renders a DiagnosticSeverity's name for logging.

type InvalidInputReferenceError

type InvalidInputReferenceError struct{ StepID string }

InvalidInputReferenceError reports a step's "input" field containing anything other than exactly one "${{ steps.<id>.output }}" reference — input is a Directory-typed field (design.md D-H); a literal string or a variables./secrets. reference can never satisfy that.

func (*InvalidInputReferenceError) Error

type MissingStepOutputError

type MissingStepOutputError struct{ StepID, ReferencedStepID string }

MissingStepOutputError reports a "${{ steps.<id>.output }}" reference to a step whose output is not available — because it has not executed yet (should not happen given Graph.Waves ordering), it failed, or it was skipped (fail-fast or a non-matching "when").

func (*MissingStepOutputError) Error

func (e *MissingStepOutputError) Error() string

type MissingWithFieldError

type MissingWithFieldError struct{ StepID, Field string }

MissingWithFieldError reports a required capability-specific "with" field (for example Artifactor's "ref") absent from a step's resolved with-values at dispatch time.

func (*MissingWithFieldError) Error

func (e *MissingWithFieldError) Error() string

type Options

type Options struct {
	FailFast    bool
	Timeout     time.Duration
	Retries     int
	MaxParallel int
}

Options configures the engine-level execution controls design.md D-K implements now: FailFast, per-step Timeout, bounded per-step Retries, and MaxParallel.

Retries is the TOTAL number of attempts made per step (not the count of additional retries after the first) — a value less than 1 is treated as 1, i.e. no retry. The manifest schema (internal/workflow/manifest) has no per-step or workflow-level "retries" field at all; Retries is therefore engine/caller-supplied only in this work unit. This is a genuine schema gap, not an oversight of this package — flagged for sdd-verify rather than reaching back into WU4's manifest package from here.

Timeout is applied per step via context.WithTimeout, using the single workflow-level spec.execution.timeout value as each step's budget (the manifest schema has no PER-STEP timeout field either — see the same gap note above). A zero Timeout means no deadline is applied. It composes with concurrent execution unchanged: each step still gets its own context.WithTimeout, derived from that wave's (possibly already-canceled, see runWave) context.

MaxParallel is the upper bound on steps of a single wave running at once (recorded from spec.execution.concurrency.maxParallel — see OptionsFromSpec). It is never a cross-wave budget: each wave gets its own fresh bound. A value <= 1 (including the unset zero value) runs a wave strictly sequentially, in manifest-declaration order — identical to this package's pre-concurrency behavior. A value greater than the wave's own step count is harmless — it is clamped to that count, never over-provisioning idle workers.

func OptionsFromSpec

func OptionsFromSpec(spec manifest.ExecutionSpec) (Options, error)

OptionsFromSpec derives Options from a manifest's ExecutionSpec. This is the concrete place spec.execution.concurrency.maxParallel is validated and recorded (tasks.md 8.5, design.md D-K): it is copied into Options.MaxParallel verbatim, and Execute now uses it to bound concurrent steps within a wave (see Options' and runWave's doc comments). It performs NO additional validation of MaxParallel (for example rejecting negative values) — that check belongs to manifest stage 3 (internal/workflow/manifest.ValidateStructure), which already rejects maxParallel < 0 at parse time.

type OutputKindMismatchError

type OutputKindMismatchError struct {
	StepID, ReferencedStepID, Field, Want string
}

OutputKindMismatchError reports a "${{ steps.<id>.output }}" reference whose resolved output kind cannot satisfy the field it is used in — the steps.<id>.output kind check design.md's Migration Sequence table and WU7's apply-progress ("Closing the WU5/WU6 loop") both name as this package's job: a step's output kind is fixed by whichever capability produced it, which is only known once that step has actually run.

func (*OutputKindMismatchError) Error

func (e *OutputKindMismatchError) Error() string

type Result

type Result struct {
	Outcomes []StepOutcome
	// Failures is the ordered list of step ids whose every attempt
	// failed. Empty means the run succeeded end-to-end.
	Failures []string
}

Result is Execute's complete report. Outcomes is ordered by execution (wave order, then manifest-declaration order within a wave) — the same determinism Kahn's algorithm already guarantees for Graph.Waves (internal/workflow/graph, design.md D-K).

func Execute

func Execute(ctx context.Context, cfg Config) (*Result, error)

Execute runs cfg's manifest steps over cfg.Graph's Kahn waves (internal/workflow/graph, design.md D-J), invoking Layer 1 capability interfaces directly through cfg.Registry (never through Plan). See this package's doc comment for the full scheduling, approval, and capability-dispatch contract.

func (*Result) Failed

func (r *Result) Failed() bool

Failed reports whether any step failed.

type StepFailedError

type StepFailedError struct {
	StepID string
	Err    error
}

StepFailedError is Execute's returned error whenever at least one step fails — it names the first failing step id (tasks.md 8.2), wrapping that step's last-attempt error.

func (*StepFailedError) Error

func (e *StepFailedError) Error() string

func (*StepFailedError) Unwrap

func (e *StepFailedError) Unwrap() error

type StepOutcome

type StepOutcome struct {
	StepID   string
	Status   StepStatus
	Attempts int
	Err      error
	// Duration is elapsed time across this step's attempt budget (design.md
	// D3); a skipped step records zero.
	Duration time.Duration
	// Provider/Capability identify this step, sourced from manifest.Step
	// (design.md D4), since dispatch never runs for a skipped or
	// fail-before-dispatch step.
	Provider   providers.Ref
	Capability string
	// Output is populated only for text-returning capabilities on success
	// (design.md D5); empty otherwise.
	Output string
	// Diagnostics carries supplementary, non-fatal observations (design.md
	// D5).
	Diagnostics []Diagnostic
}

StepOutcome records one step's terminal result, in the order it was decided (wave order, manifest-declaration order within a wave).

type StepStatus

type StepStatus int

StepStatus is the terminal state of one step in a Result.

const (
	// StatusSucceeded means the step's capability method returned no
	// error within its configured attempt budget.
	StatusSucceeded StepStatus = iota
	// StatusFailed means every configured attempt returned an error.
	StatusFailed
	// StatusSkipped means the step's "when" predicate did not match, or
	// fail-fast stopped scheduling before this step's wave ran.
	StatusSkipped
)

func (StepStatus) String

func (s StepStatus) String() string

String renders a StepStatus's name for diagnostics.

type StepTimeoutError

type StepTimeoutError struct {
	StepID  string
	Timeout time.Duration
}

StepTimeoutError reports a step whose execution exceeded Options.Timeout (tasks.md 8.3) — context.WithTimeout fired and the step was canceled, never left hanging.

func (*StepTimeoutError) Error

func (e *StepTimeoutError) Error() string

type UndeclaredSecretError

type UndeclaredSecretError struct{ StepID, Name string }

UndeclaredSecretError reports a "${{ secrets.<name> }}" reference to a name absent from spec.secrets.

func (*UndeclaredSecretError) Error

func (e *UndeclaredSecretError) Error() string

type UndeclaredVariableError

type UndeclaredVariableError struct{ StepID, Name string }

UndeclaredVariableError reports a "${{ variables.<name> }}" reference to a name absent from spec.variables.

func (*UndeclaredVariableError) Error

func (e *UndeclaredVariableError) Error() string

type UnknownCapabilityError

type UnknownCapabilityError struct{ StepID, Capability string }

UnknownCapabilityError reports a step whose Capability is not one of the five contract types. Defensive only: manifest.ValidateStructure (stage 3) already rejects this before Execute would ever see it when the normal parse→graph→engine pipeline is followed — this package does not assume that happened, mirroring internal/workflow/graph's own "does not assume stage 3 already ran" discipline.

func (*UnknownCapabilityError) Error

func (e *UnknownCapabilityError) Error() string

type UnknownStepError

type UnknownStepError struct{ StepID string }

UnknownStepError reports a Closure request naming a step id absent from g.Nodes.

func (*UnknownStepError) Error

func (e *UnknownStepError) Error() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL