engine

package
v0.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 7, 2026 License: Apache-2.0 Imports: 10 Imported by: 0

Documentation

Overview

Package engine implements the wave-scheduling execution engine for Shipwright's declarative workflow layer (design.md D-K, workflow-execution spec). It is the FIRST place in the fixed seven-stage validation pipeline (design.md D-H) where a manifest's compiled graph.Graph (stage 5), its resolved providers.Registry (stage 6), and interpolated with/input values (stage 7) are all in scope simultaneously — see WU7's apply-progress ("Closing the WU5/WU6 loop") for the exact boundary this package closes: a steps.<id>.output reference's Kind is not knowable until the producing step has actually run, and a with-field's Kind is not checkable against a provider's declared WithSchema until that provider is resolved. Both happen here, per step, as each wave executes.

Execute invokes the Layer 1 capability interfaces (pkg/shipwright. Builder/Tester/Artifactor/Deployer/Runner) DIRECTLY — never through Plan (the .dagger/ Layer 2 composition type). This is a deliberate architectural decision (design.md D-G): Plan's five-slot chain cannot express needs[]/DAG dependencies, so the workflow engine bypasses it entirely.

Scheduling (design.md D-K): Kahn's waves (internal/workflow/graph) are executed in order; within a single wave, steps run SEQUENTIALLY in manifest-declaration order — never map iteration order, never concurrently. Options.MaxParallel is validated/recorded (see OptionsFromSpec) but is NOT used to widen execution in this work unit; sequential execution is a correct schedule for any MaxParallel >= 1. Concurrent widening within a wave is explicitly deferred (design.md D-K). A manifest declaring MaxParallel > 1 is rejected before execution begins by manifest.ValidateExecutable (design.md D1), a validation-layer contract this package does not itself check.

Approval gates (design.md D-M): spec.environments.<name>.approvals is parsed metadata only. This package contains NO blocking, queueing, or "wait for approval" logic anywhere — a manifest declaring approvals executes exactly as if it had none; manifest.ValidateExecutable rejects declared approvals before Execute runs, so this is a defense-in-depth invariant for a direct, unvalidated caller.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Closure

func Closure(g *graph.Graph, stepID string) (*graph.Graph, error)

Closure computes the needs-transitive closure of stepID over g: a new *graph.Graph containing stepID and every step it depends on, directly or transitively, with Waves filtered to preserve g's own topological order and per-wave manifest-declaration order (design.md D-N: "--step <id> retargeted to a manifest step id — executes the needs-transitive closure of <id> in topological order and stops — a subgraph run, computed by reachability over the graph already built").

This function exposes ONLY the underlying reachability computation. Wiring it to the CLI's --step flag is Phase 9's job (design.md D-N); this package deliberately stops at the computed *graph.Graph so Phase 9 can pass it straight to Execute unchanged.

Closure never re-runs cycle detection: g is assumed to already be the product of graph.Build, which never returns a Graph containing a cycle (an acyclic input has no cycle to reintroduce by filtering a subset of its own edges), so the reachability walk below is unconditionally termination-safe.

Types

type Config

type Config struct {
	// Steps is the manifest's ordered step list (declaration order is
	// significant — see this package's doc comment on wave ordering).
	Steps []manifest.Step
	// Graph is the compiled DAG (internal/workflow/graph.Build's output).
	Graph *graph.Graph
	// Registry resolves each step's uses.provider/uses.module to a
	// concrete Layer 1 capability instance (internal/workflow/providers).
	Registry *providers.Registry
	// Source is the workflow's default Directory input (spec.source),
	// bound to a step's Directory-typed input whenever that step
	// declares no explicit "input" field (design.md D-H schema
	// simplification note).
	Source *dagger.Directory
	// Variables resolves "${{ variables.<name> }}" references
	// (spec.variables).
	Variables map[string]string
	// Secrets resolves "${{ secrets.<name> }}" references
	// (spec.secrets), already bound to their *dagger.Secret handles —
	// never a plaintext string (design.md D-L).
	Secrets map[string]*dagger.Secret
	// Predicates is the actual runtime value for each "when" key this
	// run evaluates against (for example {"branch": "main"}). A step
	// declaring "when: {branch: [main]}" executes only when
	// Predicates["branch"] is present in that list (design.md D-L,
	// exact-match evaluation, never an expression).
	Predicates map[string]string
	// Options configures failure strategy, per-step timeout, and
	// per-step retry.
	Options Options
	// Now is the injectable clock seam for StepOutcome.Duration (design.md
	// D3); nil falls back to time.Now.
	Now func() time.Time
}

Config bundles everything Execute needs beyond the compiled Graph and resolved Registry to run a manifest end-to-end. Steps carries the manifest's own step declarations (capability, uses, with, input, when) — deliberately separate from Graph, whose Node type carries only id+needs (internal/workflow/graph's own package doc comment: "everything else about the step belongs to a later phase's concern" — this IS that later phase).

type Diagnostic

type Diagnostic struct {
	Severity DiagnosticSeverity
	Code     string
	Message  string
}

Diagnostic is one supplementary, non-fatal observation attached to a StepOutcome (design.md D5). Code gives tests a stable identity.

type DiagnosticSeverity

type DiagnosticSeverity int

DiagnosticSeverity classifies a Diagnostic's importance. No SeverityError exists: failure is already Err/StatusFailed (design.md D5).

const (
	SeverityInfo DiagnosticSeverity = iota
	SeverityWarning
)

func (DiagnosticSeverity) String

func (s DiagnosticSeverity) String() string

String renders a DiagnosticSeverity's name for logging.

type InvalidInputReferenceError

type InvalidInputReferenceError struct{ StepID string }

InvalidInputReferenceError reports a step's "input" field containing anything other than exactly one "${{ steps.<id>.output }}" reference — input is a Directory-typed field (design.md D-H); a literal string or a variables./secrets. reference can never satisfy that.

func (*InvalidInputReferenceError) Error

type MissingStepOutputError

type MissingStepOutputError struct{ StepID, ReferencedStepID string }

MissingStepOutputError reports a "${{ steps.<id>.output }}" reference to a step whose output is not available — because it has not executed yet (should not happen given Graph.Waves ordering), it failed, or it was skipped (fail-fast or a non-matching "when").

func (*MissingStepOutputError) Error

func (e *MissingStepOutputError) Error() string

type MissingWithFieldError

type MissingWithFieldError struct{ StepID, Field string }

MissingWithFieldError reports a required capability-specific "with" field (for example Artifactor's "ref") absent from a step's resolved with-values at dispatch time.

func (*MissingWithFieldError) Error

func (e *MissingWithFieldError) Error() string

type Options

type Options struct {
	FailFast    bool
	Timeout     time.Duration
	Retries     int
	MaxParallel int
}

Options configures the engine-level execution controls design.md D-K implements now: FailFast, per-step Timeout, and bounded per-step Retries.

Retries is the TOTAL number of attempts made per step (not the count of additional retries after the first) — a value less than 1 is treated as 1, i.e. no retry. The manifest schema (internal/workflow/manifest) has no per-step or workflow-level "retries" field at all; Retries is therefore engine/caller-supplied only in this work unit. This is a genuine schema gap, not an oversight of this package — flagged for sdd-verify rather than reaching back into WU4's manifest package from here.

Timeout is applied per step via context.WithTimeout, using the single workflow-level spec.execution.timeout value as each step's budget (the manifest schema has no PER-STEP timeout field either — see the same gap note above). A zero Timeout means no deadline is applied.

MaxParallel is recorded from spec.execution.concurrency.maxParallel (see OptionsFromSpec) but is NEVER used to widen execution — see this package's doc comment.

func OptionsFromSpec

func OptionsFromSpec(spec manifest.ExecutionSpec) (Options, error)

OptionsFromSpec derives Options from a manifest's ExecutionSpec. This is the concrete place spec.execution.concurrency.maxParallel is "validated and recorded" (tasks.md 8.5, design.md D-K): it is copied into Options.MaxParallel verbatim and never used to widen execution anywhere in this package. It performs NO additional validation of MaxParallel (for example rejecting <= 0) — design.md D-K states that check belongs to manifest stage 3 (internal/workflow/manifest.ValidateStructure). At the time this package was written, ValidateStructure does NOT enforce maxParallel <= 0 — this is a confirmed gap, flagged for sdd-verify rather than fixed here (fixing it would mean reaching back into WU4's package, which is out of this work unit's scope per its own launch instructions).

type OutputKindMismatchError

type OutputKindMismatchError struct {
	StepID, ReferencedStepID, Field, Want string
}

OutputKindMismatchError reports a "${{ steps.<id>.output }}" reference whose resolved output kind cannot satisfy the field it is used in — the steps.<id>.output kind check design.md's Migration Sequence table and WU7's apply-progress ("Closing the WU5/WU6 loop") both name as this package's job: a step's output kind is fixed by whichever capability produced it, which is only known once that step has actually run.

func (*OutputKindMismatchError) Error

func (e *OutputKindMismatchError) Error() string

type Result

type Result struct {
	Outcomes []StepOutcome
	// Failures is the ordered list of step ids whose every attempt
	// failed. Empty means the run succeeded end-to-end.
	Failures []string
}

Result is Execute's complete report. Outcomes is ordered by execution (wave order, then manifest-declaration order within a wave) — the same determinism Kahn's algorithm already guarantees for Graph.Waves (internal/workflow/graph, design.md D-K).

func Execute

func Execute(ctx context.Context, cfg Config) (*Result, error)

Execute runs cfg's manifest steps over cfg.Graph's Kahn waves (internal/workflow/graph, design.md D-J), invoking Layer 1 capability interfaces directly through cfg.Registry (never through Plan). See this package's doc comment for the full scheduling, approval, and capability-dispatch contract.

func (*Result) Failed

func (r *Result) Failed() bool

Failed reports whether any step failed.

type StepFailedError

type StepFailedError struct {
	StepID string
	Err    error
}

StepFailedError is Execute's returned error whenever at least one step fails — it names the first failing step id (tasks.md 8.2), wrapping that step's last-attempt error.

func (*StepFailedError) Error

func (e *StepFailedError) Error() string

func (*StepFailedError) Unwrap

func (e *StepFailedError) Unwrap() error

type StepOutcome

type StepOutcome struct {
	StepID   string
	Status   StepStatus
	Attempts int
	Err      error
	// Duration is elapsed time across this step's attempt budget (design.md
	// D3); a skipped step records zero.
	Duration time.Duration
	// Provider/Capability identify this step, sourced from manifest.Step
	// (design.md D4), since dispatch never runs for a skipped or
	// fail-before-dispatch step.
	Provider   providers.Ref
	Capability string
	// Output is populated only for text-returning capabilities on success
	// (design.md D5); empty otherwise.
	Output string
	// Diagnostics carries supplementary, non-fatal observations (design.md
	// D5).
	Diagnostics []Diagnostic
}

StepOutcome records one step's terminal result, in the order it was decided (wave order, manifest-declaration order within a wave).

type StepStatus

type StepStatus int

StepStatus is the terminal state of one step in a Result.

const (
	// StatusSucceeded means the step's capability method returned no
	// error within its configured attempt budget.
	StatusSucceeded StepStatus = iota
	// StatusFailed means every configured attempt returned an error.
	StatusFailed
	// StatusSkipped means the step's "when" predicate did not match, or
	// fail-fast stopped scheduling before this step's wave ran.
	StatusSkipped
)

func (StepStatus) String

func (s StepStatus) String() string

String renders a StepStatus's name for diagnostics.

type StepTimeoutError

type StepTimeoutError struct {
	StepID  string
	Timeout time.Duration
}

StepTimeoutError reports a step whose execution exceeded Options.Timeout (tasks.md 8.3) — context.WithTimeout fired and the step was canceled, never left hanging.

func (*StepTimeoutError) Error

func (e *StepTimeoutError) Error() string

type UndeclaredSecretError

type UndeclaredSecretError struct{ StepID, Name string }

UndeclaredSecretError reports a "${{ secrets.<name> }}" reference to a name absent from spec.secrets.

func (*UndeclaredSecretError) Error

func (e *UndeclaredSecretError) Error() string

type UndeclaredVariableError

type UndeclaredVariableError struct{ StepID, Name string }

UndeclaredVariableError reports a "${{ variables.<name> }}" reference to a name absent from spec.variables.

func (*UndeclaredVariableError) Error

func (e *UndeclaredVariableError) Error() string

type UnknownCapabilityError

type UnknownCapabilityError struct{ StepID, Capability string }

UnknownCapabilityError reports a step whose Capability is not one of the five contract types. Defensive only: manifest.ValidateStructure (stage 3) already rejects this before Execute would ever see it when the normal parse→graph→engine pipeline is followed — this package does not assume that happened, mirroring internal/workflow/graph's own "does not assume stage 3 already ran" discipline.

func (*UnknownCapabilityError) Error

func (e *UnknownCapabilityError) Error() string

type UnknownStepError

type UnknownStepError struct{ StepID string }

UnknownStepError reports a Closure request naming a step id absent from g.Nodes.

func (*UnknownStepError) Error

func (e *UnknownStepError) Error() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL