Documentation
¶
Overview ¶
Package engine implements the wave-scheduling execution engine for Shipwright's declarative workflow layer (design.md D-K, workflow-execution spec). It is the FIRST place in the fixed seven-stage validation pipeline (design.md D-H) where a manifest's compiled graph.Graph (stage 5), its resolved providers.Registry (stage 6), and interpolated with/input values (stage 7) are all in scope simultaneously — see WU7's apply-progress ("Closing the WU5/WU6 loop") for the exact boundary this package closes: a steps.<id>.output reference's Kind is not knowable until the producing step has actually run, and a with-field's Kind is not checkable against a provider's declared WithSchema until that provider is resolved. Both happen here, per step, as each wave executes.
Execute invokes the Layer 1 capability interfaces (pkg/shipwright. Builder/Tester/Artifactor/Deployer/Runner) DIRECTLY — never through Plan (the .dagger/ Layer 2 composition type). This is a deliberate architectural decision (design.md D-G): Plan's five-slot chain cannot express needs[]/DAG dependencies, so the workflow engine bypasses it entirely.
Scheduling (design.md D-K): Kahn's waves (internal/workflow/graph) are executed in order — a later wave never starts before every step of the current wave has settled. WITHIN a wave, steps are independent by construction (Kahn only admits a step into a wave once every entry in its needs[] has already completed in an earlier wave, and graph.validateDataReferences rejects any steps.<id>.output reference not also present in needs[]) — so this package now runs them concurrently, bounded by Options.MaxParallel (the "worker pool drops into later" seam design.md D-K named for exactly this). MaxParallel <= 1 — including the zero-value/unset default — preserves the historical behavior exactly: strict, one-at-a-time, manifest-declaration-order execution. See runWave's doc comment for the full concurrency contract: ordering determinism, failFast semantics, and thread-safety argument.
Approval gates (design.md D-M): spec.environments.<name>.approvals is parsed metadata only. This package contains NO blocking, queueing, or "wait for approval" logic anywhere — a manifest declaring approvals executes exactly as if it had none; manifest.ValidateExecutable rejects declared approvals before Execute runs, so this is a defense-in-depth invariant for a direct, unvalidated caller.
Index ¶
- func Closure(g *graph.Graph, stepID string) (*graph.Graph, error)
- type Config
- type Diagnostic
- type DiagnosticSeverity
- type InvalidInputReferenceError
- type MissingStepOutputError
- type MissingWithFieldError
- type Options
- type OutputKindMismatchError
- type Result
- type StepFailedError
- type StepOutcome
- type StepStatus
- type StepTimeoutError
- type UndeclaredSecretError
- type UndeclaredVariableError
- type UnknownCapabilityError
- type UnknownStepError
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Closure ¶
Closure computes the needs-transitive closure of stepID over g: a new *graph.Graph containing stepID and every step it depends on, directly or transitively, with Waves filtered to preserve g's own topological order and per-wave manifest-declaration order (design.md D-N: "--step <id> retargeted to a manifest step id — executes the needs-transitive closure of <id> in topological order and stops — a subgraph run, computed by reachability over the graph already built").
This function exposes ONLY the underlying reachability computation. Wiring it to the CLI's --step flag is Phase 9's job (design.md D-N); this package deliberately stops at the computed *graph.Graph so Phase 9 can pass it straight to Execute unchanged.
Closure never re-runs cycle detection: g is assumed to already be the product of graph.Build, which never returns a Graph containing a cycle (an acyclic input has no cycle to reintroduce by filtering a subset of its own edges), so the reachability walk below is unconditionally termination-safe.
Types ¶
type Config ¶
type Config struct {
// Steps is the manifest's ordered step list (declaration order is
// significant — see this package's doc comment on wave ordering).
Steps []manifest.Step
// Graph is the compiled DAG (internal/workflow/graph.Build's output).
Graph *graph.Graph
// Registry resolves each step's uses.provider/uses.module to a
// concrete Layer 1 capability instance (internal/workflow/providers).
Registry *providers.Registry
// Source is the workflow's default Directory input (spec.source),
// bound to a step's Directory-typed input whenever that step
// declares no explicit "input" field (design.md D-H schema
// simplification note).
Source *dagger.Directory
// Variables resolves "${{ variables.<name> }}" references
// (spec.variables).
Variables map[string]string
// Secrets resolves "${{ secrets.<name> }}" references
// (spec.secrets), already bound to their *dagger.Secret handles —
// never a plaintext string (design.md D-L).
Secrets map[string]*dagger.Secret
// Predicates is the actual runtime value for each "when" key this
// run evaluates against (for example {"branch": "main"}). A step
// declaring "when: {branch: [main]}" executes only when
// Predicates["branch"] is present in that list (design.md D-L,
// exact-match evaluation, never an expression).
Predicates map[string]string
// Options configures failure strategy, per-step timeout, and
// per-step retry.
Options Options
// Now is the injectable clock seam for StepOutcome.Duration (design.md
// D3); nil falls back to time.Now.
Now func() time.Time
}
Config bundles everything Execute needs beyond the compiled Graph and resolved Registry to run a manifest end-to-end. Steps carries the manifest's own step declarations (capability, uses, with, input, when) — deliberately separate from Graph, whose Node type carries only id+needs (internal/workflow/graph's own package doc comment: "everything else about the step belongs to a later phase's concern" — this IS that later phase).
type Diagnostic ¶
type Diagnostic struct {
Severity DiagnosticSeverity
Code string
Message string
}
Diagnostic is one supplementary, non-fatal observation attached to a StepOutcome (design.md D5). Code gives tests a stable identity.
type DiagnosticSeverity ¶
type DiagnosticSeverity int
DiagnosticSeverity classifies a Diagnostic's importance. No SeverityError exists: failure is already Err/StatusFailed (design.md D5).
const ( SeverityInfo DiagnosticSeverity = iota SeverityWarning )
func (DiagnosticSeverity) String ¶
func (s DiagnosticSeverity) String() string
String renders a DiagnosticSeverity's name for logging.
type InvalidInputReferenceError ¶
type InvalidInputReferenceError struct{ StepID string }
InvalidInputReferenceError reports a step's "input" field containing anything other than exactly one "${{ steps.<id>.output }}" reference — input is a Directory-typed field (design.md D-H); a literal string or a variables./secrets. reference can never satisfy that.
func (*InvalidInputReferenceError) Error ¶
func (e *InvalidInputReferenceError) Error() string
type MissingStepOutputError ¶
type MissingStepOutputError struct{ StepID, ReferencedStepID string }
MissingStepOutputError reports a "${{ steps.<id>.output }}" reference to a step whose output is not available — because it has not executed yet (should not happen given Graph.Waves ordering), it failed, or it was skipped (fail-fast or a non-matching "when").
func (*MissingStepOutputError) Error ¶
func (e *MissingStepOutputError) Error() string
type MissingWithFieldError ¶
type MissingWithFieldError struct{ StepID, Field string }
MissingWithFieldError reports a required capability-specific "with" field (for example Artifactor's "ref") absent from a step's resolved with-values at dispatch time.
func (*MissingWithFieldError) Error ¶
func (e *MissingWithFieldError) Error() string
type Options ¶
Options configures the engine-level execution controls design.md D-K implements now: FailFast, per-step Timeout, bounded per-step Retries, and MaxParallel.
Retries is the TOTAL number of attempts made per step (not the count of additional retries after the first) — a value less than 1 is treated as 1, i.e. no retry. The manifest schema (internal/workflow/manifest) has no per-step or workflow-level "retries" field at all; Retries is therefore engine/caller-supplied only in this work unit. This is a genuine schema gap, not an oversight of this package — flagged for sdd-verify rather than reaching back into WU4's manifest package from here.
Timeout is applied per step via context.WithTimeout, using the single workflow-level spec.execution.timeout value as each step's budget (the manifest schema has no PER-STEP timeout field either — see the same gap note above). A zero Timeout means no deadline is applied. It composes with concurrent execution unchanged: each step still gets its own context.WithTimeout, derived from that wave's (possibly already-canceled, see runWave) context.
MaxParallel is the upper bound on steps of a single wave running at once (recorded from spec.execution.concurrency.maxParallel — see OptionsFromSpec). It is never a cross-wave budget: each wave gets its own fresh bound. A value <= 1 (including the unset zero value) runs a wave strictly sequentially, in manifest-declaration order — identical to this package's pre-concurrency behavior. A value greater than the wave's own step count is harmless — it is clamped to that count, never over-provisioning idle workers.
func OptionsFromSpec ¶
func OptionsFromSpec(spec manifest.ExecutionSpec) (Options, error)
OptionsFromSpec derives Options from a manifest's ExecutionSpec. This is the concrete place spec.execution.concurrency.maxParallel is validated and recorded (tasks.md 8.5, design.md D-K): it is copied into Options.MaxParallel verbatim, and Execute now uses it to bound concurrent steps within a wave (see Options' and runWave's doc comments). It performs NO additional validation of MaxParallel (for example rejecting negative values) — that check belongs to manifest stage 3 (internal/workflow/manifest.ValidateStructure), which already rejects maxParallel < 0 at parse time.
type OutputKindMismatchError ¶
type OutputKindMismatchError struct {
StepID, ReferencedStepID, Field, Want string
}
OutputKindMismatchError reports a "${{ steps.<id>.output }}" reference whose resolved output kind cannot satisfy the field it is used in — the steps.<id>.output kind check design.md's Migration Sequence table and WU7's apply-progress ("Closing the WU5/WU6 loop") both name as this package's job: a step's output kind is fixed by whichever capability produced it, which is only known once that step has actually run.
func (*OutputKindMismatchError) Error ¶
func (e *OutputKindMismatchError) Error() string
type Result ¶
type Result struct {
Outcomes []StepOutcome
// Failures is the ordered list of step ids whose every attempt
// failed. Empty means the run succeeded end-to-end.
Failures []string
}
Result is Execute's complete report. Outcomes is ordered by execution (wave order, then manifest-declaration order within a wave) — the same determinism Kahn's algorithm already guarantees for Graph.Waves (internal/workflow/graph, design.md D-K).
func Execute ¶
Execute runs cfg's manifest steps over cfg.Graph's Kahn waves (internal/workflow/graph, design.md D-J), invoking Layer 1 capability interfaces directly through cfg.Registry (never through Plan). See this package's doc comment for the full scheduling, approval, and capability-dispatch contract.
type StepFailedError ¶
StepFailedError is Execute's returned error whenever at least one step fails — it names the first failing step id (tasks.md 8.2), wrapping that step's last-attempt error.
func (*StepFailedError) Error ¶
func (e *StepFailedError) Error() string
func (*StepFailedError) Unwrap ¶
func (e *StepFailedError) Unwrap() error
type StepOutcome ¶
type StepOutcome struct {
StepID string
Status StepStatus
Attempts int
Err error
// Duration is elapsed time across this step's attempt budget (design.md
// D3); a skipped step records zero.
Duration time.Duration
// Provider/Capability identify this step, sourced from manifest.Step
// (design.md D4), since dispatch never runs for a skipped or
// fail-before-dispatch step.
Provider providers.Ref
Capability string
// Output is populated only for text-returning capabilities on success
// (design.md D5); empty otherwise.
Output string
// Diagnostics carries supplementary, non-fatal observations (design.md
// D5).
Diagnostics []Diagnostic
}
StepOutcome records one step's terminal result, in the order it was decided (wave order, manifest-declaration order within a wave).
type StepStatus ¶
type StepStatus int
StepStatus is the terminal state of one step in a Result.
const ( // StatusSucceeded means the step's capability method returned no // error within its configured attempt budget. StatusSucceeded StepStatus = iota // StatusFailed means every configured attempt returned an error. StatusFailed // StatusSkipped means the step's "when" predicate did not match, or // fail-fast stopped scheduling before this step's wave ran. StatusSkipped )
func (StepStatus) String ¶
func (s StepStatus) String() string
String renders a StepStatus's name for diagnostics.
type StepTimeoutError ¶
StepTimeoutError reports a step whose execution exceeded Options.Timeout (tasks.md 8.3) — context.WithTimeout fired and the step was canceled, never left hanging.
func (*StepTimeoutError) Error ¶
func (e *StepTimeoutError) Error() string
type UndeclaredSecretError ¶
type UndeclaredSecretError struct{ StepID, Name string }
UndeclaredSecretError reports a "${{ secrets.<name> }}" reference to a name absent from spec.secrets.
func (*UndeclaredSecretError) Error ¶
func (e *UndeclaredSecretError) Error() string
type UndeclaredVariableError ¶
type UndeclaredVariableError struct{ StepID, Name string }
UndeclaredVariableError reports a "${{ variables.<name> }}" reference to a name absent from spec.variables.
func (*UndeclaredVariableError) Error ¶
func (e *UndeclaredVariableError) Error() string
type UnknownCapabilityError ¶
type UnknownCapabilityError struct{ StepID, Capability string }
UnknownCapabilityError reports a step whose Capability is not one of the five contract types. Defensive only: manifest.ValidateStructure (stage 3) already rejects this before Execute would ever see it when the normal parse→graph→engine pipeline is followed — this package does not assume that happened, mirroring internal/workflow/graph's own "does not assume stage 3 already ran" discipline.
func (*UnknownCapabilityError) Error ¶
func (e *UnknownCapabilityError) Error() string
type UnknownStepError ¶
type UnknownStepError struct{ StepID string }
UnknownStepError reports a Closure request naming a step id absent from g.Nodes.
func (*UnknownStepError) Error ¶
func (e *UnknownStepError) Error() string