steps

command module
v0.0.0-...-a368ab9 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 23, 2026 License: MIT Imports: 5 Imported by: 0

README

steps

Pipelines where an agent is just another step.

steps runs Concourse-style YAML pipelines — get, task, put — from one Go binary, and adds agent: an LLM with tool calls, sub-agents, and MCP servers, sitting in the plan beside everything else. Every step is content-addressed and cached in SQLite, checked by assert:, and shown in a run transcript with what it cost.

Why

  • An agent is a step. Same inputs:/outputs: as a task, same cache, same transcript. What it can touch is the tool grant: no edit_file means it reviews and cannot change code.
  • A model proposes, a deterministic step disposes. verdicts: route the plan on the model's decision; assert: checks it wrote what it claimed; approval: parks the run for a person. An agent that reports success while writing nothing is caught by the pipeline, not by you.
  • Every step is cached, replayable, priced. An unchanged commit re-runs nothing. steps plan says what would run before it costs anything. --replay --from re-runs one expensive step. The transcript shows spend against its ceiling.

Install

brew tap jtarchie/steps https://github.com/jtarchie/steps && brew install steps   # macOS
go install github.com/jtarchie/steps@latest                                       # any platform with Go 1.26+

Linux tarballs for each arch are on the releases page; the tap ships a cask, which Homebrew on Linux does not install.

Sixty seconds

Three files. Each is the previous one plus one idea, and each runs as written.

1 · A task.

# hello.yml
jobs:
- name: hello
  plan:
  - task: greet
    run: echo "hello from steps"
steps run hello.yml

2 · A resource. git is built in, so there is no resource_types: block to write.

# build.yml
resources:
- name: repo
  type: git
  source: { uri: https://github.com/jtarchie/steps.git, branch: main }

jobs:
- name: build
  plan:
  - get: repo
  - task: compile
    inputs: [repo]          # a step sees only the artifacts it declares
    run: cd repo && go build ./...

Run it twice. The second run fetches nothing and compiles nothing: the commit, the command and the declared inputs hash the same, so every step is replayed from cache.

3 · An agent, gated by things a model cannot talk its way past.

# review.yml
resources:
- name: repo
  type: git
  source: { uri: https://github.com/jtarchie/steps.git, branch: main }

agents:
- name: reviewer
  source: { model: openrouter/qwen/qwen3.7-flash, api_key_env: OPENROUTER_API_KEY }
  system: You review changes. Be terse.
  tools: [read_file, search_files, write_file]   # no edit_file: it reviews, it cannot change code

jobs:
- name: review
  plan:
  - get: repo
  - agent: reviewer
    inputs: [repo]
    outputs: [report]
    messages:
      - Read repo/ and write a one-paragraph risk summary to report/summary.md.
    verdicts:
      - approve: publish            # the decision picks the next step
      - reject: escalate
    assert:
      files: [report/summary.md]    # ...and it must have actually written it
  - task: escalate
    run: echo paging a human
  - task: publish
    inputs: [report]
    run: cat report/summary.md
OPENROUTER_API_KEY=... steps run review.yml

Swap the model for yours: any OpenAI-compatible endpoint, OpenRouter, a gateway (vercel/, requesty/, helicone/), a local server that needs no key (lmstudio/your-model), or a CLI you already have ("@claude/sonnet"). steps validate review.yml tells you before a run whether the key is set and the model resolves.

See it run

steps web is the daemon: it serves the browser UI, holds the pipelines you upload into it, polls every trigger: true resource, and runs what they enqueue.

steps web                                  # http://127.0.0.1:8088
steps pipeline set -c review.yml           # upload a pipeline into it

The run page is the transcript at the top of this file: every step in plan order, blocks folded under a rail, a cached step dimmed and labeled, an agent step expanded into its conversation with every tool call and result, and a spend panel that shows each step's cost against the ceiling it ran under. A run still in flight streams to the page as it happens. See docs/web.md.

Real pipelines

Pipeline What it does
examples/pr-review.yml Adaptive PR review: a planner decides which review dimensions a change needs, one reviewer per dimension runs concurrently, a falsifier challenges every finding, a gatekeeper decides what blocks, a synthesizer writes the review, and a human approves before it posts. PR_REPO=owner/name steps run examples/pr-review.yml --job review
examples/self-build.yml One draft PR per open issue labeled self-build: an opus planner (read-only), a sonnet implementer, an empty-diff gate, an opus reviewer whose verdict either approves or sends the diff back, then commit, push, gh pr create --draft.
examples/release.yml How steps releases itself: a new v* tag on GitHub → the full validation suite as the gate → approval: → goreleaser as a put → download the published archive and check it reports the tag.

Every YAML example in docs/ is also a complete pipeline the test suite extracts and executes, so it runs as shown.

Commands

Command What it does
steps run <pipeline> Run one job once (--job when there is more than one). --resume <id> continues a failed run; --replay <id> --from <step> re-runs one step of one.
steps validate <pipeline> Check the file, and that this machine can run it: model names, api_key_env:, MCP binaries. --live probes the models too.
steps plan <pipeline> Show which steps a run would execute and which are cached.
steps test <pipeline> Run every job and check the assert: directives.
steps runs -p <name> What past runs recorded (steps, queue, cost, where for the other views).
steps web The daemon: browser UI, trigger polling, the run queue (docs).
steps pipeline set -c <pipeline> Upload a pipeline to a daemon — the only way a served pipeline changes (list, get, pause, unpause, rename, destroy for the rest).
steps approvals / steps questions Answer an approval: or an ask_user question a run is parked on.
steps mcp list|tools|login List, inspect, or authorize mcp_servers: entries.
steps docs [page] Read the docs in the terminal.

Exit codes: 0 success, 1 a step failed, 2 the pipeline could not be run (config or infrastructure), 130 interrupted.

Learn more

  • docs/ — the reference, one page per feature. Start with resources, then control flow or agents.
  • steps.schema.json — JSON Schema for the pipeline format. Put # yaml-language-server: $schema=./steps.schema.json on the first line of a pipeline for completion and inline errors in your editor.
  • CLAUDE.md — architecture, build constraints, and contribution notes for anyone (human or agent) changing this codebase.

Build from source

go build -v     # Go 1.26+
task            # the whole validation sequence: fmt, lint, test (-race), build, vuln

Documentation

Overview

Package main is the steps entrypoint. It holds nothing but the process boundary — hand the command line to internal/cli, turn the error it returns into an exit code — so that every command, and the end-to-end suite that drives them, lives in a package something can import.

Directories

Path Synopsis
Package docs embeds the user-facing documentation (docs/*.md) and extracts its fenced ```yaml blocks, which are the repo's tested example corpus: docs_test.go (root package) schema-validates and executes them, so a doc example cannot drift from what the runner actually accepts.
Package docs embeds the user-facing documentation (docs/*.md) and extracts its fenced ```yaml blocks, which are the repo's tested example corpus: docs_test.go (root package) schema-validates and executes them, so a doc example cannot drift from what the runner actually accepts.
internal
agent
Package agent runs an agent step's LLM tool-calling conversation: it resolves the agent's model/connection, compiles the step's granted tools, drives the request/tool-execute/append loop (see runAgentConversation), and enforces required: true tools via the provider's tool_choice.
Package agent runs an agent step's LLM tool-calling conversation: it resolves the agent's model/connection, compiles the step's granted tools, drives the request/tool-execute/append loop (see runAgentConversation), and enforces required: true tools via the provider's tool_choice.
blobstore
Package blobstore is a content-addressed store for artifact trees on S3.
Package blobstore is a content-addressed store for artifact trees on S3.
cli
Package cli implements steps' command-line grammar and every command behind it: check discovers resource versions, get fetches one via a rendered shell command, and task runs a plan step's command.
Package cli implements steps' command-line grammar and every command behind it: check discovers resource versions, get fetches one via a rendered shell command, and task runs a plan step's command.
compress
Package compress is the one opinion about zstd this repo holds.
Package compress is the one opinion about zstd this repo holds.
config
Package config parses and resolves a Concourse-style pipeline YAML file (resource_types/resources/jobs) and the config-merge logic (task and agent-invocation resolution) that both plan-time hashing and run-time execution share.
Package config parses and resolves a Concourse-style pipeline YAML file (resource_types/resources/jobs) and the config-merge logic (task and agent-invocation resolution) that both plan-time hashing and run-time execution share.
dockerapi
Package dockerapi talks to a docker engine, and is the only package in this repo that holds an engine client.
Package dockerapi talks to a docker engine, and is the only package in this repo that holds an engine client.
events
Package events is the run-event bus: a stdlib-only leaf that carries what a run is doing, as it does it, from the packages executing a job to whatever is watching.
Package events is the run-event bus: a stdlib-only leaf that carries what a run is doing, as it does it, from the packages executing a job to whatever is watching.
exprlang
Package exprlang evaluates the expression form of a resource type's check/in/out — the JSON-over-HTTP alternative to writing those three as shell commands.
Package exprlang evaluates the expression form of a resource type's check/in/out — the JSON-over-HTTP alternative to writing those three as shell commands.
mcp
Package mcp is the shared MCP (Model Context Protocol) client layer: it connects to a configured mcp_servers: entry (config.MCPServer) over either Streamable HTTP (endpoint:) or a local subprocess speaking newline- delimited JSON on stdin/stdout (command:, see stdio.go), lists its tools, and calls one.
Package mcp is the shared MCP (Model Context Protocol) client layer: it connects to a configured mcp_servers: entry (config.MCPServer) over either Streamable HTTP (endpoint:) or a local subprocess speaking newline- delimited JSON on stdin/stdout (command:, see stdio.go), lists its tools, and calls one.
merkle
Package merkle plans a job's steps into content-addressed chains and computes the hashes used to skip already-succeeded work.
Package merkle plans a job's steps into content-addressed chains and computes the hashes used to skip already-succeeded work.
outcome
Package outcome classifies a step's or job's error into the categories the hook system dispatches on: a task-level failure (nonzero exit / red verdict / a required tool that never succeeded), an infrastructure error, or an abort (the job's context was canceled).
Package outcome classifies a step's or job's error into the categories the hook system dispatches on: a task-level failure (nonzero exit / red verdict / a required tool that never succeeded), an infrastructure error, or an abort (the job's context was canceled).
pipeline
Package pipeline orchestrates a job's plan: resolving/fetching get steps, running task/put/agent steps in order, and recording each step's outcome so later runs can skip unchanged work (see internal/merkle).
Package pipeline orchestrates a job's plan: resolving/fetching get steps, running task/put/agent steps in order, and recording each step's outcome so later runs can skip unchanged work (see internal/merkle).
resource
Package resource runs a resource type's check/in/out shell commands and selects among the versions a check returns.
Package resource runs a resource type's check/in/out shell commands and selects among the versions a check returns.
retry
Package retry provides a linear-backoff retry loop.
Package retry provides a linear-backoff retry loop.
shell
Package shell runs pipeline-defined commands via `sh -c`, either on the host (HostRunner) or inside a container (DockerRunner, see docker.go).
Package shell runs pipeline-defined commands via `sh -c`, either on the host (HostRunner) or inside a container (DockerRunner, see docker.go).
shim
Package shim is the remote half of a tagged step.
Package shim is the remote half of a tagged step.
store
Package store is the contract steps holds its state database to: the row types every driver returns, the errors every driver raises, and the Store interface every driver implements.
Package store is the contract steps holds its state database to: the row types every driver returns, the errors every driver raises, and the Store interface every driver implements.
store/sqlite
Package sqlite is the sqlite driver for the state database: the schema, the SQL and the connection pragmas behind internal/store's contract.
Package sqlite is the sqlite driver for the state database: the schema, the SQL and the connection pragmas behind internal/store's contract.
store/storetest
Package storetest is the store's conformance suite: the behavior tests written against internal/store's contract rather than against any one driver, so a second driver is proven by the same tests the first one passes.
Package storetest is the store's conformance suite: the behavior tests written against internal/store's contract rather than against any one driver, so a second driver is proven by the same tests the first one passes.
template
Package template renders Go templates against source/version data.
Package template renders Go templates against source/version data.
trigger
Package trigger polls resources named by a get step's trigger: true and runs every job affected by a version change — the cross-job counterpart to internal/pipeline's single-job orchestration.
Package trigger polls resources named by a get step's trigger: true and runs every job affected by a version change — the cross-job counterpart to internal/pipeline's single-job orchestration.
venue
Package venue runs a step's commands somewhere other than this machine.
Package venue runs a step's commands somewhere other than this machine.
venue/iapdial
Package iapdial speaks the Cloud IAP TCP-forwarding relay protocol: a websocket to Google's tunnel relay carrying framed bytes, which is a byte pipe to a TCP port on a Compute Engine instance's VPC interface.
Package iapdial speaks the Cloud IAP TCP-forwarding relay protocol: a websocket to Google's tunnel relay carrying framed bytes, which is a byte pipe to a TCP port on a Compute Engine instance's VPC interface.
venue/ssmdial
Package ssmdial speaks the AWS Systems Manager session data-channel protocol: a websocket to the SSM messaging service carrying framed agent messages, which for a port-forwarding session is a byte pipe to a port on the managed node.
Package ssmdial speaks the AWS Systems Manager session data-channel protocol: a websocket to the SSM messaging service carrying framed agent messages, which for a port-forwarding session is a byte pipe to a port on the managed node.
web
Package web serves the pipeline UI: a read-and-operate view of what the runner has done and is doing, over the same sqlite store the CLI writes.
Package web serves the pipeline UI: a read-and-operate view of what the runner has done and is doing, over the same sqlite store the CLI writes.
webhook
Package webhook turns one delivery into a version; expressions reach it compiled, so one can build a string but never decide that a signature is valid.
Package webhook turns one delivery into a version; expressions reach it compiled, so one can build a string but never decide that a signature is valid.
wire
Package wire carries a step's work across a venue: the framed protocol an orchestrator and a pushed shim speak, and the codec that moves a step's directory tree between them.
Package wire carries a step's work across a venue: the framed protocol an orchestrator and a pushed shim speak, and the codec that moves a step's directory tree between them.
workspace
Package workspace materializes the per-step/per-build filesystem views a job's get/task/put/agent steps run against — either the default shared (single-mutable-directory) implementation, or, when a pipeline opts into workspace: isolation, per-step copy/btrfs-backed directories built from each step's declared inputs/outputs.
Package workspace materializes the per-step/per-build filesystem views a job's get/task/put/agent steps run against — either the default shared (single-mutable-directory) implementation, or, when a pipeline opts into workspace: isolation, per-step copy/btrfs-backed directories built from each step's declared inputs/outputs.
tools
coverdiff command
Command coverdiff reports changed lines the tests never executed.
Command coverdiff reports changed lines the tests never executed.
kindswitch command
Package kindswitch reports tagless kind dispatch that silently ignores a step kind.
Package kindswitch reports tagless kind dispatch that silently ignores a step kind.
mutants command
Command mutants is the bookkeeping around a gremlins sweep: which package is stalest, and which files a scoped run may skip.
Command mutants is the bookkeeping around a gremlins sweep: which package is stalest, and which files a scoped run may skip.
sqlscope
Package sqlscope finds SQL statements that touch a pipeline-scoped table without naming pipeline_id.
Package sqlscope finds SQL statements that touch a pipeline-scoped table without naming pipeline_id.
stamp command
Command stamp ties a commit to the tree `task` last passed on: the sequence takes eight minutes, so a hook that RAN it would be bypassed by the second day, while one that only COMPARES tree hashes costs nothing and proves more — the tree tested is the tree pushed.
Command stamp ties a commit to the tree `task` last passed on: the sequence takes eight minutes, so a hook that RAN it would be bypassed by the second day, while one that only COMPARES tree hashes costs nothing and proves more — the tree tested is the tree pushed.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL