README
¶
steps
Pipelines where an agent is just another step.
steps runs Concourse-style YAML pipelines — get, task, put — from one Go binary, and adds agent: an LLM with tool calls, sub-agents, and MCP servers, sitting in the plan beside everything else. Every step is content-addressed and cached in SQLite, checked by assert:, and shown in a run transcript with what it cost.
Status: v0.1.0, the first tagged release. macOS and Linux, amd64 and arm64. The pipeline format may still change between releases. A state database written by v0.1.0 or later is upgraded in place by the next steps run or steps web (SQLite keeps a backup copy beside the file).
Why
- An agent is a step. Same
inputs:/outputs:as a task, same cache, same transcript. What it can touch is the tool grant: noedit_filemeans it reviews and cannot change code. - A model proposes, a deterministic step disposes.
verdicts:route the plan on the model's decision;assert:checks it wrote what it claimed;approval:parks the run for a person. An agent that reports success while writing nothing is caught by the pipeline, not by you. - Every step is cached, replayable, priced. An unchanged commit re-runs nothing.
steps plansays what would run before it costs anything.--replay --fromre-runs one expensive step. The transcript shows spend against its ceiling.
Install
Homebrew (macOS). The cask lives in this repo's Casks/, so the repo is the tap:
brew tap jtarchie/steps https://github.com/jtarchie/steps
brew install --cask steps
Release archives: steps_<version>_<os>_<arch>.tar.gz for darwin/linux x amd64/arm64, plus checksums.txt, on the releases page. The macOS binary is unsigned; the cask strips the quarantine attribute, a hand-extracted one needs xattr -d com.apple.quarantine steps.
curl -fsSL https://github.com/jtarchie/steps/releases/download/v0.1.0/steps_0.1.0_linux_amd64.tar.gz | tar -xz steps
From source, Go 1.27+:
go install github.com/jtarchie/steps@latest # steps --version prints the module version it installed
Runtime needs: git for the built-in git resource, Docker only for steps that name an image:, and an API key (or a local model server) only for agent steps. Everything else runs on the host shell.
Sixty seconds
Three files. Each is the previous one plus one idea, and each runs as written.
1 · A task.
# hello.yml
jobs:
- name: hello
plan:
- task: greet
run: echo "hello from steps"
steps run hello.yml
2 · A resource. git is built in, so there is no resource_types: block to write.
# build.yml
resources:
- name: repo
type: git
source: { uri: https://github.com/jtarchie/steps.git, branch: main }
jobs:
- name: build
plan:
- get: repo
- task: compile
inputs: [repo] # a step sees only the artifacts it declares
run: cd repo && go build ./...
Run it twice. The second run fetches nothing and compiles nothing: the commit, the command and the declared inputs hash the same, so every step is replayed from cache.
3 · An agent, gated by things a model cannot talk its way past.
# review.yml
resources:
- name: repo
type: git
source: { uri: https://github.com/jtarchie/steps.git, branch: main }
agents:
- name: reviewer
source: { model: openrouter/qwen/qwen3.7-flash, api_key_env: OPENROUTER_API_KEY }
system: You review changes. Be terse.
tools: [read_file, search_files, write_file] # no edit_file: it reviews, it cannot change code
jobs:
- name: review
plan:
- get: repo
- agent: reviewer
inputs: [repo]
outputs: [report]
messages:
- Read repo/ and write a one-paragraph risk summary to report/summary.md.
verdicts:
- approve: publish # the decision picks the next step
- reject: escalate
assert:
files: [report/summary.md] # ...and it must have actually written it
- task: escalate
run: echo paging a human
- task: publish
inputs: [report]
run: cat report/summary.md
OPENROUTER_API_KEY=... steps run review.yml
Swap the model for yours: any OpenAI-compatible endpoint, OpenRouter, a gateway (vercel/, requesty/, helicone/), a local server that needs no key (lmstudio/your-model), or a CLI you already have ("@claude/sonnet"). steps validate review.yml tells you before a run whether the key is set and the model resolves.
See it run
steps web is the daemon: it serves the browser UI, holds the pipelines you upload into it, polls every trigger: true resource, and runs what they enqueue.
steps web # http://127.0.0.1:8088
steps pipeline set -c review.yml # upload a pipeline into it
The run page is the transcript at the top of this file: every step in plan order, blocks folded under a rail, a cached step dimmed and labeled, an agent step expanded into its conversation with every tool call and result, and a spend panel that shows each step's cost against the ceiling it ran under. A run still in flight streams to the page as it happens. See docs/web.md.
Real pipelines
| Pipeline | What it does |
|---|---|
examples/pr-review.yml |
Adaptive PR review: a planner decides which review dimensions a change needs, one reviewer per dimension runs concurrently, a falsifier challenges every finding, a gatekeeper decides what blocks, a synthesizer writes the review, and a human approves before it posts. PR_REPO=owner/name steps run examples/pr-review.yml --job review |
examples/self-build.yml |
One draft PR per open issue labeled self-build: an opus planner (read-only), an opus implementer, an empty-diff gate, an opus reviewer whose verdict either approves or sends the diff back, then commit, push, gh pr create --draft. |
examples/release.yml |
How steps releases itself: a new v* tag on GitHub → the full validation suite as the gate → approval: → goreleaser as a put → download the published archive and check it reports the tag. |
Every YAML example in docs/ is also a complete pipeline the test suite extracts and executes, so it runs as shown.
Commands
| Command | What it does |
|---|---|
steps run <pipeline> |
Run one job once (--job when there is more than one). --resume <id> continues a failed run; --replay <id> --from <step> re-runs one step of one. |
steps validate <pipeline> |
Check the file, and that this machine can run it: model names, api_key_env:, MCP binaries. --live probes the models too. |
steps plan <pipeline> |
Show which steps a run would execute and which are cached. |
steps test <pipeline> |
Run every job and check the assert: directives. |
steps runs -p <name> |
What past runs recorded (steps, queue, cost, where for the other views). |
steps web |
The daemon: browser UI, trigger polling, the run queue (docs). |
steps pipeline set -c <pipeline> |
Upload a pipeline to a daemon — the only way a served pipeline changes (list, get, pause, unpause, rename, destroy for the rest). |
steps approvals / steps questions |
Answer an approval: or an ask_user question a run is parked on. |
steps mcp list|tools|login |
List, inspect, or authorize mcp_servers: entries. |
steps docs [page] |
Read the docs in the terminal. |
Exit codes: 0 success, 1 a step failed, 2 the pipeline could not be run (config or infrastructure), 130 interrupted.
Learn more
docs/— the reference, one page per feature. Start with resources, then control flow or agents.steps.schema.json— JSON Schema for the pipeline format. Put# yaml-language-server: $schema=./steps.schema.jsonon the first line of a pipeline for completion and inline errors in your editor.CLAUDE.md— architecture, build constraints, and contribution notes for anyone (human or agent) changing this codebase.
Build from source
go build -v # Go 1.27+
task # the whole validation sequence: fmt, lint, test (-race), build, vuln
Tests against real AWS, GCP and DigitalOcean machines are opt-in. hack/README.md has the commands to set up credentials and bring the fixtures up and down.
Documentation
¶
Overview ¶
Package main is the steps entrypoint. It holds nothing but the process boundary — hand the command line to internal/cli, turn the error it returns into an exit code — so that every command, and the end-to-end suite that drives them, lives in a package something can import.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package docs embeds the user-facing documentation (docs/*.md) and extracts its fenced ```yaml blocks, which are the repo's tested example corpus: docs_test.go (root package) schema-validates and executes them, so a doc example cannot drift from what the runner actually accepts.
|
Package docs embeds the user-facing documentation (docs/*.md) and extracts its fenced ```yaml blocks, which are the repo's tested example corpus: docs_test.go (root package) schema-validates and executes them, so a doc example cannot drift from what the runner actually accepts. |
|
internal
|
|
|
actor
Package actor names who or what caused something steps records: a person, or the automatic thing that stood in for one.
|
Package actor names who or what caused something steps records: a person, or the automatic thing that stood in for one. |
|
agent
Package agent runs an agent step's LLM tool-calling conversation: it resolves the agent's model/connection, compiles the step's granted tools, drives the request/tool-execute/append loop (see runAgentConversation), and enforces required: true tools via the provider's tool_choice.
|
Package agent runs an agent step's LLM tool-calling conversation: it resolves the agent's model/connection, compiles the step's granted tools, drives the request/tool-execute/append loop (see runAgentConversation), and enforces required: true tools via the provider's tool_choice. |
|
agent/llmclient
Package llmclient builds the model.LLM an agent step talks to and the HTTP transports under it: retries, tool-call argument repair, OpenRouter's caching levers, and the choice of which fallback source to try next.
|
Package llmclient builds the model.LLM an agent step talks to and the HTTP transports under it: retries, tool-call argument repair, OpenRouter's caching levers, and the choice of which fallback source to try next. |
|
agent/tree
Package tree is how a file tool reaches the step's directory: directly, or through the image's own userland in the step's container.
|
Package tree is how a file tool reaches the step's directory: directly, or through the image's own userland in the step's container. |
|
blobstore
Package blobstore is a content-addressed store for artifact trees on S3.
|
Package blobstore is a content-addressed store for artifact trees on S3. |
|
cli
Package cli implements steps' command-line grammar and every command behind it.
|
Package cli implements steps' command-line grammar and every command behind it. |
|
compress
Package compress is the one opinion about zstd this repo holds.
|
Package compress is the one opinion about zstd this repo holds. |
|
config
Package config parses and resolves a Concourse-style pipeline YAML file (resource_types/resources/jobs) and the config-merge logic (task and agent-invocation resolution) that both plan-time hashing and run-time execution share.
|
Package config parses and resolves a Concourse-style pipeline YAML file (resource_types/resources/jobs) and the config-merge logic (task and agent-invocation resolution) that both plan-time hashing and run-time execution share. |
|
dockerapi
Package dockerapi talks to a docker engine, and is the only package in this repo that holds an engine client.
|
Package dockerapi talks to a docker engine, and is the only package in this repo that holds an engine client. |
|
events
Package events is the run-event bus: a stdlib-only leaf that carries what a run is doing, as it does it, from the packages executing a job to whatever is watching.
|
Package events is the run-event bus: a stdlib-only leaf that carries what a run is doing, as it does it, from the packages executing a job to whatever is watching. |
|
exprlang
Package exprlang evaluates the expression form of a resource type's check/in/out — the JSON-over-HTTP alternative to writing those three as shell commands.
|
Package exprlang evaluates the expression form of a resource type's check/in/out — the JSON-over-HTTP alternative to writing those three as shell commands. |
|
mcp
Package mcp is the shared MCP (Model Context Protocol) client layer: it connects to a configured mcp_servers: entry (config.MCPServer) over either Streamable HTTP (endpoint:) or a local subprocess speaking newline- delimited JSON on stdin/stdout (command:, see stdio.go), lists its tools, and calls one.
|
Package mcp is the shared MCP (Model Context Protocol) client layer: it connects to a configured mcp_servers: entry (config.MCPServer) over either Streamable HTTP (endpoint:) or a local subprocess speaking newline- delimited JSON on stdin/stdout (command:, see stdio.go), lists its tools, and calls one. |
|
merkle
Package merkle plans a job's steps into content-addressed chains and computes the hashes used to skip already-succeeded work.
|
Package merkle plans a job's steps into content-addressed chains and computes the hashes used to skip already-succeeded work. |
|
outcome
Package outcome classifies a step's or job's error into the categories the hook system dispatches on: a task-level failure (nonzero exit / red verdict / a required tool that never succeeded), an infrastructure error, or an abort (the job's context was canceled).
|
Package outcome classifies a step's or job's error into the categories the hook system dispatches on: a task-level failure (nonzero exit / red verdict / a required tool that never succeeded), an infrastructure error, or an abort (the job's context was canceled). |
|
pipeline
Package pipeline orchestrates a job's plan: resolving/fetching get steps, running task/put/agent steps in order, and recording each step's outcome so later runs can skip unchanged work (see internal/merkle).
|
Package pipeline orchestrates a job's plan: resolving/fetching get steps, running task/put/agent steps in order, and recording each step's outcome so later runs can skip unchanged work (see internal/merkle). |
|
redact
Package redact masks credentials in text that leaves the process.
|
Package redact masks credentials in text that leaves the process. |
|
resource
Package resource runs a resource type's check/in/out shell commands and selects among the versions a check returns.
|
Package resource runs a resource type's check/in/out shell commands and selects among the versions a check returns. |
|
retry
Package retry provides a linear-backoff retry loop.
|
Package retry provides a linear-backoff retry loop. |
|
runview
Package runview folds a run's recorded events into the step tree a reader sees.
|
Package runview folds a run's recorded events into the step tree a reader sees. |
|
scheduler
Package scheduler is the daemon's job-scheduling engine: it drains each held pipeline's durable queue in this process — workers, the --max-runs slots, the circuit breaker, pause, abort, the shutdown grace — and holds the workspace provider each pipeline's runs materialize in.
|
Package scheduler is the daemon's job-scheduling engine: it drains each held pipeline's durable queue in this process — workers, the --max-runs slots, the circuit breaker, pause, abort, the shutdown grace — and holds the workspace provider each pipeline's runs materialize in. |
|
sealbox
Package sealbox encrypts the secrets steps keeps in its state database, so a copy of the database without the key is worth nothing.
|
Package sealbox encrypts the secrets steps keeps in its state database, so a copy of the database without the key is worth nothing. |
|
shell
Package shell runs pipeline-defined commands via `sh -c`, either on the host (HostRunner) or inside a container (DockerRunner, see docker.go).
|
Package shell runs pipeline-defined commands via `sh -c`, either on the host (HostRunner) or inside a container (DockerRunner, see docker.go). |
|
store
Package store is the contract steps holds its state database to: the row types every driver returns, the errors every driver raises, and the Store interface every driver implements.
|
Package store is the contract steps holds its state database to: the row types every driver returns, the errors every driver raises, and the Store interface every driver implements. |
|
store/postgres
Package postgres is the Postgres driver for the state database: its own schema and its own SQL behind internal/store's contract, proven by the same conformance suite as the sqlite driver.
|
Package postgres is the Postgres driver for the state database: its own schema and its own SQL behind internal/store's contract, proven by the same conformance suite as the sqlite driver. |
|
store/sqlite
Package sqlite is the sqlite driver for the state database: the schema, the SQL and the connection pragmas behind internal/store's contract.
|
Package sqlite is the sqlite driver for the state database: the schema, the SQL and the connection pragmas behind internal/store's contract. |
|
store/storetest
Package storetest is the store's conformance suite: the behavior tests written against internal/store's contract rather than against any one driver, so a second driver is proven by the same tests the first one passes.
|
Package storetest is the store's conformance suite: the behavior tests written against internal/store's contract rather than against any one driver, so a second driver is proven by the same tests the first one passes. |
|
telemetry
Package telemetry is the one place steps meets the OpenTelemetry SDK, off unless OTEL_* asks: the docker and GCE clients already trace against the global provider, so installing one is all their calls need.
|
Package telemetry is the one place steps meets the OpenTelemetry SDK, off unless OTEL_* asks: the docker and GCE clients already trace against the global provider, so installing one is all their calls need. |
|
template
Package template renders Go templates against source/version data.
|
Package template renders Go templates against source/version data. |
|
testdocker
Package testdocker is a test's docker half of the setup: whether a daemon is reachable, and a directory it can bind-mount.
|
Package testdocker is a test's docker half of the setup: whether a daemon is reachable, and a directory it can bind-mount. |
|
testpostgres
Package testpostgres runs a real Postgres for tests from embedded-postgres' downloaded binaries, so the Postgres-backed tests need no docker daemon.
|
Package testpostgres runs a real Postgres for tests from embedded-postgres' downloaded binaries, so the Postgres-backed tests need no docker daemon. |
|
testsshd
Package testsshd is an in-process sshd so worker venues are testable without a network, a credential or a second machine; only OpenSSH's own config surface is missing.
|
Package testsshd is an in-process sshd so worker venues are testable without a network, a credential or a second machine; only OpenSSH's own config surface is missing. |
|
treedigest
Package treedigest lets a worker with no steps code (busybox only) name a tree exactly as the orchestrator does, so outputs can stay where they were made (steps#206).
|
Package treedigest lets a worker with no steps code (busybox only) name a tree exactly as the orchestrator does, so outputs can stay where they were made (steps#206). |
|
trigger
Package trigger polls resources named by a get step's trigger: true and runs every job affected by a version change — the cross-job counterpart to internal/pipeline's single-job orchestration.
|
Package trigger polls resources named by a get step's trigger: true and runs every job affected by a version change — the cross-job counterpart to internal/pipeline's single-job orchestration. |
|
usage
Package usage is the spend domain: what an agent step reported, the accumulator a job rolls those reports into, and the ceilings measured against it.
|
Package usage is the spend domain: what an agent step reported, the accumulator a job rolls those reports into, and the ceilings measured against it. |
|
venue
Package venue runs a step's commands somewhere other than this machine.
|
Package venue runs a step's commands somewhere other than this machine. |
|
venue/iapdial
Package iapdial speaks the Cloud IAP TCP-forwarding relay protocol: a websocket to Google's tunnel relay carrying framed bytes, which is a byte pipe to a TCP port on a Compute Engine instance's VPC interface.
|
Package iapdial speaks the Cloud IAP TCP-forwarding relay protocol: a websocket to Google's tunnel relay carrying framed bytes, which is a byte pipe to a TCP port on a Compute Engine instance's VPC interface. |
|
venue/ssmdial
Package ssmdial speaks the AWS Systems Manager session data-channel protocol: a websocket to the SSM messaging service carrying framed agent messages, which for a port-forwarding session is a byte pipe to a port on the managed node.
|
Package ssmdial speaks the AWS Systems Manager session data-channel protocol: a websocket to the SSM messaging service carrying framed agent messages, which for a port-forwarding session is a byte pipe to a port on the managed node. |
|
web
Package web serves the pipeline UI: a read-and-operate view of what the runner has done and is doing, over the same sqlite store the CLI writes.
|
Package web serves the pipeline UI: a read-and-operate view of what the runner has done and is doing, over the same sqlite store the CLI writes. |
|
webhook
Package webhook turns one delivery into a version; expressions reach it compiled, so one can build a string but never decide that a signature is valid.
|
Package webhook turns one delivery into a version; expressions reach it compiled, so one can build a string but never decide that a signature is valid. |
|
wire
Package wire is the tree codec: a step's directory as a tar stream, packed and unpacked the way every transfer of one agrees on, symlinks kept and never followed, and nothing written outside the tree on the way in.
|
Package wire is the tree codec: a step's directory as a tar stream, packed and unpacked the way every transfer of one agrees on, symlinks kept and never followed, and nothing written outside the tree on the way in. |
|
workspace
Package workspace materializes the per-step/per-build filesystem views a job's steps run against: per-step directories, copied (copy-on-write where the filesystem offers it), built from each step's declared inputs/outputs.
|
Package workspace materializes the per-step/per-build filesystem views a job's steps run against: per-step directories, copied (copy-on-write where the filesystem offers it), built from each step's declared inputs/outputs. |
|
tools
|
|
|
coverattr
command
Command coverattr says which e2e test is the only one executing which statements: every package's tests are profiled once, then each e2e test in a process of its own, and the two are compared block by block.
|
Command coverattr says which e2e test is the only one executing which statements: every package's tests are profiled once, then each e2e test in a process of its own, and the two are compared block by block. |
|
coverdiff
command
Command coverdiff reports changed lines the tests never executed.
|
Command coverdiff reports changed lines the tests never executed. |
|
kindswitch
command
Package kindswitch keeps step-kind dispatch where golangci-lint's `exhaustive` can check it.
|
Package kindswitch keeps step-kind dispatch where golangci-lint's `exhaustive` can check it. |
|
mutants
command
Command mutants measures mutation testing (sweep, all), keeps the ledger of it (stale, summary, record), and judges e2e (judge).
|
Command mutants measures mutation testing (sweep, all), keeps the ledger of it (stale, summary, record), and judges e2e (judge). |
|
sqlscope
Package sqlscope finds SQL statements that touch a pipeline-scoped table without naming pipeline_id, or a run-scoped one (run_id REFERENCES runs, no pipeline_id of its own) without naming pipeline_id or reaching runs.
|
Package sqlscope finds SQL statements that touch a pipeline-scoped table without naming pipeline_id, or a run-scoped one (run_id REFERENCES runs, no pipeline_id of its own) without naming pipeline_id or reaching runs. |
|
stamp
command
Command stamp ties a commit to the tree `task` last passed on: the sequence takes eight minutes, so a hook that RAN it would be bypassed by the second day, while one that only COMPARES tree hashes costs nothing and proves more — the tree tested is the tree pushed.
|
Command stamp ties a commit to the tree `task` last passed on: the sequence takes eight minutes, so a hook that RAN it would be bypassed by the second day, while one that only COMPARES tree hashes costs nothing and proves more — the tree tested is the tree pushed. |
|
testshard
command
Command testshard runs one test package as several processes of a single compiled test binary, so a package of serial tests finishes in the time of its slowest shard; each process keeps its own globals, which is what t.Setenv, t.Chdir, a swapped os.Stdout and a daemon stopped by SIGINT to its own pid all need, and what t.Parallel cannot give them.
|
Command testshard runs one test package as several processes of a single compiled test binary, so a package of serial tests finishes in the time of its slowest shard; each process keeps its own globals, which is what t.Setenv, t.Chdir, a swapped os.Stdout and a daemon stopped by SIGINT to its own pid all need, and what t.Parallel cannot give them. |
Click to show internal directories.
Click to hide internal directories.