vegaload

module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 6, 2026 License: Apache-2.0

README

VegaLoad

VegaLoad is a thin, open-source load testing tool: a single static binary with a scriptable core engine, four protocol drivers (HTTP/1.1, HTTP/2, gRPC, WebSocket), and a self-contained HTML report — no server, no account, no telemetry.

It's also agent-native: vegaload init registers VegaLoad as an MCP server for Claude Code, Cursor, or any other MCP-capable agent host, so an agent can run tests, read results, and explain failures through the same CLI commands a human would type. It is never agent-mandatory — every feature works from a plain terminal with nothing else installed. See AGENTS.md for the design principle behind that split.

Status

Phase 0 (core engine, CLI, protocols, scripting) and Phase 1 (the HTML/JSON report) are done. Phase 2 (the MCP server, skill bundles, and a versioned eval suite for the MCP tools) is also done — this README's walkthrough covers all three. A Harness RT bridge (--move-to-harness) is intentionally out of scope for now.

Install

With Homebrew (macOS and Linux), after the first tagged GitHub release:

brew install vegaload/tap/vegaload
vegaload version

brew tap vegaload/tap followed by brew install vegaload does the same thing. Upgrade with brew upgrade vegaload. Until a tag exists, build from source below.

Or build from source (Go 1.22+):

git clone https://github.com/vegaload/vegaload
cd vegaload
go build -o vegaload ./cmd/vegaload
./vegaload version

go install github.com/vegaload/vegaload/cmd/vegaload@latest also works once the repository is public. Release archives for Linux, macOS and Windows are attached to each GitHub release. A scratch-based Docker image builds from the included Dockerfile (docker build .); it carries nothing but the binary and CA certificates, so Python-scripted scenarios (which shell out to a local python3) need a different base image — see the Dockerfile's comment.

Getting started

Everything below uses examples/sample-app, a small widgets service with injected latency and a ~3% failure rate on creates, served over HTTP/1.1, HTTP/2, WebSocket, and gRPC, built specifically so this walkthrough has something real to point vegaload at. Start it in its own terminal first:

cd examples/sample-app
go run .

Leave it running, and do the rest from a second terminal at the repository root.

1. Run a load test, no scenario file needed

Protocol-direct mode drives a target straight from CLI flags — useful for a quick check, or for an agent that just got a URL and doesn't need to write a script first:

./vegaload run -target http://127.0.0.1:8080/widgets -protocol http1 -vus 10 -duration 30s

This runs 10 virtual users against the sample app for 30 seconds, prints a summary, and writes (and opens) a self-contained HTML report — one file, with the full latency distribution and a requests/errors-over-time chart, nothing else to host.

2. Or write a scenario file
./vegaload new my-scenario

scaffolds my-scenario.vl.js (JavaScript by default; -python for a .py scenario shelling out to a local python3). A scenario's http and ws globals make real HTTP and WebSocket calls, carrying a value from one into the next — a script can create something over HTTP, then open a WebSocket and read frames until it's done, the way examples/scenarios/http-ws-chain.vl.js does against the sample app (its Python twin, http_ws_chain.py, does the same thing):

./vegaload run -vus 5 -duration 10s examples/scenarios/http-ws-chain.vl.js

(flags before the scenario file — vegaload's flag parser stops at the first non-flag argument; run vegaload run -h for the full flag list.) A host outside localhost needs -allow-target or -yes — the same FR-CLI-06 allowlist protocol-direct mode's -target uses, just enforced per call since a script's own targets aren't known until it runs. A scenario with no network calls at all is still useful for pure per-iteration logic against VegaLoad's VU pool and executor shapes (fixed-VU, ramp, step, constant-arrival-rate) — examples/scenarios/smoke.vl.js is a minimal one:

./vegaload run -vus 5 -duration 5s examples/scenarios/smoke.vl.js

To load test an actual HTTP target without writing a script at all, use protocol-direct mode (step 1) instead — or generate a runbook of protocol-direct commands from an OpenAPI spec:

./vegaload new -from-openapi examples/sample-app/openapi.json sample-app

writes sample-app.vegaload-plan.md: one ready-to-run vegaload run command per endpoint the spec declares.

3. Explain a run's results

Keep a run's JSON summary alongside its HTML report with -out:

./vegaload run -target http://127.0.0.1:8080/widgets -protocol http1 \
  -vus 10 -duration 30s -out baseline.json
./vegaload diagnose baseline.json

diagnose reports the failure rate, flags a latency long tail (p99 more than 5x p50), and says whether failures were concentrated in one period or spread out — plus suggested pass/fail thresholds (a p95 ceiling, an error rate ceiling) derived from that same run, for writing into a CI gate. Add a bring-your-own-LLM connector for a plain-English narrative on top of the rule-based findings:

export VEGALOAD_LLM_PROVIDER=openai       # or anthropic, ollama
export VEGALOAD_LLM_API_KEY=sk-...
./vegaload diagnose baseline.json

Nothing about a run or its results leaves your machine unless you configure this yourself (-no-llm skips narration even if it's configured).

4. Compare a later run against the baseline

After another run with -out candidate.json:

./vegaload compare baseline.json candidate.json

Prints deltas (error rate, latency percentiles, totals, overall RPS) and exits non-zero if error rate or p95 latency got worse. Optional slack:

./vegaload compare -error-rate-delta 0.01 -p95-ratio 1.2 \
  baseline.json candidate.json

Checked-in fixtures under examples/scenarios/compare/ show both an ok and a regressed pair without needing a live target.

5. Hand all of this to an agent
./vegaload init

writes four things into the current project, each skippable if already present: a Claude Code skill bundle (.claude/skills/vegaload), a Cursor rules file (.cursor/rules/vegaload.mdc), and an MCP server entry merged into both .mcp.json and .cursor/mcp.json, pointing at this same compiled binary running vegaload mcp serve. Open the project in Claude Code or Cursor afterward and the agent has seven tools — create_scenario, run_test, get_results, suggest_thresholds, diagnose_failure, compare_reports, generate_from_spec — each one calling the exact CLI command shown above and parsing its -output json result; there is no agent-only path that skips the CLI.

vegaload mcp eval runs a versioned, non-LLM suite of {tool call, expected outcome} cases against those same tools directly — the thing to run in CI after upgrading, to check the tool layer itself still behaves, independent of any model's tool-picking behavior:

./vegaload mcp eval

See examples/scenarios/README.md for this same walkthrough as a standalone, copy-pasteable script.

Command reference

Command What it does
vegaload run Run a load test: a scenario file, or a protocol-direct target
vegaload new Scaffold a starter scenario file, or a runbook from an OpenAPI spec
vegaload diagnose Print environment info, or explain a report's results
vegaload mcp serve Run an MCP server over stdio for agent-native use
vegaload mcp eval Run the versioned MCP tool-calling eval suite against this binary
vegaload init Register the MCP server and skill bundles for the current project

Every command supports -output text (default), json, or (run only) jsonl. Run vegaload <command> -h for its full flag list, or vegaload help for the top-level summary.

License

Apache-2.0. See LICENSE.

Directories

Path Synopsis
cmd
vegaload command
eval.go implements `vegaload mcp eval`: FR-MCP-04's versioned MCP tool-calling evaluation suite, run against this same compiled binary.
eval.go implements `vegaload mcp eval`: FR-MCP-04's versioned MCP tool-calling evaluation suite, run against this same compiled binary.
internal
audit
Package audit implements FR-CLI-07: an audit record of every run — trigger source, target, parameters, and timestamp — appended to a local, append-only JSONL file.
Package audit implements FR-CLI-07: an audit record of every run — trigger source, target, parameters, and timestamp — appended to a local, append-only JSONL file.
compare
Package compare diffs two report.Result values (typically JSON from `vegaload run -out`) so a later run can be checked against a baseline without a baseline registry or history store — just two files.
Package compare diffs two report.Result values (typically JSON from `vegaload run -out`) so a later run can be checked against a baseline without a baseline registry or history store — just two files.
diagnose
Package diagnose implements the rule-based half of FR-MCP-05's diagnose_failure / vegaload diagnose: a set of fixed heuristics over a report.Result that flag the failure patterns a human reviewing a load test would look for first, plus a pure-arithmetic threshold suggester.
Package diagnose implements the rule-based half of FR-MCP-05's diagnose_failure / vegaload diagnose: a set of fixed heuristics over a report.Result that flag the failure patterns a human reviewing a load test would look for first, plus a pure-arithmetic threshold suggester.
engine
Package engine implements VegaLoad's core execution engine: the virtual user (VU) scheduler and the load-shape executors that decide how many VUs run, and when, over the life of a scenario.
Package engine implements VegaLoad's core execution engine: the virtual user (VU) scheduler and the load-shape executors that decide how many VUs run, and when, over the life of a scenario.
eval
Package eval implements FR-MCP-04's versioned MCP tool-calling evaluation suite: a fixed set of {tool, arguments, expected outcome} cases, embedded into the binary and versioned by directory (v1, and any v2+ added later without touching v1's cases) so a suite someone has pinned to doesn't silently change underneath them.
Package eval implements FR-MCP-04's versioned MCP tool-calling evaluation suite: a fixed set of {tool, arguments, expected outcome} cases, embedded into the binary and versioned by directory (v1, and any v2+ added later without touching v1's cases) so a suite someone has pinned to doesn't silently change underneath them.
llm
Package llm implements FR-MCP-05's optional BYO-LLM narrative layer: turning a diagnose.Findings into a short plain-English explanation by calling a user-configured LLM endpoint.
Package llm implements FR-MCP-05's optional BYO-LLM narrative layer: turning a diagnose.Findings into a short plain-English explanation by calling a user-configured LLM endpoint.
mcp
Package mcp implements FR-MCP-01/02/03's agent-native layer: a hand-rolled JSON-RPC 2.0 server speaking the Model Context Protocol over stdio (no third-party SDK — see go.mod's dependency-pinning history for why this project prefers stdlib-only solutions).
Package mcp implements FR-MCP-01/02/03's agent-native layer: a hand-rolled JSON-RPC 2.0 server speaking the Model Context Protocol over stdio (no third-party SDK — see go.mod's dependency-pinning history for why this project prefers stdlib-only solutions).
openapi
Package openapi implements FR-MCP-06's `vegaload new --from-openapi` and the generate_from_spec MCP tool: turning an OpenAPI document into a ready-to-run starting point for load testing its endpoints.
Package openapi implements FR-MCP-06's `vegaload new --from-openapi` and the generate_from_spec MCP tool: turning an OpenAPI document into a ready-to-run starting point for load testing its endpoints.
protocol
Package protocol defines the shared interface every wire protocol driver (HTTP/1.1, HTTP/2, gRPC, WebSocket) implements, plus the request and result types they exchange with the engine.
Package protocol defines the shared interface every wire protocol driver (HTTP/1.1, HTTP/2, gRPC, WebSocket) implements, plus the request and result types they exchange with the engine.
protocol/grpc
Package grpc implements a protocol.Protocol driver for gRPC.
Package grpc implements a protocol.Protocol driver for gRPC.
protocol/http1
Package http1 implements a protocol.Protocol driver that speaks plain HTTP/1.1.
Package http1 implements a protocol.Protocol driver that speaks plain HTTP/1.1.
protocol/http2
Package http2 implements a protocol.Protocol driver that speaks HTTP/2, over TLS (the normal case, negotiated via ALPN) or over plain TCP ("h2c") when the target is an http:// URL — many internal services run h2c precisely to skip TLS termination, and a load tester that only understood h2-over-TLS would be unable to test them at all.
Package http2 implements a protocol.Protocol driver that speaks HTTP/2, over TLS (the normal case, negotiated via ALPN) or over plain TCP ("h2c") when the target is an http:// URL — many internal services run h2c precisely to skip TLS termination, and a load tester that only understood h2-over-TLS would be unable to test them at all.
protocol/websocket
Package websocket implements a protocol.Protocol driver for WebSocket.
Package websocket implements a protocol.Protocol driver for WebSocket.
report
Package report implements FR-RPT-01 through 04 and FR-CLI-05: turning a run's recorded iterations into the structured data a self-contained HTML report, a JSON summary, and (eventually) the MCP get_results tool all read from.
Package report implements FR-RPT-01 through 04 and FR-CLI-05: turning a run's recorded iterations into the structured data a self-contained HTML report, a JSON summary, and (eventually) the MCP get_results tool all read from.
safety
Package safety implements FR-CLI-06: a target allowlist plus hard caps on RPS, duration, and concurrency, enforced by default on every run.
Package safety implements FR-CLI-06: a target allowlist plus hard caps on RPS, duration, and concurrency, enforced by default on every run.
scripting/js
Package js runs a scenario written as a JavaScript or TypeScript file, using the embedded Goja ECMAScript runtime — no Node.js, no network fetch of a runtime, and no separate build step for the author.
Package js runs a scenario written as a JavaScript or TypeScript file, using the embedded Goja ECMAScript runtime — no Node.js, no network fetch of a runtime, and no separate build step for the author.
scripting/netapi
Package netapi gives VegaLoad's scripting runtimes (internal/scripting/js and /python) the real HTTP and WebSocket access FR-CLI-08 asks for: a scenario script can make an actual network call and read the response (status, headers, body) back into the script, unlike internal/protocol's drivers, which exist to repeat one fixed Target and discard the response body entirely -- see protocol.Result's doc comment, and http1.Driver.Do, which io.Copy's the body straight to io.Discard.
Package netapi gives VegaLoad's scripting runtimes (internal/scripting/js and /python) the real HTTP and WebSocket access FR-CLI-08 asks for: a scenario script can make an actual network call and read the response (status, headers, body) back into the script, unlike internal/protocol's drivers, which exist to repeat one fixed Target and discard the response body entirely -- see protocol.Result's doc comment, and http1.Driver.Do, which io.Copy's the body straight to io.Discard.
scripting/python
Package python runs a scenario written as a Python file, by shelling out to a python3 interpreter the user has installed separately.
Package python runs a scenario written as a Python file, by shelling out to a python3 interpreter the user has installed separately.
skills
Package skills implements FR-MCP-04's agent skill bundles and the `vegaload init` command built on them: the actual Markdown content Claude Code and Cursor read to know when and how to call VegaLoad's MCP tools, embedded into the compiled binary at build time (via go:embed) so `vegaload init` can write them out without needing the vegaload source tree to be present on the machine it's installed on.
Package skills implements FR-MCP-04's agent skill bundles and the `vegaload init` command built on them: the actual Markdown content Claude Code and Cursor read to know when and how to call VegaLoad's MCP tools, embedded into the compiled binary at build time (via go:embed) so `vegaload init` can write them out without needing the vegaload source tree to be present on the machine it's installed on.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL