sendit

sendit

A Go CLI tool that simulates realistic user web traffic across HTTP, headless browser, DNS, and WebSocket protocols. Designed to blend into normal traffic baselines while being polite to both the local machine and target servers.

Key properties:

• Never bursts aggressively — all pacing is delay-gated before acquiring worker slots
• Per-domain token-bucket rate limits with decorrelated jitter backoff on transient errors
• Pauses dispatch when local CPU or RAM exceeds configurable thresholds
• Graceful shutdown: waits for all in-flight requests to complete on SIGINT/SIGTERM

---

Quick Start

Prerequisites

• Go 1.22+
• Chrome/Chromium (only required for type: browser targets)

Build

git clone <repo>
cd sendit
go build -o sendit ./cmd/sendit

Validate your config

./sendit validate --config config/example.yaml
# config valid

Run

./sendit start --config config/example.yaml --log-level debug

Run with a targets file

Rather than listing every target in the YAML, you can point targets_file at a plain-text list:

# config/my-targets.txt
https://example.com   http   5
example.com           dns    2

# config/simple.yaml
targets_file: "config/my-targets.txt"
target_defaults:
  http:
    timeout_s: 15
  dns:
    resolver: "8.8.8.8:53"

./sendit validate --config config/simple.yaml   # check the file parses cleanly
./sendit start   --config config/simple.yaml --log-level debug

---

CLI Commands

sendit start    [-c <path>] [--foreground] [--log-level debug|info|warn|error] [--dry-run]
sendit probe    <target>   [--type http|dns] [--interval 1s] [--timeout 5s]
sendit stop     [--pid-file <path>]
sendit status   [--pid-file <path>]
sendit validate [-c <path>]
sendit version
sendit completion <shell>

Command	Description
start	Start the engine. Writes a PID file by default so stop/status can find the process; use --foreground to skip writing the PID file.
probe	Test a single HTTP or DNS endpoint in a loop (like ping). No config file required.
stop	Send SIGTERM to a running instance via its PID file.
status	Check whether the process in the PID file is still alive.
validate	Parse and validate a config file without starting the engine. Exits 0 on success, non-zero with a message on failure.
version	Print version, commit, and build date.
completion	Generate shell autocompletion scripts (bash, zsh, fish, powershell).

start flags

Flag	Short	Default	Description
--config	-c	config/example.yaml	Path to YAML config file
--foreground		false	Skip writing the PID file (process always runs in the foreground)
--log-level		(from config)	Override log level: debug | info | warn | error
--dry-run		false	Print config summary (targets, pacing, limits) and exit without sending traffic

probe flags

Flag	Default	Description
--type	(auto-detected)	Driver type: http | dns
--interval	1s	Delay between requests
--timeout	5s	Per-request timeout
--resolver	8.8.8.8:53	DNS resolver (dns targets only)
--record-type	A	DNS record type (dns targets only)

stop / status flags

Flag	Default	Description
--pid-file	/tmp/sendit.pid	Path to the PID file written by start

validate flags

Flag	Short	Default	Description
--config	-c	config/example.yaml	Path to YAML config file

---

Configuration Reference

See config/example.yaml for a full working example. Every section has defaults so you only need to specify what you want to override.

pacing

Controls how requests are spaced in time.

Field	Default	Description
mode	human	human | rate_limited | scheduled
requests_per_minute	20	Target RPM — used by rate_limited and scheduled modes only
jitter_factor	0.4	Unused in human mode; reserved for future modes
min_delay_ms	800	Minimum inter-request delay in human mode
max_delay_ms	8000	Maximum inter-request delay in human mode
schedule	[]	List of cron windows — required when mode: scheduled

Pacing modes:

• human — random delay per request uniformly sampled from [min_delay_ms, max_delay_ms]. requests_per_minute and jitter_factor are ignored in this mode.
• rate_limited — token-bucket limiter at requests_per_minute plus a small random jitter after each token.
• scheduled — cron expressions open active windows; within each window behaves like rate_limited at the window's own RPM. Dispatch is paused between windows.

pacing:
  mode: scheduled
  schedule:
    - cron: "0 9 * * 1-5"      # weekdays 09:00
      duration_minutes: 30
      requests_per_minute: 40

limits

Concurrency and resource thresholds.

Field	Default	Description
max_workers	4	Maximum simultaneous requests across all driver types
max_browser_workers	1	Sub-limit for concurrent headless browser instances
cpu_threshold_pct	60.0	Pause dispatch when CPU usage exceeds this percentage
memory_threshold_mb	512	Pause dispatch when RAM in use exceeds this value in MB

Note: memory_threshold_mb defaults to 512 MB, which is below baseline usage on most modern machines. Set this to a value above your system's idle memory footprint (e.g. 8192 for a 16 GB machine) to avoid blocking dispatch entirely.

rate_limits

Per-domain token buckets applied after the pacing delay and before acquiring a worker slot.

Field	Default	Description
default_rps	0.5	Requests per second applied to all domains not listed in per_domain
per_domain	[]	List of {domain, rps} overrides

rate_limits:
  default_rps: 0.5
  per_domain:
    - domain: "example.com"
      rps: 0.2

backoff

Retry behaviour on transient errors (HTTP 429, 502, 503, 504, DNS SERVFAIL, network failures).

Field	Default	Description
initial_ms	1000	Base delay for the first retry, in milliseconds
max_ms	120000	Maximum delay cap, in milliseconds
multiplier	2.0	Exponential growth factor per attempt
max_attempts	3	Stop retrying after this many consecutive failures for a domain

Permanent errors (HTTP 400, 403, 404; DNS NXDOMAIN, REFUSED) are logged and skipped immediately with no retry. Context cancellation errors are dropped silently.

targets_file and target_defaults

Instead of (or in addition to) listing targets inline, you can point targets_file at a plain-text file of URL/type pairs. Targets from the file are appended to any inline targets entries, so both can be used together.

File format — one entry per line:

<url> <type> [weight]

• url — full URL (https://, wss://) or a bare hostname for DNS targets
• type — one of http | browser | dns | websocket
• weight — optional positive integer; defaults to target_defaults.weight when omitted
• Lines starting with # and blank lines are ignored

# config/targets.txt
https://example.com      http   5
https://api.example.com  http   3
example.com              dns    2
wss://ws.example.com     websocket

target_defaults supplies the remaining fields (driver settings, default weight) for every target loaded from the file. Inline targets are unaffected and use whatever fields they specify directly.

targets_file: "config/targets.txt"

target_defaults:
  weight: 1                    # used when weight is omitted from the file
  http:
    method: GET
    headers:
      User-Agent: "Mozilla/5.0 ..."
    timeout_s: 15
  browser:
    scroll: false
    timeout_s: 30
  dns:
    resolver: "8.8.8.8:53"
    record_type: A
  websocket:
    duration_s: 30
    expect_messages: 0

target_defaults field	Default	Description
weight	1	Selection weight for file targets with no explicit weight
http.method	GET	HTTP verb
http.timeout_s	15	Request timeout in seconds
browser.timeout_s	30	Page load timeout in seconds
dns.resolver	8.8.8.8:53	DNS resolver address
dns.record_type	A	DNS record type
websocket.duration_s	30	How long to hold the connection open

Note: HTTP header map keys are lowercased by the YAML parser (e.g. User-Agent is stored as user-agent). This applies to both inline targets and target_defaults.

targets

List of endpoints to request. Each target has a weight controlling selection frequency relative to the others. Selection uses the Vose alias method (O(1) per pick).

targets:
  - url: "https://example.com"
    weight: 10
    type: http
    http:
      method: GET
      headers:
        User-Agent: "Mozilla/5.0 ..."
      body: ""          # optional request body
      timeout_s: 15

  - url: "https://news.ycombinator.com"
    weight: 5
    type: browser
    browser:
      scroll: true                    # scroll to mid-page then bottom
      wait_for_selector: "#hnmain"    # wait for this CSS selector before returning
      timeout_s: 30

  - url: "example.com"
    weight: 3
    type: dns
    dns:
      resolver: "8.8.8.8:53"
      record_type: A          # A | AAAA | MX | TXT | CNAME | ...

  - url: "wss://stream.example.com/feed"
    weight: 2
    type: websocket
    websocket:
      duration_s: 30                          # hold connection open for this long
      send_messages: ['{"type":"subscribe"}'] # messages to send on connect
      expect_messages: 1                      # wait to receive this many messages

output

Optional result export to a file for offline analysis.

Field	Default	Description
enabled	false	Enable result export
file	sendit-results.jsonl	Output file path
format	jsonl	jsonl (one JSON object per line) | csv
append	false	Append to an existing file instead of truncating on start

output:
  enabled: true
  file: "results.jsonl"
  format: jsonl    # jsonl | csv
  append: false

Each JSONL record contains: ts, url, type, status, duration_ms, bytes, error. CSV output writes a header row when append: false.

metrics

Optional Prometheus exposition.

metrics:
  enabled: true
  prometheus_port: 9090     # GET http://localhost:9090/metrics

Exposed metrics:

Metric	Type	Labels
sendit_requests_total	Counter	type, status_code
sendit_errors_total	Counter	type, error_class
sendit_request_duration_seconds	Histogram	type
sendit_bytes_read_total	Counter	type

daemon

daemon:
  pid_file: "/tmp/sendit.pid"   # written by start unless --foreground is set
  log_level: info                   # debug | info | warn | error
  log_format: text                  # text (coloured console) | json

---

Dispatch Pipeline

Every task flows through the following gates in order before a worker goroutine is launched:

Scheduler.Wait        pacing delay (human jitter / token bucket / cron window)
  → resource.Admit    pause if CPU or RAM over threshold
  → backoff.Wait      per-domain delay after transient errors
  → ratelimit.Wait    per-domain token bucket
  → pool.Acquire      global semaphore + browser sub-semaphore
  → go driver.Execute
  → pool.Release

This ordering ensures that slow or rate-limited domains do not consume worker slots while waiting.

---

Architecture

cmd/sendit/main.go          cobra CLI
internal/config/                YAML loader, defaults, validator, targets_file parser
internal/task/                  Task & Result types; Vose alias weighted selector
internal/ratelimit/             Per-domain token-bucket registry; decorrelated jitter backoff
internal/resource/              gopsutil CPU/RAM monitor with Admit() gate
internal/driver/                HTTP · headless browser (chromedp) · DNS (miekg) · WebSocket
internal/engine/                Worker pool · scheduler · dispatch loop
internal/metrics/               Prometheus counters & histograms
internal/output/                JSONL / CSV result writer (non-blocking, goroutine-backed)
config/example.yaml             Full reference configuration (with target_defaults section)
config/targets.txt              Example targets file (url + type per line)
config/test.yaml                Lightweight HTTP+DNS config for local smoke-testing

Browser driver

Each browser task spawns its own chromedp.ExecAllocator — no shared browser state — which prevents memory accumulation from long-running sessions. The max_browser_workers sub-semaphore limits concurrent Chrome instances independently of the global worker pool.

DNS driver

DNS RCODEs are mapped to HTTP-like status codes so the engine's unified error classifier works across all driver types:

DNS RCODE	HTTP equivalent	Effect
NOERROR (0)	200	success
NXDOMAIN (3)	404	permanent skip
REFUSED (5)	403	permanent skip
SERVFAIL (2)	503	transient backoff
other	502	transient backoff

---

Running Tests

go test ./...
go test -race ./...      # with race detector

---

Verification

Scenario	How to test
Config validation	sendit validate --config config/example.yaml → prints "config valid", exits 0
HTTP traffic	Use config/test.yaml (points at httpbin.org); observe status codes in logs
DNS traffic	DNS targets in config/test.yaml; look for type=dns status=200 log lines
targets_file	Set targets_file: "config/targets.txt" in a config; validate checks the file, start loads all entries
targets_file error	Point targets_file at a file with a bad line (e.g. example.com grpc) → validate prints the line number and error
target_defaults	Omit method from target_defaults.http; confirm requests default to GET in logs
Resource gate	Set cpu_threshold_pct: 1 → logs show "resource monitor: over threshold, dispatch paused"
Rate limiting	Set default_rps: 0.1, max_workers: 1 → ~1 req/10s per domain observed
Backoff	Point a target at a URL returning 429; observe exponential backoff= delay in WRN logs
Graceful shutdown	Send SIGTERM during active requests → process logs "engine stopped" after in-flight tasks finish