merlon

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 14, 2026 License: MIT Imports: 7 Imported by: 0

README

Merlon

Merlon is a suite of Caddy modules that silently scores the bot-likelihood of every incoming request at the edge — without blocking anything. It combines L1 signals (static network reputation, request velocity) with L2 signals (TLS/JA4 fingerprint vs. User-Agent consistency) into a clamped 1–20 score (1 = almost certainly human, 20 = almost certainly automated), annotating Caddy vars, the request context, and logs so that later stages — a WAF, challenges, rate-limit actions — can act on it. Design: docs/superpowers/specs/2026-10-29-merlon-design.md.

Status: M5b shipped (workers, routing, Coraza option & web console), plus the B1 BGP origin feed; B5 is the documentation milestone in progress. Merlon scores every request against live IP intelligence — public feeds (RIR allocations, threat lists, GeoLite2, CAIDA relations, RPKI ROAs, and an optional full-table BGP RIB feed for observed origin ASNs) ingested into in-memory LPM tries that drive the L1 layer — enforces per-site rules through merlon_waf — bypass / deny / redirect / challenge actions plus an observe mode — runs the progressive trust ladder (attestation → proof-of-work → interactive CAPTCHA or polite block) with honeypots, and executes JavaScript workers in-process: phase-3 route rules dispatch matching requests to a QuickJS-NG sandbox (merlon_worker_router), deployed, versioned and rolled back live via the admin API. An optional embedded Coraza OWASP engine runs behind the coraza rule action, scoring settings hot-edit in place, and a web console (/id/merlon/console/) manages it all — including live SSE request inspection — without config reload.

Build

xcaddy builds a Caddy binary with all Merlon modules baked in:

xcaddy build v2.11.7 --with codeberg.org/r3boot/merlon

(From a checkout of this repo, use --with codeberg.org/r3boot/merlon=. to build against the local source.)

Quickstart

Merlon ships both config formats: Caddy JSON (this quickstart) and the Caddyfile — grammar, JSON translation and the JSON-only boundary live in docs/usage/caddyfile.md. The JSON form:

// caddy.json — comments allowed; Caddy strips them
{
  "admin": {
    // Pin the admin listener explicitly instead of relying on Caddy's
    // built-in default (this repo reserves that port for its own tests).
    // Every admin-API example below assumes 127.0.0.1:2020.
    "listen": "127.0.0.1:2020"
  },
  "apps": {
    "merlon": {},
    "http": {
      "servers": {
        "srv0": {
          "listen": [":8080"],
          "routes": [
            {
              "handle": [
                {
                  "handler": "merlon_rating",
                  "engine": {
                    "network_ranges": [
                      {"cidr": "198.51.100.0/24", "reason": "datacenter", "delta": 6}
                    ],
                    "known_browser_ja4": ["t13d1516h2_8daaf6152771_e5627efa2ab1"],
                    "velocity_per_sec": 1000,
                    "velocity_burst": 2000
                  }
                },
                {
                  "handler": "static_response",
                  "headers": {"X-Bot-Score": ["{http.vars.bot_score}"]},
                  "body": "score: {http.vars.bot_score}"
                }
              ]
            }
          ]
        }
      }
    }
  }
}
./caddy run --config caddy.json
curl -si http://localhost:8080/ | grep -i x-bot-score   # → X-Bot-Score: 13

merlon_rating never blocks; it stores the score in {http.vars.bot_score} and the fired signal names in {http.vars.bot_reasons}, logs both at DEBUG, and puts the full *score.Result on the request context. The curl above scores 13: base 1 + no_tls (4) + ua_script (8) — the shipped defaults in score/fingerprint.go (applyDefaults), combined by score/score.go (Finalize: clamp(1 + Σ deltas, 1, 20)).

For a production-shaped config — TLS handshake hook, listener wrapper, challenge and worker stages, a CA/pki setup for lab domains — see docs/usage/getting-started.md.

Documentation hub

This README is the map; the reference detail lives in docs/usage/. Every page cites the code it was written against, so a doc that contradicts the tree is a bug — file it.

Page What it owns
getting-started.md Build with xcaddy, a realistic standalone JSON config, and the verify-your-install walkthrough
configuration.md App/module JSON: db_path, trust keyring co-location, feeds snapshot dir and caps, site-key normalization
caddyfile.md Caddyfile directives — grammar, JSON translation, what stays JSON
rules.md The rule grammar in full: matcher table, action table, phase pairing, observe mode
scoring.md Score formula, the complete signal/weight table, EngineConfig keys and SQLite-override precedence
ja4plus.md The JA4+ lane: JA4H (the b1 variant + deviation table) and JA4T computation, the operator-managed fp_intel table, the four B10 signals and the family rule
challenges.md The progressive ladder, /__merlon/* endpoints, token binding, honeypots, captcha configuration
feeds.md The nine adapters, source config fields (name vs adapter), merge/origin priority, refresh scheduling
cdn.md CDN attribution setup: the five provider formats, per-provider header trust rules, GCP/Akamai caveats, mTLS boundary
identity.md The peer-gated Tailscale identity lane: Serve/tsnet topologies, allow-lists, sharp edges
webbotauth.md The Web Bot Auth lane: verifying signed AI-agent requests (draft-ietf-webbotauth-httpsig-protocol-01), the three closed outcomes, directory-fetch posture, and the honest limits before granting a pass
waf.md Coraza engine v2: rule-set profiles, per-site OWASP wiring, the pinned CRS provisioning recipe and its honest phase-1-only coverage delta
websockets.md The WS lane end to end: ws_upgrade matching, the merlon_ws frame-tee attach recipe, what the tee sees and refuses to see, kill/bump policy
vhosts.md The virtual-host store plane: the two flags and what toggling does not do, the escape-hatch truth, the collision law's two answers, the rename cascade, and how the DNS plane joins it
dns.md The DNS zone plane: the CoreDNS recipe, the three TSIG identities and rotation, slaves and DNSSEC, the reconciler's card vocabulary, the BC-6 preview gate, backups
workers.md The handle(req) ABI, QuickJS sandbox rules, deploy/rollback workflow
admin-api.md Route-by-route admin reference with a curl example per endpoint
console.md Every console view, with its behavior read from the view source — screenshots where the shot pipeline has them (images in docs/images/), prose for the Settings, FP-Lane, Feeds, FP-Intel, Vhosts and Zones views, plus the SSE and ETag behaviors
operations.md Local CI gates, the e2e suite pointer, perf/PERF.md workflow, backup and revocation notes
admin-security.md Admin-plane security posture on one page: the port-is-the-credential law, the pinned browser walls and console headers, unix-socket option, state-file modes, audit retention — each fact linked to its test or code pin
metrics.md The complete merlon_* registry: every family with type/labels/example PromQL, the scrape quickstart, lifetime semantics, absent-series honesty

Roadmap items that are not shipped — sensor.js v2 countermeasures (B8), workerd sidecar (B9) — are listed in Status & limitations and appear nowhere else as current features.

Status & limitations

Honest current state, verified against this tree:

  • Rules and settings are admin-API-managed by design. They are Tier-2 dynamic config (spec §8): rules, scoring overrides, feed sources, honeypot paths, the challenge policy and the coraza engine live in the SQLite rules/settings tables and hot-swap without a config reload. That is why rules never appear in the Caddy JSON — not an omission.
  • The admin port is unauthenticated loopback. There are no tokens, no auth headers and no console login: whatever can reach the admin listener is an admin (adminapi/adminapi.go package doc — "reachable only wherever the admin listener is bound (loopback by default)"). Reach it over SSH/VPN/ Tailscale; never move it to a routable address without real authentication in front of it.
  • The captcha provider has no admin route. See CAPTCHA (rung 3): the settings row is edited with a SQLite client. This is a known gap, not an oversight in the docs.
  • Coraza runs the request-headers phase only. merlon_waf feeds the engine URI, method and headers — never a body — so SecLang rules in coraza phases 2–5 compile but never evaluate, and a matched coraza rule fails open when the engine is disabled or unset. See Optional Coraza WAF engine (M5b).
  • Observe/enforce counters are ephemeral. GET /id/merlon/observe/matches counts are in-memory since process start and reset on restart (rules/snapshot.go: Counters); they are a rehearsal and drift signal, not an audit trail — the audit log is the durable record.
  • Not yet (roadmap): sensor.js v2 countermeasures (B8), workerd sidecar (B9). Fingerprint-rotation enforcement (the P-fwd escalation) stays gated on the sensor.js v2 device anchor — B8 ships the observation lane and candidate export only (ruling R-b8-fp-empty).

Rules & enforcement (M2)

Rules live in SQLite, opened at apps.merlon.db_path (default <caddy data dir>/merlon.db), and are edited through Merlon's admin API — never by editing Caddy config. Every mutation recompiles the site's ruleset and hot-swaps it atomically (each list response carries a monotonically bumping generation), so rule changes take effect on the next request with no config reload. merlon_waf is the only Merlon module that blocks traffic, and it must be placed after merlon_rating in a route's handler chain — it reads the score rating put on the request context (with no rating handler before it, bot_score matchers see 0).

Canonical wiring:

{
  "apps": {
    "merlon": {}, // SQLite store + per-site snapshots + observe counters
    "http": {
      "servers": {
        "srv0": {
          "listen": [":8080"],
          "routes": [
            {
              "handle": [
                { "handler": "merlon_rating" },
                { "handler": "merlon_waf", "site": "example.com" },
                { "handler": "static_response", "body": "origin" }
              ]
            }
          ]
        }
      }
    }
  }
}

(site is optional; it defaults to the request Host without port. Both are case-normalized — site keys are matched lowercase, so config and admin API always agree.)

Admin API

Served by the admin.api.merlon module on Caddy's admin listener — Caddy's own admin API, so no extra port and no extra auth surface (see the box below). Route-by-route reference, with a curl example per endpoint, is docs/usage/admin-api.md; this is the map, grouped by surface:

Surface Endpoints under /id/merlon/ Deep dive
Rules GET+POST sites/{site}/rules, PATCH+DELETE sites/{site}/rules/{id}, POST sites/{site}/rules/reorder rules.md
Observe counters GET observe/matches rules.md
Challenge ladder, honeypots, token revocation GET+PUT settings/challenges, GET+PUT settings/honeypots, POST trust/bump_epoch challenges.md
Feeds GET+PUT settings/feeds, POST feeds/refresh, GET feeds/status, GET+PUT feeds/custom feeds.md
Network intelligence GET lookup?ip=…, GET registry/asn/{asn}?limit=… feeds.md
Scoring override GET+PUT+DELETE settings/scoring scoring.md
Coraza engine GET+PUT settings/coraza Coraza, waf.md
Live inspection GET inspect/live (SSE) console.md
Audit log & config export GET audit?limit=N, POST export admin-api.md
Workers GET sites/{site}/workers, POST …/{name}/deploy, POST …/{name}/rollback, GET …/{name}/versions, GET …/{name}/code, GET+PUT+DELETE …/{name}/schedule, POST …/{name}/run workers.md
Worker assets GET …/{name}/assets, PUT+DELETE …/{name}/assets/{path...}, DELETE …/{name}/assets, GET+PUT …/{name}/assets-config workers.md
Web console GET console/{path...} console.md

Every example in this README talks to an admin listener pinned to an explicit loopback port, so it never collides with Caddy's built-in default (which this repo's own test suite reserves):

export MADMIN=http://127.0.0.1:2020   # matches the Quickstart's admin.listen

Site keys are normalized (port stripped, IPv6 brackets removed, case-folded): foo:8080, [foo] and FOO all name the same site. Rule ETags look like "v3" (the row version) and come back from create and patch responses. Errors: invalid rule → 400 with the grammar message, unknown id → 404, stale If-Match or concurrent-write conflict → 412, missing If-Match on PATCH → 428.

Keep the admin API private. It is only as exposed as Caddy's admin listener itself — loopback (localhost:2019) by default. Keep it that way; do not move admin.listen to a routable address without putting real authentication in front of it. There are no tokens and no auth headers anywhere in this surface: reachability is the authorization model (spec §9 — zero extra auth surface).

# create a phase-2 deny rule (responds 201 + ETag: "v1" + Location)
curl -si -X POST $MADMIN/id/merlon/sites/example.com/rules \
     -d '{"phase":2,"enabled":true,
          "matchers":{"bot_score":{"min":15}},
          "action":{"type":"deny"}}'

# patch = full replace; If-Match with the current ETag is mandatory
curl -si -X PATCH $MADMIN/id/merlon/sites/example.com/rules/1 \
     -H 'If-Match: "v1"' \
     -d '{"phase":2,"enabled":true,
          "matchers":{"bot_score":{"min":18}},
          "action":{"type":"deny","status":429}}'

Scoring hot-edit. PUT settings/scoring takes exactly the JSON that merlon_rating's engine block takes (see Quickstart) and takes effect on the next request: the PUT builds the engine before persisting, so an invalid payload gets 400 and never touches storage or live state. While the row exists it wins over the module's JSON config; DELETE drops it and GET answers {"configured":false} until then, and the override is reloaded at Provision so it survives restarts. One gotcha: a PUT replaces the live engine, which resets in-flight per-IP velocity counters — repeated tuning suppresses the velocity signal, so expect a warm-up window after each save.

Live inspection. GET inspect/live streams every scored request as an SSE data: frame shaped {"ts","site","method","path","ip","score","sensor_state","asn","country","signals":[{"layer","name","delta"}]} (optional fields omitted when empty) with a comment heartbeat every 15 s. Publishing is gated on subscriber presence — zero overhead while nobody watches — and a slow subscriber's oldest buffered event is dropped rather than blocking the data plane.

Audit & export. GET audit?limit=N answers audit entries newest first as {"entries":[{ts,site,kind,rule_id,before,after}]} (default limit 100, clamped to 1..1000, non-numeric → 400). POST export answers the whole configuration bundle (version 1): per-site rules, worker code with current versions inline, and the settings documents — challenges, honeypot paths, feeds, captcha, the scoring override (null when no SQLite override exists) and the coraza config. It carries no secret values: per ruling R7 the captcha config holds only the secret_env variable name, and feed license keys appear by name only, so the bundle is safe to ship off-box for backup or migration. Every successful export writes one settings.export audit entry with no payload — just the egress trace.

Web console. A browser UI for everything in this section ships inside the binary — see Web console (M5b).

Rule grammar

Phases. Rules run in three ordered phases: 1 = bypass, 2 = enforce, 3 = route (landed in M5a). merlon_waf walks phases 1→2; phase 3 is evaluated separately by merlon_worker_router, which sees only route rules (within a phase the first enabled matching rule wins that phase). A phase-1 bypass match skips all enforcement; an enforce-mode match acts immediately and stops evaluation. A fourth enum, 4 = score nudge, is deliberately not an evaluation order: nudge rules run as a cumulative pre-pass before phases 1–2 — see score below and rules.md.

Actions. bypass | deny | redirect | challenge | coraza | queue | route | score:

Action Options Defaults
bypass — stop enforcement, serve the origin
deny status (any 4xx) 403, body blocked by merlon
redirect url (required), status ∈ 301/302/307/308 302
challenge policy (only "progressive" exists so far) progressive ladder — see Trust & challenges
coraza profile (rule-set profile name, "" = default engine) run the request through the embedded Coraza engine — phase 2 only; see Coraza & waf.md
queue capacity, max_wait_ms, max_waiters, status, retry_after_s 20, 5000, 4×capacity, 429, 5 — concurrency permit with bounded waiters, not a rate limiter — phase 2 only; see rules.md
route worker (required, ^[a-z0-9][a-z0-9_-]{0,63}$) dispatch to the named worker — phase 3 only; see Workers & routing
score delta (non-zero, abs ≤ 19), reason (≤ 200 B) shift the effective bot score fed to enforcement — phase 4 only; unrated requests are never nudged; see rules.md

Validation enforces the pairing: route requires phase 3, coraza and queue require phase 2, bypass/deny/redirect/challenge require phases 1–2, and score pairs exclusively with phase 4.

Matchers. All present conditions must match (AND); an absent condition passes; at least one condition is required (match-all rules are rejected):

Matcher Matches when
path: {exact | prefix | regex} any variant matches the URL path (OR-ed within the condition)
methods: ["GET", "POST"] request method is in the list (case-insensitive)
header: {name, regex?} header present; empty/absent regex = presence check only
cookie: {name, regex?} same, for cookies
bot_score: {min} rating score ≥ min (0 when no rating handler ran)
client_cidrs: ["10.0.0.0/8"] client IP falls in any CIDR
client_lists: ["office-nets"] client IP falls in any CIDR of the named fleet network lists (merged with client_cidrs; an empty or list-only-empty rule never matches — fail closed); lists live at PUT /id/merlon/lists/networks/{name} — see rules.md
ja4_prefixes: ["t13d1516h2"] JA4 fingerprint starts with any prefix
asn: [6502] feed-resolved origin ASN is in the list (see Feeds)
country: ["de"] feed-resolved ISO 3166-1 alpha-2 country matches (case-insensitive)
sensor_state: "absent" L3 token state: absent | valid | stale | replayed (see below)
ws_upgrade: true|false request is (true) / is not (false) a WebSocket upgrade — Upgrade: websocket header, case-insensitive; negation landed in M5a

asn and country fail closed: without feeds data resolving network context for the client, they never match. General rule negation (beyond ws_upgrade:false) and the query matcher remain deferred. Matcher-by-matcher semantics, the phase-pairing validation rules and worked examples: docs/usage/rules.md.

Mode. "mode": "observe" (default "enforce") evaluates the rule but never acts on the request: every would-have-matched hit is counted, logged, and exposed by GET /id/merlon/observe/matches as {"rules":[{"rule_id":1,"observed":42,"actions":0}]} while traffic passes freely — the safe way to rehearse a rule before switching it to enforce. Counters are in-memory and reset on restart.

Trust & challenges (M3)

The progressive ladder (spec §6): score below rung1_min passes silently, then attestation page → client proof-of-work → interactive CAPTCHA or polite block (the CAPTCHA rung activates with a provider — see CAPTCHA). Every rung is configurable at runtime; failures escalate per client and reset on success. Since M20 the HTML legs are branded formatted documents — themed, localized (Accept-Language, en fallback), noindex/no-store, CSP-locked per page; machine legs are byte-identical to M3 (contract: docs/usage/challenges.md §The interstitial documents).

rung default score band browser clients JSON/API clients
0 < 9 pass silently pass silently
1 9–11 inline attestation page (loads sensor.js, auto-reload on success) pass through, scored
2 12–15 PoW puzzle page 403 + puzzle as JSON
3 ≥ 16 CAPTCHA page when a provider is configured, else polite block 403 {"error":"captcha_required"} / polite block

PoW difficulty is clamp(bot_score, 12, 18) leading zero bits of sha256(challenge + ":" + nonce) — trivial at rung 2's floor, a few seconds of JS even at the cap. Each failed verify bumps that client one rung higher (capped at rung 3); a solve resets it. PoW issuance is rate limited to 5/s per network bucket (burst 10); when exhausted, rung-2 clients get the polite block and GET /__merlon/challenge/start answers 429. Since S1 that endpoint also only serves challenged clients: it spends the grant the WAF's rung-2 mint records alongside its embedded puzzle, so a cold ask answers 403 — puzzles, and therefore clearances, are never minted for free. Challenges are bound to the issuing network bucket (/32 v4, /56 v6): verification from a different bucket is rejected without consuming the challenge — same no-consume policy as a wrong nonce (only a correct solve consumes it) — so a legitimate retry from the issuing network still succeeds.

Chain order matters: the challenge endpoints must run before the WAF.

{"handle": [
  {"handler": "merlon_rating"},
  {"handler": "merlon_challenge"},
  {"handler": "merlon_waf", "site": "example.com"},
  {"handler": "reverse_proxy", "upstreams": [{"dial": "localhost:9000"}]}
]}
cookie visibility lifetime meaning
merlon_att JS-readable 30 min sensor script reported a consistent browser (feeds L3)
merlon_clr HttpOnly 1 h passed a challenge on this site (normalized Host); caps bot_score at 4 while valid

Tokens are HMAC-signed (keyring: <db dir>/merlon_trust_keys.json; rotation = append key + restart Caddy or caddy reload --force, since a plain reload of an unchanged config is a no-op and the key file is not part of it) and bound to client network bucket (/32, v6 /56), JA4 prefix, UA class — and, since the S1 token v2, to the issuing site (rules.NormalizeSite(Host)): a clearance earned on one hosted site never validates on another. The UA-class arm is strict on the presented side — a token minted with a class must present it back. A forged or cross-fingerprint token reads as sensor_state: replayed (+10).

Challenge rules use the new action (only progressive policy exists):

curl -X POST --localhost $MADMIN/id/merlon/sites/example.com/rules -d '{
  "phase": 2, "name": "challenge suspicious", "enabled": true,
  "matchers": {"bot_score": {"min": 10}},
  "action": {"type": "challenge", "policy": "progressive"}}'

First-match-wins applies to challenge too: a matched challenge rule whose score lands below rung1_min passes the request and suppresses later rules — evaluation does not fall through to a deny further down the list.

Challenge policy & honeypots
curl --localhost $MADMIN/id/merlon/settings/challenges
# {"rung1_min":9,"rung2_min":12,"rung3_min":16}

curl -X PUT --localhost $MADMIN/id/merlon/settings/challenges \
  -d '{"rung1_min":8,"rung2_min":12,"rung3_min":17}'

curl -X PUT --localhost $MADMIN/id/merlon/settings/honeypots \
  -d '{"paths":["/admin-login","/.env"]}'   # exact paths; publish them in robots.txt

curl -X POST --localhost $MADMIN/id/merlon/trust/bump_epoch

A honeypot hit answers 404 (illusion preserved), poisons the client IP for 24 h, and adds honeypot_hit (+12) to its score — fleet-wide on this edge, not per-site. The paths are never linked from your own pages; publishing them in robots.txt leaves them for crawlers only. bump_epoch instantly revokes all outstanding attestation/clearance tokens (global epoch; per-client epochs are planned).

Endpoints served by merlon_challenge: /__merlon/sensor.js, POST /__merlon/attest (accepted only for a client the sensor was served to, rate-limited per bucket), GET /__merlon/challenge/start (grant-gated — challenged clients only), POST /__merlon/challenge/verify, POST /__merlon/captcha/verify — note the two underscores in /__merlon/, a common typo when wiring these paths into other rules. Non-HTML clients receive the PoW puzzle as JSON (spec §6). Full endpoint semantics, token-claim layout and failure-escalation rules: docs/usage/challenges.md.

Feeds (M4)

Feeds ingest public IP-intelligence sources into per-source in-memory LPM tries that supply L1 scoring with per-prefix reputation, origin ASN, country, CAIDA tier and RPKI validity — replacing hand-maintained CIDR tables as the primary network data source. Everything is stdlib-only; no module dependencies were added for M4.

Feeds are off by default: the settings document starts as {"sources": []} and operators opt in per source. PUT settings/feeds validates and persists — it never fetches (that is what POST feeds/refresh is for). Sources live under the admin API key feeds — never in Caddy config:

The one-click route (item 33). The console's Feeds view carries a Recommended feeds card that applies a curated, versioned, licence-checked set — five RIR delegated-stats feeds, the published Cloudflare/AWS/Fastly lists, Akamai with an honest partial-coverage label, and the compiled-in GCP ranges. Posture, in one sentence: nothing phones home before your click, and the click itself fetches nothing — it stores a config document through the same PUT below; the ~30 MB/day fetch weight is disclosed before you click, and rows whose licence could not be verified at the primary source render as unverified, never as green. A fresh install still scores nothing until the operator clicks (or hand-writes the PUT) — that is the design, not a gap. Keyed sources (GeoLite2, HoneyLabs watchlist, ThreatFox) ride a masked write-only credentials lane in the same view. Full terms table, cadence floors, and the operator-supplied automation recipes: docs/usage/feeds.md §Recommended feeds.

curl --localhost $MADMIN/id/merlon/settings/feeds
curl -X PUT --localhost $MADMIN/id/merlon/settings/feeds -d '{
  "sources": [
    {"name": "ripencc", "adapter": "rir", "enabled": true,
     "url": "https://ftp.ripe.net/pub/stats/ripencc/delegated-ripencc-latest"},
    {"name": "feodo", "adapter": "threat", "enabled": true, "delta": 8,
     "url": "https://feodotracker.abuse.ch/downloads/ipblocklist.txt"},
    {"name": "geolite2", "adapter": "geo", "enabled": true,
     "unpack": "zip", "basic_auth": "user:pass",
     "license_key_env": "GEOLITE2_LICENSE_KEY",
     "files": [
       {"role": "country", "url": "https://download.maxmind.com/geoip/databases/GeoLite2-Country-CSV/download?suffix=zip"},
       {"role": "asn",     "url": "https://download.maxmind.com/geoip/databases/GeoLite2-ASN-CSV/download?suffix=zip"}
     ]}
  ]}'
Adapters
Adapter format (default) Input Contributes Default Δ
rir — RIR statistics exchange format v2 (`registry|cc|type|start|value|date|status[ extensions…], one allocation per line — the real delegated-*-latestlayout).startis a CIDR, alo - hirange (both expanded to the minimal CIDR cover), or a bare first address whosevaluegives the extent: IPv4 address count (non-CIDR counts included) or/nn` prefix length; IPv6 values are prefix lengths. PI/PC types set the provider-independent flag country, PI flag
threat iplist · threatfox-csv · honeylabs-csv plain IP/CIDR lines (# comments, Feodo-style trailing fields tolerated) · ThreatFox CSV export (ioc_type=ip rows only) · HoneyLabs exploiters CSV (ip,first_seen,last_seen,…; tor_exit rows skipped; the vendor's one-IP-per-line .txt variant parses as iplist) reputation delta +8 (per-source delta)
datacenter cidr · ipinfo-csv CIDR lines · CSV with network/country/category columns (rows kept when category contains hosting or cloud; all rows kept if no category column) datacenter flag + delta +10 (per-source delta)
geo — GeoLite2 Country + ASN CSV editions, roles country and asn, each a zip download (unpack: "zip", item 70) fetched with HTTP Basic (basic_auth: "user:pass", credential stored as account_id:license_key); every *Blocks*.csv member is parsed origin ASN, org, country 0
caida — as-organizations TSV (role orgs) + as-relations pipe format (role rels, required) — ASN-keyed, no prefix rows tier (1 transit … 4 customer-only) and org name via the resolved origin ASN 0
roa — JSON {"roas":[{"prefix": "…", "asn": N}]} RPKI validity, computed at lookup time +6 fixed when invalid
bgp — One MRT TABLE_DUMP_V2 RIB snapshot (RIPE RIS latest-bview.gz or any equivalent; the adapter sniffs the gzip magic itself since RIS serves bare gzip) — first path wins per prefix observed origin ASN (the strongest origin signal, see below) 0
cdn cloudflare-ips · fastly · akamai-acl · cloudfront-ips · gcp-static The five providers' published prefix lists (JSON/text; gcp-static compiles the GCP ranges in and needs no input at all) via_cdn provider overlay — labels the socket peer's CDN, never delta or origin 0 always
intel honeylabs-fingerprints-csv HoneyLabs fingerprints CSV — fingerprint lane: kind=ja4 rows only (akin and q… QUIC-derived values skipped and counted), upsert-then-sweep into fp_intel per exact source tag, no trie neutral fp_intel rows — display-only, zero score movement (feeds.md §HoneyLabs sources, ja4plus.md §HoneyLabs import lane) 0 always

Source names match ^[a-z0-9][a-z0-9_-]{0,63}$, must be unique, and each source needs at least one of url (http(s)), file (local path — offline imports and tests; the raw archive is skipped for it), or a files list of {"role","url"} entries; files takes precedence over url/file (the one exempt shape: cdn + gcp-static, which carries its input compiled in). Per-source delta is bounded to ±25. Note that name and adapter are different fields: registry lookups key on name (it is what lookup.matches[].source reports), so canonical operator config pairs them — {"name":"bgp","adapter":"bgp"}. The full source-field table is in docs/usage/feeds.md.

GeoLite2 credentials (item 70). GeoLite2 downloads are HTTP Basic against …/geoip/databases/{edition}/download?suffix=zip: username = your MaxMind account id, password = the licence key. The credentials lane keeps that as one value, account_id:license_key (same in the GEOLITE2_LICENSE_KEY env var when it is set, where it wins); the manager splits it at use, so no part of it ever rides a URL, a status row, a log line or an audit row — and a stored value with no : (a bare licence key) is refused with an error that says what form was expected instead of sending it. Other keyed sources still use the classic lane: URLs carrying {license_key} are substituted from the env var named by license_key_env. A missing or empty env var fails that source's fetch only: GET /id/merlon/feeds/status shows the error in last_error, the last snapshot (if any) keeps serving, and every other source refreshes normally. The expanded key never appears in status output or logs.

Merge semantics

Resolution happens per lookup. Within one source the most-specific prefix wins; across sources the deltas sum. For origin attributes the observed BGP origin wins (a bgp source), then geo, then rir (ASN, country) — spec §16 superseded the earlier geo>rir ruling; CAIDA tier and org resolve through the resolved origin ASN; the datacenter flag is set by any hit. RPKI validity is evaluated against the feed-derived origin ASN: a covering ROA that does not list it yields invalid and a fixed +6. Custom operator entries are just another source (custom) — fleet-wide, merged like any feed. The static network_ranges engine config still works (most-specific wins) and adds its signal on top of the feed signals.

Each matching source with a non-zero delta contributes an L1 signal named feed:<source> with that source's delta; the geo, rir and caida feeds contribute origin attributes (ASN, country, tier), not score. An RPKI-invalid prefix fires rpki_invalid (+6). So a curl from a datacenter IP listed by two threat feeds scores: base 1 + no_tls 4 + feed:datacenter 10 + feed:feodo 8 + ….

Scheduling & refresh

A source is due when interval_hours (default 168 = weekly) plus a deterministic jitter derived from the source name (< 24 h, same name → same offset every boot) has elapsed since its last fetch. A 15-minute ticker wakes the scheduler; nothing is ever fetched at Provision — startup only loads snapshots, so restarts are instant and offline-safe. The first fetch happens on POST /id/merlon/feeds/refresh or the first scheduler tick that finds the source due:

curl -X POST --localhost $MADMIN/id/merlon/feeds/refresh          # all enabled
curl -X POST --localhost $MADMIN/id/merlon/feeds/refresh \
     -d '{"source":"feodo"}'                                                    # one source
# [{"source":"feodo","ok":true,"entries":12345}]

curl --localhost $MADMIN/id/merlon/feeds/status
# {"sources":[{"source":"feodo","adapter":"threat","enabled":true,
#              "fetched_at":"2026-11-01T04:11:38Z","entries":12345}]}
# (a source never successfully fetched yet shows "fetched_at": null)

Refreshes are synchronous (admin API is loopback-only) and all of them — manual POSTs and scheduler ticks alike — serialize on a single lock (ruling R10). The two callers handle contention differently: a scheduler tick queues behind the lock, while POST /id/merlon/feeds/refresh never queues — it answers 409 refresh in progress instead (in bulk mode, sources already imported by that call come back as audited partial progress). A hung upstream therefore stalls scheduled refreshes for up to the fetch timeout (5 min per file, 30 min per refresh) but never turns the admin API into a waiting room. The data plane never waits either: imports build a new Registry and swap it atomically, so requests keep scoring against the last snapshot throughout.

Stale ≠ absent. A failed fetch, parse or snapshot write keeps the previous generation serving; the reason surfaces in last_error. Fetched bytes are archived under <dir(db_path)>/feeds/raw/, newest 7 per source kept; compiled snapshots (gzip JSON) live in <dir(db_path)>/feeds/ and are reloaded at startup.

Custom operator entries

Your own reputation list, merged like any feed and stored in SQLite (feed_custom table). PUT is a full replacement (honeypot-paths precedent): max 10 000 entries per write, delta bounded ±25, prefixes canonicalized (127.0.0.1/8 → 127.0.0.0/8), duplicates rejected:

curl -X PUT --localhost $MADMIN/id/merlon/feeds/custom -d '{
  "entries": [
    {"prefix": "192.0.2.0/24", "delta": -3, "reason": "office-vpn"},
    {"prefix": "203.0.113.66/32", "delta": 4, "reason": "repeat-offender"}
  ]}'
The lookup probe

GET /id/merlon/lookup?ip=… runs the full merge against the live registry and always answers 200 for a valid IP (400 otherwise) — empty matches and "rpki":"unknown" when no ROA covers the address or its origin ASN could not be resolved:

{"ip": "203.0.113.66", "origin_asn": 6450, "country": "de", "tier": 2,
 "datacenter": true, "rpki": "invalid", "delta": 28,
 "matches": [
   {"source": "custom",     "prefix": "203.0.113.66/32", "delta": 4},
   {"source": "datacenter", "prefix": "203.0.113.0/24",  "delta": 10},
   {"source": "feodo",      "prefix": "203.0.113.66/32", "delta": 8},
   {"source": "rpki",       "prefix": "203.0.113.0/24",  "delta": 6}]}

The asn and country rule matchers consume exactly this context — and fail closed: with no feeds (or no covering entry), they never match.

GET /id/merlon/registry/asn/{asn}?limit=… is the reverse drill (B1): which prefixes does origin AS asn announce in the live bgp feed? It answers {"asn":…,"total":N,"prefixes":[…]} — limit defaults to 200 and is capped at 5000 (non-numeric or ≤ 0 → 400, as is asn 0), and without a bgp source total is simply 0.

ROA exports

The roa adapter wants the tiny JSON shape {"roas":[{"prefix","asn"}]}; a one-liner converts an NTT-style registry export (array of objects each carrying a prefixes list):

curl -sf <your-peering-export.json> \
  | jq '{roas:[.[] | .asn as $a | .prefixes[]? | {prefix:(.prefix // .), asn:(.asn // $a)}]}' \
  > /var/lib/merlon/roas.json

…then point a roa source at the file ("file": "/var/lib/merlon/roas.json").

BGP: RIB snapshots in, live monitoring out. M4 explicitly dropped BGP; B1 shipped the ingestion (spec §16) — an MRT TABLE_DUMP_V2 reader and the bgp adapter, so one RIPE RIS latest-bview.gz (or any equivalent dump) gives Merlon the full prefix→observed-origin table, which is the strongest origin-ASN signal and the key to registry forensics. Still out of scope: live BGP (no BMP/update feeds — snapshots only) and announcement-vs-allocation mismatch detection; RPKI validity is still evaluated against the operator-supplied ROA export above, and with no bgp source the origin ASN falls back to GeoLite2-ASN. None of it added a dependency: the MRT parser is in-repo Go.

Workers & routing (M5a)

Phase 3 opened: a rule can now route matching requests to a JavaScript worker running inside the edge process — no origin round-trip, no restart. Code is stored versioned in SQLite and dispatched by merlon_worker_router, placed after merlon_waf. Full chain order:

merlon_rating -> merlon_challenge -> merlon_waf -> merlon_worker_router -> origin
{"handle": [
  {"handler": "merlon_rating"},
  {"handler": "merlon_challenge"},
  {"handler": "merlon_waf", "site": "example.com"},
  {"handler": "merlon_worker_router"},
  {"handler": "reverse_proxy", "upstreams": [{"dial": "localhost:9000"}]}
]}

The router reads the same per-site snapshot as the WAF but evaluates phase 3 only (ruling R4): a phase-1/2 bypass skips enforcement yet still flows through the router — routing is dispatch, not blocking. A site with no rules passes through allocation-light; a request whose route rule matches gets its worker's response and never reaches the origin. If the named worker was never deployed (or its invoke fails), the router answers 502 (worker unavailable / worker error) rather than falling through behind a matched rule. Observe mode works for route rules too: counted by GET /id/merlon/observe/matches, never dispatched. Note that a route rule's actions counter ticks on successful dispatches only — both 502 paths return before it, so neither counter moves and a throwing worker looks like a non-matching rule in GET /id/merlon/observe/matches (unlike WAF rules, whose actions count every match, before enforcement).

Route rules

route is a phase-3-only action naming a per-site worker (^[a-z0-9][a-z0-9_-]{0,63}$, same grammar as feed source names):

curl -X POST --localhost $MADMIN/id/merlon/sites/example.com/rules -d '{
  "phase": 3, "name": "demo route", "enabled": true,
  "matchers": {"path": {"prefix": "/worker-demo"}},
  "action": {"type": "route", "worker": "echo"}}'
The handle(req) contract

A worker is one JS source file that defines function handle(req) — called once per request in a fresh sandbox; return a response object (or a bare string, taken as the body with status 200):

function handle(req) {
  return {
    status: 200,                                   // out-of-range/omitted => 200
    headers: {"content-type": "text/plain"},       // Content-Length/-Encoding stay host-owned
    body: "hello from " + req.path + " (score " + req.bot_score + ")"
  };
}

req fields: method, path, query, headers (object of arrays), body (string, truncated at 64 KiB), site, client_ip, bot_score. The response body is capped at 1 MiB. A handle that throws yields a 502; the guest message (with stack) is logged, capped at 512 B. The key __error in a returned object is reserved for failure signalling — workers cannot use it as data. Math.random() throws unconditionally (deterministic sandbox): QuickJS seeds its PRNG from an epoch-0 clock, so sequences are identical across instances — never usable for secrets. Field-by-field ABI reference (workers/request.go WorkerRequest): docs/usage/workers.md.

Sandbox & limits (rulings R1/R2/R3)

Guest code runs in QuickJS-NG under wazero, one fresh module instance per invocation — never pooled, no state survives a request. Limits:

Limit Value On breach
Wall-clock deadline per invoke 5 s (1 s for the deploy trial run) execution killed — 502 on a request; 400 at deploy for the trial run
Guest linear memory 64 MiB allocation fails / invoke error
Worker source size 1..512 KiB 400 at deploy
Request body into guest 64 KiB (truncated) debug log only
Response body 1 MiB invoke error

The guest artifact has zero WASM imports — no filesystem, sockets or clocks reach the host (deny-all even inside QuickJS), and there is deliberately no fetch/KV API in M5a: workers are pure request→ response transforms. Network egress and key-value storage from workers are deferred (ruling R3, backlog).

Guest artifact & provenance

The sandbox embeds one prebuilt WASM module (workers/wasm/quickjs-merlon.wasm): QuickJS-NG v0.11.0 (MIT) compiled with wasi-sdk 25 (-nostartfiles, WASI syscall stubs deny-all). The MIT license text ships with the module source as workers/wasm/LICENSE.quickjs (not embedded in the Go binary — only the wasm artifact itself is); PROVENANCE records versions, exact compile/link lines and sha256 (mirrored in PROVENANCE.sha). Rebuild reproducibly with workers/guest/build.sh (fetches pinned toolchain + sources; the Go build never invokes it — go:embed ships the committed artifact).

Deploy & rollback

Deploy trial-runs the code first (must load and define handle within 1 s) and only persists on success — a broken worker cannot be deployed (400 with the guest message, capped at 512 B). Versions are immutable; deploy appends, rollback flips the current pointer:

# deploy => {"name":"echo","version":1} (201)
curl -si -X POST --localhost \
     $MADMIN/id/merlon/sites/example.com/workers/echo/deploy \
     -d '{"code":"function handle(req){return {body:\"v1: \"+req.path}}"}'

# rollback => current pointer back to version 1 (unknown worker/version: 404)
curl -s -X POST --localhost \
     $MADMIN/id/merlon/sites/example.com/workers/echo/rollback \
     -d '{"version":1}'

Deploying again appends version 2 — versions are immutable, so every past one stays fetchable through …/versions (history, newest first, no code bodies) and …/code?version=N (raw source; …/code alone is the current one).

Deploys and rollbacks are audited (workers.deploy / workers.rollback kinds with before/after versions). A deploy takes effect on the next request: worker code is cached in-process for the routing hot path, and every deploy or rollback refreshes that cache — only a cold miss reads SQLite.

CAPTCHA (rung 3)

Rung 3 upgrades from polite block to an interactive Friendly Captcha v1 widget when a provider is configured — no SDK, plain net/http against a self-hosted solve server (any Friendly-Captcha-compatible deployment serving the /fc.min.js widget script and the siteverify endpoint). The settings document lives under the SQLite settings key captcha:

{"provider": "friendlycaptcha", "enabled": true,
 "solve_url": "https://solve.example.net/siteverify",
 "site_key": "0x00000000000000000000000",
 "secret_env": "MERLON_CAPTCHA_SECRET"}

The verification secret is never stored — secret_env names the environment variable that holds it, re-read on every verify and never logged (only the variable name appears in logs). The client IP is deliberately not forwarded to the solve server (privacy). Verification is fail-closed: unset secret, transport error, timeout (5 s cap), non-200 or malformed response all mean "not solved"; redirects are never followed (the secret is never re-sent). Siteverify speaks Friendly Captcha v1: form fields secret, solution, sitekey. There is no admin endpoint for this setting in M5a: seed/edit the captcha row of the settings table with any SQLite client and restart (or reload) Caddy — an enabled-but-invalid document fails Provision, a disabled or absent one is inert.

Rung-3 behavior:

provider state HTML GET client JSON/API client
configured & enabled CAPTCHA page (widget from solve_url origin, POSTs the token to POST /__merlon/captcha/verify) 403 {"error":"captcha_required"}
unset or disabled polite block page (formatted document since M20) 403 polite block (text/plain, unchanged from M3, byte-for-byte)

A correct solve answers {"ok":true,"redirect":"/path"} and sets the same HttpOnly clearance cookie a PoW solve grants (merlon_clr, 1 h, caps bot_score at 4); a failed verify escalates the client one rung higher like every other challenge failure. An empty token is rejected 400 without escalation; with no provider configured the endpoint answers 404.

Optional Coraza WAF engine (M5b)

merlon_waf can hand matched requests to an embedded Coraza (ModSecurity-compatible) rule engine. It is off by default and costs nothing until enabled; coraza is the only third-party engine dependency in the data plane, confined to app/. The engine config — an enabled flag plus raw SecLang directives — lives in a SQLite settings row and hot-swaps atomically through the admin API, no reload:

# enable with two inline phase-1 deny rules
curl -X PUT --localhost $MADMIN/id/merlon/settings/coraza -d '{
  "enabled": true,
  "directives": "SecRuleEngine On\nSecRule REQUEST_URI \"@streq /wp-login.php\" \"id:1,phase:1,deny,status:403\"\nSecRule REQUEST_HEADERS:User-Agent \"@rx ^sqlmap\" \"id:2,phase:1,t:lowercase,deny\""}'

# read back — {"enabled":false} while unset
curl --localhost $MADMIN/id/merlon/settings/coraza

# disable (full-document replace: engine dropped, directives cleared)
curl -X PUT --localhost $MADMIN/id/merlon/settings/coraza \
  -d '{"enabled":false}'

The PUT builds the engine before persisting: uncompilable SecLang gets a 400 and leaves both storage and the live engine untouched (store failures → 500); successful writes are audited as kind settings.coraza. An enabled-but-empty directive set arms a valid, inert engine — enabling before writing rules blocks nothing instead of failing to boot.

On the data plane, a phase-2 coraza rule in your normal ruleset gates which traffic enters the engine; the SecLang decides what is blocked:

curl -X POST --localhost $MADMIN/id/merlon/sites/example.com/rules -d '{
  "phase": 2, "name": "coraza gate", "enabled": true,
  "matchers": {"path": {"prefix": "/"}},
  "action": {"type": "coraza"}}'

Since B9-T6 the settings document also carries an optional rule-sets pool — named engines ({"rule_sets":[{"name":"owasp","directives": "Include /etc/merlon/crs/rules/*.conf"}]}, names [a-z0-9-]{1,32}) that a rule selects with "action": {"type": "coraza", "profile": "owasp"} — so one site can run OWASP while another runs your own rules. Unknown or removed profiles fail open like a disabled engine; a path-backed profile that fails to compile at boot is skipped with an ERROR naming it while the rest of the edge serves. Pool semantics, the per-site recipe and an OWASP CRS provisioning guide (with its honest coverage delta): docs/usage/waf.md.

A matched coraza rule consumes its phase-2 slot even when the engine passes the request — like every enforce action it is first-enforce-match- wins, so later deny/challenge rules in phase 2 never evaluate for that request (put stricter fallbacks in a different phase if you need them).

What the engine sees (v1 limitation). Each matched request opens a Coraza transaction fed URI, method and headers only — request bodies are never read (buffering them would break streaming and WebSocket upgrades on Caddy's hot path). Phase-1 rules on REQUEST_URI, query args and headers work; body-dependent rules (ARGS_POST, REQUEST_BODY) can never fire, and SecLang rules in coraza phases 2–5 compile but never evaluate — only the request-headers phase runs (no body phases, no logging phase). Go promotes Host out of the header map and folds Transfer-Encoding into its own field — both are re-fed to the transaction, so header rules see the full wire set (only syntactically valid Transfer-Encoding tokens get this far at all — Go's net/http rejects malformed ones before the handler runs). REMOTE_ADDR is populated (IPv6 brackets stripped); SERVER_NAME is unset in v1 — don't match on it.

Interruptions become responses. A coraza deny/drop answers locally with the engine's status (default 403) and body blocked — note this differs from plain merlon deny's blocked by merlon. A coraza redirect sets Location and uses 301/302/303/307; anything unexpected degrades to 302.

Fail-open on the data plane, fail-closed at boot. With the engine disabled or unset, a matched coraza rule simply passes the request through — a config gap must never black-hole a site. Conversely, a corrupt settings row or an uncompilable enabled config fails Provision (Caddy refuses to boot), the same discipline as the captcha and scoring rows.

No CRS bundled. Merlon ships without the OWASP Core Rule Set — rules come from your directives and, since B9-T6, from your own on-disk rule trees via profile Includes. For a pinned CRS v4 provisioning recipe — and the plain statement of which CRS families actually fire under a phase-1-only data plane — see docs/usage/waf.md; upstream engine details live in the coraza documentation.

The console's Coraza view mirrors exactly these three shipped limitations (phase-1 data plane, fail-open with no engine, full-document-replace on {"enabled":false}) — see docs/usage/console.md.

Web console (M5b)

GET http://<admin-addr>/id/merlon/console/ serves the embedded Svelte 5 console — thirteen hash-routed views over the admin API: Overview (feed health, observe counters), Rules (CRUD, mode toggle, reorder), Workers (CodeMirror editor, deploy/rollback/version history, per-worker static asset bundles — manifest, upload, delete, wipe, SPA-fallback flag), Scoring, Challenges, Coraza, Settings (client-IP trust, identity lane, WS frame-tee policy, JA4+ feature gates, honeypot trap paths, bump-trust-epoch danger card with a typed confirmation, export-bundle download card), Feeds (source forms, bulk refresh with per-source results, custom-prefix table), FP Lane (fp_block table, fired candidates, rotation-lane editor), FP Intel (fingerprint-intel registry: list + filters, CRUD, tolerant query, bulk import with a per-row errors table), IP Lookup, Audit log (client-side kind/site filters), and the Live Inspector — a capped, auto-scrolling table fed by the /inspect/live SSE stream (opening the tab is what enables publishing; the rating handler only emits while a subscriber is attached). The bolded panels are the M23 (wave-1) coverage fold; per-view behavior, every claim cited by symbol, is docs/usage/console.md.

Coverage (pre-positioning honesty, design 20 §3). The console covers every admin lane except:

  • the Web Bot Auth lane — GET/PUT settings/webbotauth and POST webbotauth/probe
  • the Pages lane — GET/PUT settings/pages
  • the network-violation-history surface — GET/DELETE net/rep

Those seven routes stay admin-API-only until their panels ship — the curl reference is admin-api.md (the per-net table + badges already carry a console hand-off spec). This list, kept identical in console.md, is what stands between today's honest "covers every lane except…" posture and any stronger one; docs-and-gate discipline, not vibes (item 20 R-20-precondition).

The console has no auth of its own: it is reachable exactly where the admin API is and rides that listener's network protection — loopback by default, so reach it over SSH/VPN/Tailscale (spec §9: zero extra auth surface). Whatever can reach the admin port is an admin.

Rules-view selector to know: the site field is free text persisted to localStorage.site; with nothing stored yet it defaults to the first site of a POST /export bundle when that call succeeds, otherwise type the site key manually (the server normalizes it anyway).

Screenshots & per-view reference. Every view — plus the attestation, proof-of-work and polite-block pages — is screenshotted and annotated one section per view in docs/usage/console.md, which also documents the deterministic regeneration command. The committed PNGs live in docs/images/ and regenerate byte-identically (verified across separate go run invocations).

Build & ship. console/build.sh (maintainer-only, node ≥ 22) runs npm ci && npm run build in console/ui/ and syncs the output to console/dist/ — the committed build artifact go:embed bakes into the binary (the Go build never invokes npm, same rule as workers/guest/). Flow: edit console/ui/, run console/build.sh, commit console/ui and console/dist in the same commit.

Dev flow. npm run dev in console/ui/ serves Vite with HMR. Point it at the admin API on the same origin, or add a local proxy for /id/merlon/* → http://127.0.0.1:2020 (your admin.listen) to your own vite.config.js — see scope ruling (d).

v1 scope rulings — the console is functional-admin, not the spec's pixel design; the §9 ideal remains valid direction for a later polish milestone:

  • (a) rules edit in a validated JSON editor holding the full wire shape — not a 1:1 matcher form;
  • (b) reorder is up/down buttons (POST reorder with the full ordered id list) — not drag-and-drop;
  • (c) settings are structured forms where fields are few (challenge rung thresholds, coraza enable + SecLang box) and JSON editors elsewhere (the scoring EngineConfig document) — no slider dashboards;
  • (d) no committed vite dev-proxy config — this README pointer is all there is;
  • the live inspector streams scored requests only (rating stage: score + per-signal breakdown) — enforcement actions and worker responses are not part of the SSE event shape yet.

Modules

Module ID Namespace Purpose
merlon core app (apps.merlon) Shared lifecycle/state for Merlon modules; owns the feed manager, snapshots and refresh scheduler
tls.context.merlon tls.context Captures the JA4 fingerprint from every TLS ClientHello
http.handlers.merlon_rating HTTP handler Scores requests (L1+L2+L3) and annotates vars/context/logs
http.handlers.merlon_challenge HTTP handler Serves the /__merlon/* trust endpoints: sensor script, attestation beacon, PoW start/verify, captcha verify (POST /__merlon/captcha/verify)
http.handlers.merlon_waf HTTP handler Enforces per-site SQLite rulesets: bypass/deny/redirect/challenge/coraza + observe mode + honeypot trap — the only blocking module
http.handlers.merlon_worker_router HTTP handler Dispatches phase-3 route rules to workers in the QuickJS-NG sandbox; 502 when the named worker cannot serve
admin.api.merlon admin router Rule CRUD with ETags, reorder, observe counters, challenge policy, honeypots, epoch revocation, feeds settings/refresh/status/custom, the IP lookup probe and the ASN→prefixes registry drill, worker deploy/rollback/list/versions/code, scoring hot-edit, coraza engine settings, live-inspect SSE, audit listing and config export — over Caddy's admin API
caddy.listeners.merlon listener wrapper Per-source-IP handshake rate limiting (kills connection storms)

Every module's config fields, defaults and provisioning behavior (including where the SQLite file, the trust keyring and the feed snapshots land): docs/usage/configuration.md.

TLS fingerprint capture

Attach the handshake hook to a server's TLS connection policies so JA4 fingerprints are computed at handshake time (Caddy's handshake_context hook is experimental and may change upstream):

"tls_connection_policies": [
  { "handshake_context": {"module": "merlon"} }
]

With this enabled, a browser-UA client whose JA4 is not in known_browser_ja4 fires ua_browser_ja4_unknown (+6); a known-browser JA4 with a browser UA stays at score 1.

Handshake rate limiting

Wrap a server's listener to cap new connections per source IP — over-budget connections are closed at accept time (config fields sit alongside "wrapper"):

"servers": {
  "srv0": {
    "listen": [":8080"],
    "listener_wrappers": [
      { "wrapper": "merlon", "per_ip_per_sec": 50, "burst": 100 }
    ],
    "routes": [ ... ]
  }
}

Scoring model

score = clamp(1 + Σ signal deltas, 1, 20), base 1. Key signals:

Signal Layer Default Δ Fires when
no_tls L2 +4 Plaintext request with no captured fingerprint
ua_script L2 +8 UA matches known HTTP libraries (curl, python-requests, …)
ua_browser_ja4_unknown L2 +6 Browser UA but JA4 not in known_browser_ja4
ja4_partial / tls_abnormal:* L2 +1 / +1.5 Degenerate or abnormal ClientHello
velocity_high L1 +4 Per-IP request rate over configured budget
sensor_valid / sensor_stale / sensor_replayed L3 −6 / +8 / +10 Attestation token verified / expired-or-revoked / forged-or-cross-fingerprint
honeypot_hit L3 +12 Client IP poisoned by a honeypot visit (24 h)
feed:<source> L1 per-source Δ Client IP matches feed source <source> (most-specific prefix; deltas sum across sources — see Feeds)
rpki_invalid L1 +6 A feed ROA covers the client prefix but not its resolved origin ASN
(configured ranges) L1 your Δ Client IP in a network_ranges CIDR (most specific wins)

A valid clearance cookie caps the finalized score at 4 (never below 1).

The complete weight tables (fingerprint / trust / velocity defaults, each applyDefaults), EngineConfig keys and the JSON-vs-SQLite precedence rules: docs/usage/scoring.md.

Metrics

Merlon is a Prometheus exporter: every collector rides the admin listener's /metrics endpoint (pinned 127.0.0.1:2020). The full registry — every merlon_* family with type, labels, meaning and example PromQL, the scrape posture (on-host static_configs; remote only via tailnet/tunnel, since the admin plane carries no auth and its reach is the access control), the documented-absent merlon_build_info, and what is deliberately not measured here — lives in docs/usage/metrics.md.

Performance

The scoring path is built for the hot loop: regexes are compiled once, maps are bounded (velocity buckets cap tracked IPs and evict LRU), and each score costs ~5 small allocations.

go test ./score/ -bench=EngineScoreSilentPath -benchtime=2s -count=3

Measured: 6,976 ns/op (≈7 µs; 287 B/op, 5 allocs/op; median of three -count=3 runs) on a 13th Gen Intel Core i9-13900K, linux/amd64, Go 1.26 — comfortably ≪ the <1 ms p99 end-to-end budget.

The M2 milestone gate re-checked this baseline (rules store, WAF handler, limiterx unification, fingerprint attribution) by A/B benchmarking M1 commit d82b02c against M2 HEAD interleaved on the same machine: 11,138 vs 11,006 ns/op medians — parity (≈−1%), allocations unchanged at 5/op. Absolute numbers scale with host load; run the gate quiet for comparable absolutes.

M3 re-ran the same gate on this host: 11,023 / 10,875 / 10,857 ns/op (-benchtime=2s -count=3), median 10,875 — parity with the 11,006 M2 baseline (−1.2%), allocations unchanged at 5/op. The trust layer is wired per-request behind optional deps; the silent-path engine pays nothing for it.

M4 re-ran it again: -benchtime=2s -count=5, median 11,791 ns/op (allocations unchanged at 5/op). Absolute numbers drifted with host load that day — an interleaved A/B against the M3 base commit on the same machine in a quiet window measured 10,916 (M4) vs 10,943 ns/op (M3 base): parity (−0.2%). The feed hook sits behind a nil-checked optional dep; with feeds loaded, resolution is a binary search per source (~µs) and not gated.

M5a A/B-interleaved against the M4 tip (d427dab) in parallel worktrees, 3 rounds of -benchtime=2s -count=5 per side (15 samples each): medians 11,385 ns/op (M5a) vs 11,335 ns/op (M4 base) — parity (+0.4%), allocations unchanged at 5/op, under heavy concurrent host load (loadavg ≈ 8–18; absolutes inflated accordingly, the interleave is what counts). The worker runtime compiles its guest once at Provision and sits entirely off the scoring path.

Development

House style (applied Uber Go subset, review-enforced rules, frozen surfaces): docs/dev/style.md.

go vet ./...
CGO_ENABLED=0 go test -p 1 ./...

Run tests with -p 1. Several caddytest-based tests share the Caddy admin endpoint localhost:2019; when test binaries run in parallel they race for that port. Serializing packages with -p 1 keeps the suite green.

Kill stray caddy processes before running tests. A leftover caddy from a previous run or build check squats :2019 (and :8080), making unrelated packages fail. Check with pgrep -a caddy and kill anything it lists.

Pre-M3 databases may contain mixed-case site keys written before site normalization; those rows are unreachable from the data plane and can be removed with any SQLite client.

Pre-M3 databases may also contain rules that the tightened M3 grammar no longer accepts — bot_score.min:0 or blank ja4_prefixes entries. Those rows fail snapshot compilation and Caddy refuses to boot; remove or edit them with any SQLite client. (ws_upgrade:false used to be on this list; negation landed in M5a and such rows are valid grammar again.)

Pre-M5b databases open cleanly: no schema additions at all — the coraza config is a plain settings row (coraza) that appears on the first PUT /id/merlon/settings/coraza; until then the engine is unset and every matched coraza rule fails open.

Pre-M5a databases may contain phase-3 rules with non-route actions — the pre-M5a grammar allowed phases 1–3 but rejected only the route action itself, so legacy deny/challenge rows in phase 3 can exist. M5a validation inverts that: phase 3 accepts route exclusively, and such legacy rows fail snapshot compilation and block boot; delete them or move them to phase 1–2 with any SQLite client.

Pre-M5a databases otherwise open cleanly: the only schema additions are the workers and worker_versions tables, created on first open (CREATE TABLE IF NOT EXISTS) — nothing changes until you deploy a worker.

Pre-M4 databases open cleanly: the only schema addition is the feed_custom table, created on first open (CREATE TABLE IF NOT EXISTS), and feeds default to off — nothing changes until you add sources.

For the full-stack suites, the local gate script (scripts/ci-gate.sh) and the make gate* targets — and the e2e suite's own ground truth in e2e/README.md — see docs/usage/operations.md.

Design & spec

Full design and roadmap: design spec · M1 plan.

License

Merlon is MIT — LICENSE — with one deliberately fenced carve-out: internal/ja4h and internal/ja4t implement FoxIO's JA4+ family names and published specification (JA4H, JA4T), and FoxIO licenses the JA4+ family for non-commercial purposes. The project takes that same posture for the island instead of pretending it away: the FoxIO License 1.1 text ships as LICENSE-JA4PLUS and NOTICE states the coverage — the implementations are clean code written from the spec text, the carve-out applies to the generators and the name/spec they implement, and consuming fingerprint data is license-free: the values you compute and the fp_intel rows you import are your data. The JA4 base in ja4/ stays BSD-3-clean, and Merlon neither bundles nor fetches any FoxIO data (ruling R-b10-nocsv). Non-commercial FLOSS posture, stated once, everywhere it applies.

Documentation

Overview

Package merlon is the umbrella package for the Merlon edge security platform. Import it (blank) to register every Merlon Caddy module: each module-bearing package in the list below self-registers through its caddy.RegisterModule init (adminapi, app, handlers, listeners, tlsctx, wstee), and caddyfile layers in the Caddyfile vocabulary. The blank-import list is the registration roster — a new module-bearing package must be added here for the umbrella import to keep its promise. Design: docs/superpowers/specs/2026-10-29-merlon-design.md

Directories

Path Synopsis
Package adminapi serves merlon's Tier-2 dynamic config over Caddy's admin API (spec §8): rule CRUD with ETags and an audit log, plus the settings lanes, workers with their schedules and assets, the fp_block and fp_intel lanes, live inspection over SSE, configuration export and the web console SPA — a non-exhaustive enumeration; API.Routes is the one pinned route table.
Package adminapi serves merlon's Tier-2 dynamic config over Caddy's admin API (spec §8): rule CRUD with ETags and an audit log, plus the settings lanes, workers with their schedules and assets, the fp_block and fp_intel lanes, live inspection over SSE, configuration export and the web console SPA — a non-exhaustive enumeration; API.Routes is the one pinned route table.
Package app implements the merlon core Caddy app: shared state for all merlon modules — the SQLite rule store, compiled per-site rule snapshots, and observe counters (spec §2).
Package app implements the merlon core Caddy app: shared state for all merlon modules — the SQLite rule store, compiled per-site rule snapshots, and observe counters (spec §2).
Package caddyfile registers Merlon's Caddyfile vocabulary — the "merlon" global option and the merlon_rating / merlon_waf / merlon_challenge / merlon_worker_router site directives — with Caddy's built-in config adapter.
Package caddyfile registers Merlon's Caddyfile vocabulary — the "merlon" global option and the merlon_rating / merlon_waf / merlon_challenge / merlon_worker_router site directives — with Caddy's built-in config adapter.
Package console embeds the built web console SPA (spec §9) and serves it on the admin port: adminapi/console.go mounts Handler behind http.StripPrefix("/id/merlon/console", …).
Package console embeds the built web console SPA (spec §9) and serves it on the admin port: adminapi/console.go mounts Handler behind http.StripPrefix("/id/merlon/console", …).
Package feeds ingests public IP-intelligence sources — RIR allocations, threat lists, datacenter ranges, GeoLite2 blocks, CAIDA relations, RPKI ROAs, CDN edge ranges and BGP RIB snapshots, one Adapter per family registered at init — into per-source LPM tries merged into a request-path Registry (spec §4).
Package feeds ingests public IP-intelligence sources — RIR allocations, threat lists, datacenter ranges, GeoLite2 blocks, CAIDA relations, RPKI ROAs, CDN edge ranges and BGP RIB snapshots, one Adapter per family registered at init — into per-source LPM tries merged into a request-path Registry (spec §4).
Package handlers implements merlon Caddy HTTP handler modules: merlon_rating (scoring), merlon_challenge (trust endpoints), merlon_waf (rule enforcement) and merlon_worker_router (phase-3 worker dispatch).
Package handlers implements merlon Caddy HTTP handler modules: merlon_rating (scoring), merlon_challenge (trust endpoints), merlon_waf (rule enforcement) and merlon_worker_router (phase-3 worker dispatch).
internal
clientip
Package clientip resolves which client IP an HTTP request is ABOUT.
Package clientip resolves which client IP an HTTP request is ABOUT.
cronx
Package cronx is a minimal dependency-free 5-field cron expression parser for B7 cron-scheduled worker runs.
Package cronx is a minimal dependency-free 5-field cron expression parser for B7 cron-scheduled worker runs.
fpintel
Package fpintel is the operator-managed fingerprint-intelligence registry (B10 spec §7.3): the validation grammar shared by the admin surface, and the atomically-swapped immutable snapshot the trust lane queries per request.
Package fpintel is the operator-managed fingerprint-intelligence registry (B10 spec §7.3): the validation grammar shared by the admin surface, and the atomically-swapped immutable snapshot the trust lane queries per request.
hostscope
Package hostscope is the Host-header preparation seam for the per-site scoped route plane (design 15 §7.1, Phase 1 scope).
Package hostscope is the Host-header preparation seam for the per-site scoped route plane (design 15 §7.1, Phase 1 scope).
inspectx
Package inspectx is a bounded in-process fan-out hub for live request inspection (spec §9 Live inspector, spec §8 /inspect/live).
Package inspectx is a bounded in-process fan-out hub for live request inspection (spec §9 Live inspector, spec §8 /inspect/live).
ja4h
Package ja4h computes the JA4H HTTP client fingerprint (FoxIO JA4+ family, https://github.com/FoxIO-LLC/ja4) from a Go request — pure computation over net/http, stdlib-only, no capture logic.
Package ja4h computes the JA4H HTTP client fingerprint (FoxIO JA4+ family, https://github.com/FoxIO-LLC/ja4) from a Go request — pure computation over net/http, stdlib-only, no capture logic.
ja4t
Package ja4t computes the JA4T TCP-stack fingerprint (FoxIO JA4+ family, https://github.com/FoxIO-LLC/ja4) from a saved SYN — a pure byte parser over the IPv4/IPv6 + TCP header pair, plus the per-connection store the capture module writes into.
Package ja4t computes the JA4T TCP-stack fingerprint (FoxIO JA4+ family, https://github.com/FoxIO-LLC/ja4) from a saved SYN — a pure byte parser over the IPv4/IPv6 + TCP header pair, plus the per-connection store the capture module writes into.
ja4x
Package ja4x computes the JA4X certificate fingerprint (FoxIO JA4+ family, https://github.com/FoxIO-LLC/ja4) from an already-parsed TLS peer chain — a pure x509/ASN.1 parse over crypto/x509, stdlib-only, no capture logic.
Package ja4x computes the JA4X certificate fingerprint (FoxIO JA4+ family, https://github.com/FoxIO-LLC/ja4) from an already-parsed TLS peer chain — a pure x509/ASN.1 parse over crypto/x509, stdlib-only, no capture logic.
limiterx
Package limiterx provides bounded per-key token buckets with O(1) LRU eviction, shared by score.Velocity and listeners ConnTap (M2 unification of the M1-duplicated bucket maps; reviewer-directed).
Package limiterx provides bounded per-key token buckets with O(1) LRU eviction, shared by score.Velocity and listeners ConnTap (M2 unification of the M1-duplicated bucket maps; reviewer-directed).
metricx
Package metricx holds Merlon's custom Prometheus collectors (B7, design §Metrics).
Package metricx holds Merlon's custom Prometheus collectors (B7, design §Metrics).
pagekit
Package pagekit renders merlon's formatted interstitial pages — the attest, proof-of-work, captcha and polite-block documents that human browsers see on customer domains (design 13).
Package pagekit renders merlon's formatted interstitial pages — the attest, proof-of-work, captcha and polite-block documents that human browsers see on customer domains (design 13).
reqctx
Package reqctx carries the per-request identity the merlon WAF middleware resolves — the ATTRIBUTED client (trusted-peer resolution per B8) and the rules-plane SITE key — down to handlers that never see the middleware's locals, notably the B9 WS frame-tee transport (ruling R-b9-ws-ctx).
Package reqctx carries the per-request identity the merlon WAF middleware resolves — the ATTRIBUTED client (trusted-peer resolution per B8) and the rules-plane SITE key — down to handlers that never see the middleware's locals, notably the B9 WS frame-tee transport (ruling R-b9-ws-ctx).
tokens
Package tokens is the site-token library for the per-site scoped route plane (design 15 §7.2, R-15-tokens — the FROZEN format ruling).
Package tokens is the site-token library for the per-site scoped route plane (design 15 §7.2, R-15-tokens — the FROZEN format ruling).
ttlx
Package ttlx observes the two network-stack facts Merlon can read honestly about an accepted TCP connection, with no new privileges and no packet tap:
Package ttlx observes the two network-stack facts Merlon can read honestly about an accepted TCP connection, with no new privileges and no packet tap:
webbotauth
Package webbotauth implements the verification path of the draft-ietf-webbotauth-httpsig-protocol-01 profile of RFC 9421 HTTP Message Signatures: header parsing, profile enforcement, signature-base reconstruction and cryptographic verification against keys supplied by a caller-provided KeySource.
Package webbotauth implements the verification path of the draft-ietf-webbotauth-httpsig-protocol-01 profile of RFC 9421 HTTP Message Signatures: header parsing, profile enforcement, signature-base reconstruction and cryptographic verification against keys supplied by a caller-provided KeySource.
wsframe
Package wsframe implements an RFC 6455 frame observer for the Merlon WebSocket frame tee (B9).
Package wsframe implements an RFC 6455 frame observer for the Merlon WebSocket frame tee (B9).
Package ja4 implements TLS client fingerprinting per the JA4 specification (https://github.com/FoxIO-LLC/ja4): Compute turns a TLS ClientHello into a Fingerprint, and the ConnStore carries that spec-compliant value from the handshake to every request on the same connection; ExtractFromState is the best-effort partial fallback when no handshake fingerprint survives, and Fingerprint.IsBrowserLike flags profiles outside typical browser shapes.
Package ja4 implements TLS client fingerprinting per the JA4 specification (https://github.com/FoxIO-LLC/ja4): Compute turns a TLS ClientHello into a Fingerprint, and the ConnStore carries that spec-compliant value from the handshake to every request on the same connection; ExtractFromState is the best-effort partial fallback when no handshake fingerprint survives, and Fingerprint.IsBrowserLike flags profiles outside typical browser shapes.
Package listeners implements merlon Caddy listener wrappers: the tap rate gate (module id caddy.listeners.merlon, Tap below) and the JA4T SYN capture (caddy.listeners.merlon_ja4t, ja4t.go).
Package listeners implements merlon Caddy listener wrappers: the tap rate gate (module id caddy.listeners.merlon, Tap below) and the JA4T SYN capture (caddy.listeners.merlon_ja4t, ja4t.go).
Package rules implements the merlon rule grammar (spec §5): structured, AND-ed matchers with bypass/deny/redirect/challenge/coraza/route/queue/score actions across four phases (score nudge is phase 4 — an enum label that evaluates *before* phases 1–3, see Rule.Phase).
Package rules implements the merlon rule grammar (spec §5): structured, AND-ed matchers with bypass/deny/redirect/challenge/coraza/route/queue/score actions across four phases (score nudge is phase 4 — an enum label that evaluates *before* phases 1–3, see Rule.Phase).
Package score contains the pure bot-scoring pipeline: scorers emit weighted signals; the engine combines them into a clamped 1–20 score.
Package score contains the pure bot-scoring pipeline: scorers emit weighted signals; the engine combines them into a clamped 1–20 score.
scripts
internal/docstrip command
Command docstrip prints the comment-stripped canonical form of one Go source file to stdout — the per-file primitive behind scripts/comments-only.sh (design 17 R-17-guard).
Command docstrip prints the comment-stripped canonical form of one Go source file to stdout — the per-file primitive behind scripts/comments-only.sh (design 17 R-17-guard).
Package sensor ships the browser attestation script (spec §6 L3/L4) and parses its reports.
Package sensor ships the browser attestation script (spec §6 L3/L4) and parses its reports.
Package store persists Merlon's dynamic state in a single SQLite file via the pure-Go modernc driver (spec §10): rules, the admin audit log, settings rows, the honeypot and fp_block TTL lanes, fp_intel rows, the fp_seen observation table, the trust revocation epoch, custom feed deltas, worker code, schedules and assets, and the per-site site_tokens credential rows (item 15 P1 — hashes only, never secrets, and never in the export bundle), the fleet-scoped net_lists network-list documents (item 32), the bot_rollups hourly traffic counters (item 31 W1), the per-domain vhosts virtual-host documents (item 29 D1), and the zones + zone_records DNS zone plane (item 29 D2).
Package store persists Merlon's dynamic state in a single SQLite file via the pure-Go modernc driver (spec §10): rules, the admin audit log, settings rows, the honeypot and fp_block TTL lanes, fp_intel rows, the fp_seen observation table, the trust revocation epoch, custom feed deltas, worker code, schedules and assets, and the per-site site_tokens credential rows (item 15 P1 — hashes only, never secrets, and never in the export bundle), the fleet-scoped net_lists network-list documents (item 32), the bot_rollups hourly traffic counters (item 31 W1), the per-domain vhosts virtual-host documents (item 29 D1), and the zones + zone_records DNS zone plane (item 29 D2).
Package tlsctx registers a Caddy TLS connection-policy handshake hook (module id tls.context.merlon) that computes and caches the JA4 fingerprint for every ClientHello.
Package tlsctx registers a Caddy TLS connection-policy handshake hook (module id tls.context.merlon) that computes and caches the JA4 fingerprint for every ClientHello.
Package trust implements spec §6: HMAC keyring, attestation and clearance tokens with fingerprint/network binding, the progressive challenge policy, failure escalation, and the proof-of-work engine.
Package trust implements spec §6: HMAC keyring, attestation and clearance tokens with fingerprint/network binding, the progressive challenge policy, failure escalation, and the proof-of-work engine.
Package vhosts renders the merlon vhost store into the running Caddy config — the projection half of item 29 (design rulings R-29-boundary, R-29-roundtrip, BC-4).
Package vhosts renders the merlon vhost store into the running Caddy config — the projection half of item 29 (design rulings R-29-boundary, R-29-roundtrip, BC-4).
Package workers runs user JavaScript in the QuickJS-NG guest under wazero.
Package workers runs user JavaScript in the QuickJS-NG guest under wazero.
wasm
Package wasm ships the prebuilt worker guest artifact: QuickJS-NG plus a small host ABI (workers/guest/main.c) compiled to a wasm32-wasip1 reactor with ZERO imports — no FS, sockets, or clocks cross the guest boundary by construction (deny-all __wrap__ stubs).
Package wasm ships the prebuilt worker guest artifact: QuickJS-NG plus a small host ABI (workers/guest/main.c) compiled to a wasm32-wasip1 reactor with ZERO imports — no FS, sockets, or clocks cross the guest boundary by construction (deny-all __wrap__ stubs).
Package wstee implements the Merlon WebSocket frame-tee transport (B9 track A, spec §1.4): a reverse-proxy transport module whose id is http.reverse_proxy.transport.merlon_ws, wrapping a fully provisioned stock HTTP transport.
Package wstee implements the Merlon WebSocket frame-tee transport (B9 track A, spec §1.4): a reverse-proxy transport module whose id is http.reverse_proxy.transport.merlon_ws, wrapping a fully provisioned stock HTTP transport.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL