Merlon
Merlon is a suite of Caddy modules that silently
scores the bot-likelihood of every incoming request at the edge — without
blocking anything. It combines L1 signals (static network reputation, request
velocity) with L2 signals (TLS/JA4 fingerprint vs. User-Agent consistency)
into a clamped 1–20 score (1 = almost certainly human, 20 = almost
certainly automated), annotating Caddy vars, the request context, and logs so
that later stages — a WAF, challenges, rate-limit actions — can act on it.
Design: docs/superpowers/specs/2026-10-29-merlon-design.md.
Status: M5b shipped (workers, routing, Coraza option & web console), plus the
B1 BGP origin feed; B5 is the documentation milestone in progress. Merlon
scores every request against live IP intelligence — public feeds (RIR
allocations, threat lists, GeoLite2, CAIDA relations, RPKI ROAs, and an
optional full-table BGP RIB feed for observed origin ASNs) ingested into
in-memory LPM tries
that drive the L1 layer — enforces per-site rules through merlon_waf —
bypass / deny / redirect / challenge actions plus an observe mode — runs
the progressive trust ladder (attestation → proof-of-work → interactive
CAPTCHA or polite block) with honeypots, and executes JavaScript workers
in-process: phase-3 route rules dispatch matching requests to a QuickJS-NG
sandbox (merlon_worker_router), deployed, versioned and rolled back live
via the admin API. An optional embedded Coraza OWASP engine runs behind the
coraza rule action, scoring settings hot-edit in place, and a web console
(/id/merlon/console/) manages it all — including live SSE request
inspection — without config reload.
Build
xcaddy builds a Caddy binary with all Merlon modules baked in:
xcaddy build v2.11.7 --with codeberg.org/r3boot/merlon
(From a checkout of this repo, use --with codeberg.org/r3boot/merlon=. to
build against the local source.)
Quickstart
Merlon ships both config formats: Caddy JSON (this quickstart) and the
Caddyfile — grammar, JSON translation and the JSON-only boundary live in
docs/usage/caddyfile.md. The JSON form:
// caddy.json — comments allowed; Caddy strips them
{
"admin": {
// Pin the admin listener explicitly instead of relying on Caddy's
// built-in default (this repo reserves that port for its own tests).
// Every admin-API example below assumes 127.0.0.1:2020.
"listen": "127.0.0.1:2020"
},
"apps": {
"merlon": {},
"http": {
"servers": {
"srv0": {
"listen": [":8080"],
"routes": [
{
"handle": [
{
"handler": "merlon_rating",
"engine": {
"network_ranges": [
{"cidr": "198.51.100.0/24", "reason": "datacenter", "delta": 6}
],
"known_browser_ja4": ["t13d1516h2_8daaf6152771_e5627efa2ab1"],
"velocity_per_sec": 1000,
"velocity_burst": 2000
}
},
{
"handler": "static_response",
"headers": {"X-Bot-Score": ["{http.vars.bot_score}"]},
"body": "score: {http.vars.bot_score}"
}
]
}
]
}
}
}
}
}
./caddy run --config caddy.json
curl -si http://localhost:8080/ | grep -i x-bot-score # → X-Bot-Score: 13
merlon_rating never blocks; it stores the score in {http.vars.bot_score}
and the fired signal names in {http.vars.bot_reasons}, logs both at DEBUG,
and puts the full *score.Result on the request context. The curl above
scores 13: base 1 + no_tls (4) + ua_script (8) — the shipped defaults in
score/fingerprint.go (applyDefaults), combined by score/score.go
(Finalize: clamp(1 + Σ deltas, 1, 20)).
For a production-shaped config — TLS handshake hook, listener wrapper, challenge
and worker stages, a CA/pki setup for lab domains — see
docs/usage/getting-started.md.
Documentation hub
This README is the map; the reference detail lives in docs/usage/. Every page
cites the code it was written against, so a doc that contradicts the tree is a
bug — file it.
| Page |
What it owns |
| getting-started.md |
Build with xcaddy, a realistic standalone JSON config, and the verify-your-install walkthrough |
| configuration.md |
App/module JSON: db_path, trust keyring co-location, feeds snapshot dir and caps, site-key normalization |
| caddyfile.md |
Caddyfile directives — grammar, JSON translation, what stays JSON |
| rules.md |
The rule grammar in full: matcher table, action table, phase pairing, observe mode |
| scoring.md |
Score formula, the complete signal/weight table, EngineConfig keys and SQLite-override precedence |
| ja4plus.md |
The JA4+ lane: JA4H (the b1 variant + deviation table) and JA4T computation, the operator-managed fp_intel table, the four B10 signals and the family rule |
| challenges.md |
The progressive ladder, /__merlon/* endpoints, token binding, honeypots, captcha configuration |
| feeds.md |
The nine adapters, source config fields (name vs adapter), merge/origin priority, refresh scheduling |
| cdn.md |
CDN attribution setup: the five provider formats, per-provider header trust rules, GCP/Akamai caveats, mTLS boundary |
| identity.md |
The peer-gated Tailscale identity lane: Serve/tsnet topologies, allow-lists, sharp edges |
| webbotauth.md |
The Web Bot Auth lane: verifying signed AI-agent requests (draft-ietf-webbotauth-httpsig-protocol-01), the three closed outcomes, directory-fetch posture, and the honest limits before granting a pass |
| waf.md |
Coraza engine v2: rule-set profiles, per-site OWASP wiring, the pinned CRS provisioning recipe and its honest phase-1-only coverage delta |
| websockets.md |
The WS lane end to end: ws_upgrade matching, the merlon_ws frame-tee attach recipe, what the tee sees and refuses to see, kill/bump policy |
| vhosts.md |
The virtual-host store plane: the two flags and what toggling does not do, the escape-hatch truth, the collision law's two answers, the rename cascade, and how the DNS plane joins it |
| dns.md |
The DNS zone plane: the CoreDNS recipe, the three TSIG identities and rotation, slaves and DNSSEC, the reconciler's card vocabulary, the BC-6 preview gate, backups |
| workers.md |
The handle(req) ABI, QuickJS sandbox rules, deploy/rollback workflow |
| admin-api.md |
Route-by-route admin reference with a curl example per endpoint |
| console.md |
Every console view, with its behavior read from the view source — screenshots where the shot pipeline has them (images in docs/images/), prose for the Settings, FP-Lane, Feeds, FP-Intel, Vhosts and Zones views, plus the SSE and ETag behaviors |
| operations.md |
Local CI gates, the e2e suite pointer, perf/PERF.md workflow, backup and revocation notes |
| admin-security.md |
Admin-plane security posture on one page: the port-is-the-credential law, the pinned browser walls and console headers, unix-socket option, state-file modes, audit retention — each fact linked to its test or code pin |
| metrics.md |
The complete merlon_* registry: every family with type/labels/example PromQL, the scrape quickstart, lifetime semantics, absent-series honesty |
Roadmap items that are not shipped — sensor.js
v2 countermeasures (B8), workerd sidecar (B9) — are listed in
Status & limitations and appear nowhere else as
current features.
Status & limitations
Honest current state, verified against this tree:
- Rules and settings are admin-API-managed by design. They are Tier-2
dynamic config (spec §8): rules, scoring overrides, feed sources, honeypot
paths, the challenge policy and the coraza engine live in the SQLite
rules/settings tables and hot-swap without a config reload. That is why
rules never appear in the Caddy JSON — not an omission.
- The admin port is unauthenticated loopback. There are no tokens, no
auth headers and no console login: whatever can reach the admin listener is
an admin (
adminapi/adminapi.go package doc — "reachable only wherever the
admin listener is bound (loopback by default)"). Reach it over SSH/VPN/
Tailscale; never move it to a routable address without real authentication
in front of it.
- The captcha provider has no admin route. See
CAPTCHA (rung 3): the settings row is edited with a
SQLite client. This is a known gap, not an oversight in the docs.
- Coraza runs the request-headers phase only.
merlon_waf feeds the engine
URI, method and headers — never a body — so SecLang rules in coraza phases
2–5 compile but never evaluate, and a matched coraza rule fails open
when the engine is disabled or unset. See
Optional Coraza WAF engine (M5b).
- Observe/enforce counters are ephemeral.
GET /id/merlon/observe/matches
counts are in-memory since process start and reset on restart
(rules/snapshot.go: Counters); they are a rehearsal and drift signal,
not an audit trail — the audit log is the durable record.
- Not yet (roadmap): sensor.js v2
countermeasures (B8), workerd sidecar (B9). Fingerprint-rotation
enforcement (the P-fwd escalation) stays gated on the sensor.js v2
device anchor — B8 ships the observation lane and candidate export
only (ruling R-b8-fp-empty).
Rules & enforcement (M2)
Rules live in SQLite, opened at apps.merlon.db_path (default
<caddy data dir>/merlon.db), and are edited through Merlon's admin API —
never by editing Caddy config. Every mutation recompiles the site's ruleset
and hot-swaps it atomically (each list response carries a monotonically
bumping generation), so rule changes take effect on the next request with
no config reload. merlon_waf is the only Merlon module that blocks
traffic, and it must be placed after merlon_rating in a route's handler
chain — it reads the score rating put on the request context (with no rating
handler before it, bot_score matchers see 0).
Canonical wiring:
{
"apps": {
"merlon": {}, // SQLite store + per-site snapshots + observe counters
"http": {
"servers": {
"srv0": {
"listen": [":8080"],
"routes": [
{
"handle": [
{ "handler": "merlon_rating" },
{ "handler": "merlon_waf", "site": "example.com" },
{ "handler": "static_response", "body": "origin" }
]
}
]
}
}
}
}
}
(site is optional; it defaults to the request Host without port. Both are
case-normalized — site keys are matched lowercase, so config and admin API
always agree.)
Admin API
Served by the admin.api.merlon module on Caddy's admin listener — Caddy's
own admin API, so no extra port and no extra auth surface (see the box below).
Route-by-route reference, with a curl example per endpoint, is
docs/usage/admin-api.md; this is the map, grouped
by surface:
| Surface |
Endpoints under /id/merlon/ |
Deep dive |
| Rules |
GET+POST sites/{site}/rules, PATCH+DELETE sites/{site}/rules/{id}, POST sites/{site}/rules/reorder |
rules.md |
| Observe counters |
GET observe/matches |
rules.md |
| Challenge ladder, honeypots, token revocation |
GET+PUT settings/challenges, GET+PUT settings/honeypots, POST trust/bump_epoch |
challenges.md |
| Feeds |
GET+PUT settings/feeds, POST feeds/refresh, GET feeds/status, GET+PUT feeds/custom |
feeds.md |
| Network intelligence |
GET lookup?ip=…, GET registry/asn/{asn}?limit=… |
feeds.md |
| Scoring override |
GET+PUT+DELETE settings/scoring |
scoring.md |
| Coraza engine |
GET+PUT settings/coraza |
Coraza, waf.md |
| Live inspection |
GET inspect/live (SSE) |
console.md |
| Audit log & config export |
GET audit?limit=N, POST export |
admin-api.md |
| Workers |
GET sites/{site}/workers, POST …/{name}/deploy, POST …/{name}/rollback, GET …/{name}/versions, GET …/{name}/code, GET+PUT+DELETE …/{name}/schedule, POST …/{name}/run |
workers.md |
| Worker assets |
GET …/{name}/assets, PUT+DELETE …/{name}/assets/{path...}, DELETE …/{name}/assets, GET+PUT …/{name}/assets-config |
workers.md |
| Web console |
GET console/{path...} |
console.md |
Every example in this README talks to an admin listener pinned to an explicit
loopback port, so it never collides with Caddy's built-in default (which this
repo's own test suite reserves):
export MADMIN=http://127.0.0.1:2020 # matches the Quickstart's admin.listen
Site keys are normalized (port stripped, IPv6 brackets removed, case-folded):
foo:8080, [foo] and FOO all name the same site. Rule ETags look like
"v3" (the row version) and come back from create and patch responses.
Errors: invalid rule → 400 with the grammar message, unknown id → 404, stale
If-Match or concurrent-write conflict → 412, missing If-Match on PATCH →
428.
Keep the admin API private. It is only as exposed as Caddy's admin
listener itself — loopback (localhost:2019) by default. Keep it that
way; do not move admin.listen to a routable address without putting
real authentication in front of it. There are no tokens and no auth
headers anywhere in this surface: reachability is the authorization
model (spec §9 — zero extra auth surface).
# create a phase-2 deny rule (responds 201 + ETag: "v1" + Location)
curl -si -X POST $MADMIN/id/merlon/sites/example.com/rules \
-d '{"phase":2,"enabled":true,
"matchers":{"bot_score":{"min":15}},
"action":{"type":"deny"}}'
# patch = full replace; If-Match with the current ETag is mandatory
curl -si -X PATCH $MADMIN/id/merlon/sites/example.com/rules/1 \
-H 'If-Match: "v1"' \
-d '{"phase":2,"enabled":true,
"matchers":{"bot_score":{"min":18}},
"action":{"type":"deny","status":429}}'
Scoring hot-edit. PUT settings/scoring takes exactly the JSON that
merlon_rating's engine block takes (see Quickstart) and
takes effect on the next request: the PUT builds the engine before
persisting, so an invalid payload gets 400 and never touches storage or live
state. While the row exists it wins over the module's JSON config; DELETE
drops it and GET answers {"configured":false} until then, and the override
is reloaded at Provision so it survives restarts. One gotcha: a PUT replaces
the live engine, which resets in-flight per-IP velocity counters — repeated
tuning suppresses the velocity signal, so expect a warm-up window after each
save.
Live inspection. GET inspect/live streams every scored request as an SSE
data: frame shaped
{"ts","site","method","path","ip","score","sensor_state","asn","country","signals":[{"layer","name","delta"}]}
(optional fields omitted when empty) with a comment heartbeat every 15 s.
Publishing is gated on subscriber presence — zero overhead while nobody
watches — and a slow subscriber's oldest buffered event is dropped rather than
blocking the data plane.
Audit & export. GET audit?limit=N answers audit entries newest first as
{"entries":[{ts,site,kind,rule_id,before,after}]} (default limit 100,
clamped to 1..1000, non-numeric → 400). POST export answers the whole
configuration bundle (version 1): per-site rules, worker code with current
versions inline, and the settings documents — challenges, honeypot paths,
feeds, captcha, the scoring override (null when no SQLite override exists)
and the coraza config. It carries no secret values: per ruling R7 the
captcha config holds only the secret_env variable name, and feed license
keys appear by name only, so the bundle is safe to ship off-box for backup or
migration. Every successful export writes one settings.export audit entry
with no payload — just the egress trace.
Web console. A browser UI for everything in this section ships inside
the binary — see Web console (M5b).
Rule grammar
Phases. Rules run in three ordered phases: 1 = bypass,
2 = enforce, 3 = route (landed in M5a). merlon_waf walks phases
1→2; phase 3 is evaluated separately by merlon_worker_router, which sees
only route rules (within a phase the first enabled matching rule wins
that phase). A phase-1 bypass match skips all enforcement; an enforce-mode
match acts immediately and stops evaluation. A fourth enum, 4 = score
nudge, is deliberately not an evaluation order: nudge rules run as a
cumulative pre-pass before phases 1–2 — see score below and
rules.md.
Actions. bypass | deny | redirect | challenge | coraza | queue | route | score:
| Action |
Options |
Defaults |
bypass |
— |
stop enforcement, serve the origin |
deny |
status (any 4xx) |
403, body blocked by merlon |
redirect |
url (required), status ∈ 301/302/307/308 |
302 |
challenge |
policy (only "progressive" exists so far) |
progressive ladder — see Trust & challenges |
coraza |
profile (rule-set profile name, "" = default engine) |
run the request through the embedded Coraza engine — phase 2 only; see Coraza & waf.md |
queue |
capacity, max_wait_ms, max_waiters, status, retry_after_s |
20, 5000, 4×capacity, 429, 5 — concurrency permit with bounded waiters, not a rate limiter — phase 2 only; see rules.md |
route |
worker (required, ^[a-z0-9][a-z0-9_-]{0,63}$) |
dispatch to the named worker — phase 3 only; see Workers & routing |
score |
delta (non-zero, abs ≤ 19), reason (≤ 200 B) |
shift the effective bot score fed to enforcement — phase 4 only; unrated requests are never nudged; see rules.md |
Validation enforces the pairing: route requires phase 3, coraza and
queue require phase 2, bypass/deny/redirect/challenge require
phases 1–2, and score pairs exclusively with phase 4.
Matchers. All present conditions must match (AND); an absent condition
passes; at least one condition is required (match-all rules are rejected):
| Matcher |
Matches when |
path: {exact | prefix | regex} |
any variant matches the URL path (OR-ed within the condition) |
methods: ["GET", "POST"] |
request method is in the list (case-insensitive) |
header: {name, regex?} |
header present; empty/absent regex = presence check only |
cookie: {name, regex?} |
same, for cookies |
bot_score: {min} |
rating score ≥ min (0 when no rating handler ran) |
client_cidrs: ["10.0.0.0/8"] |
client IP falls in any CIDR |
client_lists: ["office-nets"] |
client IP falls in any CIDR of the named fleet network lists (merged with client_cidrs; an empty or list-only-empty rule never matches — fail closed); lists live at PUT /id/merlon/lists/networks/{name} — see rules.md |
ja4_prefixes: ["t13d1516h2"] |
JA4 fingerprint starts with any prefix |
asn: [6502] |
feed-resolved origin ASN is in the list (see Feeds) |
country: ["de"] |
feed-resolved ISO 3166-1 alpha-2 country matches (case-insensitive) |
sensor_state: "absent" |
L3 token state: absent | valid | stale | replayed (see below) |
ws_upgrade: true|false |
request is (true) / is not (false) a WebSocket upgrade — Upgrade: websocket header, case-insensitive; negation landed in M5a |
asn and country fail closed: without feeds data resolving network
context for the client, they never match. General rule negation (beyond
ws_upgrade:false) and the query matcher remain deferred.
Matcher-by-matcher semantics, the phase-pairing validation rules and worked
examples: docs/usage/rules.md.
Mode. "mode": "observe" (default "enforce") evaluates the rule but
never acts on the request: every would-have-matched hit is counted, logged,
and exposed by GET /id/merlon/observe/matches as
{"rules":[{"rule_id":1,"observed":42,"actions":0}]} while traffic passes
freely — the safe way to rehearse a rule before switching it to enforce.
Counters are in-memory and reset on restart.
Trust & challenges (M3)
The progressive ladder (spec §6): score below rung1_min passes silently,
then attestation page → client proof-of-work → interactive CAPTCHA or polite
block (the CAPTCHA rung activates with a provider — see
CAPTCHA). Every rung is configurable at runtime; failures
escalate per client and reset on success. Since M20 the HTML legs are
branded formatted documents — themed, localized (Accept-Language, en
fallback), noindex/no-store, CSP-locked per page; machine legs are
byte-identical to M3 (contract:
docs/usage/challenges.md §The interstitial documents).
| rung |
default score band |
browser clients |
JSON/API clients |
| 0 |
< 9 |
pass silently |
pass silently |
| 1 |
9–11 |
inline attestation page (loads sensor.js, auto-reload on success) |
pass through, scored |
| 2 |
12–15 |
PoW puzzle page |
403 + puzzle as JSON |
| 3 |
≥ 16 |
CAPTCHA page when a provider is configured, else polite block |
403 {"error":"captcha_required"} / polite block |
PoW difficulty is clamp(bot_score, 12, 18) leading zero bits of
sha256(challenge + ":" + nonce) — trivial at rung 2's floor, a few seconds
of JS even at the cap. Each failed verify bumps that client one rung higher
(capped at rung 3); a solve resets it. PoW issuance is rate limited to 5/s
per network bucket (burst 10); when exhausted, rung-2 clients get the polite
block and GET /__merlon/challenge/start answers 429. Since S1 that
endpoint also only serves challenged clients: it spends the grant the
WAF's rung-2 mint records alongside its embedded puzzle, so a cold ask
answers 403 — puzzles, and therefore clearances, are never minted for
free. Challenges are
bound to the issuing network bucket (/32 v4, /56 v6): verification from
a different bucket is rejected without consuming the challenge — same
no-consume policy as a wrong nonce (only a correct solve consumes it) — so
a legitimate retry from the issuing network still succeeds.
Chain order matters: the challenge endpoints must run before the WAF.
{"handle": [
{"handler": "merlon_rating"},
{"handler": "merlon_challenge"},
{"handler": "merlon_waf", "site": "example.com"},
{"handler": "reverse_proxy", "upstreams": [{"dial": "localhost:9000"}]}
]}
| cookie |
visibility |
lifetime |
meaning |
merlon_att |
JS-readable |
30 min |
sensor script reported a consistent browser (feeds L3) |
merlon_clr |
HttpOnly |
1 h |
passed a challenge on this site (normalized Host); caps bot_score at 4 while valid |
Tokens are HMAC-signed (keyring: <db dir>/merlon_trust_keys.json; rotation =
append key + restart Caddy or caddy reload --force, since a plain reload of an
unchanged config is a no-op and the key file is not part of it) and bound to
client network bucket (/32, v6 /56), JA4 prefix, UA class — and, since the S1
token v2, to the issuing site (rules.NormalizeSite(Host)): a clearance
earned on one hosted site never validates on another. The UA-class arm is
strict on the presented side — a token minted with a class must present it
back. A forged or
cross-fingerprint token reads as sensor_state: replayed (+10).
Challenge rules use the new action (only progressive policy exists):
curl -X POST --localhost $MADMIN/id/merlon/sites/example.com/rules -d '{
"phase": 2, "name": "challenge suspicious", "enabled": true,
"matchers": {"bot_score": {"min": 10}},
"action": {"type": "challenge", "policy": "progressive"}}'
First-match-wins applies to challenge too: a matched challenge rule whose
score lands below rung1_min passes the request and suppresses later
rules — evaluation does not fall through to a deny further down the list.
Challenge policy & honeypots
curl --localhost $MADMIN/id/merlon/settings/challenges
# {"rung1_min":9,"rung2_min":12,"rung3_min":16}
curl -X PUT --localhost $MADMIN/id/merlon/settings/challenges \
-d '{"rung1_min":8,"rung2_min":12,"rung3_min":17}'
curl -X PUT --localhost $MADMIN/id/merlon/settings/honeypots \
-d '{"paths":["/admin-login","/.env"]}' # exact paths; publish them in robots.txt
curl -X POST --localhost $MADMIN/id/merlon/trust/bump_epoch
A honeypot hit answers 404 (illusion preserved), poisons the client IP for
24 h, and adds honeypot_hit (+12) to its score — fleet-wide on this edge,
not per-site. The paths are never linked from your own pages; publishing
them in robots.txt leaves them for crawlers only.
bump_epoch instantly revokes all outstanding attestation/clearance
tokens (global epoch; per-client epochs are planned).
Endpoints served by merlon_challenge: /__merlon/sensor.js,
POST /__merlon/attest (accepted only for a client the sensor was served
to, rate-limited per bucket), GET /__merlon/challenge/start (grant-gated
— challenged clients only),
POST /__merlon/challenge/verify, POST /__merlon/captcha/verify — note the
two underscores in /__merlon/, a common typo when wiring these paths into
other rules. Non-HTML clients receive the PoW puzzle as JSON (spec §6).
Full endpoint semantics, token-claim layout and failure-escalation rules:
docs/usage/challenges.md.
Feeds (M4)
Feeds ingest public IP-intelligence sources into per-source in-memory
LPM tries that supply L1 scoring with per-prefix reputation, origin ASN,
country, CAIDA tier and RPKI validity — replacing hand-maintained CIDR
tables as the primary network data source. Everything is stdlib-only; no
module dependencies were added for M4.
Feeds are off by default: the settings document starts as
{"sources": []} and operators opt in per source. PUT settings/feeds
validates and persists — it never fetches (that is what
POST feeds/refresh is for). Sources live under the
admin API key feeds — never in Caddy config:
The one-click route (item 33). The console's Feeds view carries a
Recommended feeds card that applies a curated, versioned,
licence-checked set — five RIR delegated-stats feeds, the published
Cloudflare/AWS/Fastly lists, Akamai with an honest partial-coverage
label, and the compiled-in GCP ranges. Posture, in one sentence:
nothing phones home before your click, and the click itself fetches
nothing — it stores a config document through the same PUT below; the
~30 MB/day fetch weight is disclosed before you click, and rows whose
licence could not be verified at the primary source render as
unverified, never as green. A fresh install still scores nothing until
the operator clicks (or hand-writes the PUT) — that is the design, not
a gap. Keyed sources (GeoLite2, HoneyLabs watchlist, ThreatFox) ride a
masked write-only credentials lane in the same view. Full terms table,
cadence floors, and the operator-supplied automation recipes:
docs/usage/feeds.md §Recommended feeds.
curl --localhost $MADMIN/id/merlon/settings/feeds
curl -X PUT --localhost $MADMIN/id/merlon/settings/feeds -d '{
"sources": [
{"name": "ripencc", "adapter": "rir", "enabled": true,
"url": "https://ftp.ripe.net/pub/stats/ripencc/delegated-ripencc-latest"},
{"name": "feodo", "adapter": "threat", "enabled": true, "delta": 8,
"url": "https://feodotracker.abuse.ch/downloads/ipblocklist.txt"},
{"name": "geolite2", "adapter": "geo", "enabled": true,
"unpack": "zip", "basic_auth": "user:pass",
"license_key_env": "GEOLITE2_LICENSE_KEY",
"files": [
{"role": "country", "url": "https://download.maxmind.com/geoip/databases/GeoLite2-Country-CSV/download?suffix=zip"},
{"role": "asn", "url": "https://download.maxmind.com/geoip/databases/GeoLite2-ASN-CSV/download?suffix=zip"}
]}
]}'
Adapters
| Adapter |
format (default) |
Input |
Contributes |
Default Δ |
rir |
— |
RIR statistics exchange format v2 (`registry|cc|type|start|value|date|status[ |
extensions…], one allocation per line — the real delegated-*-latestlayout).startis a CIDR, alo - hirange (both expanded to the minimal CIDR cover), or a bare first address whosevaluegives the extent: IPv4 address count (non-CIDR counts included) or/nn` prefix length; IPv6 values are prefix lengths. PI/PC types set the provider-independent flag |
country, PI flag |
threat |
iplist · threatfox-csv · honeylabs-csv |
plain IP/CIDR lines (# comments, Feodo-style trailing fields tolerated) · ThreatFox CSV export (ioc_type=ip rows only) · HoneyLabs exploiters CSV (ip,first_seen,last_seen,…; tor_exit rows skipped; the vendor's one-IP-per-line .txt variant parses as iplist) |
reputation delta |
+8 (per-source delta) |
datacenter |
cidr · ipinfo-csv |
CIDR lines · CSV with network/country/category columns (rows kept when category contains hosting or cloud; all rows kept if no category column) |
datacenter flag + delta |
+10 (per-source delta) |
geo |
— |
GeoLite2 Country + ASN CSV editions, roles country and asn, each a zip download (unpack: "zip", item 70) fetched with HTTP Basic (basic_auth: "user:pass", credential stored as account_id:license_key); every *Blocks*.csv member is parsed |
origin ASN, org, country |
0 |
caida |
— |
as-organizations TSV (role orgs) + as-relations pipe format (role rels, required) — ASN-keyed, no prefix rows |
tier (1 transit … 4 customer-only) and org name via the resolved origin ASN |
0 |
roa |
— |
JSON {"roas":[{"prefix": "…", "asn": N}]} |
RPKI validity, computed at lookup time |
+6 fixed when invalid |
bgp |
— |
One MRT TABLE_DUMP_V2 RIB snapshot (RIPE RIS latest-bview.gz or any equivalent; the adapter sniffs the gzip magic itself since RIS serves bare gzip) — first path wins per prefix |
observed origin ASN (the strongest origin signal, see below) |
0 |
cdn |
cloudflare-ips · fastly · akamai-acl · cloudfront-ips · gcp-static |
The five providers' published prefix lists (JSON/text; gcp-static compiles the GCP ranges in and needs no input at all) |
via_cdn provider overlay — labels the socket peer's CDN, never delta or origin |
0 always |
intel |
honeylabs-fingerprints-csv |
HoneyLabs fingerprints CSV — fingerprint lane: kind=ja4 rows only (akin and q… QUIC-derived values skipped and counted), upsert-then-sweep into fp_intel per exact source tag, no trie |
neutral fp_intel rows — display-only, zero score movement (feeds.md §HoneyLabs sources, ja4plus.md §HoneyLabs import lane) |
0 always |
Source names match ^[a-z0-9][a-z0-9_-]{0,63}$, must be unique, and each
source needs at least one of url (http(s)), file (local path — offline
imports and tests; the raw archive is skipped for it), or a files list of
{"role","url"} entries; files takes precedence over url/file (the
one exempt shape: cdn + gcp-static, which carries its input compiled
in). Per-source delta is bounded to ±25. Note that name and adapter are
different fields: registry lookups key on name (it is what
lookup.matches[].source reports), so canonical operator config pairs them —
{"name":"bgp","adapter":"bgp"}. The full source-field table is in
docs/usage/feeds.md.
GeoLite2 credentials (item 70). GeoLite2 downloads are HTTP Basic
against …/geoip/databases/{edition}/download?suffix=zip: username = your
MaxMind account id, password = the licence key. The credentials lane keeps
that as one value, account_id:license_key (same in the
GEOLITE2_LICENSE_KEY env var when it is set, where it wins); the manager
splits it at use, so no part of it ever rides a URL, a status row, a log
line or an audit row — and a stored value with no : (a bare licence key)
is refused with an error that says what form was expected instead of
sending it. Other keyed sources still use the classic lane: URLs carrying
{license_key} are substituted from the env var named by
license_key_env. A missing or empty env var fails that source's fetch
only: GET /id/merlon/feeds/status shows the error in last_error, the
last snapshot (if any) keeps serving, and every other source refreshes
normally. The expanded key never appears in status output or logs.
Merge semantics
Resolution happens per lookup. Within one source the most-specific prefix
wins; across sources the deltas sum. For origin attributes the
observed BGP origin wins (a bgp source), then geo, then rir
(ASN, country) — spec §16 superseded the earlier geo>rir ruling; CAIDA tier
and org resolve through the resolved origin ASN; the datacenter flag is set by
any hit. RPKI validity is evaluated against the feed-derived origin ASN: a
covering ROA that does not list it yields invalid and a fixed +6. Custom
operator entries are just another source (custom) — fleet-wide, merged like
any feed. The static network_ranges engine config still works (most-specific
wins) and adds its signal on top of the feed signals.
Each matching source with a non-zero delta contributes an L1 signal
named feed:<source> with that source's delta; the geo, rir and caida
feeds contribute origin attributes (ASN, country, tier), not score. An
RPKI-invalid prefix fires rpki_invalid (+6). So a curl from a
datacenter IP listed by two threat feeds scores:
base 1 + no_tls 4 + feed:datacenter 10 + feed:feodo 8 + ….
Scheduling & refresh
A source is due when interval_hours (default 168 = weekly) plus a
deterministic jitter derived from the source name (< 24 h, same name →
same offset every boot) has elapsed since its last fetch. A 15-minute
ticker wakes the scheduler; nothing is ever fetched at Provision —
startup only loads snapshots, so restarts are instant and offline-safe.
The first fetch happens on POST /id/merlon/feeds/refresh or the first
scheduler tick that finds the source due:
curl -X POST --localhost $MADMIN/id/merlon/feeds/refresh # all enabled
curl -X POST --localhost $MADMIN/id/merlon/feeds/refresh \
-d '{"source":"feodo"}' # one source
# [{"source":"feodo","ok":true,"entries":12345}]
curl --localhost $MADMIN/id/merlon/feeds/status
# {"sources":[{"source":"feodo","adapter":"threat","enabled":true,
# "fetched_at":"2026-11-01T04:11:38Z","entries":12345}]}
# (a source never successfully fetched yet shows "fetched_at": null)
Refreshes are synchronous (admin API is loopback-only) and all of them —
manual POSTs and scheduler ticks alike — serialize on a single lock
(ruling R10). The two callers handle contention differently: a scheduler tick
queues behind the lock, while POST /id/merlon/feeds/refresh never queues —
it answers 409 refresh in progress instead (in bulk mode, sources
already imported by that call come back as audited partial progress). A hung
upstream therefore stalls scheduled refreshes for up to the fetch timeout
(5 min per file, 30 min per refresh) but never turns the admin API into a
waiting room. The data plane never waits either: imports build a new
Registry and swap it atomically, so requests keep scoring against the last
snapshot throughout.
Stale ≠ absent. A failed fetch, parse or snapshot write keeps the
previous generation serving; the reason surfaces in last_error. Fetched
bytes are archived under <dir(db_path)>/feeds/raw/, newest 7 per source
kept; compiled snapshots (gzip JSON) live in <dir(db_path)>/feeds/ and
are reloaded at startup.
Custom operator entries
Your own reputation list, merged like any feed and stored in SQLite
(feed_custom table). PUT is a full replacement (honeypot-paths
precedent): max 10 000 entries per write, delta bounded ±25, prefixes
canonicalized (127.0.0.1/8 → 127.0.0.0/8), duplicates rejected:
curl -X PUT --localhost $MADMIN/id/merlon/feeds/custom -d '{
"entries": [
{"prefix": "192.0.2.0/24", "delta": -3, "reason": "office-vpn"},
{"prefix": "203.0.113.66/32", "delta": 4, "reason": "repeat-offender"}
]}'
The lookup probe
GET /id/merlon/lookup?ip=… runs the full merge against the live registry
and always answers 200 for a valid IP (400 otherwise) — empty matches
and "rpki":"unknown" when no ROA covers the address or its origin ASN
could not be resolved:
{"ip": "203.0.113.66", "origin_asn": 6450, "country": "de", "tier": 2,
"datacenter": true, "rpki": "invalid", "delta": 28,
"matches": [
{"source": "custom", "prefix": "203.0.113.66/32", "delta": 4},
{"source": "datacenter", "prefix": "203.0.113.0/24", "delta": 10},
{"source": "feodo", "prefix": "203.0.113.66/32", "delta": 8},
{"source": "rpki", "prefix": "203.0.113.0/24", "delta": 6}]}
The asn and country rule matchers consume exactly this context — and
fail closed: with no feeds (or no covering entry), they never match.
GET /id/merlon/registry/asn/{asn}?limit=… is the reverse drill (B1): which
prefixes does origin AS asn announce in the live bgp feed? It answers
{"asn":…,"total":N,"prefixes":[…]} — limit defaults to 200 and is capped
at 5000 (non-numeric or ≤ 0 → 400, as is asn 0), and without a bgp source
total is simply 0.
ROA exports
The roa adapter wants the tiny JSON shape {"roas":[{"prefix","asn"}]};
a one-liner converts an NTT-style registry export (array of objects each
carrying a prefixes list):
curl -sf <your-peering-export.json> \
| jq '{roas:[.[] | .asn as $a | .prefixes[]? | {prefix:(.prefix // .), asn:(.asn // $a)}]}' \
> /var/lib/merlon/roas.json
…then point a roa source at the file ("file": "/var/lib/merlon/roas.json").
BGP: RIB snapshots in, live monitoring out. M4 explicitly dropped BGP;
B1 shipped the ingestion (spec §16) — an MRT TABLE_DUMP_V2 reader and
the bgp adapter, so one RIPE RIS latest-bview.gz (or any equivalent
dump) gives Merlon the full prefix→observed-origin table, which is the
strongest origin-ASN signal and the key to registry forensics. Still out of
scope: live BGP (no BMP/update feeds — snapshots only) and
announcement-vs-allocation mismatch detection; RPKI validity is still
evaluated against the operator-supplied ROA export above, and with no bgp
source the origin ASN falls back to GeoLite2-ASN. None of it added a
dependency: the MRT parser is in-repo Go.
Workers & routing (M5a)
Phase 3 opened: a rule can now route matching requests to a JavaScript
worker running inside the edge process — no origin round-trip, no restart.
Code is stored versioned in SQLite and dispatched by
merlon_worker_router, placed after merlon_waf. Full chain order:
merlon_rating -> merlon_challenge -> merlon_waf -> merlon_worker_router -> origin
{"handle": [
{"handler": "merlon_rating"},
{"handler": "merlon_challenge"},
{"handler": "merlon_waf", "site": "example.com"},
{"handler": "merlon_worker_router"},
{"handler": "reverse_proxy", "upstreams": [{"dial": "localhost:9000"}]}
]}
The router reads the same per-site snapshot as the WAF but evaluates phase 3
only (ruling R4): a phase-1/2 bypass skips enforcement yet still flows
through the router — routing is dispatch, not blocking. A site with no rules
passes through allocation-light; a request whose route rule matches gets its
worker's response and never reaches the origin. If the named worker was
never deployed (or its invoke fails), the router answers 502
(worker unavailable / worker error) rather than falling through behind a
matched rule. Observe mode works for route rules too: counted by
GET /id/merlon/observe/matches, never dispatched. Note that a route rule's
actions counter ticks on successful dispatches only — both 502 paths
return before it, so neither counter moves and a throwing worker looks like a
non-matching rule in GET /id/merlon/observe/matches (unlike WAF rules, whose
actions count every match, before enforcement).
Route rules
route is a phase-3-only action naming a per-site worker
(^[a-z0-9][a-z0-9_-]{0,63}$, same grammar as feed source names):
curl -X POST --localhost $MADMIN/id/merlon/sites/example.com/rules -d '{
"phase": 3, "name": "demo route", "enabled": true,
"matchers": {"path": {"prefix": "/worker-demo"}},
"action": {"type": "route", "worker": "echo"}}'
The handle(req) contract
A worker is one JS source file that defines function handle(req) — called
once per request in a fresh sandbox; return a response object (or a bare
string, taken as the body with status 200):
function handle(req) {
return {
status: 200, // out-of-range/omitted => 200
headers: {"content-type": "text/plain"}, // Content-Length/-Encoding stay host-owned
body: "hello from " + req.path + " (score " + req.bot_score + ")"
};
}
req fields: method, path, query, headers (object of arrays),
body (string, truncated at 64 KiB), site, client_ip, bot_score.
The response body is capped at 1 MiB. A handle that throws yields a 502;
the guest message (with stack) is logged, capped at 512 B. The key
__error in a returned object is reserved for failure signalling —
workers cannot use it as data. Math.random() throws unconditionally
(deterministic sandbox): QuickJS seeds its PRNG from an epoch-0 clock, so
sequences are identical across instances — never usable for secrets.
Field-by-field ABI reference (workers/request.go WorkerRequest):
docs/usage/workers.md.
Sandbox & limits (rulings R1/R2/R3)
Guest code runs in QuickJS-NG under wazero, one fresh module instance
per invocation — never pooled, no state survives a request. Limits:
| Limit |
Value |
On breach |
| Wall-clock deadline per invoke |
5 s (1 s for the deploy trial run) |
execution killed — 502 on a request; 400 at deploy for the trial run |
| Guest linear memory |
64 MiB |
allocation fails / invoke error |
| Worker source size |
1..512 KiB |
400 at deploy |
| Request body into guest |
64 KiB (truncated) |
debug log only |
| Response body |
1 MiB |
invoke error |
The guest artifact has zero WASM imports — no filesystem, sockets or
clocks reach the host (deny-all even inside QuickJS), and there is
deliberately no fetch/KV API in M5a: workers are pure request→
response transforms. Network egress and key-value storage from workers are
deferred (ruling R3, backlog).
Guest artifact & provenance
The sandbox embeds one prebuilt WASM module
(workers/wasm/quickjs-merlon.wasm): QuickJS-NG v0.11.0 (MIT) compiled
with wasi-sdk 25 (-nostartfiles, WASI syscall stubs deny-all). The MIT
license text ships with the module source as workers/wasm/LICENSE.quickjs
(not embedded in the Go binary — only the wasm artifact itself is);
PROVENANCE records versions, exact compile/link lines and sha256 (mirrored
in PROVENANCE.sha). Rebuild reproducibly with
workers/guest/build.sh (fetches pinned toolchain + sources; the Go build
never invokes it — go:embed ships the committed artifact).
Deploy & rollback
Deploy trial-runs the code first (must load and define handle within
1 s) and only persists on success — a broken worker cannot be deployed
(400 with the guest message, capped at 512 B). Versions are immutable;
deploy appends, rollback flips the current pointer:
# deploy => {"name":"echo","version":1} (201)
curl -si -X POST --localhost \
$MADMIN/id/merlon/sites/example.com/workers/echo/deploy \
-d '{"code":"function handle(req){return {body:\"v1: \"+req.path}}"}'
# rollback => current pointer back to version 1 (unknown worker/version: 404)
curl -s -X POST --localhost \
$MADMIN/id/merlon/sites/example.com/workers/echo/rollback \
-d '{"version":1}'
Deploying again appends version 2 — versions are immutable, so every past one
stays fetchable through …/versions (history, newest first, no code bodies)
and …/code?version=N (raw source; …/code alone is the current one).
Deploys and rollbacks are audited (workers.deploy / workers.rollback
kinds with before/after versions). A deploy takes effect on the next
request: worker code is cached in-process for the routing hot path, and
every deploy or rollback refreshes that cache — only a cold miss reads
SQLite.
CAPTCHA (rung 3)
Rung 3 upgrades from polite block to an interactive Friendly Captcha v1
widget when a provider is configured — no SDK, plain net/http against a
self-hosted solve server (any Friendly-Captcha-compatible deployment
serving the /fc.min.js widget script and the siteverify endpoint). The
settings document lives under the SQLite settings key captcha:
{"provider": "friendlycaptcha", "enabled": true,
"solve_url": "https://solve.example.net/siteverify",
"site_key": "0x00000000000000000000000",
"secret_env": "MERLON_CAPTCHA_SECRET"}
The verification secret is never stored — secret_env names the
environment variable that holds it, re-read on every verify and never
logged (only the variable name appears in logs). The client IP is
deliberately not forwarded to the solve server (privacy). Verification is
fail-closed: unset secret, transport error, timeout (5 s cap), non-200 or
malformed response all mean "not solved"; redirects are never followed (the
secret is never re-sent). Siteverify speaks Friendly Captcha v1: form fields
secret, solution, sitekey. There is no admin endpoint for this
setting in M5a: seed/edit the captcha row of the settings table with
any SQLite client and restart (or reload) Caddy — an enabled-but-invalid
document fails Provision, a disabled or absent one is inert.
Rung-3 behavior:
| provider state |
HTML GET client |
JSON/API client |
| configured & enabled |
CAPTCHA page (widget from solve_url origin, POSTs the token to POST /__merlon/captcha/verify) |
403 {"error":"captcha_required"} |
| unset or disabled |
polite block page (formatted document since M20) |
403 polite block (text/plain, unchanged from M3, byte-for-byte) |
A correct solve answers {"ok":true,"redirect":"/path"} and sets the same
HttpOnly clearance cookie a PoW solve grants (merlon_clr, 1 h, caps
bot_score at 4); a failed verify escalates the client one rung higher like
every other challenge failure. An empty token is rejected 400 without
escalation; with no provider configured the endpoint answers 404.
Optional Coraza WAF engine (M5b)
merlon_waf can hand matched requests to an embedded
Coraza (ModSecurity-compatible) rule engine. It is
off by default and costs nothing until enabled; coraza is the only
third-party engine dependency in the data plane, confined to app/. The
engine config — an enabled flag plus raw SecLang directives — lives
in a SQLite settings row and hot-swaps atomically through the admin API,
no reload:
# enable with two inline phase-1 deny rules
curl -X PUT --localhost $MADMIN/id/merlon/settings/coraza -d '{
"enabled": true,
"directives": "SecRuleEngine On\nSecRule REQUEST_URI \"@streq /wp-login.php\" \"id:1,phase:1,deny,status:403\"\nSecRule REQUEST_HEADERS:User-Agent \"@rx ^sqlmap\" \"id:2,phase:1,t:lowercase,deny\""}'
# read back — {"enabled":false} while unset
curl --localhost $MADMIN/id/merlon/settings/coraza
# disable (full-document replace: engine dropped, directives cleared)
curl -X PUT --localhost $MADMIN/id/merlon/settings/coraza \
-d '{"enabled":false}'
The PUT builds the engine before persisting: uncompilable SecLang gets
a 400 and leaves both storage and the live engine untouched (store
failures → 500); successful writes are audited as kind settings.coraza.
An enabled-but-empty directive set arms a valid, inert engine — enabling
before writing rules blocks nothing instead of failing to boot.
On the data plane, a phase-2 coraza rule in your normal ruleset gates
which traffic enters the engine; the SecLang decides what is blocked:
curl -X POST --localhost $MADMIN/id/merlon/sites/example.com/rules -d '{
"phase": 2, "name": "coraza gate", "enabled": true,
"matchers": {"path": {"prefix": "/"}},
"action": {"type": "coraza"}}'
Since B9-T6 the settings document also carries an optional rule-sets
pool — named engines ({"rule_sets":[{"name":"owasp","directives": "Include /etc/merlon/crs/rules/*.conf"}]}, names [a-z0-9-]{1,32}) that
a rule selects with "action": {"type": "coraza", "profile": "owasp"} —
so one site can run OWASP while another runs your own rules. Unknown or
removed profiles fail open like a disabled engine; a path-backed profile
that fails to compile at boot is skipped with an ERROR naming it while the
rest of the edge serves. Pool semantics, the per-site recipe and an OWASP
CRS provisioning guide (with its honest coverage delta):
docs/usage/waf.md.
A matched coraza rule consumes its phase-2 slot even when the engine
passes the request — like every enforce action it is first-enforce-match-
wins, so later deny/challenge rules in phase 2 never evaluate for that
request (put stricter fallbacks in a different phase if you need them).
What the engine sees (v1 limitation). Each matched request opens a
Coraza transaction fed URI, method and headers only — request bodies are
never read (buffering them would break streaming and WebSocket upgrades on
Caddy's hot path). Phase-1 rules on REQUEST_URI, query args and headers
work; body-dependent rules (ARGS_POST, REQUEST_BODY) can never fire, and
SecLang rules in coraza phases 2–5 compile but never evaluate — only the
request-headers phase runs (no body phases, no logging phase).
Go promotes Host out of the header map and folds Transfer-Encoding into
its own field — both are re-fed to the transaction, so header rules see the
full wire set (only syntactically valid Transfer-Encoding tokens get this
far at all — Go's net/http rejects malformed ones before the handler runs).
REMOTE_ADDR is populated (IPv6 brackets stripped); SERVER_NAME is
unset in v1 — don't match on it.
Interruptions become responses. A coraza deny/drop answers locally
with the engine's status (default 403) and body blocked — note this
differs from plain merlon deny's blocked by merlon. A coraza redirect
sets Location and uses 301/302/303/307; anything unexpected degrades to
302.
Fail-open on the data plane, fail-closed at boot. With the engine
disabled or unset, a matched coraza rule simply passes the request
through — a config gap must never black-hole a site. Conversely, a corrupt
settings row or an uncompilable enabled config fails Provision (Caddy
refuses to boot), the same discipline as the captcha and scoring rows.
No CRS bundled. Merlon ships without the OWASP Core Rule Set — rules
come from your directives and, since B9-T6, from your own on-disk rule
trees via profile Includes. For a pinned CRS v4 provisioning recipe —
and the plain statement of which CRS families actually fire under a
phase-1-only data plane — see docs/usage/waf.md;
upstream engine details live in the
coraza documentation.
The console's Coraza view mirrors exactly these three shipped limitations
(phase-1 data plane, fail-open with no engine, full-document-replace on
{"enabled":false}) — see docs/usage/console.md.
Web console (M5b)
GET http://<admin-addr>/id/merlon/console/ serves the embedded Svelte 5
console — thirteen hash-routed views over the admin API: Overview
(feed health, observe counters), Rules (CRUD, mode toggle, reorder),
Workers (CodeMirror editor, deploy/rollback/version history,
per-worker static asset bundles — manifest, upload, delete, wipe,
SPA-fallback flag), Scoring, Challenges, Coraza, Settings
(client-IP trust, identity lane, WS frame-tee policy, JA4+ feature
gates, honeypot trap paths, bump-trust-epoch danger card with a
typed confirmation, export-bundle download card), Feeds (source
forms, bulk refresh with per-source results, custom-prefix table),
FP Lane (fp_block table, fired candidates, rotation-lane editor),
FP Intel (fingerprint-intel registry: list + filters, CRUD, tolerant
query, bulk import with a per-row errors table), IP Lookup, Audit
log (client-side kind/site filters), and the Live Inspector — a
capped, auto-scrolling table fed by the /inspect/live SSE stream
(opening the tab is what enables publishing; the rating handler only
emits while a subscriber is attached). The bolded panels are the M23
(wave-1) coverage fold; per-view behavior, every claim cited by symbol,
is docs/usage/console.md.
Coverage (pre-positioning honesty, design 20 §3). The console covers
every admin lane except:
- the Web Bot Auth lane —
GET/PUT settings/webbotauth and POST webbotauth/probe
- the Pages lane —
GET/PUT settings/pages
- the network-violation-history surface —
GET/DELETE net/rep
Those seven routes stay admin-API-only until their panels ship — the
curl reference is admin-api.md (the per-net
table + badges already carry a console hand-off spec). This list, kept
identical in console.md, is what stands between
today's honest "covers every lane except…" posture and any stronger one;
docs-and-gate discipline, not vibes (item 20 R-20-precondition).
The console has no auth of its own: it is reachable exactly where the
admin API is and rides that listener's network protection — loopback by
default, so reach it over SSH/VPN/Tailscale (spec §9: zero extra auth
surface). Whatever can reach the admin port is an admin.
Rules-view selector to know: the site field is free text persisted to
localStorage.site; with nothing stored yet it defaults to the first site
of a POST /export bundle when that call succeeds, otherwise type the site
key manually (the server normalizes it anyway).
Screenshots & per-view reference. Every view — plus the attestation,
proof-of-work and polite-block pages — is screenshotted and annotated one
section per view in docs/usage/console.md, which also
documents the deterministic regeneration command. The committed PNGs live in
docs/images/ and regenerate byte-identically (verified across separate
go run invocations).
Build & ship. console/build.sh (maintainer-only, node ≥ 22) runs
npm ci && npm run build in console/ui/ and syncs the output to
console/dist/ — the committed build artifact go:embed bakes into the
binary (the Go build never invokes npm, same rule as workers/guest/).
Flow: edit console/ui/, run console/build.sh, commit console/ui and
console/dist in the same commit.
Dev flow. npm run dev in console/ui/ serves Vite with HMR. Point it
at the admin API on the same origin, or add a local proxy for
/id/merlon/* → http://127.0.0.1:2020 (your admin.listen) to your own
vite.config.js — see scope ruling (d).
v1 scope rulings — the console is functional-admin, not the spec's
pixel design; the §9 ideal remains valid direction for a later polish
milestone:
- (a) rules edit in a validated JSON editor holding the full wire
shape — not a 1:1 matcher form;
- (b) reorder is up/down buttons (POST reorder with the full
ordered id list) — not drag-and-drop;
- (c) settings are structured forms where fields are few (challenge
rung thresholds, coraza enable + SecLang box) and JSON editors elsewhere
(the scoring
EngineConfig document) — no slider dashboards;
- (d) no committed vite dev-proxy config — this README pointer is
all there is;
- the live inspector streams scored requests only (rating stage:
score + per-signal breakdown) — enforcement actions and worker responses
are not part of the SSE event shape yet.
Modules
| Module ID |
Namespace |
Purpose |
merlon |
core app (apps.merlon) |
Shared lifecycle/state for Merlon modules; owns the feed manager, snapshots and refresh scheduler |
tls.context.merlon |
tls.context |
Captures the JA4 fingerprint from every TLS ClientHello |
http.handlers.merlon_rating |
HTTP handler |
Scores requests (L1+L2+L3) and annotates vars/context/logs |
http.handlers.merlon_challenge |
HTTP handler |
Serves the /__merlon/* trust endpoints: sensor script, attestation beacon, PoW start/verify, captcha verify (POST /__merlon/captcha/verify) |
http.handlers.merlon_waf |
HTTP handler |
Enforces per-site SQLite rulesets: bypass/deny/redirect/challenge/coraza + observe mode + honeypot trap — the only blocking module |
http.handlers.merlon_worker_router |
HTTP handler |
Dispatches phase-3 route rules to workers in the QuickJS-NG sandbox; 502 when the named worker cannot serve |
admin.api.merlon |
admin router |
Rule CRUD with ETags, reorder, observe counters, challenge policy, honeypots, epoch revocation, feeds settings/refresh/status/custom, the IP lookup probe and the ASN→prefixes registry drill, worker deploy/rollback/list/versions/code, scoring hot-edit, coraza engine settings, live-inspect SSE, audit listing and config export — over Caddy's admin API |
caddy.listeners.merlon |
listener wrapper |
Per-source-IP handshake rate limiting (kills connection storms) |
Every module's config fields, defaults and provisioning behavior (including
where the SQLite file, the trust keyring and the feed snapshots land):
docs/usage/configuration.md.
TLS fingerprint capture
Attach the handshake hook to a server's TLS connection policies so JA4
fingerprints are computed at handshake time (Caddy's handshake_context
hook is experimental and may change upstream):
"tls_connection_policies": [
{ "handshake_context": {"module": "merlon"} }
]
With this enabled, a browser-UA client whose JA4 is not in
known_browser_ja4 fires ua_browser_ja4_unknown (+6); a known-browser
JA4 with a browser UA stays at score 1.
Handshake rate limiting
Wrap a server's listener to cap new connections per source IP — over-budget
connections are closed at accept time (config fields sit alongside
"wrapper"):
"servers": {
"srv0": {
"listen": [":8080"],
"listener_wrappers": [
{ "wrapper": "merlon", "per_ip_per_sec": 50, "burst": 100 }
],
"routes": [ ... ]
}
}
Scoring model
score = clamp(1 + Σ signal deltas, 1, 20), base 1. Key signals:
| Signal |
Layer |
Default Δ |
Fires when |
no_tls |
L2 |
+4 |
Plaintext request with no captured fingerprint |
ua_script |
L2 |
+8 |
UA matches known HTTP libraries (curl, python-requests, …) |
ua_browser_ja4_unknown |
L2 |
+6 |
Browser UA but JA4 not in known_browser_ja4 |
ja4_partial / tls_abnormal:* |
L2 |
+1 / +1.5 |
Degenerate or abnormal ClientHello |
velocity_high |
L1 |
+4 |
Per-IP request rate over configured budget |
sensor_valid / sensor_stale / sensor_replayed |
L3 |
−6 / +8 / +10 |
Attestation token verified / expired-or-revoked / forged-or-cross-fingerprint |
honeypot_hit |
L3 |
+12 |
Client IP poisoned by a honeypot visit (24 h) |
feed:<source> |
L1 |
per-source Δ |
Client IP matches feed source <source> (most-specific prefix; deltas sum across sources — see Feeds) |
rpki_invalid |
L1 |
+6 |
A feed ROA covers the client prefix but not its resolved origin ASN |
| (configured ranges) |
L1 |
your Δ |
Client IP in a network_ranges CIDR (most specific wins) |
A valid clearance cookie caps the finalized score at 4 (never below 1).
The complete weight tables (fingerprint / trust / velocity defaults, each
applyDefaults), EngineConfig keys and the JSON-vs-SQLite precedence rules:
docs/usage/scoring.md.
Metrics
Merlon is a Prometheus exporter: every collector rides the admin
listener's /metrics endpoint (pinned 127.0.0.1:2020). The full
registry — every merlon_* family with type, labels, meaning and example
PromQL, the scrape posture (on-host static_configs; remote only via
tailnet/tunnel, since the admin plane carries no auth and its reach is
the access control), the documented-absent merlon_build_info, and what
is deliberately not measured here — lives in
docs/usage/metrics.md.
The scoring path is built for the hot loop: regexes are compiled once, maps
are bounded (velocity buckets cap tracked IPs and evict LRU), and each score
costs ~5 small allocations.
go test ./score/ -bench=EngineScoreSilentPath -benchtime=2s -count=3
Measured: 6,976 ns/op (≈7 µs; 287 B/op, 5 allocs/op; median of three
-count=3 runs) on a 13th Gen Intel Core i9-13900K, linux/amd64, Go 1.26 —
comfortably ≪ the <1 ms p99 end-to-end budget.
The M2 milestone gate re-checked this baseline (rules store, WAF handler,
limiterx unification, fingerprint attribution) by A/B benchmarking M1 commit
d82b02c against M2 HEAD interleaved on the same machine: 11,138 vs 11,006
ns/op medians — parity (≈−1%), allocations unchanged at 5/op. Absolute
numbers scale with host load; run the gate quiet for comparable absolutes.
M3 re-ran the same gate on this host: 11,023 / 10,875 / 10,857 ns/op
(-benchtime=2s -count=3), median 10,875 — parity with the 11,006 M2
baseline (−1.2%), allocations unchanged at 5/op. The trust layer is wired
per-request behind optional deps; the silent-path engine pays nothing for it.
M4 re-ran it again: -benchtime=2s -count=5, median 11,791 ns/op
(allocations unchanged at 5/op). Absolute numbers drifted with host load
that day — an interleaved A/B against the M3 base commit on the same
machine in a quiet window measured 10,916 (M4) vs 10,943 ns/op (M3 base):
parity (−0.2%). The feed hook sits behind a nil-checked optional dep; with
feeds loaded, resolution is a binary search per source (~µs) and not gated.
M5a A/B-interleaved against the M4 tip (d427dab) in parallel worktrees,
3 rounds of -benchtime=2s -count=5 per side (15 samples each): medians
11,385 ns/op (M5a) vs 11,335 ns/op (M4 base) — parity (+0.4%),
allocations unchanged at 5/op, under heavy concurrent host load
(loadavg ≈ 8–18; absolutes inflated accordingly, the interleave is what
counts). The worker runtime compiles its guest once at Provision and sits
entirely off the scoring path.
Development
House style (applied Uber Go subset, review-enforced rules, frozen surfaces):
docs/dev/style.md.
go vet ./...
CGO_ENABLED=0 go test -p 1 ./...
Run tests with -p 1. Several caddytest-based tests share the Caddy
admin endpoint localhost:2019; when test binaries run in parallel they
race for that port. Serializing packages with -p 1 keeps the suite green.
Kill stray caddy processes before running tests. A leftover caddy from
a previous run or build check squats :2019 (and :8080), making unrelated
packages fail. Check with pgrep -a caddy and kill anything it lists.
Pre-M3 databases may contain mixed-case site keys written before site normalization; those rows are unreachable from the data plane and can be removed with any SQLite client.
Pre-M3 databases may also contain rules that the tightened M3 grammar no longer accepts — bot_score.min:0 or blank ja4_prefixes entries. Those rows fail snapshot compilation and Caddy refuses to boot; remove or edit them with any SQLite client. (ws_upgrade:false used to be on this list; negation landed in M5a and such rows are valid grammar again.)
Pre-M5b databases open cleanly: no schema additions at all — the coraza
config is a plain settings row (coraza) that appears on the first
PUT /id/merlon/settings/coraza; until then the engine is unset and every
matched coraza rule fails open.
Pre-M5a databases may contain phase-3 rules with non-route actions — the pre-M5a grammar allowed phases 1–3 but rejected only the route action itself, so legacy deny/challenge rows in phase 3 can exist. M5a validation inverts that: phase 3 accepts route exclusively, and such legacy rows fail snapshot compilation and block boot; delete them or move them to phase 1–2 with any SQLite client.
Pre-M5a databases otherwise open cleanly: the only schema additions are the workers and worker_versions tables, created on first open (CREATE TABLE IF NOT EXISTS) — nothing changes until you deploy a worker.
Pre-M4 databases open cleanly: the only schema addition is the feed_custom table, created on first open (CREATE TABLE IF NOT EXISTS), and feeds default to off — nothing changes until you add sources.
For the full-stack suites, the local gate script (scripts/ci-gate.sh) and
the make gate* targets — and the e2e suite's own ground truth in
e2e/README.md — see docs/usage/operations.md.
Design & spec
Full design and roadmap: design spec ·
M1 plan.
License
Merlon is MIT — LICENSE — with one deliberately fenced
carve-out: internal/ja4h and internal/ja4t implement FoxIO's JA4+
family names and published specification (JA4H, JA4T), and FoxIO
licenses the JA4+ family for non-commercial purposes. The project
takes that same posture for the island instead of pretending it away:
the FoxIO License 1.1 text ships as
LICENSE-JA4PLUS and NOTICE states the
coverage — the implementations are clean code written from the spec
text, the carve-out applies to the generators and the name/spec they
implement, and consuming fingerprint data is license-free: the
values you compute and the fp_intel rows you import are your data.
The JA4 base in ja4/ stays BSD-3-clean, and Merlon neither bundles
nor fetches any FoxIO data (ruling R-b10-nocsv). Non-commercial FLOSS
posture, stated once, everywhere it applies.