README
¶
exitmesh-agent
The open source ExitMesh agent. One Go binary, exitmesh-agent, runs in three roles:
- node (Kubernetes DaemonSet): scrapes the local kubelet and annotated pods, tails pod logs, evaluates PromQL and LogQL rules locally against bounded windows, and queues findings and per-resource metric facts for the coordinator.
- coordinator (Kubernetes StatefulSet, one replica): keeps a compact, normalized model of cluster state, records every meaningful change as a restorable delta, builds the change graph, evaluates state rules, and holds the single outbound connection to ExitMesh.
- host (Linux systemd service): both roles in one process for a Linux host without Kubernetes, collecting host state,
node_*metrics, the journal, and log files.
The agent speaks the open History Protocol and evaluates signed rule bundles. It is licensed under the Apache License 2.0.
Read-only and outbound-only
- Read-only. Kubernetes access is
get,list, andwatchonly. No writes of any kind, no Secrets, notokenreviewsorsubjectaccessreviews, nopods/exec,pods/attach,pods/portforward,nodes/proxy, ornodes/log. The agent never executes commands and never self-updates. - One disclosed host directory. Node agents mount
/var/log/podsread-only and write only to/var/lib/exitmesh(mode 0700, capped at 1 GiB by the agent). Hosts run as the unprivilegedexitmeshuser under a hardened systemd unit whose only writable path is/var/lib/exitmesh. - Outbound-only. The coordinator (or host agent) dials out to your ExitMesh endpoint over a WebSocket tunnel. Nothing listens outside the cluster network; node agents reach the coordinator over a ClusterIP Service. Shipped NetworkPolicies make default-deny namespaces work unchanged.
- Durable store and forward. Every checkpoint, delta, metric fact, and finding is spooled before it counts as emitted and is resent byte-identical until committed, so outages leave no gaps.
- Inspectable rules. Rule bundles are signed for integrity, not secrecy. Every rule that runs on your infrastructure can be read on the coordinator volume or under
/var/lib/exitmesh.
Historical search is not an ExitMesh feature
ExitMesh keeps no archive of raw logs, raw metrics, or manifests, and historical telemetry search is not an ExitMesh feature and will not become one. To search past logs or metrics beyond what is currently available in the cluster, run your own monitoring stack (for example Grafana with Loki and Mimir, or VictoriaMetrics and VictoriaLogs) fed by your own shipper (for example Alloy, vmagent, or an OpenTelemetry Collector), and connect it to the agent as a read-only lookback source. Investigations then query it through the agent, scope-injected at the query AST. Prometheus, Mimir, VictoriaMetrics, Loki, and VictoriaLogs adapters ship in the agent; see docs/configuration.md.
Quickstart: Kubernetes
Copy the endpoint, enrollment token, and trust roots (the root public keys of your ExitMesh deployment, which sign its rule bundles) from the ExitMesh connector page, then:
helm install exitmesh-agent oci://ghcr.io/cloud-exit/charts/exitmesh-agent \
--version 0.1.0 \
--namespace default \
--set endpoint=https://<endpoint from the connector page> \
--set enrollment.token=<enrollment token> \
--set-json 'trust.roots=["<root id>:<root public key>"]'
The chart creates two namespaces: exitmesh-node (Pod Security privileged, required for the hostPath volumes) and exitmesh (Pod Security restricted). The release record lives in the namespace passed with --namespace. Profiles, GitOps installs, and pre-created namespaces are covered in docs/install-kubernetes.md.
Quickstart: Linux hosts
# Debian and Ubuntu
sudo apt install ./exitmesh-agent_0.1.0_amd64.deb
# RHEL, Fedora, Rocky, Alma
sudo dnf install ./exitmesh-agent-0.1.0-1.x86_64.rpm
sudo sed -i 's#^endpoint: ""#endpoint: "https://<endpoint from the connector page>"#' /etc/exitmesh/agent.yaml
sudo sed -i 's#^ roots: \[\]# roots: ["<root id>:<root public key>"]#' /etc/exitmesh/agent.yaml
printf '%s\n' '<enrollment token>' | sudo install -m 0640 -o root -g exitmesh /dev/stdin /etc/exitmesh/enrollment-token
sudo systemctl start exitmesh-agent
Groups, golden images, and the tarball install are covered in docs/install-host.md.
Documentation
| Topic | Document |
|---|---|
| Kubernetes install, profiles, GitOps | docs/install-kubernetes.md |
| Host install | docs/install-host.md |
| Configuration reference | docs/configuration.md |
| Security model | docs/security.md |
| Operations runbook | docs/operations.md |
| Upgrades | docs/upgrades.md |
| Air-gap profile | docs/airgap.md |
| Uninstall and removal | docs/uninstall.md |
| Architecture and internal contracts | docs/architecture.md |
| History Protocol specification | protocol/SPEC.md |
| Command line | docs/cli.md |
| Rule bundle format and signing | docs/bundle-format.md |
| State rules (CEL) | docs/state-rules.md |
| PromQL rules | docs/promql-rules.md |
| LogQL subset | docs/logql-subset.md |
| Default rules and insight coverage | docs/insight-coverage.md |
| Investigation tools and lookback | docs/investigation.md |
| Kubernetes field catalog | docs/field-catalog.md |
| Published kube_* series | docs/kube-series.md |
| Host facts and availability | docs/host-facts.md |
| Log tailing contract | docs/log-contract.md |
Building
make build # static binary in bin/, CGO_ENABLED=0, -trimpath
make test # unit tests
make chart-test # helm lint and chart assertions
See CONTRIBUTING.md for every quality gate, the DCO sign-off, and dependency rules. Report vulnerabilities as described in SECURITY.md.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
exitmesh-agent
command
Command exitmesh-agent runs the ExitMesh agent in the node, coordinator, or host role and provides its administration subcommands (docs/cli.md).
|
Command exitmesh-agent runs the ExitMesh agent in the node, coordinator, or host role and provides its administration subcommands (docs/cli.md). |
|
exitmesh-bundle
command
Command exitmesh-bundle builds, signs, verifies, and inspects rule bundles (docs/bundle-format.md).
|
Command exitmesh-bundle builds, signs, verifies, and inspects rule bundles (docs/bundle-format.md). |
|
exitmesh-refcp
command
Command exitmesh-refcp serves the reference control plane over TLS with a generated self-signed CA for local testing.
|
Command exitmesh-refcp serves the reference control plane over TLS with a generated self-signed CA for local testing. |
|
internal
|
|
|
admin
Package admin is the local unix-socket administration API used by the CLI while the agent holds its lock.
|
Package admin is the local unix-socket administration API used by the CLI while the agent holds its lock. |
|
config
Package config loads and validates the agent configuration file (docs/configuration.md).
|
Package config loads and validates the agent configuration file (docs/configuration.md). |
|
coordinator
Package coordinator wires the coordinator role: state collection, the spool and writer session, cluster rule evaluation, findings, the node agent API, and the local administration socket.
|
Package coordinator wires the coordinator role: state collection, the spool and writer session, cluster rule evaluation, findings, the node agent API, and the local administration socket. |
|
deploytest
Package deploytest renders the Helm chart and checks the repository's deployment, packaging, and CI assets.
|
Package deploytest renders the Helm chart and checks the repository's deployment, packaging, and CI assets. |
|
deploytest/releaseinfo
command
Command releaseinfo prints the protocol, schema, and rule engine versions for release notes as KEY=VALUE lines.
|
Command releaseinfo prints the protocol, schema, and rule engine versions for release notes as KEY=VALUE lines. |
|
findings
Package findings turns rule and query observations into finding episodes and records (PRD 7.7).
|
Package findings turns rule and query observations into finding episodes and records (PRD 7.7). |
|
host
Package host runs the host role: node agent and coordinator in one process on a Linux host without Kubernetes.
|
Package host runs the host role: node agent and coordinator in one process on a Linux host without Kubernetes. |
|
hostfacts
Package hostfacts collects normalized Linux host state, reports per-fact availability, and tracks it as protocol ops.
|
Package hostfacts collects normalized Linux host state, reports per-fact availability, and tracks it as protocol ops. |
|
hostfacts/journal
Package journal is a pure-Go, read-only reader of the systemd journal file format.
|
Package journal is a pure-Go, read-only reader of the systemd journal file format. |
|
hostfacts/nodemetrics
Package nodemetrics runs upstream node_exporter collectors in process so node_* series and upstream rules work unchanged.
|
Package nodemetrics runs upstream node_exporter collectors in process so node_* series and upstream rules work unchanged. |
|
hostfacts/sqlitedb
Package sqlitedb reads rowid tables of SQLite files (with committed WAL frames) in pure Go and serves them to go-rpmdb as a database/sql driver.
|
Package sqlitedb reads rowid tables of SQLite files (with committed WAL frames) in pure Go and serves them to go-rpmdb as a database/sql driver. |
|
investigate
Package investigate serves bounded live and lookback investigation tools with AST scope injection (docs/investigation.md).
|
Package investigate serves bounded live and lookback investigation tools with AST scope injection (docs/investigation.md). |
|
kv
Package kv is the small durable key-value store used for agent metadata: alert state, log offsets, bundle state, key manifest sequence, cursors.
|
Package kv is the small durable key-value store used for agent metadata: alert state, log offsets, bundle state, key manifest sequence, cursors. |
|
node
Package node wires the node agent role: scrape, logs, rules, findings, and delivery to the coordinator.
|
Package node wires the node agent role: scrape, logs, rules, findings, and delivery to the coordinator. |
|
nodeapi
Package nodeapi implements the in-cluster HTTPS API between node agents and the coordinator (docs/architecture.md).
|
Package nodeapi implements the in-cluster HTTPS API between node agents and the coordinator (docs/architecture.md). |
|
privdrop
Package privdrop re-executes the process as an unprivileged user that keeps selected capabilities as ambient capabilities.
|
Package privdrop re-executes the process as an unprivileged user that keeps selected capabilities as ambient capabilities. |
|
redact
Package redact removes secret-looking values from text before it reaches evidence rings, spools, transmission, or diagnostics (PRD L6, 10).
|
Package redact removes secret-looking values from text before it reaches evidence rings, spools, transmission, or diagnostics (PRD L6, 10). |
|
rules/bundle
Package bundle defines the rule bundle format (docs/bundle-format.md): parsing, validation, signature and key manifest verification.
|
Package bundle defines the rule bundle format (docs/bundle-format.md): parsing, validation, signature and key manifest verification. |
|
rules/engine
Package engine evaluates CEL state rules and PromQL and LogQL alerting rules (docs/promql-rules.md).
|
Package engine evaluates CEL state rules and PromQL and LogQL alerting rules (docs/promql-rules.md). |
|
rules/logql
Package logql implements the LogQL subset published in docs/logql-subset.md without Loki code.
|
Package logql implements the LogQL subset published in docs/logql-subset.md without Loki code. |
|
rules/validators
Package validators builds the bundle validators backed by the real rule engines.
|
Package validators builds the bundle validators backed by the real rule engines. |
|
rulesdefault
Package rulesdefault serves the default ExitMesh rule bundle sources embedded from rules/ and builds their archives.
|
Package rulesdefault serves the default ExitMesh rule bundle sources embedded from rules/ and builds their archives. |
|
spool
Package spool is the writer's durable record store, the node agent queue, and directory locks.
|
Package spool is the writer's durable record store, the node agent queue, and directory locks. |
|
state
Package state normalizes Kubernetes objects into protocol state, edges, scopes, and kube_* series.
|
Package state normalizes Kubernetes objects into protocol state, edges, scopes, and kube_* series. |
|
telemetry/disk
Package disk enforces the state directory cap by shrinking the TSDB first and reporting pressure.
|
Package disk enforces the state directory cap by shrinking the TSDB first and reporting pressure. |
|
telemetry/evidence
Package evidence holds redacted matched lines under a per-node byte ceiling split into rule shares.
|
Package evidence holds redacted matched lines under a per-node byte ceiling split into rule shares. |
|
telemetry/logs
Package logs tails container and host log files per the contract in docs/log-contract.md.
|
Package logs tails container and host log files per the contract in docs/log-contract.md. |
|
telemetry/metricfacts
Package metricfacts summarizes local series into per-resource facts and change thresholds.
|
Package metricfacts summarizes local series into per-resource facts and change thresholds. |
|
telemetry/scrape
Package scrape is the agent's own scrape loop with per-node budgets, staleness, and coverage status.
|
Package scrape is the agent's own scrape loop with per-node budgets, staleness, and coverage status. |
|
telemetry/tsdb
Package tsdb wraps the Prometheus TSDB with rule-derived retention, a size ceiling, and pressure shrinking.
|
Package tsdb wraps the Prometheus TSDB with rule-derived retention, a size ceiling, and pressure shrinking. |
|
tunnel
Package tunnel implements the WebSocket tunnel binding (SPEC 9) and the enrollment client.
|
Package tunnel implements the WebSocket tunnel binding (SPEC 9) and the enrollment client. |
|
pkg
|
|
|
protocol
Package protocol implements the ExitMesh History Protocol v1 described in protocol/SPEC.md.
|
Package protocol implements the ExitMesh History Protocol v1 described in protocol/SPEC.md. |
|
protocol/client
Package client implements the transport-agnostic writer session of the History Protocol (protocol/SPEC.md sections 8 and 9): hello, resume and drain, windowed replay, acknowledgements, divergence handling with rebaseline, and the reverse MCP channel.
|
Package client implements the transport-agnostic writer session of the History Protocol (protocol/SPEC.md sections 8 and 9): hello, resume and drain, windowed replay, acknowledgements, divergence handling with rebaseline, and the reverse MCP channel. |
|
protocol/client/clienttest
Package clienttest provides an in-memory writer model implementing client.Hooks over a client.MemStore, for tests.
|
Package clienttest provides an in-memory writer model implementing client.Hooks over a client.MemStore, for tests. |
|
protocol/refcp
Package refcp is an in-memory reference control plane for contract and end-to-end tests.
|
Package refcp is an in-memory reference control plane for contract and end-to-end tests. |
|
protocol
|
|
|
genvectors
command
Command genvectors deterministically writes the language-neutral vectors under protocol/vectors.
|
Command genvectors deterministically writes the language-neutral vectors under protocol/vectors. |
|
Package rules embeds the default ExitMesh rule bundle sources, one directory per target type.
|
Package rules embeds the default ExitMesh rule bundle sources, one directory per target type. |
Click to show internal directories.
Click to hide internal directories.