netrules

package
v0.6.10 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 2, 2026 License: MIT Imports: 25 Imported by: 0

Documentation

Index

Constants

View Source
const (
	BlockTable  = "aerolvm_block"
	BlockedSet  = "blocked"
	ReleasedSet = "released"
)

The block table and its sets.

View Source
const (
	BackendExec    = "exec"
	BackendNetlink = "netlink"

	ChainDockerUser  = "DOCKER-USER"
	ChainAerolvmUser = "AEROLVM-USER"
	// ChainAerolvmFC holds the per-guest-IP rules for Firecracker VMs (egress
	// Phase 4), apart from the container engines' chains: its FORWARD
	// accepts are scoped to the TAP subnet.
	ChainAerolvmFC = "AEROLVM-FC"
	// ChainAerolvmInput holds per-IP drops for traffic from restricted
	// sandboxes to the host itself (sandboxd API, SSH gateway, cluster port).
	// Shared by the dockerd and containerd managers: rules are keyed by
	// sandbox IP and the two bridges never share an IP.
	ChainAerolvmInput = "AEROLVM-INPUT"
)

Backend names for SB_NETRULES_BACKEND.

View Source
const ChainAerolvmFloor = "AEROLVM-FLOOR"

ChainAerolvmFloor names the deny floor (plans/egress-domain-filtering.md §5.10 PC-2): DROPs from a sandbox subnet to each floor address, jumped from the top of the user chain so they sit above every per-sandbox ACCEPT. A chain of its own because the floor is replaced as a whole when the operator file changes, and the rule backends cannot list rules to find the old ones.

Each sandbox subnet gets its own chain, FloorChainFor(subnet). Docker, containerd and Firecracker each floor their own subnet from their own user chain; when they shared this one chain, every SetFloor flushed it, and on a node running more than one of them only the last subnet kept a floor. This name is now only the legacy chain an older daemon filled, emptied at the first floor so its leftover jump is a no-op.

View Source
const ChainAerolvmNode = "AEROLVM-NODE"

ChainAerolvmNode is the node guard: sandboxes are default-denied from the node they run on. Every packet a sandbox subnet sends to the node itself (its bridge gateways, its private and any locally bound public address: whatever reaches INPUT) is dropped unless it answers a connection the host opened (sandboxd to the toolbox) or goes to a port on the allowlist the service computes (the egress gateway's DNS and proxy ports, and the few others setup/config-defaults.md lists).

It replaces an enumerated list of ports to drop (the control-port guard): a node also runs services AerolVM does not know about (an EDR agent, a log shipper, a node exporter, a SIEM forwarder), and every one of them was reachable from every default sandbox.

Layout, all in the filter table:

INPUT          first rule: -j AEROLVM-NODE (kept first, like the floor's jump)
AEROLVM-NODE   1: conntrack RELATED,ESTABLISHED -j RETURN
               then per sandbox subnet: -s <subnet> -j AEROLVM-NODE-<hash>-A|B
AEROLVM-NODE-<hash>-A|B
               -s <subnet> [-d <addr>] -p tcp|udp --dport N -j RETURN (the allowlist)
               -s <subnet> -j DROP

Its own chain, not rules in AEROLVM-INPUT, so no per-sandbox rule can sit above it: an allowlist RETURN for a CIDR that covers the node's address (AEROLVM-INPUT's inputPolicyReturnSpec) would otherwise skip the guard. RETURN, never ACCEPT, so a host firewall such as ufw still decides what it lets in.

Variables

View Source
var (
	// ErrNodeGuardDisabled means network rules are off on this node, so
	// nothing keeps sandboxes off the node's own services.
	ErrNodeGuardDisabled = errors.New("netrules: network rules are disabled; sandboxes can reach every service on this node")
	// ErrNodeGuardUnsupported means the rule backend cannot manage chains.
	ErrNodeGuardUnsupported = errors.New("netrules: backend cannot manage the node guard chains")
)
View Source
var ErrFloorUnsupported = errors.New("netrules: backend cannot manage the egress floor chain")

ErrFloorUnsupported means the rule backend cannot manage the floor chain.

Functions

func BlockedInSets added in v0.6.9

func BlockedInSets(ip string) (blocked, released bool)

BlockedInSets reports what the mirror holds for ip, for tests and diagnostics.

func FloorChainFor added in v0.6.9

func FloorChainFor(subnet netip.Prefix) string

FloorChainFor is the floor chain for one sandbox subnet: the prefix plus eight hex digits of the subnet's hash, 22 characters, inside iptables' 28-character chain-name limit.

func NodeChainsFor added in v0.6.9

func NodeChainsFor(subnet netip.Prefix) [2]string

NodeChainsFor is the pair of chains one sandbox subnet's guard alternates between: a new allowlist is built in the one nothing jumps to, then the jump moves, so a change never leaves the subnet unguarded (a flush and refill of one chain would, briefly). 23 characters, inside iptables' 28.

func ResetBlockSetsForTest added in v0.6.9

func ResetBlockSetsForTest()

ResetBlockSetsForTest forgets the mirror. Test-only seam: the mirror is process-wide, as a node's runtimes share one process and one table.

func ResetFloorsForTest added in v0.6.9

func ResetFloorsForTest()

ResetFloorsForTest forgets every registered floor. Test-only seam: the registry is process-wide, as a node's runtimes share one process.

func SetSandboxInterconnect added in v0.6.9

func SetSandboxInterconnect(allow bool)

SetSandboxInterconnect sets whether sandboxes on one node may connect to each other; the daemon sets it from config before the first floor.

func WarnIfLegacyIptables added in v0.5.6

func WarnIfLegacyIptables(backend string, logger *slog.Logger)

WarnIfLegacyIptables logs a boot-time warning when the netlink backend is selected on a host whose iptables binary runs in legacy (xtables) mode. The netlink backend writes to the nft tables, which legacy iptables cannot see — the kernel evaluates BOTH rule sets, so flipping SB_NETRULES_BACKEND with live sandboxes strands the previous backend's ACCEPT/DROP rules on recycled container IPs (TODOS.md "netrules backend switch on iptables-legacy hosts"). Best-effort: a missing binary or exec failure stays silent — no signal is not a legacy host.

Types

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

func New

func New(enabled bool) (*Manager, error)

func NewWithBackend added in v0.5.5

func NewWithBackend(backend RuleBackend) *Manager

NewWithBackend builds an enabled Manager over an injected backend. Test seam only — production wiring goes through New.

func NewWithOptions added in v0.5.6

func NewWithOptions(enabled bool, backend, userChain string) (*Manager, error)

NewWithOptions builds a Manager with the chosen RuleBackend. userChain selects the filter chain for per-IP rules; empty defaults to DOCKER-USER. Unknown backend names error so misconfig is loud. Enabled backends are linux-only (see newEnabledManager); other platforms always return a disabled manager.

func (*Manager) ApplyEgressPolicy added in v0.5.4

func (m *Manager) ApplyEgressPolicy(containerIP string, allowCIDRs, denyCIDRs []string) error

ApplyEgressPolicy installs a per-container selective egress policy in DOCKER-USER, scoped by source IP and comment-tagged (see egressPolicyComment). The mode follows the shared allow-wins precedence (plans/ egress-domain-filtering.md D4):

  • allowCIDRs only → allowlist: ACCEPT each CIDR, DROP everything else.
  • denyCIDRs only → blocklist: DROP each CIDR, leave the rest to Docker's default ACCEPT.
  • both → allow wins, then deny, then default ACCEPT: ACCEPTs sit above the DROPs and there is no catch-all.

"allow + deny 0.0.0.0/0" is the portable allowlist spelling (E2B's allowOut + denyOut all): normalizeLists maps it to allow-only here, in the one place every apply and clear goes through, so a clear from the stored row always mirrors what the apply installed.

Re-apply is idempotent: every rule is Exists-checked before Insert, so the start/reconcile reapply paths can call this repeatedly without duplicating.

func (*Manager) BlockAllEgress

func (m *Manager) BlockAllEgress(containerIP string) error

BlockAllEgress installs a DROP rule for traffic originating from containerIP. The rule lives in DOCKER-USER, the chain Docker explicitly reserves for operator-defined firewall rules. DOCKER-USER is jumped from FORWARD *before* DOCKER-FORWARD, so our DROP fires before Docker's blanket "iifname docker0 accept" rule that would otherwise short-circuit any rule appended directly to FORWARD. This works on iptables-legacy and on Docker 28+/iptables-nft (which writes through to nftables) alike.

func (*Manager) BlockAllEgressReport added in v0.5.6

func (m *Manager) BlockAllEgressReport(containerIP string) (bool, error)

BlockAllEgressReport is BlockAllEgress with the idempotency outcome surfaced: inserted is true only when the rule was genuinely absent and had to be re-installed. Reconcile reapplies unconditionally (the Exists guard below is what makes that safe), so "we called Apply" is a constant rate and useless as a signal — "the rule was missing when we looked" is the actual isolation-drift event worth counting. Disabled manager and empty IP report inserted=false: nothing was installed, so nothing drifted.

func (*Manager) BlockAllIngress added in v0.1.7

func (m *Manager) BlockAllIngress(containerIP string) error

BlockAllIngress installs a DROP rule for traffic destined for containerIP, the mirror of BlockAllEgress on the destination axis. Used by the network quota enforcer when net_bytes_in_limit is crossed. The honest caveat (also documented in plans/network-usage-tracking.md): host-side ingress is counted after the NIC has accepted the packet, so the meter is "what the container would have seen" rather than "bytes spent on the wire." Same chain (DOCKER-USER) and idempotency check pattern as the egress mirror.

func (*Manager) BridgeSubnet added in v0.6.6

func (m *Manager) BridgeSubnet() netip.Prefix

BridgeSubnet returns the subnet set by SetBridgeSubnet (the containerd CNI bridge), or the zero prefix.

func (*Manager) ClearAddress added in v0.6.9

func (m *Manager) ClearAddress(ip string) error

ClearAddress removes every per-address rule keyed by ip from this manager's user chain and AEROLVM-INPUT. Call it when the address's holder is gone or about to be, never while a sandbox still uses the address. A backend that can't list a chain clears the specs the manager knows without their lists: block-all, the ingress block and the hold.

func (*Manager) ClearBlockAllEgress

func (m *Manager) ClearBlockAllEgress(containerIP string) error

ClearBlockAllEgress removes containerIP's block-all for good, when the address is let go or about to be (destroy, a failed create, a stop): on the block sets it is Forget (the delete that pays the grace period); a sandbox that keeps its address and only stops being blocked is lifted instead (LiftBlockAllEgress).

func (*Manager) ClearBlockAllIngress added in v0.1.7

func (m *Manager) ClearBlockAllIngress(containerIP string) error

func (*Manager) ClearEgressPolicy added in v0.5.4

func (m *Manager) ClearEgressPolicy(containerIP string, allowCIDRs, denyCIDRs []string) error

ClearEgressPolicy removes the rules ApplyEgressPolicy would have installed for the same (containerIP, allowCIDRs, denyCIDRs). The caller passes the policy persisted on the sandbox row so cleanup is exact and comment-scoped — the blanket BlockAllEgress DROP (no comment) is left untouched.

func (*Manager) ClearHoldEgress added in v0.6.6

func (m *Manager) ClearHoldEgress(containerIP string) error

ClearHoldEgress removes the hold DROP. Only a successful attach lifts a hold; callers enforce that.

func (*Manager) Enabled

func (m *Manager) Enabled() bool

func (*Manager) EnsureChain added in v0.5.6

func (m *Manager) EnsureChain() error

EnsureChain bootstraps the filter user chain and its FORWARD jump once per Manager lifetime. Idempotent and latched like EnsureLayer4Ready: concurrent callers single-flight through chainMu; success sets chainReady.

func (*Manager) HoldEgress added in v0.6.6

func (m *Manager) HoldEgress(containerIP string) error

HoldEgress installs the hold DROP for containerIP. Idempotent.

func (*Manager) LiftBlockAllEgress added in v0.6.9

func (m *Manager) LiftBlockAllEgress(containerIP string) error

LiftBlockAllEgress lifts containerIP's block-all while the sandbox keeps its address (the gateway attach of a hostname-filtered create, a quota or policy that no longer blocks, a warm slot adopted without one): on the block sets it is one add to @released. The address stays in @blocked until Forget. On the per-address rules it is ClearBlockAllEgress.

func (*Manager) PrepareBlockSets added in v0.6.9

func (m *Manager) PrepareBlockSets() error

PrepareBlockSets moves this manager's block-alls into the block sets: it makes sure the table exists, reads it into the process's mirror (the first manager to prepare does), and converts the per-address block rules an older daemon left in this manager's chains (set element first, rule second, so the address is never open). A backend without sets, or a failure here, leaves the manager on the per-address rules, which stay correct, only slower; the error says why. Call once at boot, before the manager serves creates.

func (*Manager) ReassertBlockSets added in v0.6.9

func (m *Manager) ReassertBlockSets() (repaired bool, err error)

ReassertBlockSets checks the table against the mirror: a missing table or element is put back (repaired reports it), and a release the kernel holds that this process never made is deleted, as nothing here would ever lift a block for it. A no-op on the per-address rules.

func (*Manager) ReassertChain added in v0.5.6

func (m *Manager) ReassertChain() error

ReassertChain re-runs the chain + FORWARD-jump bootstrap WITHOUT the latch, so a caller (e.g. the containerd netns reconcile ticker) can re-assert the jump after a dockerd restart flushes/reorders FORWARD and drops it. Both steps are idempotent; a no-op when disabled or the backend cannot bootstrap.

func (*Manager) ReassertFloor added in v0.6.9

func (m *Manager) ReassertFloor() (repaired bool, err error)

ReassertFloor puts this manager's floor back if something removed it: the chain, the user chain's jump to it, or any of its DROPs (a host firewall reload, a flush). Cheap when nothing is missing: the jump check and one rule check per floor address, no rewrite, so a floor in place never has the empty moment SetFloor's flush and refill has. repaired reports that the floor was rewritten. A no-op until SetFloor has run.

func (*Manager) ReassertNodeGuard added in v0.6.9

func (m *Manager) ReassertNodeGuard() (repaired bool, err error)

ReassertNodeGuard puts the guard back if something removed it: the INPUT jump (a host firewall reload that rewrote INPUT, or one that put its own rules above ours), the established-return rule, the subnet's jump, or the drop at the end of its chain (a flush). Cheap when nothing is missing: a handful of rule checks. repaired reports that the guard was rebuilt. A no-op until SetNodeGuard has succeeded once.

func (*Manager) ResetChainLatch added in v0.5.6

func (m *Manager) ResetChainLatch()

ResetChainLatch clears the bootstrap latch. Test-only seam.

func (*Manager) SetBridgeSubnet added in v0.5.6

func (m *Manager) SetBridgeSubnet(subnet string)

SetBridgeSubnet records the sandbox bridge subnet whose forwarded traffic EnsureChain/ReassertChain must ACCEPT (below any per-IP DROP). Empty = no-op.

func (*Manager) SetFloor added in v0.6.6

func (m *Manager) SetFloor(subnet netip.Prefix, cidrs []netip.Prefix) error

SetFloor replaces the node-wide deny floor for sandboxes on subnet. An empty cidrs list clears the operator's part. The flush and refill are separate steps, so the floor is briefly empty while it is replaced; the egress gateway's own copy of the floor (its deny_floor set) stays in place throughout. A subnet seen for the first time also re-renders every other runtime's floor, so each drops traffic to it.

func (*Manager) SetNodeGuard added in v0.6.9

func (m *Manager) SetNodeGuard(subnet netip.Prefix, allow []NodePort) error

SetNodeGuard default-denies sandboxes on subnet from the node itself, except allow. Replacing the allowlist is safe while sandboxes run: the new list goes into the subnet's idle chain, the jump moves to it, then the old chain is emptied. In between, a packet passes both chains, so it gets through only if both allow it (fail closed, never open). Idempotent: the service calls it at boot and whenever the allowlist changes.

type NodePort added in v0.6.9

type NodePort struct {
	// Proto is "tcp" or "udp".
	Proto string
	// Port is the destination port, 1-65535.
	Port uint16
	// Dst narrows the entry to one node address (a resolver the sandboxes
	// are pointed at); the zero value allows the port on every address.
	Dst netip.Addr
}

NodePort is one thing sandboxes may still reach on their own node.

func (NodePort) String added in v0.6.9

func (p NodePort) String() string

String is "tcp/15080", or "udp/53@10.0.1.5" for one address.

type RuleBackend added in v0.5.5

type RuleBackend interface {
	Exists(table, chain string, rulespec ...string) (bool, error)
	Insert(table, chain string, pos int, rulespec ...string) error
	Delete(table, chain string, rulespec ...string) error
}

RuleBackend is the subset of iptables operations the Manager drives. Production always wraps *go-iptables' IPTables; tests substitute an in-memory backend so rule-state semantics (which rules survive an adopt, a clear, a reapply) are assertable without root or a linux host.

func NewNetlinkBackend added in v0.5.6

func NewNetlinkBackend() (RuleBackend, error)

NewNetlinkBackend wires the per-operation nftables connection factory. Callers must be on linux; non-linux builds use the stub that always errors.

type RuleInsert added in v0.6.9

type RuleInsert struct {
	RuleRef
	Pos int
}

RuleInsert is a rule to ensure at an iptables-style position: 1 is the top, a position past the last rule appends (the RuleBackend.Insert contract).

type RuleRef added in v0.6.9

type RuleRef struct {
	Table string
	Chain string
	Spec  []string
}

RuleRef names one rule: its table, its chain and its iptables-shaped spec.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL