netfilter

package
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 13, 2026 License: MIT Imports: 10 Imported by: 0

Documentation

Overview

Package netfilter compiles an egress authorisation into an IP-layer firewall.

The problem it solves is that, until now, cloop's egress allowlist was advisory. pkg/egressbroker is an HTTP forward proxy: it checks hosts, ports, CIDRs and the SSRF block set beautifully, and it only ever sees traffic a workload chose to send it. A harness that opens a raw socket, runs `curl --noproxy '*'`, resolves a name over DoH or speaks anything that is not HTTP walks straight past every one of those checks. The container driver said as much in its own doc comment — "it does not filter egress" — and the Kubernetes driver said the same about NetworkPolicy. Both were honest, and both left the sandbox with unrestricted outbound access the moment an operator turned the network on.

A firewall closes that, but only if it means the same thing everywhere. So there is one compiler here and several renderers: Compile turns an authorisation into an ordered, first-match-wins Policy, and the backends render that same Policy as nftables rules or as a Kubernetes NetworkPolicy. Evaluate answers "what would this policy do to that packet" without any backend at all, which is what lets a test compare the firewall against the proxy address-by-address instead of trusting that two hand-written rule sets agree.

Relationship to the proxy's block set

egressbroker.BlockReason refuses loopback, RFC1918, link-local, the cloud metadata endpoint, CGNAT and multicast unless the grant names an explicit CIDR. This package reproduces that set as prefixes and reproduces the waiver rule: an allow for a granted CIDR is emitted *before* the drop that would otherwise cover it, so an explicit CIDR — and nothing else — buys reach into blocked space. agreement_test.go checks the two implementations against each other in both directions.

The agreement is exact but for one deliberate delta. The proxy normalises IPv6 encodings that carry an IPv4 address (NAT64, 6to4, v4-translated, v4-compatible) and then checks the address inside. A packet filter cannot do arithmetic on an embedded address, so this package drops those transition prefixes wholesale. That is stricter than the proxy — 64:ff9b::8.8.8.8 is a public address the proxy would allow — and strictness is the safe direction. No sandbox reaches the Internet through a translation prefix it addresses itself; that is a gateway's job.

What a hostname allowlist becomes

It becomes the public Internet. "*.github.com" is a name, and names do not exist at layer 3. Compile therefore refuses to silently widen: opening direct egress for a host-pattern grant requires AllowPublicInternet, and the resulting Policy carries a Warning saying exactly how much wider the firewall is than the grant. The narrow configuration is the brokered one, where the only reachable address is the proxy and the host allowlist is enforced where it can be — inside it.

The package depends on nothing but the standard library, so the executor drivers can all reach it without dragging the broker's storage and crypto along behind. Compile and the renderers are pure functions; apply.go is the one file that touches the host, and it is separated precisely so that everything deciding what the firewall *means* stays testable without one.

Index

Constants

View Source
const (
	MaxPorts    = 64
	MaxPrefixes = 256
)

MaxPorts and MaxPrefixes bound a compiled policy. The limits exist because the rendered ruleset is handed to another program — nft, or the Kubernetes API server — and an unbounded list turns a config mistake into an argv overflow or a rejected object, neither of which reads as "your grant is too big".

Variables

View Source
var ErrUnavailable = errors.New("netfilter: host packet filtering is unavailable")

ErrUnavailable reports that host-side filtering cannot be performed. It is a sentinel so callers can degrade deliberately — refusing to start an unfiltered sandbox — rather than by pattern-matching an error string.

Functions

func BlockReasonForPrefix

func BlockReasonForPrefix(p netip.Prefix) string

BlockReasonForPrefix reports why a prefix is blocked, or "" when it is not.

A prefix is blocked when it is contained in a blocked one — 10.8.0.0/24 is private — and also when it *contains* one, because allowing 0.0.0.0/0 would otherwise be reported as unblocked while covering every blocked range.

func RenderNftables

func RenderNftables(p Policy, opts NftablesOptions) (string, error)

RenderNftables renders the policy as a script for `nft -f -`.

inet rather than ip and ip6 is not a convenience: two families mean two rulesets that can drift, and a v6 ruleset an operator forgot is the whole firewall bypassed by a AAAA record.

func TableName

func TableName(prefix, id string) string

TableName builds a table name for a sandbox from an identifier.

nft identifiers are narrower than the handle IDs and executor IDs cloop generates, so every character outside the grammar becomes an underscore and the result is truncated to fit. Collisions are possible in principle and harmless in practice only if the caller passes something unique; pass the handle ID, which is.

func ValidateInterfaceName

func ValidateInterfaceName(name string) error

ValidateInterfaceName checks a kernel interface name.

The name reaches nft as part of a script, so an unchecked one is command injection into the host's firewall — and unlike a table name, this one can originate from a runtime's output rather than from cloop.

func ValidateNftName

func ValidateNftName(name string) error

ValidateNftName checks a table or chain identifier.

The name reaches nft as part of a script, so an unchecked one is command injection into the host's firewall. The grammar here is narrower than nft's on purpose: cloop generates these names, so anything outside [A-Za-z0-9_-] is a bug rather than a use case.

Types

type Applier

type Applier struct {
	// contains filtered or unexported fields
}

Applier installs and removes rulesets on the host.

The nft path is resolved once at construction, for the same reason pkg/executor/container resolves its runtime once: a PATH that changes mid-run must not change which binary manipulates the host firewall.

func NewApplier

func NewApplier() (*Applier, error)

NewApplier locates nft. It performs no privileged operation, so a caller can construct one to ask Available() what would happen.

func (*Applier) Apply

func (a *Applier) Apply(ctx context.Context, p Policy, opts NftablesOptions) error

Apply installs the policy, replacing any ruleset previously installed under the same table.

Replacement rather than mutation is what makes this safe to call again for the same sandbox: the rendered script deletes the table and recreates it in one transaction, so a second Apply cannot leave rules from the first behind. That is the property a reconcile loop needs.

func (*Applier) Available

func (a *Applier) Available(ctx context.Context) error

Available reports whether this process can actually install a ruleset.

It checks by listing, which needs the same CAP_NET_ADMIN a load does but changes nothing — so a preflight can call it on a healthy host without side effects. The distinction it draws matters: "nft is missing" and "nft is present but this process is unprivileged" have completely different fixes, and a caller that reported them as one error would send an operator to install a package they already have.

func (*Applier) Path

func (a *Applier) Path() string

Path reports the resolved nft binary, for diagnostics.

func (*Applier) Remove

func (a *Applier) Remove(ctx context.Context, table string) error

Remove deletes a sandbox's table.

A table that is already gone is success. Teardown runs on paths that also run after a crash, and a cleanup that fails because it has nothing to do turns every restart into a spurious error in the log.

type Blocked

type Blocked struct {
	Prefix netip.Prefix
	Reason string
}

Blocked is one entry of the hard-block set: a prefix and the sentence that explains it.

func BlockedPrefixes

func BlockedPrefixes() []Blocked

BlockedPrefixes returns the hard-block set, in evaluation order. The slice is a copy: a caller that sorts or filters it cannot reach into the policy every sandbox on the host is compiled against.

type IPBlock

type IPBlock struct {
	CIDR   string   `json:"cidr"`
	Except []string `json:"except,omitempty"`
}

IPBlock is a CIDR with holes.

type Input

type Input struct {
	// AllowCIDRs are the address ranges the authorisation names. These are
	// the only thing that waives the block set, exactly as in the proxy.
	AllowCIDRs []netip.Prefix

	// AllowPorts bounds every destination allow. Empty means the caller
	// named no ports, and Compile refuses rather than guessing: an allow
	// rule with no port restriction is a hole, and "they probably meant
	// 80 and 443" is not a thing a firewall compiler should decide.
	AllowPorts []uint16

	// Brokers are egress proxy endpoints the sandbox must reach. Naming
	// any of them is what makes the policy brokered.
	Brokers []netip.AddrPort

	// Resolvers are DNS servers the sandbox may query directly. Each is
	// opened on both UDP and TCP — a truncated answer retries over TCP, and
	// a resolver reachable only over UDP fails on large responses in a way
	// that looks like a DNS outage rather than a firewall rule.
	//
	// A brokered policy needs none: the proxy resolves on the sandbox's
	// behalf, which is also what makes its resolve-once pinning meaningful.
	Resolvers []netip.AddrPort

	// AllowPublicInternet opens every address outside the block set on
	// AllowPorts. It exists because a hostname allowlist cannot be
	// expressed at layer 3, and it is a separate field rather than an
	// inference from HostPatterns so that widening the filter to the whole
	// Internet is something a caller states rather than something that
	// happens to it.
	AllowPublicInternet bool

	// HostPatterns is the L7 allowlist this policy could not enforce. It is
	// used only to write an accurate warning.
	HostPatterns []string
}

Input is the authorisation to compile.

It is deliberately not egressbroker.Grant. A grant carries subjects, TTLs, byte quotas and HTTP methods, none of which a packet filter can act on, and depending on that package would drag the broker's storage and crypto into something that should stay a pure function. The caller projects a grant down to the dimensions layer 3 understands; GrantInput in the executor wiring does exactly that.

type LabelSelector

type LabelSelector struct {
	MatchLabels map[string]string `json:"matchLabels,omitempty"`
}

LabelSelector selects the Pods this policy governs.

type Mode

type Mode uint8

Mode is the shape of an authorisation, derived by Compile and used for display and for the rendered header.

const (
	// ModeIsolated is a sandbox with no egress at all.
	ModeIsolated Mode = iota
	// ModeBrokered is a sandbox whose only reachable address is the egress
	// proxy. The L3 filter and the L7 allowlist coincide here: nothing can
	// leave except through the thing that enforces hosts and methods.
	ModeBrokered
	// ModeFiltered is a sandbox dialling destinations directly under a
	// CIDR/port allowlist. Narrow when the grant names CIDRs, and no
	// narrower than "the public Internet on these ports" when it names
	// hostnames.
	ModeFiltered
)

func (Mode) String

func (m Mode) String() string

String renders the mode.

type NetworkPolicy

type NetworkPolicy struct {
	APIVersion string            `json:"apiVersion"`
	Kind       string            `json:"kind"`
	Metadata   NetworkPolicyMeta `json:"metadata"`
	Spec       NetworkPolicySpec `json:"spec"`
}

NetworkPolicy is the subset of networking.k8s.io/v1 NetworkPolicy this package emits. Only the fields set here are modelled, matching the Kubernetes driver's convention for its own object types.

func RenderNetworkPolicy

func RenderNetworkPolicy(p Policy, opts NetworkPolicyOptions) (*NetworkPolicy, error)

RenderNetworkPolicy translates the policy into a NetworkPolicy object.

type NetworkPolicyMeta

type NetworkPolicyMeta struct {
	Name        string            `json:"name,omitempty"`
	Namespace   string            `json:"namespace,omitempty"`
	Labels      map[string]string `json:"labels,omitempty"`
	Annotations map[string]string `json:"annotations,omitempty"`
}

NetworkPolicyMeta is the object metadata the caller fills in.

type NetworkPolicyOptions

type NetworkPolicyOptions struct {
	Name      string
	Namespace string
	// PodSelector must match exactly the Pods this policy should govern. An
	// empty selector in Kubernetes means *every* Pod in the namespace, so
	// an empty one here is refused rather than quietly firewalling the
	// namespace's other tenants.
	PodSelector map[string]string
	Labels      map[string]string
	Annotations map[string]string

	// AllowClusterDNS opens UDP and TCP 53 to the kube-dns Pods.
	//
	// It is a selector rather than an address because cluster DNS lives on
	// a Service ClusterIP in private space, and which side of the policy
	// the CNI applies its DNAT on varies by CNI. Selecting the kube-system
	// namespace is the form that works everywhere; an ipBlock for the
	// Service CIDR is the form that works until it doesn't.
	//
	// Without it, a default-deny egress policy breaks name resolution, and
	// the failure reads as "the network is broken" rather than "DNS is
	// denied" — which is why it defaults on in the driver.
	AllowClusterDNS bool
	// DNSNamespaceLabels selects the namespace running cluster DNS.
	// Defaults to kubernetes.io/metadata.name=kube-system, the label the
	// API server sets automatically on every namespace since 1.21.
	DNSNamespaceLabels map[string]string
}

NetworkPolicyOptions carry what the Policy cannot express: the Pod selector, and the cluster-shaped exceptions.

type NetworkPolicyPeer

type NetworkPolicyPeer struct {
	IPBlock           *IPBlock       `json:"ipBlock,omitempty"`
	NamespaceSelector *LabelSelector `json:"namespaceSelector,omitempty"`
	PodSelector       *LabelSelector `json:"podSelector,omitempty"`
}

NetworkPolicyPeer is one destination: an address range, or a selector.

type NetworkPolicyPort

type NetworkPolicyPort struct {
	Protocol string `json:"protocol,omitempty"`
	Port     *int   `json:"port,omitempty"`
}

NetworkPolicyPort is one destination port.

type NetworkPolicyRule

type NetworkPolicyRule struct {
	To    []NetworkPolicyPeer `json:"to,omitempty"`
	From  []NetworkPolicyPeer `json:"from,omitempty"`
	Ports []NetworkPolicyPort `json:"ports,omitempty"`
}

NetworkPolicyRule is one egress or ingress entry.

type NetworkPolicySpec

type NetworkPolicySpec struct {
	PodSelector LabelSelector       `json:"podSelector"`
	PolicyTypes []string            `json:"policyTypes"`
	Egress      []NetworkPolicyRule `json:"egress,omitempty"`
	Ingress     []NetworkPolicyRule `json:"ingress,omitempty"`
}

NetworkPolicySpec is the policy body.

type NftablesOptions

type NftablesOptions struct {
	// Table is the inet table to install into. Deleting and recreating this
	// one table is what makes an apply atomic and idempotent, so it must
	// belong to this sandbox alone.
	Table string

	// Bridge, when set, renders the host-side form: a single forward chain
	// that filters only traffic arriving from that interface.
	//
	// This is how a control-plane host firewalls a container without ever
	// entering the container's network namespace. The bridge exists from
	// the moment the runtime creates the network, which is before any
	// workload can join it, so there is no window in which the sandbox is
	// running unfiltered — the ordering problem that makes the obvious
	// "start it, then firewall it" approach unsafe.
	//
	// Empty renders the in-namespace form instead: output, input and
	// forward chains that default to drop, for a filter installed inside
	// the sandbox's own namespace.
	Bridge string
}

NftablesOptions select where the ruleset attaches.

type Policy

type Policy struct {
	// Mode summarises the shape of the authorisation for display.
	Mode Mode
	// Rules are evaluated in order; the first match decides.
	Rules []Rule
	// Warnings record where the filter is necessarily wider than the
	// authorisation it was compiled from. They are rendered into the
	// output of every backend, because an operator reading a ruleset needs
	// to see that "*.github.com" became "the Internet".
	Warnings []string
}

Policy is a compiled firewall: an ordered rule list evaluated first-match wins, with an implicit drop at the end.

The default is not a field. A firewall whose default could be "allow" is a firewall an operator can misconfigure into uselessness, and there is no authorisation this package can be handed that would justify it.

func Compile

func Compile(in Input) (Policy, error)

Compile turns an authorisation into an ordered filter.

Rule order is the security-relevant part, so it is fixed here rather than left to a renderer:

  1. sandbox-local loopback, so a harness that binds 127.0.0.1 works;
  2. allows for explicitly granted CIDRs, ahead of the drops they waive;
  3. allows for the broker and for resolvers, which are infrastructure;
  4. drops for the block set;
  5. the public-Internet allow, if the caller asked for one;
  6. the implicit final drop.

Steps 2 and 4 are the waiver rule from egressbroker.CheckAddr expressed as ordering: a granted 10.8.0.0/24 is allowed on its ports before 10.0.0.0/8 is dropped, so the grant buys that prefix and those ports and nothing else. Putting the Internet allow last rather than first is what keeps step 4 meaningful.

func (Policy) Evaluate

func (p Policy) Evaluate(addr netip.Addr, port uint16, proto Proto) (Verdict, string)

Evaluate reports what the policy does to one packet, and why.

This is the policy's own semantics, independent of any backend: the tests that compare this package against egressbroker's address checks call it, and so does `cloop egress firewall --check`. A renderer that disagrees with Evaluate is a renderer with a bug.

func (Policy) WireOnly

func (p Policy) WireOnly() Policy

WireOnly returns the policy with its namespace-local rules removed.

It answers "what can leave the sandbox", which is the question the proxy's address checks answer too — and the only footing on which the two can be compared. The loopback allows are the whole difference: they permit the sandbox to talk to itself, a conversation that never reaches an interface and that the proxy, sitting on another host, has no opinion about.

type Proto

type Proto uint8

Proto narrows a rule to one transport, or to all of them.

const (
	// ProtoAny matches every transport, including ICMP and anything else
	// that is neither TCP nor UDP. A drop rule wants this: an exfiltration
	// channel over ICMP is still an exfiltration channel.
	ProtoAny Proto = iota
	ProtoTCP
	ProtoUDP
)

func (Proto) String

func (p Proto) String() string

String renders the transport for rule comments.

type Rule

type Rule struct {
	Verdict Verdict
	// Prefix is the destination address range.
	Prefix netip.Prefix
	// Ports is the destination port allowlist. Empty means every port,
	// which is what a drop rule wants and what an allow rule should almost
	// never have.
	Ports []uint16
	Proto Proto
	Scope Scope
	// Reason is the human sentence that lands in an nft comment, in the
	// NetworkPolicy annotation and in an operator's terminal. It is part of
	// the output, not a debugging aid.
	Reason string
}

Rule is one line of the compiled filter.

func (Rule) String

func (r Rule) String() string

String renders the rule as a single readable line.

type Scope

type Scope uint8

Scope says which filters a rule is meaningful to.

const (
	// ScopeWire covers traffic that leaves the sandbox's network namespace.
	// Every backend can express it.
	ScopeWire Scope = iota
	// ScopeLocal covers traffic that never reaches a wire — the sandbox
	// talking to its own loopback. An in-namespace packet filter has to
	// allow it explicitly or a harness that binds a local port breaks in a
	// way that looks nothing like a firewall problem. A Kubernetes
	// NetworkPolicy never sees it, so that renderer drops these rules
	// rather than emitting a meaningless 127.0.0.0/8 peer.
	ScopeLocal
)

type Verdict

type Verdict uint8

Verdict is what the filter does with a packet.

const (
	// VerdictDrop discards the packet silently. Silence rather than a
	// rejection is deliberate: a sandbox probing for reachable networks
	// learns nothing from a timeout, whereas an ICMP unreachable confirms
	// that something is listening on the other side of the filter.
	VerdictDrop Verdict = iota
	// VerdictAllow forwards the packet.
	VerdictAllow
)

func (Verdict) String

func (v Verdict) String() string

String renders the verdict for rule comments and audit rows.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL