egressbroker

package
v0.0.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 27, 2026 License: MIT Imports: 23 Imported by: 0

Documentation

Overview

Package egressbroker leases the control plane's Internet connection to isolated executors through an authenticated forward proxy.

It closes the last gap in the grantable-resource set. pkg/secretbroker brokers GitHub repositories, PATs, and Kubernetes clusters; network egress was the missing pillar, and it is the one that matters most for isolation, because a sandbox that needs the network has until now had to be given a network — which means all of it.

The shape of the answer is a proxy rather than a firewall rule. A container or Pod runs with --network=none-style isolation and no route of its own; it reaches the outside world only by asking the control plane, which decides per connection. That inverts the usual arrangement: the sandbox has no egress capability to lose, and the allowlist lives somewhere the workload cannot edit.

Three types, mirroring pkg/secretbroker so the two read alike: 

Grant    who may make outbound connections, to which hosts, ports, and
         methods, under what byte quota, until when.
Session  a short-lived redemption of a matching grant, carrying a
         single-use proxy credential and live byte counters.
Proxy    the enforcement point: an HTTP forward proxy speaking CONNECT
         for TLS and absolute-URI GET/POST/... for plain HTTP.

What is actually enforced, honestly scoped:

  • Host, port, and method are checked before a single byte leaves.
  • The destination name is resolved exactly once, every returned address is checked, and the dial goes to the resolved literal — so a name that answers "93.184.216.34" to the policy check and "127.0.0.1" to the dial has nowhere to put the second answer. See netguard.go.
  • Loopback, RFC1918, CGNAT, link-local, multicast, unspecified addresses and the cloud metadata services (pkg/cloudmeta: 169.254.169.254, AWS's fd00:ec2::254, Alibaba Cloud's 100.100.100.200 and the rest) are refused unless an allowed CIDR explicitly covers them — and a metadata service only by its own /32 or /128, never by a prefix that merely contains it.
  • Byte quotas are enforced during the copy, so an over-quota transfer is cut mid-stream rather than noticed afterwards.
  • A session's TTL applies to open tunnels, not just to new connections.

What is not enforced: anything inside a CONNECT tunnel. Those bytes are TLS, cloop does not hold the origin's key, and no CA is installed in the sandbox — deliberately, because a proxy that could read the tunnel would be a far more valuable thing to compromise than the credential it protects. So Grant.Methods gates plain HTTP only, and the tunnel is authorised at its host:port and then accounted, not inspected.

Index

Constants

View Source 
const DefaultGrantTTL = 24 * time.Hour

DefaultGrantTTL matches pkg/secretbroker's, so `--ttl` means the same thing in `cloop secret grant` and `cloop egress grant`.

View Source 
const DefaultListenAddr = "127.0.0.1:0"

DefaultListenAddr binds loopback only.

This is the safe default and frequently the wrong one, which is worth being explicit about: a container on a bridge network cannot reach the host's loopback, so an operator running sandboxes will set an address the sandbox can route to (a podman gateway, a docker0 address) — and at that moment the proxy becomes reachable by anything else on that network too. The authentication is what holds there, not the bind address.

View Source 
const DefaultSessionTTL = 15 * time.Minute

DefaultSessionTTL bounds one redemption when neither the grant nor the broker says otherwise.

Fifteen minutes matches pkg/secretbroker's lease TTL on purpose. The two are the same kind of object — a short-lived materialisation of a longer authorisation — and an operator who has learned "credentials in a sandbox last fifteen minutes and renew" should not have to learn a second number for the network.

View Source 
const HostedStatusKind = "egress_proxy"

HostedStatusKind is the hub_owners kind under which every hub process records the egress proxy it hosts (Task 20378), one row per process, so that `cloop hub doctor` — another process, possibly on another machine — can say where each hub's proxy listens, or why it does not.

View Source 
const MaxHostLength = 253

MaxHostLength bounds a destination hostname. 253 is the DNS maximum for a fully-qualified name; anything longer cannot resolve and is more likely to be an attempt to overflow something downstream.

View Source 
const MaxHostPatterns = 256

MaxHostPatterns bounds an allowlist. The limit exists because every pattern is walked on every connection; it is generous enough that no legitimate grant reaches it.

View Source 
const MaxSessionTTLCeiling = 4 * time.Hour

MaxSessionTTLCeiling is the hard upper bound on a session, regardless of what a grant or the config asks for. A proxy credential that lives for hours is not a lease, it is a configuration.

View Source 
const ProxyAuthScheme = "Basic"

ProxyAuthScheme is the only authentication scheme the proxy accepts. Basic over a loopback- or VPN-reachable proxy carries a 256-bit random token, not a password, so there is nothing to guess and nothing reused elsewhere; the scheme is chosen because every HTTP client and every language runtime already speaks it from a proxy URL's userinfo.

View Source 
const TokenBytes = 32

TokenBytes is the size of a minted proxy token. 256 bits from crypto/rand is well past the point where guessing is the attack anyone would choose, which is what lets the token be stored as a bare SHA-256 rather than run through a password KDF: there is no dictionary to attack and no low-entropy user choice to protect.

Variables

View Source 
var (
	// ErrSessionExists: the broker already holds a session with that id.
	ErrSessionExists = errors.New("egressbroker: a session with that id is already live")
	// ErrStoreUnavailable: RestoreSession could not read the session's grant.
	// That says nothing about the grant, so the session is neither restored
	// nor ended, and the caller tries again.
	ErrStoreUnavailable = errors.New("egressbroker: the grant store could not be read")
)
View Source 
var (
	// ErrHostNotAllowed: the destination hostname is outside the grant's
	// host allowlist and its resolved address is outside every allowed CIDR.
	ErrHostNotAllowed = errors.New("egressbroker: destination host not allowed")

	// ErrPortNotAllowed: the destination port is outside the grant's port
	// allowlist.
	ErrPortNotAllowed = errors.New("egressbroker: destination port not allowed")

	// ErrMethodNotAllowed: the HTTP method is outside the grant's method
	// allowlist. Only reachable on the plain-HTTP path — see Grant.Methods
	// for why a CONNECT tunnel cannot be method-filtered.
	ErrMethodNotAllowed = errors.New("egressbroker: HTTP method not allowed")

	// ErrDestinationBlocked: the destination resolved to a loopback,
	// RFC1918, carrier-grade-NAT, link-local, multicast, or unspecified
	// address, and no allowed CIDR covers it. This is the SSRF guard, and it
	// fires after resolution, so a public name that resolves inward is
	// caught as surely as a literal 169.254.169.254.
	ErrDestinationBlocked = errors.New("egressbroker: destination address is in a blocked range")

	// ErrQuotaExceeded: the session transferred more bytes than the grant
	// permits. Raised mid-stream, which tears the transfer down rather than
	// letting it finish — a quota that only applies at the end of a
	// download is not a quota.
	ErrQuotaExceeded = errors.New("egressbroker: transfer quota exceeded")

	// ErrSessionExpired: the proxy session's TTL elapsed. Also raised
	// mid-tunnel: an open CONNECT tunnel does not outlive its lease.
	ErrSessionExpired = errors.New("egressbroker: proxy session expired")

	// ErrUnauthenticated: the Proxy-Authorization header was missing,
	// malformed, named an unknown session, or carried the wrong token.
	// One sentinel for all four on purpose — distinguishing them to the
	// caller would turn the proxy into a session-ID oracle.
	ErrUnauthenticated = errors.New("egressbroker: proxy credential missing or invalid")

	// ErrGrantNotFound: no egress grant with that ID exists.
	ErrGrantNotFound = errors.New("egressbroker: grant not found")

	// ErrGrantExpired: the grant's TTL elapsed.
	ErrGrantExpired = errors.New("egressbroker: grant expired")

	// ErrGrantRevoked: the grant was revoked.
	ErrGrantRevoked = errors.New("egressbroker: grant revoked")

	// ErrNoGrant: no active grant matches the requester, so there is
	// nothing to redeem. Distinct from ErrGrantNotFound, which is about a
	// specific ID.
	ErrNoGrant = errors.New("egressbroker: no active egress grant matches this requester")

	// ErrInvalidGrant: the grant failed structural validation and was never
	// stored. A validation failure is not a denial: it never became a
	// security decision, so it does not produce a deny audit row.
	ErrInvalidGrant = errors.New("egressbroker: invalid grant")

	// ErrInvalidRequest: the proxy request was malformed — an unparseable
	// CONNECT target, a relative URI on the plain-HTTP path, an unsupported
	// scheme.
	ErrInvalidRequest = errors.New("egressbroker: malformed proxy request")

	// ErrResolveFailed: DNS lookup for the destination failed or returned
	// no usable address.
	ErrResolveFailed = errors.New("egressbroker: destination resolution failed")

	// ErrDialFailed: the connection to the (already authorised) destination
	// could not be established.
	ErrDialFailed = errors.New("egressbroker: destination dial failed")
)

Sentinel errors. Every refusal in this package wraps exactly one of them, and the choice of sentinel is what the audit row's verdict is derived from — so "why was this request refused" is answerable by errors.Is at any call site, not by string matching on a message.

The denial set is deliberately fine-grained. "Blocked" is not a useful verdict to an operator staring at a sandbox that cannot reach GitHub: the difference between "you forgot to allow port 443", "the host is not on the list", "you have burned your download quota", and "the lease died under you" is the difference between a one-line fix and an afternoon.

View Source 
var DefaultPorts = []int{80, 443}

DefaultPorts is what a grant gets when the operator names no ports. It is written into the stored grant rather than applied at decision time, so `cloop egress list` shows the ports that will actually be honoured and there is no invisible default to be surprised by later.

View Source 
var MetadataIPv4 = netip.MustParseAddr("169.254.169.254")

MetadataIPv4 is the cloud instance metadata endpoint most clouds share. It is inside link-local and would be blocked by that rule alone; it is named separately so the denial says "metadata service" rather than "link-local", because that is the sentence an operator needs to see in an audit log.

It is one entry of pkg/cloudmeta's table, which BlockReason consults whole (Task 20397): AWS's IPv6 endpoint, Alibaba Cloud's, Azure's WireServer and the rest are refused with the same sentence, from the same table the packet filter compiles its drops from.

View Source 
var ProxyEnvKeys = []string{"HTTPS_PROXY", "HTTP_PROXY", "https_proxy", "http_proxy"}

ProxyEnvKeys names the variables Env sets that carry the session's credential. A caller placing Env into a workload declares these sensitive (redact.EnvKey), so their values are scrubbed from the workload's output and left out of anything persisted about it.

Functions

func BlockReason 

func BlockReason(addr netip.Addr) string

BlockReason classifies an address against the hard-block set and returns a human-readable reason, or "" when the address is on the public Internet.

Ordering is by specificity, not by prefix length: the reason text is what lands in the audit log and in the operator's terminal, so "cloud metadata service" beats the technically-equally-true "link-local".

func FormatBytes 

func FormatBytes(n int64) string

FormatBytes renders a byte count in the same units the CLI accepts, so a value copied out of `cloop egress list` can be pasted back into `cloop egress grant`.

func FormatProxyCredential 

func FormatProxyCredential(sessionID, token string) string

FormatProxyCredential builds the header value a client sends. Used by the `cloop egress test` client and by the round-trip tests, so the encoding has one definition on both sides.

func NormalizeDestinationHost 

func NormalizeDestinationHost(host string) (string, error)

NormalizeDestinationHost canonicalises and validates a destination hostname or address literal.

Both request parsers route through it, and that shared definition is the point rather than a convenience: if CONNECT and the plain-HTTP authority disagreed about what counts as a host, one of them would be the way past the allowlist. A fuzz run found exactly that — splitHostPort had no charset rule at all and accepted "0\r0", which would have reached both a dial target and an audit row.

func ParseBytes 

func ParseBytes(s string) (int64, error)

ParseBytes reads a byte quota: a bare integer (bytes) or a suffixed size ("64k", "10m", "2g"). Empty yields 0, meaning unlimited.

func ParseConnectTarget 

func ParseConnectTarget(target string) (host string, port int, err error)

ParseConnectTarget parses the authority-form request-target of a CONNECT request ("example.com:443", "[2001:db8::1]:443") into a host and port.

RFC 9110 requires authority-form to carry a port, and this parser enforces it rather than defaulting: a CONNECT with no port is malformed, and inventing 443 for it would mean the policy check ran against a port the client never asked for.

Everything that is not unambiguously a host and a port is rejected. In particular a userinfo prefix, a path, a query, a fragment, a wildcard, any whitespace, any control byte, and an unbracketed IPv6 literal are all errors, because each of them is a way to make one reader see a different host than another.

func ParseProxyCredential 

func ParseProxyCredential(header string) (sessionID, token string, err error)

ParseProxyCredential extracts the session ID and token from a Proxy-Authorization header value.

It returns ErrUnauthenticated for every failure shape, and never says which one. A parser that distinguished "no such session" from "wrong token" would be an oracle for enumerating live sessions.

Types

type Broker 

type Broker struct {
	// contains filtered or unexported fields
}

Broker issues and enforces egress grants.

It owns the policy and the live sessions; pkg/egressbroker's Proxy owns the wire. The split is the same one pkg/secretbroker draws between broker and store: the part worth testing exhaustively should be testable without a socket, and it is — every decision below runs against an in-memory store with no network in sight.

Safe for concurrent use.

func New 

func New(store Store, opts ...Option) (*Broker, error)

New builds a Broker over store.

func (*Broker) Authenticate 

func (b *Broker) Authenticate(sessionID, token string) (*Session, error)

Authenticate resolves a presented credential to its session.

Every failure returns the same ErrUnauthenticated, and the token comparison runs even when the session ID is unknown — against a zero hash, which no token can match. Both are there so that response timing does not reveal whether a session ID exists, which would otherwise turn the proxy into an enumeration oracle for live sessions.

func (*Broker) CheckpointSession added in v0.0.5 

func (b *Broker) CheckpointSession(id string)

CheckpointSession writes a live durable session's counters to its record when they moved since the last checkpoint. A hub calls it on its keepalive's tick, which bounds what a SIGKILL can lose to one tick of traffic.

func (*Broker) CloseAllSessions added in v0.0.5 

func (b *Broker) CloseAllSessions(reason string) int

CloseAllSessions retires every live session and reports how many there were. A hub calls it on shutdown, so each session's last row says why it ended rather than the trail simply stopping.

func (*Broker) CloseSession 

func (b *Broker) CloseSession(id, reason string)

CloseSession retires a session, closes any socket it still holds, and records its final byte counts.

Closing the sockets is what makes revocation immediate rather than TTL-bounded. That is the payoff for brokering a capability instead of handing over a token: a leaked PAT cannot be recalled from a running container, but a tunnel can be severed at the proxy, mid-transfer, by the process that owns the socket.

It is idempotent: a session torn down by the proxy and then released by the orchestrator produces one close row, not two.

func (*Broker) Endpoint 

func (b *Broker) Endpoint() string

Endpoint returns the address handed to sandboxes.

func (*Broker) ExtendSession added in v0.0.5 

func (b *Broker) ExtendSession(ctx context.Context, id string) (time.Time, error)

ExtendSession renews a live session for as long as the grant behind it still authorises it, and returns the session's deadline afterwards.

It is the egress counterpart of secretbroker.Broker.Extend, and a hub calls it for the same reason: a run outlives one session period, and a session that lapsed under it would cut the run off the network part-way through. The grant is re-read from the store on every call, so a revocation or an expiry made anywhere — the dashboard, the CLI, another hub member — is refused here even though the session itself was issued before it. The deadline moves to min(now + the session TTL, the grant's own expiry), never past either, and never backwards; a call that cannot move it returns the current deadline and writes no row.

The policy the session enforces is not refreshed. It is the snapshot taken at redemption, which is what keeps a request from being evaluated against a half-updated grant; grants are immutable but for revocation, which this refuses, so nothing is lost by it.

func (*Broker) Grant 

func (b *Broker) Grant(ctx context.Context, req GrantRequest) (Grant, error)

Grant authorises a subject to reach the network through the proxy.

func (*Broker) ListGrants 

func (b *Broker) ListGrants(f GrantFilter) ([]Grant, error)

ListGrants returns grants matching the filter, newest first.

func (*Broker) ReapExpired 

func (b *Broker) ReapExpired() int

ReapExpired retires sessions whose TTL has elapsed and returns how many. The proxy calls it on a ticker so that an idle expired session does not linger in memory until someone happens to present its credential.

func (*Broker) RecheckSession added in v0.0.5 

func (b *Broker) RecheckSession(id string) error

RecheckSession reports whether the grant behind a live session still authorises it, reading the grant from the store. It writes no row: a hub asks it every minute of a run, and the answer that matters — a revoked or expired grant — is recorded by the CloseSession the caller makes on it.

It is how a revocation the hub's own broker did not perform reaches a live session promptly: `cloop egress revoke` in another process, or a revoke on another hub member, stamps the shared store and nothing else.

func (*Broker) Redeem 

func (b *Broker) Redeem(ctx context.Context, req RedeemRequest) (*Redemption, error)

Redeem mints a single-use proxy credential for the first active grant that matches the requester.

"Single-use" is about minting, not about requests: the token authorises one session, which may carry many connections, and cannot be presented to mint a second session. The plaintext is returned here and nowhere else — the broker keeps only its SHA-256, so a memory dump of the control plane taken after this call does not yield a usable credential.

When several grants match, the first by the store's newest-first ordering wins rather than the union of all of them. Unioning would let two narrow grants silently compose into a wide one, and "why can this sandbox reach that host" would stop having a single answer.

func (*Broker) RelabelSession added in v0.0.7 

func (b *Broker) RelabelSession(id string, labels map[string]string) (bool, error)

RelabelSession judges the live session id as its executor carrying labels from now on, and reports whether that changed anything. It closes nothing: the caller asks RecheckSession next, and ends a session whose grant the new labels no longer match.

func (*Broker) RestoreSession added in v0.0.5 

func (b *Broker) RestoreSession(ctx context.Context, req RestoreRequest) (*Session, error)

RestoreSession brings back a session another hub process redeemed, under the same id, token hash and deadline, enforcing the grant snapshot it was issued with and counting on from the counters its record holds.

The grant is re-read first, exactly as a renewal re-reads it: one revoked or expired while no process held the session, or no longer issued to its requester, refuses the restore with a denied egress.restore row, and the session is never served again. A grant that could not be read refuses nothing: ErrStoreUnavailable, and the caller tries again. The deadline is clamped to what a renewal would set now — the grant's expiry, the session ceiling from now — never moved later; a session already past it is refused as expired.

func (*Broker) Revoke 

func (b *Broker) Revoke(ctx context.Context, grantID, actor string) error

Revoke marks a grant unusable and closes every live session it authorised.

Closing the sessions is the difference between this and secretbroker.Revoke, and it is available here precisely because the capability is brokered rather than handed over: a leaked PAT cannot be recalled from a running container, but a proxy session can be torn down at the proxy, mid-tunnel, from the control plane. Revocation is therefore immediate rather than bounded by the session TTL.

func (*Broker) Session 

func (b *Broker) Session(id string) *Session

Session returns a live session by ID, or nil.

func (*Broker) SessionRequester added in v0.0.7 

func (b *Broker) SessionRequester(id string) (r secretbroker.Requester, live bool)

SessionRequester returns who the live session id is judged as, and whether this broker serves it.

func (*Broker) Sessions 

func (b *Broker) Sessions() []*Session

Sessions returns a snapshot of the live sessions, for the CLI and UI.

func (*Broker) SetEndpoint 

func (b *Broker) SetEndpoint(addr string)

SetEndpoint updates the advertised address. The proxy calls it after binding, so an operator who configured port 0 still gets a usable HTTPS_PROXY value rather than one naming a port nothing listens on.

func (*Broker) SuspendSession added in v0.0.5 

func (b *Broker) SuspendSession(id, reason string) bool

SuspendSession stops serving a session without ending it: it leaves the broker and its sockets close — the process is going away, or another one serves the session now — but no close row is written and its record stays open, with its counters checkpointed, for the hub process that adopts its run to restore. It reports whether the session was here.

type Destination 

type Destination struct {
	// Host is the requested name (or address literal), lowercased.
	Host string
	Port int
	// Addr is the resolved address the dial will target. Every other
	// address the name resolved to was checked too — see Resolve.
	Addr netip.Addr
	// Resolved is the full answer set, for the audit trail.
	Resolved []netip.Addr
	// Literal reports whether Host was already an address, so no lookup
	// happened and there is nothing to rebind.
	Literal bool
}

Destination is an authorised, pinned destination: the name the client asked for, and the single address that will be dialled for it.

func Resolve 

func Resolve(ctx context.Context, r Resolver, g Grant, host string, port int) (Destination, error)

Resolve turns a requested host:port into a pinned, authorised Destination, or returns the typed reason it may not be reached.

The rules, in order:

  1. The port must be allowed. Checked first because it is free and needs no network traffic — a grant that permits only 443 should not generate a DNS query for a request to port 22.
  2. The name must be on the host allowlist, unless the grant is CIDR-only. Also free, also before any lookup.
  3. The name is resolved exactly once.
  4. *Every* returned address must pass CheckAddr. Not just the one that will be dialled: a name that answers with a mix of public and private addresses is either misconfigured or hostile, and picking the public one would let the sandbox retry until the resolver's rotation handed it the other. Failing the whole name is the only reading with no race in it.
  5. The first surviving address is pinned, and the caller dials that literal.

func (Destination) AddrPort 

func (d Destination) AddrPort() string

AddrPort renders the pinned destination for net.Dial.

func (Destination) String 

func (d Destination) String() string

String renders "host:port" for logs and audit rows.

type DialFunc 

type DialFunc func(ctx context.Context, network, addr string) (net.Conn, error)

DialFunc is the connection factory the proxy uses. Tests replace it to avoid touching the real network; production leaves it nil and gets a net.Dialer.

type Grant 

type Grant struct {
	ID string `json:"id"`
	// Scope is a free-form operator label ("ci", "deps"). It carries no
	// authorisation weight, so editing it cannot widen the grant.
	Scope   string               `json:"scope,omitempty"`
	Subject secretbroker.Subject `json:"subject"`

	// Hosts is the destination allowlist, in pkg/secretbroker's host syntax:
	// an exact name, "*.example.com" for subdomains only (not the apex), or
	// "*" for everything. Matching is delegated to
	// secretbroker.Constraints.CheckHost so that "which hosts does this
	// pattern cover" has exactly one definition in the codebase.
	Hosts []string `json:"hosts,omitempty"`

	// CIDRs is a second, address-level allow dimension, and the *only* way
	// to reach a private address.
	//
	// A destination is authorised when its hostname matches Hosts or its
	// resolved address falls inside one of these prefixes. Listing a private
	// prefix here is what waives the SSRF block for that prefix and nothing
	// else — which is why there is no blanket allow_private flag. "Let this
	// sandbox reach the metadata service" should be a sentence an operator
	// has to write out as 169.254.169.254/32, not a checkbox, so a prefix
	// that merely contains a metadata service is refused unless the list
	// names the service too (Task 20397).
	CIDRs []string `json:"cidrs,omitempty"`

	// Ports is the destination port allowlist. Never empty after Normalize.
	Ports []int `json:"ports,omitempty"`

	// Methods gates plain-HTTP requests. "*" allows every method.
	//
	// It does not, and cannot, gate a CONNECT tunnel: the method lives
	// inside the TLS record layer. Normalize writes "*" rather than leaving
	// the field empty so that a listing never implies a restriction that is
	// not there.
	Methods []string `json:"methods,omitempty"`

	// MaxBytesUp and MaxBytesDown cap one session's transfer, in bytes,
	// counted from the sandbox's point of view: "up" is what it sends. Zero
	// means unlimited.
	MaxBytesUp   int64 `json:"max_bytes_up,omitempty"`
	MaxBytesDown int64 `json:"max_bytes_down,omitempty"`

	// SessionTTL bounds one redemption. Zero means the broker's default.
	// The split from ExpiresAt is the same one pkg/secretbroker draws
	// between grant and lease: a day-long authorisation does not imply a
	// day-long credential in a sandbox.
	SessionTTL time.Duration `json:"session_ttl,omitempty"`

	ExpiresAt time.Time `json:"expires_at,omitempty"`
	CreatedAt time.Time `json:"created_at"`
	CreatedBy string    `json:"created_by,omitempty"`
	RevokedAt time.Time `json:"revoked_at,omitempty"`
}

Grant authorises one subject to make outbound connections through the proxy, under constraints, until ExpiresAt.

Subject reuses pkg/secretbroker's: an egress grant is aimed at a project, an executor, or a labelled fleet by exactly the same rules as a credential grant, and an operator should not have to learn a second targeting syntax for the second resource.

func (Grant) Active 

func (g Grant) Active(now time.Time) bool

Active reports whether the grant may be redeemed at now.

func (Grant) AllowsAddr 

func (g Grant) AllowsAddr(addr netip.Addr) bool

AllowsAddr reports whether an address is inside one of the grant's CIDRs.

This is both the second allow dimension and the SSRF opt-in: netguard's blocked-range check consults it, so an operator who lists 10.0.0.0/8 gets exactly 10.0.0.0/8 and not the loopback, the metadata service, or their IPv6 equivalents.

A cloud metadata service is the one address a containing prefix does not cover: only its own /32 or /128 does (Task 20397). Validate refuses a grant whose CIDRs merely contain one, so for a grant made since, this changes nothing; for one stored before the rule — 169.254.0.0/16 granted for some link-local device — it is what keeps the metadata service closed anyway, the same way the packet filter drops it ahead of the allow.

func (Grant) CheckAddr 

func (g Grant) CheckAddr(addr netip.Addr, hostAllowed bool) error

CheckAddr is the address gate: it applies the hard-block set, then the grant's own dimensions.

The order is deliberate. The block set is consulted first and can only be waived by an explicit CIDR in the grant — never by the host allowlist. That is what stops "--hosts '*'" from being a route to the metadata service: wildcards buy reach on the public Internet, and nothing else.

hostAllowed carries the result of the name-level check so the two allow dimensions compose. A destination is authorised when its name matched, or when its address is inside an allowed CIDR.

func (Grant) CheckHost 

func (g Grant) CheckHost(host string) error

CheckHost reports whether the destination hostname is on the allowlist.

A grant with only CIDRs has no host opinion; the address check in CheckAddr is then the gate, and this returns nil so the two dimensions compose as an OR rather than an AND. A grant that lists hosts and a destination that matches none of them is refused here, before any DNS traffic is generated.

func (Grant) CheckMethod 

func (g Grant) CheckMethod(method string) error

CheckMethod reports whether a plain-HTTP method is allowed.

func (Grant) CheckPort 

func (g Grant) CheckPort(port int) error

CheckPort reports whether the destination port is allowed. An empty list denies: Normalize guarantees a stored grant always names its ports, so an empty list here means the grant was hand-built and skipped validation.

func (Grant) DenyReason 

func (g Grant) DenyReason(now time.Time) string

DenyReason returns the audit-friendly reason this grant is unusable at now, or "" when it is usable. Mirrors secretbroker.Grant.DenyReason so a denial reads the same in both audit trails.

func (Grant) DenySentinel 

func (g Grant) DenySentinel() error

DenySentinel returns the error matching DenyReason, so a caller can attach the right typed error to an inactive grant without re-deriving why.

func (Grant) HostMatches 

func (g Grant) HostMatches(host string) bool

HostMatches is CheckHost as a boolean, for the OR with the CIDR dimension.

func (*Grant) Normalize 

func (g *Grant) Normalize()

Normalize canonicalises a grant and fills in the defaults that must be visible in storage rather than applied silently at decision time.

It is called by Validate, so anything that reaches the store has already been through it.

func (Grant) SessionDeadline 

func (g Grant) SessionDeadline(now time.Time, ceiling time.Duration) time.Time

SessionDeadline returns when a session redeemed at now must expire, given the broker's ceiling. A session never outlives the grant that authorised it, which is what makes revocation effective within one session period.

func (Grant) Summary 

func (g Grant) Summary() string

Summary renders the grant's policy as a short, stable, audit-safe string. It contains allowlist patterns and limits only — an egress grant holds no credential material at all, which is one of the nicer properties of brokering a capability rather than a token.

func (*Grant) Validate 

func (g *Grant) Validate() error

Validate normalises the grant and checks its invariants.

A grant with neither hosts nor CIDRs is rejected rather than treated as "allow nothing" or "allow everything": both readings are defensible, which is precisely why the operator has to say. Pass --hosts '*' to mean it.

type GrantFilter 

type GrantFilter struct {
	// Subject keeps only grants whose subject renders to this string.
	Subject string
	// ActiveOnly drops expired and revoked grants.
	ActiveOnly bool
}

GrantFilter narrows ListGrants.

type GrantRequest 

type GrantRequest struct {
	Subject      secretbroker.Subject
	Scope        string
	Hosts        []string
	CIDRs        []string
	Ports        []int
	Methods      []string
	MaxBytesUp   int64
	MaxBytesDown int64
	SessionTTL   time.Duration
	// TTL is the grant's own lifetime. Zero means DefaultGrantTTL.
	TTL   time.Duration
	Actor string
}

GrantRequest describes an egress grant to create.

type HostedStatus added in v0.0.5 

type HostedStatus struct {
	// Enabled is executors.egress.enabled as the hub read it.
	Enabled bool `json:"enabled"`
	// Listening is the address the proxy is bound to.
	Listening string `json:"listening,omitempty"`
	// Advertised is the address sandboxes are pointed at when no route of
	// their own applies: executors.egress.advertise_addr with the bound port,
	// or the bound address.
	Advertised string `json:"advertised,omitempty"`
	// Error says why an enabled proxy is not running — a listen address that
	// would not bind, a database that would not open.
	Error string `json:"error,omitempty"`
	// HubPort, Hostname, PID and BootID identify the hub process, so a reader
	// can tell a live hub's row from one a crashed hub left behind.
	HubPort   int       `json:"hub_port,omitempty"`
	Hostname  string    `json:"hostname,omitempty"`
	PID       int       `json:"pid,omitempty"`
	BootID    string    `json:"boot_id,omitempty"`
	UpdatedAt time.Time `json:"updated_at"`
}

HostedStatus is that row's meta: what one hub process did with executors.egress.

type MemStore 

type MemStore struct {
	// contains filtered or unexported fields
}

MemStore is an in-memory Store, used by tests and by `cloop egress test` when it needs a throwaway policy.

It is exported rather than test-only because the test command genuinely needs it: constructing a proxy for a one-shot connectivity check should not require writing a grant into the project database.

func NewMemStore 

func NewMemStore() *MemStore

NewMemStore returns an empty in-memory store.

func (*MemStore) GetGrant 

func (m *MemStore) GetGrant(id string) (Grant, error)

GetGrant returns one grant by ID.

func (*MemStore) ListGrants 

func (m *MemStore) ListGrants() ([]Grant, error)

ListGrants returns every grant, newest first.

func (*MemStore) PutGrant 

func (m *MemStore) PutGrant(g Grant) error

PutGrant stores a copy of g.

func (*MemStore) RevokeGrant 

func (m *MemStore) RevokeGrant(id string, at time.Time) error

RevokeGrant stamps a grant revoked.

type Option 

type Option func(*Broker)

Option configures a Broker.

func WithAuditor 

func WithAuditor(a secretbroker.Auditor) Option

WithAuditor sets the audit sink. Without one, decisions are enforced but not recorded — acceptable for tests, never for a control plane, which is why cmd wires secretstore.NewAuditor unconditionally.

func WithClock 

func WithClock(fn func() time.Time) Option

WithClock overrides the time source, so TTL expiry can be exercised without sleeping.

func WithDefaultQuotas added in v0.0.5 

func WithDefaultQuotas(up, down int64) Option

WithDefaultQuotas sets the per-session byte quotas applied to a session whose grant names none — executors.egress.default_max_bytes_up/down on a hub.

Applied at redemption rather than written into grants, so a grant made before the default was configured, or through a path that did not apply it, is bounded all the same. A grant's own quota always wins, and a default can only bound a session that would otherwise be unlimited: it never widens one.

func WithEndpoint 

func WithEndpoint(addr string) Option

WithEndpoint sets the address sandboxes should use to reach the proxy ("127.0.0.1:8899", "host.containers.internal:8899"). It is what goes into HTTPS_PROXY, so it must be routable *from the sandbox*, which is not always the same as the address the proxy listens on.

func WithMaxSessionTTL 

func WithMaxSessionTTL(d time.Duration) Option

WithMaxSessionTTL sets the ceiling applied to every redemption. Values above MaxSessionTTLCeiling are clamped down to it: a config typo must not be able to mint an all-day credential.

func WithSessionStore added in v0.0.5 

func WithSessionStore(store SessionStore) Option

WithSessionStore records every Durable session in store (Task 20383).

type Options 

type Options struct {
	// ListenAddr is the bind address. Empty means DefaultListenAddr.
	ListenAddr string
	// Endpoint is the address sandboxes should use to reach this proxy. It
	// is what goes into HTTPS_PROXY and is frequently *not* ListenAddr: a
	// container reaches the host by a gateway address, not by the address
	// the host bound. Empty falls back to the listener's own address.
	Endpoint string
	// DialTimeout bounds the connection to an authorised origin.
	DialTimeout time.Duration
	// Resolver overrides DNS. Tests use it to simulate rebinding.
	Resolver Resolver
	// Dial overrides the outbound dialer. Tests use it to avoid the network;
	// production leaves it nil.
	Dial DialFunc
	// AwaitSession, when set, is asked about a credential naming a session
	// the broker does not hold, before it is refused (Task 20383): a hub
	// waits there, bounded, for the process adopting the session's run to
	// restore it. It reports whether the broker holds the session now; the
	// credential is then authenticated as usual. A credential naming a session
	// the broker does hold — right token or wrong — is never offered.
	AwaitSession func(ctx context.Context, sessionID, token string) bool
}

Options configure a Proxy.

type Proxy 

type Proxy struct {
	// contains filtered or unexported fields
}

Proxy is the forward proxy. Build it with NewProxy and run it with Serve or ListenAndServe.

func NewProxy 

func NewProxy(b *Broker, opts Options) (*Proxy, error)

NewProxy builds a proxy over a broker.

func (*Proxy) Addr 

func (p *Proxy) Addr() string

Addr returns the bound address, or "" before Serve.

func (*Proxy) Close 

func (p *Proxy) Close() error

Close stops the proxy. It is idempotent and safe to call from a defer alongside an explicit call.

Shutdown is deliberately not graceful in the http.Server.Shutdown sense: an open CONNECT tunnel has no request boundary to wait for, so Shutdown would block until the session TTL expired. Closing outright is also the correct security behaviour — when the control plane stops, so does every sandbox's route to the Internet.

Making that true takes two steps, not one. http.Server.Close closes the connections the server still tracks, but a hijacked connection is deliberately not among them, so every live CONNECT tunnel survives it; they are closed explicitly from the registry. Skipping that step does not merely leak a tunnel — it also wedges the drain below, because the handler owning the tunnel never returns.

The drain then waits for the handlers to *finish reporting*. Each one still has an audit row to write, and http.Server.Close does not wait for handler goroutines. Without it those rows land after the caller has closed the database, which is how a decision ends up enforced but unrecorded — the one outcome this package exists to make impossible. The wait is bounded, so a wedged handler delays shutdown rather than preventing it.

func (*Proxy) ListenAndServe 

func (p *Proxy) ListenAndServe() error

ListenAndServe binds ListenAddr and serves until Close.

It publishes the bound address to the broker before serving, so a configuration of ":0" still produces a usable HTTPS_PROXY value — which matters more than it sounds, because port 0 is the right choice for an ephemeral per-run proxy and the wrong one to have to know in advance.

func (*Proxy) Serve 

func (p *Proxy) Serve(ln net.Listener) error

Serve serves on an existing listener and returns when it is closed.

func (*Proxy) ServeHTTP 

func (p *Proxy) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP authenticates the caller and dispatches to the tunnel or plain-HTTP path.

type RedeemRequest 

type RedeemRequest struct {
	Requester secretbroker.Requester
	// TaskID is the unit of work, carried into every audit row this session
	// produces.
	TaskID string
	Actor  string
	// GrantID pins the redemption to one grant. Empty picks the widest
	// active match — see Redeem.
	GrantID string
	// RunID names the execution the session is for, recorded on every row the
	// session produces. Never matched against, like Requester.RunID.
	RunID string
	// Endpoint overrides the broker's advertised address for this session's
	// proxy URL. A hub serves executors of several kinds, and each reaches the
	// proxy by a different route — a container by its bridge gateway, a Pod by
	// a Service, a device by an address on the link between them — so the
	// address that goes into HTTPS_PROXY is chosen per redemption. Empty uses
	// the broker's.
	Endpoint string
	// NoProxy lists hosts the workload reaches directly; see
	// Redemption.NoProxy.
	NoProxy []string
	// Durable records the session in the broker's SessionStore (Task 20383),
	// so the hub process that adopts its run after this one stops can restore
	// it with its counters. A hub sets it for every run's session.
	Durable bool
}

RedeemRequest identifies who wants a proxy credential.

type Redemption 

type Redemption struct {
	Session *Session
	// Token is the plaintext proxy token. Single-use in the sense that
	// matters: it is minted for exactly one session and cannot be presented
	// to mint another.
	Token string
	// ProxyURL is the full "http://id:token@host:port" the sandbox uses. It
	// carries the credential, so it is never audited or logged.
	ProxyURL string
	// NoProxy lists further hosts the workload must reach directly rather
	// than through the proxy, beyond loopback — the hub's own endpoints, such
	// as its git proxy, which a grant does not name and should not have to.
	NoProxy []string
}

Redemption is what a caller gets back from Broker.Redeem: the session, and the one and only copy of its token.

The token is a separate field rather than a Session field so that a Session can be marshalled, logged, or returned over an API without any possibility of carrying the credential with it — the same property pkg/secretbroker gets by keeping Secret.Sealed json:"-".

func (Redemption) Env 

func (r Redemption) Env() map[string]string

Env returns the environment a workload needs to route through the proxy.

Both cases of each variable are set because the ecosystem never agreed: curl and most Unix tooling read the lowercase forms, Go's http.ProxyFromEnvironment and most language runtimes read either, and a handful of Windows-origin tools read only the uppercase. Setting one and not the other is how a sandbox ends up with a proxy that half its tools ignore — and a tool that ignores the proxy in a --network=none sandbox does not leak, it simply fails, which is a confusing way to discover the mistake.

NO_PROXY keeps loopback traffic local. Without it a workload that talks to its own sidecar on 127.0.0.1 would send that request to the proxy, which would refuse it as a blocked destination — correct, but for a request that never needed brokering. NoProxy extends it the same way for the hub's own endpoints.

CLOOP_EGRESS_EXPIRES is the deadline the session was issued with. A hub renews a session for as long as its run lives, so this is when the session would end unrenewed, not a promise that it will.

func (Redemption) EnvLines 

func (r Redemption) EnvLines() []string

EnvLines renders Env as sorted "K=V" strings, ready to append to an executor.Spec's Env.

type Resolver 

type Resolver interface {
	LookupNetIP(ctx context.Context, network, host string) ([]netip.Addr, error)
}

Resolver is the DNS interface the proxy depends on. Narrowing it to the one method used lets a test substitute a resolver that answers differently on each call — which is exactly the rebinding attack, and exactly what the resolve-once discipline is meant to survive.

type RestoreRequest added in v0.0.5 

type RestoreRequest struct {
	// Record is the session as its store recorded it.
	Record SessionRecord
	// From names the hub process the session was taken over from.
	From string
}

RestoreRequest is a session to bring back.

type Session 

type Session struct {
	ID string `json:"id"`
	// GrantID names the authority. The policy itself is snapshotted into
	// Grant below rather than re-read per request, so a request cannot be
	// evaluated against a half-updated grant; revocation lands at the next
	// redemption, bounded by the session TTL.
	GrantID string `json:"grant_id"`
	Grant   Grant  `json:"-"`

	ExecutorID string `json:"executor_id,omitempty"`
	ProjectID  string `json:"project_id,omitempty"`
	// TaskID ties every audit row to the unit of work that caused it, which
	// is what makes "what did task 20163 talk to" an answerable question.
	TaskID string `json:"task_id,omitempty"`
	// RunID names the execution the session was redeemed for (Task 20378),
	// the id the hub's dispatch rows carry, so a run's egress joins the rest
	// of its trail.
	RunID string `json:"run_id,omitempty"`
	Actor string `json:"actor,omitempty"`

	IssuedAt time.Time `json:"issued_at"`
	// contains filtered or unexported fields
}

Session is one redemption of a grant: a live proxy credential, the policy snapshot it was issued under, and the byte counters that bound it.

A session is served from the memory of the process whose proxy redeemed it, and nothing about it that authenticates is ever written down: the token exists only in the sandbox, and this process keeps its SHA-256. What may be written down, since Task 20383, is everything else — on a broker with a SessionStore, a Durable session's id, token hash, grant snapshot, deadline and byte counters are recorded when it is redeemed, kept in step as it is renewed, checkpointed as it moves bytes, and closed when it ends (durable.go). That is what lets the process adopting a run after a restart restore the run's session under the same credential, and it is why a restored session is not a meterless credential: it resumes counting from the checkpointed totals, against the same quota, and its grant is re-read before it is served again. A session whose grant was revoked or expired meanwhile is never restored. What a SIGKILL can lose is the traffic since the last checkpoint, not the quota.

Safe for concurrent use — a single session is shared by every connection the sandbox opens.

func (*Session) BindContext 

func (s *Session) BindContext(parent context.Context) (context.Context, context.CancelFunc)

BindContext returns a context bounded by the session: it is cancelled at the session's TTL, and immediately if the session is revoked.

The plain-HTTP path needs this because it has no socket of its own to put a deadline on — the response body is streamed by an http.Transport, and the only handle on it is the request context. Without it a workload could hold an endless chunked response open long past its lease.

The deadline is re-read when it is reached rather than fixed at the start, because a renewal can move it while a response is streaming; only a deadline that is still the session's when it arrives ends the exchange.

func (*Session) BytesDown 

func (s *Session) BytesDown() int64

func (*Session) BytesUp 

func (s *Session) BytesUp() int64

BytesUp and BytesDown report the running totals.

func (*Session) CloseReason added in v0.0.5 

func (s *Session) CloseReason() string

CloseReason says why the session ended, or "" while it is live.

func (*Session) Closed 

func (s *Session) Closed() bool

Closed reports whether the session has been retired.

func (*Session) Counters added in v0.0.5 

func (s *Session) Counters() SessionCounters

Counters snapshots what the session has moved.

func (*Session) Done 

func (s *Session) Done() <-chan struct{}

Done returns a channel closed when the session ends, by expiry or by revocation.

func (*Session) Durable added in v0.0.5 

func (s *Session) Durable() bool

Durable reports whether a SessionStore holds a record of the session.

func (*Session) Expired 

func (s *Session) Expired(now time.Time) bool

Expired reports whether the session's TTL has elapsed at now.

func (*Session) ExpiresAt 

func (s *Session) ExpiresAt() time.Time

ExpiresAt returns the session's current expiry: the deadline it was issued with, or a later one ExtendSession moved it to.

func (*Session) MarshalJSON added in v0.0.5 

func (s *Session) MarshalJSON() ([]byte, error)

MarshalJSON renders the session with its current expiry, which is not a field because it moves.

func (*Session) OpenTunnels added in v0.0.5 

func (s *Session) OpenTunnels() int

OpenTunnels reports how many CONNECT tunnels the session is carrying now.

func (*Session) Requests 

func (s *Session) Requests() int64

Requests reports how many proxy requests this session has made.

func (*Session) TTL 

func (s *Session) TTL(now time.Time) time.Duration

TTL returns the remaining lifetime at now, clamped at zero.

func (*Session) TunnelsCut added in v0.0.5 

func (s *Session) TunnelsCut() int

TunnelsCut reports how many open tunnels the session's end severed — the tunnels a revocation or a run's end took down mid-transfer. Zero while the session is live.

type SessionCounters added in v0.0.5 

type SessionCounters struct {
	BytesUp   int64 `json:"bytes_up"`
	BytesDown int64 `json:"bytes_down"`
	Requests  int64 `json:"requests"`
}

SessionCounters is what a session has moved.

type SessionLabelStore added in v0.0.7 

type SessionLabelStore interface {
	RelabelSession(id string, labels map[string]string) error
}

SessionLabelStore is an optional SessionStore extension that rewrites the labels a durable session's record holds (Task 20432). A store without it keeps the labels the session was redeemed with, and a process restoring the session relabels it again when it finds the record and the hub disagree.

type SessionRecord added in v0.0.5 

type SessionRecord struct {
	ID string
	// TokenSHA256 is the hex SHA-256 of the proxy token.
	TokenSHA256 string
	GrantID     string
	// Grant is the policy snapshot the session enforces, quotas included.
	Grant      Grant
	ExecutorID string
	ProjectID  string
	TaskID     string
	RunID      string
	Actor      string
	// Labels are the requester's, so a restore asks whether the grant is still
	// issued to the same party.
	Labels    map[string]string
	IssuedAt  time.Time
	ExpiresAt time.Time
	Counters  SessionCounters
}

SessionRecord is what a SessionStore keeps of a session: its identity, the SHA-256 of its token, the grant snapshot it enforces and the requester it was issued to — never the token.

type SessionStore added in v0.0.5 

type SessionStore interface {
	SaveSession(rec SessionRecord) error
	ExtendSession(id string, expiresAt time.Time) error
	CheckpointSession(id string, c SessionCounters) error
	CloseSession(id, reason string, at time.Time, c SessionCounters) error
}

SessionStore persists the record of every Durable session: its redemption, its renewals, its counters and its end. Errors are the implementation's to report — a session whose record could not be written still serves, and is noted on its redemption row.

type Store 

type Store interface {
	// PutGrant inserts or replaces a grant.
	PutGrant(g Grant) error
	// GetGrant returns one grant by ID, or an error wrapping
	// ErrGrantNotFound.
	GetGrant(id string) (Grant, error)
	// ListGrants returns every grant, including expired and revoked ones —
	// filtering is the broker's job, and an audit reader needs the history.
	ListGrants() ([]Grant, error)
	// RevokeGrant stamps a grant revoked at the given time. Revocation is a
	// stamp rather than a delete so "who could reach the Internet in March"
	// stays answerable.
	RevokeGrant(id string, at time.Time) error
}

Store persists egress grants.

Only grants. A session's record, where a hub keeps one, goes through a SessionStore instead (durable.go): it holds no credential, only the token's hash, and a restore re-reads the grant from here before serving the session again, so a revocation made while no process held it still lands.

Implementations must be safe for concurrent use.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.