tunnel

package
v0.23.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 12, 2026 License: AGPL-3.0 Imports: 67 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	ApplicationCodeOK            quic.ApplicationErrorCode = quic.ApplicationErrorCode(quic.NoError)
	ApplicationCodeInternalError quic.ApplicationErrorCode = quic.ApplicationErrorCode(quic.InternalError)
)
View Source 
const (
	// QueryParamConnAttempt is the /connect query-string key carrying the
	// 0-based dial attempt number. Its presence signals that the client
	// understands same-server rejection.
	QueryParamConnAttempt = "conn_attempt"
	// QueryParamConnFinal, when set to "1", marks the client's final dial
	// attempt: the server must accept it even if it already holds a
	// connection from this agent process. The client owns "this is my last
	// try" — deriving it server-side from a shared constant would lock old
	// agents out under version skew.
	QueryParamConnFinal = "conn_final"
	// QueryParamReplacesConnID names a dead connection this dial replaces.
	// The server excludes it from the diversity check (an agent must not be
	// rejected against its own corpse while the server waits out the QUIC
	// idle timeout) and evicts it immediately.
	QueryParamReplacesConnID = "replaces_conn"
	// HeaderRejectReason is the response header carrying a machine-readable
	// reason for a rejected /connect request.
	HeaderRejectReason = "X-Apoxy-Reject-Reason"
	// RejectReasonAgentConnExists indicates the server already holds a live
	// connection from this agent process for this tunnel.
	RejectReasonAgentConnExists = "agent-conn-exists"
	// MaxSameServerDialAttempts is the total number of dials the client makes
	// before marking an attempt final and settling for a server that already
	// has one of its connections. Client-side knob only — the server keys
	// acceptance on QueryParamConnFinal, never on this constant.
	MaxSameServerDialAttempts = 5
)

Same-server connection diversity. An agent process that dials a server already holding one of its connections is rejected with 409 Conflict and HeaderRejectReason set to RejectReasonAgentConnExists so it re-dials from a fresh UDP 4-tuple, giving the load balancer a chance to pick a different backend. Without this, min-conns > 1 buys no drain resilience: all of an agent's connections can hash onto one replica, and that replica's drain zeroes the agent's entire endpoint set. The client marks its last re-dial with QueryParamConnFinal, which the server honors unconditionally, so agents still connect when every dial lands on the same server (e.g. a single-replica region) and no shared retry-cap constant has to agree across independently deployed client and server binaries. Old clients never send the attempt param and are never rejected.

View Source 
const LabelKeyVersion = "apoxy.dev/version"

LabelKeyVersion is the agent label that carries the CLI build version. Always sent by Dial so AgentStatus.Labels can surface the agent's version.

Variables

View Source 
var (
	ErrNotConnected = errors.New("not connected")
)

Functions

func BuildClientRouter 

func BuildClientRouter(opts ...TunnelClientOption) (router.Router, error)

BuildClientRouter builds a router for the client tunnel side using provided options and sane defaults.

func SetupWithManager 

func SetupWithManager(mgr ctrl.Manager, srv *TunnelServer) error

SetupWithManager sets up the TunnelServer as a reconciler with the manager.

Types

type ClientGetter 

type ClientGetter interface {
	// GetClient returns a client for the given tunnel UUID.
	GetClient(ctx context.Context, tunUID uuid.UUID) (client.Client, error)
}

ClientGetter provides access to Kubernetes clients. In single-cluster mode, the tunnel UUID is ignored and the default client is returned. In multi-cluster mode, the tunnel UUID is used to look up which cluster's client to use.

type Conn 

type Conn struct {
	UUID uuid.UUID
	// contains filtered or unexported fields
}

func (*Conn) BFDState 

func (c *Conn) BFDState() bfdl.State

BFDState returns the current BFD session state for this connection.

func (*Conn) Close 

func (c *Conn) Close() error

func (*Conn) Context 

func (c *Conn) Context() context.Context

Context returns the context of the underlying connection that is canceled when the connection is closed.

func (*Conn) LastAlive 

func (c *Conn) LastAlive() time.Time

LastAlive returns when the last valid BFD packet was received from the server.

func (*Conn) LocalAddrs 

func (c *Conn) LocalAddrs() ([]netip.Prefix, error)

type ConnStats added in v0.23.0 

type ConnStats struct {
	// ID is the connection ID, which is also the name of the Tunnel object the
	// relay creates for it.
	ID string
	// Instance tells apart the connections that share an ID. The ID comes from
	// the connection 4-tuple, so an agent that reconnects from the same address
	// gets the ID of the connection it replaced. Every connection the relay
	// accepts gets its own instance number, so two ConnStats with the same ID
	// and a different Instance report different connections, and the counters
	// of one must not be compared with the counters of the other.
	Instance uint64
	// Network is the VPCNetwork name the connection is bound to.
	Network string
	// ProjectID is the tenant scope the connection's credential resolved to.
	// Empty on single-tenant relays.
	ProjectID string
	// AgentInstance is the per-process ID of the agent that opened the
	// connection, if it declared one.
	AgentInstance string

	RXBytes   uint64
	TXBytes   uint64
	RXPackets uint64
	TXPackets uint64
	RXDrops   uint64
	TXDrops   uint64

	// RXKeepAlives is the number of keep-alive frames received. A keep-alive
	// carries no payload, so it is counted here and not in RXPackets or
	// RXBytes. It still proves the peer is alive.
	RXKeepAlives uint64
	// TXKeepAlives is the number of keep-alive frames sent. A keep-alive
	// carries no payload, so it is counted here and not in TXPackets or
	// TXBytes.
	TXKeepAlives uint64
}

ConnStats is one connection's identity and its datapath counters.

The counters are cumulative over the life of the connection's current virtual network: they start at zero when the network is installed and go away with it, so a reader must take a decrease as a reset rather than as negative traffic. RXDrops and TXDrops fold every drop reason the datapath counts separately. Keep-alive frames carry no payload and are counted apart from the packet and byte counters, so an idle connection that only exchanges keep-alives shows no packets and no bytes.

type OnConnectFunc 

type OnConnectFunc func(ctx context.Context, connID string, tn *corev1alpha.TunnelNode)

OnConnectFunc is called when a tunnel connection is established. The connID is a UUID identifying the connection, and tn is the TunnelNode object.

type OnDisconnectFunc 

type OnDisconnectFunc func(ctx context.Context, connID string)

OnDisconnectFunc is called when a tunnel connection is closed. The connID is the same UUID that was passed to OnConnectFunc.

type Relay 

type Relay struct {
	// contains filtered or unexported fields
}

func NewRelay 

func NewRelay(name string, pc net.PacketConn, cert tls.Certificate, handler *icx.Handler, idHasher *hasher.Hasher, router router.Router) *Relay

func (*Relay) Address 

func (r *Relay) Address() netip.AddrPort

Address is the underlay address of the relay.

func (*Relay) ConnectionStats added in v0.23.0 

func (r *Relay) ConnectionStats() []ConnStats

ConnectionStats returns the datapath counters of every live connection, sorted by connection ID. A connection that has no virtual network yet — one still completing its connect — has no counters to report and is left out.

func (*Relay) DisconnectConnection added in v0.23.0 

func (r *Relay) DisconnectConnection(id string)

DisconnectConnection closes one live connection and runs its control-plane cleanup. It is safe to call after another path has already removed it.

func (*Relay) MetricsStore 

func (r *Relay) MetricsStore() *metrics.MetricsStore

MetricsStore returns the metrics store, if configured.

func (*Relay) Name 

func (r *Relay) Name() string

Name is the name of the relay.

func (*Relay) RemoveCredentials added in v0.22.0 

func (r *Relay) RemoveCredentials(tunnelName string)

RemoveCredentials revokes a tunnel's static authentication token so new connects to it fail closed. Only consulted when the default static token validator is in effect (see SetTokenValidator).

func (*Relay) SetCertProvider 

func (r *Relay) SetCertProvider(getCert func(*tls.ClientHelloInfo) (*tls.Certificate, error))

SetCertProvider overrides the source of the relay's TLS server certificate, e.g. to enable hot-reload from disk via pkg/cert/reload. Must be called before Start.

func (*Relay) SetCredentials 

func (r *Relay) SetCredentials(tunnelName, token string)

SetCredentials sets the static authentication token used by agents to authenticate with the relay for a tunnel. Only consulted when the default static token validator is in effect (see SetTokenValidator).

func (*Relay) SetEgressGateway 

func (r *Relay) SetEgressGateway(enabled bool)

SetEgressGateway enables or disables internet egress for the tunnel agents.

func (*Relay) SetLameDuckPeriod 

func (r *Relay) SetLameDuckPeriod(d time.Duration)

SetLameDuckPeriod sets how long the relay keeps forwarding traffic after announcing a drain via GOAWAY, giving agents time to establish replacement sessions before this one goes dark. Zero (the default) shuts down immediately. Must be called before Start.

func (*Relay) SetMetricsStore 

func (r *Relay) SetMetricsStore(s *metrics.MetricsStore)

SetMetricsStore configures the push-based metrics store.

func (*Relay) SetOnConnStatsFinal added in v0.23.0 

func (r *Relay) SetOnConnStatsFinal(fn func(ConnStats))

SetOnConnStatsFinal sets a callback that receives a connection's last datapath counters when it is torn down, and for every connection still live at shutdown, before the virtual networks are removed and the counters go away. It runs at most once per connection, and under no relay lock, so the callback may call back into the relay.

func (*Relay) SetOnConnect 

func (r *Relay) SetOnConnect(onConnect func(ctx context.Context, tunnelName, agentName string, conn controllers.Connection) error)

SetOnConnect sets a callback that is invoked when a new connection is established to the relay.

func (*Relay) SetOnDisconnect 

func (r *Relay) SetOnDisconnect(onDisconnect func(ctx context.Context, agentName, id string) error)

SetOnDisconnect sets a callback that is invoked when a connection is closed.

func (*Relay) SetOnDraining 

func (r *Relay) SetOnDraining(onDraining func(context.Context))

SetOnDraining sets a callback invoked at the start of shutdown, before the GOAWAY goes out and while the relay is still forwarding. Deregistration belongs here: it stops discovery from handing this relay out while its live connections ride out the lame duck.

func (*Relay) SetOnShutdown 

func (r *Relay) SetOnShutdown(onShutdown func(context.Context))

SetOnShutdown sets a callback that is invoked when the relay is shutting down.

func (*Relay) SetTokenValidator 

func (r *Relay) SetTokenValidator(v token.TokenValidator)

SetTokenValidator overrides how agent credentials are authenticated, e.g. with a JWT-backed validator. Must be called before Start.

func (*Relay) Start 

func (r *Relay) Start(ctx context.Context) error

Start starts the relay.

type SingleClusterClientGetter 

type SingleClusterClientGetter struct {
	Client client.Client
}

SingleClusterClientGetter wraps a single client.Client for use with ClientGetter.

func (*SingleClusterClientGetter) GetClient 

func (s *SingleClusterClientGetter) GetClient(_ context.Context, _ uuid.UUID) (client.Client, error)

GetClient returns the wrapped client, ignoring the tunnel UUID.

type TunnelClientMode 

type TunnelClientMode string
const (
	// TunnelClientModeKernel indicates that the tunnel client will use the kernel mode router.
	// This mode requires root privileges and is more efficient for routing traffic.
	TunnelClientModeKernel TunnelClientMode = "kernel"
	// TunnelClientModeUser indicates that the tunnel client will use the user mode router.
	TunnelClientModeUser TunnelClientMode = "user"
)

func TunnelClientModeFromString 

func TunnelClientModeFromString(mode string) (TunnelClientMode, error)

TunnelClientModeFromStringreturns the tunnel client mode for the given string.

type TunnelClientOption 

type TunnelClientOption func(*tunnelClientOptions)

func WithAuthToken 

func WithAuthToken(token string) TunnelClientOption

WithAuthToken sets the authentication token for the tunnel client.

func WithDiagRegistry 

func WithDiagRegistry(r *diag.Registry) TunnelClientOption

WithDiagRegistry enables the agent diagnostics surface. When set, the conn opens a long-lived /diag/rpc stream to tunnelproxy and runs commands from r in response to operator requests. Auth is inherited from the existing QUIC TLS to tunnelproxy.

func WithExternalInterface 

func WithExternalInterface(name string) TunnelClientOption

WithExternalInterface sets the external interface name. This is only valid in kernel mode.

func WithInsecureSkipVerify 

func WithInsecureSkipVerify(skip bool) TunnelClientOption

WithInsecureSkipVerify skips TLS certificate verification of the server.

func WithLabels 

func WithLabels(labels map[string]string) TunnelClientOption

WithLabels sets metadata labels to send on tunnel connections.

func WithMode 

func WithMode(mode TunnelClientMode) TunnelClientOption

WithMode sets the mode of the tunnel client (kernel or user).

func WithPacketObserver 

func WithPacketObserver(obs tunnelconn.PacketObserver) TunnelClientOption

WithPacketObserver sets the packet observer for the tunnel client. The observer will receive notifications for each packet passing through the tunnel.

func WithPcapPath 

func WithPcapPath(path string) TunnelClientOption

WithPcapPath sets the optional path to a packet capture file for the tunnel client.

func WithPreserveDefaultGatewayDestinations 

func WithPreserveDefaultGatewayDestinations(dsts []netip.Prefix) TunnelClientOption

WithPreserveDefaultGatewayDestinations sets destinations for which the existing default gateway will be preserved.

func WithReplacesConnID 

func WithReplacesConnID(id string) TunnelClientOption

WithReplacesConnID declares that this dial replaces a dead connection with the given ID. The server excludes that connection from its same-server diversity check (an agent must not be rejected against its own corpse while the server waits out the QUIC idle timeout) and evicts it.

func WithRootCAs 

func WithRootCAs(caCerts *x509.CertPool) TunnelClientOption

WithRootCAs sets the optional root CA certificates for TLS verification.

func WithSocksListenAddr 

func WithSocksListenAddr(addr string) TunnelClientOption

WithSocksListenAddr sets the listen address for the local SOCKS5 proxy server. Only valid in user mode.

func WithTunnelInterface 

func WithTunnelInterface(name string) TunnelClientOption

WithTunnelInterface sets the tunnel interface name. This is only valid in kernel mode.

type TunnelDialer 

type TunnelDialer struct {
	Router router.Router
	// contains filtered or unexported fields
}

TunnelDialer dials a tunnel connection. Must be started before use.

func (*TunnelDialer) Dial 

func (d *TunnelDialer) Dial(
	ctx context.Context,
	id uuid.UUID,
	addr string,
	opts ...TunnelClientOption,
) (*Conn, error)

Dial dials the TunnelNode server, establishes tunnel connection and sets up client-side routing.

type TunnelServer 

type TunnelServer struct {
	// contains filtered or unexported fields
}

TunnelServer manages QUIC tunnel connections and routes traffic via CONNECT-IP. It exposes ReconcileWithClient for use by reconcilers (standard or multicluster).

func NewTunnelServer 

func NewTunnelServer(
	cg ClientGetter,
	v token.JWTValidator,
	r router.Router,
	opts ...TunnelServerOption,
) (*TunnelServer, error)

NewTunnelServer creates a new server proxy that routes traffic via QUIC tunnels.

func (*TunnelServer) ActiveTCPConns 

func (t *TunnelServer) ActiveTCPConns() int

ActiveTCPConns returns the number of active TCP connections being tracked across all tunnels.

func (*TunnelServer) BFDServer 

func (t *TunnelServer) BFDServer() *bfdl.Server

BFDServer returns the BFD server instance, or nil if BFD is not enabled. Useful for testing (e.g. suppressing heartbeats for specific connections).

func (*TunnelServer) BeginDrain 

func (t *TunnelServer) BeginDrain()

BeginDrain signals that a graceful drain is in progress. Must be called before the server's context is cancelled so that Stop() knows not to force-close connections and the router.

func (*TunnelServer) CloseConnection 

func (t *TunnelServer) CloseConnection(connID string)

CloseConnection closes the tunnel connection with the given ID. No-op if the connection does not exist.

func (*TunnelServer) CloseConnectionsByName 

func (t *TunnelServer) CloseConnectionsByName(name string)

CloseConnectionsByName closes all active connections for the TunnelNode with the given name. WARNING: In multi-tenant environments, multiple TunnelNodes across different projects can share the same name. Prefer CloseConnectionsByUID to avoid cross-project collisions.

func (*TunnelServer) CloseConnectionsByUID 

func (t *TunnelServer) CloseConnectionsByUID(uid string)

CloseConnectionsByUID closes all active connections for the TunnelNode with the given UID. This is safe in multi-tenant environments where multiple projects may have TunnelNodes with the same name but different UIDs.

func (*TunnelServer) ConnTracker 

func (t *TunnelServer) ConnTracker() *conntrack.Tracker

ConnTracker returns the TCP connection tracker for integration with the router's packet forwarding path.

func (*TunnelServer) DiagSessions 

func (t *TunnelServer) DiagSessions() *diag.Sessions

DiagSessions returns the per-agent diag stream registry. Callers (e.g. tunnelproxy operator endpoints) look up an agent's session by its TunnelNode UID and call Invoke to drive a command.

func (*TunnelServer) Drain 

func (t *TunnelServer) Drain(ctx context.Context)

Drain sends BFD AdminDown to all peers, waits for the context to expire, then force-closes all remaining connections and the router.

func (*TunnelServer) LabelSelector 

func (srv *TunnelServer) LabelSelector() string

LabelSelector returns the label selector configured for this server.

func (*TunnelServer) MetricsStore 

func (t *TunnelServer) MetricsStore() *metrics.MetricsStore

MetricsScraper returns the agent metrics scraper, if configured. MetricsStore returns the push-based metrics store, if configured.

func (*TunnelServer) Predicates 

func (t *TunnelServer) Predicates() ([]predicate.Predicate, error)

Predicates returns the predicates to use when setting up the controller. This is useful when integrating with multicluster-runtime.

func (*TunnelServer) ReconcileWithClient 

func (t *TunnelServer) ReconcileWithClient(ctx context.Context, c client.Client, request reconcile.Request) (reconcile.Result, error)

ReconcileWithClient reconciles a TunnelNode using the provided client. This method can be used by both standard reconcilers and multicluster reconcilers.

func (*TunnelServer) Start 

func (t *TunnelServer) Start(ctx context.Context) error

func (*TunnelServer) Stop 

func (t *TunnelServer) Stop() error

func (*TunnelServer) StopAccepting 

func (t *TunnelServer) StopAccepting()

StopAccepting stops accepting new tunnel connections. Existing QUIC connections remain alive (the listener is NOT closed, to preserve the underlying UDP socket that existing connections are multiplexed on).

type TunnelServerOption 

type TunnelServerOption func(*tunnelServerOptions)

func WithBFDListenAddr 

func WithBFDListenAddr(addr netip.Addr) TunnelServerOption

WithBFDListenAddr enables the BFD server on the given overlay address.

func WithCertPath 

func WithCertPath(path string) TunnelServerOption

WithCertPath sets the path to the TLS certificate.

func WithConnTracker 

func WithConnTracker(ct *conntrack.Tracker) TunnelServerOption

WithConnTracker sets an externally-created TCP connection tracker. If set, it is used for ActiveTCPConns() reporting during graceful drain.

func WithExternalAddrs 

func WithExternalAddrs(addrs ...netip.Prefix) TunnelServerOption

WithExternalAddr sets the external IPv6 prefix. This is the IPv6 prefix used to send traffic through the tunnel.

func WithIPAMv4 

func WithIPAMv4(ipamv4 tunnet.IPAM) TunnelServerOption

WithIPAMv4 sets the IPv4 IPAM.

func WithKeyLogPath 

func WithKeyLogPath(path string) TunnelServerOption

WithKeyLogPath sets the path to the TLS key log (disabled by default).

func WithKeyPath 

func WithKeyPath(path string) TunnelServerOption

WithKeyPath sets the path to the TLS key.

func WithLabelSelector 

func WithLabelSelector(labelSelector string) TunnelServerOption

WithLabelSelector sets the label selector to filter TunnelNodes.

func WithMetricsStore 

func WithMetricsStore(s *metrics.MetricsStore) TunnelServerOption

WithMetricsStore enables push-based metrics collection from connected agents. Agents push metrics over the existing HTTP/3 connection; the store must be started separately to satisfy errgroup patterns.

func WithOnAlive 

func WithOnAlive(fn bfdl.OnAliveFunc) TunnelServerOption

WithOnAlive sets a callback invoked on each valid BFD Rx from a client.

func WithOnConnect 

func WithOnConnect(fn OnConnectFunc) TunnelServerOption

WithOnConnect sets a callback that is invoked when a tunnel connection is established.

func WithOnDisconnect 

func WithOnDisconnect(fn OnDisconnectFunc) TunnelServerOption

WithOnDisconnect sets a callback that is invoked when a tunnel connection is closed.

func WithOnDown 

func WithOnDown(fn bfdl.OnDownFunc) TunnelServerOption

WithOnDown sets a callback invoked when a BFD session's detect timer expires (Up→Down transition). This fires before the QUIC connection closes, allowing the caller to proactively mark the endpoint as not-ready so DNS-based service discovery stops routing to it.

func WithProjectIDLookup 

func WithProjectIDLookup(fn func(tunnelUID string) string) TunnelServerOption

WithProjectIDLookup sets a function that resolves a tunnel UID to its project ID. Used by the metrics scraper to label scraped metrics with the owning project.

func WithProxyAddr 

func WithProxyAddr(addr string) TunnelServerOption

WithProxyAddr sets the address to bind the proxy to.

func WithPublicAddr 

func WithPublicAddr(addr string) TunnelServerOption

WithPublicAddr sets the address tunnel proxy is reachable at. This address will be set on the TunnelNode objects that this proxy is serving.

func WithULAPrefix 

func WithULAPrefix(prefix netip.Prefix) TunnelServerOption

WithULAPrefix sets the Unique Local Address prefix.

Source Files

View all Source files

Directories

Path Synopsis
Package agent implements the tunnel agent: the client side of the vpc.apoxy.dev relay stack.
 Package agent implements the tunnel agent: the client side of the vpc.apoxy.dev relay stack.
Package bfdl implements a BFD-lite (RFC 5880 subset) protocol for application-level liveness detection between tunnel agents and the tunnelproxy server.
 Package bfdl implements a BFD-lite (RFC 5880 subset) protocol for application-level liveness detection between tunnel agents and the tunnelproxy server.
Package conntrack provides a lightweight TCP connection tracker that implements connection.PacketObserver.
 Package conntrack provides a lightweight TCP connection tracker that implements connection.PacketObserver.
Package conntrackpc provides a conntrack-style multiplexer for net.PacketConn, suitable for QUIC clients that want multiple "virtual" PacketConns over one UDP socket.
 Package conntrackpc provides a conntrack-style multiplexer for net.PacketConn, suitable for QUIC clients that want multiple "virtual" PacketConns over one UDP socket.
Package endpointselect provides endpoint selection strategies for tunnel connections.
 Package endpointselect provides endpoint selection strategies for tunnel connections.
Package fasttun implements a high-performance interface to Linux TUN devices with support for multi-queue and batched packet I/O.
 Package fasttun implements a high-performance interface to Linux TUN devices with support for multi-queue and batched packet I/O.
Package ipalloc holds the relay-side, in-process connection address allocators for the vpc.apoxy.dev relay (APO-825 §2.8, where a slot is called a "block").
 Package ipalloc holds the relay-side, in-process connection address allocators for the vpc.apoxy.dev relay (APO-825 §2.8, where a slot is called a "block").

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.