kubeguard

package
v0.0.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 27, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Overview

Package kubeguard is the monitor that sits between a sandbox and a Kubernetes cluster (Task 20277).

Why a proxy and not a narrower kubeconfig

pkg/secretbroker already minimises a kubeconfig: a grant names contexts and namespaces, MinimizeKubeconfig drops every context outside the allowlist, and the clusters and users nothing references go with them. That is real — a sandbox cannot reach a cluster whose credential it was never given.

It cannot do the two things this package exists for.

  • A namespace in a kubeconfig context is a *client-side default*. It is the value kubectl uses when you do not pass -n. `kubectl -n kube-system get secrets` ignores it entirely. So "this project may only touch namespace app" was, until now, a suggestion written on the honour system.
  • Read-only cannot be expressed in a kubeconfig at all. There is no field for it. A kubeconfig carries a credential, and the credential's authority is whatever the cluster's RBAC says — which the hub does not control and often cannot narrow, because the kubeconfig a developer uploads is frequently the one their own admin-grade user holds.

Both need something that reads each request and decides. That something must live outside the sandbox, because anything inside it is advice to code that is under no obligation to take it. So: the hub holds the real credential, the sandbox gets a token worth only what the policy allows, and every request passes through a process the sandbox does not control.

This is the same bargain pkg/gitproxy makes for git, and the packages are deliberately shaped alike.

This file

apirequest.go turns an HTTP method and path into the tuple an authorization decision is actually about: verb, API group, resource, subresource, namespace, name. It is the security boundary of the package — everything policy.go decides, it decides about the output of this file — so it is a pure function over a method, a path and a query, with no I/O, and it is tested against the path shapes the real API server accepts rather than the ones kubectl happens to send.

The parse deliberately mirrors the apiserver's own k8s.io/apiserver/pkg/endpoints/request.RequestInfoFactory. Not out of deference: if this package's idea of "which namespace is this request about" differs from the API server's by even one path shape, the difference *is* the vulnerability. A request the proxy reads as namespace "app" and the cluster reads as namespace "kube-system" is a bypass, and it would look like a working allowlist right up until it did not.

Index

Constants

View Source 
const (
	VerbGet              = "get"
	VerbList             = "list"
	VerbWatch            = "watch"
	VerbCreate           = "create"
	VerbUpdate           = "update"
	VerbPatch            = "patch"
	VerbDelete           = "delete"
	VerbDeleteCollection = "deletecollection"
)

Verb constants. These are RBAC's verbs, spelled the way RBAC spells them, because an operator writing an allowlist is going to copy it out of a Role they already have.

View Source 
const (
	// DefaultMaxBodyBytes caps a request body. Kubernetes refuses objects
	// above roughly 1.5 MiB (etcd's own limit), so this is comfortably above
	// anything the API server would accept and far below anything that would
	// cost the hub memory. Under the default read-only policy no request has
	// a body at all.
	DefaultMaxBodyBytes int64 = 3 << 20

	// MaxPatterns bounds each allowlist, so a grant cannot turn every request
	// into a few thousand glob evaluations.
	MaxPatterns = 256
)

Defaults and ceilings.

View Source 
const (
	// DefaultSessionTTL is how long a session lives when the caller does not
	// say. A task's Kubernetes work happens inside one dispatch, so an hour
	// is generous.
	DefaultSessionTTL = time.Hour

	// MaxSessionTTL is the ceiling. Twelve hours, matching
	// gitproxy.MaxSessionTTL and for the same reason: a session that outlives
	// a working day is indistinguishable from having handed the sandbox the
	// kubeconfig outright.
	MaxSessionTTL = 12 * time.Hour
)

Session lifetimes.

Variables

View Source 
var (
	// ErrUnauthenticated covers every authentication failure. It is
	// deliberately one error: distinguishing "no such session" from "wrong
	// token" would be an oracle for enumerating live sessions.
	ErrUnauthenticated = errors.New("kubeguard: unauthenticated")
	// ErrSessionExpired is reported separately from ErrUnauthenticated
	// because the remedy differs and the token was genuinely correct. It is
	// only ever returned to a caller that already proved it held the token.
	ErrSessionExpired = errors.New("kubeguard: session expired")
	// ErrSessionClosed is an operator revocation or hub shutdown.
	ErrSessionClosed = errors.New("kubeguard: session closed")
)

Errors returned by the registry.

View Source 
var DefaultNonResourcePaths = []string{
	"/",
	"/api",
	"/api/*",
	"/apis",
	"/apis/*",
	"/apis/*/*",
	"/version",
	"/openapi/**",
	"/healthz",
	"/readyz",
	"/livez",
}

DefaultNonResourcePaths is the discovery surface kubectl needs before it can issue a single useful request.

Without these, a read-only session is not read-only — it is broken. `kubectl get pods` begins by fetching /api and /apis to learn which groups exist, then /api/v1 to learn that pods are namespaced. Denying discovery produces "the server could not find the requested resource" on every command, which reads as a cluster problem rather than a policy one.

They are all reads of schema and health. None returns cluster data: /api and /apis list group names, /openapi returns the type schema, /version returns the server's build, and the health endpoints return "ok". A trailing "/**" admits any depth below the prefix.

View Source 
var ErrDenied = errors.New("request denied by kubernetes access policy")

ErrDenied is what every refusal wraps, so a caller can test with errors.Is without caring which rule fired.

View Source 
var ReadVerbs = []string{VerbGet, VerbList, VerbWatch}

ReadVerbs are the verbs that only read. This is the definition of "read access" the task is about, and it is the package's default policy.

It is exactly RBAC's read set. Notably absent is "deletecollection", which reads as a plural of something and is not.

Functions

func NormalizeVerbs 

func NormalizeVerbs(in []string) []string

NormalizeVerbs lowercases, trims, drops empties and dedupes a verb list *without* substituting the read-only default for an empty one.

Exported for pkg/config, which builds the deployment floor and must keep "unset" distinguishable from "read-only" — see Intersect.

Types

type APIRequest 

type APIRequest struct {
	// Verb is the RBAC verb this request performs.
	Verb string
	// APIGroup is "" for the core group ("/api/v1/..."), otherwise the group
	// name from "/apis/<group>/<version>/...".
	APIGroup string
	// APIVersion is the version segment.
	APIVersion string
	// Resource is the plural resource name ("pods", "deployments").
	Resource string
	// Subresource is the trailing segment when one is present ("log",
	// "status", "exec").
	Subresource string
	// Namespace is the namespace the request is about, empty for a
	// cluster-scoped request or a collection across all namespaces.
	Namespace string
	// Name is the object name, empty for a collection request.
	Name string
	// IsResource reports whether this is a resource request at all.
	IsResource bool
	// Path is the cleaned request path, always populated.
	Path string
	// Upgrade reports that the client asked to switch protocols
	// (SPDY or websocket). See policy.go for why that is refused.
	Upgrade bool
}

APIRequest is one request, reduced to what a decision is about.

A request is either about a resource (IsResource true, and the fields below are populated) or about a non-resource path such as /version or /healthz (IsResource false, and Path carries it). RBAC makes the same split, for the same reason: there is no namespace or verb-set that means anything about /openapi/v2.

func Parse 

func Parse(r *http.Request) APIRequest

Parse reduces an HTTP request to an APIRequest.

It never fails: a path this function does not recognise as a resource request becomes a non-resource request, which policy.go evaluates against the non-resource allowlist and denies by default. Returning an error for an unparseable path would push the fail-closed decision onto every caller; making it a request nobody allowed keeps it here.

func (APIRequest) ClusterScoped 

func (r APIRequest) ClusterScoped() bool

ClusterScoped reports that the request names no namespace. That is true both of a genuinely cluster-scoped resource (nodes) and of a collection request spanning every namespace (/api/v1/pods) — which is exactly why policy.go treats the two the same: both reach outside any one namespace.

func (APIRequest) Dangerous 

func (r APIRequest) Dangerous() bool

Dangerous reports that the request names a subresource that confers a capability rather than data.

func (APIRequest) ResourceString 

func (r APIRequest) ResourceString() string

ResourceString renders the resource the way RBAC writes it, for audit rows and refusal messages: "pods", "pods/log", "apps/deployments".

func (APIRequest) String 

func (r APIRequest) String() string

String renders the request for a log line: "get pods/log in namespace app".

type Denial 

type Denial struct {
	Reason  DenyReason
	Message string
}

Denial is a refusal carrying its classification and the prose the sandbox sees.

The message is written for the person reading `kubectl`'s output, because that is who will read it: it names what was attempted and what the session is allowed, so the next step is obvious. It never names the cluster credential, the grant, or another tenant.

func (*Denial) Error 

func (d *Denial) Error() string

func (*Denial) Unwrap 

func (d *Denial) Unwrap() error

type DenyReason 

type DenyReason string

DenyReason classifies a refusal. It is a metric label and an audit field, so the set is small and stable: an operator watching a fleet wants to tell "someone's chart tried to write" from "someone probed kube-system". 

const (
	DenyVerbNotAllowed       DenyReason = "verb_not_allowed"
	DenyNamespaceNotAllowed  DenyReason = "namespace_not_allowed"
	DenyClusterScope         DenyReason = "cluster_scope"
	DenyResourceNotAllowed   DenyReason = "resource_not_allowed"
	DenyDangerousSubresource DenyReason = "dangerous_subresource"
	DenyNonResourcePath      DenyReason = "non_resource_path"
	DenyProtocolUpgrade      DenyReason = "protocol_upgrade"
	DenyBodyTooLarge         DenyReason = "body_too_large"
)

func DenyReasonOf 

func DenyReasonOf(err error) (DenyReason, bool)

DenyReasonOf extracts the classification from an error returned by Decide.

type Event 

type Event struct {
	Kind      EventKind
	SessionID string
	// Cluster is the API server endpoint the session is pinned to, so a fleet
	// with several clusters can be read apart. It is a URL, never a
	// credential.
	Cluster string
	// Context is the kubeconfig context the session was minted from.
	Context string
	// Verb, Resource, Namespace and Name describe the request. Empty on
	// session lifecycle rows.
	Verb      string
	Resource  string
	Namespace string
	Name      string
	// Reason classifies a denial, for metrics.
	Reason DenyReason

	ProjectID  string
	TaskID     string
	ExecutorID string
	Actor      string
	GrantID    string
	LeaseID    string

	Detail string
	At     time.Time
}

Event is one audit row.

func (Event) String 

func (e Event) String() string

String renders the event for a log line.

type EventKind 

type EventKind string

EventKind classifies an audit row. 

const (
	EventSessionMinted EventKind = "session_minted"
	EventSessionClosed EventKind = "session_closed"
	// EventRequestDenied is the row that matters. It is the only place a
	// sandbox's attempt to write to, or read outside, its granted scope is
	// recorded, and nothing else in cloop would record it.
	EventRequestDenied EventKind = "request_denied"
	// EventRequestAllowed is sampled rather than written per request; see
	// Registry.OnEvent's contract in session.go.
	EventRequestAllowed EventKind = "request_allowed"
	// EventRejected covers refusals that happen before a session is
	// identified: no credential, an unknown token, an expired session.
	EventRejected EventKind = "rejected"
)

type MintRequest 

type MintRequest struct {
	// Kubeconfig is the credential the hub holds, as delivered by the secret
	// broker. It is parsed here and kept only in RESTConfig form.
	Kubeconfig []byte
	// Context selects which context in the kubeconfig to use. Empty means the
	// document's current-context.
	Context string
	// Policy is what the session may do. The zero value means
	// ReadOnlyPolicy.
	Policy Policy
	// TTL bounds the session. Zero means DefaultSessionTTL; above
	// MaxSessionTTL is an error rather than a silent clamp.
	TTL time.Duration

	ProjectID  string
	TaskID     string
	ExecutorID string
	Actor      string
	GrantID    string
	LeaseID    string
}

MintRequest describes a session to create.

type Minted 

type Minted struct {
	Session *Session
	// Token is the bearer token the sandbox presents. This is the only copy
	// that will ever exist; the registry keeps a hash.
	Token string
	// Kubeconfig is the document to deliver into the sandbox. It points at
	// the proxy and carries Token, and it contains no cluster credential.
	Kubeconfig []byte
}

Minted is a new session plus the one copy of its token.

type Options 

type Options struct {
	// Transport overrides the outbound transport. Tests use it; in
	// production it is nil and each session's own transport, built from its
	// kubeconfig's TLS material, is used instead.
	Transport http.RoundTripper
}

Options configures a Proxy.

type Policy 

type Policy struct {
	// Verbs is the RBAC verb allowlist. Empty means ReadVerbs.
	Verbs []string `json:"verbs,omitempty"`

	// Namespaces is a glob allowlist of namespace names. Empty means no
	// namespace confinement — every namespace the credential can reach.
	//
	// When it is non-empty it does more than filter: it also refuses
	// collection requests that name no namespace (see decideNamespace), which
	// is the enforcement a kubeconfig's default namespace could never
	// provide.
	Namespaces []string `json:"namespaces,omitempty"`

	// Resources is a glob allowlist matched against "resource" for the core
	// group and "group/resource" otherwise — the spelling RBAC uses, so a
	// pattern can be copied out of a Role. Empty means every resource.
	//
	// Patterns match the resource *without* its subresource, so "pods"
	// admits "pods/log". A subresource is governed by the verb and by the
	// dangerous-subresource rule, not by this list.
	Resources []string `json:"resources,omitempty"`

	// NonResourcePaths is a glob allowlist of non-resource URLs. Empty means
	// DefaultNonResourcePaths. A trailing "/**" admits any depth below the
	// prefix.
	NonResourcePaths []string `json:"non_resource_paths,omitempty"`

	// MaxBodyBytes caps a request body. Zero means DefaultMaxBodyBytes.
	MaxBodyBytes int64 `json:"max_body_bytes,omitempty"`
}

Policy is what one session may do against one cluster.

The zero value is not usable directly; Normalize turns it into the read-only policy, which is the answer to "the operator granted a cluster and said nothing else".

func ReadOnlyPolicy 

func ReadOnlyPolicy() Policy

ReadOnlyPolicy returns the policy this package exists to enforce: get, list and watch, every namespace, every resource, discovery allowed.

Narrow it with Namespaces to confine a project to its own namespaces.

func (Policy) AllowsNamespace 

func (p Policy) AllowsNamespace(ns string) bool

AllowsNamespace reports whether ns is inside the allowlist. An empty allowlist admits every namespace.

The glob semantics deliberately match pkg/secretbroker's matchAny: "*" admits everything, otherwise path.Match decides. The same allowlist string is written once on a grant and read in both places, so the two agreeing is a correctness requirement, not a nicety — kubeguard_constraints_test.go asserts it directly.

func (Policy) AllowsNonResourcePath 

func (p Policy) AllowsNonResourcePath(pth string) bool

AllowsNonResourcePath reports whether a non-resource URL is readable.

func (Policy) AllowsResource 

func (p Policy) AllowsResource(r APIRequest) bool

AllowsResource reports whether the request's resource is in the allowlist.

func (Policy) AllowsVerb 

func (p Policy) AllowsVerb(v string) bool

AllowsVerb reports whether the verb is in the allowlist.

func (Policy) Decide 

func (p Policy) Decide(r APIRequest) error

Decide returns nil when the request may be forwarded, or a *Denial.

The order of the checks is chosen for the message the operator reads, not for speed: the most specific and most alarming reason wins, so a sandbox that tried to exec into a pod is told that, rather than being told its verb was fine and leaving the reader to work out why the request still failed.

func (Policy) Intersect 

func (p Policy) Intersect(other Policy) (Policy, error)

Intersect narrows p by other and returns the result. Neither input is modified.

This is how a deployment-wide floor and a per-grant policy combine: the session runs under what *both* permit, so an operator who sets executors.kube_guard.verbs to read-only has made every grant on the hub read-only regardless of what its own verbs say, and a grant that asks for less than the floor allows still gets only what it asked for.

Verbs intersect exactly — there are eight of them and set intersection is the whole answer.

The glob lists cannot. There is no general way to compute the intersection of two glob languages, so this narrows conservatively instead: an entry from other survives only if p already admits it. That is sound in the direction that matters (the result never admits something p denied) and lossy in the other — a grant for "app-*" under a floor of "app-1" yields nothing, even though "app-1" is in both languages.

The lossy case is returned as an error rather than as a policy, and the distinction is load-bearing. An empty *namespace* list does not mean "no namespaces"; it means "no namespace confinement", so returning one would turn a floor the grant failed to satisfy into a session that can read every namespace in the cluster — a widening, produced by the narrowing function, in the one direction a security control must never fail. The same is true of Resources. So an empty result on any dimension is a refusal, and GuardKubeconfig turns it into a lease that fails with the two policies named.

In practice this is rare: the common floor is empty or "*", where no narrowing happens at all. An operator who does write a floor should write it as a literal superset of what grants will ask for.

func (Policy) IsZero 

func (p Policy) IsZero() bool

IsZero distinguishes "never filled in" from "deliberately empty", so a caller can tell a default apart from a choice.

func (*Policy) Normalize 

func (p *Policy) Normalize()

Normalize fills defaults, lowercases and dedupes verbs, and drops empty patterns. It is idempotent.

func (Policy) ReadOnly 

func (p Policy) ReadOnly() bool

ReadOnly reports whether every permitted verb only reads. It is what the UI and audit rows render, and it is computed from the verb list rather than tracked alongside it so the two cannot disagree.

func (Policy) Summary 

func (p Policy) Summary() string

Summary renders the policy for an audit row or a UI badge. It is short by design: "read-only, namespaces app|app-staging".

func (Policy) Validate 

func (p Policy) Validate() error

Validate reports the first structural problem. Call after Normalize.

func (Policy) ValidateAsFloor 

func (p Policy) ValidateAsFloor() error

ValidateAsFloor checks a deployment-wide floor, where an empty verb list is legitimate and means "no ceiling — let each grant decide".

Separate from Validate because the two are asked about different objects: a *session* policy permitting no verbs permits nothing and is a bug, while a *floor* permitting no verbs in particular is the default configuration.

type Proxy 

type Proxy struct {
	// contains filtered or unexported fields
}

Proxy serves the Kubernetes API to sandboxes, under policy.

func New 

func New(reg *Registry, opts Options) (*Proxy, error)

New returns a proxy that authenticates against reg.

func (*Proxy) Addr 

func (p *Proxy) Addr() string

Addr reports where the proxy is listening.

func (*Proxy) Close 

func (p *Proxy) Close() error

Close stops serving and releases pooled upstream connections.

func (*Proxy) Registry 

func (p *Proxy) Registry() *Registry

Registry returns the registry this proxy authenticates against.

func (*Proxy) Serve 

func (p *Proxy) Serve(ln net.Listener) error

Serve accepts connections on ln until Close.

func (*Proxy) ServeHTTP 

func (p *Proxy) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP is the whole decision path, in order.

type Registry 

type Registry struct {
	// BaseURL is the https endpoint sandboxes are pointed at.
	BaseURL string
	// CABundle is the PEM the sandbox should trust when connecting to
	// BaseURL. Empty means the sandbox's own trust store is expected to
	// contain it — correct for a publicly-signed certificate, and the reason
	// this is a separate field rather than an implicit skip-verify.
	CABundle []byte
	// Now is the clock, injectable for tests.
	Now func() time.Time
	// OnEvent receives audit rows. It runs on the request goroutine, so it
	// must be quick: a slow sink here delays a sandbox's API call.
	//
	// Denials are always delivered. Allowed requests are not, by default —
	// a `kubectl get pods` is several requests and a watch is one that never
	// ends, so a row per allowed request would bury the denials that matter
	// in discovery traffic. Set SampleAllowed to change that.
	OnEvent func(Event)
	// SampleAllowed, when true, emits an EventRequestAllowed for every
	// forwarded request. Off by default; see OnEvent.
	SampleAllowed bool
	// contains filtered or unexported fields
}

Registry holds the process's live sessions.

func NewRegistry 

func NewRegistry(baseURL string) (*Registry, error)

NewRegistry returns a registry whose sessions are advertised at baseURL.

baseURL must be https. The bearer token rides an Authorization header on every request, and a loopback listener is no exception: a sandbox is by construction something that may share a host with whatever else is listening on loopback.

func (*Registry) Authenticate 

func (r *Registry) Authenticate(token string) (*Session, error)

Authenticate resolves a bearer token to a live session.

The token is "<id>.<secret>". Splitting it lets the map be indexed without scanning, and the comparison that decides the answer is a constant-time compare of the full token's hash — so a caller learns nothing from timing about either half.

func (*Registry) Close 

func (r *Registry) Close(id, reason string)

Close revokes a session and records why.

The session is closed rather than deleted so an in-flight request finds it and is refused with ErrSessionClosed, and so the closing row carries the counters. ReapExpired removes it later.

func (*Registry) CloseForLease 

func (r *Registry) CloseForLease(leaseID, reason string) int

CloseForLease revokes every session minted against a lease, and reports how many it closed.

This is what makes lease revocation mean something for Kubernetes. Without it, revoking a lease wipes the credential directory in the sandbox while the session the sandbox already authenticated with keeps working until its TTL — the exact gap Task 20178 closed for the other kinds.

func (*Registry) Mint 

func (r *Registry) Mint(req MintRequest) (*Minted, error)

Mint parses the kubeconfig, stores the credential, and returns a session plus the sandbox-facing document.

The kubeconfig is parsed by pkg/executor/kubernetes.ParseKubeconfig rather than a parser of this package's own. That function already refuses the three things a tenant-supplied kubeconfig must never be allowed to do — name an `exec` credential plugin, name an `auth-provider`, or point a credential at a file path on the control-plane host — and having one parser means those refusals cannot drift apart from the driver's.

func (*Registry) ReapExpired 

func (r *Registry) ReapExpired() int

ReapExpired drops lapsed and closed sessions, and returns how many it removed.

Expiry is enforced at authentication regardless, so this is hygiene — without it the map grows for the life of the process.

func (*Registry) Session 

func (r *Registry) Session(id string) (*Session, bool)

Session returns a session by ID without authenticating, for operators.

func (*Registry) Sessions 

func (r *Registry) Sessions() []*Session

Sessions returns every session the registry holds.

type Session 

type Session struct {
	// ID identifies the session. It is not a secret: it is the first half of
	// the bearer token and appears in every audit row.
	ID string
	// ClusterURL is the API server the session is pinned to. Recorded for
	// audit; the actual destination is upstream.Server, which no caller can
	// influence.
	ClusterURL string
	// ContextName is the kubeconfig context the session was minted from.
	ContextName string
	// Policy is what this session may do. Applied to every request.
	Policy Policy

	// Attribution, all recorded on audit rows.
	ProjectID  string
	TaskID     string
	ExecutorID string
	Actor      string
	GrantID    string
	LeaseID    string

	IssuedAt  time.Time
	ExpiresAt time.Time
	// contains filtered or unexported fields
}

Session is one sandbox's scoped access to one cluster.

func (*Session) CloseReason 

func (s *Session) CloseReason() string

CloseReason returns why the session was closed, or "".

func (*Session) Closed 

func (s *Session) Closed() bool

Closed reports whether the session was revoked.

func (*Session) Describe 

func (s *Session) Describe() string

Describe renders the session for a diagnostic line, never its credential.

func (*Session) Expired 

func (s *Session) Expired(now time.Time) bool

Expired reports whether the session has lapsed.

func (*Session) Stats 

func (s *Session) Stats() Stats

Stats snapshots the counters.

func (*Session) Upstream 

func (s *Session) Upstream() *kubernetes.RESTConfig

Upstream returns the real cluster credential. Only the proxy calls it, and only to build the outbound request.

type Stats 

type Stats struct {
	Allowed  int64
	Denied   int64
	BytesIn  int64
	BytesOut int64
}

Stats is a session's traffic, for the closing audit row.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.