kubernetes

package
v0.26.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 8, 2026 License: MIT Imports: 33 Imported by: 0

Documentation

Overview

Package kubernetes implements the agent runtime backend on top of a Kubernetes cluster.

The mapping is:

HostedAgentPool -> a PriorityClass, a ResourceQuota scoped to it, and
                         one shared PersistentVolumeClaim
HostedAgentInstance   -> a Deployment, a Service, and a Secret, with the
                         pool volume mounted at a per-instance subPath

ResourceQuota cannot select on labels, and PriorityClass is the only per-pod attribute its scopeSelector understands. Tagging every sandbox with its pool's PriorityClass is therefore what lets several pools share one namespace. The PriorityClass carries no scheduling intent: every pool uses the same value and never preempts.

Sandboxes are Burstable. The quota's requests budget is the pool's capacity, and its limits budget is that capacity multiplied by the overcommit ratio. Co-location happens implicitly: the pool volume binds to whichever node the first sandbox lands on, and the scheduler then confines the rest of the pool to that node.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ReachableServerURL

func ReachableServerURL(ctx context.Context, client kclient.Client, serverURL string) (string, error)

ReachableServerURL rewrites Obot's own address into one a sandbox can use.

A sandbox is told where to find Obot, and that address is resolved from inside the cluster rather than on the machine Obot runs on. When Obot runs in the cluster the two agree. When it does not -- which is what `make dev` does, with Obot on the developer's machine and sandboxes in a cluster -- the configured address is a loopback one, and loopback inside a pod is the pod itself. An agent would then be handed a URL that resolves to nothing, with every MCP call and model request failing on a connection refused that says nothing about why.

The node's address is used because the developer's machine is the node: Obot listens on all interfaces, so a pod reaching the node on Obot's port reaches Obot. Anything that is not a loopback address is returned untouched, since it is already an address someone chose to be reachable.

This is a development convenience. In production Obot runs in the cluster and its own Service name resolves, so nothing here applies.

Types

type Backend

type Backend struct {
	// contains filtered or unexported fields
}

func New

func New(client, cachedClient kclient.Client, opts Options) (*Backend, error)

func (*Backend) AttachTerminal

AttachTerminal connects to the console of a sandbox's running process.

This attaches rather than execs. Exec would start a second process with its own TTY, which gives a shell beside the agent but not the agent's own session; attach joins the console the container was started with, so an operator sees exactly what the agent is doing. That console only exists because the harness was marked interactive, which is why a terminal requires one.

func (*Backend) DeleteInstance

DeleteInstance removes the sandbox and then erases its directory on the shared pool volume. kubelet never cleans up a subPath directory, so without this every deleted sandbox would leak its workspace.

The cleanup job is created before the workload is pruned, because it is the only place the pool identity survives: InstanceRef does not carry the pool, and the Deployment that does is about to be deleted.

func (*Backend) DeletePool

func (*Backend) DevRoute

func (b *Backend) DevRoute(backendID string) (string, http.RoundTripper, bool)

DevRoute returns an address and transport that reach a sandbox from outside the cluster, by way of the API server's service proxy.

A sandbox is only addressable at a cluster-internal name, which resolves nowhere when Obot runs on a developer's machine. The API server will proxy to a Service on our behalf, so a developer already holding cluster credentials can reach their agent without a port-forward or a tunnel.

This is for development only. Reaching a Service this way needs the services/proxy permission, which grants access to every Service in the cluster, not merely agent sandboxes. A production deployment runs Obot inside the cluster where the internal address resolves directly, so it neither needs nor should be granted that permission.

The bool reports whether a route could be produced at all; callers fall back to the sandbox's own address when it is false.

func (*Backend) GetPoolUtilization

func (b *Backend) GetPoolUtilization(ctx context.Context, ref agentbackend.PoolRef) (agentbackend.UtilizationSnapshot, error)

GetPoolUtilization reports live resource consumption for a pool.

Usage is measured per sandbox through metrics.k8s.io. Where measurement is unavailable -- no metrics-server, or a pod too new to have been sampled -- that sandbox falls back to its committed request, so a pool always reports a usable number rather than understating itself to zero. Falling back to the request rather than to zero is deliberate: a sandbox that exists has certainly reserved its request, so that is the floor of what it is using.

func (*Backend) ObserveInstance

func (*Backend) ObservePool

func (*Backend) ReconcileInstance

func (*Backend) ReconcilePool

type Options

type Options struct {
	// Namespace holds every sandbox and pool object. One namespace serves all
	// pools; they are separated by PriorityClass, not by namespace.
	Namespace string
	// ClusterDomain is used to build in-cluster service URLs.
	ClusterDomain string
	// StorageClassName provisions pool volumes. It should use
	// volumeBindingMode: WaitForFirstConsumer, which is what confines a pool to
	// a single node.
	StorageClassName string
	// RuntimeClassName is the name of the runtime class to use for the sandbox pods.
	RuntimeClassName string
	// Affinity, Tolerations, and NodeSelector constrain every pod created by
	// the backend, including cleanup jobs that must reach the pool volume.
	Affinity     *corev1.Affinity
	Tolerations  []corev1.Toleration
	NodeSelector map[string]string
	// FSGroup owns the per-instance subdirectory on the shared volume.
	FSGroup int64
	// PodSecurityLevel must match the Pod Security Admission level enforced on
	// Namespace. A sandbox that does not satisfy it is refused at admission,
	// which surfaces as a Deployment whose pod never appears. Empty means
	// restricted, which is what the Helm chart labels the namespace with.
	PodSecurityLevel PodSecurityLevel
	// ImagePullSecrets are attached to every sandbox pod.
	ImagePullSecrets []string
	// CleanupImage runs the per-instance volume cleanup job. It needs a shell
	// and coreutils; any small base image will do.
	CleanupImage string
	// ImagePullPolicy controls whether a sandbox image is re-pulled on every
	// start. Always is right for a mutable tag such as :latest, but refuses an
	// image that was preloaded into the node rather than published to a
	// registry -- which is how an air-gapped install, and a developer testing a
	// locally built harness, supply one. Empty means Always.
	ImagePullPolicy string
	// RESTConfig enables live usage reporting through metrics.k8s.io. Without
	// it, utilization reports committed requests instead of measured usage.
	RESTConfig *rest.Config
}

Options configures the backend. Everything here is deployment-wide; nothing is per-agent or per-user.

type PodSecurityLevel

type PodSecurityLevel string

PodSecurityLevel is the Pod Security Admission level the sandbox namespace enforces. Sandboxes share that namespace with MCP servers, so a sandbox that does not satisfy the namespace's level is rejected at admission rather than failing later -- the Deployment is created and no pod ever appears.

This mirrors the levels the MCP Kubernetes backend applies, and defaults to restricted for the same reason: it is what the Helm chart labels the namespace with unless an operator lowers it.

const (
	PodSecurityPrivileged PodSecurityLevel = "privileged"
	PodSecurityBaseline   PodSecurityLevel = "baseline"
	PodSecurityRestricted PodSecurityLevel = "restricted"
)

func ParsePodSecurityLevel

func ParsePodSecurityLevel(level string) PodSecurityLevel

ParsePodSecurityLevel maps configuration to a level, defaulting to restricted. An unrecognized value is treated as restricted rather than rejected: the strict reading is the safe one, and it is also the only one that keeps working when the namespace really is restricted.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL