grpc

package
v1.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 17, 2026 License: Apache-2.0 Imports: 93 Imported by: 0

Documentation

Index

Constants

View Source
const (
	MaxRecvMessageSize = 4 << 20
	MaxSendMessageSize = 64 << 20
)

MaxRecvMessageSize is the largest gRPC message the agent reads from a caller with a client certificate. Any certificate holder can send these, a request is decoded several times, and a connection may carry many streams, so it is a few MiB, not tens (the agent has 128Mi). MaxSendMessageSize is what it may answer with: a batch of up to MaxItems objects.

View Source
const (
	// DefaultClientCertTTL is how long an issued client certificate is valid.
	DefaultClientCertTTL = 30 * 24 * time.Hour
)
View Source
const DefaultEnrollMaxMessage = 64 << 10

DefaultEnrollMaxMessage is the largest message the Enroll server (the one open to callers without a certificate) reads.

View Source
const DefaultRenewMinInterval = 10 * time.Second

DefaultRenewMinInterval is how often one tenant may renew its certificate.

View Source
const (
	// MaxPrewarmImages is the most images one PrewarmImages call may name, and maxPrewarmEntries the most the agent keeps track of
	// (a goroutine starts per distinct image): past the second the oldest finished one is forgotten, and when every one is still
	// running the new image is refused.
	MaxPrewarmImages = 200
)

SetPrewarm configures cache prewarming; call it before serving.

Variables

This section is empty.

Functions

func NewVersionedClientset

func NewVersionedClientset(cfg *rest.Config) (versioned.Interface, error)

NewVersionedClientset builds the laboratory typed clientset forcing JSON content negotiation. The clientset is generated with --prefers-protobuf, but CRDs are served only as JSON, so protobuf negotiation fails at runtime.

func NodePlatforms

func NodePlatforms(k8s kubernetes.Interface, selector map[string]string) func(context.Context) []v1.Platform

NodePlatforms lists the distinct OS/architecture pairs of the nodes matching a nodeSelector.

Types

type ErrorJournalConfig

type ErrorJournalConfig struct {
	// ReleaseNamespace holds the events of the operator, node-agents and proxy; the group namespaces hold those of the VPN and
	// gateway pods. Empty: only the group namespaces are read.
	ReleaseNamespace string
	// Interval is how often the release namespace is read; GroupInterval how often the group namespaces are (one list each).
	Interval, GroupInterval time.Duration
	// Self is the agent's own aggregator (its logger feeds it); nil when it has none.
	Self *errorlog.Aggregator
}

ErrorJournalConfig tells the agent where the components' events are.

type Features

type Features struct {
	Lifecycle *protobuf.LifecycleFeature
	// StatePersistence is the cluster switch; Debounce, ExcludePaths, WriteQuota and MaxFileSize are its cluster values
	// (bytes), the ceilings of a tenant's own limits.
	StatePersistence bool
	Debounce         time.Duration
	ExcludePaths     []string
	WriteQuota       int64
	MaxFileSize      int64
	TenantQuota      int64
	MaxEntries       int
	// CacheEnabled and CacheRegistries describe the image cache.
	CacheEnabled    bool
	CacheRegistries []string
	// SchedulerEnabled and SchedulerMaxPods describe the pod queue.
	SchedulerEnabled bool
	SchedulerMaxPods int32
	// LabsDomain and VPNEndpoint are the public hosts of lab endpoints and of WireGuard.
	LabsDomain  string
	VPNEndpoint string
	// ProxyAccessTokenMaxTTL and ProxySessionMaxTTL are the L7 proxy's limits.
	ProxyAccessTokenMaxTTL time.Duration
	ProxySessionIdleTTL    time.Duration
	ProxySessionMaxTTL     time.Duration
	// DeviceProfiles are the enabled catalog profile IDs (internal/profiles).
	DeviceProfiles []string
	// Limits are the caps CreateLabs enforces.
	Limits limits.Limits
	// GroupPods is how a group's own pods are sized (the maximum CreateLabGroups accepts) and the default size without one.
	GroupPods GroupPodsFeature
}

Features are the platform's choices the backend needs to know, set once from the chart.

type GroupPodsFeature

type GroupPodsFeature struct {
	Sizing                     grouppods.Sizings
	DefaultVPN, DefaultGateway grouppods.Overhead
}

GroupPodsFeature is grouppods.Sizings plus the chart's default sizes.

type Handler

type Handler struct {
	protobuf.UnimplementedLabManagerServer
	// contains filtered or unexported fields
}

Handler implements protobuf.LabManagerServer against the laboratory typed clientset (custom resources on the workload cluster) and, for resources whose data lives partly in core Secrets (e.g. LabGroupClient's wg.conf), the plain kubernetes clientset.

func NewHandler

func NewHandler(cs versioned.Interface, k8s kubernetes.Interface, metrics metricsclient.Interface, agentID ...string) *Handler

NewHandler builds a Handler backed by the given typed clientset, plain kubernetes clientset, and (optionally) a metrics clientset. A nil metrics client disables live usage reporting without erroring.

func (*Handler) Authorize

func (h *Handler) Authorize(ctx context.Context) error

Authorize admits a call: the caller's certificate must name a Tenant that exists (an unknown CN is PermissionDenied, the default tenant included: it has a Tenant object like any other, so its certificates can be revoked too) and must belong to the tenant's current enrollment epoch.

func (*Handler) CreateLabGroupClients

CreateLabGroupClients provisions WireGuard VPN clients. The agent generates every keypair: the PRIVATE key lives only in this call and never touches the cluster. Only the public key is registered; the controller assigns an IP and assembles the config (with a private-key placeholder) into the CR status. The agent then substitutes the real private key and returns the finished config. The caller (control plane) stores it; no later call returns it. A client that already exists is EXISTS (labels it lacks are added: UPDATED) and comes back with the placeholder config.

func (*Handler) CreateLabGroups

CreateLabGroups creates LabGroup custom resources. Re-sending an item whose group exists with the same spec is EXISTS (labels it lacks are added: UPDATED); a group with a different spec is FAILED for that item.

func (*Handler) CreateLabs

CreateLabs creates Lab custom resources from variants (the spec and common variables, sent once) and items. Re-sending an item whose lab exists with the same spec is EXISTS (missing labels are added: UPDATED). Initial-runtime creates retain the existing variable rewrite behavior; lifecycle-managed copies keep their Secrets. A lab with a different immutable spec is FAILED for that item.

func (*Handler) DeleteLabGroupClients

func (h *Handler) DeleteLabGroupClients(ctx context.Context, in *protobuf.DeleteRequest) (*protobuf.BatchResult, error)

DeleteLabGroupClients deletes LabGroupClients. Deletion is asynchronous: until it is finished, creating the same name fails with a retryable TERMINATING error.

func (*Handler) DeleteLabGroups

func (h *Handler) DeleteLabGroups(ctx context.Context, in *protobuf.DeleteRequest) (*protobuf.BatchResult, error)

DeleteLabGroups deletes LabGroups. Deletion is asynchronous (finalizers): it is finished once ListLabGroups no longer returns the group. Until then Create and the other writes on that name fail with a retryable TERMINATING error.

func (*Handler) DeleteLabs

func (h *Handler) DeleteLabs(ctx context.Context, in *protobuf.DeleteRequest) (*protobuf.BatchResult, error)

DeleteLabs deletes Labs. Deletion is asynchronous: until it is finished, creating the same name fails with a retryable TERMINATING error.

func (*Handler) Enroll

Enroll exchanges a tenant's one-time token for a client certificate and registers the tenant's access public key. See the proto for the contract.

func (*Handler) ExportDeviceSnapshot

ExportDeviceSnapshot streams the latest state of a snapshot-backed device as one archive: the squashed difference to its base image. See the proto for the format.

func (*Handler) GetCapacity

func (h *Handler) GetCapacity(ctx context.Context, _ *protobuf.Empty) (*protobuf.CapacityResponse, error)

GetCapacity is the caller's tenant view: its quota, what its pods reserve and use, what is free. It never reveals cluster-wide numbers.

func (*Handler) GetFeatures

func (h *Handler) GetFeatures(ctx context.Context, _ *protobuf.Empty) (*protobuf.FeaturesResponse, error)

GetFeatures is the caller's tenant view of what the laboratory offers.

func (*Handler) ListLabGroupClients

func (h *Handler) ListLabGroupClients(ctx context.Context, in *protobuf.ListRequest) (*protobuf.LabGroupClientList, error)

ListLabGroupClients lists LabGroupClients by names (items), by selector, or all of them; lab_group narrows the listing to one group. Their config carries the private-key placeholder (the cluster never holds the real key).

func (*Handler) ListLabGroups

func (h *Handler) ListLabGroups(ctx context.Context, in *protobuf.ListRequest) (*protobuf.LabGroupList, error)

ListLabGroups lists LabGroups by names (items), by selector, or all of them.

func (*Handler) ListLabs

func (h *Handler) ListLabs(ctx context.Context, in *protobuf.ListRequest) (*protobuf.LabList, error)

ListLabs lists Labs by names (items), by selector, or all of them; lab_group narrows the listing to one LabGroup.

func (*Handler) ListMaintenanceWindows

ListMaintenanceWindows lists the maintenance windows announced by the cluster operator that apply to the caller's tenant, soonest first. Windows that are over are left out unless asked for. The agent itself refuses and delays nothing during a window: the backend plans around it.

func (*Handler) Monitoring

Monitoring streams the platform state to one subscriber. It starts with a full snapshot, or, when the subscriber names an agent epoch and sequence the journal still covers, with the updates it missed; then it sends only changes. All subscribers share one poller and one journal (see monitor), each gets its own selector, minimum interval and position.

func (*Handler) Ping

Ping answers the backend's connection health probe (mTLS and CN allowlist have already been checked by the server interceptors).

func (*Handler) PrewarmImages

PrewarmImages starts (or reports on) the cache prewarm of images. It returns at once with the current state of every requested image; repeat the call to poll.

The cache is shared, so the caller's tenant policy applies: an image the tenant may not use is reported FAILED and never fetched, and the list of every warmed image (an empty request) shows only the images the tenant may use. A tenant with registry credentials of its own pulls straight from its registries, so its images are SKIPPED: the shared cache would fetch them with the platform's credentials, or not at all.

An image whose registry matches a credential of the request is only checked: the registry is asked for its digest with those credentials (and nothing else) and the answer is DONE with the digest (for a tag of an image index, the index digest: informational), or FAILED. A digest-pinned reference is asked for too. The checks are bounded (CredTimeout each, CredTotalTimeout together), use a transport that refuses internal addresses, and fail with a generic message, never the transport's error. It is neither fetched into the cache nor remembered, so the credentials and the result live only as long as this call.

func (*Handler) RemoveAccessKey

func (h *Handler) RemoveAccessKey(ctx context.Context, in *protobuf.RemoveAccessKeyRequest) (*protobuf.Empty, error)

RemoveAccessKey removes a key of the caller's tenant; the last one stays.

func (*Handler) RenewCertificate

RenewCertificate issues a new certificate for the caller's tenant, in the tenant's current epoch. A tenant may renew once per renewMinInterval: a renewal loop is how a revoked certificate used to outlive the revocation.

func (*Handler) RescueDevices

RescueDevices switches devices between their normal start and rescue mode: the latest snapshot started with a shell instead of the image entrypoint.

func (*Handler) ResetDevices

func (h *Handler) ResetDevices(ctx context.Context, in *protobuf.DevicesRequest) (*protobuf.BatchResult, error)

ResetDevices discards the snapshots of the devices and restarts them from their base image. The operator does the work: a new reset token on the Device is what it acts on, so repeating the call resets again.

func (*Handler) RetireLabGroups added in v1.1.0

func (*Handler) RetireLabs added in v1.1.0

func (*Handler) RotateAccessKey

func (h *Handler) RotateAccessKey(ctx context.Context, in *protobuf.RotateAccessKeyRequest) (*protobuf.Empty, error)

RotateAccessKey adds an access public key to the caller's tenant.

func (*Handler) RunErrorJournal

func (h *Handler) RunErrorJournal(ctx context.Context)

RunErrorJournal reads the components' events (and the agent's own errors) until ctx ends.

func (*Handler) RunTenantStatus

func (h *Handler) RunTenantStatus(ctx context.Context, interval time.Duration)

RunTenantStatus keeps the status of every Tenant (reserved, used) fresh until ctx ends.

func (*Handler) SetClientCA

func (h *Handler) SetClientCA(certFile, keyFile string, ttl time.Duration)

SetClientCA gives the agent the CA that signs client certificates (the one that verifies them on the server side) and the lifetime of what it signs. The files are read per signature, so a rotated CA needs no restart.

func (*Handler) SetErrorJournal

func (h *Handler) SetErrorJournal(c ErrorJournalConfig)

SetErrorJournal configures the journal; call before Run.

func (*Handler) SetFeatures

func (h *Handler) SetFeatures(f Features)

SetFeatures sets the platform features the agent reports (GetFeatures, Monitoring).

func (*Handler) SetGroupOverhead

func (h *Handler) SetGroupOverhead(o grouppods.Overhead)

SetGroupOverhead sets what the VPN and gateway pods of one LabGroup request together: the service overhead of every group, which a reservation must add to its labs.

func (*Handler) SetImagePolicy

func (h *Handler) SetImagePolicy(deny []string, cacheAddrs ...string)

SetImagePolicy sets the platform's side of the image policy: the repositories no tenant may use and the addresses of the platform registry and image cache. A tenant's own allow list (Tenant.spec.images.allow) narrows the rest.

func (*Handler) SetLabGroupAccess

SetLabGroupAccess replaces the one namespaced access-policy CR of every listed LabGroup. The VPN process watches that resource and applies its full default-deny rule set. Client names need not exist yet: the policy is stored at group scope and automatically applies once such a VPN client is created. Lab names must be labs of the group. Result per policy: CREATED, UPDATED, or EXISTS when nothing changed.

func (*Handler) SetLabScheduling

func (h *Handler) SetLabScheduling(selector map[string]string, tolerations []corev1.Toleration)

SetLabScheduling tells the agent which nodes lab pods run on, so a percentage quota resolves against the same allocatable as the operator's scheduler sees.

func (*Handler) SetMonitoringConfig

func (h *Handler) SetMonitoringConfig(cfg MonitoringConfig)

SetMonitoringConfig sets the journal and poller bounds. Call it before the first Monitoring stream; later calls are ignored.

func (*Handler) SetNodeReserve

func (h *Handler) SetNodeReserve(r nodecap.Reserve)

SetNodeReserve sets the platform reserve the scheduler keeps free on the lab nodes, so the largest device the agent reports is the room the scheduler would give a lab.

func (*Handler) SetPackingReserve

func (h *Handler) SetPackingReserve(percent int)

SetPackingReserve sets the hidden packing reserve (percent, 0-49): the capacity reported to tenants is net of it, because nodes do not share memory and devices leave gaps between them, so the last part of a cluster cannot be packed full. A tenant never sees the reserve.

func (*Handler) SetPrewarm

func (h *Handler) SetPrewarm(cfg PrewarmConfig)

func (*Handler) SetRegistryAddr

func (h *Handler) SetRegistryAddr(addr string)

SetRegistryAddr tells the agent where the platform registry (zot) is, as it reaches it (host:port). Snapshot export reads the snapshots from there.

func (*Handler) SetRegistryAuth

func (h *Handler) SetRegistryAuth(user, password string)

SetRegistryAuth gives the agent the reader account of the registry (empty user: anonymous).

func (*Handler) SetRenewMinInterval

func (h *Handler) SetRenewMinInterval(d time.Duration)

SetRenewMinInterval sets how often a tenant may renew its certificate (zero: DefaultRenewMinInterval).

func (*Handler) SetRequiredSnapshotAvailable added in v1.1.0

func (h *Handler) SetRequiredSnapshotAvailable(available bool)

SetRequiredSnapshotAvailable wires the native producer gate after its proof. This enables acceptance checks, not advertised lifecycle capabilities.

func (*Handler) SetStatePersistence

func (h *Handler) SetStatePersistence(enabled bool)

SetStatePersistence tells the agent whether the cluster allows device state persistence; a topology that asks for it otherwise is refused.

func (*Handler) StartLabGroups added in v1.1.0

StartLabGroups starts group services only. Every stopped/terminal Lab is unchanged.

func (*Handler) StartLabs added in v1.1.0

StartLabs never clears terminal intent or changes immutable runtime identity.

func (*Handler) StopLabGroups added in v1.1.0

StopLabGroups accepts intent; it never force-stops children or reports release.

func (*Handler) StopLabs added in v1.1.0

StopLabs accepts durable intent only. Completion requires matching observations.

func (*Handler) UpdateLabGroupClients

func (h *Handler) UpdateLabGroupClients(ctx context.Context, in *protobuf.UpdateLabGroupClientsRequest) (*protobuf.BatchResult, error)

UpdateLabGroupClients changes the labels of the targeted clients.

func (*Handler) UpdateLabGroups

UpdateLabGroups changes labels and the mutable spec fields of the targeted groups.

func (*Handler) UpdateLabs

UpdateLabs changes labels and rewrites device Secrets of the targeted labs. The spec of a lab does not change.

type MonitoringConfig

type MonitoringConfig struct {
	// JournalSize is the most updates the journal keeps (10000).
	JournalSize int
	// JournalAge is how long an update stays in the journal (15m).
	JournalAge time.Duration
	// PollInterval is how often the one shared poller observes the platform (1s).
	// It is the finest resolution of any subscriber's stream.
	PollInterval time.Duration
	// SubscriberBuffer is how many journal updates a subscriber may lag behind
	// before it is dropped (256).
	SubscriberBuffer int
	// MaxStreamsPerTenant is how many Monitoring streams one tenant may hold open at once (0 = unlimited): each subscribe runs a full
	// collect of the cluster and keeps a buffer, so a tenant must not be able to open them without end.
	MaxStreamsPerTenant int
}

MonitoringConfig bounds the shared monitoring journal and poller. Zero fields take the defaults.

type PrewarmConfig

type PrewarmConfig struct {
	// Enabled: the platform image cache exists. Off: PrewarmImages fails with FailedPrecondition.
	Enabled bool
	// RegistryAddr is host:port of the cache (zot) as the agent reaches it.
	RegistryAddr string
	// Registries are the upstream registries the cache serves; images of any other
	// registry are SKIPPED (nodes pull them directly).
	Registries []string
	// Resolver turns a tag into the digest the operator will pin labs to (the
	// platform manifest of a single-architecture cluster, the index otherwise).
	Resolver imagecache.Resolver
	// Concurrency is how many images are fetched at once (4).
	Concurrency int
	// Timeout bounds one image (10m).
	Timeout time.Duration
	// RetryAfter is how long a FAILED image stays failed before a repeated call
	// tries it again, so that a poller sees the failure (30s).
	RetryAfter time.Duration
	// StaleAfter is the age after which a DONE image is checked again when asked
	// for; the check is a manifest request, which also counts as a pull for the
	// cache's unused-image retention (30m).
	StaleAfter time.Duration
	// CredentialedResolver builds the upstream check of an image the request brought
	// credentials for, over a keychain made of those credentials only; nil means a
	// registry probe with that keychain. A field so tests need no registry.
	CredentialedResolver func(kc authn.Keychain) imagecache.Resolver
	// CredTimeout bounds the check of one credentialed image (15s); CredTotalTimeout the
	// whole call's credentialed checks (60s).
	CredTimeout      time.Duration
	CredTotalTimeout time.Duration
	// CredTransport carries the credentialed checks; nil means a transport that refuses
	// loopback, private, link-local and unspecified addresses at dial time. A field so
	// tests can reach an in-process registry.
	CredTransport http.RoundTripper
	// Now is the clock; nil means time.Now.
	Now func() time.Time
}

PrewarmConfig makes the agent able to fill the image cache ahead of time.

type SecretKeychain

type SecretKeychain struct {
	K8s       kubernetes.Interface
	Namespace string
	Names     []string
}

SecretKeychain authenticates requests to upstream registries with dockerconfigjson Secrets of one namespace, read on every use (a rotated credential applies at once). A missing Secret means anonymous access.

func (*SecretKeychain) Resolve

Resolve implements authn.Keychain.

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server is the agent's gRPC front. With mTLS on it is two servers behind one listener: the main one, for connections that presented a client certificate, and the Enroll server for those that did not, which knows the Enroll call only, reads messages of a few kilobytes, and is rate limited. A connection's kind is decided by the TLS handshake before anything is read from it, so nothing an anonymous caller sends is decoded by the main server and no message of megabytes is buffered for it.

func New

func New(cfg *config.Config, impl protobuf.LabManagerServer) (*Server, error)

New builds the server. With mTLS off it refuses to start unless the insecure mode was asked for by name.

func (*Server) GracefulStop

func (s *Server) GracefulStop()

GracefulStop stops accepting connections and lets running calls finish.

func (*Server) Serve

func (s *Server) Serve(lis net.Listener) error

Serve serves until the listener closes or the server stops.

func (*Server) Stop

func (s *Server) Stop()

Stop closes everything at once.

type SplitLimits

type SplitLimits struct {
	// HandshakeTimeout is how long a connection may take to finish the TLS handshake.
	HandshakeTimeout time.Duration
	// MaxHandshakes is how many handshakes may run at once; a connection that finds no room is closed at once.
	MaxHandshakes int
	// MaxAnonymous is how many connections without a client certificate may be open together. There is no per-address limit: behind the
	// gateway's TLS passthrough every caller has the gateway's address, so one would be a global limit that an anonymous host could use
	// to lock out every tenant.
	MaxAnonymous int
}

SplitLimits bound what a caller without a client certificate may cost. Such a caller can reach only the Enroll server (see Server), so these protect the one door that is open to everyone.

func DefaultSplitLimits

func DefaultSplitLimits() SplitLimits

DefaultSplitLimits are what the agent runs with unless the chart says otherwise.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL