Documentation
¶
Index ¶
- Constants
- func HashEnrollmentToken(token string) string
- func NewEnrollmentToken() (token, hash string, err error)
- func NodePlatforms(reader client.Reader, nodeSelector map[string]string) func(context.Context) []v1.Platform
- type ConnectionReconciler
- type DeviceDefaults
- type DeviceReconciler
- type LabGroupClientReconciler
- type LabGroupReconciler
- type LabReconciler
- type PodSecurity
- type PullKeychain
- type RetentionSweeper
- type Scheduler
- type SchedulerConfig
- type SnapshotCatalog
- type SnapshotRegistry
- type StatePolicy
- type TenantReconciler
Constants ¶
const DefaultEnrollmentTTL = 24 * time.Hour
DefaultEnrollmentTTL is how long an unused enrollment token works.
const RetentionConfigMap = "laboratory-snapshot-retention"
RetentionConfigMap holds, per lab whose snapshots outlive it, the time the sweep first saw the lab gone. It lives in the operator namespace.
Variables ¶
This section is empty.
Functions ¶
func HashEnrollmentToken ¶
HashEnrollmentToken is the hex SHA-256 of a token, what the status keeps.
func NewEnrollmentToken ¶
NewEnrollmentToken returns a fresh random token and its hash.
Types ¶
type ConnectionReconciler ¶
type ConnectionReconciler struct {
client.Client
Scheme *runtime.Scheme
Recorder record.EventRecorder
}
ConnectionReconciler reconciles a Connection object. The operator role is minimal: node-agent writes port status; this reconciler aggregates port readiness into Connection.status.ready.
func (*ConnectionReconciler) SetupWithManager ¶
func (r *ConnectionReconciler) SetupWithManager(mgr ctrl.Manager) error
type DeviceDefaults ¶
type DeviceDefaults struct {
CPU string
Memory string
// MaxCPU and MaxMemory are the chart maxima of one device (limits.device.max*): a larger value of an existing
// object is clamped to them. Empty or "0" = no maximum beyond the absolute bound of the limits package.
MaxCPU string
MaxMemory string
}
DeviceDefaults are the CPU and memory of a device container that declares none. Empty strings leave the resource unset.
type DeviceReconciler ¶
type DeviceReconciler struct {
client.Client
Scheme *runtime.Scheme
LabNodeSelector map[string]string
LabTolerations []corev1.Toleration
// PriorityClass is the class of the device pods (the lowest; it never preempts). Empty = none.
PriorityClass string
// SchedulerName is the kube-scheduler profile of the device pods (bin-packing); empty = the default scheduler.
SchedulerName string
// TerminationGraceSeconds is the terminationGracePeriodSeconds of the device pods; 0 = the Kubernetes default.
TerminationGraceSeconds int64
// NetConfigImage is the image used for the optional init-container that
// assigns static IP/routes inside a device pod. Must contain `ip` (iproute2)
// and `sh`. Empty disables static addressing via init-container.
NetConfigImage string
// Defaults are the CPU and memory of a device container that declares none.
Defaults DeviceDefaults
// Registry is the snapshot registry, nil when state persistence is off; it
// is used to drop a device's snapshots on reset.
Registry SnapshotRegistry
// Reader reads from the API server without the cache; nil means Client. Used
// where a stale cache would start a device twice.
Reader client.Reader
// MirrorRegistries are the upstream registries the image cache serves; a
// device whose spec names a cache prefix (Spec.ImageMirror) pulls its images
// through it.
MirrorRegistries []string
// ExitSnapshotTimeout is how long a finished pod waits for the node-agent's
// exit snapshot; zero means 30s.
ExitSnapshotTimeout time.Duration
// NodeLossForceDeleteAfter is how long a snapshot-backed pod may stay Terminating on a NotReady
// or missing node before it is force-deleted; zero means 5m, negative never.
NodeLossForceDeleteAfter time.Duration
// Now is the clock; nil means time.Now. A field so tests can move time.
Now func() time.Time
// Security is the hardening of the device pods (capabilities, user namespaces, ephemeral storage).
Security PodSecurity
// Scheduled makes the device wait for the scheduler to dispatch its pod
// (see device_sched.go). Off: the pod is created as soon as the device is.
Scheduled bool
}
DeviceReconciler reconciles a Device object.
func (*DeviceReconciler) SetupWithManager ¶
func (r *DeviceReconciler) SetupWithManager(mgr ctrl.Manager) error
type LabGroupClientReconciler ¶
type LabGroupClientReconciler struct {
client.Client
Scheme *runtime.Scheme
Recorder record.EventRecorder
// VPNBaseNetwork is the full VPN address space advertised to WireGuard clients (e.g. "10.8.0.0/10").
VPNBaseNetwork string
}
LabGroupClientReconciler reconciles a LabGroupClient object.
func (*LabGroupClientReconciler) SetupWithManager ¶
func (r *LabGroupClientReconciler) SetupWithManager(mgr ctrl.Manager) error
type LabGroupReconciler ¶
type LabGroupReconciler struct {
client.Client
Scheme *runtime.Scheme
Recorder record.EventRecorder
// PublicVPNEndpoint is the publicly reachable host:port that clients dial
// (host of the WireGuard demux). Written verbatim to LabGroup.Status.VPN.Endpoint.
PublicVPNEndpoint string
// VPNBaseNetwork is the base address space for all VPN subnets (e.g. "10.8.0.0/10").
VPNBaseNetwork string
// InetBaseNetwork is the base address space for per-lab internet/gateway subnets (e.g. "10.9.0.0/10").
InetBaseNetwork string
// VPNImage is the container image for VPN pods.
VPNImage string
// GatewayImage is the container image for gateway pods.
GatewayImage string
// GroupPods are the resources of the VPN and gateway pods of a NEW group (requests = limits);
// the Deployments of existing groups are left as they are, so a live event keeps its VPN.
GroupPods grouppods.Config
// PriorityClass is the class of the VPN and gateway pods (above the devices). Empty = none.
PriorityClass string
// SchedulerName is the kube-scheduler profile of the VPN and gateway pods (bin-packing); empty = the default scheduler.
SchedulerName string
// TerminationGraceSeconds is the terminationGracePeriodSeconds of the VPN and gateway pods; 0 = the Kubernetes default.
TerminationGraceSeconds int64
// Mirror rewrites the VPN and gateway images for the image cache when their
// pods are created; the zero value (cache off) rewrites nothing.
Mirror imagecache.Rewriter
// Resolver pins those images to a digest, once, at creation; nil pins nothing.
Resolver imagecache.Resolver
// SupportEmail is the contact address shown on the VPN probe page.
SupportEmail string
// LabNodeSelector is applied to VPN and gateway pod specs.
LabNodeSelector map[string]string
// LabTolerations is applied to VPN and gateway pod specs.
LabTolerations []corev1.Toleration
// AgentEnabled gates creation of the management-agent RoleBinding in each
// group namespace. AgentSA identifies the agent ServiceAccount to bind.
AgentEnabled bool
AgentSA types.NamespacedName
// ProxyEnabled gates the RoleBinding of the L7 proxy's traffic reports in each group namespace.
ProxyEnabled bool
// NetworkPolicyEnabled gates creation of the default-deny NetworkPolicy
// baseline in each group namespace.
NetworkPolicyEnabled bool
// OperatorSA is the operator's own ServiceAccount: it is bound to the working role in each group namespace
// (empty: no binding, for tests).
OperatorSA types.NamespacedName
// VPNStatsInterval is STATS_INTERVAL of the VPN pod of a new group; zero leaves the pod's own default.
VPNStatsInterval time.Duration
// ImagePullSecrets names registry Secrets of the operator namespace. They are
// copied into every group namespace and referenced by the VPN and gateway pods.
ImagePullSecrets []string
// Scheduled makes the group's VPN and gateway pods wait for the scheduler to
// dispatch them (see labgroup_sched.go). Off: they start as soon as the group is created.
Scheduled bool
// contains filtered or unexported fields
}
LabGroupReconciler reconciles a LabGroup object.
func (*LabGroupReconciler) SetupWithManager ¶
func (r *LabGroupReconciler) SetupWithManager(mgr ctrl.Manager) error
type LabReconciler ¶
type LabReconciler struct {
client.Client
Scheme *runtime.Scheme
Recorder record.EventRecorder
// BaseDomain is the public DNS suffix under which task URLs are advertised,
// e.g. "challenges.cybericebox.com". An exposed device named "web" inside a
// lab is reachable as https://web-<code>.<BaseDomain> (names.WebHostLabel; code is 3-4 random
// base36 chars fixed by the name of the device's Service). Written to
// Lab.Status.Access on Ready.
BaseDomain string
// ProxySourceCIDRs is an optional list of CIDRs added as ipBlock peers in the
// web-exposure NetworkPolicy. Required when the proxy runs with hostNetwork (its
// source IP is the node IP, not a pod IP, so namespace/label selectors don't apply).
ProxySourceCIDRs []string
// VPNBaseNetwork is the base address space for per-lab VPN subnets (e.g. "10.8.0.0/16").
VPNBaseNetwork string
// InetBaseNetwork is the base address space for per-lab internet/gateway subnets (e.g. "10.9.0.0/16").
InetBaseNetwork string
// Reader reads from the API server without the cache; nil means Client. Used
// where a stale cache would hand out a device code twice.
Reader client.Reader
// State is the device state persistence policy applied to labs created
// while the platform switch is on.
State StatePolicy
// Mirror rewrites image references for the image cache; the zero value
// (cache off) rewrites nothing. The Lab records the decision once.
Mirror imagecache.Rewriter
// Resolver pins image tags to digests for a lab created with the image
// cache on; nil pins nothing.
Resolver imagecache.Resolver
// NetConfigImage is the image of the device netconfig init-container, pinned
// together with the device images.
NetConfigImage string
// contains filtered or unexported fields
}
LabReconciler reconciles a Lab object.
func (*LabReconciler) SetupWithManager ¶
func (r *LabReconciler) SetupWithManager(mgr ctrl.Manager) error
type PodSecurity ¶
type PodSecurity struct {
// UserNamespaces runs device pods in their own user namespace (hostUsers: false), so root in a device is not
// root on the node. Needs Kubernetes 1.33+ and a runtime that supports it (containerd 2.x, kernel 6.3+).
UserNamespaces bool
// EphemeralStorage is the limit (and request) of a device's writable layer, logs and emptyDirs; empty: none.
EphemeralStorage string
}
PodSecurity is the hardening of the device pods the operator creates (the VPN and gateway pods have a fixed, minimal set, see hardenGroupPod).
type PullKeychain ¶
PullKeychain authenticates requests to upstream registries with the operator's image pull Secrets (kubernetes.io/dockerconfigjson in Namespace). The Secrets are read on every resolution, so a rotated credential is used at once.
func (*PullKeychain) Resolve ¶
func (k *PullKeychain) Resolve(res authn.Resource) (authn.Authenticator, error)
Resolve implements authn.Keychain. A missing or unreadable Secret means anonymous access, never a failure: public images need no credentials.
type RetentionSweeper ¶
type RetentionSweeper struct {
Client client.Client
// Reader reads Labs and the ConfigMap directly from the API server: the
// operator's cache does not watch ConfigMaps cluster-wide. Nil means Client.
Reader client.Reader
Registry SnapshotCatalog
Retention time.Duration
Interval time.Duration
Namespace string
// Now is the clock; nil means time.Now.
Now func() time.Time
}
RetentionSweeper deletes the snapshots of labs that are gone once Retention has passed since the sweep first noticed the lab missing. The registry's garbage collection then frees the blobs (see the chart values statePersistence.registry.gc). A lab that reappears under the same name before the deadline cancels the deletion.
func (*RetentionSweeper) NeedLeaderElection ¶
func (s *RetentionSweeper) NeedLeaderElection() bool
NeedLeaderElection makes the sweep run on the elected operator only.
type Scheduler ¶
type Scheduler struct {
client.Client
Recorder record.EventRecorder
Config SchedulerConfig
// Defaults are the device resources a device without any gets; the same the
// DeviceReconciler applies, so the resource check sees the real load.
Defaults DeviceDefaults
// GroupPods are the resources of the VPN and gateway pods of a group; the scheduler counts them
// against the node room and the tenant's quota like any pod.
GroupPods grouppods.Config
// LabNodeSelector and LabTolerations pick the nodes the prepull asks and the node
// set of the resource check, like they shape the lab pods.
LabNodeSelector map[string]string
LabTolerations []corev1.Toleration
// Mirror rewrites image references of labs that use the image cache, so the
// prepull warms the cache through the path the lab pods will use.
Mirror imagecache.Rewriter
// ImagesNamespace holds the credentials of a prepull request; empty means names.ImagesNamespace.
ImagesNamespace string
// Now is the clock; nil means time.Now.
Now func() time.Time
// contains filtered or unexported fields
}
Scheduler dispatches the pods of Labs and LabGroups through a conveyor so a burst of objects does not start every pod at once. See planSchedule for the order and DEPLOY.md, "Scheduler". It is the only writer of the scheduling state of queued pods and runs on the leader.
func (*Scheduler) NeedLeaderElection ¶
NeedLeaderElection makes the manager run the scheduler on the leader only.
type SchedulerConfig ¶
type SchedulerConfig struct {
// MaxPods is the largest number of pods starting at once; 0 sets no limit.
MaxPods int
// StartupTimeout is how long a dispatched pod may take to become Ready.
StartupTimeout time.Duration
// RestartThreshold is the restart count that fails a pod that is not Ready.
RestartThreshold int
// PlatformReservePercent of the schedulable CPU and memory (allocatable, so after the kubelet's reserve,
// and after the requests of DaemonSets and the proxy) is never given to user labs.
PlatformReservePercent int
// PlatformReserveNode is an extra absolute reserve on every schedulable node.
PlatformReserveNode corev1.ResourceList
// ResourceCheck holds a pod back while no node has room for it.
ResourceCheck bool
// Prepull pulls the images of a group onto the nodes before its first pod starts.
Prepull bool
// PrepullTimeout bounds the wait for a prepull; dispatch goes on after it.
PrepullTimeout time.Duration
// Tick is how often the queue is processed; zero means defaultSchedulerTick.
Tick time.Duration
// StatusBudget is the largest number of queue status writes per tick; zero
// means defaultStatusBudget. It keeps a long queue from flooding the API server.
StatusBudget int
// StatusInterval is the least time between two queue status writes of one
// object; zero means defaultStatusInterval.
StatusInterval time.Duration
}
SchedulerConfig is the scheduler policy.
type SnapshotCatalog ¶
type SnapshotCatalog interface {
Repos(ctx context.Context) ([]string, error)
DeleteRepo(ctx context.Context, repo string) error
}
SnapshotCatalog is the registry side of the retention sweep.
type SnapshotRegistry ¶
SnapshotRegistry is the part of the snapshot registry the controller needs: dropping a device's snapshots on reset.
type StatePolicy ¶
type StatePolicy struct {
Enabled bool
Debounce time.Duration
ExcludePaths []string
WriteQuotaBytes int64
MaxFileBytes int64
MaxEntries int32
TenantQuota int64
MaxLayers int32
}
StatePolicy is the device state persistence configuration of the operator. It is copied onto the Devices of a Lab when they are created and never re-applied, so changing it affects only labs created afterwards.
func SetupState ¶
func SetupState(c operator.StateConfig) (StatePolicy, *snapshot.Registry, error)
SetupState turns the operator's state persistence configuration into the policy the Lab controller copies onto new devices and the snapshot registry client (nil when persistence is off or no registry is configured).
type TenantReconciler ¶
type TenantReconciler struct {
client.Client
Scheme *runtime.Scheme
// TTL is the lifetime of a token; zero means DefaultEnrollmentTTL.
TTL time.Duration
// Now is the clock; nil means time.Now.
Now func() time.Time
}
TenantReconciler gives every Tenant a one-time enrollment token. The token is random; only its hash and expiry go to Tenant.status.enrollment, and the token itself is shown once, in the Secret tenant-<name>-enrollment of the tenants namespace (the admin reads it with kubectl). The agent burns the token when a client enrolls (status.enrollment.usedAt), and the Secret goes away. An annotation asks for a new token.
func (*TenantReconciler) SetupWithManager ¶
func (r *TenantReconciler) SetupWithManager(mgr ctrl.Manager) error
Source Files
¶
- address_refs.go
- connection_controller.go
- consts.go
- device_code.go
- device_controller.go
- device_sched.go
- devicestate.go
- lab_controller.go
- labgroup_controller.go
- labgroup_sched.go
- labgroupclient_controller.go
- labstate.go
- nodeplatforms.go
- podsecurity.go
- pullkeychain.go
- pullsecrets.go
- resources.go
- retention.go
- rollout.go
- sched_capacity.go
- sched_plan.go
- sched_prepull.go
- scheduler.go
- statesetup.go
- tenant_enroll.go
- userlabels.go