controller

package
v0.1.29 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 12, 2026 License: Apache-2.0 Imports: 49 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// ClusterSizerDeploymentName is the Deployment the sizer resizes: the
	// controller's own. The RBAC Role grants get/patch on this name only.
	ClusterSizerDeploymentName = "devzero-zxporter-controller-manager"
	// ClusterSizerAnnotation holds the sizing ladder as JSON. Its absence
	// disables the sizer, so Helm opts in simply by rendering it.
	ClusterSizerAnnotation = "devzero.io/cluster-sizer"
)

Variables

This section is empty.

Functions

This section is empty.

Types

type ClusterSize added in v0.1.29

type ClusterSize struct {
	Nodes int
	Cores int
	Fit   corev1.ResourceList
	// Eligible holds the allocatable of every node the pod could land on, so
	// the resized pod can be checked against one real node rather than
	// against Fit, whose per-resource maxima may come from different nodes.
	Eligible []corev1.ResourceList
}

ClusterSize is what the ladder is evaluated against: every node, schedulable or not, and the sum of their CPU capacity in whole cores. Fit is separate: the largest allocatable per resource among the nodes the sized pod could actually land on, which bounds what a request may ask for.

type ClusterSizer added in v0.1.29

type ClusterSizer struct {
	// contains filtered or unexported fields
}

ClusterSizer keeps the controller's own resources proportional to cluster size. It scales up as soon as the ladder asks for more, and scales down only after the lower value has held for the configured delay, so a cluster that bounces across a step boundary does not restart zxporter on every tick.

func NewClusterSizer added in v0.1.29

func NewClusterSizer(
	c client.Client,
	apiReader client.Reader,
	namespace string,
	healthManager *health.HealthManager,
) *ClusterSizer

NewClusterSizer builds the sizer. apiReader must be uncached (mgr.GetAPIReader()): nodes are listed directly each tick so the sizer adds no node informer to the very process whose memory it is sizing.

func (*ClusterSizer) NeedLeaderElection added in v0.1.29

func (s *ClusterSizer) NeedLeaderElection() bool

NeedLeaderElection makes the sizer run only on the elected manager.

func (*ClusterSizer) ReconcileOnce added in v0.1.29

func (s *ClusterSizer) ReconcileOnce(ctx context.Context) (time.Duration, error)

ReconcileOnce sizes the Deployment once and returns how long to wait before the next pass.

func (*ClusterSizer) Start added in v0.1.29

func (s *ClusterSizer) Start(ctx context.Context) error

Start implements the controller-manager Runnable interface. The interval is re-read from the annotation after every pass, so a changed pollInterval takes effect without a restart.

type ClusterSizerConfig added in v0.1.29

type ClusterSizerConfig struct {
	PollInterval   string `json:"pollInterval,omitempty"`
	ScaleDownDelay string `json:"scaleDownDelay,omitempty"`
	// MaxNodePercent caps every sized request at this share of the largest
	// allocatable among nodes the pod can schedule on, so a resize never asks
	// for more than any node can give and stalls the rollout. Default 50.
	MaxNodePercent int                             `json:"maxNodePercent,omitempty"`
	Containers     map[string]ContainerSizerConfig `json:"containers"`
}

ClusterSizerConfig is the annotation payload: a ladder per container and resource, plus the shared timing and node-fit settings.

type CollectionPolicyReconciler

type CollectionPolicyReconciler struct {
	client.Client
	Scheme            *runtime.Scheme
	Log               logr.Logger
	KEDAClient        *kedaclient.Clientset
	K8sClient         *kubernetes.Clientset
	DynamicClient     *dynamic.DynamicClient
	MetadataClient    metadata.Interface
	DiscoveryClient   *discovery.DiscoveryClient
	DakrClient        transport.DakrClient
	ApiExtensions     *apiextensionsclientset.Clientset
	CollectionManager *collector.CollectionManager
	MpaServer         *server.MpaServer
	Sender            transport.DirectSender
	TelemetrySender   *transport.TelemetrySender
	TelemetryMetrics  *collector.TelemetryMetrics
	TelemetryLogger   telemetry_logger.Logger
	ZapLogger         *zap.Logger
	IsRunning         bool
	CurrentPolicyHash string
	CurrentConfig     *PolicyConfig
	RestartInProgress bool
	MpaServerPort     int
	HealthManager     *health.HealthManager
	// contains filtered or unexported fields
}

CollectionPolicyReconciler reconciles a CollectionPolicy object

func (*CollectionPolicyReconciler) Reconcile

func (r *CollectionPolicyReconciler) Reconcile(
	ctx context.Context,
	req ctrl.Request,
) (ctrl.Result, error)

Reconcile is part of the main kubernetes reconciliation loop which aims to move the current state of the cluster closer to the desired state.

func (*CollectionPolicyReconciler) SetupWithManager

func (r *CollectionPolicyReconciler) SetupWithManager(mgr ctrl.Manager) error

SetupWithManager sets up the controller with the Manager.

type ContainerSizerConfig added in v0.1.29

type ContainerSizerConfig struct {
	Requests map[corev1.ResourceName]ResourceLadder `json:"requests,omitempty"`
	Limits   map[corev1.ResourceName]ResourceLadder `json:"limits,omitempty"`
}

ContainerSizerConfig maps resource names (memory, cpu) to their ladder, for requests and limits independently. A resource that is not listed is never touched.

type EnvBasedController

type EnvBasedController struct {
	client.Client
	Scheme          *runtime.Scheme
	Log             logr.Logger
	K8sClient       kubernetes.Interface
	DynamicClient   *dynamic.DynamicClient
	DiscoveryClient *discovery.DiscoveryClient
	ApiExtensions   *apiextensionsclientset.Clientset
	Reconciler      *CollectionPolicyReconciler
	// contains filtered or unexported fields
}

EnvBasedController is a controller that uses environment variables instead of CRDs

func NewEnvBasedController

func NewEnvBasedController(
	mgr ctrl.Manager,
	healthManager *health.HealthManager,
	reconcileInterval time.Duration,
	mpaServerPort int,
) (*EnvBasedController, error)

NewEnvBasedController creates a new environment-based controller

func (*EnvBasedController) NeedLeaderElection

func (c *EnvBasedController) NeedLeaderElection() bool

NeedLeaderElection implements the LeaderElectionRunnable interface

func (*EnvBasedController) Start

func (c *EnvBasedController) Start(ctx context.Context) error

Start implements the Runnable interface for manager.Add

type GPURuntimeResolver added in v0.1.0

type GPURuntimeResolver struct {
	// contains filtered or unexported fields
}

GPURuntimeResolver keeps the managed GPU DaemonSet's RuntimeClass in sync with the RuntimeClass API when the DaemonSet opts into auto mode.

func NewGPURuntimeResolver added in v0.1.0

func NewGPURuntimeResolver(
	c client.Client,
	apiReader client.Reader,
	namespace string,
	interval time.Duration,
	healthManager *health.HealthManager,
) *GPURuntimeResolver

NewGPURuntimeResolver builds the resolver. apiReader must be an uncached reader (mgr.GetAPIReader()): the RuntimeClass read is served by a direct GET, which the narrow get-on-resourceName RBAC allows. The manager's cached client would instead back the read with a cluster-wide LIST+WATCH informer, which the resolver's RBAC intentionally forbids, so a cached read blocks forever on a cache that can never sync. client is used for the DaemonSet patch (and its already-permitted cached read).

func (*GPURuntimeResolver) NeedLeaderElection added in v0.1.0

func (r *GPURuntimeResolver) NeedLeaderElection() bool

NeedLeaderElection makes the resolver run only on the elected manager.

func (*GPURuntimeResolver) ReconcileOnce added in v0.1.0

func (r *GPURuntimeResolver) ReconcileOnce(ctx context.Context) error

ReconcileOnce resolves and, when necessary, merge-patches the managed GPU DaemonSet. Conflicts retry the entire transaction so every attempt computes desired state from a fresh DaemonSet read.

func (*GPURuntimeResolver) Start added in v0.1.0

func (r *GPURuntimeResolver) Start(ctx context.Context) error

Start implements the controller-manager Runnable interface.

type PolicyConfig

type PolicyConfig struct {
	TargetNamespaces                   []string
	ExcludedNamespaces                 []string
	ExcludedPods                       []collector.ExcludedPod
	ExcludedDeployments                []collector.ExcludedDeployment
	ExcludedStatefulSets               []collector.ExcludedStatefulSet
	ExcludedDaemonSets                 []collector.ExcludedDaemonSet
	ExcludedServices                   []collector.ExcludedService
	ExcludedPVCs                       []collector.ExcludedPVC
	ExcludedEvents                     []collector.ExcludedEvent
	ExcludedJobs                       []collector.ExcludedJob
	ExcludedCronJobs                   []collector.ExcludedCronJob
	ExcludedReplicationControllers     []collector.ExcludedReplicationController
	ExcludedIngresses                  []collector.ExcludedIngress
	ExcludedNetworkPolicies            []collector.ExcludedNetworkPolicy
	ExcludedEndpoints                  []collector.ExcludedEndpoint
	ExcludedServiceAccounts            []collector.ExcludedServiceAccount
	ExcludedLimitRanges                []collector.ExcludedLimitRange
	ExcludedResourceQuotas             []collector.ExcludedResourceQuota
	ExcludedHPAs                       []collector.ExcludedHPA
	ExcludedVPAs                       []collector.ExcludedVPA
	ExcludedRoles                      []collector.ExcludedRole
	ExcludedRoleBindings               []collector.ExcludedRoleBinding
	ExcludedClusterRoles               []string
	ExcludedClusterRoleBindings        []string
	ExcludedPDBs                       []collector.ExcludedPDB
	ExcludedPSPs                       []string
	ExcludedCRDs                       []string
	ExcludedReplicaSet                 []collector.ExcludedReplicaSet
	ExcludedStorageClasses             []string
	ExcludedPVs                        []string
	ExcludedIngressClasses             []string
	ExcludedNodes                      []string
	ExcludedCRDGroups                  []string
	WatchedCRDs                        []string
	ExcludedCSINodes                   []string
	ExcludedDatadogReplicaSets         []collector.ExcludedDatadogExtendedDaemonSetReplicaSet
	ExcludedArgoRollouts               []collector.ExcludedArgoRollout
	ExcludedKedaScaledJobs             []collector.ExcludedScaledJob
	ExcludedKedaScaledObjects          []collector.ExcludedScaledObject
	ExcludedCSIDrivers                 []string
	ExcludedCSIStorageCapacities       []collector.ExcludedCSIStorageCapacity
	ExcludedVolumeAttachments          []string
	ExcludedKubeflowNotebooks          []collector.ExcludedKubeflowNotebook
	ExcludedVolcanoJobs                []collector.ExcludedVolcanoJob
	ExcludedSparkApplications          []collector.ExcludedSparkApplication
	ExcludedScheduledSparkApplications []collector.ExcludedScheduledSparkApplication
	ExcludedCNPGClusters               []collector.ExcludedCNPGCluster
	ExcludedArgoCDApplications         []collector.ExcludedArgoCDApplication
	ExcludedArgoCDApplicationSets      []collector.ExcludedArgoCDApplicationSet
	ExcludedFlaggerCanaries            []collector.ExcludedFlaggerCanary
	ExcludedFlaggerMetricTemplates     []collector.ExcludedFlaggerMetricTemplate
	ExcludedFlaggerAlertProviders      []collector.ExcludedFlaggerAlertProvider
	ExcludedRuntimeClasses             []string

	DisabledCollectors []string

	// MigPartedConfigMapName/Namespace locate the NVIDIA mig-parted ConfigMap;
	// empty values fall back to the GPU operator defaults.
	MigPartedConfigMapName      string
	MigPartedConfigMapNamespace string

	KubeContextName         string
	DakrURL                 string
	ClusterToken            string
	DisableGPUMetrics       bool
	UpdateInterval          time.Duration
	NodeMetricsInterval     time.Duration
	ClusterSnapshotInterval time.Duration
	BufferSize              int
	MaskSecretData          bool
	// DisableEnvVarCollection removes environment variables from every object
	// before it is sent to Dakr (ENV_VAR_COLLECTION=disabled).
	DisableEnvVarCollection bool
	NumResourceProcessors   int
	NodeMetricsConcurrency  int
	NodemonRequestTimeout   time.Duration
	KubeletFallbackTimeout  time.Duration

	// SchedulerMetricsScrapeInterval bounds how often the SchedulerMetrics
	// collector scrapes dz-scheduler's /metrics endpoint. See
	// collector.DefaultSchedulerMetricsScrapeInterval's doc comment for the
	// default's reasoning.
	SchedulerMetricsScrapeInterval time.Duration

	// KEDAMetricsEndpoint is the full URL of the KEDA operator /metrics endpoint
	// scraped by the KedaScalerMetrics collector. Empty means the collector
	// applies collector.DefaultKedaMetricsEndpoint (KEDA in devzero-system).
	KEDAMetricsEndpoint string
}

PolicyConfig holds the current configuration

func (PolicyConfig) String added in v0.1.29

func (c PolicyConfig) String() string

String keeps ClusterToken out of logs and telemetry: the config is dumped with %v/%+v on every (re)initialisation and config change.

type ResourceLadder added in v0.1.29

type ResourceLadder struct {
	Base         *resource.Quantity `json:"base,omitempty"`
	Step         *resource.Quantity `json:"step,omitempty"`
	Max          *resource.Quantity `json:"max,omitempty"`
	NodesPerStep int                `json:"nodesPerStep,omitempty"`
	CoresPerStep int                `json:"coresPerStep,omitempty"`
}

ResourceLadder is one scaling function:

want = min(max, base + step * max(ceil(nodes/nodesPerStep), ceil(cores/coresPerStep)))

A zero NodesPerStep or CoresPerStep disables that term, and a nil Max means no cap.

func (ResourceLadder) Calculate added in v0.1.29

Calculate evaluates the ladder for one resource. Arithmetic is in milli-units so CPU steps below one core work; the result is formatted the way that resource is conventionally written (768Mi, 250m).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL