Documentation
¶
Index ¶
Constants ¶
const ( // ClusterSizerDeploymentName is the Deployment the sizer resizes: the // controller's own. The RBAC Role grants get/patch on this name only. ClusterSizerDeploymentName = "devzero-zxporter-controller-manager" // ClusterSizerAnnotation holds the sizing ladder as JSON. Its absence // disables the sizer, so Helm opts in simply by rendering it. ClusterSizerAnnotation = "devzero.io/cluster-sizer" )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ClusterSize ¶ added in v0.1.29
type ClusterSize struct {
Nodes int
Cores int
Fit corev1.ResourceList
// Eligible holds the allocatable of every node the pod could land on, so
// the resized pod can be checked against one real node rather than
// against Fit, whose per-resource maxima may come from different nodes.
Eligible []corev1.ResourceList
}
ClusterSize is what the ladder is evaluated against: every node, schedulable or not, and the sum of their CPU capacity in whole cores. Fit is separate: the largest allocatable per resource among the nodes the sized pod could actually land on, which bounds what a request may ask for.
type ClusterSizer ¶ added in v0.1.29
type ClusterSizer struct {
// contains filtered or unexported fields
}
ClusterSizer keeps the controller's own resources proportional to cluster size. It scales up as soon as the ladder asks for more, and scales down only after the lower value has held for the configured delay, so a cluster that bounces across a step boundary does not restart zxporter on every tick.
func NewClusterSizer ¶ added in v0.1.29
func NewClusterSizer( c client.Client, apiReader client.Reader, namespace string, healthManager *health.HealthManager, ) *ClusterSizer
NewClusterSizer builds the sizer. apiReader must be uncached (mgr.GetAPIReader()): nodes are listed directly each tick so the sizer adds no node informer to the very process whose memory it is sizing.
func (*ClusterSizer) NeedLeaderElection ¶ added in v0.1.29
func (s *ClusterSizer) NeedLeaderElection() bool
NeedLeaderElection makes the sizer run only on the elected manager.
func (*ClusterSizer) ReconcileOnce ¶ added in v0.1.29
ReconcileOnce sizes the Deployment once and returns how long to wait before the next pass.
type ClusterSizerConfig ¶ added in v0.1.29
type ClusterSizerConfig struct {
PollInterval string `json:"pollInterval,omitempty"`
ScaleDownDelay string `json:"scaleDownDelay,omitempty"`
// MaxNodePercent caps every sized request at this share of the largest
// allocatable among nodes the pod can schedule on, so a resize never asks
// for more than any node can give and stalls the rollout. Default 50.
MaxNodePercent int `json:"maxNodePercent,omitempty"`
Containers map[string]ContainerSizerConfig `json:"containers"`
}
ClusterSizerConfig is the annotation payload: a ladder per container and resource, plus the shared timing and node-fit settings.
type CollectionPolicyReconciler ¶
type CollectionPolicyReconciler struct {
client.Client
Scheme *runtime.Scheme
Log logr.Logger
KEDAClient *kedaclient.Clientset
K8sClient *kubernetes.Clientset
DynamicClient *dynamic.DynamicClient
MetadataClient metadata.Interface
DiscoveryClient *discovery.DiscoveryClient
DakrClient transport.DakrClient
ApiExtensions *apiextensionsclientset.Clientset
CollectionManager *collector.CollectionManager
MpaServer *server.MpaServer
Sender transport.DirectSender
TelemetrySender *transport.TelemetrySender
TelemetryMetrics *collector.TelemetryMetrics
TelemetryLogger telemetry_logger.Logger
ZapLogger *zap.Logger
IsRunning bool
CurrentPolicyHash string
CurrentConfig *PolicyConfig
RestartInProgress bool
MpaServerPort int
HealthManager *health.HealthManager
// contains filtered or unexported fields
}
CollectionPolicyReconciler reconciles a CollectionPolicy object
func (*CollectionPolicyReconciler) Reconcile ¶
func (r *CollectionPolicyReconciler) Reconcile( ctx context.Context, req ctrl.Request, ) (ctrl.Result, error)
Reconcile is part of the main kubernetes reconciliation loop which aims to move the current state of the cluster closer to the desired state.
func (*CollectionPolicyReconciler) SetupWithManager ¶
func (r *CollectionPolicyReconciler) SetupWithManager(mgr ctrl.Manager) error
SetupWithManager sets up the controller with the Manager.
type ContainerSizerConfig ¶ added in v0.1.29
type ContainerSizerConfig struct {
Requests map[corev1.ResourceName]ResourceLadder `json:"requests,omitempty"`
Limits map[corev1.ResourceName]ResourceLadder `json:"limits,omitempty"`
}
ContainerSizerConfig maps resource names (memory, cpu) to their ladder, for requests and limits independently. A resource that is not listed is never touched.
type EnvBasedController ¶
type EnvBasedController struct {
client.Client
Scheme *runtime.Scheme
Log logr.Logger
K8sClient kubernetes.Interface
DynamicClient *dynamic.DynamicClient
DiscoveryClient *discovery.DiscoveryClient
ApiExtensions *apiextensionsclientset.Clientset
Reconciler *CollectionPolicyReconciler
// contains filtered or unexported fields
}
EnvBasedController is a controller that uses environment variables instead of CRDs
func NewEnvBasedController ¶
func NewEnvBasedController( mgr ctrl.Manager, healthManager *health.HealthManager, reconcileInterval time.Duration, mpaServerPort int, ) (*EnvBasedController, error)
NewEnvBasedController creates a new environment-based controller
func (*EnvBasedController) NeedLeaderElection ¶
func (c *EnvBasedController) NeedLeaderElection() bool
NeedLeaderElection implements the LeaderElectionRunnable interface
type GPURuntimeResolver ¶ added in v0.1.0
type GPURuntimeResolver struct {
// contains filtered or unexported fields
}
GPURuntimeResolver keeps the managed GPU DaemonSet's RuntimeClass in sync with the RuntimeClass API when the DaemonSet opts into auto mode.
func NewGPURuntimeResolver ¶ added in v0.1.0
func NewGPURuntimeResolver( c client.Client, apiReader client.Reader, namespace string, interval time.Duration, healthManager *health.HealthManager, ) *GPURuntimeResolver
NewGPURuntimeResolver builds the resolver. apiReader must be an uncached reader (mgr.GetAPIReader()): the RuntimeClass read is served by a direct GET, which the narrow get-on-resourceName RBAC allows. The manager's cached client would instead back the read with a cluster-wide LIST+WATCH informer, which the resolver's RBAC intentionally forbids, so a cached read blocks forever on a cache that can never sync. client is used for the DaemonSet patch (and its already-permitted cached read).
func (*GPURuntimeResolver) NeedLeaderElection ¶ added in v0.1.0
func (r *GPURuntimeResolver) NeedLeaderElection() bool
NeedLeaderElection makes the resolver run only on the elected manager.
func (*GPURuntimeResolver) ReconcileOnce ¶ added in v0.1.0
func (r *GPURuntimeResolver) ReconcileOnce(ctx context.Context) error
ReconcileOnce resolves and, when necessary, merge-patches the managed GPU DaemonSet. Conflicts retry the entire transaction so every attempt computes desired state from a fresh DaemonSet read.
type PolicyConfig ¶
type PolicyConfig struct {
TargetNamespaces []string
ExcludedNamespaces []string
ExcludedPods []collector.ExcludedPod
ExcludedDeployments []collector.ExcludedDeployment
ExcludedStatefulSets []collector.ExcludedStatefulSet
ExcludedDaemonSets []collector.ExcludedDaemonSet
ExcludedServices []collector.ExcludedService
ExcludedPVCs []collector.ExcludedPVC
ExcludedEvents []collector.ExcludedEvent
ExcludedJobs []collector.ExcludedJob
ExcludedCronJobs []collector.ExcludedCronJob
ExcludedReplicationControllers []collector.ExcludedReplicationController
ExcludedIngresses []collector.ExcludedIngress
ExcludedNetworkPolicies []collector.ExcludedNetworkPolicy
ExcludedEndpoints []collector.ExcludedEndpoint
ExcludedServiceAccounts []collector.ExcludedServiceAccount
ExcludedLimitRanges []collector.ExcludedLimitRange
ExcludedResourceQuotas []collector.ExcludedResourceQuota
ExcludedHPAs []collector.ExcludedHPA
ExcludedVPAs []collector.ExcludedVPA
ExcludedRoles []collector.ExcludedRole
ExcludedRoleBindings []collector.ExcludedRoleBinding
ExcludedClusterRoles []string
ExcludedClusterRoleBindings []string
ExcludedPDBs []collector.ExcludedPDB
ExcludedPSPs []string
ExcludedCRDs []string
ExcludedReplicaSet []collector.ExcludedReplicaSet
ExcludedStorageClasses []string
ExcludedPVs []string
ExcludedIngressClasses []string
ExcludedNodes []string
ExcludedCRDGroups []string
WatchedCRDs []string
ExcludedCSINodes []string
ExcludedDatadogReplicaSets []collector.ExcludedDatadogExtendedDaemonSetReplicaSet
ExcludedArgoRollouts []collector.ExcludedArgoRollout
ExcludedKedaScaledJobs []collector.ExcludedScaledJob
ExcludedKedaScaledObjects []collector.ExcludedScaledObject
ExcludedCSIDrivers []string
ExcludedCSIStorageCapacities []collector.ExcludedCSIStorageCapacity
ExcludedVolumeAttachments []string
ExcludedKubeflowNotebooks []collector.ExcludedKubeflowNotebook
ExcludedVolcanoJobs []collector.ExcludedVolcanoJob
ExcludedSparkApplications []collector.ExcludedSparkApplication
ExcludedScheduledSparkApplications []collector.ExcludedScheduledSparkApplication
ExcludedCNPGClusters []collector.ExcludedCNPGCluster
ExcludedArgoCDApplications []collector.ExcludedArgoCDApplication
ExcludedArgoCDApplicationSets []collector.ExcludedArgoCDApplicationSet
ExcludedFlaggerCanaries []collector.ExcludedFlaggerCanary
ExcludedFlaggerMetricTemplates []collector.ExcludedFlaggerMetricTemplate
ExcludedFlaggerAlertProviders []collector.ExcludedFlaggerAlertProvider
ExcludedRuntimeClasses []string
DisabledCollectors []string
// MigPartedConfigMapName/Namespace locate the NVIDIA mig-parted ConfigMap;
// empty values fall back to the GPU operator defaults.
MigPartedConfigMapName string
MigPartedConfigMapNamespace string
KubeContextName string
DakrURL string
ClusterToken string
DisableGPUMetrics bool
UpdateInterval time.Duration
NodeMetricsInterval time.Duration
ClusterSnapshotInterval time.Duration
BufferSize int
MaskSecretData bool
// DisableEnvVarCollection removes environment variables from every object
// before it is sent to Dakr (ENV_VAR_COLLECTION=disabled).
DisableEnvVarCollection bool
NumResourceProcessors int
NodeMetricsConcurrency int
NodemonRequestTimeout time.Duration
KubeletFallbackTimeout time.Duration
// SchedulerMetricsScrapeInterval bounds how often the SchedulerMetrics
// collector scrapes dz-scheduler's /metrics endpoint. See
// collector.DefaultSchedulerMetricsScrapeInterval's doc comment for the
// default's reasoning.
SchedulerMetricsScrapeInterval time.Duration
// KEDAMetricsEndpoint is the full URL of the KEDA operator /metrics endpoint
// scraped by the KedaScalerMetrics collector. Empty means the collector
// applies collector.DefaultKedaMetricsEndpoint (KEDA in devzero-system).
KEDAMetricsEndpoint string
}
PolicyConfig holds the current configuration
func (PolicyConfig) String ¶ added in v0.1.29
func (c PolicyConfig) String() string
String keeps ClusterToken out of logs and telemetry: the config is dumped with %v/%+v on every (re)initialisation and config change.
type ResourceLadder ¶ added in v0.1.29
type ResourceLadder struct {
Base *resource.Quantity `json:"base,omitempty"`
Step *resource.Quantity `json:"step,omitempty"`
Max *resource.Quantity `json:"max,omitempty"`
NodesPerStep int `json:"nodesPerStep,omitempty"`
CoresPerStep int `json:"coresPerStep,omitempty"`
}
ResourceLadder is one scaling function:
want = min(max, base + step * max(ceil(nodes/nodesPerStep), ceil(cores/coresPerStep)))
A zero NodesPerStep or CoresPerStep disables that term, and a nil Max means no cap.
func (ResourceLadder) Calculate ¶ added in v0.1.29
func (l ResourceLadder) Calculate(size ClusterSize, res corev1.ResourceName) resource.Quantity
Calculate evaluates the ladder for one resource. Arithmetic is in milli-units so CPU steps below one core work; the result is formatted the way that resource is conventionally written (768Mi, 250m).