utils

package
v0.0.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 15, 2025 License: Apache-2.0 Imports: 31 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CalculateExpectedArrivalRate added in v0.0.4 

func CalculateExpectedArrivalRate(loadRateReqPerSec, ttftMs, itlMs, outputTokens int) float64

CalculateExpectedArrivalRate calculates the expected arrival rate (req/min) based on GuideLLM rate (req/sec), ITL/TTFT latencies, and the number of output tokens.

With constant rate type, GuideLLM maintains a number of concurrent in-flight requests approximately equal to the target rate. The actual throughput is then determined by: 

Throughput (req/s) = Concurrency / Request_Duration (s)

Where: 

Concurrency = target_rate (e.g., 5 concurrent requests for rate=5)
Request_Duration = TTFT + (ITL * output_tokens)

func CheckIfClusterExistsOrCreate 

func CheckIfClusterExistsOrCreate(maxGPUs int) (string, error)

func CreateAcceleratorUnitCostConfigMap 

func CreateAcceleratorUnitCostConfigMap(controllerNamespace string) *corev1.ConfigMap

createAcceleratorUnitCostConfigMap creates the accelerator unitcost ConfigMap

func CreateArrivalQuery 

func CreateArrivalQuery(modelID, namespace string) string

The following utility functions are used to create Prometheus queries for testing

func CreateDecToksQuery added in v0.0.3 

func CreateDecToksQuery(modelID, namespace string) string

func CreateITLQuery 

func CreateITLQuery(modelID, namespace string) string

func CreateLlmdSimDeployment added in v0.0.3 

func CreateLlmdSimDeployment(namespace, deployName, modelName, appLabel, port string, avgTTFT, avgITL int) *appsv1.Deployment

creates a llm-d-sim deployment with the specified configuration

func CreateLlmdSimService added in v0.0.3 

func CreateLlmdSimService(namespace, serviceName, appLabel string, nodePort, port int) *corev1.Service

creates a service for the llm-d-sim deployment

func CreateLlmdSimServiceMonitor added in v0.0.3 

func CreateLlmdSimServiceMonitor(name, namespace, targetNamespace, appLabel string) *unstructured.Unstructured

creates a ServiceMonitor for llm-d-sim metrics collection

func CreateLoadGeneratorJob added in v0.0.3 

func CreateLoadGeneratorJob(image, namespace, targetURL, modelName string, rate, maxSeconds, inputTokens, outputTokens int, k8sClient *kubernetes.Clientset, ctx context.Context) (*batchv1.Job, error)

CreateLoadGeneratorJob creates and launches a Kubernetes Job for load generation using GuideLLM with the specified parameters

func CreatePromptToksQuery added in v0.0.3 

func CreatePromptToksQuery(modelID, namespace string) string

func CreateServiceClassConfigMap 

func CreateServiceClassConfigMap(controllerNamespace string) *corev1.ConfigMap

createServiceClassConfigMap creates the serviceclass ConfigMap

func CreateTTFTQuery added in v0.0.3 

func CreateTTFTQuery(modelID, namespace string) string

func CreateVariantAutoscalingConfigMap 

func CreateVariantAutoscalingConfigMap(cmName, controllerNamespace string) *corev1.ConfigMap

func CreateVariantAutoscalingResource 

func CreateVariantAutoscalingResource(namespace, resourceName, modelId, acc string) *v1alpha1.VariantAutoscaling

creates a VariantAutoscaling resource with owner reference to deployment

func DetectArchitecture added in v0.0.3 

func DetectArchitecture() (string, error)

DetectArchitecture checks the host architecture

func GetInfernoReplicaMetrics 

func GetInfernoReplicaMetrics(variantName, namespace, acceleratorType string) (currentReplicas, desiredReplicas, desiredRatio float64, err error)

GetInfernoReplicaMetrics queries Prometheus for metrics emitted by the Inferno autoscaler

func GetNonEmptyLines 

func GetNonEmptyLines(output string) []string

GetNonEmptyLines converts given command output string into individual objects according to line breakers, and ignores the empty elements in it.

func GetProjectDir 

func GetProjectDir() (string, error)

GetProjectDir will return the directory where the project is

func InstallCertManager 

func InstallCertManager() error

InstallCertManager installs the cert manager bundle.

func InstallPrometheusOperator 

func InstallPrometheusOperator() error

InstallPrometheusOperator installs the prometheus Operator to be used to export the enabled metrics. Includes TLS certificate generation and configuration for HTTPS support.

func IsCertManagerCRDsInstalled 

func IsCertManagerCRDsInstalled() bool

IsCertManagerCRDsInstalled checks if any Cert Manager CRDs are installed by verifying the existence of key CRDs related to Cert Manager.

func IsPrometheusCRDsInstalled 

func IsPrometheusCRDsInstalled() bool

IsPrometheusCRDsInstalled checks if any Prometheus CRDs are installed by verifying the existence of key CRDs related to Prometheus.

func LoadImageToKindClusterWithName 

func LoadImageToKindClusterWithName(name string, maxGPUs int) error

LoadImageToKindClusterWithName loads a local docker image to the kind cluster

func LogVariantAutoscalingStatus 

func LogVariantAutoscalingStatus(ctx context.Context, vaName, namespace string, crClient client.Client) error

LogVariantAutoscalingStatus fetches and logs the status of the specified VariantAutoscaling resource

func Run 

func Run(cmd *exec.Cmd) (string, error)

Run executes the provided command within this context

func SetUpPortForward 

func SetUpPortForward(k8sClient *kubernetes.Clientset, ctx context.Context, serviceName, namespace string, localPort, servicePort int) *exec.Cmd

SetUpPortForward sets up port forwarding to a Service on the specified port

func SetupTestEnvironment added in v0.0.3 

func SetupTestEnvironment(image string, numNodes, gpusPerNode int, gpuTypes string)

setupEnvironment sets up necessary environment variables for the E2E tests

func StopCmd 

func StopCmd(cmd *exec.Cmd) error

StopCmd attempts to gracefully stop the provided command, handling early exits and timeouts.

func StopJob added in v0.0.3 

func StopJob(namespace string, job *batchv1.Job, k8sClient *kubernetes.Clientset, ctx context.Context) error

StopJob deletes a Kubernetes Job and ensures it is removed from the cluster along with its Pods.

func UncommentCode 

func UncommentCode(filename, target, prefix string) error

UncommentCode searches for target in the file and remove the comment prefix of the target content. The target content may span multiple lines.

func UninstallCertManager 

func UninstallCertManager()

UninstallCertManager uninstalls the cert manager

func UninstallPrometheusOperator 

func UninstallPrometheusOperator()

UninstallPrometheusOperator uninstalls the prometheus

func ValidateAppLabelUniqueness 

func ValidateAppLabelUniqueness(namespace, appLabel string, k8sClient *kubernetes.Clientset, crClient client.Client)

ValidateAppLabelUniqueness checks if the appLabel is already in use by other resources and fails if it's not unique

func ValidateVariantAutoscalingUniqueness 

func ValidateVariantAutoscalingUniqueness(namespace, modelId, acc string, crClient client.Client)

ValidateVariantAutoscalingUniqueness checks if the VariantAutoscaling configuration is unique within the namespace

func VerifyPortForwardReadiness 

func VerifyPortForwardReadiness(ctx context.Context, localPort int, request string) error

VerifyPortForwardReadiness checks if the port forwarding is ready by sending HTTP requests to the specified local port

Types

type MockPromAPI 

type MockPromAPI struct {
	QueryResults map[string]model.Value
	QueryErrors  map[string]error
}

MockPromAPI is a mock implementation of promv1.API for testing

func (*MockPromAPI) AlertManagers 

func (m *MockPromAPI) AlertManagers(ctx context.Context) (promv1.AlertManagersResult, error)

func (*MockPromAPI) Alerts 

func (m *MockPromAPI) Alerts(ctx context.Context) (promv1.AlertsResult, error)

func (*MockPromAPI) Buildinfo 

func (m *MockPromAPI) Buildinfo(ctx context.Context) (promv1.BuildinfoResult, error)

func (*MockPromAPI) CleanTombstones 

func (m *MockPromAPI) CleanTombstones(ctx context.Context) error

func (*MockPromAPI) Config 

func (m *MockPromAPI) Config(ctx context.Context) (promv1.ConfigResult, error)

func (*MockPromAPI) DeleteSeries 

func (m *MockPromAPI) DeleteSeries(ctx context.Context, matches []string, startTime, endTime time.Time) error

func (*MockPromAPI) Flags 

func (m *MockPromAPI) Flags(ctx context.Context) (promv1.FlagsResult, error)

func (*MockPromAPI) GetValue 

func (m *MockPromAPI) GetValue(ctx context.Context, timestamp time.Time, opts ...promv1.Option) (model.Value, promv1.Warnings, error)

func (*MockPromAPI) LabelNames 

func (m *MockPromAPI) LabelNames(ctx context.Context, matches []string, startTime, endTime time.Time, opts ...promv1.Option) ([]string, promv1.Warnings, error)

func (*MockPromAPI) LabelValues 

func (m *MockPromAPI) LabelValues(ctx context.Context, label string, matches []string, startTime, endTime time.Time, opts ...promv1.Option) (model.LabelValues, promv1.Warnings, error)

func (*MockPromAPI) Metadata 

func (m *MockPromAPI) Metadata(ctx context.Context, metric, limit string) (map[string][]promv1.Metadata, error)

func (*MockPromAPI) Query 

func (m *MockPromAPI) Query(ctx context.Context, query string, ts time.Time, opts ...promv1.Option) (model.Value, promv1.Warnings, error)

func (*MockPromAPI) QueryExemplars 

func (m *MockPromAPI) QueryExemplars(ctx context.Context, query string, startTime, endTime time.Time) ([]promv1.ExemplarQueryResult, error)

func (*MockPromAPI) QueryRange 

func (m *MockPromAPI) QueryRange(ctx context.Context, query string, r promv1.Range, opts ...promv1.Option) (model.Value, promv1.Warnings, error)

func (*MockPromAPI) Rules 

func (m *MockPromAPI) Rules(ctx context.Context) (promv1.RulesResult, error)

func (*MockPromAPI) Runtimeinfo 

func (m *MockPromAPI) Runtimeinfo(ctx context.Context) (promv1.RuntimeinfoResult, error)

func (*MockPromAPI) Series 

func (m *MockPromAPI) Series(ctx context.Context, matches []string, startTime, endTime time.Time, opts ...promv1.Option) ([]model.LabelSet, promv1.Warnings, error)

func (*MockPromAPI) Snapshot 

func (m *MockPromAPI) Snapshot(ctx context.Context, skipHead bool) (promv1.SnapshotResult, error)

func (*MockPromAPI) TSDB 

func (m *MockPromAPI) TSDB(ctx context.Context, opts ...promv1.Option) (promv1.TSDBResult, error)

func (*MockPromAPI) Targets 

func (m *MockPromAPI) Targets(ctx context.Context) (promv1.TargetsResult, error)

func (*MockPromAPI) TargetsMetadata 

func (m *MockPromAPI) TargetsMetadata(ctx context.Context, matchTarget, metric, limit string) ([]promv1.MetricMetadata, error)

func (*MockPromAPI) WalReplay 

func (m *MockPromAPI) WalReplay(ctx context.Context) (promv1.WalReplayStatus, error)

type PrometheusClient 

type PrometheusClient struct {
	// contains filtered or unexported fields
}

PrometheusClient wraps the official Prometheus client

func NewPrometheusClient 

func NewPrometheusClient(baseURL string, insecureSkipVerify bool) (*PrometheusClient, error)

creates a new Prometheus client for e2e tests

func (*PrometheusClient) QueryWithRetry 

func (p *PrometheusClient) QueryWithRetry(ctx context.Context, query string) (float64, error)

QueryWithRetry queries Prometheus API with retries and returns the metric value

type PrometheusQueryResult 

type PrometheusQueryResult struct {
	Status string `json:"status"`
	Data   struct {
		ResultType string `json:"resultType"`
		Result     []struct {
			Metric map[string]string `json:"metric"`
			Value  []any             `json:"value"`
		} `json:"result"`
	} `json:"data"`
}

PrometheusQueryResult represents the response from Prometheus API

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.