Documentation
¶
Index ¶
- Constants
- func GetKubeConfig() (*rest.Config, error)
- func GetRouterPod(t *testing.T, kubeClient kubernetes.Interface, kthenaNamespace string) *corev1.Pod
- func LoadLoRAAdapter(t *testing.T, podURL string, loraName string, loraPath string)
- func LoadMultiResourceYAMLFromFile[T any](path string) []*T
- func LoadYAMLFromFile[T any](path string) *T
- func RandomString(n int) string
- func SendChatRequest(t *testing.T, modelName string, messages []ChatMessage) *http.Response
- func SendChatRequestWithURL(t *testing.T, url string, modelName string, messages []ChatMessage) *http.Response
- func UnloadLoRAAdapter(t *testing.T, podURL string, loraName string)
- func WaitForModelServingReady(t *testing.T, ctx context.Context, kthenaClient *clientset.Clientset, ...)
- type ChatCompletionsRequest
- type ChatCompletionsResponse
- func CheckChatCompletions(t *testing.T, modelName string, messages []ChatMessage) *ChatCompletionsResponse
- func CheckChatCompletionsQuiet(t *testing.T, modelName string, messages []ChatMessage) *ChatCompletionsResponse
- func CheckChatCompletionsWithHeaders(t *testing.T, modelName string, messages []ChatMessage, ...) *ChatCompletionsResponse
- func CheckChatCompletionsWithURL(t *testing.T, url string, modelName string, messages []ChatMessage) *ChatCompletionsResponse
- func CheckChatCompletionsWithURLAndHeaders(t *testing.T, url string, modelName string, messages []ChatMessage, ...) *ChatCompletionsResponse
- func SendChatRequestWithRetry(t *testing.T, url string, modelName string, messages []ChatMessage, ...) *ChatCompletionsResponse
- func SendChatRequestWithRetryQuiet(t *testing.T, url string, modelName string, messages []ChatMessage, ...) *ChatCompletionsResponse
- type ChatMessage
- type PortForwarder
Constants ¶
const ( // DefaultRouterURL is the default URL for the router service via port-forward // Use 127.0.0.1 instead of localhost to avoid IPv6 resolution issues in CI environments DefaultRouterURL = "http://127.0.0.1:8080/v1/chat/completions" )
Variables ¶
This section is empty.
Functions ¶
func GetKubeConfig ¶
GetKubeConfig returns a Kubernetes REST config. It tries in-cluster config first, then falls back to kubeconfig file.
func GetRouterPod ¶
func GetRouterPod(t *testing.T, kubeClient kubernetes.Interface, kthenaNamespace string) *corev1.Pod
GetRouterPod is a helper function to get the router pod
func LoadLoRAAdapter ¶
LoadLoRAAdapter loads a LoRA adapter directly on the LLM-Mock pod by sending a request to /v1/load_lora_adapter The request is sent directly to the specified pod URL (e.g., http://127.0.0.1:9000/v1/load_lora_adapter) Note: This should NOT be sent through the router, as /v1/load_lora_adapter is a management endpoint
func LoadMultiResourceYAMLFromFile ¶
LoadMultiResourceYAMLFromFile loads a multi-resource YAML file from a path relative to the project root and returns all resources as a slice of the specified type. The YAML file can contain multiple resources separated by "---".
func LoadYAMLFromFile ¶
LoadYAMLFromFile loads a YAML file from a path relative to the project root and unmarshals it into the specified type.
func RandomString ¶
RandomString generates a random string of length n.
func SendChatRequest ¶
func SendChatRequestWithURL ¶
func UnloadLoRAAdapter ¶
UnloadLoRAAdapter unloads a LoRA adapter directly on the LLM-Mock pod by sending a request to /v1/unload_lora_adapter The request is sent directly to the specified pod URL (e.g., http://127.0.0.1:9000/v1/unload_lora_adapter) Note: This should NOT be sent through the router, as /v1/unload_lora_adapter is a management endpoint
Types ¶
type ChatCompletionsRequest ¶
type ChatCompletionsRequest struct {
Model string `json:"model"`
Messages []ChatMessage `json:"messages"`
Stream bool `json:"stream"`
}
ChatCompletionsRequest represents a chat completions API request
type ChatCompletionsResponse ¶
ChatCompletionsResponse represents the response from chat completions API
func CheckChatCompletions ¶
func CheckChatCompletions(t *testing.T, modelName string, messages []ChatMessage) *ChatCompletionsResponse
CheckChatCompletions sends a chat completions request to the router service and verifies the response. It uses the port-forwarded router service at localhost:8080.
func CheckChatCompletionsQuiet ¶
func CheckChatCompletionsQuiet(t *testing.T, modelName string, messages []ChatMessage) *ChatCompletionsResponse
CheckChatCompletionsQuiet is like CheckChatCompletions but does not log response status/body on success. Use it in high-volume loops (e.g. weighted distribution tests) to avoid log flood.
func CheckChatCompletionsWithHeaders ¶
func CheckChatCompletionsWithHeaders(t *testing.T, modelName string, messages []ChatMessage, headers map[string]string) *ChatCompletionsResponse
CheckChatCompletionsWithHeaders sends a chat completions request with custom headers to the default router URL.
func CheckChatCompletionsWithURL ¶
func CheckChatCompletionsWithURL(t *testing.T, url string, modelName string, messages []ChatMessage) *ChatCompletionsResponse
CheckChatCompletionsWithURL sends a chat completions request to the specified URL and verifies the response. It retries with exponential backoff if the request fails or returns a non-200 status code.
func CheckChatCompletionsWithURLAndHeaders ¶
func CheckChatCompletionsWithURLAndHeaders(t *testing.T, url string, modelName string, messages []ChatMessage, headers map[string]string) *ChatCompletionsResponse
func SendChatRequestWithRetry ¶
func SendChatRequestWithRetry(t *testing.T, url string, modelName string, messages []ChatMessage, headers map[string]string) *ChatCompletionsResponse
SendChatRequestWithRetry sends a chat completions request with retry logic but without assertions. It returns the final response regardless of status code.
func SendChatRequestWithRetryQuiet ¶
func SendChatRequestWithRetryQuiet(t *testing.T, url string, modelName string, messages []ChatMessage, headers map[string]string) *ChatCompletionsResponse
SendChatRequestWithRetryQuiet is like SendChatRequestWithRetry but does not log response status/body on success. Use it in loops or high-volume checks to avoid log flood (e.g. TestModelRouteSubsetShared weighted distribution).
type ChatMessage ¶
ChatMessage represents a chat message in the request
func NewChatMessage ¶
func NewChatMessage(role, content string) ChatMessage
NewChatMessage creates a new chat message
type PortForwarder ¶
type PortForwarder interface {
// Start runs this forwarder.
Start() error
// Close this forwarder and release any resources.
Close()
}
PortForwarder manages the forwarding of a single port. This implementation is inspired by the Istio portforwarder: https://github.com/istio/istio/blob/master/pkg/kube/portforwarder.go
func SetupPortForward ¶
func SetupPortForward(namespace, service, localPort, remotePort string) (PortForwarder, error)
SetupPortForward sets up a port-forward to a service and waits for it to be ready. It returns a PortForwarder interface that can be used to manage the port-forward. If SetupPortForward fails, the test should stop immediately as the error indicates a critical infrastructure issue that prevents the test from continuing.
func SetupPortForwardToPod ¶
func SetupPortForwardToPod(namespace, podName, localPort, podPort string) (PortForwarder, error)
SetupPortForwardToPod sets up a port-forward directly to a pod and waits for it to be ready. It returns a PortForwarder interface that can be used to manage the port-forward. If SetupPortForwardToPod fails, the test should stop immediately as the error indicates a critical infrastructure issue that prevents the test from continuing.
Source Files