Documentation
¶
Overview ¶
Package discovery provides service discovery for standalone/Docker mode.
Currently only FileProvider is implemented. Kubernetes mode still uses the informer logic in pkg/cache/informers.go directly.
TODO: Add KubernetesProvider to unify both modes under the Provider interface. The main blocker is that informers.go also watches ModelAdapters (for LoRA).
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type EndpointConfig ¶
type EndpointConfig struct {
// Address is the endpoint address in "host:port" format.
Address string `json:"address"`
// Labels are optional labels for routing (e.g., role-name, roleset-name for P/D).
Labels map[string]string `json:"labels,omitempty"`
}
EndpointConfig represents a single endpoint in the config file.
type FileConfig ¶
type FileConfig struct {
// Models is the list of models and their endpoints.
Models []ModelConfig `json:"models"`
}
FileConfig represents the complete endpoints configuration file.
type FileProvider ¶
type FileProvider struct {
// contains filtered or unexported fields
}
FileProvider implements Provider using a YAML configuration file.
func NewFileProvider ¶
func NewFileProvider(configPath string) *FileProvider
NewFileProvider creates a new file-based discovery provider.
func (*FileProvider) AddEventHandler ¶
func (p *FileProvider) AddEventHandler(_ string, _ cache.ResourceEventHandlerFuncs, _ <-chan struct{}) error
AddEventHandler implements Provider.
func (*FileProvider) Load ¶
func (p *FileProvider) Load() ([]any, error)
Load reads the config file and returns all endpoints as synthetic pods.
func (*FileProvider) Type ¶
func (p *FileProvider) Type() string
Type returns the provider type identifier.
type ModelConfig ¶
type ModelConfig struct {
// Name is the model name.
Name string `json:"name"`
// Endpoints are the backend endpoints serving this model.
Endpoints []EndpointConfig `json:"endpoints"`
}
ModelConfig represents a model and its endpoints.
type Provider ¶
type Provider interface {
// Load returns all k8s like Resources, can be *v1.Pod, *modelv1alpha1.ModelAdapter
Load() ([]any, error)
// Process Resource add/update/delete events. kind is the resource type, "Pod", "ModelAdapter", etc
AddEventHandler(kind string,
handler cache.ResourceEventHandlerFuncs,
stopCh <-chan struct{}) error
// Type returns a string identifier for the provider type.
Type() string
}
Provider defines the interface for service discovery backends.
Source Files