Documentation
¶
Overview ¶
Package virtual implements the Virtual Models service.
Responsibilities:
- CRUD operations for Agent records
- Validation of virtual model configurations
- Aggregated limits and capabilities across the fall-through list
Index ¶
- func ParseMarker(marker string) (providerID, slug string, err error)
- func ProviderMarker(providerID, slug string) string
- type Metadata
- type ProviderVMGroup
- type Service
- func (s *Service) Create(vm *models.VirtualModel) error
- func (s *Service) Delete(id string) error
- func (s *Service) Get(id string) (*models.VirtualModel, error)
- func (s *Service) GroupsForProvider(providerID string) ([]ProviderVMGroup, error)
- func (s *Service) List() ([]*models.VirtualModel, error)
- func (s *Service) LiveMembers(agent *models.VirtualModel) ([]models.ModelId, error)
- func (s *Service) SetLogger(l *slog.Logger)
- func (s *Service) SyncProviderModels(ctx context.Context, providerID string) ([]ProviderVMGroup, error)
- func (s *Service) Update(id string, vm *models.VirtualModel) error
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ParseMarker ¶ added in v0.1.2
ParseMarker splits a ManagedBy marker into provider ID and endpoint slug. The split runs at the last colon, so provider IDs containing colons (qualified "type:qualifier" rows) survive the round trip.
func ProviderMarker ¶ added in v0.1.0
ProviderMarker builds the ManagedBy marker for a provider endpoint group.
Types ¶
type Metadata ¶ added in v0.1.0
type Metadata struct {
Capabilities []string
ContextLength int64
MaxCompletionTokens int64
InputModalities []string
OutputModalities []string
Reasoning *models.ModelReasoning
SupportedParameters []string
}
Metadata is generated virtual-model metadata, folded serve-time from the currently available members. Every set is an intersection (only what all members share); limits are minima over members reporting them.
func FoldMembers ¶ added in v0.1.0
FoldMembers folds member infos into one virtual metadata record. Callers pass only available (known, enabled) members; an empty input yields a zero Metadata.
type ProviderVMGroup ¶ added in v0.1.0
type ProviderVMGroup struct {
Endpoint string `json:"endpoint"`
Label string `json:"label"`
Models []string `json:"models"`
Virtual *models.VirtualModel `json:"virtual,omitempty"`
}
ProviderVMGroup is one endpoint group with its managed virtual model.
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
Service manages virtual model records.
func (*Service) Create ¶
func (s *Service) Create(vm *models.VirtualModel) error
Create creates a new virtual model with a slug ID derived from its name.
func (*Service) Get ¶
func (s *Service) Get(id string) (*models.VirtualModel, error)
Get retrieves a virtual model by ID.
func (*Service) GroupsForProvider ¶ added in v0.1.0
func (s *Service) GroupsForProvider(providerID string) ([]ProviderVMGroup, error)
GroupsForProvider computes endpoint groups from the cached merged view (no upstream fetch) with managed markers resolved. Only non-empty groups are returned; disabled models never become members.
func (*Service) List ¶
func (s *Service) List() ([]*models.VirtualModel, error)
List returns all virtual models.
func (*Service) LiveMembers ¶ added in v0.1.0
LiveMembers returns the model ids to try for the agent, snapshotted for one request. Managed virtual models resolve their endpoint group live from the provider model list (disabled and removed models drop out on the next call); manual models use the stored list. Storage failures and corrupt markers are errors; a managed group with no models is an empty list without an error.
func (*Service) SetLogger ¶ added in v0.1.0
SetLogger wires structured logging (called once from server.New).
func (*Service) SyncProviderModels ¶ added in v0.1.0
func (s *Service) SyncProviderModels(ctx context.Context, providerID string) ([]ProviderVMGroup, error)
SyncProviderModels creates/updates/deletes managed virtual models so each non-empty endpoint group has exactly one. Members are enabled models serving the endpoint, in cache order. Managed VMs whose group emptied are deleted. Manual edits to name/description/instruction survive: only the member list is rewritten. Per-group failures accumulate into the returned error; completed groups still apply.