virtualmodels

package
v0.1.61 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 26, 2026 License: MIT Imports: 23 Imported by: 0

Documentation

Overview

Package virtualmodels unifies model aliases (redirects) and model access overrides (policies) behind one entity, the virtual model, persisted in a single virtual_models table.

A row with Targets is a REDIRECT: Source is a new addressable name that rewrites to one or more real models. A redirect with a single target is a plain alias; a redirect with several targets is load balanced, distributing requests across them by Strategy (round robin or lowest cost). A row without Targets is an ACCESS POLICY: Source is a scoped selector over existing models, gated by UserPaths.

The Service is a single native engine: it operates directly on VirtualModel rows behind one in-memory snapshot, serving both redirect resolution and policy authorization without composing other engines.

Index

Constants

View Source 
const (
	// StrategyRoundRobin rotates across targets, honoring per-target Weight. It is
	// the default when Strategy is empty.
	StrategyRoundRobin = "round_robin"
	// StrategyCost always routes to the cheapest currently-available target, ranked
	// by the model registry's per-token pricing.
	StrategyCost = "cost"
)

Load-balancing strategies for multi-target redirects.

View Source 
const (
	KindRedirect = "redirect"
	KindPolicy   = "policy"
)

Role kinds for the admin view.

Variables

View Source 
var ErrNotFound = errors.New("virtual model not found")

ErrNotFound indicates a requested virtual model was not found.

Functions

func IsValidationError 

func IsValidationError(err error) bool

IsValidationError reports whether err is a validation error.

Types

type BatchPreparer 

type BatchPreparer struct {
	// contains filtered or unexported fields
}

BatchPreparer rewrites redirect (alias) sources for native batch subrequests and validates model access before provider submission. It is the combined replacement for the two old preparers (alias rewrite then access validation).

func NewBatchPreparer 

func NewBatchPreparer(provider core.RoutableProvider, service *Service) *BatchPreparer

NewBatchPreparer creates the combined redirect-rewrite + access-validation batch preparer.

func (*BatchPreparer) PrepareBatchRequest 

func (p *BatchPreparer) PrepareBatchRequest(ctx context.Context, providerType string, req *core.BatchRequest) (*core.BatchRewriteResult, error)

PrepareBatchRequest rewrites redirect sources for inline and file-backed batch items and validates model access for each resolved selector.

type Catalog 

type Catalog interface {
	Supports(model string) bool
	// ModelAvailable is Supports narrowed to providers whose inventory is
	// fresh: target selection uses it so load balancing routes around a
	// provider whose latest model refresh failed.
	ModelAvailable(model string) bool
	GetProviderType(model string) string
	LookupModel(model string) (*core.Model, bool)
	ProviderNames() []string
}

Catalog is the combined catalog surface the native engine needs.

type ChatExecutor 

type ChatExecutor struct {
	// contains filtered or unexported fields
}

ChatExecutor applies virtual-model redirects to transport-free chat completions before delegating to the wrapped provider. It is the narrow replacement for the former redirect-aware Provider decorator, whose only production role was serving as the guardrail auxiliary-LLM executor (guardrails.ChatCompletionExecutor is a single-method interface).

func NewChatExecutor 

func NewChatExecutor(inner ChatExecutorProvider, service *Service) *ChatExecutor

NewChatExecutor creates a redirect-aware chat executor over inner.

func (*ChatExecutor) ChatCompletion 

func (e *ChatExecutor) ChatCompletion(ctx context.Context, req *core.ChatRequest) (*core.ChatResponse, error)

ChatCompletion resolves the request's redirect (user-path aware) and delegates the rewritten request to the wrapped provider.

type ChatExecutorProvider 

type ChatExecutorProvider interface {
	ChatCompletion(ctx context.Context, req *core.ChatRequest) (*core.ChatResponse, error)
	// contains filtered or unexported methods
}

ChatExecutorProvider is the slice of the router the executor needs: model support checks for redirect validation and chat dispatch.

type EffectiveState 

type EffectiveState struct {
	Selector       string   `json:"selector"`
	ProviderName   string   `json:"provider_name,omitempty"`
	Model          string   `json:"model,omitempty"`
	DefaultEnabled bool     `json:"default_enabled"`
	Enabled        bool     `json:"enabled"`
	UserPaths      []string `json:"user_paths,omitempty"`
}

EffectiveState is the compiled access decision for one concrete selector.

type MongoDBStore 

type MongoDBStore struct {
	// contains filtered or unexported fields
}

MongoDBStore stores virtual models in MongoDB.

func NewMongoDBStore 

func NewMongoDBStore(database *mongo.Database) (*MongoDBStore, error)

NewMongoDBStore creates collection indexes if needed.

func (*MongoDBStore) Close 

func (s *MongoDBStore) Close() error

func (*MongoDBStore) Delete 

func (s *MongoDBStore) Delete(ctx context.Context, source string) error

func (*MongoDBStore) Get 

func (s *MongoDBStore) Get(ctx context.Context, source string) (*VirtualModel, error)

func (*MongoDBStore) List 

func (s *MongoDBStore) List(ctx context.Context) ([]VirtualModel, error)

func (*MongoDBStore) Upsert 

func (s *MongoDBStore) Upsert(ctx context.Context, vm VirtualModel) error

type Resolution 

type Resolution struct {
	Requested core.ModelSelector
	Resolved  core.ModelSelector
	Source    string
}

Resolution captures the requested selector and the concrete selector chosen after redirect resolution. Source is the redirect name that matched, if any.

type Result 

type Result struct {
	Service *Service
	Store   Store
	// contains filtered or unexported fields
}

Result holds the initialized virtual models service and any owned resources.

func New 

func New(ctx context.Context, cfg *config.Config, shared storage.Storage, catalog Catalog, declaredProviders []string) (*Result, error)

New creates a virtual models subsystem using an existing storage connection.

func (*Result) Close 

func (r *Result) Close() error

Close releases resources held by the virtual models subsystem.

type SQLStore added in v0.1.60 

type SQLStore struct {
	// contains filtered or unexported fields
}

SQLStore stores virtual models in a SQL database.

func NewSQLStore added in v0.1.60 

func NewSQLStore(ctx context.Context, db sqlx.DB) (*SQLStore, error)

NewSQLStore creates the virtual_models table and indexes if needed.

func (*SQLStore) Close added in v0.1.60 

func (s *SQLStore) Close() error

func (*SQLStore) Delete added in v0.1.60 

func (s *SQLStore) Delete(ctx context.Context, source string) error

func (*SQLStore) Get added in v0.1.60 

func (s *SQLStore) Get(ctx context.Context, source string) (*VirtualModel, error)

func (*SQLStore) List added in v0.1.60 

func (s *SQLStore) List(ctx context.Context) ([]VirtualModel, error)

func (*SQLStore) Upsert added in v0.1.60 

func (s *SQLStore) Upsert(ctx context.Context, vm VirtualModel) error

func (*SQLStore) UpsertAll added in v0.1.60 

func (s *SQLStore) UpsertAll(ctx context.Context, vms []VirtualModel) error

UpsertAll writes every row in a single transaction, so a failed seed leaves the table untouched rather than partially populated (which would otherwise trip the "already populated" guard and suppress a re-import on the next start).

type Service 

type Service struct {
	// contains filtered or unexported fields
}

Service is the single native engine over the virtual_models store. It serves both redirect resolution (alias behavior) and policy authorization (access override behavior) from one atomically swapped in-memory snapshot.

func NewService 

func NewService(store Store, catalog Catalog, defaultEnabled bool) (*Service, error)

NewService creates a virtual models service backed by the store and catalog. defaultEnabled is the process-wide model availability default consulted when no policy matches.

func (*Service) AllowsModel 

func (s *Service) AllowsModel(ctx context.Context, selector core.ModelSelector) bool

AllowsModel reports whether selector is available for the effective request user path.

func (*Service) Delete 

func (s *Service) Delete(ctx context.Context, source string) error

Delete removes one virtual model and refreshes the in-memory snapshot.

func (*Service) EffectiveState 

func (s *Service) EffectiveState(selector core.ModelSelector) EffectiveState

EffectiveState resolves the compiled access state for one concrete selector.

func (*Service) EnabledByDefault 

func (s *Service) EnabledByDefault() bool

EnabledByDefault reports the process-wide model availability default.

func (*Service) ExposedModels 

func (s *Service) ExposedModels() []core.Model

ExposedModels returns enabled redirects projected as model-list entries.

func (*Service) ExposedModelsFiltered 

func (s *Service) ExposedModelsFiltered(allow func(core.ModelSelector) bool) []core.Model

ExposedModelsFiltered returns enabled redirects projected as model-list entries, filtered by the concrete target selector.

func (*Service) ExposedModelsForUserPath 

func (s *Service) ExposedModelsForUserPath(userPath string, allow func(core.ModelSelector) bool) []core.Model

ExposedModelsForUserPath is ExposedModelsFiltered plus per-redirect user_path scoping: a redirect carrying user_paths is hidden from callers it would not apply to, so a scoped alias is not listed (its name exposed) to callers outside its scope even though resolution would fall through for them.

func (*Service) FilterPublicModels 

func (s *Service) FilterPublicModels(ctx context.Context, models []core.Model) []core.Model

FilterPublicModels removes models that are unavailable for the effective request user path.

func (*Service) Get 

func (s *Service) Get(source string) (*VirtualModel, bool)

Get returns one cached virtual model by source.

func (*Service) GetProviderType 

func (s *Service) GetProviderType(model string) string

GetProviderType returns the resolved provider type for a redirect, or empty when unresolved.

func (*Service) List 

func (s *Service) List() []VirtualModel

List returns all cached virtual models sorted by source.

func (*Service) ListViews 

func (s *Service) ListViews() []View

ListViews returns all virtual models (redirects and policies) for the admin UI.

func (*Service) Refresh 

func (s *Service) Refresh(ctx context.Context) error

Refresh reloads virtual models from storage and atomically swaps the snapshot.

func (*Service) Rename 

func (s *Service) Rename(ctx context.Context, oldSource string, vm VirtualModel) error

Rename moves an existing virtual model to a new source: it stores the row under the new source and removes the old one, validating and refreshing like Upsert with rollback on failure. A no-op rename (old == new after normalization) delegates to Upsert. The new source must be free — renaming onto an existing row is rejected rather than silently overwriting it, since source is the primary key on every store backend.

func (*Service) Resolve 

func (s *Service) Resolve(model, provider string) (Resolution, bool, error)

Resolve resolves raw model/provider inputs through the redirect table.

func (*Service) ResolveModel 

func (s *Service) ResolveModel(requested core.RequestedModelSelector) (core.ModelSelector, bool, error)

ResolveModel resolves a requested selector and returns the concrete selector chosen for execution. It does not consult user_paths; scoped redirects are applied by ResolveModelForUserPath on the request path.

func (*Service) ResolveModelForUserPath 

func (s *Service) ResolveModelForUserPath(ctx context.Context, requested core.RequestedModelSelector) (core.ModelSelector, bool, error)

ResolveModelForUserPath resolves a requested selector honoring per-redirect user_paths against the effective request user path. A redirect scoped to user_paths the caller does not match falls through to the literal model name.

func (*Service) ResolveRefreshTarget 

func (s *Service) ResolveRefreshTarget(requested core.RequestedModelSelector) (core.ModelSelector, bool, error)

ResolveRefreshTarget returns a redirect target without consulting the current catalog so callers can refresh an unavailable target provider before normal resolution is retried.

func (*Service) ResolveUpsertEnabled 

func (s *Service) ResolveUpsertEnabled(source, oldSource string, requested *bool) bool

ResolveUpsertEnabled returns the enabled flag an upsert should persist when the request may omit it: the requested value when present; otherwise the stored value for source (or, on a rename, for oldSource, since the new source does not exist yet); defaulting to true for new rows.

func (*Service) SetConfigModels 

func (s *Service) SetConfigModels(models []VirtualModel)

SetConfigModels installs the declarative (config.yaml / VIRTUAL_MODELS) virtual models that override store rows of the same source. Call it before the first Refresh, then ValidateManagedConfig to reject invalid declarations at startup.

func (*Service) SetTargetCapacity 

func (s *Service) SetTargetCapacity(capacity func(qualifiedModel string) bool)

SetTargetCapacity installs the rate-limit capacity probe consulted by load balancing. Must be called before the service starts resolving requests.

func (*Service) StartBackgroundRefresh 

func (s *Service) StartBackgroundRefresh(interval time.Duration) func()

StartBackgroundRefresh periodically reloads virtual models until stopped.

func (*Service) Supports 

func (s *Service) Supports(model string) bool

Supports reports whether a redirect currently resolves to a concrete model.

func (*Service) Upsert 

func (s *Service) Upsert(ctx context.Context, vm VirtualModel) error

Upsert validates and stores one virtual model, replacing any existing row at the same source even when its kind changes. A policy with no metadata whose access state is identical to what it would inherit is deleted instead of persisting a redundant row.

func (*Service) ValidateManagedConfig 

func (s *Service) ValidateManagedConfig(declaredProviders []string) error

ValidateManagedConfig checks that every declarative config redirect satisfies the catalog-independent redirect invariants (valid selector, no self- or cross-redirect target, no misspelled target provider), so a malformed IaC entry fails startup loudly. declaredProviders lists the names present in the providers configuration even when they did not register — e.g. their credentials are unset in this environment — so a config shared across environments still boots; such targets only warn and stay unavailable. Call it once after the initial Refresh.

It deliberately does NOT require targets to be catalog-supported: the provider model catalog loads asynchronously and may still be warming when this runs, and an unavailable target is skipped at resolve time like any other redirect target (the background ticker also skips this gate so a transient provider-catalog gap cannot freeze the snapshot). Gating startup on availability would abort an otherwise-valid declaration on a cold cache or a momentarily-unreachable provider — availability is runtime state, not a property of the declaration.

func (*Service) ValidateModelAccess 

func (s *Service) ValidateModelAccess(ctx context.Context, selector core.ModelSelector) error

ValidateModelAccess returns a typed request error when selector is not available.

type Store 

type Store interface {
	List(ctx context.Context) ([]VirtualModel, error)
	Get(ctx context.Context, source string) (*VirtualModel, error)
	Upsert(ctx context.Context, vm VirtualModel) error
	Delete(ctx context.Context, source string) error
	Close() error
}

Store defines persistence operations for virtual models.

type Target 

type Target struct {
	Provider string  `json:"provider,omitempty" bson:"provider,omitempty"`
	Model    string  `json:"model" bson:"model"`
	Weight   float64 `json:"weight,omitempty" bson:"weight,omitempty"`
}

Target is one concrete (provider, model) destination of a redirect.

Weight biases the round-robin strategy: a target with weight 2 receives twice the share of a target with weight 1. A non-positive or unset weight is treated as 1, so single-target and unweighted redirects behave identically to before. Weight is ignored by the cost strategy, which always picks the cheapest target.

type View 

type View struct {
	Source        string    `json:"source"`
	Kind          string    `json:"kind"`
	Targets       []Target  `json:"targets,omitempty"`
	Strategy      string    `json:"strategy,omitempty"`
	ProviderName  string    `json:"provider_name,omitempty"`
	Model         string    `json:"model,omitempty"`
	UserPaths     []string  `json:"user_paths,omitempty"`
	Description   string    `json:"description,omitempty"`
	Enabled       bool      `json:"enabled"`
	Managed       bool      `json:"managed,omitempty"`
	ResolvedModel string    `json:"resolved_model,omitempty"`
	ProviderType  string    `json:"provider_type,omitempty"`
	Valid         bool      `json:"valid,omitempty"`
	ScopeKind     string    `json:"scope_kind,omitempty"`
	CreatedAt     time.Time `json:"created_at"`
	UpdatedAt     time.Time `json:"updated_at"`
}

View is the admin-facing representation of one virtual model.

type VirtualModel 

type VirtualModel struct {
	Source       string    `json:"source" bson:"_id"`
	Targets      []Target  `json:"targets,omitempty" bson:"targets,omitempty"`
	Strategy     string    `json:"strategy,omitempty" bson:"strategy,omitempty"`
	ProviderName string    `json:"provider_name,omitempty" bson:"provider_name,omitempty"`
	Model        string    `json:"model,omitempty" bson:"model,omitempty"`
	UserPaths    []string  `json:"user_paths,omitempty" bson:"user_paths,omitempty"`
	Description  string    `json:"description,omitempty" bson:"description,omitempty"`
	Enabled      bool      `json:"enabled" bson:"enabled"`
	CreatedAt    time.Time `json:"created_at" bson:"created_at"`
	UpdatedAt    time.Time `json:"updated_at" bson:"updated_at"`

	// Managed marks a virtual model supplied declaratively through config.yaml or
	// the VIRTUAL_MODELS env var rather than the admin store. It is an in-memory
	// flag only: stores never read or write it. Managed rows override store rows
	// of the same Source and are read-only to the admin API.
	Managed bool `json:"managed,omitempty" bson:"-"`
}

VirtualModel is one operator-defined model entry.

func ConfigModels 

func ConfigModels(entries []config.VirtualModelConfig) []VirtualModel

ConfigModels converts declarative config.yaml / VIRTUAL_MODELS entries into virtual model rows marked as managed. The rows are fully validated when the service builds its snapshot, so an invalid declaration fails startup loudly.

func (VirtualModel) IsRedirect 

func (v VirtualModel) IsRedirect() bool

IsRedirect reports whether this row redirects (has at least one target).

func (VirtualModel) Kind 

func (v VirtualModel) Kind() string

Kind returns the derived role: "redirect" or "policy".

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.