Documentation
¶
Overview ¶
Package virtualmodels implements public virtual model pools such as ail-compound. It deliberately lives below internal/ so SDK clients only see the stable config shape, not selector internals.
Index ¶
- Constants
- func FallbackEnabled(cfg *config.SDKConfig, model string) bool
- func IsVirtualModel(cfg *config.SDKConfig, model string) bool
- func PublicCatalog(cfg *config.SDKConfig) []map[string]any
- func RewriteModelField(payload []byte, publicModel string) []byte
- type NoEligibleBackendError
- type Requirements
- type Route
- type Router
- type RouterPoolState
- type RouterState
Constants ¶
const ( ClassChatText = "chat_text" ClassChatTextTools = "chat_text_tools" ClassChatMultiturnTools = "chat_multiturn_tools" ClassChatImageUnderstanding = "chat_image_understanding" ClassChatAudioUnderstanding = "chat_audio_understanding" ClassChatMultimodalUnderstanding = "chat_multimodal_understanding" ClassAudioTranscription = "audio_transcription" ClassImageGeneration = "image_generation" ClassSpeechGeneration = "speech_generation" ClassMusicGeneration = "music_generation" ClassLyricsGeneration = "lyrics_generation" ClassEmbeddings = "embeddings" )
Variables ¶
This section is empty.
Functions ¶
func FallbackEnabled ¶
FallbackEnabled reports whether a virtual pool should try another eligible member on recoverable upstream failures.
func IsVirtualModel ¶
IsVirtualModel reports whether model maps to a configured virtual pool.
func PublicCatalog ¶
PublicCatalog returns provider-neutral public virtual models for /v1/models.
func RewriteModelField ¶
RewriteModelField rewrites a JSON object model field when present. It also supports raw SSE payload text; data: [DONE] is preserved.
Types ¶
type NoEligibleBackendError ¶
type NoEligibleBackendError struct {
Model string
Requirements Requirements
}
NoEligibleBackendError reports explicit capability mismatch without trying an upstream provider.
func (NoEligibleBackendError) Code ¶
func (e NoEligibleBackendError) Code() string
func (NoEligibleBackendError) Error ¶
func (e NoEligibleBackendError) Error() string
type Requirements ¶
type Requirements struct {
Class string
InputText bool
InputImage bool
InputAudio bool
NeedsTools bool
HasToolHistory bool
OutputText bool
OutputImage bool
OutputAudio bool
MinContext int
Endpoint string
}
Requirements captures the capability contract required by a request.
func DetectRequirements ¶
func DetectRequirements(rawJSON []byte, endpoint string) Requirements
DetectRequirements classifies OpenAI-compatible requests by endpoint and body.
type Route ¶
type Route struct {
PublicModel string
Provider string
NativeModel string
MemberID string
Requirements Requirements
}
Route is the concrete provider/model choice for one request.
type Router ¶
type Router struct {
// contains filtered or unexported fields
}
Router keeps smooth weighted round-robin accumulators for virtual model pools. State is persisted best-effort so local process restarts continue proportional routing without changing the public OpenAI-compatible API contract.
func NewRouterWithStatePath ¶ added in v0.5.24
func (*Router) Select ¶
func (r *Router) Select(cfg *config.SDKConfig, model string, rawJSON []byte, endpoint string) (Route, error)
Select chooses one eligible member for a virtual model.
func (*Router) SelectExcluding ¶
func (r *Router) SelectExcluding(cfg *config.SDKConfig, model string, rawJSON []byte, endpoint string, exclude map[string]struct{}) (Route, error)
SelectExcluding chooses an eligible member while skipping member IDs in exclude.
func (*Router) Snapshot ¶ added in v0.5.24
func (r *Router) Snapshot() RouterState
Snapshot returns a copy of the current router state for tests and observability. It best-effort loads persisted state before returning.
type RouterPoolState ¶ added in v0.5.24
type RouterPoolState struct {
ConfigHash string `json:"config_hash"`
Current map[string]int `json:"current"`
Counts map[string]int `json:"counts"`
LastSelected string `json:"last_selected,omitempty"`
}
RouterPoolState stores the smooth weighted round-robin accumulators for one virtual model routing bucket.
type RouterState ¶ added in v0.5.24
type RouterState struct {
SchemaVersion int `json:"schema_version"`
Algorithm string `json:"algorithm"`
Pools map[string]RouterPoolState `json:"pools"`
}
RouterState is the persisted, operator-observable routing state for virtual model pools. Pool keys start with lower-case "<public-model>|<request-class>" and may include media/tool/context qualifiers that affect eligible members.
Source Files