Documentation
¶
Overview ¶
Package models defines all shared data structures for llm-router.
Index ¶
- Constants
- func EmbeddingBase64(values []float64) string
- func NormalizeBaseURL(raw string) (string, error)
- func SpeechContentType(format string) string
- type AdminUser
- type AuthFlowResult
- type AuthStepInput
- type ChatCompletionChoice
- type ChatCompletionRequest
- type ChatCompletionResponse
- type ChatCompletionUsage
- type ChatMessage
- type ChatMessageContentPart
- type ChatMessageImageURL
- type ChatMessageInputAudio
- type ChatTool
- type ChatToolCall
- type ChatToolFunction
- type CompletionTokensDetails
- type Credential
- type CredentialScope
- type Embedding
- type EmbeddingsRequest
- type EmbeddingsResponse
- type EmbeddingsUsage
- type ErrorResponse
- type ErrorType
- type ExhaustedEntry
- type GeoBanEntry
- type GeoConfig
- type ImageData
- type ImageGenerationRequest
- type ImageGenerationResponse
- type MetricEvent
- type MetricsFilters
- type MetricsOverview
- type ModelId
- type ModelInfo
- type ModelOverride
- type ModelReasoning
- type OpenAIError
- type OpenAIErrorBody
- type PluginInternalError
- type PromptTokensDetails
- type Provider
- type ProviderError
- type ProviderInstance
- type ProviderInstanceCreateRequest
- type ProviderInstanceUpdateRequest
- type ProviderStats
- type ProxyCandidate
- type ProxyConfig
- type RouterConfiguration
- type RouterToken
- type SpeechRequest
- type SpeechResponse
- type StreamChunk
- type StreamChunkChoice
- type StreamOptions
- type TimeRange
- type TimeSeriesPoint
- type TokenRules
- type TranscriptionRequest
- type TranscriptionResponse
- type TranscriptionSegment
- type TranscriptionWord
- type UINode
- type VirtualModel
- type VirtualModelEntry
Constants ¶
const ( EndpointChatCompletions = "chat/completions" EndpointAudioTranscription = "audio/transcriptions" EndpointAudioSpeech = "audio/speech" EndpointImagesGenerations = "images/generations" EndpointEmbeddings = "embeddings" )
Endpoint identifiers listed in ModelInfo.Endpoints. The router rejects a request when the resolved model declares Endpoints and the target endpoint is absent; an empty Endpoints list means chat/completions only (legacy plugins predate the field).
const ( ExhaustedScopeAccount = "account" ExhaustedScopeModel = "model" ExhaustedScopeProxy = "proxy" )
ExhaustedScope dimensions nameable in the error contract scope field. Provider (the configured instance ID) and plugin are always part of every key and need no naming.
const ( ProxyModeDisabled = "disabled" ProxyModeAuto = "auto" ProxyModeManual = "manual" )
Mode constants for ProxyConfig.
const ( GeoModeFailFast = "fail_fast" GeoModeRetrySameKey = "retry_same_key" )
Geo mode constants for the provider-level geo policy stored in ProviderInstance.Config["geo"].
const CurrentVersion = "0.3.7"
CurrentVersion is the router version plugins declare compatibility with via the @router_version manifest tag.
0.0.5 adds the transcribe handler and llm_router.multipart. 0.0.6 adds the speech and generate_image handlers. 0.0.7 replaces the proxy pool: multi @proxy_location whitelist, @proxy_default_option, per-pair rate limits and blocks. 0.1.1 replaces per-pair proxy limits and credential quota marks with the unified exhausted store: joint limit keys with a scope field on the error contract. Old plugins are rejected by the manifest gate. 0.1.2 adds the payment_required error type for upstream paywalls. 0.2.0 adds DynamicForm Step gaps (flow/grid gap and spacer size accept Step int 0..8 alongside the legacy sm|md|lg enum), input_type=secret, around/evenly justify values, and strict link URL schemes. 0.3.0 reworks the provider error contract: timeout merges into upstream, content_policy/model_unavailable/structural_fault added, rate_limit and quota_exceeded require plugin-supplied retry_after, geo becomes an indefinite per-provider proxy ban with same-key retry on another region, auth/payment_required disable the credential, structural_fault disables the provider (DisabledBy/Reason/At on both). 0.3.1 restricts proxy-source candidates to unauthenticated HTTP URLs; the proxypool library owns proxy health and cache lifecycle. 0.3.4 permits proxy scope for quota_exceeded and retries proxy-scoped rate/quota failures on another proxy with the same credential. 0.3.5 adds the transport error type for connectivity failures and retries it with the same credential on another proxy. 0.3.6 adds the plugin model_specs table: pinned per-model rows merged over discovered catalog rows in GetModelInfos. 0.3.7 adds the overloaded error type for congested backends: distinct code, same-credential proxy retry, no marks or cooldown.
const DefaultGeoMaxProxies = 3
DefaultGeoMaxProxies bounds same-key geo retries when max_proxies is absent.
const MaxGeoMaxProxies = 10
MaxGeoMaxProxies caps same-key geo retries.
Variables ¶
This section is empty.
Functions ¶
func EmbeddingBase64 ¶ added in v0.0.5
EmbeddingBase64 encodes a float vector as base64 float32 little-endian, the OpenAI encoding_format=base64 wire form.
func NormalizeBaseURL ¶ added in v0.1.2
NormalizeBaseURL trims raw and requires an http(s) URL without a trailing slash. Single source of truth for base_url validation across provider CRUD and the custom backend.
func SpeechContentType ¶ added in v0.0.5
SpeechContentType maps a speech format to its wire Content-Type.
Types ¶
type AdminUser ¶
type AdminUser struct {
Username string `json:"username"`
PasswordHash string `json:"password_hash"`
CreatedAt time.Time `json:"created_at"`
}
AdminUser is the dashboard operator account.
type AuthFlowResult ¶
type AuthFlowResult struct {
Render []*UINode `json:"render,omitempty"`
RedirectURL string `json:"redirect_url,omitempty"`
Credentials map[string]any `json:"credentials,omitempty"`
}
AuthFlowResult is one of the three auth_step/auth_initiate outcomes, discriminated by which field is set.
type AuthStepInput ¶
type AuthStepInput struct {
Action string `json:"action"`
Values map[string]any `json:"values"`
FlowID string `json:"flow_id,omitempty"`
}
AuthStepInput is the submit payload for auth_step.
type ChatCompletionChoice ¶
type ChatCompletionChoice struct {
Index int `json:"index"`
Message ChatMessage `json:"message"`
FinishReason string `json:"finish_reason"`
Logprobs any `json:"logprobs,omitempty"`
}
type ChatCompletionRequest ¶
type ChatCompletionRequest struct {
Model ModelId `json:"model" example:"openai/gpt-4o"`
Messages []ChatMessage `json:"messages"`
Tools []ChatTool `json:"tools,omitempty"`
ToolChoice any `json:"tool_choice,omitempty"`
ParallelToolCalls *bool `json:"parallel_tool_calls,omitempty"`
Stream bool `json:"stream,omitempty" example:"false"`
StreamOptions *StreamOptions `json:"stream_options,omitempty"`
MaxTokens int `json:"max_tokens,omitempty" example:"1000"`
MaxCompletionTokens *int `json:"max_completion_tokens,omitempty"`
Temperature float64 `json:"temperature,omitempty" example:"0.7"`
TopP float64 `json:"top_p,omitempty" example:"1.0"`
N *int `json:"n,omitempty"`
Stop any `json:"stop,omitempty"`
Seed *int64 `json:"seed,omitempty"`
FrequencyPenalty *float64 `json:"frequency_penalty,omitempty"`
PresencePenalty *float64 `json:"presence_penalty,omitempty"`
Logprobs *bool `json:"logprobs,omitempty"`
TopLogprobs *int `json:"top_logprobs,omitempty"`
ResponseFormat any `json:"response_format,omitempty"`
User *string `json:"user,omitempty"`
ServiceTier *string `json:"service_tier,omitempty"`
ReasoningEffort *string `json:"reasoning_effort,omitempty"`
Verbosity *string `json:"verbosity,omitempty"`
}
ChatCompletionRequest is the incoming /v1/chat/completions body.
type ChatCompletionResponse ¶
type ChatCompletionResponse struct {
ID string `json:"id"`
Object string `json:"object"`
Created int64 `json:"created"`
Model string `json:"model"`
Choices []ChatCompletionChoice `json:"choices"`
Usage ChatCompletionUsage `json:"usage"`
SystemFingerprint *string `json:"system_fingerprint,omitempty"`
ServiceTier *string `json:"service_tier,omitempty"`
}
ChatCompletionResponse mirrors the OpenAI response schema.
type ChatCompletionUsage ¶
type ChatCompletionUsage struct {
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
TotalTokens int `json:"total_tokens"`
PromptTokensDetails *PromptTokensDetails `json:"prompt_tokens_details,omitempty"`
CompletionTokensDetails *CompletionTokensDetails `json:"completion_tokens_details,omitempty"`
}
type ChatMessage ¶
type ChatMessage struct {
Role string `json:"role" example:"user" enums:"system,user,assistant,tool,developer"`
Content string `json:"-" example:"Hello, how are you?"`
ContentParts []ChatMessageContentPart `json:"-"`
ToolCalls []ChatToolCall `json:"tool_calls,omitempty"`
ToolCallID string `json:"tool_call_id,omitempty"`
Name string `json:"name,omitempty"`
Refusal *string `json:"refusal,omitempty"`
// ReasoningContent carries chain-of-thought text exposed by reasoning
// models (DeepSeek `reasoning_content`, Groq `reasoning`).
ReasoningContent string `json:"reasoning_content,omitempty"`
}
ChatMessage is a single turn in a conversation.
func (ChatMessage) MarshalJSON ¶
func (m ChatMessage) MarshalJSON() ([]byte, error)
func (ChatMessage) TextContent ¶
func (m ChatMessage) TextContent() string
func (*ChatMessage) UnmarshalJSON ¶
func (m *ChatMessage) UnmarshalJSON(data []byte) error
type ChatMessageContentPart ¶
type ChatMessageContentPart struct {
Type string `json:"type,omitempty"`
Text string `json:"text,omitempty"`
ToolUseID string `json:"tool_use_id,omitempty"`
ID string `json:"id,omitempty"`
Name string `json:"name,omitempty"`
Input json.RawMessage `json:"input,omitempty"`
ImageURL *ChatMessageImageURL `json:"image_url,omitempty"`
InputAudio *ChatMessageInputAudio `json:"input_audio,omitempty"`
Content []ChatMessageContentPart `json:"-"`
ContentText string `json:"-"`
}
func (ChatMessageContentPart) MarshalJSON ¶
func (p ChatMessageContentPart) MarshalJSON() ([]byte, error)
func (ChatMessageContentPart) TextContent ¶
func (p ChatMessageContentPart) TextContent() string
func (*ChatMessageContentPart) UnmarshalJSON ¶
func (p *ChatMessageContentPart) UnmarshalJSON(data []byte) error
type ChatMessageImageURL ¶
type ChatMessageImageURL struct {
URL string `json:"url,omitempty"`
Detail string `json:"detail,omitempty"`
}
ChatMessageImageURL is an OpenAI image_url content part. URL is either a remote http(s) URL or a data: URL carrying base64 bytes inline.
type ChatMessageInputAudio ¶
type ChatMessageInputAudio struct {
Data string `json:"data,omitempty"`
Format string `json:"format,omitempty"`
}
ChatMessageInputAudio is an OpenAI input_audio content part. Data is base64 audio, Format is wav or mp3.
type ChatTool ¶
type ChatTool struct {
Type string `json:"type,omitempty"`
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
Parameters map[string]any `json:"parameters,omitempty"`
InputSchema map[string]any `json:"input_schema,omitempty"`
Function *ChatToolFunction `json:"function,omitempty"`
Strict *bool `json:"strict,omitempty"`
}
type ChatToolCall ¶
type ChatToolCall struct {
ID string `json:"id,omitempty"`
Type string `json:"type,omitempty"`
Function ChatToolFunction `json:"function"`
ExtraContent map[string]interface{} `json:"extra_content,omitempty"`
}
type ChatToolFunction ¶
type CompletionTokensDetails ¶
type CompletionTokensDetails struct {
ReasoningTokens int `json:"reasoning_tokens,omitempty"`
AudioTokens int `json:"audio_tokens,omitempty"`
}
CompletionTokensDetails is the OpenAI breakdown of completion-side token usage.
type Credential ¶
type Credential struct {
ID string `json:"id"`
ProviderID string `json:"provider_id"`
Label string `json:"label"`
Data map[string]any `json:"data"`
ExpiresAt *time.Time `json:"expires_at,omitempty"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
LastUsedAt *time.Time `json:"last_used_at,omitempty"`
RequestCount int64 `json:"request_count"`
SuccessCount int64 `json:"success_count"`
FailureCount int64 `json:"failure_count"`
// Disabled excludes the credential from routing and fallthrough.
Disabled bool `json:"disabled,omitempty"`
// DisabledBy names who disabled the credential: "admin" (dashboard PUT)
// or "system" (auth/payment_required auto-disable). Empty when enabled.
DisabledBy string `json:"disabled_by,omitempty"`
// DisabledReason carries the human reason (system cause or admin note).
DisabledReason string `json:"disabled_reason,omitempty"`
// DisabledAt marks when the credential was disabled. Nil when enabled.
DisabledAt *time.Time `json:"disabled_at,omitempty"`
// Order is the admin-defined pool position (1-based). 0 means unordered:
// unordered credentials sort after ordered ones by computed priority.
Order int `json:"order,omitempty"`
}
Credential holds provider-specific authentication data. Data is a flexible map to support any auth scheme, including non-string values such as expiry timestamps.
func (*Credential) DataString ¶
func (c *Credential) DataString(key string) string
DataString returns a string view of a data value for Go adapters that only understand string credentials.
func (*Credential) ExpiresIn ¶
func (c *Credential) ExpiresIn() time.Duration
ExpiresIn returns the duration until expiry (negative if already expired).
func (*Credential) IncrementUsage ¶
func (c *Credential) IncrementUsage(success bool)
IncrementUsage updates usage statistics after a request.
func (*Credential) IsExpired ¶
func (c *Credential) IsExpired() bool
IsExpired reports whether the credential has passed its expiry time.
func (*Credential) Priority ¶
func (c *Credential) Priority() int
Priority returns the selection priority for this credential: unused first, then used, expired last. Limit state lives in the exhausted store, never on the credential.
type CredentialScope ¶ added in v0.1.2
type CredentialScope struct {
ProviderID string `json:"provider_id"`
All bool `json:"all"`
CredentialIDs []string `json:"credential_ids,omitempty"`
}
CredentialScope grants a token access to credentials of one provider.
type Embedding ¶ added in v0.0.5
type Embedding struct {
Index int `json:"index"`
Values []float64 `json:"-"`
B64Values string `json:"-"`
}
func (Embedding) MarshalJSON ¶ added in v0.0.5
MarshalJSON renders one embedding entry in the OpenAI shape: float array by default, base64 float32-LE when the client asked for base64.
func (*Embedding) UnmarshalJSON ¶ added in v0.0.5
UnmarshalJSON accepts the plugin-side embedding entry: the OpenAI {index, embedding:[...]} shape with float values.
type EmbeddingsRequest ¶ added in v0.0.5
type EmbeddingsRequest struct {
Model ModelId `json:"model"`
Input []string `json:"-"`
EncodingFormat string `json:"encoding_format,omitempty"` // float (default) or base64
Dimensions int `json:"dimensions,omitempty"`
}
EmbeddingsRequest is the parsed body of POST /v1/embeddings. Input is normalized to a list of strings at the edge: OpenAI accepts a single string, a list of strings, or token arrays; token arrays are decoded by clients upstream of this router and are refused here.
func (*EmbeddingsRequest) UnmarshalJSON ¶ added in v0.0.5
func (r *EmbeddingsRequest) UnmarshalJSON(raw []byte) error
embeddingsRequestJSON decodes the polymorphic OpenAI input field.
type EmbeddingsResponse ¶ added in v0.0.5
type EmbeddingsResponse struct {
Data []Embedding `json:"data"`
Model string `json:"model"`
Usage *EmbeddingsUsage `json:"usage,omitempty"`
}
EmbeddingsResponse is the normalized plugin return and the client wire response. Plugins always return float vectors; the edge renders encoding_format=base64 from them.
type EmbeddingsUsage ¶ added in v0.0.5
type ErrorResponse ¶
type ErrorResponse struct {
Error string `json:"error" example:"invalid request"`
}
ErrorResponse is the standard error response format for all API endpoints.
type ErrorType ¶
type ErrorType int
ErrorType classifies provider errors for key iteration inside backends.
const ( ErrorTypeUnknown ErrorType = iota ErrorTypeRateLimit // Temporary rate limit; plugin MUST supply RetryAfter ErrorTypeQuotaExceeded // Quota exhausted; plugin MUST supply RetryAfter ErrorTypeAuth // Auth failure: disable the credential (system) ErrorTypeUpstream // Transient upstream failure (5xx,timeout, reset) ErrorTypeTransport // Connectivity failure (EOF, reset, broken tunnel); carries no state, retries on another proxy ErrorTypeInvalidRequest // Malformed request; fatal for the pool ErrorTypeGeo // Exit geo-blocked: indefinite (provider, proxy) ban, same-key retry on another region ErrorTypeNotFound // Model does not exist upstream; drop it from the cache ErrorTypePaymentRequired // Upstream paywall: disable the credential (system) ErrorTypeContentPolicy // Upstream rejected the content; fatal for the pool ErrorTypeOverloaded // Backend congested; retryable, no marks or cooldown ErrorTypeStructuralFault // Provider config/network broken for all keys and models; disable the provider (system), fatal for the pool )
type ExhaustedEntry ¶ added in v0.1.2
type ExhaustedEntry struct {
Key string `json:"key"`
ResetsAt time.Time `json:"resets_at"`
Reason string `json:"reason,omitempty"`
}
ExhaustedEntry is one joint limit key: the stored dimensions act as a filter, and a candidate combination matching every stored dimension is skipped until ResetsAt passes.
type GeoBanEntry ¶ added in v0.2.0
type GeoBanEntry struct {
Key string `json:"key"`
Plugin string `json:"plugin"`
Provider string `json:"provider"`
Proxy string `json:"proxy"`
Reason string `json:"reason,omitempty"`
BannedAt time.Time `json:"banned_at"`
}
GeoBanEntry is one indefinite geo-block flag: proxy ProxyID is unusable for provider type Provider of plugin Plugin. No expiry: the flag lives until an admin clears it explicitly.
type GeoConfig ¶ added in v0.2.0
type GeoConfig struct {
Mode string `json:"mode"` // fail_fast (default) or retry_same_key
MaxProxies int `json:"max_proxies,omitempty"`
}
GeoConfig is the provider-level geo-block reaction policy.
type ImageGenerationRequest ¶ added in v0.0.5
type ImageGenerationRequest struct {
Model ModelId `json:"model"`
Prompt string `json:"prompt"`
N int `json:"n,omitempty"`
Size string `json:"size,omitempty"`
Quality string `json:"quality,omitempty"`
Style string `json:"style,omitempty"`
ResponseFormat string `json:"response_format,omitempty"` // url or b64_json
}
ImageGenerationRequest is the parsed body of POST /v1/images/generations.
type ImageGenerationResponse ¶ added in v0.0.5
type ImageGenerationResponse struct {
Created int64 `json:"created"`
Data []ImageData `json:"data"`
}
ImageGenerationResponse is the normalized plugin return and the client wire response: the OpenAI images shape.
type MetricEvent ¶
type MetricEvent struct {
Timestamp time.Time
ProviderID string
ProviderType string
Model ModelId
TokenID string
Duration time.Duration
StatusCode int
TokensInput int64
TokensOutput int64
ErrorType string
}
MetricEvent represents a single API request event.
type MetricsFilters ¶
type MetricsFilters struct {
ProviderID string `json:"provider_id"`
Model ModelId `json:"model"`
TimeRange TimeRange `json:"time_range"`
}
MetricsFilters for querying metrics.
type MetricsOverview ¶
type MetricsOverview struct {
TotalRequests int64 `json:"total_requests" example:"1500"`
TotalErrors int64 `json:"total_errors" example:"23"`
PeakRequests int64 `json:"peak_requests" example:"45"`
PeakInputTokens int64 `json:"peak_input_tokens" example:"12000"`
PeakOutputTokens int64 `json:"peak_output_tokens" example:"3500"`
}
MetricsOverview for dashboard display.
type ModelId ¶
type ModelId string
ModelId uniquely identifies a model in the format: provider/model-name[:version]
func (ModelId) Name ¶ added in v0.1.2
Name returns the model name without the provider prefix: everything after the first '/'. Plugins sending the bare name upstream use this instead of hand-rolled stripping. Invalid ids yield the full id unchanged.
func (ModelId) Parse ¶
Parse splits a ModelId into providerID and model name. Both parts must be non-empty: "/model" and "provider/" are rejected instead of routing with an empty half.
type ModelInfo ¶
type ModelInfo struct {
Name string `json:"name"`
DisplayName string `json:"display_name"`
Description string `json:"description,omitempty"`
RPM int64 `json:"rpm"`
TPM int64 `json:"tpm"`
RPD int64 `json:"rpd"`
ContextWindow int64 `json:"context_window,omitempty"`
MaxTokens int64 `json:"max_tokens,omitempty"`
// Capabilities are UI-facing feature chips (tools, json_mode, ...).
// Derived from the fields below when the plugin does not set them.
// Modalities stay out: they render in their own column.
Capabilities []string `json:"capabilities,omitempty"`
InputModalities []string `json:"input_modalities,omitempty"`
OutputModalities []string `json:"output_modalities,omitempty"`
SupportedParameters []string `json:"supported_parameters,omitempty"`
Reasoning *ModelReasoning `json:"reasoning,omitempty"`
// Endpoints lists the served endpoint identifiers (Endpoint* constants).
// Empty means chat/completions only.
Endpoints []string `json:"endpoints,omitempty"`
}
ModelInfo contains metadata about a specific model. Field style follows the OpenRouter model schema where applicable.
func (*ModelInfo) DeriveCapabilities ¶
func (m *ModelInfo) DeriveCapabilities()
DeriveCapabilities fills Capabilities from the OpenRouter-style fields. Explicit plugin-provided capabilities win.
func (*ModelInfo) SupportsEndpoint ¶ added in v0.0.5
SupportsEndpoint reports whether the model serves endpoint. Empty Endpoints implies chat/completions only.
type ModelOverride ¶
type ModelOverride struct {
ProviderID string `json:"provider_id"`
Name string `json:"name"`
Disabled bool `json:"disabled,omitempty"`
Custom bool `json:"custom,omitempty"`
DisplayName string `json:"display_name,omitempty"`
Capabilities []string `json:"capabilities,omitempty"`
// InputModalities and OutputModalities hold probe-verified modalities,
// keyed by full model id. Applied over upstream data when non-empty.
InputModalities []string `json:"input_modalities,omitempty"`
OutputModalities []string `json:"output_modalities,omitempty"`
}
ModelOverride is the per-provider admin override for a single model: disable from routing and /v1/models, register a custom model the upstream does not list, or adjust display metadata.
type ModelReasoning ¶
type ModelReasoning struct {
// SupportedEfforts in descending effort order (highest first).
SupportedEfforts []string `json:"supported_efforts,omitempty"`
DefaultEffort string `json:"default_effort,omitempty"`
DefaultEnabled bool `json:"default_enabled,omitempty"`
Mandatory bool `json:"mandatory,omitempty"`
}
ModelReasoning describes reasoning support of a model (OpenRouter-style).
type OpenAIError ¶
type OpenAIError struct {
Error OpenAIErrorBody `json:"error"`
}
OpenAIError wraps error responses in the OpenAI error format.
type OpenAIErrorBody ¶
type PluginInternalError ¶
PluginInternalError is a caught Lua failure at the Go/Lua boundary.
func (*PluginInternalError) Error ¶
func (e *PluginInternalError) Error() string
type PromptTokensDetails ¶
type PromptTokensDetails struct {
CachedTokens int `json:"cached_tokens,omitempty"`
AudioTokens int `json:"audio_tokens,omitempty"`
}
PromptTokensDetails is the OpenAI breakdown of prompt-side token usage.
type Provider ¶
type Provider = ProviderInstance
Provider is kept as an alias so existing call sites keep compiling while the field migration (Type -> TypeKey, BaseURL -> Config) proceeds.
type ProviderError ¶
type ProviderError struct {
StatusCode int
Message string
Type ErrorType
RetryAfter *time.Time
// Scope names the exhausted dimensions the error limits (account, model,
// proxy). Allowed only on rate_limit and quota_exceeded. Empty marks the
// full combination. Proxy-scoped limits can retry on another proxy.
// Any scope on other types rejects the error table as a plugin bug.
Scope []string
// UpstreamStatus is the raw HTTP status received from the upstream.
// UpstreamBody is the raw upstream response body (possibly truncated by
// the HTTP client). Routing never reads these; they exist for logging
// and debugging. Large bodies are spilled to disk in debug mode with
// the path carried alongside the log line.
UpstreamStatus int
UpstreamBody string
}
ProviderError represents errors returned by provider backends.
func (*ProviderError) Error ¶
func (e *ProviderError) Error() string
type ProviderInstance ¶
type ProviderInstance struct {
ID string `json:"id" example:"opencode-zen"`
Name string `json:"name" example:"OpenCode Zen"`
TypeKey string `json:"type_key" example:"opencode-zen"`
Qualifier string `json:"qualifier" example:""`
Config map[string]any `json:"config"`
IconURL string `json:"icon_url" example:"https://cdn.example.com/openai.svg"`
IsUIReadonly bool `json:"is_ui_readonly"`
IsUIHidden bool `json:"is_ui_hidden"`
// Disabled takes the provider out of routing and model listings.
// Settings and discovery keep working.
Disabled bool `json:"disabled,omitempty"`
// DisabledBy names who disabled the provider: "admin" (dashboard PUT)
// or "system" (structural_fault auto-disable). Empty when enabled.
DisabledBy string `json:"disabled_by,omitempty"`
// DisabledReason carries the human reason (system cause or admin note).
DisabledReason string `json:"disabled_reason,omitempty"`
// DisabledAt marks when the provider was disabled. Nil when enabled.
DisabledAt *time.Time `json:"disabled_at,omitempty"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
ProviderInstance is the single persisted provider record for every type: lua-plugin type keys, built-in "custom" and "virtual".
func (*ProviderInstance) BaseURL ¶
func (p *ProviderInstance) BaseURL() string
BaseURL returns Config["base_url"] for OpenAI-compatible providers.
type ProviderInstanceCreateRequest ¶
type ProviderInstanceCreateRequest struct {
Name string `json:"name"`
TypeKey string `json:"type_key"`
Qualifier string `json:"qualifier"`
Config map[string]any `json:"config"`
IconURL string `json:"icon_url"`
}
ProviderInstanceCreateRequest is the generic create body for any provider type.
type ProviderInstanceUpdateRequest ¶
type ProviderInstanceUpdateRequest struct {
Name string `json:"name"`
Config map[string]any `json:"config"`
IconURL string `json:"icon_url"`
Disabled *bool `json:"disabled"`
}
ProviderInstanceUpdateRequest is the generic update body.
type ProviderStats ¶
type ProviderStats struct {
ModelCount int `json:"model_count" example:"5"`
CredentialCount int `json:"credential_count" example:"2"`
}
ProviderStats holds aggregated statistics for a provider.
type ProxyCandidate ¶
type ProxyCandidate struct {
Protocol string `json:"protocol"`
Host string `json:"host"`
Port int `json:"port"`
Country string `json:"country"`
}
ProxyCandidate is one proxy entry produced by a source plugin.
type ProxyConfig ¶
type ProxyConfig struct {
Mode string `json:"mode"` // "disabled" (default), "auto", "manual"
IDs []string `json:"ids,omitempty"` // manual mode: chosen proxy IDs
}
ProxyConfig is the provider-level proxy mode stored in ProviderInstance.Config.
func ParseProxyConfig ¶ added in v0.1.2
func ParseProxyConfig(providerConfig map[string]any) (ProxyConfig, error)
ParseProxyConfig reads the provider-level proxy policy from a provider config map. Absent or non-map proxy sections mean disabled; an explicitly unknown mode is an error, never a silent fallback to direct.
type RouterConfiguration ¶
type RouterConfiguration struct {
IsClusterNode bool `json:"is_cluster_node"`
DisableTelemetry bool `json:"disable_telemetry"`
// ModelsFilter remembers the dashboard model visibility filter
// ("all", "enabled", "disabled"). Empty means "all".
ModelsFilter string `json:"models_filter,omitempty"`
}
RouterConfiguration holds deployment-wide instance settings.
func (RouterConfiguration) Validate ¶
func (c RouterConfiguration) Validate() error
Validate checks the configuration ranges.
type RouterToken ¶
type RouterToken struct {
ID string `json:"id" example:"token123"`
Name string `json:"name" example:"Production API Token"`
Token string `json:"token,omitempty" example:"llmr_abc123def456"`
TokenHash string `json:"token_hash,omitempty"`
Rules TokenRules `json:"rules"`
CreatedAt time.Time `json:"created_at" example:"2026-05-04T15:00:00Z"`
}
RouterToken is an opaque bearer token issued by llm-router itself.
type SpeechRequest ¶ added in v0.0.5
type SpeechRequest struct {
Model ModelId `json:"model"`
Input string `json:"input"`
Voice string `json:"voice"`
ResponseFormat string `json:"response_format,omitempty"` // mp3 (default), opus, aac, flac, wav, pcm
Speed *float64 `json:"speed,omitempty"`
Instructions string `json:"instructions,omitempty"`
}
SpeechRequest is the parsed body of POST /v1/audio/speech.
type SpeechResponse ¶ added in v0.0.5
SpeechResponse is the normalized plugin return: raw audio bytes plus the format they are encoded in. The router serves the bytes with the matching Content-Type; no transcoding happens inside the router.
type StreamChunk ¶
type StreamChunk struct {
ID string `json:"id"`
Object string `json:"object"`
Created int64 `json:"created"`
Model string `json:"model"`
Choices []StreamChunkChoice `json:"choices"`
Usage *ChatCompletionUsage `json:"usage,omitempty"`
}
StreamChunk is a single SSE data payload for streaming responses.
type StreamChunkChoice ¶
type StreamChunkChoice struct {
Index int `json:"index"`
Delta ChatMessage `json:"delta"`
FinishReason *string `json:"finish_reason"`
Logprobs any `json:"logprobs,omitempty"`
}
type StreamOptions ¶
type StreamOptions struct {
IncludeUsage bool `json:"include_usage,omitempty"`
}
StreamOptions for streaming
type TimeSeriesPoint ¶
type TimeSeriesPoint struct {
Timestamp time.Time `json:"timestamp" example:"2026-05-04T19:00:00Z"`
Value int64 `json:"value" example:"42"`
}
TimeSeriesPoint for chart data.
type TokenRules ¶
type TokenRules struct {
AllowedProviders []string `json:"allowed_providers"`
AllowAllProviders bool `json:"allow_all_providers"`
AllowedModels []ModelId `json:"allowed_models"`
AllowAllModels bool `json:"allow_all_models"`
AllowedCredentials []string `json:"allowed_credentials"`
AllowAllCredentials bool `json:"allow_all_credentials"`
// CredentialScopes grants credential access per provider: All covers
// every current and future key of the provider, CredentialIDs pins
// specific keys. Scopes union with the legacy flat credential fields.
CredentialScopes []CredentialScope `json:"credential_scopes,omitempty"`
}
TokenRules constrains what an API token is allowed to do.
func (TokenRules) Allows ¶
func (r TokenRules) Allows(model ModelId) bool
Allows reports whether the token may request the given model.
func (TokenRules) AllowsCredential ¶
func (r TokenRules) AllowsCredential(providerID, credentialID string) bool
AllowsCredential reports whether the token may use the given credential of the given provider. Scopes union with the legacy flat fields: either source granting access allows the credential.
func (TokenRules) AllowsProvider ¶
func (r TokenRules) AllowsProvider(providerID string) bool
AllowsProvider reports whether the token may use the given provider.
type TranscriptionRequest ¶ added in v0.0.5
type TranscriptionRequest struct {
Model ModelId
File []byte
FileName string
ContentType string
Language string
Prompt string
ResponseFormat string // json (default), text, srt, verbose_json, vtt
Temperature *float64
TimestampGranularities []string // word, segment
}
TranscriptionRequest is the parsed multipart body of POST /v1/audio/transcriptions. File holds the raw audio bytes.
type TranscriptionResponse ¶ added in v0.0.5
type TranscriptionResponse struct {
Text string `json:"text"`
Language string `json:"language,omitempty"`
Duration float64 `json:"duration,omitempty"`
Segments []TranscriptionSegment `json:"segments,omitempty"`
Words []TranscriptionWord `json:"words,omitempty"`
}
TranscriptionResponse is the normalized plugin return: the OpenAI verbose_json shape. The router renders the client-facing response_format from these fields.
func (*TranscriptionResponse) SRT ¶ added in v0.0.5
func (r *TranscriptionResponse) SRT() string
SRT renders segments as a SubRip subtitle document.
func (*TranscriptionResponse) VTT ¶ added in v0.0.5
func (r *TranscriptionResponse) VTT() string
VTT renders segments as a WebVTT subtitle document.
type TranscriptionSegment ¶ added in v0.0.5
type TranscriptionSegment struct {
ID int `json:"id"`
Seek int `json:"seek,omitempty"`
Start float64 `json:"start"`
End float64 `json:"end"`
Text string `json:"text"`
Tokens []int `json:"tokens,omitempty"`
Temperature float64 `json:"temperature,omitempty"`
AvgLogprob float64 `json:"avg_logprob,omitempty"`
CompressionRatio float64 `json:"compression_ratio,omitempty"`
NoSpeechProb float64 `json:"no_speech_prob,omitempty"`
}
type TranscriptionWord ¶ added in v0.0.5
type UINode ¶
type UINode struct {
Type string `json:"type"`
Text string `json:"text,omitempty"`
Name string `json:"name,omitempty"`
Label string `json:"label,omitempty"`
InputType string `json:"input_type,omitempty"`
Required bool `json:"required,omitempty"`
Options []string `json:"options,omitempty"`
OptionLabels map[string]string `json:"option_labels,omitempty"`
URL string `json:"url,omitempty"`
Variant string `json:"variant,omitempty"`
FormAction string `json:"form_action,omitempty"`
Content []*UINode `json:"content,omitempty"`
Placeholder string `json:"placeholder,omitempty"`
Value any `json:"value,omitempty"`
Direction string `json:"direction,omitempty"`
Align string `json:"align,omitempty"`
Justify string `json:"justify,omitempty"`
Gap int `json:"gap,omitempty"`
Columns int `json:"columns,omitempty"`
Title string `json:"title,omitempty"`
Subtitle string `json:"subtitle,omitempty"`
Wrap bool `json:"wrap,omitempty"`
Grow bool `json:"grow,omitempty"`
Size int `json:"size,omitempty"`
Extra map[string]any `json:"-"`
}
UINode is a single node of the lua-driven UI tree shared by config_schema, credential_schema and auth wizards.
type VirtualModel ¶ added in v0.0.5
type VirtualModel struct {
ID string `json:"id"`
Name string `json:"name"`
Description string `json:"description"`
Models []VirtualModelEntry `json:"models"`
// Instruction is prepended to requests as the first user message.
Instruction string `json:"instruction"`
// ManagedBy marks provider-maintained virtual models
// ("provider:<provider-id>:<endpoint-slug>"). Managed models resolve
// their member list live from the provider list on every call; stored
// members are ignored. Empty means manually maintained.
ManagedBy string `json:"managed_by,omitempty"`
// Disabled takes the virtual model out of routing. Probes still run.
Disabled bool `json:"disabled,omitempty"`
Version int `json:"version"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
VirtualModel routes requests to its models in list order (fall-through) and reports an error only when every model failed.
type VirtualModelEntry ¶ added in v0.0.5
type VirtualModelEntry struct {
ModelID ModelId `json:"model_id"`
}
VirtualModelEntry is one fall-through step. List order is the priority.