Documentation
¶
Overview ¶
Package models defines all shared data structures for llm-router.
Index ¶
- Constants
- func EmbeddingBase64(values []float64) string
- func NormalizeBaseURL(raw string) (string, error)
- func SpeechContentType(format string) string
- type ActiveRegion
- type AdminUser
- type AuthFlowResult
- type AuthStepInput
- type ChatCompletionChoice
- type ChatCompletionRequest
- type ChatCompletionResponse
- type ChatCompletionUsage
- type ChatMessage
- type ChatMessageContentPart
- type ChatMessageImageURL
- type ChatMessageInputAudio
- type ChatTool
- type ChatToolCall
- type ChatToolFunction
- type CompletionTokensDetails
- type Credential
- type CredentialScope
- type Embedding
- type EmbeddingsRequest
- type EmbeddingsResponse
- type EmbeddingsUsage
- type ErrorResponse
- type ErrorType
- type ExhaustedEntry
- type GeoBanEntry
- type GeoConfig
- type ImageData
- type ImageGenerationRequest
- type ImageGenerationResponse
- type MetricEvent
- type MetricsFilters
- type MetricsOverview
- type ModelId
- type ModelInfo
- type ModelOverride
- type ModelReasoning
- type OpenAIError
- type OpenAIErrorBody
- type PluginInternalError
- type PromptTokensDetails
- type Provider
- type ProviderError
- type ProviderInstance
- type ProviderInstanceCreateRequest
- type ProviderInstanceUpdateRequest
- type ProviderStats
- type Proxy
- type ProxyCandidate
- type ProxyConfig
- type RouterConfiguration
- type RouterToken
- type SpeechRequest
- type SpeechResponse
- type StreamChunk
- type StreamChunkChoice
- type StreamOptions
- type TimeRange
- type TimeSeriesPoint
- type TokenRules
- type TranscriptionRequest
- type TranscriptionResponse
- type TranscriptionSegment
- type TranscriptionWord
- type UINode
- type VirtualModel
- type VirtualModelEntry
Constants ¶
const ( EndpointChatCompletions = "chat/completions" EndpointAudioTranscription = "audio/transcriptions" EndpointAudioSpeech = "audio/speech" EndpointImagesGenerations = "images/generations" EndpointEmbeddings = "embeddings" )
Endpoint identifiers listed in ModelInfo.Endpoints. The router rejects a request when the resolved model declares Endpoints and the target endpoint is absent; an empty Endpoints list means chat/completions only (legacy plugins predate the field).
const ( ExhaustedScopeAccount = "account" ExhaustedScopeModel = "model" ExhaustedScopeProxy = "proxy" )
ExhaustedScope dimensions nameable in the error contract scope field. Provider and plugin are always part of every key and need no naming.
const ( ProxyModeDisabled = "disabled" ProxyModeAuto = "auto" ProxyModeManual = "manual" )
Mode constants for ProxyConfig.
const ( GeoModeFailFast = "fail_fast" GeoModeRetrySameKey = "retry_same_key" )
Geo mode constants for the provider-level geo policy stored in ProviderInstance.Config["geo"].
const ( DefaultMinDownloadSpeedKbps int64 = 15000 DefaultMaxProxiesPerLocation = 10 DefaultUpdateIntervalMinutes = 15 )
Default proxy pool settings.
const CurrentVersion = "0.3.0"
CurrentVersion is the router version plugins declare compatibility with via the @router_version manifest tag.
0.0.5 adds the transcribe handler and llm_router.multipart. 0.0.6 adds the speech and generate_image handlers. 0.0.7 replaces the proxy pool: multi @proxy_location whitelist, @proxy_default_option, per-pair rate limits and blocks. 0.1.1 replaces per-pair proxy limits and credential quota marks with the unified exhausted store: joint limit keys with a scope field on the error contract. Old plugins are rejected by the manifest gate. 0.1.2 adds the payment_required error type for upstream paywalls. 0.2.0 adds DynamicForm Step gaps (flow/grid gap and spacer size accept Step int 0..8 alongside the legacy sm|md|lg enum), input_type=secret, around/evenly justify values, and strict link URL schemes. 0.3.0 reworks the provider error contract: timeout merges into upstream, content_policy/model_unavailable/structural_fault added, rate_limit and quota_exceeded require plugin-supplied retry_after, geo becomes an indefinite per-provider proxy ban with same-key retry on another region, auth/payment_required disable the credential, structural_fault disables the provider (DisabledBy/Reason/At on both).
const DefaultGeoMaxProxies = 3
DefaultGeoMaxProxies bounds same-key geo retries when max_proxies is absent.
const MaxGeoMaxProxies = 10
MaxGeoMaxProxies caps same-key geo retries.
Variables ¶
This section is empty.
Functions ¶
func EmbeddingBase64 ¶ added in v0.0.5
EmbeddingBase64 encodes a float vector as base64 float32 little-endian, the OpenAI encoding_format=base64 wire form.
func NormalizeBaseURL ¶ added in v0.1.2
NormalizeBaseURL trims raw and requires an http(s) URL without a trailing slash. Single source of truth for base_url validation across provider CRUD and the custom backend.
func SpeechContentType ¶ added in v0.0.5
SpeechContentType maps a speech format to its wire Content-Type.
Types ¶
type ActiveRegion ¶ added in v0.0.7
ActiveRegion is one demanded proxy exit location.
type AdminUser ¶
type AdminUser struct {
Username string `json:"username"`
PasswordHash string `json:"password_hash"`
CreatedAt time.Time `json:"created_at"`
}
AdminUser is the dashboard operator account.
type AuthFlowResult ¶
type AuthFlowResult struct {
Render []*UINode `json:"render,omitempty"`
RedirectURL string `json:"redirect_url,omitempty"`
Credentials map[string]any `json:"credentials,omitempty"`
}
AuthFlowResult is one of the three auth_step/auth_initiate outcomes, discriminated by which field is set.
type AuthStepInput ¶
type AuthStepInput struct {
Action string `json:"action"`
Values map[string]any `json:"values"`
FlowID string `json:"flow_id,omitempty"`
}
AuthStepInput is the submit payload for auth_step.
type ChatCompletionChoice ¶
type ChatCompletionChoice struct {
Index int `json:"index"`
Message ChatMessage `json:"message"`
FinishReason string `json:"finish_reason"`
Logprobs any `json:"logprobs,omitempty"`
}
type ChatCompletionRequest ¶
type ChatCompletionRequest struct {
Model ModelId `json:"model" example:"openai/gpt-4o"`
Messages []ChatMessage `json:"messages"`
Tools []ChatTool `json:"tools,omitempty"`
ToolChoice any `json:"tool_choice,omitempty"`
ParallelToolCalls *bool `json:"parallel_tool_calls,omitempty"`
Stream bool `json:"stream,omitempty" example:"false"`
StreamOptions *StreamOptions `json:"stream_options,omitempty"`
MaxTokens int `json:"max_tokens,omitempty" example:"1000"`
MaxCompletionTokens *int `json:"max_completion_tokens,omitempty"`
Temperature float64 `json:"temperature,omitempty" example:"0.7"`
TopP float64 `json:"top_p,omitempty" example:"1.0"`
N *int `json:"n,omitempty"`
Stop any `json:"stop,omitempty"`
Seed *int64 `json:"seed,omitempty"`
FrequencyPenalty *float64 `json:"frequency_penalty,omitempty"`
PresencePenalty *float64 `json:"presence_penalty,omitempty"`
Logprobs *bool `json:"logprobs,omitempty"`
TopLogprobs *int `json:"top_logprobs,omitempty"`
ResponseFormat any `json:"response_format,omitempty"`
User *string `json:"user,omitempty"`
ServiceTier *string `json:"service_tier,omitempty"`
ReasoningEffort *string `json:"reasoning_effort,omitempty"`
Verbosity *string `json:"verbosity,omitempty"`
}
ChatCompletionRequest is the incoming /v1/chat/completions body.
type ChatCompletionResponse ¶
type ChatCompletionResponse struct {
ID string `json:"id"`
Object string `json:"object"`
Created int64 `json:"created"`
Model string `json:"model"`
Choices []ChatCompletionChoice `json:"choices"`
Usage ChatCompletionUsage `json:"usage"`
SystemFingerprint *string `json:"system_fingerprint,omitempty"`
ServiceTier *string `json:"service_tier,omitempty"`
}
ChatCompletionResponse mirrors the OpenAI response schema.
type ChatCompletionUsage ¶
type ChatCompletionUsage struct {
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
TotalTokens int `json:"total_tokens"`
PromptTokensDetails *PromptTokensDetails `json:"prompt_tokens_details,omitempty"`
CompletionTokensDetails *CompletionTokensDetails `json:"completion_tokens_details,omitempty"`
}
type ChatMessage ¶
type ChatMessage struct {
Role string `json:"role" example:"user" enums:"system,user,assistant,tool,developer"`
Content string `json:"-" example:"Hello, how are you?"`
ContentParts []ChatMessageContentPart `json:"-"`
ToolCalls []ChatToolCall `json:"tool_calls,omitempty"`
ToolCallID string `json:"tool_call_id,omitempty"`
Name string `json:"name,omitempty"`
Refusal *string `json:"refusal,omitempty"`
// ReasoningContent carries chain-of-thought text exposed by reasoning
// models (DeepSeek `reasoning_content`, Groq `reasoning`).
ReasoningContent string `json:"reasoning_content,omitempty"`
}
ChatMessage is a single turn in a conversation.
func (ChatMessage) MarshalJSON ¶
func (m ChatMessage) MarshalJSON() ([]byte, error)
func (ChatMessage) TextContent ¶
func (m ChatMessage) TextContent() string
func (*ChatMessage) UnmarshalJSON ¶
func (m *ChatMessage) UnmarshalJSON(data []byte) error
type ChatMessageContentPart ¶
type ChatMessageContentPart struct {
Type string `json:"type,omitempty"`
Text string `json:"text,omitempty"`
ToolUseID string `json:"tool_use_id,omitempty"`
ID string `json:"id,omitempty"`
Name string `json:"name,omitempty"`
Input json.RawMessage `json:"input,omitempty"`
ImageURL *ChatMessageImageURL `json:"image_url,omitempty"`
InputAudio *ChatMessageInputAudio `json:"input_audio,omitempty"`
Content []ChatMessageContentPart `json:"-"`
ContentText string `json:"-"`
}
func (ChatMessageContentPart) MarshalJSON ¶
func (p ChatMessageContentPart) MarshalJSON() ([]byte, error)
func (ChatMessageContentPart) TextContent ¶
func (p ChatMessageContentPart) TextContent() string
func (*ChatMessageContentPart) UnmarshalJSON ¶
func (p *ChatMessageContentPart) UnmarshalJSON(data []byte) error
type ChatMessageImageURL ¶
type ChatMessageImageURL struct {
URL string `json:"url,omitempty"`
Detail string `json:"detail,omitempty"`
}
ChatMessageImageURL is an OpenAI image_url content part. URL is either a remote http(s) URL or a data: URL carrying base64 bytes inline.
type ChatMessageInputAudio ¶
type ChatMessageInputAudio struct {
Data string `json:"data,omitempty"`
Format string `json:"format,omitempty"`
}
ChatMessageInputAudio is an OpenAI input_audio content part. Data is base64 audio, Format is wav or mp3.
type ChatTool ¶
type ChatTool struct {
Type string `json:"type,omitempty"`
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
Parameters map[string]any `json:"parameters,omitempty"`
InputSchema map[string]any `json:"input_schema,omitempty"`
Function *ChatToolFunction `json:"function,omitempty"`
Strict *bool `json:"strict,omitempty"`
}
type ChatToolCall ¶
type ChatToolCall struct {
ID string `json:"id,omitempty"`
Type string `json:"type,omitempty"`
Function ChatToolFunction `json:"function"`
ExtraContent map[string]interface{} `json:"extra_content,omitempty"`
}
type ChatToolFunction ¶
type CompletionTokensDetails ¶
type CompletionTokensDetails struct {
ReasoningTokens int `json:"reasoning_tokens,omitempty"`
AudioTokens int `json:"audio_tokens,omitempty"`
}
CompletionTokensDetails is the OpenAI breakdown of completion-side token usage.
type Credential ¶
type Credential struct {
ID string `json:"id"`
ProviderID string `json:"provider_id"`
Label string `json:"label"`
Data map[string]any `json:"data"`
ExpiresAt *time.Time `json:"expires_at,omitempty"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
LastUsedAt *time.Time `json:"last_used_at,omitempty"`
RequestCount int64 `json:"request_count"`
SuccessCount int64 `json:"success_count"`
FailureCount int64 `json:"failure_count"`
// Disabled excludes the credential from routing and fallthrough.
Disabled bool `json:"disabled,omitempty"`
// DisabledBy names who disabled the credential: "admin" (dashboard PUT)
// or "system" (auth/payment_required auto-disable). Empty when enabled.
DisabledBy string `json:"disabled_by,omitempty"`
// DisabledReason carries the human reason (system cause or admin note).
DisabledReason string `json:"disabled_reason,omitempty"`
// DisabledAt marks when the credential was disabled. Nil when enabled.
DisabledAt *time.Time `json:"disabled_at,omitempty"`
// Order is the admin-defined pool position (1-based). 0 means unordered:
// unordered credentials sort after ordered ones by computed priority.
Order int `json:"order,omitempty"`
}
Credential holds provider-specific authentication data. Data is a flexible map to support any auth scheme, including non-string values such as expiry timestamps.
func (*Credential) DataString ¶
func (c *Credential) DataString(key string) string
DataString returns a string view of a data value for Go adapters that only understand string credentials.
func (*Credential) ExpiresIn ¶
func (c *Credential) ExpiresIn() time.Duration
ExpiresIn returns the duration until expiry (negative if already expired).
func (*Credential) IncrementUsage ¶
func (c *Credential) IncrementUsage(success bool)
IncrementUsage updates usage statistics after a request.
func (*Credential) IsExpired ¶
func (c *Credential) IsExpired() bool
IsExpired reports whether the credential has passed its expiry time.
func (*Credential) Priority ¶
func (c *Credential) Priority() int
Priority returns the selection priority for this credential: unused first, then used, expired last. Limit state lives in the exhausted store, never on the credential.
type CredentialScope ¶ added in v0.1.2
type CredentialScope struct {
ProviderID string `json:"provider_id"`
All bool `json:"all"`
CredentialIDs []string `json:"credential_ids,omitempty"`
}
CredentialScope grants a token access to credentials of one provider.
type Embedding ¶ added in v0.0.5
type Embedding struct {
Index int `json:"index"`
Values []float64 `json:"-"`
B64Values string `json:"-"`
}
func (Embedding) MarshalJSON ¶ added in v0.0.5
MarshalJSON renders one embedding entry in the OpenAI shape: float array by default, base64 float32-LE when the client asked for base64.
func (*Embedding) UnmarshalJSON ¶ added in v0.0.5
UnmarshalJSON accepts the plugin-side embedding entry: the OpenAI {index, embedding:[...]} shape with float values.
type EmbeddingsRequest ¶ added in v0.0.5
type EmbeddingsRequest struct {
Model ModelId `json:"model"`
Input []string `json:"-"`
EncodingFormat string `json:"encoding_format,omitempty"` // float (default) or base64
Dimensions int `json:"dimensions,omitempty"`
}
EmbeddingsRequest is the parsed body of POST /v1/embeddings. Input is normalized to a list of strings at the edge: OpenAI accepts a single string, a list of strings, or token arrays; token arrays are decoded by clients upstream of this router and are refused here.
func (*EmbeddingsRequest) UnmarshalJSON ¶ added in v0.0.5
func (r *EmbeddingsRequest) UnmarshalJSON(raw []byte) error
embeddingsRequestJSON decodes the polymorphic OpenAI input field.
type EmbeddingsResponse ¶ added in v0.0.5
type EmbeddingsResponse struct {
Data []Embedding `json:"data"`
Model string `json:"model"`
Usage *EmbeddingsUsage `json:"usage,omitempty"`
}
EmbeddingsResponse is the normalized plugin return and the client wire response. Plugins always return float vectors; the edge renders encoding_format=base64 from them.
type EmbeddingsUsage ¶ added in v0.0.5
type ErrorResponse ¶
type ErrorResponse struct {
Error string `json:"error" example:"invalid request"`
}
ErrorResponse is the standard error response format for all API endpoints.
type ErrorType ¶
type ErrorType int
ErrorType classifies provider errors for key iteration inside backends.
const ( ErrorTypeUnknown ErrorType = iota ErrorTypeRateLimit // Temporary rate limit; plugin MUST supply RetryAfter ErrorTypeQuotaExceeded // Quota exhausted; plugin MUST supply RetryAfter ErrorTypeAuth // Auth failure: disable the credential (system) ErrorTypeUpstream // Transient upstream failure (5xx, timeout, reset) ErrorTypeInvalidRequest // Malformed request; fatal for the pool ErrorTypeGeo // Exit geo-blocked: indefinite (provider, proxy) ban, same-key retry on another region ErrorTypeNotFound // Model does not exist upstream; drop it from the cache ErrorTypePaymentRequired // Upstream paywall: disable the credential (system) ErrorTypeContentPolicy // Upstream rejected the content; fatal for the pool ErrorTypeStructuralFault // Provider config/network broken for all keys and models; disable the provider (system), fatal for the pool )
type ExhaustedEntry ¶ added in v0.1.2
type ExhaustedEntry struct {
Key string `json:"key"`
ResetsAt time.Time `json:"resets_at"`
Reason string `json:"reason,omitempty"`
}
ExhaustedEntry is one joint limit key: the stored dimensions act as a filter, and a candidate combination matching every stored dimension is skipped until ResetsAt passes.
type GeoBanEntry ¶ added in v0.2.0
type GeoBanEntry struct {
Key string `json:"key"`
Plugin string `json:"plugin"`
Provider string `json:"provider"`
Proxy string `json:"proxy"`
Reason string `json:"reason,omitempty"`
BannedAt time.Time `json:"banned_at"`
}
GeoBanEntry is one indefinite geo-block flag: proxy ProxyID is unusable for provider type Provider of plugin Plugin. No expiry: the flag lives until the proxy is deleted or an admin clears it explicitly.
type GeoConfig ¶ added in v0.2.0
type GeoConfig struct {
Mode string `json:"mode"` // fail_fast (default) or retry_same_key
MaxProxies int `json:"max_proxies,omitempty"`
}
GeoConfig is the provider-level geo-block reaction policy.
type ImageGenerationRequest ¶ added in v0.0.5
type ImageGenerationRequest struct {
Model ModelId `json:"model"`
Prompt string `json:"prompt"`
N int `json:"n,omitempty"`
Size string `json:"size,omitempty"`
Quality string `json:"quality,omitempty"`
Style string `json:"style,omitempty"`
ResponseFormat string `json:"response_format,omitempty"` // url or b64_json
}
ImageGenerationRequest is the parsed body of POST /v1/images/generations.
type ImageGenerationResponse ¶ added in v0.0.5
type ImageGenerationResponse struct {
Created int64 `json:"created"`
Data []ImageData `json:"data"`
}
ImageGenerationResponse is the normalized plugin return and the client wire response: the OpenAI images shape.
type MetricEvent ¶
type MetricEvent struct {
Timestamp time.Time
ProviderID string
ProviderType string
Model ModelId
TokenID string
Duration time.Duration
StatusCode int
TokensInput int64
TokensOutput int64
ErrorType string
}
MetricEvent represents a single API request event.
type MetricsFilters ¶
type MetricsFilters struct {
ProviderID string `json:"provider_id"`
Model ModelId `json:"model"`
TimeRange TimeRange `json:"time_range"`
}
MetricsFilters for querying metrics.
type MetricsOverview ¶
type MetricsOverview struct {
TotalRequests int64 `json:"total_requests" example:"1500"`
TotalErrors int64 `json:"total_errors" example:"23"`
PeakRequests int64 `json:"peak_requests" example:"45"`
PeakInputTokens int64 `json:"peak_input_tokens" example:"12000"`
PeakOutputTokens int64 `json:"peak_output_tokens" example:"3500"`
}
MetricsOverview for dashboard display.
type ModelId ¶
type ModelId string
ModelId uniquely identifies a model in the format: provider/model-name[:version]
func (ModelId) Name ¶ added in v0.1.2
Name returns the model name without the provider prefix: everything after the first '/'. Plugins sending the bare name upstream use this instead of hand-rolled stripping. Invalid ids yield the full id unchanged.
func (ModelId) Parse ¶
Parse splits a ModelId into providerID and model name. Both parts must be non-empty: "/model" and "provider/" are rejected instead of routing with an empty half.
type ModelInfo ¶
type ModelInfo struct {
Name string `json:"name"`
DisplayName string `json:"display_name"`
Description string `json:"description,omitempty"`
RPM int64 `json:"rpm"`
TPM int64 `json:"tpm"`
RPD int64 `json:"rpd"`
ContextWindow int64 `json:"context_window,omitempty"`
MaxTokens int64 `json:"max_tokens,omitempty"`
// Capabilities are UI-facing feature chips (tools, json_mode, ...).
// Derived from the fields below when the plugin does not set them.
// Modalities stay out: they render in their own column.
Capabilities []string `json:"capabilities,omitempty"`
InputModalities []string `json:"input_modalities,omitempty"`
OutputModalities []string `json:"output_modalities,omitempty"`
SupportedParameters []string `json:"supported_parameters,omitempty"`
Reasoning *ModelReasoning `json:"reasoning,omitempty"`
// Endpoints lists the served endpoint identifiers (Endpoint* constants).
// Empty means chat/completions only.
Endpoints []string `json:"endpoints,omitempty"`
}
ModelInfo contains metadata about a specific model. Field style follows the OpenRouter model schema where applicable.
func (*ModelInfo) DeriveCapabilities ¶
func (m *ModelInfo) DeriveCapabilities()
DeriveCapabilities fills Capabilities from the OpenRouter-style fields. Explicit plugin-provided capabilities win.
func (*ModelInfo) SupportsEndpoint ¶ added in v0.0.5
SupportsEndpoint reports whether the model serves endpoint. Empty Endpoints implies chat/completions only.
type ModelOverride ¶
type ModelOverride struct {
ProviderID string `json:"provider_id"`
Name string `json:"name"`
Disabled bool `json:"disabled,omitempty"`
Custom bool `json:"custom,omitempty"`
DisplayName string `json:"display_name,omitempty"`
Capabilities []string `json:"capabilities,omitempty"`
// InputModalities and OutputModalities hold probe-verified modalities,
// keyed by full model id. Applied over upstream data when non-empty.
InputModalities []string `json:"input_modalities,omitempty"`
OutputModalities []string `json:"output_modalities,omitempty"`
}
ModelOverride is the per-provider admin override for a single model: disable from routing and /v1/models, register a custom model the upstream does not list, or adjust display metadata.
type ModelReasoning ¶
type ModelReasoning struct {
// SupportedEfforts in descending effort order (highest first).
SupportedEfforts []string `json:"supported_efforts,omitempty"`
DefaultEffort string `json:"default_effort,omitempty"`
DefaultEnabled bool `json:"default_enabled,omitempty"`
Mandatory bool `json:"mandatory,omitempty"`
}
ModelReasoning describes reasoning support of a model (OpenRouter-style).
type OpenAIError ¶
type OpenAIError struct {
Error OpenAIErrorBody `json:"error"`
}
OpenAIError wraps error responses in the OpenAI error format.
type OpenAIErrorBody ¶
type PluginInternalError ¶
PluginInternalError is a caught Lua failure at the Go/Lua boundary.
func (*PluginInternalError) Error ¶
func (e *PluginInternalError) Error() string
type PromptTokensDetails ¶
type PromptTokensDetails struct {
CachedTokens int `json:"cached_tokens,omitempty"`
AudioTokens int `json:"audio_tokens,omitempty"`
}
PromptTokensDetails is the OpenAI breakdown of prompt-side token usage.
type Provider ¶
type Provider = ProviderInstance
Provider is kept as an alias so existing call sites keep compiling while the field migration (Type -> TypeKey, BaseURL -> Config) proceeds.
type ProviderError ¶
type ProviderError struct {
StatusCode int
Message string
Type ErrorType
RetryAfter *time.Time
// Scope names the exhausted dimensions the error limits (account, model,
// proxy). Allowed only on rate_limit (account, model, proxy) and
// quota_exceeded (account, model). Empty marks the full combination.
// Any scope on other types rejects the error table as a plugin bug.
Scope []string
// UpstreamStatus is the raw HTTP status received from the upstream.
// UpstreamBody is the raw upstream response body (possibly truncated by
// the HTTP client). Routing never reads these; they exist for logging
// and debugging. Large bodies are spilled to disk in debug mode with
// the path carried alongside the log line.
UpstreamStatus int
UpstreamBody string
}
ProviderError represents errors returned by provider backends.
func (*ProviderError) Error ¶
func (e *ProviderError) Error() string
type ProviderInstance ¶
type ProviderInstance struct {
ID string `json:"id" example:"opencode-zen"`
Name string `json:"name" example:"OpenCode Zen"`
TypeKey string `json:"type_key" example:"opencode-zen"`
Qualifier string `json:"qualifier" example:""`
Config map[string]any `json:"config"`
IconURL string `json:"icon_url" example:"https://cdn.example.com/openai.svg"`
IsUIReadonly bool `json:"is_ui_readonly"`
IsUIHidden bool `json:"is_ui_hidden"`
// Disabled takes the provider out of routing and model listings.
// Settings and discovery keep working.
Disabled bool `json:"disabled,omitempty"`
// DisabledBy names who disabled the provider: "admin" (dashboard PUT)
// or "system" (structural_fault auto-disable). Empty when enabled.
DisabledBy string `json:"disabled_by,omitempty"`
// DisabledReason carries the human reason (system cause or admin note).
DisabledReason string `json:"disabled_reason,omitempty"`
// DisabledAt marks when the provider was disabled. Nil when enabled.
DisabledAt *time.Time `json:"disabled_at,omitempty"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
ProviderInstance is the single persisted provider record for every type: lua-plugin type keys, built-in "custom" and "virtual".
func (*ProviderInstance) BaseURL ¶
func (p *ProviderInstance) BaseURL() string
BaseURL returns Config["base_url"] for OpenAI-compatible providers.
type ProviderInstanceCreateRequest ¶
type ProviderInstanceCreateRequest struct {
Name string `json:"name"`
TypeKey string `json:"type_key"`
Qualifier string `json:"qualifier"`
Config map[string]any `json:"config"`
IconURL string `json:"icon_url"`
}
ProviderInstanceCreateRequest is the generic create body for any provider type.
type ProviderInstanceUpdateRequest ¶
type ProviderInstanceUpdateRequest struct {
Name string `json:"name"`
Config map[string]any `json:"config"`
IconURL string `json:"icon_url"`
Disabled *bool `json:"disabled"`
}
ProviderInstanceUpdateRequest is the generic update body.
type ProviderStats ¶
type ProviderStats struct {
ModelCount int `json:"model_count" example:"5"`
CredentialCount int `json:"credential_count" example:"2"`
}
ProviderStats holds aggregated statistics for a provider.
type Proxy ¶
type Proxy struct {
ID string `json:"id"`
URL string `json:"url"` // scheme://[user:pass@]host:port
Protocol string `json:"protocol"` // http, https, socks4, socks5
Host string `json:"host"`
Port int `json:"port"`
Location string `json:"location"` // ISO 3166-1 alpha-2 exit location, empty when unknown
Source string `json:"source"` // "manual" or "list:<source type key>"
HandshakeMs int64 `json:"handshake_ms"`
SpeedKbps int64 `json:"speed_kbps"`
LastCheckAt time.Time `json:"last_check_at,omitempty"`
CreatedAt time.Time `json:"created_at"`
}
Proxy is one outbound proxy endpoint, either manually registered or synced from a proxy-list source plugin. Presence in the bucket means the proxy answered the last probe; dead proxies are deleted, never flagged.
type ProxyCandidate ¶
type ProxyCandidate struct {
Protocol string `json:"protocol"`
Host string `json:"host"`
Port int `json:"port"`
Country string `json:"country"`
}
ProxyCandidate is one proxy entry produced by a source plugin.
type ProxyConfig ¶
type ProxyConfig struct {
Mode string `json:"mode"` // "disabled" (default), "auto", "manual"
IDs []string `json:"ids,omitempty"` // manual mode: chosen proxy IDs
}
ProxyConfig is the provider-level proxy mode stored in ProviderInstance.Config.
func ParseProxyConfig ¶ added in v0.1.2
func ParseProxyConfig(providerConfig map[string]any) (ProxyConfig, error)
ParseProxyConfig reads the provider-level proxy policy from a provider config map. Absent or non-map proxy sections mean disabled; an explicitly unknown mode is an error, never a silent fallback to direct.
type RouterConfiguration ¶
type RouterConfiguration struct {
IsClusterNode bool `json:"is_cluster_node"`
DisableTelemetry bool `json:"disable_telemetry"`
// ModelsFilter remembers the dashboard model visibility filter
// ("all", "enabled", "disabled"). Empty means "all".
ModelsFilter string `json:"models_filter,omitempty"`
// MinDownloadSpeedKbps floors the pooled proxy download speed. Slower
// proxies are displaced once their location holds more than
// MaxProxiesPerLocation.
MinDownloadSpeedKbps int64 `json:"min_download_speed_kbps"`
// MaxProxiesPerLocation caps pooled proxies per exit location to the
// fastest N. Locations below the cap keep even slow proxies as fallback.
MaxProxiesPerLocation int `json:"max_proxies_per_location"`
// UpdateIntervalMinutes sets the automatic proxy rotation period.
UpdateIntervalMinutes int `json:"update_interval_minutes"`
}
RouterConfiguration holds deployment-wide instance settings.
func (RouterConfiguration) Validate ¶
func (c RouterConfiguration) Validate() error
Validate checks the configuration ranges.
type RouterToken ¶
type RouterToken struct {
ID string `json:"id" example:"token123"`
Name string `json:"name" example:"Production API Token"`
Token string `json:"token,omitempty" example:"llmr_abc123def456"`
TokenHash string `json:"token_hash,omitempty"`
Rules TokenRules `json:"rules"`
CreatedAt time.Time `json:"created_at" example:"2026-05-04T15:00:00Z"`
}
RouterToken is an opaque bearer token issued by llm-router itself.
type SpeechRequest ¶ added in v0.0.5
type SpeechRequest struct {
Model ModelId `json:"model"`
Input string `json:"input"`
Voice string `json:"voice"`
ResponseFormat string `json:"response_format,omitempty"` // mp3 (default), opus, aac, flac, wav, pcm
Speed *float64 `json:"speed,omitempty"`
Instructions string `json:"instructions,omitempty"`
}
SpeechRequest is the parsed body of POST /v1/audio/speech.
type SpeechResponse ¶ added in v0.0.5
SpeechResponse is the normalized plugin return: raw audio bytes plus the format they are encoded in. The router serves the bytes with the matching Content-Type; no transcoding happens inside the router.
type StreamChunk ¶
type StreamChunk struct {
ID string `json:"id"`
Object string `json:"object"`
Created int64 `json:"created"`
Model string `json:"model"`
Choices []StreamChunkChoice `json:"choices"`
Usage *ChatCompletionUsage `json:"usage,omitempty"`
}
StreamChunk is a single SSE data payload for streaming responses.
type StreamChunkChoice ¶
type StreamChunkChoice struct {
Index int `json:"index"`
Delta ChatMessage `json:"delta"`
FinishReason *string `json:"finish_reason"`
Logprobs any `json:"logprobs,omitempty"`
}
type StreamOptions ¶
type StreamOptions struct {
IncludeUsage bool `json:"include_usage,omitempty"`
}
StreamOptions for streaming
type TimeSeriesPoint ¶
type TimeSeriesPoint struct {
Timestamp time.Time `json:"timestamp" example:"2026-05-04T19:00:00Z"`
Value int64 `json:"value" example:"42"`
}
TimeSeriesPoint for chart data.
type TokenRules ¶
type TokenRules struct {
AllowedProviders []string `json:"allowed_providers"`
AllowAllProviders bool `json:"allow_all_providers"`
AllowedModels []ModelId `json:"allowed_models"`
AllowAllModels bool `json:"allow_all_models"`
AllowedCredentials []string `json:"allowed_credentials"`
AllowAllCredentials bool `json:"allow_all_credentials"`
// CredentialScopes grants credential access per provider: All covers
// every current and future key of the provider, CredentialIDs pins
// specific keys. Scopes union with the legacy flat credential fields.
CredentialScopes []CredentialScope `json:"credential_scopes,omitempty"`
}
TokenRules constrains what an API token is allowed to do.
func (TokenRules) Allows ¶
func (r TokenRules) Allows(model ModelId) bool
Allows reports whether the token may request the given model.
func (TokenRules) AllowsCredential ¶
func (r TokenRules) AllowsCredential(providerID, credentialID string) bool
AllowsCredential reports whether the token may use the given credential of the given provider. Scopes union with the legacy flat fields: either source granting access allows the credential.
func (TokenRules) AllowsProvider ¶
func (r TokenRules) AllowsProvider(providerID string) bool
AllowsProvider reports whether the token may use the given provider.
type TranscriptionRequest ¶ added in v0.0.5
type TranscriptionRequest struct {
Model ModelId
File []byte
FileName string
ContentType string
Language string
Prompt string
ResponseFormat string // json (default), text, srt, verbose_json, vtt
Temperature *float64
TimestampGranularities []string // word, segment
}
TranscriptionRequest is the parsed multipart body of POST /v1/audio/transcriptions. File holds the raw audio bytes.
type TranscriptionResponse ¶ added in v0.0.5
type TranscriptionResponse struct {
Text string `json:"text"`
Language string `json:"language,omitempty"`
Duration float64 `json:"duration,omitempty"`
Segments []TranscriptionSegment `json:"segments,omitempty"`
Words []TranscriptionWord `json:"words,omitempty"`
}
TranscriptionResponse is the normalized plugin return: the OpenAI verbose_json shape. The router renders the client-facing response_format from these fields.
func (*TranscriptionResponse) SRT ¶ added in v0.0.5
func (r *TranscriptionResponse) SRT() string
SRT renders segments as a SubRip subtitle document.
func (*TranscriptionResponse) VTT ¶ added in v0.0.5
func (r *TranscriptionResponse) VTT() string
VTT renders segments as a WebVTT subtitle document.
type TranscriptionSegment ¶ added in v0.0.5
type TranscriptionSegment struct {
ID int `json:"id"`
Seek int `json:"seek,omitempty"`
Start float64 `json:"start"`
End float64 `json:"end"`
Text string `json:"text"`
Tokens []int `json:"tokens,omitempty"`
Temperature float64 `json:"temperature,omitempty"`
AvgLogprob float64 `json:"avg_logprob,omitempty"`
CompressionRatio float64 `json:"compression_ratio,omitempty"`
NoSpeechProb float64 `json:"no_speech_prob,omitempty"`
}
type TranscriptionWord ¶ added in v0.0.5
type UINode ¶
type UINode struct {
Type string `json:"type"`
Text string `json:"text,omitempty"`
Name string `json:"name,omitempty"`
Label string `json:"label,omitempty"`
InputType string `json:"input_type,omitempty"`
Required bool `json:"required,omitempty"`
Options []string `json:"options,omitempty"`
OptionLabels map[string]string `json:"option_labels,omitempty"`
URL string `json:"url,omitempty"`
Variant string `json:"variant,omitempty"`
FormAction string `json:"form_action,omitempty"`
Content []*UINode `json:"content,omitempty"`
Placeholder string `json:"placeholder,omitempty"`
Value any `json:"value,omitempty"`
Direction string `json:"direction,omitempty"`
Align string `json:"align,omitempty"`
Justify string `json:"justify,omitempty"`
Gap int `json:"gap,omitempty"`
Columns int `json:"columns,omitempty"`
Title string `json:"title,omitempty"`
Subtitle string `json:"subtitle,omitempty"`
Wrap bool `json:"wrap,omitempty"`
Grow bool `json:"grow,omitempty"`
Size int `json:"size,omitempty"`
Extra map[string]any `json:"-"`
}
UINode is a single node of the lua-driven UI tree shared by config_schema, credential_schema and auth wizards.
type VirtualModel ¶ added in v0.0.5
type VirtualModel struct {
ID string `json:"id"`
Name string `json:"name"`
Description string `json:"description"`
Models []VirtualModelEntry `json:"models"`
// Instruction is prepended to requests as the first user message.
Instruction string `json:"instruction"`
// ManagedBy marks provider-maintained virtual models
// ("provider:<provider-id>:<endpoint-slug>"). Managed models resolve
// their member list live from the provider list on every call; stored
// members are ignored. Empty means manually maintained.
ManagedBy string `json:"managed_by,omitempty"`
// Disabled takes the virtual model out of routing. Probes still run.
Disabled bool `json:"disabled,omitempty"`
Version int `json:"version"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
VirtualModel routes requests to its models in list order (fall-through) and reports an error only when every model failed.
type VirtualModelEntry ¶ added in v0.0.5
type VirtualModelEntry struct {
ModelID ModelId `json:"model_id"`
}
VirtualModelEntry is one fall-through step. List order is the priority.