models

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 27, 2026 License: MIT Imports: 7 Imported by: 0

Documentation

Overview

Package models defines all shared data structures for llm-router.

Index

Constants

View Source
const (
	EndpointChatCompletions    = "chat/completions"
	EndpointAudioTranscription = "audio/transcriptions"
	EndpointAudioSpeech        = "audio/speech"
	EndpointImagesGenerations  = "images/generations"
	EndpointEmbeddings         = "embeddings"
)

Endpoint identifiers listed in ModelInfo.Endpoints. The router rejects a request when the resolved model declares Endpoints and the target endpoint is absent; an empty Endpoints list means chat/completions only (legacy plugins predate the field).

View Source
const (
	ExhaustedScopeAccount = "account"
	ExhaustedScopeModel   = "model"
	ExhaustedScopeProxy   = "proxy"
)

ExhaustedScope dimensions nameable in the error contract scope field. Provider and plugin are always part of every key and need no naming.

View Source
const (
	ProxyModeDisabled = "disabled"
	ProxyModeAuto     = "auto"
	ProxyModeManual   = "manual"
)

Mode constants for ProxyConfig.

View Source
const (
	GeoModeFailFast     = "fail_fast"
	GeoModeRetrySameKey = "retry_same_key"
)

Geo mode constants for the provider-level geo policy stored in ProviderInstance.Config["geo"].

View Source
const (
	DefaultMinDownloadSpeedKbps  int64 = 15000
	DefaultMaxProxiesPerLocation       = 10
	DefaultUpdateIntervalMinutes       = 15
)

Default proxy pool settings.

View Source
const CurrentVersion = "0.3.0"

CurrentVersion is the router version plugins declare compatibility with via the @router_version manifest tag.

0.0.5 adds the transcribe handler and llm_router.multipart. 0.0.6 adds the speech and generate_image handlers. 0.0.7 replaces the proxy pool: multi @proxy_location whitelist, @proxy_default_option, per-pair rate limits and blocks. 0.1.1 replaces per-pair proxy limits and credential quota marks with the unified exhausted store: joint limit keys with a scope field on the error contract. Old plugins are rejected by the manifest gate. 0.1.2 adds the payment_required error type for upstream paywalls. 0.2.0 adds DynamicForm Step gaps (flow/grid gap and spacer size accept Step int 0..8 alongside the legacy sm|md|lg enum), input_type=secret, around/evenly justify values, and strict link URL schemes. 0.3.0 reworks the provider error contract: timeout merges into upstream, content_policy/model_unavailable/structural_fault added, rate_limit and quota_exceeded require plugin-supplied retry_after, geo becomes an indefinite per-provider proxy ban with same-key retry on another region, auth/payment_required disable the credential, structural_fault disables the provider (DisabledBy/Reason/At on both).

View Source
const DefaultGeoMaxProxies = 3

DefaultGeoMaxProxies bounds same-key geo retries when max_proxies is absent.

View Source
const MaxGeoMaxProxies = 10

MaxGeoMaxProxies caps same-key geo retries.

Variables

This section is empty.

Functions

func EmbeddingBase64 added in v0.0.5

func EmbeddingBase64(values []float64) string

EmbeddingBase64 encodes a float vector as base64 float32 little-endian, the OpenAI encoding_format=base64 wire form.

func NormalizeBaseURL added in v0.1.2

func NormalizeBaseURL(raw string) (string, error)

NormalizeBaseURL trims raw and requires an http(s) URL without a trailing slash. Single source of truth for base_url validation across provider CRUD and the custom backend.

func SpeechContentType added in v0.0.5

func SpeechContentType(format string) string

SpeechContentType maps a speech format to its wire Content-Type.

Types

type ActiveRegion added in v0.0.7

type ActiveRegion struct {
	Region   string    `json:"region"`
	LastSeen time.Time `json:"last_seen"`
}

ActiveRegion is one demanded proxy exit location.

type AdminUser

type AdminUser struct {
	Username     string    `json:"username"`
	PasswordHash string    `json:"password_hash"`
	CreatedAt    time.Time `json:"created_at"`
}

AdminUser is the dashboard operator account.

type AuthFlowResult

type AuthFlowResult struct {
	Render      []*UINode      `json:"render,omitempty"`
	RedirectURL string         `json:"redirect_url,omitempty"`
	Credentials map[string]any `json:"credentials,omitempty"`
}

AuthFlowResult is one of the three auth_step/auth_initiate outcomes, discriminated by which field is set.

type AuthStepInput

type AuthStepInput struct {
	Action string         `json:"action"`
	Values map[string]any `json:"values"`
	FlowID string         `json:"flow_id,omitempty"`
}

AuthStepInput is the submit payload for auth_step.

type ChatCompletionChoice

type ChatCompletionChoice struct {
	Index        int         `json:"index"`
	Message      ChatMessage `json:"message"`
	FinishReason string      `json:"finish_reason"`
	Logprobs     any         `json:"logprobs,omitempty"`
}

type ChatCompletionRequest

type ChatCompletionRequest struct {
	Model               ModelId        `json:"model" example:"openai/gpt-4o"`
	Messages            []ChatMessage  `json:"messages"`
	Tools               []ChatTool     `json:"tools,omitempty"`
	ToolChoice          any            `json:"tool_choice,omitempty"`
	ParallelToolCalls   *bool          `json:"parallel_tool_calls,omitempty"`
	Stream              bool           `json:"stream,omitempty" example:"false"`
	StreamOptions       *StreamOptions `json:"stream_options,omitempty"`
	MaxTokens           int            `json:"max_tokens,omitempty" example:"1000"`
	MaxCompletionTokens *int           `json:"max_completion_tokens,omitempty"`
	Temperature         float64        `json:"temperature,omitempty" example:"0.7"`
	TopP                float64        `json:"top_p,omitempty" example:"1.0"`
	N                   *int           `json:"n,omitempty"`
	Stop                any            `json:"stop,omitempty"`
	Seed                *int64         `json:"seed,omitempty"`
	FrequencyPenalty    *float64       `json:"frequency_penalty,omitempty"`
	PresencePenalty     *float64       `json:"presence_penalty,omitempty"`
	Logprobs            *bool          `json:"logprobs,omitempty"`
	TopLogprobs         *int           `json:"top_logprobs,omitempty"`
	ResponseFormat      any            `json:"response_format,omitempty"`
	User                *string        `json:"user,omitempty"`
	ServiceTier         *string        `json:"service_tier,omitempty"`
	ReasoningEffort     *string        `json:"reasoning_effort,omitempty"`
	Verbosity           *string        `json:"verbosity,omitempty"`
}

ChatCompletionRequest is the incoming /v1/chat/completions body.

type ChatCompletionResponse

type ChatCompletionResponse struct {
	ID                string                 `json:"id"`
	Object            string                 `json:"object"`
	Created           int64                  `json:"created"`
	Model             string                 `json:"model"`
	Choices           []ChatCompletionChoice `json:"choices"`
	Usage             ChatCompletionUsage    `json:"usage"`
	SystemFingerprint *string                `json:"system_fingerprint,omitempty"`
	ServiceTier       *string                `json:"service_tier,omitempty"`
}

ChatCompletionResponse mirrors the OpenAI response schema.

type ChatCompletionUsage

type ChatCompletionUsage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`

	PromptTokensDetails     *PromptTokensDetails     `json:"prompt_tokens_details,omitempty"`
	CompletionTokensDetails *CompletionTokensDetails `json:"completion_tokens_details,omitempty"`
}

type ChatMessage

type ChatMessage struct {
	Role         string                   `json:"role" example:"user" enums:"system,user,assistant,tool,developer"`
	Content      string                   `json:"-" example:"Hello, how are you?"`
	ContentParts []ChatMessageContentPart `json:"-"`
	ToolCalls    []ChatToolCall           `json:"tool_calls,omitempty"`
	ToolCallID   string                   `json:"tool_call_id,omitempty"`
	Name         string                   `json:"name,omitempty"`
	Refusal      *string                  `json:"refusal,omitempty"`
	// ReasoningContent carries chain-of-thought text exposed by reasoning
	// models (DeepSeek `reasoning_content`, Groq `reasoning`).
	ReasoningContent string `json:"reasoning_content,omitempty"`
}

ChatMessage is a single turn in a conversation.

func (ChatMessage) MarshalJSON

func (m ChatMessage) MarshalJSON() ([]byte, error)

func (ChatMessage) TextContent

func (m ChatMessage) TextContent() string

func (*ChatMessage) UnmarshalJSON

func (m *ChatMessage) UnmarshalJSON(data []byte) error

type ChatMessageContentPart

type ChatMessageContentPart struct {
	Type        string                   `json:"type,omitempty"`
	Text        string                   `json:"text,omitempty"`
	ToolUseID   string                   `json:"tool_use_id,omitempty"`
	ID          string                   `json:"id,omitempty"`
	Name        string                   `json:"name,omitempty"`
	Input       json.RawMessage          `json:"input,omitempty"`
	ImageURL    *ChatMessageImageURL     `json:"image_url,omitempty"`
	InputAudio  *ChatMessageInputAudio   `json:"input_audio,omitempty"`
	Content     []ChatMessageContentPart `json:"-"`
	ContentText string                   `json:"-"`
}

func (ChatMessageContentPart) MarshalJSON

func (p ChatMessageContentPart) MarshalJSON() ([]byte, error)

func (ChatMessageContentPart) TextContent

func (p ChatMessageContentPart) TextContent() string

func (*ChatMessageContentPart) UnmarshalJSON

func (p *ChatMessageContentPart) UnmarshalJSON(data []byte) error

type ChatMessageImageURL

type ChatMessageImageURL struct {
	URL    string `json:"url,omitempty"`
	Detail string `json:"detail,omitempty"`
}

ChatMessageImageURL is an OpenAI image_url content part. URL is either a remote http(s) URL or a data: URL carrying base64 bytes inline.

type ChatMessageInputAudio

type ChatMessageInputAudio struct {
	Data   string `json:"data,omitempty"`
	Format string `json:"format,omitempty"`
}

ChatMessageInputAudio is an OpenAI input_audio content part. Data is base64 audio, Format is wav or mp3.

type ChatTool

type ChatTool struct {
	Type        string            `json:"type,omitempty"`
	Name        string            `json:"name,omitempty"`
	Description string            `json:"description,omitempty"`
	Parameters  map[string]any    `json:"parameters,omitempty"`
	InputSchema map[string]any    `json:"input_schema,omitempty"`
	Function    *ChatToolFunction `json:"function,omitempty"`
	Strict      *bool             `json:"strict,omitempty"`
}

type ChatToolCall

type ChatToolCall struct {
	ID           string                 `json:"id,omitempty"`
	Type         string                 `json:"type,omitempty"`
	Function     ChatToolFunction       `json:"function"`
	ExtraContent map[string]interface{} `json:"extra_content,omitempty"`
}

type ChatToolFunction

type ChatToolFunction struct {
	Name        string         `json:"name,omitempty"`
	Description string         `json:"description,omitempty"`
	Parameters  map[string]any `json:"parameters,omitempty"`
	Arguments   string         `json:"arguments,omitempty"`
	Strict      *bool          `json:"strict,omitempty"`
}

type CompletionTokensDetails

type CompletionTokensDetails struct {
	ReasoningTokens int `json:"reasoning_tokens,omitempty"`
	AudioTokens     int `json:"audio_tokens,omitempty"`
}

CompletionTokensDetails is the OpenAI breakdown of completion-side token usage.

type Credential

type Credential struct {
	ID         string         `json:"id"`
	ProviderID string         `json:"provider_id"`
	Label      string         `json:"label"`
	Data       map[string]any `json:"data"`
	ExpiresAt  *time.Time     `json:"expires_at,omitempty"`
	CreatedAt  time.Time      `json:"created_at"`
	UpdatedAt  time.Time      `json:"updated_at"`

	LastUsedAt   *time.Time `json:"last_used_at,omitempty"`
	RequestCount int64      `json:"request_count"`
	SuccessCount int64      `json:"success_count"`
	FailureCount int64      `json:"failure_count"`

	// Disabled excludes the credential from routing and fallthrough.
	Disabled bool `json:"disabled,omitempty"`
	// DisabledBy names who disabled the credential: "admin" (dashboard PUT)
	// or "system" (auth/payment_required auto-disable). Empty when enabled.
	DisabledBy string `json:"disabled_by,omitempty"`
	// DisabledReason carries the human reason (system cause or admin note).
	DisabledReason string `json:"disabled_reason,omitempty"`
	// DisabledAt marks when the credential was disabled. Nil when enabled.
	DisabledAt *time.Time `json:"disabled_at,omitempty"`
	// Order is the admin-defined pool position (1-based). 0 means unordered:
	// unordered credentials sort after ordered ones by computed priority.
	Order int `json:"order,omitempty"`
}

Credential holds provider-specific authentication data. Data is a flexible map to support any auth scheme, including non-string values such as expiry timestamps.

func (*Credential) DataString

func (c *Credential) DataString(key string) string

DataString returns a string view of a data value for Go adapters that only understand string credentials.

func (*Credential) ExpiresIn

func (c *Credential) ExpiresIn() time.Duration

ExpiresIn returns the duration until expiry (negative if already expired).

func (*Credential) IncrementUsage

func (c *Credential) IncrementUsage(success bool)

IncrementUsage updates usage statistics after a request.

func (*Credential) IsExpired

func (c *Credential) IsExpired() bool

IsExpired reports whether the credential has passed its expiry time.

func (*Credential) Priority

func (c *Credential) Priority() int

Priority returns the selection priority for this credential: unused first, then used, expired last. Limit state lives in the exhausted store, never on the credential.

type CredentialScope added in v0.1.2

type CredentialScope struct {
	ProviderID    string   `json:"provider_id"`
	All           bool     `json:"all"`
	CredentialIDs []string `json:"credential_ids,omitempty"`
}

CredentialScope grants a token access to credentials of one provider.

type Embedding added in v0.0.5

type Embedding struct {
	Index     int       `json:"index"`
	Values    []float64 `json:"-"`
	B64Values string    `json:"-"`
}

func (Embedding) MarshalJSON added in v0.0.5

func (e Embedding) MarshalJSON() ([]byte, error)

MarshalJSON renders one embedding entry in the OpenAI shape: float array by default, base64 float32-LE when the client asked for base64.

func (*Embedding) UnmarshalJSON added in v0.0.5

func (e *Embedding) UnmarshalJSON(raw []byte) error

UnmarshalJSON accepts the plugin-side embedding entry: the OpenAI {index, embedding:[...]} shape with float values.

type EmbeddingsRequest added in v0.0.5

type EmbeddingsRequest struct {
	Model          ModelId  `json:"model"`
	Input          []string `json:"-"`
	EncodingFormat string   `json:"encoding_format,omitempty"` // float (default) or base64
	Dimensions     int      `json:"dimensions,omitempty"`
}

EmbeddingsRequest is the parsed body of POST /v1/embeddings. Input is normalized to a list of strings at the edge: OpenAI accepts a single string, a list of strings, or token arrays; token arrays are decoded by clients upstream of this router and are refused here.

func (*EmbeddingsRequest) UnmarshalJSON added in v0.0.5

func (r *EmbeddingsRequest) UnmarshalJSON(raw []byte) error

embeddingsRequestJSON decodes the polymorphic OpenAI input field.

type EmbeddingsResponse added in v0.0.5

type EmbeddingsResponse struct {
	Data  []Embedding      `json:"data"`
	Model string           `json:"model"`
	Usage *EmbeddingsUsage `json:"usage,omitempty"`
}

EmbeddingsResponse is the normalized plugin return and the client wire response. Plugins always return float vectors; the edge renders encoding_format=base64 from them.

type EmbeddingsUsage added in v0.0.5

type EmbeddingsUsage struct {
	PromptTokens int `json:"prompt_tokens"`
	TotalTokens  int `json:"total_tokens"`
}

type ErrorResponse

type ErrorResponse struct {
	Error string `json:"error" example:"invalid request"`
}

ErrorResponse is the standard error response format for all API endpoints.

type ErrorType

type ErrorType int

ErrorType classifies provider errors for key iteration inside backends.

const (
	ErrorTypeUnknown          ErrorType = iota
	ErrorTypeRateLimit                  // Temporary rate limit; plugin MUST supply RetryAfter
	ErrorTypeQuotaExceeded              // Quota exhausted; plugin MUST supply RetryAfter
	ErrorTypeAuth                       // Auth failure: disable the credential (system)
	ErrorTypeUpstream                   // Transient upstream failure (5xx, timeout, reset)
	ErrorTypeInvalidRequest             // Malformed request; fatal for the pool
	ErrorTypeGeo                        // Exit geo-blocked: indefinite (provider, proxy) ban, same-key retry on another region
	ErrorTypeNotFound                   // Model does not exist upstream; drop it from the cache
	ErrorTypePaymentRequired            // Upstream paywall: disable the credential (system)
	ErrorTypeContentPolicy              // Upstream rejected the content; fatal for the pool
	ErrorTypeModelUnavailable           // Model exists but not serving; exhausted on (provider, model) for 2m
	ErrorTypeStructuralFault            // Provider config/network broken for all keys and models; disable the provider (system), fatal for the pool
)

type ExhaustedEntry added in v0.1.2

type ExhaustedEntry struct {
	Key      string    `json:"key"`
	ResetsAt time.Time `json:"resets_at"`
	Reason   string    `json:"reason,omitempty"`
}

ExhaustedEntry is one joint limit key: the stored dimensions act as a filter, and a candidate combination matching every stored dimension is skipped until ResetsAt passes.

type GeoBanEntry added in v0.2.0

type GeoBanEntry struct {
	Key      string    `json:"key"`
	Plugin   string    `json:"plugin"`
	Provider string    `json:"provider"`
	Proxy    string    `json:"proxy"`
	Reason   string    `json:"reason,omitempty"`
	BannedAt time.Time `json:"banned_at"`
}

GeoBanEntry is one indefinite geo-block flag: proxy ProxyID is unusable for provider type Provider of plugin Plugin. No expiry: the flag lives until the proxy is deleted or an admin clears it explicitly.

type GeoConfig added in v0.2.0

type GeoConfig struct {
	Mode       string `json:"mode"` // fail_fast (default) or retry_same_key
	MaxProxies int    `json:"max_proxies,omitempty"`
}

GeoConfig is the provider-level geo-block reaction policy.

func ParseGeoConfig added in v0.2.0

func ParseGeoConfig(providerConfig map[string]any) (GeoConfig, error)

ParseGeoConfig reads the provider-level geo policy from a provider config map. Absent or non-map geo sections mean fail_fast; an explicitly unknown mode is an error, never a silent fallback.

type ImageData added in v0.0.5

type ImageData struct {
	URL           string `json:"url,omitempty"`
	B64JSON       string `json:"b64_json,omitempty"`
	RevisedPrompt string `json:"revised_prompt,omitempty"`
}

type ImageGenerationRequest added in v0.0.5

type ImageGenerationRequest struct {
	Model          ModelId `json:"model"`
	Prompt         string  `json:"prompt"`
	N              int     `json:"n,omitempty"`
	Size           string  `json:"size,omitempty"`
	Quality        string  `json:"quality,omitempty"`
	Style          string  `json:"style,omitempty"`
	ResponseFormat string  `json:"response_format,omitempty"` // url or b64_json
}

ImageGenerationRequest is the parsed body of POST /v1/images/generations.

type ImageGenerationResponse added in v0.0.5

type ImageGenerationResponse struct {
	Created int64       `json:"created"`
	Data    []ImageData `json:"data"`
}

ImageGenerationResponse is the normalized plugin return and the client wire response: the OpenAI images shape.

type MetricEvent

type MetricEvent struct {
	Timestamp    time.Time
	ProviderID   string
	ProviderType string
	Model        ModelId
	TokenID      string
	Duration     time.Duration
	StatusCode   int
	TokensInput  int64
	TokensOutput int64
	ErrorType    string
}

MetricEvent represents a single API request event.

type MetricsFilters

type MetricsFilters struct {
	ProviderID string    `json:"provider_id"`
	Model      ModelId   `json:"model"`
	TimeRange  TimeRange `json:"time_range"`
}

MetricsFilters for querying metrics.

type MetricsOverview

type MetricsOverview struct {
	TotalRequests    int64 `json:"total_requests" example:"1500"`
	TotalErrors      int64 `json:"total_errors" example:"23"`
	PeakRequests     int64 `json:"peak_requests" example:"45"`
	PeakInputTokens  int64 `json:"peak_input_tokens" example:"12000"`
	PeakOutputTokens int64 `json:"peak_output_tokens" example:"3500"`
}

MetricsOverview for dashboard display.

type ModelId

type ModelId string

ModelId uniquely identifies a model in the format: provider/model-name[:version]

func (ModelId) Name added in v0.1.2

func (m ModelId) Name() string

Name returns the model name without the provider prefix: everything after the first '/'. Plugins sending the bare name upstream use this instead of hand-rolled stripping. Invalid ids yield the full id unchanged.

func (ModelId) Parse

func (m ModelId) Parse() (providerID, model string, err error)

Parse splits a ModelId into providerID and model name. Both parts must be non-empty: "/model" and "provider/" are rejected instead of routing with an empty half.

func (ModelId) ParseFull

func (m ModelId) ParseFull() (adapterType, qualifier, model string, err error)

ParseFull splits a ModelId into adapter type, qualifier, and model name.

func (ModelId) String

func (m ModelId) String() string

type ModelInfo

type ModelInfo struct {
	Name          string `json:"name"`
	DisplayName   string `json:"display_name"`
	Description   string `json:"description,omitempty"`
	RPM           int64  `json:"rpm"`
	TPM           int64  `json:"tpm"`
	RPD           int64  `json:"rpd"`
	ContextWindow int64  `json:"context_window,omitempty"`
	MaxTokens     int64  `json:"max_tokens,omitempty"`

	// Capabilities are UI-facing feature chips (tools, json_mode, ...).
	// Derived from the fields below when the plugin does not set them.
	// Modalities stay out: they render in their own column.
	Capabilities []string `json:"capabilities,omitempty"`

	InputModalities     []string        `json:"input_modalities,omitempty"`
	OutputModalities    []string        `json:"output_modalities,omitempty"`
	SupportedParameters []string        `json:"supported_parameters,omitempty"`
	Reasoning           *ModelReasoning `json:"reasoning,omitempty"`

	// Endpoints lists the served endpoint identifiers (Endpoint* constants).
	// Empty means chat/completions only.
	Endpoints []string `json:"endpoints,omitempty"`
}

ModelInfo contains metadata about a specific model. Field style follows the OpenRouter model schema where applicable.

func (*ModelInfo) DeriveCapabilities

func (m *ModelInfo) DeriveCapabilities()

DeriveCapabilities fills Capabilities from the OpenRouter-style fields. Explicit plugin-provided capabilities win.

func (*ModelInfo) SupportsEndpoint added in v0.0.5

func (m *ModelInfo) SupportsEndpoint(endpoint string) bool

SupportsEndpoint reports whether the model serves endpoint. Empty Endpoints implies chat/completions only.

type ModelOverride

type ModelOverride struct {
	ProviderID   string   `json:"provider_id"`
	Name         string   `json:"name"`
	Disabled     bool     `json:"disabled,omitempty"`
	Custom       bool     `json:"custom,omitempty"`
	DisplayName  string   `json:"display_name,omitempty"`
	Capabilities []string `json:"capabilities,omitempty"`
	// InputModalities and OutputModalities hold probe-verified modalities,
	// keyed by full model id. Applied over upstream data when non-empty.
	InputModalities  []string `json:"input_modalities,omitempty"`
	OutputModalities []string `json:"output_modalities,omitempty"`
}

ModelOverride is the per-provider admin override for a single model: disable from routing and /v1/models, register a custom model the upstream does not list, or adjust display metadata.

type ModelReasoning

type ModelReasoning struct {
	// SupportedEfforts in descending effort order (highest first).
	SupportedEfforts []string `json:"supported_efforts,omitempty"`
	DefaultEffort    string   `json:"default_effort,omitempty"`
	DefaultEnabled   bool     `json:"default_enabled,omitempty"`
	Mandatory        bool     `json:"mandatory,omitempty"`
}

ModelReasoning describes reasoning support of a model (OpenRouter-style).

type OpenAIError

type OpenAIError struct {
	Error OpenAIErrorBody `json:"error"`
}

OpenAIError wraps error responses in the OpenAI error format.

type OpenAIErrorBody

type OpenAIErrorBody struct {
	Message string  `json:"message" example:"Invalid request: missing required field 'model'"`
	Type    string  `json:"type" example:"invalid_request_error"`
	Param   *string `json:"param" example:"model"`
	Code    string  `json:"code,omitempty" example:"invalid_request"`
}

type PluginInternalError

type PluginInternalError struct {
	PluginID string
	TypeKey  string
	Cause    string
}

PluginInternalError is a caught Lua failure at the Go/Lua boundary.

func (*PluginInternalError) Error

func (e *PluginInternalError) Error() string

type PromptTokensDetails

type PromptTokensDetails struct {
	CachedTokens int `json:"cached_tokens,omitempty"`
	AudioTokens  int `json:"audio_tokens,omitempty"`
}

PromptTokensDetails is the OpenAI breakdown of prompt-side token usage.

type Provider

type Provider = ProviderInstance

Provider is kept as an alias so existing call sites keep compiling while the field migration (Type -> TypeKey, BaseURL -> Config) proceeds.

type ProviderError

type ProviderError struct {
	StatusCode int
	Message    string
	Type       ErrorType
	RetryAfter *time.Time
	// Scope names the exhausted dimensions the error limits (account, model,
	// proxy). Allowed only on rate_limit (account, model, proxy) and
	// quota_exceeded (account, model). Empty marks the full combination.
	// Any scope on other types rejects the error table as a plugin bug.
	Scope []string
	// UpstreamStatus is the raw HTTP status received from the upstream.
	// UpstreamBody is the raw upstream response body (possibly truncated by
	// the HTTP client). Routing never reads these; they exist for logging
	// and debugging. Large bodies are spilled to disk in debug mode with
	// the path carried alongside the log line.
	UpstreamStatus int
	UpstreamBody   string
}

ProviderError represents errors returned by provider backends.

func (*ProviderError) Error

func (e *ProviderError) Error() string

type ProviderInstance

type ProviderInstance struct {
	ID           string         `json:"id" example:"opencode-zen"`
	Name         string         `json:"name" example:"OpenCode Zen"`
	TypeKey      string         `json:"type_key" example:"opencode-zen"`
	Qualifier    string         `json:"qualifier" example:""`
	Config       map[string]any `json:"config"`
	IconURL      string         `json:"icon_url" example:"https://cdn.example.com/openai.svg"`
	IsUIReadonly bool           `json:"is_ui_readonly"`
	IsUIHidden   bool           `json:"is_ui_hidden"`
	// Disabled takes the provider out of routing and model listings.
	// Settings and discovery keep working.
	Disabled bool `json:"disabled,omitempty"`
	// DisabledBy names who disabled the provider: "admin" (dashboard PUT)
	// or "system" (structural_fault auto-disable). Empty when enabled.
	DisabledBy string `json:"disabled_by,omitempty"`
	// DisabledReason carries the human reason (system cause or admin note).
	DisabledReason string `json:"disabled_reason,omitempty"`
	// DisabledAt marks when the provider was disabled. Nil when enabled.
	DisabledAt *time.Time `json:"disabled_at,omitempty"`
	CreatedAt  time.Time  `json:"created_at"`
	UpdatedAt  time.Time  `json:"updated_at"`
}

ProviderInstance is the single persisted provider record for every type: lua-plugin type keys, built-in "custom" and "virtual".

func (*ProviderInstance) BaseURL

func (p *ProviderInstance) BaseURL() string

BaseURL returns Config["base_url"] for OpenAI-compatible providers.

type ProviderInstanceCreateRequest

type ProviderInstanceCreateRequest struct {
	Name      string         `json:"name"`
	TypeKey   string         `json:"type_key"`
	Qualifier string         `json:"qualifier"`
	Config    map[string]any `json:"config"`
	IconURL   string         `json:"icon_url"`
}

ProviderInstanceCreateRequest is the generic create body for any provider type.

type ProviderInstanceUpdateRequest

type ProviderInstanceUpdateRequest struct {
	Name     string         `json:"name"`
	Config   map[string]any `json:"config"`
	IconURL  string         `json:"icon_url"`
	Disabled *bool          `json:"disabled"`
}

ProviderInstanceUpdateRequest is the generic update body.

type ProviderStats

type ProviderStats struct {
	ModelCount      int `json:"model_count" example:"5"`
	CredentialCount int `json:"credential_count" example:"2"`
}

ProviderStats holds aggregated statistics for a provider.

type Proxy

type Proxy struct {
	ID       string `json:"id"`
	URL      string `json:"url"`      // scheme://[user:pass@]host:port
	Protocol string `json:"protocol"` // http, https, socks4, socks5
	Host     string `json:"host"`
	Port     int    `json:"port"`
	Location string `json:"location"` // ISO 3166-1 alpha-2 exit location, empty when unknown
	Source   string `json:"source"`   // "manual" or "list:<source type key>"

	HandshakeMs int64     `json:"handshake_ms"`
	SpeedKbps   int64     `json:"speed_kbps"`
	LastCheckAt time.Time `json:"last_check_at,omitempty"`

	CreatedAt time.Time `json:"created_at"`
}

Proxy is one outbound proxy endpoint, either manually registered or synced from a proxy-list source plugin. Presence in the bucket means the proxy answered the last probe; dead proxies are deleted, never flagged.

type ProxyCandidate

type ProxyCandidate struct {
	Protocol string `json:"protocol"`
	Host     string `json:"host"`
	Port     int    `json:"port"`
	Country  string `json:"country"`
}

ProxyCandidate is one proxy entry produced by a source plugin.

type ProxyConfig

type ProxyConfig struct {
	Mode string   `json:"mode"`          // "disabled" (default), "auto", "manual"
	IDs  []string `json:"ids,omitempty"` // manual mode: chosen proxy IDs
}

ProxyConfig is the provider-level proxy mode stored in ProviderInstance.Config.

func ParseProxyConfig added in v0.1.2

func ParseProxyConfig(providerConfig map[string]any) (ProxyConfig, error)

ParseProxyConfig reads the provider-level proxy policy from a provider config map. Absent or non-map proxy sections mean disabled; an explicitly unknown mode is an error, never a silent fallback to direct.

type RouterConfiguration

type RouterConfiguration struct {
	IsClusterNode    bool `json:"is_cluster_node"`
	DisableTelemetry bool `json:"disable_telemetry"`
	// ModelsFilter remembers the dashboard model visibility filter
	// ("all", "enabled", "disabled"). Empty means "all".
	ModelsFilter string `json:"models_filter,omitempty"`
	// MinDownloadSpeedKbps floors the pooled proxy download speed. Slower
	// proxies are displaced once their location holds more than
	// MaxProxiesPerLocation.
	MinDownloadSpeedKbps int64 `json:"min_download_speed_kbps"`
	// MaxProxiesPerLocation caps pooled proxies per exit location to the
	// fastest N. Locations below the cap keep even slow proxies as fallback.
	MaxProxiesPerLocation int `json:"max_proxies_per_location"`
	// UpdateIntervalMinutes sets the automatic proxy rotation period.
	UpdateIntervalMinutes int `json:"update_interval_minutes"`
}

RouterConfiguration holds deployment-wide instance settings.

func (RouterConfiguration) Validate

func (c RouterConfiguration) Validate() error

Validate checks the configuration ranges.

type RouterToken

type RouterToken struct {
	ID        string     `json:"id" example:"token123"`
	Name      string     `json:"name" example:"Production API Token"`
	Token     string     `json:"token,omitempty" example:"llmr_abc123def456"`
	TokenHash string     `json:"token_hash,omitempty"`
	Rules     TokenRules `json:"rules"`
	CreatedAt time.Time  `json:"created_at" example:"2026-05-04T15:00:00Z"`
}

RouterToken is an opaque bearer token issued by llm-router itself.

type SpeechRequest added in v0.0.5

type SpeechRequest struct {
	Model          ModelId  `json:"model"`
	Input          string   `json:"input"`
	Voice          string   `json:"voice"`
	ResponseFormat string   `json:"response_format,omitempty"` // mp3 (default), opus, aac, flac, wav, pcm
	Speed          *float64 `json:"speed,omitempty"`
	Instructions   string   `json:"instructions,omitempty"`
}

SpeechRequest is the parsed body of POST /v1/audio/speech.

type SpeechResponse added in v0.0.5

type SpeechResponse struct {
	Audio  []byte
	Format string // mp3, opus, aac, flac, wav, pcm
}

SpeechResponse is the normalized plugin return: raw audio bytes plus the format they are encoded in. The router serves the bytes with the matching Content-Type; no transcoding happens inside the router.

type StreamChunk

type StreamChunk struct {
	ID      string               `json:"id"`
	Object  string               `json:"object"`
	Created int64                `json:"created"`
	Model   string               `json:"model"`
	Choices []StreamChunkChoice  `json:"choices"`
	Usage   *ChatCompletionUsage `json:"usage,omitempty"`
}

StreamChunk is a single SSE data payload for streaming responses.

type StreamChunkChoice

type StreamChunkChoice struct {
	Index        int         `json:"index"`
	Delta        ChatMessage `json:"delta"`
	FinishReason *string     `json:"finish_reason"`
	Logprobs     any         `json:"logprobs,omitempty"`
}

type StreamOptions

type StreamOptions struct {
	IncludeUsage bool `json:"include_usage,omitempty"`
}

StreamOptions for streaming

type TimeRange

type TimeRange string

TimeRange represents a time window for metrics queries.

const (
	TimeRangeHour      TimeRange = "hour"
	TimeRange1Day      TimeRange = "1d"
	TimeRange7Days     TimeRange = "7d"
	TimeRange28Days    TimeRange = "28d"
	TimeRange90Days    TimeRange = "90d"
	TimeRangeThisMonth TimeRange = "month"
)

func (TimeRange) Bounds

func (tr TimeRange) Bounds() (start, end time.Time)

Bounds returns the start and end times for this time range.

type TimeSeriesPoint

type TimeSeriesPoint struct {
	Timestamp time.Time `json:"timestamp" example:"2026-05-04T19:00:00Z"`
	Value     int64     `json:"value" example:"42"`
}

TimeSeriesPoint for chart data.

type TokenRules

type TokenRules struct {
	AllowedProviders    []string  `json:"allowed_providers"`
	AllowAllProviders   bool      `json:"allow_all_providers"`
	AllowedModels       []ModelId `json:"allowed_models"`
	AllowAllModels      bool      `json:"allow_all_models"`
	AllowedCredentials  []string  `json:"allowed_credentials"`
	AllowAllCredentials bool      `json:"allow_all_credentials"`
	// CredentialScopes grants credential access per provider: All covers
	// every current and future key of the provider, CredentialIDs pins
	// specific keys. Scopes union with the legacy flat credential fields.
	CredentialScopes []CredentialScope `json:"credential_scopes,omitempty"`
}

TokenRules constrains what an API token is allowed to do.

func (TokenRules) Allows

func (r TokenRules) Allows(model ModelId) bool

Allows reports whether the token may request the given model.

func (TokenRules) AllowsCredential

func (r TokenRules) AllowsCredential(providerID, credentialID string) bool

AllowsCredential reports whether the token may use the given credential of the given provider. Scopes union with the legacy flat fields: either source granting access allows the credential.

func (TokenRules) AllowsProvider

func (r TokenRules) AllowsProvider(providerID string) bool

AllowsProvider reports whether the token may use the given provider.

type TranscriptionRequest added in v0.0.5

type TranscriptionRequest struct {
	Model                  ModelId
	File                   []byte
	FileName               string
	ContentType            string
	Language               string
	Prompt                 string
	ResponseFormat         string // json (default), text, srt, verbose_json, vtt
	Temperature            *float64
	TimestampGranularities []string // word, segment
}

TranscriptionRequest is the parsed multipart body of POST /v1/audio/transcriptions. File holds the raw audio bytes.

type TranscriptionResponse added in v0.0.5

type TranscriptionResponse struct {
	Text     string                 `json:"text"`
	Language string                 `json:"language,omitempty"`
	Duration float64                `json:"duration,omitempty"`
	Segments []TranscriptionSegment `json:"segments,omitempty"`
	Words    []TranscriptionWord    `json:"words,omitempty"`
}

TranscriptionResponse is the normalized plugin return: the OpenAI verbose_json shape. The router renders the client-facing response_format from these fields.

func (*TranscriptionResponse) SRT added in v0.0.5

func (r *TranscriptionResponse) SRT() string

SRT renders segments as a SubRip subtitle document.

func (*TranscriptionResponse) VTT added in v0.0.5

func (r *TranscriptionResponse) VTT() string

VTT renders segments as a WebVTT subtitle document.

type TranscriptionSegment added in v0.0.5

type TranscriptionSegment struct {
	ID               int     `json:"id"`
	Seek             int     `json:"seek,omitempty"`
	Start            float64 `json:"start"`
	End              float64 `json:"end"`
	Text             string  `json:"text"`
	Tokens           []int   `json:"tokens,omitempty"`
	Temperature      float64 `json:"temperature,omitempty"`
	AvgLogprob       float64 `json:"avg_logprob,omitempty"`
	CompressionRatio float64 `json:"compression_ratio,omitempty"`
	NoSpeechProb     float64 `json:"no_speech_prob,omitempty"`
}

type TranscriptionWord added in v0.0.5

type TranscriptionWord struct {
	Word  string  `json:"word"`
	Start float64 `json:"start"`
	End   float64 `json:"end"`
}

type UINode

type UINode struct {
	Type         string            `json:"type"`
	Text         string            `json:"text,omitempty"`
	Name         string            `json:"name,omitempty"`
	Label        string            `json:"label,omitempty"`
	InputType    string            `json:"input_type,omitempty"`
	Required     bool              `json:"required,omitempty"`
	Options      []string          `json:"options,omitempty"`
	OptionLabels map[string]string `json:"option_labels,omitempty"`
	URL          string            `json:"url,omitempty"`
	Variant      string            `json:"variant,omitempty"`
	FormAction   string            `json:"form_action,omitempty"`
	Content      []*UINode         `json:"content,omitempty"`
	Placeholder  string            `json:"placeholder,omitempty"`
	Value        any               `json:"value,omitempty"`
	Direction    string            `json:"direction,omitempty"`
	Align        string            `json:"align,omitempty"`
	Justify      string            `json:"justify,omitempty"`
	Gap          int               `json:"gap,omitempty"`
	Columns      int               `json:"columns,omitempty"`
	Title        string            `json:"title,omitempty"`
	Subtitle     string            `json:"subtitle,omitempty"`
	Wrap         bool              `json:"wrap,omitempty"`
	Grow         bool              `json:"grow,omitempty"`
	Size         int               `json:"size,omitempty"`
	Extra        map[string]any    `json:"-"`
}

UINode is a single node of the lua-driven UI tree shared by config_schema, credential_schema and auth wizards.

type VirtualModel added in v0.0.5

type VirtualModel struct {
	ID          string              `json:"id"`
	Name        string              `json:"name"`
	Description string              `json:"description"`
	Models      []VirtualModelEntry `json:"models"`
	// Instruction is prepended to requests as the first user message.
	Instruction string `json:"instruction"`
	// ManagedBy marks provider-maintained virtual models
	// ("provider:<provider-id>:<endpoint-slug>"). Managed models resolve
	// their member list live from the provider list on every call; stored
	// members are ignored. Empty means manually maintained.
	ManagedBy string `json:"managed_by,omitempty"`
	// Disabled takes the virtual model out of routing. Probes still run.
	Disabled  bool      `json:"disabled,omitempty"`
	Version   int       `json:"version"`
	CreatedAt time.Time `json:"created_at"`
	UpdatedAt time.Time `json:"updated_at"`
}

VirtualModel routes requests to its models in list order (fall-through) and reports an error only when every model failed.

type VirtualModelEntry added in v0.0.5

type VirtualModelEntry struct {
	ModelID ModelId `json:"model_id"`
}

VirtualModelEntry is one fall-through step. List order is the priority.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL