models

package
v0.0.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 19, 2026 License: MIT Imports: 7 Imported by: 0

Documentation

Overview

Package models defines all shared data structures for llm-router.

Index

Constants

View Source
const (
	EndpointChatCompletions    = "chat/completions"
	EndpointAudioTranscription = "audio/transcriptions"
	EndpointAudioSpeech        = "audio/speech"
	EndpointImagesGenerations  = "images/generations"
	EndpointEmbeddings         = "embeddings"
)

Endpoint identifiers listed in ModelInfo.Endpoints. The router rejects a request when the resolved model declares Endpoints and the target endpoint is absent; an empty Endpoints list means chat/completions only (legacy plugins predate the field).

View Source
const (
	ProxyModeDisabled = "disabled"
	ProxyModeAuto     = "auto"
	ProxyModeManual   = "manual"
)

Mode constants for ProxyConfig.

View Source
const CurrentVersion = "0.0.6"

CurrentVersion is the router version plugins declare compatibility with via the @router_version manifest tag.

0.0.5 adds the transcribe handler and llm_router.multipart. 0.0.6 adds the speech and generate_image handlers.

Variables

This section is empty.

Functions

func EmbeddingBase64 added in v0.0.5

func EmbeddingBase64(values []float64) string

EmbeddingBase64 encodes a float vector as base64 float32 little-endian, the OpenAI encoding_format=base64 wire form.

func SpeechContentType added in v0.0.5

func SpeechContentType(format string) string

SpeechContentType maps a speech format to its wire Content-Type.

Types

type AdminUser

type AdminUser struct {
	Username     string    `json:"username"`
	PasswordHash string    `json:"password_hash"`
	CreatedAt    time.Time `json:"created_at"`
}

AdminUser is the dashboard operator account.

type AuthFlowResult

type AuthFlowResult struct {
	Render      []*UINode      `json:"render,omitempty"`
	RedirectURL string         `json:"redirect_url,omitempty"`
	Credentials map[string]any `json:"credentials,omitempty"`
}

AuthFlowResult is one of the three auth_step/auth_initiate outcomes, discriminated by which field is set.

type AuthStepInput

type AuthStepInput struct {
	Action string         `json:"action"`
	Values map[string]any `json:"values"`
	FlowID string         `json:"flow_id,omitempty"`
}

AuthStepInput is the submit payload for auth_step.

type ChatCompletionChoice

type ChatCompletionChoice struct {
	Index        int         `json:"index"`
	Message      ChatMessage `json:"message"`
	FinishReason string      `json:"finish_reason"`
	Logprobs     any         `json:"logprobs,omitempty"`
}

type ChatCompletionRequest

type ChatCompletionRequest struct {
	Model               ModelId        `json:"model" example:"openai/gpt-4o"`
	Messages            []ChatMessage  `json:"messages"`
	Tools               []ChatTool     `json:"tools,omitempty"`
	ToolChoice          any            `json:"tool_choice,omitempty"`
	ParallelToolCalls   *bool          `json:"parallel_tool_calls,omitempty"`
	Stream              bool           `json:"stream,omitempty" example:"false"`
	StreamOptions       *StreamOptions `json:"stream_options,omitempty"`
	MaxTokens           int            `json:"max_tokens,omitempty" example:"1000"`
	MaxCompletionTokens *int           `json:"max_completion_tokens,omitempty"`
	Temperature         float64        `json:"temperature,omitempty" example:"0.7"`
	TopP                float64        `json:"top_p,omitempty" example:"1.0"`
	N                   *int           `json:"n,omitempty"`
	Stop                any            `json:"stop,omitempty"`
	Seed                *int64         `json:"seed,omitempty"`
	FrequencyPenalty    *float64       `json:"frequency_penalty,omitempty"`
	PresencePenalty     *float64       `json:"presence_penalty,omitempty"`
	Logprobs            *bool          `json:"logprobs,omitempty"`
	TopLogprobs         *int           `json:"top_logprobs,omitempty"`
	ResponseFormat      any            `json:"response_format,omitempty"`
	User                *string        `json:"user,omitempty"`
	ServiceTier         *string        `json:"service_tier,omitempty"`
	ReasoningEffort     *string        `json:"reasoning_effort,omitempty"`
	Verbosity           *string        `json:"verbosity,omitempty"`
}

ChatCompletionRequest is the incoming /v1/chat/completions body.

type ChatCompletionResponse

type ChatCompletionResponse struct {
	ID                string                 `json:"id"`
	Object            string                 `json:"object"`
	Created           int64                  `json:"created"`
	Model             string                 `json:"model"`
	Choices           []ChatCompletionChoice `json:"choices"`
	Usage             ChatCompletionUsage    `json:"usage"`
	SystemFingerprint *string                `json:"system_fingerprint,omitempty"`
	ServiceTier       *string                `json:"service_tier,omitempty"`
}

ChatCompletionResponse mirrors the OpenAI response schema.

type ChatCompletionUsage

type ChatCompletionUsage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`

	PromptTokensDetails     *PromptTokensDetails     `json:"prompt_tokens_details,omitempty"`
	CompletionTokensDetails *CompletionTokensDetails `json:"completion_tokens_details,omitempty"`
}

type ChatMessage

type ChatMessage struct {
	Role         string                   `json:"role" example:"user" enums:"system,user,assistant,tool,developer"`
	Content      string                   `json:"-" example:"Hello, how are you?"`
	ContentParts []ChatMessageContentPart `json:"-"`
	ToolCalls    []ChatToolCall           `json:"tool_calls,omitempty"`
	ToolCallID   string                   `json:"tool_call_id,omitempty"`
	Name         string                   `json:"name,omitempty"`
	Refusal      *string                  `json:"refusal,omitempty"`
	// ReasoningContent carries chain-of-thought text exposed by reasoning
	// models (DeepSeek `reasoning_content`, Groq `reasoning`).
	ReasoningContent string `json:"reasoning_content,omitempty"`
}

ChatMessage is a single turn in a conversation.

func (ChatMessage) MarshalJSON

func (m ChatMessage) MarshalJSON() ([]byte, error)

func (ChatMessage) TextContent

func (m ChatMessage) TextContent() string

func (*ChatMessage) UnmarshalJSON

func (m *ChatMessage) UnmarshalJSON(data []byte) error

type ChatMessageContentPart

type ChatMessageContentPart struct {
	Type        string                   `json:"type,omitempty"`
	Text        string                   `json:"text,omitempty"`
	ToolUseID   string                   `json:"tool_use_id,omitempty"`
	ID          string                   `json:"id,omitempty"`
	Name        string                   `json:"name,omitempty"`
	Input       json.RawMessage          `json:"input,omitempty"`
	ImageURL    *ChatMessageImageURL     `json:"image_url,omitempty"`
	InputAudio  *ChatMessageInputAudio   `json:"input_audio,omitempty"`
	Content     []ChatMessageContentPart `json:"-"`
	ContentText string                   `json:"-"`
}

func (ChatMessageContentPart) MarshalJSON

func (p ChatMessageContentPart) MarshalJSON() ([]byte, error)

func (ChatMessageContentPart) TextContent

func (p ChatMessageContentPart) TextContent() string

func (*ChatMessageContentPart) UnmarshalJSON

func (p *ChatMessageContentPart) UnmarshalJSON(data []byte) error

type ChatMessageImageURL

type ChatMessageImageURL struct {
	URL    string `json:"url,omitempty"`
	Detail string `json:"detail,omitempty"`
}

ChatMessageImageURL is an OpenAI image_url content part. URL is either a remote http(s) URL or a data: URL carrying base64 bytes inline.

type ChatMessageInputAudio

type ChatMessageInputAudio struct {
	Data   string `json:"data,omitempty"`
	Format string `json:"format,omitempty"`
}

ChatMessageInputAudio is an OpenAI input_audio content part. Data is base64 audio, Format is wav or mp3.

type ChatTool

type ChatTool struct {
	Type        string            `json:"type,omitempty"`
	Name        string            `json:"name,omitempty"`
	Description string            `json:"description,omitempty"`
	Parameters  map[string]any    `json:"parameters,omitempty"`
	InputSchema map[string]any    `json:"input_schema,omitempty"`
	Function    *ChatToolFunction `json:"function,omitempty"`
	Strict      *bool             `json:"strict,omitempty"`
}

type ChatToolCall

type ChatToolCall struct {
	ID           string                 `json:"id,omitempty"`
	Type         string                 `json:"type,omitempty"`
	Function     ChatToolFunction       `json:"function"`
	ExtraContent map[string]interface{} `json:"extra_content,omitempty"`
}

type ChatToolFunction

type ChatToolFunction struct {
	Name        string         `json:"name,omitempty"`
	Description string         `json:"description,omitempty"`
	Parameters  map[string]any `json:"parameters,omitempty"`
	Arguments   string         `json:"arguments,omitempty"`
	Strict      *bool          `json:"strict,omitempty"`
}

type CompletionTokensDetails

type CompletionTokensDetails struct {
	ReasoningTokens int `json:"reasoning_tokens,omitempty"`
	AudioTokens     int `json:"audio_tokens,omitempty"`
}

CompletionTokensDetails is the OpenAI breakdown of completion-side token usage.

type Credential

type Credential struct {
	ID         string         `json:"id"`
	ProviderID string         `json:"provider_id"`
	Label      string         `json:"label"`
	Data       map[string]any `json:"data"`
	ExpiresAt  *time.Time     `json:"expires_at,omitempty"`
	CreatedAt  time.Time      `json:"created_at"`
	UpdatedAt  time.Time      `json:"updated_at"`

	LastUsedAt   *time.Time `json:"last_used_at,omitempty"`
	RequestCount int64      `json:"request_count"`
	SuccessCount int64      `json:"success_count"`
	FailureCount int64      `json:"failure_count"`

	QuotaResetAt *time.Time `json:"quota_reset_at,omitempty"`

	// Disabled excludes the credential from routing and fallthrough.
	Disabled bool `json:"disabled,omitempty"`
	// Order is the admin-defined pool position (1-based). 0 means unordered:
	// unordered credentials sort after ordered ones by computed priority.
	Order int `json:"order,omitempty"`
}

Credential holds provider-specific authentication data. Data is a flexible map to support any auth scheme, including non-string values such as expiry timestamps.

func (*Credential) DataString

func (c *Credential) DataString(key string) string

DataString returns a string view of a data value for Go adapters that only understand string credentials.

func (*Credential) ExpiresIn

func (c *Credential) ExpiresIn() time.Duration

ExpiresIn returns the duration until expiry (negative if already expired).

func (*Credential) IncrementUsage

func (c *Credential) IncrementUsage(success bool)

IncrementUsage updates usage statistics after a request.

func (*Credential) IsExpired

func (c *Credential) IsExpired() bool

IsExpired reports whether the credential has passed its expiry time.

func (*Credential) IsQuotaExceeded

func (c *Credential) IsQuotaExceeded() bool

IsQuotaExceeded reports whether the credential's quota is currently exceeded.

func (*Credential) MarkQuotaExceeded

func (c *Credential) MarkQuotaExceeded(resetAt time.Time)

MarkQuotaExceeded marks this credential as quota-exceeded until resetAt.

func (*Credential) Priority

func (c *Credential) Priority() int

Priority returns the selection priority for this credential.

type Embedding added in v0.0.5

type Embedding struct {
	Index     int       `json:"index"`
	Values    []float64 `json:"-"`
	B64Values string    `json:"-"`
}

func (Embedding) MarshalJSON added in v0.0.5

func (e Embedding) MarshalJSON() ([]byte, error)

MarshalJSON renders one embedding entry in the OpenAI shape: float array by default, base64 float32-LE when the client asked for base64.

func (*Embedding) UnmarshalJSON added in v0.0.5

func (e *Embedding) UnmarshalJSON(raw []byte) error

UnmarshalJSON accepts the plugin-side embedding entry: the OpenAI {index, embedding:[...]} shape with float values.

type EmbeddingsRequest added in v0.0.5

type EmbeddingsRequest struct {
	Model          ModelId  `json:"model"`
	Input          []string `json:"-"`
	EncodingFormat string   `json:"encoding_format,omitempty"` // float (default) or base64
	Dimensions     int      `json:"dimensions,omitempty"`
}

EmbeddingsRequest is the parsed body of POST /v1/embeddings. Input is normalized to a list of strings at the edge: OpenAI accepts a single string, a list of strings, or token arrays; token arrays are decoded by clients upstream of this router and are refused here.

func (*EmbeddingsRequest) UnmarshalJSON added in v0.0.5

func (r *EmbeddingsRequest) UnmarshalJSON(raw []byte) error

embeddingsRequestJSON decodes the polymorphic OpenAI input field.

type EmbeddingsResponse added in v0.0.5

type EmbeddingsResponse struct {
	Data  []Embedding      `json:"data"`
	Model string           `json:"model"`
	Usage *EmbeddingsUsage `json:"usage,omitempty"`
}

EmbeddingsResponse is the normalized plugin return and the client wire response. Plugins always return float vectors; the edge renders encoding_format=base64 from them.

type EmbeddingsUsage added in v0.0.5

type EmbeddingsUsage struct {
	PromptTokens int `json:"prompt_tokens"`
	TotalTokens  int `json:"total_tokens"`
}

type ErrorResponse

type ErrorResponse struct {
	Error string `json:"error" example:"invalid request"`
}

ErrorResponse is the standard error response format for all API endpoints.

type ErrorType

type ErrorType int

ErrorType classifies provider errors for key iteration inside backends.

const (
	ErrorTypeUnknown        ErrorType = iota
	ErrorTypeRateLimit                // Temporary rate limit on this key
	ErrorTypeQuotaExceeded            // Credential quota exhausted, deprioritize (MUST have RetryAfter)
	ErrorTypeAuth                     // Auth failure, credential may be invalid
	ErrorTypeUpstream                 // Transient upstream failure (5xx, overload)
	ErrorTypeTimeout                  // Transient timeout
	ErrorTypeInvalidRequest           // Invalid request
	ErrorTypeGeo                      // Geo-blocked upstream; proxy used is at fault, mark it bad
)

type ImageData added in v0.0.5

type ImageData struct {
	URL           string `json:"url,omitempty"`
	B64JSON       string `json:"b64_json,omitempty"`
	RevisedPrompt string `json:"revised_prompt,omitempty"`
}

type ImageGenerationRequest added in v0.0.5

type ImageGenerationRequest struct {
	Model          ModelId `json:"model"`
	Prompt         string  `json:"prompt"`
	N              int     `json:"n,omitempty"`
	Size           string  `json:"size,omitempty"`
	Quality        string  `json:"quality,omitempty"`
	Style          string  `json:"style,omitempty"`
	ResponseFormat string  `json:"response_format,omitempty"` // url or b64_json
}

ImageGenerationRequest is the parsed body of POST /v1/images/generations.

type ImageGenerationResponse added in v0.0.5

type ImageGenerationResponse struct {
	Created int64       `json:"created"`
	Data    []ImageData `json:"data"`
}

ImageGenerationResponse is the normalized plugin return and the client wire response: the OpenAI images shape.

type MetricEvent

type MetricEvent struct {
	Timestamp    time.Time
	ProviderID   string
	ProviderType string
	Model        ModelId
	TokenID      string
	Duration     time.Duration
	StatusCode   int
	TokensInput  int64
	TokensOutput int64
	ErrorType    string
}

MetricEvent represents a single API request event.

type MetricsFilters

type MetricsFilters struct {
	ProviderID string    `json:"provider_id"`
	Model      ModelId   `json:"model"`
	TimeRange  TimeRange `json:"time_range"`
}

MetricsFilters for querying metrics.

type MetricsOverview

type MetricsOverview struct {
	TotalRequests    int64 `json:"total_requests" example:"1500"`
	TotalErrors      int64 `json:"total_errors" example:"23"`
	PeakRequests     int64 `json:"peak_requests" example:"45"`
	PeakInputTokens  int64 `json:"peak_input_tokens" example:"12000"`
	PeakOutputTokens int64 `json:"peak_output_tokens" example:"3500"`
}

MetricsOverview for dashboard display.

type ModelId

type ModelId string

ModelId uniquely identifies a model in the format: provider/model-name[:version]

func (ModelId) Parse

func (m ModelId) Parse() (providerID, model string, err error)

Parse splits a ModelId into providerID and model name.

func (ModelId) ParseFull

func (m ModelId) ParseFull() (adapterType, qualifier, model string, err error)

ParseFull splits a ModelId into adapter type, qualifier, and model name.

func (ModelId) String

func (m ModelId) String() string

type ModelInfo

type ModelInfo struct {
	Name          string `json:"name"`
	DisplayName   string `json:"display_name"`
	Description   string `json:"description,omitempty"`
	RPM           int64  `json:"rpm"`
	TPM           int64  `json:"tpm"`
	RPD           int64  `json:"rpd"`
	ContextWindow int64  `json:"context_window,omitempty"`
	MaxTokens     int64  `json:"max_tokens,omitempty"`

	// Capabilities are UI-facing feature chips (tools, vision, ...).
	// Derived from the fields below when the plugin does not set them.
	Capabilities []string `json:"capabilities,omitempty"`

	InputModalities     []string        `json:"input_modalities,omitempty"`
	OutputModalities    []string        `json:"output_modalities,omitempty"`
	SupportedParameters []string        `json:"supported_parameters,omitempty"`
	Reasoning           *ModelReasoning `json:"reasoning,omitempty"`

	// Endpoints lists the served endpoint identifiers (Endpoint* constants).
	// Empty means chat/completions only.
	Endpoints []string `json:"endpoints,omitempty"`
}

ModelInfo contains metadata about a specific model. Field style follows the OpenRouter model schema where applicable.

func (*ModelInfo) DeriveCapabilities

func (m *ModelInfo) DeriveCapabilities()

DeriveCapabilities fills Capabilities from the OpenRouter-style fields. Explicit plugin-provided capabilities win.

func (*ModelInfo) SupportsEndpoint added in v0.0.5

func (m *ModelInfo) SupportsEndpoint(endpoint string) bool

SupportsEndpoint reports whether the model serves endpoint. Empty Endpoints implies chat/completions only.

type ModelOverride

type ModelOverride struct {
	ProviderID   string   `json:"provider_id"`
	Name         string   `json:"name"`
	Disabled     bool     `json:"disabled,omitempty"`
	Custom       bool     `json:"custom,omitempty"`
	DisplayName  string   `json:"display_name,omitempty"`
	Capabilities []string `json:"capabilities,omitempty"`
}

ModelOverride is the per-provider admin override for a single model: disable from routing and /v1/models, register a custom model the upstream does not list, or adjust display metadata.

type ModelReasoning

type ModelReasoning struct {
	// SupportedEfforts in descending effort order (highest first).
	SupportedEfforts []string `json:"supported_efforts,omitempty"`
	DefaultEffort    string   `json:"default_effort,omitempty"`
	DefaultEnabled   bool     `json:"default_enabled,omitempty"`
	Mandatory        bool     `json:"mandatory,omitempty"`
}

ModelReasoning describes reasoning support of a model (OpenRouter-style).

type OpenAIError

type OpenAIError struct {
	Error OpenAIErrorBody `json:"error"`
}

OpenAIError wraps error responses in the OpenAI error format.

type OpenAIErrorBody

type OpenAIErrorBody struct {
	Message string  `json:"message" example:"Invalid request: missing required field 'model'"`
	Type    string  `json:"type" example:"invalid_request_error"`
	Param   *string `json:"param" example:"model"`
	Code    string  `json:"code,omitempty" example:"invalid_request"`
}

type PluginInternalError

type PluginInternalError struct {
	PluginID string
	TypeKey  string
	Cause    string
}

PluginInternalError is a caught Lua failure at the Go/Lua boundary.

func (*PluginInternalError) Error

func (e *PluginInternalError) Error() string

type PromptTokensDetails

type PromptTokensDetails struct {
	CachedTokens int `json:"cached_tokens,omitempty"`
	AudioTokens  int `json:"audio_tokens,omitempty"`
}

PromptTokensDetails is the OpenAI breakdown of prompt-side token usage.

type Provider

type Provider = ProviderInstance

Provider is kept as an alias so existing call sites keep compiling while the field migration (Type -> TypeKey, BaseURL -> Config) proceeds.

type ProviderError

type ProviderError struct {
	StatusCode int
	Message    string
	Type       ErrorType
	RetryAfter *time.Time
}

ProviderError represents errors returned by provider backends.

func (*ProviderError) Error

func (e *ProviderError) Error() string

type ProviderInstance

type ProviderInstance struct {
	ID           string         `json:"id" example:"opencode-zen"`
	Name         string         `json:"name" example:"OpenCode Zen"`
	TypeKey      string         `json:"type_key" example:"opencode-zen"`
	Qualifier    string         `json:"qualifier" example:""`
	Config       map[string]any `json:"config"`
	IconURL      string         `json:"icon_url" example:"https://cdn.example.com/openai.svg"`
	IsUIReadonly bool           `json:"is_ui_readonly"`
	IsUIHidden   bool           `json:"is_ui_hidden"`
	// Disabled takes the provider out of routing and model listings.
	// Settings and discovery keep working.
	Disabled  bool      `json:"disabled,omitempty"`
	CreatedAt time.Time `json:"created_at"`
	UpdatedAt time.Time `json:"updated_at"`
}

ProviderInstance is the single persisted provider record for every type: lua-plugin type keys, built-in "custom" and "virtual".

func (*ProviderInstance) BaseURL

func (p *ProviderInstance) BaseURL() string

BaseURL returns Config["base_url"] for OpenAI-compatible providers.

type ProviderInstanceCreateRequest

type ProviderInstanceCreateRequest struct {
	Name      string         `json:"name"`
	TypeKey   string         `json:"type_key"`
	Qualifier string         `json:"qualifier"`
	Config    map[string]any `json:"config"`
	IconURL   string         `json:"icon_url"`
}

ProviderInstanceCreateRequest is the generic create body for any provider type.

type ProviderInstanceUpdateRequest

type ProviderInstanceUpdateRequest struct {
	Name     string         `json:"name"`
	Config   map[string]any `json:"config"`
	IconURL  string         `json:"icon_url"`
	Disabled *bool          `json:"disabled"`
}

ProviderInstanceUpdateRequest is the generic update body.

type ProviderStats

type ProviderStats struct {
	ModelCount      int `json:"model_count" example:"5"`
	CredentialCount int `json:"credential_count" example:"2"`
}

ProviderStats holds aggregated statistics for a provider.

type Proxy

type Proxy struct {
	ID       string `json:"id"`
	URL      string `json:"url"`      // scheme://[user:pass@]host:port
	Protocol string `json:"protocol"` // http, https, socks4, socks5
	Host     string `json:"host"`
	Port     int    `json:"port"`
	Country  string `json:"country"` // ISO 3166-1 alpha-2, empty when unknown
	Source   string `json:"source"`  // "manual" or "list:<source type key>"

	Alive       bool      `json:"alive"`
	LatencyMs   int64     `json:"latency_ms"`
	LastCheckAt time.Time `json:"last_check_at,omitempty"`

	// Health per provider type key: a proxy region-locked by one provider
	// stays usable for the others.
	ProviderHealth map[string]ProxyHealth `json:"provider_health,omitempty"`

	CreatedAt time.Time `json:"created_at"`
}

Proxy is one outbound proxy endpoint, either manually registered or synced from a proxy-list source plugin.

type ProxyCandidate

type ProxyCandidate struct {
	Protocol string `json:"protocol"`
	Host     string `json:"host"`
	Port     int    `json:"port"`
	Country  string `json:"country"`
}

ProxyCandidate is one proxy entry produced by a source plugin.

type ProxyConfig

type ProxyConfig struct {
	Mode string   `json:"mode"`          // "disabled" (default), "auto", "manual"
	IDs  []string `json:"ids,omitempty"` // manual mode: chosen proxy IDs
}

ProxyConfig is the provider-level proxy mode stored in ProviderInstance.Config.

type ProxyHealth

type ProxyHealth struct {
	OK        bool      `json:"ok"`
	LatencyMs int64     `json:"latency_ms"`
	CheckedAt time.Time `json:"checked_at"`
}

ProxyHealth is the last probe outcome for one provider.

type ProxyPreferences

type ProxyPreferences struct {
	Location string `json:"location,omitempty"` // ISO country code, e.g. "US"
}

ProxyPreferences come from a plugin manifest: where the upstream expects requests to originate.

type RouterConfiguration

type RouterConfiguration struct {
	IsClusterNode    bool `json:"is_cluster_node"`
	DisableTelemetry bool `json:"disable_telemetry"`
	// ServerCountry is an optional manual override (ISO 3166-1 alpha-2) for
	// the geo-IP detected server location, used for proxy preference matching.
	ServerCountry string `json:"server_country,omitempty"`
}

RouterConfiguration holds deployment-wide instance settings.

func (RouterConfiguration) Validate

func (c RouterConfiguration) Validate() error

Validate checks the instance settings.

type RouterToken

type RouterToken struct {
	ID        string     `json:"id" example:"token123"`
	Name      string     `json:"name" example:"Production API Token"`
	Token     string     `json:"token,omitempty" example:"llmr_abc123def456"`
	TokenHash string     `json:"token_hash,omitempty"`
	Rules     TokenRules `json:"rules"`
	CreatedAt time.Time  `json:"created_at" example:"2026-05-04T15:00:00Z"`
}

RouterToken is an opaque bearer token issued by llm-router itself.

type SpeechRequest added in v0.0.5

type SpeechRequest struct {
	Model          ModelId  `json:"model"`
	Input          string   `json:"input"`
	Voice          string   `json:"voice"`
	ResponseFormat string   `json:"response_format,omitempty"` // mp3 (default), opus, aac, flac, wav, pcm
	Speed          *float64 `json:"speed,omitempty"`
	Instructions   string   `json:"instructions,omitempty"`
}

SpeechRequest is the parsed body of POST /v1/audio/speech.

type SpeechResponse added in v0.0.5

type SpeechResponse struct {
	Audio  []byte
	Format string // mp3, opus, aac, flac, wav, pcm
}

SpeechResponse is the normalized plugin return: raw audio bytes plus the format they are encoded in. The router serves the bytes with the matching Content-Type; no transcoding happens inside the router.

type StreamChunk

type StreamChunk struct {
	ID      string               `json:"id"`
	Object  string               `json:"object"`
	Created int64                `json:"created"`
	Model   string               `json:"model"`
	Choices []StreamChunkChoice  `json:"choices"`
	Usage   *ChatCompletionUsage `json:"usage,omitempty"`
}

StreamChunk is a single SSE data payload for streaming responses.

type StreamChunkChoice

type StreamChunkChoice struct {
	Index        int         `json:"index"`
	Delta        ChatMessage `json:"delta"`
	FinishReason *string     `json:"finish_reason"`
	Logprobs     any         `json:"logprobs,omitempty"`
}

type StreamOptions

type StreamOptions struct {
	IncludeUsage bool `json:"include_usage,omitempty"`
}

StreamOptions for streaming

type TimeRange

type TimeRange string

TimeRange represents a time window for metrics queries.

const (
	TimeRangeHour      TimeRange = "hour"
	TimeRange1Day      TimeRange = "1d"
	TimeRange7Days     TimeRange = "7d"
	TimeRange28Days    TimeRange = "28d"
	TimeRange90Days    TimeRange = "90d"
	TimeRangeThisMonth TimeRange = "month"
)

func (TimeRange) Bounds

func (tr TimeRange) Bounds() (start, end time.Time)

Bounds returns the start and end times for this time range.

type TimeSeriesPoint

type TimeSeriesPoint struct {
	Timestamp time.Time `json:"timestamp" example:"2026-05-04T19:00:00Z"`
	Value     int64     `json:"value" example:"42"`
}

TimeSeriesPoint for chart data.

type TokenRules

type TokenRules struct {
	AllowedProviders    []string  `json:"allowed_providers"`
	AllowAllProviders   bool      `json:"allow_all_providers"`
	AllowedModels       []ModelId `json:"allowed_models"`
	AllowAllModels      bool      `json:"allow_all_models"`
	AllowedCredentials  []string  `json:"allowed_credentials"`
	AllowAllCredentials bool      `json:"allow_all_credentials"`
}

TokenRules constrains what an API token is allowed to do.

func (TokenRules) Allows

func (r TokenRules) Allows(model ModelId) bool

Allows reports whether the token may request the given model.

func (TokenRules) AllowsCredential

func (r TokenRules) AllowsCredential(credentialID string) bool

AllowsCredential reports whether the token may use the given credential.

func (TokenRules) AllowsProvider

func (r TokenRules) AllowsProvider(providerID string) bool

AllowsProvider reports whether the token may use the given provider.

type TranscriptionRequest added in v0.0.5

type TranscriptionRequest struct {
	Model                  ModelId
	File                   []byte
	FileName               string
	ContentType            string
	Language               string
	Prompt                 string
	ResponseFormat         string // json (default), text, srt, verbose_json, vtt
	Temperature            *float64
	TimestampGranularities []string // word, segment
}

TranscriptionRequest is the parsed multipart body of POST /v1/audio/transcriptions. File holds the raw audio bytes.

type TranscriptionResponse added in v0.0.5

type TranscriptionResponse struct {
	Text     string                 `json:"text"`
	Language string                 `json:"language,omitempty"`
	Duration float64                `json:"duration,omitempty"`
	Segments []TranscriptionSegment `json:"segments,omitempty"`
	Words    []TranscriptionWord    `json:"words,omitempty"`
}

TranscriptionResponse is the normalized plugin return: the OpenAI verbose_json shape. The router renders the client-facing response_format from these fields.

func (*TranscriptionResponse) SRT added in v0.0.5

func (r *TranscriptionResponse) SRT() string

SRT renders segments as a SubRip subtitle document.

func (*TranscriptionResponse) VTT added in v0.0.5

func (r *TranscriptionResponse) VTT() string

VTT renders segments as a WebVTT subtitle document.

type TranscriptionSegment added in v0.0.5

type TranscriptionSegment struct {
	ID               int     `json:"id"`
	Seek             int     `json:"seek,omitempty"`
	Start            float64 `json:"start"`
	End              float64 `json:"end"`
	Text             string  `json:"text"`
	Tokens           []int   `json:"tokens,omitempty"`
	Temperature      float64 `json:"temperature,omitempty"`
	AvgLogprob       float64 `json:"avg_logprob,omitempty"`
	CompressionRatio float64 `json:"compression_ratio,omitempty"`
	NoSpeechProb     float64 `json:"no_speech_prob,omitempty"`
}

type TranscriptionWord added in v0.0.5

type TranscriptionWord struct {
	Word  string  `json:"word"`
	Start float64 `json:"start"`
	End   float64 `json:"end"`
}

type UINode

type UINode struct {
	Type         string            `json:"type"`
	Text         string            `json:"text,omitempty"`
	Name         string            `json:"name,omitempty"`
	Label        string            `json:"label,omitempty"`
	InputType    string            `json:"input_type,omitempty"`
	Required     bool              `json:"required,omitempty"`
	Options      []string          `json:"options,omitempty"`
	OptionLabels map[string]string `json:"option_labels,omitempty"`
	URL          string            `json:"url,omitempty"`
	Variant      string            `json:"variant,omitempty"`
	FormAction   string            `json:"form_action,omitempty"`
	Content      []*UINode         `json:"content,omitempty"`
	Placeholder  string            `json:"placeholder,omitempty"`
	Value        any               `json:"value,omitempty"`
	Direction    string            `json:"direction,omitempty"`
	Align        string            `json:"align,omitempty"`
	Justify      string            `json:"justify,omitempty"`
	Gap          string            `json:"gap,omitempty"`
	Columns      int               `json:"columns,omitempty"`
	Title        string            `json:"title,omitempty"`
	Subtitle     string            `json:"subtitle,omitempty"`
	Wrap         bool              `json:"wrap,omitempty"`
	Grow         bool              `json:"grow,omitempty"`
	Size         string            `json:"size,omitempty"`
	Extra        map[string]any    `json:"-"`
}

UINode is a single node of the lua-driven UI tree shared by config_schema, credential_schema and auth wizards.

type VirtualModel added in v0.0.5

type VirtualModel struct {
	ID          string              `json:"id"`
	Name        string              `json:"name"`
	Description string              `json:"description"`
	Models      []VirtualModelEntry `json:"models"`
	// Instruction is prepended to requests as the first user message.
	Instruction string `json:"instruction"`
	// Capabilities is the intersection of all models' capabilities.
	Capabilities []string `json:"capabilities,omitempty"`
	// ContextLength / MaxCompletionTokens are the minima across all models.
	ContextLength       int64     `json:"context_length,omitempty"`
	MaxCompletionTokens int64     `json:"max_completion_tokens,omitempty"`
	Version             int       `json:"version"`
	CreatedAt           time.Time `json:"created_at"`
	UpdatedAt           time.Time `json:"updated_at"`
}

VirtualModel routes requests to its models in list order (fall-through) and reports an error only when every model failed.

type VirtualModelEntry added in v0.0.5

type VirtualModelEntry struct {
	ModelID ModelId `json:"model_id"`
}

VirtualModelEntry is one fall-through step. List order is the priority.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL