Documentation
¶
Overview ¶
Package config provides configuration management for the application.
Index ¶
- Constants
- Variables
- func DefaultPIDFilePath() string
- func DeriveMCPServerSlug(name string) string
- func IsLocalModelListSource(location string) bool
- func JoinBasePath(basePath, urlPath string) string
- func LoadFailoverPolicy(cfg *FailoverConfig) error
- func MCPServerEnabled(server MCPServerConfig) bool
- func NormalizeAllowedOrigin(origin string) (string, error)
- func NormalizeBasePath(value string) string
- func NormalizeBreakerScope(scope string) string
- func NormalizeHeaderName(value, fallback string) (string, error)
- func OriginHost(normalizedOrigin string) string
- func ParseBodySizeLimitBytes(s string) (int64, error)
- func ParseResilienceStatuses(entries, defaults []string) (map[int]bool, error)
- func ProviderModelIDs(models []RawProviderModel) []string
- func ProviderModelMetadataOverrides(models []RawProviderModel) map[string]*core.ModelMetadata
- func ResolveMetricsEndpoint(endpoint string) string
- func ResolveMetricsEndpointWithPprof(endpoint string, pprofEnabled bool) string
- func SemanticCacheActive(sem *SemanticCacheConfig) bool
- func SimpleCacheEnabled(s *SimpleCacheConfig) bool
- func ValidateBodySizeLimit(s string) error
- func ValidateCacheConfig(c *CacheConfig) error
- func ValidateMCPServerConfig(server *MCPServerConfig) error
- func ValidateMCPServerName(name string) error
- func ValidateMCPServerSlug(slug string) error
- func ValidateResilience(r ResilienceConfig) error
- type AdminConfig
- type BudgetLabelConfig
- type BudgetLimitConfig
- type BudgetUserPathConfig
- type BudgetsConfig
- type CacheConfig
- type CircuitBreakerConfig
- type Config
- type ConfiguredProviderModelsMode
- type EmbedderConfig
- type FailoverConfig
- type FailoverErrorPhrase
- type FailoverMode
- type GuardrailRuleConfig
- type GuardrailsConfig
- type HTTPConfig
- type ImageBodyScope
- type LLMBasedAlteringSettings
- type LoadResult
- type LocalCacheConfig
- type LogConfig
- type MCPConfig
- type MCPServerConfig
- type MetricsConfig
- type ModelCacheConfig
- type ModelFilter
- type ModelListConfig
- type ModelsConfig
- type MongoDBStorageConfig
- type OpenTelemetryConfig
- type PGVectorConfig
- type PineconeConfig
- type PluginFileConfig
- type PluginsConfig
- type PostgreSQLStorageConfig
- type QdrantConfig
- type RateLimitModelConfig
- type RateLimitProviderConfig
- type RateLimitRuleConfig
- type RateLimitUserPathConfig
- type RateLimitsConfig
- type RawCircuitBreakerConfig
- type RawProviderConfig
- type RawProviderModel
- type RawResilienceConfig
- type RawRetryConfig
- type RedisModelConfig
- type RedisResponseConfig
- type ResilienceConfig
- type ResponseCacheConfig
- type RetryConfig
- type SQLiteStorageConfig
- type SemanticCacheConfig
- type ServerConfig
- type SessionConfig
- type SessionHeaderConfig
- type SimpleCacheConfig
- type StorageConfig
- type SystemPromptSettings
- type TaggingConfig
- type TaggingHeaderConfig
- type UsageConfig
- type UserConfig
- type VectorStoreConfig
- type VersionCheckConfig
- type VirtualModelConfig
- type VirtualModelTargetConfig
- type WeaviateConfig
- type WorkflowsConfig
Constants ¶
const ( MCPTransportHTTP = "http" MCPTransportSSE = "sse" MCPTransportStdio = "stdio" )
MCP transport names accepted in MCPServerConfig.Transport.
const ( DefaultBodySizeLimit int64 = 10 * 1024 * 1024 // 10MB MinBodySizeLimit int64 = 1 * 1024 // 1KB MaxBodySizeLimit int64 = 100 * 1024 * 1024 // 100MB )
Body size limit constants
const DefaultMCPToolTimeout = 30 * time.Second
DefaultMCPToolTimeout bounds a single upstream tools/call when the server declares no timeout of its own.
const DefaultStreamStallTimeoutSeconds = 60
DefaultStreamStallTimeoutSeconds is the default ServerConfig.StreamStallTimeout. It matches the send timeout most reverse proxies apply between two successive writes to a client.
const DefaultTaggingDelimiter = ","
DefaultTaggingDelimiter separates multiple labels inside one header value.
const DefaultVersionCheckURL = versioncheck.DefaultURL
DefaultVersionCheckURL is the public release manifest served by the GoModel website. "/core.txt" or "/pro.txt" is appended per distribution.
const LegacyPIDFilePath = "data/gomodel.pid"
LegacyPIDFilePath is the pid file location used next to a project-local ./data directory, matching where the SQLite database lands in the same setup.
const TrustAnyOrigin = "*"
TrustAnyOrigin is the mcp.allowed_origins entry that trusts every browser origin. It exists for deployments that enforce their own origin checks in front of the gateway, and disables the gateway's DNS-rebinding defense.
Variables ¶
var DefaultFailoverRetryErrors = []string{
"model not found",
"model does not exist",
"model unsupported",
"model unavailable",
"model not available",
"model deprecated",
"model retired",
"model disabled",
"upstream failed",
"upstream error",
"upstream unavailable",
"upstream timed out",
"upstream timeout",
"404 unsupported",
"404 unavailable",
"404 not available",
"404 deprecated",
"404 retired",
"404 disabled",
}
DefaultFailoverRetryErrors is the failover.retry_on_errors default. The phrases cover a model that is gone or refused whatever status the provider chose, an aggregator provider relaying a failure of its own upstream behind a 4xx, and retired-model 404s that avoid the word "model". Plain endpoint 404s and validation errors match none of them.
var DefaultFailoverRetryStatuses = []string{"429", "5xx"}
DefaultFailoverRetryStatuses is the failover.retry_on_statuses default: rate limits and every server-side failure.
Functions ¶
func DefaultPIDFilePath ¶ added in v0.1.68
func DefaultPIDFilePath() string
DefaultPIDFilePath returns the pid file path used when none is configured: LegacyPIDFilePath when a ./data directory already exists (Docker images and existing deployments), otherwise the OS-conventional per-user data directory — the same resolution the database uses, so both land together.
func DeriveMCPServerSlug ¶
DeriveMCPServerSlug creates a conservative default slug from a display name. Callers may let users edit it before creation; once persisted it is a stable identity and should not change when the display name changes.
func IsLocalModelListSource ¶ added in v0.1.88
IsLocalModelListSource reports whether a model list location names a file on the local filesystem ("file://..." or a bare path) rather than an HTTP URL. It mirrors what the catalog fetcher accepts.
func JoinBasePath ¶
JoinBasePath prefixes urlPath with the normalized public mount path.
func LoadFailoverPolicy ¶ added in v0.1.84
func LoadFailoverPolicy(cfg *FailoverConfig) error
LoadFailoverPolicy validates the retry policy fields of cfg and fills the parsed RetryStatuses and RetryErrors, applying the defaults for empty lists.
func MCPServerEnabled ¶
func MCPServerEnabled(server MCPServerConfig) bool
MCPServerEnabled reports the effective enabled state (default true).
func NormalizeAllowedOrigin ¶ added in v0.1.80
NormalizeAllowedOrigin canonicalizes one origin for comparison against an Origin header: scheme and host lowercased, port preserved. Origins are compared exactly, so anything that is not a bare "scheme://host[:port]" is rejected rather than silently widened.
func NormalizeBasePath ¶
NormalizeBasePath canonicalizes the public mount path for the HTTP server. Empty, whitespace-only, and "/" all resolve to root.
func NormalizeBreakerScope ¶ added in v0.1.88
NormalizeBreakerScope resolves the empty scope to the provider default.
func NormalizeHeaderName ¶
NormalizeHeaderName canonicalizes an HTTP header field name. Empty values fall back to fallback.
func OriginHost ¶ added in v0.1.80
OriginHost returns the "host[:port]" of an origin already canonicalized by NormalizeAllowedOrigin, which is the form a Host header carries.
func ParseBodySizeLimitBytes ¶
ParseBodySizeLimitBytes parses a configured body size limit into bytes. Accepts formats like: "10M", "10MB", "1024K", "1024KB", "104857600". Returns an error if the format is invalid or value is outside bounds (1KB - 100MB).
func ParseResilienceStatuses ¶ added in v0.1.88
ParseResilienceStatuses expands exact HTTP codes and classes. Nil uses the supplied defaults; an explicit empty list disables status-based matches.
func ProviderModelIDs ¶
func ProviderModelIDs(models []RawProviderModel) []string
ProviderModelIDs returns the ID of each model entry, preserving order and dropping entries with empty IDs.
func ProviderModelMetadataOverrides ¶
func ProviderModelMetadataOverrides(models []RawProviderModel) map[string]*core.ModelMetadata
ProviderModelMetadataOverrides returns id -> metadata for entries with non-nil Metadata. Returns nil if no entries declare metadata.
func ResolveMetricsEndpoint ¶ added in v0.1.74
ResolveMetricsEndpoint returns the normalized, safe endpoint used by the HTTP server. Extensions should use the same value when excluding Prometheus scrapes from request instrumentation.
func ResolveMetricsEndpointWithPprof ¶ added in v0.1.74
ResolveMetricsEndpointWithPprof also prevents the metrics route from shadowing an enabled pprof route.
func SemanticCacheActive ¶
func SemanticCacheActive(sem *SemanticCacheConfig) bool
SemanticCacheActive reports whether the semantic response cache should be validated and constructed. The semantic block must be present (YAML or SEMANTIC_CACHE_ENABLED=true); omitted enabled means true.
func SimpleCacheEnabled ¶
func SimpleCacheEnabled(s *SimpleCacheConfig) bool
SimpleCacheEnabled reports whether the exact-match response cache layer is allowed to run for a non-nil simple config. Omitted enabled means true.
func ValidateBodySizeLimit ¶
ValidateBodySizeLimit validates a body size limit string. Accepts formats like: "10M", "10MB", "1024K", "1024KB", "104857600" Returns an error if the format is invalid or value is outside bounds (1KB - 100MB).
func ValidateCacheConfig ¶
func ValidateCacheConfig(c *CacheConfig) error
ValidateCacheConfig validates the cache configuration in c. For the model cache, at least one backend (Local or Redis) must be configured; having neither is an error. Both may be set: Redis is preferred and Local is the fallback if Redis is unreachable at startup. When Redis is selected, its URL must be non-empty. Returns a descriptive error if any constraint is violated, or nil if the configuration is valid.
func ValidateMCPServerConfig ¶
func ValidateMCPServerConfig(server *MCPServerConfig) error
ValidateMCPServerConfig validates one server definition and applies defaults in place. It is shared by config loading and the admin API.
func ValidateMCPServerName ¶
ValidateMCPServerName accepts a human-facing Unicode display name. Machine constraints belong to the separate immutable slug.
func ValidateMCPServerSlug ¶
ValidateMCPServerSlug validates the stable ASCII identity used in routes, scope headers, and aggregated tool/prompt names.
func ValidateResilience ¶ added in v0.1.88
func ValidateResilience(r ResilienceConfig) error
ValidateResilience checks policies for both file loading and programmatic providers.
Types ¶
type AdminConfig ¶
type AdminConfig struct {
// EndpointsEnabled controls whether the admin REST API is active
// Default: true
EndpointsEnabled bool `yaml:"endpoints_enabled" env:"ADMIN_ENDPOINTS_ENABLED"`
// UIEnabled controls whether the admin dashboard UI is active
// Requires EndpointsEnabled — if endpoints are disabled and UI is enabled,
// a warning is logged and UI is forced to false.
// Default: true
UIEnabled bool `yaml:"ui_enabled" env:"ADMIN_UI_ENABLED"`
// LiveLogsEnabled controls whether the dashboard opens a realtime log stream.
// Default: true
LiveLogsEnabled bool `yaml:"live_logs_enabled" env:"DASHBOARD_LIVE_LOGS_ENABLED"`
// LiveLogsBufferSize is the in-memory replay window for dashboard live log events.
// Default: 10000
LiveLogsBufferSize int `yaml:"live_logs_buffer_size" env:"DASHBOARD_LIVE_LOGS_BUFFER_SIZE"`
// LiveLogsReplayLimit caps events replayed to one reconnecting dashboard client.
// Default: 1000
LiveLogsReplayLimit int `yaml:"live_logs_replay_limit" env:"DASHBOARD_LIVE_LOGS_REPLAY_LIMIT"`
// LiveLogsHeartbeatSeconds keeps idle stream connections and proxies active.
// Default: 15
LiveLogsHeartbeatSeconds int `yaml:"live_logs_heartbeat_seconds" env:"DASHBOARD_LIVE_LOGS_HEARTBEAT_SECONDS"`
}
AdminConfig holds configuration for the admin API and dashboard UI.
type BudgetLabelConfig ¶ added in v0.1.60
type BudgetLabelConfig struct {
Label string `yaml:"label"`
Limits []BudgetLimitConfig `yaml:"limits"`
}
BudgetLabelConfig declares one or more budget limits for a request label.
type BudgetLimitConfig ¶
type BudgetLimitConfig struct {
// Period accepts hourly, daily, weekly, or monthly. The resolved period is
// persisted as PeriodSeconds in the database.
Period string `yaml:"period" json:"period"`
// PeriodSeconds can be set directly instead of Period. Standard values are
// 3600, 86400, 604800, and 2592000.
PeriodSeconds int64 `yaml:"period_seconds" json:"period_seconds"`
// Amount is the maximum allowed tracked provider spend for the period.
Amount float64 `yaml:"amount" json:"amount"`
// PerChild is accepted only in the JSON-array environment-variable form.
// YAML keeps this setting on the enclosing user-path entry.
PerChild bool `yaml:"-" json:"per_child,omitempty"`
}
BudgetLimitConfig declares one spend limit for a reset period. The json tags support the JSON-array form of SET_BUDGET_* env values.
type BudgetUserPathConfig ¶
type BudgetUserPathConfig struct {
Path string `yaml:"path"`
PerChild bool `yaml:"per_child"`
Limits []BudgetLimitConfig `yaml:"limits"`
}
BudgetUserPathConfig declares one or more budget limits for a user path.
type BudgetsConfig ¶
type BudgetsConfig struct {
// Enabled controls whether budget checks are active.
// Default: true. Requires usage tracking because spend limits are evaluated
// from usage cost records.
Enabled bool `yaml:"enabled" env:"BUDGETS_ENABLED"`
// UserPaths declares budget limits by tracked user path.
UserPaths []BudgetUserPathConfig `yaml:"user_paths"`
// Labels declares budget limits by request label. A request carrying
// several labels is charged against every matching label budget.
//
// Labels have no env-var form: they are matched verbatim and may contain
// characters and casing that env var names cannot express. Declare them
// here or through the admin API.
Labels []BudgetLabelConfig `yaml:"labels"`
}
BudgetsConfig holds per-user-path spend limits.
type CacheConfig ¶
type CacheConfig struct {
Model ModelCacheConfig `yaml:"model"`
Response ResponseCacheConfig `yaml:"response"`
}
CacheConfig holds model and response cache configuration.
type CircuitBreakerConfig ¶
type CircuitBreakerConfig struct {
// FailureOnStatuses uses defaults when nil; an empty list disables status triggers.
FailureOnStatuses []string `yaml:"failure_on_statuses" env:"CIRCUIT_BREAKER_FAILURE_ON_STATUSES"`
Scope string `yaml:"scope" env:"CIRCUIT_BREAKER_SCOPE"`
// Enabled switches the circuit breaker on or off. When false, requests are
// never short-circuited regardless of the thresholds below.
// Default: true
Enabled bool `yaml:"enabled" env:"CIRCUIT_BREAKER_ENABLED"`
FailureThreshold int `yaml:"failure_threshold" env:"CIRCUIT_BREAKER_FAILURE_THRESHOLD"`
SuccessThreshold int `yaml:"success_threshold" env:"CIRCUIT_BREAKER_SUCCESS_THRESHOLD"`
Timeout time.Duration `yaml:"timeout" env:"CIRCUIT_BREAKER_TIMEOUT"`
}
CircuitBreakerConfig holds resolved circuit breaker settings. This is the canonical type shared between config and llmclient.
func DefaultCircuitBreakerConfig ¶
func DefaultCircuitBreakerConfig() CircuitBreakerConfig
DefaultCircuitBreakerConfig returns the default circuit breaker settings.
type Config ¶
type Config struct {
Server ServerConfig `yaml:"server"`
Models ModelsConfig `yaml:"models"`
Cache CacheConfig `yaml:"cache"`
Storage StorageConfig `yaml:"storage"`
Logging LogConfig `yaml:"logging"`
Usage UsageConfig `yaml:"usage"`
Budgets BudgetsConfig `yaml:"budgets"`
RateLimits RateLimitsConfig `yaml:"rate_limits"`
Metrics MetricsConfig `yaml:"metrics"`
OpenTelemetry OpenTelemetryConfig `yaml:"opentelemetry"`
HTTP HTTPConfig `yaml:"http"`
Admin AdminConfig `yaml:"admin"`
Guardrails GuardrailsConfig `yaml:"guardrails"`
Failover FailoverConfig `yaml:"failover"`
Workflows WorkflowsConfig `yaml:"workflows"`
Resilience ResilienceConfig `yaml:"resilience"`
Tagging TaggingConfig `yaml:"tagging"`
Session SessionConfig `yaml:"session"`
MCP MCPConfig `yaml:"mcp"`
// Plugins controls which plugin shared objects (.so) are loaded at startup.
Plugins PluginsConfig `yaml:"plugins"`
// VersionCheck controls the daily update check against the public
// release manifest. See VersionCheckConfig for what it sends.
VersionCheck VersionCheckConfig `yaml:"version_check"`
// Offline is the one switch for air-gapped installs. It turns off every
// outbound call the gateway makes on its own: the update check and the
// remote model catalog download. Calls to configured providers and to
// anything else the operator declared (OTLP, MCP upstreams, vector
// stores) are untouched. A catalog served from a local file keeps working.
// Default: false
Offline bool `yaml:"offline" env:"GOMODEL_OFFLINE"`
// Extensions holds configuration owned by custom distributions. Core keeps
// the values opaque; an extension decodes its named section with
// LoadResult.DecodeExtension.
Extensions map[string]yaml.Node `yaml:"extensions,omitempty"`
// VirtualModels declares redirects, load balancers, and access policies as
// infrastructure-as-code. They override admin-store rows of the same source.
VirtualModels []VirtualModelConfig `yaml:"virtual_models"`
// Users declares per-user-path model access policies as
// infrastructure-as-code. They shadow admin-store rows of the same path.
Users []UserConfig `yaml:"users"`
}
Config holds the application configuration.
type ConfiguredProviderModelsMode ¶
type ConfiguredProviderModelsMode string
ConfiguredProviderModelsMode controls how explicitly configured provider model lists are applied to the discovered model inventory.
const ( // ConfiguredProviderModelsModeFallback uses configured models only when the // upstream /models call fails or returns nothing. ConfiguredProviderModelsModeFallback ConfiguredProviderModelsMode = "fallback" // ConfiguredProviderModelsModeAllowlist exposes only the configured models // and skips the upstream /models call. ConfiguredProviderModelsModeAllowlist ConfiguredProviderModelsMode = "allowlist" // ConfiguredProviderModelsModeMerge unions the upstream inventory with the // configured models, so models a provider serves but does not list stay // routable without hiding the discovered ones. ConfiguredProviderModelsModeMerge ConfiguredProviderModelsMode = "merge" )
func NormalizeConfiguredProviderModelsMode ¶
func NormalizeConfiguredProviderModelsMode(mode ConfiguredProviderModelsMode) ConfiguredProviderModelsMode
NormalizeConfiguredProviderModelsMode canonicalizes a configured provider models mode.
func ResolveConfiguredProviderModelsMode ¶
func ResolveConfiguredProviderModelsMode(mode ConfiguredProviderModelsMode) ConfiguredProviderModelsMode
ResolveConfiguredProviderModelsMode canonicalizes mode and applies the process default.
func (ConfiguredProviderModelsMode) Valid ¶
func (m ConfiguredProviderModelsMode) Valid() bool
Valid reports whether mode is one of the supported configured-provider-models modes.
type EmbedderConfig ¶
EmbedderConfig selects how embeddings are generated. Provider must match a key in the top-level providers map when semantic caching is active; that provider's api_key and base_url are reused for POST /v1/embeddings. There is no default provider.
type FailoverConfig ¶
type FailoverConfig struct {
// Enabled controls failover globally. It defaults to true; configured rules
// and workflow policy decide whether any request has failover candidates.
Enabled bool `yaml:"enabled" env:"FAILOVER_ENABLED"`
// MaxAttempts caps how many failover targets one request may try after
// its primary attempt fails. Zero (the default) sweeps every remaining
// target in order.
MaxAttempts int `yaml:"max_attempts" env:"FAILOVER_MAX_ATTEMPTS"`
// RetryOnStatuses lists the upstream HTTP statuses that trigger failover:
// exact codes (408) or whole classes (5xx). Setting it replaces the
// default DefaultFailoverRetryStatuses; an empty list keeps the default.
RetryOnStatuses []string `yaml:"retry_on_statuses" env:"FAILOVER_RETRY_ON_STATUSES"`
// RetryOnErrors lists phrases that trigger failover regardless of status:
// an error qualifies when every word of a phrase appears in its code or
// message. A numeric word (404, 4xx) matches the HTTP status instead of
// the text. Setting it replaces the default DefaultFailoverRetryErrors; an
// empty list keeps the default.
RetryOnErrors []string `yaml:"retry_on_errors" env:"FAILOVER_RETRY_ON_ERRORS"`
// DefaultMode is a deprecated compatibility field. It is accepted from old
// config files and FAILOVER_MODE, but runtime failover is manual-only.
DefaultMode FailoverMode `yaml:"default_mode" env:"FAILOVER_MODE"`
// ManualRulesPath points to a JSON file that maps source model selectors to
// ordered failover model selector lists. Empty disables manual rules.
ManualRulesPath string `yaml:"manual_rules_path" env:"FAILOVER_MANUAL_RULES_PATH"`
// Rules defines manual failover rules inline in config.yaml.
Rules map[string][]string `yaml:"rules"`
// RulesJSON defines manual failover rules inline from env.
RulesJSON string `yaml:"rules_json" env:"FAILOVER_RULES_JSON"`
// DisabledModels disables failover for matching source selectors.
DisabledModels []string `yaml:"disabled_models" env:"FAILOVER_DISABLED_MODELS"`
// DisabledModelsJSON disables failover for matching source selectors from
// env. It accepts either a JSON string array or object with boolean values.
DisabledModelsJSON string `yaml:"disabled_models_json" env:"FAILOVER_DISABLED_MODELS_JSON"`
// Overrides is a removed compatibility field. Per-model failover modes are gone;
// operators migrate to DisabledModels. It is still parsed — and ignored, with a
// warning — so an old config file keeps booting under strict YAML validation.
Overrides map[string]any `yaml:"overrides"`
// Manual holds the parsed manual failover lists loaded from ManualRulesPath.
Manual map[string][]string `yaml:"-"`
// Disabled holds normalized per-model failover disables.
Disabled map[string]bool `yaml:"-"`
// RetryStatuses is the parsed RetryOnStatuses (or its default): the set of
// HTTP statuses that trigger failover.
RetryStatuses map[int]bool `yaml:"-"`
// RetryErrors is the parsed RetryOnErrors (or its default): one lower-cased
// word list per phrase.
RetryErrors []FailoverErrorPhrase `yaml:"-"`
}
FailoverConfig holds translated-route model failover policy.
type FailoverErrorPhrase ¶ added in v0.1.84
type FailoverErrorPhrase struct {
// Words are the lower-cased text fragments the error must contain.
Words []string
// Statuses are HTTP statuses the error must carry; empty means any.
Statuses map[int]bool
}
FailoverErrorPhrase is one parsed retry_on_errors entry. Every Word must appear in the error text and every status constraint must hold.
type FailoverMode ¶
type FailoverMode string
const ( FailoverModeOff FailoverMode = "off" FailoverModeManual FailoverMode = "manual" FailoverModeAuto FailoverMode = "auto" )
func ResolveFailoverDefaultMode ¶
func ResolveFailoverDefaultMode(mode FailoverMode) FailoverMode
ResolveFailoverDefaultMode canonicalizes the global failover default mode and applies the process default when unset.
func (FailoverMode) Valid ¶
func (m FailoverMode) Valid() bool
Valid reports whether mode is one of the supported failover modes.
type GuardrailRuleConfig ¶
type GuardrailRuleConfig struct {
// Name is a unique identifier for this guardrail instance (used in logs and errors)
Name string `yaml:"name"`
// Type selects the plugin the instance is built from: a built-in such as
// "system_prompt", "llm_based_altering", "string_replace", "header_edit",
// "llm_judge", "presidio", or the manifest name of a loaded plugin.
Type string `yaml:"type"`
// UserPath scopes internal auxiliary guardrail requests for workflow
// selection and audit logging. When empty, the caller user path is used.
UserPath string `yaml:"user_path"`
// Order controls execution ordering relative to other guardrails.
// Guardrails with the same order run in parallel; different orders run sequentially.
// Default: 0
Order int `yaml:"order"`
// Phase selects where the default workflow runs this instance:
// "prompt" (before the provider call), "response" (on the complete
// response), or "stream" (per streamed event).
// Default: "prompt"
Phase string `yaml:"phase"`
// Config is the plugin's own configuration, validated against the
// plugin's config schema (see GET /admin/guardrails/types). The typed
// system_prompt and llm_based_altering blocks below are folded into it.
Config map[string]any `yaml:"config"`
// FailMode selects what happens when the instance errors or times out:
// "closed" rejects the request with HTTP 500, "open" continues without it.
// Default: empty (closed for prompt/response/stream phases)
FailMode string `yaml:"fail_mode"`
// TimeoutMS bounds every hook call of the instance in milliseconds.
// Default: 0 (no per-instance timeout)
TimeoutMS int `yaml:"timeout_ms"`
// SystemPrompt holds settings when Type is "system_prompt"
SystemPrompt SystemPromptSettings `yaml:"system_prompt"`
// LLMBasedAltering holds settings when Type is "llm_based_altering"
LLMBasedAltering LLMBasedAlteringSettings `yaml:"llm_based_altering"`
}
GuardrailRuleConfig defines a single guardrail instance.
type GuardrailsConfig ¶
type GuardrailsConfig struct {
// Enabled controls whether guardrails are active
// Default: false
Enabled bool `yaml:"enabled" env:"GUARDRAILS_ENABLED"`
// EnableForBatchProcessing controls whether guardrails are applied to inline
// batch items for /v1/batches requests.
// Default: false
EnableForBatchProcessing bool `yaml:"enable_for_batch_processing" env:"ENABLE_GUARDRAILS_FOR_BATCH_PROCESSING"`
// Rules is a list of guardrail instances. Each entry defines one guardrail
// with its own name, type, order, and type-specific settings. Multiple
// instances of the same type are allowed (e.g. two system_prompt guardrails
// with different content).
Rules []GuardrailRuleConfig `yaml:"rules"`
}
GuardrailsConfig holds configuration for the request guardrails pipeline.
type HTTPConfig ¶
type HTTPConfig struct {
// Timeout is the overall HTTP request timeout in seconds (default: 600)
Timeout int `yaml:"timeout" env:"HTTP_TIMEOUT"`
// ResponseHeaderTimeout is the time to wait for response headers in seconds (default: 600)
ResponseHeaderTimeout int `yaml:"response_header_timeout" env:"HTTP_RESPONSE_HEADER_TIMEOUT"`
}
HTTPConfig holds HTTP client configuration for upstream API requests. App startup installs these values into internal/httpclient before providers are constructed; the HTTP_TIMEOUT and HTTP_RESPONSE_HEADER_TIMEOUT env vars take precedence over the YAML values.
type ImageBodyScope ¶ added in v0.1.81
type ImageBodyScope string
ImageBodyScope selects which image bytes the audit log embeds when LogImageBodies is enabled.
const ( ImageBodyScopeAll ImageBodyScope = "all" ImageBodyScopeInput ImageBodyScope = "input" ImageBodyScopeOutput ImageBodyScope = "output" )
func ResolveImageBodyScope ¶ added in v0.1.81
func ResolveImageBodyScope(value ImageBodyScope) ImageBodyScope
ResolveImageBodyScope normalizes a configured scope, defaulting to all.
func (ImageBodyScope) Inputs ¶ added in v0.1.81
func (s ImageBodyScope) Inputs() bool
Inputs reports whether uploaded images (edit sources and masks) are stored.
func (ImageBodyScope) Outputs ¶ added in v0.1.81
func (s ImageBodyScope) Outputs() bool
Outputs reports whether generated images are stored.
func (ImageBodyScope) Valid ¶ added in v0.1.81
func (s ImageBodyScope) Valid() bool
Valid reports whether the scope is one of the supported values.
type LLMBasedAlteringSettings ¶
type LLMBasedAlteringSettings struct {
// Model is the model selector used for the auxiliary rewrite call.
// This can be a concrete model name, provider-qualified selector, or alias.
Model string `yaml:"model"`
// Provider is an optional routing hint for Model.
Provider string `yaml:"provider"`
// Prompt is the system prompt used to rewrite targeted messages.
// When empty, the built-in LiteLLM-derived anonymization prompt is used.
Prompt string `yaml:"prompt"`
// Roles selects which message roles are rewritten.
// Default: ["user"]
Roles []string `yaml:"roles"`
// SkipContentPrefix skips rewriting for messages whose trimmed text begins with this prefix.
SkipContentPrefix string `yaml:"skip_content_prefix"`
// MaxTokens limits the auxiliary rewrite completion.
// Default: 4096
MaxTokens int `yaml:"max_tokens"`
}
LLMBasedAlteringSettings holds the type-specific settings for an llm_based_altering guardrail.
type LoadResult ¶
type LoadResult struct {
Config *Config
RawProviders map[string]RawProviderConfig
}
LoadResult is returned by Load and bundles the application config with the raw provider map parsed from YAML. Provider env vars and resolution are handled by the providers package.
func Load ¶
func Load() (*LoadResult, error)
Load reads configuration from file and environment using a three-layer pipeline:
defaults (code) → config.yaml (optional overlay) → env vars (always win)
The returned LoadResult contains the resolved application Config and the raw provider map parsed from YAML. Provider env var discovery, credential filtering, and resilience merging are handled by the providers package.
func (*LoadResult) DecodeExtension ¶ added in v0.1.71
func (r *LoadResult) DecodeExtension(name string, target any) (bool, error)
DecodeExtension strictly decodes one named extensions: section into target. It returns false when the section is absent. Core deliberately does not know any extension's schema, while each extension still gets unknown-key safety.
type LocalCacheConfig ¶
type LocalCacheConfig struct {
CacheDir string `yaml:"cache_dir" env:"GOMODEL_CACHE_DIR"`
}
LocalCacheConfig holds local file cache configuration.
type LogConfig ¶
type LogConfig struct {
// Enabled controls whether audit logging is active
// Default: false
Enabled bool `yaml:"enabled" env:"LOGGING_ENABLED"`
// LogBodies enables logging of full request/response bodies
// WARNING: May contain sensitive data (PII, API keys in prompts)
// Default: true
LogBodies bool `yaml:"log_bodies" env:"LOGGING_LOG_BODIES"`
// LogAudioBodies refines LogBodies for audio endpoints: when both are
// enabled, the /v1/audio/speech JSON input and binary audio output are
// stored (audio as base64 for playback) and /v1/audio/transcriptions upload
// metadata is recorded. Requires LogBodies (the master body-logging switch);
// when LogBodies is on but this is off, audio responses are recorded as a
// lightweight placeholder instead of the full bytes.
// WARNING: stores full audio in the audit log; grows storage quickly.
// Default: false
LogAudioBodies bool `yaml:"log_audio_bodies" env:"LOGGING_LOG_AUDIO_BODIES"`
// LogImageBodies refines LogBodies for the image endpoints
// (/v1/images/generations, /v1/images/edits): when both are enabled the
// image bytes (uploaded sources and masks, generated outputs) are stored as
// base64 so the dashboard can display them. Requires LogBodies; when
// LogBodies is on but this is off, image bodies keep their metadata
// (prompt, parameters, sizes, usage) and URLs but drop the pixels.
// WARNING: stores full images in the audit log; grows storage quickly.
// Default: false
LogImageBodies bool `yaml:"log_image_bodies" env:"LOGGING_LOG_IMAGE_BODIES"`
// LogImageBodiesScope narrows LogImageBodies to one direction: "all"
// stores uploaded inputs and generated outputs, "input" only the uploads
// (edit sources and masks), "output" only the generated images. Ignored
// while LogImageBodies is off.
// Default: all
LogImageBodiesScope ImageBodyScope `yaml:"log_image_bodies_scope" env:"LOGGING_LOG_IMAGE_BODIES_SCOPE"`
// LogRevisionBodies refines LogBodies for the request-revision chain:
// when both are enabled, every request rewriter that changed the body
// (for example GoModel Pro token compression) and every prompt guardrail
// that edited the prompt store the request as they left it alongside the
// original in the audit entry. Requires LogBodies.
// Disabling it keeps the revision metadata (rewriter name, sizes, tokens
// saved, change detail) but drops the rewritten body copy — roughly
// halving audit storage per compressed request.
// Default: true
LogRevisionBodies bool `yaml:"log_revision_bodies" env:"LOGGING_LOG_REVISION_BODIES"`
// LogGuardrailSteps records every prompt guardrail that edited the
// request as its own revision in the audit entry, carrying the request
// as that step left it, so a chain of edits reads step by step. Each
// step leaves a copy of the prompt behind; building and encoding the
// requests from those copies runs off the request path. Disabling it
// records the
// chain's edits as one revision (the request as forwarded) and skips
// the per-step snapshots.
// Default: true
LogGuardrailSteps bool `yaml:"log_guardrail_steps" env:"LOGGING_LOG_GUARDRAIL_STEPS"`
// LogHeaders enables logging of request/response headers
// Sensitive headers (Authorization, Cookie, etc.) are auto-redacted
// Default: true
LogHeaders bool `yaml:"log_headers" env:"LOGGING_LOG_HEADERS"`
// BufferSize is the number of log entries to buffer before flushing
// Default: 1000
BufferSize int `yaml:"buffer_size" env:"LOGGING_BUFFER_SIZE"`
// FlushInterval is how often to flush buffered logs (in seconds)
// Default: 5
FlushInterval int `yaml:"flush_interval" env:"LOGGING_FLUSH_INTERVAL"`
// RetentionDays is how long to keep logs (0 = forever)
// Default: 30
RetentionDays int `yaml:"retention_days" env:"LOGGING_RETENTION_DAYS"`
// OnlyModelInteractions limits audit logging to AI model endpoints only
// When true, only /v1/chat/completions, /v1/responses, /v1/embeddings, /v1/files, and /v1/batches are logged
// Endpoints like /health, /metrics, /admin, /v1/models are skipped
// Default: true
OnlyModelInteractions bool `yaml:"only_model_interactions" env:"LOGGING_ONLY_MODEL_INTERACTIONS"`
}
LogConfig holds audit logging configuration
type MCPConfig ¶
type MCPConfig struct {
// Enabled gates the /mcp routes. Default: true (a no-op without servers).
Enabled bool `yaml:"enabled" env:"MCP_ENABLED"`
// AllowedOrigins lists the browser origins ("scheme://host[:port]") that
// may reach /mcp. Empty — the default — trusts none, which is right for
// the MCP clients this endpoint serves, since none of them are web pages.
// It is what stops a DNS-rebound page from driving the gateway; only add
// an origin you actually serve an MCP web client from. TrustAnyOrigin
// ("*") turns the check off and is logged as a warning at startup.
AllowedOrigins []string `yaml:"allowed_origins" env:"MCP_ALLOWED_ORIGINS"`
// Servers maps stable server slugs to upstream definitions. Slugs become
// tool namespaces and URL segments, so they are restricted to [a-z0-9_-].
Servers map[string]MCPServerConfig `yaml:"servers"`
}
MCPConfig declares the MCP gateway: upstream MCP servers aggregated behind the authenticated /mcp endpoint. Declarative entries override admin-store rows with the same name and are read-only in the dashboard.
type MCPServerConfig ¶
type MCPServerConfig struct {
// URL is the upstream MCP endpoint for http/sse transports.
URL string `yaml:"url,omitempty" json:"url,omitempty"`
// Transport selects the upstream transport: "http" (streamable HTTP,
// default), "sse" (legacy HTTP+SSE), or "stdio" (spawned subprocess).
Transport string `yaml:"transport,omitempty" json:"transport,omitempty"`
// Headers are sent verbatim on every upstream request (http/sse). Values
// support ${ENV} expansion via the standard config pipeline. This is the
// credential boundary: client bearer tokens are never forwarded upstream.
Headers map[string]string `yaml:"headers,omitempty" json:"headers,omitempty"`
// Command, Args, and Env launch a stdio server as a subprocess. Stdio
// servers are deliberately declarative-only: the admin API and dashboard
// reject them, because registering subprocesses at runtime is a remote
// code execution vector.
Command string `yaml:"command,omitempty" json:"command,omitempty"`
Args []string `yaml:"args,omitempty" json:"args,omitempty"`
Env map[string]string `yaml:"env,omitempty" json:"env,omitempty"`
// Description is an optional human-readable note.
Description string `yaml:"description,omitempty" json:"description,omitempty"`
// Enabled toggles the entry. It defaults to true when omitted.
Enabled *bool `yaml:"enabled,omitempty" json:"enabled,omitempty"`
// AllowedTools restricts the tools exposed from this server (original,
// un-prefixed names). Empty means all tools.
AllowedTools []string `yaml:"allowed_tools,omitempty" json:"allowed_tools,omitempty"`
// DisallowedTools hides specific tools; applied after AllowedTools.
DisallowedTools []string `yaml:"disallowed_tools,omitempty" json:"disallowed_tools,omitempty"`
// UserPaths scopes server visibility to specific request user paths
// (subtree match, same semantics as virtual models). Empty means all.
UserPaths []string `yaml:"user_paths,omitempty" json:"user_paths,omitempty"`
// ToolTimeout bounds a single tools/call against this server.
// Default: 30s.
ToolTimeout time.Duration `yaml:"tool_timeout,omitempty" json:"tool_timeout,omitempty"`
}
MCPServerConfig declares one upstream MCP server.
type MetricsConfig ¶
type MetricsConfig struct {
// Enabled controls whether Prometheus metrics are collected and exposed
// Default: false
Enabled bool `yaml:"enabled" env:"METRICS_ENABLED"`
// Endpoint is the HTTP path where metrics are exposed
// Default: "/metrics"
Endpoint string `yaml:"endpoint" env:"METRICS_ENDPOINT"`
}
MetricsConfig holds observability configuration for Prometheus metrics
type ModelCacheConfig ¶
type ModelCacheConfig struct {
RefreshInterval int `yaml:"refresh_interval" env:"CACHE_REFRESH_INTERVAL"`
// RecheckInterval is how often (seconds) providers whose latest refresh
// failed are re-checked, so outage recovery is detected without waiting
// for the next full refresh. Zero or negative disables the fast recheck.
RecheckInterval int `yaml:"recheck_interval" env:"PROVIDER_RECHECK_INTERVAL"`
ModelList ModelListConfig `yaml:"model_list"`
Local *LocalCacheConfig `yaml:"local"`
Redis *RedisModelConfig `yaml:"redis"`
}
ModelCacheConfig holds cache configuration for model registry. At least one of Local or Redis must be non-nil. When both are set, Redis is preferred and Local is the fallback if Redis is unreachable.
type ModelFilter ¶ added in v0.1.82
type ModelFilter struct {
// Include keeps only models matching at least one pattern. Empty keeps all.
Include []string `yaml:"include"`
// Exclude drops models matching any pattern. It is applied after Include.
Exclude []string `yaml:"exclude"`
// MaxPricePerMtok caps a model's highest per-million-token rate — the
// larger of its input and output rate. A model with no known pricing is
// dropped while the cap is set: a price cap that lets unpriced models
// through is not a cap. Nil disables price filtering.
MaxPricePerMtok *float64 `yaml:"max_price_per_mtok"`
}
ModelFilter narrows a provider's discovered model inventory. It is declared under providers.<name>.model_filter and applied after metadata enrichment, so price rules see both registry pricing and whatever the provider reported.
providers:
openrouter:
model_filter:
include: ["*:free"] # keep only matching models
exclude: ["*-preview"] # then drop matching models
max_price_per_mtok: 0 # drop models priced above the cap
Patterns are globs matched case-insensitively against the raw provider model ID: `*` matches any run of characters (including `/`, unlike shell globs, so `*:free` matches `deepseek/deepseek-r1:free`) and `?` matches exactly one.
func (ModelFilter) Empty ¶ added in v0.1.82
func (f ModelFilter) Empty() bool
Empty reports whether the filter would keep every model.
func (ModelFilter) Normalize ¶ added in v0.1.82
func (f ModelFilter) Normalize() ModelFilter
Normalize trims patterns and drops empty ones.
func (ModelFilter) Validate ¶ added in v0.1.82
func (f ModelFilter) Validate(field string) error
Validate rejects a price cap that cannot express a real limit. NaN silently rejects every model, +Inf silently disables the cap, and a negative cap can never be met — each turns a cost control into something other than what was written, so they fail startup rather than mislead.
type ModelListConfig ¶
type ModelListConfig struct {
// URL is the HTTP(S) URL to fetch models.json from (empty = disabled)
URL string `yaml:"url" env:"MODEL_LIST_URL"`
}
ModelListConfig holds configuration for fetching the external model metadata registry.
type ModelsConfig ¶
type ModelsConfig struct {
// EnabledByDefault controls whether provider models are available
// when no persisted user-path access override exists.
// Default: true.
EnabledByDefault bool `yaml:"enabled_by_default" env:"MODELS_ENABLED_BY_DEFAULT"`
// KeepOnlyAliasesAtModelsEndpoint controls whether GET /v1/models hides
// provider models and returns only alias-projected model entries.
// Default: false.
KeepOnlyAliasesAtModelsEndpoint bool `yaml:"keep_only_aliases_at_models_endpoint" env:"KEEP_ONLY_ALIASES_AT_MODELS_ENDPOINT"`
// UnqualifiedModelIDsAtModelsEndpoint controls whether GET /v1/models
// returns bare model IDs (gpt-5) instead of provider-qualified ones
// (openai/gpt-5). When two providers expose the same model ID, only the
// entry for the provider an unqualified request routes to is listed.
// Default: false.
UnqualifiedModelIDsAtModelsEndpoint bool `yaml:"unqualified_model_ids_at_models_endpoint" env:"UNQUALIFIED_MODEL_IDS_AT_MODELS_ENDPOINT"`
// ConfiguredProviderModelsMode controls how providers.<name>.models and
// provider *_MODELS env vars affect the provider model inventory.
// Supported values: "fallback", "allowlist", "merge". Default: "fallback".
ConfiguredProviderModelsMode ConfiguredProviderModelsMode `yaml:"configured_provider_models_mode" env:"CONFIGURED_PROVIDER_MODELS_MODE"`
}
ModelsConfig holds global model access defaults.
type MongoDBStorageConfig ¶
type MongoDBStorageConfig struct {
// URL is the connection string; a database named in its path is honored
// (e.g., mongodb://localhost:27017/gomodel)
URL string `yaml:"url" env:"MONGODB_URL"`
// Database overrides the database named in the URL (default: gomodel)
Database string `yaml:"database" env:"MONGODB_DATABASE"`
}
MongoDBStorageConfig holds MongoDB-specific storage configuration
type OpenTelemetryConfig ¶ added in v0.1.84
type OpenTelemetryConfig struct {
// Enabled turns on OpenTelemetry export for inbound HTTP requests and
// outbound provider calls.
// Default: false
Enabled bool `yaml:"enabled" env:"OTEL_ENABLED"`
// ServiceName is the service.name resource attribute (OTEL_SERVICE_NAME).
// Default: "gomodel"
ServiceName string `yaml:"service_name"`
// ResourceAttributes adds resource attributes such as
// deployment.environment (OTEL_RESOURCE_ATTRIBUTES).
ResourceAttributes map[string]string `yaml:"resource_attributes"`
// Endpoint is the OTLP collector endpoint (OTEL_EXPORTER_OTLP_ENDPOINT).
// Default: "http://localhost:4318" for http/protobuf, "localhost:4317" for grpc
Endpoint string `yaml:"endpoint"`
// Protocol is "http/protobuf" or "grpc" (OTEL_EXPORTER_OTLP_PROTOCOL).
// Default: "http/protobuf"
Protocol string `yaml:"protocol"`
// Headers are sent with every export request, for example an
// authorization header for a hosted backend (OTEL_EXPORTER_OTLP_HEADERS).
Headers map[string]string `yaml:"headers"`
// TracesExporter is "otlp" or "none" (OTEL_TRACES_EXPORTER).
// Default: "otlp"
TracesExporter string `yaml:"traces_exporter"`
// MetricsExporter is "otlp" or "none" (OTEL_METRICS_EXPORTER).
// Default: "otlp"
MetricsExporter string `yaml:"metrics_exporter"`
// Sampler selects the trace sampler (OTEL_TRACES_SAMPLER), for example
// "parentbased_traceidratio" with SamplerArg "0.1".
// Default: "parentbased_always_on"
Sampler string `yaml:"sampler"`
// SamplerArg is the sampler argument (OTEL_TRACES_SAMPLER_ARG).
SamplerArg string `yaml:"sampler_arg"`
// Propagators is the comma-separated context propagator list
// (OTEL_PROPAGATORS): tracecontext, baggage, b3, b3multi, jaeger,
// ottrace, or none.
// Default: "tracecontext,baggage"
Propagators string `yaml:"propagators"`
}
OpenTelemetryConfig configures OTLP trace and metric export.
Only Enabled is a GoModel setting. The other fields mirror the standard OTEL_* environment variables that the OpenTelemetry SDK reads by itself, so a YAML-first deployment does not have to reach for the environment for the common settings. An OTEL_* variable that is already set in the environment wins over the YAML value, like every other GoModel setting; anything not listed here (per-signal endpoints, timeouts, compression, batch sizes, …) is available through its OTEL_* variable.
func (OpenTelemetryConfig) Environment ¶ added in v0.1.84
func (c OpenTelemetryConfig) Environment() map[string]string
Environment returns the OTEL_* variables that the configured fields stand for. Unset fields are omitted so the SDK applies its own defaults.
type PGVectorConfig ¶
type PGVectorConfig struct {
URL string `yaml:"url"`
Table string `yaml:"table"`
Dimension int `yaml:"dimension"`
}
PGVectorConfig holds connection configuration for the pgvector vector store.
type PineconeConfig ¶
type PineconeConfig struct {
Host string `yaml:"host"`
APIKey string `yaml:"api_key"`
Namespace string `yaml:"namespace"`
Dimension int `yaml:"dimension"`
}
PineconeConfig holds connection configuration for Pinecone (data-plane HTTP API).
type PluginFileConfig ¶ added in v0.1.90
type PluginFileConfig struct {
// File is the .so path: absolute, or relative to one of SearchPaths.
File string `yaml:"file"`
// SHA256 is an optional hex digest of the file. When set, a mismatch
// fails startup.
SHA256 string `yaml:"sha256"`
}
PluginFileConfig identifies one shared object to load.
type PluginsConfig ¶ added in v0.1.90
type PluginsConfig struct {
// Enabled turns the plugin system on: the built-in and loaded plugin
// types, the guardrails built from them, the routing-strategy plugins
// virtual models can select, and the /admin/plugins and /admin/guardrails
// endpoints. Guardrails need plugins, so GUARDRAILS_ENABLED=true enables
// it implicitly.
// Default: false
Enabled bool `yaml:"enabled" env:"PLUGINS_ENABLED"`
// SearchPaths lists directories searched for relative plugin files.
// A relative file must resolve inside one of these directories; the first
// match wins. Absolute file paths are used as-is.
// Default: empty (no .so loading)
SearchPaths []string `yaml:"search_paths" env:"PLUGINS_SEARCH_PATHS"`
// Load lists the shared objects to open at startup. Each file exports a
// GoModelPlugin symbol (see the pluginapi package). A file that cannot be
// resolved, verified, or opened is a startup error.
Load []PluginFileConfig `yaml:"load"`
}
PluginsConfig controls the plugin system and the loading of plugin shared objects (.so files).
A shared object is trusted code: loading one is equivalent to changing the binary. Keep search_paths root-owned and pin files with sha256 in production. Loading requires a cgo-enabled build of GoModel on Linux, macOS, or FreeBSD (the gomodel:<version>-plugins image); the default static binary reports a clear error for any configured plugin file.
type PostgreSQLStorageConfig ¶
type PostgreSQLStorageConfig struct {
// URL is the connection string (e.g., postgres://user:pass@localhost/dbname)
URL string `yaml:"url" env:"POSTGRES_URL"`
// MaxConns is the maximum connection pool size (default: 10)
MaxConns int `yaml:"max_conns" env:"POSTGRES_MAX_CONNS"`
}
PostgreSQLStorageConfig holds PostgreSQL-specific storage configuration
type QdrantConfig ¶
type QdrantConfig struct {
URL string `yaml:"url"`
Collection string `yaml:"collection"`
APIKey string `yaml:"api_key"`
}
QdrantConfig holds connection configuration for the Qdrant vector store.
type RateLimitModelConfig ¶
type RateLimitModelConfig struct {
Model string `yaml:"model"`
Limits []RateLimitRuleConfig `yaml:"limits"`
}
RateLimitModelConfig declares one or more rate limit rules for a model.
type RateLimitProviderConfig ¶
type RateLimitProviderConfig struct {
Name string `yaml:"name"`
Limits []RateLimitRuleConfig `yaml:"limits"`
}
RateLimitProviderConfig declares one or more rate limit rules for a provider.
type RateLimitRuleConfig ¶
type RateLimitRuleConfig struct {
// Period accepts minute, hour, day, or concurrent. The resolved period is
// persisted as PeriodSeconds in the database.
Period string `yaml:"period" json:"period"`
// PeriodSeconds can be set directly instead of Period for custom windows.
// 0 means the concurrent (in-flight) limit.
PeriodSeconds *int64 `yaml:"period_seconds" json:"period_seconds"`
// MaxRequests caps requests per period, or in-flight requests for the
// concurrent period.
MaxRequests *int64 `yaml:"max_requests" json:"max_requests"`
// MaxTokens caps total tokens per period. Not valid for the concurrent
// period. Requires usage tracking to be enforced.
MaxTokens *int64 `yaml:"max_tokens" json:"max_tokens"`
// PerChild is accepted only in the JSON-array environment-variable form.
// YAML keeps this setting on the enclosing user-path entry.
PerChild bool `yaml:"-" json:"per_child,omitempty"`
}
RateLimitRuleConfig declares the limits for one period. The json tags support the JSON-array form of SET_RATE_LIMIT_* env values.
type RateLimitUserPathConfig ¶
type RateLimitUserPathConfig struct {
Path string `yaml:"path"`
PerChild bool `yaml:"per_child"`
Limits []RateLimitRuleConfig `yaml:"limits"`
}
RateLimitUserPathConfig declares one or more rate limit rules for a user path.
type RateLimitsConfig ¶
type RateLimitsConfig struct {
// Enabled controls whether rate limit checks are active.
// Default: true. With no rules configured the check is a no-op.
Enabled bool `yaml:"enabled" env:"RATE_LIMITS_ENABLED"`
// FlushInterval is how often live request/token windows are written to
// storage, in seconds. Default 1. 0 disables the periodic loop; Start
// still loads and Close of an active generation still writes once.
FlushInterval int `yaml:"flush_interval" env:"RATE_LIMITS_FLUSH_INTERVAL"`
// UserPaths declares rate limit rules by tracked user path.
UserPaths []RateLimitUserPathConfig `yaml:"user_paths"`
// Providers declares rate limit rules by configured provider name.
// Provider rules cap all traffic routed to that provider instance; load
// balancing and failover skip a saturated provider while capacity exists
// elsewhere.
Providers []RateLimitProviderConfig `yaml:"providers"`
// Models declares rate limit rules by model. A provider-qualified subject
// ("openai/gpt-4o") caps one provider's model; a bare id ("gpt-4o") caps
// the model across every provider.
Models []RateLimitModelConfig `yaml:"models"`
}
RateLimitsConfig holds request, token, and concurrency limits scoped to user paths (consumers), providers, and models.
type RawCircuitBreakerConfig ¶
type RawCircuitBreakerConfig struct {
FailureOnStatuses []string `yaml:"failure_on_statuses"`
Scope *string `yaml:"scope"`
Enabled *bool `yaml:"enabled"`
FailureThreshold *int `yaml:"failure_threshold"`
SuccessThreshold *int `yaml:"success_threshold"`
Timeout *time.Duration `yaml:"timeout"`
}
RawCircuitBreakerConfig holds optional per-provider circuit breaker overrides from YAML. Nil fields inherit from the global CircuitBreakerConfig.
type RawProviderConfig ¶
type RawProviderConfig struct {
Type string `yaml:"type"`
APIKey string `yaml:"api_key"`
// APIKeys lists additional API keys for this provider. When more than one
// key is resolved (counting APIKey), identified sessions stay on one key by
// default while sessionless requests rotate round robin. Set it via
// `api_keys:` or the `<PROVIDER>_API_KEY_<n>` env vars.
APIKeys []string `yaml:"api_keys"`
// SessionStickyKeys defaults to true. Set false to restore round-robin key
// selection for every request, including requests carrying a session ID.
SessionStickyKeys *bool `yaml:"session_sticky_keys"`
BaseURL string `yaml:"base_url"`
APIVersion string `yaml:"api_version"`
Backend string `yaml:"backend"`
AuthType string `yaml:"auth_type"`
APIMode string `yaml:"api_mode"`
VertexProject string `yaml:"vertex_project"`
VertexLocation string `yaml:"vertex_location"`
ServiceAccountFile string `yaml:"service_account_file"`
ServiceAccountJSON string `yaml:"service_account_json"`
ServiceAccountJSONBase64 string `yaml:"service_account_json_base64"`
GCPScope string `yaml:"gcp_scope"`
// InferenceObjective is the trusted llm-d InferenceObjective name injected
// into outbound requests. It is ignored by provider types other than llmd.
InferenceObjective string `yaml:"inference_objective"`
// FairnessFromUserPath controls whether the llmd provider derives its
// fairness ID from GoModel's effective (authenticated) user path. It
// defaults to true; nil preserves that default.
FairnessFromUserPath *bool `yaml:"fairness_from_user_path"`
Models []RawProviderModel `yaml:"models"`
// ModelFilter narrows the provider's model inventory to the models that
// match its patterns and price cap. It applies to the final inventory, so
// it also narrows models added by `models` in merge or allowlist mode.
ModelFilter ModelFilter `yaml:"model_filter"`
Resilience *RawResilienceConfig `yaml:"resilience"`
}
RawProviderConfig is the YAML-sourced provider configuration before env var overrides, credential filtering, or resilience merging. Exported so the providers package can resolve it into a fully-configured ProviderConfig.
type RawProviderModel ¶
type RawProviderModel struct {
ID string `yaml:"id"`
Metadata *core.ModelMetadata `yaml:"metadata,omitempty"`
}
RawProviderModel is a single entry under providers.<name>.models. It supports two YAML shapes so operators can opt into rich metadata without churning simple configs:
models:
- some-model-id # bare string
- id: local-model # mapping with optional metadata
metadata:
context_window: 131072
capabilities:
tools: true
Metadata is merged onto whatever the remote model registry supplies, with config-declared fields taking precedence. This lets local providers (Ollama, custom OpenAI-compatible endpoints) advertise their context windows, pricing, and capabilities via /v1/models even when the remote registry has no entry.
func (*RawProviderModel) UnmarshalYAML ¶
func (m *RawProviderModel) UnmarshalYAML(node *yaml.Node) error
UnmarshalYAML accepts either a bare string (model ID) or a mapping with id and metadata.
type RawResilienceConfig ¶
type RawResilienceConfig struct {
Retry *RawRetryConfig `yaml:"retry"`
CircuitBreaker *RawCircuitBreakerConfig `yaml:"circuit_breaker"`
}
RawResilienceConfig holds optional per-provider resilience overrides from YAML. Nil fields inherit from the global ResilienceConfig.
type RawRetryConfig ¶
type RawRetryConfig struct {
RetryOnStatuses []string `yaml:"retry_on_statuses"`
MaxRetries *int `yaml:"max_retries"`
InitialBackoff *time.Duration `yaml:"initial_backoff"`
MaxBackoff *time.Duration `yaml:"max_backoff"`
BackoffFactor *float64 `yaml:"backoff_factor"`
JitterFactor *float64 `yaml:"jitter_factor"`
}
RawRetryConfig holds optional per-provider retry overrides from YAML. Nil fields inherit from the global RetryConfig.
type RedisModelConfig ¶
type RedisModelConfig struct {
URL string `yaml:"url" env:"REDIS_URL"`
Key string `yaml:"key" env:"REDIS_KEY_MODELS"`
TTL int `yaml:"ttl" env:"REDIS_TTL_MODELS"`
}
RedisModelConfig holds Redis connection configuration for the model registry cache.
type RedisResponseConfig ¶
type RedisResponseConfig struct {
URL string `yaml:"url"`
Key string `yaml:"key"`
TTL int `yaml:"ttl"`
}
RedisResponseConfig holds Redis connection configuration for the response cache. Uses separate env vars from RedisModelConfig for key and TTL to allow independent configuration. The URL is shared via REDIS_URL to simplify single-Redis deployments; use YAML config if different Redis instances are needed for model and response caches. Env vars are applied in Load via applyResponseSimpleEnv, only when cache.response.simple is present (see RESPONSE_CACHE_SIMPLE_ENABLED for env-only opt-in without YAML).
type ResilienceConfig ¶
type ResilienceConfig struct {
Retry RetryConfig `yaml:"retry"`
CircuitBreaker CircuitBreakerConfig `yaml:"circuit_breaker"`
}
ResilienceConfig holds resolved resilience settings (retry and circuit breaker).
type ResponseCacheConfig ¶
type ResponseCacheConfig struct {
Simple *SimpleCacheConfig `yaml:"simple"`
Semantic *SemanticCacheConfig `yaml:"semantic"`
}
ResponseCacheConfig holds configuration for response cache middleware.
type RetryConfig ¶
type RetryConfig struct {
// RetryOnStatuses uses defaults when nil; an empty list disables status triggers.
RetryOnStatuses []string `yaml:"retry_on_statuses" env:"RETRY_ON_STATUSES"`
MaxRetries int `yaml:"max_retries" env:"RETRY_MAX_RETRIES"`
InitialBackoff time.Duration `yaml:"initial_backoff" env:"RETRY_INITIAL_BACKOFF"`
MaxBackoff time.Duration `yaml:"max_backoff" env:"RETRY_MAX_BACKOFF"`
BackoffFactor float64 `yaml:"backoff_factor" env:"RETRY_BACKOFF_FACTOR"`
JitterFactor float64 `yaml:"jitter_factor" env:"RETRY_JITTER_FACTOR"`
}
RetryConfig holds resolved retry settings for an LLM client. This is the canonical type shared between config and llmclient.
func DefaultRetryConfig ¶
func DefaultRetryConfig() RetryConfig
DefaultRetryConfig returns the default retry settings.
type SQLiteStorageConfig ¶
type SQLiteStorageConfig struct {
// Path is the database file path. Default: ./data/gomodel.db when a
// ./data directory exists, otherwise the OS per-user data directory
// (e.g. ~/.local/share/gomodel/gomodel.db).
Path string `yaml:"path" env:"SQLITE_PATH"`
}
SQLiteStorageConfig holds SQLite-specific storage configuration
type SemanticCacheConfig ¶
type SemanticCacheConfig struct {
Enabled *bool `yaml:"enabled"`
SimilarityThreshold float64 `yaml:"similarity_threshold"`
TTL *int `yaml:"ttl"`
MaxConversationMessages *int `yaml:"max_conversation_messages"`
ExcludeSystemPrompt bool `yaml:"exclude_system_prompt"`
Embedder EmbedderConfig `yaml:"embedder"`
VectorStore VectorStoreConfig `yaml:"vector_store"`
}
SemanticCacheConfig holds configuration for the semantic (vector-similarity) response cache. When the semantic block is omitted from config.yaml, this layer stays off unless SEMANTIC_CACHE_ENABLED=true is set. Omitted enabled (nil) means true whenever the semantic block exists. Tuning env vars are applied in Load via applyResponseSemanticEnv when this block exists.
type ServerConfig ¶
type ServerConfig struct {
Port string `yaml:"port" env:"PORT"`
BasePath string `yaml:"base_path" env:"BASE_PATH"` // URL path prefix where the app is mounted (e.g., "/g")
MasterKey string `yaml:"master_key" env:"GOMODEL_MASTER_KEY"` // Optional: Master key for authentication
BodySizeLimit string `yaml:"body_size_limit" env:"BODY_SIZE_LIMIT"` // Max request body size (e.g., "10M", "1024K")
SwaggerEnabled bool `yaml:"swagger_enabled" env:"SWAGGER_ENABLED"` // Whether to expose the Swagger UI at /swagger/index.html
PprofEnabled bool `yaml:"pprof_enabled" env:"PPROF_ENABLED"` // Whether to expose debug profiling routes at /debug/pprof/*
// EnablePassthroughRoutes exposes provider-native passthrough endpoints under
// /p/{provider}/{endpoint}. Default: true.
EnablePassthroughRoutes bool `yaml:"enable_passthrough_routes" env:"ENABLE_PASSTHROUGH_ROUTES"`
// AllowPassthroughV1Alias allows /p/{provider}/v1/... style passthrough routes
// while keeping /p/{provider}/... as the canonical form. Default: true.
AllowPassthroughV1Alias bool `yaml:"allow_passthrough_v1_alias" env:"ALLOW_PASSTHROUGH_V1_ALIAS"`
// UserPathHeader is the inbound HTTP header used to read/write user paths.
// Default: X-GoModel-User-Path.
UserPathHeader string `yaml:"user_path_header" env:"USER_PATH_HEADER"`
// EnabledPassthroughProviders lists the provider types enabled on
// /p/{provider}/... passthrough routes. Default:
// ["openai", "anthropic", "openrouter", "kilo", "zai", "sglang", "vllm", "llmd", "deepseek"].
EnabledPassthroughProviders []string `yaml:"enabled_passthrough_providers" env:"ENABLED_PASSTHROUGH_PROVIDERS"`
// RealtimeEnabled exposes the realtime (speech-to-speech) websocket endpoints
// at /v1/realtime and /v1/realtime/translations, their WebRTC signaling
// siblings, and the /p/{provider}/v1/realtime passthrough upgrade.
// Default: true. Only providers implementing realtime accept sessions.
RealtimeEnabled bool `yaml:"realtime_enabled" env:"REALTIME_ENABLED"`
// AuthVerifyEnabled exposes GET /v1/auth/verify, which reports whether the
// API key a request carries authenticates against this gateway. It sits
// outside /admin so it keeps working when the admin API is disabled.
// Default: false — turn it on only when a service in front of the gateway
// needs to validate keys without holding a copy of them.
AuthVerifyEnabled bool `yaml:"auth_verify_enabled" env:"AUTH_VERIFY_ENABLED"`
// PIDFile records the process id of the running gateway so `gomodel --reload`
// can find it. Default: DefaultPIDFilePath(). Set it per instance when
// several gateways share a host, or to "" in config.yaml to write no pid
// file at all, which also disables `--reload` (an empty PID_FILE reads as
// unset, like every other env var here, and keeps the default). Changing it
// needs a restart — it names the process that is already running — so a
// reload only warns about it.
PIDFile string `yaml:"pid_file" env:"PID_FILE"`
// StreamStallTimeout bounds, in seconds, how long a single response write
// on a model interaction route may wait for the client to accept bytes
// before the connection is dropped. It fires only when the client has
// stopped reading (its socket buffer is full), never while the gateway is
// waiting on the provider, so slow models are unaffected. Without it a
// client that stops reading a stream pins a goroutine and the upstream
// provider connection until the provider side times out.
// Default: 60 (DefaultStreamStallTimeoutSeconds). 0 disables the limit.
StreamStallTimeout int `yaml:"stream_stall_timeout" env:"STREAM_STALL_TIMEOUT"`
}
ServerConfig holds HTTP server configuration
type SessionConfig ¶ added in v0.1.63
type SessionConfig struct {
// Enabled turns session identification on. Default: true.
Enabled bool `yaml:"enabled" env:"SESSION_KEEPING_ENABLED"`
// AutoDetect derives a session id from the conversation prefix of chat and
// responses requests when no explicit signal is present. Default: true.
AutoDetect bool `yaml:"auto_detect" env:"SESSION_AUTO_DETECT"`
// BuiltinRules enables the built-in registry of session headers and body
// fields known coding tools send. Default: true.
BuiltinRules bool `yaml:"builtin_rules" env:"SESSION_BUILTIN_RULES"`
// Headers declares additional session id headers, merged over the built-in
// registry (an entry with a built-in header name overrides it).
Headers []SessionHeaderConfig `yaml:"headers,omitempty"`
}
SessionConfig controls session keeping: identifying which requests belong to one client session for sticky load balancing and audit-log grouping.
type SessionHeaderConfig ¶ added in v0.1.63
type SessionHeaderConfig struct {
// Header is the HTTP header name to read the session id from.
Header string `yaml:"header" json:"header"`
// Transform optionally post-processes the header value. Supported:
// "session-uuid" (extract a session UUID from Anthropic metadata-style
// values). Default: use the value as-is.
Transform string `yaml:"transform,omitempty" json:"transform,omitempty"`
}
SessionHeaderConfig declares one header to read session ids from.
type SimpleCacheConfig ¶
type SimpleCacheConfig struct {
Enabled *bool `yaml:"enabled"`
Redis *RedisResponseConfig `yaml:"redis"`
}
SimpleCacheConfig holds configuration for exact-match response caching. When the simple block is omitted from config.yaml, this layer stays off unless RESPONSE_CACHE_SIMPLE_ENABLED=true is set (e.g. Helm without a response-cache YAML fragment). Omitted enabled (nil) means true whenever the simple block exists.
type StorageConfig ¶
type StorageConfig struct {
// Type specifies the storage backend: "sqlite" (default), "postgresql", or "mongodb"
Type string `yaml:"type" env:"STORAGE_TYPE"`
// SQLite configuration
SQLite SQLiteStorageConfig `yaml:"sqlite"`
// PostgreSQL configuration
PostgreSQL PostgreSQLStorageConfig `yaml:"postgresql"`
// MongoDB configuration
MongoDB MongoDBStorageConfig `yaml:"mongodb"`
}
StorageConfig holds database storage configuration (used by audit logging, usage tracking, future IAM, etc.)
func (StorageConfig) BackendConfig ¶
func (c StorageConfig) BackendConfig() storage.Config
BackendConfig converts the application storage config into the internal storage config.
type SystemPromptSettings ¶
type SystemPromptSettings struct {
// Mode controls how the system prompt is applied: "inject", "override", or "decorator"
// - inject: adds a system message only if none exists
// - override: replaces all existing system messages
// - decorator: prepends to the first existing system message
// Default: "inject"
Mode string `yaml:"mode"`
// Content is the system prompt text to apply
Content string `yaml:"content"`
}
SystemPromptSettings holds the type-specific settings for a system_prompt guardrail.
type TaggingConfig ¶
type TaggingConfig struct {
Headers []TaggingHeaderConfig `yaml:"headers"`
}
TaggingConfig declares request labelling based on HTTP headers. Headers listed here are read on every request; their values become request labels. Declarative entries override admin-store rows with the same header name and are read-only in the dashboard.
type TaggingHeaderConfig ¶
type TaggingHeaderConfig struct {
// Header is the HTTP header name to read labels from.
Header string `yaml:"header" json:"header"`
// Prefix is optionally trimmed from the front of each label. Trimming only
// affects the extracted label, never the forwarded header value.
Prefix string `yaml:"prefix,omitempty" json:"prefix,omitempty"`
// DoNotPass strips the header before forwarding the request upstream.
// Default: false (headers are passed through as-is).
DoNotPass bool `yaml:"do_not_pass,omitempty" json:"do_not_pass,omitempty"`
// Delimiter splits one header value into multiple labels. Default: ",".
Delimiter string `yaml:"delimiter,omitempty" json:"delimiter,omitempty"`
}
TaggingHeaderConfig declares one header to extract labels from.
type UsageConfig ¶
type UsageConfig struct {
// Enabled controls whether usage tracking is active
// Default: true
Enabled bool `yaml:"enabled" env:"USAGE_ENABLED"`
// EnforceReturningUsageData controls whether to ask streaming providers to return usage data when possible.
// When true, stream_options: {"include_usage": true} is added for provider paths that support it.
// Default: true
EnforceReturningUsageData bool `yaml:"enforce_returning_usage_data" env:"ENFORCE_RETURNING_USAGE_DATA"`
// PricingRecalculationEnabled controls whether the admin pricing recalculation action is available.
// Storage and pricing metadata support are still required; false always disables the feature.
// Default: true
PricingRecalculationEnabled bool `yaml:"pricing_recalculation_enabled" env:"USAGE_PRICING_RECALCULATION_ENABLED"`
// BufferSize is the number of usage entries to buffer before flushing
// Default: 1000
BufferSize int `yaml:"buffer_size" env:"USAGE_BUFFER_SIZE"`
// FlushInterval is how often to flush buffered usage entries (in seconds)
// Default: 5
FlushInterval int `yaml:"flush_interval" env:"USAGE_FLUSH_INTERVAL"`
// RetentionDays is how long to keep usage data (0 = forever)
// Default: 90
RetentionDays int `yaml:"retention_days" env:"USAGE_RETENTION_DAYS"`
}
UsageConfig holds token usage tracking configuration
type UserConfig ¶ added in v0.1.84
type UserConfig struct {
// Path is the user path (group or user) the policy applies to, e.g.
// "/acme/eng". Deeper paths inherit it.
Path string `yaml:"path" json:"path"`
// AllowedModels lists the model selectors requests under Path may use:
// exact "provider/model", provider-wide "provider/*", or model-wide
// "model". Empty means the node itself does not restrict models.
AllowedModels []string `yaml:"allowed_models,omitempty" json:"allowed_models,omitempty"`
// Description is an optional human-readable note.
Description string `yaml:"description,omitempty" json:"description,omitempty"`
}
UserConfig declares one user-path access policy in config.yaml or the USERS env var, so model access can be managed as infrastructure-as-code. A declared policy shadows the admin-store row of the same path and is read-only in the dashboard.
type VectorStoreConfig ¶
type VectorStoreConfig struct {
Type string `yaml:"type"`
Qdrant QdrantConfig `yaml:"qdrant"`
PGVector PGVectorConfig `yaml:"pgvector"`
Pinecone PineconeConfig `yaml:"pinecone"`
Weaviate WeaviateConfig `yaml:"weaviate"`
}
VectorStoreConfig selects the vector DB backend. Type must be set when semantic caching is enabled: qdrant, pgvector, pinecone, weaviate.
type VersionCheckConfig ¶ added in v0.1.83
type VersionCheckConfig struct {
// Enabled turns the daily update check on.
// Default: true
Enabled bool `yaml:"enabled" env:"GOMODEL_VERSION_CHECK_ENABLED"`
// URL is the base URL of the version manifest. The channel file
// ("core.txt" or "pro.txt") is appended to it.
// Default: https://gomodel.enterpilot.io/version
URL string `yaml:"url" env:"GOMODEL_VERSION_CHECK_URL"`
// IntervalHours is how often the background check runs. Each run is
// jittered so gateways started together do not query in lockstep.
// Default: 24
IntervalHours int `yaml:"interval_hours" env:"GOMODEL_VERSION_CHECK_INTERVAL_HOURS"`
// TimeoutSeconds bounds a single manifest request.
// Default: 5
TimeoutSeconds int `yaml:"timeout_seconds" env:"GOMODEL_VERSION_CHECK_TIMEOUT_SECONDS"`
// MaxDailyChecks caps how many manifest requests this gateway makes per
// day in total, so a hostile client cycling cookies cannot turn /version
// into an outbound request amplifier.
// Default: 500
MaxDailyChecks int `yaml:"max_daily_checks" env:"GOMODEL_VERSION_CHECK_MAX_DAILY"`
}
VersionCheckConfig controls the daily update check against the GoModel release manifest.
The check sends the running version, the distribution name, and an anonymous install identifier. It never sends API keys, provider credentials, model names, prompts, usage data, client addresses, or the hostname the gateway is served on. Set Enabled to false to stop all outbound traffic from this subsystem.
type VirtualModelConfig ¶
type VirtualModelConfig struct {
// Source is the addressable virtual model name (for a redirect/load balancer)
// or the access selector (for an access policy).
Source string `yaml:"source" json:"source"`
// Strategy selects load balancing across multiple targets: "round_robin"
// (default), "cost", "failover", "adaptive" (delegates to a registered
// routing extension and falls back to round_robin without one), or
// "plugin" (delegates to the routing-strategy plugin named by
// StrategyPlugin). Ignored for single-target aliases and access policies.
Strategy string `yaml:"strategy,omitempty" json:"strategy,omitempty"`
// StrategyPlugin names the routing-strategy plugin used when Strategy is
// "plugin"; required then, ignored otherwise. Target weights have no
// effect under a plugin strategy: the plugin picks the target.
StrategyPlugin string `yaml:"strategy_plugin,omitempty" json:"strategy_plugin,omitempty"`
// StrategyConfig is the plugin's per-virtual-model configuration. Its keys
// are the plugin's route-scoped fields (see GET /admin/plugins,
// route_fields) and are validated against them at startup.
StrategyConfig map[string]any `yaml:"strategy_config,omitempty" json:"strategy_config,omitempty"`
// SessionAffinity keeps requests of one detected client session on the
// target that served it before, while that target stays available. Defaults
// to true when omitted; set false to restore stateless balancing.
SessionAffinity *bool `yaml:"session_affinity,omitempty" json:"session_affinity,omitempty"`
// Failover retries a request that failed on the chosen target against the
// remaining targets, in declared order. Defaults to true when omitted; set
// false to serve the chosen target only. The failover strategy always
// fails over.
Failover *bool `yaml:"failover,omitempty" json:"failover,omitempty"`
// Target is shorthand for a single-target alias, e.g. "openai/gpt-4o". Use
// Targets instead to load balance across several models.
Target string `yaml:"target,omitempty" json:"target,omitempty"`
// Targets are the redirect destinations. One target is a plain alias; several
// are load balanced across by Strategy.
Targets []VirtualModelTargetConfig `yaml:"targets,omitempty" json:"targets,omitempty"`
// UserPaths scopes the entry to specific request user paths. Empty means all.
UserPaths []string `yaml:"user_paths,omitempty" json:"user_paths,omitempty"`
// Description is an optional human-readable note.
Description string `yaml:"description,omitempty" json:"description,omitempty"`
// Slowdown is an optional extra-time factor. For example, 0.5 adds 50% of
// measured inference time. Zero explicitly disables inherited slowdown;
// nil leaves the setting unspecified.
Slowdown *float64 `yaml:"slowdown,omitempty" json:"slowdown,omitempty"`
// Enabled toggles the entry. It defaults to true when omitted.
Enabled *bool `yaml:"enabled,omitempty" json:"enabled,omitempty"`
}
VirtualModelConfig declares one virtual model in config.yaml or the VIRTUAL_MODELS env var, so operators can manage redirects, load balancers, and access policies as infrastructure-as-code. Declarative virtual models override admin-store rows of the same source and are read-only in the dashboard.
type VirtualModelTargetConfig ¶
type VirtualModelTargetConfig struct {
Provider string `yaml:"provider,omitempty" json:"provider,omitempty"`
Model string `yaml:"model" json:"model"`
Weight float64 `yaml:"weight,omitempty" json:"weight,omitempty"`
}
VirtualModelTargetConfig is one load-balancing destination. Model may be a bare id (with Provider set) or a "provider/model" selector. Weight biases the round_robin strategy and defaults to 1.
type WeaviateConfig ¶
type WeaviateConfig struct {
URL string `yaml:"url"`
Class string `yaml:"class"`
APIKey string `yaml:"api_key"`
}
WeaviateConfig holds connection configuration for Weaviate.
type WorkflowsConfig ¶
type WorkflowsConfig struct {
// RefreshInterval controls how often the in-memory workflow snapshot
// is refreshed from storage. Default: 1m.
RefreshInterval time.Duration `yaml:"refresh_interval" env:"WORKFLOW_REFRESH_INTERVAL"`
}
WorkflowsConfig holds runtime refresh behavior for persisted workflows.
Source Files
¶
- admin.go
- budget.go
- cache.go
- config.go
- env.go
- failover.go
- failover_policy.go
- guardrails.go
- http.go
- logging.go
- mcp.go
- merge.go
- metrics.go
- model_filter.go
- models.go
- opentelemetry.go
- plugins.go
- provider_models.go
- providers.go
- ratelimit.go
- resilience.go
- resilience_policy.go
- server.go
- session.go
- storage.go
- tagging.go
- usage.go
- user_path_env.go
- users.go
- versioncheck.go
- virtualmodels.go
- workflows.go