Documentation
¶
Index ¶
- Constants
- func ApplyModelSettings(config *ProviderConfig, modelInfo *ModelInfo)
- func IsValidThinkingLevelForModel(level ThinkingLevel, provider, modelName string) bool
- func LoadCachedProviders() (map[string]modelsDBProvider, string)
- func LoadModelSettingsFrom(v *viper.Viper) map[string]*GenerationParams
- func LoadSystemPromptValue(input string) string
- func ParseModelString(modelString string) (provider, model string, err error)
- func ReloadGlobalRegistry()
- func RemoveCachedProviders() error
- func ResolveProviderBaseURL(providerID string) (string, error)
- func StoreCachedProviders(providers map[string]modelsDBProvider, etag string) error
- func ThinkingLevelDescription(level ThinkingLevel) string
- type Cost
- type CostConfig
- type CostTier
- type CustomModelConfig
- type GenerationParams
- type GenerationParamsConfig
- type Limit
- type LimitConfig
- type ModelInfo
- type ModelsDBProviders
- type ModelsRegistry
- func (r *ModelsRegistry) GetLLMProviders() []string
- func (r *ModelsRegistry) GetModelsForProvider(provider string) (map[string]ModelInfo, error)
- func (r *ModelsRegistry) GetProviderInfo(provider string) *ProviderInfo
- func (r *ModelsRegistry) GetSupportedProviders() []string
- func (r *ModelsRegistry) LookupModel(provider, modelID string) *ModelInfo
- func (r *ModelsRegistry) SuggestModels(provider, invalidModel string) []string
- func (r *ModelsRegistry) ValidateEnvironment(provider string, apiKey string) error
- func (r *ModelsRegistry) ValidateModelString(modelString string) error
- type OllamaLoadingResult
- type ProviderConfig
- type ProviderInfo
- type ProviderOverrideConfig
- type ProviderResult
- type ThinkingLevel
Constants ¶
const ( WireNameOpenAI = "openai" WireNameOpenAICompat = "openai-compat" WireNameAnthropic = "anthropic" WireNameGoogle = "google" )
Wire protocol names accepted in provider overrides (config `providers` section) and the --provider-wire flag. These are the user-facing, dependency-agnostic names for the wire protocols.
const ( // StatusDeprecated marks a model the provider has scheduled for removal. // It usually still answers requests, so this is advisory, not a block. StatusDeprecated = "deprecated" // StatusBeta marks a model the provider considers pre-release. StatusBeta = "beta" )
Model lifecycle markers published by the catalog.
const (
// ClaudeCodePrompt is the required system prompt for OAuth authentication.
ClaudeCodePrompt = "You are Claude Code, Anthropic's official CLI for Claude."
)
Variables ¶
This section is empty.
Functions ¶
func ApplyModelSettings ¶ added in v0.46.0
func ApplyModelSettings(config *ProviderConfig, modelInfo *ModelInfo)
ApplyModelSettings merges per-model generation parameter defaults from the registry into a ProviderConfig. Model-level params are only applied for fields where the user has not explicitly set a value (i.e., the corresponding viper key is not set via CLI flag or global config).
The lookup order is:
- modelSettings["provider/model"] from config (highest model-level priority)
- ModelInfo.Params from custom model definitions
Both are overridden by explicit CLI flags / global config values.
func IsValidThinkingLevelForModel ¶ added in v0.57.0
func IsValidThinkingLevelForModel(level ThinkingLevel, provider, modelName string) bool
IsValidThinkingLevelForModel reports whether a thinking level is valid for the given provider/model.
The answer comes from the model's published reasoning metadata: OpenAI's gpt-5.x line accepts "none" but not "minimal", the o-series accepts neither, and Anthropic's budget-based models accept any level. Models the catalog says nothing about are permitted every level, so a model missing from the catalog is never blocked on the strength of absent data.
Pass an empty provider when it isn't known; the model is then looked up across every provider and accepted if any of them grades it as supported.
func LoadCachedProviders ¶
LoadCachedProviders reads the cached provider data from disk.
Returns nil, "" if no cache exists, the cache is unreadable, or it was written under an older schema. A stale-schema cache is discarded rather than merged: it would shadow the embedded catalog with entries missing whatever fields the old binary did not model. Returning no ETag as well forces the next `kit update-models` to make an unconditional fetch, so the cache is rewritten in full instead of being confirmed as "not modified".
func LoadModelSettingsFrom ¶ added in v0.71.0
func LoadModelSettingsFrom(v *viper.Viper) map[string]*GenerationParams
LoadModelSettingsFrom loads per-model generation parameter overrides from the supplied per-instance store. When v is nil the process-global store is used. Keys are "provider/model" strings. Returns nil if no model settings are configured.
func LoadSystemPromptValue ¶ added in v0.46.0
LoadSystemPromptValue resolves a system prompt value that may be either inline text or a file path. If the value is a path to an existing file, its contents are read and returned. Otherwise the string is returned as-is. This mirrors config.LoadSystemPrompt but lives in the models package to avoid circular dependencies.
func ParseModelString ¶
ParseModelString parses a model string in "provider/model" format (e.g. "anthropic/claude-sonnet-4-5"). It splits on the first "/" to extract the provider and model name. For backward compatibility, the legacy "provider:model" format is also accepted with a deprecation warning printed to stderr. Returns the provider name, model name, and any error.
func ReloadGlobalRegistry ¶
func ReloadGlobalRegistry()
ReloadGlobalRegistry rebuilds the global registry from the current data sources (cache → embedded). Call after updating the cache.
func RemoveCachedProviders ¶
func RemoveCachedProviders() error
RemoveCachedProviders deletes the cache file, causing the registry to fall back to the embedded model database on next load.
func ResolveProviderBaseURL ¶ added in v0.73.0
ResolveProviderBaseURL returns the base API URL kit will use when talking to the given provider, applying the same resolution order as CreateProvider:
- The provider's `api` field from the models.dev registry.
- The hard-coded default base URL of its npm SDK package (e.g. @ai-sdk/groq → https://api.groq.com/openai/v1).
- Template substitution against the current process environment when the URL contains "${VAR}" placeholders (e.g. cloudflare-workers-ai needs CLOUDFLARE_ACCOUNT_ID).
It returns an error when the provider is unknown, when no URL can be derived, or when a templated URL has unset placeholders. The error message is suitable for direct display to end users.
Note: providers handled by bespoke auth schemes (amazon-bedrock SigV4, azure resource URLs, google-vertex project/location, sap-ai-core customer deployments) may return either an empty URL or a regional/templated URL — the actual endpoint is finalised inside their native handlers and depends on runtime credentials.
func StoreCachedProviders ¶
StoreCachedProviders writes provider data to the cache file on disk.
func ThinkingLevelDescription ¶ added in v0.7.0
func ThinkingLevelDescription(level ThinkingLevel) string
ThinkingLevelDescription returns a human-readable description of a thinking level.
Types ¶
type Cost ¶
type Cost struct {
Input float64
Output float64
CacheRead *float64
CacheWrite *float64
// Tiers holds long-context pricing, ordered by ascending threshold. Many
// long-context models bill roughly 2x the base rate once a request's
// prompt passes a threshold (commonly 200k tokens), so charging the base
// rate for every request under-reports the cost of large contexts. Empty
// when the model prices every request at the base rate.
Tiers []CostTier
// Published is true when the source catalog supplied a pricing block for
// the model. It distinguishes a model that is genuinely free (an explicit
// zero rate, as with openrouter's ":free" variants) from one whose price
// is simply unknown because the catalog omitted it — both otherwise look
// identical, since the zero value of Input/Output is 0.
//
// Callers that display cost should check this before rendering a figure;
// reporting "$0.00" for an unpriced model is worse than reporting nothing.
Published bool
}
Cost represents the pricing information for a model. All rates are in US dollars per one million tokens.
func (Cost) RatesFor ¶ added in v0.102.0
RatesFor returns the rates that apply to a request whose prompt is promptTokens long. The prompt is the right input for tier selection: it is the whole context sent to the model, so callers must include cached tokens as well as fresh ones.
Returns the base rates when the model publishes no tiers or the prompt sits below every threshold. When several tiers match, the highest one wins.
type CostConfig ¶ added in v0.22.0
type CostConfig struct {
Input float64 `json:"input" yaml:"input"`
Output float64 `json:"output" yaml:"output"`
}
CostConfig defines the pricing for a custom model.
type CostTier ¶ added in v0.102.0
type CostTier struct {
Threshold int
Input float64
Output float64
CacheRead *float64
CacheWrite *float64
}
CostTier is a set of rates that applies once a request's prompt exceeds Threshold tokens.
type CustomModelConfig ¶ added in v0.22.0
type CustomModelConfig struct {
Name string `json:"name" yaml:"name"`
BaseURL string `json:"baseUrl,omitempty" yaml:"baseUrl,omitempty"`
APIKey string `json:"apiKey,omitempty" yaml:"apiKey,omitempty"`
APIModelName string `json:"apiModelName,omitempty" yaml:"apiModelName,omitempty"`
Family string `json:"family,omitempty" yaml:"family,omitempty"`
Attachment bool `json:"attachment,omitempty" yaml:"attachment,omitempty"`
Reasoning bool `json:"reasoning,omitempty" yaml:"reasoning,omitempty"`
Temperature bool `json:"temperature,omitempty" yaml:"temperature,omitempty"`
Knowledge string `json:"knowledge,omitempty" yaml:"knowledge,omitempty"`
// Cost is a pointer so that an omitted "cost" block (unknown pricing) is
// distinguishable from an explicitly configured zero rate (a genuinely
// free model). A value type would collapse both to 0.
Cost *CostConfig `json:"cost,omitempty" yaml:"cost,omitempty"`
Limit LimitConfig `json:"limit" yaml:"limit"`
Params GenerationParamsConfig `json:"params,omitzero" yaml:"params,omitempty"`
}
CustomModelConfig defines a custom model configuration loaded from the config file. This is a duplicate here to avoid circular dependencies with internal/config.
type GenerationParams ¶ added in v0.46.0
type GenerationParams struct {
MaxTokens *int
Temperature *float32
TopP *float32
TopK *int32
FrequencyPenalty *float32
PresencePenalty *float32
StopSequences []string
ThinkingLevel ThinkingLevel
SystemPrompt string // Per-model system prompt (inline text or file path)
}
GenerationParams holds per-model generation parameter defaults. These are stored on ModelInfo and applied during provider creation. Nil pointer fields mean "no model-level default" — the global config or CLI flag value (if any) will be used instead.
type GenerationParamsConfig ¶ added in v0.46.0
type GenerationParamsConfig struct {
MaxTokens *int `json:"maxTokens,omitempty" yaml:"maxTokens,omitempty"`
Temperature *float32 `json:"temperature,omitempty" yaml:"temperature,omitempty"`
TopP *float32 `json:"topP,omitempty" yaml:"topP,omitempty"`
TopK *int32 `json:"topK,omitempty" yaml:"topK,omitempty"`
FrequencyPenalty *float32 `json:"frequencyPenalty,omitempty" yaml:"frequencyPenalty,omitempty"`
PresencePenalty *float32 `json:"presencePenalty,omitempty" yaml:"presencePenalty,omitempty"`
StopSequences []string `json:"stopSequences,omitempty" yaml:"stopSequences,omitempty"`
ThinkingLevel string `json:"thinkingLevel,omitempty" yaml:"thinkingLevel,omitempty"`
SystemPrompt string `json:"systemPrompt,omitempty" yaml:"systemPrompt,omitempty"`
}
GenerationParamsConfig is the JSON/YAML-serializable form of generation parameter defaults. Used in both customModels[].params and modelSettings[].
type LimitConfig ¶ added in v0.22.0
type LimitConfig struct {
Context int `json:"context" yaml:"context"`
Output int `json:"output" yaml:"output"`
}
LimitConfig defines context and output limits for a custom model.
type ModelInfo ¶
type ModelInfo struct {
ID string
Name string
Family string // Model family (e.g., "claude", "gpt", "gemini")
Attachment bool
Reasoning bool
Temperature bool
Cost Cost
Limit Limit
ProviderNPM string // Model-specific provider npm override (e.g. "@ai-sdk/anthropic")
BaseURL string // Per-model base URL override (custom models only)
APIKey string // Per-model API key override (custom models only)
APIModelName string // Per-model API model name override (custom models only)
// Params holds per-model generation parameter defaults. These are applied
// when the user hasn't explicitly set the corresponding CLI flag or global
// config value. Nil pointer fields mean "no model-level default".
Params *GenerationParams
// Status is the model's lifecycle marker from the catalog: StatusDeprecated,
// StatusBeta, or empty for normal general availability.
Status string
// ReasoningLevels lists the reasoning effort levels the model accepts, as
// published by the catalog (e.g. "none", "low", "medium", "high", "max").
//
// Empty carries two distinct meanings, separated by ReasoningIsGraded:
// either the catalog published no reasoning metadata at all, or the model
// takes a reasoning budget that has no named levels (an on/off toggle or a
// raw token budget). Callers should consult SupportsReasoningLevel rather
// than testing this slice directly.
ReasoningLevels []string
// ReasoningIsGraded is true when the catalog says the model selects
// reasoning by named effort level, meaning ReasoningLevels is an
// exhaustive list. False means either no metadata, or a toggle/budget
// model that accepts any level Kit maps onto a token budget.
ReasoningIsGraded bool
}
ModelInfo represents information about a specific model.
func LookupModelForSettings ¶ added in v0.46.0
LookupModelForSettings is a convenience function that parses a "provider/model" string and looks up the ModelInfo in the global registry. Returns nil when the model string is invalid or the model is unknown. Used by Kit.SetModel to pre-apply per-model settings before CreateProvider.
func (*ModelInfo) CacheType ¶ added in v0.30.0
CacheType returns the appropriate cache mechanism for this model family. Returns empty string if caching is not supported.
func (*ModelInfo) IsDeprecated ¶ added in v0.102.0
IsDeprecated reports whether the catalog marks this model as deprecated.
func (*ModelInfo) SupportsCaching ¶ added in v0.30.0
SupportsCaching returns true if this model family supports prompt caching. This enables automatic cost savings for supported models regardless of provider.
func (*ModelInfo) SupportsReasoningLevel ¶ added in v0.102.0
SupportsReasoningLevel reports whether the model accepts the named reasoning effort level.
Returns true when the catalog published no usable reasoning metadata, so an unknown model is never blocked on the strength of missing data. Callers that need to distinguish "known good" from "unknown" should check ReasoningIsGraded.
type ModelsDBProviders ¶
type ModelsDBProviders = map[string]modelsDBProvider
ModelsDBProviders is the top-level type for models.dev/api.json data: a map of provider ID → provider object.
type ModelsRegistry ¶
type ModelsRegistry struct {
// contains filtered or unexported fields
}
ModelsRegistry provides validation and information about models. It maintains a registry of all supported LLM providers and their models, including capabilities, pricing, and configuration requirements. The registry data comes from models.dev.
func GetGlobalRegistry ¶
func GetGlobalRegistry() *ModelsRegistry
GetGlobalRegistry returns the global models registry instance.
func NewModelsRegistry ¶
func NewModelsRegistry() *ModelsRegistry
NewModelsRegistry creates a new models registry populated from models.dev data.
func (*ModelsRegistry) GetLLMProviders ¶ added in v0.30.0
func (r *ModelsRegistry) GetLLMProviders() []string
GetLLMProviders returns provider IDs that have LLM support, either through a native provider or via openaicompat auto-routing.
func (*ModelsRegistry) GetModelsForProvider ¶
func (r *ModelsRegistry) GetModelsForProvider(provider string) (map[string]ModelInfo, error)
GetModelsForProvider returns all models for a specific provider.
func (*ModelsRegistry) GetProviderInfo ¶
func (r *ModelsRegistry) GetProviderInfo(provider string) *ProviderInfo
GetProviderInfo returns the full provider info, or nil if not found.
func (*ModelsRegistry) GetSupportedProviders ¶
func (r *ModelsRegistry) GetSupportedProviders() []string
GetSupportedProviders returns a list of all provider IDs in the registry.
func (*ModelsRegistry) LookupModel ¶
func (r *ModelsRegistry) LookupModel(provider, modelID string) *ModelInfo
LookupModel returns model metadata from the database if available. Returns nil when the model or provider is not in the database — this is expected for new models, custom fine-tunes, or providers the database doesn't track yet. Callers should treat a nil return as "unknown model" and continue with sensible defaults.
func (*ModelsRegistry) SuggestModels ¶
func (r *ModelsRegistry) SuggestModels(provider, invalidModel string) []string
SuggestModels returns similar model names when an invalid model is provided.
func (*ModelsRegistry) ValidateEnvironment ¶
func (r *ModelsRegistry) ValidateEnvironment(provider string, apiKey string) error
ValidateEnvironment checks if required credentials are available for a provider. It checks the explicit API key, stored credentials (for providers that support them, such as Anthropic OAuth), and environment variables. Returns nil for providers not in the registry (unknown providers are assumed to handle auth themselves or via --provider-api-key).
func (*ModelsRegistry) ValidateModelString ¶ added in v0.35.0
func (r *ModelsRegistry) ValidateModelString(modelString string) error
ValidateModelString checks whether a model string is well-formed and refers to a known provider. It returns a user-friendly error with suggestions when the model or provider is unrecognised. Passing validation does not guarantee that API authentication will succeed — it only catches obvious mistakes (typos, missing provider prefix, non-existent provider names) early so that callers such as subagent spawning can return fast feedback.
Unknown models under a known provider are allowed (the provider API is the authority), but a completely unknown provider is rejected.
type OllamaLoadingResult ¶
type OllamaLoadingResult struct {
Message string
}
OllamaLoadingResult contains the result of model loading with actual settings used.
type ProviderConfig ¶
type ProviderConfig struct {
ModelString string
SystemPrompt string
ProviderAPIKey string
ProviderURL string
// ProviderWire is an explicit wire protocol override ("openai",
// "openai-compat", "anthropic", "google") for auto-routed providers.
// Takes precedence over the registry's Wire declaration and the
// npm-package heuristic. Set from the --provider-wire flag.
ProviderWire string
MaxTokens int
Temperature *float32
TopP *float32
TopK *int32
FrequencyPenalty *float32
PresencePenalty *float32
StopSequences []string
NumGPU *int32
MainGPU *int32
TLSSkipVerify bool
ThinkingLevel ThinkingLevel
DisableCaching bool // Opt-out: set to true to disable automatic prompt caching
// ConfigStore is the per-instance configuration store used to resolve
// "explicitly set" precedence checks (isExplicitlySet), per-model
// settings, and right-sizing. When nil, the process-global viper store is
// used. Threading a per-Kit store here keeps generation-parameter
// precedence isolated between Kit instances in the same process.
ConfigStore *viper.Viper
// ProgressReaderFunc, when set, wraps an io.Reader with progress display
// for long operations like Ollama model pulls. The returned io.ReadCloser
// must be closed when done. When nil, the raw reader is consumed directly
// with no progress UI.
ProgressReaderFunc func(io.Reader) io.ReadCloser
}
ProviderConfig holds configuration for creating LLM providers.
type ProviderInfo ¶
type ProviderInfo struct {
ID string
Env []string
NPM string // npm package identifier from models.dev (e.g. "@ai-sdk/openai-compatible")
API string // base API URL for openai-compatible providers
Name string
// Wire is an explicit wire protocol declaration ("openai",
// "openai-compat", "anthropic", "google"). When set it takes precedence
// over the npm-package heuristic in auto-routing. Empty means "infer".
// Populated from the `providers` config section; not present in the
// models.dev data itself.
Wire string
// Headers are default HTTP headers added to every request to this
// provider (auto-routed wires only). Populated from the `providers`
// config section.
Headers map[string]string
Models map[string]ModelInfo
}
ProviderInfo represents information about a model provider.
type ProviderOverrideConfig ¶ added in v0.86.0
type ProviderOverrideConfig struct {
// Name is the human-readable provider name.
Name string `json:"name,omitempty" yaml:"name,omitempty"`
// Wire is the wire protocol this provider speaks: "openai" (Responses
// API), "openai-compat" (chat completions), "anthropic", or "google".
Wire string `json:"wire,omitempty" yaml:"wire,omitempty"`
// BaseURL is the provider's API base URL.
BaseURL string `json:"baseUrl,omitempty" yaml:"baseUrl,omitempty"`
// APIKeyEnv lists environment variable names to try (in order) when
// resolving the API key.
APIKeyEnv []string `json:"apiKeyEnv,omitempty" yaml:"apiKeyEnv,omitempty"`
// Headers are default HTTP headers added to every request.
Headers map[string]string `json:"headers,omitempty" yaml:"headers,omitempty"`
}
ProviderOverrideConfig declares or patches a provider in the model registry via the `providers` config section. All fields are optional; non-empty fields override the corresponding registry values (which come from the models.dev database). A provider ID that is not in the database is registered fresh, so users can declare entirely new providers (e.g. internal LLM gateways) without touching the database:
providers:
# Patch the wire protocol of a known provider
minimax:
wire: anthropic
# Declare a brand-new provider
corp-llm:
name: "Corp LLM Gateway"
wire: anthropic
baseUrl: https://llm.internal.corp/api
apiKeyEnv: [CORP_LLM_KEY, LLM_GATEWAY_KEY]
headers:
X-Team: platform
type ProviderResult ¶
type ProviderResult struct {
// Model is the created fantasy LanguageModel
Model fantasy.LanguageModel
// Message contains optional feedback for the user
Message string
// Closer is an optional cleanup function for providers that hold
// resources (e.g. kronk's loaded models). May be nil.
Closer io.Closer
// ProviderOptions contains provider-specific options to be passed to the
// fantasy agent (e.g. OpenAI Responses API reasoning options).
ProviderOptions fantasy.ProviderOptions
// SkipMaxOutputTokens indicates that this provider doesn't support the
// max_output_tokens parameter (e.g., OpenAI Codex OAuth API).
SkipMaxOutputTokens bool
}
ProviderResult contains the result of provider creation.
func CreateProvider ¶
func CreateProvider(ctx context.Context, config *ProviderConfig) (*ProviderResult, error)
CreateProvider creates a fantasy LanguageModel based on the provider configuration. Model metadata is looked up from the models.dev database for cost tracking and capability detection, but unknown models are passed through to the provider API — the database is advisory, not a gatekeeper.
Native providers: anthropic, openai, google, ollama, azure, google-vertex-anthropic, openrouter, bedrock, vercel. Any other provider in models.dev is auto-routed by wire protocol: its npm package (or per-model override) selects the OpenAI, Anthropic, or Google transport, using the provider's api URL as the base. Providers with an api URL but an unrecognized npm package fall back to the OpenAI-compatible wire.
type ThinkingLevel ¶ added in v0.7.0
type ThinkingLevel string
ThinkingLevel controls extended thinking / reasoning budget for supported models.
const ( ThinkingOff ThinkingLevel = "off" ThinkingNone ThinkingLevel = "none" ThinkingMinimal ThinkingLevel = "minimal" ThinkingLow ThinkingLevel = "low" ThinkingMedium ThinkingLevel = "medium" ThinkingHigh ThinkingLevel = "high" )
func ParseThinkingLevel ¶ added in v0.7.0
func ParseThinkingLevel(s string) ThinkingLevel
ParseThinkingLevel converts a string to a ThinkingLevel, defaulting to ThinkingOff.
func SuggestThinkingLevelFallback ¶ added in v0.57.0
func SuggestThinkingLevelFallback(level ThinkingLevel, provider, modelName string) ThinkingLevel
SuggestThinkingLevelFallback returns a recommended fallback level when the requested level is not valid for the model.
It prefers the nearest supported level of similar cost: "minimal" and "none" substitute for each other, and otherwise the closest supported level on the effort scale is chosen. Returns ThinkingOff when the model supports no level at all.
func SupportedThinkingLevels ¶ added in v0.102.0
func SupportedThinkingLevels(provider, modelName string) []ThinkingLevel
SupportedThinkingLevels returns the thinking levels the given model accepts, in the order ThinkingLevels reports them.
The list is derived from the catalog's reasoning metadata. When the catalog has nothing usable for the model — an unknown model, or one that takes a reasoning toggle or raw token budget rather than named levels — every level is returned, since there is no evidence any of them is unsupported.
ThinkingOff is always included: it means "don't ask for reasoning", which every model honours.
func ThinkingLevels ¶ added in v0.7.0
func ThinkingLevels() []ThinkingLevel
ThinkingLevels returns the ordered list of available thinking levels for cycling.