models

package
v0.102.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 3, 2026 License: MIT Imports: 32 Imported by: 0

Documentation

Index

Constants

View Source
const (
	WireNameOpenAI       = "openai"
	WireNameOpenAICompat = "openai-compat"
	WireNameAnthropic    = "anthropic"
	WireNameGoogle       = "google"
)

Wire protocol names accepted in provider overrides (config `providers` section) and the --provider-wire flag. These are the user-facing, dependency-agnostic names for the wire protocols.

View Source
const (
	// StatusDeprecated marks a model the provider has scheduled for removal.
	// It usually still answers requests, so this is advisory, not a block.
	StatusDeprecated = "deprecated"
	// StatusBeta marks a model the provider considers pre-release.
	StatusBeta = "beta"
)

Model lifecycle markers published by the catalog.

View Source
const (
	// ClaudeCodePrompt is the required system prompt for OAuth authentication.
	ClaudeCodePrompt = "You are Claude Code, Anthropic's official CLI for Claude."
)

Variables

This section is empty.

Functions

func ApplyModelSettings added in v0.46.0

func ApplyModelSettings(config *ProviderConfig, modelInfo *ModelInfo)

ApplyModelSettings merges per-model generation parameter defaults from the registry into a ProviderConfig. Model-level params are only applied for fields where the user has not explicitly set a value (i.e., the corresponding viper key is not set via CLI flag or global config).

The lookup order is:

  1. modelSettings["provider/model"] from config (highest model-level priority)
  2. ModelInfo.Params from custom model definitions

Both are overridden by explicit CLI flags / global config values.

func IsValidThinkingLevelForModel added in v0.57.0

func IsValidThinkingLevelForModel(level ThinkingLevel, provider, modelName string) bool

IsValidThinkingLevelForModel reports whether a thinking level is valid for the given provider/model.

The answer comes from the model's published reasoning metadata: OpenAI's gpt-5.x line accepts "none" but not "minimal", the o-series accepts neither, and Anthropic's budget-based models accept any level. Models the catalog says nothing about are permitted every level, so a model missing from the catalog is never blocked on the strength of absent data.

Pass an empty provider when it isn't known; the model is then looked up across every provider and accepted if any of them grades it as supported.

func LoadCachedProviders

func LoadCachedProviders() (map[string]modelsDBProvider, string)

LoadCachedProviders reads the cached provider data from disk.

Returns nil, "" if no cache exists, the cache is unreadable, or it was written under an older schema. A stale-schema cache is discarded rather than merged: it would shadow the embedded catalog with entries missing whatever fields the old binary did not model. Returning no ETag as well forces the next `kit update-models` to make an unconditional fetch, so the cache is rewritten in full instead of being confirmed as "not modified".

func LoadModelSettingsFrom added in v0.71.0

func LoadModelSettingsFrom(v *viper.Viper) map[string]*GenerationParams

LoadModelSettingsFrom loads per-model generation parameter overrides from the supplied per-instance store. When v is nil the process-global store is used. Keys are "provider/model" strings. Returns nil if no model settings are configured.

func LoadSystemPromptValue added in v0.46.0

func LoadSystemPromptValue(input string) string

LoadSystemPromptValue resolves a system prompt value that may be either inline text or a file path. If the value is a path to an existing file, its contents are read and returned. Otherwise the string is returned as-is. This mirrors config.LoadSystemPrompt but lives in the models package to avoid circular dependencies.

func ParseModelString

func ParseModelString(modelString string) (provider, model string, err error)

ParseModelString parses a model string in "provider/model" format (e.g. "anthropic/claude-sonnet-4-5"). It splits on the first "/" to extract the provider and model name. For backward compatibility, the legacy "provider:model" format is also accepted with a deprecation warning printed to stderr. Returns the provider name, model name, and any error.

func ReloadGlobalRegistry

func ReloadGlobalRegistry()

ReloadGlobalRegistry rebuilds the global registry from the current data sources (cache → embedded). Call after updating the cache.

func RemoveCachedProviders

func RemoveCachedProviders() error

RemoveCachedProviders deletes the cache file, causing the registry to fall back to the embedded model database on next load.

func ResolveProviderBaseURL added in v0.73.0

func ResolveProviderBaseURL(providerID string) (string, error)

ResolveProviderBaseURL returns the base API URL kit will use when talking to the given provider, applying the same resolution order as CreateProvider:

  1. The provider's `api` field from the models.dev registry.
  2. The hard-coded default base URL of its npm SDK package (e.g. @ai-sdk/groq → https://api.groq.com/openai/v1).
  3. Template substitution against the current process environment when the URL contains "${VAR}" placeholders (e.g. cloudflare-workers-ai needs CLOUDFLARE_ACCOUNT_ID).

It returns an error when the provider is unknown, when no URL can be derived, or when a templated URL has unset placeholders. The error message is suitable for direct display to end users.

Note: providers handled by bespoke auth schemes (amazon-bedrock SigV4, azure resource URLs, google-vertex project/location, sap-ai-core customer deployments) may return either an empty URL or a regional/templated URL — the actual endpoint is finalised inside their native handlers and depends on runtime credentials.

func StoreCachedProviders

func StoreCachedProviders(providers map[string]modelsDBProvider, etag string) error

StoreCachedProviders writes provider data to the cache file on disk.

func ThinkingLevelDescription added in v0.7.0

func ThinkingLevelDescription(level ThinkingLevel) string

ThinkingLevelDescription returns a human-readable description of a thinking level.

Types

type Cost

type Cost struct {
	Input      float64
	Output     float64
	CacheRead  *float64
	CacheWrite *float64

	// Tiers holds long-context pricing, ordered by ascending threshold. Many
	// long-context models bill roughly 2x the base rate once a request's
	// prompt passes a threshold (commonly 200k tokens), so charging the base
	// rate for every request under-reports the cost of large contexts. Empty
	// when the model prices every request at the base rate.
	Tiers []CostTier

	// Published is true when the source catalog supplied a pricing block for
	// the model. It distinguishes a model that is genuinely free (an explicit
	// zero rate, as with openrouter's ":free" variants) from one whose price
	// is simply unknown because the catalog omitted it — both otherwise look
	// identical, since the zero value of Input/Output is 0.
	//
	// Callers that display cost should check this before rendering a figure;
	// reporting "$0.00" for an unpriced model is worse than reporting nothing.
	Published bool
}

Cost represents the pricing information for a model. All rates are in US dollars per one million tokens.

func (Cost) RatesFor added in v0.102.0

func (c Cost) RatesFor(promptTokens int) Cost

RatesFor returns the rates that apply to a request whose prompt is promptTokens long. The prompt is the right input for tier selection: it is the whole context sent to the model, so callers must include cached tokens as well as fresh ones.

Returns the base rates when the model publishes no tiers or the prompt sits below every threshold. When several tiers match, the highest one wins.

type CostConfig added in v0.22.0

type CostConfig struct {
	Input  float64 `json:"input" yaml:"input"`
	Output float64 `json:"output" yaml:"output"`
}

CostConfig defines the pricing for a custom model.

type CostTier added in v0.102.0

type CostTier struct {
	Threshold  int
	Input      float64
	Output     float64
	CacheRead  *float64
	CacheWrite *float64
}

CostTier is a set of rates that applies once a request's prompt exceeds Threshold tokens.

type CustomModelConfig added in v0.22.0

type CustomModelConfig struct {
	Name         string `json:"name" yaml:"name"`
	BaseURL      string `json:"baseUrl,omitempty" yaml:"baseUrl,omitempty"`
	APIKey       string `json:"apiKey,omitempty" yaml:"apiKey,omitempty"`
	APIModelName string `json:"apiModelName,omitempty" yaml:"apiModelName,omitempty"`
	Family       string `json:"family,omitempty" yaml:"family,omitempty"`
	Attachment   bool   `json:"attachment,omitempty" yaml:"attachment,omitempty"`
	Reasoning    bool   `json:"reasoning,omitempty" yaml:"reasoning,omitempty"`
	Temperature  bool   `json:"temperature,omitempty" yaml:"temperature,omitempty"`
	Knowledge    string `json:"knowledge,omitempty" yaml:"knowledge,omitempty"`
	// Cost is a pointer so that an omitted "cost" block (unknown pricing) is
	// distinguishable from an explicitly configured zero rate (a genuinely
	// free model). A value type would collapse both to 0.
	Cost   *CostConfig            `json:"cost,omitempty" yaml:"cost,omitempty"`
	Limit  LimitConfig            `json:"limit" yaml:"limit"`
	Params GenerationParamsConfig `json:"params,omitzero" yaml:"params,omitempty"`
}

CustomModelConfig defines a custom model configuration loaded from the config file. This is a duplicate here to avoid circular dependencies with internal/config.

type GenerationParams added in v0.46.0

type GenerationParams struct {
	MaxTokens        *int
	Temperature      *float32
	TopP             *float32
	TopK             *int32
	FrequencyPenalty *float32
	PresencePenalty  *float32
	StopSequences    []string
	ThinkingLevel    ThinkingLevel
	SystemPrompt     string // Per-model system prompt (inline text or file path)
}

GenerationParams holds per-model generation parameter defaults. These are stored on ModelInfo and applied during provider creation. Nil pointer fields mean "no model-level default" — the global config or CLI flag value (if any) will be used instead.

type GenerationParamsConfig added in v0.46.0

type GenerationParamsConfig struct {
	MaxTokens        *int     `json:"maxTokens,omitempty" yaml:"maxTokens,omitempty"`
	Temperature      *float32 `json:"temperature,omitempty" yaml:"temperature,omitempty"`
	TopP             *float32 `json:"topP,omitempty" yaml:"topP,omitempty"`
	TopK             *int32   `json:"topK,omitempty" yaml:"topK,omitempty"`
	FrequencyPenalty *float32 `json:"frequencyPenalty,omitempty" yaml:"frequencyPenalty,omitempty"`
	PresencePenalty  *float32 `json:"presencePenalty,omitempty" yaml:"presencePenalty,omitempty"`
	StopSequences    []string `json:"stopSequences,omitempty" yaml:"stopSequences,omitempty"`
	ThinkingLevel    string   `json:"thinkingLevel,omitempty" yaml:"thinkingLevel,omitempty"`
	SystemPrompt     string   `json:"systemPrompt,omitempty" yaml:"systemPrompt,omitempty"`
}

GenerationParamsConfig is the JSON/YAML-serializable form of generation parameter defaults. Used in both customModels[].params and modelSettings[].

type Limit

type Limit struct {
	Context int
	Output  int
}

Limit represents the context and output limits for a model.

type LimitConfig added in v0.22.0

type LimitConfig struct {
	Context int `json:"context" yaml:"context"`
	Output  int `json:"output" yaml:"output"`
}

LimitConfig defines context and output limits for a custom model.

type ModelInfo

type ModelInfo struct {
	ID           string
	Name         string
	Family       string // Model family (e.g., "claude", "gpt", "gemini")
	Attachment   bool
	Reasoning    bool
	Temperature  bool
	Cost         Cost
	Limit        Limit
	ProviderNPM  string // Model-specific provider npm override (e.g. "@ai-sdk/anthropic")
	BaseURL      string // Per-model base URL override (custom models only)
	APIKey       string // Per-model API key override (custom models only)
	APIModelName string // Per-model API model name override (custom models only)

	// Params holds per-model generation parameter defaults. These are applied
	// when the user hasn't explicitly set the corresponding CLI flag or global
	// config value. Nil pointer fields mean "no model-level default".
	Params *GenerationParams

	// Status is the model's lifecycle marker from the catalog: StatusDeprecated,
	// StatusBeta, or empty for normal general availability.
	Status string

	// ReasoningLevels lists the reasoning effort levels the model accepts, as
	// published by the catalog (e.g. "none", "low", "medium", "high", "max").
	//
	// Empty carries two distinct meanings, separated by ReasoningIsGraded:
	// either the catalog published no reasoning metadata at all, or the model
	// takes a reasoning budget that has no named levels (an on/off toggle or a
	// raw token budget). Callers should consult SupportsReasoningLevel rather
	// than testing this slice directly.
	ReasoningLevels []string

	// ReasoningIsGraded is true when the catalog says the model selects
	// reasoning by named effort level, meaning ReasoningLevels is an
	// exhaustive list. False means either no metadata, or a toggle/budget
	// model that accepts any level Kit maps onto a token budget.
	ReasoningIsGraded bool
}

ModelInfo represents information about a specific model.

func LookupModelForSettings added in v0.46.0

func LookupModelForSettings(modelString string) *ModelInfo

LookupModelForSettings is a convenience function that parses a "provider/model" string and looks up the ModelInfo in the global registry. Returns nil when the model string is invalid or the model is unknown. Used by Kit.SetModel to pre-apply per-model settings before CreateProvider.

func (*ModelInfo) CacheType added in v0.30.0

func (m *ModelInfo) CacheType() string

CacheType returns the appropriate cache mechanism for this model family. Returns empty string if caching is not supported.

func (*ModelInfo) IsDeprecated added in v0.102.0

func (m *ModelInfo) IsDeprecated() bool

IsDeprecated reports whether the catalog marks this model as deprecated.

func (*ModelInfo) SupportsCaching added in v0.30.0

func (m *ModelInfo) SupportsCaching() bool

SupportsCaching returns true if this model family supports prompt caching. This enables automatic cost savings for supported models regardless of provider.

func (*ModelInfo) SupportsReasoningLevel added in v0.102.0

func (m *ModelInfo) SupportsReasoningLevel(level string) bool

SupportsReasoningLevel reports whether the model accepts the named reasoning effort level.

Returns true when the catalog published no usable reasoning metadata, so an unknown model is never blocked on the strength of missing data. Callers that need to distinguish "known good" from "unknown" should check ReasoningIsGraded.

type ModelsDBProviders

type ModelsDBProviders = map[string]modelsDBProvider

ModelsDBProviders is the top-level type for models.dev/api.json data: a map of provider ID → provider object.

type ModelsRegistry

type ModelsRegistry struct {
	// contains filtered or unexported fields
}

ModelsRegistry provides validation and information about models. It maintains a registry of all supported LLM providers and their models, including capabilities, pricing, and configuration requirements. The registry data comes from models.dev.

func GetGlobalRegistry

func GetGlobalRegistry() *ModelsRegistry

GetGlobalRegistry returns the global models registry instance.

func NewModelsRegistry

func NewModelsRegistry() *ModelsRegistry

NewModelsRegistry creates a new models registry populated from models.dev data.

func (*ModelsRegistry) GetLLMProviders added in v0.30.0

func (r *ModelsRegistry) GetLLMProviders() []string

GetLLMProviders returns provider IDs that have LLM support, either through a native provider or via openaicompat auto-routing.

func (*ModelsRegistry) GetModelsForProvider

func (r *ModelsRegistry) GetModelsForProvider(provider string) (map[string]ModelInfo, error)

GetModelsForProvider returns all models for a specific provider.

func (*ModelsRegistry) GetProviderInfo

func (r *ModelsRegistry) GetProviderInfo(provider string) *ProviderInfo

GetProviderInfo returns the full provider info, or nil if not found.

func (*ModelsRegistry) GetSupportedProviders

func (r *ModelsRegistry) GetSupportedProviders() []string

GetSupportedProviders returns a list of all provider IDs in the registry.

func (*ModelsRegistry) LookupModel

func (r *ModelsRegistry) LookupModel(provider, modelID string) *ModelInfo

LookupModel returns model metadata from the database if available. Returns nil when the model or provider is not in the database — this is expected for new models, custom fine-tunes, or providers the database doesn't track yet. Callers should treat a nil return as "unknown model" and continue with sensible defaults.

func (*ModelsRegistry) SuggestModels

func (r *ModelsRegistry) SuggestModels(provider, invalidModel string) []string

SuggestModels returns similar model names when an invalid model is provided.

func (*ModelsRegistry) ValidateEnvironment

func (r *ModelsRegistry) ValidateEnvironment(provider string, apiKey string) error

ValidateEnvironment checks if required credentials are available for a provider. It checks the explicit API key, stored credentials (for providers that support them, such as Anthropic OAuth), and environment variables. Returns nil for providers not in the registry (unknown providers are assumed to handle auth themselves or via --provider-api-key).

func (*ModelsRegistry) ValidateModelString added in v0.35.0

func (r *ModelsRegistry) ValidateModelString(modelString string) error

ValidateModelString checks whether a model string is well-formed and refers to a known provider. It returns a user-friendly error with suggestions when the model or provider is unrecognised. Passing validation does not guarantee that API authentication will succeed — it only catches obvious mistakes (typos, missing provider prefix, non-existent provider names) early so that callers such as subagent spawning can return fast feedback.

Unknown models under a known provider are allowed (the provider API is the authority), but a completely unknown provider is rejected.

type OllamaLoadingResult

type OllamaLoadingResult struct {
	Message string
}

OllamaLoadingResult contains the result of model loading with actual settings used.

type ProviderConfig

type ProviderConfig struct {
	ModelString    string
	SystemPrompt   string
	ProviderAPIKey string
	ProviderURL    string
	// ProviderWire is an explicit wire protocol override ("openai",
	// "openai-compat", "anthropic", "google") for auto-routed providers.
	// Takes precedence over the registry's Wire declaration and the
	// npm-package heuristic. Set from the --provider-wire flag.
	ProviderWire     string
	MaxTokens        int
	Temperature      *float32
	TopP             *float32
	TopK             *int32
	FrequencyPenalty *float32
	PresencePenalty  *float32
	StopSequences    []string
	NumGPU           *int32
	MainGPU          *int32
	TLSSkipVerify    bool
	ThinkingLevel    ThinkingLevel
	DisableCaching   bool // Opt-out: set to true to disable automatic prompt caching

	// ConfigStore is the per-instance configuration store used to resolve
	// "explicitly set" precedence checks (isExplicitlySet), per-model
	// settings, and right-sizing. When nil, the process-global viper store is
	// used. Threading a per-Kit store here keeps generation-parameter
	// precedence isolated between Kit instances in the same process.
	ConfigStore *viper.Viper

	// ProgressReaderFunc, when set, wraps an io.Reader with progress display
	// for long operations like Ollama model pulls. The returned io.ReadCloser
	// must be closed when done. When nil, the raw reader is consumed directly
	// with no progress UI.
	ProgressReaderFunc func(io.Reader) io.ReadCloser
}

ProviderConfig holds configuration for creating LLM providers.

type ProviderInfo

type ProviderInfo struct {
	ID   string
	Env  []string
	NPM  string // npm package identifier from models.dev (e.g. "@ai-sdk/openai-compatible")
	API  string // base API URL for openai-compatible providers
	Name string
	// Wire is an explicit wire protocol declaration ("openai",
	// "openai-compat", "anthropic", "google"). When set it takes precedence
	// over the npm-package heuristic in auto-routing. Empty means "infer".
	// Populated from the `providers` config section; not present in the
	// models.dev data itself.
	Wire string
	// Headers are default HTTP headers added to every request to this
	// provider (auto-routed wires only). Populated from the `providers`
	// config section.
	Headers map[string]string
	Models  map[string]ModelInfo
}

ProviderInfo represents information about a model provider.

type ProviderOverrideConfig added in v0.86.0

type ProviderOverrideConfig struct {
	// Name is the human-readable provider name.
	Name string `json:"name,omitempty" yaml:"name,omitempty"`
	// Wire is the wire protocol this provider speaks: "openai" (Responses
	// API), "openai-compat" (chat completions), "anthropic", or "google".
	Wire string `json:"wire,omitempty" yaml:"wire,omitempty"`
	// BaseURL is the provider's API base URL.
	BaseURL string `json:"baseUrl,omitempty" yaml:"baseUrl,omitempty"`
	// APIKeyEnv lists environment variable names to try (in order) when
	// resolving the API key.
	APIKeyEnv []string `json:"apiKeyEnv,omitempty" yaml:"apiKeyEnv,omitempty"`
	// Headers are default HTTP headers added to every request.
	Headers map[string]string `json:"headers,omitempty" yaml:"headers,omitempty"`
}

ProviderOverrideConfig declares or patches a provider in the model registry via the `providers` config section. All fields are optional; non-empty fields override the corresponding registry values (which come from the models.dev database). A provider ID that is not in the database is registered fresh, so users can declare entirely new providers (e.g. internal LLM gateways) without touching the database:

providers:
  # Patch the wire protocol of a known provider
  minimax:
    wire: anthropic

  # Declare a brand-new provider
  corp-llm:
    name: "Corp LLM Gateway"
    wire: anthropic
    baseUrl: https://llm.internal.corp/api
    apiKeyEnv: [CORP_LLM_KEY, LLM_GATEWAY_KEY]
    headers:
      X-Team: platform

type ProviderResult

type ProviderResult struct {
	// Model is the created fantasy LanguageModel
	Model fantasy.LanguageModel
	// Message contains optional feedback for the user
	Message string
	// Closer is an optional cleanup function for providers that hold
	// resources (e.g. kronk's loaded models). May be nil.
	Closer io.Closer
	// ProviderOptions contains provider-specific options to be passed to the
	// fantasy agent (e.g. OpenAI Responses API reasoning options).
	ProviderOptions fantasy.ProviderOptions
	// SkipMaxOutputTokens indicates that this provider doesn't support the
	// max_output_tokens parameter (e.g., OpenAI Codex OAuth API).
	SkipMaxOutputTokens bool
}

ProviderResult contains the result of provider creation.

func CreateProvider

func CreateProvider(ctx context.Context, config *ProviderConfig) (*ProviderResult, error)

CreateProvider creates a fantasy LanguageModel based on the provider configuration. Model metadata is looked up from the models.dev database for cost tracking and capability detection, but unknown models are passed through to the provider API — the database is advisory, not a gatekeeper.

Native providers: anthropic, openai, google, ollama, azure, google-vertex-anthropic, openrouter, bedrock, vercel. Any other provider in models.dev is auto-routed by wire protocol: its npm package (or per-model override) selects the OpenAI, Anthropic, or Google transport, using the provider's api URL as the base. Providers with an api URL but an unrecognized npm package fall back to the OpenAI-compatible wire.

type ThinkingLevel added in v0.7.0

type ThinkingLevel string

ThinkingLevel controls extended thinking / reasoning budget for supported models.

const (
	ThinkingOff     ThinkingLevel = "off"
	ThinkingNone    ThinkingLevel = "none"
	ThinkingMinimal ThinkingLevel = "minimal"
	ThinkingLow     ThinkingLevel = "low"
	ThinkingMedium  ThinkingLevel = "medium"
	ThinkingHigh    ThinkingLevel = "high"
)

func ParseThinkingLevel added in v0.7.0

func ParseThinkingLevel(s string) ThinkingLevel

ParseThinkingLevel converts a string to a ThinkingLevel, defaulting to ThinkingOff.

func SuggestThinkingLevelFallback added in v0.57.0

func SuggestThinkingLevelFallback(level ThinkingLevel, provider, modelName string) ThinkingLevel

SuggestThinkingLevelFallback returns a recommended fallback level when the requested level is not valid for the model.

It prefers the nearest supported level of similar cost: "minimal" and "none" substitute for each other, and otherwise the closest supported level on the effort scale is chosen. Returns ThinkingOff when the model supports no level at all.

func SupportedThinkingLevels added in v0.102.0

func SupportedThinkingLevels(provider, modelName string) []ThinkingLevel

SupportedThinkingLevels returns the thinking levels the given model accepts, in the order ThinkingLevels reports them.

The list is derived from the catalog's reasoning metadata. When the catalog has nothing usable for the model — an unknown model, or one that takes a reasoning toggle or raw token budget rather than named levels — every level is returned, since there is no evidence any of them is unsupported.

ThinkingOff is always included: it means "don't ask for reasoning", which every model honours.

func ThinkingLevels added in v0.7.0

func ThinkingLevels() []ThinkingLevel

ThinkingLevels returns the ordered list of available thinking levels for cycling.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL