models

package
v0.70.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 16, 2026 License: MIT Imports: 29 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// ClaudeCodePrompt is the required system prompt for OAuth authentication.
	ClaudeCodePrompt = "You are Claude Code, Anthropic's official CLI for Claude."
)

Variables

This section is empty.

Functions

func ApplyModelSettings added in v0.46.0

func ApplyModelSettings(config *ProviderConfig, modelInfo *ModelInfo)

ApplyModelSettings merges per-model generation parameter defaults from the registry into a ProviderConfig. Model-level params are only applied for fields where the user has not explicitly set a value (i.e., the corresponding viper key is not set via CLI flag or global config).

The lookup order is:

  1. modelSettings["provider/model"] from config (highest model-level priority)
  2. ModelInfo.Params from custom model definitions

Both are overridden by explicit CLI flags / global config values.

func IsValidThinkingLevelForModel added in v0.57.0

func IsValidThinkingLevelForModel(level ThinkingLevel, modelName string) bool

IsValidThinkingLevelForModel checks if a thinking level is valid for the given model. Some OpenAI models like gpt-5.4 don't support "minimal" and require "none" instead.

func LoadCachedProviders

func LoadCachedProviders() (map[string]modelsDBProvider, string)

LoadCachedProviders reads the cached provider data from disk. Returns nil, "" if no cache exists or the cache is unreadable.

func LoadModelSettingsFromConfig added in v0.46.0

func LoadModelSettingsFromConfig() map[string]*GenerationParams

LoadModelSettingsFromConfig loads per-model generation parameter overrides from the config file. Keys are "provider/model" strings. Returns nil if no model settings are configured.

func LoadSystemPromptValue added in v0.46.0

func LoadSystemPromptValue(input string) string

LoadSystemPromptValue resolves a system prompt value that may be either inline text or a file path. If the value is a path to an existing file, its contents are read and returned. Otherwise the string is returned as-is. This mirrors config.LoadSystemPrompt but lives in the models package to avoid circular dependencies.

func ParseModelString

func ParseModelString(modelString string) (provider, model string, err error)

ParseModelString parses a model string in "provider/model" format (e.g. "anthropic/claude-sonnet-4-5"). It splits on the first "/" to extract the provider and model name. For backward compatibility, the legacy "provider:model" format is also accepted with a deprecation warning printed to stderr. Returns the provider name, model name, and any error.

func ReloadGlobalRegistry

func ReloadGlobalRegistry()

ReloadGlobalRegistry rebuilds the global registry from the current data sources (cache → embedded). Call after updating the cache.

func RemoveCachedProviders

func RemoveCachedProviders() error

RemoveCachedProviders deletes the cache file, causing the registry to fall back to the embedded model database on next load.

func StoreCachedProviders

func StoreCachedProviders(providers map[string]modelsDBProvider, etag string) error

StoreCachedProviders writes provider data to the cache file on disk.

func ThinkingLevelDescription added in v0.7.0

func ThinkingLevelDescription(level ThinkingLevel) string

ThinkingLevelDescription returns a human-readable description of a thinking level.

Types

type Cost

type Cost struct {
	Input      float64
	Output     float64
	CacheRead  *float64
	CacheWrite *float64
}

Cost represents the pricing information for a model.

type CostConfig added in v0.22.0

type CostConfig struct {
	Input  float64 `json:"input" yaml:"input"`
	Output float64 `json:"output" yaml:"output"`
}

CostConfig defines the pricing for a custom model.

type CustomModelConfig added in v0.22.0

type CustomModelConfig struct {
	Name        string                 `json:"name" yaml:"name"`
	BaseURL     string                 `json:"baseUrl,omitempty" yaml:"baseUrl,omitempty"`
	APIKey      string                 `json:"apiKey,omitempty" yaml:"apiKey,omitempty"`
	Family      string                 `json:"family,omitempty" yaml:"family,omitempty"`
	Attachment  bool                   `json:"attachment,omitempty" yaml:"attachment,omitempty"`
	Reasoning   bool                   `json:"reasoning,omitempty" yaml:"reasoning,omitempty"`
	Temperature bool                   `json:"temperature,omitempty" yaml:"temperature,omitempty"`
	Knowledge   string                 `json:"knowledge,omitempty" yaml:"knowledge,omitempty"`
	Cost        CostConfig             `json:"cost" yaml:"cost"`
	Limit       LimitConfig            `json:"limit" yaml:"limit"`
	Params      GenerationParamsConfig `json:"params,omitzero" yaml:"params,omitempty"`
}

CustomModelConfig defines a custom model configuration loaded from the config file. This is a duplicate here to avoid circular dependencies with internal/config.

type GenerationParams added in v0.46.0

type GenerationParams struct {
	MaxTokens        *int
	Temperature      *float32
	TopP             *float32
	TopK             *int32
	FrequencyPenalty *float32
	PresencePenalty  *float32
	StopSequences    []string
	ThinkingLevel    ThinkingLevel
	SystemPrompt     string // Per-model system prompt (inline text or file path)
}

GenerationParams holds per-model generation parameter defaults. These are stored on ModelInfo and applied during provider creation. Nil pointer fields mean "no model-level default" — the global config or CLI flag value (if any) will be used instead.

type GenerationParamsConfig added in v0.46.0

type GenerationParamsConfig struct {
	MaxTokens        *int     `json:"maxTokens,omitempty" yaml:"maxTokens,omitempty"`
	Temperature      *float32 `json:"temperature,omitempty" yaml:"temperature,omitempty"`
	TopP             *float32 `json:"topP,omitempty" yaml:"topP,omitempty"`
	TopK             *int32   `json:"topK,omitempty" yaml:"topK,omitempty"`
	FrequencyPenalty *float32 `json:"frequencyPenalty,omitempty" yaml:"frequencyPenalty,omitempty"`
	PresencePenalty  *float32 `json:"presencePenalty,omitempty" yaml:"presencePenalty,omitempty"`
	StopSequences    []string `json:"stopSequences,omitempty" yaml:"stopSequences,omitempty"`
	ThinkingLevel    string   `json:"thinkingLevel,omitempty" yaml:"thinkingLevel,omitempty"`
	SystemPrompt     string   `json:"systemPrompt,omitempty" yaml:"systemPrompt,omitempty"`
}

GenerationParamsConfig is the JSON/YAML-serializable form of generation parameter defaults. Used in both customModels[].params and modelSettings[].

type Limit

type Limit struct {
	Context int
	Output  int
}

Limit represents the context and output limits for a model.

type LimitConfig added in v0.22.0

type LimitConfig struct {
	Context int `json:"context" yaml:"context"`
	Output  int `json:"output" yaml:"output"`
}

LimitConfig defines context and output limits for a custom model.

type ModelInfo

type ModelInfo struct {
	ID          string
	Name        string
	Family      string // Model family (e.g., "claude", "gpt", "gemini")
	Attachment  bool
	Reasoning   bool
	Temperature bool
	Cost        Cost
	Limit       Limit
	ProviderNPM string // Model-specific provider npm override (e.g. "@ai-sdk/anthropic")
	BaseURL     string // Per-model base URL override (custom models only)
	APIKey      string // Per-model API key override (custom models only)

	// Params holds per-model generation parameter defaults. These are applied
	// when the user hasn't explicitly set the corresponding CLI flag or global
	// config value. Nil pointer fields mean "no model-level default".
	Params *GenerationParams
}

ModelInfo represents information about a specific model.

func LookupModelForSettings added in v0.46.0

func LookupModelForSettings(modelString string) *ModelInfo

LookupModelForSettings is a convenience function that parses a "provider/model" string and looks up the ModelInfo in the global registry. Returns nil when the model string is invalid or the model is unknown. Used by Kit.SetModel to pre-apply per-model settings before CreateProvider.

func (*ModelInfo) CacheType added in v0.30.0

func (m *ModelInfo) CacheType() string

CacheType returns the appropriate cache mechanism for this model family. Returns empty string if caching is not supported.

func (*ModelInfo) SupportsCaching added in v0.30.0

func (m *ModelInfo) SupportsCaching() bool

SupportsCaching returns true if this model family supports prompt caching. This enables automatic cost savings for supported models regardless of provider.

type ModelsDBProviders

type ModelsDBProviders = map[string]modelsDBProvider

ModelsDBProviders is the top-level type for models.dev/api.json data: a map of provider ID → provider object.

type ModelsRegistry

type ModelsRegistry struct {
	// contains filtered or unexported fields
}

ModelsRegistry provides validation and information about models. It maintains a registry of all supported LLM providers and their models, including capabilities, pricing, and configuration requirements. The registry data comes from models.dev.

func GetGlobalRegistry

func GetGlobalRegistry() *ModelsRegistry

GetGlobalRegistry returns the global models registry instance.

func NewModelsRegistry

func NewModelsRegistry() *ModelsRegistry

NewModelsRegistry creates a new models registry populated from models.dev data.

func (*ModelsRegistry) GetLLMProviders added in v0.30.0

func (r *ModelsRegistry) GetLLMProviders() []string

GetLLMProviders returns provider IDs that have LLM support, either through a native provider or via openaicompat auto-routing.

func (*ModelsRegistry) GetModelsForProvider

func (r *ModelsRegistry) GetModelsForProvider(provider string) (map[string]ModelInfo, error)

GetModelsForProvider returns all models for a specific provider.

func (*ModelsRegistry) GetProviderInfo

func (r *ModelsRegistry) GetProviderInfo(provider string) *ProviderInfo

GetProviderInfo returns the full provider info, or nil if not found.

func (*ModelsRegistry) GetSupportedProviders

func (r *ModelsRegistry) GetSupportedProviders() []string

GetSupportedProviders returns a list of all provider IDs in the registry.

func (*ModelsRegistry) LookupModel

func (r *ModelsRegistry) LookupModel(provider, modelID string) *ModelInfo

LookupModel returns model metadata from the database if available. Returns nil when the model or provider is not in the database — this is expected for new models, custom fine-tunes, or providers the database doesn't track yet. Callers should treat a nil return as "unknown model" and continue with sensible defaults.

func (*ModelsRegistry) SuggestModels

func (r *ModelsRegistry) SuggestModels(provider, invalidModel string) []string

SuggestModels returns similar model names when an invalid model is provided.

func (*ModelsRegistry) ValidateEnvironment

func (r *ModelsRegistry) ValidateEnvironment(provider string, apiKey string) error

ValidateEnvironment checks if required credentials are available for a provider. It checks the explicit API key, stored credentials (for providers that support them, such as Anthropic OAuth), and environment variables. Returns nil for providers not in the registry (unknown providers are assumed to handle auth themselves or via --provider-api-key).

func (*ModelsRegistry) ValidateModelString added in v0.35.0

func (r *ModelsRegistry) ValidateModelString(modelString string) error

ValidateModelString checks whether a model string is well-formed and refers to a known provider. It returns a user-friendly error with suggestions when the model or provider is unrecognised. Passing validation does not guarantee that API authentication will succeed — it only catches obvious mistakes (typos, missing provider prefix, non-existent provider names) early so that callers such as subagent spawning can return fast feedback.

Unknown models under a known provider are allowed (the provider API is the authority), but a completely unknown provider is rejected.

type OllamaLoadingResult

type OllamaLoadingResult struct {
	Message string
}

OllamaLoadingResult contains the result of model loading with actual settings used.

type ProviderConfig

type ProviderConfig struct {
	ModelString      string
	SystemPrompt     string
	ProviderAPIKey   string
	ProviderURL      string
	MaxTokens        int
	Temperature      *float32
	TopP             *float32
	TopK             *int32
	FrequencyPenalty *float32
	PresencePenalty  *float32
	StopSequences    []string
	NumGPU           *int32
	MainGPU          *int32
	TLSSkipVerify    bool
	ThinkingLevel    ThinkingLevel
	DisableCaching   bool // Opt-out: set to true to disable automatic prompt caching

	// ProgressReaderFunc, when set, wraps an io.Reader with progress display
	// for long operations like Ollama model pulls. The returned io.ReadCloser
	// must be closed when done. When nil, the raw reader is consumed directly
	// with no progress UI.
	ProgressReaderFunc func(io.Reader) io.ReadCloser
}

ProviderConfig holds configuration for creating LLM providers.

type ProviderInfo

type ProviderInfo struct {
	ID     string
	Env    []string
	NPM    string // npm package identifier from models.dev (e.g. "@ai-sdk/openai-compatible")
	API    string // base API URL for openai-compatible providers
	Name   string
	Models map[string]ModelInfo
}

ProviderInfo represents information about a model provider.

type ProviderResult

type ProviderResult struct {
	// Model is the created fantasy LanguageModel
	Model fantasy.LanguageModel
	// Message contains optional feedback for the user
	Message string
	// Closer is an optional cleanup function for providers that hold
	// resources (e.g. kronk's loaded models). May be nil.
	Closer io.Closer
	// ProviderOptions contains provider-specific options to be passed to the
	// fantasy agent (e.g. OpenAI Responses API reasoning options).
	ProviderOptions fantasy.ProviderOptions
	// SkipMaxOutputTokens indicates that this provider doesn't support the
	// max_output_tokens parameter (e.g., OpenAI Codex OAuth API).
	SkipMaxOutputTokens bool
}

ProviderResult contains the result of provider creation.

func CreateProvider

func CreateProvider(ctx context.Context, config *ProviderConfig) (*ProviderResult, error)

CreateProvider creates a fantasy LanguageModel based on the provider configuration. Model metadata is looked up from the models.dev database for cost tracking and capability detection, but unknown models are passed through to the provider API — the database is advisory, not a gatekeeper.

Native providers: anthropic, openai, google, ollama, azure, google-vertex-anthropic, openrouter, bedrock, vercel. Any provider in models.dev with an api URL or openai-compatible npm package is auto-routed through fantasy's openaicompat provider.

type ThinkingLevel added in v0.7.0

type ThinkingLevel string

ThinkingLevel controls extended thinking / reasoning budget for supported models.

const (
	ThinkingOff     ThinkingLevel = "off"
	ThinkingNone    ThinkingLevel = "none"
	ThinkingMinimal ThinkingLevel = "minimal"
	ThinkingLow     ThinkingLevel = "low"
	ThinkingMedium  ThinkingLevel = "medium"
	ThinkingHigh    ThinkingLevel = "high"
)

func ParseThinkingLevel added in v0.7.0

func ParseThinkingLevel(s string) ThinkingLevel

ParseThinkingLevel converts a string to a ThinkingLevel, defaulting to ThinkingOff.

func SuggestThinkingLevelFallback added in v0.57.0

func SuggestThinkingLevelFallback(level ThinkingLevel, modelName string) ThinkingLevel

SuggestThinkingLevelFallback returns a recommended fallback level when the requested level is not valid for the model. Returns ThinkingOff if no suitable fallback exists.

func ThinkingLevels added in v0.7.0

func ThinkingLevels() []ThinkingLevel

ThinkingLevels returns the ordered list of available thinking levels for cycling.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL