models

package
v0.45.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 8, 2026 License: MIT Imports: 31 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// ClaudeCodePrompt is the required system prompt for OAuth authentication.
	ClaudeCodePrompt = "You are Claude Code, Anthropic's official CLI for Claude."
)
View Source 
const DefaultPoolTTL = 5 * time.Minute

DefaultPoolTTL is the default time-to-live for idle pooled providers.

Variables

This section is empty.

Functions

func LoadCachedProviders 

func LoadCachedProviders() (map[string]modelsDBProvider, string)

LoadCachedProviders reads the cached provider data from disk. Returns nil, "" if no cache exists or the cache is unreadable.

func ParseModelString 

func ParseModelString(modelString string) (provider, model string, err error)

ParseModelString parses a model string in "provider/model" format (e.g. "anthropic/claude-sonnet-4-5"). It splits on the first "/" to extract the provider and model name. For backward compatibility, the legacy "provider:model" format is also accepted with a deprecation warning printed to stderr. Returns the provider name, model name, and any error.

func ReloadGlobalRegistry 

func ReloadGlobalRegistry()

ReloadGlobalRegistry rebuilds the global registry from the current data sources (cache → embedded). Call after updating the cache.

func RemoveCachedProviders 

func RemoveCachedProviders() error

RemoveCachedProviders deletes the cache file, causing the registry to fall back to the embedded model database on next load.

func StoreCachedProviders 

func StoreCachedProviders(providers map[string]modelsDBProvider, etag string) error

StoreCachedProviders writes provider data to the cache file on disk.

func ThinkingLevelDescription added in v0.7.0 

func ThinkingLevelDescription(level ThinkingLevel) string

ThinkingLevelDescription returns a human-readable description of a thinking level.

Types

type Cost 

type Cost struct {
	Input      float64
	Output     float64
	CacheRead  *float64
	CacheWrite *float64
}

Cost represents the pricing information for a model.

type CostConfig added in v0.22.0 

type CostConfig struct {
	Input  float64 `json:"input" yaml:"input"`
	Output float64 `json:"output" yaml:"output"`
}

CostConfig defines the pricing for a custom model.

type CustomModelConfig added in v0.22.0 

type CustomModelConfig struct {
	Name        string      `json:"name" yaml:"name"`
	BaseURL     string      `json:"baseUrl,omitempty" yaml:"baseUrl,omitempty"`
	APIKey      string      `json:"apiKey,omitempty" yaml:"apiKey,omitempty"`
	Family      string      `json:"family,omitempty" yaml:"family,omitempty"`
	Attachment  bool        `json:"attachment,omitempty" yaml:"attachment,omitempty"`
	Reasoning   bool        `json:"reasoning,omitempty" yaml:"reasoning,omitempty"`
	Temperature bool        `json:"temperature,omitempty" yaml:"temperature,omitempty"`
	Knowledge   string      `json:"knowledge,omitempty" yaml:"knowledge,omitempty"`
	Cost        CostConfig  `json:"cost" yaml:"cost"`
	Limit       LimitConfig `json:"limit" yaml:"limit"`
}

CustomModelConfig defines a custom model configuration loaded from the config file. This is a duplicate here to avoid circular dependencies with internal/config.

type Limit 

type Limit struct {
	Context int
	Output  int
}

Limit represents the context and output limits for a model.

type LimitConfig added in v0.22.0 

type LimitConfig struct {
	Context int `json:"context" yaml:"context"`
	Output  int `json:"output" yaml:"output"`
}

LimitConfig defines context and output limits for a custom model.

type ModelInfo 

type ModelInfo struct {
	ID          string
	Name        string
	Family      string // Model family (e.g., "claude", "gpt", "gemini")
	Attachment  bool
	Reasoning   bool
	Temperature bool
	Cost        Cost
	Limit       Limit
	ProviderNPM string // Model-specific provider npm override (e.g. "@ai-sdk/anthropic")
	BaseURL     string // Per-model base URL override (custom models only)
	APIKey      string // Per-model API key override (custom models only)
}

ModelInfo represents information about a specific model.

func (*ModelInfo) CacheType added in v0.30.0 

func (m *ModelInfo) CacheType() string

CacheType returns the appropriate cache mechanism for this model family. Returns empty string if caching is not supported.

func (*ModelInfo) SupportsCaching added in v0.30.0 

func (m *ModelInfo) SupportsCaching() bool

SupportsCaching returns true if this model family supports prompt caching. This enables automatic cost savings for supported models regardless of provider.

type ModelsDBProviders 

type ModelsDBProviders = map[string]modelsDBProvider

ModelsDBProviders is the top-level type for models.dev/api.json data: a map of provider ID → provider object.

type ModelsRegistry 

type ModelsRegistry struct {
	// contains filtered or unexported fields
}

ModelsRegistry provides validation and information about models. It maintains a registry of all supported LLM providers and their models, including capabilities, pricing, and configuration requirements. The registry data comes from models.dev.

func GetGlobalRegistry 

func GetGlobalRegistry() *ModelsRegistry

GetGlobalRegistry returns the global models registry instance.

func NewModelsRegistry 

func NewModelsRegistry() *ModelsRegistry

NewModelsRegistry creates a new models registry populated from models.dev data.

func (*ModelsRegistry) GetFantasyProviders deprecated 

func (r *ModelsRegistry) GetFantasyProviders() []string

Deprecated: Use GetLLMProviders instead.

func (*ModelsRegistry) GetLLMProviders added in v0.30.0 

func (r *ModelsRegistry) GetLLMProviders() []string

GetLLMProviders returns provider IDs that have LLM support, either through a native provider or via openaicompat auto-routing.

func (*ModelsRegistry) GetModelsForProvider 

func (r *ModelsRegistry) GetModelsForProvider(provider string) (map[string]ModelInfo, error)

GetModelsForProvider returns all models for a specific provider.

func (*ModelsRegistry) GetProviderInfo 

func (r *ModelsRegistry) GetProviderInfo(provider string) *ProviderInfo

GetProviderInfo returns the full provider info, or nil if not found.

func (*ModelsRegistry) GetSupportedProviders 

func (r *ModelsRegistry) GetSupportedProviders() []string

GetSupportedProviders returns a list of all provider IDs in the registry.

func (*ModelsRegistry) LookupModel 

func (r *ModelsRegistry) LookupModel(provider, modelID string) *ModelInfo

LookupModel returns model metadata from the database if available. Returns nil when the model or provider is not in the database — this is expected for new models, custom fine-tunes, or providers the database doesn't track yet. Callers should treat a nil return as "unknown model" and continue with sensible defaults.

func (*ModelsRegistry) SuggestModels 

func (r *ModelsRegistry) SuggestModels(provider, invalidModel string) []string

SuggestModels returns similar model names when an invalid model is provided.

func (*ModelsRegistry) ValidateEnvironment 

func (r *ModelsRegistry) ValidateEnvironment(provider string, apiKey string) error

ValidateEnvironment checks if required credentials are available for a provider. It checks the explicit API key, stored credentials (for providers that support them, such as Anthropic OAuth), and environment variables. Returns nil for providers not in the registry (unknown providers are assumed to handle auth themselves or via --provider-api-key).

func (*ModelsRegistry) ValidateModelString added in v0.35.0 

func (r *ModelsRegistry) ValidateModelString(modelString string) error

ValidateModelString checks whether a model string is well-formed and refers to a known provider. It returns a user-friendly error with suggestions when the model or provider is unrecognised. Passing validation does not guarantee that API authentication will succeed — it only catches obvious mistakes (typos, missing provider prefix, non-existent provider names) early so that callers such as subagent spawning can return fast feedback.

Unknown models under a known provider are allowed (the provider API is the authority), but a completely unknown provider is rejected.

type OllamaLoadingResult 

type OllamaLoadingResult struct {
	Message string
}

OllamaLoadingResult contains the result of model loading with actual settings used.

type ProviderConfig 

type ProviderConfig struct {
	ModelString      string
	SystemPrompt     string
	ProviderAPIKey   string
	ProviderURL      string
	MaxTokens        int
	Temperature      *float32
	TopP             *float32
	TopK             *int32
	FrequencyPenalty *float32
	PresencePenalty  *float32
	StopSequences    []string
	NumGPU           *int32
	MainGPU          *int32
	TLSSkipVerify    bool
	ThinkingLevel    ThinkingLevel
	DisableCaching   bool // Opt-out: set to true to disable automatic prompt caching
}

ProviderConfig holds configuration for creating LLM providers.

type ProviderInfo 

type ProviderInfo struct {
	ID     string
	Env    []string
	NPM    string // npm package identifier from models.dev (e.g. "@ai-sdk/openai-compatible")
	API    string // base API URL for openai-compatible providers
	Name   string
	Models map[string]ModelInfo
}

ProviderInfo represents information about a model provider.

type ProviderPool added in v0.9.0 

type ProviderPool struct {
	// contains filtered or unexported fields
}

ProviderPool manages reusable LLM provider instances to reduce overhead when spawning multiple subagents or making repeated completion calls.

func GetGlobalPool added in v0.9.0 

func GetGlobalPool() *ProviderPool

GetGlobalPool returns the singleton provider pool instance.

func NewProviderPool added in v0.9.0 

func NewProviderPool(ttl time.Duration) *ProviderPool

NewProviderPool creates a provider pool with the given TTL for idle providers.

func (*ProviderPool) Close added in v0.9.0 

func (p *ProviderPool) Close()

Close shuts down the pool and releases all providers.

func (*ProviderPool) Get added in v0.9.0 

func (p *ProviderPool) Get(ctx context.Context, modelString string) (fantasy.LanguageModel, fantasy.ProviderOptions, func(), error)

Get returns a provider for the model string, creating one if needed. The returned release function must be called when the provider is no longer needed. The provider may be reused by subsequent Get calls.

type ProviderResult 

type ProviderResult struct {
	// Model is the created fantasy LanguageModel
	Model fantasy.LanguageModel
	// Message contains optional feedback for the user
	Message string
	// Closer is an optional cleanup function for providers that hold
	// resources (e.g. kronk's loaded models). May be nil.
	Closer io.Closer
	// ProviderOptions contains provider-specific options to be passed to the
	// fantasy agent (e.g. OpenAI Responses API reasoning options).
	ProviderOptions fantasy.ProviderOptions
	// SkipMaxOutputTokens indicates that this provider doesn't support the
	// max_output_tokens parameter (e.g., OpenAI Codex OAuth API).
	SkipMaxOutputTokens bool
}

ProviderResult contains the result of provider creation.

func CreateProvider 

func CreateProvider(ctx context.Context, config *ProviderConfig) (*ProviderResult, error)

CreateProvider creates a fantasy LanguageModel based on the provider configuration. Model metadata is looked up from the models.dev database for cost tracking and capability detection, but unknown models are passed through to the provider API — the database is advisory, not a gatekeeper.

Native providers: anthropic, openai, google, ollama, azure, google-vertex-anthropic, openrouter, bedrock, vercel. Any provider in models.dev with an api URL or openai-compatible npm package is auto-routed through fantasy's openaicompat provider.

type ThinkingLevel added in v0.7.0 

type ThinkingLevel string

ThinkingLevel controls extended thinking / reasoning budget for supported models. 

const (
	ThinkingOff     ThinkingLevel = "off"
	ThinkingMinimal ThinkingLevel = "minimal"
	ThinkingLow     ThinkingLevel = "low"
	ThinkingMedium  ThinkingLevel = "medium"
	ThinkingHigh    ThinkingLevel = "high"
)

func ParseThinkingLevel added in v0.7.0 

func ParseThinkingLevel(s string) ThinkingLevel

ParseThinkingLevel converts a string to a ThinkingLevel, defaulting to ThinkingOff.

func ThinkingLevels added in v0.7.0 

func ThinkingLevels() []ThinkingLevel

ThinkingLevels returns the ordered list of available thinking levels for cycling.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.