model

package
v0.4.8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 22, 2026 License: MIT Imports: 17 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var TokenTracker = &TokenUsage{}

TokenTracker is a global token usage tracker

Functions

func FormatAPIError 

func FormatAPIError(err error, attempt, maxRetries int) string

FormatAPIError produces a user-friendly error message with retry context.

func GetModelContextLimit 

func GetModelContextLimit(modelName string) int

GetModelContextLimit returns the known context limit for a given model name. Returns 0 if the model is not in the known list.

func GetTokenUsage 

func GetTokenUsage() (prompt, completion, total int64)

GetTokenUsage returns the current token usage statistics

func IsRetryable 

func IsRetryable(_ context.Context, err error) bool

IsRetryable returns true if the error should be retried. It is designed to be used as ModelRetryConfig.IsRetryAble in the Eino framework.

Context overflow errors are NOT retryable — they need compaction. Auth errors are NOT retryable — they need user action.

func NewChatModel 

func NewChatModel(_ context.Context, cfg *ChatModelConfig) (einomodel.ToolCallingChatModel, error)

func ParseProviderModel 

func ParseProviderModel(s string) (provider, model string, err error)

ParseProviderModel splits "provider/model" into its components.

func ParseRetryAfter 

func ParseRetryAfter(err error) time.Duration

ParseRetryAfter extracts a delay from an error message or OpenAI APIError. It looks for Retry-After header patterns in the error text. Returns 0 if no delay information is found.

func ResetTokenUsage 

func ResetTokenUsage()

ResetTokenUsage resets the token usage tracker

func SmartBackoff 

func SmartBackoff(ctx context.Context, attempt int) time.Duration

SmartBackoff returns a delay for the given retry attempt, respecting server-sent Retry-After hints when available. It is designed to be used as ModelRetryConfig.BackoffFunc in the Eino framework.

Strategy (matching Claude-Code & OpenCode patterns):

  1. If the error contains a Retry-After hint, use it (capped at 5 min).
  2. Otherwise fall back to exponential backoff: 500ms × 2^(attempt-1), capped at 32s, plus 0-25% random jitter.

func ValidateProvider added in v0.3.10 

func ValidateProvider(ctx context.Context, apiKey, baseURL string) error

ValidateProvider tests connectivity to a provider by making a lightweight GET /models request. Returns nil on success, or a descriptive error.

func WithRetryError 

func WithRetryError(ctx context.Context, err error) context.Context

WithRetryError stores an error in context for BackoffFunc to inspect.

func WithTokenTracker 

func WithTokenTracker(ctx context.Context, t *TokenUsage) context.Context

WithTokenTracker attaches a per-agent TokenUsage to the context. chatModel.Generate/Stream will increment this tracker in addition to the global TokenTracker.

Types

type APIErrorCategory 

type APIErrorCategory int

APIErrorCategory classifies LLM API errors into actionable categories. 

const (
	// ErrCategoryTransient — network blips, timeouts, 5xx; safe to retry.
	ErrCategoryTransient APIErrorCategory = iota
	// ErrCategoryRateLimit — 429 / "overloaded"; retry with back-off.
	ErrCategoryRateLimit
	// ErrCategoryContextOverflow — input too long; needs compaction, NOT retry.
	ErrCategoryContextOverflow
	// ErrCategoryAuth — 401/403; permanent until key is fixed.
	ErrCategoryAuth
	// ErrCategoryFatal — 400 bad request, unknown; do not retry.
	ErrCategoryFatal
)

func ClassifyError 

func ClassifyError(err error) APIErrorCategory

ClassifyError determines the category of an API error.

func (APIErrorCategory) String 

func (c APIErrorCategory) String() string

type ChatModelConfig 

type ChatModelConfig struct {
	Model   string
	APIKey  string
	BaseURL string
}

type ContextOverflowInfo 

type ContextOverflowInfo struct {
	ActualTokens int
	LimitTokens  int
	TokenGap     int // ActualTokens - LimitTokens
}

ContextOverflowInfo holds parsed token counts from an overflow error.

func ParseContextOverflow 

func ParseContextOverflow(err error) *ContextOverflowInfo

ParseContextOverflow extracts token counts from a context overflow error. Returns nil if the error is not a context overflow or counts cannot be parsed.

type ModelCost 

type ModelCost struct {
	Input      float64 `json:"input"`
	Output     float64 `json:"output"`
	CacheRead  float64 `json:"cache_read,omitempty"`
	CacheWrite float64 `json:"cache_write,omitempty"`
}

ModelCost describes per-token costs in USD per 1M tokens.

type ModelFactory 

type ModelFactory struct {
	// contains filtered or unexported fields
}

ModelFactory creates and caches ChatModel instances by "provider/model" identifier.

func NewModelFactory 

func NewModelFactory(cfg *config.Config, fallback einomodel.ToolCallingChatModel) *ModelFactory

NewModelFactory creates a model factory with the given config, fallback model, and registry.

func (*ModelFactory) Fallback 

func (f *ModelFactory) Fallback() einomodel.ToolCallingChatModel

Fallback returns the default fallback model.

func (*ModelFactory) GetModel 

func (f *ModelFactory) GetModel(ctx context.Context, providerModel string) (einomodel.ToolCallingChatModel, error)

GetModel returns a ChatModel for the given "provider/model" identifier. Empty string returns the fallback model.

func (*ModelFactory) Registry 

func (f *ModelFactory) Registry() *ModelRegistry

Registry returns the underlying ModelRegistry for metadata lookups.

type ModelInfo 

type ModelInfo struct {
	ID           string
	ContextLimit int // Maximum context window size, 0 if unknown
	Pricing      ModelPricing
}

ModelInfo contains information about a model

type ModelLimit 

type ModelLimit struct {
	Context int `json:"context"`
	Input   int `json:"input,omitempty"`
	Output  int `json:"output,omitempty"`
}

ModelLimit describes context window and output limits.

type ModelModalities 

type ModelModalities struct {
	Input  []string `json:"input,omitempty"`
	Output []string `json:"output,omitempty"`
}

ModelModalities describes input/output modalities.

type ModelPricing 

type ModelPricing struct {
	InputPer1M  float64 // cost per 1M input tokens
	OutputPer1M float64 // cost per 1M output tokens
}

ModelPricing contains cost information for a model.

type ModelRegistry 

type ModelRegistry struct {
	// contains filtered or unexported fields
}

ModelRegistry provides model metadata from models.dev and custom config. The base data is statically generated at build time via go:generate. Custom models from config are merged in at runtime.

func NewModelRegistry 

func NewModelRegistry() *ModelRegistry

NewModelRegistry creates a new ModelRegistry with a deep copy of generated data. Each RegistryProvider and its Models map are copied so that merging custom models at runtime never mutates the shared generatedProviders.

func NewModelRegistryWithConfig added in v0.4.8 

func NewModelRegistryWithConfig(cfg *config.Config) *ModelRegistry

NewModelRegistryWithConfig creates a ModelRegistry and merges custom models from config.

func (*ModelRegistry) GetModelContextLimit 

func (r *ModelRegistry) GetModelContextLimit(providerID, modelID string) int

GetModelContextLimit returns the context limit for a model looked up via registry.

func (*ModelRegistry) GetModelCost 

func (r *ModelRegistry) GetModelCost(providerID, modelID string) (inputPer1M, outputPer1M float64)

GetModelCost returns pricing info for a model.

func (*ModelRegistry) GetProvider 

func (r *ModelRegistry) GetProvider(providerID string) *RegistryProvider

GetProvider returns provider info by ID, or nil if not found.

func (*ModelRegistry) GetProviderAPI 

func (r *ModelRegistry) GetProviderAPI(providerID string) string

GetProviderAPI returns the API base URL for a provider from the registry.

func (*ModelRegistry) GetProviderEnvVars 

func (r *ModelRegistry) GetProviderEnvVars(providerID string) []string

GetProviderEnvVars returns the environment variable names for a provider.

func (*ModelRegistry) HasProvider 

func (r *ModelRegistry) HasProvider(providerID string) bool

HasProvider returns whether the given provider ID exists in the registry.

func (*ModelRegistry) ListProviderModels 

func (r *ModelRegistry) ListProviderModels(providerID string, toolCallOnly bool) []*RegistryModel

ListProviderModels returns models for a provider from the registry. If toolCallOnly is true, only models with tool_call support are returned. Models are sorted by ID.

func (*ModelRegistry) ListProviders 

func (r *ModelRegistry) ListProviders() []*RegistryProvider

ListProviders returns all providers in the curated display order.

func (*ModelRegistry) Load 

func (r *ModelRegistry) Load() (map[string]*RegistryProvider, error)

Load returns the provider/model data.

func (*ModelRegistry) LookupModel 

func (r *ModelRegistry) LookupModel(providerID, modelID string) (*RegistryProvider, *RegistryModel, bool)

LookupModel finds a model by "provider/model" identifier. Returns the provider info, model info, and whether it was found.

func (*ModelRegistry) MergeConfigProviders added in v0.4.8 

func (r *ModelRegistry) MergeConfigProviders(providers map[string]*config.ProviderConfig)

MergeConfigProviders merges custom models from config providers into the registry. For providers not in the registry, a new entry is created. For existing providers, custom models are added (existing models are not overridden).

type RegistryModel 

type RegistryModel struct {
	ID               string           `json:"id"`
	Name             string           `json:"name"`
	Family           string           `json:"family,omitempty"`
	Attachment       bool             `json:"attachment,omitempty"`
	Reasoning        bool             `json:"reasoning,omitempty"`
	ToolCall         bool             `json:"tool_call,omitempty"`
	StructuredOutput bool             `json:"structured_output,omitempty"`
	Temperature      bool             `json:"temperature,omitempty"`
	Knowledge        string           `json:"knowledge,omitempty"`
	ReleaseDate      string           `json:"release_date,omitempty"`
	LastUpdated      string           `json:"last_updated,omitempty"`
	Modalities       *ModelModalities `json:"modalities,omitempty"`
	OpenWeights      bool             `json:"open_weights,omitempty"`
	Cost             *ModelCost       `json:"cost,omitempty"`
	Limit            *ModelLimit      `json:"limit,omitempty"`
	Status           string           `json:"status,omitempty"`
	Recommended      bool             `json:"recommended,omitempty"`
	DefaultEnabled   bool             `json:"default_enabled,omitempty"`
}

RegistryModel represents a model from models.dev API.

type RegistryProvider 

type RegistryProvider struct {
	ID     string                    `json:"id"`
	Name   string                    `json:"name"`
	Env    []string                  `json:"env"`
	API    string                    `json:"api"`
	Doc    string                    `json:"doc,omitempty"`
	Models map[string]*RegistryModel `json:"models"`
}

RegistryProvider represents a provider from models.dev API.

type TokenUsage 

type TokenUsage struct {
	PromptTokens     int64
	CompletionTokens int64
	TotalTokens      int64
	CachedTokens     int64
	LastTotalTokens  int64
	// contains filtered or unexported fields
}

TokenUsage tracks token consumption across all API calls

func TokenTrackerFromContext 

func TokenTrackerFromContext(ctx context.Context) *TokenUsage

TokenTrackerFromContext retrieves the per-agent TokenUsage from the context, if any.

func (*TokenUsage) Add 

func (t *TokenUsage) Add(prompt, completion, total, cached int)

Add adds token usage to the tracker

func (*TokenUsage) AddByModel 

func (t *TokenUsage) AddByModel(model string, prompt, completion, total int)

AddByModel adds token usage attributed to a specific model name.

func (*TokenUsage) Get 

func (t *TokenUsage) Get() (prompt, completion, total int64)

Get returns the current token usage

func (*TokenUsage) GetByModel 

func (t *TokenUsage) GetByModel() map[string]int64

GetByModel returns a snapshot of per-model token totals.

func (*TokenUsage) GetLastDetail added in v0.4.4 

func (t *TokenUsage) GetLastDetail() *TokenUsageDetail

GetLastDetail returns the last API call's token usage detail.

func (*TokenUsage) GetLastTotal added in v0.3.2 

func (t *TokenUsage) GetLastTotal() int64

GetLastTotal returns the last API call's total tokens (current context usage)

func (*TokenUsage) Reset 

func (t *TokenUsage) Reset()

Reset resets the token tracker

type TokenUsageDetail added in v0.4.4 

type TokenUsageDetail struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`
	CachedTokens     int `json:"cached_tokens"`
}

TokenUsageDetail holds per-call token usage details for tracing/observability.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.