model

package
v0.3.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 21, 2026 License: MIT Imports: 16 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var TokenTracker = &TokenUsage{}

TokenTracker is a global token usage tracker

Functions

func FormatAPIError 

func FormatAPIError(err error, attempt, maxRetries int) string

FormatAPIError produces a user-friendly error message with retry context.

func GetModelContextLimit 

func GetModelContextLimit(modelName string) int

GetModelContextLimit returns the known context limit for a given model name. Returns 0 if the model is not in the known list.

func GetTokenUsage 

func GetTokenUsage() (prompt, completion, total int64)

GetTokenUsage returns the current token usage statistics

func IsRetryable 

func IsRetryable(_ context.Context, err error) bool

IsRetryable returns true if the error should be retried. It is designed to be used as ModelRetryConfig.IsRetryAble in the Eino framework.

Context overflow errors are NOT retryable — they need compaction. Auth errors are NOT retryable — they need user action.

func NewChatModel 

func NewChatModel(_ context.Context, cfg *ChatModelConfig) (einomodel.ToolCallingChatModel, error)

func ParseProviderModel 

func ParseProviderModel(s string) (provider, model string, err error)

ParseProviderModel splits "provider/model" into its components.

func ParseRetryAfter 

func ParseRetryAfter(err error) time.Duration

ParseRetryAfter extracts a delay from an error message or OpenAI APIError. It looks for Retry-After header patterns in the error text. Returns 0 if no delay information is found.

func ResetTokenUsage 

func ResetTokenUsage()

ResetTokenUsage resets the token usage tracker

func SmartBackoff 

func SmartBackoff(ctx context.Context, attempt int) time.Duration

SmartBackoff returns a delay for the given retry attempt, respecting server-sent Retry-After hints when available. It is designed to be used as ModelRetryConfig.BackoffFunc in the Eino framework.

Strategy (matching Claude-Code & OpenCode patterns):

  1. If the error contains a Retry-After hint, use it (capped at 5 min).
  2. Otherwise fall back to exponential backoff: 500ms × 2^(attempt-1), capped at 32s, plus 0-25% random jitter.

func WithRetryError 

func WithRetryError(ctx context.Context, err error) context.Context

WithRetryError stores an error in context for BackoffFunc to inspect.

func WithTokenTracker 

func WithTokenTracker(ctx context.Context, t *TokenUsage) context.Context

WithTokenTracker attaches a per-agent TokenUsage to the context. chatModel.Generate/Stream will increment this tracker in addition to the global TokenTracker.

Types

type APIErrorCategory 

type APIErrorCategory int

APIErrorCategory classifies LLM API errors into actionable categories. 

const (
	// ErrCategoryTransient — network blips, timeouts, 5xx; safe to retry.
	ErrCategoryTransient APIErrorCategory = iota
	// ErrCategoryRateLimit — 429 / "overloaded"; retry with back-off.
	ErrCategoryRateLimit
	// ErrCategoryContextOverflow — input too long; needs compaction, NOT retry.
	ErrCategoryContextOverflow
	// ErrCategoryAuth — 401/403; permanent until key is fixed.
	ErrCategoryAuth
	// ErrCategoryFatal — 400 bad request, unknown; do not retry.
	ErrCategoryFatal
)

func ClassifyError 

func ClassifyError(err error) APIErrorCategory

ClassifyError determines the category of an API error.

func (APIErrorCategory) String 

func (c APIErrorCategory) String() string

type ChatModelConfig 

type ChatModelConfig struct {
	Model   string
	APIKey  string
	BaseURL string
}

type ContextOverflowInfo 

type ContextOverflowInfo struct {
	ActualTokens int
	LimitTokens  int
	TokenGap     int // ActualTokens - LimitTokens
}

ContextOverflowInfo holds parsed token counts from an overflow error.

func ParseContextOverflow 

func ParseContextOverflow(err error) *ContextOverflowInfo

ParseContextOverflow extracts token counts from a context overflow error. Returns nil if the error is not a context overflow or counts cannot be parsed.

type ModelCost 

type ModelCost struct {
	Input      float64 `json:"input"`
	Output     float64 `json:"output"`
	CacheRead  float64 `json:"cache_read,omitempty"`
	CacheWrite float64 `json:"cache_write,omitempty"`
}

ModelCost describes per-token costs in USD per 1M tokens.

type ModelFactory 

type ModelFactory struct {
	// contains filtered or unexported fields
}

ModelFactory creates and caches ChatModel instances by "provider/model" identifier.

func NewModelFactory 

func NewModelFactory(cfg *config.Config, fallback einomodel.ToolCallingChatModel) *ModelFactory

NewModelFactory creates a model factory with the given config, fallback model, and registry.

func (*ModelFactory) Fallback 

func (f *ModelFactory) Fallback() einomodel.ToolCallingChatModel

Fallback returns the default fallback model.

func (*ModelFactory) GetModel 

func (f *ModelFactory) GetModel(ctx context.Context, providerModel string) (einomodel.ToolCallingChatModel, error)

GetModel returns a ChatModel for the given "provider/model" identifier. Empty string returns the fallback model.

func (*ModelFactory) Registry 

func (f *ModelFactory) Registry() *ModelRegistry

Registry returns the underlying ModelRegistry for metadata lookups.

type ModelInfo 

type ModelInfo struct {
	ID           string
	ContextLimit int // Maximum context window size, 0 if unknown
	Pricing      ModelPricing
}

ModelInfo contains information about a model

type ModelLimit 

type ModelLimit struct {
	Context int `json:"context"`
	Input   int `json:"input,omitempty"`
	Output  int `json:"output,omitempty"`
}

ModelLimit describes context window and output limits.

type ModelModalities 

type ModelModalities struct {
	Input  []string `json:"input,omitempty"`
	Output []string `json:"output,omitempty"`
}

ModelModalities describes input/output modalities.

type ModelPricing 

type ModelPricing struct {
	InputPer1M  float64 // cost per 1M input tokens
	OutputPer1M float64 // cost per 1M output tokens
}

ModelPricing contains cost information for a model.

type ModelRegistry 

type ModelRegistry struct {
}

ModelRegistry provides model metadata from models.dev. The data is statically generated at build time via go:generate.

func NewModelRegistry 

func NewModelRegistry() *ModelRegistry

NewModelRegistry creates a new ModelRegistry.

func (*ModelRegistry) GetModelContextLimit 

func (r *ModelRegistry) GetModelContextLimit(providerID, modelID string) int

GetModelContextLimit returns the context limit for a model looked up via registry.

func (*ModelRegistry) GetModelCost 

func (r *ModelRegistry) GetModelCost(providerID, modelID string) (inputPer1M, outputPer1M float64)

GetModelCost returns pricing info for a model.

func (*ModelRegistry) GetProvider 

func (r *ModelRegistry) GetProvider(providerID string) *RegistryProvider

GetProvider returns provider info by ID, or nil if not found.

func (*ModelRegistry) GetProviderAPI 

func (r *ModelRegistry) GetProviderAPI(providerID string) string

GetProviderAPI returns the API base URL for a provider from the registry.

func (*ModelRegistry) GetProviderEnvVars 

func (r *ModelRegistry) GetProviderEnvVars(providerID string) []string

GetProviderEnvVars returns the environment variable names for a provider.

func (*ModelRegistry) HasProvider 

func (r *ModelRegistry) HasProvider(providerID string) bool

HasProvider returns whether the given provider ID exists in the registry.

func (*ModelRegistry) ListProviderModels 

func (r *ModelRegistry) ListProviderModels(providerID string, toolCallOnly bool) []*RegistryModel

ListProviderModels returns models for a provider from the registry. If toolCallOnly is true, only models with tool_call support are returned. Models are sorted by ID.

func (*ModelRegistry) ListProviders 

func (r *ModelRegistry) ListProviders() []*RegistryProvider

ListProviders returns all providers in the curated display order.

func (*ModelRegistry) Load 

func (r *ModelRegistry) Load() (map[string]*RegistryProvider, error)

Load returns the statically generated provider/model data.

func (*ModelRegistry) LookupModel 

func (r *ModelRegistry) LookupModel(providerID, modelID string) (*RegistryProvider, *RegistryModel, bool)

LookupModel finds a model by "provider/model" identifier. Returns the provider info, model info, and whether it was found.

type RegistryModel 

type RegistryModel struct {
	ID               string           `json:"id"`
	Name             string           `json:"name"`
	Family           string           `json:"family,omitempty"`
	Attachment       bool             `json:"attachment,omitempty"`
	Reasoning        bool             `json:"reasoning,omitempty"`
	ToolCall         bool             `json:"tool_call,omitempty"`
	StructuredOutput bool             `json:"structured_output,omitempty"`
	Temperature      bool             `json:"temperature,omitempty"`
	Knowledge        string           `json:"knowledge,omitempty"`
	ReleaseDate      string           `json:"release_date,omitempty"`
	LastUpdated      string           `json:"last_updated,omitempty"`
	Modalities       *ModelModalities `json:"modalities,omitempty"`
	OpenWeights      bool             `json:"open_weights,omitempty"`
	Cost             *ModelCost       `json:"cost,omitempty"`
	Limit            *ModelLimit      `json:"limit,omitempty"`
	Status           string           `json:"status,omitempty"`
}

RegistryModel represents a model from models.dev API.

type RegistryProvider 

type RegistryProvider struct {
	ID     string                    `json:"id"`
	Name   string                    `json:"name"`
	Env    []string                  `json:"env"`
	API    string                    `json:"api"`
	Doc    string                    `json:"doc,omitempty"`
	Models map[string]*RegistryModel `json:"models"`
}

RegistryProvider represents a provider from models.dev API.

type TokenUsage 

type TokenUsage struct {
	PromptTokens     int64
	CompletionTokens int64
	TotalTokens      int64
	LastTotalTokens  int64
	// contains filtered or unexported fields
}

TokenUsage tracks token consumption across all API calls

func TokenTrackerFromContext 

func TokenTrackerFromContext(ctx context.Context) *TokenUsage

TokenTrackerFromContext retrieves the per-agent TokenUsage from the context, if any.

func (*TokenUsage) Add 

func (t *TokenUsage) Add(prompt, completion, total int)

Add adds token usage to the tracker

func (*TokenUsage) AddByModel 

func (t *TokenUsage) AddByModel(model string, prompt, completion, total int)

AddByModel adds token usage attributed to a specific model name.

func (*TokenUsage) Get 

func (t *TokenUsage) Get() (prompt, completion, total int64)

Get returns the current token usage

func (*TokenUsage) GetByModel 

func (t *TokenUsage) GetByModel() map[string]int64

GetByModel returns a snapshot of per-model token totals.

func (*TokenUsage) GetLastTotal added in v0.3.2 

func (t *TokenUsage) GetLastTotal() int64

GetLastTotal returns the last API call's total tokens (current context usage)

func (*TokenUsage) Reset 

func (t *TokenUsage) Reset()

Reset resets the token tracker

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.