providers

package
v0.11.0-beta.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 23, 2026 License: Apache-2.0, MIT Imports: 32 Imported by: 0

Documentation

Overview

Package providers — canonical provider default URLs, env vars, and base configuration. Centralises URL fragments that were previously duplicated across factory.go, fetcher.go, config/setup.go, and config/wizard.go.

Package providers — modelsdev.go fetches the live model catalog from https://models.dev/api.json with a disk cache fallback. The catalog is the source of truth for the TUI model picker; it is independent of which providers the user has actually configured.

Index

Constants

View Source
const CacheTTL = 24 * time.Hour

CacheTTL is how long a disk-cached catalog is considered fresh enough to skip the network entirely. Set generously — model lists don't move much hour-to-hour and SSH users feel every network call.

View Source
const (
	// DefaultAnthropicAPIVersion is the version string sent in the
	// anthropic-version header. Override via FactoryConfig.Headers
	// or by setting the ANTHROPIC_VERSION env var.
	DefaultAnthropicAPIVersion = "2023-06-01"
)
View Source
const DefaultOllamaURL = "http://localhost:11434"

DefaultOllamaURL is the canonical URL for a local Ollama instance. Override with the OLLAMA_HOST environment variable when connecting to a remote Ollama server.

View Source
const FetchTimeout = 5 * time.Second

FetchTimeout caps how long the live HTTP fetch may run before falling back to disk cache. 5 seconds aligns with the spec.

View Source
const OpenAIStreamUsageMetadataKey = "openai_stream_usage"

OpenAIStreamUsageMetadataKey controls the explicit compatibility opt-out. Set Request.Metadata[OpenAIStreamUsageMetadataKey] to boolean false for a gateway that rejects stream_options. The default and boolean true request usage. No rejected model request is automatically replayed without the option.

Variables

View Source
var (
	ErrProviderUnavailable = errors.New("provider: unavailable")
	ErrModelNotFound       = errors.New("provider: model not found")
	ErrRateLimited         = errors.New("provider: rate limited")
	ErrContextLength       = errors.New("provider: context length exceeded")
	ErrAuthFailed          = errors.New("provider: authentication failed")
)
View Source
var ModelsDevURL = "https://models.dev/api.json"

ModelsDevURL is the canonical models.dev API endpoint. Exposed as a var so tests can swap it for an httptest server.

View Source
var Version = "0.1.0-dev"

userAgent matches the spec: overkill/<version>. We avoid an import cycle by reading the version at call time from a package-level variable that the caller (cmd/overkill) may overwrite if it changes.

Functions

func AllProviderTypes

func AllProviderTypes() []string

AllProviderTypes returns known provider type names.

func CanonicalAPIKeyEnv

func CanonicalAPIKeyEnv(providerType string) string

CanonicalAPIKeyEnv returns the canonical env var name for a provider type.

func CanonicalBaseURL

func CanonicalBaseURL(providerType string) string

CanonicalBaseURL returns the canonical base URL for a provider type, or "" if unknown.

func CapsTags

func CapsTags(m CatalogModel) []string

CapsTags returns short capability tags ("tools", "vision", "reasoning").

func DefaultBaseURL

func DefaultBaseURL(providerType string) string

func FormatContext

func FormatContext(n int) string

FormatContext renders "128K", "1M", etc. Used by the dialog.

func FormatCost

func FormatCost(in, out float64) string

FormatCost renders "$in/$out per 1M" — used by the dialog rendering.

func OpenAICompatBaseURLs

func OpenAICompatBaseURLs() map[string]string

OpenAICompatBaseURLs returns a map from provider type to base URL, but only for providers that use the OpenAI-compatible API style. Used by the factory's fallback path and any code that needs to route by provider+URL.

func ResolveBaseURL

func ResolveBaseURL(kind, configured string) (string, error)

ResolveBaseURL returns the API prefix shared by execution and diagnostics. Preserve private gateway paths; normalize the historical official Anthropic origin default, whose native protocol endpoints require the /v1 prefix.

func ResolveOAuthAPIKey

func ResolveOAuthAPIKey(provider string) string

ResolveOAuthAPIKey looks up a stored OAuth token for the given provider and returns its access token in a form usable as an API key (Bearer header is injected by the provider client).

Returns "" if no token is stored or the file is unreadable. The factory uses this when ProviderConfig.AuthType == "oauth" and APIKey is empty.

func ResolveTokenLimits

func ResolveTokenLimits(models []Model, id string, outputOverride int) (contextWindow, outputTokens int, err error)

ResolveTokenLimits treats legacy MaxTokens as context capacity only. Unknown models use a conservative context estimate and leave output at zero so the provider chooses its default. An explicit output override must fit both the context and any known model output ceiling.

func WithRetryStream

func WithRetryStream(fn func() (<-chan Chunk, error), isRetryable func(error) bool) (<-chan Chunk, error)

WithRetryStream is a convenience wrapper that uses context.Background(). Prefer WithRetryStreamCtx for cancellation support.

func WithRetryStreamCtx

func WithRetryStreamCtx(ctx context.Context, fn func() (<-chan Chunk, error), isRetryable func(error) bool) (<-chan Chunk, error)

Types

type AnthropicProvider

type AnthropicProvider struct {
	*BaseProvider
}

func NewAnthropicProvider

func NewAnthropicProvider(apiKey string, models []Model) *AnthropicProvider

func (*AnthropicProvider) Complete

func (p *AnthropicProvider) Complete(ctx context.Context, req Request) (Response, error)

func (*AnthropicProvider) Stream

func (p *AnthropicProvider) Stream(ctx context.Context, req Request) (<-chan Chunk, error)

type Attachment

type Attachment struct {
	Kind      AttachmentKind
	MediaType string
	Data      []byte
}

Attachment is a single non-text payload bound to a Message. Data is the raw bytes (the provider layer base64-encodes when the API needs it — keeping bytes in memory lets us size-check before send without double-encoding). MediaType is the IANA media type (e.g. "image/png"); providers reject unknown types so we don't paper over a clipboard read that returned the wrong MIME.

type AttachmentKind

type AttachmentKind string

AttachmentKind enumerates the modalities the agent can submit to a provider. "image" is the only supported value today; the type stays open so audio/PDF can land without re-shaping every call site.

const (
	AttachmentImage AttachmentKind = "image"
)

type BaseProvider

type BaseProvider struct {
	// contains filtered or unexported fields
}

func NewBaseProvider

func NewBaseProvider(name, baseURL, apiKey string, models []Model) *BaseProvider

func (*BaseProvider) Models

func (bp *BaseProvider) Models() []Model

func (*BaseProvider) Name

func (bp *BaseProvider) Name() string

func (*BaseProvider) SetCustomHeaders

func (bp *BaseProvider) SetCustomHeaders(h map[string]string)

SetCustomHeaders merges the given headers into the provider's per-request header map. Existing keys are overwritten. Restricted headers (Authorization, X-Api-Key, Content-Type) are ignored to prevent credential replacement or content-type breakage.

type BedrockProvider

type BedrockProvider struct {
	*BaseProvider
	// contains filtered or unexported fields
}

BedrockProvider implements Provider for AWS Bedrock (Converse API). Uses SigV4 signing via AWS SDK v2.

func NewBedrockProvider

func NewBedrockProvider(region, accessKeyID, secretAccessKey string, models []Model) (*BedrockProvider, error)

func (*BedrockProvider) Complete

func (p *BedrockProvider) Complete(ctx context.Context, req Request) (Response, error)

Complete sends a request to the Bedrock Converse endpoint.

func (*BedrockProvider) Stream

func (p *BedrockProvider) Stream(ctx context.Context, req Request) (<-chan Chunk, error)

Stream implements basic streaming via the Bedrock ConverseStream API. Returns a channel of Chunk events. If the ConverseStream endpoint returns an error, the channel is closed with the error on the first chunk.

type Catalog

type Catalog struct {
	// contains filtered or unexported fields
}

Catalog is the in-memory queryable view over the API payload.

func FetchCatalog

func FetchCatalog(ctx context.Context) (*Catalog, error)

FetchCatalog returns the model catalog with this priority:

  1. fresh disk cache (mtime < CacheTTL old) — no network call
  2. live API — refreshes the cache on success
  3. stale disk cache — any mtime
  4. baked Go fallback

This keeps the model picker snappy: first launch fetches, every subsequent open within 24h is a single file read.

func ParseCatalog

func ParseCatalog(body []byte, source CatalogSource) (*Catalog, error)

parseCatalog decodes the models.dev payload into a Catalog. The payload is a flat object keyed by provider id; each value carries an optional `id` and a `models` map. ParseCatalog builds a Catalog from raw models.dev JSON bytes. Public entry for tests + tools that want to construct a catalog without hitting the network or the disk cache.

func RefreshCatalog

func RefreshCatalog(ctx context.Context) (*Catalog, error)

RefreshCatalog forces a network fetch, updating the disk cache. Used by background warmers or an explicit "refresh" command.

func (*Catalog) All

func (c *Catalog) All() []FlatModel

All returns every model across every provider, sorted by provider then id. Each entry's Family is set to the provider id so callers can group easily.

func (*Catalog) Models

func (c *Catalog) Models(providerID string) []CatalogModel

Models returns the models for one provider, sorted by id. Empty if the providerID is unknown.

func (*Catalog) Providers

func (c *Catalog) Providers() []CatalogProvider

Providers returns the providers sorted by id.

func (*Catalog) Source

func (c *Catalog) Source() CatalogSource

Source returns where this catalog came from (live, cache, or baked).

type CatalogCost

type CatalogCost struct {
	Input      float64 `json:"input,omitempty"`
	Output     float64 `json:"output,omitempty"`
	CacheRead  float64 `json:"cache_read,omitempty"`
	CacheWrite float64 `json:"cache_write,omitempty"`
}

type CatalogLimit

type CatalogLimit struct {
	Context int `json:"context,omitempty"`
	Output  int `json:"output,omitempty"`
}

type CatalogModalities

type CatalogModalities struct {
	Input  []string `json:"input,omitempty"`
	Output []string `json:"output,omitempty"`
}

type CatalogModel

type CatalogModel struct {
	ID          string            `json:"id"`
	Name        string            `json:"name"`
	Family      string            `json:"family,omitempty"`
	Attachment  bool              `json:"attachment,omitempty"`
	Reasoning   bool              `json:"reasoning,omitempty"`
	ToolCall    bool              `json:"tool_call,omitempty"`
	Temperature bool              `json:"temperature,omitempty"`
	OpenWeights bool              `json:"open_weights,omitempty"`
	Knowledge   string            `json:"knowledge,omitempty"`
	ReleaseDate string            `json:"release_date,omitempty"`
	LastUpdated string            `json:"last_updated,omitempty"`
	Modalities  CatalogModalities `json:"modalities,omitempty"`
	Cost        CatalogCost       `json:"cost,omitempty"`
	Limit       CatalogLimit      `json:"limit,omitempty"`
}

CatalogModel is one model entry under a provider.

type CatalogProvider

type CatalogProvider struct {
	ID     string                  `json:"id"`
	Name   string                  `json:"name"`
	NPM    string                  `json:"npm,omitempty"`
	API    string                  `json:"api,omitempty"`
	Doc    string                  `json:"doc,omitempty"`
	Env    []string                `json:"env,omitempty"`
	Models map[string]CatalogModel `json:"models"`
}

CatalogProvider is one provider entry in the live API.

type CatalogSource

type CatalogSource string

CatalogSource indicates which tier produced the loaded catalog.

const (
	SourceLive  CatalogSource = "live"
	SourceCache CatalogSource = "cache"
	SourceBaked CatalogSource = "baked"
)

type Chunk

type Chunk struct {
	Content   string
	ToolCalls []ToolCall
	Done      bool
	// Usage is a provider-reported cumulative snapshot for this request, not
	// a delta. Nil means omitted/unknown, including interrupted streams.
	Usage *Usage
	// Err carries a fatal streaming error (transport drop, scanner failure,
	// parse error on the upstream protocol). Consumers MUST treat a non-nil
	// Err as "this stream did not complete cleanly" — committing the
	// accumulated content as if Done would be a silent wrong answer. The
	// producer emits at most one Err chunk and then closes the channel.
	Err error
}

type ClassifiedError

type ClassifiedError struct {
	Original  error
	Reason    FailoverReason
	Retryable bool
	Cooldown  time.Duration
}

type ErrorClassifier

type ErrorClassifier struct {
}

func NewErrorClassifier

func NewErrorClassifier() *ErrorClassifier

func (*ErrorClassifier) Classify

func (c *ErrorClassifier) Classify(err error) ClassifiedError

func (*ErrorClassifier) Cooldown

func (c *ErrorClassifier) Cooldown(err error) time.Duration

func (*ErrorClassifier) IsRetryable

func (c *ErrorClassifier) IsRetryable(err error) bool

func (*ErrorClassifier) RetryableFunc

func (c *ErrorClassifier) RetryableFunc() func(error) bool

type FactoryConfig

type FactoryConfig struct {
	Name     string
	Type     string
	APIKey   string
	BaseURL  string
	Models   []Model
	Headers  map[string]string
	AuthType string // "oauth" to use stored OAuth tokens via ResolveOAuthAPIKey
}

type FailoverChain

type FailoverChain struct {
	// contains filtered or unexported fields
}

func NewFailoverChain

func NewFailoverChain(providers ...Provider) *FailoverChain

func (*FailoverChain) Complete

func (fc *FailoverChain) Complete(ctx context.Context, req Request) (Response, error)

func (*FailoverChain) MarkFailed

func (fc *FailoverChain) MarkFailed(name string, cooldown time.Duration)

func (*FailoverChain) ResetCooldown

func (fc *FailoverChain) ResetCooldown(name string)

func (*FailoverChain) Stream

func (fc *FailoverChain) Stream(ctx context.Context, req Request) (<-chan Chunk, error)

type FailoverReason

type FailoverReason string
const (
	ReasonAuth            FailoverReason = "auth"
	ReasonRateLimit       FailoverReason = "rate_limit"
	ReasonBilling         FailoverReason = "billing"
	ReasonNetwork         FailoverReason = "network"
	ReasonTimeout         FailoverReason = "timeout"
	ReasonFormat          FailoverReason = "format"
	ReasonContextOverflow FailoverReason = "context_overflow"
	ReasonOverloaded      FailoverReason = "overloaded"
	ReasonModelNotFound   FailoverReason = "model_not_found"
	ReasonUnknown         FailoverReason = "unknown"
)

type FlatModel

type FlatModel struct {
	ProviderID   string
	ProviderName string
	Model        CatalogModel
}

FlatModel pairs a model with its owning provider for flat list rendering.

type GeminiProvider

type GeminiProvider struct {
	*BaseProvider
}

func NewGeminiProvider

func NewGeminiProvider(apiKey string, models []Model) *GeminiProvider

func (*GeminiProvider) Complete

func (p *GeminiProvider) Complete(ctx context.Context, req Request) (Response, error)

func (*GeminiProvider) Stream

func (p *GeminiProvider) Stream(ctx context.Context, req Request) (<-chan Chunk, error)

type HTTPError

type HTTPError struct {
	StatusCode int
	Body       string
}

func (*HTTPError) Error

func (e *HTTPError) Error() string

func (*HTTPError) IsRetryable

func (e *HTTPError) IsRetryable() bool

type HTTPModelFetcher

type HTTPModelFetcher struct {
	// contains filtered or unexported fields
}

func NewModelFetcher

func NewModelFetcher() *HTTPModelFetcher

func (*HTTPModelFetcher) Fetch

func (f *HTTPModelFetcher) Fetch(ctx context.Context, providerType, apiKey, baseURL string) ([]Model, error)

func (*HTTPModelFetcher) FetchAndPersist

func (f *HTTPModelFetcher) FetchAndPersist(ctx context.Context, providerName, providerType, apiKey, baseURL, outputDir string) ([]Model, int, error)

type Message

type Message struct {
	Role       string
	Content    string
	ToolCalls  []ToolCall
	ToolCallID string
	// Attachments carry non-text inputs (images today, potentially
	// audio/PDF later). Empty for the common case so a string Content
	// still serializes as a plain string-content message to every
	// provider. When non-empty, the provider's request builder emits
	// the format-specific multi-part body (anthropic content blocks,
	// OpenAI content parts, gemini inlineData). Order is preserved.
	Attachments []Attachment
}

type MockProvider

type MockProvider struct {
	// contains filtered or unexported fields
}

func NewMockProvider

func NewMockProvider(name string, models []Model, handler func(Request) (Response, error)) *MockProvider

func (*MockProvider) Complete

func (m *MockProvider) Complete(_ context.Context, req Request) (Response, error)

func (*MockProvider) Models

func (m *MockProvider) Models() []Model

func (*MockProvider) Name

func (m *MockProvider) Name() string

func (*MockProvider) Stream

func (m *MockProvider) Stream(_ context.Context, req Request) (<-chan Chunk, error)

type Model

type Model struct {
	ID                string
	Name              string
	Family            string
	MaxTokens         int
	ContextWindow     int
	MaxOutputTokens   int // explicit completion ceiling, independent of context capacity
	DefaultMaxTokens  int
	CostIn            float64
	CostOut           float64
	CostCacheIn       float64
	CostCacheOut      float64
	SupportsTools     bool
	SupportsStreaming bool
	SupportsVision    bool
	Reasoning         bool
	StructuredOutput  bool
	Temperature       bool
	Attachment        bool
	OpenWeights       bool
	ReleaseDate       string
	LastUpdated       string
	Knowledge         string
	Status            string
	InputModalities   []string
	OutputModalities  []string
}

func AnthropicModels

func AnthropicModels() []Model

func AzureModels

func AzureModels() []Model

func BuiltinModelByID

func BuiltinModelByID(id string) *Model

BuiltinModelByID searches every built-in catalog and returns the first matching Model by ID. Returns nil when no preset has pricing for the requested model — the caller should fall back to the generic sentinel.

func CerebrasModels

func CerebrasModels() []Model

func CopilotModels

func CopilotModels() []Model

func DeepInfraModels

func DeepInfraModels() []Model

func DeepSeekModels

func DeepSeekModels() []Model

func FireworksModels

func FireworksModels() []Model

func GeminiModels

func GeminiModels() []Model

func GroqModels

func GroqModels() []Model

func MistralModels

func MistralModels() []Model

func OllamaModels

func OllamaModels() []Model

func OpenAIModels

func OpenAIModels() []Model

func OpenRouterModels

func OpenRouterModels() []Model

func PerplexityModels

func PerplexityModels() []Model

func TogetherAIModels

func TogetherAIModels() []Model

func VertexModels

func VertexModels() []Model

func XAIModels

func XAIModels() []Model

type ModelCatalog

type ModelCatalog struct {
	// contains filtered or unexported fields
}

func LoadCatalog

func LoadCatalog(dir string) (*ModelCatalog, error)

func (*ModelCatalog) ByCapability

func (mc *ModelCatalog) ByCapability(requiresToolCall bool, requiresVision bool) []*Model

func (*ModelCatalog) ByFamily

func (mc *ModelCatalog) ByFamily(family string) []*Model

func (*ModelCatalog) Get

func (mc *ModelCatalog) Get(id string) (*Model, error)

func (*ModelCatalog) List

func (mc *ModelCatalog) List() []string

type ModelFetcher

type ModelFetcher interface {
	Fetch(ctx context.Context, providerType, apiKey, baseURL string) ([]Model, error)
}

type OllamaProvider

type OllamaProvider struct {
	*OpenAIProvider
}

func NewOllamaProvider

func NewOllamaProvider(baseURL string, models []Model) *OllamaProvider

func (*OllamaProvider) Complete

func (p *OllamaProvider) Complete(ctx context.Context, req Request) (Response, error)

func (*OllamaProvider) Stream

func (p *OllamaProvider) Stream(ctx context.Context, req Request) (<-chan Chunk, error)

type OpenAICompatProvider

type OpenAICompatProvider struct {
	*OpenAIProvider
}

func NewOpenAICompatProvider

func NewOpenAICompatProvider(name, baseURL, apiKey string, models []Model) *OpenAICompatProvider

type OpenAIProvider

type OpenAIProvider struct {
	*BaseProvider
}

func NewOpenAIProvider

func NewOpenAIProvider(apiKey string, models []Model) *OpenAIProvider

func (*OpenAIProvider) Complete

func (p *OpenAIProvider) Complete(ctx context.Context, req Request) (Response, error)

func (*OpenAIProvider) Stream

func (p *OpenAIProvider) Stream(ctx context.Context, req Request) (<-chan Chunk, error)

type Provider

type Provider interface {
	Name() string
	Complete(ctx context.Context, req Request) (Response, error)
	Stream(ctx context.Context, req Request) (<-chan Chunk, error)
	Models() []Model
}

func NewProvider

func NewProvider(cfg FactoryConfig) (Provider, error)

type ProviderDefaults

type ProviderDefaults struct {
	Type      string
	Name      string
	APIKeyEnv string
	BaseURL   string
	Method    ProviderMethod // "native" for non-OpenAI-compat, "openai-compat" otherwise
}

ProviderDefaults holds the canonical defaults for a provider type.

func CanonicalProviderDefaults

func CanonicalProviderDefaults() []ProviderDefaults

CanonicalProviderDefaults returns a copy of the canonical provider defaults.

func LookupProviderDefaults

func LookupProviderDefaults(providerType string) *ProviderDefaults

LookupProviderDefaults returns the full ProviderDefaults for a given type.

type ProviderMethod

type ProviderMethod string

ProviderMethod describes how the factory constructs the provider.

const (
	MethodNative       ProviderMethod = "native"
	MethodOpenAICompat ProviderMethod = "openai-compat"
)

type Request

type Request struct {
	Model         string
	Messages      []Message
	Tools         []Tool
	MaxTokens     int
	Temperature   float64
	SystemPrompt  string
	ThinkingLevel string // off|minimal|low|medium|high|x-high — passed through from config
	Metadata      map[string]any
}

type Response

type Response struct {
	ID        string
	Model     string
	Content   string
	ToolCalls []ToolCall
	Usage     Usage
	// UsageReported distinguishes a reported all-zero usage object from
	// omitted/null usage. OpenAI Chat Completions populates this explicitly;
	// older adapters may leave it false even when numeric usage is present.
	UsageReported bool
}

func WithRetry

func WithRetry(fn func() (*Response, error), isRetryable func(error) bool) (*Response, error)

WithRetry is a convenience wrapper that uses context.Background(). Prefer WithRetryCtx for cancellation support.

func WithRetryCtx

func WithRetryCtx(ctx context.Context, fn func() (*Response, error), isRetryable func(error) bool) (*Response, error)

type TOMLCost

type TOMLCost struct {
	Input      float64 `toml:"input"`
	Output     float64 `toml:"output"`
	CacheRead  float64 `toml:"cache_read"`
	CacheWrite float64 `toml:"cache_write"`
}

type TOMLExperimental

type TOMLExperimental struct {
	Modes map[string]TOMLExperimentalMode `toml:"modes"`
}

type TOMLExperimentalMode

type TOMLExperimentalMode struct {
	Cost     TOMLCost              `toml:"cost"`
	Provider *TOMLProviderOverride `toml:"provider,omitempty"`
}

type TOMLExtends

type TOMLExtends struct {
	From string `toml:"from"`
}

type TOMLLimit

type TOMLLimit struct {
	Context int `toml:"context"`
	Output  int `toml:"output"`
	Input   int `toml:"input"`
}

type TOMLModalities

type TOMLModalities struct {
	Input  []string `toml:"input"`
	Output []string `toml:"output"`
}

type TOMLModel

type TOMLModel struct {
	Name             string                `toml:"name"`
	Family           string                `toml:"family"`
	MaxTokens        int                   `toml:"max_tokens"`
	Reasoning        *bool                 `toml:"reasoning"`
	ToolCall         *bool                 `toml:"tool_call"`
	StructuredOutput *bool                 `toml:"structured_output"`
	Temperature      *bool                 `toml:"temperature"`
	Attachment       *bool                 `toml:"attachment"`
	OpenWeights      *bool                 `toml:"open_weights"`
	Modalities       TOMLModalities        `toml:"modalities"`
	Cost             TOMLCost              `toml:"cost"`
	Extends          *TOMLExtends          `toml:"extends"`
	ReleaseDate      string                `toml:"release_date"`
	LastUpdated      string                `toml:"last_updated"`
	Knowledge        string                `toml:"knowledge"`
	Status           string                `toml:"status"`
	Interleaved      any                   `toml:"interleaved"`
	Limit            TOMLLimit             `toml:"limit"`
	Experimental     *TOMLExperimental     `toml:"experimental,omitempty"`
	ProviderOverride *TOMLProviderOverride `toml:"provider,omitempty"`
}

type TOMLProviderOverride

type TOMLProviderOverride struct {
	NPM     string            `toml:"npm"`
	API     string            `toml:"api"`
	Body    map[string]any    `toml:"body"`
	Headers map[string]string `toml:"headers"`
	Shape   string            `toml:"shape"`
}

type TokenLimitResolver

type TokenLimitResolver func(model string) (contextWindow, outputTokens int, err error)

TokenLimitResolver resolves independent context and completion limits for a model.

func TokenLimitsFor

func TokenLimitsFor(models []Model, outputOverride int) TokenLimitResolver

TokenLimitsFor keeps the caller's output override separate from model metadata, so changing models can re-resolve both limits instead of inheriting old caps.

type Tool

type Tool struct {
	Name        string
	Description string
	Parameters  json.RawMessage
}

type ToolCall

type ToolCall struct {
	ID        string
	Name      string
	Arguments string
}

type Usage

type Usage struct {
	InputTokens       int
	OutputTokens      int
	CachedInputTokens int
}

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL