Documentation
¶
Overview ¶
Package model talks to LLM providers using a single OpenAI-shaped message format.
Index ¶
- Constants
- Variables
- func AttachmentInfo(images []Image) []map[string]any
- func AttachmentLabels(images []Image) string
- func CheckProviderEnv(spec string) error
- func ContextWindow(spec string) int
- func FormatBytes(n int) string
- func Hint(spec, errMsg string) string
- func InsecureBaseURL(d ProviderDef) bool
- func IsContextOverflow(err error) bool
- func IsRetryExhausted(err error) bool
- func MaxOutputTokensSupported(spec string) error
- func MissingKeyEnv(spec string) string
- func OllamaBaseURL() string
- func OpenAITransport(spec, setting string) string
- func PricePerMTok(spec string) (in, out float64, ok bool)
- func ReasoningChars(m Message) int
- func ResolveOpenAIAPI(setting, base string) string
- func SameProvider(a, b string) bool
- func SetMaxOutputTokens(m Model, spec string, n int) error
- func SetProviders(defs []ProviderDef)
- func SetReasoningEffort(m Model, spec, effort string) error
- func Truncated(m Message) (bool, string)
- func Unpriced(spec string) bool
- func ValidOpenAIAPI(v string) bool
- func ValidReasoningEffort(v string) bool
- func WithFallbackObserver(ctx context.Context, fn func(Switch)) context.Context
- func WithRetryObserver(ctx context.Context, fn func(Retry)) context.Context
- type Anthropic
- type Codex
- type Cost
- type Fallback
- type FallbackEntry
- type FunctionCall
- type FunctionSpec
- type Image
- type Message
- type Model
- type ModelMeta
- type OpenAI
- type OpenAIResponses
- type Options
- type ProviderDef
- type ProviderEnvError
- type Retry
- type RetryExhaustedError
- type Switch
- type ToolCall
- type ToolResult
- type ToolSpec
Constants ¶
const ( ProviderAPIChat = "chat" ProviderAPIResponses = "responses" ProviderAPIAnthropic = "anthropic" )
Wire protocols a configured provider can speak.
const ( // ExtraAnthropicContent holds every content block of an Anthropic // response that contained thinking, in order, for verbatim replay. ExtraAnthropicContent = "anthropic_content" // ExtraCodexReasoning holds the Responses API reasoning output items // (with encrypted_content), in order, for replay before the message. Both // codex and the openai Responses transport use it; the name predates the // latter and is kept so existing trajectories still replay. ExtraCodexReasoning = "codex_reasoning" )
Message.Extra keys holding provider reasoning state. They are written by the provider that produced the message and ignored by every other one.
const ( OpenAIAPIAuto = "auto" OpenAIAPIResponses = "responses" OpenAIAPIChat = "chat" )
The openai provider's transports, chosen by the openai_api setting.
const ( // ExtraTruncated is true on a response cut off at the output-token limit. ExtraTruncated = "truncated" // ExtraTruncationReason is the provider's raw reason, for logs. ExtraTruncationReason = "truncation_reason" )
Message.Extra keys for output-token truncation. A provider sets them when its documented stop signal says the response was cut off at the output limit, so the agent loop reads one flag whatever the provider.
const DefaultAnthropicMaxTokens = 8192
DefaultAnthropicMaxTokens is the max_tokens sent to Anthropic when max_output_tokens is unset.
const ExtraModel = "model"
ExtraModel is the Message.Extra key naming the model spec that produced an assistant message. It is set only on runs with a fallback chain.
const ImageTokenEstimate = 1600
ImageTokenEstimate is the fixed token estimate for one image in context accounting, instead of base64 length / 4. It is close to the cost of a ~1,000x1,000 pixel image on Anthropic and OpenAI.
const MaxFallbackModels = 3
MaxFallbackModels caps the fallback chain (fallback_models, --fallback-model).
const MaxTaskImages = 10
MaxTaskImages caps how many --image attachments one task message carries.
Variables ¶
var ContextWindows = map[string]int{
"gpt-4o": 128_000,
"gpt-4o-mini": 128_000,
"gpt-4.1": 1_047_576,
"gpt-4.1-mini": 1_047_576,
"gpt-4.1-nano": 1_047_576,
"claude-": 200_000,
}
ContextWindows maps a model name prefix to its context window in tokens. The longest matching prefix wins. Unknown models return 0, meaning the window isn't known.
var ImageMIMETypes = map[string]bool{"image/png": true, "image/jpeg": true, "image/gif": true, "image/webp": true}
ImageMIMETypes are the formats tinyagent sends to providers.
var OnCredentials func(values ...string)
OnCredentials, when set, receives credential values a provider loads from somewhere other than the environment, such as the codex provider's tokens (including ones refreshed during the run), for redaction.
var OpenAIAPIs = []string{OpenAIAPIAuto, OpenAIAPIResponses, OpenAIAPIChat}
OpenAIAPIs are the accepted openai_api values.
var PriceOverride [2]float64
PriceOverride, when non-zero, replaces the price table's input and output prices. The table's cache-read multiplier still applies.
var Prices = map[string][3]float64{
"gpt-4o": {2.50, 10.00},
"gpt-4o-mini": {0.15, 0.60},
"gpt-4.1": {2.00, 8.00},
"gpt-4.1-mini": {0.40, 1.60},
"gpt-4.1-nano": {0.10, 0.40},
"claude-sonnet-4": {3.00, 15.00},
"claude-opus-4": {15.00, 75.00},
"claude-haiku-4-5": {1.00, 5.00},
"claude-3-5-haiku": {0.80, 4.00},
"claude-3-7-sonnet": {3.00, 15.00},
"claude-sonnet-5-5": {2.00, 10.00, 0.05},
"claude-opus-5-5": {4.00, 20.00, 0.05},
"claude-fable-5-1": {10.00, 50.00, 0.025},
}
Prices maps a model name prefix to USD per million input/output tokens and, optionally, the cache-read price as a multiple of the input price (0 means the standard 0.1). The longest matching prefix wins. Unknown models cost 0 unless overridden.
var ReasoningEfforts = []string{"none", "minimal", "low", "medium", "high", "xhigh", "max"}
ReasoningEfforts are the accepted reasoning_effort values. Support varies by model; unsupported values are reported by the provider.
Functions ¶
func AttachmentInfo ¶ added in v0.5.0
AttachmentInfo is an attachment's metadata for events and logs: index, base name, MIME type, size, and sha256. It never contains image data.
func AttachmentLabels ¶ added in v0.5.0
AttachmentLabels is the text added to a user message for its task attachments: a blank line, then one "[attached image N: name, mime, bytes]" line per image. It is "" without images, so image-free messages are unchanged.
func CheckProviderEnv ¶ added in v0.5.0
CheckProviderEnv reports the first variable that the configured provider of spec needs but that is unset or empty, or nil (also for built-in providers).
func ContextWindow ¶ added in v0.4.0
ContextWindow returns the context window for spec, a "<provider>/<model>" name as given to New, or 0 when it isn't in ContextWindows. For a nested name such as openrouter/anthropic/claude-x, the last segment is tried too. A configured provider's model metadata, matched by exact name, comes before the table.
func FormatBytes ¶ added in v0.5.0
FormatBytes adds thousands separators: 48213 -> 48,213.
func Hint ¶
Hint returns a one-line setup fix for a failed run of the model spec, given the error text, or "" for errors that aren't a common setup problem. It names environment variables but never prints their values.
func InsecureBaseURL ¶ added in v0.5.0
func InsecureBaseURL(d ProviderDef) bool
InsecureBaseURL reports whether d sends requests over plain http to a host other than loopback.
func IsContextOverflow ¶ added in v0.4.0
IsContextOverflow reports whether err is a provider's "request is larger than the context window" error.
func IsRetryExhausted ¶ added in v0.5.0
IsRetryExhausted reports whether err is a request that still failed with a transient cause after every retry.
func MaxOutputTokensSupported ¶ added in v0.5.0
MaxOutputTokensSupported reports whether the provider of spec accepts a per-request output-token budget, and returns an error naming it if not.
func MissingKeyEnv ¶ added in v0.5.0
MissingKeyEnv returns the API key variable a built-in provider's spec needs when it is unset, or "" when it is set or not needed. An openai spec with OPENAI_BASE_URL set may target a server without keys, so it is never reported.
func OllamaBaseURL ¶ added in v0.5.0
func OllamaBaseURL() string
OllamaBaseURL returns the Ollama server URL that OLLAMA_HOST resolves to.
func OpenAITransport ¶ added in v0.5.0
OpenAITransport returns the transport New would use for spec with setting, reading OPENAI_BASE_URL, or "" when spec's provider isn't openai (the setting only applies to it).
func PricePerMTok ¶ added in v0.7.0
PricePerMTok returns the built-in or configured USD price per million input and output tokens for spec. ok is false when the model is unpriced.
func ReasoningChars ¶ added in v0.5.0
ReasoningChars approximates the size of the reasoning state stored in m, which is sent back with the history.
func ResolveOpenAIAPI ¶ added in v0.5.0
ResolveOpenAIAPI returns the transport, "responses" or "chat", that the openai provider uses for setting (an openai_api value; "" means auto) with base, the value of OPENAI_BASE_URL ("" when unset). auto picks Responses only for OpenAI's own API, so OpenAI-compatible servers keep Chat Completions.
func SameProvider ¶ added in v0.5.0
SameProvider reports whether two model specs use the same provider; a spec without a provider prefix is an openai model.
func SetMaxOutputTokens ¶ added in v0.5.0
SetMaxOutputTokens configures m, built by New(spec, ...), to send n as its provider's per-request output-token limit. n == 0 leaves m unchanged (the provider default; 8192 for Anthropic).
func SetProviders ¶ added in v0.5.0
func SetProviders(defs []ProviderDef)
SetProviders replaces the configured providers. Definitions are assumed validated (see config.Load).
func SetReasoningEffort ¶ added in v0.5.0
SetReasoningEffort configures m, built by New(spec, ...), to send effort using its provider's native control. An empty effort leaves m unchanged.
func Truncated ¶ added in v0.5.0
Truncated reports whether m was cut off at the output-token limit, and the provider's raw reason.
func Unpriced ¶ added in v0.4.0
Unpriced reports whether spec, as passed to New, names a model billed per token whose cost tinyagent can't estimate: no Prices entry and no PriceOverride, so its cost always reads 0. Ollama and Codex are free by design and never unpriced.
func ValidOpenAIAPI ¶ added in v0.5.0
ValidOpenAIAPI reports whether v is an accepted openai_api value. "" means auto.
func ValidReasoningEffort ¶ added in v0.5.0
ValidReasoningEffort reports whether v is an accepted reasoning_effort.
func WithFallbackObserver ¶ added in v0.5.0
WithFallbackObserver returns a context whose Fallback queries report each switch to fn before re-sending the request. fn runs on the requesting goroutine.
Types ¶
type Anthropic ¶
type Anthropic struct {
BaseURL string
APIKey string
Model string
MaxTokens int // defaults to DefaultAnthropicMaxTokens
OnText func(string) // if set, stream and report text deltas
// ReasoningEffort, if set, is sent as output_config.effort.
ReasoningEffort string
// Headers, if set, are added to every request after the defaults
// (configured providers).
Headers map[string]string
// Price, if set, replaces the Prices table entry for this model
// (configured providers); PriceOverride still wins.
Price *[3]float64
// OmitEmptyKey sends no x-api-key header when APIKey is empty
// (configured providers without api_key_env).
OmitEmptyKey bool
}
Anthropic speaks the native Messages API.
type Codex ¶
type Codex struct {
BaseURL string
TokenURL string
AuthFile string
Model string
OnText func(string)
// ReasoningEffort, if set, is sent as reasoning.effort.
ReasoningEffort string
// contains filtered or unexported fields
}
Codex uses a ChatGPT subscription instead of an API key. It signs in with the tokens `codex login` saves in $CODEX_HOME/auth.json, refreshes them when they expire (writing them back so the Codex CLI stays signed in), and speaks the Responses API that the Codex backend serves.
type Cost ¶
type Cost struct {
USD float64 `json:"usd"`
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
CacheReadTokens int `json:"cache_read_tokens,omitempty"`
CacheWriteTokens int `json:"cache_write_tokens,omitempty"`
}
Cost is one response's usage. InputTokens counts every input token, including the cache reads and writes broken out after it.
type Fallback ¶ added in v0.5.0
type Fallback struct {
Entries []FallbackEntry
// contains filtered or unexported fields
}
Fallback is a Model that sends each request to the active entry and, when that request still fails with a transient cause after the provider's own retries (a RetryExhaustedError), switches to the next entry and re-sends the same request. Other errors are returned as they are. After a switch, later requests stay on the model that answered for the rest of the invocation. Entries[0] is the primary model.
func NewFallback ¶ added in v0.5.0
func NewFallback(primary FallbackEntry, fallbacks ...FallbackEntry) *Fallback
NewFallback returns a Fallback over primary followed by fallbacks.
type FallbackEntry ¶ added in v0.5.0
FallbackEntry is one model of a fallback chain: its spec as given to New and the Model built from it.
type FunctionCall ¶
type FunctionSpec ¶
type Image ¶ added in v0.5.0
type Image struct {
MIMEType string `json:"mime_type"`
Data string `json:"data"`
// Label names the source for elision notes: a workspace path or an MCP
// tool name.
Label string `json:"label,omitempty"`
}
Image is one image attached to a tool result (opt-in --images) or to a user task message (--image). Data is the base64 encoding of the raw bytes. Images travel only in provider requests and trajectories; every text channel (progress log, --json, the SQLite log, hooks) sees Placeholder instead.
func (Image) Placeholder ¶ added in v0.5.0
Placeholder is the text form shown in place of the image, e.g. "[image: image/png, 48,213 bytes, sha256:ab12cd34ef56]". It never contains image data.
type Message ¶
type Message struct {
Role string `json:"role"`
Content string `json:"content,omitempty"`
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
ToolCallID string `json:"tool_call_id,omitempty"`
// Images are tool-result images (opt-in --images) or user task
// attachments (--image), sent in each provider's image form. Content
// already holds a placeholder or label line for each one.
Images []Image `json:"images,omitempty"`
Extra map[string]any `json:"extra,omitempty"`
}
func ForModel ¶ added in v0.5.0
ForModel returns messages as they may be sent to spec: provider-specific reasoning state (Anthropic thinking blocks, Responses/Codex reasoning items) is kept only on assistant messages produced by the same provider. A message without an ExtraModel tag counts as produced by primary. The input is not modified; messages without foreign state are shared.
type Model ¶
type Model interface {
Query(ctx context.Context, messages []Message, tools []ToolSpec) (Message, Cost, error)
}
type ModelMeta ¶ added in v0.5.0
type ModelMeta struct {
InputPricePerMTok float64
OutputPricePerMTok float64
CacheReadMultiplier float64 // 0 means the standard 0.1
ContextWindow int
}
ModelMeta is per-model price and context-window metadata for a configured provider. Zero values mean unknown.
type OpenAI ¶
type OpenAI struct {
BaseURL string
APIKey string
Model string
OnText func(string) // if set, stream and report text deltas
// ReasoningEffort, if set, is sent as reasoning_effort, or as
// reasoning.effort when NestedReasoning is set (OpenRouter).
ReasoningEffort string
NestedReasoning bool
// MaxOutputTokens, if > 0, is sent under MaxOutputKey
// (max_completion_tokens, or max_tokens for Ollama).
MaxOutputTokens int
MaxOutputKey string
// Headers, if set, are added to every request after the defaults
// (configured providers).
Headers map[string]string
// Price, if set, replaces the Prices table entry for this model
// (configured providers); PriceOverride still wins.
Price *[3]float64
}
OpenAI speaks the Chat Completions API, which also covers OpenRouter, Ollama, and other OpenAI-compatible endpoints.
type OpenAIResponses ¶ added in v0.5.0
type OpenAIResponses struct {
BaseURL string
APIKey string
Model string
OnText func(string) // if set, stream and report text deltas
// ReasoningEffort, if set, is sent as reasoning.effort.
ReasoningEffort string
// MaxOutputTokens, if > 0, is sent as max_output_tokens.
MaxOutputTokens int
// Headers, if set, are added to every request after the defaults
// (configured providers).
Headers map[string]string
// Price, if set, replaces the Prices table entry for this model
// (configured providers); PriceOverride still wins.
Price *[3]float64
// contains filtered or unexported fields
}
OpenAIResponses speaks OpenAI's Responses API (POST /responses) with an API key. It is stateless (store: false) and keeps reasoning items that carry encrypted_content for replay, so tool calls and reasoning work together.
type Options ¶ added in v0.5.0
type Options struct {
// OpenAIAPI is the openai provider's transport: auto (or ""), responses,
// or chat. Other providers ignore it.
OpenAIAPI string
}
Options are provider settings beyond the model spec.
type ProviderDef ¶ added in v0.5.0
type ProviderDef struct {
ID string
API string
BaseURL string
APIKeyEnv string
Headers map[string]string
EnvHeaders map[string]string
Models map[string]ModelMeta
}
ProviderDef is a config-defined provider. APIKeyEnv and EnvHeaders name environment variables; their values are read only when a model is built.
func ConfiguredProvider ¶ added in v0.5.0
func ConfiguredProvider(spec string) (ProviderDef, bool)
ConfiguredProvider returns the configured provider spec resolves to, if any. Built-in names always resolve to the built-in provider.
type ProviderEnvError ¶ added in v0.5.0
type ProviderEnvError struct {
Provider, Env, Use string
}
ProviderEnvError reports a missing environment variable a configured provider needs. It names the variable, never a value.
func (*ProviderEnvError) Error ¶ added in v0.5.0
func (e *ProviderEnvError) Error() string
type Retry ¶ added in v0.5.0
type Retry struct {
Attempt int // the attempt about to be made, starting at 2
Reason string // "http_<status>" or "network"; never a response body
Backoff time.Duration // wait before the attempt
}
Retry describes one provider request about to be retried.
type RetryExhaustedError ¶ added in v0.5.0
type RetryExhaustedError struct {
Attempts int
Reason string // "http_<status>" or "network"; never a response body
Err error // the last attempt's error
}
RetryExhaustedError is returned when a request still failed with a transient cause (HTTP 429, 5xx or a network error) after every attempt. Errors returned without retrying (other 4xx) are never this type, so errors.As tells an outage apart from a setup or request problem.
func (*RetryExhaustedError) Error ¶ added in v0.5.0
func (e *RetryExhaustedError) Error() string
func (*RetryExhaustedError) Unwrap ¶ added in v0.5.0
func (e *RetryExhaustedError) Unwrap() error
type Switch ¶ added in v0.5.0
type Switch struct {
From string // model spec that failed
To string // model spec the request is re-sent to
Reason string // "http_<status>" or "network"; never a response body
Attempts int // attempts made on From
}
Switch describes one move down the fallback chain.
type ToolCall ¶
type ToolCall struct {
ID string `json:"id"`
Type string `json:"type"`
Function FunctionCall `json:"function"`
}
type ToolResult ¶
type ToolSpec ¶
type ToolSpec struct {
Type string `json:"type"`
Function FunctionSpec `json:"function"`
}
ToolSpec is an OpenAI function-calling tool definition.