Documentation
¶
Overview ¶
Package model talks to LLM providers using a single OpenAI-shaped message format.
Index ¶
- Constants
- Variables
- func AttachmentInfo(images []Image) []map[string]any
- func AttachmentLabels(images []Image) string
- func CheckProviderEnv(spec string) error
- func ContextWindow(spec string) int
- func DefaultPrices() map[string][3]float64
- func FormatBytes(n int) string
- func Hint(spec, errMsg string) string
- func InsecureBaseURL(d ProviderDef) bool
- func IsContextOverflow(err error) bool
- func IsRetryExhausted(err error) bool
- func MaxOutputTokensSupported(spec string) error
- func MissingKeyEnv(spec string) string
- func OllamaBaseURL() string
- func OpenAITransport(spec, setting string) string
- func OpenCodeModelID(name string) string
- func PricePerMTok(spec string) (in, out float64, ok bool)
- func ReasoningChars(m Message) int
- func ResolveOpenAIAPI(setting, base string) string
- func SameProvider(a, b string) bool
- func SetCLIPaths(claude, codex string)
- func SetMaxOutputTokens(m Model, spec string, n int) error
- func SetOpenCodePath(p string)
- func SetProviders(defs []ProviderDef)
- func SetReasoningEffort(m Model, spec, effort string) error
- func Truncated(m Message) (bool, string)
- func Unpriced(spec string) bool
- func ValidOpenAIAPI(v string) bool
- func ValidReasoningEffort(v string) bool
- func WithFallbackObserver(ctx context.Context, fn func(Switch)) context.Context
- func WithRetryObserver(ctx context.Context, fn func(Retry)) context.Context
- type Anthropic
- type ClaudeCode
- type Codex
- type CodexExec
- type Cost
- type Fallback
- type FallbackEntry
- type FunctionCall
- type FunctionSpec
- type HandoffOptions
- type Image
- type Message
- type MissingCLIError
- type Model
- type ModelMeta
- type OpenAI
- type OpenAIResponses
- type OpenCode
- type Options
- type ProviderDef
- type ProviderEnvError
- type Retry
- type RetryExhaustedError
- type Runtime
- type Switch
- type ToolCall
- type ToolResult
- type ToolSpec
Constants ¶
const ( ProviderAPIChat = "chat" ProviderAPIResponses = "responses" ProviderAPIAnthropic = "anthropic" )
Wire protocols a configured provider can speak.
const ( // ExtraAnthropicContent holds every content block of an Anthropic // response that contained thinking, in order, for verbatim replay. ExtraAnthropicContent = "anthropic_content" // ExtraCodexReasoning holds the Responses API reasoning output items // (with encrypted_content), in order, for replay before the message. Both // codex and the openai Responses transport use it; the name predates the // latter and is kept so existing trajectories still replay. ExtraCodexReasoning = "codex_reasoning" )
Message.Extra keys holding provider reasoning state. They are written by the provider that produced the message and ignored by every other one.
const ( OpenAIAPIAuto = "auto" OpenAIAPIResponses = "responses" OpenAIAPIChat = "chat" )
The openai provider's transports, chosen by the openai_api setting.
const ( // ExtraTruncated is true on a response cut off at the output-token limit. ExtraTruncated = "truncated" // ExtraTruncationReason is the provider's raw reason, for logs. ExtraTruncationReason = "truncation_reason" )
Message.Extra keys for output-token truncation. A provider sets them when its documented stop signal says the response was cut off at the output limit, so the agent loop reads one flag whatever the provider.
const ClaudeCodeProvider = "claude-code"
ClaudeCodeProvider is the provider id of the Claude Code handoff.
const CodexExecProvider = "codex-exec"
CodexExecProvider is the provider id of the codex exec handoff.
const DefaultAnthropicMaxTokens = 8192
DefaultAnthropicMaxTokens is the max_tokens sent to Anthropic when max_output_tokens is unset.
const ExtraModel = "model"
ExtraModel is the Message.Extra key naming the model spec that produced an assistant message. It is set only on runs with a fallback chain.
const HandoffDefaultModel = "default"
HandoffDefaultModel as the model name sends no --model flag, so the CLI's own configured model applies.
const ImageTokenEstimate = 1600
ImageTokenEstimate is the fixed token estimate for one image in context accounting, instead of base64 length / 4. It is close to the cost of a ~1,000x1,000 pixel image on Anthropic and OpenAI.
const MaxFallbackModels = 3
MaxFallbackModels caps the fallback chain (fallback_models, --fallback-model).
const MaxTaskImages = 10
MaxTaskImages caps how many --image attachments one task message carries.
const OpenCodeProvider = "opencode"
OpenCodeProvider is the provider id of the opencode run handoff.
const OpenCodeZenProvider = "opencode"
OpenCodeZenProvider is OpenCode's own Zen provider id.
Variables ¶
var ContextWindows = map[string]int{
"gpt-4o": 128_000,
"gpt-4o-mini": 128_000,
"gpt-4.1": 1_047_576,
"gpt-4.1-mini": 1_047_576,
"gpt-4.1-nano": 1_047_576,
"claude-": 200_000,
}
ContextWindows maps a model name prefix to its context window in tokens. The longest matching prefix wins. Unknown models return 0, meaning the window isn't known.
var DefaultRuntime = NewRuntime()
DefaultRuntime is used whenever Options.Runtime (or an adapter's Runtime field) is nil. It is process-wide: the CLI sets its PriceOverride from config once per run. Tests should build their own Runtime instead of changing it.
var ImageMIMETypes = map[string]bool{"image/png": true, "image/jpeg": true, "image/gif": true, "image/webp": true}
ImageMIMETypes are the formats tinyagent sends to providers.
var OnCredentials func(values ...string)
OnCredentials, when set, receives credential values a provider loads from somewhere other than the environment, such as the codex provider's tokens (including ones refreshed during the run), for redaction.
var OpenAIAPIs = []string{OpenAIAPIAuto, OpenAIAPIResponses, OpenAIAPIChat}
OpenAIAPIs are the accepted openai_api values.
var ReasoningEfforts = []string{"none", "minimal", "low", "medium", "high", "xhigh", "max"}
ReasoningEfforts are the accepted reasoning_effort values. Support varies by model; unsupported values are reported by the provider.
Functions ¶
func AttachmentInfo ¶ added in v0.5.0
AttachmentInfo is an attachment's metadata for events and logs: index, base name, MIME type, size, and sha256. It never contains image data.
func AttachmentLabels ¶ added in v0.5.0
AttachmentLabels is the text added to a user message for its task attachments: a blank line, then one "[attached image N: name, mime, bytes]" line per image. It is "" without images, so image-free messages are unchanged.
func CheckProviderEnv ¶ added in v0.5.0
CheckProviderEnv reports the first variable that the configured provider of spec needs but that is unset or empty, or nil (also for built-in providers).
func ContextWindow ¶ added in v0.4.0
ContextWindow returns the context window for spec, a "<provider>/<model>" name as given to New, or 0 when it isn't in ContextWindows. For a nested name such as openrouter/anthropic/claude-x, the last segment is tried too. A configured provider's model metadata, matched by exact name, comes before the table.
func DefaultPrices ¶ added in v0.15.0
DefaultPrices returns a copy of the built-in price table.
func FormatBytes ¶ added in v0.5.0
FormatBytes adds thousands separators: 48213 -> 48,213.
func Hint ¶
Hint returns a one-line setup fix for a failed run of the model spec, given the error text, or "" for errors that aren't a common setup problem. It names environment variables but never prints their values.
func InsecureBaseURL ¶ added in v0.5.0
func InsecureBaseURL(d ProviderDef) bool
InsecureBaseURL reports whether d sends requests over plain http to a host other than loopback.
func IsContextOverflow ¶ added in v0.4.0
IsContextOverflow reports whether err is a provider's "request is larger than the context window" error.
func IsRetryExhausted ¶ added in v0.5.0
IsRetryExhausted reports whether err is a request that still failed with a transient cause after every retry.
func MaxOutputTokensSupported ¶ added in v0.5.0
MaxOutputTokensSupported reports whether the provider of spec accepts a per-request output-token budget, and returns an error naming it if not.
func MissingKeyEnv ¶ added in v0.5.0
MissingKeyEnv returns the API key variable a built-in provider's spec needs when it is unset, or "" when it is set or not needed. An openai spec with OPENAI_BASE_URL set may target a server without keys, so it is never reported.
func OllamaBaseURL ¶ added in v0.5.0
func OllamaBaseURL() string
OllamaBaseURL returns the Ollama server URL that OLLAMA_HOST resolves to.
func OpenAITransport ¶ added in v0.5.0
OpenAITransport returns the transport New would use for spec with setting, reading OPENAI_BASE_URL, or "" when spec's provider isn't openai (the setting only applies to it).
func OpenCodeModelID ¶ added in v0.16.0
OpenCodeModelID maps the part of a tinyagent model name after "opencode/" to OpenCode's --model value: "" and "default" stay as they are, a bare name is a Zen model ("big-pickle" -> "opencode/big-pickle"), and "provider/model" is passed unchanged.
func PricePerMTok ¶ added in v0.7.0
PricePerMTok returns the built-in or configured USD price per million input and output tokens for spec under DefaultRuntime. ok is false when the model is unpriced.
func ReasoningChars ¶ added in v0.5.0
ReasoningChars approximates the size of the reasoning state stored in m, which is sent back with the history.
func ResolveOpenAIAPI ¶ added in v0.5.0
ResolveOpenAIAPI returns the transport, "responses" or "chat", that the openai provider uses for setting (an openai_api value; "" means auto) with base, the value of OPENAI_BASE_URL ("" when unset). auto picks Responses only for OpenAI's own API, so OpenAI-compatible servers keep Chat Completions.
func SameProvider ¶ added in v0.5.0
SameProvider reports whether two model specs use the same provider; a spec without a provider prefix is an openai model.
func SetCLIPaths ¶ added in v0.12.0
func SetCLIPaths(claude, codex string)
SetCLIPaths sets the configured binary paths of the claude-code and codex-exec providers; "" means look the binary up on PATH.
func SetMaxOutputTokens ¶ added in v0.5.0
SetMaxOutputTokens configures m, built by New(spec, ...), to send n as its provider's per-request output-token limit. n == 0 leaves m unchanged (the provider default; 8192 for Anthropic).
func SetOpenCodePath ¶ added in v0.14.0
func SetOpenCodePath(p string)
SetOpenCodePath sets the configured binary path of the opencode provider; "" means look it up on PATH. Call it after SetCLIPaths.
func SetProviders ¶ added in v0.5.0
func SetProviders(defs []ProviderDef)
SetProviders replaces the configured providers. Definitions are assumed validated (see config.Load).
func SetReasoningEffort ¶ added in v0.5.0
SetReasoningEffort configures m, built by New(spec, ...), to send effort using its provider's native control. An empty effort leaves m unchanged.
func Truncated ¶ added in v0.5.0
Truncated reports whether m was cut off at the output-token limit, and the provider's raw reason.
func Unpriced ¶ added in v0.4.0
Unpriced reports whether spec, as passed to New, names a model billed per token whose cost tinyagent can't estimate under DefaultRuntime. See Runtime.Unpriced.
func ValidOpenAIAPI ¶ added in v0.5.0
ValidOpenAIAPI reports whether v is an accepted openai_api value. "" means auto.
func ValidReasoningEffort ¶ added in v0.5.0
ValidReasoningEffort reports whether v is an accepted reasoning_effort.
func WithFallbackObserver ¶ added in v0.5.0
WithFallbackObserver returns a context whose Fallback queries report each switch to fn before re-sending the request. fn runs on the requesting goroutine.
Types ¶
type Anthropic ¶
type Anthropic struct {
BaseURL string
APIKey string
Model string
MaxTokens int // defaults to DefaultAnthropicMaxTokens
OnText func(string) // if set, stream and report text deltas
// ReasoningEffort, if set, is sent as output_config.effort.
ReasoningEffort string
// Headers, if set, are added to every request after the defaults
// (configured providers).
Headers map[string]string
// Price, if set, replaces the Runtime's price table entry for this
// model (configured providers); the Runtime's PriceOverride still wins.
Price *[3]float64
// Runtime supplies the HTTP client, retry policy and prices. nil uses
// DefaultRuntime.
Runtime *Runtime
// OmitEmptyKey sends no x-api-key header when APIKey is empty
// (configured providers without api_key_env).
OmitEmptyKey bool
}
Anthropic speaks the native Messages API.
type ClaudeCode ¶ added in v0.12.0
type ClaudeCode struct {
Path string // resolved claude binary
Model string // "" or HandoffDefaultModel sends no --model
HandoffOptions
}
ClaudeCode runs `claude -p` for each query.
func NewClaudeCode ¶ added in v0.12.0
func NewClaudeCode(name string, opts HandoffOptions) (*ClaudeCode, error)
NewClaudeCode resolves the claude binary for model name.
func (*ClaudeCode) Args ¶ added in v0.12.0
func (c *ClaudeCode) Args() []string
Args returns the claude arguments for one run.
type Codex ¶
type Codex struct {
BaseURL string
TokenURL string
AuthFile string
Model string
OnText func(string)
// ReasoningEffort, if set, is sent as reasoning.effort.
ReasoningEffort string
// Runtime supplies the HTTP client, retry policy and prices. nil uses
// DefaultRuntime.
Runtime *Runtime
// contains filtered or unexported fields
}
Codex uses a ChatGPT subscription instead of an API key. It signs in with the tokens `codex login` saves in $CODEX_HOME/auth.json, refreshes them when they expire (writing them back so the Codex CLI stays signed in), and speaks the Responses API that the Codex backend serves.
type CodexExec ¶ added in v0.12.0
type CodexExec struct {
Path string // resolved codex binary
Model string // "" or HandoffDefaultModel sends no -m
HandoffOptions
}
CodexExec runs `codex exec --json` for each query.
func NewCodexExec ¶ added in v0.12.0
func NewCodexExec(name string, opts HandoffOptions) (*CodexExec, error)
NewCodexExec resolves the codex binary for model name.
type Cost ¶
type Cost struct {
USD float64 `json:"usd"`
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
CacheReadTokens int `json:"cache_read_tokens,omitempty"`
CacheWriteTokens int `json:"cache_write_tokens,omitempty"`
}
Cost is one response's usage. InputTokens counts every input token, including the cache reads and writes broken out after it.
type Fallback ¶ added in v0.5.0
type Fallback struct {
Entries []FallbackEntry
// contains filtered or unexported fields
}
Fallback is a Model that sends each request to the active entry and, when that request still fails with a transient cause after the provider's own retries (a RetryExhaustedError), switches to the next entry and re-sends the same request. Other errors are returned as they are. After a switch, later requests stay on the model that answered for the rest of the invocation. Entries[0] is the primary model.
func NewFallback ¶ added in v0.5.0
func NewFallback(primary FallbackEntry, fallbacks ...FallbackEntry) *Fallback
NewFallback returns a Fallback over primary followed by fallbacks.
type FallbackEntry ¶ added in v0.5.0
FallbackEntry is one model of a fallback chain: its spec as given to New and the Model built from it.
type FunctionCall ¶
type FunctionSpec ¶
type HandoffOptions ¶ added in v0.12.0
type HandoffOptions struct {
// Dir is the working directory the CLI runs in (--cwd). "" keeps
// tinyagent's own working directory.
Dir string
// ReadOnly maps --read-only/read_only onto the CLI's own restriction.
ReadOnly bool
}
HandoffOptions are the TinyAgent settings a handoff provider maps onto the CLI it runs.
type Image ¶ added in v0.5.0
type Image struct {
MIMEType string `json:"mime_type"`
Data string `json:"data"`
// Label names the source for elision notes: a workspace path or an MCP
// tool name.
Label string `json:"label,omitempty"`
}
Image is one image attached to a tool result (opt-in --images) or to a user task message (--image). Data is the base64 encoding of the raw bytes. Images travel only in provider requests and trajectories; every text channel (progress log, --json, the SQLite log, hooks) sees Placeholder instead.
func (Image) Placeholder ¶ added in v0.5.0
Placeholder is the text form shown in place of the image, e.g. "[image: image/png, 48,213 bytes, sha256:ab12cd34ef56]". It never contains image data.
type Message ¶
type Message struct {
Role string `json:"role"`
Content string `json:"content,omitempty"`
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
ToolCallID string `json:"tool_call_id,omitempty"`
// Images are tool-result images (opt-in --images) or user task
// attachments (--image), sent in each provider's image form. Content
// already holds a placeholder or label line for each one.
Images []Image `json:"images,omitempty"`
Extra map[string]any `json:"extra,omitempty"`
}
func ForModel ¶ added in v0.5.0
ForModel returns messages as they may be sent to spec: provider-specific reasoning state (Anthropic thinking blocks, Responses/Codex reasoning items) is kept only on assistant messages produced by the same provider. A message without an ExtraModel tag counts as produced by primary. The input is not modified; messages without foreign state are shared.
type MissingCLIError ¶ added in v0.12.0
type MissingCLIError struct {
Provider, Binary, Configured, Install, SignIn, Docs string
}
MissingCLIError is returned when a handoff provider's CLI isn't found.
func (*MissingCLIError) Error ¶ added in v0.12.0
func (e *MissingCLIError) Error() string
type Model ¶
type Model interface {
Query(ctx context.Context, messages []Message, tools []ToolSpec) (Message, Cost, error)
}
type ModelMeta ¶ added in v0.5.0
type ModelMeta struct {
InputPricePerMTok float64
OutputPricePerMTok float64
CacheReadMultiplier float64 // 0 means the standard 0.1
ContextWindow int
}
ModelMeta is per-model price and context-window metadata for a configured provider. Zero values mean unknown.
type OpenAI ¶
type OpenAI struct {
BaseURL string
APIKey string
Model string
OnText func(string) // if set, stream and report text deltas
// ReasoningEffort, if set, is sent as reasoning_effort, or as
// reasoning.effort when NestedReasoning is set (OpenRouter).
ReasoningEffort string
NestedReasoning bool
// MaxOutputTokens, if > 0, is sent under MaxOutputKey
// (max_completion_tokens, or max_tokens for Ollama).
MaxOutputTokens int
MaxOutputKey string
// Headers, if set, are added to every request after the defaults
// (configured providers).
Headers map[string]string
// Price, if set, replaces the Runtime's price table entry for this
// model (configured providers); the Runtime's PriceOverride still wins.
Price *[3]float64
// Runtime supplies the HTTP client, retry policy and prices. nil uses
// DefaultRuntime.
Runtime *Runtime
}
OpenAI speaks the Chat Completions API, which also covers OpenRouter, Ollama, and other OpenAI-compatible endpoints.
type OpenAIResponses ¶ added in v0.5.0
type OpenAIResponses struct {
BaseURL string
APIKey string
Model string
OnText func(string) // if set, stream and report text deltas
// ReasoningEffort, if set, is sent as reasoning.effort.
ReasoningEffort string
// MaxOutputTokens, if > 0, is sent as max_output_tokens.
MaxOutputTokens int
// Headers, if set, are added to every request after the defaults
// (configured providers).
Headers map[string]string
// Price, if set, replaces the Runtime's price table entry for this
// model (configured providers); the Runtime's PriceOverride still wins.
Price *[3]float64
// Runtime supplies the HTTP client, retry policy and prices. nil uses
// DefaultRuntime.
Runtime *Runtime
// contains filtered or unexported fields
}
OpenAIResponses speaks OpenAI's Responses API (POST /responses) with an API key. It is stateless (store: false) and keeps reasoning items that carry encrypted_content for replay, so tool calls and reasoning work together.
type OpenCode ¶ added in v0.14.0
type OpenCode struct {
Path string // resolved opencode binary
Model string // "" or HandoffDefaultModel sends no --model
HandoffOptions
}
OpenCode runs `opencode run --format json` for each query.
func NewOpenCode ¶ added in v0.14.0
func NewOpenCode(name string, opts HandoffOptions) (*OpenCode, error)
NewOpenCode resolves the opencode binary for model name.
func (*OpenCode) Args ¶ added in v0.14.0
Args returns the opencode arguments for one run with prompt as the message. The working directory is not a flag: the process runs in c.Dir.
func (*OpenCode) Env ¶ added in v0.14.0
Env returns the extra environment for one run (nil when none).
type Options ¶ added in v0.5.0
type Options struct {
// OpenAIAPI is the openai provider's transport: auto (or ""), responses,
// or chat. Other providers ignore it.
OpenAIAPI string
// Handoff is what the claude-code, codex-exec and opencode providers map onto
// the CLI they run. Other providers ignore it.
Handoff HandoffOptions
// Runtime supplies the HTTP client, retry policy and prices for the
// HTTP providers. nil uses DefaultRuntime.
Runtime *Runtime
}
Options are provider settings beyond the model spec.
type ProviderDef ¶ added in v0.5.0
type ProviderDef struct {
ID string
API string
BaseURL string
APIKeyEnv string
Headers map[string]string
EnvHeaders map[string]string
Models map[string]ModelMeta
}
ProviderDef is a config-defined provider. APIKeyEnv and EnvHeaders name environment variables; their values are read only when a model is built.
func ConfiguredProvider ¶ added in v0.5.0
func ConfiguredProvider(spec string) (ProviderDef, bool)
ConfiguredProvider returns the configured provider spec resolves to, if any. Built-in names always resolve to the built-in provider.
type ProviderEnvError ¶ added in v0.5.0
type ProviderEnvError struct {
Provider, Env, Use string
}
ProviderEnvError reports a missing environment variable a configured provider needs. It names the variable, never a value.
func (*ProviderEnvError) Error ¶ added in v0.5.0
func (e *ProviderEnvError) Error() string
type Retry ¶ added in v0.5.0
type Retry struct {
Attempt int // the attempt about to be made, starting at 2
Reason string // "http_<status>" or "network"; never a response body
Backoff time.Duration // wait before the attempt
}
Retry describes one provider request about to be retried.
type RetryExhaustedError ¶ added in v0.5.0
type RetryExhaustedError struct {
Attempts int
Reason string // "http_<status>" or "network"; never a response body
Err error // the last attempt's error
}
RetryExhaustedError is returned when a request still failed with a transient cause (HTTP 429, 5xx or a network error) after every attempt. Errors returned without retrying (other 4xx) are never this type, so errors.As tells an outage apart from a setup or request problem.
func (*RetryExhaustedError) Error ¶ added in v0.5.0
func (e *RetryExhaustedError) Error() string
func (*RetryExhaustedError) Unwrap ¶ added in v0.5.0
func (e *RetryExhaustedError) Unwrap() error
type Runtime ¶ added in v0.15.0
type Runtime struct {
// HTTP sends provider requests. nil uses a client with a 10-minute
// timeout.
HTTP *http.Client
// RetryBase is the first retry's backoff; each later retry doubles it
// (plus jitter). Zero or negative uses 1s.
RetryBase time.Duration
// MaxAttempts is the total number of tries for a transient failure
// (HTTP 429, 5xx or a network error). Zero or negative uses 6.
MaxAttempts int
// Prices maps a model name prefix to USD per million input/output tokens
// and, optionally, the cache-read price as a multiple of the input price
// (0 means the standard 0.1). The longest matching prefix wins. nil uses
// the built-in table; a non-nil empty map prices every model at 0.
Prices map[string][3]float64
// PriceOverride, when non-zero, replaces the price table's input and
// output prices. The table's cache-read multiplier still applies.
PriceOverride [2]float64
}
Runtime holds the provider settings that used to be package-level globals: the HTTP client, the retry policy, and the price table. Each run or test can build its own Runtime and pass it through Options, so pricing and retry behavior are facts about a request rather than about the process. A nil *Runtime means DefaultRuntime.
func NewRuntime ¶ added in v0.15.0
func NewRuntime() *Runtime
NewRuntime returns a Runtime with today's production behavior: a 10-minute HTTP timeout, a 1s retry base, 6 attempts, the built-in price table, and no price override.
func (*Runtime) PricePerMTok ¶ added in v0.15.0
PricePerMTok returns the built-in or configured USD price per million input and output tokens for spec. ok is false when the model is unpriced.
type Switch ¶ added in v0.5.0
type Switch struct {
From string // model spec that failed
To string // model spec the request is re-sent to
Reason string // "http_<status>" or "network"; never a response body
Attempts int // attempts made on From
}
Switch describes one move down the fallback chain.
type ToolCall ¶
type ToolCall struct {
ID string `json:"id"`
Type string `json:"type"`
Function FunctionCall `json:"function"`
}
type ToolResult ¶
type ToolSpec ¶
type ToolSpec struct {
Type string `json:"type"`
Function FunctionSpec `json:"function"`
}
ToolSpec is an OpenAI function-calling tool definition.