Help us shape the future of Go! Take the 2026 Go Developer Survey

model

package
v0.16.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 1, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package model talks to LLM providers using a single OpenAI-shaped message format.

Index

Constants

View Source
const (
	ProviderAPIChat      = "chat"
	ProviderAPIResponses = "responses"
	ProviderAPIAnthropic = "anthropic"
)

Wire protocols a configured provider can speak.

View Source
const (
	// ExtraAnthropicContent holds every content block of an Anthropic
	// response that contained thinking, in order, for verbatim replay.
	ExtraAnthropicContent = "anthropic_content"
	// ExtraCodexReasoning holds the Responses API reasoning output items
	// (with encrypted_content), in order, for replay before the message. Both
	// codex and the openai Responses transport use it; the name predates the
	// latter and is kept so existing trajectories still replay.
	ExtraCodexReasoning = "codex_reasoning"
)

Message.Extra keys holding provider reasoning state. They are written by the provider that produced the message and ignored by every other one.

View Source
const (
	OpenAIAPIAuto      = "auto"
	OpenAIAPIResponses = "responses"
	OpenAIAPIChat      = "chat"
)

The openai provider's transports, chosen by the openai_api setting.

View Source
const (
	// ExtraTruncated is true on a response cut off at the output-token limit.
	ExtraTruncated = "truncated"
	// ExtraTruncationReason is the provider's raw reason, for logs.
	ExtraTruncationReason = "truncation_reason"
)

Message.Extra keys for output-token truncation. A provider sets them when its documented stop signal says the response was cut off at the output limit, so the agent loop reads one flag whatever the provider.

View Source
const ClaudeCodeProvider = "claude-code"

ClaudeCodeProvider is the provider id of the Claude Code handoff.

View Source
const CodexExecProvider = "codex-exec"

CodexExecProvider is the provider id of the codex exec handoff.

View Source
const DefaultAnthropicMaxTokens = 8192

DefaultAnthropicMaxTokens is the max_tokens sent to Anthropic when max_output_tokens is unset.

View Source
const ExtraModel = "model"

ExtraModel is the Message.Extra key naming the model spec that produced an assistant message. It is set only on runs with a fallback chain.

View Source
const HandoffDefaultModel = "default"

HandoffDefaultModel as the model name sends no --model flag, so the CLI's own configured model applies.

View Source
const ImageTokenEstimate = 1600

ImageTokenEstimate is the fixed token estimate for one image in context accounting, instead of base64 length / 4. It is close to the cost of a ~1,000x1,000 pixel image on Anthropic and OpenAI.

View Source
const MaxFallbackModels = 3

MaxFallbackModels caps the fallback chain (fallback_models, --fallback-model).

View Source
const MaxTaskImages = 10

MaxTaskImages caps how many --image attachments one task message carries.

View Source
const OpenCodeProvider = "opencode"

OpenCodeProvider is the provider id of the opencode run handoff.

View Source
const OpenCodeZenProvider = "opencode"

OpenCodeZenProvider is OpenCode's own Zen provider id.

Variables

View Source
var ContextWindows = map[string]int{
	"gpt-4o":       128_000,
	"gpt-4o-mini":  128_000,
	"gpt-4.1":      1_047_576,
	"gpt-4.1-mini": 1_047_576,
	"gpt-4.1-nano": 1_047_576,
	"claude-":      200_000,
}

ContextWindows maps a model name prefix to its context window in tokens. The longest matching prefix wins. Unknown models return 0, meaning the window isn't known.

View Source
var DefaultRuntime = NewRuntime()

DefaultRuntime is used whenever Options.Runtime (or an adapter's Runtime field) is nil. It is process-wide: the CLI sets its PriceOverride from config once per run. Tests should build their own Runtime instead of changing it.

View Source
var ImageMIMETypes = map[string]bool{"image/png": true, "image/jpeg": true, "image/gif": true, "image/webp": true}

ImageMIMETypes are the formats tinyagent sends to providers.

View Source
var OnCredentials func(values ...string)

OnCredentials, when set, receives credential values a provider loads from somewhere other than the environment, such as the codex provider's tokens (including ones refreshed during the run), for redaction.

OpenAIAPIs are the accepted openai_api values.

View Source
var ReasoningEfforts = []string{"none", "minimal", "low", "medium", "high", "xhigh", "max"}

ReasoningEfforts are the accepted reasoning_effort values. Support varies by model; unsupported values are reported by the provider.

Functions

func AttachmentInfo added in v0.5.0

func AttachmentInfo(images []Image) []map[string]any

AttachmentInfo is an attachment's metadata for events and logs: index, base name, MIME type, size, and sha256. It never contains image data.

func AttachmentLabels added in v0.5.0

func AttachmentLabels(images []Image) string

AttachmentLabels is the text added to a user message for its task attachments: a blank line, then one "[attached image N: name, mime, bytes]" line per image. It is "" without images, so image-free messages are unchanged.

func CheckProviderEnv added in v0.5.0

func CheckProviderEnv(spec string) error

CheckProviderEnv reports the first variable that the configured provider of spec needs but that is unset or empty, or nil (also for built-in providers).

func ContextWindow added in v0.4.0

func ContextWindow(spec string) int

ContextWindow returns the context window for spec, a "<provider>/<model>" name as given to New, or 0 when it isn't in ContextWindows. For a nested name such as openrouter/anthropic/claude-x, the last segment is tried too. A configured provider's model metadata, matched by exact name, comes before the table.

func DefaultPrices added in v0.15.0

func DefaultPrices() map[string][3]float64

DefaultPrices returns a copy of the built-in price table.

func FormatBytes added in v0.5.0

func FormatBytes(n int) string

FormatBytes adds thousands separators: 48213 -> 48,213.

func Hint

func Hint(spec, errMsg string) string

Hint returns a one-line setup fix for a failed run of the model spec, given the error text, or "" for errors that aren't a common setup problem. It names environment variables but never prints their values.

func InsecureBaseURL added in v0.5.0

func InsecureBaseURL(d ProviderDef) bool

InsecureBaseURL reports whether d sends requests over plain http to a host other than loopback.

func IsContextOverflow added in v0.4.0

func IsContextOverflow(err error) bool

IsContextOverflow reports whether err is a provider's "request is larger than the context window" error.

func IsRetryExhausted added in v0.5.0

func IsRetryExhausted(err error) bool

IsRetryExhausted reports whether err is a request that still failed with a transient cause after every retry.

func MaxOutputTokensSupported added in v0.5.0

func MaxOutputTokensSupported(spec string) error

MaxOutputTokensSupported reports whether the provider of spec accepts a per-request output-token budget, and returns an error naming it if not.

func MissingKeyEnv added in v0.5.0

func MissingKeyEnv(spec string) string

MissingKeyEnv returns the API key variable a built-in provider's spec needs when it is unset, or "" when it is set or not needed. An openai spec with OPENAI_BASE_URL set may target a server without keys, so it is never reported.

func OllamaBaseURL added in v0.5.0

func OllamaBaseURL() string

OllamaBaseURL returns the Ollama server URL that OLLAMA_HOST resolves to.

func OpenAITransport added in v0.5.0

func OpenAITransport(spec, setting string) string

OpenAITransport returns the transport New would use for spec with setting, reading OPENAI_BASE_URL, or "" when spec's provider isn't openai (the setting only applies to it).

func OpenCodeModelID added in v0.16.0

func OpenCodeModelID(name string) string

OpenCodeModelID maps the part of a tinyagent model name after "opencode/" to OpenCode's --model value: "" and "default" stay as they are, a bare name is a Zen model ("big-pickle" -> "opencode/big-pickle"), and "provider/model" is passed unchanged.

func PricePerMTok added in v0.7.0

func PricePerMTok(spec string) (in, out float64, ok bool)

PricePerMTok returns the built-in or configured USD price per million input and output tokens for spec under DefaultRuntime. ok is false when the model is unpriced.

func ReasoningChars added in v0.5.0

func ReasoningChars(m Message) int

ReasoningChars approximates the size of the reasoning state stored in m, which is sent back with the history.

func ResolveOpenAIAPI added in v0.5.0

func ResolveOpenAIAPI(setting, base string) string

ResolveOpenAIAPI returns the transport, "responses" or "chat", that the openai provider uses for setting (an openai_api value; "" means auto) with base, the value of OPENAI_BASE_URL ("" when unset). auto picks Responses only for OpenAI's own API, so OpenAI-compatible servers keep Chat Completions.

func SameProvider added in v0.5.0

func SameProvider(a, b string) bool

SameProvider reports whether two model specs use the same provider; a spec without a provider prefix is an openai model.

func SetCLIPaths added in v0.12.0

func SetCLIPaths(claude, codex string)

SetCLIPaths sets the configured binary paths of the claude-code and codex-exec providers; "" means look the binary up on PATH.

func SetMaxOutputTokens added in v0.5.0

func SetMaxOutputTokens(m Model, spec string, n int) error

SetMaxOutputTokens configures m, built by New(spec, ...), to send n as its provider's per-request output-token limit. n == 0 leaves m unchanged (the provider default; 8192 for Anthropic).

func SetOpenCodePath added in v0.14.0

func SetOpenCodePath(p string)

SetOpenCodePath sets the configured binary path of the opencode provider; "" means look it up on PATH. Call it after SetCLIPaths.

func SetProviders added in v0.5.0

func SetProviders(defs []ProviderDef)

SetProviders replaces the configured providers. Definitions are assumed validated (see config.Load).

func SetReasoningEffort added in v0.5.0

func SetReasoningEffort(m Model, spec, effort string) error

SetReasoningEffort configures m, built by New(spec, ...), to send effort using its provider's native control. An empty effort leaves m unchanged.

func Truncated added in v0.5.0

func Truncated(m Message) (bool, string)

Truncated reports whether m was cut off at the output-token limit, and the provider's raw reason.

func Unpriced added in v0.4.0

func Unpriced(spec string) bool

Unpriced reports whether spec, as passed to New, names a model billed per token whose cost tinyagent can't estimate under DefaultRuntime. See Runtime.Unpriced.

func ValidOpenAIAPI added in v0.5.0

func ValidOpenAIAPI(v string) bool

ValidOpenAIAPI reports whether v is an accepted openai_api value. "" means auto.

func ValidReasoningEffort added in v0.5.0

func ValidReasoningEffort(v string) bool

ValidReasoningEffort reports whether v is an accepted reasoning_effort.

func WithFallbackObserver added in v0.5.0

func WithFallbackObserver(ctx context.Context, fn func(Switch)) context.Context

WithFallbackObserver returns a context whose Fallback queries report each switch to fn before re-sending the request. fn runs on the requesting goroutine.

func WithRetryObserver added in v0.5.0

func WithRetryObserver(ctx context.Context, fn func(Retry)) context.Context

WithRetryObserver returns a context whose provider requests report each retry to fn before waiting. fn runs on the requesting goroutine.

Types

type Anthropic

type Anthropic struct {
	BaseURL   string
	APIKey    string
	Model     string
	MaxTokens int          // defaults to DefaultAnthropicMaxTokens
	OnText    func(string) // if set, stream and report text deltas
	// ReasoningEffort, if set, is sent as output_config.effort.
	ReasoningEffort string
	// Headers, if set, are added to every request after the defaults
	// (configured providers).
	Headers map[string]string
	// Price, if set, replaces the Runtime's price table entry for this
	// model (configured providers); the Runtime's PriceOverride still wins.
	Price *[3]float64
	// Runtime supplies the HTTP client, retry policy and prices. nil uses
	// DefaultRuntime.
	Runtime *Runtime
	// OmitEmptyKey sends no x-api-key header when APIKey is empty
	// (configured providers without api_key_env).
	OmitEmptyKey bool
}

Anthropic speaks the native Messages API.

func (*Anthropic) Query

func (a *Anthropic) Query(ctx context.Context, messages []Message, tools []ToolSpec) (Message, Cost, error)

type ClaudeCode added in v0.12.0

type ClaudeCode struct {
	Path  string // resolved claude binary
	Model string // "" or HandoffDefaultModel sends no --model
	HandoffOptions
}

ClaudeCode runs `claude -p` for each query.

func NewClaudeCode added in v0.12.0

func NewClaudeCode(name string, opts HandoffOptions) (*ClaudeCode, error)

NewClaudeCode resolves the claude binary for model name.

func (*ClaudeCode) Args added in v0.12.0

func (c *ClaudeCode) Args() []string

Args returns the claude arguments for one run.

func (*ClaudeCode) Query added in v0.12.0

func (c *ClaudeCode) Query(ctx context.Context, messages []Message, _ []ToolSpec) (Message, Cost, error)

type Codex

type Codex struct {
	BaseURL  string
	TokenURL string
	AuthFile string
	Model    string
	OnText   func(string)
	// ReasoningEffort, if set, is sent as reasoning.effort.
	ReasoningEffort string
	// Runtime supplies the HTTP client, retry policy and prices. nil uses
	// DefaultRuntime.
	Runtime *Runtime
	// contains filtered or unexported fields
}

Codex uses a ChatGPT subscription instead of an API key. It signs in with the tokens `codex login` saves in $CODEX_HOME/auth.json, refreshes them when they expire (writing them back so the Codex CLI stays signed in), and speaks the Responses API that the Codex backend serves.

func NewCodex

func NewCodex(model string, onText func(string)) (*Codex, error)

func (*Codex) Query

func (c *Codex) Query(ctx context.Context, messages []Message, tools []ToolSpec) (Message, Cost, error)

type CodexExec added in v0.12.0

type CodexExec struct {
	Path  string // resolved codex binary
	Model string // "" or HandoffDefaultModel sends no -m
	HandoffOptions
}

CodexExec runs `codex exec --json` for each query.

func NewCodexExec added in v0.12.0

func NewCodexExec(name string, opts HandoffOptions) (*CodexExec, error)

NewCodexExec resolves the codex binary for model name.

func (*CodexExec) Args added in v0.12.0

func (c *CodexExec) Args() []string

Args returns the codex arguments for one run; the prompt goes on stdin.

func (*CodexExec) Query added in v0.12.0

func (c *CodexExec) Query(ctx context.Context, messages []Message, _ []ToolSpec) (Message, Cost, error)

type Cost

type Cost struct {
	USD              float64 `json:"usd"`
	InputTokens      int     `json:"input_tokens"`
	OutputTokens     int     `json:"output_tokens"`
	CacheReadTokens  int     `json:"cache_read_tokens,omitempty"`
	CacheWriteTokens int     `json:"cache_write_tokens,omitempty"`
}

Cost is one response's usage. InputTokens counts every input token, including the cache reads and writes broken out after it.

type Fallback added in v0.5.0

type Fallback struct {
	Entries []FallbackEntry
	// contains filtered or unexported fields
}

Fallback is a Model that sends each request to the active entry and, when that request still fails with a transient cause after the provider's own retries (a RetryExhaustedError), switches to the next entry and re-sends the same request. Other errors are returned as they are. After a switch, later requests stay on the model that answered for the rest of the invocation. Entries[0] is the primary model.

func NewFallback added in v0.5.0

func NewFallback(primary FallbackEntry, fallbacks ...FallbackEntry) *Fallback

NewFallback returns a Fallback over primary followed by fallbacks.

func (*Fallback) Active added in v0.5.0

func (f *Fallback) Active() string

Active returns the spec of the model that serves the next request.

func (*Fallback) Query added in v0.5.0

func (f *Fallback) Query(ctx context.Context, messages []Message, tools []ToolSpec) (Message, Cost, error)

Query implements Model.

type FallbackEntry added in v0.5.0

type FallbackEntry struct {
	Spec  string
	Model Model
}

FallbackEntry is one model of a fallback chain: its spec as given to New and the Model built from it.

type FunctionCall

type FunctionCall struct {
	Name      string `json:"name"`
	Arguments string `json:"arguments"`
}

type FunctionSpec

type FunctionSpec struct {
	Name        string         `json:"name"`
	Description string         `json:"description"`
	Parameters  map[string]any `json:"parameters"`
}

type HandoffOptions added in v0.12.0

type HandoffOptions struct {
	// Dir is the working directory the CLI runs in (--cwd). "" keeps
	// tinyagent's own working directory.
	Dir string
	// ReadOnly maps --read-only/read_only onto the CLI's own restriction.
	ReadOnly bool
}

HandoffOptions are the TinyAgent settings a handoff provider maps onto the CLI it runs.

type Image added in v0.5.0

type Image struct {
	MIMEType string `json:"mime_type"`
	Data     string `json:"data"`
	// Label names the source for elision notes: a workspace path or an MCP
	// tool name.
	Label string `json:"label,omitempty"`
}

Image is one image attached to a tool result (opt-in --images) or to a user task message (--image). Data is the base64 encoding of the raw bytes. Images travel only in provider requests and trajectories; every text channel (progress log, --json, the SQLite log, hooks) sees Placeholder instead.

func NewImage added in v0.5.0

func NewImage(mime, label string, raw []byte) Image

NewImage encodes raw image bytes.

func (Image) DataURL added in v0.5.0

func (im Image) DataURL() string

DataURL returns the image as a data: URL.

func (Image) Placeholder added in v0.5.0

func (im Image) Placeholder() string

Placeholder is the text form shown in place of the image, e.g. "[image: image/png, 48,213 bytes, sha256:ab12cd34ef56]". It never contains image data.

func (Image) Size added in v0.5.0

func (im Image) Size() int

Size returns the decoded size in bytes.

type Message

type Message struct {
	Role       string     `json:"role"`
	Content    string     `json:"content,omitempty"`
	ToolCalls  []ToolCall `json:"tool_calls,omitempty"`
	ToolCallID string     `json:"tool_call_id,omitempty"`
	// Images are tool-result images (opt-in --images) or user task
	// attachments (--image), sent in each provider's image form. Content
	// already holds a placeholder or label line for each one.
	Images []Image        `json:"images,omitempty"`
	Extra  map[string]any `json:"extra,omitempty"`
}

func ForModel added in v0.5.0

func ForModel(messages []Message, spec, primary string) []Message

ForModel returns messages as they may be sent to spec: provider-specific reasoning state (Anthropic thinking blocks, Responses/Codex reasoning items) is kept only on assistant messages produced by the same provider. A message without an ExtraModel tag counts as produced by primary. The input is not modified; messages without foreign state are shared.

type MissingCLIError added in v0.12.0

type MissingCLIError struct {
	Provider, Binary, Configured, Install, SignIn, Docs string
}

MissingCLIError is returned when a handoff provider's CLI isn't found.

func (*MissingCLIError) Error added in v0.12.0

func (e *MissingCLIError) Error() string

type Model

type Model interface {
	Query(ctx context.Context, messages []Message, tools []ToolSpec) (Message, Cost, error)
}

func New

func New(spec string, onText func(string)) (Model, error)

New resolves "<provider>/<model>" to a Model. Without a "/", the provider is openai. If onText is non-nil, responses are streamed and each text delta is passed to it as it arrives.

func NewWith added in v0.5.0

func NewWith(spec string, onText func(string), opts Options) (Model, error)

NewWith is New with provider options.

type ModelMeta added in v0.5.0

type ModelMeta struct {
	InputPricePerMTok   float64
	OutputPricePerMTok  float64
	CacheReadMultiplier float64 // 0 means the standard 0.1
	ContextWindow       int
}

ModelMeta is per-model price and context-window metadata for a configured provider. Zero values mean unknown.

type OpenAI

type OpenAI struct {
	BaseURL string
	APIKey  string
	Model   string
	OnText  func(string) // if set, stream and report text deltas
	// ReasoningEffort, if set, is sent as reasoning_effort, or as
	// reasoning.effort when NestedReasoning is set (OpenRouter).
	ReasoningEffort string
	NestedReasoning bool
	// MaxOutputTokens, if > 0, is sent under MaxOutputKey
	// (max_completion_tokens, or max_tokens for Ollama).
	MaxOutputTokens int
	MaxOutputKey    string
	// Headers, if set, are added to every request after the defaults
	// (configured providers).
	Headers map[string]string
	// Price, if set, replaces the Runtime's price table entry for this
	// model (configured providers); the Runtime's PriceOverride still wins.
	Price *[3]float64
	// Runtime supplies the HTTP client, retry policy and prices. nil uses
	// DefaultRuntime.
	Runtime *Runtime
}

OpenAI speaks the Chat Completions API, which also covers OpenRouter, Ollama, and other OpenAI-compatible endpoints.

func NewOllama

func NewOllama(model string) *OpenAI

NewOllama returns a client for a local Ollama server's OpenAI-compatible endpoint.

func (*OpenAI) Query

func (o *OpenAI) Query(ctx context.Context, messages []Message, tools []ToolSpec) (Message, Cost, error)

type OpenAIResponses added in v0.5.0

type OpenAIResponses struct {
	BaseURL string
	APIKey  string
	Model   string
	OnText  func(string) // if set, stream and report text deltas
	// ReasoningEffort, if set, is sent as reasoning.effort.
	ReasoningEffort string
	// MaxOutputTokens, if > 0, is sent as max_output_tokens.
	MaxOutputTokens int

	// Headers, if set, are added to every request after the defaults
	// (configured providers).
	Headers map[string]string
	// Price, if set, replaces the Runtime's price table entry for this
	// model (configured providers); the Runtime's PriceOverride still wins.
	Price *[3]float64
	// Runtime supplies the HTTP client, retry policy and prices. nil uses
	// DefaultRuntime.
	Runtime *Runtime
	// contains filtered or unexported fields
}

OpenAIResponses speaks OpenAI's Responses API (POST /responses) with an API key. It is stateless (store: false) and keeps reasoning items that carry encrypted_content for replay, so tool calls and reasoning work together.

func (*OpenAIResponses) Query added in v0.5.0

func (o *OpenAIResponses) Query(ctx context.Context, messages []Message, tools []ToolSpec) (Message, Cost, error)

type OpenCode added in v0.14.0

type OpenCode struct {
	Path  string // resolved opencode binary
	Model string // "" or HandoffDefaultModel sends no --model
	HandoffOptions
}

OpenCode runs `opencode run --format json` for each query.

func NewOpenCode added in v0.14.0

func NewOpenCode(name string, opts HandoffOptions) (*OpenCode, error)

NewOpenCode resolves the opencode binary for model name.

func (*OpenCode) Args added in v0.14.0

func (c *OpenCode) Args(prompt string) []string

Args returns the opencode arguments for one run with prompt as the message. The working directory is not a flag: the process runs in c.Dir.

func (*OpenCode) Env added in v0.14.0

func (c *OpenCode) Env() []string

Env returns the extra environment for one run (nil when none).

func (*OpenCode) Query added in v0.14.0

func (c *OpenCode) Query(ctx context.Context, messages []Message, _ []ToolSpec) (Message, Cost, error)

func (*OpenCode) Version added in v0.14.0

func (c *OpenCode) Version(ctx context.Context) string

Version runs `opencode --version` with a short timeout and returns its first output line, or "" if it fails.

type Options added in v0.5.0

type Options struct {
	// OpenAIAPI is the openai provider's transport: auto (or ""), responses,
	// or chat. Other providers ignore it.
	OpenAIAPI string
	// Handoff is what the claude-code, codex-exec and opencode providers map onto
	// the CLI they run. Other providers ignore it.
	Handoff HandoffOptions
	// Runtime supplies the HTTP client, retry policy and prices for the
	// HTTP providers. nil uses DefaultRuntime.
	Runtime *Runtime
}

Options are provider settings beyond the model spec.

type ProviderDef added in v0.5.0

type ProviderDef struct {
	ID         string
	API        string
	BaseURL    string
	APIKeyEnv  string
	Headers    map[string]string
	EnvHeaders map[string]string
	Models     map[string]ModelMeta
}

ProviderDef is a config-defined provider. APIKeyEnv and EnvHeaders name environment variables; their values are read only when a model is built.

func ConfiguredProvider added in v0.5.0

func ConfiguredProvider(spec string) (ProviderDef, bool)

ConfiguredProvider returns the configured provider spec resolves to, if any. Built-in names always resolve to the built-in provider.

type ProviderEnvError added in v0.5.0

type ProviderEnvError struct {
	Provider, Env, Use string
}

ProviderEnvError reports a missing environment variable a configured provider needs. It names the variable, never a value.

func (*ProviderEnvError) Error added in v0.5.0

func (e *ProviderEnvError) Error() string

type Retry added in v0.5.0

type Retry struct {
	Attempt int           // the attempt about to be made, starting at 2
	Reason  string        // "http_<status>" or "network"; never a response body
	Backoff time.Duration // wait before the attempt
}

Retry describes one provider request about to be retried.

type RetryExhaustedError added in v0.5.0

type RetryExhaustedError struct {
	Attempts int
	Reason   string // "http_<status>" or "network"; never a response body
	Err      error  // the last attempt's error
}

RetryExhaustedError is returned when a request still failed with a transient cause (HTTP 429, 5xx or a network error) after every attempt. Errors returned without retrying (other 4xx) are never this type, so errors.As tells an outage apart from a setup or request problem.

func (*RetryExhaustedError) Error added in v0.5.0

func (e *RetryExhaustedError) Error() string

func (*RetryExhaustedError) Unwrap added in v0.5.0

func (e *RetryExhaustedError) Unwrap() error

type Runtime added in v0.15.0

type Runtime struct {
	// HTTP sends provider requests. nil uses a client with a 10-minute
	// timeout.
	HTTP *http.Client
	// RetryBase is the first retry's backoff; each later retry doubles it
	// (plus jitter). Zero or negative uses 1s.
	RetryBase time.Duration
	// MaxAttempts is the total number of tries for a transient failure
	// (HTTP 429, 5xx or a network error). Zero or negative uses 6.
	MaxAttempts int
	// Prices maps a model name prefix to USD per million input/output tokens
	// and, optionally, the cache-read price as a multiple of the input price
	// (0 means the standard 0.1). The longest matching prefix wins. nil uses
	// the built-in table; a non-nil empty map prices every model at 0.
	Prices map[string][3]float64
	// PriceOverride, when non-zero, replaces the price table's input and
	// output prices. The table's cache-read multiplier still applies.
	PriceOverride [2]float64
}

Runtime holds the provider settings that used to be package-level globals: the HTTP client, the retry policy, and the price table. Each run or test can build its own Runtime and pass it through Options, so pricing and retry behavior are facts about a request rather than about the process. A nil *Runtime means DefaultRuntime.

func NewRuntime added in v0.15.0

func NewRuntime() *Runtime

NewRuntime returns a Runtime with today's production behavior: a 10-minute HTTP timeout, a 1s retry base, 6 attempts, the built-in price table, and no price override.

func (*Runtime) PricePerMTok added in v0.15.0

func (r *Runtime) PricePerMTok(spec string) (in, out float64, ok bool)

PricePerMTok returns the built-in or configured USD price per million input and output tokens for spec. ok is false when the model is unpriced.

func (*Runtime) Unpriced added in v0.15.0

func (r *Runtime) Unpriced(spec string) bool

Unpriced reports whether spec, as passed to New, names a model billed per token whose cost tinyagent can't estimate: no price table entry and no PriceOverride, so its cost always reads 0. Ollama and Codex are free by design and never unpriced.

type Switch added in v0.5.0

type Switch struct {
	From     string // model spec that failed
	To       string // model spec the request is re-sent to
	Reason   string // "http_<status>" or "network"; never a response body
	Attempts int    // attempts made on From
}

Switch describes one move down the fallback chain.

type ToolCall

type ToolCall struct {
	ID       string       `json:"id"`
	Type     string       `json:"type"`
	Function FunctionCall `json:"function"`
}

type ToolResult

type ToolResult struct {
	ToolCallID string `json:"tool_call_id"`
	Content    string `json:"content"`
}

type ToolSpec

type ToolSpec struct {
	Type     string       `json:"type"`
	Function FunctionSpec `json:"function"`
}

ToolSpec is an OpenAI function-calling tool definition.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL