Documentation
¶
Overview ¶
Package provider defines the model-backend abstraction and a registry mapping a provider "kind" to a factory. Concrete implementations live in subpackages (e.g. provider/openai) and self-register via init(). The core resolves providers by kind from config and never hardcodes a specific model.
Index ¶
- Constants
- func BudgetKeyForConfig(name, baseURL, apiKey string) string
- func CanonicalizeSchema(raw json.RawMessage) json.RawMessage
- func ContentLen(content any) int
- func ContentString(content any) string
- func ImageContent(text string, imageURLs ...string) any
- func IsConnReset(err error) bool
- func IsStreamInterrupted(err error) bool
- func IsTextOnly(content any) bool
- func Kinds() []string
- func ParseImageDataURL(dataURL string) (mediaType, base64Data string, ok bool)
- func Register(kind string, f Factory)
- func RetryableStatus(s int) bool
- func SendWithRetry(ctx context.Context, httpClient *http.Client, opts SendOptions, ...) (*http.Response, error)
- func TextContent(text string) any
- func WithRetryNotify(ctx context.Context, fn RetryNotify) context.Context
- type APIError
- type AuthError
- type BudgetStatus
- type Chunk
- type ChunkType
- type Config
- type ContentPart
- type Factory
- type ImageURL
- type Message
- type Pricing
- type Provider
- type RateLimitedProvider
- type Request
- type RequestBudget
- type RetryInfo
- type RetryNotify
- type Role
- type SendOptions
- type StreamInterruptedError
- type ToolCall
- type ToolSchema
- type Usage
Constants ¶
const MaxRetries = 10
MaxRetries is the number of times SendWithRetry re-attempts the connection + header phase after the initial try (so up to MaxRetries+1 total attempts).
Variables ¶
This section is empty.
Functions ¶
func BudgetKeyForConfig ¶
BudgetKeyForConfig returns the budget bucket key for a provider Config — baseURL + the resolved API key. Two providers hitting the same endpoint with the same key share one RPM quota (matching how providers actually meter: the platform counts requests per key, regardless of which model/feature/client issued them). The name parameter is accepted for call-site symmetry with provider.Config but intentionally NOT included, so the main conversation, subagents, multimodal tools, RAG extraction/embedding, and RagAsk all draw from one bucket when they share an endpoint+key.
func CanonicalizeSchema ¶
func CanonicalizeSchema(raw json.RawMessage) json.RawMessage
CanonicalizeSchema recursively stabilizes a JSON Schema so the same logical schema always produces the same byte representation.
func ContentLen ¶
ContentLen estimates the byte size of Content for token budgeting. For multimodal content, text bytes + base64 image bytes (approximate).
func ContentString ¶
ContentString extracts the text portion of a Message.Content field. When Content is nil or a plain string, returns it directly. When Content is a structured multimodal block, extracts and concatenates text parts.
func ImageContent ¶
ImageContent creates a multimodal Content value with text and image parts.
func IsConnReset ¶
IsConnReset reports whether err is a connection-level drop (peer reset, truncated body, closed socket) as opposed to a protocol or caller error. A stream cut this way mid-body can be replayed from scratch, unlike a decode or 4xx error. The common trigger is a local proxy (v2rayN/sing-box) idle-closing the long-lived SSE connection during a reasoner's first-token gap.
func IsStreamInterrupted ¶
func IsTextOnly ¶
IsTextOnly reports whether Content contains no image parts.
func ParseImageDataURL ¶
ParseImageDataURL splits a "data:image/png;base64,AAAA..." data URL into its MIME type and raw base64 payload. Returns ("", "", false) when the prefix is not a valid data URL.
func Register ¶
Register adds a factory under a kind (e.g. "openai"). Intended for init(). It panics on a duplicate kind, since that is a compile-time wiring mistake.
func RetryableStatus ¶
RetryableStatus reports whether a backoff can plausibly recover from status s: 408 (request timeout), 429 (rate limit) and 5xx (incl. Anthropic's 529). Other 4xx (400/401/402/422, …) are caller/config problems retrying can't fix.
func SendWithRetry ¶
func SendWithRetry(ctx context.Context, httpClient *http.Client, opts SendOptions, newReq func(context.Context) (*http.Request, error)) (*http.Response, error)
SendWithRetry POSTs a streaming request built by newReq and returns the OK response. It retries the connection+header phase up to MaxRetries times on transient network errors and retryable statuses with capped exponential backoff + jitter, honoring Retry-After. 401/403 become *AuthError; other non-OK statuses become *APIError. A RetryNotify in ctx fires before each sleep. Retries cover only the header phase — once the body streams, mid-stream failures are not retried (the model has already emitted tokens).
func TextContent ¶
TextContent creates a simple text-only Content value.
func WithRetryNotify ¶
func WithRetryNotify(ctx context.Context, fn RetryNotify) context.Context
WithRetryNotify attaches a callback that SendWithRetry invokes before each backoff sleep, so the agent can surface a transient "retrying (n/m)" status.
Types ¶
type APIError ¶
APIError reports a non-OK HTTP status that isn't an auth failure. Status carries the code so the display layer can map it to an actionable, localized message; Body is a trimmed snippet of the response.
type AuthError ¶
type AuthError struct {
Provider string // the provider instance name, e.g. "openai"
KeyEnv string // the api_key_env the key is read from, when known
Status int // the HTTP status (401 or 403)
HasKey bool // a non-empty key was sent vs. no key configured
}
AuthError reports that a provider rejected the API key (HTTP 401/403). Its message is already user-facing and actionable — it names the provider and, when known, the environment variable the key comes from — so the CLI can surface it verbatim instead of dumping a raw status body. Providers should return this (rather than a generic status error) for auth failures.
type BudgetStatus ¶
type BudgetStatus struct {
RPM int `json:"rpm"`
Used int `json:"used"`
Remaining int `json:"remaining"`
ReserveMain int `json:"reserveMain"`
WindowSecs int `json:"windowSecs"` // seconds until the window resets
}
Status reports the current-window usage for a key, for the UI's "RPM: 3/5 remaining" indicator and cost estimates.
type Chunk ¶
type Chunk struct {
Type ChunkType
Text string // ChunkText, ChunkReasoning
Signature string // ChunkReasoning: opaque proof for the reasoning (Anthropic thinking signature), when issued
ToolCall *ToolCall // ChunkToolCallStart (ID+Name only), ChunkToolCall (complete)
Usage *Usage // ChunkUsage
Err error // ChunkError
}
Chunk is a single streamed event. Read the field matching Type.
type ChunkType ¶
type ChunkType int
ChunkType identifies the kind of a streamed increment.
const ( ChunkText ChunkType = iota // text delta ChunkReasoning // thinking-mode reasoning delta (before the visible answer) ChunkToolCallStart // a tool call has begun (ToolCall: ID+Name; args still streaming) ChunkToolCall // one complete tool call ChunkUsage // token usage for the completion ChunkDone // completion finished normally ChunkError // an error occurred )
type Config ¶
type Config struct {
Name string // instance name, e.g. "openai"
BaseURL string // OpenAI-compatible endpoint
Model string // model id
APIKey string // resolved from api_key_env
Extra map[string]any // kind-specific options
}
Config is a resolved provider instance configuration.
type ContentPart ¶
type ContentPart struct {
Type string `json:"type"` // "text" or "image_url"
Text string `json:"text,omitempty"` // for Type == "text"
ImageURL *ImageURL `json:"image_url,omitempty"` // for Type == "image_url"
}
ContentPart is one block in a multimodal message (OpenAI content parts format).
func ImageParts ¶
func ImageParts(content any) []ContentPart
ImageParts extracts image ContentParts from a multimodal Content value. Returns nil when Content is a plain string or contains no images.
type ImageURL ¶
type ImageURL struct {
URL string `json:"url"` // "data:image/png;base64,..."
Detail string `json:"detail,omitempty"` // "low", "high", "auto"
}
ImageURL holds a base64 data URL for inline image content.
type Message ¶
type Message struct {
Role Role `json:"role"`
Content any `json:"content,omitempty"` // string or []ContentPart (multimodal)
ReasoningContent string `json:"reasoning_content,omitempty"` // assistant: thinking-mode chain-of-thought, round-tripped on multi-turn
// ReasoningSignature is an opaque, provider-issued proof that ReasoningContent
// is genuine model output. Anthropic requires the signed thinking block be
// replayed on the next turn when a tool call followed thinking; providers
// without signed reasoning (e.g. the openai-compatible ones) leave it empty.
// Round-tripped alongside ReasoningContent.
ReasoningSignature string `json:"reasoning_signature,omitempty"`
ToolCalls []ToolCall `json:"tool_calls,omitempty"` // set by assistant
ToolCallID string `json:"tool_call_id,omitempty"` // links a tool result to its call
Name string `json:"name,omitempty"` // tool message: tool name
}
Message is a single conversation message.
func NormalizeMessages ¶
NormalizeMessages repairs a conversation history so it satisfies the tool-call contract the OpenAI-compatible and Anthropic APIs enforce: every assistant tool_calls entry must be answered by a following tool message for its id, and a tool message must follow such a call. It backfills a placeholder result for any unanswered call (so the turn stays intact), drops orphan tool messages, backfills empty tool-call names from their results (old sessions saved before name-tracking landed can carry an empty name), and closes truncated call-argument JSON (some gateways 400 on replayed half-streamed args).
This is the wire-safe entry point for provider requests. Stored session loads use NormalizeSessionMessages so they can share the assistant-turn repairs without deleting standalone tool messages that must round-trip through resume.
A well-formed history — no unanswered calls, no orphan results, no empty tool-call names, no truncated args — returns the input slice unchanged (same backing array, zero allocation). This keeps the prefix-cache key stable for healthy sessions and makes repeated normalization cheap.
Ported from DeepSeek-Reasonix (PR #4811 unifying history normalization).
func NormalizeSessionMessages ¶
NormalizeSessionMessages applies only repairs that are safe to persist in a saved session. It shares assistant-turn repairs with NormalizeMessages, but preserves existing tool messages instead of dropping or reordering them so Save/LoadSession remains a byte-for-byte conversation round trip for histories that were already on disk.
func SanitizeToolPairing ¶
SanitizeToolPairing repairs a history for a provider request (every assistant tool_calls answered by a following tool message, orphan tool messages dropped, empty tool-call names backfilled from results, truncated args closed) right before sending it to the wire — without touching the stored session. Kept as a distinct name so call sites read as "defensive wire prep" rather than "session mutation". Now a thin alias over NormalizeMessages so the wire path and the session-load path share one repair implementation.
func (*Message) UnmarshalJSON ¶
UnmarshalJSON restores Content as its concrete type. The field is `any` (string for plain text, []ContentPart for multimodal). encoding/json has no way to recover that from a generic []interface{} on reload, so without this a saved multimodal message comes back as []interface{} — and every downstream switch on `content.(type)` (ContentString, ContentLen, buildRequest) misses its []ContentPart case, dumping the image data URL as plain text. We decode content separately: a JSON string stays a string; a JSON array becomes []ContentPart, fixing the type for the whole pipeline.
type Pricing ¶
type Pricing struct {
CacheHit float64 `toml:"cache_hit"` // per 1M cached prompt tokens
Input float64 `toml:"input"` // per 1M uncached prompt tokens
Output float64 `toml:"output"` // per 1M completion tokens
Currency string `toml:"currency"`
}
Pricing is a provider's per-1M-token rates, used to estimate spend. Currency is just a display symbol (default "¥"). toml tags let config decode it.
type Provider ¶
type Provider interface {
// Name returns the provider instance name, e.g. "openai" / "anthropic".
Name() string
// Stream starts a streaming completion, pushing increments on the channel.
// Cancelling ctx must abort the underlying request; a closed channel marks
// the end of the completion.
Stream(ctx context.Context, req Request) (<-chan Chunk, error)
}
Provider is a chat-capable model backend.
func NewRateLimitedProvider ¶
func NewRateLimitedProvider(inner Provider, budget *RequestBudget, key string, priority bool) Provider
NewRateLimitedProvider wraps inner. key identifies the budget bucket (baseURL+apiKey). priority=true marks this as a main-agent provider (always granted, may use reserve); false = background (waits on reserve).
func UnwrapProvider ¶
UnwrapProvider peels decorator layers (anything implementing Unwrap() Provider) off p until it reaches a provider that has no inner, returning that base. Use this before a type assertion on a provider that may have been wrapped by boot.NewProviderWithProxy (which adds RateLimitedProvider when a global RPM budget is configured). nil-safe: a nil p returns nil.
type RateLimitedProvider ¶
type RateLimitedProvider struct {
// contains filtered or unexported fields
}
RateLimitedProvider wraps an inner Provider with request-budget enforcement.
func (*RateLimitedProvider) Name ¶
func (p *RateLimitedProvider) Name() string
func (*RateLimitedProvider) Unwrap ¶
func (p *RateLimitedProvider) Unwrap() Provider
Unwrap returns the inner Provider this decorator wraps, so callers (and tests) can reach the concrete provider underneath any number of decorators. Pairs with UnwrapProvider, the standard Go unwrap idiom: NewProviderWithProxy may wrap a provider with this rate limiter depending on the global budget, so a direct type assertion on its return value is unsafe — unwrap first.
type Request ¶
type Request struct {
Messages []Message
Tools []ToolSchema
Temperature float64
MaxTokens int
}
Request is a single completion request.
type RequestBudget ¶
type RequestBudget struct {
// contains filtered or unexported fields
}
RequestBudget meters RPM across one or more API keys. Each key (identified by baseURL+apiKey) gets its own rolling window, because different providers have independent quotas. nil / RPM=0 means no limiting.
func NewRequestBudget ¶
func NewRequestBudget(rpm, reserveMain int) *RequestBudget
NewRequestBudget builds a budget. rpm<=0 returns a disabled budget (Acquire is a no-op) so callers don't need to branch on "is limiting configured".
func (*RequestBudget) Acquire ¶
Acquire blocks until a request slot is available for the given key, or ctx is cancelled. priority=true grants immediately (main-agent); priority=false (background) waits when remaining quota <= reserve.
Returns nil on success, ctx.Err() if the context is cancelled while waiting.
func (*RequestBudget) Status ¶
func (b *RequestBudget) Status(key string) BudgetStatus
Status returns the current budget status for a key. When limiting is disabled, returns a zeroed status (RPM=0).
type RetryInfo ¶
RetryInfo describes a backoff about to happen: Attempt is the 1-based retry number (of Max) and Delay is how long SendWithRetry will wait before it.
type RetryNotify ¶
type RetryNotify func(RetryInfo)
type SendOptions ¶
type SendOptions struct {
ProvName string // provider instance name for error messages
KeyEnv string // api_key_env for AuthError
KeyPresent bool // a non-empty key was configured
RetryAuth bool // retry 401/403 up to maxAuthRetries (key previously worked)
}
SendOptions configures SendWithRetry's behaviour.
type StreamInterruptedError ¶
type StreamInterruptedError struct {
Err error
}
StreamInterruptedError marks a recoverable transport cut that happened after the caller had already received model output. Providers must not replay these requests themselves because doing so could duplicate visible text or tool calls; the agent can append a tail recovery prompt instead.
func (*StreamInterruptedError) Error ¶
func (e *StreamInterruptedError) Error() string
func (*StreamInterruptedError) Unwrap ¶
func (e *StreamInterruptedError) Unwrap() error
type ToolCall ¶
type ToolCall struct {
ID string `json:"id"`
Name string `json:"name"`
Arguments string `json:"arguments"`
}
ToolCall is a tool invocation requested by the model. Arguments is raw JSON.
type ToolSchema ¶
type ToolSchema struct {
Name string `json:"name"`
Description string `json:"description"`
Parameters json.RawMessage `json:"parameters"`
}
ToolSchema is a tool definition exposed to the model. Parameters is JSON Schema.
type Usage ¶
type Usage struct {
PromptTokens int
CompletionTokens int
TotalTokens int
CacheHitTokens int // prompt tokens served from cache
CacheMissTokens int // prompt tokens not cached
ReasoningTokens int // subset of CompletionTokens spent on chain-of-thought
FinishReason string // "stop", "tool_calls", "length", "content_filter", "repetition_truncation", …
}
Usage reports token accounting for a completion. Cache hit/miss come from either top-level prompt_cache_{hit,miss}_tokens or the OpenAI standard prompt_tokens_details.cached_tokens — the openai provider normalises both shapes into these fields. Note: some providers do not report cache tokens (both fields stay 0); the normalisation is kept for future support. ReasoningTokens is the thinking-mode subset of CompletionTokens reported by thinking-capable models. FinishReason carries the model's last reported choices[0].finish_reason so the agent can surface abnormal terminations ("length", "content_filter", "repetition_truncation").
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package anthropic implements the Anthropic Messages API provider (POST /v1/messages, SSE streaming) with a hand-written net/http client — no SDK.
|
Package anthropic implements the Anthropic Messages API provider (POST /v1/messages, SSE streaming) with a hand-written net/http client — no SDK. |
|
Package openai implements the OpenAI-compatible /chat/completions provider.
|
Package openai implements the OpenAI-compatible /chat/completions provider. |