Documentation
¶
Overview ¶
Package model defines the provider-neutral interface to large language models: messages, requests, responses, streaming and token usage.
Index ¶
- Constants
- func CloneProviderOptions(opts map[string]map[string]any) map[string]map[string]any
- func IsRetryable(err error) bool
- func Permanent(err error) error
- func RetryAfter(err error) time.Duration
- func StreamFromResponse(resp *Response) iter.Seq2[*StreamEvent, error]
- func WrapReminder(text string) string
- type APIError
- type Accumulator
- type Describer
- type Description
- type Effort
- type FinishReason
- type Func
- type Message
- type Model
- type Part
- func AudioPart(mediaType string, data []byte) Part
- func AudioURLPart(url string) Part
- func DocumentPart(mediaType string, data []byte, filename string) Part
- func DocumentURLPart(url string) Part
- func FilePart(t PartType, provider, fileID, mediaType string) Part
- func ImagePart(mediaType string, data []byte) Part
- func ImageURLPart(url string) Part
- func MediaPart(mediaType string, data []byte) Part
- func TextDocumentPart(title, text string) Part
- func TextPart(text string) Part
- func VideoPart(mediaType string, data []byte) Part
- func VideoURLPart(url string) Part
- type PartType
- type PermanentError
- type Reasoning
- type ReasoningDisplay
- type Request
- type Response
- type ResponseFormat
- type Role
- type StreamEvent
- type StreamEventType
- type ToolCall
- type ToolChoice
- type ToolChoiceMode
- type ToolGrammar
- type ToolResult
- type ToolSpec
- type Usage
Constants ¶
const FreeformInput = "input"
FreeformInput is the argument that carries the text input of a call to a freeform tool; see ToolSpec.Grammar.
Variables ¶
This section is empty.
Functions ¶
func CloneProviderOptions ¶ added in v1.1.0
CloneProviderOptions copies opts and each provider's map; the option values remain shared.
func IsRetryable ¶
IsRetryable reports whether err may succeed if the request is retried.
- Provider errors follow APIError.Retryable.
- Transport failures are retryable: errors from http.Client.Do, which come as *url.Error, and failures reading a response body, which the agentx providers also report as *url.Error. This covers refused, reset and dropped connections, HTTP/2 stream resets and GOAWAY, connections closed early and transport timeouts, including http.Client.Timeout and Transport.ResponseHeaderTimeout. Network errors outside a *url.Error, such as *net.OpError, unexpected EOF or a net.Error timeout, are retryable too.
- Failures that recur on every attempt are not retryable even then: invalid URLs, certificate verification failures, TLS alerts from the server, other TLS handshake failures, DNS lookups of unknown hosts and a plain HTTP response to an HTTPS request.
- The caller's context being cancelled or past its deadline is not retryable. A timeout of the HTTP client is not the caller's deadline and is retryable, although it also matches context.DeadlineExceeded. Retry loops should still stop once their context is done.
- Errors marked Permanent and all other errors, such as invalid requests or undecodable responses, are not retryable.
func Permanent ¶
Permanent wraps err so that IsRetryable reports false for it. It returns nil for a nil err.
func RetryAfter ¶
RetryAfter returns the delay requested by the provider in err, or 0.
func StreamFromResponse ¶
func StreamFromResponse(resp *Response) iter.Seq2[*StreamEvent, error]
StreamFromResponse converts a complete response into a stream with one delta per part followed by the done event. A nil response yields an error.
func WrapReminder ¶
WrapReminder wraps text in a <system-reminder> block, the convention for context that the application, not the user, adds to a message.
Types ¶
type APIError ¶
type APIError struct {
Provider string
StatusCode int
// Type is the provider's error type, or its code when it reports no
// type.
Type string
// Code is the provider's error code, when it reports one alongside or
// instead of a type.
Code string
Message string
// RetryAfter is the delay requested by the provider, when available.
RetryAfter time.Duration
}
APIError is an error returned by a model provider's API.
type Accumulator ¶
type Accumulator struct {
// contains filtered or unexported fields
}
Accumulator assembles stream deltas into a complete message.
func (*Accumulator) Add ¶
func (a *Accumulator) Add(ev *StreamEvent)
Add merges a delta event into the accumulated message.
func (*Accumulator) Message ¶
func (a *Accumulator) Message() *Message
Message returns the accumulated assistant message.
type Describer ¶
type Describer interface {
Describe() Description
}
Describer is implemented by models that describe themselves for telemetry. The agentx providers implement it, and model wrappers should forward it.
type Description ¶
type Description struct {
// Provider is the gen_ai.provider.name, such as "openai", "anthropic"
// or "gcp.gemini".
Provider string
// Operation is the gen_ai.operation.name of a call, such as "chat" or
// "generate_content".
Operation string
// Model is the model used when a request names none.
Model string
// ServerAddress and ServerPort locate the API endpoint.
ServerAddress string
ServerPort int
}
Description identifies a model for telemetry. Names follow the OpenTelemetry GenAI semantic conventions.
func Describe ¶
func Describe(m Model) Description
Describe returns m's description, or a zero Description if m does not implement Describer.
type Effort ¶ added in v1.1.0
type Effort string
Effort is a reasoning effort level. Providers map the levels to what a model supports, using the nearest supported level.
type FinishReason ¶
type FinishReason string
FinishReason explains why the model stopped generating.
const ( FinishStop FinishReason = "stop" FinishLength FinishReason = "length" FinishToolCalls FinishReason = "tool_calls" // FinishRefusal reports that the provider declined the request. FinishRefusal FinishReason = "refusal" // FinishContentFilter reports that output was withheld by a filter. FinishContentFilter FinishReason = "content_filter" // FinishPause reports that a long-running server-side turn was paused and // should be continued by sending the conversation again. FinishPause FinishReason = "pause" FinishUnknown FinishReason = "unknown" )
type Func ¶
Func adapts a generate function into a Model. Stream is derived from Generate.
type Message ¶
type Message struct {
Role Role `json:"role"`
Parts []Part `json:"parts"`
Name string `json:"name,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"`
}
Message is a single conversation turn.
func AssistantText ¶
AssistantText returns an assistant message containing text.
func CloneMessages ¶
CloneMessages copies a message slice, cloning each message with Message.Clone.
func NewMessage ¶
NewMessage returns a message with the given role and parts.
func SystemText ¶
SystemText returns a system message containing text.
func ToolResults ¶
func ToolResults(results ...*ToolResult) *Message
ToolResults returns a tool message carrying the given results.
func (*Message) Clone ¶
Clone copies the message and its parts, including media data and tool payloads. Metadata maps are copied shallowly: their values remain shared. A nil receiver returns nil.
func (*Message) ToolResults ¶
func (m *Message) ToolResults() []*ToolResult
ToolResults returns the tool results in the message, in order.
type Model ¶
type Model interface {
// Generate returns a complete response.
Generate(ctx context.Context, req *Request) (*Response, error)
// Stream yields incremental events and finishes with a StreamDone event
// that carries the complete response. A non-nil error ends the stream.
Stream(ctx context.Context, req *Request) iter.Seq2[*StreamEvent, error]
}
Model generates responses from a conversation.
Implementations must be safe for concurrent use.
type Part ¶
type Part struct {
Type PartType `json:"type"`
// Text holds text content, the visible reasoning for PartReasoning, or
// the transcript of audio a model produced.
Text string `json:"text,omitempty"`
// MediaType, Data and URL describe an image, document, audio or video
// part: its content is either Data, a URL or FileID. Data is encoded as
// base64 in JSON.
MediaType string `json:"media_type,omitempty"`
Data []byte `json:"data,omitempty"`
URL string `json:"url,omitempty"`
// FileID refers to a file uploaded to the provider's Files API, such
// as an OpenAI or Anthropic file ID, in place of Data or URL. Gemini
// files are referenced by their URI in URL. Set Provider to the
// provider that issued the ID, so other providers reject the part
// instead of sending an ID they do not know.
FileID string `json:"file_id,omitempty"`
// Filename names a document, audio or video part. Providers send it as
// the file name or title where the API takes one.
Filename string `json:"filename,omitempty"`
// Detail is the resolution at which providers that support it process
// an image or video: "low", "medium", "high" or "auto". It maps to
// OpenAI's detail and Gemini's mediaResolution.
Detail string `json:"detail,omitempty"`
// Citations asks providers that support it, Anthropic and Bedrock, to
// cite a document part in the answer.
Citations bool `json:"citations,omitempty"`
// Signature is an opaque provider token that must be returned unchanged
// with the part, such as a reasoning signature.
Signature string `json:"signature,omitempty"`
// Redacted marks reasoning whose content is encrypted in Data.
Redacted bool `json:"redacted,omitempty"`
ToolCall *ToolCall `json:"tool_call,omitempty"`
ToolResult *ToolResult `json:"tool_result,omitempty"`
// Provider and Raw hold a PartRaw block. Provider also names the
// provider that issued Signature, when the provider records it, so
// signatures are not sent to providers that cannot verify them.
Provider string `json:"provider,omitempty"`
Raw json.RawMessage `json:"raw,omitempty"`
}
Part is one piece of message content.
func AudioURLPart ¶
AudioURLPart returns an audio part referenced by URL.
func DocumentPart ¶
DocumentPart returns an inline document part, such as a PDF, named filename. The name may be empty.
func DocumentURLPart ¶
DocumentURLPart returns a document part referenced by URL.
func FilePart ¶
FilePart returns a part of type t, such as PartDocument or PartImage, referring to the file fileID that was uploaded to provider's Files API. mediaType may be empty when the provider knows the file's type.
func ImageURLPart ¶
ImageURLPart returns an image part referenced by URL.
func MediaPart ¶
MediaPart returns an inline part whose type follows mediaType: image/* becomes an image part, audio/* an audio part, video/* a video part and any other type a document part.
func TextDocumentPart ¶
TextDocumentPart returns a plain text document part titled title. Providers with document support, such as Anthropic and Bedrock, can cite it.
func VideoURLPart ¶
VideoURLPart returns a video part referenced by URL.
type PartType ¶
type PartType string
PartType identifies the kind of content held by a Part.
const ( PartText PartType = "text" PartImage PartType = "image" // PartDocument is a document, such as a PDF or a plain text file. PartDocument PartType = "document" // PartAudio is an audio clip. PartAudio PartType = "audio" // PartVideo is a video clip. PartVideo PartType = "video" PartReasoning PartType = "reasoning" PartToolCall PartType = "tool_call" PartToolResult PartType = "tool_result" // PartRaw carries a provider-specific content block that has no neutral // representation. Providers echo raw parts they produced back verbatim and // ignore raw parts from other providers. PartRaw PartType = "raw" )
func MediaPartType ¶
MediaPartType returns the part type for content of mediaType; see MediaPart.
type PermanentError ¶
type PermanentError struct{ Err error }
PermanentError marks an error that retrying cannot fix, such as a request that fails validation before it is sent.
func (*PermanentError) Error ¶
func (e *PermanentError) Error() string
func (*PermanentError) Unwrap ¶
func (e *PermanentError) Unwrap() error
type Reasoning ¶
type Reasoning struct {
// Effort is the reasoning effort; empty leaves it to the provider.
Effort Effort `json:"effort,omitempty"`
// Display controls whether reasoning text is returned, for providers
// that support it; empty leaves it to the provider.
Display ReasoningDisplay `json:"display,omitempty"`
// BudgetTokens limits reasoning tokens for providers that use budgets.
BudgetTokens int `json:"budget_tokens,omitempty"`
}
Reasoning configures model reasoning ("thinking").
type ReasoningDisplay ¶ added in v1.1.0
type ReasoningDisplay string
ReasoningDisplay controls whether reasoning text is returned.
const ( // DisplaySummarized returns summarized reasoning text. DisplaySummarized ReasoningDisplay = "summarized" // DisplayOmitted returns no reasoning text. DisplayOmitted ReasoningDisplay = "omitted" )
Reasoning display modes.
type Request ¶
type Request struct {
// Model overrides the provider's configured model name.
Model string `json:"model,omitempty"`
// System is the system prompt.
System string `json:"system,omitempty"`
Messages []*Message `json:"messages"`
Tools []*ToolSpec `json:"tools,omitempty"`
ToolChoice *ToolChoice `json:"tool_choice,omitempty"`
MaxTokens int `json:"max_tokens,omitempty"`
Temperature *float64 `json:"temperature,omitempty"`
// Seed asks providers that support it, such as OpenAI and Gemini, for
// deterministic sampling.
Seed *int `json:"seed,omitempty"`
TopP *float64 `json:"top_p,omitempty"`
Stop []string `json:"stop,omitempty"`
Reasoning *Reasoning `json:"reasoning,omitempty"`
ResponseFormat *ResponseFormat `json:"response_format,omitempty"`
// ProviderOptions holds provider-specific request fields, keyed by the
// provider adapter: "openai", "anthropic", "gemini" or "bedrock", the
// name in Part.Provider. An adapter merges only its own map into the
// request body, overriding generated fields with the same name, so one
// request can carry options for several providers, as a fallback
// between providers needs.
ProviderOptions map[string]map[string]any `json:"provider_options,omitempty"`
}
Request is a provider-neutral generation request.
type Response ¶
type Response struct {
ID string `json:"id,omitempty"`
Model string `json:"model,omitempty"`
Message *Message `json:"message"`
FinishReason FinishReason `json:"finish_reason"`
Usage Usage `json:"usage"`
}
Response is a complete model output.
func Collect ¶
Collect consumes a stream and returns its final response. A done event without a response or without a message gets the message accumulated from the deltas. Nil events are skipped.
type ResponseFormat ¶
type ResponseFormat struct {
Name string `json:"name"`
Schema json.RawMessage `json:"schema"`
Strict bool `json:"strict,omitempty"`
}
ResponseFormat constrains the response to JSON matching Schema.
type StreamEvent ¶
type StreamEvent struct {
Type StreamEventType `json:"type"`
// Index is the position of the part that Delta extends.
Index int `json:"index"`
// Delta is a partial part. Text, Signature and ToolCall.Arguments are
// appended to the part at Index; other fields are set when first seen.
Delta *Part `json:"delta,omitempty"`
// Response is set on StreamDone.
Response *Response `json:"response,omitempty"`
}
StreamEvent is one event of a streaming response.
type StreamEventType ¶
type StreamEventType string
StreamEventType identifies a streaming event.
const ( // StreamDelta carries an incremental piece of a message part. StreamDelta StreamEventType = "delta" // StreamDone is the final event and carries the complete response. StreamDone StreamEventType = "done" )
type ToolCall ¶
type ToolCall struct {
ID string `json:"id"`
Name string `json:"name"`
Arguments json.RawMessage `json:"arguments,omitempty"`
}
ToolCall is a model request to invoke a tool.
type ToolChoice ¶
type ToolChoice struct {
Mode ToolChoiceMode `json:"mode"`
Name string `json:"name,omitempty"`
}
ToolChoice selects the tool-calling mode. Name is used with ToolChoiceTool.
Some recent models reject forced tool use (required or a named tool); use auto with a prompt instruction for those models.
type ToolChoiceMode ¶
type ToolChoiceMode string
ToolChoiceMode controls whether and which tools the model must call.
const ( ToolChoiceAuto ToolChoiceMode = "auto" ToolChoiceNone ToolChoiceMode = "none" ToolChoiceRequired ToolChoiceMode = "required" ToolChoiceTool ToolChoiceMode = "tool" )
type ToolGrammar ¶
type ToolGrammar struct {
// Syntax is the grammar language, "lark" or "regex".
Syntax string `json:"syntax"`
// Definition is the grammar. An empty definition accepts any text.
Definition string `json:"definition,omitempty"`
}
ToolGrammar constrains the text input of a freeform tool.
type ToolResult ¶
type ToolResult struct {
CallID string `json:"call_id"`
Name string `json:"name,omitempty"`
Content string `json:"content"`
// Parts holds content that follows Content, such as the images and
// documents a tool returned. Providers put them in the tool result
// where the API allows it, or else in a user message after the
// results; see their package documentation.
Parts []Part `json:"parts,omitempty"`
IsError bool `json:"is_error,omitempty"`
// Metadata is what the tool reported for the application in
// tool.Result.Metadata. It is not sent to models, and is stored with
// the conversation, so its values should be encodable as JSON.
Metadata map[string]any `json:"metadata,omitempty"`
}
ToolResult is the outcome of a tool call.
type ToolSpec ¶
type ToolSpec struct {
Name string `json:"name"`
Description string `json:"description,omitempty"`
// Parameters is a JSON Schema object describing the arguments.
Parameters json.RawMessage `json:"parameters,omitempty"`
// Strict asks providers that support it to guarantee schema-valid
// arguments.
Strict bool `json:"strict,omitempty"`
// Grammar, when set, makes the tool freeform where the provider
// supports it, as the OpenAI Responses API does: the model writes the
// input as plain text matching the grammar instead of JSON, and the
// provider passes the text as the string argument FreeformInput.
// Other providers send Parameters, which must declare that argument.
Grammar *ToolGrammar `json:"grammar,omitempty"`
}
ToolSpec describes a tool the model may call.
type Usage ¶
type Usage struct {
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
// CacheReadTokens and CacheWriteTokens count prompt-cache activity. They
// are reported separately from InputTokens by providers that distinguish
// them.
CacheReadTokens int `json:"cache_read_tokens,omitempty"`
CacheWriteTokens int `json:"cache_write_tokens,omitempty"`
// ReasoningTokens is the portion of OutputTokens spent on reasoning when
// the provider reports it.
ReasoningTokens int `json:"reasoning_tokens,omitempty"`
// Requests and ToolCalls count the model calls and executed tool calls
// of a run, including those of nested runs such as sub-agents. The
// runtime counts them; providers leave them zero.
Requests int `json:"requests,omitempty"`
ToolCalls int `json:"tool_calls,omitempty"`
}
Usage reports token consumption for one or more model calls.
func (Usage) PromptTokens ¶
PromptTokens returns every token the model read as input.
func (Usage) TotalTokens ¶
TotalTokens returns all input and output tokens, including cache activity.