Help us shape the future of Go! Take the 2026 Go Developer Survey

model

package
v1.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 31, 2026 License: MIT Imports: 17 Imported by: 0

Documentation

Overview

Package model defines the provider-neutral interface to large language models: messages, requests, responses, streaming and token usage.

Index

Constants

View Source
const FreeformInput = "input"

FreeformInput is the argument that carries the text input of a call to a freeform tool; see ToolSpec.Grammar.

Variables

This section is empty.

Functions

func CloneProviderOptions added in v1.1.0

func CloneProviderOptions(opts map[string]map[string]any) map[string]map[string]any

CloneProviderOptions copies opts and each provider's map; the option values remain shared.

func IsRetryable

func IsRetryable(err error) bool

IsRetryable reports whether err may succeed if the request is retried.

  • Provider errors follow APIError.Retryable.
  • Transport failures are retryable: errors from http.Client.Do, which come as *url.Error, and failures reading a response body, which the agentx providers also report as *url.Error. This covers refused, reset and dropped connections, HTTP/2 stream resets and GOAWAY, connections closed early and transport timeouts, including http.Client.Timeout and Transport.ResponseHeaderTimeout. Network errors outside a *url.Error, such as *net.OpError, unexpected EOF or a net.Error timeout, are retryable too.
  • Failures that recur on every attempt are not retryable even then: invalid URLs, certificate verification failures, TLS alerts from the server, other TLS handshake failures, DNS lookups of unknown hosts and a plain HTTP response to an HTTPS request.
  • The caller's context being cancelled or past its deadline is not retryable. A timeout of the HTTP client is not the caller's deadline and is retryable, although it also matches context.DeadlineExceeded. Retry loops should still stop once their context is done.
  • Errors marked Permanent and all other errors, such as invalid requests or undecodable responses, are not retryable.

func Permanent

func Permanent(err error) error

Permanent wraps err so that IsRetryable reports false for it. It returns nil for a nil err.

func RetryAfter

func RetryAfter(err error) time.Duration

RetryAfter returns the delay requested by the provider in err, or 0.

func StreamFromResponse

func StreamFromResponse(resp *Response) iter.Seq2[*StreamEvent, error]

StreamFromResponse converts a complete response into a stream with one delta per part followed by the done event. A nil response yields an error.

func WrapReminder

func WrapReminder(text string) string

WrapReminder wraps text in a <system-reminder> block, the convention for context that the application, not the user, adds to a message.

Types

type APIError

type APIError struct {
	Provider   string
	StatusCode int
	// Type is the provider's error type, or its code when it reports no
	// type.
	Type string
	// Code is the provider's error code, when it reports one alongside or
	// instead of a type.
	Code    string
	Message string
	// RetryAfter is the delay requested by the provider, when available.
	RetryAfter time.Duration
}

APIError is an error returned by a model provider's API.

func (*APIError) Error

func (e *APIError) Error() string

func (*APIError) Retryable

func (e *APIError) Retryable() bool

Retryable reports whether the request may succeed if retried.

type Accumulator

type Accumulator struct {
	// contains filtered or unexported fields
}

Accumulator assembles stream deltas into a complete message.

func (*Accumulator) Add

func (a *Accumulator) Add(ev *StreamEvent)

Add merges a delta event into the accumulated message.

func (*Accumulator) Message

func (a *Accumulator) Message() *Message

Message returns the accumulated assistant message.

type Describer

type Describer interface {
	Describe() Description
}

Describer is implemented by models that describe themselves for telemetry. The agentx providers implement it, and model wrappers should forward it.

type Description

type Description struct {
	// Provider is the gen_ai.provider.name, such as "openai", "anthropic"
	// or "gcp.gemini".
	Provider string
	// Operation is the gen_ai.operation.name of a call, such as "chat" or
	// "generate_content".
	Operation string
	// Model is the model used when a request names none.
	Model string
	// ServerAddress and ServerPort locate the API endpoint.
	ServerAddress string
	ServerPort    int
}

Description identifies a model for telemetry. Names follow the OpenTelemetry GenAI semantic conventions.

func Describe

func Describe(m Model) Description

Describe returns m's description, or a zero Description if m does not implement Describer.

type Effort added in v1.1.0

type Effort string

Effort is a reasoning effort level. Providers map the levels to what a model supports, using the nearest supported level.

const (
	EffortNone    Effort = "none"
	EffortMinimal Effort = "minimal"
	EffortLow     Effort = "low"
	EffortMedium  Effort = "medium"
	EffortHigh    Effort = "high"
	EffortXHigh   Effort = "xhigh"
	EffortMax     Effort = "max"
)

Reasoning effort levels, from none to max.

type FinishReason

type FinishReason string

FinishReason explains why the model stopped generating.

const (
	FinishStop      FinishReason = "stop"
	FinishLength    FinishReason = "length"
	FinishToolCalls FinishReason = "tool_calls"
	// FinishRefusal reports that the provider declined the request.
	FinishRefusal FinishReason = "refusal"
	// FinishContentFilter reports that output was withheld by a filter.
	FinishContentFilter FinishReason = "content_filter"
	// FinishPause reports that a long-running server-side turn was paused and
	// should be continued by sending the conversation again.
	FinishPause   FinishReason = "pause"
	FinishUnknown FinishReason = "unknown"
)

type Func

type Func func(ctx context.Context, req *Request) (*Response, error)

Func adapts a generate function into a Model. Stream is derived from Generate.

func (Func) Generate

func (f Func) Generate(ctx context.Context, req *Request) (*Response, error)

Generate calls f.

func (Func) Stream

func (f Func) Stream(ctx context.Context, req *Request) iter.Seq2[*StreamEvent, error]

Stream calls f and streams its response. A nil response without an error yields an error.

type Message

type Message struct {
	Role     Role           `json:"role"`
	Parts    []Part         `json:"parts"`
	Name     string         `json:"name,omitempty"`
	Metadata map[string]any `json:"metadata,omitempty"`
}

Message is a single conversation turn.

func AssistantText

func AssistantText(text string) *Message

AssistantText returns an assistant message containing text.

func CloneMessages

func CloneMessages(msgs []*Message) []*Message

CloneMessages copies a message slice, cloning each message with Message.Clone.

func NewMessage

func NewMessage(role Role, parts ...Part) *Message

NewMessage returns a message with the given role and parts.

func SystemText

func SystemText(text string) *Message

SystemText returns a system message containing text.

func ToolResults

func ToolResults(results ...*ToolResult) *Message

ToolResults returns a tool message carrying the given results.

func UserText

func UserText(text string) *Message

UserText returns a user message containing text.

func (*Message) Clone

func (m *Message) Clone() *Message

Clone copies the message and its parts, including media data and tool payloads. Metadata maps are copied shallowly: their values remain shared. A nil receiver returns nil.

func (*Message) Text

func (m *Message) Text() string

Text concatenates the message's text parts.

func (*Message) ToolCalls

func (m *Message) ToolCalls() []*ToolCall

ToolCalls returns the tool calls in the message, in order.

func (*Message) ToolResults

func (m *Message) ToolResults() []*ToolResult

ToolResults returns the tool results in the message, in order.

type Model

type Model interface {
	// Generate returns a complete response.
	Generate(ctx context.Context, req *Request) (*Response, error)
	// Stream yields incremental events and finishes with a StreamDone event
	// that carries the complete response. A non-nil error ends the stream.
	Stream(ctx context.Context, req *Request) iter.Seq2[*StreamEvent, error]
}

Model generates responses from a conversation.

Implementations must be safe for concurrent use.

type Part

type Part struct {
	Type PartType `json:"type"`

	// Text holds text content, the visible reasoning for PartReasoning, or
	// the transcript of audio a model produced.
	Text string `json:"text,omitempty"`

	// MediaType, Data and URL describe an image, document, audio or video
	// part: its content is either Data, a URL or FileID. Data is encoded as
	// base64 in JSON.
	MediaType string `json:"media_type,omitempty"`
	Data      []byte `json:"data,omitempty"`
	URL       string `json:"url,omitempty"`
	// FileID refers to a file uploaded to the provider's Files API, such
	// as an OpenAI or Anthropic file ID, in place of Data or URL. Gemini
	// files are referenced by their URI in URL. Set Provider to the
	// provider that issued the ID, so other providers reject the part
	// instead of sending an ID they do not know.
	FileID string `json:"file_id,omitempty"`
	// Filename names a document, audio or video part. Providers send it as
	// the file name or title where the API takes one.
	Filename string `json:"filename,omitempty"`
	// Detail is the resolution at which providers that support it process
	// an image or video: "low", "medium", "high" or "auto". It maps to
	// OpenAI's detail and Gemini's mediaResolution.
	Detail string `json:"detail,omitempty"`
	// Citations asks providers that support it, Anthropic and Bedrock, to
	// cite a document part in the answer.
	Citations bool `json:"citations,omitempty"`

	// Signature is an opaque provider token that must be returned unchanged
	// with the part, such as a reasoning signature.
	Signature string `json:"signature,omitempty"`
	// Redacted marks reasoning whose content is encrypted in Data.
	Redacted bool `json:"redacted,omitempty"`

	ToolCall   *ToolCall   `json:"tool_call,omitempty"`
	ToolResult *ToolResult `json:"tool_result,omitempty"`

	// Provider and Raw hold a PartRaw block. Provider also names the
	// provider that issued Signature, when the provider records it, so
	// signatures are not sent to providers that cannot verify them.
	Provider string          `json:"provider,omitempty"`
	Raw      json.RawMessage `json:"raw,omitempty"`
}

Part is one piece of message content.

func AudioPart

func AudioPart(mediaType string, data []byte) Part

AudioPart returns an inline audio part.

func AudioURLPart

func AudioURLPart(url string) Part

AudioURLPart returns an audio part referenced by URL.

func DocumentPart

func DocumentPart(mediaType string, data []byte, filename string) Part

DocumentPart returns an inline document part, such as a PDF, named filename. The name may be empty.

func DocumentURLPart

func DocumentURLPart(url string) Part

DocumentURLPart returns a document part referenced by URL.

func FilePart

func FilePart(t PartType, provider, fileID, mediaType string) Part

FilePart returns a part of type t, such as PartDocument or PartImage, referring to the file fileID that was uploaded to provider's Files API. mediaType may be empty when the provider knows the file's type.

func ImagePart

func ImagePart(mediaType string, data []byte) Part

ImagePart returns an inline image part.

func ImageURLPart

func ImageURLPart(url string) Part

ImageURLPart returns an image part referenced by URL.

func MediaPart

func MediaPart(mediaType string, data []byte) Part

MediaPart returns an inline part whose type follows mediaType: image/* becomes an image part, audio/* an audio part, video/* a video part and any other type a document part.

func TextDocumentPart

func TextDocumentPart(title, text string) Part

TextDocumentPart returns a plain text document part titled title. Providers with document support, such as Anthropic and Bedrock, can cite it.

func TextPart

func TextPart(text string) Part

TextPart returns a text part.

func VideoPart

func VideoPart(mediaType string, data []byte) Part

VideoPart returns an inline video part.

func VideoURLPart

func VideoURLPart(url string) Part

VideoURLPart returns a video part referenced by URL.

func (Part) IsMedia

func (p Part) IsMedia() bool

IsMedia reports whether p is an image, document, audio or video part.

type PartType

type PartType string

PartType identifies the kind of content held by a Part.

const (
	PartText  PartType = "text"
	PartImage PartType = "image"
	// PartDocument is a document, such as a PDF or a plain text file.
	PartDocument PartType = "document"
	// PartAudio is an audio clip.
	PartAudio PartType = "audio"
	// PartVideo is a video clip.
	PartVideo      PartType = "video"
	PartReasoning  PartType = "reasoning"
	PartToolCall   PartType = "tool_call"
	PartToolResult PartType = "tool_result"
	// PartRaw carries a provider-specific content block that has no neutral
	// representation. Providers echo raw parts they produced back verbatim and
	// ignore raw parts from other providers.
	PartRaw PartType = "raw"
)

func MediaPartType

func MediaPartType(mediaType string) PartType

MediaPartType returns the part type for content of mediaType; see MediaPart.

type PermanentError

type PermanentError struct{ Err error }

PermanentError marks an error that retrying cannot fix, such as a request that fails validation before it is sent.

func (*PermanentError) Error

func (e *PermanentError) Error() string

func (*PermanentError) Unwrap

func (e *PermanentError) Unwrap() error

type Reasoning

type Reasoning struct {
	// Effort is the reasoning effort; empty leaves it to the provider.
	Effort Effort `json:"effort,omitempty"`
	// Display controls whether reasoning text is returned, for providers
	// that support it; empty leaves it to the provider.
	Display ReasoningDisplay `json:"display,omitempty"`
	// BudgetTokens limits reasoning tokens for providers that use budgets.
	BudgetTokens int `json:"budget_tokens,omitempty"`
}

Reasoning configures model reasoning ("thinking").

type ReasoningDisplay added in v1.1.0

type ReasoningDisplay string

ReasoningDisplay controls whether reasoning text is returned.

const (
	// DisplaySummarized returns summarized reasoning text.
	DisplaySummarized ReasoningDisplay = "summarized"
	// DisplayOmitted returns no reasoning text.
	DisplayOmitted ReasoningDisplay = "omitted"
)

Reasoning display modes.

type Request

type Request struct {
	// Model overrides the provider's configured model name.
	Model string `json:"model,omitempty"`
	// System is the system prompt.
	System   string     `json:"system,omitempty"`
	Messages []*Message `json:"messages"`

	Tools      []*ToolSpec `json:"tools,omitempty"`
	ToolChoice *ToolChoice `json:"tool_choice,omitempty"`

	MaxTokens   int      `json:"max_tokens,omitempty"`
	Temperature *float64 `json:"temperature,omitempty"`
	// Seed asks providers that support it, such as OpenAI and Gemini, for
	// deterministic sampling.
	Seed *int     `json:"seed,omitempty"`
	TopP *float64 `json:"top_p,omitempty"`
	Stop []string `json:"stop,omitempty"`

	Reasoning      *Reasoning      `json:"reasoning,omitempty"`
	ResponseFormat *ResponseFormat `json:"response_format,omitempty"`

	// ProviderOptions holds provider-specific request fields, keyed by the
	// provider adapter: "openai", "anthropic", "gemini" or "bedrock", the
	// name in Part.Provider. An adapter merges only its own map into the
	// request body, overriding generated fields with the same name, so one
	// request can carry options for several providers, as a fallback
	// between providers needs.
	ProviderOptions map[string]map[string]any `json:"provider_options,omitempty"`
}

Request is a provider-neutral generation request.

func (*Request) Clone

func (r *Request) Clone() *Request

Clone copies the request's Messages, Tools and Stop slices and the maps of ProviderOptions. Messages, tool specs, option pointers and the values in ProviderOptions remain shared.

type Response

type Response struct {
	ID           string       `json:"id,omitempty"`
	Model        string       `json:"model,omitempty"`
	Message      *Message     `json:"message"`
	FinishReason FinishReason `json:"finish_reason"`
	Usage        Usage        `json:"usage"`
}

Response is a complete model output.

func Collect

func Collect(stream iter.Seq2[*StreamEvent, error]) (*Response, error)

Collect consumes a stream and returns its final response. A done event without a response or without a message gets the message accumulated from the deltas. Nil events are skipped.

func (*Response) Text

func (r *Response) Text() string

Text returns the text of the response message.

func (*Response) ToolCalls

func (r *Response) ToolCalls() []*ToolCall

ToolCalls returns the tool calls requested by the response.

type ResponseFormat

type ResponseFormat struct {
	Name   string          `json:"name"`
	Schema json.RawMessage `json:"schema"`
	Strict bool            `json:"strict,omitempty"`
}

ResponseFormat constrains the response to JSON matching Schema.

type Role

type Role string

Role identifies the author of a message.

const (
	RoleSystem    Role = "system"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	// RoleTool marks a message that carries tool results back to the model.
	RoleTool Role = "tool"
)

type StreamEvent

type StreamEvent struct {
	Type StreamEventType `json:"type"`
	// Index is the position of the part that Delta extends.
	Index int `json:"index"`
	// Delta is a partial part. Text, Signature and ToolCall.Arguments are
	// appended to the part at Index; other fields are set when first seen.
	Delta *Part `json:"delta,omitempty"`
	// Response is set on StreamDone.
	Response *Response `json:"response,omitempty"`
}

StreamEvent is one event of a streaming response.

type StreamEventType

type StreamEventType string

StreamEventType identifies a streaming event.

const (
	// StreamDelta carries an incremental piece of a message part.
	StreamDelta StreamEventType = "delta"
	// StreamDone is the final event and carries the complete response.
	StreamDone StreamEventType = "done"
)

type ToolCall

type ToolCall struct {
	ID        string          `json:"id"`
	Name      string          `json:"name"`
	Arguments json.RawMessage `json:"arguments,omitempty"`
}

ToolCall is a model request to invoke a tool.

type ToolChoice

type ToolChoice struct {
	Mode ToolChoiceMode `json:"mode"`
	Name string         `json:"name,omitempty"`
}

ToolChoice selects the tool-calling mode. Name is used with ToolChoiceTool.

Some recent models reject forced tool use (required or a named tool); use auto with a prompt instruction for those models.

type ToolChoiceMode

type ToolChoiceMode string

ToolChoiceMode controls whether and which tools the model must call.

const (
	ToolChoiceAuto     ToolChoiceMode = "auto"
	ToolChoiceNone     ToolChoiceMode = "none"
	ToolChoiceRequired ToolChoiceMode = "required"
	ToolChoiceTool     ToolChoiceMode = "tool"
)

type ToolGrammar

type ToolGrammar struct {
	// Syntax is the grammar language, "lark" or "regex".
	Syntax string `json:"syntax"`
	// Definition is the grammar. An empty definition accepts any text.
	Definition string `json:"definition,omitempty"`
}

ToolGrammar constrains the text input of a freeform tool.

type ToolResult

type ToolResult struct {
	CallID  string `json:"call_id"`
	Name    string `json:"name,omitempty"`
	Content string `json:"content"`
	// Parts holds content that follows Content, such as the images and
	// documents a tool returned. Providers put them in the tool result
	// where the API allows it, or else in a user message after the
	// results; see their package documentation.
	Parts   []Part `json:"parts,omitempty"`
	IsError bool   `json:"is_error,omitempty"`
	// Metadata is what the tool reported for the application in
	// tool.Result.Metadata. It is not sent to models, and is stored with
	// the conversation, so its values should be encodable as JSON.
	Metadata map[string]any `json:"metadata,omitempty"`
}

ToolResult is the outcome of a tool call.

type ToolSpec

type ToolSpec struct {
	Name        string `json:"name"`
	Description string `json:"description,omitempty"`
	// Parameters is a JSON Schema object describing the arguments.
	Parameters json.RawMessage `json:"parameters,omitempty"`
	// Strict asks providers that support it to guarantee schema-valid
	// arguments.
	Strict bool `json:"strict,omitempty"`
	// Grammar, when set, makes the tool freeform where the provider
	// supports it, as the OpenAI Responses API does: the model writes the
	// input as plain text matching the grammar instead of JSON, and the
	// provider passes the text as the string argument FreeformInput.
	// Other providers send Parameters, which must declare that argument.
	Grammar *ToolGrammar `json:"grammar,omitempty"`
}

ToolSpec describes a tool the model may call.

type Usage

type Usage struct {
	InputTokens  int `json:"input_tokens"`
	OutputTokens int `json:"output_tokens"`
	// CacheReadTokens and CacheWriteTokens count prompt-cache activity. They
	// are reported separately from InputTokens by providers that distinguish
	// them.
	CacheReadTokens  int `json:"cache_read_tokens,omitempty"`
	CacheWriteTokens int `json:"cache_write_tokens,omitempty"`
	// ReasoningTokens is the portion of OutputTokens spent on reasoning when
	// the provider reports it.
	ReasoningTokens int `json:"reasoning_tokens,omitempty"`
	// Requests and ToolCalls count the model calls and executed tool calls
	// of a run, including those of nested runs such as sub-agents. The
	// runtime counts them; providers leave them zero.
	Requests  int `json:"requests,omitempty"`
	ToolCalls int `json:"tool_calls,omitempty"`
}

Usage reports token consumption for one or more model calls.

func (*Usage) Add

func (u *Usage) Add(other Usage)

Add accumulates other into u.

func (Usage) PromptTokens

func (u Usage) PromptTokens() int

PromptTokens returns every token the model read as input.

func (Usage) TotalTokens

func (u Usage) TotalTokens() int

TotalTokens returns all input and output tokens, including cache activity.

Directories

Path Synopsis
Package modeltest provides a scripted model for tests.
Package modeltest provides a scripted model for tests.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL