Documentation
¶
Index ¶
- func APIKeyEnvVars(provider string) []string
- func FindEnvAPIKeys(provider string) []string
- func FindEnvAPIKeysWithEnv(provider string, env ProviderEnv) []string
- func GetEnvAPIKey(provider string) string
- func GetEnvAPIKeyWithEnv(provider string, env ProviderEnv) string
- func GetProviders() []string
- func IsContextOverflow(message AssistantMessage, contextWindow int64) bool
- func OverflowPatterns() []*regexp.Regexp
- func ParseToolArguments(raw string) map[string]any
- func RepairJSON(source string) string
- func ValidateToolArguments(tool ToolDefinition, toolCall ToolCall) (map[string]any, error)
- func ValidateToolCall(tools []ToolDefinition, toolCall ToolCall) (map[string]any, error)
- type AnthropicMessagesCompatibility
- type AssistantContent
- type AssistantMessage
- type Client
- type Context
- type Event
- type EventType
- type ImageContent
- type Message
- type Model
- type ModelCompatibility
- type ModelCost
- type ModelInput
- type ModelRegistry
- type ModelThinkingLevel
- type OpenAICompletionsCompatibility
- type Protocol
- type ProtocolAdapter
- type ProviderEnv
- type Registry
- type StopReason
- type StreamOptions
- type TextContent
- type ThinkingContent
- type ToolCall
- type ToolDefinition
- type ToolResultContent
- type ToolResultMessage
- type Usage
- type UsageCost
- type UserContent
- type UserMessage
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func APIKeyEnvVars ¶
APIKeyEnvVars returns the environment variables checked for provider in precedence order. The returned slice is safe for the caller to modify.
func FindEnvAPIKeys ¶
FindEnvAPIKeys returns the names of configured API key environment variables for provider, in lookup order.
func FindEnvAPIKeysWithEnv ¶
func FindEnvAPIKeysWithEnv(provider string, env ProviderEnv) []string
FindEnvAPIKeysWithEnv is FindEnvAPIKeys with request-scoped overrides.
func GetEnvAPIKey ¶
GetEnvAPIKey returns the first configured API key for provider.
func GetEnvAPIKeyWithEnv ¶
func GetEnvAPIKeyWithEnv(provider string, env ProviderEnv) string
GetEnvAPIKeyWithEnv returns the first configured API key for provider, preferring non-empty request-scoped values over process environment values.
func GetProviders ¶
func GetProviders() []string
GetProviders returns all providers in the built-in model registry.
func IsContextOverflow ¶
func IsContextOverflow(message AssistantMessage, contextWindow int64) bool
IsContextOverflow reports whether an assistant message indicates that the input exceeded the model's context window. It covers three cases:
- Error-based overflow: most providers return StopReasonError with a recognizable error message (matched against overflowPatterns, excluding nonOverflowPatterns such as rate limits).
- Silent overflow (e.g. z.ai): the request succeeds but usage.Input exceeds the context window. Pass a non-zero contextWindow to detect this.
- Length-stop overflow (e.g. Xiaomi MiMo): the server truncates oversized input to fill the window, leaving no room to generate, so it returns StopReasonLength with zero output and input filling the window.
Pass contextWindow as the model's window size to enable cases 2 and 3; pass 0 to check error messages only.
func OverflowPatterns ¶
OverflowPatterns returns a copy of the overflow detection patterns, primarily for tests.
func ParseToolArguments ¶
ParseToolArguments decodes a tool call's accumulated JSON arguments into an object. Models occasionally emit JSON with unescaped control characters or bad escape sequences; a strict decode of that input loses every argument. So this first tries a strict decode, then retries on a repaired copy, and only then gives up. It always returns a non-nil map so callers have a usable value.
func RepairJSON ¶
RepairJSON fixes malformed JSON string literals by escaping raw control characters inside strings and doubling backslashes before invalid escape characters, while preserving valid escapes and \uXXXX sequences. Only ASCII characters carry special meaning here, so iterating over bytes leaves multi-byte UTF-8 sequences untouched.
func ValidateToolArguments ¶
func ValidateToolArguments(tool ToolDefinition, toolCall ToolCall) (map[string]any, error)
ValidateToolArguments coerces a tool call's arguments toward the tool's JSON Schema (forgiving common model mistakes such as "3" for a number), then validates them. It returns the coerced arguments, or a detailed error naming the failing fields. The original toolCall.Arguments are left unchanged.
The coercion mirrors pi's coerceWithJsonSchema. Validation covers the JSON Schema features normally emitted for tool definitions, including composition keywords and object, array, string, and numeric constraints.
func ValidateToolCall ¶
func ValidateToolCall(tools []ToolDefinition, toolCall ToolCall) (map[string]any, error)
ValidateToolCall finds the tool named by the call and validates its arguments. It mirrors pi's helper: a utility callers may invoke before dispatching a tool, not something the library calls itself. It returns the coerced arguments.
Types ¶
type AnthropicMessagesCompatibility ¶
type AnthropicMessagesCompatibility struct {
SupportsTemperature *bool `json:"supportsTemperature,omitempty"`
SupportsCacheControl *bool `json:"supportsCacheControl,omitempty"`
SupportsCacheControlTools *bool `json:"supportsCacheControlOnTools,omitempty"`
ForceAdaptiveThinking *bool `json:"forceAdaptiveThinking,omitempty"`
AllowEmptySignature *bool `json:"allowEmptySignature,omitempty"`
}
AnthropicMessagesCompatibility describes differences between providers that implement an Anthropic Messages-compatible endpoint. Pointer booleans distinguish an explicit false value from an unspecified provider default. Anthropic-compatible vendors (e.g. MiniMax) are served by pointing the base URL at their endpoint; most need no overrides at all.
func (*AnthropicMessagesCompatibility) Protocol ¶
func (*AnthropicMessagesCompatibility) Protocol() Protocol
Protocol identifies the API protocol whose request and message dialect this compatibility configuration describes.
type AssistantContent ¶
type AssistantContent interface {
// contains filtered or unexported methods
}
AssistantContent is content that can appear in an assistant message.
type AssistantMessage ¶
type AssistantMessage struct {
Content []AssistantContent `json:"content"`
Protocol Protocol `json:"protocol"`
Provider string `json:"provider"`
Model string `json:"model"`
ResponseModel string `json:"responseModel,omitempty"`
ResponseID string `json:"responseId,omitempty"`
Usage Usage `json:"usage"`
StopReason StopReason `json:"stopReason"`
ErrorMessage string `json:"errorMessage,omitempty"`
Timestamp int64 `json:"timestamp"`
}
AssistantMessage is the final or partial response returned by a provider.
func NewAssistantMessage ¶
func NewAssistantMessage(model Model) AssistantMessage
NewAssistantMessage initializes provider-independent response metadata.
func (AssistantMessage) MarshalJSON ¶
func (message AssistantMessage) MarshalJSON() ([]byte, error)
func (*AssistantMessage) UnmarshalJSON ¶
func (message *AssistantMessage) UnmarshalJSON(data []byte) error
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client routes LLM requests to the adapter registered for a model protocol.
func (*Client) Complete ¶
func (c *Client) Complete( ctx context.Context, model Model, input Context, options StreamOptions, ) (AssistantMessage, error)
Complete consumes a provider stream and returns the final assistant message.
type Context ¶
type Context struct {
SystemPrompt string `json:"systemPrompt,omitempty"`
Messages []Message `json:"messages"`
Tools []ToolDefinition `json:"tools,omitempty"`
}
Context contains the prompt, conversation history, and available tools.
func (Context) MarshalJSON ¶
func (*Context) UnmarshalJSON ¶
type Event ¶
type Event struct {
Type EventType
ContentIndex int
Delta string
Content string
ToolCall *ToolCall
Partial *AssistantMessage
Message *AssistantMessage
Err error
}
Event is a single update emitted while streaming a provider response.
type EventType ¶
type EventType string
EventType identifies the kind of update emitted by a provider stream.
const ( // EventStart marks the beginning of a provider stream. EventStart EventType = "start" // EventTextStart marks the creation of a text content block. EventTextStart EventType = "text_start" // EventTextDelta carries newly generated text. EventTextDelta EventType = "text_delta" // EventTextEnd carries the completed text content block. EventTextEnd EventType = "text_end" // EventThinkingStart marks the creation of a reasoning content block. EventThinkingStart EventType = "thinking_start" // EventThinkingDelta carries newly generated reasoning content. EventThinkingDelta EventType = "thinking_delta" // EventThinkingEnd carries the completed reasoning content block. EventThinkingEnd EventType = "thinking_end" // EventToolCallStart marks the creation of a tool call content block. EventToolCallStart EventType = "toolcall_start" // EventToolCallDelta carries a fragment of a tool call's arguments as it streams. EventToolCallDelta EventType = "toolcall_delta" // EventToolCallEnd carries a completed tool call request. EventToolCallEnd EventType = "toolcall_end" // EventDone carries the final assistant message. EventDone EventType = "done" // EventError carries a stream failure. EventError EventType = "error" )
type ImageContent ¶
ImageContent represents a base64-encoded image.
func (ImageContent) MarshalJSON ¶
func (content ImageContent) MarshalJSON() ([]byte, error)
type Message ¶
type Message interface {
// contains filtered or unexported methods
}
Message is one item in the conversation context.
func TransformMessages ¶
func TransformMessages(messages []Message, model Model, normalizeToolCallID func(string) string) []Message
TransformMessages prepares the library's provider-independent conversation history for replay against model. Provider adapters should call it before translating Message values into their own wire-format message types.
Transformation happens per request instead of modifying the Agent's canonical history. The same history may later be sent to a model with different image, reasoning, or tool capabilities. Mutating stored history would make that model switch lose information permanently.
Shared transformations are applied in this order:
- Replace images with descriptive text when the target model is text-only.
- Reconcile assistant turns produced by a different model: keep reasoning for the same model, downgrade it to text otherwise, and normalize tool-call identifiers via normalizeToolCallID when crossing providers.
- Drop assistant turns terminated by an error or cancellation because they may contain partial reasoning or half-streamed tool calls.
- Insert synthetic error results for tool calls with no matching result before the conversation continues or ends.
normalizeToolCallID rewrites a tool-call ID for the target provider; pass nil to leave identifiers unchanged.
The returned slice is new and this function does not mutate messages. Message objects requiring no changes may still be shared with the input, so callers should treat the input and result as immutable.
type Model ¶
type Model struct {
ID string `json:"id"`
Name string `json:"name"`
Protocol Protocol `json:"protocol"`
Provider string `json:"provider"`
BaseURL string `json:"baseUrl"`
Reasoning bool `json:"reasoning"`
ThinkingLevelMap map[ModelThinkingLevel]*string `json:"thinkingLevelMap,omitempty"`
Input []ModelInput `json:"input"`
Cost ModelCost `json:"cost"`
ContextWindow int64 `json:"contextWindow"`
MaxTokens int64 `json:"maxTokens"`
Headers map[string]string `json:"headers,omitempty"`
Compatibility ModelCompatibility `json:"compat,omitempty"`
}
Model identifies a model, its provider endpoint, capabilities, limits, and pricing. ThinkingLevelMap values are provider-specific; nil marks a level as unsupported while a missing key uses the provider default.
func GetModel ¶
GetModel returns a model from the package's built-in model registry. It panics when the provider/model pair is unknown. Use LookupModel when the identifiers come from dynamic or untrusted input.
func LookupModel ¶
LookupModel returns a model from the package's built-in model registry.
func (*Model) UnmarshalJSON ¶
UnmarshalJSON restores the concrete compatibility type selected by Protocol. The protocol acts as the discriminator, mirroring pi's Model<TApi> conditional compatibility type at runtime.
type ModelCompatibility ¶
type ModelCompatibility interface {
Protocol() Protocol
}
ModelCompatibility is implemented by protocol-specific compatibility configurations. It keeps Model independent from any one provider protocol while allowing registration and adapters to verify type/protocol agreement.
type ModelCost ¶
type ModelCost struct {
Input float64 `json:"input"`
Output float64 `json:"output"`
CacheRead float64 `json:"cacheRead"`
CacheWrite float64 `json:"cacheWrite"`
}
ModelCost stores prices in US dollars per million tokens.
type ModelInput ¶
type ModelInput string
ModelInput identifies an input modality accepted by a model.
const ( Text ModelInput = "text" Image ModelInput = "image" )
type ModelRegistry ¶
type ModelRegistry struct {
// contains filtered or unexported fields
}
ModelRegistry stores models by provider and model ID. It is safe for concurrent access and returns defensive copies of registered models.
func NewModelRegistry ¶
func NewModelRegistry() *ModelRegistry
NewModelRegistry creates an empty model registry.
func (*ModelRegistry) Get ¶
func (registry *ModelRegistry) Get(provider, modelID string) (Model, bool)
Get returns a model registered for provider and modelID.
func (*ModelRegistry) Models ¶
func (registry *ModelRegistry) Models(provider string) []Model
Models returns a provider's models ordered by model ID.
func (*ModelRegistry) Providers ¶
func (registry *ModelRegistry) Providers() []string
Providers returns registered provider IDs in lexical order.
func (*ModelRegistry) Register ¶
func (registry *ModelRegistry) Register(model Model) error
Register adds or replaces a model with the same provider and ID.
type ModelThinkingLevel ¶
type ModelThinkingLevel string
ModelThinkingLevel is a provider-independent reasoning effort level.
const ( ModelThinkingOff ModelThinkingLevel = "off" ModelThinkingMinimal ModelThinkingLevel = "minimal" ModelThinkingLow ModelThinkingLevel = "low" ModelThinkingMedium ModelThinkingLevel = "medium" ModelThinkingHigh ModelThinkingLevel = "high" ModelThinkingXHigh ModelThinkingLevel = "xhigh" )
func ClampThinkingLevel ¶
func ClampThinkingLevel(model Model, level ModelThinkingLevel) ModelThinkingLevel
ClampThinkingLevel adjusts a requested level to the nearest one the model supports: it prefers the requested level, then steps up, then down, and falls back to the lowest supported level (or "off").
func SupportedThinkingLevels ¶
func SupportedThinkingLevels(model Model) []ModelThinkingLevel
SupportedThinkingLevels returns the thinking levels a model accepts. A non-reasoning model supports only "off". For reasoning models, a level mapped to nil is unsupported, and "xhigh" is supported only when explicitly mapped.
type OpenAICompletionsCompatibility ¶
type OpenAICompletionsCompatibility struct {
SupportsStore *bool `json:"supportsStore,omitempty"`
SupportsDeveloperRole *bool `json:"supportsDeveloperRole,omitempty"`
SupportsReasoningEffort *bool `json:"supportsReasoningEffort,omitempty"`
MaxTokensField string `json:"maxTokensField,omitempty"`
SupportsStrictMode *bool `json:"supportsStrictMode,omitempty"`
RequiresReasoningContentOnAssistantMessages *bool `json:"requiresReasoningContentOnAssistantMessages,omitempty"`
// RequiresThinkingAsText makes replayed assistant turns carry thinking as a
// leading text content block instead of a provider reasoning field, for
// endpoints that reject reasoning fields on input.
RequiresThinkingAsText *bool `json:"requiresThinkingAsText,omitempty"`
ThinkingFormat string `json:"thinkingFormat,omitempty"`
ZAIToolStream *bool `json:"zaiToolStream,omitempty"`
}
OpenAICompletionsCompatibility describes differences between providers that implement an OpenAI-compatible Chat Completions endpoint. Pointer booleans distinguish an explicit false value from an unspecified provider default.
func (*OpenAICompletionsCompatibility) Protocol ¶
func (*OpenAICompletionsCompatibility) Protocol() Protocol
Protocol identifies the API protocol whose request and message dialect this compatibility configuration describes.
type Protocol ¶
type Protocol string
Protocol identifies the API protocol used to communicate with a model.
type ProtocolAdapter ¶
type ProtocolAdapter interface {
// Protocol returns the registry key used to select this adapter.
Protocol() Protocol
// Stream emits response events for the given model and conversation context.
Stream(ctx context.Context, model Model, input Context, options StreamOptions) (<-chan Event, error)
}
ProtocolAdapter translates between a concrete LLM protocol and the package streaming interface.
type ProviderEnv ¶
ProviderEnv contains request-scoped environment overrides. Non-empty values take precedence over process environment variables during credential lookup.
type Registry ¶
type Registry struct {
// contains filtered or unexported fields
}
Registry stores protocol adapters and is safe for concurrent access.
func (*Registry) Get ¶
func (registry *Registry) Get(protocol Protocol) (ProtocolAdapter, bool)
Get returns the adapter registered for the protocol.
func (*Registry) Register ¶
func (registry *Registry) Register(adapter ProtocolAdapter) error
Register adds or replaces an adapter for its protocol.
type StopReason ¶
type StopReason string
StopReason explains why the model stopped generating a response.
const ( // StopReasonStop marks a normal completion. StopReasonStop StopReason = "stop" // StopReasonLength marks truncation by the max output token limit. StopReasonLength StopReason = "length" // StopReasonToolUse marks a stop to let the caller execute tool calls. StopReasonToolUse StopReason = "toolUse" // StopReasonError marks a provider or runtime failure. StopReasonError StopReason = "error" // StopReasonAborted marks a cancelled request. StopReasonAborted StopReason = "aborted" )
type StreamOptions ¶
type StreamOptions struct {
APIKey string
Env ProviderEnv
// Temperature overrides the model's default sampling temperature when set.
Temperature *float64
// MaxTokens caps the output tokens for this request. Zero leaves it unset.
MaxTokens int64
// Headers are merged into the request, overriding model default headers.
Headers map[string]string
// Reasoning requests a thinking level. The provider clamps it to what the
// model supports. Empty leaves the model's default; "off" disables thinking.
Reasoning ModelThinkingLevel
}
StreamOptions contains provider-specific settings for a stream request.
type TextContent ¶
type TextContent struct {
Text string `json:"text"`
TextSignature string `json:"textSignature,omitempty"`
}
TextContent represents plain text.
func (TextContent) MarshalJSON ¶
func (content TextContent) MarshalJSON() ([]byte, error)
type ThinkingContent ¶
type ThinkingContent struct {
Thinking string `json:"thinking"`
ThinkingSignature string `json:"thinkingSignature,omitempty"`
Redacted bool `json:"redacted,omitempty"`
}
ThinkingContent represents model reasoning content.
func (ThinkingContent) MarshalJSON ¶
func (content ThinkingContent) MarshalJSON() ([]byte, error)
type ToolCall ¶
type ToolCall struct {
ID string `json:"id"`
Name string `json:"name"`
Arguments map[string]any `json:"arguments"`
ThoughtSignature string `json:"thoughtSignature,omitempty"`
}
ToolCall describes a request to invoke a named tool with JSON arguments.
func (ToolCall) MarshalJSON ¶
type ToolDefinition ¶
type ToolDefinition struct {
Name string `json:"name"`
Description string `json:"description"`
Parameters json.RawMessage `json:"parameters"`
}
ToolDefinition describes a tool that the model may call.
type ToolResultContent ¶
type ToolResultContent interface {
// contains filtered or unexported methods
}
ToolResultContent is content that can appear in a tool result message.
type ToolResultMessage ¶
type ToolResultMessage struct {
ToolCallID string `json:"toolCallId"`
ToolName string `json:"toolName"`
Content []ToolResultContent `json:"content"`
IsError bool `json:"isError"`
}
ToolResultMessage contains the result of an assistant tool call.
func (ToolResultMessage) MarshalJSON ¶
func (message ToolResultMessage) MarshalJSON() ([]byte, error)
func (*ToolResultMessage) UnmarshalJSON ¶
func (message *ToolResultMessage) UnmarshalJSON(data []byte) error
type Usage ¶
type Usage struct {
Input int64 `json:"input"`
Output int64 `json:"output"`
CacheRead int64 `json:"cacheRead"`
CacheWrite int64 `json:"cacheWrite"`
TotalTokens int64 `json:"totalTokens"`
Cost UsageCost `json:"cost"`
}
Usage records token consumption for one assistant response.
type UsageCost ¶
type UsageCost struct {
Input float64 `json:"input"`
Output float64 `json:"output"`
CacheRead float64 `json:"cacheRead"`
CacheWrite float64 `json:"cacheWrite"`
Total float64 `json:"total"`
}
UsageCost breaks down the US dollar cost of one response by token category.
func CalculateCost ¶
CalculateCost returns the US dollar cost of usage at the model's prices. Model costs are quoted per million tokens.
type UserContent ¶
type UserContent interface {
// contains filtered or unexported methods
}
UserContent is content that can appear in a user message.
type UserMessage ¶
type UserMessage struct {
Content []UserContent `json:"content"`
}
UserMessage contains content supplied by the user.
func (UserMessage) MarshalJSON ¶
func (message UserMessage) MarshalJSON() ([]byte, error)
func (*UserMessage) UnmarshalJSON ¶
func (message *UserMessage) UnmarshalJSON(data []byte) error
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
genmodels
command
Command genmodels builds llm's checked-in model catalog from the same public catalogs used by pi-ai.
|
Command genmodels builds llm's checked-in model catalog from the same public catalogs used by pi-ai. |
|
providers
|
|
|
anthropic
Package anthropic implements the Anthropic Messages protocol on top of the official anthropic-sdk-go.
|
Package anthropic implements the Anthropic Messages protocol on top of the official anthropic-sdk-go. |