Documentation
¶
Overview ¶
Package llms provides unified support for interacting with different Language Models (LLMs) from various providers. Designed with an extensible architecture, the package facilitates seamless integration of LLMs with a focus on modularity, encapsulation, and easy configurability.
The package includes the following subpackages for LLM providers: 1. Hugging Face: llms/huggingface/ 2. Local LLM: llms/local/ 3. OpenAI: llms/openai/ 4. Google AI: llms/googleai/ 5. Cohere: llms/cohere/
Each subpackage includes provider-specific LLM implementations and helper files for communication with supported LLM providers. The internal directories within these subpackages contain provider-specific client and API implementations.
The `llms.go` file contains the types and interfaces for interacting with different LLMs.
The `options.go` file provides various options and functions to configure the LLMs.
Index ¶
- Variables
- func CalculateMaxTokens(model, text string) int
- func CalculateThinkingBudget(mode ThinkingMode, maxTokens int) int
- func CountTokens(model, text string) int
- func DefaultIsReasoningModel(model string) bool
- func GenerateFromSinglePrompt(ctx context.Context, llm Model, prompt string, options ...CallOption) (string, error)
- func GetBufferString(messages []ChatMessage, humanPrefix string, aiPrefix string) (string, error)
- func GetModelContextSize(model string) int
- func IsAuthenticationError(err error) bool
- func IsCanceledError(err error) bool
- func IsContentFilterError(err error) bool
- func IsInvalidRequestError(err error) bool
- func IsNotImplementedError(err error) bool
- func IsProviderUnavailableError(err error) bool
- func IsQuotaExceededError(err error) bool
- func IsRateLimitError(err error) bool
- func IsReasoningModel(model string) bool
- func IsTimeoutError(err error) bool
- func IsTokenLimitError(err error) bool
- func ShowMessageContents(w io.Writer, msgs []MessageContent)
- func SupportsReasoningModel(llm interface{}) bool
- type AIChatMessage
- type BinaryContent
- type CacheControl
- type CachedContent
- type CallOption
- func WithCandidateCount(c int) CallOption
- func WithFrequencyPenalty(frequencyPenalty float64) CallOption
- func WithFunctionCallBehavior(behavior FunctionCallBehavior) CallOption
- func WithFunctions(functions []FunctionDefinition) CallOption
- func WithInterleaveThinking(enabled bool) CallOption
- func WithJSONMode() CallOption
- func WithMaxLength(maxLength int) CallOption
- func WithMaxTokens(maxTokens int) CallOption
- func WithMetadata(metadata map[string]interface{}) CallOption
- func WithMinLength(minLength int) CallOption
- func WithModel(model string) CallOption
- func WithN(n int) CallOption
- func WithOptions(options CallOptions) CallOption
- func WithPresencePenalty(presencePenalty float64) CallOption
- func WithPromptCaching(enabled bool) CallOption
- func WithRepetitionPenalty(repetitionPenalty float64) CallOption
- func WithResponseMIMEType(responseMIMEType string) CallOption
- func WithReturnThinking(enabled bool) CallOption
- func WithSeed(seed int) CallOption
- func WithStopWords(stopWords []string) CallOption
- func WithStreamThinking(enabled bool) CallOption
- func WithStreamingFunc(streamingFunc func(ctx context.Context, chunk []byte) error) CallOption
- func WithStreamingReasoningFunc(...) CallOption
- func WithTemperature(temperature float64) CallOption
- func WithThinking(config *ThinkingConfig) CallOption
- func WithThinkingBudget(tokens int) CallOption
- func WithThinkingMode(mode ThinkingMode) CallOption
- func WithToolChoice(choice any) CallOption
- func WithTools(tools []Tool) CallOption
- func WithTopK(topK int) CallOption
- func WithTopP(topP float64) CallOption
- type CallOptions
- type ChatMessage
- type ChatMessageModel
- type ChatMessageModelData
- type ChatMessageType
- type ContentChoice
- type ContentPart
- type ContentResponse
- type Error
- type ErrorCode
- type ErrorMapper
- type ErrorMatcher
- type FunctionCall
- type FunctionCallBehavior
- type FunctionChatMessage
- type FunctionDefinition
- type FunctionReference
- type GenericChatMessage
- type HumanChatMessage
- type ImageURLContent
- type LLMdeprecated
- type MessageContent
- type Model
- type Named
- type PromptValue
- type ReasoningModel
- type SystemChatMessage
- type TextContent
- type ThinkingConfig
- type ThinkingContent
- type ThinkingMode
- type ThinkingTokenUsage
- type Tool
- type ToolCall
- type ToolCallResponse
- type ToolChatMessage
- type ToolChoice
Constants ¶
This section is empty.
Variables ¶
var ( // ErrAuthentication is returned when authentication fails. ErrAuthentication = &Error{Code: ErrCodeAuthentication} // ErrRateLimit is returned when rate limit is exceeded. ErrRateLimit = &Error{Code: ErrCodeRateLimit} // ErrInvalidRequest is returned for invalid requests. ErrInvalidRequest = &Error{Code: ErrCodeInvalidRequest} // ErrTimeout is returned when an operation times out. ErrTimeout = &Error{Code: ErrCodeTimeout} // ErrCanceled is returned when an operation is canceled. ErrCanceled = &Error{Code: ErrCodeCanceled} // ErrQuotaExceeded is returned when quota is exceeded. ErrQuotaExceeded = &Error{Code: ErrCodeQuotaExceeded} // ErrContentFilter is returned when content is filtered. ErrContentFilter = &Error{Code: ErrCodeContentFilter} // ErrTokenLimit is returned when token limit is exceeded. ErrTokenLimit = &Error{Code: ErrCodeTokenLimit} ErrProviderUnavailable = &Error{Code: ErrCodeProviderUnavailable} // ErrNotImplemented is returned when a feature is not implemented. ErrNotImplemented = &Error{Code: ErrCodeNotImplemented} )
Common error variables for easy comparison.
var ErrUnexpectedChatMessageType = errors.New("unexpected chat message type")
ErrUnexpectedChatMessageType is returned when a chat message is of an unexpected type.
Functions ¶
func CalculateMaxTokens ¶
CalculateMaxTokens calculates the max number of tokens that could be added to a text.
func CalculateThinkingBudget ¶
func CalculateThinkingBudget(mode ThinkingMode, maxTokens int) int
CalculateThinkingBudget calculates the token budget based on mode and max tokens.
func CountTokens ¶
CountTokens gets the number of tokens the text contains.
func DefaultIsReasoningModel ¶
DefaultIsReasoningModel provides the default reasoning model detection logic. This can be used by LLM implementations that want to extend rather than replace the default detection logic.
func GenerateFromSinglePrompt ¶
func GenerateFromSinglePrompt(ctx context.Context, llm Model, prompt string, options ...CallOption) (string, error)
GenerateFromSinglePrompt is a convenience function for calling an LLM with a single string prompt, expecting a single string response. It's useful for simple, string-only interactions and provides a slightly more ergonomic API than the more general llms.Model.GenerateContent.
func GetBufferString ¶
func GetBufferString(messages []ChatMessage, humanPrefix string, aiPrefix string) (string, error)
GetBufferString gets the buffer string of messages.
func GetModelContextSize ¶
GetModelContextSize gets the max number of tokens for a language model. If the model name isn't recognized the default value 2048 is returned.
func IsAuthenticationError ¶
IsAuthenticationError returns true if the error is an authentication error.
func IsCanceledError ¶
IsCanceledError returns true if the error is a cancellation error.
func IsContentFilterError ¶
IsContentFilterError returns true if the error is a content filter error.
func IsInvalidRequestError ¶
IsInvalidRequestError returns true if the error is an invalid request error.
func IsNotImplementedError ¶
IsNotImplementedError returns true if the error is a not implemented error.
func IsProviderUnavailableError ¶
IsProviderUnavailableError returns true if the error is a provider unavailable error.
func IsQuotaExceededError ¶
IsQuotaExceededError returns true if the error is a quota exceeded error.
func IsRateLimitError ¶
IsRateLimitError returns true if the error is a rate limit error.
func IsReasoningModel ¶
IsReasoningModel returns true if the model is a reasoning/thinking model. This includes OpenAI o1/o3/GPT-5 series, Anthropic Claude 3.7+, DeepSeek reasoner, etc. For runtime checking of LLM instances, use SupportsReasoningModel instead.
func IsTimeoutError ¶
IsTimeoutError returns true if the error is a timeout error.
func IsTokenLimitError ¶
IsTokenLimitError returns true if the error is a token limit error.
func ShowMessageContents ¶
func ShowMessageContents(w io.Writer, msgs []MessageContent)
ShowMessageContents is a debugging helper for MessageContent.
func SupportsReasoningModel ¶
func SupportsReasoningModel(llm interface{}) bool
SupportsReasoningModel checks if an LLM instance supports reasoning tokens. This first checks if the LLM implements the ReasoningModel interface. If not, it falls back to checking the model string if available.
Types ¶
type AIChatMessage ¶
type AIChatMessage struct {
// Content is the content of the message.
Content string `json:"content,omitempty"`
// FunctionCall represents the model choosing to call a function.
FunctionCall *FunctionCall `json:"function_call,omitempty"`
// ToolCalls represents the model choosing to call tools.
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
// ReasoningContent is used with deepseek-reasoner model and represents
// the reasoning contents of the assistant message before the final answer.
ReasoningContent string `json:"reasoning_content,omitempty"`
// ThinkingContent contains the model's thinking/reasoning text.
// Used by Gemini 3.0 and other models that support extended thinking.
ThinkingContent string `json:"thinking_content,omitempty"`
// ThoughtSignature is an opaque verification token for Gemini 3.0 thinking.
// This must be persisted and returned in subsequent API calls to validate
// the reasoning context in multi-turn conversations.
ThoughtSignature string `json:"thought_signature,omitempty"`
}
AIChatMessage is a message sent by an AI.
func (AIChatMessage) GetContent ¶
func (m AIChatMessage) GetContent() string
func (AIChatMessage) GetFunctionCall ¶
func (m AIChatMessage) GetFunctionCall() *FunctionCall
func (AIChatMessage) GetType ¶
func (m AIChatMessage) GetType() ChatMessageType
type BinaryContent ¶
BinaryContent is content holding some binary data with a MIME type.
func BinaryPart ¶
func BinaryPart(mime string, data []byte) BinaryContent
BinaryPart creates a new BinaryContent from the given MIME type (e.g. "image/png" and binary data).
func (BinaryContent) MarshalJSON ¶
func (bc BinaryContent) MarshalJSON() ([]byte, error)
func (BinaryContent) String ¶
func (bc BinaryContent) String() string
func (*BinaryContent) UnmarshalJSON ¶
func (bc *BinaryContent) UnmarshalJSON(data []byte) error
type CacheControl ¶
type CacheControl struct {
// Type specifies the type of caching (provider-specific, e.g., "ephemeral").
Type string `json:"type,omitempty"`
// Duration specifies cache lifetime (provider-specific limits apply).
Duration time.Duration `json:"-"`
}
CacheControl represents prompt caching configuration for providers that support it.
type CachedContent ¶
type CachedContent struct {
ContentPart
CacheControl *CacheControl `json:"cache_control,omitempty"`
}
CachedContent represents content with caching instructions.
func WithCacheControl ¶
func WithCacheControl(content ContentPart, control *CacheControl) CachedContent
WithCacheControl wraps content with cache control instructions.
type CallOption ¶
type CallOption func(*CallOptions)
CallOption is a function that configures a CallOptions.
func WithCandidateCount ¶
func WithCandidateCount(c int) CallOption
WithCandidateCount specifies the number of response candidates to generate.
func WithFrequencyPenalty ¶
func WithFrequencyPenalty(frequencyPenalty float64) CallOption
WithFrequencyPenalty will add an option to set the frequency penalty for sampling.
func WithFunctionCallBehavior ¶
func WithFunctionCallBehavior(behavior FunctionCallBehavior) CallOption
WithFunctionCallBehavior will add an option to set the behavior to use when calling functions. Deprecated: Use WithToolChoice instead.
func WithFunctions ¶
func WithFunctions(functions []FunctionDefinition) CallOption
WithFunctions will add an option to set the functions to include in the request. Deprecated: Use WithTools instead.
func WithInterleaveThinking ¶
func WithInterleaveThinking(enabled bool) CallOption
WithInterleaveThinking enables interleaved thinking between tool calls. Provider support varies - check your provider's documentation.
func WithJSONMode ¶
func WithJSONMode() CallOption
WithJSONMode will add an option to set the response format to JSON. This is useful for models that return structured data.
func WithMaxLength ¶
func WithMaxLength(maxLength int) CallOption
WithMaxLength will add an option to set the maximum length of the generated text.
func WithMaxTokens ¶
func WithMaxTokens(maxTokens int) CallOption
WithMaxTokens specifies the max number of tokens to generate.
func WithMetadata ¶
func WithMetadata(metadata map[string]interface{}) CallOption
WithMetadata will add an option to set metadata to include in the request. The meaning of this field is specific to the backend in use.
func WithMinLength ¶
func WithMinLength(minLength int) CallOption
WithMinLength will add an option to set the minimum length of the generated text.
func WithModel ¶
func WithModel(model string) CallOption
WithModel specifies which model name to use.
func WithN ¶
func WithN(n int) CallOption
WithN will add an option to set how many chat completion choices to generate for each input message.
func WithPresencePenalty ¶
func WithPresencePenalty(presencePenalty float64) CallOption
WithPresencePenalty will add an option to set the presence penalty for sampling.
func WithPromptCaching ¶
func WithPromptCaching(enabled bool) CallOption
WithPromptCaching adds cache control metadata to call options. This is a generic option that can be used by any provider that supports caching. Providers should check for this metadata and handle it appropriately.
func WithRepetitionPenalty ¶
func WithRepetitionPenalty(repetitionPenalty float64) CallOption
WithRepetitionPenalty will add an option to set the repetition penalty for sampling.
func WithResponseMIMEType ¶
func WithResponseMIMEType(responseMIMEType string) CallOption
WithResponseMIMEType will add an option to set the ResponseMIMEType. Provider support varies - check your provider's documentation.
func WithReturnThinking ¶
func WithReturnThinking(enabled bool) CallOption
WithReturnThinking enables returning thinking/reasoning in the response.
func WithSeed ¶
func WithSeed(seed int) CallOption
WithSeed will add an option to use deterministic sampling.
func WithStopWords ¶
func WithStopWords(stopWords []string) CallOption
WithStopWords specifies a list of words to stop generation on.
func WithStreamThinking ¶
func WithStreamThinking(enabled bool) CallOption
WithStreamThinking enables streaming of thinking tokens.
func WithStreamingFunc ¶
func WithStreamingFunc(streamingFunc func(ctx context.Context, chunk []byte) error) CallOption
WithStreamingFunc specifies the streaming function to use.
func WithStreamingReasoningFunc ¶
func WithStreamingReasoningFunc(streamingReasoningFunc func(ctx context.Context, reasoningChunk, chunk []byte) error) CallOption
WithStreamingReasoningFunc specifies the streaming reasoning function to use.
func WithTemperature ¶
func WithTemperature(temperature float64) CallOption
WithTemperature specifies the model temperature, a hyperparameter that regulates the randomness, or creativity, of the AI's responses.
func WithThinking ¶
func WithThinking(config *ThinkingConfig) CallOption
WithThinking adds thinking configuration to call options.
func WithThinkingBudget ¶
func WithThinkingBudget(tokens int) CallOption
WithThinkingBudget sets explicit token budget for thinking.
func WithThinkingMode ¶
func WithThinkingMode(mode ThinkingMode) CallOption
WithThinkingMode sets the thinking mode for the request.
func WithToolChoice ¶
func WithToolChoice(choice any) CallOption
WithToolChoice will add an option to set the choice of tool to use. It can either be "none", "auto" (the default behavior), or a specific tool as described in the ToolChoice type.
func WithTools ¶
func WithTools(tools []Tool) CallOption
WithTools will add an option to set the tools to use.
func WithTopK ¶
func WithTopK(topK int) CallOption
WithTopK will add an option to use top-k sampling.
func WithTopP ¶
func WithTopP(topP float64) CallOption
WithTopP will add an option to use top-p sampling.
type CallOptions ¶
type CallOptions struct {
// Model is the model to use.
Model string `json:"model"`
// CandidateCount is the number of response candidates to generate.
CandidateCount int `json:"candidate_count"`
// MaxTokens is the maximum number of tokens to generate.
MaxTokens int `json:"max_tokens"`
// Temperature is the temperature for sampling, between 0 and 1.
Temperature float64 `json:"temperature"`
// StopWords is a list of words to stop on.
StopWords []string `json:"stop_words"`
// StreamingFunc is a function to be called for each chunk of a streaming response.
// Return an error to stop streaming early.
StreamingFunc func(ctx context.Context, chunk []byte) error `json:"-"`
// StreamingReasoningFunc is a function to be called for each chunk of a streaming response.
// Return an error to stop streaming early.
StreamingReasoningFunc func(ctx context.Context, reasoningChunk, chunk []byte) error `json:"-"`
// TopK is the number of tokens to consider for top-k sampling.
TopK int `json:"top_k"`
// TopP is the cumulative probability for top-p sampling.
TopP float64 `json:"top_p"`
// Seed is a seed for deterministic sampling.
Seed int `json:"seed"`
// MinLength is the minimum length of the generated text.
MinLength int `json:"min_length"`
// MaxLength is the maximum length of the generated text.
MaxLength int `json:"max_length"`
// N is how many chat completion choices to generate for each input message.
N int `json:"n"`
// RepetitionPenalty is the repetition penalty for sampling.
RepetitionPenalty float64 `json:"repetition_penalty"`
// FrequencyPenalty is the frequency penalty for sampling.
FrequencyPenalty float64 `json:"frequency_penalty"`
// PresencePenalty is the presence penalty for sampling.
PresencePenalty float64 `json:"presence_penalty"`
// JSONMode is a flag to enable JSON mode.
JSONMode bool `json:"json"`
// Tools is a list of tools to use. Each tool can be a specific tool or a function.
Tools []Tool `json:"tools,omitempty"`
// ToolChoice is the choice of tool to use, it can either be "none", "auto" (the default behavior), or a specific tool as described in the ToolChoice type.
ToolChoice any `json:"tool_choice"`
// Function defitions to include in the request.
// Deprecated: Use Tools instead.
Functions []FunctionDefinition `json:"functions,omitempty"`
// FunctionCallBehavior is the behavior to use when calling functions.
//
// If a specific function should be invoked, use the format:
// `{"name": "my_function"}`
// Deprecated: Use ToolChoice instead.
FunctionCallBehavior FunctionCallBehavior `json:"function_call,omitempty"`
// Metadata is a map of metadata to include in the request.
// The meaning of this field is specific to the backend in use.
Metadata map[string]interface{} `json:"metadata,omitempty"`
// ResponseMIMEType MIME type of the generated candidate text.
// Supported MIME types are: text/plain: (default) Text output.
// application/json: JSON response in the response candidates.
ResponseMIMEType string `json:"response_mime_type,omitempty"`
}
CallOptions is a set of options for calling models. Not all models support all options.
type ChatMessage ¶
type ChatMessage interface {
// GetType gets the type of the message.
GetType() ChatMessageType
// GetContent gets the content of the message.
GetContent() string
}
ChatMessage represents a message in a chat.
type ChatMessageModel ¶
type ChatMessageModel struct {
Type string `bson:"type" json:"type"`
Data ChatMessageModelData `bson:"data" json:"data"`
}
func ConvertChatMessageToModel ¶
func ConvertChatMessageToModel(m ChatMessage) ChatMessageModel
ConvertChatMessageToModel Convert a ChatMessage to a ChatMessageModel.
func (ChatMessageModel) ToChatMessage ¶
func (c ChatMessageModel) ToChatMessage() ChatMessage
type ChatMessageModelData ¶
type ChatMessageType ¶
type ChatMessageType string
ChatMessageType is the type of chat message.
const ( // ChatMessageTypeAI is a message sent by an AI. ChatMessageTypeAI ChatMessageType = "ai" // ChatMessageTypeHuman is a message sent by a human. ChatMessageTypeHuman ChatMessageType = "human" // ChatMessageTypeSystem is a message sent by the system. ChatMessageTypeSystem ChatMessageType = "system" // ChatMessageTypeGeneric is a message sent by a generic user. ChatMessageTypeGeneric ChatMessageType = "generic" // ChatMessageTypeFunction is a message sent by a function. ChatMessageTypeFunction ChatMessageType = "function" // ChatMessageTypeTool is a message sent by a tool. ChatMessageTypeTool ChatMessageType = "tool" )
type ContentChoice ¶
type ContentChoice struct {
// Content is the textual content of a response
Content string
// StopReason is the reason the model stopped generating output.
StopReason string
// GenerationInfo is arbitrary information the model adds to the response.
GenerationInfo map[string]any
// FuncCall is non-nil when the model asks to invoke a function/tool.
// If a model invokes more than one function/tool, this field will only
// contain the first one.
FuncCall *FunctionCall
// ToolCalls is a list of tool calls the model asks to invoke.
ToolCalls []ToolCall
// ReasoningContent is used with deepseek-reasoner model and represents
// the reasoning contents of the assistant message before the final answer.
ReasoningContent string
// ThinkingContent contains the model's thinking/reasoning text.
// Used by Gemini 3.0 and other models that support extended thinking.
ThinkingContent string
// ThoughtSignature is an opaque verification token for Gemini 3.0 thinking.
// This must be persisted and returned in subsequent API calls to validate
// the reasoning context in multi-turn conversations.
ThoughtSignature string
}
ContentChoice is one of the response choices returned by GenerateContent calls.
type ContentPart ¶
type ContentPart interface {
// contains filtered or unexported methods
}
ContentPart is an interface all parts of content have to implement.
type ContentResponse ¶
type ContentResponse struct {
Choices []*ContentChoice
}
ContentResponse is the response returned by a GenerateContent call. It can potentially return multiple content choices.
type Error ¶
type Error struct {
// Code is the standardized error code.
Code ErrorCode
// Message is a human-readable error message.
Message string
// Provider is the name of the provider that generated the error.
Provider string
// Details contains provider-specific error details.
Details map[string]interface{}
// Cause is the underlying error, if any.
Cause error
}
Error represents a standardized error from an LLM provider.
func (*Error) WithDetail ¶
WithDetail adds a detail to the error.
type ErrorCode ¶
type ErrorCode string
ErrorCode represents a standardized error code for LLM operations.
const ( // ErrCodeUnknown indicates an unknown error. ErrCodeUnknown ErrorCode = "unknown" // ErrCodeAuthentication indicates an authentication failure. ErrCodeAuthentication ErrorCode = "authentication" // ErrCodeRateLimit indicates a rate limit has been exceeded. ErrCodeRateLimit ErrorCode = "rate_limit" // ErrCodeInvalidRequest indicates the request was invalid. ErrCodeInvalidRequest ErrorCode = "invalid_request" // ErrCodeResourceNotFound indicates a requested resource was not found. ErrCodeResourceNotFound ErrorCode = "resource_not_found" // ErrCodeTimeout indicates the operation timed out. ErrCodeTimeout ErrorCode = "timeout" // ErrCodeCanceled indicates the operation was canceled. ErrCodeCanceled ErrorCode = "canceled" // ErrCodeQuotaExceeded indicates a quota has been exceeded. ErrCodeQuotaExceeded ErrorCode = "quota_exceeded" // ErrCodeContentFilter indicates content was blocked by safety filters. ErrCodeContentFilter ErrorCode = "content_filter" // ErrCodeTokenLimit indicates the token limit was exceeded. ErrCodeTokenLimit ErrorCode = "token_limit" ErrCodeProviderUnavailable ErrorCode = "provider_unavailable" // ErrCodeNotImplemented indicates a feature is not implemented. ErrCodeNotImplemented ErrorCode = "not_implemented" )
type ErrorMapper ¶
type ErrorMapper struct {
// contains filtered or unexported fields
}
ErrorMapper helps map provider-specific errors to standardized errors.
func AnthropicErrorMapper ¶
func AnthropicErrorMapper() *ErrorMapper
AnthropicErrorMapper creates an error mapper with Anthropic-specific patterns.
func GoogleAIErrorMapper ¶
func GoogleAIErrorMapper() *ErrorMapper
GoogleAIErrorMapper creates an error mapper with Google AI-specific patterns.
func NewErrorMapper ¶
func NewErrorMapper(provider string) *ErrorMapper
NewErrorMapper creates a new error mapper for a provider.
func OpenAIErrorMapper ¶
func OpenAIErrorMapper() *ErrorMapper
OpenAIErrorMapper creates an error mapper with OpenAI-specific patterns.
func (*ErrorMapper) AddMatcher ¶
func (m *ErrorMapper) AddMatcher(matcher ErrorMatcher) *ErrorMapper
AddMatcher adds a custom error matcher.
func (*ErrorMapper) Map ¶
func (m *ErrorMapper) Map(err error) error
Map is an alias for WrapError for consistency with provider error mappers.
func (*ErrorMapper) WrapError ¶
func (m *ErrorMapper) WrapError(err error) error
WrapError wraps an error with standardized error information.
type ErrorMatcher ¶
type ErrorMatcher struct {
// Match returns true if this matcher handles the error
Match func(error) bool
// Code is the error code to use
Code ErrorCode
// Transform optionally transforms the error message
Transform func(error) string
}
ErrorMatcher matches an error and returns the appropriate error code.
type FunctionCall ¶
type FunctionCall struct {
// The name of the function to call.
Name string `json:"name"`
// The arguments to pass to the function, as a JSON string.
Arguments string `json:"arguments"`
}
FunctionCall is the name and arguments of a function call.
type FunctionCallBehavior ¶
type FunctionCallBehavior string
FunctionCallBehavior is the behavior to use when calling functions.
const ( // FunctionCallBehaviorNone will not call any functions. FunctionCallBehaviorNone FunctionCallBehavior = "none" // FunctionCallBehaviorAuto will call functions automatically. FunctionCallBehaviorAuto FunctionCallBehavior = "auto" )
type FunctionChatMessage ¶
type FunctionChatMessage struct {
// Name is the name of the function.
Name string `json:"name"`
// Content is the content of the function message.
Content string `json:"content"`
}
FunctionChatMessage is a chat message representing the result of a function call. Deprecated: Use ToolChatMessage instead.
func (FunctionChatMessage) GetContent ¶
func (m FunctionChatMessage) GetContent() string
func (FunctionChatMessage) GetName ¶
func (m FunctionChatMessage) GetName() string
func (FunctionChatMessage) GetType ¶
func (m FunctionChatMessage) GetType() ChatMessageType
type FunctionDefinition ¶
type FunctionDefinition struct {
// Name is the name of the function.
Name string `json:"name"`
// Description is a description of the function.
Description string `json:"description"`
// Parameters is a list of parameters for the function.
Parameters any `json:"parameters,omitempty"`
// Strict is a flag to indicate if the function should be called strictly.
// Provider support varies - typically used for structured output guarantees.
Strict bool `json:"strict,omitempty"`
}
FunctionDefinition is a definition of a function that can be called by the model.
type FunctionReference ¶
type FunctionReference struct {
// Name is the name of the function.
Name string `json:"name"`
}
FunctionReference is a reference to a function.
type GenericChatMessage ¶
GenericChatMessage is a chat message with an arbitrary speaker.
func (GenericChatMessage) GetContent ¶
func (m GenericChatMessage) GetContent() string
func (GenericChatMessage) GetName ¶
func (m GenericChatMessage) GetName() string
func (GenericChatMessage) GetType ¶
func (m GenericChatMessage) GetType() ChatMessageType
type HumanChatMessage ¶
type HumanChatMessage struct {
Content string
}
HumanChatMessage is a message sent by a human.
func (HumanChatMessage) GetContent ¶
func (m HumanChatMessage) GetContent() string
func (HumanChatMessage) GetType ¶
func (m HumanChatMessage) GetType() ChatMessageType
type ImageURLContent ¶
type ImageURLContent struct {
URL string `json:"url"`
Detail string `json:"detail,omitempty"` // Detail is the detail of the image, e.g. "low", "high".
}
ImageURLContent is content with an URL pointing to an image.
func ImageURLPart ¶
func ImageURLPart(url string) ImageURLContent
ImageURLPart creates a new ImageURLContent from the given URL.
func ImageURLWithDetailPart ¶
func ImageURLWithDetailPart(url string, detail string) ImageURLContent
ImageURLWithDetailPart creates a new ImageURLContent from the given URL and detail.
func (ImageURLContent) MarshalJSON ¶
func (iuc ImageURLContent) MarshalJSON() ([]byte, error)
func (ImageURLContent) String ¶
func (iuc ImageURLContent) String() string
func (*ImageURLContent) UnmarshalJSON ¶
func (iuc *ImageURLContent) UnmarshalJSON(data []byte) error
type MessageContent ¶
type MessageContent struct {
Role ChatMessageType
Parts []ContentPart
}
MessageContent is the content of a message sent to a LLM. It has a role and a sequence of parts. For example, it can represent one message in a chat session sent by the user, in which case Role will be ChatMessageTypeHuman and Parts will be the sequence of items sent in this specific message.
func TextParts ¶
func TextParts(role ChatMessageType, parts ...string) MessageContent
TextParts is a helper function to create a MessageContent with a role and a list of text parts.
func (MessageContent) MarshalJSON ¶
func (mc MessageContent) MarshalJSON() ([]byte, error)
func (*MessageContent) UnmarshalJSON ¶
func (mc *MessageContent) UnmarshalJSON(data []byte) error
type Model ¶
type Model interface {
// GenerateContent asks the model to generate content from a sequence of
// messages. It's the most general interface for multi-modal LLMs that support
// chat-like interactions.
GenerateContent(ctx context.Context, messages []MessageContent, options ...CallOption) (*ContentResponse, error)
// Call is a simplified interface for a text-only Model, generating a single
// string response from a single string prompt.
//
// Deprecated: this method is retained for backwards compatibility. Use the
// more general [GenerateContent] instead. You can also use
// the [GenerateFromSinglePrompt] function which provides a similar capability
// to Call and is built on top of the new interface.
Call(ctx context.Context, prompt string, options ...CallOption) (string, error)
}
Model is an interface multi-modal models implement.
type Named ¶
type Named interface {
GetName() string
}
Named is an interface for objects that have a name.
type PromptValue ¶
type PromptValue interface {
String() string
Messages() []ChatMessage
}
PromptValue is the interface that all prompt values must implement.
type ReasoningModel ¶
type ReasoningModel interface {
Model
// SupportsReasoning returns true if the model supports reasoning/thinking tokens.
// This capability allows models to "think" through problems internally before
// generating a response, improving quality for complex tasks.
SupportsReasoning() bool
}
ReasoningModel is an interface for models that support extended reasoning/thinking. Models implementing this interface can generate internal reasoning tokens that are used to improve response quality but may not be included in the final output.
type SystemChatMessage ¶
type SystemChatMessage struct {
Content string
}
SystemChatMessage is a chat message representing information that should be instructions to the AI system.
func (SystemChatMessage) GetContent ¶
func (m SystemChatMessage) GetContent() string
func (SystemChatMessage) GetType ¶
func (m SystemChatMessage) GetType() ChatMessageType
type TextContent ¶
type TextContent struct {
Text string
}
TextContent is content with some text.
func TextPart ¶
func TextPart(s string) TextContent
TextPart creates TextContent from a given string.
func (TextContent) MarshalJSON ¶
func (tc TextContent) MarshalJSON() ([]byte, error)
func (TextContent) String ¶
func (tc TextContent) String() string
func (*TextContent) UnmarshalJSON ¶
func (tc *TextContent) UnmarshalJSON(data []byte) error
type ThinkingConfig ¶
type ThinkingConfig struct {
// Mode specifies the thinking mode (none, low, medium, high, auto).
Mode ThinkingMode `json:"mode,omitempty"`
// BudgetTokens sets explicit token budget for thinking.
// Providers may have different minimum and maximum limits.
BudgetTokens int `json:"budget_tokens,omitempty"`
// ReturnThinking controls whether thinking/reasoning is included in response.
// Provider support and behavior varies.
ReturnThinking bool `json:"return_thinking,omitempty"`
// StreamThinking enables streaming of thinking tokens as they're generated.
// Not all providers support this feature.
StreamThinking bool `json:"stream_thinking,omitempty"`
// InterleaveThinking enables thinking between tool calls.
// Provider support varies.
InterleaveThinking bool `json:"interleave_thinking,omitempty"`
}
ThinkingConfig configures thinking/reasoning behavior for models that support it.
func DefaultThinkingConfig ¶
func DefaultThinkingConfig() *ThinkingConfig
DefaultThinkingConfig returns a sensible default thinking configuration.
func GetThinkingConfig ¶
func GetThinkingConfig(opts *CallOptions) *ThinkingConfig
GetThinkingConfig safely retrieves thinking config from call options. Returns nil if no thinking config is present.
type ThinkingContent ¶
type ThinkingContent struct {
// Thinking is the model's reasoning/thinking text.
Thinking string `json:"thinking"`
// Signature is the opaque verification token that must be returned
// in subsequent API calls to validate the reasoning context.
Signature string `json:"signature,omitempty"`
}
ThinkingContent represents the model's thinking/reasoning with an optional verification signature. Used by Gemini 3.0 and similar models.
func ThinkingPart ¶
func ThinkingPart(thinking string) ThinkingContent
ThinkingPart creates a new ThinkingContent with the given thinking text.
func ThinkingPartWithSignature ¶
func ThinkingPartWithSignature(thinking, signature string) ThinkingContent
ThinkingPartWithSignature creates a new ThinkingContent with thinking text and a verification signature.
func (ThinkingContent) MarshalJSON ¶
func (tc ThinkingContent) MarshalJSON() ([]byte, error)
func (ThinkingContent) String ¶
func (tc ThinkingContent) String() string
func (*ThinkingContent) UnmarshalJSON ¶
func (tc *ThinkingContent) UnmarshalJSON(data []byte) error
type ThinkingMode ¶
type ThinkingMode string
ThinkingMode represents different thinking/reasoning modes for LLMs.
const ( // ThinkingModeNone disables thinking/reasoning. ThinkingModeNone ThinkingMode = "none" // ThinkingModeLow allocates minimal tokens for thinking (~20% of max tokens). ThinkingModeLow ThinkingMode = "low" // ThinkingModeMedium allocates moderate tokens for thinking (~50% of max tokens). ThinkingModeMedium ThinkingMode = "medium" // ThinkingModeHigh allocates maximum tokens for thinking (~80% of max tokens). ThinkingModeHigh ThinkingMode = "high" // ThinkingModeAuto lets the model decide how much thinking is needed. ThinkingModeAuto ThinkingMode = "auto" )
type ThinkingTokenUsage ¶
type ThinkingTokenUsage struct {
// ThinkingTokens is the total number of thinking/reasoning tokens used.
ThinkingTokens int `json:"thinking_tokens,omitempty"`
// ThinkingInputTokens is the number of input tokens used for thinking.
ThinkingInputTokens int `json:"thinking_input_tokens,omitempty"`
// ThinkingOutputTokens is the number of output tokens from thinking.
ThinkingOutputTokens int `json:"thinking_output_tokens,omitempty"`
// ThinkingCachedTokens is the number of cached thinking tokens (if applicable).
ThinkingCachedTokens int `json:"thinking_cached_tokens,omitempty"`
// ThinkingBudgetUsed is the actual budget used vs allocated.
ThinkingBudgetUsed int `json:"thinking_budget_used,omitempty"`
// ThinkingBudgetAllocated is the budget that was allocated.
ThinkingBudgetAllocated int `json:"thinking_budget_allocated,omitempty"`
}
ThinkingTokenUsage represents token usage specific to thinking/reasoning.
func ExtractThinkingTokens ¶
func ExtractThinkingTokens(generationInfo map[string]any) *ThinkingTokenUsage
ExtractThinkingTokens extracts thinking token information from generation info.
type Tool ¶
type Tool struct {
// Type is the type of the tool.
Type string `json:"type"`
// Function is the function to call.
Function *FunctionDefinition `json:"function,omitempty"`
}
Tool is a tool that can be used by the model.
type ToolCall ¶
type ToolCall struct {
// ID is the unique identifier of the tool call.
ID string `json:"id"`
// Type is the type of the tool call. Typically, this would be "function".
Type string `json:"type"`
// FunctionCall is the function call to be executed.
FunctionCall *FunctionCall `json:"function,omitempty"`
// ThoughtSignature is an opaque verification token for Gemini 3.0 thinking.
// When present, this must be included when sending tool call responses back
// to the model. Required for Gemini 3 Pro function calling.
ThoughtSignature string `json:"thought_signature,omitempty"`
}
ToolCall is a call to a tool (as requested by the model) that should be executed.
func (ToolCall) MarshalJSON ¶
func (*ToolCall) UnmarshalJSON ¶
type ToolCallResponse ¶
type ToolCallResponse struct {
// ToolCallID is the ID of the tool call this response is for.
ToolCallID string `json:"tool_call_id"`
// Name is the name of the tool that was called.
Name string `json:"name"`
// Content is the textual content of the response.
Content string `json:"content"`
}
ToolCallResponse is the response returned by a tool call.
func (ToolCallResponse) MarshalJSON ¶
func (tc ToolCallResponse) MarshalJSON() ([]byte, error)
func (*ToolCallResponse) UnmarshalJSON ¶
func (tc *ToolCallResponse) UnmarshalJSON(data []byte) error
type ToolChatMessage ¶
type ToolChatMessage struct {
// ID is the ID of the tool call.
ID string `json:"tool_call_id"`
// Content is the content of the tool message.
Content string `json:"content"`
}
ToolChatMessage is a chat message representing the result of a tool call.
func (ToolChatMessage) GetContent ¶
func (m ToolChatMessage) GetContent() string
func (ToolChatMessage) GetID ¶
func (m ToolChatMessage) GetID() string
func (ToolChatMessage) GetType ¶
func (m ToolChatMessage) GetType() ChatMessageType
type ToolChoice ¶
type ToolChoice struct {
// Type is the type of the tool.
Type string `json:"type"`
// Function is the function to call (if the tool is a function).
Function *FunctionReference `json:"function,omitempty"`
}
ToolChoice is a specific tool to use.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package cache provides a generic wrapper that adds caching to a `llms.Model`.
|
Package cache provides a generic wrapper that adds caching to a `llms.Model`. |
|
Package compliance provides a test suite to verify provider implementations.
|
Package compliance provides a test suite to verify provider implementations. |
|
Package ernie wrapper around the Baidu Large Language Model Platform APIs.
|
Package ernie wrapper around the Baidu Large Language Model Platform APIs. |
|
Package googleai provides caching support for Google AI models.
|
Package googleai provides caching support for Google AI models. |
|
internal/cmd
command
Code generator for vertex.go from googleai.go nolint
|
Code generator for vertex.go from googleai.go nolint |
|
palm
package palm implements a langchaingo provider for Google Vertex AI legacy PaLM models.
|
package palm implements a langchaingo provider for Google Vertex AI legacy PaLM models. |
|
vertex
package vertex implements a langchaingo provider for Google Vertex AI LLMs, including the new Gemini models.
|
package vertex implements a langchaingo provider for Google Vertex AI LLMs, including the new Gemini models. |
|
internal/localclient
Package localclient provides a client for local LLMs.
|
Package localclient provides a client for local LLMs. |
|
Package openai provides an interface to OpenAI's language models.
|
Package openai provides an interface to OpenAI's language models. |