Documentation
¶
Overview ¶
Package llm provides the LLM client interface for the agent loop.
This package defines the interface that LLM providers must implement to work with the agent. Actual implementations are injected at runtime.
Thread Safety:
All types in this package are designed for concurrent use.
Index ¶
- func FormatAsObservation(toolName string, success bool, output string) string
- func ParseToolCalls(response *Response) []agent.ToolInvocation
- func ParseToolCallsWithReAct(response *Response) ([]agent.ToolInvocation, bool)
- func ReActInstructions() string
- type AnthropicAgentAdapter
- type Client
- type CompletionCall
- type EmptyResponseError
- type GeminiAgentAdapter
- type Message
- type MockClient
- func (c *MockClient) CallCount() int
- func (c *MockClient) Complete(ctx context.Context, request *Request) (*Response, error)
- func (c *MockClient) ExpectCall(count int) error
- func (c *MockClient) GetCalls() []CompletionCall
- func (c *MockClient) LastRequest() *Request
- func (c *MockClient) Model() string
- func (c *MockClient) Name() string
- func (c *MockClient) QueueFinalResponse(content string) *MockClient
- func (c *MockClient) QueueResponse(response *Response) *MockClient
- func (c *MockClient) QueueToolCall(toolName string, arguments map[string]any) *MockClient
- func (c *MockClient) Reset()
- func (c *MockClient) SetDefaultResponse(response *Response) *MockClient
- func (c *MockClient) Verify() error
- func (c *MockClient) WithDelay(d time.Duration) *MockClient
- func (c *MockClient) WithError(err error) *MockClient
- func (c *MockClient) WithModel(model string) *MockClient
- func (c *MockClient) WithName(name string) *MockClient
- func (c *MockClient) WithResponseFunc(f func(*Request) (*Response, error)) *MockClient
- type OllamaAdapter
- type OpenAIAgentAdapter
- type ReActParseResult
- type Request
- type Response
- type ToolCall
- type ToolCallResult
- type ToolChoice
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func FormatAsObservation ¶
FormatAsObservation formats a tool result as a ReAct observation.
Description:
Converts a tool result into the "Observation:" format expected by the ReAct pattern. This is injected into the conversation after tool execution.
Inputs:
toolName - The name of the tool that was executed. success - Whether the tool execution succeeded. output - The tool output text (or error message if failed).
Outputs:
string - The formatted observation.
Example:
obs := FormatAsObservation("find_entry_points", true, "Found 5 entry points...")
// obs = "Observation: Found 5 entry points..."
Thread Safety: This function is safe for concurrent use.
func ParseToolCalls ¶
func ParseToolCalls(response *Response) []agent.ToolInvocation
ParseToolCalls extracts tool invocations from an LLM response.
Description:
Parses native tool_calls from the LLM response structure. For models that don't support native function calling, use ParseToolCallsWithReAct instead.
Inputs:
response - The LLM response
Outputs:
[]agent.ToolInvocation - Parsed tool invocations
Thread Safety: This function is safe for concurrent use.
func ParseToolCallsWithReAct ¶
func ParseToolCallsWithReAct(response *Response) ([]agent.ToolInvocation, bool)
ParseToolCallsWithReAct extracts tool invocations with ReAct fallback.
Description:
First attempts to parse native tool_calls from the response. If no native tool calls are found, falls back to parsing ReAct-style text format (Thought/Action/Action Input). This enables tool usage with models that don't support native function calling.
Inputs:
response - The LLM response
Outputs:
[]agent.ToolInvocation - Parsed tool invocations (from either source) bool - True if invocations came from ReAct parsing (not native)
Example:
invocations, usedReAct := ParseToolCallsWithReAct(response)
if usedReAct {
// Format next tool result as Observation
}
Thread Safety: This function is safe for concurrent use.
func ReActInstructions ¶
func ReActInstructions() string
ReActInstructions returns the system prompt addition for ReAct mode.
Description:
Returns the instruction text to append to the system prompt when ReAct mode is enabled. This teaches the LLM the ReAct format.
Outputs:
string - The ReAct instructions to append.
Thread Safety: This function is safe for concurrent use.
Types ¶
type AnthropicAgentAdapter ¶
type AnthropicAgentAdapter struct {
// contains filtered or unexported fields
}
AnthropicAgentAdapter adapts AnthropicClient to the agent's Client interface.
Description:
Wraps the existing AnthropicClient to provide LLM capabilities for the agent loop using Anthropic Claude models. Converts between agent message format and Anthropic's datatypes.Message format.
Thread Safety: AnthropicAgentAdapter is safe for concurrent use.
func NewAnthropicAgentAdapter ¶
func NewAnthropicAgentAdapter(client *llm.AnthropicClient, model string) *AnthropicAgentAdapter
NewAnthropicAgentAdapter creates a new AnthropicAgentAdapter.
Inputs:
- client: The AnthropicClient to wrap. Must not be nil.
- model: The model name for identification.
Outputs:
- *AnthropicAgentAdapter: The configured adapter.
func (*AnthropicAgentAdapter) Model ¶
func (a *AnthropicAgentAdapter) Model() string
Model implements Client.Model.
func (*AnthropicAgentAdapter) Name ¶
func (a *AnthropicAgentAdapter) Name() string
Name implements Client.Name.
type Client ¶
type Client interface {
// Complete sends a prompt to the LLM and returns a response.
//
// Inputs:
// ctx - Context for cancellation and timeout
// request - The completion request
//
// Outputs:
// *Response - The LLM response
// error - Non-nil if the request failed
Complete(ctx context.Context, request *Request) (*Response, error)
// Name returns the provider name (e.g., "anthropic", "openai").
Name() string
// Model returns the model being used.
Model() string
}
Client defines the interface for LLM interactions.
Implementations must be safe for concurrent use.
type CompletionCall ¶
CompletionCall records a call to Complete.
type EmptyResponseError ¶
type EmptyResponseError struct {
// Duration is how long the request took before returning empty.
Duration time.Duration
// MessageCount is the number of input messages sent.
MessageCount int
// Model is the model that returned the empty response.
Model string
}
EmptyResponseError indicates the LLM returned an empty response.
This typically indicates a problem with the model, context, or request. Empty responses should not silently pass through as they cause downstream issues in the agent loop.
Fixed in cb_30a.
func (*EmptyResponseError) Error ¶
func (e *EmptyResponseError) Error() string
Error implements the error interface.
type GeminiAgentAdapter ¶
type GeminiAgentAdapter struct {
// contains filtered or unexported fields
}
GeminiAgentAdapter adapts GeminiClient to the agent's Client interface.
Description:
Wraps the existing GeminiClient to provide LLM capabilities for the agent loop using Google Gemini models. Converts between agent message format and Gemini's datatypes.Message format.
Thread Safety: GeminiAgentAdapter is safe for concurrent use.
func NewGeminiAgentAdapter ¶
func NewGeminiAgentAdapter(client *llm.GeminiClient, model string) *GeminiAgentAdapter
NewGeminiAgentAdapter creates a new GeminiAgentAdapter.
Inputs:
- client: The GeminiClient to wrap. Must not be nil.
- model: The model name for identification.
Outputs:
- *GeminiAgentAdapter: The configured adapter.
func (*GeminiAgentAdapter) Model ¶
func (a *GeminiAgentAdapter) Model() string
Model implements Client.Model.
func (*GeminiAgentAdapter) Name ¶
func (a *GeminiAgentAdapter) Name() string
Name implements Client.Name.
type Message ¶
type Message struct {
// Role is "user", "assistant", or "system".
Role string `json:"role"`
// Content is the text content.
Content string `json:"content"`
// ToolCalls contains tool invocations (for assistant messages).
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
// ToolResults contains tool results (for tool messages).
ToolResults []ToolCallResult `json:"tool_results,omitempty"`
}
Message represents a conversation message.
type MockClient ¶
type MockClient struct {
// contains filtered or unexported fields
}
MockClient is a mock LLM client for testing.
Thread Safety:
MockClient is safe for concurrent use.
func (*MockClient) CallCount ¶
func (c *MockClient) CallCount() int
CallCount returns the number of calls made.
func (*MockClient) ExpectCall ¶
func (c *MockClient) ExpectCall(count int) error
ExpectCall returns an error if the expected number of calls wasn't made.
func (*MockClient) GetCalls ¶
func (c *MockClient) GetCalls() []CompletionCall
GetCalls returns all recorded calls.
func (*MockClient) LastRequest ¶
func (c *MockClient) LastRequest() *Request
LastRequest returns the most recent request.
func (*MockClient) Model ¶
func (c *MockClient) Model() string
Model implements the Client interface.
func (*MockClient) QueueFinalResponse ¶
func (c *MockClient) QueueFinalResponse(content string) *MockClient
QueueFinalResponse queues a final response with no tool calls.
func (*MockClient) QueueResponse ¶
func (c *MockClient) QueueResponse(response *Response) *MockClient
QueueResponse adds a response to the queue.
func (*MockClient) QueueToolCall ¶
func (c *MockClient) QueueToolCall(toolName string, arguments map[string]any) *MockClient
QueueToolCall queues a response that invokes a tool.
func (*MockClient) SetDefaultResponse ¶
func (c *MockClient) SetDefaultResponse(response *Response) *MockClient
SetDefaultResponse sets the response to return when queue is empty.
func (*MockClient) Verify ¶
func (c *MockClient) Verify() error
Verify ensures all queued responses were consumed.
func (*MockClient) WithDelay ¶
func (c *MockClient) WithDelay(d time.Duration) *MockClient
WithDelay adds artificial latency.
func (*MockClient) WithError ¶
func (c *MockClient) WithError(err error) *MockClient
WithError configures the client to return an error.
func (*MockClient) WithModel ¶
func (c *MockClient) WithModel(model string) *MockClient
WithModel sets the model name.
func (*MockClient) WithName ¶
func (c *MockClient) WithName(name string) *MockClient
WithName sets the provider name.
func (*MockClient) WithResponseFunc ¶
func (c *MockClient) WithResponseFunc(f func(*Request) (*Response, error)) *MockClient
WithResponseFunc sets a dynamic response function.
type OllamaAdapter ¶
type OllamaAdapter struct {
// contains filtered or unexported fields
}
OllamaAdapter adapts services/llm.OllamaClient to the agent's Client interface.
Description:
OllamaAdapter wraps the existing OllamaClient to provide LLM capabilities for the agent loop. This allows Trace to use local Ollama models (like gpt-oss:20b) for code assistance.
Thread Safety:
OllamaAdapter is safe for concurrent use.
func NewOllamaAdapter ¶
func NewOllamaAdapter(client *llm.OllamaClient, model string) *OllamaAdapter
NewOllamaAdapter creates a new OllamaAdapter.
Description:
Creates an adapter wrapping the provided OllamaClient.
Inputs:
client - The OllamaClient to wrap. Must not be nil. model - The model name for identification.
Outputs:
*OllamaAdapter - The configured adapter.
Example:
ollamaClient, _ := llm.NewOllamaClient() adapter := NewOllamaAdapter(ollamaClient, "gpt-oss:20b")
func (*OllamaAdapter) Complete ¶
Complete implements Client.
Description:
Sends a completion request to Ollama and returns the response. Converts between agent message format and Ollama's datatypes.Message format. If tools are provided in the request, uses ChatWithTools to enable function calling.
Inputs:
ctx - Context for cancellation and timeout. request - The completion request.
Outputs:
*Response - The LLM response. error - Non-nil if the request failed.
Thread Safety: This method is safe for concurrent use.
type OpenAIAgentAdapter ¶
type OpenAIAgentAdapter struct {
// contains filtered or unexported fields
}
OpenAIAgentAdapter adapts OpenAIClient to the agent's Client interface.
Description:
Wraps the existing OpenAIClient to provide LLM capabilities for the agent loop using OpenAI models. Converts between agent message format and OpenAI's datatypes.Message format.
Thread Safety: OpenAIAgentAdapter is safe for concurrent use.
func NewOpenAIAgentAdapter ¶
func NewOpenAIAgentAdapter(client *llm.OpenAIClient, model string) *OpenAIAgentAdapter
NewOpenAIAgentAdapter creates a new OpenAIAgentAdapter.
Inputs:
- client: The OpenAIClient to wrap. Must not be nil.
- model: The model name for identification.
Outputs:
- *OpenAIAgentAdapter: The configured adapter.
func (*OpenAIAgentAdapter) Model ¶
func (a *OpenAIAgentAdapter) Model() string
Model implements Client.Model.
func (*OpenAIAgentAdapter) Name ¶
func (a *OpenAIAgentAdapter) Name() string
Name implements Client.Name.
type ReActParseResult ¶
type ReActParseResult struct {
// Thought is the LLM's reasoning (may be empty).
Thought string
// Action is the tool name to invoke (empty if no action).
Action string
// ActionInput is the JSON arguments (empty if no action).
ActionInput string
// FinalAnswer is the final response (empty if not done).
FinalAnswer string
// RemainingText is any unparsed text.
RemainingText string
}
ReActParseResult contains parsed ReAct components from LLM response text.
Thread Safety: This type is immutable and safe for concurrent read access.
func ParseReAct ¶
func ParseReAct(text string) *ReActParseResult
ParseReAct extracts ReAct components from LLM response text.
Description:
Parses LLM output for ReAct-style structured text. The ReAct format is:
Thought: [reasoning]
Action: [tool_name]
Action Input: {"param": "value"}
Or for final responses:
Thought: [reasoning]
Final Answer: [response]
Inputs:
text - The LLM response text to parse.
Outputs:
*ReActParseResult - Parsed components. Never nil.
Example:
result := ParseReAct("Thought: I need to find tests.\nAction: find_entry_points\nAction Input: {\"type\": \"test\"}")
// result.Action = "find_entry_points"
// result.ActionInput = "{\"type\": \"test\"}"
Thread Safety: This function is safe for concurrent use.
func (*ReActParseResult) HasAction ¶
func (r *ReActParseResult) HasAction() bool
HasAction returns true if an action was parsed.
Description:
Indicates whether the LLM output contains a tool invocation.
Outputs:
bool - True if Action field is non-empty.
Thread Safety: This method is safe for concurrent use.
func (*ReActParseResult) HasFinalAnswer ¶
func (r *ReActParseResult) HasFinalAnswer() bool
HasFinalAnswer returns true if a final answer was parsed.
Description:
Indicates whether the LLM has provided a final response (meaning no more tool calls are expected).
Outputs:
bool - True if FinalAnswer field is non-empty.
Thread Safety: This method is safe for concurrent use.
func (*ReActParseResult) ToToolCall ¶
func (r *ReActParseResult) ToToolCall() *ToolCall
ToToolCall converts a ReAct action to a ToolCall.
Description:
Creates a ToolCall struct from the parsed ReAct components. Returns nil if no action was parsed.
Outputs:
*ToolCall - The tool call, or nil if no action.
Example:
result := ParseReAct("Action: find_entry_points\nAction Input: {\"type\": \"test\"}")
call := result.ToToolCall()
// call.Name = "find_entry_points"
// call.Arguments = "{\"type\": \"test\"}"
Thread Safety: This method is safe for concurrent use.
type Request ¶
type Request struct {
// SystemPrompt is the system message.
SystemPrompt string `json:"system_prompt,omitempty"`
// Messages is the conversation history.
Messages []Message `json:"messages"`
// Tools defines available tools for the LLM.
Tools []tools.ToolDefinition `json:"tools,omitempty"`
// ToolChoice controls tool selection behavior.
// If nil, defaults to "auto" (model decides).
ToolChoice *ToolChoice `json:"tool_choice,omitempty"`
// MaxTokens limits the response length.
MaxTokens int `json:"max_tokens,omitempty"`
// Temperature controls randomness (0.0-1.0). Set to 0.0 for most
// deterministic output. Set to a negative value (e.g., -1) to omit
// from the request and use the provider's default. The Go zero value
// (0.0) is treated as an explicit "most deterministic" setting.
Temperature float64 `json:"temperature,omitempty"`
// StopSequences defines sequences that stop generation.
StopSequences []string `json:"stop_sequences,omitempty"`
// ModelOverride allows using a different model for this request.
// Used for multi-model scenarios (e.g., tool routing with a fast model).
// Empty string means use the client's default model.
ModelOverride string `json:"model_override,omitempty"`
// KeepAlive controls how long the model stays loaded in VRAM.
// Values: "-1" = infinite, "5m" = 5 minutes (default), "0" = unload immediately.
// Used to prevent model thrashing when alternating between models.
KeepAlive string `json:"keep_alive,omitempty"`
}
Request represents a completion request to the LLM.
func BuildRequest ¶
func BuildRequest( ctx *agent.AssembledContext, availableTools []tools.ToolDefinition, maxTokens int, ) *Request
BuildRequest creates a request from agent context.
Description:
Converts the agent's assembled context into an LLM request suitable for the Complete method.
Inputs:
ctx - Assembled context from the agent availableTools - Tools available for this request maxTokens - Maximum response tokens
Outputs:
*Request - The built request
type Response ¶
type Response struct {
// Content is the text response.
Content string `json:"content"`
// ToolCalls contains any tool calls the LLM wants to make.
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
// StopReason indicates why generation stopped.
// Values: "end", "max_tokens", "tool_use", "stop_sequence"
StopReason string `json:"stop_reason"`
// TokensUsed is the total tokens consumed (input + output).
TokensUsed int `json:"tokens_used"`
// InputTokens is the input token count.
InputTokens int `json:"input_tokens"`
// OutputTokens is the output token count.
OutputTokens int `json:"output_tokens"`
// Duration is how long the request took.
Duration time.Duration `json:"duration"`
// Model is the model that generated this response.
Model string `json:"model,omitempty"`
// TraceStep is the CRS trace step recorded for this LLM call.
// Callers can use this to record the step in the CRS trace recorder.
// May be nil if CRS recording was not applicable.
TraceStep *crs.TraceStep `json:"trace_step,omitempty"`
}
Response represents an LLM response.
func (*Response) HasToolCalls ¶
HasToolCalls returns true if the response contains tool calls.
type ToolCall ¶
type ToolCall struct {
// ID is a unique identifier for this call.
ID string `json:"id"`
// Name is the tool name.
Name string `json:"name"`
// Arguments are the tool arguments as JSON.
Arguments string `json:"arguments"`
}
ToolCall represents a tool invocation by the LLM.
type ToolCallResult ¶
type ToolCallResult struct {
// ToolCallID links back to the tool call.
ToolCallID string `json:"tool_call_id"`
// Content is the result content.
Content string `json:"content"`
// IsError indicates if this is an error result.
IsError bool `json:"is_error,omitempty"`
}
ToolCallResult contains the result of a tool call.
type ToolChoice ¶
type ToolChoice struct {
// Type controls tool selection behavior:
// - "auto": Model decides whether to call tools (default)
// - "any": Model MUST call at least one tool
// - "tool": Model MUST call the specific named tool
// - "none": Model cannot call tools (text response only)
Type string `json:"type"`
// Name is required when Type is "tool". Specifies which tool to force.
Name string `json:"name,omitempty"`
}
ToolChoice specifies how the model should select tools.
The tool_choice parameter controls whether and which tools the model calls. This enables forcing tool usage at the API level rather than relying on prompts.
func ToolChoiceAny ¶
func ToolChoiceAny() *ToolChoice
ToolChoiceAny forces the model to call at least one tool.
func ToolChoiceAuto ¶
func ToolChoiceAuto() *ToolChoice
ToolChoiceAuto allows the model to decide whether to call tools.
func ToolChoiceNone ¶
func ToolChoiceNone() *ToolChoice
ToolChoiceNone prevents the model from calling any tools.
func ToolChoiceRequired ¶
func ToolChoiceRequired(toolName string) *ToolChoice
ToolChoiceRequired forces the model to call a specific tool by name.