Documentation
¶
Overview ¶
Package llm defines a provider-agnostic LLM client interface for the Cortex engine.
This abstraction allows the engine to call any LLM backend (Nexus, direct providers, mock) without importing provider-specific packages. Adapters convert between these types and the concrete provider types.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Chunk ¶
type Chunk struct {
// Content is the incremental text delta.
Content string
// ToolCalls contains incremental tool call deltas.
ToolCalls []ToolCall
// FinishReason is set on the final chunk.
FinishReason string
}
Chunk is a single incremental piece of a streaming response.
type Client ¶
type Client interface {
// Complete sends a synchronous completion request and returns the full response.
Complete(ctx context.Context, req *Request) (*Response, error)
// CompleteStream sends a streaming completion request and returns a stream
// that yields incremental chunks.
CompleteStream(ctx context.Context, req *Request) (Stream, error)
}
Client is the interface for sending completion requests to an LLM.
type Message ¶
type Message struct {
// Role is the message author: "user", "assistant", or "tool".
Role string
// Content is the text content of the message.
Content string
// ToolCalls contains tool invocations requested by the assistant.
ToolCalls []ToolCall
// ToolCallID identifies which tool call this message is responding to (role=tool).
ToolCallID string
}
Message is a single message in a conversation.
type MockClient ¶
type MockClient struct {
// TokenDelay is the delay between each streamed character. Default: 20ms.
TokenDelay time.Duration
}
MockClient is an echo-based LLM client for testing and fallback. It echoes the user input back as the assistant response with simulated delays.
func NewMockClient ¶
func NewMockClient() *MockClient
NewMockClient creates a new echo-based mock LLM client.
func (*MockClient) CompleteStream ¶
CompleteStream returns a stream that yields the echo response token by token.
type Request ¶
type Request struct {
// Model is the model identifier (e.g. "gpt-4o", "claude-sonnet-4-20250514", "smart").
Model string
// System is the system prompt injected before the conversation.
System string
// Messages is the conversation history.
Messages []Message
// Tools is the set of tools available for the LLM to call.
Tools []Tool
// MaxTokens limits the maximum output tokens.
MaxTokens int
// Temperature controls sampling randomness. Nil means provider default.
Temperature *float64
}
Request is a completion request sent to an LLM.
type Response ¶
type Response struct {
// Content is the assistant's text response.
Content string
// ToolCalls contains any tool invocations requested by the assistant.
ToolCalls []ToolCall
// Usage tracks token consumption.
Usage Usage
// Model is the actual model used (may differ from request if aliased).
Model string
// FinishReason indicates why generation stopped: "stop", "tool_calls", "length", etc.
FinishReason string
}
Response is the result of a synchronous completion request.
type Stream ¶
type Stream interface {
// Next returns the next chunk. Returns io.EOF when the stream is complete.
Next(ctx context.Context) (*Chunk, error)
// Close releases stream resources.
Close() error
// Usage returns the final token usage after the stream completes.
// May return nil if usage is not yet available.
Usage() *Usage
}
Stream yields incremental chunks from a streaming completion.
type Tool ¶
type Tool struct {
// Name is the tool's unique identifier.
Name string
// Description explains what the tool does.
Description string
// Parameters is a JSON Schema describing the tool's input parameters.
Parameters any
}
Tool describes a function tool available for the LLM to call.
type ToolCall ¶
type ToolCall struct {
// ID uniquely identifies this tool call (used to match tool results).
ID string
// Name is the name of the tool to invoke.
Name string
// Arguments is the JSON-encoded arguments for the tool.
Arguments string
}
ToolCall represents a tool invocation from the assistant.