llm

package
v1.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 27, 2026 License: MIT Imports: 3 Imported by: 1

Documentation

Overview

Package llm defines a provider-agnostic LLM client interface for the Cortex engine.

This abstraction allows the engine to call any LLM backend (Nexus, direct providers, mock) without importing provider-specific packages. Adapters convert between these types and the concrete provider types.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Chunk

type Chunk struct {
	// Content is the incremental text delta.
	Content string

	// ToolCalls contains incremental tool call deltas.
	ToolCalls []ToolCall

	// FinishReason is set on the final chunk.
	FinishReason string
}

Chunk is a single incremental piece of a streaming response.

type Client

type Client interface {
	// Complete sends a synchronous completion request and returns the full response.
	Complete(ctx context.Context, req *Request) (*Response, error)

	// CompleteStream sends a streaming completion request and returns a stream
	// that yields incremental chunks.
	CompleteStream(ctx context.Context, req *Request) (Stream, error)
}

Client is the interface for sending completion requests to an LLM.

type Message

type Message struct {
	// Role is the message author: "user", "assistant", or "tool".
	Role string

	// Content is the text content of the message.
	Content string

	// ToolCalls contains tool invocations requested by the assistant.
	ToolCalls []ToolCall

	// ToolCallID identifies which tool call this message is responding to (role=tool).
	ToolCallID string
}

Message is a single message in a conversation.

type MockClient

type MockClient struct {
	// TokenDelay is the delay between each streamed character. Default: 20ms.
	TokenDelay time.Duration
}

MockClient is an echo-based LLM client for testing and fallback. It echoes the user input back as the assistant response with simulated delays.

func NewMockClient

func NewMockClient() *MockClient

NewMockClient creates a new echo-based mock LLM client.

func (*MockClient) Complete

func (m *MockClient) Complete(_ context.Context, req *Request) (*Response, error)

Complete returns an echo response synchronously.

func (*MockClient) CompleteStream

func (m *MockClient) CompleteStream(_ context.Context, req *Request) (Stream, error)

CompleteStream returns a stream that yields the echo response token by token.

type Request

type Request struct {
	// Model is the model identifier (e.g. "gpt-4o", "claude-sonnet-4-20250514", "smart").
	Model string

	// System is the system prompt injected before the conversation.
	System string

	// Messages is the conversation history.
	Messages []Message

	// Tools is the set of tools available for the LLM to call.
	Tools []Tool

	// MaxTokens limits the maximum output tokens.
	MaxTokens int

	// Temperature controls sampling randomness. Nil means provider default.
	Temperature *float64
}

Request is a completion request sent to an LLM.

type Response

type Response struct {
	// Content is the assistant's text response.
	Content string

	// ToolCalls contains any tool invocations requested by the assistant.
	ToolCalls []ToolCall

	// Usage tracks token consumption.
	Usage Usage

	// Model is the actual model used (may differ from request if aliased).
	Model string

	// FinishReason indicates why generation stopped: "stop", "tool_calls", "length", etc.
	FinishReason string
}

Response is the result of a synchronous completion request.

type Stream

type Stream interface {
	// Next returns the next chunk. Returns io.EOF when the stream is complete.
	Next(ctx context.Context) (*Chunk, error)

	// Close releases stream resources.
	Close() error

	// Usage returns the final token usage after the stream completes.
	// May return nil if usage is not yet available.
	Usage() *Usage
}

Stream yields incremental chunks from a streaming completion.

type Tool

type Tool struct {
	// Name is the tool's unique identifier.
	Name string

	// Description explains what the tool does.
	Description string

	// Parameters is a JSON Schema describing the tool's input parameters.
	Parameters any
}

Tool describes a function tool available for the LLM to call.

type ToolCall

type ToolCall struct {
	// ID uniquely identifies this tool call (used to match tool results).
	ID string

	// Name is the name of the tool to invoke.
	Name string

	// Arguments is the JSON-encoded arguments for the tool.
	Arguments string
}

ToolCall represents a tool invocation from the assistant.

type Usage

type Usage struct {
	PromptTokens     int
	CompletionTokens int
	TotalTokens      int
}

Usage tracks token consumption for a completion.

Directories

Path Synopsis
Package nexus adapts the Nexus LLM gateway to the cortex llm.Client interface.
Package nexus adapts the Nexus LLM gateway to the cortex llm.Client interface.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL