llm

package
v1.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 14, 2026 License: MIT Imports: 20 Imported by: 0

Documentation

Overview

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI. This file defines StreamEvent types for granular streaming events, replacing the batch-oriented StreamChunk for tool call delivery.

Package llm provides the LLM client abstraction for Celeste CLI.

Package llm provides the LLM client for Celeste CLI. This file contains streaming-specific functionality.

Package llm provides the LLM client for Celeste CLI.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func IsValidThinkingLevel added in v1.8.0

func IsValidThinkingLevel(level string) bool

IsValidLevel reports whether level is a recognised thinking level.

func ValidThinkingLevels added in v1.8.0

func ValidThinkingLevels() []string

ValidLevels returns the accepted level strings for user-facing validation.

Types

type AnthropicBackend added in v1.8.2

type AnthropicBackend struct {
	// contains filtered or unexported fields
}

AnthropicBackend implements LLMBackend using the native Anthropic SDK. This backend supports Claude models via the Anthropic Messages API with prompt caching, extended thinking, and native streaming.

func NewAnthropicBackend added in v1.8.2

func NewAnthropicBackend(config *Config) (*AnthropicBackend, error)

NewAnthropicBackend creates a new Anthropic backend using the native SDK.

func (*AnthropicBackend) Close added in v1.8.2

func (b *AnthropicBackend) Close() error

Close cleans up resources (no-op for Anthropic backend).

func (*AnthropicBackend) SendMessageStream added in v1.8.2

func (b *AnthropicBackend) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback.

func (*AnthropicBackend) SendMessageStreamEvents added in v1.8.2

func (b *AnthropicBackend) SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamEventCallback) error

SendMessageStreamEvents sends a message with granular streaming events.

func (*AnthropicBackend) SendMessageSync added in v1.8.2

func (b *AnthropicBackend) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync sends a message and returns the complete result.

func (*AnthropicBackend) SetSystemPrompt added in v1.8.2

func (b *AnthropicBackend) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona).

func (*AnthropicBackend) SetThinkingConfig added in v1.8.2

func (b *AnthropicBackend) SetThinkingConfig(config ThinkingConfig)

SetThinkingConfig configures extended thinking for Claude models. Claude supports budget_tokens via ThinkingConfigParam.

type BackendType

type BackendType string

BackendType identifies which SDK implementation is being used.

const (
	// BackendTypeOpenAI uses the go-openai SDK (OpenAI, Venice, Anthropic, etc.)
	BackendTypeOpenAI BackendType = "openai"

	// BackendTypeGoogle uses the native Google GenAI SDK (Gemini, Vertex AI)
	BackendTypeGoogle BackendType = "google"

	// BackendTypeXAI uses the native xAI SDK with Collections support (Grok models)
	BackendTypeXAI BackendType = "xai"

	// BackendTypeAnthropic uses the native Anthropic SDK (Claude models)
	BackendTypeAnthropic BackendType = "anthropic"
)

func DetectBackendType

func DetectBackendType(baseURL string) BackendType

DetectBackendType determines which backend to use based on the base URL.

type CacheablePrompt added in v1.8.0

type CacheablePrompt struct {
	StaticPrefix  string // persona + tool schemas + grimoire (cacheable)
	DynamicSuffix string // git status + date + memories (not cached)
}

CacheablePrompt separates the system prompt into a static prefix (suitable for provider-side prompt caching) and a dynamic suffix that changes between turns. The static prefix typically contains the persona definition, tool schemas, and grimoire content. The dynamic suffix holds git status snapshots, current date, memories, and other volatile context.

func (CacheablePrompt) FullPrompt added in v1.8.0

func (cp CacheablePrompt) FullPrompt() string

FullPrompt returns the complete system prompt by joining the static and dynamic sections with a clear separator.

type ChatCompletionResult

type ChatCompletionResult struct {
	Content      string
	ToolCalls    []ToolCallResult
	FinishReason string
	Error        error
	Usage        *TokenUsage // Token usage from the API response (if available)
}

ChatCompletionResult holds the result of a chat completion.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client wraps LLM backends and provides a unified interface. It automatically selects the appropriate backend (OpenAI or Google) based on the provider.

func NewClient

func NewClient(config *Config, registry *tools.Registry) *Client

NewClient creates a new LLM client with automatic backend selection. It detects whether to use OpenAI SDK, Google GenAI SDK, or xAI SDK based on the base URL.

func (*Client) ExecuteSkill

func (c *Client) ExecuteSkill(ctx context.Context, name string, argsJSON string) (*ExecutionResult, error)

ExecuteSkill executes a skill and returns the result.

func (*Client) GetConfig

func (c *Client) GetConfig() *Config

GetConfig returns the current configuration.

func (*Client) GetSkills

func (c *Client) GetSkills() []tui.SkillDefinition

GetSkills returns skill definitions for the TUI.

func (*Client) SendMessageStream

func (c *Client) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback. This delegates to the appropriate backend (OpenAI or Google).

func (*Client) SendMessageStreamEvents added in v1.7.0

func (c *Client) SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamEventCallback) error

SendMessageStreamEvents sends a message with granular streaming events. This delegates to the appropriate backend.

func (*Client) SendMessageSync

func (c *Client) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync sends a message synchronously and returns the result. This delegates to the appropriate backend (OpenAI or Google).

func (*Client) SetSystemPrompt

func (c *Client) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona).

func (*Client) SetThinkingConfig added in v1.8.0

func (c *Client) SetThinkingConfig(config ThinkingConfig)

SetThinkingConfig configures extended thinking / reasoning effort.

func (*Client) UpdateConfig

func (c *Client) UpdateConfig(config *Config)

UpdateConfig updates the client configuration and recreates the backend if needed. This allows dynamic endpoint/model switching during runtime.

type Config

type Config struct {
	APIKey            string
	BaseURL           string
	Model             string
	Timeout           time.Duration
	SkipPersonaPrompt bool
	SimulateTyping    bool
	TypingSpeed       int // chars per second

	// Google Cloud authentication (for Gemini/Vertex AI)
	GoogleCredentialsFile string // Path to service account JSON file
	GoogleUseADC          bool   // Use Application Default Credentials

	// Collections (xAI only)
	Collections *config.CollectionsConfig
	XAIFeatures *config.XAIFeaturesConfig
}

Config holds LLM client configuration.

type ExecutionResult added in v1.7.0

type ExecutionResult struct {
	Success  bool           `json:"success"`
	Result   interface{}    `json:"result,omitempty"`
	Error    string         `json:"error,omitempty"`
	Metadata map[string]any `json:"metadata,omitempty"` // e.g. image base64 from read_file
}

ExecutionResult represents the result of a skill/tool execution.

type GoogleBackend

type GoogleBackend struct {
	// contains filtered or unexported fields
}

GoogleBackend implements LLMBackend using Google's native GenAI SDK. This backend supports Gemini AI Studio and Vertex AI with automatic authentication.

func NewGoogleBackend

func NewGoogleBackend(config *Config) (*GoogleBackend, error)

NewGoogleBackend creates a new Google GenAI backend with automatic authentication. Authentication methods (in order of priority): 1. Simple API key (for Gemini AI Studio) 2. GoogleCredentialsFile in config (service account JSON) 3. GOOGLE_APPLICATION_CREDENTIALS environment variable 4. Application Default Credentials (gcloud auth application-default login)

func (*GoogleBackend) Close

func (b *GoogleBackend) Close() error

Close cleans up resources.

func (*GoogleBackend) SendMessageStream

func (b *GoogleBackend) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback.

func (*GoogleBackend) SendMessageStreamEvents added in v1.7.0

func (b *GoogleBackend) SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamEventCallback) error

SendMessageStreamEvents sends a message with granular streaming events.

func (*GoogleBackend) SendMessageSync

func (b *GoogleBackend) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync sends a message synchronously and returns the complete result.

func (*GoogleBackend) SetSystemPrompt

func (b *GoogleBackend) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona).

func (*GoogleBackend) SetThinkingConfig added in v1.8.0

func (b *GoogleBackend) SetThinkingConfig(config ThinkingConfig)

SetThinkingConfig configures extended thinking for Gemini models. Gemini supports thinkingBudget via GenerateContentConfig.ThinkingConfig.

type LLMBackend

type LLMBackend interface {
	// SendMessageStream sends a message with streaming callback.
	// The callback receives chunks as they arrive from the LLM.
	// Returns error if the request fails.
	SendMessageStream(ctx context.Context, messages []tui.ChatMessage,
		tools []tui.SkillDefinition, callback StreamCallback) error

	// SendMessageStreamEvents sends a message with granular streaming events.
	// Unlike SendMessageStream which batches tool calls into the final chunk,
	// this method delivers tool call information incrementally as it arrives.
	SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage,
		tools []tui.SkillDefinition, callback StreamEventCallback) error

	// SendMessageSync sends a message and returns the complete result.
	// This is useful for non-streaming use cases or testing.
	// Returns the full chat completion result or error.
	SendMessageSync(ctx context.Context, messages []tui.ChatMessage,
		tools []tui.SkillDefinition) (*ChatCompletionResult, error)

	// SetSystemPrompt sets the system prompt (Celeste persona).
	// This configures the LLM's behavior and character.
	SetSystemPrompt(prompt string)

	// SetThinkingConfig configures extended thinking / reasoning effort.
	// Backends that don't support thinking silently ignore the config.
	SetThinkingConfig(config ThinkingConfig)

	// Close cleans up resources (e.g., network connections).
	// Should be called when the backend is no longer needed.
	Close() error
}

LLMBackend defines the interface all LLM backends must implement. This abstraction allows Celeste to support multiple SDK implementations: - OpenAI SDK (go-openai) for OpenAI, Grok, Venice, Anthropic, etc. - Google GenAI SDK (google.golang.org/genai) for Gemini and Vertex AI

type OpenAIBackend

type OpenAIBackend struct {
	// contains filtered or unexported fields
}

OpenAIBackend implements LLMBackend using the go-openai SDK. This backend supports OpenAI, Grok, Venice, Anthropic, and other OpenAI-compatible providers.

func NewOpenAIBackend

func NewOpenAIBackend(config *Config) *OpenAIBackend

NewOpenAIBackend creates a new OpenAI-compatible backend.

func (*OpenAIBackend) Close

func (b *OpenAIBackend) Close() error

Close cleans up resources (no-op for OpenAI backend).

func (*OpenAIBackend) SendMessageStream

func (b *OpenAIBackend) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback.

func (*OpenAIBackend) SendMessageStreamEvents added in v1.7.0

func (b *OpenAIBackend) SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamEventCallback) error

SendMessageStreamEvents sends a message with granular streaming events.

func (*OpenAIBackend) SendMessageSync

func (b *OpenAIBackend) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync sends a message synchronously and returns the complete result.

func (*OpenAIBackend) SetSystemPrompt

func (b *OpenAIBackend) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona).

func (*OpenAIBackend) SetThinkingConfig added in v1.8.0

func (b *OpenAIBackend) SetThinkingConfig(config ThinkingConfig)

SetThinkingConfig configures extended thinking / reasoning effort. For OpenAI o-series models this maps to reasoning_effort. For Anthropic (via OpenAI compat) this is a no-op for now.

type SimulatedStream

type SimulatedStream struct {
	// contains filtered or unexported fields
}

SimulatedStream simulates streaming for dump responses.

func NewSimulatedStream

func NewSimulatedStream(content string, config SimulatedStreamConfig) *SimulatedStream

NewSimulatedStream creates a new simulated stream.

func (*SimulatedStream) GetProgress

func (s *SimulatedStream) GetProgress() float64

GetProgress returns the current progress (0-1).

func (*SimulatedStream) Next

func (s *SimulatedStream) Next() (chunk string, delay time.Duration, done bool)

Next returns the next chunk and delay.

func (*SimulatedStream) Reset

func (s *SimulatedStream) Reset()

Reset resets the simulated stream.

type SimulatedStreamConfig

type SimulatedStreamConfig struct {
	TypingSpeed  int     // Characters per second
	GlitchChance float64 // Chance of corruption effect (0-1)
	MinDelay     time.Duration
	MaxDelay     time.Duration
}

SimulatedStreamConfig holds configuration for simulated streaming.

func DefaultSimulatedConfig

func DefaultSimulatedConfig() SimulatedStreamConfig

DefaultSimulatedConfig returns default simulated streaming config.

type StreamCallback

type StreamCallback func(chunk StreamChunk)

StreamCallback is called for each chunk during streaming.

type StreamChunk

type StreamChunk struct {
	Content      string
	IsFirst      bool
	IsFinal      bool
	FinishReason string
	ToolCalls    []ToolCallResult
	Usage        *TokenUsage // Only populated on final chunk with stream_options
}

type StreamEvent added in v1.7.0

type StreamEvent struct {
	// Type identifies which kind of event this is.
	Type StreamEventType

	// ContentDelta contains the text delta (only for EventContentDelta).
	ContentDelta string

	// ToolUseID is the unique identifier for the tool call
	// (set for EventToolUseStart, EventToolUseInputDelta, EventToolUseDone).
	ToolUseID string

	// ToolName is the function name being called
	// (set for EventToolUseStart, EventToolUseDone).
	ToolName string

	// InputDelta contains partial argument JSON
	// (only for EventToolUseInputDelta).
	InputDelta string

	// CompleteInput contains the fully accumulated argument JSON
	// (only for EventToolUseDone).
	CompleteInput string

	// Usage contains token usage statistics (only for EventMessageDone).
	Usage *TokenUsage

	// FinishReason indicates why the response ended (only for EventMessageDone).
	FinishReason string
}

StreamEvent represents a single granular event during LLM streaming. Unlike StreamChunk which batches tool calls into the final chunk, StreamEvent delivers tool call information incrementally as it arrives.

func (StreamEvent) IsToolEvent added in v1.7.0

func (e StreamEvent) IsToolEvent() bool

IsToolEvent returns true if this event relates to a tool use block.

type StreamEventCallback added in v1.7.0

type StreamEventCallback func(event StreamEvent)

StreamEventCallback is called for each streaming event.

type StreamEventType added in v1.7.0

type StreamEventType int

StreamEventType identifies the kind of streaming event.

const (
	// EventContentDelta is emitted when the LLM produces a text content delta.
	EventContentDelta StreamEventType = iota

	// EventToolUseStart is emitted when a new tool_use block begins streaming.
	// Contains the tool call ID and tool name.
	EventToolUseStart

	// EventToolUseInputDelta is emitted as argument JSON accumulates for a tool call.
	// Contains partial JSON input for the tool call identified by ToolUseID.
	EventToolUseInputDelta

	// EventToolUseDone is emitted when a tool call's arguments are fully received.
	// Contains the complete input JSON, ready for execution.
	EventToolUseDone

	// EventMessageDone is emitted when the entire LLM response is complete.
	// Contains usage statistics and finish reason.
	EventMessageDone
)

func (StreamEventType) String added in v1.7.0

func (t StreamEventType) String() string

String returns a human-readable name for the event type.

type StreamState

type StreamState struct {
	// contains filtered or unexported fields
}

StreamState tracks the state of a streaming response.

func NewStreamState

func NewStreamState() *StreamState

NewStreamState creates a new stream state tracker.

func (*StreamState) GetChunkCount

func (s *StreamState) GetChunkCount() int

GetChunkCount returns the number of chunks received.

func (*StreamState) GetContent

func (s *StreamState) GetContent() string

GetContent returns the accumulated content.

func (*StreamState) GetDuration

func (s *StreamState) GetDuration() time.Duration

GetDuration returns the total stream duration.

func (*StreamState) IsComplete

func (s *StreamState) IsComplete() bool

IsComplete returns whether the stream is complete.

func (*StreamState) IsDump

func (s *StreamState) IsDump() bool

IsDump returns whether the stream was a dump (all at once).

func (*StreamState) MarkComplete

func (s *StreamState) MarkComplete()

MarkComplete marks the stream as complete.

type ThinkingConfig added in v1.8.0

type ThinkingConfig struct {
	Enabled      bool
	BudgetTokens int    // 0 = provider default
	Level        string // "off", "low", "medium", "high", "max"
}

ThinkingConfig controls extended thinking / reasoning effort for LLM backends. When Enabled is true, compatible backends inject the appropriate parameters (e.g. reasoning_effort for OpenAI o-series, thinkingBudget for Gemini).

func (ThinkingConfig) LevelToBudget added in v1.8.0

func (tc ThinkingConfig) LevelToBudget() int

LevelToBudget converts a human-friendly level string to a token budget. If Level is empty or unrecognised, BudgetTokens is returned as-is (0 = provider default).

type TokenUsage

type TokenUsage struct {
	PromptTokens     int
	CompletionTokens int
	TotalTokens      int
}

StreamChunk represents a streaming chunk. TokenUsage holds token usage information from API response

type ToolCallResult

type ToolCallResult struct {
	ID        string
	Name      string
	Arguments string
}

ToolCallResult holds a tool call from the LLM.

type ToolUseAccumulator added in v1.7.0

type ToolUseAccumulator struct {
	// contains filtered or unexported fields
}

ToolUseAccumulator tracks in-progress tool uses during streaming, collecting input deltas and producing completed ToolCallResult values. It is safe for use from a single goroutine (the streaming callback goroutine).

func NewToolUseAccumulator added in v1.7.0

func NewToolUseAccumulator() *ToolUseAccumulator

NewToolUseAccumulator creates a new accumulator for tracking tool uses.

func (*ToolUseAccumulator) CompletedCalls added in v1.7.0

func (a *ToolUseAccumulator) CompletedCalls() []ToolCallResult

CompletedCalls returns all tool calls that have finished receiving input, in the order they were started.

func (*ToolUseAccumulator) HandleEvent added in v1.7.0

func (a *ToolUseAccumulator) HandleEvent(event StreamEvent)

HandleEvent processes a StreamEvent and updates internal state. For EventToolUseStart: creates a new pending tool use. For EventToolUseInputDelta: appends to the pending tool's input buffer. For EventToolUseDone: moves the tool use to the completed list. Other event types are ignored.

func (*ToolUseAccumulator) PendingCount added in v1.7.0

func (a *ToolUseAccumulator) PendingCount() int

PendingCount returns the number of tool uses still accumulating input.

func (*ToolUseAccumulator) Reset added in v1.7.0

func (a *ToolUseAccumulator) Reset()

Reset clears all state.

type XAIBackend

type XAIBackend struct {
	// contains filtered or unexported fields
}

XAIBackend implements LLMBackend using xAI's native API. This backend supports xAI-specific features like Collections (RAG).

func NewXAIBackend

func NewXAIBackend(config *Config, registry *tools.Registry) (*XAIBackend, error)

NewXAIBackend creates a new xAI backend with Collections support.

func (*XAIBackend) ChangeModel

func (b *XAIBackend) ChangeModel(model string) error

ChangeModel changes the model

func (*XAIBackend) Close

func (b *XAIBackend) Close() error

Close cleans up resources (implements LLMBackend interface)

func (*XAIBackend) GetSkills

func (b *XAIBackend) GetSkills() []tui.SkillDefinition

GetSkills returns the list of available skills from the registry

func (*XAIBackend) SendMessageStream

func (b *XAIBackend) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback.

func (*XAIBackend) SendMessageStreamEvents added in v1.7.0

func (b *XAIBackend) SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamEventCallback) error

SendMessageStreamEvents sends a message with granular streaming events.

func (*XAIBackend) SendMessageSync

func (b *XAIBackend) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync collects the xAI stream into a single synchronous result.

func (*XAIBackend) SetSystemPrompt

func (b *XAIBackend) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona).

func (*XAIBackend) SetThinkingConfig added in v1.8.0

func (b *XAIBackend) SetThinkingConfig(config ThinkingConfig)

SetThinkingConfig configures extended thinking / reasoning effort for Grok models. xAI supports reasoning_effort in the request body.

func (*XAIBackend) SwitchEndpoint

func (b *XAIBackend) SwitchEndpoint(endpoint string) error

SwitchEndpoint switches to a different endpoint (for config switching)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL