llm

package
v1.7.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 1, 2026 License: MIT Imports: 17 Imported by: 0

Documentation

Overview

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI. This file defines StreamEvent types for granular streaming events, replacing the batch-oriented StreamChunk for tool call delivery.

Package llm provides the LLM client abstraction for Celeste CLI.

Package llm provides the LLM client for Celeste CLI. This file contains streaming-specific functionality.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BackendType

type BackendType string

BackendType identifies which SDK implementation is being used.

const (
	// BackendTypeOpenAI uses the go-openai SDK (OpenAI, Venice, Anthropic, etc.)
	BackendTypeOpenAI BackendType = "openai"

	// BackendTypeGoogle uses the native Google GenAI SDK (Gemini, Vertex AI)
	BackendTypeGoogle BackendType = "google"

	// BackendTypeXAI uses the native xAI SDK with Collections support (Grok models)
	BackendTypeXAI BackendType = "xai"
)

func DetectBackendType

func DetectBackendType(baseURL string) BackendType

DetectBackendType determines which backend to use based on the base URL.

type ChatCompletionResult

type ChatCompletionResult struct {
	Content      string
	ToolCalls    []ToolCallResult
	FinishReason string
	Error        error
	Usage        *TokenUsage // Token usage from the API response (if available)
}

ChatCompletionResult holds the result of a chat completion.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client wraps LLM backends and provides a unified interface. It automatically selects the appropriate backend (OpenAI or Google) based on the provider.

func NewClient

func NewClient(config *Config, registry *tools.Registry) *Client

NewClient creates a new LLM client with automatic backend selection. It detects whether to use OpenAI SDK, Google GenAI SDK, or xAI SDK based on the base URL.

func (*Client) ExecuteSkill

func (c *Client) ExecuteSkill(ctx context.Context, name string, argsJSON string) (*ExecutionResult, error)

ExecuteSkill executes a skill and returns the result.

func (*Client) GetConfig

func (c *Client) GetConfig() *Config

GetConfig returns the current configuration.

func (*Client) GetSkills

func (c *Client) GetSkills() []tui.SkillDefinition

GetSkills returns skill definitions for the TUI.

func (*Client) SendMessageStream

func (c *Client) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback. This delegates to the appropriate backend (OpenAI or Google).

func (*Client) SendMessageStreamEvents added in v1.7.0

func (c *Client) SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamEventCallback) error

SendMessageStreamEvents sends a message with granular streaming events. This delegates to the appropriate backend.

func (*Client) SendMessageSync

func (c *Client) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync sends a message synchronously and returns the result. This delegates to the appropriate backend (OpenAI or Google).

func (*Client) SetSystemPrompt

func (c *Client) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona).

func (*Client) UpdateConfig

func (c *Client) UpdateConfig(config *Config)

UpdateConfig updates the client configuration and recreates the backend if needed. This allows dynamic endpoint/model switching during runtime.

type Config

type Config struct {
	APIKey            string
	BaseURL           string
	Model             string
	Timeout           time.Duration
	SkipPersonaPrompt bool
	SimulateTyping    bool
	TypingSpeed       int // chars per second

	// Google Cloud authentication (for Gemini/Vertex AI)
	GoogleCredentialsFile string // Path to service account JSON file
	GoogleUseADC          bool   // Use Application Default Credentials

	// Collections (xAI only)
	Collections *config.CollectionsConfig
	XAIFeatures *config.XAIFeaturesConfig
}

Config holds LLM client configuration.

type ExecutionResult added in v1.7.0

type ExecutionResult struct {
	Success bool        `json:"success"`
	Result  interface{} `json:"result,omitempty"`
	Error   string      `json:"error,omitempty"`
}

ExecutionResult represents the result of a skill/tool execution.

type GoogleBackend

type GoogleBackend struct {
	// contains filtered or unexported fields
}

GoogleBackend implements LLMBackend using Google's native GenAI SDK. This backend supports Gemini AI Studio and Vertex AI with automatic authentication.

func NewGoogleBackend

func NewGoogleBackend(config *Config) (*GoogleBackend, error)

NewGoogleBackend creates a new Google GenAI backend with automatic authentication. Authentication methods (in order of priority): 1. Simple API key (for Gemini AI Studio) 2. GoogleCredentialsFile in config (service account JSON) 3. GOOGLE_APPLICATION_CREDENTIALS environment variable 4. Application Default Credentials (gcloud auth application-default login)

func (*GoogleBackend) Close

func (b *GoogleBackend) Close() error

Close cleans up resources.

func (*GoogleBackend) SendMessageStream

func (b *GoogleBackend) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback.

func (*GoogleBackend) SendMessageStreamEvents added in v1.7.0

func (b *GoogleBackend) SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamEventCallback) error

SendMessageStreamEvents sends a message with granular streaming events.

func (*GoogleBackend) SendMessageSync

func (b *GoogleBackend) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync sends a message synchronously and returns the complete result.

func (*GoogleBackend) SetSystemPrompt

func (b *GoogleBackend) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona).

type LLMBackend

type LLMBackend interface {
	// SendMessageStream sends a message with streaming callback.
	// The callback receives chunks as they arrive from the LLM.
	// Returns error if the request fails.
	SendMessageStream(ctx context.Context, messages []tui.ChatMessage,
		tools []tui.SkillDefinition, callback StreamCallback) error

	// SendMessageStreamEvents sends a message with granular streaming events.
	// Unlike SendMessageStream which batches tool calls into the final chunk,
	// this method delivers tool call information incrementally as it arrives.
	SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage,
		tools []tui.SkillDefinition, callback StreamEventCallback) error

	// SendMessageSync sends a message and returns the complete result.
	// This is useful for non-streaming use cases or testing.
	// Returns the full chat completion result or error.
	SendMessageSync(ctx context.Context, messages []tui.ChatMessage,
		tools []tui.SkillDefinition) (*ChatCompletionResult, error)

	// SetSystemPrompt sets the system prompt (Celeste persona).
	// This configures the LLM's behavior and character.
	SetSystemPrompt(prompt string)

	// Close cleans up resources (e.g., network connections).
	// Should be called when the backend is no longer needed.
	Close() error
}

LLMBackend defines the interface all LLM backends must implement. This abstraction allows Celeste to support multiple SDK implementations: - OpenAI SDK (go-openai) for OpenAI, Grok, Venice, Anthropic, etc. - Google GenAI SDK (google.golang.org/genai) for Gemini and Vertex AI

type OpenAIBackend

type OpenAIBackend struct {
	// contains filtered or unexported fields
}

OpenAIBackend implements LLMBackend using the go-openai SDK. This backend supports OpenAI, Grok, Venice, Anthropic, and other OpenAI-compatible providers.

func NewOpenAIBackend

func NewOpenAIBackend(config *Config) *OpenAIBackend

NewOpenAIBackend creates a new OpenAI-compatible backend.

func (*OpenAIBackend) Close

func (b *OpenAIBackend) Close() error

Close cleans up resources (no-op for OpenAI backend).

func (*OpenAIBackend) SendMessageStream

func (b *OpenAIBackend) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback.

func (*OpenAIBackend) SendMessageStreamEvents added in v1.7.0

func (b *OpenAIBackend) SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamEventCallback) error

SendMessageStreamEvents sends a message with granular streaming events.

func (*OpenAIBackend) SendMessageSync

func (b *OpenAIBackend) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync sends a message synchronously and returns the complete result.

func (*OpenAIBackend) SetSystemPrompt

func (b *OpenAIBackend) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona).

type SimulatedStream

type SimulatedStream struct {
	// contains filtered or unexported fields
}

SimulatedStream simulates streaming for dump responses.

func NewSimulatedStream

func NewSimulatedStream(content string, config SimulatedStreamConfig) *SimulatedStream

NewSimulatedStream creates a new simulated stream.

func (*SimulatedStream) GetProgress

func (s *SimulatedStream) GetProgress() float64

GetProgress returns the current progress (0-1).

func (*SimulatedStream) Next

func (s *SimulatedStream) Next() (chunk string, delay time.Duration, done bool)

Next returns the next chunk and delay.

func (*SimulatedStream) Reset

func (s *SimulatedStream) Reset()

Reset resets the simulated stream.

type SimulatedStreamConfig

type SimulatedStreamConfig struct {
	TypingSpeed  int     // Characters per second
	GlitchChance float64 // Chance of corruption effect (0-1)
	MinDelay     time.Duration
	MaxDelay     time.Duration
}

SimulatedStreamConfig holds configuration for simulated streaming.

func DefaultSimulatedConfig

func DefaultSimulatedConfig() SimulatedStreamConfig

DefaultSimulatedConfig returns default simulated streaming config.

type StreamCallback

type StreamCallback func(chunk StreamChunk)

StreamCallback is called for each chunk during streaming.

type StreamChunk

type StreamChunk struct {
	Content      string
	IsFirst      bool
	IsFinal      bool
	FinishReason string
	ToolCalls    []ToolCallResult
	Usage        *TokenUsage // Only populated on final chunk with stream_options
}

type StreamEvent added in v1.7.0

type StreamEvent struct {
	// Type identifies which kind of event this is.
	Type StreamEventType

	// ContentDelta contains the text delta (only for EventContentDelta).
	ContentDelta string

	// ToolUseID is the unique identifier for the tool call
	// (set for EventToolUseStart, EventToolUseInputDelta, EventToolUseDone).
	ToolUseID string

	// ToolName is the function name being called
	// (set for EventToolUseStart, EventToolUseDone).
	ToolName string

	// InputDelta contains partial argument JSON
	// (only for EventToolUseInputDelta).
	InputDelta string

	// CompleteInput contains the fully accumulated argument JSON
	// (only for EventToolUseDone).
	CompleteInput string

	// Usage contains token usage statistics (only for EventMessageDone).
	Usage *TokenUsage

	// FinishReason indicates why the response ended (only for EventMessageDone).
	FinishReason string
}

StreamEvent represents a single granular event during LLM streaming. Unlike StreamChunk which batches tool calls into the final chunk, StreamEvent delivers tool call information incrementally as it arrives.

func (StreamEvent) IsToolEvent added in v1.7.0

func (e StreamEvent) IsToolEvent() bool

IsToolEvent returns true if this event relates to a tool use block.

type StreamEventCallback added in v1.7.0

type StreamEventCallback func(event StreamEvent)

StreamEventCallback is called for each streaming event.

type StreamEventType added in v1.7.0

type StreamEventType int

StreamEventType identifies the kind of streaming event.

const (
	// EventContentDelta is emitted when the LLM produces a text content delta.
	EventContentDelta StreamEventType = iota

	// EventToolUseStart is emitted when a new tool_use block begins streaming.
	// Contains the tool call ID and tool name.
	EventToolUseStart

	// EventToolUseInputDelta is emitted as argument JSON accumulates for a tool call.
	// Contains partial JSON input for the tool call identified by ToolUseID.
	EventToolUseInputDelta

	// EventToolUseDone is emitted when a tool call's arguments are fully received.
	// Contains the complete input JSON, ready for execution.
	EventToolUseDone

	// EventMessageDone is emitted when the entire LLM response is complete.
	// Contains usage statistics and finish reason.
	EventMessageDone
)

func (StreamEventType) String added in v1.7.0

func (t StreamEventType) String() string

String returns a human-readable name for the event type.

type StreamState

type StreamState struct {
	// contains filtered or unexported fields
}

StreamState tracks the state of a streaming response.

func NewStreamState

func NewStreamState() *StreamState

NewStreamState creates a new stream state tracker.

func (*StreamState) AddChunk

func (s *StreamState) AddChunk(content string)

AddChunk adds a chunk to the stream state.

func (*StreamState) GetChunkCount

func (s *StreamState) GetChunkCount() int

GetChunkCount returns the number of chunks received.

func (*StreamState) GetContent

func (s *StreamState) GetContent() string

GetContent returns the accumulated content.

func (*StreamState) GetDuration

func (s *StreamState) GetDuration() time.Duration

GetDuration returns the total stream duration.

func (*StreamState) IsComplete

func (s *StreamState) IsComplete() bool

IsComplete returns whether the stream is complete.

func (*StreamState) IsDump

func (s *StreamState) IsDump() bool

IsDump returns whether the stream was a dump (all at once).

func (*StreamState) MarkComplete

func (s *StreamState) MarkComplete()

MarkComplete marks the stream as complete.

type TokenUsage

type TokenUsage struct {
	PromptTokens     int
	CompletionTokens int
	TotalTokens      int
}

StreamChunk represents a streaming chunk. TokenUsage holds token usage information from API response

type ToolCallResult

type ToolCallResult struct {
	ID        string
	Name      string
	Arguments string
}

ToolCallResult holds a tool call from the LLM.

type ToolUseAccumulator added in v1.7.0

type ToolUseAccumulator struct {
	// contains filtered or unexported fields
}

ToolUseAccumulator tracks in-progress tool uses during streaming, collecting input deltas and producing completed ToolCallResult values. It is safe for use from a single goroutine (the streaming callback goroutine).

func NewToolUseAccumulator added in v1.7.0

func NewToolUseAccumulator() *ToolUseAccumulator

NewToolUseAccumulator creates a new accumulator for tracking tool uses.

func (*ToolUseAccumulator) CompletedCalls added in v1.7.0

func (a *ToolUseAccumulator) CompletedCalls() []ToolCallResult

CompletedCalls returns all tool calls that have finished receiving input, in the order they were started.

func (*ToolUseAccumulator) HandleEvent added in v1.7.0

func (a *ToolUseAccumulator) HandleEvent(event StreamEvent)

HandleEvent processes a StreamEvent and updates internal state. For EventToolUseStart: creates a new pending tool use. For EventToolUseInputDelta: appends to the pending tool's input buffer. For EventToolUseDone: moves the tool use to the completed list. Other event types are ignored.

func (*ToolUseAccumulator) PendingCount added in v1.7.0

func (a *ToolUseAccumulator) PendingCount() int

PendingCount returns the number of tool uses still accumulating input.

func (*ToolUseAccumulator) Reset added in v1.7.0

func (a *ToolUseAccumulator) Reset()

Reset clears all state.

type XAIBackend

type XAIBackend struct {
	// contains filtered or unexported fields
}

XAIBackend implements LLMBackend using xAI's native API. This backend supports xAI-specific features like Collections (RAG).

func NewXAIBackend

func NewXAIBackend(config *Config, registry *tools.Registry) (*XAIBackend, error)

NewXAIBackend creates a new xAI backend with Collections support.

func (*XAIBackend) ChangeModel

func (b *XAIBackend) ChangeModel(model string) error

ChangeModel changes the model

func (*XAIBackend) Close

func (b *XAIBackend) Close() error

Close cleans up resources (implements LLMBackend interface)

func (*XAIBackend) GetSkills

func (b *XAIBackend) GetSkills() []tui.SkillDefinition

GetSkills returns the list of available skills from the registry

func (*XAIBackend) SendMessageStream

func (b *XAIBackend) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback.

func (*XAIBackend) SendMessageStreamEvents added in v1.7.0

func (b *XAIBackend) SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamEventCallback) error

SendMessageStreamEvents sends a message with granular streaming events.

func (*XAIBackend) SendMessageSync

func (b *XAIBackend) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync collects the xAI stream into a single synchronous result.

func (*XAIBackend) SetSystemPrompt

func (b *XAIBackend) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona).

func (*XAIBackend) SwitchEndpoint

func (b *XAIBackend) SwitchEndpoint(endpoint string) error

SwitchEndpoint switches to a different endpoint (for config switching)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL