llm

package
v1.5.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 25, 2026 License: MIT Imports: 17 Imported by: 0

Documentation

Overview

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client abstraction for Celeste CLI.

Package llm provides the LLM client for Celeste CLI. This file contains streaming-specific functionality.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CalculateTargetTokens

func CalculateTargetTokens(maxTokens int) int

CalculateTargetTokens calculates the target token count for compaction. Typically aims for 70% of max context window.

func EstimateSummarySavings

func EstimateSummarySavings(messages []config.SessionMessage, count int) (int, int)

EstimateSummarySavings estimates how many tokens would be saved by summarization. Returns estimated tokens before and after summarization.

func FormatCompactionResult

func FormatCompactionResult(messagesBefore, messagesAfter, tokensBefore, tokensAfter int) string

FormatCompactionResult creates a user-friendly message about compaction results.

func MarshalSummaryMetadata

func MarshalSummaryMetadata(messagesBefore, messagesAfter, tokensBefore, tokensAfter int) (string, error)

MarshalSummaryMetadata creates JSON metadata about the summarization. Useful for logging and debugging.

func ShouldTriggerCompaction

func ShouldTriggerCompaction(session *config.Session, model string) bool

ShouldTriggerCompaction determines if automatic compaction should trigger. Checks if session has reached 80% of context window capacity.

func ValidateCompactionSavings

func ValidateCompactionSavings(currentTokens, targetTokens int) bool

ValidateCompactionSavings checks if compaction would achieve meaningful token savings. Returns true if savings would be at least 20% of current tokens.

Types

type BackendType

type BackendType string

BackendType identifies which SDK implementation is being used.

const (
	// BackendTypeOpenAI uses the go-openai SDK (OpenAI, Venice, Anthropic, etc.)
	BackendTypeOpenAI BackendType = "openai"

	// BackendTypeGoogle uses the native Google GenAI SDK (Gemini, Vertex AI)
	BackendTypeGoogle BackendType = "google"

	// BackendTypeXAI uses the native xAI SDK with Collections support (Grok models)
	BackendTypeXAI BackendType = "xai"
)

func DetectBackendType

func DetectBackendType(baseURL string) BackendType

DetectBackendType determines which backend to use based on the base URL.

type ChatCompletionResult

type ChatCompletionResult struct {
	Content      string
	ToolCalls    []ToolCallResult
	FinishReason string
	Error        error
}

ChatCompletionResult holds the result of a chat completion.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client wraps LLM backends and provides a unified interface. It automatically selects the appropriate backend (OpenAI or Google) based on the provider.

func NewClient

func NewClient(config *Config, registry *skills.Registry) *Client

NewClient creates a new LLM client with automatic backend selection. It detects whether to use OpenAI SDK, Google GenAI SDK, or xAI SDK based on the base URL.

func (*Client) ExecuteSkill

func (c *Client) ExecuteSkill(ctx context.Context, name string, argsJSON string) (*skills.ExecutionResult, error)

ExecuteSkill executes a skill and returns the result.

func (*Client) GetConfig

func (c *Client) GetConfig() *Config

GetConfig returns the current configuration.

func (*Client) GetSkills

func (c *Client) GetSkills() []tui.SkillDefinition

GetSkills returns skill definitions for the TUI.

func (*Client) SendMessageStream

func (c *Client) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback. This delegates to the appropriate backend (OpenAI or Google).

func (*Client) SendMessageSync

func (c *Client) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync sends a message synchronously and returns the result. This delegates to the appropriate backend (OpenAI or Google).

func (*Client) SetSystemPrompt

func (c *Client) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona).

func (*Client) UpdateConfig

func (c *Client) UpdateConfig(config *Config)

UpdateConfig updates the client configuration and recreates the backend if needed. This allows dynamic endpoint/model switching during runtime.

type Config

type Config struct {
	APIKey            string
	BaseURL           string
	Model             string
	Timeout           time.Duration
	SkipPersonaPrompt bool
	SimulateTyping    bool
	TypingSpeed       int // chars per second

	// Google Cloud authentication (for Gemini/Vertex AI)
	GoogleCredentialsFile string // Path to service account JSON file
	GoogleUseADC          bool   // Use Application Default Credentials

	// Collections (xAI only)
	Collections *config.CollectionsConfig
	XAIFeatures *config.XAIFeaturesConfig
}

Config holds LLM client configuration.

type GoogleBackend

type GoogleBackend struct {
	// contains filtered or unexported fields
}

GoogleBackend implements LLMBackend using Google's native GenAI SDK. This backend supports Gemini AI Studio and Vertex AI with automatic authentication.

func NewGoogleBackend

func NewGoogleBackend(config *Config) (*GoogleBackend, error)

NewGoogleBackend creates a new Google GenAI backend with automatic authentication. Authentication methods (in order of priority): 1. Simple API key (for Gemini AI Studio) 2. GoogleCredentialsFile in config (service account JSON) 3. GOOGLE_APPLICATION_CREDENTIALS environment variable 4. Application Default Credentials (gcloud auth application-default login)

func (*GoogleBackend) Close

func (b *GoogleBackend) Close() error

Close cleans up resources.

func (*GoogleBackend) SendMessageStream

func (b *GoogleBackend) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback.

func (*GoogleBackend) SendMessageSync

func (b *GoogleBackend) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync sends a message synchronously and returns the complete result.

func (*GoogleBackend) SetSystemPrompt

func (b *GoogleBackend) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona).

type LLMBackend

type LLMBackend interface {
	// SendMessageStream sends a message with streaming callback.
	// The callback receives chunks as they arrive from the LLM.
	// Returns error if the request fails.
	SendMessageStream(ctx context.Context, messages []tui.ChatMessage,
		tools []tui.SkillDefinition, callback StreamCallback) error

	// SendMessageSync sends a message and returns the complete result.
	// This is useful for non-streaming use cases or testing.
	// Returns the full chat completion result or error.
	SendMessageSync(ctx context.Context, messages []tui.ChatMessage,
		tools []tui.SkillDefinition) (*ChatCompletionResult, error)

	// SetSystemPrompt sets the system prompt (Celeste persona).
	// This configures the LLM's behavior and character.
	SetSystemPrompt(prompt string)

	// Close cleans up resources (e.g., network connections).
	// Should be called when the backend is no longer needed.
	Close() error
}

LLMBackend defines the interface all LLM backends must implement. This abstraction allows Celeste to support multiple SDK implementations: - OpenAI SDK (go-openai) for OpenAI, Grok, Venice, Anthropic, etc. - Google GenAI SDK (google.golang.org/genai) for Gemini and Vertex AI

type OpenAIBackend

type OpenAIBackend struct {
	// contains filtered or unexported fields
}

OpenAIBackend implements LLMBackend using the go-openai SDK. This backend supports OpenAI, Grok, Venice, Anthropic, and other OpenAI-compatible providers.

func NewOpenAIBackend

func NewOpenAIBackend(config *Config) *OpenAIBackend

NewOpenAIBackend creates a new OpenAI-compatible backend.

func (*OpenAIBackend) Close

func (b *OpenAIBackend) Close() error

Close cleans up resources (no-op for OpenAI backend).

func (*OpenAIBackend) SendMessageStream

func (b *OpenAIBackend) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback.

func (*OpenAIBackend) SendMessageSync

func (b *OpenAIBackend) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync sends a message synchronously and returns the complete result.

func (*OpenAIBackend) SetSystemPrompt

func (b *OpenAIBackend) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona).

type SimulatedStream

type SimulatedStream struct {
	// contains filtered or unexported fields
}

SimulatedStream simulates streaming for dump responses.

func NewSimulatedStream

func NewSimulatedStream(content string, config SimulatedStreamConfig) *SimulatedStream

NewSimulatedStream creates a new simulated stream.

func (*SimulatedStream) GetProgress

func (s *SimulatedStream) GetProgress() float64

GetProgress returns the current progress (0-1).

func (*SimulatedStream) Next

func (s *SimulatedStream) Next() (chunk string, delay time.Duration, done bool)

Next returns the next chunk and delay.

func (*SimulatedStream) Reset

func (s *SimulatedStream) Reset()

Reset resets the simulated stream.

type SimulatedStreamConfig

type SimulatedStreamConfig struct {
	TypingSpeed  int     // Characters per second
	GlitchChance float64 // Chance of corruption effect (0-1)
	MinDelay     time.Duration
	MaxDelay     time.Duration
}

SimulatedStreamConfig holds configuration for simulated streaming.

func DefaultSimulatedConfig

func DefaultSimulatedConfig() SimulatedStreamConfig

DefaultSimulatedConfig returns default simulated streaming config.

type StreamCallback

type StreamCallback func(chunk StreamChunk)

StreamCallback is called for each chunk during streaming.

type StreamChunk

type StreamChunk struct {
	Content      string
	IsFirst      bool
	IsFinal      bool
	FinishReason string
	ToolCalls    []ToolCallResult
	Usage        *TokenUsage // Only populated on final chunk with stream_options
}

type StreamState

type StreamState struct {
	// contains filtered or unexported fields
}

StreamState tracks the state of a streaming response.

func NewStreamState

func NewStreamState() *StreamState

NewStreamState creates a new stream state tracker.

func (*StreamState) AddChunk

func (s *StreamState) AddChunk(content string)

AddChunk adds a chunk to the stream state.

func (*StreamState) GetChunkCount

func (s *StreamState) GetChunkCount() int

GetChunkCount returns the number of chunks received.

func (*StreamState) GetContent

func (s *StreamState) GetContent() string

GetContent returns the accumulated content.

func (*StreamState) GetDuration

func (s *StreamState) GetDuration() time.Duration

GetDuration returns the total stream duration.

func (*StreamState) IsComplete

func (s *StreamState) IsComplete() bool

IsComplete returns whether the stream is complete.

func (*StreamState) IsDump

func (s *StreamState) IsDump() bool

IsDump returns whether the stream was a dump (all at once).

func (*StreamState) MarkComplete

func (s *StreamState) MarkComplete()

MarkComplete marks the stream as complete.

type Summarizer

type Summarizer struct {
	// contains filtered or unexported fields
}

Summarizer handles automatic conversation summarization for context management.

func NewSummarizer

func NewSummarizer(client *Client) *Summarizer

NewSummarizer creates a new conversation summarizer.

func (*Summarizer) CompactSession

func (s *Summarizer) CompactSession(session *config.Session, targetTokens int) (int, int, error)

CompactSession performs context compaction by summarizing old messages. targetTokens specifies the desired token count after compaction (typically 70% of max). Returns the number of messages before and after compaction.

func (*Summarizer) SummarizeMessages

func (s *Summarizer) SummarizeMessages(messages []config.SessionMessage, count int) (string, error)

SummarizeMessages creates a concise summary of conversation messages. Returns a summary message that preserves key context while reducing token count.

type TokenUsage

type TokenUsage struct {
	PromptTokens     int
	CompletionTokens int
	TotalTokens      int
}

StreamChunk represents a streaming chunk. TokenUsage holds token usage information from API response

type ToolCallResult

type ToolCallResult struct {
	ID        string
	Name      string
	Arguments string
}

ToolCallResult holds a tool call from the LLM.

type XAIBackend

type XAIBackend struct {
	// contains filtered or unexported fields
}

XAIBackend implements LLMBackend using xAI's native API. This backend supports xAI-specific features like Collections (RAG).

func NewXAIBackend

func NewXAIBackend(config *Config, registry *skills.Registry) (*XAIBackend, error)

NewXAIBackend creates a new xAI backend with Collections support.

func (*XAIBackend) ChangeModel

func (b *XAIBackend) ChangeModel(model string) error

ChangeModel changes the model

func (*XAIBackend) Close

func (b *XAIBackend) Close() error

Close cleans up resources (implements LLMBackend interface)

func (*XAIBackend) GetSkills

func (b *XAIBackend) GetSkills() []tui.SkillDefinition

GetSkills returns the list of available skills from the registry

func (*XAIBackend) SendMessageStream

func (b *XAIBackend) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback.

func (*XAIBackend) SendMessageSync

func (b *XAIBackend) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync sends a message synchronously (not implemented for xAI backend).

func (*XAIBackend) SetSystemPrompt

func (b *XAIBackend) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona).

func (*XAIBackend) SwitchEndpoint

func (b *XAIBackend) SwitchEndpoint(endpoint string) error

SwitchEndpoint switches to a different endpoint (for config switching)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL