Help us shape the future of Go! Take the 2026 Go Developer Survey

llm

package
v2.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 9, 2026 License: MIT Imports: 31 Imported by: 0

Documentation

Overview

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI.

Package llm provides the LLM client for Celeste CLI. This file defines StreamEvent types for granular streaming events, replacing the batch-oriented StreamChunk for tool call delivery.

Package llm provides the LLM client abstraction for Celeste CLI.

Package llm provides the LLM client for Celeste CLI.

Index

Constants

View Source 
const (
	BlocksAnthropicMessages = "anthropic-messages" // Messages API content blocks (W2, W1)
	BlocksOpenAIResponses   = "openai-responses"   // Responses API output items (W8)
)

Provider-block formats (2.0 F3): the wire format a backend's ProviderBlocks are in.

Variables

View Source 
var ErrContextOverflow = errors.New("the conversation no longer fits the model's context window")

ErrContextOverflow marks a request rejected because the conversation no longer fits the model's context window. Match it with errors.Is.

View Source 
var ErrRuleInterrupt = errors.New("stream interrupted by a steering rule")

ErrRuleInterrupt marks a stream a steering rule or the watchdog cut short (2.0 W3). withRetry never retries it: the loop re-runs the turn itself, with the rule's reminder.

View Source 
var ErrStalled = errors.New("no data from the model")

ErrStalled marks a request that received nothing from the provider for the whole stall timeout (the config's `timeout`). Match it with errors.Is.

Functions

func BlocksRejectedIn 

func BlocksRejectedIn(err error) bool

BlocksRejectedIn reports whether err says the provider refused the replayed blocks (a *BlocksRejectedError anywhere in its chain).

func MaxRequestDuration 

func MaxRequestDuration(stall time.Duration) time.Duration

MaxRequestDuration bounds one request however steadily it streams: 30 minutes, or three times the stall timeout when that is longer. The stall timeout catches a dead connection; this catches a reply that never ends.

func ProviderKey 

func ProviderKey(format, baseURL, model string) string

ProviderKey names the backend whose blocks a message carries: format|endpoint|model, the endpoint trimmed, lower-cased and without a trailing slash. tui.ReplayBlocks replays blocks only to the same key, so after an endpoint or model switch the provider-neutral Content and ToolCalls are sent instead. A backend computes it at request time from the effective base URL (after its default is applied) and the model it actually sends (after served-model resolution), never from the raw config, so the key it captures under equals the key it replays under.

Types

type AnthropicBackend 

type AnthropicBackend struct {
	// contains filtered or unexported fields
}

AnthropicBackend implements LLMBackend using the native Anthropic SDK. This backend supports Claude models via the Anthropic Messages API with prompt caching, extended thinking, and native streaming.

func NewAnthropicBackend 

func NewAnthropicBackend(config *Config) (*AnthropicBackend, error)

NewAnthropicBackend creates a new Anthropic backend using the native SDK.

func (*AnthropicBackend) Close 

func (b *AnthropicBackend) Close() error

Close cleans up resources (no-op for Anthropic backend).

func (*AnthropicBackend) SendMessageStream 

func (b *AnthropicBackend) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback.

func (*AnthropicBackend) SendMessageStreamEvents 

func (b *AnthropicBackend) SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamEventCallback) error

SendMessageStreamEvents sends a message with granular streaming events.

func (*AnthropicBackend) SendMessageSync 

func (b *AnthropicBackend) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync sends a message and returns the complete result.

func (*AnthropicBackend) SetSystemPrompt 

func (b *AnthropicBackend) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona). A change from a previous prompt makes the next request drop replayed blocks: their signatures cover the old prompt (2.0 W2 ruling 10).

func (*AnthropicBackend) SetSystemPromptParts 

func (b *AnthropicBackend) SetSystemPromptParts(static, dynamic string)

SetSystemPromptParts sets the system prompt as its byte-stable static part (the persona, prompts.Prompt.Static) and the dynamic rest (user identity, sliders, project context, git, date). Requests carry a cache breakpoint right after the static part (#309).

func (*AnthropicBackend) SetThinkingConfig 

func (b *AnthropicBackend) SetThinkingConfig(config ThinkingConfig)

SetThinkingConfig configures extended thinking for Claude models. Claude supports budget_tokens via ThinkingConfigParam.

type BackendType 

type BackendType string

BackendType identifies which SDK implementation is being used. 

const (
	// BackendTypeOpenAI uses the go-openai SDK (OpenAI, Venice, Anthropic, etc.)
	BackendTypeOpenAI BackendType = "openai"

	// BackendTypeGoogle uses the native Google GenAI SDK (Gemini, Vertex AI)
	BackendTypeGoogle BackendType = "google"

	// BackendTypeXAI uses the native xAI SDK with Collections support (Grok models)
	BackendTypeXAI BackendType = "xai"

	// BackendTypeAnthropic uses the native Anthropic SDK (Claude models)
	BackendTypeAnthropic BackendType = "anthropic"

	// BackendTypeOpenAIResponses uses OpenAI's Responses API (2.0 W8): the
	// openai provider, or any provider whose registry entry declares it.
	BackendTypeOpenAIResponses BackendType = "openai-responses"
)

func DetectBackendType 

func DetectBackendType(baseURL string) BackendType

DetectBackendType determines which backend to use based on the base URL. Anthropic, xAI and Google are decided by host, by the same rules providers.DetectProvider uses (#372, #377), so the header and the backend choice never disagree and a proxy path that names one of those domains stays OpenAI-compatible.

type BlocksRejectedError 

type BlocksRejectedError struct{ Err error }

BlocksRejectedError is a request's error when the provider refused the blocks it replayed and the resend without them failed too (W8-1 review M4). The refusal still holds: the caller strips the history's blocks (2.0 F3), as it does for a reply carrying BlocksRejected.

func (*BlocksRejectedError) Error 

func (e *BlocksRejectedError) Error() string

func (*BlocksRejectedError) Unwrap 

func (e *BlocksRejectedError) Unwrap() error

type ChatCompletionResult 

type ChatCompletionResult struct {
	Content      string
	ToolCalls    []ToolCallResult
	FinishReason string
	Error        error
	Usage        *TokenUsage // Token usage from the API response (if available)

	ProviderBlocks *tui.ProviderBlocks // the reply as the provider sent it (2.0 F3); nil from backends that keep none
	BlocksRejected bool                // the provider refused the replayed blocks; the caller strips them (2.0 F3)
}

ChatCompletionResult holds the result of a chat completion.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client wraps LLM backends and provides a unified interface. It automatically selects the appropriate backend (OpenAI or Google) based on the provider.

A Client is safe for concurrent use: the TUI switches model while a turn streams on the same client. mu guards backend, config, backendType, systemPrompt, thinking and toolMode. A request takes one snapshot of backend and config per attempt and keeps using that backend even if UpdateConfig replaces it meanwhile; no lock is held across a network call or a callback.

func NewClient 

func NewClient(config *Config, registry *tools.Registry) *Client

NewClient creates a new LLM client with automatic backend selection (resolveBackendType).

func NewClientWithBackend 

func NewClientWithBackend(config *Config, registry *tools.Registry, backend LLMBackend) *Client

NewClientWithBackend builds a Client around an existing backend, skipping backend detection. Tests use it to drive the agent loop with a fake.

func (*Client) GetConfig 

func (c *Client) GetConfig() *Config

GetConfig returns the current configuration.

func (*Client) GetSkills 

func (c *Client) GetSkills() []tui.SkillDefinition

GetSkills returns the tools the model may call in this client's mode, fitted to the context window next to the system prompt (compact.FitTools): on a small window a core set with short descriptions, plus the tools find_tools activated (the newest first while they fit). When that reduces the set, TakeToolNotice says so once.

func (*Client) SendMessageStream 

func (c *Client) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback. This delegates to the appropriate backend (OpenAI or Google).

func (*Client) SendMessageStreamEvents 

func (c *Client) SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamEventCallback) error

SendMessageStreamEvents sends a message with granular streaming events. This delegates to the appropriate backend.

func (*Client) SendMessageSync 

func (c *Client) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync sends a message synchronously and returns the result. This delegates to the appropriate backend (OpenAI or Google). Every attempt sends messages unchanged (2.0 F3: the history is append-only).

func (*Client) ServerCompaction 

func (c *Client) ServerCompaction(ctx context.Context) CompactionSupport

ServerCompaction reports whether this client's endpoint serves server-side compaction (2.0 W8; W1 consumes it). Backends without a probe report CompactionUnsupported.

func (*Client) SetSystemPrompt 

func (c *Client) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona) as one piece.

func (*Client) SetSystemPromptParts 

func (c *Client) SetSystemPromptParts(static, dynamic string)

SetSystemPromptParts sets the system prompt from prompts.Compose's parts: the byte-stable persona (Prompt.Static) and the rest (Prompt.Dynamic). A backend that caches the persona on its own gets the parts; the others get the whole prompt, prompts.Prompt.String.

func (*Client) SetThinkingConfig 

func (c *Client) SetThinkingConfig(config ThinkingConfig)

SetThinkingConfig configures extended thinking / reasoning effort. It is kept and re-applied when UpdateConfig rebuilds the backend.

func (*Client) SetToolFilter 

func (c *Client) SetToolFilter(fn func([]tui.SkillDefinition) []tui.SkillDefinition)

SetToolFilter makes GetSkills narrow the tools with fn before fitting them to the window (the chat drops submit_plan outside plan mode, and every write tool in it), so the fit spends its budget only on tools that are sent and TakeToolNotice counts those (K2). fn is called without the client's lock held and must not modify its argument.

func (*Client) SetToolMode 

func (c *Client) SetToolMode(mode tools.RuntimeMode)

SetToolMode sets which runtime mode's tools GetSkills offers the model. Agent runs call this with tools.ModeAgent; everything else stays on the default, tools.ModeChat.

func (*Client) SetWindowFunc 

func (c *Client) SetWindowFunc(fn func() int)

SetWindowFunc makes GetSkills fit the tools to the window fn returns (the chat follows its live model and context_limit). fn is called without the client's lock held, so it may call the client.

func (*Client) ShareToolNotices 

func (c *Client) ShareToolNotices(n *compact.ToolNotices)

ShareToolNotices makes TakeToolNotice dedupe through n, shared with other clients: a server whose every request has its own client tells its log once per fit, not once per request.

func (*Client) SystemPrompt 

func (c *Client) SystemPrompt() string

SystemPrompt returns the prompt last set with SetSystemPrompt.

func (*Client) TakeToolNotice 

func (c *Client) TakeToolNotice() string

TakeToolNotice returns the notice for GetSkills' last fit, once per fit for this client ("" when it was not reduced or was already told). The caller shows it, as it shows the persona guard's.

func (*Client) UpdateConfig 

func (c *Client) UpdateConfig(config *Config)

UpdateConfig switches the client to config. The backend is rebuilt whenever any field differs, so the next request goes to the new endpoint with the new model and the rest (timeout, collections, credentials) reaches the backend too; the system prompt and thinking config carry over (2.0 W8 ruling 12). A request already in flight finishes on the backend it started with, so the old backend is not closed (every backend's Close is a no-op). The backend is built outside the lock.

type CompactionProber 

type CompactionProber interface {
	ProbeCompaction(ctx context.Context) CompactionSupport
}

CompactionProber is implemented by backends that can ask their endpoint whether it serves server-side compaction. W1 decides from it whether a session compacts on the server or on the client ladder.

type CompactionSupport 

type CompactionSupport int

CompactionSupport is what a live probe learned about an endpoint's server-side compaction (/responses/compact, 2.0 W8). 

const (
	// CompactionUnknown: the probe got no definite answer (auth, rate
	// limit, server error, network). Use the client ladder this time.
	CompactionUnknown CompactionSupport = iota
	// CompactionSupported: the endpoint serves /responses/compact.
	CompactionSupported
	// CompactionUnsupported: it does not; use the client ladder.
	CompactionUnsupported
)

func (CompactionSupport) String 

func (s CompactionSupport) String() string

type Config 

type Config struct {
	APIKey  string
	BaseURL string
	Model   string
	// Backend forces a backend instead of detecting it from BaseURL. Tests
	// use it to reach a fake provider on 127.0.0.1 with a native backend.
	// Empty keeps detection.
	Backend BackendType
	Timeout time.Duration
	// FirstByteTimeout is how long a request may wait for the first byte
	// of the reply, when that is longer than Timeout (the stall timeout
	// between chunks). A local server prefilling a long prompt sends
	// nothing for minutes (#359). Zero, or anything up to Timeout, means
	// Timeout covers the first byte too. FirstByteBudget applies the cap.
	FirstByteTimeout time.Duration

	// Google Cloud authentication (for Gemini/Vertex AI)
	GoogleCredentialsFile string // Path to service account JSON file
	GoogleUseADC          bool   // Use Application Default Credentials

	// ContextLimit is the user's context_limit (0: resolved from the model
	// and endpoint). GetSkills fits the tool schemas to the window (#310).
	ContextLimit int

	// Collections (xAI only)
	Collections *config.CollectionsConfig
	XAIFeatures *config.XAIFeaturesConfig
}

Config holds LLM client configuration.

func ConfigFrom 

func ConfigFrom(cfg *config.Config) *Config

ConfigFrom is the one place a client Config is built from the user's configuration, so every path (chat, endpoint switch, single message, MCP, agent, summarizer) carries the same fields. A path that needs something different overrides it on the result at its call site.

func (*Config) FirstByteBudget 

func (c *Config) FirstByteBudget() time.Duration

FirstByteBudget is how long a request made with c may wait for the first byte of the reply: FirstByteTimeout when it is longer than the stall timeout, capped by RequestCap; otherwise the stall timeout.

func (*Config) Plain 

func (c *Config) Plain() *Config

Plain returns a copy of c for a one-shot completion that carries its own system prompt (a summary, an oracle question): no xAI collections or features. c is not modified.

func (*Config) RequestCap 

func (c *Config) RequestCap() time.Duration

RequestCap bounds one request made with c however steadily it streams: MaxRequestDuration of the stall timeout. Work that is one request to the model, such as a compaction summary, is bounded by it too (#345).

func (*Config) StallTimeout 

func (c *Config) StallTimeout() time.Duration

StallTimeout is the stall timeout of requests made with c: c.Timeout, or 60 s when c (which may be nil) carries none.

type GoogleBackend 

type GoogleBackend struct {
	// contains filtered or unexported fields
}

GoogleBackend implements LLMBackend using Google's native GenAI SDK. This backend supports Gemini AI Studio and Vertex AI with automatic authentication.

func NewGoogleBackend 

func NewGoogleBackend(config *Config) (*GoogleBackend, error)

NewGoogleBackend creates a new Google GenAI backend with automatic authentication. Authentication methods (in order of priority): 1. Simple API key (for Gemini AI Studio) 2. GoogleCredentialsFile in config (service account JSON) 3. GOOGLE_APPLICATION_CREDENTIALS environment variable 4. Application Default Credentials (gcloud auth application-default login)

func (*GoogleBackend) Close 

func (b *GoogleBackend) Close() error

Close cleans up resources.

func (*GoogleBackend) SendMessageStream 

func (b *GoogleBackend) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback: the text of SendMessageStreamEvents as chunks, then one final chunk with the complete tool calls and the finish reason.

func (*GoogleBackend) SendMessageStreamEvents 

func (b *GoogleBackend) SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamEventCallback) error

SendMessageStreamEvents sends a message with granular streaming events.

func (*GoogleBackend) SendMessageSync 

func (b *GoogleBackend) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync sends a message and returns the complete result. It streams internally (#349): a whole-reply request sends nothing until the reply is complete, so a long one (a compaction summary) looked idle and failed with ErrStalled after the stall timeout. Streamed, every chunk's bytes reach the stall watch, and each chunk also counts as activity. It accumulates SendMessageStreamEvents, the backend's one reading of the stream (audit C3).

func (*GoogleBackend) SetSystemPrompt 

func (b *GoogleBackend) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona).

func (*GoogleBackend) SetThinkingConfig 

func (b *GoogleBackend) SetThinkingConfig(config ThinkingConfig)

SetThinkingConfig configures extended thinking for Gemini models. Gemini supports thinkingBudget via GenerateContentConfig.ThinkingConfig.

type LLMBackend 

type LLMBackend interface {
	// SendMessageStream sends a message with streaming callback.
	// The callback receives chunks as they arrive from the LLM.
	// Returns error if the request fails.
	SendMessageStream(ctx context.Context, messages []tui.ChatMessage,
		tools []tui.SkillDefinition, callback StreamCallback) error

	// SendMessageStreamEvents sends a message with granular streaming events.
	// Unlike SendMessageStream which batches tool calls into the final chunk,
	// this method delivers tool call information incrementally as it arrives.
	SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage,
		tools []tui.SkillDefinition, callback StreamEventCallback) error

	// SendMessageSync sends a message and returns the complete result.
	// This is useful for non-streaming use cases or testing.
	// Returns the full chat completion result or error.
	SendMessageSync(ctx context.Context, messages []tui.ChatMessage,
		tools []tui.SkillDefinition) (*ChatCompletionResult, error)

	// SetSystemPrompt sets the system prompt (Celeste persona).
	// This configures the LLM's behavior and character.
	SetSystemPrompt(prompt string)

	// SetThinkingConfig configures extended thinking / reasoning effort.
	// Backends that don't support thinking silently ignore the config.
	SetThinkingConfig(config ThinkingConfig)

	// Close cleans up resources (e.g., network connections).
	// Should be called when the backend is no longer needed.
	Close() error
}

LLMBackend defines the interface all LLM backends must implement. This abstraction allows Celeste to support multiple SDK implementations: - OpenAI SDK (go-openai) for OpenAI, Grok, Venice, Anthropic, etc. - Google GenAI SDK (google.golang.org/genai) for Gemini and Vertex AI

type OpenAIBackend 

type OpenAIBackend struct {
	// contains filtered or unexported fields
}

OpenAIBackend implements LLMBackend using the go-openai SDK. This backend supports OpenAI, Grok, Venice, Anthropic, and other OpenAI-compatible providers.

func NewOpenAIBackend 

func NewOpenAIBackend(config *Config) *OpenAIBackend

NewOpenAIBackend creates a new OpenAI-compatible backend.

func (*OpenAIBackend) Close 

func (b *OpenAIBackend) Close() error

Close cleans up resources (no-op for OpenAI backend).

func (*OpenAIBackend) SendMessageStream 

func (b *OpenAIBackend) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback.

func (*OpenAIBackend) SendMessageStreamEvents 

func (b *OpenAIBackend) SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamEventCallback) error

SendMessageStreamEvents sends a message with granular streaming events.

func (*OpenAIBackend) SendMessageSync 

func (b *OpenAIBackend) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync sends a message synchronously and returns the complete result.

func (*OpenAIBackend) SetSystemPrompt 

func (b *OpenAIBackend) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona).

func (*OpenAIBackend) SetThinkingConfig 

func (b *OpenAIBackend) SetThinkingConfig(config ThinkingConfig)

SetThinkingConfig configures extended thinking / reasoning effort. For OpenAI o-series models this maps to reasoning_effort. For Anthropic (via OpenAI compat) this is a no-op for now.

type ResponsesBackend 

type ResponsesBackend struct {
	// contains filtered or unexported fields
}

ResponsesBackend talks to OpenAI's Responses API (POST /v1/responses) with streaming, tool calls and reasoning items (2.0 W8). Each reply's output items become the assistant message's ProviderBlocks and are replayed on later turns. The whole history is sent every time (store=false; previous_response_id is never used).

func NewResponsesBackend 

func NewResponsesBackend(config *Config) *ResponsesBackend

NewResponsesBackend creates a Responses backend for config.

func (*ResponsesBackend) Close 

func (b *ResponsesBackend) Close() error

func (*ResponsesBackend) ProbeCompaction 

func (b *ResponsesBackend) ProbeCompaction(ctx context.Context) CompactionSupport

ProbeCompaction asks the endpoint once per process whether it serves POST /responses/compact, with an empty input that any endpoint serving the route rejects without running a model. 2xx, 400 or 422 mean the route exists; an unsupported-endpoint answer (ruling 8) means it does not. Only those answers are cached; anything else is CompactionUnknown and is asked again next time. An endpoint in fallback is unsupported without a request.

func (*ResponsesBackend) SendMessageStream 

func (b *ResponsesBackend) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends messages and streams text chunks; the final chunk carries the tool calls, usage and blocks.

func (*ResponsesBackend) SendMessageStreamEvents 

func (b *ResponsesBackend) SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamEventCallback) error

SendMessageStreamEvents sends messages and streams granular events.

func (*ResponsesBackend) SendMessageSync 

func (b *ResponsesBackend) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync sends messages and returns the whole reply.

func (*ResponsesBackend) SetSystemPrompt 

func (b *ResponsesBackend) SetSystemPrompt(prompt string)

func (*ResponsesBackend) SetThinkingConfig 

func (b *ResponsesBackend) SetThinkingConfig(tc ThinkingConfig)

type StreamCallback 

type StreamCallback func(chunk StreamChunk)

StreamCallback is called for each chunk during streaming.

type StreamChunk 

type StreamChunk struct {
	Content      string
	IsFirst      bool
	IsFinal      bool
	FinishReason string
	ToolCalls    []ToolCallResult
	Usage        *TokenUsage // Only populated on final chunk with stream_options

	ProviderBlocks *tui.ProviderBlocks // only on the final chunk (2.0 F3)
	BlocksRejected bool                // only on the final chunk: the provider refused the replayed blocks (2.0 F3)
}

type StreamEvent 

type StreamEvent struct {
	// Type identifies which kind of event this is.
	Type StreamEventType

	// ContentDelta contains the text delta (only for EventContentDelta).
	ContentDelta string

	// ThinkingDelta contains reasoning text (only for EventThinkingDelta).
	ThinkingDelta string

	// ToolUseID is the unique identifier for the tool call
	// (set for EventToolUseStart, EventToolUseInputDelta, EventToolUseDone).
	ToolUseID string

	// ToolName is the function name being called
	// (set for EventToolUseStart, EventToolUseDone).
	ToolName string

	// InputDelta contains partial argument JSON
	// (only for EventToolUseInputDelta).
	InputDelta string

	// CompleteInput contains the fully accumulated argument JSON
	// (only for EventToolUseDone).
	CompleteInput string

	// ThoughtSignature carries Gemini 3.x's opaque per-call token
	// (set for EventToolUseStart and EventToolUseDone, empty for every other
	// provider). It has to travel on the event because the streaming path
	// rebuilds ToolCallResult from events alone — anything left on the
	// backend's own struct is discarded at this boundary, and Gemini then
	// rejects the next turn with "Function call is missing a
	// thought_signature".
	ThoughtSignature []byte

	// Usage contains token usage statistics (only for EventMessageDone).
	Usage *TokenUsage

	// FinishReason indicates why the response ended (only for EventMessageDone).
	FinishReason string

	// ProviderBlocks is the reply as the provider sent it, in order (only
	// for EventMessageDone; nil from backends that keep none). The loop
	// attaches it to the assistant message it records (2.0 F3).
	ProviderBlocks *tui.ProviderBlocks

	// BlocksRejected reports that the provider refused the blocks replayed
	// in this request and the backend sent the neutral history instead
	// (only for EventMessageDone). The loop then strips every message's
	// blocks so they are not sent again (2.0 F3).
	BlocksRejected bool
}

StreamEvent represents a single granular event during LLM streaming. Unlike StreamChunk which batches tool calls into the final chunk, StreamEvent delivers tool call information incrementally as it arrives.

type StreamEventCallback 

type StreamEventCallback func(event StreamEvent)

StreamEventCallback is called for each streaming event.

type StreamEventType 

type StreamEventType int

StreamEventType identifies the kind of streaming event. 

const (
	// EventContentDelta is emitted when the LLM produces a text content delta.
	EventContentDelta StreamEventType = iota

	// EventToolUseStart is emitted when a new tool_use block begins streaming.
	// Contains the tool call ID and tool name.
	EventToolUseStart

	// EventToolUseInputDelta is emitted as argument JSON accumulates for a tool call.
	// Contains partial JSON input for the tool call identified by ToolUseID.
	EventToolUseInputDelta

	// EventToolUseDone is emitted when a tool call's arguments are fully received.
	// Contains the complete input JSON, ready for execution.
	EventToolUseDone

	// EventMessageDone is emitted when the entire LLM response is complete.
	// Contains usage statistics and finish reason.
	EventMessageDone

	// EventThinkingDelta is emitted as the model's reasoning streams, for
	// a "thinking" indicator only: it is never reply text and never joins
	// the history. The OpenAI-compatible chat-completions backend emits it
	// (reasoning / reasoning_content deltas and inlined <think> blocks, L4),
	// and Google emits it for Gemini thought parts; Anthropic and Responses
	// thinking are kept as provider blocks instead.
	EventThinkingDelta
)

func (StreamEventType) String 

func (t StreamEventType) String() string

String returns a human-readable name for the event type.

type ThinkingConfig 

type ThinkingConfig struct {
	Enabled      bool
	BudgetTokens int    // 0 = provider default
	Level        string // "off", "low", "medium", "high", "max"
}

ThinkingConfig controls extended thinking / reasoning effort for LLM backends. When Enabled is true, compatible backends inject the appropriate parameters (e.g. reasoning_effort for OpenAI o-series, thinkingBudget for Gemini).

func (ThinkingConfig) LevelToBudget 

func (tc ThinkingConfig) LevelToBudget() int

LevelToBudget converts a human-friendly level string to a token budget. If Level is empty or unrecognised, BudgetTokens is returned as-is (0 = provider default).

type TokenUsage 

type TokenUsage struct {
	PromptTokens     int
	CompletionTokens int
	TotalTokens      int
	// CacheReadTokens and CacheWriteTokens are the parts of PromptTokens
	// served from or written to the prompt cache (Anthropic; OpenAI, xAI
	// and Google report CacheReadTokens only). CacheWrite1hTokens is the
	// part of CacheWriteTokens written with a 1-hour lifetime (Anthropic),
	// which is priced higher than a 5-minute write (#312).
	CacheReadTokens    int
	CacheWriteTokens   int
	CacheWrite1hTokens int
	// Estimated: celeste counted these itself, because the provider sent
	// no usage (a stream a steering rule cut short, 2.0 W3).
	Estimated bool
}

StreamChunk represents a streaming chunk. TokenUsage holds token usage information from API response

func (*TokenUsage) CostUsage 

func (u *TokenUsage) CostUsage() costs.Usage

CostUsage is u as costs.CostOf prices it: the prompt with its cache reads and writes (#312).

type ToolCallResult 

type ToolCallResult struct {
	ID        string
	Name      string
	Arguments string
	// ArgsError is non-empty when the streamed Arguments could not be assembled
	// into valid JSON (e.g. a dropped stream delta). Empty means no detected
	// corruption — consumers still validate required fields themselves.
	ArgsError string
	// ThoughtSignature carries Gemini 3.x's opaque per-call token through to the
	// assistant message so the next turn can echo it back. Empty elsewhere.
	ThoughtSignature []byte
}

ToolCallResult holds a tool call from the LLM.

type ToolUseAccumulator 

type ToolUseAccumulator struct {
	// contains filtered or unexported fields
}

ToolUseAccumulator tracks in-progress tool uses during streaming, collecting input deltas and producing completed ToolCallResult values. It is safe for use from a single goroutine (the streaming callback goroutine).

func NewToolUseAccumulator 

func NewToolUseAccumulator() *ToolUseAccumulator

NewToolUseAccumulator creates a new accumulator for tracking tool uses.

func (*ToolUseAccumulator) CompletedCalls 

func (a *ToolUseAccumulator) CompletedCalls() []ToolCallResult

CompletedCalls returns all tool calls that have finished receiving input, in the order they were started.

func (*ToolUseAccumulator) HandleEvent 

func (a *ToolUseAccumulator) HandleEvent(event StreamEvent)

HandleEvent processes a StreamEvent and updates internal state. For EventToolUseStart: creates a new pending tool use. For EventToolUseInputDelta: appends to the pending tool's input buffer. For EventToolUseDone: moves the tool use to the completed list. Other event types are ignored.

type XAIBackend 

type XAIBackend struct {
	// contains filtered or unexported fields
}

XAIBackend implements LLMBackend using xAI's native API. This backend supports xAI-specific features like Collections (RAG).

func NewXAIBackend 

func NewXAIBackend(config *Config, registry *tools.Registry) (*XAIBackend, error)

NewXAIBackend creates a new xAI backend with Collections support.

func (*XAIBackend) Close 

func (b *XAIBackend) Close() error

Close cleans up resources (implements LLMBackend interface)

func (*XAIBackend) GetSkills 

func (b *XAIBackend) GetSkills() []tui.SkillDefinition

GetSkills returns the chat-mode tools from the registry.

func (*XAIBackend) SendMessageStream 

func (b *XAIBackend) SendMessageStream(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamCallback) error

SendMessageStream sends a message with streaming callback.

func (*XAIBackend) SendMessageStreamEvents 

func (b *XAIBackend) SendMessageStreamEvents(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition, callback StreamEventCallback) error

SendMessageStreamEvents sends a message with granular streaming events.

func (*XAIBackend) SendMessageSync 

func (b *XAIBackend) SendMessageSync(ctx context.Context, messages []tui.ChatMessage, tools []tui.SkillDefinition) (*ChatCompletionResult, error)

SendMessageSync collects the xAI stream into a single synchronous result.

func (*XAIBackend) SetSystemPrompt 

func (b *XAIBackend) SetSystemPrompt(prompt string)

SetSystemPrompt sets the system prompt (Celeste persona).

func (*XAIBackend) SetThinkingConfig 

func (b *XAIBackend) SetThinkingConfig(config ThinkingConfig)

SetThinkingConfig configures extended thinking / reasoning effort for Grok models. xAI supports reasoning_effort in the request body.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.