Documentation
¶
Overview ¶
Package context provides context management for conversations.
The context engine handles:
- Sliding window to keep conversations within token limits
- History compaction to summarize older messages
- Token counting for context budget management
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrCompactionFailed = errors.New("compaction failed")
ErrCompactionFailed indicates that context summarization failed. This is a sentinel error that can be checked with errors.Is().
Functions ¶
This section is empty.
Types ¶
type CompactionError ¶ added in v0.16.0
type CompactionError struct {
// Reason describes why compaction failed.
Reason string
// Cause is the underlying error, if any.
Cause error
}
CompactionError wraps a compaction failure with additional context.
func NewCompactionError ¶ added in v0.16.0
func NewCompactionError(reason string, cause error) *CompactionError
NewCompactionError creates a CompactionError with the given reason and cause.
func (*CompactionError) Error ¶ added in v0.16.0
func (e *CompactionError) Error() string
Error implements the error interface.
func (*CompactionError) Is ¶ added in v0.16.0
func (e *CompactionError) Is(target error) bool
Is reports whether this error matches the target.
func (*CompactionError) Unwrap ¶ added in v0.16.0
func (e *CompactionError) Unwrap() error
Unwrap returns the underlying cause for errors.Is/As compatibility.
type Config ¶
type Config struct {
// MaxMessages is the maximum number of messages to include.
// Zero means no limit.
MaxMessages int
// MaxTokens is the maximum token budget for context.
// Zero means no limit.
MaxTokens int
// ReserveTokens is the number of tokens to reserve for the response.
// This is subtracted from MaxTokens when calculating available context.
ReserveTokens int
// CompactionEnabled enables automatic history compaction.
CompactionEnabled bool
// CompactionThreshold is the message count that triggers compaction.
// When messages exceed this, older messages are summarized.
CompactionThreshold int
// TokenCounter estimates token count for messages.
// If nil, a simple character-based estimator is used.
TokenCounter TokenCounter
// Summarizer condenses older messages when compaction triggers
// (RMI-OMNIAGENT-027). Required for CompactionEnabled to have any
// effect beyond a no-op — see EnableCompaction.
Summarizer SummarizeFunc
}
Config configures the context engine.
type Engine ¶
type Engine struct {
// contains filtered or unexported fields
}
Engine manages conversation context for LLM requests.
func (*Engine) Apply ¶
func (e *Engine) Apply(ctx context.Context, messages []provider.Message) ([]provider.Message, error)
Apply applies context management to a list of messages. It returns a potentially reduced set of messages that fits within limits, and any error encountered compacting them (RMI-OMNIAGENT-027). The returned messages are always usable even when err is non-nil — compaction failures fall back to plain windowing, never break the caller's turn.
func (*Engine) AvailableTokens ¶
AvailableTokens returns the remaining token budget after accounting for messages.
func (*Engine) EnableCompaction ¶ added in v0.19.0
func (e *Engine) EnableCompaction(threshold int, fn SummarizeFunc)
EnableCompaction turns on LLM-based conversation summarization (RMI-OMNIAGENT-027): once the message count exceeds threshold, older messages are condensed via fn instead of being dropped outright.
type MessagePair ¶
MessagePair represents a user-assistant exchange.
func ExtractPairs ¶
func ExtractPairs(messages []provider.Message) []MessagePair
ExtractPairs extracts user-assistant message pairs from a conversation.
type ModelTokenCounter ¶
type ModelTokenCounter struct {
// Model is the model name for token counting.
Model string
// Fallback is used if model-specific counting is unavailable.
Fallback TokenCounter
// contains filtered or unexported fields
}
ModelTokenCounter uses model-specific token counting. This provides more accurate counts for specific models.
func NewModelTokenCounter ¶ added in v0.16.0
func NewModelTokenCounter(model string) *ModelTokenCounter
NewModelTokenCounter creates a new model-specific token counter.
func (*ModelTokenCounter) Count ¶
func (c *ModelTokenCounter) Count(msg provider.Message) int
Count estimates tokens for a message using model-specific logic.
func (*ModelTokenCounter) CountText ¶
func (c *ModelTokenCounter) CountText(text string) int
CountText estimates tokens for text using model-specific logic.
type SimpleTokenCounter ¶
type SimpleTokenCounter struct {
// CharsPerToken is the average characters per token.
// Default is 4 for English text.
CharsPerToken int
}
SimpleTokenCounter provides a rough token estimate based on character count. It uses a simple heuristic: ~4 characters per token for English text. This is suitable for rough estimates but not precise billing calculations.
func (*SimpleTokenCounter) Count ¶
func (c *SimpleTokenCounter) Count(msg provider.Message) int
Count estimates tokens for a message.
func (*SimpleTokenCounter) CountText ¶
func (c *SimpleTokenCounter) CountText(text string) int
CountText estimates tokens for text.
type SummarizeFunc ¶ added in v0.19.0
SummarizeFunc condenses a batch of messages into a single summary string, used by WindowStrategySummarize (RMI-OMNIAGENT-027). Injected rather than called directly by this package so context stays free of any LLM-client dependency — agent.Agent supplies the real implementation, reusing its own client.
type TokenBudget ¶
type TokenBudget struct {
// Total is the total token budget.
Total int
// Reserved is tokens reserved for response.
Reserved int
// Used is tokens already used.
Used int
}
TokenBudget tracks token usage against a budget.
func (*TokenBudget) Available ¶
func (b *TokenBudget) Available() int
Available returns the available tokens for context.
func (*TokenBudget) Consume ¶
func (b *TokenBudget) Consume(tokens int)
Consume adds to the used token count.
func (*TokenBudget) OverBudget ¶
func (b *TokenBudget) OverBudget() bool
OverBudget returns true if usage exceeds the budget.
type TokenCounter ¶
type TokenCounter interface {
// Count returns the estimated token count for a message.
Count(msg provider.Message) int
// CountText returns the estimated token count for text.
CountText(text string) int
}
TokenCounter estimates the token count for messages.
type Window ¶
type Window struct {
// contains filtered or unexported fields
}
Window manages a sliding window of messages.
func NewWindow ¶
func NewWindow(config WindowConfig) *Window
NewWindow creates a new message window.
func (*Window) Apply ¶
func (w *Window) Apply(ctx context.Context, messages []provider.Message) ([]provider.Message, error)
Apply applies the windowing strategy to messages. Returns the windowed messages and any error encountered. For WindowStrategySummarize, a CompactionError is returned if summarization fails.
type WindowConfig ¶
type WindowConfig struct {
// Strategy is the windowing strategy.
Strategy WindowStrategy
// MaxMessages is the maximum messages to keep.
MaxMessages int
// MaxTokens is the maximum tokens to keep.
MaxTokens int
// TokenCounter for token estimation.
TokenCounter TokenCounter
// Summarizer is called for WindowStrategySummarize. If nil, that
// strategy falls back to WindowStrategyRecent and returns a
// CompactionError explaining why.
Summarizer SummarizeFunc
}
WindowConfig configures a window.
type WindowStrategy ¶
type WindowStrategy int
WindowStrategy defines how messages are windowed.
const ( // WindowStrategyRecent keeps only the most recent messages. WindowStrategyRecent WindowStrategy = iota // WindowStrategySummarize summarizes older messages before windowing. WindowStrategySummarize // WindowStrategyImportant keeps important messages (user questions, key responses). WindowStrategyImportant )