Documentation
¶
Overview ¶
Package context provides context management for conversations.
The context engine handles:
- Sliding window to keep conversations within token limits
- History compaction to summarize older messages
- Token counting for context budget management
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
// MaxMessages is the maximum number of messages to include.
// Zero means no limit.
MaxMessages int
// MaxTokens is the maximum token budget for context.
// Zero means no limit.
MaxTokens int
// ReserveTokens is the number of tokens to reserve for the response.
// This is subtracted from MaxTokens when calculating available context.
ReserveTokens int
// CompactionEnabled enables automatic history compaction.
CompactionEnabled bool
// CompactionThreshold is the message count that triggers compaction.
// When messages exceed this, older messages are summarized.
CompactionThreshold int
// TokenCounter estimates token count for messages.
// If nil, a simple character-based estimator is used.
TokenCounter TokenCounter
}
Config configures the context engine.
type Engine ¶
type Engine struct {
// contains filtered or unexported fields
}
Engine manages conversation context for LLM requests.
func (*Engine) Apply ¶
Apply applies context management to a list of messages. It returns a potentially reduced set of messages that fits within limits.
func (*Engine) AvailableTokens ¶
AvailableTokens returns the remaining token budget after accounting for messages.
type MessagePair ¶
MessagePair represents a user-assistant exchange.
func ExtractPairs ¶
func ExtractPairs(messages []provider.Message) []MessagePair
ExtractPairs extracts user-assistant message pairs from a conversation.
type ModelTokenCounter ¶
type ModelTokenCounter struct {
// Model is the model name for token counting.
Model string
// Fallback is used if model-specific counting is unavailable.
Fallback TokenCounter
}
ModelTokenCounter uses model-specific token counting. This provides more accurate counts for specific models.
func (*ModelTokenCounter) Count ¶
func (c *ModelTokenCounter) Count(msg provider.Message) int
Count estimates tokens for a message using model-specific logic.
func (*ModelTokenCounter) CountText ¶
func (c *ModelTokenCounter) CountText(text string) int
CountText estimates tokens for text using model-specific logic.
type SimpleTokenCounter ¶
type SimpleTokenCounter struct {
// CharsPerToken is the average characters per token.
// Default is 4 for English text.
CharsPerToken int
}
SimpleTokenCounter provides a rough token estimate based on character count. It uses a simple heuristic: ~4 characters per token for English text. This is suitable for rough estimates but not precise billing calculations.
func (*SimpleTokenCounter) Count ¶
func (c *SimpleTokenCounter) Count(msg provider.Message) int
Count estimates tokens for a message.
func (*SimpleTokenCounter) CountText ¶
func (c *SimpleTokenCounter) CountText(text string) int
CountText estimates tokens for text.
type TokenBudget ¶
type TokenBudget struct {
// Total is the total token budget.
Total int
// Reserved is tokens reserved for response.
Reserved int
// Used is tokens already used.
Used int
}
TokenBudget tracks token usage against a budget.
func (*TokenBudget) Available ¶
func (b *TokenBudget) Available() int
Available returns the available tokens for context.
func (*TokenBudget) Consume ¶
func (b *TokenBudget) Consume(tokens int)
Consume adds to the used token count.
func (*TokenBudget) OverBudget ¶
func (b *TokenBudget) OverBudget() bool
OverBudget returns true if usage exceeds the budget.
type TokenCounter ¶
type TokenCounter interface {
// Count returns the estimated token count for a message.
Count(msg provider.Message) int
// CountText returns the estimated token count for text.
CountText(text string) int
}
TokenCounter estimates the token count for messages.
type Window ¶
type Window struct {
// contains filtered or unexported fields
}
Window manages a sliding window of messages.
func NewWindow ¶
func NewWindow(config WindowConfig) *Window
NewWindow creates a new message window.
type WindowConfig ¶
type WindowConfig struct {
// Strategy is the windowing strategy.
Strategy WindowStrategy
// MaxMessages is the maximum messages to keep.
MaxMessages int
// MaxTokens is the maximum tokens to keep.
MaxTokens int
// TokenCounter for token estimation.
TokenCounter TokenCounter
}
WindowConfig configures a window.
type WindowStrategy ¶
type WindowStrategy int
WindowStrategy defines how messages are windowed.
const ( // WindowStrategyRecent keeps only the most recent messages. WindowStrategyRecent WindowStrategy = iota // WindowStrategySummarize summarizes older messages before windowing. WindowStrategySummarize // WindowStrategyImportant keeps important messages (user questions, key responses). WindowStrategyImportant )
Source Files