Documentation
¶
Overview ¶
Package context provides context building for LLM prompts.
Package context provides context building for LLM prompts. It orchestrates short-term memory, long-term memory, and retrieval results into an optimized context window for LLM inference.
Package context provides context building for LLM prompts.
Package context provides context building for LLM prompts.
Package context provides context building for LLM prompts.
Package context provides context building for LLM prompts.
Index ¶
- Constants
- func EstimateTokens(content string) int
- func FormatConversation(messages []*Message) string
- func FormatEpisodes(episodes []*EpisodicMemory) string
- func FormatPreferences(prefs *UserPreferences) string
- type BudgetAllocator
- type CacheProvider
- type Config
- type ContextBuilder
- type ContextPriority
- type ContextRequest
- type ContextResult
- type ContextSegment
- type ContextStats
- type EpisodicMemory
- type EpisodicProvider
- type LongTermContext
- type LongTermExtractor
- type Message
- type MessageProvider
- type PreferenceProvider
- type PriorityRanker
- type RetrievalItem
- type Service
- func (s *Service) Build(ctx context.Context, req *ContextRequest) (*ContextResult, error)
- func (s *Service) GetStats() *ContextStats
- func (s *Service) WithCache(c CacheProvider) *Service
- func (s *Service) WithEpisodicProvider(p EpisodicProvider) *Service
- func (s *Service) WithMessageProvider(p MessageProvider) *Service
- func (s *Service) WithPreferenceProvider(p PreferenceProvider) *Service
- type ShortTermExtractor
- type TokenBreakdown
- type TokenBudget
- type UserPreferences
Constants ¶
const ( DefaultMaxTokens = 4096 DefaultSystemPrompt = 500 DefaultUserPrefsRatio = 0.10 DefaultRetrievalRatio = 0.35 MinSegmentTokens = 100 )
Default token budget values.
Variables ¶
This section is empty.
Functions ¶
func EstimateTokens ¶
EstimateTokens estimates the token count for a string. Uses heuristic: Chinese chars count as ~2 tokens, ASCII as ~0.3 tokens per char.
func FormatConversation ¶
FormatConversation formats messages into conversation context.
func FormatEpisodes ¶
func FormatEpisodes(episodes []*EpisodicMemory) string
FormatEpisodes formats episodic memories into context.
func FormatPreferences ¶
func FormatPreferences(prefs *UserPreferences) string
FormatPreferences formats user preferences into context.
Types ¶
type BudgetAllocator ¶
type BudgetAllocator struct {
// contains filtered or unexported fields
}
BudgetAllocator allocates token budgets.
func NewBudgetAllocator ¶
func NewBudgetAllocator() *BudgetAllocator
NewBudgetAllocator creates a new budget allocator with defaults.
func (*BudgetAllocator) Allocate ¶
func (a *BudgetAllocator) Allocate(total int, hasRetrieval bool) *TokenBudget
Allocate allocates token budget based on total and whether retrieval is needed.
type CacheProvider ¶
type CacheProvider interface {
Get(ctx context.Context, key string) ([]byte, bool)
Set(ctx context.Context, key string, value []byte, ttl time.Duration) error
}
CacheProvider provides caching functionality.
type Config ¶
type Config struct {
MaxTurns int // Max conversation turns (default: 10)
MaxEpisodes int // Max episodic memories (default: 3)
MaxTokens int // Default max tokens (default: 4096)
CacheTTL time.Duration // Cache TTL (default: 5 minutes)
}
Config configures the context builder service.
type ContextBuilder ¶
type ContextBuilder interface {
// Build constructs the full context from various sources.
Build(ctx context.Context, req *ContextRequest) (*ContextResult, error)
// GetStats returns context building statistics.
GetStats() *ContextStats
}
ContextBuilder builds optimized context for LLM inference.
type ContextPriority ¶
type ContextPriority int
ContextPriority represents the priority level of a context segment.
const ( PrioritySystem ContextPriority = 100 // System prompt - highest PriorityUserQuery ContextPriority = 90 // Current user query PriorityRecentTurns ContextPriority = 80 // Most recent 3 turns PriorityRetrieval ContextPriority = 70 // RAG retrieval results PriorityEpisodic ContextPriority = 60 // Episodic memory PriorityPreferences ContextPriority = 50 // User preferences PriorityOlderTurns ContextPriority = 40 // Older conversation turns )
type ContextRequest ¶
type ContextRequest struct {
SessionID string
CurrentQuery string
AgentType string
RetrievalResults []*RetrievalItem
MaxTokens int
UserID int32
}
ContextRequest contains parameters for context building.
type ContextResult ¶
type ContextResult struct {
TokenBreakdown *TokenBreakdown
SystemPrompt string
ConversationContext string
RetrievalContext string
UserPreferences string
TotalTokens int
BuildTime time.Duration
}
ContextResult contains the built context.
type ContextSegment ¶
type ContextSegment struct {
Content string
Source string
Priority ContextPriority
TokenCost int
}
ContextSegment represents a piece of context with priority.
func PrioritizeAndTruncate ¶
func PrioritizeAndTruncate(segments []*ContextSegment, budget int) []*ContextSegment
PrioritizeAndTruncate is a convenience function.
type ContextStats ¶
type ContextStats struct {
TotalBuilds int64
AverageTokens float64
CacheHits int64
AverageBuildTime time.Duration
}
ContextStats tracks context building metrics.
type EpisodicMemory ¶
type EpisodicMemory struct {
Timestamp time.Time
Summary string
AgentType string
Outcome string
ID int64
}
EpisodicMemory represents a stored episode.
type EpisodicProvider ¶
type EpisodicProvider interface {
SearchEpisodes(ctx context.Context, userID int32, query string, limit int) ([]*EpisodicMemory, error)
}
EpisodicProvider provides episodic memory search.
type LongTermContext ¶
type LongTermContext struct {
Preferences *UserPreferences
Episodes []*EpisodicMemory
}
LongTermContext contains extracted long-term context.
type LongTermExtractor ¶
type LongTermExtractor struct {
// contains filtered or unexported fields
}
LongTermExtractor extracts episodic memories and user preferences.
func NewLongTermExtractor ¶
func NewLongTermExtractor(maxEpisodes int) *LongTermExtractor
NewLongTermExtractor creates a new long-term memory extractor.
func (*LongTermExtractor) Extract ¶
func (e *LongTermExtractor) Extract( ctx context.Context, episodicProvider EpisodicProvider, prefProvider PreferenceProvider, userID int32, query string, ) (*LongTermContext, error)
Extract extracts long-term context for the user.
type Message ¶
Message represents a conversation message.
func SplitByRecency ¶
SplitByRecency splits messages into recent (high priority) and older (lower priority).
type MessageProvider ¶
type MessageProvider interface {
GetRecentMessages(ctx context.Context, sessionID string, limit int) ([]*Message, error)
}
MessageProvider provides recent messages for a session.
type PreferenceProvider ¶
type PreferenceProvider interface {
GetPreferences(ctx context.Context, userID int32) (*UserPreferences, error)
}
PreferenceProvider provides user preferences.
type PriorityRanker ¶
type PriorityRanker struct{}
PriorityRanker ranks and truncates context segments by priority.
func NewPriorityRanker ¶
func NewPriorityRanker() *PriorityRanker
NewPriorityRanker creates a new priority ranker.
func (*PriorityRanker) RankAndTruncate ¶
func (r *PriorityRanker) RankAndTruncate(segments []*ContextSegment, budget int) []*ContextSegment
RankAndTruncate sorts segments by priority and truncates to fit budget.
type RetrievalItem ¶
RetrievalItem represents a single retrieval result.
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
Service implements ContextBuilder with caching support.
func NewService ¶
NewService creates a new context builder service.
func (*Service) Build ¶
func (s *Service) Build(ctx context.Context, req *ContextRequest) (*ContextResult, error)
Build constructs the context for LLM inference.
func (*Service) GetStats ¶
func (s *Service) GetStats() *ContextStats
GetStats returns context building statistics.
func (*Service) WithCache ¶
func (s *Service) WithCache(c CacheProvider) *Service
WithCache sets the cache provider.
func (*Service) WithEpisodicProvider ¶
func (s *Service) WithEpisodicProvider(p EpisodicProvider) *Service
WithEpisodicProvider sets the episodic memory provider.
func (*Service) WithMessageProvider ¶
func (s *Service) WithMessageProvider(p MessageProvider) *Service
WithMessageProvider sets the message provider.
func (*Service) WithPreferenceProvider ¶
func (s *Service) WithPreferenceProvider(p PreferenceProvider) *Service
WithPreferenceProvider sets the preference provider.
type ShortTermExtractor ¶
type ShortTermExtractor struct {
// contains filtered or unexported fields
}
ShortTermExtractor extracts recent conversation turns.
func NewShortTermExtractor ¶
func NewShortTermExtractor(maxTurns int) *ShortTermExtractor
NewShortTermExtractor creates a new short-term memory extractor.
func (*ShortTermExtractor) Extract ¶
func (e *ShortTermExtractor) Extract(ctx context.Context, provider MessageProvider, sessionID string) ([]*Message, error)
Extract extracts recent messages from the session.
type TokenBreakdown ¶
type TokenBreakdown struct {
SystemPrompt int
ShortTermMemory int
LongTermMemory int
Retrieval int
UserPrefs int
}
TokenBreakdown shows how tokens are distributed.
type TokenBudget ¶
type TokenBudget struct {
Total int
SystemPrompt int
ShortTermMemory int
LongTermMemory int
Retrieval int
UserPrefs int
}
TokenBudget represents the token allocation plan.
func AllocateBudget ¶
func AllocateBudget(total int, hasRetrieval bool) *TokenBudget
AllocateBudget is a convenience function.
type UserPreferences ¶
type UserPreferences struct {
Timezone string
CommunicationStyle string
PreferredTimes []string
DefaultDuration int
}
UserPreferences represents user preferences.
func DefaultUserPreferences ¶
func DefaultUserPreferences() *UserPreferences
DefaultUserPreferences returns default preferences.