Documentation
¶
Overview ¶
Package context provides context building for LLM prompts.
Package context provides budget profiles for dynamic token allocation. Issue #93: 动态 Token 预算分配 - 按意图类型自适应调整
Package context provides context building for LLM prompts. It orchestrates short-term memory, long-term memory, and retrieval results into an optimized context window for LLM inference.
Package context provides context building for LLM prompts.
Package context provides incremental context building for multi-turn conversations. This module implements Issue #94: Incremental context updates with 70% reduction in round 2 and 82% reduction in round 5.
Package context provides context building for LLM prompts.
Package context provides context building for LLM prompts.
Package context provides context building for LLM prompts.
Index ¶
- Constants
- Variables
- func EstimateTokens(content string) int
- func FormatConversation(messages []*Message) string
- func FormatEpisodes(episodes []*EpisodicMemory) string
- func FormatPreferences(prefs *UserPreferences) string
- type BudgetAllocator
- func (a *BudgetAllocator) Allocate(total int, hasRetrieval bool) *TokenBudget
- func (a *BudgetAllocator) AllocateForAgent(total int, hasRetrieval bool, agentType string) *TokenBudget
- func (a *BudgetAllocator) LoadProfileFromEnv()
- func (a *BudgetAllocator) WithIntentResolver(resolver *IntentResolver) *BudgetAllocator
- func (a *BudgetAllocator) WithProfileRegistry(registry *ProfileRegistry) *BudgetAllocator
- type BudgetProfile
- type CacheProvider
- type Config
- type ContextBuilder
- type ContextPriority
- type ContextRequest
- type ContextRequestWithMessages
- type ContextResult
- type ContextSegment
- type ContextSnapshot
- type ContextStats
- type Delta
- type DeltaBuilder
- func (d *DeltaBuilder) Clear()
- func (d *DeltaBuilder) ComputeDelta(sessionID string, req *ContextRequest, prevSnapshot *ContextSnapshot) *Delta
- func (d *DeltaBuilder) GetSnapshot(sessionID string) *ContextSnapshot
- func (d *DeltaBuilder) GetStats() *DeltaStats
- func (d *DeltaBuilder) SaveSnapshot(sessionID string, snapshot *ContextSnapshot)
- func (d *DeltaBuilder) SelectStrategy(req *ContextRequest, prevSnapshot *ContextSnapshot) UpdateStrategy
- type DeltaStats
- type EpisodicMemory
- type EpisodicProvider
- type IncrementalBuilder
- type IntentResolver
- type LongTermContext
- type LongTermExtractor
- type Message
- type MessageProvider
- type PreferenceProvider
- type PriorityRanker
- type ProfileRegistry
- type RetrievalItem
- type Service
- func (s *Service) Build(ctx context.Context, req *ContextRequest) (*ContextResult, error)
- func (s *Service) GetStats() *ContextStats
- func (s *Service) WithCache(c CacheProvider) *Service
- func (s *Service) WithEpisodicProvider(p EpisodicProvider) *Service
- func (s *Service) WithMessageProvider(p MessageProvider) *Service
- func (s *Service) WithPreferenceProvider(p PreferenceProvider) *Service
- type ShortTermExtractor
- type TokenBreakdown
- type TokenBudget
- type UpdateStrategy
- type UserPreferences
Constants ¶
const ( DefaultMaxTokens = 4096 DefaultSystemPrompt = 500 DefaultUserPrefsRatio = 0.10 DefaultRetrievalRatio = 0.45 // Updated per best practices MinSegmentTokens = 100 )
Default token budget values.
Token allocation strategy (with retrieval): - System Prompt: 500 tokens (fixed) - User Preferences: 10% - Short-term Memory: 40% - Long-term Memory: 15% - Retrieval Results: 45%
Reference: docs/agent-engineering/AGENT_PROMPT_BEST_PRACTICES.md
Variables ¶
var DefaultBudgetProfiles = map[string]*BudgetProfile{
"memo_search": {
Name: "memo_search",
Description: "Memo search intent - prioritize retrieval results",
ShortTermRatio: 0.30,
LongTermRatio: 0.10,
RetrievalRatio: 0.60,
UserPrefsRatio: 0.10,
},
"schedule_create": {
Name: "schedule_create",
Description: "Schedule creation - moderate conversation, low retrieval",
ShortTermRatio: 0.55,
LongTermRatio: 0.25,
RetrievalRatio: 0.20,
UserPrefsRatio: 0.10,
},
"schedule_query": {
Name: "schedule_query",
Description: "Schedule query - moderate conversation, low retrieval",
ShortTermRatio: 0.55,
LongTermRatio: 0.25,
RetrievalRatio: 0.20,
UserPrefsRatio: 0.10,
},
"amazing": {
Name: "amazing",
Description: "Amazing parrot - balanced allocation",
ShortTermRatio: 0.40,
LongTermRatio: 0.15,
RetrievalRatio: 0.45,
UserPrefsRatio: 0.10,
},
"geek": {
Name: "geek",
Description: "Geek parrot - no LLM budget (Claude Code CLI manages context)",
ShortTermRatio: 0.00,
LongTermRatio: 0.00,
RetrievalRatio: 0.00,
UserPrefsRatio: 0.00,
},
"evolution": {
Name: "evolution",
Description: "Evolution parrot - no LLM budget (Claude Code CLI manages context)",
ShortTermRatio: 0.00,
LongTermRatio: 0.00,
RetrievalRatio: 0.00,
UserPrefsRatio: 0.00,
},
"default": {
Name: "default",
Description: "Fallback profile - balanced allocation",
ShortTermRatio: 0.40,
LongTermRatio: 0.15,
RetrievalRatio: 0.45,
UserPrefsRatio: 0.10,
},
}
DefaultBudgetProfiles returns the built-in profile registry.
Functions ¶
func EstimateTokens ¶
EstimateTokens estimates the token count for a string. Uses heuristic: Chinese chars count as ~2 tokens, ASCII as ~0.3 tokens per char.
func FormatConversation ¶
FormatConversation formats messages into conversation context.
func FormatEpisodes ¶
func FormatEpisodes(episodes []*EpisodicMemory) string
FormatEpisodes formats episodic memories into context.
func FormatPreferences ¶
func FormatPreferences(prefs *UserPreferences) string
FormatPreferences formats user preferences into context.
Types ¶
type BudgetAllocator ¶
type BudgetAllocator struct {
// contains filtered or unexported fields
}
BudgetAllocator allocates token budgets.
func NewBudgetAllocator ¶
func NewBudgetAllocator() *BudgetAllocator
NewBudgetAllocator creates a new budget allocator with defaults.
func (*BudgetAllocator) Allocate ¶
func (a *BudgetAllocator) Allocate(total int, hasRetrieval bool) *TokenBudget
Allocate allocates token budget based on total and whether retrieval is needed.
func (*BudgetAllocator) AllocateForAgent ¶ added in v0.94.0
func (a *BudgetAllocator) AllocateForAgent(total int, hasRetrieval bool, agentType string) *TokenBudget
AllocateForAgent allocates token budget based on agent type (profile-based). Issue #93: 动态 Token 预算分配 - 按意图类型自适应调整
func (*BudgetAllocator) LoadProfileFromEnv ¶ added in v0.94.0
func (a *BudgetAllocator) LoadProfileFromEnv()
LoadProfileFromEnv loads profile overrides from environment variables.
func (*BudgetAllocator) WithIntentResolver ¶ added in v0.94.0
func (a *BudgetAllocator) WithIntentResolver(resolver *IntentResolver) *BudgetAllocator
WithIntentResolver sets a custom intent resolver.
func (*BudgetAllocator) WithProfileRegistry ¶ added in v0.94.0
func (a *BudgetAllocator) WithProfileRegistry(registry *ProfileRegistry) *BudgetAllocator
WithProfileRegistry sets a custom profile registry.
type BudgetProfile ¶ added in v0.94.0
type BudgetProfile struct {
Name string
Description string
// Token allocation ratios for the remaining budget (after SystemPrompt + UserPrefs)
// Must sum to 1.0
ShortTermRatio float64
LongTermRatio float64
RetrievalRatio float64
// Fixed ratios
UserPrefsRatio float64 // Default: 0.10
}
BudgetProfile defines token allocation ratios for different intents.
type CacheProvider ¶
type CacheProvider interface {
Get(ctx context.Context, key string) ([]byte, bool)
Set(ctx context.Context, key string, value []byte, ttl time.Duration) error
}
CacheProvider provides caching functionality.
type Config ¶
type Config struct {
MaxTurns int // Max conversation turns (default: 10)
MaxEpisodes int // Max episodic memories (default: 3)
MaxTokens int // Default max tokens (default: 4096)
CacheTTL time.Duration // Cache TTL (default: 5 minutes)
}
Config configures the context builder service.
type ContextBuilder ¶
type ContextBuilder interface {
// Build constructs the full context from various sources.
Build(ctx context.Context, req *ContextRequest) (*ContextResult, error)
// GetStats returns context building statistics.
GetStats() *ContextStats
}
ContextBuilder builds optimized context for LLM inference.
type ContextPriority ¶
type ContextPriority int
ContextPriority represents the priority level of a context segment.
const ( PrioritySystem ContextPriority = 100 // System prompt - highest PriorityUserQuery ContextPriority = 90 // Current user query PriorityRecentTurns ContextPriority = 80 // Most recent 3 turns PriorityRetrieval ContextPriority = 70 // RAG retrieval results PriorityEpisodic ContextPriority = 60 // Episodic memory PriorityPreferences ContextPriority = 50 // User preferences PriorityOlderTurns ContextPriority = 40 // Older conversation turns )
type ContextRequest ¶
type ContextRequest struct {
SessionID string
CurrentQuery string
AgentType string
RetrievalResults []*RetrievalItem
MaxTokens int
UserID int32
}
ContextRequest contains parameters for context building.
type ContextRequestWithMessages ¶ added in v0.94.0
type ContextRequestWithMessages struct {
ContextRequest
Messages []*Message
}
ContextRequestWithMessages extends ContextRequest with message history.
type ContextResult ¶
type ContextResult struct {
TokenBreakdown *TokenBreakdown
SystemPrompt string
ConversationContext string
RetrievalContext string
UserPreferences string
TotalTokens int
BuildTime time.Duration
}
ContextResult contains the built context.
type ContextSegment ¶
type ContextSegment struct {
Content string
Source string
Priority ContextPriority
TokenCost int
}
ContextSegment represents a piece of context with priority.
func PrioritizeAndTruncate ¶
func PrioritizeAndTruncate(segments []*ContextSegment, budget int) []*ContextSegment
PrioritizeAndTruncate is a convenience function.
type ContextSnapshot ¶ added in v0.94.0
type ContextSnapshot struct {
// Messages in the conversation.
Messages []*Message
// Retrieval results.
RetrievalResults []*RetrievalItem
// System prompt hash.
SystemPromptHash string
// User preferences hash.
UserPrefsHash string
// Episodic memories hash.
EpisodicHash string
// Timestamp of the snapshot.
Timestamp time.Time
// Token count at time of snapshot.
TokenCount int
// Query that was processed.
Query string
// Agent type used.
AgentType string
}
ContextSnapshot captures a snapshot of the context for comparison.
func CreateSnapshot ¶ added in v0.94.0
func CreateSnapshot(req *ContextRequest, result *ContextResult) *ContextSnapshot
CreateSnapshot creates a snapshot from a context result.
func (*ContextSnapshot) ToJSON ¶ added in v0.94.0
func (s *ContextSnapshot) ToJSON() (string, error)
ToJSON converts a snapshot to JSON for debugging.
type ContextStats ¶
type ContextStats struct {
TotalBuilds int64
AverageTokens float64
CacheHits int64
AverageBuildTime time.Duration
}
ContextStats tracks context building metrics.
type Delta ¶ added in v0.94.0
type Delta struct {
// NewMessages are messages added since last build.
NewMessages []*Message
// ModifiedSections are sections that changed.
ModifiedSections []string
// RemovedSections are sections that were removed.
RemovedSections []string
// NewRetrievalItems are new retrieval results.
NewRetrievalItems []*RetrievalItem
// PreviousHash is the hash of the previous context.
PreviousHash string
// CurrentHash is the hash of the current context.
CurrentHash string
// Strategy used for this delta.
Strategy UpdateStrategy
}
Delta represents the changes between two context builds.
type DeltaBuilder ¶ added in v0.94.0
type DeltaBuilder struct {
// contains filtered or unexported fields
}
DeltaBuilder computes deltas between context builds.
func NewDeltaBuilder ¶ added in v0.94.0
func NewDeltaBuilder() *DeltaBuilder
NewDeltaBuilder creates a new delta builder.
func (*DeltaBuilder) Clear ¶ added in v0.94.0
func (d *DeltaBuilder) Clear()
Clear removes all cached snapshots.
func (*DeltaBuilder) ComputeDelta ¶ added in v0.94.0
func (d *DeltaBuilder) ComputeDelta(sessionID string, req *ContextRequest, prevSnapshot *ContextSnapshot) *Delta
ComputeDelta computes the delta between the current request and the previous snapshot.
func (*DeltaBuilder) GetSnapshot ¶ added in v0.94.0
func (d *DeltaBuilder) GetSnapshot(sessionID string) *ContextSnapshot
GetSnapshot retrieves the snapshot for a session.
func (*DeltaBuilder) GetStats ¶ added in v0.94.0
func (d *DeltaBuilder) GetStats() *DeltaStats
GetStats returns delta building statistics.
func (*DeltaBuilder) SaveSnapshot ¶ added in v0.94.0
func (d *DeltaBuilder) SaveSnapshot(sessionID string, snapshot *ContextSnapshot)
SaveSnapshot saves a snapshot for a session.
func (*DeltaBuilder) SelectStrategy ¶ added in v0.94.0
func (d *DeltaBuilder) SelectStrategy(req *ContextRequest, prevSnapshot *ContextSnapshot) UpdateStrategy
SelectStrategy selects the optimal update strategy based on the request and previous snapshot.
type DeltaStats ¶ added in v0.94.0
type DeltaStats struct {
TotalDeltas int64
DeltaHits int64
FullRebuilds int64
AppendOnlyHits int64
ConversationOnlyHits int64
AverageDeltaMs int64
SavedTokens int64
HitRate float64
}
DeltaStats contains delta building statistics.
type EpisodicMemory ¶
type EpisodicMemory struct {
Timestamp time.Time
Summary string
AgentType string
Outcome string
ID int64
}
EpisodicMemory represents a stored episode.
type EpisodicProvider ¶
type EpisodicProvider interface {
SearchEpisodes(ctx context.Context, userID int32, query string, limit int) ([]*EpisodicMemory, error)
}
EpisodicProvider provides episodic memory search.
type IncrementalBuilder ¶ added in v0.94.0
type IncrementalBuilder struct {
// contains filtered or unexported fields
}
IncrementalBuilder provides incremental context building.
func NewIncrementalBuilder ¶ added in v0.94.0
func NewIncrementalBuilder(base *Service) *IncrementalBuilder
NewIncrementalBuilder creates a new incremental context builder.
func (*IncrementalBuilder) BuildIncremental ¶ added in v0.94.0
func (b *IncrementalBuilder) BuildIncremental(ctx context.Context, req *ContextRequest) (*ContextResult, error)
BuildIncremental builds context incrementally based on the optimal strategy.
func (*IncrementalBuilder) ClearCache ¶ added in v0.94.0
func (b *IncrementalBuilder) ClearCache()
ClearCache clears the incremental builder cache.
func (*IncrementalBuilder) GetDeltaStats ¶ added in v0.94.0
func (b *IncrementalBuilder) GetDeltaStats() *DeltaStats
GetDeltaStats returns delta building statistics.
type IntentResolver ¶ added in v0.94.0
type IntentResolver struct {
}
IntentResolver infers task intent from AgentType.
func NewIntentResolver ¶ added in v0.94.0
func NewIntentResolver() *IntentResolver
NewIntentResolver creates a new intent resolver.
func (*IntentResolver) Resolve ¶ added in v0.94.0
func (r *IntentResolver) Resolve(agentType string) string
Resolve infers intent from AgentType (thread-safe). Returns intent name (e.g., "memo_search", "schedule_create").
type LongTermContext ¶
type LongTermContext struct {
Preferences *UserPreferences
Episodes []*EpisodicMemory
}
LongTermContext contains extracted long-term context.
type LongTermExtractor ¶
type LongTermExtractor struct {
// contains filtered or unexported fields
}
LongTermExtractor extracts episodic memories and user preferences.
func NewLongTermExtractor ¶
func NewLongTermExtractor(maxEpisodes int) *LongTermExtractor
NewLongTermExtractor creates a new long-term memory extractor.
func (*LongTermExtractor) Extract ¶
func (e *LongTermExtractor) Extract( ctx context.Context, episodicProvider EpisodicProvider, prefProvider PreferenceProvider, userID int32, query string, ) (*LongTermContext, error)
Extract extracts long-term context for the user.
type Message ¶
Message represents a conversation message.
func SplitByRecency ¶
SplitByRecency splits messages into recent (high priority) and older (lower priority).
type MessageProvider ¶
type MessageProvider interface {
GetRecentMessages(ctx context.Context, sessionID string, limit int) ([]*Message, error)
}
MessageProvider provides recent messages for a session.
type PreferenceProvider ¶
type PreferenceProvider interface {
GetPreferences(ctx context.Context, userID int32) (*UserPreferences, error)
}
PreferenceProvider provides user preferences.
type PriorityRanker ¶
type PriorityRanker struct{}
PriorityRanker ranks and truncates context segments by priority.
func NewPriorityRanker ¶
func NewPriorityRanker() *PriorityRanker
NewPriorityRanker creates a new priority ranker.
func (*PriorityRanker) RankAndTruncate ¶
func (r *PriorityRanker) RankAndTruncate(segments []*ContextSegment, budget int) []*ContextSegment
RankAndTruncate sorts segments by priority and truncates to fit budget.
type ProfileRegistry ¶ added in v0.94.0
type ProfileRegistry struct {
// contains filtered or unexported fields
}
ProfileRegistry manages budget profiles with override support.
func NewProfileRegistry ¶ added in v0.94.0
func NewProfileRegistry() *ProfileRegistry
NewProfileRegistry creates a new profile registry with defaults.
func (*ProfileRegistry) Get ¶ added in v0.94.0
func (r *ProfileRegistry) Get(name string) (*BudgetProfile, bool)
Get returns a profile by name (thread-safe). Returns the profile and whether it was found directly (not using fallback).
func (*ProfileRegistry) LoadFromEnv ¶ added in v0.94.0
func (r *ProfileRegistry) LoadFromEnv()
LoadFromEnv overrides profiles from environment variables. Format: DIVINESENSE_BUDGET_<PROFILE>_<FIELD>=<value> Example: DIVINESENSE_BUDGET_MEMO_SEARCH_SHORT_TERM=0.35
func (*ProfileRegistry) Set ¶ added in v0.94.0
func (r *ProfileRegistry) Set(name string, profile *BudgetProfile)
Set adds or overrides a profile (thread-safe).
type RetrievalItem ¶
RetrievalItem represents a single retrieval result.
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
Service implements ContextBuilder with caching support.
func NewService ¶
NewService creates a new context builder service.
func (*Service) Build ¶
func (s *Service) Build(ctx context.Context, req *ContextRequest) (*ContextResult, error)
Build constructs the context for LLM inference.
func (*Service) GetStats ¶
func (s *Service) GetStats() *ContextStats
GetStats returns context building statistics.
func (*Service) WithCache ¶
func (s *Service) WithCache(c CacheProvider) *Service
WithCache sets the cache provider.
func (*Service) WithEpisodicProvider ¶
func (s *Service) WithEpisodicProvider(p EpisodicProvider) *Service
WithEpisodicProvider sets the episodic memory provider.
func (*Service) WithMessageProvider ¶
func (s *Service) WithMessageProvider(p MessageProvider) *Service
WithMessageProvider sets the message provider.
func (*Service) WithPreferenceProvider ¶
func (s *Service) WithPreferenceProvider(p PreferenceProvider) *Service
WithPreferenceProvider sets the preference provider.
type ShortTermExtractor ¶
type ShortTermExtractor struct {
// contains filtered or unexported fields
}
ShortTermExtractor extracts recent conversation turns.
func NewShortTermExtractor ¶
func NewShortTermExtractor(maxTurns int) *ShortTermExtractor
NewShortTermExtractor creates a new short-term memory extractor.
func (*ShortTermExtractor) Extract ¶
func (e *ShortTermExtractor) Extract(ctx context.Context, provider MessageProvider, sessionID string) ([]*Message, error)
Extract extracts recent messages from the session.
type TokenBreakdown ¶
type TokenBreakdown struct {
SystemPrompt int
ShortTermMemory int
LongTermMemory int
Retrieval int
UserPrefs int
}
TokenBreakdown shows how tokens are distributed.
type TokenBudget ¶
type TokenBudget struct {
Total int
SystemPrompt int
ShortTermMemory int
LongTermMemory int
Retrieval int
UserPrefs int
}
TokenBudget represents the token allocation plan.
func AllocateBudget ¶
func AllocateBudget(total int, hasRetrieval bool) *TokenBudget
AllocateBudget is a convenience function.
type UpdateStrategy ¶ added in v0.94.0
type UpdateStrategy int
UpdateStrategy defines the strategy for updating context.
const ( // ComputeDelta computes only the changes since last build. ComputeDelta UpdateStrategy = iota // AppendOnly appends new messages without recomputing. AppendOnly // UpdateConversationOnly updates only conversation context. UpdateConversationOnly // FullRebuild performs a full context rebuild. FullRebuild )
func (UpdateStrategy) String ¶ added in v0.94.0
func (s UpdateStrategy) String() string
String returns the string representation of the strategy.
type UserPreferences ¶
type UserPreferences struct {
Timezone string
CommunicationStyle string
PreferredTimes []string
DefaultDuration int
}
UserPreferences represents user preferences.
func DefaultUserPreferences ¶
func DefaultUserPreferences() *UserPreferences
DefaultUserPreferences returns default preferences.