context

package
v0.94.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 10, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package context provides context building for LLM prompts.

Package context provides budget profiles for dynamic token allocation. Issue #93: 动态 Token 预算分配 - 按意图类型自适应调整

Package context provides context building for LLM prompts. It orchestrates short-term memory, long-term memory, and retrieval results into an optimized context window for LLM inference.

Package context provides context building for LLM prompts.

Package context provides incremental context building for multi-turn conversations. This module implements Issue #94: Incremental context updates with 70% reduction in round 2 and 82% reduction in round 5.

Package context provides context building for LLM prompts.

Package context provides context building for LLM prompts.

Package context provides context building for LLM prompts.

Index

Constants

View Source
const (
	DefaultMaxTokens      = 4096
	DefaultSystemPrompt   = 500
	DefaultUserPrefsRatio = 0.10
	DefaultRetrievalRatio = 0.35
	MinSegmentTokens      = 100
)

Default token budget values.

Variables

View Source
var DefaultBudgetProfiles = map[string]*BudgetProfile{
	"memo_search": {
		Name:           "memo_search",
		Description:    "Memo search intent - prioritize retrieval results",
		ShortTermRatio: 0.30,
		LongTermRatio:  0.10,
		RetrievalRatio: 0.60,
		UserPrefsRatio: 0.10,
	},
	"schedule_create": {
		Name:           "schedule_create",
		Description:    "Schedule creation - moderate conversation, low retrieval",
		ShortTermRatio: 0.55,
		LongTermRatio:  0.25,
		RetrievalRatio: 0.20,
		UserPrefsRatio: 0.10,
	},
	"schedule_query": {
		Name:           "schedule_query",
		Description:    "Schedule query - moderate conversation, low retrieval",
		ShortTermRatio: 0.55,
		LongTermRatio:  0.25,
		RetrievalRatio: 0.20,
		UserPrefsRatio: 0.10,
	},
	"amazing": {
		Name:           "amazing",
		Description:    "Amazing parrot - balanced allocation",
		ShortTermRatio: 0.40,
		LongTermRatio:  0.15,
		RetrievalRatio: 0.45,
		UserPrefsRatio: 0.10,
	},
	"geek": {
		Name:           "geek",
		Description:    "Geek parrot - no LLM budget (Claude Code CLI manages context)",
		ShortTermRatio: 0.00,
		LongTermRatio:  0.00,
		RetrievalRatio: 0.00,
		UserPrefsRatio: 0.00,
	},
	"evolution": {
		Name:           "evolution",
		Description:    "Evolution parrot - no LLM budget (Claude Code CLI manages context)",
		ShortTermRatio: 0.00,
		LongTermRatio:  0.00,
		RetrievalRatio: 0.00,
		UserPrefsRatio: 0.00,
	},
	"default": {
		Name:           "default",
		Description:    "Fallback profile - balanced allocation",
		ShortTermRatio: 0.40,
		LongTermRatio:  0.15,
		RetrievalRatio: 0.45,
		UserPrefsRatio: 0.10,
	},
}

DefaultBudgetProfiles returns the built-in profile registry.

Functions

func EstimateTokens

func EstimateTokens(content string) int

EstimateTokens estimates the token count for a string. Uses heuristic: Chinese chars count as ~2 tokens, ASCII as ~0.3 tokens per char.

func FormatConversation

func FormatConversation(messages []*Message) string

FormatConversation formats messages into conversation context.

func FormatEpisodes

func FormatEpisodes(episodes []*EpisodicMemory) string

FormatEpisodes formats episodic memories into context.

func FormatPreferences

func FormatPreferences(prefs *UserPreferences) string

FormatPreferences formats user preferences into context.

Types

type BudgetAllocator

type BudgetAllocator struct {
	// contains filtered or unexported fields
}

BudgetAllocator allocates token budgets.

func NewBudgetAllocator

func NewBudgetAllocator() *BudgetAllocator

NewBudgetAllocator creates a new budget allocator with defaults.

func (*BudgetAllocator) Allocate

func (a *BudgetAllocator) Allocate(total int, hasRetrieval bool) *TokenBudget

Allocate allocates token budget based on total and whether retrieval is needed.

func (*BudgetAllocator) AllocateForAgent added in v0.94.0

func (a *BudgetAllocator) AllocateForAgent(total int, hasRetrieval bool, agentType string) *TokenBudget

AllocateForAgent allocates token budget based on agent type (profile-based). Issue #93: 动态 Token 预算分配 - 按意图类型自适应调整

func (*BudgetAllocator) LoadProfileFromEnv added in v0.94.0

func (a *BudgetAllocator) LoadProfileFromEnv()

LoadProfileFromEnv loads profile overrides from environment variables.

func (*BudgetAllocator) WithIntentResolver added in v0.94.0

func (a *BudgetAllocator) WithIntentResolver(resolver *IntentResolver) *BudgetAllocator

WithIntentResolver sets a custom intent resolver.

func (*BudgetAllocator) WithProfileRegistry added in v0.94.0

func (a *BudgetAllocator) WithProfileRegistry(registry *ProfileRegistry) *BudgetAllocator

WithProfileRegistry sets a custom profile registry.

type BudgetProfile added in v0.94.0

type BudgetProfile struct {
	Name        string
	Description string

	// Token allocation ratios for the remaining budget (after SystemPrompt + UserPrefs)
	// Must sum to 1.0
	ShortTermRatio float64
	LongTermRatio  float64
	RetrievalRatio float64

	// Fixed ratios
	UserPrefsRatio float64 // Default: 0.10
}

BudgetProfile defines token allocation ratios for different intents.

type CacheProvider

type CacheProvider interface {
	Get(ctx context.Context, key string) ([]byte, bool)
	Set(ctx context.Context, key string, value []byte, ttl time.Duration) error
}

CacheProvider provides caching functionality.

type Config

type Config struct {
	MaxTurns    int           // Max conversation turns (default: 10)
	MaxEpisodes int           // Max episodic memories (default: 3)
	MaxTokens   int           // Default max tokens (default: 4096)
	CacheTTL    time.Duration // Cache TTL (default: 5 minutes)
}

Config configures the context builder service.

func DefaultConfig

func DefaultConfig() Config

DefaultConfig returns default configuration.

type ContextBuilder

type ContextBuilder interface {
	// Build constructs the full context from various sources.
	Build(ctx context.Context, req *ContextRequest) (*ContextResult, error)

	// GetStats returns context building statistics.
	GetStats() *ContextStats
}

ContextBuilder builds optimized context for LLM inference.

type ContextPriority

type ContextPriority int

ContextPriority represents the priority level of a context segment.

const (
	PrioritySystem      ContextPriority = 100 // System prompt - highest
	PriorityUserQuery   ContextPriority = 90  // Current user query
	PriorityRecentTurns ContextPriority = 80  // Most recent 3 turns
	PriorityRetrieval   ContextPriority = 70  // RAG retrieval results
	PriorityEpisodic    ContextPriority = 60  // Episodic memory
	PriorityPreferences ContextPriority = 50  // User preferences
	PriorityOlderTurns  ContextPriority = 40  // Older conversation turns
)

type ContextRequest

type ContextRequest struct {
	SessionID        string
	CurrentQuery     string
	AgentType        string
	RetrievalResults []*RetrievalItem
	MaxTokens        int
	UserID           int32
}

ContextRequest contains parameters for context building.

type ContextRequestWithMessages added in v0.94.0

type ContextRequestWithMessages struct {
	ContextRequest
	Messages []*Message
}

ContextRequestWithMessages extends ContextRequest with message history.

type ContextResult

type ContextResult struct {
	TokenBreakdown      *TokenBreakdown
	SystemPrompt        string
	ConversationContext string
	RetrievalContext    string
	UserPreferences     string
	TotalTokens         int
	BuildTime           time.Duration
}

ContextResult contains the built context.

type ContextSegment

type ContextSegment struct {
	Content   string
	Source    string
	Priority  ContextPriority
	TokenCost int
}

ContextSegment represents a piece of context with priority.

func PrioritizeAndTruncate

func PrioritizeAndTruncate(segments []*ContextSegment, budget int) []*ContextSegment

PrioritizeAndTruncate is a convenience function.

type ContextSnapshot added in v0.94.0

type ContextSnapshot struct {
	// Messages in the conversation.
	Messages []*Message

	// Retrieval results.
	RetrievalResults []*RetrievalItem

	// System prompt hash.
	SystemPromptHash string

	// User preferences hash.
	UserPrefsHash string

	// Episodic memories hash.
	EpisodicHash string

	// Timestamp of the snapshot.
	Timestamp time.Time

	// Token count at time of snapshot.
	TokenCount int

	// Query that was processed.
	Query string

	// Agent type used.
	AgentType string
}

ContextSnapshot captures a snapshot of the context for comparison.

func CreateSnapshot added in v0.94.0

func CreateSnapshot(req *ContextRequest, result *ContextResult) *ContextSnapshot

CreateSnapshot creates a snapshot from a context result.

func (*ContextSnapshot) ToJSON added in v0.94.0

func (s *ContextSnapshot) ToJSON() (string, error)

ToJSON converts a snapshot to JSON for debugging.

type ContextStats

type ContextStats struct {
	TotalBuilds      int64
	AverageTokens    float64
	CacheHits        int64
	AverageBuildTime time.Duration
}

ContextStats tracks context building metrics.

type Delta added in v0.94.0

type Delta struct {
	// NewMessages are messages added since last build.
	NewMessages []*Message

	// ModifiedSections are sections that changed.
	ModifiedSections []string

	// RemovedSections are sections that were removed.
	RemovedSections []string

	// NewRetrievalItems are new retrieval results.
	NewRetrievalItems []*RetrievalItem

	// PreviousHash is the hash of the previous context.
	PreviousHash string

	// CurrentHash is the hash of the current context.
	CurrentHash string

	// Strategy used for this delta.
	Strategy UpdateStrategy
}

Delta represents the changes between two context builds.

func (*Delta) ToJSON added in v0.94.0

func (d *Delta) ToJSON() (string, error)

ToJSON converts a delta to JSON for debugging.

type DeltaBuilder added in v0.94.0

type DeltaBuilder struct {
	// contains filtered or unexported fields
}

DeltaBuilder computes deltas between context builds.

func NewDeltaBuilder added in v0.94.0

func NewDeltaBuilder() *DeltaBuilder

NewDeltaBuilder creates a new delta builder.

func (*DeltaBuilder) Clear added in v0.94.0

func (d *DeltaBuilder) Clear()

Clear removes all cached snapshots.

func (*DeltaBuilder) ComputeDelta added in v0.94.0

func (d *DeltaBuilder) ComputeDelta(sessionID string, req *ContextRequest, prevSnapshot *ContextSnapshot) *Delta

ComputeDelta computes the delta between the current request and the previous snapshot.

func (*DeltaBuilder) GetSnapshot added in v0.94.0

func (d *DeltaBuilder) GetSnapshot(sessionID string) *ContextSnapshot

GetSnapshot retrieves the snapshot for a session.

func (*DeltaBuilder) GetStats added in v0.94.0

func (d *DeltaBuilder) GetStats() *DeltaStats

GetStats returns delta building statistics.

func (*DeltaBuilder) SaveSnapshot added in v0.94.0

func (d *DeltaBuilder) SaveSnapshot(sessionID string, snapshot *ContextSnapshot)

SaveSnapshot saves a snapshot for a session.

func (*DeltaBuilder) SelectStrategy added in v0.94.0

func (d *DeltaBuilder) SelectStrategy(req *ContextRequest, prevSnapshot *ContextSnapshot) UpdateStrategy

SelectStrategy selects the optimal update strategy based on the request and previous snapshot.

type DeltaStats added in v0.94.0

type DeltaStats struct {
	TotalDeltas          int64
	DeltaHits            int64
	FullRebuilds         int64
	AppendOnlyHits       int64
	ConversationOnlyHits int64
	AverageDeltaMs       int64
	SavedTokens          int64
	HitRate              float64
}

DeltaStats contains delta building statistics.

type EpisodicMemory

type EpisodicMemory struct {
	Timestamp time.Time
	Summary   string
	AgentType string
	Outcome   string
	ID        int64
}

EpisodicMemory represents a stored episode.

type EpisodicProvider

type EpisodicProvider interface {
	SearchEpisodes(ctx context.Context, userID int32, query string, limit int) ([]*EpisodicMemory, error)
}

EpisodicProvider provides episodic memory search.

type IncrementalBuilder added in v0.94.0

type IncrementalBuilder struct {
	// contains filtered or unexported fields
}

IncrementalBuilder provides incremental context building.

func NewIncrementalBuilder added in v0.94.0

func NewIncrementalBuilder(base *Service) *IncrementalBuilder

NewIncrementalBuilder creates a new incremental context builder.

func (*IncrementalBuilder) BuildIncremental added in v0.94.0

func (b *IncrementalBuilder) BuildIncremental(ctx context.Context, req *ContextRequest) (*ContextResult, error)

BuildIncremental builds context incrementally based on the optimal strategy.

func (*IncrementalBuilder) ClearCache added in v0.94.0

func (b *IncrementalBuilder) ClearCache()

ClearCache clears the incremental builder cache.

func (*IncrementalBuilder) GetDeltaStats added in v0.94.0

func (b *IncrementalBuilder) GetDeltaStats() *DeltaStats

GetDeltaStats returns delta building statistics.

type IntentResolver added in v0.94.0

type IntentResolver struct {
}

IntentResolver infers task intent from AgentType.

func NewIntentResolver added in v0.94.0

func NewIntentResolver() *IntentResolver

NewIntentResolver creates a new intent resolver.

func (*IntentResolver) Resolve added in v0.94.0

func (r *IntentResolver) Resolve(agentType string) string

Resolve infers intent from AgentType (thread-safe). Returns intent name (e.g., "memo_search", "schedule_create").

type LongTermContext

type LongTermContext struct {
	Preferences *UserPreferences
	Episodes    []*EpisodicMemory
}

LongTermContext contains extracted long-term context.

type LongTermExtractor

type LongTermExtractor struct {
	// contains filtered or unexported fields
}

LongTermExtractor extracts episodic memories and user preferences.

func NewLongTermExtractor

func NewLongTermExtractor(maxEpisodes int) *LongTermExtractor

NewLongTermExtractor creates a new long-term memory extractor.

func (*LongTermExtractor) Extract

func (e *LongTermExtractor) Extract(
	ctx context.Context,
	episodicProvider EpisodicProvider,
	prefProvider PreferenceProvider,
	userID int32,
	query string,
) (*LongTermContext, error)

Extract extracts long-term context for the user.

type Message

type Message struct {
	Timestamp time.Time
	Role      string
	Content   string
}

Message represents a conversation message.

func SplitByRecency

func SplitByRecency(messages []*Message, recentCount int) (recent, older []*Message)

SplitByRecency splits messages into recent (high priority) and older (lower priority).

type MessageProvider

type MessageProvider interface {
	GetRecentMessages(ctx context.Context, sessionID string, limit int) ([]*Message, error)
}

MessageProvider provides recent messages for a session.

type PreferenceProvider

type PreferenceProvider interface {
	GetPreferences(ctx context.Context, userID int32) (*UserPreferences, error)
}

PreferenceProvider provides user preferences.

type PriorityRanker

type PriorityRanker struct{}

PriorityRanker ranks and truncates context segments by priority.

func NewPriorityRanker

func NewPriorityRanker() *PriorityRanker

NewPriorityRanker creates a new priority ranker.

func (*PriorityRanker) RankAndTruncate

func (r *PriorityRanker) RankAndTruncate(segments []*ContextSegment, budget int) []*ContextSegment

RankAndTruncate sorts segments by priority and truncates to fit budget.

type ProfileRegistry added in v0.94.0

type ProfileRegistry struct {
	// contains filtered or unexported fields
}

ProfileRegistry manages budget profiles with override support.

func NewProfileRegistry added in v0.94.0

func NewProfileRegistry() *ProfileRegistry

NewProfileRegistry creates a new profile registry with defaults.

func (*ProfileRegistry) Get added in v0.94.0

func (r *ProfileRegistry) Get(name string) (*BudgetProfile, bool)

Get returns a profile by name (thread-safe).

func (*ProfileRegistry) LoadFromEnv added in v0.94.0

func (r *ProfileRegistry) LoadFromEnv()

LoadFromEnv overrides profiles from environment variables. Format: DIVINESENSE_BUDGET_<PROFILE>_<FIELD>=<value> Example: DIVINESENSE_BUDGET_MEMO_SEARCH_SHORT_TERM=0.35

func (*ProfileRegistry) Set added in v0.94.0

func (r *ProfileRegistry) Set(name string, profile *BudgetProfile)

Set adds or overrides a profile (thread-safe).

type RetrievalItem

type RetrievalItem struct {
	ID      string
	Content string
	Source  string
	Score   float32
}

RetrievalItem represents a single retrieval result.

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service implements ContextBuilder with caching support.

func NewService

func NewService(cfg Config) *Service

NewService creates a new context builder service.

func (*Service) Build

func (s *Service) Build(ctx context.Context, req *ContextRequest) (*ContextResult, error)

Build constructs the context for LLM inference.

func (*Service) GetStats

func (s *Service) GetStats() *ContextStats

GetStats returns context building statistics.

func (*Service) WithCache

func (s *Service) WithCache(c CacheProvider) *Service

WithCache sets the cache provider.

func (*Service) WithEpisodicProvider

func (s *Service) WithEpisodicProvider(p EpisodicProvider) *Service

WithEpisodicProvider sets the episodic memory provider.

func (*Service) WithMessageProvider

func (s *Service) WithMessageProvider(p MessageProvider) *Service

WithMessageProvider sets the message provider.

func (*Service) WithPreferenceProvider

func (s *Service) WithPreferenceProvider(p PreferenceProvider) *Service

WithPreferenceProvider sets the preference provider.

type ShortTermExtractor

type ShortTermExtractor struct {
	// contains filtered or unexported fields
}

ShortTermExtractor extracts recent conversation turns.

func NewShortTermExtractor

func NewShortTermExtractor(maxTurns int) *ShortTermExtractor

NewShortTermExtractor creates a new short-term memory extractor.

func (*ShortTermExtractor) Extract

func (e *ShortTermExtractor) Extract(ctx context.Context, provider MessageProvider, sessionID string) ([]*Message, error)

Extract extracts recent messages from the session.

type TokenBreakdown

type TokenBreakdown struct {
	SystemPrompt    int
	ShortTermMemory int
	LongTermMemory  int
	Retrieval       int
	UserPrefs       int
}

TokenBreakdown shows how tokens are distributed.

type TokenBudget

type TokenBudget struct {
	Total           int
	SystemPrompt    int
	ShortTermMemory int
	LongTermMemory  int
	Retrieval       int
	UserPrefs       int
}

TokenBudget represents the token allocation plan.

func AllocateBudget

func AllocateBudget(total int, hasRetrieval bool) *TokenBudget

AllocateBudget is a convenience function.

type UpdateStrategy added in v0.94.0

type UpdateStrategy int

UpdateStrategy defines the strategy for updating context.

const (
	// ComputeDelta computes only the changes since last build.
	ComputeDelta UpdateStrategy = iota

	// AppendOnly appends new messages without recomputing.
	AppendOnly

	// UpdateConversationOnly updates only conversation context.
	UpdateConversationOnly

	// FullRebuild performs a full context rebuild.
	FullRebuild
)

func (UpdateStrategy) String added in v0.94.0

func (s UpdateStrategy) String() string

String returns the string representation of the strategy.

type UserPreferences

type UserPreferences struct {
	Timezone           string
	CommunicationStyle string
	PreferredTimes     []string
	DefaultDuration    int
}

UserPreferences represents user preferences.

func DefaultUserPreferences

func DefaultUserPreferences() *UserPreferences

DefaultUserPreferences returns default preferences.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL