context

package
v0.80.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 1, 2026 License: MIT Imports: 7 Imported by: 0

Documentation

Overview

Package context provides context building for LLM prompts.

Package context provides context building for LLM prompts. It orchestrates short-term memory, long-term memory, and retrieval results into an optimized context window for LLM inference.

Package context provides context building for LLM prompts.

Package context provides context building for LLM prompts.

Package context provides context building for LLM prompts.

Package context provides context building for LLM prompts.

Index

Constants

View Source
const (
	DefaultMaxTokens      = 4096
	DefaultSystemPrompt   = 500
	DefaultUserPrefsRatio = 0.10
	DefaultRetrievalRatio = 0.35
	MinSegmentTokens      = 100
)

Default token budget values.

Variables

This section is empty.

Functions

func EstimateTokens

func EstimateTokens(content string) int

EstimateTokens estimates the token count for a string. Uses heuristic: Chinese chars count as ~2 tokens, ASCII as ~0.3 tokens per char.

func FormatConversation

func FormatConversation(messages []*Message) string

FormatConversation formats messages into conversation context.

func FormatEpisodes

func FormatEpisodes(episodes []*EpisodicMemory) string

FormatEpisodes formats episodic memories into context.

func FormatPreferences

func FormatPreferences(prefs *UserPreferences) string

FormatPreferences formats user preferences into context.

Types

type BudgetAllocator

type BudgetAllocator struct {
	// contains filtered or unexported fields
}

BudgetAllocator allocates token budgets.

func NewBudgetAllocator

func NewBudgetAllocator() *BudgetAllocator

NewBudgetAllocator creates a new budget allocator with defaults.

func (*BudgetAllocator) Allocate

func (a *BudgetAllocator) Allocate(total int, hasRetrieval bool) *TokenBudget

Allocate allocates token budget based on total and whether retrieval is needed.

type CacheProvider

type CacheProvider interface {
	Get(ctx context.Context, key string) ([]byte, bool)
	Set(ctx context.Context, key string, value []byte, ttl time.Duration) error
}

CacheProvider provides caching functionality.

type Config

type Config struct {
	MaxTurns    int           // Max conversation turns (default: 10)
	MaxEpisodes int           // Max episodic memories (default: 3)
	MaxTokens   int           // Default max tokens (default: 4096)
	CacheTTL    time.Duration // Cache TTL (default: 5 minutes)
}

Config configures the context builder service.

func DefaultConfig

func DefaultConfig() Config

DefaultConfig returns default configuration.

type ContextBuilder

type ContextBuilder interface {
	// Build constructs the full context from various sources.
	Build(ctx context.Context, req *ContextRequest) (*ContextResult, error)

	// GetStats returns context building statistics.
	GetStats() *ContextStats
}

ContextBuilder builds optimized context for LLM inference.

type ContextPriority

type ContextPriority int

ContextPriority represents the priority level of a context segment.

const (
	PrioritySystem      ContextPriority = 100 // System prompt - highest
	PriorityUserQuery   ContextPriority = 90  // Current user query
	PriorityRecentTurns ContextPriority = 80  // Most recent 3 turns
	PriorityRetrieval   ContextPriority = 70  // RAG retrieval results
	PriorityEpisodic    ContextPriority = 60  // Episodic memory
	PriorityPreferences ContextPriority = 50  // User preferences
	PriorityOlderTurns  ContextPriority = 40  // Older conversation turns
)

type ContextRequest

type ContextRequest struct {
	SessionID        string
	CurrentQuery     string
	AgentType        string
	RetrievalResults []*RetrievalItem
	MaxTokens        int
	UserID           int32
}

ContextRequest contains parameters for context building.

type ContextResult

type ContextResult struct {
	TokenBreakdown      *TokenBreakdown
	SystemPrompt        string
	ConversationContext string
	RetrievalContext    string
	UserPreferences     string
	TotalTokens         int
	BuildTime           time.Duration
}

ContextResult contains the built context.

type ContextSegment

type ContextSegment struct {
	Content   string
	Source    string
	Priority  ContextPriority
	TokenCost int
}

ContextSegment represents a piece of context with priority.

func PrioritizeAndTruncate

func PrioritizeAndTruncate(segments []*ContextSegment, budget int) []*ContextSegment

PrioritizeAndTruncate is a convenience function.

type ContextStats

type ContextStats struct {
	TotalBuilds      int64
	AverageTokens    float64
	CacheHits        int64
	AverageBuildTime time.Duration
}

ContextStats tracks context building metrics.

type EpisodicMemory

type EpisodicMemory struct {
	Timestamp time.Time
	Summary   string
	AgentType string
	Outcome   string
	ID        int64
}

EpisodicMemory represents a stored episode.

type EpisodicProvider

type EpisodicProvider interface {
	SearchEpisodes(ctx context.Context, userID int32, query string, limit int) ([]*EpisodicMemory, error)
}

EpisodicProvider provides episodic memory search.

type LongTermContext

type LongTermContext struct {
	Preferences *UserPreferences
	Episodes    []*EpisodicMemory
}

LongTermContext contains extracted long-term context.

type LongTermExtractor

type LongTermExtractor struct {
	// contains filtered or unexported fields
}

LongTermExtractor extracts episodic memories and user preferences.

func NewLongTermExtractor

func NewLongTermExtractor(maxEpisodes int) *LongTermExtractor

NewLongTermExtractor creates a new long-term memory extractor.

func (*LongTermExtractor) Extract

func (e *LongTermExtractor) Extract(
	ctx context.Context,
	episodicProvider EpisodicProvider,
	prefProvider PreferenceProvider,
	userID int32,
	query string,
) (*LongTermContext, error)

Extract extracts long-term context for the user.

type Message

type Message struct {
	Timestamp time.Time
	Role      string
	Content   string
}

Message represents a conversation message.

func SplitByRecency

func SplitByRecency(messages []*Message, recentCount int) (recent, older []*Message)

SplitByRecency splits messages into recent (high priority) and older (lower priority).

type MessageProvider

type MessageProvider interface {
	GetRecentMessages(ctx context.Context, sessionID string, limit int) ([]*Message, error)
}

MessageProvider provides recent messages for a session.

type PreferenceProvider

type PreferenceProvider interface {
	GetPreferences(ctx context.Context, userID int32) (*UserPreferences, error)
}

PreferenceProvider provides user preferences.

type PriorityRanker

type PriorityRanker struct{}

PriorityRanker ranks and truncates context segments by priority.

func NewPriorityRanker

func NewPriorityRanker() *PriorityRanker

NewPriorityRanker creates a new priority ranker.

func (*PriorityRanker) RankAndTruncate

func (r *PriorityRanker) RankAndTruncate(segments []*ContextSegment, budget int) []*ContextSegment

RankAndTruncate sorts segments by priority and truncates to fit budget.

type RetrievalItem

type RetrievalItem struct {
	ID      string
	Content string
	Source  string
	Score   float32
}

RetrievalItem represents a single retrieval result.

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service implements ContextBuilder with caching support.

func NewService

func NewService(cfg Config) *Service

NewService creates a new context builder service.

func (*Service) Build

func (s *Service) Build(ctx context.Context, req *ContextRequest) (*ContextResult, error)

Build constructs the context for LLM inference.

func (*Service) GetStats

func (s *Service) GetStats() *ContextStats

GetStats returns context building statistics.

func (*Service) WithCache

func (s *Service) WithCache(c CacheProvider) *Service

WithCache sets the cache provider.

func (*Service) WithEpisodicProvider

func (s *Service) WithEpisodicProvider(p EpisodicProvider) *Service

WithEpisodicProvider sets the episodic memory provider.

func (*Service) WithMessageProvider

func (s *Service) WithMessageProvider(p MessageProvider) *Service

WithMessageProvider sets the message provider.

func (*Service) WithPreferenceProvider

func (s *Service) WithPreferenceProvider(p PreferenceProvider) *Service

WithPreferenceProvider sets the preference provider.

type ShortTermExtractor

type ShortTermExtractor struct {
	// contains filtered or unexported fields
}

ShortTermExtractor extracts recent conversation turns.

func NewShortTermExtractor

func NewShortTermExtractor(maxTurns int) *ShortTermExtractor

NewShortTermExtractor creates a new short-term memory extractor.

func (*ShortTermExtractor) Extract

func (e *ShortTermExtractor) Extract(ctx context.Context, provider MessageProvider, sessionID string) ([]*Message, error)

Extract extracts recent messages from the session.

type TokenBreakdown

type TokenBreakdown struct {
	SystemPrompt    int
	ShortTermMemory int
	LongTermMemory  int
	Retrieval       int
	UserPrefs       int
}

TokenBreakdown shows how tokens are distributed.

type TokenBudget

type TokenBudget struct {
	Total           int
	SystemPrompt    int
	ShortTermMemory int
	LongTermMemory  int
	Retrieval       int
	UserPrefs       int
}

TokenBudget represents the token allocation plan.

func AllocateBudget

func AllocateBudget(total int, hasRetrieval bool) *TokenBudget

AllocateBudget is a convenience function.

type UserPreferences

type UserPreferences struct {
	Timezone           string
	CommunicationStyle string
	PreferredTimes     []string
	DefaultDuration    int
}

UserPreferences represents user preferences.

func DefaultUserPreferences

func DefaultUserPreferences() *UserPreferences

DefaultUserPreferences returns default preferences.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL