Documentation
¶
Overview ¶
Package compaction — LCM-inspired dual-state context management. Three-level escalation (detailed → bullet → truncate). Wired via run.go with per-session context; checks cheapest available model at runtime.
Package compaction — importance scoring + hierarchical compaction (§8.1 Phase 5 #8, application-layer approximation of Cartridge / Neural GC / Fast KV Compaction).
Honest scope: the original papers (Eyuboglu 2025, Li 2026, Zweiger 2026) operate on the model's KV cache. We don't have model weights or kernel access — we cannot literally implement them. What we CAN build is the same shape at the application layer:
- Importance scoring: rank context segments by recency × reuse × pinned-status × token-cost. Cheap segments and frequently- re-read segments stay; expensive-and-untouched ones evict.
- Hierarchical compaction: when a segment is evicted, write a short summary into a retrievable index instead of dropping it entirely. The agent can re-fetch via journal_search if it needs the original.
- Prefetch on relevance signal: when the agent reads a segment, pull adjacent ones into the cache proactively (composes with internal/speculative — that package handles the read cache; this one handles the eviction policy).
What's HERE: the scoring + eviction logic. The compactor itself (lcm.go, compactor.go) already lives in this package; importance. go plugs into it as an alternative ranking strategy. The hierarchical summary write-path is a SummaryWriter interface so the agent can plug in whatever LLM-driven summarizer it wants.
Index ¶
- func Compact(segments []*Segment, target EvictionTarget, opts ImportanceOptions) (keep []*Segment, evict []*Segment)
- func EnsureSegmentsValid(segs []*Segment) error
- func FormatEvictionReport(originalCount, originalTokens, finalCount, finalTokens int) string
- func HierarchicalCompact(ctx context.Context, segments []*Segment, target EvictionTarget, ...) (keep []*Segment, evict []*Segment, err error)
- func Score(seg *Segment, opts ImportanceOptions) float64
- type AgentCompactor
- type AgentPromptCompressor
- type AlertSink
- type CompactOptions
- type CompactionResult
- type Compactor
- type CompressionResult
- type EvictionTarget
- type HistoryFactory
- type HistoryStrategy
- type ImportanceOptions
- type LCMCompactor
- func (c *LCMCompactor) Compact(ctx context.Context, messages []providers.Message, opts CompactOptions) (*CompactionResult, error)
- func (c *LCMCompactor) EstimateUsage(messages []providers.Message, model string) int
- func (c *LCMCompactor) SetAlertSink(s AlertSink, sessionID string)
- func (c *LCMCompactor) ShouldCompact(messages []providers.Message, opts CompactOptions) (bool, float64)
- type Level
- type PromptCompressor
- type Segment
- type SummaryWriter
- type SummaryWriterFunc
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Compact ¶
func Compact(segments []*Segment, target EvictionTarget, opts ImportanceOptions) (keep []*Segment, evict []*Segment)
Compact decides which segments to evict to bring the total token cost under target.MaxTokens. Returns (keep, evict). Eviction order is lowest-score-first; pinned segments never appear in `evict`. Falls back to keeping at least MinKeep segments.
Caller's responsibility: actually drop the evicted segments from the live context. We just decide.
func EnsureSegmentsValid ¶
EnsureSegmentsValid is a sanity check for the caller — returns the first error if any segment has an empty ID, or nil when all look good. Useful in tests + before passing to Compact.
func FormatEvictionReport ¶
FormatEvictionReport renders a one-line summary of an eviction pass: "compacted N segments (M → K tokens)". Useful for the daemon log + the user-facing toast.
func HierarchicalCompact ¶
func HierarchicalCompact( ctx context.Context, segments []*Segment, target EvictionTarget, opts ImportanceOptions, summarize func(*Segment) string, writer SummaryWriter, ) (keep []*Segment, evict []*Segment, err error)
HierarchicalCompact extends Compact with summary-write-on-evict. For each evicted segment, the supplied Summarizer produces a short prose summary, and the SummaryWriter persists it. Errors in the write path are swallowed (best-effort) so a failing downstream doesn't block compaction.
summarize may return ("", nil) to skip the summary for a particular segment (e.g. trivial tool results).
func Score ¶
func Score(seg *Segment, opts ImportanceOptions) float64
Score returns the importance score for one segment. Higher = keep. Pinned segments return +Inf so the caller's eviction loop trivially skips them.
Types ¶
type AgentCompactor ¶
type AgentCompactor struct {
// contains filtered or unexported fields
}
AgentCompactor adapts LCMCompactor to the agent.HistoryCompactor interface. Constructed via NewAgentCompactor; safe to use as the agent's compactor.
func NewAgentCompactor ¶
func NewAgentCompactor(provider providers.Provider, tok *tokenizer.Estimator, preserveLast int) *AgentCompactor
NewAgentCompactor builds an LCM-backed compactor adapter for use with Agent.SetCompactor. Defaults match DefaultCompactOptions; pass overrides via the optional functional options if needed later.
func (*AgentCompactor) Compact ¶
func (c *AgentCompactor) Compact(ctx context.Context, msgs []providers.Message, model string, maxTokens int) ([]providers.Message, string, error)
Compact satisfies agent.HistoryCompactor by delegating to the LCM 3-level escalation pipeline and returning the preserved trailing messages plus summary.
func (*AgentCompactor) SetAlertSink ¶
func (c *AgentCompactor) SetAlertSink(s AlertSink, sessionID string)
SetAlertSink threads through to the underlying LCMCompactor.
func (*AgentCompactor) SetCompactionModel ¶
func (c *AgentCompactor) SetCompactionModel(model string)
SetCompactionModel overrides the model used for the summarisation LLM call. Empty string restores the DefaultCompactOptions default.
type AgentPromptCompressor ¶
type AgentPromptCompressor struct {
// contains filtered or unexported fields
}
AgentPromptCompressor wraps a PromptCompressor to satisfy agent.PromptCompressor. The agent expects Compress(ctx, prompt) (string, int, error) where int = tokens saved.
func NewAgentPromptCompressor ¶
func NewAgentPromptCompressor(provider providers.Provider, model string) *AgentPromptCompressor
NewAgentPromptCompressor builds the adapter. model may be empty (falls back to cheapest).
type AlertSink ¶
AlertSink receives compaction-skip alerts (master plan §4.19). Tiny interface so the compaction package stays free of journal imports.
type CompactOptions ¶
type CompactOptions struct {
MaxTokens int `json:"max_tokens"`
PreserveLast int `json:"preserve_last"`
Model string `json:"model"`
CompactionModel string `json:"compaction_model"`
SoftThreshold float64 `json:"soft_threshold"`
HardThreshold float64 `json:"hard_threshold"`
}
func DefaultCompactOptions ¶
func DefaultCompactOptions() CompactOptions
type CompactionResult ¶
type CompactionResult struct {
Summary string `json:"summary"`
Messages []providers.Message `json:"messages"` // preserved trailing messages
Level Level `json:"level"`
OriginalTokens int `json:"original_tokens"`
CompactedTokens int `json:"compacted_tokens"`
Ratio float64 `json:"ratio"`
MessagesCompacted int `json:"messages_compacted"`
MessagesPreserved int `json:"messages_preserved"`
Error string `json:"error,omitempty"`
}
type CompressionResult ¶
type EvictionTarget ¶
type EvictionTarget struct {
// MaxTokens caps the total token budget after eviction. Caller
// usually sets this to the model's context window minus a
// safety margin.
MaxTokens int
// MinKeep guarantees at least this many segments survive even
// if they exceed MaxTokens. Protects the agent from over-
// compacting itself into amnesia.
MinKeep int
}
EvictionTarget describes how aggressively to compact.
type HistoryFactory ¶
type HistoryFactory struct {
Store *seahorse.PostgresStore
// contains filtered or unexported fields
}
func NewHistoryFactory ¶
func NewHistoryFactory(ctx context.Context, cfg config.CompactionConfig, db *sql.DB) (*HistoryFactory, error)
func (*HistoryFactory) Build ¶
func (f *HistoryFactory) Build(provider providers.Provider, sessionID string) (HistoryStrategy, bool, error)
func (*HistoryFactory) Strategy ¶
func (f *HistoryFactory) Strategy() string
type HistoryStrategy ¶
type HistoryStrategy interface {
Compact(context.Context, []providers.Message, string, int) ([]providers.Message, string, error)
}
HistoryStrategy is structurally identical to agent.HistoryCompactor; keeping this package independent of agent avoids a runtime/agent dependency cycle.
type ImportanceOptions ¶
type ImportanceOptions struct {
// RecencyHalfLife after which the recency contribution drops
// to 0.5. Default 30 minutes — context compaction operates on
// much shorter horizons than memory segments.
RecencyHalfLife time.Duration
// RecencyWeight, ReuseWeight, CostWeight scale each axis.
// Pinned is binary — pinned segments score Infinity, never
// evict.
RecencyWeight float64
ReuseWeight float64
// CostWeight is INVERTED: bigger segments get LOWER scores
// (more attractive to evict). Default 0.5.
CostWeight float64
// Now lets tests inject a deterministic clock.
Now func() time.Time
}
ImportanceOptions tunes the score weights.
type LCMCompactor ¶
type LCMCompactor struct {
// contains filtered or unexported fields
}
func NewLCMCompactor ¶
func NewLCMCompactor(provider providers.Provider, tok *tokenizer.Estimator) *LCMCompactor
func (*LCMCompactor) Compact ¶
func (c *LCMCompactor) Compact(ctx context.Context, messages []providers.Message, opts CompactOptions) (*CompactionResult, error)
func (*LCMCompactor) EstimateUsage ¶
func (c *LCMCompactor) EstimateUsage(messages []providers.Message, model string) int
func (*LCMCompactor) SetAlertSink ¶
func (c *LCMCompactor) SetAlertSink(s AlertSink, sessionID string)
SetAlertSink wires a sink that receives a compaction_skip alert when the LCM falls all the way to truncation (Level 3) — i.e., no LLM savings.
func (*LCMCompactor) ShouldCompact ¶
func (c *LCMCompactor) ShouldCompact(messages []providers.Message, opts CompactOptions) (bool, float64)
type PromptCompressor ¶
type PromptCompressor struct {
// contains filtered or unexported fields
}
func NewPromptCompressor ¶
func NewPromptCompressor(provider providers.Provider, model string) *PromptCompressor
func (*PromptCompressor) Compress ¶
func (pc *PromptCompressor) Compress(ctx context.Context, prompt string) (string, *CompressionResult, error)
func (*PromptCompressor) IsEnabled ¶
func (pc *PromptCompressor) IsEnabled() bool
func (*PromptCompressor) SetEnabled ¶
func (pc *PromptCompressor) SetEnabled(enabled bool)
func (*PromptCompressor) SetMinSavingRatio ¶
func (pc *PromptCompressor) SetMinSavingRatio(ratio float64)
type Segment ¶
type Segment struct {
// ID is a stable identifier used to track segments across
// scoring passes. Could be a message ID, tool-call ID, or any
// opaque caller-supplied tag.
ID string
// Content is the actual context bytes. Token-cost estimate
// derived from len(Content) when TokenCost is 0.
Content string
// TokenCost is the caller's estimate (or measured count) for
// this segment. 0 → estimate as len(Content) / 4 (rough English
// rule of thumb).
TokenCost int
// CreatedAt is when the segment first entered the context.
CreatedAt time.Time
// LastAccessed is bumped whenever the agent re-reads the
// segment. Recency component of the score.
LastAccessed time.Time
// AccessCount tracks how often the segment has been re-read.
// Reuse component of the score.
AccessCount int
// Pinned, when true, never evicts. Use for the user's current
// turn, the active plan, or anything the caller wants to keep
// regardless of score.
Pinned bool
// Kind is operator-readable: "user_input", "agent_reply",
// "tool_call", "tool_result", "system". Same vocabulary as the
// flight recorder but here it's just metadata.
Kind string
}
Segment is one chunk of context with the metadata importance scoring needs.
func Rank ¶
func Rank(segments []*Segment, opts ImportanceOptions) []*Segment
Rank returns segments sorted by Score, lowest-first — i.e. the eviction order. Pinned segments land at the end (highest score). Stable sort preserves caller-supplied order for ties.
type SummaryWriter ¶
type SummaryWriter interface {
WriteSummary(ctx context.Context, segment *Segment, summary string) error
}
SummaryWriter is the hook for hierarchical compaction. When the caller evicts a segment, it can ask a SummaryWriter to write a retrievable summary into a long-term index (typically the observation store). The agent can later journal_search to pull the summary back when relevance signal demands it.
type SummaryWriterFunc ¶
SummaryWriterFunc adapts a function to the interface.