compaction

package
v0.11.0-beta.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 23, 2026 License: Apache-2.0, MIT Imports: 14 Imported by: 0

Documentation

Overview

Package compaction — LCM-inspired dual-state context management. Three-level escalation (detailed → bullet → truncate). Wired via run.go with per-session context; checks cheapest available model at runtime.

Package compaction — importance scoring + hierarchical compaction (§8.1 Phase 5 #8, application-layer approximation of Cartridge / Neural GC / Fast KV Compaction).

Honest scope: the original papers (Eyuboglu 2025, Li 2026, Zweiger 2026) operate on the model's KV cache. We don't have model weights or kernel access — we cannot literally implement them. What we CAN build is the same shape at the application layer:

  • Importance scoring: rank context segments by recency × reuse × pinned-status × token-cost. Cheap segments and frequently- re-read segments stay; expensive-and-untouched ones evict.
  • Hierarchical compaction: when a segment is evicted, write a short summary into a retrievable index instead of dropping it entirely. The agent can re-fetch via journal_search if it needs the original.
  • Prefetch on relevance signal: when the agent reads a segment, pull adjacent ones into the cache proactively (composes with internal/speculative — that package handles the read cache; this one handles the eviction policy).

What's HERE: the scoring + eviction logic. The compactor itself (lcm.go, compactor.go) already lives in this package; importance. go plugs into it as an alternative ranking strategy. The hierarchical summary write-path is a SummaryWriter interface so the agent can plug in whatever LLM-driven summarizer it wants.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Compact

func Compact(segments []*Segment, target EvictionTarget, opts ImportanceOptions) (keep []*Segment, evict []*Segment)

Compact decides which segments to evict to bring the total token cost under target.MaxTokens. Returns (keep, evict). Eviction order is lowest-score-first; pinned segments never appear in `evict`. Falls back to keeping at least MinKeep segments.

Caller's responsibility: actually drop the evicted segments from the live context. We just decide.

func EnsureSegmentsValid

func EnsureSegmentsValid(segs []*Segment) error

EnsureSegmentsValid is a sanity check for the caller — returns the first error if any segment has an empty ID, or nil when all look good. Useful in tests + before passing to Compact.

func FormatEvictionReport

func FormatEvictionReport(originalCount, originalTokens, finalCount, finalTokens int) string

FormatEvictionReport renders a one-line summary of an eviction pass: "compacted N segments (M → K tokens)". Useful for the daemon log + the user-facing toast.

func HierarchicalCompact

func HierarchicalCompact(
	ctx context.Context,
	segments []*Segment,
	target EvictionTarget,
	opts ImportanceOptions,
	summarize func(*Segment) string,
	writer SummaryWriter,
) (keep []*Segment, evict []*Segment, err error)

HierarchicalCompact extends Compact with summary-write-on-evict. For each evicted segment, the supplied Summarizer produces a short prose summary, and the SummaryWriter persists it. Errors in the write path are swallowed (best-effort) so a failing downstream doesn't block compaction.

summarize may return ("", nil) to skip the summary for a particular segment (e.g. trivial tool results).

func Score

func Score(seg *Segment, opts ImportanceOptions) float64

Score returns the importance score for one segment. Higher = keep. Pinned segments return +Inf so the caller's eviction loop trivially skips them.

Types

type AgentCompactor

type AgentCompactor struct {
	// contains filtered or unexported fields
}

AgentCompactor adapts LCMCompactor to the agent.HistoryCompactor interface. Constructed via NewAgentCompactor; safe to use as the agent's compactor.

func NewAgentCompactor

func NewAgentCompactor(provider providers.Provider, tok *tokenizer.Estimator, preserveLast int) *AgentCompactor

NewAgentCompactor builds an LCM-backed compactor adapter for use with Agent.SetCompactor. Defaults match DefaultCompactOptions; pass overrides via the optional functional options if needed later.

func (*AgentCompactor) Compact

func (c *AgentCompactor) Compact(ctx context.Context, msgs []providers.Message, model string, maxTokens int) ([]providers.Message, string, error)

Compact satisfies agent.HistoryCompactor by delegating to the LCM 3-level escalation pipeline and returning the preserved trailing messages plus summary.

func (*AgentCompactor) SetAlertSink

func (c *AgentCompactor) SetAlertSink(s AlertSink, sessionID string)

SetAlertSink threads through to the underlying LCMCompactor.

func (*AgentCompactor) SetCompactionModel

func (c *AgentCompactor) SetCompactionModel(model string)

SetCompactionModel overrides the model used for the summarisation LLM call. Empty string restores the DefaultCompactOptions default.

type AgentPromptCompressor

type AgentPromptCompressor struct {
	// contains filtered or unexported fields
}

AgentPromptCompressor wraps a PromptCompressor to satisfy agent.PromptCompressor. The agent expects Compress(ctx, prompt) (string, int, error) where int = tokens saved.

func NewAgentPromptCompressor

func NewAgentPromptCompressor(provider providers.Provider, model string) *AgentPromptCompressor

NewAgentPromptCompressor builds the adapter. model may be empty (falls back to cheapest).

func (*AgentPromptCompressor) Compress

func (a *AgentPromptCompressor) Compress(ctx context.Context, prompt string) (string, int, error)

type AlertSink

type AlertSink interface {
	Create(alertType, message, sessionID string) error
}

AlertSink receives compaction-skip alerts (master plan §4.19). Tiny interface so the compaction package stays free of journal imports.

type CompactOptions

type CompactOptions struct {
	MaxTokens       int     `json:"max_tokens"`
	PreserveLast    int     `json:"preserve_last"`
	Model           string  `json:"model"`
	CompactionModel string  `json:"compaction_model"`
	SoftThreshold   float64 `json:"soft_threshold"`
	HardThreshold   float64 `json:"hard_threshold"`
}

func DefaultCompactOptions

func DefaultCompactOptions() CompactOptions

type CompactionResult

type CompactionResult struct {
	Summary           string              `json:"summary"`
	Messages          []providers.Message `json:"messages"` // preserved trailing messages
	Level             Level               `json:"level"`
	OriginalTokens    int                 `json:"original_tokens"`
	CompactedTokens   int                 `json:"compacted_tokens"`
	Ratio             float64             `json:"ratio"`
	MessagesCompacted int                 `json:"messages_compacted"`
	MessagesPreserved int                 `json:"messages_preserved"`
	Error             string              `json:"error,omitempty"`
}

type Compactor

type Compactor interface {
	Compact(ctx context.Context, messages []providers.Message, opts CompactOptions) (*CompactionResult, error)
	EstimateUsage(messages []providers.Message, model string) int
	ShouldCompact(messages []providers.Message, opts CompactOptions) (bool, float64)
}

type CompressionResult

type CompressionResult struct {
	OriginalTokens   int
	CompressedTokens int
	Ratio            float64
	Skipped          bool
	SkippedReason    string
	ModelUsed        string
}

type EvictionTarget

type EvictionTarget struct {
	// MaxTokens caps the total token budget after eviction. Caller
	// usually sets this to the model's context window minus a
	// safety margin.
	MaxTokens int
	// MinKeep guarantees at least this many segments survive even
	// if they exceed MaxTokens. Protects the agent from over-
	// compacting itself into amnesia.
	MinKeep int
}

EvictionTarget describes how aggressively to compact.

type HistoryFactory

type HistoryFactory struct {
	Store *seahorse.PostgresStore
	// contains filtered or unexported fields
}

func NewHistoryFactory

func NewHistoryFactory(ctx context.Context, cfg config.CompactionConfig, db *sql.DB) (*HistoryFactory, error)

func (*HistoryFactory) Build

func (f *HistoryFactory) Build(provider providers.Provider, sessionID string) (HistoryStrategy, bool, error)

func (*HistoryFactory) Strategy

func (f *HistoryFactory) Strategy() string

type HistoryStrategy

type HistoryStrategy interface {
	Compact(context.Context, []providers.Message, string, int) ([]providers.Message, string, error)
}

HistoryStrategy is structurally identical to agent.HistoryCompactor; keeping this package independent of agent avoids a runtime/agent dependency cycle.

type ImportanceOptions

type ImportanceOptions struct {
	// RecencyHalfLife after which the recency contribution drops
	// to 0.5. Default 30 minutes — context compaction operates on
	// much shorter horizons than memory segments.
	RecencyHalfLife time.Duration
	// RecencyWeight, ReuseWeight, CostWeight scale each axis.
	// Pinned is binary — pinned segments score Infinity, never
	// evict.
	RecencyWeight float64
	ReuseWeight   float64
	// CostWeight is INVERTED: bigger segments get LOWER scores
	// (more attractive to evict). Default 0.5.
	CostWeight float64
	// Now lets tests inject a deterministic clock.
	Now func() time.Time
}

ImportanceOptions tunes the score weights.

type LCMCompactor

type LCMCompactor struct {
	// contains filtered or unexported fields
}

func NewLCMCompactor

func NewLCMCompactor(provider providers.Provider, tok *tokenizer.Estimator) *LCMCompactor

func (*LCMCompactor) Compact

func (c *LCMCompactor) Compact(ctx context.Context, messages []providers.Message, opts CompactOptions) (*CompactionResult, error)

func (*LCMCompactor) EstimateUsage

func (c *LCMCompactor) EstimateUsage(messages []providers.Message, model string) int

func (*LCMCompactor) SetAlertSink

func (c *LCMCompactor) SetAlertSink(s AlertSink, sessionID string)

SetAlertSink wires a sink that receives a compaction_skip alert when the LCM falls all the way to truncation (Level 3) — i.e., no LLM savings.

func (*LCMCompactor) ShouldCompact

func (c *LCMCompactor) ShouldCompact(messages []providers.Message, opts CompactOptions) (bool, float64)

type Level

type Level int
const (
	LevelNone        Level = iota
	Level1Detailed         // LLM summary, preserve details
	Level2Aggressive       // LLM summary, bullet points, half target tokens
	Level3Truncate         // Deterministic truncation, no LLM, always succeeds
)

func (Level) String

func (l Level) String() string

type PromptCompressor

type PromptCompressor struct {
	// contains filtered or unexported fields
}

func NewPromptCompressor

func NewPromptCompressor(provider providers.Provider, model string) *PromptCompressor

func (*PromptCompressor) Compress

func (pc *PromptCompressor) Compress(ctx context.Context, prompt string) (string, *CompressionResult, error)

func (*PromptCompressor) IsEnabled

func (pc *PromptCompressor) IsEnabled() bool

func (*PromptCompressor) SetEnabled

func (pc *PromptCompressor) SetEnabled(enabled bool)

func (*PromptCompressor) SetMinSavingRatio

func (pc *PromptCompressor) SetMinSavingRatio(ratio float64)

type Segment

type Segment struct {
	// ID is a stable identifier used to track segments across
	// scoring passes. Could be a message ID, tool-call ID, or any
	// opaque caller-supplied tag.
	ID string
	// Content is the actual context bytes. Token-cost estimate
	// derived from len(Content) when TokenCost is 0.
	Content string
	// TokenCost is the caller's estimate (or measured count) for
	// this segment. 0 → estimate as len(Content) / 4 (rough English
	// rule of thumb).
	TokenCost int
	// CreatedAt is when the segment first entered the context.
	CreatedAt time.Time
	// LastAccessed is bumped whenever the agent re-reads the
	// segment. Recency component of the score.
	LastAccessed time.Time
	// AccessCount tracks how often the segment has been re-read.
	// Reuse component of the score.
	AccessCount int
	// Pinned, when true, never evicts. Use for the user's current
	// turn, the active plan, or anything the caller wants to keep
	// regardless of score.
	Pinned bool
	// Kind is operator-readable: "user_input", "agent_reply",
	// "tool_call", "tool_result", "system". Same vocabulary as the
	// flight recorder but here it's just metadata.
	Kind string
}

Segment is one chunk of context with the metadata importance scoring needs.

func Rank

func Rank(segments []*Segment, opts ImportanceOptions) []*Segment

Rank returns segments sorted by Score, lowest-first — i.e. the eviction order. Pinned segments land at the end (highest score). Stable sort preserves caller-supplied order for ties.

type SummaryWriter

type SummaryWriter interface {
	WriteSummary(ctx context.Context, segment *Segment, summary string) error
}

SummaryWriter is the hook for hierarchical compaction. When the caller evicts a segment, it can ask a SummaryWriter to write a retrievable summary into a long-term index (typically the observation store). The agent can later journal_search to pull the summary back when relevance signal demands it.

type SummaryWriterFunc

type SummaryWriterFunc func(ctx context.Context, segment *Segment, summary string) error

SummaryWriterFunc adapts a function to the interface.

func (SummaryWriterFunc) WriteSummary

func (f SummaryWriterFunc) WriteSummary(ctx context.Context, seg *Segment, summary string) error

Directories

Path Synopsis
Package seahorse implements hierarchical DAG summarization — a more sophisticated alternative to flat LCM compaction.
Package seahorse implements hierarchical DAG summarization — a more sophisticated alternative to flat LCM compaction.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL