Documentation
¶
Overview ¶
Package compaction provides context window management with token estimation, compaction triggers, and LLM-based conversation summarization.
The algorithm preserves a token budget of recent messages (KeepRecentTokens, default 20 000) rather than a fixed message count. Auto-compaction fires when estimated context usage exceeds contextWindow − ReserveTokens.
Features modelled after pi's compaction system:
- Tool result truncation (2000 char max) during serialisation
- Split turn handling: when a single turn exceeds the keep budget, the turn prefix is summarised separately and merged
- Cumulative file tracking: read and modified files extracted from tool calls and carried forward across compactions
Index ¶
- func EstimateMessageTokens(messages []fantasy.Message) int
- func FindCutPoint(messages []fantasy.Message, keepRecentTokens int) int
- func IsSplitTurn(messages []fantasy.Message, cutPoint int) bool
- func ShouldCompact(messages []fantasy.Message, contextWindow int, reserveTokens int) bool
- type CompactionOptions
- type CompactionResult
- type PreviousCompaction
- type StreamCallback
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func EstimateMessageTokens ¶
EstimateMessageTokens estimates total tokens across a slice of fantasy messages by summing the estimated tokens for every text part.
func FindCutPoint ¶
FindCutPoint walks backward from the end of messages, accumulating tokens until the keepRecentTokens budget is filled. Returns the index that separates "old" messages (0..cutPoint-1, to be summarised) from "recent" messages (cutPoint..end, to be preserved).
The cut point prefers turn boundaries (user messages). When a single turn exceeds the budget, the cut lands mid-turn (IsSplitTurn returns true).
Returns 0 if there are fewer than 2 messages or all messages fit within the keep budget.
func IsSplitTurn ¶ added in v0.19.0
IsSplitTurn returns true if the cut point lands in the middle of a turn (i.e. the message at cutPoint is not a user message, meaning we're splitting a single turn's assistant/tool messages).
Types ¶
type CompactionOptions ¶
type CompactionOptions struct {
ContextWindow int // Model's context window size (tokens)
ReserveTokens int // Tokens to reserve for LLM response, default 16384
KeepRecentTokens int // Recent tokens to preserve (not summarised), default 20000
SummaryPrompt string // Custom summary prompt (empty = use default)
}
CompactionOptions configures compaction behaviour. Token-based defaults are applied for zero-value fields.
type CompactionResult ¶
type CompactionResult struct {
Summary string // LLM-generated summary of compacted messages
OriginalTokens int // Estimated token count before compaction
CompactedTokens int // Estimated token count after compaction
MessagesRemoved int // Number of messages replaced by the summary
CutPoint int // Index in the original messages where the cut was made
ReadFiles []string // Files read during the compacted conversation
ModifiedFiles []string // Files modified during the compacted conversation
}
CompactionResult contains statistics from a compaction operation.
func Compact ¶
func Compact( ctx context.Context, model fantasy.LanguageModel, messages []fantasy.Message, opts CompactionOptions, customInstructions string, prev *PreviousCompaction, onChunk StreamCallback, ) (*CompactionResult, []fantasy.Message, error)
Compact summarises older messages using the LLM, returning the compaction result and a new message slice (summary message + preserved recent messages).
The model parameter is the same fantasy.LanguageModel used for regular generation — compaction creates a disposable fantasy agent with no tools to produce the summary.
customInstructions is optional text appended to the summary prompt (e.g. "Focus on the API design decisions"). Pass "" to use the default prompt only.
prev carries file tracking from a previous compaction for cumulative tracking. Pass nil if there is no prior compaction. onChunk is an optional callback for streaming summary text. Pass nil for non-streaming compaction.
type PreviousCompaction ¶ added in v0.19.0
PreviousCompaction carries file tracking state from a prior compaction so that file operations accumulate across multiple compactions.
type StreamCallback ¶ added in v0.33.0
StreamCallback is called for each chunk of text during streaming compaction. Return a non-nil error to cancel the stream.
Source Files