compaction

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 1, 2026 License: MIT Imports: 11 Imported by: 0

Documentation

Overview

Package compaction reduces conversation history to fit a model's context window. It provides a context provider and composable strategies — sliding window, truncation, summarization, tool-result eviction, and context-window sizing — selected by triggers evaluated over a message index.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Compact

func Compact(ctx context.Context, strategy Strategy, messages []*message.Message, tokenCounter TokenCounter) ([]*message.Message, error)

Compact applies a strategy to messages and returns the included compacted messages.

It is useful for ad-hoc compaction outside of a context provider. The input messages are first grouped into a MessageIndex, the strategy is applied, and only non-excluded messages are returned.

func DefaultToolCallFormatter

func DefaultToolCallFormatter(group *MessageGroup) string

DefaultToolCallFormatter produces a YAML-like summary of tool-call groups, including tool names, results, and deduplication counts for repeated tool names.

This is the formatter used when no custom ToolCallFormatter is supplied. It can be referenced directly in a custom formatter to augment or wrap the default output.

func NewContextProvider

func NewContextProvider(cfg ContextProviderConfig) agent.ContextProvider

NewContextProvider creates a context provider that applies compaction before each agent run.

When a local session is available, the provider stores message-group state so subsequent runs can incrementally update the index. Without a session, it still performs stateless compaction over the current message list. Service-managed sessions are skipped because the service owns history.

Types

type ContextProviderConfig

type ContextProviderConfig struct {
	// Strategy is the compaction strategy to apply before each agent run.
	Strategy Strategy

	// SourceID identifies this provider in the agent context pipeline.
	// When empty, a default compaction provider source ID is used.
	SourceID string

	// StateKey identifies where provider state is stored in the session.
	// When empty, SourceID is used.
	StateKey string

	// TokenCounter computes token counts for message groups.
	// When nil, token counts are estimated from UTF-8 byte counts.
	TokenCounter TokenCounter

	// Logger emits provider diagnostics when set.
	Logger *slog.Logger
}

ContextProviderConfig configures a compaction context provider.

type ContextWindowStrategy

type ContextWindowStrategy struct {
	// MaxContextWindowTokens is the maximum number of tokens the model's context window supports
	// (for example, 1,048,576 for gpt-4.1). Must be positive.
	MaxContextWindowTokens int

	// MaxOutputTokens is the maximum number of output tokens the model can generate per response
	// (for example, 32,768 for gpt-4.1). Must be non-negative and less than MaxContextWindowTokens.
	MaxOutputTokens int

	// ToolEvictionThreshold is the fraction of the input budget at which tool-result eviction
	// triggers. Must be in (0.0, 1.0]. Zero uses the default (0.5).
	ToolEvictionThreshold float64

	// TruncationThreshold is the fraction of the input budget at which truncation triggers.
	// Must be in (0.0, 1.0] and >= ToolEvictionThreshold. Zero uses the default (0.8).
	TruncationThreshold float64
}

ContextWindowStrategy is a compaction strategy that derives token thresholds from a model's context window size and maximum output tokens, applying a two-phase pipeline:

  1. Tool-result eviction (ToolResultStrategy) — collapses old tool-call groups into concise summaries when the token count exceeds ToolEvictionThreshold × InputBudget.
  2. Truncation (TruncationStrategy) — removes the oldest non-system message groups when the token count exceeds TruncationThreshold × InputBudget.

The input budget is MaxContextWindowTokens - MaxOutputTokens, representing the tokens available for conversation input (system messages, tools, and history).

This is a convenience wrapper around PipelineStrategy that automates threshold calculation from model specifications.

func (*ContextWindowStrategy) Compact

func (s *ContextWindowStrategy) Compact(ctx context.Context, index *MessageIndex) (bool, error)

Compact compacts index in place using a two-phase tool-result eviction and truncation pipeline derived from the model's context window and output token limits.

type GroupKind

type GroupKind int

GroupKind identifies the role a message group plays during compaction.

const (
	// GroupKindSystem contains one or more system messages.
	GroupKindSystem GroupKind = iota
	// GroupKindUser contains a single user message.
	GroupKindUser
	// GroupKindAssistantText contains a single assistant text response without tool calls.
	GroupKindAssistantText
	// GroupKindToolCall contains an assistant tool call and its matching tool result messages.
	GroupKindToolCall
	// GroupKindSummary contains a summary message produced by compaction.
	GroupKindSummary
)

type MessageGroup

type MessageGroup struct {
	// Kind is the kind of this message group.
	Kind GroupKind

	// Messages contains the messages in this group.
	Messages []*message.Message

	// MessageCount is the number of messages in this group.
	MessageCount int

	// ByteCount is the total UTF-8 byte count of this group's message content.
	ByteCount int

	// TokenCount is the estimated or counted token count for this group's messages.
	TokenCount int

	// TurnIndex identifies the user turn this group belongs to.
	//
	// System groups have a nil turn index. A turn starts with a user group and includes subsequent
	// non-user, non-system groups until the next user group or end of conversation.
	TurnIndex *int `json:",omitzero"`

	// IsExcluded indicates whether this group is omitted from the projected message list.
	IsExcluded bool

	// ExcludeReason optionally explains why this group was excluded.
	ExcludeReason string `json:",omitzero"`
}

MessageGroup represents a logical group of messages that must be kept or removed together.

Groups preserve atomic relationships such as an assistant tool call and its corresponding tool result messages. They can be marked as excluded so compacted projections omit them while the original grouped history remains available for diagnostics, storage, or later re-inclusion.

type MessageIndex

type MessageIndex struct {
	// Groups is the ordered list of message groups in the index.
	Groups []*MessageGroup

	// TokenCounter is used to compute token counts for newly created groups.
	// When nil, token counts are estimated from byte counts.
	TokenCounter TokenCounter `json:"-"`
	// contains filtered or unexported fields
}

MessageIndex groups a flat message list into atomic units and tracks compaction metrics.

Groups may be marked as excluded without being removed, allowing strategies to project a compacted message list while preserving the original grouped history for diagnostics and storage. Metrics are available for all groups and for the included, non-excluded subset.

func CreateMessageIndex

func CreateMessageIndex(messages []*message.Message, tokenCounter TokenCounter) *MessageIndex

CreateMessageIndex creates a message index from a flat message list.

The grouping algorithm preserves system messages, user turns, assistant text, summaries, and assistant tool-call/result pairs as logical groups.

func NewMessageIndex

func NewMessageIndex(groups []*MessageGroup, tokenCounter TokenCounter) *MessageIndex

NewMessageIndex creates a message index from pre-built groups.

The index restores its turn counter and last processed message from the provided groups so it can continue incremental updates when new messages are appended.

func (*MessageIndex) AddGroup

func (index *MessageIndex) AddGroup(kind GroupKind, messages []*message.Message, turnIndex *int) *MessageGroup

AddGroup creates and appends a group to the end of the index.

func (*MessageIndex) AllMessages

func (index *MessageIndex) AllMessages() []*message.Message

AllMessages returns messages from all groups, including excluded groups.

func (*MessageIndex) IncludedByteCount

func (index *MessageIndex) IncludedByteCount() int

IncludedByteCount returns the UTF-8 byte count across non-excluded groups.

func (*MessageIndex) IncludedGroupCount

func (index *MessageIndex) IncludedGroupCount() int

IncludedGroupCount returns the number of groups that are not excluded.

func (*MessageIndex) IncludedMessageCount

func (index *MessageIndex) IncludedMessageCount() int

IncludedMessageCount returns the number of messages across non-excluded groups.

func (*MessageIndex) IncludedMessages

func (index *MessageIndex) IncludedMessages() []*message.Message

IncludedMessages returns messages from groups that are not excluded.

func (*MessageIndex) IncludedNonSystemGroupCount

func (index *MessageIndex) IncludedNonSystemGroupCount() int

IncludedNonSystemGroupCount returns the number of non-system groups that are not excluded.

func (*MessageIndex) IncludedTokenCount

func (index *MessageIndex) IncludedTokenCount() int

IncludedTokenCount returns the token count across non-excluded groups.

func (*MessageIndex) IncludedTurnCount

func (index *MessageIndex) IncludedTurnCount() int

IncludedTurnCount returns the number of user turns with at least one non-excluded group.

func (*MessageIndex) InsertGroup

func (index *MessageIndex) InsertGroup(at int, kind GroupKind, messages []*message.Message, turnIndex *int) *MessageGroup

InsertGroup creates and inserts a group at the specified index.

func (*MessageIndex) RawMessageCount

func (index *MessageIndex) RawMessageCount() int

RawMessageCount returns the number of original messages represented by the index.

Summary groups are excluded from this count because they are generated during compaction.

func (*MessageIndex) TotalByteCount

func (index *MessageIndex) TotalByteCount() int

TotalByteCount returns the UTF-8 byte count across all groups, including excluded groups.

func (*MessageIndex) TotalGroupCount

func (index *MessageIndex) TotalGroupCount() int

TotalGroupCount returns the number of groups, including excluded groups.

func (*MessageIndex) TotalMessageCount

func (index *MessageIndex) TotalMessageCount() int

TotalMessageCount returns the number of messages across all groups, including excluded groups.

func (*MessageIndex) TotalTokenCount

func (index *MessageIndex) TotalTokenCount() int

TotalTokenCount returns the token count across all groups, including excluded groups.

func (*MessageIndex) TotalTurnCount

func (index *MessageIndex) TotalTurnCount() int

TotalTurnCount returns the number of user turns across all groups.

func (*MessageIndex) TurnGroups

func (index *MessageIndex) TurnGroups(turnIndex int) []*MessageGroup

TurnGroups returns all groups that belong to the specified user turn.

func (*MessageIndex) Update

func (index *MessageIndex) Update(messages []*message.Message)

Update incrementally appends new messages or rebuilds the index when the existing prefix changed.

Existing groups and exclusion state are preserved when the previous last processed message is still present. If the message list was replaced or trimmed before that point, the index is rebuilt.

type PipelineStrategy

type PipelineStrategy struct {
	// Strategies is the ordered sequence of strategies to execute.
	Strategies []Strategy
}

PipelineStrategy executes strategies sequentially against the same index.

Each strategy operates on the result of the previous one, enabling composed behaviors such as summarizing older messages and then truncating to fit a budget.

func (*PipelineStrategy) Compact

func (strategy *PipelineStrategy) Compact(ctx context.Context, index *MessageIndex) (bool, error)

Compact compacts index in place.

type SlidingWindowStrategy

type SlidingWindowStrategy struct {
	// Trigger controls whether sliding-window compaction should run.
	// When nil, compaction always runs.
	Trigger Trigger

	// Target controls when compaction stops after each excluded turn.
	// When nil, compaction stops when Trigger would no longer fire.
	Target Trigger

	// MinimumPreservedTurns is the minimum number of most-recent user turns to preserve.
	// Groups with nil or non-positive turn indexes are preserved independently of this value.
	//
	// When nil, a default floor is used. An explicit value is honored as-is, so a pointer to 0
	// disables the floor entirely; a negative value is clamped to 0.
	MinimumPreservedTurns *int
}

SlidingWindowStrategy excludes the oldest user turns while preserving recent turns.

System messages are always preserved. This strategy operates on logical turn boundaries rather than token estimates, making it predictable for bounding conversation length.

func (*SlidingWindowStrategy) Compact

func (strategy *SlidingWindowStrategy) Compact(_ context.Context, index *MessageIndex) (bool, error)

Compact compacts index in place.

type Strategy

type Strategy interface {
	// Compact applies strategy-specific compaction to index.
	//
	// It returns true when the strategy changed the index. Implementations should honor ctx for
	// cancellation when they perform blocking work.
	Compact(context.Context, *MessageIndex) (bool, error)
}

Strategy compacts a message index to reduce context size.

Strategies mutate the provided index in place by marking groups as excluded or inserting compact replacement groups such as summaries.

type SummarizationStrategy

type SummarizationStrategy struct {
	// Trigger controls whether summarization should run.
	// When nil, summarization always runs.
	Trigger Trigger

	// Target controls when summarization stops marking groups after each exclusion.
	// When nil, summarization stops when Trigger would no longer fire.
	Target Trigger

	// Summarizer generates the replacement summary text.
	// When nil, the strategy performs no compaction.
	Summarizer Summarizer

	// MinimumPreservedGroups is the minimum number of most-recent non-system groups to preserve.
	// This is a hard floor; summarization will not summarize groups within this protected window.
	//
	// When nil, a default floor is used. An explicit value is honored as-is, so a pointer to 0
	// disables the floor entirely; a negative value is clamped to 0.
	MinimumPreservedGroups *int

	// SummarizationPrompt is the system prompt prepended to messages sent to Summarizer.
	// When nil, a default prompt is used.
	SummarizationPrompt *string

	// SummaryUnavailableMessage is used when Summarizer returns only whitespace.
	// When empty, a default unavailable message is used.
	SummaryUnavailableMessage string
}

SummarizationStrategy summarizes older groups into a single assistant summary message.

The strategy protects system messages and the most recent non-system groups. Older groups are sent to Summarizer, and the resulting summary is inserted as a GroupKindSummary message. Summarizer failures are best effort: excluded groups are restored and Compact reports no change. Context cancellation and deadline errors are restored and returned to the caller.

func (*SummarizationStrategy) Compact

func (strategy *SummarizationStrategy) Compact(ctx context.Context, index *MessageIndex) (bool, error)

Compact compacts index in place.

type Summarizer

type Summarizer interface {
	// Summarize returns a textual summary of messages.
	Summarize(context.Context, []*message.Message) (string, error)
}

Summarizer generates a summary from messages selected by a summarization strategy.

type SummarizerFunc

type SummarizerFunc func(context.Context, []*message.Message) (string, error)

SummarizerFunc adapts a function to Summarizer.

func (SummarizerFunc) Summarize

func (f SummarizerFunc) Summarize(ctx context.Context, messages []*message.Message) (string, error)

Summarize calls f(ctx, messages).

type TokenCounter

type TokenCounter interface {
	// CountTokens returns the token count for text content.
	CountTokens(string) int
}

TokenCounter counts tokens for text-bearing content.

type ToolResultStrategy

type ToolResultStrategy struct {
	// Trigger controls whether tool-result compaction should run.
	// When nil, compaction always runs.
	Trigger Trigger

	// Target controls when compaction stops after each collapsed tool-call group.
	// When nil, compaction stops when Trigger would no longer fire.
	Target Trigger

	// MinimumPreservedGroups is the minimum number of most-recent non-system groups to preserve.
	// This is a hard floor; tool-call groups within this protected window are not collapsed.
	//
	// When nil, a default floor is used. An explicit value is honored as-is, so a pointer to 0
	// disables the floor entirely; a negative value is clamped to 0.
	MinimumPreservedGroups *int

	// ToolCallFormatter formats a tool-call group as a compact summary string.
	// When nil, DefaultToolCallFormatter is used, which produces a YAML-like block listing
	// each tool name and its results.
	ToolCallFormatter func(*MessageGroup) string
}

ToolResultStrategy collapses old tool-call groups into assistant summary messages.

This strategy preserves user messages and plain assistant responses. It only targets tool-call groups outside the protected recent window and replaces each with a concise assistant summary.

func (*ToolResultStrategy) Compact

func (strategy *ToolResultStrategy) Compact(_ context.Context, index *MessageIndex) (bool, error)

Compact compacts index in place.

type Trigger

type Trigger func(*MessageIndex) bool

Trigger defines a predicate over message-index metrics used to start or stop compaction.

Triggers are used both to decide whether a strategy should run and, when used as targets, to decide when incremental compaction has reduced the index enough.

func All

func All(triggers ...Trigger) Trigger

All returns a compound trigger that fires only when every trigger fires.

func Always

func Always() Trigger

Always returns a trigger that always fires regardless of index state.

func Any

func Any(triggers ...Trigger) Trigger

Any returns a compound trigger that fires when at least one trigger fires.

func GroupsExceed

func GroupsExceed(maxGroups int) Trigger

GroupsExceed returns a trigger that fires when the included group count exceeds maxGroups.

func HasToolCalls

func HasToolCalls() Trigger

HasToolCalls returns a trigger that fires when the included index contains a tool-call group.

func MessagesExceed

func MessagesExceed(maxMessages int) Trigger

MessagesExceed returns a trigger that fires when the included message count exceeds maxMessages.

func Never

func Never() Trigger

Never returns a trigger that never fires regardless of index state.

func TokensBelow

func TokensBelow(maxTokens int) Trigger

TokensBelow returns a trigger that fires when the included token count is below maxTokens.

func TokensExceed

func TokensExceed(maxTokens int) Trigger

TokensExceed returns a trigger that fires when the included token count exceeds maxTokens.

func TurnsExceed

func TurnsExceed(maxTurns int) Trigger

TurnsExceed returns a trigger that fires when the included user turn count exceeds maxTurns.

A user turn starts with a user group and includes subsequent non-user, non-system groups until the next user group or end of conversation.

type TruncationStrategy

type TruncationStrategy struct {
	// Trigger controls whether truncation should run.
	// When nil, truncation always runs.
	Trigger Trigger

	// Target controls when truncation stops after each exclusion.
	// When nil, truncation stops when Trigger would no longer fire.
	Target Trigger

	// MinimumPreservedGroups is the minimum number of most-recent non-system groups to preserve.
	// This is a hard floor; truncation will not remove groups beyond this limit.
	//
	// When nil, a default floor is used. An explicit value is honored as-is, so a pointer to 0
	// disables the floor entirely; a negative value is clamped to 0.
	MinimumPreservedGroups *int
}

TruncationStrategy excludes the oldest non-system groups while preserving recent groups.

System messages are always preserved. The strategy respects group boundaries, so tool-call groups are removed as atomic units instead of separating assistant calls from tool results.

func (*TruncationStrategy) Compact

func (strategy *TruncationStrategy) Compact(_ context.Context, index *MessageIndex) (bool, error)

Compact compacts index in place.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL