compactionfacts

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 5, 2026 License: Apache-2.0 Imports: 11 Imported by: 0

Documentation

Overview

Package compactionfacts provides protocol-neutral semantic facts extraction, deterministic item hashing, start-rule matching, and streaming fact builder primitives for compaction recognition without prompt retention.

The package serves as the single internal source of truth for compaction facts shared between the canonical compaction detector (internal/infra/compactiondetect), execution runtime, and future streaming frontend proof compilation workers.

Index

Constants

View Source 
const (
	// ItemHashSizeBytes is the memory size in bytes of one SHA-256 item hash (32 bytes).
	ItemHashSizeBytes = 32

	// DefaultMaxFactItems bounds the maximum number of retained item hashes for wire ingestion
	// (4096 items * 32 bytes/item = 128 KiB fact memory budget).
	DefaultMaxFactItems = 4096
	// HeuristicTailItems is the number of recent semantic tail fingerprints.
	HeuristicTailItems = 2
	// HeuristicPrefixItems is the number of leading item hashes folded into PrefixHash.
	HeuristicPrefixItems = 8
)
View Source 
const (
	EvidenceProtocolStrict   = compaction.EvidenceProtocolStrict
	EvidenceSignatureStrict  = compaction.EvidenceSignatureStrict
	EvidenceHistoryHeuristic = compaction.EvidenceHistoryHeuristic
)
View Source 
const (
	MarkerCodexCheckpoint     = "context checkpoint compaction"
	MarkerConversationTag     = "<conversation>"
	MarkerPiSummaryCarrier    = "<summary>"
	MarkerClineSummaryPost    = "context summary:"
	MarkerSystemNotice        = "<system_notice>"
	MarkerClineCompactedPost  = "earlier context was compacted"
	MarkerOpenCodeHistoryHead = "the following is the conversation history:"
	MarkerHermesRefOnly       = "[context compaction \u2014 reference only]"
	MarkerHermesLegacySummary = "[context summary]:"
	MarkerKiloObjective       = "objective"
	MarkerKiloDetails         = "important details"
	MarkerKiloWorkState       = "work state"
	MarkerKiloNextMove        = "next move"
	MarkerKiloFiles           = "relevant files"
	MarkerAiderUserTag        = "# user"
	MarkerAiderAssistantTag   = "# assistant"
)

Shared compaction rule markers.

View Source 
const (
	RuleProtocolContextCompaction       = "protocol.context_compaction.v1"
	RuleCodexLocalCheckpoint            = "codex.local_checkpoint.v1"
	RulePiOpenClawCompactionSummary     = "pi_openclaw.compaction_summary.v1"
	RuleClineAgenticCompaction          = "cline.agentic_compaction.v1"
	RuleClineBasicCompactionPost        = "cline.basic_compaction_post.v1"
	RuleOpenCodeAnchoredSummary         = "opencode.anchored_summary.v1"
	RuleOpenCodeCustomCompactionHistory = "opencode.custom_compaction_history.v1"
	RuleHermesLocalCompactionPost       = "hermes.local_compaction_post.v1"
	RuleHermesLegacyCompactionPost      = "hermes.legacy_compaction_post.v1"
	RuleKiloCodeAnchoredSummary         = "kilocode.anchored_summary.v1"
	RuleClaudeCodeCompaction            = "claude_code_2026_03.compaction.v1"
	RuleGeminiCLIStateSnapshot          = "gemini_cli.state_snapshot.v1"
	RuleRooCodeCondense                 = "roo_code.condense.v1"
	RuleAiderChatSummary                = "aider.chat_summary.v1"
	RuleCrushSessionSummary             = "crush.session_summary.v1"
	RuleHeuristicCompaction             = "local.compaction_heuristic.v1"
)

Shared compaction rule IDs.

Variables

View Source 
var ErrFactBudgetExceeded = errors.New("compactionfacts: item count exceeds fact budget")

ErrFactBudgetExceeded is returned when the retained facts (item hashes) exceed the configured budget.

View Source 
var StartRules = []StartRule{
	{
		ID:       RuleProtocolContextCompaction,
		Mode:     RuleModeSingle,
		Evidence: EvidenceProtocolStrict,
		Match: func(op lipapi.Operation, _ int, _ func(string) bool) bool {
			return op == lipapi.OperationContextCompaction
		},
	},
	{
		ID:       RuleCodexLocalCheckpoint,
		Mode:     RuleModeSingle,
		Evidence: EvidenceSignatureStrict,
		Match: func(_ lipapi.Operation, _ int, hasText func(string) bool) bool {
			return hasText(MarkerCodexCheckpoint)
		},
	},
	{
		ID:       RulePiOpenClawCompactionSummary,
		Mode:     RuleModeSeries,
		Evidence: EvidenceSignatureStrict,
		Match: func(_ lipapi.Operation, _ int, hasText func(string) bool) bool {
			return hasText(MarkerConversationTag) && hasText("summar") && hasText("checkpoint")
		},
	},
	{
		ID:       RuleClineAgenticCompaction,
		Mode:     RuleModeSingle,
		Evidence: EvidenceSignatureStrict,
		Match: func(_ lipapi.Operation, _ int, hasText func(string) bool) bool {
			return hasText("continuation-note") && hasText("summar")
		},
	},
	{
		ID:       RuleOpenCodeAnchoredSummary,
		Mode:     RuleModeSingle,
		Evidence: EvidenceSignatureStrict,
		Match: func(_ lipapi.Operation, _ int, hasText func(string) bool) bool {
			return hasText(MarkerConversationTag) && hasText("summar") &&
				hasText(MarkerKiloObjective) && hasText(MarkerKiloWorkState)
		},
	},
	{
		ID:       RuleOpenCodeCustomCompactionHistory,
		Mode:     RuleModeSingle,
		Evidence: EvidenceSignatureStrict,
		Match: func(_ lipapi.Operation, _ int, hasText func(string) bool) bool {
			return hasText(MarkerOpenCodeHistoryHead)
		},
	},
	{
		ID:       RuleKiloCodeAnchoredSummary,
		Mode:     RuleModeSingle,
		Evidence: EvidenceSignatureStrict,
		Match: func(_ lipapi.Operation, _ int, hasText func(string) bool) bool {
			return hasText(MarkerKiloObjective) && hasText(MarkerKiloDetails) &&
				hasText(MarkerKiloWorkState) && hasText(MarkerKiloNextMove) &&
				hasText(MarkerKiloFiles)
		},
	},
	{
		ID:       RuleClaudeCodeCompaction,
		Mode:     RuleModeSingle,
		Evidence: EvidenceSignatureStrict,
		Match: func(_ lipapi.Operation, toolCount int, hasText func(string) bool) bool {
			return toolCount == 0 && hasText("text only") && hasText("compaction") && hasText("conversation")
		},
	},
	{
		ID:       RuleGeminiCLIStateSnapshot,
		Mode:     RuleModeSeries,
		Evidence: EvidenceSignatureStrict,
		Match: func(_ lipapi.Operation, _ int, hasText func(string) bool) bool {
			if !hasText("state snapshot") {
				return false
			}
			return hasText("generate") || hasText("verify")
		},
	},
	{
		ID:       RuleRooCodeCondense,
		Mode:     RuleModeSingle,
		Evidence: EvidenceSignatureStrict,
		Match: func(_ lipapi.Operation, _ int, hasText func(string) bool) bool {
			return hasText("condense") && hasText("summar") && hasText("conversation")
		},
	},
	{
		ID:       RuleAiderChatSummary,
		Mode:     RuleModeSeries,
		Evidence: EvidenceSignatureStrict,
		Match: func(_ lipapi.Operation, _ int, hasText func(string) bool) bool {
			return hasText(MarkerAiderUserTag) && hasText(MarkerAiderAssistantTag) && hasText("summar")
		},
	},
	{
		ID:       RuleCrushSessionSummary,
		Mode:     RuleModeSingle,
		Evidence: EvidenceSignatureStrict,
		Match: func(_ lipapi.Operation, _ int, hasText func(string) bool) bool {
			return hasText("session summary") && hasText("preserve") && hasText("context")
		},
	},
}

StartRules is the ordered table of compaction start rules. The protocol rule must stay first so canonical semantics dominate text signatures.

Functions

func Digest 

func Digest(f RequestFacts, complete bool) [32]byte

Digest computes a deterministic 32-byte SHA-256 digest over ALL behavior-relevant compaction fact fields and the completion status.

func EstimateTokens 

func EstimateTokens(call lipapi.Call) int

EstimateTokens calculates characters/4 from all WalkCallTexts sources.

func HashItem 

func HashItem(it lipapi.Item) [32]byte

HashItem computes the SHA-256 semantic hash of a single canonical lipapi.Item.

func WriteField 

func WriteField(w io.Writer, name, val string)

WriteField writes a length-prefixed key-value field to w: name + "=" + strconv.Itoa(len(val)) + ":" + val + "|"

func WriteFieldHeader 

func WriteFieldHeader(w io.Writer, name string, valLen int)

WriteFieldHeader writes the length-prefixed header for a field: name + "=" + strconv.Itoa(valLen) + ":"

func WriteFieldTrailer 

func WriteFieldTrailer(w io.Writer)

WriteFieldTrailer writes the field delimiter "|".

func WriteItemCanonical 

func WriteItemCanonical(w io.Writer, it lipapi.Item)

WriteItemCanonical streams the deterministic canonical representation of an Item directly into w, matching the exact format required for semantic hashing without allocating intermediate strings.Builder payload copies.

Types

type Builder 

type Builder struct {
	// contains filtered or unexported fields
}

Builder accumulates bounded semantic facts across incremental decoded fragments.

func NewBuilder 

func NewBuilder(op lipapi.Operation, maxFactItems int) *Builder

NewBuilder creates a new facts Builder. maxFactItems bounds the number of retained ItemHashes.

  • If maxFactItems > 0: strictly bounds retained item hashes. When exceeded, the builder sets a sticky error (ErrFactBudgetExceeded) and Build returns empty facts and the sticky error. The bound is checked BEFORE item-hash allocation.
  • If maxFactItems <= 0: unbounded (retains all item hashes without limit), preserving canonical exact semantics for arbitrary item counts.

func NewBuilderWithByteBudget 

func NewBuilderWithByteBudget(op lipapi.Operation, maxFactBytes int) *Builder

NewBuilderWithByteBudget creates a Builder whose item hash budget is derived from a byte budget: maxFactItems = maxFactBytes / ItemHashSizeBytes. If maxFactBytes <= 0, the builder is unbounded.

func (*Builder) AddItem 

func (b *Builder) AddItem(it lipapi.Item) error

AddItem hashes a complete lipapi.Item and feeds its text fields into the text matcher and token counter with proper field boundaries. If the budget is exceeded, it sets and returns a sticky ErrFactBudgetExceeded BEFORE hashing or allocating.

func (*Builder) AddItemHash 

func (b *Builder) AddItemHash(h [32]byte) error

AddItemHash appends a precomputed item hash. If the budget is exceeded, it sets and returns a sticky ErrFactBudgetExceeded BEFORE allocating or appending.

func (*Builder) AddToolCount 

func (b *Builder) AddToolCount(n int)

AddToolCount increments the tool count for the request.

func (*Builder) Build 

func (b *Builder) Build() (RequestFacts, error)

Build produces the final bounded RequestFacts. If the builder encountered an error (such as ErrFactBudgetExceeded), it returns an empty RequestFacts and the sticky error.

func (*Builder) EndField 

func (b *Builder) EndField()

EndField ends the current text field, flushing any pending UTF-8 bytes and inserting the canonical '\n' field boundary.

func (*Builder) FeedField 

func (b *Builder) FeedField(text string)

FeedField feeds an entire field and ends it with a canonical field boundary.

func (*Builder) FeedTextChunk 

func (b *Builder) FeedTextChunk(chunk string)

FeedTextChunk feeds a chunk of text within the current field into the bounded text matcher and accumulates its UTF-8 rune count towards EstimatedTokens.

func (*Builder) FeedTextChunkBytes 

func (b *Builder) FeedTextChunkBytes(chunk []byte)

FeedTextChunkBytes feeds a chunk of byte text within the current field into the bounded text matcher and accumulates its UTF-8 rune count towards EstimatedTokens.

type Evidence 

type Evidence = compaction.Evidence

Evidence defines evidence levels matching canonical compaction semantics.

type ItemHasher 

type ItemHasher struct {
	// contains filtered or unexported fields
}

ItemHasher computes an item hash incrementally by streaming fields and chunks.

func NewItemHasher 

func NewItemHasher(kind lipapi.ItemKind, role lipapi.Role) *ItemHasher

NewItemHasher starts hashing an item with the given kind and role.

func (*ItemHasher) AddContentPart 

func (ih *ItemHasher) AddContentPart(kind lipapi.ContentPartKind, text, refusal, summary string)

AddContentPart records a content part on the item.

func (*ItemHasher) AddToolCall 

func (ih *ItemHasher) AddToolCall(name string)

AddToolCall records a tool call on the item.

func (*ItemHasher) AddToolResult 

func (ih *ItemHasher) AddToolResult(output string)

AddToolResult records a tool result on the item.

func (*ItemHasher) BeginContentText 

func (ih *ItemHasher) BeginContentText(valLen int) io.Writer

BeginContentText begins streaming a ContentPartText with a known byte length. It writes "content.kind=4:text|" and the header "content.text=<valLen>:". The returned io.Writer streams into the hash. Call EndContentText() after writing exactly valLen bytes to complete the part.

func (*ItemHasher) EndContentText 

func (ih *ItemHasher) EndContentText()

EndContentText closes the content.text field and emits trailing empty refusal and summary fields.

func (*ItemHasher) Sum 

func (ih *ItemHasher) Sum() [32]byte

Sum returns the final 32-byte SHA-256 hash.

func (*ItemHasher) Writer 

func (ih *ItemHasher) Writer() io.Writer

Writer returns an io.Writer to stream payload chunks directly into the item hash.

type RequestFacts 

type RequestFacts struct {
	Operation       lipapi.Operation
	ToolCount       int
	EstimatedTokens int
	ItemCount       int
	ItemHashes      [][32]byte
	TailHashes      [HeuristicTailItems][32]byte
	TailLen         int
	PrefixHash      [32]byte
	PrefixItems     int

	StartRuleMatched  bool
	StartRuleID       string
	StartRuleMode     RuleMode
	StartRuleEvidence Evidence
}

RequestFacts holds bounded semantic facts extracted from a request for compaction recognition without retaining prompt or payload content.

func ExtractFactsFromCall 

func ExtractFactsFromCall(call lipapi.Call) RequestFacts

ExtractFactsFromCall extracts exact bounded RequestFacts from a canonical Call. It preserves ALL WalkCallTexts sources with canonical '\n' field boundaries, and retains full exact semantics for arbitrary item counts without silent truncation.

func (RequestFacts) Clone 

func (f RequestFacts) Clone() RequestFacts

Clone returns an owned deep copy of RequestFacts, ensuring that mutations to ItemHashes in the clone or original do not affect each other.

type RuleMode 

type RuleMode int

RuleMode declares the transaction behavior of a compaction rule. 

const (
	// RuleModeSingle: the first matching opened request emits a start; a strict
	// later completion closes it; otherwise the transaction closes silently.
	RuleModeSingle RuleMode = iota
	// RuleModeSeries: later matching utility subcalls reuse one active transaction
	// and suppress repeated starts; the first strict/heuristic completion closes.
	RuleModeSeries
	// RuleModeCompletionOnly: no start is ever emitted; a post marker or heuristic
	// creates one completed event transaction.
	RuleModeCompletionOnly
)

type StartRule 

type StartRule struct {
	ID       string
	Mode     RuleMode
	Evidence Evidence
	Match    func(op lipapi.Operation, toolCount int, hasText func(string) bool) bool
}

StartRule defines one compaction request start rule.

func MatchStartRule 

func MatchStartRule(op lipapi.Operation, toolCount int, hasText func(string) bool) (StartRule, bool)

MatchStartRule evaluates the ordered compaction start-rule table against request facts.

type TextMatcher 

type TextMatcher struct {
	// contains filtered or unexported fields
}

TextMatcher performs bounded substring matching for start-rule markers across streamed chunks of text without buffering full request text. It distinguishes chunks within ONE field (carrying overlap and incomplete UTF-8 bytes) from different fields (separated by the canonical '\n' field boundary).

func NewTextMatcher 

func NewTextMatcher() *TextMatcher

NewTextMatcher constructs an empty bounded text matcher.

func (*TextMatcher) EndField 

func (tm *TextMatcher) EndField() int

EndField ends the current text field. It flushes any pending UTF-8 bytes and feeds the canonical '\n' separator, matching base collectCallText where each nonempty field is joined with a newline. This ensures markers cannot match across distinct field boundaries.

func (*TextMatcher) Feed 

func (tm *TextMatcher) Feed(chunk string)

Feed is an alias for FeedChunk for backward compatibility with chunk streaming.

func (*TextMatcher) FeedBytesChunk 

func (tm *TextMatcher) FeedBytesChunk(chunk []byte) int

FeedBytesChunk updates the matcher with a new byte slice within the CURRENT field. It carries overlap and incomplete UTF-8 bytes across chunks within this field, and returns the number of complete UTF-8 runes processed in this chunk.

func (*TextMatcher) FeedChunk 

func (tm *TextMatcher) FeedChunk(chunk string) int

FeedChunk updates the matcher with a new chunk of text within the CURRENT field. It carries overlap and incomplete UTF-8 bytes across chunks within this field, and returns the number of complete UTF-8 runes processed in this chunk.

func (*TextMatcher) FeedField 

func (tm *TextMatcher) FeedField(text string) int

FeedField feeds an entire field, immediately inserting the canonical field boundary.

func (*TextMatcher) HasText 

func (tm *TextMatcher) HasText(marker string) bool

HasText reports whether the specified marker was encountered in fed text.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.