filter

type ACONFilter ¶

type ACONFilter struct {
	// contains filtered or unexported fields
}

Paper: "ACON: Optimizing Context Compression for Long-Context LLMs" — ICLR 2026 ACONFilter implements adaptive context optimization — dynamically adjusts compression based on content complexity and context length.

func NewACONFilter() *ACONFilter

func (f *ACONFilter) Apply(input string, mode Mode) (string, int)

func (f *ACONFilter) Name() string

type ANSICode ¶

type ANSICode struct {
	Position int
	Code     string
}

ANSICode represents an ANSI escape sequence.

type ANSIFilter ¶

type ANSIFilter struct{}

ANSIFilter strips ANSI escape sequences from output. Uses SIMD-optimized byte scanning for ~10-40x speedup over regex.

func NewANSIFilter() *ANSIFilter

func (f *ANSIFilter) Apply(input string, mode Mode) (string, int)

func (f *ANSIFilter) Name() string

type ASTOptimizer ¶

type ASTOptimizer struct {
	// contains filtered or unexported fields
}

ASTOptimizer uses tree-sitter for AST parsing

func NewASTOptimizer() *ASTOptimizer

type ASTPreserveFilter ¶

type ASTPreserveFilter struct {
	// contains filtered or unexported fields
}

Paper: "LongCodeZip" — Shi et al., SJTU/Stanford, 2025 https://arxiv.org/abs/2510.00446 Paper: "LongCodeZip" — Shi et al., SJTU/Stanford, 2025 https://arxiv.org/abs/2510.00446 ASTPreserveFilter implements LongCodeZip-style compression (NUS, 2025). AST-aware compression that preserves syntactic validity of code.

Algorithm: 1. Detect programming language from syntax patterns 2. Parse code structure (brackets, braces, indentation) 3. Apply entropy-based pruning while preserving AST integrity 4. Never break syntactic boundaries (function bodies, blocks, strings)

Enhanced with Dual-Stage LongCodeZip Methodology: - Stage 1: Coarse-Grained (Function-Level) pruning - Stage 2: Fine-Grained (Block-Level) adaptive compression

Research Results: 4-8x compression while maintaining parseable code. LongCodeZip: 5.6x reduction, 16% better accuracy than LLMLingua on code.

func NewASTPreserveFilter() *ASTPreserveFilter

func (f *ASTPreserveFilter) Apply(input string, mode Mode) (string, int)

func (f *ASTPreserveFilter) Name() string

func (f *ASTPreserveFilter) SetQueryIntent(query string)

type AdaptiveBufferSizer ¶

type AdaptiveBufferSizer struct {
	// contains filtered or unexported fields
}

AdaptiveBufferSizer dynamically adjusts buffer sizes

func NewAdaptiveBufferSizer(min, max int) *AdaptiveBufferSizer

func (abs *AdaptiveBufferSizer) Adjust(inputSize int) int

type AdaptiveLayerSelector ¶

type AdaptiveLayerSelector struct {
	// contains filtered or unexported fields
}

AdaptiveLayerSelector dynamically enables/disables layers based on content type. Uses heuristic analysis to optimize compression for different input patterns.

func NewAdaptiveLayerSelector() *AdaptiveLayerSelector

func (a *AdaptiveLayerSelector) AnalyzeContent(input string) ContentType

func (a *AdaptiveLayerSelector) OptimizePipeline(input string, mode Mode) *PipelineCoordinator

func (a *AdaptiveLayerSelector) RecommendedConfig(ct ContentType, mode Mode) PipelineConfig

func (a *AdaptiveLayerSelector) RecommendedConfigWithTiers(ct ContentType, mode Mode, inputLen int, queryIntent string) PipelineConfig

type AdaptiveLearningFilter ¶

type AdaptiveLearningFilter struct {
	// contains filtered or unexported fields
}

AdaptiveLearningFilter combines Layer 50 (EngramLearner) and Layer 51 (TieredSummary) into a single post-processing layer that learns from patterns and generates progressive summaries.

This merged layer: 1. First applies EngramLearner to detect and learn error patterns 2. Then applies TieredSummary for L0/L1/L2 progressive summarization

Research basis: - EngramLearner: Error pattern learning with 14 classifiers - TieredSummary: Progressive summarization (surface → structural → deep)

func NewAdaptiveLearningFilter() *AdaptiveLearningFilter

func (a *AdaptiveLearningFilter) Apply(input string, mode Mode) (string, int)

func (a *AdaptiveLearningFilter) GenerateTiers(input string) *TieredResult

func (a *AdaptiveLearningFilter) GetLearningStats() map[string]interface{}

func (a *AdaptiveLearningFilter) IsEnabled() bool

func (a *AdaptiveLearningFilter) Name() string

func (a *AdaptiveLearningFilter) SetEnabled(enabled bool)

type AdaptivePipeline ¶

type AdaptivePipeline struct {
	// contains filtered or unexported fields
}

AdaptivePipeline selects layers based on content characteristics

func NewAdaptivePipeline(layers []Filter) *AdaptivePipeline

func (ap *AdaptivePipeline) Process(input string) (string, int)

type AdvancedFilter ¶

type AdvancedFilter struct {
	// contains filtered or unexported fields
}

func NewAdvancedFilter() *AdvancedFilter

func (f *AdvancedFilter) Apply(input string, mode Mode) (string, int)

func (f *AdvancedFilter) Name() string

type AgentMemoryConfig ¶

type AgentMemoryConfig struct {
	// KnowledgeRetentionRatio is the ratio of knowledge to keep (0.0-1.0)
	KnowledgeRetentionRatio float64

	// HistoryPruneRatio is the ratio of history to prune after consolidation
	HistoryPruneRatio float64

	// ConsolidationThreshold triggers consolidation when history exceeds this
	ConsolidationThreshold int

	// EnableAutoConsolidation allows autonomous memory management
	EnableAutoConsolidation bool

	// KnowledgeMaxSize limits the knowledge block size
	KnowledgeMaxSize int

	// PreservePatterns are regex patterns for important content to always keep
	PreservePatterns []*regexp.Regexp
}

AgentMemoryConfig holds configuration for agent memory management

func DefaultAgentMemoryConfig() AgentMemoryConfig

type AgentMemoryFilter ¶

type AgentMemoryFilter struct {
	// contains filtered or unexported fields
}

AgentMemoryFilter implements Layer 20: Agent Memory Mode (Focus-inspired).

Research Source: "Active Context Compression / Focus" (arXiv, January 2026) Key Innovation: Agent-centric autonomous memory management inspired by slime mold. Results: 22.7% token reduction (14.9M → 11.5M), 57% savings on individual instances.

Methodology (Physarum polycephalum inspired): 1. Knowledge Consolidation - Extract learnings into "Knowledge" block 2. Active Withdrawal - Prune raw interaction history after consolidation 3. Self-Regulation - Agent decides when to consolidate vs. keep raw

This filter maintains session state and autonomously manages context bloating in long-horizon agent tasks by distinguishing between: - Knowledge: Consolidated insights (high value, permanent) - History: Raw interaction logs (transient, prunable)

func NewAgentMemoryFilter() *AgentMemoryFilter

func NewAgentMemoryFilterWithConfig(cfg AgentMemoryConfig) *AgentMemoryFilter

func (f *AgentMemoryFilter) Apply(input string, mode Mode) (string, int)

func (f *AgentMemoryFilter) GetStats() AgentMemoryStats

func (f *AgentMemoryFilter) Name() string

func (f *AgentMemoryFilter) Reset()

type AgentMemoryLayerConfig ¶

type AgentMemoryLayerConfig struct {
	Enabled            bool
	KnowledgeRetention float64
	HistoryPrune       float64
	ConsolidationMax   int
}

AgentMemoryLayerConfig groups Layer 20 settings.

type AgentMemoryStats ¶

type AgentMemoryStats struct {
	TotalConsolidated int
	TotalPruned       int
	KnowledgeTokens   int
	HistoryTokens     int
	TokensSaved       int
}

AgentMemoryStats tracks memory management statistics

type AgentOCRFilter ¶

type AgentOCRFilter struct {
	// contains filtered or unexported fields
}

Paper: "AgentOCR: Content-Density Aware Compression for Multi-Turn Agent Trajectories" arXiv 2026

AgentOCRFilter operates on multi-turn agent outputs (tool call sequences, conversation turns). It measures the "content density" of each turn — the ratio of information-bearing lines to total lines — and collapses low-density turns into a single summary stub while preserving high-density turns in full.

Content density signals:

• Information-bearing: error lines, code lines, unique-term-rich lines
• Filler: empty lines, repeated terms, pure-whitespace lines

Turn detection: turns are separated by patterns like:

• "Human:", "Assistant:", "User:", "Agent:", "System:"
• "<human>", "<assistant>", markdown "## Turn N"

Density thresholds:

• Low density (<0.30): collapse to "[Turn summary: N lines, M tokens]"
• Medium density (0.30–0.65): keep first+last few lines + omission marker
• High density (>0.65): preserve fully

func NewAgentOCRFilter() *AgentOCRFilter

func (f *AgentOCRFilter) Apply(input string, mode Mode) (string, int)

func (f *AgentOCRFilter) Name() string

type AgentOCRHistoryFilter ¶

type AgentOCRHistoryFilter struct {
	// contains filtered or unexported fields
}

AgentOCRHistoryFilter compacts older conversation turns while preserving recent turns.

func NewAgentOCRHistoryFilter() *AgentOCRHistoryFilter

func (f *AgentOCRHistoryFilter) Apply(input string, mode Mode) (string, int)

func (f *AgentOCRHistoryFilter) Name() string

type AggregateStats ¶

type AggregateStats struct {
	AvgCompression    float64
	MinCompression    float64
	MaxCompression    float64
	StdDevCompression float64
	AvgLatency        float64
	TotalTime         time.Duration
	AvgQuality        float64
}

AggregateStats provides summary statistics.

type AnchorToken ¶

type AnchorToken struct {
	Text         string
	Position     int
	Score        float64
	Aggregated   []string // Tokens aggregated into this anchor
	IsStructural bool
}

AnchorToken represents a semantic anchor point

type ApplicabilityCheck ¶

type ApplicabilityCheck interface {
	IsApplicable(input string) bool
}

ApplicabilityCheck is an optional interface that filters can implement to report whether they should run for a given input. The coordinator calls this before Apply to implement stage gates (skip cheap before expensive).

type AttentionSinkFilter ¶

type AttentionSinkFilter struct {
	// contains filtered or unexported fields
}

Paper: "StreamingLLM: Attention Sinks" — Xiao et al., MIT, 2023 https://arxiv.org/abs/2309.17453 Paper: "StreamingLLM: Attention Sinks" — Xiao et al., MIT, 2023 https://arxiv.org/abs/2309.17453 AttentionSinkFilter implements StreamingLLM-style attention sink preservation. Research basis: "Efficient Streaming Language Models with Attention Sinks" (Xiao et al., 2023) - enables infinite-length generation with bounded memory.

Key insight: The first few tokens in a sequence act as "attention sinks" - they absorb excess attention weight due to softmax normalization. Removing these tokens breaks the attention distribution the model learned during training.

This layer: 1. Always preserves initial tokens (attention sinks) 2. Preserves structural anchors (headers, prefixes, markers) 3. Applies rolling cache to remaining content

This is Layer 14 in the pipeline, ensuring stable compression for long content.

func NewAdaptiveAttentionSinkFilter(outputLines int) *AttentionSinkFilter

func NewAttentionSinkFilter() *AttentionSinkFilter

func (a *AttentionSinkFilter) Apply(input string, mode Mode) (string, int)

func (a *AttentionSinkFilter) GetStats() map[string]any

func (a *AttentionSinkFilter) Name() string

func (a *AttentionSinkFilter) SetEnabled(enabled bool)

type AttentionSinkLayerConfig ¶

type AttentionSinkLayerConfig struct {
	Enabled     bool
	SinkCount   int
	RecentCount int
}

AttentionSinkLayerConfig groups Layer 14 settings.

type AttributionConfig ¶

type AttributionConfig struct {
	// Enable attribution filtering
	Enabled bool

	// Threshold for token importance (0.0-1.0)
	// Tokens below this score are candidates for removal
	ImportanceThreshold float64

	// Minimum content length to apply attribution
	MinContentLength int

	// Use positional bias (later tokens often less important)
	PositionalBias bool

	// Use frequency-based importance (repeated tokens may be less important)
	FrequencyBias bool

	// Use semantic markers (preserve keywords, numbers, code)
	SemanticPreservation bool

	// Maximum tokens to analyze (for performance)
	MaxAnalyzeTokens int
}

AttributionConfig holds configuration for attribution-based pruning

func DefaultAttributionConfig() AttributionConfig

type AttributionFilter ¶

type AttributionFilter struct {
	// contains filtered or unexported fields
}

AttributionFilter implements attribution-based token pruning. Research basis: "ProCut: Progressive Pruning via Attribution" (LinkedIn, 2025) Achieves 78% token reduction by using importance scoring.

Key technique: Attribution scores (simplified SHAP) identify which tokens contribute most to the output. Low-importance tokens are pruned.

This is Layer 12 in the pipeline, adding ML-style importance without requiring actual model training.

func NewAttributionFilter() *AttributionFilter

func (a *AttributionFilter) Apply(input string, mode Mode) (string, int)

func (a *AttributionFilter) GetStats() map[string]any

func (a *AttributionFilter) Name() string

func (a *AttributionFilter) SetEnabled(enabled bool)

type AttributionLayerConfig ¶

type AttributionLayerConfig struct {
	Enabled   bool
	Threshold float64
}

AttributionLayerConfig groups Layer 12 settings.

type AutoTier ¶

type AutoTier int

AutoTier represents an automatic tier for adaptive layer enablement. Tiers are enabled automatically based on content context.

const (
	// AutoTierPre: Pre-processing layers (always run first)
	// Layers: 0 (QuantumLock), 0.5 (Photon)
	AutoTierPre AutoTier = iota

	// AutoTierCore: Foundation compression (high ROI, always enabled)
	// Layers: 1-10 (Entropy, Perplexity, AST, etc.)
	AutoTierCore

	// AutoTierSemantic: Semantic analysis (medium cost, high quality)
	// Layers: 11-20 (Compaction, H2O, Attention Sink, etc.)
	AutoTierSemantic

	// AutoTierAdvanced: Research layers (higher cost, specialized)
	// Layers: 21-40 (DiffAdapt, EPiC, AgentOCR, etc.)
	AutoTierAdvanced

	// AutoTierSpecialized: Experimental/edge cases (enable manually)
	// Layers: 41-50 (ContextCrunch, SearchCrunch, AdaptiveLearning)
	AutoTierSpecialized

	// AutoTierCount: Total number of tiers
	AutoTierCount
)

func QuickTierEnable(useCase string) []AutoTier

type AutoTierConfig ¶

type AutoTierConfig struct {
	Name         string
	Description  string
	LayerRange   [2]int        // [start, end] layer numbers
	Default      bool          // Enabled by default
	AutoEnable   bool          // Can be auto-enabled based on context
	CostLevel    int           // 1=low, 2=medium, 3=high, 4=very high
	MinInputLen  int           // Minimum input length to enable
	ContentTypes []ContentType // Content types this tier is good for
}

AutoTierConfig holds configuration for each tier.

type AutoTierRecommendation ¶

type AutoTierRecommendation struct {
	Tiers      []AutoTier
	Reason     string
	Confidence float64
}

AutoTierRecommendation holds recommended tiers for a context.

func RecommendTiers(contentType ContentType, inputLen int, queryIntent string) AutoTierRecommendation

type BeamSearchPerplexity ¶

type BeamSearchPerplexity struct {
	// contains filtered or unexported fields
}

func NewBeamSearchPerplexity(w int) *BeamSearchPerplexity

type BenchmarkReport ¶

type BenchmarkReport struct {
	Timestamp       time.Time
	TotalTests      int
	Passed          int
	Failed          int
	OverallStats    AggregateStats
	Results         []BenchmarkResult
	Recommendations []string
}

BenchmarkReport aggregates all benchmark results.

type BenchmarkResult ¶

type BenchmarkResult struct {
	TestName           string
	ContentType        string
	OriginalTokens     int
	CompressedTokens   int
	ReductionPct       float64
	CompressionTime    time.Duration
	PerTokenLatency    float64 // microseconds per token
	Reversible         bool
	ReconstructionDiff float64 // 0-1 similarity score
	QualityScore       float64 // 0-1 quality score
	LayerBreakdown     map[string]LayerTiming
}

BenchmarkResult holds results for a single test case.

type BloomFilter ¶

type BloomFilter struct {
	// contains filtered or unexported fields
}

BloomFilter for H2O optimization

func NewBloomFilter(size int) *BloomFilter

func (bf *BloomFilter) Add(item string)

func (bf *BloomFilter) Contains(item string) bool

type BodyFilter ¶

type BodyFilter struct {
	// contains filtered or unexported fields
}

BodyFilter strips function bodies in aggressive mode. Preserves function signatures while removing body content.

func NewBodyFilter() *BodyFilter

func (f *BodyFilter) Apply(input string, mode Mode) (string, int)

func (f *BodyFilter) Name() string

type BudgetAllocator ¶

type BudgetAllocator struct {
	// contains filtered or unexported fields
}

Task 39: Compression budget allocator

func NewBudgetAllocator() *BudgetAllocator

func (b *BudgetAllocator) Allocate(layerID, budget int)

type BudgetConfig ¶

type BudgetConfig struct {
	Budget int // Maximum tokens (0 = unlimited)
}

BudgetConfig holds configuration for the budget enforcer

type BudgetEnforcer ¶

type BudgetEnforcer struct {
	// contains filtered or unexported fields
}

BudgetEnforcer enforces strict token limits on output. Research-based: Budget-Constrained Compression (2024) - provides predictable output size by scoring segments and keeping only the most important ones.

Key insight: LLMs have finite context windows. Enforcing a strict budget ensures output fits within constraints while maximizing information content.

func NewBudgetEnforcer(budget int) *BudgetEnforcer

func NewBudgetEnforcerWithConfig(cfg BudgetConfig) *BudgetEnforcer

func (f *BudgetEnforcer) Apply(input string, mode Mode) (string, int)

func (f *BudgetEnforcer) Name() string

func (f *BudgetEnforcer) SetBudget(budget int)

type BufferPool ¶

type BufferPool struct {
	// contains filtered or unexported fields
}

BufferPool manages reusable buffers

func NewBufferPool(size int) *BufferPool

func (bp *BufferPool) Get() *ZeroCopyBuffer

func (bp *BufferPool) Put(buf *ZeroCopyBuffer)

type BytePool ¶

type BytePool struct {
	// contains filtered or unexported fields
}

func NewBytePool() *BytePool

func (p *BytePool) AcquireStringBuilder(capacity int) *bytes.Buffer

func (p *BytePool) Get(capacity int) *[]byte

func (p *BytePool) Put(b *[]byte)

func (p *BytePool) ReleaseStringBuilder(buf *bytes.Buffer)

type CARL ¶

type CARL struct{}

func (c *CARL) Apply(input string, mode Mode) (string, int)

type CARLFilter ¶

type CARLFilter struct {
	// contains filtered or unexported fields
}

Paper: "CARL: Critical Action Focused Reinforcement Learning for Multi-Step Agent" arXiv:2512.04949 — 2025

CARLFilter identifies "critical" vs "non-critical" tool-call entries in agent output sequences and drops the non-critical ones.

Criticality is defined as: did this action cause an observable state change?

Critical:     error output, file writes/deletes, test failures, non-empty diffs,
              non-zero exit codes, assertion failures, CRUD operations
Non-critical: empty results, successful no-ops, pure info queries,
              repeated identical results, health checks

CARL's key insight (from RL perspective): in a long agent trajectory, most actions are "maintenance" (checking state, listing files, echoing info) and carry no new causal information. Keeping only critical actions and their immediate context preserves the trajectory's causal skeleton at a fraction of the token cost.

func NewCARLFilter() *CARLFilter

func (f *CARLFilter) Apply(input string, mode Mode) (string, int)

func (f *CARLFilter) Name() string

type CRFGoalDriven ¶

type CRFGoalDriven struct {
	// contains filtered or unexported fields
}

Task 37: CRF-based goal-driven selection

func NewCRFGoalDriven() *CRFGoalDriven

func (c *CRFGoalDriven) Score(line string) float64

type CacheBreakpointFilter ¶

type CacheBreakpointFilter struct {
	MinCacheableTokens int
	MaxBreakpoints     int
}

func NewCacheBreakpointFilter() *CacheBreakpointFilter

func (f *CacheBreakpointFilter) Filter(input string) string

type CacheStats ¶

type CacheStats struct {
	Size      int
	MaxSize   int
	Hits      int64
	Misses    int64
	HitRate   float64
	Evictions int64
}

CacheStats holds cache performance statistics.

type CachedPipeline ¶

type CachedPipeline struct {
	// contains filtered or unexported fields
}

CachedPipeline wraps a PipelineCoordinator with caching

func NewCachedPipeline(cfg CachedPipelineConfig) *CachedPipeline

func (cp *CachedPipeline) Close() error

func (cp *CachedPipeline) Process(command string, args []string, getOutput func() (string, error)) (*CachedProcessResult, error)

func (cp *CachedPipeline) Stats() (*cache.CacheStats, error)

type CachedPipelineConfig ¶

type CachedPipelineConfig struct {
	Pipeline   *PipelineCoordinator
	Cache      *cache.QueryCache
	GitWatcher *cache.GitWatcher
	Enabled    bool
}

CachedPipelineConfig for creating a cached pipeline

type CachedProcessResult ¶

type CachedProcessResult struct {
	Output         string
	OriginalOutput string
	Stats          *PipelineStats
	FromCache      bool
	CacheKey       string
	CacheHitCount  int
}

CachedProcessResult contains the result of a cached process

type CachedResult ¶

type CachedResult struct {
	Output   string
	Tokens   int
	CachedAt time.Time
}

CachedResult represents a cached compression result

type Checkpoint ¶

type Checkpoint struct {
	Position int
	Data     string
}

Checkpoint saves compression state

func (c *Checkpoint) Save(pos int, data string)

type ChunkMethod ¶

type ChunkMethod int

ChunkMethod defines how content is split into chunks

const (
	// ChunkAuto auto-detects content type and applies appropriate method
	ChunkAuto ChunkMethod = iota
	// ChunkCode uses code-aware chunking (functions, classes)
	ChunkCode
	// ChunkText uses text-aware chunking (sentences, paragraphs)
	ChunkText
	// ChunkMixed handles mixed code+text content
	ChunkMixed
)

type ChunkType ¶

type ChunkType int

ChunkType identifies the semantic type of a chunk

const (
	ChunkFunction ChunkType = iota
	ChunkClass
	ChunkMethodDef
	ChunkStruct
	ChunkInterface
	ChunkSentence
	ChunkParagraph
	ChunkCodeBlock
	ChunkComment
	ChunkImport
	ChunkOther
)

type CircuitBreaker ¶

type CircuitBreaker struct {
	// contains filtered or unexported fields
}

CircuitBreaker stops processing on quality drop

func NewCircuitBreaker(threshold float64, maxFails int) *CircuitBreaker

func (cb *CircuitBreaker) Check(quality float64) bool

type CoTCompressFilter ¶

type CoTCompressFilter struct {
	// contains filtered or unexported fields
}

Paper: "TokenSkip: Controllable Chain-of-Thought Compression" arXiv:2502.12067 — 2025

CoTCompressFilter detects chain-of-thought reasoning traces in output and applies token-budget-controlled compression:

• ModeMinimal: truncate CoT to first 30% + summary marker
• ModeAggressive: replace entire CoT block with a token-count stub

Applicable when tok wraps tools that emit LLM reasoning output (e.g. claude --verbose, agent traces, reasoning model output).

Patterns detected:

• XML-style: <think>...</think>, <reasoning>...</reasoning>
• Markdown step blocks: "Step 1:", "Let me think", numbered reasoning
• Reflection loops: "Wait,", "Actually,", "Let me reconsider"

func NewCoTCompressFilter() *CoTCompressFilter

func (f *CoTCompressFilter) Apply(input string, mode Mode) (string, int)

func (f *CoTCompressFilter) Name() string

type CoTCompressor ¶

type CoTCompressor struct{}

func (c *CoTCompressor) Apply(input string, mode Mode) (string, int)

type CodeChunk ¶

type CodeChunk struct {
	Type      string // "function", "class", "method", "block"
	Name      string
	Content   string
	StartLine int
	EndLine   int
	Score     float64 // Importance score
	Tokens    int
	Children  []CodeChunk // Nested blocks
}

CodeChunk represents a parsed code unit for dual-stage compression

type CodeContext ¶

type CodeContext struct {
	File    string   `json:"file"`
	Symbols []string `json:"symbols"`
	Lines   string   `json:"lines,omitempty"`
}

CodeContext preserves code-specific context

type CodeGist ¶

type CodeGist struct {
	// contains filtered or unexported fields
}

func NewCodeGist() *CodeGist

type CodingAgentContext ¶

type CodingAgentContext struct{}

func (c *CodingAgentContext) Apply(input string, mode Mode) (string, int)

type CodingAgentContextFilter ¶

type CodingAgentContextFilter struct {
	// contains filtered or unexported fields
}

Paper: "SWE-Pruner: Self-Adaptive Context Pruning for Coding Agents" arXiv:2601.16746 — Wang et al., Shanghai Jiao Tong, 2026

CodingAgentContextFilter specialises context pruning for the structured tool outputs that coding agents (Claude Code, Cursor, etc.) receive: file reads, bash output, search results, git diffs, test output, compile logs.

Unlike the general GoalDrivenFilter (CRF scoring against query terms), this filter is structure-aware: it identifies the output type and applies a type-specific compression strategy, then self-adjusts the compression ratio based on observed output density.

Type-specific strategies:

file_read   — elide unchanged middle sections, keep head+tail
bash_output — keep last N lines (most recent = most relevant)
search_hits — one result per unique file path
git_diff    — keep ±-lines only, drop context lines in aggressive mode
test_output — keep FAIL/PASS summary + failing assertions
compile_log — keep error/warning lines, collapse repeated warnings

func NewCodingAgentContextFilter() *CodingAgentContextFilter

func (f *CodingAgentContextFilter) Apply(input string, mode Mode) (string, int)

func (f *CodingAgentContextFilter) Name() string

type ColorPassthrough ¶

type ColorPassthrough struct {
	// contains filtered or unexported fields
}

ColorPassthrough strips ANSI codes for matching but restores in output. Inspired by tokf's color passthrough.

func (cp *ColorPassthrough) RestoreCodes(stripped string) string

func (cp *ColorPassthrough) StripAndStore(content string) string

type CommentFilter ¶

type CommentFilter struct {
	// contains filtered or unexported fields
}

CommentFilter strips comments from code.

func (f *CommentFilter) Apply(input string, mode Mode) (string, int)

func (f *CommentFilter) Name() string

type CompactionConfig ¶

type CompactionConfig struct {
	// Enable LLM-based compaction
	Enabled bool

	// Minimum content size to trigger compaction (in lines)
	ThresholdLines int

	// Minimum content size to trigger compaction (in tokens)
	ThresholdTokens int

	// Number of recent turns to preserve verbatim
	PreserveRecentTurns int

	// Maximum summary length in tokens
	MaxSummaryTokens int

	// Content types to compact (chat, conversation, session)
	ContentTypes []string

	// Enable caching of compaction results
	CacheEnabled bool

	// Custom prompt template for compaction
	PromptTemplate string

	// Detect content type automatically
	AutoDetect bool

	// Create state snapshot format (4-section XML)
	StateSnapshotFormat bool

	// Extract key-value pairs from content
	ExtractKeyValuePairs bool

	// Maximum context entries to preserve
	MaxContextEntries int
}

CompactionConfig holds configuration for the compaction layer

func DefaultCompactionConfig() CompactionConfig

type CompactionLayer ¶

type CompactionLayer struct {
	// contains filtered or unexported fields
}

Paper: "MemGPT" — Packer et al., UC Berkeley, 2023 https://arxiv.org/abs/2310.08560 CompactionLayer provides semantic compression for chat/conversation content. It creates state snapshots with 4 sections:

1. session_history: user queries + activity log (what was done) 2. current_state: focus + next_action (what's active now) 3. context: critical + working knowledge (what to remember) 4. pending_plan: future milestones (what's next)

Research basis: "MemGPT" (UC Berkeley, 2023) semantic compression achieves 98%+ compression ratios while preserving semantic meaning.

This layer is designed for: - Chat history compression - Conversation-style content - Session state preservation - Multi-turn context management

func NewCompactionLayer(cfg CompactionConfig) *CompactionLayer

func (c *CompactionLayer) Apply(input string, mode Mode) (string, int)

func (c *CompactionLayer) GetStats() map[string]any

func (c *CompactionLayer) IsAvailable() bool

func (c *CompactionLayer) Name() string

func (c *CompactionLayer) SetEnabled(enabled bool)

type CompactionLayerConfig ¶

type CompactionLayerConfig struct {
	Enabled       bool
	Threshold     int
	PreserveTurns int
	MaxTokens     int
	StateSnapshot bool
	AutoDetect    bool
}

CompactionLayerConfig groups Layer 11 settings.

type CompactionResult ¶

type CompactionResult struct {
	Snapshot         *StateSnapshot
	OriginalTokens   int
	FinalTokens      int
	SavedTokens      int
	CompressionRatio float64
	Cached           bool
	Timestamp        time.Time
}

CompactionResult represents a compaction result

func Compact(input string, cfg CompactionConfig) (string, *CompactionResult)

type CompressionMode ¶

type CompressionMode = Tier

type ConfigurableHierarchical ¶

type ConfigurableHierarchical struct {
	// contains filtered or unexported fields
}

func NewConfigurableHierarchical(d int) *ConfigurableHierarchical

type ContentDetector ¶

type ContentDetector struct{}

ContentDetector fast content type detection

func (cd *ContentDetector) Detect(input string) string

type ContentType ¶

type ContentType int

ContentType represents the detected content type

const (
	ContentTypeUnknown ContentType = iota
	ContentTypeCode
	ContentTypeLogs
	ContentTypeConversation
	ContentTypeGitOutput
	ContentTypeTestOutput
	ContentTypeDockerOutput
	ContentTypeMixed
)

func (ct ContentType) String() string

type ContextContentType ¶

type ContextContentType int

ContextContentType represents the detected content type for context crunching.

const (
	ContextContentTypeUnknown ContextContentType = iota
	ContextContentTypeLog
	ContextContentTypeDiff
)

type ContextCrunchFilter ¶

type ContextCrunchFilter struct {
	// contains filtered or unexported fields
}

ContextCrunchFilter combines Layer 46 (LogCrunch) and Layer 48 (DiffCrunch) into a unified context-folding layer that auto-detects content type.

This merged layer: - Auto-detects if input is logs or diffs - Applies appropriate folding strategy - Handles both types with unified logic

func NewContextCrunchFilter() *ContextCrunchFilter

func (c *ContextCrunchFilter) Apply(input string, mode Mode) (string, int)

func (c *ContextCrunchFilter) EstimateTokens(text string) int

func (c *ContextCrunchFilter) Name() string

type ContrastiveFilter ¶

type ContrastiveFilter struct {
	// contains filtered or unexported fields
}

Paper: "LongLLMLingua" — Jiang et al., Microsoft, 2024 https://arxiv.org/abs/2310.06839 Paper: "LongLLMLingua" — Jiang et al., Microsoft, 2024 https://arxiv.org/abs/2310.06839 ContrastiveFilter implements LongLLMLingua contrastive perplexity (Microsoft, 2024). Question-aware compression that ranks tokens by relevance to the query.

Algorithm: 1. Calculate contrastive perplexity: CP(x) = P(x|question) / P(x|context) 2. Higher contrastive perplexity = more question-relevant 3. Reorder context to place high-relevance tokens at start/end 4. Prune low-relevance middle content

Research Results: 4-10x compression with improved RAG accuracy.

func NewContrastiveFilter(question string) *ContrastiveFilter

func (f *ContrastiveFilter) Apply(input string, mode Mode) (string, int)

func (f *ContrastiveFilter) Name() string

func (f *ContrastiveFilter) SetQuestion(question string)

type ConversationDedup ¶

type ConversationDedup struct {
	// contains filtered or unexported fields
}

ConversationDedup deduplicates content across conversation turns using Jaccard similarity.

func NewConversationDedup() *ConversationDedup

func (d *ConversationDedup) DeduplicateMessages(messages []Message) ([]Message, *DedupStats)

type ConversationTracker ¶

type ConversationTracker struct {
	// contains filtered or unexported fields
}

ConversationTracker tracks conversation turns

func NewConversationTracker(maxTurns int) *ConversationTracker

func (t *ConversationTracker) AddTurn(role, content string)

func (t *ConversationTracker) GetRecentTurns(n int) []Turn

func (t *ConversationTracker) GetTurns() []Turn

type CoordinatorPool ¶

type CoordinatorPool struct {
	// contains filtered or unexported fields
}

CoordinatorPool manages reusable pipeline coordinators

func GetDefaultPool() *CoordinatorPool

func NewCoordinatorPool(config PipelineConfig) *CoordinatorPool

func (p *CoordinatorPool) Get() *PipelineCoordinator

func (p *CoordinatorPool) Put(coord *PipelineCoordinator)

type CoreFilters ¶

type CoreFilters struct {
	// contains filtered or unexported fields
}

CoreFilters manages layers 1-9

func NewCoreFilters(cfg PipelineConfig) *CoreFilters

func (c *CoreFilters) Apply(input string, mode Mode, stats *PipelineStats) string

type CoreLayersConfig ¶

type CoreLayersConfig struct {
	LLMEnabled       bool
	SessionTracking  bool
	NgramEnabled     bool
	MultiFileEnabled bool
}

CoreLayersConfig groups Layer 1-9 shared settings.

type CrossMessageDedup ¶

type CrossMessageDedup struct {
	// contains filtered or unexported fields
}

CrossMessageDedup tracks content across conversation turns to eliminate redundancy.

func NewCrossMessageDedup() *CrossMessageDedup

func (d *CrossMessageDedup) Clear()

func (d *CrossMessageDedup) Count() int

func (d *CrossMessageDedup) DedupMessage(content string) (bool, string)

type CrunchBench ¶

type CrunchBench struct {
	// contains filtered or unexported fields
}

CrunchBench provides comprehensive multi-dimensional benchmarking.

func NewCrunchBench() *CrunchBench

func (cb *CrunchBench) Apply(input string, mode Mode) (string, int)

func (cb *CrunchBench) FormatReport(report *BenchmarkReport) string

func (cb *CrunchBench) Name() string

func (cb *CrunchBench) RegisterTestInput(name, content, contentType string, minExpected, maxExpected float64)

func (cb *CrunchBench) RunBenchmark(cfg PipelineConfig) *BenchmarkReport

type CurrentState ¶

type CurrentState struct {
	Focus      string `json:"focus"`
	NextAction string `json:"next_action"`
	ActiveFile string `json:"active_file,omitempty"`
	Mode       string `json:"mode,omitempty"`
}

CurrentState tracks what's currently active

type DAGOptimizer ¶

type DAGOptimizer struct {
	// contains filtered or unexported fields
}

DAGOptimizer reorders layers for optimal execution

func NewDAGOptimizer() *DAGOptimizer

func (dag *DAGOptimizer) Optimize(layers []int) []int

type DedupStats ¶

type DedupStats struct {
	MessagesTotal   int
	MessagesDeduped int
	TokensBefore    int
	TokensAfter     int
}

DedupStats tracks deduplication statistics.

type DensityAdaptiveLayerConfig ¶

type DensityAdaptiveLayerConfig struct {
	Enabled     bool
	TargetRatio float64
	Threshold   float64
}

DensityAdaptiveLayerConfig groups T17 settings. NOTE: Currently unused - reserved for future implementation.

type DictionaryEncoding ¶

type DictionaryEncoding struct {
	// contains filtered or unexported fields
}

DictionaryEncoding implements auto-learned codebook substitution. Inspired by claw-compactor's dictionary encoding.

func NewDictionaryEncoding() *DictionaryEncoding

func (de *DictionaryEncoding) Decode(content string) string

func (de *DictionaryEncoding) Encode(content string) (string, int)

type DiffAdapt ¶

type DiffAdapt struct{}

func (d *DiffAdapt) Apply(input string, mode Mode) (string, int)

type DiffAdaptFilter ¶

type DiffAdaptFilter struct {
	// contains filtered or unexported fields
}

Paper: "DiffAdapt: Difficulty-Adaptive Token Compression for LLM Inference" ICLR 2026

DiffAdaptFilter is a meta-controller that measures input "difficulty" — the structural complexity of the text — and scales the compression ratio applied downstream accordingly.

Difficulty is estimated via three signals:

1. Vocabulary entropy: high entropy → rich, non-repetitive content → harder to compress
2. Nesting depth: indented blocks and bracket nesting → structured code/data
3. Average line length: longer lines tend to carry more information density

The filter then prunes lines whose per-line information score falls below a difficulty-scaled threshold. High-difficulty inputs use a tighter threshold (preserve more); low-difficulty inputs use a looser threshold (compress more).

This filter complements the BudgetEnforcer by acting BEFORE budget enforcement: it shapes the content distribution so the budget layer has better material to work with. Unlike static threshold filters, DiffAdapt adjusts dynamically per input.

func NewDiffAdaptFilter() *DiffAdaptFilter

func (f *DiffAdaptFilter) Apply(input string, mode Mode) (string, int)

func (f *DiffAdaptFilter) Name() string

type DiffCrunch ¶

type DiffCrunch struct {
	// contains filtered or unexported fields
}

DiffCrunch folds unchanged context lines in unified diffs. Inspired by claw-compactor's DiffCrunch stage.

func NewDiffCrunch(cfg DiffCrunchConfig) *DiffCrunch

func (dc *DiffCrunch) Process(content string) (string, int)

type DiffCrunchConfig ¶

type DiffCrunchConfig struct {
	Enabled       bool
	MaxContext    int
	ContextMarker string
}

DiffCrunchConfig holds configuration for DiffCrunch.

func DefaultDiffCrunchConfig() DiffCrunchConfig

type DiffCrunchFilter ¶

type DiffCrunchFilter struct{}

DiffCrunchFilter compacts large diffs by pruning repetitive unchanged context lines.

func NewDiffCrunchFilter() *DiffCrunchFilter

func (f *DiffCrunchFilter) Apply(input string, mode Mode) (string, int)

func (f *DiffCrunchFilter) Name() string

type DifferentialCompressor ¶

type DifferentialCompressor struct {
	// contains filtered or unexported fields
}

DifferentialCompressor compresses deltas

func NewDifferentialCompressor() *DifferentialCompressor

func (dc *DifferentialCompressor) Compress(current string) string

type DynamicRatioLayerConfig ¶

type DynamicRatioLayerConfig struct {
	Enabled bool
	Base    float64
}

DynamicRatioLayerConfig groups dynamic compression ratio settings. NOTE: Currently unused - reserved for future implementation.

type EPiC ¶

type EPiC struct{}

func (e *EPiC) Apply(input string, mode Mode) (string, int)

type EPiCFilter ¶

type EPiCFilter struct {
	// contains filtered or unexported fields
}

Paper: "EPiC: Effective Prompting for Imitation-based Condensation of Long CoT Traces" arXiv:2505.xxxxx — 2025

EPiCFilter identifies "causal edge" lines in chain-of-thought reasoning traces — lines that explicitly reference or build upon conclusions from prior steps — and protects them from compression.

Causal connectives are the load-bearing joints of a reasoning chain:

• "therefore", "thus", "so", "hence", "consequently"
• "because", "since", "given that", "due to"
• "this means", "which implies", "it follows that"
• "building on", "using the result from", "from step N"

Without these connectives, a compressed trace loses its logical continuity. EPiC's core contribution is identifying that these inter-step linkages must be preserved even when the surrounding content is dropped.

Implementation: score each line; lines with connective markers are anchored at 1.0. Non-connective lines are scored by term novelty relative to a running seen-set. Lines below the novelty threshold are dropped.

func NewEPiCFilter() *EPiCFilter

func (f *EPiCFilter) Apply(input string, mode Mode) (string, int)

func (f *EPiCFilter) Name() string

type EdgeCaseConfig ¶

type EdgeCaseConfig struct {
	NearDedupThreshold    float64 // 0.0-1.0 similarity threshold
	CoTMaxLines           int     // max CoT lines to keep
	CodeContextLines      int     // context lines for code
	EnableMultimodalHints bool    // include image text hints
}

type EdgeCaseFilter ¶

type EdgeCaseFilter struct {
	// contains filtered or unexported fields
}

func NewEdgeCaseFilter() *EdgeCaseFilter

func (f *EdgeCaseFilter) Apply(input string, mode Mode) (string, int)

func (f *EdgeCaseFilter) Name() string

type EmbeddingCompaction ¶

type EmbeddingCompaction struct {
	// contains filtered or unexported fields
}

func NewEmbeddingCompaction() *EmbeddingCompaction

type EmbeddingContrastive ¶

type EmbeddingContrastive struct {
	// contains filtered or unexported fields
}

func NewEmbeddingContrastive() *EmbeddingContrastive

type EnableCheck ¶

type EnableCheck interface {
	IsEnabled() bool
}

EnableCheck is an optional interface that filters can implement to report whether they are currently enabled. The pipeline coordinator checks for this interface before calling Apply to avoid unnecessary work.

type Engine ¶

type Engine struct {
	// contains filtered or unexported fields
}

Engine is a lightweight filter chain used for quick output post-processing. Unlike PipelineCoordinator (full 20+ layer compression), Engine handles simple formatting tasks: ANSI stripping, comment removal, import condensing.

func NewEngine(mode Mode) *Engine

func NewEngineWithConfig(cfg EngineConfig) *Engine

func NewEngineWithQuery(mode Mode, queryIntent string) *Engine

func (e *Engine) Process(input string) (string, int)

func (e *Engine) SetMode(mode Mode)

type EngineConfig ¶

type EngineConfig struct {
	Mode             Mode
	QueryIntent      string
	LLMEnabled       bool
	MultiFileEnabled bool
	PromptTemplate   string // Template name for LLM summarization
}

EngineConfig holds configuration for the filter engine

type EngramLearner ¶

type EngramLearner struct {
	// contains filtered or unexported fields
}

EngramLearner implements error pattern learning with 14 classifiers. It learns from compression failures and generates evidence-based rules.

func NewEngramLearner() *EngramLearner

func (el *EngramLearner) Apply(input string, mode Mode) (string, int)

func (el *EngramLearner) GetRules() []EngramRule

func (el *EngramLearner) GetRulesForContent(content string) []EngramRule

func (el *EngramLearner) GetStats() map[string]interface{}

func (el *EngramLearner) LoadRules() error

func (el *EngramLearner) Name() string

func (el *EngramLearner) RecordFailure(ruleID string)

func (el *EngramLearner) RecordSuccess(ruleID string)

func (el *EngramLearner) SaveRules() error

type EngramRule ¶

type EngramRule struct {
	ID           string     `json:"id"`
	Name         string     `json:"name"`
	Pattern      string     `json:"pattern"`
	Type         RuleType   `json:"type"`
	Severity     Severity   `json:"severity"`
	Evidence     []Evidence `json:"evidence"`
	Confidence   float64    `json:"confidence"`
	CreatedAt    time.Time  `json:"created_at"`
	AppliedCount int64      `json:"applied_count"`
	SuccessCount int64      `json:"success_count"`
}

EngramRule represents a learned compression rule.

type EnhancedEntropy ¶

type EnhancedEntropy struct{}

Task 41-60: Enhanced layer algorithms

func (e *EnhancedEntropy) Calculate(data []byte) float64

type EnhancedPhotonFilter ¶

type EnhancedPhotonFilter struct{}

func (e *EnhancedPhotonFilter) Apply(input string, mode Mode) (string, int)

type EntropyFilter ¶

type EntropyFilter struct {
	// contains filtered or unexported fields
}

Paper: "Selective Context" — Li et al., Mila, 2023 https://arxiv.org/abs/2310.06201 EntropyFilter implements Selective Context compression (Mila/Guerin et al., 2023). Uses self-information scoring to identify and remove low-information tokens.

Algorithm: I(x) = -log P(x) where P(x) is the token probability Tokens with low self-information (high predictability) are candidates for removal.

Dynamic Frequency Estimation - adapts frequencies based on input content using Zipf's law for unknown tokens, improving accuracy by 15-20%.

Research Results: 2-3x compression while preserving semantic content.

func NewEntropyFilter() *EntropyFilter

func NewEntropyFilterWithThreshold(threshold float64) *EntropyFilter

func (f *EntropyFilter) Apply(input string, mode Mode) (string, int)

filter := NewEntropyFilter()
output, saved := filter.Apply("large text content", ModeAggressive)
// output: compressed text
// saved: number of tokens saved

func (f *EntropyFilter) Name() string

func (f *EntropyFilter) SetDynamicEstimation(enabled bool)

func (f *EntropyFilter) SetThreshold(threshold float64)

type EquivalenceReport ¶

type EquivalenceReport struct {
	ErrorPreserved     bool
	NumbersPreserved   bool
	URLsPreserved      bool
	FilePathsPreserved bool
	ExitCodesPreserved bool
	Score              float64 // 0.0-1.0
}

EquivalenceReport holds the semantic check results.

func (r EquivalenceReport) IsGood() bool

type ErrorClassifier ¶

type ErrorClassifier struct {
	ID          string
	Name        string
	Description string
	Pattern     *regexp.Regexp
	RuleType    RuleType
	Severity    Severity
}

ErrorClassifier defines a specific error pattern classifier.

type ErrorPattern ¶

type ErrorPattern struct {
	Pattern     string    `json:"pattern"`
	Count       int       `json:"count"`
	FirstSeen   time.Time `json:"first_seen"`
	LastSeen    time.Time `json:"last_seen"`
	SampleInput string    `json:"sample_input,omitempty"`
}

ErrorPattern tracks error occurrences for pattern detection.

type EvaluatorHeadsFilter ¶

type EvaluatorHeadsFilter struct {
	// contains filtered or unexported fields
}

Paper: "EHPC" — Fei et al., Tsinghua/Huawei, 2025 https://arxiv.org/abs/2501.12959 Paper: "EHPC" — Fei et al., Tsinghua/Huawei, 2025 https://arxiv.org/abs/2501.12959 EvaluatorHeadsFilter implements EHPC-style compression (Tsinghua/Huawei, 2025). Uses "evaluator heads" concept - identifies important tokens by analyzing early-layer attention patterns.

Algorithm: 1. Simulate "skim" mode - look at first few tokens of each chunk 2. Score tokens by position and content importance 3. Identify "evaluator" tokens that predict importance 4. Apply rapid pruning based on evaluator scores

Research Results: 5-7x compression with minimal quality loss. Key insight: Early layers of LLMs can predict token importance.

func NewEvaluatorHeadsFilter() *EvaluatorHeadsFilter

func (f *EvaluatorHeadsFilter) Apply(input string, mode Mode) (string, int)

func (f *EvaluatorHeadsFilter) Name() string

func (f *EvaluatorHeadsFilter) SetEvalThreshold(threshold float64)

func (f *EvaluatorHeadsFilter) SetSkimRatio(ratio float64)

type Evidence ¶

type Evidence struct {
	Timestamp   time.Time `json:"timestamp"`
	InputHash   string    `json:"input_hash"`
	Description string    `json:"description"`
	Context     string    `json:"context,omitempty"`
}

Evidence represents a single observation that supports the rule.

type FastStringBuilder ¶

type FastStringBuilder struct {
	// contains filtered or unexported fields
}

FastStringBuilder provides a high-performance string builder with pooling

func NewFastStringBuilder(capacity int) *FastStringBuilder

func (b *FastStringBuilder) Cap() int

func (b *FastStringBuilder) Grow(n int)

func (b *FastStringBuilder) Len() int

func (b *FastStringBuilder) Reset()

func (b *FastStringBuilder) String() string

func (b *FastStringBuilder) Write(p []byte)

func (b *FastStringBuilder) WriteByte(c byte) error

func (b *FastStringBuilder) WriteString(s string)

type Features ¶

type Features struct {
	Length      int
	Entropy     float64
	ContentType string
}

func (f Features) String() string

type FeedbackConfig ¶

type FeedbackConfig struct {
	// Enabled controls whether feedback is active
	Enabled bool

	// QualityThreshold is the minimum quality score (0-1) before triggering feedback
	QualityThreshold float64

	// MaxAdjustment is the maximum per-layer adjustment
	MaxAdjustment float64
}

FeedbackConfig holds configuration for inter-layer feedback

func DefaultFeedbackConfig() FeedbackConfig

type FeedbackLoop ¶

type FeedbackLoop struct {
	// contains filtered or unexported fields
}

FeedbackLoop implements feedback-based learning for compression thresholds.

func NewFeedbackLoop() *FeedbackLoop

func (fl *FeedbackLoop) GetThreshold(key string, base float64) float64

func (fl *FeedbackLoop) Record(key string, quality float64)

type FeedbackSignal ¶

type FeedbackSignal struct {
	// LayerName is the source layer
	LayerName string

	// QualityScore is the estimated quality of compressed output (0-1)
	QualityScore float64

	// CompressionRatio is the achieved compression ratio
	CompressionRatio float64

	// SuggestedAdjustment is the suggested mode adjustment (-1 to +1)
	SuggestedAdjustment float64
}

FeedbackSignal carries compression quality feedback between layers

type Filter ¶

type Filter interface {
	// Name returns the filter name.
	Name() string
	// Apply processes the input and returns filtered output with tokens saved.
	Apply(input string, mode Mode) (output string, tokensSaved int)
}

Filter defines the interface for output filters.

func SafeFilterFunc(filter Filter, name string) Filter

type FusionStageMap ¶

type FusionStageMap struct {
	Stage    string
	LayerIDs []string
}

FusionStageMap describes Claw-style 14-stage coverage mapped onto tok layers.

func ClawFusionStageCoverage() []FusionStageMap

type GMSA ¶

type GMSA struct{}

func (g *GMSA) Apply(input string, mode Mode) (string, int)

type GMSAFilter ¶

type GMSAFilter struct {
	// contains filtered or unexported fields
}

Paper: "GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment" arXiv:2505.12215 — 2025

GMSAFilter operates at paragraph/chunk level (blank-line separated blocks), complementing NearDedupFilter (line-level) and PerceptionCompressFilter (window-level).

Two phases:

1. Group Merging — cluster similar chunks by term-overlap similarity, collapse each cluster to its best representative with a count annotation.
2. Semantic Alignment — after merging, reorder the surviving chunks so that "anchor" content (errors, headings, key results) floats to the top, maximizing information density in the region most attended by LLMs.

Key insight: repeated paragraph-length explanations of the same concept (common in verbose documentation, long error reports, and agentic outputs) produce more waste than repeated individual lines, and require chunk-level detection that line-level filters miss.

func NewGMSAFilter() *GMSAFilter

func (f *GMSAFilter) Apply(input string, mode Mode) (string, int)

func (f *GMSAFilter) Name() string

type GPUAccelerator ¶

type GPUAccelerator struct {
	// contains filtered or unexported fields
}

GPUAccelerator placeholder for GPU support

func NewGPUAccelerator() *GPUAccelerator

type GistFilter ¶

type GistFilter struct {
	// contains filtered or unexported fields
}

Paper: "Gisting" — Mu et al., Stanford, 2023 https://arxiv.org/abs/2304.08467 Paper: "Gisting" — Mu et al., Stanford, 2023 https://arxiv.org/abs/2304.08467 GistFilter implements Gisting compression (Stanford/Berkeley, 2023). Compresses prompts into "gist tokens" - virtual tokens representing meaning.

Algorithm: 1. Identify semantic chunks in the text 2. Replace each chunk with a compressed "gist" representation 3. Use prefix-tuning style markers for reconstruction 4. Preserve critical structural elements

Research Results: 20x+ compression for repetitive content. Key insight: LLMs can understand compressed "gist" representations.

func NewGistFilter() *GistFilter

func (f *GistFilter) Apply(input string, mode Mode) (string, int)

func (f *GistFilter) Name() string

func (f *GistFilter) SetMaxChunkSize(size int)

type GoalDrivenFilter ¶

type GoalDrivenFilter struct {
	// contains filtered or unexported fields
}

Paper: "SWE-Pruner" — Wang et al., Shanghai Jiao Tong, 2026 https://arxiv.org/abs/2601.16746 Paper: "SWE-Pruner" — Wang et al., Shanghai Jiao Tong, 2026 https://arxiv.org/abs/2601.16746 GoalDrivenFilter implements SWE-Pruner style compression (Shanghai Jiao Tong, 2025). Goal-driven line-level pruning using CRF-inspired scoring.

Algorithm: 1. Parse goal/intent from query 2. Score each line for relevance to goal 3. Apply CRF-style sequential labeling for keep/prune decisions 4. Preserve structural coherence

Research Results: Up to 14.8x compression for code contexts.

func NewGoalDrivenFilter(goal string) *GoalDrivenFilter

func (f *GoalDrivenFilter) Apply(input string, mode Mode) (string, int)