core

package
v1.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 2, 2026 License: MIT Imports: 19 Imported by: 0

Documentation

Overview

Package core provides budget enforcement for token optimization.

Package core provides entropy-based filtering for compression optimization.

Package core provides goal-driven context analysis for intelligent compression.

Package core provides compression layer implementations.

Package core provides advanced compression layers (11-20).

Package core provides final compression layers (21-31).

Package core provides perplexity-based filtering using statistical language models.

Index

Constants

This section is empty.

Variables

View Source 
var CommonModelPricing = map[string]ModelPricing{
	"gpt-4o": {
		Model:            "gpt-4o",
		InputPerMillion:  2.50,
		OutputPerMillion: 10.00,
	},
	"gpt-4o-mini": {
		Model:            "gpt-4o-mini",
		InputPerMillion:  0.15,
		OutputPerMillion: 0.60,
	},
	"claude-3.5-sonnet": {
		Model:            "claude-3.5-sonnet",
		InputPerMillion:  3.00,
		OutputPerMillion: 15.00,
	},
	"claude-3-haiku": {
		Model:            "claude-3-haiku",
		InputPerMillion:  0.25,
		OutputPerMillion: 1.25,
	},
}

CommonModelPricing provides pricing for popular models (as of 2025).

Functions

func CalculateSavings 

func CalculateSavings(tokensSaved int, model string) float64

CalculateSavings computes dollar savings from token reduction.

func CalculateTokensSaved 

func CalculateTokensSaved(original, filtered string) int

CalculateTokensSaved computes token savings between original and filtered.

func DefaultGoals added in v1.5.0 

func DefaultGoals() map[GoalType]Goal

DefaultGoals provides predefined goals.

func EstimateTokens 

func EstimateTokens(text string) int

EstimateTokens is the single source of truth for token estimation. P1.1: Uses BPE tokenization when available, falls back to heuristic. Phase 2.8: Results are cached to avoid repeated encoding.

func RegisterAdvancedLayers added in v1.5.0 

func RegisterAdvancedLayers(registry *LayerRegistry)

RegisterAdvancedLayers registers layers 11-20.

func RegisterDefaultLayers added in v1.5.0 

func RegisterDefaultLayers(registry *LayerRegistry)

RegisterDefaultLayers registers all default layers.

func RegisterFinalLayers added in v1.5.0 

func RegisterFinalLayers(registry *LayerRegistry)

RegisterFinalLayers registers layers 21-31.

Types

type AnalyzeResult added in v1.5.0 

type AnalyzeResult struct {
	Items         []ContextItem
	TotalItems    int
	FilteredItems int
	RelevanceAvg  float64
}

AnalyzeResult contains the analysis results.

type AttentionSinkLayer added in v1.5.0 

type AttentionSinkLayer struct {
	// contains filtered or unexported fields
}

Layer 14: Attention Sink Rolling Cache

func NewAttentionSinkLayer added in v1.5.0 

func NewAttentionSinkLayer() *AttentionSinkLayer

NewAttentionSinkLayer creates attention sink layer.

func (*AttentionSinkLayer) Apply added in v1.5.0 

func (l *AttentionSinkLayer) Apply(content string) (string, int)

func (*AttentionSinkLayer) Name added in v1.5.0 

func (l *AttentionSinkLayer) Name() string

func (*AttentionSinkLayer) ShouldApply added in v1.5.0 

func (l *AttentionSinkLayer) ShouldApply(contentType string) bool

type AttributionLayer added in v1.5.0 

type AttributionLayer struct {
	// contains filtered or unexported fields
}

Layer 12: Attribution Filter

func NewAttributionLayer added in v1.5.0 

func NewAttributionLayer() *AttributionLayer

NewAttributionLayer creates a new attribution layer.

func (*AttributionLayer) Apply added in v1.5.0 

func (l *AttributionLayer) Apply(content string) (string, int)

func (*AttributionLayer) Name added in v1.5.0 

func (l *AttributionLayer) Name() string

func (*AttributionLayer) ShouldApply added in v1.5.0 

func (l *AttributionLayer) ShouldApply(contentType string) bool

type BPETokenizer 

type BPETokenizer struct {
	// contains filtered or unexported fields
}

BPETokenizer wraps tiktoken for accurate BPE token counting. P1.1: Replaces heuristic len/4 with real BPE tokenization. ~20-30% more accurate than heuristic estimation.

func (*BPETokenizer) Count 

func (b *BPETokenizer) Count(text string) int

Count returns the accurate BPE token count with caching.

type BudgetConfig added in v1.5.0 

type BudgetConfig struct {
	MaxTokens     int
	Mode          BudgetMode
	PreserveRatio float64 // Ratio of budget for high-priority content
}

BudgetConfig configures budget enforcement.

func DefaultBudgetConfig added in v1.5.0 

func DefaultBudgetConfig() BudgetConfig

DefaultBudgetConfig returns default configuration.

type BudgetEnforcer added in v1.5.0 

type BudgetEnforcer struct {
	// contains filtered or unexported fields
}

BudgetEnforcer enforces token budget constraints.

func NewBudgetEnforcer added in v1.5.0 

func NewBudgetEnforcer(config BudgetConfig) *BudgetEnforcer

NewBudgetEnforcer creates a new budget enforcer.

func (*BudgetEnforcer) Check added in v1.5.0 

func (be *BudgetEnforcer) Check(content string) BudgetStatus

Check checks current content against budget.

func (*BudgetEnforcer) Enforce added in v1.5.0 

func (be *BudgetEnforcer) Enforce(content string) (string, EnforceResult)

Enforce enforces the budget on content.

type BudgetLayer added in v1.5.0 

type BudgetLayer struct {
	// contains filtered or unexported fields
}

Layer 10: Budget Enforcement

func NewBudgetLayer added in v1.5.0 

func NewBudgetLayer(maxTokens int) *BudgetLayer

NewBudgetLayer creates a new budget enforcement layer.

func (*BudgetLayer) Apply added in v1.5.0 

func (l *BudgetLayer) Apply(content string) (string, int)

func (*BudgetLayer) Name added in v1.5.0 

func (l *BudgetLayer) Name() string

func (*BudgetLayer) ShouldApply added in v1.5.0 

func (l *BudgetLayer) ShouldApply(contentType string) bool

type BudgetMode added in v1.5.0 

type BudgetMode int

BudgetMode represents the budget enforcement mode. 

const (
	BudgetModeSoft     BudgetMode = iota // Warn but don't enforce
	BudgetModeStrict                     // Strict enforcement - truncate content
	BudgetModeAdaptive                   // Adapt compression based on budget
)

type BudgetStatus added in v1.5.0 

type BudgetStatus struct {
	CurrentTokens   int
	MaxTokens       int
	RemainingTokens int
	PercentUsed     float64
	Exceeded        bool
}

BudgetStatus represents the current budget status.

type CodeFoldLayer added in v1.5.0 

type CodeFoldLayer struct {
	// contains filtered or unexported fields
}

Layer 7: Code Fold Detection

func NewCodeFoldLayer added in v1.5.0 

func NewCodeFoldLayer() *CodeFoldLayer

NewCodeFoldLayer creates a new code folding layer.

func (*CodeFoldLayer) Apply added in v1.5.0 

func (l *CodeFoldLayer) Apply(content string) (string, int)

func (*CodeFoldLayer) Name added in v1.5.0 

func (l *CodeFoldLayer) Name() string

func (*CodeFoldLayer) ShouldApply added in v1.5.0 

func (l *CodeFoldLayer) ShouldApply(contentType string) bool

type CodeFoldLayer22 added in v1.5.0 

type CodeFoldLayer22 struct {
	// contains filtered or unexported fields
}

Layer 22: Code Fold Detection

func NewCodeFoldLayer22 added in v1.5.0 

func NewCodeFoldLayer22() *CodeFoldLayer22

func (*CodeFoldLayer22) Apply added in v1.5.0 

func (l *CodeFoldLayer22) Apply(content string) (string, int)

func (*CodeFoldLayer22) Name added in v1.5.0 

func (l *CodeFoldLayer22) Name() string

func (*CodeFoldLayer22) ShouldApply added in v1.5.0 

func (l *CodeFoldLayer22) ShouldApply(contentType string) bool

type CommandRunner 

type CommandRunner interface {
	Run(ctx context.Context, args []string) (output string, exitCode int, err error)
	LookPath(name string) (string, error)
}

CommandRunner abstracts shell command execution.

type CommentRemovalLayer added in v1.5.0 

type CommentRemovalLayer struct {
	// contains filtered or unexported fields
}

Layer 9: Comment Removal Heuristics

func NewCommentRemovalLayer added in v1.5.0 

func NewCommentRemovalLayer() *CommentRemovalLayer

NewCommentRemovalLayer creates a new comment removal layer.

func (*CommentRemovalLayer) Apply added in v1.5.0 

func (l *CommentRemovalLayer) Apply(content string) (string, int)

func (*CommentRemovalLayer) Name added in v1.5.0 

func (l *CommentRemovalLayer) Name() string

func (*CommentRemovalLayer) ShouldApply added in v1.5.0 

func (l *CommentRemovalLayer) ShouldApply(contentType string) bool

type CommentRemovalLayer24 added in v1.5.0 

type CommentRemovalLayer24 struct {
	// contains filtered or unexported fields
}

Layer 24: Comment Removal Heuristics

func NewCommentRemovalLayer24 added in v1.5.0 

func NewCommentRemovalLayer24() *CommentRemovalLayer24

func (*CommentRemovalLayer24) Apply added in v1.5.0 

func (l *CommentRemovalLayer24) Apply(content string) (string, int)

func (*CommentRemovalLayer24) Name added in v1.5.0 

func (l *CommentRemovalLayer24) Name() string

func (*CommentRemovalLayer24) ShouldApply added in v1.5.0 

func (l *CommentRemovalLayer24) ShouldApply(contentType string) bool

type CompressionInfo added in v1.5.0 

type CompressionInfo struct {
	OriginalSize   int
	CompressedSize int
	Saved          int
	Regions        int
}

CompressionInfo contains compression statistics.

type ContextItem added in v1.5.0 

type ContextItem struct {
	Content   string
	Source    string
	LineNum   int
	Relevance float64
	Priority  float64
}

ContextItem represents a single item of context.

type ContrastiveLayer added in v1.5.0 

type ContrastiveLayer struct {
	// contains filtered or unexported fields
}

Layer 5: Contrastive Learning - Identify semantically similar content

func NewContrastiveLayer added in v1.5.0 

func NewContrastiveLayer() *ContrastiveLayer

NewContrastiveLayer creates a new contrastive learning layer.

func (*ContrastiveLayer) Apply added in v1.5.0 

func (l *ContrastiveLayer) Apply(content string) (string, int)

func (*ContrastiveLayer) Name added in v1.5.0 

func (l *ContrastiveLayer) Name() string

func (*ContrastiveLayer) ShouldApply added in v1.5.0 

func (l *ContrastiveLayer) ShouldApply(contentType string) bool

type CostTracker added in v1.5.0 

type CostTracker struct {
	// contains filtered or unexported fields
}

func NewCostTracker added in v1.5.0 

func NewCostTracker() *CostTracker

func (*CostTracker) GetCostByModel added in v1.5.0 

func (t *CostTracker) GetCostByModel() map[string]float64

func (*CostTracker) GetCostByProvider added in v1.5.0 

func (t *CostTracker) GetCostByProvider() map[string]float64

func (*CostTracker) GetTotalCost added in v1.5.0 

func (t *CostTracker) GetTotalCost() float64

func (*CostTracker) GetTurns added in v1.5.0 

func (t *CostTracker) GetTurns(limit int) []TurnRecord

func (*CostTracker) RecordTurn added in v1.5.0 

func (t *CostTracker) RecordTurn(cmd string, inputTokens, outputTokens int, cost float64, model, provider string)

type DiscoverAnalyzer 

type DiscoverAnalyzer struct {
	// contains filtered or unexported fields
}

func NewDiscoverAnalyzer 

func NewDiscoverAnalyzer() *DiscoverAnalyzer

func (*DiscoverAnalyzer) Analyze 

func (d *DiscoverAnalyzer) Analyze(command string) []MissedSaving

func (*DiscoverAnalyzer) AnalyzeBatch 

func (d *DiscoverAnalyzer) AnalyzeBatch(commands []string) []MissedSaving

type DupDetector added in v1.5.0 

type DupDetector struct {
	// contains filtered or unexported fields
}

func NewDupDetector added in v1.5.0 

func NewDupDetector(window time.Duration) *DupDetector

func (*DupDetector) Cleanup added in v1.5.0 

func (d *DupDetector) Cleanup()

func (*DupDetector) GetDuplicates added in v1.5.0 

func (d *DupDetector) GetDuplicates() []*DupEntry

func (*DupDetector) Record added in v1.5.0 

func (d *DupDetector) Record(command string, tokens int) *DupEntry

type DupEntry added in v1.5.0 

type DupEntry struct {
	Command     string
	Count       int
	FirstSeen   time.Time
	LastSeen    time.Time
	TotalTokens int
}

type EnforceResult added in v1.5.0 

type EnforceResult struct {
	OriginalTokens  int
	FinalTokens     int
	WasEnforced     bool
	ReductionMethod string
	StagesApplied   []string
	Warning         string
}

EnforceResult contains enforcement results.

func (EnforceResult) ReductionPercent added in v1.5.0 

func (er EnforceResult) ReductionPercent() float64

ReductionPercent returns reduction percentage.

func (EnforceResult) TokensSaved added in v1.5.0 

func (er EnforceResult) TokensSaved() int

TokensSaved returns tokens saved.

type EntropyConfig added in v1.5.0 

type EntropyConfig struct {
	WindowSize        int
	HighEntropyThresh float64 // Threshold for high entropy (random data)
	LowEntropyThresh  float64 // Threshold for low entropy (repetitive data)
	MinRepeat         int     // Minimum repeats to consider deduplication
}

EntropyConfig configures the entropy filter.

func DefaultEntropyConfig added in v1.5.0 

func DefaultEntropyConfig() EntropyConfig

DefaultEntropyConfig returns default configuration.

type EntropyFilter added in v1.5.0 

type EntropyFilter struct {
	// contains filtered or unexported fields
}

EntropyFilter implements Shannon entropy-based content filtering. It identifies high-entropy regions (random data, hashes, keys) that are less compressible and can be aggressively deduplicated.

func NewEntropyFilter added in v1.5.0 

func NewEntropyFilter(config EntropyConfig) *EntropyFilter

NewEntropyFilter creates a new entropy filter.

func (*EntropyFilter) Analyze added in v1.5.0 

func (f *EntropyFilter) Analyze(content string) []Region

Analyze analyzes content and returns regions with entropy information.

func (*EntropyFilter) Compress added in v1.5.0 

func (f *EntropyFilter) Compress(content string) (string, CompressionInfo)

Compress applies entropy-based compression.

func (*EntropyFilter) FindDuplicates added in v1.5.0 

func (f *EntropyFilter) FindDuplicates(content string) map[string][]int

FindDuplicates finds duplicate high-entropy regions.

func (*EntropyFilter) GetStats added in v1.5.0 

func (f *EntropyFilter) GetStats(content string) EntropyStats

GetStats returns entropy statistics for content.

type EntropyStats added in v1.5.0 

type EntropyStats struct {
	AvgEntropy      float64
	HighEntropyPct  float64
	LowEntropyPct   float64
	DuplicateHashes int
}

EntropyStats provides statistics about content entropy.

type EntropyType added in v1.5.0 

type EntropyType int

EntropyType classifies the entropy level. 

const (
	LowEntropy    EntropyType = iota // Repetitive, highly compressible
	MediumEntropy                    // Normal text/code
	HighEntropy                      // Random data, hashes, keys
)

type FilterInfo added in v1.5.0 

type FilterInfo struct {
	OriginalSize     int
	FilteredSize     int
	Perplexity       float64
	HighEntropyLines int
}

FilterInfo contains filtering statistics.

func (FilterInfo) ReductionPercent added in v1.5.0 

func (f FilterInfo) ReductionPercent() float64

ReductionPercent returns the percentage reduction.

type FinalPassLayer added in v1.5.0 

type FinalPassLayer struct{}

Layer 31: Final Compression Pass

func NewFinalPassLayer added in v1.5.0 

func NewFinalPassLayer() *FinalPassLayer

func (*FinalPassLayer) Apply added in v1.5.0 

func (l *FinalPassLayer) Apply(content string) (string, int)

func (*FinalPassLayer) Name added in v1.5.0 

func (l *FinalPassLayer) Name() string

func (*FinalPassLayer) ShouldApply added in v1.5.0 

func (l *FinalPassLayer) ShouldApply(contentType string) bool

type Goal added in v1.5.0 

type Goal struct {
	Type        GoalType
	Keywords    []string
	PriorityMap map[string]float64 // Keyword -> importance score
}

Goal represents a user's intent.

type GoalAnalyzer added in v1.5.0 

type GoalAnalyzer struct {
	// contains filtered or unexported fields
}

GoalAnalyzer analyzes content against user goals.

func NewGoalAnalyzer added in v1.5.0 

func NewGoalAnalyzer(goal Goal) *GoalAnalyzer

NewGoalAnalyzer creates a new goal analyzer.

func (*GoalAnalyzer) Analyze added in v1.5.0 

func (a *GoalAnalyzer) Analyze(content string, source string) AnalyzeResult

Analyze analyzes content and returns relevant items based on goal.

func (*GoalAnalyzer) ExtractRelevant added in v1.5.0 

func (a *GoalAnalyzer) ExtractRelevant(content string, source string) string

ExtractRelevant extracts only the relevant content based on goal.

func (*GoalAnalyzer) RankFiles added in v1.5.0 

func (a *GoalAnalyzer) RankFiles(files map[string]string) []RankedFile

RankFiles ranks multiple files by relevance to goal.

type GoalType added in v1.5.0 

type GoalType int

GoalType represents the type of user goal. 

const (
	GoalUnderstand GoalType = iota // Understand code structure
	GoalDebug                      // Debug an issue
	GoalRefactor                   // Refactor code
	GoalReview                     // Code review
	GoalTest                       // Write tests
	GoalDocument                   // Generate documentation
	GoalSearch                     // Find specific information
	GoalOptimize                   // Performance optimization
)

func GetGoalFromQuery added in v1.5.0 

func GetGoalFromQuery(query string) GoalType

GetGoalFromQuery infers a goal from a natural language query.

type HeavyHitterLayer added in v1.5.0 

type HeavyHitterLayer struct {
	// contains filtered or unexported fields
}

Layer 13: H2O Heavy-Hitter Optimization

func NewHeavyHitterLayer added in v1.5.0 

func NewHeavyHitterLayer() *HeavyHitterLayer

NewHeavyHitterLayer creates a new heavy hitter layer.

func (*HeavyHitterLayer) Apply added in v1.5.0 

func (l *HeavyHitterLayer) Apply(content string) (string, int)

func (*HeavyHitterLayer) Name added in v1.5.0 

func (l *HeavyHitterLayer) Name() string

func (*HeavyHitterLayer) ShouldApply added in v1.5.0 

func (l *HeavyHitterLayer) ShouldApply(contentType string) bool

type ImportCollapseLayer added in v1.5.0 

type ImportCollapseLayer struct{}

Layer 8: Import/Dependency Collapse

func NewImportCollapseLayer added in v1.5.0 

func NewImportCollapseLayer() *ImportCollapseLayer

NewImportCollapseLayer creates a new import collapse layer.

func (*ImportCollapseLayer) Apply added in v1.5.0 

func (l *ImportCollapseLayer) Apply(content string) (string, int)

func (*ImportCollapseLayer) Name added in v1.5.0 

func (l *ImportCollapseLayer) Name() string

func (*ImportCollapseLayer) ShouldApply added in v1.5.0 

func (l *ImportCollapseLayer) ShouldApply(contentType string) bool

type ImportCollapseLayer23 added in v1.5.0 

type ImportCollapseLayer23 struct{}

Layer 23: Import/Dependency Collapse

func NewImportCollapseLayer23 added in v1.5.0 

func NewImportCollapseLayer23() *ImportCollapseLayer23

func (*ImportCollapseLayer23) Apply added in v1.5.0 

func (l *ImportCollapseLayer23) Apply(content string) (string, int)

func (*ImportCollapseLayer23) Name added in v1.5.0 

func (l *ImportCollapseLayer23) Name() string

func (*ImportCollapseLayer23) ShouldApply added in v1.5.0 

func (l *ImportCollapseLayer23) ShouldApply(contentType string) bool

type KGNode added in v1.5.0 

type KGNode struct {
	ID      string
	Type    string
	Content string
	Related []string
}

type KnowledgeGraphLayer added in v1.5.0 

type KnowledgeGraphLayer struct {
	// contains filtered or unexported fields
}

Layer 20: Agent Memory Knowledge Graph

func NewKnowledgeGraphLayer added in v1.5.0 

func NewKnowledgeGraphLayer() *KnowledgeGraphLayer

NewKnowledgeGraphLayer creates knowledge graph layer.

func (*KnowledgeGraphLayer) Apply added in v1.5.0 

func (l *KnowledgeGraphLayer) Apply(content string) (string, int)

func (*KnowledgeGraphLayer) Name added in v1.5.0 

func (l *KnowledgeGraphLayer) Name() string

func (*KnowledgeGraphLayer) ShouldApply added in v1.5.0 

func (l *KnowledgeGraphLayer) ShouldApply(contentType string) bool

type LLMCompactionLayer added in v1.5.0 

type LLMCompactionLayer struct {
	// contains filtered or unexported fields
}

Layer 11: LLM Compaction with Ollama Support

func NewLLMCompactionLayer added in v1.5.0 

func NewLLMCompactionLayer() *LLMCompactionLayer

NewLLMCompactionLayer creates a new LLM compaction layer.

func (*LLMCompactionLayer) Apply added in v1.5.0 

func (l *LLMCompactionLayer) Apply(content string) (string, int)

func (*LLMCompactionLayer) Name added in v1.5.0 

func (l *LLMCompactionLayer) Name() string

func (*LLMCompactionLayer) ShouldApply added in v1.5.0 

func (l *LLMCompactionLayer) ShouldApply(contentType string) bool

type Layer added in v1.5.0 

type Layer interface {
	Name() string
	Apply(content string) (string, int)
	ShouldApply(contentType string) bool
}

Layer represents a compression layer.

type LayerRegistry added in v1.5.0 

type LayerRegistry struct {
	// contains filtered or unexported fields
}

LayerRegistry manages compression layers.

func NewLayerRegistry added in v1.5.0 

func NewLayerRegistry() *LayerRegistry

NewLayerRegistry creates a new layer registry.

func (*LayerRegistry) Apply added in v1.5.0 

func (r *LayerRegistry) Apply(content string, contentType string) (string, int)

Apply applies all registered layers.

func (*LayerRegistry) GetLayers added in v1.5.0 

func (r *LayerRegistry) GetLayers() []Layer

GetLayers returns all registered layers.

func (*LayerRegistry) Register added in v1.5.0 

func (r *LayerRegistry) Register(layer Layer)

Register registers a layer.

type LazyPrunerLayer added in v1.5.0 

type LazyPrunerLayer struct {
	// contains filtered or unexported fields
}

Layer 18: Lazy Pruner

func NewLazyPrunerLayer added in v1.5.0 

func NewLazyPrunerLayer() *LazyPrunerLayer

NewLazyPrunerLayer creates lazy pruner layer.

func (*LazyPrunerLayer) Apply added in v1.5.0 

func (l *LazyPrunerLayer) Apply(content string) (string, int)

func (*LazyPrunerLayer) Name added in v1.5.0 

func (l *LazyPrunerLayer) Name() string

func (*LazyPrunerLayer) ShouldApply added in v1.5.0 

func (l *LazyPrunerLayer) ShouldApply(contentType string) bool

type LineScore added in v1.5.0 

type LineScore struct {
	Line       int
	Text       string
	Perplexity float64
	Entropy    EntropyType
}

LineScore contains per-line perplexity score.

type LiveCommand added in v1.5.0 

type LiveCommand struct {
	Command      string
	PID          int
	StartedAt    time.Time
	InputTokens  int
	OutputTokens int
	SavedTokens  int
	Status       string
}

type LiveMonitor added in v1.5.0 

type LiveMonitor struct {
	// contains filtered or unexported fields
}

func NewLiveMonitor added in v1.5.0 

func NewLiveMonitor() *LiveMonitor

func (*LiveMonitor) Cleanup added in v1.5.0 

func (m *LiveMonitor) Cleanup(maxAge time.Duration)

func (*LiveMonitor) EndCommand added in v1.5.0 

func (m *LiveMonitor) EndCommand(pid int, inputTokens, outputTokens int)

func (*LiveMonitor) GetActive added in v1.5.0 

func (m *LiveMonitor) GetActive() []*LiveCommand

func (*LiveMonitor) GetRecent added in v1.5.0 

func (m *LiveMonitor) GetRecent(n int) []*LiveCommand

func (*LiveMonitor) StartCommand added in v1.5.0 

func (m *LiveMonitor) StartCommand(pid int, cmd string)

type MetaTokenLZ77Layer added in v1.5.0 

type MetaTokenLZ77Layer struct {
	// contains filtered or unexported fields
}

Layer 15: Meta-Token LZ77

func NewMetaTokenLZ77Layer added in v1.5.0 

func NewMetaTokenLZ77Layer() *MetaTokenLZ77Layer

NewMetaTokenLZ77Layer creates LZ77 compression layer.

func (*MetaTokenLZ77Layer) Apply added in v1.5.0 

func (l *MetaTokenLZ77Layer) Apply(content string) (string, int)

func (*MetaTokenLZ77Layer) Name added in v1.5.0 

func (l *MetaTokenLZ77Layer) Name() string

func (*MetaTokenLZ77Layer) ShouldApply added in v1.5.0 

func (l *MetaTokenLZ77Layer) ShouldApply(contentType string) bool

type MissedSaving 

type MissedSaving struct {
	Command    string
	Reason     string
	Suggestion string
	EstTokens  int
	EstSavings int
}

type ModelPricing 

type ModelPricing struct {
	Model            string
	InputPerMillion  float64 // Cost per 1M input tokens
	OutputPerMillion float64 // Cost per 1M output tokens
}

ModelPricing holds per-token pricing for a model.

type NGramModel added in v1.5.0 

type NGramModel struct {
	// contains filtered or unexported fields
}

NGramModel represents a statistical language model.

func NewNGramModel added in v1.5.0 

func NewNGramModel() *NGramModel

NewNGramModel creates a new n-gram model.

func PretrainedModel added in v1.5.0 

func PretrainedModel() *NGramModel

PretrainedModel provides a basic pretrained model for common code patterns.

func (*NGramModel) Perplexity added in v1.5.0 

func (m *NGramModel) Perplexity(text string) float64

Perplexity calculates perplexity of text using the model.

func (*NGramModel) Train added in v1.5.0 

func (m *NGramModel) Train(text string)

Train trains the model on sample text.

type NgramDeduplicationLayer added in v1.5.0 

type NgramDeduplicationLayer struct {
	// contains filtered or unexported fields
}

Layer 6: N-gram Deduplication

func NewNgramDeduplicationLayer added in v1.5.0 

func NewNgramDeduplicationLayer() *NgramDeduplicationLayer

NewNgramDeduplicationLayer creates a new n-gram deduplication layer.

func (*NgramDeduplicationLayer) Apply added in v1.5.0 

func (l *NgramDeduplicationLayer) Apply(content string) (string, int)

func (*NgramDeduplicationLayer) Name added in v1.5.0 

func (l *NgramDeduplicationLayer) Name() string

func (*NgramDeduplicationLayer) ShouldApply added in v1.5.0 

func (l *NgramDeduplicationLayer) ShouldApply(contentType string) bool

type OSCommandRunner 

type OSCommandRunner struct {
	Env []string
}

OSCommandRunner executes real shell commands using os/exec.

func NewOSCommandRunner 

func NewOSCommandRunner() *OSCommandRunner

NewOSCommandRunner creates a command runner with the current environment.

func (*OSCommandRunner) LookPath 

func (r *OSCommandRunner) LookPath(name string) (string, error)

LookPath resolves a command name to its full path.

func (*OSCommandRunner) Run 

func (r *OSCommandRunner) Run(ctx context.Context, args []string) (string, int, error)

Run executes a command and captures combined stdout+stderr.

type PerplexityConfig added in v1.5.0 

type PerplexityConfig struct {
	MaxNgram  int
	Threshold float64 // Perplexity threshold for filtering
	MinLength int     // Minimum content length to apply
	UseGPU    bool    // Enable GPU acceleration
}

PerplexityConfig configures the perplexity filter.

func DefaultPerplexityConfig added in v1.5.0 

func DefaultPerplexityConfig() PerplexityConfig

DefaultPerplexityConfig returns default configuration.

type PerplexityFilter added in v1.5.0 

type PerplexityFilter struct {
	// contains filtered or unexported fields
}

PerplexityFilter uses n-gram models to estimate content information density. High perplexity indicates unpredictable/noisy content that can be compressed.

func NewPerplexityFilter added in v1.5.0 

func NewPerplexityFilter(config PerplexityConfig) *PerplexityFilter

NewPerplexityFilter creates a new perplexity filter.

func (*PerplexityFilter) AnalyzeLines added in v1.5.0 

func (f *PerplexityFilter) AnalyzeLines(content string) []LineScore

AnalyzeLines analyzes each line separately.

func (*PerplexityFilter) BatchProcess added in v1.5.0 

func (f *PerplexityFilter) BatchProcess(documents []string) []FilterInfo

BatchProcess processes multiple documents in parallel.

func (*PerplexityFilter) Filter added in v1.5.0 

func (f *PerplexityFilter) Filter(content string) (string, FilterInfo)

Filter applies perplexity-based filtering.

func (*PerplexityFilter) Score added in v1.5.0 

func (f *PerplexityFilter) Score(content string) ScoreResult

Score scores content for information density.

func (*PerplexityFilter) TrainOnCorpus added in v1.5.0 

func (f *PerplexityFilter) TrainOnCorpus(samples []string)

TrainOnCorpus trains the model on a corpus of code/text.

type PrecisionLayer added in v1.5.0 

type PrecisionLayer struct {
	// contains filtered or unexported fields
}

Layer 27: Number Precision Reduction

func NewPrecisionLayer added in v1.5.0 

func NewPrecisionLayer() *PrecisionLayer

func (*PrecisionLayer) Apply added in v1.5.0 

func (l *PrecisionLayer) Apply(content string) (string, int)

func (*PrecisionLayer) Name added in v1.5.0 

func (l *PrecisionLayer) Name() string

func (*PrecisionLayer) ShouldApply added in v1.5.0 

func (l *PrecisionLayer) ShouldApply(contentType string) bool

type PromptDebugger added in v1.5.0 

type PromptDebugger struct {
	// contains filtered or unexported fields
}

func NewPromptDebugger added in v1.5.0 

func NewPromptDebugger(dir string) *PromptDebugger

func (*PromptDebugger) CompressionRatio added in v1.5.0 

func (d *PromptDebugger) CompressionRatio(record PromptRecord) float64

func (*PromptDebugger) List added in v1.5.0 

func (d *PromptDebugger) List(limit int) ([]PromptRecord, error)

func (*PromptDebugger) Save added in v1.5.0 

func (d *PromptDebugger) Save(record PromptRecord) error

type PromptRecord added in v1.5.0 

type PromptRecord struct {
	Timestamp        time.Time
	Command          string
	RawPrompt        string
	CompressedPrompt string
	Model            string
	InputTokens      int
	OutputTokens     int
	Duration         time.Duration
}

type RankedFile added in v1.5.0 

type RankedFile struct {
	Path      string
	Content   string
	Score     float64
	Relevance float64
}

RankedFile represents a file with relevance score.

type Region added in v1.5.0 

type Region struct {
	Start       int
	End         int
	Entropy     float64
	EntropyType EntropyType
	Content     string
}

Region represents a region of content with entropy characteristics.

type RepetitionLayer added in v1.5.0 

type RepetitionLayer struct {
	// contains filtered or unexported fields
}

Layer 30: Repetition Detection

func NewRepetitionLayer added in v1.5.0 

func NewRepetitionLayer() *RepetitionLayer

func (*RepetitionLayer) Apply added in v1.5.0 

func (l *RepetitionLayer) Apply(content string) (string, int)

func (*RepetitionLayer) Name added in v1.5.0 

func (l *RepetitionLayer) Name() string

func (*RepetitionLayer) ShouldApply added in v1.5.0 

func (l *RepetitionLayer) ShouldApply(contentType string) bool

type ScoreResult added in v1.5.0 

type ScoreResult struct {
	Perplexity  float64
	EntropyType EntropyType
	ShouldKeep  bool
	Confidence  float64
}

ScoreResult contains perplexity analysis results.

type SemanticAnchorLayer added in v1.5.0 

type SemanticAnchorLayer struct {
	// contains filtered or unexported fields
}

Layer 19: Semantic Anchor Detection

func NewSemanticAnchorLayer added in v1.5.0 

func NewSemanticAnchorLayer() *SemanticAnchorLayer

NewSemanticAnchorLayer creates semantic anchor layer.

func (*SemanticAnchorLayer) Apply added in v1.5.0 

func (l *SemanticAnchorLayer) Apply(content string) (string, int)

func (*SemanticAnchorLayer) Name added in v1.5.0 

func (l *SemanticAnchorLayer) Name() string

func (*SemanticAnchorLayer) ShouldApply added in v1.5.0 

func (l *SemanticAnchorLayer) ShouldApply(contentType string) bool

type SemanticChunkLayer added in v1.5.0 

type SemanticChunkLayer struct {
	// contains filtered or unexported fields
}

Layer 16: Semantic Chunking Boundaries

func NewSemanticChunkLayer added in v1.5.0 

func NewSemanticChunkLayer() *SemanticChunkLayer

NewSemanticChunkLayer creates semantic chunking layer.

func (*SemanticChunkLayer) Apply added in v1.5.0 

func (l *SemanticChunkLayer) Apply(content string) (string, int)

func (*SemanticChunkLayer) Name added in v1.5.0 

func (l *SemanticChunkLayer) Name() string

func (*SemanticChunkLayer) ShouldApply added in v1.5.0 

func (l *SemanticChunkLayer) ShouldApply(contentType string) bool

type SemanticSimilarityLayer added in v1.5.0 

type SemanticSimilarityLayer struct {
	// contains filtered or unexported fields
}

Layer 21: Semantic Similarity Filter

func NewSemanticSimilarityLayer added in v1.5.0 

func NewSemanticSimilarityLayer() *SemanticSimilarityLayer

NewSemanticSimilarityLayer creates a semantic similarity layer.

func (*SemanticSimilarityLayer) Apply added in v1.5.0 

func (l *SemanticSimilarityLayer) Apply(content string) (string, int)

func (*SemanticSimilarityLayer) Name added in v1.5.0 

func (l *SemanticSimilarityLayer) Name() string

func (*SemanticSimilarityLayer) ShouldApply added in v1.5.0 

func (l *SemanticSimilarityLayer) ShouldApply(contentType string) bool

type SessionInfo added in v1.5.0 

type SessionInfo struct {
	ID           string
	StartedAt    time.Time
	LastActive   time.Time
	CommandCount int
	TokensSaved  int
	Agent        string
}

type SessionTracker added in v1.5.0 

type SessionTracker struct {
	// contains filtered or unexported fields
}

func NewSessionTracker added in v1.5.0 

func NewSessionTracker() *SessionTracker

func (*SessionTracker) GetActiveSessions added in v1.5.0 

func (s *SessionTracker) GetActiveSessions() []*SessionInfo

func (*SessionTracker) GetAdoptionRate added in v1.5.0 

func (s *SessionTracker) GetAdoptionRate() float64

func (*SessionTracker) GetAllSessions added in v1.5.0 

func (s *SessionTracker) GetAllSessions() []*SessionInfo

func (*SessionTracker) RecordCommand added in v1.5.0 

func (s *SessionTracker) RecordCommand(id string, tokensSaved int)

func (*SessionTracker) StartSession added in v1.5.0 

func (s *SessionTracker) StartSession(id, agent string)

type Sketch added in v1.5.0 

type Sketch struct {
	Key      string
	Summary  string
	FullHash string
}

type SketchStoreLayer added in v1.5.0 

type SketchStoreLayer struct {
	// contains filtered or unexported fields
}

Layer 17: Sketch Store KV

func NewSketchStoreLayer added in v1.5.0 

func NewSketchStoreLayer() *SketchStoreLayer

NewSketchStoreLayer creates sketch store layer.

func (*SketchStoreLayer) Apply added in v1.5.0 

func (l *SketchStoreLayer) Apply(content string) (string, int)

func (*SketchStoreLayer) Name added in v1.5.0 

func (l *SketchStoreLayer) Name() string

func (*SketchStoreLayer) ShouldApply added in v1.5.0 

func (l *SketchStoreLayer) ShouldApply(contentType string) bool

type StringInternLayer added in v1.5.0 

type StringInternLayer struct {
	// contains filtered or unexported fields
}

Layer 26: String Interning

func NewStringInternLayer added in v1.5.0 

func NewStringInternLayer() *StringInternLayer

func (*StringInternLayer) Apply added in v1.5.0 

func (l *StringInternLayer) Apply(content string) (string, int)

func (*StringInternLayer) Name added in v1.5.0 

func (l *StringInternLayer) Name() string

func (*StringInternLayer) ShouldApply added in v1.5.0 

func (l *StringInternLayer) ShouldApply(contentType string) bool

type TruncateLayer added in v1.5.0 

type TruncateLayer struct{}

Layer 29: UUID/Hash Truncation

func NewTruncateLayer added in v1.5.0 

func NewTruncateLayer() *TruncateLayer

func (*TruncateLayer) Apply added in v1.5.0 

func (l *TruncateLayer) Apply(content string) (string, int)

func (*TruncateLayer) Name added in v1.5.0 

func (l *TruncateLayer) Name() string

func (*TruncateLayer) ShouldApply added in v1.5.0 

func (l *TruncateLayer) ShouldApply(contentType string) bool

type TurnRecord added in v1.5.0 

type TurnRecord struct {
	Timestamp    time.Time
	Command      string
	InputTokens  int
	OutputTokens int
	Cost         float64
	Model        string
	Provider     string
}

type URLLayer added in v1.5.0 

type URLLayer struct {
	// contains filtered or unexported fields
}

Layer 28: URL Shortening

func NewURLLayer added in v1.5.0 

func NewURLLayer() *URLLayer

func (*URLLayer) Apply added in v1.5.0 

func (l *URLLayer) Apply(content string) (string, int)

func (*URLLayer) Name added in v1.5.0 

func (l *URLLayer) Name() string

func (*URLLayer) ShouldApply added in v1.5.0 

func (l *URLLayer) ShouldApply(contentType string) bool

type WhitespaceLayer added in v1.5.0 

type WhitespaceLayer struct{}

Layer 25: Whitespace Normalization

func NewWhitespaceLayer added in v1.5.0 

func NewWhitespaceLayer() *WhitespaceLayer

func (*WhitespaceLayer) Apply added in v1.5.0 

func (l *WhitespaceLayer) Apply(content string) (string, int)

func (*WhitespaceLayer) Name added in v1.5.0 

func (l *WhitespaceLayer) Name() string

func (*WhitespaceLayer) ShouldApply added in v1.5.0 

func (l *WhitespaceLayer) ShouldApply(contentType string) bool

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.