optimize

package
v0.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 16, 2026 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

Package optimize provides LLM-powered config compression.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CountTokens

func CountTokens(content string) int

CountTokens estimates the token count for content using runes/4 approximation. This provides a reasonable estimate for Claude's tokenizer without requiring external dependencies. Uses rune count (not byte count) to handle unicode correctly.

func ExtractAnchors

func ExtractAnchors(content string) []string

ExtractAnchors finds critical content that must be preserved during optimization. This includes tool names, file paths, commands, and other specific identifiers. For categorized extraction with different validation strictness, use ExtractCategorizedAnchors.

func HashContent

func HashContent(content string) string

HashContent generates a SHA256 hash for content.

func ValidateAnchors

func ValidateAnchors(original, optimized string) ([]string, []string)

ValidateAnchors checks if all anchors from the original content exist in optimized content. Returns (preserved, missing) anchor lists. For categorized validation with different strictness, use ValidateAnchorsCategorized.

Types

type AnchorCategory

type AnchorCategory int

AnchorCategory indicates how strict validation should be for an anchor type.

const (
	// AnchorStrict means missing anchors should fail validation (file paths, commands).
	AnchorStrict AnchorCategory = iota
	// AnchorSoft means missing anchors generate warnings but don't fail (tool names).
	AnchorSoft
)

type CategorizedAnchors

type CategorizedAnchors struct {
	Strict []string // File paths, commands - must be preserved
	Soft   []string // Tool names - warn if missing but don't fail
}

CategorizedAnchors holds anchors grouped by their validation strictness.

func ExtractCategorizedAnchors

func ExtractCategorizedAnchors(content string) *CategorizedAnchors

ExtractCategorizedAnchors finds critical content and categorizes by validation strictness.

func (*CategorizedAnchors) All

func (c *CategorizedAnchors) All() []string

All returns all anchors as a flat list (for backward compatibility).

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client handles communication with the Claude API.

func NewClient

func NewClient(opts ...ClientOption) (*Client, error)

NewClient creates a new Claude API client. It reads the API key from the ANTHROPIC_API_KEY environment variable.

func (*Client) Optimize

func (c *Client) Optimize(ctx context.Context, content string, targetTokens int, anchors []string) (string, error)

Optimize sends content to Claude for optimization.

type ClientOption

type ClientOption func(*Client)

ClientOption configures a Client.

func WithBaseURL

func WithBaseURL(url string) ClientOption

WithBaseURL sets the API base URL.

func WithHTTPClient

func WithHTTPClient(client *http.Client) ClientOption

WithHTTPClient sets a custom HTTP client.

func WithModel

func WithModel(model string) ClientOption

WithModel sets the model to use.

type Message

type Message struct {
	Role    string `json:"role"`
	Content string `json:"content"`
}

Message represents a message in the Claude API.

type OptimizationCache

type OptimizationCache struct {
	// contains filtered or unexported fields
}

OptimizationCache manages cached optimization results.

func NewOptimizationCache

func NewOptimizationCache(paths *config.Paths) *OptimizationCache

NewOptimizationCache creates a new cache.

func (*OptimizationCache) Clear

func (c *OptimizationCache) Clear(owner, repo string) error

Clear removes cached optimization for a repo.

func (*OptimizationCache) Exists

func (c *OptimizationCache) Exists(owner, repo string) bool

Exists checks if an optimization cache entry exists.

func (*OptimizationCache) IsStale

func (c *OptimizationCache) IsStale(owner, repo, sourceHash string) bool

IsStale checks if the cached optimization is stale based on source hash.

func (*OptimizationCache) ListCached

func (c *OptimizationCache) ListCached() ([]string, error)

ListCached returns all cached optimizations.

func (*OptimizationCache) Read

func (c *OptimizationCache) Read(owner, repo string) (string, *OptimizationMeta, error)

Read retrieves cached optimization result and metadata. Returns empty string and nil metadata if not found. If content exists but metadata is missing/corrupted, cleans up orphaned files.

func (*OptimizationCache) ReadMeta

func (c *OptimizationCache) ReadMeta(owner, repo string) (*OptimizationMeta, error)

ReadMeta retrieves only the metadata for an optimization.

func (*OptimizationCache) Write

func (c *OptimizationCache) Write(owner, repo, content string, meta *OptimizationMeta) error

Write stores an optimization result with metadata.

type OptimizationMeta

type OptimizationMeta struct {
	SourceHash      string    `json:"source_hash"`
	OptimizedAt     time.Time `json:"optimized_at"`
	OriginalTokens  int       `json:"original_tokens"`
	OptimizedTokens int       `json:"optimized_tokens"`
	Model           string    `json:"model"`
	Deterministic   bool      `json:"deterministic"`
}

OptimizationMeta tracks optimization state for cached results.

type Optimizer

type Optimizer struct {
	// contains filtered or unexported fields
}

Optimizer handles the optimization pipeline.

func NewOptimizer

func NewOptimizer(paths *config.Paths) *Optimizer

NewOptimizer creates a new optimizer.

func (*Optimizer) Optimize

func (o *Optimizer) Optimize(ctx context.Context, content string, owner, repo string, opts Options) (*Result, error)

Optimize runs the full optimization pipeline on content.

func (*Optimizer) SetClient

func (o *Optimizer) SetClient(client *Client)

SetClient allows injecting a client for testing.

type Options

type Options struct {
	Target        int    // Target token count (0 = auto ~50% reduction)
	Deterministic bool   // Only apply deterministic transforms (no LLM)
	Force         bool   // Re-optimize even if cache is fresh
	NoCache       bool   // Skip cache read/write
	Model         string // Model to use for optimization
}

Options controls optimization behavior.

type PreprocessStats

type PreprocessStats struct {
	BlankLinesRemoved int
	DuplicatesRemoved int
	PhrasesStripped   int
}

PreprocessStats tracks what changes were made during preprocessing.

func Preprocess

func Preprocess(content string) (string, PreprocessStats)

Preprocess performs deterministic cleanup on content without using an LLM. It normalizes whitespace, removes duplicate rules, and strips verbose phrases.

type Result

type Result struct {
	OriginalContent  string
	OptimizedContent string
	Stats            TokenStats
	PreprocessStats  PreprocessStats
	PreservedAnchors []string
	MissingAnchors   []string // All missing (strict + soft), for backward compatibility
	MissingStrict    []string // Strict anchors missing (file paths, commands) - causes failure
	MissingSoft      []string // Soft anchors missing (tool names) - warnings only
	FromCache        bool
	Deterministic    bool
}

Result contains the optimization outcome.

type TokenStats

type TokenStats struct {
	Before int
	After  int
}

TokenStats holds before/after token statistics.

func (TokenStats) PercentReduction

func (s TokenStats) PercentReduction() float64

PercentReduction returns the percentage reduction (0-100).

func (TokenStats) Saved

func (s TokenStats) Saved() int

Saved returns the number of tokens saved.

type ValidationResult

type ValidationResult struct {
	Preserved     []string // Anchors that were preserved
	MissingStrict []string // Strict anchors that are missing (should fail)
	MissingSoft   []string // Soft anchors that are missing (warnings only)
}

ValidationResult contains the results of anchor validation.

func ValidateAnchorsCategorized

func ValidateAnchorsCategorized(original, optimized string) *ValidationResult

ValidateAnchorsCategorized checks anchors with different strictness levels.

func (*ValidationResult) AllMissing

func (v *ValidationResult) AllMissing() []string

AllMissing returns all missing anchors (for backward compatibility).

func (*ValidationResult) HasStrictFailures

func (v *ValidationResult) HasStrictFailures() bool

HasStrictFailures returns true if any strict anchors are missing.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL