Documentation
¶
Overview ¶
Package compress implements the prompt-compression algorithm GrayCode native prompt-compression implementation.
Algorithm overview:
- Split input into segments (sentences, preserving code/URLs).
- Per segment, run the auto-clarity check: segments containing security/destructive keywords are passed through verbatim.
- For safe segments, apply the dictionary (eng/v1) first (longest-match-first phrase substitutions).
- Then apply the drop-list for the chosen intensity level (Lite/Full/Ultra).
- Re-join segments, preserving original whitespace.
Package compress implements the prompt-compression algorithm GrayCode native prompt-compression implementation.
The algorithm is a pure text-rewrite engine. Three intensity levels (Lite, Full, Ultra) progressively drop articles, filler words, pleasantries, hedging, and redundant conjunctions. A built-in dictionary (eng/v1) substitutes verbose phrases with terse equivalents.
Auto-clarity rules (see safety.go) prevent compression of security warnings, destructive operations, and code-bearing segments.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func IsSafeSegment ¶
IsSafeSegment returns true if s contains no safety keywords. A segment with safety keywords must be passed through unchanged.
func SplitSafeSegments ¶
func SplitSafeSegments(s string) []segment
SplitSafeSegments splits s into segments that are safe to compress and segments that must be preserved verbatim. Returns a slice of (text, safe) pairs. Adjacent safe/unsafe segments are merged by their flag.
func SubstituteDict ¶
SubstituteDict applies the eng/v1 dictionary to s, replacing verbose phrases with terse equivalents. Case-insensitive on keys; preserves the original casing of the first character of the match.
Multi-word keys are matched as whole phrases. The longest match is preferred to avoid partial replacements.
Types ¶
type CompressResult ¶
type CompressResult struct {
// Original is the input verbatim.
Original string
// Compressed is the rewritten text.
Compressed string
// Stats for the operation.
Stats Stats
}
CompressResult reports what the compression did to the input.
type Intensity ¶
type Intensity int
Intensity controls the aggressiveness of prompt compression.
The intensity ordering is Lite < Full < Ultra. Higher intensity drops more word classes and applies more aggressive dictionary substitutions.
type Stats ¶
type Stats struct {
Intensity Intensity
OriginalBytes int
CompressedBytes int
OriginalWords int
CompressedWords int
BytesSaved int
PercentOff float64
DroppedArticles int
DroppedFiller int
DroppedHedge int
DroppedConjunctions int
DictionaryHits int
PassThroughSegments int
CompressedSegments int
BytesSavedByDict int
BytesSavedByDrops int
RefusedDueToSensitive bool
// Per-segment counts (informational; first N at most).
SensitiveKeywordsHit []string
}
Stats reports per-stage counts.
func Compress ¶
Compress applies the algorithm to s at the given intensity.
Behavior:
- Empty input returns (input, empty result) untouched.
- CJK-heavy text is passed through (compression rules assume Latin grammar).
- Sensitive segments (security/destructive keywords) are preserved verbatim regardless of intensity.
- Multi-word drop-list entries ("of course", "feel free") are matched as whole phrases.
- Dictionary substitutions are case-insensitive and preserve the case of the first character of each match.
Source Files