Documentation
¶
Overview ¶
Package compress wires reversible context compression (ctxzip) into the forge agent loop.
ctxzip shrinks bulky content before it reaches the LLM — tool outputs, logs, JSON — and offloads everything it drops to a durable local store, replaced inline by a retrievable "<<ctxzip:HASH ...>>" marker. Compression is lossy on the wire but lossless end-to-end: the model can recover any original via the context_expand tool.
Three integration seams, all owned by Runtime:
- AfterToolExecHook — compresses tool output once, at production time, before it enters Memory. Because the compressed bytes never change afterwards, the conversation prefix stays byte-stable across turns and provider prompt caches keep hitting.
- WrapClient — an llm.Client decorator that compresses the live zone of every outbound request (skipping the frozen prefix and recent turns). It is deliberately deterministic: the relevance query is pinned to the first user message of the session, never the latest turn, so the same historic message always compresses to the same bytes.
- ExpandTool — the context_expand builtin that retrieves originals from the store by marker hash.
The store is a bbolt file (default .forge/ctxzip.db) so originals survive process restarts; entries expire after TTL, at which point the disk or the original command is the source of truth (the tool's miss message says so).
Index ¶
- Constants
- func SuggestionsPath(storePath string) string
- type AuditFunc
- type Config
- type PatternStat
- type Runtime
- func (r *Runtime) AfterToolExecHook() runtime.Hook
- func (r *Runtime) Close() error
- func (r *Runtime) ExpandTool() tools.Tool
- func (r *Runtime) Store() ccr.Store
- func (r *Runtime) Suggestions() []PatternStat
- func (r *Runtime) TakeInvocationTotals(ctx context.Context) SavingsTotals
- func (r *Runtime) Totals() SavingsTotals
- func (r *Runtime) WrapClient(inner llm.Client) llm.Client
- type SavingsTotals
Constants ¶
const ( // AuditEventCompressed fires whenever content is compressed, from either // seam (tool_output hook or request wrapper). Fields: seam, tool, // tokens_before, tokens_after, saved_tokens, plus running totals // total_saved_tokens / total_compressions / total_expansions so any // single event shows the cumulative savings picture. AuditEventCompressed = "context_compressed" // AuditEventExpanded fires when the model retrieves offloaded content // via context_expand. Fields: hash, hit, bytes, plus the same running // totals — expansions are the "cost" side auditors net against savings. AuditEventExpanded = "context_expanded" )
Audit event names emitted by the compression runtime.
const ( DefaultTTL = 30 * time.Minute DefaultMinToolOutputChars = 2048 )
Defaults for Config fields left at their zero value.
const AuditEventPatternSuggested = "context_pattern_suggested"
AuditEventPatternSuggested fires once per pattern when it crosses suggestThreshold. Fields: pattern, expansions, tools.
const (
// SuggestionsFileName sits next to the CCR store under .forge/.
SuggestionsFileName = "ctxzip-suggestions.json"
)
const SystemDirective = `## Compressed context
Large tool outputs may be automatically compressed to fit your context.
Compressed sections are replaced inline by a marker like
<<ctxzip:HASH N_lines_offloaded>> — the note says how much was offloaded. The
visible remainder keeps errors, anomalies, and representative content, so for
many questions you will not need the offloaded part.
When you DO need offloaded data to answer precisely (exact counts, full
listings, a specific record you cannot see), call the ` + expandToolName + ` tool with the
marker's hash to retrieve the original content. If it reports the content
expired, re-run the tool that produced the output.`
SystemDirective is appended to the agent's system prompt whenever compression is enabled, so EVERY skill gets marker-awareness from the runtime — skill authors never need to document compression themselves. The text is constant, which keeps the system prompt byte-stable across turns (provider prompt caches stay warm).
Variables ¶
This section is empty.
Functions ¶
func SuggestionsPath ¶
SuggestionsPath returns the flywheel file for a given store path.
Types ¶
type AuditFunc ¶
AuditFunc receives compression audit events. The runner wires this to the AuditLogger (EmitFromContext, so correlation_id/task_id are stamped from ctx); nil disables audit emission. Token figures are tokenizer estimates, not provider-billed counts — directionally accurate for savings reporting.
type Config ¶
type Config struct {
// StorePath is the bbolt file for the CCR store (required),
// e.g. .forge/ctxzip.db.
StorePath string
// TTL is how long offloaded originals stay retrievable. Default 30m.
TTL time.Duration
// MinToolOutputChars is the size below which tool outputs are left
// alone by the AfterToolExec hook. Default 2048.
MinToolOutputChars int
// KeepPatterns is the builder's domain vocabulary of case-insensitive
// substrings compression must never drop (forge.yaml
// compression.keep_patterns). Union with ctxzip's built-in error floor.
KeepPatterns []string
// Logger is optional; nil disables logging.
Logger runtime.Logger
// Audit is optional; nil disables audit emission. See AuditFunc.
Audit AuditFunc
}
Config configures the compression runtime.
type PatternStat ¶
type PatternStat struct {
// Pattern preserves the first-seen casing (what the operator would put
// in keep_patterns).
Pattern string `json:"pattern"`
// Expansions counts DISTINCT expansion events whose retrieved content
// contained the token — not occurrences within one retrieval, and not
// repeat retrievals of the same content hash.
Expansions int `json:"expansions"`
Tools []string `json:"tools,omitempty"`
Suggested bool `json:"suggested"`
LastSeen time.Time `json:"last_seen"`
}
PatternStat is one tracked keep-pattern candidate.
type Runtime ¶
type Runtime struct {
// contains filtered or unexported fields
}
Runtime owns the shared CCR store and produces the hook, client wrapper, and expand tool that plug into the agent loop.
func (*Runtime) AfterToolExecHook ¶
AfterToolExecHook returns a hook that compresses tool output at production time, before the loop appends it to Memory. This is the primary compression seam: because the output is compressed exactly once, the bytes stored in history never change afterwards, keeping the conversation prefix stable for provider prompt caches.
Register it AFTER redaction/guardrail hooks so it compresses what those hooks left, not what they were about to remove.
The hook never fails the loop: on any problem it leaves ToolOutput as-is. Error results (hctx.Error != nil) are always left verbatim — dropping parts of an error the user is about to debug is the catastrophic failure mode.
func (*Runtime) ExpandTool ¶
ExpandTool returns the context_expand tool backed by this Runtime's store. Register it conditionally (like memory_get) — only when compression is on.
func (*Runtime) Suggestions ¶
func (r *Runtime) Suggestions() []PatternStat
Suggestions exposes the tracked keep-pattern candidates (for the CLI).
func (*Runtime) TakeInvocationTotals ¶
func (r *Runtime) TakeInvocationTotals(ctx context.Context) SavingsTotals
TakeInvocationTotals pops and returns the savings accumulated under the ctx's correlation ID. Call it exactly once, at the invocation's response boundary (invocation_complete emission); subsequent calls for the same invocation return zeros. Safe under concurrent invocations — each correlation ID accumulates independently.
func (*Runtime) Totals ¶
func (r *Runtime) Totals() SavingsTotals
Totals returns a snapshot of the cumulative savings picture.
func (*Runtime) WrapClient ¶
WrapClient decorates an llm.Client so every outbound request has its live zone compressed. It sits below the FallbackChain, so it also covers retry calls and the compactor's summarization call.
Cache discipline ("passthrough is sacred"): only the live zone is touched — the system prompt (frozen prefix) and the most recent turns are forwarded byte-identical. Determinism matters just as much: the relevance query is pinned to the FIRST user message of the conversation, never the latest turn. Deriving it from the latest turn would recompress the same historic message to different bytes each turn and bust the provider prompt cache.
type SavingsTotals ¶
type SavingsTotals struct {
// Compressions is how many times content was compressed (either seam).
Compressions int64
// SavedTokens is the cumulative per-EVENT token reduction — counted once
// per compression, at compression time.
SavedTokens int64
// WireSavedTokens is the cumulative REALIZED reduction: every time a
// marker rides in an outbound LLM request, that marker's saved tokens are
// tokens this request did not send. A tool output compressed once but
// resent in history across ten calls saves its delta ten times — this is
// the number that matches the provider's bill (live finding: an
// invocation reporting 1,257 event-saved tokens had actually avoided
// ~31K billed tokens through history compounding).
WireSavedTokens int64
// Expansions / ExpansionMisses count context_expand retrievals — the
// cost side to net against savings.
Expansions int64
ExpansionMisses int64
}
SavingsTotals is the process-lifetime savings picture. Token figures are tokenizer estimates.
Source Files