Documentation
¶
Overview ¶
Package contextplan manages token budget windows and compaction. Compact and Calibrated adapt provider messages to bounded context windows. See docs/plans/contextplan.md.
Index ¶
Constants ¶
const ( // DefaultSmoothingFactor is the EWMA weight Observe applies when // Calibrate receives a non-positive alpha. DefaultSmoothingFactor = 0.3 // MinCorrectionFactor is the floor Observe clamps the correction // factor to. MinCorrectionFactor = 0.5 // MaxCorrectionFactor is the ceiling Observe clamps the correction // factor to. MaxCorrectionFactor = 2.0 )
EWMA smoothing bounds for Calibrated.
const ( // DefaultTriggerPercent compacts at this percent of Window.Budget(). DefaultTriggerPercent = 100 // DefaultTargetPercent compacts down to this percent of Budget(). DefaultTargetPercent = 10 // DefaultRecentTail is the message-count bound of the tail fill. DefaultRecentTail = 8 // MaxRecentTail is the highest tail bound a caller may set. MaxRecentTail = 64 // CompactionAlgorithm names the idempotency-key fingerprint scheme. CompactionAlgorithm = "context-compact-v1" )
Compaction thresholds and bounds.
Variables ¶
var ( // ErrNoMessages is Compact's error for an empty message list. ErrNoMessages = errors.New("contextplan: no messages to compact") // ErrEstimateFailed is Compact's error when the token estimator // fails. ErrEstimateFailed = errors.New("contextplan: token estimate failed") // ErrRetentionOverflow is Compact's error when the retention set // alone exceeds the window budget. ErrRetentionOverflow = errors.New("contextplan: retention set alone exceeds the window") // ErrNoObjective is Compact's error when no user message exists to // retain as the objective. ErrNoObjective = errors.New("contextplan: no user message to retain as objective") )
Sentinel errors for Compact; test with errors.Is.
var ( // ErrMaxTokensNotPositive is Validate's error when MaxTokens <= 0. ErrMaxTokensNotPositive = errors.New("contextplan: max tokens must be positive") // ErrReserveNegative is Validate's error when Reserve < 0. ErrReserveNegative = errors.New("contextplan: reserve must not be negative") // ErrReserveTooLarge is Validate's error when Reserve >= MaxTokens. ErrReserveTooLarge = errors.New("contextplan: reserve must be less than max tokens") )
Sentinel errors for Window.Validate; test with errors.Is.
Functions ¶
This section is empty.
Types ¶
type Calibrated ¶
type Calibrated struct {
// contains filtered or unexported fields
}
Calibrated wraps a provider.TokenEstimator with an exponentially weighted moving average, corrected after each completed turn through Observe. A *Calibrated implements provider.TokenEstimator. Safe for concurrent use: one mutex guards factor, the only mutable field.
func Calibrate ¶
func Calibrate(est provider.TokenEstimator, alpha float64) *Calibrated
Calibrate wraps est with an EWMA correction factor. alpha is the EWMA smoothing weight in (0, 1]; a value outside that range, including zero or negative, falls back to DefaultSmoothingFactor. A nil est is a caller error caught at the first EstimateTokens call, not at construction, matching the wrapped interface's own contract.
func (*Calibrated) EstimateTokens ¶
func (c *Calibrated) EstimateTokens(req provider.Request) (int, error)
EstimateTokens calls the wrapped estimator, then scales the result by the current EWMA correction factor, itself always within [MinCorrectionFactor, MaxCorrectionFactor].
func (*Calibrated) Observe ¶
func (c *Calibrated) Observe(estimated, actual int)
Observe records one completed turn: estimated is the value EstimateTokens returned to the caller for that turn - the already-corrected figure, not a raw pre-scale count - and actual is the real provider.Usage.TotalTokens for the same turn. Observe corrects factor multiplicatively against estimated, so the fixed point is unchanged from before this fix: a corrected estimate that tracks actual. The result is clamped to [MinCorrectionFactor, MaxCorrectionFactor]. A non-positive estimated or a non-positive actual is a no-op.
type CompactResult ¶
type CompactResult struct {
Kept []provider.Message
Dropped []provider.Message
BeforeTokens int
AfterTokens int
TriggerTokens int
TargetTokens int
Compacted bool
Key string
}
CompactResult is Compact's output.
func Compact ¶
func Compact(msgs []provider.Message, w Window, e provider.TokenEstimator) (CompactResult, error)
Compact applies the trigger check and the retention policy. An invalid window fails Window.Validate before any estimate. A request at or above the trigger compacts; below it passes through with Compacted false. The retention set is mandatory; the tail fill is optional and stops at the first unit that breaks contiguity, the message-count bound, or the target. Kept preserves the original relative order. The Key is deterministic per input.
type Compaction ¶
type Compaction struct {
TriggerPercent int
TargetPercent int
TargetTokens int
RecentTail int
PreserveNames []string
}
Compaction configures compaction thresholds and retention. The zero value means the defaults, never "disabled": TriggerPercent zero means DefaultTriggerPercent, TargetPercent zero means DefaultTargetPercent, RecentTail zero means DefaultRecentTail.
func (Compaction) Validate ¶
func (c Compaction) Validate() error
Validate rejects percents outside [0, 100], a negative TargetTokens or RecentTail, a RecentTail over MaxRecentTail, an empty but present PreserveNames entry, and duplicate PreserveNames entries. When TargetTokens is zero, a TargetPercent at or above the resolved TriggerPercent is rejected; when TargetTokens is positive, that comparison is skipped and Window.Validate instead rejects a TargetTokens at or above Budget().
type Window ¶
type Window struct {
MaxTokens int
Reserve int
Compaction Compaction
}
Window is the token budget for one planned request. MaxTokens is the model's context window; Reserve is the headroom Plan never spends, held back for the model's own reply. Compaction carries the compaction thresholds and retention rules; its zero value means the defaults, never "disabled".
func (Window) Budget ¶
Budget returns MaxTokens - Reserve, the tokens Plan may spend on Request.Messages.
func (Window) CompactTarget ¶
CompactTarget returns the target in tokens: TargetTokens when positive, else Budget times TargetPercent, floored.
func (Window) CompactTrigger ¶
CompactTrigger returns the trigger in tokens: Budget times TriggerPercent, floored.
Source Files