Documentation
¶
Overview ¶
Package contextwindow estimates token usage for a message history. The numbers are approximate — exact tokenization depends on the model's tokenizer and varies enough between providers that "good enough for a watermark" is the operative bar.
Two callers consume this:
- The TUI status bar, which colors the token counter when usage crosses a configurable warn_threshold.
- The auto-summarization trigger, which fires before the next turn when usage crosses auto_threshold.
Both treat the numbers as ratios, not absolute counts, so a 10–15% estimation error is harmless.
The companion question — how big IS the model's window — lives in the catalog package (catalog.ResolveWindow / WindowFor): the window is a per-model/per-deployment fact sourced from the catalog, the user's override, or a live probe, none of which belong in this leaf estimation package (which would otherwise drag the catalog's config/network deps in, and form an import cycle via adapter).
Index ¶
- func DistributeBudget(demand []int, totalBudget, floor int) []int
- func EstimateImage(img adapter.ImageBlock) int
- func EstimateMessage(m adapter.Message) int
- func EstimateText(s string) int
- func EstimateTokens(messages []adapter.Message) int
- func EstimateToolSchemas(tools []adapter.Tool) int
- func SplitMessages(msgs []adapter.Message) (systemTokens, conversationTokens int)
- func ToolBudgetCaps(msgs []adapter.Message, budgetTokens, ceiling, floor int) (idxs, raw, caps []int)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DistributeBudget ¶
DistributeBudget computes, for each item's demand (its size if unconstrained), a capped allocation such that:
- no allocation exceeds that item's own demand (never inflate),
- no allocation drops below floor UNLESS the item's own demand is already below floor — an already-small item is never truncated further just to make room for others, and keeps its FULL demand regardless of totalBudget,
- subject to (1) and (2), the sum of allocations is <= totalBudget whenever achievable.
Water-filling / max-min fair sharing: items whose demand already fits an equal share of the remaining budget pass through untouched, and their unused slack rolls forward to items that need it.
Two distinct situations can make the returned sum exceed totalBudget — both are accepted trade-offs of protecting already-small items (2), not bugs:
- too many oversized (demand > floor) items for the budget: every one gets pushed down to floor, and floor*n alone may exceed totalBudget.
- many items whose OWN demand already sits below floor: each keeps its full (small) demand unconditionally per (2), so their combined total is whatever it naturally is — DistributeBudget will not shrink an already-small item further to help the sum fit, even though nothing else in the budget technically prevented it.
In both cases the alternative would be truncating some item to near-zero, which callers must not do; DistributeBudget always prefers leaving the sum over budget to producing a degenerate allocation.
Exists to replace a fixed per-item cap applied independently to each oversized item: capping N items to the same fixed ceiling bounds each one but not their sum, so a turn with many moderately-large items (e.g. 30+ tool results) could still retain several times a caller's intended total budget. DistributeBudget bounds the sum instead, in the common case where oversized items dominate.
func EstimateImage ¶ added in v0.4.0
func EstimateImage(img adapter.ImageBlock) int
EstimateImage returns the approximate token cost of a single image.
This deliberately does NOT scale with len(img.Data). A 4MB screenshot and a 200KB one of the same dimensions cost the model the same — byte size reflects compression, not context. Estimating from bytes would have scored one screenshot at a million tokens and triggered compaction constantly; ignoring images (the prior behavior) scored them at zero, so an image-heavy session could never trigger compaction at all. Pixel area is the measure both major providers actually bill on.
DecodeConfig reads only the header, so this stays cheap enough for the per-turn and status-bar call sites.
img.MediaType is honored: only formats with a registered header decoder can be dimension-read, and webp — which the read_file tool actively advertises — has none in the stdlib's image package. Rather than pull in a webp decoder (golang.org/x/image) just to read a width, we skip the sniff for any MediaType we can't decode and return the fallback. That's the same outcome the byte-sniff would have produced for an unregistered format, but explicit and stable rather than accidental.
func EstimateMessage ¶ added in v0.4.0
EstimateMessage returns the approximate token cost of one message — content, tool-call arguments, and any attached images.
Callers that walk long histories (compaction's retain-point search) use this per-message form to avoid building a slice per call.
func EstimateText ¶ added in v0.3.0
EstimateText returns an approximate token count for an arbitrary string under the same heuristic EstimateTokens uses. /context calls this for the system-prompt and memory-file buckets where the input isn't a Message slice.
func EstimateTokens ¶
EstimateTokens returns an approximate token count for the given messages. Tool-call argument JSON and tool-result content are counted; role metadata is not (it's bounded and small).
func EstimateToolSchemas ¶ added in v0.3.0
EstimateToolSchemas counts what the adapter advertises per tool: the tool name, description, and the JSON-serialized parameter schema. Mirrors what's actually sent on the wire so the /context Tools bucket matches the real cost.
Marshal errors fall back to a zero contribution for the offending tool — a malformed schema would have failed at registration time, so in practice this branch only fires for tools that pass nil/non-JSON schemas (which the adapter would reject anyway).
func SplitMessages ¶ added in v0.3.0
SplitMessages partitions a message slice into the system-prompt prefix and the conversation body, returning estimated token counts for each bucket. The split is by role: every RoleSystem message (typically just one) goes into the system bucket, everything else into conversation. /context charges the two to different buckets so the user can see the always-present system overhead separately from the growing chat history.
func ToolBudgetCaps ¶
func ToolBudgetCaps(msgs []adapter.Message, budgetTokens, ceiling, floor int) (idxs, raw, caps []int)
ToolBudgetCaps is the shared core of "retain this message tail, but keep its tool results within the remaining tail budget" — the exact scan + DistributeBudget wiring that internal/agent's compaction and internal/tui's /summarize compaction both need, factored out here after review flagged the two duplicating it independently (they still each own their own truncation marker text and mutation/return shape, which genuinely differ — only this numeric core is identical).
Scans msgs for RoleTool messages; every other role's tokens (except RoleSystem, never itself retained by either caller) count as already-spent overhead subtracted from budgetTokens before allocating. ceiling pre-clips any single tool message's demand before water-filling runs, so one oversized payload never dominates the tail even when the overall budget happens to have slack for it; floor is the per-message floor passed straight to DistributeBudget.
Returns, in encounter order, each tool message's index within msgs, its actual (uncapped) token size, and its computed cap — idxs/raw/caps are parallel slices of the same length. A caller truncates msgs[idxs[k]] to caps[k] only when raw[k] > caps[k]; raw is returned so that check never needs to re-estimate a message it already scanned once here. When the floor makes the budget infeasible, caps may still exceed the remaining tool budget; this mirrors DistributeBudget's documented trade-off of preserving a meaningful minimum fragment rather than truncating results to near-zero. All three slices are nil when msgs contains no tool messages.
Types ¶
This section is empty.
Source Files