README
¶
internal/protocol/usage
Centralized token extraction and normalization. All handlers call into this package instead of re-implementing provider rules inline.
Normalization rules
The normalization table lives in .design/stream-usage-tracking.md §2, together with the per-field containment rules, the cross-provider invariants, and the gpt-5.6 cache-write background (§12). It is deliberately not duplicated here — this file previously carried a second copy that went stale the moment cache writes landed.
The one-line version, for orientation:
InputTokens = uncached + written (OpenAI subtracts reads; Anthropic adds creation)
CacheInputTokens = cache reads
CacheWriteTokens = cache writes, a SUBSET of InputTokens — never add it on top
cache_hit_ratio = CacheInputTokens / (InputTokens + CacheInputTokens)
Anthropic streaming event split
Anthropic splits usage across two events:
message_start→input_tokens,cache_creation_input_tokens,cache_read_input_tokensmessage_delta→output_tokens(some non-standard providers also sendinput_tokenshere)
AnthropicAccumulator handles this split, priority fallback, and normalization transparently.
API
Non-streaming (pure functions)
usage.FromOpenAIChatCompletion(resp.Usage) // openai.CompletionUsage
usage.FromOpenAIResponses(resp.Usage) // responses.ResponseUsage
usage.FromAnthropicMessage(resp.Usage) // anthropic.Usage
usage.FromAnthropicBetaMessage(resp.Usage) // anthropic.BetaUsage
Streaming — Anthropic accumulator
acc := usage.NewAnthropicAccumulator()
// In event loop:
acc.Consume(&evt) // MessageStreamEventUnion (non-beta)
acc.ConsumeBeta(&evt) // BetaRawMessageStreamEventUnion (beta)
// At return:
if acc.HasUsage() {
return acc.Result(), nil
}
return protocol.ZeroTokenUsage(), nil
Coverage
internal/protocol/nonstream/
| Function | Extractor |
|---|---|
HandleOpenAIChatNonStream |
FromOpenAIChatCompletion |
HandleOpenAIResponsesNonStream |
FromOpenAIResponses |
HandleAnthropicV1NonStream |
FromAnthropicMessage |
HandleAnthropicV1BetaNonStream |
FromAnthropicBetaMessage |
internal/protocol/stream/
| Function | Mechanism |
|---|---|
HandleAnthropic |
AnthropicAccumulator.Consume |
HandleAnthropicBeta |
AnthropicAccumulator.ConsumeBeta |
AnthropicToOpenAIStreamWithMCPHooks |
AnthropicAccumulator.ConsumeBeta |
HandleAnthropicBetaToOpenAIResponsesStream |
AnthropicAccumulator.ConsumeBeta |
internal/server/ (dispatch layer)
| Code site | Extractor |
|---|---|
protocol_dispatch — Anthropic Beta non-stream (×2) |
FromAnthropicBetaMessage |
protocol_dispatch — Responses → Anthropic Beta |
FromAnthropicBetaMessage |
protocol_dispatch — OpenAI Chat non-stream (×2) |
FromOpenAIChatCompletion |
protocol_dispatch — OpenAI Responses non-stream (×2) |
FromOpenAIResponses |
anthropic_message_v1 — Responses → Anthropic v1 |
FromOpenAIResponses |
anthropic_message_beta — Responses → Anthropic Beta |
FromOpenAIResponses |
Intentional inline extraction (not migrated)
| File | Reason |
|---|---|
stream/openai_passthrough.go |
Per-chunk accumulation + estimated usage injection fallback |
stream/openai_to_anthropic*.go |
Uses StreamTokenCounter (incremental counting, not extraction) |
stream/openai_{chat,responses}_to_*.go |
state fields are dual-use: also build the wire response body |
stream/google_to_any.go |
Google SDK has no structured cache sub-fields |
nonstream/anthropic_to_openai.go |
Returns wire format (map[string]interface{}), not *TokenUsage |
nonstream/openai_to_anthropic.go |
Same |
server/protocol_dispatch — Google non-stream |
Google schema, no cached tokens in SDK struct |
Documentation
¶
Overview ¶
Package usage centralizes token extraction and normalization logic for all supported provider protocols. Every handler calls into this package instead of re-implementing provider-specific rules inline.
Normalization rules:
- OpenAI (Chat / Responses): prompt_tokens = total (cached + written + uncached). Store inputTokens = total - cached so the frontend ratio formula gives cache_read / (cache_read + uncached) = correct hit rate. cache_write_tokens (gpt-5.6+) stays inside inputTokens because it is billed at a premium rate, and is also reported as CacheWriteTokens.
- Anthropic: input_tokens = uncached only; cache_creation_input_tokens is an additional write cost that belongs in the denominator. Store inputTokens = input + creation so the formula covers total prompt cost.
Both sides therefore agree: inputTokens = uncached + written, and CacheWriteTokens is a subset of inputTokens, never an addition to it.
Index ¶
- func ChatUsage(u *protocol.TokenUsage) openai.CompletionUsage
- func FromAnthropicBetaMessage(u anthropic.BetaUsage) *protocol.TokenUsage
- func FromAnthropicMessage(u anthropic.Usage) *protocol.TokenUsage
- func FromOpenAIChatCompletion(u openai.CompletionUsage) *protocol.TokenUsage
- func FromOpenAIResponses(u responses.ResponseUsage) *protocol.TokenUsage
- func ToChatStreamUsageWire(u *protocol.TokenUsage) *wire.ChatStreamUsage
- func ToChatUsageWire(u *protocol.TokenUsage) wire.ChatCompletionUsageWire
- func ToResponsesUsageWire(u *protocol.TokenUsage) *wire.ResponsesUsageWire
- type AnthropicAccumulator
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ChatUsage ¶
func ChatUsage(u *protocol.TokenUsage) openai.CompletionUsage
ChatUsage converts normalized TokenUsage into an OpenAI Chat Completions CompletionUsage wire struct. OpenAI wire semantics: PromptTokens = TOTAL (uncached + cached), CachedTokens is a reported subset.
func FromAnthropicBetaMessage ¶
func FromAnthropicBetaMessage(u anthropic.BetaUsage) *protocol.TokenUsage
FromAnthropicBetaMessage extracts normalized TokenUsage from an Anthropic beta BetaMessage usage block. Same normalization as the non-beta path.
func FromAnthropicMessage ¶
func FromAnthropicMessage(u anthropic.Usage) *protocol.TokenUsage
FromAnthropicMessage extracts normalized TokenUsage from an Anthropic v1 (non-beta) Message usage block. CacheCreationInputTokens is added to InputTokens so the denominator covers all non-cache-read prompt cost.
func FromOpenAIChatCompletion ¶
func FromOpenAIChatCompletion(u openai.CompletionUsage) *protocol.TokenUsage
FromOpenAIChatCompletion extracts normalized TokenUsage from an OpenAI Chat Completions usage block. CachedTokens and CacheWriteTokens are disjoint SUBSETS of PromptTokens. Only the read hits are subtracted: writes are billed at 1.25x the uncached input rate (gpt-5.6+), so they stay inside InputTokens and are reported separately for cost attribution — mirroring how Anthropic's cache_creation_input_tokens is folded in.
func FromOpenAIResponses ¶
func FromOpenAIResponses(u responses.ResponseUsage) *protocol.TokenUsage
FromOpenAIResponses extracts normalized TokenUsage from an OpenAI Responses API usage block. Same semantics as Chat: InputTokens = total, CachedTokens and CacheWriteTokens are subsets.
func ToChatStreamUsageWire ¶ added in v0.260806.1
func ToChatStreamUsageWire(u *protocol.TokenUsage) *wire.ChatStreamUsage
ToChatStreamUsageWire converts normalized TokenUsage into the Chat Completions stream usage wire shape.
func ToChatUsageWire ¶ added in v0.260806.1
func ToChatUsageWire(u *protocol.TokenUsage) wire.ChatCompletionUsageWire
ToChatUsageWire converts normalized TokenUsage into the non-streaming Chat Completions usage wire shape.
func ToResponsesUsageWire ¶ added in v0.260806.1
func ToResponsesUsageWire(u *protocol.TokenUsage) *wire.ResponsesUsageWire
ToResponsesUsageWire converts normalized TokenUsage into the Responses API usage wire shape.
Types ¶
type AnthropicAccumulator ¶
type AnthropicAccumulator struct {
// contains filtered or unexported fields
}
AnthropicAccumulator accumulates token usage across a streaming Anthropic response. The Anthropic protocol splits usage across two event types:
- message_start → input_tokens, cache_creation, cache_read
- message_delta → output_tokens (occasionally input for non-standard providers)
Both non-beta (MessageStreamEventUnion) and beta (BetaRawMessageStreamEventUnion) streams are supported via Consume and ConsumeBeta respectively.
func NewAnthropicAccumulator ¶
func NewAnthropicAccumulator() *AnthropicAccumulator
NewAnthropicAccumulator returns a zeroed accumulator ready to consume events.
func (*AnthropicAccumulator) Consume ¶
func (a *AnthropicAccumulator) Consume(evt *anthropic.MessageStreamEventUnion)
Consume updates the accumulator from a non-beta streaming event. It is safe to call on every event in the stream; only usage-carrying events (message_start, message_delta) have any effect.
func (*AnthropicAccumulator) ConsumeBeta ¶
func (a *AnthropicAccumulator) ConsumeBeta(evt *anthropic.BetaRawMessageStreamEventUnion)
ConsumeBeta updates the accumulator from a beta streaming event.
func (*AnthropicAccumulator) HasUsage ¶
func (a *AnthropicAccumulator) HasUsage() bool
HasUsage reports whether any non-zero usage was observed.
func (*AnthropicAccumulator) Result ¶
func (a *AnthropicAccumulator) Result() *protocol.TokenUsage
Result returns the normalized TokenUsage built from accumulated events.
Source Files