Documentation
¶
Index ¶
- func CountBetaTokensViaTiktoken(req *anthropic.BetaMessageCountTokensParams) (int, error)
- func CountTokensViaTiktoken(req *anthropic.MessageCountTokensParams) (int, error)
- func EstimateInputTokens(req *openai.ChatCompletionNewParams) (int, error)
- func EstimateOutputTokens(content string) int
- type StreamTokenCounter
- func (c *StreamTokenCounter) AddInputTokens(tokens int)
- func (c *StreamTokenCounter) AddToOutputTokens(tokens int)
- func (c *StreamTokenCounter) ConsumeOpenAIChunk(chunk *openai.ChatCompletionChunk) (inputTokens, outputTokens int, err error)
- func (c *StreamTokenCounter) CountText(text string) int
- func (c *StreamTokenCounter) EstimateInputTokens(req *openai.ChatCompletionNewParams) (int, error)
- func (c *StreamTokenCounter) GetCounts() (inputTokens, outputTokens int)
- func (c *StreamTokenCounter) InputTokens() int
- func (c *StreamTokenCounter) OutputTokens() int
- func (c *StreamTokenCounter) Reset()
- func (c *StreamTokenCounter) SetInputTokens(tokens int)
- func (c *StreamTokenCounter) SetOutputTokens(tokens int)
- func (c *StreamTokenCounter) TotalTokens() int
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CountBetaTokensViaTiktoken ¶ added in v0.260204.1200
func CountBetaTokensViaTiktoken(req *anthropic.BetaMessageCountTokensParams) (int, error)
CountBetaTokensViaTiktoken approximates token count for OpenAI-style providers using tiktoken
func CountTokensViaTiktoken ¶ added in v0.260204.1200
func CountTokensViaTiktoken(req *anthropic.MessageCountTokensParams) (int, error)
CountTokensViaTiktoken approximates token count for OpenAI-style providers using tiktoken
func EstimateInputTokens ¶
func EstimateInputTokens(req *openai.ChatCompletionNewParams) (int, error)
EstimateInputTokens estimates input tokens from OpenAI request using tiktoken
func EstimateOutputTokens ¶
EstimateOutputTokens estimates output tokens from accumulated content
Types ¶
type StreamTokenCounter ¶ added in v0.260224.0
type StreamTokenCounter struct {
// contains filtered or unexported fields
}
StreamTokenCounter maintains token count state for streaming responses. It uses incremental counting strategy, tokenizing each delta immediately.
Usage example:
counter, _ := NewStreamTokenCounter()
counter.SetInputTokens(estimateRequestTokens(req)) // Pre-count input
for chunk := range stream {
counter.ConsumeOpenAIChunk(chunk)
}
input, output := counter.GetCounts()
func NewStreamTokenCounter ¶ added in v0.260224.0
func NewStreamTokenCounter() (*StreamTokenCounter, error)
NewStreamTokenCounter creates a new streaming token counter.
func NewStreamTokenCounterWithEncoding ¶ added in v0.260224.0
func NewStreamTokenCounterWithEncoding(encoding string) (*StreamTokenCounter, error)
NewStreamTokenCounterWithEncoding creates a new streaming token counter with a specific encoding.
func (*StreamTokenCounter) AddInputTokens ¶ added in v0.260224.0
func (c *StreamTokenCounter) AddInputTokens(tokens int)
AddInputTokens adds to the input token count.
func (*StreamTokenCounter) AddToOutputTokens ¶ added in v0.260224.0
func (c *StreamTokenCounter) AddToOutputTokens(tokens int)
AddToOutputTokens adds to the output token count.
func (*StreamTokenCounter) ConsumeOpenAIChunk ¶ added in v0.260224.0
func (c *StreamTokenCounter) ConsumeOpenAIChunk(chunk *openai.ChatCompletionChunk) (inputTokens, outputTokens int, err error)
ConsumeOpenAIChunk processes an OpenAI streaming chunk and updates token counts. Returns the current (inputTokens, outputTokens) after processing this chunk.
The function handles:
- Content deltas (text responses)
- Refusal deltas (refusal messages)
- Tool call deltas (function names and arguments)
- Usage information (typically in the final chunk when stream_options.include_usage is set)
func (*StreamTokenCounter) CountText ¶ added in v0.260224.0
func (c *StreamTokenCounter) CountText(text string) int
CountText counts tokens for any text string.
func (*StreamTokenCounter) EstimateInputTokens ¶ added in v0.260224.0
func (c *StreamTokenCounter) EstimateInputTokens(req *openai.ChatCompletionNewParams) (int, error)
EstimateInputTokens estimates input tokens from a request using the counter's encoder. This is useful to pre-set the input token count before streaming.
func (*StreamTokenCounter) GetCounts ¶ added in v0.260224.0
func (c *StreamTokenCounter) GetCounts() (inputTokens, outputTokens int)
GetCounts returns the current token counts (inputTokens, outputTokens).
func (*StreamTokenCounter) InputTokens ¶ added in v0.260224.0
func (c *StreamTokenCounter) InputTokens() int
InputTokens returns the current input token count.
func (*StreamTokenCounter) OutputTokens ¶ added in v0.260224.0
func (c *StreamTokenCounter) OutputTokens() int
OutputTokens returns the current output token count.
func (*StreamTokenCounter) Reset ¶ added in v0.260224.0
func (c *StreamTokenCounter) Reset()
Reset resets the counter to zero.
func (*StreamTokenCounter) SetInputTokens ¶ added in v0.260224.0
func (c *StreamTokenCounter) SetInputTokens(tokens int)
SetInputTokens sets the input token count. Use this when you have the exact count from the request.
func (*StreamTokenCounter) SetOutputTokens ¶ added in v0.260224.0
func (c *StreamTokenCounter) SetOutputTokens(tokens int)
SetOutputTokens sets the output token count.
func (*StreamTokenCounter) TotalTokens ¶ added in v0.260224.0
func (c *StreamTokenCounter) TotalTokens() int
TotalTokens returns the sum of input and output tokens.
Source Files