Documentation
¶
Overview ¶
Package tokenize owns gateway-side token estimation. When a provider stream ends without reporting usage, the estimator counts prompt and completion tokens with the closest known tokenizer so accounting, budgets, and clients still receive token totals. Codecs initialize at composition time and are shared across requests; counting is read-only and safe for concurrent use.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Estimator ¶
type Estimator struct {
// contains filtered or unexported fields
}
Estimator counts tokens with shared pre-built codecs. Create one with NewEstimator at composition time; the zero value has no codecs and falls back to a bytes-per-token heuristic.
func NewEstimator ¶
func NewEstimator() *Estimator
NewEstimator builds the estimator and initializes every codec it can select, so request paths never pay first-use construction and never touch the codecs' lazy decode state.
func (*Estimator) CountMessages ¶
CountMessages estimates the prompt token count of a chat request, including per-message framing overhead and the reply preamble.
type Hint ¶
type Hint struct {
// Tokenizer is the catalog tokenizer family, for example "gpt".
Tokenizer string
// Model is the exact provider model ID.
Model string
}
Hint names the routed model so the estimator can pick the closest codec. Both fields are optional; an empty hint selects the default.
Source Files