Documentation
¶
Index ¶
- func CountText(model, text string) int
- func Estimate(req *ir.Request, output []ir.ContentPart) *ir.Usage
- func FillMissing(req *ir.Request, output []ir.ContentPart, actual *ir.Usage) *ir.Usage
- func FillMissingText(req *ir.Request, outputText string, actual *ir.Usage) *ir.Usage
- func Prewarm(model string)
- func Provenance(actual *ir.Usage, filled *ir.Usage) metrics.TokenProvenance
- type RequestEstimator
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CountText ¶
CountText 使用模型对应的 tiktoken 编码计数；编码不可用时退化为字符估算。
func Estimate ¶
Estimate 基于 IR 请求和输出内容估算输入、输出及总 token 数。
func FillMissing ¶
FillMissing 用本地估算补全上游遗漏的 token 用量，同时优先保留上游已返回 的真实值。估算（对整个请求做完整 BPE 编码）是惰性的：上游真的留下缺口时 才计算。流式路径每帧补全请复用 RequestEstimator，避免重复编码请求侧文本。
func FillMissingText ¶
FillMissingText 把纯文本输出包装成 IR 内容块，再复用统一的 usage 补全逻辑。
func Prewarm ¶ added in v1.25.4
func Prewarm(model string)
Prewarm warms the tiktoken encoder for model so the first forwarded request does not block on the BPE download. Unknown model names resolve through the cl100k_base fallback, warming the shared encoder as well. Callers typically invoke it once at startup in a background goroutine; failures are recorded by the negative cache and retried after the TTL, so the result is ignored.
func Provenance ¶ added in v1.23.0
Provenance reports, per counter, whether the value came from the upstream or was estimated locally by FillMissing/FillMissingText. The distinction is recorded at fill time because the returned *ir.Usage no longer says which counters were synthesized; a later reader can only guess.
Types ¶
type RequestEstimator ¶ added in v1.25.4
type RequestEstimator struct {
// contains filtered or unexported fields
}
RequestEstimator 缓存请求侧的输入估算结果。流式转发每个 usage 帧都会补全 一次 usage：若每帧都对整个请求（系统提示 + 全部消息 + 工具 schema，上限 50MB）重新编码，CPU 开销随帧数线性放大；而输入部分在一次转发内不变，缓存 后每帧只需估算新增的输出文本。请求文本的首次编码同样是惰性的——上游完整 上报 usage 时完全不付出估算成本。零值不可用，必须由 NewRequestEstimator 构造；nil 接收者的行为与 req 为 nil 的 FillMissing 一致。
func NewRequestEstimator ¶ added in v1.25.4
func NewRequestEstimator(req *ir.Request) *RequestEstimator
NewRequestEstimator 为一次转发创建估算器。req 可为 nil，此时仅输出侧可估。
func (*RequestEstimator) Estimate ¶ added in v1.25.4
func (e *RequestEstimator) Estimate(output []ir.ContentPart) *ir.Usage
Estimate 在输入估算固定的前提下计算输出侧 token 数。
func (*RequestEstimator) FillMissing ¶ added in v1.25.4
func (e *RequestEstimator) FillMissing(output []ir.ContentPart, actual *ir.Usage) *ir.Usage
FillMissing 与包级 FillMissing 语义一致，但输入估算来自缓存。
Source Files
¶
- tokenusage.go