tokenmeter

package
v0.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 13, 2026 License: MIT Imports: 5 Imported by: 0

Documentation

Overview

Fixed-density token estimation matches token-meter/src/estimate.ts. Before actual tokenization, shared heuristics use one token per four text bytes plus block structure and message-role overhead so meters and projections agree.

Package tokenmeter measures token usage and enforces budgets (M31), matching util/token-meter. It records prompt and completion tokens per request and accumulates them per session. Once the cap is reached, the next request is denied before calling the LLM. Surface-node weights estimate costs for derived message and tool-result snapshots exposed to UI and SDK consumers.

Index

Constants

View Source
const BudgetDeniedError = "token-meter: session budget exceeded; request rejected before LLM call"

BudgetDeniedError uses a stable message for caller auditing.

View Source
const (

	// RoleOverhead accounts for per-message role framing.
	RoleOverhead = 4
)

Variables

This section is empty.

Functions

func EstimateContent

func EstimateContent(blocks []llm.ContentBlock) int

EstimateContent estimates content tokens with fixed density and block overhead.

func EstimateMessage

func EstimateMessage(msg llm.Message) int

EstimateMessage includes content and role overhead for a model-visible message.

func EstimateMessages

func EstimateMessages(msgs []llm.Message) int

EstimateMessages estimates tokens for the entire conversation history.

Types

type Meter

type Meter struct {
	// contains filtered or unexported fields
}

Meter measures session-level tokens.

func New

func New(budget int) *Meter

New creates a meter; nonpositive budgets are unlimited.

func (*Meter) Budget

func (m *Meter) Budget() int

Budget returns the budget cap.

func (*Meter) BudgetDenied

func (m *Meter) BudgetDenied() bool

BudgetDenied reports whether the budget has denied further requests.

func (*Meter) Check

func (m *Meter) Check() error

Check validates budget before a request; callers must not invoke the LLM on failure.

func (*Meter) HasBudget

func (m *Meter) HasBudget() bool

HasBudget permits another request only while below the cap. Callers must check before invoking the LLM to avoid an actual over-budget request.

func (*Meter) Record

func (m *Meter) Record(u TokenUsage)

Record records usage and updates budget denial.

func (*Meter) Records

func (m *Meter) Records() []RequestRecord

Records returns a snapshot of all measurements.

func (*Meter) Totals

func (m *Meter) Totals() (prompt, completion, total int)

Totals returns cumulative prompt, completion, and total tokens.

type RequestRecord

type RequestRecord struct {
	Seq   int        `json:"seq"`
	Usage TokenUsage `json:"usage"`
}

RequestRecord records one measurement.

type SurfaceNode

type SurfaceNode struct {
	// Kind identifies the node type, such as assistant/message.
	Kind string
	// Count is the number of occurrences.
	Count int
}

SurfaceNode counts occurrences of a derived-node type.

type SurfacePricing

type SurfacePricing map[string]int

SurfacePricing maps derived-node types to token weights; cost is weight times count.

func (SurfacePricing) CostOf

func (p SurfacePricing) CostOf(entries []SurfaceNode) int

CostOf estimates surface-node cost; unknown types have zero cost.

type TokenUsage

type TokenUsage struct {
	// PromptTokens counts input tokens.
	PromptTokens int `json:"promptTokens"`
	// CompletionTokens counts output tokens.
	CompletionTokens int `json:"completionTokens"`
}

TokenUsage records one model request's token usage.

func (TokenUsage) Total

func (u TokenUsage) Total() int

Total returns total request tokens.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL