usage

package
v7.2.97 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 23, 2026 License: MIT Imports: 6 Imported by: 0

Documentation

Index

Constants

View Source
const AutoServiceTier = "auto"

AutoServiceTier is the OpenAI request semantics when service_tier is omitted. OpenAI HTTP handlers set it explicitly, without changing other providers' historical direct-SDK default.

View Source
const DefaultServiceTier = "default"

DefaultServiceTier is retained for direct SDK and non-OpenAI usage callers.

View Source
const TokenAccountingSchemaVersion = 2

TokenAccountingSchemaVersion identifies the canonical token accounting contract.

Variables

This section is empty.

Functions

func GenerateEnabled added in v7.2.78

func GenerateEnabled(generate *bool) bool

GenerateEnabled reports whether generation is enabled for the record field. A nil value defaults to true so legacy callers that omit Generate keep the historical behavior.

func GenerateFlag added in v7.2.78

func GenerateFlag(generate bool) *bool

GenerateFlag returns a pointer suitable for Record.Generate.

func GenerateFromContext added in v7.2.78

func GenerateFromContext(ctx context.Context) bool

GenerateFromContext returns whether the client requested actual generation. Missing values default to true.

func PublishRecord

func PublishRecord(ctx context.Context, record Record)

PublishRecord publishes a record using the default manager.

func ReasoningEffortFromContext added in v7.1.18

func ReasoningEffortFromContext(ctx context.Context) string

ReasoningEffortFromContext returns the client-requested reasoning effort stored in ctx.

func RegisterNamedPlugin added in v7.1.47

func RegisterNamedPlugin(name string, plugin Plugin)

RegisterNamedPlugin registers or replaces a named plugin on the default manager.

func RegisterPlugin

func RegisterPlugin(plugin Plugin)

RegisterPlugin registers a plugin on the default manager.

func RequestedModelAliasFromContext

func RequestedModelAliasFromContext(ctx context.Context) string

RequestedModelAliasFromContext returns the client-requested model name stored in ctx.

func ServiceTierFromContext added in v7.1.27

func ServiceTierFromContext(ctx context.Context) string

ServiceTierFromContext returns the client-requested service tier stored in ctx.

func StartDefault

func StartDefault(ctx context.Context)

StartDefault starts the default manager's dispatcher.

func StopDefault

func StopDefault()

StopDefault stops the default manager's dispatcher.

func WithGenerate added in v7.2.78

func WithGenerate(ctx context.Context, generate bool) context.Context

WithGenerate stores whether the client requested actual generation for usage sinks. Missing context values default to true; only an explicit false disables generation.

func WithReasoningEffort added in v7.1.18

func WithReasoningEffort(ctx context.Context, effort string) context.Context

WithReasoningEffort stores the client-requested reasoning effort for usage sinks.

func WithRequestedModelAlias

func WithRequestedModelAlias(ctx context.Context, alias string) context.Context

WithRequestedModelAlias stores the client-requested model name for usage sinks.

func WithServiceTier added in v7.1.27

func WithServiceTier(ctx context.Context, tier string) context.Context

WithServiceTier stores the client-requested service tier for usage sinks.

Types

type Detail

type Detail struct {
	InputTokens         int64
	OutputTokens        int64
	ReasoningTokens     int64
	CachedTokens        int64
	CacheReadTokens     int64
	CacheCreationTokens int64
	TotalTokens         int64
	TokenBreakdown      TokenBreakdown
	ResponseServiceTier string
}

Detail holds the token usage breakdown.

func EnsureTokenBreakdown added in v7.2.97

func EnsureTokenBreakdown(detail Detail) Detail

EnsureTokenBreakdown attaches a valid v2 breakdown to legacy or direct SDK usage details without guessing whether reasoning is already inside output.

func EnsureTokenBreakdownForProvider added in v7.2.97

func EnsureTokenBreakdownForProvider(detail Detail, provider, executorType string) Detail

EnsureTokenBreakdownForProvider attaches a valid v2 breakdown to legacy or direct SDK usage details using the known provider's token semantics. Unknown providers remain unclassified instead of guessing how their buckets overlap.

type Failure

type Failure struct {
	StatusCode int
	Body       string
}

Failure holds HTTP failure metadata for an upstream request attempt.

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager maintains a queue of usage records and delivers them to registered plugins.

func DefaultManager

func DefaultManager() *Manager

DefaultManager returns the global usage manager instance.

func NewManager

func NewManager(buffer int) *Manager

NewManager constructs a manager with a buffered queue.

func (*Manager) Publish

func (m *Manager) Publish(ctx context.Context, record Record)

Publish enqueues a usage record for processing. If no plugin is registered the record will be discarded downstream.

func (*Manager) Register

func (m *Manager) Register(plugin Plugin)

Register appends a plugin to the delivery list.

func (*Manager) RegisterNamed added in v7.1.47

func (m *Manager) RegisterNamed(name string, plugin Plugin)

RegisterNamed registers or replaces a plugin by name.

func (*Manager) Start

func (m *Manager) Start(ctx context.Context)

Start launches the background dispatcher. Calling Start multiple times is safe.

func (*Manager) Stop

func (m *Manager) Stop()

Stop stops the dispatcher and drains the queue.

type Plugin

type Plugin interface {
	HandleUsage(ctx context.Context, record Record)
}

Plugin consumes usage records emitted by the proxy runtime.

type Record

type Record struct {
	Provider string
	// ExecutorType stores the concrete executor type that handled the request.
	ExecutorType string
	Model        string
	Alias        string
	APIKey       string
	AuthID       string
	AuthIndex    string
	AuthType     string
	Source       string
	// ReasoningEffort stores the translated upstream thinking level for request event logs.
	ReasoningEffort string
	// ServiceTier stores the client-requested service tier.
	ServiceTier string
	// RequestServiceTier is a deprecated input-only alias retained for existing
	// plugin callers. It is normalized into ServiceTier and never emitted.
	RequestServiceTier string
	// ResponseServiceTier stores the final tier reported by the upstream response.
	ResponseServiceTier string
	// Generate reports whether the client requested actual generation.
	// nil or true means generation is enabled; only an explicit false disables generation.
	// Use GenerateFlag to set the value and GenerateEnabled to read it with the default.
	Generate    *bool
	RequestedAt time.Time
	Latency     time.Duration
	TTFT        time.Duration
	Failed      bool
	Fail        Failure
	Detail      Detail
	// ResponseHeaders stores a snapshot of upstream response headers for usage sinks.
	ResponseHeaders http.Header
}

Record contains the usage statistics captured for a single provider request.

type TokenAccountingQuality added in v7.2.97

type TokenAccountingQuality string

TokenAccountingQuality describes how confidently a token total can be classified.

const (
	TokenAccountingQualityComplete     TokenAccountingQuality = "complete"
	TokenAccountingQualityInconsistent TokenAccountingQuality = "inconsistent"
	TokenAccountingQualityUnclassified TokenAccountingQuality = "unclassified"
)

type TokenBreakdown added in v7.2.97

type TokenBreakdown struct {
	SchemaVersion      int                    `json:"schema_version"`
	Quality            TokenAccountingQuality `json:"quality"`
	TotalTokens        int64                  `json:"total_tokens"`
	Input              TokenInputBreakdown    `json:"input"`
	Output             TokenOutputBreakdown   `json:"output"`
	UnclassifiedTokens int64                  `json:"unclassified_tokens"`
}

TokenBreakdown is the canonical, non-overlapping token accounting contract.

func NewIndependentTokenBreakdown added in v7.2.97

func NewIndependentTokenBreakdown(uncachedInput, cacheRead, cacheWrite, nonReasoningOutput, reasoning, total int64) TokenBreakdown

NewIndependentTokenBreakdown normalizes protocols where uncached input, cache reads, cache writes, non-reasoning output, and reasoning are separate.

func NewPartialSubsetTokenBreakdown added in v7.2.97

func NewPartialSubsetTokenBreakdown(inputTotal, cacheRead, cacheWrite, outputTotal, reasoning, total int64) TokenBreakdown

NewPartialSubsetTokenBreakdown preserves known subset buckets while assigning an authoritative remainder to the unclassified bucket.

func NewSeparateReasoningTokenBreakdown added in v7.2.97

func NewSeparateReasoningTokenBreakdown(inputTotal, cacheRead, cacheWrite, nonReasoningOutput, reasoning, total int64) TokenBreakdown

NewSeparateReasoningTokenBreakdown normalizes protocols where cache tokens are included in input totals while reasoning is separate from ordinary output.

func NewSubsetTokenBreakdown added in v7.2.97

func NewSubsetTokenBreakdown(inputTotal, cacheRead, cacheWrite, outputTotal, reasoning, total int64) TokenBreakdown

NewSubsetTokenBreakdown normalizes protocols where cache tokens are included in input totals and reasoning tokens are included in output totals.

func NewUnclassifiedTokenBreakdown added in v7.2.97

func NewUnclassifiedTokenBreakdown(total int64) TokenBreakdown

NewUnclassifiedTokenBreakdown preserves an authoritative total without guessing how an unknown protocol partitions it.

func (TokenBreakdown) Valid added in v7.2.97

func (b TokenBreakdown) Valid() bool

Valid reports whether the breakdown satisfies the v2 accounting invariants.

type TokenInputBreakdown added in v7.2.97

type TokenInputBreakdown struct {
	TotalTokens      int64 `json:"total_tokens"`
	UncachedTokens   int64 `json:"uncached_tokens"`
	CacheReadTokens  int64 `json:"cache_read_tokens"`
	CacheWriteTokens int64 `json:"cache_write_tokens"`
}

TokenInputBreakdown contains mutually exclusive input token buckets.

type TokenOutputBreakdown added in v7.2.97

type TokenOutputBreakdown struct {
	TotalTokens        int64 `json:"total_tokens"`
	NonReasoningTokens int64 `json:"non_reasoning_tokens"`
	ReasoningTokens    int64 `json:"reasoning_tokens"`
}

TokenOutputBreakdown contains mutually exclusive output token buckets.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL