usage

package
v8.0.53 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 3, 2026 License: MIT Imports: 12 Imported by: 8

Documentation

Index

Constants

View Source
const AutoServiceTier = "auto"

AutoServiceTier is the OpenAI request semantics when service_tier is omitted. OpenAI HTTP handlers set it explicitly, without changing other providers' historical direct-SDK default.

View Source
const DefaultServiceTier = "default"

DefaultServiceTier is retained for direct SDK and non-OpenAI usage callers.

View Source
const FailureStreamTruncatedCode = "stream_truncated"

FailureStreamTruncatedCode marks a locally synthesized failure for an accepted upstream stream that ended without its terminal frame.

View Source
const TokenAccountingSchemaVersion = 2

TokenAccountingSchemaVersion identifies the canonical token accounting contract.

Variables

This section is empty.

Functions

func APIKeyFromContext added in v8.0.24

func APIKeyFromContext(ctx context.Context) string

APIKeyFromContext returns the client API key stored in ctx or gin.Context.

func AccessProviderFromContext added in v8.0.24

func AccessProviderFromContext(ctx context.Context) string

AccessProviderFromContext returns the client authentication provider stored in ctx.

func BoolPtr added in v8.0.48

func BoolPtr(v bool) *bool

BoolPtr returns a pointer to v.

func CaptureIdentity added in v8.0.45

func CaptureIdentity(ctx context.Context) context.Context

CaptureIdentity snapshots the current request's usage identity into ctx while the HTTP handler is active so post-handler goroutines never read a reused gin.Context.

func CloneInt64Ptr added in v8.0.48

func CloneInt64Ptr(p *int64) *int64

CloneInt64Ptr returns a shallow copy of p when non-nil.

func ExecutionRequestIDFromContext

func ExecutionRequestIDFromContext(ctx context.Context) string

ExecutionRequestIDFromContext retrieves the execution instance request ID from the context.

func Float64Ptr added in v8.0.48

func Float64Ptr(v float64) *float64

Float64Ptr returns a pointer to v.

func GenerateEnabled

func GenerateEnabled(generate *bool) bool

GenerateEnabled reports whether generation is enabled for the record field. A nil value defaults to true so legacy callers that omit Generate keep the historical behavior.

func GenerateFlag

func GenerateFlag(generate bool) *bool

GenerateFlag returns a pointer suitable for Record.Generate.

func GenerateFromContext

func GenerateFromContext(ctx context.Context) bool

GenerateFromContext returns whether the client requested actual generation. Missing values default to true.

func HasCapturedIdentity added in v8.0.45

func HasCapturedIdentity(ctx context.Context) bool

HasCapturedIdentity reports whether ctx holds a captured usage identity snapshot.

func Int64Ptr added in v8.0.48

func Int64Ptr(v int64) *int64

Int64Ptr returns a pointer to v.

func IsNativeAccessProvider added in v8.0.24

func IsNativeAccessProvider(provider string) bool

IsNativeAccessProvider reports whether provider represents CPA's built-in inline key provider.

func IsNativeKeyFromContext added in v8.0.24

func IsNativeKeyFromContext(ctx context.Context) (bool, bool)

IsNativeKeyFromContext returns whether the client API key is native to CPA if explicitly set in ctx.

func IterationsPtr added in v8.0.48

func IterationsPtr(items []IterationUsage) *[]IterationUsage

IterationsPtr returns a pointer to a cloned non-empty IterationUsage slice, or nil when empty.

func PublishRecord

func PublishRecord(ctx context.Context, record Record)

PublishRecord publishes a record using the default manager.

func ReasoningEffortFromContext

func ReasoningEffortFromContext(ctx context.Context) string

ReasoningEffortFromContext returns the client-requested reasoning effort stored in ctx.

func RegisterNamedPlugin

func RegisterNamedPlugin(name string, plugin Plugin)

RegisterNamedPlugin registers or replaces a named plugin on the default manager.

func RegisterPlugin

func RegisterPlugin(plugin Plugin)

RegisterPlugin registers a plugin on the default manager.

func RequestedModelAliasFromContext

func RequestedModelAliasFromContext(ctx context.Context) string

RequestedModelAliasFromContext returns the client-requested model name stored in ctx.

func SelectionIDFromContext added in v8.0.44

func SelectionIDFromContext(ctx context.Context) string

SelectionIDFromContext retrieves the scheduler selection ID from the context.

func ServiceTierFromContext

func ServiceTierFromContext(ctx context.Context) string

ServiceTierFromContext returns the client-requested service tier stored in ctx.

func SetCapturedAPIKey added in v8.0.45

func SetCapturedAPIKey(ctx context.Context, apiKey string)

SetCapturedAPIKey updates the captured API key in ctx when an identity snapshot holder exists.

func StartDefault

func StartDefault(ctx context.Context)

StartDefault starts the default manager's dispatcher.

func StopDefault

func StopDefault()

StopDefault stops the default manager's dispatcher.

func StreamDeliverySupported added in v8.0.14

func StreamDeliverySupported(ctx context.Context) bool

StreamDeliverySupported reports whether an active producer understands delivery.

func StreamDeliveryTracked added in v8.0.14

func StreamDeliveryTracked(ctx context.Context) bool

StreamDeliveryTracked reports whether a consumer will acknowledge delivery.

func StreamFromContext

func StreamFromContext(ctx context.Context) bool

StreamFromContext returns whether the request was executed in streaming mode. Missing values default to false.

func SupportStreamDelivery added in v8.0.14

func SupportStreamDelivery(ctx context.Context) func()

SupportStreamDelivery opts an active producer into early terminal cancellation. The producer must reconcile cancellation with WaitStreamDelivery.

func TraceIDFromContext

func TraceIDFromContext(ctx context.Context) string

TraceIDFromContext retrieves the parent inbound HTTP request ID from the context.

func WaitStreamDelivery added in v8.0.14

func WaitStreamDelivery(ctx context.Context) (error, bool)

WaitStreamDelivery waits for the consumer, not the upstream reader, to finish. Producers must close their output channel before waiting to avoid deadlocks.

func WithAPIKey added in v8.0.24

func WithAPIKey(ctx context.Context, apiKey string) context.Context

WithAPIKey stores the client API key for usage sinks.

func WithAccessProvider added in v8.0.24

func WithAccessProvider(ctx context.Context, provider string) context.Context

WithAccessProvider stores the client authentication provider identifier for usage sinks.

func WithCapturedIdentity added in v8.0.45

func WithCapturedIdentity(ctx context.Context, apiKey, accessProvider string, isNativeKey ...bool) context.Context

WithCapturedIdentity stores an authoritative snapshot of the request's usage identity (including empty values) so later asynchronous calls do not read a reused gin.Context.

func WithExecutionRequestID

func WithExecutionRequestID(ctx context.Context, requestID string) context.Context

WithExecutionRequestID attaches a specific execution instance request ID to the context.

func WithGenerate

func WithGenerate(ctx context.Context, generate bool) context.Context

WithGenerate stores whether the client requested actual generation for usage sinks. Missing context values default to true; only an explicit false disables generation.

func WithIsNativeKey added in v8.0.24

func WithIsNativeKey(ctx context.Context, isNative bool) context.Context

WithIsNativeKey stores whether the client API key is native to CPA for usage sinks.

func WithReasoningEffort

func WithReasoningEffort(ctx context.Context, effort string) context.Context

WithReasoningEffort stores the client-requested reasoning effort for usage sinks.

func WithRequestedModelAlias

func WithRequestedModelAlias(ctx context.Context, alias string) context.Context

WithRequestedModelAlias stores the client-requested model name for usage sinks.

func WithSelectionID added in v8.0.44

func WithSelectionID(ctx context.Context, selectionID string) context.Context

WithSelectionID attaches the scheduler selection ID for the selected credential to the context.

func WithServiceTier

func WithServiceTier(ctx context.Context, tier string) context.Context

WithServiceTier stores the client-requested service tier for usage sinks.

func WithStream

func WithStream(ctx context.Context, stream bool) context.Context

WithStream stores whether the request was executed in streaming mode for usage sinks.

func WithStreamDelivery added in v8.0.14

func WithStreamDelivery(ctx context.Context) (context.Context, func(error))

WithStreamDelivery lets the HTTP consumer acknowledge delivery independently of request cancellation. The consumer must resolve the outcome on every exit.

func WithTraceID

func WithTraceID(ctx context.Context, traceID string) context.Context

WithTraceID attaches the parent inbound HTTP request ID to the context.

func WithoutStreamDelivery added in v8.0.14

func WithoutStreamDelivery(ctx context.Context) context.Context

WithoutStreamDelivery isolates an internal execution from its caller's HTTP acknowledgment. Internal consumers retain EOF-based accounting completion; cancellation and all unrelated context values remain inherited.

Types

type CacheDetails added in v8.0.48

type CacheDetails struct {
	Mode                    string           `json:"mode,omitempty"`
	TTL                     string           `json:"ttl,omitempty"`
	Retention               string           `json:"retention,omitempty"`
	Ephemeral5mInputTokens  *int64           `json:"ephemeral_5m_input_tokens,omitempty"`
	Ephemeral30mInputTokens *int64           `json:"ephemeral_30m_input_tokens,omitempty"`
	Ephemeral1hInputTokens  *int64           `json:"ephemeral_1h_input_tokens,omitempty"`
	Ephemeral12hInputTokens *int64           `json:"ephemeral_12h_input_tokens,omitempty"`
	ByTTL                   map[string]int64 `json:"by_ttl,omitempty"`
}

CacheDetails holds prompt cache configuration and TTL-bucketed write token counts.

func (*CacheDetails) Clone added in v8.0.48

func (c *CacheDetails) Clone() *CacheDetails

Clone returns a deep copy of CacheDetails.

func (*CacheDetails) SumTokens added in v8.0.48

func (c *CacheDetails) SumTokens() (int64, bool)

SumTokens returns the sum of TTL-bucketed tokens and whether at least one TTL bucket was present.

type CostDetails added in v8.0.48

type CostDetails struct {
	Cost                             *float64 `json:"cost,omitempty"`
	CostInUSDTicks                   *int64   `json:"cost_in_usd_ticks,omitempty"`
	IsBYOK                           *bool    `json:"is_byok,omitempty"`
	UpstreamInferenceCost            *float64 `json:"upstream_inference_cost,omitempty"`
	UpstreamInferencePromptCost      *float64 `json:"upstream_inference_prompt_cost,omitempty"`
	UpstreamInferenceCompletionsCost *float64 `json:"upstream_inference_completions_cost,omitempty"`
}

CostDetails holds optional upstream-reported cost information (e.g. OpenRouter).

func (*CostDetails) Clone added in v8.0.48

func (c *CostDetails) Clone() *CostDetails

Clone returns a deep copy of CostDetails.

type Detail

type Detail struct {
	InputTokens         int64
	OutputTokens        int64
	ReasoningTokens     int64
	CachedTokens        int64
	CacheReadTokens     int64
	CacheCreationTokens int64
	TotalTokens         int64
	TokenBreakdown      TokenBreakdown
	ResponseServiceTier string

	CacheCreation5mTokens    *int64
	CacheCreation30mTokens   *int64
	CacheCreation1hTokens    *int64
	CacheCreation12hTokens   *int64
	TextInputTokens          *int64
	ImageInputTokens         *int64
	AudioInputTokens         *int64
	VideoInputTokens         *int64
	FileInputTokens          *int64
	CachedTextTokens         *int64
	TextOutputTokens         *int64
	ImageOutputTokens        *int64
	AudioOutputTokens        *int64
	AcceptedPredictionTokens *int64
	RejectedPredictionTokens *int64
	CacheDetails             *CacheDetails
	WebSearchRequests        *int64
	ImageCount               *int64
	RequestedImageDetails    *ImageDetails
	ImageDetails             *ImageDetails
	MediaCounts              *MediaCounts
	ToolUsageDetails         *ToolUsageDetails
	Iterations               *[]IterationUsage
	CostDetails              *CostDetails
	Speed                    string
	InferenceGeo             string
	UpstreamServiceTier      string
}

Detail holds the token and structured usage breakdown captured from an upstream response.

func EnsureTokenBreakdown

func EnsureTokenBreakdown(detail Detail) Detail

EnsureTokenBreakdown attaches a valid v2 breakdown to legacy or direct SDK usage details without guessing whether reasoning is already inside output.

func EnsureTokenBreakdownForProvider

func EnsureTokenBreakdownForProvider(detail Detail, provider, executorType string) Detail

EnsureTokenBreakdownForProvider attaches a valid v2 breakdown to legacy or direct SDK usage details using the known provider's token semantics. Unknown providers remain unclassified instead of guessing how their buckets overlap.

func (Detail) IterationSlice added in v8.0.48

func (d Detail) IterationSlice() []IterationUsage

IterationSlice returns a deep copy of Detail.Iterations when present.

type Failure

type Failure struct {
	StatusCode int
	Body       string
	// Code carries structured failure classification (e.g. FailureStreamTruncatedCode)
	// across usage sinks so downstream schedulers do not treat locally
	// synthesized statuses as received upstream refusals.
	Code string
}

Failure holds HTTP failure metadata for an upstream request attempt.

type ImageDetails added in v8.0.48

type ImageDetails struct {
	Size           string `json:"size,omitempty"`
	Resolution     string `json:"resolution,omitempty"`
	ImageSize      string `json:"image_size,omitempty"`
	AspectRatio    string `json:"aspect_ratio,omitempty"`
	Quality        string `json:"quality,omitempty"`
	Background     string `json:"background,omitempty"`
	OutputFormat   string `json:"output_format,omitempty"`
	ResponseFormat string `json:"response_format,omitempty"`
	MimeType       string `json:"mime_type,omitempty"`
	InputFidelity  string `json:"input_fidelity,omitempty"`
}

ImageDetails holds image/video generation dimension and quality attributes.

func (*ImageDetails) Clone added in v8.0.48

func (d *ImageDetails) Clone() *ImageDetails

Clone returns a deep copy of ImageDetails.

type IterationUsage added in v8.0.48

type IterationUsage struct {
	Type                     string        `json:"type,omitempty"`
	Model                    string        `json:"model,omitempty"`
	InputTokens              *int64        `json:"input_tokens,omitempty"`
	OutputTokens             *int64        `json:"output_tokens,omitempty"`
	ThinkingTokens           *int64        `json:"thinking_tokens,omitempty"`
	ReasoningTokens          *int64        `json:"reasoning_tokens,omitempty"`
	CacheReadInputTokens     *int64        `json:"cache_read_input_tokens,omitempty"`
	CacheCreationInputTokens *int64        `json:"cache_creation_input_tokens,omitempty"`
	CacheCreation5mTokens    *int64        `json:"cache_creation_5m_tokens,omitempty"`
	CacheCreation1hTokens    *int64        `json:"cache_creation_1h_tokens,omitempty"`
	CacheCreation            *CacheDetails `json:"cache_creation,omitempty"`
}

IterationUsage holds per-iteration usage reported in usage.iterations[].

func CloneIterations added in v8.0.48

func CloneIterations(items []IterationUsage) []IterationUsage

CloneIterations returns a deep copy of a slice of IterationUsage.

func (IterationUsage) Clone added in v8.0.48

func (it IterationUsage) Clone() IterationUsage

Clone returns a deep copy of IterationUsage.

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager maintains a queue of usage records and delivers them to registered plugins.

func DefaultManager

func DefaultManager() *Manager

DefaultManager returns the global usage manager instance.

func NewManager

func NewManager(buffer int) *Manager

NewManager constructs a manager with a buffered queue.

func (*Manager) Publish

func (m *Manager) Publish(ctx context.Context, record Record)

Publish enqueues a usage record for processing. If no plugin is registered the record will be discarded downstream.

func (*Manager) Register

func (m *Manager) Register(plugin Plugin)

Register appends a plugin to the delivery list.

func (*Manager) RegisterNamed

func (m *Manager) RegisterNamed(name string, plugin Plugin)

RegisterNamed registers or replaces a plugin by name.

func (*Manager) Start

func (m *Manager) Start(ctx context.Context)

Start launches the background dispatcher. Calling Start multiple times is safe.

func (*Manager) Stop

func (m *Manager) Stop()

Stop stops the dispatcher and drains the queue.

type MediaCounts added in v8.0.48

type MediaCounts struct {
	PromptMedia     *int64   `json:"prompt_media,omitempty"`
	CompletionMedia *int64   `json:"completion_media,omitempty"`
	PromptAudio     *int64   `json:"prompt_audio,omitempty"`
	AudioSeconds    *float64 `json:"audio_seconds,omitempty"`
}

MediaCounts holds prompt/completion media item counts and audio/video duration.

func (*MediaCounts) Clone added in v8.0.48

func (m *MediaCounts) Clone() *MediaCounts

Clone returns a deep copy of MediaCounts.

type Plugin

type Plugin interface {
	HandleUsage(ctx context.Context, record Record)
}

Plugin consumes usage records emitted by the proxy runtime.

type Record

type Record struct {
	// RequestID uniquely identifies this specific model execution instance (UUID v4).
	RequestID string
	// TraceID identifies the parent inbound HTTP request when available (8-character hex).
	TraceID string
	// SelectionID identifies the scheduler selection decision that produced this usage record when available.
	SelectionID string
	Provider    string
	// BaseURL stores the configured upstream base URL when available.
	BaseURL string
	// ExecutorType stores the concrete executor type that handled the request.
	ExecutorType string
	Model        string
	Alias        string
	APIKey       string
	// IsNativeKey reports whether the client API key was authenticated by CPA's native config provider.
	IsNativeKey bool
	// AccessProvider identifies the client request authentication provider (e.g. "config-inline" or plugin ID).
	AccessProvider  string
	SessionID       string
	ParentSessionID string
	AuthID          string
	AuthIndex       string
	// AccessTokenSHA256 identifies the OAuth token version without exposing the token.
	AccessTokenSHA256 string
	AuthType          string
	Source            string
	// ReasoningEffort stores the translated upstream thinking level for request event logs.
	ReasoningEffort string
	// ServiceTier stores the client-requested service tier.
	ServiceTier string
	// EffectiveServiceTier stores the tier in the final outbound request after
	// protocol translation and payload defaults or overrides.
	EffectiveServiceTier string
	// RequestServiceTier is a deprecated input-only alias retained for existing
	// plugin callers. It is normalized into ServiceTier and never emitted.
	RequestServiceTier string
	// ResponseServiceTier stores the final tier reported by the upstream response.
	ResponseServiceTier string
	// ResponseModel stores the model name reported by the upstream response, empty when unknown.
	ResponseModel string
	// Generate reports whether the client requested actual generation.
	// nil or true means generation is enabled; only an explicit false disables generation.
	// Use GenerateFlag to set the value and GenerateEnabled to read it with the default.
	Generate *bool
	// Stream reports whether the request was executed in streaming mode.
	Stream      bool
	RequestedAt time.Time
	Latency     time.Duration
	TTFT        time.Duration
	Failed      bool
	Fail        Failure
	Detail      Detail
	// ResponseHeaders stores a snapshot of upstream response headers for usage sinks.
	ResponseHeaders http.Header
}

Record contains the usage statistics captured for a single provider request.

type TokenAccountingQuality

type TokenAccountingQuality string

TokenAccountingQuality describes how confidently a token total can be classified.

const (
	TokenAccountingQualityComplete     TokenAccountingQuality = "complete"
	TokenAccountingQualityInconsistent TokenAccountingQuality = "inconsistent"
	TokenAccountingQualityUnclassified TokenAccountingQuality = "unclassified"
)

type TokenBreakdown

type TokenBreakdown struct {
	SchemaVersion      int                    `json:"schema_version"`
	Quality            TokenAccountingQuality `json:"quality"`
	TotalTokens        int64                  `json:"total_tokens"`
	Input              TokenInputBreakdown    `json:"input"`
	Output             TokenOutputBreakdown   `json:"output"`
	UnclassifiedTokens int64                  `json:"unclassified_tokens"`
}

TokenBreakdown is the canonical, non-overlapping token accounting contract.

func NewIndependentTokenBreakdown

func NewIndependentTokenBreakdown(uncachedInput, cacheRead, cacheWrite, nonReasoningOutput, reasoning, total int64) TokenBreakdown

NewIndependentTokenBreakdown normalizes protocols where uncached input, cache reads, cache writes, non-reasoning output, and reasoning are separate.

func NewPartialSubsetTokenBreakdown

func NewPartialSubsetTokenBreakdown(inputTotal, cacheRead, cacheWrite, outputTotal, reasoning, total int64) TokenBreakdown

NewPartialSubsetTokenBreakdown preserves known subset buckets while assigning an authoritative remainder to the unclassified bucket.

func NewSeparateReasoningTokenBreakdown

func NewSeparateReasoningTokenBreakdown(inputTotal, cacheRead, cacheWrite, nonReasoningOutput, reasoning, total int64) TokenBreakdown

NewSeparateReasoningTokenBreakdown normalizes protocols where cache tokens are included in input totals while reasoning is separate from ordinary output.

func NewSubsetTokenBreakdown

func NewSubsetTokenBreakdown(inputTotal, cacheRead, cacheWrite, outputTotal, reasoning, total int64) TokenBreakdown

NewSubsetTokenBreakdown normalizes protocols where cache tokens are included in input totals and reasoning tokens are included in output totals.

func NewUnclassifiedTokenBreakdown

func NewUnclassifiedTokenBreakdown(total int64) TokenBreakdown

NewUnclassifiedTokenBreakdown preserves an authoritative total without guessing how an unknown protocol partitions it.

func (TokenBreakdown) Valid

func (b TokenBreakdown) Valid() bool

Valid reports whether the breakdown satisfies the v2 accounting invariants.

type TokenInputBreakdown

type TokenInputBreakdown struct {
	TotalTokens      int64 `json:"total_tokens"`
	UncachedTokens   int64 `json:"uncached_tokens"`
	CacheReadTokens  int64 `json:"cache_read_tokens"`
	CacheWriteTokens int64 `json:"cache_write_tokens"`
}

TokenInputBreakdown contains mutually exclusive input token buckets.

type TokenOutputBreakdown

type TokenOutputBreakdown struct {
	TotalTokens        int64 `json:"total_tokens"`
	NonReasoningTokens int64 `json:"non_reasoning_tokens"`
	ReasoningTokens    int64 `json:"reasoning_tokens"`
}

TokenOutputBreakdown contains mutually exclusive output token buckets.

type ToolUsageDetails added in v8.0.48

type ToolUsageDetails struct {
	WebSearchCalls         *int64           `json:"web_search_calls,omitempty"`
	WebSearchQueries       *int64           `json:"web_search_queries,omitempty"`
	WebFetchCalls          *int64           `json:"web_fetch_calls,omitempty"`
	NumSearchResults       *int64           `json:"num_search_results,omitempty"`
	NumSourcesUsed         *int64           `json:"num_sources_used,omitempty"`
	NumServerSideToolsUsed *int64           `json:"num_server_side_tools_used,omitempty"`
	ToolCallsRequested     *int64           `json:"tool_calls_requested,omitempty"`
	ToolCallsExecuted      *int64           `json:"tool_calls_executed,omitempty"`
	XSearchCalls           *int64           `json:"x_search_calls,omitempty"`
	XPostsFetched          *int64           `json:"x_posts_fetched,omitempty"`
	XUsersFetched          *int64           `json:"x_users_fetched,omitempty"`
	CodeExecutionCalls     *int64           `json:"code_execution_calls,omitempty"`
	FileSearchCalls        *int64           `json:"file_search_calls,omitempty"`
	MCPCalls               *int64           `json:"mcp_calls,omitempty"`
	DocumentSearchCalls    *int64           `json:"document_search_calls,omitempty"`
	ImageGenerationCalls   *int64           `json:"image_generation_calls,omitempty"`
	Extra                  map[string]int64 `json:"extra,omitempty"`
}

ToolUsageDetails holds structured server-side tool execution counters.

func (*ToolUsageDetails) Clone added in v8.0.48

func (t *ToolUsageDetails) Clone() *ToolUsageDetails

Clone returns a deep copy of ToolUsageDetails.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL