Documentation
¶
Overview ¶
Package usagemetrics turns recorded usage entries into token and cost metrics. It reads each entry once and hands the result to the enabled metric pipelines (Prometheus, OpenTelemetry); it never touches storage.
Index ¶
Constants ¶
const ( OtherValue = "{other}" OtherUserPath = "/" + OtherValue )
Overflow buckets for dimension values seen after the cap is reached, or that are too long, reserved, or not valid UTF-8.
const MaxValueBytes = 256
MaxValueBytes is the longest dimension value exported as itself. Every admitted value is copied into each of its series and into every scrape, so a longer one goes to the overflow bucket without taking a slot.
const UnknownAuthKey = "{unknown}"
UnknownAuthKey is reported for a managed key whose name cannot be resolved, for example one deleted while its request was still streaming.
Variables ¶
This section is empty.
Functions ¶
func NewTap ¶
func NewTap(inner usage.LoggerInterface, callers *Callers, recorders ...Recorder) usage.LoggerInterface
NewTap wraps the logger, or returns it unchanged when no metric pipeline takes usage samples. Wrap the logger before the markup tap, so the entry already carries its billed cost when it gets here.
func ResetCallerValues ¶
func ResetCallerValues()
ResetCallerValues forgets every admitted caller value (useful for testing).
Types ¶
type Caller ¶
type Caller struct {
// UserPath is the request's user path, cut to the configured depth.
UserPath string
// AuthKey is the name of the managed API key; empty for the master key
// and for unauthenticated traffic.
AuthKey string
// RequestLabels is the request's labels, sorted and comma-separated;
// see joinLabels for the encoding.
RequestLabels string
}
Caller is who made a request, as metric dimension values. A dimension that is switched off, or that has no value for the request, is empty.
type Callers ¶
type Callers struct {
// contains filtered or unexported fields
}
Callers resolves the opt-in caller dimensions. A nil *Callers has every dimension switched off.
func NewCallers ¶
func NewCallers(cfg config.MetricsCallerDimensionsConfig, authKeyName func(id string) string) *Callers
NewCallers builds the resolver for the configured dimensions, or returns nil when all of them are off. authKeyName maps a managed key ID to its name; without it the auth key dimension stays off.
type Recorder ¶
type Recorder interface {
RecordUsage(sample Sample)
}
Recorder is one metric pipeline's sink for usage samples. RecordUsage runs on the request path and must not block.
type Sample ¶
type Sample struct {
// Provider is the configured provider name, ProviderType its type.
Provider string
ProviderType string
// Model is the model that answered, RequestModel the one the request was
// routed to (empty when the entry does not record it).
Model string
RequestModel string
Endpoint string
// Operation is the GenAI operation of the provider call, when known.
Operation string
Caller Caller
InputTokens int64
OutputTokens int64
CachedInputTokens int64
CacheWriteInputTokens int64
ReasoningOutputTokens int64
// ProviderCost is what the provider charges for the request and
// BilledCost what the caller is billed, both in USD. Zero means free,
// unknown, or (for BilledCost) that pricing markups are off.
ProviderCost float64
BilledCost float64
}
Sample is what one usage entry adds to the token and cost metrics.
InputTokens and OutputTokens are complete: input covers the uncached prompt, provider-cached reads and cache writes, and output covers reasoning. The three detail counts are parts of those totals, never additions to them.
type Tap ¶
type Tap struct {
// contains filtered or unexported fields
}
Tap decorates a usage logger so every recorded entry also feeds the token and cost metrics. All modalities and response-cache hits funnel through Logger.Write once the entry is final — at the end of the response, or at stream close — so one tap covers them all.
func (*Tap) SetLivePublisher ¶
func (t *Tap) SetLivePublisher(p usage.LiveEventPublisher)
SetLivePublisher forwards the dashboard publisher to the wrapped logger, so wrapping it never silences live usage events.
func (*Tap) Write ¶
func (t *Tap) Write(entry *usage.UsageEntry)