Documentation
¶
Overview ¶
Package observability provides instrumentation for metrics, tracing, and logging.
Index ¶
Constants ¶
const ( TokenTypeInput = "input" TokenTypeOutput = "output" TokenTypeInputCached = "input_cached" TokenTypeInputCacheWrite = "input_cache_write" //nolint:gosec // G101: metric label value, not a credential. TokenTypeOutputReasoning = "output_reasoning" )
Values of the token_type label.
Variables ¶
var ( // RequestsTotal counts total LLM requests by provider, model, endpoint, and status RequestsTotal = promauto.NewCounterVec( prometheus.CounterOpts{ Name: "gomodel_requests_total", Help: "Total number of LLM requests", }, []string{"provider", "model", "endpoint", "status_code", "status_type", "stream"}, ) // RequestDuration measures request latency distribution // For streaming requests, this measures time to stream establishment, not total stream duration RequestDuration = promauto.NewHistogramVec( prometheus.HistogramOpts{ Name: "gomodel_request_duration_seconds", Help: "LLM request duration in seconds", Buckets: []float64{0.1, 0.25, 0.5, 1, 2, 5, 10, 30, 60}, }, []string{"provider", "model", "endpoint", "stream"}, ) // InFlightRequests tracks concurrent requests per provider InFlightRequests = promauto.NewGaugeVec( prometheus.GaugeOpts{ Name: "gomodel_requests_in_flight", Help: "Number of LLM requests currently in flight", }, []string{"provider", "endpoint", "stream"}, ) // ResponseSnapshotStoreFailures counts failures while storing response snapshots. ResponseSnapshotStoreFailures = promauto.NewCounterVec( prometheus.CounterOpts{ Name: "gomodel_response_snapshot_store_failures_total", Help: "Total number of response snapshot store failures", }, []string{"provider", "provider_name", "operation"}, ) // EmptyResponsesTotal counts provider calls that returned 200 without // choices, output, or usage. gomodel_requests_total records those calls as // successes, so alert on this counter instead. EmptyResponsesTotal = promauto.NewCounterVec( prometheus.CounterOpts{ Name: "gomodel_empty_responses_total", Help: "Total number of provider responses that returned 200 without choices, output, or usage", }, []string{"provider", "model", "reason"}, ) // CircuitBreakerState reports each provider's circuit breaker state as of // its most recent request (0=closed, 1=half-open, 2=open). The value is // updated per request, so an idle provider keeps its last observed state. CircuitBreakerState = promauto.NewGaugeVec( prometheus.GaugeOpts{ Name: "gomodel_circuit_breaker_state", Help: "Circuit breaker state per provider (0=closed, 1=half-open, 2=open)", }, []string{"provider"}, ) )
Prometheus metrics for LLM gateway observability
var ( // TokensTotal counts every token sent to and returned by providers. TokensTotal = promauto.NewCounterVec( prometheus.CounterOpts{ Name: "gomodel_tokens_total", Help: "Tokens processed by providers. token_type is input (uncached, cached, and cache-write prompt tokens) or output (reasoning included). Response-cache hits are not counted", }, []string{"provider", "model", "token_type", "user_path", "auth_key", "request_labels"}, ) // TokenDetailsTotal breaks gomodel_tokens_total down further. TokenDetailsTotal = promauto.NewCounterVec( prometheus.CounterOpts{ Name: "gomodel_token_details_total", Help: "Parts of gomodel_tokens_total: input_cached and input_cache_write are parts of the input tokens, output_reasoning is part of the output tokens. Do not add them to gomodel_tokens_total", }, []string{"provider", "model", "token_type", "user_path", "auth_key", "request_labels"}, ) // ProviderCostTotal sums what providers charge. ProviderCostTotal = promauto.NewCounterVec( prometheus.CounterOpts{ Name: "gomodel_provider_cost_usd_total", Help: "Provider cost of requests in USD, for requests with known pricing. Response-cache hits are not counted", }, []string{"provider", "model", "user_path", "auth_key", "request_labels"}, ) // BilledCostTotal sums what callers are billed. BilledCostTotal = promauto.NewCounterVec( prometheus.CounterOpts{ Name: "gomodel_billed_cost_usd_total", Help: "Cost billed to callers in USD: provider cost plus markup, including billed response-cache hits. Recorded only when pricing markups are enabled", }, []string{"provider", "model", "user_path", "auth_key", "request_labels"}, ) // CallerRequestsTotal counts provider calls per caller. It exists so the // caller labels never multiply gomodel_requests_total and its histogram: // it drops their endpoint, status_code, and stream labels, and is // recorded only while a caller dimension is enabled. CallerRequestsTotal = promauto.NewCounterVec( prometheus.CounterOpts{ Name: "gomodel_caller_requests_total", Help: "LLM requests by caller. Recorded only while a caller dimension is enabled", }, []string{"provider", "model", "status_type", "user_path", "auth_key", "request_labels"}, ) )
Token and cost metrics, fed from recorded usage entries. The caller labels (user_path, auth_key, request_labels) are opt-in and empty when switched off; Prometheus treats an empty label as an absent one.
Numbers that must not be added together live in separate metrics: the details are parts of gomodel_tokens_total, and the billed cost is the provider cost plus markup.
Functions ¶
func NewPrometheusHooks ¶
NewPrometheusHooks returns hooks that instrument LLM requests with Prometheus metrics. These hooks can be injected into llmclient.Config to enable observability without polluting business logic.
func RequestCallers ¶ added in v0.2.4
func RequestCallers() *usagemetrics.Callers
RequestCallers returns the caller dimensions SetRequestCallers selected.
func ResetMetrics ¶
func ResetMetrics()
ResetMetrics resets all metrics to zero (useful for testing)
func SetRequestCallers ¶ added in v0.2.4
func SetRequestCallers(callers *usagemetrics.Callers)
SetRequestCallers selects the caller dimensions of gomodel_caller_requests_total. nil stops recording the metric.
Types ¶
type UsageRecorder ¶ added in v0.2.4
type UsageRecorder struct{}
UsageRecorder writes usage samples to the Prometheus token and cost counters.
func (UsageRecorder) RecordUsage ¶ added in v0.2.4
func (UsageRecorder) RecordUsage(s usagemetrics.Sample)
RecordUsage adds one usage entry's tokens and cost. Zero values are skipped, so a series appears only once it has something to report.