usage

package
v7.2.59 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 16, 2026 License: MIT Imports: 15 Imported by: 0

Documentation

Overview

Package usage provides usage tracking and logging functionality for the CLI Proxy API server. It includes plugins for monitoring API usage, token consumption, and other metrics to help with observability and billing purposes.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DefaultSnapshotPath 

func DefaultSnapshotPath(configFilePath string) string

func EnsureRequestMetrics 

func EnsureRequestMetrics(ginCtx *gin.Context)

func ObserveAPIResponseChunk 

func ObserveAPIResponseChunk(ginCtx *gin.Context, bytesWritten int64)

func ObserveResponseChunkReady 

func ObserveResponseChunkReady(ginCtx *gin.Context)

func ObserveResponseChunkReadyFromContext 

func ObserveResponseChunkReadyFromContext(ctx context.Context)

func ObserveResponseWrite 

func ObserveResponseWrite(ginCtx *gin.Context, bytesWritten int64, streaming bool)

func ResetDefaultRequestStatistics 

func ResetDefaultRequestStatistics()

func SaveRequestStatisticsToFile 

func SaveRequestStatisticsToFile(path string, stats *RequestStatistics) error

func SaveSnapshotToFile 

func SaveSnapshotToFile(path string, snapshot StatisticsSnapshot) error

func SetStatisticsEnabled 

func SetStatisticsEnabled(enabled bool)

SetStatisticsEnabled toggles whether in-memory statistics are recorded.

func StatisticsEnabled 

func StatisticsEnabled() bool

StatisticsEnabled reports the current recording state.

Types

type APISnapshot 

type APISnapshot struct {
	TotalRequests int64                    `json:"total_requests"`
	SuccessCount  int64                    `json:"success_count"`
	FailureCount  int64                    `json:"failure_count"`
	TotalTokens   int64                    `json:"total_tokens"`
	Tokens        TokenStats               `json:"tokens"`
	Models        map[string]ModelSnapshot `json:"models"`
}

APISnapshot summarises metrics for a single API key.

type LoggerPlugin 

type LoggerPlugin struct{}

LoggerPlugin collects in-memory request statistics for usage analysis. It implements coreusage.Plugin to receive usage records emitted by the runtime.

func NewLoggerPlugin 

func NewLoggerPlugin() *LoggerPlugin

NewLoggerPlugin constructs a new logger plugin instance.

Returns:

  • *LoggerPlugin: A new logger plugin instance wired to the shared statistics store.

func (*LoggerPlugin) HandleUsage 

func (p *LoggerPlugin) HandleUsage(ctx context.Context, record coreusage.Record)

HandleUsage implements coreusage.Plugin. It updates the in-memory statistics store whenever a usage record is received.

Parameters:

  • ctx: The context for the usage record
  • record: The usage record to aggregate

type MergeResult 

type MergeResult struct {
	Added   int64 `json:"added"`
	Skipped int64 `json:"skipped"`
}

type ModelSnapshot 

type ModelSnapshot struct {
	TotalRequests int64           `json:"total_requests"`
	SuccessCount  int64           `json:"success_count"`
	FailureCount  int64           `json:"failure_count"`
	TotalTokens   int64           `json:"total_tokens"`
	Tokens        TokenStats      `json:"tokens"`
	Details       []RequestDetail `json:"details"`
}

ModelSnapshot summarises metrics for a specific model.

type RequestDetail 

type RequestDetail struct {
	Timestamp             time.Time  `json:"timestamp"`
	LatencyMs             int64      `json:"latency_ms"`
	FirstByteLatencyMs    *int64     `json:"first_byte_latency_ms,omitempty"`
	APIFirstByteLatencyMs *int64     `json:"api_first_byte_latency_ms,omitempty"`
	ChunkCount            int64      `json:"chunk_count"`
	ResponseBytes         int64      `json:"response_bytes"`
	APIResponseBytes      int64      `json:"api_response_bytes"`
	Source                string     `json:"source"`
	Alias                 string     `json:"alias,omitempty"`
	AuthIndex             string     `json:"auth_index"`
	ReasoningEffort       string     `json:"reasoning_effort,omitempty"`
	Tokens                TokenStats `json:"tokens"`
	Failed                bool       `json:"failed"`
}

RequestDetail stores the timestamp, latency, and token usage for a single request.

type RequestMetrics 

type RequestMetrics struct {
	ChunkCount           int64
	ResponseBytes        int64
	APIResponseBytes     int64
	FirstResponseChunkAt time.Time
	FirstResponseWriteAt time.Time
}

func SnapshotRequestMetrics 

func SnapshotRequestMetrics(ctx context.Context) RequestMetrics

func SnapshotRequestMetricsFromGin 

func SnapshotRequestMetricsFromGin(ginCtx *gin.Context) RequestMetrics

type RequestStatistics 

type RequestStatistics struct {
	// contains filtered or unexported fields
}

RequestStatistics maintains aggregated request metrics in memory.

func GetRequestStatistics 

func GetRequestStatistics() *RequestStatistics

GetRequestStatistics returns the shared statistics store.

func NewRequestStatistics 

func NewRequestStatistics() *RequestStatistics

NewRequestStatistics constructs an empty statistics store.

func (*RequestStatistics) MergeSnapshot 

func (s *RequestStatistics) MergeSnapshot(snapshot StatisticsSnapshot) MergeResult

MergeSnapshot merges an exported statistics snapshot into the current store. Existing data is preserved and duplicate request details are skipped.

func (*RequestStatistics) Record 

func (s *RequestStatistics) Record(ctx context.Context, record coreusage.Record)

Record ingests a new usage record and updates the aggregates.

func (*RequestStatistics) Reset 

func (s *RequestStatistics) Reset()

Reset clears all in-memory statistics while keeping the store instance reusable.

func (*RequestStatistics) RestoreSnapshot 

func (s *RequestStatistics) RestoreSnapshot(snapshot StatisticsSnapshot) RestoreResult

RestoreSnapshot replaces the current store with a persisted snapshot. It preserves persisted aggregate totals even when request details were trimmed.

func (*RequestStatistics) Snapshot 

func (s *RequestStatistics) Snapshot() StatisticsSnapshot

Snapshot returns a copy of the aggregated metrics for external consumption.

type RestoreResult 

type RestoreResult struct {
	Requests int64 `json:"requests"`
	Details  int64 `json:"details"`
}

type StatisticsSnapshot 

type StatisticsSnapshot struct {
	TotalRequests int64      `json:"total_requests"`
	SuccessCount  int64      `json:"success_count"`
	FailureCount  int64      `json:"failure_count"`
	TotalTokens   int64      `json:"total_tokens"`
	Tokens        TokenStats `json:"tokens"`

	APIs map[string]APISnapshot `json:"apis"`

	RequestsByDay  map[string]int64 `json:"requests_by_day"`
	RequestsByHour map[string]int64 `json:"requests_by_hour"`
	TokensByDay    map[string]int64 `json:"tokens_by_day"`
	TokensByHour   map[string]int64 `json:"tokens_by_hour"`
}

StatisticsSnapshot represents an immutable view of the aggregated metrics.

func LoadSnapshotFromFile 

func LoadSnapshotFromFile(path string) (StatisticsSnapshot, error)

type TokenStats 

type TokenStats struct {
	InputTokens         int64 `json:"input_tokens"`
	OutputTokens        int64 `json:"output_tokens"`
	ReasoningTokens     int64 `json:"reasoning_tokens"`
	CachedTokens        int64 `json:"cached_tokens"`
	CacheReadTokens     int64 `json:"cache_read_tokens"`
	CacheCreationTokens int64 `json:"cache_creation_tokens"`
	TotalTokens         int64 `json:"total_tokens"`
}

TokenStats captures the token usage breakdown for a request.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.