Documentation
¶
Overview ¶
Package usage provides usage tracking and logging functionality for the CLI Proxy API server. It includes plugins for monitoring API usage, token consumption, and other metrics to help with observability and billing purposes.
Index ¶
- func InitializePersistence(dbPath string, batchSize, flushIntervalSecs, retentionDays int) error
- func LoadRecordsFromDB(db *sql.DB, retentionDays int, stats *RequestStatistics) error
- func SetStatisticsEnabled(enabled bool)
- func StatisticsEnabled() bool
- func StopPersistence() error
- type APISnapshot
- type LoggerPlugin
- type ModelSnapshot
- type Persister
- type RequestDetail
- type RequestStatistics
- type StatisticsSnapshot
- type TokenStats
- type UsageRecord
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func InitializePersistence ¶ added in v1.0.2
InitializePersistence initializes SQLite persistence for usage records. It loads historical records from the database to rebuild in-memory stats. Returns an error if persistence fails to initialize, but this should not stop the server.
func LoadRecordsFromDB ¶ added in v1.0.2
func LoadRecordsFromDB(db *sql.DB, retentionDays int, stats *RequestStatistics) error
LoadRecordsFromDB loads usage statistics from the database using optimized SQL aggregation. Instead of loading all records and processing them one by one, it: 1. Uses SQL GROUP BY for aggregates (totals, by_day, by_hour, by_api, by_model) 2. Only loads recent N records for the Details array This reduces memory usage and speeds up recovery significantly.
func SetStatisticsEnabled ¶
func SetStatisticsEnabled(enabled bool)
SetStatisticsEnabled toggles whether in-memory statistics are recorded.
func StatisticsEnabled ¶
func StatisticsEnabled() bool
StatisticsEnabled reports the current recording state.
func StopPersistence ¶ added in v1.0.2
func StopPersistence() error
StopPersistence gracefully shuts down the persister, flushing pending writes.
Types ¶
type APISnapshot ¶
type APISnapshot struct {
TotalRequests int64 `json:"total_requests"`
TotalTokens int64 `json:"total_tokens"`
Models map[string]ModelSnapshot `json:"models"`
}
APISnapshot summarises metrics for a single API key.
type LoggerPlugin ¶
type LoggerPlugin struct {
// contains filtered or unexported fields
}
LoggerPlugin collects in-memory request statistics for usage analysis. It implements coreusage.Plugin to receive usage records emitted by the runtime.
func GetLoggerPlugin ¶ added in v1.0.2
func GetLoggerPlugin() *LoggerPlugin
GetLoggerPlugin returns the shared logger plugin instance.
func NewLoggerPlugin ¶
func NewLoggerPlugin() *LoggerPlugin
NewLoggerPlugin constructs a new logger plugin instance. Returns:
- *LoggerPlugin: A new logger plugin instance wired to the shared statistics store.
func (*LoggerPlugin) HandleUsage ¶
func (p *LoggerPlugin) HandleUsage(ctx context.Context, record coreusage.Record)
HandleUsage implements coreusage.Plugin. It updates the in-memory statistics store whenever a usage record is received. Parameters:
- ctx: The context for the usage record
- record: The usage record to aggregate
type ModelSnapshot ¶
type ModelSnapshot struct {
TotalRequests int64 `json:"total_requests"`
TotalTokens int64 `json:"total_tokens"`
Details []RequestDetail `json:"details"`
}
ModelSnapshot summarises metrics for a specific model.
type Persister ¶ added in v1.0.2
type Persister struct {
// contains filtered or unexported fields
}
Persister handles SQLite persistence for usage records with async batched writes.
func NewPersister ¶ added in v1.0.2
func NewPersister(dbPath string, batchSize, flushIntervalSecs, retentionDays int) (*Persister, error)
NewPersister initializes a new SQLite persister with the given configuration. Returns nil if dbPath is empty or creation fails.
func (*Persister) DBPath ¶ added in v1.0.2
DBPath returns the filesystem path to the SQLite database.
func (*Persister) Enqueue ¶ added in v1.0.2
func (p *Persister) Enqueue(record UsageRecord)
Enqueue adds a usage record to the persistence queue. Non-blocking; drops records if queue is full to prevent blocking.
type RequestDetail ¶
type RequestDetail struct {
Timestamp time.Time `json:"timestamp"`
Source string `json:"source"`
AuthIndex uint64 `json:"auth_index"`
Tokens TokenStats `json:"tokens"`
Failed bool `json:"failed"`
}
RequestDetail stores the timestamp and token usage for a single request.
type RequestStatistics ¶
type RequestStatistics struct {
// contains filtered or unexported fields
}
RequestStatistics maintains aggregated request metrics in memory.
func GetRequestStatistics ¶
func GetRequestStatistics() *RequestStatistics
GetRequestStatistics returns the shared statistics store.
func NewRequestStatistics ¶
func NewRequestStatistics() *RequestStatistics
NewRequestStatistics constructs an empty statistics store.
func (*RequestStatistics) Record ¶
func (s *RequestStatistics) Record(ctx context.Context, record coreusage.Record)
Record ingests a new usage record and updates the aggregates.
func (*RequestStatistics) Snapshot ¶
func (s *RequestStatistics) Snapshot() StatisticsSnapshot
Snapshot returns a copy of the aggregated metrics for external consumption.
type StatisticsSnapshot ¶
type StatisticsSnapshot struct {
TotalRequests int64 `json:"total_requests"`
SuccessCount int64 `json:"success_count"`
FailureCount int64 `json:"failure_count"`
TotalTokens int64 `json:"total_tokens"`
APIs map[string]APISnapshot `json:"apis"`
RequestsByDay map[string]int64 `json:"requests_by_day"`
RequestsByHour map[string]int64 `json:"requests_by_hour"`
TokensByDay map[string]int64 `json:"tokens_by_day"`
TokensByHour map[string]int64 `json:"tokens_by_hour"`
}
StatisticsSnapshot represents an immutable view of the aggregated metrics.
type TokenStats ¶
type TokenStats struct {
PromptTokens int64 `json:"prompt_tokens"`
CompletionTokens int64 `json:"completion_tokens"`
ReasoningTokens int64 `json:"reasoning_tokens"`
CachedTokens int64 `json:"cached_tokens"`
TotalTokens int64 `json:"total_tokens"`
AudioTokens int64 `json:"audio_tokens,omitempty"`
CacheCreationInputTokens int64 `json:"cache_creation_input_tokens,omitempty"`
CacheReadInputTokens int64 `json:"cache_read_input_tokens,omitempty"`
ToolUsePromptTokens int64 `json:"tool_use_prompt_tokens,omitempty"`
}
TokenStats captures the token usage breakdown for a request.
type UsageRecord ¶ added in v1.0.2
type UsageRecord struct {
Provider string
Model string
APIKey string
AuthID string
AuthIndex uint64
Source string
RequestedAt time.Time
Failed bool
InputTokens int64
OutputTokens int64
ReasoningTokens int64
CachedTokens int64
TotalTokens int64
AudioTokens int64
CacheCreationInputTokens int64
CacheReadInputTokens int64
ToolUsePromptTokens int64
}
UsageRecord represents a single usage record for persistence.
Source Files