analytics

package
v2.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 2, 2026 License: AGPL-3.0 Imports: 21 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var TeeQueueSize = 10000

TeeQueueSize is the per-sink queue length.

Functions

func GetAppActivity 

func GetAppActivity(db *gorm.DB, appIDs []uint, since time.Time) (map[uint]*AppActivity, error)

GetAppActivity returns activity keyed by app id for the given apps. Apps with no chat records are absent from the map.

func GetAppSpending 

func GetAppSpending(db *gorm.DB, windows []AppSpendWindow, end time.Time) (map[uint]float64, error)

GetAppSpending returns each app's spend between its own window start and `end`, in the same units as the budget service (llm_chat_records.cost is stored x10000; see proxy/analyze_utils.go). One query for every app: each app contributes an (app_id, start) predicate, so a user with twenty apps costs one round trip rather than twenty. Apps with no records are absent.

func GetBudgetUsage 

func GetBudgetUsage(db *gorm.DB, startDate, endDate *time.Time, llmID *uint) ([]models.BudgetUsage, error)

GetBudgetUsage returns usage statistics for all LLMs and Apps that have costs, with optional date range

func GetChatLogsForChatID 

func GetChatLogsForChatID(db *gorm.DB, chatID uint) ([]models.LLMChatLogEntry, error)

GetChatLogsForChatID retrieves all chat log entries for a specific chat ID

func GetCostAnalysis 

func GetCostAnalysis(db *gorm.DB, startDate, endDate time.Time, interactionType *models.InteractionType) (map[string]*ChartData, error)

GetCostAnalysis returns the total cost per day for each currency and interaction type

func GetProxyLogsForAppID 

func GetProxyLogsForAppID(db *gorm.DB, startDate, endDate time.Time, appID uint, page, pageSize int, search string) ([]models.ProxyLog, int64, error)

GetProxyLogsForAppID returns paginated proxy logs for a specific app Optional search parameter searches request_body and response_body fields

func GetProxyLogsForLLM 

func GetProxyLogsForLLM(db *gorm.DB, startDate, endDate time.Time, llmID uint, page, pageSize int, search string) ([]models.ProxyLog, int64, error)

GetProxyLogsForLLM returns paginated proxy logs for a specific LLM by filtering on vendor Optional search parameter searches request_body and response_body fields

func GetTokenUsageAndCostForApp 

func GetTokenUsageAndCostForApp(db *gorm.DB, startDate, endDate time.Time, appID uint) (*models.MultiAxisChartData, error)

GetTokenUsageAndCostForApp returns the token usage and total cost for a specific app over time

func GetToolOperationsUsageOverTime 

func GetToolOperationsUsageOverTime(db *gorm.DB, toolID uint, startDate, endDate time.Time) (*models.MultiAxisChartData, error)

GetToolOperationsUsageOverTime returns the usage count for each operation of a specific tool over time

func GetUsage 

func GetUsage(db *gorm.DB, startDate, endDate time.Time, vendor string, llmID, appID *uint, interactionType *models.InteractionType, modelName string) (*models.MultiAxisChartData, error)

GetUsage returns token usage and cost data based on provided filters modelName, when non-empty, restricts the series to a single model (matched on the recorded model name, so an alias appears under its resolved name).

func Init 

func Init(ctx context.Context, db *gorm.DB)

Init initializes analytics with default database handler (for backward compatibility)

func InitDefault 

func InitDefault(ctx context.Context, db *gorm.DB)

InitDefault initializes the default database analytics handler

func Migrate 

func Migrate(db *gorm.DB) error

Migrate creates or updates the analytics tables. pkg/studio runs it with the other migrations under the cross-instance migration lock. The handler does not migrate: it may write to a database whose schema another instance owns, so whoever owns the schema calls Migrate before recording starts.

func RecordChatLogEntry 

func RecordChatLogEntry(ctx context.Context, log *models.LLMChatLogEntry)

func RecordChatRecord 

func RecordChatRecord(ctx context.Context, record *models.LLMChatRecord)

func RecordChatRecordsBatch 

func RecordChatRecordsBatch(ctx context.Context, records []*models.LLMChatRecord)

RecordChatRecordsBatch records multiple chat records in a batch for improved performance

func RecordComplianceEvents 

func RecordComplianceEvents(ctx context.Context, events []*models.ComplianceEvent)

RecordComplianceEvents records filter script compliance events asynchronously

func RecordContentMessage 

func RecordContentMessage(
	mc *llms.MessageContent,
	cr *llms.ContentResponse,
	vendor models.Vendor,
	name, chatID string,
	timeMs int, userID, appID, llmID uint,
	t time.Time,
	svc services.ServiceInterface,
	dontLogBodies bool,
)

func RecordExchange 

func RecordExchange(ctx context.Context, log *models.ProxyLog, rec *models.LLMChatRecord)

RecordExchange records a proxied request's proxy log together with its chat record (nil when the response yielded no usage). A handler that implements ExchangeRecorder gets both in one call; any other handler gets RecordProxyLog followed by RecordChatRecord, as before.

func RecordProxyLog 

func RecordProxyLog(ctx context.Context, log *models.ProxyLog)

func RecordProxyLogsBatch 

func RecordProxyLogsBatch(ctx context.Context, logs []*models.ProxyLog)

RecordProxyLogsBatch records multiple proxy logs in a batch for improved performance

func RecordToolCall 

func RecordToolCall(ctx context.Context, name string, t time.Time, execTime int, toolID uint)

func RequestLatencyMS 

func RequestLatencyMS(ctx context.Context, end time.Time) (int, bool)

RequestLatencyMS is the time from the request's start to end, in milliseconds. end is normally the moment the response completed (a record's TimeStamp); when it is not after the start (a timestamp taken at the start of the request) the time elapsed until now is used instead. ok is false when the context carries no start.

func ResetHandler 

func ResetHandler()

ResetHandler resets the global analytics handler (useful for testing). When the handler is a stopped database recorder, it waits (briefly) for the recorder's worker to finish, so it does not outlive the Studio that stopped it: a Studio started next in the process replaces the package state (the logger) the worker still uses.

func SetBufferSize 

func SetBufferSize(n int)

SetBufferSize sets how many records wait in memory to be written (Studio passes AppConf.AnalyticsBufferSize); without it, ANALYTICS_BUFFER_SIZE decides. Call it before StartRecording.

func SetHandler 

func SetHandler(handler AnalyticsHandler)

SetHandler sets the global analytics handler implementation

func StartRecording 

func StartRecording(ctx context.Context, db *gorm.DB)

StartRecording is deprecated, use InitDefault instead

func WithRequestStart 

func WithRequestStart(ctx context.Context, start time.Time) context.Context

WithRequestStart records when the gateway received the request.

Types

type AnalyticsHandler 

type AnalyticsHandler interface {
	// RecordChatRecord records LLM chat/proxy usage
	RecordChatRecord(ctx context.Context, record *models.LLMChatRecord)

	// RecordChatLogEntry records detailed chat log entries
	RecordChatLogEntry(ctx context.Context, log *models.LLMChatLogEntry)

	// RecordProxyLog records proxy request/response logs
	RecordProxyLog(ctx context.Context, log *models.ProxyLog)

	// RecordToolCall records tool call execution
	RecordToolCall(ctx context.Context, name string, timestamp time.Time, execTime int, toolID uint)

	// SetAsGlobalHandler sets this handler as the global analytics handler
	SetAsGlobalHandler()

	// Batch processing methods for improved performance
	RecordChatRecordsBatch(ctx context.Context, records []*models.LLMChatRecord)
	RecordProxyLogsBatch(ctx context.Context, logs []*models.ProxyLog)

	// RecordComplianceEvents records filter script compliance events
	RecordComplianceEvents(ctx context.Context, events []*models.ComplianceEvent)
}

AnalyticsHandler defines the interface for analytics implementations

func GetHandler 

func GetHandler() AnalyticsHandler

GetHandler returns the current analytics handler (useful for testing)

type AppActivity 

type AppActivity struct {
	AppID        uint
	LastAccessAt *time.Time
	Requests     int64
}

AppActivity is the portal overview's per-app activity summary: when the app last completed an LLM interaction and how many it made in the window.

It reads llm_chat_records, the same table the app page's token, cost and interaction charts and the budget spend are computed from, so the overview never disagrees with the app page (proxy_logs also records rejected and failed attempts, which the app page does not count). One grouped query over the caller's apps, so the overview costs one round trip however many apps there are.

type AppSpendWindow 

type AppSpendWindow struct {
	AppID uint
	Start time.Time
}

AppSpendWindow names an app and the start of its budget period.

type ChartData 

type ChartData struct {
	Labels []string  `json:"labels"`
	Data   []float64 `json:"data"`
	Cost   []float64 `json:"cost,omitempty"`
}

func GetAppInteractionsOverTime 

func GetAppInteractionsOverTime(db *gorm.DB, startDate, endDate time.Time, appID uint) (*ChartData, error)

GetAppInteractionsOverTime returns the number of LLM interactions for a specific app over time

func GetChatInteractionsForChat 

func GetChatInteractionsForChat(db *gorm.DB, startDate, endDate time.Time, chatID string) (*ChartData, error)

GetChatInteractionsForChat returns the number of interactions for a specific chat over time

func GetChatRecordsPerDay 

func GetChatRecordsPerDay(db *gorm.DB, startDate, endDate *time.Time) (*ChartData, error)

GetChatRecordsPerDay returns the total number of chat records per day for a given time period

func GetChatRecordsPerUser 

func GetChatRecordsPerUser(db *gorm.DB, startDate, endDate time.Time) (*ChartData, error)

GetChatRecordsPerUser returns the total number of chat records per user for a given time period

func GetModelUsage 

func GetModelUsage(db *gorm.DB, startDate, endDate time.Time, modelName string) (*ChartData, error)

GetModelUsage returns the usage statistics for a specific model over time

func GetMostUsedLLMModels 

func GetMostUsedLLMModels(db *gorm.DB, startDate, endDate time.Time, interactionType *models.InteractionType) (*ChartData, error)

GetMostUsedLLMModels returns the usage count for each LLM model

func GetTokenUsageForApp 

func GetTokenUsageForApp(db *gorm.DB, startDate, endDate time.Time, appID uint) (*ChartData, error)

GetTokenUsageForApp returns the token usage for a specific app over time

func GetTokenUsagePerApp 

func GetTokenUsagePerApp(db *gorm.DB, startDate, endDate time.Time, interactionType *models.InteractionType) (*ChartData, error)

GetTokenUsagePerApp returns the total token usage for each app

func GetTokenUsagePerUser 

func GetTokenUsagePerUser(db *gorm.DB, startDate, endDate time.Time, interactionType *models.InteractionType) (*ChartData, error)

GetTokenUsagePerUser returns the total token usage for each user

func GetToolCallsPerDay 

func GetToolCallsPerDay(db *gorm.DB, startDate, endDate time.Time) (*ChartData, error)

func GetToolUsageStatistics 

func GetToolUsageStatistics(db *gorm.DB, startDate, endDate time.Time) (*ChartData, error)

GetToolUsageStatistics returns the usage count for each tool

func GetUniqueUsersPerDay 

func GetUniqueUsersPerDay(db *gorm.DB, startDate, endDate time.Time) (*ChartData, error)

GetUniqueUsersPerDay returns the number of unique users per day

func GetVendorUsage 

func GetVendorUsage(db *gorm.DB, startDate, endDate time.Time, vendor string, llmID *uint) (*ChartData, error)

GetVendorUsage returns the usage statistics for a specific vendor over time

type DatabaseHandler 

type DatabaseHandler struct {
	// contains filtered or unexported fields
}

DatabaseHandler implements AnalyticsHandler using the existing database/channel system

func NewDatabaseHandler 

func NewDatabaseHandler(ctx context.Context, db *gorm.DB) *DatabaseHandler

NewDatabaseHandler creates a new database analytics handler

func (*DatabaseHandler) RecordChatLogEntry 

func (h *DatabaseHandler) RecordChatLogEntry(_ context.Context, logEntry *models.LLMChatLogEntry)

RecordChatLogEntry implements analytics.AnalyticsHandler

func (*DatabaseHandler) RecordChatRecord 

func (h *DatabaseHandler) RecordChatRecord(_ context.Context, record *models.LLMChatRecord)

Implement AnalyticsHandler interface methods

func (*DatabaseHandler) RecordChatRecordsBatch 

func (h *DatabaseHandler) RecordChatRecordsBatch(_ context.Context, records []*models.LLMChatRecord)

RecordChatRecordsBatch records multiple chat records asynchronously using background worker This method is non-blocking and returns immediately to avoid impacting request latency

func (*DatabaseHandler) RecordComplianceEvents 

func (h *DatabaseHandler) RecordComplianceEvents(_ context.Context, events []*models.ComplianceEvent)

RecordComplianceEvents records filter script compliance events asynchronously

func (*DatabaseHandler) RecordProxyLog 

func (h *DatabaseHandler) RecordProxyLog(_ context.Context, log *models.ProxyLog)

func (*DatabaseHandler) RecordProxyLogsBatch 

func (h *DatabaseHandler) RecordProxyLogsBatch(_ context.Context, logs []*models.ProxyLog)

RecordProxyLogsBatch records multiple proxy logs asynchronously using background worker This method is non-blocking and returns immediately to avoid impacting request latency

func (*DatabaseHandler) RecordToolCall 

func (h *DatabaseHandler) RecordToolCall(_ context.Context, name string, timestamp time.Time, execTime int, toolID uint)

func (*DatabaseHandler) SetAsGlobalHandler 

func (h *DatabaseHandler) SetAsGlobalHandler()

SetAsGlobalHandler sets this handler as the global analytics handler

func (*DatabaseHandler) Stop 

func (h *DatabaseHandler) Stop()

Stop gracefully stops the database handler

type ExchangeRecorder 

type ExchangeRecorder interface {
	// RecordExchange records one proxied request. rec is nil when the
	// response yielded no usage (an error, or nothing to parse).
	RecordExchange(ctx context.Context, log *models.ProxyLog, rec *models.LLMChatRecord)
}

ExchangeRecorder is implemented by handlers that record a proxied request's proxy log and its chat record together. The gateway produces both for one request; a handler that stores them as one row needs them in one call rather than having to pair two separate calls up again.

type ModelAppUsage 

type ModelAppUsage struct {
	AppID        uint     `json:"appId"`
	AppName      string   `json:"appName"`
	AppDeleted   bool     `json:"appDeleted"`
	OwnerUserID  uint     `json:"ownerUserId"`
	OwnerEmail   string   `json:"ownerEmail"`
	RequestCount int64    `json:"requestCount"`
	TotalTokens  int64    `json:"totalTokens"`
	TotalCost    float64  `json:"totalCost"`
	FirstUsed    ScanTime `json:"firstUsed"`
	LastUsed     ScanTime `json:"lastUsed"`
}

ModelAppUsage is one app's usage of a specific model under a specific LLM entry. It answers "who is using model X?" with enough detail to contact the app owner.

func GetAppsForModel 

func GetAppsForModel(db *gorm.DB, startDate, endDate time.Time, llmID uint, modelName string, interactionType *models.InteractionType) ([]ModelAppUsage, error)

GetAppsForModel returns every app that called the given model through the given LLM entry within the date range, most recently used first. The LLM scope matters: the same model name can sit under two LLM entries (two keys for the same vendor), and the question is asked from one provider's page. Soft-deleted apps are still returned so historical usage stays attributable; they are flagged rather than hidden.

type ScanTime 

type ScanTime struct {
	time.Time
}

ScanTime is a time.Time that can be scanned from an aggregated datetime column such as MAX(time_stamp). An aggregate result column has no declared type, so the SQLite driver hands back the raw TEXT instead of a time.Time (Postgres keeps the timestamp type). Scanning such a column straight into a time.Time fails on SQLite with "unsupported Scan, storing driver.Value type string into type *time.Time"; this type accepts all three representations.

func (*ScanTime) Scan 

func (s *ScanTime) Scan(value interface{}) error

Scan implements sql.Scanner.

func (ScanTime) Value 

func (s ScanTime) Value() (driver.Value, error)

Value implements driver.Valuer so GORM treats the type as a plain column.

type Tee 

type Tee struct {
	// contains filtered or unexported fields
}

Tee sends every record to a primary handler (Studio's database handler: budgets and spend are computed from it) and to extra sinks, for a host that ships analytics into its own pipeline. Sinks never slow a request: each has a bounded queue drained by its own goroutine, and records that do not fit are dropped and counted. Sinks receive copies, so they never share a record with the primary handler (which writes IDs back into it).

func NewTee 

func NewTee(primary AnalyticsHandler, sinks ...AnalyticsHandler) *Tee

NewTee returns a Tee over primary and sinks. Stop it to drain the sinks.

func (*Tee) Dropped 

func (t *Tee) Dropped() []uint64

Dropped reports, per sink, how many records did not fit its queue.

func (*Tee) RecordChatLogEntry 

func (t *Tee) RecordChatLogEntry(ctx context.Context, log *models.LLMChatLogEntry)

func (*Tee) RecordChatRecord 

func (t *Tee) RecordChatRecord(ctx context.Context, record *models.LLMChatRecord)

func (*Tee) RecordChatRecordsBatch 

func (t *Tee) RecordChatRecordsBatch(ctx context.Context, records []*models.LLMChatRecord)

func (*Tee) RecordComplianceEvents 

func (t *Tee) RecordComplianceEvents(ctx context.Context, events []*models.ComplianceEvent)

func (*Tee) RecordExchange 

func (t *Tee) RecordExchange(ctx context.Context, log *models.ProxyLog, rec *models.LLMChatRecord)

RecordExchange keeps a proxied request's log and chat record together for handlers that store them as one (ExchangeRecorder) and splits them for the others, as the package-level RecordExchange does.

func (*Tee) RecordProxyLog 

func (t *Tee) RecordProxyLog(ctx context.Context, log *models.ProxyLog)

func (*Tee) RecordProxyLogsBatch 

func (t *Tee) RecordProxyLogsBatch(ctx context.Context, logs []*models.ProxyLog)

func (*Tee) RecordToolCall 

func (t *Tee) RecordToolCall(ctx context.Context, name string, timestamp time.Time, execTime int, toolID uint)

func (*Tee) SetAsGlobalHandler 

func (t *Tee) SetAsGlobalHandler()

SetAsGlobalHandler makes the Tee the process's analytics handler.

func (*Tee) Stop 

func (t *Tee) Stop(timeout time.Duration)

Stop waits up to timeout for the sinks to take what is queued. Records sent after Stop are lost to the sinks (the primary still gets them).

type VendorModelCost 

type VendorModelCost struct {
	Model            string  `json:"model"`
	ModelPriceID     *uint   `json:"modelPriceId"`
	TotalCost        float64 `json:"totalCost"`
	PromptCost       float64 `json:"promptCost"`
	ResponseCost     float64 `json:"responseCost"`
	CacheWriteCost   float64 `json:"cacheWriteCost"`
	CacheReadCost    float64 `json:"cacheReadCost"`
	TotalTokens      int64   `json:"totalTokens"` // Total of all token types
	PromptTokens     int64   `json:"promptTokens"`
	ResponseTokens   int64   `json:"responseTokens"`
	CacheWriteTokens int64   `json:"cacheWriteTokens"`
	CacheReadTokens  int64   `json:"cacheReadTokens"`
	// Usage columns backing the "Models in use" view on the LLM details page.
	RequestCount int64    `json:"requestCount"`
	AppCount     int64    `json:"appCount"`
	LastUsed     ScanTime `json:"lastUsed"`
}

VendorModelCost represents the total cost for a specific vendor and model

func GetTotalCostPerVendorAndModel 

func GetTotalCostPerVendorAndModel(db *gorm.DB, startDate, endDate time.Time, interactionType *models.InteractionType, llmID *uint) ([]VendorModelCost, error)

GetTotalCostPerVendorAndModel returns the total cost per vendor and model with detailed breakdowns

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.