tracking

package
v1.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 2, 2026 License: MIT Imports: 17 Imported by: 0

Documentation

Overview

Package tracking provides cost estimation and reporting.

Package tracking provides per-layer statistics tracking.

Index

Constants

View Source 
const AddAgentAttributionColumns = `` /* 155-byte string literal not displayed */

AddAgentAttributionColumns adds columns for tracking AI agent context. Enables per-model, per-provider, per-agent token savings analysis.

View Source 
const AddCompositeIndexes = `` /* 252-byte string literal not displayed */

AddCompositeIndexes adds composite indexes for common query patterns. T181: Composite indexes on (project_path, timestamp) and (command, timestamp).

View Source 
const AgentAttributionIndexes = `` /* 543-byte string literal not displayed */

AgentAttributionIndexes defines indexes for agent attribution.

View Source 
const CreateAgentSummaryView = `` /* 426-byte string literal not displayed */

CreateAgentSummaryView creates a view for per-agent statistics.

View Source 
const CreateCommandsTable = `` /* 732-byte string literal not displayed */

CreateCommandsTable creates the main commands table.

View Source 
const CreateLayerStatsTable = `` /* 466-byte string literal not displayed */

CreateLayerStatsTable tracks per-layer savings for detailed analysis. T184: Per-layer savings tracking.

View Source 
const CreateMigrationTable = `` /* 134-byte string literal not displayed */

MigrationHistory tracks applied migrations.

View Source 
const CreateParseFailuresTable = `` /* 332-byte string literal not displayed */

CreateParseFailuresTable creates a table for tracking parse failures.

View Source 
const CreateSummaryView = `` /* 341-byte string literal not displayed */

CreateSummaryView creates a view for aggregated statistics.

View Source 
const HistoryRetentionDays = 90

HistoryRetentionDays is the number of days to retain tracking data. Records older than this are automatically cleaned up on each write.

View Source 
const SchemaVersion = 4

SchemaVersion is the current database schema version.

Variables

View Source 
var CommandColumnDefs = []struct {
	Name string
	Type string
}{
	{"agent_name", "TEXT"},
	{"model_name", "TEXT"},
	{"provider", "TEXT"},
	{"model_family", "TEXT"},
	{"context_kind", "TEXT"},
	{"context_mode", "TEXT"},
	{"context_resolved_mode", "TEXT"},
	{"context_target", "TEXT"},
	{"context_related_files", "INTEGER NOT NULL DEFAULT 0"},
	{"context_bundle", "BOOLEAN NOT NULL DEFAULT 0"},
}

CommandColumnDefs defines optional columns added after the base table exists.

View Source 
var Migrations = []string{
	CreateCommandsTable,
	CreateSummaryView,
	CreateParseFailuresTable,
	AddCompositeIndexes,
	CreateLayerStatsTable,
	AddAgentAttributionColumns,
	CreateAgentSummaryView,
}

Migrations contains all migration statements in order.

View Source 
var ModelPricing = map[string]CostEstimator{
	"claude-3-opus":     {/* contains filtered or unexported fields */},
	"claude-3-sonnet":   {/* contains filtered or unexported fields */},
	"claude-3-haiku":    {/* contains filtered or unexported fields */},
	"claude-3.5-sonnet": {/* contains filtered or unexported fields */},
	"claude-3.5-haiku":  {/* contains filtered or unexported fields */},
	"gpt-4-turbo":       {/* contains filtered or unexported fields */},
	"gpt-4":             {/* contains filtered or unexported fields */},
	"gpt-3.5-turbo":     {/* contains filtered or unexported fields */},
	"default":           {/* contains filtered or unexported fields */},
}

ModelPricing contains pricing for different LLM models.

Functions

func DatabasePath 

func DatabasePath() string

DatabasePath returns the default database path. Delegates to config.DatabasePath for consistent XDG compliance.

func EstimateTokens 

func EstimateTokens(text string) int

EstimateTokens provides a heuristic token count. Delegates to core.EstimateTokens for single source of truth (T22).

func InitSchema 

func InitSchema() []string

InitSchema initializes all database tables and migrations.

Types

type Alert 

type Alert struct {
	Type      string
	Severity  string
	Message   string
	Timestamp time.Time
}

Alert represents a threshold alert.

type AlertThreshold 

type AlertThreshold struct {
	DailyTokenLimit  int64
	DailyCostLimit   float64
	WeeklyTokenLimit int64
	WeeklyCostLimit  float64
}

AlertThreshold represents alert configuration.

type CommandCost 

type CommandCost struct {
	Command     string
	TimesRun    int
	TokensSaved int
	CostSaved   float64
}

CommandCost represents cost data for a command.

type CommandFailureCount 

type CommandFailureCount struct {
	Command string `json:"command"`
	Count   int    `json:"count"`
}

CommandFailureCount represents a command and its failure count.

type CommandRecord 

type CommandRecord struct {
	ID             int64     `json:"id"`
	Command        string    `json:"command"`
	OriginalOutput string    `json:"original_output,omitempty"`
	FilteredOutput string    `json:"filtered_output,omitempty"`
	OriginalTokens int       `json:"original_tokens"`
	FilteredTokens int       `json:"filtered_tokens"`
	SavedTokens    int       `json:"saved_tokens"`
	ProjectPath    string    `json:"project_path"`
	SessionID      string    `json:"session_id,omitempty"`
	ExecTimeMs     int64     `json:"exec_time_ms"`
	Timestamp      time.Time `json:"timestamp"`
	ParseSuccess   bool      `json:"parse_success"`
	// AI Agent attribution fields
	AgentName   string `json:"agent_name,omitempty"`   // e.g., "Claude Code", "OpenCode", "Cursor"
	ModelName   string `json:"model_name,omitempty"`   // e.g., "claude-3-opus", "gpt-4", "gemini-pro"
	Provider    string `json:"provider,omitempty"`     // e.g., "Anthropic", "OpenAI", "Google"
	ModelFamily string `json:"model_family,omitempty"` // e.g., "claude", "gpt", "gemini"
	// Smart context read metadata
	ContextKind         string `json:"context_kind,omitempty"`          // e.g., "read", "delta", "mcp"
	ContextMode         string `json:"context_mode,omitempty"`          // requested mode: auto, graph, delta, ...
	ContextResolvedMode string `json:"context_resolved_mode,omitempty"` // effective mode after auto-resolution
	ContextTarget       string `json:"context_target,omitempty"`        // file path or target identifier
	ContextRelatedFiles int    `json:"context_related_files,omitempty"` // number of related files included
	ContextBundle       bool   `json:"context_bundle,omitempty"`        // whether multiple files were delivered together
}

CommandRecord represents a single command execution record.

type CommandStats 

type CommandStats struct {
	Command        string  `json:"command"`
	ExecutionCount int     `json:"execution_count"`
	TotalSaved     int     `json:"total_saved"`
	TotalOriginal  int     `json:"total_original"`
	ReductionPct   float64 `json:"reduction_percent"`
}

CommandStats represents statistics for a specific command type.

type CostEstimate 

type CostEstimate struct {
	TokensSaved      int
	EstimatedSavings float64
	Currency         string
	CostPer1MTokens  float64
}

CostEstimate contains cost estimation results.

func (CostEstimate) Format 

func (ce CostEstimate) Format() string

Format returns a formatted string representation.

type CostEstimator 

type CostEstimator struct {
	// contains filtered or unexported fields
}

CostEstimator estimates API costs based on token usage.

func NewCostEstimator 

func NewCostEstimator(model string) *CostEstimator

NewCostEstimator creates a new cost estimator.

func (*CostEstimator) EstimateCost 

func (ce *CostEstimator) EstimateCost(tokens int) float64

EstimateCost estimates the cost for a number of tokens.

func (*CostEstimator) EstimateSavings 

func (ce *CostEstimator) EstimateSavings(tokensSaved int) CostEstimate

EstimateSavings estimates cost savings from token reduction.

type CostProjection 

type CostProjection struct {
	MonthlyEstimate float64
	YearlyEstimate  float64
	GrowthRate      float64 // Monthly growth rate
}

CostProjection contains future cost projections.

type CostReport 

type CostReport struct {
	GeneratedAt      time.Time
	Period           string
	TotalTokensSaved int64
	EstimatedSavings float64
	Currency         string
	Model            string
	DailyBreakdown   []DailyCost
	TopCommands      []CommandCost
	Projections      CostProjection
}

CostReport contains comprehensive cost reporting.

func (*CostReport) ExportToCSV 

func (cr *CostReport) ExportToCSV(path string) error

ExportToCSV exports the cost report to CSV.

func (*CostReport) ExportToJSON 

func (cr *CostReport) ExportToJSON(path string) error

ExportToJSON exports the cost report to JSON.

func (*CostReport) Format 

func (cr *CostReport) Format() string

Format returns a formatted report string.

type DailyCost 

type DailyCost struct {
	Date         string
	TokensSaved  int
	CostSaved    float64
	CommandCount int
}

DailyCost represents cost data for a single day.

type InstrumentedProcessor added in v1.5.0 

type InstrumentedProcessor struct {
	// contains filtered or unexported fields
}

InstrumentedProcessor wraps a processor with tracking.

func NewInstrumentedProcessor added in v1.5.0 

func NewInstrumentedProcessor(name string, processor func(string) (string, int), tracker *LayerTracker) *InstrumentedProcessor

NewInstrumentedProcessor creates an instrumented processor.

func (*InstrumentedProcessor) Process added in v1.5.0 

func (ip *InstrumentedProcessor) Process(input string) (string, int)

Process processes content with tracking.

type LayerReport added in v1.5.0 

type LayerReport struct {
	GeneratedAt time.Time            `json:"generated_at"`
	Layers      map[string]LayerStat `json:"layers"`
	Total       LayerStat            `json:"total"`
	TopSavers   []*LayerStat         `json:"top_savers"`
	MostUsed    []*LayerStat         `json:"most_used"`
}

LayerReport contains a full statistics report.

func (*LayerReport) String added in v1.5.0 

func (r *LayerReport) String() string

String returns a formatted report.

type LayerStat added in v1.5.0 

type LayerStat struct {
	Name        string        `json:"name"`
	Invocations int64         `json:"invocations"`
	TokensIn    int64         `json:"tokens_in"`
	TokensOut   int64         `json:"tokens_out"`
	TokensSaved int64         `json:"tokens_saved"`
	TotalTime   time.Duration `json:"total_time"`
	AvgTime     time.Duration `json:"avg_time"`
	LastUsed    time.Time     `json:"last_used"`
	Errors      int64         `json:"errors"`
}

LayerStat tracks statistics for a single layer.

func (*LayerStat) CompressionRatio added in v1.5.0 

func (ls *LayerStat) CompressionRatio() float64

CompressionRatio returns the compression ratio.

func (*LayerStat) SavingsPercent added in v1.5.0 

func (ls *LayerStat) SavingsPercent() float64

SavingsPercent returns the percentage of tokens saved.

type LayerStatRecord 

type LayerStatRecord struct {
	LayerName   string
	TokensSaved int
	DurationUs  int64
}

LayerStatRecord holds per-layer statistics for database recording.

type LayerTracker added in v1.5.0 

type LayerTracker struct {
	// contains filtered or unexported fields
}

LayerTracker tracks statistics for all layers.

func GetGlobalLayerTracker added in v1.5.0 

func GetGlobalLayerTracker() *LayerTracker

GetGlobalLayerTracker returns the global tracker.

func NewLayerTracker added in v1.5.0 

func NewLayerTracker() *LayerTracker

NewLayerTracker creates a new layer tracker.

func (*LayerTracker) Get added in v1.5.0 

func (lt *LayerTracker) Get(layerName string) (*LayerStat, bool)

Get returns statistics for a layer.

func (*LayerTracker) GetAll added in v1.5.0 

func (lt *LayerTracker) GetAll() map[string]*LayerStat

GetAll returns all layer statistics.

func (*LayerTracker) GetMostUsed added in v1.5.0 

func (lt *LayerTracker) GetMostUsed(n int) []*LayerStat

GetMostUsed returns the top N most used layers.

func (*LayerTracker) GetTopSavers added in v1.5.0 

func (lt *LayerTracker) GetTopSavers(n int) []*LayerStat

GetTopSavers returns the top N layers by tokens saved.

func (*LayerTracker) Record added in v1.5.0 

func (lt *LayerTracker) Record(layerName string, tokensIn, tokensOut int, duration time.Duration)

Record records a layer invocation.

func (*LayerTracker) RecordError added in v1.5.0 

func (lt *LayerTracker) RecordError(layerName string)

RecordError records a layer error.

func (*LayerTracker) Report added in v1.5.0 

func (lt *LayerTracker) Report() *LayerReport

Report generates a statistics report.

func (*LayerTracker) Reset added in v1.5.0 

func (lt *LayerTracker) Reset()

Reset resets all statistics.

func (*LayerTracker) SetUpdateCallback added in v1.5.0 

func (lt *LayerTracker) SetUpdateCallback(cb func(string, *LayerStat))

SetUpdateCallback sets a callback for updates.

func (*LayerTracker) TotalStats added in v1.5.0 

func (lt *LayerTracker) TotalStats() LayerStat

TotalStats returns aggregated statistics.

type ParseFailureRecord 

type ParseFailureRecord struct {
	ID                int64     `json:"id"`
	Timestamp         time.Time `json:"timestamp"`
	RawCommand        string    `json:"raw_command"`
	ErrorMessage      string    `json:"error_message"`
	FallbackSucceeded bool      `json:"fallback_succeeded"`
}

ParseFailureRecord represents a single parse failure event.

type ParseFailureSummary 

type ParseFailureSummary struct {
	Total          int64                 `json:"total"`
	RecoveryRate   float64               `json:"recovery_rate"`
	TopCommands    []CommandFailureCount `json:"top_commands"`
	RecentFailures []ParseFailureRecord  `json:"recent_failures"`
}

ParseFailureSummary represents aggregated parse failure analytics.

type ReportFilter 

type ReportFilter struct {
	ProjectPath string     `json:"project_path,omitempty"`
	SessionID   string     `json:"session_id,omitempty"`
	Command     string     `json:"command,omitempty"`
	StartTime   *time.Time `json:"start_time,omitempty"`
	EndTime     *time.Time `json:"end_time,omitempty"`
}

ReportFilter represents filters for generating reports.

type SavingsSummary 

type SavingsSummary struct {
	TotalCommands int     `json:"total_commands"`
	TotalSaved    int     `json:"total_saved"`
	TotalOriginal int     `json:"total_original"`
	TotalFiltered int     `json:"total_filtered"`
	ReductionPct  float64 `json:"reduction_percent"`
}

SavingsSummary represents aggregated token savings.

type SessionInfo 

type SessionInfo struct {
	SessionID   string    `json:"session_id"`
	StartedAt   time.Time `json:"started_at"`
	ProjectPath string    `json:"project_path"`
}

SessionInfo represents information about a shell session.

type TimedExecution 

type TimedExecution struct {
	// contains filtered or unexported fields
}

TimedExecution tracks execution time and token savings.

func Start 

func Start() *TimedExecution

Start begins a timed execution for tracking.

func (*TimedExecution) Track 

func (t *TimedExecution) Track(command, tokmanCmd string, originalTokens, filteredTokens int)

Track records the execution with token savings. Automatically captures AI agent attribution from environment variables:

  • TOKMAN_AGENT: AI agent name (e.g., "Claude Code", "OpenCode", "Cursor")
  • TOKMAN_MODEL: Model name (e.g., "claude-3-opus", "gpt-4")
  • TOKMAN_PROVIDER: Provider name (e.g., "Anthropic", "OpenAI")

type Tracker 

type Tracker struct {
	// contains filtered or unexported fields
}

Tracker manages token tracking persistence.

func GetGlobalTracker 

func GetGlobalTracker() *Tracker

GetGlobalTracker returns the global tracker instance (exported for external use).

func NewTracker 

func NewTracker(dbPath string) (*Tracker, error)

NewTracker creates a new Tracker with the given database path.

func (*Tracker) CheckAlert 

func (t *Tracker) CheckAlert(threshold AlertThreshold) ([]Alert, error)

CheckAlert checks if any thresholds are exceeded.

func (*Tracker) CleanupOld 

func (t *Tracker) CleanupOld() (int64, error)

CleanupOld manually triggers cleanup of old records. Returns the number of records deleted.

func (*Tracker) CleanupWithRetention 

func (t *Tracker) CleanupWithRetention(days int) (int64, error)

CleanupWithRetention removes records older than specified days. T183: Configurable data retention policy.

func (*Tracker) Close 

func (t *Tracker) Close() error

Close closes the database connection.

func (*Tracker) CountCommandsSince 

func (t *Tracker) CountCommandsSince(since time.Time) (int64, error)

CountCommandsSince returns the count of commands executed since the given time.

func (*Tracker) DatabaseSize 

func (t *Tracker) DatabaseSize() (int64, error)

DatabaseSize returns the size of the tracking database in bytes.

func (*Tracker) ExportReport 

func (t *Tracker) ExportReport(format, outputPath string, days int) error

ExportReport exports a report in the specified format.

func (*Tracker) GenerateCostReport 

func (t *Tracker) GenerateCostReport(model string, days int) (*CostReport, error)

GenerateCostReport generates a comprehensive cost report.

func (*Tracker) GetCommandStats 

func (t *Tracker) GetCommandStats(projectPath string) ([]CommandStats, error)

GetCommandStats returns statistics grouped by command. When projectPath is empty, returns all commands without filtering.

func (*Tracker) GetDailySavings 

func (t *Tracker) GetDailySavings(projectPath string, days int) ([]struct {
	Date     string
	Saved    int
	Original int
	Commands int
}, error)

GetDailySavings returns token savings grouped by day.

func (*Tracker) GetLayerStats 

func (t *Tracker) GetLayerStats(commandID int64) ([]LayerStatRecord, error)

GetLayerStats returns per-layer statistics for a command.

func (*Tracker) GetParseFailureSummary 

func (t *Tracker) GetParseFailureSummary() (*ParseFailureSummary, error)

GetParseFailureSummary returns aggregated parse failure analytics.

func (*Tracker) GetRecentCommands 

func (t *Tracker) GetRecentCommands(projectPath string, limit int) ([]CommandRecord, error)

GetRecentCommands returns the most recent command executions. When projectPath is empty, returns all recent commands without filtering.

func (*Tracker) GetRecentCommandsForPatterns 

func (t *Tracker) GetRecentCommandsForPatterns(projectPath string, limit int, commandPatterns []string) ([]CommandRecord, error)

GetRecentCommandsForPatterns returns recent commands optionally filtered by command GLOB patterns. When commandPatterns is empty, it returns all commands.

func (*Tracker) GetRecentContextReads 

func (t *Tracker) GetRecentContextReads(projectPath, kind, mode string, limit int) ([]CommandRecord, error)

GetRecentContextReads returns recent smart-read records using structured metadata when available, with legacy command fallback for older rows.

func (*Tracker) GetSavings 

func (t *Tracker) GetSavings(projectPath string) (*SavingsSummary, error)

GetSavings returns the total token savings for a project path. Uses GLOB matching for case-sensitive path comparison. When projectPath is empty, returns all records without filtering.

func (*Tracker) GetSavingsForCommands 

func (t *Tracker) GetSavingsForCommands(projectPath string, commandPatterns []string) (*SavingsSummary, error)

GetSavingsForCommands returns token savings for commands matching any of the provided GLOB patterns. When commandPatterns is empty, it returns all records.

func (*Tracker) GetSavingsForContextReads 

func (t *Tracker) GetSavingsForContextReads(projectPath, kind, mode string) (*SavingsSummary, error)

GetSavingsForContextReads returns smart-read savings using structured metadata when available, with command-pattern fallback for older records.

func (*Tracker) GetTopLayers 

func (t *Tracker) GetTopLayers(limit int) ([]struct {
	LayerName  string
	TotalSaved int64
	AvgSaved   float64
	CallCount  int64
}, error)

GetTopLayers returns the most effective compression layers.

func (*Tracker) OverallSavingsPct 

func (t *Tracker) OverallSavingsPct() (float64, error)

OverallSavingsPct returns the overall savings percentage across all commands.

func (*Tracker) Query 

func (t *Tracker) Query(query string, args ...any) (*sql.Rows, error)

Query executes a raw SQL query and returns the rows. This is exposed for custom aggregations in the economics package.

func (*Tracker) QueryRow 

func (t *Tracker) QueryRow(query string, args ...any) *sql.Row

QueryRow executes a raw SQL query expected to return at most one row.

func (*Tracker) Record 

func (t *Tracker) Record(record *CommandRecord) error

Record saves a command execution to the database.

func (*Tracker) RecordContext 

func (t *Tracker) RecordContext(ctx context.Context, record *CommandRecord) error

RecordContext saves a command execution to the database with context support. P3: Enables cancellation for long-running database operations.

func (*Tracker) RecordLayerStats 

func (t *Tracker) RecordLayerStats(commandID int64, stats []LayerStatRecord) error

RecordLayerStats saves per-layer statistics for a command. T184: Per-layer savings tracking.

func (*Tracker) RecordParseFailure 

func (t *Tracker) RecordParseFailure(rawCommand string, errorMessage string, fallbackSucceeded bool) error

RecordParseFailure records a parse failure event.

func (*Tracker) TokensSaved24h 

func (t *Tracker) TokensSaved24h() (int64, error)

TokensSaved24h returns tokens saved in the last 24 hours.

func (*Tracker) TokensSavedTotal 

func (t *Tracker) TokensSavedTotal() (int64, error)

TokensSavedTotal returns total tokens saved across all time.

func (*Tracker) TopCommands 

func (t *Tracker) TopCommands(limit int) ([]string, error)

TopCommands returns the top N commands by execution count.

func (*Tracker) Vacuum 

func (t *Tracker) Vacuum() error

Vacuum reclaims unused space in the database.

type TrackerInterface 

type TrackerInterface interface {
	Record(record *CommandRecord) error
	GetSavings(projectPath string) (*SavingsSummary, error)
	GetRecentCommands(projectPath string, limit int) ([]CommandRecord, error)
	Query(query string, args ...any) (*sql.Rows, error)
	Close() error
}

TrackerInterface defines the contract for command tracking. Implementations can use SQLite, in-memory stores, or mocks for testing.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.