benchmarking

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 6, 2026 License: MIT Imports: 20 Imported by: 0

Documentation

Overview

Package benchmarking provides chart generation capabilities

Package benchmarking provides DSL for defining benchmarks

Package benchmarking provides comprehensive performance benchmarking for TokMan

Package benchmarking provides storage capabilities for benchmark results

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CalculateRegressionScore

func CalculateRegressionScore(regressions []Regression) float64

CalculateRegressionScore calculates an overall regression score (0-100)

func CompareReports

func CompareReports(baselineData, currentData []byte) (string, error)

CompareReports compares two benchmark reports

func CreateComment

func CreateComment(report *SuiteReport, regressions []Regression) string

CreateComment creates a formatted comment for posting to CI

func ExportReport

func ExportReport(report *SuiteReport, format string, w io.Writer) error

ExportReport exports a report in the specified format

func ExportTrendReport

func ExportTrendReport(report *TrendReport) ([]byte, error)

ExportTrendReport exports a trend report as JSON

func FormatRegressionReport

func FormatRegressionReport(report *RegressionReport) string

FormatReport formats a regression report as a string

func FormatTrendReport

func FormatTrendReport(report *TrendReport) string

FormatTrendReport formats a trend report as a string

func GenerateGitHubActionsWorkflow

func GenerateGitHubActionsWorkflow() string

GenerateGitHubActionsWorkflow generates GitHub Actions workflow YAML

func GenerateGitLabCIConfig

func GenerateGitLabCIConfig() string

GenerateGitLabCIConfig generates GitLab CI configuration

func GenerateReportCharts

func GenerateReportCharts(report *SuiteReport) string

GenerateReportCharts generates charts for a benchmark report

func GenerateTrendChart

func GenerateTrendChart(trend *Trend) string

GenerateTrendChart generates a trend chart from historical data

func IsRegression

func IsRegression(baseline, current float64, thresholdPct float64, higherIsBetter bool) bool

IsRegression checks if a single change constitutes a regression

func ResetRunner

func ResetRunner()

ResetRunner resets the global runner (useful for testing)

func StandardDSLTemplates

func StandardDSLTemplates() map[string]string

StandardDSLTemplates provides common benchmark templates

Types

type AxisOptions

type AxisOptions struct {
	Label string
	Min   *float64
	Max   *float64
}

AxisOptions represents axis options

type Benchmark

type Benchmark interface {
	Name() string
	Type() BenchmarkType
	Run(ctx context.Context) (*BenchmarkResult, error)
}

Benchmark interface for pluggable benchmarks

type BenchmarkBuilder

type BenchmarkBuilder struct {
	// contains filtered or unexported fields
}

BenchmarkBuilder builds a single benchmark definition

func (*BenchmarkBuilder) WithDescription

func (b *BenchmarkBuilder) WithDescription(desc string) *BenchmarkBuilder

WithDescription sets the description

func (*BenchmarkBuilder) WithDuration

func (b *BenchmarkBuilder) WithDuration(duration time.Duration) *BenchmarkBuilder

WithDuration sets the duration

func (*BenchmarkBuilder) WithIterations

func (b *BenchmarkBuilder) WithIterations(iterations int) *BenchmarkBuilder

WithIterations sets the iterations

func (*BenchmarkBuilder) WithParameter

func (b *BenchmarkBuilder) WithParameter(key string, value interface{}) *BenchmarkBuilder

WithParameter adds a parameter

func (*BenchmarkBuilder) WithType

func (b *BenchmarkBuilder) WithType(benchmarkType string) *BenchmarkBuilder

WithType sets the benchmark type

func (*BenchmarkBuilder) WithWarmup

func (b *BenchmarkBuilder) WithWarmup(warmup int) *BenchmarkBuilder

WithWarmup sets the warmup iterations

type BenchmarkDefinition

type BenchmarkDefinition struct {
	Name        string
	Type        string
	Description string
	Iterations  int
	Duration    time.Duration
	Warmup      int
	Parameters  map[string]interface{}
	Setup       string
	Teardown    string
	Validators  []ValidatorDefinition
}

BenchmarkDefinition represents a benchmark defined via DSL

func ParseDSLFile

func ParseDSLFile(filename string) ([]BenchmarkDefinition, error)

ParseDSLFile parses a DSL file

func (*BenchmarkDefinition) ToBenchmark

func (def *BenchmarkDefinition) ToBenchmark() (Benchmark, error)

ToBenchmark converts definition to actual benchmark

type BenchmarkPool

type BenchmarkPool struct {
	// contains filtered or unexported fields
}

BenchmarkPool manages a pool of benchmark workers

func NewBenchmarkPool

func NewBenchmarkPool(workers int) *BenchmarkPool

NewBenchmarkPool creates a new benchmark pool

func (*BenchmarkPool) Results

func (bp *BenchmarkPool) Results() <-chan BenchmarkResult

Results returns the result channel

func (*BenchmarkPool) Start

func (bp *BenchmarkPool) Start()

Start starts the worker pool

func (*BenchmarkPool) Stop

func (bp *BenchmarkPool) Stop()

Stop stops the worker pool

func (*BenchmarkPool) Submit

func (bp *BenchmarkPool) Submit(bm Benchmark) bool

Submit submits a benchmark to the pool

type BenchmarkResult

type BenchmarkResult struct {
	Name         string
	Type         BenchmarkType
	Duration     time.Duration
	TokensIn     int
	TokensOut    int
	Throughput   float64 // tokens per second
	MemoryUsedMB float64
	Allocations  uint64
	LatencyP50   time.Duration
	LatencyP95   time.Duration
	LatencyP99   time.Duration
	Errors       int
	SuccessRate  float64
	Timestamp    time.Time
	Metadata     map[string]string
}

BenchmarkResult holds the results of a single benchmark run

func (*BenchmarkResult) ToJSON

ToJSON converts BenchmarkResult to JSON format

type BenchmarkResultJSON

type BenchmarkResultJSON struct {
	Name         string            `json:"name"`
	Type         string            `json:"type"`
	Duration     string            `json:"duration"`
	TokensIn     int               `json:"tokens_in"`
	TokensOut    int               `json:"tokens_out"`
	Throughput   float64           `json:"throughput"`
	MemoryUsedMB float64           `json:"memory_used_mb"`
	Allocations  uint64            `json:"allocations"`
	LatencyP50   string            `json:"latency_p50"`
	LatencyP95   string            `json:"latency_p95"`
	LatencyP99   string            `json:"latency_p99"`
	Errors       int               `json:"errors"`
	SuccessRate  float64           `json:"success_rate"`
	Timestamp    time.Time         `json:"timestamp"`
	Metadata     map[string]string `json:"metadata,omitempty"`
}

BenchmarkResultJSON is a JSON-serializable version of BenchmarkResult

type BenchmarkType

type BenchmarkType string

BenchmarkType defines the type of benchmark to run

const (
	TypeCompression BenchmarkType = "compression"
	TypePipeline    BenchmarkType = "pipeline"
	TypeMemory      BenchmarkType = "memory"
	TypeConcurrency BenchmarkType = "concurrency"
	TypeEndToEnd    BenchmarkType = "e2e"
)

type CICDConfig

type CICDConfig struct {
	Provider         string
	Repository       string
	Branch           string
	CommitSHA        string
	PullRequestID    string
	BuildID          string
	Token            string
	ResultsPath      string
	BaselineBranch   string
	FailOnRegression bool
	Thresholds       RegressionThresholds
}

CICDConfig holds CI/CD configuration

func LoadConfigFromEnv

func LoadConfigFromEnv() CICDConfig

LoadConfigFromEnv loads configuration from environment variables

type CICDIntegration

type CICDIntegration struct {
	// contains filtered or unexported fields
}

CICDIntegration provides CI/CD integration for benchmarks

func NewCICDIntegration

func NewCICDIntegration(config CICDConfig) *CICDIntegration

NewCICDIntegration creates a new CI/CD integration

func (*CICDIntegration) RunAndReport

func (ci *CICDIntegration) RunAndReport(suite *Suite) error

RunAndReport runs benchmarks and reports results to CI/CD

type CSVFormatter

type CSVFormatter struct{}

CSVFormatter formats benchmark results as CSV

func NewCSVFormatter

func NewCSVFormatter() *CSVFormatter

NewCSVFormatter creates a new CSV formatter

func (*CSVFormatter) FormatReport

func (f *CSVFormatter) FormatReport(w io.Writer, report *SuiteReport) error

FormatReport formats a suite report as CSV

func (*CSVFormatter) FormatSummary

func (f *CSVFormatter) FormatSummary(w io.Writer, stats *SummaryStats) error

FormatSummary formats summary statistics as CSV

type Chart

type Chart struct {
	Type    ChartType
	Title   string
	Width   int
	Height  int
	Data    ChartData
	Options ChartOptions
}

Chart represents a generated chart

type ChartData

type ChartData struct {
	Labels []string
	Series []DataSeries
}

ChartData represents chart data

type ChartGenerator

type ChartGenerator struct {
	// contains filtered or unexported fields
}

ChartGenerator generates charts

func NewChartGenerator

func NewChartGenerator() *ChartGenerator

NewChartGenerator creates a new chart generator

func (*ChartGenerator) GenerateBarChart

func (cg *ChartGenerator) GenerateBarChart(title string, labels []string, values []float64) string

GenerateBarChart generates an ASCII bar chart

func (*ChartGenerator) GenerateComparisonChart

func (cg *ChartGenerator) GenerateComparisonChart(title string, baseline, current map[string]float64) string

GenerateComparisonChart generates a comparison chart

func (*ChartGenerator) GenerateHistogram

func (cg *ChartGenerator) GenerateHistogram(title string, values []float64, bins int) string

GenerateHistogram generates an ASCII histogram

func (*ChartGenerator) GenerateLineChart

func (cg *ChartGenerator) GenerateLineChart(title string, labels []string, series []DataSeries) string

GenerateLineChart generates an ASCII line chart

type ChartOptions

type ChartOptions struct {
	XAxis  AxisOptions
	YAxis  AxisOptions
	Legend bool
	Grid   bool
}

ChartOptions represents chart options

type ChartType

type ChartType string

ChartType represents the type of chart

const (
	ChartBar       ChartType = "bar"
	ChartLine      ChartType = "line"
	ChartScatter   ChartType = "scatter"
	ChartHeatmap   ChartType = "heatmap"
	ChartBox       ChartType = "box"
	ChartHistogram ChartType = "histogram"
)

type CommentPoster

type CommentPoster interface {
	Post(comment string) error
}

CommentPoster posts comments to CI systems

type CommitStatus

type CommitStatus string

CommitStatus represents commit status

const (
	StatusSuccess CommitStatus = "success"
	StatusFailure CommitStatus = "failure"
	StatusPending CommitStatus = "pending"
	StatusError   CommitStatus = "error"
)

type Comparison

type Comparison struct {
	Baseline  *SuiteReport
	Current   *SuiteReport
	Changes   map[string]MetricChange
	Improved  []string
	Regressed []string
	Unchanged []string
}

Comparison represents a comparison between two benchmark reports

type CompressionBenchmark

type CompressionBenchmark struct {
	// contains filtered or unexported fields
}

CompressionBenchmark benchmarks the compression pipeline

func NewCompressionBenchmark

func NewCompressionBenchmark(name string, inputSize int) *CompressionBenchmark

NewCompressionBenchmark creates a compression benchmark

func (*CompressionBenchmark) Name

func (b *CompressionBenchmark) Name() string

func (*CompressionBenchmark) Run

func (*CompressionBenchmark) Type

type ConcurrencyBenchmark

type ConcurrencyBenchmark struct {
	// contains filtered or unexported fields
}

ConcurrencyBenchmark tests concurrent operations

func NewConcurrencyBenchmark

func NewConcurrencyBenchmark(name string, workers, tasks int) *ConcurrencyBenchmark

NewConcurrencyBenchmark creates a concurrency benchmark

func (*ConcurrencyBenchmark) Name

func (b *ConcurrencyBenchmark) Name() string

func (*ConcurrencyBenchmark) Run

func (*ConcurrencyBenchmark) Type

func (*ConcurrencyBenchmark) WithWork

func (b *ConcurrencyBenchmark) WithWork(fn func() error) *ConcurrencyBenchmark

type ConcurrencyLimiter

type ConcurrencyLimiter struct {
	// contains filtered or unexported fields
}

ConcurrencyLimiter limits concurrent benchmark execution

func NewConcurrencyLimiter

func NewConcurrencyLimiter(maxConcurrent int) *ConcurrencyLimiter

NewConcurrencyLimiter creates a new concurrency limiter

func (*ConcurrencyLimiter) Acquire

func (cl *ConcurrencyLimiter) Acquire()

Acquire acquires a slot

func (*ConcurrencyLimiter) Release

func (cl *ConcurrencyLimiter) Release()

Release releases a slot

type DSLBuilder

type DSLBuilder struct {
	// contains filtered or unexported fields
}

DSLBuilder helps build benchmark DSL programmatically

func NewDSLBuilder

func NewDSLBuilder() *DSLBuilder

NewDSLBuilder creates a new DSL builder

func (*DSLBuilder) AddBenchmark

func (b *DSLBuilder) AddBenchmark(name string) *BenchmarkBuilder

AddBenchmark adds a benchmark definition

func (*DSLBuilder) Build

func (b *DSLBuilder) Build() string

Build generates DSL text

type DSLParser

type DSLParser struct {
	// contains filtered or unexported fields
}

DSLParser parses benchmark DSL

func NewDSLParser

func NewDSLParser() *DSLParser

NewDSLParser creates a new DSL parser

func (*DSLParser) Parse

func (p *DSLParser) Parse(dsl string) ([]BenchmarkDefinition, error)

Parse parses DSL text into benchmark definitions

type DataSeries

type DataSeries struct {
	Name   string
	Color  string
	Values []float64
}

DataSeries represents a data series

type EndToEndBenchmark

type EndToEndBenchmark struct {
	// contains filtered or unexported fields
}

EndToEndBenchmark tests the complete system

func NewEndToEndBenchmark

func NewEndToEndBenchmark(name, scenario string) *EndToEndBenchmark

NewEndToEndBenchmark creates an E2E benchmark

func (*EndToEndBenchmark) AddStep

func (b *EndToEndBenchmark) AddStep(fn func() error) *EndToEndBenchmark

func (*EndToEndBenchmark) Name

func (b *EndToEndBenchmark) Name() string

func (*EndToEndBenchmark) Run

func (*EndToEndBenchmark) Type

func (b *EndToEndBenchmark) Type() BenchmarkType

type GitHubCommentPoster

type GitHubCommentPoster struct {
	Token string
	Repo  string
	PR    string
}

GitHubCommentPoster posts to GitHub

func (*GitHubCommentPoster) Post

func (g *GitHubCommentPoster) Post(comment string) error

Post posts a comment to GitHub

type HistoricalPoint

type HistoricalPoint struct {
	Timestamp time.Time
	Value     float64
}

HistoricalPoint represents a single data point

type HistoricalResult

type HistoricalResult struct {
	Timestamp   time.Time
	Value       float64
	Metric      string
	BenchmarkID string
	Metadata    map[string]string
}

HistoricalResult represents a benchmark result at a point in time

type Hook

type Hook interface {
	BeforeBenchmark(name string)
	AfterBenchmark(result *BenchmarkResult)
}

Hook allows extensibility in the benchmark lifecycle

type JSONFormatter

type JSONFormatter struct {
	Pretty bool
}

JSONFormatter formats benchmark results as JSON

func NewJSONFormatter

func NewJSONFormatter(pretty bool) *JSONFormatter

NewJSONFormatter creates a new JSON formatter

func (*JSONFormatter) FormatReport

func (f *JSONFormatter) FormatReport(report *SuiteReport) ([]byte, error)

FormatReport formats a suite report as JSON

func (*JSONFormatter) FormatResult

func (f *JSONFormatter) FormatResult(result *BenchmarkResult) ([]byte, error)

FormatResult formats a single benchmark result as JSON

type MemStats

type MemStats struct {
	Alloc         uint64
	TotalAlloc    uint64
	Sys           uint64
	NumGC         uint32
	HeapAlloc     uint64
	HeapSys       uint64
	HeapInuse     uint64
	StackInuse    uint64
	MSpanInuse    uint64
	MCacheInuse   uint64
	GCCPUFraction float64
}

MemStats holds memory statistics

func MemorySnapshot

func MemorySnapshot() *MemStats

MemorySnapshot captures memory statistics

func (*MemStats) Diff

func (m *MemStats) Diff(other *MemStats) *MemStats

Diff returns the difference between two snapshots

type MemoryBenchmark

type MemoryBenchmark struct {
	// contains filtered or unexported fields
}

MemoryBenchmark measures memory usage patterns

func NewMemoryBenchmark

func NewMemoryBenchmark(name string, allocSize int) *MemoryBenchmark

NewMemoryBenchmark creates a memory benchmark

func (*MemoryBenchmark) Name

func (b *MemoryBenchmark) Name() string

func (*MemoryBenchmark) Run

func (*MemoryBenchmark) Type

func (b *MemoryBenchmark) Type() BenchmarkType

type MetricChange

type MetricChange struct {
	Name      string
	Baseline  float64
	Current   float64
	Change    float64
	ChangePct float64
	Direction string // "improved", "regressed", "unchanged"
}

MetricChange represents a change in a metric

type ParallelBenchmark

type ParallelBenchmark struct {
	Name        string
	Iterations  int
	Parallelism int
	Fn          func(ctx context.Context) error
}

ParallelBenchmark runs a single benchmark with parallel iterations

func (*ParallelBenchmark) Run

Run executes parallel iterations

type ParallelBenchmarkOptions

type ParallelBenchmarkOptions struct {
	MaxWorkers      int
	Ordered         bool
	ContinueOnError bool
	Timeout         time.Duration
}

ParallelBenchmarkOptions configures parallel execution

func DefaultParallelOptions

func DefaultParallelOptions() ParallelBenchmarkOptions

DefaultParallelOptions returns default parallel options

type ParallelBenchmarkResult

type ParallelBenchmarkResult struct {
	BenchmarkResult
	WorkerID  int
	StartTime time.Time
	EndTime   time.Time
}

ParallelBenchmarkResult holds results from parallel execution

type ParallelExecutor

type ParallelExecutor struct {
	// contains filtered or unexported fields
}

ParallelExecutor manages parallel benchmark execution

func NewParallelExecutor

func NewParallelExecutor(options ParallelBenchmarkOptions) *ParallelExecutor

NewParallelExecutor creates a new parallel executor

func (*ParallelExecutor) AddHook

func (pe *ParallelExecutor) AddHook(hook Hook)

AddHook adds a lifecycle hook

func (*ParallelExecutor) Execute

func (pe *ParallelExecutor) Execute(ctx context.Context, benchmarks []Benchmark) ([]BenchmarkResult, error)

Execute runs benchmarks in parallel

type ParallelRunner

type ParallelRunner struct {
	// contains filtered or unexported fields
}

ParallelRunner executes benchmarks in parallel

func NewParallelRunner

func NewParallelRunner(maxWorkers int) *ParallelRunner

NewParallelRunner creates a new parallel benchmark runner

func (*ParallelRunner) AddHook

func (pr *ParallelRunner) AddHook(hook Hook)

AddHook registers a lifecycle hook

func (*ParallelRunner) RegisterSuite

func (pr *ParallelRunner) RegisterSuite(suite *Suite)

RegisterSuite adds a benchmark suite

func (*ParallelRunner) RunSuite

func (pr *ParallelRunner) RunSuite(ctx context.Context, suiteName string) (*SuiteReport, error)

RunSuite executes a benchmark suite in parallel

type ParallelSuite

type ParallelSuite struct {
	*Suite
	Parallelism int
	Ordered     bool
}

ParallelSuite extends Suite with parallel execution options

func NewParallelSuite

func NewParallelSuite(name string, parallelism int) *ParallelSuite

NewParallelSuite creates a parallel benchmark suite

func (*ParallelSuite) WithOrdered

func (ps *ParallelSuite) WithOrdered(ordered bool) *ParallelSuite

WithOrdered sets ordered execution (preserves result order)

type PipelineBenchmark

type PipelineBenchmark struct {
	// contains filtered or unexported fields
}

PipelineBenchmark benchmarks the full pipeline with different configurations

func NewPipelineBenchmark

func NewPipelineBenchmark(name string, mode filter.Mode, input string) *PipelineBenchmark

NewPipelineBenchmark creates a pipeline benchmark

func (*PipelineBenchmark) Name

func (b *PipelineBenchmark) Name() string

func (*PipelineBenchmark) Run

func (*PipelineBenchmark) Type

func (b *PipelineBenchmark) Type() BenchmarkType

type Regression

type Regression struct {
	BenchmarkID string
	Metric      string
	Baseline    float64
	Current     float64
	Change      float64
	ChangePct   float64
	Severity    RegressionSeverity
	Confidence  float64
	DetectedAt  time.Time
	Description string
}

Regression represents a detected regression

type RegressionDetection

type RegressionDetection struct {
	Timestamp   time.Time
	CommitHash  string
	Regressions []Regression
	Score       float64
}

RegressionDetection represents a detection event

type RegressionDetector

type RegressionDetector struct {
	// contains filtered or unexported fields
}

RegressionDetector detects performance regressions

func NewRegressionDetector

func NewRegressionDetector(thresholds RegressionThresholds) *RegressionDetector

NewRegressionDetector creates a new regression detector

func (*RegressionDetector) DetectRegressions

func (rd *RegressionDetector) DetectRegressions(baseline, current []BenchmarkResult) []Regression

DetectRegressions detects regressions between baseline and current results

func (*RegressionDetector) GenerateReport

func (rd *RegressionDetector) GenerateReport(baseline, current []BenchmarkResult) *RegressionReport

GenerateReport generates a regression report

type RegressionHistory

type RegressionHistory struct {
	Detections []RegressionDetection
}

RegressionHistory tracks regression history over time

func (*RegressionHistory) AddDetection

func (rh *RegressionHistory) AddDetection(commitHash string, regressions []Regression)

AddDetection adds a detection to history

func (*RegressionHistory) Trend

func (rh *RegressionHistory) Trend() string

Trend shows regression trend over time

type RegressionReport

type RegressionReport struct {
	GeneratedAt     time.Time
	Regressions     []Regression
	Summary         RegressionSummary
	Recommendations []string
}

RegressionReport contains all detected regressions

type RegressionSeverity

type RegressionSeverity string

RegressionSeverity represents severity level

const (
	SeverityCritical RegressionSeverity = "critical"
	SeverityHigh     RegressionSeverity = "high"
	SeverityMedium   RegressionSeverity = "medium"
	SeverityLow      RegressionSeverity = "low"
)

type RegressionSummary

type RegressionSummary struct {
	TotalRegressions int
	CriticalCount    int
	HighCount        int
	MediumCount      int
	LowCount         int
	ByMetric         map[string]int
}

RegressionSummary provides summary statistics

type RegressionThresholds

type RegressionThresholds struct {
	LatencyRegression float64 // Percentage increase threshold
	ThroughputDrop    float64 // Percentage drop threshold
	MemoryIncrease    float64 // Percentage increase threshold
	ErrorRateIncrease float64 // Absolute increase threshold
	SuccessRateDrop   float64 // Percentage drop threshold
}

RegressionThresholds defines thresholds for regression detection

func DefaultRegressionThresholds

func DefaultRegressionThresholds() RegressionThresholds

DefaultRegressionThresholds returns default thresholds

type Runner

type Runner struct {
	// contains filtered or unexported fields
}

Runner executes benchmarks and collects results

func GetRunner

func GetRunner() *Runner

func NewRunner

func NewRunner() *Runner

NewRunner creates a new benchmark runner

func (*Runner) AddHook

func (r *Runner) AddHook(hook Hook)

AddHook registers a lifecycle hook

func (*Runner) RegisterSuite

func (r *Runner) RegisterSuite(suite *Suite)

RegisterSuite adds a benchmark suite

func (*Runner) RunSuite

func (r *Runner) RunSuite(ctx context.Context, suiteName string) (*SuiteReport, error)

RunSuite executes a benchmark suite

type StatusUpdater

type StatusUpdater interface {
	Update(status CommitStatus, description string) error
}

StatusUpdater updates commit status

type Storage

type Storage struct {
	// contains filtered or unexported fields
}

Storage provides persistent storage for benchmark results

func NewStorage

func NewStorage(dbPath string) (*Storage, error)

NewStorage creates a new benchmark storage

func (*Storage) Cleanup

func (s *Storage) Cleanup(olderThan time.Duration) error

Cleanup removes old data

func (*Storage) Close

func (s *Storage) Close() error

Close closes the storage

func (*Storage) Export

func (s *Storage) Export() (map[string]interface{}, error)

Export exports all data to JSON

func (*Storage) GetStats

func (s *Storage) GetStats() (StorageStats, error)

GetStats returns statistics about stored data

func (*Storage) ListSuites

func (s *Storage) ListSuites(limit int) ([]SuiteInfo, error)

ListSuites lists all benchmark suites

func (*Storage) LoadRegressions

func (s *Storage) LoadRegressions(since time.Time) ([]Regression, error)

LoadRegressions loads recent regressions

func (*Storage) LoadSuite

func (s *Storage) LoadSuite(suiteID int64) (*SuiteReport, error)

LoadSuite loads a benchmark suite by ID

func (*Storage) LoadTrends

func (s *Storage) LoadTrends(benchmarkName string) ([]Trend, error)

LoadTrends loads trends for a benchmark

func (*Storage) SaveRegression

func (s *Storage) SaveRegression(regression *Regression) error

SaveRegression saves a regression detection

func (*Storage) SaveSuite

func (s *Storage) SaveSuite(report *SuiteReport) error

SaveSuite saves a benchmark suite and its results

func (*Storage) SaveTrend

func (s *Storage) SaveTrend(trend *Trend) error

SaveTrend saves a trend analysis

type StorageStats

type StorageStats struct {
	SuiteCount      int
	ResultCount     int
	RegressionCount int
	FirstRun        time.Time
	LastRun         time.Time
}

StorageStats provides storage statistics

type Suite

type Suite struct {
	// contains filtered or unexported fields
}

Suite represents a collection of benchmarks

func NewSuite

func NewSuite(name string) *Suite

NewSuite creates a new benchmark suite

func StandardBenchmarks

func StandardBenchmarks() *Suite

StandardBenchmarks returns a suite of standard benchmarks

func (*Suite) AddBenchmark

func (s *Suite) AddBenchmark(bm Benchmark) *Suite

AddBenchmark adds a benchmark to the suite

func (*Suite) WithDuration

func (s *Suite) WithDuration(d time.Duration) *Suite

WithDuration sets benchmark duration

func (*Suite) WithIterations

func (s *Suite) WithIterations(n int) *Suite

WithIterations sets iterations per benchmark

func (*Suite) WithWarmup

func (s *Suite) WithWarmup(n int) *Suite

WithWarmup sets warmup runs

type SuiteInfo

type SuiteInfo struct {
	ID        int64
	Name      string
	CreatedAt time.Time
}

SuiteInfo provides info about a stored suite

type SuiteReport

type SuiteReport struct {
	Name      string
	StartTime time.Time
	EndTime   time.Time
	Duration  time.Duration
	Results   []BenchmarkResult
}

SuiteReport contains the results of running a suite

func (*SuiteReport) Summary

func (sr *SuiteReport) Summary() *SummaryStats

Summary provides aggregate statistics

type SummaryStats

type SummaryStats struct {
	TotalBenchmarks  int
	FailedBenchmarks int
	TotalTokens      int
	TotalErrors      int
	AvgThroughput    float64
	SuccessRate      float64
	P50Latency       time.Duration
	P95Latency       time.Duration
	P99Latency       time.Duration
	ByType           map[BenchmarkType][]BenchmarkResult
}

SummaryStats provides aggregate statistics across benchmarks

type TableFormatter

type TableFormatter struct{}

TableFormatter formats benchmark results as a table

func NewTableFormatter

func NewTableFormatter() *TableFormatter

NewTableFormatter creates a new table formatter

func (*TableFormatter) FormatReport

func (f *TableFormatter) FormatReport(report *SuiteReport) string

FormatReport formats a suite report as a table

type Trend

type Trend struct {
	BenchmarkID   string
	Metric        string
	Direction     TrendDirection
	Slope         float64
	Intercept     float64
	Correlation   float64
	CurrentValue  float64
	PredictedNext float64
	ChangePct     float64
	History       []HistoricalPoint
}

Trend represents a trend analysis for a benchmark

func DetectImprovements

func DetectImprovements(report *TrendReport, threshold float64) []Trend

DetectImprovements detects performance improvements

func DetectRegressions

func DetectRegressions(report *TrendReport, threshold float64) []Trend

DetectRegressions detects performance regressions

type TrendDirection

type TrendDirection string

TrendDirection represents the direction of a trend

const (
	TrendUp     TrendDirection = "up"
	TrendDown   TrendDirection = "down"
	TrendStable TrendDirection = "stable"
)

type TrendReport

type TrendReport struct {
	GeneratedAt time.Time
	Period      time.Duration
	Trends      []Trend
	Summary     TrendSummary
}

TrendReport contains trend analysis for multiple benchmarks

type TrendStorage

type TrendStorage interface {
	Save(history map[string][]HistoricalResult) error
	Load() (map[string][]HistoricalResult, error)
}

TrendStorage interface for persistence

type TrendSummary

type TrendSummary struct {
	TotalBenchmarks int
	Improving       int
	Degrading       int
	Stable          int
	HighCorrelation int
}

TrendSummary provides aggregate trend information

type TrendTracker

type TrendTracker struct {
	// contains filtered or unexported fields
}

TrendTracker tracks benchmark trends over time

func NewTrendTracker

func NewTrendTracker() *TrendTracker

NewTrendTracker creates a new trend tracker

func (*TrendTracker) AnalyzeTrend

func (tt *TrendTracker) AnalyzeTrend(benchmarkID, metric string) *Trend

AnalyzeTrend performs trend analysis for a benchmark

func (*TrendTracker) GenerateReport

func (tt *TrendTracker) GenerateReport(period time.Duration) *TrendReport

GenerateReport generates a trend report for all tracked benchmarks

func (*TrendTracker) Load

func (tt *TrendTracker) Load(storage TrendStorage) error

Load restores the trend history from storage

func (*TrendTracker) Record

func (tt *TrendTracker) Record(result BenchmarkResult)

Record adds a benchmark result to the history

func (*TrendTracker) Save

func (tt *TrendTracker) Save(storage TrendStorage) error

Save persists the trend history to storage

type ValidatorDefinition

type ValidatorDefinition struct {
	Metric    string
	Operator  string
	Threshold float64
	Message   string
}

ValidatorDefinition represents a validation rule

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL