loadtest

package
v1.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 1, 2026 License: Apache-2.0 Imports: 8 Imported by: 0

Documentation

Overview

Package loadtest provides a configurable mock LLM provider for load testing.

The provider simulates realistic LLM behavior — configurable latency, error injection, token streaming, and tool call responses — without consuming any real LLM tokens. This allows the entire loom stack (gRPC server, agent orchestration, session management, observability) to be hammered under concurrent load at zero cost.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Metrics

type Metrics struct {
	TotalCalls     atomic.Int64
	SuccessCount   atomic.Int64
	ErrorCount     atomic.Int64
	TotalLatencyNs atomic.Int64 // sum of all call latencies in nanoseconds
	MinLatencyNs   atomic.Int64
	MaxLatencyNs   atomic.Int64
}

Metrics tracks provider usage during a load test.

func (*Metrics) Snapshot

func (m *Metrics) Snapshot() MetricsSnapshot

Snapshot returns a point-in-time snapshot of the metrics.

type MetricsSnapshot

type MetricsSnapshot struct {
	TotalCalls   int64
	SuccessCount int64
	ErrorCount   int64
	AvgLatency   time.Duration
	MinLatency   time.Duration
	MaxLatency   time.Duration
}

Snapshot returns a copy of the current metrics as a plain struct.

type Provider

type Provider struct {
	// contains filtered or unexported fields
}

Provider is a mock LLM provider for load testing. It implements types.LLMProvider and types.StreamingLLMProvider. All methods are safe for concurrent use.

func NewProvider

func NewProvider(config ProviderConfig) *Provider

NewProvider creates a new load test provider.

func (*Provider) Chat

func (p *Provider) Chat(ctx context.Context, messages []types.Message, tools []shuttle.Tool) (*types.LLMResponse, error)

Chat implements types.LLMProvider. It simulates an LLM call with configurable latency and error injection.

func (*Provider) ChatStream

func (p *Provider) ChatStream(ctx context.Context, messages []types.Message, tools []shuttle.Tool, callback types.TokenCallback) (*types.LLMResponse, error)

ChatStream implements types.StreamingLLMProvider. It streams the response content in chunks with configurable delay between chunks.

func (*Provider) GetMetrics

func (p *Provider) GetMetrics() *Metrics

GetMetrics returns the provider's usage metrics.

func (*Provider) Model

func (p *Provider) Model() string

Model returns the model identifier.

func (*Provider) Name

func (p *Provider) Name() string

Name returns the provider name.

type ProviderConfig

type ProviderConfig struct {
	// Name is the provider name returned by Name().
	Name string

	// Model is the model identifier returned by Model().
	Model string

	// BaseLatency is the minimum latency per Chat call.
	// Simulates LLM time-to-first-token.
	BaseLatency time.Duration

	// LatencyJitter adds random jitter up to this duration on top of BaseLatency.
	// Total latency per call = BaseLatency + rand(0, LatencyJitter).
	LatencyJitter time.Duration

	// ErrorRate is the probability [0.0, 1.0] that a Chat call returns an error.
	// Simulates 429s, timeouts, and transient failures.
	ErrorRate float64

	// ErrorMessage is the error returned when error injection fires.
	// Defaults to "loadtest: simulated provider error" if empty.
	ErrorMessage string

	// Response is the canned response returned on success.
	// If nil, a default response is generated.
	Response *types.LLMResponse

	// ResponseFunc generates a dynamic response based on the input messages.
	// Takes precedence over Response if set.
	ResponseFunc func(messages []types.Message, tools []shuttle.Tool) *types.LLMResponse

	// StreamChunkSize is the number of characters per streaming chunk.
	// Only used when the provider is accessed via ChatStream.
	// Defaults to 10.
	StreamChunkSize int

	// StreamChunkDelay is the delay between streaming chunks.
	// Defaults to 5ms.
	StreamChunkDelay time.Duration

	// InputTokensPerMessage is the simulated input token count per message.
	// Defaults to 50.
	InputTokensPerMessage int

	// OutputTokens is the simulated output token count.
	// Defaults to 100.
	OutputTokens int

	// CostPerCall is the simulated cost in USD per call.
	// Defaults to 0.001.
	CostPerCall float64
}

ProviderConfig controls the mock provider's behavior.

func DefaultConfig

func DefaultConfig() ProviderConfig

DefaultConfig returns a ProviderConfig with realistic defaults.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL