Documentation
¶
Overview ¶
Package loadtest provides a configurable mock LLM provider for load testing.
The provider simulates realistic LLM behavior — configurable latency, error injection, token streaming, and tool call responses — without consuming any real LLM tokens. This allows the entire loom stack (gRPC server, agent orchestration, session management, observability) to be hammered under concurrent load at zero cost.
Index ¶
- type Metrics
- type MetricsSnapshot
- type Provider
- func (p *Provider) Chat(ctx context.Context, messages []types.Message, tools []shuttle.Tool) (*types.LLMResponse, error)
- func (p *Provider) ChatStream(ctx context.Context, messages []types.Message, tools []shuttle.Tool, ...) (*types.LLMResponse, error)
- func (p *Provider) GetMetrics() *Metrics
- func (p *Provider) Model() string
- func (p *Provider) Name() string
- type ProviderConfig
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Metrics ¶
type Metrics struct {
TotalCalls atomic.Int64
SuccessCount atomic.Int64
ErrorCount atomic.Int64
TotalLatencyNs atomic.Int64 // sum of all call latencies in nanoseconds
MinLatencyNs atomic.Int64
MaxLatencyNs atomic.Int64
}
Metrics tracks provider usage during a load test.
func (*Metrics) Snapshot ¶
func (m *Metrics) Snapshot() MetricsSnapshot
Snapshot returns a point-in-time snapshot of the metrics.
type MetricsSnapshot ¶
type MetricsSnapshot struct {
TotalCalls int64
SuccessCount int64
ErrorCount int64
AvgLatency time.Duration
MinLatency time.Duration
MaxLatency time.Duration
}
Snapshot returns a copy of the current metrics as a plain struct.
type Provider ¶
type Provider struct {
// contains filtered or unexported fields
}
Provider is a mock LLM provider for load testing. It implements types.LLMProvider and types.StreamingLLMProvider. All methods are safe for concurrent use.
func NewProvider ¶
func NewProvider(config ProviderConfig) *Provider
NewProvider creates a new load test provider.
func (*Provider) Chat ¶
func (p *Provider) Chat(ctx context.Context, messages []types.Message, tools []shuttle.Tool) (*types.LLMResponse, error)
Chat implements types.LLMProvider. It simulates an LLM call with configurable latency and error injection.
func (*Provider) ChatStream ¶
func (p *Provider) ChatStream(ctx context.Context, messages []types.Message, tools []shuttle.Tool, callback types.TokenCallback) (*types.LLMResponse, error)
ChatStream implements types.StreamingLLMProvider. It streams the response content in chunks with configurable delay between chunks.
func (*Provider) GetMetrics ¶
GetMetrics returns the provider's usage metrics.
type ProviderConfig ¶
type ProviderConfig struct {
// Name is the provider name returned by Name().
Name string
// Model is the model identifier returned by Model().
Model string
// BaseLatency is the minimum latency per Chat call.
// Simulates LLM time-to-first-token.
BaseLatency time.Duration
// LatencyJitter adds random jitter up to this duration on top of BaseLatency.
// Total latency per call = BaseLatency + rand(0, LatencyJitter).
LatencyJitter time.Duration
// ErrorRate is the probability [0.0, 1.0] that a Chat call returns an error.
// Simulates 429s, timeouts, and transient failures.
ErrorRate float64
// ErrorMessage is the error returned when error injection fires.
// Defaults to "loadtest: simulated provider error" if empty.
ErrorMessage string
// Response is the canned response returned on success.
// If nil, a default response is generated.
Response *types.LLMResponse
// ResponseFunc generates a dynamic response based on the input messages.
// Takes precedence over Response if set.
ResponseFunc func(messages []types.Message, tools []shuttle.Tool) *types.LLMResponse
// StreamChunkSize is the number of characters per streaming chunk.
// Only used when the provider is accessed via ChatStream.
// Defaults to 10.
StreamChunkSize int
// StreamChunkDelay is the delay between streaming chunks.
// Defaults to 5ms.
StreamChunkDelay time.Duration
// InputTokensPerMessage is the simulated input token count per message.
// Defaults to 50.
InputTokensPerMessage int
// OutputTokens is the simulated output token count.
// Defaults to 100.
OutputTokens int
// CostPerCall is the simulated cost in USD per call.
// Defaults to 0.001.
CostPerCall float64
}
ProviderConfig controls the mock provider's behavior.
func DefaultConfig ¶
func DefaultConfig() ProviderConfig
DefaultConfig returns a ProviderConfig with realistic defaults.