Documentation
¶
Overview ¶
Package benchmark provides a load-testing client and an in-process server factory for the virtualmodel HTTP service. The server side is a thin factory over virtualmodel/virtualserver.Service so benchmarks reuse the same vmodel registries as production code; the client side is a pooled HTTP load generator that collects throughput / latency metrics.
This package replaces the former pkg/benchmark, whose mock-server half duplicated virtualserver and whose Model type duplicated virtualmodel.Model.
Index ¶
- Constants
- type AnthropicMessageRequest
- type BenchmarkClient
- func (bc *BenchmarkClient) TestChatEndpoint(model string, messages []map[string]interface{}, concurrency int, ...) (*BenchmarkResult, error)
- func (bc *BenchmarkClient) TestMessagesEndpoint(model string, messages []map[string]interface{}, maxTokens int, ...) (*BenchmarkResult, error)
- func (bc *BenchmarkClient) TestModelsEndpoint(concurrency int, totalRequests int) (*BenchmarkResult, error)
- type BenchmarkOptions
- type BenchmarkResult
- type LocalServer
- type OpenAIChatRequest
- type RequestResult
Constants ¶
const DefaultPort = 12580
DefaultPort is the conventional benchmark server port. Callers may use LocalServer.Port() to discover an ephemeral port instead.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AnthropicMessageRequest ¶
type AnthropicMessageRequest struct {
Model string `json:"model"`
MaxTokens int `json:"max_tokens"`
Messages []map[string]interface{} `json:"messages"`
Stream bool `json:"stream,omitempty"`
}
AnthropicMessageRequest is a minimal request body for /v1/messages used by the load tester.
type BenchmarkClient ¶
type BenchmarkClient struct {
// contains filtered or unexported fields
}
func NewBenchmarkClient ¶
func NewBenchmarkClient(opts *BenchmarkOptions) *BenchmarkClient
func (*BenchmarkClient) TestChatEndpoint ¶
func (bc *BenchmarkClient) TestChatEndpoint(model string, messages []map[string]interface{}, concurrency int, totalRequests int) (*BenchmarkResult, error)
func (*BenchmarkClient) TestMessagesEndpoint ¶
func (bc *BenchmarkClient) TestMessagesEndpoint(model string, messages []map[string]interface{}, maxTokens int, concurrency int, totalRequests int) (*BenchmarkResult, error)
func (*BenchmarkClient) TestModelsEndpoint ¶
func (bc *BenchmarkClient) TestModelsEndpoint(concurrency int, totalRequests int) (*BenchmarkResult, error)
type BenchmarkOptions ¶
type BenchmarkResult ¶
type BenchmarkResult struct {
TotalRequests int `json:"totalRequests"`
SuccessRequests int `json:"successRequests"`
FailedRequests int `json:"failedRequests"`
TotalDuration time.Duration `json:"totalDuration"`
AvgResponseTime time.Duration `json:"avgResponseTime"`
MinResponseTime time.Duration `json:"minResponseTime"`
MaxResponseTime time.Duration `json:"maxResponseTime"`
RequestsPerSec float64 `json:"requestsPerSec"`
TotalBytes int64 `json:"totalBytes"`
ErrorRate float64 `json:"errorRate"`
StatusCodeCounts map[int]int `json:"statusCodeCounts"`
}
func (*BenchmarkResult) PrintSummary ¶
func (br *BenchmarkResult) PrintSummary()
type LocalServer ¶
type LocalServer struct {
// contains filtered or unexported fields
}
LocalServer wraps a virtualserver.Service exposed over an HTTP listener so the benchmark client can hit a real loopback target. It mounts the vmodel routes at the OpenAI-conventional /v1/ prefix as well as /openai/v1/ and /anthropic/v1/ for benchmark clients that exercise both protocols against a single process.
func NewLocalServer ¶
func NewLocalServer(addr string) (*LocalServer, error)
NewLocalServer starts an in-process benchmark server bound to addr (an empty string or ":0" picks an ephemeral port). The returned server is already listening; call Port() to discover the bound port and Close() to shut down. The underlying virtualmodel registries come pre-populated with the same defaults as production via virtualserver.NewService.
func (*LocalServer) BaseURL ¶
func (s *LocalServer) BaseURL() string
BaseURL returns http://localhost:<port> for use as BenchmarkOptions.BaseURL.
func (*LocalServer) Close ¶
func (s *LocalServer) Close() error
Close shuts down the server with a short grace period.
func (*LocalServer) Port ¶
func (s *LocalServer) Port() int
Port returns the TCP port the server is listening on.
func (*LocalServer) Service ¶
func (s *LocalServer) Service() *virtualserver.Service
Service returns the underlying virtualserver.Service so callers can register additional virtual models on its anthropic / openai registries.
type OpenAIChatRequest ¶
type OpenAIChatRequest struct {
Model string `json:"model"`
Messages []map[string]interface{} `json:"messages"`
Stream bool `json:"stream,omitempty"`
}
OpenAIChatRequest is a minimal request body for /v1/chat/completions used by the load tester. It is not a full SDK type — only the fields the benchmark client needs to construct a request.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
examples
|
|
|
client
command
Stand-alone benchmark client driver: starts an in-process LocalServer (vmodel-backed) and drives it with the BenchmarkClient against both the OpenAI Chat and Anthropic Messages routes, printing a metrics summary.
|
Stand-alone benchmark client driver: starts an in-process LocalServer (vmodel-backed) and drives it with the BenchmarkClient against both the OpenAI Chat and Anthropic Messages routes, printing a metrics summary. |
|
server
command
Stand-alone benchmark mock server: starts a local HTTP server backed by the production virtualmodel registries (with their default mock models pre-registered) so external benchmark drivers can hit a realistic vmodel surface over loopback.
|
Stand-alone benchmark mock server: starts a local HTTP server backed by the production virtualmodel registries (with their default mock models pre-registered) so external benchmark drivers can hit a realistic vmodel surface over loopback. |