Documentation
¶
Overview ¶
Package httpclient provides optimized HTTP clients for LLM providers with built-in monitoring and connection pooling.
This package is designed specifically for high-throughput LLM gateway scenarios where multiple providers need to be accessed concurrently with different performance characteristics.
Features:
- Optimized connection pooling per provider
- HTTP/2 support with multiplexing
- Comprehensive metrics collection
- Middleware support for rate limiting, request IDs, and timeouts
- Provider-specific configuration defaults
Basic Usage:
// Create a client with default configuration
client, err := httpclient.New("openai", httpclient.DefaultConfig())
if err != nil {
log.Fatal(err)
}
defer client.Close()
// Use the client
resp, err := client.Do(req)
With Monitoring:
// Implement MetricsCollector interface
metrics := &MyMetricsCollector{}
config := httpclient.DefaultConfig()
config.MetricsCollector = metrics
client, err := httpclient.New("anthropic", config)
With Middleware:
config := httpclient.DefaultConfig()
config.TransportWrapper = httpclient.ChainTransportWrappers(
httpclient.WithRateLimiting(100),
httpclient.WithRequestID(nil),
)
client, err := httpclient.New("google", config)
The package is designed to be used as a foundation for LLM provider connectors, handling all the complexity of HTTP communication while allowing the connectors to focus on provider-specific logic.
Index ¶
- type Client
- type Config
- type ConnectionStats
- type MetricsCollector
- type MonitoredTransport
- type NoOpMetricsCollector
- func (n *NoOpMetricsCollector) RecordConnectionClosed(_ string)
- func (n *NoOpMetricsCollector) RecordConnectionCreated(_ string)
- func (n *NoOpMetricsCollector) RecordConnectionReused(_ string)
- func (n *NoOpMetricsCollector) RecordPoolStats(_ string, _ ConnectionStats)
- func (n *NoOpMetricsCollector) RecordRequestComplete(_, _, _ string, _ int, _ time.Duration)
- func (n *NoOpMetricsCollector) RecordRequestError(_, _, _ string, _ error)
- func (n *NoOpMetricsCollector) RecordRequestStart(_, _, _ string)
- type RoundTripper
- type TransportWrapper
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client represents an HTTP client optimized for a specific provider
func (*Client) GetHTTPClient ¶
GetHTTPClient returns the underlying HTTP client
func (*Client) Stats ¶
func (c *Client) Stats() ConnectionStats
Stats returns current connection pool statistics
func (*Client) UpdateConfig ¶
UpdateConfig updates the client configuration Note: This creates a new transport, so use sparingly
type Config ¶
type Config struct {
// Connection pool settings
MaxIdleConns int // Maximum idle connections across all hosts
MaxIdleConnsPerHost int // Maximum idle connections per host
MaxConnsPerHost int // Maximum total connections per host
IdleConnTimeout time.Duration // How long idle connections are kept alive
// Timeout settings
DialTimeout time.Duration // Timeout for establishing connection
TLSHandshakeTimeout time.Duration // Timeout for TLS handshake
ResponseHeaderTimeout time.Duration // Timeout for receiving response headers
ExpectContinueTimeout time.Duration // Timeout for 100-continue response
RequestTimeout time.Duration // Overall timeout for entire request
// Feature flags
EnableHTTP2 bool // Enable HTTP/2 support
EnableCompression bool // Enable transparent compression
EnableKeepAlives bool // Enable connection keep-alive
// Monitoring and observability
MetricsCollector MetricsCollector // Optional metrics collector
// Advanced options
TransportWrapper TransportWrapper // Optional transport wrapper for middleware
}
Config defines the configuration options for creating an HTTP client
func DefaultConfig ¶
func DefaultConfig() Config
DefaultConfig returns a configuration optimized for LLM gateway usage
type ConnectionStats ¶
type ConnectionStats struct {
Provider string
IdleConnections int
ActiveConnections int
TotalConnections int
WaitingRequests int
}
ConnectionStats represents current connection pool statistics
type MetricsCollector ¶
type MetricsCollector interface {
// Connection pool metrics
RecordConnectionCreated(provider string)
RecordConnectionClosed(provider string)
RecordConnectionReused(provider string)
RecordPoolStats(provider string, stats ConnectionStats)
// Request lifecycle metrics
RecordRequestStart(provider string, method string, path string)
RecordRequestComplete(provider string, method string, path string, statusCode int, duration time.Duration)
RecordRequestError(provider string, method string, path string, err error)
}
MetricsCollector defines the interface for collecting HTTP client metrics
type MonitoredTransport ¶
type MonitoredTransport struct {
// contains filtered or unexported fields
}
MonitoredTransport wraps an http.RoundTripper to add request monitoring.
func (*MonitoredTransport) ActiveRequests ¶
func (t *MonitoredTransport) ActiveRequests() int64
ActiveRequests returns the current number of active requests
func (*MonitoredTransport) CloseIdleConnections ¶
func (t *MonitoredTransport) CloseIdleConnections()
CloseIdleConnections closes idle connections in the underlying transport
func (*MonitoredTransport) Unwrap ¶
func (t *MonitoredTransport) Unwrap() http.RoundTripper
Unwrap returns the underlying transport (useful for testing and debugging)
type NoOpMetricsCollector ¶
type NoOpMetricsCollector struct{}
NoOpMetricsCollector is a no-op implementation of MetricsCollector
func (*NoOpMetricsCollector) RecordConnectionClosed ¶
func (n *NoOpMetricsCollector) RecordConnectionClosed(_ string)
RecordConnectionClosed does nothing in NoOpMetricsCollector
func (*NoOpMetricsCollector) RecordConnectionCreated ¶
func (n *NoOpMetricsCollector) RecordConnectionCreated(_ string)
RecordConnectionCreated does nothing in NoOpMetricsCollector
func (*NoOpMetricsCollector) RecordConnectionReused ¶
func (n *NoOpMetricsCollector) RecordConnectionReused(_ string)
RecordConnectionReused does nothing in NoOpMetricsCollector
func (*NoOpMetricsCollector) RecordPoolStats ¶
func (n *NoOpMetricsCollector) RecordPoolStats(_ string, _ ConnectionStats)
RecordPoolStats does nothing in NoOpMetricsCollector
func (*NoOpMetricsCollector) RecordRequestComplete ¶
func (n *NoOpMetricsCollector) RecordRequestComplete(_, _, _ string, _ int, _ time.Duration)
RecordRequestComplete does nothing in NoOpMetricsCollector
func (*NoOpMetricsCollector) RecordRequestError ¶
func (n *NoOpMetricsCollector) RecordRequestError(_, _, _ string, _ error)
RecordRequestError does nothing in NoOpMetricsCollector
func (*NoOpMetricsCollector) RecordRequestStart ¶
func (n *NoOpMetricsCollector) RecordRequestStart(_, _, _ string)
RecordRequestStart does nothing in NoOpMetricsCollector
type RoundTripper ¶
RoundTripper is an interface for HTTP round trippers (copied to avoid circular import)
type TransportWrapper ¶
type TransportWrapper func(transport RoundTripper) RoundTripper
TransportWrapper allows wrapping the transport with middleware
func ChainTransportWrappers ¶
func ChainTransportWrappers(wrappers ...TransportWrapper) TransportWrapper
ChainTransportWrappers chains multiple transport wrappers
func WithRateLimiting ¶
func WithRateLimiting(rps int) TransportWrapper
WithRateLimiting adds rate limiting to the transport
func WithRequestID ¶
func WithRequestID(generator func() string) TransportWrapper
WithRequestID adds a unique request ID to each request
func WithTimeout ¶
func WithTimeout(timeout time.Duration) TransportWrapper
WithTimeout adds a per-request timeout
Source Files