Documentation
¶
Index ¶
- Constants
- Variables
- func BuildNamedBackoff(strategy string, initialWait, maxWait time.Duration) retryablehttp.Backoff
- func BuildStatusCodeCheckRetry(statusCodes []int) retryablehttp.CheckRetry
- func ValidateURLNotPrivate(rawURL string) error
- type AppConfig
- type CacheStats
- type CacheType
- type CircuitBreakerConfig
- type CircuitBreakerState
- type Client
- func (c *Client) CacheStats() *CacheStats
- func (c *Client) CleanExpiredCache() error
- func (c *Client) ClearCache() error
- func (c *Client) Close() error
- func (c *Client) Delete(ctx context.Context, url string) (*http.Response, error)
- func (c *Client) DeleteCacheEntry(ctx context.Context, url string) error
- func (c *Client) Do(req *http.Request) (*http.Response, error)
- func (c *Client) Get(ctx context.Context, url string) (*http.Response, error)
- func (c *Client) Post(ctx context.Context, url, contentType string, body io.Reader) (*http.Response, error)
- func (c *Client) Put(ctx context.Context, url, contentType string, body io.Reader) (*http.Response, error)
- func (c *Client) RetryableClient() *retryablehttp.Client
- func (c *Client) StandardClient() *http.Client
- func (c *Client) WarmCache(ctx context.Context, urls []string) error
- type ClientConfig
- type FileCache
- func (fc *FileCache) CleanExpired() error
- func (fc *FileCache) Clear() error
- func (fc *FileCache) Close() error
- func (fc *FileCache) Del(ctx context.Context, key string) error
- func (fc *FileCache) Get(ctx context.Context, key string) ([]byte, error)
- func (fc *FileCache) Set(ctx context.Context, key string, data []byte, _ time.Duration) error
- func (fc *FileCache) Stats() (hits, misses uint64)
- type FileCacheConfig
- type Metrics
- type NoopMetrics
- func (NoopMetrics) DecrementConcurrentRequests(context.Context)
- func (NoopMetrics) IncrementCacheHits(context.Context)
- func (NoopMetrics) IncrementCacheMisses(context.Context)
- func (NoopMetrics) IncrementConcurrentRequests(context.Context)
- func (NoopMetrics) IncrementErrorsTotal(context.Context, string, string, string, string)
- func (NoopMetrics) IncrementRequestsTotal(context.Context, string, string, string, int)
- func (NoopMetrics) IncrementRetries(context.Context, string, string, string)
- func (NoopMetrics) RecordRequestDuration(context.Context, string, string, string, int, time.Duration)
- func (NoopMetrics) RecordRequestSize(context.Context, string, string, string, float64)
- func (NoopMetrics) RecordResponseSize(context.Context, string, string, string, float64)
- func (NoopMetrics) SetCacheSizeBytes(int64)
- func (NoopMetrics) SetCircuitBreakerState(string, float64)
- type RequestHook
- type ResponseHook
Constants ¶
const ( // DefaultTimeout is the default HTTP request timeout. DefaultTimeout = 30 * time.Second // DefaultRetryMax is the default maximum number of retries. DefaultRetryMax = 3 // DefaultRetryWait is the default minimum wait time between retries. DefaultRetryWait = 1 * time.Second // DefaultRetryMaxWait is the default maximum wait time between retries. DefaultRetryMaxWait = 30 * time.Second // DefaultCacheTTL is the default time-to-live for cached responses. DefaultCacheTTL = 10 * time.Minute // DefaultCacheKeyPrefix is the default prefix for cache keys. DefaultCacheKeyPrefix = "httpc:" // DefaultMaxResponseBodySize is the default maximum HTTP response body size in bytes (100 MB). DefaultMaxResponseBodySize int64 = 100 * 1024 * 1024 // DefaultMaxCacheFileSize is the default maximum size for a single cached file in bytes (10 MB). DefaultMaxCacheFileSize int64 = 10 * 1024 * 1024 // DefaultMemoryCacheSize is the default maximum number of entries in the memory cache. DefaultMemoryCacheSize = 1000 // DefaultCircuitBreakerMaxFailures is the default number of failures before opening the circuit. DefaultCircuitBreakerMaxFailures = 5 // DefaultCircuitBreakerOpenTimeout is the default wait time before transitioning from Open to HalfOpen. DefaultCircuitBreakerOpenTimeout = 30 * time.Second // DefaultCircuitBreakerHalfOpenRequests is the default number of successful requests in HalfOpen before closing. DefaultCircuitBreakerHalfOpenRequests = 1 // DefaultMaxRedirects is the default maximum number of HTTP redirects to follow. DefaultMaxRedirects = 10 )
Default configuration values for the HTTP client.
const ( // BackoffNone returns the initial wait time for every attempt. BackoffNone = "none" // BackoffLinear increases the wait time linearly with each attempt. BackoffLinear = "linear" // BackoffExponential doubles the wait time with each attempt (default). BackoffExponential = "exponential" )
Backoff strategy names for use with BuildNamedBackoff.
Variables ¶
var ErrCacheSizeLimitExceeded = errors.New("cache: size limit exceeded")
ErrCacheSizeLimitExceeded is returned when attempting to cache data exceeds size limit
var ErrCircuitBreakerOpen = errors.New("circuit breaker is open")
ErrCircuitBreakerOpen is returned when the circuit breaker is open and prevents requests
var ErrDecompressionBombDetected = errors.New("decompressed response exceeds maximum allowed size")
ErrDecompressionBombDetected is returned when a decompressed response exceeds the size limit.
var ErrResponseBodyTooLarge = errors.New("response body exceeds maximum allowed size")
ErrResponseBodyTooLarge is returned when the response body exceeds MaxResponseBodySize
Functions ¶
func BuildNamedBackoff ¶
func BuildNamedBackoff(strategy string, initialWait, maxWait time.Duration) retryablehttp.Backoff
BuildNamedBackoff returns a Backoff function built from a named strategy.
Supported strategy values: "none", "linear", "exponential" (default). The returned durations are clamped to [initialWait, maxWait].
func BuildStatusCodeCheckRetry ¶
func BuildStatusCodeCheckRetry(statusCodes []int) retryablehttp.CheckRetry
BuildStatusCodeCheckRetry returns a CheckRetry function that retries on the given HTTP status codes as well as on any connection/network error. Passing nil or an empty slice retries only on errors.
func ValidateURLNotPrivate ¶
ValidateURLNotPrivate returns an error if rawURL's host is an IP literal that falls within a private, loopback, link-local, or CGNAT range, or if the hostname is a well-known alias for a private/metadata address.
Non-canonical IP forms (decimal, hex, octal) that net.ParseIP does not recognise are also rejected, because some HTTP stacks silently convert them to standard IPs.
Arbitrary hostnames are not pre-resolved because DNS lookups introduce TOCTOU races and are not appropriate for a CLI tool. However, well-known hostnames (localhost, metadata.google.internal) are blocked because they have static, predictable resolutions.
Types ¶
type AppConfig ¶
type AppConfig struct {
// Timeout is the HTTP request timeout as a duration string (e.g. "30s").
Timeout string `json:"timeout,omitempty" yaml:"timeout,omitempty"`
// RetryMax is the maximum number of retries.
RetryMax int `json:"retryMax,omitempty" yaml:"retryMax,omitempty"`
// RetryWaitMin is the minimum wait time between retries as a duration string.
RetryWaitMin string `json:"retryWaitMin,omitempty" yaml:"retryWaitMin,omitempty"`
// RetryWaitMax is the maximum wait time between retries as a duration string.
RetryWaitMax string `json:"retryWaitMax,omitempty" yaml:"retryWaitMax,omitempty"`
// EnableCache enables HTTP response caching.
EnableCache *bool `json:"enableCache,omitempty" yaml:"enableCache,omitempty"`
// CacheType is the cache type: "memory" or "filesystem".
CacheType string `json:"cacheType,omitempty" yaml:"cacheType,omitempty"`
// CacheDir is the directory for filesystem cache.
CacheDir string `json:"cacheDir,omitempty" yaml:"cacheDir,omitempty"`
// CacheTTL is the time-to-live for cached responses as a duration string.
CacheTTL string `json:"cacheTTL,omitempty" yaml:"cacheTTL,omitempty"`
// CacheKeyPrefix is the prefix for cache keys.
CacheKeyPrefix string `json:"cacheKeyPrefix,omitempty" yaml:"cacheKeyPrefix,omitempty"`
// MaxCacheFileSize is the maximum size for a single cached file in bytes.
MaxCacheFileSize int64 `json:"maxCacheFileSize,omitempty" yaml:"maxCacheFileSize,omitempty"`
// MemoryCacheSize is the maximum number of entries in memory cache.
MemoryCacheSize int `json:"memoryCacheSize,omitempty" yaml:"memoryCacheSize,omitempty"`
// EnableCircuitBreaker enables the circuit breaker pattern.
EnableCircuitBreaker *bool `json:"enableCircuitBreaker,omitempty" yaml:"enableCircuitBreaker,omitempty"`
// CircuitBreakerMaxFailures is the number of failures before opening the circuit.
CircuitBreakerMaxFailures int `json:"circuitBreakerMaxFailures,omitempty" yaml:"circuitBreakerMaxFailures,omitempty"`
// CircuitBreakerOpenTimeout is the wait time before half-open state as a duration string.
CircuitBreakerOpenTimeout string `json:"circuitBreakerOpenTimeout,omitempty" yaml:"circuitBreakerOpenTimeout,omitempty"`
// CircuitBreakerHalfOpenMaxRequests is the number of successful requests in half-open before closing.
CircuitBreakerHalfOpenMaxRequests int `json:"circuitBreakerHalfOpenMaxRequests,omitempty" yaml:"circuitBreakerHalfOpenMaxRequests,omitempty"`
// EnableCompression enables automatic gzip compression.
EnableCompression *bool `json:"enableCompression,omitempty" yaml:"enableCompression,omitempty"`
// AllowPrivateIPs allows HTTP requests to private/loopback/link-local IP literals.
AllowPrivateIPs *bool `json:"allowPrivateIPs,omitempty" yaml:"allowPrivateIPs,omitempty"`
// MaxResponseBodySize is the maximum HTTP response body size in bytes.
MaxResponseBodySize int64 `json:"maxResponseBodySize,omitempty" yaml:"maxResponseBodySize,omitempty"`
}
AppConfig holds string-based HTTP client configuration suitable for loading from YAML/JSON config files. Duration fields use string format (e.g. "30s", "5m") for human-readable configuration.
Use NewClientFromAppConfig to create a Client from this configuration.
func MergeAppConfig ¶
MergeAppConfig merges two AppConfig values. Fields set in override take precedence over base. Returns base if override is nil.
type CacheStats ¶
type CacheStats struct {
Hits uint64
Misses uint64
HitRate float64 // Computed as Hits / (Hits + Misses)
}
CacheStats represents cache hit and miss statistics with computed hit rate
type CircuitBreakerConfig ¶
type CircuitBreakerConfig struct {
// MaxFailures is the number of consecutive failures before opening the circuit
MaxFailures int
// OpenTimeout is how long to wait before transitioning from Open to HalfOpen
OpenTimeout time.Duration
// HalfOpenMaxRequests is the number of successful requests in HalfOpen before closing
HalfOpenMaxRequests int
}
CircuitBreakerConfig holds configuration for circuit breaker
func DefaultCircuitBreakerConfig ¶
func DefaultCircuitBreakerConfig() *CircuitBreakerConfig
DefaultCircuitBreakerConfig returns default circuit breaker configuration
type CircuitBreakerState ¶
type CircuitBreakerState int
CircuitBreakerState represents the state of a circuit breaker
const ( // StateClosed allows all requests through StateClosed CircuitBreakerState = iota // StateOpen blocks all requests StateOpen // StateHalfOpen allows a single test request through StateHalfOpen )
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client is an HTTP client with retry, timeout, and caching capabilities.
Thread-Safety: Client is safe for concurrent use by multiple goroutines. All methods can be called concurrently. The underlying retryable HTTP client, cache implementations, and circuit breaker are all thread-safe. Multiple goroutines can share a single Client instance without additional synchronization.
func NewClient ¶
func NewClient(config *ClientConfig) *Client
NewClient creates a new HTTP client with the provided configuration
func NewClientFromAppConfig ¶
NewClientFromAppConfig creates a new HTTP client using string-based application configuration. The cfg parameter can be nil, in which case defaults are used. Returns an error if any duration string or config value is invalid.
func (*Client) CacheStats ¶
func (c *Client) CacheStats() *CacheStats
CacheStats returns cache hit and miss statistics with computed hit rate Returns nil if cache stats are not available
func (*Client) CleanExpiredCache ¶
CleanExpiredCache removes expired cache entries Only supported for filesystem cache
func (*Client) ClearCache ¶
ClearCache clears all cached entries For filesystem cache, this also removes files from disk
func (*Client) Close ¶
Close gracefully shuts down the client and cleans up resources For filesystem cache, this performs a cleanup of expired entries
func (*Client) DeleteCacheEntry ¶
DeleteCacheEntry removes a specific entry from the cache by URL
func (*Client) Do ¶
Do executes an HTTP request with retry logic, hooks, and circuit breaker support
func (*Client) Post ¶
func (c *Client) Post(ctx context.Context, url, contentType string, body io.Reader) (*http.Response, error)
Post performs a POST request
func (*Client) Put ¶
func (c *Client) Put(ctx context.Context, url, contentType string, body io.Reader) (*http.Response, error)
Put performs a PUT request
func (*Client) RetryableClient ¶
func (c *Client) RetryableClient() *retryablehttp.Client
RetryableClient returns the underlying retryable HTTP client
func (*Client) StandardClient ¶
StandardClient returns the underlying standard HTTP client (useful for external libraries)
type ClientConfig ¶
type ClientConfig struct {
// Timeout is the maximum time to wait for a request to complete
Timeout time.Duration
// RetryMax is the maximum number of retries
RetryMax int
// RetryWaitMin is the minimum time to wait between retries
RetryWaitMin time.Duration
// RetryWaitMax is the maximum time to wait between retries
RetryWaitMax time.Duration
// EnableCache enables HTTP caching
EnableCache bool
// CacheType specifies the type of cache to use (memory or filesystem)
CacheType CacheType
// CacheDir is the directory to use for filesystem cache (only used when CacheType is filesystem)
CacheDir string
// CacheTTL is the time-to-live for cached responses
CacheTTL time.Duration
// CacheKeyPrefix is a prefix added to all cache keys to prevent collisions
CacheKeyPrefix string
// MaxCacheFileSize is the maximum size in bytes for a single cached file (0 = no limit)
// Only applies to filesystem cache
MaxCacheFileSize int64
// MemoryCacheSize is the maximum number of entries in the memory cache (default: 1000)
MemoryCacheSize int
// Logger is the logger to use for the client
Logger logr.Logger
// CheckRetry is a custom retry policy function
CheckRetry retryablehttp.CheckRetry
// Backoff is a custom backoff policy function
Backoff retryablehttp.Backoff
// ErrorHandler is called if retries are exhausted
ErrorHandler retryablehttp.ErrorHandler
// RequestHooks are functions called before each request
RequestHooks []RequestHook
// ResponseHooks are functions called after each response
ResponseHooks []ResponseHook
// Return the new full Authorization header value (e.g. "Bearer <new-token>") to
// inject a single transparent retry with the refreshed token.
// Return an empty string (or an error) to pass the 401 response through as-is.
OnUnauthorized func(ctx context.Context) (authorizationHeader string, err error)
// EnableCircuitBreaker enables circuit breaker pattern
EnableCircuitBreaker bool
// CircuitBreakerConfig holds circuit breaker configuration
CircuitBreakerConfig *CircuitBreakerConfig
// EnableCompression enables automatic gzip compression for requests/responses
EnableCompression bool
// Metrics is the metrics collector for the HTTP client.
// Defaults to NoopMetrics{} if not set.
Metrics Metrics
// AllowPrivateIPs allows HTTP requests to private/loopback/link-local IP literals.
// Defaults to false (deny), enforcing secure-by-default behaviour.
AllowPrivateIPs bool
// MaxRedirects is the maximum number of HTTP redirects to follow.
// Defaults to DefaultMaxRedirects (10).
MaxRedirects int
// MaxResponseBodySize is the maximum HTTP response body size in bytes.
// Defaults to DefaultMaxResponseBodySize (100 MB).
MaxResponseBodySize int64
}
ClientConfig holds the configuration for the HTTP client
func DefaultConfig ¶
func DefaultConfig() *ClientConfig
DefaultConfig returns a ClientConfig with sensible defaults
type FileCache ¶
type FileCache struct {
// contains filtered or unexported fields
}
FileCache is a filesystem-based cache implementation.
Thread-Safety: FileCache is safe for concurrent use by multiple goroutines. Get, Set, and Del operations can be called concurrently. Hit/miss statistics are tracked using atomic operations. File operations are performed atomically where possible (e.g., write-then-rename for Set). However, due to filesystem limitations, there may be race conditions if multiple processes (not goroutines) access the same cache directory simultaneously.
func NewFileCache ¶
func NewFileCache(config *FileCacheConfig) (*FileCache, error)
NewFileCache creates a new filesystem cache in the specified directory
func (*FileCache) CleanExpired ¶
CleanExpired removes expired cache entries
func (*FileCache) Close ¶
Close performs cleanup and releases resources This method cleans expired entries as a final housekeeping step
func (*FileCache) Get ¶
Get retrieves data from the cache for the given key Returns (nil, nil) for cache misses - this is not an error, it's standard cache behavior
type FileCacheConfig ¶
type FileCacheConfig struct {
// Dir is the directory to use for cache storage
Dir string
// TTL is the time-to-live for cached entries
TTL time.Duration
// KeyPrefix is a prefix added to all cache keys to prevent collisions
KeyPrefix string
// MaxSize is the maximum size in bytes for a single cached file (0 = no limit)
MaxSize int64
// FileIOTimeout is the best-effort maximum time for file I/O operations (default: 5s).
// Context is checked before and after I/O syscalls, but cannot interrupt them mid-call.
FileIOTimeout time.Duration
// Logger is used for logging cache operations
Logger logr.Logger
// Metrics is the metrics collector for cache operations
Metrics Metrics
}
FileCacheConfig holds configuration for filesystem cache
type Metrics ¶
type Metrics interface {
// RecordRequestDuration records the duration of an HTTP request.
RecordRequestDuration(ctx context.Context, method, host, pathTemplate string, statusCode int, duration time.Duration)
// IncrementRequestsTotal increments the total request counter.
IncrementRequestsTotal(ctx context.Context, method, host, pathTemplate string, statusCode int)
// IncrementErrorsTotal increments the error counter with the given error type.
IncrementErrorsTotal(ctx context.Context, method, host, pathTemplate, errorType string)
// IncrementRetries increments the retry counter.
IncrementRetries(ctx context.Context, method, host, pathTemplate string)
// IncrementCacheHits increments the cache hit counter.
IncrementCacheHits(ctx context.Context)
// IncrementCacheMisses increments the cache miss counter.
IncrementCacheMisses(ctx context.Context)
// SetCacheSizeBytes reports the total cache size in bytes.
SetCacheSizeBytes(bytes int64)
// SetCircuitBreakerState reports the circuit breaker state for a host.
// State values: 0=closed, 1=open, 2=half-open.
SetCircuitBreakerState(host string, state float64)
// IncrementConcurrentRequests increments the in-flight request counter.
IncrementConcurrentRequests(ctx context.Context)
// DecrementConcurrentRequests decrements the in-flight request counter.
DecrementConcurrentRequests(ctx context.Context)
// RecordRequestSize records the size of an HTTP request body in bytes.
RecordRequestSize(ctx context.Context, method, host, pathTemplate string, bytes float64)
// RecordResponseSize records the size of an HTTP response body in bytes.
RecordResponseSize(ctx context.Context, method, host, pathTemplate string, bytes float64)
}
Metrics defines the interface for collecting HTTP client metrics. Implement this interface to integrate with your preferred metrics backend (Prometheus, OpenTelemetry, StatsD, etc.).
All methods must be safe for concurrent use by multiple goroutines.
type NoopMetrics ¶
type NoopMetrics struct{}
NoopMetrics is a no-op implementation of Metrics that discards all data. This is the default when no metrics backend is configured.
func (NoopMetrics) DecrementConcurrentRequests ¶
func (NoopMetrics) DecrementConcurrentRequests(context.Context)
func (NoopMetrics) IncrementCacheHits ¶
func (NoopMetrics) IncrementCacheHits(context.Context)
func (NoopMetrics) IncrementCacheMisses ¶
func (NoopMetrics) IncrementCacheMisses(context.Context)
func (NoopMetrics) IncrementConcurrentRequests ¶
func (NoopMetrics) IncrementConcurrentRequests(context.Context)
func (NoopMetrics) IncrementErrorsTotal ¶
func (NoopMetrics) IncrementRequestsTotal ¶
func (NoopMetrics) IncrementRetries ¶
func (NoopMetrics) RecordRequestDuration ¶
func (NoopMetrics) RecordRequestSize ¶
func (NoopMetrics) RecordResponseSize ¶
func (NoopMetrics) SetCacheSizeBytes ¶
func (NoopMetrics) SetCacheSizeBytes(int64)
func (NoopMetrics) SetCircuitBreakerState ¶
func (NoopMetrics) SetCircuitBreakerState(string, float64)
type RequestHook ¶
RequestHook is a function that processes a request before it's sent
type ResponseHook ¶
ResponseHook is a function that processes a response after it's received