Documentation
¶
Overview ¶
Package httpclient contains helpers for doing HTTP requests (for example, round trippers for retrying and rate limiting).
Index ¶
- Constants
- Variables
- func CheckErrorIsTemporary(err error) bool
- func DefaultCheckRetry(ctx context.Context, resp *http.Response, roundTripErr error, ...) (needRetry bool, err error)
- func GetIdempotentHintFromContext(ctx context.Context) bool
- func GetRequestTypeFromContext(ctx context.Context) string
- func MustNew(cfg *Config) *http.Client
- func MustNewWithOpts(cfg *Config, opts Opts) *http.Client
- func New(cfg *Config) (*http.Client, error)
- func NewContextWithIdempotentHint(ctx context.Context, isIdempotent bool) context.Context
- func NewContextWithRequestType(ctx context.Context, requestType string) context.Context
- func NewLoggingRoundTripper(delegate http.RoundTripper) http.RoundTripper
- func NewLoggingRoundTripperWithOpts(delegate http.RoundTripper, opts LoggingRoundTripperOpts) http.RoundTripper
- func NewMetricsRoundTripper(delegate http.RoundTripper, collector MetricsCollector) http.RoundTripper
- func NewMetricsRoundTripperWithOpts(delegate http.RoundTripper, collector MetricsCollector, ...) http.RoundTripper
- func NewRequestIDRoundTripper(delegate http.RoundTripper) http.RoundTripper
- func NewRequestIDRoundTripperWithOpts(delegate http.RoundTripper, opts RequestIDRoundTripperOpts) http.RoundTripper
- func NewWithOpts(cfg *Config, opts Opts) (*http.Client, error)
- type AuthBearerRoundTripper
- type AuthBearerRoundTripperError
- type AuthBearerRoundTripperOpts
- type AuthProvider
- type AuthProviderFunc
- type AuthProviderInvalidator
- type CheckRetryFunc
- type Config
- type ConfigOption
- type ConstantBackoffConfig
- type ExponentialBackoffConfig
- type LogConfig
- type LoggingMode
- type LoggingRoundTripper
- type LoggingRoundTripperOpts
- type MetricsCollector
- type MetricsConfig
- type MetricsRoundTripper
- type MetricsRoundTripperOpts
- type Opts
- type PrometheusMetricsCollector
- type RateLimitingRoundTripper
- type RateLimitingRoundTripperAdaptation
- type RateLimitingRoundTripperOpts
- type RateLimitingWaitError
- type RateLimitsConfig
- type RequestIDRoundTripper
- type RequestIDRoundTripperOpts
- type RetriesConfig
- type RetryPolicy
- type RetryableRoundTripper
- type RetryableRoundTripperError
- type RetryableRoundTripperOpts
- type UserAgentRoundTripper
- type UserAgentRoundTripperOpts
- type UserAgentUpdateStrategy
Examples ¶
Constants ¶
const ( DefaultRateLimitingBurst = 1 DefaultRateLimitingWaitTimeout = 15 * time.Second )
Default parameter values for RateLimitingRoundTripper.
const ( DefaultMaxRetryAttempts = 10 DefaultExponentialBackoffInitialInterval = time.Second DefaultExponentialBackoffMultiplier = 2 )
Default parameter values for RetryableRoundTripper.
const DefaultAuthProviderMinInvalidationInterval = 15 * time.Minute
DefaultAuthProviderMinInvalidationInterval is the default minimum time between cache invalidation attempts for the same AuthProvider.
const RetryAttemptNumberHeader = "X-Retry-Attempt"
RetryAttemptNumberHeader is an HTTP header name that will contain the serial number of the retry attempt.
const UnlimitedRetryAttempts = -1
UnlimitedRetryAttempts should be used as RetryableRoundTripperOpts.MaxRetryAttempts value when we want to stop retries only by RetryableRoundTripperOpts.BackoffPolicy.
Variables ¶
var DefaultBackoffPolicy = retry.PolicyFunc(func() backoff.BackOff { bf := backoff.NewExponentialBackOff() bf.InitialInterval = DefaultExponentialBackoffInitialInterval bf.Multiplier = DefaultExponentialBackoffMultiplier bf.Reset() return bf })
DefaultBackoffPolicy is a default backoff policy.
Functions ¶
func CheckErrorIsTemporary ¶
CheckErrorIsTemporary checks either error is temporary or not.
func DefaultCheckRetry ¶
func DefaultCheckRetry( ctx context.Context, resp *http.Response, roundTripErr error, doneRetryAttempts int, ) (needRetry bool, err error)
DefaultCheckRetry determines whether the request should be retried. Retry policy:
- Network/transport error: retry if error is temporary (see CheckErrorIsTemporary).
- HTTP 429 Too Many Requests: always retry regardless of method. This status is usually a signal from the server that the request was rejected before it was processed due to rate limiting.
- HTTP 5xx and 408: retry only for clearly idempotent methods (GET/HEAD/OPTIONS) or when the request context carries the idempotent hint set via NewContextWithIdempotentHint(ctx, true).
func GetIdempotentHintFromContext ¶ added in v1.23.0
GetIdempotentHintFromContext extracts the "idempotent request" hint from context. Returns false when the key is not present. See NewContextWithIdempotentHint for details.
func GetRequestTypeFromContext ¶ added in v1.7.0
GetRequestTypeFromContext extracts request type from the context.
func MustNew ¶ added in v1.6.0
MustNew wraps delegate transports with logging, rate limiting, retryable, request id and panics if any error occurs.
func MustNewWithOpts ¶ added in v1.6.0
MustNewWithOpts wraps delegate transports with options logging, metrics, rate limiting, retryable, user agent, request id and panics if any error occurs.
func New ¶ added in v1.6.0
New wraps delegate transports with logging, rate limiting, retryable, request id and returns an error if any occurs.
func NewContextWithIdempotentHint ¶ added in v1.23.0
NewContextWithIdempotentHint returns a derived context that carries an "idempotent request" hint. When set to true, the request is considered idempotent even if it's not a GET/HEAD/OPTIONS request. Currently, this hint is used by RetryableRoundTripper in the DefaultCheckRetry function to decide whether it's safe to retry unsafe methods like POST and PATCH on retriable server errors.
func NewContextWithRequestType ¶ added in v1.7.0
NewContextWithRequestType creates a new context with request type.
func NewLoggingRoundTripper ¶ added in v1.6.0
func NewLoggingRoundTripper(delegate http.RoundTripper) http.RoundTripper
NewLoggingRoundTripper creates an HTTP transport that log requests.
func NewLoggingRoundTripperWithOpts ¶ added in v1.6.0
func NewLoggingRoundTripperWithOpts( delegate http.RoundTripper, opts LoggingRoundTripperOpts, ) http.RoundTripper
NewLoggingRoundTripperWithOpts creates an HTTP transport that log requests with options.
func NewMetricsRoundTripper ¶ added in v1.6.0
func NewMetricsRoundTripper(delegate http.RoundTripper, collector MetricsCollector) http.RoundTripper
NewMetricsRoundTripper creates an HTTP transport that measures requests done.
func NewMetricsRoundTripperWithOpts ¶ added in v1.6.0
func NewMetricsRoundTripperWithOpts( delegate http.RoundTripper, collector MetricsCollector, opts MetricsRoundTripperOpts, ) http.RoundTripper
NewMetricsRoundTripperWithOpts creates an HTTP transport that measures requests done.
func NewRequestIDRoundTripper ¶ added in v1.6.0
func NewRequestIDRoundTripper(delegate http.RoundTripper) http.RoundTripper
NewRequestIDRoundTripper creates an HTTP transport with X-Request-ID header support.
func NewRequestIDRoundTripperWithOpts ¶ added in v1.6.0
func NewRequestIDRoundTripperWithOpts(delegate http.RoundTripper, opts RequestIDRoundTripperOpts) http.RoundTripper
NewRequestIDRoundTripperWithOpts creates an HTTP transport with X-Request-ID header support with options.
Types ¶
type AuthBearerRoundTripper ¶
type AuthBearerRoundTripper struct {
Delegate http.RoundTripper
AuthProvider AuthProvider
// contains filtered or unexported fields
}
AuthBearerRoundTripper implements http.RoundTripper interface and sets the "Authorization: Bearer <token>" HTTP header in outgoing requests using the provided AuthProvider. Notes and caveats:
- If the request already contains the Authorization header, this RoundTripper delegates to the underlying transport without any token refresh or retry logic.
- On an authentication failure (by default HTTP 401), it will optionally invalidate the provider cache (if the provider implements AuthProviderInvalidator), obtain a new token, and retry the request once with the new token.
- Only a single retry attempt is performed by design. There is no backoff or multi‑step retry here. focuses strictly on auth-related retries).
- For requests with bodies, the component attempts to make the body rewindable to safely retry once (prefers Request.GetBody, then io.ReadSeeker, and as a last resort buffers the entire body in memory). For very large payloads, callers should prefer providing Request.GetBody or a seekable reader to avoid high memory usage.
- MinInvalidationInterval throttles how often the provider cache invalidation can happen to prevent hammering token endpoints under bursts of 401s; the throttling is maintained per RoundTripper instance.
func NewAuthBearerRoundTripper ¶
func NewAuthBearerRoundTripper(delegate http.RoundTripper, authProvider AuthProvider) *AuthBearerRoundTripper
NewAuthBearerRoundTripper creates a new AuthBearerRoundTripper.
func NewAuthBearerRoundTripperWithOpts ¶
func NewAuthBearerRoundTripperWithOpts( delegate http.RoundTripper, authProvider AuthProvider, opts AuthBearerRoundTripperOpts, ) *AuthBearerRoundTripper
NewAuthBearerRoundTripperWithOpts creates a new AuthBearerRoundTripper with options.
func (*AuthBearerRoundTripper) RoundTrip ¶
RoundTrip executes a single HTTP transaction, returning a Response for the provided Request. Behavior overview:
- If Authorization header is already present, the request is passed through without any retry/refresh.
- Otherwise, it adds a bearer token from AuthProvider and sends the request.
- If ShouldRefreshTokenAndRetry returns true (default: on 401), it may invalidate provider cache (throttled by MinInvalidationInterval), fetch a new token, safely rewind the request body if needed, and retry once.
- On transport errors (no response), no auth retry is attempted here.
- If token refresh fails or token stays the same after invalidation, the original response is returned unchanged.
type AuthBearerRoundTripperError ¶
type AuthBearerRoundTripperError struct {
Inner error
}
AuthBearerRoundTripperError is returned in RoundTrip method of AuthBearerRoundTripper when the original request cannot be potentially retried.
func (*AuthBearerRoundTripperError) Error ¶
func (e *AuthBearerRoundTripperError) Error() string
func (*AuthBearerRoundTripperError) Unwrap ¶
func (e *AuthBearerRoundTripperError) Unwrap() error
Unwrap returns the next error in the error chain.
type AuthBearerRoundTripperOpts ¶
type AuthBearerRoundTripperOpts struct {
// TokenScope is an optional list of scopes to pass to AuthProvider.GetToken.
// By default, no scopes are passed.
TokenScope []string
// MinInvalidationInterval determines the minimum time between cache invalidation attempts
// for the same AuthProvider. This prevents excessive calls to token endpoints.
// By default, DefaultAuthProviderMinInvalidationInterval const is used.
MinInvalidationInterval time.Duration
// ShouldRefreshTokenAndRetry is called after the first attempt and decides whether a token refresh attempt
// should be performed and the request should be retried once more (cache invalidation + token refresh + single retry).
// By default, it returns true only for HTTP 401 responses.
// If you need to retry only for specific endpoints or HTTP methods, you can provide your own implementation
// and use http.Response.Request field to make the decision.
ShouldRefreshTokenAndRetry func(ctx context.Context, resp *http.Response) bool
// LoggerProvider is a function that provides a context-specific logger.
// One of the typical use cases is to use an auth client in the context of a request handler,
// where the logger should produce logs with request-specific information (e.g., request ID).
LoggerProvider func(ctx context.Context) log.FieldLogger
}
AuthBearerRoundTripperOpts is options for AuthBearerRoundTripper.
type AuthProvider ¶
AuthProvider provides auth information used for bearer authorization.
type AuthProviderFunc ¶ added in v1.18.0
AuthProviderFunc allows to define get token logic in functional way.
type AuthProviderInvalidator ¶ added in v1.23.0
type AuthProviderInvalidator interface {
Invalidate()
}
AuthProviderInvalidator is implemented by AuthProviders that support cache invalidation.
type CheckRetryFunc ¶
type CheckRetryFunc func(ctx context.Context, resp *http.Response, roundTripErr error, doneRetryAttempts int) (bool, error)
CheckRetryFunc is a function that is called right after RoundTrip() method and determines if the next retry attempt is needed.
type Config ¶ added in v1.6.0
type Config struct {
// Retries is a configuration for HTTP client retries policy.
Retries RetriesConfig `mapstructure:"retries" yaml:"retries" json:"retries"`
// RateLimits is a configuration for HTTP client rate limits.
RateLimits RateLimitsConfig `mapstructure:"rateLimits" yaml:"rateLimits" json:"rateLimits"`
// Log is a configuration for HTTP client logs.
Log LogConfig `mapstructure:"log" yaml:"log" json:"log"`
// Metrics is a configuration for HTTP client metrics.
Metrics MetricsConfig `mapstructure:"metrics" yaml:"metrics" json:"metrics"`
// Timeout is the maximum time to wait for a request to be made.
Timeout config.TimeDuration `mapstructure:"timeout" yaml:"timeout" json:"timeout"`
// contains filtered or unexported fields
}
Config represents options for HTTP client configuration. Configuration can be loaded in different formats (YAML, JSON) using config.Loader, viper, or with json.Unmarshal/yaml.Unmarshal functions directly.
func NewConfig ¶ added in v1.6.0
func NewConfig(options ...ConfigOption) *Config
NewConfig creates a new instance of the Config.
func NewConfigWithKeyPrefix ¶ added in v1.6.0
NewConfigWithKeyPrefix creates a new instance of the Config with a key prefix. This prefix will be used by config.Loader. Deprecated: use NewConfig with WithKeyPrefix instead.
func NewDefaultConfig ¶ added in v1.11.0
func NewDefaultConfig(options ...ConfigOption) *Config
NewDefaultConfig creates a new instance of the Config with default values.
func (*Config) KeyPrefix ¶ added in v1.6.0
KeyPrefix returns a key prefix with which all configuration parameters should be presented. Implements config.KeyPrefixProvider interface.
func (*Config) Set ¶ added in v1.6.0
func (c *Config) Set(dp config.DataProvider) error
Set sets http client configuration based on the passed config.DataProvider. Implements config.Config interface.
func (*Config) SetProviderDefaults ¶ added in v1.6.0
func (c *Config) SetProviderDefaults(dp config.DataProvider)
SetProviderDefaults is part of config interface implementation. Implements config.Config interface.
type ConfigOption ¶ added in v1.11.0
type ConfigOption func(*configOptions)
ConfigOption is a type for functional options for the Config.
func WithKeyPrefix ¶ added in v1.11.0
func WithKeyPrefix(keyPrefix string) ConfigOption
WithKeyPrefix returns a ConfigOption that sets a key prefix for parsing configuration parameters. This prefix will be used by config.Loader.
type ConstantBackoffConfig ¶ added in v1.6.0
type ConstantBackoffConfig struct {
// Interval is the interval for constant backoff.
Interval config.TimeDuration `mapstructure:"interval" yaml:"interval" json:"interval"`
}
ConstantBackoffConfig represents configuration options for constant backoff.
type ExponentialBackoffConfig ¶ added in v1.6.0
type ExponentialBackoffConfig struct {
// InitialInterval is the initial interval for exponential backoff.
InitialInterval config.TimeDuration `mapstructure:"initialInterval" yaml:"initialInterval" json:"initialInterval"`
// Multiplier is the multiplier for exponential backoff.
Multiplier float64 `mapstructure:"multiplier" yaml:"multiplier" json:"multiplier"`
}
ExponentialBackoffConfig represents configuration options for exponential backoff.
type LogConfig ¶ added in v1.6.0
type LogConfig struct {
// Enabled is a flag that enables logging.
Enabled bool `mapstructure:"enabled" yaml:"enabled" json:"enabled"`
// SlowRequestThreshold is a threshold for slow requests.
SlowRequestThreshold config.TimeDuration `mapstructure:"slowRequestThreshold" yaml:"slowRequestThreshold" json:"slowRequestThreshold"`
// Mode of logging: [all, failed]. 'all' by default.
Mode LoggingMode `mapstructure:"mode" yaml:"mode" json:"mode"`
}
LogConfig represents configuration options for HTTP client logs.
func (*LogConfig) TransportOpts ¶ added in v1.6.0
func (c *LogConfig) TransportOpts() LoggingRoundTripperOpts
TransportOpts returns transport options.
type LoggingMode ¶ added in v1.6.0
type LoggingMode string
LoggingMode represents a mode of logging.
const ( LoggingModeAll LoggingMode = "all" LoggingModeFailed LoggingMode = "failed" )
func (LoggingMode) IsValid ¶ added in v1.6.0
func (lm LoggingMode) IsValid() bool
IsValid checks if the logger mode is valid.
type LoggingRoundTripper ¶ added in v1.6.0
type LoggingRoundTripper struct {
// Delegate is the next RoundTripper in the chain.
Delegate http.RoundTripper
// ClientType represents a type of client, it's a service component reference. e.g. 'auth-service'
ClientType string
// Mode of logging: [all, failed]. 'all' by default.
Mode LoggingMode
// SlowRequestThreshold is a threshold for slow requests.
SlowRequestThreshold time.Duration
// LoggerProvider is a function that provides a context-specific logger.
// middleware.GetLoggerFromContext is used by default.
LoggerProvider func(ctx context.Context) log.FieldLogger
}
LoggingRoundTripper implements http.RoundTripper for logging requests.
func (*LoggingRoundTripper) RoundTrip ¶ added in v1.6.0
RoundTrip adds logging capabilities to the HTTP transport. It logs HTTP requests and responses, tracking execution time and success/failure status.
Logging behavior: - If logging is disabled (no logger), the request is simply forwarded. - If the request completes successfully and is not slow, it may be ignored based on logging mode. - Slow requests, failed requests, and configured conditions determine log level (Info, Warn, Error).
Additional logged information:
- HTTP method and URL.
- Response status code and elapsed time.
- Client type if available.
- X-Request-ID from request headers.
- Request type extracted from the context.
- The request duration in milliseconds is accumulated in LoggingParams, allowing tracking of the total time spent on external requests when the handler execution is finalized. Note that this value is accurate only for sequential requests.
type LoggingRoundTripperOpts ¶ added in v1.6.0
type LoggingRoundTripperOpts struct {
// ClientType represents a type of client, it's a service component reference. e.g. 'auth-service'
ClientType string
// Mode of logging: [all, failed]. 'all' by default.
Mode LoggingMode
// SlowRequestThreshold is a threshold for slow requests.
SlowRequestThreshold time.Duration
// LoggerProvider is a function that provides a context-specific logger.
// middleware.GetLoggerFromContext is used by default.
LoggerProvider func(ctx context.Context) log.FieldLogger
}
LoggingRoundTripperOpts represents an options for LoggingRoundTripper.
type MetricsCollector ¶ added in v1.6.0
type MetricsCollector interface {
// RequestDuration observes the duration of the request and the status code.
RequestDuration(clientType, remoteAddress, summary, status, requestType string, duration float64)
}
MetricsCollector is an interface for collecting metrics for client requests.
type MetricsConfig ¶ added in v1.6.0
type MetricsConfig struct {
// Enabled is a flag that enables metrics.
Enabled bool `mapstructure:"enabled" yaml:"enabled" json:"enabled"`
}
MetricsConfig represents configuration options for HTTP client logs.
type MetricsRoundTripper ¶ added in v1.6.0
type MetricsRoundTripper struct {
// Delegate is the next RoundTripper in the chain.
Delegate http.RoundTripper
// ClientType represents a type of client, it's a service component reference. e.g. 'auth-service'
ClientType string
// MetricsCollector is a metrics collector.
MetricsCollector MetricsCollector
// ClassifyRequest does request classification, producing non-parameterized summary for given request.
ClassifyRequest func(r *http.Request, clientType string) string
}
MetricsRoundTripper is an HTTP transport that measures requests done.
type MetricsRoundTripperOpts ¶ added in v1.6.0
type MetricsRoundTripperOpts struct {
// ClientType represents a type of client, it's a service component reference. e.g. 'auth-service'
ClientType string
// ClassifyRequest does request classification, producing non-parameterized summary for given request.
ClassifyRequest func(r *http.Request, clientType string) string
}
MetricsRoundTripperOpts is an HTTP transport that measures requests done.
type Opts ¶ added in v1.6.0
type Opts struct {
// UserAgent is a user agent string.
UserAgent string
// AuthProvider provides auth information used for bearer authorization.
AuthProvider AuthProvider
// ClientType represents a type of client, it's a service component reference. e.g. 'auth-service'.
ClientType string
// Delegate is the next RoundTripper in the chain.
Delegate http.RoundTripper
// LoggerProvider is a function that provides a context-specific logger.
LoggerProvider func(ctx context.Context) log.FieldLogger
// RequestIDProvider is a function that provides a request ID.
RequestIDProvider func(ctx context.Context) string
// MetricsCollector is a metrics collector.
MetricsCollector MetricsCollector
// ClassifyRequest does request classification, producing non-parameterized summary in metrics round tripper.
ClassifyRequest func(r *http.Request, clientType string) string
}
Opts provides options for NewWithOpts and MustWithOpts functions.
type PrometheusMetricsCollector ¶ added in v1.6.0
type PrometheusMetricsCollector struct {
// Durations is a histogram of the http client requests durations.
Durations *prometheus.HistogramVec
}
PrometheusMetricsCollector is a Prometheus metrics collector.
func NewPrometheusMetricsCollector ¶ added in v1.6.0
func NewPrometheusMetricsCollector(namespace string) *PrometheusMetricsCollector
NewPrometheusMetricsCollector creates a new Prometheus metrics collector.
func (*PrometheusMetricsCollector) MustRegister ¶ added in v1.6.0
func (p *PrometheusMetricsCollector) MustRegister()
MustRegister registers the Prometheus metrics.
func (*PrometheusMetricsCollector) RequestDuration ¶ added in v1.6.0
func (p *PrometheusMetricsCollector) RequestDuration( clientType, host, summary, status, requestType string, duration float64, )
RequestDuration observes the duration of the request and the status code.
func (*PrometheusMetricsCollector) Unregister ¶ added in v1.6.0
func (p *PrometheusMetricsCollector) Unregister()
Unregister the Prometheus metrics.
type RateLimitingRoundTripper ¶
type RateLimitingRoundTripper struct {
Delegate http.RoundTripper
RateLimit int
Burst int
WaitTimeout time.Duration
Adaptation RateLimitingRoundTripperAdaptation
// contains filtered or unexported fields
}
RateLimitingRoundTripper wraps implementing http.RoundTripper interface object and provides adaptive (can use limit from response's HTTP header) rate limiting mechanism for outgoing requests.
func NewRateLimitingRoundTripper ¶
func NewRateLimitingRoundTripper(delegate http.RoundTripper, rateLimit int) (*RateLimitingRoundTripper, error)
NewRateLimitingRoundTripper creates a new RateLimitingRoundTripper with specified rate limit.
Example ¶
ExampleNewRateLimitingRoundTripper demonstrates the use of RateLimitingRoundTripper with default parameters.
Add "// Output:" in the end of the function and run:
$ go test ./httpclient -v -run ExampleNewRateLimitingRoundTripper
Output will be like:
[Req#1] 204 (0ms) [Req#2] 204 (502ms) [Req#3] 204 (497ms) [Req#4] 204 (500ms) [Req#5] 204 (503ms)
// Note: error handling is intentionally omitted so as not to overcomplicate the example.
// It is strictly necessary to handle all errors in real code.
server := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
rw.WriteHeader(http.StatusNoContent)
}))
// Let's make transport that may do maximum 2 requests per second.
tr, _ := NewRateLimitingRoundTripper(http.DefaultTransport, 2)
httpClient := &http.Client{Transport: tr}
start := time.Now()
prev := time.Now()
for i := 0; i < 5; i++ {
resp, _ := httpClient.Get(server.URL)
_ = resp.Body.Close()
now := time.Now()
_, _ = fmt.Fprintf(os.Stderr, "[Req#%d] %d (%dms)\n", i+1, resp.StatusCode, now.Sub(prev).Milliseconds())
prev = now
}
delta := time.Since(start) - time.Second*2
if delta > time.Millisecond*20 {
fmt.Println("Total time is much greater than 2s")
} else {
fmt.Println("Total time is about 2s")
}
Output: Total time is about 2s
func NewRateLimitingRoundTripperWithOpts ¶
func NewRateLimitingRoundTripperWithOpts( delegate http.RoundTripper, rateLimit int, opts RateLimitingRoundTripperOpts, ) (*RateLimitingRoundTripper, error)
NewRateLimitingRoundTripperWithOpts creates a new RateLimitingRoundTripper with specified rate limit and options. For options that are not presented, the default values will be used.
Example ¶
ExampleNewRateLimitingRoundTripperWithOpts demonstrates the use of RateLimitingRoundTripper.
// Note: error handling is intentionally omitted so as not to overcomplicate the example.
// It is strictly necessary to handle all errors in real code.
server := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
rw.WriteHeader(http.StatusNoContent)
}))
// Let's make transport that may do maximum 2 requests per second.
tr, _ := NewRateLimitingRoundTripperWithOpts(http.DefaultTransport, 2, RateLimitingRoundTripperOpts{
WaitTimeout: time.Millisecond * 100, // Wait maximum 100ms.
})
httpClient := &http.Client{Transport: tr}
for i := 0; i < 2; i++ {
resp, err := httpClient.Get(server.URL)
if err != nil {
var waitErr *RateLimitingWaitError
if errors.As(err, &waitErr) {
fmt.Printf("trying to do too many requests")
}
continue
}
_ = resp.Body.Close()
}
Output: trying to do too many requests
type RateLimitingRoundTripperAdaptation ¶
RateLimitingRoundTripperAdaptation represents a params to adapt rate limiting in accordance with value in response.
type RateLimitingRoundTripperOpts ¶
type RateLimitingRoundTripperOpts struct {
Burst int
WaitTimeout time.Duration
Adaptation RateLimitingRoundTripperAdaptation
}
RateLimitingRoundTripperOpts represents an options for RateLimitingRoundTripper.
type RateLimitingWaitError ¶
type RateLimitingWaitError struct {
Inner error
}
RateLimitingWaitError is returned in RoundTrip method of RateLimitingRoundTripper when rate limit is exceeded.
func (*RateLimitingWaitError) Error ¶
func (e *RateLimitingWaitError) Error() string
func (*RateLimitingWaitError) Unwrap ¶
func (e *RateLimitingWaitError) Unwrap() error
Unwrap returns the next error in the error chain.
type RateLimitsConfig ¶ added in v1.6.0
type RateLimitsConfig struct {
// Enabled is a flag that enables rate limiting.
Enabled bool `mapstructure:"enabled" yaml:"enabled" json:"enabled"`
// Limit is the maximum number of requests that can be made.
Limit int `mapstructure:"limit" yaml:"limit" json:"limit"`
// Burst allow temporary spikes in request rate.
Burst int `mapstructure:"burst" yaml:"burst" json:"burst"`
// WaitTimeout is the maximum time to wait for a request to be made.
WaitTimeout config.TimeDuration `mapstructure:"waitTimeout" yaml:"waitTimeout" json:"waitTimeout"`
}
RateLimitsConfig represents configuration options for HTTP client rate limits.
func (*RateLimitsConfig) TransportOpts ¶ added in v1.6.0
func (c *RateLimitsConfig) TransportOpts() RateLimitingRoundTripperOpts
TransportOpts returns transport options.
type RequestIDRoundTripper ¶ added in v1.6.0
type RequestIDRoundTripper struct {
// Delegate is the next RoundTripper in the chain.
Delegate http.RoundTripper
// Opts are the options for the request ID round tripper.
Opts RequestIDRoundTripperOpts
}
RequestIDRoundTripper for X-Request-ID header to the request.
type RequestIDRoundTripperOpts ¶ added in v1.6.0
type RequestIDRoundTripperOpts struct {
// RequestIDProvider is a function that provides a request ID.
// middleware.GetRequestIDFromContext is used by default.
RequestIDProvider func(ctx context.Context) string
}
RequestIDRoundTripperOpts for X-Request-ID header to the request options.
type RetriesConfig ¶ added in v1.6.0
type RetriesConfig struct {
// Enabled is a flag that enables retries.
Enabled bool `mapstructure:"enabled" yaml:"enabled" json:"enabled"`
// MaxAttempts is the maximum number of attempts to retry the request.
MaxAttempts int `mapstructure:"maxAttempts" yaml:"maxAttempts" json:"maxAttempts"`
// Policy of a retry: [exponential, constant].
Policy RetryPolicy `mapstructure:"policy" yaml:"policy" json:"policy"`
// ExponentialBackoff is the configuration for exponential backoff.
ExponentialBackoff ExponentialBackoffConfig `mapstructure:"exponentialBackoff" yaml:"exponentialBackoff" json:"exponentialBackoff"`
// ConstantBackoff is the configuration for constant backoff.
ConstantBackoff ConstantBackoffConfig `mapstructure:"constantBackoff" yaml:"constantBackoff" json:"constantBackoff"`
}
RetriesConfig represents configuration options for HTTP client retries policy.
func (*RetriesConfig) GetPolicy ¶ added in v1.6.0
func (c *RetriesConfig) GetPolicy() retry.Policy
GetPolicy returns a retry policy based on strategy or nil if none is provided.
func (*RetriesConfig) TransportOpts ¶ added in v1.6.0
func (c *RetriesConfig) TransportOpts() RetryableRoundTripperOpts
TransportOpts returns transport options.
type RetryPolicy ¶ added in v1.6.0
type RetryPolicy string
RetryPolicy represents a retry policy strategy.
const ( DefaultTimeout = time.Minute // RetryPolicyExponential is a policy for exponential retries. RetryPolicyExponential RetryPolicy = "exponential" // RetryPolicyConstant is a policy for constant retries. RetryPolicyConstant RetryPolicy = "constant" )
type RetryableRoundTripper ¶
type RetryableRoundTripper struct {
// Delegate is an object that implements http.RoundTripper interface
// and is used for sending HTTP requests under the hood.
Delegate http.RoundTripper
// Logger is used for logging.
// Deprecated: use LoggerProvider instead.
Logger log.FieldLogger
// LoggerProvider is a function that provides a context-specific logger.
// One of the typical use cases is to use a retryable client in the context of a request handler,
// where the logger should produce logs with request-specific information (e.g., request ID).
LoggerProvider func(ctx context.Context) log.FieldLogger
// MaxRetryAttempts determines how many maximum retry attempts can be done.
// The total number of sending HTTP request may be MaxRetryAttempts + 1 (the first request is not a retry attempt).
// If its value is UnlimitedRetryAttempts, it's supposed that retry mechanism will be stopped by BackoffPolicy.
// By default, DefaultMaxRetryAttempts const is used.
MaxRetryAttempts int
// CheckRetry is called right after RoundTrip() method and determines if the next retry attempt is needed.
// By default, DefaultCheckRetry function is used.
CheckRetry CheckRetryFunc
// IgnoreRetryAfter determines if Retry-After HTTP header of the response is parsed and
// used as a wait time before doing the next retry attempt.
// If it's true or response doesn't contain Retry-After HTTP header, BackoffPolicy will be used for computing delay.
IgnoreRetryAfter bool
// BackoffPolicy is used for computing wait time before doing the next retry attempt
// when the given response doesn't contain Retry-After HTTP header or IgnoreRetryAfter is true.
// By default, DefaultBackoffPolicy is used.
BackoffPolicy retry.Policy
}
RetryableRoundTripper wraps an object that implements http.RoundTripper interface and provides a retrying mechanism for HTTP requests.
func NewRetryableRoundTripper ¶
func NewRetryableRoundTripper(delegate http.RoundTripper) (*RetryableRoundTripper, error)
NewRetryableRoundTripper returns a new instance of RetryableRoundTripper.
Example ¶
ExampleNewRetryableRoundTripper demonstrates the use of RetryableRoundTripper with default parameters.
To execute this example: 1) Add "// Output: Got 200 after 10 retry attempts" in the end of function. 2) Run:
$ go test ./httpclient -v -run ExampleNewRetryableRoundTripper
Stderr will contain something like this:
[Req#1] wait time: 0.001s [Req#2] wait time: 1.109s [Req#3] wait time: 2.882s [Req#4] wait time: 4.661s [Req#5] wait time: 7.507s [Req#6] wait time: 14.797s [Req#7] wait time: 37.984s [Req#8] wait time: 33.942s [Req#9] wait time: 39.394s [Req#10] wait time: 35.820s [Req#11] wait time: 48.060s
// Note: error handling is intentionally omitted so as not to overcomplicate the example.
// It is strictly necessary to handle all errors in real code.
reqTimes := make(chan time.Time, DefaultMaxRetryAttempts+1)
server := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
reqTimes <- time.Now()
if n, err := strconv.Atoi(r.Header.Get(RetryAttemptNumberHeader)); err == nil && n == DefaultMaxRetryAttempts {
rw.WriteHeader(http.StatusOK)
_, _ = rw.Write([]byte("ok, you win..."))
return
}
rw.WriteHeader(http.StatusServiceUnavailable)
}))
prevTime := time.Now()
// Create RetryableRoundTripper with default params:
// + maximum retry attempts = 10 (DefaultMaxRetryAttempts)
// + respect Retry-After response HTTP header
// + exponential backoff policy (DefaultBackoffPolicy, multiplier = 2, initial interval = 1s)
tr, _ := NewRetryableRoundTripper(http.DefaultTransport)
httpClient := &http.Client{Transport: tr}
resp, err := httpClient.Get(server.URL)
if err != nil {
log.Fatal(err)
}
_ = resp.Body.Close()
close(reqTimes)
reqsCount := 0
for reqTime := range reqTimes {
reqsCount++
_, _ = fmt.Fprintf(os.Stderr, "[Req#%d] wait time: %.3fs\n", reqsCount, reqTime.Sub(prevTime).Seconds())
prevTime = reqTime
}
doneRetryAttempts := reqsCount - 1
fmt.Printf("Got %d after %d retry attempts", resp.StatusCode, doneRetryAttempts)
func NewRetryableRoundTripperWithOpts ¶
func NewRetryableRoundTripperWithOpts( delegate http.RoundTripper, opts RetryableRoundTripperOpts, ) (*RetryableRoundTripper, error)
NewRetryableRoundTripperWithOpts creates a new instance of RateLimitingRoundTripper with specified options.
type RetryableRoundTripperError ¶
type RetryableRoundTripperError struct {
Inner error
}
RetryableRoundTripperError is returned in RoundTrip method of RetryableRoundTripper when the original request cannot be potentially retried.
func (*RetryableRoundTripperError) Error ¶
func (e *RetryableRoundTripperError) Error() string
func (*RetryableRoundTripperError) Unwrap ¶
func (e *RetryableRoundTripperError) Unwrap() error
Unwrap returns the next error in the error chain.
type RetryableRoundTripperOpts ¶
type RetryableRoundTripperOpts struct {
// Logger is used for logging.
// Deprecated: use LoggerProvider instead.
Logger log.FieldLogger
// LoggerProvider is a function that provides a context-specific logger.
// One of the typical use cases is to use a retryable client in the context of a request handler,
// where the logger should produce logs with request-specific information (e.g., request ID).
LoggerProvider func(ctx context.Context) log.FieldLogger
// MaxRetryAttempts determines how many maximum retry attempts can be done.
// The total number of sending HTTP request may be MaxRetryAttempts + 1 (the first request is not a retry attempt).
// If its value is UnlimitedRetryAttempts, it's supposed that retry mechanism will be stopped by BackoffPolicy.
// By default, DefaultMaxRetryAttempts const is used.
MaxRetryAttempts int
// CheckRetryFunc is called right after RoundTrip() method and determines if the next retry attempt is needed.
// By default, DefaultCheckRetry function is used.
CheckRetryFunc CheckRetryFunc
// IgnoreRetryAfter determines if Retry-After HTTP header of the response is parsed and
// used as a wait time before doing the next retry attempt.
// If it's true or response doesn't contain Retry-After HTTP header, BackoffPolicy will be used for computing delay.
IgnoreRetryAfter bool
// BackoffPolicy is used for computing wait time before doing the next retry attempt
// when the given response doesn't contain Retry-After HTTP header or IgnoreRetryAfter is true.
// By default, DefaultBackoffPolicy is used.
BackoffPolicy retry.Policy
}
RetryableRoundTripperOpts represents an options for RetryableRoundTripper.
type UserAgentRoundTripper ¶
type UserAgentRoundTripper struct {
Delegate http.RoundTripper
UserAgent string
UpdateStrategy UserAgentUpdateStrategy
}
UserAgentRoundTripper implements http.RoundTripper interface and sets User-Agent HTTP header in all outgoing requests.
func NewUserAgentRoundTripper ¶
func NewUserAgentRoundTripper(delegate http.RoundTripper, userAgent string) *UserAgentRoundTripper
NewUserAgentRoundTripper creates a new UserAgentRoundTripper.
func NewUserAgentRoundTripperWithOpts ¶
func NewUserAgentRoundTripperWithOpts( delegate http.RoundTripper, userAgent string, opts UserAgentRoundTripperOpts, ) *UserAgentRoundTripper
NewUserAgentRoundTripperWithOpts creates a new UserAgentRoundTripper with specified options.
type UserAgentRoundTripperOpts ¶
type UserAgentRoundTripperOpts struct {
UpdateStrategy UserAgentUpdateStrategy
}
UserAgentRoundTripperOpts represents an options for UserAgentRoundTripper.
type UserAgentUpdateStrategy ¶
type UserAgentUpdateStrategy int
UserAgentUpdateStrategy represents a strategy for updating User-Agent HTTP header.
const ( UserAgentUpdateStrategySetIfEmpty UserAgentUpdateStrategy = iota UserAgentUpdateStrategyAppend UserAgentUpdateStrategyPrepend )
User-Agent update strategies.