httpclient

package
v1.29.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 23, 2025 License: MIT Imports: 18 Imported by: 0

README

httpclient

GoDoc Widget

A collection of HTTP client round trippers that can be chained together to enhance HTTP clients with reliability, observability, and traffic control features.

Additionally, the package includes a factory function to create HTTP clients with pre-configured round trippers based on YAML/JSON configuration files.

Table of Contents

Available Round Trippers

Authentication Bearer Token Round Tripper

Automatically handles Bearer token authentication with token refresh capabilities.

Features:

  • Automatic token injection
  • Token refresh on authentication failures
  • Cache invalidation with minimum intervals
  • Request body rewinding for retries after token refresh

Usage:

// Create auth provider
tokenProvider := httpclient.AuthProviderFunc(func(ctx context.Context, scope ...string) (string, error) {
    // Your token acquisition logic here
    return "your-access-token", nil
})

// Basic usage
transport := httpclient.NewAuthBearerRoundTripper(http.DefaultTransport, tokenProvider)

// With custom options
transport := httpclient.NewAuthBearerRoundTripperWithOpts(
    http.DefaultTransport,
    tokenProvider,
    httpclient.AuthBearerRoundTripperOpts{
        TokenScope: []string{"my-service:read", "my-service:write"},
        MinInvalidationInterval: 15 * time.Minute,
        ShouldRefreshTokenAndRetry: func(ctx context.Context, resp *http.Response) bool {
			// Custom condition to trigger token refresh and retry
        },
    },
)

client := &http.Client{Transport: transport}

Configuration Options:

  • TokenScope: OAuth scopes for token requests
  • MinInvalidationInterval: Minimum time between auth token provider cache invalidations
  • ShouldRefreshTokenAndRetry: Custom condition for token refresh and retry
  • LoggerProvider: Context-specific logger function
Logging Round Tripper

Provides comprehensive HTTP request and response logging.

Features:

  • Request/response logging
  • Slow request detection
  • Configurable logging modes (all/failed)
  • Client type identification

Usage:

// Basic logging
transport := httpclient.NewLoggingRoundTripper(http.DefaultTransport)

// With custom options
transport := httpclient.NewLoggingRoundTripperWithOpts(
    http.DefaultTransport,
    httpclient.LoggingRoundTripperOpts{
        ClientType:           "my-api-client",
        Mode:                 httpclient.LoggingModeAll,
        SlowRequestThreshold: 2 * time.Second,
        LoggerProvider: func(ctx context.Context) log.FieldLogger {
			// Extract logger from your context
        },
    },
)

client := &http.Client{Transport: transport}

Logging Modes:

  • LoggingModeAll: Log all requests and responses (default)
  • LoggingModeFailed: Log only failed or slow requests

Configuration Options:

  • ClientType: Type identifier for the client (used in logs and metrics)
  • Mode: Logging mode - all requests or only failed/slow requests
  • SlowRequestThreshold: Duration threshold for identifying slow requests
  • LoggerProvider: Context-specific logger function
Metrics Round Tripper

Collects Prometheus metrics for HTTP client requests.

Features:

  • Request duration histograms
  • Request classification
  • Client type labeling
  • Customizable metrics collection

Usage:

// Create metrics collector
collector := httpclient.NewPrometheusMetricsCollector("myapp")
collector.MustRegister()

// Create transport
transport := httpclient.NewMetricsRoundTripperWithOpts(
    http.DefaultTransport,
    collector,
    httpclient.MetricsRoundTripperOpts{
        ClientType: "api-client",
        ClassifyRequest: func(r *http.Request, clientType string) string {
			// Custom request classification logic
        },
    },
)

client := &http.Client{Transport: transport}

Metrics Collected:

  • http_client_request_duration_seconds: Histogram of request durations (suitable for Prometheus alerts on response times or codes)

Labels:

  • client_type: Type of client making the request
  • remote_address: Target server address
  • summary: Request classification summary
  • status: HTTP response status code
  • request_type: Type of request (from context)

Configuration Options:

  • ClientType: Type identifier for the client (included in metrics labels)
  • ClassifyRequest: Function to classify requests for metrics summary
User Agent Round Tripper

Manages User-Agent headers in outgoing requests.

Features:

  • Multiple update strategies
  • Conditional User-Agent setting
  • Header composition

Usage:

// Basic usage - set User-Agent if empty
transport := httpclient.NewUserAgentRoundTripper(http.DefaultTransport, "myapp/1.0")

// With custom strategy
transport := httpclient.NewUserAgentRoundTripperWithOpts(
    http.DefaultTransport,
    "myapp/1.0",
    httpclient.UserAgentRoundTripperOpts{
        UpdateStrategy: httpclient.UserAgentUpdateStrategyAppend,
    },
)

client := &http.Client{Transport: transport}

Configuration Options:

  • UpdateStrategy: Strategy for updating the User-Agent header

Update Strategies:

  • UserAgentUpdateStrategySetIfEmpty: Set only if User-Agent is empty (default)
  • UserAgentUpdateStrategyAppend: Append to existing User-Agent
  • UserAgentUpdateStrategyPrepend: Prepend to existing User-Agent
Rate Limiting Round Tripper

Controls outbound request rate to prevent overwhelming target services.

Features:

  • Token bucket rate limiting
  • Adaptive rate limiting based on response headers
  • Configurable burst capacity
  • Wait timeout control

Usage:

// Basic rate limiting - 10 requests per second
transport, err := httpclient.NewRateLimitingRoundTripper(http.DefaultTransport, 10)

// With custom options
transport, err := httpclient.NewRateLimitingRoundTripperWithOpts(
    http.DefaultTransport, 10,
    httpclient.RateLimitingRoundTripperOpts{
        Burst:       20,
        WaitTimeout: 5 * time.Second,
        Adaptation: httpclient.RateLimitingRoundTripperAdaptation{
            ResponseHeaderName: "X-RateLimit-Remaining",
            SlackPercent:       10,
        },
    },
)

client := &http.Client{Transport: transport}

Configuration Options:

  • Burst: Allow temporary request bursts (default: 1)
  • WaitTimeout: Maximum wait time for rate limiting (default: 15s)
  • Adaptation: Adaptive rate limiting based on response headers
Request ID Round Tripper

Automatically adds X-Request-ID headers to outgoing requests.

Features:

  • Automatic request ID generation
  • Context-aware request ID extraction
  • Request correlation support

Usage:

// Basic usage
transport := httpclient.NewRequestIDRoundTripper(http.DefaultTransport)

// With custom provider
transport := httpclient.NewRequestIDRoundTripperWithOpts(
    http.DefaultTransport,
    httpclient.RequestIDRoundTripperOpts{
        RequestIDProvider: func(ctx context.Context) string {
            // Extract request ID from your context
        },
    },
)

client := &http.Client{Transport: transport}

Configuration Options:

  • RequestIDProvider: Function to extract request ID from context
Retryable Round Tripper

Provides automatic retry functionality for failed HTTP requests with configurable backoff policies.

Features:

  • Configurable retry attempts (default: 10)
  • Multiple backoff policies (exponential, constant)
  • Retry-After header support
  • Idempotent request detection
  • Body rewinding for safe retries

Usage:

// Basic usage with default settings
transport, err := httpclient.NewRetryableRoundTripper(http.DefaultTransport)

// With custom options
transport, err := httpclient.NewRetryableRoundTripperWithOpts(
    http.DefaultTransport,
    httpclient.RetryableRoundTripperOpts{
        MaxRetryAttempts: 5,
        CheckRetryFunc: func(ctx context.Context, resp *http.Response, err error, attempts int) (bool, error) {
            // Custom retry logic, e.g., retry on 5xx status codes or network errors for idempotent methods, etc.
        },
        BackoffPolicy: retry.PolicyFunc(func() backoff.BackOff {
			// Custom backoff policy
        }),
    },
)

client := &http.Client{Transport: transport}

Configuration:

  • MaxRetryAttempts: Maximum number of retry attempts (default: 10)
  • CheckRetryFunc: Custom retry condition function
  • IgnoreRetryAfter: Ignore Retry-After response headers
  • BackoffPolicy: Backoff strategy for retry delays

Client Factory and Configuration

The package provides factory functions to create HTTP clients with pre-configured round trippers. Configuration can be loaded from YAML, JSON, environment variables, or defined programmatically.

Configuration can be loaded from:

  • YAML/JSON data using unmarshalling
  • YAML/JSON files and/or environment variables using config.Loader
  • Direct struct initialization
Creating Client from YAML Configuration

config.yaml:

httpclient:
  timeout: 30s
  retries:
    enabled: true
    maxAttempts: 5
    policy: exponential
    exponentialBackoff:
      initialInterval: 1s
      multiplier: 2.0
  rateLimits:
    enabled: true
    limit: 100
    burst: 20
    waitTimeout: 5s
  log:
    enabled: true
    mode: all
    slowRequestThreshold: 2s
  metrics:
    enabled: true

Go code:

import (
    "os"

    "gopkg.in/yaml.v3"

    "github.com/acronis/go-appkit/httpclient"
)

func createClientFromYAML() (*http.Client, error) {
    data, err := os.ReadFile("config.yaml")
    if err != nil {
        return nil, err
    }

    var config struct {
        HTTPClient httpclient.Config `yaml:"httpclient"`
    }
    if err := yaml.Unmarshal(data, &config); err != nil {
        return nil, err
    }
    
    // Create auth provider
    authProvider := httpclient.AuthProviderFunc(func(ctx context.Context, scope ...string) (string, error) {
        // Your token acquisition logic here
        return "your-access-token", nil
    })
    
    // Create metrics collector
    metricsCollector := httpclient.NewPrometheusMetricsCollector("my_service")
    metricsCollector.MustRegister()
    
    // Create client with loaded configuration
    return httpclient.NewWithOpts(&config.HTTPClient, httpclient.Opts{
        UserAgent:        "my-service/1.0.0",
        ClientType:       "external-api",
        AuthProvider:     authProvider,
        MetricsCollector: metricsCollector,
        LoggerProvider: func(ctx context.Context) log.FieldLogger {
            return middleware.GetLoggerFromContext(ctx)
        },
        RequestIDProvider: func(ctx context.Context) string {
            return middleware.GetRequestIDFromContext(ctx)
        },
    })
}

Documentation

Overview

Package httpclient contains helpers for doing HTTP requests (for example, round trippers for retrying and rate limiting).

Index

Examples

Constants

View Source
const (
	DefaultRateLimitingBurst       = 1
	DefaultRateLimitingWaitTimeout = 15 * time.Second
)

Default parameter values for RateLimitingRoundTripper.

View Source
const (
	DefaultMaxRetryAttempts                  = 10
	DefaultExponentialBackoffInitialInterval = time.Second
	DefaultExponentialBackoffMultiplier      = 2
)

Default parameter values for RetryableRoundTripper.

View Source
const DefaultAuthProviderMinInvalidationInterval = 15 * time.Minute

DefaultAuthProviderMinInvalidationInterval is the default minimum time between cache invalidation attempts for the same AuthProvider.

View Source
const RetryAttemptNumberHeader = "X-Retry-Attempt"

RetryAttemptNumberHeader is an HTTP header name that will contain the serial number of the retry attempt.

View Source
const UnlimitedRetryAttempts = -1

UnlimitedRetryAttempts should be used as RetryableRoundTripperOpts.MaxRetryAttempts value when we want to stop retries only by RetryableRoundTripperOpts.BackoffPolicy.

Variables

View Source
var DefaultBackoffPolicy = retry.PolicyFunc(func() backoff.BackOff {
	bf := backoff.NewExponentialBackOff()
	bf.InitialInterval = DefaultExponentialBackoffInitialInterval
	bf.Multiplier = DefaultExponentialBackoffMultiplier
	bf.Reset()
	return bf
})

DefaultBackoffPolicy is a default backoff policy.

Functions

func CheckErrorIsTemporary

func CheckErrorIsTemporary(err error) bool

CheckErrorIsTemporary checks either error is temporary or not.

func DefaultCheckRetry

func DefaultCheckRetry(
	ctx context.Context, resp *http.Response, roundTripErr error, doneRetryAttempts int,
) (needRetry bool, err error)

DefaultCheckRetry determines whether the request should be retried. Retry policy:

  • Network/transport error: retry if error is temporary (see CheckErrorIsTemporary).
  • HTTP 429 Too Many Requests: always retry regardless of method. This status is usually a signal from the server that the request was rejected before it was processed due to rate limiting.
  • HTTP 5xx and 408: retry only for clearly idempotent methods (GET/HEAD/OPTIONS) or when the request context carries the idempotent hint set via NewContextWithIdempotentHint(ctx, true).

func GetIdempotentHintFromContext added in v1.23.0

func GetIdempotentHintFromContext(ctx context.Context) bool

GetIdempotentHintFromContext extracts the "idempotent request" hint from context. Returns false when the key is not present. See NewContextWithIdempotentHint for details.

func GetRequestTypeFromContext added in v1.7.0

func GetRequestTypeFromContext(ctx context.Context) string

GetRequestTypeFromContext extracts request type from the context.

func MustNew added in v1.6.0

func MustNew(cfg *Config) *http.Client

MustNew wraps delegate transports with logging, rate limiting, retryable, request id and panics if any error occurs.

func MustNewWithOpts added in v1.6.0

func MustNewWithOpts(cfg *Config, opts Opts) *http.Client

MustNewWithOpts wraps delegate transports with options logging, metrics, rate limiting, retryable, user agent, request id and panics if any error occurs.

func New added in v1.6.0

func New(cfg *Config) (*http.Client, error)

New wraps delegate transports with logging, rate limiting, retryable, request id and returns an error if any occurs.

func NewContextWithIdempotentHint added in v1.23.0

func NewContextWithIdempotentHint(ctx context.Context, isIdempotent bool) context.Context

NewContextWithIdempotentHint returns a derived context that carries an "idempotent request" hint. When set to true, the request is considered idempotent even if it's not a GET/HEAD/OPTIONS request. Currently, this hint is used by RetryableRoundTripper in the DefaultCheckRetry function to decide whether it's safe to retry unsafe methods like POST and PATCH on retriable server errors.

func NewContextWithRequestType added in v1.7.0

func NewContextWithRequestType(ctx context.Context, requestType string) context.Context

NewContextWithRequestType creates a new context with request type.

func NewLoggingRoundTripper added in v1.6.0

func NewLoggingRoundTripper(delegate http.RoundTripper) http.RoundTripper

NewLoggingRoundTripper creates an HTTP transport that log requests.

func NewLoggingRoundTripperWithOpts added in v1.6.0

func NewLoggingRoundTripperWithOpts(
	delegate http.RoundTripper, opts LoggingRoundTripperOpts,
) http.RoundTripper

NewLoggingRoundTripperWithOpts creates an HTTP transport that log requests with options.

func NewMetricsRoundTripper added in v1.6.0

func NewMetricsRoundTripper(delegate http.RoundTripper, collector MetricsCollector) http.RoundTripper

NewMetricsRoundTripper creates an HTTP transport that measures requests done.

func NewMetricsRoundTripperWithOpts added in v1.6.0

func NewMetricsRoundTripperWithOpts(
	delegate http.RoundTripper,
	collector MetricsCollector,
	opts MetricsRoundTripperOpts,
) http.RoundTripper

NewMetricsRoundTripperWithOpts creates an HTTP transport that measures requests done.

func NewRequestIDRoundTripper added in v1.6.0

func NewRequestIDRoundTripper(delegate http.RoundTripper) http.RoundTripper

NewRequestIDRoundTripper creates an HTTP transport with X-Request-ID header support.

func NewRequestIDRoundTripperWithOpts added in v1.6.0

func NewRequestIDRoundTripperWithOpts(delegate http.RoundTripper, opts RequestIDRoundTripperOpts) http.RoundTripper

NewRequestIDRoundTripperWithOpts creates an HTTP transport with X-Request-ID header support with options.

func NewWithOpts added in v1.6.0

func NewWithOpts(cfg *Config, opts Opts) (*http.Client, error)

NewWithOpts wraps delegate transports with options logging, metrics, rate limiting, retryable, user agent, request id and returns an error if any occurs.

Types

type AuthBearerRoundTripper

type AuthBearerRoundTripper struct {
	Delegate     http.RoundTripper
	AuthProvider AuthProvider
	// contains filtered or unexported fields
}

AuthBearerRoundTripper implements http.RoundTripper interface and sets the "Authorization: Bearer <token>" HTTP header in outgoing requests using the provided AuthProvider. Notes and caveats:

  • If the request already contains the Authorization header, this RoundTripper delegates to the underlying transport without any token refresh or retry logic.
  • On an authentication failure (by default HTTP 401), it will optionally invalidate the provider cache (if the provider implements AuthProviderInvalidator), obtain a new token, and retry the request once with the new token.
  • Only a single retry attempt is performed by design. There is no backoff or multi‑step retry here. focuses strictly on auth-related retries).
  • For requests with bodies, the component attempts to make the body rewindable to safely retry once (prefers Request.GetBody, then io.ReadSeeker, and as a last resort buffers the entire body in memory). For very large payloads, callers should prefer providing Request.GetBody or a seekable reader to avoid high memory usage.
  • MinInvalidationInterval throttles how often the provider cache invalidation can happen to prevent hammering token endpoints under bursts of 401s; the throttling is maintained per RoundTripper instance.

func NewAuthBearerRoundTripper

func NewAuthBearerRoundTripper(delegate http.RoundTripper, authProvider AuthProvider) *AuthBearerRoundTripper

NewAuthBearerRoundTripper creates a new AuthBearerRoundTripper.

func NewAuthBearerRoundTripperWithOpts

func NewAuthBearerRoundTripperWithOpts(
	delegate http.RoundTripper, authProvider AuthProvider, opts AuthBearerRoundTripperOpts,
) *AuthBearerRoundTripper

NewAuthBearerRoundTripperWithOpts creates a new AuthBearerRoundTripper with options.

func (*AuthBearerRoundTripper) RoundTrip

func (rt *AuthBearerRoundTripper) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip executes a single HTTP transaction, returning a Response for the provided Request. Behavior overview:

  • If Authorization header is already present, the request is passed through without any retry/refresh.
  • Otherwise, it adds a bearer token from AuthProvider and sends the request.
  • If ShouldRefreshTokenAndRetry returns true (default: on 401), it may invalidate provider cache (throttled by MinInvalidationInterval), fetch a new token, safely rewind the request body if needed, and retry once.
  • On transport errors (no response), no auth retry is attempted here.
  • If token refresh fails or token stays the same after invalidation, the original response is returned unchanged.

type AuthBearerRoundTripperError

type AuthBearerRoundTripperError struct {
	Inner error
}

AuthBearerRoundTripperError is returned in RoundTrip method of AuthBearerRoundTripper when the original request cannot be potentially retried.

func (*AuthBearerRoundTripperError) Error

func (*AuthBearerRoundTripperError) Unwrap

func (e *AuthBearerRoundTripperError) Unwrap() error

Unwrap returns the next error in the error chain.

type AuthBearerRoundTripperOpts

type AuthBearerRoundTripperOpts struct {
	// TokenScope is an optional list of scopes to pass to AuthProvider.GetToken.
	// By default, no scopes are passed.
	TokenScope []string

	// MinInvalidationInterval determines the minimum time between cache invalidation attempts
	// for the same AuthProvider. This prevents excessive calls to token endpoints.
	// By default, DefaultAuthProviderMinInvalidationInterval const is used.
	MinInvalidationInterval time.Duration

	// ShouldRefreshTokenAndRetry is called after the first attempt and decides whether a token refresh attempt
	// should be performed and the request should be retried once more (cache invalidation + token refresh + single retry).
	// By default, it returns true only for HTTP 401 responses.
	// If you need to retry only for specific endpoints or HTTP methods, you can provide your own implementation
	// and use http.Response.Request field to make the decision.
	ShouldRefreshTokenAndRetry func(ctx context.Context, resp *http.Response) bool

	// LoggerProvider is a function that provides a context-specific logger.
	// One of the typical use cases is to use an auth client in the context of a request handler,
	// where the logger should produce logs with request-specific information (e.g., request ID).
	LoggerProvider func(ctx context.Context) log.FieldLogger
}

AuthBearerRoundTripperOpts is options for AuthBearerRoundTripper.

type AuthProvider

type AuthProvider interface {
	GetToken(ctx context.Context, scope ...string) (string, error)
}

AuthProvider provides auth information used for bearer authorization.

type AuthProviderFunc added in v1.18.0

type AuthProviderFunc func(ctx context.Context, scope ...string) (string, error)

AuthProviderFunc allows to define get token logic in functional way.

func (AuthProviderFunc) GetToken added in v1.18.0

func (f AuthProviderFunc) GetToken(ctx context.Context, scope ...string) (string, error)

type AuthProviderInvalidator added in v1.23.0

type AuthProviderInvalidator interface {
	Invalidate()
}

AuthProviderInvalidator is implemented by AuthProviders that support cache invalidation.

type CheckRetryFunc

type CheckRetryFunc func(ctx context.Context, resp *http.Response, roundTripErr error, doneRetryAttempts int) (bool, error)

CheckRetryFunc is a function that is called right after RoundTrip() method and determines if the next retry attempt is needed.

type Config added in v1.6.0

type Config struct {
	// Retries is a configuration for HTTP client retries policy.
	Retries RetriesConfig `mapstructure:"retries" yaml:"retries" json:"retries"`

	// RateLimits is a configuration for HTTP client rate limits.
	RateLimits RateLimitsConfig `mapstructure:"rateLimits" yaml:"rateLimits" json:"rateLimits"`

	// Log is a configuration for HTTP client logs.
	Log LogConfig `mapstructure:"log" yaml:"log" json:"log"`

	// Metrics is a configuration for HTTP client metrics.
	Metrics MetricsConfig `mapstructure:"metrics" yaml:"metrics" json:"metrics"`

	// Timeout is the maximum time to wait for a request to be made.
	Timeout config.TimeDuration `mapstructure:"timeout" yaml:"timeout" json:"timeout"`
	// contains filtered or unexported fields
}

Config represents options for HTTP client configuration. Configuration can be loaded in different formats (YAML, JSON) using config.Loader, viper, or with json.Unmarshal/yaml.Unmarshal functions directly.

func NewConfig added in v1.6.0

func NewConfig(options ...ConfigOption) *Config

NewConfig creates a new instance of the Config.

func NewConfigWithKeyPrefix added in v1.6.0

func NewConfigWithKeyPrefix(keyPrefix string) *Config

NewConfigWithKeyPrefix creates a new instance of the Config with a key prefix. This prefix will be used by config.Loader. Deprecated: use NewConfig with WithKeyPrefix instead.

func NewDefaultConfig added in v1.11.0

func NewDefaultConfig(options ...ConfigOption) *Config

NewDefaultConfig creates a new instance of the Config with default values.

func (*Config) KeyPrefix added in v1.6.0

func (c *Config) KeyPrefix() string

KeyPrefix returns a key prefix with which all configuration parameters should be presented. Implements config.KeyPrefixProvider interface.

func (*Config) Set added in v1.6.0

func (c *Config) Set(dp config.DataProvider) error

Set sets http client configuration based on the passed config.DataProvider. Implements config.Config interface.

func (*Config) SetProviderDefaults added in v1.6.0

func (c *Config) SetProviderDefaults(dp config.DataProvider)

SetProviderDefaults is part of config interface implementation. Implements config.Config interface.

type ConfigOption added in v1.11.0

type ConfigOption func(*configOptions)

ConfigOption is a type for functional options for the Config.

func WithKeyPrefix added in v1.11.0

func WithKeyPrefix(keyPrefix string) ConfigOption

WithKeyPrefix returns a ConfigOption that sets a key prefix for parsing configuration parameters. This prefix will be used by config.Loader.

type ConstantBackoffConfig added in v1.6.0

type ConstantBackoffConfig struct {
	// Interval is the interval for constant backoff.
	Interval config.TimeDuration `mapstructure:"interval" yaml:"interval" json:"interval"`
}

ConstantBackoffConfig represents configuration options for constant backoff.

type ExponentialBackoffConfig added in v1.6.0

type ExponentialBackoffConfig struct {
	// InitialInterval is the initial interval for exponential backoff.
	InitialInterval config.TimeDuration `mapstructure:"initialInterval" yaml:"initialInterval" json:"initialInterval"`

	// Multiplier is the multiplier for exponential backoff.
	Multiplier float64 `mapstructure:"multiplier" yaml:"multiplier" json:"multiplier"`
}

ExponentialBackoffConfig represents configuration options for exponential backoff.

type LogConfig added in v1.6.0

type LogConfig struct {
	// Enabled is a flag that enables logging.
	Enabled bool `mapstructure:"enabled" yaml:"enabled" json:"enabled"`

	// SlowRequestThreshold is a threshold for slow requests.
	SlowRequestThreshold config.TimeDuration `mapstructure:"slowRequestThreshold" yaml:"slowRequestThreshold" json:"slowRequestThreshold"`

	// Mode of logging: [all, failed]. 'all' by default.
	Mode LoggingMode `mapstructure:"mode" yaml:"mode" json:"mode"`
}

LogConfig represents configuration options for HTTP client logs.

func (*LogConfig) TransportOpts added in v1.6.0

func (c *LogConfig) TransportOpts() LoggingRoundTripperOpts

TransportOpts returns transport options.

type LoggingMode added in v1.6.0

type LoggingMode string

LoggingMode represents a mode of logging.

const (
	LoggingModeAll    LoggingMode = "all"
	LoggingModeFailed LoggingMode = "failed"
)

func (LoggingMode) IsValid added in v1.6.0

func (lm LoggingMode) IsValid() bool

IsValid checks if the logger mode is valid.

type LoggingRoundTripper added in v1.6.0

type LoggingRoundTripper struct {
	// Delegate is the next RoundTripper in the chain.
	Delegate http.RoundTripper

	// ClientType represents a type of client, it's a service component reference. e.g. 'auth-service'
	ClientType string

	// Mode of logging: [all, failed]. 'all' by default.
	Mode LoggingMode

	// SlowRequestThreshold is a threshold for slow requests.
	SlowRequestThreshold time.Duration

	// LoggerProvider is a function that provides a context-specific logger.
	// middleware.GetLoggerFromContext is used by default.
	LoggerProvider func(ctx context.Context) log.FieldLogger
}

LoggingRoundTripper implements http.RoundTripper for logging requests.

func (*LoggingRoundTripper) RoundTrip added in v1.6.0

func (rt *LoggingRoundTripper) RoundTrip(r *http.Request) (*http.Response, error)

RoundTrip adds logging capabilities to the HTTP transport. It logs HTTP requests and responses, tracking execution time and success/failure status.

Logging behavior: - If logging is disabled (no logger), the request is simply forwarded. - If the request completes successfully and is not slow, it may be ignored based on logging mode. - Slow requests, failed requests, and configured conditions determine log level (Info, Warn, Error).

Additional logged information:

  • HTTP method and URL.
  • Response status code and elapsed time.
  • Client type if available.
  • X-Request-ID from request headers.
  • Request type extracted from the context.
  • The request duration in milliseconds is accumulated in LoggingParams, allowing tracking of the total time spent on external requests when the handler execution is finalized. Note that this value is accurate only for sequential requests.

type LoggingRoundTripperOpts added in v1.6.0

type LoggingRoundTripperOpts struct {
	// ClientType represents a type of client, it's a service component reference. e.g. 'auth-service'
	ClientType string

	// Mode of logging: [all, failed]. 'all' by default.
	Mode LoggingMode

	// SlowRequestThreshold is a threshold for slow requests.
	SlowRequestThreshold time.Duration

	// LoggerProvider is a function that provides a context-specific logger.
	// middleware.GetLoggerFromContext is used by default.
	LoggerProvider func(ctx context.Context) log.FieldLogger
}

LoggingRoundTripperOpts represents an options for LoggingRoundTripper.

type MetricsCollector added in v1.6.0

type MetricsCollector interface {
	// RequestDuration observes the duration of the request and the status code.
	RequestDuration(clientType, remoteAddress, summary, status, requestType string, duration float64)
}

MetricsCollector is an interface for collecting metrics for client requests.

type MetricsConfig added in v1.6.0

type MetricsConfig struct {
	// Enabled is a flag that enables metrics.
	Enabled bool `mapstructure:"enabled" yaml:"enabled" json:"enabled"`
}

MetricsConfig represents configuration options for HTTP client logs.

type MetricsRoundTripper added in v1.6.0

type MetricsRoundTripper struct {
	// Delegate is the next RoundTripper in the chain.
	Delegate http.RoundTripper

	// ClientType represents a type of client, it's a service component reference. e.g. 'auth-service'
	ClientType string

	// MetricsCollector is a metrics collector.
	MetricsCollector MetricsCollector

	// ClassifyRequest does request classification, producing non-parameterized summary for given request.
	ClassifyRequest func(r *http.Request, clientType string) string
}

MetricsRoundTripper is an HTTP transport that measures requests done.

func (*MetricsRoundTripper) RoundTrip added in v1.6.0

func (rt *MetricsRoundTripper) RoundTrip(r *http.Request) (*http.Response, error)

RoundTrip measures external requests done.

type MetricsRoundTripperOpts added in v1.6.0

type MetricsRoundTripperOpts struct {
	// ClientType represents a type of client, it's a service component reference. e.g. 'auth-service'
	ClientType string

	// ClassifyRequest does request classification, producing non-parameterized summary for given request.
	ClassifyRequest func(r *http.Request, clientType string) string
}

MetricsRoundTripperOpts is an HTTP transport that measures requests done.

type Opts added in v1.6.0

type Opts struct {
	// UserAgent is a user agent string.
	UserAgent string

	// AuthProvider provides auth information used for bearer authorization.
	AuthProvider AuthProvider

	// ClientType represents a type of client, it's a service component reference. e.g. 'auth-service'.
	ClientType string

	// Delegate is the next RoundTripper in the chain.
	Delegate http.RoundTripper

	// LoggerProvider is a function that provides a context-specific logger.
	LoggerProvider func(ctx context.Context) log.FieldLogger

	// RequestIDProvider is a function that provides a request ID.
	RequestIDProvider func(ctx context.Context) string

	// MetricsCollector is a metrics collector.
	MetricsCollector MetricsCollector

	// ClassifyRequest does request classification, producing non-parameterized summary in metrics round tripper.
	ClassifyRequest func(r *http.Request, clientType string) string
}

Opts provides options for NewWithOpts and MustWithOpts functions.

type PrometheusMetricsCollector added in v1.6.0

type PrometheusMetricsCollector struct {
	// Durations is a histogram of the http client requests durations.
	Durations *prometheus.HistogramVec
}

PrometheusMetricsCollector is a Prometheus metrics collector.

func NewPrometheusMetricsCollector added in v1.6.0

func NewPrometheusMetricsCollector(namespace string) *PrometheusMetricsCollector

NewPrometheusMetricsCollector creates a new Prometheus metrics collector.

func (*PrometheusMetricsCollector) MustRegister added in v1.6.0

func (p *PrometheusMetricsCollector) MustRegister()

MustRegister registers the Prometheus metrics.

func (*PrometheusMetricsCollector) RequestDuration added in v1.6.0

func (p *PrometheusMetricsCollector) RequestDuration(
	clientType, host, summary, status, requestType string, duration float64,
)

RequestDuration observes the duration of the request and the status code.

func (*PrometheusMetricsCollector) Unregister added in v1.6.0

func (p *PrometheusMetricsCollector) Unregister()

Unregister the Prometheus metrics.

type RateLimitingRoundTripper

type RateLimitingRoundTripper struct {
	Delegate http.RoundTripper

	RateLimit   int
	Burst       int
	WaitTimeout time.Duration
	Adaptation  RateLimitingRoundTripperAdaptation
	// contains filtered or unexported fields
}

RateLimitingRoundTripper wraps implementing http.RoundTripper interface object and provides adaptive (can use limit from response's HTTP header) rate limiting mechanism for outgoing requests.

func NewRateLimitingRoundTripper

func NewRateLimitingRoundTripper(delegate http.RoundTripper, rateLimit int) (*RateLimitingRoundTripper, error)

NewRateLimitingRoundTripper creates a new RateLimitingRoundTripper with specified rate limit.

Example

ExampleNewRateLimitingRoundTripper demonstrates the use of RateLimitingRoundTripper with default parameters.

Add "// Output:" in the end of the function and run:

$ go test ./httpclient -v -run ExampleNewRateLimitingRoundTripper

Output will be like:

[Req#1] 204 (0ms)
[Req#2] 204 (502ms)
[Req#3] 204 (497ms)
[Req#4] 204 (500ms)
[Req#5] 204 (503ms)
// Note: error handling is intentionally omitted so as not to overcomplicate the example.
// It is strictly necessary to handle all errors in real code.

server := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
	rw.WriteHeader(http.StatusNoContent)
}))

// Let's make transport that may do maximum 2 requests per second.
tr, _ := NewRateLimitingRoundTripper(http.DefaultTransport, 2)
httpClient := &http.Client{Transport: tr}

start := time.Now()
prev := time.Now()
for i := 0; i < 5; i++ {
	resp, _ := httpClient.Get(server.URL)
	_ = resp.Body.Close()
	now := time.Now()
	_, _ = fmt.Fprintf(os.Stderr, "[Req#%d] %d (%dms)\n", i+1, resp.StatusCode, now.Sub(prev).Milliseconds())
	prev = now
}
delta := time.Since(start) - time.Second*2
if delta > time.Millisecond*20 {
	fmt.Println("Total time is much greater than 2s")
} else {
	fmt.Println("Total time is about 2s")
}
Output:

Total time is about 2s

func NewRateLimitingRoundTripperWithOpts

func NewRateLimitingRoundTripperWithOpts(
	delegate http.RoundTripper, rateLimit int, opts RateLimitingRoundTripperOpts,
) (*RateLimitingRoundTripper, error)

NewRateLimitingRoundTripperWithOpts creates a new RateLimitingRoundTripper with specified rate limit and options. For options that are not presented, the default values will be used.

Example

ExampleNewRateLimitingRoundTripperWithOpts demonstrates the use of RateLimitingRoundTripper.

// Note: error handling is intentionally omitted so as not to overcomplicate the example.
// It is strictly necessary to handle all errors in real code.

server := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
	rw.WriteHeader(http.StatusNoContent)
}))

// Let's make transport that may do maximum 2 requests per second.
tr, _ := NewRateLimitingRoundTripperWithOpts(http.DefaultTransport, 2, RateLimitingRoundTripperOpts{
	WaitTimeout: time.Millisecond * 100, // Wait maximum 100ms.
})
httpClient := &http.Client{Transport: tr}

for i := 0; i < 2; i++ {
	resp, err := httpClient.Get(server.URL)
	if err != nil {
		var waitErr *RateLimitingWaitError
		if errors.As(err, &waitErr) {
			fmt.Printf("trying to do too many requests")
		}
		continue
	}
	_ = resp.Body.Close()
}
Output:

trying to do too many requests

func (*RateLimitingRoundTripper) RoundTrip

func (rt *RateLimitingRoundTripper) RoundTrip(r *http.Request) (*http.Response, error)

RoundTrip executes a single HTTP transaction, returning a Response for the provided Request.

type RateLimitingRoundTripperAdaptation

type RateLimitingRoundTripperAdaptation struct {
	ResponseHeaderName string
	SlackPercent       int
}

RateLimitingRoundTripperAdaptation represents a params to adapt rate limiting in accordance with value in response.

type RateLimitingRoundTripperOpts

type RateLimitingRoundTripperOpts struct {
	Burst       int
	WaitTimeout time.Duration
	Adaptation  RateLimitingRoundTripperAdaptation
}

RateLimitingRoundTripperOpts represents an options for RateLimitingRoundTripper.

type RateLimitingWaitError

type RateLimitingWaitError struct {
	Inner error
}

RateLimitingWaitError is returned in RoundTrip method of RateLimitingRoundTripper when rate limit is exceeded.

func (*RateLimitingWaitError) Error

func (e *RateLimitingWaitError) Error() string

func (*RateLimitingWaitError) Unwrap

func (e *RateLimitingWaitError) Unwrap() error

Unwrap returns the next error in the error chain.

type RateLimitsConfig added in v1.6.0

type RateLimitsConfig struct {
	// Enabled is a flag that enables rate limiting.
	Enabled bool `mapstructure:"enabled" yaml:"enabled" json:"enabled"`

	// Limit is the maximum number of requests that can be made.
	Limit int `mapstructure:"limit" yaml:"limit" json:"limit"`

	// Burst allow temporary spikes in request rate.
	Burst int `mapstructure:"burst" yaml:"burst" json:"burst"`

	// WaitTimeout is the maximum time to wait for a request to be made.
	WaitTimeout config.TimeDuration `mapstructure:"waitTimeout" yaml:"waitTimeout" json:"waitTimeout"`
}

RateLimitsConfig represents configuration options for HTTP client rate limits.

func (*RateLimitsConfig) TransportOpts added in v1.6.0

TransportOpts returns transport options.

type RequestIDRoundTripper added in v1.6.0

type RequestIDRoundTripper struct {
	// Delegate is the next RoundTripper in the chain.
	Delegate http.RoundTripper

	// Opts are the options for the request ID round tripper.
	Opts RequestIDRoundTripperOpts
}

RequestIDRoundTripper for X-Request-ID header to the request.

func (*RequestIDRoundTripper) RoundTrip added in v1.6.0

func (rt *RequestIDRoundTripper) RoundTrip(r *http.Request) (*http.Response, error)

RoundTrip adds X-Request-ID header to the request.

type RequestIDRoundTripperOpts added in v1.6.0

type RequestIDRoundTripperOpts struct {
	// RequestIDProvider is a function that provides a request ID.
	// middleware.GetRequestIDFromContext is used by default.
	RequestIDProvider func(ctx context.Context) string
}

RequestIDRoundTripperOpts for X-Request-ID header to the request options.

type RetriesConfig added in v1.6.0

type RetriesConfig struct {
	// Enabled is a flag that enables retries.
	Enabled bool `mapstructure:"enabled" yaml:"enabled" json:"enabled"`

	// MaxAttempts is the maximum number of attempts to retry the request.
	MaxAttempts int `mapstructure:"maxAttempts" yaml:"maxAttempts" json:"maxAttempts"`

	// Policy of a retry: [exponential, constant].
	Policy RetryPolicy `mapstructure:"policy" yaml:"policy" json:"policy"`

	// ExponentialBackoff is the configuration for exponential backoff.
	ExponentialBackoff ExponentialBackoffConfig `mapstructure:"exponentialBackoff" yaml:"exponentialBackoff" json:"exponentialBackoff"`

	// ConstantBackoff is the configuration for constant backoff.
	ConstantBackoff ConstantBackoffConfig `mapstructure:"constantBackoff" yaml:"constantBackoff" json:"constantBackoff"`
}

RetriesConfig represents configuration options for HTTP client retries policy.

func (*RetriesConfig) GetPolicy added in v1.6.0

func (c *RetriesConfig) GetPolicy() retry.Policy

GetPolicy returns a retry policy based on strategy or nil if none is provided.

func (*RetriesConfig) TransportOpts added in v1.6.0

func (c *RetriesConfig) TransportOpts() RetryableRoundTripperOpts

TransportOpts returns transport options.

type RetryPolicy added in v1.6.0

type RetryPolicy string

RetryPolicy represents a retry policy strategy.

const (
	DefaultTimeout = time.Minute

	// RetryPolicyExponential is a policy for exponential retries.
	RetryPolicyExponential RetryPolicy = "exponential"

	// RetryPolicyConstant is a policy for constant retries.
	RetryPolicyConstant RetryPolicy = "constant"
)

type RetryableRoundTripper

type RetryableRoundTripper struct {
	// Delegate is an object that implements http.RoundTripper interface
	// and is used for sending HTTP requests under the hood.
	Delegate http.RoundTripper

	// Logger is used for logging.
	// Deprecated: use LoggerProvider instead.
	Logger log.FieldLogger

	// LoggerProvider is a function that provides a context-specific logger.
	// One of the typical use cases is to use a retryable client in the context of a request handler,
	// where the logger should produce logs with request-specific information (e.g., request ID).
	LoggerProvider func(ctx context.Context) log.FieldLogger

	// MaxRetryAttempts determines how many maximum retry attempts can be done.
	// The total number of sending HTTP request may be MaxRetryAttempts + 1 (the first request is not a retry attempt).
	// If its value is UnlimitedRetryAttempts, it's supposed that retry mechanism will be stopped by BackoffPolicy.
	// By default, DefaultMaxRetryAttempts const is used.
	MaxRetryAttempts int

	// CheckRetry is called right after RoundTrip() method and determines if the next retry attempt is needed.
	// By default, DefaultCheckRetry function is used.
	CheckRetry CheckRetryFunc

	// IgnoreRetryAfter determines if Retry-After HTTP header of the response is parsed and
	// used as a wait time before doing the next retry attempt.
	// If it's true or response doesn't contain Retry-After HTTP header, BackoffPolicy will be used for computing delay.
	IgnoreRetryAfter bool

	// BackoffPolicy is used for computing wait time before doing the next retry attempt
	// when the given response doesn't contain Retry-After HTTP header or IgnoreRetryAfter is true.
	// By default, DefaultBackoffPolicy is used.
	BackoffPolicy retry.Policy
}

RetryableRoundTripper wraps an object that implements http.RoundTripper interface and provides a retrying mechanism for HTTP requests.

func NewRetryableRoundTripper

func NewRetryableRoundTripper(delegate http.RoundTripper) (*RetryableRoundTripper, error)

NewRetryableRoundTripper returns a new instance of RetryableRoundTripper.

Example

ExampleNewRetryableRoundTripper demonstrates the use of RetryableRoundTripper with default parameters.

To execute this example: 1) Add "// Output: Got 200 after 10 retry attempts" in the end of function. 2) Run:

$ go test ./httpclient -v -run ExampleNewRetryableRoundTripper

Stderr will contain something like this:

[Req#1] wait time: 0.001s
[Req#2] wait time: 1.109s
[Req#3] wait time: 2.882s
[Req#4] wait time: 4.661s
[Req#5] wait time: 7.507s
[Req#6] wait time: 14.797s
[Req#7] wait time: 37.984s
[Req#8] wait time: 33.942s
[Req#9] wait time: 39.394s
[Req#10] wait time: 35.820s
[Req#11] wait time: 48.060s
// Note: error handling is intentionally omitted so as not to overcomplicate the example.
// It is strictly necessary to handle all errors in real code.

reqTimes := make(chan time.Time, DefaultMaxRetryAttempts+1)

server := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
	reqTimes <- time.Now()
	if n, err := strconv.Atoi(r.Header.Get(RetryAttemptNumberHeader)); err == nil && n == DefaultMaxRetryAttempts {
		rw.WriteHeader(http.StatusOK)
		_, _ = rw.Write([]byte("ok, you win..."))
		return
	}
	rw.WriteHeader(http.StatusServiceUnavailable)
}))

prevTime := time.Now()

// Create RetryableRoundTripper with default params:
//	+ maximum retry attempts = 10 (DefaultMaxRetryAttempts)
//	+ respect Retry-After response HTTP header
//	+ exponential backoff policy (DefaultBackoffPolicy, multiplier = 2, initial interval = 1s)
tr, _ := NewRetryableRoundTripper(http.DefaultTransport)
httpClient := &http.Client{Transport: tr}

resp, err := httpClient.Get(server.URL)
if err != nil {
	log.Fatal(err)
}
_ = resp.Body.Close()
close(reqTimes)

reqsCount := 0
for reqTime := range reqTimes {
	reqsCount++
	_, _ = fmt.Fprintf(os.Stderr, "[Req#%d] wait time: %.3fs\n", reqsCount, reqTime.Sub(prevTime).Seconds())
	prevTime = reqTime
}
doneRetryAttempts := reqsCount - 1
fmt.Printf("Got %d after %d retry attempts", resp.StatusCode, doneRetryAttempts)

func NewRetryableRoundTripperWithOpts

func NewRetryableRoundTripperWithOpts(
	delegate http.RoundTripper, opts RetryableRoundTripperOpts,
) (*RetryableRoundTripper, error)

NewRetryableRoundTripperWithOpts creates a new instance of RateLimitingRoundTripper with specified options.

func (*RetryableRoundTripper) RoundTrip

func (rt *RetryableRoundTripper) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip performs request with retry logic.

type RetryableRoundTripperError

type RetryableRoundTripperError struct {
	Inner error
}

RetryableRoundTripperError is returned in RoundTrip method of RetryableRoundTripper when the original request cannot be potentially retried.

func (*RetryableRoundTripperError) Error

func (*RetryableRoundTripperError) Unwrap

func (e *RetryableRoundTripperError) Unwrap() error

Unwrap returns the next error in the error chain.

type RetryableRoundTripperOpts

type RetryableRoundTripperOpts struct {
	// Logger is used for logging.
	// Deprecated: use LoggerProvider instead.
	Logger log.FieldLogger

	// LoggerProvider is a function that provides a context-specific logger.
	// One of the typical use cases is to use a retryable client in the context of a request handler,
	// where the logger should produce logs with request-specific information (e.g., request ID).
	LoggerProvider func(ctx context.Context) log.FieldLogger

	// MaxRetryAttempts determines how many maximum retry attempts can be done.
	// The total number of sending HTTP request may be MaxRetryAttempts + 1 (the first request is not a retry attempt).
	// If its value is UnlimitedRetryAttempts, it's supposed that retry mechanism will be stopped by BackoffPolicy.
	// By default, DefaultMaxRetryAttempts const is used.
	MaxRetryAttempts int

	// CheckRetryFunc is called right after RoundTrip() method and determines if the next retry attempt is needed.
	// By default, DefaultCheckRetry function is used.
	CheckRetryFunc CheckRetryFunc

	// IgnoreRetryAfter determines if Retry-After HTTP header of the response is parsed and
	// used as a wait time before doing the next retry attempt.
	// If it's true or response doesn't contain Retry-After HTTP header, BackoffPolicy will be used for computing delay.
	IgnoreRetryAfter bool

	// BackoffPolicy is used for computing wait time before doing the next retry attempt
	// when the given response doesn't contain Retry-After HTTP header or IgnoreRetryAfter is true.
	// By default, DefaultBackoffPolicy is used.
	BackoffPolicy retry.Policy
}

RetryableRoundTripperOpts represents an options for RetryableRoundTripper.

type UserAgentRoundTripper

type UserAgentRoundTripper struct {
	Delegate       http.RoundTripper
	UserAgent      string
	UpdateStrategy UserAgentUpdateStrategy
}

UserAgentRoundTripper implements http.RoundTripper interface and sets User-Agent HTTP header in all outgoing requests.

func NewUserAgentRoundTripper

func NewUserAgentRoundTripper(delegate http.RoundTripper, userAgent string) *UserAgentRoundTripper

NewUserAgentRoundTripper creates a new UserAgentRoundTripper.

func NewUserAgentRoundTripperWithOpts

func NewUserAgentRoundTripperWithOpts(
	delegate http.RoundTripper, userAgent string, opts UserAgentRoundTripperOpts,
) *UserAgentRoundTripper

NewUserAgentRoundTripperWithOpts creates a new UserAgentRoundTripper with specified options.

func (*UserAgentRoundTripper) RoundTrip

func (rt *UserAgentRoundTripper) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip executes a single HTTP transaction, returning a Response for the provided Request.

type UserAgentRoundTripperOpts

type UserAgentRoundTripperOpts struct {
	UpdateStrategy UserAgentUpdateStrategy
}

UserAgentRoundTripperOpts represents an options for UserAgentRoundTripper.

type UserAgentUpdateStrategy

type UserAgentUpdateStrategy int

UserAgentUpdateStrategy represents a strategy for updating User-Agent HTTP header.

const (
	UserAgentUpdateStrategySetIfEmpty UserAgentUpdateStrategy = iota
	UserAgentUpdateStrategyAppend
	UserAgentUpdateStrategyPrepend
)

User-Agent update strategies.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL