httpclient

package
v2.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 20, 2026 License: MIT Imports: 14 Imported by: 0

README

httpclient

Package httpclient provides an HTTP client that transparently adds distributed tracing and structured logging to every outgoing request.

Quick start

client := httpclient.NewWithConfig(&httpclient.Config{
    Tracer:  a.Tracer(),
    Logger:  a.Logger(),
    Timeout: 10 * time.Second,
})

resp, err := client.Get(ctx, "https://api.example.com/users")

Dependency injection

Client implements the Doer interface:

type Doer interface {
    Do(req *http.Request) (*http.Response, error)
}

Accept Doer in constructors so consumers can inject test doubles without needing httptest servers:

type Service struct {
    http httpclient.Doer
}

Configuration

Field Default Description
Tracer nil (disabled) Creates client spans and injects W3C traceparent headers
Logger nil (disabled) Emits structured log lines per request
Transport sensible defaults (5s dial, 5s TLS, 90s idle, 10 idle conns/host) Underlying http.RoundTripper for custom TLS, proxies, etc.
Timeout 30s Per-request timeout. Set to -1 to disable

Instrumentation

  • Tracing: creates a client span named "METHOD host" with OTel semantic convention v1.21+ attributes (http.request.method, url.full, server.address, http.response.status_code), and injects W3C traceparent/tracestate headers into outgoing requests.
  • Logging: emits a structured log line per request with method, URL, status, duration, and attempt number (when retrying). Uses ErrorContext for network errors.

Retry

DoWithRetry automatically retries on configurable status codes and optionally on transient network errors:

resp, err := client.DoWithRetry(req, &httpclient.RetryConfig{
    MaxAttempts:     3,
    RetryableStatus: []int{429, 503},         // default
    RetryOnError:    true,                     // retry DNS/connection failures
    BaseDelay:       2 * time.Second,          // fallback when no Retry-After
    MaxDelay:        60 * time.Second,         // caps any Retry-After value
})
Field Default Description
MaxAttempts 3 Total attempts including the first
RetryableStatus [429, 503] HTTP status codes that trigger retry
RetryOnError false Retry on transient network errors
BaseDelay 1s Fallback delay when no Retry-After header
MaxDelay 60s Upper bound on any retry delay

When no Retry-After header is present, delay increases exponentially with 25% jitter: BaseDelay * 2^(attempt-1) + jitter, capped at MaxDelay. Retry-After headers override the computed delay when present. Each attempt is logged with its number for debugging.

MaxAttempts: 1 is equivalent to a single Do call with no retry overhead. Requests with a non-rewindable body (GetBody == nil) are rejected with an error when MaxAttempts > 1 to prevent silent data corruption.

Custom transport

Layer the instrumented transport on top of a custom http.RoundTripper:

client := httpclient.NewWithConfig(&httpclient.Config{
    Tracer:    a.Tracer(),
    Transport: &http.Transport{TLSClientConfig: tlsCfg},
})

Documentation

Overview

Package httpclient provides an HTTP client that transparently adds distributed tracing and structured logging to every outgoing request.

Tracing and logging are implemented as a custom http.RoundTripper so they apply to all requests regardless of which helper method is used.

  • Tracing: creates a client span named "METHOD host", records HTTP method, URL, host, and response status as span attributes, and injects outgoing W3C traceparent / tracestate headers so downstream services can parent their spans correctly.
  • Logging: emits a structured log line after each round trip with method, URL, status, elapsed duration, and attempt number (when retrying). Uses ErrorContext for network errors and InfoContext for completed requests.

Both the tracer and logger are optional; omitting either disables that instrumentation layer.

Dependency injection

The package defines a Doer interface that Client implements. Code that makes HTTP calls should accept a Doer rather than a concrete *Client so that callers can substitute test doubles:

type Service struct {
	http httpclient.Doer
}

func NewService(http httpclient.Doer) *Service {
	return &Service{http: http}
}

Basic usage

client := httpclient.NewWithConfig(&httpclient.Config{
	Logger:  a.Logger(),
	Tracer:  a.Tracer(),
	Timeout: 10 * time.Second,
})

resp, err := client.Get(ctx, "https://api.example.com/users")
if err != nil { ... }
defer resp.Body.Close()

Retry with network error recovery

resp, err := client.DoWithRetry(req, &httpclient.RetryConfig{
	MaxAttempts:  3,
	RetryOnError: true,              // retry DNS/connection failures
	BaseDelay:    2 * time.Second,   // fallback when no Retry-After
})

Composing with a custom transport

Pass Transport to layer the instrumented transport on top of your own:

client := httpclient.NewWithConfig(&httpclient.Config{
	Tracer:    a.Tracer(),
	Logger:    a.Logger(),
	Transport: &http.Transport{TLSClientConfig: tlsCfg},
})

Testing

Use net/http/httptest.NewServer to capture outgoing requests without real network calls:

ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	w.WriteHeader(http.StatusOK)
}))
defer ts.Close()

resp, err := client.Get(ctx, ts.URL)

Alternatively, implement the Doer interface directly to build a test double without any network:

type mockDoer struct {
	resp *http.Response
	err  error
}
func (m *mockDoer) Do(req *http.Request) (*http.Response, error) {
	return m.resp, m.err
}
Example

Example shows a basic instrumented client making a GET request.

package main

import (
	"context"
	"time"

	"gitlab.com/gitlab-org/labkit/v2/httpclient"
)

func main() {
	client := httpclient.NewWithConfig(&httpclient.Config{
		// Logger: a.Logger(),
		// Tracer: a.Tracer(),
		Timeout: 10 * time.Second,
	})

	ctx := context.Background()
	resp, err := client.Get(ctx, "https://example.com/api/projects")
	if err != nil {
		panic(err)
	}
	defer resp.Body.Close()
	_ = resp
}

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewTransport

func NewTransport(cfg *Config) http.RoundTripper

NewTransport returns an http.RoundTripper that adds distributed tracing and structured logging to every request. Use this when you need to pass an instrumented transport to a third-party library that accepts *http.Client or http.RoundTripper:

transport := httpclient.NewTransport(&httpclient.Config{
    Tracer: a.Tracer(),
    Logger: a.Logger(),
})
client := &http.Client{Transport: transport}
oauthClient := oauth2.NewClient(ctx, tokenSource)
oauthClient.Transport = transport
Example

ExampleNewTransport shows how to layer the instrumented transport onto a third-party client that accepts http.RoundTripper, such as oauth2.

package main

import (
	"gitlab.com/gitlab-org/labkit/v2/httpclient"
)

func main() {
	transport := httpclient.NewTransport(&httpclient.Config{
		// Logger: a.Logger(),
		// Tracer: a.Tracer(),
	})
	// Pass to any library that accepts http.RoundTripper or *http.Client:
	//   oauthClient := oauth2.NewClient(ctx, tokenSource)
	//   oauthClient.Transport = transport
	_ = transport
}

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is an HTTP client that transparently adds distributed tracing and structured logging to every outgoing request via a custom http.RoundTripper. It implements Doer.

func New

func New() *Client

New returns a Client with default Config (30s timeout, default transport timeouts, no tracing, no logging).

func NewWithConfig

func NewWithConfig(cfg *Config) *Client

NewWithConfig returns a Client configured with cfg. A nil cfg is treated identically to an empty Config (all defaults).

func (*Client) Delete

func (c *Client) Delete(ctx context.Context, url string) (*http.Response, error)

Delete issues a DELETE request to url using ctx.

func (*Client) Do

func (c *Client) Do(req *http.Request) (*http.Response, error)

Do sends req and returns the response. The context attached to req is used to propagate trace context and to correlate log lines.

func (*Client) DoWithRetry

func (c *Client) DoWithRetry(req *http.Request, cfg *RetryConfig) (*http.Response, error)

DoWithRetry sends req and automatically retries when the server responds with a retryable status code (default: 429 Too Many Requests and 503 Service Unavailable). The Retry-After response header is honoured when present — both the integer-seconds and HTTP-date forms are supported. Retries stop when the context is cancelled, the maximum attempt count is reached, or the response status is not retryable.

When MaxAttempts is 1, DoWithRetry is equivalent to Client.Do with no retry overhead.

Requests with a body are only retried correctly when http.Request.GetBody is non-nil, which allows the body to be re-read on each attempt. The standard library sets GetBody automatically for *bytes.Buffer, *bytes.Reader, and *strings.Reader bodies. Callers that supply a plain io.Reader should set GetBody themselves or use only idempotent methods. If GetBody is nil and MaxAttempts > 1, DoWithRetry returns an error.

Example

ExampleClient_DoWithRetry shows automatic retry on 429 and 503 responses with exponential backoff. The Retry-After header is honoured when present.

package main

import (
	"context"
	"fmt"
	"io"
	"net/http"
	"net/http/httptest"
	"time"

	"gitlab.com/gitlab-org/labkit/v2/httpclient"
)

func main() {
	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprint(w, "ok")
	}))
	defer ts.Close()

	client := httpclient.New()
	req, _ := http.NewRequestWithContext(context.Background(), http.MethodGet, ts.URL, nil)

	resp, err := client.DoWithRetry(req, &httpclient.RetryConfig{
		MaxAttempts: 3,
		BaseDelay:   100 * time.Millisecond,
	})
	if err != nil {
		panic(err)
	}
	defer resp.Body.Close()

	body, _ := io.ReadAll(resp.Body)
	fmt.Println(string(body))
}
Output:
ok

func (*Client) Get

func (c *Client) Get(ctx context.Context, url string) (*http.Response, error)

Get issues a GET request to url using ctx.

func (*Client) HTTPClient

func (c *Client) HTTPClient() *http.Client

HTTPClient returns the underlying *http.Client. Use this when you need to pass the instrumented client to a third-party library that accepts *http.Client directly.

func (*Client) Head

func (c *Client) Head(ctx context.Context, url string) (*http.Response, error)

Head issues a HEAD request to url using ctx.

func (*Client) Patch

func (c *Client) Patch(ctx context.Context, url, contentType string, body io.Reader) (*http.Response, error)

Patch issues a PATCH request to url using ctx with the given content type and body.

func (*Client) Post

func (c *Client) Post(ctx context.Context, url, contentType string, body io.Reader) (*http.Response, error)

Post issues a POST request to url using ctx with the given content type and body.

Example

ExampleClient_Post shows a POST request with a JSON body.

package main

import (
	"context"
	"fmt"
	"io"
	"net/http"
	"net/http/httptest"
	"strings"

	"gitlab.com/gitlab-org/labkit/v2/httpclient"
)

func main() {
	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		body, _ := io.ReadAll(r.Body)
		fmt.Fprint(w, string(body))
	}))
	defer ts.Close()

	client := httpclient.New()
	resp, err := client.Post(
		context.Background(),
		ts.URL,
		"application/json",
		strings.NewReader(`{"name":"labkit"}`),
	)
	if err != nil {
		panic(err)
	}
	defer resp.Body.Close()
	body, _ := io.ReadAll(resp.Body)
	fmt.Println(string(body))
}
Output:
{"name":"labkit"}

func (*Client) Put

func (c *Client) Put(ctx context.Context, url, contentType string, body io.Reader) (*http.Response, error)

Put issues a PUT request to url using ctx with the given content type and body.

type Config

type Config struct {
	// Tracer is used to create client spans and inject outgoing W3C trace
	// context into request headers. When nil, tracing is disabled.
	Tracer *trace.Tracer

	// Logger is used for structured request logs. When nil, logging is disabled.
	Logger *slog.Logger

	// Transport is the underlying [http.RoundTripper]. When nil, a default
	// transport with sensible timeouts is created (5s dial, 5s TLS handshake,
	// 90s idle connection). Use this field to layer the instrumented transport
	// on top of a custom transport.
	Transport http.RoundTripper

	// Timeout is the overall timeout for a single request (not including
	// retries). Defaults to 30s. Set to -1 to disable.
	Timeout time.Duration
}

Config holds optional configuration for New / NewWithConfig.

type Doer

type Doer interface {
	Do(req *http.Request) (*http.Response, error)
}

Doer executes HTTP requests. It is the interface that Client implements and the seam point consumers should depend on for testability. Accepting a Doer in constructors allows callers to inject test doubles without needing httptest servers.

Example

ExampleDoer shows how to accept a Doer interface in your own code, making it easy to substitute a test double.

package main

import (
	"context"
	"fmt"
	"io"
	"net/http"
	"net/http/httptest"

	"gitlab.com/gitlab-org/labkit/v2/httpclient"
)

func main() {
	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprint(w, "hello")
	}))
	defer ts.Close()

	// call shows a function that depends on Doer, not *Client.
	call := func(d httpclient.Doer, url string) string {
		req, _ := http.NewRequestWithContext(context.Background(), http.MethodGet, url, nil)
		resp, err := d.Do(req)
		if err != nil {
			return ""
		}
		defer resp.Body.Close()
		b, _ := io.ReadAll(resp.Body)
		return string(b)
	}

	// In production: pass httpclient.New()
	// In tests: pass a mock that implements Doer
	fmt.Println(call(httpclient.New(), ts.URL))
}
Output:
hello

type RetryConfig

type RetryConfig struct {
	// MaxAttempts is the total number of attempts, including the first.
	// Must be >= 1. Defaults to 3.
	MaxAttempts int

	// RetryableStatus is the set of HTTP status codes that trigger a retry.
	// Defaults to [429, 503].
	RetryableStatus []int

	// RetryOnError controls whether transient network errors (DNS failures,
	// connection refused, timeouts) trigger a retry. Defaults to false for
	// backward compatibility. Enable this for services calling flaky sidecars
	// or endpoints with intermittent connectivity.
	RetryOnError bool

	// BaseDelay is the fallback delay between retries when no Retry-After
	// header is present or parseable. Defaults to 1s.
	BaseDelay time.Duration

	// MaxDelay caps the wait between retries, including any delay derived
	// from a Retry-After response header. Defaults to 60s.
	MaxDelay time.Duration
}

RetryConfig configures the retry behaviour of Client.DoWithRetry. A nil RetryConfig is treated identically to a zero value (all defaults).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL