retriable

package
v1.0.442 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 9, 2026 License: MIT Imports: 41 Imported by: 4

README

retriable

HTTP client with retry policy, JSON marshalling, header propagation, Bearer/DPoP authorization, replay nonces and file-based token storage.

Creating a client

client, err := retriable.New(retriable.ClientConfig{
	Host: "https://api.example.com",
	Request: &retriable.RequestPolicy{RetryLimit: 3, Timeout: 5 * time.Second},
}, retriable.WithUserAgent("my-cli"))
if err != nil {
	return err
}

var res struct{ ID string `json:"id"` }
hdr, status, err := client.Get(ctx, "/v1/items/1", &res)

Clients can also be loaded from YAML: LoadClient(file) reads a single ClientConfig, LoadFactory(file) reads a Config with named clients and Factory.CreateClient(name) / Factory.ForHost(url) build them.

clients:
  prod:
    host: https://api.example.com
    tls:
      cert: ~/.config/demo/client.pem
      key: ~/.config/demo/client.key
      trusted_ca: /etc/pki/cabundle.pem
    request:
      retry_limit: 3
      timeout: 2s
    storage_folder: ~/.config/demo

Interfaces

*Client implements HTTPClientWithNonce, which is the union of:

// GenericHTTP defines request helpers that take an explicit host.
type GenericHTTP interface {
	// Request sends a request to the specified host and decodes the response
	// into responseBody. Statuses >= 300 (except 204) are returned as an error.
	//
	// host should include all the protocol/host/port preamble, e.g. https://foo.bar:3444
	// path should be an absolute URI path, i.e. /foo/bar/baz
	// requestBody can be io.Reader, []byte, string, or an object to be JSON encoded
	// responseBody can be io.Writer, or a struct to decode JSON into.
	Request(ctx context.Context, method string, host string, path string, requestBody any, responseBody any) (http.Header, int, error)

	// RequestURL is similar to Request but takes a raw URL.
	RequestURL(ctx context.Context, method, rawURL string, requestBody any, responseBody any) (http.Header, int, error)

	// HeadTo makes a HEAD request against the specified host.
	HeadTo(ctx context.Context, host string, path string) (http.Header, int, error)
}

// HTTPClient groups the helpers that use the configured host.
type HTTPClient interface {
	Head(ctx context.Context, path string) (http.Header, int, error)
	Get(ctx context.Context, path string, body any) (http.Header, int, error)
	Post(ctx context.Context, path string, requestBody any, responseBody any) (http.Header, int, error)
	Put(ctx context.Context, path string, requestBody any, responseBody any) (http.Header, int, error)
	Delete(ctx context.Context, path string, body any) (http.Header, int, error)
}

Client.Do(*http.Request) is the lower-level entry point (implements Requestor): it applies headers, auth and retries and returns the raw response, which the caller must close.

Retry policy

Policy.Retries maps an HTTP status (0 for transport errors) to a ShouldRetry callback; TotalRetryLimit caps retries across all codes and RequestTimeout bounds each Request/Get/... call (not Do). DefaultPolicy() allows 5 retries in total: connection errors up to 4 times (2s wait), 502 and 503 up to 5 times (1s), and 429 up to 3 times after the response's Retry-After (1s without one). A 429 whose Retry-After asks for more than 30s is returned at once with reason LimitExceeded, as is a 429 under a policy without a 429 entry. DNS/TLS errors, 404 and other 4xx are never retried. RetryAfterShouldRetryFactory(limit, wait, maxWait, reason) builds the same Retry-After handling for custom policies (for example 503).

The wait between attempts ends when the request context is done, and Do returns the context error; while the context is live, a retry that could not start before its deadline is skipped and the last response is returned. A retried response body is drained (up to 64 KiB) and closed; an error response body is read up to 64 KiB into the returned error.

In a ClientConfig, request.retry_limit and request.timeout replace the DefaultPolicy values only when non-zero; a negative retry_limit disables retries.

client.WithPolicy(retriable.Policy{
	TotalRetryLimit: 3,
	RequestTimeout:  time.Second,
	Retries: map[int]retriable.ShouldRetry{
		http.StatusServiceUnavailable: retriable.DefaultShouldRetryFactory(2, time.Millisecond, "retry"),
	},
})

Headers and context

  • WithHeaders(ctx, map) / PropagateHeadersFromRequest(ctx, r, names...) attach headers to a context; the client sets them on outgoing requests.
  • X-Correlation-ID is added from the context (xhttp/correlation) when absent.
  • WithUserAgent(name) adds User-Agent, X-CLIENT-HOSTNAME, X-CLIENT-IP without waiting for the network; a host name or IP that cannot be resolved is not sent.

Transport

WithTLS(cfg) and WithDNSServer(addr) install a modified clone of the current *http.Transport (of http.DefaultTransport when none is set), so a transport passed to WithTransport is never changed. Call them after WithTransport, which replaces the transport with all its settings. On any other http.RoundTripper they cannot apply their setting: New returns the error, and after New every request fails with it until WithTransport replaces the transport. When a transport the client created is replaced, its idle connections are closed; a transport passed to WithTransport is never closed by the client.

New applies ClientConfig.TLS after the options, to the transport they leave, so New(cfg, WithTransport(t)) uses a clone of t with the TLS configuration from the files; when t is not an *http.Transport, New returns the error. Only a WithTLS option that no later WithTransport replaced takes precedence over ClientConfig.TLS.

RequestURL derives the host and request URI from the parsed URL and rejects URLs without a scheme, opaque URLs (https:host/path) and URLs with user info (credentials).

Authorization and storage

Storage (root = ClientConfig.StorageFolder) keeps .auth_token and DPoP keys (<thumbprint>.jwk). Load the token with ClientConfig.LoadAuthTokenOrFromEnv("MY_TOKEN") or LoadAuthToken(), then call client.SetAuthorization() to add the Authorization header; DPoP tokens are signed per request. Token format is an opaque string or access_token=...&exp=<unix>&dpop_jkt=<jkt>&token_type=DPoP. Authorization is only sent to https:// and unixs:// hosts.

New credentials folders use mode 0700; existing folder permissions are kept. An empty storage path uses the working directory. Token and key files use 0600. DPoP proofs are signed separately for each retry attempt.

Nonces

client.WithNonce("/nonce", retriable.DefaultReplayNonceHeader) installs a NonceProvider that caches nonces from response headers and fetches new ones with HEAD requests; it satisfies jose.NonceSource. NonceContext(ctx) bounds a fetch by ctx; Nonce() (which has no context) bounds it by 30s. The cache keeps the newest nonces: when its 64 entries are full, the oldest half is dropped.

Documentation

Overview

Package retriable provides an HTTP client with a configurable retry policy, JSON request/response marshalling, header propagation through context, bearer/DPoP authorization, replay-nonce handling and file-based token storage.

A Client wraps a standard *http.Client. Every request goes through Client.Do, which rewinds the body and re-sends it according to Policy.ShouldRetry; the higher-level Get/Post/Put/Delete/Request helpers additionally encode the request body and decode the response. Responses with status >= 300 (except 204) are converted into an error, preferably an *httperror.Error when the body is a JSON error document.

Basic usage: 

client, err := retriable.New(retriable.ClientConfig{
	Host: "https://api.example.com",
	Request: &retriable.RequestPolicy{RetryLimit: 3, Timeout: 5 * time.Second},
}, retriable.WithUserAgent("my-cli"))
if err != nil {
	return err
}
var res struct{ ID string `json:"id"` }
hdr, status, err := client.Get(ctx, "/v1/items/1", &res)

Clients can also be built from a YAML/JSON configuration with LoadClient (a single ClientConfig) or LoadFactory (a Config holding a map of named ClientConfig entries): 

clients:
  prod:
    host: https://api.example.com
    tls:
      cert: ~/.config/demo/client.pem
      key: ~/.config/demo/client.key
      trusted_ca: /etc/pki/cabundle.pem
    request:
      retry_limit: 3
      timeout: 2s
    storage_folder: ~/.config/demo

StorageFolder is the root of a Storage where the access token (.auth_token) and DPoP private keys (<thumbprint>.jwk) are kept; Client.SetAuthorization loads them and adds the Authorization header.

The retry policy is described by Policy: Retries maps an HTTP status code (0 for transport errors) to a ShouldRetry callback, TotalRetryLimit caps the number of retries across all codes, and RequestTimeout bounds each call made through Request/Get/Post/etc. DefaultPolicy retries connection errors, 502 and 503, and 429 after the server's Retry-After (see RetryAfterShouldRetryFactory). The wait between attempts ends when the request context is done. In a ClientConfig, a zero retry_limit or timeout keeps the DefaultPolicy value and a negative retry_limit disables retries.

All exported functions return errors rather than panic. WithTLS and WithDNSServer install modified copies of an *http.Transport; on any other http.RoundTripper they cannot apply their setting, so New and every request return an error until WithTransport replaces the transport. New applies ClientConfig.TLS after the options, so it also applies to a transport set with WithTransport, unless a later WithTLS option sets the TLS configuration.

Concurrency: the With*/Add*/Set* methods are synchronized, and a request uses the configuration current when it starts. Direct changes to the exported Name, Policy and Config fields, or to the *http.Client returned by HTTPClient, are not synchronized; make them before sharing the Client between goroutines.

Index

Constants

View Source 
const (
	// Success is returned when the request succeeded (status < 400).
	Success = "success"
	// NotFound is returned when the request returned 404.
	NotFound = "not-found"
	// LimitExceeded is returned when TotalRetryLimit was reached, on 429
	// without a Retries entry, and by RetryAfterShouldRetryFactory when the
	// server asks to wait longer than its maxWait.
	LimitExceeded = "limit-exceeded"
	// DeadlineExceeded is returned when the request context deadline passed.
	DeadlineExceeded = "deadline"
	// Cancelled is returned when the request context was cancelled.
	Cancelled = "cancelled"
	// NonRetriableError is returned for errors and statuses that are never retried.
	NonRetriableError = "non-retriable"
)

Reason strings returned by Policy.ShouldRetry when it stops retrying.

View Source 
const (
	// DefaultReplayNonceHeader is the conventional (ACME-style) header name
	// carrying a replay nonce; pass it to Client.WithNonce or NewNonceProvider.
	DefaultReplayNonceHeader = header.ReplayNonce
)

Variables

View Source 
var DefaultNonRetriableErrors = []string{
	"no such host",
	"TLS handshake error",
	"certificate signed by unknown authority",
	"client didn't provide a certificate",
	"tls: bad certificate",
	"x509: certificate",
	"x509: cannot validate certificate",
	"server gave HTTP response to HTTPS client",
	"dial tcp: lookup",
	"peer reset",
}

DefaultNonRetriableErrors lists substrings of transport error messages (DNS, TLS and certificate failures) that the default policy never retries.

Functions

func DumpRequestOut added in v0.33.326 

func DumpRequestOut(req *http.Request, body bool) ([]byte, error)

DumpRequestOut returns the wire representation of an outgoing client request, like httputil.DumpRequestOut: it includes any headers that the standard http.Transport adds, such as User-Agent. When body is true the request body is read into memory and restored on req.

Sensitive credential headers are redacted in the dump without changing req. A nil request or URL returns an error. This is adapted from the Go standard library implementation.

func HostFolderName 

func HostFolderName(host string) string

HostFolderName returns the storage sub-folder name derived from a host URL: the host[:port] part with ":" replaced by "_", e.g. "https://foo.bar:3444" -> "foo.bar_3444".

func PropagateHeadersFromRequest 

func PropagateHeadersFromRequest(ctx context.Context, r *http.Request, headers ...string) context.Context

PropagateHeadersFromRequest returns a context carrying the named headers that are present in the incoming request r, so that a Client used with that context forwards them on its outgoing requests (see WithHeaders). A nil ctx is treated as context.Background().

func WithHeaders 

func WithHeaders(ctx context.Context, headers map[string]string) context.Context

WithHeaders returns a copy of ctx carrying headers that a Client sets on every outgoing request made with that context (overriding client-level headers of the same name). It replaces, not merges, headers stored by an earlier WithHeaders or PropagateHeadersFromRequest call. A nil ctx is treated as context.Background().

Types

type AuthToken 

type AuthToken struct {
	// Raw is the original token string.
	Raw string
	// AccessToken is the value sent in the Authorization header.
	AccessToken string
	// RefreshToken is the optional refresh_token value.
	RefreshToken string
	// TokenType is "Bearer" (default) or "DPoP".
	TokenType string
	// DpopJkt is the thumbprint of the DPoP key bound to the token, if any;
	// it names the "<jkt>.jwk" file in Storage.
	DpopJkt string
	// Expires is the optional expiry time (from exp).
	Expires *time.Time
}

AuthToken is a parsed access token as stored in .auth_token or an environment variable; see ParseAuthToken.

func LoadAuthToken 

func LoadAuthToken(dir string) (*AuthToken, string, error)

LoadAuthToken reads and parses the ".auth_token" file in dir. It returns the token, the file location (also on error) and an error when the file is missing or malformed. Expiry is not checked; use AuthToken.Expired.

func ParseAuthToken 

func ParseAuthToken(rawToken, location string) (*AuthToken, string, error)

ParseAuthToken parses a token string. A value without "=" is an opaque Bearer access token; otherwise it is parsed as a query string with the keys access_token (or id_token, or token), refresh_token, dpop_jkt and exp (unix seconds). location is passed through for the caller's logging. Expiry is parsed but not validated.

func (*AuthToken) Expired 

func (t *AuthToken) Expired() bool

Expired returns true if expiry is present on the token, and is behind the current time

type BeforeSendRequest 

type BeforeSendRequest func(r *http.Request) *http.Request

BeforeSendRequest is a hook invoked once per request (before retries) that may modify or replace the outgoing request.

type Client 

type Client struct {
	// Name identifies the client in logs.
	Name string
	// Policy is the retry policy for HTTP requests; defaults to DefaultPolicy.
	Policy Policy

	// Config is the configuration the client was created from; its Storage
	// is used by SetAuthorization to load tokens and DPoP keys.
	Config ClientConfig
	// contains filtered or unexported fields
}

Client is an HTTP client with retries, JSON marshalling, header propagation and token-based authorization on top of *http.Client. Create it with New, Default, LoadClient or a Factory.

func Default 

func Default(host string) (*Client, error)

Default creates a Client for the given host with DefaultPolicy, no TLS configuration and no token storage.

func LoadClient 

func LoadClient(file string) (*Client, error)

LoadClient reads a single YAML ClientConfig from file (with ~ and $VAR expansion of all paths) and returns a Client built with New.

func New 

func New(cfg ClientConfig, opts ...ClientOption) (*Client, error)

New creates a Client from cfg and applies opts on top of it. cfg.Request (if set) overrides the RequestTimeout and TotalRetryLimit of DefaultPolicy with its non-zero values (see RequestPolicy). cfg.TLS (if set) is loaded from files and applied with WithTLS after opts, to the transport they leave, so a transport set with WithTransport gets it too; only a WithTLS option that no later WithTransport replaced takes precedence over cfg.TLS. It returns an error when the TLS files cannot be loaded, or when WithTLS (including the one for cfg.TLS) or WithDNSServer cannot change the transport: one set by WithTransport, or http.DefaultTransport when none is set, that is not an *http.Transport.

func NewForHost 

func NewForHost(cfg, host string) (*Client, error)

NewForHost loads a Factory from the cfg file and returns its client for host. When the config file cannot be loaded, a Default client for host is returned instead; only a failure to build the configured client is an error.

func (*Client) AddHeader 

func (c *Client) AddHeader(header, value string) *Client

AddHeader adds a header that is sent with every request, replacing any previous value for the same name.

func (*Client) CurrentHost 

func (c *Client) CurrentHost() string

CurrentHost returns the configured host (scheme://host[:port]).

func (*Client) DecodeResponse 

func (c *Client) DecodeResponse(resp *http.Response, body any) (http.Header, int, error)

DecodeResponse maps an HTTP response to either the body parameter or an error, and returns the response headers and status code in both cases. 204 returns immediately; for a status >= 300 at most 64 KiB of the body is read, and it is decoded as an *httperror.Error when it is a JSON error document with a "code", otherwise the (possibly truncated) body text becomes the error message. For other statuses the body is copied into body when it is an io.Writer, or JSON-decoded into it (numbers are decoded as json.Number). It does not close resp.Body.

func (*Client) Delete 

func (c *Client) Delete(ctx context.Context, path string, body any) (http.Header, int, error)

Delete makes an HTTP DELETE to the configured host (see Request) and decodes the response into body (io.Writer or JSON target). Statuses >= 300 are returned as an error, with retries applied per the client Policy. path should be an absolute URI path, i.e. /foo/bar/baz

func (*Client) Do 

func (c *Client) Do(r *http.Request) (*http.Response, error)

Do sends r with retries according to Policy and returns the final response, which the caller must close. It implements Requestor. Before sending, the client headers, context-propagated headers (see WithHeaders / PropagateHeadersFromRequest), the X-Correlation-ID, the BeforeSendRequest hook and caller-identity token are applied once; the DPoP proof is signed for each attempt. The body is buffered so it can be rewound for each retry. Do does not apply Policy.RequestTimeout: bound r's context yourself. The wait between attempts ends early when r's context is done, and Do then returns an error wrapping the context error. While the context is live, a retry that could not start before its deadline is not attempted, and the last response (or transport error) is returned instead. When retries are exhausted the last response (or transport error) is returned; a non-2xx status is not converted to an error here. Do returns the error recorded by WithTLS or WithDNSServer when they could not change the transport, and closes r.Body.

func (*Client) Get 

func (c *Client) Get(ctx context.Context, path string, body any) (http.Header, int, error)

Get makes an HTTP GET to the configured host (see Request) and decodes the response into body (io.Writer or JSON target). Statuses >= 300 are returned as an error, with retries applied per the client Policy. path should be an absolute URI path, i.e. /foo/bar/baz

func (*Client) GetNonceProvider 

func (c *Client) GetNonceProvider() NonceProvider

GetNonceProvider returns the current nonce provider, or nil.

func (*Client) HTTPClient 

func (c *Client) HTTPClient() *http.Client

HTTPClient returns the underlying *http.Client, for callers that need to tweak it directly. Such changes are not synchronized with requests in flight; make them before the client is shared.

func (*Client) Head 

func (c *Client) Head(ctx context.Context, path string) (http.Header, int, error)

Head makes a HEAD request to the configured host (see HeadTo). path should be an absolute URI path, i.e. /foo/bar/baz

func (*Client) HeadTo 

func (c *Client) HeadTo(ctx context.Context, host string, path string) (http.Header, int, error)

HeadTo makes a HEAD request against the specified host, applying the retry Policy and RequestTimeout, and returns the response headers and status code. Unlike Request, a non-2xx status is not turned into an error.

host should include all the protocol/host/port preamble, e.g. https://foo.bar:3444 path should be an absolute URI path, i.e. /foo/bar/baz

func (*Client) Post 

func (c *Client) Post(ctx context.Context, path string, requestBody any, responseBody any) (http.Header, int, error)

Post makes an HTTP POST to the configured host (see Request). requestBody is sent as-is for io.Reader, []byte and string, otherwise JSON encoded; the response is decoded into responseBody and statuses >= 300 are returned as an error, with retries applied per the client Policy. path should be an absolute URI path, i.e. /foo/bar/baz

func (*Client) Put 

func (c *Client) Put(ctx context.Context, path string, requestBody any, responseBody any) (http.Header, int, error)

Put makes an HTTP PUT to the configured host (see Request). requestBody is sent as-is for io.Reader, []byte and string, otherwise JSON encoded; the response is decoded into responseBody and statuses >= 300 are returned as an error, with retries applied per the client Policy. path should be an absolute URI path, i.e. /foo/bar/baz

func (*Client) Request 

func (c *Client) Request(ctx context.Context, method string, host string, path string, requestBody any, responseBody any) (http.Header, int, error)

Request sends a request to the specified host and decodes the response into responseBody (see DecodeResponse). It returns the HTTP headers, status code, and an optional error. For responses with status codes >= 300 (except 204) it converts the response into a Go error, preferably an *httperror.Error. The retry Policy and RequestTimeout are applied, a correlation ID is attached, and the nonce provider (if any) is fed the response headers.

host should include all the protocol/host/port preamble, e.g. https://foo.bar:3444 path should be an absolute URI path, i.e. /foo/bar/baz requestBody can be io.Reader, []byte, string, or an object to be JSON encoded responseBody can be io.Writer, or a struct to decode JSON into.

func (*Client) RequestURL 

func (c *Client) RequestURL(ctx context.Context, method, rawURL string, requestBody any, responseBody any) (http.Header, int, error)

RequestURL is similar to Request but takes a raw URL; the host (scheme://host[:port]) and the request URI (escaped path, query and fragment) are derived from the parsed URL. A URL without a scheme, an opaque URL (scheme:opaque) or one with user info (credentials) is an error; set credentials with SetAuthorization, WithCallerIdentity or headers instead.

func (*Client) SetAuthorization added in v0.35.345 

func (c *Client) SetAuthorization() error

SetAuthorization adds the Authorization header from Config.AuthToken, loading it from Storage when not yet set. For DPoP tokens the private key named by the token's dpop_jkt is loaded from Storage and every request is signed. It is a no-op for hosts that are not https:// or unixs://, and an error when the token is missing or expired.

func (*Client) SetNonceProvider 

func (c *Client) SetNonceProvider(provider NonceProvider)

SetNonceProvider replaces the nonce provider. When set, Request feeds every response's headers to it via SetFromHeader.

func (*Client) Storage 

func (c *Client) Storage() *Storage

Storage returns the token/key storage of the client's Config, creating it from Config.StorageFolder on first use.

func (*Client) WithBeforeSendRequest 

func (c *Client) WithBeforeSendRequest(hook BeforeSendRequest) *Client

WithBeforeSendRequest installs a hook that is invoked once per request (before retries) to modify or replace the outgoing request.

func (*Client) WithCallerIdentity added in v0.17.0 

func (c *Client) WithCallerIdentity(ci credentials.CallerIdentity) *Client

WithCallerIdentity installs a token provider. Before each request the client calls GetCallerIdentity when it has no token or the cached token has expired, and sets the Authorization header from the result.

func (*Client) WithDNSServer 

func (c *Client) WithDNSServer(dns string) *Client

WithDNSServer makes the transport resolve names through the given DNS server, which must be specified in <host>:<port> format. Like WithTLS, it installs a clone of the current *http.Transport (of http.DefaultTransport when none is set) with DialContext replaced, and records an error returned by New and every request when the transport is another http.RoundTripper.

func (*Client) WithHeaders 

func (c *Client) WithHeaders(headers map[string]string) *Client

WithHeaders adds headers that are sent with every request.

func (*Client) WithHost 

func (c *Client) WithHost(host string) *Client

WithHost sets the host (scheme://host[:port]) used by the Head/Get/Post/Put/Delete helpers.

func (*Client) WithName 

func (c *Client) WithName(name string) *Client

WithName modifies client's name for logging purposes.

func (*Client) WithNonce 

func (c *Client) WithNonce(path, headerName string)

WithNonce installs the default nonce provider (see NewNonceProvider) that fetches nonces with HEAD requests to path on the configured host and reads them from the headerName response header. A leading CurrentHost() prefix in path is stripped.

func (*Client) WithPolicy 

func (c *Client) WithPolicy(policy Policy) *Client

WithPolicy modifies retriable policy.

func (*Client) WithStorage added in v0.35.345 

func (c *Client) WithStorage(storage *Storage) *Client

WithStorage replaces the token/key storage used by SetAuthorization.

func (*Client) WithTLS 

func (c *Client) WithTLS(tlsConfig *tls.Config) *Client

WithTLS sets the TLS configuration of the transport. It installs a clone of the current *http.Transport with TLSClientConfig replaced, so a transport supplied with WithTransport (or http.DefaultTransport) is never modified; when no transport is set, the clone is made from http.DefaultTransport with 100 max (idle) connections per host. When the transport is another http.RoundTripper, the configuration cannot be applied: New returns an error, and every request fails with that error until WithTransport replaces the transport.

func (*Client) WithTimeout 

func (c *Client) WithTimeout(timeout time.Duration) *Client

WithTimeout sets Policy.RequestTimeout, the per-call timeout applied by Request and the Get/Post/Put/Delete/Head helpers (not by Do).

func (*Client) WithTransport 

func (c *Client) WithTransport(transport http.RoundTripper) *Client

WithTransport replaces the HTTP transport, including any TLS or DNS setting made before, and clears an error left by WithTLS or WithDNSServer. Call it before WithTLS or WithDNSServer, which install modified copies of an *http.Transport. The idle connections of a transport that WithTLS or WithDNSServer created are closed; the caller's transport is left as is. In New, ClientConfig.TLS is applied after the options, so a transport set by a WithTransport option is replaced by a clone with that TLS configuration unless a later WithTLS option sets one (see New). A nil transport, including a nil *http.Transport, selects http.DefaultTransport.

func (*Client) WithUserAgent added in v0.22.0 

func (c *Client) WithUserAgent(name string) *Client

WithUserAgent adds the User-Agent, X-CLIENT-HOSTNAME and X-CLIENT-IP headers to every request. The host name and the first non-loopback IPv4 address are read once, without waiting for the network; a value that cannot be resolved is not sent.

type ClientConfig 

type ClientConfig struct {
	// Host is the base URL of the server, e.g. https://foo.bar:3444.
	Host string `json:"host" yaml:"host"`

	// TLS provides TLS config for the client
	TLS *TLSInfo `json:"tls,omitempty" yaml:"tls,omitempty"`

	// Request provides Request Policy
	Request *RequestPolicy `json:"request,omitempty" yaml:"request,omitempty"`

	// StorageFolder specifies the root folder for keys and token.
	StorageFolder string `json:"storage_folder,omitempty" yaml:"storage_folder,omitempty"`

	// AuthToken is the access token loaded by LoadAuthToken or
	// CheckAuthTokenFromEnv and used by Client.SetAuthorization.
	AuthToken *AuthToken `json:"-" yaml:"-"`

	// TokenLocation describes where AuthToken came from
	// (a file path or "env://NAME"), for logging.
	TokenLocation string `json:"-" yaml:"-"`
	// contains filtered or unexported fields
}

ClientConfig is the configuration of a Client for one host. It is the YAML/JSON document read by LoadClient and the value of each entry in Config.Clients.

func (*ClientConfig) CheckAuthTokenFromEnv added in v0.35.346 

func (c *ClientConfig) CheckAuthTokenFromEnv(env string) (bool, error)

CheckAuthTokenFromEnv loads AuthToken from the named environment variable (see ParseAuthToken for the accepted formats). It returns false when the variable is unset or empty, and an error when the value is malformed or the token has expired.

func (*ClientConfig) LoadAuthToken added in v0.35.345 

func (c *ClientConfig) LoadAuthToken() error

LoadAuthToken sets AuthToken and TokenLocation from the .auth_token file in Storage. It does not check expiry.

func (*ClientConfig) LoadAuthTokenOrFromEnv added in v0.35.346 

func (c *ClientConfig) LoadAuthTokenOrFromEnv(env string) error

LoadAuthTokenOrFromEnv sets AuthToken from the named environment variable when it is set, otherwise from the .auth_token file in Storage.

func (*ClientConfig) Storage 

func (c *ClientConfig) Storage() *Storage

Storage returns the token/key storage rooted at StorageFolder, creating it on first use.

type ClientOption 

type ClientOption interface {
	// contains filtered or unexported methods
}

A ClientOption modifies the default behavior of Client. New applies the options after the host and request policy of the ClientConfig, and before its TLS configuration (see New).

func WithBeforeSendRequest 

func WithBeforeSendRequest(hook BeforeSendRequest) ClientOption

WithBeforeSendRequest is a ClientOption that installs a hook to modify the request before it's sent.

func WithCallerIdentity added in v0.22.0 

func WithCallerIdentity(ci credentials.CallerIdentity) ClientOption

WithCallerIdentity is a ClientOption that installs a token provider used to obtain (and refresh on expiry) the Authorization header for each request.

func WithDNSServer 

func WithDNSServer(dns string) ClientOption

WithDNSServer is a ClientOption that allows to use custom dns server for resolution dns server must be specified in <host>:<port> format; see Client.WithDNSServer. New returns an error when the transport is not an *http.Transport. 

retriable.New(cfg, retriable.WithDNSServer(dns))

This option cannot be provided for constructors which produce result objects. WithTransport replaces the transport, including the DNS setting, so when both are used, WithDNSServer must come after WithTransport: 

retriable.New(cfg, retriable.WithTransport(t), retriable.WithDNSServer(dns))

func WithHost 

func WithHost(host string) ClientOption

WithHost is a ClientOption that allows to set the host list. 

retriable.New(cfg, retriable.WithHost(host))

func WithName 

func WithName(name string) ClientOption

WithName is a ClientOption that specifies client's name for logging purposes. 

retriable.New(cfg, retriable.WithName("tlsclient"))

This option cannot be provided for constructors which produce result objects.

func WithPolicy 

func WithPolicy(policy Policy) ClientOption

WithPolicy is a ClientOption that specifies retriable policy. 

retriable.New(cfg, retriable.WithPolicy(p))

This option cannot be provided for constructors which produce result objects.

func WithTLS 

func WithTLS(tlsConfig *tls.Config) ClientOption

WithTLS is a ClientOption that specifies TLS configuration; see Client.WithTLS. New returns an error when the transport is not an *http.Transport. 

retriable.New(cfg, retriable.WithTLS(t))

This option cannot be provided for constructors which produce result objects.

func WithTimeout 

func WithTimeout(timeout time.Duration) ClientOption

WithTimeout is a ClientOption that specifies HTTP client timeout. 

retriable.New(cfg, retriable.WithTimeout(t))

This option cannot be provided for constructors which produce result objects.

func WithTransport 

func WithTransport(transport http.RoundTripper) ClientOption

WithTransport is a ClientOption that specifies HTTP Transport configuration; see Client.WithTransport. When cfg.TLS is set, New applies it (unless a later WithTLS option sets one) to a clone of the transport; New returns an error when that transport is not an *http.Transport. 

retriable.New(cfg, retriable.WithTransport(t))

This option cannot be provided for constructors which produce result objects.

func WithUserAgent added in v0.22.0 

func WithUserAgent(name string) ClientOption

WithUserAgent is a ClientOption that adds the User-Agent, X-CLIENT-HOSTNAME and X-CLIENT-IP headers to every request; see Client.WithUserAgent.

type Config 

type Config struct {
	// Clients maps a logical client name to its configuration.
	Clients map[string]*ClientConfig `json:"clients,omitempty" yaml:"clients,omitempty"`
}

Config is the multi-client configuration consumed by NewFactory and LoadFactory: a map of client name to ClientConfig.

type DeleteRequester 

type DeleteRequester interface {
	// Delete makes a DELETE request to the configured host;
	// path should be an absolute URI path, i.e. /foo/bar/baz.
	// The response body is decoded into body (io.Writer or JSON target) and
	// the HTTP headers and status code are returned.
	Delete(ctx context.Context, path string, body any) (http.Header, int, error)
}

DeleteRequester defines the HTTP DELETE helper against the configured host.

type Factory 

type Factory struct {
	// contains filtered or unexported fields
}

Factory creates Clients from a Config, looked up either by client name or by host URL. When several clients share a host, the last one wins for ForHost/ConfigForHost.

func LoadFactory 

func LoadFactory(file string) (*Factory, error)

LoadFactory reads a YAML Config from file (with ~ and $VAR expansion of all paths) and returns a Factory. Each client's StorageFolder gets a per-host sub-folder appended (see HostFolderName) unless already present.

func NewFactory 

func NewFactory(cfg Config) (*Factory, error)

NewFactory returns a Factory for cfg. It fails when a client entry has no host.

func (*Factory) ConfigForHost 

func (f *Factory) ConfigForHost(hostname string) *ClientConfig

ConfigForHost returns the ClientConfig whose Host equals hostname (full URL, e.g. https://foo.bar:3444), or nil.

func (*Factory) CreateClient 

func (f *Factory) CreateClient(clientName string) (*Client, error)

CreateClient returns Client for a specified client name. If the name is not found in the configuration, a client with default settings will be returned.

func (*Factory) ForHost 

func (f *Factory) ForHost(hostname string) (*Client, error)

ForHost returns a Client for the specified host URL. If the host is not found in the configuration, a Default client for that host is returned.

type GenericHTTP 

type GenericHTTP interface {
	// Request sends a request to the specified host and decodes the response
	// into responseBody.
	// It returns the HTTP headers, status code, and an optional error.
	// For responses with status codes >= 300 (except 204) it converts the
	// response into a Go error, preferably an *httperror.Error.
	// The client's retry Policy and RequestTimeout are applied.
	//
	// host should include all the protocol/host/port preamble, e.g. https://foo.bar:3444
	// path should be an absolute URI path, i.e. /foo/bar/baz
	// requestBody can be io.Reader, []byte, string, or an object to be JSON encoded
	// responseBody can be io.Writer, or a struct to decode JSON into.
	Request(ctx context.Context, method string, host string, path string, requestBody any, responseBody any) (http.Header, int, error)

	// RequestURL is similar to Request but takes a raw URL, from which the
	// host (scheme://host[:port]) and the path are derived.
	RequestURL(ctx context.Context, method, rawURL string, requestBody any, responseBody any) (http.Header, int, error)

	// HeadTo makes a HEAD request against the specified host and returns the
	// response headers and status without decoding a body.
	//
	// host should include all the protocol/host/port preamble, e.g. https://foo.bar:3444
	// path should be an absolute URI path, i.e. /foo/bar/baz
	HeadTo(ctx context.Context, host string, path string) (http.Header, int, error)
}

GenericHTTP defines request helpers that take an explicit host, for callers that need to address a server other than the configured one.

type GetRequester 

type GetRequester interface {
	// Get makes a GET request to the configured host;
	// path should be an absolute URI path, i.e. /foo/bar/baz.
	// The response body is decoded into body (io.Writer or JSON target) and
	// the HTTP headers and status code are returned.
	Get(ctx context.Context, path string, body any) (http.Header, int, error)
}

GetRequester defines the HTTP GET helper against the configured host.

type HTTPClient 

type HTTPClient interface {
	HeadRequester
	GetRequester
	PostRequester
	PutRequester
	DeleteRequester
}

HTTPClient is the union of the per-method helpers (Head, Get, Post, Put, Delete) that operate against the client's configured host. *Client implements it.

type HTTPClientWithNonce 

type HTTPClientWithNonce interface {
	GenericHTTP
	HTTPClient
	NonceRequester
}

HTTPClientWithNonce is the full client surface: explicit-host requests, configured-host helpers and nonce management. *Client implements it.

type HeadRequester 

type HeadRequester interface {
	// Head makes a HEAD request to the configured host.
	// path should be an absolute URI path, i.e. /foo/bar/baz
	Head(ctx context.Context, path string) (http.Header, int, error)
}

HeadRequester defines the HTTP HEAD helper against the configured host.

type KeyInfo 

type KeyInfo struct {
	// KeySize is the modulus size (RSA) or curve size (ECDSA) in bits.
	KeySize int
	// Type is "RSA" or "ECDSA".
	Type string
	// Algo is the JWS algorithm matched to the key size: RS256/384/512 or ES256/384/512.
	Algo string
	// Thumbprint is the base64url SHA-256 JWK thumbprint (the dpop_jkt value).
	Thumbprint string
	// Key is the parsed JWK.
	Key *jose.JSONWebKey
}

KeyInfo describes a stored DPoP private key.

func NewKeyInfo 

func NewKeyInfo(k *jose.JSONWebKey) (*KeyInfo, error)

NewKeyInfo computes KeyInfo for an RSA or ECDSA private JWK; other key types return an error.

type NonceProvider 

type NonceProvider interface {
	// Nonce returns an unused nonce, taking a cached one received in an
	// earlier response or fetching a fresh one from the server.
	// It implements jose.NonceSource, which passes no context; the default
	// provider bounds the fetch with a 30s timeout. Use NonceContext when a
	// context is available.
	Nonce() (string, error)
	// NonceContext is Nonce with the fetch bounded by ctx.
	NonceContext(ctx context.Context) (string, error)
	// SetFromHeader caches the nonce found in the response headers, if any.
	SetFromHeader(hdr http.Header)
}

NonceProvider supplies server-issued replay nonces for signed requests. It satisfies jose.NonceSource.

func NewNonceProvider 

func NewNonceProvider(client HTTPClient, noncePath, headerName string) NonceProvider

NewNonceProvider returns the default NonceProvider: it keeps a bounded LIFO cache (64 entries; when full, the oldest half is dropped) of nonces seen in headerName response headers, and when empty fetches one with a HEAD request to noncePath through client. It is safe for concurrent use.

type NonceRequester 

type NonceRequester interface {
	// SetNonceProvider replaces the nonce provider used by the client.
	SetNonceProvider(provider NonceProvider)
	// GetNonceProvider returns the current nonce provider, or nil.
	GetNonceProvider() NonceProvider
	// WithNonce installs the default provider that fetches nonces with a HEAD
	// request to path and reads them from the headerName response header.
	WithNonce(path, headerName string)
}

NonceRequester is implemented by clients that can attach a replay-nonce provider; see NonceProvider.

type Policy 

type Policy struct {

	// Retries specifies a map of HTTP Status code to ShouldRetry function,
	// 0 status code indicates a connection related error (network, TLS, DNS etc.)
	Retries map[int]ShouldRetry

	// TotalRetryLimit is the maximum number of retries across all status codes;
	// once reached ShouldRetry returns LimitExceeded regardless of Retries.
	// Zero or a negative value disables retries.
	TotalRetryLimit int

	// RequestTimeout, when > 0, bounds each call made through Request and the
	// Get/Post/Put/Delete/Head helpers by deriving a context with this timeout.
	// It is not applied by Do.
	RequestTimeout time.Duration

	// NonRetriableErrors is a list of substrings; a transport error whose
	// message contains one of them is never retried.
	// See DefaultNonRetriableErrors.
	NonRetriableErrors []string
}

Policy represents the retry policy of a Client. Use DefaultPolicy as a starting point.

func DefaultPolicy 

func DefaultPolicy() Policy

DefaultPolicy returns the policy used by New: TotalRetryLimit is 5; connection errors are retried up to 4 times with a 2s wait (DefaultShouldRetryFactory(3, ...) allows retries while the retry count is <= 3); 502 and 503 up to 5 times with a 1s wait; 429 up to 3 times, waiting for the Retry-After of the response, or 1s without one, and not at all when Retry-After asks for more than 30s (see RetryAfterShouldRetryFactory). There is no RequestTimeout, and DefaultNonRetriableErrors are never retried.

func (*Policy) ShouldRetry 

func (p *Policy) ShouldRetry(r *http.Request, resp *http.Response, err error, retries int) (bool, time.Duration, string)

ShouldRetry decides whether the attempt should be retried and how long to wait first. Order of evaluation: a cancelled or expired request context is never retried; TotalRetryLimit is enforced; transport errors matching NonRetriableErrors are not retried, others are delegated to Retries[0]; statuses < 400 succeed; 404 is not retried; 429 is delegated to Retries[429] if present and otherwise reports LimitExceeded; other 4xx are non-retriable; 5xx are delegated to Retries[status] if present.

type PostRequester 

type PostRequester interface {
	// Post makes an HTTP POST to the supplied path, serializing requestBody to json and sending
	// that as the HTTP body. the HTTP response will be decoded into reponseBody, and the status
	// code (and potentially an error) returned. It'll try and map errors (statusCode >= 300)
	// into a go error, waits & retries for rate limiting errors will be applied based on the
	// client config.
	// path should be an absolute URI path, i.e. /foo/bar/baz
	Post(ctx context.Context, path string, requestBody any, responseBody any) (http.Header, int, error)
}

PostRequester defines the HTTP POST helper against the configured host.

type PutRequester 

type PutRequester interface {
	// Put makes an HTTP PUT to the supplied path, serializing requestBody to json and sending
	// that as the HTTP body. the HTTP response will be decoded into reponseBody, and the status
	// code (and potentially an error) returned. It'll try and map errors (statusCode >= 300)
	// into a go error, waits & retries for rate limiting errors will be applied based on the
	// client config.
	// path should be an absolute URI path, i.e. /foo/bar/baz
	Put(ctx context.Context, path string, requestBody any, responseBody any) (http.Header, int, error)
}

PutRequester defines the HTTP PUT helper against the configured host.

type ReaderFunc 

type ReaderFunc func() (io.Reader, error)

ReaderFunc returns a fresh reader over the request body; it is called before every attempt so the body can be re-sent on retry.

type Request 

type Request struct {

	// Embed an HTTP request directly. This makes a *Request act exactly
	// like an *http.Request so that all meta methods are supported.
	*http.Request
	// contains filtered or unexported fields
}

Request is an *http.Request whose body can be rewound between retries. It is produced by NewRequest and used internally by Client.Do.

func NewRequest 

func NewRequest(method, url string, rawBody io.ReadSeeker) (*Request, error)

NewRequest creates a Request whose body is rewound (Seek to 0) for each attempt. Content-Length is set when rawBody exposes Len() (bytes.Reader, strings.Reader). The request has no context; set one with WithContext before sending.

func (*Request) AddHeader 

func (r *Request) AddHeader(header, value string) *Request

AddHeader appends a header value to the request (Header.Add).

func (*Request) WithHeaders 

func (r *Request) WithHeaders(headers map[string]string) *Request

WithHeaders appends the given headers to the request (Header.Add).

type RequestPolicy 

type RequestPolicy struct {
	// RetryLimit sets Policy.TotalRetryLimit. Zero keeps the default (5);
	// a negative value disables retries.
	RetryLimit int `json:"retry_limit,omitempty" yaml:"retry_limit,omitempty"`
	// Timeout sets Policy.RequestTimeout (e.g. "2s"). Zero keeps the
	// default (none); a negative value also means no timeout.
	Timeout time.Duration `json:"timeout,omitempty" yaml:"timeout,omitempty"`
}

RequestPolicy is the configurable subset of Policy. When present in ClientConfig, its non-zero values replace TotalRetryLimit and RequestTimeout of DefaultPolicy; zero (or an omitted field) keeps the default.

type Requestor 

type Requestor interface {
	// Do sends the request and returns the response; the caller must close
	// the response body.
	Do(r *http.Request) (*http.Response, error)
}

Requestor is the minimal interface for sending an HTTP request, satisfied by *Client and *http.Client.

type ShouldRetry 

type ShouldRetry func(r *http.Request, resp *http.Response, err error, retries int) (bool, time.Duration, string)

ShouldRetry specifies a policy for handling retries. It is called following each request with the response, error values returned by the http.Client and the number of already made retries (0 on the first call). It returns whether to retry, how long to sleep before the next attempt, and a short reason string used for logging (see the Success, LimitExceeded, ... constants). If ShouldRetry returns false, the Client stops retrying and returns the response to the caller. When retrying, the Client drains (up to 64 KiB) and closes the response body; when it stops it is up to the caller of Do to close the returned response body.

func DefaultShouldRetryFactory 

func DefaultShouldRetryFactory(limit int, wait time.Duration, reason string) ShouldRetry

DefaultShouldRetryFactory returns a ShouldRetry that retries with a fixed wait while the retry count is <= limit, reporting reason.

func RetryAfterShouldRetryFactory added in v1.0.438 

func RetryAfterShouldRetryFactory(limit int, wait, maxWait time.Duration, reason string) ShouldRetry

RetryAfterShouldRetryFactory returns a ShouldRetry that, like DefaultShouldRetryFactory, retries while the retry count is <= limit, but waits for the Retry-After of the response (delay-seconds or an HTTP date; a past date means no wait) and uses wait only when the response has no valid Retry-After. When Retry-After asks for more than maxWait, it does not retry and reports LimitExceeded; a maxWait <= 0 sets no bound. Register it for 429 or 503.

type Storage 

type Storage struct {
	// contains filtered or unexported fields
}

Storage is a folder holding the client's credentials: the access token in ".auth_token" (0600) and DPoP private keys in "<thumbprint>.jwk", plus arbitrary JSON/YAML documents via Marshal/Unmarshal.

func NewStorage added in v0.35.349 

func NewStorage(baseFolder string) *Storage

NewStorage returns a Storage rooted at baseFolder; a leading "~" (alone or followed by a path separator) is expanded to the home directory ($HOME, else the current user's home from the user database), and the folder is used as is when neither is known. "~user" and environment variables are not expanded. An empty folder uses the working directory. A missing folder is created lazily on write with mode 0700; the mode of an existing folder is not changed.

func (*Storage) Clean 

func (c *Storage) Clean()

Clean removes the whole storage folder, including tokens and keys. Errors are ignored.

func (*Storage) Folder added in v0.35.349 

func (c *Storage) Folder() string

Folder returns the storage folder path.

func (*Storage) ListKeys 

func (c *Storage) ListKeys() ([]*KeyInfo, error)

ListKeys returns the DPoP keys found in the storage folder (walked recursively, "*.jwk" files). Unreadable or unsupported keys are skipped and a missing folder yields an empty list; the error is always nil.

func (*Storage) LoadAuthToken 

func (c *Storage) LoadAuthToken() (*AuthToken, string, error)

LoadAuthToken reads and parses the .auth_token file in the storage folder; see the package-level LoadAuthToken.

func (*Storage) LoadKey 

func (c *Storage) LoadKey(label string) (*jose.JSONWebKey, string, error)

LoadKey loads the DPoP private key stored as "<label>.jwk" (label is normally the key thumbprint) and returns the key and its file path.

func (*Storage) Marshal added in v0.35.349 

func (c *Storage) Marshal(file string, v any) error

Marshal encodes v as JSON (".json" suffix) or YAML into file, relative to the storage folder. The folder must already exist.

func (*Storage) SaveAuthToken 

func (c *Storage) SaveAuthToken(token string) (string, error)

SaveAuthToken writes the raw token to the .auth_token file (mode 0600), creating the folder if needed, and returns the file location. An existing file or symlink is atomically replaced with a private regular file. The token can be an opaque string, or form encoded as access_token={token}&exp={unix_time}&dpop_jkt={jkt}&token_type={Bearer|DPoP} (see ParseAuthToken).

func (*Storage) SaveKey 

func (c *Storage) SaveKey(k *jose.JSONWebKey) (string, error)

SaveKey writes the DPoP private key to "<thumbprint>.jwk" in the storage folder (created with mode 0700 if needed) and returns the file path. An existing file or symlink is atomically replaced with a private regular file.

func (*Storage) Unmarshal added in v0.35.349 

func (c *Storage) Unmarshal(file string, v any) error

Unmarshal decodes the JSON (".json") or YAML file, relative to the storage folder, into v.

type TLSInfo 

type TLSInfo struct {
	// CertFile specifies location of the cert
	CertFile string `json:"cert,omitempty" yaml:"cert,omitempty"`

	// KeyFile specifies location of the key
	KeyFile string `json:"key,omitempty" yaml:"key,omitempty"`

	// TrustedCAFile specifies location of the trusted Root file
	TrustedCAFile string `json:"trusted_ca,omitempty" yaml:"trusted_ca,omitempty"`
}

TLSInfo specifies the client certificate and trust anchors, as file paths.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.