Documentation
¶
Overview ¶
Package httpclient is a resilient outbound-HTTP toolkit: transient-error classification, retries over one jittered exponential backoff loop, Retry-After parsing, status mapping, secret redaction, body helpers, custom-CA TLS transports and redirect allowlists. They live together because they compose into one net/http.Client. The revalidation cache that composes with them is the httpcache module (github.com/cplieger/go/httpcache).
Retry doors ¶
Three entry points share one option vocabulary and one backoff progression, and passing an option to the wrong door does not compile:
- Do retries a typed operation. You build the requests and keep the typed result.
- GetBytes is a bounded-bytes GET with redacted diagnostics. It owns the request, the body cap and the close.
- NewRetryRoundTripper retries transparently beneath any client, under a TransportConfig.
New and NewRetry build clients with a redirect policy in place.
Errors ¶
IsTransient decides retryability, extended through the Transient and RetryAfterHint interfaces or MarkTransient. Permanent marks an error non-retryable. A context deadline is the total budget and is never retried, while AttemptTimeout marks the end of one attempt, which is. For every non-2xx status, CheckStatus and GetBytes return an error whose chain holds a *StatusError, matched with errors.As or with errors.Is against ErrUnauthorized, ErrForbidden, ErrRateLimited and ErrServerError.
Secrets ¶
RedactSecret, RedactSecretString, RedactTransportError and LogSafeError keep credentials out of logs and errors, and GetBytes never logs or returns a raw URL. A caller adding a normalizing transform or a byte cap follows the order on RedactSecretString: redact, normalize, redact, cap.
The README groups the API by task and pkg.go.dev lists every identifier. docs/retries.md, docs/timeouts.md, docs/responses.md and docs/security-model.md hold the full contracts.
Index ¶
- Constants
- Variables
- func AttemptTimeout(err error) error
- func CATransport(pem []byte) (*http.Transport, error)
- func CheckStatus(resp *http.Response) error
- func CloneDefaultTransport() (*http.Transport, error)
- func ContextWithDefaultTimeout(ctx context.Context, def time.Duration) (context.Context, context.CancelFunc)
- func DefaultRedirectPolicy(req *http.Request, via []*http.Request) error
- func Do[T any](ctx context.Context, fn func(ctx context.Context) (T, error), opts ...DoOption) (T, error)
- func Drain(body io.ReadCloser)
- func DrainClose(rc io.ReadCloser)
- func GetBytes(ctx context.Context, client *http.Client, reqURL string, opts ...GetOption) ([]byte, error)
- func IsAttemptTimeout(err error) bool
- func IsPermanent(err error) bool
- func IsRetryableStatus(code int) bool
- func IsTransient(err error) bool
- func JitteredBackoff(backoff time.Duration) time.Duration
- func LogSafeError(err error) error
- func MarkTransient(err error) error
- func New(timeout time.Duration) *http.Client
- func NewRetry(base http.RoundTripper, policy CheckRedirect, cfg TransportConfig) *http.Client
- func ParseRetryAfter(h string) time.Duration
- func ParseRetryAfterResponse(resp *http.Response) time.Duration
- func Permanent(err error) error
- func ReadLimitedBody(body io.ReadCloser, limit int64) ([]byte, error)
- func RedactSecret(err error, secret string) error
- func RedactSecretString(s string, secret Secret) string
- func RedactTransportError(err error, prefix string, secret Secret) error
- func RefuseAllRedirects(*http.Request, []*http.Request) error
- func SafeDouble(d time.Duration) time.Duration
- func SleepCtx(ctx context.Context, d time.Duration) error
- type Cause
- type CheckRedirect
- type CheckRetry
- type DoOption
- type GetOption
- type OnRetry
- type Option
- type PrepareRetry
- type RedirectOption
- type RedirectReason
- type RedirectRefusedError
- type ResponseTooLargeError
- type RetryAfterHint
- type RetryRoundTripper
- type Secret
- type StatusError
- type Transient
- type TransportConfig
Examples ¶
Constants ¶
const RetryAfterCap = 60 * time.Second
RetryAfterCap is the longest wait ParseRetryAfter returns, so the longest Retry-After GetBytes and the retrying transport honor. ParseRetryAfterResponse and StatusError.RetryAfter keep the raw value; Do's rate-limit options cap it at their maxWait, and a non-positive maxWait takes RetryAfterCap. Its value is part of the v1 API, so a caller may derive its own bounds from it (a server timeout covering the longest Retry-After wait of GetBytes or the transport).
Variables ¶
var ( ErrUnauthorized = errors.New("unauthorized") // ErrForbidden matches a 403. ErrForbidden = errors.New("forbidden") // ErrRateLimited matches a 429. ErrRateLimited = errors.New("rate limited") // ErrServerError matches a 5xx, and a status of 600 or more, which RFC 9110 // section 15 says to process as a 5xx. Which of them are retried is // [IsRetryableStatus]'s rule. ErrServerError = errors.New("server error") )
Sentinels a *StatusError matches with errors.Is, by its Code.
var ErrMaxElapsed = errors.New("max elapsed time exceeded")
ErrMaxElapsed is the error a RetryRoundTripper returns, wrapped with the ceiling and the last attempt's error when there is one, when its next wait would meet or pass TransportConfig.MaxElapsedTime. Match it with errors.Is.
var ErrNoCertsInPEM = errors.New("httpclient: no PEM-encoded certificates found")
ErrNoCertsInPEM is returned by CATransport when the supplied PEM data contains no parseable certificates. A misconfigured or empty CA file therefore fails loudly rather than silently producing a transport that trusts nothing (which would reject every connection with an opaque error).
Functions ¶
func AttemptTimeout ¶
AttemptTimeout wraps err to declare that the timeout it reports bounded ONE ATTEMPT rather than the caller's total budget, making it retryable: IsTransient reports true for it (checked ahead of the context-error rejection), and the retry doors retry it. Returns nil for nil input. It is the mirror of Permanent, which forces the opposite verdict.
It exists because a context deadline alone cannot express the difference. A caller's expired deadline means "the budget is gone, stop"; the expiry of a per-attempt bound means "this attempt failed, try another" — opposite instructions from the same context.DeadlineExceeded. Only the code that installed the bound knows which one it is, so httpclient cannot infer it and defaults to the safe reading (terminal). This mark is how the installer says otherwise, and the wrapper deliberately keeps the deadline visible to errors.Is instead of hiding it, which was the only way to opt in before.
Reach for it when the per-attempt bound is NOT the retry door's own and httpclient cannot see whose bound it was:
- a context.WithTimeout a caller derives itself inside a Do callback. Its expiry carries the deadline sentinel, indistinguishable from the caller's own. WithAttemptTimeout does this for you, mark included; hand-roll it only when the bound must cover something other than one whole attempt.
- a net-level bound such as a net.Dialer Timeout. The net package maps every expired context onto one shared "i/o timeout" value, so a *net.OpError or *net.DNSError reporting a deadline cannot say whose it was and stays terminal (see IsTransient).
An http.Client.Timeout and a net/http.Transport ResponseHeaderTimeout do NOT need the mark: net/http reports those through its own timeout error, which never carries the sentinel, so IsTransient already reads them as per-attempt and retries them — including under the RetryRoundTripper's default policy. Use Permanent (or a TransportConfig.CheckRetry) if you want one of them to stop the loop instead.
The caller owns the judgment: mark ONLY a bound you installed over a single attempt. Marking a deadline that came from your own caller makes an exhausted budget look retryable, which is the failure mode the default classification prevents. WithAttemptTimeout avoids the judgment entirely by deciding from the two contexts it owns.
func CATransport ¶
CATransport builds an *http.Transport that trusts ONLY the CA certificate(s) in pem for TLS verification (pinning). It is cloned from http.DefaultTransport (via CloneDefaultTransport), so it keeps the standard connection pooling, dial/keepalive timeouts, HTTP/2 negotiation, and proxy support (ProxyFromEnvironment). A FRESH TLS config is then installed — RootCAs from pem, a TLS 1.2 minimum, and verification always ENABLED (InsecureSkipVerify is not set). Any TLS settings already on http.DefaultTransport are intentionally NOT carried over, so the returned transport's trust posture cannot be weakened by a program that globally mutated the default transport's *tls.Config (e.g. set InsecureSkipVerify or an accept-all VerifyPeerCertificate hook).
The supplied CA(s) are the SOLE trust anchors: the transport rejects any host not chaining to them, including public-CA hosts. This is the right setup for a known self-hosted endpoint presenting a private or self-signed certificate (a Plex server, an internal API).
The returned *http.Transport is concrete and mutable, so callers may tune fields such as MaxIdleConnsPerHost or pass it as the base RoundTripper of NewRetryRoundTripper:
tr, err := httpclient.CATransport(pem)
client := httpclient.NewRetry(tr, httpclient.DefaultRedirectPolicy, httpclient.TransportConfig{MaxAttempts: 3})
It returns ErrNoCertsInPEM when pem yields no certificates. The caller owns reading the PEM bytes (from a file, a secret, an env var), which keeps this function I/O-free and lets the caller bound the read as it sees fit.
CATransport requires http.DefaultTransport to be the standard library's *http.Transport (the default). If your program has replaced it with a wrapping RoundTripper (for example request instrumentation), CATransport returns an error.
func CheckStatus ¶
CheckStatus returns nil for a 2xx and otherwise a *StatusError carrying no URL, so its text carries no URL into a log. A 3xx reaches it only from a client that does not follow redirects (see RefuseAllRedirects) and is reported as a failure, not a completed request.
func CloneDefaultTransport ¶
CloneDefaultTransport returns a private clone of http.DefaultTransport, keeping the standard connection pooling, dial/keepalive timeouts, HTTP/2 negotiation, and proxy support (ProxyFromEnvironment). The clone is the caller's to mutate — set a per-attempt ResponseHeaderTimeout, tune MaxIdleConnsPerHost, or pass it as the base RoundTripper of NewRetryRoundTripper — without the global footgun of mutating http.DefaultTransport itself, which would silently reconfigure every other client in the process.
It returns an error when http.DefaultTransport has been replaced by a non-*http.Transport RoundTripper (request instrumentation, a test stub): a wrapping RoundTripper offers no concrete transport to clone, and failing loudly beats silently dropping the wrapper's behavior. It is the building block CATransport is assembled on.
func ContextWithDefaultTimeout ¶
func ContextWithDefaultTimeout(ctx context.Context, def time.Duration) (context.Context, context.CancelFunc)
ContextWithDefaultTimeout bounds ctx by def ONLY when ctx carries no deadline; a deadline the caller already set is the authoritative budget and is never undercut (or extended). A def <= 0 means "no default" and leaves ctx unbounded. The returned cancel is always non-nil and must be called (a no-op on the passthrough paths), matching context.WithTimeout's contract.
It is the per-request timeout rule for an API client: per-attempt bounds live on the transport (ResponseHeaderTimeout), the total budget is the caller's ctx, and this default applies only when the caller brought no budget of its own. Extracted from the identical requestContext helpers in the plexapi and arrapi client libraries.
func DefaultRedirectPolicy ¶
DefaultRedirectPolicy is the default redirect policy: it allows a redirect only to the same host as the original request, and refuses a same-host https->http scheme downgrade (which would forward a custom auth header onto a cleartext hop). A cross-host redirect is refused (Go forwards a custom header across a redirect, so it would leak) and an http->https upgrade is allowed. It delegates to RedirectPolicy(WithSameHost(true)), with one addition: a call with an empty via chain (which net/http never produces — via always carries at least the original request) is allowed rather than refused. Use RedirectPolicy for a custom allowlist or a higher hop cap (WithMaxHops).
func Do ¶
func Do[T any](ctx context.Context, fn func(ctx context.Context) (T, error), opts ...DoOption) (T, error)
Do calls fn up to WithMaxAttempts times (total) with jittered exponential backoff and returns the first success; a non-retryable error returns at once. By default only IsTransient errors retry (a 429 is not transient) and a positive RetryAfterHint replaces the backoff wait. After a failed attempt a canceled context returns ctx.Err(), except under WithRateLimitOnly, where the final attempt's error wins. Retries log at Debug to WithLogger; exhaustion logs at Warn, or Debug for a single-attempt budget (see exhaustedLevel).
Example ¶
package main
import (
"context"
"fmt"
"time"
"github.com/cplieger/go/httpclient"
)
func main() {
attempts := 0
// context.Background() (not t.Context()): an Example has no *testing.T.
result, err := httpclient.Do(context.Background(), func(_ context.Context) (string, error) {
attempts++
if attempts < 2 {
return "", &httpclient.StatusError{Code: 503}
}
return "done", nil
}, httpclient.WithMaxAttempts(3), httpclient.WithBaseDelay(time.Millisecond), httpclient.WithLabel("example"))
if err != nil {
fmt.Println("error:", err)
return
}
fmt.Println(result)
}
Output: done
func Drain ¶
func Drain(body io.ReadCloser)
Drain reads and discards up to 64 KB of a response body to enable HTTP connection reuse. A failed drain forfeits only that reuse, and is reported as a bare Debug line on slog.Default(). The read error itself is deliberately NOT logged.
Go 1.27 made HTTP/1 Response.Body drain itself on Close, which raises the question of whether this still earns its place. It does, on two measured grounds. The stdlib drain is HTTP/1 only — net/http/internal/http2 has no equivalent — so on an HTTP/2 response, which every transport this package builds negotiates (CloneDefaultTransport keeps ForceAttemptHTTP2), this is the only drain there is. And Drain takes an io.ReadCloser, not an *http.Response, so it also covers a body the stdlib never wraps. Where both apply they compose rather than collide: this drain runs first and, for a body inside the 64 KB budget, reaches EOF, which short-circuits the stdlib's drain entirely.
The two limits differ deliberately and do not need reconciling. 64 KB is how much effort this function spends; net/http's own budget is 256 KiB with a 50 ms deadline, and it applies only when unread content remains AND resp.ContentLength <= 256 KiB. That gate is why an over-limit body refused by ReadLimitedBody reads nothing extra: a response declaring more than 256 KiB is excluded outright. Measured on go1.27.0 against go1.26.7, a declared 10 MB body read to a 1 MB cap pulls an identical 1,114,112 bytes on both. A CHUNKED over-limit body has ContentLength == -1, which passes the gate, so its worst case is the cap plus at most 256 KiB more. Either way the caller pays no latency for it: the drain runs on net/http's read-loop goroutine, and Close() measured 24-44 microseconds against a deliberately stalled chunked body that provably passes the gate.
That omission is a CWE-532 fix, not an oversight. A body-read error's text is written by the FAR END: net/http renders a malformed chunked trailer as
malformed MIME header: missing colon: "<remote bytes>"
and for a consumer whose request URL carries its credential in the PATH (a webhook token is the canonical shape) an edge that echoes the request URI puts that credential in those bytes. Logging it needs only the consumer to be running at Debug — the level an operator raises precisely while diagnosing failing deliveries. Three properties make the site uncloseable from anywhere else, which is why the value is dropped HERE rather than by the caller:
- Nobody can route it. Drain takes no logger (it is called from defer at consumer call sites and from four sites inside this package, one of them a RoundTripper that has no logger at all), and the retry doors' WithLogger governs their own loop lines, not slog.Default().
- LogSafeError does not reduce it. That boundary is TYPE-based: it strips the *url.Error envelope this package's own machinery adds. A body-read error is not a *url.Error — a malformed trailer surfaces as textproto.ProtocolError, whose entire value IS the message — so the reduction returns it byte-identically.
- Redaction needs a secret to redact, and Drain is handed none.
No diagnosis a caller can act on is lost. Drain runs only where the body is being thrown away, so the response's own outcome is already reported by the path that discarded it (CheckStatus's error, the retry lines, ReadLimitedBody's error); a drain error is never the only signal of a failure, and no caller can repair a drain. This is the same call the package makes in urlErrorCause, which substitutes a contentless stand-in rather than log the one field it holds.
func DrainClose ¶
func DrainClose(rc io.ReadCloser)
DrainClose reads remaining bytes (up to drainLimit) from rc before closing it. It inherits Drain's logging contract: a failed drain logs a bare Debug line carrying no remote-authored text.
func GetBytes ¶
func GetBytes(ctx context.Context, client *http.Client, reqURL string, opts ...GetOption) ([]byte, error)
GetBytes performs a GET, retrying every status IsRetryableStatus accepts and transient transport errors with bounded backoff (honoring Retry-After up to RetryAfterCap), and returns the body read to WithMaxBodyBytes (*ResponseTooLargeError past it). Any other non-2xx is a permanent *StatusError. Every logged url and returned error is redacted. Each attempt is judged as the RetryRoundTripper's default policy judges a round trip. client must not be nil: pass New(timeout) or a client of your own, never http.DefaultClient, whose nil CheckRedirect follows a redirect to any host.
Example ¶
package main
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"time"
"github.com/cplieger/go/httpclient"
)
func main() {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
fmt.Fprint(w, "hello")
}))
defer srv.Close()
// context.Background() (not t.Context()): an Example has no *testing.T.
body, err := httpclient.GetBytes(context.Background(), srv.Client(), srv.URL,
httpclient.WithMaxAttempts(3),
httpclient.WithBaseDelay(100*time.Millisecond),
)
if err != nil {
fmt.Println("error:", err)
return
}
fmt.Println(string(body))
}
Output: hello
func IsAttemptTimeout ¶
IsAttemptTimeout reports whether err (or any wrapped error) was marked by AttemptTimeout as a per-attempt timeout.
func IsPermanent ¶
IsPermanent reports whether err, or an error it wraps, was marked by Permanent.
func IsRetryableStatus ¶
IsRetryableStatus is the one status rule every retry door applies: true for 408, 429 and 500 to 599 less the codes whose answer repeats, false for every other status. A code of 600 or more is a server error (ErrServerError) but not a valid status, so it is not retried. GetBytes and the RetryRoundTripper's default policy call it on each response, and StatusError.IsTransient reports it less 429.
func IsTransient ¶
IsTransient reports whether err is worth another attempt: a Transient that says so, an AttemptTimeout mark, or an error whose CauseOf is DNS, refused, reset or timeout. A Permanent mark, a *StatusError for 401, 403 or 429 and the caller's own expired or canceled context are never transient. The caller's deadline is the sentinel itself in the chain, so an http.Client.Timeout or a ResponseHeaderTimeout, which only match it, are per-attempt bounds and retried (see isCallerContextError).
func JitteredBackoff ¶
JitteredBackoff returns a duration in [backoff/2, backoff] using the "equal jitter" strategy (per AWS Builders' Library). Full jitter and decorrelated jitter are intentionally not provided — equal jitter is the recommended default for HTTP retry as it avoids thundering herd while maintaining a minimum backoff floor.
func LogSafeError ¶
LogSafeError returns an error whose message is safe to log. Every *url.Error in err's chain, which renders the raw request URL, is replaced by its cause. The marks this package renders on the fly (Permanent, MarkTransient, AttemptTimeout) and each branch of an errors.Join are kept around the reduced cause, so IsPermanent, IsTransient and every sibling survive. Any other wrapper above a *url.Error is dropped, because its message was rendered with the URL in it: reduce a transport error before you wrap it with your own context. An error with no *url.Error is returned unchanged, a chain nested more than eight *url.Error deep reduces to a contentless "request failed", nil returns nil, and a non-nil error never reduces to nil. It equals RedactTransportError(err, "", ""), the empty-secret case, which is the reduction httpclient applies to every transport error it logs or wraps.
func MarkTransient ¶
MarkTransient wraps err so IsTransient reports true, overriding a non-transient verdict already on it, for a failure your operation knows self-heals where the status rule cannot see it (a server fault inside a 200 envelope). Permanent, a *StatusError for 401, 403 or 429 and a caller context error still win; for a per-attempt deadline use AttemptTimeout. Returns nil for nil input.
func New ¶
New returns an *http.Client with the given timeout and the DefaultRedirectPolicy (same-host only). For a custom redirect allowlist, configure CheckRedirect with RedirectPolicy.
func NewRetry ¶
func NewRetry(base http.RoundTripper, policy CheckRedirect, cfg TransportConfig) *http.Client
NewRetry returns an *http.Client whose Transport is a RetryRoundTripper over base (nil means http.DefaultTransport) and whose CheckRedirect is policy. It panics on a nil policy, because nil means net/http's follow-anywhere default; pass DefaultRedirectPolicy, RefuseAllRedirects or a RedirectPolicy. It sets no Client.Timeout, which would cap the whole retry sequence: bound one attempt on the base transport and the total with a context deadline or TransportConfig.MaxElapsedTime.
func ParseRetryAfter ¶
ParseRetryAfter parses a Retry-After header value (delta-seconds or HTTP-date). Returns zero for missing/malformed values. Caps at RetryAfterCap for safety (prevents unbounded waits in retry loops). For raw uncapped values, use ParseRetryAfterResponse.
func ParseRetryAfterResponse ¶
ParseRetryAfterResponse parses the Retry-After header from an *http.Response. Returns zero if absent or unparseable. Does NOT cap — preserves the raw duration for StatusError's RetryAfter, so callers make their own decisions. For capped values suitable for retry loops, use ParseRetryAfter. A nil resp returns zero.
func Permanent ¶
Permanent wraps err so that no retry door repeats it, whatever else it carries: IsTransient reports false for it, and the rate-limit options do not retry a marked 429. Returns nil for nil input. It is the mirror of AttemptTimeout, which forces the opposite verdict.
func ReadLimitedBody ¶
func ReadLimitedBody(body io.ReadCloser, limit int64) ([]byte, error)
ReadLimitedBody reads body up to limit bytes, always closes body, and returns the bytes read. It reads one byte past limit to detect an over-limit body and returns *ResponseTooLargeError (with nil bytes) rather than a silently truncated payload — a truncated body indistinguishable from a complete one is a corruption hazard. A limit of math.MaxInt64 means "effectively unlimited" and is guarded against probe-size overflow.
It is the same cap+1 read GetBytes applies internally, exposed for callers that issue their own request and decode outside GetBytes but still want the fail-loud size bound. On any error the body is already closed.
func RedactSecret ¶
RedactSecret replaces occurrences of secret in err's message with "REDACTED", as RedactTransportError(err, "", Secret(secret)) does: a message that held the secret comes back as a new error with no chain, so classify err first. It funnels into RedactSecretString, whose doc comment carries the ordering and representation rules the caller owns.
func RedactSecretString ¶
RedactSecretString replaces every occurrence of secret in s with "REDACTED" and returns the result. It is the string-level building block behind RedactSecret and RedactTransportError, exposed for callers that must redact a secret from a plain string — a captured HTTP response body destined for an error field or a log line — rather than from an error value. An empty secret is a no-op (s is returned unchanged), matching the error-shaped variants.
The replacement is literal and byte-exact, which puts three obligations on the caller:
- Hold the needle in the SAME representation the haystack carries. A decoded token does not match a JSON-escaped or percent-encoded haystack, and the mismatch is silent: nothing is replaced and no error is reported.
- Call this on BOTH sides of any normalizing transform (textsafe Sanitize/SanitizeSingleLine, unicode/norm NFC/NFKC, strconv.Unquote, JSON/URL/HTML unescape, case folding, whitespace collapse), passing the needle in the representation each side carries. A call placed only before the transform misses a secret the transform CONSTRUCTS out of text the needle never matched; a call placed only after it misses a secret the transform REWRITES, leaving a near-complete fragment of the value.
- Put any byte cap (ReadLimitedBody, a bounded sanitizer preset, a plain truncation) LAST. A cut that splits the value leaves a surviving prefix this function can no longer match.
Composed, that order is: redact, normalize, redact, cap.
This package performs one of those transforms ¶
Every URL httpclient renders — *StatusError.Error(), and every "url" attribute in a GetBytes log line — goes through a parse-and-re-serialize step that blanks query values and drops userinfo. That step is a URL-encoding transform in rule 2's sense, so redacting a known secret out of one of those strings is the "only after the transform" mistake. Measured on go1.27.0 (identical on go1.26.7): for a secret in the request PATH, which is the one credential-bearing position the rendering keeps verbatim, a secret containing a space renders as tok%20en, a non-ASCII byte as tok%C3%A9n, and a '#' truncates the value at the fragment boundary — so a byte-exact needle held in the caller's own representation matches none of them and the value reaches the log percent-encoded. A stray '%' is the safe case: the URL fails to parse and the whole rendering collapses to a fixed placeholder.
So a caller holding a credential that rides in a URL PATH must redact it before the URL reaches this package (or pass the needle in its percent-encoded form as well). A credential in the query string or the userinfo needs nothing: the rendering removes those wholesale, without matching anything.
func RedactTransportError ¶
RedactTransportError reduces err as LogSafeError does, prefixes it, and redacts the secret from its message. Returns nil for nil input, and never nil for a non-nil one. When the message holds the secret, the result is a new error carrying only the redacted text, because every error in the chain still renders the secret: errors.Is, errors.As, IsTransient and CauseOf stop there, so classify err before you redact it. When the message does not hold the secret, the reduced and prefixed chain is returned whole. The replacement runs through RedactSecretString, whose doc comment carries the ordering and representation rules for composing redaction with a normalizing transform or a byte cap. An empty secret disables redaction: the (possibly prefixed) error is returned with its message untouched, since there is nothing to scan for.
func RefuseAllRedirects ¶
RefuseAllRedirects is a CheckRedirect policy that follows no redirect: the client returns the 3xx response itself with a nil error. Use it for a token-bearing client of an API that never redirects, because net/http forwards custom headers (X-Plex-Token, X-Api-Key) across a redirect and a hostile 302 would leak them. CheckStatus reports the surfaced 3xx as an error.
func SafeDouble ¶
SafeDouble doubles a duration, guarding against int64 overflow.
Types ¶
type Cause ¶
type Cause uint8
Cause names why a request failed below HTTP. Status answers are CheckStatus's; a Cause names only what happened to the connection or the body. It encodes as text by its name, so a JSON log field or stored state carries "refused", never the number.
const ( CauseUnknown Cause = iota // nil, or an error no other cause names CauseDNS // the host name did not resolve CauseRefused // nothing accepted the connection CauseReset // the peer reset or closed the connection mid-exchange CauseTimeout // a deadline or timeout expired CauseTLS // the TLS handshake failed: a certificate, an alert, or a plain-HTTP answer CauseTooLarge // a body went over its cap )
The causes CauseOf reports.
func CauseOf ¶
CauseOf names the first cause it recognizes in err's chain, trying too large, TLS, DNS, refused, reset, then timeout, so a DNS lookup that timed out reads CauseDNS. It sees through Permanent, MarkTransient and AttemptTimeout, and answers CauseUnknown for nil, a canceled context and a status error.
func (Cause) MarshalText ¶
MarshalText returns the cause's name, and refuses a value outside the constants.
func (Cause) String ¶
String returns the cause's snake_case name, or Cause(n) for a value outside the constants.
func (*Cause) UnmarshalText ¶
UnmarshalText reads a name Cause.MarshalText writes, and refuses any other.
type CheckRedirect ¶
CheckRedirect is the http.Client.CheckRedirect function shape. It is a type alias, so values are assignable in both directions; every shipped policy (DefaultRedirectPolicy, RefuseAllRedirects and anything built with RedirectPolicy) is a CheckRedirect.
func RedirectPolicy ¶
func RedirectPolicy(opts ...RedirectOption) CheckRedirect
RedirectPolicy returns a CheckRedirect configured with the given options; a nil option is skipped. A redirect is followed only to an allowed target (a WithAllowedHosts entry, a WithAllowedSuffixes match, the original host under WithSameHost, the original origin under WithSameOrigin) and never https->http, so a custom auth header, which Go forwards across a redirect, never rides a cleartext hop. With none of those permissions every redirect is refused. The hop cap is WithMaxHops. Every hard refusal is a *RedirectRefusedError; WithPreserveMethod(true) instead answers http.ErrUseLastResponse for a hop that would change the method.
type CheckRetry ¶
CheckRetry is the signature for a retry policy function. It receives the context, the response (may be nil on transport error), and the error (nil on successful response). It returns whether to retry and an optional error to short-circuit with. Mirrors hashicorp/go-retryablehttp CheckRetry.
type DoOption ¶
type DoOption interface {
// contains filtered or unexported methods
}
DoOption configures Do. Options constructed as Option values apply to both loop doors; Do-only options (WithLabel, WithRateLimitRetry, WithRateLimitOnly, WithAttemptTimeout) implement only this interface, so passing one to GetBytes is a compile error.
func WithAttemptTimeout ¶
WithAttemptTimeout bounds EACH attempt by d and makes that bound's expiry RETRYABLE: the per-attempt (per-try) timeout, as distinct from the caller's total budget. It is the piece that joins a per-attempt deadline to the retry loop; without it a deadline inside fn is classified terminal (see AttemptTimeout for why an error alone cannot say otherwise). A non-positive d means no per-attempt bound, which is the option's absence.
fn receives a context derived from the caller's with d applied. Context keeps the EARLIER deadline, so a caller deadline nearer than d still governs and the total budget is never extended — d caps one attempt, never the call.
The expiry is marked as the attempt's ONLY when the caller's own context is still live, so retrying a caller that is out of budget stays impossible: an expired or canceled caller context is never marked, Do returns ctx.Err() before it classifies anything, and SleepCtx refuses to wait on a dead context. The decision is made from the two contexts rather than from the error, because a context.DeadlineExceeded value is identical either way.
The attempt context is canceled as soon as fn returns, so fn must not return a value that still depends on it — read, decode, or drain a response body inside fn (the same rule a caller-derived per-attempt context already imposes).
It is a DoOption: GetBytes owns its own request and has no callback to bound. Bound that door by running it as one attempt inside Do — GetBytes(ctx, ...) with WithMaxAttempts(1) inside a Do callback carrying this option — the same composition that avoids multiplying two attempt budgets.
func WithLabel ¶
WithLabel sets the operation label used in Do's log lines ("<label> failed, retrying", "<label> retries exhausted"). Default: "operation".
func WithRateLimitOnly ¶
WithRateLimitOnly makes Do retry ONLY a 429 *StatusError (matched through wrapped errors); every other error, including transient transport errors, is returned immediately. The wait semantics match WithRateLimitRetry, the terminal Warn is "rate limit retries exhausted", and the FINAL attempt's error wins even under an already-canceled context (cancellation is observed in the always-positive inter-attempt sleep instead). The later of WithRateLimitOnly and WithRateLimitRetry wins.
func WithRateLimitRetry ¶
WithRateLimitRetry makes Do also retry a 429 *StatusError, waiting min(RetryAfter, maxWait) when it carries a positive RetryAfter, else maxWait. A non-positive maxWait means RetryAfterCap, so the wait is always positive and a canceled context is observed before every retry. Transient errors keep their usual waits (the backoff, or a positive RetryAfterHint), and the exponential base advances on every retry. WithRateLimitRetry and WithRateLimitOnly set one mode: the later of the two wins, with its maxWait.
type GetOption ¶
type GetOption interface {
// contains filtered or unexported methods
}
GetOption configures GetBytes. GetBytes-only options (WithHeaders, WithMaxBodyBytes) implement only this interface, so passing one to Do is a compile error.
func WithHeaders ¶
WithHeaders sets a function that is called to set headers on each request.
func WithMaxBodyBytes ¶
WithMaxBodyBytes sets the maximum response body size to read. Default: 10 MB (10 << 20 bytes). A non-positive value falls back to the default.
type OnRetry ¶
OnRetry is called before each retry attempt (not the initial request). attempt is 1-indexed (first retry = 1). resp may be nil on transport errors.
type Option ¶
Option is a retry option accepted by BOTH loop doors, Do and GetBytes: WithMaxAttempts, WithBaseDelay, WithLogger and WithExhaustedLevel construct Option values.
func WithBaseDelay ¶
WithBaseDelay sets the initial backoff delay. Default: one second. A non-positive value falls back to the default.
func WithExhaustedLevel ¶
WithExhaustedLevel overrides the level of the terminal "retries exhausted" line (see exhaustedLevel for the default rule).
It exists for the caller that reports the same terminal failure itself, with more context than this door can have. Such a caller wants the per-attempt retry diagnostics and NOT the verdict: leaving both produces two log lines for one failure, and silencing the whole logger to stop the second throws the diagnostics away with it. Pass slog.LevelDebug to keep the line for diagnosis while the caller's own line carries the alarm.
It does not change what is returned, and it does not suppress the line: a level below the logger's threshold simply is not emitted, which is the caller's decision to make.
func WithLogger ¶
WithLogger sets the logger for retry diagnostics. Default: slog.Default(). A nil logger falls back to the default.
func WithMaxAttempts ¶
WithMaxAttempts sets the maximum number of attempts (TOTAL, including the first call). Default: 3. A value below 1 is treated as 1, so the operation always runs at least once (never a silent no-op).
type PrepareRetry ¶
PrepareRetry is called before each retry to allow request mutation (e.g., re-signing with a fresh token). Mirrors go-retryablehttp PrepareRetry.
type RedirectOption ¶
type RedirectOption interface {
// contains filtered or unexported methods
}
RedirectOption configures a redirect policy built by RedirectPolicy. It is sealed like DoOption: only this package's With* redirect options implement it.
func WithAllowedHosts ¶
func WithAllowedHosts(hosts ...string) RedirectOption
WithAllowedHosts sets the exact hostnames allowed as redirect targets. A name matches without regard to ASCII case; an IP address entry, written without brackets ("2001:db8::a", "fe80::1%eth0"), matches by address, its IPv6 zone exactly. An empty entry matches nothing.
func WithAllowedSuffixes ¶
func WithAllowedSuffixes(suffixes ...string) RedirectOption
WithAllowedSuffixes sets the domain suffixes allowed (e.g. ".docker.com"). A suffix matches a host name only, never an IP literal.
func WithMaxHops ¶
func WithMaxHops(n int) RedirectOption
WithMaxHops makes the policy follow at most n redirects in one request and refuse the next with RedirectTooMany. n <= 0 means the default, 5.
func WithPreserveMethod ¶
func WithPreserveMethod(preserve bool) RedirectOption
WithPreserveMethod refuses, when true, a hop that would change the request method, as net/http does to a POST, PUT, PATCH or DELETE across a 301, 302 or 303 (RFC 9110 §15.4, https://go.dev/issue/18570). It returns http.ErrUseLastResponse, so the caller gets the 3xx with a nil error, as with RefuseAllRedirects. The method is compared with the original request (via[0]) and an empty via fails closed. The hop cap, target and downgrade refusals take precedence, and it grants nothing: with no allowlist, WithSameHost or WithSameOrigin every redirect is still refused.
func WithSameHost ¶
func WithSameHost(same bool) RedirectOption
WithSameHost permits a redirect whose target host equals the original request's host, as WithSameOrigin compares hosts, beside any allowlist entries. It follows that host on any port and an http->https upgrade there, so a custom auth header can reach another service on the host; WithSameOrigin keeps it on one origin. An https->http hop is still refused. Later options win, so a later WithSameHost(false) turns the permission back off.
func WithSameOrigin ¶
func WithSameOrigin(same bool) RedirectOption
WithSameOrigin permits a redirect to the original request's origin: its scheme, its port (absent reads 80 for http, 443 for https) and its host, a name compared ASCII case-insensitively and an IP literal by address with its IPv6 zone, a local implementation-defined identifier (RFC 4007 section 11.2), compared byte for byte. Unlike WithSameHost it refuses another port and an http->https upgrade. When set, the origin alone decides a hop to the original host, whatever WithSameHost or WithAllowedHosts say; the allowlist and the downgrade rule judge other hosts, and a refusal is RedirectOtherOrigin.
type RedirectReason ¶
type RedirectReason int
RedirectReason names the rule of a redirect policy that refused a hop.
const ( // RedirectNotAllowed means the policy follows no redirect at all (no // allowlist, no WithSameHost(true), no WithSameOrigin(true)). RedirectNotAllowed RedirectReason = iota + 1 // RedirectOtherOrigin means the target is neither an allowed host nor, under // WithSameHost or WithSameOrigin, the original request's host or origin. RedirectOtherOrigin // RedirectDowngrade means an https->http hop to an otherwise allowed target. RedirectDowngrade // RedirectTooMany means the hop would exceed the policy's hop cap. RedirectTooMany )
func (RedirectReason) String ¶
func (r RedirectReason) String() string
String returns the reason's snake_case name, as Cause.String does, or RedirectReason(n) for a value outside the constants.
type RedirectRefusedError ¶
type RedirectRefusedError struct {
// Origin is the refused target's origin, "scheme://host" plus ":port"
// when the port is not the scheme's default, escaped so [url.Parse]
// reads it back. It never holds the target's userinfo, path or query, so
// Error is safe to log for a target that carried a credential there.
Origin string
// Reason is the rule that refused the hop.
Reason RedirectReason
}
RedirectRefusedError is a shipped redirect policy's hard refusal of one hop: the hop cap, target permission and downgrade checks of a policy RedirectPolicy builds and of DefaultRedirectPolicy answer one, so a caller tells them apart with errors.As and Reason alone. RefuseAllRedirects and a WithPreserveMethod refusal answer http.ErrUseLastResponse instead. An http.Client returns it inside a *url.Error, and it is never transient.
func (*RedirectRefusedError) Error ¶
func (e *RedirectRefusedError) Error() string
Error renders the refusal from Reason and Origin only.
type ResponseTooLargeError ¶
type ResponseTooLargeError struct {
Limit int64
}
ResponseTooLargeError is returned by GetBytes when the response body exceeds the configured maximum (WithMaxBodyBytes, default 10 MB). The body is not returned: a truncated payload indistinguishable from a complete one is a silent-corruption hazard, so GetBytes fails loud instead. Limit is the cap that was exceeded, mirroring the stdlib *http.MaxBytesError shape.
func (*ResponseTooLargeError) Error ¶
func (e *ResponseTooLargeError) Error() string
type RetryAfterHint ¶
RetryAfterHint is implemented by errors that carry an explicit wait duration for the next retry, typically a parsed and capped Retry-After. When fn's returned error is transient AND implements this interface with a positive duration, Do waits that duration before the next attempt instead of its jittered exponential backoff. The exponential base still advances, so a later transient error without a hint resumes the normal progression. The hint MUST already be capped by the implementer (e.g. via ParseRetryAfter); Do sleeps on it verbatim and applies no ceiling of its own, so an uncapped value is an unbounded-wait hazard.
It embeds error for the same reason Transient does: the interface is read through errors.As/errors.AsType over an error tree, so it can only ever name an error.
type RetryRoundTripper ¶
type RetryRoundTripper struct {
// contains filtered or unexported fields
}
RetryRoundTripper implements http.RoundTripper with automatic retry: of the safe methods, of any request carrying a non-empty Idempotency-Key header, and with TransportConfig.RetryNonIdempotent of every request. A request whose body has no GetBody to rewind it is sent once.
Unlike hashicorp/go-retryablehttp, it takes a plain *http.Request and counts total attempts (TransportConfig.MaxAttempts), not retries.
The zero value retries over http.DefaultTransport with the TransportConfig{} defaults; NewRetryRoundTripper sets the rest.
func NewRetryRoundTripper ¶
func NewRetryRoundTripper(next http.RoundTripper, cfg TransportConfig) *RetryRoundTripper
NewRetryRoundTripper creates a RetryRoundTripper wrapping next with the given configuration. If next is nil, http.DefaultTransport is used. TransportConfig{} gives the defaults (see its field docs).
func (*RetryRoundTripper) CloseIdleConnections ¶
func (rt *RetryRoundTripper) CloseIdleConnections()
CloseIdleConnections closes the base transport's idle connections where it can, so http.Client.CloseIdleConnections reaches them.
func (*RetryRoundTripper) RoundTrip ¶
RoundTrip implements http.RoundTripper: it retries an eligible request on a failure its policy accepts, honoring Retry-After on every retried status (RFC 9110 defines it for 503 and 3xx, RFC 6585 for 429), and never mutates req.
When retries run out the last response comes back with a nil error, even a retryable 503: the caller must close its body and check resp.StatusCode.
type Secret ¶
type Secret string
Secret is a credential value being redacted: the needle the redaction helpers scan FOR, never the text they scan. It is a distinct type so the value-to-hide and the text-to-scan cannot be transposed at a call site — a reversed call turns the redactor into a leak, scanning the credential for occurrences of the log text and returning the credential untouched. With the positions typed apart, a plain string variable in the secret position is a compile error; the caller converts exactly the value that is the credential (httpclient.Secret(token)) and nothing else fits there.
Convert at the call boundary, never store one: Secret carries no logging protection of its own — formatting a Secret with %v/%s or logging it via slog prints the credential verbatim. The zero value ("") disables redaction: every helper returns its input unredacted when the secret is empty, because there is nothing to scan for. Callers redacting a possibly-absent credential must treat empty as "no redaction happened", not as "redaction succeeded".
type StatusError ¶
type StatusError struct {
Code int
// RetryAfter is the response's Retry-After, raw and uncapped, zero when
// absent or malformed. The server controls it (CWE-400): bound it before
// sleeping on it, as Do's rate-limit options do.
RetryAfter time.Duration
// contains filtered or unexported fields
}
StatusError is the error CheckStatus and GetBytes return for a non-2xx response. Match a status class with errors.Is against ErrUnauthorized, ErrForbidden, ErrRateLimited or ErrServerError, or read Code.
func (*StatusError) Error ¶
func (e *StatusError) Error() string
func (*StatusError) Is ¶
func (e *StatusError) Is(target error) bool
Is matches the sentinel for e's Code.
func (*StatusError) IsTransient ¶
func (e *StatusError) IsTransient() bool
IsTransient reports whether Do retries e by default: IsRetryableStatus less 429, which only a rate-limit option retries.
type Transient ¶
Transient is the interface an error implements to report whether it represents a transient (retryable) failure. It embeds error because every value this interface can ever name IS one: it is read through errors.As/errors.AsType over an error tree, whose nodes are errors by construction, so a non-error implementation is unreachable. Declaring the error half makes the type say that, keeps Error() available on a recovered value for diagnostics, and is what lets a caller write errors.AsType[Transient](err) — errors.AsType constrains its type parameter to error, where the older errors.As accepted any target. Same shape as net.Error.
type TransportConfig ¶
type TransportConfig struct {
// CheckRetry overrides the retry policy. nil means the default policy:
// an error [IsTransient] accepts, or a status [IsRetryableStatus] accepts.
CheckRetry CheckRetry
// OnRetry is a hook called before each retry attempt, the transport's only
// observability seam (it logs nothing itself).
OnRetry OnRetry
// PrepareRetry is called before each retry to mutate the cloned request
// (e.g. refresh an auth token).
PrepareRetry PrepareRetry
// MaxAttempts is the TOTAL attempt count including the initial request.
// Zero means unset and takes the default of 3; a NEGATIVE value
// means exactly one attempt (a zero struct field cannot distinguish absent
// from zero, so try-once is spelled as a negative).
MaxAttempts int
// BaseDelay is the initial backoff delay; non-positive means
// the default of one second. Waits are equal-jitter with overflow-safe
// doubling, and a retryable response's Retry-After header (capped at
// RetryAfterCap) overrides the computed wait.
BaseDelay time.Duration
// MaxElapsedTime is a hard ceiling on total time across retries,
// including any honored Retry-After: when now+wait would meet or pass it,
// the round-tripper aborts with [ErrMaxElapsed] instead of sleeping. It is
// checked BETWEEN attempts and cannot interrupt a stalled in-flight
// attempt; bound single attempts on the base transport (e.g.
// ResponseHeaderTimeout) and the whole call with the request context.
// Zero means no ceiling.
MaxElapsedTime time.Duration
// RetryNonIdempotent extends retries from the safe methods and keyed
// requests to every method (POST, PUT, PATCH, DELETE and the rest).
RetryNonIdempotent bool
}
TransportConfig configures a RetryRoundTripper (and, through NewRetry, the retrying client). The zero value is ready to use: three total attempts, one-second base delay, the default retry policy, no hooks, no elapsed ceiling, and replay of the safe methods and keyed requests only.