httpx

package
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 8, 2026 License: MIT Imports: 20 Imported by: 0

Documentation

Overview

Package httpx provides small HTTP transport helpers shared across the proxy.

Index

Constants

View Source
const (
	HeaderXForwardedFor = "X-Forwarded-For"
	HeaderXRealIP       = "X-Real-IP"
	HeaderForwarded     = "Forwarded"
)

Forwarding headers a trusted reverse proxy may use to convey the client IP.

View Source
const DefaultBodyReadIdleTimeout = 60 * time.Second

DefaultBodyReadIdleTimeout is how long a request body may go without a byte before reading it fails.

View Source
const DefaultUpstreamIdleTimeout = 60 * time.Second

DefaultUpstreamIdleTimeout is how long an upstream may stay silent — no response headers, or no body bytes — before the request is aborted.

View Source
const DefaultWriteIdleTimeout = 60 * time.Second

DefaultWriteIdleTimeout bounds how long one chunk written to a client may take, so a client that stops reading cannot pin a serving goroutine forever.

Variables

View Source
var ErrBadForwardedHeader = errors.New("malformed host or forwarding header")

ErrBadForwardedHeader means the address a request names for this proxy — its Host, or a forwarding header from a trusted proxy — is not a usable URL authority, scheme or path prefix.

View Source
var ErrCircuitOpen = errors.New("upstream throttled: circuit open")

ErrCircuitOpen is returned (wrapped) when a request is fast-failed because the destination host is in its post-throttle cooldown.

Functions

func BodyReadDeadline added in v1.0.0

func BodyReadDeadline(next http.Handler, idle time.Duration) http.Handler

BodyReadDeadline bounds how slowly a client may send a request body, the job a server-wide ReadTimeout would do if it did not also cancel long-running handlers. For a request with a body, a read deadline of idle is armed and re-armed on every read that makes progress; it is cleared as soon as the body hits EOF, so the server's background read (which starts at EOF) never inherits it. A body abandoned before EOF keeps its deadline, which also bounds the server's own drain of the remainder after the handler returns. Requests without a body pass straight through.

func CopyWithWriteDeadline added in v1.0.0

func CopyWithWriteDeadline(w http.ResponseWriter, src io.Reader, idle time.Duration) (int64, error)

CopyWithWriteDeadline copies src to w in chunks of up to 256 KiB, arming a write deadline of idle before each chunk, so a transfer is bounded by progress rather than total time: a multi-GB artifact to a slow but reading client succeeds, a stalled client is dropped after idle. A writer without deadline support is copied without them. The deadline is cleared on return.

func NewTransport added in v0.4.0

func NewTransport(pool *x509.CertPool) *http.Transport

NewTransport clones the default transport and points its root pool at the given one. A nil pool keeps the platform default. Cloning preserves the default connection pooling, proxy handling, and HTTP/2 negotiation.

func ParsePublicURL added in v1.0.0

func ParsePublicURL(raw string) (*url.URL, error)

ParsePublicURL validates a configured public base URL and returns it with any trailing slash removed.

func RootPool added in v0.4.0

func RootPool(caFiles []string) (*x509.CertPool, int, error)

RootPool returns the system root pool with every certificate from caFiles added, and how many certificates were added. Configured CAs supplement the public roots rather than replacing them, so a private mirror signed by a corporate CA and a public registry both verify through the same pool.

Every failure is fatal to the caller by design: a CA file that cannot be read or parsed is a misconfiguration, and continuing with silently reduced trust would surface much later as an opaque x509 error on a fetch.

func SetRetryAfter added in v1.0.0

func SetRetryAfter(h http.Header, d time.Duration)

SetRetryAfter sets a Retry-After header of d rounded up to whole seconds; d <= 0 means no hint, and no header is set.

Types

type CircuitBreaker

type CircuitBreaker struct {
	// contains filtered or unexported fields
}

CircuitBreaker is an http.RoundTripper that fast-fails requests to a host that recently returned HTTP 429/503 instead of retrying it. Retrying a throttled upstream "harder" only deepens the penalty (per Sonatype's Maven Central guidance), so on a 429/503 the breaker opens for a cooldown — honoring Retry-After, else jittered exponential backoff — during which every request to that host fails immediately with ErrCircuitOpen. The handler's multi-upstream fallback then serves from a mirror right away instead of waiting on or hammering the throttled host. The breaker closes on the first success after the cooldown.

It performs no retries and never blocks, so it does not consume the caller's request timeout while a host is throttled.

func NewCircuitBreaker

func NewCircuitBreaker(base http.RoundTripper, baseDelay, maxDelay time.Duration) *CircuitBreaker

NewCircuitBreaker wraps base. baseDelay and maxDelay bound the exponential cooldown used when an upstream sends no Retry-After. A nil base uses http.DefaultTransport.

func (*CircuitBreaker) OpenHosts added in v0.6.0

func (c *CircuitBreaker) OpenHosts() []string

OpenHosts returns the hosts (in req.URL.Host form) whose circuit is currently open, sorted. It is safe for concurrent use.

func (*CircuitBreaker) RoundTrip

func (c *CircuitBreaker) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip fast-fails when the host's circuit is open; otherwise it sends the request, opening the circuit on a 429/503 and resetting it on success.

func (*CircuitBreaker) Throttle added in v1.0.0

func (c *CircuitBreaker) Throttle(host string) (status int, retryIn time.Duration, ok bool)

Throttle reports whether host's circuit is open: the upstream status (429 or 503) that opened it and the time left on its cooldown. It is safe for concurrent use.

type ClientIPResolver added in v0.6.0

type ClientIPResolver struct {
	// contains filtered or unexported fields
}

ClientIPResolver determines the originating client IP of a request. The forwarding header is honoured only when the direct peer is a trusted proxy; otherwise the header is attacker-controlled and the peer address is used.

func NewClientIPResolver added in v0.6.0

func NewClientIPResolver(trusted []string, header string, log zerolog.Logger) (*ClientIPResolver, error)

NewClientIPResolver builds a resolver. trusted entries are CIDRs or bare IPs; header is one of the Header* constants (empty selects X-Forwarded-For).

func (*ClientIPResolver) ClientIP added in v0.6.0

func (c *ClientIPResolver) ClientIP(r *http.Request) string

ClientIP returns the client IP (no port) for r.

type ConcurrencyLimiter

type ConcurrencyLimiter struct {
	// contains filtered or unexported fields
}

ConcurrencyLimiter is an http.RoundTripper that caps the number of concurrent in-flight requests per destination host. Parallel dependency resolution (Gradle, Maven, npm) fans out many simultaneous requests to a single registry; left unbounded that burst trips the registry's rate limit, which responds with HTTP 429. The cap keeps outbound concurrency under the limit so throttling does not happen in the first place.

Requests beyond the cap block until a slot frees or the request's context is cancelled. A single limiter is meant to be shared by every http.Client that talks to upstreams (metadata, transparent proxy, artifact download) so the cap is per host across all of them, not per client.

func NewConcurrencyLimiter

func NewConcurrencyLimiter(base http.RoundTripper, maxPerHost int) *ConcurrencyLimiter

NewConcurrencyLimiter wraps base, capping concurrent requests to maxPerHost per destination host. A nil base uses http.DefaultTransport; maxPerHost <= 0 disables limiting (every request passes straight through).

func (*ConcurrencyLimiter) RoundTrip

func (l *ConcurrencyLimiter) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip acquires a per-host slot (respecting context cancellation), runs the request, then releases the slot.

type IdleTimeout added in v1.0.0

type IdleTimeout struct {
	// contains filtered or unexported fields
}

IdleTimeout is an http.RoundTripper that aborts a request when it stops making progress, instead of capping its total duration: a multi-GB artifact arriving steadily over minutes succeeds, a stalled one fails after idle. Progress is the response headers arriving, then every body read that returns bytes. The request context is derived from the caller's, so caller cancellation still aborts the request.

func NewIdleTimeout added in v1.0.0

func NewIdleTimeout(next http.RoundTripper, idle time.Duration) *IdleTimeout

NewIdleTimeout wraps next with an idle timeout. Place it inside any queueing round-trippers (rate and concurrency limits) so time spent waiting for a slot does not count as upstream silence.

func (*IdleTimeout) RoundTrip added in v1.0.0

func (t *IdleTimeout) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip implements http.RoundTripper.

type PublicBaseResolver added in v1.0.0

type PublicBaseResolver struct {
	// contains filtered or unexported fields
}

PublicBaseResolver derives the base URL a client used to reach this proxy, for links the proxy writes into documents it serves. A configured public URL wins outright; otherwise a trusted reverse proxy's X-Forwarded-Proto, X-Forwarded-Host, X-Forwarded-Port and X-Forwarded-Prefix (or the proto= and host= of Forwarded) describe it; otherwise the request's own Host and TLS state do. Forwarding headers from any other peer are ignored, with a one-time warning: a client could name any host in them.

A nil *PublicBaseResolver trusts no proxy and has no public URL.

func NewPublicBaseResolver added in v1.0.0

func NewPublicBaseResolver(publicURL string, trusted TrustedPeers, log zerolog.Logger) (*PublicBaseResolver, error)

NewPublicBaseResolver builds a resolver. publicURL is server.public_url and may be empty; when set it must be an absolute http or https URL without a query or fragment.

func (*PublicBaseResolver) Base added in v1.0.0

func (p *PublicBaseResolver) Base(r *http.Request) (*url.URL, error)

Base returns the client-facing base URL for r: scheme, host and path prefix, with no trailing slash. An error wraps ErrBadForwardedHeader.

type RateLimiter

type RateLimiter struct {
	// contains filtered or unexported fields
}

RateLimiter is an http.RoundTripper that caps the request *rate* to each destination host using a token bucket. A concurrency cap alone does not bound rate — a few fast workers can still emit hundreds of requests per second — and registries such as Maven Central throttle by rate, so this is what actually prevents HTTP 429 under sustained load. Requests block until a token is available or the request's context is cancelled.

func NewRateLimiter

func NewRateLimiter(base http.RoundTripper, perSecond float64, burst int) *RateLimiter

NewRateLimiter wraps base, limiting requests to perSecond per destination host with the given burst (clamped to >= 1). A nil base uses http.DefaultTransport; perSecond <= 0 disables limiting (every request passes straight through).

func (*RateLimiter) RoundTrip

func (l *RateLimiter) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip waits for a per-host token (respecting context cancellation), then runs the request.

type TrustedPeers added in v1.0.0

type TrustedPeers []netip.Prefix

TrustedPeers is the set of reverse proxies whose forwarding headers are believed. A peer outside it can forge any forwarding header, so every resolver that reads one consults this set first.

func ParseTrustedPeers added in v1.0.0

func ParseTrustedPeers(entries []string) (TrustedPeers, error)

ParseTrustedPeers parses server.trusted_proxies entries: CIDRs or bare IPs.

func (TrustedPeers) Contains added in v1.0.0

func (t TrustedPeers) Contains(a netip.Addr) bool

Contains reports whether a is a trusted proxy address.

func (TrustedPeers) PeerTrusted added in v1.0.0

func (t TrustedPeers) PeerTrusted(r *http.Request) bool

PeerTrusted reports whether r arrived directly from a trusted proxy.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL