Documentation
¶
Overview ¶
Package httpx provides small HTTP transport helpers shared across the proxy.
Index ¶
- Constants
- Variables
- func BodyReadDeadline(next http.Handler, idle time.Duration) http.Handler
- func CopyWithWriteDeadline(w http.ResponseWriter, src io.Reader, idle time.Duration) (int64, error)
- func NewTransport(pool *x509.CertPool) *http.Transport
- func ParsePublicURL(raw string) (*url.URL, error)
- func RootPool(caFiles []string) (*x509.CertPool, int, error)
- func SetRetryAfter(h http.Header, d time.Duration)
- type CircuitBreaker
- type ClientIPResolver
- type ConcurrencyLimiter
- type IdleTimeout
- type PublicBaseResolver
- type RateLimiter
- type TrustedPeers
Constants ¶
const ( HeaderXForwardedFor = "X-Forwarded-For" HeaderXRealIP = "X-Real-IP" HeaderForwarded = "Forwarded" )
Forwarding headers a trusted reverse proxy may use to convey the client IP.
const DefaultBodyReadIdleTimeout = 60 * time.Second
DefaultBodyReadIdleTimeout is how long a request body may go without a byte before reading it fails.
const DefaultUpstreamIdleTimeout = 60 * time.Second
DefaultUpstreamIdleTimeout is how long an upstream may stay silent — no response headers, or no body bytes — before the request is aborted.
const DefaultWriteIdleTimeout = 60 * time.Second
DefaultWriteIdleTimeout bounds how long one chunk written to a client may take, so a client that stops reading cannot pin a serving goroutine forever.
Variables ¶
var ErrBadForwardedHeader = errors.New("malformed host or forwarding header")
ErrBadForwardedHeader means the address a request names for this proxy — its Host, or a forwarding header from a trusted proxy — is not a usable URL authority, scheme or path prefix.
var ErrCircuitOpen = errors.New("upstream throttled: circuit open")
ErrCircuitOpen is returned (wrapped) when a request is fast-failed because the destination host is in its post-throttle cooldown.
Functions ¶
func BodyReadDeadline ¶ added in v1.0.0
BodyReadDeadline bounds how slowly a client may send a request body, the job a server-wide ReadTimeout would do if it did not also cancel long-running handlers. For a request with a body, a read deadline of idle is armed and re-armed on every read that makes progress; it is cleared as soon as the body hits EOF, so the server's background read (which starts at EOF) never inherits it. A body abandoned before EOF keeps its deadline, which also bounds the server's own drain of the remainder after the handler returns. Requests without a body pass straight through.
func CopyWithWriteDeadline ¶ added in v1.0.0
CopyWithWriteDeadline copies src to w in chunks of up to 256 KiB, arming a write deadline of idle before each chunk, so a transfer is bounded by progress rather than total time: a multi-GB artifact to a slow but reading client succeeds, a stalled client is dropped after idle. A writer without deadline support is copied without them. The deadline is cleared on return.
func NewTransport ¶ added in v0.4.0
NewTransport clones the default transport and points its root pool at the given one. A nil pool keeps the platform default. Cloning preserves the default connection pooling, proxy handling, and HTTP/2 negotiation.
func ParsePublicURL ¶ added in v1.0.0
ParsePublicURL validates a configured public base URL and returns it with any trailing slash removed.
func RootPool ¶ added in v0.4.0
RootPool returns the system root pool with every certificate from caFiles added, and how many certificates were added. Configured CAs supplement the public roots rather than replacing them, so a private mirror signed by a corporate CA and a public registry both verify through the same pool.
Every failure is fatal to the caller by design: a CA file that cannot be read or parsed is a misconfiguration, and continuing with silently reduced trust would surface much later as an opaque x509 error on a fetch.
Types ¶
type CircuitBreaker ¶
type CircuitBreaker struct {
// contains filtered or unexported fields
}
CircuitBreaker is an http.RoundTripper that fast-fails requests to a host that recently returned HTTP 429/503 instead of retrying it. Retrying a throttled upstream "harder" only deepens the penalty (per Sonatype's Maven Central guidance), so on a 429/503 the breaker opens for a cooldown — honoring Retry-After, else jittered exponential backoff — during which every request to that host fails immediately with ErrCircuitOpen. The handler's multi-upstream fallback then serves from a mirror right away instead of waiting on or hammering the throttled host. The breaker closes on the first success after the cooldown.
It performs no retries and never blocks, so it does not consume the caller's request timeout while a host is throttled.
func NewCircuitBreaker ¶
func NewCircuitBreaker(base http.RoundTripper, baseDelay, maxDelay time.Duration) *CircuitBreaker
NewCircuitBreaker wraps base. baseDelay and maxDelay bound the exponential cooldown used when an upstream sends no Retry-After. A nil base uses http.DefaultTransport.
func (*CircuitBreaker) OpenHosts ¶ added in v0.6.0
func (c *CircuitBreaker) OpenHosts() []string
OpenHosts returns the hosts (in req.URL.Host form) whose circuit is currently open, sorted. It is safe for concurrent use.
func (*CircuitBreaker) RoundTrip ¶
RoundTrip fast-fails when the host's circuit is open; otherwise it sends the request, opening the circuit on a 429/503 and resetting it on success.
type ClientIPResolver ¶ added in v0.6.0
type ClientIPResolver struct {
// contains filtered or unexported fields
}
ClientIPResolver determines the originating client IP of a request. The forwarding header is honoured only when the direct peer is a trusted proxy; otherwise the header is attacker-controlled and the peer address is used.
func NewClientIPResolver ¶ added in v0.6.0
func NewClientIPResolver(trusted []string, header string, log zerolog.Logger) (*ClientIPResolver, error)
NewClientIPResolver builds a resolver. trusted entries are CIDRs or bare IPs; header is one of the Header* constants (empty selects X-Forwarded-For).
type ConcurrencyLimiter ¶
type ConcurrencyLimiter struct {
// contains filtered or unexported fields
}
ConcurrencyLimiter is an http.RoundTripper that caps the number of concurrent in-flight requests per destination host. Parallel dependency resolution (Gradle, Maven, npm) fans out many simultaneous requests to a single registry; left unbounded that burst trips the registry's rate limit, which responds with HTTP 429. The cap keeps outbound concurrency under the limit so throttling does not happen in the first place.
Requests beyond the cap block until a slot frees or the request's context is cancelled. A single limiter is meant to be shared by every http.Client that talks to upstreams (metadata, transparent proxy, artifact download) so the cap is per host across all of them, not per client.
func NewConcurrencyLimiter ¶
func NewConcurrencyLimiter(base http.RoundTripper, maxPerHost int) *ConcurrencyLimiter
NewConcurrencyLimiter wraps base, capping concurrent requests to maxPerHost per destination host. A nil base uses http.DefaultTransport; maxPerHost <= 0 disables limiting (every request passes straight through).
type IdleTimeout ¶ added in v1.0.0
type IdleTimeout struct {
// contains filtered or unexported fields
}
IdleTimeout is an http.RoundTripper that aborts a request when it stops making progress, instead of capping its total duration: a multi-GB artifact arriving steadily over minutes succeeds, a stalled one fails after idle. Progress is the response headers arriving, then every body read that returns bytes. The request context is derived from the caller's, so caller cancellation still aborts the request.
func NewIdleTimeout ¶ added in v1.0.0
func NewIdleTimeout(next http.RoundTripper, idle time.Duration) *IdleTimeout
NewIdleTimeout wraps next with an idle timeout. Place it inside any queueing round-trippers (rate and concurrency limits) so time spent waiting for a slot does not count as upstream silence.
type PublicBaseResolver ¶ added in v1.0.0
type PublicBaseResolver struct {
// contains filtered or unexported fields
}
PublicBaseResolver derives the base URL a client used to reach this proxy, for links the proxy writes into documents it serves. A configured public URL wins outright; otherwise a trusted reverse proxy's X-Forwarded-Proto, X-Forwarded-Host, X-Forwarded-Port and X-Forwarded-Prefix (or the proto= and host= of Forwarded) describe it; otherwise the request's own Host and TLS state do. Forwarding headers from any other peer are ignored, with a one-time warning: a client could name any host in them.
A nil *PublicBaseResolver trusts no proxy and has no public URL.
func NewPublicBaseResolver ¶ added in v1.0.0
func NewPublicBaseResolver(publicURL string, trusted TrustedPeers, log zerolog.Logger) (*PublicBaseResolver, error)
NewPublicBaseResolver builds a resolver. publicURL is server.public_url and may be empty; when set it must be an absolute http or https URL without a query or fragment.
type RateLimiter ¶
type RateLimiter struct {
// contains filtered or unexported fields
}
RateLimiter is an http.RoundTripper that caps the request *rate* to each destination host using a token bucket. A concurrency cap alone does not bound rate — a few fast workers can still emit hundreds of requests per second — and registries such as Maven Central throttle by rate, so this is what actually prevents HTTP 429 under sustained load. Requests block until a token is available or the request's context is cancelled.
func NewRateLimiter ¶
func NewRateLimiter(base http.RoundTripper, perSecond float64, burst int) *RateLimiter
NewRateLimiter wraps base, limiting requests to perSecond per destination host with the given burst (clamped to >= 1). A nil base uses http.DefaultTransport; perSecond <= 0 disables limiting (every request passes straight through).
type TrustedPeers ¶ added in v1.0.0
TrustedPeers is the set of reverse proxies whose forwarding headers are believed. A peer outside it can forge any forwarding header, so every resolver that reads one consults this set first.
func ParseTrustedPeers ¶ added in v1.0.0
func ParseTrustedPeers(entries []string) (TrustedPeers, error)
ParseTrustedPeers parses server.trusted_proxies entries: CIDRs or bare IPs.
func (TrustedPeers) Contains ¶ added in v1.0.0
func (t TrustedPeers) Contains(a netip.Addr) bool
Contains reports whether a is a trusted proxy address.
func (TrustedPeers) PeerTrusted ¶ added in v1.0.0
func (t TrustedPeers) PeerTrusted(r *http.Request) bool
PeerTrusted reports whether r arrived directly from a trusted proxy.