httputil

package
v0.97.12 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 5, 2026 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Overview

Package httputil provides small HTTP helpers shared across go-kit services.

Security Headers

SecurityHeaders sets a conservative set of HTTP security headers on a ResponseWriter. Defaults follow go-nerv stricter policy:

  • Content-Security-Policy: default-src self; script-src self; style-src self unsafe-inline
  • X-Content-Type-Options: nosniff
  • X-Frame-Options: DENY
  • Referrer-Policy: strict-origin-when-cross-origin
  • Permissions-Policy: camera=(), microphone=(), geolocation=()
  • X-XSS-Protection: 0

Cache-Control is NOT set by default. Cache policy is orthogonal to security headers — marketing pages, API endpoints, and authed admin pages have fundamentally different caching requirements. Use WithCacheControl to set it:

SecurityHeaders(w, WithCacheControl("no-store"))           // authed admin
SecurityHeaders(w, WithCacheControl("public, max-age=3600")) // public assets

Services that require a less restrictive CSP (e.g. oxpulse-admin, which needs unsafe-inline for its embedded scripts) should call SecurityHeaders(w, WithCSP("...")) to override only that header while keeping all other defaults.

Client IP

ClientIP extracts the real client address from an HTTP request. Consults X-Real-IP, then X-Forwarded-For (first hop), then r.RemoteAddr. Each candidate is validated via net.ParseIP -- invalid or spoofed header values are silently skipped rather than returned. See ClientIP for the full trust model and deployment requirements.

SSRF Guard

This is the single, framework-owned SSRF block-list for any go-kit service that fetches a caller-supplied URL (an image src, an advertiser website, a redirect Location header, an out-of-process render delegate's target). It supersedes the two guards it consolidates -- render/html's former unexported isPrivateIP/safeDial and go-enriche's fetch/ssrf.go -- and adds the ranges neither one covered: CGNAT (100.64.0.0/10), the NAT64 well-known prefix (64:ff9b::/96), 6to4 (2002::/16), and the deprecated IPv4-compatible IPv6 form (::/96).

IsBlockedIP is the low-level predicate: loopback, RFC1918/RFC4193 private, link-local (incl. the 169.254.169.254 cloud-metadata address), unspecified, multicast, plus the four ranges above. Every other function here is built on top of it.

GuardedDialContext wraps a *net.Dialer with a Control hook that checks the ALREADY-RESOLVED address at connect time, immediately before the connect(2) syscall -- this defeats DNS-rebinding, since the check inspects the literal address about to be dialed, never a hostname string that could resolve differently between lookup and connect. Non-TCP/UDP dials -- a Unix domain socket ("unix"/"unixgram"/"unixpacket") -- pass through unchecked: a UDS dials a local filesystem path, not a network address, so it is not an SSRF-to-internal-IP vector.

NewSSRFGuardedClient wraps an *http.Client in two tiers depending on its Transport: a *http.Transport (or nil) gets DialContext replaced with GuardedDialContext AND is wrapped in a pre-request CheckURL check; any other http.RoundTripper (e.g. a stealth/fingerprint-evasion client with no exposed dial hook) gets ONLY the pre-request CheckURL check. Both layers are needed on the *http.Transport tier because a proxy-configured Transport (explicitly, or inherited via Proxy: http.ProxyFromEnvironment) causes net/http to call DialContext with the PROXY's address, never the real target -- GuardedDialContext alone would silently pass a proxied request to an internal host straight through. So: a direct (non-proxied) request gets BOTH the connect-time, DNS-rebind-proof dial guard AND the pre-request check; a proxied request gets only the pre-request check (still real protection -- it evaluates the actual destination URL, not the proxy's -- but a necessarily weaker, pre-resolve tier, the same one CheckURL itself provides for a delegate this package cannot dial-guard).

CheckURL is the pre-handoff check for a URL this package never dials itself -- e.g. before handing an "any place" URL to an out-of-process render delegate. It enforces an http/https scheme allowlist (a headless browser coerced into file:// or gopher:// bypasses every IP-based check, since those schemes never dial the checked host at all) plus IsBlockedIP on every resolved address. Call it again after each redirect hop a delegate reports, to minimize the DNS-rebind window. A host that looks like a non-standard IP encoding (decimal, octal, or hex -- e.g. "2130706433" or "0x7f000001") but fails net.ParseIP is refused outright rather than handed to DNS resolution, since some resolvers still parse those forms as literal IPs.

Index

Constants

This section is empty.

Variables

View Source
var ErrSSRFBlocked = errors.New("httputil: SSRF-blocked address")

ErrSSRFBlocked wraps every error CheckURL, GuardedDialContext, or a NewSSRFGuardedClient-wrapped transport returns when a target address is loopback, private (RFC1918 / RFC4193 ULA), link-local (including the cloud metadata address 169.254.169.254), unspecified, multicast, carrier-grade NAT (RFC 6598), or one of the IPv6 transition ranges that embed or route to an IPv4 address (NAT64, 6to4, the deprecated IPv4-compatible form) — the address classes an SSRF payload targets to reach internal infrastructure that must never be dialed from a caller-supplied target (e.g. an advertiser-provided website URL, a redirect Location header, an out-of-process render delegate's fetch target).

Functions

func CheckRawURL added in v0.95.0

func CheckRawURL(ctx context.Context, rawURL string) error

CheckRawURL parses rawURL and delegates to CheckURL. Convenience for callers holding a string (a redirect Location header, an MCP tool argument) rather than an already-parsed *url.URL.

func CheckURL added in v0.95.0

func CheckURL(ctx context.Context, u *url.URL) error

CheckURL is a pre-handoff safety check for a URL this package does not itself dial — e.g. before handing an "any place" URL to an out-of-process render delegate (a headless browser, a fetch microservice) whose own outbound dial GuardedDialContext / NewSSRFGuardedClient cannot reach. Call it as close as possible to the point of dispatch, and again after each redirect hop the delegate reports, to minimize the DNS-rebind window — CheckURL is necessarily weaker against rebinding than GuardedDialContext, since DNS can change between this resolution and the delegate's own, separate one.

Enforces two things:

  1. Scheme allowlist — only "http" and "https" are permitted; a headless browser coerced into "file://" or "gopher://" bypasses every IP-based check below, since those schemes never dial the checked host at all.
  2. Every resolved address for the URL's host passes IsBlockedIP.

A host that is a literal IP is checked directly. A host that LOOKS like a non-standard numeral encoding of an IP (decimal, octal, or hex — e.g. "2130706433", "0x7f000001", or "012.0.0.1") but fails net.ParseIP is refused outright rather than handed to DNS resolution: some resolvers (notably glibc's getaddrinfo via cgo) still parse these forms as literal IPs, which would silently defeat this check if it fell through to a same-string-but-different DNS lookup.

func ClientIP

func ClientIP(r *http.Request) string

ClientIP extracts the real client address. Services behind a reverse proxy (e.g. Caddy) typically see only the loopback address in r.RemoteAddr.

Priority:

  1. X-Real-IP — set by the proxy to the single trusted client IP; most authoritative.
  2. X-Forwarded-For first element — fallback for proxies that don't set X-Real-IP.
  3. r.RemoteAddr — last resort (always loopback behind a proxy, but correct in tests).

Each candidate is validated via net.ParseIP before use; an invalid value silently falls through to the next candidate.

Security / trust model

ClientIP trusts the X-Real-IP and X-Forwarded-For headers when their values parse as valid IPs. These headers are client-controlled and can be spoofed by any caller that can reach the service directly. Callers MUST ensure the service is fronted by a reverse proxy that strips or overrides these headers from untrusted sources before calling ClientIP. Behind a misconfigured deployment the returned IP is attacker-controlled.

func DenyBlockedAddress added in v0.95.1

func DenyBlockedAddress(network, address string) error

DenyBlockedAddress is the Control-hook body: it inspects the ALREADY-RESOLVED network/address pair — exactly what net.Dialer.Control receives, and what GuardedDialContext wires this into — and refuses anything IsBlockedIP flags. Exported (split out from GuardedDialContext originally so tests could drive it directly with a hardcoded post-resolution address, simulating exactly what net/http passes after DNS lookup, without needing a real DNS rebind) so a caller wiring a bespoke Control-hook-shaped seam on an opaque transport this package does not itself dial (e.g. a stealth/fingerprint-evasion client's own dialer hook) can reuse the identical policy GuardedDialContext uses internally, rather than duplicating it. See SSRFGuards for the paired redirect-guard closure.

func GuardedDialContext added in v0.95.0

func GuardedDialContext(base *net.Dialer) func(ctx context.Context, network, address string) (net.Conn, error)

GuardedDialContext wraps base with a Control hook that refuses to connect to a blocked address (see IsBlockedIP). The check runs on the ALREADY-RESOLVED address at connect time — after DNS lookup, immediately before the connect(2) syscall — which is what defeats DNS-rebinding: a hostname that resolves to a public IP when net/http first looks it up but resolves to a private IP by the time this fires is still caught, because the check inspects the literal address about to be dialed, never the hostname string. Any pre-existing Control hook on base still runs first.

base == nil uses a Dialer with sane guarded-fetch defaults (10s connect timeout, 30s keepalive).

func IsBlockedIP added in v0.95.0

func IsBlockedIP(ip net.IP) bool

IsBlockedIP reports whether ip must never be dialed as a fetch/render target. This is the single, framework-owned SSRF block-list — every other primitive in this file (GuardedDialContext, NewSSRFGuardedClient, CheckURL) is built on top of this one predicate. A nil IP is treated as blocked (fail closed).

Go's net.IP predicates already unwrap IPv4-mapped-IPv6 addresses (e.g. ::ffff:10.0.0.1 or ::ffff:127.0.0.1) to their IPv4 form before matching — including against extraBlockedCIDRs, since net.IPNet.Contains performs the same To4() unwrap internally — so no separate normalization step is needed here.

func NewSSRFGuardedClient added in v0.95.0

func NewSSRFGuardedClient(base *http.Client) *http.Client

NewSSRFGuardedClient returns an SSRF-guarded *http.Client built on top of base. base == nil returns a fresh client with a guardedTransport().

Two composition tiers, chosen by what base.Transport actually is:

  • nil or *http.Transport: cloned (Clone() preserves TLSClientConfig, proxy, HTTP2 settings, MaxIdleConns, etc. — every field a caller may have set) with DialContext replaced by GuardedDialContext, THEN wrapped with the pre-request CheckURL layer (wrapTransportTier) — see wrapTransportTier's doc for why a proxy-configured Transport needs both layers, not just the dial-time one.
  • any other http.RoundTripper (e.g. a stealth/fingerprint-evasion client whose Transport performs its own dial via a bespoke backend, with no DialContext/net.Dialer hook exposed at all): wrapped with ONLY the pre-request CheckURL layer — the necessarily WEAKER pre-resolve tier (a DNS-rebind can still occur between this check and the delegate's own, separate resolution), but the best guarantee available without reaching into a dial mechanism this package does not own.

The returned *http.Client is a shallow copy of base; base itself (and its Transport) is never mutated.

func SSRFGuards added in v0.95.1

func SSRFGuards() (
	redirect func(req *http.Request, via []*http.Request) error,
	dial func(network, address string) error,
)

SSRFGuards returns the pair of stdlib-typed closures an opaque-transport HTTP client (one whose Transport performs its own dial/redirect via a bespoke backend this package cannot reach — e.g. a stealth/ fingerprint-evasion client) installs to close the SAME two gaps NewSSRFGuardedClient already closes for a plain *http.Transport-backed client:

  • dial is DenyBlockedAddress itself — wire it into whatever Control-hook-shaped seam the opaque transport exposes (directly, if it accepts a net.Dialer.Control-shaped func, or behind a thin adapter otherwise) for the rebind-proof, connect-time check.
  • redirect enforces the ≤maxSSRFRedirectHops cap (see that const's doc for why re-owning it is required, not optional) and THEN CheckURL(req.Context(), req.URL) on every hop — wire it into whatever per-hop redirect-decision seam the opaque transport exposes. Its signature matches http.Client.CheckRedirect exactly, so a stdlib-backed caller can assign it directly with no adapter.

Named for the CAPABILITY (this package's framework-owned SSRF policy), not any one consumer — this package takes no dependency on whatever opaque-transport client ends up wiring these in, and the identical pair of closures can be handed to more than one backend so their behavior cannot drift apart.

func SecurityHeaders

func SecurityHeaders(w http.ResponseWriter, opts ...Option)

SecurityHeaders writes a conservative set of HTTP security headers to w. Each header is set via Header().Set (replaces, never appends). Pass Option values to override specific headers from their defaults.

Headers set by default:

  • X-Content-Type-Options: nosniff
  • X-Frame-Options: DENY
  • Referrer-Policy: strict-origin-when-cross-origin
  • Content-Security-Policy: default-src 'self'; script-src 'self'; ...
  • Permissions-Policy: camera=(), microphone=(), geolocation=()
  • X-XSS-Protection: 0

Cache-Control is NOT set by default. Pass WithCacheControl to set it. Each handler must declare its own cache policy — marketing pages, API endpoints, and authed admin pages have fundamentally different requirements.

Types

type Option

type Option func(*config)

Option modifies the header configuration applied by SecurityHeaders.

func WithCSP

func WithCSP(policy string) Option

WithCSP overrides the Content-Security-Policy header value.

func WithCacheControl added in v0.56.0

func WithCacheControl(value string) Option

WithCacheControl sets the Cache-Control header value. SecurityHeaders does not set Cache-Control by default — cache policy is orthogonal to security headers and must be declared per handler. Use this option when the caller wants a single call to cover both concerns:

SecurityHeaders(w, WithCacheControl("no-store"))          // authed admin pages
SecurityHeaders(w, WithCacheControl("public, max-age=3600")) // public assets

func WithPermissionsPolicy added in v0.55.0

func WithPermissionsPolicy(policy string) Option

WithPermissionsPolicy overrides the Permissions-Policy header value. Default: "camera=(), microphone=(), geolocation=()" (no device access).

func WithReferrerPolicy

func WithReferrerPolicy(policy string) Option

WithReferrerPolicy overrides the Referrer-Policy header value.

func WithXSSProtection added in v0.55.0

func WithXSSProtection(value string) Option

WithXSSProtection overrides the X-XSS-Protection header value. Default: "0" (disable legacy XSS auditor per OWASP/Mozilla Observatory).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL