Documentation
¶
Overview ¶
Package httpclient builds outbound HTTP clients for URLs that users control (webhooks, link previews, avatars by URL). Safe refuses to connect to loopback, private, link-local (cloud metadata), multicast and other internal addresses. The check runs on the resolved IP at connect time, so DNS rebinding and redirects to internal hosts are blocked too.
resp, err := httpclient.Safe(httpclient.Options{}).Get(userURL)
Options bound the timeout, redirects and body size and may allow extra prefixes; blocked destinations fail with ErrBlockedAddress. Get is the one-call form and Allowed the address check itself. Throttle, Retry and Breaker are transport wrappers for Options.WrapTransport that respect a third-party API's rate limits and failures.
See https://github.com/AlexGalhardo/vukem/blob/main/docs/security.md.
Example ¶
package main
import (
"fmt"
"io"
"time"
"github.com/AlexGalhardo/vukem/httpclient"
)
func main() {
// Fetch a URL a user typed in (a webhook target, an avatar link).
// Internal destinations fail with ErrBlockedAddress.
client := httpclient.Safe(httpclient.Options{Timeout: 5 * time.Second})
resp, err := client.Get("https://example.com/avatar.png")
if err != nil {
return
}
defer resp.Body.Close()
avatar, err := io.ReadAll(resp.Body) // fails past Options.MaxBody
fmt.Println(len(avatar), err)
}
Output:
Index ¶
- Variables
- func Allowed(ip netip.Addr, allow []netip.Prefix) bool
- func Breaker(o BreakerOptions) func(http.RoundTripper) http.RoundTripper
- func Get(ctx context.Context, rawURL string) (*http.Response, error)
- func Retry(p RetryPolicy) func(http.RoundTripper) http.RoundTripper
- func Safe(o Options) *http.Client
- func Throttle(l Limit) func(http.RoundTripper) http.RoundTripper
- type BreakerOptions
- type Limit
- type Options
- type RetryPolicy
- type RoundTripperFunc
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrBlockedAddress = errors.New("httpclient: destination address is not allowed")
ErrBlockedAddress is returned when the destination is an internal address.
var ErrBodyTooLarge = errors.New("httpclient: response body too large")
ErrBodyTooLarge is returned by reads past Options.MaxBody.
var ErrCircuitOpen = errors.New("httpclient: circuit open for host")
ErrCircuitOpen is returned by a Breaker transport while the host's circuit is open.
var ErrNilTransport = errors.New("httpclient: Options.WrapTransport returned a nil transport")
ErrNilTransport is returned by every request of a client whose Options.WrapTransport returned nil.
Functions ¶
func Allowed ¶
Allowed reports whether ip may be reached with allow as extra prefixes.
Example ¶
package main
import (
"fmt"
"net/netip"
"github.com/AlexGalhardo/vukem/httpclient"
)
func main() {
for _, ip := range []string{"93.184.216.34", "127.0.0.1", "10.0.0.5", "169.254.169.254"} {
fmt.Println(ip, httpclient.Allowed(netip.MustParseAddr(ip), nil))
}
}
Output: 93.184.216.34 true 127.0.0.1 false 10.0.0.5 false 169.254.169.254 false
func Breaker ¶ added in v0.34.0
func Breaker(o BreakerOptions) func(http.RoundTripper) http.RoundTripper
Breaker returns a transport wrapper with a circuit breaker per host: after o.Failures consecutive failures, requests to that host fail at once with ErrCircuitOpen for o.Cooldown; then a single trial request goes through, and its success closes the circuit while a failure opens it again. A request ended by its own context counts as neither.
func Get ¶
Get is a convenience for Safe(Options{}).Get with a context.
Example ¶
package main
import (
"context"
"fmt"
"github.com/AlexGalhardo/vukem/httpclient"
)
func main() {
resp, err := httpclient.Get(context.Background(), "https://example.com/feed.xml")
if err != nil {
return
}
defer resp.Body.Close()
fmt.Println(resp.Status)
}
Output:
func Retry ¶ added in v0.34.0
func Retry(p RetryPolicy) func(http.RoundTripper) http.RoundTripper
Retry returns a transport wrapper that retries a request after a transport error or a 429, 502, 503 or 504 response, waiting for the server's Retry-After (seconds or an HTTP date) or else an exponential backoff. Only idempotent methods (GET, HEAD, OPTIONS, TRACE, PUT, DELETE) and requests carrying an Idempotency-Key header are retried, and only when their body can be replayed (no body, or Request.GetBody, which http.NewRequest sets for in-memory bodies). Context errors, ErrBlockedAddress and ErrCircuitOpen are never retried. Client.Timeout covers all attempts together.
func Throttle ¶ added in v0.34.0
func Throttle(l Limit) func(http.RoundTripper) http.RoundTripper
Throttle returns a transport wrapper that limits each host (the URL's host:port) to l: requests wait for their turn, or fail with the request context's error when it ends first. Use one client per API with its own limit (Options.WrapTransport).
httpclient.Safe(httpclient.Options{
WrapTransport: httpclient.Throttle(httpclient.Limit{PerSecond: 4, InFlight: 8}),
})
Types ¶
type BreakerOptions ¶ added in v0.34.0
type BreakerOptions struct {
// Failures is the number of consecutive failures (transport errors and
// 5xx responses) that opens a host's circuit (default 5).
Failures int
// Cooldown is how long the circuit stays open before one trial
// request may go through (default 30s).
Cooldown time.Duration
}
BreakerOptions configures a Breaker transport.
type Limit ¶ added in v0.34.0
type Limit struct {
// PerSecond is the sustained request rate per host; 0 means no rate
// limit.
PerSecond float64
// Burst is how many requests may go at once after an idle period
// (default 1).
Burst int
// InFlight bounds the requests per host whose response body is still
// open; 0 means no bound.
InFlight int
}
Limit bounds the requests a Throttle transport sends to each host.
type Options ¶
type Options struct {
// Timeout bounds the whole request (default 10s).
Timeout time.Duration
// MaxBody bounds response bodies; reads past it fail (default 10 MiB).
MaxBody int64
// MaxRedirects (default 5).
MaxRedirects int
// Allow lists extra prefixes that may be reached (e.g. an internal
// service network). Use sparingly.
Allow []netip.Prefix
// WrapTransport, when set, wraps the client's transport (tracing,
// metrics; drivers/otel's Transport fits). It must delegate every
// request to the transport it is given: the address check runs in that
// transport's dialer, so a delegating wrapper cannot reach blocked
// addresses. A wrapper returning nil makes every request fail with
// ErrNilTransport (http.Client would otherwise use
// http.DefaultTransport, which has no address check).
WrapTransport func(http.RoundTripper) http.RoundTripper
}
Options tune Safe.
type RetryPolicy ¶ added in v0.34.0
type RetryPolicy struct {
// Attempts is the total number of tries, the first included
// (default 3).
Attempts int
// Base is the first backoff delay, doubled on every retry with full
// jitter (default 200ms).
Base time.Duration
// Max caps every delay (default 30s). A Retry-After longer than Max
// is not waited for: the response is returned as is.
Max time.Duration
}
RetryPolicy configures a Retry transport.
type RoundTripperFunc ¶ added in v0.34.0
RoundTripperFunc adapts a function to http.RoundTripper.