Documentation
¶
Overview ¶
Package httpclient is the shared HTTP client used by aptcache, aptrepo, apkindex, and pacmandb for short metadata fetches.
It exposes one *http.Client with a global timeout so a stalled mirror cannot hang the build, plus helpers for size-capped body reads and a 2xx status check. Large package downloads use grab directly and don't route through this client.
Index ¶
- Constants
- Variables
- func AtomicWrite(destPath string, fn func(io.Writer) error) error
- func CheckStatus(resp *http.Response, fetchURL string) error
- func Client() *http.Client
- func FetchBytes(ctx context.Context, url string, maxBytes int64) ([]byte, error)
- func FetchBytesConditional(ctx context.Context, url string, maxBytes int64, ifModSince time.Time) (body []byte, notModified bool, err error)
- func FetchToFile(ctx context.Context, url, destPath string, maxBytes int64) error
- func IsRetryable(err error) bool
- func LimitedBody(resp *http.Response) io.Reader
- func LimitedBodyN(resp *http.Response, maxBytes int64) io.Reader
- func SetRetryPolicy(attempts int, baseDelay time.Duration)
- func UpgradeToHTTPS(rawURL string) (string, bool)
- func WithRetry(ctx context.Context, label string, fn func() error) error
- type HTTPStatusError
Constants ¶
const DefaultMaxBytes int64 = 2 << 30 // 2 GiB
DefaultMaxBytes caps individual HTTP responses to 2 GiB. Package indexes are typically a few MB; .deb / .apk / .rpm files top out around a few hundred MB. The cap prevents OOM from a malicious or buggy mirror serving an unbounded stream.
const DefaultTimeout = 10 * time.Minute
DefaultTimeout bounds a complete HTTP request (connection, TLS handshake, header read, body read) at 10 minutes. Large .deb downloads over slow links should still fit; stalled mirrors will fail rather than hang.
Variables ¶
var ErrTooLarge = errors.New("httpclient: response body exceeds size cap")
ErrTooLarge is returned when a response body exceeds the configured cap.
Functions ¶
func AtomicWrite ¶
AtomicWrite writes to destPath via a temp file + rename so readers never see a partial file. fn receives a writer for the temp file; if fn returns an error the temp file is removed.
func CheckStatus ¶
CheckStatus returns a descriptive error if resp.StatusCode is not 2xx. The caller still owns resp.Body.Close().
func Client ¶
Client returns the package-wide *http.Client. Reuse by all aptcache / aptrepo / apkindex / pacmandb / aptinstall callers so timeout and redirect behaviour stays consistent.
func FetchBytes ¶
FetchBytes downloads url and returns the body, capped at maxBytes. A non-positive maxBytes falls back to DefaultMaxBytes. Transient failures are retried per the package retry policy.
func FetchBytesConditional ¶ added in v2.1.7
func FetchBytesConditional( ctx context.Context, url string, maxBytes int64, ifModSince time.Time, ) (body []byte, notModified bool, err error)
FetchBytesConditional is like FetchBytes but sends If-Modified-Since when ifModSince is non-zero. On HTTP 304 it returns (nil, true, nil) so the caller can use its on-disk copy. On 200 it returns (data, false, nil). On any other error it returns (nil, false, err).
This is the warm-cache primitive that lets dnfcache / aptrepo / aptcache skip re-downloading metadata files whose upstream hasn't moved.
Transient failures (network errors, HTTP 5xx) are retried per the package retry policy.
func FetchToFile ¶
FetchToFile downloads url and writes it atomically to destPath, capped at maxBytes. Content-Length is checked as a cheap preflight when available. A non-positive maxBytes falls back to DefaultMaxBytes. Transient failures are retried per the package retry policy; the atomic temp-file + rename write guarantees a failed attempt leaves no partial file.
func IsRetryable ¶ added in v2.3.3
IsRetryable reports whether err looks like a transient network failure worth retrying: transport-level errors (DNS, connection reset/refused, timeouts, mid-body EOF) and HTTP 408/429/5xx responses.
Context cancellation, response-size caps, and all other HTTP 4xx codes are NOT retryable: they are definitive answers, not blips.
func LimitedBody ¶
LimitedBody wraps resp.Body in an io.LimitReader with DefaultMaxBytes. Callers should still defer resp.Body.Close() on the original response.
func LimitedBodyN ¶
LimitedBodyN wraps resp.Body in an io.LimitReader with the supplied cap. Use when the caller knows a tighter bound (e.g. control.tar is always small). A non-positive cap falls back to DefaultMaxBytes.
func SetRetryPolicy ¶ added in v2.3.3
SetRetryPolicy overrides the global retry policy. attempts is the total number of tries (minimum 1); baseDelay is the first backoff interval (doubled on each subsequent retry). Intended for tests and callers that need faster failure (e.g. offline detection).
func UpgradeToHTTPS ¶ added in v2.4.4
UpgradeToHTTPS rewrites a plain-http URL to https. Returns ok=false for anything that isn't parseable http (including URLs already on https).
Used by aptrepo/aptcache to recover from networks that reset or drop port-80 connections outright while leaving 443 untouched — declared mirrors like archive.ubuntu.com serve identical content on both schemes, so escalating on a transient http failure is safe.
func WithRetry ¶ added in v2.3.3
WithRetry runs fn until it succeeds, fails with a non-retryable error, or the retry budget is exhausted. label identifies the fetch in retry logs (typically the URL). The last error is returned verbatim so callers can still classify it (errors.As on HTTPStatusError, etc.).
fn must be safe to re-run from scratch: helpers in this package re-issue the full request and rewrite output atomically, so a half-read body from a failed attempt is never observable.
Types ¶
type HTTPStatusError ¶
HTTPStatusError is returned by CheckStatus when the response is not 2xx. Callers can use errors.As to extract the status code without parsing the Error() string.
func (*HTTPStatusError) Error ¶
func (e *HTTPStatusError) Error() string
Error returns the error message for HTTPStatusError.
func (*HTTPStatusError) IsClientError ¶
func (e *HTTPStatusError) IsClientError() bool
IsClientError reports whether the status is in the 4xx range. Useful for classifying transient repo failures (auth, rate-limit, not-found) as non-fatal vs systemic failures (network, 5xx, disk full).
Source Files