Help us shape the future of Go! Take the 2026 Go Developer Survey

httpx

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 31, 2026 License: Apache-2.0 Imports: 38 Imported by: 0

Documentation

Overview

Package httpx sends one Flowi request: protocol selection (auto, 1.1, 2, h2c), proxy and custom CAs, redirect chain capture, timings, response decompression, request bodies (including file uploads that must stay inside the collection), large downloads streamed to disk, retries with backoff and jitter, and a cookie jar that can be saved.

It knows nothing about templates or secrets: callers build the request and mask what they record.

Index

Constants

View Source
const (
	RedirectStrip    = "strip"
	RedirectSameSite = "same_site"
	RedirectKeep     = "keep"
)

Values of Options.RedirectCredentials.

Variables

View Source
var ErrHTTP3Proxy = errors.New("HTTP/3 cannot go through a proxy; unset the proxy or choose another http.version")

ErrHTTP3Proxy is returned for HTTP/3 with a proxy configured: QUIC runs over UDP, which HTTP proxies do not carry, so the proxy would be skipped silently.

Functions

func StoredKey

func StoredKey(s StoredCookie) string

StoredKey identifies a stored cookie by name, domain and path: the triple that makes two cookies one.

func TLSConfig

func TLSConfig(caFiles []string) (*tls.Config, error)

TLSConfig is the client TLS configuration for the given extra CA files (the system pool alone when there are none).

func WriteStored

func WriteStored(file string, cookies []StoredCookie) error

WriteStored saves cookies as a jar file, as Save does.

Types

type Attempt

type Attempt struct {
	Response *Response
	Err      error
	Duration time.Duration
	Wait     time.Duration
}

Attempt is one try at sending the request. Exactly one of Response and Err is set. Wait is the backoff before the next attempt (zero for the last).

type BodySource

type BodySource interface {
	Open() (r io.ReadCloser, length int64, contentType string, err error)
}

BodySource opens a request body. It is opened once per attempt, so retries resend the whole body. length is -1 when unknown.

func BytesBody

func BytesBody(b []byte, contentType string) BodySource

BytesBody is a body held in memory.

func FileBody

func FileBody(root, rel string) (BodySource, error)

FileBody sends a file of the collection. The path is checked now, before anything is sent; the file is reopened for every attempt.

func FormBody

func FormBody(fields []FormField) BodySource

FormBody encodes fields in the order given (url.Values would sort them).

func MultipartBody

func MultipartBody(root string, parts []Part) (BodySource, error)

MultipartBody streams parts as multipart/form-data. File paths are checked now, before anything is sent.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client sends requests. One client serves one send or run; it owns its connection pool, released by CloseIdleConnections.

func NewClient

func NewClient(o Options) (*Client, error)

NewClient builds a client. Unreadable or invalid CA files are errors.

func (*Client) Close

func (c *Client) Close()

Close releases everything the client holds. Call it when the client is done: an HTTP/3 transport owns a UDP socket that idle-connection cleanup does not close.

func (*Client) CloseIdleConnections

func (c *Client) CloseIdleConnections()

CloseIdleConnections releases pooled connections.

func (*Client) Do

func (c *Client) Do(ctx context.Context, r Request) ([]Attempt, error)

Do sends r, retrying per Options.Retry, and returns every attempt in order. Transport failures are reported in Attempt.Err; the error result is only for ctx being done.

type FormField

type FormField struct{ Name, Value string }

FormField is one application/x-www-form-urlencoded field.

type Hop

type Hop struct {
	Status   int
	URL      string // the URL that answered with the redirect
	Location string
	Header   http.Header
	// Forwarded names the credential headers sent on to Location even
	// though it is another origin (RedirectCredentials, RedirectKeepHeaders).
	Forwarded []string
}

Hop is one redirect response.

type Jar

type Jar struct {
	// contains filtered or unexported fields
}

Jar is a cookie jar that can be saved and loaded. The standard jar cannot list its cookies, so Jar keeps a log of what was set (the latest value of each cookie) and replays it into a fresh standard jar on load.

func LoadJar

func LoadJar(file string, now time.Time) (*Jar, error)

LoadJar reads a saved jar, dropping cookies expired at now. A missing file is an empty jar.

func NewJar

func NewJar() *Jar

NewJar returns an empty in-memory jar.

func (*Jar) All

func (j *Jar) All() []StoredCookie

All returns every cookie currently held, in the order first set.

func (*Jar) Clone

func (j *Jar) Clone() *Jar

Clone returns an independent jar holding this jar's unexpired cookies: later changes to either do not reach the other.

func (*Jar) Cookies

func (j *Jar) Cookies(u *url.URL) []*http.Cookie

Cookies implements http.CookieJar.

func (*Jar) Save

func (j *Jar) Save(file string) error

Save writes the jar atomically, dropping cookies that have expired. Session cookies (no expiry) are kept: a series of standalone sends acts as one long browser session until the jar is cleared.

func (*Jar) SetCookies

func (j *Jar) SetCookies(u *url.URL, cookies []*http.Cookie)

SetCookies implements http.CookieJar.

type Options

type Options struct {
	// Transport, when set, sends every attempt instead of the network: a run played back from a recording.
	Transport       http.RoundTripper
	Version         string // auto | 1.1 | 2 | h2c | 3
	FollowRedirects bool
	MaxRedirects    int
	// RedirectCredentials says when a redirect may carry credential headers
	// to a new origin: "strip" (the default, and "" means the same) drops
	// them when the origin changes, "same_site" drops them only when the
	// registrable domain changes or https falls back to http, "keep" never
	// drops them. RedirectKeepHeaders names headers that are forwarded
	// whatever the mode.
	RedirectCredentials string
	RedirectKeepHeaders []string
	Timeout             time.Duration // per attempt
	Proxy               *httpproxy.Config
	CAFiles             []string
	ClientCert          *tls.Certificate // sent when the server asks for one (mutual TLS)
	RequestCompression  string           // none | gzip
	Jar                 http.CookieJar   // nil sends and keeps no cookies
	StreamThreshold     int64            // bodies larger than this stream to DownloadDir
	DownloadDir         string
	Retry               RetryPolicy
	Clock               func() time.Time // nil → time.Now
	Rand                *rand.Rand       // retry jitter; nil → runtime-seeded
	// BeforeAttempt and AfterAttempt, when set, are called around each
	// attempt (n is 0-based) on the goroutine calling Do, for live progress.
	BeforeAttempt func(n int)
	AfterAttempt  func(n int, a Attempt)
	// Gate, when set, is called before each attempt with the target host
	// and must return before the attempt starts; release is called when
	// the attempt ends. Waiting in Gate (for a concurrency slot or a rate
	// limit) does not count toward Timeout. An error stops Do.
	Gate func(ctx context.Context, host string) (release func(), err error)
	// contains filtered or unexported fields
}

Options configures a client. Every request it sends uses them.

type OutsideRootError

type OutsideRootError struct{ Path string }

OutsideRootError means a file body path leaves the collection: an absolute path, "..", or a symlink pointing elsewhere. Collections are often cloned from others, so a request must not be able to upload arbitrary local files.

func (*OutsideRootError) Error

func (e *OutsideRootError) Error() string

type Part

type Part struct {
	Name, Value       string
	File, ContentType string
}

Part is one multipart/form-data part: a value, or a file of the collection when File is set.

type Request

type Request struct {
	Method string
	URL    string
	Header http.Header
	Body   BodySource // nil sends no body
}

Request is what to send. Header is sent as is, plus Accept-Encoding when the caller did not set one.

type Response

type Response struct {
	Status    int
	Proto     string // "HTTP/1.1", "HTTP/2.0"
	Header    http.Header
	Body      []byte // decoded; nil when streamed to File
	File      string // where a streamed body was written
	SHA256    string // hex digest of the decoded body
	RawSize   int64  // bytes on the wire
	Size      int64  // decoded bytes
	Encoding  string // Content-Encoding that was decoded; "" if none or unknown
	Redirects []Hop
	Timings   Timings
}

Response is a received response.

type RetryPolicy

type RetryPolicy struct {
	Max     int      // extra attempts; 0 = no retries
	Backoff string   // exponential | constant
	Jitter  bool     // full jitter: a random delay in [0, computed]
	On      []string // 5xx | 429 | timeout | connection
	// contains filtered or unexported fields
}

RetryPolicy says which failed attempts are retried and how long to wait.

type StoredCookie

type StoredCookie struct {
	URL    string      `json:"url"`
	Cookie http.Cookie `json:"cookie"`
}

StoredCookie is a cookie with the URL that set it.

func ReadStored

func ReadStored(file string, now time.Time) ([]StoredCookie, error)

ReadStored reads the cookies of a saved jar without building a jar, dropping expired and damaged entries. A missing file holds none.

type TimeoutError

type TimeoutError struct{ After time.Duration }

TimeoutError means an attempt took longer than Options.Timeout.

func (*TimeoutError) Error

func (e *TimeoutError) Error() string

type Timings

type Timings struct {
	DNS, Connect, TLS, TTFB, Total time.Duration
}

Timings of one attempt. Total runs from sending to the end of the body.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL