Documentation
¶
Overview ¶
Package httpx sends one Flowi request: protocol selection (auto, 1.1, 2, h2c), proxy and custom CAs, redirect chain capture, timings, response decompression, request bodies (including file uploads that must stay inside the collection), large downloads streamed to disk, retries with backoff and jitter, and a cookie jar that can be saved.
It knows nothing about templates or secrets: callers build the request and mask what they record.
Index ¶
- Constants
- Variables
- func StoredKey(s StoredCookie) string
- func TLSConfig(caFiles []string) (*tls.Config, error)
- func WriteStored(file string, cookies []StoredCookie) error
- type Attempt
- type BodySource
- type Client
- type FormField
- type Hop
- type Jar
- type Options
- type OutsideRootError
- type Part
- type Request
- type Response
- type RetryPolicy
- type StoredCookie
- type TimeoutError
- type Timings
Constants ¶
const ( RedirectStrip = "strip" RedirectSameSite = "same_site" RedirectKeep = "keep" )
Values of Options.RedirectCredentials.
Variables ¶
var ErrHTTP3Proxy = errors.New("HTTP/3 cannot go through a proxy; unset the proxy or choose another http.version")
ErrHTTP3Proxy is returned for HTTP/3 with a proxy configured: QUIC runs over UDP, which HTTP proxies do not carry, so the proxy would be skipped silently.
Functions ¶
func StoredKey ¶
func StoredKey(s StoredCookie) string
StoredKey identifies a stored cookie by name, domain and path: the triple that makes two cookies one.
func TLSConfig ¶
TLSConfig is the client TLS configuration for the given extra CA files (the system pool alone when there are none).
func WriteStored ¶
func WriteStored(file string, cookies []StoredCookie) error
WriteStored saves cookies as a jar file, as Save does.
Types ¶
type Attempt ¶
Attempt is one try at sending the request. Exactly one of Response and Err is set. Wait is the backoff before the next attempt (zero for the last).
type BodySource ¶
type BodySource interface {
Open() (r io.ReadCloser, length int64, contentType string, err error)
}
BodySource opens a request body. It is opened once per attempt, so retries resend the whole body. length is -1 when unknown.
func BytesBody ¶
func BytesBody(b []byte, contentType string) BodySource
BytesBody is a body held in memory.
func FileBody ¶
func FileBody(root, rel string) (BodySource, error)
FileBody sends a file of the collection. The path is checked now, before anything is sent; the file is reopened for every attempt.
func FormBody ¶
func FormBody(fields []FormField) BodySource
FormBody encodes fields in the order given (url.Values would sort them).
func MultipartBody ¶
func MultipartBody(root string, parts []Part) (BodySource, error)
MultipartBody streams parts as multipart/form-data. File paths are checked now, before anything is sent.
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client sends requests. One client serves one send or run; it owns its connection pool, released by CloseIdleConnections.
func (*Client) Close ¶
func (c *Client) Close()
Close releases everything the client holds. Call it when the client is done: an HTTP/3 transport owns a UDP socket that idle-connection cleanup does not close.
func (*Client) CloseIdleConnections ¶
func (c *Client) CloseIdleConnections()
CloseIdleConnections releases pooled connections.
type FormField ¶
type FormField struct{ Name, Value string }
FormField is one application/x-www-form-urlencoded field.
type Hop ¶
type Hop struct {
Status int
URL string // the URL that answered with the redirect
Location string
Header http.Header
// Forwarded names the credential headers sent on to Location even
// though it is another origin (RedirectCredentials, RedirectKeepHeaders).
Forwarded []string
}
Hop is one redirect response.
type Jar ¶
type Jar struct {
// contains filtered or unexported fields
}
Jar is a cookie jar that can be saved and loaded. The standard jar cannot list its cookies, so Jar keeps a log of what was set (the latest value of each cookie) and replays it into a fresh standard jar on load.
func LoadJar ¶
LoadJar reads a saved jar, dropping cookies expired at now. A missing file is an empty jar.
func (*Jar) All ¶
func (j *Jar) All() []StoredCookie
All returns every cookie currently held, in the order first set.
func (*Jar) Clone ¶
Clone returns an independent jar holding this jar's unexpired cookies: later changes to either do not reach the other.
type Options ¶
type Options struct {
// Transport, when set, sends every attempt instead of the network: a run played back from a recording.
Transport http.RoundTripper
Version string // auto | 1.1 | 2 | h2c | 3
FollowRedirects bool
MaxRedirects int
// RedirectCredentials says when a redirect may carry credential headers
// to a new origin: "strip" (the default, and "" means the same) drops
// them when the origin changes, "same_site" drops them only when the
// registrable domain changes or https falls back to http, "keep" never
// drops them. RedirectKeepHeaders names headers that are forwarded
// whatever the mode.
RedirectCredentials string
RedirectKeepHeaders []string
Timeout time.Duration // per attempt
Proxy *httpproxy.Config
CAFiles []string
ClientCert *tls.Certificate // sent when the server asks for one (mutual TLS)
RequestCompression string // none | gzip
Jar http.CookieJar // nil sends and keeps no cookies
StreamThreshold int64 // bodies larger than this stream to DownloadDir
DownloadDir string
Retry RetryPolicy
Clock func() time.Time // nil → time.Now
Rand *rand.Rand // retry jitter; nil → runtime-seeded
// BeforeAttempt and AfterAttempt, when set, are called around each
// attempt (n is 0-based) on the goroutine calling Do, for live progress.
BeforeAttempt func(n int)
AfterAttempt func(n int, a Attempt)
// Gate, when set, is called before each attempt with the target host
// and must return before the attempt starts; release is called when
// the attempt ends. Waiting in Gate (for a concurrency slot or a rate
// limit) does not count toward Timeout. An error stops Do.
Gate func(ctx context.Context, host string) (release func(), err error)
// contains filtered or unexported fields
}
Options configures a client. Every request it sends uses them.
type OutsideRootError ¶
type OutsideRootError struct{ Path string }
OutsideRootError means a file body path leaves the collection: an absolute path, "..", or a symlink pointing elsewhere. Collections are often cloned from others, so a request must not be able to upload arbitrary local files.
func (*OutsideRootError) Error ¶
func (e *OutsideRootError) Error() string
type Part ¶
Part is one multipart/form-data part: a value, or a file of the collection when File is set.
type Request ¶
type Request struct {
Method string
URL string
Header http.Header
Body BodySource // nil sends no body
}
Request is what to send. Header is sent as is, plus Accept-Encoding when the caller did not set one.
type Response ¶
type Response struct {
Status int
Proto string // "HTTP/1.1", "HTTP/2.0"
Header http.Header
Body []byte // decoded; nil when streamed to File
File string // where a streamed body was written
SHA256 string // hex digest of the decoded body
RawSize int64 // bytes on the wire
Size int64 // decoded bytes
Encoding string // Content-Encoding that was decoded; "" if none or unknown
Redirects []Hop
Timings Timings
}
Response is a received response.
type RetryPolicy ¶
type RetryPolicy struct {
Max int // extra attempts; 0 = no retries
Backoff string // exponential | constant
Jitter bool // full jitter: a random delay in [0, computed]
On []string // 5xx | 429 | timeout | connection
// contains filtered or unexported fields
}
RetryPolicy says which failed attempts are retried and how long to wait.
type StoredCookie ¶
StoredCookie is a cookie with the URL that set it.
func ReadStored ¶
func ReadStored(file string, now time.Time) ([]StoredCookie, error)
ReadStored reads the cookies of a saved jar without building a jar, dropping expired and damaged entries. A missing file holds none.
type TimeoutError ¶
TimeoutError means an attempt took longer than Options.Timeout.
func (*TimeoutError) Error ¶
func (e *TimeoutError) Error() string