Documentation
¶
Overview ¶
Package chttp provides the HTTP client every katbyte tool talks to APIs with: per-attempt timeouts so a stalled server fails fast, retries for transient failures that are careful never to re-send a mutation whose outcome is unknown, and a trace of every exchange for whoever hands it a logger.
It imports nothing outside the standard library and logs nothing of its own accord, so an SDK can be built on it without handing whoever uses that SDK a logger they did not ask for. The application that wants the traffic in its log says so when it makes the client:
client := chttp.New(chttp.Options{Name: "Audiobookshelf", Log: clog.Log})
What counts as worth another try, how long to wait for an answer and what is too secret to print are each API's own, so each is the caller's to set (Options); the defaults are the cautious ones.
Index ¶
- Constants
- Variables
- func Dropped(err error) bool
- func Explain(err error) error
- func IsNotFound(err error) bool
- func IsWebPage(contentType string, body []byte) bool
- func KeepCredentialsOnHost(headers ...string) func(req *http.Request, via []*http.Request) error
- func MarkRetrySafe(req *http.Request) *http.Request
- func NewBaseTransport(o Options) http.RoundTripper
- func Preview(body []byte) string
- func RedactError(err error, also ...string) error
- func RedactJSON(text string, also ...string) string
- func RedactURL(raw string, also ...string) string
- func RefuseRedirects(advice string) func(req *http.Request, via []*http.Request) error
- func Refused(err error) bool
- func StatusCode(err error) int
- func Tries(resp *http.Response) int
- type Client
- type Logger
- type Options
- type RedirectError
- type Retry
- type RetryTransport
- type StatusError
- type Transport
Constants ¶
const DefaultHeaderWait = 30 * time.Second
DefaultHeaderWait is how long a server has to start answering.
const DefaultTraceBody = 8 << 10
DefaultTraceBody is how much of a body a trace prints: enough to see what was asked and what came back, and not a library's whole listing.
const DefaultTries = 3
DefaultTries is how many times a request is sent before giving up: enough to ride through a blip or a single rate-limit window without turning an outage into a minutes-long hang.
const MaxRetryAfter = 60 * time.Second
MaxRetryAfter caps how long a Retry-After header can make a retry wait. A server asking for more than this is telling an interactive tool to come back later, not to sit there.
const PreviewLen = 300
PreviewLen is how much of a response body a StatusError carries.
Variables ¶
var ErrTooLarge = errors.New("the answer is too large")
ErrTooLarge is an answer longer than the caller of Fetch said it would hold.
Functions ¶
func Dropped ¶ added in v0.5.0
Dropped reports whether a request failed because a connection that was there went away: reset or closed by the other end, or ended in the middle of an answer, over HTTP/2 as a stream the server or a proxy gave up. It is what a client takes as worth another try unless told otherwise (see Retry).
A server that could not be reached at all, a name that did not resolve, a certificate that did not check out, a wait that ran out and a request that was cancelled are not: the next attempt would meet the same.
func Explain ¶ added in v0.4.0
Explain adds, to the error of a request that never got an answer, what the operating system may be doing about it where that is known: a dial a Mac refused with "no route to host" is usually its Local Network privacy, which no retry and no server fixes, and the bare error sends its reader to the network. The error keeps what it wraps. One there is nothing to say about, and one already explained, comes back as it was.
Transport does this for every request it carries; Explain is for a client built on another transport.
func IsNotFound ¶ added in v0.4.0
IsNotFound reports whether err is a 404 from the server.
func IsWebPage ¶ added in v0.5.0
IsWebPage reports whether an answer is an HTML document where an API's own answer was expected: a wrong address or a proxy's login page answers 200 with a page, for a request that never reached the API.
The body decides. A proxy can label anything anything, so a document that starts as HTML is a page whatever it is called; and some servers label their own one-word replies as HTML, so what is called HTML is a page only when it starts with a tag.
func KeepCredentialsOnHost ¶ added in v0.4.0
KeepCredentialsOnHost is a redirect policy, for an http.Client's CheckRedirect: a redirect is followed (up to Go's usual ten), but one that leaves the first request's host, or goes from https down to http, reaches its target without the credentials.
Go itself drops only Authorization on the way to another domain, and keeps it for a subdomain; a token an API takes in a header of its own it would hand to whatever host a redirect names. This drops Authorization and the headers named. Another port on the same host is the same machine, and is left alone.
func MarkRetrySafe ¶
MarkRetrySafe declares a request safe to re-send even though its method is not idempotent: a GraphQL or JQL query is a read that happens to travel as a POST. Reads with idempotent methods (GET, HEAD, OPTIONS) need no mark.
func NewBaseTransport ¶
func NewBaseTransport(o Options) http.RoundTripper
NewBaseTransport returns http.DefaultTransport tuned with per-attempt timeouts so a stalled connection or unresponsive server fails fast instead of hanging the command: ten seconds to connect, ten for TLS, and the options' HeaderWait for the server to start answering. It is exported so callers that must build their own client (oauth2, for one) can still start from the same transport.
func Preview ¶ added in v0.4.0
Preview is the start of a response body for an error to carry: the space around it gone, cut at PreviewLen bytes on a whole character, with "..." to say there was more.
func RedactError ¶ added in v0.4.0
RedactError hides the credentials in the URL a failed request names. A request that gets no answer (a timeout, a refused connection) fails with a *url.Error that prints its whole URL, and a key sent in the query is in it, so it would reach a log or a tool's answer. The error keeps its type and what it wraps; only the URL it prints changes (see RedactURL).
It is for the place the error is first seen: a message already built around the error (fmt.Errorf with %w) keeps the text it was built with.
func RedactJSON ¶ added in v0.6.1
RedactJSON is JSON text, whole or cut short, with the value of every string field that holds a credential blanked: a field of one of the names an API passes a credential under, one of also, or one whose name ends as a credential's does (password, secret, token, apikey, api_key). It is the rule a trace hides a body's credentials by, for whatever else writes down what it was sent.
func RedactURL ¶ added in v0.4.0
RedactURL replaces the value of every credential query parameter in a URL with REDACTED, and any password in it, leaving the rest as it was sent: the parameters around it, their order and a fragment. The parameters are the common ones (api_key, apikey, api_token, access_token, token) and whatever also names, compared without case.
func RefuseRedirects ¶ added in v0.5.0
RefuseRedirects is a redirect policy, for an http.Client's CheckRedirect: no redirect is followed, and the request fails with a *RedirectError that says where it was sent and gives the advice.
It is for an API that answers where it is asked, so that a redirect means the address points at something in front of it: an http address a proxy moves to https, or a login page. Following one is worse than failing. Go turns a DELETE, PATCH or POST into a GET on a 301, 302 or 303 and drops its body, the GET answers 200, and the write reports success having done nothing. An API that does redirect wants KeepCredentialsOnHost.
func Refused ¶ added in v0.5.0
Refused reports whether a request failed because nothing was listening where it was sent. That is not a dropped connection, and a client does not try again for it unless told to: a caller whose server restarts under it says so, with func(err error) bool { return Dropped(err) || Refused(err) } as its Retry's Error.
func StatusCode ¶ added in v0.4.0
StatusCode is the status of a *StatusError in err's chain, or 0.
Types ¶
type Client ¶ added in v0.5.0
Client is an http.Client whose requests are tried again and traced as its Options say. It is used as one - Do, Timeout and CheckRedirect are all there - and adds Fetch, for an answer read whole.
func New ¶ added in v0.5.0
New returns a client made as the options say. It sets no limit on a whole request and follows redirects as Go does: set Timeout and CheckRedirect on it for an API that wants otherwise (see RefuseRedirects and KeepCredentialsOnHost).
Timeout, when set, is for everything one Do does: every try it makes and the waits between them, since the tries are made underneath it. Each time Fetch asks again is a Do of its own, with the whole Timeout to itself.
func (*Client) Fetch ¶ added in v0.5.0
Fetch sends a request and reads its whole answer, for one small enough to hold: at most limit bytes, and a longer one is an error (ErrTooLarge) rather than an answer cut to fit. A limit of math.MaxInt64 is no limit. The response comes back with its body read and closed, whatever its status.
An answer that stops part way - a proxy that drops the connection in the middle of a body - is asked for again as any other dropped connection is, which the retry inside a transport cannot do: the body is read after the transport has returned. A request that is not safe to repeat is not sent again for it, and none is sent more than Retry's Tries in all, however it failed each time.
type Logger ¶ added in v0.5.0
Logger is where a client says what it is doing: a retry at debug, and each request and answer at trace. A logrus logger is one as it stands, clog.Log among them.
What is traced is put together only when the logger formats it, and is read from the request and the answer at that moment. So a logger must format before Tracef returns, and one that drops a level without formatting, as logrus does, costs nothing while that level is off.
type Options ¶ added in v0.5.0
type Options struct {
// Name says whose traffic this is in a log, so a tool talking to two
// APIs can tell them apart.
Name string
// Log is where retries and the trace of each exchange go. Nil logs
// nothing.
Log Logger
// Retry is when a request is sent again.
Retry Retry
// HeaderWait is how long the server has to start answering one
// attempt; 0 is DefaultHeaderWait. An API with calls that are slow to
// start answering - a search it runs before it writes a byte - wants
// longer. Reading the answer is not bounded by it.
HeaderWait time.Duration
// SecretHeaders are headers whose values a trace must not show, beside
// Authorization, Proxy-Authorization, Cookie and Set-Cookie: the header
// of its own an API takes a key in.
SecretHeaders []string
// SecretNames are the names whose values a trace must not show wherever
// they appear - a query parameter, a form field, a JSON field - compared
// without case. They are beside the ones a trace hides unasked: any name
// that ends in password, secret, token, apikey or api_key, so
// proxyPassword, refresh_token and TmdbApiKey need no naming here and an
// API's "pass" or "pin" does.
SecretNames []string
// TraceBody is how much of a body a trace prints, in bytes: 0 is
// DefaultTraceBody, and a negative number prints no body at all. No
// more than this is ever read to print it.
TraceBody int
// Base is what requests are sent through underneath the retries and
// the trace; nil is NewBaseTransport. A test hands in its own, and so
// does an application with a proxy or certificates of its own.
Base http.RoundTripper
}
Options are what a client is made with. The zero value is a client that logs nothing and retries with the defaults.
type RedirectError ¶ added in v0.5.0
type RedirectError struct {
// Method and Path are the request that was redirected
Method string
Path string
// To is where it was sent, with the credentials in it hidden
To string
// Advice is what the caller knows to do about it on this API: which
// setting holds the address to put right
Advice string
}
RedirectError is a request that was redirected and not followed (see RefuseRedirects).
func (*RedirectError) Error ¶ added in v0.5.0
func (e *RedirectError) Error() string
type Retry ¶ added in v0.5.0
type Retry struct {
// Tries is the most a request is ever sent; 0 is DefaultTries and 1
// never sends it again. It is one number however the request fails: a
// read that Fetch asks for again does not start the count afresh.
Tries int
// Wait is how long to wait after the attempt numbered from 0 failed;
// nil is 1s, 2s, 4s and so on. A 429's Retry-After is used in its
// place, up to MaxRetryAfter.
Wait func(attempt int) time.Duration
// Status says whether an answer with this status is worth another try;
// nil is 502, 503 and 504, which a gateway gives for a server it could
// not reach. A plain 500 is left out: many servers answer it for what
// will never succeed, and trying those again only makes the failure
// slower. An API that uses 500 for "busy" adds it here.
Status func(code int) bool
// Error says whether a request that got no answer is worth sending
// again; nil is Dropped, a connection that was there and went. A server
// that cannot be reached at all is not one, and neither is a timeout:
// waiting the same wait again is rarely what is wanted.
Error func(err error) bool
}
Retry is when a request is sent again, and how long after. The zero value is the default: three tries a second and then two apart, for an answer of 502, 503 or 504 and for a connection that dropped.
A request is only ever sent again when that cannot do the work twice. One refused with 429 was turned away before it was acted on, so any request may be; otherwise only a request that is safe to repeat is - a GET, HEAD or OPTIONS, or one marked with MarkRetrySafe. A write that gets no answer, or an error for one, may still have been made.
type RetryTransport ¶
type RetryTransport struct {
// contains filtered or unexported fields
}
RetryTransport wraps an http.RoundTripper with the retries the options ask for (see Retry): 429 for every request, and a dropped connection or a status worth another try for requests that are safe to repeat. A 429's Retry-After header is honoured up to MaxRetryAfter, and a cancelled request context aborts the wait.
func NewRetryTransport ¶
func NewRetryTransport(o Options, next http.RoundTripper) *RetryTransport
NewRetryTransport wraps next with the retries the options ask for.
type StatusError ¶ added in v0.4.0
type StatusError struct {
Method string
Path string
// StatusCode is the status the server answered with
StatusCode int
// Expected are the statuses the request takes as success, when it
// names them
Expected []int
// Body is the start of the response body (see Preview)
Body string
// Tries is how many times the request was sent, when more than once
Tries int
// Note is what the caller knows about this status on this API: which
// key to check for a 401, what permission a 403 wants
Note string
}
StatusError is an answer with a status the request did not ask for.
func (*StatusError) Error ¶ added in v0.4.0
func (e *StatusError) Error() string
type Transport ¶
type Transport struct {
// contains filtered or unexported fields
}
Transport is an http.RoundTripper that traces each request and answer to the options' Log, with what is secret blanked (see Options) and JSON bodies pretty-printed, and that says of a request that got no answer what the operating system may be doing about it, where that is known (see Explain). With no Log it traces nothing and only explains.
While tracing is on, the start of an answer that is text is read before the answer is handed back, up to what the trace prints: an answer that trickles in is held until that much has come or it ends.
func NewTransport ¶
func NewTransport(o Options, next http.RoundTripper) *Transport
NewTransport wraps next with the trace the options ask for.