httpreq

package module
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 21, 2026 License: Apache-2.0 Imports: 27 Imported by: 0

README

httpreq

httpreq provides an abstraction API over Go's net/http. One-shot verb funcs (Get, Post, Put, Patch, Delete, Head, Options), a RequestOptions struct covering the usual pain points (JSON/XML/form bodies, multipart uploads, query structs, proxies, cookie jars, TLS/dial/request timeouts, redirect handling, OTel tracing), and Session for cookie-jar-backed request reuse.

Heavily based on levigross/grequests (archived); see NOTICE.

go get github.com/heyvito/httpreq

Usage

resp, err := httpreq.Get(ctx, "https://api.example.com/things", nil)
if err != nil {
    // network/transport error; resp.Error is also set
}
defer resp.Close()

if resp.OK { // 2xx
    var things []Thing
    if err := resp.JSON(&things); err != nil {
        // ...
    }
}

Response also exposes String(), Bytes(), XML(&v, charsetReader), DownloadToFile(path), and satisfies io.Reader. String/Bytes buffer the body internally so they can be called more than once; JSON/XML/ DownloadToFile close the response when done.

Request bodies
httpreq.Post(ctx, url, &httpreq.RequestOptions{JSON: payload})
httpreq.Post(ctx, url, &httpreq.RequestOptions{XML: payload})
httpreq.Post(ctx, url, &httpreq.RequestOptions{Data: map[string]string{"a": "b"}}) // form-urlencoded
httpreq.Post(ctx, url, &httpreq.RequestOptions{RequestBody: someReader})           // takes precedence over the above

JSON/XML accept string, []byte, or any marshalable value.

Query params
&httpreq.RequestOptions{Params: map[string]string{"q": "1"}}
// or, mutually exclusive with Params:
&httpreq.RequestOptions{QueryStruct: myStruct} // github.com/google/go-querystring tags
File uploads
files, _ := httpreq.FileUploadFromDisk("report.pdf")
httpreq.Post(ctx, url, &httpreq.RequestOptions{Files: files, Data: map[string]string{"note": "hi"}})

FileUploadFromGlob builds []FileUpload from a glob pattern, skipping files that fail to stat. Multipart (POST) accepts multiple files; PUT/ PATCH send the raw body and accept exactly one (ErrInvalidFileAmount otherwise).

Sessions
sess, _ := httpreq.NewSession(&httpreq.RequestOptions{UserAgent: "my-client/1.0"})
resp, err := sess.Get(ctx, url, nil) // shares cookie jar + client across calls
sess.CloseIdleConnections()

Per-call RequestOptions passed to a session method are merged on top of the session's defaults (UserAgent, Host, Auth, Headers).

Other options worth knowing about
  • InsecureSkipVerify, Proxies map[scheme]*url.URL, LocalAddr, DialTimeout/DialKeepAlive/TLSHandshakeTimeout/RequestTimeout, HTTPClient (bypasses all of the above), BeforeRequest func(*http.Request) error.
  • RedirectLimit (default 30, global RequestRedirectLimit) and SensitiveHTTPHeaders (default RequestSensitiveHTTPHeaders: Authorization, Proxy-Authorization, Www-Authenticate) control redirect behaviour. Sensitive headers are stripped on cross-request redirects, everything else is forwarded.
  • EnableTracing wraps the transport with otelhttp, producing a client span per round trip (incl. redirects) and injecting trace context from ctx. TracerProvider overrides the otelhttp default. Set the package var RequestEnableTracing = true to enable it process-wide.

License

Apache License 2.0 — see LICENSE. Portions derived from grequests, Copyright 2015 Levi Gross; see NOTICE for attribution.

Documentation

Overview

Package httpreq implements a friendly API over Go's existing net/http library. It is heavily based on the archived github.com/levigross/grequests; see the NOTICE file at the root of this module for attribution.

Index

Constants

This section is empty.

Variables

View Source
var (

	// ErrRedirectLimitExceeded indicates the response yielded too many
	// redirects.
	ErrRedirectLimitExceeded = errors.New("httpreq: request exceeded redirect count")

	// ErrInvalidFileAmount indicates that more than one file was provided to a
	// PATCH or PUT request, which is not supported.
	ErrInvalidFileAmount = errors.New("httpreq: PUT and PATCH requests supports at most one file")

	// RequestRedirectLimit is a tunable variable that specifies how many times
	// redirects can be made in response to a redirect. This is the global
	// variable; to set this on a request by request basis, set it within the
	// `RequestOptions` structure.
	RequestRedirectLimit = 30

	// RequestEnableTracing is a tunable variable that, when true, enables
	// OpenTelemetry tracing (see RequestOptions.EnableTracing) on every
	// request regardless of what that particular request's RequestOptions
	// set. This is the global variable, meant to be set once at startup so
	// individual call sites don't each need `EnableTracing: true`; a request
	// can still set `RequestOptions.EnableTracing = true` on its own even
	// when this is false, but — since a bare bool can't tell "unset" apart
	// from "false" — there is no per-request way to opt out while this is
	// true.
	RequestEnableTracing bool

	// RequestSensitiveHTTPHeaders is a map of sensitive HTTP headers that must
	// not be passed on a redirect. This is the global variable; to set this on
	// a request by request basis, set it within the `RequestOptions` structure.
	RequestSensitiveHTTPHeaders = map[string]struct{}{
		"Www-Authenticate":    {},
		"Authorization":       {},
		"Proxy-Authorization": {},
	}
)

Functions

func BuildHTTPClient

func BuildHTTPClient(ro RequestOptions) (*http.Client, error)

BuildHTTPClient returns a custom HTTP client based on values provided to RequestOptions.

Types

type FileUpload

type FileUpload struct {
	// Filename is the name of the file to upload. This is used to guess the mimetype as well as pass it onto the server
	FileName string

	// FileContents just needs to be an io.ReadCloser (which most files satisfy anyway)
	FileContents io.ReadCloser

	// FieldName is form field name
	FieldName string

	// FileMime represents which mimetime should be sent along with the file.
	// When empty, defaults to application/octet-stream
	FileMime string
}

FileUpload is a struct that is used to specify the file that a User wishes to upload.

func FileUploadFromDisk

func FileUploadFromDisk(fileName string) ([]FileUpload, error)

FileUploadFromDisk creates a FileUpload struct slice from a location on disk

func FileUploadFromGlob

func FileUploadFromGlob(fileSystemGlob string) ([]FileUpload, error)

FileUploadFromGlob creates a FileUpload struct slice from a glob location on disk this function will gloss over all errors in the files and only upload the files that don't return errors from the glob

type RequestOptions

type RequestOptions struct {
	// Data is a map of key values that will eventually convert into the
	// body of a POST request.
	Data map[string]string

	// Params is a map of query strings to be attached to the request.
	Params map[string]string

	// QueryStruct is a struct that encapsulates a set of URL query params
	// this parameter is mutually exclusive with `Params map[string]string`
	// (they cannot be combined). For more information, see
	// https://godoc.org/github.com/google/go-querystring/query
	QueryStruct any

	// Files accepts files to be uploaded. The use of this field is limited to
	// POST requests.
	Files []FileUpload

	// JSON, when set, marshalls its contents as JSON on request body.
	JSON any

	// XML, when set, marshalls its contents as XML on the request body.
	XML any

	// Headers defines custom HTTP headers to the request.
	Headers map[string]string

	// InsecureSkipVerify indicates if the server's TLS certificate must be
	// validated. Note that that Go's TLS verify mechanism doesn't validate if a
	// certificate has been revoked
	InsecureSkipVerify bool

	// DisableCompression, when set, disables gzip compression on this request.
	DisableCompression bool

	// UserAgent sets an arbitrary custom user agent for the request.
	UserAgent string

	// Host sets an arbitrary host for the request.
	Host string

	// Auth specifies a username and password to use with the request. It
	// will use basic HTTP authentication formatting the username and password
	// as specified per the HTTP Protocol Specification, being
	// base64(username ":" password)
	Auth []string

	// IsAjax sets whether the request should carry headers as if it was
	// sent by a browser's JavaScript engine.
	IsAjax bool

	// Cookies attaches the provided array of `http.Cookie` to the request.
	Cookies []*http.Cookie

	// UseCookieJar creates a custom HTTP client that processes and stores HTTP
	// cookies when they are sent down.
	UseCookieJar bool

	// Proxies is a map in the following format:
	// 	*protocol* => proxy address
	// Example:
	//	http => http://127.0.0.1:8080
	Proxies map[string]*stdurl.URL

	// TLSHandshakeTimeout specifies the maximum amount of time to
	// wait for a TLS handshake. Zero means no timeout.
	TLSHandshakeTimeout time.Duration

	// DialTimeout specifies the maximum amount of time a dial will wait for
	// connect to complete.
	DialTimeout time.Duration

	// DialKeepAlive specifies the keep-alive period for an active
	// network connection. If zero, keep-alive are not enabled.
	DialKeepAlive time.Duration

	// RequestTimeout specifies the maximum amount of time a whole request
	// (dial / request / redirects) may take.
	RequestTimeout time.Duration

	// ResponseHeaderTimeout, if non-zero, specifies the amount of time to
	// wait for a server's response headers after fully writing the
	// request (including its body, if any). Zero means no timeout.
	ResponseHeaderTimeout time.Duration

	// ExpectContinueTimeout, if non-zero, specifies the amount of time to
	// wait for a server's first response headers after fully writing the
	// request headers if the request has an "Expect: 100-continue"
	// header. Zero means no timeout.
	ExpectContinueTimeout time.Duration

	// MaxResponseHeaderBytes specifies a limit on how many response bytes
	// are allowed in the server's response header. Zero means the
	// default value provided by the default http.Client is used (10MB).
	MaxResponseHeaderBytes int64

	// HTTPClient can be provided to supply a custom HTTP client;
	// this is useful for using an OAuth client with the request.
	HTTPClient *http.Client

	// SensitiveHTTPHeaders specifies a map of sensitive HTTP headers that
	// should not be passed on a redirect.
	SensitiveHTTPHeaders map[string]struct{}

	// RedirectLimit specifies the maximum acceptable amount of redirects that
	// we should expect before returning an error; by default this is set to 30.
	// This can be changed globally by modifying the `httpreq.RedirectLimit`
	// variable.
	RedirectLimit int

	// RequestBody accepts anything matching an `io.Reader` for the request
	// body. This option takes precedence over any other request option
	// specified.
	RequestBody io.Reader

	// CookieJar specifies an http.CookieJar to be used with the request.
	// This option takes precedence over `UseCookieJar`.
	CookieJar http.CookieJar

	// Context can be used to maintain state and cancellation between requests.
	Context context.Context

	// BeforeRequest is a hook that allows the request object to be modified
	// before it has been fired. This is useful for adding authentication
	// and other functionality not provided in this library.
	BeforeRequest func(req *http.Request) error

	// LocalAddr specifies which local interface must be used for the request.
	LocalAddr *net.TCPAddr

	// EnableTracing wraps the request's transport with OpenTelemetry
	// instrumentation (go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp),
	// producing a client span per HTTP round trip — including one per redirect
	// hop — and injecting the trace context into outgoing request headers via
	// the configured propagator. The span's parent is read from the context
	// passed to the request (Get, Session.Get, etc.), so it composes with
	// whatever tracing setup already produced that context.
	//
	// Tracing is also enabled when the `RequestEnableTracing` package variable
	// is true, regardless of this field; set that once instead of this field
	// on every request to enable tracing process-wide.
	EnableTracing bool

	// TracerProvider overrides the TracerProvider used when tracing is
	// enabled (see EnableTracing). When nil, otelhttp falls back to the
	// global TracerProvider (otel.GetTracerProvider()). Has no effect when
	// tracing is disabled.
	TracerProvider oteltrace.TracerProvider
}

RequestOptions holds the set of options that configure a single request

type Response

type Response struct {
	// OK indicates whether the server returned a 2xx code
	OK bool

	// Error contains any error that may have occurred during the request.
	Error error

	// RawResponse contains the http.Response object generated by downstream
	// mechanisms.
	RawResponse *http.Response

	// StatusCode represents the HTTP Status Code returned by the server.
	StatusCode int

	// Header contains the set of headers returned by the server.
	Header http.Header
	// contains filtered or unexported fields
}

Response represents the result of an HTTP request

func Delete

func Delete(ctx context.Context, url string, options *RequestOptions) (*Response, error)

Delete takes a context, an url, and optional options, performs the request and returns its Response.

func Get

func Get(ctx context.Context, url string, options *RequestOptions) (*Response, error)

Get takes a context, an url, and optional options, performs the request and returns its Response.

func Head(ctx context.Context, url string, options *RequestOptions) (*Response, error)

Head takes a context, an url, and optional options, performs the request and returns its Response.

func Options

func Options(ctx context.Context, url string, options *RequestOptions) (*Response, error)

Options takes a context, an url, and optional options, performs the request and returns its Response.

func Patch

func Patch(ctx context.Context, url string, options *RequestOptions) (*Response, error)

Patch takes a context, an url, and optional options, performs the request and returns its Response.

func Post

func Post(ctx context.Context, url string, options *RequestOptions) (*Response, error)

Post takes a context, an url, and optional options, performs the request and returns its Response.

func Put

func Put(ctx context.Context, url string, options *RequestOptions) (*Response, error)

Put takes a context, an url, and optional options, performs the request and returns its Response.

func Request

func Request(ctx context.Context, verb, url string, options *RequestOptions) (*Response, error)

Request takes a context, an HTTP verb, an url, and optional options, performs the request and returns its Response.

func (*Response) Bytes

func (r *Response) Bytes() []byte

Bytes returns the response body as a byte slice

func (*Response) ClearInternalBuffer

func (r *Response) ClearInternalBuffer()

ClearInternalBuffer clears the internal buffer that maintains data used by String and Bytes.

func (*Response) Close

func (r *Response) Close() error

func (*Response) DownloadToFile

func (r *Response) DownloadToFile(fileName string) error

DownloadToFile downloads the contents of the response to a provided path.

func (*Response) JSON

func (r *Response) JSON(userStruct any) error

JSON unmarshals the obtained response body into the provided userStruct pointer.

func (*Response) Read

func (r *Response) Read(p []byte) (n int, err error)

func (*Response) String

func (r *Response) String() string

String returns the response body as a string

func (*Response) XML

func (r *Response) XML(userStruct any, charsetReader XMLCharDecoder) error

XML unmarshals the obtained response body into the provided userStruct pointer.

type Session

type Session struct {
	// RequestOptions is global options for this session.
	RequestOptions *RequestOptions

	// HTTPClient is the client that will be used to perform this session's
	// requests.
	HTTPClient *http.Client
}

Session makes use of persistent cookies across HTTP requests.

func NewSession

func NewSession(ro *RequestOptions) (*Session, error)

NewSession creates a new Session which can be used to maintain state across requests. This function unconditionally sets UseCookieJar as that is the purpose of leveraging this facility.

func (*Session) CloseIdleConnections

func (s *Session) CloseIdleConnections()

CloseIdleConnections closes the sessions' idle connections, if any.

This goes through http.Client.CloseIdleConnections rather than asserting the Transport is a *http.Transport directly, since RequestOptions.EnableTracing (and any custom RequestOptions.HTTPClient) may wrap it in another http.RoundTripper.

func (*Session) Delete

func (s *Session) Delete(ctx context.Context, url string, ro *RequestOptions) (*Response, error)

Delete takes a context, url, and optional options, executing the request through the current Session.

func (*Session) Get

func (s *Session) Get(ctx context.Context, url string, ro *RequestOptions) (*Response, error)

Get takes a context, url, and optional options, executing the request through the current Session.

func (*Session) Head

func (s *Session) Head(ctx context.Context, url string, ro *RequestOptions) (*Response, error)

Head takes a context, url, and optional options, executing the request through the current Session.

func (*Session) Options

func (s *Session) Options(ctx context.Context, url string, ro *RequestOptions) (*Response, error)

Options takes a context, url, and optional options, executing the request through the current Session.

func (*Session) Patch

func (s *Session) Patch(ctx context.Context, url string, ro *RequestOptions) (*Response, error)

Patch takes a context, url, and optional options, executing the request through the current Session.

func (*Session) Post

func (s *Session) Post(ctx context.Context, url string, ro *RequestOptions) (*Response, error)

Post takes a context, url, and optional options, executing the request through the current Session.

func (*Session) Put

func (s *Session) Put(ctx context.Context, url string, ro *RequestOptions) (*Response, error)

Put takes a context, url, and optional options, executing the request through the current Session.

type XMLCharDecoder

type XMLCharDecoder func(charset string, input io.Reader) (io.Reader, error)

XMLCharDecoder defines a function type that takes a charset and stream of bytes that's not encoded in UTF-8 through an io.Reader, and returns another io.Reader returns bytes from the input into bytes encoded into UTF-8.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL