requests

package module
v0.21.10 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 4, 2021 License: MIT Imports: 21 Imported by: 228

README

Requests GoDoc Go Report Card Gocover.io Mentioned in Awesome Go

Requests logo

HTTP requests for Gophers.

The problem: Go's net/http is powerful and versatile, but using it correctly for client requests can be extremely verbose.

The solution: The requests.Builder type is a convenient way to build, send, and handle HTTP requests. Builder has a fluent API with methods returning a pointer to the same struct, which allows for declaratively describing a request by method chaining.

Requests also comes with tools for building custom http transports, include a request recorder and replayer for testing.

Examples

Simple GET into a string
// code with net/http
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://example.com", nil)
if err != nil {
	// ...
}
res, err := http.DefaultClient.Do(req)
if err != nil {
	// ...
}
defer res.Body.Close()
b, err := io.ReadAll(res.Body)
if err != nil {
	// ...
}
s := string(b)

// equivalent code using requests
var s string
err := requests.
	URL("http://example.com").
	ToString(&s).
	Fetch(context.Background())

// 5 lines vs. 13 lines
POST a raw body
err := requests.
	URL("https://postman-echo.com/post").
	BodyBytes([]byte(`hello, world`)).
	ContentType("text/plain").
	Fetch(context.Background())

// Equivalent code with net/http
body := bytes.NewReader(([]byte(`hello, world`))
req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://postman-echo.com/post", body)
if err != nil {
	// ...
}
req.Header.Set("Content-Type", "text/plain")
res, err := http.DefaultClient.Do(req)
if err != nil {
	// ...
}
defer res.Body.Close()
_, err := io.ReadAll(res.Body)
if err != nil {
	// ...
}
// 5 lines vs. 14 lines
GET a JSON object
var post placeholder
err := requests.
	URL("https://jsonplaceholder.typicode.com").
	Pathf("/posts/%d", 1).
	ToJSON(&post).
	Fetch(context.Background())

// Equivalent code with net/http
var post placeholder
u, err := url.Parse("https://jsonplaceholder.typicode.com")
if err != nil {
	// ...
}
u.Path = fmt.Sprintf("/posts/%d", 1)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
if err != nil {
	// ...
}
res, err := http.DefaultClient.Do(req)
if err != nil {
	// ...
}
defer res.Body.Close()
b, err := io.ReadAll(res.Body)
if err != nil {
	// ...
}
err := json.Unmarshal(b, &post)
if err != nil {
	// ...
}
// 6 lines vs. 23 lines
POST a JSON object and parse the response
var res placeholder
req := placeholder{
	Title:  "foo",
	Body:   "baz",
	UserID: 1,
}
err := requests.
	URL("/posts").
	Host("jsonplaceholder.typicode.com").
	BodyJSON(&req).
	ToJSON(&res).
	Fetch(context.Background())
// net/http equivalent left as an exercise for the reader
Set custom headers for a request
// Set headers
var headers postman
err := requests.
	URL("https://postman-echo.com/get").
	UserAgent("bond/james-bond").
	ContentType("secret").
	Header("martini", "shaken").
	Fetch(context.Background())
Easily manipulate query parameters
var params postman
err := requests.
	URL("https://postman-echo.com/get?a=1&b=2").
	Param("b", "3").
	Param("c", "4").
	Fetch(context.Background())
	// URL is https://postman-echo.com/get?a=1&b=3&c=4
Record and replay responses
// record a request to the file system
cl.Transport = requests.Record(nil, "somedir")
var s1, s2 string
err := requests.URL("http://example.com").
	Client(&cl).
	ToString(&s1).
	Fetch(context.Background())
check(err)

// now replay the request in tests
cl.Transport = requests.Replay("somedir")
err = requests.URL("http://example.com").
	Client(&cl).
	ToString(&s2).
	Fetch(context.Background())
check(err)
assert(s1 == s2) // true

Documentation

Overview

Package requests is a convenience wrapper around net/http to make it faster and easier to build requests and custom transports.

Example 
package main

import (
	"context"
	"fmt"
	"strings"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Simple GET into a string
	var s string
	err := requests.
		URL("http://example.com").
		ToString(&s).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to example.com:", err)
	}
	fmt.Println(strings.Contains(s, "Example Domain"))
}

Output:
true
Example (GetJSON) 
package main

import (
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
)

type placeholder struct {
	ID     int    `json:"id,omitempty"`
	Title  string `json:"title"`
	Body   string `json:"body"`
	UserID int    `json:"userId"`
}

func main() {
	// GET a JSON object
	id := 1
	var post placeholder
	err := requests.
		URL("https://jsonplaceholder.typicode.com").
		Pathf("/posts/%d", id).
		ToJSON(&post).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to jsonplaceholder.typicode.com:", err)
	}
	fmt.Println(post.Title)
}

Output:
sunt aut facere repellat provident occaecati excepturi optio reprehenderit
Example (PostJSON) 
package main

import (
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
)

type placeholder struct {
	ID     int    `json:"id,omitempty"`
	Title  string `json:"title"`
	Body   string `json:"body"`
	UserID int    `json:"userId"`
}

func main() {
	// POST a JSON object and parse the response
	var res placeholder
	req := placeholder{
		Title:  "foo",
		Body:   "baz",
		UserID: 1,
	}
	err := requests.
		URL("/posts").
		Host("jsonplaceholder.typicode.com").
		BodyJSON(&req).
		ToJSON(&res).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to jsonplaceholder.typicode.com:", err)
	}
	fmt.Println(res)
}

Output:
{101 foo baz 1}
Example (QueryParam) 
package main

import (
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
)

// Examples with the Postman echo server
type postman struct {
	Args    map[string]string `json:"args"`
	Data    string            `json:"data"`
	Headers map[string]string `json:"headers"`
	JSON    map[string]string `json:"json"`
}

func main() {
	// Set a query parameter
	var params postman
	err := requests.
		URL("https://postman-echo.com/get?a=1&b=2").
		Param("b", "3").
		Param("c", "4").
		ToJSON(&params).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("problem with postman:", err)
	}
	fmt.Println(params.Args)
}

Output:
map[a:1 b:3 c:4]

Index

Examples

Constants

This section is empty.

Variables

View Source 
var NoFollow = MaxFollow(0)

NoFollow is a CheckRedirectPolicy that does not follow redirects.

Functions

func HasStatusErr 

func HasStatusErr(err error, codes ...int) bool

HasStatusErr returns true if err is a StatusError caused by any of the codes given.

Example 
package main

import (
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
)

func main() {
	err := requests.
		URL("http://example.com/404").
		CheckStatus(200).
		Fetch(context.Background())
	if requests.HasStatusErr(err, 404) {
		fmt.Println("got a 404")
	}
}

Output:
got a 404

func NewCookieJar added in v0.21.10 

func NewCookieJar() http.CookieJar

NewCookieJar returns a cookie jar using the standard public suffix list.

Example 
package main

import (
	"context"
	"fmt"
	"net/http"
	"net/url"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Create a client that preserve cookies between requests
	myClient := *http.DefaultClient
	myClient.Jar = requests.NewCookieJar()
	// Use the client to make a request
	err := requests.
		URL("http://httpbin.org/cookies/set/chocolate/chip").
		Client(&myClient).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to httpbin.org:", err)
	}
	// Now check that cookies we got
	for _, cookie := range myClient.Jar.Cookies(&url.URL{
		Scheme: "http",
		Host:   "httpbin.org",
	}) {
		fmt.Println(cookie)
	}
	// And we'll see that they're reused on subsequent requests
	var cookies struct {
		Cookies map[string]string
	}
	err = requests.
		URL("http://httpbin.org/cookies").
		Client(&myClient).
		ToJSON(&cookies).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to httpbin.org:", err)
	}
	fmt.Println(cookies)
}

Output:
chocolate=chip
{map[chocolate:chip]}

func Record added in v0.21.3 

func Record(rt http.RoundTripper, basepath string) http.RoundTripper

Record returns an http.RoundTripper that writes out its requests and their responses to text files in basepath. Requests are named according to a hash of their contents. Responses are named according to the request that made them.

func Replay added in v0.21.3 

func Replay(basepath string) http.RoundTripper

Replay returns an http.RoundTripper that reads its responses from text files in basepath. Responses are looked up according to a hash of the request.

func ReplayFS added in v0.21.5 

func ReplayFS(fsys fs.FS) http.RoundTripper

ReplayFS returns an http.RoundTripper that reads its responses from text files in the fs.FS. Responses are looked up according to a hash of the request. Response file names may optionally be prefixed with comments for better human organization.

Example 
package main

import (
	"context"
	"fmt"
	"net/http"
	"testing/fstest"

	"github.com/carlmjohnson/requests"
)

func main() {
	fsys := fstest.MapFS{
		"fsys.example - MKIYDwjs.res.txt": &fstest.MapFile{
			Data: []byte(`HTTP/1.1 200 OK
Content-Type: text/plain; charset=UTF-8
Date: Mon, 24 May 2021 18:48:50 GMT

An example response.`),
		},
	}
	var s string
	const expected = `An example response.`
	if err := requests.
		URL("http://fsys.example").
		Client(&http.Client{
			Transport: requests.ReplayFS(fsys),
		}).
		ToString(&s).
		Fetch(context.Background()); err != nil {
		panic(err)
	}
	fmt.Println(s == expected)
}

Output:
true

func ReplayString added in v0.21.8 

func ReplayString(rawResponse string) http.RoundTripper

ReplayString returns an http.RoundTripper that always responds with a request built from rawResponse. It is intended for use in one-off tests.

Example 
package main

import (
	"context"
	"fmt"
	"net/http"

	"github.com/carlmjohnson/requests"
)

func main() {
	const res = `HTTP/1.1 200 OK

An example response.`

	var s string
	const expected = `An example response.`
	if err := requests.
		URL("http://response.example").
		Client(&http.Client{
			Transport: requests.ReplayString(res),
		}).
		ToString(&s).
		Fetch(context.Background()); err != nil {
		panic(err)
	}
	fmt.Println(s == expected)
}

Output:
true

func UserAgentTransport added in v0.21.10 

func UserAgentTransport(rt http.RoundTripper, s string) http.RoundTripper

UserAgentTransport returns a wrapped http.RoundTripper that sets the User-Agent header on requests to s.

Types

type BodyGetter 

type BodyGetter = func() (io.ReadCloser, error)

BodyGetter provides a Builder with a source for a request body.

func BodyBytes 

func BodyBytes(b []byte) BodyGetter

BodyBytes is a BodyGetter that returns the provided raw bytes.

func BodyForm 

func BodyForm(data url.Values) BodyGetter

BodyForm is a BodyGetter that builds an encoded form body.

func BodyJSON 

func BodyJSON(v interface{}) BodyGetter

BodyJSON is a BodyGetter that marshals a JSON object.

func BodyReader 

func BodyReader(r io.Reader) BodyGetter

BodyReader is a BodyGetter that returns an io.Reader.

type Builder 

type Builder struct {
	// contains filtered or unexported fields
}

Builder is a convenient way to build, send, and handle HTTP requests. Builder has a fluent API with methods returning a pointer to the same struct, which allows for declaratively describing a request by method chaining.

Builder can be thought of as having the following phases:

Set the base URL for a request with requests.URL then customize it with Scheme, Host, Hostf, Path, Pathf, and Param.

Set the method for a request with Method or use the Get, Post, and Put methods. By default, requests without a body are GET and those with a body are POST.

Set headers with Header or set conventional header keys with Accept, CacheControl, ContentType, UserAgent, BasicAuth, and Bearer.

Set the http.Client to use for a request with Client.

Set the body of the request if any with GetBody or use built in BodyBytes, BodyJSON, BodyForm, or BodyReader.

Add a response validator to the Builder with AddValidator or use the built in CheckStatus, CheckContentType, and Peek.

Set a handler for a response with Handle or use ToJSON, ToString, ToBytesBuffer, ToBufioReader, ToHTML, or ToWriter.

Fetch creates an http.Request with Request and sends it via the underlying http.Client with Do.

In many cases, it will be possible to set most options for an API endpoint in a Builder at the package or struct level and then call Clone in a function to add request specific details for the URL, parameters, headers, body, or handler. The zero value of Builder is usable.

func URL 

func URL(baseurl string) *Builder

URL creates a new Builder suitable for method chaining.

func (*Builder) Accept added in v0.21.4 

func (rb *Builder) Accept(contentTypes string) *Builder

Accept sets the Accept header for a request.

func (*Builder) AddValidator 

func (rb *Builder) AddValidator(h ResponseHandler) *Builder

AddValidator adds a response validator to the Builder. Adding a validator disables DefaultValidator. To disable all validation, just add nil.

func (*Builder) BasicAuth 

func (rb *Builder) BasicAuth(username, password string) *Builder

BasicAuth sets the Authorization header to a basic auth credential.

func (*Builder) Bearer added in v0.21.4 

func (rb *Builder) Bearer(token string) *Builder

Bearer sets the Authorization header to a bearer token.

Example 
package main

import (
	"context"
	"fmt"
	"net/http"

	"github.com/carlmjohnson/requests"
)

func main() {
	// We get a 401 response if no bearer token is provided
	err := requests.
		URL("http://httpbin.org/bearer").
		CheckStatus(http.StatusUnauthorized).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("problem with httpbin:", err)
	}
	// But our response is accepted when we provide a bearer token
	var resp struct {
		Authenticated bool
		Token         string
	}
	err = requests.
		URL("http://httpbin.org/bearer").
		Bearer("whatever").
		ToJSON(&resp).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("problem with httpbin:", err)
	}
	fmt.Println(resp.Authenticated)
	fmt.Println(resp.Token)
}

Output:
true
whatever

func (*Builder) BodyBytes 

func (rb *Builder) BodyBytes(b []byte) *Builder

BodyBytes sets the Builder's request body to b.

Example 
package main

import (
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
)

// Examples with the Postman echo server
type postman struct {
	Args    map[string]string `json:"args"`
	Data    string            `json:"data"`
	Headers map[string]string `json:"headers"`
	JSON    map[string]string `json:"json"`
}

func main() {
	// Post a raw body
	var data postman
	err := requests.
		URL("https://postman-echo.com/post").
		BodyBytes([]byte(`hello, world`)).
		ContentType("text/plain").
		ToJSON(&data).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("problem with postman:", err)
	}
	fmt.Println(data.Data)
}

Output:
hello, world

func (*Builder) BodyForm 

func (rb *Builder) BodyForm(data url.Values) *Builder

BodyForm sets the Builder's request body to the encoded form. It also sets the ContentType to "application/x-www-form-urlencoded".

Example 
package main

import (
	"context"
	"fmt"
	"net/url"

	"github.com/carlmjohnson/requests"
)

// Examples with the Postman echo server
type postman struct {
	Args    map[string]string `json:"args"`
	Data    string            `json:"data"`
	Headers map[string]string `json:"headers"`
	JSON    map[string]string `json:"json"`
}

func main() {
	// Submit form values
	var echo postman
	err := requests.
		URL("https://postman-echo.com/put").
		Put().
		BodyForm(url.Values{
			"hello": []string{"world"},
		}).
		ToJSON(&echo).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("problem with postman:", err)
	}
	fmt.Println(echo.JSON)
}

Output:
map[hello:world]

func (*Builder) BodyJSON 

func (rb *Builder) BodyJSON(v interface{}) *Builder

BodyJSON sets the Builder's request body to the marshaled JSON. It also sets ContentType to "application/json".

func (*Builder) BodyReader 

func (rb *Builder) BodyReader(r io.Reader) *Builder

BodyReader sets the Builder's request body to r.

Example 
package main

import (
	"context"
	"fmt"
	"log"
	"os"
	"path/filepath"

	"github.com/carlmjohnson/requests"
)

// Examples with the Postman echo server
type postman struct {
	Args    map[string]string `json:"args"`
	Data    string            `json:"data"`
	Headers map[string]string `json:"headers"`
	JSON    map[string]string `json:"json"`
}

func main() {
	// temp file creation boilerplate
	dir, err := os.MkdirTemp("", "example")
	if err != nil {
		log.Fatal(err)
	}
	defer os.RemoveAll(dir) // clean up

	file := filepath.Join(dir, "tmpfile")
	if err := os.WriteFile(file, []byte(`hello, world`), 0666); err != nil {
		log.Fatal(err)
	}

	f, err := os.Open(file)
	if err != nil {
		log.Fatal(err)
	}
	defer f.Close()

	// send the raw file to server
	var echo postman
	err = requests.
		URL("https://postman-echo.com/post").
		ContentType("text/plain").
		BodyReader(f).
		ToJSON(&echo).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("problem with postman:", err)
	}
	fmt.Println(echo.Data)
}

Output:
hello, world

func (*Builder) CacheControl added in v0.21.4 

func (rb *Builder) CacheControl(directive string) *Builder

CacheControl sets the client-side Cache-Control directive for a request.

func (*Builder) CheckContentType added in v0.21.2 

func (rb *Builder) CheckContentType(cts ...string) *Builder

CheckContentType adds a validator for the content type header of a response.

func (*Builder) CheckStatus 

func (rb *Builder) CheckStatus(acceptStatuses ...int) *Builder

CheckStatus adds a validator for status code of a response.

Example 
package main

import (
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Expect a specific status code
	err := requests.
		URL("https://jsonplaceholder.typicode.com").
		Pathf("/posts/%d", 9001).
		CheckStatus(404).
		CheckContentType("application/json").
		Fetch(context.Background())
	if err != nil {
		fmt.Println("should be a 404:", err)
	} else {
		fmt.Println("OK")
	}
}

Output:
OK

func (*Builder) Client 

func (rb *Builder) Client(cl *http.Client) *Builder

Client sets the http.Client to use for requests. If nil, it uses http.DefaultClient.

func (*Builder) Clone 

func (rb *Builder) Clone() *Builder

Clone creates a new Builder suitable for independent mutation.

func (*Builder) ContentType 

func (rb *Builder) ContentType(ct string) *Builder

ContentType sets the Content-Type header on a request.

func (*Builder) Do 

func (rb *Builder) Do(req *http.Request) (err error)

Do calls the underlying http.Client and validates and handles any resulting response. The response body is closed after all validators and the handler run.

func (*Builder) Fetch 

func (rb *Builder) Fetch(ctx context.Context) (err error)

Fetch builds a request, sends it, and handles the response.

func (*Builder) Get 

func (rb *Builder) Get() *Builder

Get sets HTTP method to GET.

func (*Builder) GetBody 

func (rb *Builder) GetBody(src BodyGetter) *Builder

GetBody sets the BodySource for a request. It implicitly sets method to POST.

func (*Builder) Handle 

func (rb *Builder) Handle(h ResponseHandler) *Builder

Handle sets the response handler for a Builder. To use multiple handlers, use ChainHandlers.

func (*Builder) Head added in v0.21.7 

func (rb *Builder) Head() *Builder

Head sets HTTP method to HEAD.

func (*Builder) Header 

func (rb *Builder) Header(key, value string) *Builder

Header sets a header on a request. It overwrites the value of existing keys.

Example 
package main

import (
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
)

// Examples with the Postman echo server
type postman struct {
	Args    map[string]string `json:"args"`
	Data    string            `json:"data"`
	Headers map[string]string `json:"headers"`
	JSON    map[string]string `json:"json"`
}

func main() {
	// Set headers
	var headers postman
	err := requests.
		URL("https://postman-echo.com/get").
		UserAgent("bond/james-bond").
		BasicAuth("bondj", "007!").
		ContentType("secret").
		Header("martini", "shaken").
		ToJSON(&headers).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("problem with postman:", err)
	}
	fmt.Println(headers.Headers["user-agent"])
	fmt.Println(headers.Headers["authorization"])
	fmt.Println(headers.Headers["content-type"])
	fmt.Println(headers.Headers["martini"])
}

Output:
bond/james-bond
Basic Ym9uZGo6MDA3IQ==
secret
shaken

func (*Builder) Host 

func (rb *Builder) Host(host string) *Builder

Host sets the host for a request. It overrides the URL function.

func (*Builder) Hostf added in v0.21.2 

func (rb *Builder) Hostf(format string, a ...interface{}) *Builder

Hostf calls Host with fmt.Sprintf.

func (*Builder) Method 

func (rb *Builder) Method(method string) *Builder

Method sets the HTTP method for a request.

func (*Builder) Param 

func (rb *Builder) Param(key string, values ...string) *Builder

Param sets a query parameter on a request. It overwrites the existing values of a key.

func (*Builder) Path 

func (rb *Builder) Path(path string) *Builder

Path joins a path to a request. If the path begins with /, it overrides any existing path. If the path begins with ./ or ../, the final path will be rewritten in its absolute form.

Example 
package main

import (
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
)

type placeholder struct {
	ID     int    `json:"id,omitempty"`
	Title  string `json:"title"`
	Body   string `json:"body"`
	UserID int    `json:"userId"`
}

func main() {
	// Add an ID to a base path
	id := 1
	var post placeholder
	err := requests.
		URL("https://jsonplaceholder.typicode.com/posts").
		// inherits path /posts from baseurl
		Pathf("%d", id).
		// URL is now https://jsonplaceholder.typicode.com/posts/1
		ToJSON(&post).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to jsonplaceholder.typicode.com:", err)
	}
	fmt.Println(post.ID)
}

Output:
1

func (*Builder) Pathf added in v0.21.2 

func (rb *Builder) Pathf(format string, a ...interface{}) *Builder

Pathf calls Path with fmt.Sprintf.

func (*Builder) Peek added in v0.21.5 

func (rb *Builder) Peek(n int, f func([]byte) error) *Builder

Peek adds a validator that peeks at the first n bytes of a response body.

Example 
package main

import (
	"context"
	"fmt"
	"strings"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Check that a response has a doctype
	const doctype = "<!doctype html>"
	var s string
	err := requests.
		URL("http://example.com").
		Peek(len(doctype), func(b []byte) error {
			if string(b) != doctype {
				return fmt.Errorf("missing doctype: %q", b)
			}
			return nil
		}).
		ToString(&s).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to example.com:", err)
	}
	fmt.Println(
		// Final result still has the prefix
		strings.HasPrefix(s, doctype),
		// And the full body
		strings.HasSuffix(s, "</html>\n"),
	)
}

Output:
true true

func (*Builder) Post 

func (rb *Builder) Post() *Builder

Post sets HTTP method to POST.

func (*Builder) Put 

func (rb *Builder) Put() *Builder

Put sets HTTP method to PUT.

func (*Builder) Request 

func (rb *Builder) Request(ctx context.Context) (req *http.Request, err error)

Request builds a new http.Request with its context set.

func (*Builder) Scheme added in v0.21.7 

func (rb *Builder) Scheme(scheme string) *Builder

Scheme sets the scheme for a request. It overrides the URL function.

func (*Builder) ToBufioReader 

func (rb *Builder) ToBufioReader(f func(r *bufio.Reader) error) *Builder

ToBufioReader sets the Builder to call a callback with the response body wrapped in a bufio.Reader.

Example 
package main

import (
	"bufio"
	"context"
	"fmt"
	"io"
	"strings"

	"github.com/carlmjohnson/requests"
)

func main() {
	// read a response line by line for a sentinel
	found := false
	err := requests.
		URL("http://example.com").
		ToBufioReader(func(r *bufio.Reader) error {
			var err error
			for s := ""; err == nil; {
				if strings.Contains(s, "Example Domain") {
					found = true
					return nil
				}
				// read one line from response
				s, err = r.ReadString('\n')
			}
			if err == io.EOF {
				return nil
			}
			return err
		}).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to example.com:", err)
	}
	fmt.Println(found)
}

Output:
true

func (*Builder) ToBufioScanner added in v0.21.9 

func (rb *Builder) ToBufioScanner(f func(r *bufio.Scanner) error) *Builder

ToBufioScanner sets the Builder to call a callback with the response body wrapped in a bufio.Scanner.

Example 
package main

import (
	"bufio"
	"bytes"
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
)

func main() {
	// read a response line by line for a sentinel
	found := false
	needle := []byte("Example Domain")
	err := requests.
		URL("http://example.com").
		ToBufioScanner(func(s *bufio.Scanner) error {
			// read one line at time from response
			for s.Scan() {
				if bytes.Contains(s.Bytes(), needle) {
					found = true
					return nil
				}
			}
			return s.Err()
		}).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to example.com:", err)
	}
	fmt.Println(found)
}

Output:
true

func (*Builder) ToBytesBuffer 

func (rb *Builder) ToBytesBuffer(buf *bytes.Buffer) *Builder

ToBytesBuffer sets the Builder to write the response body to the provided bytes.Buffer.

Example 
package main

import (
	"bytes"
	"context"
	"fmt"
	"strings"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Simple GET into a buffer
	var buf bytes.Buffer
	err := requests.
		URL("http://example.com").
		ToBytesBuffer(&buf).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to example.com:", err)
	}
	fmt.Println(strings.Contains(buf.String(), "Example Domain"))
}

Output:
true

func (*Builder) ToHTML added in v0.21.7 

func (rb *Builder) ToHTML(n *html.Node) *Builder

ToHTML sets the Builder to parse the response as HTML.

Example 
package main

import (
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
	"golang.org/x/net/html"
	"golang.org/x/net/html/atom"
)

func main() {
	var doc html.Node
	err := requests.
		URL("http://example.com").
		ToHTML(&doc).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to example.com:", err)
	}
	var f func(*html.Node)
	f = func(n *html.Node) {
		if n.DataAtom == atom.A {
			for _, attr := range n.Attr {
				if attr.Key == "href" {
					fmt.Println("link:", attr.Val)
				}
			}
		}
		for c := n.FirstChild; c != nil; c = c.NextSibling {
			f(c)
		}
	}
	f(&doc)
}

Output:
link: https://www.iana.org/domains/example

func (*Builder) ToJSON 

func (rb *Builder) ToJSON(v interface{}) *Builder

ToJSON sets the Builder to decode a response as a JSON object

func (*Builder) ToString 

func (rb *Builder) ToString(sp *string) *Builder

ToString sets the Builder to write the response body to the provided string pointer.

func (*Builder) ToWriter added in v0.21.7 

func (rb *Builder) ToWriter(w io.Writer) *Builder

ToWriter sets the Builder to copy the response body into w.

Example 
package main

import (
	"context"
	"fmt"
	"log"
	"os"

	"github.com/carlmjohnson/requests"
)

func main() {
	f, err := os.CreateTemp("", "*.example.html")
	if err != nil {
		log.Fatal(err)
	}
	defer os.Remove(f.Name()) // clean up

	err = requests.
		URL("http://example.com").
		ToWriter(f).
		Fetch(context.Background())
	if err != nil {
		log.Fatal(err)
	}
	if err = f.Close(); err != nil {
		log.Fatal(err)
	}
	stat, err := os.Stat(f.Name())
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("file is %d bytes\n", stat.Size())

}

Output:
file is 1256 bytes

func (*Builder) UserAgent 

func (rb *Builder) UserAgent(s string) *Builder

UserAgent sets the User-Agent header.

type CheckRedirectPolicy added in v0.21.10 

type CheckRedirectPolicy = func(req *http.Request, via []*http.Request) error

CheckRedirectPolicy is a function suitable for use as CheckRedirect on an http.Client.

func MaxFollow added in v0.21.10 

func MaxFollow(n int) CheckRedirectPolicy

MaxFollow returns a CheckRedirectPolicy that follows a maximum of n redirects.

Example 
package main

import (
	"context"
	"fmt"
	"net/http"

	"github.com/carlmjohnson/requests"
)

func main() {
	cl := *http.DefaultClient
	cl.CheckRedirect = requests.MaxFollow(1)

	var h http.Header
	if err := requests.
		URL("https://httpbingo.org/redirect/2").
		Client(&cl).
		CheckStatus(http.StatusFound).
		Handle(func(res *http.Response) error {
			h = res.Header
			return nil
		}).
		Fetch(context.Background()); err != nil {
		panic(err)
	}
	fmt.Println(h.Get("Location"))
}

Output:
/get

type ResponseHandler 

type ResponseHandler = func(*http.Response) error

ResponseHandler is used to validate or handle the response to a request. 

var DefaultValidator ResponseHandler = CheckStatus(
	http.StatusOK,
	http.StatusCreated,
	http.StatusAccepted,
	http.StatusNonAuthoritativeInfo,
	http.StatusNoContent,
)

DefaultValidator is the validator applied by Builder unless otherwise specified.

func ChainHandlers 

func ChainHandlers(handlers ...ResponseHandler) ResponseHandler

ChainHandlers allows for the composing of validators or response handlers.

func CheckContentType added in v0.21.2 

func CheckContentType(cts ...string) ResponseHandler

CheckContentType validates that a response has one of the given content type headers.

func CheckStatus 

func CheckStatus(acceptStatuses ...int) ResponseHandler

CheckStatus validates the response has an acceptable status code.

func Peek added in v0.21.5 

func Peek(n int, f func([]byte) error) ResponseHandler

Peek wraps the body of a response in a bufio.Reader and gives f a peek at the first n bytes for validation.

func ToBufioReader 

func ToBufioReader(f func(r *bufio.Reader) error) ResponseHandler

ToBufioReader takes a callback which wraps the response body in a bufio.Reader.

func ToBufioScanner added in v0.21.9 

func ToBufioScanner(f func(r *bufio.Scanner) error) ResponseHandler

ToBufioScanner takes a callback which wraps the response body in a bufio.Scanner.

func ToBytesBuffer 

func ToBytesBuffer(buf *bytes.Buffer) ResponseHandler

ToBytesBuffer writes the response body to the provided bytes.Buffer.

func ToHTML added in v0.21.7 

func ToHTML(n *html.Node) ResponseHandler

ToHTML parses the page with x/net/html.Parse.

func ToJSON 

func ToJSON(v interface{}) ResponseHandler

ToJSON decodes a response as a JSON object.

func ToString 

func ToString(sp *string) ResponseHandler

ToString writes the response body to the provided string pointer.

func ToWriter added in v0.21.7 

func ToWriter(w io.Writer) ResponseHandler

ToWriter copies the response body to w.

type RoundTripFunc added in v0.21.3 

type RoundTripFunc func(req *http.Request) (res *http.Response, err error)

RoundTripFunc is an adaptor to use a function as an http.RoundTripper.

func (RoundTripFunc) RoundTrip added in v0.21.3 

func (rtf RoundTripFunc) RoundTrip(r *http.Request) (*http.Response, error)

RoundTrip implements http.RoundTripper.

type StatusError 

type StatusError http.Response

StatusError is the error type produced by CheckStatus.

func (*StatusError) Error 

func (se *StatusError) Error() string

Error fulfills the error interface.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.