mockhttp

package module
v0.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 14, 2024 License: MIT Imports: 19 Imported by: 0

README

go-mockhttp

build and test

TODO:

  • Make a better representation (image) , can be using tldraw alone => or if you find ways to illustrate via gif, even better => https://github.com/dai-shi/excalidraw-animate
  • Make badge for the CI result => DONE
  • Make Mock Definition specification
  • Extract example into full fledge example golang project => let's try find some good example
  • Future works => refer casbin, essentially, want more ways to provide mocks adapter (database, redis, etc...). Also, exploring ways to create rule even more flexible. Mock server for even more language agnostic approach for stubbing upstream service. => DONE

A gif / representation

Overview
Problem

  • You have http client in your Go code, and want to start testing various case on it.

  • For unit test, you can mock those response with httptest

  • What if your QA guys asked you various edge case for QA testing / for integration test that tied with external service. It might be easy if we had full control of the upstream service, but what if don't have it ? Sometimes, we might need to prepare various mocks for testing edge cases to ensure everything works as expected.

    So, what to do? Should we... :

    • Hardcoded many if statement in the codebase to prepare those edge case ?
    • Build a sandbox service to represent 3rd party / external upstream service ?
    • Give up and don't test those case at all ?

While all 3 options is definitely possible (except the last one 😠), introducing go-mockhttp...

go-mockhttp is a testing layer for Go standard http client library. It allows stubbed responses to be configured for matched HTTP requests that had been defined and can be used to test your application's service layer in unit test or even in actual test server for many various case that depends on 3rd party responses.

It use Mock Definition, a term that we use to define a specification of:

  • How to determine (match) whether a request should be mock / not
  • How to determine (match) which mock response should be used, based on request entity (using flexible rules)
  • Other misc thing that might be useful for testing

Equipped with these capabilities, now the *http.Client that you use can be extended to also supports integration / automation / manual testing easily.

WARNING! While you can definitely use it on production, it is suggested to only use this for helping testing purposes.

How it works?

As Go http client library (from Go net/http client library) use a *http.Client with definition that can be referred here.

Based of the documentation, the Client had 6 main method:

  • (c) CloseIdleConnections()
  • (c) Do(req)
  • (c) Get(url)
  • (c) Head(url)
  • (c) Post(url, contentType, body)
  • (c) PostForm(url, data)

Using this as the base reference, we could easily extend the standard Go http.Client struct into any custom struct that we want. To actually stub the 3rd party dependencies (via HTTP call), we could modify these method:

  • (c) Do(req)
  • (c) Get(url)
  • (c) Head(url)
  • (c) Post(url, contentType, body)
  • (c) PostForm(url, data)

that relates heavily on exchanging actual data to upstream service. Specifically, we apply this approach:

=> Check if req match with loaded (in runtime) Mock Definition
   => Yes? Use response defined in Mock Definition
   => No?  Continue the requests to upstream service
Get Started

Version 0.1.0 and before are requiring Go version 1.13+.

Installation
go get github.com/William9923/go-mockhttp
Examples

Using this library should look almost identical to what you would do with net/http. The most simple example of a GET request is shown below:

...

  resolver, err := mockhttp.NewFileResolverAdapter(policyDirPath)
  if err != nil {
    panic(err)
  }

  err = resolver.LoadPolicy(context.Background())
  if err != nil {
    panic(err)
  }

  mockClient := mockhttp.NewClient(resolver)
  resp, err := .Get("/foo")
  if err != nil {
    panic(err)
  }

...

The returned response object is an *http.Response, the same thing you would usually get from net/http. Had the request match with the Mock Definition loaded into *mockhttp.Client, the above call would instead be stubbed with response defined in Mock Definition.

Roadmap

What will the library try to improve in the future?

  • Provide more example for easier adoption of the library in any existing projects.
  • Additional adapter supports (inspired by casbin), to allow more ways to load Mock Definition from different storage.
  • Extending ways to use Mock Definition in other language (not only Go), as Mock Definition can be used cross-language.
  • Build mockhttp as a service instead of a library, to accomodate for non-Go service that would like to utilize it (Mock as a Service).
FAQ
How to use with stdlib *http.Client ?

Similar to go-retryablehttp, It's possible to fully convert *mockhttp.Client directly into a http.Client. This makes adoption mockhttp is applicable in many situation with minimal effort. Simply configure a *retryablehttp.Client as you wish, and then call StandardClient():

...
  mockClient := mockhttp.NewClient(resolver) // assuming resolver for mock definition had been defined...
  resp, err := mockClient.Get("/foo")
  if err != nil {
    panic(err)
  }
...
What is Mock Definition ?

A term to describe a specification (as a yaml file) that includes:

  • Host, endpoint path and HTTP Method of upstream service that we want to mock.

  • Supported http requests format is JSON, XML, Form for POST, PUT, PATCH, DELETE requests.

  • Description field that is used to describe what's the mock definition is.

  • Multiple (array) responses that can be used as the mock responses that match the host, endpoint path and HTTP method defined in the spec.

  • Each responses can includes:

    • response_headers: map of <string, string>
    • response_body : support all serializeable format (as string)
    • status_code: int
    • enable_template: allow templating for response_body, using request body information
    • delay: integer. Use milliseconds, to delay the responses before returning the response. Useful for testing timeout requests (context deadline).
    • rules: array of CEL expression that use request body information to evaluate the expression.

Example:

host: marketplace.com
path: /check-price
method: POST
desc: Testing Marketplace Price Endpoint
responses:
  - response_headers:
      Content-Type: application/json
    response_body: "{\"user_name\": \"Mocker\",\r\n \"price\": 1000}"
    status_code: 200
  - response_headers:
      Content-Type: application/json
    response_body: "{\"user_name\": \"William\",\r\n \"price\": 2000}"
    delay: 1000
    status_code: 488
    enable_template: false
    rules:
      - body.name == "William"

Match Behavior:

There are 3 ways on how the library will try to match the endpoint path:

  1. Exact Match: /v1/api/mock/1
  2. With Path Param: /v1/api/mock/:id
  3. Wildcard: /v1/api/*

What happen when the request have no matching Mock Definition?

There are 2 conditions that might happen:

  1. Request don't match host, path, and method => http client will immediately call actual upstream service
  2. Request match host, path, and method, but didn't satisfy the rules in responses => will try to use default response (response with no rules). If no default response defined, will simply call actual upstream service.
License

This project is under the MIT license.

Documentation

Overview

standard net/http client library and exposes nearly the same public API. This makes mocking http response very easy to drop into existing programs.

mockhttp performs mock if enabled. Inspired by retryablehttp by hashicorp.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrPolicyLoaded           = fmt.Errorf("policy loaded")
	ErrClientMissing          = fmt.Errorf("client missing")
	ErrNoMockResponse         = fmt.Errorf("no mock response prepared")
	ErrUnsupportedContentType = fmt.Errorf("unsupported content type")
	ErrCommon                 = fmt.Errorf("common error")
	ErrNoContentType          = fmt.Errorf("unable to find content type")
)

Functions

func ReusableReader 

func ReusableReader(r io.Reader) io.Reader

Types

type Client 

type Client struct {
	HTTPClient *http.Client // Internal HTTP client.
	Logger     interface{}  // Customer logger instance. Can be either Logger or LeveledLogger

	// RequestLogHook allows a user-supplied function to be called
	// before each httprequest  call.
	RequestLogHook RequestLogHook

	// ResponseLogHook allows a user-supplied function to be called
	// with the response from each HTTP request executed.
	ResponseLogHook ResponseLogHook

	// MockStore represents the datastore.
	// The built-in library provides file-based datastore, but it can be easily extended.
	Resolver ResolverAdapter
	// contains filtered or unexported fields
}

Client is used to make HTTP requests. It adds additional functionality like automatic retries to tolerate minor outages.

func NewClient 

func NewClient(resolver ResolverAdapter) *Client

NewClient creates a new Client with default settings.

func (*Client) Do 

func (c *Client) Do(req *Request) (*http.Response, error)

Do wraps calling an HTTP method with mock possibility. WARN: DON'T USE IT ON PRODUCTION!

func (*Client) Get 

func (c *Client) Get(url string) (*http.Response, error)

Get is a convenience helper for doing simple GET requests.

func (*Client) Head 

func (c *Client) Head(url string) (*http.Response, error)

Head is a convenience method for doing simple HEAD requests.

func (*Client) Post 

func (c *Client) Post(url, contentType string, body interface{}) (*http.Response, error)

Post is a convenience method for doing simple POST requests.

func (*Client) PostForm 

func (c *Client) PostForm(url string, data url.Values) (*http.Response, error)

PostForm is a convenience method for doing simple POST operations using pre-filled url.Values form data.

func (*Client) StandardClient 

func (c *Client) StandardClient() *http.Client

StandardClient returns a stdlib *http.Client with a custom Transport, which shims in a *mockhttp.Client for added retries.

type LenReader 

type LenReader interface {
	Len() int
}

LenReader is an interface implemented by many in-memory io.Reader's. Used for automatically sending the right Content-Length header when possible.

type LeveledLogger 

type LeveledLogger interface {
	Error(msg string, keysAndValues ...interface{})
	Info(msg string, keysAndValues ...interface{})
	Debug(msg string, keysAndValues ...interface{})
	Warn(msg string, keysAndValues ...interface{})
}

LeveledLogger is an interface that can be implemented by any logger or a logger wrapper to provide leveled logging. The methods accept a message string and a variadic number of key-value pairs. For log.Printf style formatting where message string contains a format specifier, use Logger interface.

type Logger 

type Logger interface {
	Printf(string, ...interface{})
}

Logger interface allows to use other loggers than standard log.Logger.

type ReaderFunc 

type ReaderFunc func() (io.Reader, error)

ReaderFunc is the type of function that can be given natively to NewRequest

type Request 

type Request struct {

	// Embed an HTTP request directly. This makes a *Request act exactly
	// like an *http.Request so that all meta methods are supported.
	*http.Request
	// contains filtered or unexported fields
}

Request wraps the metadata needed to create HTTP requests.

func FromRequest 

func FromRequest(r *http.Request) (*Request, error)

FromRequest wraps an http.Request in a retryablehttp.Request

func NewRequest 

func NewRequest(method, url string, rawBody interface{}) (*Request, error)

NewRequest creates a new wrapped request.

func NewRequestWithContext 

func NewRequestWithContext(ctx context.Context, method, url string, rawBody interface{}) (*Request, error)

NewRequestWithContext creates a new wrapped request with the provided context.

The context controls the entire lifetime of a request and its response: obtaining a connection, sending the request, and reading the response headers and body.

func (*Request) BodyBytes 

func (r *Request) BodyBytes() ([]byte, error)

BodyBytes allows accessing the request body. It is an analogue to http.Request's Body variable, but it returns a copy of the underlying data rather than consuming it.

This function is not thread-safe; do not call it at the same time as another call, or at the same time this request is being used with Client.Do.

func (*Request) SetBody 

func (r *Request) SetBody(rawBody interface{}) error

SetBody allows setting the request body.

It is useful if a new body needs to be set without constructing a new Request.

func (*Request) SetResponseHandler 

func (r *Request) SetResponseHandler(fn ResponseHandlerFunc)

SetResponseHandler allows setting the response handler.

func (*Request) WithContext 

func (r *Request) WithContext(ctx context.Context) *Request

WithContext returns wrapped Request with a shallow copy of underlying *http.Request with its context changed to ctx. The provided ctx must be non-nil.

func (*Request) WriteTo 

func (r *Request) WriteTo(w io.Writer) (int64, error)

WriteTo allows copying the request body into a writer.

It writes data to w until there's no more data to write or when an error occurs. The return int64 value is the number of bytes written. Any error encountered during the write is also returned. The signature matches io.WriterTo interface.

type RequestLogHook 

type RequestLogHook func(Logger, *http.Request)

RequestLogHook allows a function to run before http call executed. The HTTP request which will be made.

type ResolverAdapter 

type ResolverAdapter interface {
	LoadPolicy(ctx context.Context) error
	Resolve(ctx context.Context, req *Request) (*http.Response, error)
}

Main Resolver Contract 1. LoadPolicy : load policy spec from different datastore (file, database, etc...) 2. Resolve : check request and return mock response if exist

func NewFileResolverAdapter 

func NewFileResolverAdapter(dir string) (ResolverAdapter, error)

NewFileResolverAdapter returns new ResolverAdapter for Mock client, with file based mock policy.

param: dir (string) -> directory path where all the policy spec located.

type ResponseHandlerFunc 

type ResponseHandlerFunc func(*http.Response) error

ResponseHandlerFunc is a type of function that takes in a Response, and does something with it. The ResponseHandlerFunc is called when the HTTP client successfully receives a response and the The response body is not automatically closed. It must be closed either by the ResponseHandlerFunc or by the caller out-of-band. Failure to do so will result in a memory leak.

Main purposes: to enable delay for mocking http calls

type ResponseLogHook 

type ResponseLogHook func(Logger, *http.Response)

ResponseLogHook is like RequestLogHook, but allows running a function on each HTTP response. This function will be invoked at the end of every HTTP request executed, regardless of whether a subsequent retry needs to be performed or not. If the response body is read or closed from this method, this will affect the response returned from Do().

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.