httpcache

package module
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 25, 2020 License: MIT Imports: 22 Imported by: 2

README

#+HTML: <a href="https://github.com/sillygod/cdp-cache/actions?query=workflow%3ACI"><img src="https://github.com/sillygod/cdp-cache/workflows/CI/badge.svg?branch=master" /></a>
#+HTML: </div>

#+HTML: <a href="https://www.codacy.com/manual/sillygod/cdp-cache?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=sillygod/cdp-cache&amp;utm_campaign=Badge_Grade"><img src="https://app.codacy.com/project/badge/Grade/43d801ba437a42419e479492eca72ee2" /></a>
#+HTML: </div>


#+HTML: <a href="https://goreportcard.com/report/github.com/sillygod/cdp-cache"><img src="https://goreportcard.com/badge/github.com/sillygod/cdp-cache" /></a>
#+HTML: </div>


* Caddy Http Cache

  This is a http cache plugin for caddy 2. In fact, I am not familiar with the mechanism of the cdn cache so I borrow the code from https://github.com/nicolasazrak/caddy-cache. which is the caddy v1's cache module. Then, I migrate the codebase to be consist with the caddy2 architecture, do some change, and implement more feature for it.

  Currently, this is under development so it will change frequently.

* How to build

  In development, go to the cmd folder and type the following commands.

  #+begin_src sh
    go build -ldflags="-w -s" -o caddy
  #+end_src
  
  To build linux binary by this
  #+begin_src 
  GOOS=linux GOARCH=amd64 go build -ldflags="-w -s" -o caddy
  #+end_src
  
  Or you can use the [[https://github.com/caddyserver/xcaddy][xcaddy]] to build the executable file.
  Ensure you've install it by =go get -u github.com/caddyserver/xcaddy/cmd/xcaddy=
  #+begin_src sh
    xcaddy build v2.0.0 --with github.com/sillygod/cdp-cache 
  #+end_src
  
  Xcaddy also provide a way to develop plugin locally.
  #+begin_src sh
    xcaddy run --config cmd/Caddyfile
  #+end_src

* How to run

  You can directly run with the binary.
  #+begin_src sh
    caddy run --config [caddy file] --adapter caddyfile
  #+end_src
  
  Or if you are preferred to use the docker 
  #+begin_src sh
    docker run -it -p80:80 -p443:443 -v [the path of caddyfile]:/app/Caddyfile docker.pkg.github.com/sillygod/cdp-cache/caddy:latest
  #+end_src

* Configuration
  
  The following will list current support configs in the caddyfile.

** status_header
   The header to set cache status. default value: =X-Cache-Status=

** match_path
   Only the request's path match the condition will be cached. Ex. =/= means all request need to be cached because all request's path must start with =/=

** default_max_age
   The cache's expiration time.

** match_header
   only the req's header match the condtions 

** path
   The position where to save the file. Only applied when the =cache_type= is =file=.

** cache_key
   The key of cache entry. The default value is ={http.request.method} {http.request.host}{http.request.uri.path}?{http.request.uri.query}=

** cache_bucket_num
   The bucket number of the mod of cache_key's checksum. The default value is 256.

** cache_type
   Indicate to use which kind of cache's storage backend. Currently, there are two choices. One is =file= and the other is =in_memory=
   
** cache_max_memory_size
   The max memory usage for in_memory backend.

* Deployment
  
  In progress, currently I add github action to auto build the executable for linux platform and also the docker images.


* Benchmarks
  
  Now, I just simply compares the performance between in-memory and disk.
  
** Env
   Caddy run with the config file under directory =benchmark= and tests were run on the mac book pro (1.4 GHz Intel Core i5, 16 GB 2133 MHz LPDDR3)

** Test Result

  The following benchmark is analysized by =wrk -c 200 -d 30s --latency -t 4 http://localhost:9992/test.jpg=
  the file size is 100KB
  
  |                         | req/s | latency (50% 90% 99%)        |
  | proxy + file cache      | 10233 | 18.24ms /  22.56ms / 33.34ms |
  | proxy + in memory cache | 15810 | 12.55ms /  14.09ms / 22.88ms |

  
* Todo list
  
  - [x] in memory cache (consider to use [[https://github.com/golang/groupcache][groupcache]])
  - [X] chache control expiration issue default will add 24 hours if no rule applied (in the cacheobject library)
  - [X] research about usagepool in the caddy2 (use global variable instead)
  - [ ] purge cache entries
  - [ ] custom log format (currently only add zap logger to print info)
    Idealy, We can implement a custom log module.
  - [ ] more optimization..

Documentation

Index

Constants

View Source
const (
	BYTE = 1 << (iota * 10)
	KB
	MB
	GB
)

BYTE represents the num of byte 1MB = 2^10 BYTE 1GB = 2^10 MB

Variables

This section is empty.

Functions

This section is empty.

Types

type CacheType

type CacheType string

CacheType is the type of cache which means the backend for storing cache content.

type Config

type Config struct {
	Type                   CacheType                `json:"type,omitempty"`
	StatusHeader           string                   `json:"status_header,omitempty"`
	DefaultMaxAge          time.Duration            `json:"default_max_age,omitempty"`
	LockTimeout            time.Duration            `json:"lock_timeout,omitempty"`
	RuleMatchersRaws       []RuleMatcherRawWithType `json:"rule_matcher_raws,omitempty"`
	RuleMatchers           []RuleMatcher            `json:"-"`
	CacheBucketsNum        int                      `json:"cache_buckets_num,omitempty"`
	CacheMaxMemorySize     int                      `json:"cache_max_memory_size,omitempty"`
	Path                   string                   `json:"path,omitempty"`
	CacheKeyTemplate       string                   `json:"cache_key_template,omitempty"`
	RedisConnectionSetting string                   `json:"redis_connection_setting,omitempty"`
}

Config is the configuration for cache process

type Entry

type Entry struct {
	Request  *http.Request
	Response *Response
	// contains filtered or unexported fields
}

Entry consists of a cache key and one or more response corresponding to the prior requests. https://httpwg.org/specs/rfc7234.html#caching.overview

func NewEntry

func NewEntry(key string, request *http.Request, response *Response, config *Config) *Entry

NewEntry creates a new Entry for the given request and response and it also calculates whether it is public or not

func (*Entry) Clean

func (e *Entry) Clean() error

Clean purges the cache

func (*Entry) IsFresh

func (e *Entry) IsFresh() bool

IsFresh indicates this entry is not expired

func (*Entry) Key

func (e *Entry) Key() string

Key return the key for the entry

func (*Entry) WriteBodyTo

func (e *Entry) WriteBodyTo(w http.ResponseWriter) error

WriteBodyTo sends the body to the http.ResponseWritter

type HTTPCache

type HTTPCache struct {
	// contains filtered or unexported fields
}

HTTPCache is a http cache for http request which is focus on static files

func NewHTTPCache

func NewHTTPCache(config *Config) *HTTPCache

NewHTTPCache new a HTTPCache to hanle cache entries

func (*HTTPCache) Del added in v0.1.0

func (h *HTTPCache) Del(key string) error

Del purge the key immediately

func (*HTTPCache) Get

func (h *HTTPCache) Get(key string, request *http.Request) (*Entry, bool)

Get returns the cached response

func (*HTTPCache) Keys added in v0.2.0

func (h *HTTPCache) Keys() []string

Keys list the keys holded by this cache

func (*HTTPCache) Put

func (h *HTTPCache) Put(request *http.Request, entry *Entry)

Put adds the entry in the cache

type Handler

type Handler struct {
	Config   *Config    `json:"config,omitempty"`
	Cache    *HTTPCache `json:"-"`
	URLLocks *URLLock   `json:"-"`
	// contains filtered or unexported fields
}

Handler is a http handler as a middleware to cache the response

func (Handler) CaddyModule

func (Handler) CaddyModule() caddy.ModuleInfo

CaddyModule returns the Caddy module information

func (*Handler) Provision

func (h *Handler) Provision(ctx caddy.Context) error

Provision setups the configs

func (*Handler) ServeHTTP

func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request, next caddyhttp.Handler) error

func (*Handler) UnmarshalCaddyfile

func (h *Handler) UnmarshalCaddyfile(d *caddyfile.Dispenser) error

UnmarshalCaddyfile sets up the handler from Caddyfile

:4000 {
    reverse_proxy yourserver:5000
    http_cache {
        match_path /assets
        match_header Content-Type image/jpg image/png
        status_header X-Cache-Status
        default_max_age 15m
        path /tmp/caddy-cache
    }
}

func (*Handler) Validate

func (h *Handler) Validate() error

Validate validates httpcache's configuration.

type HeaderRuleMatcher

type HeaderRuleMatcher struct {
	Header string   `json:"header"`
	Value  []string `json:"value"`
}

HeaderRuleMatcher determines whether the request's header is matched.

type PathRuleMatcher

type PathRuleMatcher struct {
	Path string `json:"path"`
}

PathRuleMatcher determines whether the reuqest's path is matched.

type PurgePayload added in v0.1.0

type PurgePayload struct {
	Method string `json:"method"`
	Host   string `json:"host"`
	URI    string `json:"uri"`
	// contains filtered or unexported fields
}

PurgePayload holds the field which will be unmarshaled from the request's body NOTE: the format of URI can contains the query param. ex. when the client send a delete reqeust with the body

{
   "method": "GET",
   "hots": "example.com",
   "uri": "/static?ext=txt",
}

type Response

type Response struct {
	Code      int
	HeaderMap http.Header
	// contains filtered or unexported fields
}

Response encapsulates the entry

func NewResponse

func NewResponse() *Response

NewResponse returns an initialized Response.

func (*Response) Clean

func (r *Response) Clean() error

Clean performs purge the cache

func (*Response) Close

func (r *Response) Close() error

Close indicate the data is completely written to the body so that we can close it.

func (*Response) Flush

func (r *Response) Flush()

func (*Response) GetReader

func (r *Response) GetReader() (io.ReadCloser, error)

func (*Response) Header

func (r *Response) Header() http.Header

func (*Response) SetBody

func (r *Response) SetBody(body backends.Backend)

func (*Response) WaitClose

func (r *Response) WaitClose()

func (*Response) WaitHeaders

func (r *Response) WaitHeaders()

func (*Response) Write

func (r *Response) Write(buf []byte) (int, error)

func (*Response) WriteHeader

func (r *Response) WriteHeader(code int)

type RuleMatcher

type RuleMatcher interface {
	// contains filtered or unexported methods
}

RuleMatcher determines whether the request should be cached or not.

type RuleMatcherRawWithType

type RuleMatcherRawWithType struct {
	Type RuleMatcherType
	Data json.RawMessage
}

type RuleMatcherType

type RuleMatcherType string

RuleMatcherType specifies the type of matching rule to cache.

const (
	MatcherTypePath   RuleMatcherType = "path"
	MatcherTypeHeader RuleMatcherType = "header"
)

the following list the different way to decide the request whether is matched or not to be cached it's response.

type URLLock

type URLLock struct {
	// contains filtered or unexported fields
}

func NewURLLock

func NewURLLock(config *Config) *URLLock

func (*URLLock) Acquire added in v0.0.2

func (allLocks *URLLock) Acquire(key string) *sync.Mutex

Acquire a lock for given key

Directories

Path Synopsis
Package main is the entry point of the Caddy application.
Package main is the entry point of the Caddy application.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL