cache

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 9, 2021 License: MIT Imports: 18 Imported by: 2

README

High performance resilient in-memory cache for Go

This library defines cache interfaces and provides in-memory implementations.

Build Status Coverage Status GoDevDoc time tracker Code lines Comments

Documentation

Overview

Package cache defines cache interfaces and provides in-memory implementations.

Index

Examples

Constants

View Source
const (
	// ErrExpired indicates expired cache entry,
	// may implement ErrWithExpiredItem to enable stale value serving.
	ErrExpired = SentinelError("expired cache item")

	// ErrNotFound indicates missing cache entry.
	ErrNotFound = SentinelError("missing cache item")

	// ErrNothingToInvalidate indicates no caches were added to Invalidator.
	ErrNothingToInvalidate = SentinelError("nothing to invalidate")

	// ErrAlreadyInvalidated indicates recent invalidation.
	ErrAlreadyInvalidated = SentinelError("already invalidated")
)
View Source
const (
	// MetricMiss is a name of a metric to count cache miss events.
	MetricMiss = "cache_miss"
	// MetricExpired is a name of a metric to count expired cache read events.
	MetricExpired = "cache_expired"
	// MetricHit is a name of a metric to count valid cache read events.
	MetricHit = "cache_hit"
	// MetricWrite is a name of a metric to count cache write events.
	MetricWrite = "cache_write"
	// MetricItems is a name of a gauge to count number of items in cache.
	MetricItems = "cache_items"

	// MetricRefreshed is a name of a metric to count stale refresh events.
	MetricRefreshed = "cache_refreshed"
	// MetricBuild is a name of a metric to count value building events.
	MetricBuild = "cache_build"
	// MetricFailed is a name of a metric to count number of failed value builds.
	MetricFailed = "cache_failed"

	// MetricChanged is a name of a metric to count number of cache builds that changed cached value.
	MetricChanged = "cache_changed"

	// MetricEvict is a name of metric to count evictions.
	MetricEvict = "cache_evict"
)
View Source
const DefaultTTL = time.Duration(0)

DefaultTTL indicates default (unlimited ttl) value for entry expiration time.

View Source
const SkipWriteTTL = time.Duration(-1)

SkipWriteTTL is a ttl value to indicate that cache must not be stored.

Variables

This section is empty.

Functions

func GobRegister

func GobRegister(values ...interface{})

GobRegister enables cached type transferring.

func GobTypesHash

func GobTypesHash() uint64

GobTypesHash returns a fingerprint of a group of types to transfer.

func GobTypesHashReset

func GobTypesHashReset()

GobTypesHashReset resets types hash to zero value.

func SkipRead

func SkipRead(ctx context.Context) bool

SkipRead returns true if cache read is ignored in context.

func TTL

func TTL(ctx context.Context) time.Duration

TTL retrieves cache time to live from context, zero value is returned by default.

func WithSkipRead

func WithSkipRead(ctx context.Context) context.Context

WithSkipRead returns context with cache read ignored.

With such context cache.Reader should always return ErrNotFound discarding cached value.

func WithTTL

func WithTTL(ctx context.Context, ttl time.Duration, updateExisting bool) context.Context

WithTTL adds cache time to live information to context.

If there is already ttl in context and updateExisting, then ttl value in original context will be updated.

Updating existing ttl can be useful if ttl information is only available internally during cache value build, in such a case value builder can indicate ttl to external cache backend. For example cache ttl can be derived from HTTP response cache headers of value source.

When existing ttl is updated minimal non-zero value is kept. This is necessary to unite ttl requirements of multiple parties.

Types

type Config

type Config struct {
	// Logger is an instance of contextualized logger, can be nil.
	Logger ctxd.Logger

	// Stats is metrics collector, can be nil.
	Stats stats.Tracker

	// Name is cache instance name, used in stats and logging.
	Name string

	// TimeToLive is delay before entry expiration, default 5m.
	TimeToLive time.Duration

	// DeleteExpiredAfter is delay before expired entry is deleted from cache, default 24h.
	DeleteExpiredAfter time.Duration

	// DeleteExpiredJobInterval is delay between two consecutive cleanups, default 1h.
	DeleteExpiredJobInterval time.Duration

	// ItemsCountReportInterval is items count metric report interval, default 1m.
	ItemsCountReportInterval time.Duration

	// ExpirationJitter is a fraction of TTL to randomize, default 0.1.
	// Use -1 to disable.
	// If enabled, entry TTL will be randomly altered in bounds of ±(ExpirationJitter * TTL / 2).
	ExpirationJitter float64

	// HeapInUseSoftLimit sets heap in use threshold when eviction of most expired items will be triggered.
	//
	// Eviction is a part of delete expired job, eviction runs at most once per delete expired job and
	// removes most expired entries up to EvictFraction.
	HeapInUseSoftLimit uint64

	// CountSoftLimit sets count threshold when eviction of most expired items will be triggered.
	//
	// Eviction is a part of delete expired job, eviction runs at most once per delete expired job and
	// removes most expired entries up to EvictFraction.
	CountSoftLimit uint64

	// EvictFraction is a fraction (0, 1] of total count of items to be evicted when resource is overused,
	// default 0.1 (10% of items).
	EvictFraction float64
}

Config controls cache instance.

func (Config) Use

func (c Config) Use(cfg *Config)

Use is a functional option to apply configuration.

type Deleter

type Deleter interface {
	// Delete removes a cache entry with a given key
	// and returns ErrNotFound for non-existent keys.
	Delete(ctx context.Context, key []byte) error
}

Deleter deletes from cache.

type Dumper

type Dumper interface {
	Dump(w io.Writer) (int, error)
}

Dumper dumps cache entries in binary format.

type Entry

type Entry interface {
	Key() []byte
	Value() interface{}
}

Entry is cache entry with key and value.

type ErrWithExpiredItem

type ErrWithExpiredItem interface {
	error
	Value() interface{}
	ExpiredAt() time.Time
}

ErrWithExpiredItem defines an expiration error with entry details.

type Failover

type Failover struct {
	// Errors caches errors of failed updates.
	Errors *ShardedMap
	// contains filtered or unexported fields
}

Failover is a cache frontend to manage cache updates in a non-conflicting and performant way.

Please use NewFailover to create instance.

func NewFailover

func NewFailover(options ...func(cfg *FailoverConfig)) *Failover

NewFailover creates a Failover cache instance.

Build is locked per key to avoid concurrent updates, new value is served . Stale value is served during non-concurrent update (up to FailoverConfig.UpdateTTL long).

func (*Failover) Get

func (f *Failover) Get(
	ctx context.Context,
	key []byte,
	buildFunc func(ctx context.Context,
	) (interface{}, error)) (interface{}, error)

Get returns value from cache or from build function.

Example
package main

import (
	"context"
	"errors"
	"fmt"
	"strconv"
	"time"

	"github.com/bool64/cache"
)

type Value struct {
	Sequence int
	ID       int
	Country  string
}

type Service interface {
	GetByID(ctx context.Context, country string, id int) (Value, error)
}

// Real is an instance of real service that does some slow/expensive processing.
type Real struct {
	Calls int
}

func (s *Real) GetByID(ctx context.Context, country string, id int) (Value, error) {
	s.Calls++

	if id == 0 {
		return Value{}, errors.New("invalid id")
	}

	return Value{
		Country:  country,
		ID:       id,
		Sequence: s.Calls,
	}, nil
}

// Cached is a service wrapper to serve already processed results from cache.
type Cached struct {
	upstream Service
	storage  *cache.Failover
}

func (s Cached) GetByID(ctx context.Context, country string, id int) (Value, error) {
	// Prepare string cache key and call stale cache.
	// Build function will be called if there is a cache miss.
	cacheKey := country + ":" + strconv.Itoa(id)

	value, err := s.storage.Get(ctx, []byte(cacheKey), func(ctx context.Context) (interface{}, error) {
		return s.upstream.GetByID(ctx, country, id)
	})
	if err != nil {
		return Value{}, err
	}

	// Type assert and return result.
	return value.(Value), nil
}

func main() {
	var service Service

	ctx := context.Background()

	service = Cached{
		upstream: &Real{},
		storage: cache.NewFailover(func(cfg *cache.FailoverConfig) {
			cfg.BackendConfig.TimeToLive = time.Minute
		}),
	}

	fmt.Println(service.GetByID(ctx, "DE", 123))
	fmt.Println(service.GetByID(ctx, "US", 0)) // Error increased sequence, but was cached with short-ttl.
	fmt.Println(service.GetByID(ctx, "US", 0)) // This call did not hit backend.
	fmt.Println(service.GetByID(ctx, "US", 456))
	fmt.Println(service.GetByID(ctx, "DE", 123))
	fmt.Println(service.GetByID(ctx, "US", 456))
	fmt.Println(service.GetByID(ctx, "FR", 789))

}
Output:

{1 123 DE} <nil>
{0 0 } invalid id
{0 0 } invalid id
{3 456 US} <nil>
{1 123 DE} <nil>
{3 456 US} <nil>
{4 789 FR} <nil>

type FailoverConfig

type FailoverConfig struct {
	// Name is added to logs and stats.
	Name string

	// Backend is a cache instance, ShardedMap created by default.
	Backend ReadWriter

	// BackendConfig is a configuration for ShardedMap cache instance if Backend is not provided.
	BackendConfig Config

	// FailedUpdateTTL is ttl of failed build cache, default 20s, -1 disables errors cache.
	FailedUpdateTTL time.Duration

	// UpdateTTL is a time interval to retry update, default 1 minute.
	UpdateTTL time.Duration

	// SyncUpdate disables update in background, default is background update with stale value served.
	SyncUpdate bool

	// SyncRead enables backend reading in the critical section to ensure cache miss
	// will not trigger multiple updates sequentially.
	//
	// Probability of such issue is low, there is performance penalty for enabling this option.
	SyncRead bool

	// MaxStaleness is duration when value can be served after expiration.
	// If value has expired longer than this duration it won't be served unless value update failure.
	MaxStaleness time.Duration

	// Logger collects messages with context.
	Logger ctxd.Logger

	// Stats tracks stats.
	Stats stats.Tracker

	// ObserveMutability enables deep equal check with metric collection on cache update.
	ObserveMutability bool
}

FailoverConfig is optional configuration for NewFailover.

func (FailoverConfig) Use

func (fc FailoverConfig) Use(cfg *FailoverConfig)

Use is a functional option for NewFailover to apply configuration.

type Invalidator

type Invalidator struct {
	sync.Mutex

	// SkipInterval defines minimal duration between two cache invalidations (flood protection).
	SkipInterval time.Duration

	// Callbacks contains a list of functions to call on invalidate.
	Callbacks []func(ctx context.Context)
	// contains filtered or unexported fields
}

Invalidator is a registry of cache expiration triggers.

func (*Invalidator) Invalidate

func (i *Invalidator) Invalidate(ctx context.Context) error

Invalidate triggers cache expiration.

type NoOp

type NoOp struct{}

NoOp is a ReadWriter stub.

func (NoOp) Delete

func (NoOp) Delete(_ context.Context, _ []byte) error

Delete is always missing item.

func (NoOp) Read

func (NoOp) Read(_ context.Context, _ []byte) (interface{}, error)

Read is always missing item.

func (NoOp) Write

func (NoOp) Write(_ context.Context, _ []byte, _ interface{}) error

Write does nothing.

type ReadWriter

type ReadWriter interface {
	Reader
	Writer
}

ReadWriter reads from and writes to cache.

type Reader

type Reader interface {
	// Read returns cached value or error.
	Read(ctx context.Context, key []byte) (interface{}, error)
}

Reader reads from cache.

type Restorer

type Restorer interface {
	Restore(r io.Reader) (int, error)
}

Restorer restores cache entries from binary dump.

type SentinelError

type SentinelError string

SentinelError is an error.

func (SentinelError) Error

func (e SentinelError) Error() string

Error implements error.

type ShardedMap

type ShardedMap struct {
	// contains filtered or unexported fields
}

ShardedMap is an in-memory cache backend.

func NewShardedMap

func NewShardedMap(options ...func(cfg *Config)) *ShardedMap

NewShardedMap creates an instance of in-memory cache with optional configuration.

Example
package main

import (
	"context"
	"fmt"
	"time"

	"github.com/bool64/cache"
	"github.com/bool64/ctxd"
	"github.com/bool64/stats"
)

func main() {
	// Create cache instance.
	c := cache.NewShardedMap(cache.Config{
		Name:       "dogs",
		TimeToLive: 13 * time.Minute,
		Logger:     &ctxd.LoggerMock{},
		Stats:      &stats.TrackerMock{},

		// Tweak these parameters to reduce/stabilize rwMutexMap consumption at cost of cache hit rate.
		// If cache cardinality and size are reasonable, default values should be fine.
		DeleteExpiredAfter:       time.Hour,
		DeleteExpiredJobInterval: 10 * time.Minute,
		HeapInUseSoftLimit:       200 * 1024 * 1024, // 200MB soft limit for process heap in use.
		EvictFraction:            0.2,               // Drop 20% of mostly expired items (including non-expired) on heap overuse.
	}.Use)

	// Use context if available.
	ctx := context.TODO()

	// Write value to cache.
	_ = c.Write(ctx, []byte("my-key"), []int{1, 2, 3})

	// Read value from cache.
	val, _ := c.Read(ctx, []byte("my-key"))
	fmt.Printf("%v", val)

}
Output:

[1 2 3]

func (ShardedMap) Delete

func (c ShardedMap) Delete(ctx context.Context, key []byte) error

func (ShardedMap) DeleteAll

func (c ShardedMap) DeleteAll(ctx context.Context)

DeleteAll erases all entries.

func (*ShardedMap) Dump

func (c *ShardedMap) Dump(w io.Writer) (int, error)

Dump saves cached entries and returns a number of processed entries.

Dump uses encoding/gob to serialize cache entries, therefore it is necessary to register cached types in advance with cache.GobRegister.

func (ShardedMap) ExpireAll

func (c ShardedMap) ExpireAll(ctx context.Context)

ExpireAll marks all entries as expired, they can still serve stale cache.

func (ShardedMap) Len

func (c ShardedMap) Len() int

Len returns number of elements in cache.

func (ShardedMap) Read

func (c ShardedMap) Read(ctx context.Context, key []byte) (interface{}, error)

Read gets value.

func (*ShardedMap) Restore

func (c *ShardedMap) Restore(r io.Reader) (int, error)

Restore loads cached entries and returns number of processed entries.

Restore uses encoding/gob to unserialize cache entries, therefore it is necessary to register cached types in advance with cache.GobRegister.

func (ShardedMap) Walk

func (c ShardedMap) Walk(walkFn func(e Entry) error) (int, error)

Walk walks cached entries.

func (ShardedMap) Write

func (c ShardedMap) Write(ctx context.Context, k []byte, v interface{}) error

Write sets value.

type Walker

type Walker interface {
	Walk(func(entry Entry) error) (int, error)
}

Walker calls function for every entry in cache and fails on first error returned by that function.

Count of processed entries is returned.

type Writer

type Writer interface {
	// Write stores value in cache with a given key.
	Write(ctx context.Context, key []byte, value interface{}) error
}

Writer writes to cache.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL