loadingcache

package module
v0.0.0-...-f1ada25 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 30, 2025 License: MIT Imports: 8 Imported by: 0

README

loadingcache

A fork of @Hartimer's loadingcache, with extra features stapled on.

Documentation

Overview

Package loadingcache provides a way for clients to create a cache capable of loading values on demand, should they get cache misses.

You can configure the cache to expire entries after a certain amount elapses since the last write and/or read.

This project is heavily inspired by Guava Cache (https://github.com/google/guava/wiki/CachesExplained).

All errors are wrapped by github.com/pkg/errors.Wrap. If you which to check the type of it, please use github.com/pkg/errors.Is.

Index

Examples

Constants

This section is empty.

Variables

View Source
var ErrKeyNotFound error = errors.New("Key not found")

ErrKeyNotFound represents an error indicating that the key was not found

View Source
var ErrTombstone error = errors.New("tombstone")

ErrTombstone represents an error indicating that the key is a "tombstone"

Functions

func WithBackgroundEvictFrequency

func WithBackgroundEvictFrequency[Key comparable, Value any](d time.Duration) func(cacheOptions[Key, Value]) cacheOptions[Key, Value]

WithBackgroundEvictFrequency is a CacheOption which will set the overall TTL for a given key. This will spawn a background goroutine to purge keys within the specified interval.

func WithClock

func WithClock[Key comparable, Value any](c clock.Clock) func(cacheOptions[Key, Value]) cacheOptions[Key, Value]

WithClock is a CacheOption which allows you to set a specific clock implementation. This is useful in testing scenarios, to check that a key gets expired after a given interval.

func WithExpireAfterRead

func WithExpireAfterRead[Key comparable, Value any](d time.Duration) func(cacheOptions[Key, Value]) cacheOptions[Key, Value]

WithExpireAfterRead is a CacheOption which allows you to set the duration that a key will persist after being read from. For example, setting this value to 1 * time.Minute will mean that a key is expired 1 minute after it is read from, given that no new reads are done within that 1 minute interval.

func WithExpireAfterWrite

func WithExpireAfterWrite[Key comparable, Value any](d time.Duration) func(cacheOptions[Key, Value]) cacheOptions[Key, Value]

WithExpireAfterWrite is a CacheOption which allows you to set the duration that a key will persist after being written to. For example, setting this value to 1 * time.Minute will mean that a key is expired 1 minute after it is written to, given that no other writes occur to a given key within that 1 minute interval.

func WithHashCodeFunc

func WithHashCodeFunc[Key comparable, Value any](fn func(key Key) uint64) func(cacheOptions[Key, Value]) cacheOptions[Key, Value]

WithHashCodeFunc is a CacheOption which allows you to specify a stable hashing function for a given key. This is used in the sharding implementation, and when ShardCount > 1, must be specified otherwise this library will panic.

func WithLoadFunc

func WithLoadFunc[Key comparable, Value any](f LoadFunc[Key, Value]) func(cacheOptions[Key, Value]) cacheOptions[Key, Value]

WithLoadFunc is a CacheOption which allows you to set a custom function to lazy load in cache entries, if they weren't already present inside of the cache.

func WithMaxSize

func WithMaxSize[Key comparable, Value any](sz int32) func(cacheOptions[Key, Value]) cacheOptions[Key, Value]

WithMaxSize is a CacheOption which allows you to set an upper bound to the number of items stored within this cache.

func WithName

func WithName[Key comparable, Value any](name string) func(cacheOptions[Key, Value]) cacheOptions[Key, Value]

WithName is a CacheOption which sets a name for this cache, handy for metrics / logging. It's slightly useless at the moment.

func WithRemovalListeners

func WithRemovalListeners[Key comparable, Value any](li ...RemovalListener[Key, Value]) func(cacheOptions[Key, Value]) cacheOptions[Key, Value]

WithRemovalListeners is a CacheOption which allows you to listen for keys that have been removed, and react accordingly.

func WithShardCount

func WithShardCount[Key comparable, Value any](c int) func(cacheOptions[Key, Value]) cacheOptions[Key, Value]

WithShardCount is a CacheOption which allows you to specify the number of shards for this cache. This is useful if you have significant lock contention, but requires a stable hash code function to be specified as well (see: WithHashCodeFunc).

Types

type Cache

type Cache[Key comparable, Value any] interface {
	// Get returns the value associated with a given key. If no entry exists for
	// the provided key, loadingcache.ErrKeyNotFound is returned.
	Get(ctx context.Context, key Key) (Value, error)

	// Put adds a value to the cache identified by a key.
	// If a value already exists associated with that key, it
	// is replaced.
	Put(key Key, value Value)

	// Invalidate removes keys from the cache. If a key does not exists it is a noop.
	Invalidate(keys ...Key)

	// InvalidateAll invalidates all keys
	InvalidateAll()

	// Close cleans up any resources used by the cache
	Close()

	// Stats returns the curret stats
	Stats() Stats
}

Cache describe the base interface to interact with a generic cache.

This interface reduces all keys and values to a generic interface{}.

Example (AdvancedUsage)
package main

import (
	"context"
	"fmt"
	"time"

	"github.com/devzero-inc/loadingcache"
)

func main() {
	type Key int
	type Value string

	cache := loadingcache.New(
		loadingcache.WithMaxSize[Key, Value](2),
		loadingcache.WithExpireAfterRead[Key, Value](2*time.Minute),
		loadingcache.WithExpireAfterWrite[Key, Value](time.Minute),
		loadingcache.WithRemovalListeners(
			func(notification loadingcache.RemovalNotification[Key, Value]) {
				fmt.Printf("Entry removed due to %s\n", notification.Reason)
			},
		),
		loadingcache.WithLoadFunc(func(_ context.Context, key Key) (Value, error) {
			fmt.Printf("Loading key %v\n", key)
			return Value(fmt.Sprint(key)), nil
		}),
	)

	cache.Put(1, "1")
	val1, _ := cache.Get(context.TODO(), 1)
	fmt.Printf("%v\n", val1)

	val2, _ := cache.Get(context.TODO(), 2)
	fmt.Printf("%v\n", val2)

	val3, _ := cache.Get(context.TODO(), 3)
	fmt.Printf("%v\n", val3)

}
Output:
1
Loading key 2
2
Loading key 3
Entry removed due to SIZE
3
Example (SimpleUsage)
package main

import (
	"context"
	"fmt"

	"github.com/pkg/errors"

	"github.com/devzero-inc/loadingcache"
)

func main() {
	type Key string
	type Value int

	cache := loadingcache.New[Key, Value]()

	// Addign some values and reading them
	cache.Put("a", 1)
	cache.Put("b", 2)
	cache.Put("c", 3)
	val1, _ := cache.Get(context.TODO(), "a") // Don't forget to check for errors
	fmt.Printf("%v\n", val1)
	val2, _ := cache.Get(context.TODO(), "b") // Don't forget to check for errors
	fmt.Printf("%v\n", val2)

	// Getting a value that does not exist
	_, err := cache.Get(context.TODO(), "d")
	if errors.Is(err, loadingcache.ErrKeyNotFound) {
		fmt.Println("That key does not exist")
	}

	// Evicting
	cache.Invalidate("a")
	cache.Invalidate("b", "c")
	cache.InvalidateAll()

}
Output:
1
2
That key does not exist

func New

func New[Key comparable, Value any](opts ...CacheOption[Key, Value]) Cache[Key, Value]

New instantiates a new cache

type CacheOption

type CacheOption[Key comparable, Value any] func(cacheOptions[Key, Value]) cacheOptions[Key, Value]

CacheOption describes an option that can configure the cache

type LoadFunc

type LoadFunc[Key comparable, Value any] func(context.Context, Key) (Value, error)

LoadFunc represents a function that given a key, it returns a value or an error.

type LoadingError

type LoadingError interface {
	error
	CacheFor() time.Duration
}

LoadingError is an interface which extends the default error type to also provide a method on how long this error should be cached for.

func Errorf

func Errorf(t time.Duration, f string, args ...any) LoadingError

Errorf is a function which instantiates a new error abiding by the LoadingError interface. Use this when you want a "tombstone" value to be created inside of the cache, specifying how long the tombstone should be valid for.

type RemovalListener

type RemovalListener[Key comparable, Value any] func(RemovalNotification[Key, Value])

RemovalListener represents a removal listener

type RemovalNotification

type RemovalNotification[Key comparable, Value any] struct {
	Key    Key
	Value  Value
	Reason RemovalReason
}

RemovalNotification is passed to listeners everytime an entry is removed

type RemovalReason

type RemovalReason string

RemovalReason is an enum describing the causes for an entry to be removed from the cache.

const (
	// RemovalReasonExplicit means the entry was explicitly invalidated
	RemovalReasonExplicit RemovalReason = "EXPLICIT"

	// RemovalReasonReplaced means the entry was replaced by a new one
	RemovalReasonReplaced RemovalReason = "REPLACED"

	// RemovalReasonExpired means the entry expired, e.g. too much time
	// since last read/write.
	RemovalReasonExpired RemovalReason = "EXPIRED"

	// RemovalReasonSize means the entry was removed due to the cache size.
	RemovalReasonSize RemovalReason = "SIZE"
)

type Stats

type Stats interface {
	// EvictionCount is the number of times an entry has been evicted
	EvictionCount() int64

	// HitCount the number of times Cache lookup methods have returned a cached value
	HitCount() int64

	// HitRate is the ratio of cache requests which were hits. This is defined as
	// hitCount / requestCount, or 1.0 when requestCount == 0
	HitRate() float64

	// MissCount is the number of times Cache lookup methods have returned an uncached
	// (newly loaded) value
	MissCount() int64

	// MissRate is the ratio of cache requests which were misses. This is defined as
	// missCount / requestCount, or 0.0 when requestCount == 0
	MissRate() float64

	// RequestCount is the number of times Cache lookup methods have returned either a cached or
	// uncached value. This is defined as hitCount + missCount
	RequestCount() int64

	// LoadSuccessCount is the number of times Cache lookup methods have successfully
	// loaded a new value
	LoadSuccessCount() int64

	// LoadErrorCount is the number of times Cache lookup methods threw an exception while loading
	// a new value
	LoadErrorCount() int64

	// LoadErrorRate is the ratio of cache loading attempts which threw exceptions.
	// This is defined as loadExceptionCount / (loadSuccessCount + loadExceptionCount), or 0.0 when
	// loadSuccessCount + loadExceptionCount == 0
	LoadErrorRate() float64

	// LoadCount the total number of times that Cache lookup methods attempted to load new values.
	// This includes both successful load operations, as well as those that threw exceptions.
	// This is defined as loadSuccessCount + loadExceptionCount
	LoadCount() int64

	// LoadTotalTime is the total duration the cache has spent loading new values
	LoadTotalTime() time.Duration

	// AverageLoadPenalty is the average duration spent loading new values. This is defined as
	// totalLoadTime / (loadSuccessCount + loadExceptionCount).
	AverageLoadPenalty() time.Duration
}

Stats exposes cache relevant metrics.

Be aware that this interface may be exposing a live stats collector, and as such if you manually calculate rates, values may differ if calls to the cache have occurred between calls.

Directories

Path Synopsis
Package stats implements the internal logic for the statistics interface.
Package stats implements the internal logic for the statistics interface.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL