The highest tagged major version is
Documentation
¶
Overview ¶
Package memory is a cache.Cache held in a map in this process.
What choosing it commits you to ¶
Nothing here has a network, and that is most of what distinguishes it from the redis provider. There is no circuit breaker, no cache.ErrUnavailable, no connection to lose and no partial batch: a read either finds the key or reports cache.ErrNotFound, Ping always succeeds, and Flush needs no namespace because this cache wholly owns its store. Code written against *Cache[T] rather than against cache.Cache[T] therefore carries none of the handling those possibilities force on it.
What it gives up is everything the shared store was for. Entries live in one process, so two replicas have two caches that agree about nothing, a restart is a cold cache, and a write on one instance is invisible to the others — including a delete, which is the case that bites: an invalidation that reaches only the replica that performed it leaves the stale value being served everywhere else until it expires. A cache whose entries must be consistent across replicas, or must survive a deploy, wants the redis provider.
Values are shared, not copied ¶
Set stores the pointer it is given and Get hands that same pointer back. Nothing is serialized, so a caller that mutates a value it read from the cache mutates what every other reader sees, with no lock held. Store values that are treated as immutable, or copy on the way out. The redis provider does not have this property — it encodes on write and decodes into a fresh value on read — so code that relies on either behavior is not portable between the two.
Bounds ¶
By default the map is bounded only by expiry, and expired entries are reclaimed lazily, on the read that finds them or when the key is overwritten. A key written once and never read again therefore holds its memory indefinitely: pass WithJanitor to sweep on a timer, WithMaxEntries to bound the map by count, or both. A size bound requires an explicit eviction policy, since the policy decides what a full cache forgets.
WithLoader turns the cache read-through, computing what a read misses and collapsing concurrent misses on one key into a single computation.
Index ¶
- Variables
- type Cache
- func (i *Cache[T]) Close() error
- func (i *Cache[T]) Delete(ctx context.Context, key string) error
- func (i *Cache[T]) DeleteByPrefix(ctx context.Context, prefix string) error
- func (i *Cache[T]) DeleteMany(ctx context.Context, keys []string) error
- func (i *Cache[T]) Flush(ctx context.Context) error
- func (i *Cache[T]) Get(ctx context.Context, key string) (*T, error)
- func (i *Cache[T]) GetMany(ctx context.Context, keys []string) (map[string]*T, error)
- func (i *Cache[T]) Ping(ctx context.Context) error
- func (i *Cache[T]) Set(ctx context.Context, key string, value *T, opts ...cache.WriteOption) error
- func (i *Cache[T]) SetIfPresent(ctx context.Context, key string, value *T, opts ...cache.WriteOption) error
- func (i *Cache[T]) SetMany(ctx context.Context, items map[string]*T, opts ...cache.WriteOption) error
- type EvictionPolicy
- type Loader
- type Option
- func WithJanitor(ctx context.Context, interval time.Duration) Option
- func WithLoader[T any](loader Loader[T]) Option
- func WithLogger(logger logging.Logger) Option
- func WithMaxEntries(maxEntries int, policy EvictionPolicy) Option
- func WithMetricsProvider(metricsProvider metrics.Provider) Option
- func WithTracerProvider(tracerProvider tracing.Provider) Option
Constants ¶
This section is empty.
Variables ¶
var ErrLoaderTypeMismatch = errors.New("loader type does not match cache type")
ErrLoaderTypeMismatch indicates WithLoader was given a loader for a type other than the one the cache was built for.
var ErrUnknownEvictionPolicy = errors.New("unknown eviction policy")
ErrUnknownEvictionPolicy indicates WithMaxEntries was given a policy that is not one of this package's constants. It is reported at construction rather than defaulted to one of them, because the policy decides which data a full cache loses, and a caller who typed a policy is owed the one they named.
Functions ¶
This section is empty.
Types ¶
type Cache ¶
type Cache[T any] struct { // contains filtered or unexported fields }
Cache is the in-memory cache.Cache implementation. It is exported, and returned by NewInMemoryCache, so a caller can depend on this cache rather than on the interface every provider shares: nothing here is unreachable without a network, nothing returns cache.ErrUnavailable, and Flush needs no namespace, so code built against this type need not carry the handling those possibilities force on code built against cache.Cache.
func NewInMemoryCache ¶
NewInMemoryCache builds an in-memory cache. Writes expire after defaultExpiry unless overridden per call with cache.WithExpiry; a non-positive defaultExpiry means entries never expire by default.
By default expired entries are evicted lazily, on the read that discovers them or when overwritten, and the map is not otherwise size-bounded. Pass WithJanitor to sweep them on a timer instead — see that option for when the lazy default is not enough — and WithMaxEntries to bound the map by count rather than only by expiry.
The cache answers only for what has been written to it unless it is given a WithLoader, which makes reads compute what they miss and collapses concurrent misses on a key into one computation.
func (*Cache[T]) Close ¶
Close stops the janitor, if one is running. Entries are left in place: the map is unreachable once the cache is, and Close is not an eviction.
It is safe to call more than once.
func (*Cache[T]) DeleteByPrefix ¶
DeleteByPrefix removes every entry whose key begins with prefix. The memory provider wholly owns its map, so an empty prefix is permitted and clears everything.
func (*Cache[T]) DeleteMany ¶
DeleteMany removes the given keys; keys that are absent are not an error.
func (*Cache[T]) Flush ¶
Flush removes every entry. The memory provider wholly owns its store, so no namespace is needed.
func (*Cache[T]) SetIfPresent ¶
func (i *Cache[T]) SetIfPresent(ctx context.Context, key string, value *T, opts ...cache.WriteOption) error
SetIfPresent overwrites key only if it currently holds a live entry.
Presence is judged under the same lock the write takes, so an entry that expires or is deleted concurrently cannot be resurrected: the check and the write are one critical section, which is the whole point of the method.
An expired-but-not-yet-swept entry counts as absent. The janitor and the read path are both lazy about eviction, so an entry's presence in the map is not the same as its being live, and a caller asking "is it still there" means the deadline, not the bookkeeping. It is left for the sweeper rather than evicted here: this is a write path, and counting a TTL loss discovered by a refused write would mix it in with the ones reads discover.
type EvictionPolicy ¶
type EvictionPolicy uint8
EvictionPolicy selects which entry a size-bounded cache drops when a write would take it past its bound.
The zero value is not a policy: WithMaxEntries takes one explicitly, so that bounding a cache is never separable from saying what it forgets.
const ( // EvictLeastRecentlyUsed drops the entry that has gone longest without // being read or written. It is what a cache in front of an expensive // computation usually wants: the working set stays resident and the tail // pays for the bound. // // Recording a read is a mutation, so a cache using this policy takes the // write lock on the read path — Get and GetMany stop being shared reads. // Under heavy concurrent hits on a small key set that is the difference // between readers running in parallel and readers queueing, which is the // case for preferring EvictOldestWritten. EvictLeastRecentlyUsed EvictionPolicy = iota + 1 // EvictOldestWritten drops the entry written longest ago, however often it // has been read since. Overwriting a key counts as writing it, so a value // that is refreshed stays put; a value that is only read does not. // // It keeps the read path a shared read, which is what recommends it for a // memo whose entries are refreshed on a timer rather than promoted by use. EvictOldestWritten )
func ParseEvictionPolicy ¶
func ParseEvictionPolicy(name string) (EvictionPolicy, error)
ParseEvictionPolicy resolves a policy's configuration name, case- and space-insensitively, and accepts the shorthands "lru" and "fifo". It exists for configuration-driven wiring, which can only carry a string; an unrecognized name is an error rather than a default, for the reason ErrUnknownEvictionPolicy gives.
func (EvictionPolicy) String ¶
func (p EvictionPolicy) String() string
String returns the policy's configuration name, and is what ParseEvictionPolicy accepts. An undefined policy renders as "unknown" rather than its number, so a message built from it reads the same as one built from a name.
type Loader ¶
Loader computes the value for a key the cache does not hold.
Returning cache.ErrNotFound means the key genuinely has no value; the caller's Get returns cache.ErrNotFound and nothing is stored, so absence is not cached. Returning (nil, nil) is a value — a nil *T is stored and served like any other, which is how a loader says "the answer is nothing" rather than "there is no answer". Any other error is returned to the caller as-is and, again, nothing is stored: a failed computation must not become a cached one.
type Option ¶
type Option func(*options)
Option configures an in-memory cache at construction. Options are applied in the order given, and a nil option is ignored.
It carries no type parameter even though the cache does: almost nothing an Option sets depends on the cached type, and Go cannot infer a type argument from a call's result type — so an Option[T] would force every call site to spell the cached type out by hand — WithLogger[MyValue](l) — forever. WithLoader is the one option that depends on the cached type; it stays generic but still needs no annotation, because T is inferable from the loader it is handed.
func WithJanitor ¶
WithJanitor starts a background sweep that removes expired entries every interval, rather than waiting for a read to discover them.
Without it an entry is only dropped when something reads that exact key or overwrites it. That is fine for a hot cache, where the keys worth evicting are the keys being read anyway. It is not fine for a workload that writes many keys and rarely reads them back — an idempotency-key store, a long-TTL request cache — because nothing ever triggers the lazy path and the map grows without bound. The rule of thumb: enable it whenever the expiry is long relative to how often a given key is read.
A sweep bounds the map in time, not in size: it reclaims entries once they expire, and reclaims nothing before then. A keyspace that can produce entries faster than they expire — or one whose entries never expire at all — needs WithMaxEntries as well, which is the only thing here that puts a ceiling on the map.
The sweep stops on Close, and also when ctx is done — whichever happens first. Passing context.Background() and relying on Close is the ordinary shape; a cancellable ctx is for tying the sweep to something narrower than the cache's own lifetime. A nil ctx or a non-positive interval starts no goroutine at all.
func WithLoader ¶
WithLoader makes the cache read through to loader: a Get that misses runs the loader, stores what it returns, and returns it, and concurrent misses on one key produce a single loader call whose result they all share.
Without it the cache only answers for what has been written to it, so every caller that misses computes its own value. For a memo in front of an expensive computation — an aggregate query, a remote fetch — that turns each expiry into a thundering herd, one computation per concurrent reader, which is the load the memo was added to remove.
GetMany reads through too, loading its missing keys concurrently. The concurrency is the batch's, so a caller handing it a thousand missing keys gets a thousand loads in flight; batch size is the caller's to choose, and a loader that must not be called that widely should limit itself.
The loader runs on a context detached from the cancellation of whichever caller happened to start it, since its result belongs to every caller waiting on it and one of them giving up must not cancel the others' computation. Values and the surrounding trace carry over; deadlines do not, so a loader that needs a time limit has to impose its own. A caller whose own context ends first stops waiting and gets that context's error, while the load continues for the others.
T is inferred from the loader, so this needs no type argument:
memory.WithLoader(func(ctx context.Context, key string) (*Stats, error) {
return q.aggregate(ctx, key)
})
It must match the cache it configures. Because Option carries no type parameter, a loader for the wrong type cannot be rejected by the compiler; NewInMemoryCache returns ErrLoaderTypeMismatch instead, at construction.
func WithLogger ¶
WithLogger attaches a logger. An absent logger logs nowhere.
func WithMaxEntries ¶
func WithMaxEntries(maxEntries int, policy EvictionPolicy) Option
WithMaxEntries bounds the cache to maxEntries live entries, dropping one entry per policy whenever a write would take it past the bound.
Without it the map is bounded only by expiry: entries leave when a read discovers them expired, or when a janitor sweeps them, and nothing at all leaves a cache whose entries do not expire. That is fine for a keyspace whose cardinality is known — a fixed set of feature flags, one entry per tenant. It is not fine for a cache keyed by anything a caller can vary freely: a request fingerprint, a query's parameters, a rendered URL. There the map grows until something evicts it, and TTL only bounds the growth if entries arrive slower than they expire.
The bound is on entries, not bytes. This package cannot size an arbitrary T, so a caller whose values vary wildly in size should pick maxEntries against the largest, not the average.
A non-positive maxEntries leaves the cache unbounded, so a configured bound can be turned off without changing the wiring. An undefined policy is rejected by the constructor with ErrUnknownEvictionPolicy.
func WithMetricsProvider ¶
WithMetricsProvider attaches a metrics provider for the cache's hit, miss, set, delete, eviction, and load counters and its latency histogram. An absent provider records nothing.
func WithTracerProvider ¶
WithTracerProvider attaches a tracer provider, enabling spans on every cache operation. An absent tracer provider traces nowhere.
Source Files