cache

package
v0.18.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 10, 2026 License: MIT Imports: 20 Imported by: 0

Documentation

Overview

Package cache is an HTTP response-cache middleware with a pluggable storage backend. It implements a CDN-style honor-origin policy: it caches a response only when the origin opts in via explicit Cache-Control/Expires freshness; refuses private/no-store/no-cache, Set-Cookie, and Vary: *; honors Vary (keying per varied request header); GET/HEAD only; and ignores the client's request Cache-Control so a client can't bust the shared cache. Concurrent misses for one key collapse into a single origin fetch. It is fail-static: any storage error degrades to a cache miss, never an error to the client. The response is tagged X-Cache: HIT|MISS.

Two storage backends ship: an in-memory one (NewMemory, bodies held in RAM, lost on restart) and a disk-backed one (NewDisk, survives restarts, streams bodies to disk so it isn't bounded by RSS). Both bound their total size with LRU eviction and a per-object cap. Plug either (or your own Storage) into New.

store, _ := cache.NewDisk("/var/cache/app", 1<<30) // 1 GiB on disk
m := cache.New(store, cache.Options{MaxFileSize: 8 << 20})
srv.Use(m) // mount it ahead of the upstream/handler it should cache

Only origin-opted-in (public, fresh) content is cached, so per-user responses must be marked uncacheable by the origin. As a shared cache it additionally follows RFC 9111 §3.5: a response to a request bearing an Authorization header is cached only when the origin explicitly opts in via public, s-maxage, or must-revalidate.

The key is the request's host+method+scheme+uri plus the origin-declared Vary headers; it does not include request headers the origin reflects but doesn't declare in Vary (e.g. X-Forwarded-Host into a Location), so — as with any honor-origin shared cache — such a response can be poisoned. Use Options.Cacheable to exclude untrusted requests or paths from the cache.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func LogResult added in v0.18.0

func LogResult(r *http.Request, info ResultInfo)

LogResult is a ResultFunc that records the cache outcome on the request's structured-logger record as the field "cacheStatus" (HIT, MISS, STALE, STALE_ERROR, or BYPASS), so it appears in access logs alongside the upstream, status, and timing fields. It is a no-op when no logger middleware is mounted ahead of the cache (logger.Set ignores a request with no record).

Wire it via Options.OnResult, alone or composed with prom.Cache:

cache.New(store, cache.Options{OnResult: cache.LogResult})
Example

Make the cache observable: record the per-request outcome (HIT/MISS/STALE/ STALE_ERROR/BYPASS) as a "cacheStatus" field in the structured access log. Pair with prom.Cache() for Prometheus metrics (see that function's example).

package main

import (
	"github.com/moonrhythm/parapet/pkg/cache"
)

func main() {
	cache.New(cache.NewMemory(256<<20), cache.Options{
		OnResult: cache.LogResult,
	})
}

Types

type Cache

type Cache struct {
	// contains filtered or unexported fields
}

Cache is the HTTP response-cache middleware. It implements parapet.Middleware (ServeHandler). Construct with New, giving it a Storage backend.

func New

func New(storage Storage, opts Options) *Cache

New builds the cache middleware over the given Storage backend (memory or disk).

Example

Mount a disk-backed response cache ahead of the upstream whose responses it should cache.

package main

import (
	"log"

	"github.com/moonrhythm/parapet"
	"github.com/moonrhythm/parapet/pkg/cache"
)

func main() {
	// 1 GiB on disk, surviving restarts; use cache.NewMemory(size) for an in-memory
	// cache held in RAM instead.
	store, err := cache.NewDisk("/var/cache/app", 1<<30)
	if err != nil {
		log.Fatal(err)
	}

	s := parapet.New()
	s.Use(cache.New(store, cache.Options{
		MaxFileSize:  8 << 20, // don't cache bodies larger than 8 MiB
		DecoupleFill: true,    // a slow client won't stall waiting followers
	}))
	// s.Use(upstream.SingleHost(...)) — the handler whose responses get cached.
}

func (*Cache) ServeHandler

func (c *Cache) ServeHandler(next http.Handler) http.Handler

ServeHandler implements parapet.Middleware: it wraps next (the upstream/handler whose responses are cached). A hit short-circuits next; a miss fetches via next and stores.

type DiskStorage

type DiskStorage struct {
	// contains filtered or unexported fields
}

DiskStorage is a disk-backed cache backend. Entries survive restarts and the body is streamed to disk, so the cache isn't bounded by RSS. The total on-disk byte cap is held by an in-memory LRU, re-seeded from disk by a background startup scan that also reaps orphans, torn writes, and expired entries. Safe for concurrent use by a single Cache (see Storage).

Layout (sharded by the first 2 hex chars of the key):

<dir>/<aa>/<key>.body   response body bytes
<dir>/<aa>/<key>.meta   JSON sidecar (written last)
<dir>/tmp/<key>.<seq>   in-progress writes, atomically renamed on commit

func NewDisk

func NewDisk(dir string, maxSize int64) (*DiskStorage, error)

NewDisk creates (or opens) a disk storage rooted at dir, bounded to maxSize total body bytes. It starts a background scan that re-seeds the byte cap from surviving entries and reaps orphans/expired files off the serving path — the cap simply lags until the scan completes. Returns an error only if the dir can't be initialized.

func (*DiskStorage) Delete

func (s *DiskStorage) Delete(key string)

Delete removes the entry under key (meta first, so a half-deleted entry reads as a clean miss rather than a torn read) and its LRU accounting.

func (*DiskStorage) Get

func (s *DiskStorage) Get(key string) (Meta, []byte, bool)

Get reads the entry under key, touching its LRU recency on a hit. ok=false on any miss / corruption / torn read (fail-static — treated as a cache miss). The body length is checked against meta.Size so a reader can never serve an old meta paired with a concurrently-rewritten body (the two files are read non-atomically; this guards the framing-corruption case).

func (*DiskStorage) Range added in v0.17.1

func (s *DiskStorage) Range(fn func(key string, m Meta) bool)

Range walks the shard dirs reading each entry's .meta sidecar and calls fn(key, Meta), stopping early if fn returns false. It holds no lock, so fn may Delete the entry it is visiting (the keys come from a directory snapshot taken before fn runs). Unreadable/corrupt sidecars are skipped. For maintenance only, off the serving path.

func (*DiskStorage) Writer

func (s *DiskStorage) Writer(key string) (EntryWriter, error)

Writer streams a new entry's body to a temp file; Commit fsyncs + renames it into place (body first, meta last) and admits it to the byte cap; Abort discards the temp file. Returns an error if the temp file can't be created.

type EntryWriter

type EntryWriter interface {
	io.Writer
	// Commit persists the streamed body with meta and admits it to the byte cap
	// (evicting LRU victims). A failure is fail-static: the entry is not cached.
	Commit(meta Meta) error
	// Abort discards the streamed body (e.g. truncated/over-cap/panicked fill).
	Abort()
}

EntryWriter streams one cached body and finalizes it. Exactly one of Commit or Abort must be called; after either, the writer is spent. Abort after Commit (or vice versa) is a no-op. Backends admit the entry to their capacity bound (LRU) inside Commit.

type MemoryStorage

type MemoryStorage struct {
	// contains filtered or unexported fields
}

MemoryStorage is an in-memory cache backend: bodies are held in RAM and lost on restart. Total size is bounded by LRU eviction (the cap passed to NewMemory) plus the middleware's per-object cap. Safe for concurrent use. During a fill the whole body is buffered in RAM (and retained if the response is cached), so peak transient memory is up to MaxFileSize per concurrent miss, independent of the byte cap.

func NewMemory

func NewMemory(maxSize int64) *MemoryStorage

NewMemory creates an in-memory storage bounded to maxSize total body bytes.

func (*MemoryStorage) Delete

func (s *MemoryStorage) Delete(key string)

Delete removes the entry under key.

func (*MemoryStorage) Get

func (s *MemoryStorage) Get(key string) (Meta, []byte, bool)

Get returns the entry under key, touching its LRU recency on a hit. The Meta is deep-copied so a caller (e.g. the InvalidatedAfter hook) can't mutate the live stored entry; the body is returned by reference and must not be mutated (see Storage). This matches the disk backend, which returns a fresh Meta per call.

func (*MemoryStorage) Range added in v0.17.1

func (s *MemoryStorage) Range(fn func(key string, m Meta) bool)

Range snapshots the current (key, Meta) pairs under the read lock, then calls fn for each WITHOUT holding the lock — so fn may Delete entries (which takes the write lock) without deadlocking. The snapshot deep-copies each Meta (header map and Vary slice included), so a fn that mutates the Meta it receives can neither corrupt the live cached entry nor data-race concurrent serving of it.

func (*MemoryStorage) Writer

func (s *MemoryStorage) Writer(key string) (EntryWriter, error)

Writer returns a buffer-backed writer; Commit stores it (bodies are in RAM either way for this backend).

type Meta

type Meta struct {
	Header     http.Header `json:"header"`
	PrimaryHex string      `json:"primary"`        // primary key hash (host+method+scheme+uri)
	Host       string      `json:"host,omitempty"` // normalized host (lowercased, port-stripped); for out-of-band Range maintenance
	URI        string      `json:"uri,omitempty"`  // request-uri (path+query); for out-of-band Range maintenance
	Vary       []string    `json:"vary"`           // lowercased Vary header names
	Tags       []string    `json:"tags,omitempty"` // surrogate keys from the response Cache-Tag header; for out-of-band tag-scoped Range maintenance
	Created    int64       `json:"created"`        // unix nanos
	FreshUntil int64       `json:"fresh"`          // unix nanos; entry is stale after this
	// StaleWhileRevalidate and StaleIfError are RFC 5861 windows in nanoseconds
	// PAST FreshUntil: within StaleWhileRevalidate a stale entry may be served
	// while it is revalidated in the background; within StaleIfError a stale entry
	// may be served when revalidation fails. 0 means the window is not offered.
	StaleWhileRevalidate int64 `json:"swr,omitempty"`
	StaleIfError         int64 `json:"sie,omitempty"`
	Size                 int64 `json:"size"` // body bytes (== eviction weight)
	Status               int   `json:"status"`
}

Meta is the stored metadata for a cached response. Backends persist it alongside the body (the disk backend marshals it to JSON).

type Options

type Options struct {
	// InvalidatedAfter, when non-nil, is consulted on every cache hit to support
	// out-of-band invalidation (cache purge). It receives the request and the
	// stored entry's Meta and returns an invalidation epoch in unix nanos: a hit
	// whose Meta.Created is <= the returned epoch is treated as stale (reaped and
	// served as a miss), exactly like a passed FreshUntil. Return 0 (or any value
	// below the entry's Created) to keep the entry. It runs only on a hit, so it
	// costs nothing while the cache is idle; nil disables the check entirely (zero
	// overhead). The callee owns its own concurrency.
	InvalidatedAfter func(r *http.Request, m Meta) int64

	// Cacheable, when non-nil, is called for each GET/HEAD request; returning false
	// excludes the request from the cache entirely (served straight from the origin,
	// untagged), exactly as if it were uncacheable. Use it to restrict caching to
	// vetted paths, or to refuse requests carrying headers an origin might reflect
	// into the body without declaring them in Vary (an unkeyed-input poisoning
	// vector — see the package doc). nil caches everything otherwise cacheable.
	Cacheable func(r *http.Request) bool

	// Override, when non-nil, may return a forced caching policy that overrides the
	// origin's Cache-Control — letting you cache an origin that sends no (or
	// unwanted) cache headers. It is called on each GET/HEAD fill with the request
	// and the origin's response status and headers (before the body), so the
	// decision can key on anything in the request (host, path, extension) AND the
	// response (Content-Type, Content-Length, status). Return nil to honor the
	// origin for that response (the default). header is a copy of the response
	// headers, so reading it is safe and mutating it has no effect.
	//
	// The forced policy is baked into the stored entry only, so the served
	// Cache-Control stays the origin's and does not propagate downstream. How far
	// the override reaches is set per request by Override.Mode; safety refusals
	// (see Override) still apply.
	//
	// It runs on every fill, including background stale-while-revalidate refreshes,
	// and is re-evaluated against the fresh response each time (so a response-shape
	// change can change the policy). Don't key on wall-clock or random state.
	// Changing the hook does not re-policy already-stored entries.
	//
	// Forcing trusts you to target cacheable paths: the cache key ignores the
	// request's Cookie/Authorization, so do not force per-user paths (see the
	// per-mode safety notes on OverrideMode). Cacheable returning false takes
	// precedence — an excluded request is never forced.
	Override func(r *http.Request, status int, header http.Header) *Override

	// OnResult, when non-nil, is called once per request the cache serves, after it
	// decides how it was served, with the request and a ResultInfo (the outcome —
	// HIT/MISS/STALE/STALE_ERROR/BYPASS — and, on a fill, its duration). It makes
	// the cache observable without changing behavior: see prom.Cache for Prometheus
	// metrics and cache.LogResult for a structured-log field, or compose your own
	// ResultFunc. It runs synchronously on the foreground serving path only, never
	// from the background stale-while-revalidate goroutine; a panic unwinding from
	// the origin handler during a fill is not reported. nil disables it (zero
	// overhead). The callee owns its own concurrency.
	OnResult ResultFunc

	// MaxFileSize caps a cacheable response's body. A GET response larger than
	// this (by Content-Length, or mid-stream) is not cached but still served in
	// full. Defaults to 8 MiB when <= 0.
	MaxFileSize int64

	// LockTimeout bounds how long a concurrent miss (a "follower") waits for the
	// in-flight leader to populate the cache before giving up and fetching the
	// origin itself. The leader holds the fill lock for the WHOLE time it streams
	// the response to its own client and (on the disk backend) fsyncs the committed
	// entry, so a slow leader client or a saturated disk can hold followers for up
	// to this long before they fall back to the origin. Raise it to wait through a
	// slow fill (fewer origin fetches, higher follower latency); lower it to fail
	// fast (more origin fetches under load). Defaults to 2s when <= 0.
	LockTimeout time.Duration

	// RevalidateTimeout bounds a background stale-while-revalidate fetch (RFC
	// 5861): the detached request to the origin is cancelled after this long so a
	// hung origin can't pin the single-flight lock or leak a goroutine. It does
	// not apply to a normal (foreground) fill. Defaults to 30s when <= 0.
	RevalidateTimeout time.Duration

	// DefaultStaleWhileRevalidate and DefaultStaleIfError force RFC 5861 stale
	// serving for a cacheable response that does not carry the matching directive,
	// so an origin you don't control still gets stale-while-revalidate /
	// stale-if-error behavior. An explicit directive on the response wins; a
	// response marked must-revalidate / proxy-revalidate is never served stale,
	// regardless of these. Unlike injecting the directive with a headers
	// middleware, these stay private to this cache: the served Cache-Control is
	// the origin's, so the policy does not propagate to downstream clients/caches.
	// Zero (the default) forces nothing. Each is clamped to ~10y.
	DefaultStaleWhileRevalidate time.Duration
	DefaultStaleIfError         time.Duration

	// DecoupleFill, when true, stops a slow leader client (or a slow disk) from
	// holding the fill lock — and thus stalling waiting followers — while its response
	// streams to that client. It engages only when the fill is CONTENDED, i.e. at
	// least one follower is already blocked waiting for it (the stampede case
	// single-flight exists for); an uncontended fill has nothing to isolate and
	// streams to the client in lockstep with no added latency.
	//
	// When it engages, the cacheable body is read from the origin at origin speed —
	// the leader's client receives nothing until the fill is done — while it is
	// streamed to storage and also buffered in memory for the leader (so the leader's
	// response never depends on the stored entry, which may expire, be evicted, or be
	// purged). The entry is then committed, the fill lock released (waiting followers
	// immediately hit the cache), and only then is the leader's own client served from
	// the buffered body. This trades the leader's time-to-first-byte (it waits for the
	// whole fill) and an extra in-memory copy of the leader's body (bounded by
	// MaxFileSize, per in-flight decoupled fill) for follower isolation, and serves
	// the leader the sanitized response headers (hop-by-hop stripped, no Age) rather
	// than the raw origin headers. A handler that relies on incremental flushing is
	// unaffected (a Flush during a decoupled fill is ignored). When false (default),
	// the leader always streams in lockstep and holds the fill lock until that stream
	// and the commit finish (see LockTimeout).
	DecoupleFill bool
}

Options configures the cache middleware.

type Override added in v0.18.0

type Override struct {
	TTL                  time.Duration
	StaleWhileRevalidate time.Duration
	StaleIfError         time.Duration
	Mode                 OverrideMode
}

Override is a forced caching policy for one request, returned by Options.Override. TTL is the forced freshness lifetime and is required: a non-positive TTL means "do not force" (honor the origin). StaleWhileRevalidate and StaleIfError force the RFC 5861 windows. Mode selects which origin safety signals the force still respects.

Example

Force caching for an origin that sends no cache headers, deciding on both the request and the origin's response — here, only successful image responses.

package main

import (
	"net/http"
	"strings"
	"time"

	"github.com/moonrhythm/parapet/pkg/cache"
)

func main() {
	cache.New(cache.NewMemory(256<<20), cache.Options{
		Override: func(r *http.Request, status int, header http.Header) *cache.Override {
			if status == http.StatusOK && strings.HasPrefix(header.Get("Content-Type"), "image/") {
				return &cache.Override{TTL: time.Hour} // OverrideBalanced by default
			}
			return nil // everything else: respect the origin's Cache-Control
		},
	})
}

type OverrideMode added in v0.18.0

type OverrideMode int

OverrideMode selects how far an Override reaches over the origin's Cache-Control. The zero value is OverrideBalanced.

const (
	// OverrideBalanced forces freshness and overrides no-cache / max-age /
	// Expires, but still refuses a response that is unsafe to share: no-store,
	// private, Set-Cookie, Vary: *, a non-cacheable status, an oversize body, or
	// an Authorization-bearing request without a shared opt-in.
	//
	// It does NOT inspect the request's Cookie header, and the cache key ignores
	// Cookie. So a per-user response gated by a session cookie — with none of the
	// markers above (no Set-Cookie/private/no-store, no Authorization) — would be
	// force-cached and served to other users. Only target paths you know are not
	// per-user (scope the Override hook, or use Options.Cacheable). Good for static
	// assets.
	OverrideBalanced OverrideMode = iota

	// OverrideConservative only fills freshness when the origin declares none and
	// otherwise honors the origin's Cache-Control entirely (no-cache/no-store/
	// private and any explicit max-age are respected). Safest; does nothing for an
	// origin that already sends no-cache/no-store.
	OverrideConservative

	// OverrideAggressive overrides almost everything, including no-store, private,
	// and the Authorization gate. Only Set-Cookie, Vary: *, a non-cacheable status,
	// and an oversize body still refuse.
	//
	// DANGER: this can cause a CROSS-USER LEAK. Bypassing the Authorization gate
	// means a response to one user's authenticated request is stored under a key
	// that ignores Authorization (host+method+scheme+uri+declared Vary), so it is
	// then served to other — including unauthenticated — users. Use it only for
	// endpoints with no per-user or secret data, or where the origin sends
	// Vary: Authorization (which puts the credential in the key). Prefer
	// OverrideBalanced unless you are certain.
	OverrideAggressive
)

type Result added in v0.18.0

type Result string

Result is the cache outcome for one request, reported to Options.OnResult. Its string value matches the X-Cache response header where one is sent (HIT, MISS, STALE) and additionally names the two states X-Cache cannot distinguish: a stale-if-error fallback (vs a stale-while-revalidate serve, both "STALE" on the wire) and a bypass (which sends no X-Cache header at all).

const (
	// ResultHit: served from a fresh stored entry (X-Cache: HIT).
	ResultHit Result = "HIT"

	// ResultMiss: not served from a stored entry; the origin was contacted and, if
	// the response was cacheable, stored (X-Cache: MISS).
	ResultMiss Result = "MISS"

	// ResultStale: served a stale stored entry under RFC 5861
	// stale-while-revalidate and refreshed it in the background (X-Cache: STALE).
	ResultStale Result = "STALE"

	// ResultStaleError: served a stale stored entry under RFC 5861 stale-if-error
	// because revalidating with the origin failed (its X-Cache is also "STALE").
	ResultStaleError Result = "STALE_ERROR"

	// ResultBypass: the request was ineligible for caching — a non-cacheable method,
	// a protocol upgrade, a Range request, or Options.Cacheable returned false — and
	// was proxied straight to the origin. This path sends no X-Cache header, so the
	// hook is the only way to observe it.
	ResultBypass Result = "BYPASS"
)

type ResultFunc added in v0.18.0

type ResultFunc func(r *http.Request, info ResultInfo)

ResultFunc observes a cache outcome. Assign one to Options.OnResult to make the cache observable — see prom.Cache for Prometheus metrics and cache.LogResult for a structured-log field. It is invoked once per request the cache serves, synchronously on the foreground serving path, and never from the background stale-while-revalidate goroutine (which has no client request to attribute and must not touch request-scoped state such as the logger record). A panic propagating out of the origin handler during a fill is not reported (it unwinds past the hook); every request that returns normally is.

type ResultInfo added in v0.18.0

type ResultInfo struct {
	// Result is the cache decision (HIT, MISS, STALE, STALE_ERROR, BYPASS).
	Result Result

	// FillDuration is how long the foreground origin fetch took, set only when this
	// request actually contacted the origin on the serving path: a MISS fill and a
	// stale-if-error revalidation attempt. It is zero for a HIT, for a
	// stale-while-revalidate STALE (served from cache; the background refresh runs
	// detached and is never reported), and for a BYPASS.
	FillDuration time.Duration
}

ResultInfo carries the details of one request's cache outcome to a ResultFunc.

type Storage

type Storage interface {
	// Get returns the entry stored under key, or ok=false on a miss. A hit should
	// be counted as a recent use (LRU). The returned Meta (including its Header map
	// and Vary slice) must be independent of stored state — safe for the caller to
	// read or modify without affecting the cache; the returned body must not be
	// mutated. Any internal error is reported as a miss (fail-static).
	Get(key string) (meta Meta, body []byte, ok bool)

	// Writer begins storing an entry under key. The caller streams the body to the
	// returned EntryWriter and then calls Commit (to persist) or Abort (to
	// discard). It returns an error if a writer can't be opened (then the entry is
	// simply not cached). The disk backend streams to a temp file so the body
	// never has to be buffered whole in RAM.
	Writer(key string) (EntryWriter, error)

	// Delete removes the entry under key (e.g. when the middleware finds it stale).
	Delete(key string)

	// Range calls fn for each currently-stored entry (key + its Meta), stopping
	// early if fn returns false. It is for out-of-band maintenance — e.g. a purge
	// reaper that physically deletes entries matching an external predicate — not
	// the serving path. Iteration order is unspecified, and the snapshot is
	// best-effort: fn may observe an entry that is concurrently deleted, and an
	// entry written during the walk may or may not be visited. The Meta passed to
	// fn is independent of stored state (see Get), so fn may freely read or modify
	// it. fn MAY call Delete(key) on this storage (a backend must not hold a lock
	// across fn that Delete would need). fn MUST NOT call Writer.
	Range(fn func(key string, m Meta) bool)
}

Storage is the cache backend: where cached entries live and how total size is bounded. The middleware handles policy, keys, Vary, locking, and X-Cache; Storage only persists bytes and enforces capacity (LRU + per-object cap).

Implementations must be safe for concurrent use BY A SINGLE Cache instance. One backing store (e.g. a DiskStorage dir) must be owned by one Cache: concurrent same-key writes are otherwise unsynchronized. The middleware serializes same-key fills with its fill lock and only writes a key it has just missed, so within one Cache the store never sees a same-key Get racing a Set.

Directories

Path Synopsis
Package purge invalidates entries in a pkg/cache response cache.
Package purge invalidates entries in a pkg/cache response cache.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL