cache

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 13, 2026 License: Apache-2.0 Imports: 33 Imported by: 0

Documentation

Overview

Package cache provides a framework for implementing and registering different cache backends.

Index

Constants

View Source 
const (
	// RangeFull indicates the full object should be served.
	RangeFull = client.RangeFull
	// RangePartial indicates a single satisfiable byte range.
	RangePartial = client.RangePartial
	// RangeNotSatisfiable indicates the range lies outside the object.
	RangeNotSatisfiable = client.RangeNotSatisfiable
)
View Source 
const ETagKey = client.ETagKey

ETagKey is the HTTP header key used to store the ETag.

Variables

View Source 
var ErrNotFound = errors.New("cache backend not found")

ErrNotFound is returned when a cache backend is not found.

View Source 
var ErrNotModified = client.ErrNotModified

ErrNotModified is returned by Open/Stat when an If-None-Match precondition is satisfied.

View Source 
var ErrPreconditionFailed = client.ErrPreconditionFailed

ErrPreconditionFailed is returned by Open/Stat when an If-Match precondition is not met.

View Source 
var ErrRangeNotSatisfiable = client.ErrRangeNotSatisfiable

ErrRangeNotSatisfiable is returned by Open when a requested Range lies outside the object.

View Source 
var ErrStatsUnavailable = client.ErrStatsUnavailable

ErrStatsUnavailable is returned when a cache backend cannot provide statistics.

Functions

func Fetch 

func Fetch(client *http.Client, r *http.Request, c Cache) (*http.Response, error)

Fetch retrieves a response from cache or fetches from the request URL and caches it. The response is streamed without buffering. Returns HTTPError for semantic errors. The caller must close the response body.

func FetchDirect 

func FetchDirect(client *http.Client, r *http.Request, c Cache, key Key) (*http.Response, error)

FetchDirect fetches and caches the given URL without checking the cache first. The response is streamed without buffering. Returns HTTPError for semantic errors. The caller must close the response body.

func FormatETag added in v0.3.0 

func FormatETag(etag string) (string, error)

FormatETag formats a raw ETag value as a strong HTTP ETag.

func NewHTTPClient 

func NewHTTPClient(headerFunc HeaderFunc) *http.Client

NewHTTPClient creates an *http.Client that attaches headerFunc headers to every outgoing request.

func ParallelGet 

func ParallelGet(ctx context.Context, c Cache, key Key, dst io.WriterAt, chunkSize int64, concurrency int) error

ParallelGet downloads an object from any Range-capable Cache into dst, fetching it in chunkSize-byte chunks concurrently. It delegates to client.ParallelGet; see that function for the full semantics.

func RawETagFromHeader added in v0.3.0 

func RawETagFromHeader(etag string) (string, error)

RawETagFromHeader extracts a raw ETag value from a strong HTTP ETag header.

func Register 

func Register[Config any, C Cache](r *Registry, id, description string, factory Factory[Config, C])

Register a cache factory function.

func RegisterDisk 

func RegisterDisk(r *Registry)

RegisterDisk cache with the given registry.

func RegisterMemory 

func RegisterMemory(r *Registry)

func RegisterS3 

func RegisterS3(r *Registry, clientProvider s3client.ClientProvider)

RegisterS3 registers the S3 cache backend. The clientProvider supplies the shared minio client constructed from the global s3 config block.

func ValidateNamespace 

func ValidateNamespace(name string) error

ValidateNamespace checks that a namespace name is valid.

func ValidateRawETag added in v0.3.0 

func ValidateRawETag(etag string) error

ValidateRawETag verifies that etag is an unquoted cache ETag value.

func WriteFunc 

func WriteFunc(ctx context.Context, c Cache, key Key, headers http.Header, ttl time.Duration, fn func(w io.Writer) error, opts ...Option) error

WriteFunc is a convenience wrapper around Cache.Create that handles aborting the write on error. The provided function receives a writer; if it returns an error the cache entry is discarded. On success the entry is committed.

Types

type Cache 

type Cache interface {
	// String describes the Cache implementation.
	String() string
	// Namespace creates a namespaced view of this cache.
	// All operations on the returned cache will use the given namespace prefix.
	Namespace(namespace Namespace) Cache
	// Stat returns the headers of an existing object in the cache.
	//
	// Expired files MUST not be returned.
	// Must return os.ErrNotExist if the file does not exist.
	//
	// Conditional opts are evaluated against the stored ETag: a satisfied
	// If-None-Match returns ErrNotModified (with headers); a failed If-Match
	// returns ErrPreconditionFailed.
	Stat(ctx context.Context, key Key, opts ...Option) (http.Header, error)
	// Open an existing file in the cache.
	//
	// Expired files MUST NOT be returned.
	// The returned headers MUST include a Last-Modified header.
	// Must return os.ErrNotExist if the file does not exist.
	//
	// Conditional opts are evaluated against the stored ETag: a satisfied
	// If-None-Match returns ErrNotModified (with headers, no body); a failed
	// If-Match returns ErrPreconditionFailed.
	//
	// A Range opt requests a single byte range: on success the returned
	// headers carry Content-Range and a Content-Length of the range, and the
	// reader yields only those bytes. A range outside the object returns
	// ErrRangeNotSatisfiable (with headers carrying Content-Range: bytes */N).
	Open(ctx context.Context, key Key, opts ...Option) (io.ReadCloser, http.Header, error)
	// Create a new file in the cache.
	//
	// If "ttl" is zero, a maximum TTL MUST be used by the implementation.
	//
	// The file MUST NOT be available for read until completely written and closed.
	//
	// If the context is cancelled the object MUST NOT be made available in the cache.
	Create(ctx context.Context, key Key, headers http.Header, ttl time.Duration, opts ...Option) (Writer, error)
	// Delete a file from the cache.
	//
	// MUST be atomic.
	Delete(ctx context.Context, key Key) error
	// Invalidate evicts a stale local copy from the cache.
	//
	// Unlike Delete, invalidation MUST NOT remove authoritative shared storage.
	// Missing objects are treated as successfully invalidated.
	Invalidate(ctx context.Context, key Key) error
	// Stats returns health and usage statistics for the cache.
	Stats(ctx context.Context) (Stats, error)
	// ListNamespaces returns all unique namespaces in the cache in order.
	ListNamespaces(ctx context.Context) ([]string, error)
	// Close the Cache.
	Close() error
}

A Cache knows how to retrieve, create and delete objects from a cache.

Objects in the cache are not guaranteed to persist and implementations may delete them at any time.

func MaybeNewTiered 

func MaybeNewTiered(ctx context.Context, caches []Cache, metadata *metadatadb.Store) Cache

MaybeNewTiered creates a Tiered cache from one or more caches.

If no caches are passed it will panic.

func NoOpCache 

func NoOpCache() Cache

NoOpCache returns a cache that doesn't cache anything. All Open() calls return os.ErrNotExist (cache miss). All Create() calls return a writer that discards data.

type Disk 

type Disk struct {
	// contains filtered or unexported fields
}

func NewDisk 

func NewDisk(ctx context.Context, config DiskConfig) (*Disk, error)

NewDisk creates a new disk-based cache instance.

config.Root MUST be set.

This Cache implementation stores cache entries under a directory. If total usage exceeds the limit, entries are evicted based on their last access time. TTLs are stored in a bbolt database. If an entry exceeds its TTL or the default, it is evicted. The implementation is safe for concurrent use within a single Go process.

func (*Disk) Close 

func (d *Disk) Close() error

func (*Disk) Create 

func (d *Disk) Create(ctx context.Context, key Key, headers http.Header, ttl time.Duration, opts ...Option) (Writer, error)

func (*Disk) Delete 

func (d *Disk) Delete(_ context.Context, key Key) error

func (*Disk) Invalidate added in v0.3.0 

func (d *Disk) Invalidate(ctx context.Context, key Key) error

func (*Disk) ListNamespaces 

func (d *Disk) ListNamespaces(_ context.Context) ([]string, error)

ListNamespaces returns all unique namespaces in the disk cache.

func (*Disk) Namespace 

func (d *Disk) Namespace(namespace Namespace) Cache

Namespace creates a namespaced view of the disk cache.

func (*Disk) Open 

func (d *Disk) Open(ctx context.Context, key Key, opts ...Option) (io.ReadCloser, http.Header, error)

func (*Disk) Size 

func (d *Disk) Size() int64

func (*Disk) Stat 

func (d *Disk) Stat(ctx context.Context, key Key, opts ...Option) (http.Header, error)

func (*Disk) Stats 

func (d *Disk) Stats(_ context.Context) (Stats, error)

func (*Disk) String 

func (d *Disk) String() string

type DiskConfig 

type DiskConfig struct {
	Root          string        `hcl:"root,optional" help:"Root directory for the disk storage." default:"${CACHEW_STATE}/cache"`
	LimitMB       int           `hcl:"limit-mb,optional" help:"Maximum size of the disk cache in megabytes (defaults to 10GB)." default:"10240"`
	MaxTTL        time.Duration `hcl:"max-ttl,optional" help:"Maximum time-to-live for entries in the disk cache (defaults to 1 hour)." default:"1h"`
	EvictInterval time.Duration `hcl:"evict-interval,optional" help:"Interval at which to check files for eviction (defaults to 1 minute)." default:"1m"`
}

type Factory 

type Factory[Config any, C Cache] func(ctx context.Context, config Config) (C, error)

Factory is a function that creates a new cache instance from the given hcl-tagged configuration struct.

type HeaderFunc 

type HeaderFunc = client.HeaderFunc

HeaderFunc returns headers to attach to each outgoing request.

type Key 

type Key = client.Key

Key represents a unique identifier for a cached object.

func NewKey 

func NewKey(s string) Key

NewKey returns the SHA256 of s.

func ParseKey 

func ParseKey(key string) (Key, error)

ParseKey from its hex-encoded string form.

type Memory 

type Memory struct {
	// contains filtered or unexported fields
}

func NewMemory 

func NewMemory(ctx context.Context, config MemoryConfig) (*Memory, error)

func (*Memory) Close 

func (m *Memory) Close() error

func (*Memory) Create 

func (m *Memory) Create(ctx context.Context, key Key, headers http.Header, ttl time.Duration, opts ...Option) (Writer, error)

func (*Memory) Delete 

func (m *Memory) Delete(_ context.Context, key Key) error

func (*Memory) Invalidate added in v0.3.0 

func (m *Memory) Invalidate(ctx context.Context, key Key) error

func (*Memory) ListNamespaces 

func (m *Memory) ListNamespaces(_ context.Context) ([]string, error)

ListNamespaces returns all unique namespaces in the memory cache.

func (*Memory) Namespace 

func (m *Memory) Namespace(namespace Namespace) Cache

Namespace creates a namespaced view of the memory cache.

func (*Memory) Open 

func (m *Memory) Open(_ context.Context, key Key, opts ...Option) (io.ReadCloser, http.Header, error)

func (*Memory) Stat 

func (m *Memory) Stat(_ context.Context, key Key, opts ...Option) (http.Header, error)

func (*Memory) Stats 

func (m *Memory) Stats(_ context.Context) (Stats, error)

func (*Memory) String 

func (m *Memory) String() string

type MemoryConfig 

type MemoryConfig struct {
	LimitMB int           `hcl:"limit-mb,optional" help:"Maximum size of the disk cache in megabytes (defaults to 1GB)." default:"1024"`
	MaxTTL  time.Duration `hcl:"max-ttl,optional" help:"Maximum time-to-live for entries in the disk cache (defaults to 1 hour)." default:"1h"`
}

type Namespace 

type Namespace = client.Namespace

Namespace identifies a logical partition within a cache or metadata store.

func ParseNamespace 

func ParseNamespace(name string) (Namespace, error)

ParseNamespace validates and returns a Namespace from a plain string.

type Option 

type Option = client.RequestOption

Option configures conditional parameters on a cache Open or Stat.

func IfMatch 

func IfMatch(etag string) Option

IfMatch sets the If-Match precondition. Open/Stat return ErrPreconditionFailed if the stored ETag does not match.

func IfNoneMatch 

func IfNoneMatch(etag string) Option

IfNoneMatch sets the If-None-Match precondition. Open/Stat return ErrNotModified when the stored ETag matches.

func IfRange 

func IfRange(etag string) Option

IfRange gates Range on the stored ETag: the range is only applied when etag matches, otherwise the full object is returned.

func Range 

func Range(start, end int64) Option

Range requests a single half-open byte range [start, end) from Open. A negative end means "to the end of the object". The returned headers carry a Content-Range; Open returns ErrRangeNotSatisfiable if the range lies outside the object. Stat ignores Range.

func WithETag added in v0.3.0 

func WithETag(etag string) Option

WithETag sets the raw ETag value to store on Create.

type RangeOutcome 

type RangeOutcome = client.RangeOutcome

RangeOutcome classifies how a Range request should be answered, as returned by RequestOptions.ResolveRange.

type Registry 

type Registry struct {
	// contains filtered or unexported fields
}

func NewRegistry 

func NewRegistry() *Registry

func (*Registry) Create 

func (r *Registry) Create(ctx context.Context, name string, config *hcl.Block, vars map[string]string) (Cache, error)

Create a new cache instance from the given name and configuration.

Will return "ErrNotFound" if the cache backend is not found.

func (*Registry) Exists 

func (r *Registry) Exists(name string) bool

func (*Registry) Schema 

func (r *Registry) Schema() *hcl.AST

Schema returns the schema for all registered cache backends. Each entry is wrapped as a "cache <name> { ... }" block.

type Remote 

type Remote struct {
	// contains filtered or unexported fields
}

Remote implements Cache as a client for the remote cache server, wrapping a *client.Client.

func NewRemote 

func NewRemote(baseURL string, headerFunc HeaderFunc) *Remote

NewRemote creates a new remote cache client. If headerFunc is non-nil, its returned headers are added to every outgoing request.

func (*Remote) Close 

func (r *Remote) Close() error

func (*Remote) Create 

func (r *Remote) Create(ctx context.Context, key Key, headers http.Header, ttl time.Duration, opts ...Option) (Writer, error)

func (*Remote) Delete 

func (r *Remote) Delete(ctx context.Context, key Key) error

func (*Remote) Invalidate added in v0.3.0 

func (r *Remote) Invalidate(ctx context.Context, key Key) error

func (*Remote) ListNamespaces 

func (r *Remote) ListNamespaces(ctx context.Context) ([]string, error)

func (*Remote) Namespace 

func (r *Remote) Namespace(namespace Namespace) Cache

func (*Remote) Open 

func (r *Remote) Open(ctx context.Context, key Key, opts ...Option) (io.ReadCloser, http.Header, error)

func (*Remote) Stat 

func (r *Remote) Stat(ctx context.Context, key Key, opts ...Option) (http.Header, error)

func (*Remote) Stats 

func (r *Remote) Stats(ctx context.Context) (Stats, error)

func (*Remote) String 

func (r *Remote) String() string

type RequestOptions 

type RequestOptions = client.RequestOptions

RequestOptions is the resolved set of conditional and range parameters for an Open or Stat.

func NewRequestOptions 

func NewRequestOptions(opts ...Option) RequestOptions

NewRequestOptions applies opts and returns the resulting RequestOptions.

type S3 

type S3 struct {
	// contains filtered or unexported fields
}

func NewS3 

func NewS3(ctx context.Context, config S3Config, clientProvider s3client.ClientProvider) (*S3, error)

NewS3 creates a new S3-based cache instance.

The minio client is obtained from the shared clientProvider, which is constructed once from the global s3 configuration block. Cache-specific settings (bucket, TTL, upload tuning) come from the per-instance S3Config.

This Cache implementation stores cache entries in an S3-compatible object storage service. Metadata (headers and expiration time) are stored as object user metadata. The implementation uses the lightweight minio-go SDK to reduce overhead compared to the AWS SDK.

func (*S3) Close 

func (s *S3) Close() error

func (*S3) Create 

func (s *S3) Create(ctx context.Context, key Key, headers http.Header, ttl time.Duration, opts ...Option) (Writer, error)

func (*S3) Delete 

func (s *S3) Delete(ctx context.Context, key Key) error

func (*S3) Invalidate added in v0.3.0 

func (s *S3) Invalidate(ctx context.Context, key Key) error

func (*S3) ListNamespaces 

func (s *S3) ListNamespaces(_ context.Context) ([]string, error)

ListNamespaces returns all unique namespaces in the S3 cache. Not implemented for S3 - would require listing all objects.

func (*S3) Namespace 

func (s *S3) Namespace(namespace Namespace) Cache

Namespace creates a namespaced view of the S3 cache.

func (*S3) Open 

func (s *S3) Open(ctx context.Context, key Key, opts ...Option) (io.ReadCloser, http.Header, error)

func (*S3) Stat 

func (s *S3) Stat(ctx context.Context, key Key, opts ...Option) (http.Header, error)

func (*S3) Stats 

func (s *S3) Stats(_ context.Context) (Stats, error)

func (*S3) String 

func (s *S3) String() string

type S3Config 

type S3Config struct {
	Bucket              string        `hcl:"bucket" help:"S3 bucket name."`
	MaxTTL              time.Duration `hcl:"max-ttl,optional" help:"Maximum time-to-live for entries in the S3 cache (defaults to 1 hour)." default:"1h"`
	UploadConcurrency   uint          `` /* 144-byte string literal not displayed */
	UploadPartSizeMB    uint          `` /* 141-byte string literal not displayed */
	DownloadConcurrency uint          `hcl:"download-concurrency,optional" help:"Number of concurrent range-GET workers for downloads (defaults to 8)." default:"8"`
	DownloadPartSizeMB  uint          `` /* 129-byte string literal not displayed */
}

S3Config contains cache-specific S3 settings. Connection parameters (endpoint, region, SSL, credentials) are provided by the global s3client.Config block and shared via the s3client.ClientProvider.

type Stats 

type Stats = client.Stats

Stats contains health and usage statistics for a cache.

type Tiered 

type Tiered struct {
	// contains filtered or unexported fields
}

The Tiered cache combines multiple caches.

It is not directly selectable from configuration, but instead is automatically used if multiple caches are configured.

func (Tiered) Close 

func (t Tiered) Close() error

Close all underlying caches.

func (Tiered) Create 

func (t Tiered) Create(ctx context.Context, key Key, headers http.Header, ttl time.Duration, opts ...Option) (Writer, error)

Create a new object. All underlying caches will be written to in sequence.

func (Tiered) Delete 

func (t Tiered) Delete(ctx context.Context, key Key) error

Delete from all underlying caches. All errors are returned.

func (Tiered) Invalidate added in v0.3.0 

func (t Tiered) Invalidate(ctx context.Context, key Key) error

Invalidate evicts stale local copies from every non-authoritative tier. The final tier is authoritative by construction, so invalidation leaves it intact even if that backend's own Invalidate method would remove the object.

func (Tiered) ListNamespaces 

func (t Tiered) ListNamespaces(ctx context.Context) ([]string, error)

ListNamespaces returns unique namespaces from all underlying caches.

func (Tiered) Namespace 

func (t Tiered) Namespace(namespace Namespace) Cache

Namespace creates a namespaced view of the tiered cache. All underlying caches are also namespaced.

func (Tiered) Open 

func (t Tiered) Open(ctx context.Context, key Key, opts ...Option) (io.ReadCloser, http.Header, error)

Open returns a reader from the first cache that succeeds. When a higher tier hits but lower tiers missed, the returned reader transparently backfills the lowest tier as the caller reads, so that subsequent Opens are served locally.

A tier that holds a different version than the request's validators name — a failed If-Match, or an If-Range miss — is not definitive: deeper tiers are consulted for the named version, so a replica whose local tier has diverged can still satisfy a pinned request from a shared tier. When no tier holds it, the first tier's outcome stands: the full representation for an If-Range miss (per RFC 9110), ErrPreconditionFailed for a failed If-Match. A tier that errored while being probed takes precedence over both, so outages are not misreported as missing versions.

If all caches fail, all errors are returned.

func (Tiered) Stat 

func (t Tiered) Stat(ctx context.Context, key Key, opts ...Option) (http.Header, error)

Stat returns headers from the first cache that succeeds.

A tier that fails an If-Match precondition holds a different version of the object, not a definitive answer: deeper tiers are consulted for the version the validator names, and ErrPreconditionFailed is only returned when none holds it. A tier that errored while being probed takes precedence, so outages are not misreported as missing versions.

If all caches fail, all errors are returned.

func (Tiered) Stats 

func (t Tiered) Stats(ctx context.Context) (Stats, error)

func (Tiered) String 

func (t Tiered) String() string

type Writer 

type Writer = client.CacheWriter

Writer extends io.WriteCloser with abort semantics for cache writes.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.