httpcache

package
v0.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 27, 2026 License: Apache-2.0, MIT Imports: 24 Imported by: 0

README

httpcache

httpcache provides an rfc7234 compliant golang http.Handler.

wercker status

GoDoc

Example

This example is from the included CLI, it runs a caching proxy on http://localhost:8080.

proxy := &httputil.ReverseProxy{
    Director: func(r *http.Request) {
    },
}

handler := httpcache.NewHandler(httpcache.NewMemoryCache(), proxy)
handler.Shared = true

log.Printf("proxy listening on http://%s", listen)
log.Fatal(http.ListenAndServe(listen, handler))

Implemented

  • All of rfc7234, except those listed below
  • Disk and Memory storage
  • Apache-like logging via httplog package

Todo

  • Offline operation
  • Size constraints on memory/disk cache and cache eviction
  • Correctly handle mixture of HTTP1.0 clients and 1.1 upstreams
  • More detail in Via header
  • Support for weak entities with If-Match and If-None-Match
  • Invalidation based on Content-Location and request method
  • Better handling of duplicate headers and CacheControl values

Caveats

  • Conditional requests are never cached, this includes Range requests

Testing

Tests are currently conducted via the test suite and verified via the CoAdvisor tool.

Reading List

Documentation

Index

Constants

View Source 
const (
	CacheHeader     = "X-Cache"
	ProxyDateHeader = "Proxy-Date"
)
View Source 
const (
	CacheControlHeader = "Cache-Control"
)
View Source 
const DefaultCacheTTL = 7 * 24 * time.Hour

DefaultCacheTTL is the default TTL for cached items (7 days)

View Source 
const DefaultCleanupInterval = 1 * time.Hour

DefaultCleanupInterval is the default interval for cache cleanup (1 hour)

View Source 
const DefaultMaxCacheSize int64 = 10 * 1024 * 1024 * 1024

DefaultMaxCacheSize is the default maximum cache size (10 GB)

View Source 
const DefaultStaleMapTTL = 24 * time.Hour

DefaultStaleMapTTL is the default TTL for stale map entries (24 hours)

Variables

View Source 
var Clock = func() time.Time {
	return time.Now().UTC()
}
View Source 
var DebugLogging = false

DebugLogging controls whether debug messages are logged

View Source 
var ErrNotFoundInCache = errors.New("not found in cache")

Returned when a resource doesn't exist

View Source 
var Writes sync.WaitGroup

Functions

func SetLogger added in v0.8.0 

func SetLogger(log *logger.Logger)

SetLogger sets the logger instance for the httpcache package

Types

type Cache 

type Cache interface {
	Header(key string) (Header, error)
	Store(res *Resource, keys ...string) error
	Retrieve(key string) (*Resource, error)
	Invalidate(keys ...string)
	Freshen(res *Resource, keys ...string) error
}

func NewDiskCache 

func NewDiskCache(dir string) (Cache, error)

NewDiskCache returns a disk-backed cache

func NewMemoryCache 

func NewMemoryCache() Cache

NewMemoryCache returns an ephemeral cache in memory

func NewVFSCache 

func NewVFSCache(fs vfs.VFS) Cache

NewVFSCache returns a cache backend off the provided VFS

type CacheConfig added in v0.8.0 

type CacheConfig struct {
	// MaxSize is the maximum size of the cache in bytes.
	// When exceeded, the oldest items will be evicted (LRU).
	// Set to 0 to disable size-based eviction.
	MaxSize int64

	// TTL is the time-to-live for cached items.
	// Items older than this will be evicted during cleanup.
	// Set to 0 to disable TTL-based eviction.
	TTL time.Duration

	// CleanupInterval is the interval between automatic cleanup runs.
	// Set to 0 to disable automatic cleanup.
	CleanupInterval time.Duration

	// StaleMapTTL is the TTL for stale map entries.
	// Stale entries older than this will be removed.
	StaleMapTTL time.Duration
}

CacheConfig holds configuration for cache behavior

func DefaultCacheConfig added in v0.8.0 

func DefaultCacheConfig() *CacheConfig

DefaultCacheConfig returns a CacheConfig with sensible defaults

func (*CacheConfig) Validate added in v0.8.0 

func (c *CacheConfig) Validate() *CacheConfig

Validate validates the configuration and applies defaults where needed

func (*CacheConfig) WithCleanupInterval added in v0.8.0 

func (c *CacheConfig) WithCleanupInterval(interval time.Duration) *CacheConfig

WithCleanupInterval sets the cleanup interval

func (*CacheConfig) WithMaxSize added in v0.8.0 

func (c *CacheConfig) WithMaxSize(size int64) *CacheConfig

WithMaxSize sets the maximum cache size

func (*CacheConfig) WithStaleMapTTL added in v0.8.0 

func (c *CacheConfig) WithStaleMapTTL(ttl time.Duration) *CacheConfig

WithStaleMapTTL sets the stale map TTL

func (*CacheConfig) WithTTL added in v0.8.0 

func (c *CacheConfig) WithTTL(ttl time.Duration) *CacheConfig

WithTTL sets the TTL for cached items

type CacheControl 

type CacheControl map[string][]string

func ParseCacheControl 

func ParseCacheControl(input string) (CacheControl, error)

func ParseCacheControlHeaders 

func ParseCacheControlHeaders(h http.Header) (CacheControl, error)

func (CacheControl) Add 

func (cc CacheControl) Add(key, val string)

func (CacheControl) Duration 

func (cc CacheControl) Duration(key string) (time.Duration, error)

func (CacheControl) Get 

func (cc CacheControl) Get(key string) (string, bool)

func (CacheControl) Has 

func (cc CacheControl) Has(key string) bool

func (CacheControl) String 

func (cc CacheControl) String() string

type CacheMetrics added in v0.8.0 

type CacheMetrics struct {
	// CacheHits tracks the number of cache hits
	CacheHits *prometheus.CounterVec

	// CacheMisses tracks the number of cache misses
	CacheMisses *prometheus.CounterVec

	// CacheSkips tracks the number of cache skips (non-cacheable requests)
	CacheSkips prometheus.Counter

	// UpstreamDuration tracks the duration of upstream requests
	UpstreamDuration *prometheus.HistogramVec

	// CacheSizeBytes tracks the current cache size in bytes (gauge)
	CacheSizeBytes prometheus.Gauge

	// CacheItemCount tracks the current number of cached items (gauge)
	CacheItemCount prometheus.Gauge

	// CacheStaleCount tracks the current number of stale map entries (gauge)
	CacheStaleCount prometheus.Gauge

	// UpstreamErrors tracks the number of upstream errors
	UpstreamErrors *prometheus.CounterVec

	// CacheStoreOperations tracks cache store operations
	CacheStoreOperations *prometheus.CounterVec

	// CacheRetrieveOperations tracks cache retrieve operations
	CacheRetrieveOperations *prometheus.CounterVec

	// CacheEvictions tracks the number of cache evictions
	CacheEvictions *prometheus.CounterVec

	// CacheCleanupDuration tracks the duration of cleanup operations
	CacheCleanupDuration prometheus.Histogram

	// RequestsByDistro tracks total requests by distribution and status
	RequestsByDistro *prometheus.CounterVec

	// MirrorSwitches tracks mirror switch events
	MirrorSwitches *prometheus.CounterVec

	// RequestLatency tracks request latency distribution by distribution and cache status
	RequestLatency *prometheus.HistogramVec

	// ActiveConnections tracks the number of active connections
	ActiveConnections prometheus.Gauge

	// BytesTransferred tracks total bytes transferred
	BytesTransferred *prometheus.CounterVec

	// MirrorBenchmarkDuration tracks mirror benchmark operation duration
	MirrorBenchmarkDuration *prometheus.HistogramVec

	// PackageDownloads tracks package download counts by distribution
	PackageDownloads *prometheus.CounterVec

	// AuthFailures tracks authentication failures
	AuthFailures *prometheus.CounterVec
}

CacheMetrics holds Prometheus metrics for cache operations 

var DefaultMetrics *CacheMetrics

DefaultMetrics is the default metrics instance (nil until initialized)

func NewCacheMetrics added in v0.8.0 

func NewCacheMetrics(registry *metrics.Registry) *CacheMetrics

NewCacheMetrics creates and registers cache metrics with the given registry

func (*CacheMetrics) DecrementActiveConnections added in v0.8.0 

func (m *CacheMetrics) DecrementActiveConnections()

DecrementActiveConnections decrements the active connection counter

func (*CacheMetrics) IncrementActiveConnections added in v0.8.0 

func (m *CacheMetrics) IncrementActiveConnections()

IncrementActiveConnections increments the active connection counter

func (*CacheMetrics) RecordAuthFailure added in v0.8.0 

func (m *CacheMetrics) RecordAuthFailure(reason string)

RecordAuthFailure records an authentication failure

func (*CacheMetrics) RecordBytesTransferred added in v0.8.0 

func (m *CacheMetrics) RecordBytesTransferred(direction, distro string, bytes int64)

RecordBytesTransferred records bytes transferred

func (*CacheMetrics) RecordCacheEviction added in v0.8.0 

func (m *CacheMetrics) RecordCacheEviction(reason string)

RecordCacheEviction records a cache eviction

func (*CacheMetrics) RecordCacheHit added in v0.8.0 

func (m *CacheMetrics) RecordCacheHit(method string)

RecordCacheHit records a cache hit

func (*CacheMetrics) RecordCacheMiss added in v0.8.0 

func (m *CacheMetrics) RecordCacheMiss(method string)

RecordCacheMiss records a cache miss

func (*CacheMetrics) RecordCacheSkip added in v0.8.0 

func (m *CacheMetrics) RecordCacheSkip()

RecordCacheSkip records a cache skip

func (*CacheMetrics) RecordCleanupDuration added in v0.8.0 

func (m *CacheMetrics) RecordCleanupDuration(durationSeconds float64)

RecordCleanupDuration records the duration of a cleanup operation

func (*CacheMetrics) RecordMirrorBenchmarkDuration added in v0.8.0 

func (m *CacheMetrics) RecordMirrorBenchmarkDuration(distro, result string, durationSeconds float64)

RecordMirrorBenchmarkDuration records the duration of a mirror benchmark

func (*CacheMetrics) RecordMirrorSwitch added in v0.8.0 

func (m *CacheMetrics) RecordMirrorSwitch(distro, fromMirror, toMirror string)

RecordMirrorSwitch records a mirror switch event

func (*CacheMetrics) RecordPackageDownload added in v0.8.0 

func (m *CacheMetrics) RecordPackageDownload(distro, packageType string)

RecordPackageDownload records a package download

func (*CacheMetrics) RecordRequestByDistro added in v0.8.0 

func (m *CacheMetrics) RecordRequestByDistro(distro, status string)

RecordRequestByDistro records a request for a specific distribution

func (*CacheMetrics) RecordRequestLatency added in v0.8.0 

func (m *CacheMetrics) RecordRequestLatency(distro string, cacheHit bool, durationSeconds float64)

RecordRequestLatency records the latency of a request

func (*CacheMetrics) RecordRetrieveOperation added in v0.8.0 

func (m *CacheMetrics) RecordRetrieveOperation(found bool)

RecordRetrieveOperation records a cache retrieve operation

func (*CacheMetrics) RecordStoreOperation added in v0.8.0 

func (m *CacheMetrics) RecordStoreOperation(success bool)

RecordStoreOperation records a cache store operation

func (*CacheMetrics) RecordUpstreamDuration added in v0.8.0 

func (m *CacheMetrics) RecordUpstreamDuration(method string, status int, durationSeconds float64)

RecordUpstreamDuration records the duration of an upstream request

func (*CacheMetrics) RecordUpstreamError added in v0.8.0 

func (m *CacheMetrics) RecordUpstreamError(errorType string)

RecordUpstreamError records an upstream error

func (*CacheMetrics) SetCacheItemCount added in v0.8.0 

func (m *CacheMetrics) SetCacheItemCount(count int)

SetCacheItemCount sets the current number of cached items

func (*CacheMetrics) SetCacheSize added in v0.8.0 

func (m *CacheMetrics) SetCacheSize(sizeBytes int64)

SetCacheSize sets the current cache size in bytes

func (*CacheMetrics) SetCacheStaleCount added in v0.8.0 

func (m *CacheMetrics) SetCacheStaleCount(count int)

SetCacheStaleCount sets the current number of stale map entries

func (*CacheMetrics) UpdateCacheStats added in v0.8.0 

func (m *CacheMetrics) UpdateCacheStats(stats CacheStats)

UpdateCacheStats updates all cache gauge metrics from stats

type CacheStats added in v0.8.0 

type CacheStats struct {
	// TotalSize is the total size of cached items in bytes
	TotalSize int64
	// ItemCount is the number of cached items
	ItemCount int
	// StaleCount is the number of stale map entries
	StaleCount int
	// HitCount is the number of cache hits
	HitCount int64
	// MissCount is the number of cache misses
	MissCount int64
}

CacheStats holds cache statistics

type CleanupResult added in v0.8.0 

type CleanupResult struct {
	// RemovedItems is the number of items removed
	RemovedItems int
	// RemovedBytes is the number of bytes freed
	RemovedBytes int64
	// RemovedStaleEntries is the number of stale map entries removed
	RemovedStaleEntries int
	// Duration is how long the cleanup took
	Duration time.Duration
}

CleanupResult holds the result of a cleanup operation

type ExtendedCache added in v0.8.0 

type ExtendedCache interface {
	Cache
	// Stats returns current cache statistics
	Stats() CacheStats
	// Cleanup runs a manual cleanup cycle
	Cleanup() CleanupResult
	// Purge removes all cached items
	Purge() error
	// Close stops the cache and cleanup goroutines
	Close() error
}

ExtendedCache extends Cache with management capabilities

func NewDiskCacheWithConfig added in v0.8.0 

func NewDiskCacheWithConfig(dir string, config *CacheConfig) (ExtendedCache, error)

NewDiskCacheWithConfig returns a disk-backed cache with custom configuration

func NewMemoryCacheWithConfig added in v0.8.0 

func NewMemoryCacheWithConfig(config *CacheConfig) ExtendedCache

NewMemoryCacheWithConfig returns an ephemeral cache with custom configuration

func NewVFSCacheWithConfig added in v0.8.0 

func NewVFSCacheWithConfig(fs vfs.VFS, config *CacheConfig) ExtendedCache

NewVFSCacheWithConfig returns a cache backend with custom configuration

type Handler 

type Handler struct {
	Shared bool
	// contains filtered or unexported fields
}

func NewHandler 

func NewHandler(cache Cache, upstream http.Handler) *Handler

func (*Handler) ServeHTTP 

func (h *Handler) ServeHTTP(rw http.ResponseWriter, r *http.Request)

func (*Handler) SetMetrics added in v0.8.0 

func (h *Handler) SetMetrics(m *CacheMetrics)

SetMetrics sets the metrics instance for the handler 

type Header struct {
	http.Header
	StatusCode int
}

type Key 

type Key struct {
	// contains filtered or unexported fields
}

Key represents a unique identifier for a resource in the cache

func NewKey 

func NewKey(method string, u *url.URL, h http.Header) Key

NewKey returns a new Key instance

func NewRequestKey 

func NewRequestKey(r *http.Request) Key

NewRequestKey generates a Key for a request

func (Key) ForMethod 

func (k Key) ForMethod(method string) Key

ForMethod returns a new Key with a given method

func (Key) String 

func (k Key) String() string

func (Key) Vary 

func (k Key) Vary(varyHeader string, r *http.Request) Key

Vary returns a Key that is varied on particular headers in a http.Request

type ReadSeekCloser 

type ReadSeekCloser interface {
	io.Reader
	io.Seeker
	io.Closer
}

type Resource 

type Resource struct {
	ReadSeekCloser
	RequestTime, ResponseTime time.Time
	// contains filtered or unexported fields
}

func NewResource 

func NewResource(statusCode int, body ReadSeekCloser, hdrs http.Header) *Resource

func NewResourceBytes 

func NewResourceBytes(statusCode int, b []byte, hdrs http.Header) *Resource

func (*Resource) Age 

func (r *Resource) Age() (time.Duration, error)

Calculate the age of the resource

func (*Resource) DateAfter 

func (r *Resource) DateAfter(d time.Time) bool

func (*Resource) Expires 

func (r *Resource) Expires() (time.Time, error)

func (*Resource) HasExplicitExpiration 

func (r *Resource) HasExplicitExpiration() bool

func (*Resource) HasValidators 

func (r *Resource) HasValidators() bool

func (*Resource) Header 

func (r *Resource) Header() http.Header

func (*Resource) HeuristicFreshness 

func (r *Resource) HeuristicFreshness() time.Duration

func (*Resource) IsNonErrorStatus 

func (r *Resource) IsNonErrorStatus() bool

func (*Resource) IsStale 

func (r *Resource) IsStale() bool

func (*Resource) LastModified 

func (r *Resource) LastModified() time.Time

func (*Resource) MarkStale 

func (r *Resource) MarkStale()

func (*Resource) MaxAge 

func (r *Resource) MaxAge(shared bool) (time.Duration, error)

func (*Resource) MustValidate 

func (r *Resource) MustValidate(shared bool) bool

func (*Resource) RemovePrivateHeaders 

func (r *Resource) RemovePrivateHeaders()

func (*Resource) Status 

func (r *Resource) Status() int

func (*Resource) Via 

func (r *Resource) Via() string

type Validator 

type Validator struct {
	Handler http.Handler
}

func (*Validator) Validate 

func (v *Validator) Validate(req *http.Request, res *Resource) bool

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.