httpcache

package module
v1.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 4, 2025 License: MIT Imports: 13 Imported by: 2

README

httpcache

httpcache

CI Security Coverage Go Report Card License GoDoc

Package httpcache provides an http.RoundTripper implementation that works as a mostly RFC 7234 compliant cache for HTTP responses. It improves application performance by reducing redundant HTTP requests and supports various backends for use cases such as API caching, web scraping, and microservices.

Note: This is a maintained fork of gregjones/httpcache, which is no longer actively maintained. This fork aims to modernize the codebase while maintaining backward compatibility, fix bugs, and add new features.

Use Cases

  • API Caching: Reduce latency and server load by caching API responses.
  • Web Scraping: Avoid repeated requests to the same endpoints.
  • Microservices: Cache responses between services for better performance.
  • Web Applications: Improve user experience by caching dynamic content.
  • Resource Caching: Store static or frequently accessed resources locally.

Table of Contents

Features

  • ✅ RFC 7234 Compliant (~95% compliance) - Implements HTTP caching standards
    • ✅ Age header calculation (Section 4.2.3)
    • ✅ Warning headers for stale responses (Section 5.5)
    • ✅ must-revalidate directive enforcement (Section 5.2.2.1)
    • ✅ Pragma: no-cache support (Section 5.4)
    • ✅ Cache invalidation on unsafe methods (Section 4.4)
  • ✅ Multiple Backends - Memory, Disk, Redis, LevelDB, Memcache, PostgreSQL, MongoDB, NATS K/V, Hazelcast, Cloud Storage (S3/GCS/Azure)
  • ✅ Multi-Tier Caching - Combine multiple backends with automatic fallback and promotion
  • ✅ Compression Wrapper - Automatic Gzip, Brotli, or Snappy compression for cached data
  • ✅ Security Wrapper - Optional SHA-256 key hashing and AES-256 encryption
  • ✅ Thread-Safe - Safe for concurrent use
  • ✅ Zero Dependencies - Core package uses only Go standard library
  • ✅ Easy Integration - Drop-in replacement for http.Client
  • ✅ ETag & Validation - Automatic cache revalidation
  • ✅ Stale-If-Error - Resilient caching with RFC 5861 support
  • ✅ Stale-While-Revalidate - Async cache updates for better performance
  • ✅ Private Cache - Suitable for web browsers and API clients

Quick Start

package main

import (
    "fmt"
    "io"
    "net/http"
    
    "github.com/sandrolain/httpcache"
)

func main() {
    // Create a cached HTTP client
    transport := httpcache.NewMemoryCacheTransport()
    client := transport.Client()
    
    // Make requests - second request will be cached!
    resp, _ := client.Get("https://example.com")
    io.Copy(io.Discard, resp.Body)
    resp.Body.Close()
    
    // Check if response came from cache
    if resp.Header.Get(httpcache.XFromCache) == "1" {
        fmt.Println("Response was cached!")
    }
}

Installation

go get github.com/sandrolain/httpcache

Documentation

📚 Core Documentation
🔍 Quick Navigation

Getting Started:

Common Tasks:

Advanced Topics:

Practical Examples

See the examples/ directory for complete, runnable examples:

Each example includes:

  • Complete working code
  • Detailed README
  • Use case explanations
  • Best practices

Limitations

Vary Header

⚠️ Current Limitation: The Vary response header is used for validation only, not for creating separate cache entries.

Workaround: Use CacheKeyHeaders to create true separate cache entries.

See How It Works - Vary Header Support for details.

Private Directive

⚠️ Current Limitation: The Cache-Control: private directive is ignored because httpcache implements a private cache.

Impact: In multi-user scenarios, responses marked as private will still be cached and potentially shared.

Workaround: Use Cache-Control: no-store or configure CacheKeyHeaders.

See Security Considerations for details.

Performance

  • Memory cache: ~100ns per operation
  • Disk cache: ~1-5ms per operation (depends on filesystem)
  • Redis cache: ~1-3ms per operation (network latency dependent)
  • Overhead vs no-cache: < 5% for cache hits

Benchmarks are available in each backend's *_bench_test.go file.

Testing

# Run all tests
go test ./...

# Run with coverage
go test -cover ./...

# Run integration tests (requires Docker)
go test -tags=integration ./...

# Run benchmarks
go test -bench=. ./...

Contributing

Contributions are welcome! Please:

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Make your changes with tests
  4. Run golangci-lint run and govulncheck ./...
  5. Commit your changes (git commit -m 'feat: add amazing feature')
  6. Push to the branch (git push origin feature/amazing-feature)
  7. Open a Pull Request

Acknowledgments

This project is a fork of gregjones/httpcache by Greg Jones, which was archived in 2023. We're grateful for the original work and continue its development with modern Go practices.

Additional acknowledgments:

  • RFC 7234 - HTTP Caching specification
  • RFC 5861 - HTTP Cache-Control Extensions for Stale Content
  • All contributors to the original and forked projects

License

MIT License - see LICENSE.txt for details.

Support

Documentation

Overview

Package httpcache provides a http.RoundTripper implementation that works as a mostly RFC-compliant cache for http responses.

It is only suitable for use as a 'private' cache (i.e. for a web-browser or an API-client and not for a shared proxy).

Index

Constants

View Source 
const (

	// XFromCache is the header added to responses that are returned from the cache
	XFromCache = "X-From-Cache"
	// XRevalidated is the header added to responses that got revalidated
	XRevalidated = "X-Revalidated"
	// XStale is the header added to responses that are stale
	XStale = "X-Stale"
	// XFreshness is the header added to responses indicating the freshness state
	XFreshness = "X-Cache-Freshness"
	// XCachedTime is the internal header used to store when a response was cached
	XCachedTime = "X-Cached-Time"
)

Variables

View Source 
var ErrNoDateHeader = errors.New("no Date header")

ErrNoDateHeader indicates that the HTTP headers contained no Date header.

Functions

func CachedResponse 

func CachedResponse(c Cache, req *http.Request) (resp *http.Response, err error)

CachedResponse returns the cached http.Response for req if present, and nil otherwise.

func Date 

func Date(respHeaders http.Header) (date time.Time, err error)

Date parses and returns the value of the Date header.

func GetLogger 

func GetLogger() *slog.Logger

GetLogger returns the configured logger or the default slog logger.

func SetLogger 

func SetLogger(l *slog.Logger)

SetLogger sets a custom slog.Logger instance to be used by the httpcache package. If not set, the default slog logger will be used.

Types

type Cache 

type Cache interface {
	// Get returns the []byte representation of a cached response and a bool
	// set to true if the value isn't empty
	Get(key string) (responseBytes []byte, ok bool)
	// Set stores the []byte representation of a response against a key
	Set(key string, responseBytes []byte)
	// Delete removes the value associated with the key
	Delete(key string)
}

A Cache interface is used by the Transport to store and retrieve responses.

type MemoryCache 

type MemoryCache struct {
	// contains filtered or unexported fields
}

MemoryCache is an implemtation of Cache that stores responses in an in-memory map.

func NewMemoryCache 

func NewMemoryCache() *MemoryCache

NewMemoryCache returns a new Cache that will store items in an in-memory map

func (*MemoryCache) Delete 

func (c *MemoryCache) Delete(key string)

Delete removes key from the cache

func (*MemoryCache) Get 

func (c *MemoryCache) Get(key string) (resp []byte, ok bool)

Get returns the []byte representation of the response and true if present, false if not

func (*MemoryCache) Set 

func (c *MemoryCache) Set(key string, resp []byte)

Set saves response resp to the cache with key

type Transport 

type Transport struct {
	// The RoundTripper interface actually used to make requests
	// If nil, http.DefaultTransport is used
	Transport http.RoundTripper
	Cache     Cache
	// If true, responses returned from the cache will be given an extra header, X-From-Cache
	MarkCachedResponses bool
	// If true, server errors (5xx status codes) will not be served from cache
	// even if they are fresh. This forces a new request to the server.
	// Default is false to maintain backward compatibility.
	SkipServerErrorsFromCache bool
	// AsyncRevalidateTimeout is the context timeout for async requests triggered by stale-while-revalidate.
	// If zero, no timeout is applied to async revalidation requests.
	AsyncRevalidateTimeout time.Duration
	// ShouldCache allows configuring non-standard caching behaviour based on the response.
	// If set, this function is called to determine whether a non-200 response should be cached.
	// This enables caching of responses like 404 Not Found, 301 Moved Permanently, etc.
	// If nil, only 200 OK responses are cached (standard behavior).
	// The function receives the http.Response and should return true to cache it.
	// Note: This only bypasses the status code check; Cache-Control headers are still respected.
	ShouldCache func(*http.Response) bool
	// CacheKeyHeaders specifies additional request headers to include in the cache key generation.
	// This allows creating separate cache entries based on request header values.
	// Common use cases include "Authorization" for user-specific caches or "Accept-Language"
	// for locale-specific responses.
	// Header names are case-insensitive and will be canonicalized.
	// Example: []string{"Authorization", "Accept-Language"}
	// Note: This is different from the HTTP Vary response header mechanism, which is handled separately.
	CacheKeyHeaders []string
}

Transport is an implementation of http.RoundTripper that will return values from a cache where possible (avoiding a network request) and will additionally add validators (etag/if-modified-since) to repeated requests allowing servers to return 304 / Not Modified

func NewMemoryCacheTransport 

func NewMemoryCacheTransport() *Transport

NewMemoryCacheTransport returns a new Transport using the in-memory cache implementation

func NewTransport 

func NewTransport(c Cache) *Transport

NewTransport returns a new Transport with the provided Cache implementation and MarkCachedResponses set to true

func (*Transport) Client 

func (t *Transport) Client() *http.Client

Client returns an *http.Client that caches responses.

func (*Transport) RoundTrip 

func (t *Transport) RoundTrip(req *http.Request) (resp *http.Response, err error)

RoundTrip takes a Request and returns a Response

If there is a fresh Response already in cache, then it will be returned without connecting to the server.

If there is a stale Response, then any validators it contains will be set on the new request to give the server a chance to respond with NotModified. If this happens, then the cached Response will be returned.

Source Files

View all Source files

Directories

Path Synopsis
Package blobcache provides an httpcache.Cache implementation that uses Go Cloud Development Kit (CDK) blob storage for cloud-agnostic cache storage.
 Package blobcache provides an httpcache.Cache implementation that uses Go Cloud Development Kit (CDK) blob storage for cloud-agnostic cache storage.
Package diskcache provides an implementation of httpcache.Cache that uses the diskv package to supplement an in-memory map with persistent storage
 Package diskcache provides an implementation of httpcache.Cache that uses the diskv package to supplement an in-memory map with persistent storage
examples
basic command
blobcache command
cachekeyheaders command
compresscache command
custom-backend command
diskcache command
freecache command
hazelcast command
leveldb command
mongodb command
multicache command
natskv command
postgresql command
prometheus command
redis command
security-best-practices command
Package freecache provides a high-performance, zero-GC overhead implementation of httpcache.Cache using github.com/coocood/freecache as the underlying storage.
 Package freecache provides a high-performance, zero-GC overhead implementation of httpcache.Cache using github.com/coocood/freecache as the underlying storage.
Package hazelcast provides a Hazelcast interface for http caching.
 Package hazelcast provides a Hazelcast interface for http caching.
Package leveldbcache provides an implementation of httpcache.Cache that uses github.com/syndtr/goleveldb/leveldb
 Package leveldbcache provides an implementation of httpcache.Cache that uses github.com/syndtr/goleveldb/leveldb
Package memcache provides an implementation of httpcache.Cache that uses gomemcache to store cached responses.
 Package memcache provides an implementation of httpcache.Cache that uses gomemcache to store cached responses.
Package mongodb provides a MongoDB interface for http caching.
 Package mongodb provides a MongoDB interface for http caching.
Package natskv provides a NATS JetStream Key/Value store interface for http caching.
 Package natskv provides a NATS JetStream Key/Value store interface for http caching.
Package postgresql provides a PostgreSQL interface for HTTP caching.
 Package postgresql provides a PostgreSQL interface for HTTP caching.
Package redis provides a redis interface for http caching.
 Package redis provides a redis interface for http caching.
wrapper
compresscache
Package compresscache provides a cache wrapper that automatically compresses cached data to reduce storage requirements and network bandwidth usage.
 Package compresscache provides a cache wrapper that automatically compresses cached data to reduce storage requirements and network bandwidth usage.
metrics
Package metrics provides an interface for collecting HTTP cache metrics.
 Package metrics provides an interface for collecting HTTP cache metrics.
metrics/prometheus
Package prometheus provides Prometheus metrics implementation for httpcache.
 Package prometheus provides Prometheus metrics implementation for httpcache.
multicache
Package multicache provides a multi-tiered cache implementation that allows cascading through multiple cache backends with automatic fallback and promotion.
 Package multicache provides a multi-tiered cache implementation that allows cascading through multiple cache backends with automatic fallback and promotion.
securecache
Package securecache provides a security wrapper for httpcache.Cache implementations.
 Package securecache provides a security wrapper for httpcache.Cache implementations.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.