cache

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 17, 2025 License: MIT Imports: 22 Imported by: 0

Documentation

Overview

Package cache provides a comprehensive caching system for OCI (Open Container Initiative) artifacts.

This package implements a multi-tier caching strategy optimized for OCI registries, with separate handling for manifests and blobs. The cache system is designed to:

  • Reduce network requests to OCI registries
  • Provide configurable TTL (Time-To-Live) for cache entries
  • Store already-compressed OCI artifacts efficiently (no double compression)
  • Handle concurrent access safely
  • Provide detailed metrics and observability
  • Support pluggable eviction strategies

Architecture Overview

The cache system consists of several key components:

  • CacheManager: Central coordinator that manages cache configuration and metrics
  • Cache: Core interface for basic cache operations (get, put, delete, clear)
  • ManifestCache: Specialized interface for OCI manifest caching with validation
  • BlobCache: Specialized interface for OCI blob caching with streaming support
  • EvictionStrategy: Pluggable strategies for cache size management

Cache Entry Lifecycle

1. **Creation**: Entries are created with a key, data, and optional TTL 2. **Storage**: Entries are stored directly with metadata (OCI artifacts are pre-compressed) 3. **Access**: Entries track access patterns for eviction decisions 4. **Expiration**: Entries can expire based on TTL or be explicitly invalidated 5. **Eviction**: Entries may be evicted when cache size limits are reached

Error Handling

The package defines specific errors for different cache failure scenarios:

  • ErrCacheExpired: Entry exists but has exceeded its TTL
  • ErrCacheCorrupted: Entry data is corrupted or unreadable
  • ErrCacheFull: Cache cannot accept new entries due to size limits
  • ErrCacheInvalidated: Entry has been explicitly invalidated

All errors support proper wrapping with context using fmt.Errorf and %w.

Configuration

Cache behavior is controlled through CacheConfig:

  • MaxSizeBytes: Maximum cache size in bytes
  • DefaultTTL: Default time-to-live for entries

Metrics and Observability

The CacheMetrics struct tracks:

  • Hit/miss ratios
  • Eviction counts
  • Error rates
  • Storage utilization

Thread Safety

All cache implementations must be safe for concurrent use by multiple goroutines. The package uses appropriate synchronization primitives to ensure data consistency.

Implementation Notes

This package follows Go best practices:

  • Interfaces are defined at the point of use
  • Errors are wrapped with context
  • Configuration is validated on creation
  • Tests achieve high coverage with race detection
  • Code follows the Go style guide from docs/guides/go/style.md

Index

Constants

View Source
const (
	OperationManifest = "manifest"
	OperationBlob     = "blob"
)

Operation types for metrics tracking

Variables

View Source
var ErrCacheCorrupted = errors.New("cache entry is corrupted")

ErrCacheCorrupted is returned when a cache entry is found to be corrupted.

View Source
var ErrCacheExpired = errors.New("cache entry has expired")

ErrCacheExpired is returned when attempting to access an expired cache entry.

View Source
var ErrCacheFull = errors.New("cache is full")

ErrCacheFull is returned when the cache cannot store additional entries due to size limits.

View Source
var ErrCacheInvalidated = errors.New("cache entry has been invalidated")

ErrCacheInvalidated is returned when a cache entry has been invalidated and can no longer be used.

Functions

func LogCacheHit

func LogCacheHit(ctx context.Context, logger *Logger, operation Operation, size int64)

LogCacheHit logs a cache hit event.

func LogCacheMiss

func LogCacheMiss(ctx context.Context, logger *Logger, operation Operation, reason string)

LogCacheMiss logs a cache miss event.

func LogCacheOperation

func LogCacheOperation(
	ctx context.Context,
	logger *Logger,
	operation Operation,
	duration time.Duration,
	success bool,
	size int64,
	err error,
)

LogCacheOperation logs a cache operation with performance metrics.

func LogCleanup

func LogCleanup(
	ctx context.Context,
	logger *Logger,
	operation string,
	entriesRemoved int,
	bytesFreed int64,
	duration time.Duration,
)

LogCleanup logs cleanup operations.

func LogEviction

func LogEviction(ctx context.Context, logger *Logger, key string, size int64, reason string)

LogEviction logs an eviction event.

func LogPerformanceMetrics

func LogPerformanceMetrics(ctx context.Context, logger *Logger, metrics *MetricsSnapshot)

LogPerformanceMetrics logs periodic performance metrics.

func NewManifestCache

func NewManifestCache(storage *Storage, manager ConfigProvider) *manifestCache

NewManifestCache creates a new manifest cache instance.

func NewTagCache

func NewTagCache(storage *Storage, config TagResolverConfig) *tagCache

NewTagCache creates a new tag cache instance.

Types

type BlobCache

type BlobCache interface {
	// GetBlob retrieves a blob by its digest.
	// Returns a ReadCloser that the caller must close.
	GetBlob(ctx context.Context, digest string) (io.ReadCloser, error)

	// PutBlob stores a blob with the given digest.
	// The reader content is consumed and stored.
	PutBlob(ctx context.Context, digest string, reader io.Reader) error

	// HasBlob checks if a blob exists in the cache.
	// This is more efficient than GetBlob when only existence matters.
	HasBlob(ctx context.Context, digest string) (bool, error)

	// DeleteBlob removes a blob from the cache.
	// Returns nil if the blob doesn't exist (idempotent operation).
	DeleteBlob(ctx context.Context, digest string) error
}

BlobCache defines operations specific to OCI blob caching. Blobs can be large and benefit from streaming operations.

func NewBlobCache

func NewBlobCache(storage *Storage, defaultTTL time.Duration) (BlobCache, error)

NewBlobCache creates a new blob cache with the given storage backend. The defaultTTL specifies how long blobs should be cached (24 hours default).

type Cache

type Cache interface {
	// Get retrieves a cache entry by key.
	// Returns ErrCacheExpired if the entry exists but has expired.
	// Returns an error if the key is not found or if retrieval fails.
	Get(ctx context.Context, key string) (*Entry, error)

	// Put stores a cache entry with the given key.
	// Returns ErrCacheFull if the cache cannot accommodate the entry.
	// Overwrites existing entries with the same key.
	Put(ctx context.Context, key string, entry *Entry) error

	// Delete removes a cache entry by key.
	// Returns nil if the key doesn't exist (idempotent operation).
	Delete(ctx context.Context, key string) error

	// Clear removes all entries from the cache.
	Clear(ctx context.Context) error

	// Size returns the total size of all cache entries in bytes.
	Size(ctx context.Context) (int64, error)
}

Cache defines the core interface for cache operations. Implementations should be safe for concurrent use by multiple goroutines.

type CompositeEviction

type CompositeEviction struct {
	// contains filtered or unexported fields
}

CompositeEviction combines multiple eviction strategies with configurable priorities. It allows different strategies to work together for optimal cache management.

func NewCompositeEviction

func NewCompositeEviction(strategies []EvictionStrategy, priorities []int) *CompositeEviction

NewCompositeEviction creates a new composite eviction strategy.

func (*CompositeEviction) OnAccess

func (c *CompositeEviction) OnAccess(entry *Entry)

OnAccess notifies all strategies of an access event.

func (*CompositeEviction) OnAdd

func (c *CompositeEviction) OnAdd(entry *Entry)

OnAdd notifies all strategies of an add event.

func (*CompositeEviction) OnRemove

func (c *CompositeEviction) OnRemove(entry *Entry)

OnRemove notifies all strategies of a remove event.

func (*CompositeEviction) SelectForEviction

func (c *CompositeEviction) SelectForEviction(entries map[string]*Entry) []string

SelectForEviction combines results from all strategies based on priority. Higher priority strategies (lower priority numbers) are processed first.

type Config

type Config struct {
	// MaxSizeBytes is the maximum size of the cache in bytes.
	MaxSizeBytes int64
	// DefaultTTL is the default time-to-live for cache entries.
	DefaultTTL time.Duration
}

Config holds configuration for cache behavior.

func (*Config) SetDefaults

func (c *Config) SetDefaults()

SetDefaults applies default values to unset fields in the configuration.

func (*Config) Validate

func (c *Config) Validate() error

Validate checks that the cache configuration is valid.

type ConfigProvider

type ConfigProvider interface {
	Config() Config
}

ConfigProvider defines the interface that cache components need from a manager.

type Coordinator

type Coordinator struct {
	// contains filtered or unexported fields
}

Coordinator coordinates multiple cache types and implements comprehensive cache management. It provides a unified interface for manifest and blob caching with size limits, eviction, and metrics collection.

func NewCoordinator

func NewCoordinator(
	ctx context.Context,
	config Config,
	fs *billyfs.FS,
	cachePath string,
	logger *Logger,
) (*Coordinator, error)

NewCoordinator creates a new cache coordinator with the specified configuration. It initializes all cache components and starts background cleanup processes.

func (*Coordinator) Clear

func (cm *Coordinator) Clear(ctx context.Context) error

Clear removes all entries from all caches.

func (*Coordinator) Close

func (cm *Coordinator) Close() error

Close shuts down the cache manager and cleans up resources.

func (*Coordinator) Config

func (cm *Coordinator) Config() Config

Config returns the cache configuration.

func (*Coordinator) GetBlob

func (cm *Coordinator) GetBlob(
	ctx context.Context,
	digest string,
) (io.ReadCloser, error)

GetBlob retrieves a blob from the cache.

func (*Coordinator) GetManifest

func (cm *Coordinator) GetManifest(
	ctx context.Context,
	digest string,
) (*ocispec.Manifest, error)

GetManifest retrieves a manifest from the cache.

func (*Coordinator) GetMetrics

func (cm *Coordinator) GetMetrics() *DetailedMetrics

GetMetrics returns current cache metrics.

func (*Coordinator) GetStats

func (cm *Coordinator) GetStats() Stats

GetStats returns comprehensive cache statistics.

func (*Coordinator) PutBlob

func (cm *Coordinator) PutBlob(
	ctx context.Context,
	digest string,
	reader io.Reader,
) error

PutBlob stores a blob in the cache.

func (*Coordinator) PutManifest

func (cm *Coordinator) PutManifest(
	ctx context.Context,
	digest string,
	manifest *ocispec.Manifest,
) error

PutManifest stores a manifest in the cache.

func (*Coordinator) Size

func (cm *Coordinator) Size(ctx context.Context) (int64, error)

Size returns the total size of all cached entries.

type DebugInfo

type DebugInfo struct {
	Timestamp   time.Time        `json:"timestamp"`
	Config      Config           `json:"config"`
	Metrics     MetricsSnapshot  `json:"metrics"`
	IndexStats  IndexStats       `json:"index_stats"`
	CacheStats  Stats            `json:"cache_stats"`
	TopEntries  []EntryInfo      `json:"top_entries"`
	Integrity   IntegrityReport  `json:"integrity"`
	StorageInfo StorageDebugInfo `json:"storage_info"`
}

DebugInfo provides comprehensive debugging information about the cache state.

type DebugTools

type DebugTools struct {
	// contains filtered or unexported fields
}

DebugTools provides debugging and maintenance utilities for the cache.

func NewDebugTools

func NewDebugTools(coordinator *Coordinator) *DebugTools

NewDebugTools creates a new debug tools instance for the given coordinator.

func (*DebugTools) ClearExpiredEntries

func (dt *DebugTools) ClearExpiredEntries(ctx context.Context) (int, error)

ClearExpiredEntries removes all expired entries from the cache.

func (*DebugTools) CollectDebugInfo

func (dt *DebugTools) CollectDebugInfo(ctx context.Context) (*DebugInfo, error)

CollectDebugInfo gathers comprehensive debugging information about the cache.

func (*DebugTools) ExportDebugInfo

func (dt *DebugTools) ExportDebugInfo(ctx context.Context, outputPath string) error

ExportDebugInfo exports debug information to a JSON file.

func (*DebugTools) GetCacheContents

func (dt *DebugTools) GetCacheContents(ctx context.Context) ([]EntryInfo, error)

GetCacheContents returns a list of all cache entries with their metadata.

func (*DebugTools) RepairIntegrity

func (dt *DebugTools) RepairIntegrity(ctx context.Context) (*IntegrityReport, error)

RepairIntegrity attempts to repair integrity issues found during checks.

func (*DebugTools) TriggerCleanup

func (dt *DebugTools) TriggerCleanup(ctx context.Context) error

TriggerCleanup manually triggers cache cleanup operations.

func (*DebugTools) ValidateAllDigests

func (dt *DebugTools) ValidateAllDigests(ctx context.Context) (int, []string, error)

ValidateAllDigests performs digest validation on all cache entries.

type DetailedMetrics

type DetailedMetrics struct {
	// contains filtered or unexported fields
}

DetailedMetrics provides comprehensive metrics collection for cache operations. It tracks performance, bandwidth savings, and operational statistics.

func NewDetailedMetrics

func NewDetailedMetrics() *DetailedMetrics

NewDetailedMetrics creates a new DetailedMetrics instance.

func (*DetailedMetrics) GetSnapshot

func (m *DetailedMetrics) GetSnapshot() MetricsSnapshot

GetSnapshot returns a thread-safe snapshot of current metrics.

func (*DetailedMetrics) RecordDelete

func (m *DetailedMetrics) RecordDelete(bytesRemoved int64)

RecordDelete records a deletion operation.

func (*DetailedMetrics) RecordError

func (m *DetailedMetrics) RecordError()

RecordError records an operation error.

func (*DetailedMetrics) RecordEviction

func (m *DetailedMetrics) RecordEviction(bytesEvicted int64)

RecordEviction records an eviction operation.

func (*DetailedMetrics) RecordHit

func (m *DetailedMetrics) RecordHit(operationType string, bytesServed int64)

RecordHit records a cache hit and updates relevant metrics.

func (*DetailedMetrics) RecordLatency

func (m *DetailedMetrics) RecordLatency(operation string, duration time.Duration)

RecordLatency records the latency of an operation.

func (*DetailedMetrics) RecordMiss

func (m *DetailedMetrics) RecordMiss(operationType string, bytesDownloaded int64)

RecordMiss records a cache miss and updates network usage metrics.

func (*DetailedMetrics) RecordPut

func (m *DetailedMetrics) RecordPut(operationType string, bytesStored int64)

RecordPut records a successful put operation.

func (*DetailedMetrics) RecordTagResolution

func (m *DetailedMetrics) RecordTagResolution()

RecordTagResolution records a tag resolution operation.

func (*DetailedMetrics) Reset

func (m *DetailedMetrics) Reset()

Reset clears all metrics data.

type DirInfo

type DirInfo struct {
	Path      string `json:"path"`
	FileCount int    `json:"file_count"`
	TotalSize int64  `json:"total_size"`
}

DirInfo provides information about a subdirectory.

type Entry

type Entry struct {
	// Key is the unique identifier for this cache entry.
	Key string
	// Data contains the cached data.
	Data []byte
	// Metadata contains additional metadata about the entry.
	Metadata map[string]string
	// CreatedAt is when this entry was first created.
	CreatedAt time.Time
	// AccessedAt is when this entry was last accessed.
	AccessedAt time.Time
	// TTL is the time-to-live for this entry. Zero means no expiration.
	TTL time.Duration
	// AccessCount tracks how many times this entry has been accessed.
	AccessCount int64
}

Entry represents a single entry in the cache.

func (*Entry) IsExpired

func (e *Entry) IsExpired() bool

IsExpired returns true if the cache entry has expired based on its TTL.

func (*Entry) Size

func (e *Entry) Size() int64

Size returns the approximate size of the cache entry in bytes.

type EntryInfo

type EntryInfo struct {
	Key         string        `json:"key"`
	Size        int64         `json:"size"`
	CreatedAt   time.Time     `json:"created_at"`
	AccessedAt  time.Time     `json:"accessed_at"`
	TTL         time.Duration `json:"ttl"`
	AccessCount int64         `json:"access_count"`
	FilePath    string        `json:"file_path"`
	IsExpired   bool          `json:"is_expired"`
	Type        string        `json:"type"` // "manifest" or "blob"
}

EntryInfo provides detailed information about a cache entry.

type EvictionStrategy

type EvictionStrategy interface {
	// SelectForEviction chooses which cache entries should be evicted.
	// Returns a slice of keys to evict, ordered by eviction priority.
	// The entries map provides access to all current cache entries for decision making.
	SelectForEviction(entries map[string]*Entry) []string

	// OnAccess is called when a cache entry is accessed (read).
	// Implementations can update access patterns or priorities.
	OnAccess(entry *Entry)

	// OnAdd is called when a new entry is added to the cache.
	// Implementations can initialize eviction metadata.
	OnAdd(entry *Entry)

	// OnRemove is called when an entry is removed from the cache.
	// Implementations can clean up eviction metadata.
	OnRemove(entry *Entry)
}

EvictionStrategy defines how entries are selected for eviction when cache size limits are reached. Implementations should be deterministic and thread-safe.

type Index

type Index struct {
	// contains filtered or unexported fields
}

Index manages in-memory cache index with persistence and recovery capabilities.

func NewIndex

func NewIndex(indexPath string, maxEntries int) *Index

NewIndex creates a new cache index with the specified persistence path.

func (*Index) Cleanup

func (idx *Index) Cleanup(ctx context.Context, forceCompact bool) error

Cleanup removes expired entries and optionally compacts the index.

func (*Index) Compact

func (idx *Index) Compact() error

Compact removes expired entries and optimizes the index structure.

func (*Index) Delete

func (idx *Index) Delete(key string) error

Delete removes an index entry by key.

func (*Index) ExpiredKeys

func (idx *Index) ExpiredKeys() []string

ExpiredKeys returns keys of all expired entries.

func (*Index) Get

func (idx *Index) Get(key string) (*IndexEntry, bool)

Get retrieves an index entry by key.

func (*Index) Keys

func (idx *Index) Keys(filter func(*IndexEntry) bool) []string

Keys returns all keys in the index, optionally filtered by a predicate.

func (*Index) Load

func (idx *Index) Load(ctx context.Context) error

Load reads the index from persistent storage.

func (*Index) NewIterator

func (idx *Index) NewIterator(filter func(*IndexEntry) bool) *Iterator

NewIterator creates a new iterator over the index entries.

func (*Index) Persist

func (idx *Index) Persist() error

Persist writes the current index to persistent storage.

func (*Index) Put

func (idx *Index) Put(key string, entry *IndexEntry) error

Put stores or updates an index entry.

func (*Index) Size

func (idx *Index) Size() int

Size returns the current number of entries in the index.

func (*Index) Stats

func (idx *Index) Stats() IndexStats

Stats returns current index statistics.

type IndexEntry

type IndexEntry struct {
	Key         string            `json:"key"`
	Size        int64             `json:"size"`
	CreatedAt   time.Time         `json:"created_at"`
	AccessedAt  time.Time         `json:"accessed_at"`
	TTL         time.Duration     `json:"ttl"`
	AccessCount int64             `json:"access_count"`
	Metadata    map[string]string `json:"metadata,omitempty"`
	FilePath    string            `json:"file_path"` // Relative path to the data file
}

IndexEntry represents a single entry in the cache index.

func (*IndexEntry) IsExpired

func (ie *IndexEntry) IsExpired() bool

IsExpired returns true if the index entry has expired.

type IndexStats

type IndexStats struct {
	TotalEntries       int
	ExpiredEntries     int
	TotalSize          int64
	LastCompaction     time.Time
	AverageAccessCount float64
}

IndexStats returns statistics about the index.

type IntegrityReport

type IntegrityReport struct {
	TotalEntries     int           `json:"total_entries"`
	CorruptedEntries []string      `json:"corrupted_entries"`
	MissingFiles     []string      `json:"missing_files"`
	InvalidDigests   []string      `json:"invalid_digests"`
	IsHealthy        bool          `json:"is_healthy"`
	CheckDuration    time.Duration `json:"check_duration"`
}

IntegrityReport provides information about cache integrity.

type Iterator

type Iterator struct {
	// contains filtered or unexported fields
}

Iterator allows iteration over index entries with optional filtering.

func (*Iterator) Close

func (it *Iterator) Close()

Close releases any resources held by the iterator.

func (*Iterator) Entry

func (it *Iterator) Entry() *IndexEntry

Entry returns the current entry.

func (*Iterator) Key

func (it *Iterator) Key() string

Key returns the current key.

func (*Iterator) Next

func (it *Iterator) Next() bool

Next advances the iterator to the next entry.

type LRUEviction

type LRUEviction struct {
	// contains filtered or unexported fields
}

LRUEviction implements an LRU (Least Recently Used) eviction strategy. It maintains access order using a doubly-linked list and provides O(1) access tracking.

func NewLRUEviction

func NewLRUEviction() *LRUEviction

NewLRUEviction creates a new LRU eviction strategy.

func (*LRUEviction) OnAccess

func (l *LRUEviction) OnAccess(entry *Entry)

OnAccess is called when a cache entry is accessed (read). Moves the entry to the front of the access list (most recently used).

func (*LRUEviction) OnAdd

func (l *LRUEviction) OnAdd(entry *Entry)

OnAdd is called when a new entry is added to the cache. Adds the entry to the front of the access list.

func (*LRUEviction) OnRemove

func (l *LRUEviction) OnRemove(entry *Entry)

OnRemove is called when an entry is removed from the cache. Removes the entry from the access tracking.

func (*LRUEviction) SelectForEviction

func (l *LRUEviction) SelectForEviction(entries map[string]*Entry) []string

SelectForEviction chooses which cache entries should be evicted based on LRU policy. Returns keys ordered by eviction priority (oldest accessed first).

type LogConfig

type LogConfig struct {
	// Level sets the minimum log level (debug, info, warn, error)
	Level LogLevel
	// EnableCallerInfo includes file and line number in logs
	EnableCallerInfo bool
	// EnablePerformanceLogging enables detailed performance metrics logging
	EnablePerformanceLogging bool
	// EnableCacheOperations enables logging of individual cache operations
	EnableCacheOperations bool
}

LogConfig holds configuration for the cache logger.

func DefaultLogConfig

func DefaultLogConfig() LogConfig

DefaultLogConfig returns a default logging configuration.

type LogLevel

type LogLevel int

LogLevel represents different logging levels

const (
	LogLevelDebug LogLevel = iota
	LogLevelInfo
	LogLevelWarn
	LogLevelError
)

LogLevelDebug represents debug logging level

func ParseLogLevel

func ParseLogLevel(level string) (LogLevel, error)

ParseLogLevel parses a string log level into a LogLevel.

type Logger

type Logger struct {
	// contains filtered or unexported fields
}

Logger provides structured logging for the cache system. It wraps different logger implementations for consistent behavior.

func NewLogger

func NewLogger(config LogConfig) *Logger

NewLogger creates a new structured logger with the given configuration.

func NewNopLogger

func NewNopLogger() *Logger

NewNopLogger creates a no-op logger that discards all log messages.

func (*Logger) Debug

func (l *Logger) Debug(ctx context.Context, msg string, args ...any)

Debug logs debug-level messages

func (*Logger) Error

func (l *Logger) Error(ctx context.Context, msg string, args ...any)

Error logs error-level messages

func (*Logger) Info

func (l *Logger) Info(ctx context.Context, msg string, args ...any)

Info logs info-level messages

func (*Logger) Warn

func (l *Logger) Warn(ctx context.Context, msg string, args ...any)

Warn logs warning-level messages

func (*Logger) With

func (l *Logger) With(args ...any) *Logger

With returns a logger with additional context fields

func (*Logger) WithDigest

func (l *Logger) WithDigest(digest string) *Logger

WithDigest returns a logger with digest context

func (*Logger) WithDuration

func (l *Logger) WithDuration(duration time.Duration) *Logger

WithDuration returns a logger with duration context

func (*Logger) WithOperation

func (l *Logger) WithOperation(operation string) *Logger

WithOperation returns a logger with operation context

func (*Logger) WithSize

func (l *Logger) WithSize(size int64) *Logger

WithSize returns a logger with size context

type ManifestCache

type ManifestCache interface {
	// GetManifest retrieves a manifest by its digest.
	// Returns the manifest content or an error if not found.
	GetManifest(ctx context.Context, digest string) (*ocispec.Manifest, error)

	// PutManifest stores a manifest with the given digest.
	// The digest should be validated before storage.
	PutManifest(ctx context.Context, digest string, manifest *ocispec.Manifest) error

	// HasManifest checks if a manifest exists in the cache.
	// This is more efficient than GetManifest when only existence matters.
	HasManifest(ctx context.Context, digest string) (bool, error)

	// ValidateManifest performs validation of a manifest against a reference.
	// This may involve checking the manifest digest against a registry.
	ValidateManifest(ctx context.Context, reference, digest string) (bool, error)
}

ManifestCache defines operations specific to OCI manifest caching. Manifests are typically small and have short TTLs for freshness.

type MetricsSnapshot

type MetricsSnapshot struct {
	// Core hit/miss statistics
	Hits    int64   `json:"hits"`
	Misses  int64   `json:"misses"`
	HitRate float64 `json:"hit_rate"`

	// Eviction and error tracking
	Evictions int64 `json:"evictions"`
	Errors    int64 `json:"errors"`

	// Size and entry tracking
	BytesStored    int64 `json:"bytes_stored"`
	EntriesStored  int64 `json:"entries_stored"`
	BytesRetrieved int64 `json:"bytes_retrieved"`

	// Bandwidth metrics
	NetworkRequests int64 `json:"network_requests"`
	BytesDownloaded int64 `json:"bytes_downloaded"`
	BytesServed     int64 `json:"bytes_served"`
	BandwidthSaved  int64 `json:"bandwidth_saved"`

	// Operation counts by type
	ManifestGets   int64 `json:"manifest_gets"`
	ManifestPuts   int64 `json:"manifest_puts"`
	BlobGets       int64 `json:"blob_gets"`
	BlobPuts       int64 `json:"blob_puts"`
	TagResolutions int64 `json:"tag_resolutions"`

	// Latency metrics (in nanoseconds)
	AverageGetLatency    time.Duration `json:"avg_get_latency_ns"`
	AveragePutLatency    time.Duration `json:"avg_put_latency_ns"`
	AverageDeleteLatency time.Duration `json:"avg_delete_latency_ns"`

	// Time-based metrics
	Uptime                time.Duration `json:"uptime"`
	TimeSinceLastHit      time.Duration `json:"time_since_last_hit"`
	TimeSinceLastMiss     time.Duration `json:"time_since_last_miss"`
	TimeSinceLastEviction time.Duration `json:"time_since_last_eviction"`
	TimeSinceLastError    time.Duration `json:"time_since_last_error"`

	// Peak usage tracking
	PeakBytesStored   int64   `json:"peak_bytes_stored"`
	PeakEntriesStored int64   `json:"peak_entries_stored"`
	PeakHitRate       float64 `json:"peak_hit_rate"`

	// Sample counts for latency calculations
	GetLatencySamples    int `json:"get_latency_samples"`
	PutLatencySamples    int `json:"put_latency_samples"`
	DeleteLatencySamples int `json:"delete_latency_samples"`
}

MetricsSnapshot provides a point-in-time view of cache metrics.

type Operation

type Operation string

Operation represents different types of cache operations for logging.

const (
	OpGetManifest    Operation = "get_manifest"
	OpPutManifest    Operation = "put_manifest"
	OpGetBlob        Operation = "get_blob"
	OpPutBlob        Operation = "put_blob"
	OpDeleteEntry    Operation = "delete_entry"
	OpEvictEntry     Operation = "evict_entry"
	OpCleanupExpired Operation = "cleanup_expired"
	OpValidateDigest Operation = "validate_digest"
)

Operation constants for cache operations

type SizeEviction

type SizeEviction struct {
	// contains filtered or unexported fields
}

SizeEviction implements size-based eviction when cache exceeds configured limits. It prioritizes evicting larger entries and expired entries.

func NewSizeEviction

func NewSizeEviction(maxSizeBytes int64) *SizeEviction

NewSizeEviction creates a new size-based eviction strategy.

func (*SizeEviction) OnAccess

func (s *SizeEviction) OnAccess(entry *Entry)

OnAccess is called when a cache entry is accessed. For size-based eviction, this is a no-op since we don't track access patterns.

func (*SizeEviction) OnAdd

func (s *SizeEviction) OnAdd(entry *Entry)

OnAdd is called when a new entry is added to the cache. For size-based eviction, this is a no-op since we don't maintain state.

func (*SizeEviction) OnRemove

func (s *SizeEviction) OnRemove(entry *Entry)

OnRemove is called when an entry is removed from the cache. For size-based eviction, this is a no-op since we don't maintain state.

func (*SizeEviction) SelectForEviction

func (s *SizeEviction) SelectForEviction(entries map[string]*Entry) []string

SelectForEviction chooses entries to evict when cache size exceeds limits. Prioritizes expired entries, then largest entries.

type Stats

type Stats struct {
	TotalEntries       int       `json:"total_entries"`
	ExpiredEntries     int       `json:"expired_entries"`
	TotalSize          int64     `json:"total_size_bytes"`
	MaxSize            int64     `json:"max_size_bytes"`
	HitRate            float64   `json:"hit_rate"`
	Evictions          int64     `json:"total_evictions"`
	Errors             int64     `json:"total_errors"`
	LastCompaction     time.Time `json:"last_compaction"`
	AverageAccessCount float64   `json:"average_access_count"`
}

Stats provides comprehensive statistics about the cache state.

type Storage

type Storage struct {
	// contains filtered or unexported fields
}

Storage provides atomic, corruption-resistant filesystem operations for cache storage. It uses billyfs.FS for filesystem abstraction, supporting both OS and in-memory filesystems.

func NewStorage

func NewStorage(fs *billyfs.FS, rootPath string) (*Storage, error)

NewStorage creates a new storage instance with the given filesystem and root path. The filesystem abstraction allows testing with in-memory filesystems.

func (*Storage) CleanupTempFiles

func (s *Storage) CleanupTempFiles(ctx context.Context) error

CleanupTempFiles removes any leftover temporary files from failed operations.

func (*Storage) Exists

func (s *Storage) Exists(ctx context.Context, path string) (bool, error)

Exists checks if a file exists in the storage.

func (*Storage) ListFiles

func (s *Storage) ListFiles(ctx context.Context, dirPath string) ([]string, error)

ListFiles returns a list of files in the given directory path.

func (*Storage) NewStreamWriter

func (s *Storage) NewStreamWriter(ctx context.Context, path string) (*StreamWriter, error)

NewStreamWriter creates a new streaming writer for the given path.

func (*Storage) ReadWithIntegrity

func (s *Storage) ReadWithIntegrity(ctx context.Context, path string) ([]byte, error)

ReadWithIntegrity reads data from a file and verifies its integrity using checksums.

func (*Storage) Remove

func (s *Storage) Remove(ctx context.Context, path string) error

Remove removes a file from the storage.

func (*Storage) Size

func (s *Storage) Size(ctx context.Context) (int64, error)

Size returns the total size of all files in the storage.

func (*Storage) WriteAtomically

func (s *Storage) WriteAtomically(ctx context.Context, path string, data []byte) error

WriteAtomically writes data to a file atomically using a temporary file and rename. This ensures that either the complete file is written or nothing is written, preventing partial/corrupted files from being visible to readers.

type StorageDebugInfo

type StorageDebugInfo struct {
	TotalFiles     int       `json:"total_files"`
	TotalSize      int64     `json:"total_size"`
	CachePath      string    `json:"cache_path"`
	Subdirectories []DirInfo `json:"subdirectories"`
}

StorageDebugInfo provides information about storage usage.

type StreamWriter

type StreamWriter struct {
	// contains filtered or unexported fields
}

StreamWriter provides streaming write operations with integrity verification.

func (*StreamWriter) Close

func (sw *StreamWriter) Close() error

Close finalizes the stream write operation atomically.

func (*StreamWriter) Size

func (sw *StreamWriter) Size() int64

Size returns the current size of the written data.

func (*StreamWriter) Write

func (sw *StreamWriter) Write(data []byte) (int, error)

Write writes data to the stream writer.

type TTLEviction

type TTLEviction struct{}

TTLEviction implements TTL-based cleanup for expired entries. It identifies and removes entries that have exceeded their time-to-live.

func NewTTLEviction

func NewTTLEviction() *TTLEviction

NewTTLEviction creates a new TTL-based eviction strategy.

func (*TTLEviction) OnAccess

func (t *TTLEviction) OnAccess(entry *Entry)

OnAccess is called when a cache entry is accessed. For TTL eviction, updates the access time but doesn't affect eviction priority.

func (*TTLEviction) OnAdd

func (t *TTLEviction) OnAdd(entry *Entry)

OnAdd is called when a new entry is added to the cache. For TTL eviction, this is a no-op since we check expiration on demand.

func (*TTLEviction) OnRemove

func (t *TTLEviction) OnRemove(entry *Entry)

OnRemove is called when an entry is removed from the cache. For TTL eviction, this is a no-op since we don't maintain state.

func (*TTLEviction) SelectForEviction

func (t *TTLEviction) SelectForEviction(entries map[string]*Entry) []string

SelectForEviction identifies all expired entries for cleanup. Returns expired entries ordered by expiration time (oldest first).

type TagCache

type TagCache interface {
	// GetTagMapping retrieves the current digest for a tag reference.
	// Returns the tag mapping or an error if not found or expired.
	GetTagMapping(ctx context.Context, reference string) (*TagMapping, error)

	// PutTagMapping stores or updates a tag-to-digest mapping.
	// Creates history entries when updating existing mappings.
	PutTagMapping(ctx context.Context, reference, digest string) error

	// HasTagMapping checks if a tag mapping exists and is not expired.
	// This is more efficient than GetTagMapping when only existence matters.
	HasTagMapping(ctx context.Context, reference string) (bool, error)

	// DeleteTagMapping removes a tag mapping from the cache.
	// Returns nil if the mapping doesn't exist (idempotent operation).
	DeleteTagMapping(ctx context.Context, reference string) error

	// GetTagHistory retrieves the history of digests for a tag reference.
	// Returns a slice of historical entries in chronological order (oldest first).
	GetTagHistory(ctx context.Context, reference string) ([]TagHistoryEntry, error)
}

TagCache defines operations for caching tag-to-digest mappings. Tags are mutable references that can change frequently, requiring efficient resolution with TTL-based caching and history tracking.

type TagHistoryEntry

type TagHistoryEntry struct {
	// Digest the tag previously pointed to
	Digest string `json:"digest"`
	// ChangedAt when this change occurred
	ChangedAt time.Time `json:"changed_at"`
}

TagHistoryEntry represents a historical mapping of a tag to a digest.

type TagMapping

type TagMapping struct {
	// Reference is the tag reference (e.g., "myregistry.com/myimage:latest")
	Reference string `json:"reference"`
	// Digest is the content digest this tag currently points to
	Digest string `json:"digest"`
	// CreatedAt is when this mapping was first created
	CreatedAt time.Time `json:"created_at"`
	// UpdatedAt is when this mapping was last updated
	UpdatedAt time.Time `json:"updated_at"`
	// AccessCount tracks how many times this mapping has been accessed
	AccessCount int64 `json:"access_count"`
	// History contains previous digests this tag has pointed to
	History []TagHistoryEntry `json:"history,omitempty"`
}

TagMapping represents a mapping from a tag reference to a digest. This enables efficient tag resolution with history tracking and TTL management.

type TagResolver

type TagResolver struct {
	// contains filtered or unexported fields
}

TagResolver provides efficient tag resolution with multi-tier validation. It implements caching, HEAD request optimization, and tag movement detection.

func NewTagResolver

func NewTagResolver(
	tagCache TagCache,
	manifestCache ManifestCache,
	config TagResolverConfig,
) *TagResolver

NewTagResolver creates a new tag resolver with the given caches and configuration.

func (*TagResolver) BatchResolveTags

func (tr *TagResolver) BatchResolveTags(
	ctx context.Context,
	references []string,
) (map[string]string, error)

BatchResolveTags resolves multiple tag references efficiently. Uses concurrent requests and caches results.

func (*TagResolver) DetectTagMovement

func (tr *TagResolver) DetectTagMovement(
	ctx context.Context,
	reference, expectedDigest string,
) (bool, error)

DetectTagMovement checks if a tag has moved to a different digest. This is useful for cache invalidation when external changes are detected.

func (*TagResolver) GetTagHistory

func (tr *TagResolver) GetTagHistory(
	ctx context.Context,
	reference string,
) ([]TagHistoryEntry, error)

GetTagHistory retrieves the history of digests for a tag reference.

func (*TagResolver) HealthCheck

func (tr *TagResolver) HealthCheck(ctx context.Context) error

HealthCheck performs a basic health check of the resolver. Returns an error if the resolver is not functioning properly.

func (*TagResolver) InvalidateTag

func (tr *TagResolver) InvalidateTag(ctx context.Context, reference string) error

InvalidateTag removes a tag mapping from the cache. Useful when tag movement is detected externally.

func (*TagResolver) ResolveTag

func (tr *TagResolver) ResolveTag(ctx context.Context, reference string) (string, error)

ResolveTag resolves a tag reference to its digest using multi-tier validation. Strategy: 1. Check local cache first 2. If cache miss or expired, perform HEAD request to registry 3. If HEAD request succeeds, update cache and return digest 4. Handle tag movement detection and cache invalidation

func (*TagResolver) ValidateTag

func (tr *TagResolver) ValidateTag(ctx context.Context, reference string) (bool, error)

ValidateTag performs comprehensive validation of a tag reference. Returns true if the tag exists and points to valid content.

type TagResolverConfig

type TagResolverConfig struct {
	// DefaultTTL is the default TTL for tag mappings
	DefaultTTL time.Duration
	// MaxHistorySize is the maximum number of historical entries to keep per tag
	MaxHistorySize int
	// EnableHistory enables tracking of tag history (defaults to true)
	EnableHistory bool
}

TagResolverConfig contains configuration for tag resolution behavior.

func (*TagResolverConfig) SetDefaults

func (c *TagResolverConfig) SetDefaults()

SetDefaults applies default values to unset fields in the tag resolver configuration.

func (*TagResolverConfig) Validate

func (c *TagResolverConfig) Validate() error

Validate checks that the tag resolver configuration is valid.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL