Documentation
¶
Overview ¶
Package cache provides a comprehensive caching system for OCI (Open Container Initiative) artifacts.
This package implements a multi-tier caching strategy optimized for OCI registries, with separate handling for manifests and blobs. The cache system is designed to:
- Reduce network requests to OCI registries
- Provide configurable TTL (Time-To-Live) for cache entries
- Store already-compressed OCI artifacts efficiently (no double compression)
- Handle concurrent access safely
- Provide detailed metrics and observability
- Support pluggable eviction strategies
Architecture Overview ¶
The cache system consists of several key components:
- CacheManager: Central coordinator that manages cache configuration and metrics
- Cache: Core interface for basic cache operations (get, put, delete, clear)
- ManifestCache: Specialized interface for OCI manifest caching with validation
- BlobCache: Specialized interface for OCI blob caching with streaming support
- EvictionStrategy: Pluggable strategies for cache size management
Cache Entry Lifecycle ¶
1. **Creation**: Entries are created with a key, data, and optional TTL 2. **Storage**: Entries are stored directly with metadata (OCI artifacts are pre-compressed) 3. **Access**: Entries track access patterns for eviction decisions 4. **Expiration**: Entries can expire based on TTL or be explicitly invalidated 5. **Eviction**: Entries may be evicted when cache size limits are reached
Error Handling ¶
The package defines specific errors for different cache failure scenarios:
- ErrCacheExpired: Entry exists but has exceeded its TTL
- ErrCacheCorrupted: Entry data is corrupted or unreadable
- ErrCacheFull: Cache cannot accept new entries due to size limits
- ErrCacheInvalidated: Entry has been explicitly invalidated
All errors support proper wrapping with context using fmt.Errorf and %w.
Configuration ¶
Cache behavior is controlled through CacheConfig:
- MaxSizeBytes: Maximum cache size in bytes
- DefaultTTL: Default time-to-live for entries
Metrics and Observability ¶
The CacheMetrics struct tracks:
- Hit/miss ratios
- Eviction counts
- Error rates
- Storage utilization
Thread Safety ¶
All cache implementations must be safe for concurrent use by multiple goroutines. The package uses appropriate synchronization primitives to ensure data consistency.
Implementation Notes ¶
This package follows Go best practices:
- Interfaces are defined at the point of use
- Errors are wrapped with context
- Configuration is validated on creation
- Tests achieve high coverage with race detection
- Code follows the Go style guide from docs/guides/go/style.md
Index ¶
- Constants
- Variables
- func LogCacheHit(ctx context.Context, logger *Logger, operation Operation, size int64)
- func LogCacheMiss(ctx context.Context, logger *Logger, operation Operation, reason string)
- func LogCacheOperation(ctx context.Context, logger *Logger, operation Operation, ...)
- func LogCleanup(ctx context.Context, logger *Logger, operation string, entriesRemoved int, ...)
- func LogEviction(ctx context.Context, logger *Logger, key string, size int64, reason string)
- func LogPerformanceMetrics(ctx context.Context, logger *Logger, metrics *MetricsSnapshot)
- func NewManifestCache(storage *Storage, manager ConfigProvider) *manifestCache
- func NewTagCache(storage *Storage, config TagResolverConfig) *tagCache
- type BlobCache
- type Cache
- type CompositeEviction
- type Config
- type ConfigProvider
- type Coordinator
- func (cm *Coordinator) Clear(ctx context.Context) error
- func (cm *Coordinator) Close() error
- func (cm *Coordinator) Config() Config
- func (cm *Coordinator) GetBlob(ctx context.Context, digest string) (io.ReadCloser, error)
- func (cm *Coordinator) GetManifest(ctx context.Context, digest string) (*ocispec.Manifest, error)
- func (cm *Coordinator) GetMetrics() *DetailedMetrics
- func (cm *Coordinator) GetStats() Stats
- func (cm *Coordinator) PutBlob(ctx context.Context, digest string, reader io.Reader) error
- func (cm *Coordinator) PutManifest(ctx context.Context, digest string, manifest *ocispec.Manifest) error
- func (cm *Coordinator) Size(ctx context.Context) (int64, error)
- type DebugInfo
- type DebugTools
- func (dt *DebugTools) ClearExpiredEntries(ctx context.Context) (int, error)
- func (dt *DebugTools) CollectDebugInfo(ctx context.Context) (*DebugInfo, error)
- func (dt *DebugTools) ExportDebugInfo(ctx context.Context, outputPath string) error
- func (dt *DebugTools) GetCacheContents(ctx context.Context) ([]EntryInfo, error)
- func (dt *DebugTools) RepairIntegrity(ctx context.Context) (*IntegrityReport, error)
- func (dt *DebugTools) TriggerCleanup(ctx context.Context) error
- func (dt *DebugTools) ValidateAllDigests(ctx context.Context) (int, []string, error)
- type DetailedMetrics
- func (m *DetailedMetrics) GetSnapshot() MetricsSnapshot
- func (m *DetailedMetrics) RecordDelete(bytesRemoved int64)
- func (m *DetailedMetrics) RecordError()
- func (m *DetailedMetrics) RecordEviction(bytesEvicted int64)
- func (m *DetailedMetrics) RecordHit(operationType string, bytesServed int64)
- func (m *DetailedMetrics) RecordLatency(operation string, duration time.Duration)
- func (m *DetailedMetrics) RecordMiss(operationType string, bytesDownloaded int64)
- func (m *DetailedMetrics) RecordPut(operationType string, bytesStored int64)
- func (m *DetailedMetrics) RecordTagResolution()
- func (m *DetailedMetrics) Reset()
- type DirInfo
- type Entry
- type EntryInfo
- type EvictionStrategy
- type Index
- func (idx *Index) Cleanup(ctx context.Context, forceCompact bool) error
- func (idx *Index) Compact() error
- func (idx *Index) Delete(key string) error
- func (idx *Index) ExpiredKeys() []string
- func (idx *Index) Get(key string) (*IndexEntry, bool)
- func (idx *Index) Keys(filter func(*IndexEntry) bool) []string
- func (idx *Index) Load(ctx context.Context) error
- func (idx *Index) NewIterator(filter func(*IndexEntry) bool) *Iterator
- func (idx *Index) Persist() error
- func (idx *Index) Put(key string, entry *IndexEntry) error
- func (idx *Index) Size() int
- func (idx *Index) Stats() IndexStats
- type IndexEntry
- type IndexStats
- type IntegrityReport
- type Iterator
- type LRUEviction
- type LogConfig
- type LogLevel
- type Logger
- func (l *Logger) Debug(ctx context.Context, msg string, args ...any)
- func (l *Logger) Error(ctx context.Context, msg string, args ...any)
- func (l *Logger) Info(ctx context.Context, msg string, args ...any)
- func (l *Logger) Warn(ctx context.Context, msg string, args ...any)
- func (l *Logger) With(args ...any) *Logger
- func (l *Logger) WithDigest(digest string) *Logger
- func (l *Logger) WithDuration(duration time.Duration) *Logger
- func (l *Logger) WithOperation(operation string) *Logger
- func (l *Logger) WithSize(size int64) *Logger
- type ManifestCache
- type MetricsSnapshot
- type Operation
- type SizeEviction
- type Stats
- type Storage
- func (s *Storage) CleanupTempFiles(ctx context.Context) error
- func (s *Storage) Exists(ctx context.Context, path string) (bool, error)
- func (s *Storage) ListFiles(ctx context.Context, dirPath string) ([]string, error)
- func (s *Storage) NewStreamWriter(ctx context.Context, path string) (*StreamWriter, error)
- func (s *Storage) ReadWithIntegrity(ctx context.Context, path string) ([]byte, error)
- func (s *Storage) Remove(ctx context.Context, path string) error
- func (s *Storage) Size(ctx context.Context) (int64, error)
- func (s *Storage) WriteAtomically(ctx context.Context, path string, data []byte) error
- type StorageDebugInfo
- type StreamWriter
- type TTLEviction
- type TagCache
- type TagHistoryEntry
- type TagMapping
- type TagResolver
- func (tr *TagResolver) BatchResolveTags(ctx context.Context, references []string) (map[string]string, error)
- func (tr *TagResolver) DetectTagMovement(ctx context.Context, reference, expectedDigest string) (bool, error)
- func (tr *TagResolver) GetTagHistory(ctx context.Context, reference string) ([]TagHistoryEntry, error)
- func (tr *TagResolver) HealthCheck(ctx context.Context) error
- func (tr *TagResolver) InvalidateTag(ctx context.Context, reference string) error
- func (tr *TagResolver) ResolveTag(ctx context.Context, reference string) (string, error)
- func (tr *TagResolver) ValidateTag(ctx context.Context, reference string) (bool, error)
- type TagResolverConfig
Constants ¶
const ( OperationManifest = "manifest" OperationBlob = "blob" )
Operation types for metrics tracking
Variables ¶
var ErrCacheCorrupted = errors.New("cache entry is corrupted")
ErrCacheCorrupted is returned when a cache entry is found to be corrupted.
var ErrCacheExpired = errors.New("cache entry has expired")
ErrCacheExpired is returned when attempting to access an expired cache entry.
var ErrCacheFull = errors.New("cache is full")
ErrCacheFull is returned when the cache cannot store additional entries due to size limits.
var ErrCacheInvalidated = errors.New("cache entry has been invalidated")
ErrCacheInvalidated is returned when a cache entry has been invalidated and can no longer be used.
Functions ¶
func LogCacheHit ¶
LogCacheHit logs a cache hit event.
func LogCacheMiss ¶
LogCacheMiss logs a cache miss event.
func LogCacheOperation ¶
func LogCacheOperation( ctx context.Context, logger *Logger, operation Operation, duration time.Duration, success bool, size int64, err error, )
LogCacheOperation logs a cache operation with performance metrics.
func LogCleanup ¶
func LogCleanup( ctx context.Context, logger *Logger, operation string, entriesRemoved int, bytesFreed int64, duration time.Duration, )
LogCleanup logs cleanup operations.
func LogEviction ¶
LogEviction logs an eviction event.
func LogPerformanceMetrics ¶
func LogPerformanceMetrics(ctx context.Context, logger *Logger, metrics *MetricsSnapshot)
LogPerformanceMetrics logs periodic performance metrics.
func NewManifestCache ¶
func NewManifestCache(storage *Storage, manager ConfigProvider) *manifestCache
NewManifestCache creates a new manifest cache instance.
func NewTagCache ¶
func NewTagCache(storage *Storage, config TagResolverConfig) *tagCache
NewTagCache creates a new tag cache instance.
Types ¶
type BlobCache ¶
type BlobCache interface {
// GetBlob retrieves a blob by its digest.
// Returns a ReadCloser that the caller must close.
GetBlob(ctx context.Context, digest string) (io.ReadCloser, error)
// PutBlob stores a blob with the given digest.
// The reader content is consumed and stored.
PutBlob(ctx context.Context, digest string, reader io.Reader) error
// HasBlob checks if a blob exists in the cache.
// This is more efficient than GetBlob when only existence matters.
HasBlob(ctx context.Context, digest string) (bool, error)
// DeleteBlob removes a blob from the cache.
// Returns nil if the blob doesn't exist (idempotent operation).
DeleteBlob(ctx context.Context, digest string) error
}
BlobCache defines operations specific to OCI blob caching. Blobs can be large and benefit from streaming operations.
type Cache ¶
type Cache interface {
// Get retrieves a cache entry by key.
// Returns ErrCacheExpired if the entry exists but has expired.
// Returns an error if the key is not found or if retrieval fails.
Get(ctx context.Context, key string) (*Entry, error)
// Put stores a cache entry with the given key.
// Returns ErrCacheFull if the cache cannot accommodate the entry.
// Overwrites existing entries with the same key.
Put(ctx context.Context, key string, entry *Entry) error
// Delete removes a cache entry by key.
// Returns nil if the key doesn't exist (idempotent operation).
Delete(ctx context.Context, key string) error
// Clear removes all entries from the cache.
Clear(ctx context.Context) error
// Size returns the total size of all cache entries in bytes.
Size(ctx context.Context) (int64, error)
}
Cache defines the core interface for cache operations. Implementations should be safe for concurrent use by multiple goroutines.
type CompositeEviction ¶
type CompositeEviction struct {
// contains filtered or unexported fields
}
CompositeEviction combines multiple eviction strategies with configurable priorities. It allows different strategies to work together for optimal cache management.
func NewCompositeEviction ¶
func NewCompositeEviction(strategies []EvictionStrategy, priorities []int) *CompositeEviction
NewCompositeEviction creates a new composite eviction strategy.
func (*CompositeEviction) OnAccess ¶
func (c *CompositeEviction) OnAccess(entry *Entry)
OnAccess notifies all strategies of an access event.
func (*CompositeEviction) OnAdd ¶
func (c *CompositeEviction) OnAdd(entry *Entry)
OnAdd notifies all strategies of an add event.
func (*CompositeEviction) OnRemove ¶
func (c *CompositeEviction) OnRemove(entry *Entry)
OnRemove notifies all strategies of a remove event.
func (*CompositeEviction) SelectForEviction ¶
func (c *CompositeEviction) SelectForEviction(entries map[string]*Entry) []string
SelectForEviction combines results from all strategies based on priority. Higher priority strategies (lower priority numbers) are processed first.
type Config ¶
type Config struct {
// MaxSizeBytes is the maximum size of the cache in bytes.
MaxSizeBytes int64
// DefaultTTL is the default time-to-live for cache entries.
DefaultTTL time.Duration
}
Config holds configuration for cache behavior.
func (*Config) SetDefaults ¶
func (c *Config) SetDefaults()
SetDefaults applies default values to unset fields in the configuration.
type ConfigProvider ¶
type ConfigProvider interface {
Config() Config
}
ConfigProvider defines the interface that cache components need from a manager.
type Coordinator ¶
type Coordinator struct {
// contains filtered or unexported fields
}
Coordinator coordinates multiple cache types and implements comprehensive cache management. It provides a unified interface for manifest and blob caching with size limits, eviction, and metrics collection.
func NewCoordinator ¶
func NewCoordinator( ctx context.Context, config Config, fs *billyfs.FS, cachePath string, logger *Logger, ) (*Coordinator, error)
NewCoordinator creates a new cache coordinator with the specified configuration. It initializes all cache components and starts background cleanup processes.
func (*Coordinator) Clear ¶
func (cm *Coordinator) Clear(ctx context.Context) error
Clear removes all entries from all caches.
func (*Coordinator) Close ¶
func (cm *Coordinator) Close() error
Close shuts down the cache manager and cleans up resources.
func (*Coordinator) Config ¶
func (cm *Coordinator) Config() Config
Config returns the cache configuration.
func (*Coordinator) GetBlob ¶
func (cm *Coordinator) GetBlob( ctx context.Context, digest string, ) (io.ReadCloser, error)
GetBlob retrieves a blob from the cache.
func (*Coordinator) GetManifest ¶
func (cm *Coordinator) GetManifest( ctx context.Context, digest string, ) (*ocispec.Manifest, error)
GetManifest retrieves a manifest from the cache.
func (*Coordinator) GetMetrics ¶
func (cm *Coordinator) GetMetrics() *DetailedMetrics
GetMetrics returns current cache metrics.
func (*Coordinator) GetStats ¶
func (cm *Coordinator) GetStats() Stats
GetStats returns comprehensive cache statistics.
func (*Coordinator) PutManifest ¶
func (cm *Coordinator) PutManifest( ctx context.Context, digest string, manifest *ocispec.Manifest, ) error
PutManifest stores a manifest in the cache.
type DebugInfo ¶
type DebugInfo struct {
Timestamp time.Time `json:"timestamp"`
Config Config `json:"config"`
Metrics MetricsSnapshot `json:"metrics"`
IndexStats IndexStats `json:"index_stats"`
CacheStats Stats `json:"cache_stats"`
TopEntries []EntryInfo `json:"top_entries"`
Integrity IntegrityReport `json:"integrity"`
StorageInfo StorageDebugInfo `json:"storage_info"`
}
DebugInfo provides comprehensive debugging information about the cache state.
type DebugTools ¶
type DebugTools struct {
// contains filtered or unexported fields
}
DebugTools provides debugging and maintenance utilities for the cache.
func NewDebugTools ¶
func NewDebugTools(coordinator *Coordinator) *DebugTools
NewDebugTools creates a new debug tools instance for the given coordinator.
func (*DebugTools) ClearExpiredEntries ¶
func (dt *DebugTools) ClearExpiredEntries(ctx context.Context) (int, error)
ClearExpiredEntries removes all expired entries from the cache.
func (*DebugTools) CollectDebugInfo ¶
func (dt *DebugTools) CollectDebugInfo(ctx context.Context) (*DebugInfo, error)
CollectDebugInfo gathers comprehensive debugging information about the cache.
func (*DebugTools) ExportDebugInfo ¶
func (dt *DebugTools) ExportDebugInfo(ctx context.Context, outputPath string) error
ExportDebugInfo exports debug information to a JSON file.
func (*DebugTools) GetCacheContents ¶
func (dt *DebugTools) GetCacheContents(ctx context.Context) ([]EntryInfo, error)
GetCacheContents returns a list of all cache entries with their metadata.
func (*DebugTools) RepairIntegrity ¶
func (dt *DebugTools) RepairIntegrity(ctx context.Context) (*IntegrityReport, error)
RepairIntegrity attempts to repair integrity issues found during checks.
func (*DebugTools) TriggerCleanup ¶
func (dt *DebugTools) TriggerCleanup(ctx context.Context) error
TriggerCleanup manually triggers cache cleanup operations.
func (*DebugTools) ValidateAllDigests ¶
ValidateAllDigests performs digest validation on all cache entries.
type DetailedMetrics ¶
type DetailedMetrics struct {
// contains filtered or unexported fields
}
DetailedMetrics provides comprehensive metrics collection for cache operations. It tracks performance, bandwidth savings, and operational statistics.
func NewDetailedMetrics ¶
func NewDetailedMetrics() *DetailedMetrics
NewDetailedMetrics creates a new DetailedMetrics instance.
func (*DetailedMetrics) GetSnapshot ¶
func (m *DetailedMetrics) GetSnapshot() MetricsSnapshot
GetSnapshot returns a thread-safe snapshot of current metrics.
func (*DetailedMetrics) RecordDelete ¶
func (m *DetailedMetrics) RecordDelete(bytesRemoved int64)
RecordDelete records a deletion operation.
func (*DetailedMetrics) RecordError ¶
func (m *DetailedMetrics) RecordError()
RecordError records an operation error.
func (*DetailedMetrics) RecordEviction ¶
func (m *DetailedMetrics) RecordEviction(bytesEvicted int64)
RecordEviction records an eviction operation.
func (*DetailedMetrics) RecordHit ¶
func (m *DetailedMetrics) RecordHit(operationType string, bytesServed int64)
RecordHit records a cache hit and updates relevant metrics.
func (*DetailedMetrics) RecordLatency ¶
func (m *DetailedMetrics) RecordLatency(operation string, duration time.Duration)
RecordLatency records the latency of an operation.
func (*DetailedMetrics) RecordMiss ¶
func (m *DetailedMetrics) RecordMiss(operationType string, bytesDownloaded int64)
RecordMiss records a cache miss and updates network usage metrics.
func (*DetailedMetrics) RecordPut ¶
func (m *DetailedMetrics) RecordPut(operationType string, bytesStored int64)
RecordPut records a successful put operation.
func (*DetailedMetrics) RecordTagResolution ¶
func (m *DetailedMetrics) RecordTagResolution()
RecordTagResolution records a tag resolution operation.
type DirInfo ¶
type DirInfo struct {
Path string `json:"path"`
FileCount int `json:"file_count"`
TotalSize int64 `json:"total_size"`
}
DirInfo provides information about a subdirectory.
type Entry ¶
type Entry struct {
// Key is the unique identifier for this cache entry.
Key string
// Data contains the cached data.
Data []byte
// Metadata contains additional metadata about the entry.
Metadata map[string]string
// CreatedAt is when this entry was first created.
CreatedAt time.Time
// AccessedAt is when this entry was last accessed.
AccessedAt time.Time
// TTL is the time-to-live for this entry. Zero means no expiration.
TTL time.Duration
// AccessCount tracks how many times this entry has been accessed.
AccessCount int64
}
Entry represents a single entry in the cache.
type EntryInfo ¶
type EntryInfo struct {
Key string `json:"key"`
Size int64 `json:"size"`
CreatedAt time.Time `json:"created_at"`
AccessedAt time.Time `json:"accessed_at"`
TTL time.Duration `json:"ttl"`
AccessCount int64 `json:"access_count"`
FilePath string `json:"file_path"`
IsExpired bool `json:"is_expired"`
Type string `json:"type"` // "manifest" or "blob"
}
EntryInfo provides detailed information about a cache entry.
type EvictionStrategy ¶
type EvictionStrategy interface {
// SelectForEviction chooses which cache entries should be evicted.
// Returns a slice of keys to evict, ordered by eviction priority.
// The entries map provides access to all current cache entries for decision making.
SelectForEviction(entries map[string]*Entry) []string
// OnAccess is called when a cache entry is accessed (read).
// Implementations can update access patterns or priorities.
OnAccess(entry *Entry)
// OnAdd is called when a new entry is added to the cache.
// Implementations can initialize eviction metadata.
OnAdd(entry *Entry)
// OnRemove is called when an entry is removed from the cache.
// Implementations can clean up eviction metadata.
OnRemove(entry *Entry)
}
EvictionStrategy defines how entries are selected for eviction when cache size limits are reached. Implementations should be deterministic and thread-safe.
type Index ¶
type Index struct {
// contains filtered or unexported fields
}
Index manages in-memory cache index with persistence and recovery capabilities.
func (*Index) ExpiredKeys ¶
ExpiredKeys returns keys of all expired entries.
func (*Index) Get ¶
func (idx *Index) Get(key string) (*IndexEntry, bool)
Get retrieves an index entry by key.
func (*Index) Keys ¶
func (idx *Index) Keys(filter func(*IndexEntry) bool) []string
Keys returns all keys in the index, optionally filtered by a predicate.
func (*Index) NewIterator ¶
func (idx *Index) NewIterator(filter func(*IndexEntry) bool) *Iterator
NewIterator creates a new iterator over the index entries.
func (*Index) Put ¶
func (idx *Index) Put(key string, entry *IndexEntry) error
Put stores or updates an index entry.
type IndexEntry ¶
type IndexEntry struct {
Key string `json:"key"`
Size int64 `json:"size"`
CreatedAt time.Time `json:"created_at"`
AccessedAt time.Time `json:"accessed_at"`
TTL time.Duration `json:"ttl"`
AccessCount int64 `json:"access_count"`
Metadata map[string]string `json:"metadata,omitempty"`
FilePath string `json:"file_path"` // Relative path to the data file
}
IndexEntry represents a single entry in the cache index.
func (*IndexEntry) IsExpired ¶
func (ie *IndexEntry) IsExpired() bool
IsExpired returns true if the index entry has expired.
type IndexStats ¶
type IndexStats struct {
TotalEntries int
ExpiredEntries int
TotalSize int64
LastCompaction time.Time
AverageAccessCount float64
}
IndexStats returns statistics about the index.
type IntegrityReport ¶
type IntegrityReport struct {
TotalEntries int `json:"total_entries"`
CorruptedEntries []string `json:"corrupted_entries"`
MissingFiles []string `json:"missing_files"`
InvalidDigests []string `json:"invalid_digests"`
IsHealthy bool `json:"is_healthy"`
CheckDuration time.Duration `json:"check_duration"`
}
IntegrityReport provides information about cache integrity.
type Iterator ¶
type Iterator struct {
// contains filtered or unexported fields
}
Iterator allows iteration over index entries with optional filtering.
type LRUEviction ¶
type LRUEviction struct {
// contains filtered or unexported fields
}
LRUEviction implements an LRU (Least Recently Used) eviction strategy. It maintains access order using a doubly-linked list and provides O(1) access tracking.
func NewLRUEviction ¶
func NewLRUEviction() *LRUEviction
NewLRUEviction creates a new LRU eviction strategy.
func (*LRUEviction) OnAccess ¶
func (l *LRUEviction) OnAccess(entry *Entry)
OnAccess is called when a cache entry is accessed (read). Moves the entry to the front of the access list (most recently used).
func (*LRUEviction) OnAdd ¶
func (l *LRUEviction) OnAdd(entry *Entry)
OnAdd is called when a new entry is added to the cache. Adds the entry to the front of the access list.
func (*LRUEviction) OnRemove ¶
func (l *LRUEviction) OnRemove(entry *Entry)
OnRemove is called when an entry is removed from the cache. Removes the entry from the access tracking.
func (*LRUEviction) SelectForEviction ¶
func (l *LRUEviction) SelectForEviction(entries map[string]*Entry) []string
SelectForEviction chooses which cache entries should be evicted based on LRU policy. Returns keys ordered by eviction priority (oldest accessed first).
type LogConfig ¶
type LogConfig struct {
// Level sets the minimum log level (debug, info, warn, error)
Level LogLevel
// EnableCallerInfo includes file and line number in logs
EnableCallerInfo bool
// EnablePerformanceLogging enables detailed performance metrics logging
EnablePerformanceLogging bool
// EnableCacheOperations enables logging of individual cache operations
EnableCacheOperations bool
}
LogConfig holds configuration for the cache logger.
func DefaultLogConfig ¶
func DefaultLogConfig() LogConfig
DefaultLogConfig returns a default logging configuration.
type LogLevel ¶
type LogLevel int
LogLevel represents different logging levels
LogLevelDebug represents debug logging level
func ParseLogLevel ¶
ParseLogLevel parses a string log level into a LogLevel.
type Logger ¶
type Logger struct {
// contains filtered or unexported fields
}
Logger provides structured logging for the cache system. It wraps different logger implementations for consistent behavior.
func NewNopLogger ¶
func NewNopLogger() *Logger
NewNopLogger creates a no-op logger that discards all log messages.
func (*Logger) WithDigest ¶
WithDigest returns a logger with digest context
func (*Logger) WithDuration ¶
WithDuration returns a logger with duration context
func (*Logger) WithOperation ¶
WithOperation returns a logger with operation context
type ManifestCache ¶
type ManifestCache interface {
// GetManifest retrieves a manifest by its digest.
// Returns the manifest content or an error if not found.
GetManifest(ctx context.Context, digest string) (*ocispec.Manifest, error)
// PutManifest stores a manifest with the given digest.
// The digest should be validated before storage.
PutManifest(ctx context.Context, digest string, manifest *ocispec.Manifest) error
// HasManifest checks if a manifest exists in the cache.
// This is more efficient than GetManifest when only existence matters.
HasManifest(ctx context.Context, digest string) (bool, error)
// ValidateManifest performs validation of a manifest against a reference.
// This may involve checking the manifest digest against a registry.
ValidateManifest(ctx context.Context, reference, digest string) (bool, error)
}
ManifestCache defines operations specific to OCI manifest caching. Manifests are typically small and have short TTLs for freshness.
type MetricsSnapshot ¶
type MetricsSnapshot struct {
// Core hit/miss statistics
Hits int64 `json:"hits"`
Misses int64 `json:"misses"`
HitRate float64 `json:"hit_rate"`
// Eviction and error tracking
Evictions int64 `json:"evictions"`
Errors int64 `json:"errors"`
// Size and entry tracking
BytesStored int64 `json:"bytes_stored"`
EntriesStored int64 `json:"entries_stored"`
BytesRetrieved int64 `json:"bytes_retrieved"`
// Bandwidth metrics
NetworkRequests int64 `json:"network_requests"`
BytesDownloaded int64 `json:"bytes_downloaded"`
BytesServed int64 `json:"bytes_served"`
BandwidthSaved int64 `json:"bandwidth_saved"`
// Operation counts by type
ManifestGets int64 `json:"manifest_gets"`
ManifestPuts int64 `json:"manifest_puts"`
BlobGets int64 `json:"blob_gets"`
BlobPuts int64 `json:"blob_puts"`
TagResolutions int64 `json:"tag_resolutions"`
// Latency metrics (in nanoseconds)
AverageGetLatency time.Duration `json:"avg_get_latency_ns"`
AveragePutLatency time.Duration `json:"avg_put_latency_ns"`
AverageDeleteLatency time.Duration `json:"avg_delete_latency_ns"`
// Time-based metrics
Uptime time.Duration `json:"uptime"`
TimeSinceLastHit time.Duration `json:"time_since_last_hit"`
TimeSinceLastMiss time.Duration `json:"time_since_last_miss"`
TimeSinceLastEviction time.Duration `json:"time_since_last_eviction"`
TimeSinceLastError time.Duration `json:"time_since_last_error"`
// Peak usage tracking
PeakBytesStored int64 `json:"peak_bytes_stored"`
PeakEntriesStored int64 `json:"peak_entries_stored"`
PeakHitRate float64 `json:"peak_hit_rate"`
// Sample counts for latency calculations
GetLatencySamples int `json:"get_latency_samples"`
PutLatencySamples int `json:"put_latency_samples"`
DeleteLatencySamples int `json:"delete_latency_samples"`
}
MetricsSnapshot provides a point-in-time view of cache metrics.
type Operation ¶
type Operation string
Operation represents different types of cache operations for logging.
const ( OpGetManifest Operation = "get_manifest" OpPutManifest Operation = "put_manifest" OpGetBlob Operation = "get_blob" OpPutBlob Operation = "put_blob" OpDeleteEntry Operation = "delete_entry" OpEvictEntry Operation = "evict_entry" OpCleanupExpired Operation = "cleanup_expired" OpValidateDigest Operation = "validate_digest" )
Operation constants for cache operations
type SizeEviction ¶
type SizeEviction struct {
// contains filtered or unexported fields
}
SizeEviction implements size-based eviction when cache exceeds configured limits. It prioritizes evicting larger entries and expired entries.
func NewSizeEviction ¶
func NewSizeEviction(maxSizeBytes int64) *SizeEviction
NewSizeEviction creates a new size-based eviction strategy.
func (*SizeEviction) OnAccess ¶
func (s *SizeEviction) OnAccess(entry *Entry)
OnAccess is called when a cache entry is accessed. For size-based eviction, this is a no-op since we don't track access patterns.
func (*SizeEviction) OnAdd ¶
func (s *SizeEviction) OnAdd(entry *Entry)
OnAdd is called when a new entry is added to the cache. For size-based eviction, this is a no-op since we don't maintain state.
func (*SizeEviction) OnRemove ¶
func (s *SizeEviction) OnRemove(entry *Entry)
OnRemove is called when an entry is removed from the cache. For size-based eviction, this is a no-op since we don't maintain state.
func (*SizeEviction) SelectForEviction ¶
func (s *SizeEviction) SelectForEviction(entries map[string]*Entry) []string
SelectForEviction chooses entries to evict when cache size exceeds limits. Prioritizes expired entries, then largest entries.
type Stats ¶
type Stats struct {
TotalEntries int `json:"total_entries"`
ExpiredEntries int `json:"expired_entries"`
TotalSize int64 `json:"total_size_bytes"`
MaxSize int64 `json:"max_size_bytes"`
HitRate float64 `json:"hit_rate"`
Evictions int64 `json:"total_evictions"`
Errors int64 `json:"total_errors"`
LastCompaction time.Time `json:"last_compaction"`
AverageAccessCount float64 `json:"average_access_count"`
}
Stats provides comprehensive statistics about the cache state.
type Storage ¶
type Storage struct {
// contains filtered or unexported fields
}
Storage provides atomic, corruption-resistant filesystem operations for cache storage. It uses billyfs.FS for filesystem abstraction, supporting both OS and in-memory filesystems.
func NewStorage ¶
NewStorage creates a new storage instance with the given filesystem and root path. The filesystem abstraction allows testing with in-memory filesystems.
func (*Storage) CleanupTempFiles ¶
CleanupTempFiles removes any leftover temporary files from failed operations.
func (*Storage) NewStreamWriter ¶
NewStreamWriter creates a new streaming writer for the given path.
func (*Storage) ReadWithIntegrity ¶
ReadWithIntegrity reads data from a file and verifies its integrity using checksums.
func (*Storage) WriteAtomically ¶
WriteAtomically writes data to a file atomically using a temporary file and rename. This ensures that either the complete file is written or nothing is written, preventing partial/corrupted files from being visible to readers.
type StorageDebugInfo ¶
type StorageDebugInfo struct {
TotalFiles int `json:"total_files"`
TotalSize int64 `json:"total_size"`
CachePath string `json:"cache_path"`
Subdirectories []DirInfo `json:"subdirectories"`
}
StorageDebugInfo provides information about storage usage.
type StreamWriter ¶
type StreamWriter struct {
// contains filtered or unexported fields
}
StreamWriter provides streaming write operations with integrity verification.
func (*StreamWriter) Close ¶
func (sw *StreamWriter) Close() error
Close finalizes the stream write operation atomically.
func (*StreamWriter) Size ¶
func (sw *StreamWriter) Size() int64
Size returns the current size of the written data.
type TTLEviction ¶
type TTLEviction struct{}
TTLEviction implements TTL-based cleanup for expired entries. It identifies and removes entries that have exceeded their time-to-live.
func NewTTLEviction ¶
func NewTTLEviction() *TTLEviction
NewTTLEviction creates a new TTL-based eviction strategy.
func (*TTLEviction) OnAccess ¶
func (t *TTLEviction) OnAccess(entry *Entry)
OnAccess is called when a cache entry is accessed. For TTL eviction, updates the access time but doesn't affect eviction priority.
func (*TTLEviction) OnAdd ¶
func (t *TTLEviction) OnAdd(entry *Entry)
OnAdd is called when a new entry is added to the cache. For TTL eviction, this is a no-op since we check expiration on demand.
func (*TTLEviction) OnRemove ¶
func (t *TTLEviction) OnRemove(entry *Entry)
OnRemove is called when an entry is removed from the cache. For TTL eviction, this is a no-op since we don't maintain state.
func (*TTLEviction) SelectForEviction ¶
func (t *TTLEviction) SelectForEviction(entries map[string]*Entry) []string
SelectForEviction identifies all expired entries for cleanup. Returns expired entries ordered by expiration time (oldest first).
type TagCache ¶
type TagCache interface {
// GetTagMapping retrieves the current digest for a tag reference.
// Returns the tag mapping or an error if not found or expired.
GetTagMapping(ctx context.Context, reference string) (*TagMapping, error)
// PutTagMapping stores or updates a tag-to-digest mapping.
// Creates history entries when updating existing mappings.
PutTagMapping(ctx context.Context, reference, digest string) error
// HasTagMapping checks if a tag mapping exists and is not expired.
// This is more efficient than GetTagMapping when only existence matters.
HasTagMapping(ctx context.Context, reference string) (bool, error)
// DeleteTagMapping removes a tag mapping from the cache.
// Returns nil if the mapping doesn't exist (idempotent operation).
DeleteTagMapping(ctx context.Context, reference string) error
// GetTagHistory retrieves the history of digests for a tag reference.
// Returns a slice of historical entries in chronological order (oldest first).
GetTagHistory(ctx context.Context, reference string) ([]TagHistoryEntry, error)
}
TagCache defines operations for caching tag-to-digest mappings. Tags are mutable references that can change frequently, requiring efficient resolution with TTL-based caching and history tracking.
type TagHistoryEntry ¶
type TagHistoryEntry struct {
// Digest the tag previously pointed to
Digest string `json:"digest"`
// ChangedAt when this change occurred
ChangedAt time.Time `json:"changed_at"`
}
TagHistoryEntry represents a historical mapping of a tag to a digest.
type TagMapping ¶
type TagMapping struct {
// Reference is the tag reference (e.g., "myregistry.com/myimage:latest")
Reference string `json:"reference"`
// Digest is the content digest this tag currently points to
Digest string `json:"digest"`
// CreatedAt is when this mapping was first created
CreatedAt time.Time `json:"created_at"`
// UpdatedAt is when this mapping was last updated
UpdatedAt time.Time `json:"updated_at"`
// AccessCount tracks how many times this mapping has been accessed
AccessCount int64 `json:"access_count"`
// History contains previous digests this tag has pointed to
History []TagHistoryEntry `json:"history,omitempty"`
}
TagMapping represents a mapping from a tag reference to a digest. This enables efficient tag resolution with history tracking and TTL management.
type TagResolver ¶
type TagResolver struct {
// contains filtered or unexported fields
}
TagResolver provides efficient tag resolution with multi-tier validation. It implements caching, HEAD request optimization, and tag movement detection.
func NewTagResolver ¶
func NewTagResolver( tagCache TagCache, manifestCache ManifestCache, config TagResolverConfig, ) *TagResolver
NewTagResolver creates a new tag resolver with the given caches and configuration.
func (*TagResolver) BatchResolveTags ¶
func (tr *TagResolver) BatchResolveTags( ctx context.Context, references []string, ) (map[string]string, error)
BatchResolveTags resolves multiple tag references efficiently. Uses concurrent requests and caches results.
func (*TagResolver) DetectTagMovement ¶
func (tr *TagResolver) DetectTagMovement( ctx context.Context, reference, expectedDigest string, ) (bool, error)
DetectTagMovement checks if a tag has moved to a different digest. This is useful for cache invalidation when external changes are detected.
func (*TagResolver) GetTagHistory ¶
func (tr *TagResolver) GetTagHistory( ctx context.Context, reference string, ) ([]TagHistoryEntry, error)
GetTagHistory retrieves the history of digests for a tag reference.
func (*TagResolver) HealthCheck ¶
func (tr *TagResolver) HealthCheck(ctx context.Context) error
HealthCheck performs a basic health check of the resolver. Returns an error if the resolver is not functioning properly.
func (*TagResolver) InvalidateTag ¶
func (tr *TagResolver) InvalidateTag(ctx context.Context, reference string) error
InvalidateTag removes a tag mapping from the cache. Useful when tag movement is detected externally.
func (*TagResolver) ResolveTag ¶
ResolveTag resolves a tag reference to its digest using multi-tier validation. Strategy: 1. Check local cache first 2. If cache miss or expired, perform HEAD request to registry 3. If HEAD request succeeds, update cache and return digest 4. Handle tag movement detection and cache invalidation
func (*TagResolver) ValidateTag ¶
ValidateTag performs comprehensive validation of a tag reference. Returns true if the tag exists and points to valid content.
type TagResolverConfig ¶
type TagResolverConfig struct {
// DefaultTTL is the default TTL for tag mappings
DefaultTTL time.Duration
// MaxHistorySize is the maximum number of historical entries to keep per tag
MaxHistorySize int
// EnableHistory enables tracking of tag history (defaults to true)
EnableHistory bool
}
TagResolverConfig contains configuration for tag resolution behavior.
func (*TagResolverConfig) SetDefaults ¶
func (c *TagResolverConfig) SetDefaults()
SetDefaults applies default values to unset fields in the tag resolver configuration.
func (*TagResolverConfig) Validate ¶
func (c *TagResolverConfig) Validate() error
Validate checks that the tag resolver configuration is valid.