Documentation
¶
Overview ¶
Package disk implements a disk-based cache for file data.
Index ¶
- Constants
- type DiskCache
- func (dc *DiskCache) Clear() error
- func (dc *DiskCache) Commit(path string, meta cache.EntryMetadata) error
- func (dc *DiskCache) Delete(path string) bool
- func (dc *DiskCache) DeletePartial(path string) bool
- func (dc *DiskCache) Pin(path string)
- func (dc *DiskCache) PinPartial(path string)
- func (dc *DiskCache) Read(path string, buff []byte, ofst int64) (n int, err error)
- func (dc *DiskCache) ReadComplete(path string, meta cache.EntryMetadata, buff []byte, ofst int64) (n int, err error)
- func (dc *DiskCache) ReadPartial(path string, buff []byte, ofst int64) (n int, err error)
- func (dc *DiskCache) SizeBytes() int64
- func (dc *DiskCache) StartMaintenance()
- func (dc *DiskCache) Stats() *cache.Stats
- func (dc *DiskCache) StopMaintenance()
- func (dc *DiskCache) Unpin(path string)
- func (dc *DiskCache) UnpinPartial(path string)
- func (dc *DiskCache) Write(path string, buff []byte, ofst int64) (n int, err error)
- func (dc *DiskCache) WritePartial(path string, buff []byte, ofst int64) (n int, err error)
- type Option
- func WithCapacityBytes(n int64) Option
- func WithDisabled(disabled bool) Option
- func WithLogger(logger log.Logger) Option
- func WithLruFlushInterval(d time.Duration) Option
- func WithMaintenanceInterval(d time.Duration) Option
- func WithMaxAge(d time.Duration) Option
- func WithMaxFileCount(n int64) Option
Constants ¶
const ( // DefaultCacheCapacity is the maximum size of the local cache in bytes if not provided // at cache initialization. 0 means unbounded. Equivalent to 2GiB. DefaultCapacity = 2 * (1 << 30) // (1 << 30) == 1GiB // DefaultMaxAge is the maximum age of cache files, if not provided // at cache initialization. 0 means no expiration. Equivalent to 7 days. DefaultMaxAge = 7 * 24 * time.Hour // DefaultMaxFileCount is the maximum number of files in the cache if not provided // at cache initialization. 0 means unbounded. DefaultMaxFileCount = 10_000 // DefaultMaintenanceInterval is the interval at which cache maintenance tasks are performed, // if not provided at cache initialization. DefaultMaintenanceInterval = 5 * time.Minute // DefaultLruFlushInterval is the interval at which the LRU state is persisted to disk, // if not provided at cache initialization. DefaultLruFlushInterval = 1 * time.Minute )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type DiskCache ¶
type DiskCache struct {
// CacheRoot is the root directory for the cache.
CacheRoot string
// Capacity is the maximum size of the cache in bytes. 0 means unbounded.
Capacity int64
// Disabled indicates whether the cache is disabled.
//
// A disabled cache:
// - performs no I/O and returns no data.
// - allows the caller to operate without caching, while not requiring conditional code around
// all cache operations.
Disabled bool
// MaxAge is the maximum age of cache files. 0 means no expiration.
MaxAge time.Duration
// MaxFileCount is the maximum number of files in the cache. 0 means unbounded.
MaxFileCount int64
// MaintenanceInterval is the interval at which cache maintenance tasks are performed.
//
// Maintenance tasks include:
// - Stats are reloaded from disk
// - Old files are deleted based on MaxAge
// - Unpinned data files without completed metadata are deleted
MaintenanceInterval time.Duration
// LruFlushInterval is the interval at which the LRU state is persisted to disk.
LruFlushInterval time.Duration
// contains filtered or unexported fields
}
DiskCache implements a simple disk-based cache for file data with an interface that roughly matches the FUSE Read/Write methods.
TODO:
- allow concurrent reads/writes to different files in the cache by moving lock coordination out of the caller and lock by path within the cache with a RWMutex map. This will require maintaining a reservation of bytes so that concurrent goroutines can't exhaust memory that a different goroutine evicted for its Write operation.
- consider limiting the max size of individual cache files so that a single large file cannot evict the entire cache
- allow callers to distiguish between "fatal" vs "non fatal" errors with custom error types e.g. if a read fails, the caller can just read from the source instead of treating it as a fatal error but a write failure because the cache is at capacity and could not make space is more serious
- move Stats#MarshalJSON to a separate file to and apply the filescomfs_debug tag
func NewDiskCache ¶
NewDiskCache creates a DiskCache rooted at path and applies any options. The provided path must be an absolute path to a directory that already exists and is writable by the current process.
If not disabled, it ensures the directory exists and initializes stats by scanning it.
The cache root itself belongs to the caller and is left as it is. Everything the cache stores below it (file data, partial data, metadata and LRU state) is readable only by the current user: the data, partial and state directories the cache owns are created private, and an existing cache is made private before any of its state is read, so a cache written by an earlier version that created world-readable entries is protected as well. See lib/privatefile for what private means on each platform. A root that another user could replace entries in is refused.
Defaults:
- Disabled: false
- Capacity: DefaultCapacity
- MaxAge: DefaultMaxAge
- MaxFileCount: DefaultMaxFileCount
func (*DiskCache) Clear ¶ added in v3.3.218
Clear removes all unpinned file data from the cache. Pinned entries are invalidated immediately and removed when their final open handle closes.
func (*DiskCache) Commit ¶ added in v3.3.109
func (dc *DiskCache) Commit(path string, meta cache.EntryMetadata) error
func (*DiskCache) Delete ¶
Delete removes the cached file from the cache. It returns true if the file was deleted.
func (*DiskCache) DeletePartial ¶ added in v3.3.109
func (*DiskCache) Pin ¶ added in v3.2.264
Pin increments the reference count for a file, preventing it from being evicted. This should be called when a file handle is opened.
func (*DiskCache) PinPartial ¶ added in v3.3.109
func (*DiskCache) Read ¶
Read reads data from the cached file at the given path into buff starting at the provided offset.
It returns the number of bytes read, or 0 if the file is not in the cache.
func (*DiskCache) ReadComplete ¶ added in v3.3.109
func (*DiskCache) ReadPartial ¶ added in v3.3.109
func (*DiskCache) StartMaintenance ¶
func (dc *DiskCache) StartMaintenance()
StartMaintenance starts the maintenance goroutine if it is not already running.
func (*DiskCache) StopMaintenance ¶
func (dc *DiskCache) StopMaintenance()
StopMaintenance stops the maintenance goroutine if it is running.
func (*DiskCache) Unpin ¶ added in v3.2.264
Unpin decrements the reference count for a file. When the count reaches zero, the file becomes eligible for eviction. This should be called when a file handle is closed.
func (*DiskCache) UnpinPartial ¶ added in v3.3.109
type Option ¶
type Option func(*DiskCache)
Option configures a DiskCache.
func WithCapacityBytes ¶
WithCapacityBytes sets the maximum total size (in bytes) for the cache.
func WithDisabled ¶
WithDisabled disables the cache (no I/O, no directory creation).
func WithLogger ¶
WithLogger sets the logger for the cache.
func WithLruFlushInterval ¶
WithLruFlushInterval sets the interval for cache maintenance operations.
func WithMaintenanceInterval ¶
WithMaintenanceInterval sets the interval for cache maintenance operations.
func WithMaxAge ¶
WithMaxAge sets the maximum age for cache files.
func WithMaxFileCount ¶
WithMaxFileCount sets the maximum number of files for the cache.
Source Files