Documentation
¶
Overview ¶
Package segcache is Ether's on-disk cache of decoded Usenet segments. It is a disk budget with segment-granular LRU eviction, pinning and a protection callback, not a library: the stream engine stores segments here so rewatches, seeks and the next episode are instant.
Concurrency model: a single mutex guards the in-memory index (O(1) map lookups plus an intrusive LRU list); segment file IO happens outside it. Per-key striped RW locks order the file-level operations on one key: Get reads under the read lock, while the rename of a Put and the unlink of an eviction/Remove run under the write lock and re-check the index, so an eviction never deletes a newer file put concurrently and a reader never observes a file being replaced underneath it.
A segment larger than the whole budget is rejected by Put with ErrTooLarge. If everything that could be evicted is pinned or protected the cache is allowed to exceed its budget (logged at most once a minute).
Index ¶
- Variables
- func MinFreeBytes(total int64) int64
- func ValidateSharedMount(dir string, db *sql.DB) error
- type Cache
- func (c *Cache) ClearUnpinned(id string) error
- func (c *Cache) Close() error
- func (c *Cache) Describe(id, title string, size int64)
- func (c *Cache) Directory() string
- func (c *Cache) FreeFloor() (floor, free int64)
- func (c *Cache) Get(releaseID string, segment int) ([]byte, bool)
- func (c *Cache) Has(releaseID string, segment int) bool
- func (c *Cache) Metrics() Metrics
- func (c *Cache) Pin(releaseID string, reason string) error
- func (c *Cache) Pins() map[string][]string
- func (c *Cache) Protect(fn func() map[string]bool)
- func (c *Cache) Put(releaseID string, segment int, data []byte) error
- func (c *Cache) RecordPlay(id, title string, size int64)
- func (c *Cache) ReleaseBytes(releaseID string) int64
- func (c *Cache) ReleaseSizes() map[string]int64
- func (c *Cache) Releases() []ReleaseUsage
- func (c *Cache) Remove(releaseID string) error
- func (c *Cache) Run(ctx context.Context)
- func (c *Cache) SetBlockCheck(check func(string) bool)
- func (c *Cache) SetBudget(budget int64)
- func (c *Cache) SetDiskUsage(fn DiskUsageFunc)
- func (c *Cache) SetPolicy(p Policy)
- func (c *Cache) Shared() bool
- func (c *Cache) SharedPinTx(ctx context.Context, tx *sql.Tx, id, reason string, add bool) error
- func (c *Cache) StorageError() string
- func (c *Cache) Sweep()
- func (c *Cache) Unpin(releaseID string, reason string) error
- func (c *Cache) Usage() (used, budget int64)
- func (c *Cache) UsageLimit() int64
- type DiskUsageFunc
- type Metrics
- type Policy
- type ReleaseUsage
Constants ¶
This section is empty.
Variables ¶
var ( // ErrTooLarge is returned by Put for a segment bigger than the whole budget. ErrTooLarge = errors.New("segcache: segment larger than cache budget") // ErrClosed is returned by mutating calls after Close. ErrClosed = errors.New("segcache: cache closed") )
Functions ¶
func MinFreeBytes ¶ added in v0.2.48
MinFreeBytes is the free space the cache leaves on a volume of total bytes.
Types ¶
type Cache ¶
type Cache struct {
// contains filtered or unexported fields
}
Cache implements core.SegmentCache.
func Open ¶
Open opens (creating) a cache rooted at dir with a byte budget (<=0 means unlimited), rebuilding the in-memory index from disk (and cleaning stale temp files from interrupted writes). logger may be nil (slog.Default()).
func OpenShared ¶ added in v0.2.4
func OpenShared(dir string, budget int64, db *sql.DB, node string, logger *slog.Logger) (*Cache, error)
OpenShared opens a NAS pool. The directory must already exist and be mounted. The server database owns the index, pins and accounting: local SQLite for a standalone server, shared PostgreSQL for a cluster. Media files have immutable, random names. No SQLite database is placed on the NAS. Each mount must expose the same marker created by the first pool member.
func (*Cache) ClearUnpinned ¶ added in v0.2.34
ClearUnpinned refuses protected data on both backends and waits for local writers before removing bytes. Shared pins are also checked transactionally.
func (*Cache) Close ¶
Close marks the cache closed and publishes the in-memory pin set. It is idempotent.
func (*Cache) Describe ¶ added in v0.2.4
Describe records a source's display metadata without counting a play.
func (*Cache) FreeFloor ¶ added in v0.2.48
FreeFloor reports the floor and the free space last measured (0, 0 while the floor is off or unmeasured).
func (*Cache) Get ¶
Get returns the cached payload (a fresh buffer owned by the caller) and marks it as recently used. A missing, vanished or corrupt file is a miss.
func (*Cache) Pin ¶
Pin protects every segment of a release from eviction while it has at least one pin reason. Pins are persisted to pins.json. If persisting fails the pin still applies in memory and the error is returned.
func (*Cache) Protect ¶
Protect registers a callback returning release IDs that must not be evicted right now. It is only called when an eviction is needed and its result is cached for a few seconds. A nil fn clears it. fn must not call Protect.
func (*Cache) Put ¶
Put stores (or replaces) a segment crash-safely (temp file + rename) and then evicts down to the budget. A payload larger than the whole budget is rejected with ErrTooLarge and nothing is stored. Note that a stored segment may be evicted again immediately if everything else is pinned/protected.
func (*Cache) RecordPlay ¶ added in v0.2.4
RecordPlay counts actual playback admissions, never speculative reads. The metadata is local, contains no viewer identity, and survives restarts.
func (*Cache) ReleaseBytes ¶
ReleaseBytes reports how many bytes of a release are currently cached.
func (*Cache) ReleaseSizes ¶ added in v0.2.4
ReleaseSizes is a single metadata query regardless of shelf or segment count.
func (*Cache) Releases ¶
func (c *Cache) Releases() []ReleaseUsage
Releases lists every release that has cached segments or pins, most recently used first.
func (*Cache) Remove ¶
Remove drops all segments of a release from the index and disk. Pins are kept.
func (*Cache) SetBlockCheck ¶ added in v0.2.34
SetBlockCheck prevents background writers from refilling owner-blocked releases.
func (*Cache) SetBudget ¶
SetBudget changes the budget (<=0 = unlimited) and evicts immediately if the cache is now over it.
func (*Cache) SetDiskUsage ¶ added in v0.2.48
func (c *Cache) SetDiskUsage(fn DiskUsageFunc)
SetDiskUsage enables the free-space floor; nil disables it. It applies immediately.
func (*Cache) Shared ¶ added in v0.2.4
Shared reports whether this cache uses the cluster's NAS pool.
func (*Cache) SharedPinTx ¶ added in v0.2.4
SharedPinTx joins a fenced preloader transaction. This package owns the pin SQL; the caller owns its lease check. Both locks live until the same commit.
func (*Cache) StorageError ¶ added in v0.2.4
StorageError is intentionally suitable for the owner UI and contains no credentials.
func (*Cache) Unpin ¶
Unpin removes one pin reason. When the last reason goes the release becomes evictable again and the cache is evicted down to budget immediately.
func (*Cache) UsageLimit ¶ added in v0.3.2
UsageLimit is the most the cache may hold right now: the budget, or the free-space floor's last measured cap when that is lower. -1 means unbounded; 0 means the policy retains no media bytes (mode off).
type DiskUsageFunc ¶ added in v0.2.48
DiskUsageFunc reports the size and the free bytes of the cache's volume.
type Metrics ¶ added in v0.2.4
type Metrics struct {
Hits int64 `json:"hits"`
Misses int64 `json:"misses"`
ServedBytes int64 `json:"servedBytes"`
EvictedBytes int64 `json:"evictedBytes"`
}
Metrics count disk-cache reads since this process started (not RAM hits).
Source Files