Documentation
¶
Overview ¶
Package cache is the host-wide, immutable, content-addressed object cache.
Layout (root defaults to <os.UserCacheDir()>/fxvcs, override with the FXVCS_CACHE_DIR environment variable or Open):
<root>/objects/<storageDomainID>/<hh>/<hh>/<64 hex> object bytes <root>/pins/<storageDomainID>/<64 hex> pending-publication marker <root>/published/<storageDomainID>/<64 hex>/<remote> publication marker per remote <root>/tmp/ in-flight writes
Objects are addressed by the sha256 of their bytes; a stored object is therefore immutable and can be shared by every clone and worktree on the host. Concurrency across processes relies only on atomic rename: a writer streams into tmp/, fsyncs, then renames into place. Two writers racing on the same digest both produce identical bytes, so whichever rename wins is correct and the loser discards its temp file. No lock files are needed.
Readers verify: Open returns a reader that recomputes the digest and fails at EOF on mismatch, so a corrupt cache entry can never be handed out as valid content.
Pins mark objects that were ingested locally (git add) but not yet published to every required remote. Prune never evicts a pinned object. Publication markers are the temporary stand-in for the publication ledger ($GIT_COMMON_DIR/fxvcs/publication.db); the pre-push hook consults them.
Index ¶
- Constants
- Variables
- func DefaultDir() (string, error)
- func Digest(sum []byte) string
- type Cache
- func (c *Cache) BytesWritten() int64
- func (c *Cache) Has(domain, digest string) bool
- func (c *Cache) HasSized(domain, digest string, size int64) bool
- func (c *Cache) IsPinned(domain, digest string) (bool, error)
- func (c *Cache) List() ([]Entry, error)
- func (c *Cache) Open(domain, digest string) (*Reader, error)
- func (c *Cache) Pin(domain, digest string) error
- func (c *Cache) PlanPrune(targetBytes int64) (plan []Entry, kept int, err error)
- func (c *Cache) Prune(targetBytes int64) (PruneResult, error)
- func (c *Cache) Put(domain string, r io.Reader) (PutResult, error)
- func (c *Cache) PutBytes(domain string, data []byte) (PutResult, error)
- func (c *Cache) PutBytesChecked(domain string, data []byte) (PutResult, error)
- func (c *Cache) PutExpecting(domain, expect string, size int64, r io.Reader) (PutResult, error)
- func (c *Cache) PutReplacing(domain, digest string, r io.Reader) (PutResult, error)
- func (c *Cache) PutVerified(domain, digest string, data []byte) error
- func (c *Cache) Remove(domain, digest string, force bool) error
- func (c *Cache) Root() string
- func (c *Cache) Stat(domain, digest string) (Info, error)
- func (c *Cache) TotalBytes() (int64, error)
- func (c *Cache) Unpin(domain, digest string) error
- func (c *Cache) Verify(domain, digest string) error
- type CorruptError
- type Entry
- type Info
- type PruneResult
- type PutResult
- type Reader
Constants ¶
const EnvDir = "FXVCS_CACHE_DIR"
EnvDir is the environment variable that overrides the cache location.
Variables ¶
var ( ErrNotFound = errors.New("cache: object not found") ErrCorrupt = errors.New("cache: object bytes do not match digest") ErrInvalidDigest = errors.New("cache: invalid digest") ErrInvalidDomain = errors.New("cache: invalid storage domain id") )
Errors.
Functions ¶
func DefaultDir ¶
DefaultDir returns FXVCS_CACHE_DIR if set, else <UserCacheDir>/fxvcs.
Types ¶
type Cache ¶
type Cache struct {
// contains filtered or unexported fields
}
Cache is a handle on one cache root. It holds no open files or locks; a zero-cost value that can be created per process.
func (*Cache) BytesWritten ¶
BytesWritten reports how many object bytes this handle has stored since it was opened. It is a lower bound on growth, not a measurement of the cache: other processes share it, so a caller must still measure eventually.
func (*Cache) HasSized ¶
HasSized reports that the object is cached and is exactly size bytes.
This is the cheap presence predicate for hot paths. It is not a substitute for Verify: it catches a truncated or replaced entry, not bytes that rotted in place. Callers rely on it only where the bytes are hashed again before anything observable happens to them — see the package doc on reader verification.
func (*Cache) List ¶
List enumerates every cached object with its pin state, sorted by domain then digest. A missing objects directory is an empty cache.
func (*Cache) Open ¶
Open returns a verifying reader for the object; ErrNotFound if absent. Opening touches the object's mtime (best effort) so Prune treats it as recently used.
func (*Cache) PlanPrune ¶
PlanPrune returns the LRU eviction plan that Prune(targetBytes) would execute, without removing anything, plus the number of objects kept (pinned or within budget). Pinned objects never appear in the plan.
func (*Cache) Prune ¶
func (c *Cache) Prune(targetBytes int64) (PruneResult, error)
Prune evicts least-recently-used unpinned objects until the total object bytes are at or below targetBytes. Pinned (pending publication) objects are never evicted, so the result may remain above target. Recency is the file mtime, which Open refreshes; atime is not portable enough to rely on.
func (*Cache) Put ¶
Put streams r into the cache under domain and returns its digest. Bytes are written to a temp file, fsynced, then atomically renamed to their final path; an interrupted Put leaves at most a temp file under tmp/ and never a visible object.
func (*Cache) PutBytes ¶
PutBytes stores content already held in memory.
It exists because Put cannot know an object's identity until it has read the whole stream, so it writes and fsyncs a temporary file before discovering that the object was already cached. That is the wrong shape for a caller that holds the bytes: re-ingesting an unchanged file then rewrote every chunk it already had. Hashing first costs nothing extra — Put hashes anyway — and turns the common "already present" case into a stat.
func (*Cache) PutBytesChecked ¶
PutBytesChecked is PutBytes for an object small enough that confirming the stored copy is cheaper than being wrong about it.
PutBytes trusts identity and size, which is right for bulk content that is hashed again on the way out. A manifest is different: it is the map to everything else, it is a few kilobytes, and a damaged one makes its whole asset unreadable rather than one chunk. So the existing entry is read, and replaced when it does not match.
func (*Cache) PutExpecting ¶
PutExpecting stores a stream whose digest the caller already knows, and skips the write when the cache already holds that object at that size.
The stream is still read and hashed in full: callers use this to prove that the bytes in hand are the bytes the cache claims to have — unload is about to destroy them — so skipping the read would defeat the purpose. What it skips is the temporary file, the fsync and the rename, which is the part that made re-deriving an unchanged file expensive.
A stream that does not hash to expect is ErrCorrupt and nothing is stored.
func (*Cache) PutReplacing ¶
PutReplacing stores a stream the caller expects to be digest, overwriting a damaged copy of that object.
It exists so that repairing an object never destroys the only copy first. Put treats an existing key as success and writes nothing, so a repair had to delete the damaged entry before fetching — and a fetch that then failed left nothing at all, along with a dropped pin. Writing with replacement keeps the damaged bytes until the good ones are in hand, and swaps them atomically.
The expected digest is required, and content that does not match it is discarded rather than stored: a caller repairing one object must not be able to write, or replace, a different one on a remote's say-so. A mismatch is ErrCorrupt and the cache is untouched.
func (*Cache) PutVerified ¶
PutVerified stores bytes whose identity the caller has already established, replacing whatever is at that key.
It exists for repair. Put treats an existing key as success without looking at it — objects are immutable, so identity implies content — but that reasoning only holds while the stored bytes are the bytes that were stored. A caller holding a proven copy of the content (unload, reading the working file it is about to destroy) needs to be able to put the cache right, not to be told the damaged entry is fine.
func (*Cache) Remove ¶
Remove deletes an object and its pin/publication markers. It refuses to remove a pinned object unless force is set.
func (*Cache) TotalBytes ¶
TotalBytes sums the sizes of all stored objects.
type CorruptError ¶
type CorruptError struct {
Digest string // the object that was expected
Got string // the digest its bytes actually hash to ("" if not computed)
Size int64 // bytes actually read
Want int64 // bytes expected
}
CorruptError names the object whose bytes did not match, so a caller that can repair the cache knows what to evict. Verification happens while bytes stream, and by then the digest under suspicion is several call frames away from whoever can act on it; carrying it in the error is what makes an automatic evict-and-refetch possible instead of failing the whole operation.
errors.Is(err, ErrCorrupt) matches it.
func (*CorruptError) Error ¶
func (e *CorruptError) Error() string
func (*CorruptError) Unwrap ¶
func (e *CorruptError) Unwrap() error
type PruneResult ¶
PruneResult reports what Prune did.
type PutResult ¶
type PutResult struct {
Digest string // sha256:<hex>
Size int64
// Existed is true when an identical object was already present.
Existed bool
}
PutResult describes a stored object.
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader streams an object and verifies its digest as bytes pass through. Read returns ErrCorrupt (instead of io.EOF) if the full content does not hash to the expected digest, and also if the file is shorter or longer than recorded. Callers must treat any error as "no valid content".
Source Files