Documentation
¶
Overview ¶
Package blob is a small content-addressed storage layer for flate's fetched artifacts. Blobs are indexed by sha256 digest; two artifacts with identical content share the same on-disk slot regardless of which CR resolved them. The store is the substrate the cache rework builds on — see pkg/source for ref-keyed slots that compose with this CAS via separate ref tables.
Layout under root:
<root>/blobs/sha256/<hex>/
Each blob is a directory so individual files (chart.tgz, README, etc.) can sit inside it without escaping the digest namespace.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func WithSweepLock ¶
WithSweepLock acquires the exclusive sweep lock, calls fn, then releases the lock. Held across mark + sweep so no Refs.Put can finalize within the window. The error returned by fn is propagated unchanged; the lock is always released.
Types ¶
type Refs ¶
type Refs struct {
// contains filtered or unexported fields
}
Refs is a tiny on-disk key→digest lookup table that sits beside the CAS blob store. It exists so callers that resolve artifacts by some mutable identity tuple (e.g. (repo, chart, version) for a helm tarball, or (URL, ref, authID) for a source CR) can persist the "this identity currently points at this content" mapping without stat-walking the blob store on every lookup.
Each entry is one tiny file at <dir>/<urlEscape(key)> containing the hex digest. The choice of one-file-per-key keeps writes atomic (os.Rename), avoids parsing a single index file under contention, and survives partial writes — a corrupted entry just looks like a cache miss.
An in-memory cache (sync.Map) sits in front of the disk reads so the hot path — a render with N HelmReleases against the same repo — doesn't pay a syscall per lookup. The cache is populated by every successful Get/Put and invalidated by Put.
func NewRefs ¶
NewRefs constructs a Refs table for one category under the supplied Layout. category names a stable subdirectory under <root>/refs/ (e.g. "chart-tarballs") that GC and introspection tooling share with the writer. The directory is created lazily on first Put.
func (*Refs) Get ¶
Get reads the digest stored under key, or returns ("", false) when the key is unknown. Treats partial or empty entries as misses so a torn write doesn't surface as a sentinel.
Consults the in-memory cache first; a hit avoids the disk read entirely. A miss falls through to ReadFile and populates the cache via LoadOrStore — if a concurrent Put landed a newer digest into mem between our ReadFile (which may have observed the old contents) and our cache-fill, the Put's value wins. Without LoadOrStore, this Get could overwrite the Put's NEW with the disk-read OLD and poison the cache for the rest of the run.
func (*Refs) Put ¶
Put records (key → digest) durably via atomic.WriteFile. Concurrent writers to the same key serialize on the Refs mutex; different keys proceed in parallel. Overwriting an existing key is supported (an upstream tag re-resolved to a new digest) — the rename atomically replaces the file.
Takes the package-level GC shared lock for the duration of the write so a concurrent gc.Sweep can't observe a not-yet-written ref during mark and then purge its blob during sweep (see gclock.go).
syncDir=false on the underlying atomic write: refs files are cheap to rebuild on the next reconcile, so the fsync barrier is not worth the per-render I/O cost.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store manages a content-addressed blob directory on disk. Safe for concurrent use; the keylock.KeyMap serializes Put for the same digest so two callers writing the same content don't race on rename finalize.
func NewStore ¶
NewStore constructs a Store backed by the supplied Layout. The blob subtree is created lazily on first write; Layout's blob path methods are the single source of truth for on-disk positioning.
func (*Store) Path ¶
Path returns the on-disk path for digest. Does not stat — callers use Exists to check populated-ness.
func (*Store) PutBytes ¶
func (s *Store) PutBytes(ctx context.Context, content []byte, filename string) (string, string, error)
PutBytes installs content as a single file named filename inside the blob directory keyed by content's sha256. The digest is recomputed from the bytes (never trusted from caller input). Concurrent callers targeting the same digest serialize on a per-digest lock; the first finalizes via atomic rename and the rest observe ErrExists internally and return without rewriting. ctx cancellation aborts the lock acquire (no write performed).
Takes the package-level GC shared lock so a concurrent gc.Sweep can't delete a blob between the Exists check and the caller's subsequent Refs.Put. The early-return path also refreshes the blob's mtime — without that bump, a reused-but-old blob whose "fresh" ref lands after Sweep's mark walk would be age-pruned even though a live caller just touched it.
Returns the populated blob directory path and the computed digest.
Source Files