Documentation
¶
Overview ¶
Package content reconstructs full artifact content from Fossil's delta-chain storage.
Fossil stores blobs either as full content or as deltas against a source blob. Expand walks the delta chain from a given RID back to the root, decompresses each link, and applies deltas in sequence to produce the original content. Cycle detection prevents infinite loops.
Verify expands a blob and compares its hash (SHA1 or SHA3-256) against the stored UUID. IsPhantom checks whether a blob is a placeholder awaiting delivery during sync.
Index ¶
- Constants
- func AvailableByUUID(q db.Querier, uuid string) (libfossil.FslID, bool)
- func DeltaSource(q db.Querier, rid libfossil.FslID) (libfossil.FslID, error)
- func Deltify(tx *db.Tx, rid, srcRid libfossil.FslID) (saved int, err error)
- func Expand(q db.Querier, rid libfossil.FslID) (result []byte, err error)
- func GenerateClusters(q db.Querier) (int, error)
- func IsAvailable(q db.Querier, rid libfossil.FslID) bool
- func IsPhantom(q db.Querier, rid libfossil.FslID) (bool, error)
- func IsPrivate(q db.Querier, rid int64) bool
- func MakePrivate(q db.Querier, rid int64) error
- func MakePublic(q db.Querier, rid int64) error
- func Undelta(tx *db.Tx, rid libfossil.FslID) error
- func Verify(q db.Querier, rid libfossil.FslID) error
- type AvailabilityCache
- type Cache
- type CacheStats
Constants ¶
const ( // ClusterThreshold is the minimum number of unclustered, non-phantom blobs // before cluster generation triggers. ClusterThreshold = 100 // ClusterMaxSize is the maximum number of M-cards per cluster artifact. ClusterMaxSize = 800 )
Variables ¶
This section is empty.
Functions ¶
func AvailableByUUID ¶
AvailableByUUID resolves uuid to its rid and reports whether that artifact's content is available, per IsAvailable.
Use this in place of blob.Exists wherever the resolved rid is about to be passed to Expand: blob.Exists answers true for a phantom, whose content cannot be read.
func DeltaSource ¶
DeltaSource returns the rid that rid is stored as a delta against, or 0 if rid holds full content. Exported so callers outside this package (the clone send path, notably) can tell whether stored content read via blob.Load is a delta payload or full text without duplicating this query.
func Deltify ¶
Deltify tries to rewrite the artifact rid, currently stored whole, as a delta against srcRid. It returns the number of stored bytes saved; 0 means the policy above declined and rid was left untouched, which is a normal outcome and not an error.
The rewrite changes only how rid is stored, never what it expands to, so callers need not tell anyone the representation changed.
Deltify takes a *db.Tx rather than a db.Querier because rewriting an artifact is two statements -- the blob content and the delta link -- that must land together. A blob holding delta bytes with no delta row expands as though it were full content, which is silent corruption rather than a detectable error. Canonical wraps the same pair in db_begin_transaction / db_end_transaction (content.c:938-941); requiring the transaction in the signature makes that non-negotiable at compile time.
func GenerateClusters ¶
GenerateClusters creates cluster artifacts for unclustered blobs that are not phantoms, shunned, or private. Returns the number of clusters created.
func IsAvailable ¶
IsAvailable reports whether the content of rid can actually be read.
This is the availability predicate, distinct from blob.Exists, which answers only whether a row is present. It returns false for a phantom (blob.size < 0), false for an unknown rid, and — transitively — false for a delta whose chain bottoms out in a phantom, even though every blob row in that chain exists. It returns true only when the full chain is grounded in readable full-text content.
Ported from Fossil's content_is_available (src/content.c:163). Callers must never walk the delta table themselves; that logic lives here alone.
func MakePrivate ¶
MakePrivate inserts the rid into the private table (no-op if already present).
func MakePublic ¶
MakePublic removes the rid from the private table (no-op if not present).
func Undelta ¶
Undelta rewrites rid as full content, dropping its delta link. Mirrors content_undelta (src/content.c:745-769), including its update of blob.size, which content_deltify deliberately does not touch.
func Verify ¶
Verify confirms that rid's stored content hashes to its declared UUID. Expand performs this same check internally on every expansion now, so Verify is a thin, explicitly-named wrapper for callers (repo rebuild, integration tests) that want to state their intent as verification rather than retrieval.
Types ¶
type AvailabilityCache ¶
type AvailabilityCache struct {
// contains filtered or unexported fields
}
AvailabilityCache memoizes IsAvailable across many calls that share delta chains, for a caller that knows blob content is immutable for the cache's lifetime -- a single crosslink sweep. IsAvailable walks a blob's whole delta chain on every call, and crosslink calls it once per F-card of every check-in, so the same file blobs -- and the same chain suffixes -- are walked again and again; on the Fossil SCM corpus that walk was ~40% of the sweep.
A blob's availability is a property of its chain, and every node above a given node in the same chain shares that node's grounding or phantom, so one walk decides every node it passes through. The cache records that verdict for each rid, and a later walk that reaches an already-decided rid stops there. Total walk work over a sweep is then bounded by the number of distinct blobs, not by the number of F-card references.
ByUUID additionally memoizes the uuid->rid resolution the walk starts from. That lookup, not the walk, is what a sweep repeats: every F-card of every check-in names a file by hash, the same file hash appears in every revision that did not change it, and each name costs an index seek into a blob.uuid index far larger than SQLite's page cache. On the Fossil SCM repository that one statement was 27% of a whole clone's CPU -- more than delta expansion, hash verification and the receive path combined -- while the chain walk it guards was under 1%.
Scope ¶
Not safe for concurrent use. Both memos below are sound because of where this type is used, not because the facts they hold are permanent, so the scope is the invariant and it is worth stating exactly.
A cache belongs to ONE crosslink run: one whole-repository sweep (manifest.linkCandidatesInOrder) or one phantom-fill cascade (manifest.newCascadeLinker). Both build it as a local, neither hands it to a *repo.Repo or keeps it past the call, and nothing else constructs one. Over that window blob rows are only ever added -- crosslinking creates phantoms, fills them, and rewrites how content is stored, but removes nothing.
The one operation in the tree that removes a blob row is shun.Purge, which DELETEs from blob and delta. It has no production caller today; it runs as its own top-level operation, not from inside a sweep or a sync round. If that ever changes -- a sync learning to process shun cards mid-round is the plausible way -- a cache must not span it, because a memoized rid would then name a row that no longer exists. See TestAvailabilityCacheMustNotSpanAPurge, which pins that consequence so it is discovered by a failing test rather than by a wrong derivation.
Widening the lifetime past one crosslink run -- caching one per repository, or per process -- reopens exactly that question and is not a free change.
func NewAvailabilityCache ¶
func NewAvailabilityCache() *AvailabilityCache
NewAvailabilityCache returns an empty availability cache.
type Cache ¶
type Cache struct {
// contains filtered or unexported fields
}
Cache is a concurrency-safe LRU cache for expanded blob content. It reduces redundant delta-chain walks by caching the fully-expanded result of Expand, keyed by rid.
A nil *Cache is valid and acts as a passthrough to Expand.
func NewCache ¶
NewCache creates a cache bounded by maxBytes of expanded content and by maxCacheEntries metadata entries.
func (*Cache) Expand ¶
Expand returns the expanded content for rid, serving from cache when possible.
On a miss it walks rid's delta chain only as far back as the deepest ancestor the cache already holds, and caches every node it materializes on the way forward. That is what makes a sweep over a whole repository — every blob of a chain expanded once, in some order — cost one delta application per blob rather than one per (blob, chain-depth) pair.
A nil receiver falls through to Expand directly.
func (*Cache) Invalidate ¶
Invalidate removes a single rid from the cache.
func (*Cache) Remember ¶ added in v0.8.0
Remember retains a copy of already-expanded content. It is for receive paths that have just hash-verified full bytes and must examine those supplied bytes before permitting normal cache eviction; callers retain ownership of data.
func (*Cache) Stats ¶
func (c *Cache) Stats() CacheStats
Stats returns a snapshot of cache statistics.
Source Files