Help us shape the future of Go! Take the 2026 Go Developer Survey

content

package
v0.17.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 31, 2026 License: MIT Imports: 25 Imported by: 0

Documentation

Overview

Package content reconstructs full artifact content from Fossil's delta-chain storage.

Fossil stores blobs either as full content or as deltas against a source blob. Expand walks the delta chain from a given RID back to the root, decompresses each link, and applies deltas in sequence to produce the original content. Cycle detection prevents infinite loops.

Verify expands a blob and compares its hash (SHA1 or SHA3-256) against the stored UUID. IsPhantom checks whether a blob is a placeholder awaiting delivery during sync.

Index

Constants

View Source 
const (
	// ClusterThreshold is the minimum number of unclustered, non-phantom blobs
	// before cluster generation triggers.
	ClusterThreshold = 100

	// ClusterMaxSize is the maximum number of M-cards per cluster artifact.
	ClusterMaxSize = 800
)
View Source 
const DemandFetchLimit = 200

DemandFetchLimit bounds the artifacts one targeted demand fetch asks a peer for: one exchange's gimmes (sync.MaxGimmeBase), so the fetch is one round. A read needing more is the case a deepen is for.

View Source 
const MaxDeltaChainDepth = maxDeltaChainDepth

MaxDeltaChainDepth is maxDeltaChainDepth for callers that must bound their own walks the same way.

View Source 
const MergeAncestryLimit = 1 << 20

MergeAncestryLimit bounds the walk that decides whether a merge's ancestry crosses the horizon, as the origin bounds its merge-base walk.

Variables

View Source 
var ErrBelowHorizon = errors.New("history below the structural horizon")

ErrBelowHorizon is matched by the diagnostic of an operation that needs history below the store's structural horizon (draft-fossil-horizon-00 §5.5): the store does not hold it, and a deepen is what brings it.

View Source 
var ErrChainUnresolvable = errors.New("delta chain does not resolve")

ErrChainUnresolvable is a delta chain that loops back on itself or runs past maxDeltaChainDepth: no basis that arrives can make it readable.

View Source 
var ErrHashMismatch = errors.New("hash mismatch")

ErrHashMismatch marks an Expand that produced bytes not hashing to the name the blob is stored under. The content itself expanded; it is the binding to the name that failed, which a caller may need to report apart from content that cannot be read at all (draft-fossil-repo-state-00 Section 4.4).

Functions

func AncestryCrosses added in v0.15.0 

func AncestryCrosses(q db.Querier, starts []libfossil.FslID, limit int) (bool, error)

AncestryCrosses reports whether the ancestry of any of starts, walked back through every P card, reaches a parent the store does not hold as a check-in, when the store is at a structural horizon (§5.5): the merge base of such a check-in is then the origin's to name, even when a walk over what the store holds finds one. False for any other store. A walk past limit check-ins counts as crossing, since it cannot say otherwise.

func AncestryCrossing added in v0.15.0 

func AncestryCrossing(q db.Querier, side libfossil.FslID, starts []libfossil.FslID) error

AncestryCrossing decides whether the base the store's own walk found for a merge of side may stand (§5.5): not when the ancestry of any of starts, the merge's sides, crosses the structural horizon, even when the walk itself did not, since over part of the graph the common ancestor found need not be the nearest and only the origin can name the base. It returns a *MergeBaseError naming no base for such a merge, and nil otherwise.

func AvailableByUUID 

func AvailableByUUID(q db.Querier, uuid string) (libfossil.FslID, bool)

AvailableByUUID resolves uuid to its rid and reports whether that artifact's content is available, per IsAvailable. A database fault reads as unavailable; a caller that tolerates missing content on purpose must use CheckAvailableByUUID, so a fault is not mistaken for it.

Use this in place of blob.Exists wherever the resolved rid is about to be passed to Expand: blob.Exists answers true for a phantom, whose content cannot be read.

func BelowHorizon added in v0.15.0 

func BelowHorizon(ctx context.Context, q db.Querier, rid libfossil.FslID) (cone.Horizon, bool, error)

BelowHorizon reports rid as a check-in a structural store does not hold because it lies below the store's horizon (draft-fossil-horizon-00 §5.5), with that horizon. An operation that needs the check-in's tree names the horizon in its diagnostic rather than reporting a missing artifact.

func BelowLabels added in v0.15.0 

func BelowLabels(q db.Querier) (map[string]bool, error)

BelowLabels returns the boundary labels the store holds.

func CheckAvailable added in v0.15.0 

func CheckAvailable(q db.Querier, rid libfossil.FslID) (bool, error)

CheckAvailable is IsAvailable that reports a database fault as an error, distinct from content that is simply not there, which is (false, nil).

func CheckAvailableByUUID added in v0.10.0 

func CheckAvailableByUUID(q db.Querier, uuid string) (libfossil.FslID, bool, error)

CheckAvailableByUUID is AvailableByUUID that reports a database fault as an error, distinct from content that is simply not there (an unknown uuid, a phantom, or a delta on one), which is (0, false, nil).

func CompactStore added in v0.14.0 

func CompactStore(ctx context.Context, d *db.DB) (int, error)

CompactStore turns proved outside content into recoverable phantoms. Dense stores (including lost policy) are unchanged. Structural artifacts, unknown content, classification provenance and retained delta dependencies stay held. The existing write transaction excludes competing writers and rolls back every content change together on cancellation, corruption or SQL failure.

func DeltaSource 

func DeltaSource(q db.Querier, rid libfossil.FslID) (libfossil.FslID, error)

DeltaSource returns the rid that rid is stored as a delta against, or 0 if rid holds full content. Exported so callers outside this package (the clone send path, notably) can tell whether stored content read via blob.Load is a delta payload or full text without duplicating this query.

func Deltify 

func Deltify(tx *db.Tx, rid, srcRid libfossil.FslID) (saved int, err error)

Deltify tries to rewrite the artifact rid, currently stored whole, as a delta against srcRid. It returns the number of stored bytes saved; 0 means the policy above declined and rid was left untouched, which is a normal outcome and not an error.

The rewrite changes only how rid is stored, never what it expands to, so callers need not tell anyone the representation changed.

Deltify takes a *db.Tx rather than a db.Querier because rewriting an artifact is two statements -- the blob content and the delta link -- that must land together. A blob holding delta bytes with no delta row expands as though it were full content, which is silent corruption rather than a detectable error. Canonical wraps the same pair in db_begin_transaction / db_end_transaction (content.c:938-941); requiring the transaction in the signature makes that non-negotiable at compile time.

func Expand 

func Expand(q db.Querier, rid libfossil.FslID) (result []byte, err error)

func ExpandAll added in v0.14.0 

func ExpandAll(q db.Querier, visit func(rid libfossil.FslID, uuid string, data []byte, err error) error) error

ExpandAll expands every non-phantom blob exactly once and hands it to visit with its stored name, as fossil's rebuild_step does: it walks each delta tree down from its root, expanding a node from its parent's content and passing that content on to the node's own delta children. Every stored blob is read once and every delta applied once, however deep the chains run.

Expanding blob by blob instead -- even through a Cache -- re-walks a chain whenever its interiors have been evicted, which on a real repository (chains averaging hundreds of nodes) is O(depth^2) blob reads and hashes per chain.

visit receives what Cache.Expand would return for that blob alone: its content, bound to its name, or an error for the same failure (the wording can differ; a corrupt compressed delta reads "zlib reset", not "zlib decompress"). A node below a failed one gets the failure that stopped its chain. A blob no sound root reaches -- its chain cycles, ends in a phantom or a missing row, or exceeds maxDeltaChainDepth -- is expanded on its own after the walk, for that expansion's error. data is valid only during the call and must not be modified; it may be the source of later deltas.

A non-nil error from visit stops the sweep and is returned as is.

Memory held is the content of the nodes whose children are still pending: one buffer down a straight chain, more only where a tree branches.

func ExpandUnchecked added in v0.12.0 

func ExpandUnchecked(q db.Querier, rid libfossil.FslID) ([]byte, error)

ExpandUnchecked reconstructs rid's bytes as Expand does but does not check them against rid's name, as fossil's content_get reconstructs. It is for a caller that checks what it derives from the bytes: a received delta's basis, whose result is verified against the delta's own HASH (sync-protocol Section 7.7). It must not be used to serve content.

func GenerateClusters 

func GenerateClusters(q db.Querier) (int, error)

GenerateClusters creates cluster artifacts for unclustered blobs that are not phantoms, shunned, or private. Returns the number of clusters created.

func HorizonJoinedAfter added in v0.15.0 

func HorizonJoinedAfter(ctx context.Context, q db.Querier, h cone.Horizon, cursor int64) (bool, error)

HorizonJoinedAfter reports whether h's held set gained, at or below cursor, a check-in the store's held set through cursor lacked (horizon.Graph.JoinedAfter): a cursor from before such a reclassification is void (draft-fossil-horizon-00 §6.2.6). False when the store cannot place C.

func HorizonReach added in v0.15.0 

func HorizonReach(
	ctx context.Context, q db.Querier, h cone.Horizon,
) (k []libfossil.FslID, reach map[string]bool, placed bool, err error)

HorizonReach returns the held set of h and the manifests, baselines and nodes its trees reach, by name: what a structural horizon keeps of the store's tree structure (draft-fossil-horizon-00 §5.1, §6.2). placed is false when the store cannot place C.

func IsAvailable 

func IsAvailable(q db.Querier, rid libfossil.FslID) bool

IsAvailable reports whether the content of rid can actually be read.

This is the availability predicate, distinct from blob.Exists, which answers only whether a row is present. It returns false for a phantom (blob.size < 0), false for an unknown rid, and — transitively — false for a delta whose chain bottoms out in a phantom, even though every blob row in that chain exists. It returns true only when the full chain is grounded in readable full-text content.

Ported from Fossil's content_is_available (src/content.c:163). Callers must never walk the delta table themselves; that logic lives here alone.

func IsPhantom 

func IsPhantom(q db.Querier, rid libfossil.FslID) (bool, error)

func IsPrivate 

func IsPrivate(q db.Querier, rid int64) bool

IsPrivate returns true if the blob with the given rid is in the private table.

func LabelBelow added in v0.15.0 

func LabelBelow(q db.Querier, names []string) error

LabelBelow records names an origin labelled below the store's structural horizon. A name that is no artifact hash is ignored, as a malformed pragma is; labelling is idempotent.

func MakePrivate 

func MakePrivate(q db.Querier, rid int64) error

MakePrivate inserts the rid into the private table (no-op if already present).

func MakePublic 

func MakePublic(q db.Querier, rid int64) error

MakePublic removes the rid from the private table (no-op if not present).

func MergeBaseCrossing added in v0.15.0 

func MergeBaseCrossing(q db.Querier, side, base libfossil.FslID, unheld bool) error

MergeBaseCrossing decides whether the merge of side crosses the store's structural horizon (§5.5), given the base the store's own walk found, 0 for none, and whether that walk reached a check-in the store does not hold (merge.Pivot.Unheld). A base the store does not hold is a crossing. So is any walk that reached the horizon: the store holds only part of the graph, and a common ancestor found in part of it need not be the one the whole graph gives, so the origin must be asked. It returns a *MergeBaseError for a crossing and nil otherwise, so a merge never runs against a base it does not hold or one the origin would not choose.

func RequireAvailableByUUID added in v0.14.0 

func RequireAvailableByUUID(q db.Querier, uuid string) (libfossil.FslID, error)

RequireAvailableByUUID is a local-only read gate. Missing content is not a missing path or corruption: sparse stores deliberately retain some absences. Do not claim that no peer exists; a read has no authority to use a saved peer.

func SetStoreCone added in v0.14.0 

func SetStoreCone(q db.Querier, c *cone.Cone) error

SetStoreCone replaces the store cone; nil makes the store dense. Neither direction moves content: narrowing deletes nothing (§5.4), and widening only makes the newly covered elided phantoms wanted again, for the next synchronization to fetch. The boundary labels of a structural horizon (draft-fossil-horizon-00 §5.3) name what lies below one C, so a cone whose horizon differs drops them.

func StoreCone added in v0.14.0 

func StoreCone(q db.Querier) (*cone.Cone, error)

StoreCone returns the store cone, or nil when the store is dense. A cone this implementation cannot read -- a later serialization version, or bytes that do not parse -- is policy loss (§3.1, §5.2) and reads as dense, never as an error: the cost is bandwidth, not data.

func StructuralHorizon added in v0.15.0 

func StructuralHorizon(q db.Querier) (cone.Horizon, bool, error)

StructuralHorizon is the store cone's horizon when it is structural.

func Undelta 

func Undelta(tx db.Querier, rid libfossil.FslID) error

Undelta rewrites rid as full content, dropping its delta link. Mirrors content_undelta (src/content.c:745-769), including its update of blob.size, which content_deltify deliberately does not touch. Its two writes belong in the caller's transaction.

func Verify 

func Verify(q db.Querier, rid libfossil.FslID) error

Verify confirms that rid's stored content hashes to its declared UUID. Expand performs this same check internally on every expansion now, so Verify is a thin, explicitly-named wrapper for callers (repo rebuild, integration tests) that want to state their intent as verification rather than retrieval.

Types

type Absence added in v0.14.0 

type Absence int

Absence classifies an artifact the store names: held, or absent for one of the three reasons of §5.2. 

const (
	// Held is not absent: the store holds the content.
	Held Absence = iota
	// Transit is wanted and not yet arrived; synchronization requests it.
	Transit
	// Shunned is forbidden and never requested.
	Shunned
	// Elided is deliberately absent under the store cone. Synchronization
	// does not request it, and its absence is healthy.
	Elided
)

func (Absence) String added in v0.14.0 

func (a Absence) String() string

type Absences added in v0.14.0 

type Absences struct {
	// contains filtered or unexported fields
}

Absences is the classification of every phantom in a store, computed from the store cone at load time. Elision is never stored on a phantom (§5.2): a cone change is a reload, not a migration.

func LoadAbsences added in v0.14.0 

func LoadAbsences(q db.Querier) (*Absences, error)

LoadAbsences classifies the store's phantoms. Shunned names win; raw F/S provenance proves elision only outside every covered alias and structural reference. Retained delta ancestors remain transit. Crosslink references supplement raw provenance for historical names, including prior rename paths. A partial tree need not be crosslinked to classify its outside S children. A content horizon in the cone also elides the blobs below it and the clusters the store was sent.

func LoadAbsencesContext added in v0.14.0 

func LoadAbsencesContext(ctx context.Context, q db.Querier) (*Absences, error)

LoadAbsencesContext carries cancellation through raw provenance discovery.

func (*Absences) Below added in v0.15.0 

func (a *Absences) Below(rid libfossil.FslID) bool

Below reports rid as a check-in below a structural horizon.

func (*Absences) BelowHorizon added in v0.15.0 

func (a *Absences) BelowHorizon() int

BelowHorizon counts the elided phantoms that are check-ins below a structural horizon (draft-fossil-horizon-00 §5.4): its boundary hashes and the manifests only a tag or cherry-pick names.

func (*Absences) Class added in v0.14.0 

func (a *Absences) Class(rid libfossil.FslID) Absence

Class reports why rid is absent, or Held when it is not a phantom.

func (*Absences) Cone added in v0.14.0 

func (a *Absences) Cone() *cone.Cone

Cone is the store cone the classification used, nil for a dense store.

func (*Absences) Count added in v0.14.0 

func (a *Absences) Count(k Absence) int

Count is the number of phantoms in class k. Held is not counted.

func (*Absences) Unnamed added in v0.15.0 

func (a *Absences) Unnamed(rid libfossil.FslID) bool

Unnamed reports a phantom a structural horizon elides only because nothing the store holds names it, such as one a peer announced. A synchronization that heard the name announced asks for it: an origin announces only what it keeps, and a pusher what it is sending.

type AvailabilityCache 

type AvailabilityCache struct {
	// contains filtered or unexported fields
}

AvailabilityCache memoizes IsAvailable across many calls that share delta chains, for a caller that knows blob content is immutable for the cache's lifetime -- a single crosslink sweep. IsAvailable walks a blob's whole delta chain on every call, and crosslink calls it once per F-card of every check-in, so the same file blobs -- and the same chain suffixes -- are walked again and again; on the Fossil SCM corpus that walk was ~40% of the sweep.

A blob's availability is a property of its chain, and every node above a given node in the same chain shares that node's grounding or phantom, so one walk decides every node it passes through. The cache records that verdict for each rid, and a later walk that reaches an already-decided rid stops there. Total walk work over a sweep is then bounded by the number of distinct blobs, not by the number of F-card references.

ByUUID additionally memoizes the uuid->rid resolution the walk starts from. That lookup, not the walk, is what a sweep repeats: every F-card of every check-in names a file by hash, the same file hash appears in every revision that did not change it, and each name costs an index seek into a blob.uuid index far larger than SQLite's page cache. On the Fossil SCM repository that one statement was 27% of a whole clone's CPU -- more than delta expansion, hash verification and the receive path combined -- while the chain walk it guards was under 1%.

Scope

Not safe for concurrent use. Both memos below are sound because of where this type is used, not because the facts they hold are permanent, so the scope is the invariant and it is worth stating exactly.

A cache belongs to ONE crosslink run: one whole-repository sweep (manifest.linkCandidatesInOrder) or one phantom-fill cascade (manifest.newCascadeLinker). Both build it as a local, neither hands it to a *repo.Repo or keeps it past the call, and nothing else constructs one. Over that window blob rows are only ever added -- crosslinking creates phantoms, fills them, and rewrites how content is stored, but removes nothing.

The one operation in the tree that removes a blob row is shun.Purge, which DELETEs from blob and delta. It has no production caller today; it runs as its own top-level operation, not from inside a sweep or a sync round. If that ever changes -- a sync learning to process shun cards mid-round is the plausible way -- a cache must not span it, because a memoized rid would then name a row that no longer exists. See TestAvailabilityCacheMustNotSpanAPurge, which pins that consequence so it is discovered by a failing test rather than by a wrong derivation.

Widening the lifetime past one crosslink run -- caching one per repository, or per process -- reopens exactly that question and is not a free change.

func NewAvailabilityCache 

func NewAvailabilityCache() *AvailabilityCache

NewAvailabilityCache returns an empty availability cache.

func (*AvailabilityCache) Available added in v0.16.1 

func (a *AvailabilityCache) Available(q db.Querier, rid libfossil.FslID) bool

Available is IsAvailable through the cache, for a caller that asks of many rids sharing chains.

func (*AvailabilityCache) ByUUID 

func (a *AvailabilityCache) ByUUID(q db.Querier, uuid string) (libfossil.FslID, bool)

ByUUID resolves uuid to its rid and reports whether that blob's content is available, memoizing the chain walk. Semantics match AvailableByUUID.

func (*AvailabilityCache) ForgetUnavailable added in v0.15.0 

func (a *AvailabilityCache) ForgetUnavailable()

ForgetUnavailable drops every unavailable verdict and keeps the available ones. It is for a caller whose store only gains content between questions, a receive: an artifact that was available stays so, while one that was not may have had its basis filled since.

type Cache 

type Cache struct {
	// contains filtered or unexported fields
}

Cache is a concurrency-safe LRU cache for expanded blob content. It reduces redundant delta-chain walks by caching the fully-expanded result of Expand, keyed by rid.

A nil *Cache is valid and acts as a passthrough to Expand.

func NewCache 

func NewCache(maxBytes int64) *Cache

NewCache creates a cache bounded by maxBytes of expanded content and by maxCacheEntries metadata entries.

func NewUncheckedCache added in v0.15.0 

func NewUncheckedCache(maxBytes int64) *Cache

NewUncheckedCache creates a cache for Cache.ExpandUnchecked, bounded as NewCache bounds one. What it holds is what each row rebuilds to, unchecked against the row's name, so it serves no Cache.Expand.

func (*Cache) Clear 

func (c *Cache) Clear()

Clear removes all entries from the cache.

func (*Cache) Derived added in v0.15.0 

func (c *Cache) Derived(key any) (any, bool)

Derived returns the value SetDerived last stored under key, if it is still held. It lets a caller that parses or indexes cached content keep the result next to the bytes rather than redo the work for each neighbour that reads the same artifact. The value is shared: callers must treat it as read-only. A nil Cache holds nothing.

func (*Cache) DerivedSized added in v0.16.2 

func (c *Cache) DerivedSized(key any) (any, bool)

DerivedSized returns the value SetDerivedSized last stored under key, if it is still held, and marks it most recently used. The value is shared: callers must treat it as read-only. A nil Cache holds nothing.

func (*Cache) Expand 

func (c *Cache) Expand(q db.Querier, rid libfossil.FslID) ([]byte, error)

Expand returns the expanded content for rid, serving from cache when possible.

On a miss it walks rid's delta chain only as far back as the deepest ancestor the cache already holds, and caches every node it materializes on the way forward. That is what makes a sweep over a whole repository — every blob of a chain expanded once, in some order — cost one delta application per blob rather than one per (blob, chain-depth) pair.

A nil receiver falls through to Expand directly.

func (*Cache) ExpandUnchecked added in v0.15.0 

func (c *Cache) ExpandUnchecked(q db.Querier, rid libfossil.FslID) ([]byte, error)

ExpandUnchecked is ExpandUnchecked through an unchecked cache: rid's bytes, rebuilt from the deepest node of its chain the cache holds, with no node checked against its name, and every node it rebuilds kept. Like ExpandUnchecked it is for a caller that checks what it derives from the bytes, and must not be used to serve content.

func (*Cache) Holds added in v0.15.0 

func (c *Cache) Holds(rid libfossil.FslID) bool

Holds reports whether rid's content is cached, without counting a hit or a miss or moving it in the eviction order. A nil Cache holds nothing.

func (*Cache) Invalidate 

func (c *Cache) Invalidate(rid libfossil.FslID)

Invalidate removes a single rid from the cache.

func (*Cache) Remember added in v0.8.0 

func (c *Cache) Remember(rid libfossil.FslID, data []byte)

Remember retains a copy of already-expanded content. It is for receive paths that have just hash-verified full bytes and must examine those supplied bytes before permitting normal cache eviction; callers retain ownership of data.

func (*Cache) SetDerived added in v0.15.0 

func (c *Cache) SetDerived(key, v any)

SetDerived stores v under key, evicting the oldest value past maxDerived. key must be comparable. Because Invalidate and Clear drop every derived value, v may be computed from content that is immutable only while cached.

func (*Cache) SetDerivedSized added in v0.16.2 

func (c *Cache) SetDerivedSized(key, v any, size int64)

SetDerivedSized stores v under key, charged size bytes, evicting the least recently used values past the budget or past maxCacheEntries. A value over the whole budget is not kept. key must be comparable. Invalidate and Clear drop every value, as they drop Derived's.

func (*Cache) Stats 

func (c *Cache) Stats() CacheStats

Stats returns a snapshot of cache statistics.

type CacheStats 

type CacheStats struct {
	Hits   int64
	Misses int64
	// DerivedHits counts Derived lookups that found a value: reads that
	// reused work done on cached content rather than expanding it again.
	DerivedHits int64
	// SizedHits counts DerivedSized lookups that found a value, and
	// SizedStores the values SetDerivedSized added under a key not held:
	// for the sub-manifest walk, the nodes it reused and the nodes it parsed.
	SizedHits   int64
	SizedStores int64
	Size        int64
	MaxSize     int64
	Entries     int
}

CacheStats reports cache hit/miss statistics and current memory usage.

type ConeReferenceFacts added in v0.14.0 

type ConeReferenceFacts struct {
	// contains filtered or unexported fields
}

ConeReferenceFacts holds immutable reference facts for one database snapshot, without a cone key. The owner must validate that snapshot before using these facts against a database that may have changed.

Loading names every artifact in the store, but reads raw bytes only where the crosslink tables cannot answer: never a file version the crosslinker found to be no artifact, and a check-in only when its link is incomplete or its tree is not reproduced by mlink (see answerable). It serves whole-store work (a pull's inventory, compaction, verification, crosslink), never a clone round. Retained decks are classification projections, not complete artifacts suitable for marshaling.

func LoadConeReferenceFacts added in v0.14.0 

func LoadConeReferenceFacts(ctx context.Context, q db.Querier) (*ConeReferenceFacts, error)

func (*ConeReferenceFacts) Classify added in v0.14.0 

func (facts *ConeReferenceFacts) Classify(
	ctx context.Context, c *cone.Cone,
) (*ConeReferences, error)

Classify creates request-local membership; neither the facts nor previous classifications are changed. Maps of raw identities and decks are read-only.

type ConeReferences added in v0.14.0 

type ConeReferences struct {
	// contains filtered or unexported fields
}

ConeReferences is inferred from raw artifacts wherever crosslink tables cannot stand in for them: a sparse tree can be healthy before its elided children permit whole-tree crosslinking, so trees are always read raw, while a check-in linked completely is answered by its mlink, plink, cherrypick and tagxref rows (ConeReferenceFacts). Unknown artifacts stay wanted. A covered alias or structural reference wins over every outside reference.

func LoadConeReferences added in v0.14.0 

func LoadConeReferences(ctx context.Context, q db.Querier, c *cone.Cone) (*ConeReferences, error)

LoadConeReferences classifies both held and phantom artifact names. Traversal shares the single-name lookup depth and node limits across all roots.

func (*ConeReferences) Artifacts added in v0.14.0 

func (p *ConeReferences) Artifacts() int

Artifacts counts the artifact names the classification read.

func (*ConeReferences) HorizonBelow added in v0.15.0 

func (p *ConeReferences) HorizonBelow(
	ctx context.Context, q db.Querier, c *cone.Cone,
) (func(libfossil.FslID) bool, error)

HorizonBelow returns the elision predicate of c's content horizon (draft-fossil-horizon-00 §4.2) over the artifacts p names, held or not: true for a blob a held F card names that no check-in in the held set names and that is no structural reference. A synchronization applies it to what it received, as LoadAbsences applies it to phantoms. It is nil when c has no horizon or the store cannot place C: such a horizon proves no blob below it.

func (*ConeReferences) Nodes added in v0.15.0 

func (p *ConeReferences) Nodes() map[libfossil.FslID]string

Nodes returns the held sub-manifest nodes by rid, with their names.

func (*ConeReferences) Outside added in v0.14.0 

func (p *ConeReferences) Outside(rid libfossil.FslID) bool

Outside reports only proven elision candidates. Delta dependency closure is a separate operation: a sender can instead expand a retained delta in full.

func (*ConeReferences) Required added in v0.14.0 

func (p *ConeReferences) Required(rid libfossil.FslID) bool

Required reports a covered alias or a structural reference, including ones which have not been crosslinked yet.

func (*ConeReferences) RetainProvenance added in v0.14.0 

func (p *ConeReferences) RetainProvenance(ctx context.Context, drop map[libfossil.FslID]bool) error

RetainProvenance removes every held sub-manifest node from a proposed drop set. Sparse-cone Sections 5.4 and 6.3 keep manifests and sub-manifest nodes, in cone or not: a leaf's F cards are what classify the blobs it names as elided rather than transit phantoms, and an interior is the range proof that reaches the leaf. Only blobs are ever dropped. Check-ins never enter a drop set, since no S card names them.

func (*ConeReferences) SlotKeep added in v0.15.0 

func (p *ConeReferences) SlotKeep(ctx context.Context, c *cone.Cone, reach map[string]bool) (*SlotKeep, error)

SlotKeep walks the trees of every held check-in manifest, or with reach only of those reach names (a structural horizon's, §5.1), for c's path members.

type HashMismatchError added in v0.12.0 

type HashMismatchError struct {
	RID            libfossil.FslID
	Stored, Actual string
}

HashMismatchError is the error behind ErrHashMismatch. RID names the blob whose name the bytes failed, which is not the blob asked for when an interior delta base is the misnamed one.

func (*HashMismatchError) Error added in v0.12.0 

func (e *HashMismatchError) Error() string

func (*HashMismatchError) Is added in v0.12.0 

func (e *HashMismatchError) Is(target error) bool

Is makes errors.Is(err, ErrHashMismatch) hold.

type Memo added in v0.15.0 

type Memo struct {
	// contains filtered or unexported fields
}

Memo is an origin's classification memo for one open store (draft-fossil-horizon-00 §6.2). It keeps the store's cone-independent references, read once and extended by the artifacts appended since; and, keyed by cone digest and by horizon, the classifications computed over them, each keyed by the store prefix it covers and extended the same way. It also keeps, compactly and bounded, the trees its horizons' walks read, so a horizon under another C or rule reads only the trees no earlier walk did. It recomputes in full only when an appended artifact carries a name a held P, B or S card cited while it was absent, or the store changed other than by appending and filling phantoms.

A classification is read from a View, which brings the memo up to the store as one querier sees it, and the View is published back once what it read is known to be durable. A View that is never published changes nothing, so an origin classifying inside a request that rolls back leaves no trace. Every value a Memo hands out is immutable; extending copies.

func NewMemo added in v0.15.0 

func NewMemo() *Memo

NewMemo returns an empty memo.

func (*Memo) View added in v0.15.0 

func (m *Memo) View(ctx context.Context, q db.Querier) (*MemoView, error)

View reads q's store prefix and returns the memo extended to it: by the artifacts appended since the memo's prefix and the phantoms filled since, or read in full when that is the only exact answer.

type MemoView added in v0.15.0 

type MemoView struct {
	// contains filtered or unexported fields
}

MemoView is a Memo brought up to one store as one querier sees it. It is used by one goroutine; Publish hands what it computed back to the Memo.

func (*MemoView) HeldSet added in v0.17.0 

func (v *MemoView) HeldSet(ctx context.Context, q db.Querier, h cone.Horizon) ([]libfossil.FslID, bool, error)

HeldSet is horizon.Place over the view's store graph: h's held set, and whether the store holds h's C as a check-in. It is kept by horizon until the store changes, so a store asked for the same held set on every request walks its graph once, within the graph's visited-node bound.

func (*MemoView) HorizonBelow added in v0.15.0 

func (v *MemoView) HorizonBelow(
	ctx context.Context, q db.Querier, p *ConeReferences, c *cone.Cone,
) (func(libfossil.FslID) bool, error)

HorizonBelow is p.HorizonBelow over the view's store, p a classification the view returned.

func (*MemoView) HorizonJoinedAfter added in v0.15.0 

func (v *MemoView) HorizonJoinedAfter(ctx context.Context, q db.Querier, h cone.Horizon, cursor int64) (bool, error)

HorizonJoinedAfter is HorizonJoinedAfter over the view's store graph.

func (*MemoView) HorizonReach added in v0.15.0 

func (v *MemoView) HorizonReach(
	ctx context.Context, q db.Querier, h cone.Horizon,
) (k []libfossil.FslID, reach map[string]bool, placed bool, err error)

HorizonReach is HorizonReach over the view's store.

func (*MemoView) Publish added in v0.15.0 

func (v *MemoView) Publish()

Publish makes what v computed the memo's, unless the memo has since been brought further: past v's store, or to a store whose full read was numbered after v's (View says why that order is safe). The trees v read are kept either way, as no store changes what a tree says. Call it only once the store v read is durable.

func (*MemoView) Read added in v0.15.0 

func (v *MemoView) Read() int

Read is how many artifacts the view classified to bring the memo up to its store: every one on a full read, those appended or filled on an extension, none when the memo already covered the store.

func (*MemoView) References added in v0.15.0 

func (v *MemoView) References(ctx context.Context, c *cone.Cone) (*ConeReferences, error)

References returns the path classification of c (see LoadConeReferences) over the view's store: the memo's classification of c extended by what the store gained since, or one computed over the store graph.

func (*MemoView) SlotKeep added in v0.15.0 

func (v *MemoView) SlotKeep(
	ctx context.Context, p *ConeReferences, c *cone.Cone, reach map[string]bool,
) (*SlotKeep, error)

SlotKeep is p.SlotKeep over the view's store, p a classification the view returned: the memo's walk of c extended by the trees the store gained since, which reads only theirs (§6.2), or a fresh walk when the gain is not an append of later roots.

type MergeBaseError added in v0.15.0 

type MergeBaseError struct {
	Horizon, Side, Base string
}

MergeBaseError is a merge whose base lies below the structural horizon at Horizon (§5.5), or may: Base is the base the store's walk found when the store does not hold it, "" when the walk crossed the horizon, so that only the origin can name the base. Side is the check-in being merged in.

func (*MergeBaseError) Error added in v0.15.0 

func (e *MergeBaseError) Error() string

func (*MergeBaseError) Is added in v0.15.0 

func (e *MergeBaseError) Is(target error) bool

type MissingBelowHorizon added in v0.17.0 

type MissingBelowHorizon struct {
	Horizon  string
	Checkins []string
	Names    []string
}

MissingBelowHorizon is a read's refusal for artifacts the store does not hold under a store cone with a horizon at Horizon (draft-fossil-horizon-00 §4.4). Checkins are check-ins whose manifests, baselines and nodes the read needs; Names are the file payloads. They are exactly what a targeted demand fetch asks a peer for, which leaves the horizon where it is; the other remedy is a deepen, which moves it.

func MissingAtHorizon added in v0.17.0 

func MissingAtHorizon(q db.Querier, checkins, names []string) (*MissingBelowHorizon, error)

MissingAtHorizon is the refusal naming checkins and names, none of which the store holds, when its cone has a horizon. It is nil when the store cone has none, or when one of them is shunned: no peer may supply that, so the caller keeps its own diagnostic.

func (*MissingBelowHorizon) Error added in v0.17.0 

func (e *MissingBelowHorizon) Error() string

func (*MissingBelowHorizon) Is added in v0.17.0 

func (e *MissingBelowHorizon) Is(target error) bool

func (*MissingBelowHorizon) Join added in v0.17.0 

func (e *MissingBelowHorizon) Join(o *MissingBelowHorizon) *MissingBelowHorizon

Join is the refusal naming what e and o both name, either of which may be nil. A read that needs several artifacts joins its refusals so that one demand fetch asks for all of them.

type ReferenceBudget added in v0.14.0 

type ReferenceBudget struct {
	Artifacts     int
	StoredBytes   int64
	InflatedBytes int64
	ExpandedBytes int64
	Cards         int
}

ReferenceBudget bounds a whole-store reference load. Byte budgets count payload buffers, not Go allocator overhead or parser objects. The artifact/card caps separately bound that overhead. The caller must hold a stable database snapshot through preflight and materialization.

type ReferenceWork added in v0.14.0 

type ReferenceWork struct {
	PrefixReads  int
	PrefixBytes  int64
	PayloadReads int
	InflateBytes int64
	ExpandBytes  int64
	Visits       int
	// TreeReads and TreeBytes count a content horizon's manifest and
	// sub-manifest reads, charged as each one is expanded.
	TreeReads int
	TreeBytes int64
}

type SlotKeep added in v0.15.0 

type SlotKeep struct {
	// contains filtered or unexported fields
}

SlotKeep is what an origin keeps of the sub-manifest nodes of the trees a horizon keeps, for a request that also carries path members (draft-fossil-horizon-00 §6.2): the nodes reached from a kept tree's manifest through S-card slots that may hold an in-cone path, as the sparse-cone draft's §6.2 server descends them. It withholds every other held node, of every age. Each kept node records the lowest root whose walk keeps it, which is what §6.2.6's cursor voiding reads.

The walk is zit's walkSlots (src/sparse.zig at 3a2df62515): a node's last slot takes its bound from its parent, a node is walked again only under a bound that reaches further than every bound it was walked under, and a slot that may not reach the cone is never descended, since a child's slot lies inside its parent's. It reads the classification's facts, never the store, and walks roots in ascending rid order, so a later root that only widens a node's bound leaves the earlier root recorded.

func (*SlotKeep) Equal added in v0.15.0 

func (k *SlotKeep) Equal(o *SlotKeep) bool

Equal reports whether k and o keep the same nodes, each first kept by the same root.

func (*SlotKeep) Visits added in v0.15.0 

func (k *SlotKeep) Visits() int

Visits is how many node visits the walk that built k made beyond the walk it extended: every one for a fresh walk, none for an unchanged store.

func (*SlotKeep) Voids added in v0.15.0 

func (k *SlotKeep) Voids(cursor int64) bool

Voids reports whether a cursor names a position from before some kept node was kept (§6.2.6): the node lies at or below it and every root whose walk keeps the node lies above it, so a classification at the cursor withheld a node this one keeps and the cursor's holder was never told.

func (*SlotKeep) Withholds added in v0.15.0 

func (k *SlotKeep) Withholds(rid libfossil.FslID) bool

Withholds reports a held sub-manifest node no kept tree reaches through a slot that may hold an in-cone path.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.