Documentation
¶
Overview ¶
Package store implements Lantern's persistent header store.
The store is BadgerDB-backed. It keys block headers by their CID (canonical content-addressed lookup) and maintains:
- Per-epoch indexes: h:epoch:<be8> → []byte (set of header CIDs)
- Per-CID header blobs: h:cid:<bytes> → CBOR-encoded BlockHeader
- Canonical tipset per epoch: h:can:<be8> → tipset-key bytes (sorted CIDs)
- Head epoch: h:head → be8
"Canonical" means: the tipset key currently selected as part of the active chain. On reorg we rewrite the canonical pointers for the affected epochs.
The store cooperates with chain/header (per-block validation) and with a caller-provided beacon Verifier for BLS-signature replay on startup.
Written for Lantern. Not a direct lift; structural inspiration from lotus/chain/store but pure pure-Go and header-only.
Background head-sync agent for the persistent header store.
The Sync agent talks to a Lotus-compatible JSON-RPC source. It polls ChainHead at a configurable interval (default 6s, half of Filecoin's 30s block time), and on every new head epoch:
- Walks back from the new head to the most-recent epoch we already have canonical for (or HeadEpoch - MaxBacktrack, whichever is closer) fetching block headers along the way.
- For each fetched header, verifies the block CID and parent linkage.
- Calls Store.SetHead(newHead), which rewrites canonical pointers and fires OnHeadChange listeners. SetHead itself detects parent-CID mismatch against the prior canonical chain and acts as the reorg trigger.
We do not use a gossipsub or libp2p ChainNotify channel here — that's the job of net/libp2p. The Sync agent is the simple "poll for ChainHead" fallback that works against any Lotus-compatible RPC.
Index ¶
- Variables
- type Options
- type RPCSource
- type Stats
- type Store
- func (s *Store) AllHeadersAtEpoch(epoch abi.ChainEpoch) ([]*ltypes.BlockHeader, error)
- func (s *Store) Close() error
- func (s *Store) Get(c cid.Cid) (*ltypes.BlockHeader, error)
- func (s *Store) GetTipSet(tsk ltypes.TipSetKey) (*ltypes.TipSet, error)
- func (s *Store) GetTipSetByHeight(epoch abi.ChainEpoch) (*ltypes.TipSet, error)
- func (s *Store) Head() *ltypes.TipSet
- func (s *Store) HeadEpoch() abi.ChainEpoch
- func (s *Store) LatestBeaconEntry(bh *ltypes.BlockHeader) (*ltypes.BeaconEntry, error)
- func (s *Store) OnHeadChange(cb func(*ltypes.TipSet))
- func (s *Store) Put(bh *ltypes.BlockHeader) error
- func (s *Store) SetHead(_ctx context.Context, ts *ltypes.TipSet) error
- func (s *Store) Stats() Stats
- func (s *Store) Tipset(epoch abi.ChainEpoch) (*ltypes.TipSet, error)
- type Sync
- func (s *Sync) PollOnce(ctx context.Context) error
- func (s *Sync) SetBlockValidator(fn func(ctx context.Context, bh *ltypes.BlockHeader) error, fatal bool)
- func (s *Sync) SetGossipFresh(fn func() bool)
- func (s *Sync) SetGossipObservedHead(fn func() abi.ChainEpoch, tol abi.ChainEpoch)
- func (s *Sync) Start(ctx context.Context) error
- func (s *Sync) Stats() SyncStats
- func (s *Sync) Stop()
- type SyncOptions
- type SyncStats
Constants ¶
This section is empty.
Variables ¶
var ErrNotFound = errors.New("header/store: not found")
ErrNotFound is returned when a header or tipset is missing.
Functions ¶
This section is empty.
Types ¶
type Options ¶
type Options struct {
// BeaconConfig is an optional DRAND verifier used at startup to
// re-validate beacon entries on the most-recent N headers before
// accepting the persisted head as current.
BeaconConfig *beacon.Config
// StartupVerifyDepth: how many recent headers to re-verify on Open.
// 0 means "no verification" (e.g. unit tests).
StartupVerifyDepth int
}
Options configures Open.
type RPCSource ¶
type RPCSource interface {
// HeadEpoch returns the current chain head epoch.
HeadEpoch(ctx context.Context) (abi.ChainEpoch, error)
// TipsetCIDsByHeight returns the block CIDs that form the canonical
// tipset at the given epoch.
TipsetCIDsByHeight(ctx context.Context, h abi.ChainEpoch) ([]cid.Cid, error)
// FetchBlock returns a single BlockHeader, CID-verified.
FetchBlock(ctx context.Context, k cid.Cid) (*ltypes.BlockHeader, error)
}
RPCSource is the minimal RPC surface required by Sync. The Lantern glif client and gateway client both satisfy this interface.
type Stats ¶
type Stats struct {
HeadEpoch abi.ChainEpoch
OpenedAt time.Time
}
Stats captures store-level metrics.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store is a thread-safe persistent header store.
func Open ¶
Open opens (or creates) a BadgerDB-backed header store at the given path. Pass path="" to open an in-memory store (useful for tests).
func (*Store) AllHeadersAtEpoch ¶
func (s *Store) AllHeadersAtEpoch(epoch abi.ChainEpoch) ([]*ltypes.BlockHeader, error)
AllHeadersAtEpoch returns every header we've seen at the given epoch (canonical + competing forks). Useful for reorg debugging.
func (*Store) GetTipSet ¶
GetTipSet resolves a tipset directly by its key (the set of block CIDs). It reads each block header from the store and reassembles the tipset. Returns ErrNotFound if any constituent block is missing. Unlike GetTipSetByHeight this does NOT require the tipset to be on the canonical chain — it serves any tipset whose headers were persisted, which is what callers like ChainGetTipSet(key) need.
func (*Store) GetTipSetByHeight ¶
GetTipSetByHeight returns the canonical tipset at the given epoch. Walks backward for null-round epochs (returns the nearest canonical tipset with height <= epoch).
func (*Store) HeadEpoch ¶
func (s *Store) HeadEpoch() abi.ChainEpoch
HeadEpoch returns the persisted head epoch, or -1 if no head is set.
func (*Store) LatestBeaconEntry ¶
func (s *Store) LatestBeaconEntry(bh *ltypes.BlockHeader) (*ltypes.BeaconEntry, error)
LatestBeaconEntry returns the most recent beacon entry at or before block bh: if bh carries entries, the last one; else walk parents (bounded) until a tipset with entries is found. Mirrors Lotus StateManager.GetLatestBeaconEntry, used by chain/fullvalidate to supply prevBeacon for entry-less blocks (#90). Returns ErrNotFound if none is found within the walk budget.
func (*Store) OnHeadChange ¶
OnHeadChange registers a listener invoked whenever the canonical head advances. Listeners run synchronously in the goroutine that called SetHead — keep them fast.
func (*Store) Put ¶
func (s *Store) Put(bh *ltypes.BlockHeader) error
Put validates and persists a single BlockHeader. The caller is expected to have an established parent linkage; we verify that the header's declared parent CIDs exist in the store (if Height > 0).
func (*Store) SetHead ¶
SetHead marks ts as the canonical head, atomically rewiring canonical tipset pointers for every epoch from ts down to the most recent common ancestor with the previous head.
On reorg (new head's parent chain diverges from current head's), all canonical-tipset pointers between the divergence epoch and the new head are overwritten.
type Sync ¶
type Sync struct {
// contains filtered or unexported fields
}
Sync polls an RPCSource and feeds new heads into a Store.
func NewSync ¶
func NewSync(s *Store, src RPCSource, opts SyncOptions) *Sync
NewSync returns a Sync agent that has not been started.
func (*Sync) PollOnce ¶
PollOnce runs one synchronization cycle. Useful for tests + the Phase 6 demo where we want a deterministic single-shot sync.
func (*Sync) SetBlockValidator ¶
func (s *Sync) SetBlockValidator(fn func(ctx context.Context, bh *ltypes.BlockHeader) error, fatal bool)
SetBlockValidator wires the Full-tier per-block consensus validator (#90). fn runs after the tipset-shape check on each assembled block; fatal decides whether a validation error rejects the tipset (true) or is only logged (false, observe mode). Passing nil disables it. Safe to call before/after Start. Light/PDP tiers never call this.
func (*Sync) SetGossipFresh ¶
SetGossipFresh wires the gossip-freshness callback after construction. The gossipsub ingestor is created after the Sync in both daemon paths, so the callback can't be passed to NewSync; this lets the caller wire it once the ingestor exists (#71). Safe to call before or after Start. Passing nil disables the gossip-aware poll skip.
func (*Sync) SetGossipObservedHead ¶
func (s *Sync) SetGossipObservedHead(fn func() abi.ChainEpoch, tol abi.ChainEpoch)
SetGossipObservedHead wires the gossip-observed-tip callback after construction (#83). Pair with SetGossipFresh to make the #71 skip lag-aware. tol is the SkipLagTolerance in epochs; <=0 uses the default.
type SyncOptions ¶
type SyncOptions struct {
// Interval between ChainHead polls. Default 6s.
Interval time.Duration
// MaxBacktrack caps how far back the agent walks on each poll
// (defends against unbounded catch-up cost). Default 30.
MaxBacktrack abi.ChainEpoch
// BootstrapDepth is how many tipsets to ingest on the very first
// poll when the store is empty. Defaults to MaxBacktrack. Operators
// running against rate-limited public RPC providers (Glif) should
// set this small (1–3) so startup is quick; subsequent polls catch
// up incrementally.
BootstrapDepth abi.ChainEpoch
// CatchUpChunk caps how many epochs the warm-store catch-up advances
// in a single poll. When the store falls behind by more than this,
// each poll ingests a contiguous chunk of this size starting from
// currentHead+1, so lag decreases monotonically without skipping
// epochs (issue #33). 0 means "no cap" (advance straight to head).
// Default 200 — large enough that normal operation (0–2 epochs
// behind) is a single chunk, bounded enough that a deep backlog
// against rate-limited RPC is paced over several polls.
CatchUpChunk abi.ChainEpoch
// GossipFresh, when set, reports whether an external head source
// (the gossipsub block ingestor) has advanced the store's head
// recently — i.e. within roughly the last poll Interval. When it
// returns true the Sync skips its own RPCSource.HeadEpoch() poll for
// that tick, because gossip is already supplying the live head with
// no upstream-RPC dependency. The Sync then acts purely as a
// catch-up fallback: it resumes polling the RPCSource only when
// gossip goes quiet (stale store head). This is what keeps a node
// with healthy gossipsub from hammering a rate-limited public RPC
// (Glif 429) every Interval (#71). Nil disables the optimization
// (the Sync polls every tick, the pre-#71 behavior).
GossipFresh func() bool
// GossipObservedHead, when set, reports the highest chain epoch the
// gossip ingestor has observed/installed - i.e. the gossip layer's
// view of the live tip (>= the canonical store head). It makes the
// #71 GossipFresh skip LAG-AWARE (#83): GossipFresh alone only tells
// us the head moved recently, not that it reached the tip. A node
// whose gossip is fresh-but-lagging (installing some blocks while
// skipping head+N>1 blocks it can't backfill) would otherwise have
// Sync skip its catch-up poll forever, wedging the head ~10-20 epochs
// behind the tip. With this set, Sync skips only when the store head
// is within SkipLagTolerance of the observed tip; when further behind
// it runs the catch-up poll regardless of GossipFresh. Returns a
// negative epoch when no observation is available (treated as "can't
// confirm we're at the tip" -> don't skip). Nil keeps the pre-#83
// behavior (skip purely on GossipFresh).
GossipObservedHead func() abi.ChainEpoch
// SkipLagTolerance is how many epochs behind the gossip-observed tip
// the store head may be and still skip the catch-up poll (#83). Only
// consulted when GossipObservedHead is set. Default 2 (allows the
// normal 0-1 epoch gossip latency without forcing an RPC poll).
SkipLagTolerance abi.ChainEpoch
// StaleResetThreshold is the lag (live head − store head, in epochs)
// beyond which a warm store is considered hopelessly stale and is
// re-bootstrapped near the live head instead of attempting a
// contiguous backfill.
//
// This is the #51 "down for a week" fix. The contiguous catch-up
// (issue #33) walks from currentHead+1 and relies on backfillParents
// connecting each new tipset to the store's existing chain within
// MaxBacktrack epochs. After a multi-day outage the gap is tens of
// thousands of epochs — far beyond MaxBacktrack — so backfill never
// connects, SetHead never fires, and the head pointer freezes at the
// stale epoch (exactly the symptom reported 2026-06-18). Past this
// threshold we stop trying to bridge the gap and re-anchor near live
// head, the same way a cold start does. No keys, wallets, or other
// secrets are touched — only rebuildable chain state.
//
// 0 disables the behaviour (pure contiguous catch-up, legacy).
// Default 2880 (~1 day at 30s epochs): a node behind by more than a
// day re-anchors instead of grinding.
StaleResetThreshold abi.ChainEpoch
// OnStaleReset, when set, is fired once when a stale warm store is
// re-anchored, with (storeHead, liveHead). Used for logging/metrics.
OnStaleReset func(storeHead, liveHead abi.ChainEpoch)
// OnReorg is fired (after Store.SetHead) when the new head's parent
// chain replaced canonical pointers at one or more epochs. The
// argument is the divergence epoch (the deepest epoch whose
// canonical pointer changed).
OnReorg func(divergence abi.ChainEpoch)
}
SyncOptions configures a Sync agent.
type SyncStats ¶
type SyncStats struct {
Polls uint64
SkippedGossipFresh uint64 // #71: ticks where the Glif poll was skipped because gossip was fresh
HeadAdvances uint64
Reorgs uint64
StaleResets uint64
HeadersAdded uint64
LastError string
LastHeadEpoch abi.ChainEpoch
}
SyncStats reports observable Sync activity.