mithril

package
v0.92.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 9, 2026 License: Apache-2.0 Imports: 82 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// BackendV2 restores from incremental Cardano database artifacts
	// (CardanoDatabase, /artifact/cardano-database): per-immutable
	// archives verified against the certified merkle root.
	BackendV2 = "v2"
)

Mithril artifact backends.

View Source
const (
	// DefaultMaxDownloadBytes bounds each compressed archive, including any
	// resumed prefix, when the caller sets no limit. v2 objects have tighter
	// built-in limits.
	DefaultMaxDownloadBytes int64 = 512 << 30
)
View Source
const DefaultStreamNoProgressTimeout = 5 * time.Minute

DefaultStreamNoProgressTimeout is how long a held run may go without committing a batch before it gives up. A batch closes after at most BatchChunks files or MaxBatchBlocks blocks, minutes of download even for dense mainnet files on a slow link, so a longer gap means a stalled or degraded aggregator, and the node is better served syncing from peers than holding them off any longer.

View Source
const MithrilHysteresis = 1.25

MithrilHysteresis is how much faster than Ouroboros Mithril must have been measured before Decide prefers it. Without a margin, two sources measuring about the same rate would make a node switch back and forth every check.

View Source
const RewardStateRepairActiveKey = "mithril_reward_repair_active"

RewardStateRepairActiveKey marks a reward-state repair whose first database write has begun. It is ephemeral and is cleared with sync_status when the import completes.

View Source
const RewardStateRepairPendingKey = "mithril_reward_repair_pending"

RewardStateRepairPendingKey marks a legacy imported database whose reward state must be reconciled against a newer certified ledger snapshot before it can serve. The metadata migration writes it only when it finds the old imported reward-pot row at the recorded Mithril anchor.

Variables

View Source
var (
	// ErrExtractDestinationNotEmpty reports an exclusive extraction whose
	// destination already holds content.
	ErrExtractDestinationNotEmpty = errors.New(
		"mithril: extraction destination is not empty",
	)
	// ErrExtractUnsafePath reports a destination path component that is a
	// symlink, or an existing non-directory where a directory is required.
	ErrExtractUnsafePath = errors.New(
		"mithril: unsafe extraction path",
	)
	// ErrExtractConflictingOptions reports a caller asking for both
	// destination policies at once.
	ErrExtractConflictingOptions = errors.New(
		"mithril: WithMergeIntoDestination and WithReplaceDestination are mutually exclusive",
	)
)

Errors reported when an extraction destination cannot be trusted. The archive-content checks (Zip Slip, symlink entries) guard what the archive asks for; these guard the filesystem the archive is written into, which an attacker may have reached first.

View Source
var ErrArtifactNotFound = errors.New("artifact not found")

ErrArtifactNotFound reports a key that no object occupies.

View Source
var ErrChunkFeedClosed = errors.New("mithril: chunk feed is closed")

ErrChunkFeedClosed is returned by ChunkFeed methods called after Close.

View Source
var ErrChunkNotDelivered = errors.New(
	"mithril: chunk was not delivered by this feed",
)

ErrChunkNotDelivered is returned by Release for a chunk number that Next has not yet returned, including numbers outside the feed's range. Deleting such a trio would pull files out from under a download that may still be writing them.

View Source
var (
	// ErrDownloadTooLarge identifies a configured or expected size limit.
	// It is terminal: retrying cannot increase the permitted file size.
	ErrDownloadTooLarge = errors.New("download size limit exceeded")
)
View Source
var ErrExtractLimitExceeded = errors.New(
	"mithril: archive exceeds extraction limits",
)

ErrExtractLimitExceeded reports an archive with more entries, a larger member, more expanded bytes, or a higher expansion ratio than allowed.

View Source
var ErrExtractSparseMember = errors.New(
	"mithril: sparse archive member is not supported",
)

ErrExtractSparseMember reports a sparse tar member. Sparse holes are synthesized by archive/tar above the compressed-stream accounting, so no Mithril archive accepts them.

View Source
var ErrExtractUnexpectedMember = errors.New(
	"mithril: unexpected archive member",
)

ErrExtractUnexpectedMember reports an archive entry the consumer of that archive does not read.

View Source
var ErrNotOnCertifiedChain = errors.New(
	"local chain is not on the certified Mithril chain",
)

ErrNotOnCertifiedChain reports a local chain whose tip is not on the chain a Mithril certificate covers. Nothing certified can be applied on top of it; the caller must leave the database alone and sync from peers instead.

View Source
var ErrRewardStateRepairWaitingForSnapshot = errors.New(
	"reward-state repair is waiting for a certified snapshot that reaches the existing stable ledger point",
)

ErrRewardStateRepairWaitingForSnapshot means the selected certified state does not reach the database's existing stable Mithril ledger point.

View Source
var ErrSnapshotArchivesMissing = errors.New("snapshot archives are missing")

ErrSnapshotArchivesMissing reports a snapshot whose archives were not all in the store once its metadata had been written, so it was unlisted again.

View Source
var ErrSnapshotRestoreUnavailable = errors.New(
	"mithril: snapshot restore source unavailable",
)

ErrSnapshotRestoreUnavailable is wrapped by a RunSnapshotBackfill error caused by a source the restore reads from not answering usefully: the aggregator or a download location unreachable, timing out, rate limiting, failing with a server error, or answering 404 for anything but the pinned artifact. It says nothing about the database, so a caller retrying an interrupted restore should back off and retry rather than count it against the restore.

View Source
var ErrSnapshotRestoreUnrecoverable = errors.New(
	"mithril: interrupted snapshot restore cannot be finished",
)

ErrSnapshotRestoreUnrecoverable reports an interrupted restore that cannot be finished in place: the aggregator no longer serves the artifact its partly imported ledger state came from. Retrying cannot help. Every row in the database was written by the restore, so removing the database and restoring afresh is the recovery.

View Source
var ErrStaleBatch = errors.New("mithril: applied tip moved under the batch")

ErrStaleBatch reports that the applier's tip moved between the moment a batch was prepared and the moment it would have been applied, so the batch was discarded unapplied. A BatchApplier must return an error satisfying errors.Is(err, ErrStaleBatch) in that case.

View Source
var ErrStreamStalled = errors.New(
	"mithril: stream catch-up committed no batch within its no-progress timeout",
)

ErrStreamStalled reports a held run that committed no batch within its no-progress timeout.

Functions

func AcceptedBackends added in v0.66.0

func AcceptedBackends() []string

AcceptedBackends returns the recognized Mithril artifact backends. It is the single source for backend-name validation: cmd/dingo's resolveMithrilBackend derives from it, and internal/config's validation whitelist (which cannot import this package) verifies parity against it.

func AcceptedNetworks added in v0.70.0

func AcceptedNetworks() []string

AcceptedNetworks returns the recognized Mithril network identifiers. It is the single source for network-name validation: path-safety checks on aggregator-supplied snapshot metadata verify parity against it.

func AggregatorURLForNetwork

func AggregatorURLForNetwork(network string) (string, error)

AggregatorURLForNetwork returns the default aggregator URL for the given network name, or an error if the network is not recognized.

func AncillaryVerificationKeyURLForNetwork added in v0.26.0

func AncillaryVerificationKeyURLForNetwork(network string) (string, error)

AncillaryVerificationKeyURLForNetwork returns the default Mithril ancillary verification key URL for the given network.

func Decide added in v0.87.0

func Decide(in DecisionInput) bool

Decide reports whether the node should catch up from Mithril rather than Ouroboros. The gap is Target.Immutable minus LocalImmutable. A gap below MinGapChunks never uses Mithril. A gap that is large enough uses it when either rate is unknown, and otherwise only when Mithril measured more than MithrilHysteresis times the Ouroboros rate.

func DownloadSnapshot

func DownloadSnapshot(
	ctx context.Context,
	cfg DownloadConfig,
) (string, error)

DownloadSnapshot downloads a snapshot archive from the given URL to the specified destination directory. It returns the path to the downloaded file.

func EscapeText added in v0.85.0

func EscapeText(s string) string

EscapeText returns s with every character that could act on a terminal replaced by a visible escape sequence. Aggregator responses are remote-controlled, so text taken from them must pass through here before it reaches an operator's terminal.

ESC, other C0 and C1 control characters, newlines, bidirectional and other format characters, and invalid UTF-8 are escaped; printable text is returned unchanged.

func ExtractArchive

func ExtractArchive(
	ctx context.Context,
	archivePath string,
	destDir string,
	logger *slog.Logger,
	opts ...ExtractOption,
) (string, error)

ExtractArchive extracts a zstd-compressed tar archive to the specified destination directory. It returns the path to the directory where files were extracted. The context is checked between files so that long-running extractions can be cancelled.

By default the destination is exclusive: it must be empty, extraction is staged in a private directory, and the result is renamed into place only once complete. See WithReplaceDestination and WithMergeIntoDestination for the destinations that need other policies.

func GenesisVerificationKeyURLForNetwork added in v0.26.0

func GenesisVerificationKeyURLForNetwork(network string) (string, error)

GenesisVerificationKeyURLForNetwork returns the default Mithril genesis verification key URL for the given network.

func HumanBytes added in v0.26.0

func HumanBytes(b int64) string

HumanBytes formats a byte count in a human-readable form.

func ImmutableFileForSlot added in v0.87.0

func ImmutableFileForSlot(slot, chunkSlots uint64) (file uint64, ok bool)

ImmutableFileForSlot estimates the immutable file holding slot when every file spans chunkSlots slots, as the ImmutableDB lays out every Cardano network: the Byron epoch length, 10*k. ok is false when chunkSlots is zero, which gives no estimate.

func NeedsSync added in v0.53.0

func NeedsSync(cfg SyncConfig) (bool, error)

NeedsSync reports whether the database at cfg.DataDir requires a (re)sync — i.e. it is empty or its sync_status indicates an incomplete sync.

func NewServerHandler added in v0.81.0

func NewServerHandler(cfg ServerConfig) http.Handler

NewServerHandler returns the handler serving the Mithril aggregator Cardano database artifact API over cfg.Store, plus the archive downloads the artifact locations point at.

func ParseSigningKey added in v0.81.0

func ParseSigningKey(data string) (ed25519.PrivateKey, error)

ParseSigningKey parses an Ed25519 signing key in the Mithril JSON-hex format (or raw hex): a 32-byte seed or a 64-byte private key.

func PruneSnapshots added in v0.81.0

func PruneSnapshots(
	ctx context.Context,
	store ArtifactStore,
	keep int,
) ([]string, error)

PruneSnapshots deletes all but the keep newest complete snapshots from store and returns the hashes it removed. A keep below one retains everything, and a snapshot still being produced is never counted or removed, since it has no metadata yet.

func RewardStateRepairActive added in v0.73.1

func RewardStateRepairActive(db *database.Database) (bool, error)

RewardStateRepairActive reports whether an interrupted reward-state repair is the operation represented by the current in-progress sync status.

func RewardStateRepairPending added in v0.73.1

func RewardStateRepairPending(db *database.Database) (bool, error)

RewardStateRepairPending reports whether startup must reconcile a legacy imported reward state before serving the database.

func RunSnapshotBackfill added in v0.87.0

func RunSnapshotBackfill(
	ctx context.Context,
	cfg SnapshotConfig,
) (point ocommon.Point, err error)

RunSnapshotBackfill restores a Mithril snapshot into an empty database for a node that expires history, and returns the imported ledger state's point.

It verifies the latest artifact's certificate chain, downloads the immutable files from the certified tip down to the one holding the history-expiry cutoff (ledger slot minus StabilityWindow), and downloads and verifies the ancillary ledger state against its signed manifest. Blocks between the certified tip and a ledger state beyond it are fetched from a relay. All of that happens before the first database write, so a failure there leaves the database exactly as it was.

It then loads the blocks, imports the ledger state, records the Mithril trust boundary and the bootstrap marker, and clears the sync state, in the order `dingo mithril sync` does. A failure from the first write on leaves the restore resumable: SnapshotStartupState reports it as SnapshotStartupResume, and a later call resumes it against the same artifact. When the aggregator no longer serves that artifact, a restore that stored only blocks discards them and starts afresh on the latest one; one whose ledger import had begun fails with ErrSnapshotRestoreUnrecoverable, since importing another artifact over the rows already written is not known to leave none of them behind. Only a 404 for the pinned artifact itself, confirmed by a second request, counts as no longer served.

An error caused by a source not answering usefully (see ErrSnapshotRestoreUnavailable) wraps that sentinel, whichever phase it came from.

Probed files wait for the load on disk, not in memory; at most WindowChunks of them are kept, the rest re-downloaded by the load.

func SnapshotRestoreInterrupted added in v0.87.0

func SnapshotRestoreInterrupted(db *database.Database) (bool, error)

SnapshotRestoreInterrupted reports whether db holds a snapshot restore the node itself started and did not finish.

func SnapshotRestoreProgress added in v0.87.0

func SnapshotRestoreProgress(
	ctx context.Context,
	db *database.Database,
) (string, error)

SnapshotRestoreProgress returns a token that changes whenever an interrupted restore advances: its phase, its stored chain tip, or its ledger import checkpoint. Comparing it across a failed attempt tells an attempt that got further from one that did not.

func ValidateVerificationMaterial added in v0.26.0

func ValidateVerificationMaterial(material *VerificationMaterial) error

ValidateVerificationMaterial checks that the certificate's active signer metadata is consistent with the certified Mithril stake distribution used for the same epoch.

func VerifyCertificateChain

func VerifyCertificateChain(
	ctx context.Context,
	client *Client,
	certificateHash string,
) error

VerifyCertificateChain walks the Mithril certificate chain from the given hash back to the genesis certificate. This verifies the chain is unbroken. It does not verify STM cryptographic signatures (Phase 2).

func VerifyGenesisCertificateSignature added in v0.26.0

func VerifyGenesisCertificateSignature(
	cert *Certificate,
	verificationKeyText string,
) error

VerifyGenesisCertificateSignature verifies a genesis certificate signature with the configured Mithril genesis verification key.

func WasBootstrapped added in v0.67.0

func WasBootstrapped(db *database.Database) (bool, error)

WasBootstrapped reports whether db was populated by a Mithril bootstrap. It checks the durable immutable-import marker a completed Mithril sync leaves in sync_state (written after the completion clear, and never removed by normal serve operation), so it stays true for the life of a bootstrapped database. Consensus features whose ledger state cannot be reconstructed from a Mithril snapshot use it to refuse serving such a database -- notably CIP-0163 delegator inactivity, whose per-account expiration state is absent from the cardano-ledger snapshot and cannot be recovered after import.

Databases bootstrapped before the immutable-import marker existed (pre-v0.62.0) carry only the older mithril_ledger_slot trust boundary, so we fall back to it. That boundary is written solely by the Mithril import completion path (updateMithrilReadyState, sync_import.go) and never by a genesis sync, so keying on its presence cannot misclassify a genesis-synced database as bootstrapped.

A marker that stream catch-up created (syncKeyImmutableStreamOnly) does not count: that database replayed every block and imported no snapshot state.

Types

type Aggregator added in v0.81.0

type Aggregator struct {
	// contains filtered or unexported fields
}

Aggregator collects signer registrations and individual STM signatures over stored snapshots, and publishes a certificate for a snapshot once the signatures cover the quorum. An operator explicitly closes the epoch's signer set before signing starts; closure derives the aggregate verification key and issues the genesis certificate.

func NewAggregator added in v0.81.0

func NewAggregator(
	ctx context.Context,
	cfg AggregatorConfig,
) (*Aggregator, error)

NewAggregator validates cfg and restores the signer set and chain head a previous run closed, if any.

type AggregatorConfig added in v0.81.0

type AggregatorConfig struct {
	// Network is the Cardano network name written into certificates; only
	// snapshots of this network are certified.
	Network string
	// Epoch is the epoch signers register for. The genesis certificate is
	// issued at Epoch-1, so Epoch must be at least 1.
	Epoch uint64
	// Parameters are the STM protocol parameters for the signer set.
	Parameters ProtocolParameters
	// GenesisSigningKey signs the genesis certificate that roots the chain.
	GenesisSigningKey ed25519.PrivateKey
	// Store holds the snapshots to certify and receives the certificates.
	Store ArtifactStore
	// OperatorToken authorizes signer registration and registration closure.
	// Use at least 32 unpredictable bytes and transport it over TLS.
	OperatorToken string
}

AggregatorConfig configures an Aggregator.

type ArtifactStore added in v0.81.0

type ArtifactStore interface {
	// Put writes the object at key, replacing any existing object. A
	// reader of key sees either the previous or the complete new object.
	Put(ctx context.Context, key string, r io.Reader) error
	// Open returns a seekable reader over the object at key, or an error
	// wrapping ErrArtifactNotFound.
	Open(ctx context.Context, key string) (io.ReadSeekCloser, error)
	// Subdirs returns the sorted names of the immediate directory-like
	// children of prefix ("" for the store root).
	Subdirs(ctx context.Context, prefix string) ([]string, error)
	// DeletePrefix removes the object whose key is prefix and every object
	// under prefix + "/". Removing nothing is not an error.
	DeletePrefix(ctx context.Context, prefix string) error
}

ArtifactStore holds the objects of produced Mithril snapshot artifacts. Keys are slash-separated relative paths; a key's directory-like prefix groups the objects of one snapshot.

func OpenArtifactStore added in v0.81.0

func OpenArtifactStore(
	ctx context.Context,
	location string,
) (ArtifactStore, error)

OpenArtifactStore opens the store named by location: a filesystem directory, or an s3:// or gcs:// URI when the binary is built with dingo_extra_plugins. Only the artifact-producing and serving commands call it, so a mistyped scheme fails there rather than during a sync.

type BatchApplier added in v0.87.0

type BatchApplier interface {
	// Tip returns the point of the last applied block, or the zero point for
	// an empty ledger.
	Tip() ocommon.Point
	// ApplyTrustedBatch applies blocks, which extend expected, as one batch,
	// and returns the new tip. When the tip is no longer expected it applies
	// nothing and returns an error satisfying errors.Is(err, ErrStaleBatch).
	ApplyTrustedBatch(
		ctx context.Context,
		expected ocommon.Point,
		blocks []ledger.Block,
	) (ocommon.Point, error)
}

BatchApplier applies certified blocks to the ledger. The mithril package cannot import the ledger, so the node adapts its ledger state to this.

type Beacon

type Beacon struct {
	Epoch               uint64 `json:"epoch"`
	ImmutableFileNumber uint64 `json:"immutable_file_number"`
}

Beacon represents a Cardano chain position at a specific epoch and immutable file number.

type BootstrapConfig

type BootstrapConfig struct {
	// Network is the Cardano network name (e.g., "mainnet",
	// "preprod", "preview").
	Network string
	// Backend selects the Mithril artifact backend. BackendV2 restores
	// per-immutable archives verified against the certified merkle
	// root and is the only backend; empty selects it.
	Backend string
	// AggregatorURL overrides the default aggregator URL for the
	// network. If empty, the default URL for the network is used.
	AggregatorURL string
	// AllowInsecureHTTP permits AggregatorURL and snapshot artifact locations
	// to use plain HTTP or resolve to local/private addresses. Defaults to
	// false; this is an explicit escape hatch for local development and tests
	// and should not be set in production.
	AllowInsecureHTTP bool
	// DownloadDir is the directory where the snapshot archive will
	// be downloaded. If empty, a temporary directory is created.
	DownloadDir string
	// CleanupAfterLoad controls whether temporary files are removed
	// after loading completes.
	CleanupAfterLoad bool
	// VerifyCertificateChain enables STM certificate verification against the
	// aggregator. When true, GenesisVerificationKey is required and the
	// bootstrap process verifies the chain back to that pinned genesis key.
	VerifyCertificateChain bool
	// GenesisVerificationKey is the pinned Mithril trust anchor used by STM
	// certificate verification. It is loaded from Cardano network config and
	// validated for parseability before any aggregator request.
	GenesisVerificationKey string
	// AncillaryVerificationKey is the Mithril ancillary verification key loaded
	// from Cardano network config. It is validated for parseability now and
	// will be used when ancillary artifacts are verified cryptographically.
	AncillaryVerificationKey string
	// Logger is used for structured logging.
	Logger *slog.Logger
	// OnProgress is called during download with progress updates.
	OnProgress ProgressFunc
	// DownloadIdleTimeout is the maximum time to wait for download
	// response headers or body bytes before retrying. Zero uses the
	// downloader default; negative disables idle detection.
	DownloadIdleTimeout time.Duration
	// DownloadMaxIdleRetries is the number of consecutive idle retries
	// allowed without additional bytes. Zero uses the downloader default.
	DownloadMaxIdleRetries int
	// DownloadMaxBytes bounds each compressed object, including resumed
	// bytes. Zero uses DefaultMaxDownloadBytes. It is not a cumulative
	// budget across archives, mirrors, or retries.
	DownloadMaxBytes int64
	// DownloadMaxTransientRetries is the maximum number of retry attempts
	// for transient network errors (TLS handshake failures, connection
	// resets, HTTP 429, HTTP 5xx) per download. Zero uses the downloader
	// default. Negative disables transient retries.
	DownloadMaxTransientRetries int

	// CertificateCache, when set, lets certificate verification stop at
	// certificates an earlier verification in this process accepted, and
	// skip the walk for a leaf it already accepted. Nil walks every chain to
	// genesis.
	CertificateCache *CertificateCache
	// OnArtifactSelected, when set, is invoked once this run's aggregator
	// artifact has been resolved, identity-checked and (when enabled)
	// certificate-verified, and before anything is downloaded. A non-nil
	// return aborts the bootstrap.
	//
	// It runs before the first download rather than after the bootstrap so the
	// caller can record the artifact identity durably while the run still owns
	// nothing: an interruption at any later point then has an artifact to
	// resume against, including the artifact-keyed download cache.
	OnArtifactSelected func(SelectedArtifact) error
	// PinnedDigest, when set, selects that exact artifact from the aggregator
	// instead of the latest one: it is resolved as a Cardano database
	// artifact hash. A resumed
	// Mithril sync sets it from the artifact pin the interrupted run recorded,
	// so the resume imports the artifact its partial database rows and phase
	// checkpoints belong to. Empty selects the latest artifact, the normal
	// first-run behaviour.
	PinnedDigest string
	// StartImmutable is the lowest immutable file number to download and
	// extract. Files below it are assumed already present (a Mithril v2
	// catch-up sets this to the immutable-import marker so it only fetches the
	// archives missing from the existing blob store). Zero downloads the full
	// 0..N range, the normal bootstrap behaviour.
	StartImmutable uint64
	// OnChunkContiguous, when set, enables download<->processing
	// pipelining: chunks are fetched in parallel (out of order) by the
	// shared chunk feed, and this callback is invoked from the feed's
	// in-order consumer for each immutable file number in strict
	// contiguous order as soon as that prefix is fully downloaded. The order
	// runs from StartImmutable upwards, not from zero — a catch-up leaves
	// everything below the marker to the blob store this run is adding to, so
	// those archives are neither downloaded nor extracted here and the files
	// need not exist. It lets the caller copy blocks into the blob store while
	// later chunks are still downloading. When nil, downloads run to
	// completion before any processing (legacy behaviour).
	// The callback runs on a single consumer goroutine and serializes
	// processing, so it needs no internal locking.
	OnChunkContiguous func(chunk ContiguousChunk) error
	// Metrics, when not nil, records the chunk downloads of this bootstrap
	// and of any chunk feed or stream catch-up configured with it.
	Metrics *PipelineMetrics
	// contains filtered or unexported fields
}

BootstrapConfig holds configuration for the Mithril bootstrap process.

type BootstrapResult

type BootstrapResult struct {
	// Snapshot is the snapshot that was downloaded and extracted.
	Snapshot *SnapshotListItem
	// ImmutableDir is the path to the extracted ImmutableDB
	// directory. It is the name the directory was vetted under; use
	// ImmutableRoot to read it.
	ImmutableDir string
	// ImmutableRoot is an open handle on ImmutableDir, held from the moment
	// the directory was vetted until Cleanup closes it.
	//
	// The load path opens the ImmutableDB through this handle rather than
	// through ImmutableDir. The directory sits in a download area, so between
	// vetting it and reading it a concurrent writer could put a different tree
	// at that name; resolving the name at load time would then read the
	// replacement, with the vetting having been about something else. The
	// handle refers to the directory that was vetted, and keeps referring to it
	// however the name is repointed.
	//
	// The bootstrap always sets it. A result without it did not come from a
	// vetted lookup, and loading refuses rather than falling back to the
	// pathname — a silent fallback would reinstate exactly the open this
	// replaces.
	ImmutableRoot *os.Root
	// ImmutableDigests is the certified SHA-256 of every file in ImmutableDir,
	// keyed by the name beneath it ("00000.chunk"). Set by the v2 backend,
	// which downloads each immutable trio against a digest list covered by the
	// certificate's merkle root.
	//
	// It is carried beside ImmutableRoot because the two answer different
	// questions and both have to be answered. The handle says the load reads
	// the directory that was vetted. It says nothing about the files in it: a
	// writer who shares the download directory can rename a file of their own
	// over a verified one without ever leaving the directory the handle refers
	// to. The digests are what let the load refuse those bytes, checked from
	// the descriptor the read goes through rather than from a name reopened
	// after the check.
	ImmutableDigests map[string]string
	// ExtractDir is the root directory where the archive was
	// extracted. Contains db/immutable/, db/ledger/, etc.
	ExtractDir string
	// AncillaryDir is the root directory where the ancillary
	// archive was extracted. Contains ledger/<slot>/{meta,state,
	// tables/tvar}. Set on every successful bootstrap.
	AncillaryDir string
	// AncillaryRoot is an open handle on AncillaryDir, on the same terms as
	// ImmutableRoot: the ledger-state import discovers and reads through it, so
	// the tree the signed manifest was checked against is the tree that gets
	// loaded. Set on every successful bootstrap.
	AncillaryRoot *os.Root
	// AncillaryVerified reports that the ancillary tree's contents were checked
	// against the signed ancillary manifest (a verified v2 bootstrap).
	//
	// The ledger-state import accepts a verified tree's state even when it is
	// newer than the certified tip, and refuses a verified tree that yields no
	// state at all.
	AncillaryVerified bool
	// CertificateVerified reports that the bootstrap was asked to verify the
	// certificate chain, so a missing ancillary tree is a verification failure
	// rather than an unverified run that simply produced none.
	CertificateVerified bool
	// AncillaryDigests is the signed ancillary manifest's digest map: every
	// file the ancillary key vouched for, keyed by its slash-separated path
	// under AncillaryDir. Set only when AncillaryVerified is true.
	//
	// The manifest check hashes each file and closes it; the import opens the
	// state and table it selects afterwards. Between those two the tree is the
	// same tree, but a file in it need not be the same file — so the map
	// travels with the handle and the selected files are checked again from
	// the descriptors the import reads through, before anything is parsed.
	AncillaryDigests map[string]string
	// ExtractRoot is an open handle on ExtractDir.
	//
	// ExtractDir is derived inside the download directory like everything else
	// here, so it is held through a handle rather than resolved by name.
	ExtractRoot *os.Root
	// AncillaryArchivePath is the path to the downloaded ancillary
	// archive file. Empty if no ancillary data was downloaded.
	AncillaryArchivePath string

	// TempDir is the auto-created temporary directory that holds
	// all downloaded and extracted files. Set only when
	// BootstrapConfig.DownloadDir was empty. Cleanup() removes it
	// after removing its children.
	TempDir string
	// TempRoot is the retained handle for TempDir. It is set only when TempDir
	// is set and lets Cleanup remove that exact temporary tree.
	TempRoot *os.Root
	// contains filtered or unexported fields
}

BootstrapResult contains the result of a bootstrap operation.

func Bootstrap

func Bootstrap(
	ctx context.Context,
	cfg BootstrapConfig,
) (*BootstrapResult, error)

Bootstrap validates cfg, resolves the aggregator URL, and delegates to the v2 backend, which restores the ImmutableDB trio files and the ancillary ledger state from the aggregator's incremental database artifacts.

The caller is responsible for invoking the immutable DB load using the returned ImmutableDir path. If CleanupAfterLoad is true, the caller should call Cleanup() on the result after loading.

func (*BootstrapResult) Cleanup

func (r *BootstrapResult) Cleanup(logger *slog.Logger)

Cleanup removes the temporary files created during bootstrap. It removes the archive, extract directory, and ancillary directory individually rather than the entire parent directory, to avoid deleting user-specified download directories.

func (*BootstrapResult) CloseHandles added in v0.70.0

func (r *BootstrapResult) CloseHandles()

CloseHandles releases the directory handles the result carries. It is idempotent, and callers that do not clean up (CleanupAfterLoad off, which keeps the extracted tree for a later run) still owe this call once loading is done — the handles are descriptors held for the lifetime of the result.

Not safe to call while another goroutine reads the handle fields: it clears them. Call it once the work that reads them has finished — in Sync that is after the import errgroup is joined.

type CardanoDatabaseAncillary added in v0.54.0

type CardanoDatabaseAncillary struct {
	SizeUncompressed int64                     `json:"size_uncompressed"`
	Locations        []CardanoDatabaseLocation `json:"locations"`
}

CardanoDatabaseAncillary describes the ancillary component (ledger state plus the next in-progress immutable trio) of a v2 artifact.

type CardanoDatabaseDigestEntry added in v0.54.0

type CardanoDatabaseDigestEntry struct {
	ImmutableFileName string `json:"immutable_file_name"`
	Digest            string `json:"digest"`
}

CardanoDatabaseDigestEntry is one immutable-file digest from the v2 digest list.

type CardanoDatabaseDigests added in v0.54.0

type CardanoDatabaseDigests struct {
	SizeUncompressed int64                     `json:"size_uncompressed"`
	Locations        []CardanoDatabaseLocation `json:"locations"`
}

CardanoDatabaseDigests describes the digest-list component of a v2 artifact.

type CardanoDatabaseImmutables added in v0.54.0

type CardanoDatabaseImmutables struct {
	AverageSizeUncompressed int64                     `json:"average_size_uncompressed"`
	Locations               []CardanoDatabaseLocation `json:"locations"`
}

CardanoDatabaseImmutables describes the immutable-archives component of a v2 artifact.

type CardanoDatabaseLocation added in v0.54.0

type CardanoDatabaseLocation struct {
	Type                 string
	URI                  string
	URITemplate          string
	CompressionAlgorithm string
}

CardanoDatabaseLocation is one download location for a component of a Cardano database (v2) artifact. The JSON representation is internally tagged by "type"; the uri field is either a plain string or, for immutable archives, a {"Template": "..."} object whose template contains the {immutable_file_number} placeholder. Unknown location types and uri shapes are tolerated so new aggregator location kinds do not break parsing; such locations are simply skipped during download.

func (*CardanoDatabaseLocation) ImmutableArchiveURI added in v0.54.0

func (l *CardanoDatabaseLocation) ImmutableArchiveURI(num uint64) string

ImmutableArchiveURI resolves the location's URI template for the given immutable file number (zero-padded to 5 digits). Returns an empty string if the location has no URI template.

func (CardanoDatabaseLocation) MarshalJSON added in v0.54.0

func (l CardanoDatabaseLocation) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshaler for CardanoDatabaseLocation, producing the aggregator wire format (uri as a plain string, or as a {"Template": ...} object for templated locations).

func (*CardanoDatabaseLocation) UnmarshalJSON added in v0.54.0

func (l *CardanoDatabaseLocation) UnmarshalJSON(data []byte) error

UnmarshalJSON implements json.Unmarshaler for CardanoDatabaseLocation.

type CardanoDatabaseSnapshot added in v0.54.0

type CardanoDatabaseSnapshot struct {
	Hash                    string                    `json:"hash"`
	MerkleRoot              string                    `json:"merkle_root"`
	Network                 string                    `json:"network"`
	Beacon                  Beacon                    `json:"beacon"`
	CertificateHash         string                    `json:"certificate_hash"`
	TotalDbSizeUncompressed int64                     `json:"total_db_size_uncompressed"`
	Digests                 CardanoDatabaseDigests    `json:"digests"`
	Immutables              CardanoDatabaseImmutables `json:"immutables"`
	Ancillary               CardanoDatabaseAncillary  `json:"ancillary"`
	CardanoNodeVersion      string                    `json:"cardano_node_version"`
	CreatedAt               string                    `json:"created_at"`
}

CardanoDatabaseSnapshot represents the detail response for a v2 artifact (GET /artifact/cardano-database/{hash}).

func CreateSnapshot added in v0.81.0

func CreateSnapshot(
	ctx context.Context,
	cfg CreateSnapshotConfig,
) (*CardanoDatabaseSnapshot, error)

CreateSnapshot produces a Mithril Cardano database (v2) artifact from cfg.DBDir into cfg.Store: one zstd tar archive per immutable file trio, the digest list, an ancillary archive holding the newest ledger state with an Ed25519-signed manifest, and the artifact metadata. The metadata object is written last, so a snapshot is listed only once it is complete.

Archives are deterministic: entries are sorted, carry no timestamps or owners, and compression is single-threaded, so two runs over the same directory produce identical archives and the same artifact hash; only the metadata's CreatedAt differs unless it is set. When a complete snapshot with that hash is already in the store, nothing is written and the stored snapshot, with any certificate it carries, is returned.

func ListSnapshots added in v0.81.0

func ListSnapshots(
	ctx context.Context,
	store ArtifactStore,
) ([]CardanoDatabaseSnapshot, error)

ListSnapshots returns the complete snapshots in store, newest first. A snapshot is complete once its metadata object exists.

func (*CardanoDatabaseSnapshot) ComputeHash added in v0.54.0

func (s *CardanoDatabaseSnapshot) ComputeHash() string

ComputeHash matches the upstream Mithril Cardano database artifact hash: hex(sha256(beacon.epoch as 8-byte big-endian || merkle_root ASCII bytes)).

type CardanoDatabaseSnapshotListItem added in v0.54.0

type CardanoDatabaseSnapshotListItem struct {
	Hash                    string `json:"hash"`
	MerkleRoot              string `json:"merkle_root"`
	Beacon                  Beacon `json:"beacon"`
	CertificateHash         string `json:"certificate_hash"`
	TotalDbSizeUncompressed int64  `json:"total_db_size_uncompressed"`
	CardanoNodeVersion      string `json:"cardano_node_version"`
	CreatedAt               string `json:"created_at"`
}

CardanoDatabaseSnapshotListItem represents one entry returned by the aggregator's v2 artifact list endpoint (GET /artifact/cardano-database).

type CardanoStakeDistribution added in v0.26.0

type CardanoStakeDistribution struct {
	Hash            string                          `json:"hash"`
	CertificateHash string                          `json:"certificate_hash"`
	Epoch           uint64                          `json:"epoch"`
	Pools           []CardanoStakeDistributionParty `json:"pools"`
}

CardanoStakeDistribution represents a downloaded Cardano stake distribution artifact.

type CardanoStakeDistributionListItem added in v0.26.0

type CardanoStakeDistributionListItem struct {
	Hash                 string   `json:"hash"`
	CertificateHash      string   `json:"certificate_hash"`
	CreatedAt            string   `json:"created_at"`
	Locations            []string `json:"locations"`
	CompressionAlgorithm string   `json:"compression_algorithm"`
	Epoch                uint64   `json:"epoch"`
}

CardanoStakeDistributionListItem represents a Cardano stake distribution artifact entry returned by the aggregator.

type CardanoStakeDistributionParty added in v0.26.0

type CardanoStakeDistributionParty struct {
	PoolID string `json:"pool_id"`
	Stake  uint64 `json:"stake"`
}

CardanoStakeDistributionParty represents a stake pool and stake value in a certified Cardano stake distribution artifact.

type CardanoTransactionsBeacon added in v0.26.0

type CardanoTransactionsBeacon struct {
	Epoch       uint64 `json:"epoch"`
	BlockNumber uint64 `json:"block_number"`
}

CardanoTransactionsBeacon represents a Cardano chain position at a specific epoch and block number, used by the CardanoTransactions signed entity type.

type Certificate

type Certificate struct {
	Hash                     string              `json:"hash"`
	PreviousHash             string              `json:"previous_hash"`
	Epoch                    uint64              `json:"epoch"`
	SignedEntityType         SignedEntityType    `json:"signed_entity_type"`
	Metadata                 CertificateMetadata `json:"metadata"`
	ProtocolMessage          ProtocolMessage     `json:"protocol_message"`
	SignedMessage            string              `json:"signed_message"`
	AggregateVerificationKey string              `json:"aggregate_verification_key"`
	MultiSignature           string              `json:"multi_signature"`
	GenesisSignature         string              `json:"genesis_signature"`
}

Certificate represents a Mithril certificate as returned by the aggregator's certificate endpoint (GET /certificate/{hash}).

func (*Certificate) AggregateVerificationKeyBytes added in v0.26.0

func (c *Certificate) AggregateVerificationKeyBytes() ([]byte, error)

AggregateVerificationKeyBytes decodes the aggregate verification key from its encoded string representation.

func (*Certificate) ComputeHash added in v0.26.0

func (c *Certificate) ComputeHash() (string, error)

ComputeHash matches the upstream Mithril certificate hash.

func (*Certificate) IsChainingToItself

func (c *Certificate) IsChainingToItself() bool

IsChainingToItself returns true if this certificate's hash equals its previous hash (i.e., it is the root of the chain). Returns false if either hash is empty to avoid treating malformed certificates as root.

func (*Certificate) IsGenesis

func (c *Certificate) IsGenesis() bool

IsGenesis returns true if the certificate was signed with a genesis signature rather than a multi-signature.

func (*Certificate) MultiSignatureBytes added in v0.26.0

func (c *Certificate) MultiSignatureBytes() ([]byte, error)

MultiSignatureBytes decodes the multi-signature from its encoded string representation.

type CertificateCache added in v0.87.0

type CertificateCache struct {
	// contains filtered or unexported fields
}

CertificateCache remembers the certificates of every chain a verification accepted, so a later verification under the same genesis key stops walking at the first certificate it has already verified.

A certificate enters the cache only after its whole chain verified to a genesis certificate signed by the pinned key, the leaf bound to its artifact, and the stake distributions validated, so a cached certificate is as trustworthy as a freshly walked one. A new leaf still has every new certificate fetched and STM-verified, and the link from the newest of them to the cached one checked the same way as any other link: same-epoch AVK and parameters, or the parent's next AVK and next parameters across an epoch. A leaf the cache has already accepted is not walked at all.

The cache holds each accepted chain once, as a walk already holds it while it runs, and grows by the few certificates the aggregator adds per epoch. A different genesis key empties it. It is safe for concurrent use; the zero value is not usable, so build one with NewCertificateCache.

func NewCertificateCache added in v0.87.0

func NewCertificateCache() *CertificateCache

NewCertificateCache returns an empty cache.

type CertificateChainVerificationResult added in v0.26.0

type CertificateChainVerificationResult struct {
	Certificates       []*Certificate
	LeafCertificate    *Certificate
	GenesisCertificate *Certificate
	SignedEntityKind   string
}

CertificateChainVerificationResult captures the parsed certificate chain and derived leaf/root metadata that higher verification modes can build on.

func VerifyCertificateChainWithMode added in v0.26.0

func VerifyCertificateChainWithMode(
	ctx context.Context,
	client *Client,
	certificateHash string,
	mode VerificationMode,
) (*CertificateChainVerificationResult, error)

VerifyCertificateChainWithMode verifies the Mithril certificate chain using the requested verification mode.

type CertificateMetadata

type CertificateMetadata struct {
	Network     string                   `json:"network"`
	Version     string                   `json:"version"`
	Parameters  ProtocolParameters       `json:"parameters"`
	InitiatedAt string                   `json:"initiated_at"`
	SealedAt    string                   `json:"sealed_at"`
	Signers     []StakeDistributionParty `json:"signers"`
}

CertificateMetadata holds the metadata section of a certificate.

func (CertificateMetadata) ComputeHash added in v0.26.0

func (m CertificateMetadata) ComputeHash() (string, error)

ComputeHash matches the upstream Mithril certificate-metadata hash.

func (*CertificateMetadata) UnmarshalJSON added in v0.80.0

func (m *CertificateMetadata) UnmarshalJSON(data []byte) error

UnmarshalJSON bounds the signer collection while it is decoded. The certificate body limit bounds serialized bytes, but a JSON array of tiny objects expands into a much larger slice of Go structs before a caller can inspect its length.

type CertifiedTarget added in v0.87.0

type CertifiedTarget struct {
	// Immutable is the highest certified immutable file number.
	Immutable uint64
	// Tip is the last block of that file. The certificate does not carry it,
	// so it is zero from LatestCertifiedTarget and, on a feed, is filled in
	// once Next has returned the last chunk (and stays zero if that file
	// holds no blocks).
	Tip ocommon.Point
}

CertifiedTarget identifies the last immutable file the aggregator's certificate covers.

func LatestCertifiedTarget added in v0.87.0

func LatestCertifiedTarget(
	ctx context.Context,
	cfg BootstrapConfig,
) (CertifiedTarget, error)

LatestCertifiedTarget reports the newest certified immutable file number after verifying the artifact and its certificate chain. It fetches no digest list and no immutable data; the returned Tip is zero because nothing certified names it without the data.

Without cfg.CertificateCache every call walks and STM-verifies the whole chain to genesis, one request per certificate plus the stake distributions, which on mainnet is hundreds of requests. With a cache, a call whose leaf was already accepted costs only the artifact requests, and a new leaf costs one request per certificate added since plus the stake distributions; that is what makes it fit to poll.

type ChunkFeed added in v0.87.0

type ChunkFeed struct {
	// contains filtered or unexported fields
}

ChunkFeed downloads the certified immutable files of one Mithril artifact and hands them out in ascending order.

Workers only download and verify trios; Next decodes the chunk it returns, reading it back through the digest-checked database. Decoded blocks therefore exist only for chunks the consumer has taken, never for the verified trios waiting in the window.

Chunks are fetched in parallel but only inside a window that starts at the lowest chunk the consumer has not yet released, so a slow consumer stalls the downloads instead of filling the disk. The window is positional rather than a plain count of slots: with a count, workers could fill every slot with chunks above a slow one, and the consumer would then wait on a chunk that can never start.

func NewChunkFeed added in v0.87.0

func NewChunkFeed(
	ctx context.Context,
	cfg ChunkFeedConfig,
) (*ChunkFeed, error)

NewChunkFeed verifies the latest artifact's certificate chain and its digest list against the certified merkle root, then starts downloading. Nothing is downloaded from an immutable location before both checks pass. ctx bounds the feed's whole lifetime, not just construction; the caller must Close it.

func (*ChunkFeed) Close added in v0.87.0

func (f *ChunkFeed) Close() error

Close stops the feed, waits for every worker to exit, and removes the files it created. It is safe to call more than once and from any goroutine; a blocked Next returns ErrChunkFeedClosed.

func (*ChunkFeed) Next added in v0.87.0

func (f *ChunkFeed) Next(ctx context.Context) (VerifiedChunk, error)

Next returns the next chunk in ascending order, blocking until it has been downloaded and verified, and decodes it. A decode failure, a trio changed on disk among them, fails the feed. It returns io.EOF after the certified target. The first chunk failure is returned once every lower chunk has been delivered, and is returned by every later call. The consumer must Release the lowest outstanding chunk before Next can return the chunk WindowChunks positions ahead of it; until then Next blocks until ctx ends. A cancelled ctx fails only that call, and does not interrupt a decode already started; cancelling the feed's own context fails the feed with its error.

func (*ChunkFeed) Release added in v0.87.0

func (f *ChunkFeed) Release(num uint64) error

Release deletes the trio for chunk num and lets the window advance past it.

A chunk can be released only after Next has returned it: any other number, including one outside the feed's range, fails with ErrChunkNotDelivered. A repeat release of a chunk already released is a no-op. Chunks may be released in any order; the window advances over the contiguous released prefix.

func (*ChunkFeed) Target added in v0.87.0

func (f *ChunkFeed) Target() CertifiedTarget

Target returns the certified target, including the tip once Next has returned the last chunk.

type ChunkFeedConfig added in v0.87.0

type ChunkFeedConfig struct {
	// Bootstrap carries the network, aggregator, trust keys, download limits
	// and DownloadDir. VerifyCertificateChain and AncillaryVerificationKey
	// are ignored: a feed always verifies the certificate chain and never
	// reads the ancillary archive. An empty DownloadDir makes the feed create
	// and remove its own temporary directory.
	Bootstrap BootstrapConfig
	// StartImmutable is the lowest immutable file number to deliver. A value
	// above the certified target yields an empty feed.
	StartImmutable uint64
	// WindowChunks bounds how many chunk trios may exist on disk at once.
	// Zero or negative means unbounded. Trios wait in the window undecoded:
	// a chunk is decoded only when Next returns it, so decoded blocks in
	// memory are bounded by what the consumer holds, not by the window.
	WindowChunks int
	// Workers is the number of concurrent downloads. Zero or negative selects
	// the bootstrap default. It is capped at WindowChunks when that is set,
	// since a worker beyond the window could never start.
	Workers int
	// RawBlocks delivers each chunk's blocks as read from the verified trio,
	// in VerifiedChunk.Raw, instead of decoding them into VerifiedChunk.Blocks.
	RawBlocks bool
	// contains filtered or unexported fields
}

ChunkFeedConfig configures NewChunkFeed.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is an HTTP client for the Mithril aggregator REST API.

func NewClient

func NewClient(
	aggregatorURL string,
	opts ...ClientOption,
) *Client

NewClient creates a new Mithril aggregator API client. The aggregatorURL should be the base URL of the aggregator (e.g., "https://aggregator.release-preprod.api.mithril.network/aggregator").

func (*Client) GetCardanoDatabaseDigests added in v0.54.0

func (c *Client) GetCardanoDatabaseDigests(
	ctx context.Context,
) ([]CardanoDatabaseDigestEntry, error)

GetCardanoDatabaseDigests retrieves the immutable-file digest list for the latest v2 snapshot directly from the aggregator. Corresponds to GET /artifact/cardano-database/digests.

func (*Client) GetCardanoDatabaseSnapshot added in v0.54.0

func (c *Client) GetCardanoDatabaseSnapshot(
	ctx context.Context,
	hash string,
) (*CardanoDatabaseSnapshot, error)

GetCardanoDatabaseSnapshot retrieves the details of a specific v2 Cardano database snapshot by its hash. Corresponds to GET /artifact/cardano-database/{hash}.

func (*Client) GetCardanoStakeDistribution added in v0.26.0

func (c *Client) GetCardanoStakeDistribution(
	ctx context.Context,
	identifier string,
) (*CardanoStakeDistribution, error)

GetCardanoStakeDistribution retrieves a Cardano stake distribution by hash or unique identifier.

func (*Client) GetCertificate

func (c *Client) GetCertificate(
	ctx context.Context,
	hash string,
) (*Certificate, error)

GetCertificate retrieves a certificate by its hash. Corresponds to GET /certificate/{hash}.

func (*Client) GetLatestCardanoDatabaseSnapshot added in v0.54.0

func (c *Client) GetLatestCardanoDatabaseSnapshot(
	ctx context.Context,
) (*CardanoDatabaseSnapshot, error)

GetLatestCardanoDatabaseSnapshot returns the most recent v2 Cardano database snapshot from the aggregator, sorted by epoch (descending) with immutable file number as tie-breaker.

func (*Client) GetMithrilStakeDistribution added in v0.26.0

func (c *Client) GetMithrilStakeDistribution(
	ctx context.Context,
	hash string,
) (*MithrilStakeDistribution, error)

GetMithrilStakeDistribution retrieves a Mithril stake distribution by hash.

func (*Client) ListCardanoDatabaseSnapshots added in v0.54.0

func (c *Client) ListCardanoDatabaseSnapshots(
	ctx context.Context,
) ([]CardanoDatabaseSnapshotListItem, error)

ListCardanoDatabaseSnapshots retrieves the list of available v2 Cardano database snapshots from the aggregator. Corresponds to GET /artifact/cardano-database.

func (*Client) ListCardanoStakeDistributions added in v0.26.0

func (c *Client) ListCardanoStakeDistributions(
	ctx context.Context,
) ([]CardanoStakeDistributionListItem, error)

ListCardanoStakeDistributions retrieves available Cardano stake distributions from the aggregator.

func (*Client) ListMithrilStakeDistributions added in v0.26.0

func (c *Client) ListMithrilStakeDistributions(
	ctx context.Context,
) ([]MithrilStakeDistributionListItem, error)

ListMithrilStakeDistributions retrieves available Mithril stake distributions from the aggregator.

type ClientOption

type ClientOption func(*Client)

ClientOption is a functional option for configuring a Client.

func WithAllowInsecureHTTP added in v0.70.0

func WithAllowInsecureHTTP() ClientOption

WithAllowInsecureHTTP permits the client to send requests to a plain-HTTP aggregator or local/private address. By default, NewClient's aggregatorURL and every request it issues must use HTTPS and resolve to public addresses; this is an explicit escape hatch for local development and tests (e.g. against an httptest server) and should not be set in production.

func WithHTTPClient

func WithHTTPClient(hc *http.Client) ClientOption

WithHTTPClient sets a custom *http.Client for the Mithril client. Unless WithAllowInsecureHTTP is also set, its redirect policy is wrapped and its *http.Transport is cloned with proxy, custom-dialer, and private-address bypasses removed. Other RoundTripper implementations are rejected at use.

type ContiguousChunk added in v0.70.0

type ContiguousChunk struct {
	// Dir is the directory the chunks are extracted into. It is the name the
	// directory was vetted under, for messages; read through Root.
	Dir string
	// Root is the open handle extraction writes through. Read through it
	// rather than through Dir: the name can be repointed while the download
	// runs and the handle cannot.
	Root *os.Root
	// Digests is the certified SHA-256 of every file the download covers,
	// keyed by the name beneath Dir. The handle settles which directory is
	// read and these settle which bytes, which is the half a handle cannot
	// carry — see BootstrapResult.ImmutableDigests.
	Digests map[string]string
	// Start is the lowest immutable file number this run covers, from
	// BootstrapConfig.StartImmutable. Anything below it was left to the blob
	// store the run is adding to and is not in Dir.
	Start uint64
	// Num is the highest immutable file number whose trio is complete. Every
	// number in [Start, Num] has been downloaded, verified and extracted;
	// below Start, nothing has, which is why the range is given rather than
	// implied. Chunks are addressed by number — a number is not a position in
	// Dir's listing unless Start is zero.
	Num uint64
}

ContiguousChunk describes the contiguous immutable prefix a pipelined bootstrap has finished downloading, verifying and extracting.

type CreateSnapshotConfig added in v0.81.0

type CreateSnapshotConfig struct {
	// Network is the Cardano network name recorded in the artifact.
	Network string
	// DBDir is a cardano-node style database directory holding a sealed
	// immutable/ directory and the ledger/ snapshots taken at its tip.
	DBDir string
	// AncillarySigningKey signs the ancillary manifest. Clients verify it
	// with the matching ancillary verification key.
	AncillarySigningKey ed25519.PrivateKey
	// CardanoNodeVersion is recorded verbatim in the artifact.
	CardanoNodeVersion string
	// CreatedAt is recorded in the artifact metadata; the zero value means
	// now. The archives are a function of DBDir and AncillarySigningKey.
	CreatedAt time.Time
	// Store receives the produced objects.
	Store ArtifactStore
	// Logger receives progress; nil discards it.
	Logger *slog.Logger
}

CreateSnapshotConfig describes one snapshot production run.

type DecisionInput added in v0.87.0

type DecisionInput struct {
	// LocalImmutable is the immutable file holding the local ledger tip.
	LocalImmutable uint64
	// Target is the aggregator's certified target. Only Target.Immutable is
	// read: LatestCertifiedTarget leaves Tip zero.
	Target CertifiedTarget
	// OuroborosBPS is the blocks per second Ouroboros applied over the last
	// check interval. Zero means unknown.
	OuroborosBPS float64
	// MithrilBPS is the blocks per second the last Mithril catch-up applied.
	// Zero means unknown.
	MithrilBPS float64
	// MinGapChunks is the smallest certified gap, in immutable files, worth
	// a catch-up. Zero is treated as one: any gap at all.
	MinGapChunks uint64
}

DecisionInput is what Decide weighs.

type DigestMismatchError added in v0.70.8

type DigestMismatchError struct {
	// FileName is the immutable file the digest list names ("05471.chunk").
	FileName string
	// Expected is the certified digest; Observed is what the bytes hashed to.
	Expected string
	Observed string
}

DigestMismatchError reports a file whose bytes do not match the digest the artifact's certified digest list carries for it.

It is a diagnosis, never a relaxation: every path that produces one still refuses the bytes. It exists so the refusal can name the file, both digests and the source that served them, which is what an operator needs to tell a bad replica apart from a bad local cache and to report a mis-published archive.

func (*DigestMismatchError) Error added in v0.70.8

func (e *DigestMismatchError) Error() string

type DownloadConfig

type DownloadConfig struct {
	// URL is the download URL for the snapshot archive.
	URL string
	// DestDir is the directory where the archive will be saved.
	DestDir string
	// Filename is the name of the downloaded file. If empty, a
	// default name is generated from the snapshot digest.
	Filename string
	// ExpectedSize is the expected file size in bytes. When > 0,
	// the downloaded file size is verified after download. A
	// mismatch returns an error.
	ExpectedSize int64
	// MaxBytes bounds the complete downloaded file, including resumed bytes,
	// even when ExpectedSize or Content-Length is absent. Zero selects
	// DefaultMaxDownloadBytes; negative values are invalid. This is a
	// per-object bound, not an aggregate bootstrap or transfer-byte budget.
	MaxBytes int64
	// Logger is used for logging download progress.
	Logger *slog.Logger
	// OnProgress is called periodically with download progress.
	OnProgress ProgressFunc
	// IdleTimeout is the maximum time to wait for response headers
	// or body bytes before retrying. If zero, a conservative default
	// is used. If negative, idle detection is disabled.
	IdleTimeout time.Duration
	// MaxIdleRetries is the number of consecutive retry attempts
	// after idle timeouts that make no additional download progress.
	// If zero, a conservative default is used.
	MaxIdleRetries int
	// MaxTransientRetries is the maximum number of retry attempts for
	// transient network errors (TLS handshake failures, connection
	// resets, unexpected EOF, HTTP 429, HTTP 5xx). If zero, a
	// conservative default is used. If negative, transient retries
	// are disabled.
	MaxTransientRetries int

	// HTTPClient, when non-nil, is reused for the download instead of
	// constructing a fresh client per call. Callers that fetch many
	// files (the v2 immutable pool) pass one shared keep-alive client so
	// connections are pooled across files. Unless AllowInsecureHTTP is set,
	// custom transports are cloned and restricted before use. When nil, a
	// per-call client with keep-alives disabled is used.
	HTTPClient *http.Client
	// AllowInsecureHTTP permits URL to use plain HTTP or resolve to a
	// local/private address. By default, Validate rejects either; this is an
	// explicit escape hatch for local development and tests (e.g. against an
	// httptest server) and should not be set in production.
	AllowInsecureHTTP bool
	// contains filtered or unexported fields
}

DownloadConfig holds configuration for downloading a snapshot archive.

func (DownloadConfig) Validate added in v0.47.0

func (cfg DownloadConfig) Validate() error

Validate checks DownloadConfig values before use.

type DownloadProgress

type DownloadProgress struct {
	BytesDownloaded int64
	TotalBytes      int64
	Percent         float64
	BytesPerSecond  float64
	// Artifact identifies the artifact whose progress is being reported.
	// It is populated by the bootstrap orchestration layer so concurrent
	// downloads can be distinguished by callers consuming one callback.
	Artifact string
	// SnapshotHash identifies the Mithril snapshot or Cardano database
	// artifact that owns the download.
	SnapshotHash string
	// ArtifactsCompleted and ArtifactsTotal are populated for aggregate
	// progress, such as the v2 immutable archive worker pool.
	ArtifactsCompleted uint64
	ArtifactsTotal     uint64
}

DownloadProgress reports download progress to a callback.

type ExtractOption added in v0.70.0

type ExtractOption func(*extractConfig)

ExtractOption configures how ExtractArchive treats its destination.

func WithMergeIntoDestination added in v0.70.0

func WithMergeIntoDestination() ExtractOption

WithMergeIntoDestination extracts directly into the destination, adding to whatever is already there.

This exists for destinations that several archives populate together — the parallel immutable-archive download builds one directory from many archives, so it can neither refuse a non-empty destination nor swap the directory out from under a concurrent extraction. Merging forgoes the private-staging guarantee, so every write still goes through the per-component symlink checks below.

func WithReplaceDestination added in v0.70.0

func WithReplaceDestination() ExtractOption

WithReplaceDestination allows an exclusive extraction to proceed when the destination already holds content, replacing it with the freshly extracted tree instead of refusing.

This is the recovery path for a destination left behind by an interrupted or superseded run. Replacement is a swap of a directory staged elsewhere, never a write into the existing one, so pre-existing content is discarded rather than merged with or written through.

func WithZstdLimits added in v0.70.11

func WithZstdLimits(maxWindow, maxMemory uint64) ExtractOption

WithZstdLimits overrides the decoder resource limits for an extraction. Both values must be positive; zero keeps the safe defaults.

type ImmutableArchiveAttempt added in v0.70.8

type ImmutableArchiveAttempt struct {
	// Source is the local cache label or the redacted location URI.
	Source string
	// Location is the 1-based index into the artifact's immutable location
	// list, or 0 for the local cache.
	Location int
	// Err is why the attempt was rejected.
	Err error
}

ImmutableArchiveAttempt records one source tried for an immutable trio.

func (ImmutableArchiveAttempt) Mismatch added in v0.70.8

Mismatch returns the digest mismatch this attempt was rejected for, or nil when it failed for another reason (a download or extraction failure).

type ImmutableArchiveError added in v0.70.8

type ImmutableArchiveError struct {
	ArtifactHash        string
	Epoch               uint64
	ImmutableFileNumber uint64
	// Attempts is every source tried, in order, including the local cache
	// when a cached trio was present and rejected.
	Attempts []ImmutableArchiveAttempt
	// Locations is how many published locations the artifact carried.
	Locations int
}

ImmutableArchiveError reports that no source produced an immutable trio matching the artifact's certified digest list, with the per-source evidence needed to compare replicas.

Fail-closed is the point: the digest list is verified against the artifact's certificate merkle root before any archive is fetched, so bytes that disagree with it are refused whatever their source. This type only makes the refusal actionable.

func (*ImmutableArchiveError) Error added in v0.70.8

func (e *ImmutableArchiveError) Error() string

func (*ImmutableArchiveError) Unwrap added in v0.70.8

func (e *ImmutableArchiveError) Unwrap() error

Unwrap exposes the last attempt's cause so errors.Is/errors.As still reach the underlying download, extraction or mismatch error.

type MithrilStakeDistribution added in v0.26.0

type MithrilStakeDistribution struct {
	Hash            string                          `json:"hash"`
	CertificateHash string                          `json:"certificate_hash"`
	Epoch           uint64                          `json:"epoch"`
	Signers         []MithrilStakeDistributionParty `json:"signers"`
}

MithrilStakeDistribution represents a downloaded Mithril stake distribution artifact.

type MithrilStakeDistributionListItem added in v0.26.0

type MithrilStakeDistributionListItem struct {
	Hash                 string   `json:"hash"`
	CertificateHash      string   `json:"certificate_hash"`
	CreatedAt            string   `json:"created_at"`
	Locations            []string `json:"locations"`
	CompressionAlgorithm string   `json:"compression_algorithm"`
	Epoch                uint64   `json:"epoch"`
}

MithrilStakeDistributionListItem represents a Mithril stake distribution artifact entry returned by the aggregator.

type MithrilStakeDistributionParty added in v0.26.0

type MithrilStakeDistributionParty struct {
	PartyID         string `json:"party_id"`
	Stake           uint64 `json:"stake"`
	VerificationKey string `json:"verification_key"`
}

MithrilStakeDistributionParty represents a Mithril signer and its associated stake and verification key in the certified stake distribution.

func (*MithrilStakeDistributionParty) VerificationKeyBytes added in v0.26.0

func (p *MithrilStakeDistributionParty) VerificationKeyBytes() ([]byte, error)

VerificationKeyBytes decodes the signer's verification key from its encoded string representation.

type Mode added in v0.87.0

type Mode int

Mode is how a serving node uses Mithril while it is behind the certified immutable tip.

const (
	// ModeManual leaves Mithril to the explicit `dingo mithril sync` command.
	ModeManual Mode = iota
	// ModeStream replays certified blocks through the ledger for effects.
	ModeStream
	// ModeSnapshot restores a snapshot and backfills block history.
	ModeSnapshot
)

func SelectMode added in v0.87.0

func SelectMode(enabled, historyExpiry bool) Mode

SelectMode picks the node-managed Mithril mode from mithril.enabled and historyExpiry.enabled. Stream mode keeps all history, so a node that expires history restores a snapshot instead of replaying it.

type NetworkConfig added in v0.26.0

type NetworkConfig struct {
	AggregatorURL               string
	GenesisVerificationKeyURL   string
	AncillaryVerificationKeyURL string
}

NetworkConfig describes the Mithril trust endpoints for a specific Cardano network.

func NetworkConfigForNetwork added in v0.26.0

func NetworkConfigForNetwork(network string) (NetworkConfig, error)

NetworkConfigForNetwork returns the default Mithril network configuration for the given network name, or an error if the network is not recognized.

type PipelineMetrics added in v0.87.0

type PipelineMetrics struct {
	// contains filtered or unexported fields
}

PipelineMetrics instruments the chunk feed and the stream runner. Every method is safe on a nil *PipelineMetrics and then records nothing, so a caller without a registry pays only the nil check.

func NewPipelineMetrics added in v0.87.0

func NewPipelineMetrics(reg prometheus.Registerer) (*PipelineMetrics, error)

NewPipelineMetrics registers the pipeline metrics on reg. A nil reg returns nil metrics. Collectors already registered on reg, by an earlier call on the same registry, are reused.

type ProgressFunc

type ProgressFunc func(DownloadProgress)

ProgressFunc is a callback invoked periodically during download to report progress.

type ProtocolMessage

type ProtocolMessage struct {
	MessageParts map[string]string `json:"message_parts"`
}

ProtocolMessage represents the protocol message included in a certificate.

func (ProtocolMessage) ComputeHash added in v0.26.0

func (p ProtocolMessage) ComputeHash() string

ComputeHash matches the upstream Mithril protocol-message hash.

func (*ProtocolMessage) UnmarshalJSON added in v0.80.0

func (p *ProtocolMessage) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes the message parts, failing once they exceed maxProtocolMessageParts entries.

type ProtocolParameters

type ProtocolParameters struct {
	K    uint64  `json:"k"`
	M    uint64  `json:"m"`
	PhiF float64 `json:"phi_f"`
}

ProtocolParameters represents the Mithril protocol parameters used during signing.

func (ProtocolParameters) ComputeHash added in v0.26.0

func (p ProtocolParameters) ComputeHash() string

ComputeHash matches the upstream Mithril protocol-parameters hash.

type ResolvedStakeDistribution added in v0.26.0

type ResolvedStakeDistribution struct {
	Kind                     string
	MithrilStakeDistribution *MithrilStakeDistribution
	CardanoStakeDistribution *CardanoStakeDistribution
}

ResolvedStakeDistribution bundles the stake-distribution artifact selected for a certificate verification flow.

func ResolveStakeDistributionForCertificate added in v0.26.0

func ResolveStakeDistributionForCertificate(
	ctx context.Context,
	client *Client,
	verification *CertificateChainVerificationResult,
) (*ResolvedStakeDistribution, error)

ResolveStakeDistributionForCertificate locates the stake-distribution artifact referenced by a verified certificate chain result.

type SelectedArtifact added in v0.70.8

type SelectedArtifact struct {
	Backend         string
	Network         string
	Digest          string
	Beacon          Beacon
	CertificateHash string
}

SelectedArtifact is the identity of the aggregator artifact a bootstrap run resolved, reported to BootstrapConfig.OnArtifactSelected before any download begins. Digest is the Cardano database artifact hash, matching what BootstrapConfig.PinnedDigest accepts.

type ServerConfig added in v0.81.0

type ServerConfig struct {
	// Store holds the produced artifacts.
	Store ArtifactStore
	// PublicBaseURL is the configured public HTTP(S) origin used in artifact
	// download locations. It is required for detail responses and must use
	// HTTPS except for loopback origins used by local deployments and tests.
	PublicBaseURL string
	// RedirectBaseURL, when set, answers artifact file requests with a
	// redirect to RedirectBaseURL + "/" + key instead of streaming the
	// object from Store. It is meant for remote stores whose objects are
	// publicly readable under that base URL.
	RedirectBaseURL string
	// Aggregator, when set, mounts its signer registration, signature and
	// stake distribution endpoints beside the artifact API.
	Aggregator *Aggregator
	// Logger receives request failures; nil discards them.
	Logger *slog.Logger
}

ServerConfig configures the snapshot artifact HTTP handler.

type SignedEntityType

type SignedEntityType struct {
	// contains filtered or unexported fields
}

SignedEntityType represents the type and parameters of the signed entity in a certificate. The JSON representation uses a tagged union where the key is the entity type name.

func (*SignedEntityType) CardanoDatabase added in v0.54.0

func (s *SignedEntityType) CardanoDatabase() *Beacon

CardanoDatabase attempts to parse the signed entity as a CardanoDatabase beacon. Returns nil if the entity type does not match.

func (*SignedEntityType) CardanoImmutableFilesFull

func (s *SignedEntityType) CardanoImmutableFilesFull() *Beacon

CardanoImmutableFilesFull attempts to parse the signed entity as a CardanoImmutableFilesFull beacon. Returns nil if the entity type does not match.

func (*SignedEntityType) CardanoStakeDistribution added in v0.26.0

func (s *SignedEntityType) CardanoStakeDistribution() *Beacon

CardanoStakeDistribution attempts to parse the signed entity as a CardanoStakeDistribution beacon. Returns nil if the entity type does not match.

func (*SignedEntityType) CardanoTransactions added in v0.26.0

func (s *SignedEntityType) CardanoTransactions() *CardanoTransactionsBeacon

CardanoTransactions attempts to parse the signed entity as a CardanoTransactions beacon containing epoch and block_number. Returns nil if the entity type does not match.

func (*SignedEntityType) Kind added in v0.26.0

func (s *SignedEntityType) Kind() (string, error)

Kind returns the tagged union key for the signed entity type.

func (SignedEntityType) MarshalJSON

func (s SignedEntityType) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshaler for SignedEntityType.

func (*SignedEntityType) MithrilStakeDistribution added in v0.26.0

func (s *SignedEntityType) MithrilStakeDistribution() *Beacon

MithrilStakeDistribution attempts to parse the signed entity as a MithrilStakeDistribution beacon. Returns nil if the entity type does not match.

func (*SignedEntityType) Raw

func (s *SignedEntityType) Raw() json.RawMessage

Raw returns the raw JSON of the signed entity type.

func (*SignedEntityType) UnmarshalJSON

func (s *SignedEntityType) UnmarshalJSON(data []byte) error

UnmarshalJSON implements json.Unmarshaler for SignedEntityType.

type SnapshotBase

type SnapshotBase struct {
	Digest               string   `json:"digest"`
	Network              string   `json:"network"`
	Beacon               Beacon   `json:"beacon"`
	CertificateHash      string   `json:"certificate_hash"`
	Size                 int64    `json:"size"`
	AncillarySize        int64    `json:"ancillary_size"`
	CreatedAt            string   `json:"created_at"`
	Locations            []string `json:"locations"`
	AncillaryLocations   []string `json:"ancillary_locations"`
	CompressionAlgorithm string   `json:"compression_algorithm"`
	CardanoNodeVersion   string   `json:"cardano_node_version"`
}

SnapshotBase contains the fields shared by both the list and detail snapshot responses from the Mithril aggregator.

func (*SnapshotBase) CreatedAtTime

func (s *SnapshotBase) CreatedAtTime() (time.Time, error)

CreatedAtTime parses the CreatedAt string into a time.Time value.

type SnapshotConfig added in v0.87.0

type SnapshotConfig struct {
	// Feed carries the aggregator, trust keys and download limits. The
	// ancillary verification key is required. StartImmutable is ignored:
	// the restore derives the range from the ledger state. WindowChunks
	// bounds the probed files kept on disk for the load, and separately
	// each feed that loads the rest, so the trios on disk stay within it
	// except when the cutoff lies below the probe bound: the feed loading
	// the files below the probes then runs while the kept probes wait, for
	// a peak of twice WindowChunks. Zero keeps up to maxCutoffProbes probed
	// files.
	Feed ChunkFeedConfig
	// DB is the empty database to restore into, or the one holding the
	// restore to resume.
	DB *database.Database
	// StabilityWindow is the history-expiry retention, in slots: every block
	// at or above the ledger state's slot minus it is loaded.
	StabilityWindow uint64
	// CardanoNodeConfig supplies era epoch lengths to the ledger import.
	CardanoNodeConfig *cardano.CardanoNodeConfig
	// DataDir is the node's data directory. The restore downloads under it
	// when Feed.Bootstrap.DownloadDir is empty. One of the two is required.
	DataDir    string
	Logger     *slog.Logger
	OnProgress SyncProgressFunc
	// contains filtered or unexported fields
}

SnapshotConfig configures RunSnapshotBackfill.

type SnapshotListItem

type SnapshotListItem struct {
	SnapshotBase
}

SnapshotListItem represents a snapshot entry returned by the aggregator's list endpoint (GET /artifact/snapshots).

type SnapshotStartup added in v0.87.0

type SnapshotStartup int

SnapshotStartup is what a restore at node startup does with a database.

const (
	// SnapshotStartupFresh is an empty database: no sync in progress, no
	// stored block and no ledger tip. A restore starts from scratch.
	SnapshotStartupFresh SnapshotStartup = iota
	// SnapshotStartupResume is a restore of the node's own that was
	// interrupted. A restore resumes it against the artifact it pinned.
	SnapshotStartupResume
	// SnapshotStartupPopulated is anything else: a database with chain data,
	// or one another sync left incomplete. Nothing is restored over it.
	SnapshotStartupPopulated
)

func SnapshotStartupState added in v0.87.0

func SnapshotStartupState(
	ctx context.Context,
	db *database.Database,
) (SnapshotStartup, error)

SnapshotStartupState classifies db for a restore at node startup.

type StakeDistributionParty

type StakeDistributionParty struct {
	PartyID string `json:"party_id"`
	Stake   uint64 `json:"stake"`
}

StakeDistributionParty represents a signer in the certificate metadata with their party ID and stake.

func (StakeDistributionParty) ComputeHash added in v0.26.0

func (p StakeDistributionParty) ComputeHash() string

ComputeHash matches the upstream Mithril signer hash used in certificate metadata hashing.

type StoragePlugins added in v0.68.0

type StoragePlugins struct {
	Blob     plugin.Selection
	Metadata plugin.Selection
}

StoragePlugins contains canonical storage provider selections used during Mithril bootstrap. Empty provider names select Badger blob storage and SQLite metadata storage.

type StreamConfig added in v0.87.0

type StreamConfig struct {
	// Feed configures each chunk feed the run opens. Its StartImmutable is
	// ignored: the run derives the start from the ledger tip. An empty
	// Bootstrap.DownloadDir selects a directory under DataDir. The download
	// directory must belong to this run alone: stale feed directories in it
	// are deleted before each feed opens.
	Feed ChunkFeedConfig
	// DataDir is the node's data directory. The run downloads under it when
	// Feed.Bootstrap.DownloadDir is empty, so a crash leaves nothing in the
	// OS temp directory that no later run would clean up. One of the two is
	// required.
	DataDir string
	// ChunkSlots is the number of slots each immutable file spans: the Byron
	// epoch length, 10*k from the Byron genesis, for every Cardano network.
	// It turns the ledger tip's slot into an estimate of the tip's file. Zero
	// means no estimate, and the file is found by bisection.
	ChunkSlots uint64
	// BatchChunks is the most immutable files whose last block one batch may
	// carry. Zero or negative means one.
	BatchChunks int
	// MaxBatchBlocks is the most blocks one ApplyTrustedBatch call carries;
	// a chunk with more is split across batches. Zero or negative means no
	// limit.
	MaxBatchBlocks int
	// DB holds the immutable-import marker the run resumes from and advances.
	DB *database.Database
	// Applier is the ledger the blocks are applied to. Its tip must be on the
	// certified chain.
	Applier BatchApplier
	Logger  *slog.Logger
	// Hold, when set, is called once, after the run has verified its
	// artifact and digest list and located the applier's tip, and before it
	// re-reads the tip to apply anything; the run calls the returned release
	// on every exit, after its no-progress watchdog has stopped, so a release
	// may take longer than NoProgressTimeout. Everything that asks the aggregator about the artifact
	// happens before it, so a slow or failing aggregator delays only the
	// unheld half. Nil holds nothing.
	Hold func() (release func())
	// NoProgressTimeout ends a run, with ErrStreamStalled, once it has held
	// for this long without committing a batch. Zero or negative selects
	// DefaultStreamNoProgressTimeout.
	NoProgressTimeout time.Duration
	// contains filtered or unexported fields
}

StreamConfig configures RunStreamCatchUp.

type StreamResult added in v0.87.0

type StreamResult struct {
	// Tip is the applier's tip when the run ended.
	Tip ocommon.Point
	// Immutable is the highest immutable file whose blocks are all committed,
	// as recorded in the import marker. It is zero when no marker was written.
	Immutable uint64
	// Restarts counts replays after the applier's tip moved under a batch.
	Restarts int
}

StreamResult reports where a stream catch-up stopped.

func RunStreamCatchUp added in v0.87.0

func RunStreamCatchUp(
	ctx context.Context,
	cfg StreamConfig,
) (StreamResult, error)

RunStreamCatchUp streams the certified immutable files above the local chain into the applier, in batches, until the certified target.

The run starts at the immutable file after the import marker, or at file 0 with no marker. Certified blocks at or below the applier's tip are skipped, and the tip must be found on the certified chain before anything is applied: either as one of those blocks, as the parent of the first block after it, or, when the tip is above every certified block, as a descendant of the last one (a no-op). A tip found nowhere fails with ErrNotOnCertifiedChain and nothing is written.

A chunk is released, and recorded in the marker, only once its last block has committed. When the applier reports ErrStaleBatch the feed is closed and the run starts over from the applier's new tip.

Stream catch-up imports no ledger snapshot, so it never records the Mithril trust boundary, and its marker does not make WasBootstrapped true.

Callers must treat every non-nil error, whatever it wraps, as a reason to sync from peers instead: the run stops at a committed batch boundary, so the ledger is always consistent when it returns.

type SyncConfig added in v0.53.0

type SyncConfig struct {
	Network                string                     // "mainnet" | "preprod" | "preview"
	DataDir                string                     // node database path
	StorageMode            string                     // "api" | "core"
	CardanoNodeConfig      *cardano.CardanoNodeConfig // genesis + Mithril verification keys; if nil, loaded from EmbeddedConfigFS for Network
	CardanoConfigPath      string                     // optional explicit config path (else the network's embedded config)
	Backend                string                     // Mithril artifact backend; same semantics as BootstrapConfig.Backend (empty selects v2)
	AggregatorURL          string                     // optional; defaults per-network
	AllowInsecureHTTP      bool                       // permit insecure/local destinations; local dev/test only
	DownloadDir            string                     // optional; defaults to <DataDir>/.mithril-cache
	DownloadIdleTimeout    string                     // optional; passed to BootstrapConfig
	DownloadMaxIdleRetries int                        // must be >= 0
	DownloadMaxBytes       int64                      // per compressed object; zero uses DefaultMaxDownloadBytes
	PinnedDigest           string                     // optional exact artifact identity for a fresh bootstrap (database hash)
	VerifyCertChain        bool
	CleanupAfterLoad       bool
	// RepairLegacyRewardState forces a same-chain Mithril reconciliation for
	// an existing database marked by the reward-pot migration. The node may
	// serve only after an artifact covers the local tip and the reconciliation
	// completes; a clean bootstrap is never selected for this path.
	RepairLegacyRewardState bool
	StoragePlugins          StoragePlugins
	RunMode                 string
	BackfillBatchSize       int
	DatabaseWorkers         int
	Tracing                 bool             // OpenTelemetry tracing enabled; forwarded to the metadata pool
	Logger                  *slog.Logger     // optional; defaults to slog.Default()
	OnProgress              SyncProgressFunc // optional
	Metrics                 *PipelineMetrics // optional; records the immutable chunk downloads
}

SyncConfig is the input to a full Mithril bootstrap of a node database.

type SyncPhase added in v0.53.0

type SyncPhase string

SyncPhase identifies a stage of a Mithril bootstrap.

const (
	PhaseBootstrap     SyncPhase = "bootstrap"
	PhaseLedgerImport  SyncPhase = "ledger_import"
	PhaseImmutableCopy SyncPhase = "immutable_copy"
	PhaseGapBlocks     SyncPhase = "gap_blocks"
	PhasePostLedger    SyncPhase = "post_ledger_state"
	PhaseBackfill      SyncPhase = "backfill"
	PhaseIndexRebuild  SyncPhase = "index_rebuild"
	PhaseComplete      SyncPhase = "complete"
)

type SyncProgress added in v0.53.0

type SyncProgress struct {
	Phase           SyncPhase
	Active          bool    // true while the phase is active (begin + every mid-phase tick); false only marks the phase end
	BytesDownloaded int64   // download phase
	TotalBytes      int64   // download phase
	BytesPerSecond  float64 // download/copy/backfill rate
	CurrentSlot     uint64
	TipSlot         uint64
	Percent         float64
	Count           int    // generic counter (gap blocks, blocks copied)
	Total           int    // Total complements Count for non-byte progress (e.g. ledger-import items).
	Description     string // free-form (era/stage label)
}

SyncProgress is a flat, dependency-free progress report emitted during Sync. It deliberately exposes no internal/node or ledgerstate types so external embedders can consume it without importing dingo internals.

type SyncProgressFunc added in v0.53.0

type SyncProgressFunc func(SyncProgress)

SyncProgressFunc receives progress updates. It must be fast and non-blocking.

type SyncResult added in v0.53.0

type SyncResult struct {
	Snapshot   *SnapshotListItem
	LedgerSlot uint64
}

SyncResult summarises a completed bootstrap.

func Sync added in v0.53.0

func Sync(
	ctx context.Context,
	cfg SyncConfig,
) (syncResult SyncResult, syncErr error)

Sync performs a full Mithril bootstrap of the database at cfg.DataDir: download + verify + extract a snapshot, import ledger state and immutable blocks, close the volatile gap, backfill metadata, and mark the sync complete. The resulting database is servable by dingo.Node.Run.

type VerificationKey added in v0.26.0

type VerificationKey struct {
	Type        string
	Description string
	CborHex     string
	RawKeyBytes []byte
}

VerificationKey is a parsed Mithril/Cardano verification key file.

func ParseVerificationKey added in v0.26.0

func ParseVerificationKey(data string) (*VerificationKey, error)

ParseVerificationKey parses a verification key from either Cardano text envelope JSON or a raw hex-encoded key string.

func (*VerificationKey) RawKeyBytesHex added in v0.26.0

func (v *VerificationKey) RawKeyBytesHex() string

RawKeyBytesHex returns the raw key bytes as a hex-encoded string.

type VerificationMaterial added in v0.26.0

type VerificationMaterial struct {
	CertificateChain         *CertificateChainVerificationResult
	MithrilCertificate       *Certificate
	CardanoCertificate       *Certificate
	MithrilStakeDistribution *MithrilStakeDistribution
	CardanoStakeDistribution *CardanoStakeDistribution
}

VerificationMaterial bundles the non-cryptographic inputs needed for full Mithril STM certificate verification.

func BuildVerificationMaterial added in v0.26.0

func BuildVerificationMaterial(
	ctx context.Context,
	client *Client,
	verification *CertificateChainVerificationResult,
) (*VerificationMaterial, error)

BuildVerificationMaterial assembles the current verification inputs for a certificate chain. This does not perform aggregate signature verification; it only prepares the inputs required for that future step.

type VerificationMode added in v0.26.0

type VerificationMode uint8

VerificationMode selects the level of Mithril certificate verification.

const (
	// VerificationModeStructural verifies certificate chain linkage.
	VerificationModeStructural VerificationMode = iota + 1
	// VerificationModeSTM verifies the structural certificate chain and the
	// aggregate multi-signature of each non-genesis certificate.
	VerificationModeSTM
)

type VerifiedChunk added in v0.87.0

type VerifiedChunk struct {
	Num    uint64
	Blocks []ledger.Block
	// Raw holds the chunk's blocks undecoded, when the feed was configured
	// with RawBlocks; Blocks is then empty.
	Raw []immutable.Block
}

VerifiedChunk is one immutable file whose trio matched the certified digest list before it was decoded.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL