blob

package
v0.70.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 2, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BackupValidator added in v0.70.1

type BackupValidator interface {
	ValidateBackup(ctx context.Context, r io.Reader) error
}

BackupValidator is implemented by blob store plugins that can validate a backup stream completely without writing to the target store. Live restore uses this before either store is mutated, so a truncated record, a checksum mismatch, or trailing data cannot be discovered only after metadata has already been replaced.

type Backuper added in v0.69.0

type Backuper interface {
	// Backup streams a backup of the current contents of the store to w.
	Backup(ctx context.Context, w io.Writer) error
}

Backuper is implemented by blob store plugins that can stream a consistent point-in-time copy of their contents. Implementations should use their native MVCC/versioned backup mechanism so this does not require blocking concurrent writers for the duration of the backup.

type BlobStore

type BlobStore interface {
	// Transaction management
	Close() error
	NewTransaction(bool) types.Txn

	// KV operations (plugins map these to their internals with transaction context)
	Get(txn types.Txn, key []byte) ([]byte, error)
	Set(txn types.Txn, key, val []byte) error
	Delete(txn types.Txn, key []byte) error
	NewIterator(
		txn types.Txn,
		opts types.BlobIteratorOptions,
	) types.BlobIterator

	// Commit timestamp management
	GetCommitTimestamp() (int64, error)
	// SetCommitTimestamp stores the last commit timestamp; parameter order is
	// (timestamp, txn) to keep the transaction as the final parameter.
	SetCommitTimestamp(int64, types.Txn) error

	// Block operations
	SetBlock(
		txn types.Txn,
		slot uint64,
		hash []byte,
		cbor []byte,
		id uint64,
		blockType uint,
		height uint64,
		prevHash []byte,
	) error
	GetBlock(
		txn types.Txn,
		slot uint64,
		hash []byte,
	) ([]byte, types.BlockMetadata, error)
	DeleteBlock(txn types.Txn, slot uint64, hash []byte, id uint64) error
	// TombstoneBlock replaces the block's CBOR with an expired-history
	// marker while leaving index pointers (bi, bh) and metadata in place.
	// GetBlock and iterator ValueCopy must return types.ErrHistoryExpired
	// for the block so archive wrappers can proxy it if configured.
	// DeleteBlock continues to fully remove a block for orphan rollbacks
	// where there is no retained history entry.
	TombstoneBlock(txn types.Txn, slot uint64, hash []byte) error
	GetBlockURL(
		ctx context.Context,
		txn types.Txn,
		point ocommon.Point,
	) (types.SignedURL, types.BlockMetadata, error)
	// UTxO operations
	SetUtxo(txn types.Txn, txId []byte, outputIdx uint32, cbor []byte) error
	GetUtxo(txn types.Txn, txId []byte, outputIdx uint32) ([]byte, error)
	DeleteUtxo(txn types.Txn, txId []byte, outputIdx uint32) error

	// Transaction operations
	SetTx(txn types.Txn, txHash []byte, offsetData []byte) error
	GetTx(txn types.Txn, txHash []byte) ([]byte, error)
	DeleteTx(txn types.Txn, txHash []byte) error

	// DiskSize returns the on-disk size of the blob store in bytes.
	// Returns 0 for cloud-backed stores where local size is not meaningful.
	DiskSize() (int64, error)

	// Sync flushes everything committed so far to durable storage, so it
	// survives an unclean shutdown of the process or host.
	//
	// This exists because committing a blob transaction is not the same as
	// making it durable. The combined blob+metadata commit in database.Txn
	// deliberately commits blob first so the blob store can only ever be ahead
	// of the metadata tip, never behind -- startup reconciliation knows how to
	// trim a blob store that is ahead (cleanupOrphanedBlobs) but cannot
	// reconstruct blocks the metadata tip already references. That ordering
	// only holds on disk if the blob commit is durable before the metadata
	// commit is, so Txn.Commit calls Sync between the two. Skipping it inverts
	// the invariant on an unclean host shutdown, because the two stores flush on
	// very different schedules: SQLite reaches disk at WAL checkpoints (every
	// 1000 pages by default) while Badger can hold committed writes in a 128MiB
	// memtable for hours at chain tip. The metadata tip then survives while the
	// blocks it references are silently discarded.
	//
	// Implementations whose writes are already durable on commit (remote
	// object stores) may return nil.
	Sync() error
}

BlobStore defines the interface for a blob storage provider. All transactional methods (Get, Set, Delete, NewIterator, SetCommitTimestamp) require a non-nil types.Txn created by NewTransaction(). Passing nil will result in types.ErrNilTxn.

Important: iterators returned by `NewIterator` yield `Item()` values that must only be accessed while the transaction used to create the iterator is still active. Implementations may validate transaction state at access time (for example `ValueCopy` may fail if the transaction has been committed or rolled back). Typical usage iterates and accesses item values within the same transaction scope.

type LocalBlockReader added in v0.70.1

type LocalBlockReader interface {
	GetBlockLocal(
		txn types.Txn,
		slot uint64,
		hash []byte,
	) ([]byte, types.BlockMetadata, error)
}

LocalBlockReader is an optional extension for wrappers that can bypass a remote archive fallback. Database code uses it for bounded local probes where a cache miss must remain a miss.

type ProviderDependencies added in v0.68.0

type ProviderDependencies struct {
	DataDir      string
	RunMode      string
	StorageMode  string
	Logger       *slog.Logger
	PromRegistry prometheus.Registerer
}

ProviderDependencies are shared application settings injected into any storage.blob provider.

type Resettable added in v0.70.1

type Resettable interface {
	Reset(ctx context.Context) error
}

Resettable is implemented by remote blob stores whose configured target is independent of the local data directory passed to the provider. A live restore cannot obtain an empty target for these providers by restoring into a sibling directory, so it takes a rollback backup and then calls Reset before loading the replacement. Callers must not use Reset without first retaining a restorable copy of the current contents.

type Restorer added in v0.69.0

type Restorer interface {
	// Restore replaces the store's contents with the backup read from r.
	Restore(ctx context.Context, r io.Reader) error
}

Restorer is implemented by blob store plugins that can replace their contents from a backup produced by Backuper.Backup. Restore must only be called against a freshly created, empty store — it is not a merge.

Directories

Path Synopsis
internal
blobbackup
Package blobbackup implements the shared backup/restore stream format used by cloud blob store plugins (s3, gcs) that have no native point-in-time snapshot primitive of their own -- a plain length-prefixed key/value stream produced by walking the store's existing Get/Set/NewIterator interface, distinct from badger's own native Backup/Load format.
Package blobbackup implements the shared backup/restore stream format used by cloud blob store plugins (s3, gcs) that have no native point-in-time snapshot primitive of their own -- a plain length-prefixed key/value stream produced by walking the store's existing Get/Set/NewIterator interface, distinct from badger's own native Backup/Load format.
compensate
Package compensate provides a disk-spooled compensation log for cloud blob transactions.
Package compensate provides a disk-spooled compensation log for cloud blob transactions.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL