Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type BackupValidator ¶ added in v0.70.1
BackupValidator is implemented by blob store plugins that can validate a backup stream completely without writing to the target store. Live restore uses this before either store is mutated, so a truncated record, a checksum mismatch, or trailing data cannot be discovered only after metadata has already been replaced.
type Backuper ¶ added in v0.69.0
type Backuper interface {
// Backup streams a backup of the current contents of the store to w.
Backup(ctx context.Context, w io.Writer) error
}
Backuper is implemented by blob store plugins that can stream a consistent point-in-time copy of their contents. Implementations should use their native MVCC/versioned backup mechanism so this does not require blocking concurrent writers for the duration of the backup.
type BlobStore ¶
type BlobStore interface {
// Transaction management
Close() error
NewTransaction(bool) types.Txn
// KV operations (plugins map these to their internals with transaction context)
Get(txn types.Txn, key []byte) ([]byte, error)
Set(txn types.Txn, key, val []byte) error
Delete(txn types.Txn, key []byte) error
NewIterator(
txn types.Txn,
opts types.BlobIteratorOptions,
) types.BlobIterator
// Commit timestamp management
GetCommitTimestamp() (int64, error)
// SetCommitTimestamp stores the last commit timestamp; parameter order is
// (timestamp, txn) to keep the transaction as the final parameter.
SetCommitTimestamp(int64, types.Txn) error
// Block operations
SetBlock(
txn types.Txn,
slot uint64,
hash []byte,
cbor []byte,
id uint64,
blockType uint,
height uint64,
prevHash []byte,
) error
GetBlock(
txn types.Txn,
slot uint64,
hash []byte,
) ([]byte, types.BlockMetadata, error)
DeleteBlock(txn types.Txn, slot uint64, hash []byte, id uint64) error
// TombstoneBlock replaces the block's CBOR with an expired-history
// marker while leaving index pointers (bi, bh) and metadata in place.
// GetBlock and iterator ValueCopy must return types.ErrHistoryExpired
// for the block so archive wrappers can proxy it if configured.
// DeleteBlock continues to fully remove a block for orphan rollbacks
// where there is no retained history entry.
TombstoneBlock(txn types.Txn, slot uint64, hash []byte) error
GetBlockURL(
ctx context.Context,
txn types.Txn,
point ocommon.Point,
) (types.SignedURL, types.BlockMetadata, error)
// UTxO operations
SetUtxo(txn types.Txn, txId []byte, outputIdx uint32, cbor []byte) error
GetUtxo(txn types.Txn, txId []byte, outputIdx uint32) ([]byte, error)
DeleteUtxo(txn types.Txn, txId []byte, outputIdx uint32) error
// Transaction operations
SetTx(txn types.Txn, txHash []byte, offsetData []byte) error
GetTx(txn types.Txn, txHash []byte) ([]byte, error)
DeleteTx(txn types.Txn, txHash []byte) error
// DiskSize returns the on-disk size of the blob store in bytes.
// Returns 0 for cloud-backed stores where local size is not meaningful.
DiskSize() (int64, error)
// Sync flushes everything committed so far to durable storage, so it
// survives an unclean shutdown of the process or host.
//
// This exists because committing a blob transaction is not the same as
// making it durable. The combined blob+metadata commit in database.Txn
// deliberately commits blob first so the blob store can only ever be ahead
// of the metadata tip, never behind -- startup reconciliation knows how to
// trim a blob store that is ahead (cleanupOrphanedBlobs) but cannot
// reconstruct blocks the metadata tip already references. That ordering
// only holds on disk if the blob commit is durable before the metadata
// commit is, so Txn.Commit calls Sync between the two. Skipping it inverts
// the invariant on an unclean host shutdown, because the two stores flush on
// very different schedules: SQLite reaches disk at WAL checkpoints (every
// 1000 pages by default) while Badger can hold committed writes in a 128MiB
// memtable for hours at chain tip. The metadata tip then survives while the
// blocks it references are silently discarded.
//
// Implementations whose writes are already durable on commit (remote
// object stores) may return nil.
Sync() error
}
BlobStore defines the interface for a blob storage provider. All transactional methods (Get, Set, Delete, NewIterator, SetCommitTimestamp) require a non-nil types.Txn created by NewTransaction(). Passing nil will result in types.ErrNilTxn.
Important: iterators returned by `NewIterator` yield `Item()` values that must only be accessed while the transaction used to create the iterator is still active. Implementations may validate transaction state at access time (for example `ValueCopy` may fail if the transaction has been committed or rolled back). Typical usage iterates and accesses item values within the same transaction scope.
type LocalBlockReader ¶ added in v0.70.1
type LocalBlockReader interface {
GetBlockLocal(
txn types.Txn,
slot uint64,
hash []byte,
) ([]byte, types.BlockMetadata, error)
}
LocalBlockReader is an optional extension for wrappers that can bypass a remote archive fallback. Database code uses it for bounded local probes where a cache miss must remain a miss.
type ProviderDependencies ¶ added in v0.68.0
type ProviderDependencies struct {
DataDir string
RunMode string
StorageMode string
Logger *slog.Logger
PromRegistry prometheus.Registerer
}
ProviderDependencies are shared application settings injected into any storage.blob provider.
type Resettable ¶ added in v0.70.1
Resettable is implemented by remote blob stores whose configured target is independent of the local data directory passed to the provider. A live restore cannot obtain an empty target for these providers by restoring into a sibling directory, so it takes a rollback backup and then calls Reset before loading the replacement. Callers must not use Reset without first retaining a restorable copy of the current contents.
type Restorer ¶ added in v0.69.0
type Restorer interface {
// Restore replaces the store's contents with the backup read from r.
Restore(ctx context.Context, r io.Reader) error
}
Restorer is implemented by blob store plugins that can replace their contents from a backup produced by Backuper.Backup. Restore must only be called against a freshly created, empty store — it is not a merge.
Directories
¶
| Path | Synopsis |
|---|---|
|
internal
|
|
|
blobbackup
Package blobbackup implements the shared backup/restore stream format used by cloud blob store plugins (s3, gcs) that have no native point-in-time snapshot primitive of their own -- a plain length-prefixed key/value stream produced by walking the store's existing Get/Set/NewIterator interface, distinct from badger's own native Backup/Load format.
|
Package blobbackup implements the shared backup/restore stream format used by cloud blob store plugins (s3, gcs) that have no native point-in-time snapshot primitive of their own -- a plain length-prefixed key/value stream produced by walking the store's existing Get/Set/NewIterator interface, distinct from badger's own native Backup/Load format. |
|
committimestamp
Package committimestamp decodes a blob-stored commit timestamp shared by every blob backend (badger, S3, GCS), rejecting a value that does not actually fit the int64 the rest of the codebase carries it as, rather than each backend separately risking silent truncation or wraparound.
|
Package committimestamp decodes a blob-stored commit timestamp shared by every blob backend (badger, S3, GCS), rejecting a value that does not actually fit the int64 the rest of the codebase carries it as, rather than each backend separately risking silent truncation or wraparound. |
|
compensate
Package compensate provides a disk-spooled compensation log for cloud blob transactions.
|
Package compensate provides a disk-spooled compensation log for cloud blob transactions. |