Documentation
¶
Overview ¶
Package block_store_writeback keeps every block write in a local store and uploads it to a remote store in the background.
A write is acknowledged once the local store holds the block and a durable marker names it. Upload drains the markers to the remote store and clears each marker only after the remote store holds its block, so writes made while the remote is unreachable upload after it returns, across restarts. Backfill queues the blocks stored before a remote was chosen.
Index ¶
- type Mark
- type MarkFunc
- type MarkingStore
- func (m *MarkingStore) BeginDeferFlush()
- func (m *MarkingStore) BeginReadOperation(ctx context.Context) (block.StoreOps, func(), error)
- func (m *MarkingStore) EndDeferFlush(ctx context.Context) error
- func (m *MarkingStore) GetBlock(ctx context.Context, ref *block.BlockRef) ([]byte, bool, error)
- func (m *MarkingStore) GetBlockExists(ctx context.Context, ref *block.BlockRef) (bool, error)
- func (m *MarkingStore) GetBlockExistsBatch(ctx context.Context, refs []*block.BlockRef) ([]bool, error)
- func (m *MarkingStore) GetHashType() hash.HashType
- func (m *MarkingStore) GetSupportedFeatures() block.StoreFeature
- func (m *MarkingStore) PutBlock(ctx context.Context, data []byte, opts *block.PutOpts) (*block.BlockRef, bool, error)
- func (m *MarkingStore) PutBlockBatch(ctx context.Context, entries []*block.PutBatchEntry) error
- func (m *MarkingStore) RmBlock(ctx context.Context, ref *block.BlockRef) error
- func (m *MarkingStore) StatBlock(ctx context.Context, ref *block.BlockRef) (*block.BlockStat, error)
- func (m *MarkingStore) Sync(ctx context.Context) (bool, error)
- type Status
- type Store
- func (s *Store) Backfill(ctx context.Context, target string, ...) error
- func (s *Store) GetStatus() (Status, <-chan struct{})
- func (s *Store) SetEnabled(ctx context.Context, enabled bool) error
- func (s *Store) SetError(err error)
- func (s *Store) Upload(ctx context.Context, remote block.StoreOps) error
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Mark ¶
type Mark struct {
// Hash is the block hash.
Hash *hash.Hash
// Size is the block size in bytes.
Size int64
}
Mark names a written block.
type MarkFunc ¶
MarkFunc durably records written blocks before their writes are acknowledged. A batch write passes all of its blocks in one call.
type MarkingStore ¶
type MarkingStore struct {
// contains filtered or unexported fields
}
MarkingStore acknowledges writes only after mark retains each written block.
A write that fails to mark returns the error with the block stored; repeating the write repairs the marker.
func NewMarkingStore ¶
func NewMarkingStore(store block.StoreOps, mark MarkFunc) *MarkingStore
NewMarkingStore wraps store, calling mark after each successful write.
func (*MarkingStore) BeginDeferFlush ¶
func (m *MarkingStore) BeginDeferFlush()
BeginDeferFlush forwards the GC ref-batch scope to the inner store.
func (*MarkingStore) BeginReadOperation ¶
BeginReadOperation opens a read scope on the inner store.
func (*MarkingStore) EndDeferFlush ¶
func (m *MarkingStore) EndDeferFlush(ctx context.Context) error
EndDeferFlush forwards the GC ref-batch scope to the inner store.
func (*MarkingStore) GetBlockExists ¶
GetBlockExists checks if a block exists.
func (*MarkingStore) GetBlockExistsBatch ¶
func (m *MarkingStore) GetBlockExistsBatch(ctx context.Context, refs []*block.BlockRef) ([]bool, error)
GetBlockExistsBatch forwards batched existence probes to the inner store.
func (*MarkingStore) GetHashType ¶
func (m *MarkingStore) GetHashType() hash.HashType
GetHashType returns the inner store hash type.
func (*MarkingStore) GetSupportedFeatures ¶
func (m *MarkingStore) GetSupportedFeatures() block.StoreFeature
GetSupportedFeatures returns the inner store native feature bitset.
func (*MarkingStore) PutBlock ¶
func (m *MarkingStore) PutBlock(ctx context.Context, data []byte, opts *block.PutOpts) (*block.BlockRef, bool, error)
PutBlock stores a block and marks it, even when it already existed.
func (*MarkingStore) PutBlockBatch ¶
func (m *MarkingStore) PutBlockBatch(ctx context.Context, entries []*block.PutBatchEntry) error
PutBlockBatch marks every successful non-tombstone write in one call. A failed marker returns an error; repeating the batch repairs the markers.
type Status ¶
type Status struct {
// Enabled reports whether writes are queued for upload.
Enabled bool
// Pending is the number of blocks awaiting upload.
Pending int
// PendingBytes is the size of the blocks awaiting upload.
PendingBytes int64
// Err is the last upload failure, cleared by the next successful batch.
Err error
// Target is the remote every block stored before it was chosen is queued
// for, empty until a backfill completes.
Target string
}
Status is the upload state of a Store.
type Store ¶
type Store struct {
*MarkingStore
// contains filtered or unexported fields
}
Store is a local block store whose writes queue for upload.
func NewStore ¶
func NewStore(ctx context.Context, local block.StoreOps, markers kvtx.Store, enabled bool) (*Store, error)
NewStore builds a Store over local, counting the markers left by a previous run.
When enabled is false, writes are not queued until SetEnabled enables them.
func (*Store) Backfill ¶
func (s *Store) Backfill( ctx context.Context, target string, list func(ctx context.Context) ([]*block.BlockRef, error), ) error
Backfill queues every block list names for upload to target, unless a backfill for target already completed. Call it after enabling uploads for target, so the backfill and new writes together cover every block.
A block the local store does not hold is skipped. The completed target is recorded durably, so an interrupted backfill runs again on the next call.
func (*Store) SetEnabled ¶
SetEnabled starts or stops queueing writes for upload.
Disabling drops the queued markers and the backfill target: the local store keeps every block, and the remote store no longer needs them.
Source Files