block

package
v0.51.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 1, 2026 License: Apache-2.0 Imports: 29 Imported by: 0

Documentation

Overview

Package block defines a common pattern for interacting with block reference structures in Hydra and in memory.

Index

Constants

View Source 
const (
	// StoreFeatureNativeBatchPut means PutBlockBatch is implemented natively.
	StoreFeatureNativeBatchPut = StoreFeature_STORE_FEATURE_NATIVE_BATCH_PUT
	// StoreFeatureNativeBatchExists means GetBlockExistsBatch is implemented natively.
	StoreFeatureNativeBatchExists = StoreFeature_STORE_FEATURE_NATIVE_BATCH_EXISTS
	// StoreFeatureNativeBackgroundPut means PutBlockBackground actually deprioritizes writes.
	StoreFeatureNativeBackgroundPut = StoreFeature_STORE_FEATURE_NATIVE_BACKGROUND_PUT
	// StoreFeatureNativeFlush means Flush has buffered work to publish.
	StoreFeatureNativeFlush = StoreFeature_STORE_FEATURE_NATIVE_FLUSH
	// StoreFeatureNativeDeferFlush means BeginDeferFlush and EndDeferFlush batch flush work.
	StoreFeatureNativeDeferFlush = StoreFeature_STORE_FEATURE_NATIVE_DEFER_FLUSH
)
View Source 
const DefaultHashType = hash.HashType_HashType_BLAKE3

DefaultHashType is the default hash type for refs.

View Source 
const MaxBlockSize = 10485760

MaxBlockSize is the default maximum size in bytes (10MB) for a single serialized block accepted on the wire by transports such as DEX.

This is a sanity / DoS ceiling, not a structural limit on any one block type. Individual block types are naturally bounded well below this value:

  • blob.Blob (root): inline RawData is capped by blob.DefRawHighWaterMark = blob.DefChunkingMaxSize = 786432 (768 KiB). Above that the Blob auto-converts to CHUNKED, where the root only stores a ChunkIndex of references and individual chunk data lives in separate byteslice blocks.
  • blob chunk data (byteslice.ByteSlice): one chunk per block, capped by blob.DefChunkingMaxSize = 786432 (768 KiB).
  • blob.ChunkIndex / IAVL node / other index blocks: hold only references and small metadata, well under 1 MiB in practice.

10 MiB therefore leaves roughly an order of magnitude of headroom over the largest block any current producer will emit, while still bounding the per-message buffer a peer must allocate when receiving a single block.

Variables

View Source 
var (
	StoreFeature_name = map[int32]string{
		0:  "STORE_FEATURE_UNKNOWN",
		1:  "STORE_FEATURE_NATIVE_BATCH_PUT",
		2:  "STORE_FEATURE_NATIVE_BATCH_EXISTS",
		4:  "STORE_FEATURE_NATIVE_BACKGROUND_PUT",
		8:  "STORE_FEATURE_NATIVE_FLUSH",
		16: "STORE_FEATURE_NATIVE_DEFER_FLUSH",
	}
	StoreFeature_value = map[string]int32{
		"STORE_FEATURE_UNKNOWN":               0,
		"STORE_FEATURE_NATIVE_BATCH_PUT":      1,
		"STORE_FEATURE_NATIVE_BATCH_EXISTS":   2,
		"STORE_FEATURE_NATIVE_BACKGROUND_PUT": 4,
		"STORE_FEATURE_NATIVE_FLUSH":          8,
		"STORE_FEATURE_NATIVE_DEFER_FLUSH":    16,
	}
)

Enum value maps for StoreFeature.

View Source 
var (
	OverlayMode_name = map[int32]string{
		0: "UPPER_ONLY",
		1: "LOWER_ONLY",
		2: "UPPER_CACHE",
		3: "LOWER_CACHE",
		4: "UPPER_READ_CACHE",
		5: "LOWER_READ_CACHE",
		6: "UPPER_WRITE_CACHE",
		7: "LOWER_WRITE_CACHE",
		8: "UPPER_READBACK_CACHE",
	}
	OverlayMode_value = map[string]int32{
		"UPPER_ONLY":           0,
		"LOWER_ONLY":           1,
		"UPPER_CACHE":          2,
		"LOWER_CACHE":          3,
		"UPPER_READ_CACHE":     4,
		"LOWER_READ_CACHE":     5,
		"UPPER_WRITE_CACHE":    6,
		"LOWER_WRITE_CACHE":    7,
		"UPPER_READBACK_CACHE": 8,
	}
)

Enum value maps for OverlayMode.

View Source 
var (
	// ErrBlockStoreUnavailable is returned when Fetch is called against a nil block store.
	ErrBlockStoreUnavailable = errors.New("block store is unavailable")
	// ErrUnexpectedType is returned if a type assertion failed.
	ErrUnexpectedType = errors.New("block: unexpected object type")
	// ErrNilCursor is returned when a non-nil block cursor is required.
	ErrNilCursor = errors.New("block cursor cannot be nil")
	// ErrNilBlock is returned when a non-nil block is required.
	ErrNilBlock = errors.New("block cannot be nil")
	// ErrEmptyBlock is returned when a non-empty block is required.
	ErrEmptyBlock = errors.New("block data cannot be nil")
	// ErrEmptyBlockRef is returned a ref was required but was empty.
	ErrEmptyBlockRef = errors.New("empty block reference")
	// ErrNotBlock is returned if the object did not implement Block.
	ErrNotBlock = errors.New("object must be a block")
	// ErrNotSubBlock is returned if the block did not implement SubBlock.
	ErrNotSubBlock = errors.New("block must be a sub-block")
	// ErrNotBlockWithSubBlocks is returned if the block did not implement BlockWithSubBlocks.
	ErrNotBlockWithSubBlocks = errors.New("block must implement block with sub-blocks")
	// ErrEmptyChanges is returned if a slice of changes was unexpectedly empty.
	ErrEmptyChanges = errors.New("changes set cannot be empty")
	// ErrNotFound is returned when a block was not found but was required.
	ErrNotFound = errors.New("block not found")
	// ErrNotClonable is returned if a block could not be cloned.
	ErrNotClonable = errors.New("block: unable to clone")
	// ErrBlockRefMismatch is returned if the data does not match the expected ref.
	ErrBlockRefMismatch = errors.New("block: block ref hash mismatch")
	// ErrBufferedStoreFull is returned when a buffered store reaches its memory limits.
	ErrBufferedStoreFull = errors.New("block: buffered store is full")
)

Functions

func ApplySubBlock 

func ApplySubBlock[T SubBlock](r *T, next SubBlock) error

ApplySubBlock applies a sub-block to a field.

func CloneBlock 

func CloneBlock(blk any) (any, error)

CloneBlock tries to clone an input block.

The block should implement proto.Message or BlockWithClone.

returns ErrUnexpectedType or ErrNotClonable if the block was not clonable.

func CompareNamedSubBlocks 

func CompareNamedSubBlocks[T ComparableNamedSubBlock](a, b []T) (added, removed, changed []T)

CompareNamedSubBlocks compares two sets of ComparableNamedSubBlock. Returns the added, removed, and changed values.

func IsNamedSubBlocksSorted 

func IsNamedSubBlocksSorted[T NamedSubBlock](namedSubBlocks []T) bool

IsNamedSubBlocksSorted checks if the set of NamedSubBlock is sorted.

func MarshalBlockRefJSON 

func MarshalBlockRefJSON(ref *BlockRef) ([]byte, error)

MarshalBlockRefJSON marshals an BlockRef to JSON.

Returns "null" if the ref is nil.

func NewTransaction 

func NewTransaction(

	store StoreOps,

	transformer Transformer,

	rootRef *BlockRef,

	putOpts *PutOpts,
) (*Transaction, *Cursor)

NewTransaction builds a new transaction with a root cursor.

func SortNamedSubBlocks 

func SortNamedSubBlocks[T NamedSubBlock](namedSubBlocks []T)

SortNamedSubBlocks sorts a set of NamedSubBlock.

func UnmarshalBlock 

func UnmarshalBlock[T Block](ctx context.Context, bcs *Cursor, ctor func() Block) (T, error)

UnmarshalBlock unmarshals the block from the cursor & type-asserts it. Returns ErrUnexpectedType if the type returned was not T. Incorrect type happens if the cursor already contains a block w/ different type. If bcs == nil, returns empty, nil. If unmarshal() returns nil, returns empty, nil.

Types

type Block 

type Block interface {
	// MarshalBlock marshals the block to binary.
	// This is the initial step of marshaling, before transformations.
	MarshalBlock() ([]byte, error)
	// UnmarshalBlock unmarshals the block to the object.
	// This is the final step of decoding, after transformations.
	UnmarshalBlock(data []byte) error
}

Block defines an in-memory decoded block structure. A block should contain a minimal amount of data with some pointers to other blocks.

func CastToBlock 

func CastToBlock(sb any) (Block, error)

CastToBlock casts a object to a block or returns an error.

func NewBlockRefBlock 

func NewBlockRefBlock() Block

NewBlockRefBlock constructs a new block reference block.

type BlockRef 

type BlockRef struct {

	// Hash is the hash of the object.
	Hash *hash.Hash `protobuf:"bytes,1,opt,name=hash,proto3" json:"hash,omitempty"`
	// contains filtered or unexported fields
}

BlockRef is a block content ID reference.

protobuf-go-lite:disable-json

func BuildBlockRef 

func BuildBlockRef(data []byte, putOpts *PutOpts) (*BlockRef, error)

BuildBlockRef builds a block ref from put opts by hashing the data. If putOpts are nil, uses default hash type.

func CloneBlockRefs 

func CloneBlockRefs(refs []*BlockRef) []*BlockRef

CloneBlockRefs returns a deep copy of refs, preserving nil entries. Returns nil for empty input.

func ExtractBlockRefs 

func ExtractBlockRefs(blk any) ([]*BlockRef, error)

ExtractBlockRefs extracts all outgoing BlockRefs from a block by walking BlockWithRefs and recursing into BlockWithSubBlocks.

This captures refs at all nesting levels: direct refs on the block, refs inside sub-blocks (e.g., BlockRefSlice), and refs inside nested sub-blocks.

func NewBlockRef 

func NewBlockRef(hash *hash.Hash) *BlockRef

NewBlockRef constructs a new block reference.

func PutBlock 

func PutBlock(ctx context.Context, bk StoreOps, b Block) (*BlockRef, bool, error)

PutBlock marshals & puts a block into a bucket.

func UnmarshalBlockRefB58 

func UnmarshalBlockRefB58(ref string) (*BlockRef, error)

UnmarshalBlockRefB58 unmarshals a b58 string block ref.

func UnmarshalBlockRefBlock 

func UnmarshalBlockRefBlock(ctx context.Context, bcs *Cursor) (*BlockRef, error)

UnmarshalBlockRefBlock unmarshals a BlockRef from a block cursor.

func UnmarshalBlockRefJSON 

func UnmarshalBlockRefJSON(data []byte) (*BlockRef, error)

UnmarshalBlockRefJSON attempts to unmarshal an BlockRef from JSON.

func (*BlockRef) ApplyBlockRef 

func (b *BlockRef) ApplyBlockRef(id uint32, ptr *BlockRef) error

ApplyBlockRef applies a ref change with a field id. The reference may be nil if the child block is nil.

func (*BlockRef) Clone 

func (b *BlockRef) Clone() *BlockRef

Clone clones the block ref.

func (*BlockRef) CloneMessageVT 

func (m *BlockRef) CloneMessageVT() protobuf_go_lite.CloneMessage

func (*BlockRef) CloneVT 

func (m *BlockRef) CloneVT() *BlockRef

func (*BlockRef) EqualMessageVT 

func (this *BlockRef) EqualMessageVT(thatMsg any) bool

func (*BlockRef) EqualVT 

func (this *BlockRef) EqualVT(that *BlockRef) bool

func (*BlockRef) EqualsRef 

func (b *BlockRef) EqualsRef(oref *BlockRef) bool

EqualsRef checks if two refs are equal.

func (*BlockRef) GetBlockRefCtor 

func (b *BlockRef) GetBlockRefCtor(id uint32) Ctor

GetBlockRefCtor returns the constructor for the block at the ref id. Return nil to indicate invalid ref ID or unknown.

func (*BlockRef) GetBlockRefs 

func (b *BlockRef) GetBlockRefs() (map[uint32]*BlockRef, error)

GetBlockRefs returns all block references by ID. May return nil, and values may also be nil. Note: this does not include pending references (in a cursor)

func (*BlockRef) GetEmpty 

func (b *BlockRef) GetEmpty() bool

GetEmpty returns if the ref is empty.

func (*BlockRef) GetHash 

func (x *BlockRef) GetHash() *hash.Hash

func (*BlockRef) LessThan 

func (b *BlockRef) LessThan(other *BlockRef) bool

LessThan checks if the ref is less than another. 1. Empty is sorted to the end. 2. Hash type is sorted. 3. Hash itself is sorted in bytes ordering

func (*BlockRef) MarshalBlock 

func (b *BlockRef) MarshalBlock() ([]byte, error)

MarshalBlock marshals the block to binary. This is the initial step of marshaling, before transformations.

func (*BlockRef) MarshalJSON 

func (b *BlockRef) MarshalJSON() ([]byte, error)

MarshalJSON marshals the BlockRef to JSON.

func (*BlockRef) MarshalKey 

func (b *BlockRef) MarshalKey() ([]byte, error)

MarshalKey marshals the block ref for use as a key. The format should be reproducible and identical between versions.

func (*BlockRef) MarshalLog 

func (b *BlockRef) MarshalLog() string

MarshalLog marshals the reference for logging. If nil or empty returns <nil>

func (*BlockRef) MarshalProtoJSON 

func (b *BlockRef) MarshalProtoJSON(s *json.MarshalState)

MarshalProtoJSON marshals the BlockRef message to JSON.

func (*BlockRef) MarshalProtoText 

func (x *BlockRef) MarshalProtoText() string

func (*BlockRef) MarshalString 

func (b *BlockRef) MarshalString() string

MarshalString marshals the reference to a string form.

func (*BlockRef) MarshalToSizedBufferVT 

func (m *BlockRef) MarshalToSizedBufferVT(dAtA []byte) (int, error)

func (*BlockRef) MarshalToVT 

func (m *BlockRef) MarshalToVT(dAtA []byte) (int, error)

func (*BlockRef) MarshalVT 

func (m *BlockRef) MarshalVT() (dAtA []byte, err error)

func (*BlockRef) ParseFromB58 

func (b *BlockRef) ParseFromB58(ref string) error

ParseFromB58 parses the object ref from a base58 string.

func (*BlockRef) ProtoMessage 

func (*BlockRef) ProtoMessage()

func (*BlockRef) Reset 

func (x *BlockRef) Reset()

func (*BlockRef) SizeVT 

func (m *BlockRef) SizeVT() (n int)

func (*BlockRef) String 

func (x *BlockRef) String() string

func (*BlockRef) UnmarshalBlock 

func (b *BlockRef) UnmarshalBlock(data []byte) error

UnmarshalBlock unmarshals the block to the object. This is the final step of decoding, after transformations.

func (*BlockRef) UnmarshalJSON 

func (b *BlockRef) UnmarshalJSON(dat []byte) error

UnmarshalJSON unmarshals the BlockRef from JSON.

func (*BlockRef) UnmarshalProtoJSON 

func (b *BlockRef) UnmarshalProtoJSON(s *json.UnmarshalState)

UnmarshalProtoJSON unmarshals the BlockRef message from JSON.

func (*BlockRef) UnmarshalVT 

func (m *BlockRef) UnmarshalVT(dAtA []byte) error

func (*BlockRef) Validate 

func (b *BlockRef) Validate(allowEmpty bool) error

Validate validates the block ref.

func (*BlockRef) VerifyData 

func (b *BlockRef) VerifyData(data []byte, errDetails bool) error

VerifyData checks the given data matches the block ref. If errDetails is set, wraps the error with the unexpected and expected refs if mismatch.

type BlockStat 

type BlockStat struct {
	// Ref is the block reference.
	Ref *BlockRef
	// Size is the block data size in bytes.
	// May be -1 if the size is unknown.
	Size int64
}

BlockStat contains metadata about a stored block.

type BlockWithAttributes 

type BlockWithAttributes interface {
	// GetBlockGraphAttributes returns the block graph attributes.
	GetBlockGraphAttributes() []encoding.Attribute
}

BlockWithAttributes returns a block with graph attributes.

type BlockWithClone 

type BlockWithClone interface {
	CloneBlock() (Block, error)
}

BlockWithClone defines a block with a clone function. The clone should share nothing with the original.

type BlockWithCloneVT 

type BlockWithCloneVT[T Block] interface {
	// CloneVT clones the block object with VTprotobuf.
	CloneVT() T
}

BlockWithCloneVT defines a block with a VTProtobuf clone function.

type BlockWithPreWriteHook 

type BlockWithPreWriteHook interface {
	// BlockPreWriteHook is called when writing the block.
	BlockPreWriteHook() error
}

BlockWithPreWriteHook is a block with a function called when writing. This can also be applied to a sub-block.

type BlockWithRefs 

type BlockWithRefs interface {
	// ApplyBlockRef applies a ref change with a field id.
	// The reference may be nil if the child block is nil.
	ApplyBlockRef(id uint32, ptr *BlockRef) error
	// GetBlockRefs returns all block references by ID.
	// May return nil, and values may also be nil.
	// Note: this does not include pending references (in a cursor)
	GetBlockRefs() (map[uint32]*BlockRef, error)
	// GetBlockRefCtor returns the constructor for the block at the ref id.
	// Return nil to indicate invalid ref ID or unknown.
	GetBlockRefCtor(id uint32) Ctor
}

BlockWithRefs has references keyed by ID. Each field can contain a reference.

type BlockWithSubBlocks 

type BlockWithSubBlocks interface {
	// ApplySubBlock applies a sub-block change with a field id.
	ApplySubBlock(id uint32, next SubBlock) error
	// GetSubBlocks returns all constructed sub-blocks by ID.
	// May return nil, and values may also be nil.
	GetSubBlocks() map[uint32]SubBlock
	// GetSubBlockCtor returns a function which creates or returns the existing
	// sub-block at reference id. Can return nil to indicate invalid reference id.
	GetSubBlockCtor(id uint32) SubBlockCtor
}

BlockWithSubBlocks is a block containing sub-blocks as fields.

type BufferedStore 

type BufferedStore struct {
	// contains filtered or unexported fields
}

BufferedStore buffers PutBlock calls in memory, drains them in the background, and exposes Flush as a durability fence.

func NewBufferedStore 

func NewBufferedStore(ctx context.Context, inner StoreOps) *BufferedStore

NewBufferedStore constructs a buffered store around an inner store and uses ctx for background draining.

func NewBufferedStoreWithSettings 

func NewBufferedStoreWithSettings(
	ctx context.Context,
	inner StoreOps,
	settings *BufferedStoreSettings,
) *BufferedStore

NewBufferedStoreWithSettings constructs a buffered store with explicit settings.

func (*BufferedStore) BeginDeferFlush 

func (s *BufferedStore) BeginDeferFlush()

BeginDeferFlush opens a nested deferred flush scope.

func (*BufferedStore) EndDeferFlush 

func (s *BufferedStore) EndDeferFlush(ctx context.Context) error

EndDeferFlush closes a deferred flush scope and flushes at the outermost end.

func (*BufferedStore) Flush 

func (s *BufferedStore) Flush(ctx context.Context) error

Flush waits for background block draining through the current fence.

func (*BufferedStore) GetBlock 

func (s *BufferedStore) GetBlock(ctx context.Context, ref *BlockRef) ([]byte, bool, error)

GetBlock gets a block by reference.

func (*BufferedStore) GetBlockExists 

func (s *BufferedStore) GetBlockExists(ctx context.Context, ref *BlockRef) (bool, error)

GetBlockExists checks if a block exists.

func (*BufferedStore) GetBlockExistsBatch 

func (s *BufferedStore) GetBlockExistsBatch(ctx context.Context, refs []*BlockRef) ([]bool, error)

GetBlockExistsBatch checks if blocks exist.

func (*BufferedStore) GetHashType 

func (s *BufferedStore) GetHashType() hash.HashType

GetHashType returns the preferred hash type.

func (*BufferedStore) GetSupportedFeatures 

func (s *BufferedStore) GetSupportedFeatures() StoreFeature

GetSupportedFeatures returns the native feature bitmask for the store.

func (*BufferedStore) PutBlock 

func (s *BufferedStore) PutBlock(ctx context.Context, data []byte, opts *PutOpts) (*BlockRef, bool, error)

PutBlock buffers a block in memory and starts background draining if needed.

func (*BufferedStore) PutBlockBackground 

func (s *BufferedStore) PutBlockBackground(ctx context.Context, data []byte, opts *PutOpts) (*BlockRef, bool, error)

PutBlockBackground forwards to PutBlock because buffered writes already drain asynchronously.

func (*BufferedStore) PutBlockBatch 

func (s *BufferedStore) PutBlockBatch(ctx context.Context, entries []*PutBatchEntry) error

PutBlockBatch loops through PutBlock and RmBlock using the buffered store.

func (*BufferedStore) RmBlock 

func (s *BufferedStore) RmBlock(ctx context.Context, ref *BlockRef) error

RmBlock deletes a block by reference.

func (*BufferedStore) StatBlock 

func (s *BufferedStore) StatBlock(ctx context.Context, ref *BlockRef) (*BlockStat, error)

StatBlock returns metadata about a block without reading its data.

type BufferedStoreSettings 

type BufferedStoreSettings struct {
	MaxPendingEntries int
	MaxPendingBytes   int
	DrainBatchEntries int
}

BufferedStoreSettings configures buffered block writeback behavior.

func DefaultBufferedStoreSettings 

func DefaultBufferedStoreSettings() *BufferedStoreSettings

DefaultBufferedStoreSettings returns the default buffered store settings.

type ComparableNamedSubBlock 

type ComparableNamedSubBlock interface {
	NamedSubBlock

	// Equals compares the block to the other block for equality.
	Equals(ot ComparableNamedSubBlock) bool
}

ComparableNamedSubBlock is a NamedSubBlock that has an Equals function.

type ComparableSubBlock 

type ComparableSubBlock interface {
	comparable
	SubBlock
}

ComparableSubBlock is the type constraint for SubBlock.

type Ctor 

type Ctor func() Block

Ctor is a block constructor.

type Cursor 

type Cursor struct {
	// contains filtered or unexported fields
}

Cursor tracks traversal of a block reference DAG structure with an associated Transaction. Manages interacting with block handles, the transaction cache, the decoder and marshaller, and the transformers.

func (*Cursor) ClearAllRefs 

func (c *Cursor) ClearAllRefs()

ClearAllRefs clears all references. Cursors that were generated by Follow() will be detached.

Note: does not clear sub-blocks from the parent object. Note; does not clear the BlockRef from the parent object.

func (*Cursor) ClearRef 

func (c *Cursor) ClearRef(refID uint32)

ClearRef removes the reference handle to the given ref ID. Noop if FollowRef has not been previously called with refID. Cursors that were generated by Follow() will be detached.

Note: does not clear sub-blocks from the parent object. Note; does not clear the BlockRef from the parent object.

func (*Cursor) CloneBlock 

func (c *Cursor) CloneBlock() (any, error)

CloneBlock tries to clone the contained block.

returns ErrUnexpectedType or ErrNotClonable if the block was not clonable.

func (*Cursor) CopyToRecursive 

func (c *Cursor) CopyToRecursive(targetPos *Cursor, cloneBlocks, markDirty bool)

CopyToRecursive copies the cursor and referenced positions to another cursor. The cursor can use a different block transaction. Note: the same block refs will be reused (underlying data is not copied). If target tx == nil: is equivalent to DetachRecursive(false, cloneBlocks) If markDirty is set, marks all target positions as dirty.

func (*Cursor) Detach 

func (c *Cursor) Detach(keepRefs bool) *Cursor

Detach clones the cursor position.

If keepRefs is set, adds the new location as a parent of all previously referenced blocks.

Note: does not copy/clone the Block object.

func (*Cursor) DetachRecursive 

func (c *Cursor) DetachRecursive(detachTx, cloneBlocks, markDirty bool) *Cursor

DetachRecursive clones the cursor position and all referenced positions.

Note: if !cloneBlocks, does not copy/clone the Block objects.

func (*Cursor) DetachTransaction 

func (c *Cursor) DetachTransaction() *Cursor

DetachTransaction creates a new ephemeral transaction rooted at the cursor.

func (*Cursor) Fetch 

func (c *Cursor) Fetch(ctx context.Context) ([]byte, bool, error)

Fetch fetches the block data into memory. Fetching is performed using a block lookup. Returns the transformed decoded version of the data. Returns data, dataIsSet, err. Returns nil, false, ErrBlockStoreUnavailable if the block store is unset. Returns nil, false, nil if the reference is empty. Returns nil, false, ErrNotFound if not found (block unavailable).

func (*Cursor) FollowRef 

func (c *Cursor) FollowRef(
	refID uint32,
	blkRef *BlockRef,
) *Cursor

FollowRef follows a block reference, returning a cursor pointing to the next block and enqueuing the block for fetching. Does not wait for the block to be fetched to return. If the reference is empty, will create a new block.

func (*Cursor) FollowSubBlock 

func (c *Cursor) FollowSubBlock(refID uint32) *Cursor

FollowSubBlock follows a sub-block reference, returning a cursor pointing to the same block but at a sub-block inside a field. The block is constructed or retrieved using the BlockWithSubBlocks interface.

Once FollowSubBlock has been called, the field will be overwritten if dirty. If ClearRef is called on the parent then this relation is removed.

Note: there may already be a reference with the same ID, which would be returned. The cursor must have the block decoded or set with SetBlock. The cursor block blk must be a BlockWithSubBlocks. If these conditions are not met, returns nil

func (*Cursor) GetAllRefs 

func (c *Cursor) GetAllRefs(existingOnly bool) (map[uint32]*Cursor, error)

GetBlockRefs returns cursors to all references.

If existingOnly, only returns references that have already been traversed. If !existingOnly uses GetSubBlocks and/or GetBlockRefs to list all references. If the position blk is empty, returns an empty map.

func (*Cursor) GetBlock 

func (c *Cursor) GetBlock() (any, bool)

GetBlock returns the current loaded block at the position. May be nil if Fetch or Unmarshal or SetBlock have not been called. Returns isSubBlock.

func (*Cursor) GetBlockStore 

func (c *Cursor) GetBlockStore() (StoreOps, bool)

GetBlockStore returns the block store used for the transaction.

func (*Cursor) GetExistingRef 

func (c *Cursor) GetExistingRef(refID uint32) *Cursor

GetExistingRef checks if the reference has been traversed already. Returns nil if no handle exists for that ref.

func (*Cursor) GetRef 

func (c *Cursor) GetRef() *BlockRef

GetRef returns the current cursor reference.

func (*Cursor) GetTransaction 

func (c *Cursor) GetTransaction() *Transaction

GetTransaction returns the cursor's associated transaction, may be nil.

func (*Cursor) IsDirty 

func (c *Cursor) IsDirty() bool

IsDirty indicates if the position or any sub-positions were changed.

func (*Cursor) IsSubBlock 

func (c *Cursor) IsSubBlock() bool

IsSubBlock indicates if the cursor is currently at a sub-block position.

func (*Cursor) MarkDirty 

func (c *Cursor) MarkDirty()

MarkDirty marks the cursor location dirty, so that it will be re-written.

Note: if cursor is ephemeral (no transaction) this is no-op.

func (*Cursor) Parents 

func (c *Cursor) Parents() []*Cursor

Parents returns new cursors pointing to the parent blocks. Note: the parent list is completely dependent on the order the graph was traversed. Note: returns nil if the cursor is ephemeral (with Detach call).

func (*Cursor) SetAsSubBlock 

func (c *Cursor) SetAsSubBlock(refID uint32, parent *Cursor) error

SetAsSubBlock sets the cursor position as a sub-block of another block.

Clears any existing parent references. Immediately calls ApplySubBlock on the parent block.

May return ErrNotSubBlock or ErrUnexpectedType if the parent is not a block with sub-blocks.

func (*Cursor) SetBlock 

func (c *Cursor) SetBlock(b any, dirty bool)

SetBlock sets a block at the location, and marks the block as dirty. If the location is a Block, b should implement Block interface. If it is a SubBlock, b should implement the SubBlock interface. If dirty is set, sets the block as dirty.

Clears BlockPreWrite.

func (*Cursor) SetBlockStore 

func (c *Cursor) SetBlockStore(store StoreOps)

SetBlockStore sets the store to read from for this cursor and all sub-cursors. If nil, will use the default bucket attached to the block transaction.

func (*Cursor) SetPreWriteHook 

func (c *Cursor) SetPreWriteHook(h func(b any) error)

SetPreWriteHook sets a hook for final transforms to the block.

Note: this should not call any cursor functions that will be locked during the Write process.

Also valid for sub-blocks.

func (*Cursor) SetRef 

func (c *Cursor) SetRef(refID uint32, cursor *Cursor)

SetRef sets a block reference to the handle at the cursor. Adds c to the list of parents for cursor. The cursors must be from the same transaction. Note: this should only be used if refID and cursor are not sub-blocks.

func (*Cursor) SetRefAtCursor 

func (c *Cursor) SetRefAtCursor(ref *BlockRef, clearBlock bool)

SetRefAtCursor sets the reference at the cursor location. If ref is not equal to the existing ref, and clearBlock is set, blk is set to nil.

func (*Cursor) Unmarshal 

func (c *Cursor) Unmarshal(ctx context.Context, ctor func() Block) (Block, error)

Unmarshal fetches and unmarshals the data to a block. If already unmarshaled, returns existing data. If a sub-block, the sub-block must implement Block. If a sub-block, will return the sub-block value or nil. ctor is ignored if the cursor is a sub-block. Returns nil, nil if ctor is nil and the block is nil. Returns nil, nil if the cursor is nil. Returns value from ctor() without calling Unmarshal if empty. Returns nil, block.ErrNotFound if not found.

type NamedSubBlock 

type NamedSubBlock interface {
	// SubBlock indicates this is a sub-block.
	SubBlock
	// GetName returns the name of the block.
	GetName() string
}

NamedSubBlock is a sub-block with a name attached.

type NopStoreOps 

type NopStoreOps struct{}

NopStoreOps implements StoreOps defaults for tests.

Production wrappers must not embed this type. Test mocks may embed it and override only the methods exercised by the test.

func (NopStoreOps) BeginDeferFlush 

func (NopStoreOps) BeginDeferFlush()

BeginDeferFlush opens a no-op defer-flush scope.

func (NopStoreOps) EndDeferFlush 

func (NopStoreOps) EndDeferFlush(context.Context) error

EndDeferFlush closes a no-op defer-flush scope.

func (NopStoreOps) Flush 

func (NopStoreOps) Flush(context.Context) error

Flush returns nil.

func (NopStoreOps) GetBlock 

func (NopStoreOps) GetBlock(context.Context, *BlockRef) ([]byte, bool, error)

GetBlock returns a missing block.

func (NopStoreOps) GetBlockExists 

func (NopStoreOps) GetBlockExists(context.Context, *BlockRef) (bool, error)

GetBlockExists returns false.

func (NopStoreOps) GetBlockExistsBatch 

func (n NopStoreOps) GetBlockExistsBatch(ctx context.Context, refs []*BlockRef) ([]bool, error)

GetBlockExistsBatch loops calling GetBlockExists per ref.

func (NopStoreOps) GetHashType 

func (NopStoreOps) GetHashType() hash.HashType

GetHashType returns the preferred hash type for the store.

func (NopStoreOps) GetSupportedFeatures 

func (NopStoreOps) GetSupportedFeatures() StoreFeature

GetSupportedFeatures returns the native feature bitmask for the store.

func (NopStoreOps) PutBlock 

func (NopStoreOps) PutBlock(context.Context, []byte, *PutOpts) (*BlockRef, bool, error)

PutBlock returns ErrBlockStoreUnavailable.

func (NopStoreOps) PutBlockBackground 

func (n NopStoreOps) PutBlockBackground(ctx context.Context, data []byte, opts *PutOpts) (*BlockRef, bool, error)

PutBlockBackground forwards to PutBlock.

func (NopStoreOps) PutBlockBatch 

func (n NopStoreOps) PutBlockBatch(ctx context.Context, entries []*PutBatchEntry) error

PutBlockBatch loops calling PutBlock or RmBlock per entry.

func (NopStoreOps) RmBlock 

func (NopStoreOps) RmBlock(context.Context, *BlockRef) error

RmBlock returns nil.

func (NopStoreOps) StatBlock 

func (NopStoreOps) StatBlock(context.Context, *BlockRef) (*BlockStat, error)

StatBlock returns nil.

type OverlayMode 

type OverlayMode int32

OverlayMode controls the mode for the block store overlay. 

const (
	// UPPER_ONLY uses the upper store only for reads and writes.
	// reads go to the upper store only.
	// writes go to the upper store only.
	// removes go to the upper store only.
	// the lower store is not used.
	OverlayMode_UPPER_ONLY OverlayMode = 0
	// LOWER_ONLY uses the lower store only for reads and writes.
	// reads go to the lower store only.
	// writes go to the lower store only.
	// removes go to the lower store only.
	// the upper store is not used.
	OverlayMode_LOWER_ONLY OverlayMode = 1
	// UPPER_CACHE uses the upper store as a cache for the lower store.
	// reads go to the upper store first, then the lower store.
	// writes go to both stores.
	// removes go to both stores.
	// reads from lower are written back to upper.
	OverlayMode_UPPER_CACHE OverlayMode = 2
	// LOWER_CACHE uses the lower store as a cache for the upper store.
	// reads go to the lower store first, then the upper store.
	// writes go to both stores.
	// removes go to both stores.
	// reads from upper are written back to lower.
	OverlayMode_LOWER_CACHE OverlayMode = 3
	// UPPER_READ_CACHE uses the upper store as a cache for the lower store.
	// reads go to the upper store first, then the lower store.
	// writes go to the lower store only.
	// removes go to the lower store only.
	// reads from lower are not written back to upper.
	OverlayMode_UPPER_READ_CACHE OverlayMode = 4
	// LOWER_READ_CACHE uses the lower store as a cache for the upper store.
	// reads go to the lower store first, then the upper store.
	// writes go to the upper store only.
	// removes go to both stores.
	// reads from upper are not written back to lower.
	OverlayMode_LOWER_READ_CACHE OverlayMode = 5
	// UPPER_WRITE_CACHE uses the upper store as a write cache for the lower store.
	// reads go to the upper store first, then the lower store.
	// writes go to the upper store only.
	// removes go to both stores.
	// reads from lower are not written back to upper.
	OverlayMode_UPPER_WRITE_CACHE OverlayMode = 6
	// LOWER_WRITE_CACHE uses the lower store as a write cache for the upper store.
	// reads go to the lower store first, then the upper store.
	// writes go to the lower store only.
	// removes go to both stores.
	// reads from upper are not written back to lower.
	OverlayMode_LOWER_WRITE_CACHE OverlayMode = 7
	// UPPER_READBACK_CACHE treats the lower store as read-only and immutable
	// (e.g. remote packfiles) while caching reads back into the upper store.
	// reads go to the upper store first, then the lower store.
	// reads from lower are written back to upper.
	// writes go to the upper store only.
	// removes go to the upper store only (lower lifecycle is external, e.g.
	// packfile garbage collection / repack).
	OverlayMode_UPPER_READBACK_CACHE OverlayMode = 8
)

func (OverlayMode) Enum 

func (x OverlayMode) Enum() *OverlayMode

func (OverlayMode) MarshalJSON 

func (x OverlayMode) MarshalJSON() ([]byte, error)

MarshalJSON marshals the OverlayMode to JSON.

func (OverlayMode) MarshalProtoJSON 

func (x OverlayMode) MarshalProtoJSON(s *json.MarshalState)

MarshalProtoJSON marshals the OverlayMode to JSON.

func (OverlayMode) MarshalProtoText 

func (x OverlayMode) MarshalProtoText() string

func (OverlayMode) MarshalText 

func (x OverlayMode) MarshalText() ([]byte, error)

MarshalText marshals the OverlayMode to text.

func (OverlayMode) String 

func (x OverlayMode) String() string

func (*OverlayMode) UnmarshalJSON 

func (x *OverlayMode) UnmarshalJSON(b []byte) error

UnmarshalJSON unmarshals the OverlayMode from JSON.

func (*OverlayMode) UnmarshalProtoJSON 

func (x *OverlayMode) UnmarshalProtoJSON(s *json.UnmarshalState)

UnmarshalProtoJSON unmarshals the OverlayMode from JSON.

func (*OverlayMode) UnmarshalText 

func (x *OverlayMode) UnmarshalText(b []byte) error

UnmarshalText unmarshals the OverlayMode from text.

type PutBatchEntry 

type PutBatchEntry struct {
	// Ref is the expected content-addressed block reference.
	Ref *BlockRef
	// Data is the encoded block payload.
	Data []byte
	// Refs are outgoing block references recorded with this write.
	Refs []*BlockRef
	// Tombstone marks the block ref as deleted.
	Tombstone bool
}

PutBatchEntry describes one durable block write in a lower-layer batch.

type PutOpts 

type PutOpts struct {

	// HashType is the hash type to use.
	// If unset (0 value) will use default for the store.
	HashType hash.HashType `protobuf:"varint,1,opt,name=hash_type,json=hashType,proto3" json:"hashType,omitempty"`
	// ForceBlockRef forces the block ref to equal the given ref.
	// Can be unset to indicate none (allow any).
	//
	// If the generated BlockRef does not match, the storage operation is aborted
	// and returns block.ErrBlockRefMismatch.
	ForceBlockRef *BlockRef `protobuf:"bytes,2,opt,name=force_block_ref,json=forceBlockRef,proto3" json:"forceBlockRef,omitempty"`
	// Refs are outgoing block references recorded with this write.
	// GC-aware stores consume these refs as part of the put. Stores that do not
	// participate in GC ignore this field.
	Refs []*BlockRef `protobuf:"bytes,3,rep,name=refs,proto3" json:"refs,omitempty"`
	// contains filtered or unexported fields
}

PutOpts are options that can be passed to PutBlock.

func (*PutOpts) CloneMessageVT 

func (m *PutOpts) CloneMessageVT() protobuf_go_lite.CloneMessage

func (*PutOpts) CloneVT 

func (m *PutOpts) CloneVT() *PutOpts

func (*PutOpts) EqualMessageVT 

func (this *PutOpts) EqualMessageVT(thatMsg any) bool

func (*PutOpts) EqualVT 

func (this *PutOpts) EqualVT(that *PutOpts) bool

func (*PutOpts) GetForceBlockRef 

func (x *PutOpts) GetForceBlockRef() *BlockRef

func (*PutOpts) GetHashType 

func (x *PutOpts) GetHashType() hash.HashType

func (*PutOpts) GetRefs 

func (x *PutOpts) GetRefs() []*BlockRef

func (*PutOpts) MarshalB58 

func (o *PutOpts) MarshalB58() string

MarshalB58 marshals the put opts to a base58 string form.

func (*PutOpts) MarshalBlock 

func (o *PutOpts) MarshalBlock() ([]byte, error)

MarshalBlock marshals the block to binary. This is the initial step of marshaling, before transformations.

func (*PutOpts) MarshalJSON 

func (x *PutOpts) MarshalJSON() ([]byte, error)

MarshalJSON marshals the PutOpts to JSON.

func (*PutOpts) MarshalProtoJSON 

func (x *PutOpts) MarshalProtoJSON(s *json.MarshalState)

MarshalProtoJSON marshals the PutOpts message to JSON.

func (*PutOpts) MarshalProtoText 

func (x *PutOpts) MarshalProtoText() string

func (*PutOpts) MarshalString 

func (o *PutOpts) MarshalString() string

MarshalString marshals the put opts to b58 string.

func (*PutOpts) MarshalToSizedBufferVT 

func (m *PutOpts) MarshalToSizedBufferVT(dAtA []byte) (int, error)

func (*PutOpts) MarshalToVT 

func (m *PutOpts) MarshalToVT(dAtA []byte) (int, error)

func (*PutOpts) MarshalVT 

func (m *PutOpts) MarshalVT() (dAtA []byte, err error)

func (*PutOpts) ProtoMessage 

func (*PutOpts) ProtoMessage()

func (*PutOpts) Reset 

func (x *PutOpts) Reset()

func (*PutOpts) SelectHashType 

func (o *PutOpts) SelectHashType(defHashType hash.HashType) hash.HashType

SelectHashType selects the hash type to use for the operation. The given hash type should be the default value to use.

func (*PutOpts) SizeVT 

func (m *PutOpts) SizeVT() (n int)

func (*PutOpts) String 

func (x *PutOpts) String() string

func (*PutOpts) UnmarshalB58 

func (o *PutOpts) UnmarshalB58(ref string) error

UnmarshalB58 unmarshals the put opts from base58 string form.

func (*PutOpts) UnmarshalBlock 

func (o *PutOpts) UnmarshalBlock(data []byte) error

UnmarshalBlock unmarshals the block to the object. This is the final step of decoding, after transformations.

func (*PutOpts) UnmarshalJSON 

func (x *PutOpts) UnmarshalJSON(b []byte) error

UnmarshalJSON unmarshals the PutOpts from JSON.

func (*PutOpts) UnmarshalProtoJSON 

func (x *PutOpts) UnmarshalProtoJSON(s *json.UnmarshalState)

UnmarshalProtoJSON unmarshals the PutOpts message from JSON.

func (*PutOpts) UnmarshalVT 

func (m *PutOpts) UnmarshalVT(dAtA []byte) error

func (*PutOpts) Validate 

func (o *PutOpts) Validate() error

Validate validates the put opts.

type StoreFeature 

type StoreFeature int32

StoreFeature is a bitmask of native block store capabilities. Each non-zero enum value is a single bit and values are combined with bitwise OR. The values are intentionally not sequential. 

const (
	// STORE_FEATURE_UNKNOWN indicates no native capabilities are reported.
	StoreFeature_STORE_FEATURE_UNKNOWN StoreFeature = 0
	// STORE_FEATURE_NATIVE_BATCH_PUT means PutBlockBatch is implemented natively.
	StoreFeature_STORE_FEATURE_NATIVE_BATCH_PUT StoreFeature = 1
	// STORE_FEATURE_NATIVE_BATCH_EXISTS means GetBlockExistsBatch is implemented natively.
	StoreFeature_STORE_FEATURE_NATIVE_BATCH_EXISTS StoreFeature = 2
	// STORE_FEATURE_NATIVE_BACKGROUND_PUT means PutBlockBackground actually deprioritizes writes.
	StoreFeature_STORE_FEATURE_NATIVE_BACKGROUND_PUT StoreFeature = 4
	// STORE_FEATURE_NATIVE_FLUSH means Flush has buffered work to publish.
	StoreFeature_STORE_FEATURE_NATIVE_FLUSH StoreFeature = 8
	// STORE_FEATURE_NATIVE_DEFER_FLUSH means BeginDeferFlush and EndDeferFlush batch flush work.
	StoreFeature_STORE_FEATURE_NATIVE_DEFER_FLUSH StoreFeature = 16
)

func (StoreFeature) Enum 

func (x StoreFeature) Enum() *StoreFeature

func (StoreFeature) Has 

func (s StoreFeature) Has(feat StoreFeature) bool

Has reports whether this feature bitset contains every requested feature.

func (StoreFeature) MarshalJSON 

func (x StoreFeature) MarshalJSON() ([]byte, error)

MarshalJSON marshals the StoreFeature to JSON.

func (StoreFeature) MarshalProtoJSON 

func (x StoreFeature) MarshalProtoJSON(s *json.MarshalState)

MarshalProtoJSON marshals the StoreFeature to JSON.

func (StoreFeature) MarshalProtoText 

func (x StoreFeature) MarshalProtoText() string

func (StoreFeature) MarshalText 

func (x StoreFeature) MarshalText() ([]byte, error)

MarshalText marshals the StoreFeature to text.

func (StoreFeature) String 

func (x StoreFeature) String() string

func (*StoreFeature) UnmarshalJSON 

func (x *StoreFeature) UnmarshalJSON(b []byte) error

UnmarshalJSON unmarshals the StoreFeature from JSON.

func (*StoreFeature) UnmarshalProtoJSON 

func (x *StoreFeature) UnmarshalProtoJSON(s *json.UnmarshalState)

UnmarshalProtoJSON unmarshals the StoreFeature from JSON.

func (*StoreFeature) UnmarshalText 

func (x *StoreFeature) UnmarshalText(b []byte) error

UnmarshalText unmarshals the StoreFeature from text.

type StoreOps 

type StoreOps interface {
	// GetHashType returns the preferred hash type for the store.
	// This should return as fast as possible (called frequently).
	// If 0 is returned, uses a default defined by Hydra.
	GetHashType() hash.HashType
	// GetSupportedFeatures returns the native feature bitmask for the store.
	// This should return as fast as possible (called frequently) and remain
	// static for the lifetime of the store.
	GetSupportedFeatures() StoreFeature
	// PutBlock puts a block into the store.
	// The ref should not be modified after return.
	// The second return value can optionally indicate if the block already existed.
	// If the hash type is unset, use the type from GetHashType().
	PutBlock(ctx context.Context, data []byte, opts *PutOpts) (*BlockRef, bool, error)
	// PutBlockBatch writes a batch of block operations.
	// Implementations without native batching fall back internally.
	PutBlockBatch(ctx context.Context, entries []*PutBatchEntry) error
	// PutBlockBackground writes a block at background priority.
	// Implementations without native background priority fall back to PutBlock.
	PutBlockBackground(ctx context.Context, data []byte, opts *PutOpts) (*BlockRef, bool, error)
	// GetBlock gets a block with the given reference.
	// The ref should not be modified or retained by GetBlock.
	// Returns data, found, error.
	// Returns nil, false, nil if not found.
	// Note: the block may not be in the specified bucket.
	GetBlock(ctx context.Context, ref *BlockRef) ([]byte, bool, error)
	// GetBlockExists checks if a block exists with a cid reference.
	// The ref should not be modified or retained by GetBlock.
	// Note: the block may not be in the specified bucket.
	GetBlockExists(ctx context.Context, ref *BlockRef) (bool, error)
	// GetBlockExistsBatch checks whether each block reference exists.
	// Implementations without native batching fall back internally.
	GetBlockExistsBatch(ctx context.Context, refs []*BlockRef) ([]bool, error)
	// RmBlock deletes a block from the bucket.
	// Does not return an error if the block was not present.
	// In some cases, will return before confirming delete.
	RmBlock(ctx context.Context, ref *BlockRef) error
	// StatBlock returns metadata about a block without reading its data.
	// Returns nil, nil if the block does not exist.
	StatBlock(ctx context.Context, ref *BlockRef) (*BlockStat, error)
	// Flush publishes buffered writes when the store has a durability boundary.
	// Implementations without buffering return nil.
	Flush(ctx context.Context) error
	// BeginDeferFlush opens a defer-flush scope.
	BeginDeferFlush()
	// EndDeferFlush closes a defer-flush scope.
	EndDeferFlush(ctx context.Context) error
}

StoreOps can read/write blocks.

func NewStoreRW 

func NewStoreRW(readHandle, writeHandle StoreOps) StoreOps

NewStoreRW constructs a new Store handle using a read handle and an optional write handle. If the write handle is not nil, the write (put and delete) calls will go to it. Otherwise, all calls are sent to the read handle.

type StoreOverlay 

type StoreOverlay struct {
	// contains filtered or unexported fields
}

StoreOverlay layers an upper block store over a lower store.

ctx is used for writeback requests

func NewOverlay 

func NewOverlay(
	ctx context.Context,
	lower,
	upper StoreOps,
	mode OverlayMode,
	writebackTimeout time.Duration,
	writebackPutOpts *PutOpts,
) *StoreOverlay

NewOverlay constructs a new overlay store.

ctx is used for writeback requests

func (*StoreOverlay) BeginDeferFlush 

func (o *StoreOverlay) BeginDeferFlush()

BeginDeferFlush forwards to upper and lower stores that support deferred flushing.

func (*StoreOverlay) EndDeferFlush 

func (o *StoreOverlay) EndDeferFlush(ctx context.Context) error

EndDeferFlush forwards to upper and lower stores that support deferred flushing.

func (*StoreOverlay) Flush 

func (o *StoreOverlay) Flush(ctx context.Context) error

Flush forwards the durability boundary to stores used by the overlay.

func (*StoreOverlay) GetBlock 

func (o *StoreOverlay) GetBlock(ctx context.Context, ref *BlockRef) ([]byte, bool, error)

GetBlock gets a block with the given reference. The ref should not be modified or retained by GetBlock. Returns data, found, error. Returns nil, false, nil if not found. Note: the block may not be in the specified bucket.

func (*StoreOverlay) GetBlockExists 

func (o *StoreOverlay) GetBlockExists(ctx context.Context, ref *BlockRef) (bool, error)

GetBlockExists checks if a block exists with a cid reference. The ref should not be modified or retained by GetBlock. Note: the block may not be in the specified bucket.

func (*StoreOverlay) GetBlockExistsBatch 

func (o *StoreOverlay) GetBlockExistsBatch(ctx context.Context, refs []*BlockRef) ([]bool, error)

GetBlockExistsBatch checks block existence using the same read policy as GetBlockExists.

func (*StoreOverlay) GetHashType 

func (o *StoreOverlay) GetHashType() hash.HashType

GetHashType returns the preferred hash type for the store. This should return as fast as possible (called frequently). If 0 is returned, uses a default defined by Hydra.

func (*StoreOverlay) GetSupportedFeatures 

func (o *StoreOverlay) GetSupportedFeatures() StoreFeature

GetSupportedFeatures returns the native feature bitmask for the overlay.

func (*StoreOverlay) PutBlock 

func (o *StoreOverlay) PutBlock(ctx context.Context, data []byte, opts *PutOpts) (*BlockRef, bool, error)

PutBlock puts a block into the store. The ref should not be modified after return. The second return value can optionally indicate if the block already existed. If the hash type is unset, use the type from GetHashType().

func (*StoreOverlay) PutBlockBackground 

func (o *StoreOverlay) PutBlockBackground(ctx context.Context, data []byte, opts *PutOpts) (*BlockRef, bool, error)

PutBlockBackground writes a block in the background using the same target-store policy as PutBlock.

func (*StoreOverlay) PutBlockBatch 

func (o *StoreOverlay) PutBlockBatch(ctx context.Context, entries []*PutBatchEntry) error

PutBlockBatch writes a batch of blocks using the same target-store policy as PutBlock.

func (*StoreOverlay) RmBlock 

func (o *StoreOverlay) RmBlock(ctx context.Context, ref *BlockRef) error

RmBlock deletes a block from the bucket. Does not return an error if the block was not present. In some cases, will return before confirming delete.

func (*StoreOverlay) StatBlock 

func (o *StoreOverlay) StatBlock(ctx context.Context, ref *BlockRef) (*BlockStat, error)

StatBlock returns metadata about a block without reading its data. Returns nil, nil if the block does not exist.

type StoreRW 

type StoreRW struct {
	// contains filtered or unexported fields
}

StoreRW combines a read and write store together.

func (*StoreRW) BeginDeferFlush 

func (b *StoreRW) BeginDeferFlush()

BeginDeferFlush forwards to the write handle if it supports deferred flushing.

func (*StoreRW) EndDeferFlush 

func (b *StoreRW) EndDeferFlush(ctx context.Context) error

EndDeferFlush forwards to the write handle if it supports deferred flushing.

func (*StoreRW) Flush 

func (b *StoreRW) Flush(ctx context.Context) error

Flush forwards the durability boundary to the write handle.

func (*StoreRW) GetBlock 

func (b *StoreRW) GetBlock(ctx context.Context, ref *BlockRef) ([]byte, bool, error)

GetBlock gets a block with a cid reference. The ref should not be modified or retained by GetBlock. Note: the block may not be in the specified bucket.

func (*StoreRW) GetBlockExists 

func (b *StoreRW) GetBlockExists(ctx context.Context, ref *BlockRef) (bool, error)

GetBlockExists checks if a block exists with a cid reference. The ref should not be modified or retained by GetBlock. Note: the block may not be in the specified bucket.

func (*StoreRW) GetBlockExistsBatch 

func (b *StoreRW) GetBlockExistsBatch(ctx context.Context, refs []*BlockRef) ([]bool, error)

GetBlockExistsBatch forwards batched existence probes to the read handle when supported.

func (*StoreRW) GetHashType 

func (b *StoreRW) GetHashType() hash.HashType

GetHashType returns the preferred hash type for the store. This should return as fast as possible (called frequently). If 0 is returned, uses a default defined by Hydra.

func (*StoreRW) GetSupportedFeatures 

func (b *StoreRW) GetSupportedFeatures() StoreFeature

GetSupportedFeatures returns the native feature bitmask for the store.

func (*StoreRW) PutBlock 

func (b *StoreRW) PutBlock(ctx context.Context, data []byte, opts *PutOpts) (*BlockRef, bool, error)

PutBlock puts a block into the store. The ref should not be modified after return.

func (*StoreRW) PutBlockBackground 

func (b *StoreRW) PutBlockBackground(ctx context.Context, data []byte, opts *PutOpts) (*BlockRef, bool, error)

PutBlockBackground forwards to the write handle if it supports background writes.

func (*StoreRW) PutBlockBatch 

func (b *StoreRW) PutBlockBatch(ctx context.Context, entries []*PutBatchEntry) error

PutBlockBatch forwards to the write handle if it supports batched writes.

func (*StoreRW) RmBlock 

func (b *StoreRW) RmBlock(ctx context.Context, ref *BlockRef) error

RmBlock deletes a block from the bucket. Does not return an error if the block was not present. In some cases, will return before confirming delete.

func (*StoreRW) StatBlock 

func (b *StoreRW) StatBlock(ctx context.Context, ref *BlockRef) (*BlockStat, error)

StatBlock returns metadata about a block without reading its data. Returns nil, nil if the block does not exist.

type SubBlock 

type SubBlock interface {
	// IsNil checks if the object is nil.
	IsNil() bool
}

SubBlock is a object contained inside a Block. May optionally implement Block or other Block interfaces.

type SubBlockCtor 

type SubBlockCtor func(create bool) SubBlock

SubBlockCtor constructs a sub-block. If create == false, returns nil if the field is not set.

func NewSubBlockCtor 

func NewSubBlockCtor[T ComparableSubBlock](r *T, ctor func() T) SubBlockCtor

NewSubBlockCtor constructs a new SubBlock constructor. returns nil if r is nil usage: block.NewSubBlockCtor(r, func() *ChangeLogLL { return &ChangeLogLL{} })

type Transaction 

type Transaction struct {
	// contains filtered or unexported fields
}

Transaction tracks refs traversed between blocks, batching writes and propagating changes through the merkle graph.

The decoded object form of the block can be stored / attached to a block handle. Changes are written to storage with a topological reference sort.

Empty blocks are not written to storage: they are instead represented with a nil BlockRef. SetBlockRef should handle nil BlockRef objects correctly.

func (*Transaction) GetBlockGraph 

func (t *Transaction) GetBlockGraph() graph.Graph

GetBlockGraph returns a handle to the internal block graph state. Do not modify this, used for analysis.

func (*Transaction) GetPutOpts 

func (t *Transaction) GetPutOpts() *PutOpts

GetPutOpts returns the transaction's put options.

func (*Transaction) GetStoreOps 

func (t *Transaction) GetStoreOps() StoreOps

GetStoreOps returns the transaction's store operations.

func (*Transaction) GetTransformer 

func (t *Transaction) GetTransformer() Transformer

GetTransformer returns the transaction's block transformer.

func (*Transaction) SetBufferedStoreSettings 

func (t *Transaction) SetBufferedStoreSettings(s *BufferedStoreSettings)

SetBufferedStoreSettings overrides the BufferedStore settings used to wrap the write store inside WriteAtRoot. Pass nil to reset to defaults. This must be called before Write/WriteAtRoot begins committing for the override to take effect on that commit.

func (*Transaction) SetRoot 

func (t *Transaction) SetRoot(cursor *Cursor) error

SetRoot sets the root of the transaction to a different position. Clears all parent blocks from the new root.

func (*Transaction) SetStoreOps 

func (t *Transaction) SetStoreOps(store StoreOps)

SetStoreOps replaces the transaction's store implementation. Used to swap in GCStoreOps after the RefGraph is available.

func (*Transaction) Write 

func (t *Transaction) Write(ctx context.Context, clearTree bool) (
	res *BlockRef,
	rcursor *Cursor,
	rerr error,
)

Write writes the dirty blocks to the store, propagating reference changes up the tree. Clears the blocks cache if clearTree is set, otherwise the updated references are written to the cursor tree. The final block in the event list will be the new root. The new root cursor is returned. Blocks that are not referenced by the root directly or indirectly are "cut" and removed.

Note: after Write with clearTree, use the new returned rcursor only.

func (*Transaction) WriteAtRoot 

func (t *Transaction) WriteAtRoot(ctx context.Context, clearTree bool, subRoot *Cursor) (
	res *BlockRef,
	rcursor *Cursor,
	rerr error,
)

WriteAtRoot writes dirty blocks to the store starting from a sub-tree root. If subRoot is nil, writes from the transaction root (same as Write). If subRoot is non-nil, writes only the sub-tree rooted at that cursor. Blocks outside the sub-tree are not touched. After writing, the sub-tree nodes are non-dirty with refs set, and their block data is freed if clearTree is set. The parent transaction's Write() will skip these nodes.

type Transformer 

type Transformer interface {
	// EncodeBlock encodes the block according to the config.
	// May reuse the same byte slice if possible.
	EncodeBlock([]byte) ([]byte, error)
	// DecodeBlock decodes the block according to the config.
	// May reuse the same byte slice if possible.
	DecodeBlock([]byte) ([]byte, error)
}

Transformer encodes and decodes blocks for storage.

Source Files

View all Source files

Directories

Path Synopsis
Package block_copy provides functions for copying block DAGs between stores.
 Package block_copy provides functions for copying block DAGs between stores.
gc
Package block_gc implements garbage collection for Hydra block stores.
 Package block_gc implements garbage collection for Hydra block stores.
gcgraph
Package gcgraph implements the GC reference graph on OPFS.
 Package gcgraph implements the GC reference graph on OPFS.
rpc
rpc/client
rpc/server
wal
rpc
client
server
bucket
controller
http
http/lookup
http/server
http/test
inmem
kvfile
kvfile/http
kvtx
overlay
redis
ristretto
rpc
rpc/lookup
rpc/server
rpc/server/bucket
s3
s3/lookup
test
vlogger
all
blockenc
chksum
e2e
lz4
s2
snappy
viz
dot
dot/demo command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.