storage

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 27, 2025 License: Apache-2.0, MIT Imports: 21 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AsIdentity 

func AsIdentity(key string) (digest []byte, ok bool, err error)

Types

type CachingTempStore 

type CachingTempStore struct {
	// contains filtered or unexported fields
}

CachingTempStore is a ReadableWritableStorage that is intended for temporary use. It uses DeferredStorageCar as a backing store, so the underlying CAR file is lazily created on the first write (none will be created if there are no writes).

A provided BlockWriteOpener will receive blocks for each Put operation, this is intended to be used to write a properly ordered CARv1 file.

func NewCachingTempStore 

func NewCachingTempStore(outWriter linking.BlockWriteOpener, store *DeferredStorageCar) *CachingTempStore

func (*CachingTempStore) Close 

func (ttrw *CachingTempStore) Close() error

Close will clean up any temporary resources used by the storage.

func (*CachingTempStore) Get 

func (ttrw *CachingTempStore) Get(ctx context.Context, key string) ([]byte, error)

func (*CachingTempStore) GetStream 

func (ttrw *CachingTempStore) GetStream(ctx context.Context, key string) (io.ReadCloser, error)

func (*CachingTempStore) Has 

func (ttrw *CachingTempStore) Has(ctx context.Context, key string) (bool, error)

func (*CachingTempStore) Put 

func (ttrw *CachingTempStore) Put(ctx context.Context, key string, data []byte) error

Put writes both to temporary readwrite caching storage (available for read operations) and to the underlying write-only CARv1 output at the same time.

type DeferredStorageCar 

type DeferredStorageCar struct {
	// contains filtered or unexported fields
}

DeferredStorageCar is a wrapper around github.com/ipld/go-car/v2/storage.StorageCar that defers creating the CAR until the first Put() operation. In this way it can be optimistically instantiated and no file will be created if it is never written to (such as in the case of an error).

func NewDeferredStorageCar 

func NewDeferredStorageCar(tempDir string, root cid.Cid) *DeferredStorageCar

NewDeferredStorageCar creates a new DeferredStorageCar.

func (*DeferredStorageCar) Close 

func (dcs *DeferredStorageCar) Close() error

Close will clean up any temporary resources used by the storage.

func (*DeferredStorageCar) Get 

func (dcs *DeferredStorageCar) Get(ctx context.Context, key string) ([]byte, error)

Get returns data from the underlying CARv1.

func (*DeferredStorageCar) GetStream 

func (dcs *DeferredStorageCar) GetStream(ctx context.Context, key string) (io.ReadCloser, error)

GetStream returns data from the underlying CARv1.

func (*DeferredStorageCar) Has 

func (dcs *DeferredStorageCar) Has(ctx context.Context, key string) (bool, error)

Has returns true if the underlying CARv1 has the key.

func (*DeferredStorageCar) Put 

func (dcs *DeferredStorageCar) Put(ctx context.Context, key string, data []byte) error

Put writes data to the underlying CARv1 which will be initialised on the first call to Put.

type DeferredWriter 

type DeferredWriter interface {
	ipldstorage.WritableStorage
	io.Closer
	BlockWriteOpener() linking.BlockWriteOpener
	OnPut(cb func(int), once bool)
}

type DuplicateAdderCar 

type DuplicateAdderCar struct {
	*deferred.DeferredCarWriter
	// contains filtered or unexported fields
}

func NewDuplicateAdderCarForPath 

func NewDuplicateAdderCarForPath(
	ctx context.Context,
	outPath string,
	root cid.Cid,
	path string,
	scope trustlessutils.DagScope,
	bytes *trustlessutils.ByteRange,
	store *DeferredStorageCar,
) *DuplicateAdderCar

func NewDuplicateAdderCarForStream 

func NewDuplicateAdderCarForStream(
	ctx context.Context,
	outStream io.Writer,
	root cid.Cid,
	path string,
	scope trustlessutils.DagScope,
	bytes *trustlessutils.ByteRange,
	store *DeferredStorageCar,
) *DuplicateAdderCar

func (*DuplicateAdderCar) BlockWriteOpener 

func (da *DuplicateAdderCar) BlockWriteOpener() linking.BlockWriteOpener

func (*DuplicateAdderCar) Close 

func (da *DuplicateAdderCar) Close() error

Close closes the dup stream, verifying completion, if one was created.

type ReadableWritableStorage 

type ReadableWritableStorage interface {
	ipldstorage.ReadableStorage
	ipldstorage.WritableStorage
	ipldstorage.StreamingReadableStorage
}

type StreamingStore added in v1.0.0 

type StreamingStore struct {
	// contains filtered or unexported fields
}

StreamingStore is a storage that streams blocks directly to output without retaining block data. It maintains only a set of seen CIDs for deduplication of requests (not output).

Get() always returns NotFound since block data is not retained. Has() checks the seen set to determine if a block was already written. Put() writes directly to output and records the CID as seen.

func NewStreamingStore added in v1.0.0 

func NewStreamingStore(outWriter linking.BlockWriteOpener) *StreamingStore

NewStreamingStore creates a new streaming storage that writes blocks directly to the provided BlockWriteOpener without retaining block data.

func (*StreamingStore) Close added in v1.0.0 

func (s *StreamingStore) Close() error

Close marks the store as closed. Further operations will return errClosed.

func (*StreamingStore) Get added in v1.0.0 

func (s *StreamingStore) Get(ctx context.Context, key string) ([]byte, error)

Get always returns NotFound. StreamingStore does not retain block data - it streams directly to output. Callers needing block data must fetch from the network.

func (*StreamingStore) GetStream added in v1.0.0 

func (s *StreamingStore) GetStream(ctx context.Context, key string) (io.ReadCloser, error)

GetStream always returns NotFound. StreamingStore does not retain block data.

func (*StreamingStore) Has added in v1.0.0 

func (s *StreamingStore) Has(ctx context.Context, key string) (bool, error)

Has returns true if the CID has been written to this store. This is used for request deduplication, not output deduplication.

func (*StreamingStore) Put added in v1.0.0 

func (s *StreamingStore) Put(ctx context.Context, key string, data []byte) error

Put writes the block directly to the output stream and records the CID as seen. If the block has already been written (based on CID), this is a no-op.

func (*StreamingStore) Seen added in v1.0.0 

func (s *StreamingStore) Seen() int

Seen returns the number of unique blocks that have been written.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.