storage

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 29, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package storage provides the key/value storage backends OrbitDB uses for log entries, heads, indexes, keys and identities.

It mirrors src/storage in @orbitdb/core: MemoryStorage, LRUStorage, LevelStorage, IPFSBlockStorage and ComposedStorage all implement Storage, so any of them can be plugged in wherever a database or log accepts one.

Index

Constants

View Source 
const DefaultIPFSTimeout = 30 * time.Second

DefaultIPFSTimeout bounds a single block Get or Put, matching the 30s default of @orbitdb/core.

View Source 
const DefaultLRUSize = 1000000

DefaultLRUSize is the capacity used when NewLRUStorage is given size <= 0.

View Source 
const DefaultLevelPath = "./level"

DefaultLevelPath is the directory used when NewLevelStorage is given "".

Variables

View Source 
var ErrNotFound = errors.New("storage: key not found")

ErrNotFound is returned by Get when a key is not present.

Functions

This section is empty.

Types

type BlockService 

type BlockService interface {
	GetBlock(ctx context.Context, c cid.Cid) (blocks.Block, error)
	AddBlock(ctx context.Context, b blocks.Block) error
}

BlockService is the subset of a block service IPFSBlockStorage needs. boxo's blockservice.BlockService satisfies it; with a bitswap exchange attached, Get fetches blocks the local node does not have from peers.

type ComposedStorage 

type ComposedStorage struct {
	// contains filtered or unexported fields
}

ComposedStorage layers two storages, typically a fast cache (LRU) in front of a durable or remote one (Level, IPFS). Writes go to both; reads try the first and fall back to the second, copying a hit back into the first.

func NewComposedStorage 

func NewComposedStorage(first, second Storage) *ComposedStorage

NewComposedStorage composes first (checked first on reads) with second.

func (*ComposedStorage) Clear 

func (s *ComposedStorage) Clear(ctx context.Context) error

Clear implements Storage.

func (*ComposedStorage) Close 

func (s *ComposedStorage) Close() error

Close implements Storage. Both layers are closed even if the first fails.

func (*ComposedStorage) Del 

func (s *ComposedStorage) Del(ctx context.Context, key string) error

Del implements Storage.

func (*ComposedStorage) Get 

func (s *ComposedStorage) Get(ctx context.Context, key string) ([]byte, error)

Get implements Storage.

func (*ComposedStorage) Iterator 

func (s *ComposedStorage) Iterator(ctx context.Context, opts IteratorOptions) iter.Seq2[Pair, error]

Iterator implements Storage. It yields the pairs of the first storage and then those of the second whose keys have not been seen yet. opts is passed to both, so Amount bounds each storage separately, as in @orbitdb/core.

func (*ComposedStorage) Merge 

func (s *ComposedStorage) Merge(ctx context.Context, other Storage) error

Merge implements Storage. Like @orbitdb/core it merges in both directions: afterwards this storage and other each hold the union of both.

func (*ComposedStorage) Persist 

func (s *ComposedStorage) Persist(ctx context.Context, key string) error

Persist implements Persister by persisting key in whichever layers support it.

func (*ComposedStorage) Put 

func (s *ComposedStorage) Put(ctx context.Context, key string, value []byte) error

Put implements Storage.

type IPFSBlockStorage 

type IPFSBlockStorage struct {
	// contains filtered or unexported fields
}

IPFSBlockStorage stores values as IPFS blocks keyed by their CID. Keys are CID strings (OrbitDB uses base58btc CIDv1 dag-cbor hashes). Values are verified against the key's multihash before being stored, so a peer cannot plant bytes under a hash they do not match.

Like @orbitdb/core, Del, Iterator, Merge and Clear are no-ops: blocks are content-addressed and may be shared with other databases.

func NewIPFSBlockStorage 

func NewIPFSBlockStorage(bs BlockService, opts IPFSBlockStorageOptions) (*IPFSBlockStorage, error)

NewIPFSBlockStorage returns a storage backed by bs.

func (*IPFSBlockStorage) Clear 

func (s *IPFSBlockStorage) Clear(context.Context) error

Clear implements Storage. It is a no-op.

func (*IPFSBlockStorage) Close 

func (s *IPFSBlockStorage) Close() error

Close implements Storage. It aborts in-flight Gets and Puts.

func (*IPFSBlockStorage) Del 

func (s *IPFSBlockStorage) Del(context.Context, string) error

Del implements Storage. It is a no-op.

func (*IPFSBlockStorage) Get 

func (s *IPFSBlockStorage) Get(ctx context.Context, key string) ([]byte, error)

Get implements Storage. When the block service has a network exchange the call blocks until a peer provides the block or the timeout expires.

func (*IPFSBlockStorage) Iterator 

func (s *IPFSBlockStorage) Iterator(context.Context, IteratorOptions) iter.Seq2[Pair, error]

Iterator implements Storage. It yields nothing: a block service cannot be enumerated.

func (*IPFSBlockStorage) Merge 

func (s *IPFSBlockStorage) Merge(context.Context, Storage) error

Merge implements Storage. It is a no-op.

func (*IPFSBlockStorage) Persist 

func (s *IPFSBlockStorage) Persist(ctx context.Context, key string) error

Persist implements Persister by pinning the block, if a Pinner is set.

func (*IPFSBlockStorage) Put 

func (s *IPFSBlockStorage) Put(ctx context.Context, key string, value []byte) error

Put implements Storage.

type IPFSBlockStorageOptions 

type IPFSBlockStorageOptions struct {
	// Pinner, if set, is used to pin every block that is put or persisted.
	Pinner Pinner
	// Timeout bounds each Get and Put. Zero means DefaultIPFSTimeout.
	Timeout time.Duration
}

IPFSBlockStorageOptions configures NewIPFSBlockStorage.

type IteratorOptions 

type IteratorOptions struct {
	// Amount limits the number of pairs yielded. Zero or negative means no
	// limit.
	Amount int
	// Reverse iterates in descending key order. Backends without a key order
	// (LRU, IPFS) ignore it.
	Reverse bool
}

IteratorOptions controls Storage.Iterator.

type LRUStorage 

type LRUStorage struct {
	// contains filtered or unexported fields
}

LRUStorage keeps the most recently used pairs in memory, evicting the least recently used one once it holds size pairs. It is used as the cache layer in front of slower storages.

func NewLRUStorage 

func NewLRUStorage(size int) (*LRUStorage, error)

NewLRUStorage returns an LRUStorage holding at most size pairs.

func (*LRUStorage) Clear 

func (s *LRUStorage) Clear(context.Context) error

Clear implements Storage.

func (*LRUStorage) Close 

func (s *LRUStorage) Close() error

Close implements Storage. It is a no-op.

func (*LRUStorage) Del 

func (s *LRUStorage) Del(_ context.Context, key string) error

Del implements Storage.

func (*LRUStorage) Get 

func (s *LRUStorage) Get(_ context.Context, key string) ([]byte, error)

Get implements Storage.

func (*LRUStorage) Iterator 

func (s *LRUStorage) Iterator(_ context.Context, opts IteratorOptions) iter.Seq2[Pair, error]

Iterator implements Storage. Pairs are yielded from the least to the most recently used; Reverse flips that order. Iterating does not change recency.

func (*LRUStorage) Merge 

func (s *LRUStorage) Merge(ctx context.Context, other Storage) error

Merge implements Storage.

func (*LRUStorage) Put 

func (s *LRUStorage) Put(_ context.Context, key string, value []byte) error

Put implements Storage.

type LevelStorage 

type LevelStorage struct {
	// contains filtered or unexported fields
}

LevelStorage persists pairs in a LevelDB database on disk. The on-disk format is plain LevelDB, so a keystore written by the JavaScript implementation can be opened directly.

func NewLevelStorage 

func NewLevelStorage(path string) (*LevelStorage, error)

NewLevelStorage opens (creating if needed) the LevelDB database at path.

func (*LevelStorage) Clear 

func (s *LevelStorage) Clear(context.Context) error

Clear implements Storage.

func (*LevelStorage) Close 

func (s *LevelStorage) Close() error

Close implements Storage.

func (*LevelStorage) Del 

func (s *LevelStorage) Del(_ context.Context, key string) error

Del implements Storage.

func (*LevelStorage) Get 

func (s *LevelStorage) Get(_ context.Context, key string) ([]byte, error)

Get implements Storage.

func (*LevelStorage) Iterator 

func (s *LevelStorage) Iterator(_ context.Context, opts IteratorOptions) iter.Seq2[Pair, error]

Iterator implements Storage. Pairs are yielded in key order, or descending key order when opts.Reverse is set.

func (*LevelStorage) Merge 

func (s *LevelStorage) Merge(ctx context.Context, other Storage) error

Merge implements Storage.

func (*LevelStorage) Put 

func (s *LevelStorage) Put(_ context.Context, key string, value []byte) error

Put implements Storage.

type MemoryStorage 

type MemoryStorage struct {
	// contains filtered or unexported fields
}

MemoryStorage keeps pairs in a map. It is the default storage for logs that are not given one.

func NewMemoryStorage 

func NewMemoryStorage() *MemoryStorage

NewMemoryStorage returns an empty MemoryStorage.

func (*MemoryStorage) Clear 

func (s *MemoryStorage) Clear(context.Context) error

Clear implements Storage.

func (*MemoryStorage) Close 

func (s *MemoryStorage) Close() error

Close implements Storage. It is a no-op.

func (*MemoryStorage) Del 

func (s *MemoryStorage) Del(_ context.Context, key string) error

Del implements Storage.

func (*MemoryStorage) Get 

func (s *MemoryStorage) Get(_ context.Context, key string) ([]byte, error)

Get implements Storage.

func (*MemoryStorage) Iterator 

func (s *MemoryStorage) Iterator(_ context.Context, opts IteratorOptions) iter.Seq2[Pair, error]

Iterator implements Storage. Pairs are yielded in key order from a snapshot taken when iteration starts, so the storage may be modified while iterating.

func (*MemoryStorage) Merge 

func (s *MemoryStorage) Merge(ctx context.Context, other Storage) error

Merge implements Storage.

func (*MemoryStorage) Put 

func (s *MemoryStorage) Put(_ context.Context, key string, value []byte) error

Put implements Storage.

type Pair 

type Pair struct {
	Key   string
	Value []byte
}

Pair is a single key/value record yielded by Storage.Iterator.

type Persister 

type Persister interface {
	Persist(ctx context.Context, key string) error
}

Persister is implemented by storages that can make a stored value durable beyond their normal retention, e.g. IPFSBlockStorage pins the block.

type Pinner 

type Pinner interface {
	IsPinned(ctx context.Context, c cid.Cid) (bool, error)
	Pin(ctx context.Context, c cid.Cid) error
}

Pinner pins blocks so a garbage-collecting blockstore keeps them.

type Storage 

type Storage interface {
	// Put stores value under key, replacing any existing value.
	Put(ctx context.Context, key string, value []byte) error

	// Get returns the value stored under key, or an error wrapping
	// ErrNotFound if there is none.
	Get(ctx context.Context, key string) ([]byte, error)

	// Del removes key. Removing a missing key is not an error.
	Del(ctx context.Context, key string) error

	// Iterator yields every stored pair. If the backend fails mid-way it
	// yields a zero Pair with the error and stops.
	Iterator(ctx context.Context, opts IteratorOptions) iter.Seq2[Pair, error]

	// Merge copies every pair of other into this storage.
	Merge(ctx context.Context, other Storage) error

	// Clear removes every pair.
	Clear(ctx context.Context) error

	// Close releases the resources held by the storage.
	Close() error
}

Storage is a byte-oriented key/value store.

Implementations must be safe for concurrent use.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.