meta

package
v0.51.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 4, 2026 License: GPL-3.0 Imports: 39 Imported by: 1

Documentation

Index

Constants

View Source 
const (
	// ReviveStatusGraveyard is the type of revival status of an object from tombstone.
	ReviveStatusGraveyard reviveStatusType = iota
	// ReviveStatusGarbage is the type of revival status of an object from the garbage bucket.
	ReviveStatusGarbage
	// ReviveStatusError is the type of status when an error occurs during revive.
	ReviveStatusError
)

Variables

View Source 
var ErrDegradedMode = logicerr.New("metabase is in a degraded mode")

ErrDegradedMode is returned when metabase is in a degraded mode.

View Source 
var ErrEndOfListing = logicerr.New("end of object listing")

ErrEndOfListing is returned from object listing with cursor when storage can't return any more objects after provided cursor. Use nil cursor object to start listing again.

View Source 
var ErrInterruptIterator = logicerr.New("iterator is interrupted")

ErrInterruptIterator is returned by iteration handlers as a "break" keyword.

View Source 
var ErrLockObjectRemoval = logicerr.New("lock object removal")

ErrLockObjectRemoval is returned when inhume operation is being performed on lock object, and it is not a forced object removal.

View Source 
var ErrObjectIsExpired = logicerr.New("object is expired")

ErrObjectIsExpired is returned when the requested object's epoch is less than the current one. Such objects are considered as removed and should not be returned from the Storage Engine.

View Source 
var ErrObjectWasNotRemoved = logicerr.New("object neither has tombstone nor was marked with GC mark")

ErrObjectWasNotRemoved is returned when object neither has tombstone nor was marked with GC mark.

View Source 
var ErrOutdatedVersion = logicerr.New("invalid version, resynchronization is required")

ErrOutdatedVersion is returned on initializing an existing metabase that is not compatible with the current code version.

View Source 
var ErrReadOnlyMode = logicerr.New("metabase is in a read-only mode")

ErrReadOnlyMode is returned when metabase is in a read-only mode.

View Source 
var ErrReviveFromContainerGarbage = logicerr.New("revive from container marked with GC mark")

ErrReviveFromContainerGarbage is returned when the object is in the container that marked with GC mark.

Functions

func IsErrRemoved added in v0.28.0 

func IsErrRemoved(err error) bool

IsErrRemoved checks if error returned by Shard Exists/Get/Put method corresponds to removed object.

func PutMetadataForObject added in v0.45.2 

func PutMetadataForObject(tx *bbolt.Tx, hdr object.Object, phy bool) error

PutMetadataForObject fills object meta-data indexes using bbolt transaction. Transaction must be writable. Additional bucket for container's meta-data may be created using {255, CID...} form as a key.

Types

type ContainerInfo added in v0.49.0 

type ContainerInfo struct {
	StorageSize   uint64
	ObjectsNumber uint64
}

ContainerInfo groups metabase's objects status for a certain container.

type Containers added in v0.45.1 

type Containers interface {
	// Exists checks presence of the referenced container.
	Exists(cid.ID) (bool, error)
}

Containers provides access to information about NeoFS containers required for DB to process.

type Cursor added in v0.27.0 

type Cursor struct {
	// contains filtered or unexported fields
}

Cursor is a type for continuous object listing.

type DB 

type DB struct {
	// contains filtered or unexported fields
}

DB represents local metabase of storage node.

func New added in v0.13.0 

func New(opts ...Option) *DB

New creates and returns new Metabase instance.

func (*DB) Close 

func (db *DB) Close() error

Close closes boltDB instance.

func (*DB) CollectRawWithAttribute added in v0.51.0 

func (db *DB) CollectRawWithAttribute(cnr cid.ID, attr string, val []byte) ([]oid.ID, error)

CollectRawWithAttribute allows to fetch the list of objects precisely matching given attribute and value. No expiration/lock/tombstone checks are made.

func (*DB) Containers added in v0.13.0 

func (db *DB) Containers() (list []cid.ID, err error)

func (*DB) Delete 

func (db *DB) Delete(addrs []oid.Address) (DeleteRes, error)

Delete removes object records from metabase indexes. Does not stop on an error if there are more objects to handle requested; returns the first error appeared with a number of deleted objects wrapped.

Delete also looks up for objects that are hardly linked with elements of addrs list but not in the list themselves. If there are any, they are also deleted.

func (*DB) DeleteContainer added in v0.40.0 

func (db *DB) DeleteContainer(cID cid.ID) error

DeleteContainer removes any information that the metabase has associated with the provided container (its objects) except the graveyard-related one.

func (*DB) DumpInfo added in v0.13.0 

func (db *DB) DumpInfo() Info

DumpInfo returns information about the DB.

func (*DB) Exists added in v0.13.0 

func (db *DB) Exists(addr oid.Address, ignoreExpiration bool) (bool, error)

Exists returns ErrAlreadyRemoved if addr was marked as removed. Otherwise it returns true if addr is in primary index or false if it is not.

Returns an error of type apistatus.ObjectAlreadyRemoved if there is a tombstone associated with this object. Returns the object.ErrObjectIsExpired if the object is presented but already expired.

If referenced object is a parent of some stored objects, Exists returns [ParentError] wrapping: - *object.SplitInfoError wrapping object.SplitInfo collected from stored parts; - [ErrParts] if referenced object is EC.

func (*DB) Get 

func (db *DB) Get(addr oid.Address, raw bool) (*object.Object, error)

Get returns partial object header data for specified address (as stored in metabase). This header contains all user-defined attributes and mandatory header fields that can be used as search filters like size, checksum and owner. It does not contain parent header or session token.

"raw" flag controls virtual object processing, when false (default) a proper object header is returned, when true only SplitInfo of virtual object is returned.

Returns an error of type apistatus.ObjectNotFound if object is missing in DB. Returns an error of type apistatus.ObjectAlreadyRemoved if object has been placed in graveyard. Returns the object.ErrObjectIsExpired if the object is presented but already expired.

If raw and the object is a parent of some stored objects, Get returns: - object.SplitInfoError wrapping object.SplitInfo collected from parts if object is split; - iec.ErrPartitionedObject if object is EC.

func (*DB) GetContainerInfo added in v0.49.0 

func (db *DB) GetContainerInfo(id cid.ID) (ContainerInfo, error)

GetContainerInfo returns statistics about stored objects for specified container. If no info is found, empty ContainerInfo with no error are returned.

func (*DB) GetGarbage added in v0.40.0 

func (db *DB) GetGarbage(limit int) ([]oid.Address, []cid.ID, error)

GetGarbage returns garbage according to the metabase state. Garbage includes objects marked with GC mark (expired, tombstoned but not deleted from disk, extra replicated, etc.) and removed containers. The first return value describes garbage objects. These objects should be removed. The second return value describes garbage containers whose _all_ garbage objects were included in the first return value and, therefore, these containers can be deleted (if their objects are handled and deleted too).

func (*DB) InhumeContainer added in v0.40.0 

func (db *DB) InhumeContainer(cID cid.ID) (uint64, error)

InhumeContainer marks every object in a container as removed. Any further DB.Get calls will return apistatus.ObjectNotFound errors. Returns number of available objects marked with GC. There is no any LOCKs, forced GC marks and any relations checks, every object that belongs to a provided container will be marked as a removed one.

func (*DB) Init added in v0.13.0 

func (db *DB) Init() error

Init initializes metabase. It creates static (CID-independent) buckets in underlying BoltDB instance.

Returns ErrOutdatedVersion if a database at the provided path is outdated.

Does nothing if metabase has already been initialized and filled. To roll back the database to its initial state, use Reset.

func (*DB) IsLocked added in v0.36.0 

func (db *DB) IsLocked(addr oid.Address) (bool, error)

IsLocked checks is the provided object is locked by any `LOCK`. Not found object is considered as non-locked.

Returns only non-logical errors related to underlying database.

func (*DB) IterateExpired added in v0.16.0 

func (db *DB) IterateExpired(epoch uint64, h ExpiredObjectHandler) error

IterateExpired iterates over all objects in DB which are out of date relative to epoch. Locked objects are not included (do not confuse with objects of type LOCK).

If h returns ErrInterruptIterator, nil returns immediately. Returns other errors of h directly.

func (*DB) IterateOverGarbage added in v0.28.0 

func (db *DB) IterateOverGarbage(h func(oid.ID) error, cnr cid.ID, offset oid.ID) error

IterateOverGarbage iterates over all objects marked with GC mark in the given container.

The handler will be applied to the next after the specified offset if any are left.

Note: if offset is not found in db, iteration starts from the element that WOULD BE the following after the offset if offset was presented. That means that it is safe to delete offset element and pass if to the iteration once again: iteration would start from the next element.

Zero offset means start an iteration from the beginning.

If h returns ErrInterruptIterator, nil returns immediately. Returns other errors of h directly.

func (*DB) ListWithCursor added in v0.27.0 

func (db *DB) ListWithCursor(count int, cursor *Cursor, attrs ...string) ([]objectcore.AddressWithAttributes, *Cursor, error)

ListWithCursor lists physical objects available in metabase starting from cursor. Includes objects of all types. Does not include inhumed objects. Use cursor value from response for consecutive requests.

Optional attrs specifies attributes to include in the result. If object does not have requested attribute, corresponding element in the result is empty.

Returns ErrEndOfListing if there are no more objects to return or count parameter set to zero.

func (*DB) MarkGarbage added in v0.44.0 

func (db *DB) MarkGarbage(addrs ...oid.Address) (uint64, []oid.Address, error)

MarkGarbage marks objects to be physically removed from shard. force flag allows to override any restrictions imposed on object deletion (to be used by control service and other manual intervention cases).

func (*DB) ObjectCounters added in v0.32.0 

func (db *DB) ObjectCounters() (ObjectCounters, error)

ObjectCounters returns object counters that metabase has tracked since it was opened and initialized.

Returns only the errors that do not allow reading counter in Bolt database.

func (*DB) ObjectStatus added in v0.39.0 

func (db *DB) ObjectStatus(address oid.Address) (ObjectStatus, error)

ObjectStatus returns the status of the object in the Metabase. It contains state, path and indexed information about an object.

func (*DB) Open added in v0.13.0 

func (db *DB) Open(readOnly bool) error

Open boltDB instance for metabase.

func (*DB) Put 

func (db *DB) Put(obj *object.Object) error

Put updates metabase indexes for the given object.

Returns an error of type apistatus.ObjectAlreadyRemoved if object has been placed in graveyard. Returns the object.ErrObjectIsExpired if the object is presented but already expired.

Returns apistatus.ErrObjectAlreadyRemoved if obj is of object.TypeLock type and there is an object of object.TypeTombstone type associated with the same target.

func (*DB) ReadLastResyncEpoch added in v0.44.0 

func (db *DB) ReadLastResyncEpoch() (epoch uint64, err error)

ReadLastResyncEpoch reads from db last epoch when metabase was resynchronized. If id is missing, returns 0, nil.

func (*DB) ReadShardID added in v0.27.7 

func (db *DB) ReadShardID() ([]byte, error)

ReadShardID reads shard id from db. If id is missing, returns nil, nil.

func (*DB) Reload added in v0.34.0 

func (db *DB) Reload(opts ...Option) (bool, error)

Reload reloads part of the configuration. It returns true iff database was reopened. If a config option is invalid, it logs an error and returns nil. If there was a problem with applying new configuration, an error is returned.

If a metabase was couldn't be reopened because of an error, ErrDegradedMode is returned.

func (*DB) Reset added in v0.25.0 

func (db *DB) Reset() error

Reset resets metabase. Works similar to Init but cleans up all static buckets and removes all dynamic (CID-dependent) ones in non-blank BoltDB instances.

func (*DB) ResolveECPart added in v0.49.0 

func (db *DB) ResolveECPart(cnr cid.ID, parent oid.ID, pi iec.PartInfo) (oid.ID, error)

ResolveECPart resolves object that carries EC part produced within cnr for parent object and indexed by pi, checks its availability and returns its ID.

If the object is not EC part but of object.TypeTombstone, object.TypeLock or object.TypeLink type, ResolveECPart returns its ID instead.

If the object is not EC part but it is size-split, ResolveECPart returns *object.SplitInfoError with object.SplitInfo depending on stored relatives. If DB contains the linker, its ID is attached. If DB contains only the last child, its ID is attached. If DB contains both objects, linker ID is always attached while last child's ID may or may not be attached depending on object order.

If DB is disabled by mode (e.g. DB.SetMode), ResolveECPart returns ErrDegradedMode.

If object has expired, ResolveECPart returns ErrObjectIsExpired.

If object exists but tombstoned (via [DB.Inhume] or stored tombstone object), ResolveECPart returns apistatus.ErrObjectAlreadyRemoved.

If object is marked as garbage (via DB.MarkGarbage), ResolveECPart returns apistatus.ErrObjectNotFound.

If object is locked (via [DB.Lock] or stored locker object), ResolveECPart ignores expiration, tombstone and garbage marks.

func (*DB) ResolveECPartWithPayloadLen added in v0.50.0 

func (db *DB) ResolveECPartWithPayloadLen(cnr cid.ID, parent oid.ID, pi iec.PartInfo) (oid.ID, uint64, error)

ResolveECPartWithPayloadLen is like DB.ResolveECPart but also returns payload length.

func (*DB) ReviveObject added in v0.44.0 

func (db *DB) ReviveObject(addr oid.Address) (res ReviveStatus, err error)

ReviveObject revives object by oid.Address. Removes GCMark/Tombstone records in the corresponding buckets and restore metrics.

func (*DB) Search added in v0.45.0 

func (db *DB) Search(cnr cid.ID, fs []objectcore.SearchFilter, attrs []string, cursor *objectcore.SearchCursor, count uint16) ([]client.SearchResultItem, []byte, error)

Search selects up to count container's objects from the given container matching the specified filters.

func (*DB) Select 

func (db *DB) Select(cnr cid.ID, filters object.SearchFilters) ([]oid.Address, error)

Select returns list of addresses of objects that match search filters.

Only creation epoch, payload size, user attributes and unknown system ones are allowed with numeric operators. Values of numeric filters must be base-10 integers.

Returns object.ErrInvalidSearchQuery if specified query is invalid.

func (*DB) SetLogger added in v0.30.0 

func (db *DB) SetLogger(l *zap.Logger)

SetLogger sets logger. It is used after the shard ID was generated to use it in logs.

func (*DB) SetMode added in v0.30.0 

func (db *DB) SetMode(m mode.Mode) error

SetMode sets the metabase mode of operation. If the mode assumes no operation metabase, the database is closed.

func (*DB) SyncCounters added in v0.32.0 

func (db *DB) SyncCounters() error

SyncCounters forces to synchronize the object counters.

func (*DB) WriteShardID added in v0.27.7 

func (db *DB) WriteShardID(id []byte) error

WriteShardID writes shard it to db.

type DeleteRes added in v0.13.0 

type DeleteRes struct {
	// Actually removed objects. First len(addrs) elements always contain addrs
	// passed to [DB.Delete], but order is different in general.
	RemovedObjects []RemovedObject
	// PhyRemoved contains the number of removed physical objects.
	PhyRemoved uint64
	// AvailableRemoved contains the number of removed available objects.
	AvailableRemoved uint64
}

DeleteRes groups the resulting values of Delete operation.

type EpochState added in v0.31.0 

type EpochState interface {
	// CurrentEpoch must return current epoch height.
	CurrentEpoch() uint64
}

EpochState is an interface that provides access to the current epoch number.

type ExpiredObjectHandler added in v0.16.0 

type ExpiredObjectHandler func(oid.Address, object.Type) error

ExpiredObjectHandler is an expired object handling function.

type HeaderField added in v0.46.0 

type HeaderField struct {
	K []byte
	V []byte
}

HeaderField is object header's field index.

type Info added in v0.13.0 

type Info struct {
	// Full path to the metabase.
	Path string

	// Permission of database file.
	Permission fs.FileMode

	// LastResyncEpoch is a last epoch when metabase was resynchronized.
	LastResyncEpoch uint64
}

Info groups the information about DB.

type ObjectCounters added in v0.32.0 

type ObjectCounters struct {
	// contains filtered or unexported fields
}

ObjectCounters groups object counter according to metabase state.

func (ObjectCounters) Logic added in v0.32.0 

func (o ObjectCounters) Logic() uint64

Logic returns logical object counter.

func (ObjectCounters) Phy added in v0.32.0 

func (o ObjectCounters) Phy() uint64

Phy returns physical object counter.

type ObjectStatus added in v0.39.0 

type ObjectStatus struct {
	Version     uint64
	HeaderIndex []HeaderField
	State       []string
	Path        string
	Error       error
}

ObjectStatus represents the status of the object in the Metabase.

type Option 

type Option func(*cfg)

Option is an option of DB constructor.

func WithBoltDBOptions added in v0.13.0 

func WithBoltDBOptions(opts *bbolt.Options) Option

WithBoltDBOptions returns option to specify BoltDB options.

func WithContainers added in v0.45.1 

func WithContainers(cs Containers) Option

WithContainers return option to specify container source.

func WithEpochState added in v0.31.0 

func WithEpochState(s EpochState) Option

WithEpochState return option to specify a source of current epoch height.

func WithInitContext added in v0.45.1 

func WithInitContext(ctx context.Context) Option

WithInitContext allows to specify context for DB.Init operation.

func WithLogger 

func WithLogger(l *zap.Logger) Option

WithLogger returns option to set logger of DB.

func WithMaxBatchDelay added in v0.29.0 

func WithMaxBatchDelay(d time.Duration) Option

WithMaxBatchDelay returns option to specify maximum time to wait before the batch of concurrent transactions is processed. This option is missing from `bbolt.Options` but is set right after DB is open.

func WithMaxBatchSize added in v0.29.0 

func WithMaxBatchSize(s int) Option

WithMaxBatchSize returns option to specify maximum concurrent operations to be processed in a single transactions. This option is missing from `bbolt.Options` but is set right after DB is open.

func WithPath added in v0.13.0 

func WithPath(path string) Option

WithPath returns option to set system path to Metabase.

func WithPermissions added in v0.13.0 

func WithPermissions(perm fs.FileMode) Option

WithPermissions returns option to specify permission bits of Metabase system path.

type RemovedObject added in v0.51.0 

type RemovedObject struct {
	Address    oid.Address
	PayloadLen uint64
}

RemovedObjects describes single item handled by DB.Delete.

type ReviveStatus added in v0.44.0 

type ReviveStatus struct {
	// contains filtered or unexported fields
}

ReviveStatus groups the resulting values of ReviveObject operation. Contains the type of revival status and message for details.

func (*ReviveStatus) Message added in v0.44.0 

func (s *ReviveStatus) Message() string

Message returns message of status.

func (*ReviveStatus) StatusType added in v0.44.0 

func (s *ReviveStatus) StatusType() reviveStatusType

StatusType returns the type of revival status.

func (*ReviveStatus) TombstoneAddress added in v0.49.0 

func (s *ReviveStatus) TombstoneAddress() oid.Address

TombstoneAddress returns the tombstone address.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.