meta

package
v0.49.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 15, 2025 License: GPL-3.0 Imports: 40 Imported by: 1

Documentation

Index

Constants

View Source 
const (
	// ReviveStatusGraveyard is the type of revival status of an object from a graveyard.
	ReviveStatusGraveyard reviveStatusType = iota
	// ReviveStatusGarbage is the type of revival status of an object from the garbage bucket.
	ReviveStatusGarbage
	// ReviveStatusError is the type of status when an error occurs during revive.
	ReviveStatusError
)

Variables

View Source 
var ErrDegradedMode = logicerr.New("metabase is in a degraded mode")

ErrDegradedMode is returned when metabase is in a degraded mode.

View Source 
var ErrEndOfListing = logicerr.New("end of object listing")

ErrEndOfListing is returned from object listing with cursor when storage can't return any more objects after provided cursor. Use nil cursor object to start listing again.

View Source 
var (
	ErrIncorrectRootObject = errors.New("invalid root object")
)
View Source 
var ErrInterruptIterator = logicerr.New("iterator is interrupted")

ErrInterruptIterator is returned by iteration handlers as a "break" keyword.

View Source 
var ErrLockObjectRemoval = logicerr.New("lock object removal")

ErrLockObjectRemoval is returned when inhume operation is being performed on lock object, and it is not a forced object removal.

View Source 
var ErrObjectIsExpired = logicerr.New("object is expired")

ErrObjectIsExpired is returned when the requested object's epoch is less than the current one. Such objects are considered as removed and should not be returned from the Storage Engine.

View Source 
var ErrObjectWasNotRemoved = logicerr.New("object neither in the graveyard nor was marked with GC mark")

ErrObjectWasNotRemoved is returned when object neither in the graveyard nor was marked with GC mark.

View Source 
var ErrOutdatedVersion = logicerr.New("invalid version, resynchronization is required")

ErrOutdatedVersion is returned on initializing an existing metabase that is not compatible with the current code version.

View Source 
var ErrReadOnlyMode = logicerr.New("metabase is in a read-only mode")

ErrReadOnlyMode is returned when metabase is in a read-only mode.

View Source 
var ErrReviveFromContainerGarbage = logicerr.New("revive from container marked with GC mark")

ErrReviveFromContainerGarbage is returned when the object is in the container that marked with GC mark.

Functions

func IsErrRemoved added in v0.28.0 

func IsErrRemoved(err error) bool

IsErrRemoved checks if error returned by Shard Exists/Get/Put method corresponds to removed object.

func PutMetadataForObject added in v0.45.2 

func PutMetadataForObject(tx *bbolt.Tx, hdr object.Object, phy bool) error

PutMetadataForObject fills object meta-data indexes using bbolt transaction. Transaction must be writable. Additional bucket for container's meta-data may be created using {255, CID...} form as a key.

Types

type BucketValue added in v0.43.0 

type BucketValue struct {
	BucketIndex int
	Value       []byte
}

BucketValue pairs a bucket index and a value that relates an object.

type ContainerInfo added in v0.49.1 

type ContainerInfo struct {
	StorageSize   uint64
	ObjectsNumber uint64
}

ContainerInfo groups metabase's objects status for a certain container.

type Containers added in v0.45.1 

type Containers interface {
	// Exists checks presence of the referenced container.
	Exists(cid.ID) (bool, error)
}

Containers provides access to information about NeoFS containers required for DB to process.

type Cursor added in v0.27.0 

type Cursor struct {
	// contains filtered or unexported fields
}

Cursor is a type for continuous object listing.

type DB 

type DB struct {
	// contains filtered or unexported fields
}

DB represents local metabase of storage node.

func New added in v0.13.0 

func New(opts ...Option) *DB

New creates and returns new Metabase instance.

func (*DB) Close 

func (db *DB) Close() error

Close closes boltDB instance.

func (*DB) Containers added in v0.13.0 

func (db *DB) Containers() (list []cid.ID, err error)

func (*DB) Delete 

func (db *DB) Delete(addrs []oid.Address) (DeleteRes, error)

Delete removes object records from metabase indexes. Does not stop on an error if there are more objects to handle requested; returns the first error appeared with a number of deleted objects wrapped.

func (*DB) DeleteContainer added in v0.40.0 

func (db *DB) DeleteContainer(cID cid.ID) error

DeleteContainer removes any information that the metabase has associated with the provided container (its objects) except the graveyard-related one.

func (*DB) DoNotMove added in v0.13.0 

func (db *DB) DoNotMove(addr oid.Address) error

DoNotMove removes `MoveIt` mark from the object.

func (*DB) DropExpiredTSMarks added in v0.44.0 

func (db *DB) DropExpiredTSMarks(epoch uint64) (int, error)

DropExpiredTSMarks run through the graveyard and drops tombstone marks with tombstones whose expiration is _less_ than provided epoch. Returns number of marks dropped.

func (*DB) DumpInfo added in v0.13.0 

func (db *DB) DumpInfo() Info

DumpInfo returns information about the DB.

func (*DB) Exists added in v0.13.0 

func (db *DB) Exists(addr oid.Address, ignoreExpiration bool) (bool, error)

Exists returns ErrAlreadyRemoved if addr was marked as removed. Otherwise it returns true if addr is in primary index or false if it is not.

Returns an error of type apistatus.ObjectAlreadyRemoved if object has been placed in graveyard. Returns the object.ErrObjectIsExpired if the object is presented but already expired.

If referenced object is a parent of some stored objects, Exists returns [ParentError] wrapping: - *objectSDK.SplitInfoError wrapping objectSDK.SplitInfo collected from stored parts; - [ErrParts] if referenced object is EC.

func (*DB) FilterExpired added in v0.38.0 

func (db *DB) FilterExpired(addresses []oid.Address) ([]oid.Address, error)

FilterExpired filters expired object from `addresses` and return them. Uses internal epoch state provided via the WithEpochState option.

func (*DB) FreeLockedBy added in v0.28.0 

func (db *DB) FreeLockedBy(lockers []oid.Address) ([]oid.Address, error)

FreeLockedBy unlocks all objects in DB which are locked by lockers. Returns unlocked objects if any.

func (*DB) Get 

func (db *DB) Get(addr oid.Address, raw bool) (*objectSDK.Object, error)

Get returns partial object header data for specified address (as stored in metabase). This header contains all user-defined attributes and mandatory header fields that can be used as search filters like size, checksum and owner. It does not contain parent header or session token.

"raw" flag controls virtual object processing, when false (default) a proper object header is returned, when true only SplitInfo of virtual object is returned.

Returns an error of type apistatus.ObjectNotFound if object is missing in DB. Returns an error of type apistatus.ObjectAlreadyRemoved if object has been placed in graveyard. Returns the object.ErrObjectIsExpired if the object is presented but already expired.

If raw and the object is a parent of some stored objects, Get returns: - objectSDK.SplitInfoError wrapping objectSDK.SplitInfo collected from parts if object is split; - iec.ErrPartitionedObject if object is EC.

func (*DB) GetContainerInfo added in v0.49.1 

func (db *DB) GetContainerInfo(id cid.ID) (ContainerInfo, error)

GetContainerInfo returns statistics about stored objects for specified container. If no info is found, empty ContainerInfo with no error are returned.

func (*DB) GetGarbage added in v0.40.0 

func (db *DB) GetGarbage(limit int) ([]oid.Address, []cid.ID, error)

GetGarbage returns garbage according to the metabase state. Garbage includes objects marked with GC mark (expired, tombstoned but not deleted from disk, extra replicated, etc.) and removed containers. The first return value describes garbage objects. These objects should be removed. The second return value describes garbage containers whose _all_ garbage objects were included in the first return value and, therefore, these containers can be deleted (if their objects are handled and deleted too).

func (*DB) Inhume added in v0.13.0 

func (db *DB) Inhume(tombstone oid.Address, tombExpiration uint64, handleLocks bool, addrs ...oid.Address) (uint64, []oid.Address, error)

Inhume marks objects as removed but not removes it from metabase.

Allows inhuming non-locked objects only. Returns apistatus.ObjectLocked if at least one object is locked. Returns ErrLockObjectRemoval if inhuming is being performed on lock (not locked) object.

Returns the number of available objects that were inhumed and a list of deleted LOCK objects (if handleLocks parameter is set).

func (*DB) InhumeContainer added in v0.40.0 

func (db *DB) InhumeContainer(cID cid.ID) (uint64, error)

InhumeContainer marks every object in a container as removed. Any further DB.Get calls will return apistatus.ObjectNotFound errors. Returns number of available objects marked with GC. There is no any LOCKs, forced GC marks and any relations checks, every object that belongs to a provided container will be marked as a removed one.

func (*DB) Init added in v0.13.0 

func (db *DB) Init() error

Init initializes metabase. It creates static (CID-independent) buckets in underlying BoltDB instance.

Returns ErrOutdatedVersion if a database at the provided path is outdated.

Does nothing if metabase has already been initialized and filled. To roll back the database to its initial state, use Reset.

func (*DB) IsLocked added in v0.36.0 

func (db *DB) IsLocked(addr oid.Address) (bool, error)

IsLocked checks is the provided object is locked by any `LOCK`. Not found object is considered as non-locked.

Returns only non-logical errors related to underlying database.

func (*DB) IterateExpired added in v0.16.0 

func (db *DB) IterateExpired(epoch uint64, h ExpiredObjectHandler) error

IterateExpired iterates over all objects in DB which are out of date relative to epoch. Locked objects are not included (do not confuse with objects of type LOCK).

If h returns ErrInterruptIterator, nil returns immediately. Returns other errors of h directly.

func (*DB) IterateOverGarbage added in v0.28.0 

func (db *DB) IterateOverGarbage(h GarbageHandler, offset *oid.Address) error

IterateOverGarbage iterates over all objects marked with GC mark.

The handler will be applied to the next after the specified offset if any are left.

Note: if offset is not found in db, iteration starts from the element that WOULD BE the following after the offset if offset was presented. That means that it is safe to delete offset element and pass if to the iteration once again: iteration would start from the next element.

Nil offset means start an integration from the beginning.

If h returns ErrInterruptIterator, nil returns immediately. Returns other errors of h directly.

func (*DB) IterateOverGraveyard added in v0.16.0 

func (db *DB) IterateOverGraveyard(h TombstonedHandler, offset *oid.Address) error

IterateOverGraveyard iterates over all graves in DB.

The handler will be applied to the next after the specified offset if any are left.

Note: if offset is not found in db, iteration starts from the element that WOULD BE the following after the offset if offset was presented. That means that it is safe to delete offset element and pass if to the iteration once again: iteration would start from the next element.

Nil offset means start an integration from the beginning.

If h returns ErrInterruptIterator, nil returns immediately. Returns other errors of h directly.

func (*DB) ListWithCursor added in v0.27.0 

func (db *DB) ListWithCursor(count int, cursor *Cursor, attrs ...string) ([]objectcore.AddressWithAttributes, *Cursor, error)

ListWithCursor lists physical objects available in metabase starting from cursor. Includes objects of all types. Does not include inhumed objects. Use cursor value from response for consecutive requests.

Optional attrs specifies attributes to include in the result. If object does not have requested attribute, corresponding element in the result is empty.

Returns ErrEndOfListing if there are no more objects to return or count parameter set to zero.

func (*DB) Lock added in v0.28.0 

func (db *DB) Lock(cnr cid.ID, locker oid.ID, locked []oid.ID) error

Lock marks objects as locked with another object. All objects are from the specified container.

Allows locking regular objects only (otherwise returns apistatus.LockNonRegularObject).

Locked list should be unique. Panics if it is empty.

Returns apistatus.ErrObjectAlreadyRemoved if there is an object of objectSDK.TypeTombstone type associated with the locked one.

func (*DB) MarkGarbage added in v0.44.0 

func (db *DB) MarkGarbage(force bool, handleLocks bool, addrs ...oid.Address) (uint64, []oid.Address, error)

MarkGarbage marks objects to be physically removed from shard. force flag allows to override any restrictions imposed on object deletion (to be used by control service and other manual intervention cases). Otherwise similar to DB.Inhume, but doesn't need a tombstone.

func (*DB) Movable added in v0.13.0 

func (db *DB) Movable() ([]oid.Address, error)

Movable returns list of marked objects to move into other shard.

func (*DB) ObjectCounters added in v0.32.0 

func (db *DB) ObjectCounters() (ObjectCounters, error)

ObjectCounters returns object counters that metabase has tracked since it was opened and initialized.

Returns only the errors that do not allow reading counter in Bolt database.

func (*DB) ObjectStatus added in v0.39.0 

func (db *DB) ObjectStatus(address oid.Address) (ObjectStatus, error)

ObjectStatus returns the status of the object in the Metabase. It contains state, path and indexed information about an object.

func (*DB) Open added in v0.13.0 

func (db *DB) Open(readOnly bool) error

Open boltDB instance for metabase.

func (*DB) Put 

func (db *DB) Put(obj *objectSDK.Object) error

Put updates metabase indexes for the given object.

Returns an error of type apistatus.ObjectAlreadyRemoved if object has been placed in graveyard. Returns the object.ErrObjectIsExpired if the object is presented but already expired.

Returns apistatus.ErrObjectAlreadyRemoved if obj is of objectSDK.TypeLock type and there is an object of objectSDK.TypeTombstone type associated with the same target.

func (*DB) ReadLastResyncEpoch added in v0.44.0 

func (db *DB) ReadLastResyncEpoch() (epoch uint64, err error)

ReadLastResyncEpoch reads from db last epoch when metabase was resynchronized. If id is missing, returns 0, nil.

func (*DB) ReadShardID added in v0.27.7 

func (db *DB) ReadShardID() ([]byte, error)

ReadShardID reads shard id from db. If id is missing, returns nil, nil.

func (*DB) Reload added in v0.34.0 

func (db *DB) Reload(opts ...Option) (bool, error)

Reload reloads part of the configuration. It returns true iff database was reopened. If a config option is invalid, it logs an error and returns nil. If there was a problem with applying new configuration, an error is returned.

If a metabase was couldn't be reopened because of an error, ErrDegradedMode is returned.

func (*DB) Reset added in v0.25.0 

func (db *DB) Reset() error

Reset resets metabase. Works similar to Init but cleans up all static buckets and removes all dynamic (CID-dependent) ones in non-blank BoltDB instances.

func (*DB) ResolveECPart added in v0.49.1 

func (db *DB) ResolveECPart(cnr cid.ID, parent oid.ID, pi iec.PartInfo) (oid.ID, error)

ResolveECPart resolves object that carries EC part produced within cnr for parent object and indexed by pi, checks its availability and returns its ID.

If the object is not EC part but of [object.TypeTombstone] or [object.TypeLock] type, ResolveECPart returns its ID instead.

If DB is disabled by mode (e.g. DB.SetMode), ResolveECPart returns ErrDegradedMode.

If object has expired, ResolveECPart returns ErrObjectIsExpired.

If object exists but tombstoned (via DB.Inhume or stored tombstone object), ResolveECPart returns apistatus.ErrObjectAlreadyRemoved.

If object is marked as garbage (via DB.MarkGarbage), ResolveECPart returns apistatus.ErrObjectNotFound.

If object is locked (via DB.Lock or stored locker object), ResolveECPart ignores expiration, tombstone and garbage marks.

func (*DB) ReviveObject added in v0.44.0 

func (db *DB) ReviveObject(addr oid.Address) (res ReviveStatus, err error)

ReviveObject revives object by oid.Address. Removes GCMark/Tombstone records in the corresponding buckets and restore metrics.

func (*DB) Search added in v0.45.0 

func (db *DB) Search(cnr cid.ID, fs []objectcore.SearchFilter, attrs []string, cursor *objectcore.SearchCursor, count uint16) ([]client.SearchResultItem, []byte, error)

Search selects up to count container's objects from the given container matching the specified filters.

func (*DB) Select 

func (db *DB) Select(cnr cid.ID, filters object.SearchFilters) ([]oid.Address, error)

Select returns list of addresses of objects that match search filters.

Only creation epoch, payload size, user attributes and unknown system ones are allowed with numeric operators. Values of numeric filters must be base-10 integers.

Returns [object.ErrInvalidSearchQuery] if specified query is invalid.

func (*DB) SetLogger added in v0.30.0 

func (db *DB) SetLogger(l *zap.Logger)

SetLogger sets logger. It is used after the shard ID was generated to use it in logs.

func (*DB) SetMode added in v0.30.0 

func (db *DB) SetMode(m mode.Mode) error

SetMode sets the metabase mode of operation. If the mode assumes no operation metabase, the database is closed.

func (*DB) SyncCounters added in v0.32.0 

func (db *DB) SyncCounters() error

SyncCounters forces to synchronize the object counters.

func (*DB) ToMoveIt added in v0.13.0 

func (db *DB) ToMoveIt(addr oid.Address) error

ToMoveIt marks objects to move it into another shard. This useful for faster HRW fetching.

func (*DB) WriteShardID added in v0.27.7 

func (db *DB) WriteShardID(id []byte) error

WriteShardID writes shard it to db.

type DeleteRes added in v0.13.0 

type DeleteRes struct {
	// RawRemoved contains the number of removed raw objects.
	RawRemoved uint64
	// AvailableRemoved contains the number of removed available objects.
	AvailableRemoved uint64
	// Sizes contains the sizes of removed objects.
	// The order of the sizes is the same as in addresses'
	// slice that was provided in the [DB.Delete] address list,
	// meaning that i-th size equals the number of freed up bytes
	// after removing an object by i-th address. A zero size is
	// allowed, it claims a missing object.
	Sizes []uint64
}

DeleteRes groups the resulting values of Delete operation.

type EpochState added in v0.31.0 

type EpochState interface {
	// CurrentEpoch must return current epoch height.
	CurrentEpoch() uint64
}

EpochState is an interface that provides access to the current epoch number.

type ExpiredObject added in v0.16.0 

type ExpiredObject struct {
	// contains filtered or unexported fields
}

ExpiredObject is a descriptor of expired object from DB.

func (*ExpiredObject) Address added in v0.16.0 

func (e *ExpiredObject) Address() oid.Address

Address returns address of the expired object.

func (*ExpiredObject) Type added in v0.16.0 

func (e *ExpiredObject) Type() object.Type

Type returns type of the expired object.

type ExpiredObjectHandler added in v0.16.0 

type ExpiredObjectHandler func(*ExpiredObject) error

ExpiredObjectHandler is an ExpiredObject handling function.

type GarbageHandler added in v0.28.0 

type GarbageHandler func(GarbageObject) error

GarbageHandler is a GarbageObject handling function.

type GarbageObject added in v0.28.0 

type GarbageObject struct {
	// contains filtered or unexported fields
}

GarbageObject represents descriptor of the object that has been marked with GC.

func (GarbageObject) Address added in v0.28.0 

func (g GarbageObject) Address() oid.Address

Address returns garbage object address.

type HeaderField added in v0.46.0 

type HeaderField struct {
	K []byte
	V []byte
}

HeaderField is object header's field index.

type Info added in v0.13.0 

type Info struct {
	// Full path to the metabase.
	Path string

	// Permission of database file.
	Permission fs.FileMode

	// LastResyncEpoch is a last epoch when metabase was resynchronized.
	LastResyncEpoch uint64
}

Info groups the information about DB.

type ObjectCounters added in v0.32.0 

type ObjectCounters struct {
	// contains filtered or unexported fields
}

ObjectCounters groups object counter according to metabase state.

func (ObjectCounters) Logic added in v0.32.0 

func (o ObjectCounters) Logic() uint64

Logic returns logical object counter.

func (ObjectCounters) Phy added in v0.32.0 

func (o ObjectCounters) Phy() uint64

Phy returns physical object counter.

type ObjectStatus added in v0.39.0 

type ObjectStatus struct {
	Version     uint64
	Buckets     []BucketValue
	HeaderIndex []HeaderField
	State       []string
	Path        string
	Error       error
}

ObjectStatus represents the status of the object in the Metabase.

type Option 

type Option func(*cfg)

Option is an option of DB constructor.

func WithBoltDBOptions added in v0.13.0 

func WithBoltDBOptions(opts *bbolt.Options) Option

WithBoltDBOptions returns option to specify BoltDB options.

func WithContainers added in v0.45.1 

func WithContainers(cs Containers) Option

WithContainers return option to specify container source.

func WithEpochState added in v0.31.0 

func WithEpochState(s EpochState) Option

WithEpochState return option to specify a source of current epoch height.

func WithInitContext added in v0.45.1 

func WithInitContext(ctx context.Context) Option

WithInitContext allows to specify context for DB.Init operation.

func WithLogger 

func WithLogger(l *zap.Logger) Option

WithLogger returns option to set logger of DB.

func WithMaxBatchDelay added in v0.29.0 

func WithMaxBatchDelay(d time.Duration) Option

WithMaxBatchDelay returns option to specify maximum time to wait before the batch of concurrent transactions is processed. This option is missing from `bbolt.Options` but is set right after DB is open.

func WithMaxBatchSize added in v0.29.0 

func WithMaxBatchSize(s int) Option

WithMaxBatchSize returns option to specify maximum concurrent operations to be processed in a single transactions. This option is missing from `bbolt.Options` but is set right after DB is open.

func WithPath added in v0.13.0 

func WithPath(path string) Option

WithPath returns option to set system path to Metabase.

func WithPermissions added in v0.13.0 

func WithPermissions(perm fs.FileMode) Option

WithPermissions returns option to specify permission bits of Metabase system path.

type ReviveStatus added in v0.44.0 

type ReviveStatus struct {
	// contains filtered or unexported fields
}

ReviveStatus groups the resulting values of ReviveObject operation. Contains the type of revival status and message for details.

func (*ReviveStatus) Message added in v0.44.0 

func (s *ReviveStatus) Message() string

Message returns message of status.

func (*ReviveStatus) StatusType added in v0.44.0 

func (s *ReviveStatus) StatusType() reviveStatusType

StatusType returns the type of revival status.

func (*ReviveStatus) TombstoneAddress added in v0.49.1 

func (s *ReviveStatus) TombstoneAddress() oid.Address

TombstoneAddress returns the tombstone address.

type TombstonedHandler added in v0.28.0 

type TombstonedHandler func(object TombstonedObject) error

TombstonedHandler is a TombstonedObject handling function.

type TombstonedObject added in v0.28.0 

type TombstonedObject struct {
	// contains filtered or unexported fields
}

TombstonedObject represents descriptor of the object that has been covered with tombstone.

func (TombstonedObject) Address added in v0.28.0 

func (g TombstonedObject) Address() oid.Address

Address returns tombstoned object address.

func (TombstonedObject) Tombstone added in v0.28.0 

func (g TombstonedObject) Tombstone() oid.Address

Tombstone returns address of a tombstone that covers object.

func (TombstonedObject) TombstoneExpiration added in v0.44.0 

func (g TombstonedObject) TombstoneExpiration() uint64

TombstoneExpiration returns tombstone's expiration. It can be zero if metabase version does not support expiration indexing or if TS does not expire.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.