replication

package
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 10, 2020 License: GPL-3.0 Imports: 14 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Address

type Address = refs.Address

Address is a type alias of Address from refs package of neofs-api-go.

type AddressStore

type AddressStore interface {
	SelfAddr() (multiaddr.Multiaddr, error)
}

AddressStore is an interface of local peer's network address storage.

type CID

type CID = refs.CID

CID is a type alias of CID from refs package of neofs-api-go.

type ContainerActualityChecker

type ContainerActualityChecker interface {
	Actual(ctx context.Context, cid CID) bool
}

ContainerActualityChecker is an interface of entity for checking local node presence in container Return true if no errors && local node is in container

type EpochReceiver

type EpochReceiver interface {
	Epoch() uint64
}

EpochReceiver is an interface of entity for getting current epoch number.

type LocationDetectorParams

type LocationDetectorParams struct {
	WeightComparator
	ObjectLocator
	ReservationRatioReceiver
	PresenceChecker
	*zap.Logger

	TaskChanCap   int
	ResultTimeout time.Duration
}

LocationDetectorParams groups the parameters of location detector's constructor.

type Manager

type Manager interface {
	Process(ctx context.Context)
	HandleEpoch(ctx context.Context, epoch uint64)
}

Manager is an interface of object manager,

func NewManager

func NewManager(p ManagerParams) (Manager, error)

NewManager is an object manager's constructor.

type ManagerParams

type ManagerParams struct {
	Interval                time.Duration
	PushTaskTimeout         time.Duration
	PlacementHonorerEnabled bool
	ReplicateTaskChanCap    int
	RestoreTaskChanCap      int
	GarbageChanCap          int
	InitPoolSize            int
	ExpansionRate           float64

	ObjectPool
	ObjectVerifier

	PlacementHonorer
	ObjectLocationDetector
	StorageValidator
	ObjectReplicator
	ObjectRestorer

	*zap.Logger

	Scheduler
}

ManagerParams groups the parameters of object manager's constructor.

type MultiSolver

MultiSolver is an interface that encapsulates other different utilities.

func NewMultiSolver

func NewMultiSolver(p MultiSolverParams) (MultiSolver, error)

NewMultiSolver is a multi solver constructor.

type MultiSolverParams

type MultiSolverParams struct {
	AddressStore
	Placement placement.Component
}

MultiSolverParams groups the parameters of multi solver constructor.

type Object

type Object = object.Object

Object is a type alias of Object from object package of neofs-api-go.

type ObjectCleaner

type ObjectCleaner interface {
	Del(Address) error
}

ObjectCleaner Entity for removing object by address from somewhere

type ObjectLocation

type ObjectLocation struct {
	Node          multiaddr.Multiaddr
	WeightGreater bool // true if Node field value has less index in placement vector than localhost
}

ObjectLocation groups the information about object current remote location.

type ObjectLocationDetector

type ObjectLocationDetector interface {
	Process(ctx context.Context) chan<- Address
	Subscribe(ch chan<- *ObjectLocationRecord)
}

ObjectLocationDetector is an interface of entity that listens tasks to detect object current locations in network.

func NewLocationDetector

func NewLocationDetector(p *LocationDetectorParams) (ObjectLocationDetector, error)

NewLocationDetector is an object location detector's constructor.

type ObjectLocationRecord

type ObjectLocationRecord struct {
	Address
	ReservationRatio int
	Locations        []ObjectLocation
}

ObjectLocationRecord groups the information about all current locations.

type ObjectLocator

type ObjectLocator interface {
	LocateObject(ctx context.Context, objAddr Address) ([]multiaddr.Multiaddr, error)
}

ObjectLocator is an itnerface of entity for building list current object remote nodes by address

type ObjectPool

type ObjectPool interface {
	Update([]Address)
	Pop() (Address, error)
	Undone() int
}

ObjectPool is a queue of objects selected for data audit. It is updated once in epoch.

func NewObjectPool

func NewObjectPool() ObjectPool

NewObjectPool is an object pool constructor.

type ObjectReceptacle

type ObjectReceptacle interface {
	Put(ctx context.Context, params ObjectStoreParams) error
}

ObjectReceptacle is an interface of object storage with write access.

type ObjectReplicator

type ObjectReplicator interface {
	Process(ctx context.Context) chan<- *ReplicateTask
	Subscribe(ch chan<- *ReplicateResult)
}

ObjectReplicator is an interface of entity that listens object replication tasks. Result includes new object storage list.

func NewReplicator

func NewReplicator(p ObjectReplicatorParams) (ObjectReplicator, error)

NewReplicator is an object replicator's constructor.

type ObjectReplicatorParams

type ObjectReplicatorParams struct {
	RemoteStorageSelector
	ObjectSource
	ObjectReceptacle
	PresenceChecker
	*zap.Logger

	TaskChanCap   int
	ResultTimeout time.Duration
}

ObjectReplicatorParams groups the parameters of replicator's constructor.

type ObjectRestorer

type ObjectRestorer interface {
	Process(ctx context.Context) chan<- Address
	Subscribe(ch chan<- Address)
}

ObjectRestorer is an interface of entity that listen tasks to restore object by address. Restorer doesn't recheck if object is actually corrupted. Restorer writes result to subscriber only if restoration was successful.

func NewObjectRestorer

func NewObjectRestorer(p *ObjectRestorerParams) (ObjectRestorer, error)

NewObjectRestorer is an object restorer's constructor.

type ObjectRestorerParams

type ObjectRestorerParams struct {
	ObjectVerifier
	ObjectReceptacle
	EpochReceiver
	RemoteStorageSelector
	PresenceChecker
	*zap.Logger

	TaskChanCap   int
	ResultTimeout time.Duration
}

ObjectRestorerParams groups the parameters of object restorer's constructor.

type ObjectSource

type ObjectSource interface {
	Get(ctx context.Context, addr Address) (*Object, error)
}

ObjectSource is an interface of the object storage with read access.

type ObjectStoreParams

type ObjectStoreParams struct {
	*Object
	Nodes   []ObjectLocation
	Handler func(ObjectLocation, bool)
}

ObjectStoreParams groups the parameters for object storing.

type ObjectVerificationParams

type ObjectVerificationParams struct {
	Address
	Node         multiaddr.Multiaddr
	Handler      func(valid bool, obj *Object)
	LocalInvalid bool
}

ObjectVerificationParams groups the parameters of stored object verification.

type ObjectVerifier

type ObjectVerifier interface {
	Verify(ctx context.Context, params *ObjectVerificationParams) bool
}

ObjectVerifier is an interface of stored object verifier.

type OwnerID

type OwnerID = object.OwnerID

OwnerID is a type alias of OwnerID from object package of neofs-api-go.

type PlacementHonorer

type PlacementHonorer interface {
	Process(ctx context.Context) chan<- *ObjectLocationRecord
	Subscribe(ch chan<- *ObjectLocationRecord)
}

PlacementHonorer is an interface of entity that listens tasks to piece out placement rule of container for particular object.

func NewPlacementHonorer

func NewPlacementHonorer(p PlacementHonorerParams) (PlacementHonorer, error)

NewPlacementHonorer is a placement honorer's constructor.

type PlacementHonorerParams

type PlacementHonorerParams struct {
	ObjectSource
	ObjectReceptacle
	RemoteStorageSelector
	PresenceChecker
	*zap.Logger

	TaskChanCap   int
	ResultTimeout time.Duration
}

PlacementHonorerParams groups the parameters of placement honorer's constructor.

type PresenceChecker

type PresenceChecker interface {
	Has(address Address) (bool, error)
}

PresenceChecker is an interface of object storage with presence check access.

type RemoteStorageSelector

type RemoteStorageSelector interface {
	SelectRemoteStorages(ctx context.Context, addr Address, excl ...multiaddr.Multiaddr) ([]ObjectLocation, error)
}

RemoteStorageSelector is an interface of entity for getting remote nodes from placement for object by address Result doesn't contain nodes from exclude list

type ReplicateResult

type ReplicateResult struct {
	*ReplicateTask
	NewStorages []multiaddr.Multiaddr
}

ReplicateResult groups the information about object replication task result.

type ReplicateTask

type ReplicateTask struct {
	Address
	Shortage     int
	ExcludeNodes []multiaddr.Multiaddr
}

ReplicateTask groups the information about object replication task. Task solver should not process nodes from exclude list, Task solver should perform up to Shortage replications.

type ReservationRatioReceiver

type ReservationRatioReceiver interface {
	ReservationRatio(ctx context.Context, objAddr Address) (int, error)
}

ReservationRatioReceiver is an interface of entity for getting reservation ratio value of object by address.

type Scheduler

type Scheduler interface {
	SelectForReplication(limit int) ([]Address, error)
}

Scheduler returns slice of addresses for data audit. These addresses put into ObjectPool.

func NewReplicationScheduler

func NewReplicationScheduler(p SchedulerParams) (Scheduler, error)

NewReplicationScheduler is a replication scheduler constructor.

type SchedulerParams

type SchedulerParams struct {
	ContainerActualityChecker
	localstore.Iterator
}

SchedulerParams groups the parameters of scheduler constructor.

type StorageValidator

type StorageValidator interface {
	Process(ctx context.Context) chan<- *ObjectLocationRecord
	SubscribeReplication(ch chan<- *ReplicateTask)
	SubscribeGarbage(ch chan<- Address)
}

StorageValidator is an interface of entity that listens and performs task of storage validation on remote nodes. Validation can result to the need to replicate or clean object.

func NewStorageValidator

func NewStorageValidator(p StorageValidatorParams) (StorageValidator, error)

NewStorageValidator is a storage validator's constructor.

type StorageValidatorParams

type StorageValidatorParams struct {
	ObjectVerifier
	PresenceChecker
	*zap.Logger

	TaskChanCap   int
	ResultTimeout time.Duration
	AddrStore     AddressStore
}

StorageValidatorParams groups the parameters of storage validator's constructor.

type WeightComparator

type WeightComparator interface {
	CompareWeight(ctx context.Context, addr Address, node multiaddr.Multiaddr) int
}

WeightComparator is an itnerface of entity for comparing weight by address of local node with passed node returns -1 if local node is weightier or on error returns 0 if weights are equal returns 1 if passed node is weightier

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL