proc

package
v1.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 9, 2019 License: AGPL-3.0 Imports: 18 Imported by: 0

Documentation

Overview

Package proc provides actual actions to be performed once the sync has filtered all events

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func MergeNodes 

func MergeNodes(left *TreeNode, right *TreeNode, diff *SourceDiff)

MergeNodes will recursively detect differences between two hash trees.

Types

type ChildrenCursor 

type ChildrenCursor struct {
	// contains filtered or unexported fields
}

ChildrenCursor provides a Nexter for browsing a node children

func (*ChildrenCursor) Next 

func (c *ChildrenCursor) Next() *TreeNode

Next sends the next child or nil

type Merger 

type Merger struct {
	BatchesChannel  chan *filters.Batch
	RequeueChannels map[PathSyncSource]chan EventInfo
	LocksChannel    chan filters.LockEvent
	UnlocksChannel  chan filters.UnlockEvent
	JobsInterrupt   chan bool
	GlobalContext   context.Context
	// contains filtered or unexported fields
}

func NewMerger 

func NewMerger(ctx context.Context) *Merger

func (*Merger) AddRequeueChannel 

func (b *Merger) AddRequeueChannel(source PathSyncSource, channel chan EventInfo)

func (*Merger) Logger 

func (b *Merger) Logger() *zap.Logger

func (*Merger) ProcessBatches 

func (b *Merger) ProcessBatches()

func (*Merger) RegisterEventChannel 

func (b *Merger) RegisterEventChannel(out chan ProcessorEvent)

func (*Merger) Shutdown 

func (b *Merger) Shutdown()

type ProcessFunc 

type ProcessFunc func(event *filters.BatchedEvent, operationId string) error

type SourceDiff 

type SourceDiff struct {
	Left         common.PathSyncSource
	Right        common.PathSyncSource
	MissingLeft  []*tree.Node
	MissingRight []*tree.Node
	Context      context.Context
}

SourceDiff represents basic differences between two sources. It can be then transformed to Batch, depending on the sync being unidirectional (transform to Creates and Deletes) or bidirectional (transform only to Creates)

func ComputeSourcesDiff 

func ComputeSourcesDiff(ctx context.Context, left common.PathSyncSource, right common.PathSyncSource, strong bool, statusChan chan filters.BatchProcessStatus) (diff *SourceDiff, err error)

ComputeSourceDiff loads the diff by crawling the sources in parallel, filling up a Hash Tree and performing the merge.

func (*SourceDiff) FilterMissing 

func (diff *SourceDiff) FilterMissing(source common.PathSyncSource, target common.PathSyncTarget, in []*tree.Node, folders bool, removes bool) (out map[string]*filters.BatchedEvent)

FilterMissing transforms Missing slices to BatchEvents

func (*SourceDiff) Stats 

func (diff *SourceDiff) Stats() map[string]int

func (*SourceDiff) String 

func (diff *SourceDiff) String() string

String provides a string representation of this diff

func (*SourceDiff) ToBidirectionalBatch 

func (diff *SourceDiff) ToBidirectionalBatch(leftTarget common.PathSyncTarget, rightTarget common.PathSyncTarget) (batch *filters.BidirectionalBatch, err error)

ToBidirectionalBatch transforms this diff to a batch

func (*SourceDiff) ToUnidirectionalBatch 

func (diff *SourceDiff) ToUnidirectionalBatch(direction string) (batch *filters.Batch, err error)

ToUnidirectionalBatch transforms this diff to a batch

type TreeNode 

type TreeNode struct {
	tree.Node
	sync.Mutex
	// contains filtered or unexported fields
}

TreeNode builds a Merkle Tree but with N children and the ability to compute the COLLECTION Nodes hashes to detect changes in branches more rapidly

func NewTreeNode 

func NewTreeNode(n *tree.Node) *TreeNode

NewTreeNode creates a new node from a tree.Node. Can be a root, a COLL or a LEAF.

func TreeNodeFromSource 

func TreeNodeFromSource(source common.PathSyncSource) (*TreeNode, error)

TreeNodeFromSource populates a hash tree with leafs and folders by walking a source. When it comes accross a LEAF without Etag value, it asks the source to recompute it in a parallel fashion with throttling (max 15 at the same time). At the end of the operation, the tree should be fully loaded with all LEAF etags (but not COLL etags).

func (*TreeNode) AddChild 

func (t *TreeNode) AddChild(n *TreeNode)

AddChild appends a child to the children map (with lock)

func (*TreeNode) Enqueue 

func (t *TreeNode) Enqueue(missings []*tree.Node) []*tree.Node

Enqueue recursively appends al tree.Node and the children's tree.Node to a slice

func (*TreeNode) GetCursor 

func (t *TreeNode) GetCursor() *ChildrenCursor

GetCursor gives a cursor to crawl the current node children

func (*TreeNode) GetHash 

func (t *TreeNode) GetHash() string

GetHash returns the Etag of the node. For leaf it should be available, for Folders if it is not already computed, it will compute an etag from the children recursively, using their name and Etag.

func (*TreeNode) GetLevel 

func (t *TreeNode) GetLevel() int

GetLevel computes the current level of this node (depth)

func (*TreeNode) Label 

func (t *TreeNode) Label() string

Label returns the basename of the path

func (*TreeNode) ParentPath 

func (t *TreeNode) ParentPath() string

ParentPath returns the parent Dir path

func (*TreeNode) PrintOut 

func (t *TreeNode) PrintOut()

PrintOut sends to fmt.Println a tree version of this node

func (*TreeNode) SortedChildren 

func (t *TreeNode) SortedChildren() []*TreeNode

SortedChildren sorts children by their labels. An internal flag avoids resorting if it was already sorted once.

Source Files

  • hash-tree-merger.go
  • hash-tree.go
  • merger-create-file.go
  • merger-create-folder.go
  • merger-delete.go
  • merger-move.go
  • merger-refresh-files.go
  • merger.go
  • snapshot_diff.go

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.