volume

package
v0.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 28, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// TickerDuration is the duration of the internal loop of the volume
	// to recalculate the state
	TickerDuration = 30 * time.Second
)

Variables

View Source
var ErrClusterFull = errors.New("cluster has no available storage capacity")

ErrClusterFull is returned by storing.Service.CreateFile when no volume in the cluster has enough capacity for the incoming file. It is defined here (rather than in package storing) to avoid a test-only import cycle: storing/replica_test.go (package storing) imports storing/transport/http, which would cycle back if transport imported storing.

View Source
var ErrNoSpace = errors.New("no space left on volume")

ErrNoSpace is returned by CreateFile when the volume has no capacity for the incoming file.

Functions

This section is empty.

Types

type FileStat added in v0.5.0

type FileStat struct {
	Size     int64
	ETag     string
	ModTime  time.Time
	VolumeID string
}

FileStat holds metadata about a stored file, used for S3-compatible HEAD responses.

type Local

type Local interface {
	Volume

	// Close will try to make a clean shutdown
	io.Closer

	// ID returns the ID of the Volume
	ID() string

	// NextReplica returns the next replica
	// inline. A "not found" error means
	// no replica is needed
	NextReplica(ctx context.Context) (*replica.Replica, error)

	// UpdateReplica updates the rp of the index and the File to include
	// the vID as a volume with the Replica
	UpdateReplica(ctx context.Context, rp *replica.Replica, vID string) error

	// DeleteReplica removes a pending replication job from the queue.
	// Used to clean up stale jobs when the source file no longer exists.
	DeleteReplica(ctx context.Context, rp *replica.Replica) error

	// NextDeletion returns the next pending remote-deletion job.
	// A "not found" error means the queue is empty.
	NextDeletion(ctx context.Context) (*deletion.Deletion, error)

	// DeleteDeletion removes a processed deletion job from the queue.
	DeleteDeletion(ctx context.Context, d *deletion.Deletion) error

	// NextScrub returns the next pending scrub-repair job.
	// A "not found" error means the queue is empty.
	NextScrub(ctx context.Context) (*scrub.Scrub, error)

	// DeleteScrub removes a processed scrub-repair job from the queue.
	DeleteScrub(ctx context.Context, s *scrub.Scrub) error

	// OverwriteFileContent replaces the on-disk content for sig by streaming
	// from r, verifying the SHA1 matches sig after writing.
	OverwriteFileContent(ctx context.Context, sig string, r io.Reader) error

	// SynchronizeReplicas checks the replicas related with vID and
	// if this volume is the responsible (next after the removed ID on the files)
	// will start replication of those files which have to
	SynchronizeReplicas(ctx context.Context, vID string) error

	// GetState returns the current State of the volume
	GetState(ctx context.Context) (*state.State, error)

	// Reset will clean all the data of the volume and even change the ID
	Reset(ctx context.Context) error

	// PrepareForDrain creates Replica jobs for all local files that don't have
	// enough external replicas to cover the configured replica count.
	PrepareForDrain(ctx context.Context) error

	// HasPendingReplicas reports whether there are any pending replica jobs queued.
	HasPendingReplicas(ctx context.Context) (bool, error)

	// PurgeAllFiles removes all local files and their keys from this volume
	// without creating deletion jobs for remote nodes (remote copies are preserved).
	PurgeAllFiles(ctx context.Context) error

	// ReconcileReplicas clears the stale replica queue and rebuilds it from
	// the current file state. It enqueues a replica job for every locally-owned
	// file (VolumeIDs[0] == l.ID()) that has fewer copies than required.
	// Call this once at startup after a crash (non-reset path).
	ReconcileReplicas(ctx context.Context) error

	// AllFiles returns all file records stored on this volume.
	AllFiles(ctx context.Context) ([]*file.File, error)

	// FileVolumeIDs returns the VolumeIDs and replica count for the file with the given key.
	FileVolumeIDs(ctx context.Context, key string) ([]string, int, error)

	// PurgeFile removes the local file and index entries for key without
	// creating deletion jobs for remote nodes (unlike DeleteFile).
	PurgeFile(ctx context.Context, key string) error
}

Local is the definition of a Local volume which is an extension of the volume.Volume

func New

func New(root string, fileSystem afero.Fs, logger *slog.Logger, suow uow.StartUnitOfWork, scrubInterval time.Duration, replicaCheckInterval time.Duration) (Local, error)

New returns an implementation of the volume.Local interface using the provided parameters it can return an error because when initialized it also creates the needed directories if they are missing which are $root/file and $root/tmps and also the ID To define a total size of the volume it has to be appended to the root like `/v1:1GB`

type Volume

type Volume interface {
	// CreateFile creates a new file from the reader with the key, ttl and time of creation (if empty will be set to now)
	// There are 4 different use cases to consider:
	// * New key and reader
	// * New key with already known reader
	// * Already known key with new reader
	// * Already known key and reader
	CreateFile(ctx context.Context, key string, reader io.ReadCloser, replica int, ttl time.Duration, ca time.Time) error

	// GetFile search for the file with the key
	GetFile(ctx context.Context, key string) (io.ReadCloser, int64, error)

	// StatFile returns metadata about the file without reading its content.
	StatFile(ctx context.Context, key string) (*FileStat, error)

	// HasFile checks if a file with the key exists and returns the volumeID
	// of where is it.
	// It's possible to return a vid but false that means we know which volume
	// has it but it's not this one
	HasFile(ctx context.Context, key string) (string, bool, error)

	// DeleteFile deletes the key, if the key points to a
	// file with 2 keys, then just the key will be deleted
	// and not the content
	DeleteFile(ctx context.Context, key string) error

	// UpdateFileReplica updates the Replica information of the file
	// with the given one basically replacing it
	UpdateFileReplica(ctx context.Context, key string, volumeIDs []string, replica int) error
}

Volume is an interface to deal with the simples actions and basic ones

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL