fsops

package
v1.32.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 13, 2026 License: GPL-2.0 Imports: 12 Imported by: 0

Documentation

Overview

Package fsops defines the Backend interface that abstracts filesystem operations for qui's services. Current implementations: Local (delegates to os.* on the qui host), Remote (SFTP over the instance's pooled SSH connection; reads only for now) and Noop (returns ErrNoFilesystemAccess for instances without filesystem access). Services resolve their backend per instance through Pool.GetBackend, so the transport is transparent to them.

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrNoFilesystemAccess is returned by the NoopBackend for instances that
	// have no filesystem access configured (neither local nor remote).
	ErrNoFilesystemAccess = errors.New("filesystem access is not configured for this instance")

	// ErrNotLocal is returned by Pool.LocalBackend for an instance that no
	// longer has local filesystem access.
	ErrNotLocal = errors.New("instance does not have local filesystem access")

	// ErrNotCapable is returned by Pool.Require for an instance whose
	// filesystem mode does not grant the capability asked for.
	ErrNotCapable = errors.New("instance filesystem mode does not grant this capability")

	// ErrRemoteBackendNotWired is returned by a Pool built without a remote
	// factory for an instance in remote mode: a wiring mistake must fail
	// loudly rather than read as "not configured".
	ErrRemoteBackendNotWired = errors.New("remote filesystem backend is not wired into this pool")

	// ErrConnectionLost marks a remote read that failed because the backend
	// could not be reached (the transport dropped, or the pool would not
	// dial), not because of the path. It ends a walk as its one Err entry, and
	// a consumer that skips per-directory errors checks for it so that a walk
	// cut short is not read as complete. The cause is kept as text only, apart
	// from sshpool's own sentinels, so a lost connection never also matches
	// fs.ErrPermission or fs.ErrNotExist.
	ErrConnectionLost = errors.New("connection to the filesystem backend was lost")

	// ErrUnsupported reports a per-host fact: this server lacks the extension
	// the operation needs, or this transport has no such operation at all.
	// Distinct from ErrNoFilesystemAccess, which means nothing was configured.
	ErrUnsupported = fmt.Errorf("operation is not supported by this filesystem backend: %w", errors.ErrUnsupported)
)

Sentinel errors returned by Backend implementations.

Functions

This section is empty.

Types

type Backend

type Backend interface {

	// Stat returns metadata for a single path, following symlinks — FileID
	// and Nlinks describe the target, so symlinked torrent data keeps its
	// identity. Returns a non-nil error wrapping fs.ErrNotExist if the path
	// does not exist. A failure to resolve identity is reported in
	// LstatInfo.FileIDErr, not as a Stat error, so one identity-opaque file
	// cannot fail callers that only need metadata.
	Stat(ctx context.Context, path string) (*LstatInfo, error)

	// Lstat is like Stat but does not follow symlinks.
	Lstat(ctx context.Context, path string) (*LstatInfo, error)

	// ReadDir returns directory entries.
	ReadDir(ctx context.Context, path string) ([]DirEntry, error)

	// WalkDir walks a directory tree and streams entries on the returned channel.
	// Entries within one directory arrive in lexical order and a directory's
	// own entry precedes everything beneath it; nothing more is promised about
	// order, as a backend may list sibling directories concurrently. The
	// channel is closed when the walk completes, is cancelled via ctx, or
	// hits an unrecoverable error. Callers must drain the channel or cancel
	// ctx; abandoning it leaks the walk goroutine. Entries whose metadata
	// cannot be read are skipped, not emitted — WalkEntry.Err carries only
	// enumeration-level walk failures. Setting opts.EmitStatErrors emits
	// those entries instead, with WalkEntry.StatErr set and only Path/RelPath
	// valid, so delete preflights can fail closed on unverifiable paths.
	WalkDir(ctx context.Context, root string, opts WalkOptions) (<-chan WalkEntry, error)

	// Statfs returns free/total bytes for the filesystem containing path.
	Statfs(ctx context.Context, path string) (*StatfsResult, error)

	// SameFilesystem returns true if both paths reside on the same filesystem
	// (same device ID on Unix, same volume serial on Windows).
	SameFilesystem(ctx context.Context, p1, p2 string) (bool, error)

	// MkdirAll creates a directory and all parents. Equivalent to os.MkdirAll.
	// For torrent content and link-tree dirs, pass fsutil.ContentDirMode /
	// fsutil.LinkTreeBaseDirMode rather than a hand-typed mode (#1704, #2086).
	MkdirAll(ctx context.Context, path string, perm fs.FileMode) error

	// Remove removes a file or directory. If opts.Recursive is true, removes
	// the entire tree (like os.RemoveAll).
	Remove(ctx context.Context, path string, opts RemoveOptions) error

	// HardlinkTree creates a hardlink tree from plan. Rolls back what it
	// created on partial failure. The result records the files and dirs this
	// call made, for a later RemoveTree.
	HardlinkTree(ctx context.Context, plan *hardlinktree.TreePlan) (*TreeCreateResult, error)

	// ReflinkTree creates a reflink (CoW) tree from plan. Rolls back what it
	// created on partial failure. The result records the files and dirs this
	// call made, for a later RemoveTree.
	ReflinkTree(ctx context.Context, plan *hardlinktree.TreePlan) (*TreeCreateResult, error)

	// RemoveTree removes exactly the files and dirs recorded in created —
	// never the whole plan, which could delete links shared with sibling
	// torrents (discussion #2282). A plan root that already existed before
	// the create is therefore NOT removed; callers own pruning it. Safe to
	// call with a nil result.
	RemoveTree(ctx context.Context, created *TreeCreateResult) error

	// Paths is the dialect of this backend's filesystem. Every Join, Dir,
	// IsAbs or Rel on a path that goes to or comes from this backend uses it.
	Paths() PathDialect

	// SupportsReflink returns whether the filesystem at path supports CoW
	// reflinks. The string return is a human-readable reason when unsupported.
	SupportsReflink(ctx context.Context, path string) (bool, string, error)
}

Backend abstracts filesystem operations so services work identically against a local filesystem or an SSH-backed remote. It covers exactly the operations qui's services need: syscall-level primitives (stat, walk, mkdir, remove) plus the high-level tree operations (HardlinkTree, ReflinkTree, RemoveTree) that create-and-rollback as a unit, and the path dialect (Paths) that callers manipulate backend paths with, since a remote backend's paths are not in the host's grammar.

Every method accepts a context.Context and must respect cancellation.

Error semantics are portable across implementations: a missing path is reported compatibly with errors.Is(err, fs.ErrNotExist) and a denied one with errors.Is(err, fs.ErrPermission) — from Stat, Lstat, ReadDir, WalkDir (on entries), Remove, and Statfs alike. Remote backends must map their transport's errors onto the same sentinels.

type DirEntry

type DirEntry struct {
	Name      string
	IsDir     bool
	IsSymlink bool
}

DirEntry holds one entry from a ReadDir call.

type FileInfo

type FileInfo struct {
	Path      string
	Size      int64
	ModTime   time.Time
	IsDir     bool
	IsSymlink bool
	Mode      fs.FileMode
}

FileInfo holds metadata from a Stat call.

type LstatInfo

type LstatInfo struct {
	FileInfo
	FileID hardlink.FileID
	Nlinks uint64
	// FileIDErr is set when FileID/Nlinks could not be resolved (e.g. a
	// Windows ACL denial on an otherwise statable file). The rest of the
	// struct is still valid; callers that need identity must check this
	// instead of treating the whole stat as failed.
	FileIDErr error
}

LstatInfo holds metadata from an Lstat call, including hardlink identity.

type PathDialect added in v1.31.0

type PathDialect interface {
	// Join takes two parts, which is every backend-path join in qui; a
	// variadic slice would escape through the interface call on each file.
	Join(base, name string) string
	Dir(p string) string
	Base(p string) string
	Clean(p string) string
	IsAbs(p string) bool
	Rel(base, target string) (string, error)
	// FromSlash and ToSlash convert between slash-delimited names, which is
	// how qBittorrent reports every file, and the dialect's own form.
	FromSlash(p string) string
	ToSlash(p string) string
	Separator() string
}

PathDialect is the path grammar of a backend's filesystem. A path that goes to or comes from a Backend is manipulated with that backend's dialect, never with the host's filepath: a remote unix path on a Windows host is still slash-delimited and still absolute. Ext is absent on purpose, it reads the same under both grammars.

var HostPaths PathDialect = hostPaths{}

HostPaths is the dialect of the host filesystem: path/filepath.

var SlashPaths PathDialect = slashPaths{}

SlashPaths is the dialect of a slash-delimited POSIX filesystem regardless of the host: path. FromSlash and ToSlash are the identity.

type Pool

type Pool struct {
	// contains filtered or unexported fields
}

Pool resolves an instance ID to the appropriate Backend. For instances with local filesystem access it returns the local backend; for instances pinned for SSH access it returns a remote backend built for that instance; for instances without access it returns a noop backend that errors on every call.

func NewPool

func NewPool(store instanceGetter, local Backend) *Pool

NewPool creates a Backend pool backed by the given instance store and local backend. Without a remote factory a remote-mode instance is an error, not a silent "not configured": NewPoolWithRemote is the constructor that wires one.

func NewPoolWithRemote added in v1.31.0

func NewPoolWithRemote(store instanceGetter, local Backend, remote func(*models.Instance) Backend) *Pool

NewPoolWithRemote is NewPool plus a remote-backend factory, called per resolution: the backend it returns is a thin handle over the shared SSH connection pool, which owns the connections and their lifetime.

func (*Pool) GetBackend

func (p *Pool) GetBackend(ctx context.Context, instanceID int) (Backend, error)

GetBackend returns the appropriate Backend for the given instance ID. Returns ErrNoFilesystemAccess wrapped in a noop backend for instances without filesystem access configured.

func (*Pool) LocalBackend added in v1.31.0

func (p *Pool) LocalBackend(ctx context.Context, instanceID int) (Backend, error)

LocalBackend is Resolve for a caller that admitted the instance on local access from an earlier read of it. Any other mode fails with ErrNotLocal, so a caller whose read went stale never runs local paths against another host.

func (*Pool) Require added in v1.31.0

func (p *Pool) Require(ctx context.Context, instanceID int, capability models.FilesystemCapability) (Backend, *models.Instance, error)

Require returns the backend and the instance row for an instance whose mode grants capability, both from one read, so a caller cannot check one row and act on another. Any other instance fails with ErrNotCapable.

func (*Pool) Resolve added in v1.31.0

func (p *Pool) Resolve(ctx context.Context, instanceID int) (Backend, models.FilesystemMode, error)

Resolve is GetBackend plus the mode the backend was chosen for, both from one instance read, so a caller that records the mode cannot label a walk with a mode the instance changed to in between.

type RemoveOptions

type RemoveOptions struct {
	Recursive bool
}

RemoveOptions controls the behavior of a Remove call.

type StatfsResult

type StatfsResult struct {
	BytesAvailable int64
	BytesTotal     int64
}

StatfsResult holds filesystem space information.

type TreeCreateResult

type TreeCreateResult struct {
	Created       int
	SkippedExists int
	Files         []string
	Dirs          []string
}

TreeCreateResult holds the outcome of a HardlinkTree or ReflinkTree call. Files and Dirs record what the call actually created on disk (pre-existing paths are excluded), so RemoveTree can undo exactly this call's work without touching links shared with sibling torrents (discussion #2282).

type WalkEntry

type WalkEntry struct {
	LstatInfo
	RelPath string
	Err     error
	// StatErr is set when the entry was enumerated but its metadata lookup
	// failed; only Path and RelPath are valid then. Such entries are emitted
	// only when WalkOptions.EmitStatErrors is set and skipped otherwise.
	StatErr error
}

WalkEntry is emitted by WalkDir for each filesystem entry encountered.

type WalkOptions

type WalkOptions struct {
	SkipHidden bool
	// IgnoreDirNames are directory basenames to skip, matched case-insensitively
	// (OS/NAS metadata dirs like $RECYCLE.BIN and @eaDir vary in on-disk case).
	IgnoreDirNames []string
	// IgnoreDirNamePrefixes are matched case-insensitively.
	IgnoreDirNamePrefixes []string
	IgnorePaths           []string
	// WantFileID populates FileID and Nlinks on regular-file entries.
	WantFileID bool
	// EmitStatErrors emits entries whose metadata lookup failed with StatErr
	// set instead of silently skipping them. Consumers that delete based on
	// walk results use this to fail closed on paths they cannot verify.
	EmitStatErrors bool
}

WalkOptions controls the behavior of a WalkDir call.

func (*WalkOptions) Skip added in v1.31.0

func (o *WalkOptions) Skip(name, fullPath string, isDir, isRoot bool) bool

Skip reports whether a walk leaves out an entry, and its subtree when the entry is a directory. fullPath is in the backend's own path dialect, since IgnorePaths is an exact string compare. The walk root is exempt from every filter except IgnorePaths, so an ignored root gives an empty walk.

Directories

Path Synopsis
Package local implements fsops.Backend by delegating to the local filesystem via os.*, pkg/hardlinktree, pkg/reflinktree, pkg/hardlink, and pkg/fsutil.
Package local implements fsops.Backend by delegating to the local filesystem via os.*, pkg/hardlinktree, pkg/reflinktree, pkg/hardlink, and pkg/fsutil.
Package remote implements fsops.Backend over SFTP, on the pooled SSH connection the instance's stored credentials open.
Package remote implements fsops.Backend over SFTP, on the pooled SSH connection the instance's stored credentials open.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL