scan

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 16, 2026 License: AGPL-3.0 Imports: 14 Imported by: 0

Documentation

Overview

Package scan walks filesystem trees into immutable node trees carrying the full metadata a PXAR archive needs: mode, ownership, timestamps, symlink targets, hardlink identity, device numbers, xattrs, POSIX ACLs, file capabilities and quota project IDs.

Metadata is captured at scan time; file contents are deliberately not read here — they are streamed later by the archive emitter. Platform specifics live behind the MetadataReader interface; the full implementation targets Linux via golang.org/x/sys/unix.

Entries can be left out with exclude patterns in proxmox-backup-client syntax (Options.Exclude, Pattern), per-directory .pxarexclude files (Options.PxarExcludeFiles) and a free-form Options.Filter callback.

Index

Constants

View Source
const (
	ModeTypeMask uint64 = 0o170000
	ModeSocket   uint64 = 0o140000
	ModeSymlink  uint64 = 0o120000
	ModeRegular  uint64 = 0o100000
	ModeBlockDev uint64 = 0o060000
	ModeDir      uint64 = 0o040000
	ModeCharDev  uint64 = 0o020000
	ModeFifo     uint64 = 0o010000
)

File-type bits of Stat.Mode (the universal S_IF* values; kept local so the portable core does not depend on platform packages).

Variables

This section is empty.

Functions

This section is empty.

Types

type ACLs

type ACLs = pxar.ACLs

ACLs holds a node's POSIX ACL entries (see pxar.ACLs). A nil *ACLs means the node has no ACL records to emit.

type Follower added in v0.4.0

type Follower interface {
	Stat(path string) (Stat, error)
}

Follower is implemented by readers that can stat through a final symlink. The scanner uses it for a root that is itself a symbolic link (or junction) to a directory — a Volume Shadow Copy exposed through a link, for example — so the tree behind the link is archived while every path below is still opened through the link, which the OS follows transparently.

type Kind

type Kind int

Kind classifies a node. Devices carry block/char in the Mode type bits.

const (
	KindDirectory Kind = iota
	KindFile
	KindSymlink
	KindHardlink
	KindBlockDevice
	KindCharDevice
	KindFifo
	KindSocket
	// KindStream is a virtual file entry backed by an io.Reader with a
	// caller-declared size.
	KindStream
)

type MatchType added in v0.3.0

type MatchType int

MatchType is the verdict of a pattern hit.

const (
	// MatchExclude: the entry is left out of the archive.
	MatchExclude MatchType = iota
	// MatchInclude: the entry is kept even if an earlier pattern excluded it.
	MatchInclude
)

type MetadataReader

type MetadataReader interface {
	Lstat(path string) (Stat, error)
	// ReadDirNames returns the child names of a directory sorted in byte
	// order — the order pxar archives require.
	ReadDirNames(path string) ([]string, error)
	Readlink(path string) (string, error)
	// Xattrs returns generic extended attributes sorted by name, excluding
	// those represented by dedicated records (POSIX ACLs, security.capability).
	Xattrs(path string) ([]Xattr, error)
	ACLs(path string) (*ACLs, error)
	FCaps(path string) ([]byte, error)
	// QuotaProjID returns the filesystem quota project id and whether one is
	// set (project id 0 counts as unset).
	QuotaProjID(path string) (uint64, bool, error)
	// ReadFile returns the contents of a regular file (the scanner reads
	// .pxarexclude files with it). A missing file must be reported with an
	// error satisfying errors.Is(err, fs.ErrNotExist).
	ReadFile(path string) ([]byte, error)
}

MetadataReader is the platform seam: everything the scanner asks of the filesystem. The full-fidelity implementation targets Linux; other platforms can supply degraded implementations without touching the walk or encoders.

All methods must resolve the path without following a final symlink (l-variants). Absent metadata (no xattrs, no ACL, no fcaps) is a nil/empty result, not an error.

func DefaultReader

func DefaultReader() (MetadataReader, error)

DefaultReader returns the platform MetadataReader.

type Node

type Node struct {
	Name string // basename within the parent (archive naming)
	Path string // absolute source path; empty for virtual nodes
	Kind Kind

	Stat Stat

	// LinkTarget holds the symlink target (KindSymlink) or the
	// archive-relative path of the hardlink's first occurrence (KindHardlink).
	LinkTarget string

	Xattrs      []Xattr
	ACL         *ACLs
	FCaps       []byte
	QuotaProjID uint64 // 0 = none
	Children    []*Node

	// Stream backs KindStream nodes; Stat.Size holds the declared size.
	Stream io.Reader
}

Node is one entry in a scanned tree. Metadata is captured at scan time; file contents are read later by the archive emitter.

func StreamNode

func StreamNode(name string, size int64, r io.Reader) (*Node, error)

StreamNode returns a virtual regular-file node backed by r with the declared size. The pad/truncate policy applies if r yields a different byte count. Mode is a plain 0644 regular file owned by uid/gid 0; the caller may adjust Stat before planning.

func VirtualRoot

func VirtualRoot(name string, children []*Node) (*Node, error)

VirtualRoot synthesizes the top-level directory used when an archive holds multiple roots or bare files: mode drwxrwxrwx, uid/gid 0. Children are sorted into archive order; names must be unique. The caller stamps the mtime (plan time).

type Options

type Options struct {
	// Reader overrides the platform MetadataReader (used in tests and by
	// future non-Linux implementations).
	Reader MetadataReader
	// SkipOnError downgrades per-entry scan failures to warnings, omitting
	// the entry, instead of failing the scan.
	SkipOnError bool
	// OnWarn receives warnings; nil discards them.
	OnWarn func(Warning)
	// SkipQuotaProjIDs disables the per-node quota project id lookup, which
	// costs one open+ioctl per directory and regular file.
	SkipQuotaProjIDs bool

	// Exclude lists exclude patterns in proxmox-backup-client syntax (see
	// Pattern), matched against archive-relative paths; an excluded
	// directory's subtree is not scanned. Invalid patterns fail NewScanner.
	// The archive package records these patterns in the archive the way the
	// official client does (.pxarexclude-cli in v1, the prelude in v2).
	Exclude []string
	// PxarExcludeFiles honours .pxarexclude files found in scanned
	// directories, like the official client: each line is a pattern
	// (Pattern syntax; '#' comments and blank lines ignored) applying to
	// that directory's subtree only, with anchored patterns relative to the
	// directory holding the file. Lines that fail to parse are reported to
	// OnWarn (with a *PatternError) and ignored; the file itself is
	// archived. Off by default: a library should not silently obey control
	// files found inside the data it backs up.
	PxarExcludeFiles bool
	// Filter, when set, is consulted for every entry below a root that the
	// patterns did not exclude; returning false omits the entry and its
	// subtree. It sees the on-disk path, the archive-relative path and the
	// entry's lstat data. Unlike Exclude it is never recorded in the archive.
	Filter func(path, archivePath string, st Stat) bool
	// OnExclude, when set, receives every entry omitted by Exclude, a
	// .pxarexclude file or Filter; nil discards them.
	OnExclude func(path, archivePath string)
}

Options configures a Scanner. The zero value scans with the platform default reader, fails on the first error, and reads quota project ids.

type Pattern added in v0.3.0

type Pattern struct {
	Text     string // body: no leading '!' or '/', no trailing '/'
	Anchored bool   // leading '/'
	DirOnly  bool   // trailing '/'
	Include  bool   // leading '!'
	// contains filtered or unexported fields
}

Pattern is one exclude pattern in proxmox-backup-client syntax:

  • a leading '!' turns the pattern into an include (re-include) pattern;
  • a leading '/' anchors it: it must match the whole archive-relative path. Unanchored patterns match at any depth (every suffix of the path starting at a component boundary is tried, so "*.tmp" matches by basename and "foo/bar" matches "a/b/foo/bar");
  • a trailing '/' restricts the pattern to directories;
  • the body is an fnmatch(3)-style glob with FNM_PATHNAME semantics: '*', '?' and '[…]' classes never match '/' ('[^…]' negates a class), '\' escapes the next character, and '**' as a whole component matches zero or more components ("a/**/b" matches "a/b" and "a/x/y/b"), or at least one at the end of a pattern ("a/**" matches a's contents, not a).

Patterns without glob metacharacters are compared literally. The rules match the official client's, verified against `pxar create --exclude`.

func ParsePattern added in v0.3.0

func ParsePattern(s string) (Pattern, error)

ParsePattern parses one pattern; errors are *PatternError.

func (Pattern) Match added in v0.3.0

func (p Pattern) Match(archivePath string, isDir bool) bool

Match reports whether the pattern applies to the entry at archivePath (a slash-separated path relative to the archive root, no leading slash). The archive root itself is never matched.

func (Pattern) String added in v0.3.0

func (p Pattern) String() string

String renders the pattern in its source form — the line format proxmox-backup-client records in .pxarexclude-cli and the v2 prelude.

type PatternError added in v0.3.0

type PatternError struct {
	Pattern string
	Err     error
}

PatternError reports an exclude pattern that failed to parse.

func (*PatternError) Error added in v0.3.0

func (e *PatternError) Error() string

func (*PatternError) Unwrap added in v0.3.0

func (e *PatternError) Unwrap() error

type PatternList added in v0.3.0

type PatternList []Pattern

PatternList is an ordered list of patterns evaluated last-match-wins.

func ParsePatterns added in v0.3.0

func ParsePatterns(ss []string) (PatternList, error)

ParsePatterns parses each pattern in order, stopping at the first error.

func (PatternList) Match added in v0.3.0

func (l PatternList) Match(archivePath string, isDir bool) (mt MatchType, ok bool)

Match evaluates the list last-match-wins: the latest pattern that matches decides. ok is false when no pattern matches (the entry is included).

func (PatternList) String added in v0.3.0

func (l PatternList) String() string

String renders the list one pattern per line, newline-terminated — the exact content of proxmox-backup-client's .pxarexclude-cli / v2 prelude.

type Scanner

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner walks filesystem trees into Node trees. A single Scanner may scan several roots (multiple AddDirectory/AddFile calls); hardlink identity is tracked across all of them so cross-root hardlinks resolve correctly.

func NewScanner

func NewScanner(opts Options) (*Scanner, error)

NewScanner returns a Scanner for the given options. It fails on platforms without a default MetadataReader when none is supplied.

func (*Scanner) ScanDirectory

func (s *Scanner) ScanDirectory(path, archivePath string) (*Node, error)

ScanDirectory scans the tree rooted at path. archivePath is the node's path within the archive ("" when this directory is the archive root); it prefixes the recorded hardlink targets of everything below.

func (*Scanner) ScanFile

func (s *Scanner) ScanFile(path, archivePath string) (*Node, error)

ScanFile scans a single non-directory entry (top-level AddFile).

type Stat

type Stat struct {
	Mode       uint64 // full st_mode: type + permission bits
	UID        uint32
	GID        uint32
	MtimeSecs  int64
	MtimeNanos uint32
	Size       int64  // regular files: byte size at scan time
	Nlink      uint64 // hardlink count
	Dev        uint64 // containing device (hardlink identity)
	Ino        uint64 // inode number (hardlink identity)
	Rdev       pxar.Device
}

Stat is the lstat subset the scanner needs, in encoder-ready form.

type Warning

type Warning struct {
	Path string
	Err  error
}

Warning is a non-fatal scan event (skipped entry under SkipOnError).

type Xattr

type Xattr = pxar.Xattr

Xattr is one extended attribute (see pxar.Xattr).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL