pxar

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 16, 2026 License: AGPL-3.0 Imports: 16 Imported by: 0

Documentation

Overview

Package pxar implements the PXAR archive format: record framing constants, encoders for every entry type (v1 and the v2 split-archive additions), and the goodbye-table construction (siphash naming hash plus the casync implicit binary search tree layout), and Reader, a validating decoder for v1 archives and v2 split pairs.

The package does no filesystem access and has no I/O policy: encoders append bytes, Reader consumes the io.Reader/io.ReaderAt it is given. Every encoder has a matching size function used by the archive planner, so planned and emitted sizes can't diverge.

Index

Constants

View Source
const (
	ACLRead    uint64 = 4
	ACLWrite   uint64 = 2
	ACLExecute uint64 = 1
	// ACLNoMask marks an absent mask in ACLDefault.
	ACLNoMask uint64 = ^uint64(0)
)

ACL permission bits (per entry, rwx as in POSIX ACLs).

View Source
const (
	ACLUserSize     = HeaderSize + 16
	ACLGroupSize    = HeaderSize + 16
	ACLGroupObjSize = HeaderSize + 8
	ACLDefaultSize  = HeaderSize + 32
)

Encoded sizes of the fixed-layout ACL records.

View Source
const (
	// TypeEntry is the stat record: mode, flags, uid, gid, mtime.
	TypeEntry uint64 = 0xd5956474e588acef
	// TypeEntryV1 is the obsolete stat record layout of early pxar versions.
	// Decode-only concern; never emitted.
	TypeEntryV1 uint64 = 0x11da850a1c1cceff
	// TypeFilename precedes each directory child: name bytes + NUL.
	TypeFilename uint64 = 0x16701121063917b3
	// TypeSymlink carries the link target: target bytes + NUL.
	TypeSymlink uint64 = 0x27f971e7dbf5dc5f
	// TypeDevice carries major/minor numbers for block and char devices.
	TypeDevice uint64 = 0x9fc9e906586d5ce9
	// TypeXAttr carries one extended attribute: name + NUL + value.
	TypeXAttr uint64 = 0x0dab0229b57dcd03
	// ACL record types; bodies are defined in encode.go.
	TypeACLUser         uint64 = 0x2ce8540a457d55b8
	TypeACLGroup        uint64 = 0x136e3eceb04c03ab
	TypeACLGroupObj     uint64 = 0x10868031e9582876
	TypeACLDefault      uint64 = 0xbbbb13415a6896f5
	TypeACLDefaultUser  uint64 = 0xc89357b40532cd1f
	TypeACLDefaultGroup uint64 = 0xf90a8a5816038ffe
	// TypeFCaps carries the raw security.capability xattr value.
	TypeFCaps uint64 = 0x2da9dd9db5f7fb67
	// TypeQuotaProjID carries the filesystem quota project id.
	TypeQuotaProjID uint64 = 0xe07540e82f7d1cbb
	// TypeHardlink references an earlier entry: offset (u64) + path + NUL.
	TypeHardlink uint64 = 0x51269c8422bd7275
	// TypePayload frames regular file content.
	TypePayload uint64 = 0x28147a1b0b7c1a25
	// TypeGoodbye terminates a directory: a BST of per-child items.
	TypeGoodbye uint64 = 0x2fec4fa642d5731d
	// GoodbyeTailMarker is the hash value of the final goodbye item.
	GoodbyeTailMarker uint64 = 0xef5eed5b753e1555

	// v2 (split archive) additions.
	TypeFormatVersion  uint64 = 0x730f6c75df16a40d
	TypePrelude        uint64 = 0xe309d79d9f7b771b
	TypePayloadRef     uint64 = 0x419d3d6bc4ba977e
	PayloadStartMarker uint64 = 0x834c68c2194a4ed2
	PayloadTailMarker  uint64 = 0x6c72b78b984c81b5
)

Record type identifiers. Every item in a pxar stream is framed as a 16-byte header — type (u64 LE) followed by length (u64 LE, including the header) — and a body. Values match proxmox/pxar src/format/mod.rs.

View Source
const DeviceSize = HeaderSize + 16

DeviceSize is the full encoded size of a Device record.

View Source
const EntrySize = HeaderSize + 40

EntrySize is the full encoded size of an Entry record.

View Source
const FormatVersionSize = HeaderSize + 8

FormatVersionSize is the full encoded size of a TypeFormatVersion record.

View Source
const HeaderSize = 16

HeaderSize is the size of the type+length framing of every record.

View Source
const MarkerSize = HeaderSize

MarkerSize is the encoded size of the bare-header payload stream markers.

View Source
const PayloadRefSize = HeaderSize + 16

PayloadRefSize is the full encoded size of a TypePayloadRef record.

View Source
const QuotaProjIDSize = HeaderSize + 8

QuotaProjIDSize is the full encoded size of a TypeQuotaProjID record.

Variables

This section is empty.

Functions

func AppendACLDefault

func AppendACLDefault(dst []byte, a ACLDefault) []byte

AppendACLDefault encodes the default-ACL object permissions (TypeACLDefault).

func AppendACLDefaultGroup

func AppendACLDefaultGroup(dst []byte, a ACLGroup) []byte

AppendACLDefaultGroup encodes a default named-group ACL entry (TypeACLDefaultGroup).

func AppendACLDefaultUser

func AppendACLDefaultUser(dst []byte, a ACLUser) []byte

AppendACLDefaultUser encodes a default named-user ACL entry (TypeACLDefaultUser).

func AppendACLGroup

func AppendACLGroup(dst []byte, a ACLGroup) []byte

AppendACLGroup encodes a named-group ACL entry (TypeACLGroup).

func AppendACLGroupObj

func AppendACLGroupObj(dst []byte, permissions uint64) []byte

AppendACLGroupObj encodes the owning-group permissions (TypeACLGroupObj), present when a mask entry makes the stat group bits carry the mask instead.

func AppendACLUser

func AppendACLUser(dst []byte, a ACLUser) []byte

AppendACLUser encodes a named-user ACL entry (TypeACLUser).

func AppendDevice

func AppendDevice(dst []byte, d Device) []byte

AppendDevice encodes d as a TypeDevice record.

func AppendEntry

func AppendEntry(dst []byte, e Entry) []byte

AppendEntry encodes e as a TypeEntry record.

func AppendFCaps

func AppendFCaps(dst []byte, data []byte) []byte

AppendFCaps encodes the raw security.capability xattr value.

func AppendFilename

func AppendFilename(dst []byte, name string) []byte

AppendFilename encodes a directory child's name (NUL-terminated).

func AppendFormatVersion

func AppendFormatVersion(dst []byte, version uint64) []byte

AppendFormatVersion encodes the format version record that opens a v2 metadata stream (version 1 is expressed by the record's absence).

func AppendGoodbye

func AppendGoodbye(dst []byte, items []GoodbyeItem, dirEntryStart, goodbyeStart uint64) []byte

AppendGoodbye encodes a directory's goodbye table. dirEntryStart is the absolute offset of the directory's own TypeEntry record (i.e. after its filename record; for the root directory, the archive start), goodbyeStart the absolute offset where this goodbye record begins. Items may be in any order; they are sorted by hash and permuted into the BST layout. Item offsets are encoded as distances backwards from goodbyeStart.

func AppendHardlink(dst []byte, offset uint64, target string) []byte

AppendHardlink encodes a hardlink record: offset is the distance in bytes from the start of this hardlink's own TypeFilename record back to the start of the link target's TypeFilename record (the upstream encoder's current_position - target_position); target is the archive-relative path of the target (NUL-terminated).

func AppendPayloadHeader

func AppendPayloadHeader(dst []byte, size uint64) []byte

AppendPayloadHeader encodes the header of a TypePayload record. The size bytes of raw file content follow the header in the stream; they are written by the emitter, not by this package.

func AppendPayloadRef

func AppendPayloadRef(dst []byte, offset, size uint64) []byte

AppendPayloadRef encodes a v2 payload reference: offset addresses the payload record's *header* in the payload stream, size is the content byte count (the referenced record's length minus its header).

func AppendPayloadStartMarker

func AppendPayloadStartMarker(dst []byte) []byte

AppendPayloadStartMarker encodes the record that opens a payload stream.

func AppendPayloadTailMarker

func AppendPayloadTailMarker(dst []byte) []byte

AppendPayloadTailMarker encodes the record that closes a payload stream.

func AppendPrelude added in v0.3.0

func AppendPrelude(dst, data []byte) []byte

AppendPrelude encodes the v2 prelude: opaque bytes following the format version record. proxmox-backup-client stores its exclude pattern list here (one pattern per line) and omits the record when there are none.

func AppendQuotaProjID

func AppendQuotaProjID(dst []byte, projid uint64) []byte

AppendQuotaProjID encodes a filesystem quota project id.

func AppendSymlink(dst []byte, target string) []byte

AppendSymlink encodes a symlink target (NUL-terminated).

func AppendXAttr

func AppendXAttr(dst []byte, name string, value []byte) []byte

AppendXAttr encodes one extended attribute: name + NUL + value.

func DecodeMetadata added in v0.3.0

func DecodeMetadata(r io.Reader, visit func(MetaFile) error) (version int, err error)

DecodeMetadata walks a pxar v1 archive or v2 metadata stream in order and calls visit for every regular file with its encoded metadata and payload reference. Payload content of a v1 archive is skipped. It returns the format version (1 or 2). Change detection uses it to index the previous snapshot's metadata stream.

func FileMode added in v0.4.0

func FileMode(mode uint64) fs.FileMode

FileMode converts an st_mode value to an fs.FileMode.

func Hash

func Hash(name string) uint64

Hash returns the goodbye-table hash of a directory child's basename.

func SizeFCaps

func SizeFCaps(data []byte) uint64

SizeFCaps returns the encoded size of a TypeFCaps record.

func SizeFilename

func SizeFilename(name string) uint64

SizeFilename returns the encoded size of a TypeFilename record.

func SizeGoodbye

func SizeGoodbye(n int) uint64

SizeGoodbye returns the full encoded size of a goodbye record for a directory with n children (n items plus the tail marker).

func SizeHardlink(target string) uint64

SizeHardlink returns the encoded size of a TypeHardlink record.

func SizePayload

func SizePayload(size uint64) uint64

SizePayload returns the full encoded size of a payload record whose content is size bytes: the header plus the raw content that follows it.

func SizePrelude added in v0.3.0

func SizePrelude(data []byte) uint64

SizePrelude returns the encoded size of a TypePrelude record.

func SizeSymlink(target string) uint64

SizeSymlink returns the encoded size of a TypeSymlink record.

func SizeXAttr

func SizeXAttr(name string, value []byte) uint64

SizeXAttr returns the encoded size of a TypeXAttr record.

func ValidateFilename

func ValidateFilename(name string) error

ValidateFilename reports whether name is usable as a pxar directory child name: non-empty, no '/', no NUL, and not "." or "..".

Types

type ACLDefault

type ACLDefault struct {
	UserObjPermissions  uint64
	GroupObjPermissions uint64
	OtherPermissions    uint64
	MaskPermissions     uint64 // ACLNoMask when no mask entry exists
}

ACLDefault is the body of a TypeACLDefault record.

type ACLGroup

type ACLGroup struct {
	GID         uint64
	Permissions uint64
}

ACLGroup is the body of TypeACLGroup and TypeACLDefaultGroup records.

type ACLUser

type ACLUser struct {
	UID         uint64
	Permissions uint64
}

ACLUser is the body of TypeACLUser and TypeACLDefaultUser records.

type ACLs added in v0.4.0

type ACLs struct {
	Users    []ACLUser  // named users (access ACL)
	Groups   []ACLGroup // named groups (access ACL)
	GroupObj *uint64    // owning-group permissions; set only when a mask entry exists

	Default       *ACLDefault // default ACL object permissions (directories)
	DefaultUsers  []ACLUser
	DefaultGroups []ACLGroup
}

ACLs holds a node's POSIX ACL entries in encoder-ready form. A nil *ACLs means the node has no ACL records (a trivial ACL that only mirrors the mode bits does not count).

type Device

type Device struct {
	Major uint64
	Minor uint64
}

Device is the body of a TypeDevice record.

type Entry

type Entry struct {
	Mode       uint64 // full st_mode: type bits + permission bits
	Flags      uint64
	UID        uint32
	GID        uint32
	MtimeSecs  int64  // seconds since epoch (may be negative)
	MtimeNanos uint32 // nanoseconds within MtimeSecs
}

Entry is the stat record body (TypeEntry).

type FS added in v0.4.0

type FS struct {
	// contains filtered or unexported fields
}

FS is a read-only, random-access view of an archive as an io/fs file system. It implements fs.FS, fs.ReadDirFS, fs.StatFS and fs.ReadLinkFS.

Paths are looked up through the directories' goodbye tables, so only the records on the way are read: over a remote index (see pbs OpenDynamicIndexAt) a lookup fetches a handful of chunks, not the archive. Listing a v2 directory reads the metadata stream only; payload records are checked when a file is opened.

Open and Stat follow symbolic links inside the archive; links with an absolute target, or a relative one leaving the archive, do not resolve. A hardlink resolves to the entry it links to, under its own name. FileInfo.Sys returns the *Node with the full metadata.

The file system is safe for concurrent use if its io.ReaderAt sources are.

func OpenV1 added in v0.4.0

func OpenV1(r io.ReaderAt, size int64) (*FS, error)

OpenV1 opens a pxar v1 archive of the given size.

func OpenV2 added in v0.4.0

func OpenV2(meta io.ReaderAt, metaSize int64, payload io.ReaderAt) (*FS, error)

OpenV2 opens a v2 split archive: the metadata stream of metaSize bytes and the payload stream. payload may be nil when only metadata is needed; opening a regular file then fails.

func (*FS) Lstat added in v0.4.0

func (f *FS) Lstat(name string) (fs.FileInfo, error)

Lstat returns a FileInfo for the named file without following a final symbolic link.

func (*FS) Open added in v0.4.0

func (f *FS) Open(name string) (fs.File, error)

Open opens the named file, following symbolic links.

func (*FS) ReadDir added in v0.4.0

func (f *FS) ReadDir(name string) ([]fs.DirEntry, error)

ReadDir reads the named directory, following symbolic links, and returns its entries sorted by name.

func (f *FS) ReadLink(name string) (string, error)

ReadLink returns the target of the named symbolic link.

func (*FS) Stat added in v0.4.0

func (f *FS) Stat(name string) (fs.FileInfo, error)

Stat returns a FileInfo for the named file, following symbolic links.

type GoodbyeItem

type GoodbyeItem struct {
	Hash   uint64 // Hash(basename)
	Start  uint64
	Length uint64
}

GoodbyeItem describes one directory child for goodbye-table construction. Start and Length are in stream coordinates: Start is the absolute offset of the child's TypeFilename record, Length the child's total encoded length (filename record through goodbye table or payload, inclusive).

type MetaFile added in v0.3.0

type MetaFile struct {
	// Path is the archive-relative path ("" for the root, "dir/name" below it).
	Path string
	// Meta is the file's ENTRY record followed by its metadata records
	// (xattrs, ACLs, fcaps, quota project id), byte for byte as encoded.
	Meta []byte
	// Size is the content byte count.
	Size uint64
	// PayloadOffset is the position of the file's payload record header in
	// the payload stream (v2 only; 0 in a v1 archive).
	PayloadOffset uint64
}

MetaFile is one regular file found while decoding a pxar metadata stream.

type Node added in v0.4.0

type Node struct {
	// Path is the archive-relative path ("" for the root, "dir/name" below it).
	Path string
	// Entry is the stat record. It is zero for a hardlink, which carries no
	// metadata of its own.
	Entry

	// Hardlink marks a hardlink to an earlier entry of the archive.
	Hardlink bool
	// LinkTarget holds the symlink target, or for a hardlink the
	// archive-relative path of the entry it links to.
	LinkTarget string

	Xattrs      []Xattr
	ACL         *ACLs
	FCaps       []byte
	QuotaProjID uint64 // 0 = none
	Device      Device // block and char devices

	// Size is the content byte count of a regular file; Reader.Read returns
	// exactly that many bytes.
	Size uint64
	// PayloadOffset is the position of the file's payload record header in
	// the payload stream (v2 only).
	PayloadOffset uint64
	// contains filtered or unexported fields
}

Node is one archive entry as decoded by Reader.

type Reader added in v0.4.0

type Reader struct {
	// contains filtered or unexported fields
}

Reader reads a pxar archive entry by entry, in archive order, like archive/tar: Next advances to the next entry, Read returns the content of the current regular file. Unread content is skipped by Next.

Every entry is validated as it is decoded: filenames and hardlink targets must be plain relative names, records must have their exact sizes, and a stream must end exactly after the root directory's goodbye table.

func NewReaderV1 added in v0.4.0

func NewReaderV1(r io.Reader) *Reader

NewReaderV1 reads a pxar v1 archive.

func NewReaderV2 added in v0.4.0

func NewReaderV2(meta io.Reader, payload io.ReaderAt) *Reader

NewReaderV2 reads a v2 split archive: the metadata stream (.mpxar) in order, and the payload stream (.ppxar) at the offsets the metadata stream references. payload may be nil when only metadata is needed; Read then fails.

func (*Reader) Next added in v0.4.0

func (r *Reader) Next() (*Node, error)

Next advances to the next entry. The root directory comes first, with Path "". At the end of the archive Next returns io.EOF.

func (*Reader) Read added in v0.4.0

func (r *Reader) Read(p []byte) (int, error)

Read reads the content of the current regular file. Other entries have no content.

type Xattr added in v0.4.0

type Xattr struct {
	Name  string
	Value []byte
}

Xattr is one extended attribute. Values may contain arbitrary bytes.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL