Documentation
¶
Overview ¶
Package pxar implements the PXAR archive format: record framing constants, encoders for every entry type (v1 and the v2 split-archive additions), and the goodbye-table construction (siphash naming hash plus the casync implicit binary search tree layout), and Reader, a validating decoder for v1 archives and v2 split pairs.
The package does no filesystem access and has no I/O policy: encoders append bytes, Reader consumes the io.Reader/io.ReaderAt it is given. Every encoder has a matching size function used by the archive planner, so planned and emitted sizes can't diverge.
Index ¶
- Constants
- func AppendACLDefault(dst []byte, a ACLDefault) []byte
- func AppendACLDefaultGroup(dst []byte, a ACLGroup) []byte
- func AppendACLDefaultUser(dst []byte, a ACLUser) []byte
- func AppendACLGroup(dst []byte, a ACLGroup) []byte
- func AppendACLGroupObj(dst []byte, permissions uint64) []byte
- func AppendACLUser(dst []byte, a ACLUser) []byte
- func AppendDevice(dst []byte, d Device) []byte
- func AppendEntry(dst []byte, e Entry) []byte
- func AppendFCaps(dst []byte, data []byte) []byte
- func AppendFilename(dst []byte, name string) []byte
- func AppendFormatVersion(dst []byte, version uint64) []byte
- func AppendGoodbye(dst []byte, items []GoodbyeItem, dirEntryStart, goodbyeStart uint64) []byte
- func AppendHardlink(dst []byte, offset uint64, target string) []byte
- func AppendPayloadHeader(dst []byte, size uint64) []byte
- func AppendPayloadRef(dst []byte, offset, size uint64) []byte
- func AppendPayloadStartMarker(dst []byte) []byte
- func AppendPayloadTailMarker(dst []byte) []byte
- func AppendPrelude(dst, data []byte) []byte
- func AppendQuotaProjID(dst []byte, projid uint64) []byte
- func AppendSymlink(dst []byte, target string) []byte
- func AppendXAttr(dst []byte, name string, value []byte) []byte
- func DecodeMetadata(r io.Reader, visit func(MetaFile) error) (version int, err error)
- func FileMode(mode uint64) fs.FileMode
- func Hash(name string) uint64
- func SizeFCaps(data []byte) uint64
- func SizeFilename(name string) uint64
- func SizeGoodbye(n int) uint64
- func SizeHardlink(target string) uint64
- func SizePayload(size uint64) uint64
- func SizePrelude(data []byte) uint64
- func SizeSymlink(target string) uint64
- func SizeXAttr(name string, value []byte) uint64
- func ValidateFilename(name string) error
- type ACLDefault
- type ACLGroup
- type ACLUser
- type ACLs
- type Device
- type Entry
- type FS
- type GoodbyeItem
- type MetaFile
- type Node
- type Reader
- type Xattr
Constants ¶
const ( ACLRead uint64 = 4 ACLWrite uint64 = 2 ACLExecute uint64 = 1 // ACLNoMask marks an absent mask in ACLDefault. ACLNoMask uint64 = ^uint64(0) )
ACL permission bits (per entry, rwx as in POSIX ACLs).
const ( ACLUserSize = HeaderSize + 16 ACLGroupSize = HeaderSize + 16 ACLGroupObjSize = HeaderSize + 8 ACLDefaultSize = HeaderSize + 32 )
Encoded sizes of the fixed-layout ACL records.
const ( // TypeEntry is the stat record: mode, flags, uid, gid, mtime. TypeEntry uint64 = 0xd5956474e588acef // TypeEntryV1 is the obsolete stat record layout of early pxar versions. // Decode-only concern; never emitted. TypeEntryV1 uint64 = 0x11da850a1c1cceff // TypeFilename precedes each directory child: name bytes + NUL. TypeFilename uint64 = 0x16701121063917b3 // TypeSymlink carries the link target: target bytes + NUL. TypeSymlink uint64 = 0x27f971e7dbf5dc5f // TypeDevice carries major/minor numbers for block and char devices. TypeDevice uint64 = 0x9fc9e906586d5ce9 // TypeXAttr carries one extended attribute: name + NUL + value. TypeXAttr uint64 = 0x0dab0229b57dcd03 // ACL record types; bodies are defined in encode.go. TypeACLUser uint64 = 0x2ce8540a457d55b8 TypeACLGroup uint64 = 0x136e3eceb04c03ab TypeACLGroupObj uint64 = 0x10868031e9582876 TypeACLDefault uint64 = 0xbbbb13415a6896f5 TypeACLDefaultUser uint64 = 0xc89357b40532cd1f TypeACLDefaultGroup uint64 = 0xf90a8a5816038ffe // TypeFCaps carries the raw security.capability xattr value. TypeFCaps uint64 = 0x2da9dd9db5f7fb67 // TypeQuotaProjID carries the filesystem quota project id. TypeQuotaProjID uint64 = 0xe07540e82f7d1cbb // TypeHardlink references an earlier entry: offset (u64) + path + NUL. TypeHardlink uint64 = 0x51269c8422bd7275 // TypePayload frames regular file content. TypePayload uint64 = 0x28147a1b0b7c1a25 // TypeGoodbye terminates a directory: a BST of per-child items. TypeGoodbye uint64 = 0x2fec4fa642d5731d // GoodbyeTailMarker is the hash value of the final goodbye item. GoodbyeTailMarker uint64 = 0xef5eed5b753e1555 // v2 (split archive) additions. TypeFormatVersion uint64 = 0x730f6c75df16a40d TypePrelude uint64 = 0xe309d79d9f7b771b TypePayloadRef uint64 = 0x419d3d6bc4ba977e PayloadStartMarker uint64 = 0x834c68c2194a4ed2 PayloadTailMarker uint64 = 0x6c72b78b984c81b5 )
Record type identifiers. Every item in a pxar stream is framed as a 16-byte header — type (u64 LE) followed by length (u64 LE, including the header) — and a body. Values match proxmox/pxar src/format/mod.rs.
const DeviceSize = HeaderSize + 16
DeviceSize is the full encoded size of a Device record.
const EntrySize = HeaderSize + 40
EntrySize is the full encoded size of an Entry record.
const FormatVersionSize = HeaderSize + 8
FormatVersionSize is the full encoded size of a TypeFormatVersion record.
const HeaderSize = 16
HeaderSize is the size of the type+length framing of every record.
const MarkerSize = HeaderSize
MarkerSize is the encoded size of the bare-header payload stream markers.
const PayloadRefSize = HeaderSize + 16
PayloadRefSize is the full encoded size of a TypePayloadRef record.
const QuotaProjIDSize = HeaderSize + 8
QuotaProjIDSize is the full encoded size of a TypeQuotaProjID record.
Variables ¶
This section is empty.
Functions ¶
func AppendACLDefault ¶
func AppendACLDefault(dst []byte, a ACLDefault) []byte
AppendACLDefault encodes the default-ACL object permissions (TypeACLDefault).
func AppendACLDefaultGroup ¶
AppendACLDefaultGroup encodes a default named-group ACL entry (TypeACLDefaultGroup).
func AppendACLDefaultUser ¶
AppendACLDefaultUser encodes a default named-user ACL entry (TypeACLDefaultUser).
func AppendACLGroup ¶
AppendACLGroup encodes a named-group ACL entry (TypeACLGroup).
func AppendACLGroupObj ¶
AppendACLGroupObj encodes the owning-group permissions (TypeACLGroupObj), present when a mask entry makes the stat group bits carry the mask instead.
func AppendACLUser ¶
AppendACLUser encodes a named-user ACL entry (TypeACLUser).
func AppendDevice ¶
AppendDevice encodes d as a TypeDevice record.
func AppendEntry ¶
AppendEntry encodes e as a TypeEntry record.
func AppendFCaps ¶
AppendFCaps encodes the raw security.capability xattr value.
func AppendFilename ¶
AppendFilename encodes a directory child's name (NUL-terminated).
func AppendFormatVersion ¶
AppendFormatVersion encodes the format version record that opens a v2 metadata stream (version 1 is expressed by the record's absence).
func AppendGoodbye ¶
func AppendGoodbye(dst []byte, items []GoodbyeItem, dirEntryStart, goodbyeStart uint64) []byte
AppendGoodbye encodes a directory's goodbye table. dirEntryStart is the absolute offset of the directory's own TypeEntry record (i.e. after its filename record; for the root directory, the archive start), goodbyeStart the absolute offset where this goodbye record begins. Items may be in any order; they are sorted by hash and permuted into the BST layout. Item offsets are encoded as distances backwards from goodbyeStart.
func AppendHardlink ¶
AppendHardlink encodes a hardlink record: offset is the distance in bytes from the start of this hardlink's own TypeFilename record back to the start of the link target's TypeFilename record (the upstream encoder's current_position - target_position); target is the archive-relative path of the target (NUL-terminated).
func AppendPayloadHeader ¶
AppendPayloadHeader encodes the header of a TypePayload record. The size bytes of raw file content follow the header in the stream; they are written by the emitter, not by this package.
func AppendPayloadRef ¶
AppendPayloadRef encodes a v2 payload reference: offset addresses the payload record's *header* in the payload stream, size is the content byte count (the referenced record's length minus its header).
func AppendPayloadStartMarker ¶
AppendPayloadStartMarker encodes the record that opens a payload stream.
func AppendPayloadTailMarker ¶
AppendPayloadTailMarker encodes the record that closes a payload stream.
func AppendPrelude ¶ added in v0.3.0
AppendPrelude encodes the v2 prelude: opaque bytes following the format version record. proxmox-backup-client stores its exclude pattern list here (one pattern per line) and omits the record when there are none.
func AppendQuotaProjID ¶
AppendQuotaProjID encodes a filesystem quota project id.
func AppendSymlink ¶
AppendSymlink encodes a symlink target (NUL-terminated).
func AppendXAttr ¶
AppendXAttr encodes one extended attribute: name + NUL + value.
func DecodeMetadata ¶ added in v0.3.0
DecodeMetadata walks a pxar v1 archive or v2 metadata stream in order and calls visit for every regular file with its encoded metadata and payload reference. Payload content of a v1 archive is skipped. It returns the format version (1 or 2). Change detection uses it to index the previous snapshot's metadata stream.
func SizeFilename ¶
SizeFilename returns the encoded size of a TypeFilename record.
func SizeGoodbye ¶
SizeGoodbye returns the full encoded size of a goodbye record for a directory with n children (n items plus the tail marker).
func SizeHardlink ¶
SizeHardlink returns the encoded size of a TypeHardlink record.
func SizePayload ¶
SizePayload returns the full encoded size of a payload record whose content is size bytes: the header plus the raw content that follows it.
func SizePrelude ¶ added in v0.3.0
SizePrelude returns the encoded size of a TypePrelude record.
func SizeSymlink ¶
SizeSymlink returns the encoded size of a TypeSymlink record.
func ValidateFilename ¶
ValidateFilename reports whether name is usable as a pxar directory child name: non-empty, no '/', no NUL, and not "." or "..".
Types ¶
type ACLDefault ¶
type ACLDefault struct {
UserObjPermissions uint64
GroupObjPermissions uint64
OtherPermissions uint64
MaskPermissions uint64 // ACLNoMask when no mask entry exists
}
ACLDefault is the body of a TypeACLDefault record.
type ACLs ¶ added in v0.4.0
type ACLs struct {
Users []ACLUser // named users (access ACL)
Groups []ACLGroup // named groups (access ACL)
GroupObj *uint64 // owning-group permissions; set only when a mask entry exists
Default *ACLDefault // default ACL object permissions (directories)
DefaultUsers []ACLUser
DefaultGroups []ACLGroup
}
ACLs holds a node's POSIX ACL entries in encoder-ready form. A nil *ACLs means the node has no ACL records (a trivial ACL that only mirrors the mode bits does not count).
type Entry ¶
type Entry struct {
Mode uint64 // full st_mode: type bits + permission bits
Flags uint64
UID uint32
GID uint32
MtimeSecs int64 // seconds since epoch (may be negative)
MtimeNanos uint32 // nanoseconds within MtimeSecs
}
Entry is the stat record body (TypeEntry).
type FS ¶ added in v0.4.0
type FS struct {
// contains filtered or unexported fields
}
FS is a read-only, random-access view of an archive as an io/fs file system. It implements fs.FS, fs.ReadDirFS, fs.StatFS and fs.ReadLinkFS.
Paths are looked up through the directories' goodbye tables, so only the records on the way are read: over a remote index (see pbs OpenDynamicIndexAt) a lookup fetches a handful of chunks, not the archive. Listing a v2 directory reads the metadata stream only; payload records are checked when a file is opened.
Open and Stat follow symbolic links inside the archive; links with an absolute target, or a relative one leaving the archive, do not resolve. A hardlink resolves to the entry it links to, under its own name. FileInfo.Sys returns the *Node with the full metadata.
The file system is safe for concurrent use if its io.ReaderAt sources are.
func OpenV2 ¶ added in v0.4.0
OpenV2 opens a v2 split archive: the metadata stream of metaSize bytes and the payload stream. payload may be nil when only metadata is needed; opening a regular file then fails.
func (*FS) Lstat ¶ added in v0.4.0
Lstat returns a FileInfo for the named file without following a final symbolic link.
func (*FS) ReadDir ¶ added in v0.4.0
ReadDir reads the named directory, following symbolic links, and returns its entries sorted by name.
type GoodbyeItem ¶
GoodbyeItem describes one directory child for goodbye-table construction. Start and Length are in stream coordinates: Start is the absolute offset of the child's TypeFilename record, Length the child's total encoded length (filename record through goodbye table or payload, inclusive).
type MetaFile ¶ added in v0.3.0
type MetaFile struct {
// Path is the archive-relative path ("" for the root, "dir/name" below it).
Path string
// Meta is the file's ENTRY record followed by its metadata records
// (xattrs, ACLs, fcaps, quota project id), byte for byte as encoded.
Meta []byte
// Size is the content byte count.
Size uint64
// PayloadOffset is the position of the file's payload record header in
// the payload stream (v2 only; 0 in a v1 archive).
PayloadOffset uint64
}
MetaFile is one regular file found while decoding a pxar metadata stream.
type Node ¶ added in v0.4.0
type Node struct {
// Path is the archive-relative path ("" for the root, "dir/name" below it).
Path string
// Entry is the stat record. It is zero for a hardlink, which carries no
// metadata of its own.
Entry
// Hardlink marks a hardlink to an earlier entry of the archive.
Hardlink bool
// LinkTarget holds the symlink target, or for a hardlink the
// archive-relative path of the entry it links to.
LinkTarget string
Xattrs []Xattr
ACL *ACLs
FCaps []byte
QuotaProjID uint64 // 0 = none
Device Device // block and char devices
// Size is the content byte count of a regular file; Reader.Read returns
// exactly that many bytes.
Size uint64
// PayloadOffset is the position of the file's payload record header in
// the payload stream (v2 only).
PayloadOffset uint64
// contains filtered or unexported fields
}
Node is one archive entry as decoded by Reader.
type Reader ¶ added in v0.4.0
type Reader struct {
// contains filtered or unexported fields
}
Reader reads a pxar archive entry by entry, in archive order, like archive/tar: Next advances to the next entry, Read returns the content of the current regular file. Unread content is skipped by Next.
Every entry is validated as it is decoded: filenames and hardlink targets must be plain relative names, records must have their exact sizes, and a stream must end exactly after the root directory's goodbye table.
func NewReaderV1 ¶ added in v0.4.0
NewReaderV1 reads a pxar v1 archive.
func NewReaderV2 ¶ added in v0.4.0
NewReaderV2 reads a v2 split archive: the metadata stream (.mpxar) in order, and the payload stream (.ppxar) at the offsets the metadata stream references. payload may be nil when only metadata is needed; Read then fails.