archive

package
v0.33.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 5, 2026 License: Apache-2.0 Imports: 24 Imported by: 0

Documentation

Overview

Package archive: deterministic naming + crash-safe file I/O for the snapshot output tree.

Package archive provides deterministic naming helpers for the snapshot output tree. All functions are pure and free of I/O, except FindBlockData and ClassifyBlockPayload, which read a node directory's entries to locate an existing block-volume file.

Index

Constants

View Source 
const (
	// PAXFSCodec identifies the codec applied to one regular filesystem entry.
	PAXFSCodec = "D8.snapshot.fs.codec"
	// PAXFSOriginalPath preserves the source path independently of the stored codec suffix.
	PAXFSOriginalPath = "D8.snapshot.fs.originalPath"
	// PAXFSRawSize records the exact plaintext byte count before compression.
	PAXFSRawSize = "D8.snapshot.fs.rawSize"
)
View Source 
const (
	// SnapshotYAMLName is the filename for the per-node snapshot manifest plus checksum.
	SnapshotYAMLName = "snapshot.yaml"

	// ManifestsDirName is the subdirectory that holds own-scope manifests (always present).
	ManifestsDirName = "manifests"

	// SnapshotsDirName is the subdirectory that holds child node directories.
	// Present only when a node has children.
	SnapshotsDirName = "snapshots"

	// NodeIdentityMarkerName is the identity sidecar written into a node directory
	// on FIRST touch (WriteNodeIdentityMarker), before any chunk/staging/volume
	// data lands. It records the node's snapshot identity so a resume scan can
	// prove a PARTIAL (not-yet-finalized) directory belongs to the planned node —
	// snapshot.yaml, the only other identity record, is written just at finalize,
	// so without this marker a partial dir carries no identity and could be
	// silently resumed into by a DIFFERENT snapshot of the same source object.
	//
	// LIFECYCLE: the marker lives ONLY for the not-yet-finalized window. Once
	// snapshot.yaml is durably written, snapshot.yaml is the authoritative
	// identity record (every Done classification reads identity from it, not the
	// marker), so volume.FinalizeNode removes the marker strictly AFTER that
	// write — a crash at any earlier point leaves it in place, so a partial dir
	// always carries exactly one identity record (inv. #9 preserved). The two
	// Done=true scan branches (classifyCompleteDir and ScanAbsolute) also remove
	// a leftover marker via healNodeIdentityMarker, self-healing the crash window
	// between the snapshot.yaml write and the finalize remove, and archives from
	// older builds. This keeps a finalized node's on-disk layout to exactly
	// snapshot.yaml + manifests/ + optional snapshots/ + at most one volume
	// payload — no stray identity.json.
	//
	// It deliberately does NOT end in ".tmp", so resume.go's stale-*.tmp sweep
	// (removeTmpFiles) never touches it, and it is not one of the fixed file/dir
	// names ComputeNodeChecksum reads (manifests/, data.bin*, data.tar, data/),
	// so its presence never perturbs a node's checksum. At codec "none" (ext == "")
	// no user/server payload is named "identity.json": block payloads are
	// data.bin, FS payloads live only inside data.tar, and per-file entries are
	// tar members, never files in the node dir — so the name cannot collide with
	// user- or server-provided content (inv. #10a).
	NodeIdentityMarkerName = "identity.json"

	// DataBlockBase is the base filename (without codec extension) for the completed
	// block-volume output file. The actual filename is DataBlockName(codec.Ext()).
	DataBlockBase = "data.bin"

	// FsTarName is the output filename for a single-volume filesystem volume.
	// The tar container is uncompressed; each file entry inside is individually
	// compressed with the selected codec and named <path><ext> (ext empty for none).
	FsTarName = "data.tar"

	// FsTarStagingDirName is the temporary directory that holds raw per-file downloads
	// while a filesystem volume is being assembled. It lives next to data.tar inside
	// the node directory and is removed after the tar is assembled.
	FsTarStagingDirName = "data.tar.d"

	// FSMetaDirName is the reserved metadata subdirectory inside the FS staging
	// directory (FsTarStagingDirName). It holds the
	// download machinery's OWN internal artifacts — the per-file sizes sidecar and,
	// under FSChunksDirName, every per-file chunk directory. It is dot-prefixed and
	// clearly-internal, and the FS ingestion checkpoint (volume.sanitizeRelPath)
	// rejects any server-provided path whose FIRST segment equals it, so no
	// user/server file can ever stage into this namespace — including at codec none
	// (ext == ""), where a staged user blob is a plain file in the staging root.
	// Everything under it is thus provably disjoint from the server-provided
	// staged-blob namespace (stagingDir/<relPath><ext>) at EVERY codec (inv. #10a).
	// The SSOT for this name lives here; volume.FSMetaDirName aliases it.
	FSMetaDirName = ".d8-meta"

	// FSChunksDirName is the subdirectory under FSMetaDirName holding every
	// per-file chunk directory for a chunked FS file (see FsFileChunksDirName).
	// Placing chunk dirs here — rather than beside the staged blobs — guarantees a
	// chunk-dir path can never alias a staged user blob at codec none, so
	// MergeBlockChunks' post-merge os.RemoveAll(chunkDir) can never delete a
	// user's already-staged blob (inv. #10a).
	FSChunksDirName = "chunks"

	// FSFileChunksLeafBase is the fixed leaf directory name inside a per-file
	// chunk directory (see FsFileChunksDirName). Dot-prefixed and kept ending in
	// ".d" purely as a naming convention consistent with FsTarStagingDirName/
	// BlockChunksDirName — checksum exclusion does NOT depend on this suffix (see
	// FsFileChunksDirName doc for why).
	FSFileChunksLeafBase = ".d8-chunks"

	// DataDirName is the top-level directory for multi-volume output files.
	//
	// Multi-volume block files:       data/<pvc>.bin[.<ext>]
	// Multi-volume FS tar files:      data/<pvc>.tar
	// Multi-volume block staging:     data/<pvc>.bin.d/
	// Multi-volume FS staging:        data/<pvc>.tar.d/
	DataDirName = "data"

	// BlockChunksDirName is the temporary directory that holds individual block-volume
	// frames while the volume is being downloaded. It lives next to the merged output
	// file inside the node directory and is removed after the frames are merged.
	BlockChunksDirName = DataBlockBase + ".d"
)

Fixed names in the per-node directory layout.

View Source 
const (
	// ChecksumAlgorithmSHA256 is the only checksum algorithm the archive uses; it is the value
	// ComputeNodeChecksum records and ValidateSnapshotYAML requires in NodeChecksum.Algorithm.
	ChecksumAlgorithmSHA256 = "sha256"
	// SnapshotFormatVersionLegacy identifies archives written before explicit envelope versioning.
	// Version zero is accepted only through an explicit unauthenticated compatibility option.
	SnapshotFormatVersionLegacy = 0
	// SnapshotFormatVersionCurrent is written by every snapshot.yaml marshal. Version 2 adds
	// the mandatory authenticated direct-child commitment (ChildrenChecksum).
	SnapshotFormatVersionCurrent = 2
)
View Source 
const (
	VolumeModeBlock      = "Block"
	VolumeModeFilesystem = "Filesystem"
)

Volume modes recorded in VolumeInfo.VolumeMode. They mirror the corev1.PersistentVolumeMode values written by the download side (see volume.nodeDataToVolumeInfo) and MUST agree with the on-disk payload kind: a block payload (data.bin[.<ext>]) is "Block", a filesystem payload (data.tar) is "Filesystem". ValidateSnapshotYAML enforces that agreement.

View Source 
const ChunkMetaFileName = "chunks.meta"

ChunkMetaFileName is the sidecar filename recording the exact chunk geometry an in-progress chunk directory was produced with. It lives inside the chunk dir alongside the chunk_NNNNN[.<ext>] files it describes.

It is never hashed into a node checksum: collectNodeFiles never walks a single-volume flat chunk dir (BlockChunksDirName / FsFileChunksDirName both live outside the "data/" subtree it walks) and, for the multi-volume layout, that walk skips every directory whose name ends in ".d" — which every chunk dir does, by construction.

View Source 
const ChunkMetaMaxEncodedSize = 4 << 10

ChunkMetaMaxEncodedSize bounds metadata before JSON decoding.

Variables

View Source 
var (
	ErrArchiveLocked      = errors.New("snapshot archive is locked")
	ErrArchiveLockChanged = errors.New("snapshot archive lock binding changed")
)

ErrArchiveLocked is returned when an incompatible archive reader or writer owns the lock.

View Source 
var ErrArchiveMountBoundaryUnsupported = errors.New("archive mount-boundary verification unsupported")

ErrArchiveMountBoundaryUnsupported marks a platform or runtime that cannot prove an opened archive descendant remained on its parent's mount. Upload traversal fails closed in this case.

View Source 
var ErrAtomicReplaceUnsupported = errors.New("atomic replacement is unsupported on this platform")

ErrAtomicReplaceUnsupported reports that the platform cannot provide the AtomicWriter replacement contract without weakening its guarantees. Windows returns this error before publication when the final path exists.

View Source 
var ErrChecksumMismatch = errors.New("checksum mismatch")

ErrChecksumMismatch is returned when the recomputed checksum differs from the value recorded in snapshot.yaml.

View Source 
var ErrChildrenChecksumMismatch = errors.New("children checksum mismatch")

ErrChildrenChecksumMismatch is returned when a node's authenticated direct-child commitment does not match its physical direct children (a "hybrid tree": a child was added, removed, replaced, duplicated, or its digest no longer matches).

View Source 
var ErrChildrenMetadataBudgetExceeded = errors.New("aggregate direct-child metadata exceeds budget")

ErrChildrenMetadataBudgetExceeded is returned when the aggregate size of direct children's snapshot.yaml files exceeds maxChildrenMetadataBytes while computing a ChildrenChecksum.

View Source 
var ErrCorruptChunkMeta = errors.New("chunk metadata is corrupt")

ErrCorruptChunkMeta indicates ChunkMetaFileName exists but its contents could not be parsed as JSON — e.g. a torn write from a crash mid-write (see WriteChunkMeta's use of WriteFileAtomic, which makes this rare but not impossible) or filesystem-level corruption. Callers MUST treat this identically to a geometry mismatch (purge the chunk dir and re-download), never as a fatal error: the sidecar's only purpose is recording the byte ranges chunks were written under, and an unparseable sidecar means those ranges can no longer be trusted.

View Source 
var ErrIdentityMismatch = errors.New("output directory belongs to a different snapshot")

ErrIdentityMismatch is returned by ScanAbsolute when the target directory contains a complete snapshot whose stored identity does not match the planned node. The caller must choose a different output path rather than overwriting the data.

View Source 
var ErrInvalidBlockPayload = errors.New("invalid block payload")

ErrInvalidBlockPayload is returned by ClassifyBlockPayload when a node directory's data.bin*-prefixed contents do not resolve to exactly one recognized block payload: an unrecognized or chained suffix, more than one block file, or a block file coexisting with the filesystem volume (FsTarName).

View Source 
var ErrInvalidFSMetadata = errors.New("invalid filesystem tar metadata")

ErrInvalidFSMetadata is returned when a regular filesystem tar entry does not carry a complete, internally consistent set of d8 PAX metadata.

View Source 
var ErrInvalidSnapshotYAML = errors.New("invalid snapshot.yaml")

ErrInvalidSnapshotYAML is returned by ValidateSnapshotYAML/ValidateNodeMetadata when a node's snapshot.yaml violates a structural metadata invariant.

View Source 
var ErrLegacySnapshotFormat = errors.New(
	"legacy snapshot.yaml metadata is unauthenticated; explicit compatibility mode is required",
)

ErrLegacySnapshotFormat is returned when an unversioned archive is read without explicit permission to trust its unauthenticated snapshot.yaml metadata.

View Source 
var ErrNonRegularArchiveArtifact = errors.New("non-regular archive artifact")

ErrNonRegularArchiveArtifact marks an archive path whose host-filesystem type is unsafe.

View Source 
var ErrSnapshotMetadataChecksumMismatch = errors.New("snapshot.yaml metadata checksum mismatch")

ErrSnapshotMetadataChecksumMismatch is returned when versioned snapshot.yaml metadata differs from the canonical metadata digest recorded when the archive was written.

View Source 
var ErrSnapshotYAMLMissing = errors.New("snapshot.yaml not found")

ErrSnapshotYAMLMissing is returned when snapshot.yaml does not exist in a node directory.

View Source 
var ErrTooManyDirectChildren = errors.New("direct child count exceeds bound")

ErrTooManyDirectChildren is returned when a node's snapshots/ directory carries more than maxDirectChildren entries.

View Source 
var ErrUnsupportedSnapshotFormat = errors.New("unsupported snapshot.yaml format version")

ErrUnsupportedSnapshotFormat is returned when snapshot.yaml declares an unknown format version.

View Source 
var ErrVerifiedArchiveChanged = errors.New("verified archive view changed")

ErrVerifiedArchiveChanged is returned when a file or directory differs from the exact identity and content captured by archive verification.

Functions

func ChunkFileName 

func ChunkFileName(i int, ext string) string

ChunkFileName returns the filename for block-volume chunk index i inside BlockChunksDirName. Indices are zero-padded to five digits. ext is the codec extension (e.g. ".zst", ".lz4", ".gz", or "" for the none codec). Examples: ChunkFileName(0, ".zst") → "chunk_00000.zst", ChunkFileName(3, "") → "chunk_00003".

func CollisionNodeDir 

func CollisionNodeDir(parentDir, kind, name, short string) string

CollisionNodeDir returns the path for a node directory with a short-checksum suffix: 

<parentDir>/<NodeDirName(kind,name)>__<short>

Use this when the primary directory already holds complete data for a different snapshot and the new node's own short checksum disambiguates the two.

func ConfirmFileDurability 

func ConfirmFileDurability(ctx context.Context, path string) error

ConfirmFileDurability applies the platform durability confirmation before an already published final file is trusted. Unix syncs the parent directory. A successful Windows AtomicWriter create publication is write-through, so no separate supported directory operation exists or is required. Cancellation observed before confirmation prevents it from starting; once it starts, its result wins.

func ConfirmRootedFileDurability 

func ConfirmRootedFileDurability(ctx context.Context, destination *RootedDestination, path string) error

ConfirmRootedFileDurability confirms a published file through destination.

func DataBlockName 

func DataBlockName(ext string) string

DataBlockName returns the output filename for a block-volume with the given codec extension. ext is codec.Ext() (e.g. ".zst", ".lz4", ".gz", or "" for none). Examples: DataBlockName(".zst") → "data.bin.zst", DataBlockName("") → "data.bin".

func EnsureDir 

func EnsureDir(path string) error

EnsureDir creates path and all parents with the platform durability contract. Unix persists every containing-directory entry back to the filesystem root. Windows has no documented unprivileged directory-flush API, so directory creation cannot be given the same explicit POSIX durability guarantee.

func FSCodecExtension 

func FSCodecExtension(codec string) (string, error)

FSCodecExtension maps a validated filesystem codec name to its stored suffix.

func FindBlockData 

func FindBlockData(nodeDir string) (string, bool, error)

FindBlockData searches nodeDir for a completed block-volume file (any file whose name starts with DataBlockBase, excluding the staging directory DataBlockBase+".d"). The first non-directory match is returned as an absolute path. The second return value is false when no such file exists. An I/O error is returned in the third return value.

func FsFileChunksDirName 

func FsFileChunksDirName(relPath, ext string) string

FsFileChunksDirName returns the per-file chunk directory path for one filesystem-volume file, RELATIVE to the FS staging directory: ".d8-meta/chunks/<relPath>/<leaf>", where <leaf> is FsFileChunksLeafName(ext). relPath is the item's forward-slash relative path within the volume (e.g. "disk/payload.bin") and ext is the codec extension (e.g. ".zst", or "" for the none codec).

An EARLIER version of this function suffixed the FILE's own name instead ("chunks/<relPath><ext>.d"). At codec none (ext == "") that made the synthesized chunk-dir path for a file "x" literally "chunks/x.d" — which collides with a perfectly ordinary volume layout: a file "x" alongside a REAL directory "x.d/" holding further files (the standard unix conf.d pattern, e.g. "/etc/sudoers" + "/etc/sudoers.d/override"). Once that happens, "x"'s chunk dir sits ON THE PATH of "x.d/override"'s own chunk dir ("chunks/x.d/override.d"), so processing "x" alone can destroy "override"'s in-progress chunk work: MergeBlockChunks' post-merge os.RemoveAll(chunkDir) deletes "chunks/x.d" wholesale once "x" merges, and ensureChunkGeometry's stale-geometry purge does the same the moment it sees "chunks/x.d" already exists without its OWN chunks.meta at that level. Both failures are fail-closed (the affected file just re-downloads after a loud ErrMissingChunk-class error), but the collision is real and not limited to concurrent merges.

Joining "/" + relPath (rather than appending to relPath) is what makes the NEW path scheme collision-free: for one file's chunk dir to be an ancestor of another's, relPath would have to name BOTH an ordinary file (so it has its own chunk dir) and a directory (so something nests under it) at the same time — impossible on any filesystem, since a file has no children. That is a strictly weaker precondition to break than the old scheme's ("no volume directory is named exactly <some file>ext + \".d\""), which the ordinary sudoers/sudoers.d/ layout above violates outright. The leaf still ends in ".d" — see FSFileChunksLeafBase — but only as a naming convention; nothing here or in checksum.go (collectNodeFiles never walks the staging dir at all; collectLegacyDataFiles skips the WHOLE top-level staging directory by its own ".d" suffix and never recurses into it) depends on that suffix for correctness.

A caller joins this (via filepath.FromSlash) under the FS staging directory (FsTarStagingDirName). Chunks accumulate under the leaf while a known-size file is downloaded via Range GETs and are merged into "<relPath><ext>" (the same path DownloadBlockChunks/MergeBlockChunks use for a single block volume) once complete — which also removes the leaf directory itself. The intermediate directory "chunks/<relPath>/" this adds is NOT removed at that point (only its leaf child was); it is left empty and cleaned up only when the FS staging directory's own lifecycle ends (removed whole once the volume's tar is assembled). In-flight chunk dirs from trees written before this relocation (either the original flat "stagingDir/<relPath><ext>.d", or this function's own earlier "chunks/<relPath><ext>.d" form) are simply abandoned — such a file re-downloads once.

func FsFileChunksLeafName 

func FsFileChunksLeafName(ext string) string

FsFileChunksLeafName returns the fixed leaf directory name for a per-file chunk directory at the given codec extension. ext is retained in the leaf name so switching codecs between runs never mixes frames from two codecs in the same directory.

func ManifestFileName 

func ManifestFileName(kind, name, apiGroup string) string

ManifestFileName returns the filename for a single Kubernetes manifest in manifests/.

  • Normal form (no collision): "<kindlower>_<name>.yaml"
  • Collision fallback (same kind+name, different API groups): "<kindlower>.<apiGroup>_<name>.yaml"

Pass an empty apiGroup for the normal (non-collision) form.

func NodeDirName 

func NodeDirName(kind, name string) string

NodeDirName returns the directory name for a child snapshot node. The name is "<kindlower>_<name>" per the directory-tree layout rules. For the root node, callers use the user-supplied output directory name directly.

func OpenRegularFile 

func OpenRegularFile(path string) (*os.File, error)

OpenRegularFile rejects unsafe path components and opens the final regular file relative to a pinned descriptor for its parent directory.

func ReadDirectory 

func ReadDirectory(path string) ([]os.DirEntry, error)

ReadDirectory returns entries from a pinned descriptor opened as a real directory.

func ShortChecksum 

func ShortChecksum(hex string) string

ShortChecksum returns the first 8 hex characters of hex. The short form is used as a suffix when a node directory name already exists with a different checksum, preventing silent data overwrite.

func ValidateNodeMetadata 

func ValidateNodeMetadata(nodeDir string) error

ValidateNodeMetadata reads nodeDir's snapshot.yaml and strictly validates its envelope and metadata via ValidateSnapshotYAML, deriving the node's data-payload flags from the directory itself (ClassifyBlockPayload for data.bin[.<ext>], OpenRegularFile for data.tar). It complements VerifyNode's content checksum. Returns ErrSnapshotYAMLMissing when snapshot.yaml is absent, and propagates ClassifyBlockPayload's ErrInvalidBlockPayload for a malformed payload.

func ValidateNodeMetadataWithOptions 

func ValidateNodeMetadataWithOptions(nodeDir string, options SnapshotYAMLReadOptions) error

ValidateNodeMetadataWithOptions validates one node under an explicit compatibility policy.

func ValidateSnapshotYAML 

func ValidateSnapshotYAML(sy SnapshotYAML, hasBlockData, hasFilesystemData bool) error

ValidateSnapshotYAML strictly validates the current authenticated snapshot.yaml envelope and structural metadata.

hasBlockData and hasFilesystemData report the node's on-disk volume payload (data.bin[.<ext>] and data.tar respectively); ValidateNodeMetadata derives them from the directory. A node is a data node when it carries either payload. The rules:

  • apiVersion, kind and name are required.
  • checksum.algorithm is "sha256", checksum.hex is 64 lowercase hex chars, and checksum.short is the first 8 chars of hex (ShortChecksum).
  • sourceObjectRef is all-or-nothing: omitted, or all of apiVersion/kind/name set.
  • at most one volume (Variant A cardinality, decision #9).
  • a data node carries exactly one volume with a complete target and artifact identity (apiVersion/kind/name each), a storageClassName, a positive parseable size, and a volumeMode that agrees with the payload kind (Block for data.bin, Filesystem for data.tar).
  • a non-data node carries no volume.

func ValidateSnapshotYAMLWithOptions 

func ValidateSnapshotYAMLWithOptions(
	sy SnapshotYAML,
	hasBlockData, hasFilesystemData bool,
	options SnapshotYAMLReadOptions,
) error

ValidateSnapshotYAMLWithOptions validates snapshot.yaml under an explicit compatibility policy.

func VerifyNode 

func VerifyNode(nodeDir string) error

VerifyNode validates snapshot.yaml's versioned metadata checksum, then recomputes the node content checksum and compares it with the stored value. Returns ErrSnapshotYAMLMissing if snapshot.yaml is absent, ErrSnapshotMetadataChecksumMismatch if versioned metadata differs, and ErrChecksumMismatch if the content digests differ.

func VerifyNodeChildrenChecksumRooted 

func VerifyNodeChildrenChecksumRooted(source *RootedSource) error

VerifyNodeChildrenChecksumRooted verifies one pinned node's authenticated direct-child commitment without reading any child payload bytes.

func VerifyNodeChildrenChecksumRootedWithOptions 

func VerifyNodeChildrenChecksumRootedWithOptions(source *RootedSource, options SnapshotYAMLReadOptions) error

VerifyNodeChildrenChecksumRootedWithOptions verifies one pinned node under an explicit envelope compatibility policy; the childrenChecksum commitment itself remains mandatory.

func VerifyNodeWithOptions 

func VerifyNodeWithOptions(nodeDir string, options SnapshotYAMLReadOptions) error

VerifyNodeWithOptions verifies one node under an explicit snapshot.yaml compatibility policy.

func WithDirectorySyncHook 

func WithDirectorySyncHook(ctx context.Context, hook DirectorySyncHook) context.Context

WithDirectorySyncHook returns a context that applies hook to AtomicWriter.CommitContext and ConfirmFileDurability confirmations.

func WriteChunkMeta 

func WriteChunkMeta(dir string, meta ChunkMeta) error

WriteChunkMeta atomically writes meta as ChunkMetaFileName inside dir. dir must already exist.

func WriteFileAtomic 

func WriteFileAtomic(path string, r io.Reader) error

WriteFileAtomic is WriteFileAtomicContext with a non-cancellable context.

func WriteFileAtomicContext 

func WriteFileAtomicContext(ctx context.Context, path string, r io.Reader) error

WriteFileAtomicContext copies r into path using an AtomicWriter. Pre-publication errors remove the temporary file and leave the old final unchanged. A PublicationPublished error means the complete final file is visible but its parent-directory durability remains unconfirmed.

func WriteFileAtomicRooted 

func WriteFileAtomicRooted(
	ctx context.Context,
	destination *RootedDestination,
	path string,
	reader io.Reader,
) error

WriteFileAtomicRooted copies r into path beneath destination.

func WriteManifest 

func WriteManifest(nodeDir string, obj unstructured.Unstructured) error

WriteManifest serialises obj as uncompressed YAML and writes it atomically into <nodeDir>/manifests/. The filename is determined by ManifestFileName:

  • Normal: <kindlower>_<name>.yaml.
  • Collision fallback: if a file with the same kind/name already exists but belongs to a different API group, <kindlower>.<apiGroup>_<name>.yaml is used instead.

Rewriting the same object (same kind, name, and API group) is idempotent.

func WriteManifestRooted 

func WriteManifestRooted(
	destination *RootedDestination,
	nodeDir string,
	obj unstructured.Unstructured,
) error

WriteManifestRooted writes a manifest through destination's locked view.

func WriteNodeIdentityMarker 

func WriteNodeIdentityMarker(dir string, id NodeIdentity) error

WriteNodeIdentityMarker writes the identity marker for id into dir, but ONLY when no marker is already present. The marker records the FIRST toucher's identity — precisely the identity a later resume must match — so an existing marker is left untouched and this is safe to call on every reconcile of the same node. The write is crash-safe (WriteFileAtomic: .tmp -> fsync -> rename -> dir fsync).

func WriteNodeIdentityMarkerRooted 

func WriteNodeIdentityMarkerRooted(
	ctx context.Context,
	destination *RootedDestination,
	dir string,
	id NodeIdentity,
) error

WriteNodeIdentityMarkerRooted writes the marker through destination.

func WriteSnapshotYAML 

func WriteSnapshotYAML(nodeDir string, sy SnapshotYAML) error

WriteSnapshotYAML is WriteSnapshotYAMLContext with a non-cancellable context.

func WriteSnapshotYAMLContext 

func WriteSnapshotYAMLContext(ctx context.Context, nodeDir string, sy SnapshotYAML) error

WriteSnapshotYAMLContext serialises sy to YAML and writes it atomically to <nodeDir>/snapshot.yaml. An existing file at that path is replaced.

func WriteSnapshotYAMLRooted 

func WriteSnapshotYAMLRooted(
	ctx context.Context,
	destination *RootedDestination,
	nodeDir string,
	snapshot SnapshotYAML,
) error

WriteSnapshotYAMLRooted writes snapshot.yaml atomically through destination.

Types

type AtomicWriter 

type AtomicWriter struct {
	// contains filtered or unexported fields
}

AtomicWriter writes data to "<finalPath>.tmp" and syncs it before publication. Unix atomically replaces the final path, then syncs its parent directory. Windows durably creates a previously absent final with a write-through move, but fails with ErrAtomicReplaceUnsupported before replacing an existing final: documented Windows APIs do not combine atomic replacement with write-through namespace durability on every supported filesystem. Call Abort to remove the temporary file when an error occurs.

func NewAtomicWriter 

func NewAtomicWriter(path string) (*AtomicWriter, error)

NewAtomicWriter creates (or truncates) "<path>.tmp" and returns a writer ready to receive data. The caller must call either Commit or Abort.

func NewRootedAtomicWriter 

func NewRootedAtomicWriter(destination *RootedDestination, path string) (*AtomicWriter, error)

NewRootedAtomicWriter creates an AtomicWriter beneath destination.

func (*AtomicWriter) Abort 

func (w *AtomicWriter) Abort()

Abort closes and removes the temporary file. Safe to call even if Write returned an error. Errors from close/remove are intentionally suppressed because the caller's original error takes precedence.

func (*AtomicWriter) Commit 

func (w *AtomicWriter) Commit() error

Commit is CommitContext with a non-cancellable context. After Commit the AtomicWriter must not be used again.

func (*AtomicWriter) CommitContext 

func (w *AtomicWriter) CommitContext(ctx context.Context) error

CommitContext syncs and closes the temporary file, checks cancellation, and publishes it according to the platform contract documented on AtomicWriter.

Publication begins at the cancellation checkpoint immediately before Rename. Cancellation observed before that point removes the temporary file and leaves the final path unchanged. Once the checkpoint succeeds, cancellation no longer changes the result: publication and its platform-specific durability confirmation determine the return value, so CommitContext never reports pre-publication cancellation after publishing.

func (*AtomicWriter) OpenTempReader 

func (w *AtomicWriter) OpenTempReader() (io.ReadCloser, error)

OpenTempReader opens the unpublished temporary file for validation. The caller must close the returned reader before calling Commit or Abort.

func (*AtomicWriter) Write 

func (w *AtomicWriter) Write(p []byte) (int, error)

Write implements io.Writer.

type AuthenticatedReadStats 

type AuthenticatedReadStats struct {
	ChunkSize   int64
	SourceBytes int64
	HashedBytes int64
	ChunkLoads  int64
	CacheHits   int64
	Resets      int64
}

AuthenticatedReadStats reports chunk authentication performed while serving Read and ReadAt. It excludes the intentional full-file scans that build the index and perform final Verify.

type BlockPayload 

type BlockPayload struct {
	// Path is the absolute path to the payload file.
	Path string
	// Ext is the payload's codec extension: "" (raw/none codec), ".zst",
	// ".gz", or ".lz4" — matching compress.Codec.Ext. Callers MUST use this
	// value and never re-derive it via filepath.Ext(Path): filepath.Ext on
	// the raw name "data.bin" returns ".bin" (the base name's own suffix,
	// since "data.bin" has no separate codec suffix of its own), not "" —
	// exactly the bug this field exists to prevent downstream.
	Ext string
}

BlockPayload identifies the single block-volume payload resolved by ClassifyBlockPayload for one node directory.

func ClassifyBlockPayload 

func ClassifyBlockPayload(nodeDir string) (BlockPayload, bool, error)

ClassifyBlockPayload resolves nodeDir's block-volume payload strictly against the fixed data.bin[.<ext>] allow-list (blockPayloadExts), replacing the earlier first-glob-match behavior of FindBlockData. It is the single classifier ComputeNodeChecksum and snapimport.BuildPlan both call, so a node's checksum and its upload always agree on what "the block payload" is.

Accepted names (exactly): "data.bin" (ext ""), "data.bin.zst", "data.bin.gz", "data.bin.lz4". The staging directory (BlockChunksDirName, "data.bin.d") is the only directory entry ignored. Every other entry whose name starts with DataBlockBase — an unrecognized codec suffix ("data.bin.foo"), a chained suffix ("data.bin.zst.bak"), or any directory other than the staging dir — is rejected as ErrInvalidBlockPayload rather than silently skipped: silently ignoring it would mean the checksum or the upload picks a DIFFERENT file than a human inspecting the directory would expect, or drops volume bytes outright. More than one recognized block file, or a recognized block file coexisting with the filesystem volume (FsTarName, "data.tar"), is rejected for the same reason — a node owns AT MOST ONE volume payload.

Returns (BlockPayload{}, false, nil) when nodeDir carries no block payload at all (not an error: the normal shape for a filesystem-volume or purely structural node, and for a nodeDir that does not exist yet).

func ClassifyBlockPayloadIn 

func ClassifyBlockPayloadIn(source archiveDirectory) (BlockPayload, bool, error)

ClassifyBlockPayloadIn resolves a block payload relative to a pinned node source.

type ChildCommitment 

type ChildCommitment struct {
	APIVersion       string
	Kind             string
	Name             string
	Namespace        string
	UID              string
	NodeChecksum     NodeChecksum
	ChildrenChecksum NodeChecksum
}

ChildCommitment is the canonical record committed by a parent's ChildrenChecksum for one direct child: the child's identity plus its own recursively-authenticated digests. Chaining NodeChecksum (the child's content) and ChildrenChecksum (the child's own direct children) makes a single ChildrenChecksum comparison at a node cover only that node's direct children; full-tree authentication requires verifying every node's own commitment (see snapimport.verifyCommitmentTree / archive.VerifyNodeWithOptions applied recursively).

type ChunkMeta 

type ChunkMeta struct {
	ChunkSize int64 `json:"chunkSize"`
	TotalSize int64 `json:"totalSize"`
}

ChunkMeta records the chunk geometry — chunk size and total volume/file size — that an in-progress chunk directory was produced with. Chunk k's byte range is computed purely from these two values ([k*chunkSize, min((k+1)*chunkSize,totalSize))), and neither is encoded in ChunkFileName (only the index and codec extension are) — so a resumed download with a different chunkSize would otherwise silently reuse a chunk file that covers the wrong byte range. Comparing against the geometry a chunk dir was actually created with is the only reliable way to detect that.

func ReadChunkMeta 

func ReadChunkMeta(dir string) (ChunkMeta, bool, error)

ReadChunkMeta reads ChunkMetaFileName from dir. found is false (with a nil error) when the metadata file does not exist — the valid case for a chunk dir that predates this guard or was never fully initialized, which callers must treat as an untrusted/incompatible geometry, not as "no geometry recorded yet, anything goes".

func ReadChunkMetaFrom 

func ReadChunkMetaFrom(ctx context.Context, reader io.Reader, source string) (ChunkMeta, bool, error)

ReadChunkMetaFrom decodes bounded chunk metadata from an already-secured reader.

type CommitError 

type CommitError struct {
	// contains filtered or unexported fields
}

CommitError preserves the commit operation's original cause and records whether the final path was published before that operation failed.

func (*CommitError) Error 

func (e *CommitError) Error() string

Error implements error.

func (*CommitError) PublicationState 

func (e *CommitError) PublicationState() PublicationState

PublicationState returns the final-path state at the failure boundary.

func (*CommitError) Unwrap 

func (e *CommitError) Unwrap() error

Unwrap exposes the original commit failure for errors.Is/errors.As.

type DirectorySyncHook 

type DirectorySyncHook func(path string, next func() error) error

DirectorySyncHook wraps a platform durability confirmation. Calling next performs the real confirmation: a parent-directory sync on Unix and the post-write-through no-op on Windows. The hook is scoped to a context so deterministic operation injection does not affect concurrent writers.

type FSMetadata 

type FSMetadata struct {
	Codec        string
	OriginalPath string
	RawSize      int64
}

FSMetadata is the checksum-covered format contract for one regular data.tar entry.

func NewFSMetadata 

func NewFSMetadata(codec, originalPath string, rawSize int64) (FSMetadata, error)

NewFSMetadata validates and constructs metadata for a new regular entry.

func ParseFSMetadata 

func ParseFSMetadata(hdr *tar.Header) (FSMetadata, error)

ParseFSMetadata strictly decodes the required PAX records from hdr and verifies that hdr.Name is the canonical stored path for the declared codec.

func (FSMetadata) PAXRecords 

func (m FSMetadata) PAXRecords() map[string]string

PAXRecords returns a fresh map containing the three required records.

func (FSMetadata) StoredPath 

func (m FSMetadata) StoredPath() (string, error)

StoredPath returns the canonical tar member name for this metadata.

type Lock 

type Lock struct {
	// contains filtered or unexported fields
}

Lock is a held cooperative archive lock bound to one stable path-domain record, pinned root, and descriptor-relative lock inode. Callers must Unlock it. A lock acquired from an existing RootedSource does not close that source; a path-based acquisition owns and closes its internally opened source.

func AcquireReadLock 

func AcquireReadLock(root string) (*Lock, error)

AcquireReadLock takes a non-blocking shared lock. Multiple uploads may coexist, while an upload and a download writer exclude one another.

func AcquireReadLockContext 

func AcquireReadLockContext(ctx context.Context, root string) (*Lock, error)

AcquireReadLockContext is AcquireReadLock with cancellation propagated through acquisition.

func AcquireRootedReadLock 

func AcquireRootedReadLock(ctx context.Context, source *RootedSource) (*Lock, error)

AcquireRootedReadLock takes a shared lock through source's already-pinned root descriptor. The source and lock become one namespace-bound view until Unlock returns.

func AcquireWriteLock 

func AcquireWriteLock(root string) (*Lock, error)

AcquireWriteLock takes a non-blocking exclusive lock used by archive writers.

func AcquireWriteLockContext 

func AcquireWriteLockContext(ctx context.Context, root string) (*Lock, error)

AcquireWriteLockContext is AcquireWriteLock with cancellation propagated through acquisition.

func (*Lock) Unlock 

func (l *Lock) Unlock() error

Unlock releases the lock entry, then the root lock, and closes every owned handle. It is idempotent. The harmless regular lock file remains in the archive for future processes.

func (*Lock) Verify 

func (l *Lock) Verify() error

Verify proves that the external domain, current root name, and in-root lock entry still identify the pinned handles.

type MutationBoundaryHook 

type MutationBoundaryHook func(phase MutationPhase, path string)

MutationBoundaryHook makes rooted mutation races deterministic in tests.

type MutationPhase 

type MutationPhase string

MutationPhase identifies a rooted destination operation boundary. Hooks run before namespace verification and before the operation can mutate the pinned tree. 

const (
	MutationCreate  MutationPhase = "create"
	MutationMkdir   MutationPhase = "mkdir"
	MutationRemove  MutationPhase = "remove"
	MutationRename  MutationPhase = "rename"
	MutationSync    MutationPhase = "sync"
	MutationStat    MutationPhase = "stat"
	MutationOpen    MutationPhase = "open"
	MutationReadDir MutationPhase = "read_dir"
)

type NodeChecksum 

type NodeChecksum struct {
	// Algorithm is always "sha256".
	Algorithm string `json:"algorithm"`
	// Hex is the full lowercase hex-encoded SHA-256 digest.
	Hex string `json:"hex"`
	// Short is the first 8 characters of Hex, used as a collision-suffix when
	// a node directory with the same name already exists with a different checksum.
	Short string `json:"short"`
}

NodeChecksum is a locally-computed integrity digest. SnapshotYAML.Checksum covers the node's manifests and volume data; SnapshotYAML.MetadataChecksum covers its canonical envelope fields.

func ComputeChildrenChecksum 

func ComputeChildrenChecksum(commitments []ChildCommitment) (NodeChecksum, error)

ComputeChildrenChecksum computes the canonical authenticated digest committing to an exact set of direct-child identities and digests. It is deterministic regardless of input order: commitments are canonicalized by sorting on the full (apiVersion, kind, namespace, name, uid) identity tuple before hashing. It rejects more than maxDirectChildren commitments, any commitment with an incomplete identity or malformed checksum, and duplicate identities (ErrInvalidSnapshotYAML).

func ComputeNodeChecksum 

func ComputeNodeChecksum(nodeDir string) (NodeChecksum, error)

ComputeNodeChecksum computes a deterministic SHA-256 digest over the node's own files.

Covered files (in sorted-relpath order):

  • manifests/*.yaml
  • data.bin[.<ext>] (block volume, single-volume flat layout, if present)
  • data.tar (filesystem volume, single-volume flat layout, if present)
  • data/<pvc>.bin[.<ext>] / data/<pvc>.tar (multi-volume layout, if data/ present)

Excluded: snapshot.yaml itself and the snapshots/ child directory. Versioned snapshot.yaml semantic fields are covered separately by SnapshotYAML.MetadataChecksum.

Each file contributes its relative path (null-terminated) followed by its raw content to an independent per-file SHA-256. All per-file digests are then fed in sorted order into a final SHA-256 to produce the node checksum.

func ComputeNodeChildrenChecksum 

func ComputeNodeChildrenChecksum(nodeDir string) (NodeChecksum, error)

ComputeNodeChildrenChecksum computes nodeDir's commitment from its physical direct children.

func ComputeNodeChildrenChecksumRooted 

func ComputeNodeChildrenChecksumRooted(source *RootedSource) (NodeChecksum, error)

ComputeNodeChildrenChecksumRooted computes a commitment through an already pinned source.

func EmptyChildrenChecksum 

func EmptyChildrenChecksum() NodeChecksum

EmptyChildrenChecksum is the canonical ChildrenChecksum committed by a node with no direct children (a leaf, or an aggregator whose snapshots/ directory is absent or empty).

type NodeIdentity 

type NodeIdentity struct {
	APIVersion string
	Kind       string
	// Name is the CR metadata.name used for resume identity matching (stored in
	// snapshot.yaml and compared by matchesIdentity). It is NOT the on-disk dir name.
	Name string
	// DirName is the on-disk directory-name component: NodeDirName(Kind, DirName).
	// For domain snapshot nodes it is the source-ref .name (the captured object name);
	// for orphan leaf volume nodes it is the captured PVC name.
	// When empty, Name is used as the fallback (root nodes that use ScanAbsolute
	// with a user-supplied path and domain nodes without a source annotation).
	DirName   string
	Namespace string
	// UID is the snapshot CR's metadata.uid. It is the identity component that ties a
	// directory to the exact snapshot CR (Variant A: readable dir base from source name,
	// uniqueness/resume identity from the CR identity incl UID). matchesIdentity and the
	// collision discriminator use it.
	UID string
}

NodeIdentity describes the planned identity of a snapshot node. It is used for collision detection: if a complete primary directory holds data for a different identity, the new node is redirected to a collision-suffixed path.

type NodeIdentityMarker 

type NodeIdentityMarker struct {
	APIVersion string `json:"apiVersion"`
	Kind       string `json:"kind"`
	Name       string `json:"name"`
	Namespace  string `json:"namespace,omitempty"`
	UID        string `json:"uid,omitempty"`
}

NodeIdentityMarker is the on-disk identity sidecar (NodeIdentityMarkerName) written into a node directory on first touch. Its fields are exactly the identity fields matchesIdentity compares (the on-disk DirName is intentionally excluded — it is a naming detail, not an identity).

func ReadNodeIdentityMarker 

func ReadNodeIdentityMarker(dir string) (NodeIdentityMarker, bool, error)

ReadNodeIdentityMarker reads the identity marker from dir. found is false with a nil error when the marker is absent.

type NodeResumePlan 

type NodeResumePlan struct {
	// TargetDir is the absolute path to use for this node.  For a collision-
	// redirected node this will be CollisionNodeDir(...) rather than the primary
	// directory.
	TargetDir string

	// Done is the ONLY resume decision the pipeline consumes: true means the node
	// directory already holds a complete, identity-verified download whose
	// snapshot.yaml directory durability was confirmed, so the pipeline skips it
	// entirely. Every not-done node is (re)driven through the normal download path,
	// which re-proves what to (re)fetch from disk probes — those probes, NOT this
	// plan, are the single source of truth for resume.
	Done bool

	// Observed is a NON-AUTHORITATIVE label of what the scan saw on disk (see
	// ObservedState). It is log-only and never an input to any resume decision.
	Observed ObservedState
}

NodeResumePlan is the result of scanning one planned node on disk.

func ScanAbsolute 

func ScanAbsolute(nodeDir string, id NodeIdentity) (NodeResumePlan, error)

ScanAbsolute is ScanAbsoluteContext with a non-cancellable context.

func ScanAbsoluteContext 

func ScanAbsoluteContext(ctx context.Context, nodeDir string, id NodeIdentity) (NodeResumePlan, error)

ScanAbsoluteContext classifies the on-disk state of an absolute node directory path, removing stale *.tmp files. Unlike ScanNode it does not derive the path from a parent directory + NodeDirName convention, and it does not redirect to a collision-suffixed path on identity mismatch. Instead it returns ErrIdentityMismatch so the caller can abort and ask the user to choose a different output path.

Suitable for the root output directory where the path name is user-controlled.

func ScanAbsoluteRootedContext 

func ScanAbsoluteRootedContext(
	ctx context.Context,
	destination *RootedDestination,
	nodeDir string,
	id NodeIdentity,
) (NodeResumePlan, error)

ScanAbsoluteRootedContext is ScanAbsoluteContext rooted in destination.

func ScanNode 

func ScanNode(parentDir string, id NodeIdentity) (NodeResumePlan, error)

ScanNode is ScanNodeContext with a non-cancellable context.

func ScanNodeContext 

func ScanNodeContext(ctx context.Context, parentDir string, id NodeIdentity) (NodeResumePlan, error)

ScanNodeContext inspects parentDir for an existing node directory whose name is NodeDirName(id.Kind, nodeDirComponent(id)), removes any stale *.tmp files, and returns a NodeResumePlan describing the on-disk state for the planned node.

The directory name is derived from id.DirName (the source object name) when set, falling back to id.Name (the CR name) for nodes without a source annotation. Identity matching (matchesIdentity) uses id.APIVersion/Kind/Name/Namespace/UID, which are the values written into snapshot.yaml.

Collision rule: if the primary directory is complete (VerifyNode passes) but its stored identity does not match id, the primary directory belongs to a different node. ScanNode returns a not-done plan with TargetDir set to CollisionNodeDir(parentDir, id.Kind, nodeDirComponent(id), short), where short is derived from the existing complete node's checksum. This prevents the pipeline from overwriting unrelated completed data.

func ScanNodeRootedContext 

func ScanNodeRootedContext(
	ctx context.Context,
	destination *RootedDestination,
	parentDir string,
	id NodeIdentity,
) (NodeResumePlan, error)

ScanNodeRootedContext is ScanNodeContext rooted in destination's locked view.

type ObservedState 

type ObservedState string

ObservedState is a human-readable, NON-AUTHORITATIVE label describing what the resume scan saw on disk for a planned node directory. It exists solely for log output (the pipeline's "resume_state" attribute) so an operator can see how a node was classified; it MUST NOT drive any resume decision.

The pipeline re-proves every real resume decision from disk probes at each site — FindBlockData / a data.tar stat / chunk geometry re-derivation — so a stale or approximate label here can never cause wrong data to be reused. Only NodeResumePlan.Done gates whether a node is skipped. In particular the collision-redirect paths report ObservedPending for a fresh redirect target they do not scan the contents of; that is fine precisely because nothing reads the label to decide anything. 

const (
	// ObservedPending: the node directory does not exist, is effectively empty
	// (a genuinely fresh dir), or the node was redirected to a not-yet-scanned
	// collision path.
	ObservedPending ObservedState = "pending"

	// ObservedBlockPartial: a block chunk staging dir (BlockChunksDirName) is
	// present, i.e. a single-volume block download was in progress.
	ObservedBlockPartial ObservedState = "block_partial"

	// ObservedFSPartial: an FS tar staging dir (FsTarStagingDirName) or the
	// multi-volume data/ directory is present, i.e. a filesystem download was in
	// progress.
	ObservedFSPartial ObservedState = "fs_partial"

	// ObservedManifestsOnly: the directory exists (proven-fresh or manifests-only)
	// with no volume-staging artifact and no snapshot.yaml.
	ObservedManifestsOnly ObservedState = "manifests_only"

	// ObservedDone: snapshot.yaml is present, VerifyNode passed for the planned
	// identity, and its directory durability was confirmed — the node is complete.
	ObservedDone ObservedState = "done"
)

type OpenBoundaryHook 

type OpenBoundaryHook func(path string)

OpenBoundaryHook runs immediately before a rooted descendant or enumeration open. It exists to make adversarial replacement tests deterministic; production callers pass nil.

type PinnedDirectory 

type PinnedDirectory struct {
	// contains filtered or unexported fields
}

PinnedDirectory is a descriptor-relative directory beneath a RootedSource. It remains confined to the originally opened root even if path components are replaced after opening.

func (*PinnedDirectory) Close 

func (d *PinnedDirectory) Close() error

Close releases the descriptor retained by directory.

func (*PinnedDirectory) OpenDirectory 

func (d *PinnedDirectory) OpenDirectory(name string) (*PinnedDirectory, error)

OpenDirectory opens one real child directory relative to the pinned descriptor.

func (*PinnedDirectory) OpenRegularFile 

func (d *PinnedDirectory) OpenRegularFile(name string) (*os.File, error)

OpenRegularFile opens one regular child without following links.

func (*PinnedDirectory) Path 

func (d *PinnedDirectory) Path() string

Path returns the diagnostic host path represented by directory.

func (*PinnedDirectory) ReadDirectory 

func (d *PinnedDirectory) ReadDirectory(count int) ([]os.DirEntry, error)

ReadDirectory reads the next bounded batch from the pinned directory.

func (*PinnedDirectory) ReadDirectoryBounded 

func (d *PinnedDirectory) ReadDirectoryBounded(maxEntries int) ([]os.DirEntry, error)

ReadDirectoryBounded reads at most maxEntries entries from the pinned directory, failing with ErrTooManyDirectChildren instead of returning a larger slice when more entries exist.

type PublicationState 

type PublicationState uint8

PublicationState describes whether an AtomicWriter commit error happened before or after the final path became visible. 

const (
	// PublicationUnpublished means rename did not complete.
	PublicationUnpublished PublicationState = iota
	// PublicationPublished means rename completed but parent-directory
	// durability is not yet confirmed.
	PublicationPublished
)

func CommitPublicationState 

func CommitPublicationState(err error) PublicationState

CommitPublicationState returns the publication state carried by err. Errors that did not originate from an AtomicWriter commit are treated as PublicationUnpublished.

type RootedDestination 

type RootedDestination struct {
	// contains filtered or unexported fields
}

RootedDestination confines every operation to the exact archive root held by a write lock. The os.Root and RootedSource handles are independently opened and identity-matched once. Descendant walks retain at most two rolling source/mutation pairs plus short-lived identity probes, independent of path depth; rename may hold two completed parent pairs. Rooted paths are limited to 16 KiB and 4096 components, leaving room for node and staging prefixes around the exporter's 4096-byte relative-path contract. Each operation remains descriptor-relative and revalidates the lock binding before it can mutate.

func NewLockedRootedDestination 

func NewLockedRootedDestination(lock *Lock, hook MutationBoundaryHook) (*RootedDestination, error)

NewLockedRootedDestination binds a mutation view to lock's exact pinned root.

func OpenRootedDestination 

func OpenRootedDestination(path string, hook MutationBoundaryHook) (*RootedDestination, error)

OpenRootedDestination opens an unbound rooted mutation view. Production downloads use NewLockedRootedDestination; this entry point keeps archive and volume unit tests independent of command lock setup.

func (*RootedDestination) BindingError 

func (d *RootedDestination) BindingError() error

BindingError returns the first observed destination binding failure.

func (*RootedDestination) Close 

func (d *RootedDestination) Close() error

Close releases the mutation root. A lock-owned RootedSource remains owned by its Lock and is closed only after this destination has been closed.

func (*RootedDestination) ComputeNodeChecksum 

func (d *RootedDestination) ComputeNodeChecksum(nodeDir string) (NodeChecksum, error)

ComputeNodeChecksum hashes one node through the locked rooted view.

func (*RootedDestination) ComputeNodeChildrenChecksum 

func (d *RootedDestination) ComputeNodeChildrenChecksum(nodeDir string) (NodeChecksum, error)

ComputeNodeChildrenChecksum hashes one node's direct children through the locked rooted view.

func (*RootedDestination) CreateExclusive 

func (d *RootedDestination) CreateExclusive(path string, perm os.FileMode) (*os.File, error)

CreateExclusive creates a new regular file without following any component.

func (*RootedDestination) EnsureDir 

func (d *RootedDestination) EnsureDir(path string) error

EnsureDir creates path descriptor-relatively and confirms the leaf plus every containing directory back to the pinned root. The rolling traversal retains one source/mutation pair regardless of depth. Each containing directory is leaf is confirmed first, then each containing directory is confirmed in one rolling root-to-leaf pass. Every call repeats the complete chain because a pre-existing entry may be residue from an earlier failed confirmation.

func (*RootedDestination) FindBlockData 

func (d *RootedDestination) FindBlockData(nodeDir string) (BlockPayload, bool, error)

FindBlockData classifies one node's block payload through the locked view.

func (*RootedDestination) OpenPinnedDirectory 

func (d *RootedDestination) OpenPinnedDirectory(path string) (*PinnedDirectory, error)

OpenPinnedDirectory opens one read-only directory through the locked source.

func (*RootedDestination) OpenRegular 

func (d *RootedDestination) OpenRegular(path string) (*os.File, error)

OpenRegular opens a no-follow regular file beneath the locked root.

func (*RootedDestination) OpenRegularFile 

func (d *RootedDestination) OpenRegularFile(
	path string,
	flag int,
	perm os.FileMode,
) (*os.File, error)

OpenRegularFile opens an existing regular file for descriptor-bound reads or writes. Truncation at open time is rejected because identity must be compared before the first mutation; callers may call Truncate on the returned handle.

func (*RootedDestination) Path 

func (d *RootedDestination) Path() string

Path returns the diagnostic absolute path represented by destination.

func (*RootedDestination) ReadChunkMeta 

func (d *RootedDestination) ReadChunkMeta(
	ctx context.Context,
	dir string,
) (ChunkMeta, bool, error)

ReadChunkMeta reads bounded chunk geometry through the locked view.

func (*RootedDestination) ReadDir 

func (d *RootedDestination) ReadDir(path string) ([]os.DirEntry, error)

ReadDir reads one directory beneath the locked root.

func (*RootedDestination) ReadFile 

func (d *RootedDestination) ReadFile(path string) ([]byte, error)

ReadFile reads a no-follow regular file beneath the locked root.

func (*RootedDestination) ReadSnapshotYAML 

func (d *RootedDestination) ReadSnapshotYAML(nodeDir string) (SnapshotYAML, error)

ReadSnapshotYAML reads snapshot.yaml for one node through the locked view.

func (*RootedDestination) ReadSnapshotYAMLWithOptions 

func (d *RootedDestination) ReadSnapshotYAMLWithOptions(
	nodeDir string,
	options SnapshotYAMLReadOptions,
) (SnapshotYAML, error)

ReadSnapshotYAMLWithOptions reads snapshot.yaml through the locked view under an explicit compatibility policy.

func (*RootedDestination) Relative 

func (d *RootedDestination) Relative(path string) (string, error)

Relative converts an absolute destination path to a safe root-relative path.

func (*RootedDestination) Remove 

func (d *RootedDestination) Remove(path string) error

Remove removes one entry without following links.

func (*RootedDestination) RemoveAll 

func (d *RootedDestination) RemoveAll(path string) error

RemoveAll recursively removes one rooted subtree without crossing a mount, following a link, or reopening any absolute pathname.

func (*RootedDestination) Rename 

func (d *RootedDestination) Rename(oldPath, newPath string) error

Rename atomically renames two entries beneath the same locked root.

func (*RootedDestination) SetBindingLossHandler 

func (d *RootedDestination) SetBindingLossHandler(handler func(error))

SetBindingLossHandler installs the cancellation callback used by the pipeline. The first namespace-binding failure wins.

func (*RootedDestination) SetDirectorySyncHook 

func (d *RootedDestination) SetDirectorySyncHook(hook DirectorySyncHook)

SetDirectorySyncHook installs a destination-scoped wrapper around rooted directory confirmations. Tests use it to inject operation failures without changing concurrent destinations.

func (*RootedDestination) SetTraversalContext 

func (d *RootedDestination) SetTraversalContext(ctx context.Context)

SetTraversalContext installs the cancellation context checked by bounded rooted path walks and recursive cleanup. A destination is bound to one pipeline run at a time; callers reset the context after that run finishes.

func (*RootedDestination) Stat 

func (d *RootedDestination) Stat(path string) (os.FileInfo, error)

Stat inspects a no-follow entry beneath the locked root.

func (*RootedDestination) SyncParent 

func (d *RootedDestination) SyncParent(path string) error

SyncParent confirms the containing directory of path through its pinned handle.

func (*RootedDestination) Verify 

func (d *RootedDestination) Verify() error

Verify checks both the lock namespace and the independently pinned mutation root.

func (*RootedDestination) VerifyNode 

func (d *RootedDestination) VerifyNode(nodeDir string) error

VerifyNode verifies one node through the locked rooted view.

func (*RootedDestination) VerifyNodeWithOptions 

func (d *RootedDestination) VerifyNodeWithOptions(
	nodeDir string,
	options SnapshotYAMLReadOptions,
) error

VerifyNodeWithOptions verifies one node through the locked view under an explicit compatibility policy.

func (*RootedDestination) WriteChunkMeta 

func (d *RootedDestination) WriteChunkMeta(
	ctx context.Context,
	dir string,
	meta ChunkMeta,
) error

WriteChunkMeta writes chunk geometry atomically through the locked view.

type RootedSource 

type RootedSource struct {
	// contains filtered or unexported fields
}

RootedSource pins one verified archive directory and opens descendants relative to its descriptor or handle. Retaining the source prevents path replacement from redirecting later opens. Every child source also verifies its retained parent chain before use, so replacing an already-validated archive directory fails closed instead of silently continuing through a detached tree.

func OpenRootedSource 

func OpenRootedSource(path string) (*RootedSource, error)

OpenRootedSource opens path as a real directory without following its final component and pins the resulting descriptor or handle. Callers must close the returned source.

func OpenRootedSourceWithHook 

func OpenRootedSourceWithHook(path string, hook OpenBoundaryHook) (*RootedSource, error)

OpenRootedSourceWithHook is OpenRootedSource with a deterministic boundary hook.

func (*RootedSource) Close 

func (s *RootedSource) Close() error

Close releases the directory descriptor or handle retained by source.

func (*RootedSource) OpenDirectory 

func (s *RootedSource) OpenDirectory(name string) (*RootedSource, error)

OpenDirectory opens one real child directory relative to source. The parent source must remain open until the child is closed.

func (*RootedSource) OpenDirectoryPath 

func (s *RootedSource) OpenDirectoryPath(path string) (*PinnedDirectory, error)

OpenDirectoryPath securely descends through a relative directory path while retaining at most two transient descendant descriptors.

func (*RootedSource) OpenRegularFile 

func (s *RootedSource) OpenRegularFile(name string) (*os.File, error)

OpenRegularFile opens one regular child file without following a final symlink or reparse point. Ordinary hard links remain supported because they are regular files; host link count does not imply archive/tar hard-link semantics.

func (*RootedSource) OpenRegularPath 

func (s *RootedSource) OpenRegularPath(path string) (*os.File, error)

OpenRegularPath descends through real directories beneath source and opens the final regular file without following links at any component.

func (*RootedSource) Path 

func (s *RootedSource) Path() string

Path returns the diagnostic host path represented by source.

func (*RootedSource) ReadDirectory 

func (s *RootedSource) ReadDirectory() ([]os.DirEntry, error)

ReadDirectory reads source through a fresh descriptor so repeated enumeration is stable.

func (*RootedSource) ReadDirectoryBounded 

func (s *RootedSource) ReadDirectoryBounded(maxEntries int) ([]os.DirEntry, error)

ReadDirectoryBounded reads at most maxEntries entries from source through a fresh descriptor. It fails with ErrTooManyDirectChildren instead of returning a larger slice when more entries exist, bounding memory use against an archive directory with an adversarially large count.

type SnapshotYAML 

type SnapshotYAML struct {
	// FormatVersion identifies the snapshot.yaml envelope schema. Missing decodes as legacy
	// version zero, which normal readers reject; writers always stamp SnapshotFormatVersionCurrent.
	FormatVersion int `json:"formatVersion,omitempty"`
	// APIVersion is the apiVersion of the snapshot CR (e.g. "state-snapshotter.deckhouse.io/v1alpha1").
	APIVersion string `json:"apiVersion"`
	// Kind is the kind of the snapshot CR (e.g. "Snapshot", "DemoVirtualDiskSnapshot").
	Kind string `json:"kind"`
	// Name is the metadata.name of the snapshot CR.
	Name string `json:"name"`
	// Namespace is the namespace of the snapshot CR. Omitted for cluster-scoped resources.
	Namespace string `json:"namespace,omitempty"`
	// UID is the metadata.uid of the snapshot CR. It is the identity component the resume
	// scan matches (matchesIdentity), tying a node directory to the exact snapshot CR
	// (including UID) rather than to the source-object name.
	UID string `json:"uid,omitempty"`
	// SourceName is the metadata.name of the original captured source object
	// (status.sourceRef.name), recorded for readability. Omitted when the node has no
	// source (e.g. some import nodes). It is NOT an identity component (resume uses UID).
	SourceName string `json:"sourceName,omitempty"`
	// SourceObjectRef carries the structured spec.sourceRef from a domain snapshot CR
	// ({apiVersion,kind,name} of the source object). Absent for core Snapshot nodes and
	// CSI VolumeSnapshot data leaves.
	SourceObjectRef *SourceObjectRef `json:"sourceObjectRef,omitempty"`
	// Checksum is the locally-computed node integrity digest.
	Checksum NodeChecksum `json:"checksum"`
	// ChildrenChecksum authenticates the canonical set of this node's direct children: their
	// identities (apiVersion/kind/name/namespace/uid) and their own Checksum/ChildrenChecksum
	// digests. A node with no children commits to EmptyChildrenChecksum. It is mandatory on
	// every snapshot.yaml (see validateSnapshotEnvelope): local inspection, upload, and
	// restore all fail closed on an archive that lacks it, and on any archive whose physical
	// direct-child set does not match the committed one (a "hybrid tree").
	ChildrenChecksum *NodeChecksum `json:"childrenChecksum,omitempty"`
	// MetadataChecksum covers the canonical versioned envelope except this field itself.
	// It is absent only on compatible legacy version-zero archives.
	MetadataChecksum *NodeChecksum `json:"metadataChecksum,omitempty"`
	// Volumes lists the captured PVC volumes owned by this node.
	//
	//   - A node that captured its own volume (namespaced status.data present) carries
	//     exactly one VolumeInfo (Variant A, cardinality ≤1) — this covers both
	//     non-aggregator domain nodes and orphan leaf volume nodes.
	//   - Aggregator snapshot nodes and purely-manifest nodes carry no volumes
	//     and the field is omitted (omitempty).
	Volumes []VolumeInfo `json:"volumes,omitempty"`
}

SnapshotYAML is the per-node file written at <nodeDir>/snapshot.yaml. It records the versioned snapshot CR identity and locally-computed integrity checksums. sigs.k8s.io/yaml uses json struct tags for marshaling and unmarshaling.

func ReadSnapshotYAML 

func ReadSnapshotYAML(nodeDir string) (SnapshotYAML, error)

ReadSnapshotYAML reads and deserialises <nodeDir>/snapshot.yaml, rejecting legacy and unsupported envelope versions and invalid versioned metadata checksums. Returns an error wrapping os.ErrNotExist when the file is absent.

func ReadSnapshotYAMLWithOptions 

func ReadSnapshotYAMLWithOptions(
	nodeDir string,
	options SnapshotYAMLReadOptions,
) (SnapshotYAML, error)

ReadSnapshotYAMLWithOptions reads snapshot.yaml under an explicit compatibility policy.

func UnmarshalSnapshotYAML 

func UnmarshalSnapshotYAML(data []byte, options SnapshotYAMLReadOptions) (SnapshotYAML, error)

UnmarshalSnapshotYAML decodes YAML bytes under the caller-selected compatibility policy. The zero options value enforces the current authenticated envelope.

func (SnapshotYAML) MarshalJSON 

func (sy SnapshotYAML) MarshalJSON() ([]byte, error)

MarshalJSON stamps the current envelope version and canonical metadata checksum. Both rooted and path-based snapshot.yaml writers use sigs.k8s.io/yaml, which delegates to this method.

func (*SnapshotYAML) UnmarshalJSON 

func (sy *SnapshotYAML) UnmarshalJSON(data []byte) error

UnmarshalJSON validates envelope version and metadata integrity before exposing semantic fields. It deliberately fails closed on missing or zero formatVersion; explicit legacy callers must use UnmarshalSnapshotYAML with AllowUnauthenticatedLegacy.

type SnapshotYAMLReadOptions 

type SnapshotYAMLReadOptions struct {
	AllowUnauthenticatedLegacy bool
}

SnapshotYAMLReadOptions controls compatibility when decoding snapshot.yaml. The zero value fails closed. AllowUnauthenticatedLegacy must be selected explicitly by migration or inspection callers that accept legacy metadata without an integrity checksum. It never bypasses the mandatory authenticated ChildrenChecksum direct-child commitment: every snapshot.yaml, legacy or current, must carry one.

type SourceObjectRef 

type SourceObjectRef struct {
	// APIVersion is the apiVersion of the source object (e.g. "demo.deckhouse.io/v1alpha1").
	APIVersion string `json:"apiVersion"`
	// Kind is the kind of the source object (e.g. "DemoVirtualDisk").
	Kind string `json:"kind"`
	// Name is the metadata.name of the source object.
	Name string `json:"name"`
}

SourceObjectRef is the structured spec.sourceRef from a domain snapshot CR, persisted in snapshot.yaml so the import side can recreate the CR in import mode. The fields mirror the domain CR's spec.sourceRef (apiVersion/kind/name of the source object). Omitted for core Snapshot nodes and CSI VolumeSnapshot data leaves.

type VerifiedArchive 

type VerifiedArchive struct {
	// contains filtered or unexported fields
}

VerifiedArchive retains one rooted archive descriptor for the upload lifetime. Individual payload handles are opened only for active workers, keeping descriptor use bounded.

func OpenVerifiedArchive 

func OpenVerifiedArchive(root string) (*VerifiedArchive, error)

OpenVerifiedArchive pins root for planning, verification, upload, and final readiness.

func OpenVerifiedArchiveWithOptions 

func OpenVerifiedArchiveWithOptions(
	root string,
	options SnapshotYAMLReadOptions,
) (*VerifiedArchive, error)

OpenVerifiedArchiveWithOptions pins root under an explicit snapshot.yaml compatibility policy.

func (*VerifiedArchive) Close 

func (a *VerifiedArchive) Close() error

Close releases the pinned archive root. All VerifiedHandles must already be closed.

func (*VerifiedArchive) OpenVerifiedFile 

func (a *VerifiedArchive) OpenVerifiedFile(ctx context.Context, expected *VerifiedFile) (*VerifiedHandle, error)

OpenVerifiedFile opens, verifies, and rewinds the exact descriptor later consumed by upload.

func (*VerifiedArchive) RootSource 

func (a *VerifiedArchive) RootSource() *RootedSource

RootSource returns the pinned source used to build the plan. The caller must not close it.

func (*VerifiedArchive) VerifyNode 

func (a *VerifiedArchive) VerifyNode(ctx context.Context, nodeDir string) (*VerifiedNode, error)

VerifyNode captures a node's checksum-covered file identities and bytes from the pinned root.

func (*VerifiedArchive) VerifyNodeChildrenChecksum 

func (a *VerifiedArchive) VerifyNodeChildrenChecksum(ctx context.Context, nodeDir string) error

VerifyNodeChildrenChecksum verifies one node's authenticated direct-child commitment through the archive's pinned root, without opening any child payload bytes. Callers use it as a fast preflight (e.g. against a selected-subtree import's ancestor chain) before the fuller VerifyNode pass runs on the nodes actually being mutated.

type VerifiedFile 

type VerifiedFile struct {
	// contains filtered or unexported fields
}

VerifiedFile records the identity and content of one checksum-covered archive file.

func (*VerifiedFile) DigestMatches 

func (f *VerifiedFile) DigestMatches(digest [sha256.Size]byte) bool

DigestMatches reports whether digest identifies the verified file bytes.

func (*VerifiedFile) IdentityMatches 

func (f *VerifiedFile) IdentityMatches(info os.FileInfo) bool

IdentityMatches reports whether info is the same archive file identity verified.

type VerifiedHandle 

type VerifiedHandle struct {
	// contains filtered or unexported fields
}

VerifiedHandle is one exact regular-file descriptor whose identity and bytes were verified. The handle must be closed before its VerifiedArchive.

func (*VerifiedHandle) AuthenticatedReadStats 

func (h *VerifiedHandle) AuthenticatedReadStats() AuthenticatedReadStats

AuthenticatedReadStats returns a concurrency-safe snapshot of authenticated read work.

func (*VerifiedHandle) Close 

func (h *VerifiedHandle) Close() error

Close releases the pinned payload descriptor.

func (*VerifiedHandle) Read 

func (h *VerifiedHandle) Read(p []byte) (int, error)

Read authenticates fixed-size chunks before exposing bytes from the pinned descriptor.

func (*VerifiedHandle) ReadAt 

func (h *VerifiedHandle) ReadAt(p []byte, offset int64) (int, error)

ReadAt authenticates fixed-size chunks before exposing bytes from the pinned descriptor.

func (*VerifiedHandle) ResetAuthenticatedRead 

func (h *VerifiedHandle) ResetAuthenticatedRead()

ResetAuthenticatedRead starts a fresh authenticated consumption pass. It prevents a retry, range, or parser pass from relying on bytes cached by an earlier logical consumer.

func (*VerifiedHandle) Seek 

func (h *VerifiedHandle) Seek(offset int64, whence int) (int64, error)

Seek changes the logical authenticated-read offset.

func (*VerifiedHandle) Stat 

func (h *VerifiedHandle) Stat() (os.FileInfo, error)

Stat returns metadata for the pinned descriptor.

func (*VerifiedHandle) Verify 

func (h *VerifiedHandle) Verify(ctx context.Context) error

Verify proves both the pinned descriptor bytes and the current rooted namespace entry still match verification. It preserves the descriptor offset.

type VerifiedNode 

type VerifiedNode struct {
	// contains filtered or unexported fields
}

VerifiedNode is the immutable verification result for one archive node.

func (*VerifiedNode) Checksum 

func (n *VerifiedNode) Checksum() NodeChecksum

Checksum returns the recomputed node checksum.

func (*VerifiedNode) File 

func (n *VerifiedNode) File(relPath string) (*VerifiedFile, bool)

File returns a verified checksum-covered file by node-relative path.

func (*VerifiedNode) SnapshotDigestMatches 

func (n *VerifiedNode) SnapshotDigestMatches(digest [sha256.Size]byte) bool

SnapshotDigestMatches reports whether digest identifies the verified snapshot.yaml bytes.

func (*VerifiedNode) SnapshotIdentityMatches 

func (n *VerifiedNode) SnapshotIdentityMatches(info os.FileInfo) bool

SnapshotIdentityMatches reports whether info is the same snapshot.yaml identity verified.

type VolumeInfo 

type VolumeInfo struct {
	// Target is the source PVC that was captured (its apiVersion/kind/name/namespace/uid).
	Target VolumeObjectRef `json:"target"`
	// Artifact is the VolumeSnapshotContent that held the durable data artifact at capture
	// time. Recorded for provenance/debugging; the re-import path no longer consumes it.
	Artifact VolumeObjectRef `json:"artifact"`
	// VolumeMode records the source volume mode (Block or Filesystem). On re-import it is sent
	// as the PopulateData DataImport's spec.storageParams.volumeMode (optional).
	VolumeMode string `json:"volumeMode,omitempty"`
	// StorageClassName records the source StorageClass of the captured volume. On re-import it
	// is sent as the PopulateData DataImport's spec.storageParams.storageClassName (required).
	StorageClassName string `json:"storageClassName,omitempty"`
	// Size records the real allocated size of the captured volume (e.g. "10Gi"), taken from
	// VolumeSnapshotContent.status.restoreSize. On re-import it is sent as the PopulateData
	// DataImport's spec.storageParams.size (required).
	Size string `json:"size,omitempty"`
}

VolumeInfo describes the captured volume associated with a volume node. It is written into the volume block of snapshot.yaml so the archive is self-describing.

type VolumeObjectRef 

type VolumeObjectRef struct {
	// APIVersion is the apiVersion of the referenced object.
	APIVersion string `json:"apiVersion"`
	// Kind is the kind of the referenced object.
	Kind string `json:"kind"`
	// Name is the metadata.name of the referenced object.
	Name string `json:"name"`
	// Namespace is the namespace of the referenced object. Omitted for cluster-scoped objects.
	Namespace string `json:"namespace,omitempty"`
	// UID is the metadata.uid of the referenced object. Omitted when unknown.
	UID string `json:"uid,omitempty"`
}

VolumeObjectRef is a reference to a Kubernetes object stored in the volume block of snapshot.yaml. It captures the identity fields needed to correlate the archive entry with live cluster resources.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.