Documentation
¶
Index ¶
- Constants
- Variables
- func GetType(t byte) (string, error)
- func WriteZstdChunkedManifest(dest io.Writer, outMetadata map[string]string, offset uint64, ...) error
- func ZstdWriterWithLevel(dest io.Writer, level int) (*zstd.Encoder, error)
- type FileMetadata
- type TOC
- type TarSplitData
- type ZstdChunkedFooterData
Constants ¶
const ( ChunkTypeData = "" ChunkTypeZeros = "zeros" )
const ( // The following types correspond to regular types of entries that can // appear in a tar archive. TypeReg = "reg" TypeLink = "hardlink" TypeChar = "char" TypeBlock = "block" TypeDir = "dir" TypeFifo = "fifo" TypeSymlink = "symlink" // TypeChunk is special; in zstd:chunked not only are files individually // compressed and indexable, there is a "rolling checksum" used to compute // "chunks" of individual file contents, that are also added to the TOC TypeChunk = "chunk" )
const ( // ManifestChecksumKey is a hexadecimal sha256 digest of the compressed manifest digest. ManifestChecksumKey = "io.github.containers.zstd-chunked.manifest-checksum" // ManifestInfoKey is an annotation that signals the start of the TOC (manifest) // contents which are embedded as a skippable zstd frame. It has a format of // four decimal integers separated by `:` as follows: // <offset>:<length>:<uncompressed length>:<type> // The <type> is ManifestTypeCRFS which should have the value `1`. ManifestInfoKey = "io.github.containers.zstd-chunked.manifest-position" // TarSplitInfoKey is an annotation that signals the start of the "tar-split" metadata // contents which are embedded as a skippable zstd frame. It has a format of // three decimal integers separated by `:` as follows: // <offset>:<length>:<uncompressed length> TarSplitInfoKey = "io.github.containers.zstd-chunked.tarsplit-position" // ManifestTypeCRFS is a manifest file compatible with the CRFS TOC file. ManifestTypeCRFS = 1 // Newer versions of the image format might increase this value, so reject // any version that is not supported. FooterSizeSupported = 64 )
Variables ¶
var TarTypes = map[byte]string{ tar.TypeReg: TypeReg, tar.TypeLink: TypeLink, tar.TypeChar: TypeChar, tar.TypeBlock: TypeBlock, tar.TypeDir: TypeDir, tar.TypeFifo: TypeFifo, tar.TypeSymlink: TypeSymlink, }
var (
ZstdChunkedFrameMagic = []byte{0x47, 0x4e, 0x55, 0x6c, 0x49, 0x6e, 0x55, 0x78}
)
Functions ¶
func WriteZstdChunkedManifest ¶
func WriteZstdChunkedManifest(dest io.Writer, outMetadata map[string]string, offset uint64, tarSplitData *TarSplitData, metadata []FileMetadata, level int) error
Types ¶
type FileMetadata ¶
type FileMetadata struct {
// The metadata below largely duplicates that in the tar headers.
Type string `json:"type"`
Name string `json:"name"`
Linkname string `json:"linkName,omitempty"`
Mode int64 `json:"mode,omitempty"`
Size int64 `json:"size,omitempty"`
UID int `json:"uid,omitempty"`
GID int `json:"gid,omitempty"`
ModTime *time.Time `json:"modtime,omitempty"`
AccessTime *time.Time `json:"accesstime,omitempty"`
ChangeTime *time.Time `json:"changetime,omitempty"`
Devmajor int64 `json:"devMajor,omitempty"`
Devminor int64 `json:"devMinor,omitempty"`
Xattrs map[string]string `json:"xattrs,omitempty"`
// Digest is a hexadecimal sha256 checksum of the file contents; it
// is empty for empty files
Digest string `json:"digest,omitempty"`
Offset int64 `json:"offset,omitempty"`
EndOffset int64 `json:"endOffset,omitempty"`
ChunkSize int64 `json:"chunkSize,omitempty"`
ChunkOffset int64 `json:"chunkOffset,omitempty"`
ChunkDigest string `json:"chunkDigest,omitempty"`
ChunkType string `json:"chunkType,omitempty"`
}
FileMetadata is an entry in the TOC that includes both generic file metadata that duplicates what can found in the tar header (and should match), but also special/custom content (see below).
Regular files may optionally be represented as a sequence of “chunks”, which may be ChunkTypeData or ChunkTypeZeros (and ChunkTypeData boundaries are heuristically determined to increase chance of chunk matching / reuse similar to rsync). In that case, the regular file is represented as an initial TypeReg entry (with all metadata for the file as a whole) immediately followed by zero or more TypeChunk entries (containing only Type, Name and Chunk* fields); if there is at least one TypeChunk entry, the Chunk* fields are relevant in all of these entries, including the initial TypeReg one.
Note that the metadata here, when fetched by a zstd:chunked aware client, is used instead of that in the tar stream. The contents of the tar stream are not used in this scenario.
func NewFileMetadata ¶ added in v1.55.0
func NewFileMetadata(hdr *tar.Header) (FileMetadata, error)
NewFileMetadata creates a basic FileMetadata entry for hdr. The caller must set DigestOffset/EndOffset, and the Chunk* values, separately.
type TOC ¶
type TOC struct {
// Version is currently expected to be 1
Version int `json:"version"`
// Entries is the list of file metadata in this TOC.
// The ordering in this array currently defaults to being the same
// as that of the tar stream; however, this should not be relied on.
Entries []FileMetadata `json:"entries"`
// TarSplitDigest is the checksum of the "tar-split" data which
// is included as a distinct skippable zstd frame before the TOC.
TarSplitDigest digest.Digest `json:"tarSplitDigest,omitempty"`
}
TOC is short for Table of Contents and is used by the zstd:chunked file format to effectively add an overall index into the contents of a tarball; it also includes file metadata.
type TarSplitData ¶ added in v1.47.0
type ZstdChunkedFooterData ¶ added in v1.50.0
type ZstdChunkedFooterData struct {
}
ZstdChunkedFooterData contains all the data stored in the zstd:chunked footer. This footer exists to make the blobs self-describing, our implementation never reads it: Partial pull security hinges on the TOC digest, and that exists as a layer annotation; so we are relying on the layer annotations anyway, and doing so means we can avoid a round-trip to fetch this binary footer.
Source Files