ctf

package module
v0.0.0-...-9e33677 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 27, 2025 License: Apache-2.0 Imports: 24 Imported by: 0

Documentation

Overview

Package ctf provides various interfaces and types for working with Common Transport Format Archives (CTF)

The Common Transport Format describes a file system structure that can be used for the representation of [content](https://github.com/opencontainers/image-spec) of an OCI repository.

Therefore, it can be used to describe a subset of repositories of an OCI registry with a subset of artifacts, that can then be imported again into any OCI registry.

A CTF is a file or directory containing

  • artifact-index.json: This JSON file describes the contained artifact (versions).

  • blobs

    The blobs directory contains the blobs described by the artifact index as a flat file list. These are layer blobs or artifact blobs for the artifact descriptors.

    Every file has a filename according to its digest (https://github.com/opencontainers/image-spec/blob/main/descriptor.md#digests). Hereby the algorithm separator character is replaced by a dot (".").

    Every file SHOULD be referenced, directly or indirectly, in the artifact descriptor by a descriptor according the OCI Image Specification (https://github.com/opencontainers/image-spec/blob/main/descriptor.md).

    The artifact index describes the OCI manifests (image manifests and index manifests), which refer to further non-manifest blobs. Files not referenced by the artifacts described by the index SHOULD be ignored.

The FileFormat of a CTF can differ: as directory of an operating system file system or a virtual file system (FormatDirectory) or as content of a TAR archive (unzipped - FormatTAR or zipped - FormatTGZ). The descriptor SHOULD be the first file if stored in an archive.

This package also offers a legacy compatibility layer access for the ArtifactSet, a now no longer recommended artifact format that was used in the past by OCM CLI to package local blobs. We now instead recommend packaging in the format of OCI Image Layouts as they are almost identical. See ArtifactSet for more details.

Index

Constants

View Source 
const (
	// O_RDONLY indicates that the CTF is opened in read-only mode.
	O_RDONLY = os.O_RDONLY
	// O_RDWR indicates that the CTF is opened in read-write mode.
	O_RDWR = os.O_RDWR
	// O_CREATE indicates that the CTF is created if it does not exist.
	O_CREATE = os.O_CREATE
)

Flags to OpenCTF. They are not bound to a type because the underlying type changes based on syscall interfaces.

View Source 
const ArtifactSetMediaType = "application/vnd.oci.image.manifest.v1+tar+gzip"

ArtifactSetMediaType is the definition of the ArtifactSet Media Type in old format CTFs It is generated by OCM when transferring resources into a CTF and written into a localBlob mediaType as an access hint: 

access:
  localReference: sha256:e40e3a2f1ab1a98328dfd14539a79d27aff5c4d5c34cd16a85f0288bfa76490b
  mediaType: application/vnd.oci.image.manifest.v1+tar+gzip
  referenceName: jakobmoellerdev/podinfo/podinfo:6.7.1
  type: localBlob

This ArtifactSetMediaType can be used on artifact introspection on the resource together with NewArtifactSetFromBlob to access that ArtifactSet.

View Source 
const (
	BlobsDirectoryName = "blobs"
)

Variables

View Source 
var ErrUnsupportedFormat = fmt.Errorf("unsupported format")

Functions

func Archive 

func Archive(ctx context.Context, ctf CTF, path string, format FileFormat) error

Archive creates an archive from the provided CTF and writes it to the specified path. The format of the archive is determined by the format parameter. Supported formats are FormatTAR, FormatTGZ, and FormatDirectory. If the format is FormatDirectory, the filesystem is copied to the specified path.

func ArchiveDirectory 

func ArchiveDirectory(ctx context.Context, ctf CTF, path string) error

ArchiveDirectory archives the CTF to the specified path with FormatDirectory. The blobs are copied to the directory and the index is written to the index file. The CTF is not modified and only read from. The directory is created if it does not exist. The blobs are written to the blobs directory concurrently.

func ArchiveTAR 

func ArchiveTAR(ctx context.Context, ctf CTF, path string, format FileFormat) (err error)

ArchiveTAR archives the CTF to the specified path. The blobs are written to the blobs directory and the index is written to the index file. The CTF is not modified and only read from. The file is created if it does not exist.

see ArchiveTARToWriter for more details.

func ArchiveTARToWriter 

func ArchiveTARToWriter(ctx context.Context, ctf CTF, writer io.Writer, format FileFormat) (err error)

ArchiveTARToWriter archives the CTF to the specified writer. The file can be optionally targeted as a tgz by specifying FormatTGZ, FormatTAR otherwise.

The blobs are written to the blobs directory sequentially due to the nature of TAR archives. The blobs are written in the order they are returned by ListBlobs. The index is written to the index file as first entry.

func ConvertToOCIImageLayout 

func ConvertToOCIImageLayout(ctx context.Context, as *ArtifactSet, writer io.Writer, manifestNameFn func(ctx context.Context, digest digest.Digest, oldName string) (string, error)) (err error)

ConvertToOCIImageLayout converts an ArtifactSet to an OCI Image Layout in tar format. It will write the index.json and all blobs to the writer. It converts old blobs according to a given manifestNameFn.

The manifestNameFn is used to convert the old name of the blob to the new name. Example: 

func manifestNameFn(digest digest.Digest, oldName string) (string, error) {
	return fmt.Sprintf("ghcr.io/open-component-model/%s", digest), nil
}

This is needed due to the lossy typing that requires the component descriptor (see ArtifactSet for information).

func OpenCTFByFileExtension 

func OpenCTFByFileExtension(ctx context.Context, path string, flag int) (archive CTF, discovered FileFormat, err error)

OpenCTFByFileExtension opens a CTF at the specified path by determining the format from the file extension. For FormatDirectory, the path is treated as a directory, otherwise the path is interpreted as a file with an extension that determines its behavior. For more information on how flag behaves for FormatTAR (and FormatTGZ), see ExtractTAR.

func ToBlobFileName 

func ToBlobFileName(dig string) (string, error)

ToBlobFileName converts a digest to a blob file name by replacing the ":" with ".", which is the default separator for blobs in the CTF under BlobsDirectoryName.

func ToDigest 

func ToDigest(blobFileName string) string

ToDigest converts a blob file name to a digest by replacing the "." with ":", which is the default separator for digests in standard notation.

func WorkWithinCTF 

func WorkWithinCTF(ctx context.Context, path string, flag int, work func(ctx context.Context, ctf CTF) error) error

WorkWithinCTF opens a CTF at the specified path and calls the work function with the CTF. If the CTF is backed by a TAR or TGZ archive, the CTF is archived into its originally discovered format after the work function is called. If an error occurs during the work function, the CTF is not archived if the format is FormatTAR or FormatTGZ However, if the format is FormatDirectory, the CTF is edited in place, which can lead to non atomic failures. To avoid this, by default (flag not set to O_RDWR), the CTF is not rearchived and opened in read-only mode.

Types

type ArtifactBlob 

type ArtifactBlob struct {
	// contains filtered or unexported fields
}

ArtifactBlob is a blob.ReadOnlyBlob that is backed by an ArtifactSet.

func (*ArtifactBlob) Digest 

func (a *ArtifactBlob) Digest() (digest string, known bool)

func (*ArtifactBlob) ReadCloser 

func (a *ArtifactBlob) ReadCloser() (io.ReadCloser, error)

func (*ArtifactBlob) Size 

func (a *ArtifactBlob) Size() (size int64)

type ArtifactSet 

type ArtifactSet struct {
	// contains filtered or unexported fields
}

ArtifactSet is the equivalent (deprecated) implementation of https://github.com/open-component-model/ocm/tree/2091216b223a5c084895cf501a0570a4de485c09/api/oci/extensions/repositories/artifactset

NOTE: Even though it does look similar to an OCI Image Layout, it is not the same. Notable differences are:

  • The blobs directory orders blobs in the form

    blobs/digest-algo.digest

    instead of

    blobs/digest-algo/digest.

  • The index.json file IS a valid OCI Image Index json, but does not maintain the ociimagespec.AnnotationRefName correctly, instead it sets it to the version of the resource. Additionally, it maintains the software.ocm/tags annotation. This means it is MANDATORY to understand the resource access.referenceName in place of the usual meaning of the refName to fully target an OCI Image out of an ArtifactSet.

  • The index.json file contains a software.ocm/main annotation, that declares the main blob to be introspected (useful for single-layer artifacts or multi-layer artifacts with one main layer and multiple metadata layers).

Altogether, this makes the ArtifactSet a custom format that is not compatible with OCI Image Layouts or CTF readings that are unaware of the Component Descriptor.

It is thus deprecated and should not be used anymore.

This ArtifactSetMediaType now only serves to read from old LocalBlobs and CTFs and is maintained to ensure compatibility.

New CTFs should and will always include localBlobs not as artifact sets, but as proper OCI Image Layouts.

func NewArtifactSetFromBlob 

func NewArtifactSetFromBlob(b blob.ReadOnlyBlob) (*ArtifactSet, error)

NewArtifactSetFromBlob creates a new ArtifactSet from a blob. It will start owning the readcloser in the blob and needs to be closed due to this. It is backed by a tar filesystem that is either read from a tar file or a gzip file. The gzip format is not detected by ArtifactSetMediaType, but by the magic number in the first 512 bytes of the file. The blob must be a valid ArtifactSet, otherwise an error is returned.

The ArtifactSet merely supports read-only and any future write operation is no longer supported. All ArtifactSet's encountered in the wild MUST be converted to OCI Image Layouts.

See ArtifactSet for more information about the format.

func (*ArtifactSet) Close 

func (a *ArtifactSet) Close() error

func (*ArtifactSet) GetBlob 

func (a *ArtifactSet) GetBlob(ctx context.Context, digest string) (blob.ReadOnlyBlob, error)

func (*ArtifactSet) GetIndex 

func (a *ArtifactSet) GetIndex() ociimagespec.Index

func (*ArtifactSet) ListBlobs 

func (a *ArtifactSet) ListBlobs(ctx context.Context) (digests []string, err error)

type BlobStore 

type BlobStore interface {
	ReadOnlyBlobStore
	// SaveBlob saves the blob to the CTF.
	SaveBlob(ctx context.Context, blob blob.ReadOnlyBlob) (err error)
	// DeleteBlob deletes the blob with the specified digest from the CTF.
	DeleteBlob(ctx context.Context, digest string) (err error)
}

BlobStore provides access to the blobs of a CTF.

type CASFileBlob 

type CASFileBlob struct {
	// contains filtered or unexported fields
}

CASFileBlob represents a content-addressable storage (CAS) file blob.

func NewCASFileBlob 

func NewCASFileBlob(fs filesystem.FileSystem, path string) *CASFileBlob

NewCASFileBlob creates a new CASFileBlob instance.

func (*CASFileBlob) Digest 

func (b *CASFileBlob) Digest() (digest string, known bool)

Digest returns the digest of the blob.

func (*CASFileBlob) HasPrecalculatedDigest 

func (b *CASFileBlob) HasPrecalculatedDigest() bool

HasPrecalculatedDigest checks if a digest is already stored.

func (*CASFileBlob) ReadCloser 

func (b *CASFileBlob) ReadCloser() (io.ReadCloser, error)

ReadCloser returns an io.ReadCloser for the blob.

func (*CASFileBlob) SetPrecalculatedDigest 

func (b *CASFileBlob) SetPrecalculatedDigest(digest string)

SetPrecalculatedDigest sets the digest, ensuring thread safety.

func (*CASFileBlob) Size 

func (b *CASFileBlob) Size() int64

Size returns the size of the blob.

type CTF 

type CTF interface {
	Format() FileFormat

	IndexStore
	BlobStore
}

CTF represents the CommonTransportFormat. It is an interface that provides access to an index and blobs through IndexStore and BlobStore interfaces. Depending on the FileFormat, the CTF may be backed by a filesystem or an archive.

In practice, the CTF is almost always backed with FormatDirectory, and working on FormatTAR and FormatTGZ is handled by 1. Extracting the CTF into a Directory format 2. Working on the Directory format 3. Archiving the Directory format back into the original format

func OpenCTF 

func OpenCTF(ctx context.Context, path string, format FileFormat, flag int) (CTF, error)

OpenCTF opens a CTF at the specified path with the specified format. the CTF may be backed by a temporary directory if the format is FormatTAR or FormatTGZ. In this case, the temporary directory is used to extract the archive before returning access on that path.

type FileFormat 

type FileFormat int

FileFormat represents the format of a CTF. A FileFormat can be translated to any other FileFormat without loss of information. The zero value of FileFormat is FormatDirectory, and also the default. 

const (
	// FormatUnknown represents an unknown format.
	FormatUnknown FileFormat = iota
	// FormatDirectory represents a CTF stored as a directory at a root path.
	FormatDirectory FileFormat = iota
	// FormatTAR represents a CTF stored as a Tape (TAR) archive.
	FormatTAR FileFormat = iota
	// FormatTGZ represents a CTF stored as a Tape (TAR) archive compressed with GZip with arbitrary compression.
	FormatTGZ FileFormat = iota
)

func (FileFormat) String 

func (f FileFormat) String() string

type FileSystemCTF 

type FileSystemCTF struct {
	// contains filtered or unexported fields
}

FileSystemCTF is a CTF implementation that uses any filesystem.FileSystem as the underlying storage. It is used to read and write CTFs from a directory structure. This is the canonical implementation of the CTF interface, accessing

  • the index file at v1.ArtifactIndexFileName
  • the blobs at BlobsDirectoryName

The CTF offered will always be of type FormatDirectory.

func ExtractTAR 

func ExtractTAR(ctx context.Context, base, path string, format FileFormat, flag int) (extracted *FileSystemCTF, err error)

ExtractTAR extracts a CTF from a file at the given path and writes it to the given base directory. The base directory must exist and will form the parent directory of the extracted CTF. The format of the file must be one of the supported formats (FormatTAR, FormatTGZ). The extracted CTF is not modified and only read from after extraction, and the TAR itself is not modified. If the flag O_RDONLY is set, the extracted CTF will be read-only as well, however the CTF will be first opened as O_RDWR to copy the data from the TAR into the new FileSystemCTF.

func NewFileSystemCTF 

func NewFileSystemCTF(filesystem filesystem.FileSystem) *FileSystemCTF

NewFileSystemCTF opens a CTF with the specified filesystem as its root

func OpenCTFFromOSPath 

func OpenCTFFromOSPath(path string, flag int) (*FileSystemCTF, error)

OpenCTFFromOSPath opens a CTF at the specified path with the specified flags. Supported flags are O_RDONLY, O_RDWR, and O_CREATE, other flags can lead to undefined behavior.

func (*FileSystemCTF) DeleteBlob 

func (c *FileSystemCTF) DeleteBlob(_ context.Context, digest string) (err error)

DeleteBlob deletes the blob with the given digest from the CTF by removing the file from BlobsDirectoryName.

func (*FileSystemCTF) FS 

func (c *FileSystemCTF) FS() filesystem.FileSystem

FS returns the underlying filesystem.FileSystem of the CTF. Note that write operations to the FileSystem can affect the integrity of the CTF. TODO(jakobmoellerdev): restrict returned FileSystem to only allow read operations.

func (*FileSystemCTF) Format 

func (c *FileSystemCTF) Format() FileFormat

Format always returns FormatDirectory for FileSystemCTF.

func (*FileSystemCTF) GetBlob 

func (c *FileSystemCTF) GetBlob(_ context.Context, digest string) (blob.ReadOnlyBlob, error)

GetBlob returns the blob with the given digest from the CTF by reading the file from BlobsDirectoryName.

func (*FileSystemCTF) GetIndex 

func (c *FileSystemCTF) GetIndex(_ context.Context) (index v1.Index, err error)

GetIndex returns the v1.ArtifactIndexFileName parsed as v1.Index of the CTF. If the CTF is empty, an empty index is returned so it can be set with SetIndex.

func (*FileSystemCTF) ListBlobs 

func (c *FileSystemCTF) ListBlobs(_ context.Context) (digests []string, err error)

ListBlobs returns a list of all blobs in the CTF by listing the files in BlobsDirectoryName.

func (*FileSystemCTF) SaveBlob 

func (c *FileSystemCTF) SaveBlob(ctx context.Context, b blob.ReadOnlyBlob) (err error)

func (*FileSystemCTF) SetIndex 

func (c *FileSystemCTF) SetIndex(_ context.Context, index v1.Index) (err error)

SetIndex sets the v1.ArtifactIndexFileName of the CTF to the given index.

type IndexStore 

type IndexStore interface {
	ReadOnlyIndexStore
	// SetIndex sets the artifact index of the CTF.
	SetIndex(ctx context.Context, index v1.Index) (err error)
}

IndexStore provides access to the index of a CTF.

type ReadOnlyBlobStore 

type ReadOnlyBlobStore interface {
	// ListBlobs returns a list of all blobs in the CTF irrespective of if they are referenced by the index.
	ListBlobs(ctx context.Context) ([]string, error)
	// GetBlob returns the blob with the specified digest.
	GetBlob(ctx context.Context, digest string) (blob.ReadOnlyBlob, error)
}

type ReadOnlyIndexStore 

type ReadOnlyIndexStore interface {
	// GetIndex returns the artifact index of the CTF.
	GetIndex(ctx context.Context) (v1.Index, error)
}

Source Files

View all Source files

Directories

Path Synopsis
index
v1

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.