blob

package module
v0.0.0-...-9e33677 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 27, 2025 License: Apache-2.0 Imports: 7 Imported by: 1

Documentation

Overview

Package blob provides various interfaces and types for working with Binary Large Object's (BLOBs).

When working with BLOBs through this package, it is important to understand the following concepts:

  • ReadOnlyBlob: An interface that represents a read-only BLOB.
  • Blob: An interface that represents a BLOB that can be both read and written to.
  • SizeAware: An interface that represents any arbitrary object that can be sized.
  • DigestAware: An interface that represents any arbitrary object that can be digested.
  • MediaTypeAware: An interface that represents any arbitrary object that can have a media type.

These core concepts make it possible to describe any arbitrary data without actually introspecting it. The Blob is the fundamental interface that can be used by implementations to provide abstractions over dynamic content-ware, size-aware and content-proof data.

Additionally, the package provides convenience implementations of typical blob scenarios:

  • EagerBufferedReader: A reader that eagerly buffers the data it reads, to make it easier to introspect the data. This is useful for cases where the size and digest of the data are not known in advance but still need to be computed (through buffering in-memory instead of directly streaming).

Note that filesystem-backed blobs are located in a separated sub-package.

Index

Constants

View Source
const DefaultArchiveBlobBufferSize = 128 * 1024 // 128 KiB

DefaultArchiveBlobBufferSize is the default buffer size used to archive blobs. It is slightly larger than the default buffer size used by io.Copy as most blobs encountered in practice are larger than the default buffer size.

View Source
const SizeUnknown = -1

SizeUnknown is a constant that represents an unknown size of a blob.

Variables

This section is empty.

Functions

func ArchiveBlob

func ArchiveBlob(name string, size int64, digest string, b ReadOnlyBlob, writer *tar.Writer, buf []byte) (err error)

ArchiveBlob archives a ReadOnlyBlob to the tar writer. it assumes that size and digest are already known to compute its header. The buffer is used to copy the blob data, if nil, a new buffer is allocated.

Types

type Blob

type Blob interface {
	ReadOnlyBlob
	WriteableBlob
}

Blob is an interface that represents a Binary Large Object. It's main purpose is to provide an abstraction over purpose to be able to only interact with content on it's data/byte level. A Blob can be interacted with both for reading and writing.

type DigestAware

type DigestAware interface {
	// Digest returns the blob digest if known.
	Digest() (digest string, known bool)
}

DigestAware is an interface that represents any arbitrary object that can be digested.

Digest is used to always determine the digest of the object.

type DigestPrecalculatable

type DigestPrecalculatable interface {
	// HasPrecalculatedDigest returns true if a precalculated digest of the blob is known in advance.
	HasPrecalculatedDigest() bool
	// SetPrecalculatedDigest sets the digest of the blob if it is known in advance.
	// Once set, HasPrecalculatedDigest must return true.
	// It MUST be safe for concurrent use, serializing as necessary.
	// It MUST be safe to call multiple times, overwriting the digest every time.
	SetPrecalculatedDigest(digest string)
}

DigestPrecalculatable is an interface that represents any arbitrary object that can be set with Digests ahead of a potential read/write operation.

HasPrecalculatedDigest is used to determine if the digest of the object is known in advance. SetPrecalculatedDigest is used to set the digest of the object if it is known in advance.

type DirectReadOnlyBlob

type DirectReadOnlyBlob struct {
	*EagerBufferedReader
}

DirectReadOnlyBlob is a read-only blob that reads from an io.Reader.

func NewDirectReadOnlyBlob

func NewDirectReadOnlyBlob(r io.Reader) *DirectReadOnlyBlob

NewDirectReadOnlyBlob forwards a given io.Reader to be able to be used as a ReadOnlyBlob. It does this by wrapping the io.Reader in a EagerBufferedReader, to allow for catching information such as the digest and size of the blob. Note that this should only be used if no sizing and digest information is present.

func (*DirectReadOnlyBlob) ReadCloser

func (d *DirectReadOnlyBlob) ReadCloser() (io.ReadCloser, error)

type EagerBufferedReader

type EagerBufferedReader struct {
	// contains filtered or unexported fields
}

EagerBufferedReader is a reader that can calculate its digest and size By eagerly loading the data, and proxying a buffer for it in memory. This can lead to extraordinary memory usage for large files, so one should be careful when using this without considering the potential size of the data.

func NewEagerBufferedReader

func NewEagerBufferedReader(r io.Reader) *EagerBufferedReader

NewEagerBufferedReader creates a new EagerBufferedReader instance.

func (*EagerBufferedReader) Close

func (b *EagerBufferedReader) Close() error

func (*EagerBufferedReader) Digest

func (b *EagerBufferedReader) Digest() (string, bool)

func (*EagerBufferedReader) HasPrecalculatedDigest

func (b *EagerBufferedReader) HasPrecalculatedDigest() bool

func (*EagerBufferedReader) HasPrecalculatedSize

func (b *EagerBufferedReader) HasPrecalculatedSize() bool

func (*EagerBufferedReader) LoadEagerly

func (b *EagerBufferedReader) LoadEagerly() error

func (*EagerBufferedReader) Loaded

func (b *EagerBufferedReader) Loaded() bool

func (*EagerBufferedReader) MediaType

func (b *EagerBufferedReader) MediaType() (string, bool)

func (*EagerBufferedReader) Read

func (b *EagerBufferedReader) Read(p []byte) (n int, err error)

func (*EagerBufferedReader) SetMediaType

func (b *EagerBufferedReader) SetMediaType(mediaType string)

func (*EagerBufferedReader) SetPrecalculatedDigest

func (b *EagerBufferedReader) SetPrecalculatedDigest(digest string)

func (*EagerBufferedReader) SetPrecalculatedSize

func (b *EagerBufferedReader) SetPrecalculatedSize(size int64)

func (*EagerBufferedReader) Size

func (b *EagerBufferedReader) Size() int64

type MediaTypeAware

type MediaTypeAware interface {
	// MediaType returns the media type of the blob if known.
	MediaType() (mediaType string, known bool)
}

MediaTypeAware is an interface that represents any arbitrary object that can be interpreted as an object that is associated with a MediaType.

Even though Media Types are not a part of the core specification, they are used in many places for content-type awareness. SetMediaType is used to set the media type of the object if it is known.

type MediaTypeOverrideable

type MediaTypeOverrideable interface {
	// SetMediaType overrides the media type of the blob if it is known in advance.
	// It MUST be safe for concurrent use, serializing as necessary.
	// It MUST be safe to call multiple times, overwriting the mediaType every time.
	SetMediaType(mediaType string)
}

MediaTypeOverrideable is an interface that represents any arbitrary object that can be overwritten with custom MediaTypes ahead of a potential read/write operation.

type ReadOnlyBlob

type ReadOnlyBlob interface {
	// ReadCloser returns a reader to incrementally access byte stream content
	// It is the caller's responsibility to close the reader.
	//
	// ReadCloser MUST be safe for concurrent use, serializing access as necessary.
	// ReadCloser MUST be able to be called multiple times, which each invocation
	// returning a new reader, that starts from the beginning of the blob.
	//
	// Note that this behavior is not parallel to WriteableBlob.WriteCloser
	ReadCloser() (io.ReadCloser, error)
}

type SizeAware

type SizeAware interface {
	// Size returns the blob size in bytes if known.
	// If the size is unknown, it MUST return SizeUnknown.
	Size() (size int64)
}

SizeAware is an interface that represents any arbitrary object that can be sized.

Size is used to always determine the size of the object in bytes.

type SizePrecalculatable

type SizePrecalculatable interface {
	// HasPrecalculatedSize returns true if a precalculated size of the blob is known in advance.
	HasPrecalculatedSize() bool
	// SetPrecalculatedSize sets the size of the blob if it is known in advance.
	// It MUST be safe for concurrent use, serializing as necessary.
	// It MUST be safe to call multiple times, with the same or greater different size.
	// Once called, HasPrecalculatedSize MUST return true.
	// If called with a size less than the current size, it MUST ignore this size.
	SetPrecalculatedSize(size int64)
}

SizePrecalculatable is an interface that represents any arbitrary object that can be set with Sizes ahead of a potential read/write operation.

HasPrecalculatedSize is used to determine if the size of the object is known in advance. SetPrecalculatedSize is used to set the size of the object if it is known in advance.

type WriteableBlob

type WriteableBlob interface {
	// WriteCloser returns a writer to incrementally write byte stream content
	// It is the caller's responsibility to close the writer.
	//
	// WriteCloser MUST be safe for concurrent use, serializing access as necessary.
	// WriteCloser MUST be able to be called multiple times, which each invocation
	// returning a new writer, that appends to the previous one.
	WriteCloser() (io.WriteCloser, error)
}

WriteableBlob is an interface that represents a Binary Large Object that can be written to.

Directories

Path Synopsis
Package filesystem provides various interfaces and types for working with Binary Large Object's (BLOBs) based on a FileSystem.
Package filesystem provides various interfaces and types for working with Binary Large Object's (BLOBs) based on a FileSystem.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL