storage

package
v0.0.0-...-cf8f15b Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 5, 2026 License: Apache-2.0 Imports: 70 Imported by: 32

Documentation

Index

Constants

View Source 
const (
	// DefaultCompressFrameSize is the default uncompressed size of each
	// compression frame (2 MiB). Overridable via CompressConfig.FrameSizeKB.
	// The last frame in a file may be shorter.
	//
	// The chunker fetches one frame at a time from storage on a cache miss.
	// Larger frame sizes mean more data cached per fetch (faster warm-up and
	// fewer remote storage round-trips), but higher memory and I/O cost per
	// miss.
	//
	// This MUST be multiple of every block/page size:
	//   - header.HugepageSize (2 MiB) — UFFD huge-page size, also used by prefetch
	//   - header.RootfsBlockSize (4 KiB) — NBD / rootfs block size
	DefaultCompressFrameSize = 2 * 1024 * 1024

	// Use case identifiers for per-use-case compression targeting via LaunchDarkly.
	UseCaseBuild = "build"
	UseCasePause = "pause"
)
View Source 
const (
	CompressionNone = CompressionType(iota)
	CompressionZstd
	CompressionLZ4
)
View Source 
const (
	AttrSource   = "source"
	AttrCodec    = "codec"
	AttrOutcome  = "outcome"
	AttrFileType = "file_type"
	AttrTrigger  = "trigger"
)

Attribute vocabulary for the orchestrator.read.* / orchestrator.chunk.* / orchestrator.writeback metrics.

View Source 
const (
	TriggerRead  = "read"
	TriggerWrite = "write"
)

Writeback triggers: a read-miss cache fill vs a build/store write-through.

View Source 
const (
	OutcomeOK           = "ok"
	OutcomeNotFound     = "not_found"
	OutcomeErrCanceled  = "err_canceled"
	OutcomeErrIO        = "err_io"
	OutcomeErrTimeout   = "err_timeout"
	OutcomeTransitioned = "transitioned"
	// OutcomeContended is a writeback skipped because another goroutine held the
	// NFS chunk lock — normal cache dedup, nothing written.
	OutcomeContended = "contended"
)
View Source 
const (
	GuestEnvdPath = "/usr/bin/envd"

	MemfileName  = "memfile"
	RootfsName   = "rootfs.ext4"
	SnapfileName = "snapfile"
	MetadataName = "metadata.json"

	HeaderSuffix = ".header"
)
View Source 
const (
	ObjectMetadataTeamID      = storageopts.ObjectMetadataTeamID
	ObjectMetadataTemplateID  = storageopts.ObjectMetadataTemplateID
	ObjectMetadataBuildOrigin = storageopts.ObjectMetadataBuildOrigin
	ObjectMetadataLogicalSize = storageopts.ObjectMetadataLogicalSize
	ObjectMetadataMappedSize  = storageopts.ObjectMetadataMappedSize
	ObjectMetadataDiffSize    = storageopts.ObjectMetadataDiffSize

	ObjectOriginPause              = storageopts.ObjectOriginPause
	ObjectOriginTemplateBuild      = storageopts.ObjectOriginTemplateBuild
	ObjectOriginTemplateBuildCache = storageopts.ObjectOriginTemplateBuildCache
	ObjectOriginSnapshotTemplate   = storageopts.ObjectOriginSnapshotTemplate
)
View Source 
const ObjectMetadataSoftDeleted = storageopts.ObjectMetadataSoftDeleted

ObjectMetadataSoftDeleted is the storage-index soft-delete tombstone key.

Variables

View Source 
var (
	ErrOffsetUnaligned = errors.New("offset must be a multiple of chunk size")
	ErrBufferTooSmall  = errors.New("buffer is too small")
	ErrMultipleChunks  = errors.New("cannot read multiple chunks")
	ErrBufferTooLarge  = errors.New("buffer is too large")
)
View Source 
var BuildCacheStorageConfig = StorageConfig{
	GetLocalBasePath: func() string {
		return env.GetEnv("LOCAL_BUILD_CACHE_STORAGE_BASE_PATH", "/tmp/build-cache")
	},
	GetBucketName: func() string {
		return utils.RequiredEnv("BUILD_CACHE_BUCKET_NAME", "Bucket for storing build cache files")
	},
}
View Source 
var ErrMetadataUnsupported = errors.New("blob does not support reading custom metadata")

ErrMetadataUnsupported means the blob's backend cannot read custom metadata (no MetadataReader). It is distinct from "read succeeded, no metadata" so callers (e.g. soft-delete enforcement) can fail closed on an unverifiable backend instead of assuming there is no tombstone.

View Source 
var ErrObjectNotExist = errors.New("object does not exist")
View Source 
var ErrObjectRateLimited = errors.New("object access rate limited")

ErrObjectRateLimited means per-object mutation rate limiting — multiple concurrent writers racing to write the same content-addressed object.

View Source 
var ErrObjectSoftDeleted = errors.New("object soft-deleted by storage index")

ErrObjectSoftDeleted means the storage index has marked this object for deletion (soft-delete tombstone in custom metadata) and enforcement is on.

View Source 
var TemplateStorageConfig = StorageConfig{
	GetLocalBasePath: func() string {
		return env.GetEnv("LOCAL_TEMPLATE_STORAGE_BASE_PATH", "/tmp/templates")
	},
	GetBucketName: func() string {
		return utils.RequiredEnv("TEMPLATE_BUCKET_NAME", "Bucket for storing template files")
	},
}
View Source 
var UncompressedFrameTable = &FrameTable{}

UncompressedFrameTable is the canonical sentinel for "data is stored uncompressed" — an empty FrameTable has no frames to decompress, so a reader handed this back skips the U→C translation entirely. Shared across callers to avoid per-call allocation; read-only and must never be mutated.

View Source 
var UncompressedFullFrameTable = &FullFrameTable{}

UncompressedFullFrameTable is the Full counterpart of UncompressedFrameTable: the canonical sentinel an authoritative source latches when a build is known to be stored uncompressed end-to-end. Shared across callers; read-only.

Functions

func ComputeUploadHMAC 

func ComputeUploadHMAC(key []byte, path string, expires int64) string

func ErrAttrs 

func ErrAttrs(o SeekableObjectType, s Source, c CompressionType, err error) metric.MeasurementOption

func GetBlob 

func GetBlob(ctx context.Context, b Blob) ([]byte, error)

GetBlob is a convenience wrapper that wraps b.WriteTo interface to return a byte slice.

func IsLocal 

func IsLocal() bool

IsLocal reports whether the configured storage provider is the local filesystem backend.

func OKAttrs 

func OKAttrs(o SeekableObjectType, s Source, c CompressionType) metric.MeasurementOption

func Outcome 

func Outcome(err error) string

Outcome maps a read-path error to the outcome enum. PeerTransitionedError is a routing signal, not a failure, so it gets its own bucket (not err_io).

func ReclaimSandboxFiles 

func ReclaimSandboxFiles(tempDir, sandboxCacheDir string) (int, []error)

ReclaimSandboxFiles removes leaked sandbox files matching SandboxFileGlobs, left over from sandboxes that did not shut down cleanly. It returns the number of files removed and any per-file removal failures. Files that no longer exist are treated as already reclaimed.

func RecordPipelineEfficiency 

func RecordPipelineEfficiency(ctx context.Context, ratio float64, attrs metric.MeasurementOption)

func RecordReadBlob 

func RecordReadBlob(ctx context.Context, dur time.Duration, bytes int64, path string, src Source, err error)

blob file_type is not a SeekableObjectType, so read.blob* build attrs inline rather than via the precomputed OKAttrs table.

func RecordReadBlobDecompress 

func RecordReadBlobDecompress(ctx context.Context, dur time.Duration, bytes int64, path string, ct CompressionType, err error)

func RecordReadFetch 

func RecordReadFetch(ctx context.Context, dur time.Duration, bytes int64, attrs metric.MeasurementOption)

func RecordReadOpen 

func RecordReadOpen(ctx context.Context, dur time.Duration, ot SeekableObjectType, src Source, ct CompressionType, err error)

RecordReadOpen records one layer's own open attempt (not the delegated inner call), so a slow peer/NFS open isn't misattributed to the resolved source.

func RecordReadRead 

func RecordReadRead(ctx context.Context, dur time.Duration, bytes int64, attrs metric.MeasurementOption)

func RecordReadSize 

func RecordReadSize(ctx context.Context, dur time.Duration, ot SeekableObjectType, src Source, err error)

func SandboxFileGlobs 

func SandboxFileGlobs(tempDir, sandboxCacheDir string) []string

SandboxFileGlobs returns glob patterns matching the on-disk files a sandbox creates: firecracker/uffd sockets and the metrics fifo under tempDir, plus the rootfs overlay and link files under sandboxCacheDir. The patterns mirror the path builders above and are the single source of truth used by startup reclaim. sandboxCacheDir may be empty, in which case the cache patterns are omitted.

func SizeSidecar 

func SizeSidecar(objectPath string) string

SizeSidecar returns the sidecar path that stores the original uncompressed size for a compressed object (e.g. "/data/memfile.zstd" → "/data/memfile.zstd.uncompressed-size"). Used by the FS backend where GCS-style object metadata is unavailable.

func SplitPath 

func SplitPath(path string) (buildID, fileName string)

SplitPath splits a storage path of the form "{buildID}/{fileName}" back into its components. This is the inverse of the path methods.

func StripCompression 

func StripCompression(name string) string

StripCompression removes a known compression suffix from a file name. For example: "memfile.zstd" → "memfile". If no known suffix is present, the name is returned unchanged.

func UploadBlob 

func UploadBlob(ctx context.Context, provider StorageProvider, remotePath string, localPath string, opts ...PutOption) error

func ValidateUploadToken 

func ValidateUploadToken(key []byte, path string, expires int64, token string) bool

ValidateUploadToken validates an HMAC token for a local upload URL. Exported so that the upload handler in the orchestrator can use it.

func WithSkipCacheWriteback 

func WithSkipCacheWriteback(ctx context.Context) context.Context

WithSkipCacheWriteback returns a context that signals the NFS cache layer to skip writing fetched data back to the local cache. This is used by the prefetcher to avoid polluting the shared NFS cache with prefetch-specific reads.

Types

type Blob 

type Blob interface {
	WriteTo(ctx context.Context, dst io.Writer) (int64, error)
	Put(ctx context.Context, data []byte, opts ...storageopts.PutOption) error
	Exists(ctx context.Context) (bool, error)
}

type CachePaths 

type CachePaths struct {
	Paths

	// CacheIdentifier is used to distinguish between each entry in the cache to prevent deleting the cache files when the template cache entry is being closed and a new one is being created.
	CacheIdentifier string
	// contains filtered or unexported fields
}

func (CachePaths) CacheMetadata 

func (c CachePaths) CacheMetadata() string

func (CachePaths) CacheSnapfile 

func (c CachePaths) CacheSnapfile() string

func (CachePaths) Close 

func (c CachePaths) Close() error

func (CachePaths) NewSandboxFiles 

func (c CachePaths) NewSandboxFiles(sandboxID string) *SandboxFiles

func (CachePaths) NewSandboxFilesWithStaticID 

func (c CachePaths) NewSandboxFilesWithStaticID(sandboxID string, staticID string) *SandboxFiles

type CompleteMultipartUpload 

type CompleteMultipartUpload struct {
	XMLName string `xml:"CompleteMultipartUpload"`
	Parts   []Part `xml:"Part"`
}

type CompressConfig 

type CompressConfig struct {
	Enabled            bool   `env:"COMPRESS_ENABLED"              envDefault:"false"`
	Type               string `env:"COMPRESS_TYPE"                 envDefault:""`
	Level              int    `env:"COMPRESS_LEVEL"                envDefault:"0"`
	FrameSizeKB        int    `env:"COMPRESS_FRAME_SIZE_KB"        envDefault:"0"`
	MinPartSizeMB      int    `env:"COMPRESS_MIN_PART_SIZE_MB"     envDefault:"0"`
	FrameEncodeWorkers int    `env:"COMPRESS_FRAME_ENCODE_WORKERS" envDefault:"0"`
	EncoderConcurrency int    `env:"COMPRESS_ENCODER_CONCURRENCY"  envDefault:"0"`
}

CompressConfig is the base compression configuration, loaded from environment variables at startup. Feature flags may override individual fields at runtime at the upload boundary. Zero value means compression disabled.

func CompressConfigFromOpts 

func CompressConfigFromOpts(p PutOptions) CompressConfig

CompressConfigFromOpts returns the typed CompressConfig set by WithCompressConfig, or the zero value if absent.

func (CompressConfig) CompressionType 

func (c CompressConfig) CompressionType() CompressionType

CompressionType returns the parsed CompressionType.

func (CompressConfig) FrameSize 

func (c CompressConfig) FrameSize() int

FrameSize returns the frame size in bytes.

func (CompressConfig) IsCompressionEnabled 

func (c CompressConfig) IsCompressionEnabled() bool

IsCompressionEnabled reports whether compression is configured and active.

func (CompressConfig) MinPartSize 

func (c CompressConfig) MinPartSize() int64

MinPartSize returns the minimum compressed part size in bytes. Parts accumulate frames until they reach this threshold.

type CompressionType 

type CompressionType byte

func (CompressionType) String 

func (ct CompressionType) String() string

func (CompressionType) Suffix 

func (ct CompressionType) Suffix() string

type Config 

type Config struct {
	CompressConfig

	SandboxCacheDir  string `env:"SANDBOX_CACHE_DIR,expand"  envDefault:"${ORCHESTRATOR_BASE_PATH}/sandbox"`
	SnapshotCacheDir string `env:"SNAPSHOT_CACHE_DIR,expand" envDefault:"/mnt/snapshot-cache"`
	TemplateCacheDir string `env:"TEMPLATE_CACHE_DIR,expand" envDefault:"${ORCHESTRATOR_BASE_PATH}/template"`
}

type FrameSink 

type FrameSink = storageopts.FrameSink

type FrameSize 

type FrameSize struct {
	U int32
	C int32
}

FrameSize holds the uncompressed (U) and compressed (C) byte size of a single frame.

func (FrameSize) String 

func (s FrameSize) String() string

type FrameTable 

type FrameTable struct {
	// contains filtered or unexported fields
}

FrameTable is the decompression index for a compressed diff file. Immutable after construction; safe to share across goroutines. Sparse tables (gaps between entries) are supported.

func DeserializeFrameTable 

func DeserializeFrameTable(r io.Reader) (*FrameTable, error)

DeserializeFrameTable reads a FrameTable from r. Returns nil for uncompressed builds (compressionType=0 or numFrames=0).

func (*FrameTable) CompressedSize 

func (ft *FrameTable) CompressedSize() int64

CompressedSize returns the total compressed size across all frames. Nil-safe: returns 0 for nil (uncompressed leg in mixed-mode V4 upload).

func (*FrameTable) CompressionType 

func (ft *FrameTable) CompressionType() CompressionType

CompressionType returns the compression type. Nil-safe: returns CompressionNone for nil.

func (*FrameTable) FrameAt 

func (ft *FrameTable) FrameAt(i int) (startU, endU, startC, endC int64)

func (*FrameTable) IsCompressed 

func (ft *FrameTable) IsCompressed() bool

IsCompressed reports whether ft is non-nil and has a compression type set.

func (*FrameTable) LocateCompressed 

func (ft *FrameTable) LocateCompressed(offset int64) (Range, error)

LocateCompressed maps a U-space offset to its C-space byte range. This is the final step of the read path: after GetShiftedMapping resolves the virtual offset to a build-local U-offset, this locates the compressed bytes to fetch from storage.

func (*FrameTable) LocateUncompressed 

func (ft *FrameTable) LocateUncompressed(offset int64) (Range, error)

LocateUncompressed returns the uncompressed byte range for the frame containing the given uncompressed offset.

func (*FrameTable) NumFrames 

func (ft *FrameTable) NumFrames() int

func (*FrameTable) Serialize 

func (ft *FrameTable) Serialize(w io.Writer) error

Serialize writes the frame table to w in binary little-endian format. Nil-safe: writes zeros for type and count.

func (*FrameTable) TrimToRanges 

func (ft *FrameTable) TrimToRanges(ranges []Range) *FrameTable

TrimToRanges returns a new FrameTable containing only the frames that overlap with at least one of the given U-space byte ranges. Used during V4 header serialization to keep headers compact when a build has many frames but only a few are referenced in the current layer. Nil-safe: returns ft unchanged when ft is nil or ranges is empty.

func (*FrameTable) UncompressedSize 

func (ft *FrameTable) UncompressedSize() int64

UncompressedSize returns the total uncompressed size across all frames. Nil-safe: returns 0 for nil (uncompressed leg in mixed-mode V4 upload).

type FullFrameTable 

type FullFrameTable struct {
	// contains filtered or unexported fields
}

FullFrameTable marks a FrameTable that covers an entire file with no gaps. Produced by compressStream / CompressBytes / StoreFile / UploadFramed. The inner FrameTable is unexported and unembedded so methods are not promoted; consumers must reach functionality through nil-safe Table().

func CompressBytes 

func CompressBytes(ctx context.Context, data []byte, cfg CompressConfig) (*FullFrameTable, []byte, [32]byte, error)

func FullFromTable 

func FullFromTable(ft *FrameTable) *FullFrameTable

FullFromTable promotes a *FrameTable to a *FullFrameTable. The caller MUST guarantee ft describes the entire file with no gaps — the only legitimate promotion is from a self-build's loaded.Builds[self].FrameData entry, which build_upload_v4 unconditionally populates from the full upload. Returns the UncompressedFullFrameTable sentinel for nil or empty inputs.

func NewFullFrameTable 

func NewFullFrameTable(ct CompressionType, sizes []FrameSize) *FullFrameTable

NewFullFrameTable creates a FullFrameTable from consecutive frame sizes, computing absolute offsets starting from zero.

func UploadFramed 

func UploadFramed(ctx context.Context, provider StorageProvider, remotePath string, localPath string, opts ...PutOption) (*FullFrameTable, [32]byte, error)

func (*FullFrameTable) Table 

func (ft *FullFrameTable) Table() *FrameTable

Table returns the underlying *FrameTable, nil-safely.

type InitiateMultipartUploadResult 

type InitiateMultipartUploadResult struct {
	Bucket   string `xml:"Bucket"`
	Key      string `xml:"Key"`
	UploadID string `xml:"UploadId"`
}

type MetadataReader 

type MetadataReader interface {
	Metadata(ctx context.Context) (ObjectMetadata, error)
}

MetadataReader is an optional Blob capability: read the object's custom metadata without downloading it. Backends that can't answer cheaply omit it.

type MockBlob 

type MockBlob struct {
	mock.Mock
}

MockBlob is an autogenerated mock type for the Blob type

func NewMockBlob 

func NewMockBlob(t interface {
	mock.TestingT
	Cleanup(func())
}) *MockBlob

NewMockBlob creates a new instance of MockBlob. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations. The first argument is typically a *testing.T value.

func (*MockBlob) EXPECT 

func (_m *MockBlob) EXPECT() *MockBlob_Expecter

func (*MockBlob) Exists 

func (_mock *MockBlob) Exists(ctx context.Context) (bool, error)

Exists provides a mock function for the type MockBlob

func (*MockBlob) Put 

func (_mock *MockBlob) Put(ctx context.Context, data []byte, opts ...storageopts.PutOption) error

Put provides a mock function for the type MockBlob

func (*MockBlob) WriteTo 

func (_mock *MockBlob) WriteTo(ctx context.Context, dst io.Writer) (int64, error)

WriteTo provides a mock function for the type MockBlob

type MockBlob_Exists_Call 

type MockBlob_Exists_Call struct {
	*mock.Call
}

MockBlob_Exists_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Exists'

func (*MockBlob_Exists_Call) Return 

func (_c *MockBlob_Exists_Call) Return(b bool, err error) *MockBlob_Exists_Call

func (*MockBlob_Exists_Call) Run 

func (_c *MockBlob_Exists_Call) Run(run func(ctx context.Context)) *MockBlob_Exists_Call

func (*MockBlob_Exists_Call) RunAndReturn 

func (_c *MockBlob_Exists_Call) RunAndReturn(run func(ctx context.Context) (bool, error)) *MockBlob_Exists_Call

type MockBlob_Expecter 

type MockBlob_Expecter struct {
	// contains filtered or unexported fields
}

func (*MockBlob_Expecter) Exists 

func (_e *MockBlob_Expecter) Exists(ctx interface{}) *MockBlob_Exists_Call

Exists is a helper method to define mock.On call

  • ctx context.Context

func (*MockBlob_Expecter) Put 

func (_e *MockBlob_Expecter) Put(ctx interface{}, data interface{}, opts ...interface{}) *MockBlob_Put_Call

Put is a helper method to define mock.On call

  • ctx context.Context
  • data []byte
  • opts ...storageopts.PutOption

func (*MockBlob_Expecter) WriteTo 

func (_e *MockBlob_Expecter) WriteTo(ctx interface{}, dst interface{}) *MockBlob_WriteTo_Call

WriteTo is a helper method to define mock.On call

  • ctx context.Context
  • dst io.Writer

type MockBlob_Put_Call 

type MockBlob_Put_Call struct {
	*mock.Call
}

MockBlob_Put_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Put'

func (*MockBlob_Put_Call) Return 

func (_c *MockBlob_Put_Call) Return(err error) *MockBlob_Put_Call

func (*MockBlob_Put_Call) Run 

func (_c *MockBlob_Put_Call) Run(run func(ctx context.Context, data []byte, opts ...storageopts.PutOption)) *MockBlob_Put_Call

func (*MockBlob_Put_Call) RunAndReturn 

func (_c *MockBlob_Put_Call) RunAndReturn(run func(ctx context.Context, data []byte, opts ...storageopts.PutOption) error) *MockBlob_Put_Call

type MockBlob_WriteTo_Call 

type MockBlob_WriteTo_Call struct {
	*mock.Call
}

MockBlob_WriteTo_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'WriteTo'

func (*MockBlob_WriteTo_Call) Return 

func (_c *MockBlob_WriteTo_Call) Return(n int64, err error) *MockBlob_WriteTo_Call

func (*MockBlob_WriteTo_Call) Run 

func (_c *MockBlob_WriteTo_Call) Run(run func(ctx context.Context, dst io.Writer)) *MockBlob_WriteTo_Call

func (*MockBlob_WriteTo_Call) RunAndReturn 

func (_c *MockBlob_WriteTo_Call) RunAndReturn(run func(ctx context.Context, dst io.Writer) (int64, error)) *MockBlob_WriteTo_Call

type MockFeatureFlagsClient 

type MockFeatureFlagsClient struct {
	mock.Mock
}

MockFeatureFlagsClient is an autogenerated mock type for the featureFlagsClient type

func NewMockFeatureFlagsClient 

func NewMockFeatureFlagsClient(t interface {
	mock.TestingT
	Cleanup(func())
}) *MockFeatureFlagsClient

NewMockFeatureFlagsClient creates a new instance of MockFeatureFlagsClient. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations. The first argument is typically a *testing.T value.

func (*MockFeatureFlagsClient) BoolFlag 

func (_mock *MockFeatureFlagsClient) BoolFlag(ctx context.Context, flag featureflags.BoolFlag, ldctx ...ldcontext.Context) bool

BoolFlag provides a mock function for the type MockFeatureFlagsClient

func (*MockFeatureFlagsClient) EXPECT 

func (_m *MockFeatureFlagsClient) EXPECT() *MockFeatureFlagsClient_Expecter

func (*MockFeatureFlagsClient) IntFlag 

func (_mock *MockFeatureFlagsClient) IntFlag(ctx context.Context, flag featureflags.IntFlag, ldctx ...ldcontext.Context) int

IntFlag provides a mock function for the type MockFeatureFlagsClient

type MockFeatureFlagsClient_BoolFlag_Call 

type MockFeatureFlagsClient_BoolFlag_Call struct {
	*mock.Call
}

MockFeatureFlagsClient_BoolFlag_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'BoolFlag'

func (*MockFeatureFlagsClient_BoolFlag_Call) Return 

func (_c *MockFeatureFlagsClient_BoolFlag_Call) Return(b bool) *MockFeatureFlagsClient_BoolFlag_Call

func (*MockFeatureFlagsClient_BoolFlag_Call) Run 

func (_c *MockFeatureFlagsClient_BoolFlag_Call) Run(run func(ctx context.Context, flag featureflags.BoolFlag, ldctx ...ldcontext.Context)) *MockFeatureFlagsClient_BoolFlag_Call

func (*MockFeatureFlagsClient_BoolFlag_Call) RunAndReturn 

func (_c *MockFeatureFlagsClient_BoolFlag_Call) RunAndReturn(run func(ctx context.Context, flag featureflags.BoolFlag, ldctx ...ldcontext.Context) bool) *MockFeatureFlagsClient_BoolFlag_Call

type MockFeatureFlagsClient_Expecter 

type MockFeatureFlagsClient_Expecter struct {
	// contains filtered or unexported fields
}

func (*MockFeatureFlagsClient_Expecter) BoolFlag 

func (_e *MockFeatureFlagsClient_Expecter) BoolFlag(ctx interface{}, flag interface{}, ldctx ...interface{}) *MockFeatureFlagsClient_BoolFlag_Call

BoolFlag is a helper method to define mock.On call

  • ctx context.Context
  • flag featureflags.BoolFlag
  • ldctx ...ldcontext.Context

func (*MockFeatureFlagsClient_Expecter) IntFlag 

func (_e *MockFeatureFlagsClient_Expecter) IntFlag(ctx interface{}, flag interface{}, ldctx ...interface{}) *MockFeatureFlagsClient_IntFlag_Call

IntFlag is a helper method to define mock.On call

  • ctx context.Context
  • flag featureflags.IntFlag
  • ldctx ...ldcontext.Context

type MockFeatureFlagsClient_IntFlag_Call 

type MockFeatureFlagsClient_IntFlag_Call struct {
	*mock.Call
}

MockFeatureFlagsClient_IntFlag_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'IntFlag'

func (*MockFeatureFlagsClient_IntFlag_Call) Return 

func (_c *MockFeatureFlagsClient_IntFlag_Call) Return(n int) *MockFeatureFlagsClient_IntFlag_Call

func (*MockFeatureFlagsClient_IntFlag_Call) Run 

func (_c *MockFeatureFlagsClient_IntFlag_Call) Run(run func(ctx context.Context, flag featureflags.IntFlag, ldctx ...ldcontext.Context)) *MockFeatureFlagsClient_IntFlag_Call

func (*MockFeatureFlagsClient_IntFlag_Call) RunAndReturn 

func (_c *MockFeatureFlagsClient_IntFlag_Call) RunAndReturn(run func(ctx context.Context, flag featureflags.IntFlag, ldctx ...ldcontext.Context) int) *MockFeatureFlagsClient_IntFlag_Call

type MockReader 

type MockReader struct {
	mock.Mock
}

MockReader is an autogenerated mock type for the Reader type

func NewMockReader 

func NewMockReader(t interface {
	mock.TestingT
	Cleanup(func())
}) *MockReader

NewMockReader creates a new instance of MockReader. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations. The first argument is typically a *testing.T value.

func (*MockReader) EXPECT 

func (_m *MockReader) EXPECT() *MockReader_Expecter

func (*MockReader) Read 

func (_mock *MockReader) Read(p []byte) (int, error)

Read provides a mock function for the type MockReader

type MockReader_Expecter 

type MockReader_Expecter struct {
	// contains filtered or unexported fields
}

func (*MockReader_Expecter) Read 

func (_e *MockReader_Expecter) Read(p interface{}) *MockReader_Read_Call

Read is a helper method to define mock.On call

  • p []byte

type MockReader_Read_Call 

type MockReader_Read_Call struct {
	*mock.Call
}

MockReader_Read_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Read'

func (*MockReader_Read_Call) Return 

func (_c *MockReader_Read_Call) Return(n int, err error) *MockReader_Read_Call

func (*MockReader_Read_Call) Run 

func (_c *MockReader_Read_Call) Run(run func(p []byte)) *MockReader_Read_Call

func (*MockReader_Read_Call) RunAndReturn 

func (_c *MockReader_Read_Call) RunAndReturn(run func(p []byte) (int, error)) *MockReader_Read_Call

type MockSeekable 

type MockSeekable struct {
	mock.Mock
}

MockSeekable is an autogenerated mock type for the Seekable type

func NewMockSeekable 

func NewMockSeekable(t interface {
	mock.TestingT
	Cleanup(func())
}) *MockSeekable

NewMockSeekable creates a new instance of MockSeekable. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations. The first argument is typically a *testing.T value.

func (*MockSeekable) EXPECT 

func (_m *MockSeekable) EXPECT() *MockSeekable_Expecter

func (*MockSeekable) OpenRangeReader 

func (_mock *MockSeekable) OpenRangeReader(ctx context.Context, offsetU int64, length int64, frameTable *FrameTable) (RangeReader, Source, error)

OpenRangeReader provides a mock function for the type MockSeekable

func (*MockSeekable) Size 

func (_mock *MockSeekable) Size(ctx context.Context) (int64, error)

Size provides a mock function for the type MockSeekable

func (*MockSeekable) StoreFile 

func (_mock *MockSeekable) StoreFile(ctx context.Context, path string, opts ...PutOption) (*FullFrameTable, [32]byte, error)

StoreFile provides a mock function for the type MockSeekable

type MockSeekable_Expecter 

type MockSeekable_Expecter struct {
	// contains filtered or unexported fields
}

func (*MockSeekable_Expecter) OpenRangeReader 

func (_e *MockSeekable_Expecter) OpenRangeReader(ctx interface{}, offsetU interface{}, length interface{}, frameTable interface{}) *MockSeekable_OpenRangeReader_Call

OpenRangeReader is a helper method to define mock.On call

  • ctx context.Context
  • offsetU int64
  • length int64
  • frameTable *FrameTable

func (*MockSeekable_Expecter) Size 

func (_e *MockSeekable_Expecter) Size(ctx interface{}) *MockSeekable_Size_Call

Size is a helper method to define mock.On call

  • ctx context.Context

func (*MockSeekable_Expecter) StoreFile 

func (_e *MockSeekable_Expecter) StoreFile(ctx interface{}, path interface{}, opts ...interface{}) *MockSeekable_StoreFile_Call

StoreFile is a helper method to define mock.On call

  • ctx context.Context
  • path string
  • opts ...PutOption

type MockSeekable_OpenRangeReader_Call 

type MockSeekable_OpenRangeReader_Call struct {
	*mock.Call
}

MockSeekable_OpenRangeReader_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'OpenRangeReader'

func (*MockSeekable_OpenRangeReader_Call) Return 

func (_c *MockSeekable_OpenRangeReader_Call) Return(rangeReader RangeReader, source Source, err error) *MockSeekable_OpenRangeReader_Call

func (*MockSeekable_OpenRangeReader_Call) Run 

func (_c *MockSeekable_OpenRangeReader_Call) Run(run func(ctx context.Context, offsetU int64, length int64, frameTable *FrameTable)) *MockSeekable_OpenRangeReader_Call

func (*MockSeekable_OpenRangeReader_Call) RunAndReturn 

func (_c *MockSeekable_OpenRangeReader_Call) RunAndReturn(run func(ctx context.Context, offsetU int64, length int64, frameTable *FrameTable) (RangeReader, Source, error)) *MockSeekable_OpenRangeReader_Call

type MockSeekable_Size_Call 

type MockSeekable_Size_Call struct {
	*mock.Call
}

MockSeekable_Size_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Size'

func (*MockSeekable_Size_Call) Return 

func (_c *MockSeekable_Size_Call) Return(n int64, err error) *MockSeekable_Size_Call

func (*MockSeekable_Size_Call) Run 

func (_c *MockSeekable_Size_Call) Run(run func(ctx context.Context)) *MockSeekable_Size_Call

func (*MockSeekable_Size_Call) RunAndReturn 

func (_c *MockSeekable_Size_Call) RunAndReturn(run func(ctx context.Context) (int64, error)) *MockSeekable_Size_Call

type MockSeekable_StoreFile_Call 

type MockSeekable_StoreFile_Call struct {
	*mock.Call
}

MockSeekable_StoreFile_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'StoreFile'

func (*MockSeekable_StoreFile_Call) Return 

func (_c *MockSeekable_StoreFile_Call) Return(fullFrameTable *FullFrameTable, bytes [32]byte, err error) *MockSeekable_StoreFile_Call

func (*MockSeekable_StoreFile_Call) Run 

func (_c *MockSeekable_StoreFile_Call) Run(run func(ctx context.Context, path string, opts ...PutOption)) *MockSeekable_StoreFile_Call

func (*MockSeekable_StoreFile_Call) RunAndReturn 

func (_c *MockSeekable_StoreFile_Call) RunAndReturn(run func(ctx context.Context, path string, opts ...PutOption) (*FullFrameTable, [32]byte, error)) *MockSeekable_StoreFile_Call

type MockStorageProvider 

type MockStorageProvider struct {
	mock.Mock
}

MockStorageProvider is an autogenerated mock type for the StorageProvider type

func NewMockStorageProvider 

func NewMockStorageProvider(t interface {
	mock.TestingT
	Cleanup(func())
}) *MockStorageProvider

NewMockStorageProvider creates a new instance of MockStorageProvider. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations. The first argument is typically a *testing.T value.

func (*MockStorageProvider) DeleteObjectsWithPrefix 

func (_mock *MockStorageProvider) DeleteObjectsWithPrefix(ctx context.Context, prefix string) error

DeleteObjectsWithPrefix provides a mock function for the type MockStorageProvider

func (*MockStorageProvider) EXPECT 

func (_m *MockStorageProvider) EXPECT() *MockStorageProvider_Expecter

func (*MockStorageProvider) GetDetails 

func (_mock *MockStorageProvider) GetDetails() string

GetDetails provides a mock function for the type MockStorageProvider

func (*MockStorageProvider) OpenBlob 

func (_mock *MockStorageProvider) OpenBlob(ctx context.Context, path string) (Blob, error)

OpenBlob provides a mock function for the type MockStorageProvider

func (*MockStorageProvider) OpenSeekable 

func (_mock *MockStorageProvider) OpenSeekable(ctx context.Context, path string) (Seekable, error)

OpenSeekable provides a mock function for the type MockStorageProvider

func (*MockStorageProvider) UploadSignedURL 

func (_mock *MockStorageProvider) UploadSignedURL(ctx context.Context, path string, ttl time.Duration) (string, error)

UploadSignedURL provides a mock function for the type MockStorageProvider

type MockStorageProvider_DeleteObjectsWithPrefix_Call 

type MockStorageProvider_DeleteObjectsWithPrefix_Call struct {
	*mock.Call
}

MockStorageProvider_DeleteObjectsWithPrefix_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'DeleteObjectsWithPrefix'

func (*MockStorageProvider_DeleteObjectsWithPrefix_Call) Return 

func (_c *MockStorageProvider_DeleteObjectsWithPrefix_Call) Return(err error) *MockStorageProvider_DeleteObjectsWithPrefix_Call

func (*MockStorageProvider_DeleteObjectsWithPrefix_Call) Run 

func (_c *MockStorageProvider_DeleteObjectsWithPrefix_Call) Run(run func(ctx context.Context, prefix string)) *MockStorageProvider_DeleteObjectsWithPrefix_Call

func (*MockStorageProvider_DeleteObjectsWithPrefix_Call) RunAndReturn 

func (_c *MockStorageProvider_DeleteObjectsWithPrefix_Call) RunAndReturn(run func(ctx context.Context, prefix string) error) *MockStorageProvider_DeleteObjectsWithPrefix_Call

type MockStorageProvider_Expecter 

type MockStorageProvider_Expecter struct {
	// contains filtered or unexported fields
}

func (*MockStorageProvider_Expecter) DeleteObjectsWithPrefix 

func (_e *MockStorageProvider_Expecter) DeleteObjectsWithPrefix(ctx interface{}, prefix interface{}) *MockStorageProvider_DeleteObjectsWithPrefix_Call

DeleteObjectsWithPrefix is a helper method to define mock.On call

  • ctx context.Context
  • prefix string

func (*MockStorageProvider_Expecter) GetDetails 

func (_e *MockStorageProvider_Expecter) GetDetails() *MockStorageProvider_GetDetails_Call

GetDetails is a helper method to define mock.On call

func (*MockStorageProvider_Expecter) OpenBlob 

func (_e *MockStorageProvider_Expecter) OpenBlob(ctx interface{}, path interface{}) *MockStorageProvider_OpenBlob_Call

OpenBlob is a helper method to define mock.On call

  • ctx context.Context
  • path string

func (*MockStorageProvider_Expecter) OpenSeekable 

func (_e *MockStorageProvider_Expecter) OpenSeekable(ctx interface{}, path interface{}) *MockStorageProvider_OpenSeekable_Call

OpenSeekable is a helper method to define mock.On call

  • ctx context.Context
  • path string

func (*MockStorageProvider_Expecter) UploadSignedURL 

func (_e *MockStorageProvider_Expecter) UploadSignedURL(ctx interface{}, path interface{}, ttl interface{}) *MockStorageProvider_UploadSignedURL_Call

UploadSignedURL is a helper method to define mock.On call

  • ctx context.Context
  • path string
  • ttl time.Duration

type MockStorageProvider_GetDetails_Call 

type MockStorageProvider_GetDetails_Call struct {
	*mock.Call
}

MockStorageProvider_GetDetails_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'GetDetails'

func (*MockStorageProvider_GetDetails_Call) Return 

func (_c *MockStorageProvider_GetDetails_Call) Return(s string) *MockStorageProvider_GetDetails_Call

func (*MockStorageProvider_GetDetails_Call) Run 

func (_c *MockStorageProvider_GetDetails_Call) Run(run func()) *MockStorageProvider_GetDetails_Call

func (*MockStorageProvider_GetDetails_Call) RunAndReturn 

func (_c *MockStorageProvider_GetDetails_Call) RunAndReturn(run func() string) *MockStorageProvider_GetDetails_Call

type MockStorageProvider_OpenBlob_Call 

type MockStorageProvider_OpenBlob_Call struct {
	*mock.Call
}

MockStorageProvider_OpenBlob_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'OpenBlob'

func (*MockStorageProvider_OpenBlob_Call) Return 

func (_c *MockStorageProvider_OpenBlob_Call) Return(blob Blob, err error) *MockStorageProvider_OpenBlob_Call

func (*MockStorageProvider_OpenBlob_Call) Run 

func (_c *MockStorageProvider_OpenBlob_Call) Run(run func(ctx context.Context, path string)) *MockStorageProvider_OpenBlob_Call

func (*MockStorageProvider_OpenBlob_Call) RunAndReturn 

func (_c *MockStorageProvider_OpenBlob_Call) RunAndReturn(run func(ctx context.Context, path string) (Blob, error)) *MockStorageProvider_OpenBlob_Call

type MockStorageProvider_OpenSeekable_Call 

type MockStorageProvider_OpenSeekable_Call struct {
	*mock.Call
}

MockStorageProvider_OpenSeekable_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'OpenSeekable'

func (*MockStorageProvider_OpenSeekable_Call) Return 

func (_c *MockStorageProvider_OpenSeekable_Call) Return(seekable Seekable, err error) *MockStorageProvider_OpenSeekable_Call

func (*MockStorageProvider_OpenSeekable_Call) Run 

func (_c *MockStorageProvider_OpenSeekable_Call) Run(run func(ctx context.Context, path string)) *MockStorageProvider_OpenSeekable_Call

func (*MockStorageProvider_OpenSeekable_Call) RunAndReturn 

func (_c *MockStorageProvider_OpenSeekable_Call) RunAndReturn(run func(ctx context.Context, path string) (Seekable, error)) *MockStorageProvider_OpenSeekable_Call

type MockStorageProvider_UploadSignedURL_Call 

type MockStorageProvider_UploadSignedURL_Call struct {
	*mock.Call
}

MockStorageProvider_UploadSignedURL_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'UploadSignedURL'

func (*MockStorageProvider_UploadSignedURL_Call) Return 

func (_c *MockStorageProvider_UploadSignedURL_Call) Return(s string, err error) *MockStorageProvider_UploadSignedURL_Call

func (*MockStorageProvider_UploadSignedURL_Call) Run 

func (_c *MockStorageProvider_UploadSignedURL_Call) Run(run func(ctx context.Context, path string, ttl time.Duration)) *MockStorageProvider_UploadSignedURL_Call

func (*MockStorageProvider_UploadSignedURL_Call) RunAndReturn 

func (_c *MockStorageProvider_UploadSignedURL_Call) RunAndReturn(run func(ctx context.Context, path string, ttl time.Duration) (string, error)) *MockStorageProvider_UploadSignedURL_Call

type MultipartUploader 

type MultipartUploader struct {
	// contains filtered or unexported fields
}

func NewMultipartUploaderWithRetryConfig 

func NewMultipartUploaderWithRetryConfig(ctx context.Context, bucketName, objectName string, retryConfig RetryConfig, metadata ObjectMetadata) (*MultipartUploader, error)

func (*MultipartUploader) Close 

func (m *MultipartUploader) Close() error

func (*MultipartUploader) Complete 

func (m *MultipartUploader) Complete(ctx context.Context) error

Complete finalizes the GCS multipart upload with all collected parts.

func (*MultipartUploader) Start 

func (m *MultipartUploader) Start(ctx context.Context) error

Start initiates the GCS multipart upload.

func (*MultipartUploader) UploadFileInParallel 

func (m *MultipartUploader) UploadFileInParallel(ctx context.Context, filePath string, maxConcurrency int, hasher hash.Hash) (int64, error)

func (*MultipartUploader) UploadPart 

func (m *MultipartUploader) UploadPart(ctx context.Context, partIndex int, data ...[]byte) error

UploadPart uploads a single part to GCS. Multiple data slices are hashed and uploaded without copying into a single contiguous buffer.

type ObjectMetadata 

type ObjectMetadata = storageopts.ObjectMetadata

func BlobCustomMetadata 

func BlobCustomMetadata(ctx context.Context, b Blob) (ObjectMetadata, error)

BlobCustomMetadata returns the blob's custom metadata, or ErrMetadataUnsupported when the backend can't read it — so callers can distinguish "no tombstone" from "couldn't check" and fail closed under enforcement.

type ObjectOrigin 

type ObjectOrigin = storageopts.ObjectOrigin

type ObjectType 

type ObjectType int
const (
	UnknownObjectType ObjectType = iota
	MemfileHeaderObjectType
	RootFSHeaderObjectType
	SnapfileObjectType
	MetadataObjectType
	BuildLayerFileObjectType
	LayerMetadataObjectType
)

type Part 

type Part struct {
	PartNumber int    `xml:"PartNumber"`
	ETag       string `xml:"ETag"`
}

type Paths 

type Paths struct {
	BuildID string `json:"build_id"`
}

func (Paths) Cache 

func (p Paths) Cache(config Config) (CachePaths, error)

func (Paths) CacheKey 

func (p Paths) CacheKey() string

Key for the cache. Unique for template-build pair.

func (Paths) DataFile 

func (p Paths) DataFile(name string, ct CompressionType) string

DataFile returns the storage path for a data file (e.g. "memfile", "rootfs.ext4"), with compression suffix appended if ct is not CompressionNone.

func (Paths) HeaderFile 

func (p Paths) HeaderFile(name string) string

HeaderFile returns the storage path for a header sidecar of a data file (e.g. "memfile" → "{buildID}/memfile.header").

func (Paths) Memfile 

func (p Paths) Memfile() string

func (Paths) MemfileCompressed 

func (p Paths) MemfileCompressed(ct CompressionType) string

func (Paths) MemfileHeader 

func (p Paths) MemfileHeader() string

func (Paths) Metadata 

func (p Paths) Metadata() string

func (Paths) Rootfs 

func (p Paths) Rootfs() string

func (Paths) RootfsCompressed 

func (p Paths) RootfsCompressed(ct CompressionType) string

func (Paths) RootfsHeader 

func (p Paths) RootfsHeader() string

func (Paths) Snapfile 

func (p Paths) Snapfile() string

func (Paths) StorageDir 

func (p Paths) StorageDir() string

type PeerTransitionedError 

type PeerTransitionedError struct {
	RetryAfter time.Duration
}

PeerTransitionedError is returned by the peer Seekable when the remote storage upload has completed; the caller should re-load the V4 header from storage.

func (*PeerTransitionedError) Error 

func (e *PeerTransitionedError) Error() string

type Provider 

type Provider string
const (
	GCPStorageProvider   Provider = "GCPBucket"
	AWSStorageProvider   Provider = "AWSBucket"
	LocalStorageProvider Provider = "Local"

	DefaultStorageProvider Provider = GCPStorageProvider

	// MemoryChunkSize must always be bigger or equal to the block size.
	MemoryChunkSize = 4 * 1024 * 1024 // 4 MB

	// MetadataKeyUncompressedSize stores the original size so that Size()
	// returns the uncompressed size for compressed objects.
	MetadataKeyUncompressedSize = "uncompressed-size"
)

func GetProviderType 

func GetProviderType() Provider

GetProviderType returns the configured storage provider type from the STORAGE_PROVIDER environment variable, defaulting to GCPBucket.

type PutOption 

type PutOption = storageopts.PutOption

func WithChecksumSHA256 

func WithChecksumSHA256() PutOption

func WithCompressConfig 

func WithCompressConfig(cfg CompressConfig) PutOption

WithCompressConfig threads a typed CompressConfig through PutOptions. It is stored as `any` in storageopts to avoid importing storage from there; backends use CompressConfigFromOpts to pull it back out.

func WithFrameSink 

func WithFrameSink(s FrameSink) PutOption

func WithMetadata 

func WithMetadata(metadata ObjectMetadata) PutOption

type PutOptions 

type PutOptions = storageopts.PutOptions

func ApplyPutOptions 

func ApplyPutOptions(opts []PutOption) PutOptions

type Range 

type Range struct {
	Offset int64
	Length int
}

func (Range) String 

func (r Range) String() string

type RangeOpener 

type RangeOpener interface {
	OpenRangeReader(ctx context.Context, offsetU int64, length int64, frameTable *FrameTable) (RangeReader, Source, error)
}

RangeOpener supports progressive reads via a streaming range reader. OpenRangeReader returns the Source that served the bytes.

type RangeReader 

type RangeReader interface {
	io.Reader
	// Close returns the reader's lifetime stats, or nil if it doesn't meter.
	Close(ctx context.Context) (*ReadStats, error)
}

func NewDecompressReader 

func NewDecompressReader(inner RangeReader, ct CompressionType, src Source, ot SeekableObjectType) (RangeReader, error)

NewDecompressReader wraps inner with a decoder for ct, attributing read.decompress to src/ot. Callers without that context (tests) pass UnknownSource / UnknownSeekableObjectType.

func NewRangeReader 

func NewRangeReader(rc io.ReadCloser) RangeReader

type ReadStats 

type ReadStats struct {
	StoredBytes    int64
	DeliveredBytes int64
	Read           time.Duration // source I/O wall, excluding open and decompression
	Decompress     time.Duration
}

ReadStats is what a RangeReader did over its lifetime; returned from Close.

type RetryConfig 

type RetryConfig struct {
	MaxAttempts       int
	InitialBackoff    time.Duration
	MaxBackoff        time.Duration
	BackoffMultiplier float64
}

RetryConfig holds the configuration for retry logic

func DefaultRetryConfig 

func DefaultRetryConfig() RetryConfig

DefaultRetryConfig returns the default retry configuration matching storage_google.go

type SandboxFiles 

type SandboxFiles struct {
	CachePaths

	SandboxID string
	// contains filtered or unexported fields
}

func (*SandboxFiles) SandboxCacheRootfsLinkPath 

func (s *SandboxFiles) SandboxCacheRootfsLinkPath(config Config) string

func (*SandboxFiles) SandboxCacheRootfsPath 

func (s *SandboxFiles) SandboxCacheRootfsPath(config Config) string

func (*SandboxFiles) SandboxCgroupName 

func (s *SandboxFiles) SandboxCgroupName() string

func (*SandboxFiles) SandboxFirecrackerSocketPath 

func (s *SandboxFiles) SandboxFirecrackerSocketPath() string

func (*SandboxFiles) SandboxMetricsFifoPath 

func (s *SandboxFiles) SandboxMetricsFifoPath() string

func (*SandboxFiles) SandboxUffdSocketPath 

func (s *SandboxFiles) SandboxUffdSocketPath() string

type Seekable 

type Seekable interface {
	RangeOpener
	SeekableWriter
	Size(ctx context.Context) (int64, error)
}

type SeekableObjectType 

type SeekableObjectType int
const (
	UnknownSeekableObjectType SeekableObjectType = iota
	MemfileObjectType
	RootFSObjectType
)

func (SeekableObjectType) String 

func (t SeekableObjectType) String() string

type SeekableWriter 

type SeekableWriter interface {
	// Store entire file. Compression is opt-in via WithCompressConfig.
	StoreFile(ctx context.Context, path string, opts ...PutOption) (*FullFrameTable, [32]byte, error)
}

type Source 

type Source int8

Source identifies the backend that served a read. 

const (
	// Order is latency-ascending and load-bearing: Slice records the slowest
	// source it touched via max() over per-fetch sources.
	UnknownSource Source = iota
	SourceMmap
	SourceFS
	SourcePeer
	SourceNFS
	SourceGCS
	SourceAWS
)

func (Source) String 

func (s Source) String() string

type StorageConfig 

type StorageConfig struct {
	GetLocalBasePath func() string
	GetBucketName    func() string
	// contains filtered or unexported fields
}

StorageConfig holds the configuration for creating a storage provider. Both GetLocalBasePath and GetBucketName are evaluated lazily so that callers who set environment variables at runtime (e.g. via os.Setenv or t.Setenv in tests) see their overrides respected.

func (StorageConfig) WithLimiter 

func (c StorageConfig) WithLimiter(limiter *limit.Limiter) StorageConfig

WithLimiter returns a copy of the config with the given limiter set.

func (StorageConfig) WithLocalUpload 

func (c StorageConfig) WithLocalUpload(uploadBaseURL string, hmacKey []byte) StorageConfig

WithLocalUpload returns a copy of the config with the given local upload parameters set. These are only used when STORAGE_PROVIDER=Local to let the filesystem storage provider generate signed URLs for file uploads.

type StorageProvider 

type StorageProvider interface {
	DeleteObjectsWithPrefix(ctx context.Context, prefix string) error
	UploadSignedURL(ctx context.Context, path string, ttl time.Duration) (string, error)
	OpenBlob(ctx context.Context, path string) (Blob, error)
	OpenSeekable(ctx context.Context, path string) (Seekable, error)
	GetDetails() string
}

func GetStorageProvider 

func GetStorageProvider(ctx context.Context, cfg StorageConfig) (StorageProvider, error)

func NewGCP 

func NewGCP(ctx context.Context, bucketName string, limiter *limit.Limiter) (StorageProvider, error)

func WrapInNFSCache 

func WrapInNFSCache(
	ctx context.Context,
	rootPath string,
	inner StorageProvider,
	flags *featureflags.Client,
) StorageProvider

Source Files

View all Source files

Directories

Path Synopsis
Package storageopts holds option types for the storage package, kept separate so generated mocks can reference them without an import cycle.
 Package storageopts holds option types for the storage package, kept separate so generated mocks can reference them without an import cycle.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.