Documentation
¶
Overview ¶
Package blob is the storage-layer port (ADR-0007): opaque byte objects behind the blob.Bucket port. It is the bytes substrate, sibling to the store (records) — deliberately named "blob", not "storage", to stay distinct from "store".
This package imports no driver library; the gocloud.dev/blob adapter (memory / file / S3, all pure-Go) lives in internal/blob/gocloud.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ( ErrCallerMissing = fault.NewKnownCause(fault.Forbidden, "blob.caller-missing", "The calling Function no longer exists, so it binds no prefix.", "Call from a Function that exists; a deleted Function's worker stops.") ErrNoBinding = fault.NewKnownCause(fault.Forbidden, "blob.no-binding", "The Function's spec.bindings.blob has no entry for the alias.", "Add the binding to spec.bindings.blob, or call an alias that the Function binds.") ErrBucketMissing = fault.NewKnownCause(fault.NotFound, "blob.bucket-missing", "The binding names a Bucket that does not exist.", "Create the Bucket, or bind one that exists.") ErrReadDenied = fault.NewKnownCause(fault.Forbidden, "blob.read-denied", "A Policy forbids the Function to read the prefix.", "Change or remove the Policy that forbids s3::read on the prefix.") ErrWriteDenied = fault.NewKnownCause(fault.Forbidden, "blob.write-denied", "The Function does not own the prefix, no RolesAssignment makes it a writer, a Policy forbids the write, or the "+ "Bucket does not exist.", "Create the Bucket if it is missing; then write from the prefix's owner, or grant the Function s3::write with a "+ "RolesAssignment, such as the Blob Data Writer role.") ErrObjectNotFound = fault.NewKnownCause(fault.NotFound, "blob.object-not-found", "The object does not exist.", "Write the object first, or handle its absence.") ErrObjectOverCap = fault.NewKnownCause(fault.PayloadTooLarge, "blob.object-over-cap", "The object is larger than the Bucket's spec.quota.maxObjectBytes.", "Write a smaller object, or raise the Bucket's spec.quota.maxObjectBytes.") ErrSignPutCapped = fault.NewKnownCause(fault.Forbidden, "blob.sign-put-capped", "The Bucket sets maxObjectBytes or a size quota, or the prefix sets a size quota, which funcd could not enforce "+ "on the upload of a presigned PUT.", "Write the object with put, or sign a GET or a DELETE.") ErrSignUnsupported = fault.NewKnownCause(fault.Unavailable, "blob.sign-unsupported", "The blob store that funcd runs with signs no URL, as no store that funcd opens does.", "Read and write the object with get and put, or serve it from a Function.") ErrKeyNotUTF8 = fault.NewKnownCause(fault.Invalid, "blob.key-not-utf8", "The key is not valid UTF-8.", "Use a key of UTF-8 text.") ErrKeyReserved = fault.NewKnownCause(fault.Invalid, "blob.key-reserved", "funcd's file store refuses the key: it ends in .attrs or .funcd-tmp or holds __0x, which the store reserves, "+ "or it has a . segment or a last segment of .., which the store would keep under another key's path.", "Use another key.") ErrKeyUnstorable = fault.NewKnownCause(fault.Invalid, "blob.key-unstorable", "funcd's file store cannot hold the key: the key or one of its segments is longer than the file system takes, "+ "or the key needs the path of another object, as a/b does while a is an object.", "Use another key, or delete the object whose path the key needs.") )
The known causes of the blob service's errors (ADR-0330 Decision 2): internal/services/blob, the views of this package, the gocloud driver and the worker node's local API raise them, and each endpoint of that API that answers one declares it.
var ErrCorrupt = errors.New("the object does not match its attributes")
ErrCorrupt reports an object whose bytes do not match the MD5 its attributes record, or whose attributes do not decode; a driver wraps it in fault.Internal and serves none of the object (ADR-0278).
Functions ¶
This section is empty.
Types ¶
type Attributes ¶
type Attributes struct {
Key string
Size int64
ModTime time.Time
// MD5 is the MD5 of the content; empty when the driver has no digest for the object (ADR-0159).
MD5 []byte
ContentType string
Metadata map[string]string // user metadata, lowercase keys
}
Attributes describes one object. List fills Key, Size, ModTime and MD5; Bucket.Attributes fills all.
type Bucket ¶
type Bucket interface {
Get(ctx context.Context, key string) ([]byte, error)
Put(ctx context.Context, key string, data []byte, opts PutOptions) error
Delete(ctx context.Context, key string) error
Exists(ctx context.Context, key string) (bool, error)
List(ctx context.Context, prefix string) ([]Attributes, error)
// ListAfter returns at most limit objects under prefix whose key sorts strictly after `after`, sorted by key,
// with the fields List fills; more is true exactly when a further key under prefix exists. limit < 1 is
// fault.Invalid (ADR-0184).
ListAfter(ctx context.Context, prefix, after string, limit int) (items []Attributes, more bool, err error)
// Attributes reads one object's attributes without its content; fault.NotFound if absent.
Attributes(ctx context.Context, key string) (Attributes, error)
SignedURL(ctx context.Context, key string, opts SignOptions) (string, error)
Close() error
}
Bucket is the storage-layer port: opaque byte objects keyed by a path string. Errors are api/fault kinds; every method is ctx-first.
func Capped ¶ added in v0.2.1
Capped returns a view of inner whose Put refuses an object larger than maxBytes with fault.PayloadTooLarge: the Bucket.spec.quota.maxObjectBytes per-object policy (ADR-0080, ADR-0148, ADR-0292). It also refuses to sign a PUT URL with fault.Forbidden, since the upload then bypasses Put and SignOptions carries no size limit to bind it. maxBytes <= 0 returns inner unchanged. It forwards the optional RangeReader capability when inner implements it.
func NoSign ¶ added in v0.10.0
NoSign returns a view of inner whose SignedURL is fault.Unavailable for every method (ADR-0208 Decision 1): a remote store's driver would sign with the store credential, which deletes. It forwards the optional RangeReader capability when inner implements it, as Capped does.
func Prefixed ¶
Prefixed returns a Bucket view of inner whose keys are all transparently prefixed with keyPrefix (ADR-0080): it scopes a single shared substrate bucket into many logical buckets (e.g. one per (namespace, Bucket) S3 domain) without a separate backend per bucket. A trailing "/" on keyPrefix is recommended for a clean boundary. It forwards the optional RangeReader capability when inner implements it.
func Quoted ¶ added in v0.33.0
func Quoted(inner Bucket, chk quota.Checker, bucket quota.Subject, capped func(prefix string) bool) Bucket
Quoted returns a view of inner whose Put asks chk before it stores (ADR-0292): the key's prefix, then the Bucket, each a subject of its own and asked only when capped reports a size for it, so a key's leading segment that no quota names is never tracked; an uncapped Bucket's place goes to the zero Subject, for the free-space floor (ADR-0296); a refusal by the Bucket releases the prefix's reservation. A delete reports itself to the same subjects; reads pass through. capped reports whether the Bucket (prefix "") or a declared prefix sets a size; a presigned PUT under one is fault.Forbidden, since its upload bypasses Put, as Capped refuses one. A nil chk is quota.None. It forwards RangeReader.
type PrefixMover ¶ added in v0.30.0
PrefixMover is an optional capability of a Bucket (ADR-0288 Decision 2): MovePrefix moves each object under src to dst plus its suffix, both prefixes ending in "/". A file store does it in one rename of the directory, both parents synced; another store copies each object server side, then deletes it, so a cut move run again moves what src still holds. A src with no object is no error.
type PutOptions ¶ added in v0.4.0
type PutOptions struct {
ContentType string
Metadata map[string]string
// IfNotExist creates the object only when its key is absent; a present key is fault.Conflict and the object
// stays unchanged (ADR-0203).
IfNotExist bool
}
PutOptions configures a Put. The zero value stores no metadata and lets the driver pick the content type.
type RangeReader ¶
type RangeReader interface {
// GetRange returns bytes [offset, offset+length) of the object at key, and no more whatever its store answers.
// A negative length means "to end". offset must be >= 0. fault.NotFound if the
// key is absent; fault.Invalid for a negative offset.
GetRange(ctx context.Context, key string, offset, length int64) ([]byte, error)
}
RangeReader is an OPTIONAL capability a blob.Bucket driver MAY implement: a byte-range read of an object (ADR-0080). The s3gateway type-asserts a Bucket to RangeReader to serve DuckDB's ranged Parquet reads efficiently and FALLS BACK to a full Get+slice when the driver does not implement it. Keeping it a separate, optional interface keeps the blob.Bucket port minimal and the contract tests additive.
type RestoreReport ¶ added in v0.10.0
RestoreReport counts a RestoreAt's keys: copied from an older version, deleted (a new delete marker) and left as they were; NoVersion lists the deleted keys that had no version at the time.
type SignMethod ¶
type SignMethod string
SignMethod is the typed HTTP method a SignedURL grants (ADR-0002: typed over magic strings).
const ( SignGet SignMethod = "GET" SignPut SignMethod = "PUT" SignDelete SignMethod = "DELETE" )
type SignOptions ¶
type SignOptions struct {
Method SignMethod
Expiry time.Duration
}
SignOptions configures a SignedURL request. A zero Method means SignGet; a zero Expiry means the driver's default (15m).
type Versioned ¶ added in v0.10.0
type Versioned interface {
// Versioning reads the bucket's versioning status and Object Lock configuration (ADR-0208 Decision 2).
Versioning(ctx context.Context) (Versioning, error)
// RestoreAt makes every key under the store's prefix hold its version at at (Decision 6): keep is
// blob.versionRetention, and 0 or an at older than keep is fault.Invalid. No version is removed.
RestoreAt(ctx context.Context, at time.Time, keep time.Duration) (RestoreReport, error)
}
Versioned is an OPTIONAL capability, found by type assertion as RangeReader is: a store whose bucket keeps every version of an object (ADR-0208). A Bucket without it cannot be protected or restored to a time; the caller answers fault.Invalid.
type Versioning ¶ added in v0.10.0
type Versioning struct{ Enabled, ObjectLock bool }
Versioning is a bucket's protection: versioning Enabled and Object Lock enabled.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package blobcontract is the shared conformance suite for the blob.Bucket port (ADR-0007).
|
Package blobcontract is the shared conformance suite for the blob.Bucket port (ADR-0007). |
|
Package gocloud is the gocloud.dev/blob driver for the blob.Bucket port (ADR-0007): memory / file / S3 by URL, all pure-Go (no cgo).
|
Package gocloud is the gocloud.dev/blob driver for the blob.Bucket port (ADR-0007): memory / file / S3 by URL, all pure-Go (no cgo). |
|
Package s3gateway is the funcd S3-protocol frontend over the blob.Bucket substrate (ADR-0080), with in-platform identity via funcd-managed per-principal SigV4 keypairs resolved by an in-process versitygw auth.IAMService (ADR-0085).
|
Package s3gateway is the funcd S3-protocol frontend over the blob.Bucket substrate (ADR-0080), with in-platform identity via funcd-managed per-principal SigV4 keypairs resolved by an in-process versitygw auth.IAMService (ADR-0085). |