Documentation
¶
Overview ¶
The bucket package is the provider-agnostic interface to object storage: S3, Cloud Storage, Azure Blob Storage.
A handle is taken by naming the blueprint resource:
uploads := resources.Bucket(app, "uploadsBucket")
Implementations are per provider and live in their own modules, such as resources/aws.
Index ¶
- Variables
- func AllVersions(ctx context.Context, store Store, prefix string, opts ...ListOption) iter.Seq2[ObjectVersion, error]
- func Objects(ctx context.Context, store Store, prefix string, opts ...ListOption) iter.Seq2[ObjectInfo, error]
- type CopyOption
- type CopyOptions
- type CopyResult
- type Cursor
- type DeleteFailure
- type DeleteManyResult
- type DeleteOption
- type DeleteOptions
- type DeleteUnsent
- type Destination
- type ExistsOption
- type ExistsOptions
- type GetOption
- type GetOptions
- type InfoOption
- type InfoOptions
- type ListOption
- type ListOptions
- type Object
- type ObjectInfo
- type ObjectRef
- type ObjectVersion
- type PutOption
- type PutOptions
- type PutResult
- type Range
- type SignAction
- type SignOption
- type SignOptions
- type SignedURL
- type Store
Constants ¶
This section is empty.
Variables ¶
var ErrNotFound = errors.New("celerity: not found")
ErrNotFound reports that the bucket holds no object under the key asked for.
if errors.Is(err, bucket.ErrNotFound) { ... }
Not used by a delete, which is idempotent: a key that holds nothing is not a failure.
Functions ¶
func AllVersions ¶
func AllVersions( ctx context.Context, store Store, prefix string, opts ...ListOption, ) iter.Seq2[ObjectVersion, error]
AllVersions returns every version of every object under a prefix, fetching a page at a time.
for version, err := range bucket.AllVersions(ctx, uploads, "invoices/") {
if err != nil {
return err
}
if version.DeleteMarker {
continue
}
...
}
func Objects ¶
func Objects( ctx context.Context, store Store, prefix string, opts ...ListOption, ) iter.Seq2[ObjectInfo, error]
Objects returns every object under a prefix, fetching a page at a time.
for object, err := range bucket.Objects(ctx, uploads, "invoices/") {
if err != nil {
return err
}
...
}
Types ¶
type CopyOptions ¶
CopyOptions is the resolved configuration for a copy.
The copy carries the source's content type and metadata where neither is given here. Giving either replaces both, the two travel together in the stores, so one cannot be overridden while the other is inherited.
func (CopyOptions) Replaces ¶
func (o CopyOptions) Replaces() bool
Replaces reports whether these options replace the source's content type and metadata rather than carrying them over.
type CopyResult ¶
type CopyResult struct {
ETag string
// VersionID names the version the copy created in the destination.
VersionID string
}
CopyResult is what the store recorded about the object a copy created.
type Cursor ¶
type Cursor string
Cursor is a position in a listing, to resume it from.
A cursor is opaque what is in one belongs to the store that produced it, and only that store can read it back. Empty means a listing reached its end.
type DeleteFailure ¶
type DeleteFailure struct {
Ref ObjectRef
// Code is the store's own name for the refusal.
Code string
// Message describes the refusal.
Message string
}
DeleteFailure is an object the store refused to remove.
type DeleteManyResult ¶
type DeleteManyResult struct {
// Deleted is what the store removed.
Deleted []ObjectRef
// Failed is what the store refused.
Failed []DeleteFailure
// Unsent is what the store was never able to answer for.
Unsent []DeleteUnsent
}
DeleteManyResult reports what became of each object of a multiple delete.
Every reference handed over appears in exactly one of the three lists.
type DeleteOptions ¶
type DeleteOptions struct {
// VersionID removes one version for good rather than hiding the current one
// behind a marker. Empty deletes the current version.
VersionID string
}
DeleteOptions is the resolved configuration for a delete.
type DeleteUnsent ¶
type DeleteUnsent struct {
Ref ObjectRef
// Err is what stopped the delete, and is the same failure for every unsent
// reference.
Err error
}
DeleteUnsent is an object the store was never able to answer for, because the request carrying it did not complete or because an earlier one did not and the rest were not attempted.
type Destination ¶
type Destination struct {
// Key is what the copy is stored under.
Key string
// Store is another bucket to copy into, taken the same way this one was.
// Nil copies within the same bucket.
//
// Both buckets have to be the same provider's, since the store does the
// copying and one service cannot reach into another's.
Store Store
}
Destination is where a copy puts the object.
type ExistsOption ¶
type ExistsOption func(*ExistsOptions)
ExistsOption configures a presence check.
type ExistsOptions ¶
type ExistsOptions struct {
// VersionID asks whether one version is there rather than whether the key
// holds a current version.
VersionID string
}
ExistsOptions is the resolved configuration for a presence check.
type GetOptions ¶
type GetOptions struct {
// Range reads part of an object rather than the whole of it. The zero Range
// reads the whole of it.
Range Range
// VersionID reads one version rather than the current one. Empty reads the
// current one.
VersionID string
}
GetOptions is the resolved configuration for a read.
type InfoOptions ¶
type InfoOptions struct {
// VersionID reads one version's metadata rather than the current one's.
VersionID string
}
InfoOptions is the resolved configuration for a metadata read.
type ListOptions ¶
type ListOptions struct {
// Limit is the most objects one page holds. Zero leaves it to the store.
Limit int
// Cursor resumes a listing where an earlier one stopped. See [Cursor].
Cursor string
}
ListOptions is the resolved configuration for a listing.
type Object ¶
type Object struct {
// Body is the content, which has to be closed.
//
// Where a [Range] was asked for this is that part of the object rather than
// the whole of it.
Body io.ReadCloser
// Info is what the store knows about the object. Size is the whole object's
// even where a range was read, since that is what the store reports.
Info ObjectInfo
// ContentLength is how many bytes Body yields, which for a ranged read is
// the part rather than the whole. Equal to Info.Size for a read of the
// whole object.
ContentLength int64
}
Object is a stored object, including its content, and what the store knows about it.
type ObjectInfo ¶
type ObjectInfo struct {
Key string
Size int64
LastModified time.Time
ETag string
// ContentType is the type the object was stored with, where the store
// recorded one.
//
// This is empty in a listing, a store answers a listing with the keys and their
// sizes rather than with each object's headers, so reading this needs
// [Store.Info] or [Store.Get].
ContentType string
// Metadata is what was stored alongside the object by [PutOptions].
//
// Empty in a listing, for the same reason as ContentType.
Metadata map[string]string
// VersionID names the version this describes, where the bucket has
// versioning on. Empty where it does not, and empty in a listing. A listing
// reports the current version of each key, so there is no version to name.
VersionID string
}
ObjectInfo describes one stored object.
type ObjectRef ¶
type ObjectRef struct {
// Key is the object's key.
Key string
// VersionID names one version. Empty means the current one.
VersionID string
}
ObjectRef names one object, and optionally one version of it.
type ObjectVersion ¶
type ObjectVersion struct {
Key string
VersionID string
Size int64
LastModified time.Time
ETag string
// Current reports whether this is the version a read without a version
// would return.
Current bool
// DeleteMarker reports a version that records a deletion rather than
// content. There is nothing to read, this is what makes the versions under it
// stop being found, and removing it is what brings them back.
DeleteMarker bool
}
ObjectVersion is one version of a stored object.
type PutOptions ¶
PutOptions is the resolved configuration for a write.
type PutResult ¶
type PutResult struct {
// ETag identifies the content, and is what a conditional request elsewhere
// compares against. Its format is the store's own: compare one with another
// from the same store and never across two.
ETag string
// VersionID names the version this write created, where the bucket has
// versioning on. Empty where it does not.
VersionID string
}
PutResult is what the store recorded about an object it stored.
type Range ¶
type Range struct {
// Start is the first byte to read, counted from zero.
Start int64
// Length is how many bytes to read. Zero reads to the end of the object.
Length int64
}
Range is the part of an object to read.
Measured as a start and a length rather than as two positions, so the zero value means the whole object.
type SignAction ¶
type SignAction string
SignAction is what a signed URL grants.
const ( // SignRead grants a download. SignRead SignAction = "read" // SignWrite grants an upload, in a single request. SignWrite SignAction = "write" )
type SignOptions ¶
type SignOptions struct {
// ContentType is the type an upload has to declare.
//
// Signed into the URL where the store supports it, so an upload declaring
// anything else is refused rather than stored under a type nobody expected.
// Ignored by [SignRead], which has nothing to constrain.
ContentType string
}
SignOptions is the resolved configuration for a signed URL.
type SignedURL ¶
type SignedURL struct {
// URL is the signed URL.
URL string
// ExpiresAt is when it stops working.
//
// Computed from the expiry asked for. A URL signed with credentials that
// expire sooner stops working sooner, which on a serverless platform is the
// usual case.
ExpiresAt time.Time
}
SignedURL is a URL granting temporary access to one object, and the moment it stops working.
type Store ¶
type Store interface {
// Get returns an object's content and what the store knows about it.
// The body needs to be closed on read.
Get(ctx context.Context, key string, opts ...GetOption) (*Object, error)
// Put stores an object, and reports what the store recorded about it.
Put(ctx context.Context, key string, body io.Reader, opts ...PutOption) (PutResult, error)
// Delete removes an object, and is idempotent. A key that holds nothing
// does not produce an error.
//
// Deletes the current version where the bucket has versioning on, which
// hides the versions rather than removing them. Naming a version in
// [DeleteOptions] removes that one for good.
Delete(ctx context.Context, key string, opts ...DeleteOption) error
// DeleteMany removes several objects in as few requests as the store
// allows. A delete of many is taken partially, so every reference comes
// back in exactly one of the three lists of [DeleteManyResult].
DeleteMany(ctx context.Context, refs []ObjectRef) (DeleteManyResult, error)
// Info returns what the store knows about an object without fetching it.
//
// Reports [ErrNotFound] where the key doesn't hold anything.
Info(ctx context.Context, key string, opts ...InfoOption) (ObjectInfo, error)
// Exists reports whether the store holds an object under the key. A key
// that holds nothing produces false rather than [ErrNotFound].
//
// Without a version this asks about the current one, which on a versioned
// bucket is false where the newest version is a deletion marker, even
// though every version under it is still there. Naming a version asks
// about that version, which can be there while the current one is not.
Exists(ctx context.Context, key string, opts ...ExistsOption) (bool, error)
// List returns one page of objects under a prefix, and the cursor to
// resume from. Use [Objects] to read a listing to the end.
//
// Lists the current version of each key. Use [Store.Versions] to see what
// else a versioned bucket holds.
List(ctx context.Context, prefix string, opts ...ListOption) ([]ObjectInfo, Cursor, error)
// Versions returns one page of the versions of the objects under a prefix,
// newest first per key, and the cursor to resume from. Use [AllVersions] to
// read it to the end.
//
// Every version of every matching key, including the deletion markers a
// delete left behind, so a restore reads the newest version that is not a
// marker. A bucket without versioning answers with one version per key.
Versions(ctx context.Context, prefix string, opts ...ListOption) ([]ObjectVersion, Cursor, error)
// Copy an object, within this bucket or into another, and report
// what the store recorded about the copy.
//
// The store does the copying, so the content does not pass through the
// handler and large objects can be copied without loading the contents
// at runtime for an application.
Copy(
ctx context.Context, sourceKey string, dest Destination, opts ...CopyOption,
) (CopyResult, error)
// SignedURL returns a URL granting access to one object until it expires,
// so a browser can read or write it without the bytes passing through a
// handler and without credentials of its own.
SignedURL(
ctx context.Context, key string, action SignAction, expiry time.Duration, opts ...SignOption,
) (SignedURL, error)
}
Store is a bucket/container in an object storage service (e.g. S3, Cloud Storage, Azure Blob Storage).