Documentation
¶
Overview ¶
Package blob stores opaque byte objects under opaque keys.
It is the seam for a value too large to hold in memory or to move through a transaction. The package internal/storage owns the other kind: a small record that changes, with transactions, batches, and compare-and-set. A multi-megabyte value would move through every one of those operations, so file bytes get their own contract.
The contract is deliberately narrow. A store knows nothing about accounts, files, purposes, or expiry. It reads and writes bytes at a key. The owner of the key holds every meaning the bytes carry.
Index ¶
- Constants
- Variables
- func ValidateKey(key string) error
- type Filesystem
- func (f *Filesystem) Backend() string
- func (f *Filesystem) Delete(ctx context.Context, key string) error
- func (f *Filesystem) Get(ctx context.Context, key string) (io.ReadCloser, error)
- func (f *Filesystem) Put(ctx context.Context, key string, r io.Reader) (Info, error)
- func (f *Filesystem) Stat(ctx context.Context, key string) (Info, error)
- type Info
- type ObjectStore
- func (o *ObjectStore) Backend() string
- func (o *ObjectStore) Delete(ctx context.Context, key string) error
- func (o *ObjectStore) Get(ctx context.Context, key string) (io.ReadCloser, error)
- func (o *ObjectStore) Put(ctx context.Context, key string, r io.Reader) (Info, error)
- func (o *ObjectStore) Stat(ctx context.Context, key string) (Info, error)
- type ObjectStoreOptions
- type Store
Constants ¶
const MaxKeyLength = 256
MaxKeyLength bounds a key. The bound keeps a key inside the length every backing medium accepts, including a filesystem path component and an object store key.
Variables ¶
var ( // ErrInvalidKey reports a key the contract refuses. A store returns it // before it touches its backing medium, so a rejected key never leaves a // partial object behind. ErrInvalidKey = errors.New("blob: invalid key") // ErrNotFound reports that no object exists at the key. ErrNotFound = errors.New("blob: object not found") )
Errors the contract defines. A backend returns these rather than an error of its own, so a caller can branch on the reason without naming a backend.
Functions ¶
func ValidateKey ¶
ValidateKey reports whether the key is one this package stores under.
A key is opaque: it carries no structure a store reads. The rules exist so that one key means the same object on every backend. A path separator is the case that matters most, because a filesystem would read it as a directory step and an object store would read it as a prefix, and the same key would then name two different places.
Types ¶
type Filesystem ¶
type Filesystem struct {
// contains filtered or unexported fields
}
Filesystem stores objects as files under a configured root directory.
It suits one node. A deployment that runs more than one Starport process against the same files needs the object store backend instead, which FIL2 implements behind this same contract.
func NewFilesystem ¶
func NewFilesystem(root string) (*Filesystem, error)
NewFilesystem opens a filesystem store rooted at the directory. It creates the root and its two subdirectories when they are absent, and it refuses a root it cannot write.
func (*Filesystem) Delete ¶
func (f *Filesystem) Delete(ctx context.Context, key string) error
Delete implements Store.
func (*Filesystem) Get ¶
func (f *Filesystem) Get(ctx context.Context, key string) (io.ReadCloser, error)
Get implements Store.
type Info ¶
type Info struct {
// Key is the key the object is stored under.
Key string
// Size is the stored length in bytes.
Size int64
}
Info describes one stored object.
type ObjectStore ¶
type ObjectStore struct {
// contains filtered or unexported fields
}
ObjectStore stores objects in an S3-compatible bucket. It serves every node of a deployment, which the filesystem backend cannot.
func NewObjectStore ¶
func NewObjectStore(ctx context.Context, options ObjectStoreOptions) (*ObjectStore, error)
NewObjectStore opens a store against the bucket the options name.
It builds the client and returns. It does not reach the bucket, because a network call at construction would make startup depend on a remote service that the first upload will reach anyway.
func (*ObjectStore) Delete ¶
func (o *ObjectStore) Delete(ctx context.Context, key string) error
Delete implements Store.
func (*ObjectStore) Get ¶
func (o *ObjectStore) Get(ctx context.Context, key string) (io.ReadCloser, error)
Get implements Store.
func (*ObjectStore) Put ¶
Put implements Store.
The uploader sends the whole object in one request, or in parts when the stream is large. Either way the object becomes reachable only after the last part lands, so a failed put leaves no readable object at a key that held none, and leaves the prior object intact at a key that did.
type ObjectStoreOptions ¶
type ObjectStoreOptions struct {
Bucket string
Region string
Endpoint string
Prefix string
// AccessKeyID and SecretAccessKey state static credentials. Both empty
// selects the ambient AWS credential chain.
AccessKeyID string
SecretAccessKey string
}
ObjectStoreOptions addresses one S3-compatible bucket.
One client reaches AWS S3, Cloudflare R2, MinIO, and Backblaze B2, because each of them serves the same API. Endpoint selects the implementation, and an absent Endpoint selects AWS itself.
type Store ¶
type Store interface {
// Put stores the bytes the reader yields at the key. It reads until the
// reader reports io.EOF.
//
// A put that fails leaves no readable object at a key that held none
// before, and leaves the prior object intact at a key that did. A backend
// therefore stages the bytes and makes them reachable in one final step.
//
// Put returns ErrInvalidKey before it writes anything when the key breaks
// the key rules.
Put(ctx context.Context, key string, r io.Reader) (Info, error)
// Get opens the stored object for reading. The caller closes the reader.
// Get returns ErrNotFound when no object exists at the key.
Get(ctx context.Context, key string) (io.ReadCloser, error)
// Stat reports the object without reading its bytes. Stat returns
// ErrNotFound when no object exists at the key.
Stat(ctx context.Context, key string) (Info, error)
// Delete removes the object. Delete of an absent key returns nil, so a
// repeated delete is safe.
Delete(ctx context.Context, key string) error
// Backend names the implementation, such as "filesystem". An operator
// reads it once at startup to confirm where the bytes land.
Backend() string
}
Store reads and writes opaque bytes at an opaque key.
Every method takes a context and honors its cancellation. A backend that reaches a network respects the deadline the caller sets.
Source Files