Documentation
¶
Overview ¶
Package remoteblob implements the casstore blobstore.Storage interface for an mlfs node's direct-S3 data plane (ADR-001 §2.2, §5).
A node never holds tenant S3 credentials. Instead it asks blobgw for a short-lived presigned URL (via remotepresign.Client over NATS) and then transfers bytes directly against S3 over plain HTTP (via s3http.Client). BlobStore wires those two primitives behind the kopia blob.Storage contract so the existing casstore chunk path can write/read packs without change.
Scope: what this adapter does NOT do ¶
The node data plane only ever calls PutBlob (whole pack) and GetBlob (whole pack today; ranged reads are wired through for #18). Listing, metadata and deletion are garbage-collection concerns, and GC runs on blobgw — it owns the per-tenant credentials and the authoritative index. Therefore GetMetadata, ListBlobs and DeleteBlob return ErrNotSupported rather than guessing at an implementation the node has no business performing.
Blob-ID → S3-object-key mapping ¶
casstore blob IDs use the convention "pack-<domain>-<hash>" / "chunk-<domain>-<hash>" (see casstore/blobstore.ID). Those strings are already valid, stable, printable-ASCII S3 object keys with no leading slash, so the default keyFn maps the ID verbatim: key == string(id). Both the node and blobgw must agree on the object key for a given blob; mapping the ID verbatim keeps that agreement trivial and avoids re-deriving structure on two sides. Callers that need a different layout (e.g. a "blobs/" prefix) supply WithKeyFunc.
Idempotent PutBlob ¶
Blob IDs are content hashes, so an unconditional presigned PUT is byte- idempotent: re-putting the same ID re-uploads identical bytes. We therefore IGNORE PutOptions.DoNotRecreate and never return ErrBlobAlreadyExists — the CAS contract is satisfied by the content-addressing itself.
Index ¶
- Variables
- type BlobStore
- func (b *BlobStore) ConnectionInfo() kblob.ConnectionInfo
- func (b *BlobStore) DeleteBlob(_ context.Context, _ blobstore.ID) error
- func (b *BlobStore) DisplayName() string
- func (b *BlobStore) GetBlob(ctx context.Context, id blobstore.ID, offset, length int64, ...) error
- func (b *BlobStore) GetMetadata(_ context.Context, _ blobstore.ID) (blobstore.Metadata, error)
- func (b *BlobStore) ListBlobs(_ context.Context, _ blobstore.ID, _ func(blobstore.Metadata) error) error
- func (b *BlobStore) PutBlob(ctx context.Context, id blobstore.ID, data blobstore.Bytes, ...) error
- type KeyFunc
- type Option
Constants ¶
This section is empty.
Variables ¶
var ErrNotSupported = errors.New("remoteblob: not supported on node: GC and listing are blobgw-owned")
ErrNotSupported is returned by GetMetadata, ListBlobs and DeleteBlob. Those operations are owned by blobgw (it holds the credentials and the index and runs GC); the node's direct-S3 data plane intentionally does not implement them. errors.Is friendly.
Functions ¶
This section is empty.
Types ¶
type BlobStore ¶
type BlobStore struct {
kblob.DefaultProviderImplementation
// contains filtered or unexported fields
}
BlobStore implements casstore blobstore.Storage over presigned-URL direct-S3 transfers. It embeds kblob.DefaultProviderImplementation to inherit the no-op Close/FlushCaches/GetCapacity/ExtendBlobRetention/IsReadOnly methods. It is safe for concurrent use (its dependencies are).
func NewBlobStore ¶
func NewBlobStore(presign *remotepresign.Client, http *s3http.Client, tenant string, opts ...Option) *BlobStore
NewBlobStore constructs a BlobStore. presign and http must not be nil; tenant is the domain whose bucket the node transfers against (it is forwarded to remotepresign for subject-tenant validation).
func (*BlobStore) ConnectionInfo ¶
func (b *BlobStore) ConnectionInfo() kblob.ConnectionInfo
ConnectionInfo returns a minimal description of this storage. The node data plane has no serializable credential config to expose (credentials live on blobgw), so only the Type is set.
func (*BlobStore) DeleteBlob ¶
DeleteBlob is not supported on the node. See ErrNotSupported and the package doc: metadata/listing/deletion are blobgw-owned GC concerns.
func (*BlobStore) DisplayName ¶
DisplayName returns a human-readable identifier for this storage.
func (*BlobStore) GetBlob ¶
func (b *BlobStore) GetBlob(ctx context.Context, id blobstore.ID, offset, length int64, output blobstore.OutputBuffer) error
GetBlob fetches full or partial blob contents into output.
Per the kopia Reader.GetBlob contract (repo/blob/storage.go): length<0 fetches the whole blob (the casstore case), length>0 fetches the range [offset, offset+length). A negative offset is invalid and returns blobstore.ErrInvalidRange. length==0 is also invalid (a zero-byte ranged read is undefined by the contract and would produce a degenerate HTTP Range header); it returns ErrInvalidRange without issuing any S3 request. The whole-blob path uses s3http.Get; the ranged path uses s3http.GetRange (wired through for #18 readahead).
func (*BlobStore) GetMetadata ¶
GetMetadata is not supported on the node. See ErrNotSupported and the package doc: metadata/listing/deletion are blobgw-owned GC concerns.
func (*BlobStore) ListBlobs ¶
func (b *BlobStore) ListBlobs(_ context.Context, _ blobstore.ID, _ func(blobstore.Metadata) error) error
ListBlobs is not supported on the node. See ErrNotSupported and the package doc: metadata/listing/deletion are blobgw-owned GC concerns.
func (*BlobStore) PutBlob ¶
func (b *BlobStore) PutBlob(ctx context.Context, id blobstore.ID, data blobstore.Bytes, _ blobstore.PutOptions) error
PutBlob uploads data to the blob's S3 object via a presigned PUT.
PutOptions.DoNotRecreate is ignored: blob IDs are content hashes so the PUT is byte-idempotent (see package doc). All other PutOptions are unused by the direct-S3 data plane.
type Option ¶
type Option func(*options)
Option is a functional option for NewBlobStore.
func WithKeyFunc ¶
WithKeyFunc overrides the blob-ID → S3-object-key mapping. The default maps the ID verbatim (key == string(id)).
func WithPresignTTL ¶
WithPresignTTL overrides the requested presigned-URL validity. The default is 15 minutes. blobgw may cap the effective TTL lower.
Source Files