bsw

bsw

Block storage wrapper: one interface to issue presigned upload and download URLs for different object storages.

go get github.com/axkit/bsw

Requires Go 1.26 or newer.

Usage

svc := s3.New(&s3.Config{Region: aws.String("eu-central-1") /* , Credentials... */})
if err := svc.Init(ctx); err != nil {
	return err
}

o := bsw.NewObject(svc, "bucket", "dir/report.pdf").SetMetadata("owner", "42")

uploadURL, err := o.UploadURL(15 * time.Minute)
downloadURL, err := o.DownloadURL(15 * time.Minute)

Multipart upload (S3 only):

o := bsw.NewObject(svc, "bucket", "video.mp4", bsw.WithMultiParts(3))
urls, uploadID, err := o.MultipartUploadURLs(time.Hour)

// the client uploads every part with PUT and collects the ETag response headers

err = o.CompleteMultipartUpload(uploadID, []bsw.CompletedPart{
	&bsw.Part{ETag: etag1, PartNumber: 1},
	&bsw.Part{ETag: etag2, PartNumber: 2},
	&bsw.Part{ETag: etag3, PartNumber: 3},
})

Storages that do not support multipart upload return bsw.ErrNotSupported, which also matches errors.ErrUnsupported.

Storages

s3

Config.Endpoint and Config.ForcePathStyle allow using S3-compatible storages. The URL timeout must be positive and not longer than 7 days. If generating part URLs fails after the multipart upload was created, the upload is aborted. CompleteMultipartUpload accepts parts in any order and rejects nil parts, empty ETags, part numbers outside 1..10000 and duplicates.

azure

All objects are stored in one container, the bucket is the beginning of the blob name: <container>/<bucket>/<key>; it may have several elements, e.g. org1/docs. Config.ServiceURL overrides the service URL, e.g. for Azurite.

fs

fs.Service issues signed tokens instead of URLs. PreSignPutObjectURL returns a token for POST /api/v1/bos/upload?dest=<token> (multipart form, file in the file field; dest may also be a form field), PreSignGetObjectURL a token for GET /api/v1/bos/download?src=<token>. The endpoints are provided by fs.FileSystemStorageServer.Endpoints().

A token is base64url(payload).base64url(HMAC-SHA256(payload)), the payload holds the operation, bucket, key and expiration time. A token can not be changed without the key, and an upload token can not be used for download. Config.URLEncryptionKey must be at least 32 bytes long; use a random value.

Files are stored as <basePath>/<bucket>/<key>; the bucket may be a nested directory like org1/docs. Buckets and keys with .., . or empty path elements, absolute paths and paths with backslashes are rejected; on Windows also reserved names (NUL, ...), : and names ending with a dot or space. All file operations go through os.Root, so a symlink inside the base path can not lead outside of it.

On case-insensitive file systems (Windows, macOS by default) names that differ only in case refer to the same file: tokens for TenantA/report and tenanta/report give access to the same object. Use case-normalized bucket names and keys, or a case-sensitive file system, if this matters. An upload replaces the existing file atomically. A download is streamed from the file, it is not read into memory. The upload size is limited by the fasthttp server setting MaxRequestBodySize.

Errors

Errors are axkit/errors templates; check them with errors.Is(err, bsw.ErrInvalidArgument). Codes:

Upgrading from v0

• Go 1.26 is required.
• fs: the endpoints require vatel v1.26 or newer (v0.14 is not supported); see its MIGRATION.md for the application side.
• github.com/axkit/errors v1 is required; error values are templates now.
• fs: tokens are HMAC-signed instead of AES-CFB encrypted, tokens issued by older versions are rejected. URLEncryptionKey must be at least 32 bytes. DecodeSignedURL takes the expected operation (fs.OpPut or fs.OpGet). Config.RetryCount is removed. WriteObject takes an io.Reader.
• s3.AwsCompletedPart, azure.AzureCompletedPart and fs.AwsCompletedPart are aliases of bsw.Part.
• azure and fs return bsw.ErrNotSupported for multipart upload instead of empty results.

License

MIT