README
¶
S3 object storage
Use arkoren.dev/driver/aws/awsstorage to store portable objects in an existing
S3 bucket. Install with go get arkoren.dev/driver/aws and import awsstorage
from arkoren.dev/driver/aws/awsstorage. Use your startup context and bucket:
driver, err := awsstorage.New(ctx, "uploads", func(store *awsstorage.Builder) {
store.Prefix("tenant-a")
})
Pass the driver to storage.New from arkoren.dev/core/service/storage.
Construction loads AWS configuration through its default credential chain; it
does not provision a bucket or test permissions. Callbacks run once in order on
one fresh draft. Owned settings are detached; native credential providers stay
shared. NewFrom borrows its client and applies only Prefix, leaving region,
endpoint, credentials, middleware, and transport cleanup caller-owned.
Signing an upload
Bind both MIME type and exact length when granting an upload URL:
link, err := driver.SignURL(ctx, "avatars/ada.png", storage.SignOptions{
Method: http.MethodPut,
TTL: 10 * time.Minute,
ContentType: "image/png",
ContentLength: 2048,
})
Give the uploader link and both bound header values. Zero length leaves size
unbound; S3 PUT cannot express a maximum-size range. If borrowed middleware drops
or changes the requested signed type, SignURL fails with storage.ErrUnsupported
instead of returning a weaker URL. The driver does not modify that middleware.
Temporary credentials can expire before the requested TTL; the maximum TTL is
seven days. A signed GET does not check object existence during signing.
Reading and listing
Callers must close bodies returned by Open. Nonseekable Put readers are buffered in memory; use the native S3 upload manager for very large streams. An already canceled call consumes nothing, but cancellation cannot force an arbitrary blocking reader or native dependency to return.
Listings omit nonportable native keys. To determine whether another logical object exists, listing can read ahead through physical S3 pages; deployed native cursor strings remain unchanged. Use a cursor only with the same driver and logical prefix. Concurrent bucket changes can affect subsequent pages.
Config formatting records credential-provider types, never provider contents. Endpoint and Prefix are not redacted; keep secrets out of diagnostic strings.
See the storage driver guide
and the awsstorage API reference.
Documentation
¶
Overview ¶
Package awsstorage provides an Amazon S3 storage driver. Its Open and Stat methods wrap storage.ErrNotFound when a key is absent. Constructors accept ordered Builder callbacks invoked directly once on one fresh draft. Nil entries panic; expanded nil/empty slices omit edits. Completed settings are copied; native clients and providers are not cloned. NewFrom applies only Prefix and neither copies nor closes its native client. PUT signing accepts a content type without a length. If native middleware or an injected presigner omits or changes that requested signed type, SignURL returns storage.ErrUnsupported rather than an unbound upload URL. Positive lengths bind an exact size; zero leaves size unbound. See Driver.SignURL. List filters nonportable native keys and may read ahead to validate logical continuation without changing native cursor strings. Opened bodies belong to the caller. Already-canceled calls perform no I/O; arbitrary readers/native dependencies must cooperate with cancellation once an operation has started.
Example:
driver, err := awsstorage.New(ctx, "uploads", func(store *awsstorage.Builder) {
store.Prefix("tenant-a")
})
if err != nil {
return err
}
_ = driver
Index ¶
- type Builder
- type BuilderFunc
- type Config
- type Driver
- func (driver *Driver) Client() *s3.Client
- func (driver *Driver) Delete(ctx context.Context, key string) error
- func (driver *Driver) List(ctx context.Context, options storage.ListOptions) (storage.Page, error)
- func (driver *Driver) Open(ctx context.Context, key string) (storage.Object, error)
- func (driver *Driver) Put(ctx context.Context, key string, contents io.Reader) (storage.ObjectInfo, error)
- func (driver *Driver) SignURL(ctx context.Context, key string, options storage.SignOptions) (*url.URL, error)
- func (driver *Driver) Stat(ctx context.Context, key string) (storage.ObjectInfo, error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Builder ¶
type Builder struct {
// contains filtered or unexported fields
}
Builder configures S3 construction. NewFrom applies Prefix only: its borrowed client already owns connection settings. Publication copies settings, not native credential providers or clients.
Example:
configure := func(store *Builder) {
store.Prefix("tenant-a")
}
driver, err := New(ctx, "uploads", configure)
if err != nil {
return err
}
_ = driver
func (*Builder) Credentials ¶
func (store *Builder) Credentials(credentials aws.CredentialsProvider) *Builder
Credentials replaces the shared provider; nil selects AWS's default chain. NewFrom ignores this setting. It returns the same construction draft.
Example:
configure := func(store *Builder) {
store.Credentials(nil)
}
driver, err := New(ctx, "uploads", configure)
if err != nil {
return err
}
_ = driver
func (*Builder) Endpoint ¶
Endpoint replaces the service endpoint and enables path-style addressing in New. Empty uses AWS endpoints; NewFrom ignores it. It returns the same draft.
Example:
configure := func(store *Builder) {
store.Endpoint("http://localhost:4566")
}
driver, err := New(ctx, "uploads", configure)
if err != nil {
return err
}
_ = driver
func (*Builder) Prefix ¶
Prefix replaces the logical object-key scope. Empty selects the bucket root; constructors validate it and normalize a trailing slash. It returns the draft.
Example:
configure := func(store *Builder) {
store.Prefix("tenant-a")
}
driver, err := New(ctx, "uploads", configure)
if err != nil {
return err
}
_ = driver
func (*Builder) Region ¶
Region replaces the AWS region. Empty uses the default chain; NewFrom ignores this setting. It returns the same draft without changing a native client.
Example:
configure := func(store *Builder) {
store.Region("eu-west-1")
}
driver, err := New(ctx, "uploads", configure)
if err != nil {
return err
}
_ = driver
type BuilderFunc ¶
type BuilderFunc = func(*Builder)
BuilderFunc edits one S3 construction draft in argument order. Nil entries panic. NewFrom applies Prefix only; connection settings belong to its client.
Example:
var configure awsstorage.BuilderFunc = func(store *awsstorage.Builder) {
store.Prefix("tenant-a")
}
driver, err := awsstorage.New(ctx, "uploads", configure)
if err != nil {
return err
}
_ = driver
type Config ¶
type Config struct {
// Region selects the AWS region for requests. The default is the region
// resolved by the AWS default configuration chain.
// Example: record := awsstorage.Config{Region: "eu-west-1"}
Region string
// Endpoint overrides the AWS service endpoint and enables path-style
// addressing. The default is the regular AWS endpoint for Region.
// Example: record.Endpoint = "http://localhost:9000"
Endpoint string
// Prefix scopes logical object keys within the bucket. The default is
// empty, which stores keys at the bucket root.
// Example: record.Prefix = "tenant-a"
Prefix string
// Credentials provides request credentials when the default chain is
// unsuitable. The default is nil, which uses the AWS default credential
// chain.
// Example: record.Credentials = provider // Shared, not recursively copied.
Credentials aws.CredentialsProvider
}
Config is a redacted reporting record for optional S3 settings. It is not a constructor input: use Builder callbacks and pass the required bucket directly to New or NewFrom. Formatting redacts credentials, but not endpoint or prefix; do not store secrets in those strings.
Example:
record := awsstorage.Config{
Region: "eu-west-1",
Prefix: "uploads",
}
slog.Info("storage settings", "config", record)
func (Config) GoString ¶
GoString formats config like Config.String for %#v. Native provider objects remain shared but only their type enters the result.
Example:
diagnostic := fmt.Sprintf("%#v", awsstorage.Config{Credentials: provider})
_ = diagnostic
func (Config) LogValue ¶
LogValue implements slog.LogValuer with the redacted form of config. It does not retrieve credentials or invoke native provider formatters.
Example:
slog.Info("storage configuration", "config", awsstorage.Config{Credentials: provider})
type Driver ¶
type Driver struct {
// contains filtered or unexported fields
}
Driver stores portable objects in one S3 bucket and optional key prefix. It borrows native dependencies; callers own opened bodies and native transport cleanup. Use the storage facade for typed helpers and operation observations.
Example:
driver, err := awsstorage.NewFrom(client, "uploads")
if err != nil {
return err
}
files := storage.New(driver)
_ = files
func New ¶
New creates a Driver for bucket using configured credentials or the AWS default credential chain.
bucket is required and has no default; an empty bucket returns an error naming the S3 bucket. configure adjusts the optional settings; an invalid Config.Prefix also returns an error. No request is sent until the first operation. Callbacks run directly once, synchronously in order on a fresh draft before validation. Nil entries panic; expanded nil/empty slices omit edits. Completed settings are copied, not native credential providers.
Example:
store, err := awsstorage.New(ctx, "uploads", func(store *awsstorage.Builder) {
store.Prefix("tenant-a")
})
if err != nil {
return err
}
_ = store
func NewFrom ¶
func NewFrom(injectedClient client, bucket string, configure ...BuilderFunc) (*Driver, error)
NewFrom creates a Driver for bucket from a caller-owned S3 client.
The client is not copied or closed by the driver. bucket is validated like in New. Only Builder.Prefix applies from configure; Region, Endpoint, and Credentials are already carried by the client and are ignored. A nil client returns an error. Callbacks follow New's once-only, ordered draft rules; supplied nil entries panic before client validation.
URL signing uses the client's own presigner: an *s3.Client is wrapped with s3.NewPresignClient, and any other client that also implements PresignGetObject and PresignPutObject is used directly. For a client with neither, Driver.SignURL returns storage.ErrUnsupported.
Example:
store, err := awsstorage.NewFrom(s3.NewFromConfig(cfg), "uploads")
if err != nil {
return err
}
_ = store
func (*Driver) Client ¶
Client returns the underlying S3 client, or nil when the driver was built from a test double. Native calls bypass the logical Prefix, portable key validation, and facade observations; the client remains shared and borrowed.
Example:
if native := driver.Client(); native != nil {
_ = native.Options().Region
}
func (*Driver) Delete ¶
Delete removes key. S3 successful deletion is idempotent. Only the one prefixed key is removed, not objects below a similar path.
Example:
if err := driver.Delete(ctx, "reports/retired.pdf"); err != nil {
return err
}
func (*Driver) List ¶
List returns objects whose portable logical keys match options.Prefix. Native objects outside that syntax are omitted. Continuation checks can read ahead through physical pages containing only omitted objects; cursors retain their native S3 representation and are empty when no further logical object exists. Concurrent native bucket changes can still affect subsequent requests.
Example:
page, err := driver.List(ctx, storage.ListOptions{Prefix: "reports/", Limit: 20})
if err != nil {
return err
}
_ = page.Objects
_ = page.NextCursor // Continue with the same driver and logical prefix.
func (*Driver) Open ¶
Open returns an S3 object body that the caller must close. Missing objects return an error wrapping storage.ErrNotFound. An already canceled context takes precedence over key validation.
Example:
object, err := driver.Open(ctx, "reports/a.pdf")
if err != nil {
return err
}
data, err := io.ReadAll(object.Contents)
closeErr := object.Contents.Close()
if err != nil {
return err
}
if closeErr != nil {
return closeErr
}
_ = data
func (*Driver) Put ¶
func (driver *Driver) Put(ctx context.Context, key string, contents io.Reader) (storage.ObjectInfo, error)
Put replaces key then reads canonical metadata from S3.
PutObject needs the content length up front. A contents reader that cannot seek, such as an HTTP request body, is buffered in memory first; use the S3 upload manager through Driver.Client for very large streams. An already canceled context consumes nothing. Cancellation during a read still requires the caller's reader to cooperate; arbitrary Read cannot be forced.
Example:
info, err := driver.Put(ctx, "reports/a.json", strings.NewReader(`{"ready":true}`))
if err != nil {
return err
}
_ = info.Size
func (*Driver) SignURL ¶
func (driver *Driver) SignURL(ctx context.Context, key string, options storage.SignOptions) (*url.URL, error)
SignURL returns an AWS Signature Version 4 presigned URL that grants GET or PUT on key until options.TTL elapses. Validation follows storage.Driver.SignURL; the TTL may not exceed seven days, the SigV4 limit, and a longer one returns an error wrapping storage.ErrInvalidTTL. Config.Prefix applies, so the URL addresses the object that Driver.Put would store for key. A driver built by NewFrom from a client that cannot presign returns an error wrapping storage.ErrUnsupported.
For PUT, a nonempty options.ContentType and a positive options.ContentLength bind the Content-Type and Content-Length headers. Zero leaves length unbound; a type can be bound independently. If the presigner drops or changes the requested type, signing returns an error wrapping storage.ErrUnsupported instead of a URL with a weaker binding. This can occur when a borrowed native client's middleware computes zero length before the SDK removes Content-Type. The driver does not modify the client's middleware to override that behavior. Custom presigners must report the exact requested type as the sole signed Content-Type value in v4.PresignedHTTPRequest.SignedHeader. Set both bindings for user uploads: without them the URL holder picks the type (for example text/html) and the size. S3 presigned PUT can bind only an exact length, never a maximum; size ranges need a presigned POST policy, which this driver does not implement. The client must send exactly the signed headers.
A URL signed with temporary credentials, such as STS or IRSA, stops working when that session expires, even if the TTL is longer.
Example:
link, err := store.SignURL(ctx, "avatars/ada.png", storage.SignOptions{
Method: http.MethodPut,
TTL: 10 * time.Minute,
ContentType: "image/png",
ContentLength: 2048,
})
if err != nil {
return err
}
_ = link // Give the uploader this URL and the bound header values.
func (*Driver) Stat ¶
Stat returns canonical S3 metadata for key. Missing objects return an error wrapping storage.ErrNotFound. It applies the same logical key prefix as Open and Put without opening a body.
Example:
info, err := driver.Stat(ctx, "reports/a.pdf")
if err != nil {
return err
}
_ = info.ModifiedAt
Source Files