Documentation
¶
Overview ¶
Package s3 provides an S3-compatible backend for omnistorage.
This backend works with:
- AWS S3
- Cloudflare R2
- MinIO
- Wasabi
- DigitalOcean Spaces
- Any S3-compatible object storage
Basic usage:
backend, err := s3.New(s3.Config{
Bucket: "my-bucket",
Region: "us-east-1",
})
For S3-compatible services:
backend, err := s3.New(s3.Config{
Bucket: "my-bucket",
Endpoint: "https://play.min.io",
UsePathStyle: true,
})
Index ¶
- Variables
- func NewFromConfig(configMap map[string]string) (omnistorage.Backend, error)
- type Backend
- func (b *Backend) Close() error
- func (b *Backend) Copy(ctx context.Context, src, dst string) error
- func (b *Backend) Delete(ctx context.Context, p string) error
- func (b *Backend) Exists(ctx context.Context, p string) (bool, error)
- func (b *Backend) Features() omnistorage.Features
- func (b *Backend) List(ctx context.Context, prefix string) ([]string, error)
- func (b *Backend) Mkdir(ctx context.Context, p string) error
- func (b *Backend) Move(ctx context.Context, src, dst string) error
- func (b *Backend) NewReader(ctx context.Context, p string, opts ...omnistorage.ReaderOption) (io.ReadCloser, error)
- func (b *Backend) NewWriter(ctx context.Context, p string, opts ...omnistorage.WriterOption) (io.WriteCloser, error)
- func (b *Backend) Rmdir(ctx context.Context, p string) error
- func (b *Backend) Stat(ctx context.Context, p string) (omnistorage.ObjectInfo, error)
- type Config
Constants ¶
This section is empty.
Variables ¶
var (
ErrBucketRequired = errors.New("s3: bucket is required")
)
Errors specific to the S3 backend.
Functions ¶
func NewFromConfig ¶
func NewFromConfig(configMap map[string]string) (omnistorage.Backend, error)
NewFromConfig creates a new S3 backend from a config map. This is used by the omnistorage registry.
Types ¶
type Backend ¶
type Backend struct {
// contains filtered or unexported fields
}
Backend implements omnistorage.ExtendedBackend for S3-compatible storage.
func (*Backend) Features ¶
func (b *Backend) Features() omnistorage.Features
Features returns the capabilities of the S3 backend.
func (*Backend) NewReader ¶
func (b *Backend) NewReader(ctx context.Context, p string, opts ...omnistorage.ReaderOption) (io.ReadCloser, error)
NewReader creates a reader for the given path.
func (*Backend) NewWriter ¶
func (b *Backend) NewWriter(ctx context.Context, p string, opts ...omnistorage.WriterOption) (io.WriteCloser, error)
NewWriter creates a writer for the given path.
func (*Backend) Stat ¶
func (b *Backend) Stat(ctx context.Context, p string) (omnistorage.ObjectInfo, error)
Stat returns metadata about an object.
type Config ¶
type Config struct {
// Bucket is the S3 bucket name (required).
Bucket string
// Region is the AWS region (e.g., "us-east-1").
// If empty, uses AWS_REGION or AWS_DEFAULT_REGION environment variable.
Region string
// Endpoint is a custom endpoint URL for S3-compatible services.
// Examples:
// - MinIO: "http://localhost:9000"
// - Cloudflare R2: "https://<account_id>.r2.cloudflarestorage.com"
// - Wasabi: "https://s3.wasabisys.com"
// Leave empty for AWS S3.
Endpoint string
// Prefix is an optional prefix for all keys.
// Useful for organizing data within a bucket.
Prefix string
// AccessKeyID is the AWS access key ID.
// If empty, uses AWS_ACCESS_KEY_ID environment variable or IAM role.
AccessKeyID string
// SecretAccessKey is the AWS secret access key.
// If empty, uses AWS_SECRET_ACCESS_KEY environment variable or IAM role.
SecretAccessKey string
// SessionToken is an optional session token for temporary credentials.
SessionToken string
// UsePathStyle forces path-style addressing instead of virtual-hosted-style.
// Required for some S3-compatible services like MinIO.
// Set to true for: MinIO, some older S3-compatible services.
// Set to false for: AWS S3, Cloudflare R2, Wasabi.
UsePathStyle bool
// DisableSSL disables HTTPS for the endpoint.
// Only use for local development (e.g., local MinIO).
DisableSSL bool
// PartSize is the size in bytes for multipart upload parts.
// Default: 5MB (minimum for S3).
// Increase for better performance with large files.
PartSize int64
// Concurrency is the number of concurrent upload/download goroutines.
// Default: 5.
Concurrency int
}
Config holds configuration for the S3 backend.
func ConfigFromEnv ¶
func ConfigFromEnv() Config
ConfigFromEnv creates a Config from environment variables. Environment variables:
- OMNISTORAGE_S3_BUCKET or AWS_S3_BUCKET: bucket name
- OMNISTORAGE_S3_REGION or AWS_REGION or AWS_DEFAULT_REGION: region
- OMNISTORAGE_S3_ENDPOINT: custom endpoint
- OMNISTORAGE_S3_PREFIX: key prefix
- AWS_ACCESS_KEY_ID: access key
- AWS_SECRET_ACCESS_KEY: secret key
- AWS_SESSION_TOKEN: session token
- OMNISTORAGE_S3_USE_PATH_STYLE: "true" for path-style addressing
- OMNISTORAGE_S3_DISABLE_SSL: "true" to disable SSL
func ConfigFromMap ¶
ConfigFromMap creates a Config from a string map. Supported keys:
- bucket: bucket name (required)
- region: AWS region
- endpoint: custom endpoint URL
- prefix: key prefix
- access_key_id: AWS access key
- secret_access_key: AWS secret key
- session_token: session token
- use_path_style: "true" for path-style addressing
- disable_ssl: "true" to disable SSL
- part_size: multipart upload part size in bytes
- concurrency: number of concurrent operations
func DefaultConfig ¶
func DefaultConfig() Config
DefaultConfig returns a Config with default values.