driver

package
v1.8.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 5, 2026 License: MIT Imports: 8 Imported by: 9

Documentation

Overview

Package driver defines the core interfaces and types that every Trove storage backend must implement.

The Driver interface is intentionally minimal — advanced features are exposed via capability interfaces (MultipartDriver, PresignDriver, etc.) and the Unwrap pattern.

Error contract

Callers classify failures with errors.Is, so implementations MUST wrap the sentinels declared in errors.go rather than returning a bare descriptive error. Keep the descriptive message — wrap with %w, do not replace:

return nil, fmt.Errorf("mydriver: object %q not found in bucket %q: %w",
    key, bucket, driver.ErrObjectNotFound)

The conditions that MUST be classified:

  • a missing object → ErrObjectNotFound (Get, Head, Copy source, GetRange)
  • a missing bucket → ErrBucketNotFound (any operation naming a bucket)
  • an existing bucket → ErrBucketExists (CreateBucket)
  • an unauthorized op → ErrPermissionDenied
  • a quota/rate limit → ErrQuotaExceeded

This matters to schedulers built on Trove: a permanently missing object must fail its work item immediately, while a transient backend failure should be retried. Without the sentinel the two are indistinguishable and every retry is burned before the failure is recognized as permanent.

Backends that report these conditions with a typed error or an HTTP status code MUST use that signal rather than matching on message text. Delete is idempotent by convention and returns nil for a missing object.

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrNotFound reports that a requested resource does not exist.
	// It is the parent of ErrObjectNotFound and ErrBucketNotFound; match it
	// when any missing resource should be handled the same way.
	ErrNotFound = errors.New("trove: not found")

	// ErrObjectNotFound reports that an object does not exist.
	// It unwraps to ErrNotFound.
	ErrObjectNotFound error = &categoryError{msg: "trove: object not found", parent: ErrNotFound}

	// ErrBucketNotFound reports that a bucket or container does not exist.
	// It unwraps to ErrNotFound.
	ErrBucketNotFound error = &categoryError{msg: "trove: bucket not found", parent: ErrNotFound}

	// ErrBucketExists reports that a bucket being created already exists.
	ErrBucketExists = errors.New("trove: bucket already exists")

	// ErrPermissionDenied reports that the credentials in use are not
	// authorized for the operation. Retrying will not help until the
	// credentials or the backend's access policy change.
	ErrPermissionDenied = errors.New("trove: permission denied")

	// ErrQuotaExceeded reports that a storage quota or rate limit was
	// exceeded. Unlike ErrPermissionDenied, retrying later may succeed.
	ErrQuotaExceeded = errors.New("trove: quota exceeded")

	// ErrInvalidPath reports that a bucket name or object key is not
	// addressable — it is empty, contains a NUL byte, is rooted, or resolves
	// outside the container it was addressed to. Drivers that map keys onto a
	// hierarchical namespace wrap it from their containment guard.
	//
	// It is deliberately outside the not-found family. The resource was never
	// addressable, so nothing was looked up: a caller must be able to answer
	// "the request is malformed" (HTTP 400) rather than "it isn't there"
	// (404), which would both misdescribe the failure and let a client use
	// rejected keys to probe what exists.
	ErrInvalidPath = errors.New("trove: invalid path")
)

Sentinel errors that drivers must wrap so callers can classify failures with errors.Is, independent of which backend produced them.

These live in the driver package rather than the root trove package so that out-of-tree drivers (which live in their own Go modules) can depend on them without importing the whole Trove facade. The root package re-exports every one of them, and since errors.Is compares by identity, trove.ErrObjectNotFound and driver.ErrObjectNotFound are interchangeable.

The not-found sentinels form a small hierarchy: ErrObjectNotFound and ErrBucketNotFound both unwrap to ErrNotFound, so a caller that only cares whether something is missing can match the general sentinel, while a caller that needs to know what was missing can match the specific one.

errors.Is(err, driver.ErrObjectNotFound) // this exact object is gone
errors.Is(err, driver.ErrNotFound)       // something is gone — also true

Functions

func Drivers

func Drivers() []string

Drivers returns the names of all registered drivers.

func Permanent

func Permanent(err error) bool

Permanent reports whether err describes a condition that retrying cannot resolve, so the caller should fail the operation now rather than spend its backoff budget on attempts that are certain to fail the same way.

Permanent:

  • ErrNotFound, and with it ErrObjectNotFound and ErrBucketNotFound — the resource is gone and will not reappear on its own.
  • ErrPermissionDenied — nothing changes until the credentials or the backend's access policy do.
  • ErrBucketExists — the bucket will not stop existing because you asked again.
  • ErrInvalidPath — a malformed key does not become well-formed on the next attempt.

Not permanent:

  • ErrQuotaExceeded — the operation is refused now but may be granted later. Retry it, ideally with a longer backoff than a network failure.
  • Everything else, including unclassified errors. A driver that has not classified a condition is assumed to have hit a transient one; treating unrecognized errors as permanent would silently discard work whenever a backend grows a failure mode Trove does not yet map.

The quota check is evaluated first, so an error carrying both a quota signal and a not-found signal is reported as retryable.

if err != nil {
    if driver.Permanent(err) {
        return job.Fail(err)
    }
    return job.Retry(err)
}

func Register

func Register(name string, factory Factory)

Register registers a driver factory under the given name. It panics if a driver with the same name is already registered.

Types

type BucketConfig

type BucketConfig struct {
	// Region overrides the default region for the bucket.
	Region string
}

BucketConfig holds options for a CreateBucket operation.

func ApplyBucketOptions

func ApplyBucketOptions(opts ...BucketOption) BucketConfig

ApplyBucketOptions applies a set of BucketOptions to a BucketConfig.

type BucketInfo

type BucketInfo struct {
	Name      string    `json:"name"`
	CreatedAt time.Time `json:"created_at"`
}

BucketInfo holds bucket metadata.

type BucketOption

type BucketOption func(*BucketConfig)

BucketOption configures a CreateBucket operation.

func WithBucketRegion

func WithBucketRegion(region string) BucketOption

WithBucketRegion sets the region for a new bucket.

type Config

type Config struct {
	// Region is the cloud region for the storage backend.
	Region string

	// Endpoint overrides the default service endpoint.
	Endpoint string

	// ForcePathStyle forces path-style addressing for S3-compatible backends.
	ForcePathStyle bool
}

Config holds driver-level configuration applied during Open.

func ApplyOptions

func ApplyOptions(opts ...Option) Config

ApplyOptions applies a set of options to a Config.

type CopyConfig

type CopyConfig struct {
	// Metadata overrides the source object's metadata on the copy.
	Metadata map[string]string
}

CopyConfig holds options for a Copy operation.

func ApplyCopyOptions

func ApplyCopyOptions(opts ...CopyOption) CopyConfig

ApplyCopyOptions applies a set of CopyOptions to a CopyConfig.

type CopyOption

type CopyOption func(*CopyConfig)

CopyOption configures a Copy operation.

func WithCopyMetadata

func WithCopyMetadata(meta map[string]string) CopyOption

WithCopyMetadata overrides the source object's metadata on the copy.

type DSNConfig

type DSNConfig struct {
	// Scheme is the protocol scheme (e.g., "s3", "gcs", "file", "mem").
	Scheme string

	// User is the username or access key.
	User string

	// Password is the password or secret key.
	Password string

	// Host is the hostname and optional port.
	Host string

	// Path is the path component (e.g., bucket, directory).
	Path string

	// Params holds query string parameters.
	Params url.Values
}

DSNConfig holds parsed DSN components.

func ParseDSN

func ParseDSN(dsn string) (*DSNConfig, error)

ParseDSN parses a DSN string into its components.

Supported formats:

scheme://host/path?params
scheme://user:password@host/path?params
scheme:///path (host-less, e.g., file:///tmp/storage)

Its errors never quote the DSN, or any part of it but the scheme. A DSN carries credentials (an access key and secret in the userinfo, an account key or a connection string in the query), and an error travels into logs and onto trace spans: forge records a failed Register's error text on its lifecycle span. An error names the scheme when the DSN starts with a valid one followed by "://", and says what kind of problem it found.

type DeleteConfig

type DeleteConfig struct {
	// VersionID deletes a specific version.
	VersionID string
}

DeleteConfig holds options for a Delete operation.

func ApplyDeleteOptions

func ApplyDeleteOptions(opts ...DeleteOption) DeleteConfig

ApplyDeleteOptions applies a set of DeleteOptions to a DeleteConfig.

type DeleteOption

type DeleteOption func(*DeleteConfig)

DeleteOption configures a Delete operation.

type Driver

type Driver interface {
	// Name returns the driver identifier (e.g., "s3", "gcs", "azure", "local", "mem").
	Name() string

	// Open initializes the driver connection with the given DSN and options.
	Open(ctx context.Context, dsn string, opts ...Option) error

	// Close gracefully shuts down the driver, draining active streams.
	Close(ctx context.Context) error

	// Ping verifies backend connectivity.
	Ping(ctx context.Context) error

	// Put stores an object from a reader. Returns the stored object metadata.
	Put(ctx context.Context, bucket, key string, r io.Reader, opts ...PutOption) (*ObjectInfo, error)

	// Get retrieves an object, returning a ReadCloser for the content.
	Get(ctx context.Context, bucket, key string, opts ...GetOption) (*ObjectReader, error)

	// Delete removes an object.
	Delete(ctx context.Context, bucket, key string, opts ...DeleteOption) error

	// Head returns object metadata without content.
	Head(ctx context.Context, bucket, key string) (*ObjectInfo, error)

	// List returns objects matching the given options.
	List(ctx context.Context, bucket string, opts ...ListOption) (*ObjectIterator, error)

	// Copy copies an object within or across buckets.
	Copy(ctx context.Context, srcBucket, srcKey, dstBucket, dstKey string, opts ...CopyOption) (*ObjectInfo, error)

	// CreateBucket creates a new bucket/container.
	CreateBucket(ctx context.Context, name string, opts ...BucketOption) error

	// DeleteBucket removes a bucket.
	DeleteBucket(ctx context.Context, name string) error

	// ListBuckets returns all accessible buckets.
	ListBuckets(ctx context.Context) ([]BucketInfo, error)
}

Driver is the core interface every storage backend implements. It exposes the minimum surface area for object CRUD and streaming.

type Factory

type Factory func() Driver

Factory creates a new driver instance.

func Lookup

func Lookup(name string) (Factory, bool)

Lookup returns the driver factory registered under the given name.

type GetConfig

type GetConfig struct {
	// Range specifies a byte range to retrieve (e.g., "bytes=0-1023").
	Range string

	// VersionID retrieves a specific version of the object.
	VersionID string
}

GetConfig holds options for a Get operation.

func ApplyGetOptions

func ApplyGetOptions(opts ...GetOption) GetConfig

ApplyGetOptions applies a set of GetOptions to a GetConfig.

type GetOption

type GetOption func(*GetConfig)

GetOption configures a Get operation.

func WithRange

func WithRange(r string) GetOption

WithRange sets the byte range for a Get operation.

func WithVersionID

func WithVersionID(vid string) GetOption

WithVersionID retrieves a specific version.

type KeyPage

type KeyPage struct {
	// Keys are the object keys on this page, in order.
	Keys []string
	// Prefixes are the common prefixes on this page, in order.
	Prefixes []string
	// NextToken is the last item on this page when more items follow,
	// and empty when the listing is complete.
	NextToken string
}

KeyPage is one page of a listing, computed from a full sorted key set.

func PageKeys

func PageKeys(sorted []string, cfg ListConfig) KeyPage

PageKeys computes one page of a listing for drivers that hold, or can walk, their whole key set. The keys must be sorted lexicographically and free of duplicates.

It applies cfg.Prefix, folds keys into common prefixes when cfg.Delimiter is set, skips every item at or before cfg.Cursor, and stops after cfg.MaxKeys items, counting prefixes and keys together. A MaxKeys of zero or less means 1000. The returned NextToken is the last item emitted, key or prefix, which is what the next call's Cursor must be.

type LifecycleDriver

type LifecycleDriver interface {
	Driver
	SetLifecycle(ctx context.Context, bucket string, rules []LifecycleRule) error
	GetLifecycle(ctx context.Context, bucket string) ([]LifecycleRule, error)
}

LifecycleDriver extends Driver with lifecycle management.

type LifecycleRule

type LifecycleRule struct {
	ID                string `json:"id"`
	Prefix            string `json:"prefix"`
	ExpirationDays    int    `json:"expiration_days,omitempty"`
	TransitionDays    int    `json:"transition_days,omitempty"`
	TransitionStorage string `json:"transition_storage,omitempty"`
	Enabled           bool   `json:"enabled"`
}

LifecycleRule defines a storage lifecycle transition or expiration rule.

type ListConfig

type ListConfig struct {
	// Prefix filters objects by key prefix.
	Prefix string

	// Delimiter groups keys by a delimiter character (e.g., "/"). Keys that
	// contain the delimiter after Prefix are folded into a common prefix and
	// come back in ObjectIterator.CommonPrefixes() instead of as objects.
	Delimiter string

	// MaxKeys is the maximum number of objects and common prefixes returned
	// together in one page.
	MaxKeys int

	// Cursor is the NextToken from a previous List call, passed back
	// unchanged. It is opaque: drivers may use a key, a backend
	// continuation token or anything else, so never construct one.
	Cursor string
}

ListConfig holds options for a List operation.

func ApplyListOptions

func ApplyListOptions(opts ...ListOption) ListConfig

ApplyListOptions applies a set of ListOptions to a ListConfig.

type ListOption

type ListOption func(*ListConfig)

ListOption configures a List operation.

func WithCursor

func WithCursor(cursor string) ListOption

WithCursor sets the pagination cursor.

func WithDelimiter

func WithDelimiter(delimiter string) ListOption

WithDelimiter groups keys by a delimiter character.

func WithMaxKeys

func WithMaxKeys(n int) ListOption

WithMaxKeys sets the maximum number of objects to return.

func WithPrefix

func WithPrefix(prefix string) ListOption

WithPrefix filters objects by key prefix.

type MultipartDriver

type MultipartDriver interface {
	Driver
	InitiateMultipart(ctx context.Context, bucket, key string, opts ...PutOption) (uploadID string, err error)
	UploadPart(ctx context.Context, bucket, key, uploadID string, partNum int, r io.Reader) (*PartInfo, error)
	CompleteMultipart(ctx context.Context, bucket, key, uploadID string, parts []PartInfo) (*ObjectInfo, error)
	AbortMultipart(ctx context.Context, bucket, key, uploadID string) error
}

MultipartDriver extends Driver with multipart upload support.

type NotificationDriver

type NotificationDriver interface {
	Driver
	Watch(ctx context.Context, bucket string, opts ...WatchOption) (<-chan ObjectEvent, error)
}

NotificationDriver extends Driver with change notification support.

type ObjectEvent

type ObjectEvent struct {
	Type   ObjectEventType `json:"type"`
	Bucket string          `json:"bucket"`
	Key    string          `json:"key"`
	Info   *ObjectInfo     `json:"info,omitempty"`
}

ObjectEvent represents a change notification.

type ObjectEventType

type ObjectEventType string

ObjectEventType identifies the type of object change event.

const (
	EventCreated ObjectEventType = "object.created"
	EventDeleted ObjectEventType = "object.deleted"
	EventUpdated ObjectEventType = "object.updated"
)

type ObjectInfo

type ObjectInfo struct {
	Key          string            `json:"key"`
	Size         int64             `json:"size"`
	ContentType  string            `json:"content_type"`
	ETag         string            `json:"etag"`
	LastModified time.Time         `json:"last_modified"`
	Metadata     map[string]string `json:"metadata,omitempty"`
	VersionID    string            `json:"version_id,omitempty"`
	StorageClass string            `json:"storage_class,omitempty"`
}

ObjectInfo holds the metadata returned by backend operations.

type ObjectIterator

type ObjectIterator struct {
	// contains filtered or unexported fields
}

ObjectIterator provides cursor-based listing of objects.

NextToken is opaque. Pass it back unchanged as the Cursor of the next List call; an empty NextToken means the listing is complete. When the List call set a Delimiter, CommonPrefixes holds the prefixes that keys below them were folded into, and MaxKeys counted them alongside objects.

func NewObjectIterator

func NewObjectIterator(objects []ObjectInfo, nextToken string) *ObjectIterator

NewObjectIterator creates an iterator from a slice of objects and an optional continuation token for the next page.

func NewObjectIteratorWithPrefixes

func NewObjectIteratorWithPrefixes(objects []ObjectInfo, prefixes []string, nextToken string) *ObjectIterator

NewObjectIteratorWithPrefixes creates an iterator that also reports the common prefixes a delimiter listing folded keys into.

func (*ObjectIterator) All

func (it *ObjectIterator) All(ctx context.Context) ([]ObjectInfo, error)

All collects all remaining objects from the iterator into a slice.

func (*ObjectIterator) Close

func (it *ObjectIterator) Close() error

Close releases any resources held by the iterator.

func (*ObjectIterator) CommonPrefixes

func (it *ObjectIterator) CommonPrefixes() []string

CommonPrefixes returns the prefixes this page folded keys into. It is empty when the List call set no Delimiter.

func (*ObjectIterator) Next

func (it *ObjectIterator) Next(_ context.Context) (*ObjectInfo, error)

Next returns the next object in the iterator. Returns nil, io.EOF when no more objects are available.

func (*ObjectIterator) NextToken

func (it *ObjectIterator) NextToken() string

NextToken returns the continuation token for fetching the next page. Returns empty string when all results have been returned.

type ObjectReader

type ObjectReader struct {
	io.ReadCloser
	Info *ObjectInfo
}

ObjectReader wraps content with metadata.

type ObjectRef

type ObjectRef struct {
	Bucket string `json:"bucket"`
	Key    string `json:"key"`
}

ObjectRef identifies an object by bucket and key.

type Option

type Option func(*Config)

Option configures a driver during Open.

func WithEndpoint

func WithEndpoint(endpoint string) Option

WithEndpoint overrides the default service endpoint.

func WithForcePathStyle

func WithForcePathStyle(force bool) Option

WithForcePathStyle forces path-style addressing.

func WithRegion

func WithRegion(region string) Option

WithRegion sets the cloud region.

type PartInfo

type PartInfo struct {
	PartNumber int    `json:"part_number"`
	ETag       string `json:"etag"`
	Size       int64  `json:"size"`
}

PartInfo holds metadata about an uploaded part.

type PresignDriver

type PresignDriver interface {
	Driver
	PresignGet(ctx context.Context, bucket, key string, expires time.Duration) (string, error)
	PresignPut(ctx context.Context, bucket, key string, expires time.Duration) (string, error)
}

PresignDriver extends Driver with pre-signed URL generation.

type PutConfig

type PutConfig struct {
	ContentType  string
	Metadata     map[string]string
	Tags         map[string]string
	StorageClass string
}

PutConfig holds options for a Put operation.

func ApplyPutOptions

func ApplyPutOptions(opts ...PutOption) PutConfig

ApplyPutOptions applies a set of PutOptions to a PutConfig.

type PutOption

type PutOption func(*PutConfig)

PutOption configures a Put operation.

func WithContentType

func WithContentType(ct string) PutOption

WithContentType sets the object's content type.

func WithMetadata

func WithMetadata(meta map[string]string) PutOption

WithMetadata sets custom metadata on the object.

func WithStorageClass

func WithStorageClass(class string) PutOption

WithStorageClass sets the storage class for the object.

func WithTags

func WithTags(tags map[string]string) PutOption

WithTags sets custom tags on the object.

type RangeDriver

type RangeDriver interface {
	Driver
	// GetRange retrieves a byte range of an object. The offset is the
	// starting byte position and length is the number of bytes to read.
	// A length of -1 reads from offset to end.
	GetRange(ctx context.Context, bucket, key string, offset, length int64) (*ObjectReader, error)
}

RangeDriver extends Driver with byte-range read support. This enables resumable downloads and partial content retrieval.

type ServerCopyDriver

type ServerCopyDriver interface {
	Driver
	ServerCopy(ctx context.Context, src, dst ObjectRef) (*ObjectInfo, error)
}

ServerCopyDriver extends Driver with server-side copy (avoids download+reupload).

type VersionInfo

type VersionInfo struct {
	VersionID    string    `json:"version_id"`
	Key          string    `json:"key"`
	Size         int64     `json:"size"`
	LastModified time.Time `json:"last_modified"`
	IsLatest     bool      `json:"is_latest"`
}

VersionInfo holds metadata about an object version.

type VersioningDriver

type VersioningDriver interface {
	Driver
	ListVersions(ctx context.Context, bucket, key string) ([]VersionInfo, error)
	GetVersion(ctx context.Context, bucket, key, versionID string) (*ObjectReader, error)
	DeleteVersion(ctx context.Context, bucket, key, versionID string) error
	RestoreVersion(ctx context.Context, bucket, key, versionID string) (*ObjectInfo, error)
}

VersioningDriver extends Driver with version management.

type WatchConfig

type WatchConfig struct {
	Prefix     string
	EventTypes []ObjectEventType
}

WatchConfig holds options for a Watch operation.

type WatchOption

type WatchOption func(*WatchConfig)

WatchOption configures a Watch operation.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL