storage

package
v1.27.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 11, 2026 License: Apache-2.0 Imports: 15 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	Private = provider.Private
	Public  = provider.Public

	ProviderS3    = provider.ProviderS3
	ProviderGCS   = provider.ProviderGCS
	ProviderAzure = provider.ProviderAzure
	ProviderLocal = provider.ProviderLocal
)

Variables

View Source 
var ErrNoTenantInContext = errors.New("storage: no tenant in context — this operation would use the shared (unprefixed) key space; carry the request scope into the context, or disable multitenant.require_tenant_storage if shared keys are intended")

ErrNoTenantInContext is returned by a strict TenantStore when an operation runs with no tenant resolvable from the context — the case that would otherwise read or write the SHARED (unprefixed) key space. See TenantStoreOptions.Strict.

Functions

func CleanupTempKey 

func CleanupTempKey(purpose string) string

CleanupTempKey generates a temporary key with the configured prefix. Use this for import uploads and other temporary storage that should be cleaned up automatically.

func IsTempKey 

func IsTempKey(key string) bool

IsTempKey checks if a key is a temporary key.

func RegisterProvider 

func RegisterProvider(name string, factory ProviderFactory) error

RegisterProvider makes a storage backend selectable by name from configuration (`storage.provider`). It delegates to provider.Register; a new backend should call that one and avoid importing this package at all.

func RegisteredProviders 

func RegisteredProviders() []string

RegisteredProviders returns every selectable provider name, sorted.

Types

type AzureConfig 

type AzureConfig = provider.AzureConfig

The contract a third-party storage backend implements now lives in pkg/storage/provider, a leaf package.

The reason is a measurement: this package links 301 third-party packages — the AWS, Azure and Google Cloud SDKs, because the S3, Blob and GCS implementations live here — and until now that was the price of implementing the Store interface. Someone writing a backend for Ceph, for an internal object store, or for a filesystem they control inherited three cloud SDKs they will never call. The leaf links two.

The names below are ALIASES, not copies: storage.Store and provider.Store are the same interface, storage.Config and provider.Config the same struct, so everything that compiled before still compiles. A new backend should import pkg/storage/provider and pay for what it uses.

type CircuitBreakerConfig 

type CircuitBreakerConfig = provider.CircuitBreakerConfig

The contract a third-party storage backend implements now lives in pkg/storage/provider, a leaf package.

The reason is a measurement: this package links 301 third-party packages — the AWS, Azure and Google Cloud SDKs, because the S3, Blob and GCS implementations live here — and until now that was the price of implementing the Store interface. Someone writing a backend for Ceph, for an internal object store, or for a filesystem they control inherited three cloud SDKs they will never call. The leaf links two.

The names below are ALIASES, not copies: storage.Store and provider.Store are the same interface, storage.Config and provider.Config the same struct, so everything that compiled before still compiles. A new backend should import pkg/storage/provider and pay for what it uses.

type Cleaner 

type Cleaner struct {
	// contains filtered or unexported fields
}

Cleaner runs periodic cleanup of temporary objects.

func NewCleaner 

func NewCleaner(store Store, cfg CleanupConfig, logger *slog.Logger) (*Cleaner, error)

NewCleaner creates a background cleanup goroutine.

func (*Cleaner) Start 

func (c *Cleaner) Start()

Start begins the background cleanup loop. Call Stop() to terminate gracefully. Start begins the periodic sweep. It is a no-op when cleanup is disabled in config, which is the whole point of the switch: run() sweeps once immediately, before the first tick, so a cleaner that starts at all has already deleted by the time anyone could stop it.

func (*Cleaner) Stop 

func (c *Cleaner) Stop()

Stop signals the cleaner to terminate.

type CleanupConfig 

type CleanupConfig = provider.CleanupConfig

The contract a third-party storage backend implements now lives in pkg/storage/provider, a leaf package.

The reason is a measurement: this package links 301 third-party packages — the AWS, Azure and Google Cloud SDKs, because the S3, Blob and GCS implementations live here — and until now that was the price of implementing the Store interface. Someone writing a backend for Ceph, for an internal object store, or for a filesystem they control inherited three cloud SDKs they will never call. The leaf links two.

The names below are ALIASES, not copies: storage.Store and provider.Store are the same interface, storage.Config and provider.Config the same struct, so everything that compiled before still compiles. A new backend should import pkg/storage/provider and pay for what it uses.

type Config 

type Config = provider.Config

Config holds the complete storage configuration.

func DefaultConfig 

func DefaultConfig() Config

DefaultConfig returns the default storage configuration.

type CredentialSource 

type CredentialSource = provider.CredentialSource

The contract a third-party storage backend implements now lives in pkg/storage/provider, a leaf package.

The reason is a measurement: this package links 301 third-party packages — the AWS, Azure and Google Cloud SDKs, because the S3, Blob and GCS implementations live here — and until now that was the price of implementing the Store interface. Someone writing a backend for Ceph, for an internal object store, or for a filesystem they control inherited three cloud SDKs they will never call. The leaf links two.

The names below are ALIASES, not copies: storage.Store and provider.Store are the same interface, storage.Config and provider.Config the same struct, so everything that compiled before still compiles. A new backend should import pkg/storage/provider and pay for what it uses.

type ErrInvalidKey 

type ErrInvalidKey = provider.ErrInvalidKey

The contract a third-party storage backend implements now lives in pkg/storage/provider, a leaf package.

The reason is a measurement: this package links 301 third-party packages — the AWS, Azure and Google Cloud SDKs, because the S3, Blob and GCS implementations live here — and until now that was the price of implementing the Store interface. Someone writing a backend for Ceph, for an internal object store, or for a filesystem they control inherited three cloud SDKs they will never call. The leaf links two.

The names below are ALIASES, not copies: storage.Store and provider.Store are the same interface, storage.Config and provider.Config the same struct, so everything that compiled before still compiles. A new backend should import pkg/storage/provider and pay for what it uses.

type ErrNotFound 

type ErrNotFound = provider.ErrNotFound

The contract a third-party storage backend implements now lives in pkg/storage/provider, a leaf package.

The reason is a measurement: this package links 301 third-party packages — the AWS, Azure and Google Cloud SDKs, because the S3, Blob and GCS implementations live here — and until now that was the price of implementing the Store interface. Someone writing a backend for Ceph, for an internal object store, or for a filesystem they control inherited three cloud SDKs they will never call. The leaf links two.

The names below are ALIASES, not copies: storage.Store and provider.Store are the same interface, storage.Config and provider.Config the same struct, so everything that compiled before still compiles. A new backend should import pkg/storage/provider and pay for what it uses.

type GCSConfig 

type GCSConfig = provider.GCSConfig

The contract a third-party storage backend implements now lives in pkg/storage/provider, a leaf package.

The reason is a measurement: this package links 301 third-party packages — the AWS, Azure and Google Cloud SDKs, because the S3, Blob and GCS implementations live here — and until now that was the price of implementing the Store interface. Someone writing a backend for Ceph, for an internal object store, or for a filesystem they control inherited three cloud SDKs they will never call. The leaf links two.

The names below are ALIASES, not copies: storage.Store and provider.Store are the same interface, storage.Config and provider.Config the same struct, so everything that compiled before still compiles. A new backend should import pkg/storage/provider and pay for what it uses.

type ListOptions 

type ListOptions = provider.ListOptions

The contract a third-party storage backend implements now lives in pkg/storage/provider, a leaf package.

The reason is a measurement: this package links 301 third-party packages — the AWS, Azure and Google Cloud SDKs, because the S3, Blob and GCS implementations live here — and until now that was the price of implementing the Store interface. Someone writing a backend for Ceph, for an internal object store, or for a filesystem they control inherited three cloud SDKs they will never call. The leaf links two.

The names below are ALIASES, not copies: storage.Store and provider.Store are the same interface, storage.Config and provider.Config the same struct, so everything that compiled before still compiles. A new backend should import pkg/storage/provider and pay for what it uses.

type ListResult 

type ListResult = provider.ListResult

The contract a third-party storage backend implements now lives in pkg/storage/provider, a leaf package.

The reason is a measurement: this package links 301 third-party packages — the AWS, Azure and Google Cloud SDKs, because the S3, Blob and GCS implementations live here — and until now that was the price of implementing the Store interface. Someone writing a backend for Ceph, for an internal object store, or for a filesystem they control inherited three cloud SDKs they will never call. The leaf links two.

The names below are ALIASES, not copies: storage.Store and provider.Store are the same interface, storage.Config and provider.Config the same struct, so everything that compiled before still compiles. A new backend should import pkg/storage/provider and pay for what it uses.

type LocalConfig 

type LocalConfig = provider.LocalConfig

The contract a third-party storage backend implements now lives in pkg/storage/provider, a leaf package.

The reason is a measurement: this package links 301 third-party packages — the AWS, Azure and Google Cloud SDKs, because the S3, Blob and GCS implementations live here — and until now that was the price of implementing the Store interface. Someone writing a backend for Ceph, for an internal object store, or for a filesystem they control inherited three cloud SDKs they will never call. The leaf links two.

The names below are ALIASES, not copies: storage.Store and provider.Store are the same interface, storage.Config and provider.Config the same struct, so everything that compiled before still compiles. A new backend should import pkg/storage/provider and pay for what it uses.

type LocalStore 

type LocalStore struct {
	// contains filtered or unexported fields
}

LocalStore implements Store using the local filesystem. Suitable ONLY for development. Do NOT use in production.

func NewLocalStore 

func NewLocalStore(cfg LocalConfig) (*LocalStore, error)

NewLocalStore creates a local filesystem storage backend.

func (*LocalStore) Close 

func (s *LocalStore) Close() error

func (*LocalStore) Copy 

func (s *LocalStore) Copy(ctx context.Context, srcKey, dstKey string) (ObjectInfo, error)

func (*LocalStore) Delete 

func (s *LocalStore) Delete(ctx context.Context, key string) error

func (*LocalStore) Exists 

func (s *LocalStore) Exists(ctx context.Context, key string) (bool, error)

func (*LocalStore) Get 

func (s *LocalStore) Get(ctx context.Context, key string) (io.ReadCloser, ObjectInfo, error)

func (*LocalStore) List 

func (s *LocalStore) List(ctx context.Context, opts ListOptions) (ListResult, error)

func (*LocalStore) PublicURL 

func (s *LocalStore) PublicURL(ctx context.Context, key string, opts URLConfig) (string, error)

func (*LocalStore) Put 

func (s *LocalStore) Put(ctx context.Context, key string, reader io.Reader, opts PutOptions) (ObjectInfo, error)

func (*LocalStore) SignedURL 

func (s *LocalStore) SignedURL(ctx context.Context, key string, expires time.Duration, opts URLConfig) (string, error)

type ObjectInfo 

type ObjectInfo = provider.ObjectInfo

The contract a third-party storage backend implements now lives in pkg/storage/provider, a leaf package.

The reason is a measurement: this package links 301 third-party packages — the AWS, Azure and Google Cloud SDKs, because the S3, Blob and GCS implementations live here — and until now that was the price of implementing the Store interface. Someone writing a backend for Ceph, for an internal object store, or for a filesystem they control inherited three cloud SDKs they will never call. The leaf links two.

The names below are ALIASES, not copies: storage.Store and provider.Store are the same interface, storage.Config and provider.Config the same struct, so everything that compiled before still compiles. A new backend should import pkg/storage/provider and pay for what it uses.

type ProviderFactory 

type ProviderFactory = provider.Factory

ProviderFactory builds a Store from the configuration.

type ProviderType 

type ProviderType = provider.ProviderType

The contract a third-party storage backend implements now lives in pkg/storage/provider, a leaf package.

The reason is a measurement: this package links 301 third-party packages — the AWS, Azure and Google Cloud SDKs, because the S3, Blob and GCS implementations live here — and until now that was the price of implementing the Store interface. Someone writing a backend for Ceph, for an internal object store, or for a filesystem they control inherited three cloud SDKs they will never call. The leaf links two.

The names below are ALIASES, not copies: storage.Store and provider.Store are the same interface, storage.Config and provider.Config the same struct, so everything that compiled before still compiles. A new backend should import pkg/storage/provider and pay for what it uses.

type PublicMapper 

type PublicMapper struct {
	// contains filtered or unexported fields
}

PublicMapper handles the mapping between public URL paths and storage keys. It is used both for generating public URLs and for mounting HTTP handlers.

func NewPublicMapper 

func NewPublicMapper(store Store, publicPaths map[string]string, publicURLBase string) *PublicMapper

NewPublicMapper creates a public path mapper.

func NewPublicMapperForConfig 

func NewPublicMapperForConfig(store Store, cfg Config) *PublicMapper

NewPublicMapper creates a public URL mapper for the given store.

func (*PublicMapper) IsPublicKey 

func (m *PublicMapper) IsPublicKey(key string) bool

IsPublicKey checks if a key falls under any public path mapping.

func (*PublicMapper) Mount 

func (m *PublicMapper) Mount(mux interface {
	Get(string, ...router.Handler)
}, publicPath string, storagePrefix string)

Mount registers HTTP handlers for all configured public paths. Requests to /media/* will be served directly from the storage backend, bypassing the need for signed URLs.

Example: 

app.Storage.Public().Mount(router, "/media", "storage/public/media/")
// GET /media/blog/hero.png -> serves storage/public/media/blog/hero.png from storage

func (*PublicMapper) MountAll 

func (m *PublicMapper) MountAll(mux interface {
	Get(string, ...router.Handler)
})

MountAll registers HTTP handlers for ALL configured public paths.

func (*PublicMapper) PublicURL 

func (m *PublicMapper) PublicURL(ctx context.Context, key string, opts URLConfig) (string, error)

PublicURL constructs a public URL for a storage key. It checks if the key matches any configured public path and returns the corresponding public URL. Returns empty string if no mapping exists.

type PutOptions 

type PutOptions = provider.PutOptions

The contract a third-party storage backend implements now lives in pkg/storage/provider, a leaf package.

The reason is a measurement: this package links 301 third-party packages — the AWS, Azure and Google Cloud SDKs, because the S3, Blob and GCS implementations live here — and until now that was the price of implementing the Store interface. Someone writing a backend for Ceph, for an internal object store, or for a filesystem they control inherited three cloud SDKs they will never call. The leaf links two.

The names below are ALIASES, not copies: storage.Store and provider.Store are the same interface, storage.Config and provider.Config the same struct, so everything that compiled before still compiles. A new backend should import pkg/storage/provider and pay for what it uses.

type S3Config 

type S3Config = provider.S3Config

The contract a third-party storage backend implements now lives in pkg/storage/provider, a leaf package.

The reason is a measurement: this package links 301 third-party packages — the AWS, Azure and Google Cloud SDKs, because the S3, Blob and GCS implementations live here — and until now that was the price of implementing the Store interface. Someone writing a backend for Ceph, for an internal object store, or for a filesystem they control inherited three cloud SDKs they will never call. The leaf links two.

The names below are ALIASES, not copies: storage.Store and provider.Store are the same interface, storage.Config and provider.Config the same struct, so everything that compiled before still compiles. A new backend should import pkg/storage/provider and pay for what it uses.

type Store 

type Store = provider.Store

Store is the durable object storage interface. See provider.Store.

func New 

func New(cfg Config, logger *slog.Logger) (Store, error)

New creates a Store from configuration. This is the primary entry point for the storage package.

Example: 

store, err := storage.New(cfg, logger)
if err != nil {
    log.Fatal(err)
}
defer store.Close()

type TenantKey 

type TenantKey struct{}

TenantKey is the context key for tenant ID.

type TenantStore 

type TenantStore struct {
	// contains filtered or unexported fields
}

TenantStore wraps a Store and automatically prefixes all keys with the current tenant ID (extracted from context). This provides automatic tenant isolation at the storage level without requiring application code changes.

Key transformation example: 

// App code:
store.Put(ctx, "uploads/invoice.pdf", ...)

// Actual S3 key (if tenant "acme" is in context):
"acme/uploads/invoice.pdf"

If no tenant is in context, keys are stored without prefix — by default SILENTLY, which is exactly how a background job without request scope ends up writing tenant data into the shared key space (NF-12). Construct with NewTenantStoreWithOptions to get a WARN on the first unprefixed operation, or Strict mode that rejects them with ErrNoTenantInContext.

func NewTenantStore 

func NewTenantStore(store Store, getter func(context.Context) string) *TenantStore

NewTenantStore creates a tenant-prefixing wrapper. The getter function extracts the tenant ID from context. Pass nil for getter to disable tenant prefixing.

func NewTenantStoreWithOptions 

func NewTenantStoreWithOptions(store Store, getter func(context.Context) string, opts TenantStoreOptions) *TenantStore

NewTenantStoreWithOptions is NewTenantStore with an explicit policy for tenant-less operations. Both Strict and Logger are inert when getter is nil — a nil getter means prefixing is deliberately disabled.

func NewWithTenant 

func NewWithTenant(store Store, tenantGetter func(context.Context) string) *TenantStore

NewWithTenant wraps a store with tenant prefixing. The tenantGetter extracts tenant ID from context. Pass nil for tenantGetter to disable tenant prefixing.

func (*TenantStore) Close 

func (t *TenantStore) Close() error

func (*TenantStore) Copy 

func (t *TenantStore) Copy(ctx context.Context, srcKey, dstKey string) (ObjectInfo, error)

func (*TenantStore) Delete 

func (t *TenantStore) Delete(ctx context.Context, key string) error

func (*TenantStore) Exists 

func (t *TenantStore) Exists(ctx context.Context, key string) (bool, error)

func (*TenantStore) Get 

func (t *TenantStore) Get(ctx context.Context, key string) (io.ReadCloser, ObjectInfo, error)

func (*TenantStore) List 

func (t *TenantStore) List(ctx context.Context, opts ListOptions) (ListResult, error)

func (*TenantStore) PublicURL 

func (t *TenantStore) PublicURL(ctx context.Context, key string, opts URLConfig) (string, error)

func (*TenantStore) Put 

func (t *TenantStore) Put(ctx context.Context, key string, reader io.Reader, opts PutOptions) (ObjectInfo, error)

func (*TenantStore) SignedURL 

func (t *TenantStore) SignedURL(ctx context.Context, key string, expires time.Duration, opts URLConfig) (string, error)

func (*TenantStore) Unwrap 

func (t *TenantStore) Unwrap() Store

Unwrap returns the underlying store (for type assertions to provider-specific features).

func (*TenantStore) UnwrapIfCleaner 

func (t *TenantStore) UnwrapIfCleaner() Store

UnwrapIfCleaner returns the underlying store for cleanup configuration.

type TenantStoreOptions 

type TenantStoreOptions struct {
	// Strict rejects tenant-less operations with ErrNoTenantInContext
	// instead of degrading to the shared key space. Wire it from
	// multitenant.require_tenant_storage.
	Strict bool

	// Logger, when non-nil and Strict is off, receives ONE warning the
	// first time an operation degrades to the shared key space, so the
	// degradation is at least visible without flooding the log on every
	// background-job call.
	Logger *slog.Logger
}

TenantStoreOptions configures how a TenantStore treats an operation whose context resolves to NO tenant (NF-12). The zero value preserves the historical behaviour: unprefixed keys, in silence.

type URLConfig 

type URLConfig = provider.URLConfig

The contract a third-party storage backend implements now lives in pkg/storage/provider, a leaf package.

The reason is a measurement: this package links 301 third-party packages — the AWS, Azure and Google Cloud SDKs, because the S3, Blob and GCS implementations live here — and until now that was the price of implementing the Store interface. Someone writing a backend for Ceph, for an internal object store, or for a filesystem they control inherited three cloud SDKs they will never call. The leaf links two.

The names below are ALIASES, not copies: storage.Store and provider.Store are the same interface, storage.Config and provider.Config the same struct, so everything that compiled before still compiles. A new backend should import pkg/storage/provider and pay for what it uses.

type Visibility 

type Visibility = provider.Visibility

The contract a third-party storage backend implements now lives in pkg/storage/provider, a leaf package.

The reason is a measurement: this package links 301 third-party packages — the AWS, Azure and Google Cloud SDKs, because the S3, Blob and GCS implementations live here — and until now that was the price of implementing the Store interface. Someone writing a backend for Ceph, for an internal object store, or for a filesystem they control inherited three cloud SDKs they will never call. The leaf links two.

The names below are ALIASES, not copies: storage.Store and provider.Store are the same interface, storage.Config and provider.Config the same struct, so everything that compiled before still compiles. A new backend should import pkg/storage/provider and pay for what it uses.

Source Files

View all Source files

Directories

Path Synopsis
Package storage provides a durable, provider-agnostic file storage interface for Nucleus applications.
 Package storage provides a durable, provider-agnostic file storage interface for Nucleus applications.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.