storage

package
v0.45.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 8, 2025 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	S3Provider       = storagetypes.S3Provider
	R2Provider       = storagetypes.R2Provider
	GCSProvider      = storagetypes.GCSProvider
	DiskProvider     = storagetypes.DiskProvider
	DatabaseProvider = storagetypes.DatabaseProvider
	// Presign mode constants
	PresignModeProvider = storagetypes.PresignModeProvider
	PresignModeProxy    = storagetypes.PresignModeProxy
)

Provider type constants so we can range, switch, etc

View Source 
const (
	DefaultMaxFileSize   = 32 << 20 // 32MB
	DefaultMaxMemory     = 32 << 20 // 32MB
	DefaultUploadFileKey = "uploadFile"
)

Configuration constants

View Source 
const (
	// ErrNoFilesUploaded is returned when no files were uploaded in the request
	ErrNoFilesUploaded = errorMsg("objects: no uploadable files found in request")
)
View Source 
const (
	// MIMEDetectionBufferSize defines the buffer size for MIME type detection
	MIMEDetectionBufferSize = 512
)

Variables

View Source 
var (
	// ErrJSONParseFailed is returned when JSON parsing fails
	ErrJSONParseFailed = errors.New("failed to parse JSON")
	// ErrYAMLParseFailed is returned when YAML parsing fails
	ErrYAMLParseFailed = errors.New("failed to parse YAML")
	// ErrProviderResolutionRequired is returned when provider resolution is required but not available
	ErrProviderResolutionRequired = errors.New("provider resolution required - use external orchestration layer")
)
View Source 
var (
	DefaultValidationFunc ValidationFunc = func(_ File) error {
		return nil
	}

	DefaultNameGeneratorFunc = func(originalName string) string {
		return originalName
	}

	DefaultSkipper = func(_ *http.Request) bool {
		return false
	}

	DefaultErrorResponseHandler = func(err error, statusCode int) http.HandlerFunc {
		return func(w http.ResponseWriter, _ *http.Request) {
			http.Error(w, err.Error(), statusCode)
		}
	}
)

Default function implementations

Functions

func DetectContentType added in v0.38.1 

func DetectContentType(reader io.ReadSeeker) (string, error)

DetectContentType detects the MIME type of the provided reader using gabriel-vasile/mimetype library

func ParseDocument added in v0.38.1 

func ParseDocument(reader io.Reader, mimeType string) (any, error)

ParseDocument parses a document based on its MIME type

Types

type DownloadOptions added in v0.38.1 

type DownloadOptions = storagetypes.DownloadFileOptions

Alias types from storage/types to maintain clean imports having a bunch of smaller subpackages seemed to just complicate things

type DownloadedMetadata added in v0.38.1 

type DownloadedMetadata = storagetypes.DownloadedFileMetadata

Alias types from storage/types to maintain clean imports having a bunch of smaller subpackages seemed to just complicate things

type ErrResponseHandler added in v0.38.1 

type ErrResponseHandler func(err error, statusCode int) http.HandlerFunc

ErrResponseHandler is a custom error handler for upload failures

type File added in v0.38.1 

type File = storagetypes.File

Alias types from storage/types to maintain clean imports having a bunch of smaller subpackages seemed to just complicate things

func NewUploadFile added in v0.38.1 

func NewUploadFile(path string) (*File, error)

NewUploadFile creates a new File from a file path

type FileMetadata added in v0.38.1 

type FileMetadata = storagetypes.FileMetadata

Alias types from storage/types to maintain clean imports having a bunch of smaller subpackages seemed to just complicate things

type Files added in v0.38.1 

type Files map[string][]File

Files is a map of file uploads organized by key

type NameGeneratorFunc added in v0.38.1 

type NameGeneratorFunc func(originalName string) string

NameGeneratorFunc generates names for uploaded files

type ObjectService added in v0.38.1 

type ObjectService struct {
	// contains filtered or unexported fields
}

ObjectService provides pure object management functionality without provider resolution

func NewObjectService added in v0.38.1 

func NewObjectService() *ObjectService

NewObjectService creates a new object service instance with default configuration

func (*ObjectService) Delete added in v0.38.1 

func (s *ObjectService) Delete(ctx context.Context, provider Provider, file *storagetypes.File, opts *storagetypes.DeleteFileOptions) error

Delete deletes a file using a specific storage provider client

func (*ObjectService) Download added in v0.38.1 

func (s *ObjectService) Download(ctx context.Context, provider Provider, file *storagetypes.File, opts *DownloadOptions) (*DownloadedMetadata, error)

Download downloads a file using a specific storage provider client

func (*ObjectService) ErrorResponseHandler added in v0.38.1 

func (s *ObjectService) ErrorResponseHandler() ErrResponseHandler

ErrorResponseHandler returns the configured error response handler

func (*ObjectService) GetPresignedURL added in v0.38.1 

func (s *ObjectService) GetPresignedURL(ctx context.Context, provider Provider, file *storagetypes.File, opts *storagetypes.PresignedURLOptions) (string, error)

GetPresignedURL gets a presigned URL for a file using a specific storage provider client

func (*ObjectService) IgnoreNonExistentKeys added in v0.38.1 

func (s *ObjectService) IgnoreNonExistentKeys() bool

IgnoreNonExistentKeys returns whether to ignore non-existent form keys

func (*ObjectService) Keys added in v0.38.1 

func (s *ObjectService) Keys() []string

Keys returns the configured form keys

func (*ObjectService) MaxMemory added in v0.38.1 

func (s *ObjectService) MaxMemory() int64

MaxMemory returns the configured maximum memory for multipart forms

func (*ObjectService) MaxSize added in v0.38.1 

func (s *ObjectService) MaxSize() int64

MaxSize returns the configured maximum file size

func (*ObjectService) Skipper added in v0.38.1 

func (s *ObjectService) Skipper() SkipperFunc

Skipper returns the configured skipper function

func (*ObjectService) Upload added in v0.38.1 

func (s *ObjectService) Upload(ctx context.Context, provider Provider, reader io.Reader, opts *UploadOptions) (*File, error)

Upload uploads a file using a specific storage provider client

func (*ObjectService) WithUploader added in v0.38.1 

func (s *ObjectService) WithUploader(uploader UploaderFunc) *ObjectService

WithUploader returns a new ObjectService with the specified uploader function

func (*ObjectService) WithValidation added in v0.38.1 

func (s *ObjectService) WithValidation(validationFunc ValidationFunc) *ObjectService

WithValidation returns a new ObjectService with the specified validation function

type ParentObject added in v0.38.1 

type ParentObject = storagetypes.ParentObject

Alias types from storage/types to maintain clean imports having a bunch of smaller subpackages seemed to just complicate things

type PresignMode added in v0.39.2 

type PresignMode = storagetypes.PresignMode

Alias types from storage/types to maintain clean imports having a bunch of smaller subpackages seemed to just complicate things

type Provider added in v0.38.1 

type Provider = storagetypes.Provider

Alias types from storage/types to maintain clean imports having a bunch of smaller subpackages seemed to just complicate things

type ProviderConfig added in v0.38.1 

type ProviderConfig struct {
	// Enabled indicates if object storage is enabled
	Enabled bool `json:"enabled" koanf:"enabled" default:"true"`
	// Keys are the form field keys that will be processed for uploads
	Keys []string `json:"keys" koanf:"keys" default:"[uploadFile]"`
	// MaxSizeMB is the maximum file size allowed in MB
	MaxSizeMB int64 `json:"maxSizeMB" koanf:"maxSizeMB"`
	// MaxMemoryMB is the maximum memory to use for file uploads in MB
	MaxMemoryMB int64 `json:"maxMemoryMB" koanf:"maxMemoryMB"`
	// DevMode enables simple file upload handling for local development and testing
	DevMode bool `json:"devMode" koanf:"devMode" default:"false"`
	// Providers contains configuration for each storage provider
	Providers Providers `json:"providers" koanf:"providers"`
}

ProviderConfig contains configuration for object storage providers

type ProviderConfigs added in v0.38.1 

type ProviderConfigs struct {
	// Enabled indicates if this provider is enabled
	Enabled bool `json:"enabled" koanf:"enabled" default:"false"`
	// EnsureAvailable enforces provider availability before completing server startup
	EnsureAvailable bool `json:"ensureAvailable" koanf:"ensureAvailable" default:"false"`
	// Region for cloud providers
	Region string `json:"region" koanf:"region"`
	// Bucket name for cloud providers
	Bucket string `json:"bucket" koanf:"bucket"`
	// Endpoint for custom endpoints
	Endpoint string `json:"endpoint" koanf:"endpoint"`
	// ProxyPresignEnabled toggles proxy-signed download URL generation
	ProxyPresignEnabled bool `json:"proxyPresignEnabled" koanf:"proxyPresignEnabled" default:"false"`
	// BaseURL is the prefix for proxy download URLs (e.g., http://localhost:17608/v1/files).
	BaseURL string `json:"baseURL" koanf:"baseURL" default:"http://localhost:17608/v1/files"`
	// Credentials contains the credentials for accessing the provider
	Credentials ProviderCredentials `json:"credentials" koanf:"credentials"`
}

ProviderConfigs contains configuration for all storage providers This is structured to allow easy extension for additional providers in the future

type ProviderCredentials added in v0.38.1 

type ProviderCredentials struct {
	// AccessKeyID for cloud providers
	AccessKeyID string `json:"accessKeyID" koanf:"accessKeyID" sensitive:"true"`
	// SecretAccessKey for cloud providers
	SecretAccessKey string `json:"secretAccessKey" koanf:"secretAccessKey" sensitive:"true"`
	// ProjectID for GCS
	ProjectID string `json:"projectID" koanf:"projectID" sensitive:"true"`
	// AccountID for Cloudflare R2
	AccountID string `json:"accountID" koanf:"accountID" sensitive:"true"`
	// APIToken for Cloudflare R2
	APIToken string `json:"apiToken" koanf:"apiToken" sensitive:"true"`
}

ProviderCredentials contains credentials for a storage provider

type ProviderHints added in v0.38.1 

type ProviderHints = storagetypes.ProviderHints

Alias types from storage/types to maintain clean imports having a bunch of smaller subpackages seemed to just complicate things

type ProviderOption added in v0.38.1 

type ProviderOption func(*ProviderOptions)

ProviderOption configures runtime provider options

func WithBasePath added in v0.38.1 

func WithBasePath(path string) ProviderOption

WithBasePath sets the local base path for disk providers

func WithBucket added in v0.3.1 

func WithBucket(bucket string) ProviderOption

WithBucket sets the bucket/path value

func WithCredentials added in v0.38.1 

func WithCredentials(creds ProviderCredentials) ProviderOption

WithCredentials sets provider credentials

func WithEndpoint added in v0.3.1 

func WithEndpoint(endpoint string) ProviderOption

WithEndpoint sets the custom endpoint

func WithExtra added in v0.38.1 

func WithExtra(key string, value any) ProviderOption

WithExtra attaches provider specific metadata

func WithLocalURL added in v0.6.10 

func WithLocalURL(url string) ProviderOption

WithLocalURL sets the local URL used for presigned links

func WithProxyPresignConfig added in v0.39.2 

func WithProxyPresignConfig(cfg *ProxyPresignConfig) ProviderOption

WithProxyPresignConfig sets proxy presign runtime dependencies.

func WithProxyPresignEnabled added in v0.39.2 

func WithProxyPresignEnabled(enabled bool) ProviderOption

WithProxyPresignEnabled toggles proxy URL generation.

func WithRegion added in v0.3.1 

func WithRegion(region string) ProviderOption

WithRegion sets the region value

type ProviderOptions added in v0.38.1 

type ProviderOptions struct {
	Credentials         ProviderCredentials
	Bucket              string
	Region              string
	Endpoint            string
	BasePath            string
	LocalURL            string
	ProxyPresignEnabled bool
	ProxyPresignConfig  *ProxyPresignConfig
	// contains filtered or unexported fields
}

ProviderOptions captures runtime configuration shared across storage providers

func NewProviderOptions added in v0.38.1 

func NewProviderOptions(opts ...ProviderOption) *ProviderOptions

NewProviderOptions constructs ProviderOptions applying the supplied options

func (*ProviderOptions) Apply added in v0.38.1 

func (p *ProviderOptions) Apply(opts ...ProviderOption)

Apply applies option functions to ProviderOptions

func (*ProviderOptions) Clone added in v0.38.1 

func (p *ProviderOptions) Clone() *ProviderOptions

Clone returns a deep copy of ProviderOptions

func (*ProviderOptions) Extra added in v0.38.1 

func (p *ProviderOptions) Extra(key string) (any, bool)

Extra returns provider specific metadata

type ProviderType added in v0.38.1 

type ProviderType = storagetypes.ProviderType

Alias types from storage/types to maintain clean imports having a bunch of smaller subpackages seemed to just complicate things

type Providers added in v0.38.1 

type Providers struct {
	// S3 provider configuration
	S3 ProviderConfigs `json:"s3" koanf:"s3"`
	// CloudflareR2 provider configuration
	CloudflareR2 ProviderConfigs `json:"cloudflareR2" koanf:"cloudflareR2"`
	// GCS provider configuration
	GCS ProviderConfigs `json:"gcs" koanf:"gcs"`
	// Disk provider configuration
	Disk ProviderConfigs `json:"disk" koanf:"disk"`
	// Database provider configuration
	Database ProviderConfigs `json:"database" koanf:"database"`
}

type ProxyPresignConfig added in v0.39.2 

type ProxyPresignConfig struct {
	TokenManager  *tokens.TokenManager
	TokenIssuer   string
	TokenAudience string
	BaseURL       string
}

ProxyPresignConfig carries runtime dependencies for proxy download URL generation.

func ApplyProxyPresignOptions added in v0.39.2 

func ApplyProxyPresignOptions(cfg *ProxyPresignConfig, opts ...ProxyPresignOption) *ProxyPresignConfig

ApplyProxyPresignOptions applies options to the provided config, allocating one if needed.

func NewProxyPresignConfig added in v0.39.2 

func NewProxyPresignConfig(opts ...ProxyPresignOption) *ProxyPresignConfig

NewProxyPresignConfig builds a ProxyPresignConfig applying the supplied options.

func (*ProxyPresignConfig) Apply added in v0.39.2 

func (p *ProxyPresignConfig) Apply(opts ...ProxyPresignOption) *ProxyPresignConfig

Apply applies the supplied options to the existing ProxyPresignConfig.

type ProxyPresignOption added in v0.39.2 

type ProxyPresignOption func(*ProxyPresignConfig)

ProxyPresignOption configures a ProxyPresignConfig.

func WithProxyPresignBaseURL added in v0.39.2 

func WithProxyPresignBaseURL(baseURL string) ProxyPresignOption

WithProxyPresignBaseURL sets the base URL for generated download links.

func WithProxyPresignTokenAudience added in v0.39.2 

func WithProxyPresignTokenAudience(audience string) ProxyPresignOption

WithProxyPresignTokenAudience sets the token audience when provided.

func WithProxyPresignTokenIssuer added in v0.39.2 

func WithProxyPresignTokenIssuer(issuer string) ProxyPresignOption

WithProxyPresignTokenIssuer sets the token issuer when provided.

func WithProxyPresignTokenManager added in v0.39.2 

func WithProxyPresignTokenManager(tm *tokens.TokenManager) ProxyPresignOption

WithProxyPresignTokenManager sets the token manager when provided.

type SkipperFunc added in v0.38.1 

type SkipperFunc func(r *http.Request) bool

SkipperFunc defines a function to skip middleware processing

type UploadOptions added in v0.38.1 

type UploadOptions = storagetypes.UploadFileOptions

Alias types from storage/types to maintain clean imports having a bunch of smaller subpackages seemed to just complicate things

type UploadedMetadata added in v0.38.1 

type UploadedMetadata = storagetypes.UploadedFileMetadata

Alias types from storage/types to maintain clean imports having a bunch of smaller subpackages seemed to just complicate things

type UploaderFunc added in v0.38.1 

type UploaderFunc func(ctx context.Context, service *ObjectService, files []File) ([]File, error)

UploaderFunc handles the file upload process and returns uploaded files

type ValidationFunc added in v0.38.1 

type ValidationFunc func(f File) error

ValidationFunc is a type that can be used to dynamically validate a file

Source Files

View all Source files

Directories

Path Synopsis
providers
database
disk
Package disk is the local disk storage provider for objects service
 Package disk is the local disk storage provider for objects service
r2
Package r2 is the Cloudflare R2 storage provider for objects service
 Package r2 is the Cloudflare R2 storage provider for objects service
s3
Package s3 is the AWS S3 storage provider for objects service
 Package s3 is the AWS S3 storage provider for objects service
Package proxy implements a storage proxy that provides presigned URL generation
 Package proxy implements a storage proxy that provides presigned URL generation
Package storagetypes contains types used across the storage package
 Package storagetypes contains types used across the storage package

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.