backend

package
v1.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 22, 2026 License: Apache-2.0 Imports: 33 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// this is the media type for directories in AWS and Nextcloud
	DirContentType = "application/x-directory"

	// this is the minimum allowed size for mp parts
	MinPartSize = 5 * 1024 * 1024
)

Variables

View Source 
var ErrSkipObj = errors.New("skip this object")

Functions

func AreEtagsSame added in v1.0.14 

func AreEtagsSame(e1, e2 string) bool

AreEtagsSame compares 2 etags by ignoring quotes

func CompressData added in v1.6.0 

func CompressData(data []byte) ([]byte, error)

func ComputeMultipartETagFromPartETags added in v1.8.0 

func ComputeMultipartETagFromPartETags(uploadID string, parts []types.CompletedPart) (string, error)

ComputeMultipartETagFromPartETags computes the S3 multipart ETag ("<md5>-<partCount>") from the completed-part ETags. uploadID is only used to report the upload in an InvalidPart error.

func DecompressData added in v1.6.0 

func DecompressData(data []byte) ([]byte, error)

func EvaluateObjectDeletePreconditions added in v1.0.18 

func EvaluateObjectDeletePreconditions(etag string, modTime time.Time, size int64, preconditions ObjectDeletePreconditions) error

EvaluateObjectDeletePreconditions evaluates the DeleteObject conditions against an existing object. If-Match "*" matches any object.

func EvaluateObjectPutPreconditions added in v1.1.0 

func EvaluateObjectPutPreconditions(etag string, ifMatch, ifNoneMatch *string, objExists bool) error

EvaluateObjectPutPreconditions evaluates if-match and if-none-match preconditions for object PUT(PutObject, CompleteMultipartUpload, CopyObject) actions. The unsupported header combinations and values are rejected first, in this order, whether or not the object exists.

func EvaluatePreconditions added in v1.0.18 

func EvaluatePreconditions(etag string, modTime time.Time, preconditions PreConditions) error

EvaluatePreconditions takes the object ETag, the last modified time and evaluates the read preconditions: - if-match, - if-none-match - if-modified-since - if-unmodified-since if-match and if-none-match are ETag comparisons if-match "*" matches any ETag, and if-none-match "*" matches no ETag if-modified-since and if-unmodified-since are last modifed time comparisons

func GenerateEtag added in v1.0.14 

func GenerateEtag(h hash.Hash) string

GenerateEtag generates a new quoted etag from the provided hash.Hash

func GetPtrFromString added in v1.0.8 

func GetPtrFromString(str string) *string

func GetStringFromPtr added in v1.0.9 

func GetStringFromPtr(str *string) string

func GetTimePtr 

func GetTimePtr(t time.Time) *time.Time

func HasSSEC added in v1.9.0 

func HasSSEC(vals ...*string) bool

HasSSEC reports whether any of the given SSE-C (customer-provided key) values are set.

func IsPermissionErr added in v1.9.0 

func IsPermissionErr(err error) bool

IsPermissionErr reports whether err is a filesystem permission failure, EACCES or EPERM. Windows ERROR_ACCESS_DENIED is left out: Windows also returns it for transient states, such as a name whose delete is still pending.

func IsValidBucketName 

func IsValidBucketName(name string) bool

func IsValidDirectoryName added in v1.0.19 

func IsValidDirectoryName(name string) bool

IsValidDirectoryName returns true if the string is a valid name for a directory

func MapPermissionErr added in v1.9.0 

func MapPermissionErr(err *error)

MapPermissionErr replaces a filesystem permission failure in *err with AccessDenied; filesystem backend methods that write defer it. The gateway process lacking a permission is not transient: a retry fails the same way, while the InternalError it would otherwise become has S3 clients retry it several times before giving up. The cause is logged at debug level, since the gateway's filesystem permissions denied the request, not its authorization.

func MarshalMpUploadMetadata added in v1.5.0 

func MarshalMpUploadMetadata(mpMeta MpUploadMetadata, base64Encode bool) ([]byte, error)

MarshalMpUploadMetadata returns a compressed representation of multipart metadata. When base64Encode is true, the compressed bytes are base64-encoded so they can be sent in azure string-only metadata headers

func MarshalWebsiteConfig added in v1.6.0 

func MarshalWebsiteConfig(website []byte, base64Encode bool) ([]byte, error)

MarshalWebsiteConfig returns a compressed representation of a website configuration. When base64Encode is true, the compressed bytes are base64-encoded so they can be stored in azure string-only metadata values.

func MkdirAll 

func MkdirAll(path string, uid, gid int, doChown bool, dirPerm fs.FileMode) error

MkdirAll is similar to os.MkdirAll but it will return ErrObjectParentIsFile when appropriate MkdirAll creates a directory named path, along with any necessary parents, and returns nil, or else returns an error. The permission bits perm (before umask) are used for all directories that MkdirAll creates. Any newly created directory is set to provided uid/gid ownership. If path is already a directory, MkdirAll does nothing and returns nil. Any directory created will be set to provided uid/gid ownership if doChown is true.

Unlike os.MkdirAll it tolerates directories being removed while it runs: the implicit parent directories of objects are removed by deletes of other objects under them, without any coordination with the writers that make them.

func MoveFile added in v1.0.13 

func MoveFile(source, destination string, perm os.FileMode) error

MoveFile moves a file from source to destination.

func ParseCopySource added in v1.0.8 

func ParseCopySource(copySourceHeader string) (string, string, string, error)

ParseCopySource parses x-amz-copy-source header and returns source bucket, source object, versionId, error respectively

func ParseCopySourceRange added in v1.0.11 

func ParseCopySourceRange(size int64, acceptRange string) (int64, int64, error)

ParseCopySourceRange parses input range header and returns startoffset, length and error. If no endoffset specified, then length is set to the object size

func ParseCreateBucketTags added in v1.0.19 

func ParseCreateBucketTags(tagging []types.Tag) (map[string]string, error)

ParseCreateBucketTags parses and validates the bucket tagging from CreateBucket input

func ParseObjectRange added in v1.0.14 

func ParseObjectRange(size int64, acceptRange string) (int64, int64, bool, error)

ParseObjectRange parses input range header and returns startoffset, length, isValid and error. If no endoffset specified, then length is set to the object size for invalid inputs, it returns isValid=false and error=nil `InvalidRange` error is returned only if startoffset is greater than the object size

func ParseObjectTags added in v1.0.12 

func ParseObjectTags(tagging string) (map[string]string, error)

ParseObjectTags parses the url encoded input string into map[string]string with unescaped key/value pair

func ParseTagSet added in v1.9.0 

func ParseTagSet(tags []types.Tag, limit int, limitErr s3err.ErrorCode) (map[string]string, error)

ParseTagSet validates the tags of an XML tag set (Tagging or CreateBucketConfiguration) and returns them as a key/value map. limitErr is returned when there are more than limit tags.

func TrimEtag added in v1.0.11 

func TrimEtag(etag *string) *string

func UnmarshalWebsiteConfig added in v1.6.0 

func UnmarshalWebsiteConfig(data []byte, base64Decode bool) ([]byte, error)

UnmarshalWebsiteConfig decodes data produced by MarshalWebsiteConfig.

func UploadPartHasIntegrityCheck added in v1.9.0 

func UploadPartHasIntegrityCheck(input *s3.UploadPartInput) bool

UploadPartHasIntegrityCheck reports whether a part upload carries an integrity check of its body: Content-MD5, an x-amz-checksum-* header, or an x-amz-checksum-* trailer, which the body reader then carries. A part of a multipart upload with Object Lock parameters must carry one.

func ValidateObjectPutPreconditionHeaders added in v1.10.0 

func ValidateObjectPutPreconditionHeaders(ifMatch, ifNoneMatch *string) error

ValidateObjectPutPreconditionHeaders rejects the if-match and if-none-match header combinations and values an object PUT doesn't support. It depends on the headers alone.

func ValidateUUIDUploadID added in v1.10.0 

func ValidateUUIDUploadID(uploadID string) error

ValidateUUIDUploadID returns NoSuchUpload for an upload ID that isn't a UUID in canonical form, the form uuid.New().String() generates.

Types

type Backend 

type Backend interface {
	fmt.Stringer
	Shutdown()
	NormalizeObjectKey(bucket, object string) string
	StoresDotSegmentKeys() bool
	ValidateUploadID(uploadID string) error

	// bucket operations
	ListBuckets(context.Context, s3response.ListBucketsInput) (s3response.ListAllMyBucketsResult, error)
	HeadBucket(context.Context, *s3.HeadBucketInput) (*s3.HeadBucketOutput, error)
	GetBucketAcl(context.Context, *s3.GetBucketAclInput) ([]byte, error)
	CreateBucket(_ context.Context, _ *s3.CreateBucketInput, defaultACL []byte) error
	PutBucketAcl(_ context.Context, bucket string, data []byte) error
	DeleteBucket(_ context.Context, bucket string) error
	PutBucketVersioning(_ context.Context, bucket string, status types.BucketVersioningStatus) error
	GetBucketVersioning(_ context.Context, bucket string) (s3response.GetBucketVersioningOutput, error)
	PutBucketPolicy(_ context.Context, bucket string, policy []byte) error
	GetBucketPolicy(_ context.Context, bucket string) ([]byte, error)
	DeleteBucketPolicy(_ context.Context, bucket string) error
	PutBucketOwnershipControls(_ context.Context, bucket string, ownership types.ObjectOwnership) error
	GetBucketOwnershipControls(_ context.Context, bucket string) (types.ObjectOwnership, error)
	DeleteBucketOwnershipControls(_ context.Context, bucket string) error
	PutBucketCors(_ context.Context, bucket string, cors []byte) error
	GetBucketCors(_ context.Context, bucket string) ([]byte, error)
	DeleteBucketCors(_ context.Context, bucket string) error
	PutBucketWebsite(_ context.Context, bucket string, website []byte) error
	GetBucketWebsite(_ context.Context, bucket string) ([]byte, error)
	DeleteBucketWebsite(_ context.Context, bucket string) error

	// multipart operations
	CreateMultipartUpload(context.Context, s3response.CreateMultipartUploadInput) (s3response.InitiateMultipartUploadResult, error)
	CompleteMultipartUpload(context.Context, *s3.CompleteMultipartUploadInput) (_ s3response.CompleteMultipartUploadResult, versionid string, _ error)
	AbortMultipartUpload(context.Context, *s3.AbortMultipartUploadInput) error
	ListMultipartUploads(context.Context, *s3.ListMultipartUploadsInput) (s3response.ListMultipartUploadsResult, error)
	ListParts(context.Context, *s3.ListPartsInput) (s3response.ListPartsResult, error)
	UploadPart(context.Context, *s3.UploadPartInput) (*s3.UploadPartOutput, error)
	UploadPartCopy(context.Context, *s3.UploadPartCopyInput) (s3response.CopyPartResult, error)

	// standard object operations
	PutObject(context.Context, s3response.PutObjectInput) (s3response.PutObjectOutput, error)
	HeadObject(context.Context, *s3.HeadObjectInput) (*s3.HeadObjectOutput, error)
	GetObject(context.Context, *s3.GetObjectInput) (*s3.GetObjectOutput, error)
	GetObjectAcl(context.Context, *s3.GetObjectAclInput) (*s3.GetObjectAclOutput, error)
	GetObjectAttributes(context.Context, *s3.GetObjectAttributesInput) (s3response.GetObjectAttributesResponse, error)
	CopyObject(context.Context, s3response.CopyObjectInput) (s3response.CopyObjectOutput, error)
	ListObjects(context.Context, *s3.ListObjectsInput) (s3response.ListObjectsResult, error)
	ListObjectsV2(context.Context, *s3.ListObjectsV2Input) (s3response.ListObjectsV2Result, error)
	DeleteObject(context.Context, *s3.DeleteObjectInput) (*s3.DeleteObjectOutput, error)
	DeleteObjects(context.Context, *s3.DeleteObjectsInput) (s3response.DeleteResult, error)
	PutObjectAcl(context.Context, *s3.PutObjectAclInput) error
	ListObjectVersions(context.Context, *s3.ListObjectVersionsInput) (s3response.ListVersionsResult, error)

	// special case object operations
	RestoreObject(context.Context, *s3.RestoreObjectInput) error
	SelectObjectContent(ctx context.Context, input *s3.SelectObjectContentInput) func(w *bufio.Writer)

	// bucket tagging operations
	GetBucketTagging(_ context.Context, bucket string) (map[string]string, error)
	PutBucketTagging(_ context.Context, bucket string, tags map[string]string) error
	DeleteBucketTagging(_ context.Context, bucket string) error

	// object tagging operations
	GetObjectTagging(_ context.Context, bucket, object, versionId string) (map[string]string, error)
	PutObjectTagging(_ context.Context, bucket, object, versionId string, tags map[string]string) error
	DeleteObjectTagging(_ context.Context, bucket, object, versionId string) error

	// object lock operations
	PutObjectLockConfiguration(_ context.Context, bucket string, config []byte) error
	GetObjectLockConfiguration(_ context.Context, bucket string) ([]byte, error)
	PutObjectRetention(_ context.Context, bucket, object, versionId string, retention []byte) error
	GetObjectRetention(_ context.Context, bucket, object, versionId string) ([]byte, error)
	PutObjectLegalHold(_ context.Context, bucket, object, versionId string, status bool) error
	GetObjectLegalHold(_ context.Context, bucket, object, versionId string) (*bool, error)

	// non AWS actions
	ChangeBucketOwner(_ context.Context, bucket, owner string) error
	ListBucketsAndOwners(context.Context) ([]s3response.Bucket, error)
}

func New 

func New() Backend

type BackendUnsupported 

type BackendUnsupported struct{}

func (BackendUnsupported) AbortMultipartUpload 

func (BackendUnsupported) AbortMultipartUpload(context.Context, *s3.AbortMultipartUploadInput) error

func (BackendUnsupported) ChangeBucketOwner 

func (BackendUnsupported) ChangeBucketOwner(_ context.Context, bucket, owner string) error

func (BackendUnsupported) CompleteMultipartUpload 

func (BackendUnsupported) CompleteMultipartUpload(context.Context, *s3.CompleteMultipartUploadInput) (s3response.CompleteMultipartUploadResult, string, error)

func (BackendUnsupported) CopyObject 

func (BackendUnsupported) CopyObject(context.Context, s3response.CopyObjectInput) (s3response.CopyObjectOutput, error)

func (BackendUnsupported) CreateBucket 

func (BackendUnsupported) CreateBucket(context.Context, *s3.CreateBucketInput, []byte) error

func (BackendUnsupported) CreateMultipartUpload 

func (BackendUnsupported) CreateMultipartUpload(context.Context, s3response.CreateMultipartUploadInput) (s3response.InitiateMultipartUploadResult, error)

func (BackendUnsupported) DeleteBucket 

func (BackendUnsupported) DeleteBucket(_ context.Context, bucket string) error

func (BackendUnsupported) DeleteBucketCors added in v1.0.13 

func (BackendUnsupported) DeleteBucketCors(_ context.Context, bucket string) error

func (BackendUnsupported) DeleteBucketOwnershipControls added in v1.0.3 

func (BackendUnsupported) DeleteBucketOwnershipControls(_ context.Context, bucket string) error

func (BackendUnsupported) DeleteBucketPolicy 

func (BackendUnsupported) DeleteBucketPolicy(_ context.Context, bucket string) error

func (BackendUnsupported) DeleteBucketTagging 

func (BackendUnsupported) DeleteBucketTagging(_ context.Context, bucket string) error

func (BackendUnsupported) DeleteBucketWebsite added in v1.6.0 

func (BackendUnsupported) DeleteBucketWebsite(_ context.Context, bucket string) error

func (BackendUnsupported) DeleteObject 

func (BackendUnsupported) DeleteObject(context.Context, *s3.DeleteObjectInput) (*s3.DeleteObjectOutput, error)

func (BackendUnsupported) DeleteObjectTagging 

func (BackendUnsupported) DeleteObjectTagging(_ context.Context, bucket, object, versionId string) error

func (BackendUnsupported) DeleteObjects 

func (BackendUnsupported) DeleteObjects(context.Context, *s3.DeleteObjectsInput) (s3response.DeleteResult, error)

func (BackendUnsupported) GetBucketAcl 

func (BackendUnsupported) GetBucketAcl(context.Context, *s3.GetBucketAclInput) ([]byte, error)

func (BackendUnsupported) GetBucketCors added in v1.0.13 

func (BackendUnsupported) GetBucketCors(_ context.Context, bucket string) ([]byte, error)

func (BackendUnsupported) GetBucketOwnershipControls added in v1.0.3 

func (BackendUnsupported) GetBucketOwnershipControls(_ context.Context, bucket string) (types.ObjectOwnership, error)

func (BackendUnsupported) GetBucketPolicy 

func (BackendUnsupported) GetBucketPolicy(_ context.Context, bucket string) ([]byte, error)

func (BackendUnsupported) GetBucketTagging 

func (BackendUnsupported) GetBucketTagging(_ context.Context, bucket string) (map[string]string, error)

func (BackendUnsupported) GetBucketVersioning 

func (BackendUnsupported) GetBucketVersioning(_ context.Context, bucket string) (s3response.GetBucketVersioningOutput, error)

func (BackendUnsupported) GetBucketWebsite added in v1.6.0 

func (BackendUnsupported) GetBucketWebsite(_ context.Context, bucket string) ([]byte, error)

func (BackendUnsupported) GetObject 

func (BackendUnsupported) GetObject(context.Context, *s3.GetObjectInput) (*s3.GetObjectOutput, error)

func (BackendUnsupported) GetObjectAcl 

func (BackendUnsupported) GetObjectAcl(context.Context, *s3.GetObjectAclInput) (*s3.GetObjectAclOutput, error)

func (BackendUnsupported) GetObjectAttributes 

func (BackendUnsupported) GetObjectAttributes(context.Context, *s3.GetObjectAttributesInput) (s3response.GetObjectAttributesResponse, error)

func (BackendUnsupported) GetObjectLegalHold 

func (BackendUnsupported) GetObjectLegalHold(_ context.Context, bucket, object, versionId string) (*bool, error)

func (BackendUnsupported) GetObjectLockConfiguration 

func (BackendUnsupported) GetObjectLockConfiguration(_ context.Context, bucket string) ([]byte, error)

func (BackendUnsupported) GetObjectRetention 

func (BackendUnsupported) GetObjectRetention(_ context.Context, bucket, object, versionId string) ([]byte, error)

func (BackendUnsupported) GetObjectTagging 

func (BackendUnsupported) GetObjectTagging(_ context.Context, bucket, object, versionId string) (map[string]string, error)

func (BackendUnsupported) HeadBucket 

func (BackendUnsupported) HeadBucket(context.Context, *s3.HeadBucketInput) (*s3.HeadBucketOutput, error)

func (BackendUnsupported) HeadObject 

func (BackendUnsupported) HeadObject(context.Context, *s3.HeadObjectInput) (*s3.HeadObjectOutput, error)

func (BackendUnsupported) ListBuckets 

func (BackendUnsupported) ListBuckets(context.Context, s3response.ListBucketsInput) (s3response.ListAllMyBucketsResult, error)

func (BackendUnsupported) ListBucketsAndOwners 

func (BackendUnsupported) ListBucketsAndOwners(context.Context) ([]s3response.Bucket, error)

func (BackendUnsupported) ListMultipartUploads 

func (BackendUnsupported) ListMultipartUploads(context.Context, *s3.ListMultipartUploadsInput) (s3response.ListMultipartUploadsResult, error)

func (BackendUnsupported) ListObjectVersions 

func (BackendUnsupported) ListObjectVersions(context.Context, *s3.ListObjectVersionsInput) (s3response.ListVersionsResult, error)

func (BackendUnsupported) ListObjects 

func (BackendUnsupported) ListObjects(context.Context, *s3.ListObjectsInput) (s3response.ListObjectsResult, error)

func (BackendUnsupported) ListObjectsV2 

func (BackendUnsupported) ListObjectsV2(context.Context, *s3.ListObjectsV2Input) (s3response.ListObjectsV2Result, error)

func (BackendUnsupported) ListParts 

func (BackendUnsupported) ListParts(context.Context, *s3.ListPartsInput) (s3response.ListPartsResult, error)

func (BackendUnsupported) NormalizeObjectKey added in v1.5.0 

func (BackendUnsupported) NormalizeObjectKey(_, object string) string

func (BackendUnsupported) PutBucketAcl 

func (BackendUnsupported) PutBucketAcl(_ context.Context, bucket string, data []byte) error

func (BackendUnsupported) PutBucketCors added in v1.0.13 

func (BackendUnsupported) PutBucketCors(context.Context, string, []byte) error

func (BackendUnsupported) PutBucketOwnershipControls added in v1.0.3 

func (BackendUnsupported) PutBucketOwnershipControls(_ context.Context, bucket string, ownership types.ObjectOwnership) error

func (BackendUnsupported) PutBucketPolicy 

func (BackendUnsupported) PutBucketPolicy(_ context.Context, bucket string, policy []byte) error

func (BackendUnsupported) PutBucketTagging 

func (BackendUnsupported) PutBucketTagging(_ context.Context, bucket string, tags map[string]string) error

func (BackendUnsupported) PutBucketVersioning 

func (BackendUnsupported) PutBucketVersioning(_ context.Context, bucket string, status types.BucketVersioningStatus) error

func (BackendUnsupported) PutBucketWebsite added in v1.6.0 

func (BackendUnsupported) PutBucketWebsite(_ context.Context, bucket string, website []byte) error

func (BackendUnsupported) PutObject 

func (BackendUnsupported) PutObject(context.Context, s3response.PutObjectInput) (s3response.PutObjectOutput, error)

func (BackendUnsupported) PutObjectAcl 

func (BackendUnsupported) PutObjectAcl(context.Context, *s3.PutObjectAclInput) error

func (BackendUnsupported) PutObjectLegalHold 

func (BackendUnsupported) PutObjectLegalHold(_ context.Context, bucket, object, versionId string, status bool) error

func (BackendUnsupported) PutObjectLockConfiguration 

func (BackendUnsupported) PutObjectLockConfiguration(_ context.Context, bucket string, config []byte) error

func (BackendUnsupported) PutObjectRetention 

func (BackendUnsupported) PutObjectRetention(_ context.Context, bucket, object, versionId string, retention []byte) error

func (BackendUnsupported) PutObjectTagging 

func (BackendUnsupported) PutObjectTagging(_ context.Context, bucket, object, versionId string, tags map[string]string) error

func (BackendUnsupported) RestoreObject 

func (BackendUnsupported) RestoreObject(context.Context, *s3.RestoreObjectInput) error

func (BackendUnsupported) SelectObjectContent 

func (BackendUnsupported) SelectObjectContent(ctx context.Context, input *s3.SelectObjectContentInput) func(w *bufio.Writer)

func (BackendUnsupported) Shutdown 

func (BackendUnsupported) Shutdown()

func (BackendUnsupported) StoresDotSegmentKeys added in v1.10.0 

func (BackendUnsupported) StoresDotSegmentKeys() bool

func (BackendUnsupported) String 

func (BackendUnsupported) String() string

func (BackendUnsupported) UploadPart 

func (BackendUnsupported) UploadPart(context.Context, *s3.UploadPartInput) (*s3.UploadPartOutput, error)

func (BackendUnsupported) UploadPartCopy 

func (BackendUnsupported) UploadPartCopy(context.Context, *s3.UploadPartCopyInput) (s3response.CopyPartResult, error)

func (BackendUnsupported) ValidateUploadID added in v1.10.0 

func (BackendUnsupported) ValidateUploadID(string) error

type ByBucketName 

type ByBucketName []s3response.ListAllMyBucketsEntry

func (ByBucketName) Len 

func (d ByBucketName) Len() int

func (ByBucketName) Less 

func (d ByBucketName) Less(i, j int) bool

func (ByBucketName) Swap 

func (d ByBucketName) Swap(i, j int)

type ByObjectName 

type ByObjectName []types.Object

func (ByObjectName) Len 

func (d ByObjectName) Len() int

func (ByObjectName) Less 

func (d ByObjectName) Less(i, j int) bool

func (ByObjectName) Swap 

func (d ByObjectName) Swap(i, j int)

type FileSectionReadCloser added in v1.0.3 

type FileSectionReadCloser struct {
	R io.Reader
	F *os.File
}

func (*FileSectionReadCloser) Close added in v1.0.3 

func (f *FileSectionReadCloser) Close() error

func (*FileSectionReadCloser) Read added in v1.0.3 

func (f *FileSectionReadCloser) Read(p []byte) (int, error)

type GetObjFunc 

type GetObjFunc func(path string, d fs.DirEntry) (s3response.Object, error)

type GetVersionsFunc added in v1.0.8 

type GetVersionsFunc func(path, versionIdMarker string, pastVersionIdMarker *bool, availableObjCount int, d fs.DirEntry) (*ObjVersionFuncResult, error)

type ListMultipartUploadsPage added in v1.2.0 

type ListMultipartUploadsPage struct {
	Uploads            []s3response.Upload
	CommonPrefixes     []s3response.CommonPrefix
	IsTruncated        bool
	NextKeyMarker      string
	NextUploadIDMarker string
}

ListMultipartUploadsPage is the lister output

func ListMultipartUploads added in v1.2.0 

func ListMultipartUploads(uploads []s3response.Upload, prefix, delimiter, keyMarker, uploadIdMarker string, maxUploads int) (*ListMultipartUploadsPage, error)

ListMultipartUploads initializes a multipart upload lister and calls Run()

type MpUploadMetadata added in v1.4.1 

type MpUploadMetadata struct {
	UploadID string  `json:"uploadId"`
	Parts    []int64 `json:"parts"`
}

MpUploadMetadata is stored alongside the final object after a multipart upload completes. It records the uploadId and the cumulative byte offsets of each part: parts[i] = sum of sizes of parts 1..i+1. This allows O(1) total size lookup (parts[last]) and O(1) part-start/size derivation for GetObject/HeadObject part-number requests.

func UnmarshalMpUploadMetadata added in v1.5.0 

func UnmarshalMpUploadMetadata(data []byte, base64Decode bool) (MpUploadMetadata, error)

UnmarshalMpUploadMetadata decodes metadata produced by MarshalMpUploadMetadata. It also accepts the legacy raw JSON form so existing multipart objects remain readable

type MultipartUploadLister added in v1.2.0 

type MultipartUploadLister struct {
	Uploads        []s3response.Upload
	Prefix         string
	Delimiter      string
	MaxUploads     int
	KeyMarker      string
	UploadIDMarker string
}

MultipartUploadLister emits a ListMultipartUploads-compatible page from an already-sorted, already prefix- and key-marker-filtered upload list.

Assumptions about input Uploads:

  • Sorted by (Key asc, Initiated asc)
  • Filtered by Prefix
  • Filtered to start strictly after key-marker when key-marker was provided, or at key-marker when upload-id-marker was provided as well.

func (*MultipartUploadLister) Run added in v1.2.0 

func (l *MultipartUploadLister) Run() (*ListMultipartUploadsPage, error)

Run validates marker constraints, then performs a single-pass list that: - collapses uploads into CommonPrefixes when delimiter is set - enforces MaxUploads over (Uploads + CommonPrefixes) - computes truncation and next markers

type ObjVersionFuncResult added in v1.0.8 

type ObjVersionFuncResult struct {
	ObjectVersions      []s3response.ObjectVersion
	DelMarkers          []s3response.DeleteMarkerEntry
	NextVersionIdMarker string
	Truncated           bool
}

type ObjectDeletePreconditions added in v1.0.18 

type ObjectDeletePreconditions struct {
	IfMatch            *string
	IfMatchLastModTime *time.Time
	IfMatchSize        *int64
}

func (ObjectDeletePreconditions) IsSet added in v1.9.0 

func (p ObjectDeletePreconditions) IsSet() bool

IsSet reports whether the delete has any condition. A conditional delete of a key with no current version, including one whose current version is a delete marker, fails with NoSuchKey instead of succeeding.

type PreConditions added in v1.0.18 

type PreConditions struct {
	IfMatch       *string
	IfNoneMatch   *string
	IfModSince    *time.Time
	IfUnmodeSince *time.Time
}

type SkipDirs added in v1.10.0 

type SkipDirs struct {
	// Root names directories skipped only at the top of the walked file
	// system, such as the gateway's own metadata directory.
	Root []string
	// Any names directories skipped at every depth.
	Any []string
}

SkipDirs names the directories a walk never descends into or reports as objects. Regular files with these names are ordinary objects. Names are matched ignoring case: a case-insensitive file system resolves a name in any case to the same directory, and matching the same way on every file system keeps the listings in agreement with backends that reject object keys through these directories ignoring case.

type WalkDirFunc added in v1.2.0 

type WalkDirFunc func(path string, d fs.DirEntry, err error) error

WalkDirFunc is the type of the function called by [walkDirRoot] to visit each file or directory.

The path argument contains the argument to [walkDirRoot] as a prefix. That is, if WalkDir is called with root argument "dir" and finds a file named "a" in that directory, the walk function will be called with argument "dir/a".

The d argument is the [DirEntry] for the named path.

The error result returned by the function controls how [walkDirRoot] continues. If the function returns the special value [SkipDir], WalkDir skips the current directory (path if d.IsDir() is true, otherwise path's parent directory). If the function returns the special value [SkipAll], WalkDir skips all remaining files and directories. Otherwise, if the function returns a non-nil error, WalkDir stops entirely and returns that error.

The err argument reports an error related to path, signaling that [walkDirRoot] will not walk into that directory. The function can decide how to handle that error; as described earlier, returning the error will cause WalkDir to stop walking the entire tree.

[walkDirRoot] calls the function with a non-nil err argument in two cases.

First, if the initial [Stat] on the root directory fails, WalkDir calls the function with path set to root, d set to nil, and err set to the error from fs.Stat.

Second, if a directory's ReadDir method (see [ReadDirFile]) fails, WalkDir calls the function with path set to the directory's path, d set to an [DirEntry] describing the directory, and err set to the error from ReadDir. In this second case, the function is called twice with the path of the directory: the first call is before the directory read is attempted and has err set to nil, giving the function a chance to return [SkipDir] or [SkipAll] and avoid the ReadDir entirely. The second call is after a failed ReadDir and reports the error from ReadDir. (If ReadDir succeeds, there is no second call.)

The differences between WalkDirFunc compared to path/filepath.WalkFunc are:

  • The second argument has type [DirEntry] instead of [FileInfo].
  • The function is called before reading a directory, to allow [SkipDir] or [SkipAll] to bypass the directory read entirely or skip all remaining files and directories respectively.
  • If a directory read fails, the function is called a second time for that directory to report the error.

type WalkResults 

type WalkResults struct {
	CommonPrefixes []types.CommonPrefix
	Objects        []s3response.Object
	Truncated      bool
	NextMarker     string
}

func Walk 

func Walk(ctx context.Context, fileSystem fs.FS, prefix, delimiter, marker string, max int32, getObj GetObjFunc, skipDirs SkipDirs) (WalkResults, error)

Walk walks the supplied fs.FS and returns results compatible with list objects responses

type WalkVersioningResults added in v1.0.8 

type WalkVersioningResults struct {
	CommonPrefixes      []types.CommonPrefix
	ObjectVersions      []s3response.ObjectVersion
	DelMarkers          []s3response.DeleteMarkerEntry
	Truncated           bool
	NextMarker          string
	NextVersionIdMarker string
}

func WalkVersions added in v1.0.8 

func WalkVersions(ctx context.Context, fileSystem fs.FS, prefix, delimiter, keyMarker, versionIdMarker string, max int, getObj GetVersionsFunc, skipDirs SkipDirs) (WalkVersioningResults, error)

WalkVersions walks the supplied fs.FS and returns results compatible with ListObjectVersions action response

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.