utils

package
v1.0.19 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 20, 2025 License: Apache-2.0 Imports: 37 Imported by: 0

Documentation

Overview

code modified from golang std library src/internal/filepathlite/path.go to support path separator '/' for all platforms.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AddCRCChecksum added in v1.0.15 

func AddCRCChecksum(algo types.ChecksumAlgorithm, crc, partCrc string, partLen int64) (string, error)

AddCRCChecksum calculates the composite CRC checksum after adding the part crc. Only CRC32, CRC32C, and CRC64NVME are supported. The input checksums must be base64-encoded strings.

func ApplyOverride added in v1.0.18 

func ApplyOverride(original, override *string) *string

ApplyOverride returns the override value if it exists and status is 200, otherwise returns original

func Base64SumString added in v1.0.11 

func Base64SumString(b []byte) string

Base64SumString converts the hash bytes to the b64 encoded string checksum value

func CheckPresignedSignature 

func CheckPresignedSignature(ctx *fiber.Ctx, auth AuthData, secret string, streamBody bool) error

CheckPresignedSignature validates presigned request signature

func CheckValidSignature 

func CheckValidSignature(ctx *fiber.Ctx, auth AuthData, secret, checksum string, tdate time.Time, contentLen int64, streamBody bool) error

CheckValidSignature validates the ctx v4 auth signature

func ConvertPtrToStringPtr added in v1.0.17 

func ConvertPtrToStringPtr[T any](val *T) *string

Converst any pointer to a string pointer

func ConvertToStringPtr added in v1.0.17 

func ConvertToStringPtr[T any](val T) *string

Converts any type to a string pointer

func ExtractChecksumType added in v1.0.15 

func ExtractChecksumType(ctx *fiber.Ctx) (checksumType, error)

Extracts and validates the checksum type from the 'X-Amz-Trailer' header

func FilterObjectAttributes 

func FilterObjectAttributes(attrs map[s3response.ObjectAttributes]struct{}, output s3response.GetObjectAttributesResponse) s3response.GetObjectAttributesResponse

func FormatDatePtrToString added in v1.0.17 

func FormatDatePtrToString(date *time.Time, format string) *string

Formats the date with the given formatting and returns a string pointer

func FromSlash added in v1.0.15 

func FromSlash(path string) string

func GetInt64 added in v1.0.17 

func GetInt64(n *int64) int64

GetInt64 returns the value of int64 pointer

func GetQueryParam added in v1.0.18 

func GetQueryParam(ctx *fiber.Ctx, key string) *string

GetQueryParam returns a pointer to the query parameter value if it exists

func GetStringPtr added in v1.0.17 

func GetStringPtr(str string) *string

Returns the provided string pointer

func GetUserMetaData 

func GetUserMetaData(headers *fasthttp.RequestHeader) (metadata map[string]string)

func IsAnonymousPayloadHashSupported added in v1.0.19 

func IsAnonymousPayloadHashSupported(hash string) error

IsAnonymousPayloadHashSupported returns error if payload hash is streaming signed. e.g. "STREAMING-AWS4-HMAC-SHA256-PAYLOAD", "STREAMING-AWS4-ECDSA-P256-SHA256-PAYLOAD" ...

func IsChecksumAlgorithmValid added in v1.0.11 

func IsChecksumAlgorithmValid(alg types.ChecksumAlgorithm) error

func IsChecksumComposable added in v1.0.15 

func IsChecksumComposable(algo types.ChecksumAlgorithm) bool

IsChecksumComposable tests if the final foll object crc can be calculated based on the part crc values.

func IsChecksumTypeValid added in v1.0.11 

func IsChecksumTypeValid(t types.ChecksumType) error

Validates the provided checksum type

func IsObjectNameValid added in v1.0.15 

func IsObjectNameValid(name string) bool

func IsPresignedURLAuth added in v1.0.19 

func IsPresignedURLAuth(ctx *fiber.Ctx) bool

IsPresignedURLAuth determines if the request is presigned: which is authorization with query params

func IsSpecialPayload 

func IsSpecialPayload(str string) bool

IsSpecialPayload checks for special authorization types

func IsStreamingPayload added in v1.0.11 

func IsStreamingPayload(str string) bool

IsChunkEncoding checks for streaming/unsigned authorization types

func IsUnsignedPaylod added in v1.0.19 

func IsUnsignedPaylod(hash string) bool

IsUnsignedPaylod checks if the provided payload hash type is "UNSIGNED-PAYLOAD"

func IsUnsignedStreamingPayload added in v1.0.15 

func IsUnsignedStreamingPayload(str string) bool

Checks if the provided string is unsigned payload trailer type

func IsValidBucketName 

func IsValidBucketName(bucket string) bool

func IsValidChecksum added in v1.0.11 

func IsValidChecksum(checksum string, algorithm types.ChecksumAlgorithm) bool

func IsValidOwnership added in v1.0.3 

func IsValidOwnership(val types.ObjectOwnership) bool

func IsValidSh256PayloadHeader added in v1.0.17 

func IsValidSh256PayloadHeader(value string) bool

IsValidSh256PayloadHeader checks if the provided x-amz-content-sha256 paylod header is valid special paylod type or a valid sh256 hash

func NewChunkReader 

func NewChunkReader(ctx *fiber.Ctx, r io.Reader, authdata AuthData, region, secret string, date time.Time) (io.Reader, error)

func NewSignedChunkReader added in v1.0.10 

func NewSignedChunkReader(r io.Reader, authdata AuthData, region, secret string, date time.Time, chType checksumType) (io.Reader, error)

NewChunkReader reads from request body io.Reader and parses out the chunk metadata in stream. The headers are validated for proper signatures. Reading from the chunk reader will read only the object data stream without the chunk headers/trailers.

func ParsObjectLockHdrs 

func ParsObjectLockHdrs(ctx *fiber.Ctx) (*objLockCfg, error)

func ParseCreateMpChecksumHeaders added in v1.0.11 

func ParseCreateMpChecksumHeaders(ctx *fiber.Ctx) (types.ChecksumAlgorithm, types.ChecksumType, error)

Parses and validates the x-amz-checksum-algorithm and x-amz-checksum-type headers

func ParseDeleteObjects 

func ParseDeleteObjects(objs []types.ObjectIdentifier) (result []string)

func ParseIfMatchSize added in v1.0.18 

func ParseIfMatchSize(ctx *fiber.Ctx) *int64

ParseIfMatchSize parses the 'x-amz-if-match-size' to *int64 if parsing fails, returns nil

func ParseObjectAttributes 

func ParseObjectAttributes(ctx *fiber.Ctx) (map[s3response.ObjectAttributes]struct{}, error)

func ParsePreconditionDateHeader added in v1.0.18 

func ParsePreconditionDateHeader(date string) *time.Time

ParsePreconditionDateHeader tries to parse the given date string as - RFC1123 - RFC3339 both are valid

func ParsePreconditionDateHeaders added in v1.0.18 

func ParsePreconditionDateHeaders(ctx *fiber.Ctx, opts ...preconditionOpt) (*time.Time, *time.Time)

ParsePreconditionDateHeaders parses the "If-Modified-Since" and "If-Unmodified-Since" headers from fiber context to *time.Time

func ParsePreconditionMatchHeaders added in v1.0.18 

func ParsePreconditionMatchHeaders(ctx *fiber.Ctx, opts ...preconditionOpt) (*string, *string)

ParsePreconditionMatchHeaders extracts "If-Match" and "If-None-Match" headers from fiber Ctx

func ParseTagging added in v1.0.13 

func ParseTagging(data []byte, limit TagLimit) (map[string]string, error)

Parses and validates tagging

func ParseUint 

func ParseUint(str string) (int32, error)

func SetBucketNameValidationStrict added in v1.0.19 

func SetBucketNameValidationStrict(strict bool)

func SetMetaHeaders 

func SetMetaHeaders(ctx *fiber.Ctx, meta map[string]string)

func SetResponseHeaders 

func SetResponseHeaders(ctx *fiber.Ctx, headers []CustomHeader)

func StreamResponseBody added in v1.0.10 

func StreamResponseBody(ctx *fiber.Ctx, rdr io.ReadCloser, bodysize int)

Streams the response body by chunks

func ValidateCopySource added in v1.0.17 

func ValidateCopySource(copysource string) error

ValidateCopySource parses and validates the copy-source

func ValidateDate 

func ValidateDate(date time.Time) error

func ValidateVersionId added in v1.0.19 

func ValidateVersionId(versionId string) error

ValidateVersionId check if the input versionId is 'ulid' compatible

func WithCopySource added in v1.0.18 

func WithCopySource() preconditionOpt

Types

type AuthData 

type AuthData struct {
	Algorithm     string
	Access        string
	Region        string
	SignedHeaders string
	Signature     string
	Date          string
}

AuthData is the parsed authorization data from the header

func ParseAuthorization 

func ParseAuthorization(authorization string) (AuthData, error)

ParseAuthorization returns the parsed fields for the aws v4 auth header example authorization string from aws docs: Authorization: AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/20130524/us-east-1/s3/aws4_request, SignedHeaders=host;range;x-amz-date, Signature=fe5f80f77d5fa3beca038a248ff027d0445342fe2855ddc963176630326f1024

func ParsePresignedURIParts 

func ParsePresignedURIParts(ctx *fiber.Ctx) (AuthData, error)

https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-string-auth.html

ParsePresignedURIParts parses and validates request URL query parameters

?X-Amz-Algorithm=AWS4-HMAC-SHA256 &X-Amz-Credential=access-key-id/20130721/us-east-1/s3/aws4_request &X-Amz-Date=20130721T201207Z &X-Amz-Expires=86400 &X-Amz-SignedHeaders=host &X-Amz-Signature=1e68ad45c1db540284a4a1eca3884c293ba1a0ff63ab9db9a15b5b29dfa02cd8

type AuthReader 

type AuthReader struct {
	// contains filtered or unexported fields
}

AuthReader is an io.Reader that validates the request authorization once the underlying reader returns io.EOF. This is needed for streaming data requests where the data size and checksum are not known until the data is completely read.

func NewAuthReader 

func NewAuthReader(ctx *fiber.Ctx, r io.Reader, auth AuthData, secret string) *AuthReader

NewAuthReader initializes an io.Reader that will verify the request v4 auth when the underlying reader returns io.EOF. This postpones the authorization check until the reader is consumed. So it is important that the consumer of this reader checks for the auth errors while reading.

func (*AuthReader) Read 

func (ar *AuthReader) Read(p []byte) (int, error)

Read allows *AuthReader to be used as an io.Reader

type ChecksumValues added in v1.0.13 

type ChecksumValues map[types.ChecksumAlgorithm]string

func ParseCalculatedChecksumHeaders added in v1.0.17 

func ParseCalculatedChecksumHeaders(ctx *fiber.Ctx) (ChecksumValues, error)

ParseCalculatedChecksumHeaders parses and validates x-amz-checksum-x header keys e.g x-amz-checksum-crc32, x-amz-checksum-sha256 ...

func ParseChecksumHeadersAndSdkAlgo added in v1.0.17 

func ParseChecksumHeadersAndSdkAlgo(ctx *fiber.Ctx) (types.ChecksumAlgorithm, ChecksumValues, error)

ParseChecksumHeadersAndSdkAlgo parses/validates 'x-amz-sdk-checksum-algorithm' and 'x-amz-checksum-x' precalculated request headers

func ParseCompleteMpChecksumHeaders added in v1.0.19 

func ParseCompleteMpChecksumHeaders(ctx *fiber.Ctx) (ChecksumValues, error)

ParseCompleteMpChecksumHeaders parses and validates the 'CompleteMultipartUpload' x-amz-checksum-x headers by supporting both 'checksum' and 'checksum-<part_length>' formats

func (ChecksumValues) Headers added in v1.0.13 

func (cv ChecksumValues) Headers() string

Headers concatinates checksum algorithm by prefixing each with 'x-amz-checksum-' e.g. "x-amz-checksum-crc64nvme, x-amz-checksum-sha1"

type ChunkReader 

type ChunkReader struct {
	// contains filtered or unexported fields
}

ChunkReader reads from chunked upload request body, and returns object data stream

func (*ChunkReader) Read 

func (cr *ChunkReader) Read(p []byte) (int, error)

Read satisfies the io.Reader for this type

type CompositeChecksumReader added in v1.0.11 

type CompositeChecksumReader struct {
	// contains filtered or unexported fields
}

func NewCompositeChecksumReader added in v1.0.11 

func NewCompositeChecksumReader(ht HashType) (*CompositeChecksumReader, error)

NewCompositeChecksumReader initializes a composite checksum processor, which decodes and validates the provided checksums and returns the final checksum based on the previous processings.

The supported checksum types are: - CRC32 - CRC32C - SHA1 - SHA256

func (*CompositeChecksumReader) Process added in v1.0.11 

func (ccr *CompositeChecksumReader) Process(checksum string) error

Decodes and writes the checksum in the hasher

func (*CompositeChecksumReader) Sum added in v1.0.11 

func (ccr *CompositeChecksumReader) Sum() string

Returns the base64 encoded composite checksum

type ConditionalHeaders added in v1.0.18 

type ConditionalHeaders struct {
	IfMatch       *string
	IfNoneMatch   *string
	IfModSince    *time.Time
	IfUnmodeSince *time.Time
}

ConditionalHeaders holds the conditional header values

func ParsePreconditionHeaders added in v1.0.18 

func ParsePreconditionHeaders(ctx *fiber.Ctx, opts ...preconditionOpt) ConditionalHeaders

ParsePreconditionHeaders parses the precondition headers: - If-Match - If-None-Match - If-Modified-Since - If-Unmodified-Since

type ContextKey added in v1.0.15 

type ContextKey string

Region, StartTime, IsRoot, Account, AccessKey context locals are set to default values in middlewares.SetDefaultValues to avoid the nil interface conversions 

const (
	ContextKeyRegion         ContextKey = "region"
	ContextKeyStartTime      ContextKey = "start-time"
	ContextKeyIsRoot         ContextKey = "is-root"
	ContextKeyRootAccessKey  ContextKey = "root-access-key"
	ContextKeyAccount        ContextKey = "account"
	ContextKeyAuthenticated  ContextKey = "authenticated"
	ContextKeyPublicBucket   ContextKey = "public-bucket"
	ContextKeyParsedAcl      ContextKey = "parsed-acl"
	ContextKeySkipResBodyLog ContextKey = "skip-res-body-log"
	ContextKeyBodyReader     ContextKey = "body-reader"
	ContextKeySkip           ContextKey = "__skip"
	ContextKeyStack          ContextKey = "stack"
)

func (ContextKey) Delete added in v1.0.17 

func (ck ContextKey) Delete(ctx *fiber.Ctx)

func (ContextKey) Get added in v1.0.15 

func (ck ContextKey) Get(ctx *fiber.Ctx) any

func (ContextKey) IsSet added in v1.0.15 

func (ck ContextKey) IsSet(ctx *fiber.Ctx) bool

func (ContextKey) Set added in v1.0.15 

func (ck ContextKey) Set(ctx *fiber.Ctx, val any)

func (ContextKey) Values added in v1.0.15 

func (ck ContextKey) Values() []ContextKey

type CustomHeader 

type CustomHeader struct {
	Key   string
	Value string
}

type HashReader 

type HashReader struct {
	// contains filtered or unexported fields
}

HashReader is an io.Reader that calculates the checksum as the data is read

func NewHashReader 

func NewHashReader(r io.Reader, expectedSum string, ht HashType) (*HashReader, error)

NewHashReader intializes an io.Reader from an underlying io.Reader that calculates the checksum while the reader is being read from. If the sum provided is not "", the reader will return an error when the underlying reader returns io.EOF if the checksum does not match the provided expected checksum. If the provided sum is "", then the Sum() method can still be used to get the current checksum for the data read so far.

func (*HashReader) Read 

func (hr *HashReader) Read(p []byte) (int, error)

Read allows *HashReader to be used as an io.Reader

func (*HashReader) SetReader added in v1.0.11 

func (hr *HashReader) SetReader(r io.Reader)

func (*HashReader) Sum 

func (hr *HashReader) Sum() string

Sum returns the checksum hash of the data read so far

func (*HashReader) Type added in v1.0.11 

func (hr *HashReader) Type() HashType

type HashType 

type HashType string

HashType identifies the checksum algorithm to be used 

const (
	// HashTypeMd5 generates MD5 checksum for the data stream
	HashTypeMd5 HashType = "md5"
	// HashTypeSha256 generates SHA256 Base64-Encoded checksum for the data stream
	HashTypeSha256 HashType = "sha256"
	// HashTypeSha256Hex generates SHA256 hex encoded checksum for the data stream
	HashTypeSha256Hex HashType = "sha256-hex"
	// HashTypeSha1 generates SHA1 Base64-Encoded checksum for the data stream
	HashTypeSha1 HashType = "sha1"
	// HashTypeCRC32 generates CRC32 Base64-Encoded checksum for the data stream
	HashTypeCRC32 HashType = "crc32"
	// HashTypeCRC32C generates CRC32C Base64-Encoded checksum for the data stream
	HashTypeCRC32C HashType = "crc32c"
	// HashTypeCRC64NVME generates CRC64NVME Base64-Encoded checksum for the data stream
	HashTypeCRC64NVME HashType = "crc64nvme"
	// HashTypeNone is a no-op checksum for the data stream
	HashTypeNone HashType = "none"
)

type PresignedAuthReader 

type PresignedAuthReader struct {
	// contains filtered or unexported fields
}

PresignedAuthReader is an io.Reader that validates presigned request authorization once the underlying reader returns io.EOF. This is needed for streaming data requests where the data size is not known until the data is completely read.

func NewPresignedAuthReader 

func NewPresignedAuthReader(ctx *fiber.Ctx, r io.Reader, auth AuthData, secret string) *PresignedAuthReader

func (*PresignedAuthReader) Read 

func (pr *PresignedAuthReader) Read(p []byte) (int, error)

Read allows *PresignedAuthReader to be used as an io.Reader

type TagLimit added in v1.0.13 

type TagLimit int

TagLimit specifies the allowed tag count in a tag set 

const (
	// Tag limit for bucket tagging
	TagLimitBucket TagLimit = 50
	// Tag limit for object tagging
	TagLimitObject TagLimit = 10
)

type UnsignedChunkReader added in v1.0.10 

type UnsignedChunkReader struct {
	// contains filtered or unexported fields
}

func NewUnsignedChunkReader added in v1.0.10 

func NewUnsignedChunkReader(r io.Reader, ct checksumType) (*UnsignedChunkReader, error)

func (*UnsignedChunkReader) Read added in v1.0.10 

func (ucr *UnsignedChunkReader) Read(p []byte) (int, error)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.