Documentation
¶
Overview ¶
Package s3service defines s3 operations available
Index ¶
- Variables
- func CheckPrefix(prefix string) error
- func NewStore() (s3interfaces.S3ServiceInterface, error)
- type DeleteKeysError
- type MockS3Service
- func (m *MockS3Service) BulkDeleteFiles(ctx context.Context, uploadedKeys []string) int
- func (m *MockS3Service) BulkUploadFiles(ctx context.Context, payloads []*s3.PutObjectInput) ([]string, error)
- func (m *MockS3Service) DeleteFile(ctx context.Context, s3Path, filename, s3Key *string) error
- func (m *MockS3Service) DeleteKeys(ctx context.Context, keys []string) error
- func (m *MockS3Service) DeletePrefix(ctx context.Context, prefix string) error
- func (m *MockS3Service) FileExists(ctx context.Context, s3Path, filename string) (bool, error)
- func (m *MockS3Service) GeneratePreSignedDownloadURL(ctx context.Context, s3Path, filename string, expirationSeconds int64) (string, error)
- func (m *MockS3Service) GeneratePresignedURL(ctx context.Context, s3Key string, expiry time.Duration) (string, error)
- func (m *MockS3Service) GenerateUploadPayload(ctx context.Context, data []byte, s3Path, filename string, ...) (*s3.PutObjectInput, error)
- func (m *MockS3Service) ListObjects(ctx context.Context, prefix string, page func([]s3interfaces.ObjectInfo) error) error
- func (m *MockS3Service) UploadFile(ctx context.Context, data []byte, s3Path, filename string, ...) error
- type Service
- func (s *Service) BulkDeleteFiles(ctx context.Context, uploadedKeys []string) int
- func (s *Service) BulkUploadFiles(ctx context.Context, payloads []*s3.PutObjectInput) ([]string, error)
- func (s *Service) DeleteFile(ctx context.Context, s3Path, filename, s3Key *string) error
- func (s *Service) DeleteKeys(ctx context.Context, keys []string) error
- func (s *Service) DeletePrefix(ctx context.Context, prefix string) error
- func (s *Service) FileExists(ctx context.Context, s3Path, filename string) (bool, error)
- func (s *Service) GeneratePreSignedDownloadURL(ctx context.Context, s3Path, filename string, expirationSeconds int64) (string, error)
- func (s *Service) GeneratePresignedURL(ctx context.Context, s3Key string, expiry time.Duration) (string, error)
- func (s *Service) GenerateUploadPayload(ctx context.Context, data []byte, s3Path, filename string, ...) (*s3.PutObjectInput, error)
- func (s *Service) ListObjects(ctx context.Context, prefix string, page func([]s3interfaces.ObjectInfo) error) error
- func (s *Service) UploadFile(ctx context.Context, data []byte, s3Path, filename string, ...) error
Constants ¶
This section is empty.
Variables ¶
var ErrKeyOutsidePrefix = errors.New("S3 listed a key outside the requested prefix")
ErrKeyOutsidePrefix means storage listed a key the prefix does not cover.
var ErrListingCutShort = errors.New("S3 listing incomplete: storage looped or omitted its continuation token")
ErrListingCutShort means the listing is incomplete: storage said more objects follow but gave no token, or went round again (a token or a key seen before, or empty pages without end).
Functions ¶
func CheckPrefix ¶ added in v0.18.0
CheckPrefix refuses a prefix that is empty or lacks its trailing slash: "a/u1" would also match "a/u10/...", "" the whole bucket, "a//" an empty id.
func NewStore ¶ added in v0.19.0
func NewStore() (s3interfaces.S3ServiceInterface, error)
NewStore is NewS3Service as the interface callers hold: on failure it returns a true nil, never a nil *Service wrapped in a non-nil interface.
Types ¶
type DeleteKeysError ¶ added in v0.19.0
type DeleteKeysError = s3interfaces.DeleteKeysError
DeleteKeysError is s3interfaces.DeleteKeysError, kept here for callers of this package.
type MockS3Service ¶
MockS3Service provides a mock implementation of S3Service
func (*MockS3Service) BulkDeleteFiles ¶
func (m *MockS3Service) BulkDeleteFiles(ctx context.Context, uploadedKeys []string) int
BulkDeleteFiles deletes multiple files from S3
func (*MockS3Service) BulkUploadFiles ¶
func (m *MockS3Service) BulkUploadFiles(ctx context.Context, payloads []*s3.PutObjectInput) ([]string, error)
BulkUploadFiles uploads multiple files given a list of payloads
func (*MockS3Service) DeleteFile ¶
func (m *MockS3Service) DeleteFile(ctx context.Context, s3Path, filename, s3Key *string) error
DeleteFile deletes a file from S3
func (*MockS3Service) DeleteKeys ¶
func (m *MockS3Service) DeleteKeys(ctx context.Context, keys []string) error
DeleteKeys deletes the given keys
func (*MockS3Service) DeletePrefix ¶
func (m *MockS3Service) DeletePrefix(ctx context.Context, prefix string) error
DeletePrefix deletes every object under prefix
func (*MockS3Service) FileExists ¶
FileExists checks if a file exists in S3
func (*MockS3Service) GeneratePreSignedDownloadURL ¶
func (m *MockS3Service) GeneratePreSignedDownloadURL(ctx context.Context, s3Path, filename string, expirationSeconds int64) (string, error)
GeneratePreSignedDownloadURL generates a presigned download URL
func (*MockS3Service) GeneratePresignedURL ¶
func (m *MockS3Service) GeneratePresignedURL(ctx context.Context, s3Key string, expiry time.Duration) (string, error)
GeneratePresignedURL generates a presigned URL for the given S3 key
func (*MockS3Service) GenerateUploadPayload ¶
func (m *MockS3Service) GenerateUploadPayload(ctx context.Context, data []byte, s3Path, filename string, metadata map[string]string) (*s3.PutObjectInput, error)
GenerateUploadPayload generates a payload for uploading a file to S3
func (*MockS3Service) ListObjects ¶ added in v0.18.0
func (m *MockS3Service) ListObjects(ctx context.Context, prefix string, page func([]s3interfaces.ObjectInfo) error) error
ListObjects mocks listing. Set it up with Return(pages, err): pages is a []s3interfaces.ObjectInfo (one page) or a [][]s3interfaces.ObjectInfo (one call to page each, empty ones skipped as the service does) or nil; err comes after every page, unless page itself fails first.
func (*MockS3Service) UploadFile ¶
func (m *MockS3Service) UploadFile(ctx context.Context, data []byte, s3Path, filename string, metadata map[string]string) error
UploadFile uploads a file to S3 with the given path and filename
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
Service provides methods for interacting with AWS S3
func NewS3Service ¶
NewS3Service creates a new S3 service instance
func (*Service) BulkDeleteFiles ¶
BulkDeleteFiles deletes multiple files from S3 one by one and returns how many deletes failed, so callers can tell an orphaned object from a clean delete.
func (*Service) BulkUploadFiles ¶
func (s *Service) BulkUploadFiles(ctx context.Context, payloads []*s3.PutObjectInput) ([]string, error)
BulkUploadFiles uploads multiple files given a list of payloads. Auto rollback if there are errors
func (*Service) DeleteFile ¶
DeleteFile deletes a file from S3
func (*Service) DeleteKeys ¶
DeleteKeys deletes every key, in DeleteObjects batches. A key that does not exist counts as deleted. If any key could not be deleted it returns a *DeleteKeysError; the others are deleted anyway, so a retry only has the rest to do. Once ctx ends it sends no more batches.
func (*Service) DeletePrefix ¶
DeletePrefix deletes every object whose key starts with prefix; the prefix must pass CheckPrefix. It deletes page by page and stops at the first page with a failed delete or a listing error, naming the prefix: the pages before are already deleted, and a retry deletes the rest.
func (*Service) FileExists ¶
FileExists checks if a file exists in S3
func (*Service) GeneratePreSignedDownloadURL ¶
func (s *Service) GeneratePreSignedDownloadURL(ctx context.Context, s3Path, filename string, expirationSeconds int64) (string, error)
GeneratePreSignedDownloadURL generates a pre-signed URL for downloading a file from S3 The URL will be valid for the specified duration (in seconds)
func (*Service) GeneratePresignedURL ¶
func (s *Service) GeneratePresignedURL(ctx context.Context, s3Key string, expiry time.Duration) (string, error)
GeneratePresignedURL generates a short-lived S3 GET URL for the given full object key.
func (*Service) GenerateUploadPayload ¶
func (s *Service) GenerateUploadPayload(ctx context.Context, data []byte, s3Path, filename string, metadata map[string]string) (*s3.PutObjectInput, error)
GenerateUploadPayload generates a payload for uploading a file to S3 have the same signature as UploadFile, but return the payload instead of uploading for bulk upload
func (*Service) ListObjects ¶ added in v0.18.0
func (s *Service) ListObjects(ctx context.Context, prefix string, page func([]s3interfaces.ObjectInfo) error) error
ListObjects calls page with each non-empty page of the objects whose key starts with prefix (up to 1000 per page), and stops at the first error page returns. An incomplete listing (ErrListingCutShort) or a key outside the prefix (ErrKeyOutsidePrefix) is an error, never a short success. On an error, the pages already handed to page stay handed: a caller acting on each page has acted on those.