Documentation
¶
Overview ¶
Package s3 is the small client of the S3 protocol the replicator uses: PUT, multipart upload, ranged GET, ListObjectsV2 and DELETE, signed with SigV4, against AWS S3 and the stores that speak its protocol (Cloudflare R2, MinIO and the like).
Every body is signed by its SHA-256, which the store checks, and no request asks for the checksums newer AWS SDKs add, which other stores do not all take. Redirects are not followed: a request goes to the endpoint it was given or fails. No error carries a secret.
Index ¶
- Variables
- func Retryable(err error) bool
- type Client
- func (c *Client) CreateBucket(ctx context.Context) error
- func (c *Client) Delete(ctx context.Context, key string) error
- func (c Client) Format(f fmt.State, verb rune)
- func (c *Client) Get(ctx context.Context, key string, off, n int64) (io.ReadCloser, error)
- func (c *Client) Head(ctx context.Context, key string) (Info, error)
- func (c *Client) List(ctx context.Context, prefix, startAfter string) iter.Seq2[Object, error]
- func (c *Client) Put(ctx context.Context, key string, body io.ReaderAt, size int64, opt PutOptions) error
- type Config
- type Credentials
- type CredentialsProvider
- type EnvCredentials
- type Error
- type Info
- type Object
- type PutOptions
Constants ¶
This section is empty.
Variables ¶
var ErrNoAnswer = errors.New("s3: no answer")
ErrNoAnswer is a request that got no answer: the connection failed or the answer was cut. It may have taken effect or not. A PUT with IfNoneMatch sent again after it can fail with ErrPreconditionFailed because of its own first try: Head tells.
var ErrNotFound = errors.New("s3: no such key")
ErrNotFound is the answer of a GET of a key the bucket does not have. A missing bucket is another error.
var ErrPreconditionFailed = errors.New("s3: precondition failed")
ErrPreconditionFailed is a 412 answer: an object written with IfNoneMatch is there already.
Functions ¶
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client talks to one bucket. It is safe for concurrent use.
func New ¶
New checks cfg and makes a client. It makes no request.
The bucket goes in the host (virtual-hosted style) on an amazonaws.com endpoint when its name has no dot, and in the path otherwise, which the other stores all take.
func (*Client) CreateBucket ¶
CreateBucket makes the client's bucket, in its region.
func (*Client) Get ¶
Get reads the object at key: n bytes from byte off, or all from off when n < 0. The answer is shorter than n only when the object ends first. A range that starts at or past the end, also of an empty object, fails with a 416. The caller closes the reader.
func (*Client) Head ¶
Head reads the size and metadata of the object at key. Its 404 is ErrNotFound: a HEAD answer has no body to say NoSuchKey, so a missing bucket reads the same.
func (*Client) List ¶
List gives the objects whose keys start with prefix and come after startAfter (when not empty), in key order, a page of up to 1000 at a time. It stops at the first error.
func (*Client) Put ¶
func (c *Client) Put(ctx context.Context, key string, body io.ReaderAt, size int64, opt PutOptions) error
Put writes size bytes of body as the object at key. An object larger than a part goes up in parts (a multipart upload), each part hashed and sent from body in turn, so nothing is held in memory. A failed multipart upload is aborted, but for one whose start got no answer, which has no ID to abort: a lifecycle rule of the bucket (AbortIncompleteMultipartUpload) removes its parts. body must not change during the call.
type Config ¶
type Config struct {
// Endpoint is the scheme and host of the store, such as
// https://s3.us-east-1.amazonaws.com or
// https://<account>.r2.cloudflarestorage.com, with no path.
Endpoint string
// Region is the region requests are signed for: "auto" for R2, and
// often "us-east-1" for a store that has no regions.
Region string
Bucket string
// Credentials sign each request.
Credentials CredentialsProvider
// AllowHTTP lets the endpoint be plain http, such as a local MinIO.
// Without it only https is taken.
AllowHTTP bool
}
Config says where the bucket is and how to sign.
type Credentials ¶
type Credentials struct {
AccessKeyID string
SecretAccessKey string
SessionToken string // empty for long-term keys
}
Credentials sign requests. Every fmt verb on them gives the access key ID and hides the rest. fmt calls no method on a field that is not exported, so a struct that keeps them in such a field prints them in full: keep them in a provider (Static), which holds them behind a pointer.
type CredentialsProvider ¶
type CredentialsProvider interface {
Credentials(ctx context.Context) (Credentials, error)
}
A CredentialsProvider gives the credentials of each request, so that credentials that expire can be renewed. It is called for every request and caches what it fetches.
func Static ¶
func Static(c Credentials) CredentialsProvider
Static gives fixed credentials. They are kept behind a pointer, so that a struct that holds the provider prints an address and not the secret, however deep fmt looks.
type EnvCredentials ¶
type EnvCredentials struct{}
EnvCredentials reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN at each request.
func (EnvCredentials) Credentials ¶
func (EnvCredentials) Credentials(context.Context) (Credentials, error)
Credentials reads the variables.
type Error ¶
type Error struct {
Op string // GET, PUT, LIST, DELETE, or a step of a multipart upload
Key string
StatusCode int
Code string // S3's error code, such as NoSuchKey, if the answer had one
Message string
RequestID string
}
Error is an answer of the store that is not a success.
type Info ¶
type Info struct {
Size int64
LastModified time.Time
ETag string
Metadata map[string]string // by the names given to Put
}
Info is what Head gives of an object.
type PutOptions ¶
type PutOptions struct {
// IfNoneMatch writes the object only if the key has none: Put fails
// with ErrPreconditionFailed when it has.
IfNoneMatch bool
// Multipart sends the object as a multipart upload even when it fits
// one PUT, as one part: a probe of whether the store makes the
// completion of an upload conditional too.
Multipart bool
// Metadata is stored with the object as x-amz-meta-<name>. Names are
// lower case letters, digits and '-', values printable ASCII.
Metadata map[string]string
}
PutOptions are the options of Put.