Documentation
¶
Overview ¶
Package objectstore keeps the last valid snapshot of each source in an S3 bucket, the production store of ADR 0005: Scaleway Object Storage in fr-par, shared by the sync job and the API container, which have no volume in common.
Each source is one object whose key is the file name of package store (amf.snapshot) and whose content is its file format, envelope and digest included: Load checks a downloaded object as store.Disk checks a file. A PUT replaces an object at once, so a reader gets the previous snapshot or the new one, never part of one.
Load remembers the entity tag of the last object it accepted for each source and sends it in If-None-Match: while the object is unchanged, the bucket answers 304 and the remembered snapshot is returned without a download. The API reloads every minute; the sync job writes a few times a day.
Requests go path-style to the endpoint host, signed with AWS Signature Version 4, through the given HTTP client: in production the platform client, which allows that host only and bounds time and response size.
Index ¶
Constants ¶
const ( Host = "s3.fr-par.scw.cloud" Region = "fr-par" )
Scaleway Object Storage in fr-par (ADR 0005), the defaults of Options.
const ( AWSParisHost = "s3.eu-west-3.amazonaws.com" AWSParisRegion = "eu-west-3" )
Amazon S3 in Paris, where the backups of the key store go, out of Scaleway (#75).
const DefaultAttempts = 3
DefaultAttempts is the number of tries of a request the bucket answers with a 5xx, a throttle or a network error, the first one included.
Variables ¶
var ErrInvalidOptions = errors.New("invalid object store options")
ErrInvalidOptions means New was given Options it cannot use.
var ErrPresignedURL = errors.New("not a presigned URL of an object of Amazon S3 in eu-west-3")
ErrPresignedURL means a URL is not a presigned URL of an object of Amazon S3 in Paris. The error never quotes the URL: its query is a credential.
Functions ¶
This section is empty.
Types ¶
type Archive ¶ added in v0.57.0
type Archive struct {
// contains filtered or unexported fields
}
Archive adds the sealed backups of the key store to a bucket out of Scaleway (#75). Its credentials may only add objects: it never reads, replaces or removes one. It implements app.BackupArchive.
func NewArchive ¶ added in v0.57.0
NewArchive returns an Archive for opts, or ErrInvalidOptions.
type KeyCheck ¶ added in v0.59.0
type KeyCheck struct {
// contains filtered or unexported fields
}
KeyCheck checks that a new key reads a bucket before it is put in service (ADR 0007). It implements app.CredentialInstaller and changes nothing.
func NewKeyCheck ¶ added in v0.59.0
func NewKeyCheck(opts KeyCheckOptions) (*KeyCheck, error)
NewKeyCheck returns a KeyCheck of opts, or ErrInvalidOptions.
func (*KeyCheck) InstallCredential ¶ added in v0.59.0
InstallCredential lists at most one object of the bucket with c, until a listing succeeds or the attempts are over.
type KeyCheckOptions ¶ added in v0.59.0
type KeyCheckOptions struct {
// Endpoint is the HTTPS URL of the S3 API; "https://" + Host when
// empty.
Endpoint string
// Bucket is the bucket the key must read. Required.
Bucket string
// HTTP sends the requests. Required.
HTTP s3.HTTPClient
// Attempts is the number of listings tried, 6 when zero; Pause the
// wait between two of them, a minute when zero.
Attempts int
Pause time.Duration
// Sleep waits d, or returns the error of ctx once it is done; the
// system clock when nil.
Sleep func(ctx context.Context, d time.Duration) error
}
KeyCheckOptions configures a KeyCheck.
type Options ¶
type Options struct {
// Endpoint is the HTTPS URL of the S3 API; "https://" + Host when
// empty.
Endpoint string
// Region signs the requests; Region when empty.
Region string
// Bucket holds the objects. Required.
Bucket string
// AccessKey and SecretKey are the credentials of the bucket. Required.
AccessKey string
SecretKey config.Secret
// HTTP sends the requests: the platform client, which allows the
// endpoint host only and bounds time and size. Required.
HTTP s3.HTTPClient
// Attempts is the number of tries of a request; DefaultAttempts when
// 0.
Attempts int
}
Options configures a Store.
type PresignedObject ¶ added in v0.58.0
type PresignedObject struct {
// contains filtered or unexported fields
}
PresignedObject reads one object by its presigned URL, with no credentials of its own (#297). It implements app.KeyDumpFetcher.
func NewPresignedObject ¶ added in v0.58.0
func NewPresignedObject(u PresignedURL, client getter) *PresignedObject
NewPresignedObject returns the object of u, fetched with client, which must allow u.Host() only and never quote the query of a URL in its errors.
type PresignedURL ¶ added in v0.58.0
type PresignedURL struct {
// contains filtered or unexported fields
}
PresignedURL is a presigned URL of an object of Amazon S3 in Paris, such as the one `aws s3 presign` prints, checked by ParsePresigned. Every fmt verb prints it without its query.
func ParsePresigned ¶ added in v0.58.0
func ParsePresigned(rawURL string) (PresignedURL, error)
ParsePresigned returns rawURL checked: HTTPS, without credentials nor port, on AWSParisHost (path style) or on a bucket of it (virtual-hosted), naming an object, with a SigV4 signature in its query. Otherwise it returns ErrPresignedURL.
func (PresignedURL) Format ¶ added in v0.58.0
func (u PresignedURL) Format(f fmt.State, _ rune)
Format implements fmt.Formatter: every verb writes the URL without its query, which is a credential.
func (PresignedURL) Host ¶ added in v0.58.0
func (u PresignedURL) Host() string
Host returns the host of the URL, the one host its client may reach.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store keeps one object per source in a bucket. It implements app.SnapshotStore and is safe for concurrent use.
Source Files