Documentation
¶
Overview ¶
Package s3cache implements callbacks for a gocache.Server that store data into an S3 bucket through a local directory.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Cache ¶
type Cache struct {
// Local is the local cache directory where actions and objects are staged.
// It must be non-nil. A local stage is required because the Go toolchain
// needs direct access to read the files reported by the cache.
// It is safe to use a tmpfs directory.
Local *cachedir.Dir
// S3Client is the S3 client used to read and write cache entries to the
// backing store. It must be non-nil.
S3Client *s3.Client
// S3Bucket is the name of the S3 bucket where cache entries are stored.
// It must be non-empty.
S3Bucket string
// KeyPrefix, if non-empty, is prepended to each key stored into S3, with an
// intervening slash.
KeyPrefix string
// MinUploadSize, if positive, defines a minimum object size in bytes below
// which the cache will not write the object to S3.
MinUploadSize int64
// UploadConcurrency, if positive, defines the maximum number of concurrent
// tasks for writing cache entries to S3. If zero or negative, it uses
// runtime.NumCPU.
UploadConcurrency int
// contains filtered or unexported fields
}
Cache implements callbacks for a gocache.Server using an S3 bucket for backing store with a local directory for staging.
Remote Cache Layout ¶
Within the designated S3 bucket, keys are organized into two groups. Each action is stored in a file named:
[<prefix>/]action/<xx>/<action-id>
Each output object is stored in a file named:
[<prefix>/]object/<xx>/<object-id>
The object and action IDs are encoded as lower-case hexadecimal strings, with "<xx>" denoting the first two bytes of the ID to partition the space.
The contents of each action file have the format:
<object-id> <timestamp>
where the object ID is hex encoded and the timestamp is Unix nanoseconds. The object file contains just the binary data of the object.