Documentation
¶
Overview ¶
Package blobstore is a content-addressed store for artifact trees on S3.
A blob is one artifact packed exactly as it crosses a venue's wire — the tar codec's stream, zstd-wrapped — keyed by the digest internal/workspace computes over the tree. The key IS the identity: there is no index object, no manifest, and no state in the bucket. Which digests mean what stays in the orchestrator's own SQLite, which is why a lifecycle rule expiring untouched objects is always safe — the worst case is a re-upload, never a wrong skip.
Deliberately GET/PUT by key over plain HTTP, never a mounted filesystem: every mount surveyed for #80 fails the working tree (dropped exec bits, missing rename, no-op flock under SQLite).
One caveat to the safety claim above, recorded rather than hidden: a blob is never OVERWRITTEN (publishing HEAD-skips an existing key), so if a wrong tree ever lands under a digest — reachable only by racing two processes over one workspace.root, which the one-process-per-state-file doctrine already forbids — the verify-on-fetch turns it into a permanent miss for that digest rather than corruption, and only lifecycle expiry clears it.
Index ¶
- Variables
- type Options
- type Store
- func (s *Store) Delete(ctx context.Context, key string) error
- func (s *Store) Get(ctx context.Context, key string) (io.ReadCloser, error)
- func (s *Store) GetTree(ctx context.Context, digest, dir string) error
- func (s *Store) Has(ctx context.Context, key string) (bool, error)
- func (s *Store) HasTree(ctx context.Context, digest string) (bool, error)
- func (s *Store) PresignGet(ctx context.Context, key string, ttl time.Duration) (string, error)
- func (s *Store) PresignPut(ctx context.Context, key string, ttl time.Duration) (string, error)
- func (s *Store) PutFile(ctx context.Context, key, name string) error
- func (s *Store) PutTree(ctx context.Context, digest, dir string) error
Constants ¶
This section is empty.
Variables ¶
var ErrMissingBlob = errors.New("blob not in the artifact store")
ErrMissingBlob is a digest the store does not hold. Distinct so a caller can treat it as an ordinary cache miss rather than a broken store.
var ErrStore = errors.New("invalid artifact store")
ErrStore is a store URL that cannot work as written.
Functions ¶
This section is empty.
Types ¶
type Options ¶
type Options struct {
// URL is the option string as written, kept for error messages.
URL string
Bucket string
// Prefix namespaces this store's objects inside the bucket, so one bucket
// can carry several stores. Blobs live under <prefix>/blobs/<digest>.
Prefix string
// Region overrides the ambient AWS region.
Region string
// Endpoint points at an S3-compatible server that is not AWS — minio, or
// a test. Path-style addressing is used with it, because virtual-host
// style needs DNS an alternative endpoint does not have.
Endpoint string
// Limits bound what GetTree unpacks; zero means wire.FetchLimits. A blob
// is unpacked before its digest is checked, so the bound is all that
// stands between a bucket writer and this machine's disk.
Limits wire.UnpackOptions
}
Options say where the store is. Everything about the machine reaching it — credentials, proxies — comes from the ambient AWS configuration, the same chain every other AWS tool reads.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store is one bucket-and-prefix, holding trees by digest.
Five object verbs signed with the SDK's own SigV4 signer rather than the S3 service client: that client was ~4MB of the binary (every S3 operation is kept, because kong and text/template defeat the linker's dead-method pruning) for HEAD, GET, PUT, DELETE and a presign.
func New ¶
New opens the store. Opening reads only local configuration — the first network round trip is the first blob operation, so a misconfigured store fails on the run that uses it, with the operation in the error.
func (*Store) Delete ¶
Delete removes the object at key. Absence is success — the object being gone is the outcome asked for.
func (*Store) GetTree ¶
GetTree materializes digest's blob into dir, which must not already exist. The caller owns verifying that what arrived digests to what was asked for — the digest function lives beside the step cache, not here.
func (*Store) HasTree ¶
HasTree reports whether the store already holds digest, so a Put can be skipped — the point of content addressing is that a key never needs re-uploading.
func (*Store) PresignGet ¶
PresignGet mints a URL that reads key with no credentials but the URL itself — how a venue with zero AWS identity is handed exactly the blobs its job needs and nothing else.
func (*Store) PresignPut ¶
PresignPut is PresignGet for writing key.
func (*Store) PutFile ¶
PutFile uploads one local file as the object at key. A file rather than a reader because S3 signs the payload: an upload needs a length and a seekable body.
Source Files