Documentation
¶
Overview ¶
Package blobstore is a provider-agnostic object-storage layer: a minimal presigned-URL contract plus path-safety primitives, with no knowledge of what the blobs are. Code bundles, session exports, and user image uploads each sit on top of it via their own key builders.
The S3 implementation works against AWS S3, Cloudflare R2, Tigris, MinIO, and any other S3-compatible API. The Memory implementation is for tests.
Path safety: ValidateComponent / ValidateContentHash reject any path component that could escape its tenant prefix ("..", "/", "\\", a leading dot, or an over-long value). Key builders in dependent packages MUST route every untrusted component through them; userID in particular MUST come from authenticated token claims, never request input.
Index ¶
- Variables
- func ValidateComponent(name, v string) error
- func ValidateContentHash(h string) error
- type LAN
- func (l *LAN) BaseURL() string
- func (l *LAN) Close() error
- func (l *LAN) DeletePrefix(_ context.Context, prefix string) error
- func (l *LAN) Exists(_ context.Context, key string) (bool, error)
- func (l *LAN) PresignGet(_ context.Context, key string, ttl time.Duration) (string, error)
- func (l *LAN) PresignPut(_ context.Context, key string, ttl time.Duration) (string, error)
- type Memory
- func (m *Memory) BaseURL() string
- func (m *Memory) Close()
- func (m *Memory) DeletePrefix(_ context.Context, prefix string) error
- func (m *Memory) Exists(_ context.Context, key string) (bool, error)
- func (m *Memory) Get(key string) ([]byte, bool)
- func (m *Memory) GetCount(key string) int
- func (m *Memory) Keys() []string
- func (m *Memory) PresignGet(_ context.Context, key string, ttl time.Duration) (string, error)
- func (m *Memory) PresignPut(_ context.Context, key string, ttl time.Duration) (string, error)
- func (m *Memory) Put(key string, data []byte)
- type S3
- func (s *S3) DeletePrefix(ctx context.Context, prefix string) error
- func (s *S3) Exists(ctx context.Context, key string) (bool, error)
- func (s *S3) PresignGet(ctx context.Context, key string, ttl time.Duration) (string, error)
- func (s *S3) PresignPut(ctx context.Context, key string, ttl time.Duration) (string, error)
- type S3Config
- type Storage
Constants ¶
This section is empty.
Variables ¶
var ErrInvalidPathComponent = errors.New("blobstore: invalid path component")
ErrInvalidPathComponent is returned by ValidateComponent / ValidateContentHash when an input would produce an unsafe storage key. Surfaced as a typed error so handlers can return 400 not 500.
var ErrNotFound = errors.New("blobstore: object not found")
ErrNotFound is returned when no object exists at the requested key. Wrapped, not unwrapped — callers should errors.Is.
ErrUnavailable is returned when a Storage implementation has no reachable backing yet (e.g. a LAN blobstore before any address is bound). Distinguishes a temporary not-ready state from a genuine failure so callers can surface 503 instead of 500.
Functions ¶
func ValidateComponent ¶
ValidateComponent rejects empty strings, anything containing path separators or escape sequences, anything starting with a dot (would shadow ".gitignore"-style hidden entries), and anything over 128 chars. name is only used to render the error.
func ValidateContentHash ¶
ValidateContentHash requires exactly 64 lowercase-hex chars (a sha256 digest) so a caller can't smuggle an arbitrary path component through a content-hash slot.
Types ¶
type LAN ¶
type LAN struct {
// contains filtered or unexported fields
}
LAN is a self-serving, disk-backed Storage for single-binary deployments (clank preview) where there is no S3. It runs its own HTTP server on a LAN-bound listener and mints HMAC-signed presigned URLs that point back at itself, so a phone can PUT an image and the local clank-host can GET it — both over the LAN, no object store required.
Presigned URLs are signed over (key, op, exp) with a per-process key, so a peer on the same network can't PUT or GET by guessing storage keys: only URLs this server minted are honored, and only until they expire.
func NewLAN ¶
NewLAN starts the blob server on bindAddr (e.g. "0.0.0.0:0") and advertises advertiseHost (the LAN IP) in every minted URL so the URLs resolve from both the phone and the local host. It owns a private temp directory for blobs. signKey must be non-empty. Caller MUST Close.
func (*LAN) BaseURL ¶
BaseURL is the advertised origin minted URLs hang off of, e.g. http://192.168.1.20:7879.
func (*LAN) DeletePrefix ¶
DeletePrefix removes every blob whose key starts with prefix. Walks the tree (the store is small + ephemeral) so partial prefixes work, not just directory boundaries.
func (*LAN) PresignGet ¶
type Memory ¶
type Memory struct {
// contains filtered or unexported fields
}
Memory is an in-memory Storage implementation backed by a httptest server, intended for tests. Real PUT/GET against the presigned URLs works (the embedded server handles them) so the same client code path can be exercised end-to-end without a real S3.
Not safe for production use: presigned URLs do not carry signatures (only TTL), there is no access control, and all data lives in process memory.
func NewMemory ¶
func NewMemory() *Memory
NewMemory constructs a Memory backend with an embedded test server. Callers MUST invoke Close() to release the server when done.
func (*Memory) BaseURL ¶
BaseURL returns the embedded test server's base URL — useful for tests that need to hand the URL to a separate component.
func (*Memory) GetCount ¶
GetCount returns how many times key has been fetched via a presigned GET URL (counting attempts, hit or miss). Test-only instrumentation for asserting an object was — or was not — read from storage.
func (*Memory) PresignGet ¶
func (*Memory) PresignPut ¶
type S3 ¶
type S3 struct {
// contains filtered or unexported fields
}
S3 is the S3-compatible Storage implementation.
func NewS3 ¶
NewS3 constructs an S3 backend. Returns an error if Bucket / Region / credentials are missing — fail fast at startup, never silently fall back to anonymous access.
func (*S3) DeletePrefix ¶
DeletePrefix lists every object under prefix and deletes them in batches of deleteBatchSize. Paginates on the continuation token until the listing is exhausted. Uses the direct (internal-endpoint) client, not the presigner. A NoSuchKey on an individual object is treated as already-gone; only a hard error aborts (the caller retries — the operation is idempotent on the same prefix).
func (*S3) PresignGet ¶
type S3Config ¶
type S3Config struct {
// Bucket name. Must already exist; we don't auto-create.
Bucket string
// Region (required by AWS even for S3-alikes; e.g. R2 wants "auto").
Region string
// Endpoint is the URL the gateway uses for its own direct SDK
// calls. Should be reachable
// from inside the gateway — for docker-compose dev that's the
// internal docker hostname like http://clank-minio:9000. Leave
// empty for AWS S3.
Endpoint string
// PublicEndpoint is the URL baked into presigned URLs handed out to
// the laptop and any remote sprite. Must resolve from BOTH ends to
// the same backing storage (because SigV4 binds the host into the
// signature). When empty, falls back to Endpoint.
//
// Why two endpoints: the docker dev stack wraps minio behind a
// Cloudflare quick tunnel so a fly.io sprite can pull from it; the
// tunnel rewrites the Host header on inbound requests, which breaks
// SigV4 if the gateway itself goes through it. The gateway short-
// circuits to the docker-internal hostname for its own calls while
// minting presigned URLs with the tunnel hostname.
PublicEndpoint string
// AccessKey + SecretKey for the bucket. Required.
AccessKey string
SecretKey string
// UsePathStyle forces path-style addressing (bucket as URL path
// segment, not subdomain). Required for MinIO and most R2 setups.
UsePathStyle bool
}
S3Config configures an S3-compatible Storage backend. Works with AWS S3, Cloudflare R2, Tigris, MinIO, and any other S3-compatible API by setting Endpoint and UsePathStyle appropriately.
type Storage ¶
type Storage interface {
// PresignPut returns a presigned PUT URL valid for ttl. The URL is
// itself the capability — anyone holding it can upload to that key
// until ttl expires. Callers MUST scope key construction via a
// validated key builder, never accept raw paths from untrusted input.
PresignPut(ctx context.Context, key string, ttl time.Duration) (url string, err error)
// PresignGet returns a presigned GET URL valid for ttl. Same
// capability semantics as PresignPut.
PresignGet(ctx context.Context, key string, ttl time.Duration) (url string, err error)
// Exists reports whether an object exists at key. Used for
// content-addressed dedup — if the keyed object is already there, the
// caller can skip the PUT URL entirely.
Exists(ctx context.Context, key string) (bool, error)
// DeletePrefix removes every object whose key starts with prefix.
// Idempotent — deleting an empty/already-gone prefix is not an error.
// Used for tenant erasure (account deletion): one sweep of "<userID>/"
// purges all of a user's blobs. Callers MUST build the prefix via a
// validated key builder so an empty/escaped value can't widen the
// sweep to the whole bucket.
DeletePrefix(ctx context.Context, prefix string) error
}
Storage is the minimal contract for object storage. Implementations MUST be safe for concurrent use.