blobstore

package
v0.2.8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 17, 2026 License: AGPL-3.0 Imports: 24 Imported by: 0

Documentation

Overview

Package blobstore is a provider-agnostic object-storage layer: a minimal presigned-URL contract plus path-safety primitives, with no knowledge of what the blobs are. Code bundles, session exports, and user image uploads each sit on top of it via their own key builders.

The S3 implementation works against AWS S3, Cloudflare R2, Tigris, MinIO, and any other S3-compatible API. The Memory implementation is for tests.

Path safety: ValidateComponent / ValidateContentHash reject any path component that could escape its tenant prefix ("..", "/", "\\", a leading dot, or an over-long value). Key builders in dependent packages MUST route every untrusted component through them; userID in particular MUST come from authenticated token claims, never request input.

Index

Constants

This section is empty.

Variables

View Source
var ErrInvalidPathComponent = errors.New("blobstore: invalid path component")

ErrInvalidPathComponent is returned by ValidateComponent / ValidateContentHash when an input would produce an unsafe storage key. Surfaced as a typed error so handlers can return 400 not 500.

View Source
var ErrNotFound = errors.New("blobstore: object not found")

ErrNotFound is returned when no object exists at the requested key. Wrapped, not unwrapped — callers should errors.Is.

View Source
var ErrUnavailable = errors.New("blobstore: storage unavailable")

ErrUnavailable is returned when a Storage implementation has no reachable backing yet (e.g. a LAN blobstore before any address is bound). Distinguishes a temporary not-ready state from a genuine failure so callers can surface 503 instead of 500.

Functions

func ValidateComponent

func ValidateComponent(name, v string) error

ValidateComponent rejects empty strings, anything containing path separators or escape sequences, anything starting with a dot (would shadow ".gitignore"-style hidden entries), and anything over 128 chars. name is only used to render the error.

func ValidateContentHash

func ValidateContentHash(h string) error

ValidateContentHash requires exactly 64 lowercase-hex chars (a sha256 digest) so a caller can't smuggle an arbitrary path component through a content-hash slot.

Types

type LAN

type LAN struct {
	// contains filtered or unexported fields
}

LAN is a self-serving, disk-backed Storage for single-binary deployments (clank preview) where there is no S3. It runs its own HTTP server on a LAN-bound listener and mints HMAC-signed presigned URLs that point back at itself, so a phone can PUT an image and the local clank-host can GET it — both over the LAN, no object store required.

Presigned URLs are signed over (key, op, exp) with a per-process key, so a peer on the same network can't PUT or GET by guessing storage keys: only URLs this server minted are honored, and only until they expire.

func NewLAN

func NewLAN(bindAddr, advertiseHost string, signKey []byte) (*LAN, error)

NewLAN starts the blob server on bindAddr (e.g. "0.0.0.0:0") and advertises advertiseHost (the LAN IP) in every minted URL so the URLs resolve from both the phone and the local host. It owns a private temp directory for blobs. signKey must be non-empty. Caller MUST Close.

func (*LAN) BaseURL

func (l *LAN) BaseURL() string

BaseURL is the advertised origin minted URLs hang off of, e.g. http://192.168.1.20:7879.

func (*LAN) Close

func (l *LAN) Close() error

Close stops the server and removes the blob directory.

func (*LAN) DeletePrefix

func (l *LAN) DeletePrefix(_ context.Context, prefix string) error

DeletePrefix removes every blob whose key starts with prefix. Walks the tree (the store is small + ephemeral) so partial prefixes work, not just directory boundaries.

func (*LAN) Exists

func (l *LAN) Exists(_ context.Context, key string) (bool, error)

func (*LAN) PresignGet

func (l *LAN) PresignGet(_ context.Context, key string, ttl time.Duration) (string, error)

func (*LAN) PresignPut

func (l *LAN) PresignPut(_ context.Context, key string, ttl time.Duration) (string, error)

type Memory

type Memory struct {
	// contains filtered or unexported fields
}

Memory is an in-memory Storage implementation backed by a httptest server, intended for tests. Real PUT/GET against the presigned URLs works (the embedded server handles them) so the same client code path can be exercised end-to-end without a real S3.

Not safe for production use: presigned URLs do not carry signatures (only TTL), there is no access control, and all data lives in process memory.

func NewMemory

func NewMemory() *Memory

NewMemory constructs a Memory backend with an embedded test server. Callers MUST invoke Close() to release the server when done.

func (*Memory) BaseURL

func (m *Memory) BaseURL() string

BaseURL returns the embedded test server's base URL — useful for tests that need to hand the URL to a separate component.

func (*Memory) Close

func (m *Memory) Close()

Close shuts down the embedded test server.

func (*Memory) DeletePrefix

func (m *Memory) DeletePrefix(_ context.Context, prefix string) error

func (*Memory) Exists

func (m *Memory) Exists(_ context.Context, key string) (bool, error)

func (*Memory) Get

func (m *Memory) Get(key string) ([]byte, bool)

Get reads data at key directly, bypassing the presigned-URL path.

func (*Memory) GetCount

func (m *Memory) GetCount(key string) int

GetCount returns how many times key has been fetched via a presigned GET URL (counting attempts, hit or miss). Test-only instrumentation for asserting an object was — or was not — read from storage.

func (*Memory) Keys

func (m *Memory) Keys() []string

Keys returns all keys currently stored. Useful for assertions.

func (*Memory) PresignGet

func (m *Memory) PresignGet(_ context.Context, key string, ttl time.Duration) (string, error)

func (*Memory) PresignPut

func (m *Memory) PresignPut(_ context.Context, key string, ttl time.Duration) (string, error)

func (*Memory) Put

func (m *Memory) Put(key string, data []byte)

Put writes data at key directly, bypassing the presigned-URL path. Useful in tests for arrange-phase setup.

type S3

type S3 struct {
	// contains filtered or unexported fields
}

S3 is the S3-compatible Storage implementation.

func NewS3

func NewS3(ctx context.Context, cfg S3Config) (*S3, error)

NewS3 constructs an S3 backend. Returns an error if Bucket / Region / credentials are missing — fail fast at startup, never silently fall back to anonymous access.

func (*S3) DeletePrefix

func (s *S3) DeletePrefix(ctx context.Context, prefix string) error

DeletePrefix lists every object under prefix and deletes them in batches of deleteBatchSize. Paginates on the continuation token until the listing is exhausted. Uses the direct (internal-endpoint) client, not the presigner. A NoSuchKey on an individual object is treated as already-gone; only a hard error aborts (the caller retries — the operation is idempotent on the same prefix).

func (*S3) Exists

func (s *S3) Exists(ctx context.Context, key string) (bool, error)

func (*S3) PresignGet

func (s *S3) PresignGet(ctx context.Context, key string, ttl time.Duration) (string, error)

func (*S3) PresignPut

func (s *S3) PresignPut(ctx context.Context, key string, ttl time.Duration) (string, error)

type S3Config

type S3Config struct {
	// Bucket name. Must already exist; we don't auto-create.
	Bucket string

	// Region (required by AWS even for S3-alikes; e.g. R2 wants "auto").
	Region string

	// Endpoint is the URL the gateway uses for its own direct SDK
	// calls. Should be reachable
	// from inside the gateway — for docker-compose dev that's the
	// internal docker hostname like http://clank-minio:9000. Leave
	// empty for AWS S3.
	Endpoint string

	// PublicEndpoint is the URL baked into presigned URLs handed out to
	// the laptop and any remote sprite. Must resolve from BOTH ends to
	// the same backing storage (because SigV4 binds the host into the
	// signature). When empty, falls back to Endpoint.
	//
	// Why two endpoints: the docker dev stack wraps minio behind a
	// Cloudflare quick tunnel so a fly.io sprite can pull from it; the
	// tunnel rewrites the Host header on inbound requests, which breaks
	// SigV4 if the gateway itself goes through it. The gateway short-
	// circuits to the docker-internal hostname for its own calls while
	// minting presigned URLs with the tunnel hostname.
	PublicEndpoint string

	// AccessKey + SecretKey for the bucket. Required.
	AccessKey string
	SecretKey string

	// UsePathStyle forces path-style addressing (bucket as URL path
	// segment, not subdomain). Required for MinIO and most R2 setups.
	UsePathStyle bool
}

S3Config configures an S3-compatible Storage backend. Works with AWS S3, Cloudflare R2, Tigris, MinIO, and any other S3-compatible API by setting Endpoint and UsePathStyle appropriately.

type Storage

type Storage interface {
	// PresignPut returns a presigned PUT URL valid for ttl. The URL is
	// itself the capability — anyone holding it can upload to that key
	// until ttl expires. Callers MUST scope key construction via a
	// validated key builder, never accept raw paths from untrusted input.
	PresignPut(ctx context.Context, key string, ttl time.Duration) (url string, err error)

	// PresignGet returns a presigned GET URL valid for ttl. Same
	// capability semantics as PresignPut.
	PresignGet(ctx context.Context, key string, ttl time.Duration) (url string, err error)

	// Exists reports whether an object exists at key. Used for
	// content-addressed dedup — if the keyed object is already there, the
	// caller can skip the PUT URL entirely.
	Exists(ctx context.Context, key string) (bool, error)

	// DeletePrefix removes every object whose key starts with prefix.
	// Idempotent — deleting an empty/already-gone prefix is not an error.
	// Used for tenant erasure (account deletion): one sweep of "<userID>/"
	// purges all of a user's blobs. Callers MUST build the prefix via a
	// validated key builder so an empty/escaped value can't widen the
	// sweep to the whole bucket.
	DeletePrefix(ctx context.Context, prefix string) error
}

Storage is the minimal contract for object storage. Implementations MUST be safe for concurrent use.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL