storage

package
v0.5.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 9, 2026 License: Apache-2.0 Imports: 12 Imported by: 0

Documentation

Overview

Package storage is the object storage interface an application writes against, with the backend selected by configuration, per requirement:object-storage.

A handler resolves a configured bucket by the name it is addressed by and stores, reads, lists and deletes objects through it. Which store answers — a directory under the project, an S3-compatible endpoint, or an R2 binding inside a Cloudflare Worker — is the bucket's backend, so the handler runs on every host from one source:

bucket, err := storage.Open(ctx, "uploads")
err = bucket.Put(ctx, key, body, storage.PutOptions{ContentType: "image/png"})

A backend registers itself from a blank import, the way an engine does:

import _ "github.com/shibukawa/popcornweb/storage/local"

Index

Constants

View Source
const DefaultPresignExpiry = 15 * time.Minute

DefaultPresignExpiry is the life of a presigned URL when the caller set none, on the backends that serve their own.

View Source
const SignedPathPrefix = "/_storage/"

SignedPathPrefix is where the application serves the URLs the local backend presigns: SignedPathPrefix + bucket name + "/" + key, with the expiry and the signature in the query. A backend that serves its own presigned requests implements SelfServing, and the framework mounts SignedHandler ahead of the application when such a bucket is configured.

Variables

View Source
var ErrBadSignature = errors.New("storage: signed URL is invalid or expired")

ErrBadSignature reports a self-served request whose signature or expiry does not hold.

View Source
var ErrNotFound = errors.New("storage: object not found")

ErrNotFound reports a key the bucket does not hold. Every backend returns it for a missing object, so a caller tests one condition with errors.Is.

View Source
var ErrPresignUnavailable = errors.New("storage: presigned URLs are unavailable for this bucket")

ErrPresignUnavailable reports a bucket that cannot issue a presigned URL as configured; the message names what it lacks.

Functions

func Backends

func Backends() []string

Backends lists the registered backend names in order.

func ReadAll

func ReadAll(object *Object) ([]byte, error)

ReadAll reads a body whole and closes it, for a caller that needs the bytes in memory, such as one serving a Range request from them.

func RegisterBackend

func RegisterBackend(name string, factory Factory)

RegisterBackend registers factory under name. A backend package calls it from init, so a blank import is what puts a backend in a binary. A duplicate or empty name panics: two backends answering one configuration value is a build mistake.

func Reset

func Reset()

Reset forgets every opened bucket, for a test that changes the configuration between cases.

func SelfServed

func SelfServed(ctx context.Context) bool

SelfServed reports whether the configured set holds a bucket that serves its own presigned URLs, which is when the framework mounts SignedHandler.

func SignedHandler

func SignedHandler() http.Handler

SignedHandler serves requests under SignedPathPrefix for every configured bucket whose backend is SelfServing. A GET streams the object with its media type and ETag, a PUT stores the body under the signed key, and a request whose signature does not hold is 403; a bucket that does not serve itself is 404 here, because its URLs never point here.

func SignedHandlerFor

func SignedHandlerFor(open func(context.Context, string) (Bucket, error)) http.Handler

SignedHandlerFor is SignedHandler over a resolver of the caller's own, for a test that builds its buckets outside the configuration.

func Validate

func Validate(config pwruntime.StorageConfig) error

Validate is the startup check: every configured bucket names a registered backend and passes its own validation, so a missing blank import is a startup failure rather than a first-request one.

Types

type Bucket

type Bucket interface {
	Get(ctx context.Context, key string) (*Object, error)
	Head(ctx context.Context, key string) (*ObjectInfo, error)
	Put(ctx context.Context, key string, body io.Reader, options PutOptions) error
	Delete(ctx context.Context, key string) error
	List(ctx context.Context, options ListOptions) (*ListPage, error)
	Presign(ctx context.Context, key string, options PresignOptions) (*url.URL, error)
}

Bucket is one configured bucket. Every method takes a context, and every backend answers ErrNotFound for a missing key.

Presign returns a URL a client may use for one request against key without the application's credentials, for a bounded time. The S3 and R2 backends sign it with SigV4 query parameters, so the client talks to the store directly; the local backend answers with a path under SignedPathPrefix that the application serves itself, which is also what a deployment that exposes no bucket gets. A backend that cannot issue one as configured wraps ErrPresignUnavailable.

func Open

func Open(ctx context.Context, name string) (Bucket, error)

type Factory

type Factory func(ctx context.Context, config pwruntime.StorageBucketConfig) (Bucket, error)

Factory opens a bucket from its configuration. It is called once per bucket per process and the result kept.

type ListOptions

type ListOptions struct {
	Prefix string
	// Limit bounds one page; zero takes the backend's default.
	Limit int
	// Cursor continues a listing from a previous page's NextCursor.
	Cursor string
}

ListOptions page a listing by prefix.

type ListPage

type ListPage struct {
	Objects []ObjectInfo
	// NextCursor continues the listing when non-empty.
	NextCursor string
}

ListPage is one page of a listing.

type Object

type Object struct {
	ObjectInfo
	Body io.ReadCloser
}

Object is an object's metadata and its body. The body is a stream the caller closes; a caller needing a piece of it asks the backend for a range rather than seeking.

type ObjectInfo

type ObjectInfo struct {
	Key          string
	Size         int64
	ETag         string
	ContentType  string
	LastModified time.Time
	// Metadata is user metadata, without any backend prefix.
	Metadata map[string]string
}

ObjectInfo is what a store keeps beside an object's bytes.

type PresignOptions

type PresignOptions struct {
	// Method is GET, PUT, HEAD or DELETE; empty means GET.
	Method string
	// Expires bounds the URL's life; zero takes the backend's default, and
	// more than a backend accepts is an error rather than a clamp.
	Expires time.Duration
	// ContentType, when set, is signed on a PUT, so the sender must send
	// exactly it.
	ContentType string
	// Headers are further request headers the sender must reproduce, such
	// as Content-Disposition on a PUT.
	Headers map[string]string
}

PresignOptions describe the request a presigned URL authorizes.

type PutOptions

type PutOptions struct {
	// ContentType is the media type served back with the object. It is
	// required; a store that guesses from a key would guess differently on
	// two backends.
	ContentType string
	// ContentLength is the body's size when known, which lets a backend
	// stream rather than buffer.
	ContentLength int64
	// Metadata is user metadata kept beside the object.
	Metadata map[string]string
}

PutOptions describe a Put beyond its bytes.

type SelfServing

type SelfServing interface {
	VerifySignedRequest(r *http.Request, key string) (method string, err error)
}

SelfServing is a backend whose presigned URLs point back at the application. VerifySignedRequest checks the signature and the expiry of a request under SignedPathPrefix and returns the key it authorizes and the method it was signed for.

Directories

Path Synopsis
Package local keeps objects in a directory under the project, which is the backend api:cli-dev and a test run on: a developer loop that needs an S3 endpoint before a file can be stored is one nobody starts with.
Package local keeps objects in a directory under the project, which is the backend api:cli-dev and a test run on: a developer loop that needs an S3 endpoint before a file can be stored is one nobody starts with.
Package s3 reaches an S3-compatible store through system:tinygodriver's client: S3 itself, MinIO, and R2 through its S3 API from a process host.
Package s3 reaches an S3-compatible store through system:tinygodriver's client: S3 itself, MinIO, and R2 through its S3 API from a process host.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL