storage

package
v0.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 31, 2026 License: MIT Imports: 16 Imported by: 0

Documentation

Overview

Package storage keeps uploaded bytes outside the database. A Store is a small object store: Put streams bytes in and returns a generated key, Stat describes an object, Open reads a range of it and Delete removes it. There is no listing and no model field: the host keeps the key and the metadata in its own model, authorizes every access in its own View, and can use Serve to deliver an object it has authorized.

A Store never logs; it returns typed errors the host matches with errors.Is. Keys are always generated here (see NewKey), never derived from a client filename.

Index

Constants

View Source
const (
	// DefaultUploadField is the form field Upload reads when
	// UploadOptions.Field is empty.
	DefaultUploadField = "file"
)

Variables

View Source
var (
	// ErrNotFound is returned for a key with no object.
	ErrNotFound = errors.New("storage: object not found")
	// ErrExists is returned when a create-only write meets an existing key.
	// With generated keys this is a collision and is retried before it
	// reaches the caller.
	ErrExists = errors.New("storage: object already exists")
	// ErrTooLarge is returned for bytes past PutOptions.MaxSize. Match the
	// limit with errors.As and *TooLargeError.
	ErrTooLarge = errors.New("storage: object is too large")
	// ErrTypeNotAllowed is returned for a sniffed type outside
	// PutOptions.AllowedTypes. Match the type with errors.As and
	// *TypeNotAllowedError.
	ErrTypeNotAllowed = errors.New("storage: content type is not allowed")
	// ErrInvalidKey is returned for any key that is not in the form NewKey
	// produces.
	ErrInvalidKey = errors.New("storage: invalid key")
	// ErrInvalidRange is returned for an Open offset that is negative or
	// beyond the end of the object.
	ErrInvalidRange = errors.New("storage: invalid range")
	// ErrInvalidOptions is returned for PutOptions that cannot be honoured,
	// such as a MaxSize that is not positive.
	ErrInvalidOptions = errors.New("storage: invalid options")
)
View Source
var (
	// ErrNotMultipart is returned for a request that is not
	// multipart/form-data.
	ErrNotMultipart = errors.New("storage: request is not multipart/form-data")
	// ErrNoFile is returned when the request carries no file in the field
	// Upload reads.
	ErrNoFile = errors.New("storage: request has no file in the upload field")
	// ErrMultipleFiles is returned when the request carries more than one
	// file part. Nothing is left stored.
	ErrMultipleFiles = errors.New("storage: request has more than one file")
	// ErrTooManyParts is returned when the request has more parts than
	// Upload reads. Nothing is left stored.
	ErrTooManyParts = errors.New("storage: request has too many parts")
)

Functions

func ETagFor

func ETagFor(sha256hex string) string

ETagFor returns the quoted ETag for an object whose SHA-256 is sha256hex.

func NewKey

func NewKey() string

NewKey returns a new object key: 160 random bits as 32 lowercase base32 characters. A key is opaque. It never carries a filename or an extension, and it is not an authorization decision.

func Serve

func Serve(w http.ResponseWriter, r *http.Request, store Store, key string, opts ServeOptions) error

Serve writes the object at key to w for a GET or HEAD request. The host calls it only after it has authorized the request: Serve checks nothing about who is asking, and never uses a key, a filename or a client-declared type as an access decision.

It supports a single Range, If-None-Match, If-Modified-Since and If-Range; several ranges, or a range it cannot parse, get the whole object. The Content-Type is the type sniffed when the object was stored, and every response carries X-Content-Type-Options: nosniff. A missing object and an invalid key give the same 404.

Serve returns nil whenever it answered, and returns any unexpected store error without writing a response, so the host's View can return it and let the framework answer 500 and log it. Serve itself never logs.

func ValidKey

func ValidKey(key string) bool

ValidKey reports whether key has the form NewKey produces. Every Store refuses any other string with ErrInvalidKey before it touches storage, so a key can never name a path.

Types

type Info

type Info struct {
	Key         string
	Size        int64
	SHA256      string
	ContentType string
	// ETag is the quoted validator, derived from the SHA-256.
	ETag    string
	ModTime time.Time
}

Info describes a stored object.

type Inspector

type Inspector struct {
	// contains filtered or unexported fields
}

Inspector wraps the reader of a write. Every Store uses it, so size, type and hash behave the same everywhere. It sniffs the content type from the first bytes, enforces opts, and hashes and counts what passes through.

func Inspect

func Inspect(r io.Reader, opts PutOptions) (*Inspector, error)

Inspect reads the start of r, sniffs its content type, and refuses it with ErrTypeNotAllowed before any more is read. Read the returned Inspector to the end, then call Object. Reading past opts.MaxSize fails with ErrTooLarge.

func (*Inspector) Object

func (i *Inspector) Object(key string) Object

Object returns what was read, under key. Call it after Read returned io.EOF.

func (*Inspector) Read

func (i *Inspector) Read(p []byte) (int, error)

Read implements io.Reader.

type Object

type Object struct {
	// Key is the generated, opaque key. Keep it; it is the only handle.
	Key string
	// Size is the number of bytes stored.
	Size int64
	// SHA256 is the lowercase hex SHA-256 of the bytes.
	SHA256 string
	// ContentType is the type sniffed from the bytes, such as "image/png".
	ContentType string
}

Object is what a successful Put stored. The host copies it into its own metadata model.

type PutOptions

type PutOptions struct {
	// MaxSize is the most bytes the object may hold. It must be positive.
	MaxSize int64
	// AllowedTypes, when not empty, lists the sniffed media types accepted:
	// "image/png", or "image/*" for any subtype. Parameters and case are
	// ignored. Nothing a client declares is consulted.
	AllowedTypes []string
}

PutOptions are the host's rules for one write.

type ServeOptions

type ServeOptions struct {
	// Filename is the name offered to the browser in Content-Disposition. It
	// comes from the host (its metadata), never from the stored object. It
	// is reduced to a base name, stripped of control characters, and encoded
	// per RFC 6266 and RFC 5987; if nothing usable is left no name is sent.
	Filename string
	// InlineTypes lists the stored media types the browser may show inline,
	// such as "image/png" or "image/*". Every other type is sent as an
	// attachment. Active content (HTML, XHTML, SVG, XML, JavaScript, CSS) is
	// always an attachment, even when listed, so a stored file cannot run
	// in the application's origin.
	InlineTypes []string
	// CacheControl replaces the default "private, no-store".
	CacheControl string
	// SkipSandboxPolicy omits the default Content-Security-Policy that sandboxes
	// a served object. Set it only when the object is trusted to run its own
	// active content, or when a browser must render it outside the sandbox.
	// Nothing else in the response changes.
	SkipSandboxPolicy bool
}

ServeOptions shape one response from Serve.

type Store

type Store interface {
	// Put streams r into a new object and returns its generated key. The
	// write is atomic (the object is fully visible or absent) and
	// create-only. It always sniffs the content type from the stream, and
	// fails with ErrTooLarge past opts.MaxSize or ErrTypeNotAllowed outside
	// opts.AllowedTypes, leaving nothing stored.
	Put(ctx context.Context, r io.Reader, opts PutOptions) (Object, error)
	// Stat describes the object at key, or fails with ErrNotFound.
	Stat(ctx context.Context, key string) (Info, error)
	// Open reads length bytes of the object at key from offset; a negative
	// length, or one that reaches or exceeds the remaining bytes (math.MaxInt64
	// included), reads to the end. An
	// offset beyond the end fails with ErrInvalidRange. The caller closes
	// the reader.
	Open(ctx context.Context, key string, offset, length int64) (io.ReadCloser, error)
	// Delete removes the object at key. Deleting a missing object is not an
	// error.
	Delete(ctx context.Context, key string) error
}

Store is an object store. Implementations are safe for concurrent use and pass the conformance suite in package storagetest.

type TooLargeError

type TooLargeError struct{ Max int64 }

TooLargeError carries the limit a write exceeded. It matches ErrTooLarge.

func (*TooLargeError) Error

func (e *TooLargeError) Error() string

func (*TooLargeError) Is

func (e *TooLargeError) Is(target error) bool

type TypeNotAllowedError

type TypeNotAllowedError struct{ Type string }

TypeNotAllowedError carries the type that was sniffed. It matches ErrTypeNotAllowed.

func (*TypeNotAllowedError) Error

func (e *TypeNotAllowedError) Error() string

func (*TypeNotAllowedError) Is

func (e *TypeNotAllowedError) Is(target error) bool

type UploadOptions

type UploadOptions struct {
	// Field is the form field holding the file. It defaults to "file".
	Field string
	// PutOptions are passed to Store.Put unchanged, so the size cap and the
	// allowed-types policy apply exactly as they do to a direct Put.
	PutOptions
}

UploadOptions are the host's rules for reading one upload.

type UploadedFile

type UploadedFile struct {
	Object
	// Filename is the client's filename reduced to a safe display name: no
	// directories, no control or format characters, valid UTF-8, at most 255 bytes.
	// It may be empty.
	Filename string
	// DeclaredType is the Content-Type the client sent for the part. It is
	// not the stored type; Object.ContentType is the sniffed one.
	DeclaredType string
}

UploadedFile is what Upload stored: the Object, plus what the client said about it. Filename and DeclaredType are host metadata only. They are never used as a key, a path, a validation input or an authorization decision.

func Upload

func Upload(store Store, r *http.Request, opts UploadOptions) (UploadedFile, error)

Upload reads the one file in a multipart/form-data request and stores it with store.Put. It streams the file: nothing is buffered in memory, and a file past opts.MaxSize is refused with ErrTooLarge as soon as the cap is passed, without reading the rest. Other form fields are read, bounded, and ignored.

The request must hold exactly one file part, within a bounded number of parts; otherwise Upload fails with ErrNoFile, ErrMultipleFiles, ErrTooManyParts or ErrNotMultipart and leaves nothing stored. A request body cut off by a body limit such as tango.MaxBodySize surfaces as ErrTooLarge too. Map these to statuses in the calling View; Upload writes no response and logs nothing.

Directories

Path Synopsis
gcs module
Package local is a storage.Store on the local filesystem, for development, tests and single-host deployments with a persistent volume.
Package local is a storage.Store on the local filesystem, for development, tests and single-host deployments with a persistent volume.
s3 module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL