Documentation
¶
Overview ¶
Package storage reads and writes files on disks: the local filesystem (through os.Root, so paths can never escape the disk) and memory (tests); S3-compatible storage lives in the drivers/s3 module.
disk, err := storage.NewLocal("storage/app/public", "/storage")
path, err := storage.PutUploaded(ctx, disk, fileHeader, "avatars", storage.Images(2<<20))
url := disk.URL(path)
Every disk implements Disk; Clean validates paths, Handler serves a disk over HTTP and TemporaryURL signs links to private files. Streaming, sniffed uploads live in package upload.
See https://github.com/AlexGalhardo/vukem/blob/main/docs/subsystems.md#storage.
Example ¶
package main
import (
"context"
"fmt"
"github.com/AlexGalhardo/vukem/storage"
)
func main() {
ctx := context.Background()
// Memory is for tests; apps use storage.NewLocal or drivers/s3.
disk := storage.NewMemory("/storage")
_ = storage.PutBytes(ctx, disk, "avatars/1.txt", []byte("hello"))
data, _ := storage.Get(ctx, disk, "avatars/1.txt")
paths, _ := disk.List(ctx, "avatars")
fmt.Println(string(data), paths, disk.URL("avatars/1.txt"))
}
Output: hello [avatars/1.txt] /storage/avatars/1.txt
Index ¶
- Variables
- func Check(ctx context.Context, disk Disk) error
- func Clean(p string) (string, error)
- func Get(ctx context.Context, d Disk, p string) ([]byte, error)
- func Handler(d Disk, signer *signed.Signer) http.Handler
- func PutBytes(ctx context.Context, d Disk, p string, data []byte) error
- func PutUploaded(ctx context.Context, d Disk, fh *multipart.FileHeader, dir string, ...) (string, error)
- func TemporaryURL(signer *signed.Signer, prefix, p string, ttl time.Duration) (string, error)
- type Disk
- type Local
- func (l *Local) Close() error
- func (l *Local) Delete(_ context.Context, p string) error
- func (l *Local) Exists(_ context.Context, p string) (bool, error)
- func (l *Local) List(_ context.Context, dir string) ([]string, error)
- func (l *Local) Open(_ context.Context, p string) (io.ReadCloser, error)
- func (l *Local) Put(_ context.Context, p string, r io.Reader) error
- func (l *Local) URL(p string) string
- type Memory
- func (m *Memory) Delete(_ context.Context, p string) error
- func (m *Memory) Exists(_ context.Context, p string) (bool, error)
- func (m *Memory) List(_ context.Context, dir string) ([]string, error)
- func (m *Memory) Open(_ context.Context, p string) (io.ReadCloser, error)
- func (m *Memory) Put(_ context.Context, p string, r io.Reader) error
- func (m *Memory) URL(p string) string
- type UploadRules
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrInvalidPath = errors.New("storage: invalid path")
ErrInvalidPath rejects absolute paths, "..", backslashes and empty names.
var ErrNotFound = fs.ErrNotExist
ErrNotFound is returned (wrapped) for missing files.
var ErrUnhealthy = errors.New("storage: disk is not healthy")
ErrUnhealthy reports that a disk did not complete a round trip. Every error Check returns wraps it, and none of them wraps the driver's own: an S3 error carries the endpoint, the bucket and sometimes the access key, and the error a health handler receives is the one a naive handler puts in the response body.
Functions ¶
func Check ¶ added in v0.30.0
Check reports whether a disk can actually store a file: it writes a small object under a unique key, reads it back, compares the bytes and deletes it. Use it in a readiness check (App.AddHealthCheck), never in a liveness probe — it does real I/O against the remote.
The round trip is the point. The obvious check is whether a known object exists, and that one passes when nothing works: given the wrong credentials several S3 implementations answer "object does not exist" rather than refusing, so an existence check reports a healthy disk while no upload can succeed. Only a write distinguishes the two, and reading the bytes back also catches a disk that is up and returning the wrong object.
The delete runs even when ctx is already done, through context.WithoutCancel, so a probe the client abandoned does not leave an object behind on every request. Errors wrap ErrUnhealthy and never the driver's error.
func Clean ¶
Clean validates a relative path and returns it normalized.
Example ¶
package main
import (
"errors"
"fmt"
"github.com/AlexGalhardo/vukem/storage"
)
func main() {
p, err := storage.Clean("a/./b.txt")
fmt.Println(p, err)
_, err = storage.Clean("../etc/passwd")
fmt.Println(errors.Is(err, storage.ErrInvalidPath))
}
Output: a/b.txt <nil> true
func Handler ¶
Handler serves a disk's files under a URL prefix (mount it with http.StripPrefix). Directory listings, dotfiles and files under dot directories are never served; responses carry a sandbox CSP. Pass a Signer to require signed URLs (private files, TemporaryURL).
func PutUploaded ¶
func PutUploaded(ctx context.Context, d Disk, fh *multipart.FileHeader, dir string, rules UploadRules) (string, error)
PutUploaded stores an uploaded file under dir with a random name and an extension derived from the detected type, after checking its size and type. It returns the stored path. The client's file name is never used.
Types ¶
type Disk ¶
type Disk interface {
Put(ctx context.Context, p string, r io.Reader) error
Open(ctx context.Context, p string) (io.ReadCloser, error)
Exists(ctx context.Context, p string) (bool, error)
Delete(ctx context.Context, p string) error
// List returns the file paths under dir (recursively), sorted.
List(ctx context.Context, dir string) ([]string, error)
// URL is the public URL of p ("" for private disks).
URL(p string) string
}
Disk stores files under slash-separated relative paths.
type Local ¶
type Local struct {
// contains filtered or unexported fields
}
Local is a disk on the local filesystem. Every operation goes through an os.Root, so symlinks and ".." cannot leave the directory.
func NewLocal ¶
NewLocal opens (creating it if needed) dir as a disk; baseURL prefixes public URLs ("" for a private disk). Close it on shutdown.
type Memory ¶
type Memory struct {
// contains filtered or unexported fields
}
Memory is an in-memory disk for tests.
type UploadRules ¶
type UploadRules struct {
// MaxSize in bytes (required).
MaxSize int64
// Types are allowed MIME types, detected from the content (not the
// client's Content-Type or file name) (required: an empty list is a
// configuration error, never "any type").
Types []string
}
UploadRules restrict PutUploaded.
func Images ¶
func Images(maxSize int64) UploadRules
Images allows JPEG, PNG, GIF and WebP up to maxSize bytes.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package storagetest is the conformance suite every storage.Disk passes (local and memory in core, S3 in drivers/s3).
|
Package storagetest is the conformance suite every storage.Disk passes (local and memory in core, S3 in drivers/s3). |