upload

package
v0.0.0-...-3ac6500 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 24, 2026 License: Apache-2.0 Imports: 34 Imported by: 0

README

upload

github.com/ubgo/filesystem/upload is how bytes arrive from browsers and phones: streaming server uploads, multipart form posts, presigned direct uploads confirmed by the server, browser multipart uploads in the protocol of Uppy's AWS S3 plugin, imports from a URL behind an SSRF guard, and sweeping of uploads nobody finished. Every path enforces the same Policy against the bytes actually stored.

The concepts, guarantees and examples are in Uploads and Security.

The policy every example in the uploads guide uses:

policy · examples/concepts-uploads/main.go:43-47

var policy = upload.Policy{
	MaxSize: maxUpload,
	Allowed: []string{"image/*", "application/pdf"},
	Key:     upload.RandomKey(uploadDir),
}

Policy

Name What it is
Policy what an upload path accepts and where it stores it: MaxSize, MinSize, Allowed, Denied, AllowActiveContent, AllowActive (named active types accepted while every other protection stays on, such as SVG for contrib/svg), Key, KeyContext, Visibility (applied by Receive and Uppy, and by tickets only with SignVisibility), AllowPublic (lets a ticket request ask for a public object; off, such a request is CodePermission), SignVisibility (opt-in: Presign signs Visibility into every ticket whatever the client sends; a request for another visibility is CodePermission; needs a Visibility, and public still needs AllowPublic, else Validate answers CodeInvalidArgument), Checksums (more filesystem.Checksum digests beside SHA-256, reported in Result.Checksums: Receive has Disk.Write compute them, Confirm reads the stored object once, multipart included; a Want that does not match is CodeInvalidArgument in Receive and CodeRejected, object deleted, in Confirm; nil computes nothing), Headers, Collision, Extensions, MatchExtension, Detector (a sniff.Detector for formats the built-in detection does not know, asked only when the bytes come back octet-stream, its answer still checked by every rule), Process, Inspect
Policy.Validate() checks the policy itself (a size, at least one type, well-formed extensions, a known visibility, not both Key and KeyContext); CodeInvalidArgument otherwise
Policy.Accepts(contentType) whether the type is allowed and not denied; CodeRejected otherwise
Policy.AcceptsName(name, contentType) the name rules (DefaultDeniedExtensions, Extensions, MatchExtension); CodeRejected otherwise
DefaultDenied types refused unless AllowActiveContent: sniff.ActiveTypes plus Windows and Linux executables and WebAssembly
DefaultDeniedExtensions file-name extensions refused in any dotted segment unless AllowActiveContent: server-side scripts and executables (.php, .jsp, .exe, .htaccess, …)
KeyFunc func(Info) (string, error): chooses a storage key
KeyContextFunc, Policy.KeyContext func(ctx, Info) (string, error): a key function that also gets the upload's context (the request's, for the handlers), so a key can follow the tenant a middleware stored on it without a Resolver; set Key or KeyContext, not both
ShardedKey(dir, depth) a KeyFunc like RandomKey that splits the random name into depth levels of two-character directories (photos/q3/7a/q37a….png), so a local disk never holds one huge directory; depth in [1, MaxShardDepth] (4), otherwise every key fails with CodeInvalidArgument
NameKey(dir) a KeyFunc storing under dir by the client's name made safe, for products where users pick names; pair it with Policy.Collision
Collision what an upload does when its key holds an object: CollisionOverwrite (default), CollisionFail (CodeExists), CollisionRename (the first free name (1).ext); fail and rename write with IfNotExists and need ConditionalWrite; Presign refuses anything but overwrite
FreeKey(ctx, disk, key) key when free, else the first free of name (1).ext … name (100).ext (MaxRenameAttempts), else CodeExists; it only looks, so write with IfNotExists
RandomKey(dir) a KeyFunc storing under dir with 128 random bits and the detected type's extension; the default
HeadersFunc func(Info) (filesystem.HTTPHeaders, error): chooses the HTTP headers stored with a file (Policy.Headers); stored by Receive and Uppy, pinned in the signed PUT by Presign where the disk can (CodeUnsupported where it cannot); the disk needs Capabilities.Headers
StaticHeaders(h) a HeadersFunc storing h with every file
Attachment(cacheControl) a HeadersFunc storing attachment; filename=<the client's safe name> (RFC 2231 encoded when needed) and the given Cache-Control
Info what a KeyFunc, HeadersFunc or Processor sees: Filename (made safe) and ContentType (detected)
SafeFilename(name) a client's file name reduced to a safe display name: base name only, no control characters, at most 255 bytes, file when empty
Processor func(ctx, Info, io.Reader) (io.Reader, Info, error): transforms a file before it is stored (Policy.Process)
Inspector func(ctx, *filesystem.Disk, Result) error: examines a stored file; an error deletes it (Policy.Inspect)
Inspectors(list...) runs several inspectors in order, stopping at the first error; nil entries are skipped

Server-side uploads

Name What it is
Receive(ctx, disk, body, filename, policy) stores one stream: detect, check, process, key, stream with the size cap, inspect
Result a stored upload: Key, Filename, ContentType, Size, SHA256, Checksums (JSON tags key, filename, content_type, size, sha256, checksums; Checksums is nil and omitted unless Policy.Checksums asked for digests, which makes Result a struct to compare with reflect.DeepEqual, not ==)
Handler(disk, policy, HandlerOptions) a multipart/form-data POST handler; all files or none, or Partial
HandlerOptions Field (default DefaultField), MaxFiles (default 1), Partial, MaxTotal (bytes of all files of one request together; CodeTooLarge beyond), Resolve, OnStored, Errors (an fshttp.ErrorWriter for every failed request; nil is JSONErrors), Respond (an upload.SuccessWriter for the success body; nil is JSONSuccess, unchanged)
Resolver func(r *http.Request, base Policy) (Policy, error): the policy for one request (per-plan limits, per-user keys), run before any byte of the body is read, so also the hook for authentication, quota and rate-limit refusals; its error is answered with its code's status, fshttp.ClientMessage and any Retry-After; a returned policy that fails Validate is answered 500 with the status text only
DefaultField "file", the form field Handler reads
Response the success body: files, and failed with Partial
FileFailure one file a Partial request did not store: filename, error, code
ErrorResponse the body of a failed request: error, code
SuccessWriter, SuccessBody, JSONSuccess, ChunkProgress func(w, r, status, body SuccessBody): renders a successful response in the application's own envelope; SuccessBody is the closed set Response, ChunkProgress, Ticket, Limits; Handler, PresignHandlerWith, Chunked and LimitsHandler take one as Respond, and nil is JSONSuccess, byte for byte the default body. Uppy and tus bodies are protocol-fixed and have none
JSONErrors upload's default fshttp.ErrorWriter, writing an ErrorResponse with fshttp.ClientMessage; every handler here takes an Errors option to answer in another envelope instead

Direct uploads

Name What it is
TicketRequest what a client declares: filename, content_type, size
Ticket an authorised direct upload: key, method, url, headers, expires_at
Presign(ctx, disk, TicketRequest, policy, expires) checks the declaration and signs a PUT pinning the content type and the policy's headers (listed in Ticket.Headers, which the client must send); the disk needs Presign or a URL signer, and for headers PresignHeaders or a header-pinning signer
Policy.PostForm Presign issues a presigned POST form (Ticket.Fields, method POST) through Disk.PresignPost: the backend enforces the exact declared size, the type and the key; needs PresignPost
PresignHandler(disk, policy, expires) answers POSTed TicketRequest JSON with a Ticket
PresignHandlerWith(disk, policy, expires, PresignHandlerOptions{Resolve, Idempotency, Errors, Respond}) PresignHandler with a per-request Resolver, an ErrorWriter and a SuccessWriter
TicketRequest Filename, ContentType, Size, an optional SHA256 (hex; its form is checked) for ConfirmExpected, and an optional Visibility signed into the upload (public needs Policy.AllowPublic; the disk needs PresignVisibility, S3 and GCS in ACL mode, else CodeUnsupported; empty signs nothing, as before)
ConfirmExpected(ctx, disk, key, filename, policy, Expected{Size, SHA256}) Confirm that also refuses, deleting the object, a stored size or SHA-256 other than the declared one; zero fields are not checked
Confirm(ctx, disk, key, filename, policy) checks the stored object (size, detected type, name, a stored, declared Content-Type that would make a browser render the bytes as an active or text type they are not (aliases such as audio/wav for audio/wave pass; disks without the ContentType capability are not compared), Inspect) and deletes it if refused; the client must send the Content-Type the Ticket carries, because a backend that cannot pin it (an Azure SAS) stores whatever was sent
Chunked(disk, policy, ChunkedOptions{Staging, Owner, Fields, MaxChunkBytes, MaxUploadsPerOwner, AssemblyTimeout, StagingDisk, Resolve, OnStored, Now, Errors, Respond}) Dropzone, FilePond and resumable.js chunk protocols: chunks in any order and retried, staged per owner and upload ID within the declared size, progress in a compare-and-swap manifest, assembled through Receive by the request that completes them (201; others 200 with progress); returns an error at setup when the staging disk lacks ConditionalWrite and CompareAndSwap; the per-owner cap is one compare-and-swap of an owner index, and an assembler taken over after AssemblyTimeout is fenced by a manifest generation; Now (nil means time.Now) is the clock claims are stamped and judged by, so takeover is testable with a fake clock
ChunkFields, DropzoneFields(), ResumableFields() the form field names of a chunk library
Idempotency{Store, Owner}, HandlerOptions.Idempotency, PresignHandlerOptions.Idempotency honours the Idempotency-Key header: a retry of the same request gets the first response (200, Idempotent-Replayed: true), the key reused for another request 422, a running twin 409, a failure is not recorded; keys scoped by owner
IdempotencyStore, Recorded (Fingerprint, Body, ContentType: a replay answers with the first response's type, application/json when none was kept; a store that persists Recorded keeps all three), NewMemoryIdempotency(MemoryIdempotencyOptions{TTL, MaxEntries, Now}) the response store (Reserve, Complete, Release) and the in-process one, bounded by TTL and entry count
LimitsHandler(policy, LimitsOptions{MaxWidth, MaxHeight, Resolve, Errors, Respond}), LimitsOf, Limits the JSON limits a client follows to resize and filter files before uploading
SafePath(name), FolderKey(dir), Result.Path, Info.Path folder uploads: the client's relative path made safe, and a KeyFunc keeping the layout under dir
Policy.Progress, ProgressStep bytes received, reported about every 64 KiB and at the end
ContentAddressed(dir, ContentAddressing{Owner, Shared}), Policy.Settle, Settler content-derived keys through the Settle hook (which runs after Inspect and may move the upload): duplicates share one object within one owner (dir/<owner digest>/<sha256>.<ext>) by default, across owners only with Shared, and only with the same visibility and headers
Uppy(disk, policy, UppyOptions) the endpoints of Uppy's AWS S3 multipart plugin; the disk needs Multipart and PresignPart, otherwise every route answers 501
UppyOptions Expires (part URL lifetime, default 15 minutes), Authorize(r, key) (checks every request that continues an upload), Resolve (per-request policy; must give one upload the same limits at create and complete), OnStored(ctx, r, Result) (runs after the completed file passes Confirm, typically to create its record; its error deletes the file and fails the request), Errors (an fshttp.ErrorWriter; nil is JSONErrors). The complete request may carry filename beside parts; it is checked by the policy's name rules and reported in Result.Filename

Upload sessions

A server-issued token valid for a window (UP-15): one key or key prefix, limits no wider than the endpoint's policy, an expiry always judged on the server. Guide: Upload sessions.

Name What it is
NewSessions(store, SessionsOptions{Now}) issues and redeems sessions over a SessionStore; a nil store is CodeInvalidArgument
Sessions.Issue(ctx, policy, SessionRequest{Key|Prefix, MaxSize, Allowed, TTL}) records a session and returns IssuedSession{Session, Token}, the token shown only here; a request that widens the policy, has neither or both of Key and Prefix, or a TTL outside (0, MaxSessionLifetime] is CodeInvalidArgument
Sessions.Redeem(ctx, token, policy) the policy narrowed to the session (size, types re-checked against the policy as it is now, key or prefix); any unusable token is one CodePermission error
Sessions.Resolver() a Resolver reading HeaderUploadSession (Upload-Session), for Handler, Chunked, Uppy, PresignHandlerWith and LimitsHandler
Sessions.Revoke(ctx, id), Sessions.Sweep(ctx) end a session now; delete the expired ones (schedule it)
Session, Session.Expired(now) what a store keeps: ID, SecretHash (SHA-256 of the secret, never the secret), Key or Prefix, MaxSize, Allowed, CreatedAt, ExpiresAt
SessionStore, MemorySessions SaveSession, LoadSession, DeleteSession, SweepSessions, and the in-process store (zero value ready); records/uploadsessions is the database one
SessionIDPrefix, MaxSessionLifetime, MaxSessionTypes ups_, 7 days, 32

Imports from a URL

Name What it is
FromURL(ctx, disk, rawURL, policy, FetchOptions) fetches with the SSRF guard and stores through Receive
FromBase64(ctx, disk, encoded, filename, policy) decodes standard base64 while streaming and stores through Receive; invalid base64 is CodeInvalidArgument
FromDataURI(ctx, disk, uri, filename, policy) the same for a base64 data URI; its declared type is ignored, non-base64 URIs are CodeInvalidArgument
Promote(ctx, cache, cacheDir, key, store, storeKey) moves a cached upload to its permanent key (same disk: a move; another disk: copy, then delete); a key outside cacheDir is CodePermission and nothing moves; a root cacheDir is CodeInvalidArgument
Fetch(ctx, rawURL, FetchOptions) the guarded fetch alone; the caller closes Remote.Body
FetchOptions Timeout, MaxRedirects (negative follows none), MaxSize (refuses a larger announced length), AllowAddr (admits an address PublicAddr refuses), Now (the clock a Retry-After date is measured from; nil means time.Now), Header (sent with every request: a User-Agent, an Authorization for an internal source; Go drops credentials on a redirect to another host), Proxy (an http://host:port CONNECT proxy such as Smokescreen: the fetch still resolves the name, checks the address with PublicAddr and AllowAddr, and asks the proxy for that address, never the name, so the SSRF guard holds for the final destination)
FetchOptions.Client() the guarded *http.Client itself (the address checked after DNS resolution on every connection, no environment proxy, FetchOptions.Proxy only through the guarded tunnel, bounded redirects), for any other request to an address a user chose, such as webhooks
Remote a fetched response: Body, Filename, Size (-1 when unknown), URL (the URL finally fetched, after redirects, for resolving relative links)
PublicAddr(addr) whether an address is on the public internet; refuses loopback, private, link-local, CGNAT, multicast, reserved, NAT64 and IPv4-mapped forms
DefaultFetchTimeout one minute, for the whole fetch
DefaultMaxRedirects 5

Housekeeping

Name What it is
Sweep(ctx, disk, dir, cutoff) deletes every object under dir modified before cutoff and returns the count; the root is refused; deletes by age only

Errors

Every function returns *filesystem.Error codes: CodeRejected for a refused type, name or inspection; CodeTooLarge over a size limit; CodeInvalidArgument for an invalid policy, request, URL or address; CodeNotFound for a missing object (Confirm) or a 404/410 answer (Fetch); CodeUnavailable for network failures and 5xx answers; CodePermission when UppyOptions.Authorize refuses or an upload session token is unusable. Handlers map codes to statuses with fshttp.StatusOf and pass a backend's requested wait (filesystem.Error.RetryAfter) on as a Retry-After header. A JSON error message is the error's own cause, and only for codes about the client's input; the annotated text, which names the disk and the storage key, is never sent. See Errors.

Documentation

Overview

Package upload is how bytes arrive from browsers and phones: streaming server uploads, presigned direct uploads confirmed by the server, browser multipart uploads in the protocol Uppy's AWS S3 plugin speaks, and the sweeping of uploads nobody finished.

Every path enforces the same Policy, and every check runs on the server against the bytes actually stored: the type is detected from the content, never taken from the client's Content-Type or file extension, and the size is counted while streaming, never taken from a header.

Index

Constants

View Source
const (
	// DefaultMaxChunkBytes caps one chunk unless ChunkedOptions says
	// otherwise; client libraries default to 1 to 10 MB.
	DefaultMaxChunkBytes = 16 << 20
	// MaxChunks bounds the chunks of one file.
	MaxChunks = 10000
	// DefaultMaxUploadsPerOwner is how many chunked uploads one owner may
	// have in progress unless ChunkedOptions says otherwise.
	DefaultMaxUploadsPerOwner = 16
	// DefaultAssemblyTimeout is how long an assembly may run before
	// another request may take it over.
	DefaultAssemblyTimeout = 15 * time.Minute
)

Limits of chunked uploads.

View Source
const (
	// DefaultFetchTimeout bounds a whole fetch, body included.
	DefaultFetchTimeout = time.Minute
	// DefaultMaxRedirects is how many redirects a fetch follows.
	DefaultMaxRedirects = 5
)

Defaults of FetchOptions.

View Source
const (
	// MaxSessionLifetime caps a session's TTL: a session is a window for
	// one upload flow, and a store that is never swept still empties
	// within it.
	MaxSessionLifetime = 7 * 24 * time.Hour
	// MaxSessionTypes caps SessionRequest.Allowed.
	MaxSessionTypes = 32
)

Session limits.

View Source
const DefaultField = "file"

DefaultField is the multipart form field Handler reads files from.

View Source
const DefaultIdempotencyEntries = 100_000

DefaultIdempotencyEntries bounds MemoryIdempotency unless its options say otherwise.

View Source
const HeaderIdempotencyKey = "Idempotency-Key"

HeaderIdempotencyKey is the request header a client sets to make an upload safe to retry (Stripe's convention): the same key returns the first response instead of storing the file again.

View Source
const HeaderIdempotentReplayed = "Idempotent-Replayed"

HeaderIdempotentReplayed marks a response replayed from an earlier request with the same Idempotency-Key.

View Source
const HeaderUploadSession = "Upload-Session"

HeaderUploadSession is the request header carrying an upload session's token. A header, not a query parameter, so the token stays out of access logs and Referer headers.

View Source
const MaxPathDepth = 32

MaxPathDepth bounds the directories SafePath keeps from a client's relative path; deeper folders keep their innermost directories.

View Source
const MaxRenameAttempts = 100

MaxRenameAttempts bounds how many suffixed names FreeKey tries.

View Source
const MaxShardDepth = 4

MaxShardDepth bounds ShardedKey's depth: four levels of 1,024 directories each already spread far more files than any disk holds.

View Source
const ProgressStep = 64 << 10

ProgressStep is how many bytes Receive streams between two calls of Policy.Progress: often enough for a progress bar, rarely enough that a callback per call costs nothing next to the write.

View Source
const SessionIDPrefix = "ups_"

SessionIDPrefix starts every session ID, so an ID says what it is in logs.

Variables

CollisionValues lists every Collision.

View Source
var DefaultDenied = append(slices.Clone(sniff.ActiveTypes),
	"application/x-msdownload",
	"application/x-executable",
	"application/wasm",
)

DefaultDenied lists the content types refused by every policy unless Policy.AllowActiveContent is set: the types that run script when a browser renders them (sniff.ActiveTypes), which served from the application's origin are stored XSS, and executables.

View Source
var DefaultDeniedExtensions = []string{
	".php", ".php3", ".php4", ".php5", ".php7", ".php8", ".phtml", ".phar", ".pht",
	".asp", ".aspx", ".ashx", ".asmx", ".cer", ".jsp", ".jspx", ".cgi", ".pl", ".py", ".rb",
	".sh", ".bash", ".zsh", ".ps1", ".psm1", ".bat", ".cmd", ".com", ".exe", ".dll", ".msi",
	".scr", ".vbs", ".vbe", ".wsf", ".jar", ".hta", ".htaccess", ".shtml", ".lnk",
}

DefaultDeniedExtensions are file-name extensions refused by every policy unless Policy.AllowActiveContent is set: server-side scripts and executables, which a web server or a user's machine may run. Every dotted segment of a name is checked, not only the last, because some servers execute "shell.php.jpg".

Functions

func Chunked

func Chunked(disk *filesystem.Disk, base Policy, opts ChunkedOptions) (http.Handler, error)

Chunked accepts files sent in chunks by Dropzone, FilePond, resumable.js and similar libraries: each POST carries one chunk as multipart form data, and chunks may arrive in any order and be retried. Chunks wait in opts.Staging, with their progress in a small manifest updated by compare-and-swap; the request that completes the set assembles them in order and stores the file through Receive, so the policy applies to the whole file exactly as for a single upload (detected type, size, name rules, Process, Inspect, Settle), then answers 201 with Response{Files}. Other chunks answer 200 with how many have arrived.

A first chunk whose type the policy refuses is refused at once; a declared total size over the policy's MaxSize, chunks that would stage more than the declared size, or that do not add up to it, are refused too, as is a new upload past the owner's MaxUploadsPerOwner.

The options are checked here, once: a Staging that is the root or not a valid directory, a nil Owner, or a staging disk without ConditionalWrite and CompareAndSwap is CodeInvalidArgument, so a misconfigured endpoint fails at startup rather than on every chunk.

func ContentAddressed

func ContentAddressed(dir string, o ContentAddressing) (KeyFunc, Settler)

ContentAddressed returns the Key and Settle steps of a policy whose files are named by their content, as Git, container registries and backup tools store blobs: identical files share one object, keys never collide, and a key proves its content. By default sharing stays within one owner: "<dir>/<owner digest>/<sha256>.<ext>", the owner coming from o.Owner:

p.Key, p.Settle = upload.ContentAddressed("blobs", upload.ContentAddressing{Owner: tenantFromContext})

With o.Shared every upload under dir shares, "<dir>/<sha256>.<ext>", and two consequences follow: deleting one owner's file deletes the object every other owner's identical file points to (count references, as records' FileBySHA256 allows, before deleting), and an upload is only deduplicated against an object stored with the same visibility and headers, so one owner's public upload can never make another's private file public (a duplicate with other settings is CodeConflict; the check applies within one owner too). Neither Owner nor Shared refuses every upload with CodeInvalidArgument.

The upload is first written under "<dir>/.incoming/" with a random name while the Disk hashes it, then moved to its content key. When that key already holds the same content (a duplicate upload) the incoming copy is deleted and the existing object reused; two identical uploads racing to one key both write the same bytes, so neither can corrupt the other.

func FindStale

func FindStale(ctx context.Context, disk *filesystem.Disk, dir string, cutoff time.Time, found func(filesystem.Entry)) (int, error)

FindStale is Sweep's dry run: it calls found with every object Sweep would delete for the same dir and cutoff, in listing order, and returns how many it found, deleting nothing. It refuses the root as Sweep does.

func FreeKey

func FreeKey(ctx context.Context, disk *filesystem.Disk, key string) (string, error)

FreeKey returns key when nothing is stored there, else the first of "dir/name (1).ext" … "dir/name (MaxRenameAttempts).ext" that is free, or CodeExists when all are taken. It only looks: write with WriteOptions.IfNotExists so a racing writer cannot be overwritten between this check and the write.

func Handler

func Handler(disk *filesystem.Disk, base Policy, opts HandlerOptions) http.Handler

Handler accepts multipart/form-data POST uploads into disk under policy p. Files stream straight to the disk (no temporary copy, no memory buffer of the whole body); the whole request is capped at MaxFiles × MaxSize plus a small allowance for form overhead. All files of a request are stored, or none: a failure deletes the ones already stored.

func JSONErrors

func JSONErrors(w http.ResponseWriter, _ *http.Request, status int, err error)

JSONErrors is upload's default fshttp.ErrorWriter: an ErrorResponse whose text is fshttp.ClientMessage(err), so internals never reach the client, and whose code is filesystem.CodeOf(err). Every handler of this package answers with it unless its options set Errors.

func JSONSuccess

func JSONSuccess(w http.ResponseWriter, _ *http.Request, status int, body SuccessBody)

JSONSuccess is the default SuccessWriter: body as JSON with status.

func LimitsHandler

func LimitsHandler(base Policy, o LimitsOptions) http.Handler

LimitsHandler answers GET with the JSON Limits of the policy (resolved per request when o.Resolve is set), the contract a client follows to downscale and filter files before uploading. Mount it beside the upload endpoint, behind the same authentication when limits depend on the user.

func PresignHandler

func PresignHandler(disk *filesystem.Disk, p Policy, expires time.Duration) http.Handler

PresignHandler answers POSTed TicketRequest JSON with a Ticket. Mount it behind the application's authentication. A KeyFunc sees only the file's Info, never the request, so to place keys per user build the handler per request with a Policy whose Key closes over that user:

http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	p := base
	p.Key = upload.RandomKey("users/" + userID(r))
	upload.PresignHandler(disk, p, time.Minute).ServeHTTP(w, r)
})

func PresignHandlerWith

func PresignHandlerWith(disk *filesystem.Disk, base Policy, expires time.Duration, opts PresignHandlerOptions) http.Handler

PresignHandlerWith is PresignHandler with options, chiefly a Resolver, so per-user limits and keys need no handler built per request.

func Promote

func Promote(ctx context.Context, cache *filesystem.Disk, cacheDir, key string, store *filesystem.Disk, storeKey string) (filesystem.Entry, error)

Promote moves an upload kept in a cache area to its permanent key, the second step of two-phase uploads: store the file under cacheDir as soon as it arrives (Receive with a RandomKey(cacheDir) policy), let the form be redisplayed or the record be validated, then promote it, possibly to another disk. Uploads that are never promoted are left for Sweep on cacheDir.

key usually comes back from the client, so it must lie under cacheDir on cache, or the call fails with CodePermission and nothing moves: a client cannot promote, and so take over, any other object. cacheDir must not be the root (CodeInvalidArgument). Within one disk this is a Move; across disks a copy, then the cached object's deletion (filesystem.MoveAcross).

func PublicAddr

func PublicAddr(a netip.Addr) bool

PublicAddr reports whether a is an address of the public internet. It refuses loopback, private (RFC 1918, unique local), link-local (cloud metadata services live at 169.254.169.254), multicast, unspecified, CGNAT, reserved and NAT64 addresses, and IPv4-mapped forms of any of them. Use it wherever a server connects to an address a user chose: imports, webhooks, link previews.

func SafeFilename

func SafeFilename(name string) string

SafeFilename reduces a client-supplied file name to something safe to store and display: the base name only (no directories, either separator), no control characters, at most 255 bytes, and "file" when nothing is left.

func SafePath

func SafePath(name string) string

SafePath reduces a client-supplied relative path from a folder upload ("Holiday/2024/IMG_1.jpg", either separator) to one safe to show and to use under a prefix: each segment made safe as SafeFilename does it, empty, "." and ".." segments dropped (a client cannot climb out of the folder it dropped), at most MaxPathDepth directories, and the file name last. A name without directories comes back as SafeFilename(name).

func Sweep

func Sweep(ctx context.Context, disk *filesystem.Disk, dir string, cutoff time.Time) (int, error)

Sweep deletes every object under dir last modified before cutoff and returns how many it deleted. Point it at the prefix where direct uploads land before Confirm (and where server uploads wait to be attached), and schedule it: uploads that are never confirmed would otherwise cost storage forever. The library starts no background goroutine; the application decides when sweeping runs.

It deletes as it lists, so a failure part-way returns the count so far with the error, and a rerun continues where it stopped.

dir must not be the root: sweeping a whole disk by age is never what an upload area wants, so it is refused with CodeInvalidArgument. An object whose backend reports no modification time is kept: its age is unknown, and a zero time would otherwise count as older than any cutoff.

func Uppy

func Uppy(disk *filesystem.Disk, p Policy, opts UppyOptions) http.Handler

Uppy returns the server side of browser multipart uploads, speaking the protocol of Uppy's AWS S3 plugin. Mount it with the prefix stripped:

mux.Handle("/uploads/s3/", http.StripPrefix("/uploads/s3", upload.Uppy(disk, policy, opts)))

Routes: POST /multipart (create), GET /multipart/{id}/{part} (sign a part), GET /multipart/{id} (list parts), POST /multipart/{id}/complete, DELETE /multipart/{id}. Every route but create takes the key as ?key=.

The declared type is checked at create; on complete the assembled object is checked with Confirm (size and detected type) and deleted if refused. The disk needs the Multipart and PresignPart capabilities (S3-compatible backends); on any other disk every route answers 501.

Types

type ChunkFields

type ChunkFields struct {
	// UploadID is the client's identifier of the whole file.
	UploadID string
	// Index is the chunk's number, counted from IndexBase.
	Index string
	// Total is how many chunks the file has.
	Total string
	// TotalSize is the whole file's size in bytes.
	TotalSize string
	// File is the field holding the chunk's bytes.
	File string
	// IndexBase is the first chunk's number: 0 for Dropzone, 1 for
	// resumable.js.
	IndexBase int
}

ChunkFields names the multipart form fields of one chunk, which differ between client libraries. Every field must come before the file part, as Dropzone and resumable.js send them.

func DropzoneFields

func DropzoneFields() ChunkFields

DropzoneFields are the field names of Dropzone's chunked uploads (and FilePond configured the same way).

func ResumableFields

func ResumableFields() ChunkFields

ResumableFields are the field names of resumable.js and flow.js.

type ChunkProgress

type ChunkProgress struct {
	Received int `json:"received"`
	Total    int `json:"total"`
}

ChunkProgress is the body Chunked answers a chunk that did not complete its file with: how many of the file's chunks have arrived, of how many.

type ChunkedOptions

type ChunkedOptions struct {
	// Staging is the directory where chunks wait until the file is
	// complete. Required, not the root. Sweep it (upload.Sweep, or the
	// jobs.UploadSweep job) for chunks of uploads nobody finished.
	Staging string
	// Owner names who sends a request (a user or tenant ID), so one
	// client can never add chunks to, or complete, another's upload.
	// Required: a deployment without users returns a constant.
	Owner func(r *http.Request) (string, error)
	// Fields are the client library's field names; the zero value means
	// DropzoneFields.
	Fields ChunkFields
	// MaxChunkBytes caps each chunk; 0 means DefaultMaxChunkBytes. A
	// chunk is also capped by what is left of the file's declared size,
	// so the chunks of one upload can never stage more than the file.
	MaxChunkBytes int64
	// MaxUploadsPerOwner caps the uploads one owner may have in progress
	// at once; 0 means DefaultMaxUploadsPerOwner. Starting one more is
	// CodeQuotaExceeded, so a client cannot stage without bound by
	// opening upload after upload. The count is exact under concurrency:
	// uploads are admitted into a small per-owner index by
	// compare-and-swap. An upload leaves it when it is assembled (or its
	// assembly fails); an abandoned one stops counting once Sweep has
	// removed its manifest, checked only when the owner is at the cap.
	MaxUploadsPerOwner int
	// AssemblyTimeout is how long an assembler may go without renewing
	// its claim before another request may take over (the first one
	// crashed or stalled); 0 means DefaultAssemblyTimeout. The assembler
	// renews its claim before settling the file (just before OnStored)
	// and before removing the chunks, and stops when it was taken over, so the
	// file is stored and reported once. Streaming the file is one step:
	// a stalled stream is the case a takeover exists for.
	AssemblyTimeout time.Duration
	// StagingDisk holds the chunks; nil means the destination disk. It
	// needs CompareAndSwap and ConditionalWrite, which keep an upload's
	// progress consistent across concurrent chunks and instances: point
	// it at a local or memory disk when the destination lacks them.
	StagingDisk *filesystem.Disk
	// Resolve gives each request its own policy; see Resolver. It must
	// return the same policy for every chunk of one file.
	Resolve Resolver
	// OnStored runs once the file is assembled and stored, typically to
	// create its database row; an error deletes the file and fails the
	// request.
	OnStored func(ctx context.Context, r *http.Request, file Result) error
	// Now is the clock assembly claims are stamped and judged by against
	// AssemblyTimeout; nil means time.Now. Every instance serving one
	// staging directory should share a clock, or a skewed one takes over
	// an assembly early or late. A fake clock makes takeover testable
	// without sleeping.
	Now func() time.Time
	// Errors renders every failed chunk request; see fshttp.ErrorWriter.
	// nil is JSONErrors.
	Errors fshttp.ErrorWriter
	// Respond renders every successful chunk request (a Response for the
	// chunk that completes the file, a ChunkProgress for the others); see
	// SuccessWriter. nil is JSONSuccess, today's bodies unchanged.
	Respond SuccessWriter
}

ChunkedOptions configure Chunked.

type Collision

type Collision string

Collision says what an upload does when its key already holds an object. It matters only for keys the client's name chooses (NameKey); random keys do not collide.

const (
	// CollisionOverwrite (the zero value) replaces the existing object.
	CollisionOverwrite Collision = ""
	// CollisionFail refuses the upload with CodeExists, atomically.
	CollisionFail Collision = "fail"
	// CollisionRename stores the upload under the first free name of
	// "photo (1).png", "photo (2).png", … (see FreeKey), as desktop
	// systems do, and still refuses with CodeExists, never overwriting,
	// if a racing upload takes that name first.
	CollisionRename Collision = "rename"
)

The closed set of collision policies.

type ContentAddressing

type ContentAddressing struct {
	// Owner names who is uploading (a user or tenant ID), from the
	// request's context, where the application's authentication put it.
	// Identical files are shared only within one owner's uploads. Required
	// unless Shared.
	Owner func(ctx context.Context) (string, error)
	// Shared lets every upload under dir share objects, across owners: a
	// deliberate choice, with the consequences ContentAddressed lists.
	Shared bool
}

ContentAddressing configures ContentAddressed: whose uploads may share an object.

type ErrorResponse

type ErrorResponse struct {
	Error string          `json:"error"`
	Code  filesystem.Code `json:"code"`
}

ErrorResponse is the JSON body of a failed request. Code is the filesystem.Code, so clients can tell "too large" from "wrong type".

type Expected

type Expected struct {
	Size   int64
	SHA256 string
}

Expected is what the client declared about a direct upload, for ConfirmExpected to hold the stored object to: the size it announced (catches a truncated or padded upload) and, when it sent one, the SHA-256 of its bytes (catches corruption and a swapped file). Zero fields are not checked.

type FetchOptions

type FetchOptions struct {
	// Timeout bounds the whole fetch; 0 means DefaultFetchTimeout.
	Timeout time.Duration
	// MaxRedirects caps redirects; 0 means DefaultMaxRedirects, a negative
	// value follows none.
	MaxRedirects int
	// MaxSize refuses a response that announces a larger Content-Length
	// before reading it (CodeTooLarge); 0 does not check. FromURL sets it
	// from the policy. The body itself is not capped here: Receive and
	// Disk.Write count it as it streams.
	MaxSize int64
	// AllowAddr permits an address PublicAddr refuses, for imports from a
	// known internal server. Every connection, redirects included, is
	// checked against PublicAddr and then AllowAddr.
	AllowAddr func(netip.Addr) bool
	// Now is the clock a Retry-After given as an HTTP date is measured
	// from, for the wait a CodeUnavailable error carries
	// (filesystem.Error.RetryAfter); nil means time.Now.
	Now func() time.Time
	// Header is sent with every request of the fetch, redirects included
	// (Go drops Authorization, Cookie and WWW-Authenticate on a redirect
	// to another host, so a credential meant for one server does not
	// follow a redirect elsewhere): a User-Agent for sites that refuse
	// Go's default, an Authorization for an internal source, an Accept.
	// Host cannot be set through it (the URL's host is always used). nil
	// sends Go's defaults only.
	Header http.Header
	// Proxy, when set, sends every connection through this HTTP proxy
	// with CONNECT (Smokescreen, Squid, an egress gateway): only
	// "http://host:port", optionally with user:password for
	// Proxy-Authorization. The SSRF guard still decides the destination:
	// the fetch resolves the target's name itself, checks the address
	// against PublicAddr and AllowAddr exactly as without a proxy, and
	// asks the proxy to CONNECT to that address, never to the name, so
	// the proxy cannot resolve it elsewhere (DNS rebinding) and an
	// internal address is refused before the proxy is asked. TLS still
	// verifies the target's name end to end. The proxy's own address is
	// not checked: it is the operator's choice, not the user's. nil
	// connects directly, and the environment's HTTP_PROXY is never read.
	Proxy *url.URL
}

FetchOptions configures Fetch and FromURL.

func (FetchOptions) Client

func (o FetchOptions) Client() *http.Client

Client returns the guarded HTTP client Fetch uses, for any other request to an address a user chose (webhooks, link previews): every connection, redirects included, goes only to addresses PublicAddr (or AllowAddr) accepts, checked after DNS resolution so a name that resolves or rebinds to an internal address is refused; it uses no proxy (a proxy would be dialed instead of the target, defeating the check), bounds the dial, TLS handshake and response headers, and follows at most MaxRedirects http(s) redirects (none when negative, where a redirect is an error). Timeout is not applied to it, since Fetch bounds each fetch with its context: set Client.Timeout on the result to bound a request. It keeps no connection alive, so build one per caller and reuse it.

type FileFailure

type FileFailure struct {
	Filename string          `json:"filename"`
	Error    string          `json:"error"`
	Code     filesystem.Code `json:"code"`
}

FileFailure is one file a Partial request did not store.

type HandlerOptions

type HandlerOptions struct {
	// Field is the form field holding files. Empty means DefaultField.
	Field string
	// MaxFiles caps the files per request. 0 means 1.
	MaxFiles int
	// Partial stores every file it can and reports the others in
	// Response.Failed, for bulk uploads where one bad file should not sink
	// the rest. Without it a request stores all its files or none, which
	// is right for a form whose files belong together. Request-level
	// failures (not multipart, over the request cap, malformed) still fail
	// the request, after deleting what it stored.
	Partial bool
	// Resolve chooses the policy for each request from the one given to
	// Handler, before the body is read; its error refuses the request.
	// See Resolver. nil uses the given policy.
	Resolve Resolver
	// MaxTotal, when positive, caps the bytes of all the files of one
	// request together, on top of the policy's per-file MaxSize: each file
	// may use only what the earlier ones left, and one that does not fit
	// is CodeTooLarge (a failed file under Partial). The request body is
	// capped at MaxTotal plus form overhead when that is lower than
	// MaxFiles × MaxSize.
	MaxTotal int64
	// Idempotency, when set, makes retries of one upload (same
	// Idempotency-Key header) return the first response instead of
	// storing the files again: offline queues and flaky mobile links
	// retry without duplicates. nil ignores the header.
	Idempotency *Idempotency
	// OnStored runs after the request's files are stored (with Partial, the
	// ones that were), typically to create database records. If it fails,
	// the stored files are deleted and the request fails.
	OnStored func(ctx context.Context, r *http.Request, files []Result) error
	// Errors renders every failed request (the policy's refusals, the
	// Disk's errors, a malformed form, a refused Idempotency-Key); see
	// fshttp.ErrorWriter. nil is JSONErrors, the ErrorResponse body. A
	// Partial request that stored nothing still answers its Response
	// with Failed, which is a result, not an error.
	Errors fshttp.ErrorWriter
	// Respond renders the Response of every request that stored files,
	// and of a Partial request that stored none; see SuccessWriter. nil
	// is JSONSuccess, today's body unchanged.
	Respond SuccessWriter
}

HandlerOptions configure Handler.

type HeadersFunc

type HeadersFunc func(Info) (filesystem.HTTPHeaders, error)

HeadersFunc chooses the HTTP headers stored with an upload; see Policy.Headers. Info.Filename is the client's name made safe, fit for a Content-Disposition filename and nothing else. An error refuses the upload and should carry a filesystem.Code.

func Attachment

func Attachment(cacheControl string) HeadersFunc

Attachment returns a HeadersFunc that makes every upload download under its original name rather than display in the browser: `attachment; filename=report.pdf` (mime.FormatMediaType quotes a name only when it is not a plain token, `filename="my report.pdf"`, and encodes one a quoted string cannot hold per RFC 2231, `filename*=utf-8”f%C3%AAte.png`), and cacheControl as the Cache-Control ("" stores none). Random keys lose the client's name; this is how a download gets it back.

func StaticHeaders

func StaticHeaders(h filesystem.HTTPHeaders) HeadersFunc

StaticHeaders returns a HeadersFunc that stores h with every upload, such as a long Cache-Control for random, never-reused keys:

p.Headers = upload.StaticHeaders(filesystem.HTTPHeaders{CacheControl: "public, max-age=31536000, immutable"})

type Idempotency

type Idempotency struct {
	// Store keeps the responses. Required.
	Store IdempotencyStore
	// Owner names who sent a request (a user or tenant ID), so one
	// client's key can never replay another's response. Required: a
	// deployment without users returns a constant.
	Owner func(r *http.Request) (string, error)
}

Idempotency makes an upload handler honour HeaderIdempotencyKey. A request without the header runs as usual.

type IdempotencyStore

type IdempotencyStore interface {
	// Reserve claims key for a request about to run. It returns the
	// recorded response and true when a request with key already finished,
	// or CodeConflict while one is still running; otherwise it reserves key
	// and returns false.
	Reserve(ctx context.Context, key string) (Recorded, bool, error)
	// Complete records the response of the request that reserved key.
	Complete(ctx context.Context, key string, r Recorded) error
	// Release drops a reservation whose request failed, so a retry runs.
	Release(ctx context.Context, key string) error
}

IdempotencyStore remembers responses by idempotency key for a while. It is the seam for sharing them across instances (a database- or Redis-backed store); MemoryIdempotency is the in-process one.

type Info

type Info struct {
	// Filename is the client's file name reduced to a safe base name
	// (SafeFilename). Display only.
	Filename string
	// ContentType is the type detected from the content.
	ContentType string
	// Path is the client's relative path made safe (SafePath), ending in
	// Filename: folder uploads keep their layout through it.
	Path string
}

Info describes an incoming file to a KeyFunc.

type Inspector

type Inspector func(ctx context.Context, disk *filesystem.Disk, r Result) error

Inspector examines a stored upload; see Policy.Inspect. It reads the file from disk at r.Key. Errors should carry a filesystem.Code (CodeRejected for a refused file, CodeUnavailable when the inspection itself could not run).

func Inspectors

func Inspectors(list ...Inspector) Inspector

Inspectors runs several inspectors in order, stopping at the first error: a virus scan, then a decode check, then a moderation call. nil entries are skipped.

type IssuedSession

type IssuedSession struct {
	Session
	Token string
}

IssuedSession is a new session with its token, which is shown only here: hand it to the client, which sends it in HeaderUploadSession.

type KeyContextFunc

type KeyContextFunc func(ctx context.Context, info Info) (string, error)

KeyContextFunc is a KeyFunc that also receives the request's context: the tenant or user an authentication middleware stored on it, a trace, a deadline for a lookup. Set it as Policy.KeyContext. The same rule holds: Info.Filename is cosmetic, never part of a key's identity.

type KeyFunc

type KeyFunc func(Info) (string, error)

KeyFunc chooses the storage key for an upload. It must not trust Info.Filename for anything but cosmetics: two users can upload the same name, and names can carry path tricks.

func FolderKey

func FolderKey(dir string) KeyFunc

FolderKey returns a KeyFunc that stores each file under dir at the relative path its client sent (Info.Path), so a dropped folder keeps its layout: "Holiday/2024/IMG_1.jpg" becomes "<dir>/Holiday/2024/IMG_1.jpg". Two files with one path meet at one key: choose what happens with Policy.Collision. Under several users' uploads, put the user in dir.

func NameKey

func NameKey(dir string) KeyFunc

NameKey returns a KeyFunc that stores files under dir by the client's file name made safe (SafeFilename), for products where users see and pick names (a shared drive). Two uploads of one name then meet at one key: choose what happens with Policy.Collision. The policy's name rules (DefaultDeniedExtensions, Extensions, MatchExtension) apply to the name.

func RandomKey

func RandomKey(dir string) KeyFunc

RandomKey returns a KeyFunc that stores files under dir with a random name and the extension of the detected type, e.g. "avatars/3QX…7A.png". The client's name never reaches the key, so it cannot collide, overwrite or smuggle a path.

func ShardedKey

func ShardedKey(dir string, depth int) KeyFunc

ShardedKey returns a KeyFunc like RandomKey that also splits the random name into depth levels of directories, two characters each: "photos/q3/7a/q37a…x2.png" for depth 2. A local disk then never holds a huge flat directory, which slows listing and some filesystems. depth must be in [1, MaxShardDepth]; any other value makes every key fail with CodeInvalidArgument, so the mistake shows at the first upload.

type Limits

type Limits struct {
	MaxBytes   int64    `json:"max_bytes"`
	MinBytes   int64    `json:"min_bytes,omitempty"`
	Allowed    []string `json:"allowed"`
	Denied     []string `json:"denied,omitempty"`
	Extensions []string `json:"extensions,omitempty"`
	MaxWidth   int      `json:"max_width,omitempty"`
	MaxHeight  int      `json:"max_height,omitempty"`
}

Limits is what a client needs to prepare a file before sending it: the sizes, types and extensions the policy accepts, and, when the server checks them, the largest image dimensions, so a phone resizes a photo before uploading rather than having it refused after.

func LimitsOf

func LimitsOf(p Policy, o LimitsOptions) Limits

LimitsOf returns the limits p enforces, for clients.

type LimitsOptions

type LimitsOptions struct {
	// MaxWidth and MaxHeight publish the image dimensions the server
	// enforces (for instance with an Inspect step that checks them); 0
	// publishes none. They describe the server's check; they do not add
	// one.
	MaxWidth, MaxHeight int
	// Resolve gives each request its own policy, as the upload handlers'
	// Resolve does, so a client sees the limits of its own plan.
	Resolve Resolver
	// Errors renders a failed request; see fshttp.ErrorWriter. nil is
	// JSONErrors.
	Errors fshttp.ErrorWriter
	// Respond renders the Limits; see SuccessWriter. nil is JSONSuccess,
	// today's body unchanged.
	Respond SuccessWriter
}

LimitsOptions configure LimitsHandler.

type MemoryIdempotency

type MemoryIdempotency struct {
	// contains filtered or unexported fields
}

MemoryIdempotency is an IdempotencyStore in this process's memory, bounded by TTL and MaxEntries. Responses do not survive a restart and are not shared between instances; behind a load balancer, use a shared store. Create it with NewMemoryIdempotency.

func NewMemoryIdempotency

func NewMemoryIdempotency(opts MemoryIdempotencyOptions) (*MemoryIdempotency, error)

NewMemoryIdempotency returns an in-memory store. A non-positive TTL or a negative MaxEntries is CodeInvalidArgument.

func (*MemoryIdempotency) Complete

func (m *MemoryIdempotency) Complete(_ context.Context, key string, r Recorded) error

Complete implements IdempotencyStore.

func (*MemoryIdempotency) Release

func (m *MemoryIdempotency) Release(_ context.Context, key string) error

Release implements IdempotencyStore.

func (*MemoryIdempotency) Reserve

func (m *MemoryIdempotency) Reserve(_ context.Context, key string) (Recorded, bool, error)

Reserve implements IdempotencyStore. It drops expired entries from the old end as it goes, touching only those.

type MemoryIdempotencyOptions

type MemoryIdempotencyOptions struct {
	// TTL is how long a response (or a running request's reservation) is
	// kept. Required, positive.
	TTL time.Duration
	// MaxEntries bounds the keys kept; 0 means DefaultIdempotencyEntries.
	// When full, the oldest entry is dropped: a flood of keys can then
	// cost an old retry its dedup, never the store's memory.
	MaxEntries int
	// Now is the clock; nil means time.Now.
	Now func() time.Time
}

MemoryIdempotencyOptions configure NewMemoryIdempotency.

type MemorySessions

type MemorySessions struct {
	// contains filtered or unexported fields
}

MemorySessions is a SessionStore in this process's memory: sessions do not survive a restart and are not shared between instances (behind a load balancer, use a shared store such as records/uploadsessions). The zero value is ready to use.

func (*MemorySessions) DeleteSession

func (m *MemorySessions) DeleteSession(_ context.Context, id string) error

DeleteSession implements SessionStore.

func (*MemorySessions) LoadSession

func (m *MemorySessions) LoadSession(_ context.Context, id string) (Session, error)

LoadSession implements SessionStore.

func (*MemorySessions) SaveSession

func (m *MemorySessions) SaveSession(_ context.Context, s Session) error

SaveSession implements SessionStore.

func (*MemorySessions) SweepSessions

func (m *MemorySessions) SweepSessions(_ context.Context, now time.Time) (int, error)

SweepSessions implements SessionStore.

type Policy

type Policy struct {
	// MaxSize is the largest accepted file in bytes. Required.
	MaxSize int64
	// MinSize, when positive, is the smallest accepted file in bytes;
	// MinSize: 1 refuses empty files, which are almost always a client bug
	// (a picker that lost its file, a cancelled read). 0 accepts any size.
	// Refused with CodeRejected; Receive and Confirm delete the stored
	// object, Presign refuses the declared size before signing.
	MinSize int64
	// Allowed lists accepted MIME patterns ("image/*", "application/pdf").
	// Required: there is no accept-everything default.
	Allowed []string
	// Denied lists refused patterns, checked after Allowed; DefaultDenied
	// is always added unless AllowActiveContent is set.
	Denied []string
	// AllowActiveContent lifts DefaultDenied and DefaultDeniedExtensions,
	// for endpoints that really must accept HTML, SVG or scripts (a code
	// host) and serve them from another origin.
	AllowActiveContent bool
	// AllowActive names the active types (sniff.IsActive) this policy
	// accepts while every other protection stays on: each listed type is
	// exempt from DefaultDenied and the active-content rule, and nothing
	// else is; DefaultDeniedExtensions still apply. It is the narrow
	// escape hatch AllowActiveContent is not: contrib/svg sets it to SVG,
	// which it sanitises, without letting HTML, XHTML or any other +xml
	// type through.
	AllowActive []string
	// Detector labels formats the built-in detection does not know
	// (DICOM, STL, camera RAW, Parquet), so Allowed can name them instead
	// of allowing application/octet-stream. It is consulted only when
	// sniff.Detect answers octet-stream, so it can never relabel a known
	// type, and its answer goes through Allowed, Denied and the
	// active-content rule like any other; an empty or malformed answer
	// stays octet-stream (sniff.DetectWith). Receive, Confirm (presigned
	// and Uppy uploads) and the chunked handler use it. nil means the
	// built-in detection only.
	Detector sniff.Detector
	// Key chooses storage keys. nil means RandomKey("").
	Key KeyFunc
	// KeyContext chooses storage keys like Key, with the context of the
	// upload (the request's, for the HTTP handlers), so a key can follow
	// the tenant a middleware put on it without a Resolver building a
	// Policy per request. Set Key or KeyContext, not both: Validate
	// refuses a policy with both, because which one wins would be a
	// guess. nil uses Key.
	KeyContext KeyContextFunc
	// Visibility of stored files; empty means the disk's default.
	// Receive and the Uppy handler apply it. Presign applies it only with
	// SignVisibility; otherwise a ticket signs a visibility only when its
	// TicketRequest asks for one (see AllowPublic), so a policy shared
	// with Receive never changes what existing tickets do.
	Visibility filesystem.Visibility
	// SignVisibility makes Presign sign Visibility into every ticket, so
	// the server, not the client's JSON, decides that an avatar or a
	// public gallery upload is stored public. Off, the default, keeps the
	// historical rule above.
	//
	// Invariants: it needs a Visibility (Validate refuses it without
	// one, since signing "the disk's default" signs nothing); a public
	// Visibility still needs AllowPublic (Validate refuses it otherwise),
	// because the ticket holder creates the object; and a TicketRequest
	// asking for another visibility than the policy's is refused with
	// CodePermission, never silently overridden, so a client learns its
	// request was not honoured.
	SignVisibility bool
	// AllowPublic lets a ticket request (TicketRequest.Visibility) ask for
	// filesystem.VisibilityPublic. Off, the default, such a request is
	// refused with CodePermission before anything is signed.
	//
	// Why it is a separate switch: whoever holds a ticket creates the
	// object, so a ticket that may be public lets any client publish a
	// file under the application's domain. Only the server decides that,
	// per policy (avatars yes, invoices no); the client can only ask.
	// Asking for private, or for nothing, needs no allow. The disk must
	// pin the visibility in the upload (filesystem.PresignOptions.
	// Visibility); one that cannot refuses with CodeUnsupported.
	AllowPublic bool
	// Headers, when set, chooses the HTTP headers stored with each upload
	// (Cache-Control, Content-Disposition, Content-Encoding), which the
	// backend's URLs and fshttp then send with it. StaticHeaders and
	// Attachment cover the common cases. It sees the file as it will be
	// stored (after Process). The disk must store headers
	// (Capabilities.Headers), or every upload fails with CodeUnsupported
	// rather than losing them. Receive and the Uppy handler store them;
	// Presign pins them in the signed PUT where the disk can (see
	// filesystem.PresignOptions.Headers) and refuses with CodeUnsupported
	// where it cannot, so the backend itself refuses an upload without
	// them.
	Headers HeadersFunc
	// PostForm makes Presign issue a presigned POST form (Ticket.Fields)
	// instead of a signed PUT: the backend itself then refuses an upload
	// whose size is not exactly the declared one, whose Content-Type is not
	// the declared one, or whose key differs, rather than Confirm finding
	// out afterwards. The disk needs Capabilities.PresignPost (S3, GCS);
	// without it Presign fails with CodeUnsupported. Confirm is still
	// required: the bytes are the client's.
	PostForm bool
	// Collision says what happens when the key already holds an object:
	// overwrite (the default), fail with CodeExists, or rename to a free
	// "name (1).ext". Fail and rename write with IfNotExists, so the disk
	// needs Capabilities.ConditionalWrite (CodeUnsupported otherwise).
	// Receive and the Uppy handler apply it; Presign refuses anything but
	// overwrite with CodeUnsupported, because a presigned PUT replaces
	// whatever is there.
	Collision Collision
	// Extensions, when set, lists the file-name extensions accepted
	// (".pdf", ".jpg"), compared without case. A name without an
	// extension passes; the type check still applies to it.
	Extensions []string
	// MatchExtension refuses a file whose name's extension conventionally
	// means another type than the one detected from its bytes
	// (sniff.TypeByExtension), such as a PNG named "invoice.pdf". Names
	// whose extension the table does not know pass.
	MatchExtension bool
	// Process, when set, transforms every accepted file before it is
	// stored: strip a photo's EXIF and GPS by re-encoding it
	// (imaging.Sanitize), convert a format, compress. The type it reports
	// is checked against the policy again, and MaxSize applies to what it
	// returns. Receive runs it, so it covers form uploads, imports and tus;
	// presigned and Uppy uploads go straight to the bucket and cannot be
	// processed (Confirm only checks them).
	Process Processor
	// Progress, when set, is told how many bytes of a file Receive has
	// streamed to the disk: about every ProgressStep bytes and once more,
	// with the total, when the file is fully read. It covers the form
	// handler, FromURL and tus, which all receive through Receive. It runs
	// in the request's goroutine and must return quickly (push to a
	// channel, update a counter).
	Progress func(ctx context.Context, info Info, received int64)
	// Settle, when set, runs after Inspect on every stored upload and may
	// move it, returning the Result for where it now is; ContentAddressed
	// uses it to give files content-derived keys. An error deletes the
	// upload and is returned. Receive, Confirm and the Uppy handler run it.
	Settle Settler
	// Checksums asks for more digests of every stored file beside its
	// SHA-256 (MD5 for an existing checksum column, CRC32C, BLAKE3
	// for content addressing), reported in Result.Checksums by
	// filesystem.Checksum.Name. Receive (and so the form handler, FromURL,
	// tus and the chunked handler) has Disk.Write compute them while it
	// streams; Confirm (presigned and Uppy uploads, multipart included)
	// reads the stored object once to compute them, because those bytes
	// never passed through the server. A Want, which a per-request
	// Resolver can set from what the client declared, is verified on both
	// paths: Receive fails with CodeInvalidArgument before anything is
	// committed, Confirm deletes the object and refuses it with
	// CodeRejected. They describe the bytes as stored, after Process.
	// Validate checks the list (filesystem.ValidateChecksums). nil, the
	// default, computes nothing more and leaves Result.Checksums nil.
	Checksums []filesystem.Checksum
	// Inspect, when set, examines every accepted file once it is stored
	// and before the upload reports success: a virus scan (package scan),
	// a full image decode, a moderation call. An error deletes the object
	// and is returned as the upload's error, so a file that fails
	// inspection is never reported as stored. Receive and Confirm run it,
	// which covers every upload path in this module.
	Inspect Inspector
}

Policy describes what an upload endpoint accepts and where it stores it.

func (Policy) Accepts

func (p Policy) Accepts(contentType string) error

Accepts reports whether the policy accepts contentType, and if not, why.

func (Policy) AcceptsName

func (p Policy) AcceptsName(name, contentType string) error

AcceptsName applies the name rules (DefaultDeniedExtensions, Extensions, MatchExtension) to a file name and the content type detected or declared for it; CodeRejected when they refuse it.

func (Policy) Validate

func (p Policy) Validate() error

Validate checks the policy itself: a MaxSize, at least one allowed type, a known visibility. Every upload path calls it, so a misconfigured endpoint fails with CodeInvalidArgument rather than accepting everything; integrations call it at construction to fail earlier.

type PresignHandlerOptions

type PresignHandlerOptions struct {
	// Resolve chooses the policy for each ticket request from the one
	// given, before the body is read; its error refuses the request. See
	// Resolver. nil uses the given policy.
	Resolve Resolver
	// Idempotency, when set, makes a retried ticket request (same
	// Idempotency-Key) return the first ticket instead of a new key.
	Idempotency *Idempotency
	// Errors renders a failed ticket request; see fshttp.ErrorWriter.
	// nil is JSONErrors.
	Errors fshttp.ErrorWriter
	// Respond renders the Ticket of a successful request; see
	// SuccessWriter. nil is JSONSuccess, today's body unchanged.
	Respond SuccessWriter
}

PresignHandlerOptions configure PresignHandlerWith.

type Processor

type Processor func(ctx context.Context, in Info, body io.Reader) (io.Reader, Info, error)

Processor transforms an upload's bytes before they are stored; see Policy.Process. It returns the new body and what it now is (ContentType and Filename); an error refuses the upload.

type Recorded

type Recorded struct {
	// Fingerprint identifies the request: its method, path and content
	// (for a multipart form, each part's name, file name and bytes, so a
	// retry with a new boundary still matches).
	Fingerprint string
	// Body is the response body.
	Body []byte
	// ContentType is the first response's Content-Type, so a replay
	// answers in the same media type a custom SuccessWriter chose
	// (application/vnd.api+json, a protobuf, text/html). Empty, as in a
	// response recorded before this field existed or one that set none,
	// replays as application/json, the default writer's type. A store
	// that persists Recorded must keep it beside Body.
	ContentType string
}

Recorded is a response kept for replay, with the fingerprint of the request that produced it.

type Remote

type Remote struct {
	Body io.ReadCloser
	// Filename comes from Content-Disposition, else the URL's last path
	// segment, else "download", and is made safe with SafeFilename.
	Filename string
	// Size is the announced Content-Length, or -1.
	Size int64
	// URL is the URL finally fetched, after redirects. Relative links in
	// the body (an HTML page's og:image) resolve against it, not against
	// the URL first asked for.
	URL string
}

Remote is a fetched response, ready to stream. The caller closes Body.

func Fetch

func Fetch(ctx context.Context, rawURL string, o FetchOptions) (*Remote, error)

Fetch GETs rawURL with the SSRF guard and returns the response to stream. Errors: CodeInvalidArgument for a refused URL, address, redirect or 4xx answer other than those below; CodeNotFound for 404 and 410; CodeTooLarge for an announced size over MaxSize; CodeUnavailable for network failures, timeouts, 429 and 5xx answers.

type Resolver

type Resolver func(r *http.Request, base Policy) (Policy, error)

Resolver chooses the policy for one request from the handler's base policy, before any byte of the request body is read: limits that follow the user's plan or role (a 5 MB free tier, a 100 MB paid one), a key directory per user, a stricter type list per collection. It is also the hook for refusing early: authentication, a quota that is already used up, or a rate limit (return CodeUnavailable with WithRetryAfter and the client gets 503 and Retry-After). The error is answered with its code's status and fshttp.ClientMessage: its own words reach the client only for the codes about the client's input (invalid_argument, rejected, …), the status text otherwise. The returned policy is validated like any other; one that fails is the server's misconfiguration, not the client's mistake, so it is answered 500 with the status text only.

type Response

type Response struct {
	Files  []Result      `json:"files"`
	Failed []FileFailure `json:"failed,omitempty"`
}

Response is the JSON body Handler answers with on success. Failed is only ever set with HandlerOptions.Partial.

type Result

type Result struct {
	// Key is where the file is stored on the disk.
	Key string `json:"key"`
	// Filename is the client's file name, made safe, for display.
	Filename string `json:"filename"`
	// ContentType is the type detected from the content.
	ContentType string `json:"content_type"`
	// Size is the stored size in bytes.
	Size int64 `json:"size"`
	// SHA256 is the hex digest of the stored bytes, for deduplication and
	// integrity checks.
	SHA256 string `json:"sha256"`
	// Path is the client's relative path made safe (SafePath), for folder
	// uploads: "photos/2024/a.jpg"; the bare file name when the client sent
	// no directories. Display only, unless FolderKey builds keys from it.
	Path string `json:"path,omitempty"`
	// Checksums holds the digests Policy.Checksums asked for, by
	// filesystem.Checksum.Name, in lowercase hex; nil when the policy
	// asked for none, so a response without them is unchanged.
	Checksums map[string]string `json:"checksums,omitempty"`
}

Result describes a stored upload.

func Confirm

func Confirm(ctx context.Context, disk *filesystem.Disk, key, filename string, p Policy) (Result, error)

Confirm checks a directly uploaded object against p using the stored bytes: its size, its type detected from its first bytes, and, where the backend stores one, that the stored Content-Type names that same type (the client must send the Content-Type the Ticket's Headers carry). An object that fails is deleted and the error returned (CodeTooLarge or CodeRejected), so nothing the policy refuses stays on the disk. filename is the name the client reported, for the Result only.

func ConfirmExpected

func ConfirmExpected(ctx context.Context, disk *filesystem.Disk, key, filename string, p Policy, want Expected) (Result, error)

ConfirmExpected is Confirm that also holds the stored object to what the client declared when it asked for the ticket: a stored size other than want.Size, or a SHA-256 other than want.SHA256 (compared without case), deletes the object and refuses it with CodeRejected. A malformed want.SHA256 is CodeInvalidArgument and nothing is read or deleted.

func FromBase64

func FromBase64(ctx context.Context, disk *filesystem.Disk, encoded, filename string, p Policy) (Result, error)

FromBase64 stores a file sent as standard base64 text (RFC 4648 with padding; line breaks are ignored), as JSON APIs and mobile clients often send small files. It decodes while streaming and is Receive in every other respect: the type comes from the decoded bytes, the policy's size cap counts decoded bytes, nothing refused is stored. Text that is not valid base64 is CodeInvalidArgument. filename is for display only.

func FromDataURI

func FromDataURI(ctx context.Context, disk *filesystem.Disk, uri, filename string, p Policy) (Result, error)

FromDataURI stores a file sent as a base64 data URI, "data:image/png;base64,iVBOR…", as a canvas export or a pasted image arrives. The media type the URI declares is ignored: the type is detected from the decoded bytes, as for every upload. Only base64 data URIs are accepted; anything else is CodeInvalidArgument. See FromBase64.

func FromURL

func FromURL(ctx context.Context, disk *filesystem.Disk, rawURL string, p Policy, o FetchOptions) (Result, error)

FromURL fetches rawURL (see Fetch) and stores it on disk under policy p (see Receive): the type is detected from the bytes, never from the server's Content-Type, and the size is capped as the body streams. A Remote's announced size over p.MaxSize is refused before any byte is read.

func Receive

func Receive(ctx context.Context, disk *filesystem.Disk, body io.Reader, filename string, p Policy) (Result, error)

Receive stores one file from body on disk under policy p: it detects the type from the first bytes, refuses it with CodeRejected when the policy does not accept it, chooses the key with the policy's KeyFunc, and streams the rest with the policy's size cap (CodeTooLarge, nothing stored). filename is the client's name and is used for display only.

type Session

type Session struct {
	// ID identifies the session (SessionIDPrefix and random base32); it
	// is half of the token and is not a secret on its own.
	ID string
	// SecretHash is the hex SHA-256 of the token's secret half: never the
	// secret, which is shown once, by Issue.
	SecretHash string
	// Key, when set, is the one key uploads through the session are
	// stored at (each upload replaces the last, under the policy's
	// Collision).
	Key string
	// Prefix, when set, is the directory uploads are stored under: the
	// policy's key chosen inside it (RandomKey(Prefix) when the policy
	// has no Key).
	Prefix string
	// MaxSize, when positive, is the per-file size cap, at most the
	// policy's; 0 keeps the policy's.
	MaxSize int64
	// Allowed, when set, narrows the policy's types: each pattern lies
	// within the policy's Allowed. nil keeps the policy's.
	Allowed []string
	// CreatedAt and ExpiresAt bound the session; it is refused from
	// ExpiresAt on.
	CreatedAt time.Time
	ExpiresAt time.Time
}

Session is an upload session as a SessionStore keeps it. Exactly one of Key and Prefix is set.

func (Session) Expired

func (s Session) Expired(now time.Time) bool

Expired reports whether s has expired at now.

type SessionRequest

type SessionRequest struct {
	// Key or Prefix (exactly one) is where uploads may go; it is cleaned
	// and traversal-checked like every key.
	Key    string
	Prefix string
	// MaxSize, when positive, caps each file below the policy's MaxSize
	// (and not below its MinSize).
	MaxSize int64
	// Allowed, when set, narrows the policy's types; each pattern must lie
	// within the policy's Allowed ("image/png" within "image/*").
	Allowed []string
	// TTL is how long the session lasts: required, at most
	// MaxSessionLifetime.
	TTL time.Duration
}

SessionRequest is what the application asks of a new session.

type SessionStore

type SessionStore interface {
	// SaveSession records a new session; an ID already present is
	// CodeExists.
	SaveSession(ctx context.Context, s Session) error
	// LoadSession returns the session with id, expired or not;
	// CodeNotFound when there is none.
	LoadSession(ctx context.Context, id string) (Session, error)
	// DeleteSession removes the session with id (revocation); an unknown
	// id is not an error.
	DeleteSession(ctx context.Context, id string) error
	// SweepSessions deletes every session expired at now and returns how
	// many it deleted.
	SweepSessions(ctx context.Context, now time.Time) (int, error)
}

SessionStore keeps upload sessions. It is the seam for sharing them across instances; MemorySessions is the in-process one and records/uploadsessions the database one. Implementations store and return sessions as given; the Sessions type does every check.

type Sessions

type Sessions struct {
	// contains filtered or unexported fields
}

Sessions issues and redeems upload sessions over a SessionStore. Create it with NewSessions; it is safe for concurrent use.

func NewSessions

func NewSessions(store SessionStore, opts SessionsOptions) (*Sessions, error)

NewSessions returns Sessions over store. A nil store is CodeInvalidArgument.

func (*Sessions) Issue

func (m *Sessions) Issue(ctx context.Context, base Policy, req SessionRequest) (IssuedSession, error)

Issue records a new session for uploads under base, the policy of the endpoint the token will be redeemed at, and returns it with its token. The request may only narrow base: a MaxSize above base's (or below its MinSize), a type outside base's Allowed, no or both of Key and Prefix, an unclean key or a TTL outside (0, MaxSessionLifetime] is CodeInvalidArgument, and an invalid base fails as Validate does.

func (*Sessions) Redeem

func (m *Sessions) Redeem(ctx context.Context, token string, base Policy) (Policy, Session, error)

Redeem checks token and returns base narrowed to its session (size, types, key or prefix) with the session. Any unusable token is the same CodePermission error. The session's types are re-checked against base as it is now: a type base no longer allows is dropped, and a session left with none is refused (CodeRejected), so changing the endpoint's policy narrows sessions already issued. Receive, Confirm and every handler then enforce the returned policy on the bytes.

func (*Sessions) Resolver

func (m *Sessions) Resolver() Resolver

Resolver returns an upload Resolver that redeems the token in each request's HeaderUploadSession, for the Resolve option of Handler, Chunked, Uppy, PresignHandlerWith and LimitsHandler: a request without a usable session is refused before its body is read (403), and one with a session runs under the session's narrowed policy. Every request of a chunked or multipart upload carries the token, so a session that expires mid-upload stops it at the next request.

func (*Sessions) Revoke

func (m *Sessions) Revoke(ctx context.Context, id string) error

Revoke deletes the session with id, so its token is refused from now on. An unknown id is not an error.

func (*Sessions) Sweep

func (m *Sessions) Sweep(ctx context.Context) (int, error)

Sweep deletes the sessions expired by the Sessions clock and returns how many. Schedule it; the library starts no background work. An expired session is refused whether or not it was swept.

type SessionsOptions

type SessionsOptions struct {
	// Now is the clock every expiry is judged by; nil means time.Now.
	Now func() time.Time
}

SessionsOptions configure NewSessions.

type Settler

type Settler func(ctx context.Context, disk *filesystem.Disk, r Result) (Result, error)

Settler finishes a stored upload, possibly moving it; see Policy.Settle.

type SuccessBody

type SuccessBody interface {
	// contains filtered or unexported methods
}

SuccessBody is a body this package's handlers answer a successful request with. The set is closed, so a SuccessWriter can switch on it exhaustively:

  • Response, from Handler (201, or 200 when a Partial request stored some files; a Partial request that stored none answers its Response with the first failure's status, because the failures are a result, not an error) and from Chunked when the last chunk completes a file (201);
  • ChunkProgress, from Chunked for every other chunk (200);
  • Ticket, from PresignHandler (200);
  • Limits, from LimitsHandler (200).

The Uppy and tus protocol bodies are not in the set: their clients parse them as the protocols define, so they cannot be reshaped.

type SuccessWriter

type SuccessWriter func(w http.ResponseWriter, r *http.Request, status int, body SuccessBody)

SuccessWriter renders a successful response for a handler of this package: the opt-in seam for an API whose responses follow one envelope ({data: ...}, JSON:API, camelCase keys), as fshttp.ErrorWriter is for errors. status is the one the default answers with; body is one of the SuccessBody types. It is called before anything is written, and it owns the status line, the Content-Type and the body.

Invariant: a nil SuccessWriter in an options struct means JSONSuccess, whose output is byte for byte what every handler answered before the seam existed (Uppy's, the upload JSON contract and clients built on it keep working). With HandlerOptions.Idempotency a replayed response returns the recorded body under the Content-Type the first response set (Recorded.ContentType), so an envelope in another media type replays as itself; only the status becomes 200.

type Ticket

type Ticket struct {
	Key       string              `json:"key"`
	Method    string              `json:"method"`
	URL       string              `json:"url"`
	Headers   map[string][]string `json:"headers,omitempty"`
	Fields    map[string]string   `json:"fields,omitempty"`
	ExpiresAt time.Time           `json:"expires_at"`
}

Ticket authorises one direct upload: the client sends the file with Method to URL with Headers, then asks the server to Confirm Key. A form ticket (Policy.PostForm) has Method POST and Fields instead of Headers: the client posts a multipart form with every field, then the file in the field named "file", last.

func Presign

func Presign(ctx context.Context, disk *filesystem.Disk, req TicketRequest, p Policy, expires time.Duration) (Ticket, error)

Presign issues a Ticket for uploading straight to the disk's backend, so large files never pass through the application. The declared type and size are checked against p, but they are the client's word: Confirm must run after the upload, because only then can the server see the bytes. The signed URL pins the declared content type, so the backend refuses a different Content-Type header. The policy's Headers are pinned the same way: the Ticket's Headers list them, the client must send them, and the backend stores them. A disk that cannot pin them in a presigned PUT (see filesystem.PresignOptions.Headers) refuses with CodeUnsupported rather than issue a ticket that would lose them. A request asking for a visibility (TicketRequest.Visibility) has it signed the same way; public needs Policy.AllowPublic (CodePermission otherwise).

type TicketRequest

type TicketRequest struct {
	Filename    string `json:"filename"`
	ContentType string `json:"content_type"`
	Size        int64  `json:"size"`
	// SHA256 is the hex SHA-256 the client computed, optional. Presign
	// checks its form only; keep it with the ticket and pass it to
	// ConfirmExpected, which refuses an object whose bytes do not match.
	SHA256 string `json:"sha256,omitempty"`
	// Visibility, optional, asks for the uploaded object to be stored
	// public or private; the ticket then signs it, so the client cannot
	// change it at upload time. Public needs Policy.AllowPublic. Empty
	// signs nothing and the object gets the backend's default, as before
	// this field existed.
	Visibility filesystem.Visibility `json:"visibility,omitempty"`
}

TicketRequest is what a client sends to ask for a direct upload.

type UppyOptions

type UppyOptions struct {
	// Expires is the lifetime of each signed part URL. 0 means 15 minutes.
	Expires time.Duration
	// Authorize is called with every request that continues an upload
	// (sign, list, complete, abort) and the key it names. The upload ID and
	// key come from the client, so without this check one user could
	// complete or abort another's upload. nil allows everything, which is
	// only safe when the endpoints are not reachable by untrusted users.
	Authorize func(r *http.Request, key string) error
	// Resolve chooses the policy for each request (create, and complete,
	// whose Confirm checks the assembled file) from the one given to Uppy;
	// its error refuses the request. See Resolver. It must return the
	// same limits for the requests of one upload, or a file accepted at
	// create can be refused at complete. nil uses the given policy.
	Resolve Resolver
	// OnStored runs once a completed upload has passed Confirm, with the
	// file as it is now stored, typically to create its database row (a
	// records.Store.Create). If it fails, the file is deleted and the
	// complete request fails with its error, so a stored file never
	// lacks the row the application expected. nil does nothing more.
	OnStored func(ctx context.Context, r *http.Request, file Result) error
	// Errors renders every failed request; see fshttp.ErrorWriter. nil
	// is JSONErrors.
	Errors fshttp.ErrorWriter
}

UppyOptions configure Uppy.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL