Documentation
¶
Overview ¶
Package filebody answers the one question every surface that serves file bytes has had to ask since uploads became staged: where are this file's bytes RIGHT NOW?
Before staged uploads (internal/staging) the answer was always "on the storage driver", so every read surface — the app's download/preview, share links, the folder-share browse page, thumbnails, WebDAV, the archive and AI endpoints — called drv.Read/drv.Stat directly and that was correct. A staged upload breaks that assumption on purpose: the node is created and listed the moment the client commits, while a background op is still streaming the bytes from filex's staging area to the driver. During that window the driver does not have the object (or, on an overwrite, still has the PREVIOUS one), and a surface that asks the driver answers 404 — or worse, serves the old file as if it were the new one.
So there is exactly one helper, here, and every read surface goes through it. This is deliberate and it is the standing project rule: a fix that lands on one surface and not the others turns one product into several. If a surface cannot use it, that is a finding to report, not a special case to write.
What it decides ¶
Where the bytes are. nodes.transfer_state == "staged" or "failed" → the staging area (a failed transfer keeps its staging, and those bytes are the only copy); anything else → the driver. The node is consulted FIRST, before the driver, because an overwrite leaves a perfectly readable stale object behind and "the driver answered" is not evidence that it answered with the right file.
What Stat says while staged. The committed values: the size the client committed (the staging manifest is the authority, and it is the number already published on the node) and an ETag computed from the staged parts — never the driver's absence, and never the pre-overwrite ETag, which would let a cached client keep the old file forever.
What a FAILED transfer means. Nothing, to a reader. The staging directory is kept on failure precisely so the transfer can be retried without re-sending a byte, and the node stays "staged" — so reads keep coming out of staging exactly as they did while the transfer was running.
What "staged, but staging is gone" means. A clean, loud failure (ErrStagingGone), never a body. This state is reachable: the sweeper removes a `failed` session once it has been idle for a full TTL, and the node it belongs to keeps saying "staged" forever. There is deliberately NO fallback to the driver in that case — on an overwrite the driver holds the previous version of the same path at plausibly the same size, and serving that as the new file is a silent wrong answer, which is worse than an error a human can read. A node whose bytes DID land is settled to "stored" by the storage sync and the staged-upload boot pass, on complete evidence (model.TransferLanded) — never guessed at here.
Whether a locally prepared copy should answer instead of the backend. internal/filecache keeps big files from slow storages on local disk; this is where it is consulted, for the same reason as everything above: one door, so every read surface gets it without per-surface code. The accelerating half is invisible — Open/ReadRange simply come off the local file when there is one. The visible half (the "preparing… %" answer) is Prepare, which a surface must ask for, because only a surface knows whether it can express "not yet" to its client.
Index ¶
- Variables
- type Prep
- type Resolver
- type Source
- func (s *Source) CanRange() bool
- func (s *Source) Open(ctx context.Context) (io.ReadCloser, error)
- func (s *Source) Prepare(ctx context.Context, stat storage.Object) *Prep
- func (s *Source) ReadRange(ctx context.Context, off, length int64) (io.ReadCloser, error)
- func (s *Source) Size() int64
- func (s *Source) Stat(ctx context.Context) (storage.Object, error)
- func (s *Source) Status(ctx context.Context, stat storage.Object) *Prep
Constants ¶
This section is empty.
Variables ¶
var ErrStagingGone = errors.New("filebody: file is still transferring and its staged copy is gone")
ErrStagingGone is returned when a node says its bytes are staged but the staging area cannot produce them. Callers must surface it as an error with a message — never as a 404 (the file exists) and never as a partial body.
Functions ¶
This section is empty.
Types ¶
type Prep ¶
type Prep struct {
// Ready means the local copy exists; just serve, it will come off disk.
Ready bool
// Percent is fetch progress, 0..99, meaningful only while !Ready.
Percent int
// Size is the file's length in bytes, so a caller can say how big the
// thing being prepared is.
Size int64
}
Prep is what the cache is doing for one file. A nil *Prep means "serve this the way you always did".
type Resolver ¶
type Resolver struct {
// contains filtered or unexported fields
}
Resolver resolves byte sources. A nil *Resolver is usable and always answers "the driver" — that is what keeps every construction site (tests, embedders, list-only environments) working without wiring staging in.
func New ¶
New returns a Resolver over the node catalogue and the staging area. Either may be nil; the result then degrades to driver-only reads.
func (*Resolver) Resolve ¶
func (r *Resolver) Resolve(ctx context.Context, drv storage.Driver, storageID int64, rel string, node *model.Node) (*Source, error)
Resolve reports where the bytes of `rel` on storage `storageID` live.
`node` is optional: pass it when the caller already holds the row (share links, thumbnails, the queue workers) to save a lookup, or nil to have the catalogue consulted by path. A path that is not in the catalogue resolves to the driver, which is what keeps freshly-written, not-yet-synced files readable.
The returned error is ErrStagingGone (wrapped, with the node and path in the message) when the node claims to be staged and the staging area cannot back that claim. Resolve fails BEFORE any header or byte is written, which is the whole reason the manifest is read here and not lazily at Open time.
func (*Resolver) WithCache ¶
WithCache attaches the local file cache and returns the same Resolver, so wiring reads as one expression. A nil cache (or one with no directory) is fine and leaves every read going to the driver, which is what makes this package usable from tests and embedders that never configured a cache.
type Source ¶
type Source struct {
// Staged reports that the bytes are served out of filex's staging area
// because the transfer to the driver has not finished (or has failed and
// is waiting to be retried).
Staged bool
// UploadID is the staging session id, empty unless Staged.
UploadID string
// NodeID is the catalogue row the decision was taken from, 0 when the
// path is not in the catalogue (pre-sync files, driver-only reads).
NodeID int64
// contains filtered or unexported fields
}
Source is one file's resolved byte source. Obtain it from Resolve; the zero value is not usable.
func (*Source) CanRange ¶
CanRange reports whether ReadRange will work. Staging always can — the assembled staging reader is a seekable io.ReadSeekCloser across part boundaries — so a file being transferred is seekable in a video player and resumable in a downloader exactly like a stored one. So can a prepared local copy, which is how a slow backend that cannot range gains seeking.
func (*Source) Open ¶
Open returns the whole body.
A driver read is wrapped so the bytes and the time they took land in internal/throughput. This is the single read funnel — every surface that streams a file (browser, WebDAV, share link, ShareX, the desktop app) goes through here — which is why the measurement belongs here and not in each of them.
⚠ Only driver-backed bodies are measured. The staging area and the prepared cache copy are local disk; folding their speed into the average would tell the meter that a NAS runs at NVMe speed, and the storage would never be classified slow again — which would switch the cache off for exactly the backends it exists for. That is why the cache-hit return above is NOT wrapped, while the fill that populates the cache (Prepare) IS: the fill is the one read that really does come off the driver.
func (*Source) Prepare ¶
Prepare decides whether this file should be served from a locally prepared copy and, if it should, makes sure one is being made.
nil means the file does not qualify (too small, storage not slow, bytes still in staging, caching off, or the cache has no room) — the caller serves exactly as it did before this existed. A *Prep with Ready=false is the answer the owner asked for: tell the user we are preparing, show a percentage, and start the download when it is ready.
`stat` is the live driver description the caller already holds; passing it avoids a second round trip to the backend, which on a slow backend is the exact cost this feature exists to remove.
⚠ A cache failure is never a download failure. Anything that goes wrong here answers nil and the request streams from the driver.
func (*Source) ReadRange ¶
ReadRange returns a window of the body, with the same contract as storage.RangeReader: off >= 0, length < 0 means "to the end", length == 0 yields an immediate EOF, an offset at or past EOF is EOF rather than an error, and a length past EOF is clamped.
func (*Source) Stat ¶
Stat describes the file. While staged it answers with the COMMITTED description — the size that was committed and an ETag derived from the staged parts — because the driver has nothing to describe yet, and because on an overwrite its answer would describe the previous version.
func (*Source) Status ¶
Status reports on a prepared copy WITHOUT starting one. It is what a progress poll asks: a poll that could start work would let anyone warm the cache for any file they can see, and would restart a fetch that had just failed, once per poll.
nil means "nothing is being prepared" — for a poller that is the signal to go ahead and ask for the file.
Source Files