Documentation
¶
Overview ¶
Package file implements a backend.Backend over a local directory tree. Keys map to files under a root; writes are atomic (temp file + rename) and durable (the temp file is fsynced, then every directory entry up to one already known on the disk), so a reader never observes a partially written object and a power cut never takes a name a write already reported as stored — the property the "manifest written last" part commit relies on. The deferred operations of backend.DeferredSyncer are the exception: their names are owed only once File.SyncPrefix returns.
Index ¶
- type File
- func (f *File) CompareAndSwap(ctx context.Context, key string, expected backend.Version, data []byte) (backend.Version, bool, error)
- func (f *File) CreateObject(_ context.Context, key string) (backend.ObjectWriter, error)
- func (f *File) CreateObjectDeferred(_ context.Context, key string) (backend.ObjectWriter, error)
- func (f *File) Delete(_ context.Context, key string) error
- func (f *File) DeleteDeferred(_ context.Context, key string) error
- func (f *File) FreeInodes(context.Context) (int64, error)
- func (f *File) FreeSpace(context.Context) (int64, error)
- func (*File) IsEphemeral() bool
- func (*File) IsNodeLocal() bool
- func (f *File) List(_ context.Context, prefix string) ([]string, error)
- func (f *File) PutIfAbsent(_ context.Context, key string, data []byte) (written bool, rerr error)
- func (f *File) Read(_ context.Context, key string) ([]byte, error)
- func (f *File) ReadAt(_ context.Context, key string, off, n int64) ([]byte, error)
- func (f *File) ReadVersioned(ctx context.Context, key string) ([]byte, backend.Version, error)
- func (f *File) Size(_ context.Context, key string) (int64, error)
- func (*File) StreamsWrites() bool
- func (f *File) SyncPrefix(_ context.Context, prefix string) error
- func (f *File) Write(_ context.Context, key string, data []byte) error
- func (f *File) WriteDeferred(_ context.Context, key string, data []byte) error
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type File ¶
type File struct {
// contains filtered or unexported fields
}
File is a directory-backed backend.Backend. Keys are slash-delimited and map to paths under root. Safe for concurrent use (the filesystem serializes renames; reads and writes touch distinct temp files).
Every path operation goes through a vfs.FS opened for that operation alone — an os.Root in production. That buys two things over joining strings onto a root path. Containment stops being a check this package has to get right — a symlink inside the tree pointing out of it is refused, not merely a "..". And on Windows the file handles it opens carry FILE_SHARE_DELETE, without which a concurrent reader blocks the rename that publishes an object: plain os.Open there asks for FILE_SHARE_READ|FILE_SHARE_WRITE only, and MoveFileEx over a destination someone holds open fails with ERROR_ACCESS_DENIED. Rename-over-an-open-file is a POSIX guarantee the write path depends on; os.Root is what extends it to Windows.
The handle is per-operation rather than held on the File because os.OpenRoot opens its own directory handle through the same syscall.Open that omits FILE_SHARE_DELETE (os/root_windows.go). A root kept for the backend's lifetime would therefore pin its data directory against removal on Windows, and backend.Backend has no Close with which to release it. The cost is one extra open and close per operation.
func (*File) CompareAndSwap ¶ added in v0.40.0
func (f *File) CompareAndSwap( ctx context.Context, key string, expected backend.Version, data []byte, ) (backend.Version, bool, error)
CompareAndSwap stores data under key if the file's current contents still hash to expected, writing through the same atomic temp+fsync+rename as File.Write. Implements backend.Backend.
func (*File) CreateObject ¶ added in v0.37.0
CreateObject builds key's object incrementally in the temp file that File.Write uses for its whole-object write, renaming it over the final path on commit. Nothing is visible under key until then, so the atomicity contract is the same one; the difference is only that the bytes reach the filesystem as they are produced rather than all at once. Implements backend.ObjectCreator.
func (*File) CreateObjectDeferred ¶ added in v0.47.0
CreateObjectDeferred is File.CreateObject whose commit leaves the name for File.SyncPrefix. Implements backend.DeferredSyncer.
func (*File) Delete ¶
Delete removes key, or returns an backend.ErrNotExist-wrapping error if absent.
func (*File) DeleteDeferred ¶ added in v0.47.0
DeleteDeferred is File.Delete without the directory syncs: a power cut may bring the object back, as an orphan. Implements backend.DeferredSyncer.
func (*File) FreeInodes ¶ added in v0.40.0
FreeInodes reports how many more files can be created on the filesystem holding the root directory. A filesystem that allocates inodes dynamically (btrfs, and tmpfs on some kernels) reports a zero total; it has no ceiling to report, so that is backend.ErrSpaceUnknown rather than "none left" — the difference between an unbounded backend and a wedged one.
func (*File) FreeSpace ¶ added in v0.37.0
FreeSpace reports the bytes available on the filesystem holding the root directory. It takes the unprivileged figure, so the reserved root allowance is never counted as usable.
func (*File) IsEphemeral ¶
IsEphemeral reports false: data persists on disk.
func (*File) IsNodeLocal ¶ added in v0.39.0
IsNodeLocal reports true: a directory tree is private to its node unless the root happens to be a shared mount, which the backend cannot tell. See backend.NodeLocal for how to read that.
func (*File) List ¶
List returns, sorted ascending, every key with the given prefix.
The prefix bounds the work, not just the result: keys map to paths, so only the subtree under the prefix's directory component is traversed, and within it only the children whose name can still extend into the prefix's final (possibly partial) segment.
func (*File) PutIfAbsent ¶
PutIfAbsent stores data under key only if it does not already exist, returning whether the write happened. It writes a temp file then hard-links it to the final path: the link fails with EEXIST if the destination exists, giving an atomic, exclusive create (the conditional commit primitive). A reader never sees a partial object — the link publishes a fully written file.
func (*File) Read ¶
Read returns the value stored under key, or an backend.ErrNotExist-wrapping error.
func (*File) ReadAt ¶ added in v0.37.0
ReadAt returns the object's [off, off+n) range, clamped to its end, with one pread — the file backend never maps a part column into memory to hand back a slice of it. Implements backend.ReaderAt.
func (*File) ReadVersioned ¶ added in v0.40.0
ReadVersioned returns the value under key and the digest identifying it. Implements backend.Backend.
func (*File) Size ¶ added in v0.12.0
Size returns the byte size of the object stored under key (a stat, no read), or an backend.ErrNotExist-wrapping error if absent. It implements backend.Sizer.
func (*File) StreamsWrites ¶ added in v0.48.0
StreamsWrites reports true: the bytes reach the filesystem as they are produced. Implements backend.ObjectCreator.
func (*File) SyncPrefix ¶ added in v0.47.0
SyncPrefix makes every deferred write and delete under the directory prefix names durable: each directory of its subtree children first, then the entries of the ancestors not known durable, innermost first — so no entry reaches the disk before what it names. It keeps no record of what the deferred operations touched, so an aborted part leaves nothing behind to leak. A prefix with no directory syncs nothing. Implements backend.DeferredSyncer.
func (*File) Write ¶
Write stores data under key atomically and durably: it writes a temp file in the destination directory, fsyncs it, renames it over the final path, and fsyncs the directories the new name hangs from that are not yet known durable.
func (*File) WriteDeferred ¶ added in v0.47.0
WriteDeferred is File.Write without the directory syncs: the bytes are fsynced, the name waits for File.SyncPrefix. Implements backend.DeferredSyncer.
Source Files