Help us shape the future of Go! Take the 2026 Go Developer Survey

fsutil

package
v0.4.5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 16, 2026 License: MIT Imports: 14 Imported by: 0

Documentation

Overview

Atomic file write via a sibling temp file plus rename, fsyncing the temp file and parent directory.

Package fsutil provides shared filesystem helpers: directory creation, atomic-write, and related primitives.

Index

Constants

View Source
const (
	MaxVCSEntries = 64
	MaxVCSBytes   = 256 * 1024
)

MaxVCSEntries and MaxVCSBytes bound the inspection of a project template folder. The entry bound counts every directory entry examined, not only the candidates: the loader skips directories, so a candidate-only count is no bound at all against a root that creates subdirectories ahead of its first template.

Variables

View Source
var ErrNoReplaceUnsupported = errors.New("fsutil: atomic no-overwrite rename is unsupported on this platform")

ErrNoReplaceUnsupported reports that the running platform has no atomic no-overwrite rename primitive, so MoveNoReplace cannot move safely. The helper fails closed with this rather than emulating no-overwrite as a preflight stat plus a plain rename, which would clobber a destination that materializes after the check. Consumers detect it with errors.Is.

View Source
var ErrParentMoved = errors.New("fsutil: a parent directory was renamed or replaced during the move")

ErrParentMoved reports that a source or destination parent directory was renamed or replaced while MoveNoReplace ran, so the helper cannot confirm the entry landed at the authorized path. Consumers detect it with errors.Is. The helper rolls the rename back (best effort) before returning this, so a move reported as failed did not silently relocate the entry.

View Source
var ErrPublishUnreconciled = errors.New("fsutil: the write was refused and a file it did not stage is retained under the name given in the wrapping error")

ErrPublishUnreconciled reports that a write neither published nor put the directory back as it found it: a file the call did not stage survives under the scratch or target name given in the wrapping error, because removing it or displacing it would have destroyed data this call does not own — the planned original, which could not be linked back under its own name, or a file another writer left under a name this call was about to clean up. Nothing is lost, but the directory needs reconciling by hand. Consumers detect it with errors.Is.

View Source
var ErrSnapshotNotRegular = errors.New("fsutil: snapshot target is not a regular file")

ErrSnapshotNotRegular reports that a snapshot target exists but is a directory, symlink, FIFO, socket, or device rather than a regular file. Consumers detect it with errors.Is.

View Source
var ErrSnapshotTooLarge = errors.New("fsutil: snapshot target exceeds the byte cap")

ErrSnapshotTooLarge reports that a snapshot target's bytes exceed the caller's cap. Consumers detect it with errors.Is.

View Source
var ErrTargetIdentityChanged = errors.New("fsutil: the publish target is not the file the write was planned from")

ErrTargetIdentityChanged reports that the entry occupying the publish target name was not the file the caller planned its bytes from, so the publish was refused and nothing at that name was overwritten. Consumers detect it with errors.Is.

Functions

func CollectVCSCandidates

func CollectVCSCandidates(dir string) (files []VCSCandidate, present bool, failure VCSFailure)

CollectVCSCandidates reads every immediate non-directory entry of dir under the same bounded, no-follow, stable-file rules ReadSettingsBytes applies per file, so a caller overlays bytes this call read rather than reopening the folder. present=false with no failure is the absent state.

func EnsureDir

func EnsureDir(path string) (created bool, cat fserrors.Category, err error)

EnsureDir creates path (and any missing parents) with mode 0755 if it does not exist. The returned created reports whether the target path was absent before the call. On failure it returns the error category. Intermediate directories are not fsynced.

func EnsureDirAt

func EnsureDirAt(parentDir *os.File, relPath string, hooks ...DirLifecycle) (created bool, createdDirs []string, cat fserrors.Category, err error)

EnsureDirAt creates relPath (and any missing parent segments) under the already-open, authorized directory parentDir, descending one path component at a time relative to a held directory descriptor. It is the fd-relative, post-authorization counterpart of EnsureDir: where EnsureDir re-resolves an absolute path through os.MkdirAll (and so can be redirected by a symlink or rename swapped in after the caller authorized the location), EnsureDirAt never re-resolves by path. Each segment is created with unix.Mkdirat and then re-opened with O_DIRECTORY|O_NOFOLLOW before the next segment is created in it, so a component that is (or is swapped to) a symlink is rejected with ELOOP and a non-directory component is rejected with ENOTDIR — the create can never escape the authorized directory tree. The caller owns parentDir and must close it after this call.

relPath must be a clean, relative, slash-separated path of one or more ordinary components; an empty, absolute, or ".", "..", empty-, or NUL-containing-segment path is rejected fail-closed with EINVAL before any filesystem change. Existing directories are idempotent success: created reports whether the final target segment was absent before the call (mirroring EnsureDir). createdDirs lists the slash-joined relative paths (under parentDir) of exactly the segments this call created — a segment found already present (EEXIST) is skipped, so a caller rolling the create back removes only directories it actually made and never a pre-existing one a concurrent process slipped in. It is meaningful only on a nil-error return; the order is shallowest first, so a rollback walks it in reverse to unlink deepest child first. Every newly created segment is made durable before the call descends into it: the parent directory that received the new entry is fsynced, mirroring WriteAtomicCreateAt's parent fsync, so a nested create never reports success with an earlier parent entry left unsynced. On parent-fsync failure, leave the newly created segment: unlinking by name could remove a concurrent replacement.

hooks is at most one optional DirLifecycle; passing none runs the plain descent. It is variadic so the callers that predate the lifecycle hooks keep compiling unchanged.

func ReadSettingsBytes

func ReadSettingsBytes(path string) (data []byte, found, ok bool)

ReadSettingsBytes reads the covered settings file under the bound and the descriptor checks the trust gate itself uses, so a post-gate consumer never reads bytes the gate would have refused to fingerprint. found=false with ok=true is the absent state; ok=false is an inspection failure, which no admitted snapshot matches.

func RenameNoReplaceAt

func RenameNoReplaceAt(oldDir *os.File, oldName string, newDir *os.File, newName string) error

func SetBeforeCleanupForTest

func SetBeforeCleanupForTest(fn func()) func()

SetBeforeCleanupForTest replaces the pre-cleanup hook with fn and returns a restorer the caller MUST run to put the no-op default back. Test-only; production must not call it. intentional: test-only seam (no production caller by design).

func SetBeforeMoveRenameForTest

func SetBeforeMoveRenameForTest(fn func()) func()

SetBeforeMoveRenameForTest replaces the pre-rename hook with fn and returns a restorer the caller MUST run to put the no-op default back. Test-only; production must not call it. intentional: test-only seam (no production caller by design).

func SetBeforePublishRenameForTest

func SetBeforePublishRenameForTest(fn func()) func()

SetBeforePublishRenameForTest replaces the pre-publication hook with fn and returns a restorer the caller MUST run to put the no-op default back. Test-only; production must not call it. intentional: test-only seam (no production caller by design).

func SetRecoveryNameForTest

func SetRecoveryNameForTest(fn func(basename string) (string, error)) func()

SetRecoveryNameForTest replaces the recovery-name generator with fn and returns a restorer the caller MUST run to put the production generator back. Test-only; production must not call it. intentional: test-only seam (no production caller by design).

func SetRenameNoReplaceAtForTest

func SetRenameNoReplaceAtForTest(fn func(oldDirFd int, oldName string, newDirFd int, newName string) error) func()

SetRenameNoReplaceAtForTest replaces the platform no-overwrite rename seam with fn and returns a restorer the caller MUST run to put the production seam back. Test-only; production must not call it. intentional: test-only seam (no production caller by design).

func SetSyncDirForTest

func SetSyncDirForTest(fn func(*os.File) error) func()

SetSyncDirForTest replaces the parent-dir fsync seam used by WriteAtomic with fn and returns a restorer the caller MUST run to put the production seam back. Test-only; production code must not call this. intentional: test-only seam (no production caller by design).

func SetSyncFileForTest

func SetSyncFileForTest(fn func(*os.File) error) func()

SetSyncFileForTest replaces the temp-file fsync seam used by WriteAtomic with fn and returns a restorer the caller MUST run to put the production seam back. Test-only; production code must not call this. intentional: test-only seam (no production caller by design).

Types

type AtomicWriteStep

type AtomicWriteStep string

AtomicWriteStep identifies which step of WriteAtomic failed so callers can map it to their own reason tokens without parsing error strings. The zero value means success.

const (
	WriteStep      AtomicWriteStep = "write"
	RenameStep     AtomicWriteStep = "rename"
	ParentSyncStep AtomicWriteStep = "parent_sync"
	// LinkStep marks a failure of WriteAtomicCreate's no-replace link publish — most importantly the
	// publish-time existing-target case (err wraps fs.ErrExist), where the target appeared after
	// resolution and create must not overwrite it.
	LinkStep AtomicWriteStep = "link"
	// LoadStep marks a pre-write failure surfaced by callers that compose load/parse/marshal steps in front of WriteAtomic; WriteAtomic itself never returns it. Defined here so the shared enum stays the single source of truth for step tokens.
	LoadStep AtomicWriteStep = "load"
)

func MoveNoReplace

func MoveNoReplace(src, dst string, observe ...MoveLifecycle) (step AtomicWriteStep, cat fserrors.Category, err error)

MoveNoReplace renames src to dst on the same filesystem and never overwrites an existing destination. It opens both parent directories without following a final symlink and performs the rename relative to those held descriptors using an atomic no-replace primitive, so the operation cannot be redirected by a symlink swap and a destination that materializes after a caller's preflight is never clobbered. It is same-filesystem only: a cross-device attempt fails with the kernel's EXDEV and the helper never falls back to copy+unlink. After the rename it revalidates that each held parent is still the directory reachable at its path (os.SameFile), rolling the rename back and failing closed if a parent moved mid-operation. The caller owns no descriptors; MoveNoReplace opens and closes everything it uses.

On failure it returns the step that failed and the error category. The underlying error is returned alongside so a consumer can branch without parsing strings: an existing destination wraps fs.ErrExist (RenameStep), a missing source wraps fs.ErrNotExist (RenameStep), a cross-device attempt wraps syscall.EXDEV (RenameStep), an unsupported platform wraps ErrNoReplaceUnsupported (RenameStep), and a parent that moved wraps ErrParentMoved (ParentSyncStep). The zero step and empty category mean success.

func WriteAtomic

func WriteAtomic(targetPath string, data []byte, mode os.FileMode) (step AtomicWriteStep, cat fserrors.Category, err error)

WriteAtomic writes data to targetPath via a sibling temp file plus rename, fsyncing the temp file and parent directory. The parent directory must already exist. On failure it returns the step that failed and the error category.

func WriteAtomicCreateAt

func WriteAtomicCreateAt(parentDir *os.File, targetName string, data []byte, mode os.FileMode, beforePublish ...PrePublish) (step AtomicWriteStep, cat fserrors.Category, published os.FileInfo, err error)

WriteAtomicCreateAt writes data to targetName only when nothing already exists there, using an already open parent directory. It never resolves the parent by path: temp creation, no-replace publish, cleanup, and parent sync all operate relative to parentDir. It is the create-only counterpart of WriteAtomic: where WriteAtomic publishes with os.Rename (which silently overwrites an existing target), this publishes with a hard link, which fails if targetName already exists — so a target that materialized between a caller's pre-authorization check and the publish is never clobbered. The published file carries exactly mode's permission bits regardless of the process umask. Every returned failure leaves no target and no temp residue. The caller owns parentDir and must close it after this call. targetName must be a single path component, not a relative path.

beforePublish is at most one optional pre-publish hook; passing none is the plain publish. It is variadic so the path-free create callers that predate the hook keep compiling unchanged.

published is the identity of the inode this call linked at targetName, and it is non-nil exactly when those bytes became visible under that name — including alongside a parent-sync failure, whose compensating removal happens after the entry was already reachable. A caller that has to reconcile or undo the create identifies it by that inode and never by the name, because another writer can replace the name between publication and the reconciliation.

func WriteAtomicReplaceAt

func WriteAtomicReplaceAt(parentDir *os.File, targetName string, data []byte, mode os.FileMode, planned os.FileInfo, beforePublish PrePublish) (step AtomicWriteStep, cat fserrors.Category, published os.FileInfo, err error)

WriteAtomicReplaceAt writes data to targetName inside the already open parent directory, replacing whatever is there, and never resolves the parent by path: temp creation, the rename publish, cleanup, and the parent sync all run relative to parentDir. It is the descriptor-relative counterpart of WriteAtomic — a parent renamed or replaced after the caller opened it cannot redirect the write to a directory swapped in at the old path, because the held descriptor pins the inode. The published file carries exactly mode's permission bits regardless of the process umask. beforePublish is optional and, when non-nil, runs after the staged entry is durable and before anything becomes visible; its error aborts the publish leaving the target unchanged and no temp residue. The caller owns parentDir and must close it after this call. targetName must be a single path component, not a relative path.

planned is the identity of the file the caller computed data from, and is what closes the gap the descriptor cannot: the held descriptor pins the directory, not the entry inside it, so a plain rename replaces whatever answers to targetName at the instant the kernel resolves it — after any check the caller could have made. When planned is non-nil the publish refuses, wrapping ErrTargetIdentityChanged and leaving both files untouched, unless the entry it displaces is that exact file. Pass nil only when the caller has no expectation about what occupies the name.

published is the identity of the inode now answering to targetName, and it is non-nil exactly when publication happened — including alongside an error, because the parent fsync runs after the bytes are already visible and its failure does not take them back. A caller that may have to undo its own publish keeps it and passes it back as planned for that undo: the file it published is the only thing that undo may overwrite, and by then anything can occupy the name.

type DirLifecycle

type DirLifecycle struct {
	Intent  func(relPath string) error
	Confirm func(seg DirSegment) error
}

DirLifecycle carries EnsureDirAt's optional per-segment hooks so a directory it creates stays identifiable after a crash. Intent runs before each Mkdirat; Confirm resolves that intent once the segment's outcome is known — with Created and an identity for a directory that now exists, or with Created false and a zero identity when the attempt made nothing (an existing segment, or a Mkdirat that failed outright). Either hook's error aborts the call the way any other failure does, except on a path that is already failing: there the filesystem error is what returns, so a journalling problem cannot mask why the descent stopped.

An intent that gets no Confirm — a crash, a fsync failure, or a failure between Mkdirat and the re-open — is unrecovered state, not a rollback candidate, and nothing here removes it.

type DirSegment

type DirSegment struct {
	Path    string
	Created bool
	Dev     uint64
	Ino     uint64
}

DirSegment reports what EnsureDirAt found or made at one attempted segment. Created is true only when this call's Mkdirat made the entry, and Dev/Ino identify the directory that was then re-opened O_NOFOLLOW. A record with Created true and a non-zero identity is the only rollback-eligible outcome: ownership is proved by that inode identity, never by Path, so a directory swapped in at the same name can never be mistaken for one this call made. Dev and Ino are zero when nothing exists to identify.

type FileSnapshot

type FileSnapshot struct {
	Exists bool
	Data   []byte
	Mode   os.FileMode
	Dev    uint64
	Ino    uint64
}

FileSnapshot is the state SnapshotFileAt observed through one descriptor. When Exists is false every other field is zero. Dev and Ino identify the inode the bytes and mode were read from, so a consumer restoring later can prove it is acting on that same file rather than on whatever now answers to the name.

func SnapshotFileAt

func SnapshotFileAt(parentDir *os.File, name string, maxBytes int64) (FileSnapshot, error)

SnapshotFileAt reads name under the already open directory parentDir and reports either that the entry is absent, or that it is a regular file together with its bytes, permission mode, and inode identity. Everything after the open — the type check, the size check, and the read — uses that one descriptor, so nothing is re-resolved by name between the check and the read and the bytes returned are the bytes of the inode that was checked.

name must be a single path component. A symlink is refused rather than followed (O_NOFOLLOW), and any non-regular entry is refused with ErrSnapshotNotRegular. The open is non-blocking so a FIFO or device entry is refused instead of hanging the caller. A file whose bytes exceed maxBytes is refused with ErrSnapshotTooLarge: the pre-read size check rejects an already oversized entry without reading it, and the read itself stops one byte past the cap so a file that grows between the two still fails rather than returning a snapshot larger than the caller allowed.

type MoveLifecycle

type MoveLifecycle struct {
	BeforeRename func(srcParent, dstParent *os.File) error
	AfterRename  func(srcParent, dstParent *os.File)
}

AfterRename runs once the rename and any rollback of it have finished, whether or not the move succeeded.

type PrePublish

type PrePublish func(targetName string, data []byte, mode os.FileMode) error

PrePublish observes a staged temporary entry that already holds the final bytes, the exact requested permission bits, and a completed file fsync — immediately before the rename or link makes anything visible. mode is the permission mode read back from the staged descriptor, not the mode the caller asked for, so a recorded mode is the one publication will make durable. Returning an error aborts the publish, removes the temporary entry, and surfaces as a WriteStep failure.

type VCSCandidate

type VCSCandidate struct {
	Name  string
	Bytes []byte
}

VCSCandidate is one collected project template candidate. Name is an immediate directory entry name, never a path.

type VCSFailure

type VCSFailure string
const (
	VCSFailureNone        VCSFailure = ""
	VCSFailureUnreadable  VCSFailure = "unreadable"
	VCSFailureUnsafeEntry VCSFailure = "unsafe_entry"
	VCSFailureEntryLimit  VCSFailure = "entry_limit"
	VCSFailureByteLimit   VCSFailure = "byte_limit"
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL