atomicfile

package module
v2.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 12, 2026 License: GPL-2.0, GPL-3.0 Imports: 12 Imported by: 5

README

atomicfile

CI Go Reference Go Report Card OpenSSF Scorecard

Crash-safe atomic file writes for Go

A standalone Go library providing atomic file writes (temp→fsync→rename→dir-fsync), path-traversal validation, bounded reads, and streaming writes. Standard-library only, no external runtime dependencies.

Platform Support

Linux only (including Docker/containers). Windows is unsupported by design — os.Rename cannot guarantee atomicity on Windows (golang/go#22397). macOS/BSD may work but is untested.

Install

go get github.com/cplieger/atomicfile/v2@latest

Usage

package main

import (
	"context"
	"log"
	"strings"

	"github.com/cplieger/atomicfile/v2"
)

func main() {
	ctx := context.Background()

	// Atomic write with default mode (0644). The returned Result reports the
	// final path and whether the write is crash-durable.
	res, err := atomicfile.WriteFile(ctx, "/tmp/data.txt", []byte("hello"))
	if err != nil {
		log.Fatal(err)
	}
	log.Printf("wrote %s (durable=%v)", res.Path, res.Durable)

	// Atomic write with custom mode.
	atomicfile.WriteFile(ctx, "/tmp/secret.txt", []byte("s3cr3t"),
		atomicfile.WithMode(0o600))

	// Streaming write from an io.Reader (mode via WithMode).
	atomicfile.WriteReader(ctx, "/tmp/stream.txt", strings.NewReader("streamed"),
		atomicfile.WithMode(0o644))

	// PendingFile for incremental writes (mirrors google/renameio).
	pf, _ := atomicfile.NewPendingFile(ctx, "/tmp/pending.txt")
	defer pf.Cleanup()
	pf.Write([]byte("incremental"))
	pf.Commit(ctx)

	// Preserve existing file permissions across replace.
	atomicfile.WriteFile(ctx, "/tmp/data.txt", []byte("updated"),
		atomicfile.WithPreserveMode())

	// Auto-create parent directories.
	atomicfile.WriteFile(ctx, "/tmp/nested/dir/file.txt", []byte("deep"),
		atomicfile.WithMkdirMode(0o755))

	// Skip fsync for speed (atomicity without durability).
	atomicfile.WriteFile(ctx, "/tmp/cache.txt", []byte("fast"),
		atomicfile.WithNoSync())

	// Bounded read.
	data, _ := atomicfile.ReadBounded(ctx, "/tmp/data.txt", 1<<20)
	log.Printf("read %d bytes", len(data))
}

API

Write Functions

All write primitives return (Result, error). A nil error means the data reached its final path; inspect Result.Durable for crash durability.

  • WriteFile(ctx, path, data, opts ...Option) (Result, error) — atomic write (default mode 0644)
  • WriteReader(ctx, path, r, opts ...Option) (Result, error) — atomic write from io.Reader (uses the io.WriterTo fast path when available; mode via WithMode)
Streaming Writer
  • NewPendingFile(ctx, path, opts ...Option) (*PendingFile, error) — open a temp file for incremental writing (mode via WithMode)
  • (*PendingFile).Commit(ctx) (Result, error) — chmod + fsync + close + rename + dir-fsync (finalize). Idempotent: repeated calls return the first result.
  • (*PendingFile).Cleanup() error — close + remove (abort; no-op after Commit). Safe to defer immediately after NewPendingFile.

PendingFile embeds *os.File, providing the full io.Writer/io.ReaderFrom/fmt.Fprintf surface.

Read Functions
  • ReadBounded(ctx, path, maxBytes) ([]byte, error) — size-checked read; returns ErrFileTooLarge past the limit
Utilities
  • CleanupStaleTemps(dir, maxAge, opts ...Option) (removed int, err error) — remove orphaned temp files left by interrupted writes, returning how many were removed. Only files matching the package's exact temp shape — .atomicfile-<digits>.tmp — older than maxAge are reclaimed; caller-owned files that merely share the prefix or suffix (e.g. .atomicfile-notes.tmp, config.tmp-backup) are never touched.

Result and Durability

type Result struct {
	Path    string // cleaned, absolute final path
	Durable bool   // true only when file + parent-dir fsync both completed
}

A nil error means the data reached its final path: the write either fully succeeded, or — at worst — was renamed into place but the parent-directory fsync failed. Result.Durable distinguishes those two outcomes, so a caller that cares about crash durability inspects Result.Durable rather than decoding error types. A non-nil error always means the data did not reach its final path.

Functional Options

All write functions accept variadic Option values. Omit options for defaults.

Option Description
WithLogger(l) Custom *slog.Logger for diagnostic output (default: slog.Default())
WithMode(mode) File permission (default: 0o644)
WithMkdirMode(mode) Create the parent directory (and missing ancestors) with this mode before writing. Without it, a missing parent is an error.
WithPreserveMode() Stat the target and reuse its mode (like renameio.WithExistingPermissions), falling back to WithMode if it does not exist
WithPreserveOwnership() Stat the target and chown the temp to match its uid/gid (requires CAP_CHOWN; no-op when the target is absent or off Unix)
WithNoSync() Skip fsync for speed (atomicity without durability). Result.Durable is then always false.
WithAllowSymlinkTarget() Permit writing to a symlink path (default: refuse with ErrSymlinkTarget)

Errors

Sentinel Meaning
ErrEmptyPath The path argument was empty
ErrUnsafePath The path failed the local safety check (relative, null byte, or .. traversal)
ErrFileTooLarge The file exceeded the ReadBounded size limit
ErrSymlinkTarget The target is a symlink and WithAllowSymlinkTarget was not set

Hard write failures are reported as *WriteError{Err, Phase}, where Phase identifies the failed step: PhaseTempCreate, PhaseTempWrite, PhaseTempChmod, PhaseTempSync, PhaseTempClose, or PhaseRename. A *WriteError always means the data did not reach its final path; use errors.As to inspect it.

Durability Guarantees

Every atomic write follows the sequence: create temp file → write data → fsync temp → close → rename to final path → fsync parent directory. This ensures that after a crash, the file contains either the complete old content or the complete new content — never a partial write. The directory fsync makes the rename durable even if the system loses power immediately after the call returns.

If the parent-directory fsync fails (for example an EIO from a failing disk), the rename has already completed, so the new content is present at the final path. The write therefore returns a nil error with Result.Durable == false, and logs the fsync failure at Warn. Callers that require strict durability treat Durable == false as actionable (retry or alert); callers that only need atomicity can ignore it or use WithNoSync() to skip the directory fsync entirely.

Note on auto-created directories. When WithMkdirMode creates new parent directories, only the file's immediate parent is fsynced. The newly created intermediate directories are not fsynced into their own parents, so the durability guarantee above applies only when the full parent path already exists. If you need durability into a freshly created directory tree, create and fsync the directories before writing.

By default, all write functions refuse to write to a path that is currently a symlink, returning ErrSymlinkTarget. This is because os.Rename replaces the symlink itself rather than the file it points to — which is rarely the caller's intent and can lead to data loss or security issues.

To opt in to writing through a symlink (replacing the symlink with a regular file), use WithAllowSymlinkTarget(). To write to the link's target instead, resolve it with filepath.EvalSymlinks first.

Temp Files

Every temp file this package creates is named .atomicfile-<digits>.tmp (os.CreateTemp replaces the pattern's * with a decimal random string). CleanupStaleTemps reclaims orphaned temps of exactly that shape and nothing else, so it never deletes a caller-owned file.

Unsupported by Design

Feature Rationale
Windows rename-over semantics Target platform is Linux. os.Rename is atomic on Linux. Windows cannot guarantee atomicity (golang/go#22397). google/renameio also refuses Windows.
fs.FS interop fs.FS is a read-only interface. Atomic writes are inherently outside its scope.
Atomic symlink replacement Out of scope. Use google/renameio if needed.
Umask-aware permissions The library uses Chmod for exact permissions (ignoring umask). This is the correct secure default for server/CLI tools. Equivalent to renameio's WithStaticPermissions.
TempDir cross-mount detection Temp files are always created in the target directory (same mount point), the only correct approach for atomic rename.
os.Root scoped operations Niche security feature not needed by target consumers.

License

GPL-3.0 — see LICENSE.

Documentation

Overview

Package atomicfile provides crash-safe atomic file writes via the temp → fsync → rename → dir-fsync sequence, with path validation and bounded reads. Standard-library only.

Result and durability

The write primitives (WriteFile, WriteReader, and PendingFile.Commit) return a Result alongside an error. A nil error means the data reached its final path; the write either fully succeeded or, at worst, was renamed into place but the parent-directory fsync failed. Result.Durable distinguishes those two outcomes: it is true only when both the file and its parent directory were fsynced, so a caller that cares about crash durability inspects Result.Durable rather than decoding error types. A non-nil error always means the data did NOT reach its final path.

Temp files

Every temp file this package creates is named ".atomicfile-<digits>.tmp" (os.CreateTemp replaces the pattern's "*" with a decimal random string). CleanupStaleTemps reclaims orphaned temps of exactly that shape and nothing else, so it never deletes a caller-owned file.

Index

Examples

Constants

This section is empty.

Variables

View Source
var (
	// ErrEmptyPath is returned when a path argument is empty.
	ErrEmptyPath = errors.New("atomicfile: empty path")
	// ErrUnsafePath is returned when a path fails the local safety check.
	ErrUnsafePath = errors.New("atomicfile: unsafe path")
	// ErrFileTooLarge is returned when a file exceeds the read size limit.
	ErrFileTooLarge = errors.New("atomicfile: file too large")
	// ErrSymlinkTarget is returned when the target path is a symlink and
	// WithAllowSymlinkTarget was not set.
	ErrSymlinkTarget = errors.New("atomicfile: target is a symlink")
)

Sentinel errors.

Functions

func CleanupStaleTemps

func CleanupStaleTemps(dir string, maxAge time.Duration, opts ...Option) (removed int, err error)

CleanupStaleTemps removes temp files in dir that this package created (".atomicfile-<digits>.tmp") and whose mtime is older than maxAge. It returns the number removed. A missing dir is not an error. Best-effort per file: individual stat/remove failures are logged at Debug and skipped; only a readdir failure is returned.

Example
package main

import (
	"fmt"
	"os"
	"path/filepath"
	"time"

	"github.com/cplieger/atomicfile/v2"
)

func main() {
	dir, _ := os.MkdirTemp("", "example")
	defer os.RemoveAll(dir)
	// Simulate an orphaned temp left by an interrupted write.
	stale := filepath.Join(dir, ".atomicfile-123456.tmp")
	_ = os.WriteFile(stale, []byte("partial"), 0o644)
	old := time.Now().Add(-2 * time.Hour)
	_ = os.Chtimes(stale, old, old)

	removed, _ := atomicfile.CleanupStaleTemps(dir, time.Hour)
	fmt.Println(removed)
}
Output:
1

func ReadBounded

func ReadBounded(ctx context.Context, path string, maxBytes int64) ([]byte, error)

ReadBounded opens path, validates its size against maxBytes, and reads it via an io.LimitReader. Returns ErrFileTooLarge if the file exceeds maxBytes (including if it grows past the limit during the read). ctx is checked before the open and before the read.

Unlike the write primitives, ReadBounded does NOT refuse symlink targets: os.Open follows symlinks, so a symlink at path is resolved and its target is read. Callers reading from a directory writable by a less-trusted principal should resolve and confine the path themselves (e.g. filepath.EvalSymlinks plus a root check) before calling.

Example
package main

import (
	"context"
	"fmt"
	"os"
	"path/filepath"

	"github.com/cplieger/atomicfile/v2"
)

func main() {
	dir, _ := os.MkdirTemp("", "example")
	defer os.RemoveAll(dir)
	path := filepath.Join(dir, "data.txt")
	_ = os.WriteFile(path, []byte("bounded"), 0o644)
	data, _ := atomicfile.ReadBounded(context.Background(), path, 1<<20)
	fmt.Println(string(data))
}
Output:
bounded

Types

type Option

type Option func(*cfg)

Option configures an atomic write operation.

func WithAllowSymlinkTarget

func WithAllowSymlinkTarget() Option

WithAllowSymlinkTarget permits writing to a path that is currently a symlink. By default symlink targets are refused with ErrSymlinkTarget. Note the atomic rename REPLACES the symlink with a regular file; it does not write through to the link's target. Resolve with filepath.EvalSymlinks first if that is the intent.

func WithLogger

func WithLogger(l *slog.Logger) Option

WithLogger sets a custom logger. If not provided, slog.Default() is used.

func WithMkdirMode

func WithMkdirMode(mode os.FileMode) Option

WithMkdirMode creates the parent directory (and any missing ancestors) with the given permission before writing. Without it, a missing parent directory is an error.

func WithMode

func WithMode(mode os.FileMode) Option

WithMode sets the permission applied to the written file. Defaults to 0o644.

func WithNoSync

func WithNoSync() Option

WithNoSync skips the fsync on both the temp file and the parent directory, providing atomicity without durability. Result.Durable is then always false.

func WithPreserveMode

func WithPreserveMode() Option

WithPreserveMode stats the target before writing and applies its existing mode to the new file, falling back to the WithMode value if the target does not exist.

func WithPreserveOwnership

func WithPreserveOwnership() Option

WithPreserveOwnership stats the target before writing and chowns the temp file to match the target's uid/gid. Requires CAP_CHOWN or root; a no-op when the target does not exist or off Unix. Best-effort: the chown runs after the temp-file fsync, so unlike content and mode it is not crash-covered. A chown failure is logged at Warn and does not fail the write (the file lands with the writer's ownership).

type PendingFile

type PendingFile struct {
	*os.File
	// contains filtered or unexported fields
}

PendingFile is a temp file open for writing, destined to atomically replace path on Commit. It embeds *os.File, so callers get the full io.Writer/io.ReaderFrom/fmt.Fprintf surface. The configuration is captured at construction and reused at Commit, so durability options cannot drift between the two calls.

Example
package main

import (
	"context"
	"fmt"
	"os"
	"path/filepath"

	"github.com/cplieger/atomicfile/v2"
)

func main() {
	dir, _ := os.MkdirTemp("", "example")
	defer os.RemoveAll(dir)
	path := filepath.Join(dir, "pending.txt")
	pf, _ := atomicfile.NewPendingFile(context.Background(), path)
	defer func() { _ = pf.Cleanup() }()
	_, _ = pf.Write([]byte("incremental"))
	_, _ = pf.Commit(context.Background())
	data, _ := os.ReadFile(path)
	fmt.Println(string(data))
}
Output:
incremental

func NewPendingFile

func NewPendingFile(ctx context.Context, path string, opts ...Option) (*PendingFile, error)

NewPendingFile creates a temp file destined to atomically replace path. Write to it, then call Commit to finalize or Cleanup to abort. Mode defaults to 0o644 (override with WithMode). ctx is checked before the temp is created.

Example
package main

import (
	"context"
	"fmt"
	"os"
	"path/filepath"

	"github.com/cplieger/atomicfile/v2"
)

func main() {
	dir, _ := os.MkdirTemp("", "example")
	defer os.RemoveAll(dir)
	path := filepath.Join(dir, "pending.txt")
	pf, _ := atomicfile.NewPendingFile(context.Background(), path)
	defer func() { _ = pf.Cleanup() }()
	_, _ = pf.Write([]byte("incremental"))
	res, _ := pf.Commit(context.Background())
	data, _ := os.ReadFile(res.Path)
	fmt.Println(string(data))
}
Output:
incremental

func (*PendingFile) Cleanup

func (p *PendingFile) Cleanup() error

Cleanup closes and removes the temp file. It is a no-op once Commit has been called. Safe to defer immediately after NewPendingFile.

func (*PendingFile) Commit

func (p *PendingFile) Commit(ctx context.Context) (Result, error)

Commit runs the durability barrier (chmod, optional fsync, close), applies ownership, atomically renames the temp into place, and fsyncs the parent directory. After Commit, Cleanup is a no-op. Commit is idempotent: repeated calls return the first result. ctx is checked through the temp-side barrier (before chmod, around the fsync, and before close); a context cancelled at or before close aborts and removes the temp. Once the barrier passes, ownership and the rename run without a further ctx check, so a context cancelled in the narrow window between close and rename still commits.

type Result

type Result struct {
	// Path is the cleaned, absolute final path the data was written to.
	Path string
	// Durable reports whether the write is guaranteed to survive a crash:
	// true only when the file and its parent directory were both fsynced.
	// It is false when WithNoSync was set, or when the parent-directory fsync
	// failed after the rename — in that case the data is present at Path but
	// may not survive an immediate power loss, and the fsync failure is logged
	// at Warn.
	Durable bool
}

Result reports the outcome of an atomic write that reached its final path.

func WriteFile

func WriteFile(ctx context.Context, path string, data []byte, opts ...Option) (Result, error)

WriteFile atomically writes data to path. Mode defaults to 0o644 (override with WithMode). A nil error means the data is at path; check Result.Durable for crash durability.

Example
package main

import (
	"context"
	"fmt"
	"os"
	"path/filepath"

	"github.com/cplieger/atomicfile/v2"
)

func main() {
	dir, _ := os.MkdirTemp("", "example")
	defer os.RemoveAll(dir)
	path := filepath.Join(dir, "data.txt")
	res, _ := atomicfile.WriteFile(context.Background(), path, []byte("hello"))
	data, _ := os.ReadFile(path)
	fmt.Println(string(data), res.Durable)
}
Output:
hello true
Example (WithMkdirMode)
package main

import (
	"context"
	"fmt"
	"os"
	"path/filepath"

	"github.com/cplieger/atomicfile/v2"
)

func main() {
	dir, _ := os.MkdirTemp("", "example")
	defer os.RemoveAll(dir)
	path := filepath.Join(dir, "nested", "dir", "file.txt")
	_, _ = atomicfile.WriteFile(context.Background(), path, []byte("deep"),
		atomicfile.WithMkdirMode(0o755))
	data, _ := os.ReadFile(path)
	fmt.Println(string(data))
}
Output:
deep
Example (WithMode)
package main

import (
	"context"
	"fmt"
	"os"
	"path/filepath"

	"github.com/cplieger/atomicfile/v2"
)

func main() {
	dir, _ := os.MkdirTemp("", "example")
	defer os.RemoveAll(dir)
	path := filepath.Join(dir, "secret.txt")
	_, _ = atomicfile.WriteFile(context.Background(), path, []byte("s3cr3t"),
		atomicfile.WithMode(0o600))
	fi, _ := os.Stat(path)
	fmt.Println(fi.Mode().Perm())
}
Output:
-rw-------
Example (WithNoSync)
package main

import (
	"context"
	"fmt"
	"os"
	"path/filepath"

	"github.com/cplieger/atomicfile/v2"
)

func main() {
	dir, _ := os.MkdirTemp("", "example")
	defer os.RemoveAll(dir)
	path := filepath.Join(dir, "cache.txt")
	res, _ := atomicfile.WriteFile(context.Background(), path, []byte("fast"),
		atomicfile.WithNoSync())
	// WithNoSync trades durability for speed, so the result is not durable.
	fmt.Println(res.Durable)
}
Output:
false

func WriteReader

func WriteReader(ctx context.Context, path string, r io.Reader, opts ...Option) (Result, error)

WriteReader atomically writes the contents of r to path. Mode defaults to 0o644 (override with WithMode). If r implements io.WriterTo it is used for efficient copying; that fast path bypasses the per-Read context check, so cancellation is coarse (per-chunk for chunked sources, post-copy for single-shot sources). ctx is still honored at the durability barrier, so a cancelled write leaves no partial target.

Example
package main

import (
	"context"
	"fmt"
	"os"
	"path/filepath"
	"strings"

	"github.com/cplieger/atomicfile/v2"
)

func main() {
	dir, _ := os.MkdirTemp("", "example")
	defer os.RemoveAll(dir)
	path := filepath.Join(dir, "stream.txt")
	_, _ = atomicfile.WriteReader(context.Background(), path,
		strings.NewReader("streamed"), atomicfile.WithMode(0o600))
	data, _ := os.ReadFile(path)
	fmt.Println(string(data))
}
Output:
streamed
Example (WithMode)
package main

import (
	"context"
	"fmt"
	"os"
	"path/filepath"
	"strings"

	"github.com/cplieger/atomicfile/v2"
)

func main() {
	dir, _ := os.MkdirTemp("", "example")
	defer os.RemoveAll(dir)
	path := filepath.Join(dir, "stream.txt")
	_, _ = atomicfile.WriteReader(context.Background(), path,
		strings.NewReader("streamed"), atomicfile.WithMode(0o644))
	data, _ := os.ReadFile(path)
	fmt.Println(string(data))
}
Output:
streamed

type WriteError

type WriteError struct {
	Err   error
	Phase WritePhase
}

WriteError wraps a hard atomic-write failure with the phase that failed. A WriteError always means the data did NOT reach its final path.

func (*WriteError) Error

func (e *WriteError) Error() string

func (*WriteError) Unwrap

func (e *WriteError) Unwrap() error

type WritePhase

type WritePhase int

WritePhase identifies which step of an atomic write failed. Each value appears only on a WriteError, which is returned exclusively for hard failures (the data did not reach its final path). Two outcomes are deliberately absent from this enum because they are not hard failures: a parent-directory fsync failure (surfaced via Result.Durable) and a preserve-ownership chown failure (best-effort, logged at Warn).

const (
	// PhaseTempCreate indicates failure creating the temp file.
	PhaseTempCreate WritePhase = iota + 1
	// PhaseTempWrite indicates failure writing to the temp file.
	PhaseTempWrite
	// PhaseTempChmod indicates failure setting permissions on the temp file.
	PhaseTempChmod
	// PhaseTempSync indicates failure syncing the temp file.
	PhaseTempSync
	// PhaseTempClose indicates failure closing the temp file.
	PhaseTempClose
	// PhaseRename indicates failure renaming temp to the final path.
	PhaseRename
)

func (WritePhase) String

func (p WritePhase) String() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL