Help us shape the future of Go! Take the 2026 Go Developer Survey

walker

package
v1.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 9, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package walker handles directory traversal and file processing

Package walker handles directory traversal and file processing

Package walker handles directory traversal and file processing

Package walker handles directory traversal and file processing

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ContainPath added in v1.4.0

func ContainPath(root, path string) (string, error)

ContainPath resolves all symlinks in path and verifies the result stays inside root. Containment is checked on a path-separator boundary so a root like /home/u/code never admits /home/u/code-evil. It returns the resolved absolute path for opening, so the check and the subsequent read race as little as possible. An error means the file must be skipped, never read.

func IsBinaryContent added in v1.3.0

func IsBinaryContent(content []byte) bool

IsBinaryContent sniffs already-read bytes, avoiding the second open that DetectFile would cost after a read. Only a bounded prefix is examined, and a NUL byte short-circuits before mimetype runs. Callers must have applied the extension fast-paths first via IsBinaryExt/IsTextExt.

func IsBinaryExt added in v1.3.0

func IsBinaryExt(path string) bool

IsBinaryExt reports whether path has a known-binary extension, without any disk I/O. It backs the metadata walk so the skeleton never opens files.

func IsBinaryFile

func IsBinaryFile(path string) bool

IsBinaryFile checks whether path points to a binary file. 1. Fast-path: Known source/text extensions return false (0 disk I/O). 2. Fast-path: Known binary extensions return true (0 disk I/O). 3. Fallback: Magic-number sniffing via gabriel-vasile/mimetype.

func IsSymlink(path string) bool

IsSymlink reports whether path itself is a symlink, without following it.

func IsTextExt added in v1.3.0

func IsTextExt(path string) bool

IsTextExt reports whether path has a known-text extension, without any disk I/O.

func WalkMeta

func WalkMeta(rootDir string, matcher *ignore.IgnoreMatcher, opts ...Option) ([]FileMeta, []SkippedItem, error)

WalkMeta enumerates a directory tree and returns file metadata only: no full file read. It applies the same ignore, extension, heavy-dir, path, and size filters as Walk. Binary filtering is intentionally limited to known-binary extensions here (zero I/O); unknown-extension sniffing happens once in the content walk over bytes already read, so the two walks never open the same file twice just to classify it.

Types

type FileMeta

type FileMeta struct {
	Path       string
	Size       int64
	IsDir      bool
	ModTime    time.Time
	Hidden     bool
	GitIgnored bool
}

FileMeta describes a file without its content, backing the picker's structure-first skeleton so the TUI can open before any reads happen.

type Option

type Option func(*WalkOptions)

func WithConcurrency

func WithConcurrency(enabled bool) Option

func WithContext

func WithContext(ctx context.Context) Option

func WithExtensionMap

func WithExtensionMap(extMap map[string]struct{}) Option

func WithExtensions

func WithExtensions(extensions []string) Option

func WithIncludeBinary

func WithIncludeBinary(include bool) Option

WithIncludeBinary controls whether binary files are included in the output. By default binary files are skipped.

func WithLogger

func WithLogger(logger utils.Logger) Option

func WithMaxFileSize

func WithMaxFileSize(maxBytes int64) Option

func WithMaxWorkers

func WithMaxWorkers(workers int) Option

func WithPathFilter

func WithPathFilter(fn func(relativePath string) bool) Option

WithPathFilter sets a predicate deciding whether a relative path is processed. It runs after all other skip rules (ignore, extension) but before any file is read, so excluded paths never hit the disk.

func WithPreReadFilter

func WithPreReadFilter(fn func(relativePath string) bool) Option

WithPreReadFilter excludes paths before stat/binary detection and content reads. It is intended for cheap name-only rules such as lockfiles and generated bundles.

func WithProgress

func WithProgress(fn ProgressCallback) Option

func WithStats

func WithStats(fn StatsCallback) Option

WithStats receives one aggregate snapshot after a walk completes.

type ProgressCallback

type ProgressCallback func(stats ProgressStats)

type ProgressStats

type ProgressStats struct {
	TotalFiles      int64  // Total files seen
	ProcessedFiles  int64  // Files that passed all filters and were processed
	SkippedFiles    int64  // Files that were skipped for any reason
	TotalDirs       int64  // Total directories seen
	SkippedDirs     int64  // Directories that were skipped
	CurrentFilePath string // Path of the current file being processed (relative)
}

type SkippedItem

type SkippedItem struct {
	Path   string        `json:"path"`
	Reason SkippedReason `json:"reason"`
	IsDir  bool          `json:"is_dir"`
}

SkippedItem holds information about a skipped path.

func Walk

func Walk(rootDir string, matcher *ignore.IgnoreMatcher, walkFn WalkFunc, opts ...Option) ([]SkippedItem, error)

Walk traverses the directory tree starting from rootDir. It returns a list of skipped items and any critical error that occurred.

type SkippedReason

type SkippedReason string

SkippedReason clarifies why a file/directory was not processed.

const (
	ReasonIgnoredRule       SkippedReason = "Ignored (Gitignore/Custom Rule)"
	ReasonFilteredExtension SkippedReason = "Filtered (Extension Mismatch)"
	ReasonFilteredPath      SkippedReason = "Filtered (Path Rule)"
	ReasonSkippedSizeLimit  SkippedReason = "Skipped (Size Limit Exceeded)"
	ReasonSkippedNotRegular SkippedReason = "Skipped (Not a Regular File)"
	ReasonSkippedPermError  SkippedReason = "Skipped (Permission Error)"
	ReasonSkippedWalkError  SkippedReason = "Skipped (Walk Error)"
	ReasonSkippedReadError  SkippedReason = "Skipped (Read Error)"
	ReasonSkippedBinary     SkippedReason = "Skipped (Binary File)"
	ReasonSkippedInfoError  SkippedReason = "Skipped (File Info Error)"
	ReasonSkippedPathError  SkippedReason = "Skipped (Path Calculation Error)"
	ReasonSkippedSmart      SkippedReason = "Skipped (Smart Filter)"
	// ReasonSkippedSymlinkEscape marks a symlink whose target resolves
	// outside the scan root. The link is never read.
	ReasonSkippedSymlinkEscape SkippedReason = "Skipped (Symlink Escapes Root)"
)

type SkippedTracker

type SkippedTracker struct {
	// contains filtered or unexported fields
}

func NewSkippedTracker

func NewSkippedTracker(capacity int) *SkippedTracker

func (*SkippedTracker) Dropped added in v1.2.0

func (st *SkippedTracker) Dropped() int

Dropped reports how many skip events were not recorded because the retention cap was reached, so callers can know the skip list was truncated.

func (*SkippedTracker) Items

func (st *SkippedTracker) Items() []SkippedItem

func (*SkippedTracker) Track

func (st *SkippedTracker) Track(path string, reason SkippedReason, isDir bool)

Track adds a skipped item to the tracker, up to the retention cap. Beyond that the item is dropped and counted so callers can detect overflow.

type StatsCallback

type StatsCallback func(WalkStats)

StatsCallback receives aggregate walk metrics after traversal and workers have finished.

type WalkFunc

type WalkFunc func(relativePath string, content []byte, err error) error

type WalkOptions

type WalkOptions struct {
	Logger        utils.Logger
	Concurrent    bool
	MaxWorkers    int
	MaxFileSize   int64
	ExtensionMap  map[string]struct{}
	IncludeBinary bool
	Context       context.Context
	ProgressFn    ProgressCallback // Add progress callback function
	PreReadFilter func(relativePath string) bool
	StatsFn       StatsCallback
	// PathFilter, when set, is consulted before a file is read. Returning
	// false excludes the file from processing, so filtered paths never cost
	// I/O or token work.
	PathFilter func(relativePath string) bool
}

type WalkStats

type WalkStats struct {
	TotalFiles     int64
	ProcessedFiles int64
	SkippedFiles   int64
	TotalDirs      int64
	SkippedDirs    int64
	BytesRead      int64
	BinarySkipped  int64
	SizeSkipped    int64
	SmartSkipped   int64
	Duration       time.Duration
}

WalkStats summarizes the work performed by a directory walk.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL