Help us shape the future of Go! Take the 2026 Go Developer Survey

file

package
v0.31.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 13, 2026 License: AGPL-3.0 Imports: 25 Imported by: 0

Documentation

Overview

Package file provides functionality for managing, scanning and cleaning files and folders.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrNotReaderAt = errors.New("invalid reader: does not implement io.ReaderAt")
)
View Source 
var ErrZipFileNotExist = errors.New("zip file does not exist")

Functions

func Destroy added in v0.17.0 

func Destroy(ctx context.Context, destroyer models.FileDestroyer, f models.File, fileDeleter *Deleter, deleteFile bool) error

func GetFileSize added in v0.31.0 

func GetFileSize(f models.FS, path string, info fs.FileInfo) (int64, error)

GetFileSize gets the size of the file, taking into account symlinks.

func GetOrCreateFolderHierarchy added in v0.20.0 

func GetOrCreateFolderHierarchy(ctx context.Context, fc models.FolderFinderCreator, path string, rootPaths []string) (*models.Folder, error)

GetOrCreateFolderHierarchy gets the folder for the given path, or creates a folder hierarchy for the given path if one if no existing folder is found. Creates folder entries for each level of the hierarchy that doesn't already exist, up to the provided root paths. Does not create any folders in the file system.

func ModTime added in v0.31.0 

func ModTime(info fs.FileInfo) time.Time

ModTime returns the modification time truncated to seconds.

func SymWalk added in v0.31.0 

func SymWalk(fs models.FS, path string, walkFn fs.WalkDirFunc) error

SymWalk extends filepath.Walk to also follow symlinks

Types

type CleanHandler added in v0.17.0 

type CleanHandler interface {
	HandleFile(ctx context.Context, fileDeleter *Deleter, fileID models.FileID) error
	HandleFolder(ctx context.Context, fileDeleter *Deleter, folderID models.FolderID) error
}

CleanHandler provides a handler for cleaning Files and Folders.

type CleanOptions added in v0.17.0 

type CleanOptions struct {
	Paths []string

	// IgnoreZipFileContents will skip checking the contents of zip files when determining whether to clean a file.
	// This can significantly speed up the clean process, but will potentially miss removed files within zip files.
	// Where users do not modify zip files contents directly, this should be safe to use.
	IgnoreZipFileContents bool

	// Do a dry run. Don't delete any files
	DryRun bool

	// PathFilter are used to determine if a file should be included.
	// Excluded files are marked for cleaning.
	PathFilter PathFilter
}

CleanOptions provides options for scanning files.

type Cleaner added in v0.17.0 

type Cleaner struct {
	FS         models.FS
	Repository Repository

	Handlers  []CleanHandler
	TrashPath string
}

Cleaner scans through stored file and folder instances and removes those that are no longer present on disk.

func (*Cleaner) Clean added in v0.17.0 

func (s *Cleaner) Clean(ctx context.Context, options CleanOptions, progress *job.Progress)

Clean starts the clean process.

type Decorator added in v0.17.0 

type Decorator interface {
	Decorate(ctx context.Context, fs models.FS, f models.File) (models.File, error)
	IsMissingMetadata(ctx context.Context, fs models.FS, f models.File) bool
}

Decorator wraps the Decorate method to add additional functionality while scanning files.

type Deleter added in v0.12.0 

type Deleter struct {
	RenamerRemover RenamerRemover

	TrashPath string // if set, files will be moved to this directory instead of being permanently deleted
	// contains filtered or unexported fields
}

Deleter is used to safely delete files and directories from the filesystem. During a transaction, files and directories are marked for deletion using the Files and Dirs methods. If TrashPath is set, files are moved to trash immediately. Otherwise, they are renamed with a .delete suffix. If the transaction is rolled back, then the files/directories can be restored to their original state with the Rollback method. If the transaction is committed, the marked files are then deleted from the filesystem using the Commit method.

func NewDeleter added in v0.12.0 

func NewDeleter() *Deleter

func NewDeleterWithTrash added in v0.30.0 

func NewDeleterWithTrash(trashPath string) *Deleter

func (*Deleter) Commit added in v0.12.0 

func (d *Deleter) Commit()

Commit deletes all files marked for deletion and clears the marked list. When using trash, files have already been moved during renameForDelete, so this just clears the tracking. Otherwise, permanently delete the .delete files. Any errors encountered are logged. All files will be attempted, regardless of the errors encountered.

func (*Deleter) Dirs added in v0.12.0 

func (d *Deleter) Dirs(paths []string) error

Dirs designates directories to be deleted. Each directory marked will be renamed to add a `.delete` suffix. An error is returned if a directory could not be renamed. Note that if an error is returned, then some directories may be left renamed. Abort should be called to restore marked files/directories if this function returns an error.

func (*Deleter) DirsWithoutTrash added in v0.30.0 

func (d *Deleter) DirsWithoutTrash(paths []string) error

DirsWithoutTrash designates directories to be deleted, bypassing the trash directory. Directories will be permanently deleted even if TrashPath is configured. This is useful for deleting generated directories that can be easily recreated.

func (*Deleter) Files added in v0.12.0 

func (d *Deleter) Files(paths []string) error

Files designates files to be deleted. Each file marked will be renamed to add a `.delete` suffix. An error is returned if a file could not be renamed. Note that if an error is returned, then some files may be left renamed. Abort should be called to restore marked files if this function returns an error.

func (*Deleter) FilesWithoutTrash added in v0.30.0 

func (d *Deleter) FilesWithoutTrash(paths []string) error

FilesWithoutTrash designates files to be deleted, bypassing the trash directory. Files will be permanently deleted even if TrashPath is configured. This is useful for deleting generated files that can be easily recreated.

func (*Deleter) RegisterHooks added in v0.17.0 

func (d *Deleter) RegisterHooks(ctx context.Context)

RegisterHooks registers post-commit and post-rollback hooks.

func (*Deleter) Rollback added in v0.12.0 

func (d *Deleter) Rollback()

Rollback tries to rename all marked files and directories back to their original names and clears the marked list. Any errors encountered are logged. All files will be attempted regardless of any errors occurred.

type DirMakerStatRenamer added in v0.20.0 

type DirMakerStatRenamer interface {
	Statter
	Renamer
	Mkdir(name string, perm os.FileMode) error
	Remove(name string) error
}

type Filter added in v0.17.0 

type Filter interface {
	Accept(ctx context.Context, f models.File) bool
}

Filter provides a filter function for Files.

type FilterFunc added in v0.17.0 

type FilterFunc func(ctx context.Context, f models.File) bool

func (FilterFunc) Accept added in v0.17.0 

func (ff FilterFunc) Accept(ctx context.Context, f models.File) bool

type FilteredDecorator added in v0.17.0 

type FilteredDecorator struct {
	Decorator
	Filter
}

func (*FilteredDecorator) Decorate added in v0.17.0 

func (d *FilteredDecorator) Decorate(ctx context.Context, fs models.FS, f models.File) (models.File, error)

Decorate runs the decorator if the filter accepts the file.

func (*FilteredDecorator) IsMissingMetadata added in v0.17.0 

func (d *FilteredDecorator) IsMissingMetadata(ctx context.Context, fs models.FS, f models.File) bool

type FilteredHandler added in v0.17.0 

type FilteredHandler struct {
	Handler
	Filter
}

FilteredHandler is a Handler runs only if the filter accepts the file.

func (*FilteredHandler) Handle added in v0.17.0 

func (h *FilteredHandler) Handle(ctx context.Context, f models.File, oldFile models.File) error

Handle runs the handler if the filter accepts the file.

type FingerprintCalculator added in v0.17.0 

type FingerprintCalculator interface {
	CalculateFingerprints(f *models.BaseFile, o Opener, useExisting bool) ([]models.Fingerprint, error)
}

FingerprintCalculator calculates a fingerprint for the provided file.

type Handler added in v0.17.0 

type Handler interface {
	Handle(ctx context.Context, f models.File, oldFile models.File) error
}

Handler provides a handler for Files.

type Importer added in v0.23.0 

type Importer struct {
	ReaderWriter models.FileFinderCreator
	FolderStore  models.FolderFinderCreator
	Input        jsonschema.DirEntry
	// contains filtered or unexported fields
}

func (*Importer) Create added in v0.23.0 

func (i *Importer) Create(ctx context.Context) (*int, error)

func (*Importer) FindExistingID added in v0.23.0 

func (i *Importer) FindExistingID(ctx context.Context) (*int, error)

func (*Importer) Name added in v0.23.0 

func (i *Importer) Name() string

func (*Importer) PostImport added in v0.23.0 

func (i *Importer) PostImport(ctx context.Context, id int) error

func (*Importer) PreImport added in v0.23.0 

func (i *Importer) PreImport(ctx context.Context) error

func (*Importer) Update added in v0.23.0 

func (i *Importer) Update(ctx context.Context, id int) error

type Mover added in v0.20.0 

type Mover struct {
	Renamer DirMakerStatRenamer
	Files   models.FileFinderUpdater
	Folders models.FolderReaderWriter
	// contains filtered or unexported fields
}

func NewMover added in v0.20.0 

func NewMover(fileStore models.FileFinderUpdater, folderStore models.FolderReaderWriter, rootPaths []string) *Mover

func (*Mover) CreateFolderHierarchy added in v0.20.0 

func (m *Mover) CreateFolderHierarchy(path string) error

func (*Mover) Move added in v0.20.0 

func (m *Mover) Move(ctx context.Context, f models.File, folder *models.Folder, basename string) error

Move moves the file to the given folder and basename. If basename is empty, then the existing basename is used. Assumes that the parent folder exists in the filesystem.

func (*Mover) RegisterHooks added in v0.20.0 

func (m *Mover) RegisterHooks(ctx context.Context)

type Opener added in v0.17.0 

type Opener interface {
	Open() (io.ReadCloser, error)
}

Opener provides an interface to open a file.

type OsFS added in v0.17.0 

type OsFS struct{}

OsFS is a file system backed by the OS.

func (*OsFS) Create added in v0.20.0 

func (f *OsFS) Create(name string) (*os.File, error)

func (*OsFS) IsPathCaseSensitive added in v0.17.2 

func (f *OsFS) IsPathCaseSensitive(path string) (bool, error)

func (*OsFS) Lstat added in v0.17.0 

func (f *OsFS) Lstat(name string) (fs.FileInfo, error)

func (*OsFS) MkdirAll added in v0.20.0 

func (f *OsFS) MkdirAll(path string, perm fs.FileMode) error

func (*OsFS) Open added in v0.17.0 

func (f *OsFS) Open(name string) (fs.ReadDirFile, error)

func (*OsFS) OpenZip added in v0.17.0 

func (f *OsFS) OpenZip(name string, size int64) (models.ZipFS, error)

func (*OsFS) Remove added in v0.20.0 

func (f *OsFS) Remove(name string) error

func (*OsFS) RemoveAll added in v0.20.0 

func (f *OsFS) RemoveAll(path string) error

func (*OsFS) Rename added in v0.20.0 

func (f *OsFS) Rename(oldpath, newpath string) error

func (*OsFS) Stat added in v0.17.2 

func (f *OsFS) Stat(name string) (fs.FileInfo, error)

type PathFilter added in v0.17.0 

type PathFilter interface {
	Accept(ctx context.Context, path string, info fs.FileInfo, zipFilePath string) bool
}

PathFilter provides a filter function for paths.

type PathFilterFunc added in v0.17.0 

type PathFilterFunc func(path string) bool

func (PathFilterFunc) Accept added in v0.17.0 

func (pff PathFilterFunc) Accept(path string) bool

type Renamer added in v0.20.0 

type Renamer interface {
	Rename(oldpath, newpath string) error
}

type RenamerRemover added in v0.12.0 

type RenamerRemover interface {
	Renamer
	Remove(name string) error
	RemoveAll(path string) error
	Statter
}

RenamerRemover provides access to the Rename and Remove functions.

type Repository added in v0.17.0 

type Repository struct {
	TxnManager models.TxnManager

	File   models.FileReaderWriter
	Folder models.FolderReaderWriter
}

Repository provides access to storage methods for files and folders.

func NewRepository added in v0.24.0 

func NewRepository(repo models.Repository) Repository

func (*Repository) WithDB added in v0.24.0 

func (r *Repository) WithDB(ctx context.Context, fn txn.TxnFunc) error

func (*Repository) WithReadTxn added in v0.24.0 

func (r *Repository) WithReadTxn(ctx context.Context, fn txn.TxnFunc) error

func (*Repository) WithTxn added in v0.24.0 

func (r *Repository) WithTxn(ctx context.Context, fn txn.TxnFunc) error

type ScanFileResult added in v0.31.0 

type ScanFileResult struct {
	File               models.File
	New                bool
	Renamed            bool
	Updated            bool
	FingerprintChanged bool
}

func (ScanFileResult) IsUnchanged added in v0.31.0 

func (r ScanFileResult) IsUnchanged() bool

type ScannedFile added in v0.31.0 

type ScannedFile struct {
	*models.BaseFile
	FS   models.FS
	Info fs.FileInfo
}

ScannedFile represents a file being scanned.

type Scanner 

type Scanner struct {
	FS                    models.FS
	Repository            Repository
	FingerprintCalculator FingerprintCalculator

	// ZipFileExtensions is a list of file extensions that are considered zip files.
	// Extension does not include the . character.
	ZipFileExtensions []string

	// ScanFilters are used to determine if a file should be scanned.
	ScanFilters []PathFilter

	// HandlerRequiredFilters are used to determine if an unchanged file needs to be handled
	HandlerRequiredFilters []Filter

	// FileDecorators are applied to files as they are scanned.
	FileDecorators []Decorator

	// handlers are called after a file has been scanned.
	FileHandlers []Handler

	// RootPaths form the top-level paths for the library.
	// Used to determine the root of the folder hierarchy when creating folders.
	RootPaths []string

	// Rescan indicates whether files should be rescanned even if they haven't changed.
	Rescan bool
	// contains filtered or unexported fields
}

Scanner scans files into the database.

The scan process works using two goroutines. The first walks through the provided paths in the filesystem. It runs each directory entry through the provided ScanFilters. If none of the filter Accept methods return true, then the file/directory is ignored. Any folders found are handled immediately. Files inside zip files are also handled immediately. All other files encountered are sent to the second goroutine queue.

Folders are handled by checking if the folder exists in the database, by its full path. If a folder entry already exists, then its mod time is updated (if applicable). If the folder does not exist in the database, then a new folder entry its created.

Files are handled by first querying for the file by its path. If the file entry exists in the database, then the mod time is compared to the value in the database. If the mod time is different then file is marked as updated - it recalculates any fingerprints and fires decorators, then the file entry is updated and any applicable handlers are fired.

If the file entry does not exist in the database, then fingerprints are calculated for the file. It then determines if the file is a rename of an existing file by querying for file entries with the same fingerprint. If any are found, it checks each to see if any are missing in the file system. If one is, then the file is treated as renamed and its path is updated. If none are missing, or many are, then the file is treated as a new file.

If the file is not a renamed file, then the decorators are fired and the file is created, then the applicable handlers are fired.

func (*Scanner) AcceptEntry added in v0.31.0 

func (s *Scanner) AcceptEntry(ctx context.Context, path string, info fs.FileInfo, zipFilePath string) bool

AcceptEntry determines if the file entry should be accepted for scanning

func (*Scanner) IsZipFile added in v0.31.0 

func (s *Scanner) IsZipFile(path string) bool

IsZipFile determines if the provided path is a zip file based on its extension.

func (*Scanner) ScanFile added in v0.31.0 

func (s *Scanner) ScanFile(ctx context.Context, f ScannedFile) (*ScanFileResult, error)

ScanFile scans the provided file into the database, returning the scan result.

func (*Scanner) ScanFolder added in v0.31.0 

func (s *Scanner) ScanFolder(ctx context.Context, file ScannedFile) (*models.Folder, error)

ScanFolder scans the provided folder into the database, returning the folder entry. If the folder already exists, it is updated if necessary.

type StashIgnoreFilter added in v0.31.0 

type StashIgnoreFilter struct {
	// contains filtered or unexported fields
}

StashIgnoreFilter implements PathFilter to exclude files/directories based on .stashignore files with gitignore-style patterns.

func NewStashIgnoreFilter added in v0.31.0 

func NewStashIgnoreFilter() *StashIgnoreFilter

NewStashIgnoreFilter creates a new StashIgnoreFilter.

func (*StashIgnoreFilter) Accept added in v0.31.0 

func (f *StashIgnoreFilter) Accept(ctx context.Context, path string, info fs.FileInfo, libraryRoot string, zipFilePath string) bool

Accept returns true if the path should be included in the scan. It checks for .stashignore files in the directory hierarchy and applies gitignore-style pattern matching. The libraryRoot parameter bounds the search for .stashignore files - only directories within the library root are checked. zipFilepath is the path of the zip file if the file is inside a zip. .stashignore files will not be read within zip files.

type Statter added in v0.20.0 

type Statter interface {
	Stat(name string) (fs.FileInfo, error)
}

type ZipDestroyer added in v0.17.0 

type ZipDestroyer struct {
	FileDestroyer   models.FileFinderDestroyer
	FolderDestroyer models.FolderFinderDestroyer
}

func (*ZipDestroyer) DestroyZip added in v0.17.0 

func (d *ZipDestroyer) DestroyZip(ctx context.Context, f models.File, fileDeleter *Deleter, deleteFile bool) error

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.