Help us shape the future of Go! Take the 2026 Go Developer Survey

collectionplan

package
v1.12.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 26, 2026 License: Apache-2.0 Imports: 12 Imported by: 0

Documentation

Overview

Package collectionplan proposes how to group the files of an archive into collections, from where they are and what they are, without reading any of them. It is the quick plan of the automatic collections: a folder of the archive becomes a collection, the small folders are gathered, a very big folder is split along its subfolders, and what cannot be read, what is a duplicate and what is not worth indexing is listed with its reason instead of being lost. The plan is a proposal: the person edits it before anything is created.

The package does no input or output beyond ScanDir: the same Plan serves a folder of the computer and the listing of a drive, whatever lists the entries.

Index

Constants

This section is empty.

Variables

View Source
var ErrTooManyFiles = errors.New("collectionplan: the archive has more files than a plan can hold")

ErrTooManyFiles is returned for a listing longer than Options.MaxPlanFiles.

Functions

func DefaultReadable

func DefaultReadable(ext string) bool

DefaultReadable is the set of formats the reader turns into text: PDF, the Office formats, HTML and plain text.

Types

type Collection

type Collection struct {
	Name  string
	Files []string
	Bytes int64
	// Folders are the folders of the archive the files come from, relative to its root ("" is the root itself).
	Folders []string
}

Collection is a group of files proposed as one collection.

type Duplicate

type Duplicate struct {
	Path string
	Of   string
}

Duplicate is a file with the same content as an earlier one (by path); only the first is in a collection.

type Entry

type Entry struct {
	// Path is slash-separated and relative to the root of the archive.
	Path    string
	Size    int64
	ModTime time.Time
	// Hash is the hash of the content when the scanner computed one; two entries with the same non-empty hash are the same file.
	Hash string
	// Link is set for a symbolic link: it is not followed, and the plan leaves it out.
	Link bool
	// Problem is set when the scanner could not read the entry (a folder or a file it has no right to): it is listed, not lost.
	Problem string
}

Entry is one file of the archive, as listed by whatever scanned the source. Nothing in it needs the content of the file.

func ScanDir

func ScanDir(root string, hash bool) ([]Entry, error)

ScanDir lists the files under root without reading them, unless hash is set: then each regular file is read once, on this computer, to give its content hash, which is what finds duplicates. A symbolic link is listed as a link and not followed. What cannot be read (a folder without the right to list it, a file that cannot be opened for its hash) is listed with a Problem instead of stopping the scan. Paths are slash-separated and relative to root.

type Excluded

type Excluded struct {
	Path   string
	Reason string
}

Excluded is a file the plan leaves out, with why.

type Options

type Options struct {
	// Readable says whether a file extension (lower case, with its dot) is one the reader can turn into text. Nil means DefaultReadable.
	Readable func(ext string) bool
	// MinFiles is how many readable files a folder needs to be a collection of its own; smaller folders are gathered into one collection
	// named MiscName. Default 2: only a folder of a single file is gathered. A higher value gathers real groups: on the test bench
	// (scripts/collections-eval), with half of the folders right, gathering the folders of up to 1, 2 and 4 files leaves 52 %, 54 % and
	// 58 % of the files to fix afterwards, against 51 % when nothing is gathered.
	MinFiles int
	// MaxFiles is the size above which a collection is split along its subfolders, when it has subfolders big enough to stand alone.
	// Default 400.
	MaxFiles int
	// MaxPlanFiles refuses a listing longer than this (ErrTooManyFiles). Default 100000.
	MaxPlanFiles int
	// MiscName is the name of the collection that gathers the small folders and the files at the root. Default "Miscellaneous".
	MiscName string
}

Options tunes the plan. The zero value is usable.

type Plan

type Plan struct {
	Collections []Collection
	Excluded    []Excluded
	Duplicates  []Duplicate
}

Plan is the proposal.

func Make

func Make(entries []Entry, opts Options) (Plan, error)

Make proposes the collections of an archive. The same entries, in any order, give the same plan.

func (Plan) Assignments

func (p Plan) Assignments() map[string]string

Assignments maps each file of the plan to the name of its collection.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL