Documentation
¶
Overview ¶
Package collectionplan proposes how to group the files of an archive into collections, from where they are and what they are, without reading any of them. It is the quick plan of the automatic collections: a folder of the archive becomes a collection, the small folders are gathered, a very big folder is split along its subfolders, and what cannot be read, what is a duplicate and what is not worth indexing is listed with its reason instead of being lost. The plan is a proposal: the person edits it before anything is created.
The package does no input or output beyond ScanDir: the same Plan serves a folder of the computer and the listing of a drive, whatever lists the entries.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrTooManyFiles = errors.New("collectionplan: the archive has more files than a plan can hold")
ErrTooManyFiles is returned for a listing longer than Options.MaxPlanFiles.
Functions ¶
func DefaultReadable ¶
DefaultReadable is the set of formats the reader turns into text: PDF, the Office formats, HTML and plain text.
Types ¶
type Collection ¶
type Collection struct {
Name string
Files []string
Bytes int64
// Folders are the folders of the archive the files come from, relative to its root ("" is the root itself).
Folders []string
}
Collection is a group of files proposed as one collection.
type Duplicate ¶
Duplicate is a file with the same content as an earlier one (by path); only the first is in a collection.
type Entry ¶
type Entry struct {
// Path is slash-separated and relative to the root of the archive.
Path string
Size int64
ModTime time.Time
// Hash is the hash of the content when the scanner computed one; two entries with the same non-empty hash are the same file.
Hash string
// Link is set for a symbolic link: it is not followed, and the plan leaves it out.
Link bool
// Problem is set when the scanner could not read the entry (a folder or a file it has no right to): it is listed, not lost.
Problem string
}
Entry is one file of the archive, as listed by whatever scanned the source. Nothing in it needs the content of the file.
func ScanDir ¶
ScanDir lists the files under root without reading them, unless hash is set: then each regular file is read once, on this computer, to give its content hash, which is what finds duplicates. A symbolic link is listed as a link and not followed. What cannot be read (a folder without the right to list it, a file that cannot be opened for its hash) is listed with a Problem instead of stopping the scan. Paths are slash-separated and relative to root.
type Options ¶
type Options struct {
// Readable says whether a file extension (lower case, with its dot) is one the reader can turn into text. Nil means DefaultReadable.
Readable func(ext string) bool
// MinFiles is how many readable files a folder needs to be a collection of its own; smaller folders are gathered into one collection
// named MiscName. Default 2: only a folder of a single file is gathered. A higher value gathers real groups: on the test bench
// (scripts/collections-eval), with half of the folders right, gathering the folders of up to 1, 2 and 4 files leaves 52 %, 54 % and
// 58 % of the files to fix afterwards, against 51 % when nothing is gathered.
MinFiles int
// MaxFiles is the size above which a collection is split along its subfolders, when it has subfolders big enough to stand alone.
// Default 400.
MaxFiles int
// MaxPlanFiles refuses a listing longer than this (ErrTooManyFiles). Default 100000.
MaxPlanFiles int
// MiscName is the name of the collection that gathers the small folders and the files at the root. Default "Miscellaneous".
MiscName string
}
Options tunes the plan. The zero value is usable.
type Plan ¶
type Plan struct {
Collections []Collection
Excluded []Excluded
Duplicates []Duplicate
}
Plan is the proposal.
func Make ¶
Make proposes the collections of an archive. The same entries, in any order, give the same plan.
func (Plan) Assignments ¶
Assignments maps each file of the plan to the name of its collection.