Documentation
¶
Overview ¶
Package filter decides which files a scan should process. Filtering rules are loaded from several sources (built-in defaults, a project's scanoss.json, and the tree's .gitignore), merged into a single deduplicated set, and applied in one pass over the source tree. Skipped files are simply excluded from the scan; the list of them is not tracked, only counted.
The package is standalone: it does not import any other scanoss package, so it can be consumed on its own.
Index ¶
Constants ¶
const DefaultMaxFileSize int64 = 0
DefaultMaxFileSize is the maximum file size (bytes) to scan. 0 means unlimited, matching scanoss.py's default.
const DefaultMinFileSize int64 = 100
DefaultMinFileSize is the minimum file size (bytes) to scan. Files smaller than this are skipped. Mirrors scanoss's historical 100-byte minimum.
Variables ¶
var DefaultSkippedDirExts = []string{
".egg-info",
}
DefaultSkippedDirExts are directory-name suffixes that are skipped (e.g. a directory named "foo.egg-info").
var DefaultSkippedDirs = []string{
"nbproject",
"nbbuild",
"nbdist",
"__pycache__",
"venv",
"_yardoc",
"eggs",
"wheels",
"htmlcov",
"__pypackages__",
"example",
"examples",
"node_modules",
"vendor",
}
DefaultSkippedDirs are directory names that are skipped wholesale (the whole subtree is pruned).
var DefaultSkippedExts = []string{}/* 153 elements not displayed */
DefaultSkippedExts are file-name extensions (suffixes including the leading dot) that are skipped. Compound extensions such as ".min.js" are matched as a full suffix.
var DefaultSkippedFileEndings = []string{
"-doc",
"changelog",
"config",
"copying",
"license",
"authors",
"news",
"licenses",
"notice",
"readme",
"swiftdoc",
"texidoc",
"todo",
"version",
"ignore",
"manifest",
"sqlite",
"sqlite3",
}
DefaultSkippedFileEndings are file-name suffixes that are not extensions (no leading dot), matched case-insensitively against the whole file name. These correspond to scanoss.py's "file endings" entries.
var DefaultSkippedFiles = []string{
"gradlew",
"gradlew.bat",
"mvnw",
"mvnw.cmd",
"gradle-wrapper.jar",
"maven-wrapper.jar",
"thumbs.db",
"babel.config.js",
"license.txt",
"license.md",
"copying.lib",
"makefile",
}
DefaultSkippedFiles are exact file names that are skipped.
Functions ¶
This section is empty.
Types ¶
type CollectResult ¶
CollectResult is the outcome of a Collect: the absolute paths to scan and how many files were skipped. The skipped files themselves are not retained.
func Collect ¶
func Collect(root string, o Options) (*CollectResult, error)
Collect walks root once, returning the files to scan and a count of those skipped. Rules are loaded from the enabled sources (defaults, scanoss.json, .gitignore), deduplicated, and applied as a single composite. Hidden files and directories (names beginning with ".") are always skipped, preserving prior behavior. Symlinked directories are not followed. Returned paths are absolute.
type Composite ¶
type Composite struct {
// contains filtered or unexported fields
}
Composite holds many matchers and skips a path if any of them does. It is itself a Matcher, so composites can nest.
func Build ¶
Build concatenates matchers from every source and removes duplicates by Key(), then wraps the result in a single Composite. Sources are processed in the order given, and the first matcher seen for a given Key() wins (later duplicates are dropped) — so a rule shared across sources (e.g. ".png" in both the defaults and a scanoss.json pattern) is applied once, not repeatedly.
type Defaults ¶
type Defaults struct {
Dirs []string // directory names skipped wholesale
DirExts []string // directory-name suffixes skipped (e.g. ".egg-info")
Files []string // exact file names skipped
Exts []string // file extensions skipped (leading dot)
Endings []string // non-extension file-name suffixes skipped
MinSize int64 // minimum file size; 0 disables
MaxSize int64 // maximum file size; 0 disables (unlimited)
}
Defaults holds the skip lists and size bounds a DefaultSource turns into matchers. Callers normally start from StdDefaults and override fields as needed.
func StdDefaults ¶
func StdDefaults() Defaults
StdDefaults returns the built-in default skip lists and size bounds.
type Matcher ¶
type Matcher interface {
// Match reports whether the path (rel, relative to the scan root) should be
// skipped. info describes the entry.
Match(rel string, info os.FileInfo) bool
// Key is a stable identity used to deduplicate matchers built from different
// sources (e.g. "ext:.png", "dir:vendor").
Key() string
}
Matcher decides whether a single path should be skipped. It is the leaf of the composite pattern; Composite implements it too, so a set of matchers is itself a Matcher. Match returns true when the path should be skipped. Skipped files are not tracked, so a boolean (no reason) is all that is needed.
func DefaultSource ¶
DefaultSource turns the default skip lists and size bounds into matchers.
func GitIgnoreSource ¶
GitIgnoreSource reads the .gitignore at the root of the tree (if present) and returns a matcher for its patterns. If root is a file rather than a directory, its parent directory is used. Missing file is not an error.
func NewMatcher ¶
NewMatcher builds a per-path skip Matcher from o, for callers that evaluate entries one at a time (e.g. a streaming archive extractor) instead of walking a tree with Collect. It applies the default and scanoss.json (Settings) rules and honors PreserveDependencyManifests, so a caller filters exactly the way Collect does — from the same Options. It does NOT apply .gitignore (that needs the whole tree; use Collect). Match returns true when the entry should be skipped.
func SettingsSource ¶
SettingsSource turns scanoss.json skip rules into matchers. Returns nil when s is nil.
type Options ¶
type Options struct {
// Skip* replace the matching built-in default list when non-nil.
SkipDirs []string
SkipFiles []string
SkipExtensions []string
MinSize int64 // minimum file size; 0 uses DefaultMinFileSize
MaxSize int64 // maximum file size; 0 uses DefaultMaxFileSize (unlimited)
Defaults bool // apply the built-in default skip lists
GitIgnore bool // honor .gitignore
// Settings is the scanoss.json skip/folders rules, already resolved to a
// single operation. Nil when there is no scanoss.json.
Settings *Settings
// PreserveDependencyManifests keeps dependency manifest files
// (package.json, go.mod, pom.xml, … — see pkg/manifests) even when a skip
// rule would otherwise drop them. Use it for stages that consume manifests
// (extraction/upload feeding the dependency parser) while still pruning
// everything else. Fingerprint scanning leaves this false — manifests are
// not useful for matching. Default false → unchanged behavior.
PreserveDependencyManifests bool
HFH bool // reserved: high-file-hashing (folder hashing) variants; not yet used
}
Options configures a Collect call. Set the Skip* fields to replace the corresponding default list. Use DefaultOptions for the common case where the built-in defaults and .gitignore are applied.
func DefaultOptions ¶
func DefaultOptions() Options
DefaultOptions returns the common-case Options: the built-in default skip lists and .gitignore are applied, with no scanoss.json.
func IngestOptions ¶
func IngestOptions() Options
IngestOptions returns the options for materialising files a later stage consumes (extraction/upload feeding the dependency parser): the same prune as ScanOptions, but dependency manifests are preserved.
func ScanOptions ¶
func ScanOptions() Options
ScanOptions returns the options for fingerprint scanning: the built-in defaults and .gitignore, with dependency manifests skipped (they are not useful for matching). Alias of DefaultOptions, named for intent.
type Settings ¶
type Settings struct {
Skip Skip
}
Settings is the local, dependency-free mirror of the scanoss.json bits filter needs. Callers map settings.Settings into this.