Documentation
¶
Overview ¶
Package filter decides which files a scan should process. Filtering rules are loaded from several sources (built-in defaults, a project's scanoss.json, and the tree's .gitignore), merged into a single deduplicated set, and applied in one pass over the source tree. Skipped files are simply excluded from the scan; the list of them is not tracked, only counted.
The package is low-level: it never imports pkg/scanoss or anything that talks to the API, so file selection can be reasoned about — and used — without a client.
Index ¶
Constants ¶
const DefaultMaxFileSize int64 = 0
DefaultMaxFileSize is the maximum file size (bytes) to scan. 0 means unlimited, matching scanoss.py's default.
const DefaultMinFileSize int64 = 0
DefaultMinFileSize is the minimum file size (bytes) to scan. 0 means no minimum: a file is collected however small it is, unless another rule skips it. Raise it with Options.MinSize, or per pattern with a scanoss.json skip.sizes rule.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type CollectResult ¶
CollectResult is the outcome of a Collect: the absolute paths to scan and how many files were skipped. The skipped files themselves are not retained.
func Collect ¶
func Collect(root string, o Options) (*CollectResult, error)
Collect walks root once, returning the files to scan and a count of those skipped. Rules are loaded from the enabled sources (defaults, scanoss.json, .gitignore), deduplicated, and applied as a single composite — including the hidden-entry rule, unless Options.IncludeHidden says otherwise. Zero-byte files and symbolic links are always skipped, unconditionally. Symlinked directories are not followed. Returned paths are absolute.
type Options ¶
type Options struct {
// BuiltinFolderRules applies the built-in directory lists (node_modules, vendor,
// build output, …). BuiltinFileRules applies the built-in file lists: exact names,
// extensions and name endings, which together answer one question — is this file
// worth fingerprinting.
//
// Turning either off does not touch the Skip* fields below: those are the caller's
// own rules, not part of the built-in policy. "No built-in folder lists" and "no
// folder rules at all" are different requests, and this flag only makes the first.
BuiltinFolderRules bool
BuiltinFileRules bool
// The caller's own skip rules, layered on top of whatever the flags above admit.
//
// These are names, not paths and not globs: "node_modules", never "/node_modules"
// or "src/**". A name matches at any depth. For paths and patterns use SkipPatterns.
SkipDirs []string // directory names
SkipDirExts []string // directory-name suffixes, e.g. ".egg-info"
SkipFiles []string // exact file names
SkipExts []string // file extensions, leading dot: ".png"
// Size bounds, applied as their own rule so they survive turning the built-in
// lists off. 0 means no bound on that side; the built-in values are
// DefaultMinFileSize / DefaultMaxFileSize.
MinSize int64
MaxSize int64
GitIgnore bool // honor .gitignore
// IncludeHidden collects entries whose name begins with a dot. They are excluded by default: a
// scan wants the project's source, not its tooling. Setting it reaches version-control
// metadata too — .git and friends are dotted like anything else — which matches the reference
// implementation, where the equivalent flag has the same reach.
IncludeHidden bool
// SkipPatterns are gitignore-style globs: "src/**", "*.min.js". SizeRules bound the
// size of files matching a glob. The profiles fill both from the project's
// scanoss.json; a caller may add its own.
//
// Unlike the name-based fields above, these are final — KeepManifests never
// overrides one, because a caller that named a path meant that path.
SkipPatterns []string
SizeRules []SizeRule
// KeepManifests keeps dependency manifest files (package.json, go.mod, pom.xml, …
// — see pkg/manifests) even when a skip rule would otherwise drop them. Use it for
// stages that consume manifests while still pruning everything else. Fingerprint
// scanning leaves it false: a manifest is a declaration, not a file worth matching.
KeepManifests bool
}
Options configures a Collect call. Start from Scanning, Fingerprinting or Dependencies or build one literally — a zero Options filters nothing beyond what is never scannable (empty files, symlinks) and hidden entries.
func Dependencies ¶ added in v0.8.0
Dependencies collects the files a dependency stage should see. It keeps the manifests the file rules would otherwise drop, but it does not select only manifests: a caller that wants just those filters the result by what its parser handles.
Three things differ from Scanning, and all three are deliberate:
- the directory list prunes generated trees instead of the scanning ones;
- manifests are preserved, since they live behind skipped extensions;
- .gitignore is NOT applied. It answers "should this be versioned", not "is this a dependency": a lock file excluded from git still declares what the project uses, and losing a declaration is worse than analysing one extra.
The built-in folder lists are off and the directory rules are given explicitly: the scanning lists prune venv/ and examples/, where manifests legitimately live. dist/, build/ and target/ are pruned here instead, since a manifest under one of them is build output. The directory-suffix list is the built-in one, which those lists do not disagree on.
func Fingerprinting ¶ added in v0.8.0
Fingerprinting is Scanning's ruleset over the fingerprinting section. The two apply the same rules and differ only in which rules the project wrote for them.