filter

package
v0.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 10, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package filter decides which files a scan should process. Filtering rules are loaded from several sources (built-in defaults, a project's scanoss.json, and the tree's .gitignore), merged into a single deduplicated set, and applied in one pass over the source tree. Skipped files are simply excluded from the scan; the list of them is not tracked, only counted.

The package is low-level: it never imports pkg/scanoss or anything that talks to the API, so file selection can be reasoned about — and used — without a client.

Index

Constants

View Source
const DefaultMaxFileSize int64 = 0

DefaultMaxFileSize is the maximum file size (bytes) to scan. 0 means unlimited, matching scanoss.py's default.

View Source
const DefaultMinFileSize int64 = 0

DefaultMinFileSize is the minimum file size (bytes) to scan. 0 means no minimum: a file is collected however small it is, unless another rule skips it. Raise it with Options.MinSize, or per pattern with a scanoss.json skip.sizes rule.

Variables

This section is empty.

Functions

This section is empty.

Types

type CollectResult

type CollectResult struct {
	Files        []string
	SkippedCount int
}

CollectResult is the outcome of a Collect: the absolute paths to scan and how many files were skipped. The skipped files themselves are not retained.

func Collect

func Collect(root string, o Options) (*CollectResult, error)

Collect walks root once, returning the files to scan and a count of those skipped. Rules are loaded from the enabled sources (defaults, scanoss.json, .gitignore), deduplicated, and applied as a single composite — including the hidden-entry rule, unless Options.IncludeHidden says otherwise. Zero-byte files and symbolic links are always skipped, unconditionally. Symlinked directories are not followed. Returned paths are absolute.

type Options

type Options struct {
	// BuiltinFolderRules applies the built-in directory lists (node_modules, vendor,
	// build output, …). BuiltinFileRules applies the built-in file lists: exact names,
	// extensions and name endings, which together answer one question — is this file
	// worth fingerprinting.
	//
	// Turning either off does not touch the Skip* fields below: those are the caller's
	// own rules, not part of the built-in policy. "No built-in folder lists" and "no
	// folder rules at all" are different requests, and this flag only makes the first.
	BuiltinFolderRules bool
	BuiltinFileRules   bool

	// The caller's own skip rules, layered on top of whatever the flags above admit.
	//
	// These are names, not paths and not globs: "node_modules", never "/node_modules"
	// or "src/**". A name matches at any depth. For paths and patterns use SkipPatterns.
	SkipDirs    []string // directory names
	SkipDirExts []string // directory-name suffixes, e.g. ".egg-info"
	SkipFiles   []string // exact file names
	SkipExts    []string // file extensions, leading dot: ".png"

	// Size bounds, applied as their own rule so they survive turning the built-in
	// lists off. 0 means no bound on that side; the built-in values are
	// DefaultMinFileSize / DefaultMaxFileSize.
	MinSize int64
	MaxSize int64

	GitIgnore bool // honor .gitignore

	// IncludeHidden collects entries whose name begins with a dot. They are excluded by default: a
	// scan wants the project's source, not its tooling. Setting it reaches version-control
	// metadata too — .git and friends are dotted like anything else — which matches the reference
	// implementation, where the equivalent flag has the same reach.
	IncludeHidden bool

	// SkipPatterns are gitignore-style globs: "src/**", "*.min.js". SizeRules bound the
	// size of files matching a glob. The profiles fill both from the project's
	// scanoss.json; a caller may add its own.
	//
	// Unlike the name-based fields above, these are final — KeepManifests never
	// overrides one, because a caller that named a path meant that path.
	SkipPatterns []string
	SizeRules    []SizeRule

	// KeepManifests keeps dependency manifest files (package.json, go.mod, pom.xml, …
	// — see pkg/manifests) even when a skip rule would otherwise drop them. Use it for
	// stages that consume manifests while still pruning everything else. Fingerprint
	// scanning leaves it false: a manifest is a declaration, not a file worth matching.
	KeepManifests bool
}

Options configures a Collect call. Start from Scanning, Fingerprinting or Dependencies or build one literally — a zero Options filters nothing beyond what is never scannable (empty files, symlinks) and hidden entries.

func Dependencies added in v0.8.0

func Dependencies(scanossSettings *settings.Settings) Options

Dependencies collects the files a dependency stage should see. It keeps the manifests the file rules would otherwise drop, but it does not select only manifests: a caller that wants just those filters the result by what its parser handles.

Three things differ from Scanning, and all three are deliberate:

  • the directory list prunes generated trees instead of the scanning ones;
  • manifests are preserved, since they live behind skipped extensions;
  • .gitignore is NOT applied. It answers "should this be versioned", not "is this a dependency": a lock file excluded from git still declares what the project uses, and losing a declaration is worse than analysing one extra.

The built-in folder lists are off and the directory rules are given explicitly: the scanning lists prune venv/ and examples/, where manifests legitimately live. dist/, build/ and target/ are pruned here instead, since a manifest under one of them is build output. The directory-suffix list is the built-in one, which those lists do not disagree on.

func Fingerprinting added in v0.8.0

func Fingerprinting(scanossSettings *settings.Settings) Options

Fingerprinting is Scanning's ruleset over the fingerprinting section. The two apply the same rules and differ only in which rules the project wrote for them.

func Scanning added in v0.8.0

func Scanning(scanossSettings *settings.Settings) Options

Scanning selects the files worth matching: the built-in skip lists apply and .gitignore is honoured. Dependency manifests are dropped — a declaration is not a file to match against the knowledge base.

type SizeRule

type SizeRule struct {
	Patterns []string
	Min      int64
	Max      int64
}

SizeRule is one scanoss.json skip.sizes entry: files matching any of Patterns are skipped when smaller than Min or larger than Max (0 disables a bound).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL