Documentation
¶
Overview ¶
* Unless explicitly stated otherwise all files in this repository are licensed under the Apache-2.0 License. * * This product includes software developed at Datadog (https://www.datadoghq.com) Copyright 2024 Datadog, Inc.
* Unless explicitly stated otherwise all files in this repository are licensed under the Apache-2.0 License. * * This product includes software developed at Datadog (https://www.datadoghq.com) Copyright 2024 Datadog, Inc.
* Unless explicitly stated otherwise all files in this repository are licensed under the Apache-2.0 License. * * This product includes software developed at Datadog (https://www.datadoghq.com) Copyright 2024 Datadog, Inc.
Index ¶
- Variables
- func ChartRoots(paths []string) []string
- func GetExcludePaths(pathExpressions string) ([]string, error)
- func IgnoreDamagedFile(ctx context.Context, path string) bool
- func IsHelmChartFile(path string, chartRoots []string) bool
- func IsNestedRenderedChart(root string, renderedRoots []string) bool
- func IsTerraformCacheDir(path string) bool
- type ExtractedPath
- type FileFilter
- type FileSystemSourceProvider
- func (s *FileSystemSourceProvider) AddUnfilteredPaths(paths []string)
- func (s *FileSystemSourceProvider) BuildInventoryFromPrebuilt(ctx context.Context, extensions model.Extensions, chartPool utils.PoolOptions, ...) ([]InventoryFile, error)
- func (s *FileSystemSourceProvider) ChartFilter() *PathFilter
- func (s *FileSystemSourceProvider) ContentCache() map[string][]byte
- func (s *FileSystemSourceProvider) GetBasePaths() []string
- func (s *FileSystemSourceProvider) ReleaseContentCache()
- func (s *FileSystemSourceProvider) SetChartFilter(f *PathFilter)
- func (s *FileSystemSourceProvider) SetPrebuiltWalk(paths, chartRoots []string, contentCache map[string][]byte)
- func (s *FileSystemSourceProvider) TerraformFiles(ctx context.Context) ([]string, error)
- func (s *FileSystemSourceProvider) WalkInventory(ctx context.Context, extensions model.Extensions, chartPool utils.PoolOptions, ...) ([]InventoryFile, error)
- type InventoryFile
- type MemorySourceProvider
- func (m *MemorySourceProvider) ExcludesAny(paths ...string) bool
- func (m *MemorySourceProvider) GetBasePaths() []string
- func (m *MemorySourceProvider) ReadFile(p string) ([]byte, error)
- func (m *MemorySourceProvider) RecordMissing(path string)
- func (m *MemorySourceProvider) WalkInventory(ctx context.Context, extensions model.Extensions, chartPool utils.PoolOptions, ...) ([]InventoryFile, error)
- type PathFilter
- type SourceProvider
Constants ¶
This section is empty.
Variables ¶
var ( // ErrNotSupportedFile - error representing when a file format is not supported by the scanner ErrNotSupportedFile = errors.New("invalid file format") )
Functions ¶
func ChartRoots ¶ added in v1.20.0
ChartRoots returns the sorted, slash-separated directories of the Chart.yaml files among paths: the Helm chart roots of pushed content, in the form the analyzer produces for a disk walk.
func GetExcludePaths ¶
GetExcludePaths gets all the files that should be excluded
func IgnoreDamagedFile ¶ added in v1.7.2
IgnoreDamagedFile reports whether a damaged or inaccessible file should be skipped.
func IsHelmChartFile ¶ added in v1.17.0
IsHelmChartFile reports whether path is part of the Helm structure of one of chartRoots: a file Helm reads at the root (Chart.yaml, values*.yaml, ...) or anything under its templates/, crds/ or charts/. Other files that merely sit under a chart root (Terraform, plain manifests, CI workflows) are not, so a rendered chart does not hide them from their own parsers. A nested chart root is covered only when it sits under a parent's charts/, the one place Helm loads subcharts from. A "." root is a chart at the workspace (or scan) root.
func IsNestedRenderedChart ¶ added in v1.17.0
IsNestedRenderedChart reports whether root is a subchart of an already rendered chart, which rendered it as part of its own tree.
func IsTerraformCacheDir ¶ added in v1.7.2
IsTerraformCacheDir reports whether path is a Terraform/Terragrunt cache directory that should be skipped during repository walks.
Types ¶
type ExtractedPath ¶
type ExtractedPath struct {
Path []string
ExtractionMap map[string]model.ExtractedPathObject
}
ExtractedPath is a struct that contains the paths, temporary paths to remove and extraction map path of the sources Path is the slice of paths to scan ExtractionMap is a map that correlates the temporary path to the given path RemoveTmp is the slice containing temporary paths to be removed
func GetKuberneterSources ¶
func GetKuberneterSources(ctx context.Context, source []string, destinationPath string) (ExtractedPath, error)
GetKuberneterSources uses Kubernetes API to download runtime resources After downloading, scan the files as normal local files
func GetSources ¶
GetSources goes through the source slice, and determines the of source type (ex: zip, git, local). It than extracts the files to be scanned. If the source given is not local, a temp dir will be created where the files will be stored.
type FileFilter ¶ added in v1.20.0
FileFilter reports whether a file is left out of the scan by its path filters. Rendered Helm files are checked against it because a chart renders the files of dependencies from anywhere in the scan.
type FileSystemSourceProvider ¶
type FileSystemSourceProvider struct {
// contains filtered or unexported fields
}
FileSystemSourceProvider provides a path to be scanned and a list of files which will not be scanned
func NewFileSystemSourceProvider ¶
func NewFileSystemSourceProvider(ctx context.Context, paths, excludes, onlyPaths []string) (*FileSystemSourceProvider, error)
NewFileSystemSourceProvider initializes a FileSystemSourceProvider with path and files that will be ignored
func (*FileSystemSourceProvider) AddUnfilteredPaths ¶ added in v1.9.0
func (s *FileSystemSourceProvider) AddUnfilteredPaths(paths []string)
func (*FileSystemSourceProvider) BuildInventoryFromPrebuilt ¶ added in v1.7.2
func (s *FileSystemSourceProvider) BuildInventoryFromPrebuilt(ctx context.Context, extensions model.Extensions, chartPool utils.PoolOptions, chartFn func(ctx context.Context, chartPath string) (skip bool)) ([]InventoryFile, error)
BuildInventoryFromPrebuilt renders Helm charts on chartPool and filters pre-collected paths.
func (*FileSystemSourceProvider) ChartFilter ¶ added in v1.20.0
func (s *FileSystemSourceProvider) ChartFilter() *PathFilter
ChartFilter returns the filter that decides which charts render.
func (*FileSystemSourceProvider) ContentCache ¶ added in v1.7.2
func (s *FileSystemSourceProvider) ContentCache() map[string][]byte
ContentCache returns bytes read during analyzer classification, keyed by path.
func (*FileSystemSourceProvider) GetBasePaths ¶
func (s *FileSystemSourceProvider) GetBasePaths() []string
GetBasePaths returns base path of FileSystemSourceProvider
func (*FileSystemSourceProvider) ReleaseContentCache ¶ added in v1.16.0
func (s *FileSystemSourceProvider) ReleaseContentCache()
ReleaseContentCache drops the cached file bytes: only needed during the sink phase (parsed files keep their content in OriginalData), and the map is shared with the scan Client, so clearing here releases both references.
func (*FileSystemSourceProvider) SetChartFilter ¶ added in v1.20.0
func (s *FileSystemSourceProvider) SetChartFilter(f *PathFilter)
SetChartFilter replaces the filter that decides which charts render, which defaults to the provider's own excludes and only-paths. A scan sets the user's filters when its excludes also list files kept out of the raw scan only, such as JSON templates, whose rendered documents are still scanned.
func (*FileSystemSourceProvider) SetPrebuiltWalk ¶ added in v1.7.2
func (s *FileSystemSourceProvider) SetPrebuiltWalk(paths, chartRoots []string, contentCache map[string][]byte)
SetPrebuiltWalk reuses analyzer walk inventory to skip a second tree walk.
func (*FileSystemSourceProvider) TerraformFiles ¶ added in v1.9.0
func (s *FileSystemSourceProvider) TerraformFiles(ctx context.Context) ([]string, error)
TerraformFiles returns Terraform/OpenTofu config paths used for module discovery before scanning: native .tf/.tofu files plus .tf.json/.tofu.json files.
func (*FileSystemSourceProvider) WalkInventory ¶ added in v1.7.2
func (s *FileSystemSourceProvider) WalkInventory(ctx context.Context, extensions model.Extensions, chartPool utils.PoolOptions, chartFn func(ctx context.Context, chartPath string) (skip bool)) ([]InventoryFile, error)
WalkInventory collects matching files, calling chartFn at each Helm chart root. The analyzer-prebuilt inventory renders charts on chartPool (see renderChartsShallowFirst); a directory walk calls chartFn as it goes.
type InventoryFile ¶ added in v1.7.2
InventoryFile is a discovered file and its matched extension token.
type MemorySourceProvider ¶ added in v1.7.0
type MemorySourceProvider struct {
// contains filtered or unexported fields
}
MemorySourceProvider serves a fixed set of files (pushed over HTTP) to the scan pipeline, reading their content through a vfs.FS. It replaces the disk-walking FileSystemSourceProvider in the server's content-push path, so there is no filesystem walk and no symlink/SameFile handling.
The server's services share one provider, so their scans go through the shared prepare (runner.PrepareMemorySources), which lists files and renders pushed Helm charts via WalkInventory.
func NewMemorySourceProvider ¶ added in v1.7.0
func NewMemorySourceProvider(fsys vfs.FS, paths, ignorePaths, onlyPaths []string) *MemorySourceProvider
NewMemorySourceProvider builds a provider over the given paths, reading each file's content from fsys (the same in-memory FS used for cross-file resolution, so content is stored once). ignorePaths/onlyPaths are the config's global path filters; they apply the same ignore-paths/only-paths semantics the disk source provider applies, so server-mode scans honor them too.
func (*MemorySourceProvider) ExcludesAny ¶ added in v1.20.0
func (m *MemorySourceProvider) ExcludesAny(paths ...string) bool
ExcludesAny reports whether ignore-paths or only-paths leave out a file known by paths, as PathFilter.ExcludesAny does for the disk provider.
func (*MemorySourceProvider) GetBasePaths ¶ added in v1.7.0
func (m *MemorySourceProvider) GetBasePaths() []string
GetBasePaths returns a synthetic root. Pushed paths are reported back exactly as they were pushed, whether relative or absolute, so there is no real base to name. The server's entry point (scan.Client.Scan) never consults this: it returns the raw vulnerabilities, skipping the summary step that relativizes file names against a base path.
func (*MemorySourceProvider) ReadFile ¶ added in v1.17.0
func (m *MemorySourceProvider) ReadFile(p string) ([]byte, error)
ReadFile reads a pushed file through the provider's FS.
func (*MemorySourceProvider) RecordMissing ¶ added in v1.17.0
func (m *MemorySourceProvider) RecordMissing(path string)
RecordMissing records an escalation request when the provider's FS tracks missing paths (the in-memory one does; the real disk cannot).
func (*MemorySourceProvider) WalkInventory ¶ added in v1.17.0
func (m *MemorySourceProvider) WalkInventory(ctx context.Context, extensions model.Extensions, chartPool utils.PoolOptions, chartFn func(ctx context.Context, chartPath string) (rendered bool)) ([]InventoryFile, error)
WalkInventory is the pushed-content counterpart of the disk provider's WalkInventory: it calls chartFn for each pushed chart root the path filters keep (shallow-first, subcharts skipped once their parent rendered, on chartPool) and returns the eligible pushed files minus the Helm files of the charts that rendered or that the filters leave out.
type PathFilter ¶ added in v1.20.0
type PathFilter struct {
// contains filtered or unexported fields
}
PathFilter applies a scan's ignore-paths and only-paths, to Helm charts and the files they render. It does no I/O and never changes, so it is safe for concurrent use. A nil PathFilter excludes nothing.
A pattern with glob syntax matches the paths it names on disk and also the paths it matches as text, as the in-memory and rule filters do. A rendered file of a packaged dependency names no file on disk, so a glob over the disk alone would drop it.
func ExpandPathFilter ¶ added in v1.20.0
func ExpandPathFilter(ignorePatterns, onlyPatterns []string) (*PathFilter, error)
ExpandPathFilter builds a PathFilter from ignore-paths and only-paths patterns, expanding their globs on disk and keeping them to match as text.
func NewLexicalPathFilter ¶ added in v1.20.0
func NewLexicalPathFilter(ignorePatterns, onlyPatterns []string) *PathFilter
NewLexicalPathFilter builds a PathFilter that matches patterns as text only, for files that are not on disk, such as pushed ones. Every pattern is kept as a glob, which matches a literal path or directory the same way.
func NewPathFilter ¶ added in v1.20.0
func NewPathFilter(ignorePaths, onlyPaths []string) *PathFilter
NewPathFilter builds a PathFilter from expanded ignore-paths and only-paths.
func (*PathFilter) ChartInScope ¶ added in v1.20.0
func (f *PathFilter) ChartInScope(root string) bool
ChartInScope reports whether the chart at root is rendered: neither it nor its Chart.yaml is ignored, and with only-paths, the chart lies within one or contains one (only-paths naming a single template still renders its chart; the files outside are dropped from the output). A glob's literal leading directories stand in for the paths it may name, so a chart a glob could reach renders even when no file of it is on disk.
func (*PathFilter) ExcludesAny ¶ added in v1.20.0
func (f *PathFilter) ExcludesAny(paths ...string) bool
ExcludesAny reports whether the filters leave out a file known by paths, such as a file inside a chart archive and the declaration it is reported at: a path that is ignored, or one below an ignored directory, drops it, and with only-paths, a path within one of them keeps it. Paths need not exist.
type SourceProvider ¶
type SourceProvider interface {
GetBasePaths() []string
}
SourceProvider is the interface a service's source provider implements. GetBasePaths returns the base paths of the provider.