provider

package
v1.18.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 28, 2026 License: Apache-2.0 Imports: 21 Imported by: 0

Documentation

Overview

* Unless explicitly stated otherwise all files in this repository are licensed under the Apache-2.0 License. * * This product includes software developed at Datadog (https://www.datadoghq.com) Copyright 2024 Datadog, Inc.

* Unless explicitly stated otherwise all files in this repository are licensed under the Apache-2.0 License. * * This product includes software developed at Datadog (https://www.datadoghq.com) Copyright 2024 Datadog, Inc.

* Unless explicitly stated otherwise all files in this repository are licensed under the Apache-2.0 License. * * This product includes software developed at Datadog (https://www.datadoghq.com) Copyright 2024 Datadog, Inc.

Index

Constants

This section is empty.

Variables

View Source 
var (

	// ErrNotSupportedFile - error representing when a file format is not supported by the scanner
	ErrNotSupportedFile = errors.New("invalid file format")
)

Functions

func GetExcludePaths 

func GetExcludePaths(pathExpressions string) ([]string, error)

GetExcludePaths gets all the files that should be excluded

func IgnoreDamagedFile added in v1.7.2 

func IgnoreDamagedFile(ctx context.Context, path string) bool

IgnoreDamagedFile reports whether a damaged or inaccessible file should be skipped.

func IsHelmChartFile added in v1.17.0 

func IsHelmChartFile(path string, chartRoots []string) bool

IsHelmChartFile reports whether path is part of the Helm structure of one of chartRoots: a file Helm reads at the root (Chart.yaml, values*.yaml, ...) or anything under its templates/, crds/ or charts/. Other files that merely sit under a chart root (Terraform, plain manifests, CI workflows) are not, so a rendered chart does not hide them from their own parsers. A nested chart root is covered only when it sits under a parent's charts/, the one place Helm loads subcharts from. A "." root is a chart at the workspace (or scan) root.

func IsNestedRenderedChart added in v1.17.0 

func IsNestedRenderedChart(root string, renderedRoots []string) bool

IsNestedRenderedChart reports whether root is a subchart of an already rendered chart, which rendered it as part of its own tree.

func IsTerraformCacheDir added in v1.7.2 

func IsTerraformCacheDir(path string) bool

IsTerraformCacheDir reports whether path is a Terraform/Terragrunt cache directory that should be skipped during repository walks.

Types

type ExtractedPath 

type ExtractedPath struct {
	Path          []string
	ExtractionMap map[string]model.ExtractedPathObject
}

ExtractedPath is a struct that contains the paths, temporary paths to remove and extraction map path of the sources Path is the slice of paths to scan ExtractionMap is a map that correlates the temporary path to the given path RemoveTmp is the slice containing temporary paths to be removed

func GetKuberneterSources 

func GetKuberneterSources(ctx context.Context, source []string, destinationPath string) (ExtractedPath, error)

GetKuberneterSources uses Kubernetes API to download runtime resources After downloading, scan the files as normal local files

func GetSources 

func GetSources(ctx context.Context, source []string, downloadDir string) (ExtractedPath, error)

GetSources goes through the source slice, and determines the of source type (ex: zip, git, local). It than extracts the files to be scanned. If the source given is not local, a temp dir will be created where the files will be stored.

type FileSystemSourceProvider 

type FileSystemSourceProvider struct {
	// contains filtered or unexported fields
}

FileSystemSourceProvider provides a path to be scanned and a list of files which will not be scanned

func NewFileSystemSourceProvider 

func NewFileSystemSourceProvider(ctx context.Context, paths, excludes, onlyPaths []string) (*FileSystemSourceProvider, error)

NewFileSystemSourceProvider initializes a FileSystemSourceProvider with path and files that will be ignored

func (*FileSystemSourceProvider) AddUnfilteredPaths added in v1.9.0 

func (s *FileSystemSourceProvider) AddUnfilteredPaths(paths []string)

func (*FileSystemSourceProvider) BuildInventoryFromPrebuilt added in v1.7.2 

func (s *FileSystemSourceProvider) BuildInventoryFromPrebuilt(ctx context.Context,
	extensions model.Extensions,
	chartFn func(ctx context.Context, chartPath string) (skip bool)) ([]InventoryFile, error)

BuildInventoryFromPrebuilt renders Helm charts and filters pre-collected paths.

func (*FileSystemSourceProvider) ContentCache added in v1.7.2 

func (s *FileSystemSourceProvider) ContentCache() map[string][]byte

ContentCache returns bytes read during analyzer classification, keyed by path.

func (*FileSystemSourceProvider) GetBasePaths 

func (s *FileSystemSourceProvider) GetBasePaths() []string

GetBasePaths returns base path of FileSystemSourceProvider

func (*FileSystemSourceProvider) GetParallelSources added in v0.0.3 

func (s *FileSystemSourceProvider) GetParallelSources(ctx context.Context,
	extensions model.Extensions, sink Sink, resolverSink ResolverSink) error

GetParallelSources is an alternative to GetSources, parallelising the task

func (*FileSystemSourceProvider) GetSources 

func (s *FileSystemSourceProvider) GetSources(ctx context.Context,
	extensions model.Extensions, sink Sink, resolverSink ResolverSink) error

GetSources tries to open file or directory and execute sink function on it

func (*FileSystemSourceProvider) ReleaseContentCache added in v1.16.0 

func (s *FileSystemSourceProvider) ReleaseContentCache()

ReleaseContentCache drops the cached file bytes: only needed during the sink phase (parsed files keep their content in OriginalData), and the map is shared with the scan Client, so clearing here releases both references.

func (*FileSystemSourceProvider) SetPrebuiltWalk added in v1.7.2 

func (s *FileSystemSourceProvider) SetPrebuiltWalk(paths, chartRoots []string, contentCache map[string][]byte)

SetPrebuiltWalk reuses analyzer walk inventory to skip a second tree walk.

func (*FileSystemSourceProvider) TerraformFiles added in v1.9.0 

func (s *FileSystemSourceProvider) TerraformFiles(ctx context.Context) ([]string, error)

TerraformFiles returns Terraform/OpenTofu config paths used for module discovery before scanning: native .tf/.tofu files plus .tf.json/.tofu.json files.

func (*FileSystemSourceProvider) WalkInventory added in v1.7.2 

func (s *FileSystemSourceProvider) WalkInventory(ctx context.Context,
	extensions model.Extensions,
	chartFn func(ctx context.Context, chartPath string) (skip bool)) ([]InventoryFile, error)

WalkInventory collects matching files, calling chartFn at each Helm chart root.

type InventoryFile added in v1.7.2 

type InventoryFile struct {
	Path string
	Ext  string
}

InventoryFile is a discovered file and its matched extension token.

type MemorySourceProvider added in v1.7.0 

type MemorySourceProvider struct {
	// contains filtered or unexported fields
}

MemorySourceProvider serves a fixed set of files (pushed over HTTP) to the scan pipeline, reading their content through a vfs.FS. It replaces the disk-walking FileSystemSourceProvider in the server's content-push path, so there is no filesystem walk and no symlink/SameFile handling.

The server's services share one provider, so their scans go through the shared prepare (runner.PrepareMemorySources), which lists files and renders pushed Helm charts via WalkInventory. GetSources/GetParallelSources serve only services that do not share it; they ignore the ResolverSink and render no charts.

func NewMemorySourceProvider added in v1.7.0 

func NewMemorySourceProvider(fsys vfs.FS, paths, ignorePaths, onlyPaths []string) *MemorySourceProvider

NewMemorySourceProvider builds a provider over the given paths, reading each file's content from fsys (the same in-memory FS used for cross-file resolution, so content is stored once). ignorePaths/onlyPaths are the config's global path filters; they apply the same ignore-paths/only-paths semantics the disk source provider applies, so server-mode scans honor them too.

func (*MemorySourceProvider) GetBasePaths added in v1.7.0 

func (m *MemorySourceProvider) GetBasePaths() []string

GetBasePaths returns a synthetic root. Pushed paths are reported back exactly as they were pushed, whether relative or absolute, so there is no real base to name. The server's entry point (scan.Client.Scan) never consults this: it returns the raw vulnerabilities, skipping the summary step that relativizes file names against a base path.

func (*MemorySourceProvider) GetParallelSources added in v1.7.0 

func (m *MemorySourceProvider) GetParallelSources(ctx context.Context,
	extensions model.Extensions, sink Sink, _ ResolverSink) error

GetParallelSources fans the per-file sink (which parses the content into a document tree — the CPU-heavy step) across a bounded worker pool. There is no I/O to parallelize for in-memory content, but parsing is CPU-bound and a non-trivial share of a warm server scan, so this can speed up large pushes. It draws from the shared process-wide CPU budget (CPUBound) like the engine's other CPU-heavy pools, so concurrent scans don't oversubscribe the cores. The same sink is called concurrently by the disk provider, so it is safe for concurrent use.

func (*MemorySourceProvider) GetSources added in v1.7.0 

func (m *MemorySourceProvider) GetSources(ctx context.Context,
	extensions model.Extensions, sink Sink, _ ResolverSink) error

GetSources feeds each pushed file whose extension a parser supports into the sink, reading its content through the vfs.FS.

func (*MemorySourceProvider) ReadFile added in v1.17.0 

func (m *MemorySourceProvider) ReadFile(p string) ([]byte, error)

ReadFile reads a pushed file through the provider's FS.

func (*MemorySourceProvider) RecordMissing added in v1.17.0 

func (m *MemorySourceProvider) RecordMissing(path string)

RecordMissing records an escalation request when the provider's FS tracks missing paths (the in-memory one does; the real disk cannot).

func (*MemorySourceProvider) WalkInventory added in v1.17.0 

func (m *MemorySourceProvider) WalkInventory(ctx context.Context,
	extensions model.Extensions,
	chartFn func(ctx context.Context, chartPath string) (rendered bool)) ([]InventoryFile, error)

WalkInventory is the pushed-content counterpart of the disk provider's WalkInventory: it calls chartFn for each pushed chart root (shallow-first, subcharts skipped once their parent rendered) and returns the eligible pushed files minus the Helm files of the charts that rendered.

type ResolverSink 

type ResolverSink func(ctx context.Context, filename string) ([]string, error)

ResolverSink defines a sink function to be passed as reference to functions for resolved files/templates

type Sink 

type Sink func(ctx context.Context, filename string, content io.ReadCloser) error

Sink defines a sink function to be passed as reference to functions

type SourceProvider 

type SourceProvider interface {
	GetBasePaths() []string
	GetSources(ctx context.Context, extensions model.Extensions, sink Sink, resolverSink ResolverSink) error
	GetParallelSources(ctx context.Context, extensions model.Extensions, sink Sink, resolverSink ResolverSink) error
}

SourceProvider is the interface that wraps the basic GetSources method. GetBasePath returns base path of FileSystemSourceProvider GetSources receives context, receive ID, extensions supported and a sink function to save sources

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.