langdb

package
v0.8.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 7, 2026 License: MIT Imports: 14 Imported by: 0

Documentation

Overview

Package langdb reads the installed-package layouts of the language ecosystems that ship inside container images: Python's site-packages, Node's node_modules, and Java's jar, war and ear archives.

It is the language-ecosystem counterpart of internal/pkgdb, and answers the same two questions: which distributions are installed at which versions, and which files each one owns. It answers a third that pkgdb does not need -- which names the code imports itself by -- because a Python distribution's project name and its import name are routinely different (PyYAML installs "yaml"), and the import name is what an import graph can be rooted at.

It differs from pkgdb in one structural way. A dpkg or rpm database is a single file at a known path, so pkgdb's Reader can Detect by stat'ing it. site-packages, node_modules and jars can be anywhere and there can be many of them, so finding them means walking the tree. Scan therefore does one walk for every format at once, rather than giving each Reader its own Detect.

Nothing here talks to OSV or to the network. Readers parse a filesystem and return what they found, or an error -- never a partial answer.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func FindRoots

func FindRoots(fsys target.RootFS) (map[Format][]string, error)

FindRoots walks the tree once and returns, per format, what holds that format's installed packages: a directory for most, an archive file for a FileReader.

A matched directory is not descended into. Nesting is real -- npm stores a conflicting version in a node_modules inside a package -- but it is the npm reader that understands what the nesting means, and having the generic walk report inner directories as top-level roots would flatten exactly the structure the resolver needs.

func MavenPURL

func MavenPURL(name, version string) string

MavenPURL renders an artifact as a package URL. The groupId is the purl namespace, so the OSV name's colon becomes a slash.

func NormalizePyPI

func NormalizePyPI(name string) string

NormalizePyPI applies PEP 503 name normalization, which is how OSV, PyPI and every purl consumer key a Python project: lowercase, with runs of "-", "_" and "." collapsed to a single "-". "PyYAML" and "ruamel.yaml.clib" become "pyyaml" and "ruamel-yaml-clib".

Types

type FileReader

type FileReader interface {
	Reader
	// FileSuffixes are the file extensions that identify this format's
	// packages, lowercase and including the dot.
	FileSuffixes() []string
}

FileReader is a Reader whose installed packages are individual files rather than directories.

Java is the reason it exists. A jar is a single self-describing file that can sit anywhere -- a servlet container's lib directory, /usr/share/java, an application's working directory -- so there is no directory base name to key on the way site-packages and node_modules are keyed on. The roots handed to Read are then the archive paths themselves.

type Format

type Format string

Format identifies a language ecosystem's installed-package layout.

const (
	FormatPyPI  Format = "pypi"
	FormatNPM   Format = "npm"
	FormatMaven Format = "maven"
)

type Maven

type Maven struct{}

Maven reads Java artifacts out of the jar, war and ear files in an image.

A Java archive is a zip, which makes this the only ecosystem here whose presence test can go below the package: listing the central directory names every compiled class without executing anything, so "this artifact does not contain the vulnerable class" is a checkable claim rather than an inference. That case is not hypothetical -- the canonical Log4Shell mitigation was deleting JndiLookup.class from a jar whose version stayed 2.14.1, which every version-matching scanner still reports as vulnerable.

What the archive does not reliably carry is its own coordinates. Maven writes META-INF/maven/<groupId>/<artifactId>/pom.properties, and OSV keys Java on exactly that pair; Gradle writes nothing of the sort, so spring-core-6.1.14.jar has no groupId anywhere in it. readCoords therefore descends through four tiers, and records which one answered: a coordinate this package guessed may never support a claim that something is absent from it.

func (*Maven) DirNames

func (*Maven) DirNames() []string

DirNames implements Reader. Java archives are files that live anywhere -- a servlet container's lib directory, /usr/share/java, an app's working directory -- so there is no directory to key on and FileSuffixes does the work instead.

func (*Maven) FileSuffixes

func (*Maven) FileSuffixes() []string

FileSuffixes implements FileReader.

func (*Maven) Format

func (*Maven) Format() Format

Format implements Reader.

func (*Maven) Read

func (r *Maven) Read(fsys target.RootFS, roots []string) (Result, error)

Read implements Reader. Its roots are archive paths rather than directories.

type NPM

type NPM struct{}

NPM reads Node packages out of node_modules.

There is no installed-file manifest to read and none is needed: a Node package owns its directory outright, minus any nested node_modules, which belong to the packages inside them. That nesting is not an accident of layout -- it is how npm installs two versions of one package in the same tree -- so each nesting level is reported as a separate installed instance, with its own Dir. The resolver needs that, because which copy of a package a file sees depends on where the file is.

func (*NPM) DirNames

func (*NPM) DirNames() []string

DirNames implements Reader.

func (*NPM) Format

func (*NPM) Format() Format

Format implements Reader.

func (*NPM) Read

func (r *NPM) Read(fsys target.RootFS, roots []string) (Result, error)

Read implements Reader.

type Package

type Package struct {
	Format Format `json:"format"`

	// Name is the package name *as OSV keys it*: the PEP 503 normalized
	// project name for PyPI, the manifest name verbatim (scope included) for
	// npm.
	Name string `json:"name"`

	// AltNames are other names worth querying OSV under -- for PyPI, the
	// un-normalized project name as the metadata spells it. Same reasoning as
	// pkgdb.Package.OSVNames: a name that matches nothing costs one entry in a
	// batch query, while missing the right one reports a vulnerable package as
	// clean.
	AltNames []string `json:"alt_names,omitempty"`

	Version string `json:"version,omitempty"`

	// ImportNames are the top-level names the code is imported by: "yaml" and
	// "_yaml" for PyYAML, and just the package name for npm.
	ImportNames []string `json:"import_names,omitempty"`

	// ImportNamesKnown reports whether ImportNames came from the
	// distribution's own metadata rather than being guessed from its project
	// name.
	//
	// It exists because the two failure directions are not symmetric. A guessed
	// import name that is wrong makes the distribution unreachable in the
	// import graph, and an unreachable distribution reads as
	// vulnerable_code_not_in_execute_path -- a false clean. A consumer must be
	// able to refuse that conclusion, so the provenance travels with the data.
	ImportNamesKnown bool `json:"import_names_known"`

	// Files are the tree-absolute paths the distribution installs.
	Files []string `json:"files,omitempty"`

	// FilesKnown reports whether Files came from the distribution's own
	// manifest (RECORD, installed-files.txt) rather than being reconstructed
	// by walking its directories.
	//
	// Same asymmetry as ImportNamesKnown: an empty file list means "this
	// distribution ships no code", which is a not_present conclusion. Only a
	// manifest can support it.
	FilesKnown bool `json:"files_known"`

	// Dir is the tree-absolute directory the distribution's metadata lives in:
	// the .dist-info/.egg-info directory, or the package directory under
	// node_modules. The import resolver needs it to know which nested
	// node_modules an instance sees.
	Dir string `json:"dir,omitempty"`

	// DB is the site-packages or node_modules directory this came from, for
	// evidence and error messages.
	DB string `json:"db,omitempty"`

	// Requires are the packages this one declares it may load, named the way
	// Name is. It bounds what a computed import inside this package could
	// reach, which is the only thing that makes a dynamic-import taint able to
	// block anything narrower than the whole image.
	//
	// Only the PyPI reader fills this in. npm's resolver reads dependencies
	// from package.json itself, because for npm a name is not enough: which
	// copy of "tar" a file sees depends on the directory it is required from,
	// and that is precisely what nested node_modules means. Python's sys.path
	// is a single global search order, so a name resolves the same everywhere
	// and can be indexed once.
	Requires []Requirement `json:"requires,omitempty"`

	// RequiresKnown reports whether Requires came from readable metadata.
	//
	// Same asymmetry as FilesKnown. An empty Requires means "this package
	// depends on nothing", which narrows a taint's scope to the package
	// itself; only metadata that was actually read can support that.
	RequiresKnown bool `json:"requires_known"`

	// CoordsKnown reports whether Name came from metadata the build wrote
	// rather than from the file name or the code's own layout.
	//
	// Only the Maven reader ever sets it false. A PyPI or npm name is read from
	// a manifest that the package manager requires; a jar frequently carries no
	// statement of its own groupId at all, and the name is then reconstructed
	// from the file name and the classes' package prefixes.
	//
	// Same asymmetry as the rest of this family, one level up. A guessed
	// coordinate is still worth querying OSV with -- a name that matches nothing
	// costs one entry in a batch. It may not support a negative conclusion:
	// asserting that an artifact does not contain some class, when the artifact
	// is only what this reader thinks the jar is, stacks a guess on a guess.
	CoordsKnown bool `json:"coords_known,omitempty"`
}

Package is one installed distribution.

func (Package) OSVNames

func (p Package) OSVNames() []string

OSVNames returns the names to query OSV with, likeliest first.

type PyPI

type PyPI struct{}

PyPI reads Python distributions out of site-packages and dist-packages.

The metadata it reads is standardized: PEP 376 gives every installed distribution a .dist-info directory holding METADATA (the project name and version) and RECORD (every file the installation wrote). RECORD is the exact analog of dpkg's /var/lib/dpkg/info/<name>.list, and is what connects a CVE against "PyYAML" to the .py files that would have to be imported for it to matter.

Not every installed distribution has all of it. Distributions installed by a distro package manager frequently ship an older .egg-info instead, and even a modern .dist-info may be missing RECORD -- pip's own dist-info as Homebrew installs it has METADATA but no RECORD. Every fallback below exists because a real installation was missing the thing above it.

func (*PyPI) DirNames

func (*PyPI) DirNames() []string

DirNames implements Reader. "dist-packages" is Debian's rename of site-packages for distro-installed modules; both appear in the same image.

func (*PyPI) Format

func (*PyPI) Format() Format

Format implements Reader.

func (*PyPI) Read

func (r *PyPI) Read(fsys target.RootFS, roots []string) (Result, error)

Read implements Reader.

type Reader

type Reader interface {
	// Format names the ecosystem this reader understands.
	Format() Format
	// DirNames are the directory base names that hold this format's installed
	// packages, so one tree walk can find the roots for every reader.
	DirNames() []string
	// Read parses every root. A root that exists and cannot be listed is an
	// error; an individual manifest that will not parse is reported through
	// Result.Unreadable.
	Read(fsys target.RootFS, roots []string) (Result, error)
}

Reader parses one language's installed-package layout out of the roots that Scan found for it.

func Readers

func Readers() []Reader

Readers returns every backend, in a stable order.

type Requirement

type Requirement struct {
	Name string `json:"name"`

	// Conditional reports whether the declaration carries an environment
	// marker -- an extra, a Python version bound, a platform test.
	//
	// It exists to separate two very different kinds of absence. An
	// unconditional dependency that is not installed means the environment is
	// not what the metadata describes, and nothing about it can be trusted to
	// bound anything. A conditional one that is not installed is the marker
	// working as designed: an unselected extra is *expected* to be missing,
	// and treating that as an unknown would push nearly every Python image
	// into a global taint.
	Conditional bool `json:"conditional,omitempty"`
}

Requirement is one declared dependency.

type Result

type Result struct {
	Format Format `json:"format"`

	// Roots are what the tree walk found for this format: the site-packages or
	// node_modules directories, or, for a format whose packages are files, the
	// archive paths themselves.
	Roots []string `json:"roots"`

	Packages []Package `json:"packages"`

	// Unreadable are manifests that were found and could not be parsed.
	//
	// Unlike pkgdb, this is not a fatal error. A dpkg status file is one file
	// describing every package, so failing to parse it means knowing nothing;
	// here each distribution carries its own manifest, and a node_modules tree
	// routinely contains deliberately malformed fixtures. It is never silent
	// either: a distribution whose manifest could not be read is one whose
	// absence must not be asserted, so the plugin turns this into a taint.
	Unreadable []string `json:"unreadable,omitempty"`

	// Unidentified are archives that opened cleanly and declare no coordinates
	// anywhere -- no META-INF/maven, no usable manifest, no conventional file
	// name.
	//
	// It is Unreadable's sibling for the Maven reader, and it exists because the
	// two are the same failure wearing different clothes: something is installed
	// here and this scan cannot say what. A plugin asked whether an artifact is
	// present must not answer no while one of these is on the disk.
	Unidentified []string `json:"unidentified,omitempty"`
}

Result is what one reader found.

func Scan

func Scan(fsys target.RootFS) ([]Result, error)

Scan finds every root and runs the reader that owns it.

A format with no roots produces no Result at all, which is how a plugin learns it does not apply to this image.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL