python

package
v0.3.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 19, 2026 License: Apache-2.0 Imports: 26 Imported by: 0

Documentation

Overview

Package python implements dependency extraction for Python codebases.

This file holds the naming rules. Python package names are compared after normalization (PEP 503): Flask, flask and flask are all one project, and an index, a lockfile and a wheel may each spell the name differently. Every name read from project metadata goes through NormalizeName before it is used as an identity anywhere.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func InstalledNodeList

func InstalledNodeList(dists []*InstalledDistribution, includeFiles bool) (*sbom.NodeList, error)

InstalledNodeList builds the graph an installed environment holds. When includeFiles is set, every file the RECORDs own becomes a node related to its package.

func NormalizeName

func NormalizeName(name string) string

NormalizeName returns the canonical form of a Python package name: every run of dashes, underscores and dots becomes one dash, and the result is lowercased. This is the form purls and lockfile lookups use.

Types

type Artifact

type Artifact struct {
	URL  string `toml:"url"`
	Hash string `toml:"hash"`
	Size int64  `toml:"size"`
}

Artifact is one distribution file: a wheel or an sdist.

func (*Artifact) HashValue

func (a *Artifact) HashValue() (algorithm, value string)

HashValue splits the artifact's hash into its algorithm and hex value. uv writes them as "sha256:abc...".

type Decomposer

type Decomposer struct{}

Decomposer reads dependency data from Python codebases managed by uv. The whole graph comes from uv.lock: versions, edges, markers and artifact hashes are all resolved in the lock, so extraction needs no network and no Python interpreter.

func New

func New() *Decomposer

func (*Decomposer) DefaultOptions

func (d *Decomposer) DefaultOptions() any

DefaultOptions returns the default options for the Python decomposer.

func (*Decomposer) Extract

func (d *Decomposer) Extract(opts *api.DecomposerOptions) (*sbom.NodeList, error)

Extract builds the dependency graph the target environment sees, from whichever lockfile the codebase has. A project migrating between tools may carry both; uv.lock wins, being the one uv keeps current.

func (*Decomposer) FindCodeBases

func (d *Decomposer) FindCodeBases(index *code.PathIndex) ([]string, error)

FindCodeBases locates Python codebases by their lockfiles.

func (*Decomposer) Requirements

func (d *Decomposer) Requirements(_ *api.DecomposerOptions) []api.Requirement

Requirements returns nothing: extraction is pure Go, offline.

type Dependency

type Dependency struct {
	Name string `toml:"name"`

	// Version and Source disambiguate the target when the resolution
	// forked: a lock holding numpy at three versions states which one this
	// edge points to. Both empty mean the name alone identifies it.
	Version string `toml:"version"`
	Source  Source `toml:"source"`

	// Extra lists the target's extras this dependency enables, as in
	// requests[socks].
	Extra []string `toml:"extra"`

	// Marker is the environment marker the edge is conditional on, empty
	// when it holds everywhere.
	Marker string `toml:"marker"`
}

Dependency is one edge of the resolved graph.

type DirectURL

type DirectURL struct {
	URL     string `json:"url"`
	VCSInfo struct {
		VCS               string `json:"vcs"`
		CommitID          string `json:"commit_id"`
		RequestedRevision string `json:"requested_revision"`
	} `json:"vcs_info"`
	DirInfo struct {
		Editable bool `json:"editable"`
	} `json:"dir_info"`
}

DirectURL is a parsed direct_url.json.

type Environment

type Environment struct {
	// The PEP 508 environment variables, named as the markers name them.
	OSName                       string // os_name: "posix", "nt"
	SysPlatform                  string // sys_platform: "linux", "darwin", "win32"
	PlatformSystem               string // platform_system: "Linux", "Darwin", "Windows"
	PlatformMachine              string // platform_machine: "x86_64", "aarch64", "arm64", "AMD64"
	PythonVersion                string // python_version: "3.12"
	PythonFullVersion            string // python_full_version: "3.12.0"
	ImplementationName           string // implementation_name: "cpython"
	PlatformPythonImplementation string // platform_python_implementation: "CPython"

	// Extras are the enabled extras, normalized. The "extra" marker
	// variable evaluates against this set.
	Extras []string
}

Environment is the concrete target environment markers evaluate against: one operating system, one architecture, one Python version, one set of enabled extras. A lockfile resolves for every environment at once; an extraction is for one of these.

func NewEnvironment

func NewEnvironment(goos, goarch, pythonVersion string) (*Environment, error)

NewEnvironment builds the environment for an operating system and architecture in Go's vocabulary (GOOS and GOARCH names), and a Python version stated as "3.12" or "3.12.4". Empty os or arch mean the platform unpack itself runs on.

The translation is the reason this constructor exists: Python spells the same platform several ways, and the values must agree with what real interpreters report — linux/arm64 is aarch64, but a Mac's arm64 stays arm64 and Windows spells it ARM64.

func (*Environment) Evaluate

func (env *Environment) Evaluate(marker string) (bool, error)

Evaluate parses and evaluates a marker in this environment. An empty marker holds everywhere: a dependency with no marker is unconditional.

type InstalledDistribution

type InstalledDistribution struct {
	// Name is normalized; the dist-info directory spells it however the
	// wheel did (PySocks-1.7.1.dist-info).
	Name    string
	Version string

	Summary        string
	RequiresPython string
	HomePage       string
	ProjectURLs    map[string]string

	// The three licence tiers, as METADATA states them; LicensesFromMetadata
	// of the node builder triages them.
	LicenseExpression string
	License           string
	Classifiers       []string

	// RequiresDist are the declared dependencies, raw PEP 508.
	RequiresDist []string

	// Installer names the tool that installed the package (the INSTALLER
	// file), and Requested says the install was asked for rather than
	// pulled in (the REQUESTED marker, PEP 376).
	Installer string
	Requested bool

	// DirectURL is the provenance of a package installed from a URL or a
	// repository rather than an index (direct_url.json, PEP 610).
	DirectURL *DirectURL

	// Files is the RECORD: every file the installation owns.
	Files []InstalledFile

	// Path is the dist-info directory, relative to the FS the environment
	// was read from.
	Path string
}

InstalledDistribution is one installed package, read from its dist-info.

func FindDistributions

func FindDistributions(fsys fs.FS) ([]*InstalledDistribution, error)

FindDistributions walks a filesystem and reads every installed distribution on it, wherever its site-packages lives: system interpreters, virtualenvs and --target directories all qualify. A distribution whose metadata cannot be read is skipped: one broken package should not hide an environment.

func ReadDistribution

func ReadDistribution(fsys fs.FS, dir string) (*InstalledDistribution, error)

ReadDistribution reads one dist-info directory.

type InstalledFile

type InstalledFile struct {
	// Path is relative to the site-packages directory (entries may climb
	// out of it: ../../../bin/pip).
	Path string

	// Algorithm and Digest are the recorded hash, the digest in the
	// urlsafe base64 RECORD states it in. Empty for the files an
	// installer rewrites, RECORD itself among them.
	Algorithm string
	Digest    string

	Size int64
}

InstalledFile is one RECORD entry.

type Lockfile

type Lockfile struct {
	// Version is the lock schema version, and Revision the revision within
	// it. Additions bump the revision; only breaking changes bump the
	// version, which is the one compatibility is judged by.
	Version  int `toml:"version"`
	Revision int `toml:"revision"`

	// RequiresPython is the project's Python requirement, as a PEP 440
	// specifier set such as ">=3.10".
	RequiresPython string `toml:"requires-python"`

	// ResolutionMarkers split the environments the resolution forked over;
	// SupportedMarkers restrict the environments the project supports at
	// all (tool.uv.environments). Both absent mean one universal
	// resolution.
	ResolutionMarkers []string `toml:"resolution-markers"`
	SupportedMarkers  []string `toml:"supported-markers"`

	Packages []Package `toml:"package"`
}

Lockfile is a parsed uv.lock.

func ParseLockfile

func ParseLockfile(data []byte) (*Lockfile, error)

ParseLockfile reads a uv.lock document.

func ReadLockfile

func ReadLockfile(path string) (*Lockfile, error)

ReadLockfile reads a uv.lock from a file.

type Marker

type Marker struct {
	// contains filtered or unexported fields
}

Marker is a parsed environment marker, ready to evaluate.

func ParseMarker

func ParseMarker(s string) (*Marker, error)

ParseMarker parses an environment marker expression such as "python_full_version < '3.11' and sys_platform == 'linux'".

func (*Marker) Eval

func (m *Marker) Eval(env *Environment) (bool, error)

Eval says whether the marker holds in the given environment.

type Options

type Options struct {
	// Platform targets an operating system and architecture, in GOOS/GOARCH
	// vocabulary ("linux/arm64"), an os alone ("windows"), or empty for the
	// platform unpack runs on. A lockfile resolves for every platform at
	// once; the extraction reads it for this one.
	Platform string

	// PythonVersion targets an interpreter version, "3.12" or fuller.
	// Empty picks the newest version the lockfile's own resolution forks
	// mention, so the default describes a current install with no Python
	// on the machine unpack runs on.
	PythonVersion string

	// Concurrency controls the parallel requests to PyPI when enriching
	// (default: 10).
	Concurrency int
}

Options configures the Python dependency extraction.

type Package

type Package struct {
	Name    string `toml:"name"`
	Version string `toml:"version"`
	Source  Source `toml:"source"`

	// Dependencies are the package's resolved runtime dependencies.
	// OptionalDependencies holds those of each extra, keyed by extra name,
	// and DevDependencies those of each dependency group (PEP 735), keyed
	// by group name.
	Dependencies         []Dependency            `toml:"dependencies"`
	OptionalDependencies map[string][]Dependency `toml:"optional-dependencies"`
	DevDependencies      map[string][]Dependency `toml:"dev-dependencies"`

	// ResolutionMarkers claim this entry for part of the environment space
	// when the resolution forked: the numpy for one Python version range
	// carries the markers telling it apart from the numpy for another.
	ResolutionMarkers []string `toml:"resolution-markers"`

	// Sdist and Wheels are the package's distribution artifacts. Hashes
	// live here: PyPI has no package-level hash, only per-artifact ones.
	Sdist  *Artifact  `toml:"sdist"`
	Wheels []Artifact `toml:"wheels"`
}

Package is one locked package. A package appears once per resolved version: the same name shows up several times when the resolution forked over environments, each entry claimed by its resolution markers.

type PoetryDependency

type PoetryDependency struct {
	Name   string
	Marker string
	Extras []string
}

PoetryDependency is one edge of the resolved graph, normalized.

type PoetryFile

type PoetryFile struct {
	File string `toml:"file"`
	Hash string `toml:"hash"`
}

PoetryFile is one distribution artifact.

func (*PoetryFile) HashValue

func (f *PoetryFile) HashValue() (algorithm, value string)

HashValue splits the file's hash into its algorithm and hex value, written as "sha256:abc...".

type PoetryLockfile

type PoetryLockfile struct {
	Packages []PoetryPackage `toml:"package"`
	Metadata PoetryMetadata  `toml:"metadata"`
}

PoetryLockfile is a parsed poetry.lock.

func ParsePoetryLockfile

func ParsePoetryLockfile(data []byte) (*PoetryLockfile, error)

ParsePoetryLockfile reads a poetry.lock document.

func ReadPoetryLockfile

func ReadPoetryLockfile(path string) (*PoetryLockfile, error)

ReadPoetryLockfile reads a poetry.lock from a file.

type PoetryMetadata

type PoetryMetadata struct {
	LockVersion    string `toml:"lock-version"`
	PythonVersions string `toml:"python-versions"`
	ContentHash    string `toml:"content-hash"`
}

PoetryMetadata is the lock's trailer.

type PoetryPackage

type PoetryPackage struct {
	Name           string `toml:"name"`
	Version        string `toml:"version"`
	Description    string `toml:"description"`
	Optional       bool   `toml:"optional"`
	PythonVersions string `toml:"python-versions"`

	// Groups lists the dependency groups the package serves (lock 2.1).
	// Empty in a 2.0 lock, where membership is derived from the manifest.
	Groups []string `toml:"groups"`

	// markers says where the package applies at all. A package serving
	// several groups may apply to each under a different condition, so the
	// lock writes either one marker or a group-to-marker table; the parser
	// normalizes both into the table, the plain form under every group.
	RawMarkers any               `toml:"markers"`
	Markers    map[string]string `toml:"-"`

	// RawDependencies is the resolved dependency table: target name to a
	// constraint, which may be a bare version string, an object carrying a
	// marker and extras, or a list of such objects when the constraint
	// differs by environment. Dependencies is its normalized form.
	RawDependencies map[string]any     `toml:"dependencies"`
	Dependencies    []PoetryDependency `toml:"-"`

	// Extras names the package's extras and their requirements. The values
	// are requirement strings kept only for completeness: the packages an
	// enabled extra pulls in are resolved through Dependencies.
	Extras map[string][]string `toml:"extras"`

	// Files are the distribution artifacts with their hashes, named by
	// filename rather than URL.
	Files []PoetryFile `toml:"files"`

	// Source is set when the package does not come from an index.
	Source *PoetrySource `toml:"source"`
}

PoetryPackage is one locked package.

type PoetrySource

type PoetrySource struct {
	Type string `toml:"type"` // "git", "url", "directory", "legacy"
	URL  string `toml:"url"`

	// Reference is what was asked for (a branch, tag or rev) and
	// ResolvedReference the commit it resolved to.
	Reference         string `toml:"reference"`
	ResolvedReference string `toml:"resolved_reference"`
}

PoetrySource says where a non-index package comes from.

type PyPIClient

type PyPIClient struct {
	Agent   *khttp.Agent
	BaseURL string
}

PyPIClient fetches package metadata from the PyPI JSON API.

func NewPyPIClient

func NewPyPIClient(concurrency int) *PyPIClient

NewPyPIClient creates a client for the PyPI JSON API.

func (*PyPIClient) FetchAll

func (c *PyPIClient) FetchAll(packages []packageKey) map[packageKey]*pypiInfo

FetchAll fetches metadata for the packages in parallel. Failures are skipped: enrichment adds what it can and never breaks an extraction.

type PyProject

type PyProject struct {
	Project struct {
		Name                 string              `toml:"name"`
		Version              string              `toml:"version"`
		RequiresPython       string              `toml:"requires-python"`
		Dependencies         []string            `toml:"dependencies"`
		OptionalDependencies map[string][]string `toml:"optional-dependencies"`
	} `toml:"project"`

	// DependencyGroups is the standard groups table (PEP 735). Entries are
	// requirement strings, or include tables this reader skips.
	DependencyGroups map[string][]any `toml:"dependency-groups"`

	Tool struct {
		Poetry struct {
			Name         string              `toml:"name"`
			Version      string              `toml:"version"`
			Dependencies map[string]any      `toml:"dependencies"`
			Extras       map[string][]string `toml:"extras"`
			Group        map[string]struct {
				Dependencies map[string]any `toml:"dependencies"`
			} `toml:"group"`
			// DevDependencies is the oldest spelling of the dev group.
			DevDependencies map[string]any `toml:"dev-dependencies"`
		} `toml:"poetry"`
	} `toml:"tool"`
}

PyProject is a parsed pyproject.toml, reduced to what the graph needs.

func ReadPyProject

func ReadPyProject(path string) (*PyProject, error)

ReadPyProject reads a pyproject.toml file.

func (*PyProject) ExtraDependencies

func (p *PyProject) ExtraDependencies() (map[string][]*requirement, error)

ExtraDependencies returns the direct dependencies of every extra.

func (*PyProject) GroupDependencies

func (p *PyProject) GroupDependencies() (map[string][]*requirement, error)

GroupDependencies returns the direct dependencies of every dependency group, whichever of the three spellings declares them.

func (*PyProject) MainDependencies

func (p *PyProject) MainDependencies() ([]*requirement, error)

MainDependencies returns the project's direct runtime dependencies.

In the legacy table, python is a pseudo-dependency and an entry marked optional belongs to the extras rather than to the runtime set.

func (*PyProject) RootName

func (p *PyProject) RootName() string

RootName returns the project's name, normalized, from whichever table states it.

func (*PyProject) RootVersion

func (p *PyProject) RootVersion() string

RootVersion returns the project's version from whichever table states it.

type Source

type Source struct {
	// Registry is the index URL the package was resolved from.
	Registry string `toml:"registry"`

	// Virtual and Editable mark the project's own packages: Virtual a
	// package that is not installed (the usual project root), Editable one
	// installed in place (workspace members). Both hold the directory,
	// relative to the lock.
	Virtual  string `toml:"virtual"`
	Editable string `toml:"editable"`

	// Git, Path, Directory and URL are direct dependencies on a repository,
	// a local archive, a local directory, and a remote archive.
	Git       string `toml:"git"`
	Path      string `toml:"path"`
	Directory string `toml:"directory"`
	URL       string `toml:"url"`
}

Source says where a package comes from. Exactly one field is set.

func (*Source) IsProject

func (s *Source) IsProject() bool

IsProject says whether the package is part of the project itself rather than something fetched: the roots of the dependency graph.

type Version

type Version struct {
	Epoch   int
	Release []int

	// Pre is the pre-release phase, in order: "a" < "b" < "rc". The long
	// spellings (alpha, beta, c, pre, preview) normalize into those three.
	Pre    int
	PreN   int
	HasPre bool

	Post    int
	HasPost bool

	Dev    int
	HasDev bool

	Local string
}

Version is a parsed PEP 440 version.

func ParseVersion

func ParseVersion(s string) (*Version, error)

ParseVersion parses a PEP 440 version string.

func (*Version) Compare

func (v *Version) Compare(o *Version) int

Compare orders two versions per PEP 440: -1, 0 or 1 as v sorts before, with or after o. Trailing zeros in the release are insignificant (1.0 == 1.0.0), a dev release sorts before pre-releases, pre-releases before the final, the final before post-releases, and a local version after the same version without one.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL