normalize

package
v1.14.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 17, 2026 License: AGPL-3.0 Imports: 24 Imported by: 0

Documentation

Overview

Package normalize provides utilities for normalizing package identifiers. This file handles mapping binary package names to their source package names for Linux distributions (Debian, Alpine), enabling vulnerability matching against security advisories that reference source packages.

Index

Constants

View Source 
const GraphRootNodeID = "ROOT"

GraphRootNodeID marks the artifact itself at the head of a dependency path, as opposed to one of its components.

View Source 
const MerkleRootID = "ROOT"

MerkleRootID is the identity every SBOM root is hashed under. It is a sentinel rather than the artifact name on purpose: two artifacts with an identical dependency set must reach the same root hash, or the whole tree is stored twice. The artifact name lives in the sboms row instead.

Variables

View Source 
var PURLEcosystems = map[string]string{
	"Alpine":    "apk",
	"crates.io": "cargo",
	"Debian":    "deb",
	"Go":        "golang",
	"Hackage":   "hackage",
	"Hex":       "hex",
	"Maven":     "maven",
	"npm":       "npm",
	"NuGet":     "nuget",
	"OSS-Fuzz":  "generic",
	"Packagist": "composer",
	"Pub":       "pub",
	"PyPI":      "pypi",
	"RubyGems":  "gem",
}

PURL conversion utilities

View Source 
var ValidSemverRegex = regexp.MustCompile(`^(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$`)

Regex for validating a correct semver.

Functions

func ArtifactName added in v1.13.3 

func ArtifactName(name string) string

ArtifactName canonicalizes an artifact name by url-decoding it exactly once. Artifact names are looked up via a url-decoded path parameter (see shared.GetArtifactName), so a name that is stored without going through the same decoding step can end up indistinguishable, once placed in a URL, from a different, already-existing artifact whose name happens to decode to the same value. Applying this at every write site guarantees there is exactly one canonical stored form per artifact.

If the name is not validly percent-encoded, it is returned unchanged.

func ArtifactPurl 

func ArtifactPurl(scanner string, assetName string) string

func AssetName added in v1.9.0 

func AssetName(assetName string) (string, error)

func AssetSlugPath added in v1.9.0 

func AssetSlugPath(assetName string) (string, error)

AssetSlugPath expands a normalized "<organization>/<project>/<asset>" name into the web UI path "<organization>/projects/<project>/assets/<asset>".

func BomIsSBOM 

func BomIsSBOM(bom *cdx.BOM) bool

BomIsSBOM reports whether a document is an SBOM rather than a VEX report.

func CheckVersionInRange added in v1.13.3 

func CheckVersionInRange(exactVersion, introduced, fixed *string, lookingForVersion, affectedComponentType string) (bool, error)

func ConvertToSemver 

func ConvertToSemver(originalVersion string) (string, error)

ConvertToSemver converts various version formats to semantic versioning format. It handles: - Epoch prefixes (e.g., "2:1.2.3" -> "1.2.3") - "v" prefixes (e.g., "v1.2.3" -> "1.2.3") - Pre-release identifiers with "-" (e.g., "1.2.3-rc1") - Build metadata with "+" (e.g., "1.2.3+build1") - Tilde versions "~" (e.g., "1.2.3~rc1" -> "1.2.3-rc1") - Missing version segments (e.g., "1.2" -> "1.2.0")

Returns an error if: - Version contains invalid characters (only 0-9 and . allowed in version part) - Version has more than 3 numeric segments

func EncodeCanonical 

func EncodeCanonical(obj any) (out []byte, err error)

EncodeCanonical JSON canonicalizes the passed object and returns it as a byte slice. It uses the OLPC canonical JSON specification (see http://wiki.laptop.org/go/Canonical_JSON). If canonicalization fails the byte slice is nil and the second return value contains the error.

func FixFixedVersion 

func FixFixedVersion(purl string, fixedVersion *string) *string

func HashSubtree added in v1.14.0 

func HashSubtree(componentID string, childSubtreeHashes []uuid.UUID) uuid.UUID

HashSubtree computes the hash covering componentID and its children. Children are sorted first: the same subtree ingested in any order must hash the same, or deduplication silently stops working.

The hash is the leading 128 bits of the sha256, matching utils.HashToUUID. That is identity here - a collision would silently serve one subtree in place of another - so it is deliberately not truncated further.

func PURLToString 

func PURLToString(purl packageurl.PackageURL) (string, error)

func Purlify 

func Purlify(artifactName string, assetVersionName string) string

func SanitizeExternalReferencesURL 

func SanitizeExternalReferencesURL(url string) string

SanitizeExternalReferencesURL reverts the escaping cosign applies when attesting, which turns each "&" into the six literal characters &.

func SemverCompare 

func SemverCompare(v1, v2 string) int

func SemverSort 

func SemverSort(versions []string)

func SortStringsSlice 

func SortStringsSlice(slice []string) []string

func ToPurlWithoutVersion 

func ToPurlWithoutVersion(purl packageurl.PackageURL) string

func UppercaseCVEID 

func UppercaseCVEID(cveID string) string

Types

type Adjacency added in v1.14.0 

type Adjacency struct {
	// Children maps a ref to the refs it depends on directly.
	Children map[string][]string
	// ComponentIDs maps a ref to its component identity (a purl). A ref missing
	// here contributes itself, as happens for documents that identify
	// components by something other than a purl.
	ComponentIDs map[string]string
}

Adjacency is the minimal input needed to hash a document bottom-up. Refs are whatever handle the document uses to cross-reference components (a CycloneDX bom-ref, say); they are a parsing detail and never reach the database.

Deliberately not a graph type: parsing into this and hashing it is the whole ingest path, so there is no second in-memory SBOM representation to keep in step with the stored one.

type BOMMetadata 

type BOMMetadata struct {
	AssetVersionSlug      string
	AssetSlug             string
	OrgSlug               string
	ProjectSlug           string
	FrontendURL           string
	ArtifactName          string
	AssetID               uuid.UUID
	AddExternalReferences bool
	RootName              string // defaults to ArtifactName if empty
	AssetVersionName      string
}

BOMMetadata is everything needed to render an SBOM as a CycloneDX document that points back at this instance.

type MerkleEdge added in v1.14.0 

type MerkleEdge struct {
	SubtreeHash                 uuid.UUID
	DirectDependencySubtreeHash uuid.UUID
}

MerkleEdge is one persisted edge - a pure pivot between two node hashes. A component with n children yields n edges sharing a SubtreeHash; a leaf yields none, since its component id is carried by its node row instead.

type MerkleForest added in v1.14.0 

type MerkleForest []*MerkleTree

MerkleForest is a set of SBOMs scanned together - in practice every origin of one artifact, or every SBOM of an asset version.

The trees stay separate rather than being merged into one graph: two origins may legitimately disagree about a shared component, and both answers matter when reporting where a vulnerability comes from.

func (MerkleForest) ComponentIDs added in v1.14.0 

func (f MerkleForest) ComponentIDs() []string

ComponentIDs returns every distinct component across the forest, sorted.

func (MerkleForest) ComponentsInMultipleSBOMs added in v1.14.0 

func (f MerkleForest) ComponentsInMultipleSBOMs() []string

ComponentsInMultipleSBOMs returns components that more than one SBOM in the forest reports. Such a component cannot be marked fixed off a single scan: one source dropping it says nothing about the others.

func (MerkleForest) PathsToPURL added in v1.14.0 

func (f MerkleForest) PathsToPURL(purl string, limit int) []Path

PathsToPURL returns the dependency paths to purl across every SBOM, deduped. A limit of 0 means unlimited; otherwise it caps the total, keeping the most direct paths first.

type MerkleNode added in v1.14.0 

type MerkleNode struct {
	SubtreeHash uuid.UUID
	ComponentID string
	// sorted, so iteration is deterministic however the tree was built
	Children []uuid.UUID
}

MerkleNode is one component with the exact child set its hash covers.

func (*MerkleNode) IsLeaf added in v1.14.0 

func (n *MerkleNode) IsLeaf() bool

IsLeaf reports whether this component has no dependencies in this SBOM. It may still have children in another SBOM, under a different subtree hash.

type MerkleTree added in v1.14.0 

type MerkleTree struct {
	// Root identifies the SBOM by content: identical SBOMs share this hash.
	Root uuid.UUID
	// contains filtered or unexported fields
}

MerkleTree is one SBOM, keyed by subtree hash rather than by component id.

hash(component_id, sorted child hashes) covers a component's entire child set, so SBOMs that disagree about a shared component get different hashes and both descriptions survive.

One tree is one row of the sboms table. The artifact name and origin live in that row, so every ComponentID here is a real component.

func BuildMerkleTree added in v1.14.0 

func BuildMerkleTree(adj Adjacency, rootRef string) *MerkleTree

BuildMerkleTree converts the document below rootRef into a content-addressed tree.

rootComponentID is the identity the root is hashed under. A document's root is a parse artifact, not a real component, so passing the artifact's purl (or its name, when it has none) keeps every stored component id a real one.

func MerkleTreeFromNodesAndEdges added in v1.14.0 

func MerkleTreeFromNodesAndEdges(nodes []MerkleNode, edges []MerkleEdge, root uuid.UUID) (*MerkleTree, error)

MerkleTreeFromNodesAndEdges rebuilds a tree from persisted rows. root selects which SBOM to materialize, since the rows handed in may cover several.

Nodes carry the component ids and edges only the shape, so a leaf is a node with no outgoing edge rather than an edge with a nil child. Children on the nodes handed in are ignored - the edges are the authority on shape.

func (*MerkleTree) ComponentIDs added in v1.14.0 

func (t *MerkleTree) ComponentIDs() []string

ComponentIDs returns every distinct component in the SBOM, sorted, excluding the root - the root is the artifact itself, not one of its dependencies.

func (*MerkleTree) DirectDependencies added in v1.14.0 

func (t *MerkleTree) DirectDependencies() []string

DirectDependencies returns the component ids the SBOM depends on directly.

func (*MerkleTree) Edges added in v1.14.0 

func (t *MerkleTree) Edges() []MerkleEdge

Edges renders the tree's shape as rows to persist. Leaves contribute nothing: they have no outgoing edge, and Nodes carries their component id.

Insert them with ON CONFLICT DO NOTHING: rows for subtrees the instance has already stored are no-ops, which is where the storage saving comes from.

func (*MerkleTree) Len added in v1.14.0 

func (t *MerkleTree) Len() int

Len reports the number of distinct subtrees. This counts subtrees, not components: one component appearing with two different child sets is two nodes, and one subtree shared by two parents is one node.

func (*MerkleTree) MerkleNodes added in v1.14.0 

func (t *MerkleTree) MerkleNodes() iter.Seq[*MerkleNode]

MerkleNodes iterates every distinct subtree in deterministic order.

func (*MerkleTree) Node added in v1.14.0 

func (t *MerkleTree) Node(subtreeHash uuid.UUID) *MerkleNode

Node returns the node for a subtree hash, or nil.

func (*MerkleTree) Nodes added in v1.14.0 

func (t *MerkleTree) Nodes() []MerkleNode

Nodes renders the tree's component identities as rows to persist, one per distinct subtree. Insert these before Edges: an edge references two of them.

func (*MerkleTree) Parents added in v1.14.0 

func (t *MerkleTree) Parents() map[uuid.UUID][]uuid.UUID

Parents maps a subtree hash to the hashes of the subtrees that depend on it directly. Parents are sorted by component id (then by hash, to break ties between two different child sets of the same component) so traversal order is deterministic and independent of map iteration.

func (*MerkleTree) PathsToPURL added in v1.14.0 

func (t *MerkleTree) PathsToPURL(purl string, limit int) []Path

PathsToPURL returns every dependency path from a direct dependency of this SBOM down to purl, as component ids. A limit of 0 means unlimited.

Paths are found breadth-first, so shorter paths come first and a limit keeps the most direct ones. The root is not part of a path: a path starts at the direct dependency that pulls the component in.

Because the tree is keyed by subtree hash, a component appearing with two different child sets is two distinct nodes, so their paths stay separate.

func (*MerkleTree) RootNode added in v1.14.0 

func (t *MerkleTree) RootNode() *MerkleNode

RootNode returns the root of the SBOM.

func (*MerkleTree) SubtreesFor added in v1.14.0 

func (t *MerkleTree) SubtreesFor(purl string) []uuid.UUID

SubtreesFor returns the subtree hashes whose component matches purl, sorted so results do not depend on map iteration order. A component can match more than one subtree when different SBOM positions give it different child sets.

type ParsedSBOM added in v1.14.0 

type ParsedSBOM struct {
	Tree *MerkleTree
	// Components is keyed by component id (purl), not by bom-ref.
	Components map[string]cdx.Component
}

ParsedSBOM is what one ingested document yields: the tree to store, and the component metadata that goes to the components table keyed by purl.

The conversion that produces it lives in the transformer package; only the data lives here, so the service interfaces can name it without dragging the whole CycloneDX layer along.

type Path 

type Path []string

Path is a dependency path through an SBOM, as component ids.

func (Path) String 

func (p Path) String() string

String returns the path as a comma-separated string.

func (Path) ToStringSlice 

func (p Path) ToStringSlice() []string

ToStringSlice returns the path as a plain slice.

type PurlMatchContext 

type PurlMatchContext struct {
	SearchPurl                  string                    `json:"searchPurl"` // purl without version and qualifiers, used for database matching
	NormalizedVersion           string                    `json:"normalizedVersion"`
	OriginalVersion             string                    `json:"originalVersion"`
	HowToInterpretVersionString VersionInterpretationType `json:"howToInterpretVersionString"`
	Qualifiers                  packageurl.Qualifiers     `json:"qualifiers"`
	Namespace                   string                    `json:"namespace"`
}

PurlMatchContext holds the parsed purl information for matching

func ParsePurlForMatching 

func ParsePurlForMatching(purl packageurl.PackageURL) *PurlMatchContext

ParsePurlForMatching parses a purl and version into a context for database matching

type SBOMSource added in v1.14.0 

type SBOMSource struct {
	Source string
	SBOM   *ParsedSBOM
}

SBOMSource pairs a parsed SBOM with the source it came from, so several upstream documents can be stored as the separate SBOMs they are rather than merged into one.

type VersionInterpretationType 

type VersionInterpretationType string
const (
	ExactVersionString       VersionInterpretationType = "exact"
	SemanticVersionString    VersionInterpretationType = "semver_range"
	EmptyVersion             VersionInterpretationType = "empty_version"
	EcosystemSpecificVersion VersionInterpretationType = "ecosystem_specific"
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.