Documentation
¶
Overview ¶
Package vulns reports which nodes run software affected by known vulnerabilities. osctrl-tls stores each node's inventory (inventory.go); osctrl-api downloads advisory data, matches it and serves the findings (worker.go, read.go). Everything is inert unless --vuln-enabled is set.
Index ¶
- Constants
- Variables
- func AdvisoryKey(osvEcosystem string) string
- func IsVulnQuery(name string) bool
- func Migrate(db *gorm.DB) error
- func OSKey(platform, version, major string) string
- func PackageKey(ecosystem, name string) string
- func ParseEcosystems(s string) []string
- func Profile(id string) (posture.PostureProfile, bool)
- func Profiles() []posture.PostureProfile
- func RegisterMetrics(reg prometheus.Registerer, db *gorm.DB)
- func RequestSync(db *gorm.DB, at time.Time) error
- type Advisory
- type AdvisoryDetail
- type Affected
- type Alias
- type CPEProduct
- type Config
- type CountRow
- type Escalation
- type Event
- type FeedStatus
- type Finding
- type FindingFilter
- type Inventory
- func (inv *Inventory) EscalationsAfter(ctx context.Context, afterID uint, limit int) ([]NewFinding, error)
- func (inv *Inventory) FindingsAfter(ctx context.Context, afterID uint, limit int) ([]NewFinding, error)
- func (inv *Inventory) Ingest(nodeUUID string, envID uint, results []types.LogResultData)
- func (inv *Inventory) LatestEscalationID(ctx context.Context) (uint, error)
- func (inv *Inventory) LatestFindingID(ctx context.Context) (uint, error)
- type KEV
- type Matcher
- type NVDSource
- type NewFinding
- type NodeReport
- type NodeSoftware
- type NodeState
- type OSVSource
- type Range
- type Reader
- func (r *Reader) Advisory(id string, envID uint) (AdvisoryDetail, error)
- func (r *Reader) Feeds() (FeedStatus, error)
- func (r *Reader) Findings(f FindingFilter) ([]Finding, int64, error)
- func (r *Reader) HealthComponent(ctx context.Context) health.Component
- func (r *Reader) Node(uuid string, envID uint) (NodeReport, error)
- func (r *Reader) ScoreEvidence(nodeUUIDs []string) (map[string][]posture.NodePosture, error)
- func (r *Reader) ScoringRules() []posture.ScoringRule
- func (r *Reader) Summary(envID uint) (Summary, error)
- type Summary
- type SyncResult
- type SyncState
- type Worker
- type WorkerState
Constants ¶
const ( CategoryOS = "os" CategoryDeb = "deb" CategoryRPM = "rpm" CategoryPython = "python" CategoryNPM = "npm" // CPE-only categories: matched by product name against NVD, and only // while --vuln-nvd-enabled is on. CategoryHomebrew = "homebrew" CategoryPrograms = "programs" CategoryApps = "apps" CategoryChocolatey = "chocolatey" )
Inventory categories: the suffix of each osctrl:vuln: scheduled query.
const ( ConfidenceConfirmed = "confirmed" ConfidencePossible = "possible" )
Finding confidence. Confirmed is an exact package-ecosystem match against an advisory; possible is a name-and-version match against NVD CPE data, shown but never alerted on or scored.
const ( SeverityCritical = "critical" SeverityHigh = "high" SeverityMedium = "medium" SeverityLow = "low" SeverityUnknown = "unknown" )
Severity buckets, derived from a CVSS score.
const ( AdvisorySourceOSV = "osv" AdvisorySourceNVD = "nvd" )
Advisory sources: the feed that owns a record.
const ( StateOpen = "open" StateResolved = "resolved" StateAll = "all" )
Finding states for FindingFilter.State.
const ( DefaultOSVURL = "https://osv-vulnerabilities.storage.googleapis.com" DefaultKEVURL = "https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json" DefaultSyncInterval = 6 * time.Hour DefaultMaxDownload = int64(2048) << 20 )
const (
DefaultNVDURL = "https://services.nvd.nist.gov/rest/json/cves/2.0"
)
const EvidenceCategory = "vulnerabilities"
EvidenceCategory is the synthetic posture category findings are scored under.
const MaxRowsPerCategory = 50000
MaxRowsPerCategory bounds one snapshot. A bigger result is rejected rather than truncated: a truncated inventory reads as "not vulnerable".
const QueryPrefix = "osctrl:vuln:"
QueryPrefix names the scheduled queries whose results are inventory.
Variables ¶
var ErrBadFilter = errors.New("invalid filter")
ErrBadFilter is a filter value the API must reject with 400.
var ErrTooLarge = errors.New("download exceeds the size limit")
ErrTooLarge is returned when a download is bigger than the configured cap.
var ErrUnassessable = errors.New("version cannot be assessed")
ErrUnassessable means a version could not be compared. The package is reported as "not assessed", never as "not affected".
Functions ¶
func AdvisoryKey ¶
AdvisoryKey normalizes an OSV affected[].package.ecosystem value to the key node packages are matched under, or "" for ecosystems osctrl does not match. Ubuntu Pro/FIPS and Red Hat EUS streams ship packages a standard install does not have, so matching them would only produce noise.
func IsVulnQuery ¶
IsVulnQuery reports whether a scheduled query's results are inventory.
func Migrate ¶
Migrate creates the vulnerability tables. Called only when the feature is enabled, so a disabled deployment never has them.
func OSKey ¶
OSKey returns the ecosystem key for a node's distro packages, in the form AdvisoryKey produces ("Debian:12", "Ubuntu:22.04", "Red Hat:9"), or "" when the OS is not one osctrl can assess. Inputs are os_version columns.
func PackageKey ¶
PackageKey is the identity of a package name within an ecosystem. PyPI names are normalized per PEP 503; everything else compares exactly.
func ParseEcosystems ¶
ParseEcosystems splits the --vuln-ecosystems value.
func Profile ¶
func Profile(id string) (posture.PostureProfile, bool)
Profile returns one profile by id.
func Profiles ¶
func Profiles() []posture.PostureProfile
Profiles returns the inventory schedules, one per platform, in the shape posture profiles use so the existing apply workflow can merge them.
func RegisterMetrics ¶
func RegisterMetrics(reg prometheus.Registerer, db *gorm.DB)
RegisterMetrics registers the vulnerability gauges.
Types ¶
type Advisory ¶
type Advisory struct {
ID string `gorm:"primaryKey;type:varchar(128)" json:"id"`
// Source is the feed that owns the record. Existing rows predate NVD
// support and are OSV records.
Source string `gorm:"type:varchar(8);default:osv;index" json:"source"`
Summary string `gorm:"type:text" json:"summary"`
Details string `gorm:"type:text" json:"details"`
CVSSVector string `gorm:"type:varchar(255)" json:"cvss_vector"`
CVSSScore float64 `json:"cvss_score"`
Severity string `gorm:"type:varchar(16);index" json:"severity"`
// CVSSFrom is the aliased NVD CVE an OSV record without its own CVSS
// borrowed its score from. Empty when the score is the record's own.
CVSSFrom string `gorm:"type:varchar(32);default:''" json:"cvss_from"`
KEV bool `gorm:"index" json:"kev"`
RefURLs string `gorm:"type:text" json:"-"` // JSON array of URLs
Published time.Time `json:"published"`
Modified time.Time `json:"modified"`
}
Advisory is one vulnerability record (an OSV id such as DSA-5678-1 or RHSA-2026:1234). KEV is owned by the KEV sync, never by the record.
type AdvisoryDetail ¶
type AdvisoryDetail struct {
Advisory Advisory `json:"advisory"`
Aliases []string `json:"aliases"`
References []string `json:"references"`
Findings []Finding `json:"findings"`
}
AdvisoryDetail is one advisory and the nodes in one environment it affects.
type Affected ¶
type Affected struct {
ID uint `gorm:"primarykey"`
AdvisoryID string `gorm:"type:varchar(128);index"`
Ecosystem string `gorm:"type:varchar(64);index:idx_vuln_affected_pkg"`
Package string `gorm:"type:varchar(255);index:idx_vuln_affected_pkg"`
Ranges string `gorm:"type:text"` // JSON []Range
Versions string `gorm:"type:text"` // JSON []string
}
Affected is one (ecosystem, package) an advisory affects. Ecosystem is the normalized key from AdvisoryKey; Package is the PackageKey of the name.
type Alias ¶
type Alias struct {
AdvisoryID string `gorm:"primaryKey;type:varchar(128)"`
Alias string `gorm:"primaryKey;type:varchar(128);index"`
}
Alias links an advisory to another identifier for it, usually a CVE.
type CPEProduct ¶
type CPEProduct struct {
Vendor string `gorm:"primaryKey;type:varchar(128)"`
Product string `gorm:"primaryKey;type:varchar(128);index"`
}
CPEProduct is one vendor:product that a stored NVD CVE lists. It is the CPE dictionary name matching looks names up in: only products some CVE affects can produce a finding, so the full NVD dictionary would add nothing.
func (CPEProduct) TableName ¶
func (CPEProduct) TableName() string
type Config ¶
type Config struct {
OSVURL string
KEVURL string
Ecosystems []string // OSV directories; empty = what the fleet reports
SyncInterval time.Duration
Retention time.Duration
MaxDownload int64
HTTPClient *http.Client
// NVD CPE matching (--vuln-nvd-*). The key is a secret: it travels only
// as a request header.
NVDEnabled bool
NVDURL string
NVDAPIKey string
}
Config is the worker's view of the --vuln-* flags.
type Escalation ¶
type Escalation struct {
ID uint `gorm:"primarykey"`
FindingID uint `gorm:"index"`
PrevSeverity string `gorm:"type:varchar(16)"`
PrevKEV bool
CreatedAt time.Time `gorm:"index"`
}
Escalation records an open confirmed finding that became known-exploited or rose in severity: news the alert watcher reports once, by ID, like a new finding. Rows are pruned after escalationRetention.
func (Escalation) TableName ¶
func (Escalation) TableName() string
type Event ¶
type Event struct {
Introduced string `json:"introduced,omitempty"`
Fixed string `json:"fixed,omitempty"`
LastAffected string `json:"last_affected,omitempty"`
Limit string `json:"limit,omitempty"`
}
Event is one OSV range event; exactly one field is set.
type FeedStatus ¶
type FeedStatus struct {
Sources []SyncState `json:"sources"`
LastSyncAt *time.Time `json:"last_sync_at"`
SyncRequestedAt *time.Time `json:"sync_requested_at"`
LastSyncFailed bool `json:"last_sync_failed"`
Loaded bool `json:"loaded"`
Stale bool `json:"stale"`
}
FeedStatus is the admin view of the feeds and the worker.
type Finding ¶
type Finding struct {
ID uint `gorm:"primarykey" json:"id"`
NodeUUID string `gorm:"type:varchar(64);uniqueIndex:idx_vuln_finding" json:"node_uuid"`
EnvironmentID uint `gorm:"index" json:"environment_id"`
AdvisoryID string `gorm:"type:varchar(128);uniqueIndex:idx_vuln_finding;index" json:"advisory_id"`
Ecosystem string `gorm:"type:varchar(64);uniqueIndex:idx_vuln_finding" json:"ecosystem"`
Package string `gorm:"type:varchar(255);uniqueIndex:idx_vuln_finding" json:"package"`
InstalledVersion string `gorm:"type:varchar(255)" json:"installed_version"`
FixedVersion string `gorm:"type:varchar(255)" json:"fixed_version"`
Confidence string `gorm:"type:varchar(16);index" json:"confidence"`
Severity string `gorm:"type:varchar(16);index" json:"severity"`
KEV bool `json:"kev"`
FirstSeen time.Time `json:"first_seen"`
LastSeen time.Time `json:"last_seen"`
ResolvedAt *time.Time `gorm:"index" json:"resolved_at"`
// PeakSeverity and EverKEV are the highest state the finding reached:
// escalations are measured against them, so falling and coming back is
// not news. ReopenedAt is the last reopen: escalations recorded before
// it are not news either.
PeakSeverity string `gorm:"type:varchar(16);default:''" json:"-"`
EverKEV bool `gorm:"default:false" json:"-"`
ReopenedAt *time.Time `json:"-"`
}
Finding is one advisory affecting one package on one node. Resolved findings are kept (ResolvedAt set) so "fixed on" stays visible.
type FindingFilter ¶
type FindingFilter struct {
EnvironmentID uint
NodeUUID string
Severity string
Confidence string
KEV bool
State string
Package string
Advisory string
Page int
PageSize int
}
FindingFilter selects findings. EnvironmentID is mandatory.
type Inventory ¶
Inventory stores the software nodes report through the osctrl:vuln: scheduled queries. It is the only part of this feature osctrl-tls runs.
func NewInventory ¶
NewInventory migrates the vulnerability tables and returns the store.
func (*Inventory) EscalationsAfter ¶
func (inv *Inventory) EscalationsAfter(ctx context.Context, afterID uint, limit int) ([]NewFinding, error)
EscalationsAfter lists escalations of open confirmed findings with escalation id > afterID, lowest first, at most limit. ID is the escalation's; the rest is the finding as it is now.
func (*Inventory) FindingsAfter ¶
func (inv *Inventory) FindingsAfter(ctx context.Context, afterID uint, limit int) ([]NewFinding, error)
FindingsAfter lists open confirmed findings with id > afterID, lowest id first, at most limit, with the node's hostname (empty when the node was deleted). "Possible" findings never alert, and a finding resolved before the watcher saw it is no longer news.
func (*Inventory) Ingest ¶
func (inv *Inventory) Ingest(nodeUUID string, envID uint, results []types.LogResultData)
Ingest stores every osctrl:vuln: snapshot in results. Differential results are ignored: the profile schedules snapshots, and an added/removed row is not a whole inventory to replace the stored one with.
func (*Inventory) LatestEscalationID ¶
LatestEscalationID returns the highest escalation id, 0 when there is none.
type KEV ¶
type KEV struct {
CVEID string `gorm:"primaryKey;type:varchar(32)"`
}
KEV is one CVE in the CISA Known Exploited Vulnerabilities catalog.
type Matcher ¶
type Matcher struct {
DB *gorm.DB
// CPE turns on name matching against NVD data (--vuln-nvd-enabled).
CPE bool
// contains filtered or unexported fields
}
Matcher recomputes findings from stored inventory and advisories.
type NVDSource ¶
type NVDSource struct {
// contains filtered or unexported fields
}
NVDSource syncs CVEs that list application CPEs from the NVD CVE API 2.0, or from a mirror serving the same responses.
type NewFinding ¶
type NewFinding struct {
ID uint
NodeUUID string
Hostname string
EnvironmentID uint
AdvisoryID string
Package string
InstalledVersion string
FixedVersion string
Severity string
KEV bool
Escalated bool
PrevSeverity string
PrevKEV bool
// FindingID is the escalated finding (ID is the escalation's own).
FindingID uint
}
NewFinding is one open confirmed finding, or one escalation of it, for the alert watcher.
type NodeReport ¶
type NodeReport struct {
Findings []Finding `json:"findings"`
NotAssessed int `json:"not_assessed"`
InventoryAt *time.Time `json:"inventory_at"`
MatchedAt *time.Time `json:"matched_at"`
Loaded bool `json:"loaded"`
Stale bool `json:"stale"`
}
NodeReport is one node's findings and assessment coverage.
type NodeSoftware ¶
type NodeSoftware struct {
ID uint `gorm:"primarykey" json:"-"`
NodeUUID string `gorm:"type:varchar(64);index:idx_vuln_sw_node" json:"node_uuid"`
EnvironmentID uint `gorm:"index" json:"environment_id"`
Category string `gorm:"type:varchar(32);index:idx_vuln_sw_node" json:"category"`
Name string `gorm:"type:varchar(255)" json:"name"`
SourceName string `gorm:"type:varchar(255)" json:"source_name"`
Version string `gorm:"type:varchar(255)" json:"version"`
Arch string `gorm:"type:varchar(32)" json:"arch"`
Vendor string `gorm:"type:varchar(255)" json:"vendor"`
LastSeen time.Time `json:"last_seen"`
}
NodeSoftware is one installed package on one node, from the latest inventory snapshot for its category.
func (NodeSoftware) TableName ¶
func (NodeSoftware) TableName() string
type NodeState ¶
type NodeState struct {
NodeUUID string `gorm:"primaryKey;type:varchar(64)" json:"node_uuid"`
EnvironmentID uint `gorm:"index" json:"environment_id"`
OSPlatform string `gorm:"type:varchar(64)" json:"os_platform"`
OSVersion string `gorm:"type:varchar(128)" json:"os_version"`
OSMajor string `gorm:"type:varchar(16)" json:"os_major"`
InventoryAt time.Time `gorm:"index" json:"inventory_at"`
MatchedAt *time.Time `gorm:"index" json:"matched_at"`
NotAssessed int `json:"not_assessed"`
// Assessed counts packages matched against an exact ecosystem feed
// (OSV). It is what a clean result means for scoring; CPE matches never
// count.
Assessed int `json:"assessed"`
}
NodeState is per-node bookkeeping: the OS (which picks the advisory ecosystem for distro packages) and whether findings are current. A node needs matching when MatchedAt is NULL or older than InventoryAt.
type OSVSource ¶
type OSVSource struct {
// contains filtered or unexported fields
}
OSVSource syncs ecosystems from an OSV bucket (or a mirror of its layout): <base>/<dir>/all.zip, <base>/<dir>/modified_id.csv, <base>/<dir>/<id>.json.
type Reader ¶
type Reader struct {
DB *gorm.DB
StaleAfter time.Duration
// contains filtered or unexported fields
}
Reader answers the API's questions. Every query is scoped by environment (or node, which the handler has already checked against the environment).
func NewReader ¶
NewReader returns a reader that reports feeds as stale after three missed sync intervals.
func (*Reader) Advisory ¶
func (r *Reader) Advisory(id string, envID uint) (AdvisoryDetail, error)
Advisory returns gorm.ErrRecordNotFound for an unknown id.
func (*Reader) Feeds ¶
func (r *Reader) Feeds() (FeedStatus, error)
Feeds reports each feed's progress.
func (*Reader) Findings ¶
func (r *Reader) Findings(f FindingFilter) ([]Finding, int64, error)
Findings returns one page of findings and the total matching the filter.
func (*Reader) HealthComponent ¶
HealthComponent reports feed health: degraded when a feed is failing, unknown before the first OSV sync, stale when data is old. It reads under ctx so a stalled database cannot hang the health page.
func (*Reader) Node ¶
func (r *Reader) Node(uuid string, envID uint) (NodeReport, error)
Node returns the open findings of one node in environment envID.
func (*Reader) ScoreEvidence ¶
ScoreEvidence implements posture.EvidenceSource: one record per node with the counts of its open confirmed findings.
A node gets a record only once advisory data is loaded and the node either has confirmed findings, or its last match assessed at least one package against an exact ecosystem feed. Anything less would let an unassessed node pass (Amazon Linux, an OS-only Windows node, an ecosystem whose feed never synced, a node only NVD names cover). An evaluated node stays evaluated while an advisory sync has cleared its matched_at, so its score does not flicker after every sync.
func (*Reader) ScoringRules ¶
func (r *Reader) ScoringRules() []posture.ScoringRule
ScoringRules implements posture.EvidenceSource.
type Summary ¶
type Summary struct {
Loaded bool `json:"loaded"`
Stale bool `json:"stale"`
BySeverity map[string]map[string]int64 `json:"by_severity"`
KEV int64 `json:"kev"`
// Possible counts open possible (NVD CPE) findings. KEV, AffectedNodes
// and the top lists count confirmed findings only.
Possible int64 `json:"possible"`
// KEVPossible counts open possible findings that are known-exploited.
KEVPossible int64 `json:"kev_possible"`
AffectedNodes int64 `json:"affected_nodes"`
NotAssessed int64 `json:"not_assessed"`
TopAdvisories []CountRow `json:"top_advisories"`
TopPackages []CountRow `json:"top_packages"`
}
Summary is the environment overview.
type SyncResult ¶
type SyncResult struct{ Written, Skipped int }
SyncResult reports what one sync processed.
type SyncState ¶
type SyncState struct {
Source string `gorm:"primaryKey;type:varchar(128)" json:"source"`
Cursor time.Time `json:"cursor"`
LastSuccess *time.Time `json:"last_success"`
LastWritten int `json:"last_written"`
LastSkipped int `json:"last_skipped"`
LastError string `gorm:"type:text" json:"last_error"`
LastErrorAt *time.Time `json:"last_error_at"`
// ResumeIndex and ResumeStart let a full NVD sync that failed part-way
// continue from its next page; ResumeStart is when that sync began.
ResumeIndex int `json:"-"`
ResumeStart *time.Time `json:"-"`
}
SyncState is one feed's progress: "osv:<dir>" or "kev". Cursor only advances after a successful sync.
type Worker ¶
type Worker struct {
// contains filtered or unexported fields
}
Worker syncs feeds and keeps findings current. Every osctrl-api replica runs one; a lease row lets exactly one of them work at a time.
type WorkerState ¶
type WorkerState struct {
Name string `gorm:"primaryKey;type:varchar(32)" json:"-"`
Owner string `gorm:"type:varchar(128)" json:"-"`
LeaseUntil time.Time `json:"-"`
SyncRequestedAt *time.Time `json:"sync_requested_at"`
LastSyncAt *time.Time `json:"last_sync_at"`
LastSyncFailed bool `json:"last_sync_failed"`
LastHousekeepAt *time.Time `json:"-"`
// NVDActiveAt is the last tick of a replica with --vuln-nvd-enabled.
// Replicas with NVD off keep NVD data while it is recent.
NVDActiveAt *time.Time `json:"-"`
// NVDActiveBy is the host that stamped NVDActiveAt: a host's own stamp,
// from before it restarted with NVD off, does not delay the drop.
NVDActiveBy string `gorm:"type:varchar(255)" json:"-"`
}
WorkerState is the single coordination row shared by every osctrl-api replica: who holds the lease and what the worker last did.
func (WorkerState) TableName ¶
func (WorkerState) TableName() string