urlhaus

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 28, 2026 License: MIT Imports: 14 Imported by: 0

Documentation

Overview

Package urlhaus adds an abuse.ch URLhaus lookup to yarad: URLs pulled from a message (and from the decompressed VBA/RTF the extract package surfaces) are checked against a locally-cached feed of known malware-distribution URLs.

Design (matches the high-volume constraints):

  • The feed is downloaded ONCE per refresh interval (>=5 min, fair-use) into an in-memory set; lookups are pure local map hits, never a per-message remote API call.
  • A failed refresh keeps the previous set (fail-static) and is counted.
  • Cheap, bounded defanging ("hxxp", "[.]", "(dot)") catches URLs hidden in document code; a hit found only after defanging is flagged Deobf.
  • Matching is most-specific-wins: exact normalized URL (high confidence) else the hostname (a known-bad host). Per-message URL count is bounded.

Requires an abuse.ch Auth-Key (free, https://auth.abuse.ch/), sent as the Auth-Key header. With no key the checker is disabled (New returns nil).

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Checker 

type Checker struct {
	// contains filtered or unexported fields
}

Checker holds the cached feed and serves lookups. The zero value is not usable; use New.

func New 

func New(key string, refresh time.Duration, cacheDir string, logf func(string, ...any)) *Checker

New builds a Checker and starts its background refresher. It returns nil when key is empty (feature disabled), so callers can guard on `c != nil`. refresh is clamped to the fair-use floor. When cacheDir is non-empty the feed snapshot is persisted there and loaded on startup, so a restart serves immediately from the last-good feed instead of an empty set until the first network refresh.

func (*Checker) Check 

func (c *Checker) Check(data []byte, maxURLs int) []Hit

Check extracts URLs from data (and from a cheaply-defanged copy) via urlcand.Extract, looks each up in the feed, and returns the matches. maxURLs bounds the work per buffer. It is safe for concurrent use. Delegates to CheckCandidates.

func (*Checker) CheckCandidates 

func (c *Checker) CheckCandidates(cands []urlcand.Candidate, maxURLs int) []Hit

CheckCandidates looks up pre-extracted URL candidates in the feed. cands is produced by urlcand.Extract; maxURLs caps how many candidates are processed. A shared dedup map prevents duplicate normalized URLs across the candidate list. c.hits is incremented if any hit is found.

func (*Checker) Close 

func (c *Checker) Close()

Close stops the background refresher. Safe to call more than once and on a nil *Checker (the disabled-feature case), so shutdown code can call it unconditionally.

func (*Checker) Metrics 

func (c *Checker) Metrics() Metrics

Metrics returns a snapshot for /metrics.

type Hit 

type Hit struct {
	URL   string // the matched (normalized) URL or host
	Host  bool   // matched at host level (less specific) rather than exact URL
	Deobf bool   // only found after defanging (hxxp/[.] etc.) — more suspicious
}

Hit is one URL in a scanned buffer that matched the feed.

func (Hit) Rule 

func (h Hit) Rule() string

Rule returns the synthetic rule name for a hit, so the scanner can surface it as a match alongside YARA rules and the rspamd plugin can route it.

type Metrics 

type Metrics struct {
	Enabled         bool
	FeedURLs        int64
	FeedHosts       int64
	LastRefreshUnix int64
	RefreshFailures uint64
	Lookups         uint64 // buffers checked
	Hits            uint64 // buffers with >=1 hit
}

Metrics is a snapshot for /metrics.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.