secscan

package
v1.0.217 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 25, 2026 License: MIT Imports: 19 Imported by: 0

Documentation

Overview

Package secscan is the security-scan orchestrator: it ties ClamAV, YARA, the threat feed, the hash cache, and the admin hash allowlist into the single pipeline the proxy hot path calls. Extracted from package main per ADR-0006 Slice 2; the collaborators arrive as constructor-injected interfaces (Deps), so the engines stay decoupled and the decision tree is unit-testable with fakes. package main keeps the free-function glue (safeScanBody, scanBlock*, bodyNeedsBuffering, the status map) and the production adapters (security_scan.go shim).

Pipeline for every proxied request/response:

  1. URL / domain threat-feed check (CheckURL / CheckDomain) Instant, in-memory lookup against URLhaus + OpenPhish data. Applied in handleRequest before the request is forwarded upstream.

  2. Response body scan (ScanBody) a. SHA-256 hash looked up in cache → return cached verdict immediately. b. ClamAV INSTREAM scan (binary + text content). c. YARA rule matching (all loaded *.yar / *.yara rules). d. Result stored in hash cache. Applied in handleHTTP and handleTunnelInspect after reading the body.

All components are optional:

  • ClamAV is skipped when no address is configured or daemon is unreachable.
  • YARA is skipped when no matcher is injected or it reports not loaded.
  • Threat feeds are skipped when no checker is injected or it is disabled.

Index

Constants

View Source 
const MaxDecompressBytes = 64 << 20

MaxDecompressBytes limits decompressed data to 64 MB to guard against gzip bombs.

View Source 
const ScanBodyTimeout = 10 * time.Second

ScanBodyTimeout caps the total time for all body scanners (ClamAV + YARA). If the combined scan doesn't finish in time, the content is blocked (fail-closed).

Variables

This section is empty.

Functions

func AddRemoteScanFail 

func AddRemoteScanFail()

AddRemoteScanFail records a remote scan sidecar failure (incremented by the remote scanner client in package main).

func AddScanSkipped 

func AddScanSkipped()

AddScanSkipped records a response forwarded unscanned because it exceeded the scan buffer limit (incremented by package main's logScanLimitExceeded).

func DecompressForScan 

func DecompressForScan(data []byte, contentEncoding string) []byte

DecompressForScan transparently decompresses a response body based on its Content-Encoding header so that ClamAV/YARA signatures can match the actual content. Supports gzip, deflate, and identity (no-op). Brotli ("br") is attempted as gzip (some servers mislabel); on failure the raw bytes are returned so the scan still runs (defense in depth).

Returned data is limited to MaxDecompressBytes to guard against gzip bombs. If decompression fails, the original data is returned unchanged — the scan still runs on compressed bytes (fail-open for availability, but the signature gap is closed for the common case).

func ReadScanBuffer added in v1.0.207 

func ReadScanBuffer(r io.Reader, limit, contentLength int64) ([]byte, error)

ReadScanBuffer reads from r until EOF or limit bytes, whichever comes first, and returns the bytes read. It is a drop-in replacement for io.ReadAll(io.LimitReader(r, limit)) that uses contentLength — the origin's declared body size, or a value <= 0 when it declared none (chunked) — to allocate the destination buffer once instead of growing it.

contentLength is a HINT ONLY and is never trusted for correctness: the read still runs to EOF or to limit, so an origin that under-declares its body does not get the tail past its declaration forwarded unscanned, and one that over-declares simply yields a shorter buffer.

func RemoteScanFailTotal added in v1.0.211 

func RemoteScanFailTotal() int64

RemoteScanFailTotal reports fail-OPEN sidecar faults so far. Used by the rate-limited degradation log, where the counter carries the magnitude the suppressed lines would have.

func RemoteScanInflight added in v1.0.211 

func RemoteScanInflight() int64

RemoteScanInflight reports remote scan round trips currently in flight.

func ScanInflight added in v1.0.209 

func ScanInflight() int64

ScanInflight reports body scans currently running (abandoned ones included). It is a saturation gauge: a value that stays near or above the ClamAV concurrency limit means scans are queueing, which is the leading indicator of the timeout-and-abandon regime.

Types

type ClamScanner 

type ClamScanner interface {
	Ping() error
	Scan(data []byte) (name string, found bool, err error)
}

ClamScanner is the ClamAV surface ScanBody/ClamAVStatus need.

type CounterSnapshot 

type CounterSnapshot struct {
	ClamBlocked       int64
	YARABlocked       int64
	ThreatFeedBlocked int64
	ScanTimeout       int64
	ScanSkipped       int64
	RemoteScanFail    int64
	ClamScanError     int64
	ClamSaturated     int64
	ScanLateDiscarded int64
	ScanInflight      int64

	// Remote (sidecar) deployment. Every counter above except RemoteScanFail is
	// structurally zero there, so before CHAOS-53 a sidecar node exported no
	// scanning signal at all.
	RemoteScanSaturated int64
	RemoteScanInflight  int64
}

CounterSnapshot is a point-in-time copy of the scan counters.

func Counters 

func Counters() CounterSnapshot

Counters returns a snapshot of all scan counters.

type Deps 

type Deps struct {
	Clam     ClamScanner
	Yara     YARAMatcher
	Feed     ThreatChecker
	Excl     HashExcluder
	Cache    *hashcache.HashCache
	MaxBytes int64
}

Deps carries the injectable collaborators for New. Nil collaborators behave as absent (that engine/check is skipped), so partial injection is fine.

type HashExcluder 

type HashExcluder interface{ IsHashExcluded(hash string) bool }

HashExcluder is the admin hash-allowlist surface ScanBody needs.

type RemoteScanner 

type RemoteScanner struct {
	// contains filtered or unexported fields
}

RemoteScanner sends scan requests to a scan microservice.

func (*RemoteScanner) Enabled 

func (rs *RemoteScanner) Enabled() bool

Enabled reports whether remote scanning is configured.

Lock-free by contract — see the enabled field. TestRemoteScannerEnabled_IsLockFree pins the property.

func (*RemoteScanner) Health 

func (rs *RemoteScanner) Health() error

Health checks the remote scan service liveness.

func (*RemoteScanner) Init 

func (rs *RemoteScanner) Init(baseURL string)

Init configures the remote scanner client.

The client timeout is a BACKSTOP only. Every request this client makes now carries its own context deadline — the shared scan budget for /scan, five seconds for the admin probes — and those always fire first. It used to be the only bound on a scan, at 60 s, i.e. six times the budget the local path gives the identical decision.

func (*RemoteScanner) ScanBody 

func (rs *RemoteScanner) ScanBody(data []byte, contentType string) *Result

ScanBody sends data to the remote scan service and returns the result.

POSTURE — the whole point of this function, and previously the thing it got wrong. Three outcomes, not two:

  • VERDICT: the sidecar answered. Blocked → block, clean → forward. The answer must be AFFIRMATIVE; see the Clean check below.
  • BUDGET or CAPACITY: the scan did not finish in time, or the sidecar reported itself full. Fail CLOSED, exactly as the local path does for the identical condition. This is the inversion CHAOS-53 exists to fix: the deadline was private (30 s, three times the local budget), so exceeding it surfaced as an ordinary transport error, was classified as a sidecar fault, and took the fail-OPEN branch — a security control switched off by SLOWNESS, on healthy infrastructure, with the process's own fail-closed budget never consulted. It is the same defect CHAOS-52 fixed in the local path (WK-15), left standing in the deployment the CHAOS-52 runbook RECOMMENDS as the remedy for it.
  • FAULT: the sidecar is unreachable, erroring, or unintelligible. Fail OPEN, counted and alerted. This posture is unchanged and remains the recorded owner decision (WK-2b, the sibling of the local path's WK-1b); what changes is that it is now reached only by an actual fault.

func (*RemoteScanner) SetExclusions added in v1.0.211 

func (rs *RemoteScanner) SetExclusions(excl HashExcluder)

SetExclusions injects the admin-managed hash allowlist so the remote path honours it exactly as Scanner.ScanBody does.

Without this the allowlist was structurally dead in sidecar deployments: an admin clearing a false positive by hash saw the entry accepted, persisted and audited, and the object kept being blocked, with nothing anywhere saying the setting did not apply to this deployment.

func (*RemoteScanner) Status 

func (rs *RemoteScanner) Status() (map[string]interface{}, error)

Status fetches the remote scanner status (mirrors /api/security-scan/status).

func (*RemoteScanner) URL 

func (rs *RemoteScanner) URL() string

URL returns the configured base URL.

type Result 

type Result struct {
	Blocked bool
	Reason  string // virus name, YARA rule name, or feed source
	Source  string // "clamav", "yara", or "threatfeed"
	Hash    string // SHA-256 hex of scanned content (body scans only)
}

Result describes the outcome of a scan that triggered a block.

type ScanResponse 

type ScanResponse struct {
	Clean      bool   `json:"clean"`
	Blocked    bool   `json:"blocked"`
	Reason     string `json:"reason,omitempty"`
	Source     string `json:"source,omitempty"`      // "clamav", "yara", "dpi"
	Hash       string `json:"hash,omitempty"`        // SHA-256 of scanned content
	DPIPattern string `json:"dpi_pattern,omitempty"` // matched DPI pattern
	ElapsedMS  int64  `json:"elapsed_ms"`
}

ScanResponse is the JSON wire type between the scan sidecar and this client. The sidecar server (package main, scan_svc.go) produces it; keep the field set in sync with the server handler.

type Scanner 

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner ties together ClamAV, YARA, the threat feed, and the hash cache into a single, easy-to-use interface for the proxy pipeline.

func New 

func New(deps Deps) *Scanner

New builds a scanner from injected collaborators (ADR-0006). The scanner starts DISABLED — call Init to enable it — mirroring the pre-extraction package-main singleton (a struct literal with cache+maxBytes defaults that only Init flipped on). Collaborators are fixed at construction and never mutated afterwards (Init only touches clam/cache/maxBytes/enabled).

func (*Scanner) BodyScanEnabled 

func (ss *Scanner) BodyScanEnabled() bool

BodyScanEnabled reports whether body scanning (ClamAV and/or YARA) is active. Unlike Enabled it still takes mu: the clam/yara collaborators are ordinary mu-guarded fields and this is a response-stage check, not the request gate.

func (*Scanner) CacheClear 

func (ss *Scanner) CacheClear()

CacheClear empties the scan hash cache.

func (*Scanner) CacheEvict 

func (ss *Scanner) CacheEvict(hash string) bool

CacheEvict removes a single hash from the scan cache, reporting whether it was present.

func (*Scanner) CacheGet 

func (ss *Scanner) CacheGet(hash string) (hashcache.ScanCacheResult, bool)

CacheGet looks up a scan verdict (test support).

func (*Scanner) CacheReady 

func (ss *Scanner) CacheReady() bool

CacheReady reports whether the scan hash cache is initialised.

func (*Scanner) CacheSet 

func (ss *Scanner) CacheSet(hash string, r hashcache.ScanCacheResult)

CacheSet stores a scan verdict directly (test support and remote-scan result adoption).

func (*Scanner) CacheStats 

func (ss *Scanner) CacheStats() (hits, misses int64, size int)

CacheStats returns hash-cache hit/miss counters and current size.

func (*Scanner) CheckDomain 

func (ss *Scanner) CheckDomain(domain string) *Result

CheckDomain checks a bare hostname against the threat feed. Returns nil when no threat is found.

func (*Scanner) CheckURL 

func (ss *Scanner) CheckURL(rawURL string) *Result

CheckURL checks a full URL against the threat feed. Returns nil when no threat is found.

func (*Scanner) ClamAVStatus 

func (ss *Scanner) ClamAVStatus() string

ClamAVStatus returns a human-readable daemon connectivity string. Tier 2.3: Result is cached for clamStatusTTL to avoid hammering the ClamAV daemon on every admin dashboard poll. Cache is invalidated on Init().

func (*Scanner) ClamAVVersion added in v1.0.22 

func (ss *Scanner) ClamAVVersion() (clamav.Version, bool)

ClamAVVersion returns the ClamAV engine + signature database version, so operators can see whether virus definitions are current. Returns ok=false when ClamAV is disabled, the client does not support VERSION, or the daemon is unreachable. Cached for clamVersionTTL (definitions change rarely) and invalidated on Init, mirroring ClamAVStatus.

func (*Scanner) Enabled 

func (ss *Scanner) Enabled() bool

Enabled reports whether the scanner has been initialised.

Lock-free by contract — see the enabled field. This is the gate the proxy consults before any threat check, so it must stay free of mu. TestScannerEnabled_IsLockFree pins the property.

func (*Scanner) Init 

func (ss *Scanner) Init(clamAddr string, maxBytes int64, cache *hashcache.HashCache)

Init configures the scanner. 

clamAddr — ClamAV address string (see clamav.New); "" disables ClamAV.
maxBytes — maximum bytes to buffer per response (0 = use default 5 MiB).
cache    — hash cache to adopt; nil keeps the current one (ADR-0006: the
           cache is handed over here instead of being poked from outside).

func (*Scanner) MaxBytes 

func (ss *Scanner) MaxBytes() int64

MaxBytes returns the buffer limit for body scanning.

func (*Scanner) ScanBody 

func (ss *Scanner) ScanBody(data []byte) *Result

ScanBody scans a response body with ClamAV and YARA. Results are cached by SHA-256 to avoid redundant work. The entire scan is bounded by ScanBodyTimeout (fail-closed: blocks on timeout). Returns nil when the content is clean (or no scanner is enabled).

type ThreatChecker 

type ThreatChecker interface {
	Enabled() bool
	CheckURL(rawURL string) (bool, string)
	CheckDomain(domain string) (bool, string)
}

ThreatChecker is the threat-feed surface CheckURL/CheckDomain need.

type YARAMatcher 

type YARAMatcher interface {
	Loaded() bool
	Enabled() bool
	Match(data []byte) []string
}

YARAMatcher is the YARA surface the scan pipeline needs. Loaded and Enabled are deliberately distinct to preserve pre-ADR-0006 behavior verbatim: BodyScanEnabled keys on rules-present only (Loaded), while the body scan additionally honors the runtime enable toggle (Enabled).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.