scan

package
v0.0.0-...-3ac6500 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 24, 2026 License: Apache-2.0 Imports: 15 Imported by: 0

README

scan

github.com/ubgo/filesystem/scan checks uploaded files for malware. It defines the Scanner contract, a client for ClamAV's clamd daemon, and Inspect, which plugs any scanner into an upload.Policy so every upload path scans the stored file before reporting success.

Scanning fails closed: a file is accepted only on a clean verdict, so a scanner that is down refuses uploads (CodeUnavailable) instead of letting them through unscanned.

The concepts are in Security and a complete set-up in Virus scanning. Writing a scanner for a cloud API is in Write a scanner.

Plugging a scanner into a policy (the guide uses a stand-in scanner; production passes scan.Clamd{Network: "tcp", Address: "127.0.0.1:3310"}):

scanning · examples/concepts-security/main.go:178-194

func scanning(ctx context.Context, _ *filesystem.Disk, out io.Writer) {
	for _, c := range []struct {
		data string
		down bool
	}{{"PNG EICAR", false}, {pngBytes, true}, {pngBytes, false}} {
		disk, err := filesystem.NewDisk("scanned", memory.New())
		if err != nil {
			fmt.Fprintln(out, err)
			return
		}
		p := images
		p.Allowed = []string{"image/*", "text/plain"}
		p.Inspect = scan.Inspect(demoScanner(c.down))
		_, err = upload.Receive(ctx, disk, strings.NewReader(c.data), "file", p)
		fmt.Fprintf(out, "scan (down=%t): %q stored=%d %v\n", c.down, filesystem.CodeOf(err), objects(ctx, disk), err)
	}
}

API

Name What it is
Scanner Scan(ctx, io.Reader) (Verdict, error); an error means the scan did not complete and says nothing about the content
ScannerFunc adapts a function to Scanner, for cloud scanning APIs and tests
Verdict Clean (true when nothing was found) and Signature (what was found, such as Win.Test.EICAR_HDB-1)
Inspect(scanner) an upload.Inspector: reads the stored file and scans it; infected is CodeRejected, a failed scan CodeUnavailable unless the scanner already classified its error; either way the upload deletes the file
Clamd a clamd client: Network ("tcp" or "unix"), Address ("127.0.0.1:3310" or a socket path), Timeout, ChunkSize; streams with INSTREAM, so the daemon needs no access to the files
Clamd.Scan(ctx, r) implements Scanner
Clamd.Ping(ctx) checks that clamd answers, for a readiness probe
DefaultTimeout two minutes, for one scan including the connection
DefaultChunkSize 64 KiB per INSTREAM chunk
Digest, Fingerprinter, MD5, SHA256 a content fingerprint (algorithm and sum) and the functions that compute one while streaming
HashMatcher, HashMatcherFunc looks a fingerprint up in a block list (a perceptual-hash service, a known-content list) and returns a Verdict
HashSet, NewHashSet(signature, digests...) an in-memory HashMatcher over a fixed set of digests
HashScanner(f, m) a Scanner that fingerprints the content with f and asks m, so moderation lists plug into Inspect like an antivirus

To scan after the upload instead of inside it, record the file quarantined in records (records.Quarantine()) and let a worker call records.Store.Scan(ctx, scope, id, scanner) with any Scanner: the file is unusable until the scan passes (Quarantine until scanned).

Errors from Clamd

Situation Code
clamd cannot be reached, the connection breaks, or it answers an error CodeUnavailable
the file is larger than clamd's StreamMaxLength (100 MiB by default in current ClamAV) CodeTooLarge

Keep StreamMaxLength at least as large as the largest upload a policy allows, or such uploads fail. task test:clamav runs the client against a real clamd in Docker.

Documentation

Overview

Package scan checks uploaded files for malware. It defines the Scanner contract, a client for ClamAV's clamd daemon (the free scanner most self-hosted products use), and Inspect, which plugs any Scanner into an upload.Policy so every upload path scans before it reports success.

Scanning fails closed: a file is accepted only on a clean verdict, so a scanner that is down refuses uploads (CodeUnavailable) instead of letting them through unscanned. An app that prefers to stay up wraps its Scanner and decides there.

Index

Constants

View Source
const (
	// DefaultTimeout bounds one scan, connection included.
	DefaultTimeout = 2 * time.Minute
	// DefaultChunkSize is the size of the INSTREAM chunks sent to clamd.
	DefaultChunkSize = 64 << 10
)

Defaults of Clamd.

View Source
const (
	AlgorithmMD5    = "md5"
	AlgorithmSHA256 = "sha256"
)

The exact-hash algorithms this package computes.

Variables

This section is empty.

Functions

func Inspect

func Inspect(s Scanner) upload.Inspector

Inspect returns an upload.Inspector that scans each stored upload with s: an infected file is CodeRejected (and deleted by the upload), a failed scan CodeUnavailable unless s already classified its error.

Types

type Clamd

type Clamd struct {
	Network string
	Address string
	// Timeout bounds one scan; 0 means DefaultTimeout.
	Timeout time.Duration
	// ChunkSize of the stream; 0 means DefaultChunkSize.
	ChunkSize int
}

Clamd scans with a clamd daemon over TCP ("tcp", "127.0.0.1:3310") or a Unix socket ("unix", "/run/clamav/clamd.ctl"), streaming the file with the INSTREAM command, so the daemon needs no access to the files. clamd refuses streams over its StreamMaxLength (100 MiB by default in current ClamAV, 25 MB in old releases); keep it at least as large as the largest upload a policy allows, or such uploads fail with CodeTooLarge.

func (Clamd) Ping

func (c Clamd) Ping(ctx context.Context) error

Ping checks that clamd answers, for a readiness probe.

func (Clamd) Scan

func (c Clamd) Scan(ctx context.Context, r io.Reader) (Verdict, error)

Scan implements Scanner.

type Digest

type Digest struct {
	// Algorithm names the fingerprint: AlgorithmMD5, AlgorithmSHA256, or
	// the name a perceptual Fingerprinter gives its own.
	Algorithm string
	// Value is the fingerprint, lower-case hex for the exact hashes.
	Value string
}

Digest is a fingerprint of a file's content: an exact hash (MD5 and SHA-256 are what published hash lists use) or a perceptual hash that survives resizing and re-encoding (PDQ, PhotoDNA), computed by a Fingerprinter.

func MD5

func MD5(ctx context.Context, r io.Reader) (Digest, error)

MD5 is the Fingerprinter for MD5 lists, which most shared hash lists of known abuse and malware material still publish.

func SHA256

func SHA256(ctx context.Context, r io.Reader) (Digest, error)

SHA256 is the Fingerprinter for SHA-256 lists.

type Fingerprinter

type Fingerprinter func(ctx context.Context, r io.Reader) (Digest, error)

Fingerprinter computes one Digest of r. Perceptual hashing needs an image decoder and lives in the application (or a module with its own dependencies); it plugs in here.

type HashMatcher

type HashMatcher interface {
	Match(ctx context.Context, d Digest) (Verdict, error)
}

HashMatcher looks a fingerprint up in a list of known content: a hash list loaded in memory (HashSet), a moderation service, or a database. A match is a Verdict with Clean false and Signature naming the list entry; an error means the lookup did not complete.

type HashMatcherFunc

type HashMatcherFunc func(ctx context.Context, d Digest) (Verdict, error)

HashMatcherFunc adapts a function to HashMatcher, for a moderation API call.

func (HashMatcherFunc) Match

func (f HashMatcherFunc) Match(ctx context.Context, d Digest) (Verdict, error)

Match implements HashMatcher.

type HashSet

type HashSet struct {
	// contains filtered or unexported fields
}

HashSet is a HashMatcher over a fixed list of digests held in memory, for lists that are published as files and loaded at start-up. It is immutable once built; load a new list into a new HashSet. Exact hashes match case-insensitively.

func NewHashSet

func NewHashSet(signature string, digests ...Digest) *HashSet

NewHashSet returns a HashSet matching digests, reporting signature (the list's name, such as "known-abuse:2026-10") on a match; the matched digest is not echoed, because the signature reaches the uploader.

func (*HashSet) Len

func (s *HashSet) Len() int

Len is the number of digests in the set.

func (*HashSet) Match

func (s *HashSet) Match(_ context.Context, d Digest) (Verdict, error)

Match implements HashMatcher.

type Scanner

type Scanner interface {
	Scan(ctx context.Context, r io.Reader) (Verdict, error)
}

Scanner scans a stream. An error means the scan did not complete; it says nothing about the content.

func HashScanner

func HashScanner(f Fingerprinter, m HashMatcher) Scanner

HashScanner returns a Scanner that fingerprints the stream with f and asks m, so a hash list plugs into Inspect like a virus scanner and fails closed the same way. Chain it with a malware Scanner through upload.Inspectors to run both.

type ScannerFunc

type ScannerFunc func(ctx context.Context, r io.Reader) (Verdict, error)

ScannerFunc adapts a function to Scanner, for cloud scanning APIs and tests.

func (ScannerFunc) Scan

func (f ScannerFunc) Scan(ctx context.Context, r io.Reader) (Verdict, error)

Scan implements Scanner.

type Verdict

type Verdict struct {
	// Clean is true when nothing was found.
	Clean bool
	// Signature names what was found when Clean is false
	// ("Win.Test.EICAR_HDB-1").
	Signature string
}

Verdict is the outcome of a scan.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL