pebbledb

package
v3.1.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 31, 2026 License: MIT Imports: 19 Imported by: 0

Documentation

Index

Constants

View Source 
const (

	// CurrentDBVersion tracks the semantic version of the data format.
	CurrentDBVersion = "3.0.0"

	// CurrentSchemaVersion enforces binary compatibility.
	// Increment this only if the fundamental serialization format (e.g. Gob struct shape) changes.
	CurrentSchemaVersion = 3

	// BatchSizeLimitBytes limits the memory usage of a batch before commit (10MB).
	BatchSizeLimitBytes = 10 * 1024 * 1024
)

Variables

This section is empty.

Functions

func FormatEntropyKey 

func FormatEntropyKey(entropy float64, id string) string

Types

type DatabaseMetadata 

type DatabaseMetadata struct {
	Version        string            `json:"version"`
	Description    string            `json:"description"`
	CreatedAt      time.Time         `json:"created_at"`
	LastUpdatedAt  time.Time         `json:"last_updated_at"`
	SignatureCount int               `json:"signature_count"`
	SourceHash     string            `json:"source_hash"`
	Custom         map[string]string `json:"custom,omitempty"`
}

type PebbleScanner 

type PebbleScanner struct {
	// contains filtered or unexported fields
}

PebbleScanner performs semantic malware detection using CockroachDB's Pebble. It leverages LSM trees for high write throughput and efficient range scans.

func NewPebbleScanner 

func NewPebbleScanner(dbPath string, opts PebbleScannerOptions) (*PebbleScanner, error)

NewPebbleScanner opens or creates a Pebble backed signature database. It includes retry logic to handle transient file locks common in containerized environments.

func (*PebbleScanner) AddSignature 

func (s *PebbleScanner) AddSignature(sig *detection.Signature) error

AddSignature atomically saves a signature and updates all indexes. It uses a pointer to update the ID if it was auto generated.

func (*PebbleScanner) AddSignatures 

func (s *PebbleScanner) AddSignatures(sigs []*detection.Signature) error

AddSignatures adds multiple signatures in a single batch. Takes pointers to allow ID propagation.

func (*PebbleScanner) Checkpoint 

func (s *PebbleScanner) Checkpoint() error

func (*PebbleScanner) Close 

func (s *PebbleScanner) Close() error

func (*PebbleScanner) Compact 

func (s *PebbleScanner) Compact() error

func (*PebbleScanner) CountSignatures 

func (s *PebbleScanner) CountSignatures() (int, error)

func (*PebbleScanner) DeleteMetadata 

func (s *PebbleScanner) DeleteMetadata(key string) error

func (*PebbleScanner) DeleteSignature 

func (s *PebbleScanner) DeleteSignature(id string) error

func (*PebbleScanner) ExportToJSON 

func (s *PebbleScanner) ExportToJSON(jsonPath string) error

func (*PebbleScanner) GetAllMetadata 

func (s *PebbleScanner) GetAllMetadata() (*DatabaseMetadata, error)

func (*PebbleScanner) GetMetadata 

func (s *PebbleScanner) GetMetadata(key string) (string, error)

func (*PebbleScanner) GetSignature 

func (s *PebbleScanner) GetSignature(id string) (*detection.Signature, error)

func (*PebbleScanner) GetSignatureByTopology 

func (s *PebbleScanner) GetSignatureByTopology(topoHash string) (*detection.Signature, error)

func (*PebbleScanner) GetSnapshot 

func (s *PebbleScanner) GetSnapshot() *pebble.Snapshot

func (*PebbleScanner) InitializeMetadata 

func (s *PebbleScanner) InitializeMetadata(version, description string) error

func (*PebbleScanner) ListSignatureIDs 

func (s *PebbleScanner) ListSignatureIDs() ([]string, error)

func (*PebbleScanner) MarkFalsePositive 

func (s *PebbleScanner) MarkFalsePositive(id string, notes string) error

func (*PebbleScanner) MigrateFromJSON 

func (s *PebbleScanner) MigrateFromJSON(jsonPath string) (int, error)

func (*PebbleScanner) RebuildIndexes 

func (s *PebbleScanner) RebuildIndexes() error

RebuildIndexes clears and rebuilds all indexes. Optimized to stream signatures instead of loading all into memory.

func (*PebbleScanner) ScanBatch 

func (s *PebbleScanner) ScanBatch(topologies map[string]*topology.FunctionTopology) map[string][]detection.ScanResult

func (*PebbleScanner) ScanByEntropyRange 

func (s *PebbleScanner) ScanByEntropyRange(minEntropy, maxEntropy float64) ([]detection.Signature, error)

func (*PebbleScanner) ScanCandidates 

func (s *PebbleScanner) ScanCandidates(topo *topology.FunctionTopology) ([]*detection.Signature, error)

ScanCandidates implements the SignatureProvider interface. Uses a snapshot to ensure consistent view between Index Scan and Data Retrieval.

func (*PebbleScanner) ScanTopology 

func (s *PebbleScanner) ScanTopology(topo *topology.FunctionTopology, funcName string) ([]detection.ScanResult, error)

ScanTopology checks a function topology against the signature database. Refactored to wrapper around ScanTopologyWithSnapshot to centralize logic.

func (*PebbleScanner) ScanTopologyExact 

func (s *PebbleScanner) ScanTopologyExact(topo *topology.FunctionTopology, funcName string) (*detection.ScanResult, error)

ScanTopologyExact performs a high speed lookup for exact topology hash matches. It bypasses the fuzzy index entirely for performance critical paths.

func (*PebbleScanner) ScanTopologyWithSnapshot 

func (s *PebbleScanner) ScanTopologyWithSnapshot(snap *pebble.Snapshot, topo *topology.FunctionTopology, funcName string) ([]detection.ScanResult, error)

Allows scanning using an external snapshot for multi threaded consistency. This is used by batch processors to view the DB at a single point in time.

func (*PebbleScanner) SetAllMetadata 

func (s *PebbleScanner) SetAllMetadata(meta *DatabaseMetadata) error

func (*PebbleScanner) SetEntropyTolerance 

func (s *PebbleScanner) SetEntropyTolerance(tolerance float64)

func (*PebbleScanner) SetMetadata 

func (s *PebbleScanner) SetMetadata(key, value string) error

func (*PebbleScanner) SetThreshold 

func (s *PebbleScanner) SetThreshold(threshold float64)

func (*PebbleScanner) Stats 

func (s *PebbleScanner) Stats() (*PebbleScannerStats, error)

func (*PebbleScanner) TouchLastUpdated 

func (s *PebbleScanner) TouchLastUpdated() error

type PebbleScannerOptions 

type PebbleScannerOptions struct {
	MatchThreshold   float64
	EntropyTolerance float64
	ReadOnly         bool
	CacheSize        int64
}

PebbleScannerOptions configures the PebbleScanner initialization.

func DefaultPebbleScannerOptions 

func DefaultPebbleScannerOptions() PebbleScannerOptions

DefaultPebbleScannerOptions returns sensible defaults for a standard deployment.

type PebbleScannerStats 

type PebbleScannerStats struct {
	SignatureCount    int
	TopoIndexCount    int
	FuzzyIndexCount   int
	EntropyIndexCount int
	DiskSpaceUsed     int64
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.