detection

package
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 21, 2026 License: MIT Imports: 15 Imported by: 0

README

Declarative detection rules

The built-in pack is embedded from rules/*.yaml and compiled once when the ingestion worker starts evaluating a job. LoadPack can validate an external directory for a bounded backtest or a future rule-management API.

Each rule has a stable id and semantic version. source is an allow-list for provider/event types; when.expression is CEL evaluated with only these bindings:

  • event: immutable event metadata and provider payload
  • org.config: non-secret tenant configuration supplied by the caller
  • org.allowlists: non-secret tenant allowlists supplied by the caller
  • now: RFC3339 timestamp for deterministic evaluation

Finding strings and evidence use a logic-free template subset. Dotted paths, first_nonempty(path, path, ...), and external_recipient(path, path, owner_path) are supported. Templates cannot call CEL, access the filesystem, or execute Go code. Unknown YAML fields, invalid semantic versions, invalid severities, oversized expressions, and duplicate rule IDs are rejected before compilation. A pack has one active semantic version per rule ID; replacing a rule version is an explicit pack rollout rather than an in-place overlap.

auto_resolve_when emits a resolution draft only. Persistence must scope the state transition by organization_id, integration_id, rule ID, and rendered dedupe target; the evaluator never mutates storage.

The evaluator exposes version-aware dedupe material for backtests. The current worker preserves the existing persisted ID-plus-target hash during migration and records the rule version in finding evidence; changing that database key requires a separate collision/backfill rollout.

The JSON schema in rule.schema.json is a portable authoring contract. The Go loader additionally compiles every expression, which catches CEL errors before activation.

Documentation

Overview

Package detection evaluates versioned, declarative detection rules.

The package deliberately owns no database or provider credentials. A rule receives an immutable event and organization context and returns finding drafts. Persistence, deduplication against SecurityFinding, and provider remediation remain in the ingestion worker. This keeps rule evaluation deterministic and makes the same engine usable by backtests and tests.

Index

Constants

This section is empty.

Variables

View Source 
var BuiltinFS embed.FS

BuiltinFS contains the reviewed P1 rules shipped with Aperio. Callers can use LoadPack on a filesystem for operator-supplied packs; production worker startup uses BuiltinEngine so it does not depend on the process cwd.

Functions

func DedupeMaterial 

func DedupeMaterial(rule Rule, target string) string

DedupeMaterial is provided for callers that need to inspect the exact versioned material without duplicating the hash implementation.

func SortedRuleIDs 

func SortedRuleIDs(rules []Rule) []string

SortedRuleIDs is useful for support-matrix and diagnostics output.

func VersionedDedupeHash 

func VersionedDedupeHash(rule Rule, target string) string

VersionedDedupeHash returns a stable content hash for external backtest reports and future version-aware persistence. The current worker keeps the legacy ID-plus-target persisted key so existing findings do not fork during this migration, and stores RuleVersion in finding provenance.

Types

type BacktestMatch 

type BacktestMatch struct {
	EventType    string `json:"eventType"`
	OccurredAt   string `json:"occurredAt"`
	Target       string `json:"target"`
	DedupeTarget string `json:"dedupeTarget"`
	Severity     string `json:"severity,omitempty"`
	Resolution   bool   `json:"resolution,omitempty"`
}

BacktestMatch is a small evidence sample suitable for CLI/API rendering.

type BacktestReport 

type BacktestReport struct {
	RuleID       string          `json:"ruleId"`
	RuleVersion  string          `json:"ruleVersion"`
	Events       int             `json:"events"`
	Candidates   int             `json:"candidates"`
	Matches      int             `json:"matches"`
	Resolutions  int             `json:"resolutions"`
	MatchSamples []BacktestMatch `json:"matchSamples,omitempty"`
}

BacktestReport summarizes deterministic replay over a fixture/event slice.

type Condition 

type Condition struct {
	Expression string `yaml:"expression" json:"expression"`
}

Condition is intentionally only an expression string. CEL is compiled at pack-load time and executes without access to Go functions, I/O, or a database.

type DedupeSpec 

type DedupeSpec struct {
	TargetTemplate string `yaml:"target_template" json:"targetTemplate"`
}

DedupeSpec defines the stable subject used by the worker's tenant-scoped dedupe key. Templates are deliberately logic-free.

type Engine 

type Engine struct {
	// contains filtered or unexported fields
}

Engine compiles a rule pack once and evaluates it without I/O. CEL's standard environment is intentionally limited to three dynamic bindings; rules cannot call Go functions or access process state.

func NewEngine 

func NewEngine(rules []Rule) (*Engine, error)

NewEngine validates and compiles rules. Compilation is the trust boundary: callers should refuse to activate a pack when this returns an error.

func (*Engine) AutoResolve 

func (e *Engine) AutoResolve(event Event, org OrgContext, overrides Overrides) ([]ResolutionDraft, error)

AutoResolve evaluates clean-event predicates for open finding state. It does not mutate storage; the ingestion worker must perform the scoped transition using the returned rule ID and target.

func (*Engine) Backtest 

func (e *Engine) Backtest(ruleID string, events []Event, org OrgContext, overrides Overrides) (BacktestReport, error)

Backtest replays events through one rule. It is intentionally in-memory so callers can use fixture events in CI or a bounded database query without granting the evaluator database access.

func (*Engine) Evaluate 

func (e *Engine) Evaluate(event Event, org OrgContext, overrides Overrides) ([]FindingDraft, error)

Evaluate produces one draft per matching rule. Event type filtering and tenant overrides happen before CEL evaluation. A runtime expression error is returned rather than treated as a match; callers can fail closed and preserve an error/metrics record for operators.

func (*Engine) Rules 

func (e *Engine) Rules() []Rule

Rules returns a stable copy of the compiled pack metadata.

type Event 

type Event struct {
	OrganizationID string
	IntegrationID  string
	Provider       string
	EventType      string
	Source         string
	Actor          string
	OccurredAt     time.Time
	Payload        map[string]any
}

Event is the provider-neutral input to the evaluator.

type FindingDraft 

type FindingDraft struct {
	RuleID           string
	RuleVersion      string
	Title            string
	Description      string
	Severity         string
	RiskScore        int
	Tags             []string
	RemediationSteps []string
	Target           string
	DedupeTarget     string
	Evidence         map[string]any
}

FindingDraft is pure evaluator output. DedupeTarget is the rendered, version-stable subject; the worker adds organization/integration identity before hashing the persisted dedupe key and records RuleVersion alongside the finding for migration-safe provenance.

type FindingSpec 

type FindingSpec struct {
	TargetTemplate   string            `yaml:"target_template,omitempty" json:"targetTemplate,omitempty"`
	Title            string            `yaml:"title" json:"title"`
	Description      string            `yaml:"description" json:"description"`
	RemediationSteps []string          `yaml:"remediation_steps" json:"remediationSteps"`
	Evidence         map[string]string `yaml:"evidence,omitempty" json:"evidence,omitempty"`
}

FindingSpec is the operator-facing output of a rule.

type OrgContext 

type OrgContext struct {
	Config     map[string]any
	Allowlists map[string]any
}

OrgContext contains non-secret tenant configuration explicitly made available to a rule. It is kept as data rather than a callback so a rule cannot perform I/O or escape the evaluator sandbox.

type Overrides 

type Overrides struct {
	Disabled          map[string]bool
	SeverityOverrides map[string]string
}

Overrides are applied after a rule is selected. Disabled is the existing integration disabled_checks mechanism. SeverityOverrides is intentionally a separate map so a caller can preserve disabled-check expiry metadata while adding a severity policy in the same integration configuration JSON.

type ResolutionDraft 

type ResolutionDraft struct {
	RuleID       string
	RuleVersion  string
	DedupeTarget string
	Evidence     map[string]any
}

ResolutionDraft identifies an open finding that a later clean event should resolve. The worker owns the state transition and must scope it by tenant, integration, rule ID, and rendered target.

type Rule 

type Rule struct {
	ID              string      `yaml:"id" json:"id"`
	Version         string      `yaml:"version" json:"version"`
	Name            string      `yaml:"name" json:"name"`
	Severity        string      `yaml:"severity" json:"severity"`
	RiskScore       int         `yaml:"risk_score" json:"riskScore"`
	Tags            []string    `yaml:"tags,omitempty" json:"tags,omitempty"`
	Source          Source      `yaml:"source" json:"source"`
	When            Condition   `yaml:"when" json:"when"`
	Dedupe          DedupeSpec  `yaml:"dedupe" json:"dedupe"`
	Finding         FindingSpec `yaml:"finding" json:"finding"`
	AutoResolveWhen *Condition  `yaml:"auto_resolve_when,omitempty" json:"autoResolveWhen,omitempty"`
}

Rule is the YAML representation of one stateless detection rule.

Rule IDs and versions are public contracts. A rule version changes when its predicate, finding semantics, or dedupe target changes; callers can retain the old version while evaluating a new one during a controlled migration.

func LoadEmbeddedPack 

func LoadEmbeddedPack(source fs.FS, pattern string) ([]Rule, error)

LoadEmbeddedPack loads YAML rules from an fs.FS, which makes the built-in rules easy to test without relying on the machine filesystem.

func LoadPack 

func LoadPack(dir string) ([]Rule, error)

LoadPack reads and validates all YAML rule files directly in dir. Nested directories are not walked implicitly: packs are versioned directories and callers must choose the exact provider pack they intend to activate.

func ValidateRules 

func ValidateRules(rules []Rule) ([]Rule, error)

ValidateRules validates a complete pack and rejects duplicate rule IDs. There is one active version per ID in an Engine; a version change is a replacement that must be rolled out deliberately rather than two programs competing for the same persisted finding key. Returning a fresh slice also makes a caller's subsequent append unable to mutate the loader's registry accidentally.

type Source 

type Source struct {
	Provider   string   `yaml:"provider" json:"provider"`
	EventTypes []string `yaml:"event_types" json:"eventTypes"`
}

Source restricts evaluation before CEL runs. Event types are normalized in the same way as ingestion worker event types, so providers can use either audit-log names (for example repository.publicized) or canonical names.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.