Documentation
¶
Overview ¶
Package store persists findings so they survive a restart.
Two properties drive the design.
The packet loop must never wait on a disk. A sensor that stalls when storage gets slow drops traffic, and dropped traffic is undetected traffic, so alerts are handed to a buffered queue and written by a background goroutine. If the queue fills, alerts are dropped and counted rather than allowed to back up into the capture path. Losing a record of a finding is bad; losing the packets that would have produced the next one is worse.
And it stays cgo-free. modernc.org/sqlite is a pure-Go translation of SQLite, so `CGO_ENABLED=0 go build` still produces the static binary the rest of the project depends on. The alternative, mattn/go-sqlite3, is faster and would have cost the single-binary property outright.
Index ¶
- Constants
- type Options
- type Query
- type Stats
- type Store
- func (s *Store) Alerts(ctx context.Context, q Query) ([]model.Alert, error)
- func (s *Store) Close() error
- func (s *Store) CountAlerts(ctx context.Context) (int, error)
- func (s *Store) Devices(ctx context.Context) ([]model.Device, error)
- func (s *Store) Enqueue(a model.Alert)
- func (s *Store) Prune(ctx context.Context, before time.Time) (int64, error)
- func (s *Store) PruneToCount(ctx context.Context, keep int) (int64, error)
- func (s *Store) SaveDevices(ctx context.Context, devices []model.Device) error
- func (s *Store) SizeOnDisk(ctx context.Context) (int64, error)
- func (s *Store) Stats() Stats
- func (s *Store) Vacuum(ctx context.Context) error
Constants ¶
const ( // DefaultQueueSize is how many alerts may be waiting to be written. Sized // so that a burst of findings during an incident is absorbed rather than // dropped, while still bounding memory. DefaultQueueSize = 4096 // DefaultBatchSize is how many alerts go into one transaction. Committing // per alert makes SQLite fsync per alert, which is roughly two orders of // magnitude slower than batching. DefaultBatchSize = 256 // DefaultFlushInterval bounds how long an alert can sit unwritten when // traffic is quiet and no batch fills up. DefaultFlushInterval = 2 * time.Second )
Tuning for the background writer.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Query ¶
Query filters a history lookup. The zero value returns the most recent alerts at any severity.
type Stats ¶
type Stats struct {
Written uint64 `json:"written"`
Dropped uint64 `json:"dropped"`
Failed uint64 `json:"failed"`
Queued int `json:"queued"`
}
Stats reports what the writer has done.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store is a SQLite-backed record of alerts and observed devices.
func (*Store) CountAlerts ¶
CountAlerts reports how many alerts are stored.
func (*Store) Enqueue ¶
Enqueue hands an alert to the background writer. It never blocks.
This is called from the packet-processing goroutine, so blocking here would stall capture. A full queue means storage cannot keep up, and the honest response is to drop the record and count it rather than to stop detecting.
func (*Store) Prune ¶
Prune deletes alerts older than the cutoff and returns how many went.
Without this a sensor left running fills its disk, which is a failure mode that arrives quietly and then stops the sensor entirely. Alerts are the only unbounded table: devices are keyed by address, so that one is bounded by the size of the network.
Deleting rows does not shrink the file. SQLite keeps the freed pages for reuse, which is the right default for a database that keeps being written to. Call Vacuum when the space actually needs returning to the filesystem.
func (*Store) PruneToCount ¶
PruneToCount keeps only the newest n alerts.
A time-based cutoff is the usual policy, but it gives no bound during an incident, when a single hour can produce more findings than a normal month. This provides the hard ceiling that keeps the file from growing without limit no matter what the traffic does.
func (*Store) SaveDevices ¶
SaveDevices upserts the asset inventory.
Written in one transaction on a cadence rather than per change: a device's byte counters move on every packet, and persisting that would turn the inventory into the busiest table in the database for no analytical gain.
func (*Store) SizeOnDisk ¶
SizeOnDisk reports the database size in bytes, including the write-ahead log.
Source Files