Documentation
¶
Index ¶
- Variables
- type Cache
- type Event
- type Index
- type Indexer
- type NoopIndexer
- type WAL
- func (w *WAL) Append(ev Event) error
- func (w *WAL) Close() error
- func (w *WAL) Open(dir string) error
- func (w *WAL) Path() string
- func (w *WAL) WithCache(c *Cache)
- func (w *WAL) WithIndexer(i Indexer)
- func (w *WAL) WithMaxBytes(n int64)
- func (w *WAL) WithRedactors(r ...*regexp.Regexp)
- func (w *WAL) WithSyncEvery(d time.Duration)
Constants ¶
This section is empty.
Variables ¶
var ErrCacheNotOpen = errors.New("recorder: cache not open")
ErrCacheNotOpen is returned by Cache methods when the underlying badger DB has not been initialized via Open.
Functions ¶
This section is empty.
Types ¶
type Cache ¶
type Cache struct {
// contains filtered or unexported fields
}
Cache is a badger-backed key/value store used by the recorder to deduplicate work across the WAL (e.g. tracking which normalized path+method "schema:<path>" keys have already been seen) so it can be updated on the fly instead of re-derived on every event. This is the real badger/v4 implementation; see badger_stub.go for the "nobadger" build-tagged no-op fallback with the same method set.
func (*Cache) Close ¶
Close closes the underlying badger database. It is a no-op (returns nil) if the Cache was never opened.
func (*Cache) Exists ¶
Exists reports whether key is present in the cache: (false, nil) if it is absent, (true, nil) if present, or (false, err) if the lookup itself failed for a reason other than key-not-found. It returns ErrCacheNotOpen if the Cache has not been opened yet.
func (*Cache) Get ¶
Get reads the value stored under key, returning a copy of the bytes badger holds internally (safe to retain past the read transaction). It returns ErrCacheNotOpen if the Cache has not been opened, or badger's ErrKeyNotFound (via the underlying transaction error) if key is absent.
type Event ¶
type Event any
Event is a generic placeholder for whatever value WAL.Append is asked to persist -- typically a model.Event -- kept as `any` here so this package does not need to import the model package just to accept it.
type Index ¶
type Index struct {
// contains filtered or unexported fields
}
Index is a sqlite-backed implementation of Indexer that also doubles as a small key/value store for inferred schema snapshots (via SetInferred). It persists two tables: "idx" (one row per observed request, appended by Add) and "inferred" (one row per named schema blob, upserted by SetInferred).
func (*Index) Add ¶
func (i *Index) Add(ctx context.Context, host, proto, method, normalizedPath string, status int) error
Add inserts one row into the "idx" table recording an observed (host, proto, method, normalizedPath, status) request, implementing Indexer.Add.
func (*Index) Close ¶
Close closes the underlying sqlite database, implementing Indexer.Close. It is a no-op (returns nil) if Open was never called.
type Indexer ¶
type Indexer interface {
Open(ctx context.Context, path string) error
Close() error
Add(ctx context.Context, host, proto, method, normalizedPath string, status int) error
}
Indexer is the pluggable secondary index a WAL can attach via WithIndexer: every event WAL.Append writes is also mirrored into the index as one (host, proto, method, normalizedPath, status) row, so implementations (e.g. the sqlite-backed Index in sqlite.go) can support querying observed traffic without scanning the raw JSONL files. Open must be called before Add; Close releases any underlying resources.
type NoopIndexer ¶
type NoopIndexer struct{}
NoopIndexer is the discard-everything Indexer used when a WAL has no secondary index configured: every method is a no-op that always succeeds.
func (NoopIndexer) Add ¶
Add is a no-op: the observed (host, proto, method, normalizedPath, status) row is discarded rather than indexed, and it always returns nil.
func (NoopIndexer) Close ¶
func (NoopIndexer) Close() error
Close is a no-op: it always returns nil.
type WAL ¶
type WAL struct {
// contains filtered or unexported fields
}
WAL is a write-ahead log recorder: it appends JSON-encoded events, one per line, to a day-rotated file ("ev-YYYYMMDD.jsonl") under a directory created owner-only (0o700/0o600, SEC-0012) because captured traffic may contain sensitive data. Optional regex redactors scrub matched substrings from each encoded line before it's written, and an attached Indexer and/or Cache are updated best-effort on every Append. A WAL is safe for concurrent use; all state is guarded by an internal mutex.
func (*WAL) Append ¶
Append JSON-encodes ev, applies every configured redactor (each match replaced with "<redacted>") to the encoded bytes, and writes it as one line to the current WAL file, rotating first if the day or the configured max size (WithMaxBytes) has been exceeded. It flushes on every call and fsyncs according to WithSyncEvery's interval. After writing, it best-effort re-decodes the (post-redaction) bytes to extract host/proto/method/path/status, template-normalizes the path, and mirrors the observation into the attached Indexer and records the normalized path in the attached Cache (as "schema:<path>") if one hasn't been recorded yet; failures in that best-effort step do not fail the Append. It returns an error if the WAL is not open or if rotation/write/flush/sync fails.
func (*WAL) Close ¶
Close flushes and fsyncs the current WAL file, closes it, and closes the attached Indexer and Cache (if any). It is safe to call once the WAL is done being written to; it returns the first error encountered.
func (*WAL) Open ¶
Open creates dir (owner-only, 0o700) if needed and opens today's day-rotated WAL file for appending, initializing the WAL for use. It must be called before Append.
func (*WAL) Path ¶
Path returns the filesystem path of the WAL's currently open file (empty until Open has succeeded).
func (*WAL) WithCache ¶
WithCache attaches c as the WAL's schema-dedup cache: Append uses it to avoid redundant "schema:<path>" cache writes for a normalized path once it's already been recorded.
func (*WAL) WithIndexer ¶
WithIndexer attaches i as the WAL's secondary index: every Append also mirrors the event's (host, proto, method, normalizedPath, status) into i. It mutates the WAL in place rather than returning a new value; call it before Open (or before the next Append) to take effect.
func (*WAL) WithMaxBytes ¶
WithMaxBytes sets the size, in bytes, at which the WAL rotates to a new file even within the same day (checked on every Append via rotateIfNeeded). n <= 0 disables the size-based rotation, leaving only the daily rotation.
func (*WAL) WithRedactors ¶
WithRedactors appends r to the WAL's list of redaction patterns. Every regex in the list is applied, in order, to each event's JSON encoding before it is written, replacing every match with the literal "<redacted>". Existing redactors (from a prior call) are kept, not replaced.
func (*WAL) WithSyncEvery ¶
WithSyncEvery sets the minimum interval between fsync calls: Append syncs immediately if d <= 0 or if at least d has elapsed since the last sync, and otherwise defers the fsync to a later Append.
Source Files