Documentation
¶
Overview ¶
Package catdb is Layer 2 of the two-tier URL categorisation engine — the community category store. It is a self-contained leaf extracted from the flat package main per ADR-0002; it contains the BadgerDB dependency (the proxy's only category-DB use of Badger) behind a narrow API.
Storage: BadgerDB (v4, pure-Go, no CGo).
Key layout: []byte(domain) e.g. "facebook.com" Value layout: []byte(mapped category) e.g. "Social"
Subdomain matching: domain walking — query most-specific label first, then strip the leftmost label and retry, stopping before a bare TLD. e.g. "sub.facebook.com" → "facebook.com" → stop (next would be "com").
Layer 1 (the admin-managed catStore in package main) is always consulted first; this community store is the fallback for entries not in the admin-managed lists.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func OpenResilient ¶ added in v1.0.207
func OpenResilient(dir string) (*CommunityDB, Recovery, error)
OpenResilient opens the community store, recovering from a store the previous run could not survive. It never panics on behalf of badger — the panic case is handled by refusing to hand badger a directory a previous process died inside of — and it never destroys data it has not first moved aside.
Returns (db, recovery, nil) on success. On failure the caller MUST degrade (Layer-1-only categorisation) rather than treat it as fatal: this store is a cache, and refusing to boot an in-line gateway over a damaged cache is a self-inflicted outage.
func QuarantinedCopies ¶ added in v1.0.207
QuarantinedCopies returns the `.corrupt.*` siblings of dir, newest last. Exported so the caller can re-surface an unreconciled quarantine from a PRIOR boot: the in-memory record is process-local, so without this a self-healed store looks pristine on the next restart while the evidence — and the disk it occupies — is still sitting there.
Types ¶
type CommunityDB ¶
type CommunityDB struct {
// contains filtered or unexported fields
}
CommunityDB wraps a BadgerDB instance for URL category lookups. All exported methods are safe for concurrent use.
func Open ¶
func Open(dir string) (*CommunityDB, error)
Open opens (or creates) a BadgerDB at the given directory.
It is NOT crash-tolerant on its own, and the comment that used to claim otherwise ("Truncate is enabled so a crashed container can restart without manual intervention") was false in two ways: badger v4 removed the Truncate option entirely, and the worst crash damage does not surface as an error at all — a corrupt `.sst` makes this call PANIC from a goroutine badger spawns, which no caller can recover from. Boot paths must therefore call OpenResilient (resilient.go), which detects and quarantines a store a previous process could not survive. Direct Open is for callers that already know the directory is sound (tests, and OpenResilient itself).
func (*CommunityDB) BulkWrite ¶
func (c *CommunityDB) BulkWrite(entries map[string]string) error
BulkWrite atomically writes a batch of domain→category pairs into BadgerDB. Existing entries for the same domain are overwritten. Uses WriteBatch for high-throughput ingestion without holding a long-lived transaction — safe to call while the DB serves concurrent reads.
func (*CommunityDB) Close ¶
func (c *CommunityDB) Close() error
Close flushes and closes the underlying BadgerDB. Must be called on graceful shutdown to prevent value-log corruption.
func (*CommunityDB) Lookup ¶
func (c *CommunityDB) Lookup(host string) (string, bool)
Lookup returns the mapped category for host (or any of its parent domains). Uses domain walking: tries host, then strips the leftmost label and retries, stopping when no further parent exists above the TLD. Returns ("", false) when no entry is found.
func (*CommunityDB) Stats ¶
func (c *CommunityDB) Stats() (keys int64)
Stats returns the estimated number of keys stored in the DB.
type Recovery ¶ added in v1.0.207
type Recovery struct {
// Trigger is what prompted the recovery attempt (TriggerNone when the
// store opened cleanly on the first try).
Trigger RecoveryTrigger
// Cause is the human-readable reason behind Trigger.
Cause string
// Quarantined reports whether the damaged directory was moved aside.
Quarantined bool
// QuarantinePath is where it was moved to (empty unless Quarantined).
QuarantinePath string
// Skipped explains why a recovery that was triggered did NOT quarantine —
// a live lock holder, a missing directory, or a rename that failed. Empty
// when nothing was skipped.
Skipped string
// ResidualQuarantines lists `.corrupt.*` siblings present at open time,
// including any created by this call. A non-empty list after the operator
// has reconciled the incident is their signal to clean up.
ResidualQuarantines []string
}
Recovery reports what OpenResilient had to do. It is data, not logging: the caller owns the log line, the alert, and the metrics, so this package stays free of policy about how a degradation is surfaced.
type RecoveryTrigger ¶ added in v1.0.207
type RecoveryTrigger string
RecoveryTrigger names what caused a recovery attempt.
const ( // TriggerNone means the store opened normally. TriggerNone RecoveryTrigger = "" // TriggerPoisonMarker means a previous process died inside badger.Open on // this directory (uncatchable panic, SIGKILL, OOM, or a fatal exit). TriggerPoisonMarker RecoveryTrigger = "poison_marker" // TriggerOpenError means badger.Open returned an error identified as // corruption of the on-disk store. TriggerOpenError RecoveryTrigger = "open_error" )
Source Files