Documentation
¶
Overview ¶
Package blocklist is the host blocklist engine: O(labels) exact/wildcard matching with a never-block exceptions list, block/allow modes, sidecar persistence (.mode/.manual/.exceptions/.sources), per-feed attribution, and hosts-format line normalization. Extracted from package main's store.go per ADR-0002 (store.go decomposition, Phase A). The matcher (IsBlocked) sits on the proxy/SOCKS5 per-request hot path and moved VERBATIM — same RWMutex, same probe sequence. package main keeps the process-wide `bl` singleton and every admin/cluster surface behind type aliases.
Index ¶
- func NormalizeLine(raw string) (string, bool)
- type Entry
- type Store
- func (b *Store) Add(host string)
- func (b *Store) AddException(host string)
- func (b *Store) AddManual(host string)
- func (b *Store) AddManualBulk(hosts []string) int
- func (b *Store) ClearAll()
- func (b *Store) Count() int
- func (b *Store) CountByFeedSource(feedURL string) int
- func (b *Store) IsBlocked(host string) bool
- func (b *Store) List() []string
- func (b *Store) ListExceptions() []string
- func (b *Store) ListWithSource() []Entry
- func (b *Store) Load(path string) error
- func (b *Store) MergeFromLines(lines []string, source string) int
- func (b *Store) Mode() string
- func (b *Store) Remove(host string)
- func (b *Store) RemoveByFeedSource(feedURL string) int
- func (b *Store) RemoveException(host string)
- func (b *Store) RemoveUnattributedFeedEntries() int
- func (b *Store) ReplaceFeedEntries(hosts []string)
- func (b *Store) RestoreFeedSources(snap map[string]string)
- func (b *Store) Save()
- func (b *Store) SetMode(mode string)
- func (b *Store) SnapshotFeedSources() map[string]string
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NormalizeLine ¶
NormalizeLine extracts the blockable host from one feed or blocklist-file line. It handles plain domain lists, /etc/hosts format ("0.0.0.0 domain" / "127.0.0.1 domain"), inline comments, accidental schemes, and trailing paths/ports. Returns ok=false for lines that carry no blockable host (comments, hosts-file boilerplate, unspecified/loopback IPs). Wildcard entries ("*.example.com") pass through untouched.
Types ¶
type Entry ¶
type Entry struct {
Host string `json:"host"`
Source string `json:"source"` // "manual" or "feed"
Feed string `json:"feed,omitempty"` // feed URL that imported this entry, when known
}
Entry is a single blocklist host with its origin.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store holds two separate maps for O(1) host lookups:
- exact: e.g. "ads.example.com"
- wildcards: keyed by dot-prefix, e.g. ".example.com" (from "*.example.com")
IsBlocked walks the host's own dot-labels to probe the wildcards map, so lookup cost is O(labels) ≈ O(1) for real-world domain names, regardless of how many wildcard rules are loaded. All methods are safe for concurrent use.
func New ¶
func New() *Store
New returns an empty Store with initialized maps and the default "block" mode — the same shape as the pre-extraction package-main literal.
func (*Store) Add ¶
Add inserts host ("*.example.com" → wildcard, otherwise exact) without persisting — the feed-sync path batches its own Save.
func (*Store) AddException ¶
AddException marks host as permanently exempt from blocking. Feed syncs will still add the host to the blocklist, but IsBlocked will always return false for it.
func (*Store) AddManual ¶
AddManual adds a host and marks it as manually managed by an admin. Unlike Add (used by the feed syncer), this persists both the source attribution (the .manual sidecar via saveManual) AND the enforcement state (the main blocklist file via Save). The dual save makes the call self-durable so a caller path that bails before its own deferred Save (e.g. the apiBlocklist POST handler returning early on an invalid wildcard mid-loop, ui_policy.go) cannot leave manual entries in memory + sidecar but missing from the main file — which would not survive restart, because Load reads the main file into b.exact / b.wildcards (the maps IsBlocked consults) and the .manual sidecar only restores attribution metadata.
For bulk admin requests, prefer AddManualBulk: it does one save for N hosts instead of N saves, avoiding the O(hosts × blocklist-size) rewrite when the main file is large (e.g. a feed-backed blocklist with hundreds of thousands of entries). Codex P2 review on PR #283.
func (*Store) AddManualBulk ¶
AddManualBulk adds multiple hosts as manually-managed admin entries under a single write lock with a single saveManual + single Save call, instead of one save per host. Use this for bulk admin requests; the per-host normalization and dedupe match AddManual exactly (lowercase + trim, empty hosts skipped, "*." → wildcard, otherwise exact). The caller is responsible for any validation (length cap, wildcard format) before invoking this method — invalid entries reaching here are silently accepted, mirroring AddManual.
Returns the number of unique normalized entries actually stored by THIS call — i.e. hosts whose admin attribution (b.manual) went from false to true. Within-batch duplicates count once; cross-call repeats of an already-attributed host count as 0. This matches the caller's expectation that "added: N" in the API response and audit line reflects net new admin entries, not raw non-empty input count. A zero return means no on-disk write happens, so calling with an empty or all-blank slice (or an all-duplicates slice) is a cheap no-op.
Added per the Codex P2 review on PR #283: with per-call Save() inside AddManual, a bulk POST of N hosts to a feed-backed blocklist rewrote the entire main file N times (O(hosts × blocklist-size) disk work). This save-once path preserves the durability guarantee while keeping bulk cost O(blocklist-size).
func (*Store) ClearAll ¶
func (b *Store) ClearAll()
ClearAll removes all blocklist entries (exact, wildcard, manual) but preserves exceptions and mode. Used by config import "replace" mode.
func (*Store) CountByFeedSource ¶
CountByFeedSource reports how many currently-listed, non-manual entries are attributed to feedURL (the number a cascade delete would remove).
func (*Store) IsBlocked ¶
IsBlocked reports whether a request to host should be blocked. In "block" mode (default): listed hosts are blocked. In "allow" mode: only listed hosts are allowed; all others blocked. Exceptions always pass through regardless of mode or list membership.
func (*Store) ListExceptions ¶
ListExceptions returns a sorted list of all exception hosts.
func (*Store) ListWithSource ¶
ListWithSource returns all blocklist entries annotated with their origin: "manual" if added by an admin via the UI/API, "feed" if imported from a feed.
func (*Store) Load ¶
Load reads the main blocklist file and its sidecars (.mode, .manual, .exceptions, .sources) from path, which becomes the persistence path.
func (*Store) MergeFromLines ¶
MergeFromLines adds all valid host entries from lines to the blocklist and saves it. Existing entries are NOT removed — safe to call on a live blocklist. Lines starting with '#' or empty are skipped; /etc/hosts-format lines are normalized to their hostname (see NormalizeLine). source is the feed URL recorded as per-entry attribution ("" = none). Returns the number of newly-added entries.
func (*Store) Remove ¶
Remove deletes host from the enforcement maps, the manual set, and the feed attribution, persisting the .manual sidecar.
func (*Store) RemoveByFeedSource ¶
RemoveByFeedSource removes every entry attributed to feedURL (cascade delete when the admin removes a feed AND opts to purge its imports). Admin-added (manual) entries always survive. Returns the removed count.
func (*Store) RemoveException ¶
RemoveException removes an exception, allowing the host to be blocked again.
func (*Store) RemoveUnattributedFeedEntries ¶
RemoveUnattributedFeedEntries removes every feed-owned entry with no recorded source — the legacy cohort imported before per-feed attribution existed and no longer present in any current feed (a host still carried by a configured feed is re-stamped on every sync, so it never stays unattributed for long). Admin-added entries always survive. Returns the removed count.
func (*Store) ReplaceFeedEntries ¶
ReplaceFeedEntries replaces the feed-pushed entries (exact + wildcards) in place, leaving DP-local state (path, mode, manual, exceptions) intact. Used by applyConfigSnapshot to avoid the wholesale-replacement pattern that previously zeroed those local fields and orphaned the persistence path. Per-host parsing mirrors Add: "*.example.com" → wildcard, otherwise → exact.
IMPORTANT: AddManual (below) writes admin-added hosts to BOTH the metadata map (b.manual) AND the enforcement maps (b.exact / b.wildcards). The enforcement maps are what IsBlocked consults; b.manual is just the attribution set. We therefore re-inject every b.manual host into the new enforcement maps before the swap so admin-added blocks survive every cluster sync. Without this re-injection (pre-fix and my first-pass ReplaceFeedEntries had the same defect; flagged by Codex on PR #249), admin manual blocks would silently disappear from enforcement on every snapshot apply.
func (*Store) RestoreFeedSources ¶
RestoreFeedSources re-stamps feed attribution onto currently-listed, non-manual entries after a wholesale rebuild. Config rollback and import-replace go through Remove/ClearAll + Add, which would otherwise strand every feed entry as "unknown origin" — making them prey for the unattributed-cleanup operation (Codex P1, PR #447). Attribution already present (e.g. re-stamped by a sync mid-rebuild) is not overwritten.
func (*Store) Save ¶
func (b *Store) Save()
Save persists the main blocklist file and the pruned .sources attribution sidecar (fsynced atomic writes). No-op when no path is set.
func (*Store) SetMode ¶
SetMode sets the list mode ("allow"; anything else means "block") and persists it to the .mode sidecar.
func (*Store) SnapshotFeedSources ¶
SnapshotFeedSources returns a copy of the per-entry feed attribution map. Taken before a wholesale rebuild (config rollback / import-replace) so RestoreFeedSources can re-stamp surviving entries afterwards.
Source Files