Documentation
¶
Overview ¶
Package blocklistfeed is the remote blocklist syncer: it periodically downloads one or more remote blocklists (one domain per line, '#' comments allowed) and merges new entries into the live blocklist without removing manually-added entries. Extracted from package main per ADR-0002; it depends on the Blocklist hub only through the narrow Merger interface (main passes its *Blocklist) and on the internal/ssrf seam for the outbound fetch guard. package main keeps the blFeedSyncer singleton and the admin API handler (blocklist_feed.go shim).
Multi-feed: each feed has its own URL, sync interval, and status. A single scheduler goroutine (Start) wakes every tickInterval and syncs every feed whose interval has elapsed since its last attempt. The scheduler is ALWAYS started at startup — feeds added later via the admin API are picked up on the next tick without a restart.
Index ¶
- Constants
- type Feed
- type Merger
- type Syncer
- func (bs *Syncer) ClearFeeds()
- func (bs *Syncer) Feeds() []Feed
- func (bs *Syncer) RemoveFeed(feedURL string) bool
- func (bs *Syncer) SetFeed(feedURL string, interval time.Duration)
- func (bs *Syncer) Start(ctx context.Context)
- func (bs *Syncer) SyncAll() (int, error)
- func (bs *Syncer) SyncFeed(feedURL string) (int, error)
Constants ¶
const ( // DefaultInterval is the fallback auto-sync interval when none is // configured. Exported for the main-side callers (admin settings restore, // startup config, API handler) that pick the interval. DefaultInterval = 24 * time.Hour )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Feed ¶
type Feed struct {
URL string
Interval time.Duration // 0 = auto-sync disabled (manual Sync Now only)
LastSync time.Time // last successful sync (zero = never)
LastError string // error from the most recent attempt ("" = ok)
ImportedCount int64 // cumulative new domains merged from this feed
}
Feed is a point-in-time status snapshot of one configured feed, returned by Feeds() for the admin API and settings persistence.
type Merger ¶
Merger is the blocklist surface the syncer needs: fold downloaded feed lines into the live blocklist and report how many were newly added. main's *Blocklist satisfies this (MergeFromLines).
type Syncer ¶
type Syncer struct {
// contains filtered or unexported fields
}
Syncer downloads and merges remote domain feeds into a Merger. The zero value is safe for all methods (feeds map is lazily initialized).
func New ¶
New creates an empty syncer for bl. Feeds are added via SetFeed (startup config seed or admin API).
func (*Syncer) ClearFeeds ¶
func (bs *Syncer) ClearFeeds()
ClearFeeds removes every configured feed. Used when restoring persisted settings, which are authoritative over the YAML/CLI startup seed. Domains already merged into the blocklist are NOT removed.
func (*Syncer) Feeds ¶
Feeds returns a snapshot of all configured feeds, sorted by URL for deterministic API output and persistence. A nil syncer (feeds never configured/initialized) has no feeds — return empty rather than panic, so the read-only /api/blocklist/feed endpoint stays a 200 in that state. (Upstream fix ae1b1c1, ported into the extracted engine at merge time.)
func (*Syncer) RemoveFeed ¶
RemoveFeed deletes a feed by URL. Returns false when no such feed exists. Domains already merged into the blocklist are NOT removed (feed imports are merge-only by design).
func (*Syncer) SetFeed ¶
SetFeed adds or updates a feed. interval 0 disables auto-sync for that feed (it can still be synced manually). Existing sync statistics are preserved on update.
func (*Syncer) Start ¶
Start launches the background scheduler. It runs an immediate pass, then wakes every tickInterval and syncs each feed whose interval has elapsed since its last attempt. Feeds with interval 0 are skipped.
Source Files