Documentation
¶
Overview ¶
Package monitor provides runtime statistics, Prometheus metrics, and alerting for Foxhound scraping hunts.
Index ¶
- type AlertRule
- type Alerter
- type DomainSnapshot
- type LogSink
- type PrometheusExporter
- func (p *PrometheusExporter) RecordBlocked()
- func (p *PrometheusExporter) RecordError(domain string, errType string)
- func (p *PrometheusExporter) RecordEscalation()
- func (p *PrometheusExporter) RecordItems(n int)
- func (p *PrometheusExporter) RecordRequest(domain string, status int, duration time.Duration)
- func (p *PrometheusExporter) SetActiveWalkers(n int)
- func (p *PrometheusExporter) SetQueueSize(n int)
- func (p *PrometheusExporter) Start() error
- func (p *PrometheusExporter) Stop() error
- type Stats
- func (s *Stats) BlockRate() float64
- func (s *Stats) Rate() float64
- func (s *Stats) RecordCacheHit()
- func (s *Stats) RecordCacheMiss()
- func (s *Stats) RecordDedup()
- func (s *Stats) RecordEscalation()
- func (s *Stats) RecordItems(n int64)
- func (s *Stats) RecordRequest(success bool, blocked bool, bytes int64)
- func (s *Stats) RecordRetry()
- func (s *Stats) SuccessRate() float64
- func (s *Stats) Summary() string
- type StatsCollector
- type StatsSink
- type StatsSnapshot
- type StatsSource
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AlertRule ¶
type AlertRule struct {
// Name identifies the rule in alert payloads and logs.
Name string
// Condition returns true when the alert should fire.
Condition func(stats *Stats) bool
// Message produces the human-readable alert body from current stats.
Message func(stats *Stats) string
// Cooldown is the minimum duration between successive fires of this rule.
// A zero value means no cooldown (fire on every Check).
Cooldown time.Duration
// contains filtered or unexported fields
}
AlertRule defines a named condition that triggers a webhook alert.
func BlockRateRule ¶
BlockRateRule returns a rule that fires when the fraction of blocked requests exceeds threshold (0.0–1.0).
type Alerter ¶
type Alerter struct {
// contains filtered or unexported fields
}
Alerter evaluates AlertRules against Stats and POSTs JSON payloads to a webhook URL when conditions are met.
func NewAlerter ¶
NewAlerter returns an Alerter that will POST to webhookURL when any of the provided rules fires.
type DomainSnapshot ¶
DomainSnapshot contains per-domain metrics.
type PrometheusExporter ¶
type PrometheusExporter struct {
// contains filtered or unexported fields
}
PrometheusExporter exposes Foxhound metrics on a Prometheus-compatible HTTP endpoint. It uses its own isolated registry so it does not pollute the default global registry.
func NewPrometheus ¶
func NewPrometheus(namespace string, port int) *PrometheusExporter
NewPrometheus creates a PrometheusExporter using the given metric namespace and HTTP port. Call Start to begin serving metrics.
func (*PrometheusExporter) RecordBlocked ¶
func (p *PrometheusExporter) RecordBlocked()
RecordBlocked increments the blocked requests counter.
func (*PrometheusExporter) RecordError ¶
func (p *PrometheusExporter) RecordError(domain string, errType string)
RecordError records an error keyed by domain and error type label.
func (*PrometheusExporter) RecordEscalation ¶
func (p *PrometheusExporter) RecordEscalation()
RecordEscalation increments the static→browser escalation counter.
func (*PrometheusExporter) RecordItems ¶
func (p *PrometheusExporter) RecordItems(n int)
RecordItems adds n to the items counter.
func (*PrometheusExporter) RecordRequest ¶
func (p *PrometheusExporter) RecordRequest(domain string, status int, duration time.Duration)
RecordRequest records a completed HTTP request with its domain, status code, and duration.
func (*PrometheusExporter) SetActiveWalkers ¶
func (p *PrometheusExporter) SetActiveWalkers(n int)
SetActiveWalkers sets the active walkers gauge to n.
func (*PrometheusExporter) SetQueueSize ¶
func (p *PrometheusExporter) SetQueueSize(n int)
SetQueueSize sets the queue size gauge to n.
func (*PrometheusExporter) Start ¶
func (p *PrometheusExporter) Start() error
Start begins serving the Prometheus metrics endpoint in a background goroutine. Returns an error only if the server fails to bind (i.e. port already in use).
func (*PrometheusExporter) Stop ¶
func (p *PrometheusExporter) Stop() error
Stop gracefully shuts down the metrics HTTP server.
type Stats ¶
type Stats struct {
// StartedAt is set when NewStats is called and used for rate calculations.
StartedAt time.Time
// Request counters.
Requests atomic.Int64
Success atomic.Int64
Errors atomic.Int64
Blocked atomic.Int64
Bytes atomic.Int64
// Pipeline / crawler counters.
Items atomic.Int64
Escalations atomic.Int64
CacheHits atomic.Int64
CacheMisses atomic.Int64
DedupSkipped atomic.Int64
RetryCount atomic.Int64
}
Stats collects runtime statistics for a hunt using lock-free atomic counters. All methods are safe for concurrent use.
func NewStats ¶
func NewStats() *Stats
NewStats returns a Stats with StartedAt set to the current time.
func (*Stats) BlockRate ¶
BlockRate returns the percentage of requests that were blocked (0–100). Returns 0 when no requests have been recorded.
func (*Stats) Rate ¶
Rate returns the number of requests per second since StartedAt. Returns 0 if no time has elapsed yet.
func (*Stats) RecordCacheHit ¶
func (s *Stats) RecordCacheHit()
RecordCacheHit records one cache hit.
func (*Stats) RecordCacheMiss ¶
func (s *Stats) RecordCacheMiss()
RecordCacheMiss records one cache miss.
func (*Stats) RecordDedup ¶
func (s *Stats) RecordDedup()
RecordDedup records one item skipped by the deduplication filter.
func (*Stats) RecordEscalation ¶
func (s *Stats) RecordEscalation()
RecordEscalation records one static→browser escalation.
func (*Stats) RecordItems ¶
RecordItems adds n to the items counter.
func (*Stats) RecordRequest ¶
RecordRequest records one completed request. success=true increments Success; false increments Errors. blocked=true additionally increments Blocked. bytes is the response size in bytes.
func (*Stats) SuccessRate ¶
SuccessRate returns the percentage of requests that succeeded (0–100). Returns 0 when no requests have been recorded.
type StatsCollector ¶
type StatsCollector struct {
// contains filtered or unexported fields
}
StatsCollector periodically collects metrics from a StatsSource and pushes them to registered sinks (Prometheus, alerting, logging).
func NewStatsCollector ¶
func NewStatsCollector(source StatsSource, interval time.Duration, sinks ...StatsSink) *StatsCollector
NewStatsCollector creates a collector that polls source every interval.
func (*StatsCollector) Start ¶
func (c *StatsCollector) Start(ctx context.Context)
Start begins periodic collection in a background goroutine.
type StatsSink ¶
type StatsSink interface {
Record(snapshot StatsSnapshot)
}
StatsSink receives collected metrics.
type StatsSnapshot ¶
type StatsSnapshot struct {
Requests int64
Success int64
Errors int64
Blocked int64
Items int64
Escalations int64
Bytes int64
Domains map[string]DomainSnapshot
}
StatsSnapshot is a point-in-time copy of all metrics.
type StatsSource ¶
type StatsSource interface {
Snapshot() StatsSnapshot
}
StatsSource provides metrics to collect. monitor.Stats implements this via its Snapshot method.