monitor

package
v0.0.24 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 14, 2026 License: MIT Imports: 12 Imported by: 0

Documentation

Overview

Package monitor provides runtime statistics, Prometheus metrics, and alerting for Foxhound scraping hunts.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AlertRule

type AlertRule struct {
	// Name identifies the rule in alert payloads and logs.
	Name string
	// Condition returns true when the alert should fire.
	Condition func(stats *Stats) bool
	// Message produces the human-readable alert body from current stats.
	Message func(stats *Stats) string
	// Cooldown is the minimum duration between successive fires of this rule.
	// A zero value means no cooldown (fire on every Check).
	Cooldown time.Duration
	// contains filtered or unexported fields
}

AlertRule defines a named condition that triggers a webhook alert.

func BlockRateRule

func BlockRateRule(threshold float64, cooldown time.Duration) AlertRule

BlockRateRule returns a rule that fires when the fraction of blocked requests exceeds threshold (0.0–1.0).

func ErrorRateRule

func ErrorRateRule(threshold float64, cooldown time.Duration) AlertRule

ErrorRateRule returns a rule that fires when the fraction of failed requests exceeds threshold (0.0–1.0). Pass 0 for cooldown to fire on every Check.

type Alerter

type Alerter struct {
	// contains filtered or unexported fields
}

Alerter evaluates AlertRules against Stats and POSTs JSON payloads to a webhook URL when conditions are met.

func NewAlerter

func NewAlerter(webhookURL string, rules ...AlertRule) *Alerter

NewAlerter returns an Alerter that will POST to webhookURL when any of the provided rules fires.

func (*Alerter) Check

func (a *Alerter) Check(stats *Stats)

Check evaluates all rules against stats and fires alerts for any that match and are outside their cooldown window.

func (*Alerter) Close

func (a *Alerter) Close() error

Close releases resources held by the Alerter.

type DomainSnapshot

type DomainSnapshot struct {
	Requests   int64
	Errors     int64
	Blocked    int64
	AvgLatency time.Duration
}

DomainSnapshot contains per-domain metrics.

type LogSink

type LogSink struct{}

LogSink logs stats snapshots at info level.

func (*LogSink) Record

func (l *LogSink) Record(s StatsSnapshot)

Record logs the snapshot fields.

type PrometheusExporter

type PrometheusExporter struct {
	// contains filtered or unexported fields
}

PrometheusExporter exposes Foxhound metrics on a Prometheus-compatible HTTP endpoint. It uses its own isolated registry so it does not pollute the default global registry.

func NewPrometheus

func NewPrometheus(namespace string, port int) *PrometheusExporter

NewPrometheus creates a PrometheusExporter using the given metric namespace and HTTP port. Call Start to begin serving metrics.

func (*PrometheusExporter) RecordBlocked

func (p *PrometheusExporter) RecordBlocked()

RecordBlocked increments the blocked requests counter.

func (*PrometheusExporter) RecordError

func (p *PrometheusExporter) RecordError(domain string, errType string)

RecordError records an error keyed by domain and error type label.

func (*PrometheusExporter) RecordEscalation

func (p *PrometheusExporter) RecordEscalation()

RecordEscalation increments the static→browser escalation counter.

func (*PrometheusExporter) RecordItems

func (p *PrometheusExporter) RecordItems(n int)

RecordItems adds n to the items counter.

func (*PrometheusExporter) RecordRequest

func (p *PrometheusExporter) RecordRequest(domain string, status int, duration time.Duration)

RecordRequest records a completed HTTP request with its domain, status code, and duration.

func (*PrometheusExporter) SetActiveWalkers

func (p *PrometheusExporter) SetActiveWalkers(n int)

SetActiveWalkers sets the active walkers gauge to n.

func (*PrometheusExporter) SetQueueSize

func (p *PrometheusExporter) SetQueueSize(n int)

SetQueueSize sets the queue size gauge to n.

func (*PrometheusExporter) Start

func (p *PrometheusExporter) Start() error

Start begins serving the Prometheus metrics endpoint in a background goroutine. Returns an error only if the server fails to bind (i.e. port already in use).

func (*PrometheusExporter) Stop

func (p *PrometheusExporter) Stop() error

Stop gracefully shuts down the metrics HTTP server.

type Stats

type Stats struct {
	// StartedAt is set when NewStats is called and used for rate calculations.
	StartedAt time.Time

	// Request counters.
	Requests atomic.Int64
	Success  atomic.Int64
	Errors   atomic.Int64
	Blocked  atomic.Int64
	Bytes    atomic.Int64

	// Pipeline / crawler counters.
	Items        atomic.Int64
	Escalations  atomic.Int64
	CacheHits    atomic.Int64
	CacheMisses  atomic.Int64
	DedupSkipped atomic.Int64
	RetryCount   atomic.Int64
}

Stats collects runtime statistics for a hunt using lock-free atomic counters. All methods are safe for concurrent use.

func NewStats

func NewStats() *Stats

NewStats returns a Stats with StartedAt set to the current time.

func (*Stats) BlockRate

func (s *Stats) BlockRate() float64

BlockRate returns the percentage of requests that were blocked (0–100). Returns 0 when no requests have been recorded.

func (*Stats) Rate

func (s *Stats) Rate() float64

Rate returns the number of requests per second since StartedAt. Returns 0 if no time has elapsed yet.

func (*Stats) RecordCacheHit

func (s *Stats) RecordCacheHit()

RecordCacheHit records one cache hit.

func (*Stats) RecordCacheMiss

func (s *Stats) RecordCacheMiss()

RecordCacheMiss records one cache miss.

func (*Stats) RecordDedup

func (s *Stats) RecordDedup()

RecordDedup records one item skipped by the deduplication filter.

func (*Stats) RecordEscalation

func (s *Stats) RecordEscalation()

RecordEscalation records one static→browser escalation.

func (*Stats) RecordItems

func (s *Stats) RecordItems(n int64)

RecordItems adds n to the items counter.

func (*Stats) RecordRequest

func (s *Stats) RecordRequest(success bool, blocked bool, bytes int64)

RecordRequest records one completed request. success=true increments Success; false increments Errors. blocked=true additionally increments Blocked. bytes is the response size in bytes.

func (*Stats) RecordRetry

func (s *Stats) RecordRetry()

RecordRetry records one retry attempt.

func (*Stats) SuccessRate

func (s *Stats) SuccessRate() float64

SuccessRate returns the percentage of requests that succeeded (0–100). Returns 0 when no requests have been recorded.

func (*Stats) Summary

func (s *Stats) Summary() string

Summary returns a human-readable single-line statistics summary.

type StatsCollector

type StatsCollector struct {
	// contains filtered or unexported fields
}

StatsCollector periodically collects metrics from a StatsSource and pushes them to registered sinks (Prometheus, alerting, logging).

func NewStatsCollector

func NewStatsCollector(source StatsSource, interval time.Duration, sinks ...StatsSink) *StatsCollector

NewStatsCollector creates a collector that polls source every interval.

func (*StatsCollector) Start

func (c *StatsCollector) Start(ctx context.Context)

Start begins periodic collection in a background goroutine.

func (*StatsCollector) Stop

func (c *StatsCollector) Stop()

Stop halts the collector.

type StatsSink

type StatsSink interface {
	Record(snapshot StatsSnapshot)
}

StatsSink receives collected metrics.

type StatsSnapshot

type StatsSnapshot struct {
	Requests    int64
	Success     int64
	Errors      int64
	Blocked     int64
	Items       int64
	Escalations int64
	Bytes       int64
	Domains     map[string]DomainSnapshot
}

StatsSnapshot is a point-in-time copy of all metrics.

type StatsSource

type StatsSource interface {
	Snapshot() StatsSnapshot
}

StatsSource provides metrics to collect. monitor.Stats implements this via its Snapshot method.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL