Documentation
¶
Overview ¶
Package dns is the host-resolution cache: it maps a host to its resolved IP, honors TTLs, and feeds the per-IP politeness bucket so two hosts behind one address do not dodge rate limits.
The point of the package is to keep DNS off the dispatch hot path. Resolution is slow and bursty, so the dispatcher never blocks on a lookup: it asks the Cache for a host with Lookup, and on a miss it queues the host with Prefetch and moves on. A bounded worker pool resolves queued hosts in the background and fills the positive cache. A name that fails to resolve lands in the negative cache and is suppressed for a while so a dead host does not get retried on every dispatch.
The real resolver runs over the pure-Go net resolver (PreferGo: true). The cgo resolver blocks an OS thread per lookup and exhausts the thread limit at crawl scale; the pure-Go resolver uses goroutines instead.
DialContext rewrites a dial address to the cached IP so a connection goes straight to the resolved address while the hostname still rides in the Host header and TLS SNI. That keeps the per-IP politeness bucket keyed on exactly the IP we dialed.
Index ¶
- Constants
- type Cache
- func (c *Cache) Close()
- func (c *Cache) DialContext(base *net.Dialer) func(ctx context.Context, network, addr string) (net.Conn, error)
- func (c *Cache) Failures(host string) int
- func (c *Cache) Lookup(host string) (ip [16]byte, ok bool)
- func (c *Cache) Prefetch(hosts ...string)
- func (c *Cache) Resolve(ctx context.Context, host string) (ip [16]byte, ok bool)
- func (c *Cache) Stats() Stats
- func (c *Cache) Suppressed(host string) bool
- func (c *Cache) Wait()
- type Option
- type Resolver
- type Stats
- type SystemResolver
Constants ¶
const DefaultTTL = 5 * time.Minute
DefaultTTL is the lifetime SystemResolver reports for a record. The Go resolver does not surface the DNS TTL, so we report a sane default and let the cache clamp it into [MinTTL, MaxTTL].
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Cache ¶
type Cache struct {
// contains filtered or unexported fields
}
Cache is the resolve cache: a positive cache (host -> ip, expiry) and a negative cache (host -> fail-until), refreshed by a bounded prefetch pool.
func NewCache ¶
NewCache builds a cache over r. now is an injectable clock for tests: pass a func returning the current time; nil means time.Now.
func (*Cache) Close ¶
func (c *Cache) Close()
Close stops the worker pool. It is safe to call more than once.
func (*Cache) DialContext ¶
func (c *Cache) DialContext(base *net.Dialer) func(ctx context.Context, network, addr string) (net.Conn, error)
DialContext returns a net.Dialer DialContext func that dials the cached IP for the request's host directly (the per-IP politeness bucket is keyed on exactly this IP), falling back to the system dial when the host is uncached. The hostname still goes in the Host header and TLS SNI upstream; this only rewrites the dial address to ip:port.
func (*Cache) Failures ¶
Failures returns the current consecutive failure count for host, zero when the host is not suppressed. A run of failures is the signal to flag a dead host.
func (*Cache) Lookup ¶
Lookup returns the cached IP for host when present and unexpired. ok is false on a miss or an expired entry (the caller should Prefetch and defer the host, never resolve synchronously on the dispatch path).
func (*Cache) Prefetch ¶
Prefetch enqueues hosts for asynchronous resolution off the dispatch path. Hosts already fresh in the positive cache or still suppressed by the negative cache are skipped. It returns immediately; resolution happens in the pool.
func (*Cache) Resolve ¶
Resolve does a synchronous resolve-and-cache, for the rare path that needs it (and for tests). It honors the negative cache and the TTL clamps.
func (*Cache) Stats ¶
Stats snapshots the cache sizes. Pending counts hosts reserved for or under resolution right now.
func (*Cache) Suppressed ¶
Suppressed reports whether host is currently negatively cached (a recent failure), so the caller can flag a dead host after repeated suppression.
type Option ¶
type Option func(*config)
Option configures a Cache.
func WithMaxTTL ¶
WithMaxTTL caps a huge TTL so the cache eventually re-resolves.
func WithMinTTL ¶
WithMinTTL floors a tiny TTL so a short-lived record does not churn the cache.
func WithNegativeTTL ¶
WithNegativeTTL sets how long a failed name is suppressed before a retry.
func WithWorkers ¶
WithWorkers sets the prefetch pool concurrency. A value below 1 is ignored so the pool always has at least one worker.
type Resolver ¶
type Resolver interface {
Resolve(ctx context.Context, host string) (ip [16]byte, ttl time.Duration, err error)
}
Resolver resolves a host name to an IP and the record's TTL. The production implementation is SystemResolver over the pure-Go net resolver; tests pass a stub. It must be safe for concurrent use.
type Stats ¶
type Stats struct{ Positive, Negative, Pending int }
Stats reports cache state for the gate and tests.
type SystemResolver ¶
type SystemResolver struct {
// contains filtered or unexported fields
}
SystemResolver resolves over &net.Resolver{PreferGo: true}. Note: the Go resolver does not surface the record TTL, so Resolve returns DefaultTTL; the cache clamps TTLs into [MinTTL, MaxTTL] regardless.
Source Files