Documentation
¶
Overview ¶
Package providers turns a set of upstream resolvers into one upstream that keeps working when some of them do not.
A Provider is a named upstream: a transport, plus the health record that says whether it is currently worth asking. A Group is several providers and a policy for choosing between them. Both satisfy the same interface the transports do, so a resolver holding one group and a resolver holding one socket are the same code.
What counts as a failure ¶
This is the distinction the whole package turns on, and getting it wrong is how a resolver either fails over constantly or never fails over at all.
A transport error — a timeout, a refused connection, a malformed reply, a forged datagram — is a failure of the upstream. It says nothing about the name being asked for, so another upstream is worth trying and the failure counts against this one's health.
SERVFAIL and REFUSED are also failures, even though they arrived as well-formed DNS. SERVFAIL means the upstream could not answer; REFUSED usually means it will not serve us at all, which is a configuration problem that will not fix itself. Both are worth retrying elsewhere.
NXDOMAIN and NODATA are ANSWERS. They are the upstream doing its job, and a resolver that treats them as failures will ask every provider in turn about every typo, turn one query into six, and finally return the same answer the first provider gave — having quadrupled its own latency and leaked the name to five more operators. Classify draws this line in one place so that no caller has to redraw it.
Health, and why a failing upstream is not simply skipped ¶
An upstream that has just failed is likely to fail again, so continuing to send it a share of traffic wastes a query per attempt. But an upstream that is skipped forever never comes back, and outages end.
Health resolves that with a circuit breaker: consecutive failures trip it, a tripped provider is passed over, and after a cooldown one query is allowed through to see whether the outage is over. A single success closes it again. The cooldown grows as repeated probes fail, so a provider that is genuinely gone is probed less and less rather than at a fixed rate forever.
The probe is what makes this safe: without it the first outage would be permanent, and with an unbounded probe rate a dead upstream would receive a steady share of production traffic to no purpose.
Choosing between healthy providers ¶
Strategy is the policy, and the choice is a real trade rather than a default worth defending:
- Sequential asks the first healthy provider and moves down the list on failure. Deterministic, cheapest, and the right answer when one upstream is genuinely preferred — but a query that fails over pays both latencies.
- Race asks several at once and takes the first good answer. Fastest at the tail, and the only strategy where one slow provider costs nothing. It multiplies query volume and leaks every name to every provider raced, which is a privacy decision as much as a performance one.
- RoundRobin and Weighted spread load. Useful when the providers are equivalent and the aim is not to depend on any one of them.
- Random spreads it without a shared counter, and spreads the FAILOVER wave too: under round robin every query starting at one provider falls back to the same neighbour, so an outage moves a predictable block of traffic onto a predictable machine.
What this package does not do ¶
It does not cache, retry the same provider, chase CNAMEs, or interpret an answer beyond classifying it. Those belong to the resolver above it. It holds no sockets of its own: every provider is constructed from a transport the caller supplied or from a preset the caller named.
Index ¶
- Constants
- Variables
- func ParseEndpoint(s string) (scheme, addr string, err error)
- type Allower
- type BuildOptions
- type Concurrent
- type Group
- type GroupOptions
- type Health
- type HealthOptions
- type HealthState
- type Options
- type Outcome
- type Preset
- type Provider
- type ProviderError
- type RCodeError
- type RandSource
- type Registry
- type SelectAppender
- type State
- type Strategy
- type StrategyValidator
Examples ¶
Constants ¶
const ( // DefaultFailureThreshold is how many consecutive failures trip the // breaker. Five is high enough that a single lost datagram or a connection // the peer closed does not take an upstream out of service, and low enough // that a genuinely broken one is skipped within a second or two of traffic. DefaultFailureThreshold = 5 // DefaultCooldown is how long a tripped provider is passed over before one // probe is allowed through. DefaultCooldown = 5 * time.Second // DefaultMaxCooldown caps the backoff. A provider that has been gone for an // hour should not be probed every five seconds forever, and should not stop // being probed either: five minutes means a returning upstream is noticed // within five minutes without the probes amounting to traffic. DefaultMaxCooldown = 5 * time.Minute )
Defaults for HealthOptions.
const ( // SchemeUDP is classic DNS over a datagram, RFC 1035. Fastest and // cleartext: every observer on the path reads the name being resolved. SchemeUDP = "udp" // SchemeTCP is classic DNS over a stream, RFC 7766. Also cleartext; useful // where datagrams are blocked or answers are routinely truncated. SchemeTCP = "tcp" // SchemeTLS is DNS over TLS, RFC 7858, conventionally port 853. SchemeTLS = "tls" // SchemeHTTPS is DNS over HTTPS, RFC 8484. The address is a full URL rather // than a host and port, because the path is part of the endpoint. SchemeHTTPS = "https" )
Endpoint schemes. The scheme is how a preset says which wire an address is reached over, because an address alone does not say it: 9.9.9.9 is a datagram upstream, a stream upstream and, at a different port, a TLS upstream, and the three have entirely different privacy properties.
Variables ¶
var ( // ErrNoProviders reports a group asked to resolve with nothing to resolve // through. It cannot arise from a validated [GroupOptions]; it exists so // that a zero [Group] reached through a nil-safe path fails with a sentence // rather than a nil dereference. ErrNoProviders = errors.New("providers: group has no providers") // ErrNilQuery reports Exchange called with no question. It is a caller bug // rather than a network condition, and naming it keeps that distinction // visible in a log full of timeouts. ErrNilQuery = errors.New("providers: nil query") // ErrNoReply reports an upstream returning neither a message nor an error, // which is a broken [transport.Exchanger] rather than a broken upstream. ErrNoReply = errors.New("providers: upstream returned no reply and no error") )
Errors a Group returns in its own right. Failures attributable to an upstream arrive wrapped in a ProviderError instead.
var ( // ErrUnknownScheme reports an endpoint whose scheme is not one this engine // can carry queries over. It is a distinct error because "https" misspelt // and "quic" unimplemented are the same mistake to a configuration file and // different mistakes to the operator making it. ErrUnknownScheme = errors.New("providers: unsupported endpoint scheme") // ErrBadEndpoint reports an endpoint whose scheme was recognised but whose // address is not usable — empty, a URL where a host and port belongs, or an // IPv6 literal written without the brackets that separate it from a port. ErrBadEndpoint = errors.New("providers: malformed endpoint") // ErrNoEndpoint reports that a preset publishes nothing matching the schemes // the caller asked for. It is an error rather than a fallback on purpose: // an operator who asked for tls and silently received cleartext udp would // have exactly the privacy they were trying to buy, and no way to notice. ErrNoEndpoint = errors.New("providers: no endpoint matches the preferred schemes") // ErrDuplicatePreset reports a name already present in a registry. // Registration refuses rather than overwrites so that adding a preset can // never quietly redirect an existing name to a different operator's // servers; [Registry.Replace] is the way to say you meant it. ErrDuplicatePreset = errors.New("providers: preset name already registered") )
Errors this file returns. Test with errors.Is.
Functions ¶
func ParseEndpoint ¶
ParseEndpoint splits an endpoint string into the scheme that selects a transport and the address that transport wants.
The two are not the same shape, and the asymmetry is the point of having this function rather than a strings.Cut at every call site: for udp, tcp and tls the address is a bare host and port with the scheme removed, because that is what transport.Options.Addr means for those transports; for https the address is the WHOLE URL, scheme included, because a DoH endpoint is identified by its path as much as its host and transport.DoHOptions takes a URL.
The port may be omitted, in which case each transport applies its own default — 53, 53, 853. An IPv6 literal must be bracketed when it carries a port, and may be bracketed when it does not. That requirement is not a formality: "2620:fe::fe:53" is itself a valid IPv6 address, so it is read as one and the intended port is lost with nothing to warn about. Write "[2620:fe::fe]:53".
Types ¶
type Allower ¶
type Allower interface {
// Allow reports whether a query may be sent, claiming a probe if the
// provider is half-open. It is called once per attempt, immediately before
// the attempt.
Allow() bool
}
Allower is the optional interface a Provider implements when it can gate its own traffic — in practice, when it can claim the single probe a half-open circuit permits.
A Group uses it in preference to reading Provider.Health, and the difference matters at exactly one moment: when a tripped provider's cooldown expires, every goroutine that reads the snapshot sees "available" at once and an outage is met with a burst. Health.Allow hands that one probe to one caller. A provider that does not implement this is gated on the snapshot, which is correct but less careful.
type BuildOptions ¶
type BuildOptions struct {
// Prefer lists schemes in descending preference, as in
// {SchemeTLS, SchemeHTTPS}. Empty takes the preset's own first endpoint.
//
// If none of the listed schemes is published, [Build] fails with
// [ErrNoEndpoint] rather than falling back to one that was not asked for.
// An operator who wrote tls and was quietly given cleartext udp has lost
// the only thing they were configuring, and no log line would say so.
Prefer []string `json:"prefer"`
// Name overrides the provider name. Empty uses the preset's name. It is
// here for the deployment running the same upstream twice with different
// settings, which would otherwise produce two identically named providers
// and one uninterpretable metric.
Name string `json:"name"`
// Dialer opens connections. Nil selects a [net.Dialer].
//
// Setting it is how an embedder routes DNS through a VPN interface, a proxy
// or a bootstrap resolver — which the tls:// and https:// endpoints need,
// since their addresses are host names that themselves have to be resolved.
Dialer transport.Dialer `json:"-"`
// Timeout bounds one exchange. Zero selects [transport.DefaultTimeout], and
// it never overrules an earlier deadline on the caller's context.
Timeout time.Duration `json:"timeout"`
// IdleTimeout is how long an unused stream connection is held. Zero selects
// [transport.DefaultIdleTimeout]. Ignored by the datagram leg, which holds
// nothing.
IdleTimeout time.Duration `json:"idle_timeout"`
// MaxConns bounds concurrent connections to the upstream. Zero selects
// [transport.DefaultMaxConns].
MaxConns int `json:"max_conns"`
// Randomize enables DNS-0x20 case randomisation. Worth having on the
// cleartext schemes, where a forged reply is a real threat, and close to
// pointless on tls and https, where TLS has already authenticated the peer.
Randomize bool `json:"randomize"`
// UDPSize is the EDNS payload size advertised on udp:// endpoints. Zero
// selects [transport.DefaultUDPSize].
UDPSize uint16 `json:"udp_size"`
// TLS configures tls:// and https:// endpoints. Nil takes the platform
// roots with a TLS 1.2 floor, which is what a public resolver expects.
TLS *tls.Config `json:"-"`
// PinnedSPKI overrides [Preset.PinnedSPKI]. It applies to tls:// endpoints
// only, and is silently unused on the others, because a preset that pins
// its DoT endpoint is still allowed to publish a udp:// one.
PinnedSPKI []string `json:"pinned_spki,omitempty"`
// Health configures the circuit breaker on the resulting provider. The zero
// value selects the documented defaults; set its Clock in tests.
Health HealthOptions `json:"health"`
}
BuildOptions configure turning a Preset into a live Provider.
The fields are the settings that belong to the deployment rather than to the upstream: how connections are made, how long to wait, how hard to try. The address, the transport and the operator's own policy come from the preset.
type Concurrent ¶
type Concurrent interface {
Strategy
// Concurrency reports how many of the selected providers to query at once.
// One means sequential failover, which is what a strategy that does not
// implement this interface is assumed to want.
Concurrency() int
}
Concurrent is the optional interface a Strategy implements when the providers it selects are meant to be queried at the SAME TIME rather than one after another.
This is an interface on the strategy rather than a field on a per-call plan for two reasons. Strategy.Select returns a slice, and adding a plan struct to carry one integer would either allocate per query or widen the signature every future policy has to implement. And concurrency is a property of the policy, not of a particular selection: Race is concurrent for every query it will ever answer, so a group can ask once when it is built and never again on the query path.
type Group ¶
type Group struct {
// contains filtered or unexported fields
}
Group is several providers and a policy for choosing between them, presented as one provider.
It implements Provider rather than something narrower for one reason: GROUPS NEST. A Race group of two fast resolvers can be the first entry of a Sequential group whose second entry is a slow-but-reliable upstream, and the outer group neither knows nor needs to know that its first "provider" is three sockets and a policy. That composition is how the two properties a resolver wants — the tail latency of a race and the depth of a fallback chain — are had at once, and it costs nothing structurally because the inner group already has a name, a health record and an Exchange method.
A Group is safe for concurrent use and holds no sockets of its own.
Example (Nested) ¶
ExampleGroup_nested shows why providers.Group implements providers.Provider: groups nest, and that is how a resolver gets the tail latency of a race and the depth of a fallback chain at the same time.
The inner group races two fast upstreams. The outer group treats that whole arrangement as a single preferred provider, with a slow-but-reliable upstream behind it — and neither group needs to know what the other is.
package main
import (
"context"
"fmt"
"log"
"net/netip"
"github.com/daboss2003/dns/dnsmsg"
"github.com/daboss2003/dns/providers"
"github.com/daboss2003/dns/transport"
)
// stubExchanger answers every query the same way, which is all an example needs
// from an upstream.
type stubExchanger struct {
reply *dnsmsg.Message
err error
}
func (s stubExchanger) Exchange(_ context.Context, q *dnsmsg.Message) (*dnsmsg.Message, error) {
if s.err != nil {
return nil, s.err
}
reply := *s.reply
reply.SetReply(q)
reply.SetRCode(s.reply.RCode)
return &reply, nil
}
func (stubExchanger) Close() error { return nil }
// answering returns an upstream that resolves everything to one address.
func answering(addr string) transport.Exchanger {
var m dnsmsg.Message
m.Answers = append(m.Answers, dnsmsg.RR{
Name: dnsmsg.MustParseName("example.com."),
Type: dnsmsg.TypeA,
Class: dnsmsg.ClassINET,
TTL: 300,
Data: &dnsmsg.A{Addr: netip.MustParseAddr(addr)},
})
return stubExchanger{reply: &m}
}
// mustProvider builds a provider or gives up, which an example may do and a
// resolver may not: [providers.New] fails only on a configuration mistake, and
// the point of these examples is what happens after construction.
func mustProvider(name string, ex transport.Exchanger) providers.Provider {
p, err := providers.New(providers.Options{Name: name, Exchanger: ex})
if err != nil {
log.Fatal(err)
}
return p
}
func query(name string) *dnsmsg.Message {
var m dnsmsg.Message
m.SetQuestion(dnsmsg.MustParseName(name), dnsmsg.TypeA, dnsmsg.ClassINET)
return &m
}
func main() {
fastA := mustProvider("fast-a", answering("192.0.2.1"))
fastB := mustProvider("fast-b", answering("192.0.2.2"))
reliable := mustProvider("reliable", answering("192.0.2.3"))
fast, err := providers.NewGroup(providers.GroupOptions{
Name: "fast",
Providers: []providers.Provider{fastA, fastB},
Strategy: providers.Race(2),
})
if err != nil {
log.Fatal(err)
}
// The outer group's first "provider" is three sockets and a policy.
outer, err := providers.NewGroup(providers.GroupOptions{
Name: "upstreams",
Providers: []providers.Provider{fast, reliable},
Strategy: providers.Sequential(),
})
if err != nil {
log.Fatal(err)
}
defer outer.Close()
if _, err := outer.Exchange(context.Background(), query("example.com.")); err != nil {
log.Fatal(err)
}
for _, p := range outer.Providers() {
fmt.Printf("%s: %s\n", p.Name(), p.Health().State)
}
}
Output: fast: closed reliable: closed
func NewGroup ¶
func NewGroup(opts GroupOptions) (*Group, error)
NewGroup returns a group over opts.Providers.
The providers are not adopted in the sense of ownership — the caller built them and may have put the same provider in two groups — but Group.Close does close them, because a group is a transport.Exchanger and closing one has to release what it uses. Closing is idempotent at both levels, so a shared provider closed twice is not an error.
Example ¶
ExampleNewGroup builds the ordinary case: a preferred upstream, a fallback, and one query that has to use the fallback because the first one SERVFAILs.
package main
import (
"context"
"fmt"
"log"
"net/netip"
"github.com/daboss2003/dns/dnsmsg"
"github.com/daboss2003/dns/providers"
"github.com/daboss2003/dns/transport"
)
// stubExchanger answers every query the same way, which is all an example needs
// from an upstream.
type stubExchanger struct {
reply *dnsmsg.Message
err error
}
func (s stubExchanger) Exchange(_ context.Context, q *dnsmsg.Message) (*dnsmsg.Message, error) {
if s.err != nil {
return nil, s.err
}
reply := *s.reply
reply.SetReply(q)
reply.SetRCode(s.reply.RCode)
return &reply, nil
}
func (stubExchanger) Close() error { return nil }
// answering returns an upstream that resolves everything to one address.
func answering(addr string) transport.Exchanger {
var m dnsmsg.Message
m.Answers = append(m.Answers, dnsmsg.RR{
Name: dnsmsg.MustParseName("example.com."),
Type: dnsmsg.TypeA,
Class: dnsmsg.ClassINET,
TTL: 300,
Data: &dnsmsg.A{Addr: netip.MustParseAddr(addr)},
})
return stubExchanger{reply: &m}
}
// failingWith returns an upstream that answers with an RCODE and no records.
func failingWith(rc dnsmsg.RCode) transport.Exchanger {
var m dnsmsg.Message
m.SetRCode(rc)
return stubExchanger{reply: &m}
}
func query(name string) *dnsmsg.Message {
var m dnsmsg.Message
m.SetQuestion(dnsmsg.MustParseName(name), dnsmsg.TypeA, dnsmsg.ClassINET)
return &m
}
func main() {
primary, err := providers.New(providers.Options{
Name: "primary",
Exchanger: failingWith(dnsmsg.RCodeServerFailure),
})
if err != nil {
log.Fatal(err)
}
fallback, err := providers.New(providers.Options{
Name: "fallback",
Exchanger: answering("192.0.2.1"),
})
if err != nil {
log.Fatal(err)
}
// Sequential reads the list as what it looks like: a preference order.
g, err := providers.NewGroup(providers.GroupOptions{
Name: "upstreams",
Providers: []providers.Provider{primary, fallback},
Strategy: providers.Sequential(),
})
if err != nil {
log.Fatal(err)
}
defer g.Close()
reply, err := g.Exchange(context.Background(), query("example.com."))
if err != nil {
log.Fatal(err)
}
fmt.Println("rcode:", reply.RCode)
fmt.Println("primary failures:", primary.Health().Failures)
fmt.Println("fallback successes:", fallback.Health().Successes)
}
Output: rcode: NOERROR primary failures: 1 fallback successes: 1
func (*Group) Allow ¶
Allow implements Allower for the group as a whole, so that a group nested inside another is gated exactly as a single provider is.
It is the claim half of Group.Health: an outer group asks this before sending, and a tripped inner group hands out its one probe to one query rather than letting every goroutine in the process discover the cooldown expired at the same moment. Nesting is the point of Group implementing Provider, and a nested group that could not be gated would be the one kind of provider whose recovery arrived as a thundering herd.
func (*Group) Close ¶
Close implements transport.Exchanger, closing every provider once.
The errors are joined rather than discarded or shortened to the first: a resolver shutting down wants to know that one of its six upstreams could not be closed cleanly, and which. Closing is idempotent, so a provider shared between two groups survives both of them closing it.
func (*Group) Exchange ¶
Exchange implements transport.Exchanger, trying upstreams until one answers.
The query is treated as read-only and may be handed to several providers at once; the transports in this module copy it before assigning a message ID, so racing on one message is safe. A provider that mutated the caller's query would be broken for retries too, not only for races.
func (*Group) Health ¶
func (g *Group) Health() HealthState
Health implements Provider, reporting the GROUP's standing rather than any member's.
It trips when queries stop being answerable by anyone in the group, which is the only fact a containing group can act on: an outer group cannot do anything useful with "two of your five upstreams are down" except keep sending, and that is what a closed circuit already says.
func (*Group) Providers ¶
Providers returns the group's upstreams in configuration order.
The slice is a copy, so a caller inspecting a running resolver cannot reorder the list a query is about to be selected from. The providers themselves are shared: their health records are the live ones, which is the point of asking.
type GroupOptions ¶
type GroupOptions struct {
// Name identifies the group in metrics, logs and in any group that contains
// it. Required, for the same reason a provider's name is.
Name string `json:"name"`
// Providers are the upstreams, in preference order. At least one is
// required. The group does not copy the providers themselves — they are
// live objects with health records that outlive any one query — but it does
// copy the slice, so the caller may reuse it.
Providers []Provider `json:"-"`
// Strategy chooses between them. Nil selects [Sequential], which reads the
// list as what it looks like: a preference order.
Strategy Strategy `json:"-"`
// Clock is the time source used to measure attempts. Nil selects
// [clock.System].
Clock clock.Clock `json:"-"`
// Metrics records per-attempt upstream timings. Nil selects [metrics.Nop].
Metrics metrics.Recorder `json:"-"`
// Events publishes upstream failures and recoveries. Nil publishes nothing,
// and so does a bus with no subscriber: the group asks before it builds an
// event, because converting one to an interface allocates whether or not
// anybody is listening.
Events *events.Bus `json:"-"`
// Logger receives failover decisions at debug level and total failures at
// warn. Nil selects [logging.Discard].
Logger *slog.Logger `json:"-"`
// MaxAttempts bounds how many upstreams one query may be sent to. Zero
// means the number of providers, which is the natural reading of "try them
// all".
//
// It exists because failover multiplies latency as well as reliability: a
// group of six providers with a two-second timeout each can spend twelve
// seconds on a query whose client gave up after five. Setting this to two
// says that if the first two upstreams cannot answer, the sixth almost
// certainly cannot either, and the client would rather have SERVFAIL now.
MaxAttempts int `json:"max_attempts"`
}
GroupOptions configure a Group.
The fields carrying live objects — providers, strategy, clock, recorder, bus, logger — are marked `json:"-"`: they are wired in code by whoever owns them. What remains in JSON is what an operator sets in a configuration file, and a configuration loader is expected to fill the rest in.
func (GroupOptions) Validate ¶
func (o GroupOptions) Validate() error
Validate reports every problem with o, not merely the first, so an operator fixing a configuration file sees the whole list in one pass.
type Health ¶
type Health struct {
// contains filtered or unexported fields
}
Health is a circuit breaker over one provider.
It is safe for concurrent use: every provider in a busy server records outcomes from many goroutines at once.
func NewHealth ¶
func NewHealth(opts HealthOptions) (*Health, error)
NewHealth returns a breaker configured by opts.
func (*Health) Allow ¶
Allow reports whether a query may be sent, and is the call a Strategy makes.
Unlike Health.State it has a side effect, and that is the point: the single probe a half-open circuit permits has to be claimed by exactly one caller, or every goroutine arriving in that instant sends one and the outage is met with a burst rather than a question.
func (*Health) Reset ¶
func (h *Health) Reset()
Reset returns the breaker to its initial state, discarding the failure run and the backoff. It exists for an operator forcing a provider back into service and for tests; lifetime totals are preserved so that metrics remain monotonic.
func (*Health) State ¶
func (h *Health) State() HealthState
State returns a snapshot, moving an expired open circuit to half-open first.
The transition happens on read rather than on a timer because a breaker with no traffic does not need to change state: nothing is waiting to hear about it, and a timer per provider would be a goroutine per provider to schedule a transition nobody is watching.
type HealthOptions ¶
type HealthOptions struct {
// FailureThreshold is the number of CONSECUTIVE failures that trip the
// breaker. Consecutive rather than a rate: a provider answering four
// queries in five is degraded but useful, and taking it out of service
// would move that load onto its neighbours.
FailureThreshold int `json:"failure_threshold"`
// Cooldown is the first interval a tripped provider is passed over for.
Cooldown time.Duration `json:"cooldown"`
// MaxCooldown caps the exponential growth of Cooldown across repeated
// failed probes.
MaxCooldown time.Duration `json:"max_cooldown"`
// Clock is the time source. Nil selects [clock.System].
Clock clock.Clock `json:"-"`
}
HealthOptions configure the circuit breaker. The zero value selects the defaults above.
func (HealthOptions) Validate ¶
func (o HealthOptions) Validate() error
Validate reports every problem it finds.
type HealthState ¶
type HealthState struct {
State State `json:"state"`
// ConsecutiveFailures is the current run of failures; it resets on any
// success.
ConsecutiveFailures int `json:"consecutive_failures"`
// Successes and Failures are lifetime totals, for metrics.
Successes uint64 `json:"successes"`
Failures uint64 `json:"failures"`
// OpenedAt is when the breaker last tripped, zero if it never has.
OpenedAt time.Time `json:"opened_at,omitzero"`
// RetryAt is when the next probe becomes permitted, meaningful only while
// the state is open.
RetryAt time.Time `json:"retry_at,omitzero"`
// Cooldown is the interval currently in force, which grows with repeated
// failed probes.
Cooldown time.Duration `json:"cooldown"`
}
HealthState is an immutable snapshot of a provider's standing.
func (HealthState) Available ¶
func (s HealthState) Available() bool
Available reports whether a query may be sent, which is what a Strategy asks. A half-open provider is available for exactly the one probe that state exists to permit.
type Options ¶
type Options struct {
// Name identifies the provider. Required: an unnamed upstream cannot be
// reported on, and a group of them cannot be debugged.
Name string `json:"name"`
// Exchanger carries the queries. Required.
Exchanger transport.Exchanger `json:"-"`
// Health configures the circuit breaker. The zero value selects the
// documented defaults.
Health HealthOptions `json:"health"`
}
Options configure a Provider built from a transport.
type Outcome ¶
type Outcome uint8
Outcome is how an exchange ended, from the point of view of failover.
It exists because the useful question is not "did this return an error" but "is another upstream worth asking", and the two differ in both directions: a SERVFAIL is a well-formed response that should be retried elsewhere, and an NXDOMAIN is a real answer that should not.
const ( // OutcomeAnswered is a usable response, including NXDOMAIN and NODATA. The // provider did its job. OutcomeAnswered Outcome = iota // OutcomeFailed is a failure attributable to this upstream: a transport // error, a timeout, a forged or malformed reply, SERVFAIL or REFUSED. Worth // asking someone else, and counted against this provider's health. OutcomeFailed // OutcomeAborted is the caller giving up — a cancelled context or an // expired deadline that came from above rather than from the upstream. // It is NOT counted against health: punishing a provider because the client // hung up would take healthy upstreams out of service during a load spike, // which is exactly when they are needed. OutcomeAborted )
Exchange outcomes.
func Classify ¶
Classify decides what an exchange result means for failover.
It is the single place the rule lives, because the rule is easy to state and easy to get subtly wrong: a response is a failure when the upstream could not answer, not when the answer is unwelcome. See the package documentation for why NXDOMAIN must not be treated as a failure.
Example ¶
ExampleClassify shows the distinction the whole package turns on.
The useful question is not "did this return an error" but "is another upstream worth asking", and the two differ in both directions. Getting this wrong is silent: a resolver that fails over on NXDOMAIN still returns the right answer, having asked every provider about every typo to get it.
package main
import (
"context"
"errors"
"fmt"
"github.com/daboss2003/dns/dnsmsg"
"github.com/daboss2003/dns/providers"
)
func main() {
var timeout = errors.New("i/o timeout")
for _, c := range []struct {
what string
reply *dnsmsg.Message
err error
}{
{"NOERROR with an answer", &dnsmsg.Message{}, nil},
{"NXDOMAIN", rcode(dnsmsg.RCodeNameError), nil},
{"SERVFAIL", rcode(dnsmsg.RCodeServerFailure), nil},
{"REFUSED", rcode(dnsmsg.RCodeRefused), nil},
{"a transport timeout", nil, timeout},
{"the caller cancelling", nil, context.Canceled},
} {
fmt.Printf("%-22s %s\n", c.what, providers.Classify(c.reply, c.err))
}
}
func rcode(rc dnsmsg.RCode) *dnsmsg.Message {
var m dnsmsg.Message
m.SetRCode(rc)
return &m
}
Output: NOERROR with an answer answered NXDOMAIN answered SERVFAIL failed REFUSED failed a transport timeout failed the caller cancelling aborted
func ClassifyWithContext ¶
ClassifyWithContext is Classify with the caller's context taken into account, so that a deadline reached because the CALLER ran out of time is not charged to the upstream.
Prefer it wherever the context is at hand. A group that charges its providers for its own impatience will trip every breaker it owns during a load spike.
type Preset ¶
type Preset struct {
// Name is how an operator refers to this upstream in configuration, and
// becomes the [Provider] name in metrics and logs. It is matched
// case-insensitively; write it lower case.
Name string `json:"name"`
// Description is one line an operator sees in a list of choices.
Description string `json:"description"`
// Endpoints are the operator's published addresses in the preset author's
// preference order, each prefixed with the scheme that selects a transport:
// udp://, tcp://, tls:// or https://. See [ParseEndpoint].
//
// Several addresses of the same scheme are the redundancy the operator
// publishes — usually a primary, a secondary and their IPv6 equivalents.
// [Build] uses one of them; building a [Provider] per address and grouping
// them is what uses them all, and is the caller's decision because it
// doubles the query volume seen by that one operator.
Endpoints []string `json:"endpoints"`
// PinnedSPKI are base64 SHA-256 SPKI digests for the tls:// endpoints,
// carried through to [transport.StreamOptions.PinnedSPKI].
//
// It is empty for every preset shipped here, and that is a deliberate
// finding rather than an oversight: none of these operators publishes a
// long-lived pin, and a pin baked into a released library outlives the
// certificate it was taken from. When it expires, every query to that
// upstream fails at the handshake — a total outage caused by the library
// rather than by the operator. Pin an upstream you control, from a value
// you can update as fast as it rotates.
PinnedSPKI []string `json:"pinned_spki,omitempty"`
// Notes are what an operator should read before choosing this upstream:
// filtering behaviour, stated logging policy, jurisdiction. Factual, and
// not a recommendation — see the type documentation.
Notes string `json:"notes"`
}
Preset is a named upstream operator's published endpoints, as data.
The rule this package follows is that the resolver never knows the name of a provider. A preset is a row in a Registry, not a case in a switch, so a deployment can add its own upstream, or replace one of these, without patching this package and without a release of it.
A preset is a convenience, not an endorsement ¶
This is worth being blunt about, because a list of well-known resolvers shipped inside a DNS engine reads as a recommendation and is not one. Every entry here is a company or a foundation that will see every name the resolver asks for. Which of them is acceptable depends on who is deploying, under what law, against what threat — and that is a decision the operator has to make, not one a library can make on their behalf. The addresses are here because typing them from memory is how a resolver ends up pointed at an address that belongs to somebody else; the choice is still the operator's.
Preset.Notes exists to make that choice an informed one: who operates the service, what it filters, what it says it logs, and under whose jurisdiction. It is deliberately factual and deliberately not a ranking. Providers change their policies, and the notes here are a starting point for reading the operator's own current documentation rather than a substitute for it.
type Provider ¶
type Provider interface {
transport.Exchanger
// Name identifies the provider in metrics, logs and configuration. It is
// stable for the life of the provider and is what an operator sees.
Name() string
// Health reports the provider's current standing. The result is a snapshot;
// reading it does not block a query in flight.
Health() HealthState
}
Provider is one named upstream resolver.
It is a transport.Exchanger with an identity and a health record, which is what lets a group reason about it: metrics and logs need a name, and failover needs to know whether asking again is worth a query.
func Build ¶
func Build(p Preset, opts BuildOptions) (Provider, error)
Build turns a preset into a live Provider over one of its endpoints.
One endpoint, not all of them: an operator publishing four addresses is publishing redundancy within their own service, and deciding to use all four means sending that one operator four times the connections. Callers who want that range Preset.Endpoints and call BuildEndpoint per address, then put the results in a group.
Choosing udp:// yields UDP paired with TCP to the same address rather than bare UDP, because a truncated reply is not an answer — it is the upstream saying "ask again over a stream", and a provider that could not do so would fail every query with a large response. See transport.NewAutoUDPTCP.
Nothing is dialled here. The transports connect on their first exchange, so building providers for a dozen configured upstreams costs a dozen structs and no handshakes.
func BuildEndpoint ¶
func BuildEndpoint(name, endpoint string, opts BuildOptions) (Provider, error)
BuildEndpoint builds one Provider from one endpoint string, without a preset in between.
It is what Build is made of, and it is exported because the endpoint an operator wants is often not in any registry: a configuration file saying "tls://dns.example:853" deserves the same construction path as a named preset, and a caller building a provider per address of one preset needs to name each of them separately anyway.
BuildOptions.Prefer is ignored: the endpoint has already been chosen.
type ProviderError ¶
type ProviderError struct {
// Provider is the name of the upstream that failed.
Provider string
// Err is what went wrong, unwrappable so that errors.Is on a transport
// sentinel still works through the join.
Err error
}
ProviderError attributes a failure to the upstream that produced it.
A group that has exhausted its providers returns every failure joined together, and a joined error is only useful if each part says whose failure it was: "all four upstreams timed out" is an operational fact, while "i/o timeout" repeated four times is a puzzle.
func (*ProviderError) Error ¶
func (e *ProviderError) Error() string
Error implements the error interface.
func (*ProviderError) Unwrap ¶
func (e *ProviderError) Unwrap() error
Unwrap returns the underlying failure.
type RCodeError ¶
type RCodeError struct {
// Provider names the upstream that returned the code.
Provider string
// RCode is the response code received.
RCode dnsmsg.RCode
}
RCodeError reports an upstream that answered, in well-formed DNS, that it could not answer.
SERVFAIL, REFUSED and NOTIMP are failures with no error value of their own, so this manufactures one. It is a distinct type rather than a formatted string because "every upstream returned REFUSED" and "every upstream timed out" call for different actions — the first is a configuration or an entitlement problem and will not fix itself — and an operator should be able to tell them apart with errors.As rather than by reading prose.
func (*RCodeError) Error ¶
func (e *RCodeError) Error() string
Error implements the error interface.
type RandSource ¶
type RandSource interface {
// IntN returns a uniformly distributed value in [0,n) and panics for n
// less than one, matching [math/rand/v2.IntN].
IntN(n int) int
}
RandSource is the randomness Random draws from.
It is injected because a strategy whose output cannot be predicted cannot be tested: a test that asserts "the second provider was asked" needs to know which provider that is, and a test that asserts a distribution needs to run the same sequence twice. The interface is one method wide so that a test satisfies it with a counter, and *math/rand/v2.Rand satisfies it already.
Implementations must be safe for concurrent use. Selection happens on every query from every goroutine answering one, and a source guarded by nothing will be found by the race detector immediately.
func CryptoRand ¶
func CryptoRand() RandSource
CryptoRand returns a RandSource backed by crypto/rand.
Choosing an upstream is not a security decision — the sixteen bits an off-path attacker has to guess live in the message ID, which the transport randomises with crypto/rand already — so this is not the default and costs noticeably more than one. It is here for deployments whose threat model says that which resolver saw which name must not be predictable from outside, and for auditors who would rather not have to make that argument.
type Registry ¶
type Registry struct {
// contains filtered or unexported fields
}
Registry maps names to Preset values.
It is the seam that keeps "never hardcode a provider" true in practice: the resolver looks a name up, a deployment registers whatever it likes, and a third party ships an upstream without this package knowing it exists.
The zero Registry is valid and empty, so a Registry declared as a field of a configuration struct works without remembering NewRegistry. Lookups take a read lock and copy, which makes this a configuration-time object rather than a per-query one: resolve the presets you need into providers once at startup and hold the providers.
A Registry is safe for concurrent use.
Example ¶
ExampleRegistry shows presets as what they are: data an operator names, not a switch statement inside the resolver. A deployment registers its own upstream the same way this package registers the ones it ships, and can replace any of them without patching the package.
package main
import (
"errors"
"fmt"
"log"
"github.com/daboss2003/dns/providers"
)
func main() {
r := providers.NewRegistry()
if err := r.Register(providers.Preset{
Name: "corp",
Description: "the resolver on the office network",
Endpoints: []string{"tls://dns.corp.example:853", "udp://198.51.100.53:53"},
Notes: "operated by us; logs queries for 24h; UK jurisdiction",
}); err != nil {
log.Fatal(err)
}
// Registering the same name twice is refused rather than silently winning:
// a preset is an address queries will be sent to.
err := r.Register(providers.Preset{Name: "CORP", Endpoints: []string{"udp://203.0.113.1:53"}})
fmt.Println("duplicate:", errors.Is(err, providers.ErrDuplicatePreset))
// Lookup is case-insensitive, because the name came from a file a human
// typed.
p, ok := r.Lookup(" Corp ")
fmt.Println("found:", ok, p.Endpoints[0])
scheme, addr, err := providers.ParseEndpoint(p.Endpoints[0])
if err != nil {
log.Fatal(err)
}
fmt.Printf("scheme=%s addr=%s\n", scheme, addr)
}
Output: duplicate: true found: true tls://dns.corp.example:853 scheme=tls addr=dns.corp.example:853
func DefaultRegistry ¶
func DefaultRegistry() *Registry
DefaultRegistry returns the process-wide registry seeded with the presets this package ships. It is built on first use rather than at package initialisation, so a program that never names a preset never pays for one.
The presets in it are a convenience and not an endorsement; see Preset for what that means and why Preset.Notes is there. Callers adding their own upstreams should prefer Clone — the returned registry is shared with every other user of the package, and registering into it affects all of them.
func NewRegistry ¶
func NewRegistry() *Registry
NewRegistry returns an empty Registry. Call Registry.RegisterBuiltins to seed it with the presets this package ships, or leave it empty for a deployment that wants only upstreams it has named itself.
func (*Registry) Clone ¶
Clone returns an independent copy, which is the safe way to add a preset to the built-in set without mutating a registry other code is reading — most of all DefaultRegistry, which is process-wide.
func (*Registry) Lookup ¶
Lookup returns the preset registered under name, matched case-insensitively and ignoring surrounding whitespace — because the name arrives from a configuration file a human typed, and "Cloudflare" is not a different upstream from "cloudflare".
The returned Preset shares no memory with the registry, so a caller may edit the copy freely.
func (*Registry) Names ¶
Names returns the registered names, sorted.
Sorted rather than in map order because this is what an operator sees from `--list-presets` and what a configuration dump writes, and an ordering that changes between runs makes both of those unreadable and undiffable. The names are the canonical lower-case keys, which is exactly what Registry.Lookup accepts.
func (*Registry) Register ¶
Register adds p under its name, refusing a name already present.
Refusing rather than replacing is the safe default here in a way it is not for a decoder table. A preset is an address queries will be sent to, so a second registration silently winning would point a resolver at a different operator's servers while the configuration file still said "quad9". A caller that means to override says so with Registry.Replace.
func (*Registry) RegisterBuiltins ¶
func (r *Registry) RegisterBuiltins()
RegisterBuiltins adds every preset this package ships to r, replacing any entry of the same name. It is the documented seeding function: a caller who wants the built-in set plus their own starts here rather than reaching for a package-level variable.
func (*Registry) Replace ¶
Replace registers p, overwriting any preset already under its name.
It exists for the deployment that wants "cloudflare" to mean its own on-premises forwarder, or that is correcting an address this package got wrong without waiting for a release. Prefer Registry.Register everywhere else, so that a genuine name collision is reported rather than resolved by whichever call happened to run second.
type SelectAppender ¶
type SelectAppender interface {
Strategy
// SelectAppend appends the selected providers to dst and returns the
// result. dst may be nil.
SelectAppend(dst []Provider, ps []Provider) []Provider
}
SelectAppender is the optional interface a Strategy implements when it can write its selection into a buffer the caller owns.
Selection happens once per query, so at ten thousand queries a second a Strategy.Select that allocates a slice per call allocates ten thousand slices a second for a value that dies microseconds later. A group keeps a pool of buffers and offers one through this interface; the same strategy still satisfies Strategy for callers who do not care.
Implementations must append to dst and return the extended slice, exactly as the built-in append does, and must not assume anything about dst's contents, length or capacity.
type State ¶
type State uint8
State is a circuit breaker state.
const ( // StateClosed is the healthy state: queries flow normally. StateClosed State = iota // StateOpen means the provider has failed enough consecutive times to be // passed over. It is not asked again until the cooldown expires. StateOpen // StateHalfOpen means the cooldown has expired and exactly one query is // allowed through to find out whether the outage is over. StateHalfOpen )
Circuit states.
type Strategy ¶
type Strategy interface {
// Select returns the providers to try, in order. It must not block and
// must not mutate the input slice.
//
// The result is filtered to providers that are currently worth asking,
// judged by [HealthState.Available]. It may legitimately be empty — every
// upstream tripped is a state a group must handle, and a strategy that
// invented a provider to avoid returning nothing would be hiding the
// outage from the one component able to make a judgement about it.
Select(ps []Provider) []Provider
// Name identifies the policy in logs, metrics and configuration. It is
// what an operator reads when asking why a particular upstream was chosen.
Name() string
}
Strategy is the policy a Group uses to decide which upstreams to ask, and in what order.
It is deliberately the smallest thing that can express the choice: given the group's providers, hand back the ones worth asking. It holds no reference to the group, sends nothing itself and learns nothing from the result, which is what lets one strategy value describe a policy that a test can exercise without a network, a clock or a query.
func Race ¶
Race returns the strategy that asks n providers simultaneously and takes the first good answer.
It is the only strategy under which one slow upstream costs nothing: the query's latency is the fastest of the n rather than the first one's, and a provider that has stopped answering without failing — the case a circuit breaker is slowest to catch — stops mattering immediately rather than after a timeout. That is the best tail latency available from a resolver.
It is also the most expensive thing in the package, in two currencies. It multiplies query volume by n against every upstream raced, and it sends every name to all n of them, so a group that races a public resolver has told that operator about every query whether or not it used the answer. Racing is a privacy decision before it is a performance one.
A raced group does not fail over past its own width: the n it selected are the n it asks. Sequential depth and concurrent speed compose by nesting instead — a Race group of two fast resolvers as the first entry of a Sequential group whose second entry is the slow reliable one gives the fast path a race and the failure path a fallback, which is exactly the shape Group implementing Provider exists to allow.
func Random ¶
func Random(src RandSource) Strategy
Random returns the strategy that puts the available providers in a random order on every query.
It spreads load like RoundRobin without a shared counter, and spreads FAILOVER load too: under round robin every query that starts at provider one falls back to provider two, so an outage moves a predictable block of traffic onto a predictable neighbour. Random spreads that second wave as well, which is the reason to prefer it when the concern is a thundering herd rather than an even long-run split.
A nil src selects the math/rand/v2 global generator, which is safe for concurrent use, seeded per process and costs a couple of nanoseconds. Pass CryptoRand for unpredictability, or a fake for a deterministic test; see RandSource for why the seam is here at all.
The shuffle is Fisher-Yates over the available providers: one draw each, no allocation when the caller supplies a buffer.
func RoundRobin ¶
func RoundRobin() Strategy
RoundRobin returns the strategy that spreads queries evenly by starting each one at a different provider and wrapping around.
It is for equivalent upstreams, where the aim is not to depend on any single one of them: three public resolvers that answer the same questions, or two instances of the same service. Failover order is preserved after the starting point, so a query that begins at the third provider tries the first next.
Coordination is one atomic increment, which is what makes it usable in a server with a goroutine per query and no scheduler to consult. The counter is per strategy value, so each call to RoundRobin returns an independent rotation and one value must not be shared between groups that should not share a position in the sequence.
func Sequential ¶
func Sequential() Strategy
Sequential returns the strategy that asks providers in configuration order, moving to the next only when one fails.
It is the right default and the right answer whenever the upstreams are not equivalent — a resolver on the local network followed by a public one, a paid filtering service followed by a free fallback. Order is a preference, and this is the only strategy that honours it. The cost is that a query which fails over pays both latencies in series, which is what Race exists to avoid and what a group of two nested strategies can have both ways.
func Weighted ¶
Weighted returns the strategy that gives each provider a share of the queries proportional to its weight.
It is for upstreams that are interchangeable but not equal: a resolver on the local network that should take most of the traffic and a public one that should take a little to stay warm, or a migration that moves load from one service to another a tenth at a time. Weights are positions in the list, so weights[i] belongs to providers[i]; a mismatch is reported by GroupOptions.Validate rather than discovered by a strange split. A weight of zero means standby: that provider is never chosen first while anything with a weight is available, and is tried last when they have all failed.
The algorithm, and what it costs ¶
This is smooth weighted round robin, the scheme nginx uses. Each provider keeps a running credit. On every selection each available provider's credit grows by its weight, the provider with the most credit is chosen first, and its credit is then reduced by the total weight of all available providers. Over any run of queries each provider is chosen exactly in proportion to its weight, and — this is the "smooth" — the choices are interleaved rather than delivered in bursts: weights of 5 and 1 produce one query to the second provider every sixth query, not five to the first and then one.
It is deliberately deterministic. A weighted random draw reaches the same long-run proportions but gets there by luck, so a short window can be badly skewed, and a test asserting the split has to average over thousands of queries or accept a tolerance. This one is exact after a full cycle and reproducible from the first query, which is worth more than the unpredictability it gives up: nothing about a load split needs to be a secret.
The cost is one mutex acquisition and two linear passes over the providers per query, and no allocation when the caller supplies a buffer. For the handful of upstreams a resolver actually has, that is a few tens of nanoseconds; it is not the right shape for hundreds of backends, where the lock would become the contended thing in the system.
The credit vector is state, so each call to Weighted returns an independent strategy and one value must not be shared between groups.
type StrategyValidator ¶
type StrategyValidator interface {
Strategy
// Validate reports every problem with using this strategy over ps.
Validate(ps []Provider) error
}
StrategyValidator is the optional interface a Strategy implements when it can be misconfigured against a particular set of providers.
Weighted needs one weight per provider and Race needs a positive width, and both are mistakes an operator makes in a configuration file rather than conditions that arise at runtime. Reporting them from GroupOptions.Validate means they surface when the group is built, next to every other problem with it, instead of as a strange selection an hour into a shift.