Documentation
¶
Overview ¶
Package breaker provides a circuit breaker for calls to a dependency that may fail or slow down. After a run of consecutive failures the breaker opens and rejects calls immediately with ErrOpen, so callers fail fast instead of waiting on a dependency that is down. After a cool-down it lets one trial call through (half-open): success closes it again, failure re-opens it.
Index ¶
Examples ¶
Constants ¶
const ( DefaultFailureThreshold = 5 DefaultCoolDown = 30 * time.Second )
Default thresholds, used when a Config field is zero.
const ( DefaultMaxAttempts = 3 DefaultInitialBackoff = 100 * time.Millisecond DefaultMaxBackoff = 30 * time.Second DefaultMultiplier = 2.0 )
Defaults for RetryConfig, used when a field is zero.
Variables ¶
var ErrOpen = errors.New("circuit breaker open")
ErrOpen is returned without calling the dependency while the breaker is open.
Functions ¶
func Retry ¶
Retry calls fn until it succeeds, returns an error cfg does not consider retryable, or cfg.MaxAttempts calls have been made, waiting an exponentially growing, capped and jittered delay between calls. fn receives ctx. Progress is logged at debug level through the logger carried by ctx (zerolog.Ctx), if any.
It returns nil on success and a non-retryable error unchanged. When attempts run out the error wraps the last one. When ctx is done it stops at once and returns an error carrying both ctx.Err() and the last error.
To stop retrying while a dependency is down, call the breaker inside fn and treat ErrOpen as not retryable (the default).
Example ¶
Retry composes with a breaker: ErrOpen is not retried, so a retry loop stops as soon as the breaker opens.
package main
import (
"context"
"errors"
"fmt"
"time"
"github.com/dioad/net/breaker"
)
func main() {
b := breaker.New(breaker.Config{FailureThreshold: 1})
attempts := 0
err := breaker.Retry(context.Background(), breaker.RetryConfig{
MaxAttempts: 5,
InitialBackoff: time.Millisecond,
}, func(context.Context) error {
return b.Do(func() error {
attempts++
return errors.New("down")
})
})
fmt.Println(attempts, errors.Is(err, breaker.ErrOpen))
}
Output: 1 true
Types ¶
type Breaker ¶
type Breaker struct {
// contains filtered or unexported fields
}
Breaker is a circuit breaker. It is safe for concurrent use.
func (*Breaker) Health ¶
Health reports ErrOpen while the breaker is open and nil otherwise, matching the shape of a readiness check.
Example ¶
A breaker guards calls to a dependency. Its Health method has the shape of a lifecycle.CheckFunc, so registering it as a degraded check reports the open breaker without taking the service out of rotation.
package main
import (
"context"
"errors"
"fmt"
"time"
"github.com/dioad/lifecycle"
"github.com/dioad/net/breaker"
)
func main() {
b := breaker.New(breaker.Config{
FailureThreshold: 2,
CoolDown: time.Minute,
// Each service keeps its own metric names.
OnStateChange: func(from, to breaker.State) { fmt.Println(from, "->", to) },
})
readiness := lifecycle.NewReadiness()
readiness.AddCheck("payments", b.Health, lifecycle.WithDegraded())
errDown := errors.New("down")
for range 2 {
_ = b.Do(func() error { return errDown })
}
fmt.Println(b.Health(context.Background()))
}
Output: closed -> open circuit breaker open
type Config ¶
type Config struct {
// FailureThreshold is the number of consecutive failures that opens the breaker.
FailureThreshold int
// CoolDown is how long the breaker stays open before allowing a trial call.
CoolDown time.Duration
// IsFailure decides whether an error counts against the dependency. The
// default counts every error except the caller's own cancellation.
IsFailure func(error) bool
// OnStateChange, when set, is called after every transition. It must not
// block or call back into the breaker.
OnStateChange func(from, to State)
// Now is the clock; it defaults to time.Now and exists for tests.
Now func() time.Time
}
Config tunes a Breaker. Zero values select the defaults.
type RetryConfig ¶
type RetryConfig struct {
// MaxAttempts is the number of calls made in all, including the first.
MaxAttempts int
// InitialBackoff is the delay before the first retry, before jitter.
InitialBackoff time.Duration
// MaxBackoff caps the delay before any retry, before jitter.
MaxBackoff time.Duration
// Multiplier is the factor the delay grows by after each retry.
Multiplier float64
// NoJitter turns jitter off. By default each delay is drawn uniformly
// from zero to the backoff ("full jitter"), so clients that failed
// together do not retry together.
NoJitter bool
// Retryable decides whether an error is worth another attempt. The
// default retries every error except the caller's own cancellation and
// ErrOpen, which a retry within the cool-down cannot fix.
Retryable func(error) bool
// Rand returns a value in [0, 1) for jitter; it defaults to a random
// source and exists for tests.
Rand func() float64
// Sleep waits for d or until ctx is done; it defaults to a timer and
// exists for tests.
Sleep func(ctx context.Context, d time.Duration) error
}
RetryConfig tunes Retry. Zero values select the defaults.