README
¶
Go Consul Resolver
A library of composable layers that is designed to provide client-side load balancing for (but not limited to) HTTP client-server communication, using Consul as the service discovery layer
Example Usage
(See advanced usage patterns in the test package)
Components
The library provides three main components which - when combined - allow for transparent load balancing to occur when making HTTP requests.
The provided implementations rely on Consul as the service discovery backend, however additional providers can be supported.
Load Balancer
The load balancer is used to choose the physical node address to dispatch the request to, based on the list of available nodes that it is provided with.
The default implementation provided out of the box uses a Round Robin algorithm and supports entities based on Consul's API.
This allows for implementing more advanced load balancers, with awareness to service tags, DC location, etc.
Tag Aware Load Balancer
Given a list of tags this load balancer will prefer nodes with the provided tags. If no tagged nodes found and fallback allowed it will choose the next node using round robin algorithm.
Custom Load Balancer
may be added by implementing the Balancer API:
import "github.com/hashicorp/consul/api"
type Balancer interface {
Select() (*api.ServiceEntry, error)
UpdateTargets(targets []*api.ServiceEntry)
}
Resolver
The resolver is responsible for resolving a service name to physical node addresses, using Consul as the service discovery provider.
It does so by querying Consul's Health API with blocking queries, which allows it to be notified when changes occur in the list of available nodes for a given service.
When initializing a new Consul Resolver (via NewConsulResolver), you must provide a context and a configuration struct.
The context is used to gracefully terminate the go routine which is used to watch Consul - note that when the context is cancelled,
the resolver will become stale and will immediately return an error (based on the context's Err output) when trying to use it.
The configuration allows specifying the following parameters:
- ServiceSpec - the spec of the service being resolved (service name, port, etc.)
- Balancer - the load balancer to use
- Client - a Consul API client
- Query - the Consul query options, if you wish to override the defaults
- LogFn - A custom logging function
Once initialized with a load balancer, the resolver can be used as a stand-alone component to load balance between the various instances of the service name it was provided with.
HTTP Transport
The LoadBalancedTransport serves as the intermediary layer between the Resolver and a vanilla HTTP client.
It intercepts outbound HTTP requests and - based on the request's host - delegates IP resolution to the matching resolver, or to the base transport layer if no resolver matches.
The transport implements the http.RoundTriper interface, using *http.Transport as the underlying implementation by default (unless another implementation was provided).
By default, the library obtains the base transport from http.DefaultTransport however you may provide any type implementing http.RoundTripper via the configuration.
Upon successful address resolution from Consul, the transport will replace the Host part of the requests' URL struct with the IP address of the target node. Thus, a request to http://my-cool-service/api will be cloned, and its URL changed to http://10.1.2.3/api.
The configuration allows specifying the following parameters:
- Resolvers - a list of
Resolverinstances that will be used by the transport to resolve hostnames - Base - a base
http.RoundTripperinstance to override the default transport - NetResolverFallback - a boolean flag that controls the transport's behavior in case of a resolution error.
By default (false), aResolvererror will propagate up the call stack, and fail the HTTP request.
If set to true, the transport will attempt to resolve the address by delegating the request to the base transport implementation (which will resolve it via DNS). - LogFn - A custom logging function
Known Limitations
- TLS - in order to support TLS, you can provide a custom Base
http.Transportwith theServerNamein it'sTLSClientConfigset to the hostname presented by your certificate.
Multi-DC Support
The library provides support for multiple data centers by specifying a list of fallback data-centers to use.
If no instances are available in the local data center, the library will select instances from one of the fallback data-centers, prioritized by the order of data-centers provided by the user in the FallbackDatacenters property of the ResolverConfig struct.
Example
package main
import (
"context"
"net/http"
"github.com/hashicorp/consul/api"
consulresolver "github.com/AppsFlyer/go-consul-resolver"
"github.com/AppsFlyer/go-consul-resolver/lb"
)
func main() {
consulClient, _ := api.NewClient(&api.Config{Address: "localhost:8500")})
coolServiceResolver, _ := consulresolver.NewConsulResolver(context.Background(), consulresolver.ResolverConfig{
ServiceSpec: consulresolver.ServiceSpec{
ServiceName: "cool-service",
},
Balancer: &lb.TagAwareLoadBalancer{
Tags: []string{"az-eu-west-1c", "az-eu-east-1c"},
},
Client: consulClient,
})
transport, _ := consulresolver.NewLoadBalancedTransport(
consulresolver.TransportConfig{
Resolvers: []consulresolver.Resolver{coolServiceResolver},
})
client := &http.Client{Transport: transport}
// this will resolve via Consul
res, _ := client.Get("http://cool-service/_/health.json")
// this will resolve via DNS
res, _ := client.Get("http://google.com")
}
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Balancer ¶
type Balancer interface {
// Select returns a *api.ServiceEntry describing the selected target.
// If Select failed to provide a viable target, it should return a non-nil error.
// Important: Select must be non-blocking!
Select() (*api.ServiceEntry, error)
// UpdateTargets will be called periodically to refresh the Balancer's targets list from which the Balancer is allowed to select
UpdateTargets(targets []*api.ServiceEntry)
}
Balancer interface provides methods for selecting a target and updating its state
type LoadBalancedTransport ¶
type LoadBalancedTransport struct {
// contains filtered or unexported fields
}
func NewLoadBalancedTransport ¶
func NewLoadBalancedTransport(conf TransportConfig) (*LoadBalancedTransport, error)
type Resolver ¶
type Resolver interface {
// Resolve should return a ServiceAddress, or a non-nil error if resolution failed
Resolve(context.Context) (ServiceAddress, error)
// ServiceName should return the name of the service the resolver is providing targets for
// Target resolution will be skipped for hosts that do not match the name returned by ServiceName
ServiceName() string
}
type ResolverConfig ¶
type ResolverConfig struct {
// A function that will be used for logging.
// Optional
// Default: log.Printf
Log LogFn
// The Service Spec the resolver will handle
// Mandatory
ServiceSpec ServiceSpec
// The Balancer that will be used to select targets
// Optional
// Default: RoundRobinLoadBalancer
Balancer Balancer
// The consul client
// Mandatory
Client *api.Client
// The consul query options configuration
// Optional
Query *api.QueryOptions
// A list of datacenters to query, ordered by priority.
// Optional. Will use only the local DC if not provided.
FallbackDatacenters []string
}
type ServiceAddress ¶
type ServiceProvider ¶
type ServiceProvider interface {
ServiceMultipleTags(service string, tags []string, passingOnly bool, q *api.QueryOptions) ([]*api.ServiceEntry, *api.QueryMeta, error)
}
ServiceProvider provides a method for obtaining a list of *api.ServiceEntry entities from Consul
type ServiceResolver ¶
type ServiceResolver struct {
// contains filtered or unexported fields
}
func NewConsulResolver ¶
func NewConsulResolver(ctx context.Context, conf ResolverConfig) (*ServiceResolver, error)
NewConsulResolver creates a new Consul Resolver ctx - a context used for graceful termination of the consul-watcher go routine. Note that canceling the context will render the resolver stale, and any attempt to use it will immediately return an error conf - the resolver's config
func (*ServiceResolver) Resolve ¶
func (r *ServiceResolver) Resolve(ctx context.Context) (ServiceAddress, error)
Resolve returns a single ServiceAddress instance of the resolved target
func (*ServiceResolver) ServiceName ¶
func (r *ServiceResolver) ServiceName() string
ServiceName returns the service name that the resolver is looking up
type ServiceSpec ¶
type ServiceSpec struct {
// The name of the service in Consul.
// Mandatory
ServiceName string
// The port to use, if different from the `Service.Port` in Consul
// If set to a value other than 0, this will override the service port discovered in Consul.
// Optional
// Default: 0
ServicePort int
// Filter service instances by Consul tags.
// Optional
// Default: nil
Tags []string
// Filter service instances by Health status.
// Optional
// Default: false (only healthy endpoints are used)
IncludeUnhealthy bool
}
type TransportConfig ¶
type TransportConfig struct {
// A function that will be used for logging.
// Optional
// Default: log.Printf
Log LogFn
// The resolvers to be used for address resolution.
// Multiple resolvers are supported, and will be looked up by the `ServiceName`
// Mandatory
Resolvers []Resolver
// If true, the transport will fallback to net/Resolver on resolver error
// Optional
// Default: false
NetResolverFallback bool
// A base transport to be used for the underlying request handling.
// Optional
// Default: http.DefaultTransport
Base http.RoundTripper
}
Source Files
Directories