Consul Dataplane^BETA

Consul Dataplane is a lightweight process that manages Envoy for Consul service mesh workloads.

Consul Dataplane's design removes the need to run Consul client agents. Removing Consul client agents results in the following benefits:

• Fewer network requirements: Consul client agents use multiple network protocols and require bi-directional communication between Consul client and server agents for the gossip protocol. Consul Dataplane does not use gossip and instead uses a single gRPC connection out to the Consul servers.
• Simplified set up: Consul Dataplane does not need to be configured with a gossip encryption key and operators do not need to distribute separate ACL tokens for Consul client agents.
• Additional runtime support: Consul Dataplane runs as a sidecar alongside your workload, making it easier to support various runtimes. For example, it runs on serverless platforms when you do not have access to the host machine where the Consul client agent can be run.
• Easier upgrades: Deploying new Consul versions no longer requires upgrading Consul client agents. Consul Dataplane has better compatibility across Consul server versions, so you only need to upgrade the Consul servers to take advantage of new Consul features.

Refer to the Documentation section for more information on Consul Dataplane.

Note: We take Consul's security and our users' trust seriously. If you believe you have found a security issue in Consul, please responsibly disclose by contacting us at security@hashicorp.com.

Development

Build

Binary

make dev

Docker Image

make docker

Testing

Unit Tests

make unit-tests

Documentation

Consul Dataplane is currently in beta. It currently supports the following features:

• Consul server discovery: Consul Dataplane discovers Consul server addresses using DNS or by running a script, and is quickly notified of new Consul servers using a ServerWatch gRPC stream.
• Consul server connection: Consul Dataplane maintains a gRPC connection to a Consul server and automatically switches to another Consul server as needed.
• Feature discovery: Consul Dataplane checks Consul server feature support to facilitate version compatibility.
• Envoy management: Consul Dataplane configures, starts, and manages an Envoy sub-process.
• Envoy ADS proxy: Consul Dataplane proxies Envoy's Aggregated Discovery Service (ADS) to a Consul server.
• Consul DNS proxy: Consul Dataplane runs a local DNS server and proxy DNS requests over a gRPC connection to a Consul server to enable service discovery.
• Merged metrics: Consul Dataplane supports Prometheus, StatsD, and DogstatsD metrics. Prometheus metrics for Consul Dataplane, Envoy, and your application are merged and served from a single endpoint.

We plan to add the following features in a subsequent release:

• Envoy SDS: Consul Dataplane will implement a Secret Discovery Service (SDS) for Envoy to generate secret keys and certificate signing requests in order to offload cryptographic operations from the Consul servers.
• Config files and environment variables: Consul Dataplane currently supports CLI flags and will support configuration using both files and environment variables for configuration.

Requirements

• Consul server version 1.14+
• A compatible version of Envoy. The envoy binary must be found on the PATH.

Usage

You should run the consul-dataplane binary as a sidecar alongside your service mesh workloads in place of Envoy. You should not run Envoy directly. Instead, use consul-dataplane to configure and start Envoy for you.

In containerized environments, use the hashicorp/consul-dataplane image in place of an Envoy image. This image includes the consul-dataplane, envoy and go-discover binaries.

The following minimal example shows how to start consul-dataplane:

consul-dataplane \
    -addresses "exec=./get-addresses.sh" \
    -service-node-name my-test-node \
    -proxy-service-id my-svc-id-sidecar-proxy \
    ...

Consul Dataplane connects to Consul servers specified in -addresses. Then it starts an Envoy sub-process configured to run as a sidecar proxy for the service specified by -proxy-service-id on the node specified by -service-node-name in the Consul service catalog.

Server Discovery

Consul Dataplane connects directly to your Consul servers. It supports two forms of address discovery in the -addresses field which uses the syntax supported by go-netaddrs.

• DNS: Consul Dataplane resolves a domain name to discover Consul server IP addresses.

  consul-dataplane -addresses my.consul.example.com
  

• Executable Command: Consul Dataplane runs a script that, on success, returns one or more IP addresses separate by whitespace:

  $ ./my-script.sh
  172.20.0.1
  172.20.0.2
  172.20.0.3
  $ consul-dataplane -addresses "exec=./my-script.sh"
  

  The go-discover binary is included in the hashicorp/consul-dataplane image for use with this mode of server discovery, which functions in a way similar to Cloud Auto-join. The following example demonstrates how to use the go-discover binary with Consul Dataplane.

  consul-dataplane -addresses "exec=discover -q addrs provider=aws region=us-west-2 tag_key=consul-server tag_value=true"
  

Credentials

Consul Dataplane requires an ACL token when ACLs are enabled on the Consul servers. An ACL token can be specified one of two ways.

• Static token: A static ACL token is passed to Consul Dataplane.

  consul-dataplane -credential-type "static"` -static-token "12345678-90ab-cdef-0000-12345678abcd"
  

• Auth method login: Consul Dataplane logs in to one of Consul's supported auth methods.

  consul-dataplane -credential-type "login"
    -login-auth-method <method> \
    -login-bearer-token <token> \  # Or, -login-bearer-token-path
    -login-datacenter <datacenter> \
    -login-meta key1=val1 -login-meta key2=val2 \
    -login-namespace <namespace> \
    -login-partition <partition>
  

Refer to the Configuration Reference for a description of each option passed to consul-dataplane.

Consul Servers Behind a Load Balancer

When Consul servers are behind a load balancer, you must pass -server-watch-disabled to Consul Dataplane:

consul-dataplane -server-watch-disabled

By default, Consul Dataplane opens a server watch stream to a Consul server, which enables the server to inform Consul Dataplane of new or different Consul server addresses. However, if Consul Dataplane is connecting through a load balancer, then it must ignore the Consul server addresses that are returned from the server watch stream.

Configuration Reference

The consul-dataplane binary supports the following flags.