2018-06-12 13:19:09 +00:00
---
2020-04-07 18:55:19 +00:00
layout: docs
page_title: Connect in Production
sidebar_current: docs-guides-connect-production
description: This guide describes best practices for running Consul Connect in production.
2018-06-12 13:19:09 +00:00
---
2018-10-11 09:44:42 +00:00
# Running Connect in Production
2018-06-12 13:19:09 +00:00
2019-04-04 14:52:35 +00:00
Consul Connect can secure all inter-service communication with mutual TLS. It's
2018-06-13 15:38:11 +00:00
designed to work with [minimal configuration out of the
2019-04-04 14:52:35 +00:00
box](https://learn.hashicorp.com/consul/getting-started/connect), however, completing the [security
2020-04-09 23:46:54 +00:00
checklist](/docs/connect/security) and understanding the [Consul security
model](/docs/internals/security) are prerequisites for production
2018-06-12 13:19:09 +00:00
deployments.
2020-04-06 20:27:35 +00:00
After completing this guide, you will be able to configure Connect to
2019-04-04 14:52:35 +00:00
secure services. First, you will secure your Consul cluster with ACLs and
2020-04-06 20:27:35 +00:00
TLS encryption. Next, you will configure Connect on the servers and host.
2019-04-04 14:52:35 +00:00
Finally, you will configure your services to use Connect.
2018-06-12 13:19:09 +00:00
2019-04-04 14:52:35 +00:00
~> Note: To complete this guide you should already have a Consul cluster
2020-04-06 20:27:35 +00:00
with an appropriate number of servers and
2019-04-04 14:52:35 +00:00
clients deployed according to the other reference material including the
2020-04-09 23:46:54 +00:00
[deployment](/docs/guides/deployment) and
[performance](/docs/install/performance) guides.
2018-06-13 15:38:11 +00:00
2018-06-13 15:19:44 +00:00
The steps we need to get to a secure Connect cluster are:
2018-06-12 13:19:09 +00:00
2020-04-06 20:27:35 +00:00
1. [Configure ACLs](#configure-acls)
1. [Configure Agent Transport Encryption](#configure-agent-transport-encryption)
1. [Bootstrap Connect's Certificate Authority](#bootstrap-certificate-authority)
1. [Setup Host Firewall](#setup-host-firewall)
1. [Configure Service Instances](#configure-service-instances)
2018-06-12 13:19:09 +00:00
2019-04-04 14:52:35 +00:00
For existing Consul deployments, it may be necessary to incrementally adopt Connect
2020-04-06 20:27:35 +00:00
service-by-service. In this case, step one and two should already be complete.
2019-04-04 14:52:35 +00:00
However, we recommend reviewing all steps since the final deployment goal is to be compliant with all the security recommendations in this guide.
2018-06-12 13:19:09 +00:00
## Configure ACLs
2019-04-04 14:52:35 +00:00
Consul Connect's security is based on service identity. In practice, the identity
2018-06-12 13:19:09 +00:00
of the service is only enforcible with sufficiently restrictive ACLs.
This section will not replace reading the full [ACL
2020-04-09 23:46:54 +00:00
guide](/docs/guides/acl) but will highlight the specific requirements
2018-06-12 13:19:09 +00:00
Connect relies on to ensure it's security properties.
A service's identity, in the form of an x.509 certificate, will only be issued
to an API client that has `service:write` permission for that service. In other
words, any client that has permission to _register_ an instance of a service
2018-06-24 12:35:39 +00:00
will be able to identify as that service and access all of the resources that that
2018-06-12 13:19:09 +00:00
service is allowed to access.
2019-04-04 14:52:35 +00:00
A secure ACL setup must meet the following criteria.
2018-06-12 13:19:09 +00:00
2020-04-06 20:27:35 +00:00
1. **[ACL default
2020-04-09 23:46:54 +00:00
policy](/docs/agent/options#acl_default_policy)
2019-04-04 14:52:35 +00:00
must be `deny`.** If for any reason you cannot use the default policy of
`deny`, you must add an explicit ACL denying anonymous `service:write`. Note, in this case the Connect intention graph will also default to
2018-06-13 15:19:44 +00:00
`allow` and explicit `deny` intentions will be needed to restrict service
2018-06-13 15:38:11 +00:00
access. Also note that explicit rules to limit who can manage intentions are
necessary in this case. It is assumed for the remainder of this guide that
ACL policy defaults to `deny`.
2020-04-06 20:27:35 +00:00
2. **Each service must have a unique ACL token** that is restricted to
`service:write` only for the named service. You can review the [Securing Consul with ACLs](https://learn.hashicorp.com/consul/advanced/day-1-operations/production-acls#apply-individual-tokens-to-the-services) guide for a
2019-04-04 14:52:35 +00:00
service token example. Note, it is best practices for each instance to get a unique token as described below.
2018-06-12 13:19:09 +00:00
2019-04-04 14:52:35 +00:00
~> Individual Service Tokens: It is best practice to create a unique ACL token per service _instance_ because
it limits the blast radius of a compromise. However, since Connect intentions manage access based only on service identity, it is
possible to create only one ACL token per _service_ and share it between
2018-06-12 13:19:09 +00:00
instances.
2018-06-13 15:19:44 +00:00
In practice, managing per-instance tokens requires automated ACL provisioning,
2018-06-12 13:19:09 +00:00
for example using [HashiCorp's
2020-04-09 23:20:00 +00:00
Vault](https://www.vaultproject.io/docs/secrets/consul).
2018-06-12 13:19:09 +00:00
## Configure Agent Transport Encryption
Consul's gossip (UDP) and RPC (TCP) communications need to be encrypted
2018-06-24 12:35:39 +00:00
otherwise attackers may be able to see ACL tokens while in flight
2018-11-01 21:44:49 +00:00
between the server and client agents (RPC) or between client agent and
application (HTTP). Certificate private keys never leave the host they
are used on but are delivered to the application or proxy over local
HTTP so local agent traffic should be encrypted where potentially
2018-06-24 12:35:39 +00:00
untrusted parties might be able to observe localhost agent API traffic.
2018-06-12 13:19:09 +00:00
2019-04-04 14:52:35 +00:00
Follow the [encryption guide](https://learn.hashicorp.com/consul/advanced/day-1-operations/agent-encryption) to ensure
2018-06-24 12:35:39 +00:00
both gossip encryption and RPC/HTTP TLS are configured securely.
2018-06-12 13:19:09 +00:00
2019-04-04 14:52:35 +00:00
## Bootstrap Connect's Certificate Authority
2018-06-12 13:19:09 +00:00
2019-04-04 14:52:35 +00:00
Consul Connect comes with a built-in Certificate Authority (CA) that will
bootstrap by default when you first [enable](https://www.consul.io/docs/agent/options.html#connect_enabled) Connect on your servers.
2018-06-12 13:19:09 +00:00
To use the built-in CA, enable it in the server's configuration.
```text
connect {
enabled = true
}
```
2019-04-04 14:52:35 +00:00
This configuration change requires a Consul server restart, which you can perform one server at a time
2018-06-12 13:19:09 +00:00
to maintain availability in an existing cluster.
As soon as a server that has Connect enabled becomes the leader, it will
bootstrap a new CA and generate it's own private key which is written to the
Raft state.
Alternatively, an external private key can be provided via the [CA
2020-04-09 23:46:54 +00:00
configuration](/docs/connect/ca#specifying-a-private-key-and-root-certificate).
2018-06-12 13:19:09 +00:00
2019-04-04 14:52:35 +00:00
~> External CAs: Connect has been designed with a pluggable CA component so external CAs can be
integrated. For production workloads we recommend using [Vault or another external
2020-04-09 23:46:54 +00:00
CA](/docs/connect/ca#external-ca-certificate-authority-providers) once
2018-06-13 15:19:44 +00:00
available such that the root key is not stored within Consul state at all.
2018-06-12 13:19:09 +00:00
## Setup Host Firewall
2018-06-13 15:19:44 +00:00
In order to enable inbound connections to connect proxies, you may need to
configure host or network firewalls to allow incoming connections to proxy
ports.
In addition to Consul agent's [communication
2020-04-09 23:46:54 +00:00
ports](/docs/agent/options#ports) any
[proxies](/docs/connect/proxies) will need to have
2018-06-13 15:19:44 +00:00
ports open to accept incoming connections.
2018-10-11 09:44:42 +00:00
If using [sidecar service
2020-04-09 23:46:54 +00:00
registration](/docs/connect/proxies/sidecar-service) Consul will by default
2018-10-11 09:44:42 +00:00
assign ports from [a configurable
2020-04-09 23:46:54 +00:00
range](/docs/agent/options#sidecar_min_port) the default range is 21000 -
2020-04-06 20:27:35 +00:00
2018-10-11 09:44:42 +00:00
21255. If this feature is used, the agent assumes all ports in that range are
2020-04-06 20:27:35 +00:00
both free to use (no other processes listening on them) and are exposed in the
firewall to accept connections from other service hosts.
2018-06-12 13:19:09 +00:00
2018-10-11 09:44:42 +00:00
It is possible to prevent automated port selection by [configuring
`sidecar_min_port` and
2020-04-09 23:46:54 +00:00
`sidecar_max_port`](/docs/agent/options#sidecar_min_port) to both be `0`,
2018-10-11 09:44:42 +00:00
forcing any sidecar service registrations to need an explicit port configured.
2018-06-13 15:19:44 +00:00
It then becomes the same problem as opening ports necessary for any other
application and might be managed by configuration management or a scheduler.
2018-06-12 13:19:09 +00:00
## Configure Service Instances
2018-06-13 15:19:44 +00:00
With [necessary ACL tokens](#configure-acls) in place, all service registrations
need to have an appropriate ACL token present.
For on-disk configuration the `token` parameter of the service definition must
2020-04-06 20:27:35 +00:00
be set.
2019-04-04 14:52:35 +00:00
```json
2020-04-06 20:27:35 +00:00
{
"service": {
"name": "cassandra_db",
"port": 9002,
2019-04-04 14:52:35 +00:00
"token: "<your_token_here>"
2020-04-06 20:27:35 +00:00
}
2019-04-04 14:52:35 +00:00
}
```
2018-06-13 15:19:44 +00:00
2019-04-04 14:52:35 +00:00
For registration via the API the token is passed in the [request
2020-04-09 23:20:00 +00:00
header](/api#authentication), `X-Consul-Token`, or by using the [Go
2018-06-13 15:19:44 +00:00
client configuration](https://godoc.org/github.com/hashicorp/consul/api#Config).
To avoid the overhead of a proxy, applications may [natively
2020-04-09 23:46:54 +00:00
integrate](/docs/connect/native) with connect.
2018-06-13 15:19:44 +00:00
2019-04-04 14:52:35 +00:00
~> Protect Application Listener: If using any kind of proxy for connect, the application must ensure no untrusted
2018-06-13 15:19:44 +00:00
connections can be made to it's unprotected listening port. This is typically
done by binding to `localhost` and only allowing loopback traffic, but may also
2018-06-24 12:35:39 +00:00
be achieved using firewall rules or network namespacing.
2019-04-04 14:52:35 +00:00
For examples of proxy service definitions see the [proxy
2020-04-09 23:46:54 +00:00
documentation](/docs/connect/proxies).
2019-04-04 14:52:35 +00:00
## Summary
2020-04-06 20:27:35 +00:00
After securing your Consul cluster with ACLs and TLS encryption, you
2019-04-04 14:52:35 +00:00
can use Connect to secure service-to-service communication. If you
2020-04-06 20:27:35 +00:00
encounter any issues while setting up Consul Connect, there are
2019-04-04 14:52:35 +00:00
many [community](https://www.consul.io/community.html) resources where you can find help.