github
/
consul
mirror of https://github.com/hashicorp/consul
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
Browse Source

CE-654 - TLS Encryption docs + CE-713 - Gossip Encryption key rotation (#21509) 

* New proposed structure

* Fix structure and add some content

* Fix structure and add some content

* Fix structure and add some content

* Add content

* Add content

* mtls steps

* Encryption docs structure change

* Encryption docs structure change

* Encryption docs structure change

* Encryption docs structure change

* Encryption docs structure change

* Encryption docs structure change

* Encryption docs structure change

* Encryption docs structure change

* Encryption docs structure change

* Encryption docs structure change

* Encryption docs structure change

* Encryption docs structure change

* Encryption docs structure change

* Encryption docs structure change

* Encryption docs structure change

* Encryption docs structure change

* Encryption docs structure change

* Encryption docs structure change

* Encryption docs structure change

* spacing fixes

* Replace <CodeTabs>

* <CodeBlockConfig> alignment

* indent fixes

* spacing

* More Code tabs fixes

* Structure chenges

* Structure chenges

* Extra content and CE-713 migration

* Extra content

* Extra content

* Extra content

* Apply suggestions from code review

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Apply suggestions from code review

* Test CodeTabs

* Test CodeTabs

* Apply suggestions from code review

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

---------

Co-authored-by: boruszak <jeffrey.boruszak@hashicorp.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
pull/21771/head
danielehc 2 months ago committed by GitHub
parent
ac9e694b98
commit
250b1dece5
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
6 changed files with 943 additions and 124 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Unified View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 111
      website/content/docs/security/encryption.mdx
  2. 302
      website/content/docs/security/encryption/gossip.mdx
  3. 55
      website/content/docs/security/encryption/index.mdx
  4. 553
      website/content/docs/security/encryption/mtls.mdx
  5. 15
      website/content/docs/security/index.mdx
  6. 31
      website/data/docs-nav-data.json

111
website/content/docs/security/encryption.mdx
Unescape View File

@ -1,111 +0,0 @@
---
layout: docs
page_title: Encryption Systems
description: >-
Consul supports encrypting all of its network traffic. Remote Process Calls (RPCs) between client and server agents can be encrypted with TLS and authenticated with certificates. Gossip communication between all agents can also be encrypted.
---
# Encryption
The Consul agent supports encrypting all of its network traffic. The exact
method of encryption is described on the [encryption internals page](/consul/docs/security).
There are two separate encryption systems, one for gossip traffic and one for RPC.
To configure the encryption systems on a new cluster, review this following tutorials to
[enable gossip encryption](/consul/tutorials/security/gossip-encryption-secure?utm_source=consul.io&utm_medium=docs) and
[TLS encryption for agent communication](/consul/tutorials/security/tls-encryption-secure?utm_source=docs).
## Gossip Encryption
Enabling gossip encryption only requires that you set an encryption key when
starting the Consul agent. The key can be set via the `encrypt` parameter.
~> **WAN Joined Datacenters Note:** If using multiple WAN joined datacenters, be sure to use _the same encryption key_ in all datacenters.
The key must be 32-bytes, Base64 encoded. As a convenience, Consul provides the
[`consul keygen`](/consul/commands/keygen) command to generate a
cryptographically suitable key:
```shell-session
$ consul keygen
pUqJrVyVRj5jsiYEkM/tFQYfWyJIv4s3XkvDwy7Cu5s=
```
With that key, you can enable encryption on the agent. If encryption is enabled,
the output of [`consul agent`](/consul/commands/agent) will include "Encrypt: true":
```shell-session
$ cat encrypt.json
{"encrypt": "pUqJrVyVRj5jsiYEkM/tFQYfWyJIv4s3XkvDwy7Cu5s="}
$ consul agent -data-dir=/tmp/consul -config-file=encrypt.json
==> WARNING: LAN keyring exists but -encrypt given, using keyring
==> WARNING: WAN keyring exists but -encrypt given, using keyring
==> Starting Consul agent...
==> Starting Consul agent RPC...
==> Consul agent running!
Node name: 'Armons-MacBook-Air.local'
Datacenter: 'dc1'
Server: false (bootstrap: false)
Client Addr: 127.0.0.1 (HTTP: 8500, HTTPS: -1, DNS: 8600, RPC: 8400)
Cluster Addr: 10.1.10.12 (LAN: 8301, WAN: 8302)
Gossip encrypt: true, RPC-TLS: false, TLS-Incoming: false
...
```
All nodes within a Consul cluster must share the same encryption key in
order to send and receive cluster information.
## Configuring Gossip Encryption on an existing cluster
As of version 0.8.4, Consul supports upshifting to encrypted gossip on a running cluster
through the following process. Review this [step-by-step tutorial](/consul/tutorials/security/gossip-encryption-secure?utm_source=consul.io&utm_medium=docs#enable-gossip-encryption-existing-cluster)
to encrypt gossip on an existing cluster.
## RPC Encryption with TLS
Consul supports using TLS to verify the authenticity of servers and clients. To enable this,
Consul requires that all clients and servers have key pairs that are generated by a single
Certificate Authority. This can be a private CA, used only internally. The
CA then signs keys for each of the agents, as in
[this tutorial on generating both a CA and signing keys](/consul/tutorials/security/tls-encryption-secure?utm_source=docs).
~> Certificates need to be created with x509v3 extendedKeyUsage attributes for both clientAuth and serverAuth since Consul uses a single cert/key pair for both server and client communications.
TLS can be used to verify the authenticity of the servers or verify the authenticity of clients.
These modes are controlled by the [`verify_outgoing`](/consul/docs/agent/config/config-files#tls_internal_rpc_verify_outgoing),
[`verify_server_hostname`](/consul/docs/agent/config/config-files#tls_internal_rpc_verify_server_hostname),
and [`verify_incoming`](/consul/docs/agent/config/config-files#tls_internal_rpc_verify_incoming) options, respectively.
If [`verify_outgoing`](/consul/docs/agent/config/config-files#tls_internal_rpc_verify_outgoing) is set, agents verify the
authenticity of Consul for outgoing connections. Server nodes must present a certificate signed
by a common certificate authority present on all agents, set via the agent's
[`ca_file`](/consul/docs/agent/config/config-files#tls_internal_rpc_ca_file) and [`ca_path`](/consul/docs/agent/config/config-files#tls_internal_rpc_ca_path)
options. All server nodes must have an appropriate key pair set using [`cert_file`](/consul/docs/agent/config/config-files#tls_internal_rpc_cert_file) and [`key_file`](/consul/docs/agent/config/config-files#tls_internal_rpc_key_file).
If [`verify_server_hostname`](/consul/docs/agent/config/config-files#tls_internal_rpc_verify_server_hostname) is set, then
outgoing connections perform hostname verification. All servers must have a certificate
valid for `server.<datacenter>.<domain>` or the client will reject the handshake. This is
a new configuration as of 0.5.1, and it is used to prevent a compromised client from being
able to restart in server mode and perform a MITM (Man-In-The-Middle) attack. New deployments should set this
to true, and generate the proper certificates, but this is defaulted to false to avoid breaking
existing deployments.
If [`verify_incoming`](/consul/docs/agent/config/config-files#tls_internal_rpc_verify_incoming) is set, the servers verify the
authenticity of all incoming connections. All clients must have a valid key pair set using
[`cert_file`](/consul/docs/agent/config/config-files#tls_internal_rpc_cert_file) and
[`key_file`](/consul/docs/agent/config/config-files#tls_internal_rpc_key_file). Servers will
also disallow any non-TLS connections. To force clients to use TLS,
[`verify_outgoing`](/consul/docs/agent/config/config-files#tls_internal_rpc_verify_outgoing) must also be set.
TLS is used to secure the RPC calls between agents, but gossip between nodes is done over UDP
and is secured using a symmetric key. See above for enabling gossip encryption.
## Configuring TLS on an existing cluster
As of version 0.8.4, Consul supports migrating to TLS-encrypted traffic on a running cluster
without downtime. This process assumes a starting point with no TLS settings configured and involves
an intermediate step in order to get to full TLS encryption. Review the
[Securing RPC Communication with TLS Encryption tutorial](/consul/tutorials/security/tls-encryption-secure?utm_source=docs)
for the step-by-step process to configure TLS on a new or existing cluster. Note the call outs there
for existing cluster configuration.

302
website/content/docs/security/encryption/gossip.mdx
Unescape View File

@ -0,0 +1,302 @@
---
layout: docs
page_title: Manage gossip encryption
description: >-
Consul supports encrypting all of its network traffic. Learn how to secure gossip communication between all agents by creating and rotating a symmetric key.
---
# Manage gossip encryption
This topic describes the steps to enable gossip encryption on a Consul datacenter and rotate the gossip encryption key to maintain secure communication between agents.
## Workflows
We recommend enabling gossip encryption to all new deployed Consul datacenters. You can also update the agents in an existing datacenter to use gossip encryption.
The workflow to enable gossip encryption changes depending on whether your datacenter has client agents running.
To [enable gossip encryption on a new datacenter](#enable-gossip-encryption-on-a-new-datacenter):
1. Use `consul keygen` to generate a new gossip encryption key.
1. Create a configuration file that sets the `encrypt` parameter to the newly generated key.
1. Distribute the configuration file to all agent nodes that are part of the datacenter. Then start the Consul agent on all the nodes.
To [enable gossip encryption on an existing datacenter](#enable-gossip-encryption-on-an-existing-datacenter):
1. Use `consul keygen` to generate a new gossip encryption key.
1. Create a configuration file that sets the `encrypt` parameter to the newly generated key and sets `encrypt_verify_incoming` and `encrypt_verify_outgoing` to `false`.
1. Distribute the configuration file to all agent nodes that are part of the datacenter. Then perform a rolling restart of all Consul agents.
1. Update the `encrypt_verify_outgoing` setting to `true` and perform a rolling restart of all Consul agents.
1. Update the `encrypt_verify_incoming` setting to `true` and perform a rolling restart of all Consul agents.
If you have multiple datacenters joined in WAN federation, be sure to use _the same encryption key_ in all datacenters.
## Enable gossip encryption on a new datacenter
We recommend enabling gossip encryption on all new Consul datacenters.
### Generate a gossip encryption key
First, generate an encryption key on a Consul server. The Consul CLI includes a `consul keygen` command to generate a key.
```shell-session
$ consul keygen
YwgWlBvicJN17UOYcutXLpJSZsw5aWpLEEWqgK635zA=
```
You can generate a new gossip key using any method that can create 32 random bytes encoded in base64. For example, you can use `openssl` or `dd` to create a key on Linux with one of the following commands:
- `openssl rand -base64 32`
- `dd if=/dev/urandom bs=32 count=1 status=none | base64`
### Add the key to the agent configuration
Create a configuration that sets `encrypt` parameter to the newly generated key. You can edit the existing agent configuration, or you can add a file to the configuration directory. For more information, refer to [configuring Consul agents](/consul/docs/agent#configuring-consul-agents).
<CodeTabs>
```hcl
encrypt = "YwgWlBvicJN17UOYcutXLpJSZsw5aWpLEEWqgK635zA="
```
```json
{
"encrypt": "YwgWlBvicJN17UOYcutXLpJSZsw5aWpLEEWqgK635zA="
}
```
</CodeTabs>
### Add the key to all agent configurations
Distribute the gossip key to all the agent nodes that need to be pert of the datacenter. Then start the Consul agent on all the nodes.
When gossip encryption is properly configured, `Gossip Encryption: true` appears in the Consul logs at startup.
<CodeBlockConfig filename="consul.log" hideClipboard highlight="10">
```log
==> Starting Consul agent...
Version: '1.19.0'
Build Date: '2024-06-12 13:59:10 +0000 UTC'
Node ID: 'e74b1ade-e932-1707-cdf1-6579b8b2536c'
Node name: 'consul-server-0'
Datacenter: 'dc1' (Segment: '<all>')
Server: true (Bootstrap: false)
Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: 8443, gRPC: -1, gRPC-TLS: 8503, DNS: 53)
Cluster Addr: 172.19.0.7 (LAN: 8301, WAN: 8302)
Gossip Encryption: true
Auto-Encrypt-TLS: true
ACL Enabled: true
Reporting Enabled: false
ACL Default Policy: deny
HTTPS TLS: Verify Incoming: false, Verify Outgoing: true, Min Version: TLSv1_2
gRPC TLS: Verify Incoming: false, Min Version: TLSv1_2
Internal RPC TLS: Verify Incoming: true, Verify Outgoing: true (Verify Hostname: true), Min Version: TLSv1_2
## ...
```
</CodeBlockConfig>
## Enable gossip encryption on an existing datacenter
You can also enable gossip encryption on existing Consul datacenters.
### Generate a gossip encryption key
First, generate an encryption key on a Consul server. The Consul CLI includes a `consul keygen` command to generate a key.
```shell-session
$ consul keygen
YwgWlBvicJN17UOYcutXLpJSZsw5aWpLEEWqgK635zA=
```
### Add the key to the agent configuration
Create a configuration that sets the `encrypt` parameter to the newly generated key and sets the `encrypt_verify_incoming` and `encrypt_verify_outgoing` parameters to `false`. You can edit the existing agent configuration, or you can add a file to the configuration directory. For more information, refer to [configuring Consul agents](/consul/docs/agent#configuring-consul-agents).
<CodeTabs>
<CodeBlockConfig filename="/etc/consul.d/encryption.hcl">
```hcl
encrypt = "YwgWlBvicJN17UOYcutXLpJSZsw5aWpLEEWqgK635zA="
encrypt_verify_incoming = false
encrypt_verify_outgoing = false
```
</CodeBlockConfig>
<CodeBlockConfig filename="/etc/consul.d/encryption.json">
```json
{
"encrypt": "YwgWlBvicJN17UOYcutXLpJSZsw5aWpLEEWqgK635zA=",
"encrypt_verify_incoming" : false,
"encrypt_verify_outgoing" : false
}
```
</CodeBlockConfig>
</CodeTabs>
### Add the configuration to all agents
Distribute the configuration to all the agent nodes that need to be part of the datacenter, and then initiate a rolling update that restarts each agent.
You must restart all of the agents. The `consul reload` and `kill -HUP <process_id>` commands are not sufficient when changing the gossip configuration.
### Update outgoing encryption
The agents can decrypt gossip communication with the `encrypt` parameter set, but they are not able to send encrypted traffic.
Update the `encrypt_verify_outgoing` setting to `true` and then perform another rolling update of all Consul agents. Complete the process on all the nodes before you begin [updates to incoming encryption](#update-the-incoming-encryption).
<CodeTabs>
<CodeBlockConfig filename="/etc/consul.d/encryption.hcl">
```hcl
encrypt = "YwgWlBvicJN17UOYcutXLpJSZsw5aWpLEEWqgK635zA="
encrypt_verify_incoming = false
encrypt_verify_outgoing = true
```
</CodeBlockConfig>
<CodeBlockConfig filename="/etc/consul.d/encryption.json">
```json
{
"encrypt": "YwgWlBvicJN17UOYcutXLpJSZsw5aWpLEEWqgK635zA=",
"encrypt_verify_incoming": false,
"encrypt_verify_outgoing": true
}
```
</CodeBlockConfig>
</CodeTabs>
### Update incoming encryption
The agents can send encrypted gossip but still allow unencrypted incoming traffic. Update the `encrypt_verify_incoming` setting to `true` and then perform a final rolling update on all the agents.
<CodeTabs>
<CodeBlockConfig filename="/etc/consul.d/encryption.hcl">
```hcl
encrypt = "YwgWlBvicJN17UOYcutXLpJSZsw5aWpLEEWqgK635zA="
encrypt_verify_incoming = true
encrypt_verify_outgoing = true
```
</CodeBlockConfig>
<CodeBlockConfig filename="/etc/consul.d/encryption.json">
```json
{
"encrypt": "YwgWlBvicJN17UOYcutXLpJSZsw5aWpLEEWqgK635zA=",
"encrypt_verify_incoming": true,
"encrypt_verify_outgoing": true
}
```
</CodeBlockConfig>
</CodeTabs>
## Rotate the gossip encryption key
It is important to periodically rotate the gossip encryption key your Consul datacenter uses.
The process of rotating the gossip encryption key is centralized so that you can perform it on a single datacenter node.
The process to rotate a gossip encryption key consists of the following steps:
1. Generate a new encryption key using the `consul keygen` command.
1. Install the new encryption key using the `consul keyring -install` command.
1. Verify the new key is installed in your Consul datacenter with the `consul keyring -list` command.
1. Instruct Consul to use the new key with the `consul keyring -use` command.
1. Remove the old key using the `consul keyring -remove` command.
### Generate a new encryption key
Generate a new key using `consul keygen`:
```shell-session
$ consul keygen
FfRV9j6NXU9LlCI4zLZjjpZdj4Nrqsdm7R8YgzSHzHw=
```
### Add new key to the keyring
Add your newly generated key to the keyring.
```shell-session
$ consul keyring -install FfRV9j6NXU9LlCI4zLZjjpZdj4Nrqsdm7R8YgzSHzHw=
==> Installing new gossip encryption key...
```
### Verify that the new key is installed
After you add the key to one of the Consul agents, Consul propagates it across the entire datacenter. You do not need to repeat the command on other agents.
To ensure that the key has been propagated to all agents, list the installed encryption keys and verify that the number of agents that recognize the key is equal to the total number of agents in the datacenter.
```shell-session
$ consul keyring -list
==> Gathering installed encryption keys...
WAN:
FfRV9j6NXU9LlCI4zLZjjpZdj4Nrqsdm7R8YgzSHzHw= [1/1]
YwgWlBvicJN17UOYcutXLpJSZsw5aWpLEEWqgK635zA= [1/1]
dc1 (LAN):
YwgWlBvicJN17UOYcutXLpJSZsw5aWpLEEWqgK635zA= [7/7]
FfRV9j6NXU9LlCI4zLZjjpZdj4Nrqsdm7R8YgzSHzHw= [7/7]
```
Confirm that the two keys are installed in the datacenter and recognized by all agents, including server agents. The server agents are listed in the `WAN` section. Do not proceed to the next step unless all agents have the new key.
### Promote the new key to primary
After all agents recognize the key, it is possible to promote it to be the new primary encryption key.
```shell-session
$ consul keyring -use FfRV9j6NXU9LlCI4zLZjjpZdj4Nrqsdm7R8YgzSHzHw=
==> Changing primary gossip encryption key...
```
### Remove the old key from the keyring
Unused keys in the keyring are a potential security risk to your Consul cluster. We recommended that you remove the former primary key from the keyring after a new key is installed.
```shell-session
$ consul keyring -remove YwgWlBvicJN17UOYcutXLpJSZsw5aWpLEEWqgK635zA=
==> Removing gossip encryption key...
```
Verify that the keyring contains only one key.
```shell-session
$ consul keyring -list
==> Gathering installed encryption keys...
WAN:
FfRV9j6NXU9LlCI4zLZjjpZdj4Nrqsdm7R8YgzSHzHw= [1/1]
dc1 (LAN):
FfRV9j6NXU9LlCI4zLZjjpZdj4Nrqsdm7R8YgzSHzHw= [7/7]
```
## Next steps
Documentation for the commands used in this topic is available at [Consul agent configuration - Encryption Parameters](/consul/docs/agent/config/config-files#encryption-parameters). You can find more information over the gossip protocol used by Consul at [Gossip Protocol](/consul/docs/architecture/gossip).
After you enable gossip encryption, you can continue to process to secure your Consul datacenter by enabling mutual TLS encryption. For more information, refer to [Mutual TLS (mTLS) Encryption](/consul/docs/security/encryption/mtls).
To learn how to automate gossip key rotation using HashiCorp Vault and consul-template, refer to the [Automatically Rotate Gossip Encryption Keys Secured in Vault tutorial](/consul/tutorials/operate-consul/vault-kv-consul-secure-gossip).

55
website/content/docs/security/encryption/index.mdx
Unescape View File

@ -0,0 +1,55 @@
---
layout: docs
page_title: Encrypted communication between Consul agents
description: >-
Consul supports encrypting all of its network traffic. Remote Procedure Calls (RPCs) between client and server agents can be encrypted with TLS and authenticated with certificates. Gossip communication between all agents can also be encrypted.
---
# Encrypted communication between Consul agents
This topic provides an overview of the two distinct encryption systems available in Consul. [Gossip encryption](/consul/docs/security/encryption#gossip-encryption) and [Mutual TLS encryption](/consul/docs/security/encryption#mutual-tls-mtls-encryption) are the foundations of a secure Consul datacenter.
The guidelines in the [Consul security model](/consul/docs/security/security-models/core) for operating a secure Consul deployment recommends using both encryption systems.
## Gossip Encryption
Consul uses a [gossip protocol](/consul/docs/architecture/gossip) to perform the following cluster operations:
- Identify datacenter members.
- Quickly detect failed members and notify the rest of the cluster.
- Broadcast events and queries that can trigger custom workflows.
The gossip protocol, as well as its membership management and message broadcasting features, use the [Serf library](https://www.serf.io/).
In a default Consul configuration, the gossip protocol uses [port `8301`](/consul/docs/install/ports#lan-serf) for LAN communications and [port `8302`](/consul/docs/install/ports#lan-serf) for WAN communications between federated datacenters. Enabling gossip encryption on a Consul datacenter is required to secure traffic on these two ports.
Gossip encryption is symmetric and based on a single key that is shared across all members of the datacenter. You can configure gossip encryption in Consul using the following parameters:
- [`encrypt`](/consul/docs/agent/config/config-files#encrypt).
- [`encrypt_verify_incoming`](/consul/docs/agent/config/config-files#encrypt_verify_incoming). Only used when upshifting from unencrypted to encrypted gossip on a running cluster.
- [`encrypt_verify_outgoing`](/consul/docs/agent/config/config-files#encrypt_verify_outgoing). Only used when upshifting from unencrypted to encrypted gossip on a running cluster.
To learn more about enabling gossip encryption on your Consul datacenter and rotating gossip keys, refer to [manage gossip encryption](/consul/docs/security/encryption/gossip).
## Mutual TLS (mTLS) Encryption
Consul uses several communication protocols over different ports that you can secure using mTLS:
- A [consensus protocol](/consul/docs/architecture/consensus) provides data consistency between Consul servers. It typically uses [port `8300`](/consul/docs/install/ports#server-rpc).
- Remote Procedure Calls (RPC) forward requests from client agents to server agents. They use the same port the consensus protocol uses.
- An HTTP or HTTPS interface permits client communication with the Consul API, CLI, and UI. It typically uses [port `8500`](/consul/docs/install/ports#http) and [port `8501`](/consul/docs/install/ports#https).
- A gRPC interface receives incoming traffic from the gateways and Envoy proxies registered to the agent node. It typically uses [port `8502`](/consul/docs/install/ports#client-grpc) and [port `8503`](/consul/docs/install/ports#client-grpc-tls).
Consul uses mTLS to verify the authenticity of server and client agents. It requires that all clients and servers have key pairs that are generated by a single Certification Authority (CA). We recommend using a private CA that is not shared with other applications.
You can configure mTLS in Consul using the [`tls` stanza in agent configuration files](/consul/docs/agent/config/config-files#tls-1).
You can configure mTLS encryption for each protocol separately using the following parameters in the agent configuration file:
- [`tls.defaults`](/consul/docs/agent/config/config-files#tls_defaults) provides default settings that Consul applies to every interface unless explicitly overridden by protocol-specific configurations.
- [`tls.internal_rpc`](/consul/docs/agent/config/config-files#tls_internal_rpc) provides settings for the internal server RPC interface.
- [`tls.https`](/consul/docs/agent/config/config-files#tls_https) provides settings for the HTTP/HTTPS interface.
- [`tls.grpc`](/consul/docs/agent/config/config-files#tls_grpc) provides settings for the gRPC/xDS interface.
To learn more about enabling mTLS on your Consul datacenter, refer to [Manage mTLS encryption](/consul/docs/security/encryption/mTLS).

553
website/content/docs/security/encryption/mtls.mdx
Unescape View File

@ -0,0 +1,553 @@
---
layout: docs
page_title: Manage agent mTLS encryption
description: >-
Consul supports encrypting all of its network traffic. Learn how to encrypt and authenticate Remote Process Calls (RPCs) between client and server agents with TLS certificates.
---
# Manage agent mTLS encryption
This topic describes mutual TLS (mTLS) encryption between agents in a Consul datacenter.
This mTLS encryption is distinct from Consul service mesh mTLS encryption. To learn about encrypted mTLS communication between services, refer to [service mesh certificate authority](/consul/docs/connect/ca).
## Benefits of TLS encryption
Consul supports TLS certificates that verify the authenticity of servers and clients to secure `RCP`, `gRPC`, and `HTTP` traffic.
Implementing TLS encryption in Consul datacenters improves your deployment's security in the following ways:
- **Encrypt data in transit:** TLS encrypts data transmitted between Consul datacenter nodes and user interfaces such as the UI or CLI to ensure that sensitive information is not exposed to unauthorized parties.
- **Agent authentication:** TLS ensures that only verified parties can communicate with each other using certificates. This security measure prevents unauthorized nodes, services, and users from interacting with your Consul datacenter.
- **Prevent man-in-the-middle (MITM) attacks:** Without TLS, attackers may intercept and change communications within your Consul deployments. TLS mitigates this risk by encrypting data and verifying the identity of the communication participants.
- **Comply with security regulations and standards:** Compliance frameworks and regulations like PCI-DSS and HIPAA mandate the encryption of data in transit, which makes TLS a requirement for Consul deployments in regulated environments.
Mutual TLS (mTLS) requires that all clients and servers have key pairs that are generated by a single Certification Authority (CA). We recommend using a private CA that is not shared with other applications.
The following parameters in agent configuration files define the agent verification behavior:
- [`tls.defaults.verify_incoming`](/consul/docs/agent/config/config-files#tls_defaults_verify_incoming)
- [`tls.defaults.verify_outgoing`](/consul/docs/agent/config/config-files#tls_defaults_verify_outgoing)
- [`tls.defaults.verify_server_hostname`](/consul/docs/agent/config/config-files#tls_default_verify_server_hostname)
## Workflow
The process to enable TLS encryption in Consul deployments consists of the following steps:
1. [Initialize the built-in CA](/consul/docs/security/encryption/mtls#initialize-the-built-in-ca) used to sign all certificates.
1. [Create server certificates](/consul/docs/security/encryption/mtls#create-server-certificates) to secure Consul servers.
1. [Configure server agents](/consul/docs/security/encryption/mtls#configure-server-agents) for TLS encryption.
1. [Start server agents](/consul/docs/security/encryption/mtls#start-consul-servers).
1. [Configure Consul client agents](/consul/docs/security/encryption/mtls#configure-client-agents).
1. [Start client agents](/consul/docs/security/encryption/mtls#start-consul-clients).
## Initialize the built-in CA
The first step to configure mTLS for Consul is to initialize the certificate authority (CA) that signs the certificates. To prevent unauthorized datacenter access, Consul requires that all certificates are signed by the same CA. We recommend using a private CA because any certificate it signs will be allowed to communicate with the Consul datacenter.
Consul supports a variety of tools for creating and managing your own CA, like the [PKI secrets engine in Vault](/vault/docs/secrets/pki) or the [Terraform TLS provider](https://registry.terraform.io/providers/hashicorp/tls/latest/docs). Consul also ships with built-in TLS tools to create and manage certificates.
If you have the Consul binary installed on your path, you can create the CA and certificates, even before you start a Consul server agent.
```shell-session
$ consul tls ca create -domain=consul
==> Saved consul-agent-ca.pem
==> Saved consul-agent-ca-key.pem
```
The command generates two files, `<domain>-agent-ca.pem` and `<domain>-agent-ca-key.pem`. In this example, the domain used to generate the certificates is the default one, `consul`.
The CA certificate, `consul-agent-ca.pem`, contains the public key necessary to validate Consul certificates. You must distribute this certificate to every node where a Consul agent runs.
The CA key, `consul-agent-ca-key.pem`, signs certificates for Consul nodes. You must keep this key private. Possession of this key allows anyone to run Consul as a trusted server or generate new valid certificates for the datacenter. Malicious actors may use this key to obtain access to all Consul data, including ACL tokens.
## Create server certificates
To authenticate Consul servers, servers are provided with a special certificate that lists `server.<domain>.<datacenter>` in the `Common Name` and `Subject Alternative Name` fields. When you enable [`tls.defaults.verify_server_hostname`](/consul/docs/agent/config/config-files#tls_default_verify_server_hostname), only agents that provide this certificate are allowed to boot as a server.
Without `tls.defaults.verify_server_hostname = true`, an attacker who compromises a Consul client agent can restart the agent as a server to get access to all the data in your datacenter. You must keep the server key private to protect your Consul data.
The following example creates a server certificate for datacenter `dc1` in the `consul` domain. If your datacenter or domain is different, modify the command to use the appropriate flag values.
```shell-session
$ consul tls cert create -server -dc=dc1 -domain=consul
==> WARNING: Server Certificates grants authority to become a
server and access all state in the cluster including root keys
and all ACL tokens. Do not distribute them to production hosts
that are not server nodes. Store them as securely as CA keys.
==> Using consul-agent-ca.pem and consul-agent-ca-key.pem
==> Saved dc1-server-consul-0.pem
==> Saved dc1-server-consul-0-key.pem
```
Repeat this process on the same node where you created the CA until the number of certificates is equal to the number of servers in the datacenter. You can run the command multiple times in a row, and it automatically increases the certificate and key numbers each time.
### Federated Consul datacenter
A federated Consul environment requires the server certificate to include the names of all Consul datacenters that are within the federated environment.
Use the `-additional-dnsname` flag to provide an additional DNS names for Subject Alternative Names. `localhost` is always included. You can provide this flag multiple times in a single command.
The following example creates a certificate for a federated environment containing two Consul datacenters named `dc1` and `dc2`.
```shell-session
$ consul tls cert create -server -dc dc1 -domain=consul -additional-dnsname="server.dc2.consul"
==> WARNING: Server Certificates grants authority to become a
server and access all state in the cluster including root keys
and all ACL tokens. Do not distribute them to production hosts
that are not server nodes. Store them as securely as CA keys.
==> Using consul-agent-ca.pem and consul-agent-ca-key.pem
==> Saved dc1-server-consul-0.pem
==> Saved dc1-server-consul-0-key.pem
```
## Configure server agents
After you generate the server certificates, distribute them to the Consul servers and modify the agent configuration file to include their local location.
Copy the following files to each Consul server:
- `consul-agent-ca.pem`: CA public certificate.
- `dc1-server-consul-0.pem`: Consul server node public certificate for first server node in the `dc1` datacenter in the `consul` domain.
- `dc1-server-consul-0-key.pem`: Consul server node private key for first server node in the `dc1` datacenter in the `consul` domain.
There are two methods for configuring Consul server agents depending on the way you want to distribute certificates to the client agents:
- The _auto encryption method_ uses the Consul Connect CA to generate client certificates and then automatically distributes them to all Consul clients.
- The _operator method_ requires you to manually generate client certificates and distribute them to each client agent individually.
We recommend using the auto-encryption method with the built-in CA because Consul can then automatically rotate certificates without requiring operator intervention.
Use the operator method if you need to use a third-party CA or need more fine-grained control over certificate management.
<Tabs>
<Tab heading="Auto-encryption method" group="auto">
<CodeTabs tabs={[ "HCL", "JSON" ]}>
```hcl
addresses = {
https = "0.0.0.0"
}
ports {
https = 8501
}
tls {
defaults {
ca_file = "consul-agent-ca.pem"
cert_file = "dc1-server-consul-0.pem"
key_file = "dc1-server-consul-0-key.pem"
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true
}
}
auto_encrypt {
allow_tls = true
}
```
```json
{
"addresses": {
"https" : "0.0.0.0"
},
"ports": {
"https" : 8501
},
"tls" : {
"defaults": {
"ca_file": "consul-agent-ca.pem",
"cert_file": "dc1-server-consul-0.pem",
"key_file": "dc1-server-consul-0-key.pem",
"verify_incoming": true,
"verify_outgoing": true,
"verify_server_hostname": true
}
},
"auto_encrypt" : {
"allow_tls" : true
}
}
```
</CodeTabs>
</Tab>
<Tab heading="Operator method" group="manual">
<CodeTabs tabs={[ "HCL", "JSON" ]}>
```hcl
addresses = {
https = "0.0.0.0"
}
ports {
https = 8501
}
tls {
defaults {
ca_file = "consul-agent-ca.pem"
cert_file = "dc1-server-consul-0.pem"
key_file = "dc1-server-consul-0-key.pem"
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true
}
}
```
```json
{
"addresses": {
"https": "0.0.0.0"
},
"ports": {
"https": 8501
},
"tls": {
"defaults": {
"ca_file": "consul-agent-ca.pem",
"cert_file": "dc1-server-consul-0.pem",
"key_file": "dc1-server-consul-0-key.pem",
"verify_incoming": true,
"verify_outgoing": true,
"verify_server_hostname": true
}
}
}
```
</CodeTabs>
</Tab>
</Tabs>
Consul does not enable TLS for HTTP unless the `https` port is assigned a port number greater than `0`. We recommend using `8501`, the default number for the `https` port, because this default is designed to work automatically with other tools.
## Start Consul servers
After you configure your servers, start the Consul process.
```shell-session
$ systemctl start consul
```
Your Consul servers can now communicate with TLS encryption for RPC and consensus.
## Configure client agents
Next, configure your client agents with the same method you used to [configure the server agent](#distribute-the-server-certificates)
<Tabs>
<Tab heading="Auto-encryption method" group="auto">
The only file you need on the local disk to configure the Consul client agents using auto-encryption is the CA certificate `consul-agent-ca-.pem`.
<CodeTabs tabs={[ "HCL", "JSON" ]}>
```hcl
addresses = {
https = "0.0.0.0"
}
ports {
https = 8501
}
tls {
defaults {
ca_file = "consul-agent-ca.pem"
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true
}
}
auto_encrypt = {
tls = true
}
```
```json
{
"addresses": {
"https": "0.0.0.0"
},
"ports": {
"https": 8501
},
"tls": {
"defaults": {
"ca_file": "consul-agent-ca.pem",
"verify_incoming": true,
"verify_outgoing": true,
"verify_server_hostname": true
}
},
"auto_encrypt" : {
"tls" : true
}
}
```
</CodeTabs>
</Tab>
<Tab heading="Operator method" group="manual">
On the node where you created the CA and server certificates, generate a client certificate with `consul tls cert create -client` command.
```shell-session
$ consul tls cert create -client -dc=dc1 -domain=consul
==> Using consul-agent-ca.pem and consul-agent-ca-key.pem
==> Saved dc1-client-consul-0.pem
==> Saved dc1-client-consul-0-key.pem
```
The client certificate is also signed by the same CA used for the server certificates, but it does not contain `server.<domain>.<datacenter>` in the `Common Name` and in the `Subject Alternative Name`. Because `verify_server_hostname` is enabled, a compromised client cannot use this certificate to start as a server and access datacenter data.
Distribute the client certificates and the CA certificate `consul-agent-ca.pem` to every Consul client in the datacenter. Then add them to the client agent configuration.
<CodeTabs tabs={[ "HCL", "JSON" ]}>
```hcl
addresses = {
https = "0.0.0.0"
}
ports {
https = 8501
}
tls {
defaults {
ca_file = "consul-agent-ca.pem"
cert_file = "dc1-client-consul-0.pem"
key_file = "dc1-client-consul-0-key.pem"
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true
}
}
```
```json
{
"addresses": {
"https": "0.0.0.0"
},
"ports": {
"https": 8501
},
"tls": {
"defaults": {
"ca_file": "consul-agent-ca.pem",
"cert_file": "dc1-client-consul-0.pem",
"key_file": "dc1-client-consul-0-key.pem",
"verify_incoming": true,
"verify_outgoing": true,
"verify_server_hostname": true
}
}
}
```
</CodeTabs>
</Tab>
</Tabs>
## Start Consul clients
After you configure each client, start the Consul process on the node.
```shell-session
$ systemctl start consul
```
Your client agents now communicate using mutual TLS encryption.
## Rotate TLS certificates
To maintain the security offered by TLS encryption, we recommend that you rotate TLS certificates often.
TLS certificates are part of [Consul's reloadable configuration](/consul/docs/agent/config#reloadable-configuration), so you do not need to restart the Consul agents when you renew certificates. As a result, there is no risk of downtime.
To rotate certificates for Consul server agents complete the following steps:
1. [Generate new certificates for all server agents](/consul/docs/security/encryption/mtls#create-server-certificates) to replace the old ones.
1. Distribute the new certificates to the server nodes.
1. Reload Consul configuration on each server with the `consul reload` command.
To rotate certificates for Consul client agents complete the following steps:
<Tabs>
<Tab heading="Auto-encryption method" group="auto">
When using the auto-encryption method, Consul automatically rotates the client certificates without operator intervention.
</Tab>
<Tab heading="Operator method" group="manual">
1. [Generate new certificates for all client agents](/consul/docs/security/encryption/mtls#configure-client-agents) to replace the old ones.
1. Distribute the new certificates to the client nodes.
1. Reload Consul configuration on all clients with `consul reload` command.
</Tab>
</Tabs>
## API, CLI, and UI interactions
The configuration snippets provided in this page are valid to configure complete mTLS for your Consul datacenter. This means that all interfaces require the client to provide a valid certificate in order to communicate with the Consul agent. This is valid for all requests, API, CLI, and UI.
Since Consul v1.12 it is possible to have different settings for:
- the HTTP protocol, used for the Consul's REST API, the CLI integration, and the UI
- the RPC protocol, used for internal communications between the Consul agents.
### Interact with Consul without a client certificate
If you want to avoid the need to present a valid client certificate every time you interact with Consul using the HTTP API, CLI, or UI, configure Consul to trust all incoming HTTPS connections by setting `tls.https.verify_incoming` to `false`. RPC communications are still mTLS encrypted.
<CodeTabs>
```hcl
addresses = {
https = "0.0.0.0"
}
ports {
https = 8501
http = -1
}
tls {
https {
ca_file = "/etc/consul.d/consul-agent-ca.pem"
cert_file = "/etc/consul.d/consul-agent.pem"
key_file = "/etc/consul.d/consul-agent-key.pem"
verify_incoming = false
verify_outgoing = true
}
internal_rpc {
ca_file = "/etc/consul.d/consul-agent-ca.pem"
cert_file = "/etc/consul.d/consul-agent.pem"
key_file = "/etc/consul.d/consul-agent-key.pem"
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true
}
}
```
```json
{
"addresses": {
"https": "0.0.0.0",
"http": "127.0.0.1",
},
"ports": {
"https": 8501,
"http": 8500
},
"tls": {
"https": {
"ca_file": "consul-agent-ca.pem",
"cert_file": "consul-agent.pem",
"key_file": "consul-agent-key.pem",
"verify_incoming": false,
"verify_outgoing": true
},
"internal_rpc": {
"ca_file": "consul-agent-ca.pem",
"cert_file": "consul-agent.pem",
"key_file": "consul-agent-key.pem",
"verify_incoming": true,
"verify_outgoing": true,
"verify_server_hostname": true
}
}
}
```
</CodeTabs>
### Use HTTP for local client interaction
If you want to avoid the need to present a valid client certificate or to use the CA certificate every time you interact with the local Consul agent, configure Consul to keep the HTTP listener active on the `localhost` interface only and set `tls.https.verify_incoming` to `false`. External requests to the API or UI are still protected by TLS encryption, but requests that originate locally do not need to present a client certificate. RPC communications are still mTLS encrypted.
<CodeTabs>
```hcl
addresses = {
https = "0.0.0.0"
http = "127.0.0.1"
}
ports {
https = 8501
http = 8500
}
tls {
https {
ca_file = "/etc/consul.d/consul-agent-ca.pem"
cert_file = "/etc/consul.d/consul-agent.pem"
key_file = "/etc/consul.d/consul-agent-key.pem"
verify_incoming = false
verify_outgoing = true
}
internal_rpc {
ca_file = "/etc/consul.d/consul-agent-ca.pem"
cert_file = "/etc/consul.d/consul-agent.pem"
key_file = "/etc/consul.d/consul-agent-key.pem"
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true
}
}
```
```json
{
"addresses": {
"https": "0.0.0.0",
"http": "127.0.0.1",
},
"ports": {
"https": 8501,
"http": 8500
},
"tls": {
"https": {
"ca_file": "consul-agent-ca.pem",
"cert_file": "consul-agent.pem",
"key_file": "consul-agent-key.pem",
"verify_incoming": false,
"verify_outgoing": true
},
"internal_rpc": {
"ca_file": "consul-agent-ca.pem",
"cert_file": "consul-agent.pem",
"key_file": "consul-agent-key.pem",
"verify_incoming": true,
"verify_outgoing": true,
"verify_server_hostname": true
}
}
}
```
</CodeTabs>
## Next steps
Your agents are now configured with mTLS for encrypted communication. With the auto encryption method, your
client certificates are managed by the server. With the operator method, you distributed all the certificates manually, but have a more flexible configuration.
Documentation for the commands used in this topic is available at [Consul agent configuration - TLS configuration reference](/consul/docs/agent/config/config-files#tls-configuration-reference) and the [`consul tls` CLI command reference](/consul/commands/tls).
To learn how to automate TLS certificate generation and rotation, refer to the [Generate mTLS Certificates for Consul with Vault](/consul/tutorials/operate-consul/vault-pki-consul-secure-tls) tutorial.
To continue securing your Consul deployment, add [gossip encryption](/consul/docs/security/encryption/gossip) and enable [Access Control Lists (ACLs)](/consul/docs/security/acl) with the default deny policy.

15
website/content/docs/security/index.mdx
Unescape View File

@ -5,11 +5,15 @@ description: >-
Security requirements and recommendations for Consul vary depending on workloads and environments. Learn how ACLs and encryption can protect access to and communication within your datacenter. Security requirements and recommendations for Consul vary depending on workloads and environments. Learn how ACLs and encryption can protect access to and communication within your datacenter.
--- ---
# Consul security
This topic describes the security requirements and recommendations for a Consul deployment.
## Security Models ## Security Models
Requirements and recommendations for operating a secure Consul deployment may vary drastically depending on your Requirements and recommendations for operating a secure Consul deployment may vary drastically depending on your
intended workloads, operating system, and environment. You can find detailed information about the various personas, intended workloads, operating system, and environment. You can find detailed information about the various personas,
recommendations, requirements, and threats [here](/consul/docs/security/security-models). recommendations, requirements, and threats in the [Security Models](/consul/docs/security/security-models) section.
## ACLs ## ACLs
@ -18,6 +22,9 @@ to data and APIs.
## Encryption ## Encryption
The Consul agent supports encrypting all of its network traffic. The exact method of encryption is described on the The Consul agent supports encryption for all of its network traffic. There are two separate encryption systems:
[encryption security page](/consul/docs/security/encryption). There are two separate encryption systems, one for gossip
traffic and one for HTTP + RPC. - A gossip encryption system
- An mTLS encryption system for HTTP and RPC
For more information about these two different encryption systems, as well as configuration guidance, refer to [Consul encryption](/consul/docs/security/encryption).

31
website/data/docs-nav-data.json
Unescape View File

@ -1078,6 +1078,23 @@
"title": "Overview", "title": "Overview",
"path": "security" "path": "security"
}, },
{
"title": "Security Models",
"routes": [
{
"title": "Overview",
"path": "security/security-models"
},
{
"title": "Core",
"path": "security/security-models/core"
},
{
"title": "Network Infrastructure Automation",
"path": "security/security-models/nia"
}
]
},
{ {
"title": "Access Control (ACLs)", "title": "Access Control (ACLs)",
"routes": [ "routes": [
@ -1188,22 +1205,18 @@
}, },
{ {
"title": "Encryption", "title": "Encryption",
"path": "security/encryption"
},
{
"title": "Security Models",
"routes": [ "routes": [
{ {
"title": "Overview", "title": "Overview",
"path": "security/security-models" "path": "security/encryption"
}, },
{ {
"title": "Core", "title": "Gossip encryption",
"path": "security/security-models/core" "path": "security/encryption/gossip"
}, },
{ {
"title": "Network Infrastructure Automation", "title": "mTLS",
"path": "security/security-models/nia" "path": "security/encryption/mtls"
} }
] ]
} }

Write Preview
Loading…
Cancel
Save