mirror of https://github.com/hashicorp/consul
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
263 lines
17 KiB
263 lines
17 KiB
--- |
|
layout: docs |
|
page_title: Service Mesh Certificate Authority - Overview |
|
description: >- |
|
Consul uses a certificate authority (CA) to generate, use, manage, sign, and store certificates for your service mesh. Learn about certificate management, including configuration, root cert rotation, cross-signing, and regenerating the CA. |
|
--- |
|
|
|
# Service Mesh Certificate Authority Overview |
|
|
|
Service mesh certificate management is done centrally through the Consul |
|
servers using the configured service mesh CA (Certificate Authority) provider. A CA provider |
|
manages root and intermediate certificates and performs certificate signing |
|
operations. The Consul leader orchestrates CA provider operations as necessary, |
|
such as when a service needs a new certificate or during CA rotation events. |
|
|
|
The CA provider abstraction enables Consul to support multiple systems for |
|
storing and signing certificates. Consul ships with a |
|
[built-in CA](/consul/docs/connect/ca/consul) which generates and stores the |
|
root certificate and private key on the Consul servers. Consul also has |
|
support for using |
|
[Vault as a CA](/consul/docs/connect/ca/vault). With Vault, the root certificate |
|
and private key material remain with the Vault cluster. |
|
|
|
## CA and Certificate relationship |
|
|
|
This diagram shows the relationship between the CA certificates in a Consul primary datacenter and a |
|
secondary Consul datacenter. |
|
|
|
 |
|
|
|
Leaf certificates are created for two purposes: |
|
- the Leaf Cert Service is used by envoy proxies in the mesh to perform mTLS with other |
|
services. |
|
- the Leaf Cert Client Agent is created by auto-encrypt and auto-config. It is used by |
|
client agents for HTTP API TLS, and for mTLS for RPC requests to servers. |
|
|
|
Any secondary datacenters use their CA provider to generate an intermediate certificate |
|
signing request (CSR) to be signed by the primary root CA. They receive an intermediate |
|
CA certificate, which is used to sign leaf certificates in the secondary datacenter. |
|
|
|
You can use different providers across primary and secondary datacenters. |
|
For example, an operator may use a Vault CA provider for extra security in the primary |
|
datacenter but choose to use the built-in CA provider in the secondary datacenter, which |
|
may not have a reachable Vault cluster. The following table compares the built-in and Vault providers. |
|
|
|
## CA Provider Comparison |
|
|
|
| | Consul built-in | Vault | |
|
|------------|------------------------------------|-----------------------------------------------------------------------------------| |
|
| Security | CA private keys are stored on disk | CA private keys are stored in Vault and are never exposed to Consul server agents | |
|
| Resiliency | No dependency on external systems. If Consul is available, it can sign certificates | Dependent on Vault availability | |
|
| Latency | Consul signs certificates locally | A network call to Vault is required to sign certificates | |
|
|
|
## CA Bootstrapping |
|
|
|
CA initialization happens automatically when a new Consul leader is elected |
|
as long as |
|
[service mesh is enabled](/consul/docs/connect/configuration#agent-configuration), |
|
and the CA system has not already been initialized. This initialization process |
|
will generate the initial root certificates and setup the internal Consul server |
|
state. |
|
|
|
For the initial bootstrap, the CA provider can be configured through the |
|
[Agent configuration](/consul/docs/agent/config/config-files#connect_ca_config). After |
|
initialization, the CA can only be updated through the |
|
[Update CA Configuration API endpoint](/consul/api-docs/connect/ca#update-ca-configuration). |
|
If a CA is already initialized, any changes to the CA configuration in the |
|
agent configuration file (including removing the configuration completely) |
|
will have no effect. |
|
|
|
If no specific provider is configured when service mesh is enabled, the built-in |
|
Consul CA provider will be used and a private key and root certificate will |
|
be generated automatically. |
|
|
|
## Viewing Root Certificates |
|
|
|
Root certificates can be queried with the |
|
[list CA Roots endpoint](/consul/api-docs/connect/ca#list-ca-root-certificates). |
|
With this endpoint, you can see the list of currently trusted root certificates. |
|
When a cluster first initializes, this will only list one trusted root. Multiple |
|
roots may appear as part of |
|
[rotation](#root-certificate-rotation). |
|
|
|
```shell-session |
|
$ curl http://localhost:8500/v1/connect/ca/roots |
|
{ |
|
"ActiveRootID": "31:6c:06:fb:49:94:42:d5:e4:55:cc:2e:27:b3:b2:2e:96:67:3e:7e", |
|
"TrustDomain": "36cb52cd-4058-f811-0432-6798a240c5d3.consul", |
|
"Roots": [ |
|
{ |
|
"ID": "31:6c:06:fb:49:94:42:d5:e4:55:cc:2e:27:b3:b2:2e:96:67:3e:7e", |
|
"Name": "Consul CA Root Cert", |
|
"SerialNumber": 7, |
|
"SigningKeyID": "19:45:8b:30:a1:45:84:ae:23:52:db:8d:1b:ff:a9:09:db:fc:2a:72:39:ae:da:11:53:f4:37:5c:de:d1:68:d8", |
|
"ExternalTrustDomain": "a1499528-fbf6-df7b-05e5-ae81e1873fc4", |
|
"NotBefore": "2018-06-06T17:35:25Z", |
|
"NotAfter": "2028-06-03T17:35:25Z", |
|
"RootCert": "-----BEGIN CERTIFICATE-----\nMIICmDCCAj6gAwIBAgIBBzAKBggqhkjOPQQDAjAWMRQwEgYDVQQDEwtDb25zdWwg\nQ0EgNzAeFw0xODA2MDYxNzM1MjVaFw0yODA2MDMxNzM1MjVaMBYxFDASBgNVBAMT\nC0NvbnN1bCBDQSA3MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEgo09lpx63bHw\ncSXeeoSpHpHgyzX1Q8ewJ3RUg6Ie8Howbs/QBz1y/kGxsF35HXij3YrqhgQyPPx4\nbQ8FH2YR4aOCAXswggF3MA4GA1UdDwEB/wQEAwIBhjAPBgNVHRMBAf8EBTADAQH/\nMGgGA1UdDgRhBF8xOTo0NTo4YjozMDphMTo0NTo4NDphZToyMzo1MjpkYjo4ZDox\nYjpmZjphOTowOTpkYjpmYzoyYTo3MjozOTphZTpkYToxMTo1MzpmNDozNzo1Yzpk\nZTpkMTo2ODpkODBqBgNVHSMEYzBhgF8xOTo0NTo4YjozMDphMTo0NTo4NDphZToy\nMzo1MjpkYjo4ZDoxYjpmZjphOTowOTpkYjpmYzoyYTo3MjozOTphZTpkYToxMTo1\nMzpmNDozNzo1YzpkZTpkMTo2ODpkODA/BgNVHREEODA2hjRzcGlmZmU6Ly8zNmNi\nNTJjZC00MDU4LWY4MTEtMDQzMi02Nzk4YTI0MGM1ZDMuY29uc3VsMD0GA1UdHgEB\n/wQzMDGgLzAtgiszNmNiNTJjZC00MDU4LWY4MTEtMDQzMi02Nzk4YTI0MGM1ZDMu\nY29uc3VsMAoGCCqGSM49BAMCA0gAMEUCIHl6UDdouw8Fzn/oDHputAxt3UFbVg/U\nvC6jWPuqqMwmAiEAkvMadtwjtNU7m/AQRJrj1LeG3eXw7dWO8SlI2fEs0yY=\n-----END CERTIFICATE-----\n", |
|
"IntermediateCerts": null, |
|
"Active": true, |
|
"PrivateKeyType": "", |
|
"PrivateKeyBits": 0, |
|
"CreateIndex": 8, |
|
"ModifyIndex": 8 |
|
} |
|
] |
|
} |
|
``` |
|
|
|
## CA Configuration |
|
|
|
After initialization, the CA provider configuration can be viewed with the |
|
[Get CA Configuration API endpoint](/consul/api-docs/connect/ca#get-ca-configuration). |
|
Consul will filter sensitive values from this endpoint depending on the |
|
provider in use, so the configuration may not be complete. |
|
|
|
```shell-session |
|
$ curl http://localhost:8500/v1/connect/ca/configuration |
|
{ |
|
"Provider": "consul", |
|
"Config": { |
|
"LeafCertTTL": "72h", |
|
"IntermediateCertTTL": "8760h" |
|
}, |
|
"CreateIndex": 5, |
|
"ModifyIndex": 5 |
|
} |
|
``` |
|
|
|
The CA provider can be reconfigured using the |
|
[Update CA Configuration API endpoint](/consul/api-docs/connect/ca#update-ca-configuration). |
|
Specific options for reconfiguration can be found in the specific |
|
CA provider documentation in the sidebar to the left. |
|
|
|
## Root Certificate Rotation |
|
|
|
Whenever the CA's configuration is updated in a way that causes the root key to |
|
change, a special rotation process will be triggered in order to smoothly |
|
transition to the new certificate. This rotation is automatically orchestrated |
|
by Consul. |
|
|
|
~> If the current CA Provider doesn't support cross-signing, this process can't |
|
be followed. See [Forced Rotation Without |
|
Cross-Signing](#forced-rotation-without-cross-signing). |
|
|
|
This also automatically occurs when a completely different CA provider is |
|
configured (since this changes the root key). Therefore, this automatic rotation |
|
process can also be used to cleanly transition between CA providers. For example, |
|
updating the service mesh to use Vault instead of the built-in CA. |
|
|
|
During rotation, an intermediate CA certificate is requested from the new root, |
|
which is then cross-signed by the old root. This cross-signed certificate is |
|
then distributed alongside any newly-generated leaf certificates used by the |
|
proxies once the new root becomes active, and provides a chain of trust back to |
|
the old root certificate in the event that a certificate signed by the new root |
|
is presented to a proxy that has not yet updated its bundle of trusted root CA |
|
certificates to include the new root. |
|
|
|
After the cross-signed certificate has been successfully generated and the new root |
|
certificate or CA provider has been set up, the new root becomes the active one |
|
and is immediately used for signing any new incoming certificate requests. |
|
|
|
If we check the [list CA roots |
|
endpoint](/consul/api-docs/connect/ca#list-ca-root-certificates) after updating the |
|
configuration with a new root certificate, we can see both the old and new root |
|
certificates are present, and the currently active root has an intermediate |
|
certificate which has been generated and cross-signed automatically by the old |
|
root during the rotation process: |
|
|
|
```shell-session |
|
$ curl localhost:8500/v1/connect/ca/roots |
|
{ |
|
"ActiveRootID": "d2:2c:41:94:1e:50:04:ea:86:fc:08:d6:b0:45:a4:af:8a:eb:76:a0", |
|
"TrustDomain": "36cb52cd-4058-f811-0432-6798a240c5d3.consul", |
|
"Roots": [ |
|
{ |
|
"ID": "31:6c:06:fb:49:94:42:d5:e4:55:cc:2e:27:b3:b2:2e:96:67:3e:7e", |
|
"Name": "Consul CA Root Cert", |
|
"SerialNumber": 7, |
|
"SigningKeyID": "19:45:8b:30:a1:45:84:ae:23:52:db:8d:1b:ff:a9:09:db:fc:2a:72:39:ae:da:11:53:f4:37:5c:de:d1:68:d8", |
|
"ExternalTrustDomain": "a1499528-fbf6-df7b-05e5-ae81e1873fc4", |
|
"NotBefore": "2018-06-06T17:35:25Z", |
|
"NotAfter": "2028-06-03T17:35:25Z", |
|
"RootCert": "-----BEGIN CERTIFICATE-----\nMIICmDCCAj6gAwIBAgIBBzAKBggqhkjOPQQDAjAWMRQwEgYDVQQDEwtDb25zdWwg\nQ0EgNzAeFw0xODA2MDYxNzM1MjVaFw0yODA2MDMxNzM1MjVaMBYxFDASBgNVBAMT\nC0NvbnN1bCBDQSA3MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEgo09lpx63bHw\ncSXeeoSpHpHgyzX1Q8ewJ3RUg6Ie8Howbs/QBz1y/kGxsF35HXij3YrqhgQyPPx4\nbQ8FH2YR4aOCAXswggF3MA4GA1UdDwEB/wQEAwIBhjAPBgNVHRMBAf8EBTADAQH/\nMGgGA1UdDgRhBF8xOTo0NTo4YjozMDphMTo0NTo4NDphZToyMzo1MjpkYjo4ZDox\nYjpmZjphOTowOTpkYjpmYzoyYTo3MjozOTphZTpkYToxMTo1MzpmNDozNzo1Yzpk\nZTpkMTo2ODpkODBqBgNVHSMEYzBhgF8xOTo0NTo4YjozMDphMTo0NTo4NDphZToy\nMzo1MjpkYjo4ZDoxYjpmZjphOTowOTpkYjpmYzoyYTo3MjozOTphZTpkYToxMTo1\nMzpmNDozNzo1YzpkZTpkMTo2ODpkODA/BgNVHREEODA2hjRzcGlmZmU6Ly8zNmNi\nNTJjZC00MDU4LWY4MTEtMDQzMi02Nzk4YTI0MGM1ZDMuY29uc3VsMD0GA1UdHgEB\n/wQzMDGgLzAtgiszNmNiNTJjZC00MDU4LWY4MTEtMDQzMi02Nzk4YTI0MGM1ZDMu\nY29uc3VsMAoGCCqGSM49BAMCA0gAMEUCIHl6UDdouw8Fzn/oDHputAxt3UFbVg/U\nvC6jWPuqqMwmAiEAkvMadtwjtNU7m/AQRJrj1LeG3eXw7dWO8SlI2fEs0yY=\n-----END CERTIFICATE-----\n", |
|
"IntermediateCerts": null, |
|
"Active": false, |
|
"PrivateKeyType": "", |
|
"PrivateKeyBits": 0, |
|
"CreateIndex": 8, |
|
"ModifyIndex": 24 |
|
}, |
|
{ |
|
"ID": "d2:2c:41:94:1e:50:04:ea:86:fc:08:d6:b0:45:a4:af:8a:eb:76:a0", |
|
"Name": "Consul CA Root Cert", |
|
"SerialNumber": 16238269036752183483, |
|
"SigningKeyID": "", |
|
"ExternalTrustDomain": "a1499528-fbf6-df7b-05e5-ae81e1873fc4", |
|
"NotBefore": "2018-06-06T17:37:03Z", |
|
"NotAfter": "2028-06-03T17:37:03Z", |
|
"RootCert": "-----BEGIN CERTIFICATE-----\nMIIDijCCAnKgAwIBAgIJAOFZ66em1qC7MA0GCSqGSIb3DQEBCwUAMGIxCzAJBgNV\nBAYTAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlhMRYwFAYDVQQHDA1TYW4gRnJhbmNp\nc2NvMRIwEAYDVQQKDAlIYXNoaUNvcnAxEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0x\nODA2MDYxNzM3MDNaFw0yODA2MDMxNzM3MDNaMGIxCzAJBgNVBAYTAlVTMRMwEQYD\nVQQIDApDYWxpZm9ybmlhMRYwFAYDVQQHDA1TYW4gRnJhbmNpc2NvMRIwEAYDVQQK\nDAlIYXNoaUNvcnAxEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEB\nBQADggEPADCCAQoCggEBAK6ostXN6W093EpI3RDNQDwC1Gq3lPNoodL5XRaVVIBU\n3X5iC+Ttk02p67cHUguh4ZrWr3o3Dzxm+gKK0lfZLW0nNYNPAIGZWQD9zVSx1Lqt\n8X0pd+fhMV5coQrh3YIG/vy17IBTSBuRUX0mXOKjOeJJlrw1HQZ8pfm7WX6LFul2\nXszvgn5K1XR+9nhPy6K2bv99qsY0sm7AqCS2BjYBW8QmNngJOdLPdhyFh7invyXe\nPqgujc/KoA3P6e3/G7bJZ9+qoQMK8uwD7PxtA2hdQ9t0JGPsyWgzhwfBxWdBWRzV\nRvVi6Yu2tvw3QrjdeKQ5Ouw9FUb46VnTU7jTO974HjkCAwEAAaNDMEEwPwYDVR0R\nBDgwNoY0c3BpZmZlOi8vMzZjYjUyY2QtNDA1OC1mODExLTA0MzItNjc5OGEyNDBj\nNWQzLmNvbnN1bDANBgkqhkiG9w0BAQsFAAOCAQEATHgCro9VXj7JbH/tlB6f/KWf\n7r98+rlUE684ZRW9XcA9uUA6y265VPnemsC/EykPsririoh8My1jVPuEfgMksR39\n9eMDJKfutvSpLD1uQqZE8hu/hcYyrmQTFKjW71CfGIl/FKiAg7wXEw2ljLN9bxNv\nGG118wrJyMZrRvFjC2QKY025QQSJ6joNLFMpftsZrJlELtRV+nx3gMabpiDRXhIw\nJM6ti26P1PyVgGRPCOG10v+OuUtwe0IZoOqWpPJN8jzSuqZWf99uolkG0xuqLNz6\nd8qvTp1YF9tTmysgvdeGALez/02HTF035RVTsQfH9tM/+4yG1UnmjLpz3p4Fow==\n-----END CERTIFICATE-----", |
|
"IntermediateCerts": [ |
|
"-----BEGIN CERTIFICATE-----\nMIIDTzCCAvWgAwIBAgIBFzAKBggqhkjOPQQDAjAWMRQwEgYDVQQDEwtDb25zdWwg\nQ0EgNzAeFw0xODA2MDYxNzM3MDNaFw0yODA2MDMxNzM3MDNaMGIxCzAJBgNVBAYT\nAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlhMRYwFAYDVQQHDA1TYW4gRnJhbmNpc2Nv\nMRIwEAYDVQQKDAlIYXNoaUNvcnAxEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJ\nKoZIhvcNAQEBBQADggEPADCCAQoCggEBAK6ostXN6W093EpI3RDNQDwC1Gq3lPNo\nodL5XRaVVIBU3X5iC+Ttk02p67cHUguh4ZrWr3o3Dzxm+gKK0lfZLW0nNYNPAIGZ\nWQD9zVSx1Lqt8X0pd+fhMV5coQrh3YIG/vy17IBTSBuRUX0mXOKjOeJJlrw1HQZ8\npfm7WX6LFul2Xszvgn5K1XR+9nhPy6K2bv99qsY0sm7AqCS2BjYBW8QmNngJOdLP\ndhyFh7invyXePqgujc/KoA3P6e3/G7bJZ9+qoQMK8uwD7PxtA2hdQ9t0JGPsyWgz\nhwfBxWdBWRzVRvVi6Yu2tvw3QrjdeKQ5Ouw9FUb46VnTU7jTO974HjkCAwEAAaOC\nARswggEXMGgGA1UdDgRhBF8xOTo0NTo4YjozMDphMTo0NTo4NDphZToyMzo1Mjpk\nYjo4ZDoxYjpmZjphOTowOTpkYjpmYzoyYTo3MjozOTphZTpkYToxMTo1MzpmNDoz\nNzo1YzpkZTpkMTo2ODpkODBqBgNVHSMEYzBhgF8xOTo0NTo4YjozMDphMTo0NTo4\nNDphZToyMzo1MjpkYjo4ZDoxYjpmZjphOTowOTpkYjpmYzoyYTo3MjozOTphZTpk\nYToxMTo1MzpmNDozNzo1YzpkZTpkMTo2ODpkODA/BgNVHREEODA2hjRzcGlmZmU6\nLy8zNmNiNTJjZC00MDU4LWY4MTEtMDQzMi02Nzk4YTI0MGM1ZDMuY29uc3VsMAoG\nCCqGSM49BAMCA0gAMEUCIBp46tRDot7GFyDXu7egq7lXBvn+UUHD5MmlFvdWmtnm\nAiEAwKBzEMcLd5kCBgFHNGyksRAMh/AGdEW859aL6z0u4gM=\n-----END CERTIFICATE-----\n" |
|
], |
|
"Active": true, |
|
"PrivateKeyType": "", |
|
"PrivateKeyBits": 0, |
|
"CreateIndex": 24, |
|
"ModifyIndex": 24 |
|
} |
|
] |
|
} |
|
``` |
|
|
|
The old root certificate will be automatically removed once enough time has elapsed |
|
for any leaf certificates signed by it to expire. |
|
|
|
### Forced Rotation Without Cross-Signing |
|
|
|
If the CA provider that is currently in use does not support cross-signing, then |
|
attempts to change the root key or CA provider will fail. This is to ensure |
|
operators don't make the change without understanding that there is additional |
|
risk involved. |
|
|
|
It is possible to force the change to happen anyway by setting the |
|
`ForceWithoutCrossSigning` field in the CA configuration to `true`. |
|
|
|
The downside is that all new certificates will immediately start being signed |
|
with the new root key, but it will take some time for agents throughout the |
|
cluster to observe the root CA change and reconfigure applications and proxies |
|
to accept certificates signed by this new root. This will mean connections made |
|
with a new certificate may fail for a short period after the CA change. |
|
|
|
Typically all connected agents will have observed the new roots within seconds |
|
even in a large deployment so the impact should be contained. But it is possible |
|
for a disconnected, overloaded or misconfigured agent to not see the new root |
|
for an unbounded amount of time during which new connections to services on that |
|
host will fail. The issue will resolve as soon as the agent can reconnect to |
|
servers. |
|
|
|
Currently both Consul and Vault CA providers _do_ support cross signing. As more |
|
providers are added this documentation will list any that this section applies |
|
to. |
|
|
|
### Recovering From Expired Certificates |
|
If the built-in CA provider is misconfigured or unavailable, Consul service mesh requests eventually |
|
stop functioning due to expiration of intermediate and root certificates. To recover manually, use the |
|
[CLI helper](/consul/commands/tls/ca#consul-tls-ca-create) to generate CA certificates. |
|
|
|
|
|
#### Example - Regenerating the built in CA |
|
```shell-session |
|
$ consul tls ca create -cluster-id test -common-name "Consul Agent CA" -days=365 -domain consul |
|
==> Saved consul-agent-ca.pem |
|
==> Saved consul-agent-ca-key.pem |
|
``` |
|
The example above generates a new CA with a validity of 365 days. The cluster-id argument is specific |
|
to each cluster and can be looked up by examining the `TrustDomain` field in |
|
the [List CA Roots](/consul/api-docs/connect/ca#list-ca-root-certificates) endpoint. |
|
|
|
The contents of the generated cert and private key files from the above step should then be used with |
|
the [Update CA Configuration](/consul/api-docs/connect/ca#update-ca-configuration) endpoint. Once the CA configuration is |
|
updated on the primary datacenter, all secondary datacenters will pick up the changes and regenerate their intermediate |
|
and leaf certificates, after which any new requests that require certificate verification will succeed.
|
|
|