mirror of https://github.com/hashicorp/consul
Luke Kysow
5 years ago
committed by
GitHub
24 changed files with 962 additions and 186 deletions
@ -1,18 +0,0 @@
|
||||
--- |
||||
layout: docs |
||||
page_title: Consul on Azure Cloud |
||||
sidebar_title: Azure Kubernetes Service (AKS) |
||||
description: Consul can run directly on Azure Kubernetes Service (AKS). |
||||
--- |
||||
|
||||
# Consul on Azure Kubernetes Service |
||||
|
||||
Consul can run directly on Azure Kubernetes Service (AKS). To get hands-on experience, you can follow the [Deploy Consul on Azure Kubernetes Service (AKS)](https://learn.hashicorp.com/consul/kubernetes/azure-k8s?utm_source=consul.io&utm_medium=docs&utm_content=k8s&utm_term=aks) guide. |
||||
|
||||
The Learn guide includes: |
||||
|
||||
- AKS configuration recommendations |
||||
- How to install Consul with the official Helm chart |
||||
- Deploying two services in the Consul Connect service mesh. |
||||
|
||||
To complete this guide, you will need a Azure Cloud account. |
||||
@ -1,19 +0,0 @@
|
||||
--- |
||||
layout: 'docs' |
||||
page_title: 'Running Consul on Amazon EKS' |
||||
sidebar_title: 'Amazon Elastic Kubernetes Service (EKS)' |
||||
description: |- |
||||
Consul can run directly on Amazon Elastic Kubernetes Service (EKS). |
||||
--- |
||||
|
||||
# Consul on Amazon Elastic Kubernetes Service |
||||
|
||||
Consul can run directly on Amazon Elastic Kubernetes Service (EKS). To get hands-on experience, you can use the [Deploy Consul on Amazon Elastic Kubernetes Service (EKS)](https://learn.hashicorp.com/consul/kubernetes/aws-k8s?utm_source=consul.io&utm_medium=docs&utm_content=k8s&utm_term=eks) guide. |
||||
|
||||
The Learn guide includes: |
||||
|
||||
- EKS configuration recommendations |
||||
- How to install Consul on Amazon EKS with the official Helm chart |
||||
- Accessing Consul with the Consul UI, CLI, and API. |
||||
|
||||
You will need a Amazon Web Services account to complete this guide. |
||||
@ -1,18 +0,0 @@
|
||||
--- |
||||
layout: docs |
||||
page_title: Running Consul on Google Cloud |
||||
sidebar_title: Google Kubernetes Service (GKE) |
||||
description: Consul can run directly on Google Kubernetes Engine (GKE). |
||||
--- |
||||
|
||||
# Consul on Google Kubernetes Engine |
||||
|
||||
Consul can run directly on Google Kubernetes Engine (GKE). To get hands-on experience, you can use the [Deploy Consul on Google Kubernetes Engine (GKE)](https://learn.hashicorp.com/consul/kubernetes/google-cloud-k8s?utm_source=consul.io&utm_medium=docs&utm_content=k8s&utm_term=gke) guide. |
||||
|
||||
The Learn guide includes: |
||||
|
||||
- GKE configuration recommendations |
||||
- How to install Consul on GKE with the official Helm chart |
||||
- Accessing Consul with the Consul UI, CLI, and API. |
||||
|
||||
You will need a Google Cloud account to complete this guide. |
||||
@ -1,16 +0,0 @@
|
||||
--- |
||||
layout: docs |
||||
page_title: Running Consul on Minikube |
||||
sidebar_title: Minikube |
||||
description: Consul can run directly on Minikube for testing. |
||||
--- |
||||
|
||||
# Consul on Minikube |
||||
|
||||
Consul can run directly on Minikube for testing. To get hands-on experience, you can use the [Consul on Minikube via Helm](https://learn.hashicorp.com/consul/kubernetes/minikube?utm_source=consul.io&utm_medium=docs&utm_content=k8s&utm_term=mk) guide. |
||||
|
||||
The Learn guide includes how to: |
||||
|
||||
- Install Consul on Minikube with the official Helm chart. After |
||||
- Access Consul with the Consul UI, CLI, and API |
||||
- Deploy services in the Consul Connect service mesh. |
||||
@ -0,0 +1,402 @@
|
||||
--- |
||||
layout: docs |
||||
page_title: Federation Between Kubernetes Clusters |
||||
sidebar_title: Federation Between Kubernetes Clusters |
||||
description: >- |
||||
Federating multiple Kubernetes clusters. |
||||
--- |
||||
|
||||
# Federation Between Kubernetes Clusters |
||||
|
||||
-> **1.8.0+:** This feature is available in Consul versions 1.8.0 and higher |
||||
|
||||
~> This topic requires familiarity with [Mesh Gateways](/docs/connect/mesh_gateway) and [WAN Federation Via Mesh Gateways](/docs/connect/wan-federation-via-mesh-gateways). |
||||
|
||||
-> Looking for a step-by-step guide? Please follow our Learn Guide: [Secure and Route Service Mesh Communication Across Kubernetes](https://learn.hashicorp.com/consul/kubernetes/mesh-gateways). |
||||
|
||||
This page describes how to federate multiple Kubernetes clusters. See [Multi-Cluster Overview](/docs/k8s/installation/multi-cluster/overview) |
||||
for more information on use-cases and how it works. |
||||
|
||||
## Primary Datacenter |
||||
|
||||
Consul treats each Kubernetes cluster as a separate Consul datacenter. |
||||
In order to federate clusters, one cluster must be designated the |
||||
primary datacenter. This datacenter will be |
||||
responsible for creating the certificate authority that signs the TLS certificates |
||||
Connect uses to encrypt and authorize traffic. It also handles validating global ACL tokens. All other clusters |
||||
that are federated are considered secondaries. |
||||
|
||||
#### First Time Installation |
||||
|
||||
If you haven't installed Consul on your cluster, continue reading below. If you've |
||||
already installed Consul on a cluster and want to upgrade it to |
||||
support federation, see [Upgrading An Existing Cluster](#upgrading-an-existing-cluster). |
||||
|
||||
You will need to use the following `config.yaml` file for your primary cluster, |
||||
with the possible modifications listed below. |
||||
|
||||
```yaml |
||||
global: |
||||
name: consul |
||||
image: consul:1.8.0-beta1 |
||||
datacenter: dc1 |
||||
|
||||
# TLS configures whether Consul components use TLS. |
||||
tls: |
||||
# TLS must be enabled for federation in Kubernetes. |
||||
enabled: true |
||||
|
||||
federation: |
||||
enabled: true |
||||
# This will cause a Kubernetes secret to be created that |
||||
# can be imported by secondary datacenters to configure them |
||||
# for federation. |
||||
createFederationSecret: true |
||||
|
||||
acls: |
||||
manageSystemACLs: true |
||||
# If ACLs are enabled, we must create a token for secondary |
||||
# datacenters to replicate ACLs. |
||||
createReplicationToken: true |
||||
|
||||
# Gossip encryption secures the protocol Consul uses to quickly |
||||
# discover new nodes and detect failure. |
||||
gossipEncryption: |
||||
secretName: consul-gossip-encryption-key |
||||
secretKey: key |
||||
|
||||
connectInject: |
||||
# Consul Connect service mesh must be enabled for federation. |
||||
enabled: true |
||||
|
||||
meshGateway: |
||||
# Mesh gateways are gateways between datacenters. They must be enabled |
||||
# for federation in Kubernetes since the communication between datacenters |
||||
# goes through the mesh gateways. |
||||
enabled: true |
||||
``` |
||||
|
||||
Modifications: |
||||
|
||||
1. Note the Docker image: |
||||
```yaml |
||||
global: |
||||
image: consul:1.8.0-beta1 |
||||
``` |
||||
Kubernetes support for federation is currently only available in Consul 1.8.0 beta. |
||||
1. The Consul datacenter name is `dc1`. The datacenter name in each federated |
||||
cluster **must be unique**. |
||||
1. ACLs are enabled in the above config file. They can be disabled by setting: |
||||
```yaml |
||||
global: |
||||
acls: |
||||
manageSystemACLs: false |
||||
createReplicationToken: false |
||||
``` |
||||
ACLs secure Consul by requiring every API call to present an ACL token that |
||||
is validated to ensure it has the proper permissions. If you are only testing Consul, |
||||
this is not required. |
||||
1. Gossip encryption is enabled in the above config file. To disable it, comment |
||||
out or delete the `gossipEncryption` key: |
||||
|
||||
```yaml |
||||
global: |
||||
# gossipEncryption: |
||||
# secretName: consul-gossip-encryption-key |
||||
# secretKey: key |
||||
``` |
||||
|
||||
Gossip encryption encrypts the communication layer used to discover other |
||||
nodes in the cluster and report on failure. If you are only testing Consul, |
||||
this is not required. |
||||
|
||||
**NOTE:** This config assumes you've already |
||||
created a Kubernetes secret called `consul-gossip-encryption-key`. See |
||||
[the docs for this setting](/docs/k8s/helm#v-global-gossipencryption) for |
||||
more information on how to create this secret. |
||||
|
||||
1. The default mesh gateway configuration |
||||
creates a Kubernetes Load Balancer service. If you wish to customize the |
||||
mesh gateway, for example using a Node Port service or a custom DNS entry, |
||||
see the [Helm reference](/docs/k8s/helm#v-meshgateway) for that setting. |
||||
|
||||
With your `config.yaml` ready to go, follow our [Installation Guide](/docs/k8s/installation/overview |
||||
to install Consul on your primary cluster and then skip ahead to the [Federation Secret](#federation-secret) |
||||
section. |
||||
|
||||
#### Upgrading An Existing Cluster |
||||
|
||||
If you have an existing cluster, you will need to upgrade it to ensure it has |
||||
the following config: |
||||
|
||||
```yaml |
||||
global: |
||||
image: consul:1.8.0-beta1 |
||||
tls: |
||||
enabled: true |
||||
federation: |
||||
enabled: true |
||||
createFederationSecret: true |
||||
acls: |
||||
manageSystemACLs: true |
||||
createReplicationToken: true |
||||
meshGateway: |
||||
enabled: true |
||||
``` |
||||
|
||||
1. `global.image` must be set to `consul:1.8.0-beta1` because Kubernetes support |
||||
for federation is currently only available in Consul 1.8.0 beta. |
||||
1. `global.tls.enabled` must be `true`. See [Configuring TLS on an Existing Cluster](/docs/k8s/operations/tls-on-existing-cluster) |
||||
for more information on safely upgrading a cluster to use TLS. |
||||
|
||||
If you've set `enableAutoEncrypt: true`, this is also supported. |
||||
|
||||
1. `global.federation.enabled` must be set to `true`. This is a new config setting. |
||||
1. If using ACLs, you'll already have `global.acls.manageSystemACLs: true`. For the |
||||
primary cluster, you'll also need to set `global.acls.createReplicationToken: true`. |
||||
This ensures that an ACL token is created that secondary clusters can use to authenticate |
||||
with the primary. |
||||
1. Mesh Gateways are enabled with the default configuration. The default configuration |
||||
creates a Kubernetes Load Balancer service. If you wish to customize the |
||||
mesh gateway, see the [Helm reference](/docs/k8s/helm#v-meshgateway) for that setting. |
||||
|
||||
With the above settings added to your existing config, follow the [Upgrading](/localhost:3000/docs/k8s/operations/upgrading) |
||||
guide to upgrade your cluster and then come back to the [Federation Secret](#federation-secret) section. |
||||
|
||||
## Federation Secret |
||||
|
||||
The federation secret is a Kubernetes secret containing information needed |
||||
for secondary datacenters/clusters to federate with the primary. This secret is created |
||||
automatically by setting: |
||||
|
||||
```yaml |
||||
global: |
||||
federation: |
||||
createFederationSecret: true |
||||
``` |
||||
|
||||
After the installation into your primary cluster you will need to export |
||||
this secret: |
||||
|
||||
```sh |
||||
$ kubectl get secret consul-federation -o yaml > consul-federation-secret.yaml |
||||
``` |
||||
|
||||
!> **Security note:** The federation secret makes it possible to gain |
||||
full admin privileges in Consul. This secret must be kept securely, i.e. |
||||
it should be deleted from your filesystem after importing it to your secondary |
||||
cluster and you should use RBAC permissions to ensure only administrators |
||||
can read it from Kubernetes. |
||||
|
||||
~> **Secret doesn't exist?** If you haven't set `global.name` to `consul` then the name of the secret will |
||||
be your Helm release name suffixed with `-consul-federation` e.g. `helm-release-consul-federation`. |
||||
|
||||
Now you're ready to import the secret into your secondary cluster(s). |
||||
|
||||
Switch `kubectl` context to your secondary Kubernetes cluster. In this example |
||||
our context for our secondary cluster is `dc2`: |
||||
|
||||
```sh |
||||
$ kubectl config use-context dc2 |
||||
Switched to context "dc2". |
||||
``` |
||||
|
||||
And import the secret: |
||||
|
||||
```sh |
||||
$ kubectl apply -f consul-federation-secret.yaml |
||||
secret/consul-federation configured |
||||
``` |
||||
|
||||
#### Federation Secret Contents |
||||
|
||||
The automatically generated federation secret contains: |
||||
|
||||
- **Server certificate authority certificate** - This is the certificate authority |
||||
used to sign Consul server-to-server communication. This is required by secondary |
||||
clusters because they must communicate with the Consul servers in the primary cluster. |
||||
- **Server certificate authority key** - This is the signing key for the server certificate |
||||
authority. This is required by secondary clusters because they need to create |
||||
server certificates for each Consul server using the same certificate authority |
||||
as the primary. |
||||
|
||||
!> **Security note:** The certificate authority key would enable an attacker to compromise Consul, |
||||
it should be kept securely. |
||||
- **Consul server config** - This is a JSON snippet that must be used as part of the server config for secondary datacenters. |
||||
It sets: |
||||
|
||||
- [`primary_datacenter`](/docs/agent/options.html#primary_datacenter) to the name of the primary datacenter. |
||||
- [`primary_gateways`](/docs/agent/options.html#primary_gateways) to an array of IPs or hostnames |
||||
for the mesh gateways in the primary datacenter. These are the addresses that |
||||
Consul servers in secondary clusters will use to communicate with the primary |
||||
datacenter. |
||||
|
||||
Even if there are multiple secondary datacenters, only the primary gateways |
||||
need to be configured. Upon first connection with a primary datacenter, the |
||||
addresses for other secondary datacenters will be discovered. |
||||
|
||||
- **ACL replication token** - If ACLs are enabled, secondary datacenters need |
||||
an ACL token in order to authenticate with the primary datacenter. This ACL |
||||
token is also used to replicate ACLs from the primary datacenter so that |
||||
components in each datacenter can authenticate with one another. |
||||
- **Gossip encryption key** - If gossip encryption is enabled, secondary datacenters |
||||
need the gossip encryption key in order to be part of the gossip pool. |
||||
Gossip is the method by which Consul discovers the addresses and health of other |
||||
nodes. |
||||
|
||||
!> **Security note:** This gossip encryption key would enable an attacker to compromise Consul, |
||||
it should be kept securely. |
||||
|
||||
## Secondary Cluster(s) |
||||
|
||||
With the primary cluster up and running, and the [federation secret](/docs/installation/multi-cluster#federation-secret) imported |
||||
into the secondary cluster, we can now install Consul into the secondary |
||||
cluster. |
||||
|
||||
You will need to use the following `config.yaml` file for your secondary cluster(s), |
||||
with the possible modifications listed below. |
||||
|
||||
-> **NOTE: ** You must use a separate Helm config file for each cluster (primary and secondaries) since their |
||||
settings are different. |
||||
|
||||
```yaml |
||||
global: |
||||
name: consul |
||||
image: consul:1.8.0-beta1 |
||||
datacenter: dc2 |
||||
tls: |
||||
enabled: true |
||||
|
||||
# Here we're using the shared certificate authority from the primary |
||||
# datacenter that was exported via the federation secret. |
||||
caCert: |
||||
secretName: consul-federation |
||||
secretKey: caCert |
||||
caKey: |
||||
secretName: consul-federation |
||||
secretKey: caKey |
||||
|
||||
acls: |
||||
manageSystemACLs: true |
||||
|
||||
# Here we're importing the replication token that was |
||||
# exported from the primary via the federation secret. |
||||
replicationToken: |
||||
secretName: consul-federation |
||||
secretKey: replicationToken |
||||
|
||||
federation: |
||||
enabled: true |
||||
gossipEncryption: |
||||
secretName: consul-federation |
||||
secretKey: gossipEncryptionKey |
||||
connectInject: |
||||
enabled: true |
||||
meshGateway: |
||||
enabled: true |
||||
server: |
||||
# Here we're including the server config exported from the primary |
||||
# via the federation secret. This config includes the addresses of |
||||
# the primary datacenter's mesh gateways so Consul can begin federation. |
||||
extraVolumes: |
||||
- type: secret |
||||
name: consul-federation |
||||
items: |
||||
- key: serverConfigJSON |
||||
path: config.json |
||||
load: true |
||||
``` |
||||
|
||||
Modifications: |
||||
|
||||
1. Note `global.image: consul:1.8.0-beta1`. Kubernetes support for federation is currently only available |
||||
in Consul 1.8.0 beta. |
||||
1. The Consul datacenter name is `dc2`. The primary datacenter's name was `dc1`. |
||||
The datacenter name in **each** federated cluster **must be unique**. |
||||
1. ACLs are enabled in the above config file. They can be disabled by removing |
||||
the whole `acls` block: |
||||
|
||||
```yaml |
||||
acls: |
||||
manageSystemACLs: false |
||||
replicationToken: |
||||
secretName: consul-federation |
||||
secretKey: replicationToken |
||||
``` |
||||
|
||||
If ACLs are enabled in one datacenter, they must be enabled in all datacenters |
||||
because in order to communicate with that one datacenter ACL tokens are required. |
||||
|
||||
1. Gossip encryption is enabled in the above config file. To disable it, don't |
||||
set the `gossipEncryption` key: |
||||
|
||||
```yaml |
||||
global: |
||||
# gossipEncryption: |
||||
# secretName: consul-federation |
||||
# secretKey: gossipEncryptionKey |
||||
``` |
||||
|
||||
If gossip encryption is enabled in one datacenter, it must be enabled in all datacenters |
||||
because in order to communicate with that one datacenter the encryption key is required. |
||||
|
||||
1. The default mesh gateway configuration |
||||
creates a Kubernetes Load Balancer service. If you wish to customize the |
||||
mesh gateway, for example using a Node Port service or a custom DNS entry, |
||||
see the [Helm reference](/docs/k8s/helm#v-meshgateway) for that setting. |
||||
|
||||
With your `config.yaml` ready to go, follow our [Installation Guide](/docs/k8s/installation/overview) |
||||
to install Consul on your secondary cluster(s). |
||||
|
||||
## Verifying Federation |
||||
|
||||
To verify that both datacenters are federated, run the |
||||
`consul members -wan` command on one of the Consul server pods: |
||||
|
||||
```sh |
||||
$ kubectl exec statefulset/consul-server -- consul members -wan |
||||
Node Address Status Type Build Protocol DC Segment |
||||
consul-server-0.dc1 10.32.4.216:8302 alive server 1.8.0 2 dc1 <all> |
||||
consul-server-0.dc2 192.168.2.173:8302 alive server 1.8.0 2 dc2 <all> |
||||
consul-server-1.dc1 10.32.5.161:8302 alive server 1.8.0 2 dc1 <all> |
||||
consul-server-1.dc2 192.168.88.64:8302 alive server 1.8.0 2 dc2 <all> |
||||
consul-server-2.dc1 10.32.1.175:8302 alive server 1.8.0 2 dc1 <all> |
||||
consul-server-2.dc2 192.168.35.174:8302 alive server 1.8.0 2 dc2 <all> |
||||
``` |
||||
|
||||
In this example (run from `dc1`), you can see that this datacenter knows about |
||||
the servers in dc2 and that they have status `alive`. |
||||
|
||||
You can also use the `consul catalog services` command with the `-datacenter` flag to ensure |
||||
each datacenter can read each other's services. In this example, our `kubectl` |
||||
context is `dc1` and we're querying for the list of services in `dc2`: |
||||
|
||||
```sh |
||||
$ kubectl exec statefulset/consul-server -- consul catalog services -datacenter dc2 |
||||
consul |
||||
mesh-gateway |
||||
``` |
||||
|
||||
You can switch kubectl contexts and run the same command in `dc2` with the flag |
||||
`-datacenter dc1` to ensure `dc2` can communicate with `dc1`. |
||||
|
||||
### Consul UI |
||||
|
||||
We can also use the Consul UI to verify federation. |
||||
See [Viewing the Consul UI](docs/k8s/installation/overview#viewing-the-consul-ui) |
||||
for instructions on how to view the UI. |
||||
|
||||
~> NOTE: If ACLs are enabled, your kubectl context must be in the primary datacenter |
||||
to retrieve the bootstrap token mentioned in the UI documentation. |
||||
|
||||
With the UI open, you'll be able to switch between datacenters via the dropdown |
||||
in the top left: |
||||
|
||||
 |
||||
|
||||
## Next Steps |
||||
|
||||
With your Kubernetes clusters federated, try out using Consul service mesh to |
||||
route between services deployed on each cluster by following our Learn Guide: [Secure and Route Service Mesh Communication Across Kubernetes](https://learn.hashicorp.com/consul/kubernetes/mesh-gateways#deploy-microservices). |
||||
|
||||
You can also read our in-depth documentation on [Consul Service Mesh In Kubernetes](/docs/k8s/connect). |
||||
@ -0,0 +1,73 @@
|
||||
--- |
||||
layout: docs |
||||
page_title: Multi-Cluster Federation Overview |
||||
sidebar_title: Overview |
||||
description: >- |
||||
Installing on multiple Kubernetes clusters. |
||||
--- |
||||
|
||||
# Multi-Cluster Federation Overview |
||||
|
||||
In Consul, federation is the act of joining two or more Consul datacenters. |
||||
When datacenters are joined, Consul servers in each datacenter can communicate |
||||
with one another. This enables the following features: |
||||
|
||||
- Services on all clusters can make calls to each other through Consul Service Mesh. |
||||
- [Intentions](/docs/connect/intentions) can be used to enforce rules about which services can communicate across all clusters. |
||||
- [L7 Routing Rules](/docs/connect/l7-traffic-management) can enable multi-cluster failover |
||||
and traffic splitting. |
||||
- The Consul UI has a drop-down menu that lets you navigate between datacenters. |
||||
|
||||
## Traditional WAN Federation vs. WAN Federation Via Mesh Gateways |
||||
|
||||
Consul provides two mechanisms for WAN (Wide Area Network) federation: |
||||
|
||||
1. Traditional WAN Federation |
||||
1. WAN Federation Via Mesh Gateways (newly available in Consul 1.8.0) |
||||
|
||||
### Traditional WAN Federation |
||||
|
||||
With traditional WAN federation, all Consul servers must be exposed on the wide area |
||||
network. In the Kubernetes context this is often difficult to set up. It would require that |
||||
each Consul server pod is running on a Kubernetes node with an IP address that is routable from |
||||
all other Kubernetes clusters. Often Kubernetes clusters are deployed into private |
||||
subnets that other clusters cannot route to without additional network devices and configuration. |
||||
|
||||
The Kubernetes solution to the problem of exposing pods is load balancer services but these can't be used |
||||
with traditional WAN federation because it requires proxying both UDP and TCP and Kubernetes load balancers only proxy TCP. |
||||
In addition, each Consul server would need its own load balancer because each |
||||
server needs a unique address. This would increase cost and complexity. |
||||
|
||||
 |
||||
|
||||
### WAN Federation Via Mesh Gateways |
||||
|
||||
To solve the problems that occurred with traditional WAN federation, |
||||
Consul 1.8.0 now supports WAN federation **via mesh gateways**. This mechanism |
||||
only requires that mesh gateways are exposed with routable addresses, not Consul servers. We can front |
||||
the mesh gateway pods with a single Kubernetes service and all traffic flows between |
||||
datacenters through the mesh gateways. |
||||
|
||||
 |
||||
|
||||
## Network Requirements |
||||
|
||||
Clusters/datacenters can be federated even if they have overlapping pod IP spaces or if they're |
||||
on different cloud providers or platforms. Kubernetes clusters can even be |
||||
federated with Consul datacenters running on virtual machines (and vice versa). |
||||
Because the communication between clusters is end-to-end encrypted, mesh gateways |
||||
can even be exposed on the public internet. |
||||
|
||||
The only requirement is that the mesh gateways for each cluster can route to |
||||
one another. For example, if using a load balancer service in front of each cluster's |
||||
mesh gateway, the load balancer IP must be routable from the other mesh gateway pods. |
||||
|
||||
If using a public load balancer, this is guaranteed. If using a private load balancer |
||||
then you'll need to make sure that its IP is routable from your other clusters. |
||||
|
||||
## Next Steps |
||||
|
||||
Now that you have an overview of federation, proceed to either the |
||||
[Federation Between Kubernetes Clusters](/docs/k8s/installation/multi-cluster/kubernetes) |
||||
or [Federation Between VMs and Kubernetes](/docs/k8s/installation/multi-cluster/vms-and-kubernetes) |
||||
pages depending on your use case. |
||||
@ -0,0 +1,295 @@
|
||||
--- |
||||
layout: docs |
||||
page_title: Federation Between VMs and Kubernetes |
||||
sidebar_title: Federation Between VMs and Kubernetes |
||||
description: >- |
||||
Federating Kubernetes clusters and VMs. |
||||
--- |
||||
|
||||
# Federation Between VMs and Kubernetes |
||||
|
||||
-> **1.8.0+:** This feature is available in Consul versions 1.8.0 and higher |
||||
|
||||
~> This topic requires familiarity with [Mesh Gateways](/docs/connect/mesh_gateway) and [WAN Federation Via Mesh Gateways](/docs/connect/wan-federation-via-mesh-gateways). |
||||
|
||||
Consul datacenters running on non-kubernetes platforms like VMs or bare metal can |
||||
be federated with Kubernetes datacenters. Just like with Kubernetes, one datacenter |
||||
must be the [primary](/docs/k8s/installation/multi-cluster/installation#primary-datacenter). |
||||
|
||||
## Kubernetes as the Primary |
||||
|
||||
If your primary datacenter is running on Kubernetes, use the Helm config from the |
||||
[Primary Datacenter](/docs/k8s/installation/multi-cluster/kubernetes#primary-datacenter) section to install Consul. |
||||
|
||||
Once installed, you'll need to export the following information from the primary Kubernetes |
||||
cluster: |
||||
|
||||
1. The certificate authority cert: |
||||
|
||||
```sh |
||||
kubectl get secrets/consul-ca-cert --template='{{index .data "tls.crt" }}' | |
||||
base64 -D > consul-agent-ca.pem |
||||
``` |
||||
|
||||
and the certificate authority signing key: |
||||
|
||||
```sh |
||||
kubectl get secrets/consul-ca-key --template='{{index .data "tls.key" }}' | |
||||
base64 -D > consul-agent-ca-key.pem |
||||
``` |
||||
|
||||
With the `consul-agent-ca.pem` and `consul-agent-ca-key.pem` files you can |
||||
create certificates for your servers and clients running on VMs that share the |
||||
same certificate authority as your Kubernetes servers. |
||||
|
||||
You can use the `consul tls` commands to generate those certificates: |
||||
|
||||
```sh |
||||
# NOTE: consul-agent-ca.pem and consul-agent-ca-key.pem must be in the current |
||||
# directory. |
||||
$ consul tls cert create -server -dc=vm-dc |
||||
==> WARNING: Server Certificates grants authority to become a |
||||
server and access all state in the cluster including root keys |
||||
and all ACL tokens. Do not distribute them to production hosts |
||||
that are not server nodes. Store them as securely as CA keys. |
||||
==> Using consul-agent-ca.pem and consul-agent-ca-key.pem |
||||
==> Saved vm-dc-server-consul-0.pem |
||||
==> Saved vm-dc-server-consul-0-key.pem |
||||
``` |
||||
|
||||
See the help for output of `consul tls cert create -h` to see more options |
||||
for generating server certificates. |
||||
|
||||
These certificates can be used in your server config file: |
||||
|
||||
```hcl |
||||
# server.hcl |
||||
cert_file = "vm-dc-server-consul-0.pem" |
||||
key_file = "vm-dc-server-consul-0-key.pem" |
||||
ca_file = "consul-agent-ca.pem" |
||||
``` |
||||
|
||||
For clients, you can generate TLS certs with: |
||||
|
||||
```sh |
||||
$ consul tls cert create -client |
||||
==> Using consul-agent-ca.pem and consul-agent-ca-key.pem |
||||
==> Saved dc1-client-consul-0.pem |
||||
==> Saved dc1-client-consul-0-key.pem |
||||
``` |
||||
|
||||
Or use the [auto_encrypt](/docs/agent/options.html#auto_encrypt) feature. |
||||
|
||||
1. The WAN addresses of the mesh gateways: |
||||
|
||||
```sh |
||||
$ kubectl exec statefulset/consul-server -- sh -c \ |
||||
'curl -sk https://localhost:8501/v1/catalog/service/mesh-gateway | jq ".[].ServiceTaggedAddresses.wan"' |
||||
{ |
||||
"Address": "1.2.3.4", |
||||
"Port": 443 |
||||
} |
||||
{ |
||||
"Address": "1.2.3.4", |
||||
"Port": 443 |
||||
} |
||||
``` |
||||
|
||||
In this example, the addresses are the same because both mesh gateway pods are |
||||
fronted by the same Kubernetes load balancer. |
||||
|
||||
These addresses will be used in the server config for the `primary_gateways` |
||||
setting: |
||||
|
||||
```hcl |
||||
primary_gateways = ["1.2.3.4:443"] |
||||
``` |
||||
|
||||
1. If ACLs are enabled, you'll also need the replication ACL token: |
||||
|
||||
```sh |
||||
$ kubectl get secrets/consul-acl-replication-acl-token --template='{{.data.token}}' |
||||
e7924dd1-dc3f-f644-da54-81a73ba0a178 |
||||
``` |
||||
|
||||
This token will be used in the server config for the replication token. |
||||
You must also create your own agent policy and token. |
||||
|
||||
```hcl |
||||
acls { |
||||
tokens { |
||||
agent = "<your agent token>" |
||||
replication = "e7924dd1-dc3f-f644-da54-81a73ba0a178" |
||||
} |
||||
} |
||||
``` |
||||
|
||||
1. If gossip encryption is enabled, you'll need the key as well. The command |
||||
to retrieve the key will depend on which Kubernetes secret you've stored it in. |
||||
|
||||
This key will be used in server and client configs for the `encrypt` setting: |
||||
|
||||
```hcl |
||||
encrypt = "uF+GsbI66cuWU21kiXLze5JLEX5j4iDFlDTb0ZWNpDI=" |
||||
``` |
||||
|
||||
A final example server config file might look like: |
||||
|
||||
```hcl |
||||
# From above |
||||
cert_file = "vm-dc-server-consul-0.pem" |
||||
key_file = "vm-dc-server-consul-0-key.pem" |
||||
ca_file = "consul-agent-ca.pem" |
||||
primary_gateways = ["1.2.3.4:443"] |
||||
acl { |
||||
enabled = true |
||||
default_policy = "deny" |
||||
down_policy = "extend-cache" |
||||
tokens { |
||||
agent = "e7924dd1-dc3f-f644-da54-81a73ba0a178" |
||||
replication = "e7924dd1-dc3f-f644-da54-81a73ba0a178" |
||||
} |
||||
} |
||||
encrypt = "uF+GsbI66cuWU21kiXLze5JLEX5j4iDFlDTb0ZWNpDI=" |
||||
|
||||
# Other server settings |
||||
server = true |
||||
datacenter = "vm-dc" |
||||
data_dir = "/opt/consul" |
||||
enable_central_service_config = true |
||||
primary_datacenter = "dc1" |
||||
connect { |
||||
enabled = true |
||||
enable_mesh_gateway_wan_federation = true |
||||
} |
||||
verify_incoming_rpc = true |
||||
verify_outgoing = true |
||||
verify_server_hostname = true |
||||
ports { |
||||
https = 8501 |
||||
http = -1 |
||||
grpc = 8502 |
||||
} |
||||
``` |
||||
|
||||
## Kubernetes as the Secondary |
||||
|
||||
If you're running your primary datacenter on VMs then you'll need to manually |
||||
construct the [Federation Secret](#federation-secret) in order to federate |
||||
Kubernetes clusters as secondaries. |
||||
|
||||
-> Your VM cluster must be running mesh gateways, and have mesh gateway WAN |
||||
federation enabled. See [WAN Federation via Mesh Gateways](/docs/connect/wan-federation-via-mesh-gateways). |
||||
|
||||
You'll need: |
||||
|
||||
1. The root certificate authority cert placed in `consul-agent-ca.pem`. |
||||
1. The root certificate authority key placed in `consul-agent-ca-key.pem`. |
||||
1. The IP addresses of the mesh gateways running in your VM datacenter. These must |
||||
be routable from the Kubernetes cluster. |
||||
1. If ACLs are enabled you must create an ACL replication token with the following rules: |
||||
|
||||
```hcl |
||||
acl = "write" |
||||
operator = "write" |
||||
agent_prefix "" { |
||||
policy = "read" |
||||
} |
||||
node_prefix "" { |
||||
policy = "write" |
||||
} |
||||
service_prefix "" { |
||||
policy = "read" |
||||
intentions = "read" |
||||
} |
||||
``` |
||||
|
||||
This token is used for ACL replication and for automatic ACL management in Kubernetes. |
||||
|
||||
If you're running Consul Enterprise you'll need the rules: |
||||
|
||||
```hcl |
||||
acl = "write" |
||||
operator = "write" |
||||
agent_prefix "" { |
||||
policy = "read" |
||||
} |
||||
node_prefix "" { |
||||
policy = "write" |
||||
} |
||||
namespace_prefix "" { |
||||
service_prefix "" { |
||||
policy = "read" |
||||
intentions = "read" |
||||
} |
||||
} |
||||
``` |
||||
|
||||
1. If gossip encryption is enabled, you'll need the key. |
||||
|
||||
With that data ready, you can create the Kubernetes federation secret: |
||||
|
||||
```sh |
||||
kubectl create secret generic consul-federation \ |
||||
--from-literal=caCert=$(cat consul-agent-ca.pem) \ |
||||
--from-literal=caKey=$(cat consul-agent-ca-key.pem) |
||||
# If ACLs are enabled uncomment. |
||||
# --from-literal=replicationToken="<your acl replication token>" \ |
||||
# If using gossip encryption uncomment. |
||||
# --from-literal=gossipEncryptionKey="<your gossip encryption key>" |
||||
``` |
||||
|
||||
Then use the following Helm config file: |
||||
|
||||
```yaml |
||||
global: |
||||
name: consul |
||||
image: consul:1.8.0-beta1 |
||||
datacenter: dc2 |
||||
tls: |
||||
enabled: true |
||||
caCert: |
||||
secretName: consul-federation |
||||
secretKey: caCert |
||||
caKey: |
||||
secretName: consul-federation |
||||
secretKey: caKey |
||||
|
||||
# Delete this acls section if ACLs are disabled. |
||||
acls: |
||||
manageSystemACLs: true |
||||
replicationToken: |
||||
secretName: consul-federation |
||||
secretKey: replicationToken |
||||
|
||||
federation: |
||||
enabled: true |
||||
|
||||
# Delete this gossipEncryption section if gossip encryption is disabled. |
||||
gossipEncryption: |
||||
secretName: consul-federation |
||||
secretKey: gossipEncryptionKey |
||||
|
||||
connectInject: |
||||
enabled: true |
||||
meshGateway: |
||||
enabled: true |
||||
server: |
||||
extraConfig: | |
||||
{ |
||||
"primary_datacenter": "<your VM datacenter name>", |
||||
"primary_gateways": ["<ip of your VM mesh gateway>", "<other ip>", ...] |
||||
} |
||||
``` |
||||
|
||||
-> **NOTE: ** You must fill out the `server.extraConfig` section with the datacenter |
||||
name of your primary datacenter running on VMs and with the IPs of your mesh |
||||
gateways running on VMs. |
||||
|
||||
With your config file ready to go, follow our [Installation Guide](/docs/k8s/installation/overview |
||||
to install Consul on your secondary cluster(s). |
||||
|
||||
## Next Steps |
||||
|
||||
Read the [Verifying Federation](/docs/k8s/installation/multi-cluster/kubernetes#verifying-federation) |
||||
section to verify that federation is working as expected. |
||||
|
After Width: | Height: | Size: 157 KiB |
|
After Width: | Height: | Size: 490 KiB |
|
After Width: | Height: | Size: 451 KiB |
Loading…
Reference in new issue