@ -51,24 +51,31 @@ Regardless of whether you connect your clusters through WAN federation or cluste
The following resources are available to help you use Consul's cluster peering features.
**Tutorials:**
### Tutorials
- To learn how to peer clusters and connect services across peers in AWS Elastic Kubernetes Service (EKS) and Google Kubernetes Engine (GKE) environments, complete the [Consul Cluster Peering on Kubernetes tutorial](/consul/tutorials/developer-mesh/cluster-peering).
This reference topic describes the technical specifications associated with using cluster peering in your deployments. These specifications include required Consul components and their configurations.
This reference topic describes the technical specifications associated with using cluster peering in your deployments. These specifications include required Consul components and their configurations. To learn more about Consul's cluster peering feature, refer to [cluster peering overview](/consul/docs/connect/cluster-peering).
For cluster peering requirements in Kubernetes deployments, refer to [cluster peering on Kubernetes technical specifications](/consul/docs/k8s/connect/cluster-peering/tech-specs).
## Requirements
To use cluster peering features, make sure your Consul environment meets the following prerequisites:
Consul's default configuration supports cluster peering connections directly between clusters. In production environments, we recommend using mesh gateways to securely route service mesh traffic between partitions with cluster peering connections.
In addition, make sure your Consul environment meets the following prerequisites:
- Consul v1.14 or higher.
- A local Consul agent is required to manage mesh gateway configuration.
- Use [Envoy proxies](/consul/docs/connect/proxies/envoy). Envoy is the only proxy with mesh gateway capabilities in Consul.
- A local Consul agent is required to manage mesh gateway configurations.
In addition, the following service mesh components are required in order to establish cluster peering connections:
- [Exported service requirements](#exported-service-requirements)
- [ACL requirements](#acl-requirements)
## Mesh gateway specifications
### Mesh gateway requirements
To change Consul's default configuration and enable cluster peering through mesh gateways, use a mesh configuration entry to update your network's service mesh proxies globally:
Mesh gateways are required for routing service mesh traffic between partitions with cluster peering connections. Consider the following general requirements for mesh gateways when using cluster peering:
1. In a `mesh` configuration entry, set `PeerThroughMeshGateways` to `true`:
- A cluster requires a registered mesh gateway in order to export services to peers.
- For Enterprise, this mesh gateway must also be registered in the same partition as the exported services and their `exported-services` configuration entry.
- To use the `local` mesh gateway mode, you must register a mesh gateway in the importing cluster.
<CodeBlockConfig filename="mesh-config.hcl">
```hcl
Kind = "mesh"
Peering {
PeerThroughMeshGateways = true
}
```
</CodeBlockConfig>
In addition, you must define the `Proxy.Config` settings using opaque parameters compatible with your proxy. Refer to the [Gateway options](/consul/docs/connect/proxies/envoy#gateway-options) and [Escape-hatch Overrides](/consul/docs/connect/proxies/envoy#escape-hatch-overrides) documentation for additional Envoy proxy configuration information.
1. Write the configuration entry to Consul:
#### Mesh gateway modes
```shell
$ consul config write mesh-config.hcl
```
By default, all cluster peering connections use mesh gateways in [remote mode](/consul/docs/connect/gateways/mesh-gateway/service-to-service-traffic-wan-datacenters#remote). Be aware of these additional requirements when changing a mesh gateway's mode.
When cluster peering through mesh gateways, consider the following deployment requirements:
- A cluster requires a registered mesh gateway in order to export services to peers in other regions or cloud providers.
- The mesh gateway must also be registered in the same admin partition as the exported services and their `exported-services` configuration entry. An enterprise license is required to use multiple admin partitions with a single cluster of Consul servers.
- To use the `local` mesh gateway mode, you must register a mesh gateway in the importing cluster.
- Define the `Proxy.Config` settings using opaque parameters compatible with your proxy. Refer to the [Gateway options](/consul/docs/connect/proxies/envoy#gateway-options) and [Escape-hatch Overrides](/consul/docs/connect/proxies/envoy#escape-hatch-overrides) documentation for additional Envoy proxy configuration information.
### Mesh gateway modes
By default, cluster peering connections use mesh gateways in [remote mode](/consul/docs/connect/gateways/mesh-gateway/service-to-service-traffic-wan-datacenters#remote). Be aware of these additional requirements when changing a mesh gateway's mode.
- For mesh gateways that connect peered clusters, you can set the `mode` as either `remote` or `local`.
- The `none` mode is invalid for mesh gateways with cluster peering connections.
Refer to [mesh gateway modes](/consul/docs/connect/gateways/mesh-gateway#modes) for more information.
## Sidecar proxy requirements
## Sidecar proxy specifications
The Envoy proxies that function as sidecars in your service mesh require configuration in order to properly route traffic to peers. Sidecar proxies are defined in the [service definition](/consul/docs/services/usage/define-services).
@ -55,15 +69,13 @@ The Envoy proxies that function as sidecars in your service mesh require configu
- The `proxy.upstreams.destination_peer` parameter must be configured to enable cross-cluster traffic.
- The `proxy.upstream/destination_namespace` configuration is only necessary if the destination service is in a non-default namespace.
## Exported service requirements
The `exported-services` configuration entry is required in order for services to communicate across partitions with cluster peering connections.
## Exported service specifications
Basic guidance on using the `exported-services` configuration entry is included in [Establish cluster peering connections](/consul/docs/k8s/connect/cluster-peering/usage/establish-peering).
The `exported-services` configuration entry is required in order for services to communicate across partitions with cluster peering connections. Basic guidance on using the `exported-services` configuration entry is included in [Establish cluster peering connections](/consul/docs/connect/cluster-peering/usage/establish-peering#export-services-between-clusters).
Refer to the [`exported-services` configuration entry](/consul/docs/connect/config-entries/exported-services) reference for more information.
## ACL requirements
## ACL specifications
If ACLs are enabled, you must add tokens to grant the following permissions:
@ -16,7 +16,7 @@ This page details the process for establishing a cluster peering connection betw
Cluster peering between services cannot be established until all four steps are complete.
For Kubernetes-specific guidance for establishing cluster peering connections, refer to [Establish cluster peering connections on Kubernetes](/consul/docs/k8s/connect/cluster-peering/usage/establish-peering).
For Kubernetes guidance, refer to [Establish cluster peering connections on Kubernetes](/consul/docs/k8s/connect/cluster-peering/usage/establish-peering). For HCP Consul guidance, refer to [Establish cluster peering connections on HCP Consul](/hcp/docs/consul/usage/cluster-peering/create-connections).
## Requirements
@ -29,11 +29,9 @@ If you need to make services available to an admin partition in the same datacen
### Mesh gateway requirements
Mesh gateways are required for all cluster peering connections. Consider the following architectural requirements when creating mesh gateways:
Consul's default configuration supports cluster peering connections directly between clusters. In production environments, we recommend using mesh gateways to securely route service mesh traffic between partitions with cluster peering connections.
- A registered mesh gateway is required in order to export services to peers.
- For Enterprise, the mesh gateway that exports services must be registered in the same partition as the exported services and their `exported-services` configuration entry.
- To use the `local` mesh gateway mode, you must register a mesh gateway in the importing cluster.
To enable cluster peering through mesh gateways and configure mesh gateways to support cluster peering, refer to [mesh gateway specifications](/consul/docs/connect/cluster-peering/tech-specs#mesh-gateway-specifications).
page_title: Cluster Peering on Kubernetes Technical Specifications
description: >-
In Kubernetes deployments, cluster peering connections interact with mesh gateways, sidecar proxies, exported services, and ACLs. Learn about requirements specific to k8s, including required Helm values and CRDs.
In Kubernetes deployments, cluster peering connections interact with mesh gateways, exported services, and ACLs. Learn about requirements specific to k8s, including required Helm values and custom resource definitions (CRDs).
---
# Cluster peering on Kubernetes technical specifications
This reference topic describes the technical specifications associated with using cluster peering in your Kubernetes deployments. These specifications include [required Helm values](#helm-requirements) and [required custom resource definitions (CRDs)](#crd-requirements), as well as required Consul components and their configurations.
This reference topic describes the technical specifications associated with using cluster peering in your Kubernetes deployments. These specifications include [required Helm values](#helm-requirements) and [required custom resource definitions (CRDs)](#crd-requirements), as well as required Consul components and their configurations. To learn more about Consul's cluster peering feature, refer to [cluster peering overview](/consul/docs/connect/cluster-peering).
For cluster peering requirements in non-Kubernetes deployments, refer to [cluster peering technical specifications](/consul/docs/connect/cluster-peering/tech-specs).
## General Requirements
## General requirements
To use cluster peering features, make sure your Consul environment meets the following prerequisites:
Make sure your Consul environment meets the following prerequisites:
- Consul v1.14 or higher
- Consul on Kubernetes v1.0.0 or higher
- At least two Kubernetes clusters
In addition, the following service mesh components are required in order to establish cluster peering connections:
You must also configure the following service mesh components in order to establish cluster peering connections:
Mesh gateways are required when establishing cluster peering connections. The following values must be set in the Helm chart to enable mesh gateways:
Consul's default configuration supports cluster peering connections directly between clusters. In production environments, we recommend using mesh gateways to securely route service mesh traffic between partitions with cluster peering connections. The following values must be set in the Helm chart to enable mesh gateways:
After mesh gateways are enabled in the Helm chart, you can separately [configure Mesh CRDs](#mesh-gateway-configuration-for-kubernetes).
## CRD requirements
## CRD specifications
You must create the following CRDs in order to establish a peering connection:
@ -100,28 +100,9 @@ spec:
</CodeBlockConfig>
</CodeTabs>
## Mesh gateway requirements
## Mesh gateway specifications
Mesh gateways are required for routing service mesh traffic between partitions with cluster peering connections. Consider the following general requirements for mesh gateways when using cluster peering:
- A cluster requires a registered mesh gateway in order to export services to peers.
- For Enterprise, this mesh gateway must also be registered in the same partition as the exported services and their `exported-services` configuration entry.
- To use the `local` mesh gateway mode, you must register a mesh gateway in the importing cluster.
In addition, you must define the `Proxy.Config` settings using opaque parameters compatible with your proxy. Refer to the [Gateway options](/consul/docs/connect/proxies/envoy#gateway-options) and [Escape-hatch Overrides](/consul/docs/connect/proxies/envoy#escape-hatch-overrides) documentation for additional Envoy proxy configuration information.
### Mesh gateway modes
By default, all cluster peering connections use mesh gateways in [remote mode](/consul/docs/connect/gateways/mesh-gateway/service-to-service-traffic-wan-datacenters#remote). Be aware of these additional requirements when changing a mesh gateway's mode.
- For mesh gateways that connect peered clusters, you can set the `mode` as either `remote` or `local`.
- The `none` mode is invalid for mesh gateways with cluster peering connections.
Refer to [mesh gateway modes](/consul/docs/connect/gateways/mesh-gateway#modes) for more information.
### Mesh gateway configuration for Kubernetes
Mesh gateways are required for cluster peering connections. Complete the following steps to add mesh gateways to your deployment so that you can establish cluster peering connections:
To change Consul's default configuration and enable cluster peering through mesh gateways, use a mesh configuration entry to update your network's service mesh proxies globally:
1. In `cluster-01` create the `Mesh` custom resource with `peeringThroughMeshGateways` set to `true`.
@ -139,39 +120,47 @@ Mesh gateways are required for cluster peering connections. Complete the followi
</CodeBlockConfig>
1. Apply the mesh gateway to `cluster-01`. Replace `CLUSTER1_CONTEXT` to target the first Consul cluster.
1. Repeat the process to create and apply a mesh gateway with cluster peering enabled to `cluster-02`. Replace `CLUSTER2_CONTEXT` to target the second Consul cluster.
For help setting up the cluster context variables used in this example, refer to [assign cluster IDs to environmental variables](/consul/docs/k8s/connect/cluster-peering/usage/establish-peering#assign-cluster-ids-to-environmental-variables).
</Note>
When cluster peering through mesh gateways, consider the following deployment requirements:
- A Consul cluster requires a registered mesh gateway in order to export services to peers in other regions or cloud providers.
- The mesh gateway must also be registered in the same admin partition as the exported services and their `exported-services` configuration entry. An enterprise license is required to use multiple admin partitions with a single cluster of Consul servers.
- To use the `local` mesh gateway mode, you must register a mesh gateway in the importing cluster.
- Define the `Proxy.Config` settings using opaque parameters compatible with your proxy. For additional Envoy proxy configuration information, refer to [Gateway options](/consul/docs/connect/proxies/envoy#gateway-options) and [Escape-hatch overrides](/consul/docs/connect/proxies/envoy#escape-hatch-overrides).
### Mesh gateway modes
By default, all cluster peering connections use mesh gateways in [remote mode](/consul/docs/connect/gateways/mesh-gateway/service-to-service-traffic-wan-datacenters#remote). Be aware of these additional requirements when changing a mesh gateway's mode.
- For mesh gateways that connect peered clusters, you can set the `mode` as either `remote` or `local`.
- The `none` mode is invalid for mesh gateways with cluster peering connections.
To learn how to change the mesh gateway mode to `local` on your Kubernetes deployment, refer to [configure the mesh gateway mode for traffic between services](/consul/docs/k8s/connect/cluster-peering/usage/establish-peering#configure-the-mesh-gateway-mode-for-traffic-between-services).
## Exported service specifications
The `exported-services` CRD is required in order for services to communicate across partitions with cluster peering connections.
The `exported-services` CRD is required in order for services to communicate across partitions with cluster peering connections. Basic guidance on using the `exported-services` configuration entry is included in [Establish cluster peering connections](/consul/docs/k8s/connect/cluster-peering/usage/establish-peering#export-services-between-clusters).
Refer to the [`exported-services` configuration entry](/consul/docs/connect/config-entries/exported-services) reference for more information.
Refer to [`exported-services` configuration entry](/consul/docs/connect/config-entries/exported-services) for more information.
## ACL requirements
## ACL specifications
If ACLs are enabled, you must add tokens to grant the following permissions:
This topic describes how to dynamically query the Consul catalog using prepared queries. Prepared queries are configurations that enable you to register a complex service query and execute it on demand. For information about how to perform standard node and service lookups, refer to [Perform Static DNS Queries](/consul/docs/services/discovery/dns-static-lookups).
## Introduction
Prepared queries provide a rich set of lookup features, such as filtering by multiple tags and automatically failing over to look for services in remote datacenters if no healthy nodes are available in the local datacenter. You can also create prepared query templates that match names using a prefix match, allowing a single template to apply to potentially many services. Refer to [Query Consul Nodes and Services Overview](/consul/docs/services/discovery/dns-overview) for additional information about DNS query behaviors.
## Requirements
Consul 0.6.4 or later is required to create prepared query templates.
### ACLs
If ACLs are enabled, the querying service must present a token linked to permissions that enable read access for query, service, and node resources. Refer to the following documentation for information about creating policies to enable read access to the necessary resources:
Refer to the [prepared query reference](/consul/api-docs/query#create-prepared-query) for usage information.
1. Specify the prepared query options in JSON format. The following prepared query targets all instances of the `redis` service in `dc1` and `dc2`:
@ -64,6 +68,7 @@ Refer to the [prepared query reference](/consul/api-docs/query#create-prepared-q
$ curl --request POST --data @payload.json http://127.0.0.1:8500/v1/query
{"ID":"014af5ff-29e6-e972-dcf8-6ee602137127"}%
```
1. To run the query, send a GET request to the endpoint and specify the ID returned from the POST call.
```shell-session
@ -71,6 +76,7 @@ Refer to the [prepared query reference](/consul/api-docs/query#create-prepared-q
```
## Execute prepared queries
You can execute a prepared query using the standard lookup format or the strict RFC 2782 SRV lookup.
### Standard lookup
@ -84,6 +90,7 @@ Use the following format to execute a prepared query using the standard lookup f
Refer [Standard lookups](/consul/docs/services/discovery/dns-static-lookups#standard-lookups) for additional information about the standard lookup format in Consul.
### RFC 2782 SRV lookup
Use the following format to execute a prepared query using the RFC 2782 lookup format:
```
@ -93,9 +100,11 @@ _<query name or id>._tcp.query[.<datacenter>].<domain>
For additional information about following the RFC 2782 SRV lookup format in Consul, refer to [RFC 2782 Lookup](/consul/docs/services/discovery/dns-static-lookups#rfc-2782-lookup). For general information about the RFC 2782 specification, refer to [A DNS RR for specifying the location of services \(DNS SRV\)](https://tools.ietf.org/html/rfc2782).
### Lookup options
The `datacenter` subdomain is optional. By default, the lookup queries the datacenter of this Consul agent.
The `query name` or `id` subdomain is the name or ID of an existing prepared query.
## Results
To allow for simple load balancing, Consul returns the set of nodes in random order for each query. Prepared queries support A and SRV records. SRV records provide the port that a service is registered. Consul only serves SRV records if the client specifically requests them.
Jeff Boruszak
2 years ago
committed by