github
/
consul
mirror of https://github.com/hashicorp/consul
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Join external service to k8s

pull/15384/head
boruszak 2022-11-15 09:34:27 -06:00
parent 8f995aa000
commit c77c1fce44
1 changed files with 51 additions and 61 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

112
website/content/docs/k8s/deployment-configurations/clients-outside-kubernetes.mdx
View File

@ -1,80 +1,65 @@
---
layout: docs
page_title: Join External Clients to Consul on Kubernetes
page_title: Join External Services to Consul on Kubernetes
description: >-
Client agents running on VMs can join a Consul datacenter running on Kubernetes. Configure the Kubernetes installation to accept communication from external clients.
Services running on a virtual machine (VM) can join a Consul datacenter running on Kubernetes. Learn how to configure the Kubernetes installation to accept communication from external services.
---
# Join External Clients to Consul on Kubernetes
# Join External Services to Consul on Kubernetes
Consul clients running on non-Kubernetes nodes can join a Consul cluster running within Kubernetes.
## Networking
Within one datacenter, Consul typically requires a fully connected
[network](/docs/architecture). This means the IPs of every client and server
agent should be routable by every other client and server agent in the
datacenter. Clients need to be able to [gossip](/docs/architecture/gossip) with
every other agent and make RPC calls to servers. Servers need to be able to
gossip with every other agent. See [Architecture](/docs/architecture) for more details.
-> **Consul Enterprise customers** may use [network
segments](/docs/enterprise/network-segments) to enable non-fully-connected
topologies. However, out-of-cluster nodes must still be able to communicate
with the server pod or host IP addresses.
Services running on non-Kubernetes nodes can join a Consul cluster running within Kubernetes.
## Auto-join
The recommended way to join a cluster running within Kubernetes is to
use the ["k8s" cloud auto-join provider](/docs/install/cloud-auto-join#kubernetes-k8s).
The auto-join provider dynamically discovers IP addresses to join using
the Kubernetes API. It authenticates with Kubernetes using a standard
`kubeconfig` file. This works with all major hosted Kubernetes offerings
`kubeconfig` file. Auto-join works with all major hosted Kubernetes offerings
as well as self-hosted installations. The token in the `kubeconfig` file
needs to have permissions to list pods in the namespace where Consul servers
are deployed.
The auto-join string below will join a Consul server cluster that is
started using the [official Helm chart](/docs/k8s/helm):
The auto-join string below joins a Consul server agent to a cluster using the [official Helm chart](/docs/k8s/helm):
```shell-session
$ consul agent -retry-join 'provider=k8s label_selector="app=consul,component=server"'
```
-> **Note:** This auto-join command only connects on the default gossip port
8301, whether you are joining on the pod network or via host ports. Either a
consul server or client that is already a member of the datacenter should be
listening on this port for the external client agent to be able to use
8301, whether you are joining on the pod network or via host ports. A
Consul server that is already a member of the datacenter should be
listening on this port for the external service to connect through
auto-join.
### Auto-join on the Pod network
In the default Consul Helm chart installation, Consul clients and servers are
routable only via their pod IPs for server RPCs and gossip (HTTP
API calls to Consul clients can also be made through host IPs). This means any
external client agents joining the Consul cluster running on Kubernetes would
need to be able to have connectivity to those pod IPs.
In many hosted Kubernetes environments, you will need to explicitly configure
In the default Consul Helm chart installation, Consul servers are
routable through their pod IPs for server RPCs. As a result, any
external agents joining the Consul cluster running on Kubernetes
need to be able to connect to those pod IPs.
In many hosted Kubernetes environments, you need to explicitly configure
your hosting provider to ensure that pod IPs are routable from external VMs.
See [Azure AKS
CNI](https://docs.microsoft.com/en-us/azure/aks/concepts-network#azure-cni-advanced-networking),
[AWS EKS
CNI](https://docs.aws.amazon.com/eks/latest/userguide/pod-networking.html) and
[GKE VPC-native
clusters](https://cloud.google.com/kubernetes-engine/docs/concepts/alias-ips).
For more information, refer to [Azure AKS CNI](https://docs.microsoft.com/en-us/azure/aks/concepts-network#azure-cni-advanced-networking),
[AWS EKS CNI](https://docs.aws.amazon.com/eks/latest/userguide/pod-networking.html) and
[GKE VPC-native clusters](https://cloud.google.com/kubernetes-engine/docs/concepts/alias-ips).
Given you have the [official Helm chart](/docs/k8s/helm) installed with the default values, do the following to join an external client agent.
To join external agents with Consul on Kubernetes deployments installed with default values through the [official Helm chart](/docs/k8s/helm):
1. Make sure the pod IPs of the clients and servers in Kubernetes are
1. Make sure the pod IPs of the servers in Kubernetes are
routable from the VM and that the VM can access port 8301 (for gossip) and
port 8300 (for server RPC) on those pod IPs.
1. Make sure that the client and server pods running in Kubernetes can route
1. Make sure that the server pods running in Kubernetes can route
to the VM's advertise IP on its gossip port (default 8301).
2. Make sure you have the `kubeconfig` file for the Kubernetes cluster in `$HOME/.kube/config` on the external VM.
1. Make sure you have the `kubeconfig` file for the Kubernetes cluster in `$HOME/.kube/config` on the external VM.
2. On the external VM, run:
```bash
1. On the external VM, run:
```shell-session
consul agent \
-advertise="$ADVERTISE_IP" \
-retry-join='provider=k8s label_selector="app=consul,component=server"' \
@ -86,7 +71,8 @@ Given you have the [official Helm chart](/docs/k8s/helm) installed with the defa
-data-dir=$DATA_DIR \
```
3. Check if the join was successful by running `consul members`. Sample output:
1. Run `consul members` to check if the join was successful.
```shell-session
/ $ consul members
Node Address Status Type Build Protocol DC Segment
@ -97,10 +83,10 @@ Given you have the [official Helm chart](/docs/k8s/helm) installed with the defa
gke-external-agent-default-pool-32d15192-vo7k 10.138.0.42:8301 alive client 1.9.1 2 dc1 <default>
```
### Auto-join via host ports
If your external VMs can't connect to Kubernetes pod IPs, but they can connect
to the internal host IPs of the nodes in the Kubernetes cluster, you have the
option to expose the clients and server ports on the host IP instead.
### Auto-join through host ports
If your external VMs cannot connect to Kubernetes pod IPs but they can connect
to the internal host IPs of the nodes in the Kubernetes cluster, you can join the two by exposing ports on the host IP instead.
1. Install the [official Helm chart](/docs/k8s/helm) with the following values:
```yaml
@ -114,19 +100,20 @@ option to expose the clients and server ports on the host IP instead.
# Note that this needs to be different than 8301, to avoid conflicting with the client gossip hostPort
port: 9301
```
This will expose the client gossip ports, the server gossip ports and the server RPC port at `hostIP:hostPort`. Note that `hostIP` is the **internal** IP of the VM that the client/server pods are deployed on.
This installation exposes the client gossip ports, the server gossip ports and the server RPC port at `hostIP:hostPort`. Note that `hostIP` is the **internal** IP of the VM that the client/server pods are deployed on.
1. Make sure the IPs of the Kubernetes nodes are routable from the VM and
that the VM can access ports 8301 and 9301 (for gossip) and port 8300 (for
server RPC) on those node IPs.
1. Make sure the client and server pods running in Kubernetes can route to
1. Make sure the server pods running in Kubernetes can route to
the VM's advertise IP on its gossip port (default 8301).
3. Make sure you have the `kubeconfig` file for the Kubernetes cluster in `$HOME/.kube/config` on the external VM.
1. Make sure you have the `kubeconfig` file for the Kubernetes cluster in `$HOME/.kube/config` on the external VM.
4. On the external VM, run (note the addition of `host_network=true` in the retry-join argument):
```bash
1. On the external VM, run:
```shell-session
consul agent \
-advertise="$ADVERTISE_IP" \
-retry-join='provider=k8s host_network=true label_selector="app=consul,component=server"'
@ -137,7 +124,11 @@ option to expose the clients and server ports on the host IP instead.
-datacenter=$DATACENTER \
-data-dir=$DATA_DIR \
```
3. Check if the join was successful by running `consul members`. Sample output:
Note the addition of `host_network=true` in the retry-join argument.
1. Run `consul members` to check if the join was successful.
```shell-session
/ $ consul members
Node Address Status Type Build Protocol DC Segment
@ -149,13 +140,12 @@ option to expose the clients and server ports on the host IP instead.
```
## Manual join
If you are unable to use auto-join, you can also follow the instructions in
either of the auto-join sections but instead of using a `provider` key in the
`-retry-join` flag, you would need to pass the address of at least one
consul server, e.g: `-retry-join=$CONSUL_SERVER_IP:$SERVER_SERFLAN_PORT`. A
`kubeconfig` file is not required when using manual join.
However, rather than hardcoding the IP, it's recommended to set up a DNS entry
that would resolve to the consul servers' pod IPs (if using the pod network) or
host IPs that the server pods are running on (if using host ports).
If you are unable to use auto-join, try following the instructions in
either of the auto-join sections, but instead of using a `provider` key in the
`-retry-join` flag, pass the address of at least one Consul server. Example: `-retry-join=$CONSUL_SERVER_IP:$SERVER_SERFLAN_PORT`.
A `kubeconfig` file is not required when using manual join.
Instead of hardcoding an IP address, we recommend you set up a DNS entry
that resolves to the pod IPs or host IPs that the Consul server pods are running on.