github
/
k3s
mirror of https://github.com/k3s-io/k3s
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Improve the ipvs/README.md 

Improve the ipvs/README.md
pull/564/head
AdamDang 2019-01-31 18:42:17 +08:00 committed by GitHub
parent 29e9ff36a9
commit 62105c87d4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 26 additions and 26 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

52
pkg/proxy/ipvs/README.md
View File

@ -1,8 +1,8 @@
- [IPVS](#ipvs)
- [What is IPVS](#what-is-ipvs)
- [IPVS vs. IPTABLES](#ipvs-vs-iptables)
- [When ipvs falls back to iptables](#when-ipvs-falls-back-to-iptables)
- [Run kube-proxy in ipvs mode](#run-kube-proxy-in-ipvs-mode)
- [When IPVS falls back to IPTABLES](#when-ipvs-falls-back-to-iptables)
- [Run kube-proxy in IPVS mode](#run-kube-proxy-in-ipvs-mode)
- [Prerequisite](#prerequisite)
- [Local UP Cluster](#local-up-cluster)
- [GCE Cluster](#gce-cluster)
@ -16,7 +16,7 @@
This document intends to show users
- what is IPVS
- difference between IPVS and IPTABLES
- how to run kube-proxy in ipvs mode and info on debugging
- how to run kube-proxy in IPVS mode and info on debugging
## What is IPVS
@ -32,16 +32,16 @@ Differences between IPVS mode and IPTABLES mode are as follows:
1. IPVS provides better scalability and performance for large clusters.
2. IPVS supports more sophisticated load balancing algorithms than iptables (least load, least connections, locality, weighted, etc.).
2. IPVS supports more sophisticated load balancing algorithms than IPTABLES (least load, least connections, locality, weighted, etc.).
3. IPVS supports server health checking and connection retries, etc.
### When ipvs falls back to iptables
IPVS proxier will employ iptables in doing packet filtering, SNAT or masquerade.
Specifically, ipvs proxier will use ipset to store source or destination address of traffics that need DROP or do masquerade, to make sure the number of iptables rules be constant, no metter how many services we have.
### When IPVS falls back to IPTABLES
IPVS proxier will employ IPTABLES in doing packet filtering, SNAT or masquerade.
Specifically, IPVS proxier will use ipset to store source or destination address of traffics that need DROP or do masquerade, to make sure the number of IPTABLES rules be constant, no metter how many services we have.
Here is the table of ipset sets that ipvs proxier used.
Here is the table of ipset sets that IPVS proxier used.
| set name | members | usage |
| :----------------------------- | ---------------------------------------- | ---------------------------------------- |
@ -58,11 +58,11 @@ Here is the table of ipset sets that ipvs proxier used.
| KUBE-NODE-PORT-LOCAL-UDP | nodeport type service UDP port with `externalTrafficPolicy=local` | accept packages to nodeport service with `externalTrafficPolicy=local` |
IPVS proxier will fall back on iptables in the following scenarios.
IPVS proxier will fall back on IPTABLES in the following scenarios.
**1. kube-proxy starts with --masquerade-all=true**
If kube-proxy starts with `--masquerade-all=true`, ipvs proxier will masquerade all traffic accessing service Cluster IP, which behaves the same as what iptables proxier. Suppose kube-proxy have flag `--masquerade-all=true` specified, then the iptables installed by ipvs proxier should be like what is shown below.
If kube-proxy starts with `--masquerade-all=true`, IPVS proxier will masquerade all traffic accessing service Cluster IP, which behaves the same as what IPTABLES proxier. Suppose kube-proxy have flag `--masquerade-all=true` specified, then the IPTABLES installed by IPVS proxier should be like what is shown below.
```shell
# iptables -t nat -nL
@ -96,7 +96,7 @@ ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-CLU
**2. Specify cluster CIDR in kube-proxy startup**
If kube-proxy starts with `--cluster-cidr=<cidr>`, ipvs proxier will masquerade off-cluster traffic accessing service Cluster IP, which behaves the same as what iptables proxier. Suppose kube-proxy is provided with the cluster cidr `10.244.16.0/24`, then the iptables installed by ipvs proxier should be like what is shown below.
If kube-proxy starts with `--cluster-cidr=<cidr>`, IPVS proxier will masquerade off-cluster traffic accessing service Cluster IP, which behaves the same as what IPTABLES proxier. Suppose kube-proxy is provided with the cluster cidr `10.244.16.0/24`, then the IPTABLES installed by IPVS proxier should be like what is shown below.
```shell
# iptables -t nat -nL
@ -130,10 +130,10 @@ ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-CLU
**3. Load Balancer type service**
For loadBalancer type service, ipvs proxier will install iptables with match of ipset `KUBE-LOAD-BALANCER`.
For loadBalancer type service, IPVS proxier will install IPTABLES with match of ipset `KUBE-LOAD-BALANCER`.
Specially when service's `LoadBalancerSourceRanges` is specified or specified `externalTrafficPolicy=local`,
ipvs proxier will create ipset sets `KUBE-LOAD-BALANCER-LOCAL`/`KUBE-LOAD-BALANCER-FW`/`KUBE-LOAD-BALANCER-SOURCE-CIDR`
and install iptables accordingly, which should looks like what is shown below.
IPVS proxier will create ipset sets `KUBE-LOAD-BALANCER-LOCAL`/`KUBE-LOAD-BALANCER-FW`/`KUBE-LOAD-BALANCER-SOURCE-CIDR`
and install IPTABLES accordingly, which should looks like what is shown below.
```shell
# iptables -t nat -nL
@ -182,9 +182,9 @@ ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-LOA
**4. NodePort type service**
For NodePort type service, ipvs proxier will install iptables with match of ipset `KUBE-NODE-PORT-TCP/KUBE-NODE-PORT-UDP`.
When specified `externalTrafficPolicy=local`,ipvs proxier will create ipset sets `KUBE-NODE-PORT-LOCAL-TC/KUBE-NODE-PORT-LOCAL-UDP`
and install iptables accordingly, which should looks like what is shown below.
For NodePort type service, IPVS proxier will install IPTABLES with match of ipset `KUBE-NODE-PORT-TCP/KUBE-NODE-PORT-UDP`.
When specified `externalTrafficPolicy=local`,IPVS proxier will create ipset sets `KUBE-NODE-PORT-LOCAL-TC/KUBE-NODE-PORT-LOCAL-UDP`
and install IPTABLES accordingly, which should looks like what is shown below.
Suppose service with TCP type nodePort.
@ -222,8 +222,8 @@ KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUB
**5. Service with externalIPs specified**
For service with `externalIPs` specified, ipvs proxier will install iptables with match of ipset `KUBE-EXTERNAL-IP`,
Suppose we have service with `externalIPs` specified, iptables rules should looks like what is shown below.
For service with `externalIPs` specified, IPVS proxier will install IPTABLES with match of ipset `KUBE-EXTERNAL-IP`,
Suppose we have service with `externalIPs` specified, IPTABLES rules should looks like what is shown below.
```shell
Chain PREROUTING (policy ACCEPT)
@ -254,7 +254,7 @@ ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-EXT
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-EXTERNAL-IP dst,dst ADDRTYPE match dst-type LOCAL
```
## Run kube-proxy in ipvs mode
## Run kube-proxy in IPVS mode
Currently, local-up scripts, GCE scripts and kubeadm support switching IPVS proxy mode via exporting environment variables or specifying flags.
@ -313,9 +313,9 @@ Kube-proxy will fall back to IPTABLES mode if those requirements are not met.
### Local UP Cluster
Kube-proxy will run in iptables mode by default in a [local-up cluster](https://github.com/kubernetes/community/blob/master/contributors/devel/running-locally.md).
Kube-proxy will run in IPTABLES mode by default in a [local-up cluster](https://github.com/kubernetes/community/blob/master/contributors/devel/running-locally.md).
To use IPVS mode, users should export the env `KUBE_PROXY_MODE=ipvs` to specify the ipvs mode before [starting the cluster](https://github.com/kubernetes/community/blob/master/contributors/devel/running-locally.md#starting-the-cluster):
To use IPVS mode, users should export the env `KUBE_PROXY_MODE=ipvs` to specify the IPVS mode before [starting the cluster](https://github.com/kubernetes/community/blob/master/contributors/devel/running-locally.md#starting-the-cluster):
```shell
# before running `hack/local-up-cluster.sh`
export KUBE_PROXY_MODE=ipvs
@ -323,7 +323,7 @@ export KUBE_PROXY_MODE=ipvs
### GCE Cluster
Similar to local-up cluster, kube-proxy in [clusters running on GCE](https://kubernetes.io/docs/getting-started-guides/gce/) run in iptables mode by default. Users need to export the env `KUBE_PROXY_MODE=ipvs` before [starting a cluster](https://kubernetes.io/docs/getting-started-guides/gce/#starting-a-cluster):
Similar to local-up cluster, kube-proxy in [clusters running on GCE](https://kubernetes.io/docs/getting-started-guides/gce/) run in IPTABLES mode by default. Users need to export the env `KUBE_PROXY_MODE=ipvs` before [starting a cluster](https://kubernetes.io/docs/getting-started-guides/gce/#starting-a-cluster):
```shell
#before running one of the commands chosen to start a cluster:
# curl -sS https://get.k8s.io | bash
@ -360,7 +360,7 @@ kubeadm init --feature-gates=SupportIPVSProxyMode=true
to specify the ipvs mode before deploying the cluster.
**Notes**
If ipvs mode is successfully on, you should see ipvs proxy rules (use `ipvsadm`) like
If ipvs mode is successfully on, you should see IPVS proxy rules (use `ipvsadm`) like
```shell
# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
@ -374,7 +374,7 @@ or similar logs occur in kube-proxy logs (for example, `/tmp/kube-proxy.log` for
Using ipvs Proxier.
```
While there is no ipvs proxy rules or the following logs ocuurs indicate that the kube-proxy fails to use ipvs mode:
While there is no IPVS proxy rules or the following logs ocuurs indicate that the kube-proxy fails to use IPVS mode:
```
Can't use ipvs proxier, trying iptables proxier
Using iptables Proxier.
@ -422,4 +422,4 @@ Check whether the kube-proxy mode has been set to `ipvs`.
**3. Install required kernel modules and packages**
Check whether the ipvs required kernel modules have been compiled into the kernel and packages installed. (see Prerequisite)
Check whether the IPVS required kernel modules have been compiled into the kernel and packages installed. (see Prerequisite)