Automatic merge from submit-queue (batch tested with PRs 65024, 65287, 65345, 64693, 64941). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
complete ipvs proxier ut
**What this PR does / why we need it**:
Complete ipvs proxier ut:
1. Finish loadbalancer type service test.
2. Add external ip local case.
3. Add UDP type nodeport test.
4. Add TestCleanupLeftovers test.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61390
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
update ipvs readme file
**What this PR does / why we need it**:
Since some iptables rules that ipvs proxier installed have changed, we need update ipvs readme file.
**Special notes for your reviewer**:
1. fix some error, like nodePort realization.
2. Update iptables rules.
3. Add externalIP
4. Add ipset sets introduction
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Do not need to set SupportIPVSProxyMode after 1.10.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
`SupportIPVSProxyMode` is `true` by default since 1.10, there is no need to set it explicitly any more.
/area ipvs
/cc @m1093782566
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
iptables proxier_test typo
**What this PR does / why we need it**:
The definition of `makeTestService` is
```
func makeTestService(namespace, name string, svcFunc func(*api.Service)) api.Service {
...
}
```
but in function `TestClusterIPReject`, use
makeTestService(svcPortName.Namespace, svcPortName.`Namespace`, func(svc *api.Service)
should be
makeTestService(svcPortName.Namespace, svcPortName.`Name`, func(svc *api.Service)
I think it's a typo
/area kube-proxy
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix bug excludeCIDRs was not assign in func NewProxier
**What this PR does / why we need it**:
fix bug excludeCIDRs was not assign in func NewProxier
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Init ipvsInterface only when ipvs modules are present
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61074
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update ipvs docs --- check the prerequisite
**What this PR does / why we need it**:
As #63850 have been merged, the `Prerequisite` should be updated.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64034, 64072, 64146, 64059, 64161). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix session affinity for LoadBalancer service with ESIPP
**What this PR does / why we need it**:
fix session affinity for LoadBalancer service with ESIPP
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#63351
**Special notes for your reviewer**:
In cases that loadbalancer type service with externaltrafficpolicy=local and session-affinity specified, traffic to loadbalancer should only route to backends that in the same node with kube-proxy.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Enable checking whether ipvs modules are built-in or not
**What this PR does / why we need it**:
Enable checking whether ipvs modules are built-in or not.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#63801
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63787, 62003). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix localport open with --nodeport-addresses specified
**What this PR does / why we need it**:
Fix localport open with --nodeport-addresses specified.
**Which issue(s) this PR fixes**:
Fixes#61953
**Special notes for your reviewer**:
@ephur
**Release note**:
```release-note
Services can listen on same host ports on different interfaces with --nodeport-addresses specified
```
Automatic merge from submit-queue (batch tested with PRs 62833, 63585). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add comments for ipset entries in kube-proxy
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#63584
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
should use time.Since instead of time.Now().Sub
**What this PR does / why we need it**:
should use time.Since instead of time.Now().Sub
**Special notes for your reviewer**:
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix duplicate comment in iptables rule for non-local public-port rule
**What this PR does / why we need it**:
When kubernetes creates an iptable rule for "from-non-local public-port"
rule the rule gets created with two identical comment section.
The function `iptablesNonLocalNodePortArgs` creates a list of arguments
for the rule from iptablesCommonPortalArgs function. This function
already appends the arguments for the rules comments and therefore does
not require appending the comment again.
**Special notes for your reviewer**:
I'm actually not a go programmer at all and have no idea how your tests work.
I doubt that my fix will require adjustements on tests but I'm not 100% sure.
If additional work is needed, I would be glad if someone could point it out to me so I can take care of it.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix problem that ipvs can't work with hostPort
**What this PR does / why we need it**:
Make ipvs proxy mode can work with pods that have hostPort.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61938#60688 and #60305 are related too.
**Special notes for your reviewer**:
IPVS proxier will create dummy device named `kube-ipvs0`, which will maintain all ipvs virtual service address. That means all ipvs maintained clusterIP/externalIP/ingress will be treat as local address.
Then if we have a pod with hostPort, cni will attach this rule to `PREROUTING` chain:
```
KUBE-HOSTPORTS all -- 0.0.0.0/0 0.0.0.0/0 /* kube hostport portals */ ADDRTYPE match dst-type LOCAL
```
so if a service have same port with pod's hostport, then this service can't be access.
In this pr, we added `ACCESS` rule for traffic that aim to ipvs virtual service, to prevent those traffic from be blocked by other rules.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63129, 63066, 60009, 63136, 63086). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
ipvs proxy bind externalIP and lb ingress.IP to dummy interface
**What this PR does / why we need it**:
IPVS proxy bind externalIP and lb ingress.IP to dummy interface.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59976
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add --ipvs-exclude-cidrs flag to kube-proxy.
**What this PR does / why we need it**:
Add a flag to kube-proxy called --ipvs-exclude-cidrs. This flag allows a user to specify a list of CIDR ranges that should not be included in the cleanup of IPVS rules.
Fixes: #59507
**Release note**:
```
Use --ipvs-exclude-cidrs to specify a list of CIDR's which the IPVS proxier should not touch when cleaning up IPVS rules.
```
/assign @m1093782566
When kubernetes creates an iptable rule for "from-non-local public-port"
rule the rule gets created with two identical comment section.
The function `iptablesNonLocalNodePortArgs` creates a list of arguments
for the rule from iptablesCommonPortalArgs function. This function
already appends the arguments for the rules comments and therefore does
not require appending the comment again.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix externaltrafficpolicy=local related ipvs ci case
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61328
**Special notes for your reviewer**:
To realize externalTrafficPolicy=local, but do not affect traffic inside the cluster.
If thie pr got merged, the iptables rules of ipvs proxy mode ESIPP should be like(for loadbalance case):
```
Chain KUBE-FIRE-WALL (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-LB-INGRESS-LOCAL dst,dst
KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0 /* mark MASQ for external traffic policy not local */
Chain KUBE-MARK-DROP (0 references)
target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x8000
Chain KUBE-MARK-MASQ (3 references)
target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x4000
Chain KUBE-NODE-PORT (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-NODE-PORT-LOCAL-TCP dst
KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0 /* mark MASQ for external traffic policy not local */
Chain KUBE-POSTROUTING (0 references)
target prot opt source destination
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service traffic requiring SNAT */ mark match 0x4000/0x4000
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-LOOP-BACK dst,dst,src
Chain KUBE-SERVICES (2 references)
target prot opt source destination
KUBE-MARK-MASQ all -- !10.64.0.0/14 0.0.0.0/0 match-set KUBE-CLUSTER-IP dst,dst
KUBE-FIRE-WALL all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-LOAD-BALANCER-MASQ dst,dst
KUBE-NODE-PORT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp match-set KUBE-NODE-PORT-TCP dst
```
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61806, 61508, 62075, 62079, 62052). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
make use of auto generated LoadBalancerStatus.DeepCopy
Remove LoadBalancerStatusDeepCopy in helpers.go, and use the auto generated `LoadBalancerStatus.DeepCopy` in pkg/apis/core/zz_generated.deepcopy.go.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update IPVS doc
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59505
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove YEAR field of all generated files and fix kubernetes boilerplate checker
**What this PR does / why we need it**:
Remove YEAR field of all generated files and fix kubernetes boilerplate checker
xref: [remove YEAR fileds in gengo #91](https://github.com/kubernetes/gengo/pull/91)
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes [#gengo/issues/24](https://github.com/kubernetes/gengo/issues/24)
**Special notes for your reviewer**:
/cc @thockin @lavalamp @sttts
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 53689, 56880, 55856, 59289, 60249). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Harden kube-proxy for unmatched IP versions
**What this PR does / why we need it**:
This PR makes kube-proxy omits & logs & emits event for unmatched IP versions configuration (IPv6 address in IPv4 mode or IPv4 address in IPv6 mode).
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57219
**Special notes for your reviewer**:
**Release note**:
```release-note
Fix the issue in kube-proxy iptables/ipvs mode to properly handle incorrect IP version.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove old featureGate flag in Readme
**What this PR does / why we need it**:
This PR corrects the Readme for ipvs mode: https://github.com/kubernetes/kubernetes/blob/master/pkg/proxy/ipvs/README.md. `kubeadm init` doesn't support the `--feature-gates=SupportIPVSProxyMode=true` flag anymore since IPVS is in beta and not alpha anymore.
```
sudo kubeadm init --help
Run this command in order to set up the Kubernetes master.
Usage:
kubeadm init [flags]
Flags:
--apiserver-advertise-address string The IP address the API Server will advertise it's listening on. Specify '0.0.0.0' to use the address of the default network interface.
--apiserver-bind-port int32 Port for the API Server to bind to. (default 6443)
--apiserver-cert-extra-sans stringSlice Optional extra Subject Alternative Names (SANs) to use for the API Server serving certificate. Can be both IP addresses and DNS names.
--cert-dir string The path where to save and store the certificates. (default "/etc/kubernetes/pki")
--config string Path to kubeadm config file. WARNING: Usage of a configuration file is experimental.
--cri-socket string Specify the CRI socket to connect to. (default "/var/run/dockershim.sock")
--dry-run Don't apply any changes; just output what would be done.
--feature-gates string A set of key=value pairs that describe feature gates for various features. Options are:
CoreDNS=true|false (ALPHA - default=false)
DynamicKubeletConfig=true|false (ALPHA - default=false)
SelfHosting=true|false (ALPHA - default=false)
StoreCertsInSecrets=true|false (ALPHA - default=false)
--ignore-preflight-errors stringSlice A list of checks whose errors will be shown as warnings. Example: 'IsPrivilegedUser,Swap'. Value 'all' ignores errors from all checks.
--kubernetes-version string Choose a specific Kubernetes version for the control plane. (default "stable-1.9")
--node-name string Specify the node name.
--pod-network-cidr string Specify range of IP addresses for the pod network. If set, the control plane will automatically allocate CIDRs for every node.
--service-cidr string Use alternative range of IP address for service VIPs. (default "10.96.0.0/12")
--service-dns-domain string Use alternative domain for services, e.g. "myorg.internal". (default "cluster.local")
--skip-token-print Skip printing of the default bootstrap token generated by 'kubeadm init'.
--token string The token to use for establishing bidirectional trust between nodes and masters.
--token-ttl duration The duration before the bootstrap token is automatically deleted. If set to '0', the token will never expire. (default 24h0m0s)
```
**Special notes for your reviewer**:
Used kubeadm version:
```bash
sudo kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.3", GitCommit:"d2835416544f298c919e2ead3be3d0864b52323b", GitTreeState:"clean", BuildDate:"2018-02-07T11:55:20Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}
```
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60430, 60115, 58052, 60355, 60116). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Make nodeport ip configurable
**What this PR does / why we need it**:
By default, kube-proxy accepts everything from NodePort without any filter. It can be a problem for nodes which has both public and private NICs, and people only want to provide a service in private network and avoid exposing any internal service on the public IPs.
This PR makes nodeport ip configurable.
**Which issue(s) this PR fixes**:
Closes: #21070
**Special notes for your reviewer**:
Design proposal see: https://github.com/kubernetes/community/pull/1547
Issue in feature repo: https://github.com/kubernetes/features/issues/539
**Release note**:
```release-note
Make NodePort IP addresses configurable
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Only run connection-rejecting rules on new connections
Kube-proxy has two iptables chains full of rules to reject incoming connections to services that don't have any endpoints. Currently these rules get tested against all incoming packets, but that's unnecessary; if a connection to a given service has already been established, then we can't have been rejecting connections to that service. By only checking the first packet in each new connection, we can get rid of a lot of unnecessary checks on incoming traffic.
Fixes#56842
**Release note**:
```release-note
Additional changes to iptables kube-proxy backend to improve performance on clusters with very large numbers of services.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Delete stale UDP conntrack entries that use hostPort
**What this PR does / why we need it**:
This PR introduces a change to delete stale conntrack entries for UDP connections, specifically for udp connections that use hostPort. When the pod listening on that udp port get updated/restarted(and gets a new ip address), these entries need to be flushed so that ongoing udp connections can recover once the pod is back and the new iptables rules have been installed.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59033
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55637, 57461, 60268, 60290, 60210). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Don't create no-op iptables rules for services with no endpoints
Currently for all services we create `-t nat -A KUBE-SERVICES` rules that match the destination IPs (ClusterIP, ExternalIP, NodePort IPs, etc) and then jump to the appropriate `KUBE-SVC-XXXXXX` chain. But if the service has no endpoints then the `KUBE-SVC-XXXXXX` chain will be empty and so nothing happens except that we wasted time (a) forcing iptables-restore to parse the match rules, and (b) forcing the kernel to test matches that aren't going to have any effect.
This PR gets rid of the match rules in this case. Which is to say, it changes things so that every incoming service packet is matched *either* by nat rules to rewrite it *or* by filter rules to ICMP reject it, but not both. (Actually, that's not quite true: there are no filter rules to reject Ingress-addressed packets, and I *think* that's a bug?)
I also got rid of some comments that seemed redundant.
The patch is mostly reindentation, so best viewed with `diff -w`.
Partial fix for #56842 / Related to #56164 (which it conflicts with but I'll fix that after one or the other merges).
**Release note**:
```release-note
Removed some redundant rules created by the iptables proxier, to improve performance on systems with very many services.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Migrate FeatureGates type of kube-proxy from string to map[string]bool
**What this PR does / why we need it**:
Migration of FeatureGates type. This is a follow-up of #53025.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
ref: #53025https://github.com/kubernetes/kubernetes/pull/57754#discussion_r160023416
**Special notes for your reviewer**:
/cc @luxas @mtaufen @ncdc
**Release note**:
```release-note
action required: kube-proxy: feature gates are now specified as a map when provided via a JSON or YAML KubeProxyConfiguration, rather than as a string of key-value pairs.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Split KUBE-SERVICES chain to re-shrink the INPUT chain
**What this PR does / why we need it**:
#43972 added an iptables rule "`-A INPUT -j KUBE-SERVICES`" to make NodePort ICMP rejection work. (Previously the KUBE-SERVICES chain was only run from OUTPUT, not INPUT.) #44547 extended that patch for ExternalIP rejection as well.
However, the KUBE-SERVICES chain may potentially have a very large number of ICMP reject rules for plain ClusterIP services (the ones that get run from OUTPUT), and it seems that for some reason the kernel is much more sensitive to the length of the INPUT chain than it is to the length of the OUTPUT chain. So a node that worked fine with kube 1.6 (when KUBE-SERVICES was only run from OUTPUT) might fall over with kube 1.7 (with KUBE-SERVICES being run from both INPUT and OUTPUT).
(Specifically, a node with about 5000 ClusterIP reject rules that ran fine with OpenShift 3.6 [kube 1.6] slowed almost to a complete halt with OpenShift 3.7 [kube 1.7].)
This PR fixes things by splitting out the "new" part of KUBE-SERVICES (NodePort and ExternalIP reject rules) into a separate KUBE-EXTERNAL-SERVICES chain run from INPUT, and moves KUBE-SERVICES back to being only run from OUTPUT. (So, yes, this assumes that you don't have 5000 NodePort/ExternalIP services, but, if you do, there's not much we can do, since those rules *have* to be run on the INPUT side.)
Oh, and I left in the code to clean up the "`-A INPUT -j KUBE-SERVICES`" rule even though we don't generate it any more, so it gets fixed on upgrade.
**Release note**:
```release-note
Reorganized iptables rules to fix a performance regression on clusters with thousands of services.
```
@kubernetes/sig-network-bugs @kubernetes/rh-networking
There's a disconnect between 12d4eac038
and the earlier a6af827898. One says
KUBEPROXY_MODE and another says KUBE_PROXY_MODE. Let's just pick one.
Currently we fail with the following error:
hack/local-up-cluster.sh: line 808: KUBEPROXY_MODE: unbound variable
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Refactor kube-proxy service/endpoints update so that can be consumed among different proxiers
**What this PR does / why we need it**:
There are huge duplication among different proxiers. For example, the service/endpoints list/watch part in iptables, ipvs and windows kernel mode(to be get in soon).
I think the more places this is replicated the harder it becomes to keep correct. We may need to refactor it and let different proxiers consume the same code.
**Which issue this PR fixes**:
fixes#52464
**Special notes for your reviewer**:
* This refactor reduces **500** Lines in iptables proxy, so it will reduce **500*N**(number of proxiers) lines in total. People no need to care the service/endpoints update logic any more and can be more focus on proxy logic.
* I would like to do the following things in follow-ups:
1. rsync it to ipvs proxier
2. rsync it to winkernel proxier
**Release note**:
```release-note
Refactor kube-proxy service/endpoints update so that can be consumed among different proxiers
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix all the typos across the project
**What this PR does / why we need it**:
There are lots of typos across the project. We should avoid small PRs on fixing those annoying typos, which is time-consuming and low efficient.
This PR does fix all the typos across the project currently. And with #59463, typos could be avoided when a new PR gets merged.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
/sig testing
/area test-infra
/sig release
/cc @ixdy
/assign @fejta
**Release note**:
```release-note
None
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Abstract some duplicated code in the iptables proxier
Reorganizes the iptables proxier code so we only have the list of "-A FOO -j KUBE-BAR" rules in one place rather than duplicating the same list in multiple places. Split out from #56164 for ease of review/merging.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
validate ipset set and entry - ipvs proxier validate entry before adding it and validate set before creating it
**What this PR does / why we need it**:
validate ipset util.
**Which issue(s) this PR fixes**:
Fixes#57269
**Special notes for your reviewer**:
I splitted this PR into some small pieces for the sake of code review.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add kubeproxyconfig round trip test
**What this PR does / why we need it**:
This is a follow-up of https://github.com/kubernetes/kubernetes/pull/55961.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
ref: #55959
**Special notes for your reviewer**:
/cc @luxas @mtaufen @ncdc
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kube-proxy: fix field name comments & json tags
**What this PR does / why we need it**: correct some minor issues in the comments and json tags for some of the fields in the kube-proxy config structs.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
The udpTimeoutMilliseconds field in the kube-proxy configuration file has been renamed to udpIdleTimeout. Action required: administrators need to update their files accordingly.
```
This was extracted from my currently unmerged f074b28fe9, as requested [here](https://github.com/kubernetes/kubernetes/pull/52198#pullrequestreview-85538637).
@kubernetes/sig-network-pr-reviews @luxas
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Initialize IPVS proxy OWNERS file
**What this PR does / why we need it**:
Initialize IPVS proxy OWNERS file. There are 3 initial members: @thockin, @brendandburns and me :)
@thockin and @brendandburns has reviewed most of the IPVS proxier PRs.
I initialized this package and authored most of the commits.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55606, 59185, 58763, 59072, 59251). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
make ipvs 'cleanupIptablesLeftovers' simplier
Since there are only four iptables chains in ipvs mode, no need to restore all chains when cleanup iptables chain created by ipvs.
**What this PR does / why we need it**:
Make ipvs `cleanupIptablesLeftovers` function much simplier.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56689
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Ignore address not exist error when unbind IPVS serivce address from dummy device
**What this PR does / why we need it**:
Ignore address not exist error when unbind IPVS serivce address from dummy device.
**Which issue(s) this PR fixes**:
Fixes#58681
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 53778, 58378, 58830). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add more ipv6 support in userspace proxier
Signed-off-by: Ma Shimiao <mashimiao.fnst@cn.fujitsu.com>
**What this PR does / why we need it**:
format `%s:%d/%s"` is not suitable for ipv6 IP, replace with net.JoinHostPort
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 58300, 58530, 57942, 58543). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix nodeport localhost martian source error
**What this PR does / why we need it**:
kube-proxy NodePort access via localhost, with externalTrafficPolicy=Local will trigger martian source error.
This PR fixes nodeport localhost martian source error.
**Which issue(s) this PR fixes**:
Fixes#57922
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove ExternalTrafficLocalOnly from kube_feature gate
*What this PR does / why we need it**:
This PR is for v1.10.
External Source IP Preservation (ESIPP) had been promoted to GA since 1.7. Following the proposal on https://github.com/kubernetes/kubernetes/issues/46404#issuecomment-303939180, we should be able to remove it from feature gate now.
Added release note to announce this.
Also ref the previous attempt: https://github.com/kubernetes/kubernetes/pull/45857.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56645
**Special notes for your reviewer**:
**Release note**:
```release-note
"ExternalTrafficLocalOnly" has been removed from feature gate. It has been a GA feature since v1.7.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix ipvs proxy mode kubeadm usage
**What this PR does / why we need it**:
Fix ipvs proxy mode usage of kubeadm in ipvs README file.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58375, 58306, 58386, 57420, 58035). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix windows UT for proxy mode
**What this PR does / why we need it**:
/kind bug
Fix windows UT for proxy mode.
Run UTs for `kubeproxyconfig/validation` in windows,
```
=== RUN TestValidateKubeProxyConfiguration
--- FAIL: TestValidateKubeProxyConfiguration (0.00s)
validation_test.go:83: expected success: [KubeProxyConfiguration.Mode.ProxyMode: Invalid value: "ipvs": must be kernelspace,userspace or blank (blank means the most-available proxy [currently userspace])]
```
That's because proxy mode "IPVS" is not valid in windows.
**Which issue(s) this PR fixes**:
Fixes#58037
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57906, 57425, 56939, 57317, 57762). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add "Not Found" error for ipset set & entry delete
**What this PR does / why we need it**:
Add "not found error" for ipset set and entry delete. For example,
```
[root@SHA1000130405 k8s-yml]# ipset list
Name: KUBE-LOOP-BACK
Type: hash:ip,port,ip
Revision: 2
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 16776
References: 1
Members:
172.17.0.2,udp:53,172.17.0.2
172.17.0.2,tcp:53,172.17.0.2
100.106.89.164,tcp:6443,100.106.89.164
[root@SHA1000130405 k8s-yml]# ipset delete foo
ipset v6.19: The set with the given name does not exist
[root@SHA1000130405 k8s-yml]# ipset del KUBE-LOOP-BACK 1.2.3.4,tcp:80,1.2.3.4
ipset v6.19: Syntax error: Third element is missing from 1.2.3.4,tcp:80,1.2.3.4.
```
Just need to filter the error message "does not exist" or "element is missing". We should ignore these error when try to make sure they are deleted.
**Which issue(s) this PR fixes**:
Fixes#57318
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update gengo version to include goimports formatter
Update gengo which now uses goimports to format code and organize imports.
Fixes#55542
**Special notes for your reviewer**:
Updates version of k8s.io/gengo
Takes new dependency on golang.org/x/tools/imports and golang.org/x/tools/go/ast/astutil
**Release Notes**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57699, 57657). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix ipvs virutal server update
**What this PR does / why we need it**:
Fix ipvs virutal server update.
**Which issue(s) this PR fixes**:
Fixes#57698
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57584, 57679). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix the problem that kube-proxy can't create clusterIP ipvs rules when externalTrafficPolicy=Local
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57678
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57292, 56274, 57435, 57438, 57429). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
replace syscall with sys/unix pkg in ipvs/netlink call
**What this PR does / why we need it**:
This PR replaces syscall with sys/unix pkg in ipvs/netlink call as the Go doc for syscall says:
NOTE: This package is locked down.
Code outside the standard Go repository should be migrated to use the corresponding package in the golang.org/x/sys repository.
That is also where updates required by new systems or versions should be applied.
See https://golang.org/s/go1.4-syscall for more information.
**Which issue(s) this PR fixes**:
Fixes #57430
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @thockin @brendandburns
Automatic merge from submit-queue (batch tested with PRs 54379, 56593, 56685, 54174, 57309). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix ipvs proxier nodeport
**What this PR does / why we need it**:
Fix ipvs proxier nodeport.
**Which issue(s) this PR fixes**:
Fixes#55923
**Special notes for your reviewer**:
We bump the netwlink version in the 1st commit because:
* We call `netlink.RouteListFiltered()` to filter LOCAL type addresses from kernel route table. `netlink.RouteListFiltered()` exists in newer-version netlink package
* newer-version netlink package migrate `syscall` to `golang.org/x/sys/unix`, k8s cross-build can benefit from it as well. The Go doc for syscall says:
> NOTE: This package is locked down. Code outside the standard Go repository should be migrated to use the corresponding package in the golang.org/x/sys repository. That is also where updates required by new systems or versions should be applied. See https://golang.org/s/go1.4-syscall for more information.
**Release note**:
```release-note
Fix ipvs proxier nodeport eth* assumption
```
Automatic merge from submit-queue (batch tested with PRs 57127, 57011, 56754, 56601, 56483). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Enhance proxy mode validation
**What this PR does / why we need it**:
Proxy modes are list multiple times in validateProxyModeWindows and validateProxyModeLinux. This PR enhances the validation logic and only list them once.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Refer https://github.com/kubernetes/kubernetes/pull/56529/files#r156448205.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @thockin
Automatic merge from submit-queue (batch tested with PRs 56894, 56940, 55223, 56554, 54985). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove dead code in winkernel proxy UT
**What this PR does / why we need it**:
There is a `+build windows` tag in `pkg/proxy/winkernel/proxier_test.go`, it means the UTs will not run in upstream CI. Plus, the UT codes in `pkg/proxy/winkernel/proxier_test.go` are not for winkernel proxier, instead they are for iptables proxier!
We should remove the dead code in winkernel proxy UT and then create the winkernel-specific UTs.
**Which issue(s) this PR fixes**:
Fixes#56555
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Test Proxy Mode
**What this PR does / why we need it**:
Currently, IPVS-based kube-proxy is protected by feature gateway and there is no UT for it. We should test ipvs proxy mode when feature gateway is set.
**Which issue(s) this PR fixes**:
Fixes#56404
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57211, 56150, 56368, 56271, 55957). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Validate IPVSConfiguration only when IPVS mode is enabled.
**What this PR does / why we need it**:
It's strange for users that they should configuration valid values for ipvs params even when they don't use ipvs mode. Users don't need to care about these params in this case.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
/cc @m1093782566
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55954, 56037, 55866, 55984, 54994). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove dead code
**What this PR does / why we need it**:
remove dead code in IPVS kube-proxy
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @thockin
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update IPVS README
**What this PR does / why we need it**:
People complained that they feel confused about how to use IPVS-based kube-proxy correctly, probably we may need to provide them a better README.
**Which issue(s) this PR fixes**:
xref: #56319
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixed typo in IPVS documentation
**What this PR does / why we need it**:
Fixed documentation typo
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixed documentation typo in IPVS README.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Using ipset doing SNAT and packet filter in IPVS kube-proxy
**What this PR does / why we need it**:
Try ipset in ipvs proxy mode.
**Which issue this PR fixes**:
fixes#54203
xref: #53393, #53775
**Special notes for your reviewer**:
**Release note**:
```release-note
Using ipset doing SNAT and packet filtering in IPVS kube-proxy
```
/sig network
/area kube-proxy
Automatic merge from submit-queue (batch tested with PRs 55988, 53555, 55858). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add IPv6 and negative UT test cases for proxier's deleteEndpointConnections
This change adds IPv6 and negative UT test cases for the proxier's deleteEndpointConnections.
Changes include:
- Add IPv6 UT test cases to TestDeleteEndpointConnections.
- Add negative UT test case to TestDeleteEndpointConnections for
handling case where no connections need clearing (benign error).
- Add negative UT test case to test unexpected error.
- Reorganize UT in TestDeleteEndpointConnections so that the fake
command executor's command and scripted responses are generated on
the fly based on the test case table (rather than using a fixed
set of commands/responses that will need to be updated every time
test cases are added/deleted).
- Create the proxier service map in real time, based on the test case
table (rather than using a fixed service map that will need to be updated
every time test cases are added/deleted).
fixes#53554
**What this PR does / why we need it**:
This change adds IPv6 and negative UT test cases for the proxier's
deleteEndpointConnections.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#53554
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55757, 55741, 55817, 55834, 51530). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixes Issue 55816: Removes unneeded IPPart error
**What this PR does / why we need it**:
Previously proxy was incorrectly logging an error message for the IPPart function. The PR fixes this logging behavior to only log `invalid IP part` for invalid IP:Port combinations.
**Which issue(s) this PR fixes**:
Fixes # https://github.com/kubernetes/kubernetes/issues/55816
**Special notes for your reviewer**:
None
**Release note**:
```
none
```
Automatic merge from submit-queue (batch tested with PRs 55757, 55741, 55817, 55834, 51530). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove AddToGroupVersion in kubeproxy scheme
ref https://github.com/kubernetes/kubernetes/pull/53645#discussion_r148850279
Signed-off-by: yanxuean <yan.xuean@zte.com.cn>
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @ncdc
Automatic merge from submit-queue (batch tested with PRs 53780, 55663, 55321, 52421, 55659). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Validate kube-proxy options
**What this PR does / why we need it**:
Validate ipvs proxy options
**Which issue this PR fixes** : fixes#53852
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55009, 55532, 55601, 52569, 55533). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Kube-proxy adds forward rules to ensure NodePorts work
**What this PR does / why we need it**:
Updates kube-proxy to set up proper forwarding so that NodePorts work with docker 1.13 without depending on iptables FORWARD being changed manually/externally.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#39823
**Special notes for your reviewer**:
@thockin I used option number 2 that I mentioned in the #39823 issue, please let me know what you think about this change. If you are happy with the change then I can try to add tests but may need a little direction about what and where to add them.
**Release note**:
```release-note
Add iptables rules to allow Pod traffic even when default iptables policy is to reject.
```
Automatic merge from submit-queue (batch tested with PRs 55009, 55532, 55601, 52569, 55533). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix ipvs/proxy getLocalIPs inconsistency with iptables/proxy
**What this PR does / why we need it**:
* Fix ipvs/proxy `getLocalIPs()` inconsistency with iptables/proxy
* validate the ip address before pkg/proxy/util IPPart() return ip string.
**Which issue(s) this PR fixes** :
Fixes#55612
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 53580, 55568). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Replace sets.List() with sets.UnsortedList() in pkg/proxy
**What this PR does / why we need it**:
Replace sets.List() with sets.UnsortedList() in pkg/proxy - sets.List() will sort the result array, we don't need sorted array in pkg/proxy. Using sets.UnsortedList() can reduce the unnecessary overhead spending.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
@wojtek-t wdyt ^_^
**Release note**:
```release-note
NONE
```
/sig network
Automatic merge from submit-queue (batch tested with PRs 55247, 55324, 55261, 55147, 54052). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Implement dummy device operation by netlink for ipvs proxier
**What this PR does / why we need it**:
Currently, we implement network interface operations, e.g. create a dummy device and delete a dummy interface via ip cmd and then collect the exist code. For example, we assume the following command:
```
ip link add kube-ipvs0 type dummy
```
returns 2 means the dummy device `kube-ipvs0` exists, see https://github.com/kubernetes/kubernetes/blob/master/pkg/proxy/ipvs/proxier.go#L1529 .
However, from the man page of ip command, exit status 2 just means an error was reported by the kernel, see http://man7.org/linux/man-pages/man8/ip.8.html#EXIT_STATUS.
So, that's a bug.
This PR implements dummy device operation by netlink for ipvs proxier so that can get ride of ip command operations.
**Which issue this PR fixes**:
fixes#54054
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig network
/area kube-proxy
/kind bug
Automatic merge from submit-queue (batch tested with PRs 55247, 55324, 55261, 55147, 54052). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Restore kube-proxy's support for 0 values for conntrack settings
**What this PR does / why we need it**: re-allow 0 values for kube-proxy conntrack min, max, max per core, tcp close wait timeout, tcp established timeout.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#50787
**Special notes for your reviewer**:
- I adjusted validation to allow for 0 values for some of the conntrack settings, as otherwise the "leave the limit as-is" logic wouldn't be allowed.
- I moved the loading of the config file from the cobra command's Validate method to Complete. This way, the config is fully resolved before validation happens. Otherwise, it just validates the default config values first, and _then_ the config is loaded.
- I think I got all the default values & nil checking correct, but please review carefully!
**Release note**:
```release-note
Restored kube-proxy's support for 0 values for conntrack min, max, max per core, tcp close wait timeout, and tcp established timeout.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
update ipvs readme file
Add the way to choose ipvs mode when using kubeadm with a configuration file
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix lint errors in kubeproxyconfig types.
**What this PR does / why we need it**:
Fix some outstanding lint errors reported by k8s-ci-robot in https://github.com/kubernetes/kubernetes/pull/53645/files#r149212051.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#53645
**Special notes for your reviewer**:
/cc @cblecker @ncdc
**Release note**:
```release-note
NONE
```
When kube-proxy was refactored to use a configuration file, the ability
to use 0 for conntrack min, max, max per core, and tcp timeouts was
inadvertently broken; if you specified 0, it would instead apply the
default value from defaults.go.
This change restores the ability to use 0 to mean 0.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add windows kernel mode kubeproxy reviewers
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixing 'targetport' to service 'port' mapping
Looks like the order should be reversed for this to work.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#54968
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kube-proxy IPVS: Fix IPVS availability check
**What this PR does / why we need it**:
The current implementation of `CanUseIPVSProxier()` reads `/proc/modules`
to check whether IPVS related kernel modules can be loaded.
You might get a false-negative when the kernel modules are installed to
the system but haven't been loaded into the kernel.
This patch firstly try to run `modprobe` to load specified kernel
modules, then just log warnings if error occured.
Secondly, it will check loaded kernel modules by reading
`/proc/modules`, return an error if any required module is missing.
This change will not break the compatability of existing implementation.
Running kube-proxy in a container without mounting `/lib/modules` will
cause `modprobe` warnings, but not raise an error if all required modules are
present in the host kernel.
**Special notes for your reviewer**:
**Release note**:
```release-note
Fix IPVS availability check
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
clean up legacy ipv4/32 in ipvs proxy
**What this PR does / why we need it**:
clean up legacy ipv4/32 in ipvs proxy
**Which issue(s) this PR fixes**: closes#51866
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig network
/area ipv6
/assign @thockin
cc @danehans
The current implementation of `CanUseIPVSProxier()` reads `/proc/modules`
to check whether IPVS related kernel modules can be loaded.
You might get a false-negative when the kernel modules are installed to
the system but haven't been loaded into the kernel.
This patch firstly try to run `modprobe` to load specified kernel
modules, then just log warnings if error occured.
Secondly, it will check loaded kernel modules by reading
`/proc/modules`, return an error if any required module is missing.
This change will not break the compatability of existing implementation.
Running kube-proxy in a container without mounting `/lib/modules` will
cause `modprobe` warnings, but not raise an error if all required modules are
present in the host kernel.
Automatic merge from submit-queue (batch tested with PRs 54774, 54820, 52192, 54827). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix IPV6 judgement bug and add UTs
**What this PR does / why we need it**:
Fix IPV6 judgement bug and add UTs.
**Which issue this PR fixes** : closes#54775
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig network
/area ipv6
/kind bug
Automatic merge from submit-queue (batch tested with PRs 53796, 54666, 54516, 54625, 54704). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove +linux restriction in ipvs/fake for running UTs in !linux platform
**What this PR does / why we need it**:
Remove +linux restriction in ipvs/fake for running UTs in !linux platform
**Which issue this PR fixes**:
fixes#54667
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig network
/kind bug
Automatic merge from submit-queue (batch tested with PRs 54635, 54250, 54657, 54696, 54700). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove iptables log on restore failure
Don't log the set of rules at v2 in kube-proxy on error.
The rules are displayed at v5 before the restore is attempted.
In a large cluster the report can generate up to 100000 lines.
A partial report is only helpful if the problem is displayed
in the partial report.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Don't log the set of rules at v2 in kube-proxy on error.
The rules are displayed at v5 before the restore is attempted.
In a large cluster the report can generate up to 100000 lines.
A partial report is only helpful if the problem is displayed
in the partial report.
Automatic merge from submit-queue (batch tested with PRs 54656, 54552, 54389, 53634, 54408). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove hard code of session affinity timeout in winows kernel proxy
**What this PR does / why we need it**:
Remove hard code of session affinity timeout in winows kernel proxy - we have already done this in userspace, iptables and ipvs proxy.
**Which issue this PR fixes**:
fixes#53636
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig network
/area kube-proxy
Automatic merge from submit-queue (batch tested with PRs 52479, 53956). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Register sync proxy rules latency metrics in app level
**What this PR does / why we need it**:
IMO, should may should register proxy metrics in app level instead of in specific proxy mode, e.g. iptables, ipvs, winkernel...
By registering sync proxy rules latency metrics in app level, we can reuse codes among different proxiers.
**Which issue this PR fixes**:
closes#53957
**Special notes for your reviewer**:
@wojtek-t What do you think about it?
**Release note**:
```release-note
NONE
```
Jeff Grafton