There's a disconnect between 12d4eac038
and the earlier a6af827898. One says
KUBEPROXY_MODE and another says KUBE_PROXY_MODE. Let's just pick one.
Currently we fail with the following error:
hack/local-up-cluster.sh: line 808: KUBEPROXY_MODE: unbound variable
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Refactor kube-proxy service/endpoints update so that can be consumed among different proxiers
**What this PR does / why we need it**:
There are huge duplication among different proxiers. For example, the service/endpoints list/watch part in iptables, ipvs and windows kernel mode(to be get in soon).
I think the more places this is replicated the harder it becomes to keep correct. We may need to refactor it and let different proxiers consume the same code.
**Which issue this PR fixes**:
fixes#52464
**Special notes for your reviewer**:
* This refactor reduces **500** Lines in iptables proxy, so it will reduce **500*N**(number of proxiers) lines in total. People no need to care the service/endpoints update logic any more and can be more focus on proxy logic.
* I would like to do the following things in follow-ups:
1. rsync it to ipvs proxier
2. rsync it to winkernel proxier
**Release note**:
```release-note
Refactor kube-proxy service/endpoints update so that can be consumed among different proxiers
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix all the typos across the project
**What this PR does / why we need it**:
There are lots of typos across the project. We should avoid small PRs on fixing those annoying typos, which is time-consuming and low efficient.
This PR does fix all the typos across the project currently. And with #59463, typos could be avoided when a new PR gets merged.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
/sig testing
/area test-infra
/sig release
/cc @ixdy
/assign @fejta
**Release note**:
```release-note
None
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Abstract some duplicated code in the iptables proxier
Reorganizes the iptables proxier code so we only have the list of "-A FOO -j KUBE-BAR" rules in one place rather than duplicating the same list in multiple places. Split out from #56164 for ease of review/merging.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
validate ipset set and entry - ipvs proxier validate entry before adding it and validate set before creating it
**What this PR does / why we need it**:
validate ipset util.
**Which issue(s) this PR fixes**:
Fixes#57269
**Special notes for your reviewer**:
I splitted this PR into some small pieces for the sake of code review.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add kubeproxyconfig round trip test
**What this PR does / why we need it**:
This is a follow-up of https://github.com/kubernetes/kubernetes/pull/55961.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
ref: #55959
**Special notes for your reviewer**:
/cc @luxas @mtaufen @ncdc
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kube-proxy: fix field name comments & json tags
**What this PR does / why we need it**: correct some minor issues in the comments and json tags for some of the fields in the kube-proxy config structs.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
The udpTimeoutMilliseconds field in the kube-proxy configuration file has been renamed to udpIdleTimeout. Action required: administrators need to update their files accordingly.
```
This was extracted from my currently unmerged f074b28fe9, as requested [here](https://github.com/kubernetes/kubernetes/pull/52198#pullrequestreview-85538637).
@kubernetes/sig-network-pr-reviews @luxas
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Initialize IPVS proxy OWNERS file
**What this PR does / why we need it**:
Initialize IPVS proxy OWNERS file. There are 3 initial members: @thockin, @brendandburns and me :)
@thockin and @brendandburns has reviewed most of the IPVS proxier PRs.
I initialized this package and authored most of the commits.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55606, 59185, 58763, 59072, 59251). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
make ipvs 'cleanupIptablesLeftovers' simplier
Since there are only four iptables chains in ipvs mode, no need to restore all chains when cleanup iptables chain created by ipvs.
**What this PR does / why we need it**:
Make ipvs `cleanupIptablesLeftovers` function much simplier.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56689
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Ignore address not exist error when unbind IPVS serivce address from dummy device
**What this PR does / why we need it**:
Ignore address not exist error when unbind IPVS serivce address from dummy device.
**Which issue(s) this PR fixes**:
Fixes#58681
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 53778, 58378, 58830). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add more ipv6 support in userspace proxier
Signed-off-by: Ma Shimiao <mashimiao.fnst@cn.fujitsu.com>
**What this PR does / why we need it**:
format `%s:%d/%s"` is not suitable for ipv6 IP, replace with net.JoinHostPort
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 58300, 58530, 57942, 58543). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix nodeport localhost martian source error
**What this PR does / why we need it**:
kube-proxy NodePort access via localhost, with externalTrafficPolicy=Local will trigger martian source error.
This PR fixes nodeport localhost martian source error.
**Which issue(s) this PR fixes**:
Fixes#57922
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove ExternalTrafficLocalOnly from kube_feature gate
*What this PR does / why we need it**:
This PR is for v1.10.
External Source IP Preservation (ESIPP) had been promoted to GA since 1.7. Following the proposal on https://github.com/kubernetes/kubernetes/issues/46404#issuecomment-303939180, we should be able to remove it from feature gate now.
Added release note to announce this.
Also ref the previous attempt: https://github.com/kubernetes/kubernetes/pull/45857.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56645
**Special notes for your reviewer**:
**Release note**:
```release-note
"ExternalTrafficLocalOnly" has been removed from feature gate. It has been a GA feature since v1.7.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix ipvs proxy mode kubeadm usage
**What this PR does / why we need it**:
Fix ipvs proxy mode usage of kubeadm in ipvs README file.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58375, 58306, 58386, 57420, 58035). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix windows UT for proxy mode
**What this PR does / why we need it**:
/kind bug
Fix windows UT for proxy mode.
Run UTs for `kubeproxyconfig/validation` in windows,
```
=== RUN TestValidateKubeProxyConfiguration
--- FAIL: TestValidateKubeProxyConfiguration (0.00s)
validation_test.go:83: expected success: [KubeProxyConfiguration.Mode.ProxyMode: Invalid value: "ipvs": must be kernelspace,userspace or blank (blank means the most-available proxy [currently userspace])]
```
That's because proxy mode "IPVS" is not valid in windows.
**Which issue(s) this PR fixes**:
Fixes#58037
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57906, 57425, 56939, 57317, 57762). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add "Not Found" error for ipset set & entry delete
**What this PR does / why we need it**:
Add "not found error" for ipset set and entry delete. For example,
```
[root@SHA1000130405 k8s-yml]# ipset list
Name: KUBE-LOOP-BACK
Type: hash:ip,port,ip
Revision: 2
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 16776
References: 1
Members:
172.17.0.2,udp:53,172.17.0.2
172.17.0.2,tcp:53,172.17.0.2
100.106.89.164,tcp:6443,100.106.89.164
[root@SHA1000130405 k8s-yml]# ipset delete foo
ipset v6.19: The set with the given name does not exist
[root@SHA1000130405 k8s-yml]# ipset del KUBE-LOOP-BACK 1.2.3.4,tcp:80,1.2.3.4
ipset v6.19: Syntax error: Third element is missing from 1.2.3.4,tcp:80,1.2.3.4.
```
Just need to filter the error message "does not exist" or "element is missing". We should ignore these error when try to make sure they are deleted.
**Which issue(s) this PR fixes**:
Fixes#57318
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update gengo version to include goimports formatter
Update gengo which now uses goimports to format code and organize imports.
Fixes#55542
**Special notes for your reviewer**:
Updates version of k8s.io/gengo
Takes new dependency on golang.org/x/tools/imports and golang.org/x/tools/go/ast/astutil
**Release Notes**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57699, 57657). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix ipvs virutal server update
**What this PR does / why we need it**:
Fix ipvs virutal server update.
**Which issue(s) this PR fixes**:
Fixes#57698
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57584, 57679). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix the problem that kube-proxy can't create clusterIP ipvs rules when externalTrafficPolicy=Local
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57678
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57292, 56274, 57435, 57438, 57429). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
replace syscall with sys/unix pkg in ipvs/netlink call
**What this PR does / why we need it**:
This PR replaces syscall with sys/unix pkg in ipvs/netlink call as the Go doc for syscall says:
NOTE: This package is locked down.
Code outside the standard Go repository should be migrated to use the corresponding package in the golang.org/x/sys repository.
That is also where updates required by new systems or versions should be applied.
See https://golang.org/s/go1.4-syscall for more information.
**Which issue(s) this PR fixes**:
Fixes #57430
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @thockin @brendandburns
Automatic merge from submit-queue (batch tested with PRs 54379, 56593, 56685, 54174, 57309). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix ipvs proxier nodeport
**What this PR does / why we need it**:
Fix ipvs proxier nodeport.
**Which issue(s) this PR fixes**:
Fixes#55923
**Special notes for your reviewer**:
We bump the netwlink version in the 1st commit because:
* We call `netlink.RouteListFiltered()` to filter LOCAL type addresses from kernel route table. `netlink.RouteListFiltered()` exists in newer-version netlink package
* newer-version netlink package migrate `syscall` to `golang.org/x/sys/unix`, k8s cross-build can benefit from it as well. The Go doc for syscall says:
> NOTE: This package is locked down. Code outside the standard Go repository should be migrated to use the corresponding package in the golang.org/x/sys repository. That is also where updates required by new systems or versions should be applied. See https://golang.org/s/go1.4-syscall for more information.
**Release note**:
```release-note
Fix ipvs proxier nodeport eth* assumption
```
Automatic merge from submit-queue (batch tested with PRs 57127, 57011, 56754, 56601, 56483). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Enhance proxy mode validation
**What this PR does / why we need it**:
Proxy modes are list multiple times in validateProxyModeWindows and validateProxyModeLinux. This PR enhances the validation logic and only list them once.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Refer https://github.com/kubernetes/kubernetes/pull/56529/files#r156448205.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @thockin
Automatic merge from submit-queue (batch tested with PRs 56894, 56940, 55223, 56554, 54985). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove dead code in winkernel proxy UT
**What this PR does / why we need it**:
There is a `+build windows` tag in `pkg/proxy/winkernel/proxier_test.go`, it means the UTs will not run in upstream CI. Plus, the UT codes in `pkg/proxy/winkernel/proxier_test.go` are not for winkernel proxier, instead they are for iptables proxier!
We should remove the dead code in winkernel proxy UT and then create the winkernel-specific UTs.
**Which issue(s) this PR fixes**:
Fixes#56555
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Test Proxy Mode
**What this PR does / why we need it**:
Currently, IPVS-based kube-proxy is protected by feature gateway and there is no UT for it. We should test ipvs proxy mode when feature gateway is set.
**Which issue(s) this PR fixes**:
Fixes#56404
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57211, 56150, 56368, 56271, 55957). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Validate IPVSConfiguration only when IPVS mode is enabled.
**What this PR does / why we need it**:
It's strange for users that they should configuration valid values for ipvs params even when they don't use ipvs mode. Users don't need to care about these params in this case.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
/cc @m1093782566
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55954, 56037, 55866, 55984, 54994). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove dead code
**What this PR does / why we need it**:
remove dead code in IPVS kube-proxy
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @thockin
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update IPVS README
**What this PR does / why we need it**:
People complained that they feel confused about how to use IPVS-based kube-proxy correctly, probably we may need to provide them a better README.
**Which issue(s) this PR fixes**:
xref: #56319
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixed typo in IPVS documentation
**What this PR does / why we need it**:
Fixed documentation typo
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixed documentation typo in IPVS README.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Using ipset doing SNAT and packet filter in IPVS kube-proxy
**What this PR does / why we need it**:
Try ipset in ipvs proxy mode.
**Which issue this PR fixes**:
fixes#54203
xref: #53393, #53775
**Special notes for your reviewer**:
**Release note**:
```release-note
Using ipset doing SNAT and packet filtering in IPVS kube-proxy
```
/sig network
/area kube-proxy
Automatic merge from submit-queue (batch tested with PRs 55988, 53555, 55858). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add IPv6 and negative UT test cases for proxier's deleteEndpointConnections
This change adds IPv6 and negative UT test cases for the proxier's deleteEndpointConnections.
Changes include:
- Add IPv6 UT test cases to TestDeleteEndpointConnections.
- Add negative UT test case to TestDeleteEndpointConnections for
handling case where no connections need clearing (benign error).
- Add negative UT test case to test unexpected error.
- Reorganize UT in TestDeleteEndpointConnections so that the fake
command executor's command and scripted responses are generated on
the fly based on the test case table (rather than using a fixed
set of commands/responses that will need to be updated every time
test cases are added/deleted).
- Create the proxier service map in real time, based on the test case
table (rather than using a fixed service map that will need to be updated
every time test cases are added/deleted).
fixes#53554
**What this PR does / why we need it**:
This change adds IPv6 and negative UT test cases for the proxier's
deleteEndpointConnections.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#53554
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55757, 55741, 55817, 55834, 51530). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixes Issue 55816: Removes unneeded IPPart error
**What this PR does / why we need it**:
Previously proxy was incorrectly logging an error message for the IPPart function. The PR fixes this logging behavior to only log `invalid IP part` for invalid IP:Port combinations.
**Which issue(s) this PR fixes**:
Fixes # https://github.com/kubernetes/kubernetes/issues/55816
**Special notes for your reviewer**:
None
**Release note**:
```
none
```
Automatic merge from submit-queue (batch tested with PRs 55757, 55741, 55817, 55834, 51530). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove AddToGroupVersion in kubeproxy scheme
ref https://github.com/kubernetes/kubernetes/pull/53645#discussion_r148850279
Signed-off-by: yanxuean <yan.xuean@zte.com.cn>
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @ncdc
Automatic merge from submit-queue (batch tested with PRs 53780, 55663, 55321, 52421, 55659). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Validate kube-proxy options
**What this PR does / why we need it**:
Validate ipvs proxy options
**Which issue this PR fixes** : fixes#53852
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55009, 55532, 55601, 52569, 55533). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Kube-proxy adds forward rules to ensure NodePorts work
**What this PR does / why we need it**:
Updates kube-proxy to set up proper forwarding so that NodePorts work with docker 1.13 without depending on iptables FORWARD being changed manually/externally.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#39823
**Special notes for your reviewer**:
@thockin I used option number 2 that I mentioned in the #39823 issue, please let me know what you think about this change. If you are happy with the change then I can try to add tests but may need a little direction about what and where to add them.
**Release note**:
```release-note
Add iptables rules to allow Pod traffic even when default iptables policy is to reject.
```
Automatic merge from submit-queue (batch tested with PRs 55009, 55532, 55601, 52569, 55533). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix ipvs/proxy getLocalIPs inconsistency with iptables/proxy
**What this PR does / why we need it**:
* Fix ipvs/proxy `getLocalIPs()` inconsistency with iptables/proxy
* validate the ip address before pkg/proxy/util IPPart() return ip string.
**Which issue(s) this PR fixes** :
Fixes#55612
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 53580, 55568). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Replace sets.List() with sets.UnsortedList() in pkg/proxy
**What this PR does / why we need it**:
Replace sets.List() with sets.UnsortedList() in pkg/proxy - sets.List() will sort the result array, we don't need sorted array in pkg/proxy. Using sets.UnsortedList() can reduce the unnecessary overhead spending.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
@wojtek-t wdyt ^_^
**Release note**:
```release-note
NONE
```
/sig network
Automatic merge from submit-queue (batch tested with PRs 55247, 55324, 55261, 55147, 54052). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Implement dummy device operation by netlink for ipvs proxier
**What this PR does / why we need it**:
Currently, we implement network interface operations, e.g. create a dummy device and delete a dummy interface via ip cmd and then collect the exist code. For example, we assume the following command:
```
ip link add kube-ipvs0 type dummy
```
returns 2 means the dummy device `kube-ipvs0` exists, see https://github.com/kubernetes/kubernetes/blob/master/pkg/proxy/ipvs/proxier.go#L1529 .
However, from the man page of ip command, exit status 2 just means an error was reported by the kernel, see http://man7.org/linux/man-pages/man8/ip.8.html#EXIT_STATUS.
So, that's a bug.
This PR implements dummy device operation by netlink for ipvs proxier so that can get ride of ip command operations.
**Which issue this PR fixes**:
fixes#54054
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig network
/area kube-proxy
/kind bug
Automatic merge from submit-queue (batch tested with PRs 55247, 55324, 55261, 55147, 54052). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Restore kube-proxy's support for 0 values for conntrack settings
**What this PR does / why we need it**: re-allow 0 values for kube-proxy conntrack min, max, max per core, tcp close wait timeout, tcp established timeout.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#50787
**Special notes for your reviewer**:
- I adjusted validation to allow for 0 values for some of the conntrack settings, as otherwise the "leave the limit as-is" logic wouldn't be allowed.
- I moved the loading of the config file from the cobra command's Validate method to Complete. This way, the config is fully resolved before validation happens. Otherwise, it just validates the default config values first, and _then_ the config is loaded.
- I think I got all the default values & nil checking correct, but please review carefully!
**Release note**:
```release-note
Restored kube-proxy's support for 0 values for conntrack min, max, max per core, tcp close wait timeout, and tcp established timeout.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
update ipvs readme file
Add the way to choose ipvs mode when using kubeadm with a configuration file
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix lint errors in kubeproxyconfig types.
**What this PR does / why we need it**:
Fix some outstanding lint errors reported by k8s-ci-robot in https://github.com/kubernetes/kubernetes/pull/53645/files#r149212051.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#53645
**Special notes for your reviewer**:
/cc @cblecker @ncdc
**Release note**:
```release-note
NONE
```
When kube-proxy was refactored to use a configuration file, the ability
to use 0 for conntrack min, max, max per core, and tcp timeouts was
inadvertently broken; if you specified 0, it would instead apply the
default value from defaults.go.
This change restores the ability to use 0 to mean 0.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add windows kernel mode kubeproxy reviewers
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixing 'targetport' to service 'port' mapping
Looks like the order should be reversed for this to work.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#54968
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kube-proxy IPVS: Fix IPVS availability check
**What this PR does / why we need it**:
The current implementation of `CanUseIPVSProxier()` reads `/proc/modules`
to check whether IPVS related kernel modules can be loaded.
You might get a false-negative when the kernel modules are installed to
the system but haven't been loaded into the kernel.
This patch firstly try to run `modprobe` to load specified kernel
modules, then just log warnings if error occured.
Secondly, it will check loaded kernel modules by reading
`/proc/modules`, return an error if any required module is missing.
This change will not break the compatability of existing implementation.
Running kube-proxy in a container without mounting `/lib/modules` will
cause `modprobe` warnings, but not raise an error if all required modules are
present in the host kernel.
**Special notes for your reviewer**:
**Release note**:
```release-note
Fix IPVS availability check
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
clean up legacy ipv4/32 in ipvs proxy
**What this PR does / why we need it**:
clean up legacy ipv4/32 in ipvs proxy
**Which issue(s) this PR fixes**: closes#51866
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig network
/area ipv6
/assign @thockin
cc @danehans
The current implementation of `CanUseIPVSProxier()` reads `/proc/modules`
to check whether IPVS related kernel modules can be loaded.
You might get a false-negative when the kernel modules are installed to
the system but haven't been loaded into the kernel.
This patch firstly try to run `modprobe` to load specified kernel
modules, then just log warnings if error occured.
Secondly, it will check loaded kernel modules by reading
`/proc/modules`, return an error if any required module is missing.
This change will not break the compatability of existing implementation.
Running kube-proxy in a container without mounting `/lib/modules` will
cause `modprobe` warnings, but not raise an error if all required modules are
present in the host kernel.
Automatic merge from submit-queue (batch tested with PRs 54774, 54820, 52192, 54827). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix IPV6 judgement bug and add UTs
**What this PR does / why we need it**:
Fix IPV6 judgement bug and add UTs.
**Which issue this PR fixes** : closes#54775
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig network
/area ipv6
/kind bug
Automatic merge from submit-queue (batch tested with PRs 53796, 54666, 54516, 54625, 54704). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove +linux restriction in ipvs/fake for running UTs in !linux platform
**What this PR does / why we need it**:
Remove +linux restriction in ipvs/fake for running UTs in !linux platform
**Which issue this PR fixes**:
fixes#54667
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig network
/kind bug
Automatic merge from submit-queue (batch tested with PRs 54635, 54250, 54657, 54696, 54700). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove iptables log on restore failure
Don't log the set of rules at v2 in kube-proxy on error.
The rules are displayed at v5 before the restore is attempted.
In a large cluster the report can generate up to 100000 lines.
A partial report is only helpful if the problem is displayed
in the partial report.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Don't log the set of rules at v2 in kube-proxy on error.
The rules are displayed at v5 before the restore is attempted.
In a large cluster the report can generate up to 100000 lines.
A partial report is only helpful if the problem is displayed
in the partial report.
Automatic merge from submit-queue (batch tested with PRs 54656, 54552, 54389, 53634, 54408). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove hard code of session affinity timeout in winows kernel proxy
**What this PR does / why we need it**:
Remove hard code of session affinity timeout in winows kernel proxy - we have already done this in userspace, iptables and ipvs proxy.
**Which issue this PR fixes**:
fixes#53636
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig network
/area kube-proxy
Automatic merge from submit-queue (batch tested with PRs 52479, 53956). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Register sync proxy rules latency metrics in app level
**What this PR does / why we need it**:
IMO, should may should register proxy metrics in app level instead of in specific proxy mode, e.g. iptables, ipvs, winkernel...
By registering sync proxy rules latency metrics in app level, we can reuse codes among different proxiers.
**Which issue this PR fixes**:
closes#53957
**Special notes for your reviewer**:
@wojtek-t What do you think about it?
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add m1093782566 to pkg/proxy/OWNERS reviewers
**What this PR does / why we need it**:
Add @m1093782566 to pkg/proxy/OWNERS reviewers. I would love to help review kube-proxy, especially ipvs mode codes.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 52471, 53708). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
clean up winkernel proxy
**What this PR does / why we need it**:
clean up winkernel proxy
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix typos: remove duplicated word in comments
**What this PR does / why we need it**: Remove the duplicated word `the` in comments
**Which issue this PR fixes** : fixes #
**Special notes for your reviewer**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix duplicate unbind action in kube-proxy
**What this PR does / why we need it**:
Fix duplicate unbind action in kube-proxy. It will generate unnecessary error info If unbind multi-ports on one service .
**Which issue this PR fixes**:
fixes#51694
**Release-note**:
```release-note
NONE
```
This change adds IPv6 and negative UT test cases for the proxier's
deleteEndpointConnections.
Changes include:
- Add IPv6 UT test cases to TestDeleteEndpointConnections.
- Add negative UT test case to TestDeleteEndpointConnections for
handling case where no connections need clearing (benign error).
- Add negative UT test case to test unexpected error.
- Reorganize UT in TestDeleteEndpointConnections so that the fake
command executor's command and scripted responses are generated on
the fly based on the test case table (rather than using a fixed
set of commands/responses that will need to be updated every time
test cases are added/deleted).
- Create the proxier service map in real time, based on the test case
table (rather than using a fixed service map that will need to be updated
every time test cases are added/deleted).
fixes#53554
Automatic merge from submit-queue (batch tested with PRs 52768, 51898, 53510, 53097, 53058). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Run IPVS proxier UTs everywhere - include !linux platfrom
**What this PR does / why we need it**:
IPVS proxier UTs should run everywhere, including !linux platfrom, which will help a lot when developing in windows platfrom.
**Which issue this PR fixes**:
fixes#53099
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 53454, 53446, 52935, 53443, 52917). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove ipv4 constraint of Node IPs in ipvs proxier
**What this PR does / why we need it**:
We are targeting to IPV6. So, we should remove ipv4 constraint of Node IPs in ipvs proxier.
Besides, adding some log messages.
**Which issue this PR fixes**:
xref #51866
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add check for IPv6 '::' when calculating kube-proxy's node IP
kube-proxy currently checks for a bind address of 0.0.0.0 (IPv4 all-zeros)
when calculating kube-proxy's node IP, but it does not check for
an address of '::' (IPv6 all-zeros). For either of those all-zeros
addresses, the node IP should be determined based on the hostname,
rather than using the address directly.
Also added a helpful log message when the kube-proxy protocol is
determined to be IPv6.
fixes#52613
**What this PR does / why we need it**:
This PR adds a check for a bind address of IPv6 '::' (all zeros) when kube-proxy is calculating its node IP,
similar to what is done for a bind address of IPv4 0.0.0.0. For either of these all-zeros addresses,
kube-proxy should derive the node IP based on the hostname, rather than use the bind address
directly.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#52613
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add required family flag for conntrack IPv6 operation
This change causes kube-proxy to supply the required "-f ipv6"
family flag whenever the conntrack utility is executed and the
associated service is using IPv6.
This change is required for IPv6-only operation.
Note that unit test coverage for the 2-line changes in
pkg/proxy/iptables/proxier.go and /pkg/proxy/ipvs/proxier.go will need
to be added after support for IPv6 service addresses is added to these
files. For pkg/proxy/iptables/proxier.go, this coverage will be added
either with PR #48551.
fixes#52027
**What this PR does / why we need it**:
Kube-proxy is currently not supplying the required "-f ipv6" family flag whenever it
calls the conntrack utility and the associated service is using an IPv6 service IP address.
This means that for IPv6-only operation, conntrack is not properly cleaning up
stale UDP connections, and this may be effecting ip6tables operation.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes # 52027
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
kube-proxy currently checks for a bind address of 0.0.0.0 (IPv4 all-zeros)
when calculating kube-proxy's node IP, but it does not check for
an address of '::' (IPv6 all-zeros). For either of those all-zeros
addresses, the node IP should be determined based on the hostname,
rather than using the address directly.
Also added a helpful log message when the kube-proxy protocol is
determined to be IPv6.
fixes#52613
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
bazel: build/test almost everything
**What this PR does / why we need it**: Miscellaneous cleanups and bug fixes. The main motivating idea here was to make `bazel build //...` and `bazel test //...` mostly work. (There's a few reasons these still don't work, but we're a lot closer.)
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @BenTheElder @mikedanese @spxtr
Automatic merge from submit-queue (batch tested with PRs 50068, 52406, 52394, 48551, 52131). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Add IPv6 support to iptables proxier
Add IPv6 support to iptables proxier
The following changes are proposed for the iptables proxier:
- There are three places where a string specifying IP:port is parsed
using something like this:
if index := strings.Index(e.endpoint, ":"); index != -1 {
This will fail for IPv6 since V6 addresses contain colons. Also,
the V6 address is expected to be surrounded by square brackets
(i.e. [<ipv6-addr>]:<port>). Fix this by replacing call to Index with
call to LastIndex() and stripping out square brackets.
- The String() method for the localPort struct should put square brackets
around IPv6 addresses.
- The logging in the merge() method for proxyServiceMap should put brackets
around IPv6 addresses.
- There are several places where filterRules destination is hardcoded to
<clusterIP>/32. This should be a /128 for IPv6 case.
- Add IPv6 unit test cases
Note: I've left out most of the UT test cases that I had included in my original version of this
PR because the number of lines of code change were much too large for a single review.
I'm including a minimum of UT with this current version of the PR.
fixes#48550
**What this PR does / why we need it**:
This PR addresses several issues in the iptables proxier for handling IPv6 addresses
that were found via visual code inspection, including:
- There are three places where a string specifying IP:port using something like the following:
if index := strings.Index(e.endpoint, ":"); index != -1 {
This will fail for IPv6 since V6 addresses contains many colons, and the V6 address is expected
to be enclosed in square brackets when followed by :<port>.
- The String() method for the localPort struct should put square brackets around IPv6 addresses.
- The logging in the merge() method for proxyServiceMap should put brackets around IPv6
addresses.
- There are several places where filterRules destination is hardcoded to /32.
Should be a /128 for IPv6 case.
- More IPv6 unit test cases are needed.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#48550
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 50068, 52406, 52394, 48551, 52131). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
add FlagPersistent flag in nodePort and other situation
**What this PR does / why we need it**:
For AffinityClientIP service, origin code in ipvs didn't add Persistent flag, which is a bug, so I added it.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#52626
**Special notes for your reviewer**:
add FlagPersistent flag in nodePort and other situation
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 50294, 50422, 51757, 52379, 52014). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
rsync ipvs proxier to HEAD of iptables
**What this PR does / why we need it**:
rsync ipvs proxier to HEAD of iptables.
**Which issue this PR fixes**:
xref #51679
**Special notes for your reviewer**:
Obviously, @Lion-Wei has done part of this work, ref: #51922. It's fine that let #51922 get in first.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
add readme file of ipvs
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
None
```
(*) Fix cleanup of NodePort resources. (*) Fix the logic to select existing policies
Fix review comment
Fix Bazel
Update GoDep License
Fix NodePort forwarding to target port
Fix Darwin Build break. +1
Implement IsCompatible to validate kernel support for kernel mode
The following changes are proposed for the iptables proxier:
* There are three places where a string specifying IP:port is parsed
using something like this:
if index := strings.Index(e.endpoint, ":"); index != -1 {
This will fail for IPv6 since V6 addresses contain colons. Also,
the V6 address is expected to be surrounded by square brackets
(i.e. []:). Fix this by replacing call to Index with
call to LastIndex() and stripping out square brackets.
* The String() method for the localPort struct should put square brackets
around IPv6 addresses.
* The logging in the merge() method for proxyServiceMap should put brackets
around IPv6 addresses.
* There are several places where filterRules destination is hardcoded to
/32. This should be a /128 for IPv6 case.
* Add IPv6 unit test cases
fixes#48550
Windows Kernel now exposes "Internal Load Balancing"
using VFP (Virtual Filtering Platform) part of Virtual Switch. An inbuild
windows service HNS (Host Networking Service) acts as interface to program
the VFP. VFP is synonymous to iptables in functionality. HNS uses json based
data as input.
With the help of the interface available in github.com/Microsoft/hcsshim,
these APIs are exposed to the world in github to program HNS and use
the feature.
*** More info about the changes in this PR ***
(1) For every endpoint available in the system, an HNS Endpoint is added
(1.a) for local endpoints, a local HNS Endpoint would already exist, as part of
container creation.
(1.b) For all remote endpoints, a remote HNS Endpoint is created via HNS
(2) For every Service, a HNS ILB LoadBalancer is added referring the endpoints
created in (1)
Sample Input to HNS:
{
"Policies": [
{
"ExternalPort": 80,
"InternalPort": 80,
"Protocol": 6,
"Type": "ELB",
"VIPs": [
"11.0.98.129"
]
}
],
"References": [
"/endpoints/ca8b877b-ab90-499a-bc0e-7d736c425632",
"/endpoints/ee0ef08b-8434-4f8b-b748-393884e77465"
]
}
(2-a) This is done for Cluster IP, LoadBalancer Ingress IP, NodePort, External IP
Following the regular service and endpoint updates,
the HNS is notified of the updates and the system is kept in sync.
This change causes kube-proxy to supply the required "-f ipv6"
family flag whenever the conntrack utility is executed and the
associated service is using IPv6.
This change is required for IPv6-only operation.
Note that unit test coverage for the 2-line changes in
pkg/proxy/iptables/proxier.go and /pkg/proxy/ipvs/proxier.go will need
to be added after support for IPv6 service addresses is added to these
files. For pkg/proxy/iptables/proxier.go, this coverage will be added
either with PR #48551.
fixes#52027
Automatic merge from submit-queue
rsync IPVS proxier to the HEAD of iptables
**What this PR does / why we need it**:
There was a significant performance improvement made to iptables. Since IPVS proxier makes use of iptables in some use cases, I think we should rsync IPVS proxier to the HEAD of iptables.
**Which issue this PR fixes** :
xref #51679
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51819, 51706, 51761, 51818, 51500)
fix kube-proxy panic because of nil sessionAffinityConfig
**What this PR does / why we need it**:
fix kube-proxy panic because of nil sessionAffinityConfig
**Which issue this PR fixes**: closes#51499
**Special notes for your reviewer**:
I apology that this bug is introduced by #49850 :(
@thockin @smarterclayton @gnufied
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49850, 47782, 50595, 50730, 51341)
Paramaterize `stickyMaxAgeMinutes` for service in API
**What this PR does / why we need it**:
Currently I find `stickyMaxAgeMinutes` for a session affinity type service is hard code to 180min. There is a TODO comment, see
https://github.com/kubernetes/kubernetes/blob/master/pkg/proxy/iptables/proxier.go#L205
I think the seesion sticky max time varies from service to service and users may not aware of it since it's hard coded in all proxier.go - iptables, userspace and winuserspace.
Once we parameterize it in API, users can set/get the values for their different services.
Perhaps, we can introduce a new field `api.ClientIPAffinityConfig` in `api.ServiceSpec`.
There is an initial discussion about it in sig-network group. See,
https://groups.google.com/forum/#!topic/kubernetes-sig-network/i-LkeHrjs80
**Which issue this PR fixes**:
fixes#49831
**Special notes for your reviewer**:
**Release note**:
```release-note
Paramaterize session affinity timeout seconds in service API for Client IP based session affinity.
```
Automatic merge from submit-queue (batch tested with PRs 50094, 48966, 49478, 50593, 49140)
[kube-proxy] Move UDP conntrack operations together to pkg/proxy/util/conntrack.go
**What this PR does / why we need it**:
Fix TODO in pkg/proxy/iptables.go, see
https://github.com/kubernetes/kubernetes/blob/master/pkg/proxy/iptables/proxier.go#L1632
Move UDP conntrack operations together to from `pkg/proxy/iptables/proxier.go` to `pkg/proxy/util/conntrack.go` so that make them more consistent and add some UTs.
**Which issue this PR fixes**
Fixes#49477
**Special notes for your reviewer**:
```release-note
NONE
```
Automatic merge from submit-queue
Fix winspace proxier wrong comment message
**What this PR does / why we need it**:
Since winspace proxier has nothing to do with iptables, this PR remove the wrong comment message on iptables.
**Which issue this PR fixes**:
Fixes#50524
Automatic merge from submit-queue
Switch from package syscall to golang.org/x/sys/unix
**What this PR does / why we need it**:
The syscall package is locked down and the comment in https://github.com/golang/go/blob/master/src/syscall/syscall.go#L21-L24 advises to switch code to use the corresponding package from golang.org/x/sys. This PR does so and replaces usage of package syscall with package golang.org/x/sys/unix where applicable. This will also allow to get updates and fixes
without having to use a new go version.
In order to get the latest functionality, golang.org/x/sys/ is re-vendored. This also allows to use Eventfd() from this package instead of calling the eventfd() C function.
**Special notes for your reviewer**:
This follows previous works in other Go projects, see e.g. moby/moby#33399, cilium/cilium#588
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Log abridged set of rules at v2 in kube-proxy on error
**What this PR does / why we need it**:
this is a follow-on to https://github.com/kubernetes/kubernetes/pull/48085
**Special notes for your reviewer**:
we hit this in operations where we typically run in v2, and would like to log abridged set of output rather than full output.
**Release note**:
```release-note
NONE
```
The syscall package is locked down and the comment in [1] advises to
switch code to use the corresponding package from golang.org/x/sys. Do
so and replace usage of package syscall with package
golang.org/x/sys/unix where applicable.
[1] https://github.com/golang/go/blob/master/src/syscall/syscall.go#L21-L24
This will also allow to get updates and fixes for syscall wrappers
without having to use a new go version.
Errno, Signal and SysProcAttr aren't changed as they haven't been
implemented in /x/sys/. Stat_t from syscall is used if standard library
packages (e.g. os) require it. syscall.SIGTERM is used for
cross-platform files.
Automatic merge from submit-queue (batch tested with PRs 47232, 48625, 48613, 48567, 39173)
proxy/userspace: honor listen IP address as host IP if given
Allows the proxier to be used on an interface that's not the default route,
otherwise hostIP gets set to the default route interface even if that's
not what the user intended.
If listen IP isn't given, falls back to previous behavior.
```release-note
To allow the userspace proxy to work correctly on multi-interface hosts when using the non-default-route interface, you may now set the `bindAddress` configuration option to an IP address assigned to a network interface. The proxy will use that IP address for any required NAT operations instead of the IP address of the interface which has the default route.
```
@kubernetes/sig-network-misc @thockin @wojtek-t
Allows the proxier to be used on an interface that's not the default route,
otherwise hostIP gets set to the default route interface even if that's
not what the user intended.
If listen IP isn't given, falls back to previous behavior.
Automatic merge from submit-queue
Raise a warning instead of info if br-netfilter is missing or unset
Took quite a while to figure out why service VIP is unreachable on my cluster. It turns out br-nf-call-iptables is unset. I wish this message could be a warning to attract considerable attention.
Automatic merge from submit-queue
Move the remaining controllers to shared informers
Completes work done in 1.6 to move the last two hold outs to shared informers - tokens controller and scheduler. Adds a few more tools to allow informer reuse (like filtering the informer, or maintaining a mutation cache).
The mutation cache is identical to #45838 and will be removed when that merges
@ncdc @deads2k extracted from openshift/origin#14086
Automatic merge from submit-queue
Fix the issue in Windows kube-proxy when processing unqualified name. This is for DNS client such as ping or iwr that validate name in response and original question.
**What this PR does / why we need it**:
This PR is an additional fix to #41618 and [the corresponding commit](b9dfb69dd7). The DNS client such as nslookup does not validate name matching in response and original question. That works fine when we append DNS suffix to unqualified name in DNS query in Windows kube-proxy. However, for DNS client such as ping or Invoke-WebRequest that validates name in response and original question, the issue arises and the DNS query fails although the received DNS response has no error.
This PR fixes the additional issue by restoring the original question name in DNS response. Further, this PR refactors DNS message routines by using miekg's DNS library.
This PR affects the Windows kube-proxy only.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#42605
**Special notes for your reviewer**:
**Release note**:
```release-note
Fix DNS suffix search list support in Windows kube-proxy.
```
Automatic merge from submit-queue (batch tested with PRs 45953, 45889)
Add /metrics and profiling handlers to kube-proxy
Also expose "syncProxyRules latency" as a prometheus metrics.
Fix https://github.com/kubernetes/kubernetes/issues/45876
Automatic merge from submit-queue (batch tested with PRs 45623, 45241, 45460, 41162)
Promotes Source IP preservation for Virtual IPs from Beta to GA
Fixes#33625. Feature issue: kubernetes/features#27.
Bullet points:
- Declare 2 fields (ExternalTraffic and HealthCheckNodePort) that mirror the ESIPP annotations.
- ESIPP alpha annotations will be ignored.
- Existing ESIPP beta annotations will still be fully supported.
- Allow promoting beta annotations to first class fields or reversely.
- Disallow setting invalid ExternalTraffic and HealthCheckNodePort on services. Default ExternalTraffic field for nodePort or loadBalancer type service to "Global" if not set.
**Release note**:
```release-note
Promotes Source IP preservation for Virtual IPs to GA.
Two api fields are defined correspondingly:
- Service.Spec.ExternalTrafficPolicy <- 'service.beta.kubernetes.io/external-traffic' annotation.
- Service.Spec.HealthCheckNodePort <- 'service.beta.kubernetes.io/healthcheck-nodeport' annotation.
```
Automatic merge from submit-queue (batch tested with PRs 45685, 45572, 45624, 45723, 45733)
Remove reasons from iptables syncProxyRules
The reasons are no longer useful, since we know if something changed anyway, I think.
Automatic merge from submit-queue (batch tested with PRs 45571, 45657, 45638, 45663, 45622)
Use real proxier inside hollow-proxy but with mocked syscalls
Fixes https://github.com/kubernetes/kubernetes/issues/43701
This should make hollow-proxy better mimic the real kube-proxy in performance.
Maybe next we should have a more realistic implementation even for fake iptables (adding/updating/deleting rules/chains in an table, just not on the real one)? Though I'm not sure how important it is.
cc @kubernetes/sig-scalability-misc @kubernetes/sig-network-misc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 44727, 45409, 44968, 45122, 45493)
Separate healthz server from metrics server in kube-proxy
From #14661, proposal is on kubernetes/community#552.
Couple bullet points as in commit:
- /healthz will be served on 0.0.0.0:10256 by default.
- /metrics and /proxyMode will be served on port 10249 as before.
- Healthz handler will verify timestamp in iptables mode.
/assign @nicksardo @bowei @thockin
**Release note**:
```release-note
NONE
```
- /healthz will be served on 0.0.0.0:10256 by default.
- /metrics and /proxyMode will be served on port 10249
as before.
- Healthz handler will verify timestamp in iptables mode.
Automatic merge from submit-queue (batch tested with PRs 44590, 44969, 45325, 45208, 44714)
Fix onlylocal endpoint's healthcheck nodeport logic
I was in the middle of rebasing #41162, surprisingly found the healthcheck nodeport logic in kube-proxy is still buggy. Separate this fix out as it isn't GA related.
/assign @freehan @thockin
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Edge-based userspace LB in kube-proxy
@thockin @bowei - if one of you could take a look if that PR doesn't break some basic kube-proxy assumptions. The similar change for winuserproxy should be pretty trivial.
And we should also do that for iptables, but that requires splitting the iptables code to syncProxyRules (which from what I know @thockin already started working on so we should probably wait for it to be done).
Automatic merge from submit-queue
Restore "Setting endpoints" log message
**What this PR does / why we need it**:
The "Setting endpoints" message from kube-proxy at high verbosity was
lost as part of a larger simplification in kubernetes/kubernetes#42747.
This change brings it back, simply outputting the just-constructed
addresses list.
I need this message to monitor delays in propagating endpoints changes across nodes.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 43871, 44053)
Proxy healthchecks overhaul
The first commit is #44051
These three commits are tightly coupled, but should be reviewed one-by-one. The first adds tests for healthchecks, and found a bug. The second basically rewrites the healthcheck pkg to be much simpler and less flexible (since we weren't using the flexibility). The third tweaks how healthchecks are handled in endpoints-path to be more like they are in services-path.
@MrHohn because I know you were in here for source-IP GA work.
@wojtek-t
The "Setting endpoints" message from kube-proxy at high verbosity was
lost as part of a larger simplification in kubernetes/kubernetes#42747.
This change brings it back, simply outputting the just-constructed
addresses list.
Automatic merge from submit-queue
kube-proxy: filter INPUT as well as OUTPUT
We need to apply filter rules on the way in (nodeports) and out (cluster
IPs). Testing here is insufficient to have caught this - will come back
for that.
Fixes#43969
@justinsb since you have the best repro, can you test? It passes what I think is repro.
@ethernetdan we will want this in 1.6.x
```release-note
Fix bug with service nodeports that have no backends not being rejected, when they should be. This is not a regression vs v1.5 - it's a fix that didn't quite fix hard enough.
```
The existing healthcheck lib was pretty complicated and was hiding some
bugs (like the count always being 1), This is a reboot of the interface
and implementation to be significantly simpler and better tested.
Automatic merge from submit-queue
Use shared informers for proxy endpoints and service configs
Use shared informers instead of creating local controllers/reflectors
for the proxy's endpoints and service configs. This allows downstream
integrators to pass in preexisting shared informers to save on memory &
cpu usage.
This also enables the cache mutation detector for kube-proxy for those
presubmit jobs that already turn it on.
Follow-up to #43295 cc @wojtek-t
Will race with #43937 for conflicting changes 😄 cc @thockin
cc @smarterclayton @sttts @liggitt @deads2k @derekwaynecarr @eparis @kubernetes/rh-cluster-infra
Adding test cases for HC updates found a bug with an update that
simultaneously removes one port and adds another. Map iteration is
randomized, so sometimes no HC would be created.
Use shared informers instead of creating local controllers/reflectors
for the proxy's endpoints and service configs. This allows downstream
integrators to pass in preexisting shared informers to save on memory &
cpu usage.
This also enables the cache mutation detector for kube-proxy for those
presubmit jobs that already turn it on.
We need to apply filter rules on the way in (nodeports) and out (cluster
IPs). Testing here is insufficient to have caught this - will come back
for that.
We don't need the svcPortToInfoMap. Its only purpose was to
send "valid" local endpoints (those with valid IP and >0 port) to the
health checker. But we shouldn't be sending invalid endpoints to
the health checker anyway, because it can't do anything with them.
If we exclude invalid endpoints earlier, then we don't need
flattenValidEndpoints().
And if we don't need flattenValidEndpoints() it makes no sense to have
svcPortToInfoMap store hostPortInfo, since endpointsInfo is the same
thing as hostPortInfo except with a combined host:port.
And if svcPortToInfoMap now only stores valid endpointsInfos, it is
exactly the same thing as newEndpoints.
Automatic merge from submit-queue (batch tested with PRs 41672, 42084, 42233, 42165, 42273)
Don't sync IPtables before underlying store/reflector is fully synced
Ref #42108
Build on top of #42108 - only the second commit is unique.
Automatic merge from submit-queue
Extensible Userspace Proxy
This PR refactors the userspace proxy to allow for custom proxy socket implementations.
It changes the the ProxySocket interface to ensure that other packages can properly implement it (making sure all arguments are publicly exposed types, etc), and adds in a mechanism for an implementation to create an instance of the userspace proxy with a non-standard ProxySocket.
Custom ProxySockets are useful to inject additional logic into the actual proxying. For example, our idling proxier uses a custom proxy socket to hold connections and notify the cluster that idled scalable resources need to be woken up.
Also-Authored-By: Ben Bennett bbennett@redhat.com
Automatic merge from submit-queue (batch tested with PRs 42316, 41618, 42201, 42113, 42191)
Support unqualified and partially qualified domain name in DNS query in Windows kube-proxy
**What this PR does / why we need it**:
In Windows container networking, --dns-search is not currently supported on Windows Docker. Besides, even with --dns-suffix, inside Windows container DNS suffix is not appended to DNS query names. That makes unqualified domain name or partially qualified domain name in DNS query not able to resolve.
This PR provides a solution to resolve unqualified domain name or partially qualified domain name in DNS query for Windows container in Windows kube-proxy. It uses well-known Kubernetes DNS suffix as well host DNS suffix search list to append to the name in DNS query. DNS packet in kube-proxy UDP stream is modified as appropriate.
This PR affects the Windows kube-proxy only.
**Special notes for your reviewer**:
This PR is based on top of Anthony Howe's commit 48647fb, 0e37f0a and 7e2c71f which is already included in the PR 41487. Please only review commit b9dfb69.
**Release note**:
```release-note
Add DNS suffix search list support in Windows kube-proxy.
```
Automatic merge from submit-queue (batch tested with PRs 42200, 39535, 41708, 41487, 41335)
Update kube-proxy support for Windows
**What this PR does / why we need it**:
The kube-proxy is built upon the sophisticated iptables NAT rules. Windows does not have an equivalent capability. This introduces a change to the architecture of the user space mode of the Windows version of kube-proxy to match the capabilities of Windows.
The proxy is organized around service ports and portals. For each service a service port is created and then a portal, or iptables NAT rule, is opened for each service ip, external ip, node port, and ingress ip. This PR merges the service port and portal into a single concept of a "ServicePortPortal" where there is one connection opened for each of service IP, external ip, node port, and ingress IP.
This PR only affects the Windows kube-proxy. It is important for the Windows kube-proxy because it removes the limited portproxy rule and RRAS service and enables full tcp/udp capability to services.
**Special notes for your reviewer**:
**Release note**:
```
Add tcp/udp userspace proxy support for Windows.
```
This changes the userspace proxy so that it cleans up its conntrack
settings when a service is removed (as the iptables proxy already
does). This could theoretically cause problems when a UDP service
as deleted and recreated quickly (with the same IP address). As
long as packets from the same UDP source IP and port were going to
the same destination IP and port, the the conntrack would apply and
the packets would be sent to the old destination.
This is astronomically unlikely if you did not specify the IP address
to use in the service, and even then, only happens with an "established"
UDP connection. However, in cases where a service could be "switched"
between using the iptables proxy and the userspace proxy, this case
becomes much more frequent.
This commit makes the userspace proxy keep an ObjectReference to the
service being proxied. This allows the consumers of the `ServiceInfo`
struct, like `ProxySockets` to emit events about or otherwise refer to
the service.
This commit adds a new method for constructing userspace proxiers,
`NewCustomProxier`. `NewCustomProxier` functions identically to
`NewProxier`, except that it allows a custom constructor method to
be passed in to construct instances of ProxySocket.
This commit makes it possible for the `ProxySocket` interface to be
implemented by types outside of the `userspace` package. It mainly just
exposes relevant types and fields as public.
This commit adds a method to the `LoadBalancer` interface in the
userspace proxy which allows consumers of the `LoadBalancer` to check if
it thinks a given service has endpoints available.
This makes it more obvious that they run together and makes the upcoming
rate-limited syncs easier.
Also make test use ints for ports, so it's easier to see when a port is
a literal value vs a name.
Jeff Grafton