Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update bazelbuild/rules_go, kubernetes/repo-infra, and gazelle dependencies
**What this PR does / why we need it**: updates our bazelbuild/rules_go dependency in order to bump everything to go1.9.4. I'm separating this effort into two separate PRs, since updating rules_go requires a large cleanup, removing an attribute from most build rules.
**Release note**:
```release-note
NONE
```
With d7ddcca231, we lost the logging
of the flags. We should at least log what the command line flags
were used to start processes as those incredibly useful for trouble shooting.
Automatic merge from submit-queue (batch tested with PRs 59103, 58433). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
bugfix(mount): lstat with abs path of parent instead of '/..'
**What this PR does / why we need it**:
If a nfs volume with improper permission is mounted on a Pod, operation of deleting this Pod will fail and the pod itself will be stuck at a 'TERMINATING' status. Kubelet cannot reconcile it correctly.
This is because kubelet will try to find the mount-point with '..' file which needs `x` permission of dir. When it's forbidden, the nfs volume will never umount without a correct mount-point finded.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57095
**Special notes for your reviewer**:
**Release note**:
```release-note
Get parent dir via canonical absolute path when trying to judge mount-point
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add reviewers to util/mount
**What this PR does / why we need it**:
Add some more reviewers
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix all the typos across the project
**What this PR does / why we need it**:
There are lots of typos across the project. We should avoid small PRs on fixing those annoying typos, which is time-consuming and low efficient.
This PR does fix all the typos across the project currently. And with #59463, typos could be avoided when a new PR gets merged.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
/sig testing
/area test-infra
/sig release
/cc @ixdy
/assign @fejta
**Release note**:
```release-note
None
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
return error if New-SmbGlobalMapping failed when mounting azure file on Windows
**What this PR does / why we need it**:
This PR will return error if New-SmbGlobalMapping failed when mounting azure file on Windows.
User wants to create a pod moutning with azure file pvc using static provisioning([doc](https://github.com/andyzhangx/Demo/tree/master/windows/azurefile#static-provisioning-for-azure-file-on-windows-server-version-1709support-from-v17x])) and user uses a wrong storage account name or key, current behavior is if mount azure file on Windows, it will create an empty directory, and user would not know actually it fails.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59538
**Special notes for your reviewer**:
**Release note**:
```
return error if New-SmbGlobalMapping failed when mounting azure file on Windows
```
Automatic merge from submit-queue (batch tested with PRs 59466, 58912, 59605, 59548). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
return a more human readable error message if mount an unformatted vo…
**What this PR does / why we need it**:
If an unformatted volume is requested as read only mode, according device mount operation will fail, and the message is verbose and obscure. We should check this scenario and return a more human readable message.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#58911
**Release note**:
```release-note
NONE
```
/sig storage
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Redesign and implement volume reconstruction work
This PR is the first part of redesign of volume reconstruction work. The detailed design information is https://github.com/kubernetes/community/pull/1601
The changes include
1. Remove dependency on volume spec stored in actual state for volume
cleanup process (UnmountVolume and UnmountDevice)
Modify AttachedVolume struct to add DeviceMountPath so that volume
unmount operation can use this information instead of constructing from
volume spec
2. Modify reconciler's volume reconstruction process (syncState). Currently workflow
is when kubelet restarts, syncState() is only called once before
reconciler starts its loop.
a. If volume plugin supports reconstruction, it will use the
reconstructed volume spec information to update actual state as before.
b. If volume plugin cannot support reconstruction, it will use the
scanned mount path information to clean up the mounts.
In this PR, all the plugins still support reconstruction (except
glusterfs), so reconstruction of some plugins will still have issues.
The next PR will modify those plugins that cannot support reconstruction
well.
This PR addresses issue #52683
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Cleanup of ipvs utils
**What this PR does / why we need it**:
This PR is a small cleanup of pkg/util/ipvs.
The changes are as follows:
1. Rename toBackendService -> toIPVSService and toBackendDestination -> toIPVSDestination. The use of the term 'backend' makes things really confusing and this renamig makes it more explicit about what we are doing.
2. Give the libnetwork/ipvs package the name libipvs. This makes it less confusing since the package these files are in is also ipvs.
3. Some variable naming cleanup to make things easier to read.
/assign @m1093782566
**Release note**:
```release-note
None
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
validate ipset set and entry - ipvs proxier validate entry before adding it and validate set before creating it
**What this PR does / why we need it**:
validate ipset util.
**Which issue(s) this PR fixes**:
Fixes#57269
**Special notes for your reviewer**:
I splitted this PR into some small pieces for the sake of code review.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51323, 59306, 58991, 59050). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Proposal: Use `blkid` to detect fs type of device instead of `lsblk`.
**What this PR does / why we need it**:
Use `blkid` to detect fs type of device instead of `lsblk`.
`lsblk` depends on `udev`, and device driver's udev rules. If udev rules are not installed properly, `lsblk` could not get fs type of disk. This will cause problems, e.g. expanding volume will fail because it could not detect fs type of disk.
Take `rbd` as example, `lsblk -n -o FSTYPE /dev/rbd0` command actually read fs type from `/run/udev/data/b252:0` (may vary on different machines, see real file from `strace lsblk -n -o FSTYPE /dev/rbd0` ).
This file `/run/udev/data/b252:0` is generated by `udev` and device driver's udev files. If machine don't have `/lib/udev/rules.d/60-ceph-by-parttypeuuid.rules` udev rule file installed (this file is from `ceph-osd` package on ubuntu), it will not be properly generated, e.g:
```
# cat /run/udev/data/b251:0
S:rbd/<pool>/<image>
I:13234059587579
E:ID_FS_TYPE=
E:net.ifnames=0
G:systemd
```
`lsblk -n -o FSTYPE /dev/rbd0` will get empty fs type.
[60-ceph-by-parttypeuuid.rules](https://github.com/ceph/ceph/blob/v13.0.0/udev/60-ceph-by-parttypeuuid.rules) is udev rule, which underlyingly runs `blkid` commands to get infos of device, then store them in udev file.
If we use `blkid` to get fs type, kubelet volume manager will not depend on `udev` stuffs. Currently, if kubelet node does not have `60-ceph-by-parttypeuuid.rules` installed (from `ceph-osd` package), it will fail to get fs type of rbd image. Even administrator install `ceph-osd` later, it can not get fs type of previous mapped images (udev data files not be to updated automatically).
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
`udevadm test` logs:
- Ubuntu 16.04 (ceph-common installed), without ceph-osd: [without-ceph-osd.log](https://github.com/kubernetes/kubernetes/files/1678512/without-ceph-osd.log)
- Ubuntu 16.04 (ceph-common installed), with ceph-osd: [with-ceph-osd.log](https://github.com/kubernetes/kubernetes/files/1678509/with-ceph-osd.log)
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58317, 58687, 57872, 59063, 59328). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix TC resource Leak
ref: https://github.com/kubernetes/kubernetes/issues/56840 fix
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
This PR is the first part of redesign of volume reconstruction work. The
changes include
1. Remove dependency on volume spec stored in actual state for volume
cleanup process (UnmountVolume and UnmountDevice)
Modify AttachedVolume struct to add DeviceMountPath so that volume
unmount operation can use this information instead of constructing from
volume spec
2. Modify reconciler's volume reconstruction process (syncState). Currently workflow
is when kubelet restarts, syncState() is only called once before
reconciler starts its loop.
a. If volume plugin supports reconstruction, it will use the
reconstructed volume spec information to update actual state as before.
b. If volume plugin cannot support reconstruction, it will use the
scanned mount path information to clean up the mounts.
In this PR, all the plugins still support reconstruction (except
glusterfs), so reconstruction of some plugins will still have issues.
The next PR will modify those plugins that cannot support reconstruction
well.
This PR addresses issue #52683, #54108 (This PR includes the changes to
update devicePath after local attach finishes)
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Perform resize of mounted volume if necessary
Under certain conditions - we must perform resize of volume even when it is mounted. This enables us to get around problem of resizing volumes used by deployments etc.
```release-note
Allow expanding mounted volumes
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add pkg/util/ipset OWNERS file
**What this PR does / why we need it**:
I initialized `pkg/util/ipset` for wraping ipset exec call and authored most of the commits.
This package is used in IPVS proxier for reducing iptables call - possibly we can use ipset in iptables proxier in the future. Because reviewing this package needs some ipset background knowledge, I am creating this OWNERS file for efficient code review. Also, I am willing to share code review burden.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @brendandburns @thockin because I also added your names.
`lsblk` reads fs type info from udev files. If udev rules are not
installed. `lsblk` could not get correct fs type. This will cause
problems, e.g. expanding volume depends on fs type of disk.
Automatic merge from submit-queue (batch tested with PRs 57868, 58284, 56370, 58400, 58439). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Show findmnt command output in case of error
**What this PR does / why we need it**:
We don't print the error output from the `findmnt` command, which makes it pretty hard to debug.
The current implementation just print:
`
I0115 11:59:36.078285 18230 nsenter_mount.go:168] Failed findmnt command for path /var/lib/origin/openshift.local.volumes/pods/8d723ec8-f9e9-11e7-bc52-025000000001/volumes/kubernetes.io~secret/pvinstaller-token-bxh42: exit status 1
`
With this patch, we can actually see the error:
`
I0115 12:04:30.908164 23301 nsenter_mount.go:168] Failed findmnt command for path /var/lib/origin/openshift.local.volumes/pods/8ff8bb98-f9e9-11e7-bc52-025000000001/volumes/kubernetes.io~secret/deployer-token-g4sg9: nsenter: failed to execute /findmnt: No such file or directory
exit status 1
`
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 53631, 56960). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove unused code in UT files in pkg/
**What this PR does / why we need it**:
Remove unused code in UT files in pkg/ .
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixed TearDown of NFS with root squash.
NFS plugin should not use `IsLikelyNotMountPoint()`, as it uses `lstat()` / `stat()` to determine if the NFS volume is still mounted - NFS server may use root_squash and kubelet may not be allowed to do `lstat()` / `stat()` there.
It must use slower `IsNotMountPoint()` instead, including in `TearDown()` function.
**Release note**:
```release-note
NONE
```
/assign @gnufied @rootfs
NFS plugin should not use IsLikelyNotMountPoint(), as it uses lstat() / stat()
to determine if the NFS volume is still mounted - NFS server may use
root_squash and kubelet may not be allowed to do lstat() / stat() there.
It must use slower IsNotMountPoint() instead, including TearDown() function.
Automatic merge from submit-queue (batch tested with PRs 57906, 57425, 56939, 57317, 57762). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add "Not Found" error for ipset set & entry delete
**What this PR does / why we need it**:
Add "not found error" for ipset set and entry delete. For example,
```
[root@SHA1000130405 k8s-yml]# ipset list
Name: KUBE-LOOP-BACK
Type: hash:ip,port,ip
Revision: 2
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 16776
References: 1
Members:
172.17.0.2,udp:53,172.17.0.2
172.17.0.2,tcp:53,172.17.0.2
100.106.89.164,tcp:6443,100.106.89.164
[root@SHA1000130405 k8s-yml]# ipset delete foo
ipset v6.19: The set with the given name does not exist
[root@SHA1000130405 k8s-yml]# ipset del KUBE-LOOP-BACK 1.2.3.4,tcp:80,1.2.3.4
ipset v6.19: Syntax error: Third element is missing from 1.2.3.4,tcp:80,1.2.3.4.
```
Just need to filter the error message "does not exist" or "element is missing". We should ignore these error when try to make sure they are deleted.
**Which issue(s) this PR fixes**:
Fixes#57318
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57651, 56411, 56779, 57523, 57624). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix fake IPVS Real Server delete
**What this PR does / why we need it**:
Delete `Weight` equal in identify ipvs Realservice equal, since `Address` + `port` can indetify an unique `Realservice`.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57674
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49856, 56257, 57027, 57695, 57432). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Change to pkg/util/node.UpdateNodeStatus
**What this PR does / why we need it**:
> // TODO: Change to pkg/util/node.UpdateNodeStatus.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
/cc @brendandburns @dchen1107 @lavalamp
**Release note**:
```release-note
None
```
Kubernetes Submit Queue