Automatic merge from submit-queue (batch tested with PRs 63424, 63657). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix#63183 Pods on different nodes mount Ceph RBD PVC stuck on ContainerCreating
**What this PR does / why we need it**:
Ceph/RBD pvc can be attached on different nodes when use ReadOnlyMany.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#63183
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 59034, 63565, 63533). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[RBD block device ]ignore the loopbackdevice error, or the rbd volume will not get detached
The rbd volume loop device maybe not found and then the volume will not get detached.
so need to ignore the error.
@mtanino @sbezverk
Can you take a look?
Release note:
```
ignore the loopback device not found error when `GetLoopDevice`
```
Automatic merge from submit-queue (batch tested with PRs 63298, 63339). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: volume: do not create event on mount success
xref https://github.com/openshift/origin/issues/19497
There is an argument to be made that we should not emit events on volume mount successes. Many pods have lots of volumes that generate lots of events, especially if the pods are short lived (batch, etc). This creates load on the API server and add noise in the events.
Creating events on volume mount failures is obviously a good idea. Seems like the absence of such events or the "Started container" event implies mount success for all volumes.
/sig node
/sig storage
/release-note-none
@smarterclayton @jsafrane @saad-ali
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
aws, azure: replace path with filepath
**What this PR does / why we need it**:
This PR replaces usage of `path` with `filepath` as it uses OS-specific path separators.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62916
**Special notes for your reviewer**:
This PR addresses the following volume plugins:
- `pkg/volume/aws_ebs`
- `pkg/volume/azure_dd`
- `pkg/volume/azure_file`
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61725, 63261). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use cloudprovider.NotImplemented in AddSSHKeyToAllInstances
**What this PR does / why we need it**:
Minor cleanup: In looking at implementations of this method,
noticed a few providers created their own errors instead of using cloudprovider.NotImplemented
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62657, 63278, 62903, 63375). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add more volume types in e2e and fix part of them.
**What this PR does / why we need it**:
- Add dir-link/dir-bindmounted/dir-link-bindmounted/bockfs volume types for e2e tests.
- Fix fsGroup related e2e tests partially.
- Return error if we cannot resolve volume path.
- Because we should not fallback to volume path, if it's a symbolic link, we may get wrong results.
To safely set fsGroup on local volume, we need to implement these two methods correctly for all volume types both on the host and in container:
- get volume path kubelet can access
- paths on the host and in container are different
- get mount references
- for directories, we cannot use its mount source (device field) to identify mount references, because directories on same filesystem have same mount source (e.g. tmpfs), we need to check filesystem's major:minor and directory root path on it
Here is current status:
| | (A) volume-path (host) | (B) volume-path (container) | (C) mount-refs (host) | (D) mount-refs (container) |
| --- | --- | --- | --- | --- |
| (1) dir | OK | FAIL | FAIL | FAIL |
| (2) dir-link | OK | FAIL | FAIL | FAIL |
| (3) dir-bindmounted | OK | FAIL | FAIL | FAIL |
| (4) dir-link-bindmounted | OK | FAIL | FAIL | FAIL |
| (5) tmpfs| OK | FAIL | FAIL | FAIL |
| (6) blockfs| OK | FAIL | OK | FAIL |
| (7) block| NOTNEEDED | NOTNEEDED | NOTNEEDED | NOTNEEDED |
| (8) gce-localssd-scsi-fs| NOTTESTED | NOTTESTED | NOTTESTED | NOTTESTED |
- This PR uses `nsenter ... readlink` to resolve path in container as @msau42 @jsafrane [suggested](https://github.com/kubernetes/kubernetes/pull/61489#pullrequestreview-110032850). This fixes B1:B6 and D6, , the rest will be addressed in https://github.com/kubernetes/kubernetes/pull/62102.
- C5:D5 marked `FAIL` because `tmpfs` filesystems can share same mount source, we cannot rely on it to check mount references. e2e tests passes due to we use unique mount source string in tests.
- A7:D7 marked `NOTNEEDED` because we don't set fsGroup on block devices in local plugin. (TODO: Should we set fsGroup on block device?)
- A8:D8 marked `NOTTESTED` because I didn't test it, I leave it to `pull-kubernetes-e2e-gce`. I think it should be same as `blockfs`.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update all script shebangs to use /usr/bin/env interpreter instead of /bin/interpreter
This is required to support systems where bash doesn't reside in /bin (such as NixOS, or the *BSD family) and allow users to specify a different interpreter version through $PATH manipulation.
https://www.cyberciti.biz/tips/finding-bash-perl-python-portably-using-env.html
```release-note
Use /usr/bin/env in all script shebangs to increase portability.
```
Minor cleanup: In looking at implementations of this method,
noticed a few providers created their own errors instead of using cloudprovider.NotImplemented
Automatic merge from submit-queue (batch tested with PRs 63251, 59166, 63250, 63180, 63169). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
clean one redundant comment of rbd.go
**What this PR does / why we need it**:
This is a small PR for cleanup.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix cephfs fuse mount bug when user is not admin
**What this PR does / why we need it**:
fix cephfs fuse mount bug when user is not admin
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubernetes/issues/59393
**Special notes for your reviewer**:
**Release note**:
```release-note
fix cephfs fuse mount bug when user is not admin
```
Automatic merge from submit-queue (batch tested with PRs 63129, 63066, 60009, 63136, 63086). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add node shutdown taint
**What this PR does / why we need it**: we need node stopped taint in order to detach volumes immediately without waiting timeout. More info in issue ticket #58635
**Which issue(s) this PR fixes**
Fixes#58635
**Special notes for your reviewer**: this was reverted, original PR https://github.com/kubernetes/kubernetes/pull/59323 Hopefully now bugs are fixed. However, I will execute more tests manually today.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62590, 62818, 63015, 62922, 63000). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix some bugs inside CSI volume plugin unit test TestAttacherMountDevice
**What this PR does / why we need it**:
Fix two bugs inside CSI volume plugin unit test `TestAttacherMountDevice`.
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 62780, 62886). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Only count local mounts that are from other pods
**What this PR does / why we need it**:
In GCE, we mount the same local SSD in two different paths (for backwards compatability). This makes the fsGroup conflict check fail because it thinks the 2nd mount is from another pod. For the fsgroup check, we only want to detect if other pods are mounting the same volume, so this PR filters the mount list to only those mounts under "/var/lib/kubelet".
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62867
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62876, 62733, 62827). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix csi data race in csi_attacher_test.go
**What this PR does / why we need it**:
fix csi data race in csi_attacher_test.go#TestAttacherWaitForVolumeAttachment
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62630
**Special notes for your reviewer**:
run `stress -p 500 ./csi.test -v 5 -alsologtostderr` , There is another failure
I think we should fix it in another PR.
```
--- FAIL: TestAttacherMountDevice (0.07s)
csi_attacher_test.go:495: Running test case: normal
csi_attacher_test.go:534: test should not fail, but error occurred: mkdir path2: file exists
```
**Release note**:
```release-note
NONE
```
/sig storage
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add metrics to cinder volume
**What this PR does / why we need it**: We need cinder volume metrics to be exposed
**Which issue(s) this PR fixes**:
Fixes#62644
**Special notes for your reviewer**: This is not tested in running cluster.
**Release note**:
```release-note
Make volume usage metrics available for Cinder
```
cc @gnufied @dims
Automatic merge from submit-queue (batch tested with PRs 62803, 62787). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add `volumenameprefix` tests for glusterfs dynamic provisioner.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 62694, 62569, 62646, 61633, 62433). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Report events to apiserver in local volume plugin.
**What this PR does / why we need it**:
See https://github.com/kubernetes/kubernetes/pull/62102#discussion_r179238429.
Report events to apiserver in local volume plugin.
- Add VolumeHost.GetEventRecorder() method
- Add related e2e tests
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62248
**Special notes for your reviewer**:
Example output of `kubectl describe pods`:
```
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 7s default-scheduler Successfully assigned e2e-tests-persistent-local-volumes-test-x4h5x/security-context-670da435-4174-11e8-9098-000c29bb0377 to 127.0.0.1
Warning AlreadyMountedVolume 7s kubelet, 127.0.0.1 The requested fsGroup is 4321, but the volume local-pvfbb76 has GID 1234. The volume may not be shareable.
Normal SuccessfulMountVolume 7s kubelet, 127.0.0.1 MountVolume.SetUp succeeded for volume "default-token-996xr"
Normal SuccessfulMountVolume 7s kubelet, 127.0.0.1 MountVolume.SetUp succeeded for volume "local-pvfbb76"
Normal Pulled 6s kubelet, 127.0.0.1 Container image "k8s.gcr.io/busybox:1.24" already present on machine
Normal Created 6s kubelet, 127.0.0.1 Created container
Normal Started 6s kubelet, 127.0.0.1 Started container
```
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62694, 62569, 62646, 61633, 62433). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use filepath.Clean() instead of path.Clean()
**What this PR does / why we need it**:
Use filepath.Clean() instead of path.Clean() across `pkg/volume`. This could fix potential issues for windows containers.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
followup of #62375.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @msau42
Automatic merge from submit-queue (batch tested with PRs 62378, 62734). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add `snapfactor` storageclass parameter for GlusterFS provisioner.
This patch add a new parameter called `snapfactor` to glusterfs
storageclass. This is an optional parameter and value should
fall into the range of 1-100. When set the thin pool calculation
respect this snapfactor and create a thinpool accordingly.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
storageclass. This is an optional parameter and value should
fall into the range of 1-100. When set the thin pool calculation
respect this snapfactor and create a thinpool accordingly.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Automatic merge from submit-queue (batch tested with PRs 62676, 62612). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix WaitForAttach failure issue for azure disk
**What this PR does / why we need it**:
From v1.10, `devicePath` will be updated due to following code change:
568afb4ecc/pkg/volume/util/operationexecutor/operation_generator.go (L517-L518)
So in v1.10.0, MountVolume.WaitForAttach will fail in the azure disk remount, error logs would be like following:
```
MountVolume.WaitForAttach failed for volume "pvc-f1562ecb-3e5f-11e8-ab6b-000d3af9f967" : azureDisk - Wait for attach expect device path as a lun number, instead got: /dev/disk/azure/scsi1/lun1 (strconv.Atoi: parsing "/dev/disk/azure/scsi1/lun1": invalid syntax)
Warning FailedMount 1m (x10 over 21m) kubelet, k8s-agentpool-66825246-0 Unable to mount volumes for pod
```
This PR does not use `devicePath` anymore since it could be changed, instead, it use `diskController.GetDiskLun(diskName, volumeSource.DataDiskURI, nodeName)` to get disk LUN, this ARM api call would cost about 0.12s
The GCE disk won't have this issue since `devicePath` is not used in [WaitForAttach func](https://github.com/kubernetes/kubernetes/blob/master/pkg/volume/gce_pd/attacher.go#L133), while aws disk is also using `devicePath` in [WaitForAttach func](https://github.com/kubernetes/kubernetes/blob/master/pkg/volume/aws_ebs/attacher.go#L145), I think there is potentical issue for aws_ebs
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62540
**Special notes for your reviewer**:
should cherry-pick to v1.10
**Release note**:
```
fix WaitForAttach failure issue for azure disk
```
/assign @feiskyer
/sig azure
FYI @khenidak
Automatic merge from submit-queue (batch tested with PRs 62650, 62303, 62545, 62375). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix user visible files creation for windows
**What this PR does / why we need it**:
Fix user visible files creation for windows. Without this, [createUserVisibleFiles](https://github.com/kubernetes/kubernetes/blob/master/pkg/volume/util/atomic_writer.go#L415:24) will get linkname with subpath included, and then symlink will fail. This is because "/" is used in pod spec (e.g. `"new/path/data-1"`) while "\" is used on Windows to get linkname.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62338
**Special notes for your reviewer**:
Should also be cherry-picked to old releases.
**Release note**:
```release-note
Fix user visible files creation for windows
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix volume node affinity to OR node selector terms
**What this PR does / why we need it**:
Fixes node selector terms to be ORed, to be consistent with documentation and Pod.NodeAffinity. Also handles the "node selector term nil or empty matches nothing" behavior.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62551
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixes issue where PersistentVolume.NodeAffinity.NodeSelectorTerms were ANDed instead of ORed.
```
In WatchPod(), if one of the two channels being watched (pod updates and
events) is closed, the for/select loop turns into a tight infinite loop because
the select immediately falls through due to the channel being closed. Watch
them independently instead.
Automatic merge from submit-queue (batch tested with PRs 62486, 62471, 62183). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
CSI - Update to apply fsGroup volume ownership
**What this PR does / why we need it**:
This PR correctly fixes the CSI internal driver to apply fsGroup volume ownership value during mount.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62413
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60476, 62462, 61391, 62535, 62394). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Revert "git: Use VolumeHost.GetExec() to execute stuff in volume plugins"
This reverts commit c578542ad7 (PR #51098). The PR added support for containerized git, on the other hand it required git 1.8.5. This breaks git volumes on older distros (CentOS 7, Ubuntu 14.04) that have old git.
Git volumes are getting deprecated (https://github.com/kubernetes/kubernetes/issues/60999) so we should restore it to the last working state and not touch it any longer.
**Release note**:
```release-note
gitRepo volumes in pods no longer require git 1.8.5 or newer, older git versions are supported too now.
```
I'd like to cherry-pick it into 1.10.
/sig storage
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
The err has checked in TearDownAt func/kind bug
**What this PR does / why we need it**:
The err has checked in TearDownAt func/kind bug
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62324, 61459, 62475, 62476, 61914). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Return error in mount_unsupported for unsupported platforms
**What this PR does / why we need it**:
Presently, `util/mount_unsupported.go` does not return any errors. For unsupported platforms, this hides failures. This PR returns errors, thereby properly informing users attempting to run on an unsupported platform.
**Which issue(s) this PR fixes** :
Fixes https://github.com/kubernetes/kubernetes/issues/61919
**Special notes for your reviewer**:
There are a few methods that simply call through to other methods. I did not return errors from those methods.
I've also updated an error check and message in `volume/fc/fc_test.go`, since it was ignoring an error on unsupported platforms.
**Release note**:
```release-note
NONE
```
Kubernetes Submit Queue