Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove duplicated check of device path in aws attacher
**What this PR does / why we need it**:
The `devicePath` parameter is already validated in this [code](b7100f1ee7/pkg/volume/aws_ebs/attacher.go (L158)), so no need to check it again in the `for loop` as it won't be modified.
This can make the code clearer.
**Release note**:
```release-note
NONE
```
/sig storage
/kind cleanup
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Make ExpandVolumeDevice() idempotent if existing volume capacity meets the requested size.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add binary configmap
Reviving code from https://github.com/kubernetes/kubernetes/pull/33549 submitted by @zreigz
**What this PR does / why we need it**:
Add support for binary files in ConfigMap
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#32432
**Special notes for your reviewer**:
**Release note**:
```release-note
ConfigMap objects now support binary data via a new `binaryData` field. When using `kubectl create configmap --from-file`, files containing non-UTF8 data will be placed in this new field in order to preserve the non-UTF8 data. Use of this feature requires 1.10+ apiserver and kubelets.
```
Automatic merge from submit-queue (batch tested with PRs 58626, 58791). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use correct pv annotation to fetch volume ID.
Fix , Issue #58627
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 58480, 58549). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use backup location to load cloud config for OpenStack
**What this PR does / why we need it**:
Since we are transitioning to external cloud provider, we need a way
to use the existing cinder volume plugin (from kubelet). With external
cloud manager kubelet will be run with --cloud=provider=external and
no --cloud-config file will be in the command line. So we need a way
to load the openstack config file from somewhere.
Taking a cue from kubeadm, which currently is picking up "/etc/kubernetes/cloud-config"
https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/phases/controlplane/manifests.go#L44
let's support the scenario where we fall back to this static location if
there is no cloud provider specified in the command line.
This has been tested with local-up-cluster using the following params:
EXTERNAL_CLOUD_PROVIDER=true
CLOUD_PROVIDER=openstack
CLOUD_CONFIG=/etc/kubernetes/cloud-config
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58438, 58523, 58513, 57948). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix csi ut print
**What this PR does / why we need it**:
fix csi ut print
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58438, 58523, 58513, 57948). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add Namespace to glusterfs custom volume names.
For better isolation of volumes, users/admins would like to include `namespace` in custom volume names. This patch adds that support.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Since we are transitioning to external cloud provider, we need a way
to use the existing cinder volume plugin (from kubelet). With external
cloud manager kubelet will be run with --cloud=provider=external and
no --cloud-config file will be in the command line. So we need a way
to load the openstack config file from somewhere.
Taking a cue from kubeadm, which currently is picking up "/etc/kubernetes/cloud-config"
https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/phases/controlplane/manifests.go#L44
let's support the scenario where we fall back to this static location if
there is no cloud provider specified in the command line.
This has been tested with local-up-cluster using the following params:
EXTERNAL_CLOUD_PROVIDER=true
CLOUD_PROVIDER=openstack
CLOUD_CONFIG=/etc/kubernetes/cloud-config
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
csi: Fix versioning error message
**What this PR does / why we need it**:
Incorrect error message
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#58092
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add FSType for CSI volume source
**What this PR does / why we need it**:
Add FSType for CSI volume source to specify filesystems (alpha defaults to `ext4`)
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#58183
**Special notes for your reviewer**:
**Release note**:
```release-note
Add FSType for CSI volume source to specify filesystems
```
/assign @saad-ali
cc @vladimirvivien
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Rework method of updating atomic-updated data volumes
**What this PR does / why we need it**:
This change affects the way that secret, configmap, downwardAPI and projected volumes (which all use the same underlying code) implement their data update functionality.
* Instead of creating a subdirectory hierarchy that will contain symlinks to each actual data file, create only symlinks to items in the root of the volume, whether they be files or directories.
* Rather than comparing the user-visible data directory to see if an update is needed, compare with the current version of the data directory.
* Fix data dir timestamp format year
* Create `..data` symlink even when a data volume has no data so consumers can have simplified update watch logic.
**Which issue(s) this PR fixes**:
Fixes#57421
**Release note**:
```release-note
Correct issues that arise when volumes are mounted beneath another secret, configmap, downwardAPI or projected volume
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixed TearDown of NFS with root squash.
NFS plugin should not use `IsLikelyNotMountPoint()`, as it uses `lstat()` / `stat()` to determine if the NFS volume is still mounted - NFS server may use root_squash and kubelet may not be allowed to do `lstat()` / `stat()` there.
It must use slower `IsNotMountPoint()` instead, including in `TearDown()` function.
**Release note**:
```release-note
NONE
```
/assign @gnufied @rootfs
This change affects the way that secret, configmap, downwardAPI and projected
volumes (which all use the same underlying code) implement their data update
functionality.
* Instead of creating a subdirectory hierarchy that itself
will contain symlinks to each actual data file, create only
symlinks to items in the root of the volume, whether they
be files or directories.
* Rather than comparing the user-visible data directory
to see if an update is needed, compare with the current
version of the data directory.
* Fix data dir timestamp format year
* Create ..data symlink even when a data volume has no data so
consumers can have simplified update watch logic.
This PR add comments for the background why plugin gets loopback
device and removes loopback device even if operation_generator has
same functionality.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
set fsGroup by securityContext.fsGroup in azure file
**What this PR does / why we need it**:
set fsGroup by securityContext.fsGroup in azure file
This PR respects `securityContext.fsGroup`, while if user both sets `gid=xxx` in mountOptions in azure storage class and `securityContext.fsGroup`, `gid=xxx` setting in mountOptions will win.
```
apiVersion: v1
kind: Pod
metadata:
name: security-context-demo
spec:
securityContext:
fsGroup: 2000
volumes:
- name: sec-ctx-vol
emptyDir: {}
containers:
- name: sec-ctx-demo
image: gcr.io/google-samples/node-hello:1.0
volumeMounts:
- name: sec-ctx-vol
mountPath: /data/demo
securityContext:
allowPrivilegeEscalation: false
```
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#44085
**Special notes for your reviewer**:
**Release note**:
```
set fsGroup by securityContext.fsGroup in azure file
```
/assign @karataliu
cc @rootfs @avaranovich
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[FC Plugin] Create proper volumeSpec during ConstructVolumeSpec
**What this PR does / why we need it**:
Currently, FC plugin returns volume name and empty FCVolumeSource during ConstrutVolumeSpec of filesystem volume's reconstruction.
In this fix, ConstructVolumeSpec retrieves global mount path, and analyzes volume parameters such as
WWN, LUN, WWID from the path.
**Which issue(s) this PR fixes** :
Fixes#58085
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
cc @rootfs @jsafrane
/sig storage
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add custom volume name based on SC parameter.
At present glusterfs dynamic PVs are created with
random names. However an admin would like to have some
handle on the volume names created dynamically for
various purposes. One example would be having a filter
for sorting out PVs created for a particular storage class.
This patch enables the functionality by having a custom
volume name as a prefix to dynamic PVs. This is an optional
parameter in SC and if set, the dynamic volumes are created
in below format where `_` is the field seperator/delimiter:
customvolumeprefix_PVCname_randomUUID
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
git: Use VolumeHost.GetExec() to execute stuff in volume plugins
This prepares volume plugins to run things in containers instead of running
them on the host.
**Special notes for your reviewer**:
* instead of `cmd.SetDir(<dir>); cmd.Exec("git <command>")`, we do `cmd.Exec("git -C <dir> <command>")` - mounter.Exec does not have SetDir()
* there are lot of changes in the tests because a different exec interface is used.
@kubernetes/sig-storage-pr-reviews
```release-note
gitRepo volumes in pods require git 1.8.5 or later
```
/assign @rootfs
Currently, FC plugin returns volume name and empty
FCVolumeSource during ConstrutVolumeSpec during
filesystem volume's reconstruction.
In this fix, ConstructVolumeSpec retrieves global
mount path, analyzes volume parameters such as
WWN, LUN, WWID from the path.
Fixes#58085
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix bug:Kubelet failure to umount mount points
What this PR does / why we need it:
Fix bug#41141:Kubelet failure to umount glusterfs mount points
kubelet failure to umount mount moints in these cases:
1. glusterfs volume : transport endpoint is not connected
2. nfs volume : stale NFS file handle
this PR will fix it
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
fixes#41141
Special notes for your reviewer:
NONE
Release note:
NONE
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix cinder detach problems
**What this PR does / why we need it**: We have currently huge problems in cinder volume detach. This PR tries to fix these issues.
**Which issue(s) this PR fixes**:
Fixes#50004Fixes#57497
**Special notes for your reviewer**:
**Release note**:
```release-note
openstack cinder detach problem is fixed if nova is shutdowned
```
At present glusterfs dynamic PVs are created with
random names. However an admin would like to have some
handle on the volume names created dynamically for
various purposes. One example would be having a filter
for sorting out PVs created for a particular storage class.
This patch enables the functionality by having a custom
volume name as a prefix to dynamic PVs. This is an optional
parameter in SC and if set, the dynamic volumes are created
in below format where `_` is the field seperator/delimiter:
customvolumeprefix_PVCname_randomUUID
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Automatic merge from submit-queue (batch tested with PRs 54230, 58100, 57861, 54752). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Block volumes Support: iSCSI plugin update
**What this PR does / why we need it**:
Add interface changes to iSCSI volume plugin to enable block volumes support feature.
**Which issue this PR fixes**:
Based on this proposal (kubernetes/community#805 & kubernetes/community#1265) and this feature issue: kubernetes/features#351
**Special notes for your reviewer**:
This PR temporarily includes following changes except iSCSI plugin change for reviewing purpose.
These changes will be removed from the PR once they are merged.
- (#50457) API Change
- (#51494) Container runtime interface change, volumemanager changes, operationexecutor changes
There are another PRs related to this functionality.
(#50457) API Change
(#53385) VolumeMode PV-PVC Binding change
(#51494) Container runtime interface change, volumemanager changes, operationexecutor changes
(#55112) Block volume: Command line printer update
Plugins
(#51493) Block volumes Support: FC plugin update
(#54752) Block volumes Support: iSCSI plugin update
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57823, 58091, 58093, 58096, 57020). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix rbd ConstructVolumeSpec bug
**What this PR does / why we need it**:
Fix rbd `ConstructVolumeSpec` bug as its global device mount path is not canonical. Full details are in #57744 .
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57744
**Release note**:
```release-note
NONE
```
/sig storage
/kind bug
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
check function return err
**What this PR does / why we need it**:
check function return err
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 57733, 57613, 57953). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix device name change issue for azure disk: add remount logic
**What this PR does / why we need it**:
fix device name change issue for azure disk: add remount logic
Accoding to [Troubleshoot Linux VM device name change](https://docs.microsoft.com/en-us/azure/virtual-machines/linux/troubleshoot-device-names-problems), there is possibility of device name change, so when kubelet is restarted, we need to check whether the following two paths are still valid:
1. `/var/lib/kubelet/plugins/kubernetes.io/azure-disk/mounts/m358246426`: in MountDevice func
2. `/var/lib/kubelet/pods/950f2eb8-d4e7-11e7-bc95-000d3a041274/volumes/kubernetes.io~azure-disk/pvc-67e4e319-d4e7-11e7-bc95-000d3a041274`: in SetUpAt func
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57952
**Special notes for your reviewer**:
this is a corresponding fix of https://github.com/kubernetes/kubernetes/pull/57549, #57549 uses '/dev/disk/by-id', and this PR would check whether the mountPath is valid when kubelet restart(e.g. after VM reboot since device name may change), if not valid, remount, remember '/dev/disk/by-id' will be always valid.
**Release note**:
```
fix device name change issue for azure disk: add remount logic
```
Automatic merge from submit-queue (batch tested with PRs 57993, 58028). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add volumemetrics for glusterfs plugin.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix binary check for cephfs.go
**What this PR does / why we need it**:
unify the check method. see ref issue： #56479
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56479
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57784, 56651). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Adding support for Block Volume to rbd plugin
Adding support for Block Volume to rbd plugin
```release-note
Adding support for Block Volume type to rbd plugin.
```
NFS plugin should not use IsLikelyNotMountPoint(), as it uses lstat() / stat()
to determine if the NFS volume is still mounted - NFS server may use
root_squash and kubelet may not be allowed to do lstat() / stat() there.
It must use slower IsNotMountPoint() instead, including TearDown() function.
Kubernetes Submit Queue