Commit Graph

1791 Commits (7e75a09db6dc13218b4f5e06493115ae96f83774)

Author SHA1 Message Date
Humble Chirammal e8f59b0797 Avoid explicit mention of glusterfs in error strings.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2018-02-22 11:39:19 +05:30
andyzhangx f3324a6c26 add lock before detaching azure disk
fix build error
2018-02-22 03:54:54 +00:00
Kubernetes Submit Queue d1cb55c8a7
Merge pull request #59928 from humblec/more-correction
Automatic merge from submit-queue (batch tested with PRs 59901, 59302, 59928). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Reformat and update error strings.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note

```
2018-02-21 07:02:38 -08:00
Kubernetes Submit Queue d94bc9ef85
Merge pull request #60036 from mtanino/issue/60025
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix device unmap for non-attachable plugin case

**What this PR does / why we need it**:

This PR fixes following problem.

- If plugin is non-attachable, global unmap path isn't stored in asw,
  then plugin fails to unmap volume. To store the path, this PR moves
  MarkDeviceAsMounted operation from the `if volumeAttacher != nil` block.

**Which issue(s) this PR fixes**: Fixes #60025

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

/sig storage
/cc @msau42 @jingxu97 
@dhirajh @ianchakeres @mlmhl
2018-02-20 16:30:41 -08:00
Kubernetes Submit Queue 6ba46963f8
Merge pull request #59391 from msau42/topology-beta
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Move volume scheduling and local storage to beta

**What this PR does / why we need it**:
* Move the feature gates and APIs for volume scheduling and local storage to beta
* Update tests to use the beta fields
@kubernetes/sig-storage-pr-reviews 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #59390

**Special notes for your reviewer**:

**Release note**:

```release-note
ACTION REQUIRED: VolumeScheduling and LocalPersistentVolume features are beta and enabled by default.  The PersistentVolume NodeAffinity alpha annotation is deprecated and will be removed in a future release.
```
2018-02-20 13:26:07 -08:00
Mitsuhiro Tanino 9ad102717f Fix device unmap for non-attachable plugin case
If plugin is non-attachable, global unmap path isn't stored in asw
then plugin fails to unmap volume. To store the path, this PR moves
MarkDeviceAsMounted operation from the `if volumeAttacher != nil` block.

Fixes #60025
2018-02-20 13:03:08 -05:00
Humble Chirammal 04a83da019 Reformat and update error strings.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2018-02-20 20:07:03 +05:30
Kubernetes Submit Queue aec91e112b
Merge pull request #60050 from obnoxxx/glusterfs-comment-typo
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

glusterfs: improve a few comments

**What this PR does / why we need it**:

This PR fixes a couple of comments in the glusterfs module:
* fixes a typo in a comment
* removes an outdated comment that is not correct (any more)
* updates one comment to refer to upstream gluster documentation instead of downstream red hat product documentation
2018-02-20 03:55:20 -08:00
Kubernetes Submit Queue 96ec318718
Merge pull request #59842 from ixdy/update-rules_go-02-2018
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

 Update bazelbuild/rules_go, kubernetes/repo-infra, and gazelle dependencies

**What this PR does / why we need it**: updates our bazelbuild/rules_go dependency in order to bump everything to go1.9.4. I'm separating this effort into two separate PRs, since updating rules_go requires a large cleanup, removing an attribute from most build rules.

**Release note**:

```release-note
NONE
```
2018-02-19 22:23:05 -08:00
Michael Adam 6bf35daca7 glusterfs: refer to upstream gluster documentation
Do not refer to downstream Red Hat documentation
in the upstream kubernetes code, if there is upstream
documentation to refer to.

Signed-off-by: Michael Adam <obnox@redhat.com>
2018-02-19 21:30:02 +01:00
Michael Adam 67adc29f8f glusterfs: fix a comment typo
Signed-off-by: Michael Adam <obnox@redhat.com>
2018-02-19 21:30:02 +01:00
Michael Adam 2ec681632b glusterfs: Remove an outdated comment about GB vs GiB
This was originally added due to a misunderstanding
of the documentation of Heketi (using a different
convention). Heketi's documentation has meanwhile
been clarified.

Signed-off-by: Michael Adam <obnox@redhat.com>
2018-02-19 21:30:02 +01:00
Serguei Bezverkhi 348a02395d Fixing CSI E2E test 2018-02-17 18:13:06 -05:00
Kubernetes Submit Queue 220bdf26b3
Merge pull request #59209 from sbezverk/csi_0.2.0_breaking_changes
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

CSI 0.2.0 breaking changes

Refactor kubernetes CSI bits to support CSI version 0.2.0

```release-note
Addressing breaking changes introduced by new 0.2.0 release of CSI spec
```
2018-02-16 21:27:58 -08:00
Kubernetes Submit Queue 6d0b71740f
Merge pull request #59968 from kubernetes/revert-59323-nodetaint
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Revert "add node shutdown taint"

Reverts kubernetes/kubernetes#59323

Node becomes unready, but is never removed. I've found the following in [kube-controller-manager.log](https://storage.googleapis.com/kubernetes-jenkins/logs/ci-kubernetes-e2e-gci-gce-autoscaling/6055/artifacts/bootstrap-e2e-master/cluster-autoscaler.log) from test run for one such node:

`E0216 01:14:27.084923       1 node_lifecycle_controller.go:686] Error determining if node bootstrap-e2e-minion-group-01b1 shutdown in cloud: failed to get instance ID from cloud provider: instance not found`

This goes on for the rest of the run (~6h). Looks like the node is stuck in Unready state because of this check: https://github.com/kubernetes/kubernetes/blob/master/pkg/controller/nodelifecycle/node_lifecycle_controller.go#L684. Previously, there was no such check and the node was removed.

Reverting as this would affect all users attempting to resize their node groups on GCE.

```release-note
NONE
```
2018-02-16 20:12:56 -08:00
Michelle Au 9aa82b6e7d Volume node affinity enforcement 2018-02-16 17:54:10 -08:00
Jeff Grafton ef56a8d6bb Autogenerated: hack/update-bazel.sh 2018-02-16 13:43:01 -08:00
Serguei Bezverkhi ea4df51b3b Refactor k8s core csi bits for CSI Spec 0.2.0 2018-02-16 13:29:34 -05:00
Kubernetes Submit Queue 11ecad2629
Merge pull request #59914 from iliastsi/feature-csi-stale-path
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

csi: Remove stale volume path

**What this PR does / why we need it**:

The CSI mounter creates the following paths during SetUp():

   * .../pods/\<podID\>/volumes/kubernetes.io~csi/\<specVolId\>/mount/
   * .../pods/\<podID\>/volumes/kubernetes.io~csi/\<specVolId\>/volume_data.json

During TearDown(), it does not remove the `.../kubernetes.io~csi/<specVolId>/`
directory, leaving behind orphan volumes: method cleanupOrphanedPodDirs()
complains with 'Orphaned pod found, but volume paths are still present
on disk'.

Fix that by removing the above directory in removeMountDir().

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-02-16 08:04:32 -08:00
Aleksandra Malinowska 2d54ba3e0f
Revert "add node shutdown taint" 2018-02-16 12:24:27 +01:00
Kubernetes Submit Queue 99c87cf679
Merge pull request #59923 from jsafrane/volumemanager-logs
Automatic merge from submit-queue (batch tested with PRs 59873, 59933, 59923, 59944, 59953). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Rework volume manager log levels

- all normal logs to go to level 4
- too frequent / duplicate logs go to level 5 (e.g. when something else logged similar message not too far away).

I checked that there is no excessive spam in the log - reconciler runs every 100ms, but it does not log anything if there is nothing to do.

**What this PR does / why we need it**:
This will help us debug flakes. E2e tests do not log levels 10-12 used in volume manager

**Release note**:

```release-note
NONE
```

/sig storage
/sig node
cc: @jingxu97 @sjenning
2018-02-15 20:16:38 -08:00
Kubernetes Submit Queue 27daaab224
Merge pull request #59323 from zetaab/nodetaint
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add node shutdown taint

**What this PR does / why we need it**: we need node stopped taint in order to detach volumes immediately without waiting timeout. More info in issue ticket #58635 

**Which issue(s) this PR fixes** 
Fixes #58635

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```
2018-02-15 09:52:10 -08:00
Jan Safranek e260096392 Rework volume manager log levels
- all normal logs to go to level 4
- too frequent / duplicate logs go to level 5 (e.g. when something else logged similar message not too far away).
2018-02-15 16:33:17 +01:00
Ilias Tsitsimpis 6590c3fe35 csi: Remove stale volume path
The CSI mounter creates the following paths during SetUp():

   * .../pods/<podID>/volumes/kubernetes.io~csi/<specVolId>/mount/
   * .../pods/<podID>/volumes/kubernetes.io~csi/<specVolId>/volume_data.json

During TearDown(), it does not remove the `.../kubernetes.io~csi/<specVolId>/`
directory, leaving behind orphan volumes: method cleanupOrphanedPodDirs()
complains with 'Orphaned pod found, but volume paths are still present
on disk'.

Fix that by removing the above directory in removeMountDir().

Signed-off-by: Ilias Tsitsimpis <iliastsi@arrikto.com>
2018-02-15 13:30:36 +02:00
Humble Chirammal 64e8111f75 Correct error strings and variable name.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2018-02-14 23:19:19 +05:30
Kubernetes Submit Queue d89e64110a
Merge pull request #59716 from feiskyer/vmss-disk
Automatic merge from submit-queue (batch tested with PRs 59489, 59716). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add AzureDisk support for vmss nodes

**What this PR does / why we need it**:

This PR adds AzureDisk support for vmss nodes. Changes include

- Upgrade vmss API to 2017-12-01
- Upgrade vmss clients with new version API
- Abstract AzureDisk operations for vmss and vmas
- Added AzureDisk support for vmss
- Unit tests and fake clients fix
 
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #43287

**Special notes for your reviewer**:

~~Depending on #59652 (the first two commits are from #59652).~~

**Release note**:

```release-note
Add AzureDisk support for vmss nodes
```
2018-02-14 00:14:34 -08:00
Kubernetes Submit Queue 4afdc33c57
Merge pull request #56454 from mtanino/volumehandler-refactor
Automatic merge from submit-queue (batch tested with PRs 59767, 56454, 59237, 59730, 55479). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Block Volume: Refactor volumehandler in operationexecutor

**What this PR does / why we need it**:

Based on discussion with @saad-ali at #51494, we need refactor volumehandler in operationexecutor
for Block Volume feature. We don't need to add volumehandler as separated object.

```
VolumeHandler does not need to be a separate object that is constructed inline like this. 
You can create a new operation, e.g. UnmountOperation to which you pass the spec,
and it can return either a UnmountVolume or UnmapVolume.
```

**Which issue(s) this PR fixes** : no related issue.

**Special notes for your reviewer**:

@saad-ali @msau42 

**Release note**:

```release-note
NONE
```
2018-02-12 15:44:33 -08:00
Kubernetes Submit Queue 66ccfcb4c7
Merge pull request #59640 from jianglingxia/jlx-2018020917
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

correct the ConstructVolumeSpec func path value

**What this PR does / why we need it**:
the path value is incorrect
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-02-12 07:15:13 -08:00
Pengfei Ni 6dcd56549a Use full instanceID as lun lock key
This is because the last part of VMSS instances are numbers, which may
be same if there are multiple VMSS within same cluster.
2018-02-12 14:07:05 +08:00
Wang Guoliang 31aad75316 more concise to merge the array 2018-02-11 21:27:11 +08:00
Kubernetes Submit Queue cfa6774540
Merge pull request #56557 from andyzhangx/azurefile-createaccount
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix the create azure file pvc failure if there is no storage account in current resource group

**What this PR does / why we need it**:
When create an azure file PVC, there will be error if there is no storage account in current resource group.
With this PR, a storage account will be created if there is no storage account in current resource group.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56556

**Special notes for your reviewer**:
1. rephrase the code logic of `CreateFileShare` func.
```
if accountName is empty, then 
    find a storage account that matches accountType
    if no storage account found, then
        create a new account
else
    we only use user specified storage account

create a file share according to found storage account
```
2. Use func `getStorageAccountName` to get a unique storage account name by UUID, a storage account for azure file would be like `f0b2b0bd40c010112e897fa`. And in next PR, I will use this function to create storage account for azure disk, the storage account for azure disk would be like `d8f3ad8ad92000f1e1e88bd`.

**Release note**:

```
fix the create azure file pvc failure if there is no storage account in current resource group
```
/sig azure
/assign @rootfs
2018-02-11 05:03:32 -08:00
andyzhangx aa21bef677 create storage account if necessary when create azure file pvc
use new storage account name generation method

use uuid to generate account name

change azure file account prefix

use uniqueID to generate a storage account name

fix comments

fix comments

fix comments

fix a storage account matching bug

only use UUID in getStorageAccountName func

use shorter storage account prefix for azure file

fix comments

fix comments

fix comments

fix rebase build error

rewrite CreateFileShare code logic

fix gofmt issue

fix test error

fix comments

fix a location matching bug
2018-02-11 06:29:05 +00:00
Kubernetes Submit Queue 317853c90c
Merge pull request #59464 from dixudx/fix_all_typos
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix all the typos across the project

**What this PR does / why we need it**:
There are lots of typos across the project. We should avoid small PRs on fixing those annoying typos, which is time-consuming and low efficient.

This PR does fix all the typos across the project currently. And with #59463, typos could be avoided when a new PR gets merged.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
/sig testing
/area test-infra
/sig release
/cc @ixdy 
/assign @fejta 

**Release note**:

```release-note
None
```
2018-02-10 22:12:45 -08:00
Davanum Srinivas 7ef4f207c2 New github id - FengyunPan -> FengyunPan2
PanFengyun <pan_feng_yun@163.com>'s previous github id was @FengyunPan

Due to some problem with github, he lost access to @FengyunPan and
is not using @FengyunPan2. So let's switch over to the new id. Github
has promised to release the previous id back in 6 months, so we may
have to switch it back later.
2018-02-10 22:28:51 -05:00
Di Xu 48388fec7e fix all the typos across the project 2018-02-11 11:04:14 +08:00
mtanino bc86537f18 Autogenerated files 2018-02-09 15:41:15 -05:00
mtanino 973583e781 Refactor volumehandler in operationexecutor 2018-02-09 15:39:55 -05:00
Kubernetes Submit Queue a35acccaf8
Merge pull request #59447 from verult/pd-alpha
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fixes the regression of GCEPD not provisioning correctly on GKE alpha clusters.

Fixes the regression by better distinguishing between single-zone and multi-zone PDs.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #59047

**Special notes for your reviewer**: All PD and dynamic provisioning e2e tests pass, manually verified provision, delete, attach, and detach of both single-zone and multi-zone PDs. Will create e2e tests for multizone PDs in a separate PR.

**Release note**:


/sig storage
/assign @saad-ali
2018-02-09 11:55:10 -08:00
Kubernetes Submit Queue 977d03bd8b
Merge pull request #59232 from liubin/fix4
Automatic merge from submit-queue (batch tested with PRs 59607, 59232). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Make log content more information

And change some `fmt.Errorf` to `fmt.Error`
2018-02-09 08:54:34 -08:00
Kubernetes Submit Queue afaabe05f7
Merge pull request #59607 from harsh-px/px-pvc
Automatic merge from submit-queue (batch tested with PRs 59607, 59232). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Pass pvc namespace and annotations to Portworx Create API

Signed-off-by: Harsh Desai <harsh@portworx.com>

**What this PR does / why we need it**:

Fixes #59606

**Release note**:

```release-note
NONE
```
2018-02-09 08:54:30 -08:00
Kubernetes Submit Queue aee2cff1b8
Merge pull request #59548 from linyouchong/lyc-201802082
Automatic merge from submit-queue (batch tested with PRs 59466, 58912, 59605, 59548). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix incorrect logic in canSupport

**What this PR does / why we need it**:
if `spec.PersistentVolume` is nil or `spec.Volume` is nil, func `canSupport` should return false.

**Release note**:
```
NONE
```
/release-note-none
/sig storage
2018-02-09 03:44:41 -08:00
Kubernetes Submit Queue 8268f6d5e7
Merge pull request #59466 from humblec/hide-configuration
Automatic merge from submit-queue (batch tested with PRs 59466, 58912, 59605, 59548). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Gluster: Remove provisioner configuration from info message.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note

```
2018-02-09 03:44:32 -08:00
jianglingxia 2d805c727e correct the ConstructVolumeSpec func path value 2018-02-09 17:29:24 +08:00
linyouchong b76a3c0fd6 fix incorrect logic in glusterfs.go#canSupport 2018-02-09 14:25:16 +08:00
linyouchong e92f6eb0fe fix incorrect logic in canSupport 2018-02-09 11:45:15 +08:00
Kubernetes Submit Queue d6625f857a
Merge pull request #58177 from jingxu97/Jan/reconstruct
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Redesign and implement volume reconstruction work

This PR is the first part of redesign of volume reconstruction work. The detailed design information is https://github.com/kubernetes/community/pull/1601

The changes include
1. Remove dependency on volume spec stored in actual state for volume
cleanup process (UnmountVolume and UnmountDevice)

Modify AttachedVolume struct to add DeviceMountPath so that volume
unmount operation can use this information instead of constructing from
volume spec

2. Modify reconciler's volume reconstruction process (syncState). Currently workflow
is when kubelet restarts, syncState() is only called once before
reconciler starts its loop.
a. If volume plugin supports reconstruction, it will use the
reconstructed volume spec information to update actual state as before.
b. If volume plugin cannot support reconstruction, it will use the
scanned mount path information to clean up the mounts.

In this PR, all the plugins still support reconstruction (except
glusterfs), so reconstruction of some plugins will still have issues.
The next PR will modify those plugins that cannot support reconstruction
well.

This PR addresses issue #52683
2018-02-08 18:21:34 -08:00
Harsh Desai 79ea511ec6 Pass pvc namespace and annotations to Portworx Create API
Closes #59606

Signed-off-by: Harsh Desai <harsh@portworx.com>
2018-02-08 17:46:08 -08:00
Cheng Xing 02352460f6 Fixes the regression of GCEPD not provisioning correctly on alpha clusters. 2018-02-08 10:46:06 -08:00
Kubernetes Submit Queue fb340a4695
Merge pull request #57824 from thockin/gcr-vanity
Automatic merge from submit-queue (batch tested with PRs 57824, 58806, 59410, 59280). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

2nd try at using a vanity GCR name

The 2nd commit here is the changes relative to the reverted PR.  Please focus review attention on that.

This is the 2nd attempt.  The previous try (#57573) was reverted while we
figured out the regional mirrors (oops).
    
New plan: k8s.gcr.io is a read-only facade that auto-detects your source
region (us, eu, or asia for now) and pulls from the closest.  To publish
an image, push k8s-staging.gcr.io and it will be synced to the regionals
automatically (similar to today).  For now the staging is an alias to
gcr.io/google_containers (the legacy URL).
    
When we move off of google-owned projects (working on it), then we just
do a one-time sync, and change the google-internal config, and nobody
outside should notice.
    
We can, in parallel, change the auto-sync into a manual sync - send a PR
to "promote" something from staging, and a bot activates it.  Nice and
visible, easy to keep track of.

xref https://github.com/kubernetes/release/issues/281

TL;DR:
  *  The new `staging-k8s.gcr.io` is where we push images.  It is literally an alias to `gcr.io/google_containers` (the existing repo) and is hosted in the US.
  * The contents of `staging-k8s.gcr.io` are automatically synced to `{asia,eu,us)-k8s.gcr.io`.
  * The new `k8s.gcr.io` will be a read-only alias to whichever regional repo is closest to you.
  * In the future, images will be promoted from `staging` to regional "prod" more explicitly and auditably.

 ```release-note
Use "k8s.gcr.io" for pulling container images rather than "gcr.io/google_containers".  Images are already synced, so this should not impact anyone materially.
    
Documentation and tools should all convert to the new name. Users should take note of this in case they see this new name in the system.
```
2018-02-08 03:29:32 -08:00
Jesse Haka 3cf5b172fa add node shutdown taint
shutdowned -> stopped

use shutdown everywhere

use patch in taints api call

use notimplemented in clouds use AddOrUpdateTaintOnNode

correct log text

add fake cloud

try to fix bazel

add shutdown tests

add context
2018-02-08 12:56:06 +02:00
Tim Hockin 3586986416 Switch to k8s.gcr.io vanity domain
This is the 2nd attempt.  The previous was reverted while we figured out
the regional mirrors (oops).

New plan: k8s.gcr.io is a read-only facade that auto-detects your source
region (us, eu, or asia for now) and pulls from the closest.  To publish
an image, push k8s-staging.gcr.io and it will be synced to the regionals
automatically (similar to today).  For now the staging is an alias to
gcr.io/google_containers (the legacy URL).

When we move off of google-owned projects (working on it), then we just
do a one-time sync, and change the google-internal config, and nobody
outside should notice.

We can, in parallel, change the auto-sync into a manual sync - send a PR
to "promote" something from staging, and a bot activates it.  Nice and
visible, easy to keep track of.
2018-02-07 21:14:19 -08:00
Kubernetes Submit Queue c37b5d757d
Merge pull request #59303 from dhirajh/localblock
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Block Volume Support: Local Volume Plugin update

**What this PR does / why we need it**:

Introduce block volume support to local volumes plugin.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #59500

**Special notes for your reviewer**:
@msau42 @mtanino @ianchakeres 

Adding support for block volumes as per https://github.com/kubernetes/features/issues/121

Other related PRs:
(#50457) API Change
(#53385) VolumeMode PV-PVC Binding change
(#51494) Container runtime interface change, volumemanager changes, operationexecutor changes

**Release note**:
```
Added support for Block Volume type to local-volume plugin.
```
2018-02-07 13:27:50 -08:00
Kubernetes Submit Queue 117780b908
Merge pull request #58287 from dg-i/ceph-provisioner-userid
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add keyring parameter in Ceph RBD provisioner

**What this PR does / why we need it**:
userSecretName is a mandatory parameter for the kubernetes.io/rbd storage provisioner. Ceph RBD persistent volumes however don't need the secretRef. If no secret is given, a local keyring file (/etc/ceph/keyring) on the node will be used. This behaviour should also work with the storage provisioner. If no userSecretName parameter is given, no secretRef should be created in the resulting persistent volume.

**Special notes for your reviewer**:

**Release note**:
```release-note
Add "keyring" parameter for Ceph RBD provisioner
```
2018-02-07 08:00:28 -08:00
Kubernetes Submit Queue a1437feb18
Merge pull request #59400 from jianglingxia/jlx-2018020614
Automatic merge from submit-queue (batch tested with PRs 57230, 59400). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

make sure mounter not nil and fix some typo

**What this PR does / why we need it**:
make sure mounter not nil  then and fix some typo
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-02-07 02:58:31 -08:00
Mathias Merscher dbf5f58022
add keyring parameter in Ceph RBD provisioner 2018-02-07 11:50:49 +01:00
Humble Chirammal 922766a7cf Remove provisioner configuration from info message.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2018-02-07 13:30:42 +05:30
Kubernetes Submit Queue 11104d75f1
Merge pull request #57194 from linyouchong/linyouchong-20171214
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix TODO:change to a api-server watch

What this PR does / why we need it:
fix TODO (vladimirvivien) instead of polling api-server, change to a api-server watch

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #58031

Special notes for your reviewer:

Release note:
NONE

@vladimirvivien,@jsafrane,@saad-ali
I saw some TODO there and I am very interested in fixing it.
Please let me know if it is not inappropriate to do this.
2018-02-06 23:35:13 -08:00
Kubernetes Submit Queue e5b6026db6
Merge pull request #59287 from cheftako/cloud-context-level
Automatic merge from submit-queue (batch tested with PRs 59441, 58264, 59287, 59396, 59439). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add context to all relevant cloud APIs

**What this PR does / why we need it**:

This adds context to all the relevant cloud provider interface signatures.
Callers of those APIs are currently satisfied using context.TODO().
There will be follow on PRs to push the context through the stack.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #815

**Special notes for your reviewer**:
For an idea of the full scope of this change please look at PR #58532.

**Release note**:
```release-note
Implementers of the cloud provider interface will note the addition of a context to this interface. Trivial code modification will be necessary for a cloud provider to continue to compile.
```
2018-02-06 20:27:39 -08:00
Kubernetes Submit Queue e740fe68c5
Merge pull request #58264 from WanLinghao/log_fix
Automatic merge from submit-queue (batch tested with PRs 59441, 58264, 59287, 59396, 59439). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

	fix some log param error

this patch fix some log parameter mistakes.



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note 
/release-note-none
```
2018-02-06 20:27:36 -08:00
bin liu 053e7c9cfd Change log format: replace () to [] 2018-02-07 11:05:39 +08:00
Kubernetes Submit Queue 3d4eaf7307
Merge pull request #57092 from linyouchong/linyouchong-20171212
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix TODO: moving Driver name check in API validation

**What this PR does / why we need it**:
fix TODO in csi_plugin.go ,moving Driver name check in API validation
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #58163

**Special notes for your reviewer**:

**Release note**:
NONE

@vladimirvivien,@jsafrane,@saad-ali
I saw some TODO there and I am very interested in fixing it.
Please let me know if it is not  inappropriate to do this.
2018-02-06 18:49:36 -08:00
Dhiraj f92e13f01e Removed unnecessary test code. 2018-02-06 18:38:52 -08:00
linyouchong efb68ddf73 fix TODO: moving driver name check in API validation 2018-02-07 09:03:47 +08:00
Dhiraj 8b8b85604a Initial local PV block device plugin checkin.
Unit tests for block.

Add docs/test

Fix gofmt issues.

Address code review comments.

Remove evalsymlink fro setupDevice()

Address review comments.
2018-02-06 14:49:44 -08:00
Walter Fender e18e8ec3c0 Add context to all relevant cloud APIs
This adds context to all the relevant cloud provider interface signatures.
Callers of those APIs are currently satisfied using context.TODO().
There will be follow on PRs to push the context through the stack.
For an idea of the full scope of this change please look at PR #58532.
2018-02-06 12:49:17 -08:00
Kubernetes Submit Queue 4bd22b5467
Merge pull request #58415 from gnufied/fix-volume-resize-messages
Automatic merge from submit-queue (batch tested with PRs 52942, 58415). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Improve messaging on volume expansion

- we now provide clear message to user what to do when cloudprovider resizing is finished
  and file system resizing is needed.
- add a event when resizing is successful
- Use PATCH both in controller-manager and kubelet for updating PVC status
- Remove code duplication between controller-manager and kubelet for updating PVC status
- Only remove conditions that are managed by resize controller



```release-note
Improve messages user gets during and after volume resizing is done.
```
2018-02-06 07:55:32 -08:00
Kubernetes Submit Queue 9a8bb0098b
Merge pull request #59388 from jianglingxia/jlx-020610
Automatic merge from submit-queue (batch tested with PRs 54680, 59388, 57455). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix TestPlugin func has two same if code/kind bug

**What this PR does / why we need it**:
Fix TestPlugin func has two same if code
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-02-06 06:40:31 -08:00
Simon Croome 1a91403e43 StorageOS support for containerized kubelet and mount options
This change queries the StorageOS API to retrieve the volume
device dir when a volume is attached, rather than relying on
the default /var/lib/storageos/volumes directory.  This allows
all or some nodes to have kubelet running in a container with
the StorageOS volume path mounted in, which is required for
Azure ACS support.

Volume mount options are also supported.
2018-02-06 13:52:33 +00:00
jianglingxia 763fd53adf make sure mounter not nil and fix some typo 2018-02-06 18:28:07 +08:00
Kubernetes Submit Queue d9632f66af
Merge pull request #59328 from vmware/vSphereVolumeMetricsProvider
Automatic merge from submit-queue (batch tested with PRs 58317, 58687, 57872, 59063, 59328). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Volume metrics support for vSphere Cloud Provider

**What this PR does / why we need it**:
This PR adds Volume metrics support for vSphere Cloud Provider. With this we can monitor the available space, capacity and used space on volumes created using vSphere. 

**Which issue(s) this PR fixes**
Fixes # https://github.com/kubernetes/kubernetes/issues/43527
Fixes # https://github.com/vmware/kubernetes/issues/208

**Special notes for your reviewer**:
Performed following steps to ensure metrics are reported correctly for vSphere Volumes.

- Created Dynamic PV using the PV claim of size 1Gi.
```
$ kubectl get pvc
NAME        STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
thinclaim   Bound     pvc-bbc18f6a-0a04-11e8-bb1c-0050569c126b   1Gi        RWO            thin           5m
```

- Created Pod using this claim.
- Verified volume stats metrics.

```
/ # df -h | grep /dev/sdb
Filesystem                Size      Used Available Use% Mounted on
/dev/sdb                975.9M      1.3M    923.4M   0% /mnt/volume1
```

```
# HELP kubelet_volume_stats_available_bytes Number of available bytes in the volume
# TYPE kubelet_volume_stats_available_bytes gauge
kubelet_volume_stats_available_bytes{namespace="default",persistentvolumeclaim="thinclaim"} 9.68261632e+08

# HELP kubelet_volume_stats_capacity_bytes Capacity in bytes of the volume
# TYPE kubelet_volume_stats_capacity_bytes gauge
kubelet_volume_stats_capacity_bytes{namespace="default",persistentvolumeclaim="thinclaim"} 1.02330368e+09

# HELP kubelet_volume_stats_used_bytes Number of used bytes in the volume
# TYPE kubelet_volume_stats_used_bytes gauge
kubelet_volume_stats_used_bytes{namespace="default",persistentvolumeclaim="thinclaim"} 1.355776e+06
```

- Download 512MB file in the Pod on attached volume and Verified volume stats metrics is updated successfully.

```
/mnt/volume1 # df -h | grep /dev/sdb
Filesystem                Size      Used Available Use% Mounted on
/dev/sdb                975.9M    513.8M    410.9M  56% /mnt/volume1
```
```
# HELP kubelet_volume_stats_available_bytes Number of available bytes in the volume
# TYPE kubelet_volume_stats_available_bytes gauge
kubelet_volume_stats_available_bytes{namespace="default",persistentvolumeclaim="thinclaim"} 4.30862336e+08

# HELP kubelet_volume_stats_capacity_bytes Capacity in bytes of the volume
# TYPE kubelet_volume_stats_capacity_bytes gauge
kubelet_volume_stats_capacity_bytes{namespace="default",persistentvolumeclaim="thinclaim"} 1.02330368e+09

# HELP kubelet_volume_stats_used_bytes Number of used bytes in the volume
# TYPE kubelet_volume_stats_used_bytes gauge
kubelet_volume_stats_used_bytes{namespace="default",persistentvolumeclaim="thinclaim"} 5.38755072e+08
```

**Release note**:

```release-note
Volume metrics support for vSphere Cloud Provider
```
2018-02-05 18:43:43 -08:00
jianglingxia a5865ddd7b Fix TestPlugin func has two same if code/kind bug 2018-02-06 10:29:28 +08:00
Jing Xu 9588d2098a Redesign and implement volume reconstruction work
This PR is the first part of redesign of volume reconstruction work. The
changes include
1. Remove dependency on volume spec stored in actual state for volume
cleanup process (UnmountVolume and UnmountDevice)

Modify AttachedVolume struct to add DeviceMountPath so that volume
unmount operation can use this information instead of constructing from
volume spec

2. Modify reconciler's volume reconstruction process (syncState). Currently workflow
is when kubelet restarts, syncState() is only called once before
reconciler starts its loop.
a. If volume plugin supports reconstruction, it will use the
reconstructed volume spec information to update actual state as before.
b. If volume plugin cannot support reconstruction, it will use the
scanned mount path information to clean up the mounts.

In this PR, all the plugins still support reconstruction (except
glusterfs), so reconstruction of some plugins will still have issues.
The next PR will modify those plugins that cannot support reconstruction
well.

This PR addresses issue #52683, #54108 (This PR includes the changes to
update devicePath after local attach finishes)
2018-02-05 13:14:09 -08:00
Kubernetes Submit Queue ffda1e2200
Merge pull request #57017 from andyzhangx/azurefile-growsize
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add PV size grow feature for azure file

**What this PR does / why we need it**:
According to kubernetes/features#284, add size grow feature for azure file

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56462 

**Special notes for your reviewer**:
Since azure file is using SMB 3.0 protocal, there is no necessary to resize filesystem on agent side, the agent node will detect the changed size automatically.

**Release note**:

```
add size grow feature for azure file
```
/sig azure
@gnufied @rootfs @brendandburns
2018-02-05 11:25:48 -08:00
Divyen Patel 46edaa7269 Adding volume metrics support for vSphere Cloud Provider 2018-02-04 16:10:15 -08:00
Davanum Srinivas 73b46ff7db Fix golint for openstack and cinder packages 2018-02-04 17:15:32 -05:00
Kubernetes Submit Queue a8581f41e6
Merge pull request #59097 from zhangxiaoyu-zidif/delete-redundant-get-volumesource
Automatic merge from submit-queue (batch tested with PRs 59097, 57076, 59295). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

delete duplicate function for getting volume source

**What this PR does / why we need it**:
these two methods have same function.
```go
func getVolumeSource(
	spec *volume.Spec) (*v1.GlusterfsVolumeSource, bool, error) {
	if spec.Volume != nil && spec.Volume.Glusterfs != nil {
		return spec.Volume.Glusterfs, spec.Volume.Glusterfs.ReadOnly, nil
	} else if spec.PersistentVolume != nil &&
		spec.PersistentVolume.Spec.Glusterfs != nil {
		return spec.PersistentVolume.Spec.Glusterfs, spec.ReadOnly, nil
	}

	return nil, false, fmt.Errorf("Spec does not reference a Glusterfs volume type")
}

func (plugin *glusterfsPlugin) getGlusterVolumeSource(spec *volume.Spec) (*v1.GlusterfsVolumeSource, bool) {
	// Glusterfs volumes used directly in a pod have a ReadOnly flag set by the pod author.
	// Glusterfs volumes used as a PersistentVolume gets the ReadOnly flag indirectly through the persistent-claim volume used to mount the PV
	if spec.Volume != nil && spec.Volume.Glusterfs != nil {
		return spec.Volume.Glusterfs, spec.Volume.Glusterfs.ReadOnly
	}
	return spec.PersistentVolume.Spec.Glusterfs, spec.ReadOnly
}
```

`getVolumeSource`  seems to be much better.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-02-02 19:37:34 -08:00
bin liu e44779d86f Reformat log to show more details 2018-02-03 11:01:54 +08:00
Kubernetes Submit Queue 49ac543a56
Merge pull request #59072 from sbezverk/fc_iscsi_loopback_cleanup
Automatic merge from submit-queue (batch tested with PRs 55606, 59185, 58763, 59072, 59251). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Cleaning up loopback removal process

Handling more graceful the process of loopback removal. In case loopback device does not exist any longer, we should not fail the detachment process. 
```release-note

```
2018-02-02 18:26:41 -08:00
Kubernetes Submit Queue c0a7919ac1
Merge pull request #58794 from gnufied/perform-online-resize-if-mounted
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Perform resize of mounted volume if necessary

Under certain conditions - we must perform resize of volume even when it is mounted. This enables us to get around problem of resizing volumes used by deployments etc.

```release-note
Allow expanding mounted volumes
```
2018-02-02 12:48:53 -08:00
Serguei Bezverkhi 40e0c38988 Cleaning up loopback removal process 2018-02-02 10:33:25 -05:00
zhangxiaoyu-zidif a7dd7f14c9 delete duplicate function for getting volume source 2018-02-02 18:58:24 +08:00
linyouchong ddf4c2d62c fix TODO:change to a api-server watch 2018-02-02 17:41:45 +08:00
Kubernetes Submit Queue d3b783d5ec
Merge pull request #58743 from NickrenREN/pv-protection
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Postpone PV deletion with finalizer when it is being used

Postpone PV deletion if it is bound to a PVC

xref: https://github.com/kubernetes/community/pull/1608


**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #33355

**Special notes for your reviewer**:

**Release note**:
```release-note
Postpone PV deletion when it is being bound to a PVC
```

WIP, assign to myself first

/assign @NickrenREN
2018-02-01 19:39:52 -08:00
Kubernetes Submit Queue 8363129ee6
Merge pull request #58462 from NickrenREN/va-to-beta
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add V1beta1 VolumeAttachment API

**What this PR does / why we need it**:
Add V1beta1 VolumeAttachment API, co-existing with Alpha API object

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #58461

**Special notes for your reviewer**:

**Release note**:
```release-note
Add V1beta1 VolumeAttachment API, co-existing with Alpha API object
```
2018-02-01 18:50:25 -08:00
zhangxiaoyu-zidif 03d338a776 clean one redundant comment of rbd.go 2018-02-01 11:52:33 +08:00
Kubernetes Submit Queue 1150de9ce6
Merge pull request #58888 from lpabon/b58813
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

csi: Update version comparison model

**What this PR does / why we need it**:
CSI version matching needed to be updated to be able to support different patch levels during 0.x.x versions, and different minor.patch levels during >=1.x.x versions.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #58813

```release-note
NONE
```
2018-01-31 10:12:12 -08:00
NickrenREN cbfa0cc85a reuse PVC protection admission plugin for PV protection 2018-01-31 20:02:01 +08:00
NickrenREN 7b9d2c046f Use v1beta1 VolumeAttachment 2018-01-31 18:46:11 +08:00
andyzhangx fc988d429b initial work for azure file grow size implementation
enable azure file grow size

fix according to comments

fix comments

fix review comments

fix comments
2018-01-30 13:36:29 +00:00
zhangxiaoyu-zidif e873c7b28b fix mistaken info print 2018-01-30 14:50:19 +08:00
Hemant Kumar afeb53e5ee Perform resize of mounted volume if necessary
Add e2e test for mounted volume resize
2018-01-29 17:49:50 -05:00
Hemant Kumar 1fa8cbc5e4 Improve messaging on resize
- we now provide clear message to user what to do when cloudprovider resizing is finished
  and file system resizing is needed.
- add a event when resizing is successful.
- Use Patch for updating PVCs in both kubelet and controller-manager
- Extract updating pvc util function in one place.
- Only update resize conditions on progress
2018-01-29 15:07:51 -05:00
Kubernetes Submit Queue 59bc6d0ed6
Merge pull request #58862 from wackxu/vlm
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Mini fix about typo

```release-note
NONE
```
2018-01-29 07:29:33 -08:00
Kubernetes Submit Queue a0acd86569
Merge pull request #58431 from mlmhl/aws_volume_attacher
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

remove duplicated check of device path in aws attacher

**What this PR does / why we need it**:

The `devicePath` parameter is already validated in this [code](b7100f1ee7/pkg/volume/aws_ebs/attacher.go (L158)), so no need to check it again in the `for loop` as it won't be modified.

This can make the code clearer.

**Release note**:

```release-note
NONE
```

/sig storage
/kind cleanup
2018-01-26 11:43:16 -08:00
Luis Pabón 4abcb12296 csi: Update version comparison model
If our supported version is still at 0.X.X, then check
also the minor number. If our supported version is >= 1.X.X
then check only the major number.

Closes #58813
2018-01-26 12:51:10 -05:00
Kubernetes Submit Queue 175df0cba0
Merge pull request #58617 from humblec/expand-idmpt
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Make ExpandVolumeDevice() idempotent if existing volume capacity meets the requested size.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note

```
2018-01-26 09:07:41 -08:00
Kubernetes Submit Queue 27d01b5ab9
Merge pull request #57938 from dims/add-binary-configmap
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add binary configmap

Reviving code from https://github.com/kubernetes/kubernetes/pull/33549 submitted by @zreigz

**What this PR does / why we need it**:
Add support for binary files in ConfigMap

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #32432

**Special notes for your reviewer**:

**Release note**:

```release-note
ConfigMap objects now support binary data via a new `binaryData` field. When using `kubectl create configmap --from-file`, files containing non-UTF8 data will be placed in this new field in order to preserve the non-UTF8 data. Use of this feature requires 1.10+ apiserver and kubelets.
```
2018-01-26 04:34:33 -08:00
wackxu cd48a6745a mini fix about typo 2018-01-26 15:42:57 +08:00
WanLinghao 70ef581ecc fix some log param error
modified:   pkg/cloudprovider/providers/vsphere/vsphere_util.go
	modified:   pkg/controller/certificates/cleaner/cleaner.go
	modified:   pkg/controller/volume/pvcprotection/pvc_protection_controller.go
	modified:   pkg/volume/azure_dd/azure_mounter.go
2018-01-26 12:16:00 +08:00
Kubernetes Submit Queue 48c5e90d3b
Merge pull request #58626 from humblec/correct-ann
Automatic merge from submit-queue (batch tested with PRs 58626, 58791). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Use correct pv annotation to fetch volume ID.

Fix , Issue #58627
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note

```
2018-01-25 14:06:34 -08:00
Kubernetes Submit Queue 191f9dd5ff
Merge pull request #58549 from dims/backup-default-location-for-cloud-config
Automatic merge from submit-queue (batch tested with PRs 58480, 58549). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Use backup location to load cloud config for OpenStack

**What this PR does / why we need it**:

Since we are transitioning to external cloud provider, we need a way
to use the existing cinder volume plugin (from kubelet). With external
cloud manager kubelet will be run with --cloud=provider=external and
no --cloud-config file will be in the command line. So we need a way
to load the openstack config file from somewhere.

Taking a cue from kubeadm, which currently is picking up "/etc/kubernetes/cloud-config"
https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/phases/controlplane/manifests.go#L44

let's support the scenario where we fall back to this static location if
there is no cloud provider specified in the command line.

This has been tested with local-up-cluster using the following params:
EXTERNAL_CLOUD_PROVIDER=true
CLOUD_PROVIDER=openstack
CLOUD_CONFIG=/etc/kubernetes/cloud-config


**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-01-23 05:01:31 -08:00
Lukasz Zajaczkowski 7e158fb4f6 Add support for binary file in configmap 2018-01-23 07:28:34 -05:00
Kubernetes Submit Queue 5f6e12793b
Merge pull request #57948 from zhangxiaoyu-zidif/fix-ut-print
Automatic merge from submit-queue (batch tested with PRs 58438, 58523, 58513, 57948). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix csi ut print

**What this PR does / why we need it**:
fix csi ut print

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-01-23 03:53:33 -08:00
Kubernetes Submit Queue 1cb86fa1eb
Merge pull request #58513 from humblec/custom-name2
Automatic merge from submit-queue (batch tested with PRs 58438, 58523, 58513, 57948). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add Namespace to glusterfs custom volume names.

For better isolation of volumes, users/admins would like to include `namespace` in custom volume names. This patch adds that support.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note

```
2018-01-23 03:53:31 -08:00
Humble Chirammal 6e6b5acbb5 Use correct pv annotation to fetch volume ID.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2018-01-23 00:41:53 +05:30
Humble Chirammal 14a2671d08 Make ExpandVolumeDevice() idempotent if existing volume
capacity meets the requested size.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2018-01-22 23:23:19 +05:30
Davanum Srinivas 205b2eb3fd Use backup location to load cloud config for OpenStack
Since we are transitioning to external cloud provider, we need a way
to use the existing cinder volume plugin (from kubelet). With external
cloud manager kubelet will be run with --cloud=provider=external and
no --cloud-config file will be in the command line. So we need a way
to load the openstack config file from somewhere.

Taking a cue from kubeadm, which currently is picking up "/etc/kubernetes/cloud-config"
https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/phases/controlplane/manifests.go#L44

let's support the scenario where we fall back to this static location if
there is no cloud provider specified in the command line.

This has been tested with local-up-cluster using the following params:
EXTERNAL_CLOUD_PROVIDER=true
CLOUD_PROVIDER=openstack
CLOUD_CONFIG=/etc/kubernetes/cloud-config
2018-01-22 09:40:09 -05:00
Kubernetes Submit Queue 5a44d2ccac
Merge pull request #58226 from lpabon/b58092
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

csi: Fix versioning error message

**What this PR does / why we need it**:
Incorrect error message

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #58092
2018-01-19 13:52:54 -08:00
Humble Chirammal 2c34b9cf77 Add Namespace to glusterfs custom volume names.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2018-01-19 17:07:08 +05:30
Kubernetes Submit Queue 8cc5ccfb49
Merge pull request #58209 from NickrenREN/csi-fstype
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add FSType for CSI volume source

**What this PR does / why we need it**:
Add FSType for CSI volume source to specify filesystems (alpha defaults to `ext4`)

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #58183

**Special notes for your reviewer**:

**Release note**:
```release-note
Add FSType for CSI volume source to specify filesystems
```

/assign @saad-ali 
cc @vladimirvivien
2018-01-18 09:35:58 -08:00
Kubernetes Submit Queue 3c99777d38
Merge pull request #57422 from joelsmith/nested_data_vol
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Rework method of updating atomic-updated data volumes

**What this PR does / why we need it**:

This change affects the way that secret, configmap, downwardAPI and projected volumes (which all use the same underlying code) implement their data update functionality.

* Instead of creating a subdirectory hierarchy that will contain symlinks to each actual data file, create only symlinks to items in the root of the volume, whether they be files or directories.
* Rather than comparing the user-visible data directory to see if an update is needed, compare with the current version of the data directory.
* Fix data dir timestamp format year
* Create `..data` symlink even when a data volume has no data so consumers can have simplified update watch logic.

**Which issue(s) this PR fixes**:
Fixes #57421

**Release note**:
```release-note
Correct issues that arise when volumes are mounted beneath another secret, configmap, downwardAPI or projected volume
```
2018-01-18 03:20:19 -08:00
mlmhl edd66c9bdf remove duplicated check of device path in aws attacher 2018-01-18 12:36:14 +08:00
Kubernetes Submit Queue 74f2305c4f
Merge pull request #57967 from jsafrane/fix-nfs-root-squash
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fixed TearDown of NFS with root squash.

NFS plugin should not use `IsLikelyNotMountPoint()`, as it uses `lstat()` / `stat()` to determine if the NFS volume is still mounted - NFS server may use root_squash and kubelet may not be allowed to do `lstat()` / `stat()` there.

It must use slower `IsNotMountPoint()` instead, including in `TearDown()` function.

**Release note**:

```release-note
NONE
```

/assign @gnufied @rootfs
2018-01-17 13:32:03 -08:00
Joel Smith b4e0923785 Rework method of updating atomic-updated data volumes
This change affects the way that secret, configmap, downwardAPI and projected
volumes (which all use the same underlying code) implement their data update
functionality.

* Instead of creating a subdirectory hierarchy that itself
  will contain symlinks to each actual data file, create only
  symlinks to items in the root of the volume, whether they
  be files or directories.
* Rather than comparing the user-visible data directory
  to see if an update is needed, compare with the current
  version of the data directory.
* Fix data dir timestamp format year
* Create ..data symlink even when a data volume has no data so
  consumers can have simplified update watch logic.
2018-01-17 12:09:20 -07:00
Mitsuhiro Tanino 6fa583381e Update comments for getting and removing loopback device at iSCSI,FC,RBD
This PR add comments for the background why plugin gets loopback
device and removes loopback device even if operation_generator has
same functionality.
2018-01-17 10:22:39 -05:00
Kubernetes Submit Queue 20f27a77ee
Merge pull request #58316 from andyzhangx/azurefile-setfsgroup
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

set fsGroup by securityContext.fsGroup in azure file

**What this PR does / why we need it**:
set fsGroup by securityContext.fsGroup in azure file
This PR respects `securityContext.fsGroup`, while if user both sets `gid=xxx` in mountOptions in azure storage class and `securityContext.fsGroup`,  `gid=xxx` setting in mountOptions will win.

```
apiVersion: v1
kind: Pod
metadata:
  name: security-context-demo
spec:
  securityContext:
    fsGroup: 2000
  volumes:
  - name: sec-ctx-vol
    emptyDir: {}
  containers:
  - name: sec-ctx-demo
    image: gcr.io/google-samples/node-hello:1.0
    volumeMounts:
    - name: sec-ctx-vol
      mountPath: /data/demo
    securityContext:
      allowPrivilegeEscalation: false
```

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #44085

**Special notes for your reviewer**:

**Release note**:

```
set fsGroup by securityContext.fsGroup in azure file
```

/assign @karataliu 
cc @rootfs @avaranovich
2018-01-17 04:29:03 -08:00
andyzhangx c38e7589a4 set fsGroup by securityContext.fsGroup in azure file
fix comments

fix comments

fix comments
2018-01-17 09:24:50 +00:00
Cao Shufeng 4e7398b67b remove duplicated import 2018-01-17 09:34:59 +08:00
NickrenREN 87cebae738 Add fsType for CSI 2018-01-16 12:34:04 +08:00
Luis Pabón ea31f8f7ec csi: Fix versioning error message
Closes #58092
2018-01-15 10:27:00 -08:00
Kubernetes Submit Queue b5eb9149ee
Merge pull request #58089 from mtanino/issue/58085
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

[FC Plugin] Create proper volumeSpec during ConstructVolumeSpec

**What this PR does / why we need it**:

Currently, FC plugin returns volume name and empty FCVolumeSource during ConstrutVolumeSpec of filesystem volume's reconstruction.

In this fix, ConstructVolumeSpec retrieves global mount path, and analyzes volume parameters such as
WWN, LUN, WWID from the path.

**Which issue(s) this PR fixes** :
Fixes #58085

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

cc @rootfs @jsafrane 
/sig storage
2018-01-12 16:26:23 -08:00
Kubernetes Submit Queue 744a6f0cde
Merge pull request #57516 from humblec/custom
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add custom volume name based on SC parameter.

At present glusterfs dynamic PVs are created with
    random names. However an admin would like to have some
    handle on the volume names created dynamically for
    various purposes. One example would be having a filter
    for sorting out PVs created for a particular storage class.
    
    This patch enables the functionality by having a custom
    volume name as a prefix to dynamic PVs. This is an optional
    parameter in SC and if set, the dynamic volumes are created
    in below format where `_` is the field seperator/delimiter:
    
    customvolumeprefix_PVCname_randomUUID

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2018-01-12 04:51:54 -08:00
Kubernetes Submit Queue 820ea04f08
Merge pull request #51098 from jsafrane/exec-git
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

git: Use VolumeHost.GetExec() to execute stuff in volume plugins

This prepares volume plugins to run things in containers instead of running
them on the host.

**Special notes for your reviewer**:

* instead of `cmd.SetDir(<dir>); cmd.Exec("git <command>")`, we do `cmd.Exec("git -C <dir> <command>")` - mounter.Exec does not have SetDir()
* there are lot of changes in the tests because a different exec interface is used.

@kubernetes/sig-storage-pr-reviews 

```release-note
gitRepo volumes in pods require git 1.8.5 or later
```

/assign @rootfs
2018-01-11 09:22:21 -08:00
mtanino 41cb533ad6 [FC Plugin] Create proper volumeSpec during ConstructVolumeSpec
Currently, FC plugin returns volume name and empty
FCVolumeSource during ConstrutVolumeSpec during
filesystem volume's reconstruction.

In this fix, ConstructVolumeSpec retrieves global
mount path, analyzes volume parameters such as
WWN, LUN, WWID from the path.

Fixes #58085
2018-01-11 12:04:56 -05:00
Kubernetes Submit Queue e4851c1c85
Merge pull request #52324 from linyouchong/linyouchong-20170912
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix bug:Kubelet failure to umount  mount points

What this PR does / why we need it:
Fix bug#41141:Kubelet failure to umount glusterfs mount points
kubelet failure to umount mount moints in these cases:
1. glusterfs volume : transport endpoint is not connected
2. nfs volume : stale NFS file handle
this PR will fix it

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
fixes #41141
Special notes for your reviewer:
NONE
Release note:
NONE
2018-01-11 07:04:07 -08:00
Kubernetes Submit Queue 53a8ac753b
Merge pull request #56846 from zetaab/fixvolumeattached
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix cinder detach problems

**What this PR does / why we need it**: We have currently huge problems in cinder volume detach. This PR tries to fix these issues.

**Which issue(s) this PR fixes**:
Fixes #50004 
Fixes #57497

**Special notes for your reviewer**: 
**Release note**:

```release-note
openstack cinder detach problem is fixed if nova is shutdowned
```
2018-01-11 04:24:09 -08:00
Humble Chirammal 66c7fdb892 Update bazel.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2018-01-11 14:14:29 +05:30
Humble Chirammal e826a77919 Add custom volumename option to GlusterFS dynamic PVs.
At present glusterfs dynamic PVs are created with
random names. However an admin would like to have some
handle on the volume names created dynamically for
various purposes. One example would be having a filter
for sorting out PVs created for a particular storage class.

This patch enables the functionality by having a custom
volume name as a prefix to dynamic PVs. This is an optional
parameter in SC and if set, the dynamic volumes are created
in below format where `_` is the field seperator/delimiter:

customvolumeprefix_PVCname_randomUUID

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2018-01-11 14:01:07 +05:30
Kubernetes Submit Queue 2f17d782eb
Merge pull request #54752 from mtanino/pr/BlockVolumesSupport-iscsi
Automatic merge from submit-queue (batch tested with PRs 54230, 58100, 57861, 54752). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Block volumes Support: iSCSI plugin update

**What this PR does / why we need it**:

Add interface changes to iSCSI volume plugin to enable block volumes support feature.

**Which issue this PR fixes**: 
Based on this proposal (kubernetes/community#805 & kubernetes/community#1265) and this feature issue: kubernetes/features#351

**Special notes for your reviewer**:

This PR temporarily includes following changes except iSCSI plugin change for reviewing purpose.
These changes will be removed from the PR once they are merged.
- (#50457) API Change 
- (#51494) Container runtime interface change, volumemanager changes, operationexecutor changes

There are another PRs related to this functionality.
(#50457) API Change
(#53385) VolumeMode PV-PVC Binding change
(#51494) Container runtime interface change, volumemanager changes, operationexecutor changes
(#55112) Block volume: Command line printer update
Plugins
(#51493) Block volumes Support: FC plugin update
(#54752) Block volumes Support: iSCSI plugin update

**Release note**:

```
NONE
```
2018-01-10 13:29:38 -08:00
Kubernetes Submit Queue 70ee7d826f
Merge pull request #57823 from mlmhl/rbd_device_mount_path
Automatic merge from submit-queue (batch tested with PRs 57823, 58091, 58093, 58096, 57020). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix rbd ConstructVolumeSpec bug

**What this PR does / why we need it**:

Fix rbd `ConstructVolumeSpec` bug as its global device mount path is not canonical. Full details are in #57744 .

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #57744 

**Release note**:

```release-note
NONE
```

/sig storage
/kind bug
2018-01-10 12:37:38 -08:00
Jesse Haka 4e1b5c6a32 move detach out of os volumes attach
add test

add test

fix bazel

fix tests

change loglevel, remove else statement
2018-01-10 19:09:32 +02:00
mtanino 96509d4f5b generated code for iSCSI plugin change 2018-01-10 11:39:01 -05:00
Mitsuhiro Tanino a6d979dd88 Block volumes Support: iSCSI plugin update
This patch adds block volume support to iSCSI volume plugin.
2018-01-10 11:38:36 -05:00
Kubernetes Submit Queue f65be3747d
Merge pull request #56823 from humblec/volid-delete-resize
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add volID based delete() and resize() if volID is available in pv spec.
2018-01-10 07:17:47 -08:00
linyouchong 19003486bf Fix bug:Kubelet failure to umount mount points 2018-01-10 21:36:05 +08:00
Humble Chirammal fc6443ce2c Add volID based delete() and resize() if volID is available in pv spec.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2018-01-10 18:30:55 +05:30
Kubernetes Submit Queue 6b925ac24b
Merge pull request #56851 from zouyee/ceph-1
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

check function return err

**What this PR does / why we need it**:
check function return err

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```NONE

```
2018-01-10 03:12:28 -08:00
Kubernetes Submit Queue b9a62423c6
Merge pull request #57953 from andyzhangx/azuredisk-remount-fix
Automatic merge from submit-queue (batch tested with PRs 57733, 57613, 57953). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix device name change issue for azure disk: add remount logic

**What this PR does / why we need it**:
fix device name change issue for azure disk: add remount logic

Accoding to [Troubleshoot Linux VM device name change](https://docs.microsoft.com/en-us/azure/virtual-machines/linux/troubleshoot-device-names-problems), there is possibility of device name change, so when kubelet is restarted, we need to check whether the following two paths are still valid:
1. `/var/lib/kubelet/plugins/kubernetes.io/azure-disk/mounts/m358246426`: in MountDevice func
2. `/var/lib/kubelet/pods/950f2eb8-d4e7-11e7-bc95-000d3a041274/volumes/kubernetes.io~azure-disk/pvc-67e4e319-d4e7-11e7-bc95-000d3a041274`: in SetUpAt func

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #57952

**Special notes for your reviewer**:
 this is a corresponding fix of https://github.com/kubernetes/kubernetes/pull/57549, #57549 uses '/dev/disk/by-id', and this PR would check whether the mountPath is valid when kubelet restart(e.g.  after VM reboot since device name may change), if not valid, remount,  remember '/dev/disk/by-id' will be always valid.

**Release note**:

```
fix device name change issue for azure disk: add remount logic
```
2018-01-09 21:51:40 -08:00
Kubernetes Submit Queue 33134cb02c
Merge pull request #57993 from humblec/volume-metrics
Automatic merge from submit-queue (batch tested with PRs 57993, 58028). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

 Add volumemetrics for glusterfs plugin.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note

```
2018-01-09 19:39:29 -08:00
Kubernetes Submit Queue 389089846a
Merge pull request #57264 from zhangxiaoyu-zidif/fix-binary-check-cephfs
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix binary check for cephfs.go

**What this PR does / why we need it**:
unify the check method. see ref issue: #56479

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56479

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-01-09 13:26:54 -08:00
Humble Chirammal dc0b7da5b7 Add volumemetrics for glusterfs plugin.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2018-01-09 22:44:33 +05:30
mlmhl 2ccf22a5d0 fix rbd ConstructVolumeSpec bug 2018-01-09 10:27:05 +08:00
Kubernetes Submit Queue 8a9954d471
Merge pull request #56651 from sbezverk/block_rbd_volume_plugin
Automatic merge from submit-queue (batch tested with PRs 57784, 56651). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Adding support for Block Volume to rbd plugin

Adding support for Block Volume to rbd plugin

```release-note
Adding support for Block Volume type to rbd plugin.
```
2018-01-08 09:12:49 -08:00
Jan Safranek 45d21ee36b Fixed TearDown of NFS with root squash.
NFS plugin should not use IsLikelyNotMountPoint(), as it uses lstat() / stat()
to determine if the NFS volume is still mounted - NFS server may use
root_squash and kubelet may not be allowed to do lstat() / stat() there.

It must use slower IsNotMountPoint() instead, including TearDown() function.
2018-01-08 14:01:33 +01:00
andyzhangx 1bfb5d0670 add remount logic if original mount path is invalid 2018-01-08 06:01:42 +00:00
zhangxiaoyu-zidif 0209ce9137 fix csi attach ut print 2018-01-08 12:03:55 +08:00
zhangxiaoyu-zidif ed3912e9e3 fix csi mounter ut print 2018-01-08 10:42:05 +08:00
Serguei Bezverkhi 335c5d959f Adding support for Block Volume to rbd plugin 2018-01-05 23:56:37 -05:00
Kubernetes Submit Queue c02f54851f
Merge pull request #56315 from humblec/return-volid
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Record volumeID in GlusterFS PV spec.
2018-01-04 13:26:12 -08:00
Kubernetes Submit Queue d8680a3c43
Merge pull request #57549 from andyzhangx/azurediskid-fix
Automatic merge from submit-queue (batch tested with PRs 56382, 57549). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix azure disk not available issue when device name changed

**What this PR does / why we need it**:
There is possibility that device name(`/dev/sd*`) would change when attach/detach data disk in Azure VM according to  [Troubleshoot Linux VM device name change](https://docs.microsoft.com/en-us/azure/virtual-machines/linux/troubleshoot-device-names-problems). 
And We did hit this issue, see customer [case](https://github.com/Azure/acs-engine/issues/1918). 
This PR would use `/dev/disk/by-id` instead of `/dev/sd*` for azure disk and `/dev/disk/by-id` would not change even device name changed.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #57444

**Special notes for your reviewer**:
In a customer [case](https://github.com/Azure/acs-engine/issues/1918), customer is unable to use azure disk in StatefulSet since /dev/sd* changed after detach/attach disk. 
we are using `/dev/sd*`(code is [here](https://github.com/kubernetes/kubernetes/blob/master/pkg/volume/azure_dd/azure_common_linux.go#L140)) to "mount -bind" k8s path, while `/dev/sd*` could be changed when VM is attach/detaching data disks, see [Troubleshoot Linux VM device name change](https://docs.microsoft.com/en-us/azure/virtual-machines/linux/troubleshoot-device-names-problems)
And I have also checked related AWS, GCE code, they are using `/dev/disk/by-id/` other than `/dev/sd*`, see [aws code](https://github.com/kubernetes/kubernetes/blob/master/pkg/volume/aws_ebs/aws_util.go#L228)  
[gce code](https://github.com/kubernetes/kubernetes/blob/master/pkg/volume/gce_pd/gce_util.go#L278)

**Release note**:

```
fix azure disk not available when device name changed
```
/sig azure
/assign @rootfs 
@karataliu @brendandburns @khenidak
2018-01-04 06:40:34 -08:00
zhangxiaoyu-zidif ec6b9eb956 fix-binary-check-cephfs 2018-01-04 11:08:40 +08:00
Kubernetes Submit Queue 5e70562c6a
Merge pull request #57512 from cofyc/improve_rbd_highavailability
Automatic merge from submit-queue (batch tested with PRs 57572, 57512, 57770). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

RBD Plugin: Pass monitors addresses in a comma-separed list instead of trying one by one.

**What this PR does / why we need it**:

In production, monitors may crash (or have a network problem), if we try monitors one by one, rbd
command will hang a long time (e.g. `rbd map -m <unconnectable_host_ip>`
on linux 4.4 timed out in 6 minutes) when trying a unconnectable monitor. This is unacceptable.

Actually, we can simply pass a comma-separated list monitor addresses to `rbd`
command utility. Kernel rbd/libceph modules will pick monitor randomly
and try one by one, `rbd` command utility succeed soon if there is a
good one in monitors list.

[Docs](http://docs.ceph.com/docs/jewel/man/8/rbd/#cmdoption-rbd-m) about `-m` option of `rbd` is wrong,  'rbd' utility simply pass '-m <mon>' parameter to kernel rbd/libceph modules, which
takes a comma-seprated list of one or more monitor addresses (e.g. ip1[:port1][,ip2[:port2]...]) in its first version in linux (see 602adf4002/net/ceph/ceph_common.c (L239)). Also, libceph choose monitor randomly, so we can simply pass all addresses without randomization (see 602adf4002/net/ceph/mon_client.c (L132)).

From what I saw, there is no need to iterate monitor hosts one by one.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

Run `rbd map` against unconnectable monitor address logs on Linux 4.4:

```
root@myhost:~# uname -a
Linux myhost 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
root@myhost:~# time rbd map kubernetes-dynamic-pvc-941ff4d2-b951-11e7-8836-049fca8e58df --pool <pool> --id <id> -m <unconnectable_host_ip> --key=<password>
rbd: sysfs write failed
2017-12-20 18:55:11.810583 7f7ec56863c0  0 monclient(hunting): authenticate timed out after 300
2017-12-20 18:55:11.810638 7f7ec56863c0  0 librados: client.<id> authentication error (110) Connection timed out
rbd: couldn't connect to the cluster!
In some cases useful info is found in syslog - try "dmesg | tail" or so.
rbd: map failed: (110) Connection timed out

real	6m0.018s
user	0m0.052s
sys	0m0.064s
```  

We can simply pass a comma-separated list of monitors, if there is a good one in them, `rbd map` succeed soon.

```
root@myhost:~# time rbd map kubernetes-dynamic-pvc-941ff4d2-b951-11e7-8836-049fca8e58df --pool <pool> --id <id> -m <unconnectable_host_ip>,<good_host_ip> --key=<password>

/dev/rbd3

real	0m0.426s
user	0m0.008s
sys	0m0.008s
```

**Release note**:

```release-note
NONE
```
2018-01-03 13:46:32 -08:00
Kubernetes Submit Queue b729472952
Merge pull request #57779 from jolestar/flex-volume-fix
Automatic merge from submit-queue (batch tested with PRs 57366, 57779). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Remove unused command waitfordetach from flex volume driver

**What this PR does / why we need it**:
Clean unused code.

**Special notes for your reviewer**:
See #35629
Original PR is https://github.com/kubernetes/kubernetes/pull/50754 , the PR's repo has bean deleted,so I create a new PR and merge upstream for test.
**Release note**:

```release-note
NONE
```
2018-01-03 12:09:34 -08:00
Kubernetes Submit Queue 6dfc4b38fb
Merge pull request #57702 from mlmhl/volume_resize_event
Automatic merge from submit-queue (batch tested with PRs 57702, 57128). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

format error message and remove duplicated event for resize volume failure

**What this PR does / why we need it**:

1. The `operationGenerator.resizeFileSystem` method returns errors generated by `volumeToMount.GenerateErrorDetailed`, and the outside code(`operationGenerator.GenerateMountVolumeFunc`) uses `volumeToMount.GenerateError` to generate a new error again, which lead to the event message redundant and confused, we should use `volumeToMount.GenerateError` inside `operationGenerator.resizeFileSystem` only, in outside code is not necessary.

2. The `eventRecorderFunc` will record an event if `resizeFileSystem` returns an error, so we needn't to record event inside `resizeFileSystem` itself.

**Release note**:

```release-note
NONE
```

/sig storage
/kind enhancement
2018-01-03 08:30:30 -08:00
andyzhangx a4786fc8b0 prefer /dev/disk/azure/scsi1/ over by-id for azure disk
remove string conversion
2018-01-03 09:33:42 +00:00
jolestar eb35cee9f8 Remove unused command waitfordetach from flex volume driver 2018-01-03 16:02:31 +08:00
Kubernetes Submit Queue 7253c35ceb
Merge pull request #57675 from NickrenREN/expand-panic-fix
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix ExpandController panic bug

If we want to expand a volume which is not supported for now, for example, in release 1.8, we want to expand RBD volume, controller-manager will panic.
We do not check if volume plugin is nil in `GenerateExpandVolumeFunc`  PTAL, thanks
```
Dec 27 23:32:26 master kube-controller-manager[4192]: /usr/local/go/src/runtime/asm_amd64.s:2337
Dec 27 23:32:26 master kube-controller-manager[4192]: panic: runtime error: invalid memory address or nil pointer dereference [recovered]
Dec 27 23:32:26 master kube-controller-manager[4192]:         panic: runtime error: invalid memory address or nil pointer dereference
Dec 27 23:32:26 master kube-controller-manager[4192]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x1a1e95a]
Dec 27 23:32:26 master kube-controller-manager[4192]: goroutine 1157 [running]:
Dec 27 23:32:26 master kube-controller-manager[4192]: k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
Dec 27 23:32:26 master kube-controller-manager[4192]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:58 +0x111
Dec 27 23:32:26 master kube-controller-manager[4192]: panic(0x2cc10c0, 0x4e280c0)
Dec 27 23:32:26 master kube-controller-manager[4192]:         /usr/local/go/src/runtime/panic.go:491 +0x283
Dec 27 23:32:26 master kube-controller-manager[4192]: k8s.io/kubernetes/pkg/volume/util/operationexecutor.(*operationGenerator).GenerateExpandVolumeFunc(0xc420872780, 0xc42163b400, 0x4d7bee0, 0xc4208727b0, 0xc422187cd0, 0xc422187cd0, 0x40e84f, 0x10, 0x2a901a0)
Dec 27 23:32:26 master kube-controller-manager[4192]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/volume/util/operationexecutor/operation_generator.go:787 +0x2ba
Dec 27 23:32:26 master kube-controller-manager[4192]: k8s.io/kubernetes/pkg/volume/util/operationexecutor.(*operationExecutor).ExpandVolume(0xc420dfa7a0, 0xc42163b400, 0x4d7bee0, 0xc4208727b0, 0x1, 0x1)
Dec 27 23:32:26 master kube-controller-manager[4192]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/volume/util/operationexecutor/operation_executor.go:726 +0x5c
Dec 27 23:32:26 master kube-controller-manager[4192]: k8s.io/kubernetes/pkg/controller/volume/expand.(*syncResize).Sync(0xc420d09780)
Dec 27 23:32:26 master kube-controller-manager[4192]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/controller/volume/expand/sync_volume_resize.go:81 +0x324
Dec 27 23:32:26 master kube-controller-manager[4192]: k8s.io/kubernetes/pkg/controller/volume/expand.(*syncResize).Sync-fm()
Dec 27 23:32:26 master kube-controller-manager[4192]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/controller/volume/expand/sync_volume_resize.go:60 +0x2a
Dec 27 23:32:26 master kube-controller-manager[4192]: k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc42128a7b0)
Dec 27 23:32:26 master kube-controller-manager[4192]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x5e
Dec 27 23:32:26 master kube-controller-manager[4192]: k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc422187fb0, 0x6fc23ac00, 0x0, 0xc4202b6101, 0xc420070d80)
Dec 27 23:32:26 master kube-controller-manager[4192]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134 +0xbd
Dec 27 23:32:26 master kube-controller-manager[4192]: k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.Until(0xc42128a7b0, 0x6fc23ac00, 0xc420070d80)
Dec 27 23:32:26 master kube-controller-manager[4192]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88 +0x4d
Dec 27 23:32:26 master kube-controller-manager[4192]: k8s.io/kubernetes/pkg/controller/volume/expand.(*syncResize).Run(0xc420d09780, 0xc420070d80)
Dec 27 23:32:26 master kube-controller-manager[4192]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/controller/volume/expand/sync_volume_resize.go:60 +0x53
Dec 27 23:32:26 master kube-controller-manager[4192]: created by k8s.io/kubernetes/pkg/controller/volume/expand.(*expandController).Run
Dec 27 23:32:26 master kube-controller-manager[4192]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/controller/volume/expand/expand_controller.go:154 +0x18e
Dec 27 23:32:26 master systemd[1]: kube-controller-manager.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Dec 27 23:32:26 master systemd[1]: kube-controller-manager.service: Unit entered failed state.
Dec 27 23:32:26 master systemd[1]: kube-controller-manager.service: Failed with result 'exit-code'.
```

**What this PR does / why we need it**:
Fix controller-manager panic

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #57684

**Special notes for your reviewer**:
I find some other places where we do not check if volumeplugin is nil, so i modify the `Find...PluginBy...` functions too.
Just like these do:
https://github.com/kubernetes/kubernetes/blob/master/pkg/volume/plugins.go#L576
https://github.com/kubernetes/kubernetes/blob/master/pkg/volume/plugins.go#L602

**Release note**:
```release-note
NONE
```

/kind bug
/sig storage

/cc @gnufied 
@left2right
2018-01-02 22:29:39 -08:00
Yecheng Fu b196301b67 RBD Plugin: Fix comments and remove unnecessary locking code. 2018-01-03 12:43:30 +08:00
NickrenREN 74b197e7fe fix expand panic 2018-01-03 10:31:57 +08:00
mlmhl 2bf6b54f05 format error message and remove duplicated event for resize volume
failure
2018-01-03 10:28:54 +08:00
Kubernetes Submit Queue 462db17df0
Merge pull request #57475 from stmcginnis/iscsi_node_startup
Automatic merge from submit-queue (batch tested with PRs 57399, 57751, 57475). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

iscsi: set node.startup to manual

If the default iSCSI node.startup is set to automatic, if there is a node failure,
any pods on that node will get rescheduled to another node. If the failed node is
later brought back up it will then try to log back in to any iSCSI sessions it had
prior to the failure, which may no longer exist or may be now in-use by the other
nodes.

It appears most platforms keep the open-iscsi default of node.startup-automatic.
But in case this system-wide setting has been changed, and just to be explicit, this
sets node.startup values for kubernetes controlled volumes to manual.

Fixes #21305

```release-note
iSCSI sessions managed by kubernetes will now explicitly set startup.mode to 'manual' to
prevent automatic login after node failure recovery. This is the default open-iscsi mode, so
this change will only impact users who have changed their startup.mode to be 'automatic'
in /etc/iscsi/iscsid.conf.
```
2018-01-02 10:57:57 -08:00
Jeff Grafton efee0704c6 Autogenerate BUILD files 2017-12-23 13:12:11 -08:00
Tim Hockin e9dd8a68f6 Revert k8s.gcr.io vanity domain
This reverts commit eba5b6092a.

Fixes https://github.com/kubernetes/kubernetes/issues/57526
2017-12-22 14:36:16 -08:00
andyzhangx 636a181d48 use /dev/disk/by-id instead of /dev/sd* for azure disk
change uuid to id
2017-12-22 08:26:12 +00:00
Kubernetes Submit Queue 5001198f0e
Merge pull request #57387 from andyzhangx/add-owners
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add Dong Liu as approver and add OWNERS in credentialprovider

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #57540

**Special notes for your reviewer**:

**Release note**:

```
none
```

/sig azure
/assign @brendandburns
2017-12-21 22:32:56 -08:00
Kubernetes Submit Queue 4562198ae6
Merge pull request #56925 from wenjun93/fix_rbd
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix rbd volume ConstructVolumeSpec

**What this PR does / why we need it**:
1. rbd plugin.ConstructVolumeSpec() construct volume spec with fake value, cause kubelet volume manager will create two volumesInUse in node Status.
2. change plugin.GetVolumeName(), create volumeName using rbd pool instead of monitors, because monitors is a group of IPs, which makes the volumeName too long. Also, this is to fit plugin.ConstructVolumeSpec() since makeGlobalPDName only uses rbd pool and image.

```
before fix:
volumesAttached:
  - devicePath: ""
    name: kubernetes.io/rbd/[xxxxxxx:6789 xxxxxxxxx:6789]:volume-9a106847-4def-4d1e-9603-4c7099b22a31
  volumesInUse:
  - 'kubernetes.io/rbd/[]:'
  - kubernetes.io/rbd/[xxxxxxx:6789 xxxxxxxxx:6789]:volume-9a106847-4def-4d1e-9603-4c7099b22a31

after fix:
volumesAttached:
  - devicePath: ""
    name: kubernetes.io/rbd/volumes:volume-9a106847-4def-4d1e-9603-4c7099b22a31
  volumesInUse:
  - kubernetes.io/rbd/volumes:volume-9a106847-4def-4d1e-9603-4c7099b22a31
2017-12-21 22:32:47 -08:00
Yecheng Fu 08d1c0b412 RBD Plugin: Pass monitors addresses in a comma-separated list instead of
trying one by one.

In production, monitors may crash (or have a network problem), if we try
monitors one by one, rbd command will hang a long time (e.g. `rbd map -m
<unconnectable_host_ip>` on linux 4.4 timed out in 6 minutes) when
trying a unconnectable monitor. This is unacceptable.

Actually, we can simply pass a comma-separed list monitor addresses
to `rbd` command utility. Kernel rbd/libceph modules will pick
monitor randomly and try one by one, `rbd` command utility succeed soon
if there is a good one in monitors list.
2017-12-21 21:02:25 +08:00
Sean McGinnis 9f97d61de4
Fixed space/tab indentation
Some lines had spaces for indentation instead of tabs.
2017-12-20 23:48:15 -06:00
Kubernetes Submit Queue 8f4f07fcc1
Merge pull request #57489 from brendandburns/errs
Automatic merge from submit-queue (batch tested with PRs 57215, 57489). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Print/return the text from a number of errors that were silent before.

@rootfs @andyzhangx @khendiak @feiskyer
2017-12-20 18:31:34 -08:00
Kubernetes Submit Queue 21cae16bda
Merge pull request #56742 from zouyee/patch-12
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

outdent err block

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```NONE

```
2017-12-20 16:47:34 -08:00
Brendan Burns da6cc5d7a5 Print/return the text from a number of errors that were silent before. 2017-12-21 00:25:46 +00:00
Sean McGinnis 69a2deea9c iscsi: set node.startup to manual
If the default iSCSI node.startup is set to automatic, if there is a node failure,
any pods on that node will get rescheduled to another node. If the failed node is
later brought back up it will then try to log back in to any iSCSI sessions it had
prior to the failure, which may no longer exist or may be now in-use by the other
nodes.

It appears most platforms keep the open-iscsi default of node.startup-automatic.
But in case this system-wide setting has been changed, and just to be explicit, this
sets node.startup values for kubernetes controlled volumes to manual.

Closes issue #21305
2017-12-20 15:37:22 -06:00
Kubernetes Submit Queue 7ede2a012e
Merge pull request #56600 from edisonxiang/fixvolumesize
Automatic merge from submit-queue (batch tested with PRs 56600, 56814). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix volume size allocation in gcd_pd

**What this PR does / why we need it**:
GCE PDs are allocated in chunks of GBs not GiB but CreateVolume function incorrectly creates volume in chunks of GiB.  
1 GiB = 1024 * 1024 * 1024 Bytes
1 GB = 1000 * 1000 * 1000 Bytes

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56081

**Special notes for your reviewer**:

**Release note**:

```release-note
Fixed dynamic provisioning of GCE PDs to round to the next GB instead of GiB
```
2017-12-19 14:08:38 -08:00
zouyee b6e1fecde7 check function return err 2017-12-19 23:24:35 +08:00
andyzhangx 8d99c55bba add Dong Liu as approver and add OWNERS in credentialprovider 2017-12-19 09:31:43 +00:00
Kubernetes Submit Queue 8d24ce1137
Merge pull request #57367 from feiskyer/owners
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add azure owners

**What this PR does / why we need it**:

Add azure owners and sort owner files.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

cc @khenidak @andyzhangx 

**Release note**:

```release-note
NONE
```

/assign @brendandburns
2017-12-18 21:47:08 -08:00
Kubernetes Submit Queue 5b55f614d0
Merge pull request #57260 from davidz627/attachMountLogFix
Automatic merge from submit-queue (batch tested with PRs 55475, 57155, 57260, 57222). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Improved mount/attach error logging and added attach event.

Fixed kubelet error message to be more descriptive. Added Attach success event for help in debugging.

The attach event is helpful when the node may not have the correct information about attachment status, it allows the user to see whether the Attach was run at all. If there is no success/failure attach message we can infer that there was no attach started at all.

Fixes #57217
2017-12-18 19:45:43 -08:00
Kubernetes Submit Queue 199b110f72
Merge pull request #57155 from mtanino/issue/57153
Automatic merge from submit-queue (batch tested with PRs 55475, 57155, 57260, 57222). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

FC plugin: Update detach logic for block volume if devicePath is empty

**What this PR does / why we need it**:
Currently, FC plugin doesn't have a chance to update attached volume's devicePath into volume object.
Therefore, at the volume teardown path, FC plugin gets empty devicePath from kubelet then fails volume detach operations especially volumeMode is Block.

This PR adds logic to obtain devicePath from global map path if passed devicePath from kubelet is empty.

I think this PR isn't complete solution but is a workaround to avoid teardown failures. In order to solve the root cause, we need to discuss the way to update devicePath at kubelet side again.(https://github.com/kubernetes/kubernetes/pull/54264)

If volume is managed by multipath, FC plugin fails to detach volume from kubelet node on both Filesystem and Block cases. This PR also fix the problem.

**Which issue(s) this PR fixes** : Fixes #57153

**Special notes for your reviewer**:
@rootfs @jsafrane 

**Release note**:

```
NONE
```
2017-12-18 19:45:41 -08:00
Pengfei Ni 477def0b4d Add azure owners 2017-12-19 11:42:02 +08:00
edisonxiang d9dc64227f using RoundUpToGB function directly 2017-12-19 09:32:15 +08:00
Kubernetes Submit Queue c0d005c832
Merge pull request #57293 from linyouchong/linyouchong-20171218
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix accessmode mapping error

What this PR does / why we need it:
fix accessmode mapping error in csi_client.go

`// can be mounted in read-only mode to many hosts`
`ReadOnlyMany PersistentVolumeAccessMode = "ReadOnlyMany"`

I think api.ReadOnlyMany should be mapping to csipb.VolumeCapability_AccessMode_MULTI_NODE_READER_ONLY

Which issue this PR fixes (optional, in fixes #(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
NONE

Special notes for your reviewer:
NONE

Release note:
NONE
2017-12-18 15:47:27 -08:00
Tim Hockin eba5b6092a Use k8s.gcr.io vanity domain for container images 2017-12-18 09:18:34 -08:00
linyouchong 5bc7e2212a fix accessmode mapping error 2017-12-18 17:51:38 +08:00
Kubernetes Submit Queue a73aee954f
Merge pull request #56872 from davidz627/logImprovement
Automatic merge from submit-queue (batch tested with PRs 56375, 56872, 57053, 57165, 57218). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Improved event generation in volume mount,attach, and extend operations

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #57217

refactored mount, attach, extend operation's so that all failures generate events and event generation is more consistent.

```release-note
Improved event generation in volume mount, attach, and extend operations
```
2017-12-17 05:33:41 -08:00
edisonxiang d80dbe7fea Fix GCE CreateVolume allocates in chunks of GiB incorrectly 2017-12-17 19:43:37 +08:00
Kubernetes Submit Queue a28e708ec6
Merge pull request #57091 from humblec/volume-mode
Automatic merge from submit-queue (batch tested with PRs 57148, 57123, 57091, 57141, 57131). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add VolumeMode in GlusterFS PV spec.

This ensures that GlusterFS pvs are file backed storage.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note

```
2017-12-16 21:00:46 -08:00
Kubernetes Submit Queue 5669c4c8bb
Merge pull request #57123 from linyouchong/linyouchong-20171213
Automatic merge from submit-queue (batch tested with PRs 57148, 57123, 57091, 57141, 57131). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix incorrect log in csi_plugin

What this PR does / why we need it:
fix incorrect log in csi_plugin_test.go and csi_mounter.go

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
NONE

Special notes for your reviewer:
NONE

Release note:
NONE
2017-12-16 21:00:44 -08:00
Kubernetes Submit Queue ea55d70ee7
Merge pull request #56921 from andyzhangx/flexvolume-windows
Automatic merge from submit-queue (batch tested with PRs 56650, 55813, 56911, 56921, 56871). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

enable flexvolume on Windows node

**What this PR does / why we need it**:
enable flexvolume on Windows node:
current PodVolumeDir is like 
`\var\lib\kubelet\pods\f54c5a74-da63-11e7-b71a-000d3a02c330\volumes\test~hostpath.cmd\flextest`
which is a unix path, with this PR, PodVolumeDir in **windows** kubelet  will be converted into like
`c:\var\lib\kubelet\pods\f54c5a74-da63-11e7-b71a-000d3a02c330\volumes\test~hostpath.cmd\flextest` 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56875

**Special notes for your reviewer**:
Detailed steps about how to use flexvolume on Windows with this PR:
https://github.com/andyzhangx/Demo/tree/master/windows/flexvolume

**Release note**:

```
enable flexvolume on Windows node
```

/sig windows
2017-12-16 12:10:51 -08:00
Kubernetes Submit Queue 88ae1ef52b
Merge pull request #56658 from wenjun93/master
Automatic merge from submit-queue (batch tested with PRs 56480, 56675, 56624, 56648, 56658). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

minor fix error typo of rbd volume teardown

fix typo in rbd volume plugin `diskTearDown`
2017-12-16 03:24:51 -08:00
Humble Chirammal 95bece666d Record volumeID in GlusterFS PV spec.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2017-12-16 10:36:14 +05:30
Kubernetes Submit Queue 691436ad64
Merge pull request #56581 from gnufied/fix-gluster-disk-sizes
Automatic merge from submit-queue (batch tested with PRs 56401, 56506, 56551, 56298, 56581). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Heketi documentats incorrectly about sizes in GBs

Heketi documentation incorrectly says that volume size is created in
GB but in fact is in GiB. Fix both resizing and create volume
functions to relfect that.

Fixes https://github.com/kubernetes/kubernetes/issues/56192

```release-note
Use GiB unit for creating and resizing volumes for Glusterfs
```
2017-12-15 20:29:39 -08:00
Kubernetes Submit Queue 3abbd6fb1f
Merge pull request #56298 from pospispa/566-improvements-suggested-by-thockin-during-review-of-PR55824
Automatic merge from submit-queue (batch tested with PRs 56401, 56506, 56551, 56298, 56581). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Addressing Comments from Code Review

**What this PR does / why we need it**: addressing comments from code review: https://github.com/kubernetes/kubernetes/pull/55824#pullrequestreview-78597250

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: N/A


**Special notes for your reviewer**:
@thockin @jsafrane @msau42 PTAL

**Release note**:

```release-note
NONE
```
2017-12-15 20:29:36 -08:00
Kubernetes Submit Queue cf582af090
Merge pull request #56551 from andyzhangx/default-dir-file-mode
Automatic merge from submit-queue (batch tested with PRs 56401, 56506, 56551, 56298, 56581). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

change default azure file/dir mode to 0755

**What this PR does / why we need it**:
change default azure file/dir mode to 0755, users complain current `0700` value a lot.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #54610

**Special notes for your reviewer**:

**Release note**:

```
none
```
/sig azure
/assign @rootfs
2017-12-15 20:29:33 -08:00
Kubernetes Submit Queue e9e190c57c
Merge pull request #56487 from zhangxiaoyu-zidif/fix-binary-check-glusterfs
Automatic merge from submit-queue (batch tested with PRs 56413, 56322, 56490, 56460, 56487). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix binary check for glusterfs.go

**What this PR does / why we need it**:
err check can't show that the binary does not exist.
If the binary does not exist, the err still be nil ever.
we should use "test -x" to check if the binary exist.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56479

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-12-15 16:43:57 -08:00
Kubernetes Submit Queue d9b45d08c0
Merge pull request #56460 from liggitt/flex-pv-secret
Automatic merge from submit-queue (batch tested with PRs 56413, 56322, 56490, 56460, 56487). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Allow FlexVolume PV secret namespaces

Completes the secret namespace PV refactor, so all PV volume sources that specify secrets can reference them outside the PVC namespace.

Finished the secret-related aspect of https://github.com/kubernetes/kubernetes/issues/32131

```release-note
PersistentVolume flexVolume sources can now reference secrets in a namespace other than the PersistentVolumeClaim's namespace.
```
2017-12-15 16:43:55 -08:00
David Zhu fffd152e0d Fixed kubelet error message to be more descriptive. Added Attach success event for help in debugging. 2017-12-15 15:36:59 -08:00
mtanino c8fffea765 Autogenerated code 2017-12-15 11:56:49 -05:00
mtanino 1443b1bd1f Update detach logic for block volume if devicePath is empty 2017-12-15 11:56:14 -05:00
Kubernetes Submit Queue bf5f96d64a
Merge pull request #56147 from andyzhangx/mountoptions-azuredisk
Automatic merge from submit-queue (batch tested with PRs 57172, 55382, 56147, 56146, 56158). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add mount options support for azure disk

**What this PR does / why we need it**:
add mount options support for azure disk

example azure class with mount options:
```
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: hdd
provisioner: kubernetes.io/azure-disk
mountOptions:
  - barrier=1
  - acl
parameters:
  skuname: Standard_LRS
  kind: Managed
  fstype: ext3
```

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56131

**Special notes for your reviewer**:

**Release note**:

```
add mount options support for azure disk
```
/sig azure
/assign @rootfs
2017-12-14 12:38:15 -08:00
David Zhu e3f8f64c17 refactored mount, attach, resize operation's so that all failures generate events and event generation is more consistent.
refactored operation generator and operation executor to use more general generated functions for operations, completions, and events.
2017-12-14 11:09:12 -08:00