Commit Graph

1611 Commits (d336607679743a53ed93edbebf89f45dd957e46d)

Author SHA1 Message Date
Humble Chirammal 64e8111f75 Correct error strings and variable name.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2018-02-14 23:19:19 +05:30
Kubernetes Submit Queue d89e64110a
Merge pull request #59716 from feiskyer/vmss-disk
Automatic merge from submit-queue (batch tested with PRs 59489, 59716). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add AzureDisk support for vmss nodes

**What this PR does / why we need it**:

This PR adds AzureDisk support for vmss nodes. Changes include

- Upgrade vmss API to 2017-12-01
- Upgrade vmss clients with new version API
- Abstract AzureDisk operations for vmss and vmas
- Added AzureDisk support for vmss
- Unit tests and fake clients fix
 
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #43287

**Special notes for your reviewer**:

~~Depending on #59652 (the first two commits are from #59652).~~

**Release note**:

```release-note
Add AzureDisk support for vmss nodes
```
2018-02-14 00:14:34 -08:00
Kubernetes Submit Queue 4afdc33c57
Merge pull request #56454 from mtanino/volumehandler-refactor
Automatic merge from submit-queue (batch tested with PRs 59767, 56454, 59237, 59730, 55479). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Block Volume: Refactor volumehandler in operationexecutor

**What this PR does / why we need it**:

Based on discussion with @saad-ali at #51494, we need refactor volumehandler in operationexecutor
for Block Volume feature. We don't need to add volumehandler as separated object.

```
VolumeHandler does not need to be a separate object that is constructed inline like this. 
You can create a new operation, e.g. UnmountOperation to which you pass the spec,
and it can return either a UnmountVolume or UnmapVolume.
```

**Which issue(s) this PR fixes** : no related issue.

**Special notes for your reviewer**:

@saad-ali @msau42 

**Release note**:

```release-note
NONE
```
2018-02-12 15:44:33 -08:00
Kubernetes Submit Queue 66ccfcb4c7
Merge pull request #59640 from jianglingxia/jlx-2018020917
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

correct the ConstructVolumeSpec func path value

**What this PR does / why we need it**:
the path value is incorrect
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-02-12 07:15:13 -08:00
Pengfei Ni 6dcd56549a Use full instanceID as lun lock key
This is because the last part of VMSS instances are numbers, which may
be same if there are multiple VMSS within same cluster.
2018-02-12 14:07:05 +08:00
Kubernetes Submit Queue cfa6774540
Merge pull request #56557 from andyzhangx/azurefile-createaccount
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix the create azure file pvc failure if there is no storage account in current resource group

**What this PR does / why we need it**:
When create an azure file PVC, there will be error if there is no storage account in current resource group.
With this PR, a storage account will be created if there is no storage account in current resource group.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56556

**Special notes for your reviewer**:
1. rephrase the code logic of `CreateFileShare` func.
```
if accountName is empty, then 
    find a storage account that matches accountType
    if no storage account found, then
        create a new account
else
    we only use user specified storage account

create a file share according to found storage account
```
2. Use func `getStorageAccountName` to get a unique storage account name by UUID, a storage account for azure file would be like `f0b2b0bd40c010112e897fa`. And in next PR, I will use this function to create storage account for azure disk, the storage account for azure disk would be like `d8f3ad8ad92000f1e1e88bd`.

**Release note**:

```
fix the create azure file pvc failure if there is no storage account in current resource group
```
/sig azure
/assign @rootfs
2018-02-11 05:03:32 -08:00
andyzhangx aa21bef677 create storage account if necessary when create azure file pvc
use new storage account name generation method

use uuid to generate account name

change azure file account prefix

use uniqueID to generate a storage account name

fix comments

fix comments

fix comments

fix a storage account matching bug

only use UUID in getStorageAccountName func

use shorter storage account prefix for azure file

fix comments

fix comments

fix comments

fix rebase build error

rewrite CreateFileShare code logic

fix gofmt issue

fix test error

fix comments

fix a location matching bug
2018-02-11 06:29:05 +00:00
Kubernetes Submit Queue 317853c90c
Merge pull request #59464 from dixudx/fix_all_typos
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix all the typos across the project

**What this PR does / why we need it**:
There are lots of typos across the project. We should avoid small PRs on fixing those annoying typos, which is time-consuming and low efficient.

This PR does fix all the typos across the project currently. And with #59463, typos could be avoided when a new PR gets merged.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
/sig testing
/area test-infra
/sig release
/cc @ixdy 
/assign @fejta 

**Release note**:

```release-note
None
```
2018-02-10 22:12:45 -08:00
Davanum Srinivas 7ef4f207c2 New github id - FengyunPan -> FengyunPan2
PanFengyun <pan_feng_yun@163.com>'s previous github id was @FengyunPan

Due to some problem with github, he lost access to @FengyunPan and
is not using @FengyunPan2. So let's switch over to the new id. Github
has promised to release the previous id back in 6 months, so we may
have to switch it back later.
2018-02-10 22:28:51 -05:00
Di Xu 48388fec7e fix all the typos across the project 2018-02-11 11:04:14 +08:00
mtanino bc86537f18 Autogenerated files 2018-02-09 15:41:15 -05:00
mtanino 973583e781 Refactor volumehandler in operationexecutor 2018-02-09 15:39:55 -05:00
Kubernetes Submit Queue a35acccaf8
Merge pull request #59447 from verult/pd-alpha
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fixes the regression of GCEPD not provisioning correctly on GKE alpha clusters.

Fixes the regression by better distinguishing between single-zone and multi-zone PDs.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #59047

**Special notes for your reviewer**: All PD and dynamic provisioning e2e tests pass, manually verified provision, delete, attach, and detach of both single-zone and multi-zone PDs. Will create e2e tests for multizone PDs in a separate PR.

**Release note**:


/sig storage
/assign @saad-ali
2018-02-09 11:55:10 -08:00
Kubernetes Submit Queue 977d03bd8b
Merge pull request #59232 from liubin/fix4
Automatic merge from submit-queue (batch tested with PRs 59607, 59232). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Make log content more information

And change some `fmt.Errorf` to `fmt.Error`
2018-02-09 08:54:34 -08:00
Kubernetes Submit Queue afaabe05f7
Merge pull request #59607 from harsh-px/px-pvc
Automatic merge from submit-queue (batch tested with PRs 59607, 59232). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Pass pvc namespace and annotations to Portworx Create API

Signed-off-by: Harsh Desai <harsh@portworx.com>

**What this PR does / why we need it**:

Fixes #59606

**Release note**:

```release-note
NONE
```
2018-02-09 08:54:30 -08:00
Kubernetes Submit Queue aee2cff1b8
Merge pull request #59548 from linyouchong/lyc-201802082
Automatic merge from submit-queue (batch tested with PRs 59466, 58912, 59605, 59548). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix incorrect logic in canSupport

**What this PR does / why we need it**:
if `spec.PersistentVolume` is nil or `spec.Volume` is nil, func `canSupport` should return false.

**Release note**:
```
NONE
```
/release-note-none
/sig storage
2018-02-09 03:44:41 -08:00
Kubernetes Submit Queue 8268f6d5e7
Merge pull request #59466 from humblec/hide-configuration
Automatic merge from submit-queue (batch tested with PRs 59466, 58912, 59605, 59548). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Gluster: Remove provisioner configuration from info message.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note

```
2018-02-09 03:44:32 -08:00
jianglingxia 2d805c727e correct the ConstructVolumeSpec func path value 2018-02-09 17:29:24 +08:00
linyouchong e92f6eb0fe fix incorrect logic in canSupport 2018-02-09 11:45:15 +08:00
Kubernetes Submit Queue d6625f857a
Merge pull request #58177 from jingxu97/Jan/reconstruct
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Redesign and implement volume reconstruction work

This PR is the first part of redesign of volume reconstruction work. The detailed design information is https://github.com/kubernetes/community/pull/1601

The changes include
1. Remove dependency on volume spec stored in actual state for volume
cleanup process (UnmountVolume and UnmountDevice)

Modify AttachedVolume struct to add DeviceMountPath so that volume
unmount operation can use this information instead of constructing from
volume spec

2. Modify reconciler's volume reconstruction process (syncState). Currently workflow
is when kubelet restarts, syncState() is only called once before
reconciler starts its loop.
a. If volume plugin supports reconstruction, it will use the
reconstructed volume spec information to update actual state as before.
b. If volume plugin cannot support reconstruction, it will use the
scanned mount path information to clean up the mounts.

In this PR, all the plugins still support reconstruction (except
glusterfs), so reconstruction of some plugins will still have issues.
The next PR will modify those plugins that cannot support reconstruction
well.

This PR addresses issue #52683
2018-02-08 18:21:34 -08:00
Harsh Desai 79ea511ec6 Pass pvc namespace and annotations to Portworx Create API
Closes #59606

Signed-off-by: Harsh Desai <harsh@portworx.com>
2018-02-08 17:46:08 -08:00
Cheng Xing 02352460f6 Fixes the regression of GCEPD not provisioning correctly on alpha clusters. 2018-02-08 10:46:06 -08:00
Kubernetes Submit Queue fb340a4695
Merge pull request #57824 from thockin/gcr-vanity
Automatic merge from submit-queue (batch tested with PRs 57824, 58806, 59410, 59280). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

2nd try at using a vanity GCR name

The 2nd commit here is the changes relative to the reverted PR.  Please focus review attention on that.

This is the 2nd attempt.  The previous try (#57573) was reverted while we
figured out the regional mirrors (oops).
    
New plan: k8s.gcr.io is a read-only facade that auto-detects your source
region (us, eu, or asia for now) and pulls from the closest.  To publish
an image, push k8s-staging.gcr.io and it will be synced to the regionals
automatically (similar to today).  For now the staging is an alias to
gcr.io/google_containers (the legacy URL).
    
When we move off of google-owned projects (working on it), then we just
do a one-time sync, and change the google-internal config, and nobody
outside should notice.
    
We can, in parallel, change the auto-sync into a manual sync - send a PR
to "promote" something from staging, and a bot activates it.  Nice and
visible, easy to keep track of.

xref https://github.com/kubernetes/release/issues/281

TL;DR:
  *  The new `staging-k8s.gcr.io` is where we push images.  It is literally an alias to `gcr.io/google_containers` (the existing repo) and is hosted in the US.
  * The contents of `staging-k8s.gcr.io` are automatically synced to `{asia,eu,us)-k8s.gcr.io`.
  * The new `k8s.gcr.io` will be a read-only alias to whichever regional repo is closest to you.
  * In the future, images will be promoted from `staging` to regional "prod" more explicitly and auditably.

 ```release-note
Use "k8s.gcr.io" for pulling container images rather than "gcr.io/google_containers".  Images are already synced, so this should not impact anyone materially.
    
Documentation and tools should all convert to the new name. Users should take note of this in case they see this new name in the system.
```
2018-02-08 03:29:32 -08:00
Tim Hockin 3586986416 Switch to k8s.gcr.io vanity domain
This is the 2nd attempt.  The previous was reverted while we figured out
the regional mirrors (oops).

New plan: k8s.gcr.io is a read-only facade that auto-detects your source
region (us, eu, or asia for now) and pulls from the closest.  To publish
an image, push k8s-staging.gcr.io and it will be synced to the regionals
automatically (similar to today).  For now the staging is an alias to
gcr.io/google_containers (the legacy URL).

When we move off of google-owned projects (working on it), then we just
do a one-time sync, and change the google-internal config, and nobody
outside should notice.

We can, in parallel, change the auto-sync into a manual sync - send a PR
to "promote" something from staging, and a bot activates it.  Nice and
visible, easy to keep track of.
2018-02-07 21:14:19 -08:00
Kubernetes Submit Queue c37b5d757d
Merge pull request #59303 from dhirajh/localblock
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Block Volume Support: Local Volume Plugin update

**What this PR does / why we need it**:

Introduce block volume support to local volumes plugin.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #59500

**Special notes for your reviewer**:
@msau42 @mtanino @ianchakeres 

Adding support for block volumes as per https://github.com/kubernetes/features/issues/121

Other related PRs:
(#50457) API Change
(#53385) VolumeMode PV-PVC Binding change
(#51494) Container runtime interface change, volumemanager changes, operationexecutor changes

**Release note**:
```
Added support for Block Volume type to local-volume plugin.
```
2018-02-07 13:27:50 -08:00
Kubernetes Submit Queue 117780b908
Merge pull request #58287 from dg-i/ceph-provisioner-userid
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add keyring parameter in Ceph RBD provisioner

**What this PR does / why we need it**:
userSecretName is a mandatory parameter for the kubernetes.io/rbd storage provisioner. Ceph RBD persistent volumes however don't need the secretRef. If no secret is given, a local keyring file (/etc/ceph/keyring) on the node will be used. This behaviour should also work with the storage provisioner. If no userSecretName parameter is given, no secretRef should be created in the resulting persistent volume.

**Special notes for your reviewer**:

**Release note**:
```release-note
Add "keyring" parameter for Ceph RBD provisioner
```
2018-02-07 08:00:28 -08:00
Kubernetes Submit Queue a1437feb18
Merge pull request #59400 from jianglingxia/jlx-2018020614
Automatic merge from submit-queue (batch tested with PRs 57230, 59400). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

make sure mounter not nil and fix some typo

**What this PR does / why we need it**:
make sure mounter not nil  then and fix some typo
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-02-07 02:58:31 -08:00
Mathias Merscher dbf5f58022
add keyring parameter in Ceph RBD provisioner 2018-02-07 11:50:49 +01:00
Humble Chirammal 922766a7cf Remove provisioner configuration from info message.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2018-02-07 13:30:42 +05:30
Kubernetes Submit Queue 11104d75f1
Merge pull request #57194 from linyouchong/linyouchong-20171214
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix TODO:change to a api-server watch

What this PR does / why we need it:
fix TODO (vladimirvivien) instead of polling api-server, change to a api-server watch

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #58031

Special notes for your reviewer:

Release note:
NONE

@vladimirvivien,@jsafrane,@saad-ali
I saw some TODO there and I am very interested in fixing it.
Please let me know if it is not inappropriate to do this.
2018-02-06 23:35:13 -08:00
Kubernetes Submit Queue e5b6026db6
Merge pull request #59287 from cheftako/cloud-context-level
Automatic merge from submit-queue (batch tested with PRs 59441, 58264, 59287, 59396, 59439). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add context to all relevant cloud APIs

**What this PR does / why we need it**:

This adds context to all the relevant cloud provider interface signatures.
Callers of those APIs are currently satisfied using context.TODO().
There will be follow on PRs to push the context through the stack.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #815

**Special notes for your reviewer**:
For an idea of the full scope of this change please look at PR #58532.

**Release note**:
```release-note
Implementers of the cloud provider interface will note the addition of a context to this interface. Trivial code modification will be necessary for a cloud provider to continue to compile.
```
2018-02-06 20:27:39 -08:00
Kubernetes Submit Queue e740fe68c5
Merge pull request #58264 from WanLinghao/log_fix
Automatic merge from submit-queue (batch tested with PRs 59441, 58264, 59287, 59396, 59439). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

	fix some log param error

this patch fix some log parameter mistakes.



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note 
/release-note-none
```
2018-02-06 20:27:36 -08:00
bin liu 053e7c9cfd Change log format: replace () to [] 2018-02-07 11:05:39 +08:00
Kubernetes Submit Queue 3d4eaf7307
Merge pull request #57092 from linyouchong/linyouchong-20171212
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix TODO: moving Driver name check in API validation

**What this PR does / why we need it**:
fix TODO in csi_plugin.go ,moving Driver name check in API validation
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #58163

**Special notes for your reviewer**:

**Release note**:
NONE

@vladimirvivien,@jsafrane,@saad-ali
I saw some TODO there and I am very interested in fixing it.
Please let me know if it is not  inappropriate to do this.
2018-02-06 18:49:36 -08:00
Dhiraj f92e13f01e Removed unnecessary test code. 2018-02-06 18:38:52 -08:00
linyouchong efb68ddf73 fix TODO: moving driver name check in API validation 2018-02-07 09:03:47 +08:00
Dhiraj 8b8b85604a Initial local PV block device plugin checkin.
Unit tests for block.

Add docs/test

Fix gofmt issues.

Address code review comments.

Remove evalsymlink fro setupDevice()

Address review comments.
2018-02-06 14:49:44 -08:00
Walter Fender e18e8ec3c0 Add context to all relevant cloud APIs
This adds context to all the relevant cloud provider interface signatures.
Callers of those APIs are currently satisfied using context.TODO().
There will be follow on PRs to push the context through the stack.
For an idea of the full scope of this change please look at PR #58532.
2018-02-06 12:49:17 -08:00
Kubernetes Submit Queue 4bd22b5467
Merge pull request #58415 from gnufied/fix-volume-resize-messages
Automatic merge from submit-queue (batch tested with PRs 52942, 58415). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Improve messaging on volume expansion

- we now provide clear message to user what to do when cloudprovider resizing is finished
  and file system resizing is needed.
- add a event when resizing is successful
- Use PATCH both in controller-manager and kubelet for updating PVC status
- Remove code duplication between controller-manager and kubelet for updating PVC status
- Only remove conditions that are managed by resize controller



```release-note
Improve messages user gets during and after volume resizing is done.
```
2018-02-06 07:55:32 -08:00
Kubernetes Submit Queue 9a8bb0098b
Merge pull request #59388 from jianglingxia/jlx-020610
Automatic merge from submit-queue (batch tested with PRs 54680, 59388, 57455). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix TestPlugin func has two same if code/kind bug

**What this PR does / why we need it**:
Fix TestPlugin func has two same if code
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-02-06 06:40:31 -08:00
jianglingxia 763fd53adf make sure mounter not nil and fix some typo 2018-02-06 18:28:07 +08:00
Kubernetes Submit Queue d9632f66af
Merge pull request #59328 from vmware/vSphereVolumeMetricsProvider
Automatic merge from submit-queue (batch tested with PRs 58317, 58687, 57872, 59063, 59328). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Volume metrics support for vSphere Cloud Provider

**What this PR does / why we need it**:
This PR adds Volume metrics support for vSphere Cloud Provider. With this we can monitor the available space, capacity and used space on volumes created using vSphere. 

**Which issue(s) this PR fixes**
Fixes # https://github.com/kubernetes/kubernetes/issues/43527
Fixes # https://github.com/vmware/kubernetes/issues/208

**Special notes for your reviewer**:
Performed following steps to ensure metrics are reported correctly for vSphere Volumes.

- Created Dynamic PV using the PV claim of size 1Gi.
```
$ kubectl get pvc
NAME        STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
thinclaim   Bound     pvc-bbc18f6a-0a04-11e8-bb1c-0050569c126b   1Gi        RWO            thin           5m
```

- Created Pod using this claim.
- Verified volume stats metrics.

```
/ # df -h | grep /dev/sdb
Filesystem                Size      Used Available Use% Mounted on
/dev/sdb                975.9M      1.3M    923.4M   0% /mnt/volume1
```

```
# HELP kubelet_volume_stats_available_bytes Number of available bytes in the volume
# TYPE kubelet_volume_stats_available_bytes gauge
kubelet_volume_stats_available_bytes{namespace="default",persistentvolumeclaim="thinclaim"} 9.68261632e+08

# HELP kubelet_volume_stats_capacity_bytes Capacity in bytes of the volume
# TYPE kubelet_volume_stats_capacity_bytes gauge
kubelet_volume_stats_capacity_bytes{namespace="default",persistentvolumeclaim="thinclaim"} 1.02330368e+09

# HELP kubelet_volume_stats_used_bytes Number of used bytes in the volume
# TYPE kubelet_volume_stats_used_bytes gauge
kubelet_volume_stats_used_bytes{namespace="default",persistentvolumeclaim="thinclaim"} 1.355776e+06
```

- Download 512MB file in the Pod on attached volume and Verified volume stats metrics is updated successfully.

```
/mnt/volume1 # df -h | grep /dev/sdb
Filesystem                Size      Used Available Use% Mounted on
/dev/sdb                975.9M    513.8M    410.9M  56% /mnt/volume1
```
```
# HELP kubelet_volume_stats_available_bytes Number of available bytes in the volume
# TYPE kubelet_volume_stats_available_bytes gauge
kubelet_volume_stats_available_bytes{namespace="default",persistentvolumeclaim="thinclaim"} 4.30862336e+08

# HELP kubelet_volume_stats_capacity_bytes Capacity in bytes of the volume
# TYPE kubelet_volume_stats_capacity_bytes gauge
kubelet_volume_stats_capacity_bytes{namespace="default",persistentvolumeclaim="thinclaim"} 1.02330368e+09

# HELP kubelet_volume_stats_used_bytes Number of used bytes in the volume
# TYPE kubelet_volume_stats_used_bytes gauge
kubelet_volume_stats_used_bytes{namespace="default",persistentvolumeclaim="thinclaim"} 5.38755072e+08
```

**Release note**:

```release-note
Volume metrics support for vSphere Cloud Provider
```
2018-02-05 18:43:43 -08:00
jianglingxia a5865ddd7b Fix TestPlugin func has two same if code/kind bug 2018-02-06 10:29:28 +08:00
Jing Xu 9588d2098a Redesign and implement volume reconstruction work
This PR is the first part of redesign of volume reconstruction work. The
changes include
1. Remove dependency on volume spec stored in actual state for volume
cleanup process (UnmountVolume and UnmountDevice)

Modify AttachedVolume struct to add DeviceMountPath so that volume
unmount operation can use this information instead of constructing from
volume spec

2. Modify reconciler's volume reconstruction process (syncState). Currently workflow
is when kubelet restarts, syncState() is only called once before
reconciler starts its loop.
a. If volume plugin supports reconstruction, it will use the
reconstructed volume spec information to update actual state as before.
b. If volume plugin cannot support reconstruction, it will use the
scanned mount path information to clean up the mounts.

In this PR, all the plugins still support reconstruction (except
glusterfs), so reconstruction of some plugins will still have issues.
The next PR will modify those plugins that cannot support reconstruction
well.

This PR addresses issue #52683, #54108 (This PR includes the changes to
update devicePath after local attach finishes)
2018-02-05 13:14:09 -08:00
Kubernetes Submit Queue ffda1e2200
Merge pull request #57017 from andyzhangx/azurefile-growsize
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add PV size grow feature for azure file

**What this PR does / why we need it**:
According to kubernetes/features#284, add size grow feature for azure file

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56462 

**Special notes for your reviewer**:
Since azure file is using SMB 3.0 protocal, there is no necessary to resize filesystem on agent side, the agent node will detect the changed size automatically.

**Release note**:

```
add size grow feature for azure file
```
/sig azure
@gnufied @rootfs @brendandburns
2018-02-05 11:25:48 -08:00
Divyen Patel 46edaa7269 Adding volume metrics support for vSphere Cloud Provider 2018-02-04 16:10:15 -08:00
Davanum Srinivas 73b46ff7db Fix golint for openstack and cinder packages 2018-02-04 17:15:32 -05:00
Kubernetes Submit Queue a8581f41e6
Merge pull request #59097 from zhangxiaoyu-zidif/delete-redundant-get-volumesource
Automatic merge from submit-queue (batch tested with PRs 59097, 57076, 59295). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

delete duplicate function for getting volume source

**What this PR does / why we need it**:
these two methods have same function.
```go
func getVolumeSource(
	spec *volume.Spec) (*v1.GlusterfsVolumeSource, bool, error) {
	if spec.Volume != nil && spec.Volume.Glusterfs != nil {
		return spec.Volume.Glusterfs, spec.Volume.Glusterfs.ReadOnly, nil
	} else if spec.PersistentVolume != nil &&
		spec.PersistentVolume.Spec.Glusterfs != nil {
		return spec.PersistentVolume.Spec.Glusterfs, spec.ReadOnly, nil
	}

	return nil, false, fmt.Errorf("Spec does not reference a Glusterfs volume type")
}

func (plugin *glusterfsPlugin) getGlusterVolumeSource(spec *volume.Spec) (*v1.GlusterfsVolumeSource, bool) {
	// Glusterfs volumes used directly in a pod have a ReadOnly flag set by the pod author.
	// Glusterfs volumes used as a PersistentVolume gets the ReadOnly flag indirectly through the persistent-claim volume used to mount the PV
	if spec.Volume != nil && spec.Volume.Glusterfs != nil {
		return spec.Volume.Glusterfs, spec.Volume.Glusterfs.ReadOnly
	}
	return spec.PersistentVolume.Spec.Glusterfs, spec.ReadOnly
}
```

`getVolumeSource`  seems to be much better.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-02-02 19:37:34 -08:00
bin liu e44779d86f Reformat log to show more details 2018-02-03 11:01:54 +08:00
Kubernetes Submit Queue 49ac543a56
Merge pull request #59072 from sbezverk/fc_iscsi_loopback_cleanup
Automatic merge from submit-queue (batch tested with PRs 55606, 59185, 58763, 59072, 59251). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Cleaning up loopback removal process

Handling more graceful the process of loopback removal. In case loopback device does not exist any longer, we should not fail the detachment process. 
```release-note

```
2018-02-02 18:26:41 -08:00