Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add generic cache for Azure VMSS
**What this PR does / why we need it**:
This PR adds a generic cache for VMSS and removes old list-based cache.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Continue of ##58770.
**Special notes for your reviewer**:
Depends on #59520.
**Release note**:
```release-note
Add generic cache for Azure VMSS
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix the create azure file pvc failure if there is no storage account in current resource group
**What this PR does / why we need it**:
When create an azure file PVC, there will be error if there is no storage account in current resource group.
With this PR, a storage account will be created if there is no storage account in current resource group.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56556
**Special notes for your reviewer**:
1. rephrase the code logic of `CreateFileShare` func.
```
if accountName is empty, then
find a storage account that matches accountType
if no storage account found, then
create a new account
else
we only use user specified storage account
create a file share according to found storage account
```
2. Use func `getStorageAccountName` to get a unique storage account name by UUID, a storage account for azure file would be like `f0b2b0bd40c010112e897fa`. And in next PR, I will use this function to create storage account for azure disk, the storage account for azure disk would be like `d8f3ad8ad92000f1e1e88bd`.
**Release note**:
```
fix the create azure file pvc failure if there is no storage account in current resource group
```
/sig azure
/assign @rootfs
use new storage account name generation method
use uuid to generate account name
change azure file account prefix
use uniqueID to generate a storage account name
fix comments
fix comments
fix comments
fix a storage account matching bug
only use UUID in getStorageAccountName func
use shorter storage account prefix for azure file
fix comments
fix comments
fix comments
fix rebase build error
rewrite CreateFileShare code logic
fix gofmt issue
fix test error
fix comments
fix a location matching bug
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix all the typos across the project
**What this PR does / why we need it**:
There are lots of typos across the project. We should avoid small PRs on fixing those annoying typos, which is time-consuming and low efficient.
This PR does fix all the typos across the project currently. And with #59463, typos could be avoided when a new PR gets merged.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
/sig testing
/area test-infra
/sig release
/cc @ixdy
/assign @fejta
**Release note**:
```release-note
None
```
PanFengyun <pan_feng_yun@163.com>'s previous github id was @FengyunPan
Due to some problem with github, he lost access to @FengyunPan and
is not using @FengyunPan2. So let's switch over to the new id. Github
has promised to release the previous id back in 6 months, so we may
have to switch it back later.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add generic cache for Azure VM/LB/NSG/RouteTable
**What this PR does / why we need it**:
Part of #58770. This PR adds a generic cache of Azure VM/LB/NSG/RouteTable for reducing ARM calls.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of #58770
**Special notes for your reviewer**:
**Release note**:
```release-note
Add generic cache for Azure VM/LB/NSG/RouteTable
```
fix function
fix gofmt
fix function return value
fix tests
skip notimplemented error
remove factory unused
in openstack we should try to find instanceid from all states instead of ACTIVE, all other cloudproviders do this already
fix tests and lint
fix gofmt
fix nodelifecycletest
fix lint errors
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixes the regression of GCEPD not provisioning correctly on GKE alpha clusters.
Fixes the regression by better distinguishing between single-zone and multi-zone PDs.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59047
**Special notes for your reviewer**: All PD and dynamic provisioning e2e tests pass, manually verified provision, delete, attach, and detach of both single-zone and multi-zone PDs. Will create e2e tests for multizone PDs in a separate PR.
**Release note**:
/sig storage
/assign @saad-ali
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update azure_loadbalancer.md to fix typo
fix typo
incase -> in case
selction -> selection
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
fix typo
incase -> in case
selction -> selection
```
Automatic merge from submit-queue (batch tested with PRs 58437, 59490, 55684). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[GCE] Unit test ExternalLoadBalancer
**What this PR does / why we need it**:
- Unit tests for `pkg/cloudprovider/providers/gce/gce_loadbalancer_external.go`
- Tests creating, updating, and deleting an external LoadBalancer
**Future Improvements**
In order to further test `gce_loadbalancer_external.go`, we should add tests for the following cases:
- Network Tiers - when the current/desired network tier doesn't match, existing resources with the wrong tier should be torn down
- Expect an error when the TargetPool does not exist
- Expect an error when the LoadBalancer Firewall does not exist
- Case when TargetPool needs to be recreated
- Case when IP needs to be released (calls gce.DeleteRegionAddress)
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Make AWS attach/detach operations faster
Most attach/detach operations on AWS finish within 1-4seconds.
By using a shorter time interval and higher exponetial
factor we can shorten time taken for attach and detach to complete.
After this change retry interval looks like:
```
[1, 1.8, 3.24, 5.832000000000001, 10.4976]
```
Before it was:
```
[10, 12.0, 14.399999999999999, 17.279999999999998]
```
/sig aws
```release-note
AWS: Make attach/detach operations faster. from 10-12s to 2-6s
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
AWS: Do not ignore errors from EC2::DescribeVolume in DetachDisk
The DetachDisk method of AWS cloudprovider indirectly calls
EC2::DescribeVolume AWS API function to check if the volume
being detached is really attached to the specified node.
The AWS API call may fail and return error which is logged however
the DetachDisk then finishes successfully. This may cause the AWS
volumes to remain attached to the instances forever because the
attach/detach controller will mark the volume as attached. The PV
controller will never be able to delete those disks and they need
to be detached manually.
This patch ensures the error from DescribeVolume is propagated to
attach/detach controller and the detach operation is re-tried.
cc: @gnufied, @jsafrane
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
vclib: enable VM disk attach test
**What this PR does / why we need it**:
Follow up to PR #58534 , where this test was disabled due to a limitation in
govmomi/simulator. The test passes as expected with godeps update of govmomi.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
This PR is 1-line update to the vSphere Cloud Provider tests and godep update of the vendor'd vmware/govmomi repo.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Report InstanceID for vSphere Cloud Provider as UUID obtained from product_serial file
**What this PR does / why we need it**:
vSphere Cloud Provider is not able to find the nodes for VMs created on vSphere v1.6.5. Kubelet fetches SystemUUID from file ```/sys/class/dmi/id/product_uuid```. vSphere Cloud Provider uses this uuid as VM identifier to get node information from vCenter. vCenter v1.6.5 doesn't recognize this uuids, as a result, nodes are not found.
UUID present in file ```/sys/class/dmi/id/product_serial``` is recognized by vCenter. Yet, Kubelet doesn't report this. Therefore, in this PR InstanceID is reported as UUID which is fetched from file
```/sys/class/dmi/id/product_serial```.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubernetes/issues/58927
**Special notes for your reviewer**:
Internally review here: https://github.com/vmware/kubernetes/pull/452
Tested:
Launched K8s cluster using kubeadm (Used Ubuntu VM compatible with vSphere version 6.5.)
_**Note: Installed Ubuntu from ISO**_
Observed following:
```
Master
> cat /sys/class/dmi/id/product_uuid
743F0E42-84EA-A2F9-7736-6106BB5DBF6B
> cat /sys/class/dmi/id/product_serial
VMware-42 0e 3f 74 ea 84 f9 a2-77 36 61 06 bb 5d bf 6b
Node
> cat /sys/class/dmi/id/product_uuid
956E0E42-CC9D-3D89-9757-F27CEB539B76
> cat /sys/class/dmi/id/product_serial
VMware-42 0e 6e 95 9d cc 89 3d-97 57 f2 7c eb 53 9b 76
```
With this fix controller manager was able to find the nodes.
**controller manager logs**
```
{"log":"I0205 22:43:00.106416 1 nodemanager.go:183] Found node ubuntu-node as vm=VirtualMachine:vm-95 in vc=10.161.120.115 and datacenter=vcqaDC\n","stream":"stderr","time":"2018-02-05T22:43:00.421010375Z"}
```
**Release note**:
```release-note
vSphere Cloud Provider supports VMs provisioned on vSphere v1.6.5
```
TestRoutes assumes that there is at least one vm in the OpenStack it
is connecting to. So let's limit this test to run properly only when
we are running in a VM or one was created already outside of the
test harness
The DetachDisk method of AWS cloudprovider indirectly calls
EC2::DescribeVolume AWS API function to check if the volume
being detached is really attached to the specified node.
The AWS API call may fail and return error which is logged however
the DetachDisk then finishes successfully. This may cause the AWS
volumes to remain attached to the instances forever because the
attach/detach controller will mark the volume as attached. The PV
controller will never be able to delete those disks and they need
to be detached manually.
This patch ensures on error from DescribeVolume is propagated to
attach/detach controller and the detach operation is re-tried.
shutdowned -> stopped
use shutdown everywhere
use patch in taints api call
use notimplemented in clouds use AddOrUpdateTaintOnNode
correct log text
add fake cloud
try to fix bazel
add shutdown tests
add context
Follow up to PR #58534 , where this test was disabled due to a limitation in
govmomi/simulator. The test passes as expected with godeps update of govmomi.
Automatic merge from submit-queue (batch tested with PRs 59394, 58769, 59423, 59363, 59245). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Only populate alias range for nic0 when invoking instance.UpdateNetworkInterface.
**What this PR does / why we need it**:
Without the fix, GCP will emit errors, e.g., googleapi: Error 400: Invalid value for field 'resource.accessConfigs': ''. Updating this field is not supported., invalid
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
N/A
**Special notes for your reviewer**:
**Release note**:
```release-note
"NONE"
```
Automatic merge from submit-queue (batch tested with PRs 59441, 58264, 59287, 59396, 59439). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add context to all relevant cloud APIs
**What this PR does / why we need it**:
This adds context to all the relevant cloud provider interface signatures.
Callers of those APIs are currently satisfied using context.TODO().
There will be follow on PRs to push the context through the stack.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#815
**Special notes for your reviewer**:
For an idea of the full scope of this change please look at PR #58532.
**Release note**:
```release-note
Implementers of the cloud provider interface will note the addition of a context to this interface. Trivial code modification will be necessary for a cloud provider to continue to compile.
```
Automatic merge from submit-queue (batch tested with PRs 59441, 58264, 59287, 59396, 59439). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix some log param error
this patch fix some log parameter mistakes.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
/release-note-none
```
This adds context to all the relevant cloud provider interface signatures.
Callers of those APIs are currently satisfied using context.TODO().
There will be follow on PRs to push the context through the stack.
For an idea of the full scope of this change please look at PR #58532.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Ensure public IP removed after service deleted
**What this PR does / why we need it**:
When creating many LoadBalancer services, some services may exceed Azure basic LB's FrontendIPConfiguations quota (default is 10). Public IPs are created for all services, but it is not removed after deleting the kubernetes services.
This PR fixes the problem.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59255
**Special notes for your reviewer**:
Should cherry-pick to v1.9.
**Release note**:
```release-note
Ensure Azure public IP removed after service deleted
```
Automatic merge from submit-queue (batch tested with PRs 59165, 59386). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Refactor and add some tests.
Improve test coverage of the Azure cloud provider.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use beta instead of alpha GCE Compute API to add an alias range to an instance.
… instance.
**What this PR does / why we need it**:
We use beta instead of alpha GCE Compute API to add an alias range to an instance. Without this change, such an API is reserved for GCP projects which has whitelisted for this feature.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
N/A
**Special notes for your reviewer**:
**Release note**:
```release-note
"NONE".
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add PV size grow feature for azure file
**What this PR does / why we need it**:
According to kubernetes/features#284, add size grow feature for azure file
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56462
**Special notes for your reviewer**:
Since azure file is using SMB 3.0 protocal, there is no necessary to resize filesystem on agent side, the agent node will detect the changed size automatically.
**Release note**:
```
add size grow feature for azure file
```
/sig azure
@gnufied @rootfs @brendandburns
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add UT test to openstack and two para in configFromEnv
**What this PR does / why we need it**:
configFromEnv fun miss some para that the type define and add ut to TestToAuthOptions fun
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
/assign @dims
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix non-interface type ErrResourceNotFound on left
Related to #58145
The gophercloud.ErrResourceNotFound is not a interface, so should
use reflect to get its type then do a check.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Correct the URL of openstack and make test case more detail
**What this PR does / why we need it**:
correct the url of openstack doc and make the test case more detail,thanks!
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#54044
**Special notes for your reviewer**:
/assign @dims
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59097, 57076, 59295). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
GCE: sort firewall parameters
**What this PR does / why we need it**:
Make the firewall arguments deterministic.
Fixes#59294
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix a typo in pkg/cloudprovider/providers/azure/azure_loadbalancer.go
**What this PR does / why we need it**:
fix typo
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add UT test TestCheckOpenStackOptsfunc
**What this PR does / why we need it**:
checkOpenStackOpts func has three case that the test case not Covered，so add it,thanks
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Ensure IP is set for Azure internal loadbalancer
**What this PR does / why we need it**:
Internal Load Balancer created and associated with availability set but no target network ip configurations on Azure. And kube-controller-manager would panic because of nil pointer dereference.
This PR ensures it is set correctly.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59046
**Special notes for your reviewer**:
Should cherry-pick to v1.9
**Release note**:
```release-note
Ensure IP is set for Azure internal load balancer.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
cloudprovider/openstack: fix bug that tries to use octavia client to query flip
**What this PR does / why we need it**:
This fixes a bug that [potentially] tries to use an Octavia client to query a floating ip. Neutron should always handle those.
**Release note**:
```release-note
cloudprovider/openstack: fix bug the tries to use octavia client to query flip
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
use glog.info instead of glog.infof when no format
**What this PR does / why we need it**:
use glog.info instead of glog.infof when no format
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 59012, 58871). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Support GetLabelsForVolume In OpenStack
**What this PR does / why we need it**:
Since PersistentVolumeLabelController will invoke ```GetLabelsForVolume``` interface
in Cloud-Controller-Manager, OpenStack Provider should support it.
https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/cloud.go#L213
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#58870
**Special notes for your reviewer**:
**Release note**:
```release-note
Support GetLabelsForVolume in OpenStack Provider
```
Automatic merge from submit-queue (batch tested with PRs 58777, 58978, 58977, 58775). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix url parsing for staging/dev endpoint
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Clean up unused functions and consts
**What this PR does / why we need it**:
Clean up unused functions and consts.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58661, 58764, 58368, 58739, 58773). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[GCE cloud provider] Ensure hosts are updated in EnsureLoadBalancer()
**What this PR does / why we need it**:
From https://github.com/kubernetes/kubernetes/issues/56527, the `EnsureLoadBalancer()` implementation in GCE external LB doesn't always update the hosts (nodes). This PR makes it to do so.
Previously, the only situation where `ensureExternalLoadBalancer()` will not update hosts is when hosts are updated but there is no other changes that trigger target pool update (for which we delete&recreate target pool and hence updates the hosts). So the main change here is detecting that condition and call `updateTargetPool()`.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56527
**Special notes for your reviewer**:
Turned out it could be a small change, so I gave it a try.
/assign @nicksardo @bowei
**Release note**:
```release-note
NONE
```
Reuse the
service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags
annotation for these tags.
We think that this is a simpler and less intrusive approach than adding
a new annotation, and most users will want the same set of tags applied.
Signed-off-by: Brett Kochendorfer <brett.kochendorfer@gmail.com>
Automatic merge from submit-queue (batch tested with PRs 58697, 58658, 58676, 58674). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Expose the generate stub for compute API
This allows clients such as Ingress to begin migration to the newly
generated stubs.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58697, 58658, 58676, 58674). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix possible panic when getting Azure primary IPConfig
**What this PR does / why we need it**:
kube-controller-manager panic when removing a lot of nodes from kubernetes cluster (see #58675).
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#58675
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58590, 58667). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix logs message formating
**What this PR does / why we need it**:
Fix logs message formating.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57867, 58490, 58502, 58134). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add additional unit tests for Azure cloud provider.
@feiskyer @andyzhangx @khenidak
Automatic merge from submit-queue (batch tested with PRs 57867, 58490, 58502, 58134). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Openstack: register metadata.hostname as node name
**What this PR does / why we need it**:
Currently Openstack can boot up instances with the name like `xyz/abc`, which is not a valid kubelet node name. While `hostname` retrieved from `meta_data.json` has already been sanitized
by Openstack to valid DNS-1123 format string. It's safe to register this `metadata.hostname` as valid kubelet node name.
/kind bug
/sig openstack
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57765
**Special notes for your reviewer**:
/assign @dims @FengyunPan
**Release note**:
```release-note
Openstack: register metadata.hostname as node name
```
Automatic merge from submit-queue (batch tested with PRs 57867, 58490, 58502, 58134). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
GCE: neg to use generated code
GCE: neg to use generated code
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58412, 56132, 58506, 58542, 58394). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update Instances to use generated code
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58547, 57228, 58528, 58499, 58618). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove getOldSecurityGroupName() from OpenStack cloud provider
Related to #53764
The getOldSecurityGroupName() is used to get the old security
group name, we can remove it now.
**What this PR does / why we need it**:
#53764
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58300, 58530, 57942, 58543). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Ability to specify OS_* variables for OpenStack configuration
**What this PR does / why we need it**:
When we convert the OpenStack cloud provider to run in an external
process, we should be able to use kubernetes Secrets capability to
inject the OS_* variables. This way we can specify the cloud
configuration as a configmap, specify secrets for the userid/password
information. The configmap can be mounted as a file. the secrets can
be made available as environment variables. the external controller
itself can run as a pod/daemonset.
For backward compat, we preload all the OS_* variables, if anything
is in the config file, then that overrides the environment variables.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Authentication information for OpenStack cloud provider can now be specified as environment variables
```
Automatic merge from submit-queue (batch tested with PRs 56948, 58365, 58501). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update Zones to use generated code
Update Zones to use generated code
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
GCE: Change routes to use the generated code
GCE: Change routes to use the generated code
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57908, 58436). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Updates UrlMap, BackendService, Healthcheck, Certs, InstanceGroup to use the generated code
Updates UrlMap, BackendService, Healthcheck, Certs, InstanceGroup to use the generated code
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58104, 58492, 58491). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
GCE: forwarding rules to use generated code
GCE: forwarding rules to use generated code
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58104, 58492, 58491). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
GCE: addresses to use generated code
GCE: addresses to use generated code
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55918, 57258). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add multi-vc configuration for e2e tests
**What this PR does / why we need it**:
Currently, we accept configuration for only single VC in e2e tests. This PR adds support for multiple VC configuration for e2e tests.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/vmware/kubernetes/issues/412
**Special notes for your reviewer**:
Internally reviewed here: https://github.com/vmware/kubernetes/pull/418
**Release note**:
```release-note
NONE
```
// cc @divyenpatel @shaominchen
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add vSphere Cloud Provider simulator based tests
**What this PR does / why we need it**:
Initial set of vSphere Cloud Provider functional tests against the vCenter simulator, provides test coverage without having to run against a real vCenter instance.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
The vsphere simulator recently moved from vmware/vic to govmomi, I had discussed the idea of introducing it for testing with vSphere Cloud Provider maintainers. These tests provide 90%+ coverage for vclib/datacenter.go, but we can expand further of course.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58422, 58229, 58421, 58435, 58475). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add handling for method that use Pages() to retrieve results
- Add handling for method that use Pages() to retrieve results
- Make functions take in *Key rather than value type.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58422, 58229, 58421, 58435, 58475). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update gce call to use wrapper in gce_loadbalancer_external
**What this PR does / why we need it**:
Ack https://github.com/kubernetes/kubernetes/pull/58368#discussion_r162139441, replacing some direct compute api calls to use wrapper.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #NONE
**Special notes for your reviewer**:
/assign @nicksardo
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
openstack: remove orphaned routes from terminated instances
**What this PR does / why we need it**:
At the moment the openstack cloudprovider only returns routes where the `NextHop` address points to an existing openstack instance. This is a problem when an instance is terminated before the corresponding node is removed from k8s. The existing route is not returned by the cloudprovider anymore and therefore never considered for deletion by the route controller. When the route's `DestinationCIDR` is reassigned to a new node the router ends up with two routes pointing to a different `NextHop` leading to broken networking.
This PR removes skipping routes pointing to unknown next hops when listing routes. This should cause [this conditional](93dc3763b0/pkg/controller/route/route_controller.go (L208)) in the route controller to succeed and have the route removed if the route controller [feels responsible](93dc3763b0/pkg/controller/route/route_controller.go (L206)).
```release-note
OpenStack cloudprovider: Ensure orphaned routes are removed.
```
When we convert the OpenStack cloud provider to run in an external
process, we should be able to use kubernetes Secrets capability to
inject the OS_* variables. This way we can specify the cloud
configuration as a configmap, specify secrets for the userid/password
information. The configmap can be mounted as a file. the secrets can
be made available as environment variables. the external controller
itself can run as a pod/daemonset.
For backward compat, we preload all the OS_* variables, if anything
is in the config file, then that overrides the environment variables.
Automatic merge from submit-queue (batch tested with PRs 53631, 56960). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove unused code in UT files in pkg/
**What this PR does / why we need it**:
Remove unused code in UT files in pkg/ .
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Convert ComputerName to lower case
**What this PR does / why we need it**:
When instances count is greater than 10, azure will generate instance hostname with upper cases. But kubelet always converts hostname to lower case.
So azure cloud provider should also covert OsProfile.ComputerName to lower case.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#58274
**Special notes for your reviewer**:
This PR also adds more unit tests for scale sets.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Log message at a better level
**What this PR does / why we need it**:
We don't really need to log this meessage at level 1.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix typeos in cloud-controller-manager
**What this PR does / why we need it**:
fix typeos
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
NONE
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Instrument the Azure API calls for Prometheus monitoring
**What this PR does / why we need it**:
Instruments the Azure API calls for Prometheus monitoring.
**Special notes for your reviewer**:
This is version 2 based on the wrapped clients.
**Release note**:
```release-note
Instrument the Azure cloud provider for Prometheus monitoring.
```
cc @feiskyer @andyzhangx @jdumars @khenidak
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
The lbaas.opts.SubnetId should be set by subnet id.
Fix#58145
The getSubnetIDForLB() should return subnet id rather than net id.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add `cloud` for the generated GCE interfaces, support structs
Note: this does not wire the generated code.
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Wrap azure client calls
**What this PR does / why we need it**:
This is a clean up for azure client calls. It adds wrappers over azure clients and moves verbose logs and rate limiter inside.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/cc @cosmincojocar @andyzhangx
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix cinder detach problems
**What this PR does / why we need it**: We have currently huge problems in cinder volume detach. This PR tries to fix these issues.
**Which issue(s) this PR fixes**:
Fixes#50004Fixes#57497
**Special notes for your reviewer**:
**Release note**:
```release-note
openstack cinder detach problem is fixed if nova is shutdowned
```
Automatic merge from submit-queue (batch tested with PRs 54230, 58100, 57861, 54752). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
GCE: Use existing subnetwork of ILB forwarding rule
Fixes#57860
**Test Cases**:
Clusters using auto network with existence of a manual subnet in same region.
- [x] Upgrade 1.7 cluster with existing ILBs to latest. Confirm existing ILBs still are synced.
Version 1.7 does not attempt to fill in the subnetwork, so the forwarding rule was created with the correct subnetwork.
- [x] Upgrade 1.8 cluster with existing ILBs to latest. Confirm existing ILBs (using wrong subnet) still are synced.
- [x] Latest version creates ILBs using the correct subnet.
Clusters with manual subnets have always and will continue to use the subnet specified in gce.conf.
- [x] Upgrade 1.8 cluster with existing ILBs to latest. Confirm existing ILBs (using manual subnet) still are synced.
Clusters with legacy networks have always and will continue to use an empty subnet.
- [x] Upgrade 1.8 cluster with existing ILBs to latest. Confirm existing ILBs (using legacy network) still are synced.
**Release note**:
```release-note
GCE: Allows existing internal load balancers to continue using an outdated subnetwork
```
Automatic merge from submit-queue (batch tested with PRs 57511, 57978). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Renews cached NodeInfo with new vSphere connection
**What this PR does / why we need it**:
This PR modifies two public functions of nodemanager.go- GetNodeInfo and GetNodeDetails. For both these functions NodeInfo object is renewed with new GoVmomiClient and new vclib VirtualMachine and Datacenter.
**Which issue(s) this PR fixes** :
Fixes vmware#404
**Special notes for your reviewer**:
Code has been structured to minimize impact on existing 1.9 release code and any side-effects due to NodeInfo modification. This is a quick solution for vSphere connection renewal problem. A more enhanced solution is target for upcoming major release.
Testing:
- [x] Successfully tried out pod creation, deletion with dynamic volume.
- [x] Successfully ran e2e tests.
**Release note**:
```release-note
Fixes authentication problem faced during various vSphere operations.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix NLB icmp permission duplication
**What this PR does / why we need it**:
Fixes an issue with the ICMP rule for MTU during the creation of a NLB
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56703
Automatic merge from submit-queue (batch tested with PRs 57991, 57789). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix exists status for azure GetLoadBalancer
**What this PR does / why we need it**:
We see a lot of log indicating load balancer not found in azure:
```
E0109 07:00:31.126306 1 service_controller.go:776] Failed to process service kube-system/heapster. Retrying in 5m0s: error getting LB for service kube-system/heapster: Service(kube-system/heapster) - Loadbalancer not found
I0109 07:00:31.126384 1 event.go:218] Event(v1.ObjectReference{Kind:"Service", Namespace:"kube-system", Name:"heapster", UID:"400266e7-f507-11e7-bbc2-000d3af86f66", APIVersion:"v1", ResourceVersion:"450", FieldPath:""}): type: 'Warning' reason: 'CreatingLoadBalancerFailed' Error creating load balancer (will retry): error getting LB for service kube-system/heapster: Service(kube-system/heapster) - Loadbalancer not found
I0109 07:00:31.158858 1 azure_backoff.go:177] LoadBalancerClient.List(name) - backoff: success
E0109 07:00:31.158930 1 service_controller.go:776] Failed to process service kube-system/kubernetes-dashboard. Retrying in 5m0s: error getting LB for service kube-system/kubernetes-dashboard: Service(kube-system/kubernetes-dashboard) - Loadbalancer not found
I0109 07:00:31.158988 1 event.go:218] Event(v1.ObjectReference{Kind:"Service", Namespace:"kube-system", Name:"kubernetes-dashboard", UID:"4052f12b-f507-11e7-bbc2-000d3af86f66", APIVersion:"v1", ResourceVersion:"498", FieldPath:""}): type: 'Warning' reason: 'CreatingLoadBalancerFailed' Error creating load balancer (will retry): error getting LB for service kube-system/kubernetes-dashboard: Service(kube-system/kubernetes-dashboard) - Loadbalancer not found
```
It's interesting that those service does not need loadbalancer, and caller is just checking whether one loadbalancer exists.
009701f181/pkg/controller/service/service_controller.go (L287)
And in we can see when err is not nil, it will not check exists value. Thus we should not return error when exists=false.
This was changed in:
edfb2ad552 (diff-c901394068476b4ccb003a6c6efad57cR63)
The PR removes the error when exists=false.
**Which issue(s) this PR fixes**
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Do not set BaseURI twice
**What this PR does / why we need it**:
Do not set BaseURI again. BaseURI has been set by NewAccountsClientWithBaseURI and NewDisksClientWithBaseURI method.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57951
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @karataliu
add getNodeNameByID and use volume.AttachedDevice as devicepath
use uppercase functionname
do not delete automatically nodes if node is shutdowned in openstack
do not delete node
fix gofmt
fix cinder detach if instance is not in active state
fix gofmt
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Ensure Azure LB removable when VMSS is enabled
**What this PR does / why we need it**:
When VMSS enabled, Azure LB not removed after all LoadBalancer services deleted.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57826
**Special notes for your reviewer**:
This PR upgrades Azure GO SDK to latest release and adds a workaround to fix the problem.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add generic interface for azure clients
**What this PR does / why we need it**:
Continue of #43287. Moving remaining clients to generic interfaces.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Continue of #43287.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Minor commenting fixes for Azure Disk Controllers from CR
**What this PR does / why we need it**:
Minor commenting fixes for Azure Disk Controllers from code review.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Uniform Azure VM api calls
**What this PR does / why we need it**:
There is still a call to 'VirtualMachinesClient.Get' directly in azure_backoff, which does not go through the cache approach.
This PR uniforms all calls for getting azure vm to use 'getVirtualMachine'. Also refine unused 'exists' return value.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Related #57031
Follow-up #57432
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57696, 57821, 56317). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Move DefaultMaxEBSVolumes constant into scheduler
**What this PR does / why we need it**:
A constant only used by the scheduler lives in the aws cloudprovider package. Moving the constant into the only package where it is used reduces import bloat. Testing with the dockerized build environment, the kube-scheduler binary went from 61748499 bytes to 47339144 bytes on amd64 with this change.
**Release note**:
```release-note
NONE
```
A constant only used by the scheduler lives in the aws cloudprovider
package. Moving the constant into the only package where it is used
reduces import bloat.
Automatic merge from submit-queue (batch tested with PRs 49856, 56257, 57027, 57695, 57432). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add cache for VirtualMachinesClient.Get in azure cloud provider
**What this PR does / why we need it**:
Add a timed cache for 'VirtualMachinesClient.Get'
Currently cloud provider will send several get calls to same URL in short period, which is not necessary.
**Which issue(s) this PR fixes**:
Fixes#57031
**Special notes for your reviewer**:
**Release note**:
NONE
Automatic merge from submit-queue (batch tested with PRs 49856, 56257, 57027, 57695, 57432). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix vmss listing for Azure cloud provider
**What this PR does / why we need it**:
Fix a stupid bug of vmss listing: if there is only one instance, listScaleSetsWithRetry and listScaleSetVMsWithRetry will return empty list.
This PR also adds more verbose logs.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Related of #43287.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixing vSphere Cloud Provider to use "vsphere-cloud-provider" to create ClientBuilder
**What this PR does / why we need it**:
vSphere cloud Provider is not using lower case naming while creating clientBuilder.
With this fix, ClientBuilder is created using lowercase naming.
With mixed upper-lower case name, controller manager is crashing.
**Which issue(s) this PR fixes**
Fixes # https://github.com/kubernetes/kubernetes/issues/57279
**Special notes for your reviewer**:
None
**Release note**:
```release-note
This fixes controller manager crash in certain vSphere cloud provider environment.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove useInstanceMetadata param from Azure cloud provider
**What this PR does / why we need it**:
With out-of-tree Azure cloud provider (#50752), metadata won't work any more (kubelet won't call those metadata functions any more).
This PR removes the parameter useInstanceMetadata from Azure cloud provider.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57646.
**Special notes for your reviewer**:
**Release note**:
```release-note
Remove useInstanceMetadata parameter from Azure cloud provider.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Reduce VirtualMachineScaleSetsClient#List calls for Azure cloud provider
**What this PR does / why we need it**:
For master nodes not managed by VMSS, current cloud provider would updateCaches each time when finding master nodes info. This could result in call limits of `VirtualMachineScaleSetsClient#List`.
This PR adds a caches to those nodes which reduces the cache updating significantly.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Continue of #43287.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57502, 57543). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Support multiple scale sets in Azure cloud provider
**What this PR does / why we need it**:
This PR adds multiple scale sets support in Azure cloud provider.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Continue of #43287.
**Special notes for your reviewer**:
- Adds a local cache of basic scale sets information
- Update the cache when new nodes are not found or periodically
- Since azure doesn't support getting the scale set which contains the node, the cache is updated via listing all scale sets and their virtual machines
**Release note**:
```release-note
Support multiple scale sets in Azure cloud provider.
```
/assign @brendandburns @andyzhangx
Automatic merge from submit-queue (batch tested with PRs 57351, 55654). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
GCE: Get automatically created subnetwork if none is specified for auto network
Fixes#57350
**Release note**:
```release-note
GCE: Fixes ILB creation on automatic networks with manually created subnetworks.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add Dong Liu as approver and add OWNERS in credentialprovider
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57540
**Special notes for your reviewer**:
**Release note**:
```
none
```
/sig azure
/assign @brendandburns
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Allow use resource ID to specify public IP address in azure_loadbalancer
**What this PR does / why we need it**: Currently the Azure load balancer assumes that a Public IP address is in the same resource group as the cluster. This is not necessarily true in all environments, in addition to accepting a Public IP, we should allow an annotation to the `Service` object that indicates what resource group the IP is present in.
**Which issue this PR fixes**: fixes#53274#52129
**Special notes for your reviewer**: *first time golang user, please forgive the amateurness*
Release note
```release-note
Allow use resource ID to specify public IP address in azure_loadbalancer
```
Automatic merge from submit-queue (batch tested with PRs 57282, 57484). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix a bug in validating node existence.
**What this PR does / why we need it**:
Fixes an bug where if an error was returned that was not an `autorest.DetailedError` we would return `"not found", nil` which would result in Nodes going `NotReady`
**Which issue(s) this PR fixes**
Fixes#57483
**Release note**:
```release-note
Fixes an bug where if an error was returned that was not an `autorest.DetailedError` we would return `"not found", nil` which caused nodes to go to `NotReady` state.
```
@feiskyer @khendiak
Automatic merge from submit-queue (batch tested with PRs 57282, 57484). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove dead code in cloudprovider
**What this PR does / why we need it**:
Remove dead code in `cloudprovider`
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55751, 57337, 56406, 56864, 57347). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
should reuse code rather than rewrite it
**What this PR does / why we need it**:
should reuse `dc.GetDatastoreByName()`, instead of rewrite it
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 56375, 56872, 57053, 57165, 57218). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Compare correct file names for volume detach operation
**What this PR does / why we need it**:
Current volume detach code compares volume path with disk path, as it is. This PR removes '.vmdk' extension from both paths and then compares them. This makes sure that correct comparison is done irrespective of a missing '.vmdk' extension from one of the paths.
**Which issue(s) this PR fixes**:
Fixes https://github.com/vmware/kubernetes/issues/392
**Special notes for your reviewer**:
Deployed cluster on vSphere and provisioned a static volume. Verified that a statically provisioned volume gets detached even when volume path didn't contain any .vmdk extension and disk path had .vmdk extension.
**Release note**:
```vSphere cloud provider: Fix detach operation for volumes, when .vmdk extension is not specified in volume path.```
Automatic merge from submit-queue (batch tested with PRs 57122, 57142, 57016, 56927, 56678). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
should not ignore return messages from wait function
**What this PR does / why we need it**:
It should not ignore return messages for `wait*` function. When it go wrong, need `return` at once.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57148, 57123, 57091, 57141, 57131). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Support LoadBalancer for Azure Virtual Machine Scale Sets
**What this PR does / why we need it**:
Continue of #43287, this PR adds LoadBalancer support for Azure Virtual Machine Scale Sets. To achieve this, this PR also
- Add a general VMSet interfaces for VMSS and VMAS, so that we won't add much if-else blocks for different logics
- Add scale sets implementation and availability sets implementation to VMSet
- Add vmSet property to Azure cloud provider and call vmSet instead of direct azure clients
- Add LoadBalancer support based vmSet
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of #43287.
**Special notes for your reviewer**:
**Release note**:
```release-note
Support LoadBalancer for Azure Virtual Machine Scale Sets
```
/assign @brendandburns
Automatic merge from submit-queue (batch tested with PRs 56997, 57008, 56984, 56975, 56955). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove unused ScrubDNS interface from cloudprovider
**What this PR does / why we need it**:
DNS scrubber from kubelet has been removed in #36785 and cloudprovider's `ScrubDNS()` interface is not used anywhere.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56953.
**Special notes for your reviewer**:
**Release note**:
```release-note
Remove ScrubDNS interface from cloudprovider.
```
Automatic merge from submit-queue (batch tested with PRs 56997, 57008, 56984, 56975, 56955). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Sort default cidrs for reproducible builds
**What this PR does / why we need it**:
In different distros or environments, we may end up with a different
order of the default string printed during help and man page generation,
So we should sort so the string we print is the same everytime.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#52269
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 56639, 56746, 56715, 56673, 56726). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix issue #390
**What this PR does / why we need it**:
When VM node is removed from vSphere Inventory, the corresponding Kubernetes node is unregistered and removed from registeredNodes cache in nodemanager. However, it is not removed from the other node info cache in nodemanager. The fix is to update the other cache accordingly.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/vmware/kubernetes/issues/390
**Special notes for your reviewer**:
Internally review PR here: https://github.com/vmware/kubernetes/pull/402
**Release note**:
```
NONE
```
Testing Done:
1. Removed the node VM from vSphere inventory.
2. Create storageclass and pvc to provision volume dynamically
Automatic merge from submit-queue (batch tested with PRs 56480, 56675, 56624, 56648, 56658). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix session out in vSphere Cloud Provider
**What this PR does / why we need it**:
When the disk is attached error is returned in case of VM migration but the disk is attached successfully. When pvc is created for provisioning volume dynamically the volume is not provisioned since the vc session was expired and not renewed. This PR fixes both the issues.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/vmware/kubernetes/issues/393https://github.com/vmware/kubernetes/issues/391
**Special notes for your reviewer**:
Internally review PR here: https://github.com/vmware/kubernetes/pull/396
**Release note**:
```release-note
NONE
```
Test Done:
Test for fix https://github.com/vmware/kubernetes/issues/391 (Prints error on attach disk)
- Create storageclass and pvc to provision volume dynamically.
- Migrated VM to different VC.
- Created Pod with volume provisioned at step 1.
- Executed command kubectl describe pod.
After Fix: Didn't find the error message.
Test for fix https://github.com/vmware/kubernetes/pull/396 (Session out)
Tests which reported this issue: go run hack/e2e.go --check-version-skew=false --v --test '--test_args=--ginkgo.focus=Selector-Label\sVolume\sBinding:vsphere'
After Fix: This tests didn't report any errors.
Automatic merge from submit-queue (batch tested with PRs 56480, 56675, 56624, 56648, 56658). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove unnecessary condition
**What this PR does / why we need it**:
Now that we have judgement `loadbalancer == nil` in `L1286`, the condition is uncessary.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Pengfei Ni