Automatic merge from submit-queue (batch tested with PRs 41756, 36344, 34259, 40843, 41526)
add InternalDNS/ExternalDNS node address types
This PR adds internal/external DNS names to the types of NodeAddresses that can be reported by the kubelet.
will spawn follow up issues for cloud provider owners to include these when possible
```release-note
Nodes can now report two additional address types in their status: InternalDNS and ExternalDNS. The apiserver can use `--kubelet-preferred-address-types` to give priority to the type of address it uses to reach nodes.
```
This change migrates the 'openstack' provider and 'keystone'
authenticator plugin to the newer gophercloud/gophercloud library.
Note the 'rackspace' provider still uses rackspace/gophercloud.
Fixes#30404
We recognize a few AWS tags, and skip over masters when finding zones
for dynamic volumes. This will fix#34583.
This is not perfect, in that really the scheduler is the only component
that can correctly choose the zone, but should address the common
problem.
Automatic merge from submit-queue (batch tested with PRs 37137, 41506, 41239, 41511, 37953)
e2e test for storage class diskformat verification for vsphere cloud provider
**What this PR does / why we need it**:
This PR adds a new e2e test for vsphere cloud provider.
Test is to verify diskformat specified in storage-class is being honored while volume creation.
Steps:
1. Create StorageClass with diskformat set to valid type (supported options are `eagerzeroedthick`, `zeroedthick` and `thin`)
2. Create PVC which uses the StorageClass created in step 1.
3. Wait for PV to be provisioned.
4. Wait for PVC's status to become Bound
5. Create POD using PVC on specific node.
6. Wait for Disk to be attached to the node.
7. Get node VM's devices and find PV's Volume Disk.
8. Get Backing Info of the Volume Disk and obtain Property of `VirtualDiskFlatVer2BackingInfo` - `EagerlyScrub` and `ThinProvisioned`
9. Based on the value of `EagerlyScrub` and `ThinProvisioned`, verify if diskformat is correct.
10. Delete POD and Wait for Volume Disk to be detached from the Node.
11. Delete PVC, PV and Storage Class
**Which issue this PR fixes** *
fixes #
**Special notes for your reviewer**:
Test is executed against v1.6.0-alpha.1
Test is failing on v1.4.8
**Release Note**
```release-note
NONE
```
@kerneltime @BaluDontu @abrarshivani please review this PR
Automatic merge from submit-queue
Fix AWS device allocator to only use valid device names
According to
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/device_naming.html
we can only use /dev/xvd[b-c][a-z] as device names - so we can only
allocate upto 52 ebs volumes on a node.
fixes#41453
cc @justinsb @kubernetes/sig-storage-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 40297, 41285, 41211, 41243, 39735)
fix variables in openstack.go to keep camel casing and remove unused var
Automatic merge from submit-queue (batch tested with PRs 41337, 41375, 41363, 41034, 41350)
use instance's Name to attach gce disk
**What this PR does / why we need it**:
fix#40427
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#40427
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
In cases where insecure OpenStack endpoint is to be used
(e.g., when testing), gophercloud will fail to connect
to such endpoints. This patch adds support for custom CA
file configuration option, which, when provided, will
make gophercloud validate OpenStack endpoint against
certificate(s) read from file specified in that option.
Automatic merge from submit-queue (batch tested with PRs 41223, 40892, 41220, 41207, 41242)
Fixes#40819 and Fixes#33114
**What this PR does / why we need it**:
Start looking up the virtual machine by it's UUID in vSphere again. Looking up by IP address is problematic and can either not return a VM entirely, or could return the wrong VM.
Retrieves the VM's UUID in one of two methods - either by a `vm-uuid` entry in the cloud config file on the VM, or via sysfs. The sysfs route requires root access, but restores the previous functionality.
Multiple VMs in a vCenter cluster can share an IP address - for example, if you have multiple VM networks, but they're all isolated and use the same address range. Additionally, flannel network address ranges can overlap.
vSphere seems to have a limitation of reporting no more than 16 interfaces from a virtual machine, so it's possible that the IP address list on a VM is completely untrustworthy anyhow - it can either be empty (because the 16 interfaces it found were veth interfaces with no IP address), or it can report the flannel IP.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Fixes#40819Fixes#33114
**Special notes for your reviewer**:
**Release note**:
```release-note
Reverts to looking up the current VM in vSphere using the machine's UUID, either obtained via sysfs or via the `vm-uuid` parameter in the cloud configuration file.
```
Start looking up the virtual machine by it's UUID in vSphere again. Looking up by IP address is problematic and can either not return a VM entirely, or could return the wrong VM.
Retrieves the VM's UUID in one of two methods - either by a `vm-uuid` entry in the cloud config file on the VM, or via sysfs. The sysfs route requires root access, but restores the previous functionality.
Multiple VMs in a vCenter cluster can share an IP address - for example, if you have multiple VM networks, but they're all isolated and use the same address range. Additionally, flannel network address ranges can overlap.
vSphere seems to have a limitation of reporting no more than 16 interfaces from a virtual machine, so it's possible that the IP address list on a VM is completely untrustworthy anyhow - it can either be empty (because the 16 interfaces it found were veth interfaces with no IP address), or it can report the flannel IP.
Automatic merge from submit-queue (batch tested with PRs 41037, 40118, 40959, 41084, 41092)
Fix for detach volume when node is not present/ powered off
Fixes#33061
When a vm is reported as no longer present in cloud provider and is deleted by node controller, there are no attempts to detach respective volumes. For example, if a VM is powered off or paused, and pods are migrated to other nodes. In the case of vSphere, the VM cannot be started again because the VM still holds mount points to volumes that are now mounted to other VMs.
In order to re-join this node again, you will have to manually detach these volumes from the powered off vm before starting it.
The current fix will make sure the mount points are deleted when the VM is powered off. Since all the mount points are deleted, the VM can be powered on again.
This is a workaround proposal only. I still don't see the kubernetes issuing a detach request to the vsphere cloud provider which should be the case. (Details in original issue #33061 )
@luomiao @kerneltime @pdhamdhere @jingxu97 @saad-ali
Automatic merge from submit-queue
Add OWNERS file for GCE cloud provider
GCE cloud provider does not have OWNERS file and all PRs need to be approved by owner of pkg/cloudprovider, which is currently only @mikedanese. Adding more options would be helpful to speed up reviews.
Feel free to add/remove some names, this first version is just my qualified guess. It's hard to distinguish generic Kubernetes refactoring from real cloud provider work in git log.
```release-note
NONE
```
addressed review comments
Addressed review comment for pv_reclaimpolicy.go to verify content of the volume
addressed 2nd round of review comments
addressed 3rd round of review comments from jeffvance
1. Update the configuration file for Photon Controller cloud provider
2. Only master nodes can communicate with Photon Controller endpoint
3. Enable support for authentication-enabled Photon Controller endpoint
4. Update NodeAddresses function for query from local node
Automatic merge from submit-queue (batch tested with PRs 40638, 40742, 40710, 40718, 40763)
Reduce log noise when aws cloud cannot find public-ip4 metadata
**What this PR does / why we need it**:
The PR removes excessive log spam produced by the aws cloud provider for normal operating conditions.
**Which issue this PR fixes**
fixes https://github.com/kubernetes/kubernetes/issues/23155
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Set NetworkInterfaces and Subnets to nil before updating Azure security groups
**What this PR does / why we need it**: This is a workaround until we have an upstream fix in azure-sdk-for-go/go-autorest. Corresponding issues are https://github.com/kubernetes/kubernetes/issues/40332 and https://github.com/Azure/go-autorest/issues/112
In k8s 1.5.2, an update to azure-sdk-for-go was cherry-picked, which broke creation/updating of LBs on Azure. As we should have it back to a working state ASAP, I'd like to do a workaround for now and later when the upstream fix comes in, remove the workaround again.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#40332
**Release note**:
```release-note
Fix failing load balancers in Azure
```
CC @colemickens
Automatic merge from submit-queue
genericapiserver: cut off more dependencies – episode 3
Compare commit subjects.
This is the first step to get `apiserver_test.go` and `watch_test.go` in `pkg/genericapiserver/endpoints` cutoff from k8s.io/kubernetes dependencies.
After this we have to sync client-go and then "episode 4" can go in.
approved based on #40363
Automatic merge from submit-queue
Adding vmdk file extension for vmDiskPath in vsphere DeleteVolume
**What this PR does / why we need it**:
This fix is required to handle situation when volume path is not set with vmdk file extension.
DiskAttach call does not require vmdk file extension to be explicitly set in the file path, so in PV yaml file, users have option to set such file path.
Issue happens when such PV is created with persistentVolumeReclaimPolicy set to Delete, and user deletes associated PVC. Volume deletion will fail with the message: file was not found
**Which issue this PR fixes** *
Fixes: #40560
**Special notes for your reviewer**:
Tested fix with custom image: divyen/hyperkube-fix-deletevolume:latest (Image built using base v1.5.3-beta.0)
@kerneltime @BaluDontu please review this fix.
Automatic merge from submit-queue
AWS: trust region if found from AWS metadata
```release-note
AWS: trust region if found from AWS metadata
```
Means we can run in newly announced regions without a code change.
We don't register the ECR provider in new regions, so we will still need
a code change for now.
Fix#35014
Automatic merge from submit-queue
Set custom PollingDelay of 5 seconds for Azure VirtualMachinesClient
The default polling delay of 1 minute results in very long delays when
an Azure Disk is attached to a node. It gets worse as go-autorest
doubles the default delay to 2 minutes.
Please see: https://github.com/kubernetes/kubernetes/issues/35180#issuecomment-273085063
Only the PollingDelay for VirtualMachinesClient is modified here to
avoid too much pressure on Azure quotas.
Release Nodes:
```release-note
Reduce time needed to attach Azure disks
```
Automatic merge from submit-queue
Prune reviewers from pkg/cloudprovider
**What this PR does / why we need it**
Per discussion in https://github.com/kubernetes/kubernetes/pull/36530 the `OWNERS` file for `pkg/cloudprovider` should not contain additional reviewers at this time.
**Special notes for your reviewer**:
Sorry for the extra work in review
**Release note**:
`NONE`
Automatic merge from submit-queue
Curating Owners: pkg/cloudprovider
cc @runseb @justinsb @kerneltime @mikedanese @svanharmelen @anguslees @brendandburns @abrarshivani @imkin @luomiao @colemickens @ngtuna @dagnello @abithap
In an effort to expand the existing pool of reviewers and establish a
two-tiered review process (first someone lgtms and then someone
experienced in the project approves), we are adding new reviewers to
existing owners files.
If You Care About the Process:
------------------------------
We did this by algorithmically figuring out who’s contributed code to
the project and in what directories. Unfortunately, that doesn’t work
well: people that have made mechanical code changes (e.g change the
copyright header across all directories) end up as reviewers in lots of
places.
Instead of using pure commit data, we generated an excessively large
list of reviewers and pruned based on all time commit data, recent
commit data and review data (number of PRs commented on).
At this point we have a decent list of reviewers, but it needs one last
pass for fine tuning.
Also, see https://github.com/kubernetes/contrib/issues/1389.
TLDR:
-----
As an owner of a sig/directory and a leader of the project, here’s what
we need from you:
1. Use PR https://github.com/kubernetes/kubernetes/pull/35715 as an example.
2. The pull-request is made editable, please edit the `OWNERS` file to
remove the names of people that shouldn't be reviewing code in the
future in the **reviewers** section. You probably do NOT need to modify
the **approvers** section. Names asre sorted by relevance, using some
secret statistics.
3. Notify me if you want some OWNERS file to be removed. Being an
approver or reviewer of a parent directory makes you a reviewer/approver
of the subdirectories too, so not all OWNERS files may be necessary.
4. Please use ALIAS if you want to use the same list of people over and
over again (don't hesitate to ask me for help, or use the pull-request
above as an example)
Automatic merge from submit-queue (batch tested with PRs 39625, 39842)
AWS: Remove duplicate calls to DescribeInstance during volume operations
This change removes all duplicate calls to describeInstance
from aws volume code path.
**What this PR does / why we need it**:
This PR removes the duplicate calls present in disk check code paths in AWS. I can confirm that `getAWSInstance` actually returns all instance information already and hence there is no need of making separate `describeInstance` call.
Related to - https://github.com/kubernetes/kubernetes/issues/39526
cc @justinsb @jsafrane
Means we can run in newly announced regions without a code change.
We don't register the ECR provider in new regions, so we will still need
a code change for now.
This also means we do trust config / instance metadata, and don't reject
incorrectly configured zones.
Fix#35014
The default polling delay of 1 minute results in very long delays when
an Azure Disk is attached to a node. It gets worse as go-autorest
doubles the default delay to 2 minutes.
Please see: https://github.com/kubernetes/kubernetes/issues/35180#issuecomment-273085063
Only the PollingDelay for VirtualMachinesClient is modified here to
avoid too much pressure on Azure quotas.
Automatic merge from submit-queue
AWS: Add exponential backoff to waitForAttachmentStatus() and createTags()
We should use exponential backoff while waiting for a volume to get attached/detached to/from a node. This will lower AWS load and reduce API call throttling.
This partly fixes#33088
@justinsb, can you please take a look?
GCE PD names generated by Kubernetes are guaranteed to be unique - they
contain name of the cluster and UID of the PVC that is behind it.
Presence of a GCE PD that has the same name as we want to provision
indicates that previous provisioning did not go well and most probably
the controller manager process was restarted in the meantime.
Kubernetes should reuse this volume and not provision a new one.
Automatic merge from submit-queue
Changed default scsi controller type in vSphere Cloud Provider
This PR changes default scsi controller to ```pvscsi``` in vSphere Cloud Provider. Fixes#37527
Automatic merge from submit-queue (batch tested with PRs 38818, 38813, 38820)
AWS: Add sequential allocator for device names.
On AWS, we should not reuse device names as long as possible, see https://aws.amazon.com/premiumsupport/knowledge-center/ebs-stuck-attaching/
> "If you specify a device name that is not in use by EC2, but is being used by the block device driver within the EC2 instance, the attachment of the EBS volume does not succeed and the EBS volume is stuck in the attaching state."
This patch adds a device name allocator that tries to find a name that's next to the last used device name instead of using the first available one. This way we will loop through all device names ("xvdba" .. "xvdzz") before a device name is reused.
Fixes: #31891
@wongma7, @gnufied, @childsb PTAL
On AWS, we should not reuse device names as long as possible, see
https://aws.amazon.com/premiumsupport/knowledge-center/ebs-stuck-attaching/
"If you specify a device name that is not in use by EC2, but is being used by
the block device driver within the EC2 instance, the attachment of the EBS
volume does not succeed and the EBS volume is stuck in the attaching state."
This patch adds a device name allocator that tries to find a name that's next
to the last used device name instead of using the first available one.
This way we will loop through all device names ("xvdba" .. "xvdzz") before
a device name is reused.
We should use exponential backoff while waiting for a volume to get attached/
detached to/from a node. This will lower AWS load and reduce its API call
throttling.
Automatic merge from submit-queue (batch tested with PRs 38638, 38334)
Remove Azure Subnet RouteTable check
**What this PR does / why we need it**:
PR Removes the subnet configuration check for Azure cloudprovider. The subnet check ensures that the subnet is associated with the Route Table. However if the VNET is in a different Azure Resource Group then the check fails, even if the subnet is already valid. This a stop gap fix, to allow Kubernetes to be deployed to Custom VNETs in Azure, that may reside in a different resource group to the cluster.
fixes#38134
@colemickens
Automatic merge from submit-queue
Bad conditional in vSphereLogin function
```release-note
Fixes NotAuthenticated errors that appear in the kubelet and kube-controller-manager due to never logging in to vSphere
```
With this conditional being == instead of !=, a login would never actually be attempted by this provider, and disk attachments would fail with a NotAuthenticated error from vSphere.
This method has been unused by k8s for some time, and yet is the last
piece of the cloud provider API that encourages provider names to be
human-friendly strings (this method applies a regex to instance names).
Actually removing this deprecated method is part of a long effort to
migrate from instance names to instance IDs in at least the OpenStack
provider plugin.
Automatic merge from submit-queue
openstack: Implement the `Routes` provider API
``` release-note
Implement the Routes provider API for OpenStack using Neutron extraroute extension. This removes the need for flannel/etc where supported. To use, ensure all your nodes are on the same Neutron (private) network and specify the router ID in new `[Route]` section of provider config:
[Route]
router-id = <router UUID>
```
Automatic merge from submit-queue (batch tested with PRs 36543, 38189, 38289, 38291, 36724)
context.Context should be the first parameter of a function in vsphere
**What this PR does / why we need it**:
Change the position of the context.Context parameter.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
golint
**Release note**:
```release-note
```
Signed-off-by: yupeng <yu.peng36@zte.com.cn>
Check for error conditions from the vSphere API and return the err if one occurs. The vSphere API does not return an err for unauthenticated users, it just returns a nil user object.
Automatic merge from submit-queue (batch tested with PRs 38076, 38137, 36882, 37634, 37558)
Allow backendpools in Azure Load Balancers which are not owned by cloud provider
**What this PR does / why we need it**: It fixes#36880
**Which issue this PR fixes**: fixes#36880
**Special notes for your reviewer**:
**Release note**:
```release-note
Allow backendpools in Azure Load Balancers which are not owned by cloud provider
```
Instead of bailing out when we find another backend pool, we just ignore
other backend pools and add ours to the list of existing.
Fixes#36880
This change implements the Routes API using Neutron's "extraroute"
extension.
To use, this requires all the nodes to be on the same Neutron network
and the UUID of the Neutron router on that network.
Required cloud provider config section:
[Route]
router-id = <UUID of Neutron router>
Ensure kube-controllermanager is started with (non-default)
`--allocate-node-cidrs=true` and set `--cluster-cidr` to the POD
super-subnet (a private /16 would be reasonable).
Based on an earlier version by @timbyr (#19473)
Update EnsureLoadBalancer/UpdateLoadBalancer API to use node objects.
In particular, this allows us to take the node address directly from the
node.Status.Addresses and avoids a name -> instance lookup.
Many providers need to do some sort of node name -> IP or instanceID
lookup before they can use the list of hostnames passed to
EnsureLoadBalancer/UpdateLoadBalancer.
This change just passes the full Node object instead of simply the node
name, allowing providers to use the node's provider ID and cached
addresses without additional lookups. Using `node.Name` reproduces the
old behaviour.
Automatic merge from submit-queue
Support persistent volume usage for kubernetes running on Photon Controller platform
**What this PR does / why we need it:**
Enable the persistent volume usage for kubernetes running on Photon platform.
Photon Controller: https://vmware.github.io/photon-controller/
_Only the first commit include the real code change.
The following commits are for third-party vendor dependency and auto-generated code/docs updating._
Two components are added:
pkg/cloudprovider/providers/photon: support Photon Controller as cloud provider
pkg/volume/photon_pd: support Photon persistent disk as volume source for persistent volume
Usage introduction:
a. Photon Controller is supported as cloud provider.
When choosing to use photon controller as a cloud provider, "--cloud-provider=photon --cloud-config=[path_to_config_file]" is required for kubelet/kube-controller-manager/kube-apiserver. The config file of Photon Controller should follow the following usage:
```
[Global]
target = http://[photon_controller_endpoint_IP]
ignoreCertificate = true
tenant = [tenant_name]
project = [project_name]
overrideIP = true
```
b. Photon persistent disk is supported as volume source/persistent volume source.
yaml usage:
```
volumes:
- name: photon-storage-1
photonPersistentDisk:
pdID: "643ed4e2-3fcc-482b-96d0-12ff6cab2a69"
```
pdID is the persistent disk ID from Photon Controller.
c. Enable Photon Controller as volume provisioner.
yaml usage:
```
kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
name: gold_sc
provisioner: kubernetes.io/photon-pd
parameters:
flavor: persistent-disk-gold
```
The flavor "persistent-disk-gold" needs to be created by Photon platform admin before hand.
Automatic merge from submit-queue
cloudprovider/cloudstack: Fix a bug where we assume IP addresses instead of a hostnames
Because of how our test environment was setup, we didn’t notice that we were assuming the load balancer hosts list to always be IP addresses, while they actually are hostnames.
So without this PR, the load balancer code will not work as expected as it will not be able to find the nodes that need to be load balanced.
Also updated some comments and added a check to prevent trying to release a public IP if we don’t have one.
Automatic merge from submit-queue
azure: loadbalancer rules use DSR
**What this PR does / why we need it**:
Enables "direct server return" on the load balancer in Azure, which causes the DIP to be preserved when traffic goes through the load balancer. This enables service traffic to go to the Service Port rather than having to go through the NodePort.
**Special notes for your reviewer**:
N/A.
**Tested with...**:
```shell
kubectl run nginx --image=nginx
kubectl run nginx2 --image=nginx
kubectl expose deployment nginx --port=80 --type=LoadBalancer
kubectl expose deployment nginx2 --port=80 --type=LoadBalancer
```
Ensuring that both services got external IPs and that the resources created looked correct.
**Release note**:
```release-note
azure: load balancer preserves destination ip address
```
CC: @brendandburns
Automatic merge from submit-queue
[RFC] Prepare for deprecating NodeLegacyHostIP
Ref https://github.com/kubernetes/kubernetes/issues/9267#issuecomment-257994766
*What this PR does*
- Add comments saying "LegacyHostIP" will be deprecated in 1.7;
- Add v1.NodeLegacyHostIP to be consistent with the internal API (useful for client-go migration #35159)
- Let cloudproviders who used to only set LegacyHostIP set the IP as both InternalIP and ExternalIP
- Master used to ssh tunnel to node's ExternalIP or LegacyHostIP to do [healthz check](https://github.com/kubernetes/kubernetes/blame/master/pkg/master/master.go#L328-L332). OTOH, if on-prem, kubelet only [sets](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/kubelet_node_status.go#L430-L431) LegacyHostIP or InternalIP. In order to deprecate LegacyHostIP in 1.7, I let healthz check to use InternalIP if ExternalIP is not available. (The healthz check is the only consumer of LegacyHostIP in k8s.)
@liggitt @justinsb @bgrant0607
```release-note
LegacyHostIP will be deprecated in 1.7.
```
Automatic merge from submit-queue
Fix LBaaS version detection in openstack cloudprovider
`lbversion` is the local variable used for version detection when `os.lbOpts.LBVersion` is not specified.
xref https://bugzilla.redhat.com/show_bug.cgi?id=1391837
@ncdc @derekwaynecarr @anguslees
Mark NodeLegacyHostIP will be deprecated in 1.7;
Let cloudprovider that used to only set NodeLegacyHostIP set the IP as both InternalIP and ExternalIP, to allow dprecation in 1.7
Automatic merge from submit-queue
Don't rely on device name provided by Cinder
See issue #33128
We can't rely on the device name provided by Cinder, and thus must perform
detection based on the drive serial number (aka It's cinder ID) on the
kubelet itself.
This patch re-works the cinder volume attacher to ignore the supplied
deviceName, and instead defer to the pre-existing GetDevicePath method to
discover the device path based on it's serial number and /dev/disk/by-id
mapping.
This new behavior is controller by a config option, as falling back
to the cinder value when we can't discover a device would risk devices
not showing up, falling back to cinder's guess, and detecting the wrong
disk as attached.
Automatic merge from submit-queue
Corect filtering of OpenStack LBaaS resources to delete
Neutron's API ignores unknown parameters. When listing pools etc, K8
attempts to filter on "LoadBalancerID", which is not a valid filter.
As such, it is ignored by Neutron, and a list of all pools is
returned. K8 then proceeds to delete each of the pools.
Instead, we now double check the resources really belong to the LB
we're trying to delete.
Fixes issue #33759
See issue #33128
We can't rely on the device name provided by Cinder, and thus must perform
detection based on the drive serial number (aka It's cinder ID) on the
kubelet itself.
This patch re-works the cinder volume attacher to ignore the supplied
deviceName, and instead defer to the pre-existing GetDevicePath method to
discover the device path based on it's serial number and /dev/disk/by-id
mapping.
This new behavior is controller by a config option, as falling back
to the cinder value when we can't discover a device would risk devices
not showing up, falling back to cinder's guess, and detecting the wrong
disk as attached.
We are more liberal in what we accept as a volume id in k8s, and indeed
we ourselves generate names that look like `aws://<zone>/<id>` for
dynamic volumes.
This volume id (hereafter a KubernetesVolumeID) cannot directly be
compared to an AWS volume ID (hereafter an awsVolumeID).
We introduce types for each, to prevent accidental comparison or
confusion.
Issue #35746
GetDevicePath was currently coded to only support Nova+KVM style device
paths, update so we also support Nova+ESXi and leave the code such that
new pattern additions are easy.
Neutron's API ignores unknown paramaters. When listing pools etc, K8
attempts to filter on "LoadBalancerID", which is not a valid filter.
As such, it is ignored by Neutron, and a list of all pools is
returned. K8 then proceeds to update each of the pools.
Instead, we now double check the resources really belong to the LB
we're trying to update.
At master volume reconciler, the information about which volumes are
attached to nodes is cached in actual state of world. However, this
information might be out of date in case that node is terminated (volume
is detached automatically). In this situation, reconciler assume volume
is still attached and will not issue attach operation when node comes
back. Pods created on those nodes will fail to mount.
This PR adds the logic to periodically sync up the truth for attached volumes kept in the actual state cache. If the volume is no longer attached to the node, the actual state will be updated to reflect the truth. In turn, reconciler will take actions if needed.
To avoid issuing many concurrent operations on cloud provider, this PR
tries to add batch operation to check whether a list of volumes are
attached to the node instead of one request per volume.
More details are explained in PR #33760
Automatic merge from submit-queue
vSphere cloud provider: re-use session for vCenter logins
This change allows for the re-use of a vCenter client session. Addresses #34491
In order to be able to use new mounter library, this PR adds the
mounterPath flag to kubelet which passes the flag to the mount
interface. If flag is empty, mount uses default mount path.
Automatic merge from submit-queue
Loadbalanced client src ip preservation enters beta
Sounds like we're going to try out the proposal (https://github.com/kubernetes/kubernetes/issues/30819#issuecomment-249877334) for annotations -> fields on just one feature in 1.5 (scheduler). Or do we want to just convert to fields right now?
'names' is an array of FQDNs. 'instances' is a map indexed by canonicalized
name. Clearly these two won't always match, so when building the final
instance array to return, make sure to look up map entries by their canonicalized
name.
In the below example, "ocp-master-5pob" is clearly found as a GCE instance
but when building the final instance array it cannot be matched as the code
is looking for "ocp-master-5pob.c.ose-refarch.internal" instead. The node
is then deleted from the cluster as it cannot be found by the cloud provider.
gce.go:2519] ### getInstancesByNames([ocp-master-5pob.c.ose-refarch.internal]): initial node prefix ocp-
gce.go:2530] ### getInstancesByNames([ocp-master-5pob.c.ose-refarch.internal]): looking for instances map[ocp-master-5pob:<nil>]
gce.go:2533] ### getInstancesByNames([ocp-master-5pob.c.ose-refarch.internal]): getting zone 'europe-west1-c' (remaining 1)
gce.go:2563] ### getInstancesByNames([ocp-master-5pob.c.ose-refarch.internal]): instance name <omitted> not requested
gce.go:2563] ### getInstancesByNames([ocp-master-5pob.c.ose-refarch.internal]): instance name <omitted> not requested
gce.go:2533] ### getInstancesByNames([ocp-master-5pob.c.ose-refarch.internal]): getting zone 'europe-west1-b' (remaining 1)
gce.go:2563] ### getInstancesByNames([ocp-master-5pob.c.ose-refarch.internal]): instance name <omitted> not requested
gce.go:2576] ### getInstancesByNames([ocp-master-5pob.c.ose-refarch.internal]): found instance 'ocp-master-5pob' remaining 0
gce.go:2563] ### getInstancesByNames([ocp-master-5pob.c.ose-refarch.internal]): instance name <omitted> not requested
gce.go:2533] ### getInstancesByNames([ocp-master-5pob.c.ose-refarch.internal]): getting zone 'europe-west1-d' (remaining 0)
gce.go:2588] Failed to retrieve instance: "ocp-master-5pob.c.ose-refarch.internal"
gce.go:2624] ### getInstanceByName(ocp-master-5pob.c.ose-refarch.internal): got []: instance not found
gce.go:2626] getInstanceByName/multiple-zones: failed to get instance ocp-master-5pob.c.ose-refarch.internal; err: instance not found
nodecontroller.go:587] Deleting node (no longer present in cloud provider): ocp-master-5pob.c.ose-refarch.internal
nodecontroller.go:664] Recording Deleting Node ocp-master-5pob.c.ose-refarch.internal because it's not present according to cloud provider event message for node ocp-master-5pob.c.ose-refarch.internal
Automatic merge from submit-queue
azure: lower log priority for skipped nic update message
**What this PR does / why we need it**: Very minor, just wanted to remove some log noise I introduced in #34526.
I chose `V(3)` since it aligns with the other nicupdate message printed out here, and will be hidden for the usual default of `--v=2`.
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
NONE
```
This allows security groups to be created and attached to the neutron
port that the loadbalancer is using on the subnet.
The security group ID that is assigned to the nodes needs to be
provided, to allow for traffic from the loadbalancer to the nodePort
to be refelected in the rules.
This adds two config items to the LoadBalancer options -
ManageSecurityGroups (bool)
NodeSecurityGroupID (string)
- test pd-ssd and pd-standard on GCE,
- test all four volume types on AWS
- test just the default volume type on OpenStack (right now, there is no API
to get list of them)
Automatic merge from submit-queue
openstack: Support config-drive and improve CurrentNodeName, GetZone
This PR adds support for fetching local instance metadata via config-drive (as well as querying metadata service), and surfaces some additional metadata information (from either source):
- `CurrentNodeName` now returns the OpenStack instance name, rather than the current hostname (they might not be the same)
- `GetZone` includes availability zone label in `FailureDomain`
Thanks to @kiall for a WIP implementation of the latter.
Automatic merge from submit-queue
OpenStack LBaaSV2: EnsureLoadBalancer now updates instead of recreates existing LBs
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**: Current LBaaSV2 integration recreates existing LBs and causes service downtime and floating ip rotation. New implementation updates LBs without service downtime or any ip rotation.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#32794
**Special notes for your reviewer**: I really need this before we can move to production with kubernetes. Getting this to v1.4 would be really great. I have performed plenty of testing; lb and listener creation, port changing and listener update, multiple listeners for multi-port LBs, and deletion. Seems to work flawlessly.
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
```
Previously the OpenStack provider just returned the hostname in
CurrentNodeName. With this change, we return the local OpenStack
instance name, as the API intended.
Config-drive is an alternate no-network method for publishing local
instance metadata on OpenStack. This change implements support for
fetching data from config-drive, and tries it before querying the
network metadata service (since config-drive will fail quickly if not
available).
Note config-drive involves mounting the filesystem with label
"config-2", so anyone using config-drive and running kubelet in a
container will need to ensure /dev/disk/by-label/config-2 is available
inside the container (read-only).
Contination of #1111
I tried to keep this PR down to just a simple search-n-replace to keep
things simple. I may have gone too far in some spots but its easy to
roll those back if needed.
I avoided renaming `contrib/mesos/pkg/minion` because there's already
a `contrib/mesos/pkg/node` dir and fixing that will require a bit of work
due to a circular import chain that pops up. So I'm saving that for a
follow-on PR.
I rolled back some of this from a previous commit because it just got
to big/messy. Will follow up with additional PRs
Signed-off-by: Doug Davis <dug@us.ibm.com>
Because of how our test environment was setup, we didn’t notice that we were assuming the load balancer hosts list to be IP addresses, while they actually are hostnames.
Also updated some comments and added a check to prevent trying to release a public IP if we don’t have one.
We had another bug where we confused the hostname with the NodeName.
To avoid this happening again, and to make the code more
self-documenting, we use types.NodeName (a typedef alias for string)
whenever we are referring to the Node.Name.
A tedious but mechanical commit therefore, to change all uses of the
node name to use types.NodeName
Also clean up some of the (many) places where the NodeName is referred
to as a hostname (not true on AWS), or an instanceID (not true on GCE),
etc.
Automatic merge from submit-queue
vSphere cloud provider: ExternalID/InstanceID not returning appropriate error for non-existing VM
Addresses #33215.
When vCenter returns error vm not found, this is now being translated to
the appropriate error 'cloudprovider.InstanceNotFound' which indicates
to Kubernetes node controller that the VM is in fact not found.
Automatic merge from submit-queue
Fixed a bug that causes k8s to delete all healthmonitors on your OpenStack tenant
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
The OpenStack LBaaS v2 api does not support filtering health monitors by pool_id, so /lbaas/healthmonitors?pool_id=abc123 will always return all health monitors in your OpenStack tenant.
This presents a problem when, in the very next block of code, we loop over the list of monitorIDs and delete them one-by-one. This will delete all the health monitors in your tenant without warning.
Fortunately, we already got the healthmonitor IDs when we built the list of pools. Using those, we can delete only those healthmonitors associated with our pool(s).
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
The main issue here was the use of v2_monitors.List(lbaas.network, v2_monitors.ListOpts{PoolID: poolID}). This is trying to filter healthmonitors by pool_id, but that is not supported by the API. It creates a call like /lbaas/healthmonitors?pool_id=abc123. The API server ignores the pool_id parameter and returns a list of all healthmonitors (which k8s then tries to delete).
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
```
Automatic merge from submit-queue
Do not report error when deleting an attached volume
Persistent volume controller should not send warning events to a PV and mark the PV as failed when the volume is still attached.
This happens when a user quickly deletes a pod and associated PVC - PV is slowly detaching, while the PVC is already deleted and the PV enters Failed phase.
`Deleter.Deleter` can now return `tryAgainError`, which is sent as INFO to the PV to let the user know we did not forget to delete the PV, however the PV stays in Released state. The controller tries again in the next sync (15 seconds by default).
Fixes#31511
Addresses #33215.
When vCenter returns error vm not found, this is now being translated to
the appropriate error 'cloudprovider.InstanceNotFound' which indicates
to Kubernetes node controller that the VM is in fact not found.
When we are mounting a lot of volumes, we frequently hit rate limits.
Reduce the frequency with which we poll the status; introduces a bit of
latency but probably matches common attach times pretty closely, and
avoids causing rate limit problems everywhere.
Also, we now poll for longer, as when we timeout, the volume is in an
indeterminate state: it may be about to complete. The volume controller
can tolerate a slow attach/detach, but it is harder to tolerate the
indeterminism.
Finally, we ignore a sequence of errors in DescribeVolumes (up to 5 in a
row currently). So we will eventually return an error, but a one
off-failure (e.g. due to rate limits) does not cause us to spuriously
fail.
Bump version of golang.org/x/oauth2
Vendor google.golang.org/cloud/
Vendor google.golang.org/api/
Vendor cloud.google.com/go/compute/
Replace google.golang.org/cloud with cloud.google.com/go/
Fixes#30069
Automatic merge from submit-queue
update pkg/cloudprovider OWNERS to spread the review load
This is going to make the mungebot start assigning reviews in your cloudprovider packages.
fyi @runseb @dagnello @imkin @anguslees @dagnello
Automatic merge from submit-queue
Do not allow creation of GCE PDs in unmanaged zones.
Such volumes then couldn't be deleted as `getDiskByNameUnknownZone` goes through managed zones only.
Fixes: #31948
@kubernetes/rh-storage
@saad-ali, PTAL.
Automatic merge from submit-queue
Fix: Dynamic provisioning for vSphere
This PR does the following,
1. Fixes an error 'A specified parameter was not correct:' occurs while dynamically provisioning the volumes.
2. Adds VSAN support for dynamic provisioning.
Automatic merge from submit-queue
Service behaviour not correct when ESIPP annotation is toggled or deleted
Fixes#31649
GCE controller: Correctly add/remove http health checks based on whether the annotation was added or removed
kube-proxy: Create/Delete XLB local-endpoints balancer chain and change jump actions from KUBE-FW- chain based on the annotation.
Automatic merge from submit-queue
vSphere Cloud provider null pointer exception
This PR addresses issue #31823.
SelectByType function in govmomi will panic if deviceType is not Array,
Chan, Map, Ptr, or Slice. Also checking if vmDevices or vm are nil,
there is nothing to cleanup.
Automatic merge from submit-queue
Make a vSphere cluster the failure_zone
vSphere cloud provider returns the FailureZone as Cluster, if the VM belongs to a ResourcePool under a Cluster.
fixes: #30933
* Currently the vSphere cloud provider treats Datacenter as the failure
Zone. This doesn't necessarily work since in the current implemention
Kubernetes nodes cannot span Datacenters.
* This change introduces Clusters as the failure zone, while treating
Datacenters as Regions
* Also updated tests for Zones
Automatic merge from submit-queue
Typos and englishify pkg/cloudprovider + pkg/dns + pkg/kubectl
**What this PR does / why we need it**: Just fixed some typos + "englishify" in pkg/cloudprovider + pkg/dns + pkg/kubectl
**Which issue this PR fixes** : None
**Special notes for your reviewer**: It's just fixes typos
**Release note**: `NONE`
Automatic merge from submit-queue
retry oauth token fetch in gce cloudprovider
Fixes https://github.com/kubernetes/kubernetes/issues/31560
The oauth client fetches a token on the initial request of that client. Let's warm the cache.
cc @goltermann @lavalamp
Kubernetes Submit Queue