Automatic merge from submit-queue
Implement API usage metrics for gce storage
**What this PR does / why we need it**:
This PR implements support for emitting metrics from GCE about storage operations.
**Which issue this PR fixes**
Fixes https://github.com/kubernetes/features/issues/182
**Release note**:
```
Add support for emitting metrics from GCE cloudprovider about storage operations.
```
Automatic merge from submit-queue
Print dereferenced pod status fields when logging status update
Before: "Terminated:0xc421932af0"
After:"Terminated:&ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:0001-01-01 00:00:00 +0000 UTC,FinishedAt:2017-03-07 14:50:48 -0500 EST,ContainerID:docker://bd453bb969264b3ace2b3934a568af7679a0d51fee543a5f8a82429ff654970e,}"
"Ignoring same status for pod" messages already print status fully, these "Status for pod updated" messages should too IMO
```release-note
NONE
```
Automatic merge from submit-queue
Create subPaths and set their permissions like we do mountPaths
fixes https://github.com/kubernetes/kubernetes/issues/41638
If a subPath does not exist at the time MountVolume.Setup happens, SetVolumeOwnership will not have walked to the subPath and set appropriate permissions on it, leading to the above issue
So later, at makeMounts when we are parsing subPaths, let's create all subPaths and set their permissions according to how the parent mountPath looks.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42617, 43247, 43509, 43644, 43820)
[GCE] Support legacy-https and generic health checks
**What this PR does / why we need it**:
- Adds CRUD functions to manage `compute.HttpsHealthChecks`
The legacy HTTPS healthchecks will be used by the GLBC (GCE Load balancer Controller)
- Adds CRUD functions to manage `compute.HealthChecks`
These are required for the internal load balancer
- Removes the logic that disregards NotFound errors on DeleteHttpHealthChecks as this is useful information for callers. Here are the three known invocations within kubernetes:
[gce/gce_loadbalancer.go#L457](bc6e77d42f/pkg/cloudprovider/providers/gce/gce_loadbalancer.go (L457)): Only prints warning that HC wasn't deleted -> acceptable
[gce/gce_loadbalancer.go#L465](bc6e77d42f/pkg/cloudprovider/providers/gce/gce_loadbalancer.go (L465)): Err is ignored if not nil -> acceptable
[e2e/framework/ingress_utils.go#L530](bc6e77d42f/test/e2e/framework/ingress_utils.go (L530)): Already checks if is NotFound error -> acceptable
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
Step one of https://github.com/kubernetes/ingress/issues/494
Step one of #33483
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
validate activeDeadlineSeconds in rs/rc
**What this PR does / why we need it**:
if setting activeDeadlineSeconds, deployment will continuously created new pods after old pod dies.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#38684
**Special notes for your reviewer**:
**Release note**:
```release-note
ActiveDeadlineSeconds is validated in workload controllers now, make sure it's not set anywhere (it shouldn't be set by default and having it set means your controller will restart the Pods at some point)
```
Automatic merge from submit-queue (batch tested with PRs 41135, 43699)
azure: reduce polling delay for all Azure clients
**What this PR does / why we need it**: Reduces the polling delay for all azure clients to 5 seconds. This should speed up some additional operations at the cost of some quota.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: n/a
**Special notes for your reviewer**: This stacks on top of #43697, please only look at latest commit. Alternatively I can rebase this on top of master.
**Release note**:
```release-note
azure: all clients poll duration is now 5 seconds
```
I don't think the StorageClient polls anywhere, and it's a different type of client, so I didn't touch it.
Automatic merge from submit-queue
fix deleted typos
**What this PR does / why we need it**:
**Which issue this PR fixes** :
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
Automatic merge from submit-queue
unit test for kubectl config unset
**What this PR does / why we need it**:
unit test for kubectl config unset
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
think you
**Release note**:
```release-note
```
updating with PR changes requested.
latest changes to having short for human readable only, and error cases moved a bit to the end.
rebase fixes
latest pr. changes.
small change moving return nil out of switch.
updated the nil check for the error in the humanreadable case.
more optimization in humanreadable code.
pushed up current test changes, this is purely temporary
finished writing tests
updated test and function names.
changed output extensions from .sh to output.
updated version, version struct now just called Version and not VersionObj.
made a few changes to testing.
fixed testing issues, created better test and cleanup
go format change.
The cloudprovider is being refactored out of kubernetes core. This is being
done by moving all the cloud-specific calls from kube-apiserver, kubelet and
kube-controller-manager into a separately maintained binary(by vendors) called
cloud-controller-manager. The Kubelet relies on the cloudprovider to detect information
about the node that it is running on. Some of the cloudproviders worked by
querying local information to obtain this information. In the new world of things,
local information cannot be relied on, since cloud-controller-manager will not
run on every node. Only one active instance of it will be run in the cluster.
Today, all calls to the cloudprovider are based on the nodename. Nodenames are
unqiue within the kubernetes cluster, but generally not unique within the cloud.
This model of addressing nodes by nodename will not work in the future because
local services cannot be queried to uniquely identify a node in the cloud. Therefore,
I propose that we perform all cloudprovider calls based on ProviderID. This ID is
a unique identifier for identifying a node on an external database (such as
the instanceID in aws cloud).
Automatic merge from submit-queue (batch tested with PRs 42835, 42974)
VSAN policy support for storage volume provisioning inside kubernetes
The vsphere users will have the ability to specify custom Virtual SAN Storage Capabilities during dynamic volume provisioning. You can now define storage requirements, such as performance and availability, in the form of storage capabilities during dynamic volume provisioning. The storage capability requirements are converted into a Virtual SAN policy which are then pushed down to the Virtual SAN layer when a storage volume (virtual disk) is being created. The virtual disk is distributed across the Virtual SAN datastore to meet the requirements.
For example, User creates a storage class with VSAN storage capabilities:
> kind: StorageClass
> apiVersion: storage.k8s.io/v1beta1
> metadata:
> name: slow
> provisioner: kubernetes.io/vsphere-volume
> parameters:
> hostFailuresToTolerate: "2"
> diskStripes: "1"
> cacheReservation: "20"
> datastore: VSANDatastore
The vSphere Cloud provider provisions a virtual disk (VMDK) on VSAN with the policy configured to the disk.
When you know storage requirements of your application that is being deployed on a container, you can specify these storage capabilities when you create a storage class inside Kubernetes.
@pdhamdhere @tthole @abrarshivani @divyenpatel
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 42835, 42974)
remove legacy insecure port options from genericapiserver
The insecure port has been a source of problems and it will prevent proper aggregation into a cluster, so the genericapiserver has no need for it. In addition, there's no reason for it to be in the main kube-apiserver flow either. This pull removes it from genericapiserver and removes it from the shared kube-apiserver code. It's still wired up in the command, but its no longer possible for someone to mess up and start using in mainline code.
@kubernetes/sig-api-machinery-misc @ncdc
Automatic merge from submit-queue (batch tested with PRs 42087, 43383, 43622)
move category expansion out of restmapper
RESTMapping isn't related to CategoryExpansion (the bit that expands "all" into items to be RESTMapped). This provides that separation and simplifies the RESTMapper interface.
@kubernetes/sig-cli-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 43681, 40423, 43562, 43008, 43381)
Openstack cinder v1/v2/auto API support
**What this PR does / why we need it**:
It adds support for v2 cinder API + autodetection of available cinder API level (as in LBs).
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#39572
**Special notes for your reviewer**:
Based on work by @anguslees. The first two commits are just rebased from https://github.com/kubernetes/kubernetes/pull/36344 which already had a lgtm by @jbeda
**Release note**:
```
Add support for v2 cinder API for openstack cloud provider. By default it autodetects the available version.
```
Automatic merge from submit-queue (batch tested with PRs 42900, 43044, 42896, 43308, 43621)
require codecfactory
The genericapiserver requires a codec to start. Help new comers to the API by forcing them to set it when they create a new config.
Automatic merge from submit-queue (batch tested with PRs 42900, 43044, 42896, 43308, 43621)
Convert PrefixWriter to an interface
Convert PrefixWriter to an interface to allow other packages to create instances. This is needed for downstream projects such as OpenShift.
cc @sttts @smarterclayton @oulinbao @deads2k @liggitt @kargakis @soltysh @kubernetes/sig-cli-pr-reviews
Automatic merge from submit-queue
remove extra function call
We have read user info from context in previous line. No need to call
this function again.
**Special notes for your reviewer**:
**Release note**:
```NONE
```
Automatic merge from submit-queue
fix createProvisionedPV result err judgenment bug
When ctrl.kubeClient.Core().PersistentVolumes().Create(volume) returns no err, and storeVolumeUpdate() fails, we save PV sucessfully ,but here err is not nil,we should not run the codes next in block if err != nil {} to delete the storage asset.
same in the deletion retries below
change the err names to make it clear
**Release note**:
```release-note
NONE
```
@jsafrane @saad-ali PTAL. Thanks
Automatic merge from submit-queue
Fix PVC Annotations
annDynamicallyProvisioned is added to PV, while PVC has annStorageProvisioner.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
kubelet: check and enforce minimum docker api version
**What this PR does / why we need it**:
This PR adds enforcing a minimum docker api version (same with what we have do for dockertools).
**Which issue this PR fixes**
Fixes#42696.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 43429, 43416, 43312, 43141, 43421)
Create controller to auto register TPRs with the aggregator
Builds on https://github.com/kubernetes/kubernetes/pull/42732 (already lgtmed)
Creates a simple controller to wire TPRs with the API Service autoregistration controller.
@kubernetes/sig-api-machinery-misc @ncdc
Automatic merge from submit-queue (batch tested with PRs 43429, 43416, 43312, 43141, 43421)
add singular resource names to discovery
Adds the singular resource name to our resource for discovery. This is something we've discussed to remove our pseudo-pluralization library which is unreliable even for english and really has no hope of properly handling other languages or variations we can expect from TPRs and aggregated API servers.
This pull simply adds the information to discovery, it doesn't not re-wire any RESTMappers.
@kubernetes/sig-cli-misc @kubernetes/sig-apimachinery-misc @kubernetes/api-review
```release-note
API resource discovery now includes the `singularName` used to refer to the resource.
```
Automatic merge from submit-queue (batch tested with PRs 43429, 43416, 43312, 43141, 43421)
remove duplicated import
**What this PR does / why we need it**:
**Which issue this PR fixes**:
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 43378, 43216, 43384, 43083, 43428)
Fix tiny typo
**What this PR does / why we need it**:
**Which issue this PR fixes**
Fix type typo introduced by PR #43368.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 43378, 43216, 43384, 43083, 43428)
Remove unused argument of 'groupJobsByParent' in cronjob controller
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 43378, 43216, 43384, 43083, 43428)
Kubelet:rkt Create any missing hostPath Volumes
When using a `hostPath` inside the `Pod.spec.volumes`, this PR allows to creates any missing directory on the node.
**What this PR does / why we need it**:
With rkt as the container runtime we cannot use `hostPath` volumes if the directory is missing.
**Special notes for your reviewer**:
This PR follows [#39965](https://github.com/kubernetes/kubernetes/pull/39965)
The labels should be
> area/rkt
> area/kubelet
Automatic merge from submit-queue (batch tested with PRs 43378, 43216, 43384, 43083, 43428)
Do not reformat devices with partitions
`lsblk` reports FSTYPE of devices with partition tables as empty string `""`,
which is indistinguishable from empty devices. We must look for dependent
devices (i.e. partitions) to see that the device is really empty and report
error otherwise.
The main point of this patch is to run `lsblk` without `"-n"`, i.e. print all
dependent devices and check it output.
Sample output:
```
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
10s 10s 1 default-scheduler Normal Scheduled Successfully assigned testpod to ip-172-18-11-149.ec2.internal
2s 2s 1 kubelet, ip-172-18-11-149.ec2.internal Warning FailedMount MountVolume.MountDevice failed for volume "kubernetes.io/aws-ebs/vol-0fa9da8b91913b187" (spec.Name: "vol") pod "b74f68c5-0d6a-11e7-9233-0e11251010c0" (UID: "b74f68c5-0d6a-11e7-9233-0e11251010c0") with: failed to mount the volume as "ext4", it already contains unknown data, probably partitions. Mount error: mount failed: exit status 32
Mounting command: mount
Mounting arguments: /dev/xvdbb /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/vol-0fa9da8b91913b187 ext4 [defaults]
Output: mount: wrong fs type, bad option, bad superblock on /dev/xvdbb,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so.
```
Without this patch, the device would be reformatted and all data in the device partitions would be lost.
Fixes#13212
Release note:
```release-note
NONE
```
@kubernetes/sig-storage-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41297, 42638, 42666, 43039, 42567)
Add the ability to lock on ConfigMaps to support HA for self hosted components
**What this PR does / why we need it**:
This PR add the ability to lock on ConfigMap objects. As we self-host more and more components, ConfigMaps provides a useful resource to lock on that applies to just the component that is being added.
This allows components such as scheduler+controller manager, etc. to have a simple mechanism to allow to active-passive HA without the cost of endpoint updates.
**Related Issues**
xref: https://github.com/kubernetes/client-go/issues/28
**Special notes for your reviewer**:
We may want to deprecate endpoints locking at some point.
**Release note**:
```
Add the ability to lock on ConfigMaps to support HA for self hosted components
```
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews @mikedanese
Automatic merge from submit-queue (batch tested with PRs 42998, 42902, 42959, 43020, 42948)
[cli] fix Generator's error messages
Invalid variables are used when format error messages. This change
fixes them.
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 42998, 42902, 42959, 43020, 42948)
Add Host field to TCPSocketAction
Currently, TCPSocketAction always uses Pod's IP in connection. But when a pod uses the host network, sometimes firewall rules may prevent kubelet from connecting through the Pod's IP.
This PR introduces the 'Host' field for TCPSocketAction, and if it is set to non-empty string, the probe will be performed on the configured host rather than the Pod's IP. This gives users an opportunity to explicitly specify 'localhost' as the target for the above situations.
```release-note
Add Host field to TCPSocketAction
```
Automatic merge from submit-queue
Unit test quota for nodeport associated with loadbalancer
**What this PR does / why we need it**:
This PR adds unit tests to ensure node ports associated with loadbalancers are charged to quota appropriately. The original PR that added that feature to quota lacked a unit test (https://github.com/kubernetes/kubernetes/pull/39364)
Automatic merge from submit-queue
allow fallthrough handling from go-restful routes
This sets up the gorestful routes to fall through to a default handler and reorders the API to be ahead of the other endpoints. This makes it possible to cleanly support cases of "match, fail, try this other handler" which we'll need for API server composition.
@kubernetes/sig-api-machinery-pr-reviews @ncdc
Automatic merge from submit-queue (batch tested with PRs 42672, 42770, 42818, 42820, 40849)
Return early from eviction debug helpers if !glog.V(3)
Should keep us from running a bunch of loops needlessly.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 43653, 43654, 43652)
CRI: Check nil pointer to avoid kubelet panic.
When working on the containerd kubernetes integration, I casually returns an empty `sandboxStatus.Linux{}`, but it cause kubelet to panic.
This won't happen when runtime returns valid data, but we should not make the assumption here.
/cc @yujuhong @feiskyer
When ctrl.kubeClient.Core().PersistentVolumes().Create(volume) returns no err, but storeVolumeUpdate() failed, we save PV sucessfully ,but here err is not nil,
we should not run the codes next in block if err != nil {}
same in the deletion retries below
Automatic merge from submit-queue (batch tested with PRs 43642, 43170, 41813, 42170, 41581)
Enable storage class support in Azure File volume
**What this PR does / why we need it**:
Support StorageClass in Azure file volume
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Support StorageClass in Azure file volume
```
Automatic merge from submit-queue (batch tested with PRs 43642, 43170, 41813, 42170, 41581)
Be able to specify the timeout to wait for pod for kubectl logs/attach
Fixes https://github.com/kubernetes/kubernetes/issues/41786
current flag is `get-pod-timeout`, we can have a discussion if you have better one, default unit is seconds, above 0
@soltysh @kargakis ptal, thanks
@kubernetes/sig-cli-feature-requests
Automatic merge from submit-queue (batch tested with PRs 42522, 42545, 42556, 42006, 42631)
optimize the binding logic of bindClaimToVolume
extract var shouldSetBoundByController and do not need to judge volumename twice
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42522, 42545, 42556, 42006, 42631)
Fixes MountVolume.NewMounter errors not displayed to users via describe events
Fixes#42004
This fixes the problem of mount errors being eaten and not displayed to users again. Specifically erros caught in MountVolume.NewMounter (like missing endpoints, etc...)
Current behavior for any mount failure:
```
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
12m 12m 1 default-scheduler Normal Scheduled Successfully assigned glusterfs-bb-pod1 to 127.0.0.1
10m 1m 5 kubelet, 127.0.0.1 Warning FailedMount Unable to mount volumes for pod "glusterfs-bb-pod1_default(67c9dfa7-f9f5-11e6-aee2-5254003a59cf)": timeout expired waiting for volumes to attach/mount for pod "default"/"glusterfs-bb-pod1". list of unattached/unmounted volumes=[glusterfsvol]
10m 1m 5 kubelet, 127.0.0.1 Warning FailedSync Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "default"/"glusterfs-bb-pod1". list of unattached/unmounted volumes=[glusterfsvol]
```
New Behavior:
For example on glusterfs - deliberately didn't create endpoints, now correct message is displayed:
```
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
2m 2m 1 default-scheduler Normal Scheduled Successfully assigned glusterfs-bb-pod1 to 127.0.0.1
54s 54s 1 kubelet, 127.0.0.1 Warning FailedMount Unable to mount volumes for pod "glusterfs-bb-pod1_default(8edd2c25-fa09-11e6-92ae-5254003a59cf)": timeout expired waiting for volumes to attach/mount for pod "default"/"glusterfs-bb-pod1". With error timed out waiting for the condition. list of unattached/unmounted volumes=[glusterfsvol]
54s 54s 1 kubelet, 127.0.0.1 Warning FailedSync Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "default"/"glusterfs-bb-pod1". With error timed out waiting for the condition. list of unattached/unmounted volumes=[glusterfsvol]
2m 6s 814 kubelet, 127.0.0.1 Warning FailedMount MountVolume.NewMounter failed for volume "kubernetes.io/glusterfs/8edd2c25-fa09-11e6-92ae-5254003a59cf-glusterfsvol" (spec.Name: "glusterfsvol") pod "8edd2c25-fa09-11e6-92ae-5254003a59cf" (UID: "8edd2c25-fa09-11e6-92ae-5254003a59cf") with: endpoints "glusterfs-cluster" not found
```
Automatic merge from submit-queue (batch tested with PRs 42522, 42545, 42556, 42006, 42631)
Use pod sandbox id in checkpoint
**What this PR does / why we need it**: we should log out sandbox id when checkpoint error
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 42237, 42297, 42279, 42436, 42551)
should replace errors.New(fmt.Sprintf(...)) with fmt.Errorf(...)
Signed-off-by: yupengzte <yu.peng36@zte.com.cn>
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 41139, 41186, 38882, 37698, 42034)
Add support for bring-your-own ip address for Services on Azure
@colemickens @codablock
Automatic merge from submit-queue (batch tested with PRs 41139, 41186, 38882, 37698, 42034)
Make kubelet never delete files on mounted filesystems
With bug #27653, kubelet could remove mounted volumes and delete user data.
The bug itself is fixed, however our trust in kubelet is significantly lower.
Let's add an extra version of RemoveAll that does not cross mount boundary
(rm -rf --one-file-system).
It calls lstat(path) three times for each removed directory - once in
RemoveAllOneFilesystem and twice in IsLikelyNotMountPoint, however this way
it's platform independent and the directory that is being removed by kubelet
should be almost empty.
Automatic merge from submit-queue (batch tested with PRs 41139, 41186, 38882, 37698, 42034)
create configmap from-env-file
Allow ConfigMaps to be created from Docker based env files.
See proposal https://github.com/kubernetes/community/issues/165
**Release-note:**
```release-note
1. create configmap has a new option --from-env-file that populates a configmap from file which follows a key=val format for each line.
2. create secret has a new option --from-env-file that populates a configmap from file which follows a key=val format for each line.
```
Automatic merge from submit-queue
config set cluster api value
Related bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1420280
**Release note**:
```release-note
release-note-none
```
This patch sets a specified api version if one is given while setting cluster values in kubeconfig
cc @AdoHe @pweil-
Automatic merge from submit-queue
Add approvers to the aws OWNERS file
Without this it was picking up reviewers from a much higher directory.
```release-note
NONE
```
Automatic merge from submit-queue
fix typos
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
small change to ControlleeExpectations result judgement
GetByKey() will never return err, so err != nil {} is redundant,remove it and remove the err return too
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Add gnufied as reviewer for pkg/volume
I have helped review and contributed code to this
area already.
cc @saad-ali @jsafrane @childsb
Automatic merge from submit-queue
fc: Drop multipath.conf snippet
**What this PR does / why we need it**:
Removes multipath.conf - The code does not make use of it - or ensure s that it's getting used - and it should in addition be handled elsewehre.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
A minimalistic multipath.conf got written, but it was useless, as
it is unclear if multipathd is running and there was also no
config reload triggered.
This patch drops this snippet. In general it's probably a better idea
to leave the multipath.conf to the component managing the host.
Signed-off-by: Fabian Deutsch <fabiand@fedoraproject.org>
Out-of-tree external provisioners have the same purpose as in-tree provisioners. As external provisioners work with PV and PVC datastructures it's an advantage to import certain Kubernetes packages instead of copy-pasting the Kubernetes code.
That's why the constants are made public so that they can be used in an external provisioner.
Each major interface is now in its own file. Any package private
functions that are only referenced by a particular module was also moved
to the corresponding file. All common helper functions were moved to
gce_util.go.
This change is a pure movement of code; no semantic changes were made.
Automatic merge from submit-queue
Fix adding disks to more than one scsi adapter. Fixes#42399
**What this PR does / why we need it**: Allows a single node to use more than 16 disks.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#42399
**Special notes for your reviewer**:
**Release note**:
```release-note
Fix adding disks to more than one scsi adapter.
```
Auto-generated via:
git grep -l [Ss]uccesfully | xargs sed -ri 's/([sS])uccesfully/\1uccessfully/g'
I noticed this when running kube-scheduler with --v4 and it is annoying.
Then manually reverted changed to the vendored bits.
Automatic merge from submit-queue (batch tested with PRs 43533, 43539)
kuberuntime: don't override the pod IP for pods using host network
This fixes the issue of not passing pod IP via downward API for host network pods.
Automatic merge from submit-queue (batch tested with PRs 43465, 43529, 43474, 43521)
kubelet/cni: hook network plugin Status() up to CNI network discovery
Ensure that the plugin returns NotReady status until there is a
CNI network available which can be used to set up pods.
Fixes: https://github.com/kubernetes/kubernetes/issues/43014
I think the only reason it wasn't done like this in the first place was that the dynamic "reread /etc/cni/net.d every 10s forever" was added long after the Status() hook was. What do you think?
@freehan @caseydavenport @luxas @jbeda
Automatic merge from submit-queue
Disable readyReplicas validation for Deployments
Because there is no field in 1.5, when we update to 1.6 and the
controller tries to update the Deployment, it will be denied by
validation because the pre-existing availableReplicas field is greater
than readyReplicas (normally readyReplicas should always be greater or
equal).
Fixes https://github.com/kubernetes/kubernetes/issues/43392
@kubernetes/sig-apps-bugs
Because there is no field in 1.5, when we update to 1.6 and the
controller tries to update the Deployment, it will be denied by
validation because the pre-existing availableReplicas field is greater
than readyReplicas (normally readyReplicas should always be greater or
equal).
Automatic merge from submit-queue
Add validation for affinities and taints/tolerations annotations.
It fixes annotations validation issues for pod/node affinities and taints/tolerations annotations for 1.5 to 1.6 upgrade tests as discussed in the issue https://github.com/kubernetes/kubernetes/issues/43228 .
@davidopp @derekwaynecarr @kubernetes/sig-scheduling-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 43481, 43419, 42741, 43480)
controller: work around milliseconds skew in AddAfter
AddAfter is not requeueing precisely after the provided time and may
skew for some millieseconds. This is really important because controllers
don't relist often so a missed check because of ms difference is
essentially dropping the key. For example, in [1] the test requeues a
Deployment for a progress check after 10s[2] but the Deployment is synced
9ms earlier ending up in the controller not recognizing the Deployment as
failed thus dropping it from the queue w/o any error. The drop is fixed by
forcing the controller to resync the Deployment but we are going to resync
after the full duration.
@deads2k if you don't like this I am going to handle this on a case by case basis
[1] https://github.com/kubernetes/kubernetes/issues/39785#issuecomment-279959133
[2] c48b2cab0f/test/e2e/deployment.go (L1122)
Automatic merge from submit-queue
Rate limit HPA controller to sync period
Since the HPA controller pulls information from an external source that
makes no guarantees about consistency, it's possible for the HPA
to get into an infinite update loop -- if the metrics change with
every query, the HPA controller will run it's normal reconcilation,
post a status update, see that status update itself, fetch new metrics,
and if those metrics are different, post another status update, and
repeat. This can lead to continuously updating a single HPA.
By rate-limiting each HPA to once per sync interval, we prevent this
from happening.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Ensure slices are serialized as zero-length, not null
Fixes https://github.com/kubernetes/kubernetes/issues/43203 null serialization of slices to prevent NPE errors in clients that store and expect to receive non-null JSON values in these fields.
Ensures when we are converting to an external slice field that will be serialized even if empty (has `json` tag that does not include `omitempty`), we populate it with `[]`, not `nil`
Other places I considered putting this logic instead:
* When unmarshaling
* Would have to be done for both protobuf and ugorji
* Would still have to be done here (or on marshal) to handle cases where we construct objects to return
* When marshaling
* Would have to switch to use custom json marshaler (currently we use stdlib)
* When defaulting
* Defaulting isn't run on some fields, notably, pod template in rc/deployment spec
* Would still have to be done here (or on marshal) to handle cases where we construct objects to return
```release-note
API fields that previously serialized null arrays as `null` and empty arrays as `[]` no longer distinguish between those values and always output `[]` when serializing to JSON.
```
Automatic merge from submit-queue
Install a REJECT rule for nodeport with no backend
Rather than actually accepting the connection, REJECT. This will avoid
CLOSE_WAIT.
Fixes#43212
@justinsb @felipejfc @spiddy
Automatic merge from submit-queue
Use uid in config.go instead of pod full name.
For https://github.com/kubernetes/kubernetes/issues/43397.
In config.go, use pod uid in pod cache.
Previously, if we update the static pod, even though a new UID is generated in file.go, config.go will only reference the pod with pod full name, and never update the pod UID in the internal cache. This causes:
1) If we change container spec, kubelet will restart the corresponding container because the container hash is changed.
2) If we change pod spec, kubelet will do nothing.
With this fix, kubelet will always restart pod whenever pod spec (including container spec) is changed.
@yujuhong @bowei @dchen1107
/cc @kubernetes/sig-node-bugs
Automatic merge from submit-queue
Use Semantic.DeepEqual to compare DaemonSet template on updates
Switch to `Semantic.DeepEqual` when comparing templates on DaemonSet updates, since we can't distinguish between `null` and `[]` in protobuf. This avoids unnecessary DaemonSet pods restarts.
I didn't touch `reflect.DeepEqual` used in controller because it's close to release date, and the DeepEqual in the controller doesn't cause serious issues (except for maybe causing more enqueues than needed).
Fixes#43218
@liggitt @kargakis @lukaszo @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 42452, 43399)
Modify getInstanceByName to avoid calling getInstancesByNames
This PR modify getInstanceByname to loop through all management zones
directly instead of calling getInstancesByNames. Currently
getInstancesByNames use a node name prefix as a filter to list the
instances. If the prefix does not match, it will return all instances
which is very wasteful since getInstanceByName only query one instance
with a specific name.
Partially fix issue #42445
Automatic merge from submit-queue (batch tested with PRs 43398, 43368)
CRI: add support for dns cluster first policy
**What this PR does / why we need it**:
PR #29378 introduces ClusterFirstWithHostNet policy but only dockertools was updated to support the feature.
This PR updates kuberuntime to support it for all runtimes.
**Which issue this PR fixes**
fixes#43352
**Special notes for your reviewer**:
Candidate for v1.6.
**Release note**:
```release-note
NONE
```
cc @thockin @luxas @vefimova @Random-Liu
Automatic merge from submit-queue
Deflake TestSyncDeploymentDeletionRace
**What this PR does / why we need it**:
The cache was sometimes catching up while we were testing the case
where the cache is not yet caught up.
Before this fix, I could reproduce the failure with the following
command. After the fix, it passes.
```
go test -count 100000 -run TestSyncDeploymentDeletionRace
```
I checked the other controllers, and they all were already not starting informers for the deletion race test. I also checked that the deletion race tests for other controllers all pass with `-count 100000`.
**Which issue this PR fixes**:
Fixes#43390
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42659, 43370)
dockershim: process protocol correctly for port mapping
**What this PR does / why we need it**:
dockershim: process protocol correctly for port mapping.
**Which issue this PR fixes**
Fixes#43365.
**Special notes for your reviewer**:
Should be included in v1.6.
**Release note**:
```release-note
NONE
```
cc/ @Random-Liu @justinsb @kubernetes/sig-node-pr-reviews
The cache was sometimes catching up while we were testing the case
where the cache is not yet caught up.
Before this fix, I could reproduce the failure with the following
command. After the fix, it passes.
```
go test -count 100000 -run TestSyncDeploymentDeletionRace
```
lsblk reports FSTYPE of devices with partition tables as empty string "",
which is indistinguishable from empty devices. We must look for dependent
devices (i.e. partitions) to see that the device is really empty and report
error otherwise.
I checked that LVM, LUKS and MD RAID have their own FSTYPE in lsblk output,
so it should be only a partition table that has empty FSTYPE.
The main point of this patch is to run lsblk without "-n", i.e. print all
dependent devices and check if they're there.
PR #29378 introduces ClusterFirstWithHostNet, but docker doesn't support
setting dns options togather with hostnetwork. This commit rewrites
resolv.conf same as dockertools.
PR #29378 introduces ClusterFirstWithHostNet policy but only dockertools
was updated to support the feature. This PR updates kuberuntime to
support it for all runtimes.
Also fixes#43352.
Automatic merge from submit-queue
GC: Fix re-adoption race when orphaning dependents.
**What this PR does / why we need it**:
The GC expects that once it sees a controller with a non-nil
DeletionTimestamp, that controller will not attempt any adoption.
There was a known race condition that could cause a controller to
re-adopt something orphaned by the GC, because the controller is using a
cached value of its own spec from before DeletionTimestamp was set.
This fixes that race by doing an uncached quorum read of the controller
spec just before the first adoption attempt. It's important that this
read occurs after listing potential orphans. Note that this uncached
read is skipped if no adoptions are attempted (i.e. at steady state).
**Which issue this PR fixes**:
Fixes#42639
**Special notes for your reviewer**:
**Release note**:
```release-note
```
cc @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 42828, 43116)
Apply taint tolerations for NoExecute for all static pods.
Fixed https://github.com/kubernetes/kubernetes/issues/42753
**Release note**:
```
Apply taint tolerations for NoExecute for all static pods.
```
cc/ @davidopp
The GC expects that once it sees a controller with a non-nil
DeletionTimestamp, that controller will not attempt any adoption.
There was a known race condition that could cause a controller to
re-adopt something orphaned by the GC, because the controller is using a
cached value of its own spec from before DeletionTimestamp was set.
This fixes that race by doing an uncached quorum read of the controller
spec just before the first adoption attempt. It's important that this
read occurs after listing potential orphans. Note that this uncached
read is skipped if no adoptions are attempted (i.e. at steady state).
Automatic merge from submit-queue (batch tested with PRs 43313, 43257, 43271, 43307)
Fix AWS untagged instances
To revert to 1.5 behaviour we need to consider untagged
instances if no clusterID has been specified or found.
Fixes https://github.com/kubernetes/kubernetes/issues/43063
cc @justinsb
Automatic merge from submit-queue (batch tested with PRs 43313, 43257, 43271, 43307)
Remove 'all namespaces' meaning of empty list in PodAffinityTerm
Removes the distinction between `null` and `[]` for the PodAffinityTerm#namespaces field (option 4 discussed in https://github.com/kubernetes/kubernetes/issues/43203#issuecomment-287237992), since we can't distinguish between them in protobuf (and it's a less than ideal API)
Leaves the door open to reintroducing "all namespaces" function via a dedicated field or a dedicated token in the list of namespaces
Wanted to get a PR open and tests green in case we went with this option.
Not sure what doc/release-note is needed if the "all namespaces" function is not present in 1.6
Automatic merge from submit-queue
kubectl: Use v1.5-compatible ownership logic when listing dependents.
**What this PR does / why we need it**:
This restores compatibility between kubectl 1.6 and clusters running Kubernetes 1.5.x. It introduces transitional ownership logic in which the client considers ControllerRef when it exists, but does not require it to exist.
If we were to ignore ControllerRef altogether (pre-1.6 client behavior), we would introduce a new failure mode in v1.6 because controllers that used to get stuck due to selector overlap will now make progress. For example, that means when reaping ReplicaSets of an overlapping Deployment, we would risk deleting ReplicaSets belonging to a different Deployment that we aren't about to delete.
This transitional logic avoids such surprises in 1.6 clusters, and does no worse than kubectl 1.5 did in 1.5 clusters. To prevent this when kubectl 1.5 is used against 1.6 clusters, we can cherrypick this change.
**Which issue this PR fixes**:
Fixes#43159
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
Fix revision when SetDeploymentRevision
When some oldRSs be deleted or cleared(eg. revisionHistoryLimit set 0), the revision for SetDeploymentRevision is incorrect
Automatic merge from submit-queue
protobuf generation modifies types.go, which needs to be copied out
This was broken when we moved to the build container, but no one
noticed. Made it so that we get a test error if a field in a registered type has a json tag with no protobuf tag.
Fixes#35486
Automatic merge from submit-queue
construction of GC should not fail for restmapper error caused by tpr
Fix https://github.com/kubernetes/kubernetes/issues/43147.
The issue is that GC will fail its initialization due to an RESTMapper error cause by tpr. This PR lets GC log the error instead of failing.
Automatic merge from submit-queue
Fix polarity of a test in NodePort allocation
The result of this was that an update to a Service would release the
NodePort temporarily (the repair loop would fix it in a minute). During
that window, another Service could get allocated that Port.
Fixes#43233
Automatic merge from submit-queue (batch tested with PRs 43254, 43255, 43184, 42509)
Update comment on the default DeletionPropagationPolicy
This is fixing the API doc, so apply the 1.6 milestone.
I'll run update-all.sh before merge.
The result of this was that an update to a Service would release the
NodePort temporarily (the repair loop would fix it in a minute). During
that window, another Service could get allocated that Port.
David Ashpole