Automatic merge from submit-queue (batch tested with PRs 62209, 62281, 61792, 62285, 61973). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Updating kubemci remove-clusters e2e test to check for error in output string
RunKubemciWithKubeconfig returns an error only if there was an error in running the command. Since remove-clusters command runs fine, it does not return an error. Updating the test to check for output string to contain the error message rather than expecting RunKubemciWithKubeconfig to return an error.
Also added a check to verify that status still returns the same status when remove-clusters command is run without --force flag.
cc @G-Harmon
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62209, 62281, 61792, 62285, 61973). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add ingress e2e test for multiple TLS (SNI) support
**What this PR does / why we need it**:
Add an e2e test for multiple TLS support in ingress-gce.
**Release note**:
```release-note
None
```
/assign @MrHohn
/hold
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Adding rramkumar1 and MrHohn as reviewer & approver to pkg/test
We both have been working quite extensively with the e2e testing framework, especially for ingress.
**Release note**:
```release-note
None
```
/cc @MrHohn
/assign @bowei
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fixtodo:Move ServeHostnameService constants and default settings to service_util.go
**What this PR does / why we need it**:
fixtodo:Move ServeHostnameService constants and default settings to service_util.go
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62245, 62271). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix disruptive regional tests 3
With GKE multizonal and regional clusters (when nodes are in created in more than one zones), in test framework we are passing:
1. --num-nodes equal to number of nodes in a single zone
2. --node-instance-groups equal to MIG names in that zone
The (1) is consistent with end-user experience, since the same semantic for "--num-nodes" is used in gcloud. (2) is strictly connected to that, and we don't want to change that.
As a result, the number of expected nodes, mig size etc. can't be explicitly based on those numbers. Also commands for nodes restarts etc. need to be splitted across zones and correct zone parameter has to be passed.
Automatic merge from submit-queue (batch tested with PRs 61183, 58807). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add RBAC information to audit logs
Depends on: https://github.com/kubernetes/kubernetes/pull/58806
**Release note**:
```release-note
RBAC information is included in audit logs via audit.Event annotations:
authorization.k8s.io/decision = {allow, forbid}
authorization.k8s.io/reason = human-readable reason for the decision
```
Automatic merge from submit-queue (batch tested with PRs 62208, 62114, 62144, 60460, 62214). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
removes job scaler, continued
Builds on https://github.com/kubernetes/kubernetes/pull/61912 (original commit is there for credit/blame)
This keeps all the updates to the scaler building and all the test and reaper cleanup. It just keeps a fake job scaler around for a different command path and the reaper.
/assign @p0lyn0mial
/assign @soltysh
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62208, 62114, 62144, 60460, 62214). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add unreleased tag to http2 test
**What this PR does / why we need it**:
[HTTP2](https://github.com/kubernetes/ingress-gce/pull/146) isn't yet in an Ingress release. Add Unreleased label to disable this test for gci-gke-ingress until Ingress gets a new release.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixes restartKubelet in test/e2e_node failure.
Looks like there is some recent change on how we start kubelet service
in test_e2e_node. Fixes restartKubelet() to get right kubelet service
name to cope with the change.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes kubelet-serial-gce-e2e test failure:
https://k8s-testgrid.appspot.com/wg-resource-management#kubelet-serial-gce-e2e
Thanks a lot to @mindprince for noticing this!
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use pause manifest image
**What this PR does / why we need it**:
As pause manifest code is merged part of https://github.com/kubernetes/kubernetes/pull/57723, now its time to remove all architecture-dependent pause imagename.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61918, 62180, 62198). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Pass 2: k8s GCR vanity URL
Also push out the old URL deprecation since we have not started the community transition yet and there are some instances of it still floating about.
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Migrating test images to gcr.io/kubernetes-e2e-test-images
**What this PR does / why we need it**:
Currently e2e test images are distributed between 2 different registry locations, k8s.gcr.io and gcr.io/kubernetes-e2e-test-images. This is part of a multi-step initiative to house all the images in gcr.io/kubernetes-e2e-test-images.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62131
**Special notes for your reviewer**:
1. I am starting off with migrating images under k8s/test/images/volumes-tester.
2. I did not move ceph and nfs images since they are marked for [deprecation and removal](https://github.com/kubernetes/kubernetes/tree/master/test/images/volumes-tester). Let me know if we want them moved as well.
3. I have made a copy of the images in gcr.io/kubernetes-e2e-test-images so the references are not broken post the PR merge. Will work on removing the images from k8s.gcr.io once this change sticks.
Automatic merge from submit-queue (batch tested with PRs 60102, 59970, 60021, 62011, 62080). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[GCE] Ingress HTTP2 e2e test
**What this PR does / why we need it**:
- Adds e2e test for bringing up an HTTP2 Ingress, converting it to HTTPS, then back to HTTP2
- Update echoserver image to 1.10
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62063, 62169, 62155, 62139, 61445). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Wait longer in pod cleanup
I've checked the [garbage collector tests](827b3d77cd/test/e2e/apimachinery/garbage_collector.go (L557)) and usually they're waiting up to a minute to clean the resources. So I'm proposing to use that same values when waiting for the pod cleanup. I was not able to reproduce the problem locally, but from reading the logs it looks like the problem might be with just the timing nothing else.
Related to #61484
/assign @Liujingfang1 @mengqiy
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
apiserver: enforce global RequestContextMapper in delegation chain
Having a request context mapper in the generic config makes it easy to forget setting
a common mapper for the whole delegation chain. This PR moves it into the New
constructors that already have the delegationTarget argument. So it is obvious
that the mapper is passed down as well.
Fixes https://github.com/kubernetes/kubernetes/issues/60610
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Adding a kubemci e2e test to verify that single and multicluster ingresses can exist together
Adding an e2e test to verify that single and multicluster ingresses can exist together.
This is the first test that creates 2 ingresses, so there is some funkiness in having to update jig.Ingress and jig.Class since the jig assumes that there is only one ingress in each test.
cc @nicksardo @G-Harmon @csbell
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixing ip address leak in kubemci e2e tests by always cleaning up cloud resources
Updating AfterEach in kubemci e2e tests to always call CleanupGCEIngressController, even when the ingress has already been deleted.
This is required to delete the IP address.
Our e2e tests are failing right now due to leaking IP addresses: https://k8s-testgrid.appspot.com/sig-multicluster-kubemci#kubemci-ingress-conformance
Relevant log:
https://storage.googleapis.com/kubernetes-jenkins/logs/ci-kubemci-ingress-conformance/1085/build-log.txt
```
W0403 17:45:06.821] 2018/04/03 17:45:06 main.go:299: Something went wrong: encountered 1 errors: [Error: 2 leaked resources
W0403 17:45:06.821] [ addresses ]
W0403 17:45:06.821] +NAME REGION ADDRESS STATUS
W0403 17:45:06.822] +kubemci-edf6f1d1-3761-11e8-bd10-0a580a3c52c2 35.201.100.224 RESERVED]
```
cc @G-Harmon @nicksardo @MrHohn
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Ensure /etc/hosts has a header always - Fix conformance test
**What this PR does / why we need it**:
We need to be able to tell if an /etc/hosts in a container has been touched by kubernetes or not (whether we use the host network or not).
We have 2 scenarios where we copy /etc/hosts
- with host network (we just copy the /etc/hosts from node)
- without host network (create a fresh /etc/hosts from pod info)
We are having trouble figuring out whether a /etc/hosts in a
pod/container has been "fixed-up" or not. And whether we used
host network or a fresh /etc/hosts in the various ways we start
up the tests which are:
- VM/box against a remote cluster
- As a container inside the k8s cluster
- DIND scenario in CI where test runs inside a managed container
Please see previous mis-guided attempt to fix this problem at
ba20e63446 In this commit we revert
the code from there as well.
So we should make sure:
- we always add a header if we touched the file
- we add slightly different headers so we can figure out if we used the
host network or not.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60938
**Special notes for your reviewer**:
Also see
- https://github.com/kubernetes/kubernetes/pull/61405
- https://github.com/kubernetes/kubernetes/pull/60939
- https://github.com/kubernetes/kubernetes/issues/60938
**Release note**:
```release-note
NONE
```
Looks like there is some recent change on how we start kubelet service
in test_e2e_node. Fixes restartKubelet() to get right kubelet service
name to cope with the change.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Retry node pool deletion in autoscaling tests.
**What this PR does / why we need it**:
Currently, when there is already an operation running on the cluster, the node pool deletion will fail. This will in turn make other tests that try to add extra node pool fail.
With this PR we will retry the node pool deletion operation 3 times over the course of ~15 minutes which will hopefully be enough to finish any operations running on the cluster.
This will deflake autoscaling e2e tests on gke regional clusters.
**Release note**:
```
NONE
```
@aleksandra-malinowska
Currently all our e2e test images are distributed between 2 registry locations (i) google-containers (k8s.gcr.io) and (ii) gcr.io/kubernetes-e2e-test-images. This PR is part of the initiative to house all test images at gcr.io/kubernetes-e2e-test-images eventually.
Automatic merge from submit-queue (batch tested with PRs 60878, 62118, 62126). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove crassirostris from owners and reviewers
/cc @piosz @x13n
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61705, 61609, 62103, 62113, 62115). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Cleanup CRD/CR confusion in webhook e2e tests
**What this PR does / why we need it**:
We keep referring to custom resources as CRDs in these which is really confusing because we also refer to CRDs as CRDs
```release-note
NONE
```
This will deflake autoscaling e2e tests on gke. When there is already an operation running on the cluster, the node pool deletion will fail. This will in turn make other tests that add extra node pool fail.
Automatic merge from submit-queue (batch tested with PRs 58420, 60483). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Kube apiserver ServerRunOptions set default called before use
**What this PR does / why we need it**:
move `ServerRunOptions` set default function `defaultOptions` out of `CreateKubeAPIServerConfig`, it should be called before real use `CreateNodeDialer`. So move it to cobra.Command just after kube-apiserver flags parsed.
Similarly `ServerRunOptions.Validate` move there too.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61959, 62037). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Support custom test configurations
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Support custom test configuration for IPAM performance integration tests
```
Automatic merge from submit-queue (batch tested with PRs 61404, 61025). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add e2e test for CRD Watch
**What this PR does / why we need it**:
This adds an e2e test to watch for custom resources.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#55538
**Special notes for your reviewer**:
This PR depends on some functions from #60331, and shouldn't be merged before that one gets merged.
**Release note**:
```release-note
Add e2e test for CRD Watch
```
Automatic merge from submit-queue (batch tested with PRs 61404, 61025). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
bug fix: create/update CRD panic when mutating webhook configured
apiextentions apiserver use its own scheme to create a admission chain before creating the apiextensions apiserver config.
Fixes#61355
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
We have 2 scenarios where we copy /etc/hosts
- with host network (we just copy the /etc/hosts from node)
- without host network (create a fresh /etc/hosts from pod info)
We are having trouble figuring out whether a /etc/hosts in a
pod/container has been "fixed-up" or not. And whether we used
host network or a fresh /etc/hosts in the various ways we start
up the tests which are:
- VM/box against a remote cluster
- As a container inside the k8s cluster
- DIND scenario in CI where test runs inside a managed container
Please see previous mis-guided attempt to fix this problem at
ba20e63446 In this commit we revert
the code from there as well.
So we should make sure:
- we always add a header if we touched the file
- we add slightly different headers so we can figure out if we used the
host network or not.
Update the test case to inject /etc/hosts from node to another path
(/etc/hosts-original) as well and use that to compare.
Automatic merge from submit-queue (batch tested with PRs 61498, 62030). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove kube-apiserver option that is always force to true
All production flows forced the controllers to true. I don't think that test code should be depending on it either. Removing the option.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61498, 62030). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Delete in-tree support for NVIDIA GPUs.
This removes the alpha Accelerators feature gate which was deprecated in 1.10 (#57384).
The alternative feature DevicePlugins went beta in 1.10 (#60170).
Fixes#54012
```release-note
Support for "alpha.kubernetes.io/nvidia-gpu" resource which was deprecated in 1.10 is removed. Please use the resource exposed by DevicePlugins instead ("nvidia.com/gpu").
```
Automatic merge from submit-queue (batch tested with PRs 61096, 61955, 61542, 60597). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update to use go1.10.1
**What this PR does / why we need it**:
Update to use go1.10.1
**Release note**:
```release-note
Update to use go1.10.1
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use typed events client directly
**What this PR does / why we need it**:
This PR addresses the TODO - the typed events client can be used directly without wrapping.
**Release note**:
```release-note
NONE
```
/kind cleanup
Automatic merge from submit-queue (batch tested with PRs 61894, 61369). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use range in loops; misc fixes
**What this PR does / why we need it**:
It is cleaner to use `range` in for loops to iterate over channel until it is closed.
**Release note**:
```release-note
NONE
```
/kind cleanup
Automatic merge from submit-queue (batch tested with PRs 54997, 61869, 61816, 61909, 60525). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Updating multicluster e2e test to ensure that controller only creates instance groups
Ref https://github.com/kubernetes/ingress-gce/issues/182
Updating the multicluster ingress e2e test to also verify that the controller does not create any other resource than instance group.
I was planning to do that using annotations first, but turns out that we have the right check for annotations: 61d574a2c7/pkg/controller/controller.go (L317) :)
Am using `length(ListResource) == 0` now but have still kept the annotation check.
cc @nicksardo @MrHohn @G-Harmon
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[e2e ingress-gce] Run preshared-cert and backend HTTPS tests with kubemci
**What this PR does / why we need it**:
Make preshared-cert and backside-reencryption tests compatible with kubemci.
Test is currently failing with the symptom described on https://github.com/GoogleCloudPlatform/k8s-multicluster-ingress/issues/131#issuecomment-377098544.
@nikhiljindal @g-harmon
/hold
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #NONE
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Adding integration tests for statefulset
**What this PR does / why we need it**:
Creates a base statefulset integration test framework.
Basic tests added.
**Which issue(s) this PR fixes**:
None
**Special notes for your reviewer**:
Cloned pull/52583 and updated it to work with latest API changes.
**Release note**:
None.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix patch conflict detection in apiserver
Patching conflict for merging list with mergeKey is not determined in the correct way.
```release-note
None
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use old resource model in External Metrics API e2e tests
**What this PR does / why we need it**:
Currently e2e tests for External Metrics are failing, because the tests projects are not whitelisted to use new Stackdriver resource model. They will be in a week, but this particular test doesn't have to depend on it.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add e2e test for external metrics with Stackdriver
**What this PR does / why we need it**:
Adds e2e tests for external metrics using Stackdriver adapter.
Rename the file to note that these are Stackdriver tests in anticipation of tests running with fake custom metrics apiserver. Refactor the tests to be more structured.
**Release note**:
```
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add namespace name into e2e event verify function
**What this PR does / why we need it**:
Scheduler now events assigned msg with `ns/podname`, but the e2e is still expect `podname` only.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61916
**Special notes for your reviewer**:
cc @Random-Liu @bsalamat
@kubernetes/sig-scheduling-pr-reviews
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61482, 61740). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Make systemd service name for kubelet use a timestamp in e2e-node tests.
**What this PR does / why we need it**:
This makes it easier to figure out which execution was last when looking at the output of `systemd list-units kubelet-*.service`.
We try to find the name of the /tmp/node-e2e-* directory and use the same timestamp if we can. Otherwise, we just call Now() again, which isn't as nice (as the unit name and directory name will not match) but will still produce unit names that will be ordered when launching multiple subsequent executions on the same host.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
N/A
**Special notes for your reviewer**:
Tested using `make test-e2e-node REMOTE=true` and then checking `systemctl list-units kubelet-*.service` on the target host.
```
$ systemctl list-units kubelet-*.service
kubelet-20180326T142016.service loaded active exited /tmp/node-e2e-20180326T142016/kubelet --kubeconfig /tmp/node-e2e-20180326T142016/kubeconfig --root-dir /var/lib/kubelet ...
kubelet-20180326T143550.service loaded active exited /tmp/node-e2e-20180326T143550/kubelet --kubeconfig /tmp/node-e2e-20180326T143550/kubeconfig --root-dir /var/lib/kubelet ...
```
The units are sorted in the order they were launched.
**Release note**:
```release-note
NONE
```
The numbers will only be available when docker.service has its own
memory and cpu cgroups, which doesn't necessarily happen unless the unit
has Delegate=yes configured.
Let's work around that by checking the status of Delegate, in the case
where we are:
* running Docker
* running Systemd
* able to check the status through systemctl
* the status is explicitly Delegate=no (the default)
If all of those are true, let's make CPU and Memory expectations
optional.
Tested: make test-e2e-node REMOTE=true HOSTS=centos-e2e-node FOCUS="Summary API"
This is necessary to show any RootFs usage on systems where the backing
filesystem of overlay2 is xfs.
The current test only created directories (for mount points) in the
upper layer of the overlay. Outside of the mount namespace, only the
directories are visible. When running `du` on those, usually filesystems
will show some usage, but not xfs, which shows a disk usage of 0 for
directories.
Fix this by creating a file in the root directory, outside the volumes,
in order to trigger some disk usage that can be measured by `du`.
Tested: make test-e2e-node REMOTE=true HOSTS=centos-e2e-node FOCUS="Summary API"
Automatic merge from submit-queue (batch tested with PRs 61829, 61908, 61307, 61872, 60100). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
node authorizer sets up access rules for dynamic config
This PR makes the node authorizer automatically set up access rules for
dynamic Kubelet config.
I also added some validation to the node strategy, which I discovered we
were missing while writing this.
This PR is based on another WIP from @liggitt.
```release-note
The node authorizer now automatically sets up rules for Node.Spec.ConfigSource when the DynamicKubeletConfig feature gate is enabled.
```
Automatic merge from submit-queue (batch tested with PRs 61829, 61908, 61307, 61872, 60100). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use curl instead of wget to fetch the CNI tarball in e2e-node test
**What this PR does / why we need it**:
Curl is more ubiquitous than wget. For instance, the GCE centos-7 and rhel-7 image families ship curl by default, but not wget.
Looking at the shell scripts under cluster/, they tend to use curl more than wget. (The ones that use wget, such as get-kube.sh, try curl first and only fallback to wget if it's not available.)
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes running node e2e test on centos-7 or rhel-7 image families provided in GCE.
**Special notes for your reviewer**:
Tested by running node-e2e-test on Ubuntu, COS and CentOS.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57658, 61304, 61560, 61859, 61870). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
test: Disable ui dashboard test for gke
The dashboard is disabled on GKE 1.10, the test is failing because it's
not alive. Let's just skip this use-case, and keep a todo that it'd be
nice to enable the dashboard to test it.
**What this PR does / why we need it**:
Disable dashboard test for GKE.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61559
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
cc @mml @konryd
Automatic merge from submit-queue (batch tested with PRs 61904, 61565, 61401, 61432, 61772). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add retry to AssertCleanup
**What this PR does / why we need it**:
Add retry in AssertCleanup for e2e kubectl test to remove the flakeness
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61484
**Special notes for your reviewer**:
This function is only used in e2e kubectl test. It doesn't affect other tests' behavior.
**Release note**:
```release-note
NONE
```
This makes it easier to figure out which execution was last when looking
at the output of `systemd list-units kubelet-*.service`.
We try to find the name of the /tmp/node-e2e-* directory and use the
same timestamp if we can. Otherwise, we just call Now() again, which
isn't as nice (as the unit name and directory name will not match) but
will still produce unit names that will be ordered when launching
multiple subsequent executions on the same host.
Rename the file to note that these are Stackdriver tests in anticipation of tests with fake custom metrics apiserver. Refactor the tests to be more structured.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix 61854, skip for short tests
**What this PR does / why we need it**:
Skip for short tests, so as to avoid running for every PR.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61854
**Special notes for your reviewer**:
**Release note**:
```release-note
Disable ipamperf integration tests as part of every PR verification.
```
Automatic merge from submit-queue (batch tested with PRs 61848, 61188, 56363, 61357, 61838). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
reduce garbage_collector test cases running time
**What this PR does / why we need it**:
test cases in test\integration\garbagecollector\garbage_collector_test.go wait.Poll interval is too long. So the case running time is too long.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61848, 61188, 56363, 61357, 61838). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Affinity test
**What this PR does / why we need it**:
Add e2e test for service session affinity, including all three types of services and externalTravicPolicy=Local.
**Which issue(s) this PR fixes**:
Fixes#55520
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61790, 61808, 60339, 61615, 61757). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Deployment to stop adding pod-template-hash labels/selector on adoption
**What this PR does / why we need it**: This is a blocker for #55714, because ReplicaSet selector becomes immutable in `apps/v1`. With controller ref, Deployment's ReplicaSets and Pods can avoid fighting with each others without unique label/selector (pod-template-hash), so it's safe to stop adding hash label/selector on adoption.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61433
**Special notes for your reviewer**: This is a behavioral change to Deployment controller that will affect all versions of Deployment APIs (`apps/v1`, `extensions/v1beta1`, `apps/v1beta1`, `apps/v1beta2`).
**Release note**:
```release-note
Deployment will stop adding pod-template-hash labels/selector to ReplicaSets and Pods it adopts. Resources created by Deployments are not affected (will still have pod-template-hash labels/selector).
```
Automatic merge from submit-queue (batch tested with PRs 60465, 61773, 61371, 61146). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add more e2e subpath tests
**What this PR does / why we need it**:
Adds e2e tests for detecting subpath cleanup failures:
* Add wait for pod deletion to the end of subpath tests
* Add subpath as file test case
* Add subpath as file with container restart test case
* Refactor atomic volume tests
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61178
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61402, 61143, 61427, 60592). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Performance tests and fix for IPAM controller.
Tests the four modes of allocations. Can be run using
./test-performance.sh under tests/integration/ipamperf
directory. See ./test-performance.sh -h for supported flags.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
Please see the implementation notes comment block in cloud.go for core details of how
the mocking works. README.md has details on how the tests can be run on the
command line.
**Release note**:
```release-note
Performance test framework and basic tests for the IPAM controller, to simulate behavior
of the four supported modes under lightly loaded and loaded conditions, where load is
defined as the number of operations to perform as against the configured kubernetes
API server QPS.
```
Automatic merge from submit-queue (batch tested with PRs 61402, 61143, 61427, 60592). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
delete some unused code
**What this PR does / why we need it**:
delete some unused code
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Curl is more ubiquitous than wget. For instance, the GCE centos-7 and
rhel-7 image families ship curl by default, but not wget.
Looking at the shell scripts under cluster/, they tend to use curl more
than wget. (The ones that use wget, such as get-kube.sh, try curl first
and only fallback to wget if it's not available.)
Tested: by running node-e2e-test on Ubuntu, COS and CentOS.
This PR makes the node authorizer automatically set up access rules for
dynamic Kubelet config.
I also added some validation to the node strategy, which I discovered we
were missing while writing this.
Automatic merge from submit-queue (batch tested with PRs 61644, 61624, 61743, 61019, 61287). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Added e2e test for local-volume provisioner that does not create PV for discovered non-bind-mounted filesystem.
**What this PR does / why we need it**:
For v2+ of the [local volume provisioner](https://github.com/kubernetes-incubator/external-storage/tree/master/local-volume) non-bind mounted filesystems in a discovery directory will no longer result in local PVs. This change was put in place to handle the non-atomic nature of other methods of adding directories (e.g. https://github.com/kubernetes-incubator/external-storage/issues/482).
This PR tests this change in behavior, and it validates that non-bind mounted directories within a discovery directory do NOT result in a local PV.
**Which issue(s) this PR fixes**:
Fixes https://github.com/kubernetes/kubernetes/issues/61020
**Special notes for your reviewer**:
This test can be executed using the following commands:
```
KUBE_FEATURE_GATES="BlockVolume=true" NUM_NODES=1 go run hack/e2e.go -- --up
go run hack/e2e.go -- --test --test_args='--ginkgo.focus=PersistentVolumes-local.*Local.*volume.*provisioner'
```
If you get the logs from a local volume provisioner pod, you will see the following log messages:
```
$ kubectl logs local-volume-provisioner-94ddb -n e2e-tests-persistent-local-volumes-test-6ls4z
<snip>
I0311 19:01:30.350504 1 controller.go:73] Controller started
E0311 19:01:30.350849 1 discovery.go:172] Path "/mnt/local-storage/notbindmount" is not an actual mountpoint
```
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60519, 61099, 61218, 61166, 61714). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kube-apiserver add BuiltInAuthorizationOptions validation
Validate BuiltInAuthorizationOptions after flags parsed.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60519, 61099, 61218, 61166, 61714). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Automatically add system critical priority classes at cluster boostrapping
**What this PR does / why we need it**:
We had two PriorityClasses that were hardcoded and special cased in our code base. These two priority classes never existed in API server. Priority admission controller had code to resolve these two names. This PR removes the hardcoded PriorityClasses and adds code to create these PriorityClasses automatically when API server starts.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60178
ref/ #57471
**Special notes for your reviewer**:
**Release note**:
```release-note
Automatically add system critical priority classes at cluster boostrapping.
```
/sig scheduling
Automatic merge from submit-queue (batch tested with PRs 61434, 61501, 59609, 61467, 61531). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Grammar and spelling update
**What this PR does / why we need it**:
Some minor documentation grammar and spelling update
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61546, 61038, 61575, 60779, 61496). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use random port to avoid conflict with other important pods that might be listening on 80.
**What this PR does / why we need it**:
Fix https://github.com/kubernetes/kubernetes/issues/61018
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes # https://github.com/kubernetes/kubernetes/issues/61018
**Special notes for your reviewer**:
**Release note**:
```release-note
None.
```
@smarterclayton
Automatic merge from submit-queue (batch tested with PRs 60455, 61365, 61375, 61597, 61491). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Updated lv-provisioner image to v2.1.0 in e2e tests
**What this PR does / why we need it**:
This PR updates the [local volume provisioner](https://github.com/kubernetes-incubator/external-storage/tree/master/local-volume) to v2.1.0 in the e2e tests for k8s v1.10.
**Which issue(s) this PR fixes**:
Fixes#61596
**Special notes for your reviewer**:
The following commands can be used to perform these tests:
```
$ make WHAT=test/e2e/e2e.test
$ KUBE_FEATURE_GATES="BlockVolume=true" NUM_NODES=1 go run hack/e2e.go -- --up
$ go run hack/e2e.go -- --test --test_args='--ginkgo.focus=PersistentVolumes-local.*Local.*volume.*provisioner’
```
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add e2e test for Custom Metrics API with new Stackdriver resource model and External Metrics API.
**What this PR does / why we need it**:
Add e2e test for Custom Metrics API with new Stackdriver resource model and External Metrics API.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[GCE] Remove validation of Alpha Feature Gates
**What this PR does / why we need it**:
As per discussion with @bowei, @freehan, and @nicksardo , remove validation on Alpha Feature Gates so we can be more flexible with adding or removing new features.
**Release note**:
```release-note
NONE
```
in some environments, we see a combination of start latency
and the corresponding effect on sync pod latency causing status
manager to fail to report within the 2 minute window.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
e2e:Enable CSI tests
**What this PR does / why we need it**:
Enable CSI e2e tests now that CSI objects in Kubernetes are Beta.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
update metrics to true like it is for kube-apiserver
The kube-apiserver always hardcoded to true, but nothing else knew too. I changed the default to true (I can't think of why you wouldn't want it even if you don't often use it).
@kubernetes/sig-api-machinery-bugs
@MikeSpreitzer see if this fixes you?
Inspect daemonset provisioner logs for validation.
The logs are examined for the log lines showing that the provisioner discovered the directories and did NOT create a local PV.
This commit also adds SkipUnlessProvider supports SSH, which is required for local PV e2e tests.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix todo: use the ServiceTestJig replace of service in e2e/network/service.go
**What this PR does / why we need it**:
fix todo: use the ServiceTestJig replace of service in e2e/network/service.go
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
This commit adds a test to verify Volume access in a situation where
VMware vCenter's vpxd service is down. The test mainly verifies
Volume/file access before, during, and after restarting the vpxd
service on the vCenter host.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Replace package "golang.org/x/net/context" with "context"
**What this PR does / why we need it**:
Replace package "golang.org/x/net/context" with "context"
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60560
**Special notes for your reviewer**:
As of Go 1.7 this package(golang.org/x/net/context) is available in the standard library under the name context. see (https://godoc.org/golang.org/x/net/context)
It is almost machinery replace.
**Release note**:
```release-note
NONE
```
The dashboard is disabled on GKE 1.10, the test is failing because it's
not alive. Let's just skip this use-case, and keep a todo that it'd be
nice to enable the dashboard to test it.
Tests the four modes of allocations. Can be run using
./test-performance.sh under tests/integration/ipamperf
directory. See ./test-performance.sh -h for supported flags.
Automatic merge from submit-queue (batch tested with PRs 61378, 60915, 61499, 61507, 61478). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Capture pod startup phases as metrics
Learning from https://github.com/kubernetes/kubernetes/issues/60589, we should also start collecting and graphing sub-parts of pod-startup latency.
/sig scalability
/kind feature
/priority important-soon
/cc @wojtek-t
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61378, 60915, 61499, 61507, 61478). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Bump image for ingress downgrade test
**What this PR does / why we need it**:
Bumps the image to the latest released version. This should also fix the failing test.
/assign @bowei
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 61378, 60915, 61499, 61507, 61478). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix "Garbage collector should support cascading deletion of custom resources" flake
**What this PR does / why we need it**:
These webhook configuration objects were intended to have no effect on admission requests, and are just used to ensure that webhook configuration objects can always be deleted.
However, I think they are causing an entirely separate test to flake (Garbage collector should support cascading deletion of custom resources) because of the interaction between these two webhooks, which are run on all admission requests, and #61355 (Unable to create/update CRD when mutating webhook configured) when the two tests are run at around the same time.
This PR will make the effects of the webhook e2e test on other tests easier to reason about.
The flakiness is being caused by a legitimate failure of the underlying code (That CRDs currently don't work with webhooks), but not a failure which either of "Garbage collector should support cascading deletion of custom resources" or "AdmissionWebhooks Should not be able to prevent deleting validating-webhook-configurations or mutating-webhook-configurations" is supposed to reliably catch.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59997
**Release note**:
```release-note
NONE
```
/sig api-machinery
/kind flake
Automatic merge from submit-queue (batch tested with PRs 61453, 61393, 61379, 61373, 61494). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use inner volume name instead of outer volume name for subpath directory
**What this PR does / why we need it**:
Fixes volume reconstruction for PVCs with subpath
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61372
**Special notes for your reviewer**:
**Release note**:
```release-note
ACTION REQUIRED: In-place node upgrades to this release from versions 1.7.14, 1.8.9, and 1.9.4 are not supported if using subpath volumes with PVCs. Such pods should be drained from the node first.
```
Automatic merge from submit-queue (batch tested with PRs 61453, 61393, 61379, 61373, 61494). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixing kubemci conformance tests
Ref https://github.com/GoogleCloudPlatform/k8s-multicluster-ingress/issues/131
Changes:
* Add a static ip annotation while running the tests for kubemci. kubemci requires the IP to be preallocated.
* Add a default backend service. I have added it in the spec directly, so the change will be for ingress controller as well which should be fine. kubemci requires users to specify a default backend service that they need to ensure exists in all clusters.
* Disabled update SSL cert test for kubemci since it does not support that.
* Minor logging fixes.
Verified by running the tests locally that they now pass.
```
$ KUBECONFIG=~/.kube/config KUBE_MASTER_IP="<IP>" go run hack/e2e.go -- --test --test_args="--ginkgo.focus=kubemci"
• [SLOW TEST:629.179 seconds]
Loadbalancing: L7
/usr/local/google/home/nikhiljindal/code/src/github.com/kubernetes/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/network/framework.go:22
GCE [Slow] [Feature:kubemci]
/usr/local/google/home/nikhiljindal/code/src/github.com/kubernetes/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/network/ingress.go:604
should conform to Ingress spec
/usr/local/google/home/nikhiljindal/code/src/github.com/kubernetes/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/network/ingress.go:637
------------------------------
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSMar 19 19:45:59.242: INFO: Running AfterSuite actions on all node
Mar 19 19:45:59.242: INFO: Running AfterSuite actions on node 1
Ran 1 of 820 Specs in 631.245 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 819 Skipped PASS
Ginkgo ran 1 suite in 10m32.602848558s
Test Suite Passed
```
cc @G-Harmon @MrHohn
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61396, 61321, 61443, 60911, 61461). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Change pods memory boundary.
Node e2e tests are running in parallel. There might be many pods running during summary api test. `20mb` is a bit too low.
@dashpole
/cc @kubernetes/sig-node-pr-reviews
Signed-off-by: Lantao Liu <lantaol@google.com>
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
none
```
Automatic merge from submit-queue (batch tested with PRs 60980, 61273, 60811, 61021, 61367). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use apps/v1 ReplicaSet in controller and tests.
This updates the RS/RC controller and RS integration/e2e tests to use apps/v1 ReplicaSet, as part of #55714.
It does *not* update the Deployment controller, nor its integration/e2e tests, to use apps/v1 ReplicaSet. That will be done in a separate PR (#61419) because Deployment has many more tendrils embedded throughout the system.
```release-note
Conformance: ReplicaSet must be supported in the `apps/v1` version.
```
/assign @janetkuo
Automatic merge from submit-queue (batch tested with PRs 60980, 61273, 60811, 61021, 61367). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Added e2e test for local volume provisioner discovery of new mountpoints while running.
**What this PR does / why we need it**:
This PR adds an e2e test for the local volume provisioner leveraging of the MountPropogation feature (https://github.com/kubernetes/kubernetes/pull/46444) to discover newly created mountpoints. The MountPropogation feature is now in beta (https://github.com/kubernetes/kubernetes/blob/master/pkg/features/kube_features.go#L146).
**Which issue(s) this PR fixes**:
Fixes https://github.com/kubernetes/kubernetes/issues/58416
**Special notes for your reviewer**:
```
KUBE_FEATURE_GATES="BlockVolume=true" NUM_NODES=1 go run hack/e2e.go -- --up
go run hack/e2e.go -- --test --test_args='--ginkgo.focus=PersistentVolumes-local.*Local.*volume.*provisioner'
```
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60793, 61181, 61267, 61252, 61334). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Refactor disruptive tests to use more volume types
Refactor storage disruptive tests to use different volume types. Also mark existing tests to be NFS specific. Fixes https://github.com/kubernetes/kubernetes/issues/61150
cc @jeffvance @jingxu97
/sig storage
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 60632, 60806, 59471, 61251, 61013). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove DNS service from kubectl comformance test
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #https://github.com/kubernetes/kubernetes/issues/61177
**Special notes for your reviewer**:
/assign @pwittrock
/assign @soltysh
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59536, 61104, 61030, 59013, 61169). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add rolling update daemonset existing pod adoption integration test
**What this PR does / why we need it**:
This PR adds rolling update DaemonSet existing pod adoption integration test. It also shifts all helper functions to a new utility file.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
xref #52191
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59536, 61104, 61030, 59013, 61169). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
stop using AlwaysAdmit admission
`AlwaysAdmit ` was deprecated, and stop using it in test.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60919, 60953, 61085, 61083, 60971). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove unused `pkg/api/unversioned`
**What this PR does / why we need it**:
clean code, see #61084
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61084
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60759, 60531, 60923, 60851, 58717). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Implement preemption for extender with a verb and new interface
**What this PR does / why we need it**:
This is an alternative way of implementing #51656
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#51656
**Special notes for your reviewer**:
We will also want to compare with #56296 to see which one is the best solution. See: https://github.com/kubernetes/kubernetes/pull/56296#discussion_r163381235
cc @ravigadde @bsalamat
**Release note**:
```release-note
Implement preemption for extender with a verb and new interface
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Adding details to Conformance Tests using RFC 2119 standards.
This PR is part of the conformance documentation. This is to provide more formal specification using RFC 2119 keywords to describe the test so that who ever is running conformance tests do not have to go through the code to understand why and what is tested.
The documentation information added here into each of the tests eventually result into a document which is currently checked in at location https://github.com/cncf/k8s-conformance/blob/master/docs/KubeConformance-1.9.md
I would like to have this PR reviewed for v1.10 as I consider it important to strengthen the conformance documents.
Automatic merge from submit-queue (batch tested with PRs 60574, 60666, 60831, 60877, 60357). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove potential sources of flakes for kms_transformation_test.go.
**What this PR does / why we need it**:
Remove potential sources for flakes in TestKMSPlugin test.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
#60614
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59637, 60611, 60788, 60489, 60687). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix todo: use a better way to keep this label unique in the test
**What this PR does / why we need it**:
fix todo: use a better way to keep this label unique in the test in test/e2e/apimachinery/garbage_collector.go
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60457, 60331, 54970, 58731, 60562). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add e2e test for watch
**What this PR does / why we need it**:
Currently watch is only tested by kubectl tests, but all clients of kubernetes should be able to reliably watch resources for changes. This test should be able to accommodate testing watch for custom resources without many changes.
/sig api-machinery
```release-note
Added e2e test for watch
```
Automatic merge from submit-queue (batch tested with PRs 60457, 60331, 54970, 58731, 60562). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
tests: e2e: empty msg from channel other than stdout should be non-fatal
Currently, if the exec websocket encounters a message that is not in the stdout stream, it immediately fails. However it also currently requests the stderr steam in the query params. There doesn't seem to be any guarantee that we don't get an empty message on the stderr stream.
Requesting the stderr stream in the query is desirable if, for some reason, something in the container fails and writes to stderr.
However, we do not need fail the test if we get an empty message on another stream. If the message is not empty, then that _does_ indicate and error and we should fail.
This is the situation we are currently observing with docker 1.13 in the origin CI https://github.com/openshift/origin/issues/18726
@derekwaynecarr @smarterclayton @gabemontero @liggitt @deads2k
/sig node
Automatic merge from submit-queue (batch tested with PRs 59740, 59728, 60080, 60086, 58714). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
vSphere: Minimize property collection via Finder
The 'All' parameter of the 'NewFinder' function controls property collection while searching the inventory.
When 'All' is set to 'false', Finder collects the minimal set of object properties required to search inventory.
When 'All' is set to 'true', Finder collects *all* object properties, which are *not* required to search inventory.
Setting 'All' to 'true' is only useful when inspecting all properties of an object,
such as by certain govc commands when the '-json' or '-dump' flags are specified.
Changing All=false in VCP minimizes the SOAP payload size and marshalling required on both sides, without impacting any functionality.
**What this PR does / why we need it**:
Changing All=false in VCP minimizes the SOAP payload size and marshalling required on both sides, without impacting any functionality.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59740, 59728, 60080, 60086, 58714). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
more concise to merge the slice
**What this PR does / why we need it**:
more concise to merge the slice
**Special notes for your reviewer**:
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix test images
These commits fix volume_io tests for iSCSI and Ceph RBD. Both server images were quite old (Fedora 22), so I updated them to ~~something more stable (CentOS 7) and to newer Ceph (Jewel, 10.2.7).~~ something newer (Fedora 26).
The most important fix is that the test volumes have 120 MB so volume_io test can actually run - the tests put 100MB file to the volume to check its persistence.
When mount containers in #53440 are merged I'll try to run the tests regularly with every PR (or merge) so we catch regressions quickly.
```release-note
NONE
```
/sig testing
/sig storage
/assign @jeffvance
Fixes: #56725
Automatic merge from submit-queue (batch tested with PRs 61284, 61119, 61201). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix creation of subpath with SUID/SGID directories.
SafeMakeDir() should apply SUID/SGID/sticky bits to the directory it creates.
Fixes#61283
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Added unschedulable taint
Signed-off-by: Da K. Ma <klaus1982.cn@gmail.com>
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
part of #59194; fixes#61050
**Release note**:
```release-note
When `TaintNodesByCondition` enabled, added `node.kubernetes.io/unschedulable:NoSchedule`
taint to the node if `spec.Unschedulable` is true.
When `ScheduleDaemonSetPods` enabled, `node.kubernetes.io/unschedulable:NoSchedule`
toleration is added automatically to DaemonSet Pods; so the `unschedulable` field of
a node is not respected by the DaemonSet controller.
```
Automatic merge from submit-queue (batch tested with PRs 61118, 60579). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Increase apiserver mem-threshold in density test
Ref: https://github.com/kubernetes/kubernetes/issues/60500#issuecomment-372682659 (fixes part of that issue)
/sig scalability
/kind bug
/priority important-soon
/cc @wojtek-t
/cc @crassirostris (for the release-note)
```release-note
Audit logging with buffering enabled can increase apiserver memory usage (e.g. up to 200MB in 100-node cluster). The increase is bounded by the buffer size (configurable). Ref: issue #60500
```
Automatic merge from submit-queue (batch tested with PRs 61111, 61069). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix subpath e2e tests on multizone cluster.
Use dynamically provisioned PV to run GCE PD tests. This will make sure that the pod is scheduled to the right zone and GCE PD can be attached to a node.
**Which issue(s) this PR fixes**:
Fixes#61101
**Release note**:
```release-note
NONE
```
/sig storage
@msau42 @verult
Automatic merge from submit-queue (batch tested with PRs 60737, 60739, 61080, 60968, 60951). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix broken gke regional logging test.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60882
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60737, 60739, 61080, 60968, 60951). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Detect backsteps correctly in base path detection
Avoids false positives with atomic writer `..<timestamp>` directories
Fixes#61076
/assign @msau42 @jsafrane
```release-note
Fix a regression that prevented using `subPath` volume mounts with secret, configMap, projected, and downwardAPI volumes
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Bump to etcd 3.1.12 to pick up critical fix
etcd [3.1.12](https://github.com/coreos/etcd/releases/tag/v3.1.12) (as well as 3.2.17 and 3.3.2) was released yesterday to fix a bug critical to kubernetes:
Fix [mvcc "unsynced" watcher restore operation](https://github.com/coreos/etcd/pull/9297).
- "unsynced" watcher is watcher that needs to be in sync with events that have happened.
- That is, "unsynced" watcher is the slow watcher that was requested on old revision.
- "unsynced" watcher restore operation was not correctly populating its underlying watcher group.
- Which possibly causes [missing events from "unsynced" watchers](https://github.com/coreos/etcd/issues/9086).
This will be backported to 1.9 as well.
Release note:
```release-note
Upgrade the default etcd server version to 3.1.12 to pick up critical etcd "mvcc "unsynced" watcher restore operation" fix.
```
cc @gyuho @wojtek-t @shyamjvs @timothysc @jdumars
Use dynamically provisioned PV to run GCE PD tests. This will make sure
that the pod is scheduled to the right zone and GCE PD can be attached
to a node.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixes the races around devicemanager Allocate() and endpoint deletion.
There is a race in predicateAdmitHandler Admit() that getNodeAnyWayFunc()
could get Node with non-zero deviceplugin resource allocatable for a
non-existing endpoint. That race can happen when a device plugin fails,
but is more likely when kubelet restarts as with the current registration
model, there is a time gap between kubelet restart and device plugin
re-registration. During this time window, even though devicemanager could
have removed the resource initially during GetCapacity() call, Kubelet
may overwrite the device plugin resource capacity/allocatable with the
old value when node update from the API server comes in later. This
could cause a pod to be started without proper device runtime config set.
To solve this problem, introduce endpointStopGracePeriod. When a device
plugin fails, don't immediately remove the endpoint but set stopTime in
its endpoint. During kubelet restart, create endpoints with stopTime set
for any checkpointed registered resource. The endpoint is considered to be
in stopGracePeriod if its stoptime is set. This allows us to track what
resources should be handled by devicemanager during the time gap.
When an endpoint's stopGracePeriod expires, we remove the endpoint and
its resource. This allows the resource to be exported through other channels
(e.g., by directly updating node status through API server) if there is such
use case. Currently endpointStopGracePeriod is set as 5 minutes.
Given that an endpoint is no longer immediately removed upon disconnection,
mark all its devices unhealthy so that we can signal the resource allocatable
change to the scheduler to avoid scheduling more pods to the node.
When a device plugin endpoint is in stopGracePeriod, pods requesting the
corresponding resource will fail admission handler.
Tested:
Ran GPUDevicePlugin e2e_node test 100 times and all passed now.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubernetes/issues/60176
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixes the races around devicemanager Allocate() and endpoint deletion.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[e2e service] Fix CleanupGCEResources for regional test
*What this PR does / why we need it**:
From https://k8s-testgrid.appspot.com/google-gke-staging#gke-staging-1-8-1-9-upgrade-regional-cluster&width=20, regional cluster test is failing because the GCE resource cleanup function attempts to parse region from `--gcp-zone` while regional cluster only set `--gcp-region`.
This PR pipes region into the cleanup function as well. This will need to be cherrypicked to 1.8 and 1.9.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #NONE
**Special notes for your reviewer**:
/assign @bowei @wojtek-t
cc @nikhiljindal to see if there is anything should be fixed for federation.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[e2e service] Move apiserver restart validation logic into util
**What this PR does / why we need it**:
Follow up of #60906, on GKE apiserver pod is invisible on k8s, hence test is failing.
This PR bakes the restart validation logic into the util function instead so it could be env-awared.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60761
**Special notes for your reviewer**:
Sorry for the noise.
/assign @rramkumar1 @bowei
cc @krzyzacy
**Release note**:
```release-note
NONE
```
There is a race in predicateAdmitHandler Admit() that getNodeAnyWayFunc()
could get Node with non-zero deviceplugin resource allocatable for a
non-existing endpoint. That race can happen when a device plugin fails,
but is more likely when kubelet restarts as with the current registration
model, there is a time gap between kubelet restart and device plugin
re-registration. During this time window, even though devicemanager could
have removed the resource initially during GetCapacity() call, Kubelet
may overwrite the device plugin resource capacity/allocatable with the
old value when node update from the API server comes in later. This
could cause a pod to be started without proper device runtime config set.
To solve this problem, introduce endpointStopGracePeriod. When a device
plugin fails, don't immediately remove the endpoint but set stopTime in
its endpoint. During kubelet restart, create endpoints with stopTime set
for any checkpointed registered resource. The endpoint is considered to be
in stopGracePeriod if its stoptime is set. This allows us to track what
resources should be handled by devicemanager during the time gap.
When an endpoint's stopGracePeriod expires, we remove the endpoint and
its resource. This allows the resource to be exported through other channels
(e.g., by directly updating node status through API server) if there is such
use case. Currently endpointStopGracePeriod is set as 5 minutes.
Given that an endpoint is no longer immediately removed upon disconnection,
mark all its devices unhealthy so that we can signal the resource allocatable
change to the scheduler to avoid scheduling more pods to the node.
When a device plugin endpoint is in stopGracePeriod, pods requesting the
corresponding resource will fail admission handler.