Automatic merge from submit-queue
Node E2E: Add image white list
This is part of #29081. Fixes#29155.
As is discussed with @yujuhong in #29155, it is difficult to maintain the prepull image list if it is not enforced.
This PR added an image white list in the test framework, only images in the white list could be used in the test. If the image is not in the white list, the test will fail with reason:
```
Image "XXX" is not in the white list, consider adding it to CommonImageWhiteList in test/e2e/common/util.go or NodeImageWhiteList in test/e2e_node/image_list.go
```
Notice that if image pull policy is `PullAlways`, the image is not necessary to be in the white list or prepulled, because the test expects the image to be pulled during the test.
Currently, the image white list is only enabled in node e2e, because the image puller in e2e test is not integrated with the image white list yet.
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Implement cleanup methods to cleanup underlying cluster resources in federated service tests.
cc @kubernetes/sig-cluster-federation @colhom
Automatic merge from submit-queue
Make container exec failures in e2e easier to debug
Makes container exec failures in e2e tests easier to debug. Found while chasing some SELinux bugs :)
@pwittrock I'm adding this to the 1.4 milestone because it makes e2e failures easier to debug.
Automatic merge from submit-queue
Pet Set Example for Cassandra
- updating cassandra to 3.7
- added pet set example
- adding pet set for Cassandra e2e tests
- changed service as we do not want a lb service, as we are running C*
- updated docs
cc @bgrant0607
cc @kubernetes/examples
We can probably close a couple of other open PR, since I did some other stuff.
Automatic merge from submit-queue
Decrease timeout for namespace creation in test
If apiserver is unresponsive (e.g. because of crashloop or sth), we are wasting a lot of test time on retries.
Automatic merge from submit-queue
Add kubectl run ScheduledJob e2e test
**What this PR does / why we need it**:
This add another `kubectl run` e2e test, this time verifying proper creation of a ScheduledJob.
@janetkuo ptal
@deads2k that should give you more confidence when ditching manual clients
Automatic merge from submit-queue
Add Viper parametrization as E2E config option.
do-not-merge
Fixes#18099 via viper rather than inis.
Wont build until we remove BurntSushi/ COPYING based deps from upstream viper.
I'll dig into those issues independently and update later, before pushing the updated godeps into this PR.
Automatic merge from submit-queue
Allow to use GetSigner with vagrant provider
In order to run tests that require ssh access to a node on vagrant
we need to provide path to private ssh key.
Now it will be possible to do using VAGRANT_SSH_KEY environment variable
Automatic merge from submit-queue
Move nginx ingress e2e to slow
Normal GCE L7 e2e takes ~15m and runs in a feature private suite. This e2e ensure that the api isn't broken, by creating an nginx controller. I plan to write a really slimmed down version for presubmit, but I need to shave off a minute to get it below 5m.
Fixes https://github.com/kubernetes/kubernetes/issues/23416
Automatic merge from submit-queue
Add test for --quiet flag for kubectl run
This adds a test for the changes introduced in #30247 and #28801.
Ref #28695
Automatic merge from submit-queue
Only skip petset test if resource is missing
**What this PR does / why we need it**:
Unblock testing petset on other providers.
cc @pwittrock. Would like to cherrypick onto 1.4 but this is test code only, so it can wait til after release cut.
Automatic merge from submit-queue
Adding support for upgrading testing across image types.
Adds support for upgrade testing across image types.
@spxtr @vishh @ixdy @pwittrock
This change only affects upgrade testing. This does not touch production code and hence should be safe for cherrypicks into the 1.4 release branch.
Automatic merge from submit-queue
Re-enable Federated Ingress e2e test to check connectivity to global load balancer
...Now that it works properly.
Should not merge before #31600, as it will fail until then.
In order to run tests that require ssh access to a node on vagrant
we need to provide path to private ssh key.
Now it will be possible to do using VAGRANT_SSH_KEY environment variable
Change-Id: Ic5fe0037edd46d0db3b8036ad7fc03cf1ea07574
Automatic merge from submit-queue
Ensure that we are closing files.
**What this PR does / why we need it**: In several places we are leaking file descriptors. This could be problematic on systems with low ulimits for them.
**Release note**:
```release-note
```
Automatic merge from submit-queue
Generate 1 5 clientset
Generate the 1.5 clientset. Stop updating 1.4 clientset. Remove 1.2 clientset.
@nikhiljindal @lavalamp
I will rebase #31994 atop of this one.
Automatic merge from submit-queue
remove the rest of the non-generated clients from the kubectl code
Die `Client` Die!
It's always bigger than you think. Last bit @kargakis after this, it's gone.
Automatic merge from submit-queue
Remove long sleep in provisioning e2e tests.
PV controller sync is now 15 seconds, i.e. the controller re-tries to delete a PV four times in a minute until it succeeds. There is no need to wait for three minutes.
@kubernetes/sig-storage
Automatic merge from submit-queue
Provide an e2e skip helper checking for available resource
@janetkuo @dims this is the promised util function, but unfortunately I just learned that dynamic client suffers from the problem I've fixed in the manually written one (https://github.com/kubernetes/kubernetes/pull/29187) I need to look into the dynamic client in that case :/
Automatic merge from submit-queue
Update container image version for downward api volume tests
Some tests were using 0.7, and some were using 0.6, so updating all to 0.7.
@kubernetes/rh-cluster-infra
Automatic merge from submit-queue
Revert "tag scheduledjob e2e as [Feature:ScheduledJob]"
Reverts kubernetes/kubernetes#32233
The way the e2e jobs are configured, `[Feature:...]` tests can't easily be run in jenkins-pr or any of submit-queue blocking jobs.
Automatic merge from submit-queue
Replace gcloud shelling out with cloudprovider calls.
gcloud flakes a lot leading to resource leak. Also fixes https://github.com/kubernetes/kubernetes/issues/16636 by verifying instance-groups, ssl-certs and firewall-rules and cleaned up.
Automatic merge from submit-queue
Add e2e tests that check for wrapped volume race
This PR adds two new e2e tests that reproduce the race condition fixed in #29641 (see e.g. #29297)
In order to observe the race, you need to revert the PR that fixes it, via e.g.
```
git revert -n df1e925143
```
or
```
curl -sL https://github.com/kubernetes/kubernetes/pull/29641.patch | patch -p1 -R
```
The tests are `[Slow]` because they need to run several passes that involve creating pods with many volumes. They also are `[Serial]` because the load on the cluster may affect reproducibility of the race. They take about ~450s each when they fail on standard GCE cluster created by `go run hack/e2e.go -v --up`. `git_repo` test takes about 66s to run when it succeeds (fix PR not reverted) and `configmap` test takes about 546s in this case because configmap mounting is slower and still requires 3 passes x 5 pods x 50 configmap volumes to fail constantly with fix PR reverted. Probably these times can be reduced but frankly I've already spent quite a bit of time on tuning the numbers to find a balance between reproducibility and speed.
Managed to reproduce the problem in more or less reliable way for `configMap` and `gitRepo` volumes. Tried to reproduce it for `secret` volumes too but without success so far because they use tmpfs-based `emptyDir` variety. For `downwardAPI` volumes I expect the same problems with race reproducibility as with `secret` volumes, although I think some e2e races were caused by the bug, e.g. #29633.
The tests operate by creating several pods (via an RC) with many volumes and waiting for them to become Running. It sets node affinity for pods so that they all get created on a single node (the first one in the node list). The race condition leads to volume mount failures with slow retries, thus causing the test to time out.
The test failures look like this:
configmap:
```
• Failure [435.547 seconds]
[k8s.io] Wrapped EmptyDir volumes
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:709
should not cause race condition when used for configmaps [Serial] [Slow] [It]
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/wrapped_empty_dir.go:170
Failed waiting for pod wrapped-volume-race-8c097734-6376-11e6-9ffa-5254003793ad-acbtt to enter running state
Expected error:
<*errors.errorString | 0xc8201758d0>: {
s: "timed out waiting for the condition",
}
timed out waiting for the condition
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/wrapped_empty_dir.go:395
```
You'll see errors like this in kubelet log on the first node in the cluster:
```
E0816 00:27:23.319431 3510 configmap.go:174] Error creating atomic writer: stat /var/lib/kubelet/pods/e5986355-6347-11e6-a5d7-42010af00002/volumes/kubernetes.io~configmap/racey-configmap-14: no such file or directory
E0816 00:27:23.319478 3510 nestedpendingoperations.go:232] Operation for "\"kubernetes.io/configmap/e5986355-6347-11e6-a5d7-42010af00002-racey-configmap-14\" (\"e5986355-6347-11e6-a5d7-42010af00002\")" failed. No retries permitted until 2016-08-16 00:28:27.319450118 +0000 UTC (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "kubernetes.io/configmap/e5986355-6347-11e6-a5d7-42010af00002-racey-configmap-14" (spec.Name: "racey-configmap-14") pod "e5986355-6347-11e6-a5d7-42010af00002" (UID: "e5986355-6347-11e6-a5d7-42010af00002") with: stat /var/lib/kubelet/pods/e5986355-6347-11e6-a5d7-42010af00002/volumes/kubernetes.io~configmap/racey-configmap-14: no such file or directory
```
git_repo:
```
• Failure [455.035 seconds] [0/1882]
[k8s.io] Wrapped EmptyDir volumes
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:709
should not cause race condition when used for git_repo [Serial] [Slow] [It]
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/wrapped_empty_dir.go:179
Failed waiting for pod wrapped-volume-race-71b12b3d-6375-11e6-9ffa-5254003793ad-b0slz to enter running state
Expected error:
<*errors.errorString | 0xc8201758d0>: {
s: "timed out waiting for the condition",
}
timed out waiting for the condition
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/wrapped_empty_dir.go:395
```
Errors in kubelet log:
```
E0815 23:41:08.670203 3510 nestedpendingoperations.go:232] Operation for "\"kubernetes.io/git-repo/97636bd8-6341-11e6-a5d7-42010af00002-racey-git-repo-8\" (\"97636bd8-6341-11e6-a5d7-42010af00002\")" failed. No retries permitted until 2016-08-15 23:42:12.670181604 +0000 UTC (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "kubernetes.io/git-repo/97636bd8-6341-11e6-a5d7-42010af00002-racey-git-repo-8" (spec.Name: "racey-git-repo-8") pod "97636bd8-6341-11e6-a5d7-42010af00002" (UID: "97636bd8-6341-11e6-a5d7-42010af00002") with: failed to exec 'git clone http://10.0.68.35:2345 test': : chdir /var/lib/kubelet/pods/97636bd8-6341-11e6-a5d7-42010af00002/volumes/kubernetes.io~git-repo/racey-git-repo-8: no such file or directory
```
Generally, the races cause unexpected "no such directory" errors in kubelet logs with subsequent volume mount failures.
I've added race tests to e2e test `empty_dir_wrapper.go` ("EmptyDir wrapper volumes"). This test was added in #18445, the same PR that introduced the race bug. The original purpose of the test was making sure that no conflicts occur between different wrapped emptyDir volumes, so I've replaced "should becomes" with "should not conflict" in the first `It(...)`.
Automatic merge from submit-queue
update taints e2e, restrict taints operation with key, effect
Since taints are now unique by key, effect on a node, this PR is to restrict existing taints adding/removing/updating operations in taints e2e.
Also fixes https://github.com/kubernetes/kubernetes/issues/31066#issuecomment-242870101
Related prior Issue/PR #29362 and #30590
Automatic merge from submit-queue
Log pressure condition, memory usage, events in memory eviction test
I want to log this to help us debug some of the latest memory eviction test flakes, where we are seeing burstable "fail" before the besteffort. I saw (in the logs) attempts by the eviction manager to evict besteffort a while before burstable phase changed to "Failed", but the besteffort's phase appeared to remain "Running". I want to see the pressure condition interleaved with the pod phases to get a sense of the eviction manager's knowledge vs. pod phase.
Automatic merge from submit-queue
Plumb --feature-gates from TEST_ARGS to components in node e2e tests
This means you can set `TEST_ARGS` on the command line, in a `.properties` config for a Jenkins job, etc, to toggle gated features. For example:
`TEST_ARGS='--feature-gates=DynamicKubeletConfig=true'`
/cc @vishh @jlowdermilk
Automatic merge from submit-queue
re-enable provisioning test
Reverts https://github.com/kubernetes/kubernetes/pull/32199 for when the gke control plane is updated. This should be merged AFTER gke is ready.
@kubernetes/sig-storage @wojtek-t
Automatic merge from submit-queue
Networking test rewrite
Decomposes the old kubeproxy tests into (tcp, udp) tests for each of the following:
* intra-pod
* node-pod
* pod-Service
* node-Service
* endpoint-Service
* delete endpoints, confirm unreachability
* delete nodeport, confirm unreachability
* kube-proxy /proxymode, /healthz
Also gets rid of the old network conformance test that used apiserver proxy to check reported peer count of a webserver in a container (the netexec pod used in this test does the same thing without apiserver proxy).
Fixes https://github.com/kubernetes/kubernetes/issues/26490, https://github.com/kubernetes/kubernetes/issues/14204
[Feature:...] tag is recognized by most e2e suites and will prevent
test from being run in suites where it should not. This pattern is
used by other alpha feature tests.
Automatic merge from submit-queue
federated ingress e2e - retry reads properly
@quinton-hoole I made this rookie mistake when I wrote the code and missed your comment in last PR.
- Corrected the reference of constant test/e2e/federated-ingress.go
- Move federation ingress query call to wait. PollImmediate()
- set private method to non-capital
Automatic merge from submit-queue
Use federated namespace instead of the bootstrap cluster's namespace in Ingress e2e tests.
This should fix#31825.
cc @kubernetes/sig-cluster-federation @quinton-hoole
Automatic merge from submit-queue
Move StorageClass to a storage group
We discussed the pros and cons in sig-api-machinery yesterday. Choosing a particular group name means that clients (including our internal code) require less work and re-swizzling to handle promotions between versions. Even if you choose a group you end up not liking, the amount of work remains the same as the incubator work case: you move the affected kind, resource, and storage.
This moves the `StorageClass` type to the `storage.k8s.io` group (named for consistency with authentication, authorization, rbac, and imagepolicy). There are two commits, one for manaul changes and one for generated code.
- updating java to 3.7
- added pet set example
- adding pet set for Cassandra e2e tests
- changed service as we do not want a lb service, as we are running C*
- updated docs
fixing headers and adding exception for run.sh
adding documentation, thank god for reflog
Did not mean to commit that as the README ... fixing
fixing problems in README
fixing more problems in README
more README tweaks
munge updates
updating examples_test for PetSet in Cassandra examples
updating petset to no use better security context
Automatic merge from submit-queue
Add e2e tests for eviction subresource.
This branch includes changes pending in both #31638 and #31721. I will rebase
once those merge.
Job controller synchronizes objects only when job itself or underlying pod
changes. Or, when full resync is performed once 10 mins. This leads e2e test
to unnecessarily wait that longer timeout, sometimes at least. I've added job
modification action which triggers resync, if the job wasn't terminated within
shorter period of time.
Automatic merge from submit-queue
Add e2e test for Source IP preservation (pod to service cluster IP)
Working on #27134.
This PR added the e2e test for source ip preservation (pod to service cluster IP) in service.go. Test scenario described as below:
- Pick two different nodes in cluster.
- Create a clusterIP type service.
- Create an echo server, which echoes back client IP, to be part of the service.
- Create a client on another node. Hit the server through service cluster IP.
- Verify the source IP.
@girishkalele @freehan
Automatic merge from submit-queue
Bumped memory limit for resource consumer. Fixes#31591.
Bumped memory limit for resource consumer from 100 MB to 200 MB, increased request sizes so that the number of consumers will be smaller. Fixes#31591.
Automatic merge from submit-queue
Check server version when running scheduled job e2e tests
@janetkuo this is the promised followup to #30575 which is checking minimal server version when running ScheduledJob e2e's.
Automatic merge from submit-queue
update e2e test for federation replicaset controlelr
e2e test to verify replicases synced to underlying clusters.
@quinton-hoole @nikhiljindal @deepak-vij @kshafiee @mwielgus
Automatic merge from submit-queue
Return detailed error message for better debugging.
Try to provide more details error message for debugging when this flake #31561 happens again.
@pwittrock
Automatic merge from submit-queue
Bump nfs server image tag in pv e2e
Image modified in https://github.com/kubernetes/kubernetes/pull/30084 has been pushed, so we can bump this back up to enable the part where pod writes to server with restrictive permissions
Automatic merge from submit-queue
Adding namespaces/finalizer subresource to federation apiserver
Fixes https://github.com/kubernetes/kubernetes/issues/31077
cc @kubernetes/sig-cluster-federation @mwielgus
Verified manually that I can delete federation namespaces now.
Will update federation-namespace e2e test to verify that namespace is deleted fine
Automatic merge from submit-queue
test/e2e: fix flake in kubelet expose should create services for rc
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
NONE
```
Add a loop to retry the request to account for the TLS Timeout and API
credential error responses outlined by the flakes in #29227.
Fixes#29227
Automatic merge from submit-queue
Skip hazelcast E2E test
**What this PR does / why we need it**:
Skip hazelcast e2e test due to flakiness, which in turn is (most likely) due to a race condition upstream. See https://github.com/pires/hazelcast-kubernetes-bootstrapper/issues/9 for comments.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
https://github.com/kubernetes/kubernetes/issues/30672
**Special notes for your reviewer**:
This is temporary pending upstream changes.
**Release note**:
NONE
This gives the node e2e test binary a --feature-gates flag that populates a
FeatureGates field on the test context. The value of this field is forwarded
to the kubelet's --feature-gates flag and is also used to populate the global
DefaultFeatureGate object so that statically-linked components see the same
feature gate settings as provided via the flag.
This means that you can set feature gates via the TEST_ARGS environment
variable when running node e2e tests. For example:
TEST_ARGS='--feature-gates=DynamicKubeletConfig=true'
Automatic merge from submit-queue
Make a scheduler predicates test resiliant to race for scheduledCondi…
Fix#31341
@pwittrock - this fixes a P1 flake.
FYI @mwielgus - I don't think that the race that caused this flake can impact cluster autoscaling, but you probably should know about it.
cc @wojtek-t
Automatic merge from submit-queue
Rewrite disruption e2e test to use versioned client.
This currently includes the changes from #31638. I will rebase once that is merged.
Automatic merge from submit-queue
e2e: log wget output on CheckConnectivityToHost error
Log output might help to diagnose e2e flakes, whether they are caused by dns issues or connection timeouts.
Might help with flake https://github.com/kubernetes/kubernetes/issues/28188.
Automatic merge from submit-queue
Add e2e tests for Federated Ingress
This is e2e code for federation ingress controller.
Based util functions, add federation ingress e2e cases(reuse current k8s ones) and add logic to validate the result.
Automatic merge from submit-queue
E2E tests for the Source IP Preservation for LoadBalancers
Breaking out E2E changes from the main PR - these tests require the Alpha feature gate turned on for this feature otherwise they will consistently fail.
Add a loop to retry the request to account for the TLS Timeout and API
credential error responses outlined by the flakes in #29227.
Fixes#29227
Signed-off-by: Jess Frazelle <me@jessfraz.com>
Automatic merge from submit-queue
add retries for add/update/remove taints on node in taints e2e
fixes taint update conflict in taints e2e by adding retries for add/update/remove taints on node.
ref #27655 and #31066
Automatic merge from submit-queue
Get network name via e2e environment.
This should work, right? I plan to pipe it through into the TestContext soon, just not today, and I'd like some test runtime over the weekend. Open to suggestions.
jayunit100