Automatic merge from submit-queue
skip benchmark in jenkins serial test
This PR changes jenkins-serial.properties to skip benchmark tests (with tag [Benchmark]) in jenkins serial tests. It also add more comments in run_e2e.go.
Automatic merge from submit-queue
[Kubelet] Optionally consume configuration from <node-name> named config maps
This extends the Kubelet to check the API server for new node-specific config, and exit when it finds said new config.
/cc @kubernetes/sig-node @mikedanese @timstclair @vishh
**Release note**:
```
Extends Kubelet with Alpha Dynamic Kubelet Configuration. Please note that this alpha feature does not currently work with cloud provider auto-detection.
```
Automatic merge from submit-queue
[GarbageCollector] Allow per-resource default garbage collection behavior
What's the bug:
When deleting an RC with `deleteOptions.OrphanDependents==nil`, garbage collector is supposed to treat it as `deleteOptions.OrphanDependents==true", and orphan the pods created by it. But the apiserver is not doing that.
What's in the pr:
Allow each resource to specify the default garbage collection behavior in the registry. For example, RC registry's default GC behavior is Orphan, and Pod registry's default GC behavior is CascadingDeletion.
Automatic merge from submit-queue
federation: Adding support for namespace admission controls in federation-apiserver
Now that we have namespaces in federation apiserver, we can support namespace admission controls.
There are 3 of these:
namespace/autoprovision, namespace/exists and namespace/lifecycle.
namespace/autoprovision, namespace/exists should be deprecated in kubernetes(https://github.com/kubernetes/kubernetes/issues/31195). Adding support for namespace/lifecycle to federation-apiserver.
As in kube-apiserver, enabling namespace/lifecycle by default.
```release-note
Action required: If you have a running federation control plane, you will have to ensure that for all federation resources, the corresponding namespace exists in federation control plane.
federation-apiserver now supports NamespaceLifecycle admission control, which is enabled by default. Set the --admission-control flag on the server to change that.
```
cc @kubernetes/sig-cluster-federation @quinton-hoole
1. /validate service does not exist, so remove the test for it and add some that actually do exist
2. The namespace does not exist so this will always return NotFound
Note: DoRaw() ignores the StatusCode.
This is in preparation for the next commit
Automatic merge from submit-queue
[e2e density test] Fix unnecessary Delete RC requests when not running latency test
As the following code block
https://github.com/kubernetes/kubernetes/blob/master/test/e2e/density.go#L666-L670
shows, after running each density test case, it will attempt to delete "additional replication controllers" even though there is **no additional replication controller**.
When we are not running latency test, API Server will return "404 error code". So, I propose to move the above code block inside thedetermine statementsif `itArg.runLatencyTest{ }` , looks like:
```
if itArg.runLatencyTest {
...
for i := 1; i <= nodeCount; i++ {
name := additionalPodsPrefix + "-" + strconv.Itoa(i)
c.ReplicationControllers(ns).Delete(name, nil)
}
}
```
In this way, removing RC will be executed only if we set `itArg.runLatencyTest` to be `true`. It can avoid post some necessary requests to API Server.
Issuse is #30977
Automatic merge from submit-queue
[e2e test] Fix e2e test pause image hard code
Use `framework.GetPauseImageName(f.Client)` instead of hard code(such as `"gcr.io/google_containers/pause-amd64:3.0"`) to represent pause image name.
Related issus is #30967
Automatic merge from submit-queue
Node E2E: Remove fatal error in e2e_node_suite_test.go
Addresses https://github.com/kubernetes/kubernetes/issues/30779#issuecomment-240532190.
Currently we run node e2e test in parallel, and ginkgo makes sure that we only initialize test framework in the first test node.
However, because we throw out some fatal error during the initialization. Once there is an fatal error, the first test node will die immediately without reporting any error, and the other nodes will exit because the first node is gone with meaningless error.
If kubelet start fails, we'll get something like:
```
------------------------------
Failure [132.485 seconds]
[BeforeSuite] BeforeSuite
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
BeforeSuite on Node 1 failed
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
------------------------------
......
------------------------------
Failure [132.465 seconds]
[BeforeSuite] BeforeSuite
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
BeforeSuite on Node 1 failed
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
```
This PR replaces these fatal errors with gomega assertion, with this PR, we'll get:
```
Failure [132.482 seconds]
[BeforeSuite] BeforeSuite
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
should be able to start node services.
Expected success, but got an error:
<*errors.errorString | 0xc8203351b0>: {
s: "failed to run server start command \"/tmp/ginkgo869068712/e2e_node.test --run-services-mode --server-start-timeout 2m0s --report-dir --node-name lantaol0.mtv.corp.google.com --disable-kubenet=true --cgroups-per-qos=false --manifest-path /tmp/node-e2e-pod221291440 --eviction-hard memory.available<250Mi\": exit status 255",
}
failed to run server start command "/tmp/ginkgo869068712/e2e_node.test --run-services-mode --server-start-timeout 2m0s --report-dir --node-name lantaol0.mtv.corp.google.com --disable-kubenet=true --cgroups-per-qos=false --manifest-path /tmp/node-e2e-pod221291440 --eviction-hard memory.available<250Mi": exit status 255
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:117
------------------------------
Failure [132.485 seconds]
[BeforeSuite] BeforeSuite
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
BeforeSuite on Node 1 failed
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
------------------------------
......
------------------------------
Failure [132.465 seconds]
[BeforeSuite] BeforeSuite
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
BeforeSuite on Node 1 failed
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
```
This is much more informative.
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Node E2E: Wait for node ready before the node e2e test started.
Fixes https://github.com/kubernetes/kubernetes/issues/30252.
This PR makes node e2e test wait for exactly one node ready before running other test.
@ronnielai @mtaufen
Automatic merge from submit-queue
pkg/genericapiserver/options: don't import pkg/apiserver
Refactor the authorization options for the API server so
pkg/apiserver isn't directly imported by the options package.
Closes#28544
cc @smarterclayton
@madhusudancs, @nikhiljindal I've updated `federation/cmd/federation-apiserver/app/server.go` to include the RBAC options with this change. I don't know if this was intentionally left out in the first place but would like your feedback.
Automatic merge from submit-queue
Add benchmark to jenkins
This PR contains the following changes:
1. Add more tests in density benchmark test;
2. Add the peak value (100%) in latency and CPU usage statistic data;
3. Move the Ginkgo focus flag from e2e_remote.go to run_e2e.go;
4. Support running benchmark in run_e2e.go. The benchmark configuration file is an extension of image configuration. Each item requires additional GCE machine type (e.g. n1-standard-1, default value will be used if empty) and test names (Ginkgo focus regex strings). A test item is regarded as benchmark if the tests field is non-empty.
Automatic merge from submit-queue
Node Conformance Test: Statically link apiserver
For #30122, #30174.
Based on #30116, #30198, #30200.
**Please only review the 4th PR.**
This PR is part of our roadmap to package node conformance test.
The 1st commit is from #30116, which started e2e services in a separate process.
The 2nd commit is from #30198, it statically linked etcd into the node e2e framework.
The 3rd commit is from #30200, it moved namespace controller into e2e services.
The 4th commit is new, it statically linked apiserver into the node e2e framework.
@dchen1107 @vishh
/cc @kubernetes/sig-node @kubernetes/sig-testing
Automatic merge from submit-queue
Node Conformance Test: Move namespace controller to services
For #30122, #30174.
Based on #30116, #30198.
**Please only review the 3rd PR.**
This PR is part of our roadmap to package node conformance test.
The 1st commit is from #30116, which started e2e services in a separate process.
The 2nd commit is from #30198, it statically linked etcd into the node e2e framework.
The 3rd commit is new, it moved namespace controller into e2e services.
@dchen1107 @vishh
/cc @kubernetes/sig-node @kubernetes/sig-testing
Automatic merge from submit-queue
Scheduledjobs e2e
@janetkuo resubmitted e2e for SJ, I've updated all scripts to consume `KUBE_RUNTIME_CONFIG` properly in 2nd commit, ptal
Automatic merge from submit-queue
Unblock iterative development on pod-level cgroups
In order to allow forward progress on this feature, it takes the commits from #28017#29049 and then it globally disables the flag that allows these features to be exercised in the kubelet. The flag can be re-added to the kubelet when its actually ready.
/cc @vishh @dubstack @kubernetes/rh-cluster-infra
Automatic merge from submit-queue
When running inside docker, activate service account ASAP
Also switching to just use `GOOGLE_APPLICATION_CREDENTIALS`, rather than both.
x-ref https://github.com/kubernetes/test-infra/issues/318
Automatic merge from submit-queue
Wait for memory to be reclaimed after node_e2e MemoryEviction test
This helps prevent interference with other tests that run immediately after the MemoryEviction test.
/cc @Random-Liu @coufon
Automatic merge from submit-queue
Gubernator bug fixes: mv and GCS bucket permissions
Fixed issue where results file was not moved correctly, and also the permissions issue with the GCS bucket.
Will rebase after #30414 is merged
@timstclair
Automatic merge from submit-queue
Add logging time series to benchmark test
This PR adds a new file benchmark_util.go which contains tool functions for benchmark (we can migrate benchmark related functions into it).
The PR logs time series data for density benchmark test.
Automatic merge from submit-queue
OpenAPI / Swagger2 spec generation
This is alpha version of OpenAPI spec generation. Generated "/swagger.json" file (accessible on api server) is a valid OpenAPI spec with some warnings that will be fixed in next versions of spec generation. Currently it is possible to generate a client using this spec though I did not test the clients.
reference: #13414
**Release note**:
```release-note
Alpha support for OpenAPI (aka. Swagger 2.0) specification serves on /swagger.json
```
Automatic merge from submit-queue
extract common function in ingress and federation e2e to uil
Separate refactoring existing e2e code part from #29773
1. extract common constants, structs and functions in federated-service.go to federation-util.go, move existing util functions from federated-service.go to federation-util.go
2. extract common constants, structs and functions in ingress.go to ingress-util.go, move existing util functions from ingress.go to ingress-util.go
@quinton-hoole
Automatic merge from submit-queue
Node E2E: Make readiness check handling process exits with 0 exit code.
As is mentioned by @mtaufen:
"there is a problem with the way service `start` is currently implemented in test/e2e_node/e2e_service.go. If the Kubelet exits with status 0 before the health check completes, cmdErrorChan will be closed and, as a result, nil will be read from that channel, and you will return a nil error from `start`."
This PR changes the logic to:
1) If the err channel returns an error, return the error
2) If the err channel returns a nil, ignore it and continue checking readiness.
3) If the err channel is closed before readiness check succeeds, replace it with `blockCh` and continue checking readiness.
@mtaufen
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Move UTs that block on apiserver to integration tests.
In validating etcd.v3client we had uncovered that a change in the behavior of the client https://github.com/coreos/etcd/issues/6162 , caused a number of unit tests to fail. These test failures were due to the fact that the unit tests were trying to standup a apiserver even though there was no etcd backend stood up.
This PR simply shuffles those tests to integration tests, which is where they should be.
/cc @kubernetes/sig-scalability @wojtek-t @hongchaodeng @xiang90
Automatic merge from submit-queue
Add GUBERNATOR flag which produces g8r link for node e2e tests
When you run 'make tests-e2e-node REMOTE=true GUBERNATOR=true' outputs a URL to view the test results on Gubernator. ~~Should work after my PR for Gubernator is merged.~~
@timstclair
Convert single GV and lists of GVs into an interface that can handle
more complex scenarios (everything internal, nothing supported). Pass
the interface down into conversion.
Automatic merge from submit-queue
Implement dynamic provisioning (beta) of PersistentVolumes via StorageClass
Implemented according to PR #26908. There are several patches in this PR with one huge code regen inside.
* Please review the API changes (the first patch) carefully, sometimes I don't know what the code is doing...
* `PV.Spec.Class` and `PVC.Spec.Class` is not implemented, use annotation `volume.alpha.kubernetes.io/storage-class`
* See e2e test and integration test changes - Kubernetes won't provision a thing without explicit configuration of at least one `StorageClass` instance!
* Multiple provisioning volume plugins can coexist together, e.g. HostPath and AWS EBS. This is important for Gluster and RBD provisioners in #25026
* Contradicting the proposal, `claim.Selector` and `volume.alpha.kubernetes.io/storage-class` annotation are **not** mutually exclusive. They're both used for matching existing PVs. However, only `volume.alpha.kubernetes.io/storage-class` is used for provisioning, configuration of provisioning with `Selector` is left for (near) future.
* Documentation is missing. Can please someone write some while I am out?
For now, AWS volume plugin accepts classes with these parameters:
```
kind: StorageClass
metadata:
name: slow
provisionerType: kubernetes.io/aws-ebs
provisionerParameters:
type: io1
zone: us-east-1d
iopsPerGB: 10
```
* parameters are case-insensitive
* `type`: `io1`, `gp2`, `sc1`, `st1`. See AWS docs for details
* `iopsPerGB`: only for `io1` volumes. I/O operations per second per GiB. AWS volume plugin multiplies this with size of requested volume to compute IOPS of the volume and caps it at 20 000 IOPS (maximum supported by AWS, see AWS docs).
* of course, the plugin will use some defaults when a parameter is omitted in a `StorageClass` instance (`gp2` in the same zone as in 1.3).
GCE:
```
apiVersion: extensions/v1beta1
kind: StorageClass
metadata:
name: slow
provisionerType: kubernetes.io/gce-pd
provisionerParameters:
type: pd-standard
zone: us-central1-a
```
* `type`: `pd-standard` or `pd-ssd`
* `zone`: GCE zone
* of course, the plugin will use some defaults when a parameter is omitted in a `StorageClass` instance (SSD in the same zone as in 1.3 ?).
No OpenStack/Cinder yet
@kubernetes/sig-storage
Automatic merge from submit-queue
Adding e2e test for federation replicasets
Its a basic test which tests that we can create and delete replicasets. Will enhance it when we write the replicaset controller.
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Allow setting permission mode bits on secrets, configmaps and downwardAPI files
cc @thockin @pmorie
Here is the first round to implement: https://github.com/kubernetes/kubernetes/pull/28733.
I made two commits: one with the actual change and the other with the auto-generated code. I think it's easier to review this way, but let me know if you prefer in some other way.
I haven't written any tests yet, I wanted to have a first glance and not write them till this (and the API) are more close to the "LGTM" :)
There are some things:
* I'm not sure where to do the "AND 0777". I'll try to look better in the code base, but suggestions are always welcome :)
* The write permission on group and others is not set when you do an `ls -l` on the running container. It does work with write permissions to the owner. Debugging seems to show that is something happening after this is correctly set on creation. Will look closer.
* The default permission (when the new fields are not specified) are the same that on kubernetes v1.3
* I do realize there are conflicts with master, but I think this is good enough to have a look. The conflicts is with the autog-enerated code, so the actual code is actually the same (and it takes like ~30 minutes to generate it here)
* I didn't generate the docs (`generated-docs` and `generated-swagger-docs` from `hack/update-all.sh`) because my machine runs out of mem. So that's why it isn't in this first PR, will try to investigate and see why it happens.
Other than that, this works fine here with some silly scripts I did to create a secret&configmap&downwardAPI, a pod and check the file permissions. Tested the "defaultMode" and "mode" for all. But of course, will write tests once this is looking fine :)
Thanks a lot again!
Rodrigo
Automatic merge from submit-queue
Continue on #30774: Change podNamespacer API
continue on #30774, credit to @wojtek-t, Ref #30759
I just fixed a test and converted IsActivePod to operate on *Pod.
Automatic merge from submit-queue
Add tag [benchmark] to node-e2e-test where performance limits are not verified
This PR adds a new tag "[benchmark]" to density and resource-usage node e2e test. The performance limits will not be verified at the end of benchmark tests.
Automatic merge from submit-queue
Implement federation API server authentication e2e tests.
This PR depends on #30397. Please review only the last commit here.
Fixes: Issue #28602.
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Node Conformance Test: Statically link etcd
For #30122, #30174.
This PR is part of our roadmap to package node conformance test.
It statically linked etcd into the node e2e framework. In the future all e2e services will be linked in, and print log into the same log file `services.log`.
@dchen1107 @vishh
/cc @kubernetes/sig-node
This implements the proposal in:
docs/proposals/secret-configmap-downwarapi-file-mode.md
Fixes: #28317.
The mounttest image is updated so it returns the permissions of the linked file
and not the symlink itself.
Automatic merge from submit-queue
Fix default resource limits (node allocatable) for downward api volumes and env vars
@kubernetes/rh-cluster-infra @pmorie @derekwaynecarr
Automatic merge from submit-queue
two new pv e2e tests
Added two more pv e2e tests: 1) creating a claim before the pv (both not pre-bound), 2) creating a claim before the pv with the claim pre-bound to the PV via Spec.Volumename.
Automatic merge from submit-queue
Run CI Jenkins node e2e tests in project k8s-jkns-ci-node-e2e
Fixes#27648.
If node VMs leak, they should only harm themselves, not the rest of Jenkins.
This also lets us do VM cleanup without worrying that we might accidentally delete important Jenkins VMs.
The `k8s-jkns-ci-node-e2e` should have the right ACLs in place already. The quota is at defaults, but I don't think we'll need to increase it at this point.
Automatic merge from submit-queue
Let load and density e2e tests use GC if it's on
I've run the 100 and 500 nodes tests and they both pass.
The test-infra half of the PR is https://github.com/kubernetes/test-infra/pull/369
cc @lavalamp
Automatic merge from submit-queue
Move new etcd storage (low level storage) into cacher
In an effort for #29888, we are pushing forward this:
What?
- It changes creating etcd storage.Interface impl into creating config
- In creating cacher storage (StorageWithCacher), it passes config created above and new etcd storage inside.
Why?
- We want to expose the information of (etcd) kv client to cacher. Cacher storage uses this information to talk to remote storage.
Automatic merge from submit-queue
Update core etcd references to use 3.0.4
This updates the core references to use 3.0.4.
There are still legacy references in the code base that should be cleaned, or just removed but I'm reluctant to purge.
/cc @kubernetes/sig-scalability
Kubernetes Submit Queue