Commit Graph

961 Commits (803cb9303bce084fce9c492d9bf67e0b8b3ebed7)

Author SHA1 Message Date
Kubernetes Submit Queue a5e1eac1f8 Merge pull request #48858 from feiskyer/readonlyrootfs-test
Automatic merge from submit-queue (batch tested with PRs 46913, 48910, 48858, 47160)

Add e2e test for readOnlyRootFilesystem containers

**What this PR does / why we need it**:

This PR adds node e2e test for readOnlyRootFilesystem containers.

**Which issue this PR fixes**

Part of #44118.

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-07-25 23:00:33 -07:00
Kubernetes Submit Queue 2189314895 Merge pull request #40050 from mtaufen/standalone-mode
Automatic merge from submit-queue (batch tested with PRs 48976, 49474, 40050, 49426, 49430)

Use presence of kubeconfig file to toggle standalone mode

Fixes #40049 

```release-note
The deprecated --api-servers flag has been removed. Use --kubeconfig to provide API server connection information instead. The --require-kubeconfig flag is now deprecated. The default kubeconfig path is also deprecated. Both --require-kubeconfig and the default kubeconfig path will be removed in Kubernetes v1.10.0.
```

/cc @kubernetes/sig-cluster-lifecycle-misc @kubernetes/sig-node-misc
2017-07-25 12:14:43 -07:00
Kubernetes Submit Queue 68182cea8b Merge pull request #49396 from yguo0905/docker-validation-3
Automatic merge from submit-queue (batch tested with PRs 48224, 45431, 45946, 48775, 49396)

Update cos-dev image in benchmark tests to cos-dev-61-9759-0-0

Ref: https://github.com/kubernetes/kubernetes/issues/42926

`cos-dev-61-9759-0-0` contains a fix in Linux utility `du` that would affect the measurement of docker performance in kubelet. I'd like to update the benchmark to use the new image.

**Release note**:
```
None
```

/assign @tallclair 
/cc @kewu1992 @abgworrall
2017-07-25 11:06:55 -07:00
Kubernetes Submit Queue e623fed778 Merge pull request #48636 from jingxu97/July/allocatable
Automatic merge from submit-queue (batch tested with PRs 48636, 49088, 49251, 49417, 49494)

Fix issues for local storage allocatable feature

This PR fixes the following issues:
1. Use ResourceStorageScratch instead of ResourceStorage API to represent
local storage capacity
2. In eviction manager, use container manager instead of node provider
(kubelet) to retrieve the node capacity and reserved resources. Node
provider (kubelet) has a feature gate so that storagescratch information
may not be exposed if feature gate is not set. On the other hand,
container manager has all the capacity and allocatable resource
information.

This PR fixes issue #47809
2017-07-24 19:30:33 -07:00
Kubernetes Submit Queue fe8f6a1599 Merge pull request #49309 from yujuhong/add-node-e2e-owner
Automatic merge from submit-queue

Add yujuhong to test/e2e_node/OWNERS
2017-07-24 11:06:16 -07:00
Michael Taufen 38aee0464d Providing kubeconfig file is now the switch for standalone mode
Replaces use of --api-servers with --kubeconfig in Kubelet args across
the turnup scripts. In many cases this involves generating a kubeconfig
file for the Kubelet and placing it in the correct location on the node.
2017-07-24 11:03:00 -07:00
Yang Guo 324b091002 Update cos-dev image in benchmark tests to cos-dev-61-9759-0-0 2017-07-21 10:30:48 -07:00
Kubernetes Submit Queue 947700d146 Merge pull request #49207 from dixudx/remove_redundant_param_e2e_remote
Automatic merge from submit-queue

remove redundant param in e2e_node/remote

**What this PR does / why we need it**:

* remove redundant param in e2e_node/remote/remote.go
* fix a small typo

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
None
```
2017-07-20 20:16:38 -07:00
Kubernetes Submit Queue 6c5b24b564 Merge pull request #49064 from yguo0905/ubuntu-gke
Automatic merge from submit-queue (batch tested with PRs 49316, 46117, 49064, 48073, 49323)

Test Ubuntu image using GKE image spec on master

Ref: https://github.com/kubernetes/kubernetes/issues/46891

This PR changes the files referenced in test-infra for running Ubuntu image tests against GKE system spec on master.

The two properties files are shared by the tests against all k8s branches but the `SYSTEM_SPEC_NAME` is only available on master. This should be fine because the tests in the non master branches will just ignore the unknown env variable.

**Release note**:
```
None
```

/assign @yujuhong
2017-07-20 17:02:50 -07:00
Yu-Ju Hong d51c698181 Add yujuhong to test/e2e_node/OWNERS 2017-07-20 09:48:54 -07:00
Kubernetes Submit Queue 3e0dde91b6 Merge pull request #49062 from yguo0905/docker-validation-2
Automatic merge from submit-queue (batch tested with PRs 48377, 48940, 49144, 49062, 49148)

Add cos-dev-61-9733-0-0 to the benchmark tests

Ref: https://github.com/kubernetes/kubernetes/issues/42926

m60 has docker 1.13.1 while m61 has 17.03. This PR adds m61 to the benchmark tests so that we will have more data to compare.

PS: We will support fetching the latest image in an image family in the node e2e tests in the future.

**Release note**:

```
None
```

/assign @yujuhong 
/cc @kewu1992 @abgworrall
2017-07-19 19:10:16 -07:00
Di Xu 769929ba49 remove redundant param in e2e_node/remote 2017-07-19 22:25:31 +08:00
jeff vance a113d8ac41 volume i/o tests for storage plugins 2017-07-18 17:59:15 -07:00
Yang Guo c979d7f167 Test Ubuntu image using GKE image spec 2017-07-17 16:18:17 -07:00
Yang Guo 248930bc7d Add cos-beta-60-9592-52-0 to the benchmark tests 2017-07-17 15:53:15 -07:00
Jacob Simpson a765b8cfca Migrate api.Scheme to scheme.Scheme 2017-07-17 15:05:38 -07:00
Jacob Simpson 29c1b81d4c Scripted migration from clientset_generated to client-go. 2017-07-17 15:05:37 -07:00
Kubernetes Submit Queue 226b39c6b5 Merge pull request #48896 from yguo0905/docker-validation-m60
Automatic merge from submit-queue (batch tested with PRs 48890, 46893, 48872, 48896)

Add cos-beta-60-9592-52-0 to the benchmark tests

This PR depends on https://github.com/kubernetes/kubernetes/pull/48824.

This PR adds new resource usage tests for cos-beta-60-9592-52-0 (docker 1.13.1).

Ref: #42926

**Release note**:
```
None
```
/sig node
/area node-e2e
/assign @dchen1107
/cc @abgworrall
2017-07-14 16:49:55 -07:00
Kubernetes Submit Queue cab07f3af0 Merge pull request #46893 from yguo0905/image-spec
Automatic merge from submit-queue (batch tested with PRs 48890, 46893, 48872, 48896)

Support customized system spec in the node conformance test and create the GKE system spec

ref: https://github.com/kubernetes/kubernetes/issues/46891

- System specs are located in `test/e2e_node/system/specs`. Created one for validating GKE images in `test/e2e_node/system/specs/gke.yaml`.
- `--image-spec-name` can be used to specify a system spec in node e2e and conformance tests. This option maps to `SYSTEM_SPEC_NAME` in a test properties file, which is the user facing configuration. So, users can specify `SYSTEM_SPEC_NAME=gke` to run the image validation using the GKE system spec.
- If `SYSTEM_SPEC_NAME` is unspecified, the default spec (`system.DefaultSysSpec`) will be used.
- We can also use `make test-e2e-node SYSTEM_SPEC_NAME=gke` to run tests using GKE image spec.

**Release note**:
`None`
2017-07-14 16:49:52 -07:00
Kubernetes Submit Queue 23e60ac9ad Merge pull request #48308 from yguo0905/docker-api
Automatic merge from submit-queue

Update Docker API in Kubelet

Ref: https://github.com/kubernetes/kubernetes/issues/34308

The Kubelet currently uses deprecated docker API (https://godoc.org/github.com/docker/engine-api). This PR changes it to use the new one (https://godoc.org/github.com/moby/moby/client).

This PR updated the docker package from 1.11 to 1.13.1-rc2.

**Release note**:
```
None
```

/assign @Random-Liu 
/cc @yujuhong
2017-07-14 15:30:59 -07:00
Yang Guo 22c9e23202 Supports customized system spec in the node conformance test and creates the GKE system spec 2017-07-14 09:39:19 -07:00
Kubernetes Submit Queue a14abaabab Merge pull request #48824 from yguo0905/docker-validation
Automatic merge from submit-queue (batch tested with PRs 48082, 48815, 48901, 48824)

Add test image name to the OS image field of the perf metrics

I'd like to add the resource usage benchmarks for COS m60 (docker 1.13.1) but don't want to remove the existing m59 (docker 1.11.2) [ones](https://github.com/kubernetes/kubernetes/blob/master/test/e2e_node/jenkins/benchmark/benchmark-config.yaml#L51-L71), in order to compare the results between the two docker versions.

The `image` reported in the metrics is from `Node.Status.NodeInfo.OSImage`, which is always "Container-Optimized OS from Google" (from `/etc/os-releases`) for COS. So there's no way to differentiate two milestones in the metrics.

This PR attaches the [image name](https://github.com/kubernetes/kubernetes/blob/master/test/e2e_node/jenkins/benchmark/benchmark-config.yaml#L52) to the `image` field of the metrics. So it will become "Container-Optimized OS from Google (cos-stable-59-9460-64-0)".

See the results of the test run:

[performance-memory-containervm-resource1-resource_0.json](https://storage.googleapis.com/ygg-gke-dev-bucket/e2e-node-test/ci-kubernetes-node-kubelet-benchmark/13/artifacts/performance-memory-containervm-resource1-resource_0.json)
[performance-memory-coreos-resource1-resource_0.json](https://storage.googleapis.com/ygg-gke-dev-bucket/e2e-node-test/ci-kubernetes-node-kubelet-benchmark/13/artifacts/performance-memory-coreos-resource1-resource_0.json)
[performance-memory-gci-resource1-resource_0.json](https://storage.googleapis.com/ygg-gke-dev-bucket/e2e-node-test/ci-kubernetes-node-kubelet-benchmark/13/artifacts/performance-memory-gci-resource1-resource_0.json)

**Release note**:
```
None
```

Ref: https://github.com/kubernetes/kubernetes/issues/42926

/sig node
/area node-e2e
/assign @dchen1107
2017-07-13 22:44:00 -07:00
Kubernetes Submit Queue 8ad1be7833 Merge pull request #44475 from freehan/checkpoint-test
Automatic merge from submit-queue

add dockershim checkpoint node e2e test

Add a bunch of disruptive cases to test kubelet/dockershim's checkpoint work flow.

Some steps are quite hacky. Not sure if there is better ways to do things.
2017-07-13 18:50:10 -07:00
Yang Guo bf2ced837c Updates Docker Engine API 2017-07-13 12:55:07 -07:00
Yang Guo 22253a6e6a Add cos-beta-60-9592-52-0 to benchmark tests 2017-07-13 12:06:59 -07:00
Jing Xu bb1920edcc Fix issues for local storage allocatable feature
This PR fixes the following issues:
1. Use ResourceStorageScratch instead of ResourceStorage API to represent
local storage capacity
2. In eviction manager, use container manager instead of node provider
(kubelet) to retrieve the node capacity and reserved resources. Node
provider (kubelet) has a feature gate so that storagescratch information
may not be exposed if feature gate is not set. On the other hand,
container manager has all the capacity and allocatable resource
information.
2017-07-13 12:06:19 -07:00
Maru Newby 6ba0e92bf4 fed: Enable the namespace controller in integration tests 2017-07-13 09:50:07 -07:00
Pengfei Ni 721047fe49 Add e2e test for readOnlyRootFilesystem containers 2017-07-13 17:21:29 +08:00
Yang Guo b17c6a1769 Add test image name to the OS image field of the perf metrics 2017-07-12 14:51:45 -07:00
Kubernetes Submit Queue 0e461035cb Merge pull request #48734 from tallclair/namechange
Automatic merge from submit-queue (batch tested with PRs 48698, 48712, 48516, 48734, 48735)

Name change: s/timstclair/tallclair/

I changed my name, and I'm migrating my user name to be consistent.
2017-07-12 04:56:32 -07:00
Kubernetes Submit Queue de30789bf5 Merge pull request #48598 from gmarek/metrics
Automatic merge from submit-queue (batch tested with PRs 46865, 48661, 48598, 48658, 48614)

Move metrics_grabbert to test/e2e

cc @aleksandra-malinowska
2017-07-12 03:02:19 -07:00
Tim Allclair a2f2e1d491 Name change: s/timstclair/tallclair/ 2017-07-10 14:05:46 -07:00
Cao Shufeng 0c577c47d5 Use glog.*f when a format string is passed
ref:
https://godoc.org/github.com/golang/glog

I use the following commands to search all the invalid usage:
$ grep "glog.Warning(" * -r | grep %
$ grep "glog.Info(" * -r | grep %
$ grep "glog.Error(" * -r | grep %
$ grep ").Info(" * -r | grep % | grep "glog.V("
2017-07-10 19:04:03 +08:00
gmarek 55880e6b4b Move metrics_grabbert to test/e2e 2017-07-07 13:13:44 +02:00
Minhan Xia 6da0c11063 add dockershim checkpoint node e2e test 2017-06-29 13:26:09 -07:00
Pengfei Ni 00eeb7f53a Add node e2e tests for runAsUser 2017-06-29 09:17:14 +08:00
Kubernetes Submit Queue 165c94aa7b Merge pull request #47549 from yguo0905/change-tested-images
Automatic merge from submit-queue

Changes node e2e tests to use the new Ubuntu image

ref: https://github.com/kubernetes/kubernetes/issues/46891

`ubuntu-docker10` and `ubuntu-docker12` images are deprecated in favor of the new one.

**Release note**:
```
None
```
/sig node
/area node-e2e
/assign @dchen1107
2017-06-27 23:30:24 -07:00
Kubernetes Submit Queue 98ee52ed78 Merge pull request #48001 from yguo0905/report-prefix
Automatic merge from submit-queue (batch tested with PRs 47675, 48001)

Encodes ReportPrefix into the generated metrics file names

Ref: https://github.com/kubernetes/kubernetes/issues/44003

Adds the test prefix to be part of the name. Otherwise the same test case running on different images will override each other. Nothing needs to be changed at the node-perf-dash side.

See test run at https://console.cloud.google.com/storage/browser/ygg-gke-dev-bucket/e2e-node-test/ci-kubernetes-node-kubelet-benchmark/10.


**Release note**:
```
None
```

/sig node
/area node-e2e
/assign @Random-Liu
2017-06-27 16:11:07 -07:00
Kubernetes Submit Queue 0dad2d0803 Merge pull request #47983 from yguo0905/memcg
Automatic merge from submit-queue (batch tested with PRs 48092, 47894, 47983)

Enables memcg notification in cluster/node e2e tests

Ref: https://github.com/kubernetes/kubernetes/issues/42676

This PR sets Kubelet flag `--experimental-kernel-memcg-notification=true` when running cluster/node e2e tests on COS and Ubuntu images.

Tested:
```
e2e-node-cos:
I0623 00:09:06.641776    1080 server.go:147] Starting server "kubelet" with command "/usr/bin/systemd-run --unit=kubelet-777178888.service --slice=runtime.slice --remain-after-exit /tmp/node-e2e-20170622T170739/kubelet --kubelet-cgroups=/kubelet.slice --cgroup-root=/ --api-servers http://localhost:8080 --address 0.0.0.0 --port 10250 --read-only-port 10255 --volume-stats-agg-period 10s --allow-privileged true --serialize-image-pulls false --pod-manifest-path /tmp/node-e2e-20170622T170739/pod-manifest571288056 --file-check-frequency 10s --pod-cidr 10.100.0.0/24 --eviction-pressure-transition-period 30s --feature-gates  --eviction-hard memory.available<250Mi,nodefs.available<10%%,nodefs.inodesFree<5%% --eviction-minimum-reclaim nodefs.available=5%%,nodefs.inodesFree=5%% --v 4 --logtostderr --network-plugin=kubenet --cni-bin-dir /tmp/node-e2e-20170622T170739/cni/bin --cni-conf-dir /tmp/node-e2e-20170622T170739/cni/net.d --hostname-override tmp-node-e2e-bfe5799d-cos-stable-59-9460-64-0 --experimental-mounter-path=/tmp/node-e2e-20170622T170739/cluster/gce/gci/mounter/mounter --experimental-kernel-memcg-notification=true"

e2e-node-ubuntu:
I0623 00:03:28.526984    2279 server.go:147] Starting server "kubelet" with command "/usr/bin/systemd-run --unit=kubelet-1407651753.service --slice=runtime.slice --remain-after-exit /tmp/node-e2e-20170622T170203/kubelet --kubelet-cgroups=/kubelet.slice --cgroup-root=/ --api-servers http://localhost:8080 --address 0.0.0.0 --port 10250 --read-only-port 10255 --volume-stats-agg-period 10s --allow-privileged true --serialize-image-pulls false --pod-manifest-path /tmp/node-e2e-20170622T170203/pod-manifest083943734 --file-check-frequency 10s --pod-cidr 10.100.0.0/24 --eviction-pressure-transition-period 30s --feature-gates  --eviction-hard memory.available<250Mi,nodefs.available<10%%,nodefs.inodesFree<5%% --eviction-minimum-reclaim nodefs.available=5%%,nodefs.inodesFree=5%% --v 4 --logtostderr --network-plugin=kubenet --cni-bin-dir /tmp/node-e2e-20170622T170203/cni/bin --cni-conf-dir /tmp/node-e2e-20170622T170203/cni/net.d --hostname-override tmp-node-e2e-e48cdd73-ubuntu-gke-1604-xenial-v20170420-1 --experimental-kernel-memcg-notification=true"

e2e-node-containervm:
I0623 00:14:35.392383    2774 server.go:147] Starting server "kubelet" with command "/tmp/node-e2e-20170622T171318/kubelet --runtime-cgroups=/docker-daemon --kubelet-cgroups=/kubelet --cgroup-root=/ --system-cgroups=/system --api-servers http://localhost:8080 --address 0.0.0.0 --port 10250 --read-only-port 10255 --volume-stats-agg-period 10s --allow-privileged true --serialize-image-pulls false --pod-manifest-path /tmp/node-e2e-20170622T171318/pod-manifest507536807 --file-check-frequency 10s --pod-cidr 10.100.0.0/24 --eviction-pressure-transition-period 30s --feature-gates  --eviction-hard memory.available<250Mi,nodefs.available<10%,nodefs.inodesFree<5% --eviction-minimum-reclaim nodefs.available=5%,nodefs.inodesFree=5% --v 4 --logtostderr --network-plugin=kubenet --cni-bin-dir /tmp/node-e2e-20170622T171318/cni/bin --cni-conf-dir /tmp/node-e2e-20170622T171318/cni/net.d --hostname-override tmp-node-e2e-9e3fdd7c-e2e-node-containervm-v20161208-image"

e2e-cos:
Jun 23 17:54:38 e2e-test-ygg-minion-group-t5r0 kubelet[2005]: I0623 17:54:38.646374    2005 flags.go:52] FLAG: --experimental-kernel-memcg-notification="true"

e2e-ubuntu:
Jun 23 18:25:27 e2e-test-ygg-minion-group-19qp kubelet[1547]: I0623 18:25:27.722253    1547 flags.go:52] FLAG: --experimental-kernel-memcg-notification="true"

e2e-containervm:
I0623 18:55:51.886632    3385 flags.go:52] FLAG: --experimental-kernel-memcg-notification="false"
```

**Release note**:
```
None
```

/sig node
/area node-e2e
/assign @dchen1107 @dashpole
2017-06-26 21:08:10 -07:00
Kubernetes Submit Queue 36ae4ae4e3 Merge pull request #47971 from yujuhong/bump-usage-limit
Automatic merge from submit-queue (batch tested with PRs 48074, 47971, 48044, 47514, 47647)

e2e: bump kubelet's resurce usage limit

We don't have per-OS image limits. Bumping these to more generous
numbers to not fail the tests.
2017-06-26 11:40:51 -07:00
Yang Guo 50d49d9c51 Enables memcg notification in cluster/node e2e tests 2017-06-26 11:40:22 -07:00
Kubernetes Submit Queue 14edc46c2e Merge pull request #47892 from ajitak/npd-config
Automatic merge from submit-queue (batch tested with PRs 47993, 47892, 47591, 47469, 47845)

Bump up npd version to v0.4.1

```
Bump up npd version to v0.4.1
```

Fixes #47219
2017-06-23 18:05:46 -07:00
Yang Guo 8ab15e3774 Encodes ReportPrefix into the generated metrics file names 2017-06-23 16:11:25 -07:00
Yu-Ju Hong 71bd92ce3b e2e: bump kubelet's resurce usage limit
We don't have per-OS image limits. Bumping these to more generous
numbers to not fail the tests.
2017-06-23 09:55:18 -07:00
Kubernetes Submit Queue 467705be00 Merge pull request #47195 from dims/bind-cadvisor-on-kubelet-interface
Automatic merge from submit-queue (batch tested with PRs 47922, 47195, 47241, 47095, 47401)

Run cAdvisor on the same interface as kubelet

**What this PR does / why we need it**:

cAdvisor currently binds to all interfaces. Currently the only
solution is to use iptables to block access to the port. We
are better off making cAdvisor to bind to the interface that
kubelet uses for better security.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

Fixes #11710

**Special notes for your reviewer**:

**Release note**:

```release-note
cAdvisor binds only to the interface that kubelet is running on instead of all interfaces.
```
2017-06-22 21:33:27 -07:00
Ajit Kumar caff16c678 Bump up npd version to v0.4.1 2017-06-22 13:13:50 -07:00
Chao Xu 60604f8818 run hack/update-all 2017-06-22 11:31:03 -07:00
Chao Xu f4989a45a5 run root-rewrite-v1-..., compile 2017-06-22 10:25:57 -07:00
mbohlool c91a12d205 Remove all references to types.UnixUserID and types.UnixGroupID 2017-06-21 04:09:07 -07:00
Kubernetes Submit Queue cc645a8c6f Merge pull request #46327 from supereagle/mark-network-plugin-dir-deprecated
Automatic merge from submit-queue (batch tested with PRs 46327, 47166)

mark --network-plugin-dir deprecated for kubelet

**What this PR does / why we need it**:

**Which issue this PR fixes** : fixes #43967

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-06-19 11:23:54 -07:00