Commit Graph

1129 Commits (88db9a750b7d8991f39b4075ebb8836d4d737591)

Author SHA1 Message Date
Kubernetes Submit Queue eb86cc5e87 Merge pull request #51634 from verb/sharedpid-default-off
Automatic merge from submit-queue (batch tested with PRs 51984, 51351, 51873, 51795, 51634)

Revert to using isolated PID namespaces in Docker

**What this PR does / why we need it**: Reverts to the previous docker default of using isolated PID namespaces for containers in a pod. There exist container images that expect always to be PID 1 which we want to support unmodified in 1.8.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #48937

**Special notes for your reviewer**:

**Release note**:

```release-note
Sharing a PID namespace between containers in a pod is disabled by default in 1.8. To enable for a node, use the --docker-disable-shared-pid=false kubelet flag. Note that PID namespace sharing requires docker >= 1.13.1.
```
2017-09-05 18:40:33 -07:00
David Ashpole e5a6a79fd7 update cadvisor, docker, and runc godeps 2017-09-05 12:38:57 -07:00
Kubernetes Submit Queue cdcccaab34 Merge pull request #51845 from Random-Liu/update-sysspec
Automatic merge from submit-queue (batch tested with PRs 51845, 51868, 51864)

Update sys spec to support docker 1.11-1.13 and overlay2.

Fixes https://github.com/kubernetes/kubernetes/issues/32536.

Update docker spec to:
1) Support overlay2;
2) Support docker version 1.11-1.13.

@dchen1107 @yguo0905 @luxas 
/cc @kubernetes/sig-node-pr-reviews 

```release-note
Kubernetes 1.8 supports docker version 1.11.x, 1.12.x and 1.13.x. And also supports overlay2.
```
2017-09-03 21:31:55 -07:00
Kubernetes Submit Queue 5d72d5c31d Merge pull request #50602 from dixudx/user_arm64v8_instead_aarch64
Automatic merge from submit-queue (batch tested with PRs 50602, 51561, 51703, 51748, 49142)

Use arm32v7|arm64v8 images instead of the deprecated armhf|aarch64 image organizations

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #50601

**Special notes for your reviewer**:
/assign @ixdy @jbeda @zmerlynn
**Release note**:

```release-note
Use arm32v7|arm64v8 images instead of the deprecated armhf|aarch64 image organizations
```
2017-09-03 01:12:04 -07:00
Kubernetes Submit Queue da7ee10913 Merge pull request #49457 from mkumatag/tests_multiarch
Automatic merge from submit-queue

Use the right image for the right platform in the e2e tests

**What this PR does / why we need it**:
This PR is for enabling kubernetes tests for multi architecture platform
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #38067

**Special notes for your reviewer**:
This will enable conformance tests for all the supported architectures.
**Release note**:

```release-note
Make all e2e tests lookup image to use from a centralized place. In that centralized place, add support for multiple platforms.
```

x-ref #38067
2017-09-02 15:18:10 -07:00
Shyam JVS 3bba914496 Revert "Remove deprecated and experimental fields from KubeletConfiguration" 2017-09-02 16:30:56 +02:00
Lantao Liu 73d5f53465 Update sys spec to support docker 1.11-1.13 and overlay2. 2017-09-02 00:56:25 +00:00
Kubernetes Submit Queue 9b535b06a6 Merge pull request #51307 from mtaufen/kc-type-refactor
Automatic merge from submit-queue (batch tested with PRs 50381, 51307, 49645, 50995, 51523)

Remove deprecated and experimental fields from KubeletConfiguration

As we work towards providing a stable (v1) kubeletconfig API,
we cannot afford to have deprecated or "experimental" (alpha) fields
living in the KubeletConfiguration struct. This removes all existing
experimental or deprecated fields, and places them in KubeletFlags
instead.

I'm going to send another PR after this one that organizes the remaining
fields into substructures for readability. Then, we should try to move
to v1 ASAP (maybe not v1 in 1.8, given how close we are, but definitely in 1.9).

It makes far more sense to focus on a clean API in kubeletconfig v2,
than to try and further clean up the existing "API" that everyone
already depends on.

fixes: #51657

**Release note**:
```release-note
NONE
```
2017-09-01 16:33:59 -07:00
Lee Verberne 765374ce03 Explicitly enable docker shared-pid for e2e_node
This also renames isSharedPIDNamespaceEnabled() to
isSharedPIDNamespaceSupported() to be more accurate.
2017-09-01 23:50:11 +02:00
Kubernetes Submit Queue aa50c0f54c Merge pull request #51490 from NickrenREN/eviction-podLocalEphemeralStorageUsage
Automatic merge from submit-queue (batch tested with PRs 51628, 51637, 51490, 51279, 51302)

Fix pod local ephemeral storage usage calculation

We use podDiskUsage to calculate pod local ephemeral storage which is not correct, because podDiskUsage also contains HostPath volume  which is considered as persistent storage
This pr fixes it
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #51489

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```

/assign @jingxu97  @vishh 
cc @ddysher
2017-09-01 00:11:17 -07:00
Kubernetes Submit Queue 17dffc1ef5 Merge pull request #51448 from kastenhq/pvc_ref_volstats
Automatic merge from submit-queue (batch tested with PRs 51513, 51515, 50570, 51482, 51448)

Add PVCRef to VolumeStats

**What this PR does / why we need it**:
For pod volumes that reference a PVC, add a PVCRef to the corresponding
volume stat. This allows metrics to be indexed/queried by PVC name
which is more user-friendly than Pod reference

**Which issue this PR fixes** : [#363](https://github.com/kubernetes/features/issues/363)

**Special notes for your reviewer**:

**Release note**:
```
`VolumeStats` reported by the kubelet stats summary API 
(http://<node>:10255/stats/summary) now include a PVCRef
field describing the PVC referenced by the volume (if any). 
```
2017-08-31 22:09:20 -07:00
Manjunath A Kumatagi ee4d54c70c Port e2e tests for multi architecture 2017-09-01 05:40:52 +05:30
Manjunath A Kumatagi 22c3a590d1 Fix bazel 2017-09-01 05:39:00 +05:30
Derek Carr 566f411b08 Support remote runtimes with native cAdvisor support 2017-08-31 16:41:53 -04:00
Michael Taufen c18626de4a Remove deprecated and experimental fields from KubeletConfiguration
As we work towards providing a stable (v1) kubeletconfig API,
we cannot afford to have deprecated or "experimental" (alpha) fields
living in the KubeletConfiguration struct. This removes all existing
experimental or deprecated fields, and places them in KubeletFlags
instead.

I'm going to send another PR after this one that organizes the remaining
fields into substructures for readability. Then, we should try to move
to v1 ASAP.

It makes far more sense to focus on a clean API in kubeletconfig v2,
than to try and further clean up the existing "API" that everyone
already depends on.
2017-08-30 11:54:21 -07:00
Jing Xu 4d6da1fd9a Change SizeLimit to a pointer
This PR fixes issue #50121
2017-08-30 11:50:35 -07:00
Kubernetes Submit Queue 1fc7cd3d1d Merge pull request #51545 from sttts/sttts-deepcopy-e2e
Automatic merge from submit-queue (batch tested with PRs 47054, 50398, 51541, 51535, 51545)

e2e/integration: simplify deepcopy calls
2017-08-30 01:51:37 -07:00
Vaibhav Kamra 1ac56d8cbb Add PVCRef to VolumeStats
For pod volumes that reference a PVC, add a PVCRef to the corresponding
volume stat. This allows metrics to be indexed/queried by PVC name
which is more user-friendly than Pod reference
2017-08-29 23:12:20 -07:00
NickrenREN 4ca27417d9 Add pod local ephemeral storage usage e2e test cases 2017-08-30 13:54:26 +08:00
Dr. Stefan Schimanski 637fe0844c e2e/integration: simplify deepcopy calls 2017-08-29 20:11:50 +02:00
Yang Guo 039178b27f Use the pre-built docker binaries on Ubuntu for benchmark tests 2017-08-28 14:06:23 -07:00
Kubernetes Submit Queue 6368c1fc82 Merge pull request #51348 from rmmh/coreos-no-password
Automatic merge from submit-queue

Make coreos test images sshd not allow password login.

This will prevent security scanners from triggering.

Configuration is verbatim from:
https://coreos.com/os/docs/latest/customizing-sshd.html

```release-note
NONE
```
2017-08-26 04:19:11 -07:00
NickrenREN 27901ad5df Change eviction policy to manage one single local storage resource 2017-08-26 05:14:49 +08:00
Ryan Hitchman a7e64aaa66 Make coreos test images sshd not allow password login.
Configuration is based on:
https://coreos.com/os/docs/latest/customizing-sshd.html

The specific SSHD config is:

    # Use most defaults for sshd configuration.
    UsePrivilegeSeparation sandbox
    Subsystem sftp internal-sftp
    ClientAliveInterval 180
    UseDNS no
    UsePAM yes
    PrintLastLog no # handled by PAM
    PrintMotd no # handled by PAM
    AuthenticationMethods publickey

This will prevent security scanners from triggering.
2017-08-25 11:49:34 -07:00
Kubernetes Submit Queue 4a94363c7e Merge pull request #51158 from yguo0905/overlay2
Automatic merge from submit-queue (batch tested with PRs 51224, 51191, 51158, 50669, 51222)

Enable overlay2 on cos-m60 in node e2e tests

Ref: https://github.com/kubernetes/kubernetes/issues/42926

- Restart docker with `-s overlay2` in cloud-init before running all node e2e tests. I have to copy the systemd unit file to `/etc/systemd/system` because the `/usr/lib/systemd/system/` is read only.
- Updated node e2e tests to use the new cos-m60 image.
- The name of the cloud init file (`cos-init-live-restore.yaml`) does not indicate overlay2 will be enabled, but I can't just change the name in this PR, since it's referenced in test-infra.

**Release note**:

```
None
```

/assign @Random-Liu
2017-08-24 22:59:33 -07:00
Kubernetes Submit Queue 55a20bb901 Merge pull request #51206 from yguo0905/update-cos
Automatic merge from submit-queue (batch tested with PRs 47115, 51196, 51204, 51208, 51206)

Update cos-m61 image in benchmark tests

Ref: https://github.com/kubernetes/kubernetes/issues/51205

**Release note**:
```
None
```
2017-08-24 07:20:16 -07:00
Kubernetes Submit Queue c041567b5a Merge pull request #46597 from dixudx/implement_proposal_34058
Automatic merge from submit-queue (batch tested with PRs 51113, 46597, 50397, 51052, 51166)

implement proposal 34058: hostPath volume type

**What this PR does / why we need it**:
implement proposal #34058

**Which issue this PR fixes** : fixes #46549

**Special notes for your reviewer**:
cc @thockin @luxas @euank PTAL
2017-08-23 23:16:27 -07:00
Yang Guo a1c5c14eff Update cos-m61 image in benchmark tests 2017-08-23 09:30:20 -07:00
Kubernetes Submit Queue 178a5ff314 Merge pull request #50665 from xiangpengzhao/hardcode-to-const
Automatic merge from submit-queue (batch tested with PRs 50257, 50247, 50665, 50554, 51077)

Replace hard-code "cpu" and "memory" to consts

**What this PR does / why we need it**:
There are many places using hard coded "cpu" and "memory" as resource name. This PR replace them to consts.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:
/kind cleanup

**Release note**:

```release-note
NONE
```
2017-08-23 02:35:09 -07:00
Di Xu 6f74af94ef update e2e tests and yaml files 2017-08-23 14:05:21 +08:00
Yang Guo 755ce10e9b Enable overlay2 on cos-m60 in node e2e tests 2017-08-22 17:08:52 -07:00
Kubernetes Submit Queue c6980e7247 Merge pull request #51033 from mtaufen/revert-51008-revert-50789-fix-scheme
Automatic merge from submit-queue (batch tested with PRs 50967, 50505, 50706, 51033, 51028)

Revert "Merge pull request #51008 from kubernetes/revert-50789-fix-scheme"

I'm spinning up a cluster right now to test this fix, but I'm pretty sure this was the problem.
There doesn't seem to be a way to confirm from logs, because AFAICT the logs from the hollow kubelet containers are not collected as part of the kubemark test.

**What this PR does / why we need it**:

This reverts commit f4afdecef8, reversing
changes made to e633a1604f.

This also fixes a bug where Kubemark was still using the core api scheme
to manipulate the Kubelet's types, which was the cause of the initial
revert.

**Which issue this PR fixes**: fixes #51007

**Release note**:

```release-note
NONE
```

/cc @shyamjvs @wojtek-t
2017-08-22 10:48:21 -07:00
Kubernetes Submit Queue 0867802bbc Merge pull request #50831 from Random-Liu/instance-metadata-from-flag
Automatic merge from submit-queue (batch tested with PRs 50693, 50831, 47506, 49119, 50871)

Add instance metadata from flag even when using image config.

Also add instance metadata from flag even when we are using image config.

* Sometimes we need to dynamically generate instance metadata, it's troublesome to put them into image config.
* Sometimes we want to apply instance metadata to all images, it's duplicated to add them to each image in the image config.

/assign @yguo0905 Could you help me review this?
2017-08-21 14:29:57 -07:00
Michael Taufen a90d81620b Revert "Merge pull request #51008 from kubernetes/revert-50789-fix-scheme"
This reverts commit f4afdecef8, reversing
changes made to e633a1604f.

This also fixes a bug where Kubemark was still using the core api scheme
to manipulate the Kubelet's types, which was the cause of the initial
revert.
2017-08-21 11:28:05 -07:00
Shyam JVS 5591914d62 Revert "Don't register the kubeletconfig group with the default Scheme" 2017-08-21 11:15:27 +02:00
Di Xu d4aa1611bd use more-specific arm64v8 instead of deprecated aarch64 organization 2017-08-21 10:18:19 +08:00
Di Xu 25a786f74d use more-specific arm32v7 instead of deprecated armhf organization 2017-08-21 10:17:43 +08:00
Michael Taufen 0af9f756cd Don't register the kubeletconfig group with the default Scheme 2017-08-18 13:51:39 -07:00
Kubernetes Submit Queue a4f6ae4402 Merge pull request #50277 from yguo0905/live-restore-test
Automatic merge from submit-queue

Add node e2e test for Docker's live-restore

Ref: https://github.com/kubernetes/kubernetes/issues/42926

This PR adds a test for docker live-restore. If this is fine, we can close the unfinished PR https://github.com/kubernetes/kubernetes/pull/40364.

**Release note**:
```
None
```
2017-08-17 21:44:09 -07:00
Yang Guo 9f1f83020b Add node e2e test for Docker's live-restore 2017-08-17 16:58:21 -07:00
Random-Liu 2c129e4d6a Add instance metadata from flag even when using image config. 2017-08-17 16:42:25 -07:00
Nick Sardo a0e95f9475 Fix e2e_node for changes to /api/compute/v0.beta package 2017-08-17 10:29:58 -07:00
xiangpengzhao 1c4dbcf5ca Replace hard-code "cpu" and "memory" to consts 2017-08-16 16:37:50 +08:00
Michael Taufen 24bab4c20f move KubeletConfiguration out of componentconfig API group 2017-08-15 08:12:42 -07:00
Yang Guo 1fb12b84dd Allow passing image description from e2e node test config 2017-08-14 17:11:05 -07:00
Kubernetes Submit Queue cf80b91a9e Merge pull request #50479 from yguo0905/node-perf-m60
Automatic merge from submit-queue (batch tested with PRs 49847, 49743, 49853, 50225, 50479)

Add node benchmark tests for cos-m60 with docker 1.12.6

Ref: https://github.com/kubernetes/kubernetes/issues/42926

This PR adds a benchmark tests against cos-m60 with docker 1.12.6 on http://node-perf-dash.k8s.io. This test is useful for docker validation -- we can compare the performance of different dockers on the same OS.

cos-m60 comes with docker 1.13.1 by default, so we need to use cloud-init to downgrade the version to 1.12.6.

**Release note**:
```
None
```

/assign @dchen1107
2017-08-12 02:36:01 -07:00
Jeff Grafton a7f49c906d Use buildozer to delete licenses() rules except under third_party/ 2017-08-11 09:32:39 -07:00
Jeff Grafton 33276f06be Use buildozer to remove deprecated automanaged tags 2017-08-11 09:31:50 -07:00
Jeff Grafton cf55f9ed45 Autogenerate BUILD files 2017-08-11 09:30:23 -07:00
Yang Guo 8ca49e0989 Add node benchmark tests for cos-m60 with docker 1.12.6 2017-08-10 16:48:10 -07:00
Kubernetes Submit Queue cb49706c00 Merge pull request #48857 from feiskyer/privileged
Automatic merge from submit-queue (batch tested with PRs 49725, 50367, 50391, 48857, 50181)

Add e2e test for privileged containers

**What this PR does / why we need it**:

This PR adds node e2e test for privileged containers.

**Which issue this PR fixes**

Part of #44118.

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

/assign @Random-Liu
2017-08-10 01:47:19 -07:00
Kubernetes Submit Queue 458cc04330 Merge pull request #46254 from mtaufen/dkcfg
Automatic merge from submit-queue (batch tested with PRs 50016, 49583, 49930, 46254, 50337)

Alpha Dynamic Kubelet Configuration

Feature: https://github.com/kubernetes/features/issues/281

This proposal contains the alpha implementation of the Dynamic Kubelet Configuration feature proposed in ~#29459~ [community/contributors/design-proposals/dynamic-kubelet-configuration.md](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/dynamic-kubelet-configuration.md). 

Please note:
- ~The proposal doc is not yet up to date with this implementation, there are some subtle differences and some more significant ones. I will update the proposal doc to match by tomorrow afternoon.~
- ~This obviously needs more tests. I plan to write several O(soon). Since it's alpha and feature-gated, I'm decoupling this review from the review of the tests.~ I've beefed up the unit tests, though there is still plenty of testing to be done.
- ~I'm temporarily holding off on updating the generated docs, api specs, etc, for the sake of my reviewers 😄~ these files now live in a separate commit; the first commit is the one to review.

/cc @dchen1107 @vishh @bgrant0607 @thockin @derekwaynecarr 

```release-note
Adds (alpha feature) the ability to dynamically configure Kubelets by enabling the DynamicKubeletConfig feature gate, posting a ConfigMap to the API server, and setting the spec.configSource field on Node objects. See the proposal at https://github.com/kubernetes/community/blob/master/contributors/design-proposals/dynamic-kubelet-configuration.md for details.
```
2017-08-09 14:14:32 -07:00
Kubernetes Submit Queue 8c4a269b83 Merge pull request #49771 from feiskyer/wait-for-failure
Automatic merge from submit-queue

Add waitForFailure for e2e test framework

**What this PR does / why we need it**:

Add waitForFailure for e2e test framework, this could reduce the reliance on logs.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: 

Part of #44118. Refer https://github.com/kubernetes/kubernetes/pull/48858#discussion_r128331726

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-08-08 20:56:51 -07:00
Michael Taufen 443d58e40a Dynamic Kubelet Configuration
Alpha implementation of the Dynamic Kubelet Configuration feature.
See the proposal doc in #29459.
2017-08-08 12:21:37 -07:00
Kubernetes Submit Queue 02d04de81e Merge pull request #49914 from yguo0905/shared-pid-ns
Automatic merge from submit-queue (batch tested with PRs 50087, 39587, 50042, 50241, 49914)

Add node e2e test for Docker's shared PID namespace

Ref: https://github.com/kubernetes/kubernetes/issues/42926

This PR adds a simple test for the shared PID namespace that's enabled when Docker is 1.13.1+.

/sig node
/area node-e2e
/assign @yujuhong 

**Release note**:
```
None
```
2017-08-07 10:59:04 -07:00
Mik Vyatskov e79a228a78 Move the sig-instrumentation test to a dedicated folder 2017-08-07 10:33:03 +02:00
Dr. Stefan Schimanski 1910b5a1dd Fix code implicitly casting clientsets to getters 2017-08-06 15:30:13 +02:00
Kubernetes Submit Queue 7c9ba69617 Merge pull request #48487 from dixudx/validate_cadvisor_rootpath
Automatic merge from submit-queue (batch tested with PRs 48487, 49009, 49862, 49843, 49700)

validate cadvisor rootpath

**What this PR does / why we need it**:

When working on issue #48452, I found [KubeletConfiguration.RootDirectory](https://github.com/kubernetes/kubernetes/blob/master/cmd/kubelet/app/server.go#L525) was never been validated. The default value is set to ["/var/lib/kubelet"](https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/componentconfig/v1alpha1/defaults.go#L342).

If this directory does not exist in the file system, the [cadvisor.manager](https://github.com/kubernetes/kubernetes/blob/master/vendor/github.com/google/cadvisor/manager/manager.go#L679) will fail to gather the information for metrics.

> error trying to get filesystem Device for dir /var/lib/kubelet: err: stat failed on /var/lib/kubelet with error: no such file or directory

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

/cc @feiskyer @k82cn

**Release note**:

```release-note
validate cadvisor rootpath
```
2017-08-04 23:40:00 -07:00
Yang Guo 026a082a7f Add node e2e test for Docker's shared PID namespace 2017-08-04 15:01:55 -07:00
Kubernetes Submit Queue 7c0e9852b4 Merge pull request #49916 from yguo0905/coreos
Automatic merge from submit-queue (batch tested with PRs 49916, 50050)

Update images used in the node e2e benchmark tests

Ref: https://github.com/kubernetes/kubernetes/issues/42926

- Update the cosbeta image since the new version contains a 'du' command fix that affects Docker performance.
- Add the coreos and ubuntu image that run Docker 1.12.6 so that we will have more data to compare.

**Release note**:
```
None
```
2017-08-02 23:29:49 -07:00
Pengfei Ni 3027d9bac3 Add e2e test for privileged containers 2017-08-01 15:50:22 +08:00
Yang Guo 7c31be8ec4 Update images used in the node e2e benchmark tests 2017-07-31 18:11:02 -07:00
Kubernetes Submit Queue 72c6251508 Merge pull request #47019 from jessfraz/allowPrivilegeEscalation
Automatic merge from submit-queue (batch tested with PRs 49651, 49707, 49662, 47019, 49747)

Add support for `no_new_privs` via AllowPrivilegeEscalation

**What this PR does / why we need it**:
Implements kubernetes/community#639
Fixes #38417

Adds `AllowPrivilegeEscalation` and `DefaultAllowPrivilegeEscalation` to `PodSecurityPolicy`.
Adds `AllowPrivilegeEscalation` to container `SecurityContext`.

Adds the proposed behavior to `kuberuntime`, `dockershim`, and `rkt`. Adds a bunch of unit tests to ensure the desired default behavior and that when `DefaultAllowPrivilegeEscalation` is explicitly set.

Tests pass locally with docker and rkt runtimes. There are also a few integration tests with a `setuid` binary for sanity.

**Release note**:

```release-note
Adds AllowPrivilegeEscalation to control whether a process can gain more privileges than it's parent process
```
2017-07-31 16:56:58 -07:00
Kubernetes Submit Queue 5f6d16527d Merge pull request #49443 from yguo0905/gke-tests
Automatic merge from submit-queue (batch tested with PRs 45813, 49594, 49443, 49167, 47539)

Add node e2e tests for GKE environment

Ref: https://github.com/kubernetes/kubernetes/issues/46891

This PR adds node e2e tests for validating images used on GKE.

- We pass the `SYSTEM_SPEC_NAME` to the node e2e test process via the flag `--system-spec-name` so that we can skip the environment specific tests using `RunIfSystemSpecNameIs()`.
- Also added `SkipIfContainerRuntimeIs()` as the opposite of `RunIfContainerRuntimeIs()`.

**Release note**:
```
None
```
2017-07-28 07:22:36 -07:00
Pengfei Ni 983ecaa73d Add waitForFailure for e2e test framework 2017-07-28 17:15:43 +08:00
Kubernetes Submit Queue a5e1eac1f8 Merge pull request #48858 from feiskyer/readonlyrootfs-test
Automatic merge from submit-queue (batch tested with PRs 46913, 48910, 48858, 47160)

Add e2e test for readOnlyRootFilesystem containers

**What this PR does / why we need it**:

This PR adds node e2e test for readOnlyRootFilesystem containers.

**Which issue this PR fixes**

Part of #44118.

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-07-25 23:00:33 -07:00
Di Xu 6c7245d464 validate cadvisor rootpath 2017-07-26 10:05:29 +08:00
Kubernetes Submit Queue 2189314895 Merge pull request #40050 from mtaufen/standalone-mode
Automatic merge from submit-queue (batch tested with PRs 48976, 49474, 40050, 49426, 49430)

Use presence of kubeconfig file to toggle standalone mode

Fixes #40049 

```release-note
The deprecated --api-servers flag has been removed. Use --kubeconfig to provide API server connection information instead. The --require-kubeconfig flag is now deprecated. The default kubeconfig path is also deprecated. Both --require-kubeconfig and the default kubeconfig path will be removed in Kubernetes v1.10.0.
```

/cc @kubernetes/sig-cluster-lifecycle-misc @kubernetes/sig-node-misc
2017-07-25 12:14:43 -07:00
Kubernetes Submit Queue 68182cea8b Merge pull request #49396 from yguo0905/docker-validation-3
Automatic merge from submit-queue (batch tested with PRs 48224, 45431, 45946, 48775, 49396)

Update cos-dev image in benchmark tests to cos-dev-61-9759-0-0

Ref: https://github.com/kubernetes/kubernetes/issues/42926

`cos-dev-61-9759-0-0` contains a fix in Linux utility `du` that would affect the measurement of docker performance in kubelet. I'd like to update the benchmark to use the new image.

**Release note**:
```
None
```

/assign @tallclair 
/cc @kewu1992 @abgworrall
2017-07-25 11:06:55 -07:00
Kubernetes Submit Queue e623fed778 Merge pull request #48636 from jingxu97/July/allocatable
Automatic merge from submit-queue (batch tested with PRs 48636, 49088, 49251, 49417, 49494)

Fix issues for local storage allocatable feature

This PR fixes the following issues:
1. Use ResourceStorageScratch instead of ResourceStorage API to represent
local storage capacity
2. In eviction manager, use container manager instead of node provider
(kubelet) to retrieve the node capacity and reserved resources. Node
provider (kubelet) has a feature gate so that storagescratch information
may not be exposed if feature gate is not set. On the other hand,
container manager has all the capacity and allocatable resource
information.

This PR fixes issue #47809
2017-07-24 19:30:33 -07:00
Kubernetes Submit Queue fe8f6a1599 Merge pull request #49309 from yujuhong/add-node-e2e-owner
Automatic merge from submit-queue

Add yujuhong to test/e2e_node/OWNERS
2017-07-24 11:06:16 -07:00
Michael Taufen 38aee0464d Providing kubeconfig file is now the switch for standalone mode
Replaces use of --api-servers with --kubeconfig in Kubelet args across
the turnup scripts. In many cases this involves generating a kubeconfig
file for the Kubelet and placing it in the correct location on the node.
2017-07-24 11:03:00 -07:00
Jess Frazelle ce70619a47
allowPrivilegeEscalation: add integration test with setuid binary
Signed-off-by: Jess Frazelle <acidburn@google.com>
2017-07-24 12:52:45 -04:00
Yang Guo 78f04e2abf Add node e2e tests for GKE environment 2017-07-23 20:59:11 -07:00
Yang Guo 324b091002 Update cos-dev image in benchmark tests to cos-dev-61-9759-0-0 2017-07-21 10:30:48 -07:00
Kubernetes Submit Queue 947700d146 Merge pull request #49207 from dixudx/remove_redundant_param_e2e_remote
Automatic merge from submit-queue

remove redundant param in e2e_node/remote

**What this PR does / why we need it**:

* remove redundant param in e2e_node/remote/remote.go
* fix a small typo

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
None
```
2017-07-20 20:16:38 -07:00
Kubernetes Submit Queue 6c5b24b564 Merge pull request #49064 from yguo0905/ubuntu-gke
Automatic merge from submit-queue (batch tested with PRs 49316, 46117, 49064, 48073, 49323)

Test Ubuntu image using GKE image spec on master

Ref: https://github.com/kubernetes/kubernetes/issues/46891

This PR changes the files referenced in test-infra for running Ubuntu image tests against GKE system spec on master.

The two properties files are shared by the tests against all k8s branches but the `SYSTEM_SPEC_NAME` is only available on master. This should be fine because the tests in the non master branches will just ignore the unknown env variable.

**Release note**:
```
None
```

/assign @yujuhong
2017-07-20 17:02:50 -07:00
Yu-Ju Hong d51c698181 Add yujuhong to test/e2e_node/OWNERS 2017-07-20 09:48:54 -07:00
Kubernetes Submit Queue 3e0dde91b6 Merge pull request #49062 from yguo0905/docker-validation-2
Automatic merge from submit-queue (batch tested with PRs 48377, 48940, 49144, 49062, 49148)

Add cos-dev-61-9733-0-0 to the benchmark tests

Ref: https://github.com/kubernetes/kubernetes/issues/42926

m60 has docker 1.13.1 while m61 has 17.03. This PR adds m61 to the benchmark tests so that we will have more data to compare.

PS: We will support fetching the latest image in an image family in the node e2e tests in the future.

**Release note**:

```
None
```

/assign @yujuhong 
/cc @kewu1992 @abgworrall
2017-07-19 19:10:16 -07:00
Di Xu 769929ba49 remove redundant param in e2e_node/remote 2017-07-19 22:25:31 +08:00
jeff vance a113d8ac41 volume i/o tests for storage plugins 2017-07-18 17:59:15 -07:00
Yang Guo c979d7f167 Test Ubuntu image using GKE image spec 2017-07-17 16:18:17 -07:00
Yang Guo 248930bc7d Add cos-beta-60-9592-52-0 to the benchmark tests 2017-07-17 15:53:15 -07:00
Jacob Simpson a765b8cfca Migrate api.Scheme to scheme.Scheme 2017-07-17 15:05:38 -07:00
Jacob Simpson 29c1b81d4c Scripted migration from clientset_generated to client-go. 2017-07-17 15:05:37 -07:00
Kubernetes Submit Queue 226b39c6b5 Merge pull request #48896 from yguo0905/docker-validation-m60
Automatic merge from submit-queue (batch tested with PRs 48890, 46893, 48872, 48896)

Add cos-beta-60-9592-52-0 to the benchmark tests

This PR depends on https://github.com/kubernetes/kubernetes/pull/48824.

This PR adds new resource usage tests for cos-beta-60-9592-52-0 (docker 1.13.1).

Ref: #42926

**Release note**:
```
None
```
/sig node
/area node-e2e
/assign @dchen1107
/cc @abgworrall
2017-07-14 16:49:55 -07:00
Kubernetes Submit Queue cab07f3af0 Merge pull request #46893 from yguo0905/image-spec
Automatic merge from submit-queue (batch tested with PRs 48890, 46893, 48872, 48896)

Support customized system spec in the node conformance test and create the GKE system spec

ref: https://github.com/kubernetes/kubernetes/issues/46891

- System specs are located in `test/e2e_node/system/specs`. Created one for validating GKE images in `test/e2e_node/system/specs/gke.yaml`.
- `--image-spec-name` can be used to specify a system spec in node e2e and conformance tests. This option maps to `SYSTEM_SPEC_NAME` in a test properties file, which is the user facing configuration. So, users can specify `SYSTEM_SPEC_NAME=gke` to run the image validation using the GKE system spec.
- If `SYSTEM_SPEC_NAME` is unspecified, the default spec (`system.DefaultSysSpec`) will be used.
- We can also use `make test-e2e-node SYSTEM_SPEC_NAME=gke` to run tests using GKE image spec.

**Release note**:
`None`
2017-07-14 16:49:52 -07:00
Kubernetes Submit Queue 23e60ac9ad Merge pull request #48308 from yguo0905/docker-api
Automatic merge from submit-queue

Update Docker API in Kubelet

Ref: https://github.com/kubernetes/kubernetes/issues/34308

The Kubelet currently uses deprecated docker API (https://godoc.org/github.com/docker/engine-api). This PR changes it to use the new one (https://godoc.org/github.com/moby/moby/client).

This PR updated the docker package from 1.11 to 1.13.1-rc2.

**Release note**:
```
None
```

/assign @Random-Liu 
/cc @yujuhong
2017-07-14 15:30:59 -07:00
Yang Guo 22c9e23202 Supports customized system spec in the node conformance test and creates the GKE system spec 2017-07-14 09:39:19 -07:00
Kubernetes Submit Queue a14abaabab Merge pull request #48824 from yguo0905/docker-validation
Automatic merge from submit-queue (batch tested with PRs 48082, 48815, 48901, 48824)

Add test image name to the OS image field of the perf metrics

I'd like to add the resource usage benchmarks for COS m60 (docker 1.13.1) but don't want to remove the existing m59 (docker 1.11.2) [ones](https://github.com/kubernetes/kubernetes/blob/master/test/e2e_node/jenkins/benchmark/benchmark-config.yaml#L51-L71), in order to compare the results between the two docker versions.

The `image` reported in the metrics is from `Node.Status.NodeInfo.OSImage`, which is always "Container-Optimized OS from Google" (from `/etc/os-releases`) for COS. So there's no way to differentiate two milestones in the metrics.

This PR attaches the [image name](https://github.com/kubernetes/kubernetes/blob/master/test/e2e_node/jenkins/benchmark/benchmark-config.yaml#L52) to the `image` field of the metrics. So it will become "Container-Optimized OS from Google (cos-stable-59-9460-64-0)".

See the results of the test run:

[performance-memory-containervm-resource1-resource_0.json](https://storage.googleapis.com/ygg-gke-dev-bucket/e2e-node-test/ci-kubernetes-node-kubelet-benchmark/13/artifacts/performance-memory-containervm-resource1-resource_0.json)
[performance-memory-coreos-resource1-resource_0.json](https://storage.googleapis.com/ygg-gke-dev-bucket/e2e-node-test/ci-kubernetes-node-kubelet-benchmark/13/artifacts/performance-memory-coreos-resource1-resource_0.json)
[performance-memory-gci-resource1-resource_0.json](https://storage.googleapis.com/ygg-gke-dev-bucket/e2e-node-test/ci-kubernetes-node-kubelet-benchmark/13/artifacts/performance-memory-gci-resource1-resource_0.json)

**Release note**:
```
None
```

Ref: https://github.com/kubernetes/kubernetes/issues/42926

/sig node
/area node-e2e
/assign @dchen1107
2017-07-13 22:44:00 -07:00
Kubernetes Submit Queue 8ad1be7833 Merge pull request #44475 from freehan/checkpoint-test
Automatic merge from submit-queue

add dockershim checkpoint node e2e test

Add a bunch of disruptive cases to test kubelet/dockershim's checkpoint work flow.

Some steps are quite hacky. Not sure if there is better ways to do things.
2017-07-13 18:50:10 -07:00
Yang Guo bf2ced837c Updates Docker Engine API 2017-07-13 12:55:07 -07:00
Yang Guo 22253a6e6a Add cos-beta-60-9592-52-0 to benchmark tests 2017-07-13 12:06:59 -07:00
Jing Xu bb1920edcc Fix issues for local storage allocatable feature
This PR fixes the following issues:
1. Use ResourceStorageScratch instead of ResourceStorage API to represent
local storage capacity
2. In eviction manager, use container manager instead of node provider
(kubelet) to retrieve the node capacity and reserved resources. Node
provider (kubelet) has a feature gate so that storagescratch information
may not be exposed if feature gate is not set. On the other hand,
container manager has all the capacity and allocatable resource
information.
2017-07-13 12:06:19 -07:00
Maru Newby 6ba0e92bf4 fed: Enable the namespace controller in integration tests 2017-07-13 09:50:07 -07:00
Pengfei Ni 721047fe49 Add e2e test for readOnlyRootFilesystem containers 2017-07-13 17:21:29 +08:00
Yang Guo b17c6a1769 Add test image name to the OS image field of the perf metrics 2017-07-12 14:51:45 -07:00
Kubernetes Submit Queue 0e461035cb Merge pull request #48734 from tallclair/namechange
Automatic merge from submit-queue (batch tested with PRs 48698, 48712, 48516, 48734, 48735)

Name change: s/timstclair/tallclair/

I changed my name, and I'm migrating my user name to be consistent.
2017-07-12 04:56:32 -07:00
Kubernetes Submit Queue de30789bf5 Merge pull request #48598 from gmarek/metrics
Automatic merge from submit-queue (batch tested with PRs 46865, 48661, 48598, 48658, 48614)

Move metrics_grabbert to test/e2e

cc @aleksandra-malinowska
2017-07-12 03:02:19 -07:00
Tim Allclair a2f2e1d491 Name change: s/timstclair/tallclair/ 2017-07-10 14:05:46 -07:00
Cao Shufeng 0c577c47d5 Use glog.*f when a format string is passed
ref:
https://godoc.org/github.com/golang/glog

I use the following commands to search all the invalid usage:
$ grep "glog.Warning(" * -r | grep %
$ grep "glog.Info(" * -r | grep %
$ grep "glog.Error(" * -r | grep %
$ grep ").Info(" * -r | grep % | grep "glog.V("
2017-07-10 19:04:03 +08:00
gmarek 55880e6b4b Move metrics_grabbert to test/e2e 2017-07-07 13:13:44 +02:00
Minhan Xia 6da0c11063 add dockershim checkpoint node e2e test 2017-06-29 13:26:09 -07:00
Pengfei Ni 00eeb7f53a Add node e2e tests for runAsUser 2017-06-29 09:17:14 +08:00
Kubernetes Submit Queue 165c94aa7b Merge pull request #47549 from yguo0905/change-tested-images
Automatic merge from submit-queue

Changes node e2e tests to use the new Ubuntu image

ref: https://github.com/kubernetes/kubernetes/issues/46891

`ubuntu-docker10` and `ubuntu-docker12` images are deprecated in favor of the new one.

**Release note**:
```
None
```
/sig node
/area node-e2e
/assign @dchen1107
2017-06-27 23:30:24 -07:00
Kubernetes Submit Queue 98ee52ed78 Merge pull request #48001 from yguo0905/report-prefix
Automatic merge from submit-queue (batch tested with PRs 47675, 48001)

Encodes ReportPrefix into the generated metrics file names

Ref: https://github.com/kubernetes/kubernetes/issues/44003

Adds the test prefix to be part of the name. Otherwise the same test case running on different images will override each other. Nothing needs to be changed at the node-perf-dash side.

See test run at https://console.cloud.google.com/storage/browser/ygg-gke-dev-bucket/e2e-node-test/ci-kubernetes-node-kubelet-benchmark/10.


**Release note**:
```
None
```

/sig node
/area node-e2e
/assign @Random-Liu
2017-06-27 16:11:07 -07:00
Kubernetes Submit Queue 0dad2d0803 Merge pull request #47983 from yguo0905/memcg
Automatic merge from submit-queue (batch tested with PRs 48092, 47894, 47983)

Enables memcg notification in cluster/node e2e tests

Ref: https://github.com/kubernetes/kubernetes/issues/42676

This PR sets Kubelet flag `--experimental-kernel-memcg-notification=true` when running cluster/node e2e tests on COS and Ubuntu images.

Tested:
```
e2e-node-cos:
I0623 00:09:06.641776    1080 server.go:147] Starting server "kubelet" with command "/usr/bin/systemd-run --unit=kubelet-777178888.service --slice=runtime.slice --remain-after-exit /tmp/node-e2e-20170622T170739/kubelet --kubelet-cgroups=/kubelet.slice --cgroup-root=/ --api-servers http://localhost:8080 --address 0.0.0.0 --port 10250 --read-only-port 10255 --volume-stats-agg-period 10s --allow-privileged true --serialize-image-pulls false --pod-manifest-path /tmp/node-e2e-20170622T170739/pod-manifest571288056 --file-check-frequency 10s --pod-cidr 10.100.0.0/24 --eviction-pressure-transition-period 30s --feature-gates  --eviction-hard memory.available<250Mi,nodefs.available<10%%,nodefs.inodesFree<5%% --eviction-minimum-reclaim nodefs.available=5%%,nodefs.inodesFree=5%% --v 4 --logtostderr --network-plugin=kubenet --cni-bin-dir /tmp/node-e2e-20170622T170739/cni/bin --cni-conf-dir /tmp/node-e2e-20170622T170739/cni/net.d --hostname-override tmp-node-e2e-bfe5799d-cos-stable-59-9460-64-0 --experimental-mounter-path=/tmp/node-e2e-20170622T170739/cluster/gce/gci/mounter/mounter --experimental-kernel-memcg-notification=true"

e2e-node-ubuntu:
I0623 00:03:28.526984    2279 server.go:147] Starting server "kubelet" with command "/usr/bin/systemd-run --unit=kubelet-1407651753.service --slice=runtime.slice --remain-after-exit /tmp/node-e2e-20170622T170203/kubelet --kubelet-cgroups=/kubelet.slice --cgroup-root=/ --api-servers http://localhost:8080 --address 0.0.0.0 --port 10250 --read-only-port 10255 --volume-stats-agg-period 10s --allow-privileged true --serialize-image-pulls false --pod-manifest-path /tmp/node-e2e-20170622T170203/pod-manifest083943734 --file-check-frequency 10s --pod-cidr 10.100.0.0/24 --eviction-pressure-transition-period 30s --feature-gates  --eviction-hard memory.available<250Mi,nodefs.available<10%%,nodefs.inodesFree<5%% --eviction-minimum-reclaim nodefs.available=5%%,nodefs.inodesFree=5%% --v 4 --logtostderr --network-plugin=kubenet --cni-bin-dir /tmp/node-e2e-20170622T170203/cni/bin --cni-conf-dir /tmp/node-e2e-20170622T170203/cni/net.d --hostname-override tmp-node-e2e-e48cdd73-ubuntu-gke-1604-xenial-v20170420-1 --experimental-kernel-memcg-notification=true"

e2e-node-containervm:
I0623 00:14:35.392383    2774 server.go:147] Starting server "kubelet" with command "/tmp/node-e2e-20170622T171318/kubelet --runtime-cgroups=/docker-daemon --kubelet-cgroups=/kubelet --cgroup-root=/ --system-cgroups=/system --api-servers http://localhost:8080 --address 0.0.0.0 --port 10250 --read-only-port 10255 --volume-stats-agg-period 10s --allow-privileged true --serialize-image-pulls false --pod-manifest-path /tmp/node-e2e-20170622T171318/pod-manifest507536807 --file-check-frequency 10s --pod-cidr 10.100.0.0/24 --eviction-pressure-transition-period 30s --feature-gates  --eviction-hard memory.available<250Mi,nodefs.available<10%,nodefs.inodesFree<5% --eviction-minimum-reclaim nodefs.available=5%,nodefs.inodesFree=5% --v 4 --logtostderr --network-plugin=kubenet --cni-bin-dir /tmp/node-e2e-20170622T171318/cni/bin --cni-conf-dir /tmp/node-e2e-20170622T171318/cni/net.d --hostname-override tmp-node-e2e-9e3fdd7c-e2e-node-containervm-v20161208-image"

e2e-cos:
Jun 23 17:54:38 e2e-test-ygg-minion-group-t5r0 kubelet[2005]: I0623 17:54:38.646374    2005 flags.go:52] FLAG: --experimental-kernel-memcg-notification="true"

e2e-ubuntu:
Jun 23 18:25:27 e2e-test-ygg-minion-group-19qp kubelet[1547]: I0623 18:25:27.722253    1547 flags.go:52] FLAG: --experimental-kernel-memcg-notification="true"

e2e-containervm:
I0623 18:55:51.886632    3385 flags.go:52] FLAG: --experimental-kernel-memcg-notification="false"
```

**Release note**:
```
None
```

/sig node
/area node-e2e
/assign @dchen1107 @dashpole
2017-06-26 21:08:10 -07:00
Kubernetes Submit Queue 36ae4ae4e3 Merge pull request #47971 from yujuhong/bump-usage-limit
Automatic merge from submit-queue (batch tested with PRs 48074, 47971, 48044, 47514, 47647)

e2e: bump kubelet's resurce usage limit

We don't have per-OS image limits. Bumping these to more generous
numbers to not fail the tests.
2017-06-26 11:40:51 -07:00
Yang Guo 50d49d9c51 Enables memcg notification in cluster/node e2e tests 2017-06-26 11:40:22 -07:00
Kubernetes Submit Queue 14edc46c2e Merge pull request #47892 from ajitak/npd-config
Automatic merge from submit-queue (batch tested with PRs 47993, 47892, 47591, 47469, 47845)

Bump up npd version to v0.4.1

```
Bump up npd version to v0.4.1
```

Fixes #47219
2017-06-23 18:05:46 -07:00
Yang Guo 8ab15e3774 Encodes ReportPrefix into the generated metrics file names 2017-06-23 16:11:25 -07:00
Yu-Ju Hong 71bd92ce3b e2e: bump kubelet's resurce usage limit
We don't have per-OS image limits. Bumping these to more generous
numbers to not fail the tests.
2017-06-23 09:55:18 -07:00
Kubernetes Submit Queue 467705be00 Merge pull request #47195 from dims/bind-cadvisor-on-kubelet-interface
Automatic merge from submit-queue (batch tested with PRs 47922, 47195, 47241, 47095, 47401)

Run cAdvisor on the same interface as kubelet

**What this PR does / why we need it**:

cAdvisor currently binds to all interfaces. Currently the only
solution is to use iptables to block access to the port. We
are better off making cAdvisor to bind to the interface that
kubelet uses for better security.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

Fixes #11710

**Special notes for your reviewer**:

**Release note**:

```release-note
cAdvisor binds only to the interface that kubelet is running on instead of all interfaces.
```
2017-06-22 21:33:27 -07:00
Ajit Kumar caff16c678 Bump up npd version to v0.4.1 2017-06-22 13:13:50 -07:00
Chao Xu 60604f8818 run hack/update-all 2017-06-22 11:31:03 -07:00
Chao Xu f4989a45a5 run root-rewrite-v1-..., compile 2017-06-22 10:25:57 -07:00
mbohlool c91a12d205 Remove all references to types.UnixUserID and types.UnixGroupID 2017-06-21 04:09:07 -07:00
Kubernetes Submit Queue cc645a8c6f Merge pull request #46327 from supereagle/mark-network-plugin-dir-deprecated
Automatic merge from submit-queue (batch tested with PRs 46327, 47166)

mark --network-plugin-dir deprecated for kubelet

**What this PR does / why we need it**:

**Which issue this PR fixes** : fixes #43967

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-06-19 11:23:54 -07:00
Kubernetes Submit Queue b6faf34862 Merge pull request #47530 from mindprince/issue-47388-remove-dead-code
Automatic merge from submit-queue (batch tested with PRs 47530, 47679)

Use cos-stable-59-9460-64-0 instead of cos-beta-59-9460-20-0.

Remove dead code that has now moved to another repo as part of #47467

**Release note**:
```release-note
NONE
```

/sig node
2017-06-16 20:57:58 -07:00
Rohit Agarwal 3a86c97cf6 Use cos-stable-59-9460-64-0 instead of cos-beta-59-9460-20-0.
- It contains a fix for ipaliasing.
- It contains a fix which decouples GPU driver installation from kernel
version.

Remove dead code that has now moved to another repo as part of #47467
2017-06-16 13:48:50 -07:00
Bowei Du 1ed4afca80 Fix hardcoded CIDR in the validation_test
The ideal fix is to not hardcode these values.

fixes #47479
2017-06-15 22:15:56 -07:00
Kubernetes Submit Queue d797c219b3 Merge pull request #47260 from yguo0905/perf-dash
Automatic merge from submit-queue (batch tested with PRs 47470, 47260, 47411, 46852, 46135)

Logs node e2e perf data to standalone json files

Fixes the node-dash-perf issue in https://github.com/kubernetes/kubernetes/issues/44003.

- Move perf data types to `test/e2e/perftype/perftype.go` so that the node-perf-dash can depend on.
- Logs the perf data to standalone json files so that node-perf-dash can consume it easily. A sample run of `ci-kubernetes-node-kubelet-benchmark` is at https://console.cloud.google.com/storage/browser/ygg-gke-dev-bucket/e2e-node-test/ci-kubernetes-node-kubelet-benchmark/1.

The corresponding changes in node-perf-dash is at https://github.com/kubernetes/contrib/pull/2628.

**Release note**:
`None`

/sig node
/area node-e2e
/assign @Random-Liu
2017-06-14 12:52:18 -07:00
Yang Guo 404cda2777 Changes node e2e tests to use new Ubuntu image 2017-06-14 11:44:25 -07:00
Rohit Agarwal 9c0bf19f80 Use cos-stable-59-9460-60-0 and newer installer for GPU node e2e tests. 2017-06-13 15:36:20 -07:00
Kubernetes Submit Queue f4d2c7b931 Merge pull request #46441 from dashpole/eviction_time
Automatic merge from submit-queue

Shorten eviction tests, and increase test suite timeout

After #43590, the eviction manager is less aggressive when evicting pods.  Because of that, many runs in the flaky suite time out.
To shorten the inode eviction test, I have lowered the eviction threshold.
To shorten the allocatable eviction test, I now set KubeReserved = NodeMemoryCapacity - 200Mb, so that any pod using 200Mb will be evicted.  This shortens this test from 40 minutes, to 10 minutes.
While this should be enough to not hit the flaky suite timeout anymore, it is better to keep lower individual test timeouts than a lower suite timeout, since hitting the suite timeout means that even successful test runs are not reported.

/assign @Random-Liu @mtaufen 

issue: #31362
2017-06-13 12:58:22 -07:00
Yang Guo 29b2db5af3 Logs node e2e perf data to standalone json files 2017-06-12 14:27:56 -07:00
David Ashpole 3365cca78a shorten eviction testst and lengthen flaky suite timeout 2017-06-12 12:56:45 -07:00
Rohit Agarwal f7a563435f Fix bad check in node e2e tests for GPUs.
When no nvidia device was attached, the -ne check had a syntax error:

    sh: -ne: argument expected

This resulted in 'Success' being echoed and the test passing incorrectly.
This was found while debugging issue #47216
2017-06-11 19:25:35 -07:00
Kubernetes Submit Queue 3040cba17d Merge pull request #47144 from jingxu97/May/emptyDir
Automatic merge from submit-queue

Fix local capacity isolation test
2017-06-09 12:17:19 -07:00
Kubernetes Submit Queue f75478875a Merge pull request #47113 from feiskyer/cri
Automatic merge from submit-queue

Kubelet: rename cri package name to pkg/kubelet/apis/cri/v1alpha1/runtime

**What this PR does / why we need it**:

We have moved CRI from api/v1alpha1/runtime to apis/cri/v1alpha1, which changed the package name of CRI. This would cause a significant problem: old-versioned runtime (based on CRI in v1.6) doesn't work with latest kubelet v1.7, and vice versa.

This PR renames cri package name to `pkg/kubelet/apis/cri/v1alpha1/runtime` for fixing the problem.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: 

fixes #47012

**Special notes for your reviewer**:

Should be included in v1.7.

**Release note**:

```release-note
CRI has been moved to package `pkg/kubelet/apis/cri/v1alpha1/runtime`.
```
2017-06-09 10:08:36 -07:00
Kubernetes Submit Queue 3a5df705fe Merge pull request #47190 from mindprince/faster-node-e2e-gci
Automatic merge from submit-queue

Move the nvidia installer to the beginning.

When the installer runs for the first time, it disables loadpin and restarts
the node. So, it is better to run it in the beginning so that we can avoid
redoing the later steps. One of the later steps include downloading a tar file
and untarring it. Doing that only once saves around 1m30s in test runtime for
the gci image.

/sig node
/area node-e2e

```release-note
NONE
```
2017-06-09 09:19:16 -07:00
Pengfei Ni 22e99504d7 Update CRI references 2017-06-09 10:16:40 +08:00
Kubernetes Submit Queue 3a96c31de5 Merge pull request #46885 from kewu1992/test_gci_next_canary
Automatic merge from submit-queue (batch tested with PRs 46885, 47197)

Let COS docker validation node test against gci-next-canary

**What this PR does / why we need it**:
This is for COS docker validation node test. We plan to use family gci-next-canary in container-vm-image-staging for future Docker upgration and validation.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #47134

**Special notes for your reviewer**:

**Release note**:

```release-note
```
2017-06-08 15:46:41 -07:00
Davanum Srinivas 7e5c43a042 Run cAdvisor on the same interface as kubelet
cAdvisor currently binds to all interfaces. Currently the only
solution is to use iptables to block access to the port. We
are better off making cAdvisor to bind to the interface that
kubelet uses for better security.

Fixes #11710
2017-06-08 16:43:38 -04:00
Rohit Agarwal 4a5badfafa Move the nvidia installer to the beginning.
When the installer runs for the first time, it disables loadpin and restarts
the node. So, it is better to run it in the beginning so that we can avoid
redoing the later steps. One of the later steps include downloading a tar file
and untarring it. Doing that only once saves around 1m30s in test runtime for
the gci image.
2017-06-08 09:55:14 -07:00
Kubernetes Submit Queue 9c1b2aa9b5 Merge pull request #46743 from Random-Liu/bump-up-npd
Automatic merge from submit-queue

Bump up npd version to v0.4.0

Fixes #47070.

Bump up npd version to [v0.4.0](https://github.com/kubernetes/node-problem-detector/releases/tag/v0.4.0).

```release-note
Bump up Node Problem Detector version to v0.4.0, which added support of parsing log from /dev/kmsg and ABRT.
```

/cc @dchen1107 @ajitak
2017-06-08 08:24:18 -07:00
Jing Xu 426d44ded4 Fix local capacity isolation test
Fix issue #47128, also add flaky tag for this evition test
2017-06-08 06:30:29 -07:00
Kubernetes Submit Queue 6ee028249b Merge pull request #46385 from rickypai/rpai/host_mapping_node_e2e_test
Automatic merge from submit-queue (batch tested with PRs 43005, 46660, 46385, 46991, 47103)

add e2e node test for Pod hostAliases feature

**What this PR does / why we need it**: adds node e2e test for #45148 

tests requested in https://github.com/kubernetes/kubernetes/issues/43632#issuecomment-298434125

**Release note**:
```release-note
NONE
```

@yujuhong @thockin
2017-06-07 13:31:00 -07:00
Random-Liu 1d3979190c Bump up npd version to v0.4.0 2017-06-06 16:30:02 -07:00
Kubernetes Submit Queue 6ed4bc7b97 Merge pull request #46828 from cblecker/links-update
Automatic merge from submit-queue (batch tested with PRs 46718, 46828, 46988)

Update docs/ links to point to main site

**What this PR does / why we need it**:
This updates various links to either point to kubernetes.io or to the kubernetes/community repo instead of the legacy docs/ tree in k/k
Pre-requisite for #46813

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```

@kubernetes/sig-docs-maintainers @chenopis @ahmetb @thockin
2017-06-06 11:43:18 -07:00
Kubernetes Submit Queue 3338d784ba Merge pull request #46899 from mindprince/issue-46889-node-e2e-gpu-cos-fix
Automatic merge from submit-queue (batch tested with PRs 46897, 46899, 46864, 46854, 46875)

Wait for cloud-init to finish before starting tests.

This fixes #46889.


**Release note**:
```release-note
NONE
```
2017-06-06 05:22:42 -07:00
Christoph Blecker 1bdc7a29ae
Update docs/ URLs to point to proper locations 2017-06-05 22:13:54 -07:00
Jing Xu 0b13aee0c0 Add EmptyDir Volume and local storage for container overlay Isolation
This PR adds two features:
1. add support for isolating the emptyDir volume use. If user
sets a size limit for emptyDir volume, kubelet's eviction manager
monitors its usage
and evict the pod if the usage exceeds the limit.
2. add support for isolating the local storage for container overlay. If
the container's overly usage exceeds the limit defined in container
spec, eviction manager will evict the pod.
2017-06-05 12:05:48 -07:00
Rohit Agarwal 1561f55c4c Wait for cloud-init to finish before starting tests.
This fixes #46889.
2017-06-05 10:50:24 -07:00
Kubernetes Submit Queue 6fef1a1deb Merge pull request #46810 from vishh/gpu-cos-image-validation
Automatic merge from submit-queue (batch tested with PRs 46734, 46810, 46759, 46259, 46771)

Update the COS kernel sha for node e2e gpu installer

cc @mindprince

Relevant COS image - https://github.com/kubernetes/kubernetes/blob/master/test/e2e_node/jenkins/image-config-serial.yaml#L19
2017-06-05 06:51:23 -07:00
Kubernetes Submit Queue e6c74bbaaf Merge pull request #46221 from FengyunPan/close-file
Automatic merge from submit-queue

Close file after os.Open()

None
2017-06-03 04:42:00 -07:00
Kubernetes Submit Queue b8c9ee8abb Merge pull request #46456 from jingxu97/May/allocatable
Automatic merge from submit-queue

Add local storage (scratch space) allocatable support

This PR adds the support for allocatable local storage (scratch space).
This feature is only for root file system which is shared by kubernetes
componenets, users' containers and/or images. User could use
--kube-reserved flag to reserve the storage for kube system components.
If the allocatable storage for user's pods is used up, some pods will be
evicted to free the storage resource.

This feature is part of local storage capacity isolation and described in the proposal https://github.com/kubernetes/community/pull/306

**Release note**:

```release-note
This feature exposes local storage capacity for the primary partitions, and supports & enforces storage reservation in Node Allocatable 
```
2017-06-03 00:24:29 -07:00
Kubernetes Submit Queue d063ce213f Merge pull request #46801 from dashpole/summary_container_restart
Automatic merge from submit-queue

[Flaky PR Test] Fix summary test

fixes issue: #46797 

As we can see in the [example failure build log](https://storage.googleapis.com/kubernetes-jenkins/logs/ci-kubernetes-node-kubelet/4319/build-log.txt), the summary containers are pinging google 100s of times a second.  This causes the summary container to be killed occasionally, and fail the test.  The summary containers are only supposed to ping every 10 seconds according to the current test.  As it turns out, we were missing a semicolon, and were not sleeping between pings.  For background, we ping google to generate network traffic, so that the summary test can validate network metrics.

This PR adds the semicolon to make the container sleep between calls, and decreases the sleep time from 10 seconds to 1 second, as 1 call / 10 seconds did not produce enough activity.

cc @kubernetes/kubernetes-build-cops @dchen1107
2017-06-02 18:02:19 -07:00
Ricky Pai 4e7fed4479 e2e node test for PodSpec HostAliases 2017-06-02 17:01:44 -07:00
Kubernetes Submit Queue a6f0033164 Merge pull request #46238 from yguo0905/package-validator
Automatic merge from submit-queue (batch tested with PRs 46648, 46500, 46238, 46668, 46557)

Support validating package versions in node conformance test

**What this PR does / why we need it**:

This PR adds a package validator in node conformance test for checking whether the locally installed packages meet the image spec.

**Special notes for your reviewer**:

The image spec for GKE (which has the package spec) will be in a separate PR. Then we will publish a new node conformance test image for GKE whose name should use the convention in https://github.com/kubernetes/kubernetes/issues/45760 and have `gke` in it.


**Release note**:
```
NONE
```
2017-06-02 15:20:47 -07:00