Commit Graph

1465 Commits (b01e2a387d015b36527fa5dc6d4fc6afe764f571)

Author SHA1 Message Date
Jeff Grafton efee0704c6 Autogenerate BUILD files 2017-12-23 13:12:11 -08:00
Tim Hockin e9dd8a68f6 Revert k8s.gcr.io vanity domain
This reverts commit eba5b6092a.

Fixes https://github.com/kubernetes/kubernetes/issues/57526
2017-12-22 14:36:16 -08:00
Kubernetes Submit Queue 8f8579d439
Merge pull request #57392 from chrisglass/gke-spec-tweaks
Automatic merge from submit-queue (batch tested with PRs 57532, 57392). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Gke spec tweaks

**What this PR does / why we need it**:
This PR removes two unnecessary requirements for the GKE node image validation spec:

- The "vim" package doesn't need to be installed, as a vim environment is already available and the full "vim" pacakge installation takes some precious disk space.
- The linux headers are not needed for the kubernetes node and cluster tests to succeed, and again, take unnecessary disk space.

**Special notes for your reviewer**:
None.

**Release note**:

```release-note
NONE
```
2017-12-21 20:00:34 -08:00
Kubernetes Submit Queue fb4dc98016
Merge pull request #57532 from mtaufen/fix-hyphen-dkcfg-test
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

dynamic config test: use a hyphen between the config name and the unique suffix

These are painful to read right now due to the lack of hyphen.

```release-note
NONE
```
2017-12-21 19:27:31 -08:00
Michael Taufen 33d4fe4074 dynamic config test: use a hyphen between the config name and the unique suffix 2017-12-21 16:05:50 -08:00
Michael Taufen 6ee191ab74 Refactor kubelet config controller bootstrap process
This makes the bootstrap feel much more linear and as a result it is
easier to read.

Also simplifies status reporting for local config.
2017-12-21 15:24:56 -08:00
Kubernetes Submit Queue 2a1cdfffaa
Merge pull request #57221 from mtaufen/kc-event
Automatic merge from submit-queue (batch tested with PRs 57434, 57221, 57417, 57474, 57481). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Send an event just before the Kubelet restarts to use a new config

**What this PR does / why we need it**:
This PR makes the Kubelet send events for configuration changes. This makes it much easier to see a recent history of configuration changes for the Kubelet. 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56895

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```

/cc @dchen1107 @liggitt @dashpole
2017-12-20 17:42:37 -08:00
Kubernetes Submit Queue 09e5ce3056
Merge pull request #57434 from yguo0905/fix-gke-env-test
Automatic merge from submit-queue (batch tested with PRs 57434, 57221, 57417, 57474, 57481). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Node e2e test: do not return error if Docker's check-config.sh fails

**What this PR does / why we need it**:

This PR fixes the issue in https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2enode-ubuntudev2-k8sbeta-gkespec/1#k8sio-gke-system-requirements-conformance-featuregkeenv-the-docker-configuration-validation-should-pass

https://github.com/moby/moby/pull/28007 changed the Docker's check-config.sh to return a non-zero exit code if the check fails. But we expect certain configs to be missing, so we should ignore the exit code.

This needs to be cherry-picked into 1.9.

**Release note**:

```release-note
None
```

/area node-e2e
/kind bug
/assign @dchen1107
2017-12-20 17:42:34 -08:00
Michael Taufen d5d7d6d684 Send an event just before the Kubelet restarts to use a new config 2017-12-20 13:02:55 -08:00
Kubernetes Submit Queue 666b95743a
Merge pull request #54278 from foxyriver/fix-exit
Automatic merge from submit-queue (batch tested with PRs 54278, 56259, 56762). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

make sure of deleting archive

**What this PR does / why we need it**:

Exit() causes the current program to exit with the given status code, but deferred function does not run.
2017-12-20 12:32:32 -08:00
Yang Guo 7e1d74ead2 node_e2e: do not return error if Docker's check-config.sh fails 2017-12-19 20:50:09 -08:00
Wojciech Tyczynski 4e8526dc6b
Revert "Version bump to etcd v3.2.11, grpc v1.7.5" 2017-12-19 15:25:06 +01:00
Kubernetes Submit Queue a7b404ec7f
Merge pull request #57160 from jpbetz/etcd-client-3.2.11
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Version bump to etcd v3.2.11, grpc v1.7.5

Fix https://github.com/kubernetes/kubernetes/issues/56114: Update to etcd client 3.2.11

Version bumps:

- etcd from 3.1.10 to 3.2.11
- grpc from 1.3.0 to 1.7.5
- grpc-gateway from v1.1.0-25-g84398b9 to v1.3.0

TODO:

- [x] Apply etcd [3.2 client upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md)
- [x] Apply grpc API changes in 1.6.0 and 1.7.0 [release notes](https://github.com/grpc/grpc-go/releases)
- [x] bbolt was pulled in transitively, why? We have tests that embed etcd, so we must vendor the etcd server and all it's dependencies.
- [x] Upgrade to containerd v1.0.0? Currently kubernetes depends on containerd v1.0.0-beta.2-159-g27d450a0 which depends on grpc v1.3.0, but  containerd v1.0.0 depends on grpc 1.7.2. Not needed. The containerd grpc upgrade required [no code changes](ce3e32680d).
- [x] Fix all failing tests
- [x] Ensure we can safely upgrade grpc to 1.7.5 given that docker and cAdvisor still depend on grpc 1.3.0 (both in the versions we vend and on master for both projects). Should we hold off on this change until we have a docker release that uses gprc 1.7.x?
- [x] Wait for grpc 1.7.5 to be released (it will include https://github.com/grpc/grpc-go/pull/1747). Once released, bump grpc version in this PR and remove workarounds in `hack/godep-save.sh`.

Transitive dependencies on grpc:
- docker depends on grpc, but according to the package dependency graph (`go list -f '{{ .Deps }}'`) there are no dependencies from kubernetes to grpc via docker packages.
- containerd v1.0.0 depends on grpc 1.7.2, we should upgrade to containerd v1.0.0 soon, this can be done in a separate PR
- cadvisor depends on grpc 1.3.0 on master, it should upgrade it to grpc 1.7.5, this can be done in a separate PR

**Release note**:

```release-note
Upgrade to etcd client 3.2.11 and grpc 1.7.5 to improve HA etcd cluster stability.
```
2017-12-19 02:14:36 -08:00
Chris Glass 3e3385821a Do not require the linux headers to be installed.
The linux headers take significant disk space and are not necessary to
run kubernetes on a GKE node. User logging on to a node can trivially
install the kernel headers should they need to by running "apt-get install
linux-headers-$(uname -r)".

Signed-off-by: Chris Glass <chris.glass@canonical.com>
2017-12-19 10:58:13 +01:00
Chris Glass be41d1e2ce Do not require the vim package to be installed
The minimal Ubuntu image used on GKE nodes provides the vim editor as
part of system packages, as "vim.tiny". People logging on the nodes have a vim
environment available despite the "vim" package not being installed.

Signed-off-by: Chris Glass <chris.glass@canonical.com>
2017-12-19 10:58:13 +01:00
Kubernetes Submit Queue c16a336eaa
Merge pull request #55475 from ScorpioCPH/fix-e2e-local-test
Automatic merge from submit-queue (batch tested with PRs 55475, 57155, 57260, 57222). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix e2e local test

**What this PR does / why we need it**:

Fix some issue on local e2e_node test: `Can't start e2e service "kubelet"`

**Which issue(s) this PR fixes**:

Fixes https://github.com/kubernetes/kubernetes/issues/54622

**Special notes for your reviewer**:

**Release note**:

```release-note

```
/sig node
2017-12-18 19:45:38 -08:00
Kubernetes Submit Queue 6eb6955371
Merge pull request #56108 from tpepper/trivial
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

e2e_node: small fixes to setup_host.sh for Ubuntu Trusty

**What this PR does / why we need it**:
Two small fixes for how nsenter is built from source and installed on Ubuntu Trusty:
1. Use mktemp for the creating the build directory instead of a hard coded name
2. Use current (2.31) util-linux instead of 3+ year old version

**Which issue(s) this PR fixes**:

Fixes #56106 

**Special notes for your reviewer**:

See https://github.com/kubernetes/kubernetes/issues/56106 for some thoughts on other ways to address this.  My patch for util-linux 2.31 may just be a band-aid?

**Release note**:
```release-note
NONE
```
2017-12-18 16:36:26 -08:00
Joe Betz 94f2ed6849 Fix build and test errors from etcd 3.2.11 upgrade 2017-12-18 16:17:42 -08:00
Tim Pepper 3f4294cdf9 e2e_node: use newer util-linux
The e2e_node test environment setup script is hard coded to pull down a
quite old version of util-linux in order to build nsenter on trusty,
which sadly is well known to not include an nsenter executable.

While "just a test", it's unfortunate to be building from really old
util-linux sources when newer are available.

Signed-off-by: Tim Pepper <tpepper@vmware.com>
2017-12-18 13:18:36 -08:00
Tim Pepper 5256e26f68 e2e_node: use mktemp when building nsenter on trusty
There's a bit of a hack in place to insure nsenter is present on Ubuntu
trusty, which doesn't otherwise include it.  This downloads util-linux
to a hard coded directory in /tmp which is a bad practice.  Even though
"this is just a test case" it should properly use mktemp.

Signed-off-by: Tim Pepper <tpepper@vmware.com>
2017-12-18 13:18:36 -08:00
Tim Hockin f7be352a67 gcloud docker now auths k8s.gcr.io by default 2017-12-18 09:18:34 -08:00
Tim Hockin eba5b6092a Use k8s.gcr.io vanity domain for container images 2017-12-18 09:18:34 -08:00
Kubernetes Submit Queue 0a940bfd10
Merge pull request #56250 from ingvagabund/extend-node-e2e-test-suite-with-containerized-kubelet
Automatic merge from submit-queue (batch tested with PRs 56250, 56809, 56812, 56792, 56724). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Extend node e2e test suite with containerized Kubelet

How to run it?
```sh
make test-e2e-node \
    TEST_ARGS='--kubelet-containerized=true --hyperkube-image=hyperkube-amd64:1.9 --kubelet-flags="<FLAGS>"' \
    FOCUS="Conformance"
```
2017-12-16 07:46:38 -08:00
Kubernetes Submit Queue 8415e0c608
Merge pull request #56661 from xiangpengzhao/move-kubelet-constants
Automatic merge from submit-queue (batch tested with PRs 56410, 56707, 56661, 54998, 56722). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Move some kubelet constants to a common place

**What this PR does / why we need it**:
More context, see: https://github.com/kubernetes/kubernetes/issues/56516
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56516
[thanks @ixdy for verifying this!]

**Special notes for your reviewer**:
@ixdy how can I verify #56516 against this locally?

/cc @ixdy @mtaufen 

**Release note**:

```release-note
NONE
```
2017-12-16 05:46:35 -08:00
Kubernetes Submit Queue 6c5eb50c8d
Merge pull request #55984 from derekwaynecarr/summary-tests
Automatic merge from submit-queue (batch tested with PRs 55954, 56037, 55866, 55984, 54994). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

kubelet summary api test updates

**What this PR does / why we need it**
Fixes https://github.com/kubernetes/kubernetes/issues/55985
Improve the accuracy of the test as follows:
- ensure memory bound checks for unconstrained group are limited by actual node capacity
- grow the fs capacity bounds so we can run on larger drives (i.e. my dev laptop)

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```
2017-12-13 23:25:59 -08:00
Kubernetes Submit Queue 7b0e6b7f14
Merge pull request #56909 from dashpole/fix_local_storage_test
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

[Test Fix] Fix Broken LocalStorageEviction test

**What this PR does / why we need it**:
In #56643, I added a new test case, but did not rename the pod.  The test failed because the new pod can not be created since there is a name collision.
It fixes these new test failures: https://k8s-testgrid.appspot.com/sig-node-kubelet#kubelet-flaky-gce-e2e&include-filter-by-regex=LocalStorageCapacityIsolationEviction

**Special notes for your reviewer**:
I tested it this time

**Release note**:
```release-note
NONE
```

/assign @yujuhong 
/sig node
/kind bug
/priority critical-urgent
2017-12-07 01:30:05 -08:00
Kubernetes Submit Queue f875ee596d
Merge pull request #56519 from jiayingz/e2e-node-readiness
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Make sure node is ready before calling getLocalNode to fix test failure.

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubernetes/issues/56518

**Special notes for your reviewer**:

**Release note**:

```release-note

```
2017-12-06 18:29:19 -08:00
David Ashpole 239217f84a fix test 2017-12-06 16:55:28 -08:00
David Ashpole 7294a1d8fa make requirements less precise for disk eviction test 2017-12-06 12:58:53 -08:00
Kubernetes Submit Queue a48f11c225
Merge pull request #55448 from yguo0905/spec-change-for-new-kernel
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix GKE system spec for OS images with kernel version 4.10+

Two changes are required for validating images with kernel version 4.10+.

1. Update the spec in gke.yaml to allow 4.10+ kernel.
2. Update the GKE environment docker validation to allow `CONFIG_DEVPTS_MULTIPLE_INSTANCES` to be missing in >= 4.8 kernel because this option has been removed in 4.8.

**Release note**:

```
None
```

/assign @dchen1107
2017-12-06 08:20:24 -08:00
xiangpengzhao 1f2262e6b0 Move some kubelet constants to a common place. 2017-12-01 11:24:04 +08:00
Kubernetes Submit Queue f9c93154fc
Merge pull request #56497 from rphillips/fixes/fix_disk_metrics_coreos
Automatic merge from submit-queue (batch tested with PRs 56497, 56500, 55018, 56544, 56425). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

e2e: eviction test redirect dd stderr

**What this PR does / why we need it**: Redirects `dd` stderr to /dev/null

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56234 

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```
2017-11-29 15:25:58 -08:00
Jiaying Zhang 8d9a2e09c4 Make sure node is ready before calling getLocalNode to fix test failure. 2017-11-28 15:18:17 -08:00
Kubernetes Submit Queue 8226973ae8
Merge pull request #52144 from andyxning/fix_network_value_for_stats_summary
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix network value for stats summary for multiple network interfaces

This PR is part of [Heapster #1788](https://github.com/kubernetes/heapster/pull/1788). 

The original reason is when there are more than one none `lo`, `docker0`, `veth` network interfaces instead of just one `eth0`, the network interface value is only partial and does not correct. For now, summary stats api only gets the eth0 network interface values.

The original issues about this can be find in [Heapster #1058](https://github.com/kubernetes/heapster/issues/1058) and [Cadvisor #1593](https://github.com/google/cadvisor/issues/1593).

```release-note
Fix stats summary network value when multiple network interfaces are available.
```

/cc @DirectXMan12 @piosz @xiangpengzhao @vishh @timstclair
2017-11-28 14:59:08 -08:00
Ryan Phillips 769b9eed35 e2e: eviction test redirect dd stderr
* redirect stderr to /dev/null

Fixes #56234
2017-11-28 08:42:17 -06:00
Jan Chaloupka 607e863f85 WIP: extend node e2e test suite with containerized Kubelet 2017-11-28 15:29:27 +01:00
Aaron Crickenberger 040b80d9a7 Add [sig-node] to some unowned e2e_node tests
Follow the SIGDescribe pattern used in test/e2e/foo tests
2017-11-27 11:35:44 -05:00
Kubernetes Submit Queue 1098a3012a
Merge pull request #56228 from balajismaniam/cpuman-test-feature-tag
Automatic merge from submit-queue (batch tested with PRs 52767, 55065, 55148, 56228, 56221). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add Feature tag to CPU Manager node e2e test.

**What this PR does / why we need it**: Adds a `Feature` tag to the CPU manager node e2e tests.

CC @ConnorDoyle
2017-11-22 19:49:39 -08:00
Balaji Subramaniam 7acdf3e15e Add Feature tag to CPU Manager node e2e test. 2017-11-22 11:49:35 -08:00
Connor Doyle 6b05ac10f0 Add balajismaniam, ConnorDoyle node-e2e approvers 2017-11-22 10:01:14 -08:00
Jing Xu a66ee2eb3f Add pod-level metric for CPU and memory stats
This PR adds the pod-level metrics for CPU and memory stats. cAdvisor
can get all pod cgroup information so we can add this pod-level CPU and
memory stats information from the corresponding pod cgroup
2017-11-22 09:25:23 -08:00
Jordan Liggitt ae7dccf2e9
Revert "Kubelet flags take precedence over config from files/ConfigMaps"
This reverts commit cbebb61450.
2017-11-21 23:55:43 -05:00
Kubernetes Submit Queue d7a96d5e88
Merge pull request #56097 from mtaufen/kc-file-e2e-test
Automatic merge from submit-queue (batch tested with PRs 51494, 56097, 56072, 56175). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Kubelet flags take precedence over config from files/ConfigMaps

Changes the Kubelet configuration flag precedence order so that flags
take precedence over config from files/ConfigMaps.

See:
https://docs.google.com/document/d/18-MsChpTkrMGCSqAQN9QGgWuuFoK90SznBbwVkfZryo/

Also modifies e2e node test suite to transform all relevant Kubelet flags into
a config file before starting tests when the KubeletConfigFile feature gate is
true, and turns on the KubeletConfigFile gate for all e2e node tests.
This allows the alpha dynamic Kubelet config feature to continue to 
work in tests after the precedence change.

fixes #56171

Related: https://github.com/kubernetes/features/issues/281


```release-note
CLI flags passed to the Kubelet now take precedence over Kubelet config files and dynamic Kubelet config. This helps ensure backwards compatible behavior across Kubelet binary updates.
```
2017-11-21 19:49:27 -08:00
Kubernetes Submit Queue 754017bef4
Merge pull request #56105 from balajismaniam/enable-cpuman-only-when-not-skipped
Automatic merge from submit-queue (batch tested with PRs 55340, 55329, 56168, 56170, 56105). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Enable cpu manager only if the node e2e test is not skipped.

**What this PR does / why we need it**: This PR enables cpu manager in Kubelet only if the node e2e tests are not skipped. This change fixes the failures seen in https://k8s-testgrid.appspot.com/sig-node-kubelet#kubelet-serial-gce-e2e. 

Fixes #56144
2017-11-21 18:56:39 -08:00
Kubernetes Submit Queue 3bb6eeeb07
Merge pull request #55340 from jiayingz/metrics
Automatic merge from submit-queue (batch tested with PRs 55340, 55329, 56168, 56170, 56105). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Adds device plugin allocation latency metric.

For #53497


**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note

```
2017-11-21 18:56:29 -08:00
Kubernetes Submit Queue 94a8d81172
Merge pull request #55447 from jingxu97/Nov/podmetric
Automatic merge from submit-queue (batch tested with PRs 55812, 55752, 55447, 55848, 50984). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add Pod-level local ephemeral storage metric in Summary API

This PR adds pod-level ephemeral storage metric into Summary API.
Pod-level ephemeral storage usage is the sum of all containers and local
ephemeral volume including EmptyDir (if not backed up by memory or
hugepages), configueMap, and downwardAPI.
Address issue #55978

**Release note**:
```release-note
Add pod-level local ephemeral storage metric in Summary API. Pod-level ephemeral storage reports the total filesystem usage for the containers and emptyDir volumes in the measured Pod.
```
2017-11-21 17:57:34 -08:00
Michael Taufen cbebb61450 Kubelet flags take precedence over config from files/ConfigMaps
Changes the Kubelet configuration flag precedence order so that flags
take precedence over config from files/ConfigMaps.

See issue #56171 for more details.

Also modifies e2e node test suite to transform all relevant Kubelet
flags into a config file before starting tests when the
KubeletConfigFile feature gate is true, and turns on the
KubeletConfigFile gate for all e2e node tests. This allows the alpha
dynamic Kubelet config feature to continue to work in tests after
the precedence change.
2017-11-21 16:02:27 -08:00
Kubernetes Submit Queue 03b7d77be4
Merge pull request #54316 from dashpole/disk_request_eviction
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Take disk requests into account during evictions

fixes #54314

This PR is part of the local storage feature, and it makes the eviction manager take disk requests into account during disk evictions.
This uses the same eviction strategy as we do for memory.
Disk requests are only considered when the LocalStorageCapacityIsolation feature gate is enabled.  This is enforced by adding a check for the feature gate in getRequests().
I have added unit testing to ensure that previous behavior is preserved when the feature gate is disabled.
Most of the changes are testing.  Reviewers should focus on changes in **eviction/helpers.go**

/sig node
/assign @jingxu97  @vishh
2017-11-21 14:31:47 -08:00
Jiaying Zhang 048bafdd0b Adds device plugin registration count metric and allocation latency metric. 2017-11-21 13:44:10 -08:00
Balaji Subramaniam 16e0f12253 Enable cpu manager only if the test is not skipped.
- Also, if KubeReserved is nil, allocate a map.
2017-11-21 10:48:54 -08:00
David Ashpole 8b3bd5ae60 take disk requests into account during evictions 2017-11-21 10:21:30 -08:00
Jiaying Zhang 990113ce60 Extends gpu_device_plugin e2e_node test to verify that scheduled pods
can continue to run even after device plugin deletion and kubelet
restarts.
2017-11-20 23:40:27 -08:00
Kubernetes Submit Queue 9fe2a62b90
Merge pull request #55338 from dashpole/remove_disk_allocatable
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Remove Ephemeral Storage Allocatable Evictions

Issue #52336

Rationale and docs change: https://github.com/kubernetes/community/pull/1275

cc @kubernetes/sig-node-pr-reviews 
cc @derekwaynecarr @vishh 
/assign @jingxu97 
/assign @dchen1107
2017-11-20 21:43:24 -08:00
Jing Xu 75ef18c4d3 Add Pod-level local ephemeral storage metric in Summary API
This PR adds pod-level ephemeral storage metric into Summary API.
Pod-level ephemeral storage usage is the sum of all containers and local
ephemeral volume including EmptyDir (if not backed up by memory or
hugepages), configueMap, and downwardAPI.
2017-11-20 16:32:38 -08:00
Kubernetes Submit Queue 3679b54b19
Merge pull request #55898 from dashpole/fix_flaky_allocatable
Automatic merge from submit-queue (batch tested with PRs 54837, 55970, 55912, 55898, 52977). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix Flaky Allocatable Setup Tests

**What this PR does / why we need it**:
Fixes a flaky node e2e serial test.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #55830

**Special notes for your reviewer**:
The test was flaking because we were reading the node status before the restarted kubelet had written it.
This fixes this by waiting until we see an updated node status (looking at the condition's heartbeat time).
This also fixes an incorrect error message.

**Release note**:
```release-note
NONE
```
2017-11-18 13:13:24 -08:00
Kubernetes Submit Queue 7d1085e122
Merge pull request #54837 from xiangpengzhao/conf-test
Automatic merge from submit-queue (batch tested with PRs 54837, 55970, 55912, 55898, 52977). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Use framework.ConformanceIt for node e2e conformance tests

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
ref #54726 #53909

**Special notes for your reviewer**:
/cc @mml 

**Release note**:

```release-note
NONE
```
2017-11-18 13:13:17 -08:00
Kubernetes Submit Queue ef3b27cbd4
Merge pull request #55642 from dashpole/disable_cadvisor_disk_for_cri
Automatic merge from submit-queue (batch tested with PRs 55642, 55897, 55835, 55496, 55313). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Disable container disk metrics when using the CRI stats integration

Issue: https://github.com/kubernetes/kubernetes/issues/51798

As explained in the issue, runtimes which make use of the CRI Stats API still have the performance overhead of collecting those same stats through cAdvisor.
The CRI Stats API has metrics for CPU, Memory, and Disk.  This PR significantly reduces the added overhead due to collecting these stats in both cAdvisor and in the runtime.
This PR disables container disk metrics, which are very expensive to collect.

This PR does not disable node-level disk stats, as the "Raw" container handler does not currently respect ignoring DiskUsageMetrics.
This PR factors out the logic for determining whether or not to use the CRI stats provider into a helper function, as cAdvisor is instantiated before it is passed to the kubelet as a dependency.

cc @kubernetes/sig-node-pr-reviews @derekwaynecarr  
/kind feature
/sig node

/assign @Random-Liu @derekwaynecarr
2017-11-18 10:46:30 -08:00
David Ashpole 527611ee41 remove disk allocatable evictions 2017-11-18 10:34:59 -08:00
Derek Carr db89b46ce7 kubelet summary api test updates 2017-11-17 22:30:49 -05:00
Andy Xie 64a8edfbcf fix network value for stats summary 2017-11-18 10:17:59 +08:00
xiangpengzhao 7fdea2b0cf Use framework.ConformanceIt for node e2e conformance tests 2017-11-17 17:28:20 +08:00
Michael Taufen 1085b6f730 Lift embedded structure out of eviction-related KubeletConfiguration fields
- Changes the following KubeletConfiguration fields from `string` to
`map[string]string`:
  - `EvictionHard`
  - `EvictionSoft`
  - `EvictionSoftGracePeriod`
  - `EvictionMinimumReclaim`
- Adds flag parsing shims to maintain Kubelet's public flags API, while
enabling structured input in the file API.
- Also removes `kubeletconfig.ConfigurationMap`, which was an ad-hoc flag
parsing shim living in the kubeletconfig API group, and replaces it
with the `MapStringString` shim introduced in this PR. Flag parsing
shims belong in a common place, not in the kubeletconfig API.
I manually audited these to ensure that this wouldn't cause errors
parsing the command line for syntax that would have previously been
error free (`kubeletconfig.ConfigurationMap` was unique in that it
allowed keys to be provided on the CLI without values. I believe this was
done in `flags.ConfigurationMap` to facilitate the `--node-labels` flag,
which rightfully accepts value-free keys, and that this shim was then
just copied to `kubeletconfig`). Fortunately, the affected fields
(`ExperimentalQOSReserved`, `SystemReserved`, and `KubeReserved`) expect
non-empty strings in the values of the map, and as a result passing the
empty string is already an error. Thus requiring keys shouldn't break
anyone's scripts.
- Updates code and tests accordingly.

Regarding eviction operators, directionality is already implicit in the
signal type (for a given signal, the decision to evict will be made when
crossing the threshold from either above or below, never both). There is
no need to expose an operator, such as `<`, in the API. By changing
`EvictionHard` and `EvictionSoft` to `map[string]string`, this PR
simplifies the experience of working with these fields via the
`KubeletConfiguration` type. Again, flags stay the same.

Other things:
- There is another flag parsing shim, `flags.ConfigurationMap`, from the
shared flag utility. The `NodeLabels` field still uses
`flags.ConfigurationMap`. This PR moves the allocation of the
`map[string]string` for the `NodeLabels` field from
`AddKubeletConfigFlags` to the defaulter for the external
`KubeletConfiguration` type. Flags are layered on top of an internal
object that has undergone conversion from a defaulted external object,
which means that previously the mere registration of flags would have
overwritten any previously-defined defaults for `NodeLabels` (fortunately
there were none).
2017-11-16 18:35:13 -08:00
David Ashpole 8f3e2f315e fix flaky allocatable test 2017-11-16 11:16:58 -08:00
Kubernetes Submit Queue 779105673a
Merge pull request #55188 from mindprince/accelerator-monitoring
Automatic merge from submit-queue (batch tested with PRs 55798, 49579, 54862, 55188, 51990). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add monitoring support for hardware accelerators

Currently only NVIDIA GPU monitoring is implemented.

Feature repo issue: https://github.com/kubernetes/features/issues/369
cAdvisor PR: https://github.com/google/cadvisor/pull/1762

/kind feature
/sig node
/sig instrumentation
/area hw-accelerators

**Release note**:
```release-note
Kubelet now exposes metrics for NVIDIA GPUs attached to the containers.
```
2017-11-16 03:09:21 -08:00
Yang Guo 7eb7cfe3ef Add a cloud-init script to disable live-restore 2017-11-14 21:40:13 -08:00
David Ashpole 220edbc6e3 disable container disk metrics when using the CRI stats integration 2017-11-14 11:43:08 -08:00
Kubernetes Submit Queue 41fe3ed5bc
Merge pull request #54405 from resouer/clean-docker-dep
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

[Part 1] Remove docker dep in kubelet startup

**What this PR does / why we need it**:

Remove dependency of docker during kubelet start up.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: 

Part 1 of #54090 

**Special notes for your reviewer**:
Changes include:

1. Move docker client initialization into dockershim pkg.
2. Pass a docker `ClientConfig` from kubelet to dockershim
3. Pass parameters needed by `FakeDockerClient` thru `ClientConfig` to dockershim

(TODO, the second part) Make dockershim tolerate when dockerd is down, otherwise it will still fail kubelet

Please note after this PR, kubelet will still fail if dockerd is down, this will be fixed in the subsequent PR by making dockershim tolerate dockerd failure (initializing docker client in a separate goroutine), and refactoring cgroup and log driver detection. 

**Release note**:

```release-note
Remove docker dependency during kubelet start up 
```
2017-11-13 03:59:53 -08:00
Rohit Agarwal 9c38abd482 Expose accelerator metrics in the summary API. 2017-11-10 14:59:43 -08:00
Yang Guo ed8cd396dd Use whitelisted test image 2017-11-10 14:16:27 -08:00
Yang Guo 8ea9417a37 Adjust GKE spec to validate images with kernel version 4.10+ 2017-11-10 09:47:08 -08:00
Penghao Cen 22b04c828b Append --feature-gates option iff TestContext.FeatureGates is not nil 2017-11-10 19:42:22 +08:00
Dr. Stefan Schimanski bec617f3cc Update generated files 2017-11-09 12:14:08 +01:00
Dr. Stefan Schimanski 012b085ac8 pkg/apis/core: mechanical import fixes in dependencies 2017-11-09 12:14:08 +01:00
Kubernetes Submit Queue f7dc3966a4
Merge pull request #47497 from mikedanese/binary
Automatic merge from submit-queue (batch tested with PRs 54773, 52523, 47497, 55356, 49429). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

don't check in mounter binary

```release-note
GCI mounter is moved from the manifests tarball to the server tarball.
```
2017-11-08 22:11:53 -08:00
Kubernetes Submit Queue fdeeed1001
Merge pull request #54688 from yanxuean/besteffort
Automatic merge from submit-queue (batch tested with PRs 53645, 54734, 54586, 55015, 54688). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

e2e-node:the value of bestEffortCgroup is wrong

Signed-off-by: yanxuean <yan.xuean@zte.com.cn>

**What this PR does / why we need it**:
The value of bestEffortCgroup is wrong in e2e-node. The test case is invalid actually.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```
2017-11-06 15:33:50 -08:00
zouyee 68c5ce19b8 [test/e2e_node]Redirect dl.k8s.io to the kubernetes-release GCS bucket 2017-11-02 12:18:50 +08:00
Harry Zhang de1c305356 Remove docker dep in kubelet startup
Update bazel
2017-11-01 10:03:01 +08:00
Kubernetes Submit Queue 94935721d5
Merge pull request #54160 from mtaufen/runtime-config-to-flags
Automatic merge from submit-queue (batch tested with PRs 54160, 54016). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Move runtime-related flags from KubeletConfiguration to KubeletFlags

With respect to https://github.com/kubernetes/kubernetes/pull/53833#issuecomment-336317287, move runtime-related flags out of KubeletConfiguration.

Broader issue: https://github.com/kubernetes/features/issues/281

```release-note
NONE
```
2017-10-31 01:23:15 -07:00
Mike Danese bef68f7dbc cluster: build gci mounter like other go binaries 2017-10-30 13:56:09 -07:00
Kubernetes Submit Queue eff1a84638
Merge pull request #52256 from feiskyer/credential-provider-test
Automatic merge from submit-queue (batch tested with PRs 49762, 52256). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add node e2e tests for pulling images from credential providers

**What this PR does / why we need it**:

Add node e2e tests for pulling images from credential providers.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: 

Refer https://github.com/kubernetes/kubernetes/pull/51870#issuecomment-328234010

**Special notes for your reviewer**:

/assign @yujuhong @Random-Liu 

1. We still need to add ResetDefaultDockerProviderExpiration for facilitating tests
2. Do we need a separate image for pulling private image from credential provider?
3. Any suggestion of also adding this for sandbox images? the pause image is a global config of kubelet, but we only need to set a private one for just one test case. 

**Release note**:

```release-note
NONE
```
2017-10-27 22:48:28 -07:00
yanxuean f849ebdefa e2e-node:the value of bestEffortCgroup is wrong
Signed-off-by: yanxuean <yan.xuean@zte.com.cn>
2017-10-27 17:10:53 +08:00
Kevin 4c8539cece use core client with explicit version globally 2017-10-27 15:48:32 +08:00
Kubernetes Submit Queue 931bc9edf4 Merge pull request #53730 from bsteciuk/kubeadm-windows-e2e_node
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add Windows support to the system verification check

**What this PR does / why we need it**:  This PR (in conjunction with https://github.com/kubernetes/kubernetes/pull/53553 ) adds initial support for adding a Windows worker node to a Kubernetes cluster using
 kubeadm.  It was suggested on that PR to open a separate PR for the changes in test/e2e_node for review by sig-node devs.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #364 in conjuction with #53553 

**Special notes for your reviewer**:

**Release note**:

```release-note
Add Windows support to the system verification check
```
2017-10-26 19:23:01 -07:00
Pengfei Ni 2bbba1f662 Add node e2e tests for pulling images from credential providers 2017-10-26 20:55:13 +08:00
Bob Steciuk 94db64fcb3 Add Windows support to the system verification check
Pulled SysSpecs out of types.go and created two os specific implementations with build tags

Similarly created conditionally compiled implementations of KernelValidationHelper to get Kernel version in os specific manner, as well as os specific docker endpoints (socket vs named pipes)
2017-10-25 18:55:47 -04:00
Kubernetes Submit Queue 1336cc0b05 Merge pull request #53051 from tanshanshan/test925
Automatic merge from submit-queue (batch tested with PRs 53051, 52489, 53920). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix todo

**What this PR does / why we need it**:
fix todo 
thanks
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```
2017-10-24 21:38:17 -07:00
Michael Taufen f90b46c784 Move runtime-related flags from KubeletConfiguration to KubeletFlags 2017-10-23 11:15:48 -07:00
Sen Lu 0538b65421 Add a notice for node e2e config files 2017-10-20 16:10:13 -07:00
foxyriver 7d71129ff0 delete archive 2017-10-20 11:40:52 +08:00
Di Xu f7f3577035 use multi-arch busybox for e2e 2017-10-19 10:36:31 +08:00
Dr. Stefan Schimanski cad0364e73 Update bazel 2017-10-18 17:24:04 +02:00
Dr. Stefan Schimanski 7773a30f67 pkg/api/legacyscheme: fixup imports 2017-10-18 17:23:55 +02:00
Kubernetes Submit Queue 855551dc80 Merge pull request #51250 from dixudx/bump_cni_v0.6.0
Automatic merge from submit-queue (batch tested with PRs 53106, 52193, 51250, 52449, 53861). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

bump CNI to v0.6.0

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #49480

**Special notes for your reviewer**:
/assign @luxas @bboreham @feiskyer 

**Release note**:

```release-note
bump CNI to v0.6.0
```
2017-10-16 14:47:23 -07:00
Jeff Grafton aee5f457db update BUILD files 2017-10-15 18:18:13 -07:00
Di Xu dba448c2a6 Update all binary download references to v0.6.0 2017-10-14 22:24:49 +08:00
David Ashpole 539fddb49d kubelet evictions take priority into account 2017-10-12 13:15:05 -07:00
Kubernetes Submit Queue 0f5f82fa44 Merge pull request #53416 from krzyzacy/nodeconfig-path
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add a flag to customize config relative dir

So while migrating nodee2e configs to test-infra, I found out that I'd need to have a better support for [user-data](https://github.com/kubernetes/test-infra/blob/master/jobs/e2e_node/image-config.yaml#L11). However it's not wise to use an [absolute path](https://github.com/kubernetes/test-infra/blob/master/jobs/config.json#L9309), having the config dir to be configurable will be a better solution here, and as well for later on support run local node tests from test-infra.

Currently the job references to the image configs from test-infra, but read metadata from kubernetes, which is wrong :-\


/assign @yguo0905 @Random-Liu
2017-10-12 00:57:29 -07:00
Kubernetes Submit Queue 8c8709d4de Merge pull request #53581 from Random-Liu/add-containerd-validation-node-e2e
Automatic merge from submit-queue (batch tested with PRs 53668, 53624, 52639, 53581, 51215). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add extra log and node env metadata support.

This PR:
1) Make log collection logic extensible via flags, so that we could collect more daemon logs in this PR. (e.g. `containerd.log` and `cri-containerd.log`)
2) Add extra node metadata from specified environment variable. (e.g. `PULL_REFS` in prow).

@krzyzacy I'll change the test-infra side soon. Let's discuss whether we should move/copy this code to test infra in your refactoring.

/cc @dchen1107 @yujuhong @abhi @mikebrow 

```release-note
NONE
```
2017-10-11 17:00:06 -07:00
Michael Taufen 8180536bed Mulligan: Remove deprecated and experimental fields from KubeletConfiguration
Revert "Merge pull request #51857 from kubernetes/revert-51307-kc-type-refactor"

This reverts commit 9d27d92420, reversing
changes made to 2e69d4e625.

See original: #51307

We punted this from 1.8 so it could go through an API review. The point
of this PR is that we are trying to stabilize the kubeletconfig API so
that we can move it out of alpha, and unblock features like Dynamic
Kubelet Config, Kubelet loading its initial config from a file instead
of flags, kubeadm and other install tools having a versioned API to rely
on, etc.

We shouldn't rev the version without both removing all the deprecated
junk from the KubeletConfiguration struct, and without (at least
temporarily) removing all of the fields that have "Experimental" in
their names. It wouldn't make sense to lock in to deprecated fields.
"Experimental" fields can be audited on a 1-by-1 basis after this PR,
and if found to be stable (or sufficiently alpha-gated), can be restored
to the KubeletConfiguration without the "Experimental" prefix.
2017-10-11 09:52:39 -07:00
Random-Liu 6d132e8e18 Add extra log and node env support.
Signed-off-by: Random-Liu <taotaotheripper@gmail.com>
2017-10-10 18:07:08 -07:00
Michael Taufen 131b419596 Make feature gates loadable from a map[string]bool
Command line flag API remains the same. This allows ComponentConfig
structures (e.g. KubeletConfiguration) to express the map structure
behind feature gates in a natural way when written as JSON or YAML.

For example:

KubeletConfiguration Before:
```
apiVersion: kubeletconfig/v1alpha1
kind: KubeletConfiguration
featureGates: "DynamicKubeletConfig=true,Accelerators=true"
```

KubeletConfiguration After:
```
apiVersion: kubeletconfig/v1alpha1
kind: KubeletConfiguration
featureGates:
  DynamicKubeletConfig: true
  Accelerators: true
```
2017-10-10 09:37:51 -07:00
Kubernetes Submit Queue aaf14d4619 Merge pull request #53525 from sttts/sttts-scheme-copier-romoval
Automatic merge from submit-queue (batch tested with PRs 53525, 53652). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

apimachinery: remove ObjectCopier interface(s)

The big commit is a mechanical, transitive removal of the copier interfaces in all structs and function calls.
2017-10-10 08:31:41 -07:00
Dr. Stefan Schimanski ecb65a6a71 Update generated files 2017-10-07 11:28:47 +02:00
Kubernetes Submit Queue d9bc7f0896 Merge pull request #52606 from Random-Liu/local-node-e2e-return-error
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Let local node e2e return error.

Fixes #52665

Let `make test-e2e-node` return error when it fails. Now it always returns exit code 0, whenever it fails or not.

@yguo0905 Could you help me review this?

Signed-off-by: Lantao Liu <lantaol@google.com>
2017-10-06 21:53:03 -07:00
Dr. Stefan Schimanski ed586da147 apimachinery: remove Scheme.DeepCopy 2017-10-06 14:59:17 +02:00
Sen Lu 86936539b2 Add a flag to customize config relative dir 2017-10-03 20:20:46 -07:00
Kubernetes Submit Queue 762d1e42dc Merge pull request #53336 from jiayingz/e2e-flaky
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fixes test/e2e_node/gpu_device_plugin.go test failure.

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
fixes https://github.com/kubernetes/kubernetes/issues/53354

**Special notes for your reviewer**:

**Release note**:

```release-note
```
2017-10-03 18:22:07 -07:00
Jiaying Zhang b73f4acdee Fixes test/e2e_node/gpu_device_plugin.go test failure. 2017-10-02 17:31:10 -07:00
Kubernetes Submit Queue 471d0bb716 Merge pull request #53267 from dashpole/fix_eviction
Automatic merge from submit-queue (batch tested with PRs 53234, 53252, 53267, 53276, 53107). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Prepull images after disk eviction tests

Example failure: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-node-kubelet-flaky/2855

Disk eviction tests trigger image garbage collection.  It can remove images required for subsequent tests.

This results in the error during pod creation:
`timed out waiting for the condition`

You can see in the events after the test:
`I0929 15:47:05.884] I0929 15:17:09.376591    2309 util.go:4734] Event(v1.ObjectReference{Kind:"Pod", Namespace:"e2e-tests-localstorage-eviction-test-mn5v4", Name:"container-disk-hog-pod", UID:"8dba851c-a528-11e7-a9a6-42010a800fd7", APIVersion:"v1", ResourceVersion:"116", FieldPath:"spec.containers{container-disk-hog-container}"}): type: 'Warning' reason: 'ErrImageNeverPull' Container image "busybox" is not present with pull policy of Never`

/assign @Random-Liu
2017-09-29 20:17:41 -07:00
David Ashpole 03bc96208f prepull images after disk eviction tests 2017-09-29 11:58:38 -07:00
Lantao Liu 55dc6f67d3 Let local node e2e return error.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-09-29 17:46:22 +00:00
Sen Lu afec30c720 Abort if not default nor conformance 2017-09-28 16:10:33 -07:00
Sen Lu 69df66c738 Let node test subcommand be an arg 2017-09-28 13:47:51 -07:00
tanshanshan f6ea2a61da improve code 2017-09-28 08:47:22 +08:00
Kubernetes Submit Queue 2be6982e3d Merge pull request #53110 from feiskyer/53901
Automatic merge from submit-queue (batch tested with PRs 52630, 53110, 53136, 53075). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix host network flake tests

**What this PR does / why we need it**:

Fix flaky test "Security Context when creating a pod in the host network namespace should listen on same port in the host network containers".

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #53091

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-09-27 12:58:18 -07:00
Kubernetes Submit Queue 1f45cd06b3 Merge pull request #52250 from RenaudWasTaken/e2e-device-plugin-failure
Automatic merge from submit-queue (batch tested with PRs 50988, 50509, 52660, 52663, 52250). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Added device plugin e2e kubelet failure test

Signed-off-by: Renaud Gaubert <renaud.gaubert@gmail.com>

**What this PR does / why we need it**:
This is part of issue #52859 (fixes #52859)

This PR adds a e2e_node test for the device plugin.
Specifically it implements testing of failure handling by the device plugin components in case Kubelet restart / crashes.

I might try to refactor the GPU tests in a later PR.

**Special notes for your reviewer**:
@jiayingz @vishh 

**Release note**:
```release-note
NONE
```
2017-09-27 05:32:30 -07:00
Pengfei Ni 5d75282a62 Fix host network flake tests 2017-09-27 13:44:22 +08:00
Szymon Scharmach c76ae27ffb Improve HT detection 2017-09-26 13:48:48 +02:00
Kubernetes Submit Queue 407bef47f8 Merge pull request #52373 from dashpole/eviction_cleanup
Automatic merge from submit-queue (batch tested with PRs 52960, 52373). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..

Refactor eviction tests

fixes: #52203
We have a bunch of eviction tests, which each break independently, and take a large amount of time to fix.
This refactors these tests to share the core eviction testing logic.  Each tests needs only to set kubelet flags, and specify which pods to run.
I decided to omit the memory eviction tests because they work.  Best not to disturb them.

A large portion of the code changes are the renaming of inode_eviction_test.go -> eviction_test.go

This should probably wait until after https://github.com/kubernetes/kubernetes/pull/50392

/assign @mtaufen @Random-Liu
2017-09-25 11:17:45 -07:00
Kubernetes Submit Queue 7c9e614cbb Merge pull request #52873 from ixdy/bazel-cleanup
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..

bazel: build/test almost everything

**What this PR does / why we need it**: Miscellaneous cleanups and bug fixes. The main motivating idea here was to make `bazel build //...` and `bazel test //...` mostly work. (There's a few reasons these still don't work, but we're a lot closer.)

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

/assign @BenTheElder @mikedanese @spxtr
2017-09-24 00:04:36 -07:00
David Ashpole 828c2d9630 refactor tests, and add soft eviction test 2017-09-23 20:44:55 -07:00
Kubernetes Submit Queue a85b94eca1 Merge pull request #52697 from mkumatag/nonewprivs
Automatic merge from submit-queue (batch tested with PRs 51902, 52718, 52687, 52137, 52697). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..

Multi-arch allowPrivilegeEscalation tests

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #52698 

**Special notes for your reviewer**:

**Release note**:

```NONE
```
2017-09-23 19:49:57 -07:00
Kubernetes Submit Queue 3dea17fc64 Merge pull request #50392 from dashpole/fix_inode_eviction
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..

inode eviction tests fill a constant number of inodes

Issue: #52203

inode eviction tests pass often on some OS distributions, and almost never on others.  See [these testgrid tests](https://k8s-testgrid.appspot.com/sig-node#kubelet-flaky-gce-e2e&include-filter-by-regex=Inode)
These differences are most likely because different images have fewer or greater inode capacity, and thus percentage based rules (e.g. inodesFree<50%) make the test more stressful for some OS distributions than others.
This changes the test to require that a constant number of inodes are consumed, regardless of the number of inodes in the filesystem, by setting the new threshold to:
nodefs.inodesFree<(current_inodes_free - 200k)
so that after pods consume 200k inodes, they will be evicted.  It requires querying the summary API until we successfully determine the current number of free Inodes.
2017-09-23 07:05:23 -07:00
Jiaying Zhang ba40bee5c1 Modified test/e2e_node/gpu-device-plugin.go to make sure it passes. 2017-09-22 20:21:26 +02:00
Renaud Gaubert 6993612cec Added device plugin e2e kubelet failure test
Signed-off-by: Renaud Gaubert <renaud.gaubert@gmail.com>
2017-09-22 01:24:01 +02:00
Jeff Grafton 04b0468464 add tags to e2e and integration tests 2017-09-21 15:53:23 -07:00
Yang Guo 9fbbec1afc Fix: update system spec to support Docker 17.03 2017-09-19 10:40:25 -07:00
Manjunath A Kumatagi 945d8cd87b Multi-arch allowPrivilegeEscalation tests 2017-09-19 19:17:03 +05:30
Kubernetes Submit Queue a63e3deec3 Merge pull request #51041 from balajismaniam/cpuman-e2e-tests
Automatic merge from submit-queue

Node e2e tests for the CPU Manager. 

**What this PR does / why we need it**:
- Adds node e2e tests for the CPU Manager implementation in https://github.com/kubernetes/kubernetes/pull/49186.

**Special notes for your reviewer**: 
- Previous PR in this series: #51180
- Only `test/e2e_node/cpu_manager_test.go` must be reviewed as a part of this PR (i.e., the last commit). Rest of the comments belong in #51357 and #51180.
- The tests have been on run on `n1-standard-n4` and `n1-standard-n2` instances on GCE. 

To run this node e2e test, use the following command:
```sh
make test-e2e-node TEST_ARGS='--feature-gates=DynamicKubeletConfig=true' FOCUS="CPU Manager" SKIP="" PARALLELISM=1
```

CC @ConnorDoyle @sjenning
2017-09-12 10:46:06 -07:00
Derek Carr c59715e9cb Summary tests should report rss usage now 2017-09-11 13:12:04 -04:00
Balaji Subramaniam affa182fde Added node e2e tests for the CPU Manager feature. 2017-09-11 09:29:24 -07:00
Kubernetes Submit Queue d6df4a5127 Merge pull request #52063 from mtaufen/dkcfg-e2enode
Automatic merge from submit-queue (batch tested with PRs 52047, 52063, 51528)

Improve dynamic kubelet config e2e node test and fix bugs

Rather than just changing the config once to see if dynamic kubelet
config at-least-sort-of-works, this extends the test to check that the
Kubelet reports the expected Node condition and the expected configuration
values after several possible state transitions.

Additionally, this adds a stress test that changes the configuration 100
times. It is possible for resource leaks across Kubelet restarts to
eventually prevent the Kubelet from restarting. For example, this test
revealed that cAdvisor's leaking journalctl processes (see:
https://github.com/google/cadvisor/issues/1725) could break dynamic
kubelet config. This test will help reveal these problems earlier.

This commit also makes better use of const strings and fixes a few bugs
that the new testing turned up.

Related issue: #50217

I had been sitting on this until the cAdvisor fix merged in #51751, as these tests fail without that fix.

**Release note**:

```release-note
NONE
```
2017-09-08 16:06:56 -07:00
Michael Taufen a846ba191c Improve dynamic kubelet config e2e node test and fix bugs
Rather than just changing the config once to see if dynamic kubelet
config at-least-sort-of-works, this extends the test to check that the
Kubelet reports the expected Node condition and the expected configuration
values after several possible state transitions.

Additionally, this adds a stress test that changes the configuration 100
times. It is possible for resource leaks across Kubelet restarts to
eventually prevent the Kubelet from restarting. For example, this test
revealed that cAdvisor's leaking journalctl processes (see:
https://github.com/google/cadvisor/issues/1725) could break dynamic
kubelet config. This test will help reveal these problems earlier.

This commit also makes better use of const strings and fixes a few bugs
that the new testing turned up.

Related issue: #50217
2017-09-07 15:50:17 -07:00
David Ashpole fbb29749ef inode eviction only requires filling 200k inodes 2017-09-07 13:47:33 -07:00
Kubernetes Submit Queue b6545a086c Merge pull request #51728 from derekwaynecarr/cadvisor-stats
Automatic merge from submit-queue (batch tested with PRs 51728, 49202)

Enable CRI-O stats from cAdvisor

**What this PR does / why we need it**:
cAdvisor may support multiple container runtimes (docker, rkt, cri-o, systemd, etc.)

As long as the kubelet continues to run cAdvisor, runtimes with native cAdvisor support may not want to run multiple monitoring agents to avoid performance regression in production.  Pending kubelet running a more light-weight monitoring solution, this PR allows remote runtimes to have their stats pulled from cAdvisor when cAdvisor is registered stats provider by introspection of the runtime endpoint.

See issue https://github.com/kubernetes/kubernetes/issues/51798

**Special notes for your reviewer**:
cAdvisor will be bumped to pick up https://github.com/google/cadvisor/pull/1741

At that time, CRI-O will support fetching stats from cAdvisor.

**Release note**:
```release-note
NONE
```
2017-09-06 20:00:57 -07:00
Kubernetes Submit Queue eb86cc5e87 Merge pull request #51634 from verb/sharedpid-default-off
Automatic merge from submit-queue (batch tested with PRs 51984, 51351, 51873, 51795, 51634)

Revert to using isolated PID namespaces in Docker

**What this PR does / why we need it**: Reverts to the previous docker default of using isolated PID namespaces for containers in a pod. There exist container images that expect always to be PID 1 which we want to support unmodified in 1.8.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #48937

**Special notes for your reviewer**:

**Release note**:

```release-note
Sharing a PID namespace between containers in a pod is disabled by default in 1.8. To enable for a node, use the --docker-disable-shared-pid=false kubelet flag. Note that PID namespace sharing requires docker >= 1.13.1.
```
2017-09-05 18:40:33 -07:00
David Ashpole e5a6a79fd7 update cadvisor, docker, and runc godeps 2017-09-05 12:38:57 -07:00
Kubernetes Submit Queue cdcccaab34 Merge pull request #51845 from Random-Liu/update-sysspec
Automatic merge from submit-queue (batch tested with PRs 51845, 51868, 51864)

Update sys spec to support docker 1.11-1.13 and overlay2.

Fixes https://github.com/kubernetes/kubernetes/issues/32536.

Update docker spec to:
1) Support overlay2;
2) Support docker version 1.11-1.13.

@dchen1107 @yguo0905 @luxas 
/cc @kubernetes/sig-node-pr-reviews 

```release-note
Kubernetes 1.8 supports docker version 1.11.x, 1.12.x and 1.13.x. And also supports overlay2.
```
2017-09-03 21:31:55 -07:00
Kubernetes Submit Queue 5d72d5c31d Merge pull request #50602 from dixudx/user_arm64v8_instead_aarch64
Automatic merge from submit-queue (batch tested with PRs 50602, 51561, 51703, 51748, 49142)

Use arm32v7|arm64v8 images instead of the deprecated armhf|aarch64 image organizations

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #50601

**Special notes for your reviewer**:
/assign @ixdy @jbeda @zmerlynn
**Release note**:

```release-note
Use arm32v7|arm64v8 images instead of the deprecated armhf|aarch64 image organizations
```
2017-09-03 01:12:04 -07:00
Kubernetes Submit Queue da7ee10913 Merge pull request #49457 from mkumatag/tests_multiarch
Automatic merge from submit-queue

Use the right image for the right platform in the e2e tests

**What this PR does / why we need it**:
This PR is for enabling kubernetes tests for multi architecture platform
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #38067

**Special notes for your reviewer**:
This will enable conformance tests for all the supported architectures.
**Release note**:

```release-note
Make all e2e tests lookup image to use from a centralized place. In that centralized place, add support for multiple platforms.
```

x-ref #38067
2017-09-02 15:18:10 -07:00
Shyam JVS 3bba914496 Revert "Remove deprecated and experimental fields from KubeletConfiguration" 2017-09-02 16:30:56 +02:00
Lantao Liu 73d5f53465 Update sys spec to support docker 1.11-1.13 and overlay2. 2017-09-02 00:56:25 +00:00
Kubernetes Submit Queue 9b535b06a6 Merge pull request #51307 from mtaufen/kc-type-refactor
Automatic merge from submit-queue (batch tested with PRs 50381, 51307, 49645, 50995, 51523)

Remove deprecated and experimental fields from KubeletConfiguration

As we work towards providing a stable (v1) kubeletconfig API,
we cannot afford to have deprecated or "experimental" (alpha) fields
living in the KubeletConfiguration struct. This removes all existing
experimental or deprecated fields, and places them in KubeletFlags
instead.

I'm going to send another PR after this one that organizes the remaining
fields into substructures for readability. Then, we should try to move
to v1 ASAP (maybe not v1 in 1.8, given how close we are, but definitely in 1.9).

It makes far more sense to focus on a clean API in kubeletconfig v2,
than to try and further clean up the existing "API" that everyone
already depends on.

fixes: #51657

**Release note**:
```release-note
NONE
```
2017-09-01 16:33:59 -07:00
Lee Verberne 765374ce03 Explicitly enable docker shared-pid for e2e_node
This also renames isSharedPIDNamespaceEnabled() to
isSharedPIDNamespaceSupported() to be more accurate.
2017-09-01 23:50:11 +02:00
Kubernetes Submit Queue aa50c0f54c Merge pull request #51490 from NickrenREN/eviction-podLocalEphemeralStorageUsage
Automatic merge from submit-queue (batch tested with PRs 51628, 51637, 51490, 51279, 51302)

Fix pod local ephemeral storage usage calculation

We use podDiskUsage to calculate pod local ephemeral storage which is not correct, because podDiskUsage also contains HostPath volume  which is considered as persistent storage
This pr fixes it
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #51489

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```

/assign @jingxu97  @vishh 
cc @ddysher
2017-09-01 00:11:17 -07:00
Kubernetes Submit Queue 17dffc1ef5 Merge pull request #51448 from kastenhq/pvc_ref_volstats
Automatic merge from submit-queue (batch tested with PRs 51513, 51515, 50570, 51482, 51448)

Add PVCRef to VolumeStats

**What this PR does / why we need it**:
For pod volumes that reference a PVC, add a PVCRef to the corresponding
volume stat. This allows metrics to be indexed/queried by PVC name
which is more user-friendly than Pod reference

**Which issue this PR fixes** : [#363](https://github.com/kubernetes/features/issues/363)

**Special notes for your reviewer**:

**Release note**:
```
`VolumeStats` reported by the kubelet stats summary API 
(http://<node>:10255/stats/summary) now include a PVCRef
field describing the PVC referenced by the volume (if any). 
```
2017-08-31 22:09:20 -07:00
Manjunath A Kumatagi ee4d54c70c Port e2e tests for multi architecture 2017-09-01 05:40:52 +05:30
Manjunath A Kumatagi 22c3a590d1 Fix bazel 2017-09-01 05:39:00 +05:30
Derek Carr 566f411b08 Support remote runtimes with native cAdvisor support 2017-08-31 16:41:53 -04:00
Michael Taufen c18626de4a Remove deprecated and experimental fields from KubeletConfiguration
As we work towards providing a stable (v1) kubeletconfig API,
we cannot afford to have deprecated or "experimental" (alpha) fields
living in the KubeletConfiguration struct. This removes all existing
experimental or deprecated fields, and places them in KubeletFlags
instead.

I'm going to send another PR after this one that organizes the remaining
fields into substructures for readability. Then, we should try to move
to v1 ASAP.

It makes far more sense to focus on a clean API in kubeletconfig v2,
than to try and further clean up the existing "API" that everyone
already depends on.
2017-08-30 11:54:21 -07:00
Jing Xu 4d6da1fd9a Change SizeLimit to a pointer
This PR fixes issue #50121
2017-08-30 11:50:35 -07:00
Kubernetes Submit Queue 1fc7cd3d1d Merge pull request #51545 from sttts/sttts-deepcopy-e2e
Automatic merge from submit-queue (batch tested with PRs 47054, 50398, 51541, 51535, 51545)

e2e/integration: simplify deepcopy calls
2017-08-30 01:51:37 -07:00
Vaibhav Kamra 1ac56d8cbb Add PVCRef to VolumeStats
For pod volumes that reference a PVC, add a PVCRef to the corresponding
volume stat. This allows metrics to be indexed/queried by PVC name
which is more user-friendly than Pod reference
2017-08-29 23:12:20 -07:00
NickrenREN 4ca27417d9 Add pod local ephemeral storage usage e2e test cases 2017-08-30 13:54:26 +08:00
Dr. Stefan Schimanski 637fe0844c e2e/integration: simplify deepcopy calls 2017-08-29 20:11:50 +02:00
Yang Guo 039178b27f Use the pre-built docker binaries on Ubuntu for benchmark tests 2017-08-28 14:06:23 -07:00
Kubernetes Submit Queue 6368c1fc82 Merge pull request #51348 from rmmh/coreos-no-password
Automatic merge from submit-queue

Make coreos test images sshd not allow password login.

This will prevent security scanners from triggering.

Configuration is verbatim from:
https://coreos.com/os/docs/latest/customizing-sshd.html

```release-note
NONE
```
2017-08-26 04:19:11 -07:00
NickrenREN 27901ad5df Change eviction policy to manage one single local storage resource 2017-08-26 05:14:49 +08:00
Ryan Hitchman a7e64aaa66 Make coreos test images sshd not allow password login.
Configuration is based on:
https://coreos.com/os/docs/latest/customizing-sshd.html

The specific SSHD config is:

    # Use most defaults for sshd configuration.
    UsePrivilegeSeparation sandbox
    Subsystem sftp internal-sftp
    ClientAliveInterval 180
    UseDNS no
    UsePAM yes
    PrintLastLog no # handled by PAM
    PrintMotd no # handled by PAM
    AuthenticationMethods publickey

This will prevent security scanners from triggering.
2017-08-25 11:49:34 -07:00
Kubernetes Submit Queue 4a94363c7e Merge pull request #51158 from yguo0905/overlay2
Automatic merge from submit-queue (batch tested with PRs 51224, 51191, 51158, 50669, 51222)

Enable overlay2 on cos-m60 in node e2e tests

Ref: https://github.com/kubernetes/kubernetes/issues/42926

- Restart docker with `-s overlay2` in cloud-init before running all node e2e tests. I have to copy the systemd unit file to `/etc/systemd/system` because the `/usr/lib/systemd/system/` is read only.
- Updated node e2e tests to use the new cos-m60 image.
- The name of the cloud init file (`cos-init-live-restore.yaml`) does not indicate overlay2 will be enabled, but I can't just change the name in this PR, since it's referenced in test-infra.

**Release note**:

```
None
```

/assign @Random-Liu
2017-08-24 22:59:33 -07:00
Kubernetes Submit Queue 55a20bb901 Merge pull request #51206 from yguo0905/update-cos
Automatic merge from submit-queue (batch tested with PRs 47115, 51196, 51204, 51208, 51206)

Update cos-m61 image in benchmark tests

Ref: https://github.com/kubernetes/kubernetes/issues/51205

**Release note**:
```
None
```
2017-08-24 07:20:16 -07:00
Kubernetes Submit Queue c041567b5a Merge pull request #46597 from dixudx/implement_proposal_34058
Automatic merge from submit-queue (batch tested with PRs 51113, 46597, 50397, 51052, 51166)

implement proposal 34058: hostPath volume type

**What this PR does / why we need it**:
implement proposal #34058

**Which issue this PR fixes** : fixes #46549

**Special notes for your reviewer**:
cc @thockin @luxas @euank PTAL
2017-08-23 23:16:27 -07:00
Yang Guo a1c5c14eff Update cos-m61 image in benchmark tests 2017-08-23 09:30:20 -07:00
Kubernetes Submit Queue 178a5ff314 Merge pull request #50665 from xiangpengzhao/hardcode-to-const
Automatic merge from submit-queue (batch tested with PRs 50257, 50247, 50665, 50554, 51077)

Replace hard-code "cpu" and "memory" to consts

**What this PR does / why we need it**:
There are many places using hard coded "cpu" and "memory" as resource name. This PR replace them to consts.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:
/kind cleanup

**Release note**:

```release-note
NONE
```
2017-08-23 02:35:09 -07:00
Di Xu 6f74af94ef update e2e tests and yaml files 2017-08-23 14:05:21 +08:00
Yang Guo 755ce10e9b Enable overlay2 on cos-m60 in node e2e tests 2017-08-22 17:08:52 -07:00
Kubernetes Submit Queue c6980e7247 Merge pull request #51033 from mtaufen/revert-51008-revert-50789-fix-scheme
Automatic merge from submit-queue (batch tested with PRs 50967, 50505, 50706, 51033, 51028)

Revert "Merge pull request #51008 from kubernetes/revert-50789-fix-scheme"

I'm spinning up a cluster right now to test this fix, but I'm pretty sure this was the problem.
There doesn't seem to be a way to confirm from logs, because AFAICT the logs from the hollow kubelet containers are not collected as part of the kubemark test.

**What this PR does / why we need it**:

This reverts commit f4afdecef8, reversing
changes made to e633a1604f.

This also fixes a bug where Kubemark was still using the core api scheme
to manipulate the Kubelet's types, which was the cause of the initial
revert.

**Which issue this PR fixes**: fixes #51007

**Release note**:

```release-note
NONE
```

/cc @shyamjvs @wojtek-t
2017-08-22 10:48:21 -07:00
Kubernetes Submit Queue 0867802bbc Merge pull request #50831 from Random-Liu/instance-metadata-from-flag
Automatic merge from submit-queue (batch tested with PRs 50693, 50831, 47506, 49119, 50871)

Add instance metadata from flag even when using image config.

Also add instance metadata from flag even when we are using image config.

* Sometimes we need to dynamically generate instance metadata, it's troublesome to put them into image config.
* Sometimes we want to apply instance metadata to all images, it's duplicated to add them to each image in the image config.

/assign @yguo0905 Could you help me review this?
2017-08-21 14:29:57 -07:00
Michael Taufen a90d81620b Revert "Merge pull request #51008 from kubernetes/revert-50789-fix-scheme"
This reverts commit f4afdecef8, reversing
changes made to e633a1604f.

This also fixes a bug where Kubemark was still using the core api scheme
to manipulate the Kubelet's types, which was the cause of the initial
revert.
2017-08-21 11:28:05 -07:00
Shyam JVS 5591914d62 Revert "Don't register the kubeletconfig group with the default Scheme" 2017-08-21 11:15:27 +02:00
Di Xu d4aa1611bd use more-specific arm64v8 instead of deprecated aarch64 organization 2017-08-21 10:18:19 +08:00
Di Xu 25a786f74d use more-specific arm32v7 instead of deprecated armhf organization 2017-08-21 10:17:43 +08:00
Michael Taufen 0af9f756cd Don't register the kubeletconfig group with the default Scheme 2017-08-18 13:51:39 -07:00
Kubernetes Submit Queue a4f6ae4402 Merge pull request #50277 from yguo0905/live-restore-test
Automatic merge from submit-queue

Add node e2e test for Docker's live-restore

Ref: https://github.com/kubernetes/kubernetes/issues/42926

This PR adds a test for docker live-restore. If this is fine, we can close the unfinished PR https://github.com/kubernetes/kubernetes/pull/40364.

**Release note**:
```
None
```
2017-08-17 21:44:09 -07:00
Yang Guo 9f1f83020b Add node e2e test for Docker's live-restore 2017-08-17 16:58:21 -07:00
Random-Liu 2c129e4d6a Add instance metadata from flag even when using image config. 2017-08-17 16:42:25 -07:00
Nick Sardo a0e95f9475 Fix e2e_node for changes to /api/compute/v0.beta package 2017-08-17 10:29:58 -07:00
xiangpengzhao 1c4dbcf5ca Replace hard-code "cpu" and "memory" to consts 2017-08-16 16:37:50 +08:00
Michael Taufen 24bab4c20f move KubeletConfiguration out of componentconfig API group 2017-08-15 08:12:42 -07:00
Yang Guo 1fb12b84dd Allow passing image description from e2e node test config 2017-08-14 17:11:05 -07:00
Kubernetes Submit Queue cf80b91a9e Merge pull request #50479 from yguo0905/node-perf-m60
Automatic merge from submit-queue (batch tested with PRs 49847, 49743, 49853, 50225, 50479)

Add node benchmark tests for cos-m60 with docker 1.12.6

Ref: https://github.com/kubernetes/kubernetes/issues/42926

This PR adds a benchmark tests against cos-m60 with docker 1.12.6 on http://node-perf-dash.k8s.io. This test is useful for docker validation -- we can compare the performance of different dockers on the same OS.

cos-m60 comes with docker 1.13.1 by default, so we need to use cloud-init to downgrade the version to 1.12.6.

**Release note**:
```
None
```

/assign @dchen1107
2017-08-12 02:36:01 -07:00
Jeff Grafton a7f49c906d Use buildozer to delete licenses() rules except under third_party/ 2017-08-11 09:32:39 -07:00
Jeff Grafton 33276f06be Use buildozer to remove deprecated automanaged tags 2017-08-11 09:31:50 -07:00
Jeff Grafton cf55f9ed45 Autogenerate BUILD files 2017-08-11 09:30:23 -07:00
Yang Guo 8ca49e0989 Add node benchmark tests for cos-m60 with docker 1.12.6 2017-08-10 16:48:10 -07:00
Kubernetes Submit Queue cb49706c00 Merge pull request #48857 from feiskyer/privileged
Automatic merge from submit-queue (batch tested with PRs 49725, 50367, 50391, 48857, 50181)

Add e2e test for privileged containers

**What this PR does / why we need it**:

This PR adds node e2e test for privileged containers.

**Which issue this PR fixes**

Part of #44118.

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

/assign @Random-Liu
2017-08-10 01:47:19 -07:00
Kubernetes Submit Queue 458cc04330 Merge pull request #46254 from mtaufen/dkcfg
Automatic merge from submit-queue (batch tested with PRs 50016, 49583, 49930, 46254, 50337)

Alpha Dynamic Kubelet Configuration

Feature: https://github.com/kubernetes/features/issues/281

This proposal contains the alpha implementation of the Dynamic Kubelet Configuration feature proposed in ~#29459~ [community/contributors/design-proposals/dynamic-kubelet-configuration.md](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/dynamic-kubelet-configuration.md). 

Please note:
- ~The proposal doc is not yet up to date with this implementation, there are some subtle differences and some more significant ones. I will update the proposal doc to match by tomorrow afternoon.~
- ~This obviously needs more tests. I plan to write several O(soon). Since it's alpha and feature-gated, I'm decoupling this review from the review of the tests.~ I've beefed up the unit tests, though there is still plenty of testing to be done.
- ~I'm temporarily holding off on updating the generated docs, api specs, etc, for the sake of my reviewers 😄~ these files now live in a separate commit; the first commit is the one to review.

/cc @dchen1107 @vishh @bgrant0607 @thockin @derekwaynecarr 

```release-note
Adds (alpha feature) the ability to dynamically configure Kubelets by enabling the DynamicKubeletConfig feature gate, posting a ConfigMap to the API server, and setting the spec.configSource field on Node objects. See the proposal at https://github.com/kubernetes/community/blob/master/contributors/design-proposals/dynamic-kubelet-configuration.md for details.
```
2017-08-09 14:14:32 -07:00
Kubernetes Submit Queue 8c4a269b83 Merge pull request #49771 from feiskyer/wait-for-failure
Automatic merge from submit-queue

Add waitForFailure for e2e test framework

**What this PR does / why we need it**:

Add waitForFailure for e2e test framework, this could reduce the reliance on logs.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: 

Part of #44118. Refer https://github.com/kubernetes/kubernetes/pull/48858#discussion_r128331726

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-08-08 20:56:51 -07:00
Michael Taufen 443d58e40a Dynamic Kubelet Configuration
Alpha implementation of the Dynamic Kubelet Configuration feature.
See the proposal doc in #29459.
2017-08-08 12:21:37 -07:00
Kubernetes Submit Queue 02d04de81e Merge pull request #49914 from yguo0905/shared-pid-ns
Automatic merge from submit-queue (batch tested with PRs 50087, 39587, 50042, 50241, 49914)

Add node e2e test for Docker's shared PID namespace

Ref: https://github.com/kubernetes/kubernetes/issues/42926

This PR adds a simple test for the shared PID namespace that's enabled when Docker is 1.13.1+.

/sig node
/area node-e2e
/assign @yujuhong 

**Release note**:
```
None
```
2017-08-07 10:59:04 -07:00
Mik Vyatskov e79a228a78 Move the sig-instrumentation test to a dedicated folder 2017-08-07 10:33:03 +02:00
Dr. Stefan Schimanski 1910b5a1dd Fix code implicitly casting clientsets to getters 2017-08-06 15:30:13 +02:00
Kubernetes Submit Queue 7c9ba69617 Merge pull request #48487 from dixudx/validate_cadvisor_rootpath
Automatic merge from submit-queue (batch tested with PRs 48487, 49009, 49862, 49843, 49700)

validate cadvisor rootpath

**What this PR does / why we need it**:

When working on issue #48452, I found [KubeletConfiguration.RootDirectory](https://github.com/kubernetes/kubernetes/blob/master/cmd/kubelet/app/server.go#L525) was never been validated. The default value is set to ["/var/lib/kubelet"](https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/componentconfig/v1alpha1/defaults.go#L342).

If this directory does not exist in the file system, the [cadvisor.manager](https://github.com/kubernetes/kubernetes/blob/master/vendor/github.com/google/cadvisor/manager/manager.go#L679) will fail to gather the information for metrics.

> error trying to get filesystem Device for dir /var/lib/kubelet: err: stat failed on /var/lib/kubelet with error: no such file or directory

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

/cc @feiskyer @k82cn

**Release note**:

```release-note
validate cadvisor rootpath
```
2017-08-04 23:40:00 -07:00
Yang Guo 026a082a7f Add node e2e test for Docker's shared PID namespace 2017-08-04 15:01:55 -07:00
Kubernetes Submit Queue 7c0e9852b4 Merge pull request #49916 from yguo0905/coreos
Automatic merge from submit-queue (batch tested with PRs 49916, 50050)

Update images used in the node e2e benchmark tests

Ref: https://github.com/kubernetes/kubernetes/issues/42926

- Update the cosbeta image since the new version contains a 'du' command fix that affects Docker performance.
- Add the coreos and ubuntu image that run Docker 1.12.6 so that we will have more data to compare.

**Release note**:
```
None
```
2017-08-02 23:29:49 -07:00
Pengfei Ni 3027d9bac3 Add e2e test for privileged containers 2017-08-01 15:50:22 +08:00
Yang Guo 7c31be8ec4 Update images used in the node e2e benchmark tests 2017-07-31 18:11:02 -07:00
Kubernetes Submit Queue 72c6251508 Merge pull request #47019 from jessfraz/allowPrivilegeEscalation
Automatic merge from submit-queue (batch tested with PRs 49651, 49707, 49662, 47019, 49747)

Add support for `no_new_privs` via AllowPrivilegeEscalation

**What this PR does / why we need it**:
Implements kubernetes/community#639
Fixes #38417

Adds `AllowPrivilegeEscalation` and `DefaultAllowPrivilegeEscalation` to `PodSecurityPolicy`.
Adds `AllowPrivilegeEscalation` to container `SecurityContext`.

Adds the proposed behavior to `kuberuntime`, `dockershim`, and `rkt`. Adds a bunch of unit tests to ensure the desired default behavior and that when `DefaultAllowPrivilegeEscalation` is explicitly set.

Tests pass locally with docker and rkt runtimes. There are also a few integration tests with a `setuid` binary for sanity.

**Release note**:

```release-note
Adds AllowPrivilegeEscalation to control whether a process can gain more privileges than it's parent process
```
2017-07-31 16:56:58 -07:00
Kubernetes Submit Queue 5f6d16527d Merge pull request #49443 from yguo0905/gke-tests
Automatic merge from submit-queue (batch tested with PRs 45813, 49594, 49443, 49167, 47539)

Add node e2e tests for GKE environment

Ref: https://github.com/kubernetes/kubernetes/issues/46891

This PR adds node e2e tests for validating images used on GKE.

- We pass the `SYSTEM_SPEC_NAME` to the node e2e test process via the flag `--system-spec-name` so that we can skip the environment specific tests using `RunIfSystemSpecNameIs()`.
- Also added `SkipIfContainerRuntimeIs()` as the opposite of `RunIfContainerRuntimeIs()`.

**Release note**:
```
None
```
2017-07-28 07:22:36 -07:00
Pengfei Ni 983ecaa73d Add waitForFailure for e2e test framework 2017-07-28 17:15:43 +08:00
Kubernetes Submit Queue a5e1eac1f8 Merge pull request #48858 from feiskyer/readonlyrootfs-test
Automatic merge from submit-queue (batch tested with PRs 46913, 48910, 48858, 47160)

Add e2e test for readOnlyRootFilesystem containers

**What this PR does / why we need it**:

This PR adds node e2e test for readOnlyRootFilesystem containers.

**Which issue this PR fixes**

Part of #44118.

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-07-25 23:00:33 -07:00
Di Xu 6c7245d464 validate cadvisor rootpath 2017-07-26 10:05:29 +08:00
Kubernetes Submit Queue 2189314895 Merge pull request #40050 from mtaufen/standalone-mode
Automatic merge from submit-queue (batch tested with PRs 48976, 49474, 40050, 49426, 49430)

Use presence of kubeconfig file to toggle standalone mode

Fixes #40049 

```release-note
The deprecated --api-servers flag has been removed. Use --kubeconfig to provide API server connection information instead. The --require-kubeconfig flag is now deprecated. The default kubeconfig path is also deprecated. Both --require-kubeconfig and the default kubeconfig path will be removed in Kubernetes v1.10.0.
```

/cc @kubernetes/sig-cluster-lifecycle-misc @kubernetes/sig-node-misc
2017-07-25 12:14:43 -07:00
Kubernetes Submit Queue 68182cea8b Merge pull request #49396 from yguo0905/docker-validation-3
Automatic merge from submit-queue (batch tested with PRs 48224, 45431, 45946, 48775, 49396)

Update cos-dev image in benchmark tests to cos-dev-61-9759-0-0

Ref: https://github.com/kubernetes/kubernetes/issues/42926

`cos-dev-61-9759-0-0` contains a fix in Linux utility `du` that would affect the measurement of docker performance in kubelet. I'd like to update the benchmark to use the new image.

**Release note**:
```
None
```

/assign @tallclair 
/cc @kewu1992 @abgworrall
2017-07-25 11:06:55 -07:00
Kubernetes Submit Queue e623fed778 Merge pull request #48636 from jingxu97/July/allocatable
Automatic merge from submit-queue (batch tested with PRs 48636, 49088, 49251, 49417, 49494)

Fix issues for local storage allocatable feature

This PR fixes the following issues:
1. Use ResourceStorageScratch instead of ResourceStorage API to represent
local storage capacity
2. In eviction manager, use container manager instead of node provider
(kubelet) to retrieve the node capacity and reserved resources. Node
provider (kubelet) has a feature gate so that storagescratch information
may not be exposed if feature gate is not set. On the other hand,
container manager has all the capacity and allocatable resource
information.

This PR fixes issue #47809
2017-07-24 19:30:33 -07:00
Kubernetes Submit Queue fe8f6a1599 Merge pull request #49309 from yujuhong/add-node-e2e-owner
Automatic merge from submit-queue

Add yujuhong to test/e2e_node/OWNERS
2017-07-24 11:06:16 -07:00
Michael Taufen 38aee0464d Providing kubeconfig file is now the switch for standalone mode
Replaces use of --api-servers with --kubeconfig in Kubelet args across
the turnup scripts. In many cases this involves generating a kubeconfig
file for the Kubelet and placing it in the correct location on the node.
2017-07-24 11:03:00 -07:00
Jess Frazelle ce70619a47
allowPrivilegeEscalation: add integration test with setuid binary
Signed-off-by: Jess Frazelle <acidburn@google.com>
2017-07-24 12:52:45 -04:00
Yang Guo 78f04e2abf Add node e2e tests for GKE environment 2017-07-23 20:59:11 -07:00
Jason Brooks fa03b1eca7 add kernel config locations for fedora and atomic
* Fedora stores its kernel configs in /usr/lib/modules/$(uname -r)
* Fedora/CentOS/RHEL atomic hosts use /usr/lib/ostree-boot, though this location is deprecated
* The lack of these locations in the validator is causing kubeadm to hang on "failed to parse kernel config" in its preflight checking on fedora and atomic host
2017-07-21 13:16:13 -07:00
Yang Guo 324b091002 Update cos-dev image in benchmark tests to cos-dev-61-9759-0-0 2017-07-21 10:30:48 -07:00
Kubernetes Submit Queue 947700d146 Merge pull request #49207 from dixudx/remove_redundant_param_e2e_remote
Automatic merge from submit-queue

remove redundant param in e2e_node/remote

**What this PR does / why we need it**:

* remove redundant param in e2e_node/remote/remote.go
* fix a small typo

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
None
```
2017-07-20 20:16:38 -07:00
Kubernetes Submit Queue 6c5b24b564 Merge pull request #49064 from yguo0905/ubuntu-gke
Automatic merge from submit-queue (batch tested with PRs 49316, 46117, 49064, 48073, 49323)

Test Ubuntu image using GKE image spec on master

Ref: https://github.com/kubernetes/kubernetes/issues/46891

This PR changes the files referenced in test-infra for running Ubuntu image tests against GKE system spec on master.

The two properties files are shared by the tests against all k8s branches but the `SYSTEM_SPEC_NAME` is only available on master. This should be fine because the tests in the non master branches will just ignore the unknown env variable.

**Release note**:
```
None
```

/assign @yujuhong
2017-07-20 17:02:50 -07:00
Yu-Ju Hong d51c698181 Add yujuhong to test/e2e_node/OWNERS 2017-07-20 09:48:54 -07:00
Kubernetes Submit Queue 3e0dde91b6 Merge pull request #49062 from yguo0905/docker-validation-2
Automatic merge from submit-queue (batch tested with PRs 48377, 48940, 49144, 49062, 49148)

Add cos-dev-61-9733-0-0 to the benchmark tests

Ref: https://github.com/kubernetes/kubernetes/issues/42926

m60 has docker 1.13.1 while m61 has 17.03. This PR adds m61 to the benchmark tests so that we will have more data to compare.

PS: We will support fetching the latest image in an image family in the node e2e tests in the future.

**Release note**:

```
None
```

/assign @yujuhong 
/cc @kewu1992 @abgworrall
2017-07-19 19:10:16 -07:00
Di Xu 769929ba49 remove redundant param in e2e_node/remote 2017-07-19 22:25:31 +08:00
jeff vance a113d8ac41 volume i/o tests for storage plugins 2017-07-18 17:59:15 -07:00
Yang Guo c979d7f167 Test Ubuntu image using GKE image spec 2017-07-17 16:18:17 -07:00
Yang Guo 248930bc7d Add cos-beta-60-9592-52-0 to the benchmark tests 2017-07-17 15:53:15 -07:00
Jacob Simpson a765b8cfca Migrate api.Scheme to scheme.Scheme 2017-07-17 15:05:38 -07:00
Jacob Simpson 29c1b81d4c Scripted migration from clientset_generated to client-go. 2017-07-17 15:05:37 -07:00
Kubernetes Submit Queue 226b39c6b5 Merge pull request #48896 from yguo0905/docker-validation-m60
Automatic merge from submit-queue (batch tested with PRs 48890, 46893, 48872, 48896)

Add cos-beta-60-9592-52-0 to the benchmark tests

This PR depends on https://github.com/kubernetes/kubernetes/pull/48824.

This PR adds new resource usage tests for cos-beta-60-9592-52-0 (docker 1.13.1).

Ref: #42926

**Release note**:
```
None
```
/sig node
/area node-e2e
/assign @dchen1107
/cc @abgworrall
2017-07-14 16:49:55 -07:00
Kubernetes Submit Queue cab07f3af0 Merge pull request #46893 from yguo0905/image-spec
Automatic merge from submit-queue (batch tested with PRs 48890, 46893, 48872, 48896)

Support customized system spec in the node conformance test and create the GKE system spec

ref: https://github.com/kubernetes/kubernetes/issues/46891

- System specs are located in `test/e2e_node/system/specs`. Created one for validating GKE images in `test/e2e_node/system/specs/gke.yaml`.
- `--image-spec-name` can be used to specify a system spec in node e2e and conformance tests. This option maps to `SYSTEM_SPEC_NAME` in a test properties file, which is the user facing configuration. So, users can specify `SYSTEM_SPEC_NAME=gke` to run the image validation using the GKE system spec.
- If `SYSTEM_SPEC_NAME` is unspecified, the default spec (`system.DefaultSysSpec`) will be used.
- We can also use `make test-e2e-node SYSTEM_SPEC_NAME=gke` to run tests using GKE image spec.

**Release note**:
`None`
2017-07-14 16:49:52 -07:00
Kubernetes Submit Queue 23e60ac9ad Merge pull request #48308 from yguo0905/docker-api
Automatic merge from submit-queue

Update Docker API in Kubelet

Ref: https://github.com/kubernetes/kubernetes/issues/34308

The Kubelet currently uses deprecated docker API (https://godoc.org/github.com/docker/engine-api). This PR changes it to use the new one (https://godoc.org/github.com/moby/moby/client).

This PR updated the docker package from 1.11 to 1.13.1-rc2.

**Release note**:
```
None
```

/assign @Random-Liu 
/cc @yujuhong
2017-07-14 15:30:59 -07:00
Yang Guo 22c9e23202 Supports customized system spec in the node conformance test and creates the GKE system spec 2017-07-14 09:39:19 -07:00
Kubernetes Submit Queue a14abaabab Merge pull request #48824 from yguo0905/docker-validation
Automatic merge from submit-queue (batch tested with PRs 48082, 48815, 48901, 48824)

Add test image name to the OS image field of the perf metrics

I'd like to add the resource usage benchmarks for COS m60 (docker 1.13.1) but don't want to remove the existing m59 (docker 1.11.2) [ones](https://github.com/kubernetes/kubernetes/blob/master/test/e2e_node/jenkins/benchmark/benchmark-config.yaml#L51-L71), in order to compare the results between the two docker versions.

The `image` reported in the metrics is from `Node.Status.NodeInfo.OSImage`, which is always "Container-Optimized OS from Google" (from `/etc/os-releases`) for COS. So there's no way to differentiate two milestones in the metrics.

This PR attaches the [image name](https://github.com/kubernetes/kubernetes/blob/master/test/e2e_node/jenkins/benchmark/benchmark-config.yaml#L52) to the `image` field of the metrics. So it will become "Container-Optimized OS from Google (cos-stable-59-9460-64-0)".

See the results of the test run:

[performance-memory-containervm-resource1-resource_0.json](https://storage.googleapis.com/ygg-gke-dev-bucket/e2e-node-test/ci-kubernetes-node-kubelet-benchmark/13/artifacts/performance-memory-containervm-resource1-resource_0.json)
[performance-memory-coreos-resource1-resource_0.json](https://storage.googleapis.com/ygg-gke-dev-bucket/e2e-node-test/ci-kubernetes-node-kubelet-benchmark/13/artifacts/performance-memory-coreos-resource1-resource_0.json)
[performance-memory-gci-resource1-resource_0.json](https://storage.googleapis.com/ygg-gke-dev-bucket/e2e-node-test/ci-kubernetes-node-kubelet-benchmark/13/artifacts/performance-memory-gci-resource1-resource_0.json)

**Release note**:
```
None
```

Ref: https://github.com/kubernetes/kubernetes/issues/42926

/sig node
/area node-e2e
/assign @dchen1107
2017-07-13 22:44:00 -07:00
Kubernetes Submit Queue 8ad1be7833 Merge pull request #44475 from freehan/checkpoint-test
Automatic merge from submit-queue

add dockershim checkpoint node e2e test

Add a bunch of disruptive cases to test kubelet/dockershim's checkpoint work flow.

Some steps are quite hacky. Not sure if there is better ways to do things.
2017-07-13 18:50:10 -07:00
Yang Guo bf2ced837c Updates Docker Engine API 2017-07-13 12:55:07 -07:00
Yang Guo 22253a6e6a Add cos-beta-60-9592-52-0 to benchmark tests 2017-07-13 12:06:59 -07:00
Jing Xu bb1920edcc Fix issues for local storage allocatable feature
This PR fixes the following issues:
1. Use ResourceStorageScratch instead of ResourceStorage API to represent
local storage capacity
2. In eviction manager, use container manager instead of node provider
(kubelet) to retrieve the node capacity and reserved resources. Node
provider (kubelet) has a feature gate so that storagescratch information
may not be exposed if feature gate is not set. On the other hand,
container manager has all the capacity and allocatable resource
information.
2017-07-13 12:06:19 -07:00
Maru Newby 6ba0e92bf4 fed: Enable the namespace controller in integration tests 2017-07-13 09:50:07 -07:00
Pengfei Ni 721047fe49 Add e2e test for readOnlyRootFilesystem containers 2017-07-13 17:21:29 +08:00
Yang Guo b17c6a1769 Add test image name to the OS image field of the perf metrics 2017-07-12 14:51:45 -07:00
Kubernetes Submit Queue 0e461035cb Merge pull request #48734 from tallclair/namechange
Automatic merge from submit-queue (batch tested with PRs 48698, 48712, 48516, 48734, 48735)

Name change: s/timstclair/tallclair/

I changed my name, and I'm migrating my user name to be consistent.
2017-07-12 04:56:32 -07:00
Kubernetes Submit Queue de30789bf5 Merge pull request #48598 from gmarek/metrics
Automatic merge from submit-queue (batch tested with PRs 46865, 48661, 48598, 48658, 48614)

Move metrics_grabbert to test/e2e

cc @aleksandra-malinowska
2017-07-12 03:02:19 -07:00
Tim Allclair a2f2e1d491 Name change: s/timstclair/tallclair/ 2017-07-10 14:05:46 -07:00
Cao Shufeng 0c577c47d5 Use glog.*f when a format string is passed
ref:
https://godoc.org/github.com/golang/glog

I use the following commands to search all the invalid usage:
$ grep "glog.Warning(" * -r | grep %
$ grep "glog.Info(" * -r | grep %
$ grep "glog.Error(" * -r | grep %
$ grep ").Info(" * -r | grep % | grep "glog.V("
2017-07-10 19:04:03 +08:00
gmarek 55880e6b4b Move metrics_grabbert to test/e2e 2017-07-07 13:13:44 +02:00
Minhan Xia 6da0c11063 add dockershim checkpoint node e2e test 2017-06-29 13:26:09 -07:00
Pengfei Ni 00eeb7f53a Add node e2e tests for runAsUser 2017-06-29 09:17:14 +08:00
Kubernetes Submit Queue 165c94aa7b Merge pull request #47549 from yguo0905/change-tested-images
Automatic merge from submit-queue

Changes node e2e tests to use the new Ubuntu image

ref: https://github.com/kubernetes/kubernetes/issues/46891

`ubuntu-docker10` and `ubuntu-docker12` images are deprecated in favor of the new one.

**Release note**:
```
None
```
/sig node
/area node-e2e
/assign @dchen1107
2017-06-27 23:30:24 -07:00
Kubernetes Submit Queue 98ee52ed78 Merge pull request #48001 from yguo0905/report-prefix
Automatic merge from submit-queue (batch tested with PRs 47675, 48001)

Encodes ReportPrefix into the generated metrics file names

Ref: https://github.com/kubernetes/kubernetes/issues/44003

Adds the test prefix to be part of the name. Otherwise the same test case running on different images will override each other. Nothing needs to be changed at the node-perf-dash side.

See test run at https://console.cloud.google.com/storage/browser/ygg-gke-dev-bucket/e2e-node-test/ci-kubernetes-node-kubelet-benchmark/10.


**Release note**:
```
None
```

/sig node
/area node-e2e
/assign @Random-Liu
2017-06-27 16:11:07 -07:00
Kubernetes Submit Queue 0dad2d0803 Merge pull request #47983 from yguo0905/memcg
Automatic merge from submit-queue (batch tested with PRs 48092, 47894, 47983)

Enables memcg notification in cluster/node e2e tests

Ref: https://github.com/kubernetes/kubernetes/issues/42676

This PR sets Kubelet flag `--experimental-kernel-memcg-notification=true` when running cluster/node e2e tests on COS and Ubuntu images.

Tested:
```
e2e-node-cos:
I0623 00:09:06.641776    1080 server.go:147] Starting server "kubelet" with command "/usr/bin/systemd-run --unit=kubelet-777178888.service --slice=runtime.slice --remain-after-exit /tmp/node-e2e-20170622T170739/kubelet --kubelet-cgroups=/kubelet.slice --cgroup-root=/ --api-servers http://localhost:8080 --address 0.0.0.0 --port 10250 --read-only-port 10255 --volume-stats-agg-period 10s --allow-privileged true --serialize-image-pulls false --pod-manifest-path /tmp/node-e2e-20170622T170739/pod-manifest571288056 --file-check-frequency 10s --pod-cidr 10.100.0.0/24 --eviction-pressure-transition-period 30s --feature-gates  --eviction-hard memory.available<250Mi,nodefs.available<10%%,nodefs.inodesFree<5%% --eviction-minimum-reclaim nodefs.available=5%%,nodefs.inodesFree=5%% --v 4 --logtostderr --network-plugin=kubenet --cni-bin-dir /tmp/node-e2e-20170622T170739/cni/bin --cni-conf-dir /tmp/node-e2e-20170622T170739/cni/net.d --hostname-override tmp-node-e2e-bfe5799d-cos-stable-59-9460-64-0 --experimental-mounter-path=/tmp/node-e2e-20170622T170739/cluster/gce/gci/mounter/mounter --experimental-kernel-memcg-notification=true"

e2e-node-ubuntu:
I0623 00:03:28.526984    2279 server.go:147] Starting server "kubelet" with command "/usr/bin/systemd-run --unit=kubelet-1407651753.service --slice=runtime.slice --remain-after-exit /tmp/node-e2e-20170622T170203/kubelet --kubelet-cgroups=/kubelet.slice --cgroup-root=/ --api-servers http://localhost:8080 --address 0.0.0.0 --port 10250 --read-only-port 10255 --volume-stats-agg-period 10s --allow-privileged true --serialize-image-pulls false --pod-manifest-path /tmp/node-e2e-20170622T170203/pod-manifest083943734 --file-check-frequency 10s --pod-cidr 10.100.0.0/24 --eviction-pressure-transition-period 30s --feature-gates  --eviction-hard memory.available<250Mi,nodefs.available<10%%,nodefs.inodesFree<5%% --eviction-minimum-reclaim nodefs.available=5%%,nodefs.inodesFree=5%% --v 4 --logtostderr --network-plugin=kubenet --cni-bin-dir /tmp/node-e2e-20170622T170203/cni/bin --cni-conf-dir /tmp/node-e2e-20170622T170203/cni/net.d --hostname-override tmp-node-e2e-e48cdd73-ubuntu-gke-1604-xenial-v20170420-1 --experimental-kernel-memcg-notification=true"

e2e-node-containervm:
I0623 00:14:35.392383    2774 server.go:147] Starting server "kubelet" with command "/tmp/node-e2e-20170622T171318/kubelet --runtime-cgroups=/docker-daemon --kubelet-cgroups=/kubelet --cgroup-root=/ --system-cgroups=/system --api-servers http://localhost:8080 --address 0.0.0.0 --port 10250 --read-only-port 10255 --volume-stats-agg-period 10s --allow-privileged true --serialize-image-pulls false --pod-manifest-path /tmp/node-e2e-20170622T171318/pod-manifest507536807 --file-check-frequency 10s --pod-cidr 10.100.0.0/24 --eviction-pressure-transition-period 30s --feature-gates  --eviction-hard memory.available<250Mi,nodefs.available<10%,nodefs.inodesFree<5% --eviction-minimum-reclaim nodefs.available=5%,nodefs.inodesFree=5% --v 4 --logtostderr --network-plugin=kubenet --cni-bin-dir /tmp/node-e2e-20170622T171318/cni/bin --cni-conf-dir /tmp/node-e2e-20170622T171318/cni/net.d --hostname-override tmp-node-e2e-9e3fdd7c-e2e-node-containervm-v20161208-image"

e2e-cos:
Jun 23 17:54:38 e2e-test-ygg-minion-group-t5r0 kubelet[2005]: I0623 17:54:38.646374    2005 flags.go:52] FLAG: --experimental-kernel-memcg-notification="true"

e2e-ubuntu:
Jun 23 18:25:27 e2e-test-ygg-minion-group-19qp kubelet[1547]: I0623 18:25:27.722253    1547 flags.go:52] FLAG: --experimental-kernel-memcg-notification="true"

e2e-containervm:
I0623 18:55:51.886632    3385 flags.go:52] FLAG: --experimental-kernel-memcg-notification="false"
```

**Release note**:
```
None
```

/sig node
/area node-e2e
/assign @dchen1107 @dashpole
2017-06-26 21:08:10 -07:00
Kubernetes Submit Queue 36ae4ae4e3 Merge pull request #47971 from yujuhong/bump-usage-limit
Automatic merge from submit-queue (batch tested with PRs 48074, 47971, 48044, 47514, 47647)

e2e: bump kubelet's resurce usage limit

We don't have per-OS image limits. Bumping these to more generous
numbers to not fail the tests.
2017-06-26 11:40:51 -07:00
Yang Guo 50d49d9c51 Enables memcg notification in cluster/node e2e tests 2017-06-26 11:40:22 -07:00
Kubernetes Submit Queue 14edc46c2e Merge pull request #47892 from ajitak/npd-config
Automatic merge from submit-queue (batch tested with PRs 47993, 47892, 47591, 47469, 47845)

Bump up npd version to v0.4.1

```
Bump up npd version to v0.4.1
```

Fixes #47219
2017-06-23 18:05:46 -07:00
Yang Guo 8ab15e3774 Encodes ReportPrefix into the generated metrics file names 2017-06-23 16:11:25 -07:00
Yu-Ju Hong 71bd92ce3b e2e: bump kubelet's resurce usage limit
We don't have per-OS image limits. Bumping these to more generous
numbers to not fail the tests.
2017-06-23 09:55:18 -07:00
Kubernetes Submit Queue 467705be00 Merge pull request #47195 from dims/bind-cadvisor-on-kubelet-interface
Automatic merge from submit-queue (batch tested with PRs 47922, 47195, 47241, 47095, 47401)

Run cAdvisor on the same interface as kubelet

**What this PR does / why we need it**:

cAdvisor currently binds to all interfaces. Currently the only
solution is to use iptables to block access to the port. We
are better off making cAdvisor to bind to the interface that
kubelet uses for better security.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

Fixes #11710

**Special notes for your reviewer**:

**Release note**:

```release-note
cAdvisor binds only to the interface that kubelet is running on instead of all interfaces.
```
2017-06-22 21:33:27 -07:00
Ajit Kumar caff16c678 Bump up npd version to v0.4.1 2017-06-22 13:13:50 -07:00
Chao Xu 60604f8818 run hack/update-all 2017-06-22 11:31:03 -07:00
Chao Xu f4989a45a5 run root-rewrite-v1-..., compile 2017-06-22 10:25:57 -07:00
mbohlool c91a12d205 Remove all references to types.UnixUserID and types.UnixGroupID 2017-06-21 04:09:07 -07:00
Kubernetes Submit Queue cc645a8c6f Merge pull request #46327 from supereagle/mark-network-plugin-dir-deprecated
Automatic merge from submit-queue (batch tested with PRs 46327, 47166)

mark --network-plugin-dir deprecated for kubelet

**What this PR does / why we need it**:

**Which issue this PR fixes** : fixes #43967

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-06-19 11:23:54 -07:00
Kubernetes Submit Queue b6faf34862 Merge pull request #47530 from mindprince/issue-47388-remove-dead-code
Automatic merge from submit-queue (batch tested with PRs 47530, 47679)

Use cos-stable-59-9460-64-0 instead of cos-beta-59-9460-20-0.

Remove dead code that has now moved to another repo as part of #47467

**Release note**:
```release-note
NONE
```

/sig node
2017-06-16 20:57:58 -07:00
Rohit Agarwal 3a86c97cf6 Use cos-stable-59-9460-64-0 instead of cos-beta-59-9460-20-0.
- It contains a fix for ipaliasing.
- It contains a fix which decouples GPU driver installation from kernel
version.

Remove dead code that has now moved to another repo as part of #47467
2017-06-16 13:48:50 -07:00
Bowei Du 1ed4afca80 Fix hardcoded CIDR in the validation_test
The ideal fix is to not hardcode these values.

fixes #47479
2017-06-15 22:15:56 -07:00
Kubernetes Submit Queue d797c219b3 Merge pull request #47260 from yguo0905/perf-dash
Automatic merge from submit-queue (batch tested with PRs 47470, 47260, 47411, 46852, 46135)

Logs node e2e perf data to standalone json files

Fixes the node-dash-perf issue in https://github.com/kubernetes/kubernetes/issues/44003.

- Move perf data types to `test/e2e/perftype/perftype.go` so that the node-perf-dash can depend on.
- Logs the perf data to standalone json files so that node-perf-dash can consume it easily. A sample run of `ci-kubernetes-node-kubelet-benchmark` is at https://console.cloud.google.com/storage/browser/ygg-gke-dev-bucket/e2e-node-test/ci-kubernetes-node-kubelet-benchmark/1.

The corresponding changes in node-perf-dash is at https://github.com/kubernetes/contrib/pull/2628.

**Release note**:
`None`

/sig node
/area node-e2e
/assign @Random-Liu
2017-06-14 12:52:18 -07:00
Yang Guo 404cda2777 Changes node e2e tests to use new Ubuntu image 2017-06-14 11:44:25 -07:00
Rohit Agarwal 9c0bf19f80 Use cos-stable-59-9460-60-0 and newer installer for GPU node e2e tests. 2017-06-13 15:36:20 -07:00
Kubernetes Submit Queue f4d2c7b931 Merge pull request #46441 from dashpole/eviction_time
Automatic merge from submit-queue

Shorten eviction tests, and increase test suite timeout

After #43590, the eviction manager is less aggressive when evicting pods.  Because of that, many runs in the flaky suite time out.
To shorten the inode eviction test, I have lowered the eviction threshold.
To shorten the allocatable eviction test, I now set KubeReserved = NodeMemoryCapacity - 200Mb, so that any pod using 200Mb will be evicted.  This shortens this test from 40 minutes, to 10 minutes.
While this should be enough to not hit the flaky suite timeout anymore, it is better to keep lower individual test timeouts than a lower suite timeout, since hitting the suite timeout means that even successful test runs are not reported.

/assign @Random-Liu @mtaufen 

issue: #31362
2017-06-13 12:58:22 -07:00
Yang Guo 29b2db5af3 Logs node e2e perf data to standalone json files 2017-06-12 14:27:56 -07:00
David Ashpole 3365cca78a shorten eviction testst and lengthen flaky suite timeout 2017-06-12 12:56:45 -07:00
Rohit Agarwal f7a563435f Fix bad check in node e2e tests for GPUs.
When no nvidia device was attached, the -ne check had a syntax error:

    sh: -ne: argument expected

This resulted in 'Success' being echoed and the test passing incorrectly.
This was found while debugging issue #47216
2017-06-11 19:25:35 -07:00
Kubernetes Submit Queue 3040cba17d Merge pull request #47144 from jingxu97/May/emptyDir
Automatic merge from submit-queue

Fix local capacity isolation test
2017-06-09 12:17:19 -07:00
Kubernetes Submit Queue f75478875a Merge pull request #47113 from feiskyer/cri
Automatic merge from submit-queue

Kubelet: rename cri package name to pkg/kubelet/apis/cri/v1alpha1/runtime

**What this PR does / why we need it**:

We have moved CRI from api/v1alpha1/runtime to apis/cri/v1alpha1, which changed the package name of CRI. This would cause a significant problem: old-versioned runtime (based on CRI in v1.6) doesn't work with latest kubelet v1.7, and vice versa.

This PR renames cri package name to `pkg/kubelet/apis/cri/v1alpha1/runtime` for fixing the problem.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: 

fixes #47012

**Special notes for your reviewer**:

Should be included in v1.7.

**Release note**:

```release-note
CRI has been moved to package `pkg/kubelet/apis/cri/v1alpha1/runtime`.
```
2017-06-09 10:08:36 -07:00
Kubernetes Submit Queue 3a5df705fe Merge pull request #47190 from mindprince/faster-node-e2e-gci
Automatic merge from submit-queue

Move the nvidia installer to the beginning.

When the installer runs for the first time, it disables loadpin and restarts
the node. So, it is better to run it in the beginning so that we can avoid
redoing the later steps. One of the later steps include downloading a tar file
and untarring it. Doing that only once saves around 1m30s in test runtime for
the gci image.

/sig node
/area node-e2e

```release-note
NONE
```
2017-06-09 09:19:16 -07:00
Pengfei Ni 22e99504d7 Update CRI references 2017-06-09 10:16:40 +08:00
Kubernetes Submit Queue 3a96c31de5 Merge pull request #46885 from kewu1992/test_gci_next_canary
Automatic merge from submit-queue (batch tested with PRs 46885, 47197)

Let COS docker validation node test against gci-next-canary

**What this PR does / why we need it**:
This is for COS docker validation node test. We plan to use family gci-next-canary in container-vm-image-staging for future Docker upgration and validation.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #47134

**Special notes for your reviewer**:

**Release note**:

```release-note
```
2017-06-08 15:46:41 -07:00
Davanum Srinivas 7e5c43a042 Run cAdvisor on the same interface as kubelet
cAdvisor currently binds to all interfaces. Currently the only
solution is to use iptables to block access to the port. We
are better off making cAdvisor to bind to the interface that
kubelet uses for better security.

Fixes #11710
2017-06-08 16:43:38 -04:00
Rohit Agarwal 4a5badfafa Move the nvidia installer to the beginning.
When the installer runs for the first time, it disables loadpin and restarts
the node. So, it is better to run it in the beginning so that we can avoid
redoing the later steps. One of the later steps include downloading a tar file
and untarring it. Doing that only once saves around 1m30s in test runtime for
the gci image.
2017-06-08 09:55:14 -07:00
Kubernetes Submit Queue 9c1b2aa9b5 Merge pull request #46743 from Random-Liu/bump-up-npd
Automatic merge from submit-queue

Bump up npd version to v0.4.0

Fixes #47070.

Bump up npd version to [v0.4.0](https://github.com/kubernetes/node-problem-detector/releases/tag/v0.4.0).

```release-note
Bump up Node Problem Detector version to v0.4.0, which added support of parsing log from /dev/kmsg and ABRT.
```

/cc @dchen1107 @ajitak
2017-06-08 08:24:18 -07:00
Jing Xu 426d44ded4 Fix local capacity isolation test
Fix issue #47128, also add flaky tag for this evition test
2017-06-08 06:30:29 -07:00
Kubernetes Submit Queue 6ee028249b Merge pull request #46385 from rickypai/rpai/host_mapping_node_e2e_test
Automatic merge from submit-queue (batch tested with PRs 43005, 46660, 46385, 46991, 47103)

add e2e node test for Pod hostAliases feature

**What this PR does / why we need it**: adds node e2e test for #45148 

tests requested in https://github.com/kubernetes/kubernetes/issues/43632#issuecomment-298434125

**Release note**:
```release-note
NONE
```

@yujuhong @thockin
2017-06-07 13:31:00 -07:00
Random-Liu 1d3979190c Bump up npd version to v0.4.0 2017-06-06 16:30:02 -07:00
Kubernetes Submit Queue 6ed4bc7b97 Merge pull request #46828 from cblecker/links-update
Automatic merge from submit-queue (batch tested with PRs 46718, 46828, 46988)

Update docs/ links to point to main site

**What this PR does / why we need it**:
This updates various links to either point to kubernetes.io or to the kubernetes/community repo instead of the legacy docs/ tree in k/k
Pre-requisite for #46813

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```

@kubernetes/sig-docs-maintainers @chenopis @ahmetb @thockin
2017-06-06 11:43:18 -07:00
Kubernetes Submit Queue 3338d784ba Merge pull request #46899 from mindprince/issue-46889-node-e2e-gpu-cos-fix
Automatic merge from submit-queue (batch tested with PRs 46897, 46899, 46864, 46854, 46875)

Wait for cloud-init to finish before starting tests.

This fixes #46889.


**Release note**:
```release-note
NONE
```
2017-06-06 05:22:42 -07:00
Christoph Blecker 1bdc7a29ae
Update docs/ URLs to point to proper locations 2017-06-05 22:13:54 -07:00
Jing Xu 0b13aee0c0 Add EmptyDir Volume and local storage for container overlay Isolation
This PR adds two features:
1. add support for isolating the emptyDir volume use. If user
sets a size limit for emptyDir volume, kubelet's eviction manager
monitors its usage
and evict the pod if the usage exceeds the limit.
2. add support for isolating the local storage for container overlay. If
the container's overly usage exceeds the limit defined in container
spec, eviction manager will evict the pod.
2017-06-05 12:05:48 -07:00
Rohit Agarwal 1561f55c4c Wait for cloud-init to finish before starting tests.
This fixes #46889.
2017-06-05 10:50:24 -07:00
Kubernetes Submit Queue 6fef1a1deb Merge pull request #46810 from vishh/gpu-cos-image-validation
Automatic merge from submit-queue (batch tested with PRs 46734, 46810, 46759, 46259, 46771)

Update the COS kernel sha for node e2e gpu installer

cc @mindprince

Relevant COS image - https://github.com/kubernetes/kubernetes/blob/master/test/e2e_node/jenkins/image-config-serial.yaml#L19
2017-06-05 06:51:23 -07:00
Kubernetes Submit Queue e6c74bbaaf Merge pull request #46221 from FengyunPan/close-file
Automatic merge from submit-queue

Close file after os.Open()

None
2017-06-03 04:42:00 -07:00
Kubernetes Submit Queue b8c9ee8abb Merge pull request #46456 from jingxu97/May/allocatable
Automatic merge from submit-queue

Add local storage (scratch space) allocatable support

This PR adds the support for allocatable local storage (scratch space).
This feature is only for root file system which is shared by kubernetes
componenets, users' containers and/or images. User could use
--kube-reserved flag to reserve the storage for kube system components.
If the allocatable storage for user's pods is used up, some pods will be
evicted to free the storage resource.

This feature is part of local storage capacity isolation and described in the proposal https://github.com/kubernetes/community/pull/306

**Release note**:

```release-note
This feature exposes local storage capacity for the primary partitions, and supports & enforces storage reservation in Node Allocatable 
```
2017-06-03 00:24:29 -07:00
Kubernetes Submit Queue d063ce213f Merge pull request #46801 from dashpole/summary_container_restart
Automatic merge from submit-queue

[Flaky PR Test] Fix summary test

fixes issue: #46797 

As we can see in the [example failure build log](https://storage.googleapis.com/kubernetes-jenkins/logs/ci-kubernetes-node-kubelet/4319/build-log.txt), the summary containers are pinging google 100s of times a second.  This causes the summary container to be killed occasionally, and fail the test.  The summary containers are only supposed to ping every 10 seconds according to the current test.  As it turns out, we were missing a semicolon, and were not sleeping between pings.  For background, we ping google to generate network traffic, so that the summary test can validate network metrics.

This PR adds the semicolon to make the container sleep between calls, and decreases the sleep time from 10 seconds to 1 second, as 1 call / 10 seconds did not produce enough activity.

cc @kubernetes/kubernetes-build-cops @dchen1107
2017-06-02 18:02:19 -07:00
Ricky Pai 4e7fed4479 e2e node test for PodSpec HostAliases 2017-06-02 17:01:44 -07:00
Kubernetes Submit Queue a6f0033164 Merge pull request #46238 from yguo0905/package-validator
Automatic merge from submit-queue (batch tested with PRs 46648, 46500, 46238, 46668, 46557)

Support validating package versions in node conformance test

**What this PR does / why we need it**:

This PR adds a package validator in node conformance test for checking whether the locally installed packages meet the image spec.

**Special notes for your reviewer**:

The image spec for GKE (which has the package spec) will be in a separate PR. Then we will publish a new node conformance test image for GKE whose name should use the convention in https://github.com/kubernetes/kubernetes/issues/45760 and have `gke` in it.


**Release note**:
```
NONE
```
2017-06-02 15:20:47 -07:00
Ke Wu cb28ed1f95 Let COS docker validation node tests against gci-next-canary
We plan to use family gci-next-canary in container-vm-image-staging
for future Docker upgration and validation.
2017-06-02 14:52:01 -07:00
Vishnu kannan d45286c575 update cos kernel sha for node e2e GPU installer
Signed-off-by: Vishnu kannan <vishnuk@google.com>
2017-06-01 17:09:18 -07:00
Jing Xu 943fc53bf7 Add predicates check for local storage request
This PR adds the check for local storage request when admitting pods. If
the local storage request exceeds the available resource, pod will be
rejected.
2017-06-01 15:57:50 -07:00
Jing Xu dd67e96c01 Add local storage (scratch space) allocatable support
This PR adds the support for allocatable local storage (scratch space).
This feature is only for root file system which is shared by kubernetes
componenets, users' containers and/or images. User could use
--kube-reserved flag to reserve the storage for kube system components.
If the allocatable storage for user's pods is used up, some pods will be
evicted to free the storage resource.
2017-06-01 15:57:50 -07:00
David Ashpole d1545e1e47 add semicolon 2017-06-01 13:32:59 -07:00
supereagle dc9f0f9729 mark --network-plugin-dir deprecated for kubelet, and update related bootstrap scripts 2017-06-01 22:06:44 +08:00
Yang Guo ecf214729d Support validating package versions in node conformance test 2017-05-30 17:44:40 -07:00
David Ashpole e2718f3bc5 fix crossbuild, verify container restarts, and restart only once 2017-05-30 13:15:22 -07:00
Kubernetes Submit Queue 54a47a6f1d Merge pull request #46308 from dashpole/summary_container_restart
Automatic merge from submit-queue (batch tested with PRs 46429, 46308, 46395, 45867, 45492)

Summary Test looks at pods that have containers that restart.

Occasionally, the node can report extra containers that had been restarted through the summary API.
This test change tests a pod that restarts, and hopefully should allow us to reproduce and debug this behavior.

/assign @dchen1107 

/release-note-none
2017-05-25 22:42:04 -07:00
Kubernetes Submit Queue ed8843406e Merge pull request #46303 from Random-Liu/fix-cos-image-project
Automatic merge from submit-queue (batch tested with PRs 46299, 46309, 46311, 46303, 46150)

Fix cos image project to cos-cloud.

Addressed https://github.com/kubernetes/kubernetes/pull/45136#discussion_r118092211.

@vishh @yujuhong @dchen1107
2017-05-24 23:19:09 -07:00
Kubernetes Submit Queue 8d88c55231 Merge pull request #46311 from dashpole/disable_ubuntu_gpu_test
Automatic merge from submit-queue (batch tested with PRs 46299, 46309, 46311, 46303, 46150)

Dont attach a GPU to ubuntu test machines for node e2e serial tests

This should fix flakes in the e2e_node serial suite.

@vishh I think this is what you were asking for...

/assign @vishh
2017-05-24 23:19:07 -07:00
Kubernetes Submit Queue b71ca6691b Merge pull request #46309 from Random-Liu/move-docker-validation-to-separate-project
Automatic merge from submit-queue (batch tested with PRs 46299, 46309, 46311, 46303, 46150)

Move docker validation test to separate project.

Docker validation test is leaking VMs because new docker version `DOCKER_VERSION=17.05.0-c` totally breaks the new gci image `GCE_IMAGES=gci-test-60-9579-0-0` with the `gci-docker-version` metadata specified.

The test successfully created the instance, but timed out when checking VM aliveness, and leaked the VM.

I've cleaned up all leaked VMs. This PR moves docker validation node e2e test into a separate project to not influencing other node e2e test.

@kewu1992 We should fix the docker automated validation test.

/cc @dchen1107 @yujuhong @abgworrall
2017-05-24 23:19:05 -07:00
David Ashpole 1a6572fc6c summary test now tests a pod that has containers that have restarted 2017-05-24 13:27:57 -07:00
Random-Liu 82f588b483 Fix cos image project to cos-cloud. 2017-05-23 15:12:03 -07:00
David Ashpole 8341d544f3 remove unused test properties 2017-05-23 14:39:18 -07:00
David Ashpole 20eb016597 dont attach a GPU to ubuntu machines 2017-05-23 14:34:18 -07:00
Random-Liu dc023144a3 Move docker validation test to separate project. 2017-05-23 14:07:15 -07:00
FengyunPan 287f703d3a Close file after os.Open() 2017-05-22 21:51:11 +08:00
Vishnu kannan 86b5edb79a Update COS version to m59
Signed-off-by: Vishnu kannan <vishnuk@google.com>
2017-05-20 21:17:19 -07:00
Vishnu kannan 1e77594958 Adding an installer script that installs Nvidia drivers in Container Optimized OS
Packaged the script as a docker container stored in gcr.io/google-containers
A daemonset deployment is included to make it easy to consume the installer
A cluster e2e has been added to test the installation daemonset along with verifying installation
by using a sample CUDA application.
Node e2e for GPUs updated to avoid running on nodes without GPU devices.

Signed-off-by: Vishnu kannan <vishnuk@google.com>
2017-05-20 21:17:19 -07:00
Kubernetes Submit Queue 112ed869c7 Merge pull request #46053 from dashpole/test_eviction_metrics
Automatic merge from submit-queue (batch tested with PRs 46033, 46122, 46053, 46018, 45981)

Log age of stats used for evictions during eviction tests

I recently added prometheus metrics for the age of the metrics used for evictions #43031.  It would be nice to surface these during eviction tests, so I can better assess how old stats are, and whether or not the age of stats causes extra evictions.

This isnt super-high priority, and can be done after code-freeze, since it is a testing improvement.  Feel free to take a look whenever either of you has time.

/assign @mtaufen 
/assign @Random-Liu
2017-05-19 23:29:28 -07:00
Kubernetes Submit Queue 51f3ac1b99 Merge pull request #45004 from feiskyer/hostnetwork
Automatic merge from submit-queue

Add node e2e tests for hostNetwork

**What this PR does / why we need it**:

Add node e2e tests for hostNetwork.

**Which issue this PR fixes**

Part of #44118.

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

/assign @Random-Liu @yujuhong
2017-05-19 01:18:07 -07:00
David Ashpole 0bd0d705e3 log age of stats used for evictions during eviction tests 2017-05-18 13:51:23 -07:00
Kubernetes Submit Queue 23f0fe8632 Merge pull request #45901 from Random-Liu/fix-node-e2e
Automatic merge from submit-queue (batch tested with PRs 44520, 45253, 45838, 44685, 45901)

Fix node e2e panic when not using image config file.

In https://github.com/kubernetes/kubernetes/pull/45430, `resources` field in image config is a pointer, and we only initialize it when using image config file.

However, we still have test specifying images directly without image config file, this will cause those test to panic. See:
* https://k8s-testgrid.appspot.com/google-docker#kubelet
* https://k8s-testgrid.appspot.com/google-docker#e2e-cos

This PR fixes this.

@vishh @mtaufen @kewu1992
2017-05-16 21:28:02 -07:00
Kubernetes Submit Queue 85775105f1 Merge pull request #44520 from dashpole/test_eviction_fix
Automatic merge from submit-queue (batch tested with PRs 44520, 45253, 45838, 44685, 45901)

Ensure ordering of using dynamic kubelet config and setting up tests.

This PR simply places the body of the eviction test within its own context.  This ensures that the kubelet config is set before the pods are created, and that the kubelet config is reverted only after the pods are deleted.
2017-05-16 21:27:54 -07:00
Pengfei Ni f9eafea8bf Add node e2e tests for hostNetwork 2017-05-17 10:31:17 +08:00
Kubernetes Submit Queue d823a6e228 Merge pull request #45899 from vishh/fix-nodee2e-sone
Automatic merge from submit-queue (batch tested with PRs 45247, 45810, 45034, 45898, 45899)

Fix zone in node e2e serial tests

```shell
$ gcloud compute regions describe us-west1 --project k8s-jkns-ci-node-e2e
creationTimestamp: '2016-06-14T17:29:18.761-07:00'
description: us-west1
id: '1210'
kind: compute#region
name: us-west1
quotas:
- limit: 100.0
  metric: CPUS
  usage: 0.0
- limit: 10240.0
  metric: DISKS_TOTAL_GB
  usage: 0.0
- limit: 7.0
  metric: STATIC_ADDRESSES
  usage: 0.0
- limit: 100.0
  metric: IN_USE_ADDRESSES
  usage: 0.0
- limit: 2048.0
  metric: SSD_TOTAL_GB
  usage: 0.0
- limit: 10240.0
  metric: LOCAL_SSD_TOTAL_GB
  usage: 0.0
- limit: 100.0
  metric: INSTANCE_GROUPS
  usage: 0.0
- limit: 50.0
  metric: INSTANCE_GROUP_MANAGERS
  usage: 0.0
- limit: 1000.0
  metric: INSTANCES
  usage: 0.0
- limit: 50.0
  metric: AUTOSCALERS
  usage: 0.0
- limit: 20.0
  metric: REGIONAL_AUTOSCALERS
  usage: 0.0
- limit: 20.0
  metric: REGIONAL_INSTANCE_GROUP_MANAGERS
  usage: 0.0
selfLink: https://www.googleapis.com/compute/v1/projects/k8s-jkns-ci-node-e2e/regions/us-west1
status: UP
zones:
- https://www.googleapis.com/compute/v1/projects/k8s-jkns-ci-node-e2e/zones/us-west1-a
- https://www.googleapis.com/compute/v1/projects/k8s-jkns-ci-node-e2e/zones/us-west1-b
```
2017-05-16 19:02:03 -07:00
Random-Liu 56803ec97d Fix node e2e panic when not using image config file. 2017-05-16 11:36:50 -07:00
Vishnu Kannan 03b3d2e119 fix zone in node e2e serial tests
Signed-off-by: Vishnu Kannan <vishnuk@google.com>
2017-05-16 11:03:07 -07:00
David Ashpole 9f098a9c1a add context around test 2017-05-16 09:38:41 -07:00
zhengjiajin de1150385d fix typo 2017-05-15 12:07:36 +08:00
Vishnu kannan d1b4dba440 adding support for gpus in node e2e
Signed-off-by: Vishnu kannan <vishnuk@google.com>
2017-05-13 16:35:54 -07:00
Kubernetes Submit Queue 3619c33350 Merge pull request #42759 from mtaufen/kubelet-apis-reorg
Automatic merge from submit-queue

Reorganize kubelet tree so apis can be independently versioned

@yujuhong @lavalamp @thockin @bgrant0607 
This is an example of how we might reorganize `pkg/kubelet` so the apis it exposes can be independently versioned. This would also provide a logical place to put the `KubeletConfiguration` type, which currently lives in `pkg/apis/componentconfig`; it could live in e.g. `pkg/kubelet/apis/config` instead.

Take a look when you have a chance and let me know what you think. The most significant change in this PR is reorganizing `pkg/kubelet/api` to `pkg/kubelet/apis`, the rest is pretty much updating import paths and `BUILD` files.
2017-05-12 17:43:22 -07:00
Michael Taufen cbad320205 Reorganize kubelet tree so apis can be independently versioned 2017-05-12 10:02:33 -07:00
Kubernetes Submit Queue 3d704fa40f Merge pull request #45676 from mtaufen/fix-dkcfg-test-gci
Automatic merge from submit-queue

Fix flag formatting errors in the node tests

There were three problems:
- Lack of a trailing space after prepending flags.
- Passing multiple flags in a string to --kubelet-flags seems to confuse
the flag parser; it stops parsing ALL flags as soon as it sees the
second kubelet flag. Fortunately, all instances of --kubelet-flags are
combined together, so we can just pass two of those.
- --feature-gates should be passed to the test framework, which then
forwards it to the kubelet, instead of using --kubelet-flags.

This hopefully fixes the dynamic config test failures on COS, which
started after #45602. (See: https://k8s-testgrid.appspot.com/google-node#kubelet-serial-gce-e2e)
2017-05-12 09:04:38 -07:00
Kubernetes Submit Queue e1bb9a5177 Merge pull request #45667 from yujuhong/mv-pull-tests
Automatic merge from submit-queue (batch tested with PRs 45691, 45667, 45698, 45715)

dockertools: migrate the unit tests and delete the package
2017-05-12 04:09:41 -07:00
Kubernetes Submit Queue ee4e5e79f3 Merge pull request #45437 from kewu1992/cos-docker-validation
Automatic merge from submit-queue

Add properties file for cos-docker-validation test job

**What this PR does / why we need it**: 
This is forked from test/e2e_node/jenkins/docker_validation/jenkins-validation.properties. It is used for COS docker validation test.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```NONE
```
2017-05-11 20:58:49 -07:00
Michael Taufen abb5c3fd5a Fix flag formatting errors in the node tests
There were three problems:
- Lack of a trailing space after prepending flags.
- Passing multiple flags in a string to --kubelet-flags seems to confuse
the flag parser; it stops parsing ALL flags as soon as it sees the
second kubelet flag. Fortunately, all instances of --kubelet-flags are
combined together, so we can just pass two of those.
- --feature-gates should be passed to the test framework, which then
forwards it to the kubelet, instead of using --kubelet-flags.

This hopefully fixes the dynamic config test failures on COS, which
started after #45602.
2017-05-11 11:28:49 -07:00
Yu-Ju Hong fccf34ccb6 Remove various references of dockertools
Also update the bazel files.
2017-05-11 10:01:41 -07:00
Kubernetes Submit Queue a507d30833 Merge pull request #45602 from dashpole/enable_memcg_for_all_tests
Automatic merge from submit-queue (batch tested with PRs 45569, 45602, 45604, 45478, 45550)

Enable kernel memcg notification for node and cluster GCI/COS testing.

Sets --experimental-kernel-memcg-notification=true when running on the GCI/COS image.  It sets this for master and nodes for cluster e2e tests, and for the node in node e2e tests.

Issue #42676 

cc @dchen1107 @Random-Liu
2017-05-10 21:34:39 -07:00
David Ashpole 0b1e45c5ff enable memcg on all testing 2017-05-10 11:38:26 -07:00
Kubernetes Submit Queue 51a3413371 Merge pull request #45307 from yujuhong/mv-docker-client
Automatic merge from submit-queue (batch tested with PRs 45453, 45307, 44987)

Migrate the docker client code from dockertools to dockershim

Move docker client code from dockertools to dockershim/libdocker. This includes
DockerInterface (renamed to Interface), FakeDockerClient, etc.

This is part of #43234
2017-05-09 20:23:44 -07:00
Pengfei Ni 2b540b6d74 Add node e2e tests for hostIPC 2017-05-09 18:25:19 +08:00
Ke Wu bdcfad15ce Add properties file for cos-docker-validation test job 2017-05-05 14:49:25 -07:00
Yu-Ju Hong cf3635c876 Update bazel BUID files 2017-05-05 11:48:08 -07:00
Yu-Ju Hong 389c140eaf Move docker client code from dockertools to dockershim/dockerlib
The code affected include DockerInterface (renamed to Interface),
FakeDockerClient, etc.
2017-05-05 11:48:08 -07:00
Jamie Hannaford 9440a68744 Use dedicated Unix User and Group ID types 2017-05-05 14:07:38 +02:00
Yu-Ju Hong 5644587e07 More dockertools cleanup
Move some constants/functions to dockershim and remove unused tests.
2017-05-03 11:22:06 -07:00
Kubernetes Submit Queue 4998d78f89 Merge pull request #43883 from yujuhong/rm_non-cri-configs
Automatic merge from submit-queue (batch tested with PRs 43884, 44712, 45124, 43883)

Node e2e: Remove CRI/non-CRI configs

This depends on https://github.com/kubernetes/test-infra/pull/2363
2017-05-01 15:49:13 -07:00
Yang Guo f7c2efa42b adds Ubuntu node e2e tests 2017-05-01 12:22:11 -07:00
Kubernetes Submit Queue 8efb5c9957 Merge pull request #44983 from caesarxuchao/easy-remove-client-go-api-scheme
Automatic merge from submit-queue (batch tested with PRs 45052, 44983, 41254)

Non-controversial part of #44523

For easier review of #44523, i extracted the non-controversial part out to this PR.
2017-04-27 17:14:04 -07:00
Kubernetes Submit Queue 8ab63dd9ea Merge pull request #42740 from mtaufen/tarball-cleanup
Automatic merge from submit-queue (batch tested with PRs 42740, 44980, 45039, 41627, 45044)

Cleanup some of the tarball producing code for e2e node tests

This is some e2e node cleanup work I found sitting in a local branch while deleting old local git branches. It looks like it's still useful.
2017-04-27 13:27:00 -07:00
Chao Xu 958903509c bazel 2017-04-27 09:41:53 -07:00
Chao Xu 3fa7b7824a easy changes 2017-04-27 09:41:53 -07:00
Kubernetes Submit Queue 7670341f56 Merge pull request #43395 from sjenning/selinux-e2e-node-lifecycle
Automatic merge from submit-queue (batch tested with PRs 43395, 44960)

e2e-node: refactor lifecycle test to avoid selinux issues

Fixes #42905

Previously, the exec hook tests mounted a HostPath volume from /tmp and touched a file as a indicator that the hook had run.  This is prohibited by selinux policy on Fedora/RHEL/Centos.

This PR refactors the test to avoid filesystem indication and use the same indication that the HTTP hooks use; a GET to a http endpoint.  The exec hooks run `curl` to hit this endpoint and trigger the indication.  This simplifies this test quite a bit as well, removing over 85 lines of code.

REVIEWER NOTE: The diff is a mess on this one.  Probably better to just review the new version of the file.

@derekwaynecarr
2017-04-26 14:29:41 -07:00
Ryan Hitchman 0c6fd62582 Fix the last deprecated "gcloud docker push args" usage. 2017-04-26 11:20:25 -07:00
NickrenREN d4376599ba Cleanup: replace some hardcoded codes and remove unused functions 2017-04-25 09:38:25 +08:00
Maru Newby 9413071ce8 e2e: Prefer kubeconfig host to default 2017-04-19 14:58:43 -07:00
Kubernetes Submit Queue 4f50a8d7cd Merge pull request #44457 from dnardo/e2e_node_cni
Automatic merge from submit-queue (batch tested with PRs 43000, 44500, 44457, 44553, 44267)

Updates e2e_node test to allow both kubenet and cni to be specified f…

…or the network plugin.

This adds a simple CNI configuration which is added to the node during test setup.
This also modifies the default flags in services/kubelet.go to specify the "cni-bin-dir"
and the "cni-conf-dir" and removes the "network-plugin-dir" flag.  This leaves the default
network plugin to kubenet.



**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```
2017-04-18 13:19:08 -07:00
Chao Xu 4f9591b1de move pkg/api/v1/ref.go and pkg/api/v1/resource.go to subpackages. move some functions in resource.go to pkg/api/v1/node and pkg/api/v1/pod 2017-04-17 11:38:11 -07:00
Mike Danese a05c3c0efd autogenerated 2017-04-14 10:40:57 -07:00
Daniel Nardo 29b2708046 Add comment to setupCNI. 2017-04-13 15:11:24 -07:00
Daniel Nardo 4e458ce001 Updates e2e_node test to allow both kubenet and cni to be specified for the network plugin.
This adds a simple CNI configuration which is added to the node during test setup.
This also modifies the default flags in services/kubelet.go to specify the "cni-bin-dir"
and the "cni-conf-dir" and removes the "network-plugin-dir" flag.  This leaves the default
network plugin to kubenet.
2017-04-13 09:57:46 -07:00
Timothy St. Clair 442713aaaf Remove leagcy init that no longer works. 2017-04-11 08:48:59 -05:00