Commit Graph

8510 Commits (58fe20f0e6f4315df685c47c7dfe171b82e53427)

Author SHA1 Message Date
Jing Xu 4d6da1fd9a Change SizeLimit to a pointer
This PR fixes issue #50121
2017-08-30 11:50:35 -07:00
Kubernetes Submit Queue ad6c85ca2e Merge pull request #51622 from shyamjvs/correct-hollow-node-listing
Automatic merge from submit-queue

Only list hollow-node pods while trying to count them

Otherwise it's counting 1 extra due to heapster pod.
2017-08-30 09:19:08 -07:00
Kubernetes Submit Queue 022919d1a4 Merge pull request #51483 from yujuhong/e2e-net-tiers
Automatic merge from submit-queue

e2e: Add tests for network tiers in GCE

This test depends on #51301, which adds the new feature. Only the `e2e: Add tests for network tiers in GCE` commit is new.
#51301 should pass this new test.
2017-08-30 06:55:35 -07:00
Shyam Jeedigunta 1f6809b746 Only list hollow-node pods while trying to count them 2017-08-30 14:02:33 +02:00
Kubernetes Submit Queue a0fe767fa5 Merge pull request #51539 from saad-ali/enableAlphaGCEDiskAPI
Automatic merge from submit-queue (batch tested with PRs 51439, 51361, 51140, 51539, 51585)

Enable alpha GCE disk API

This PR builds on top of #50467 to allow the GCE disk API to use either the alpha or stable APIs.

CC @freehan
2017-08-30 03:59:35 -07:00
Kubernetes Submit Queue 1fc7cd3d1d Merge pull request #51545 from sttts/sttts-deepcopy-e2e
Automatic merge from submit-queue (batch tested with PRs 47054, 50398, 51541, 51535, 51545)

e2e/integration: simplify deepcopy calls
2017-08-30 01:51:37 -07:00
Kubernetes Submit Queue 04bc4ec716 Merge pull request #50398 from pci/gcloud-compute-list
Automatic merge from submit-queue (batch tested with PRs 47054, 50398, 51541, 51535, 51545)

Switch away from gcloud deprecated flags in compute resource listings

**What is fixed**

Remove deprecated `gcloud compute` flags, see linked issue.

**Which issue this PR fixes**:

fixes #49673 

**Special notes for your reviewer**:

The change in `gcloudComputeResourceList` in `test/e2e/framework/ingress_utils.go` isn't strictly needed as currently no affected resources are called on within that file, however the function has the _potential_ to access affected resources so I covered it as well. Happy to change if deemed unnecessary.

**Release note**:

```release-note
NONE
```
2017-08-30 01:51:29 -07:00
Kubernetes Submit Queue b4d08cb9b5 Merge pull request #50940 from MrHohn/kube-proxy-ds-upgrade-tests
Automatic merge from submit-queue (batch tested with PRs 51228, 50185, 50940, 51544, 51543)

Add upgrades tests for kube-proxy daemonset migration path

**What this PR does / why we need it**:
From #23225, this is a part of setting up CIs to validate the kube-proxy migration path (static pods -> daemonset and reverse).
The other part of the works (adding real CIs that run these tests) will be in a separate PR against [kubernetes/test-infra](https://github.com/kubernetes/test-infra).

Though this is currently blocked by #50705.

**Special notes for your reviewer**:
cc @roberthbailey  @pwittrock 

**Release note**:

```release-note
NONE
```
2017-08-29 23:54:30 -07:00
Vaibhav Kamra 1ac56d8cbb Add PVCRef to VolumeStats
For pod volumes that reference a PVC, add a PVCRef to the corresponding
volume stat. This allows metrics to be indexed/queried by PVC name
which is more user-friendly than Pod reference
2017-08-29 23:12:20 -07:00
Kubernetes Submit Queue 01e961b380 Merge pull request #49749 from sbezverk/e2e_selinux_local_starage_test
Automatic merge from submit-queue (batch tested with PRs 51377, 46580, 50998, 51466, 49749)

Adding e2e SELinux test for local storage

Adding e2e test for SELinux enabled local storage
/sig storage
Closes #45054
2017-08-29 22:57:11 -07:00
Kubernetes Submit Queue 4b8a78cb8e Merge pull request #51466 from yguo0905/ubuntu-docker
Automatic merge from submit-queue (batch tested with PRs 51377, 46580, 50998, 51466, 49749)

Use the pre-built docker binaries on Ubuntu for benchmark tests

- Tested manually.
- The `ubuntu-init-docker.yaml` is copied from `cos-init-docker.yaml` with the following changes needed by Ubuntu. This change is temporary -- we will remove the script and the tests once we know the performance of using the pre-built Docker 1.12 on Ubuntu.

```
71,72c71,72
<         mount --bind "${install_location}"/docker-containerd /usr/bin/docker-containerd
<         mount --bind "${install_location}"/docker-containerd-shim /usr/bin/docker-containerd-shim
---
>         mount --bind "${install_location}"/docker-containerd /usr/bin/containerd
>         mount --bind "${install_location}"/docker-containerd-shim /usr/bin/containerd-shim
75c75
<         mount --bind "${install_location}"/docker-runc /usr/bin/docker-runc
---
>         mount --bind "${install_location}"/docker-runc /usr/sbin/runc
88c88
<         local requested_version="$(get_metadata "gci-docker-version")"
---
>         local requested_version="$(get_metadata "ubuntu-docker-version")"
93,98d92
<         # Check if we have the requested version installed.
<         if check_installed /usr/bin/docker "${requested_version}"; then
<           echo "Requested version already installed. Exiting."
<           exit 0
<         fi
< 
100c94
<         /usr/bin/systemctl stop docker
---
>         systemctl stop docker
106c100
<         /usr/bin/systemctl start docker && exit $rc
---
>         systemctl start docker && exit $rc
```

- Updated all tests to use the latest Ubuntu image.

**Release note**:
```
None
```

/assign @Random-Liu
2017-08-29 22:57:08 -07:00
NickrenREN 4ca27417d9 Add pod local ephemeral storage usage e2e test cases 2017-08-30 13:54:26 +08:00
Philip Ingrey 697f92a5d2
Switch away from gcloud deprecated flags in compute resource listings 2017-08-30 06:41:09 +01:00
Kubernetes Submit Queue 9d9a0428b8 Merge pull request #50005 from NickrenREN/local-storage-e2e
Automatic merge from submit-queue (batch tested with PRs 49961, 50005, 50738, 51045, 49927)

Add cluster e2es to verify scheduler local storage support

Add cluster e2es to verify scheduler local storage support and remove some unused private functions

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:  
part of #50818

**Release note**:
```release-note
Add cluster e2es to verify scheduler local ephemeral storage support
```

/assign @jingxu97 
/cc @ddysher
2017-08-29 21:43:32 -07:00
saadali 4ca1dbc589 Enable switching to alpha GCE disk API 2017-08-29 17:23:25 -07:00
Eric Chiang 21666682eb test/e2e/auth: fix audit log test format parsing 2017-08-29 17:15:38 -07:00
Shyam JVS 36910232ab Merge pull request #51343 from shyamjvs/correct-cluster-ip-range
Correct default cluster-ip-range subnet
2017-08-30 01:31:50 +02:00
Shyam Jeedigunta 2df4698473 Correct default cluster-ip-range subnet 2017-08-29 23:15:23 +02:00
Dr. Stefan Schimanski 637fe0844c e2e/integration: simplify deepcopy calls 2017-08-29 20:11:50 +02:00
Zihong Zheng 5dc0845e36 Add upgrades tests for kube-proxy daemonset migration path 2017-08-29 10:16:37 -07:00
Kubernetes Submit Queue 25da6e64e2 Merge pull request #48454 from weiwei04/check-job-activeDeadlineSeconds
Automatic merge from submit-queue (batch tested with PRs 44719, 48454)

check job ActiveDeadlineSeconds

**What this PR does / why we need it**:

enqueue a sync task after ActiveDeadlineSeconds

**Which issue this PR fixes** *: 

fixes #32149

**Special notes for your reviewer**:

**Release note**:

```release-note
enqueue a sync task to wake up jobcontroller to check job ActiveDeadlineSeconds in time
```
2017-08-29 08:25:06 -07:00
Kubernetes Submit Queue 28f6b3fcc0 Merge pull request #50910 from wasylkowski/autoscaler-test-6
Automatic merge from submit-queue

Added an end-to-end test ensuring that Cluster Autoscaler does not scale up when all pending pods are unschedulable

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-08-29 06:00:03 -07:00
Wei Wei 46239ea30b check job ActiveDeadlineSeconds 2017-08-29 20:15:11 +08:00
Kubernetes Submit Queue d25a78a692 Merge pull request #51410 from nicksardo/gce-consume-netproj
Automatic merge from submit-queue (batch tested with PRs 50919, 51410, 50099, 51300, 50296)

GCE: Read networkProjectID param

Fixes #48515 

/assign bowei

The first commit is the original PR cherrypicked. The master's kubelet isn't provided a cloud config path, so the project is retrieved via instance metadata. In the GKE case, this project cannot be retrieved by the master and caused an error.

**Release note**:
```release-note
NONE
```
2017-08-29 03:20:02 -07:00
Andrzej Wasylkowski 0c1ab5597e Renamed ClusterSize and WaitForClusterSize to NumberOfReadyNodes and WaitForReadyNodes, respectively. 2017-08-29 11:53:17 +02:00
Andrzej Wasylkowski 9b0f4c9f7c Added an end-to-end test ensuring that Cluster Autoscaler does not scale up when all pending pods are unschedulable. 2017-08-29 11:52:26 +02:00
Kubernetes Submit Queue 12d73c31a9 Merge pull request #51436 from liggitt/initializer-feature
Automatic merge from submit-queue (batch tested with PRs 51471, 50561, 50435, 51473, 51436)

Feature gate initializers field

The metadata.initializers field should be feature gated and disabled by default while in alpha, especially since enforcement of initializer permission that keeps users from submitting objects with their own initializers specified is done via an admission plugin most clusters do not enable yet.

Not gating the field and tests caused tests added in https://github.com/kubernetes/kubernetes/issues/51429 to fail on clusters that don't enable the admission plugin.

This PR:
* adds an `Initializers` feature gate, auto-enables the feature gate if the admission plugin is enabled
* clears the `metadata.initializers` field of objects on create/update if the feature gate is not set
* marks the e2e tests as feature-dependent (will follow up with PR to test-infra to enable the feature and opt in for GCE e2e tests)

```release-note
Use of the alpha initializers feature now requires enabling the `Initializers` feature gate. This feature gate is auto-enabled if the `Initialzers` admission plugin is enabled.
```
2017-08-29 02:22:19 -07:00
Maciej Szulik 2de214b044
Enable batch/v1beta1.CronJobs by default 2017-08-29 09:31:39 +02:00
Yu-Ju Hong f33c37e102 e2e: Add tests for network tiers in GCE 2017-08-28 18:40:20 -07:00
Chao Xu 85ee09e4c9 update initializer names to valid ones in tests 2017-08-28 16:17:57 -07:00
Serguei Bezverkhi d904e52570 Adding e2e SELinux test for local storage
Also changing provisioner bootstrapper frpm Pod to Job
2017-08-28 19:12:17 -04:00
Yang Guo 039178b27f Use the pre-built docker binaries on Ubuntu for benchmark tests 2017-08-28 14:06:23 -07:00
Nick Sardo a48fe406be Consume new config value for network project id 2017-08-28 11:11:05 -07:00
Jordan Liggitt 658956f063
Feature gate initializers field 2017-08-28 11:11:48 -04:00
Shyam Jeedigunta 240a1ae5ab Make threshold for glbc mem-usage scale with nodes in density test 2017-08-28 13:24:24 +02:00
Kubernetes Submit Queue daf591c193 Merge pull request #51117 from k82cn/k8s_50360_2
Automatic merge from submit-queue

Moved node condition filter into a predicates.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #50360

**Release note**:

```release-note
A new predicates, named 'CheckNodeCondition', was added to replace node condition filter. 'NetworkUnavailable', 'OutOfDisk' and 'NotReady' maybe reported as a reason when failed to schedule pods.
```
2017-08-28 01:22:27 -07:00
Kubernetes Submit Queue 877ee91930 Merge pull request #51082 from caesarxuchao/repair-null-pending-initializer
Automatic merge from submit-queue (batch tested with PRs 50953, 51082)

Fix mergekey of initializers; Repair invalid update of initializers

Fix https://github.com/kubernetes/kubernetes/issues/51131

The PR did two things to make parallel patching `metadata.initializers.pending` possible:
* Add mergekey to initializers.pending
* Let the initializer admission plugin set the `metadata.intializers` to nil if an update makes the `pending` and the `result` both nil, instead of returning a validation error. Otherwise if multiple initializer controllers sending the patch removing themselves from `pending` at the same time, one of them will get a validation error.


```release-note
The patch to remove the last initializer from metadata.initializer.pending will result in metadata.initializer to be set to nil (assuming metadata.initializer.result is also nil), instead of resulting in an validation error.
```
2017-08-26 23:03:01 -07:00
Kubernetes Submit Queue 6109b4015c Merge pull request #51417 from kubernetes/revert-42689-enable-oir-e2e
Automatic merge from submit-queue

Revert "Re-enable OIR e2e tests."

Reverts kubernetes/kubernetes#42689

e2e tests are very flaky since that pr merged, and it's very likely related:
https://storage.googleapis.com/k8s-gubernator/triage/index.html?pr=1&test=Opaque%20resources
2017-08-26 20:06:46 -07:00
Kubernetes Submit Queue 562d2be7a0 Merge pull request #49006 from CaoShuFeng/is_forbidden
Automatic merge from submit-queue

Fix forbidden message format

Before this change:
 $ kubectl get pods --as=tom
 Error from server (Forbidden): pods "" is forbidden: User "tom" cannot list pods in the namespace "default".
After this change:
 $ kubectl get pods --as=tom
 Error from server (Forbidden): pods is forbidden: User "tom" cannot list pods in the namespace "default".



**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```
Fix forbidden message format, remove extra ""
```
2017-08-26 19:01:21 -07:00
Antoine Pelisse 281630b0b0 Revert "Re-enable OIR e2e tests." 2017-08-26 13:09:21 -07:00
Kubernetes Submit Queue 4b7135513f Merge pull request #51382 from nicksardo/revert-51038-gce-netproj
Automatic merge from submit-queue (batch tested with PRs 51174, 51363, 51087, 51382, 51388)

Revert "GCE: Consume new config value for network project id"

Reverts kubernetes/kubernetes#51038

Broke GKE tests
2017-08-26 06:43:33 -07:00
Kubernetes Submit Queue 1e5d85a0bb Merge pull request #51174 from caesarxuchao/fix-resourcequota
Automatic merge from submit-queue

Let the quota evaluator handle mutating specs of pod & pvc

### Background
The final goal is to address https://github.com/kubernetes/kubernetes/issues/47837, which aims to allow more mutation for uninitialized objects.

To do that, we [decided](https://github.com/kubernetes/kubernetes/issues/47837#issuecomment-321462433) to let the admission controllers to handle mutation of uninitialized objects.

### Issue
#50399 attempted to fix all admission controllers so that can handle mutating uninitialized objects. It was incomplete. I didn't realize although the resourcequota admission plugin handles the update operation, the underlying evaluator didn't. This PR updated the evaluators to handle updates of uninitialized pods/pvc.

### TODO
We still miss another piece. The [quota replenish controller](https://github.com/kubernetes/kubernetes/blob/master/pkg/controller/resourcequota/replenishment_controller.go) uses the sharedinformer, which doesn't observe the deletion of uninitialized pods at the moment. So there is a quota leak if a pod is deleted before it's initialized. It will be addressed with https://github.com/kubernetes/kubernetes/issues/48893.
2017-08-26 06:07:29 -07:00
Kubernetes Submit Queue 6368c1fc82 Merge pull request #51348 from rmmh/coreos-no-password
Automatic merge from submit-queue

Make coreos test images sshd not allow password login.

This will prevent security scanners from triggering.

Configuration is verbatim from:
https://coreos.com/os/docs/latest/customizing-sshd.html

```release-note
NONE
```
2017-08-26 04:19:11 -07:00
Kubernetes Submit Queue 932e07af53 Merge pull request #50031 from verult/ConnectedProbe
Automatic merge from submit-queue (batch tested with PRs 51054, 51101, 50031, 51296, 51173)

Dynamic Flexvolume plugin discovery, probing with filesystem watch.

**What this PR does / why we need it**: Enables dynamic Flexvolume plugin discovery. This model uses a filesystem watch (fsnotify library), which notifies the system that a probe is necessary only if something changes in the Flexvolume plugin directory.

This PR uses the dependency injection model in https://github.com/kubernetes/kubernetes/pull/49668.

**Release Note**:
```release-note
Dynamic Flexvolume plugin discovery. Flexvolume plugins can now be discovered on the fly rather than only at system initialization time.
```

/sig-storage

/assign @jsafrane @saad-ali 
/cc @bassam @chakri-nelluri @kokhang @liggitt @thockin
2017-08-26 02:05:34 -07:00
NickrenREN 18f086c6f6 Add cluster e2es to verify scheduler local storage support 2017-08-26 14:44:15 +08:00
NickrenREN 40bcad482f Remove private and unused codes 2017-08-26 14:21:05 +08:00
Kubernetes Submit Queue b65d665b99 Merge pull request #51264 from m1093782566/e2e-maxTries
Automatic merge from submit-queue (batch tested with PRs 50889, 51347, 50582, 51297, 51264)

Fix e2e network util wrong output message

**What this PR does / why we need it**:

See https://github.com/kubernetes/kubernetes/blob/master/test/e2e/framework/networking_utils.go#L217

and 

https://github.com/kubernetes/kubernetes/blob/master/test/e2e/framework/networking_utils.go#L273

I assume it should be `minTries` -> `MaxTries`

This PR fixes the wrong output message.

**Which issue this PR fixes**: fixes #51265

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-08-25 22:43:37 -07:00
Kubernetes Submit Queue 76c520cea3 Merge pull request #50889 from NickrenREN/local-storage-eviction
Automatic merge from submit-queue (batch tested with PRs 50889, 51347, 50582, 51297, 51264)

Change eviction manager to manage one single local storage resource

**What this PR does / why we need it**:
We decided to manage one single resource name, eviction policy should be modified too.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:  part of #50818

**Special notes for your reviewer**:

**Release note**:
```release-note
Change eviction manager to manage one single local ephemeral storage resource
```

/assign @jingxu97
2017-08-25 22:43:26 -07:00
Cao Shufeng ab09186737 Fix forbidden message format
Before this change:
 # kubectl get pods --as=tom
 Error from server (Forbidden): pods "" is forbidden: User "tom" cannot list pods in the namespace "default".
After this change:
 # kubectl get pods --as=tom
 Error from server (Forbidden): pods is forbidden: User "tom" cannot list pods in the namespace "default".
2017-08-26 10:27:35 +08:00
Kubernetes Submit Queue 65da3ce246 Merge pull request #51235 from cheftako/aggregator
Automatic merge from submit-queue

Fixed gke auth update wait condition.

Lookup whoami on gke using gcloud auth list.
Make sure we do not run the test on any cluster older than 1.7.

**What this PR does / why we need it**: Fixes issue with aggregator e2e test on GKE

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #50945 

**Special notes for your reviewer**: There is a TODO, follow up will be provided when the immediate problem is resolved.

**Release note**: ```release-note
NONE
```
2017-08-25 18:52:46 -07:00
Klaus Ma 18dc690c7c Moved node condition filter into a predicates. 2017-08-26 09:08:07 +08:00
Nick Sardo 0d55f6bdcb Revert "GCE: Consume new config value for network project id" 2017-08-25 18:02:10 -07:00
NickrenREN 27901ad5df Change eviction policy to manage one single local storage resource 2017-08-26 05:14:49 +08:00
Kubernetes Submit Queue a235ba4e49 Merge pull request #51327 from wasylkowski/ensure-ca-is-on
Automatic merge from submit-queue (batch tested with PRs 51134, 51122, 50562, 50971, 51327)

Made the tests ensure that Cluster Autoscaler is on before running.

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-08-25 14:01:36 -07:00
Ryan Hitchman a7e64aaa66 Make coreos test images sshd not allow password login.
Configuration is based on:
https://coreos.com/os/docs/latest/customizing-sshd.html

The specific SSHD config is:

    # Use most defaults for sshd configuration.
    UsePrivilegeSeparation sandbox
    Subsystem sftp internal-sftp
    ClientAliveInterval 180
    UseDNS no
    UsePAM yes
    PrintLastLog no # handled by PAM
    PrintMotd no # handled by PAM
    AuthenticationMethods publickey

This will prevent security scanners from triggering.
2017-08-25 11:49:34 -07:00
Cheng Xing 396c3c7c6f Adding dynamic Flexvolume plugin discovery capability, using filesystem watch. 2017-08-25 11:42:32 -07:00
Walter Fender 3b9485bba3 Fixed gke auth update wait condition.
Lookup whoami on gke using gcloud auth list.
Make sure we do not run the test on any cluster older than 1.7.
Fix for Mehdy
Fixes for LavaLamp
2017-08-25 11:11:59 -07:00
Nick Sardo 0028385e20 Consume new config value for network project id 2017-08-25 08:42:28 -07:00
Kubernetes Submit Queue 11299e363c Merge pull request #51282 from shyamjvs/new-allowed-not-ready-semantics
Automatic merge from submit-queue

AllowedNotReadyNodes allowed to be not ready for absolutely *any* reason

It's as good as we allow those many nodes to be not part of the cluster at all, ever.

Btw - currently our 5k-node correctness test fails if "kubelet stopped posting node status" or "route not created", etc (ref: https://storage.googleapis.com/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-scale-correctness/3/build-log.txt)

cc @kubernetes/sig-scalability-misc
2017-08-25 05:00:32 -07:00
Kubernetes Submit Queue d1783e0bd6 Merge pull request #51194 from bskiba/run_on_each_node
Automatic merge from submit-queue (batch tested with PRs 51244, 50559, 49770, 51194, 50901)

Distribute pods efficiently in CA scalability tests

**What this PR does / why we need it**:
Instead of using runReplicatedPodOnEachNode method
which is suited to a small number of nodes,
distribute pods on the nodes with desired load
using RCs that eat up all the space we want to be
empty after distribution.

**Release note**:
```
NONE
```
2017-08-25 04:11:13 -07:00
Andrzej Wasylkowski 6e1fbf32b0 Made the tests ensure that Cluster Autoscaler is on before running. 2017-08-25 11:11:38 +02:00
Kubernetes Submit Queue 81363abc20 Merge pull request #51230 from enisoc/sts-deflake-exec
Automatic merge from submit-queue (batch tested with PRs 50213, 50707, 49502, 51230, 50848)

StatefulSet: Deflake e2e `kubectl exec` commands.

This may help with another source of flakiness found while investigating #48031.

We seem to get a lot of flakes due to "connection refused" while running `kubectl exec`. I can't find any reason this would be caused by the test flow, so I'm adding retries to see if that helps.
2017-08-25 01:10:35 -07:00
Kubernetes Submit Queue 4a94363c7e Merge pull request #51158 from yguo0905/overlay2
Automatic merge from submit-queue (batch tested with PRs 51224, 51191, 51158, 50669, 51222)

Enable overlay2 on cos-m60 in node e2e tests

Ref: https://github.com/kubernetes/kubernetes/issues/42926

- Restart docker with `-s overlay2` in cloud-init before running all node e2e tests. I have to copy the systemd unit file to `/etc/systemd/system` because the `/usr/lib/systemd/system/` is read only.
- Updated node e2e tests to use the new cos-m60 image.
- The name of the cloud init file (`cos-init-live-restore.yaml`) does not indicate overlay2 will be enabled, but I can't just change the name in this PR, since it's referenced in test-infra.

**Release note**:

```
None
```

/assign @Random-Liu
2017-08-24 22:59:33 -07:00
Kubernetes Submit Queue ce3e2d9b10 Merge pull request #51224 from enisoc/sts-deflake-restart
Automatic merge from submit-queue (batch tested with PRs 51224, 51191, 51158, 50669, 51222)

StatefulSet: Deflake e2e "restart" phase.

This addresses another source of flakiness found while investigating #48031.

The test used to scale the StatefulSet down to 0, wait for ListPods to return 0 matching Pods, and then scale the StatefulSet back up.

This was prone to a race in which StatefulSet was told to scale back up before it had observed its own deletion of the last Pod, as evidenced by logs showing the creation of Pod ss-1 prior to the creation of the replacement Pod ss-0.

Instead, we now wait for the controller to observe all deletions before scaling it back up. This should fix flakes of the form:

```
Too many pods scheduled, expected 1 got 2
```
2017-08-24 22:59:28 -07:00
xiangpengzhao 8719b4a8ea Remove deprecated init-container in annotations 2017-08-25 13:39:29 +08:00
Clayton Coleman b16af45454 Revert "Skip "Simple pod should support exec through kubectl proxy" test"
This reverts commit 3bc6ceac38.
2017-08-24 20:18:13 -04:00
Chao Xu 4928c8d1bf let resourcequota evaluator handle uninitialid pod and pvc 2017-08-24 14:50:03 -07:00
Anthony Yeh 05d6c8a6c2
StatefulSet: Deflake e2e `kubectl exec` commands.
We seem to get a lot of flakes due to "connection refused" while running
`kubectl exec`. I can't find any reason this would be caused by the test
flow, so I'm adding retries to see if that helps.
2017-08-24 11:42:05 -07:00
Chao Xu fcd646d80e Let the initializer admission plugin set the metadata.intializers to nil
if an update makes the pendings and the result both nil
2017-08-24 11:23:51 -07:00
Shyam Jeedigunta b374416807 AllowedNotReadyNodes allowed to be not ready for absolutely *any* reason 2017-08-24 19:39:26 +02:00
Huamin Chen 4525446af2 azure file volume: add secret namespace api
Signed-off-by: Huamin Chen <hchen@redhat.com>
2017-08-24 14:49:58 +00:00
Kubernetes Submit Queue 55a20bb901 Merge pull request #51206 from yguo0905/update-cos
Automatic merge from submit-queue (batch tested with PRs 47115, 51196, 51204, 51208, 51206)

Update cos-m61 image in benchmark tests

Ref: https://github.com/kubernetes/kubernetes/issues/51205

**Release note**:
```
None
```
2017-08-24 07:20:16 -07:00
Beata Skiba 6e08007ce1 Distribute pods efficiently in CA scalability tests
Instead of using runReplicatedPodOnEachNode method
which is suited to a small number of nodes,
distribute pods on the nodes with desired load
using RCs that eat up all the space we want to be
empty after distribution.
2017-08-24 15:21:32 +02:00
m1093782566 b8edd9b885 fix e2e network wrong output message 2017-08-24 19:39:42 +08:00
Kubernetes Submit Queue ce3b118959 Merge pull request #42689 from intelsdi-x/enable-oir-e2e
Automatic merge from submit-queue (batch tested with PRs 51193, 51154, 42689, 51189, 51200)

Re-enable OIR e2e tests.

Re-enabling test skeleton for opaque integer resources originally submitted as part of #41870. The e2e was disabled since it was flaky. This is the first step toward re-enabling them. Currently all cases are skipped, so this exercises only the BeforeEach behavior and the deferred removal of OIRs from a node.

cc @timothysc
2017-08-24 04:38:07 -07:00
Kubernetes Submit Queue db928095a0 Merge pull request #50947 from shyamjvs/clusterIpRange-ginkgo
Automatic merge from submit-queue (batch tested with PRs 51108, 51035, 50539, 51160, 50947)

Auto-calculate CLUSTER_IP_RANGE based on cluster size

In preparation for eliminating CLUSTER_IP_RANGE env var from job configs, making it less error prone while folks try to start their own large cluster tests (https://github.com/kubernetes/kubernetes/issues/50907).

/cc @kubernetes/sig-scalability-misc @wojtek-t @gmarek
2017-08-24 02:32:14 -07:00
xiangpengzhao 3a12acee59 Removes redundant prefix in cluster-lifecycle e2e test names 2017-08-24 16:40:05 +08:00
Kubernetes Submit Queue 14cc8cdfa4 Merge pull request #50397 from bdbauer/statefulTesting
Automatic merge from submit-queue (batch tested with PRs 51113, 46597, 50397, 51052, 51166)

Add statefulset upgrade tests to cluster_upgrade

**What this PR does / why we need it**:
Adds already created statefulset upgrade tests to cluster_upgrade.go. With further test infra changes, this will allow them to be continuously run, giving better signals.

Detect and prevent issues like https://github.com/kubernetes/kubernetes/issues/48327

**Release note**:

```release-note
NONE
```
2017-08-23 23:16:30 -07:00
Kubernetes Submit Queue c041567b5a Merge pull request #46597 from dixudx/implement_proposal_34058
Automatic merge from submit-queue (batch tested with PRs 51113, 46597, 50397, 51052, 51166)

implement proposal 34058: hostPath volume type

**What this PR does / why we need it**:
implement proposal #34058

**Which issue this PR fixes** : fixes #46549

**Special notes for your reviewer**:
cc @thockin @luxas @euank PTAL
2017-08-23 23:16:27 -07:00
Kubernetes Submit Queue 3b2e403a37 Merge pull request #51011 from xilabao/rbac-v1-in-yaml
Automatic merge from submit-queue (batch tested with PRs 50489, 51070, 51011, 51022, 51141)

update to rbac v1 in yaml file

**What this PR does / why we need it**:
ref to https://github.com/kubernetes/kubernetes/pull/49642
ref https://github.com/kubernetes/features/issues/2

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:
cc @liggitt 

**Release note**:

```release-note
NONE
```
2017-08-23 19:54:28 -07:00
Kubernetes Submit Queue ea3a8a7570 Merge pull request #51047 from apelisse/remove-gke-test
Automatic merge from submit-queue

Skip "Simple pod should support exec through kubectl proxy" test

As reported in https://github.com/kubernetes/kubernetes/issues/50466,
this test doesn't work in GKE because it uses a bearer token and the feature only works with client certs.

As the feature that is broken in GKE is new and didn't work before, it
is safe to juste ignore the test and consider the feature as "still not
working" in GKE.

**What this PR does / why we need it**: Fixes the broken test in https://k8s-testgrid.appspot.com/release-master-blocking#gke

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: works-around #50466

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```
2017-08-23 17:41:58 -07:00
Anthony Yeh ce3fad326f
StatefulSet: Deflake e2e "restart" phase.
The test used to scale the StatefulSet down to 0, wait for ListPods to
return 0 matching Pods, and then scale the StatefulSet back up.

This was prone to a race in which StatefulSet was told to scale back up
before it had observed its own deletion of the last Pod, as evidenced by
logs showing the creation of Pod ss-1 prior to the creation of the
replacement Pod ss-0.

We now wait for the controller to observe all deletions before
scaling it back up. This should fix flakes of the form:

```
Too many pods scheduled, expected 1 got 2
```
2017-08-23 15:08:58 -07:00
Yang Guo a1c5c14eff Update cos-m61 image in benchmark tests 2017-08-23 09:30:20 -07:00
Kubernetes Submit Queue 178a5ff314 Merge pull request #50665 from xiangpengzhao/hardcode-to-const
Automatic merge from submit-queue (batch tested with PRs 50257, 50247, 50665, 50554, 51077)

Replace hard-code "cpu" and "memory" to consts

**What this PR does / why we need it**:
There are many places using hard coded "cpu" and "memory" as resource name. This PR replace them to consts.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:
/kind cleanup

**Release note**:

```release-note
NONE
```
2017-08-23 02:35:09 -07:00
Kubernetes Submit Queue 172f05bc53 Merge pull request #46902 from thockin/remove-obsolete-bins
Automatic merge from submit-queue (batch tested with PRs 50980, 46902, 51051, 51062, 51020)

Remove seemingly obsolete binaries

It's hard to tell if these are safe to remove.  Let CI tell me.
2017-08-22 23:13:59 -07:00
Di Xu 6f74af94ef update e2e tests and yaml files 2017-08-23 14:05:21 +08:00
Kubernetes Submit Queue 49c36f4b33 Merge pull request #50546 from apelisse/plumb-openapi-validation
Automatic merge from submit-queue (batch tested with PRs 51039, 50512, 50546, 50965, 50467)

Kubectl: Plumb openapi validation (disabled by default)

**What this PR does / why we need it**: Creates a new flag '--openapi' and plumb in the validation code so that it can be used by default to validate objects against the openapi schema.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: partially https://github.com/kubernetes/kubectl/issues/49

**Special notes for your reviewer**:

This is not complete, the name of the variable must change for example.

**Release note**:
```release-note
Kubectl uses openapi for validation. If OpenAPI is not available on the server, it defaults back to the old Swagger.
```
2017-08-22 21:16:11 -07:00
Kubernetes Submit Queue a44e538dbc Merge pull request #51039 from enisoc/deflake-sts-saturate
Automatic merge from submit-queue

StatefulSet: Deflake e2e "Saturate" phase.

This should reduce one source of flakiness found while investigating #48031.

The "Saturate" phase of StatefulSet e2e tests verifies orderly startup by controlling when each Pod is allowed to report Ready. If a Pod unexepectedly goes down during the test, the replacement Pod
created by the controller will forget if it was already allowed to report Ready.

After this change, the signal that allows each Pod to report Ready is persisted in the Pod's PVC. Thus, the replacement Pod will remember that it was already told to proceed to a Ready state.
2017-08-22 21:13:13 -07:00
Yang Guo 755ce10e9b Enable overlay2 on cos-m60 in node e2e tests 2017-08-22 17:08:52 -07:00
Kubernetes Submit Queue 36b5e0eca6 Merge pull request #51037 from MrHohn/sig-network-e2e-fix-describe
Automatic merge from submit-queue (batch tested with PRs 51102, 50712, 51037, 51044, 51059)

[sig-network-e2e] Remove redundant sig prefix from tests

**What this PR does / why we need it**:
Remove redundant sig prefix from:
```
[sig-network] Networking [sig-network] Granular Checks: Services [Slow] should function for endpoint-Service: http
[sig-network] Networking [sig-network] Granular Checks: Services [Slow] should function for endpoint-Service: udp
[sig-network] Networking [sig-network] Granular Checks: Services [Slow] should function for node-Service: http
[sig-network] Networking [sig-network] Granular Checks: Services [Slow] should function for node-Service: udp
[sig-network] Networking [sig-network] Granular Checks: Services [Slow] should function for pod-Service: http
[sig-network] Networking [sig-network] Granular Checks: Services [Slow] should function for pod-Service: udp
[sig-network] Networking [sig-network] Granular Checks: Services [Slow] should update endpoints: http
[sig-network] Networking [sig-network] Granular Checks: Services [Slow] should update endpoints: udp
[sig-network] Networking [sig-network] Granular Checks: Services [Slow] should update nodePort: http [Slow]
[sig-network] Networking [sig-network] Granular Checks: Services [Slow] should update nodePort: udp [Slow]
[sig-network] Loadbalancing: L7 [sig-network] GCE [Slow] [Feature:Ingress] should conform to Ingress spec
[sig-network] Loadbalancing: L7 [sig-network] GCE [Slow] [Feature:Ingress] should create ingress with given static-ip
```

Umbrella issue #49161

**Special notes for your reviewer**:
cc @xiangpengzhao 

**Release note**:

```release-note
NONE
```
2017-08-22 12:28:02 -07:00
Kubernetes Submit Queue 622bc55598 Merge pull request #51028 from ironcladlou/gc-int-flake
Automatic merge from submit-queue (batch tested with PRs 50967, 50505, 50706, 51033, 51028)

Fix GC integration test race

During TestCreateWithNonExistentOwner, when creating a pod with a
non-existent owner, assume it's possible the pod will be deleted before
we start checking for the pod's existence. Assuming that the pod still
exists immediately after Create returns is flaky if the GC reacts very
quickly.

```release-note
NONE
```

Might fix https://github.com/kubernetes/kubernetes/issues/50943; without the additional test context provided by this PR, it's not entirely possible to assess the root cause of the reported failure (as we don't know whether the original assertion failure was due to there being 0 or >1 pods).

/cc @caesarxuchao
2017-08-22 10:48:26 -07:00
Kubernetes Submit Queue c6980e7247 Merge pull request #51033 from mtaufen/revert-51008-revert-50789-fix-scheme
Automatic merge from submit-queue (batch tested with PRs 50967, 50505, 50706, 51033, 51028)

Revert "Merge pull request #51008 from kubernetes/revert-50789-fix-scheme"

I'm spinning up a cluster right now to test this fix, but I'm pretty sure this was the problem.
There doesn't seem to be a way to confirm from logs, because AFAICT the logs from the hollow kubelet containers are not collected as part of the kubemark test.

**What this PR does / why we need it**:

This reverts commit f4afdecef8, reversing
changes made to e633a1604f.

This also fixes a bug where Kubemark was still using the core api scheme
to manipulate the Kubelet's types, which was the cause of the initial
revert.

**Which issue this PR fixes**: fixes #51007

**Release note**:

```release-note
NONE
```

/cc @shyamjvs @wojtek-t
2017-08-22 10:48:21 -07:00
Antoine Pelisse 3bc6ceac38 Skip "Simple pod should support exec through kubectl proxy" test
As reported in https://github.com/kubernetes/kubernetes/issues/50466,
this test doesn't work in GKE because the transport layer doesn't work
with dialing.

As the feature that is broken in GKE is new and didn't work before, it
is safe to juste ignore the test and consider the feature as "still not
working" in GKE.
2017-08-22 10:30:16 -07:00
Kubernetes Submit Queue c61468f29b Merge pull request #51091 from resouer/fix-perf
Automatic merge from submit-queue

Should generate files before scheduler perf

**What this PR does / why we need it**:
For a newly cloned project,  generated files are not included. Then scheduler_perf will fail:
```
undefined: openapi.GetOpenAPIDefinitions
```

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: 

fixes: #51090

**Special notes for your reviewer**:
2017-08-22 08:28:26 -07:00
Kubernetes Submit Queue fdf14b8218 Merge pull request #50913 from shyamjvs/list-call-slo
Automatic merge from submit-queue (batch tested with PRs 50893, 50913, 50963, 50629, 50640)

Increase latency threshold for list api calls

This is only a short-term solution to make our density test green. In the long-term, we should measure as per our new SLIs.
From @wojtek-t's [doc](https://docs.google.com/document/d/1Q5qxdeBPgTTIXZxdsFILg7kgqWhvOwY8uROEf0j5YBw) on the new SLIs/SLOs, we have the following SLO for list calls:

```
SLO1: In default Kubernetes installation, 99th percentile of SLI2 per cluster-day:
<= 1s if total number of objects of the same type as resource in the system <= X
<= 5s if total number of objects of the same type as resource in the system <= Y
<= 30s if total number of objects of the same types as resource in the system <= Z
```

I would guess that 170,000 pods would fall into the 2nd bracket (at least) and hence the new value of 5s. WDYT?

cc @kubernetes/sig-scalability-misc @wojtek-t @gmarek
2017-08-22 05:31:07 -07:00
Harry Zhang 388e0b39bf generate files before scheduler perf 2017-08-22 16:40:16 +08:00
Kubernetes Submit Queue cb8ade18c6 Merge pull request #50950 from k82cn/revert_50360
Automatic merge from submit-queue

Revert #50362.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: part of #50884

**Release note**:

```release-note
None
```
2017-08-21 16:50:53 -07:00
Kubernetes Submit Queue 0867802bbc Merge pull request #50831 from Random-Liu/instance-metadata-from-flag
Automatic merge from submit-queue (batch tested with PRs 50693, 50831, 47506, 49119, 50871)

Add instance metadata from flag even when using image config.

Also add instance metadata from flag even when we are using image config.

* Sometimes we need to dynamically generate instance metadata, it's troublesome to put them into image config.
* Sometimes we want to apply instance metadata to all images, it's duplicated to add them to each image in the image config.

/assign @yguo0905 Could you help me review this?
2017-08-21 14:29:57 -07:00
Anthony Yeh 3bc7676024
StatefulSet: Deflake e2e "Saturate" phase.
The "Saturate" phase of StatefulSet e2e tests verifies orderly startup
by controlling when each Pod is allowed to report Ready.
If a Pod unexepectedly goes down during the test, the replacement Pod
created by the controller will forget if it was already allowed to
report Ready.

After this change, the signal that allows each Pod to report Ready is
persisted in the Pod's PVC. Thus, the replacement Pod will remember that
it was already told to proceed to a Ready state.
2017-08-21 13:52:15 -07:00
Michael Taufen a90d81620b Revert "Merge pull request #51008 from kubernetes/revert-50789-fix-scheme"
This reverts commit f4afdecef8, reversing
changes made to e633a1604f.

This also fixes a bug where Kubemark was still using the core api scheme
to manipulate the Kubelet's types, which was the cause of the initial
revert.
2017-08-21 11:28:05 -07:00