Commit Graph

5475 Commits (b4235038b312fff7d67d01fa0c2bfdc81b1ce21f)

Author SHA1 Message Date
Jeff Grafton 511bdc11ae Bump etcd-empty-dir-cleanup to 3.0.14.0 2017-02-22 13:22:04 -08:00
Jeff Grafton 1f3ba7f484 Base etcd-empty-dir-cleanup on busybox, run as nobody, and update to etcdctl 3.0.14 2017-02-22 13:22:03 -08:00
Jeff Grafton 1e7b589977 Bump gcr.io/google-containers/rescheduler to v0.2.2 2017-02-22 10:42:16 -08:00
Charles Butler 3c5009d00a Remove ivan4th from reviewers
Per ivans request in #41351 he would like to be removed from the
reviewers list in this directory tree. This commit addresses that
request.
2017-02-22 12:06:00 -06:00
Charles Butler 4408b82a90 Fixes for #41019
This branch sync's the crypto keys, and flat-files used for auth with
all the masters when scaling the kubernetes-master units. This should
fix the mis-matched crypto keys seen when rebooting units after first
deploy.
2017-02-22 10:20:51 -06:00
George Kraft d697b10885 Fix UpdateAddonsTactic to use local repo, add KUBE_VERSION option 2017-02-22 10:19:05 -06:00
Kubernetes Submit Queue 44aa1679c9 Merge pull request #41657 from bowei/update-dns
Automatic merge from submit-queue (batch tested with PRs 41349, 41532, 41256, 41587, 41657)

Update dns

```release-note
NONE
```
2017-02-22 08:12:48 -08:00
Kubernetes Submit Queue fe34705f8a Merge pull request #41587 from MrHohn/addon-manager-fix-hpa
Automatic merge from submit-queue (batch tested with PRs 41349, 41532, 41256, 41587, 41657)

Update kubectl in addon-manager to use HPA in autoscaling/v1

Addon-manager is broken since HPA objects were removed from extensions api group.

Came across the logs from [the latest addon-manager on Jenkins](https://storage.googleapis.com/kubernetes-jenkins/logs/ci-kubernetes-e2e-gci-gce/4290/artifacts/bootstrap-e2e-master/kube-addon-manager.log):
```
INFO: == Entering periodical apply loop at 2017-02-16T17:33:37+0000 ==
error: error pruning namespaced object extensions/v1beta1, Kind=HorizontalPodAutoscaler: the server could not find the requested resource
WRN: == Failed to execute /usr/local/bin/kubectl  apply --namespace=kube-system -f /etc/kubernetes/addons     --prune=true -l kubernetes.io/cluster-service=true --recursive >/dev/null at 2017-02-16T17:33:38+0000. 2 tries remaining. ==
error: error pruning namespaced object extensions/v1beta1, Kind=HorizontalPodAutoscaler: the server could not find the requested resource
WRN: == Failed to execute /usr/local/bin/kubectl  apply --namespace=kube-system -f /etc/kubernetes/addons     --prune=true -l kubernetes.io/cluster-service=true --recursive >/dev/null at 2017-02-16T17:33:46+0000. 1 tries remaining. ==
error: error pruning namespaced object extensions/v1beta1, Kind=HorizontalPodAutoscaler: the server could not find the requested resource
WRN: == Failed to execute /usr/local/bin/kubectl  apply --namespace=kube-system -f /etc/kubernetes/addons     --prune=true -l kubernetes.io/cluster-service=true --recursive >/dev/null at 2017-02-16T17:33:53+0000. 0 tries remaining. ==
WRN: == Kubernetes addon update completed with errors at 2017-02-16T17:33:58+0000 ==
```

And notice this commit (f66679a4e9) came in two weeks ago, which removed HorizontalPodAutoscaler from extensions/v1beta1.

Addon-manager is now partially functioning that it could successfully create and update addons, but will fail to prune objects, which means upgrade tests may mostly fail.

Pushed another version of addon-manager with kubectl v1.6.0-alpha.2 ([release 2 days ago](https://github.com/kubernetes/kubernetes/releases/tag/v1.6.0-alpha.2)) for fixing, including below images:
- gcr.io/google-containers/kube-addon-manager:v6.4-alpha.2
- gcr.io/google-containers/kube-addon-manager-amd64:v6.4-alpha.2
- gcr.io/google-containers/kube-addon-manager-arm:v6.4-alpha.2
- gcr.io/google-containers/kube-addon-manager-arm64:v6.4-alpha.2
- gcr.io/google-containers/kube-addon-manager-ppc64le:v6.4-alpha.2
- gcr.io/google-containers/kube-addon-manager-s390x:v6.4-alpha.2

@mikedanese 

cc @wojtek-t @shyamjvs
2017-02-22 08:12:46 -08:00
Kubernetes Submit Queue b29bdee735 Merge pull request #41256 from mbruzek/mbruzek-juju-lint-fixes
Automatic merge from submit-queue (batch tested with PRs 41349, 41532, 41256, 41587, 41657)

Lint fixes for the master and worker Python code.

**What this PR does / why we need it**: lint fixes for the python code.

**Which issue this PR fixes** none

**Special notes for your reviewer**: This is lint fixes for the Juju python code.

**Release note**:

```release-note
NONE
```

Please consider these changes so we can pass flake8 lint tests in our build process.
2017-02-22 08:12:43 -08:00
Shyam Jeedigunta d5a28b3618 Remove unnecessary metrics (http/process/go) from being exposed by etcd-version-monitor 2017-02-22 13:11:00 +01:00
Ritesh H Shukla fa3b8d0b3b Default storage class for vSphere Fixes: 40070 2017-02-22 00:27:14 +00:00
Christoph Blecker c3de31c8d0
Bump golang versions to 1.7.5 2017-02-21 13:02:16 -08:00
Madhusudan.C.S 2cb2200847 Move kube-dns ConfigMap creation/deletion out of federated services e2e tests to federation-up.sh/federation-down.sh where the clusters are joined/unjoined. 2017-02-21 10:27:31 -08:00
Shyam JVS 746cc5d284 Merge pull request #41800 from shyamjvs/fix-hollow-node-logging
Whitelist kubemark in node_ssh_supported_providers for log dump
2017-02-21 19:13:08 +01:00
Dawn Chen 3d510461a3 Bump GCI to gci-stable-56-9000-84-2 2017-02-21 10:03:14 -08:00
Kevin cd427fa4be enable DefaultTolerationSeconds admission controller by default 2017-02-22 00:45:56 +08:00
Kubernetes Submit Queue 409d7d0a91 Merge pull request #41326 from ncdc/ci-cache-mutation
Automatic merge from submit-queue (batch tested with PRs 41364, 40317, 41326, 41783, 41782)

Add ability to enable cache mutation detector in GCE

Add the ability to enable the cache mutation detector in GCE. The current default behavior (disabled) is retained.

When paired with https://github.com/kubernetes/test-infra/pull/1901, we'll be able to detect shared informer cache mutations in gce e2e PR jobs.
2017-02-21 07:45:42 -08:00
Shyam Jeedigunta 3bc6bf6b70 Whitelist kubemark in node_ssh_supported_providers for log dump 2017-02-21 14:02:17 +01:00
Mik Vyatskov 5d59d4d27b Turn fluentd supervisor off for fluentd-gcp 2017-02-21 13:50:47 +01:00
Kubernetes Submit Queue 70c9eebd21 Merge pull request #41739 from shyamjvs/hollow-node-logs
Automatic merge from submit-queue (batch tested with PRs 41706, 39063, 41330, 41739, 41576)

[Kubemark] Add option to log hollow-node logs

Ref https://github.com/kubernetes/kubernetes/issues/41613

Added an option to log kubemark hollow-node logs which includes kubelet, kubeproxy and npd logs for each hollow-node.
Setting the env var `ENABLE_HOLLOW_NODE_LOGS=true` should now enable logging for tests.

cc @kubernetes/sig-scalability-misc @wojtek-t @gmarek @yujuhong @Random-Liu
2017-02-21 02:24:43 -08:00
Zihong Zheng 2c8e89820a Update kubectl in addon-manager to use HPA in autoscaling/v1 instead of extensions/v1beta1 2017-02-20 10:49:10 -08:00
deads2k 36b586d5d7 move kube-dns to a separate service account 2017-02-20 07:35:08 -05:00
Shyam Jeedigunta ed0ab3cd8e [Kubemark] Add option to log hollow-node logs 2017-02-20 11:52:49 +01:00
Vishnu kannan 6438efeeda protect kubeproxy from system OOMs until it can be moved to Guaranteed QoS Class
Signed-off-by: Vishnu kannan <vishnuk@google.com>
2017-02-18 18:46:50 -08:00
Kubernetes Submit Queue ff12e5688c Merge pull request #40206 from Random-Liu/add-standalone-npd
Automatic merge from submit-queue

Add standalone npd on GCI.

This PR added standalone NPD in GCE GCI cluster. I already verified the PR, and it should work.

/cc @dchen1107 @fabioy @andyxning @kubernetes/sig-node-misc
2017-02-18 02:00:20 -08:00
Kubernetes Submit Queue 4b3a097ecd Merge pull request #41525 from yujuhong/fix_output
Automatic merge from submit-queue

Fix the output of health-mointor.sh

The script show prints the errors/response of the health check, but not
show the progress of `curl`.
2017-02-17 16:57:29 -08:00
Random-Liu d40c0a7099 Add standalone npd on GCI. 2017-02-17 16:18:08 -08:00
Bowei Du 9f75db3c69 Update kube-dns image versions to the latest stable release 2017-02-17 11:12:25 -08:00
Kubernetes Submit Queue 6d5b2ef49e Merge pull request #41080 from shyamjvs/etcd-version-monitor
Automatic merge from submit-queue

Added a basic monitor for providing etcd version related info

Fixes #41071 

This tool scrapes metrics partly from etcd's /version and /metrics endpoints and partly using etcdctl and exposes them as prometheus metrics at `http://localhost:9101/metrics` endpoint on the master. Here is a summary of the metrics it exposes (self-explanatory from the code):
-        etcdVersionFetchCount   = prometheus.NewCounterVec(
                prometheus.CounterOpts{
                        Namespace: "etcd",
                        Name: "version_info_fetch_count",
                        Help: "Number of times etcd's version info was fetched, labeled by etcd's server binary and cluster version",
                },
                []string{"serverversion", "clusterversion"})
-         etcdGRPCRequestsTotal   = prometheus.NewCounterVec(
                prometheus.CounterOpts{
                        Namespace: namespace,
                        Name: "grpc_requests_total",
                        Help: "Counter of received grpc requests, labeled by grpc method and grpc service names",
                },
                []string{"grpc_method", "grpc_service"})

For further info on how to run this as a binary/docker-container/kubernetes-pod and checking the metrics, have a look at the README.md file.

cc @fgrzadkowski @wojtek-t @piosz
2017-02-17 10:18:48 -08:00
Kubernetes Submit Queue 46cd8ec91b Merge pull request #41637 from wojtek-t/expose_storage_format_as_env
Automatic merge from submit-queue

Expose storage media type as env variable

Ref #40636

@mml
2017-02-17 08:15:27 -08:00
Andy Goldstein 688c19ec71 Allow cache mutation detector enablement by PRs
Allow cache mutation detector enablement by PRs in an attempt to find
mutations before they're merged in to the code base. It's just for the
apiserver and controller-manager for now. If/when the other components
start using a SharedInformerFactory, we should set them up just like
this as well.
2017-02-17 10:03:13 -05:00
Kubernetes Submit Queue 3b14667afe Merge pull request #41604 from shyamjvs/kubemark-num-nodes
Automatic merge from submit-queue

Reduce default value of kubemark's NUM_NODES to 10

Changing the default value of kubemark's NUM_NODES from 100 to 10, as it would then be possible to start kubemark on gce clusters that have been started using kube-up that uses the default config of three n1-standard-2 nodes. I've already been asked by a couple of people about why kubemark is not starting on their cluster because of this. More people shouldn't be facing this issue in future.

cc @kubernetes/sig-scalability-misc @wojtek-t @gmarek
2017-02-17 06:49:21 -08:00
Wojciech Tyczynski 3695e85b34 Expose storage media type as env variable 2017-02-17 14:16:55 +01:00
Shyam Jeedigunta 7e6b8ac26b Added a basic monitor for watching etcd version and size related info 2017-02-17 12:52:54 +01:00
Shyam Jeedigunta 94d2ed5e34 Reduce default value of kubemark's NUM_NODES to 10 2017-02-16 23:35:39 +01:00
Matt Bruzek 3b29b6a9ef Lint fixes for the master and worker Python code. 2017-02-16 14:01:30 -06:00
Mik Vyatskov 8d2d91070a Add fluentd monitoring to fluentd-gcp image 2017-02-16 19:04:32 +01:00
Kubernetes Submit Queue 30e8953fad Merge pull request #41564 from Crassirostris/fluentd-gcp-plugin-version-bump
Automatic merge from submit-queue

Bump fluentd-gcp google_cloud plugin version

Bump the version of `fluent-plugin-google-cloud` in fluentd-gcp image, because it's broken for version `0.5.2`.

Recently, gem `google-api-client` was updated to version `0.10.0`. The new version broke `fluent-plugin-google-cloud` which doesn't specify the upper version of `google-api-client` gem. I'm bumping the version used in our image to allow future changes in this release to be run and tested.

This PR doesn't bump the version, since no effective changes has happened, leaving this for the next PR to do.

CC @igorpeshansky
2017-02-16 09:20:12 -08:00
Mik Vyatskov e8de31623f Bump fluentd-gcp google_cloud plugin version 2017-02-16 16:49:16 +01:00
Kubernetes Submit Queue 627c6ce2b8 Merge pull request #41489 from Crassirostris/fluentd-add-toleration
Automatic merge from submit-queue (batch tested with PRs 40000, 41508, 41489)

Add toleration to fluentd daemonset to make it run on master

Because of https://github.com/kubernetes/kubernetes/pull/41172 fluentd pods stopped being allocated on master node.

This PR introduces toleration for master taint for fluentd.

CC @davidopp @janetkuo @kubernetes/sig-scheduling-bugs

Unfortunately, we don't have e2e tests to ensure that master logs are being ingested. This problem is a great signal to work on https://github.com/kubernetes/kubernetes/issues/41411
2017-02-16 01:52:08 -08:00
Kubernetes Submit Queue 5ff9a72ea0 Merge pull request #41508 from Crassirostris/fluentd-dns-problem-fix
Automatic merge from submit-queue (batch tested with PRs 40000, 41508, 41489)

Make fluentd use default dns instead of cluster dns to make it work o…

Fix https://github.com/kubernetes/kubernetes/issues/41415

Fluentd for Stackdriver requires external urls (e.g. `logging.googleapis.com`) to be available in order to work. If fluentd runs on master, it cannot access the service endpoint of cluster DNS. This change makes fluentd use default dns to fix this problem.

CC @thockin @bowei
2017-02-16 01:52:06 -08:00
Yu-Ju Hong d3e24e1085 Fix the output of health-mointor.sh
The script show prints the errors/response of the health check, but not
show the progress of `curl`.
2017-02-15 18:08:27 -08:00
Kubernetes Submit Queue 01393e34d6 Merge pull request #40722 from micmro/40721
Automatic merge from submit-queue (batch tested with PRs 41104, 41245, 40722, 41439, 41502)

openstack-heat: do not daemonize salt-minion

_openstack-heat_ does currently not setup a _salt-master_, so it is not necessary to  daemonize it.

**What this PR does / why we need it**:
as stated in #40721:

> The _openstack-heat_ provider only installs _salt-minions_, no _salt-master_. The configuration does not take this into account which causes the following issues:
> 
> - the _salt minion_ is not able to DNS resolve `salt` (see fist part of error log below)
> - the _salt-minion_ is daemonized and fails finding the master (second part of error log below). From my understanding is not required when there is no salt-master, as the setup uses `salt-call` 
> anyway (see [gce provider](https://github.com/kubernetes/kubernetes/blob/master/cluster/gce/configure-vm.sh#L328-L339) as reference).
> 
> ```
> Jan 31 03:00:04 kube-stack-master salt-minion[9795]: [ERROR   ] DNS lookup of 'salt' failed.
> Jan 31 03:00:04 kube-stack-master salt-minion[9795]: [ERROR   ] Master hostname: 'salt' not found. Retrying in 30 seconds
> ...
> Jan 31 02:35:30 kube-stack-master salt-minion[9690]: [ERROR   ] Error while bringing up minion for multi-master. Is master at salt responding?
> ```
> 

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #40721

**Release note**:
```release-note
Do not daemonize `salt-minion` for the openstack-heat provider.
```
2017-02-15 16:28:09 -08:00
Kubernetes Submit Queue e62866444f Merge pull request #41245 from wojtek-t/rollback_2_2_1
Automatic merge from submit-queue (batch tested with PRs 41104, 41245, 40722, 41439, 41502)

Change the etcd rollback tool to do rollback to 2.2.1 version.

I did some tests of it and for my 3-node cluster with 1 deployment it worked fine.

But before merging this, we should probably do way more testing (we should rerun tests that @mml was doing for the previous script).

@lavalamp @xiang90
2017-02-15 16:28:08 -08:00
Kubernetes Submit Queue 1fc1e5efb5 Merge pull request #41395 from gmarek/owners
Automatic merge from submit-queue

Add gmarek and jszczepkowski to cluster/gce owners

cc @mikedanese @zmerlynn @roberthbailey
2017-02-15 12:45:39 -08:00
Mik Vyatskov f6730bd334 Make fluentd use default dns instead of cluster dns to make it work on master 2017-02-15 20:53:32 +01:00
Kubernetes Submit Queue 33aedca59d Merge pull request #41332 from jszczepkowski/etcd-cluster-state-16
Automatic merge from submit-queue

Added configurable etcd initial-cluster-state to kube-up script.

Added configurable etcd initial-cluster-state to kube-up script. This
allows creation of multi-master cluster from scratch. This is a
cherry-pick of #41320 from 1.5 branch.

```release-note
Added configurable etcd initial-cluster-state to kube-up script.
```
2017-02-15 10:04:31 -08:00
Mik Vyatskov cbba60cc7d Add toleration to fluentd daemonset to make it run on master 2017-02-15 18:11:45 +01:00
Mik Vyatskov cabb9890ac Use docker log rotation mechanism instead of logrotate 2017-02-15 16:22:29 +01:00
Kubernetes Submit Queue 5cc2f73bc9 Merge pull request #41134 from shyamjvs/refactor-final-blow
Automatic merge from submit-queue (batch tested with PRs 41134, 41410, 40177, 41049, 41313)

Refactored kubemark code into provider-specific and provider-independent parts [Part-3]

Fixes #38967
Applying final part of the changes in PR #39033 (which refactored kubemark code completely). The changes included in this PR are:

- Removed `test/kubemark/common.sh` and moved relevant parts of its code to the right places in start-kubemark/stop-kubemark scripts.
- Added DOCKER_REGISTRY, PROJECT, KUBEMARK_IMAGE_MAKE_TARGET variables to `/test/kubemark/cloud-provider-config.sh` to make the kubemark image push location variable wrt provider.
- Removed get-real-pod-for-hollow-node.sh as it doesn't seem to do anything useful.

@kubernetes/sig-scalability-misc @wojtek-t @gmarek
2017-02-15 05:58:15 -08:00
Kubernetes Submit Queue 80be6a259f Merge pull request #41430 from mikedanese/preserve-key
Automatic merge from submit-queue (batch tested with PRs 41360, 41423, 41430, 40647, 41352)

preserve kube-master-cert metadata over upgrades
2017-02-15 05:06:10 -08:00
Kubernetes Submit Queue 2fde8f8efe Merge pull request #41360 from enisoc/fluentd-audit-log
Automatic merge from submit-queue

fluentd-gcp: Add kube-apiserver-audit.log.

**What this PR does / why we need it**:

Add `kube-apiserver-audit.log` from https://github.com/kubernetes/kubernetes/pull/41211 to fluentd config, so the audit log gets sent to the same place as `kube-apiserver.log`.

**Which issue this PR fixes**:

**Special notes for your reviewer**:

We would like to backport this to release-1.5 also.

**Release note**:
```release-note
The apiserver audit log (`/var/log/kube-apiserver-audit.log`) will be sent through fluentd if enabled.
```
2017-02-15 05:01:54 -08:00
Kubernetes Submit Queue 0e86d98f99 Merge pull request #41449 from zmerlynn/container-vm-v20170214
Automatic merge from submit-queue (batch tested with PRs 41196, 41252, 41300, 39179, 41449)

Bump GCE ContainerVM to container-vm-v20170214

`container-vm-v20170214` is a re-build of the `docker-runc` in `container-vm-v20170201`, and should clear the GCE slow tests.

c.f. #40828

```release-note
Bump GCE ContainerVM to container-vm-v20170214 to address CVE-2016-9962.
```
2017-02-15 04:14:17 -08:00
Kubernetes Submit Queue 4c02f29196 Merge pull request #41211 from enisoc/configure-audit-log
Automatic merge from submit-queue (batch tested with PRs 40297, 41285, 41211, 41243, 39735)

cluster/gce: Add env var to enable apiserver basic audit log.

For now, this is focused on a fixed set of flags that makes the audit
log show up under /var/log/kube-apiserver-audit.log and behave similarly
to /var/log/kube-apiserver.log. Allowing other customization would
require significantly more complex changes.

Audit log rotation is handled the same as for `kube-apiserver.log`.

**What this PR does / why we need it**:

Add a knob to enable [basic audit logging](https://kubernetes.io/docs/admin/audit/) in GCE.

**Which issue this PR fixes**:

**Special notes for your reviewer**:

We would like to cherrypick/port this to release-1.5 also.

**Release note**:
```release-note
The kube-apiserver [basic audit log](https://kubernetes.io/docs/admin/audit/) can be enabled in GCE by exporting the environment variable `ENABLE_APISERVER_BASIC_AUDIT=true` before running `cluster/kube-up.sh`. This will log to `/var/log/kube-apiserver-audit.log` and use the same `logrotate` settings as `/var/log/kube-apiserver.log`.
```
2017-02-15 03:25:12 -08:00
Kubernetes Submit Queue e4a4fe4a89 Merge pull request #41285 from liggitt/kube-scheduler-role
Automatic merge from submit-queue (batch tested with PRs 40297, 41285, 41211, 41243, 39735)

Secure kube-scheduler

This PR:
* Adds a bootstrap `system:kube-scheduler` clusterrole
* Adds a bootstrap clusterrolebinding to the `system:kube-scheduler` user
* Sets up a kubeconfig for kube-scheduler on GCE (following the controller-manager pattern)
* Switches kube-scheduler to running with kubeconfig against secured port (salt changes, beware)
* Removes superuser permissions from kube-scheduler in local-up-cluster.sh
* Adds detailed RBAC deny logging

```release-note
On kube-up.sh clusters on GCE, kube-scheduler now contacts the API on the secured port.
```
2017-02-15 03:25:10 -08:00
Michael Mrowetz 11ed1a9565 #40721 openstack-heat: salt-minion not daemonize
openstack-heat does currently not setup a salt-master, so it is not necessary to  daemonize it.
2017-02-15 17:51:07 +09:00
Kubernetes Submit Queue 0a56830520 Merge pull request #41383 from liggitt/v1beta1-cleanup
Automatic merge from submit-queue

Update rbac data to v1beta1

Update RBAC fixtures to v1beta1
2017-02-14 22:35:05 -08:00
Jordan Liggitt cc11d7367a
Switch kube-scheduler to secure API access 2017-02-15 01:05:42 -05:00
Zach Loafman b7229ed565 Bump GCE ContainerVM to container-vm-v20170214
container-vm-v20170214 is a re-build of the docker-runc in
container-vm-v20170201, and should clear the GCE slow tests.

c.f. #40828
2017-02-14 16:36:02 -08:00
Anthony Yeh 7500746e7f cluster/gce: Add env var to enable apiserver basic audit log.
For now, this is focused on a fixed set of flags that makes the audit
log show up under /var/log/kube-apiserver-audit.log and behave similarly
to /var/log/kube-apiserver.log. Allowing other customization would
require significantly more complex changes.

Audit log rotation is handled externally by the wildcard /var/log/*.log
already configured in configure-helper.sh.
2017-02-14 15:18:10 -08:00
Anthony Yeh 257a8745e3 fluentd-gcp: Add kube-apiserver-audit.log. 2017-02-14 14:23:36 -08:00
Kubernetes Submit Queue a48284862c Merge pull request #41407 from Crassirostris/fluentd-gcp-sysmted-fix
Automatic merge from submit-queue (batch tested with PRs 41382, 41407, 41409, 41296, 39636)

Fix copying systemd libraries upon fluentd-gcp startup

Fix https://github.com/kubernetes/kubernetes/issues/40936
Revert https://github.com/kubernetes/kubernetes/pull/40017
2017-02-14 13:04:21 -08:00
Kubernetes Submit Queue 90e1977a1c Merge pull request #41325 from wojtek-t/fix_etcd_migrate
Automatic merge from submit-queue (batch tested with PRs 41299, 41325, 41386, 41329, 41418)

Migrate etcd data using correct etcd version in case of previous crash

Fix #41324
Fix #41323

@mml
2017-02-14 11:42:35 -08:00
Mike Danese e17e4e110e preserve kube-master-cert metadata over upgrades 2017-02-14 11:02:11 -08:00
gmarek e6e1d3066e Add gmarek and jszczepkowski to cluster/gce owners 2017-02-14 17:53:39 +01:00
Fabian Deutsch f6ee79b2ec addonManager: Add note about labeling
The cluster manager is only picking up addons if they are labeled correctly.
2017-02-14 15:43:47 +01:00
Mik Vyatskov a1ec542d7c Fix copying systemd libraries upon fluentd-gcp startup 2017-02-14 15:41:15 +01:00
Wojciech Tyczynski 1ce544db9e Migrate etcd data using correct etcd version in case of previous crash 2017-02-14 11:30:00 +01:00
Jordan Liggitt 9e6a3496b4
Update rbac data to v1beta1 2017-02-14 00:50:31 -05:00
Kubernetes Submit Queue 1f4e2efc5b Merge pull request #41184 from liggitt/subject-apigroup
Automatic merge from submit-queue (batch tested with PRs 41357, 41178, 41280, 41184, 41278)

Switch RBAC subject apiVersion to apiGroup in v1beta1

Referencing a subject from an RBAC role binding, the API group and kind of the subject is needed to fully-qualify the reference.

The version is not, and adds complexity around re-writing the reference when returning the binding from different versions of the API, and when reconciling subjects.

This PR:
* v1beta1: change the subject `apiVersion` field to `apiGroup` (to match roleRef)
* v1alpha1: convert apiVersion to apiGroup for backwards compatibility
* all versions: add defaulting for the three allowed subject kinds
* all versions: add validation to the field so we can count on the data in etcd being good until we decide to relax the apiGroup restriction

```release-note
RBAC `v1beta1` RoleBinding/ClusterRoleBinding subjects changed `apiVersion` to `apiGroup` to fully-qualify a subject. ServiceAccount subjects default to an apiGroup of `""`, User and Group subjects default to an apiGroup of `"rbac.authorization.k8s.io"`.
```

@deads2k @kubernetes/sig-auth-api-reviews @kubernetes/sig-auth-pr-reviews
2017-02-13 21:07:10 -08:00
Bowei Du da291a7beb Send only cluster domain queries to kube-dns
Note: all PTR request must still traverse kube-dns. We can restrict
this to just the clusterCIDR in the future to reduce the amount of
PTR traffic.
2017-02-13 13:27:09 -08:00
Jordan Liggitt 2a76fa1c8f
Switch RBAC subject apiVersion to apiGroup in v1beta1 2017-02-13 15:33:09 -05:00
Jerzy Szczepkowski 80e57b7016 Added configurable etcd initial-cluster-state to kube-up script.
Added configurable etcd initial-cluster-state to kube-up script. This
allows creation of multi-master cluster from scratch. This is a
cherry-pick of #41320 from 1.5 branch.
2017-02-13 16:10:47 +01:00
Kubernetes Submit Queue e80afed777 Merge pull request #41035 from vishh/fluentd-critical
Automatic merge from submit-queue

Make fluentd a critical pod

For #40573
Based on https://github.com/kubernetes/kubernetes/pull/40655#issuecomment-277790544

```release-note
If `experimentalCriticalPodAnnotation` feature gate is set to true, fluentd pods will not be evicted by the kubelet.
```
2017-02-13 05:10:19 -08:00
Kubernetes Submit Queue 19ddde6b4f Merge pull request #41182 from brendandburns/storage
Automatic merge from submit-queue (batch tested with PRs 41182, 41290)

Add a default storage class for Azure Disk

Part of https://github.com/kubernetes/kubernetes/issues/40071

@jsafrane @colemickens @codablock @rootfs
2017-02-11 23:19:36 -08:00
Kubernetes Submit Queue 32dd68ad2a Merge pull request #41017 from luxas/symlink_hyperkube
Automatic merge from submit-queue (batch tested with PRs 38252, 41122, 36101, 41017, 41264)

Add alternative names for the server binaries to hyperkube

**What this PR does / why we need it**:

Right now one can't swap a server image to the hyperkube image without touching the `command` field in the yaml spec, and that's daunting and leading to extra and unnecessary logic for example in kubeadm.

This makes the hyperkube image directly swappable, so now `/usr/local/bin/kube-*` is a portable first argument (or simply `kube-*` if there's a shell).

**Special notes for your reviewer**:

**Release note**:

```release-note
Align the hyperkube image to support running binaries at /usr/local/bin/ like the other server images
```
@jessfraz @thockin @ixdy
2017-02-10 15:59:43 -08:00
Kubernetes Submit Queue b88b31cff4 Merge pull request #41242 from wojtek-t/dont_always_backup
Automatic merge from submit-queue (batch tested with PRs 41223, 40892, 41220, 41207, 41242)

Backup etcd only before migration

There is a bug currently that trigger backup on every run of a script (when we are running 2.2.1 version).

@mml
2017-02-10 13:35:47 -08:00
Kubernetes Submit Queue 3b7440ca9a Merge pull request #41207 from mikedanese/rerevert
Automatic merge from submit-queue (batch tested with PRs 41223, 40892, 41220, 41207, 41242)

reenable kubelet auth

revert #41132 

This reverts commit fd56078298, reversing
changes made to d953402cdf.
2017-02-10 13:35:45 -08:00
Wojciech Tyczynski 9e1a37061c Change migration tool to rollback to 2.2.1 2017-02-10 13:56:01 +01:00
Wojciech Tyczynski 3c503bc244 Change etcd rollback script to 2.2.1 version 2017-02-10 13:56:01 +01:00
Wojciech Tyczynski 5c0c445b7b Backup etcd only before migration 2017-02-10 11:29:45 +01:00
Kubernetes Submit Queue 3f25bbcd17 Merge pull request #41037 from bprashanth/glbc_version
Automatic merge from submit-queue (batch tested with PRs 41037, 40118, 40959, 41084, 41092)

Bump up GLBC version from 0.9.0-beta to 0.9.1

Tests have been green, moving the beta to a release.
2017-02-09 16:44:38 -08:00
Mike Danese c8ce55fef4 Revert "Merge pull request #41132 from kubernetes/revert-40893-kubelet-auth"
This reverts commit fd56078298, reversing
changes made to d953402cdf.
2017-02-09 15:55:12 -08:00
Kubernetes Submit Queue b7772e4f89 Merge pull request #40048 from mtaufen/remove-deprecated-flags
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)

Remove deprecated kubelet flags that look safe to remove

Removes:
```
--config
--auth-path
--resource-container
--system-container
```
which have all been marked deprecated since at least 1.4 and look safe to remove.

```release-note
The deprecated flags --config, --auth-path, --resource-container, and --system-container were removed.
```
2017-02-09 14:27:45 -08:00
Lucas Käldström 6f6ddc09c4
Make symlinks to /usr/local/bin/ in the hyperkube image 2017-02-09 21:46:00 +02:00
bprashanth 906b16d8d6 Bump up GLBC version from 0.9.0-beta to 0.9.1 2017-02-09 11:33:45 -08:00
Brendan Burns 43aa78c70a Add a default storage class for Azure Disk 2017-02-08 23:29:58 -08:00
Shyam Jeedigunta 3ac0e22f62 Refactored kubemark code into provider-specific and provider-independent parts [Part-3] 2017-02-08 17:03:13 +01:00
Wojciech Tyczynski 3828188049 Set quota bytes by default 2017-02-08 10:40:24 +01:00
Wojciech Tyczynski 3255898fba Fail etcd migration script if env var are unset 2017-02-08 10:40:24 +01:00
Wojciech Tyczynski 63531e56c5 Default TARGET_STORAGE to etcd3 in etcd manifest 2017-02-08 10:40:24 +01:00
Michael Taufen 982df56c52 Replace uses of --config with --pod-manifest-path 2017-02-07 14:32:37 -08:00
Kubernetes Submit Queue 5034d96bfb Merge pull request #40861 from lucab/to-k8s/bump-test-images
Automatic merge from submit-queue (batch tested with PRs 40345, 38183, 40236, 40861, 40900)

test: bump mounttest and mounttest-users images

This PR bumps two test images to latest versions:
 * mounttest to 0.8
 * mounttest-user to 0.5

It is a followup to https://github.com/kubernetes/kubernetes/pull/40613 and https://github.com/kubernetes/kubernetes/pull/40821.
2017-02-07 11:33:44 -08:00
Kubernetes Submit Queue 51901ef84a Merge pull request #40382 from liggitt/kube-dns-configmap
Automatic merge from submit-queue (batch tested with PRs 40382, 41060)

Make kube-dns mount optional configmap

Switches add-on templates to use an optional mounted configmap for dns

Uses options added in https://github.com/kubernetes/dns/pull/39

Blocks https://github.com/kubernetes/kubernetes/pull/38816
2017-02-07 02:36:05 -08:00
Kubernetes Submit Queue 5d0377d2e2 Merge pull request #41027 from dchen1107/master
Automatic merge from submit-queue (batch tested with PRs 40971, 41027, 40709, 40903, 39369)

Bump GCI to gci-beta-56-9000-80-0

cc/ @Random-Liu @adityakali 

Changelogs since gci-dev-56-8977-0-0 (currently used in Kubernetes):
 - "net.ipv4.conf.eth0.forwarding" and "net.ipv4.ip_forward" may get reset to 0
 - Track CVE-2016-9962 in Docker in GCI
 - Linux kernel CVE-2016-7097
 - Linux kernel CVE-2015-8964
 - Linux kernel CVE-2016-6828
 - Linux kernel CVE-2016-7917
 - Linux kernel CVE-2016-7042
 - Linux kernel CVE-2016-9793
 - Linux kernel CVE-2016-7039 and CVE-2016-8666
 - Linux kernel CVE-2016-8655
 - Toolbox: allow docker image to be loaded from local tarball
 - Update compute-image-package in GCI 
 - Change the product name on /etc/os-release (to COS)
 - Remove 'dogfood' from HWID_OVERRIDE in /etc/lsb-release
 - Include Google NVME extensions to optimize LocalSSD performance.
 - /proc/<pid>/io missing on GCI (enables process stats accounting)
 - Enable BLK_DEV_THROTTLING

cc/ @roberthbailey @fabioy for GKE cluster update
2017-02-06 20:57:14 -08:00
Vishnu kannan 10e7902a12 make fluentd a critical pod
Signed-off-by: Vishnu kannan <vishnuk@google.com>
2017-02-06 12:16:32 -08:00
Dawn Chen e710891dbb Bump GCI to gci-beta-56-9000-80-0 2017-02-03 18:00:16 -08:00
Luca Bruno 85b1def175
test: update to use mounttest:0.8 and mounttest-user:0.5 2017-02-02 20:41:18 +00:00
Vishnu Kannan c967ab7b99 Avoid evicting critical pods in Kubelet if a special feature gate is enabled
Signed-off-by: Vishnu Kannan <vishnuk@google.com>
2017-02-02 11:32:20 -08:00
Vishnu Kannan f85bbcb78d update kube proxy critical pod annotation comments to reflect reality
Signed-off-by: Vishnu Kannan <vishnuk@google.com>
2017-02-02 10:41:24 -08:00
Kubernetes Submit Queue 2dc1586f11 Merge pull request #40719 from alindeman/kube-temp-exported
Automatic merge from submit-queue (batch tested with PRs 39169, 40719, 38954, 40808, 40689)

Exports KUBE_TEMP for use in Vagrantfile

In #40147, the logic for setting `KUBE_TEMP` was refactored into `common.sh`. However, it was overlooked that `KUBE_TEMP` [needs to be exported for vagrant to work properly](https://github.com/kubernetes/kubernetes/pull/40147/files#diff-b19d3d93456020e2168c7f304f722969).

This PR restores the `export` so that `Vagrantfile` can use `ENV["KUBE_TEMP"]` properly.

👀 @rthallisey @shyamjvs @timothysc
2017-02-02 02:05:19 -08:00
Kubernetes Submit Queue 1a95922be6 Merge pull request #39169 from remonlam/master
Automatic merge from submit-queue (batch tested with PRs 39169, 40719, 38954, 40808, 40689)

add systemd notify
2017-02-02 02:05:17 -08:00
Kubernetes Submit Queue 0477100f98 Merge pull request #33684 from fraenkel/port_forward_ws
Automatic merge from submit-queue

Add websocket support for port forwarding

#32880

**Release note**:
```release-note
Port forwarding can forward over websockets or SPDY.
```
2017-02-01 23:19:02 -08:00
Zach Loafman 49062897d2 Bump GCE to container-vm-v20170201
Addresses CVE-2016-9962
2017-02-01 15:45:40 -08:00
Jordan Liggitt f139645c20
Make kube-dns mount optional configmap 2017-02-01 16:55:47 -05:00
Kubernetes Submit Queue 0bb31560c2 Merge pull request #40776 from MrHohn/addon-manager-v6.4
Automatic merge from submit-queue (batch tested with PRs 40758, 39145, 40776)

Bumps addon-manager to v6.4-alpha.1 for supporting optional ConfigMap

From #40382. Bumps up addon-manager to use v1.6.0-alpha.1 kubectl for the optional ConfigMap feature. Below images have been pushed:
- gcr.io/google-containers/kube-addon-manager:v6.4-alpha.1
- gcr.io/google-containers/kube-addon-manager-amd64:v6.4-alpha.1
- gcr.io/google-containers/kube-addon-manager-arm:v6.4-alpha.1
- gcr.io/google-containers/kube-addon-manager-arm64:v6.4-alpha.1
- gcr.io/google-containers/kube-addon-manager-ppc64le:v6.4-alpha.1
- gcr.io/google-containers/kube-addon-manager-s390x:v6.4-alpha.1

@liggitt @bowei
2017-02-01 13:30:46 -08:00
Zihong Zheng c91d605124 Bumps addon-manager to v6.4-alpha.1 for supporting optional ConfigMap 2017-02-01 09:22:43 -08:00
Michael Fraenkel beb53fb71a Port forward over websockets
- split out port forwarding into its own package

Allow multiple port forwarding ports
- Make it easy to determine which port is tied to which channel
- odd channels are for data
- even channels are for errors

- allow comma separated ports to specify multiple ports

Add  portfowardtester 1.2 to whitelist
2017-02-01 06:32:04 -07:00
Kubernetes Submit Queue 1ed1ba44d5 Merge pull request #40274 from muteor/fix-libvirt-coreos-bin-permissions
Automatic merge from submit-queue (batch tested with PRs 40111, 40368, 40342, 40274, 39443)

Libvirt-coreos - Add execute permissions to kubernetes/bin

**What this PR does / why we need it**:

The master node was failing to start for me due to the permission errors on the kubernetes server binaries.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-01-31 19:18:42 -08:00
Kubernetes Submit Queue fd13413ba7 Merge pull request #40368 from sc68cal/patch/centos_name
Automatic merge from submit-queue (batch tested with PRs 40111, 40368, 40342, 40274, 39443)

Change OPENSTACK_IMAGE_NAME to be more specific

There may already be other images in the cloud named CentOS7,
but since we are fetching a very specific version (1604) we should
go ahead and make the image name very specific as well.



**What this PR does / why we need it**:

Some clouds already have `Centos7` as an image that is available, however it may not be the *specific* version that openstack-heat looks for and downloads from CentOS.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: 

**Special notes for your reviewer**:

**Release note**:

```release-note 
OpenStack-Heat will now look for an image named "CentOS-7-x86_64-GenericCloud-1604". To restore the previous behavior set OPENSTACK_IMAGE_NAME="CentOS7"
```
2017-01-31 19:18:39 -08:00
Kubernetes Submit Queue 60c66dd3f4 Merge pull request #38772 from zmerlynn/kube-up-aws-dead
Automatic merge from submit-queue (batch tested with PRs 38772, 38797, 40732, 40740)

AWS: Deprecate the bash deployment

**What this PR does / why we need it**:  Add a strong deprecation warning to the `kube-up.sh` AWS deployment.

**Release note**:

```release-note
The bash AWS deployment via kube-up.sh has been deprecated. See http://kubernetes.io/docs/getting-started-guides/aws/ for alternatives.
```
2017-01-31 11:00:44 -08:00
Kubernetes Submit Queue 16164c6a54 Merge pull request #40711 from spxtr/owner
Automatic merge from submit-queue

Remove spxtr from several owners files, add rmmh as test reviewer.

**Release note**:
```release-note
NONE
```
2017-01-31 05:03:53 -08:00
Kubernetes Submit Queue f8f34187a7 Merge pull request #40707 from piosz/nanny-version
Automatic merge from submit-queue

Bumped version of addon resizer

ref https://github.com/kubernetes/contrib/issues/2337
2017-01-31 02:45:47 -08:00
Kubernetes Submit Queue 499bab1b99 Merge pull request #40713 from mml/force-new-cluster
Automatic merge from submit-queue (batch tested with PRs 40392, 39242, 40579, 40628, 40713)

Add --force-new-cluster when running etcd for migrations.

This is required to avoid etcd trying to create quorum during
migrations.

Might fix #40110
2017-01-31 01:16:56 -08:00
Kubernetes Submit Queue 70717dce15 Merge pull request #40691 from Cynerva/gkk/fix-master-ready-status
Automatic merge from submit-queue (batch tested with PRs 40691, 40551, 40683, 40700, 40702)

Juju kubernetes-master charm: improve status messages

**What this PR does / why we need it**:

This update to the kubernetes-master charm does the following:
1. Remove "Kubernetes master services ready" status which was occurring too early
2. Add "Waiting for kube-system pods to start" status
3. Replace "Rendering the Kubernetes DNS files." status with "Deploying KubeDNS"
4. Add "Waiting to retry KubeDNS deployment" status

The purpose of this is to give better feedback to the operator during cluster deployment.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:

Fixes https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/143, which we are tracking in a separate repository

**Special notes for your reviewer**:

This is a rebase of https://github.com/juju-solutions/kubernetes/pull/103, where prior review was done, though it was targeted against a fork.

**Release note**:

```release-note
Juju kubernetes-master charm: improve status messages
```
2017-01-30 19:14:39 -08:00
Andy Lindeman afb5e8ceb7
Exports KUBE_TEMP for use in Vagrantfile 2017-01-30 22:05:07 -05:00
Kubernetes Submit Queue d4c5aa2e3f Merge pull request #40093 from rthallisey/kubemark-makefile
Automatic merge from submit-queue (batch tested with PRs 40703, 40093, 40618, 40659, 39810)

Change kubemark Makefile to be provider independent

Ref issue #38967

The Kubemark Makefile is defaulted to gcr.io. Instead, make it
provider independent.

The kubemark makefile is set to push the kubemark image to the gcr.io registry. In order to make kubemark not as provider specific, allow the developer to choose a registry.
2017-01-30 17:14:52 -08:00
Matt Liggett cc7c142011 Add --force-new-cluster when running etcd for migrations.
This is required to avoid etcd trying to create quorum during
migrations.
2017-01-30 14:22:53 -08:00
Joe Finney 3ec286adda Remove spxtr from several owners files, add rmmh as test reviewer. 2017-01-30 13:45:21 -08:00
Piotr Szczesniak b2ea40fb97 Bumped version of addon resizer 2017-01-30 21:42:36 +01:00
Jorge O. Castro 1d6d9e13a3 Update Ubuntu OWNERS 2017-01-30 14:56:26 -05:00
George Kraft b0a9c00bbb improve status messages around KubeDNS 2017-01-30 10:09:10 -06:00
George Kraft a371b1bc27 Add "Waiting for kube-system pods to start" status message 2017-01-30 09:05:29 -06:00
George Kraft 5c5fa43700 Remove "Kubernetes master services ready." status message
This message was occurring prematurely - the master is running,
but the charm still has work to do. e.g. deploy KubeDNS.
2017-01-30 09:05:29 -06:00
George Kraft df48acb41d slightly clean up kubedns reactive step 2017-01-30 09:05:28 -06:00
Ryan Hallisey db9951b2ce Change kubemark Makefile to be provider independent
The Kubemark Makefile is defaulted to gcr.io. Instead, make it
provider independent.
2017-01-30 07:45:30 -05:00
Kubernetes Submit Queue 7ee9abe447 Merge pull request #40339 from noseka1/master
Automatic merge from submit-queue (batch tested with PRs 40549, 40339)

Invalid node names when deploying with Heat

OpenStack Heat templates create Kubernetes nodes with invalid
hostnames. Capital letters are not allowed in the hostnames:

Unable to register node "kubernetes-node-6s8OizYe" with API server: Node "kubernetes-node-6s8OizYe" is invalid: metadata.name: Invalid value: "kubernetes-node-6s8OizYe": must match the regex [a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)* (e.g. 'example.com')

This patch prevents Heat generating hostnames that contain
capital letters.



**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```
2017-01-29 23:52:07 -08:00
Kubernetes Submit Queue 3897fc00db Merge pull request #38926 from luxas/update_cross_go18
Automatic merge from submit-queue

Improve the multiarch situation; armel => armhf; reenable pcc64le; remove the patched golang

**What this PR does / why we need it**: 
 - Improves the multiarch situation as described in #38067 
 - Tries to bump to go1.8 for arm (and later enable ppc64le)
 - GOARM 6 => GOARM 7
 - Remove the golang 1.7 patch
 - armel => armhf
 - Bump QEMU version to v2.7.0

**Release note**:

```release-note
Improve the ARM builds and make hyperkube on ARM working again by upgrading the Go version for ARM to go1.8beta2
```

@kubernetes/sig-testing-misc @jessfraz @ixdy @jbeda @david-mcmahon @pwittrock
2017-01-28 03:51:19 -08:00
Kubernetes Submit Queue 25eb57ad5c Merge pull request #40584 from madhusudancs/federation-kubefed.sh
Automatic merge from submit-queue

Use a wrapper script to locate kubefed and kubectl binaries instead of directly constructing their paths.

This fixes the e2e failures that is now switched to using kubefed.

cc @kubernetes/sig-federation-pr-reviews
2017-01-27 19:19:34 -08:00
Kubernetes Submit Queue 29a7b6f826 Merge pull request #40554 from roberthbailey/chown
Automatic merge from submit-queue (batch tested with PRs 40497, 39769, 40554, 40569, 40597)

When calling chown, use : instead of . to separate the user and group for cross platform compatibility.

**What this PR does / why we need it**: Makes it possible to build on a Mac, which was broken by #39515. 

**Special notes for your reviewer**:

**Release note**:
`NONE`
2017-01-27 17:38:26 -08:00
Lucas Käldström 914824cc09
Update go version to go1.8rc2 and fix the addon-manager changelog 2017-01-27 22:44:18 +02:00
Kubernetes Submit Queue 0107e93cc6 Merge pull request #40557 from shyamjvs/refactor-returns
Automatic merge from submit-queue (batch tested with PRs 39469, 40557)

Refactored kubemark code into provider-specific and provider-independent parts [Part-1]

Applying part of the changes of PR https://github.com/kubernetes/kubernetes/pull/39033 (which refactored kubemark code completely). The changes included in this PR are:

The following are the major changes as part of this refactoring:
- Moved cluster-kubemark/config-default.sh -> cluster-kubemark/gce/config-default.sh (as the config is gce-specific)
- Changed kubernetes/cluster/kubemark/util.sh to source the right scripts based on the cloud-provider
- Added the file test/kubemark/cloud-provider-config.sh which sets the variable CLOUD_PROVIDER that is later picked up by various scripts (run-e2e-tests.sh, common.sh)
- Removed useless code and restructured start-kubemark.sh and stop-kubemark.sh scripts.

@kubernetes/sig-scalability-misc @wojtek-t @gmarek
2017-01-27 10:38:33 -08:00
Lucas Käldström 84006601a0
Upgrade go version in Makefiles to 1.7, use qemu 2.7, armel => armhf and goarm=6 => goarm=7 and use go 1.7.4 2017-01-27 20:04:24 +02:00
gmarek b51df7de67 add OWNERS to cluster/images/kubemark 2017-01-27 16:24:45 +01:00
Kubernetes Submit Queue e80da46b94 Merge pull request #40565 from bprashanth/glbc-version
Automatic merge from submit-queue (batch tested with PRs 40126, 40565, 38777, 40564, 40572)

Bump up glbc version to 0.9.0-beta.1

I plan to bump up the version to 0.9.0 proper in time for the next 1.5.x release, and cherry-pick both this and the future pr.

Previously we were just using a single version, but the "-beta/alpha" is consistent with how we release kube and gives us a convenient revert target. It also forces us to remove the "beta" tag before code freeze, and track the kubernetes release cycle.
2017-01-27 01:34:17 -08:00
Madhusudan.C.S baab2e37a9 Refactor the common parts of cluster/kube{ctl,adm}.sh into a util script. 2017-01-26 21:29:49 -08:00
bprashanth b1e0bd0fa4 Bump up glbc version to beta.1 2017-01-26 14:53:50 -08:00
Shyam Jeedigunta c62e5214c3 Refactored kubemark code into provider-specific and provider-independent parts [Part-1] 2017-01-26 22:54:14 +01:00
Robert Bailey a6c65f3035 When calling chown, use : instead of . to separate the user and
group for cross platform compatibility.
2017-01-26 13:39:06 -08:00
Kubernetes Submit Queue 2de4223cba Merge pull request #40353 from Crassirostris/fluentd-config-timestamp-fix
Automatic merge from submit-queue

Fix fluentd-gcp configuration to handle different timezones on the node

Fix https://github.com/kubernetes/kubernetes/issues/40226

Probably a bug in the fluentd configuration
2017-01-26 09:21:35 -08:00
Kubernetes Submit Queue b1279a31b9 Merge pull request #40172 from liggitt/abac-upgrade
Automatic merge from submit-queue (batch tested with PRs 38739, 40480, 40495, 40172, 40393)

Use existing ABAC policy file when upgrading GCE cluster

When upgrading, continue loading an existing ABAC policy file so that existing system components continue working as-is

```
When upgrading an existing 1.5 GCE cluster using `cluster/gce/upgrade.sh`, an existing ABAC policy file located at /etc/srv/kubernetes/abac-authz-policy.jsonl (the default location in 1.5) will enable the ABAC authorizer in addition to the RBAC authorizer. To switch an upgraded 1.5 cluster completely to RBAC, ensure the control plane components and your superuser have been granted sufficient RBAC permissions, move the legacy ABAC policy file to a backup location, and restart the apiserver.
```
2017-01-25 21:20:32 -08:00
Kubernetes Submit Queue 52aa408aac Merge pull request #39784 from micmro/keystone-v3-support-for-openstack-heat
Automatic merge from submit-queue

Add support of Keystone v3 'domain-name' to 'openstack-heat' cluster setup

**What this PR does / why we need it**:
Keystone v3 authentication by user name [requires the domain (name or ID)](http://developer.openstack.org/api-ref/identity/v3/index.html?expanded=password-authentication-with-scoped-authorization-detail). If `domain-name` is not provided kubelet fails as seen below:
  
```
kubelet: error: failed to run Kubelet: could not init cloud provider "openstack": You must provide exactly one of DomainID or DomainName to authenticate by Username
systemd: kubelet.service: main process exited, code=exited, status=1/FAILURE
systemd: Unit kubelet.service entered failed state.
systemd: kubelet.service failed.
```

To solve this I pass a new`OS_USER_DOMAIN_NAME` environment variable through openstack-heat's heat-templates to write it as `domain-name` in `/srv/kubernetes/openstack.conf`. 

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #39783

**Special notes for your reviewer**:

**Release note**:

```
domain-name support for Keystone v3 added to openstack-heat cluster setup
```
2017-01-25 19:30:32 -08:00
Kubernetes Submit Queue 75c6990966 Merge pull request #39462 from Shawyeok/master
Automatic merge from submit-queue

Able to quick create a HA cluster by kube-up.sh centos provider

Make `kube-up.sh` `centos provider` support quick create a HA cluster, as I said above [#39430](https://github.com/kubernetes/kubernetes/issues/39430), it's more flexible than `kops` or `kubeadm` for some people in a limited network region.

I'm new to k8s dev, so if this pull request need to change, please let me know.

```release-note
Added support for creating HA clusters for centos using kube-up.sh.
```
2017-01-25 16:40:57 -08:00
Kubernetes Submit Queue d98615db9a Merge pull request #40089 from sc68cal/install_docs
Automatic merge from submit-queue

[OpenStack-Heat] Add link to the OpenStack CLI install documentation

**What this PR does / why we need it**:

More helpful diagnostic text 

**Special notes for your reviewer**:

Trivial patch

**Release note**:

```release-note
NONE
```
2017-01-25 15:55:00 -08:00
Jordan Liggitt a7f52b276f
Add test shell stack traces 2017-01-25 13:34:16 -05:00
shawyeok 78170bf113 kube-up.sh centos provider support create a HA cluster
Fix: cannot get default master advertise address correctly

Set default value of NUM_MASTERS and NUM_NODES by MASTERS and NODES themself

Code cleanup and documented

Using runtime reconfiguration for etcd cluster instead of etcd discovery

Add exceptions for verify-flags
2017-01-25 22:53:27 +08:00
Shyam Jeedigunta b48de58311 Added OWNERS to kubemark subdirectories 2017-01-25 14:37:57 +01:00
Michael Mrowetz 5bdae77e0c add default for OS_USER_DOMAIN_NAME 2017-01-25 10:46:47 +09:00
Michael Mrowetz 87ee6b34ce pass openstack-heat OS_USER_DOMAIN_NAME config to domain-name in openstack.conf 2017-01-25 10:39:49 +09:00
Kubernetes Submit Queue 84e98251ed Merge pull request #40315 from madhusudancs/fed-log-uniq-cluster-names
Automatic merge from submit-queue (batch tested with PRs 40335, 40320, 40324, 39103, 40315)

Use the e2e zone name as the cluster name.

This needs a revamp, but for now e2e zone name is used as the
unique cluster identifier in our e2e tests and we will continue
to use that pattern.

This is a follow up to PR #38638.

cc @kubernetes/sig-federation-pr-reviews @nikhiljindal
2017-01-24 17:30:10 -08:00
Kubernetes Submit Queue e3ba25714f Merge pull request #40324 from chuckbutler/upstream-rebase-forreal
Automatic merge from submit-queue (batch tested with PRs 40335, 40320, 40324, 39103, 40315)

Splitting master/node services into separate charm layers

**What this PR does / why we need it**:

This branch includes a roll-up series of commits from a fork of the
Kubernetes repository pre 1.5 release because we didn't make the code freeze.
This additional effort has been fully tested and has results submit into
the gubernator to enhance confidence in this code quality vs. the single
layer, posing as both master/node.

To reference the gubernator results, please see:
https://k8s-gubernator.appspot.com/builds/canonical-kubernetes-tests/logs/kubernetes-gce-e2e-node/

Apologies in advance for the large commit however, we did not want to
submit without having successful upstream automated testing results.

This commit includes:

 - Support for CNI networking plugins
 - Support for durable storage provided by Ceph
 - Building from upstream templates (read: kubedns - no more template
 drift!)
 - An e2e charm-layer to make running validation tests much simpler/repeatable
 - Changes to support the 1.5.x series of Kubernetes



**Special notes for your reviewer**:

Additional note: We will be targeting -all- future work against upstream
so large pull requests of this magnitude will not occur again.

**Release note**:




```release-note
- Splits Juju Charm layers into master/worker roles
- Adds support for 1.5.x series of Kubernetes
- Introduces a tactic for keeping templates in sync with upstream eliminating template drift
- Adds CNI support to the Juju Charms
- Adds durable storage support to the Juju Charms
- Introduces an e2e Charm layer for repeatable testing efforts and validation of clusters

```
2017-01-24 17:30:06 -08:00
Kubernetes Submit Queue 7fbb458f6d Merge pull request #40213 from jszczepkowski/ha-e2e-tests
Automatic merge from submit-queue (batch tested with PRs 39260, 40216, 40213, 40325, 40333)

Fixed propagation of kube master certs during master replication.

Fixed propagation of kube-master-certs during master replication.
2017-01-24 16:26:02 -08:00
Kubernetes Submit Queue 6dbad57bce Merge pull request #37264 from yuanying/use-lowercase-character
Automatic merge from submit-queue (batch tested with PRs 39275, 40327, 37264)

Fix invalid node name in openstack-heat provider

Cluster node name must follow name syntax in RFC 1123.
But currently, openstack-heat provider generate invalid
node name which contains upper-case characters.
This patch fixes it.
2017-01-24 10:52:12 -08:00
Madhusudan.C.S 63dfa129f4 Use the e2e zone name as the cluster name.
This needs a revamp, but for now e2e zone name is used as the
unique cluster identifier in our e2e tests and we will continue
to use that pattern.
2017-01-24 10:50:51 -08:00
Kubernetes Submit Queue 054c84e22f Merge pull request #40299 from lucab/to-k8s/rkt-1.23.0
Automatic merge from submit-queue (batch tested with PRs 40299, 40311)

cluster: update default rkt version to 1.23.0

This updates cluster configurations to current stable rkt version.
2017-01-24 08:59:57 -08:00
Sean M. Collins 8a4038a2e1 Change OPENSTACK_IMAGE_NAME to be more specific
There may already be other images in the cloud named CentOS7,
but since we are fetching a very specific version (1604) we should
go ahead and make the image name very specific as well.
2017-01-24 11:53:10 -05:00
Matt Bruzek 3fcf279cfb Splitting master/node services into separate charm layers
This branch includes a rollup series of commits from a fork of the
kubernetes repository pre 1.5 release because we didn't make the code freeze.
This additional effort has been fully tested and has results submit into
the gubernator to enhance confidence in this code quality vs. the single
layer, posing as both master/node.

To reference the gubernator results, please see:
https://k8s-gubernator.appspot.com/builds/canonical-kubernetes-tests/logs/kubernetes-gce-e2e-node/

Apologies in advance for the large commit, however we did not want to
submit without having successful upstream automated testing results.

This commit includes:

 - Support for CNI networking plugins
 - Support for durable storage provided by ceph
 - Building from upstream templates (read: kubedns - no more template
 drift!)
 - An e2e charm-layer to make running validation tests much simpler/repeatable
 - Changes to support the 1.5.x series of kubernetes

Additional note: We will be targeting -all- future work against upstream
so large pull requests of this magnitude will not occur again.
2017-01-24 09:42:25 -06:00
Mik Vyatskov 7b194d496f Fix fluentd-gcp configuration to handle different timezones on the node 2017-01-24 11:53:15 +01:00
Wojciech Tyczynski fbd5c7c380 Revert "Refactored kubemark into cloud-provider independent code and GCE specific code" 2017-01-24 10:42:17 +01:00
Ales Nosek 307a7556da Invalid node names when deploying with Heat
OpenStack Heat templates create Kubernetes nodes with invalid
hostnames. Capital letters are not allowed in the hostnames:

Unable to register node "kubernetes-node-6s8OizYe" with API server: Node "kubernetes-node-6s8OizYe" is invalid: metadata.name: Invalid value: "kubernetes-node-6s8OizYe": must match the regex [a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)* (e.g. 'example.com')

This patch prevents Heat generating hostnames that contain
capital letters.
2017-01-23 18:34:32 -08:00
Clayton Coleman 469df12038
refactor: move ListOptions references to metav1 2017-01-23 17:52:46 -05:00
Kubernetes Submit Queue da14a4d836 Merge pull request #40309 from apelisse/complete-owners
Automatic merge from submit-queue

OWNERS: Update latest OWNERS files

These files have been created lately, so we don't have much information
about them anyway, so let's just:
- Remove assignees and make them approvers
- Copy approves as reviewers
2017-01-23 11:24:51 -08:00
Kubernetes Submit Queue 7065c477cb Merge pull request #40251 from mml/sudo
Automatic merge from submit-queue (batch tested with PRs 40251, 40171)

Only run gcloud as root if we plan to change something.

Only run gcloud as root if we plan to change something.

Fixes bug introduced in #36292 @jlowdermilk @ixdy
2017-01-23 11:24:06 -08:00
Antoine Pelisse 62af7dd33d OWNERS: Update latest OWNERS files
These files have been created lately, so we don't have much information
about them anyway, so let's just:
- Remove assignees and make them approvers
- Copy approves as reviewers
2017-01-23 10:05:48 -08:00
Luca Bruno b4bc44b9ff
cluster: update default rkt version to 1.23.0 2017-01-23 15:22:33 +00:00
Kubernetes Submit Queue 920d8fd024 Merge pull request #36250 from apprenda/win_kube_scripts
Automatic merge from submit-queue

Powershell script to start kubelet and kube-proxy

**What this PR does / why we need it**:
This PR adds a powershell script to run kubelet and kube-proxy on Windows. It expects the required arguments like `API Server` location and uses appropriate defaults.

**Which issue this PR fixes** : 
fixes # https://github.com/kubernetes/kubernetes/issues/34270

**Special notes for your reviewer**:
This PR is for supporting Windows Server Containers for k8s, the work for which is covered under https://github.com/kubernetes/features/issues/116
This PR should be merged after https://github.com/kubernetes/kubernetes/pull/31707 and https://github.com/kubernetes/kubernetes/pull/36079 PRs are merged

**Release note**:

```release-note
```
2017-01-23 02:28:39 -08:00
Keith Pope ce778bf53b Add execute permissions to the kubernetes/bin as master was failing to start due to permission denied 2017-01-22 16:00:04 +00:00
Mik Vyatskov cd4ee14019 Fix Elasticsearch image for logging and bump version 2017-01-21 17:48:32 +01:00
Matt Liggett 0a4981ebf4 Only run gcloud as root if we plan to change something. 2017-01-20 17:18:05 -08:00
Kubernetes Submit Queue 21f021449d Merge pull request #40232 from mikedanese/gce-signer
Automatic merge from submit-queue

pass CA key to signer in GCE
2017-01-20 14:16:31 -08:00
Mike Danese 513994a9f8 pass CA key to signer in GCE 2017-01-20 11:10:19 -08:00
Kubernetes Submit Queue c752a03bed Merge pull request #40011 from vwfs/optional_cachebust
Automatic merge from submit-queue (batch tested with PRs 40011, 40159)

Make CACHEBUST for hyperkube build optional

**What this PR does / why we need it**: It makes CACHEBUST for the hyperkube build optional. Currently, building the hyperkube always results in a full rebuild, including retrieving and installing of all debian packages. This is a good thing for releases, but makes live as a dev hard.

This allows to do something like this:
```
$ REGISTRY=<registry> VERSION=<version> CACHEBUST=0 ./hack/dev-push-hyperkube.sh
```

Probably we should even make CACHEBUST=0 the default when calling dev-xxx.sh scripts.

CC: @aaronlevy
2017-01-20 10:48:43 -08:00
Kubernetes Submit Queue 093ceb9528 Merge pull request #39033 from shyamjvs/provider-independent-kubemark
Automatic merge from submit-queue (batch tested with PRs 36693, 40154, 40170, 39033)

Refactored kubemark into cloud-provider independent code and GCE specific code

Ref issue #38967 

The following are the major changes as part of this refactoring:
- Moved cluster-kubemark/config-default.sh -> cluster-kubemark/gce/config-default.sh (as the config is gce-specific)
- Changed kubernetes/cluster/kubemark/util.sh to source the right scripts based on the cloud-provider
- Added test/kubemark/skeleton/util.sh which defines a well-commented interface that any cloud-provider should implement to run kubemark. (We have this interface defined only for gce currently)
  This includes functions like creating the master machine instance along with its resources, executing a given command on the master (like ssh), scp, deleting the master instance and its resources.
  All these functions have to be overrided by each cloud provider inside the file /test/kubemark/$CLOUD_PROVIDER/util.sh
- Added the file test/kubemark/cloud-provider-config.sh which sets the variable CLOUD_PROVIDER that is later picked up by various scripts (start-kubemark.sh, stop-kubemark.sh, run-e2e-tests.sh)
- Removed test/kubemark/common.sh and moved whatever provider-independent code it had into start-kubemark.sh (the only place where the scipt is called) and moved the little gce-specific code
  into test/kubemark/gce/util.sh.
- Finally, removed useless code and restructured start-kubemark.sh and stop-kubemark.sh scripts.

@kubernetes/sig-scalability-misc @wojtek-t @gmarek
2017-01-20 09:18:54 -08:00
Kubernetes Submit Queue 22a405055d Merge pull request #40170 from deads2k/client-10-restclient
Automatic merge from submit-queue (batch tested with PRs 36693, 40154, 40170, 39033)

make client-go authoritative for pkg/client/restclient

Moves client/restclient to client-go and a util/certs, util/testing as transitives.
2017-01-20 09:18:52 -08:00
Kubernetes Submit Queue 1430597f7e Merge pull request #39966 from liggitt/cert-users
Automatic merge from submit-queue (batch tested with PRs 40168, 40165, 39158, 39966, 40190)

Include system:masters group in the bootstrap admin client certificate

Sets up the bootstrap admin client certificate for new clusters to be in the system:masters group

Removes the need for an explicit grant to the kubecfg user in e2e-bindings

```release-note
The default client certificate generated by kube-up now contains the superuser `system:masters` group
```
2017-01-20 08:28:51 -08:00
deads2k ee6752ef20 find and replace 2017-01-20 08:04:53 -05:00
Jerzy Szczepkowski d1a73fa5cd Fixed propagation of kube master certs during master replication.
Fixed propagation of kube master certs during master replication.
2017-01-20 13:24:09 +01:00
Kubernetes Submit Queue 0610a23986 Merge pull request #40164 from apelisse/update-root-approvers-files
Automatic merge from submit-queue

Update root approvers files

Replaces #40040 

Update top level OWNERS files mostly to set assignees to approvers. Also remove @bgrant0607 from everywhere but the very top level OWNERS file.
2017-01-19 17:02:18 -08:00
Jordan Liggitt e6a63c8f40
Use existing ABAC policy file when upgrading GCE cluster 2017-01-19 16:46:08 -05:00
Kubernetes Submit Queue cc2250cb98 Merge pull request #40147 from rthallisey/common-ensure-temp-dir
Automatic merge from submit-queue

Use ensure-temp-dir in the common.sh script

Ref issue #38967

Instead of having an ensure-temp-dir function in multiple
places, add it to the common.sh script which is sourced by
all the providers.
2017-01-19 11:57:35 -08:00
Garrett Rodrigues ad1e5e98c2 Updated top level owners file to match new format 2017-01-19 11:29:16 -08:00
thomasschickinger 42fbf93fb0 Add rule for detecting exceptions to fluentd config for GKE logging.
Bump version of gcp-fluentd container to 1.34
2017-01-19 15:51:47 +01:00
Shyam Jeedigunta d2fadbe30f Refactored kubemark code into provider-specific and provider-independent parts 2017-01-19 15:34:13 +01:00
Ryan Hallisey dbb92f9836 Use ensure-temp-dir in the common.sh script
Instead of having an ensure-temp-dir function in multiple
places, add it to the common.sh script which is sourced by
all the providers.
2017-01-19 09:30:50 -05:00
Kubernetes Submit Queue 5e4625cad7 Merge pull request #40017 from Crassirostris/fluentd-gcp-image-fix
Automatic merge from submit-queue (batch tested with PRs 40003, 40017)

Remove library copying from fluentd image

It seems that fluentd can no longer copy systemd libraries from host to be able to read journals.
2017-01-19 05:54:08 -08:00
Kubernetes Submit Queue 29e2d8be09 Merge pull request #40113 from maisem/cos
Automatic merge from submit-queue

Adding cos as an alias for gci.

**What this PR does / why we need it**: Adding COS as an alias for GCI.

cc: @adityakali @wonderfly
2017-01-18 18:40:43 -08:00
Kubernetes Submit Queue 0c61553cbc Merge pull request #40105 from sc68cal/bugs/40102
Automatic merge from submit-queue (batch tested with PRs 40105, 40095)

[OpenStack-Heat] Fix regex used to get object-store URL

**Release note**:

```release-note

Fixes a bug in the OpenStack-Heat kubernetes provider, in the handling of differences between the Identity v2 and Identity v3 APIs

```
2017-01-18 15:54:08 -08:00
Maisem Ali 52b6c9bb41 Adding cos as an alias for gci. 2017-01-18 15:14:25 -08:00
Kubernetes Submit Queue b29d9cdbcf Merge pull request #39898 from ixdy/bazel-release-tars
Automatic merge from submit-queue

Build release tars using bazel

**What this PR does / why we need it**: builds equivalents of the various kubernetes release tarballs, solely using bazel.

For example, you can now do
```console
$ make bazel-release
$ hack/e2e.go -v -up -test -down
```

**Special notes for your reviewer**: this is currently dependent on 3b29803eb5, which I have yet to turn into a pull request, since I'm still trying to figure out if this is the best approach.

Basically, the issue comes up with the way we generate the various server docker image tarfiles and load them on nodes:
* we `md5sum` the binary being encapsulated (e.g. kube-proxy) and save that to `$binary.docker_tag` in the server tarball
* we then build the docker image and tag using that md5sum (e.g. `gcr.io/google_containers/kube-proxy:$MD5SUM`)
* we `docker save` this image, which embeds the full tag in the `$binary.tar` file.
* on cluster startup, we `docker load` these tarballs, which are loaded with the tag that we'd created at build time. the nodes then use the `$binary.docker_tag` file to find the right image.

With the current bazel `docker_build` rule, the tag isn't saved in the docker image tar, so the node is unable to find the image after `docker load`ing it.

My changes to the rule save the tag in the docker image tar, though I don't know if there are subtle issues with it. (Maybe we want to only tag when `--stamp` is given?)

Also, the docker images produced by bazel have the timestamp set to the unix epoch, which is not great for debugging. Might be another thing to change with a `--stamp`.

Long story short, we probably need to follow up with bazel folks on the best way to solve this problem.

**Release note**:

```release-note
NONE
```
2017-01-18 14:24:48 -08:00
Kubernetes Submit Queue 76d023ca90 Merge pull request #40094 from zmerlynn/cvm-v20170117
Automatic merge from submit-queue (batch tested with PRs 36467, 36528, 39568, 40094, 39042)

Bump GCE to container-vm-v20170117

Base image update only, no kubelet or Docker updates.

```release-note
Update GCE ContainerVM deployment to container-vm-v20170117 to pick up CVE fixes in base image.
```
2017-01-18 13:37:12 -08:00
Sean M. Collins 8ad7e1613a [OpenStack-Heat] Fix regex used to get object-store URL
"publicURL" is used for endpoints in the Identity v2 API, while in the
Identity v3 API it has been changed to just "public"

Fixes #40102
2017-01-18 16:29:41 -05:00
Zach Loafman a0b8fd618f Bump GCE to container-vm-v20170117
Base image update only, no kubelet or Docker updates.
2017-01-18 10:50:17 -08:00
Kubernetes Submit Queue 6dfe5c49f6 Merge pull request #38865 from vwfs/ext4_no_lazy_init
Automatic merge from submit-queue

Enable lazy initialization of ext3/ext4 filesystems

**What this PR does / why we need it**: It enables lazy inode table and journal initialization in ext3 and ext4.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #30752, fixes #30240

**Release note**:
```release-note
Enable lazy inode table and journal initialization for ext3 and ext4
```

**Special notes for your reviewer**:
This PR removes the extended options to mkfs.ext3/mkfs.ext4, so that the defaults (enabled) for lazy initialization are used.

These extended options come from a script that was historically located at */usr/share/google/safe_format_and_mount* and later ported to GO so this dependency to the script could be removed. After some search, I found the original script here: https://github.com/GoogleCloudPlatform/compute-image-packages/blob/legacy/google-startup-scripts/usr/share/google/safe_format_and_mount

Checking the history of this script, I found the commit [Disable lazy init of inode table and journal.](4d7346f7f5). This one introduces the extended flags with this description:
```
Now that discard with guaranteed zeroing is supported by PD,
initializing them is really fast and prevents perf from being affected
when the filesystem is first mounted.
```

The problem is, that this is not true for all cloud providers and all disk types, e.g. Azure and AWS. I only tested with magnetic disks on Azure and AWS, so maybe it's different for SSDs on these cloud providers. The result is that this performance optimization dramatically increases the time needed to format a disk in such cases.

When mkfs.ext4 is told to not lazily initialize the inode tables and the check for guaranteed zeroing on discard fails, it falls back to a very naive implementation that simply loops and writes zeroed buffers to the disk. Performance on this highly depends on free memory and also uses up all this free memory for write caching, reducing performance of everything else in the system. 

As of https://github.com/kubernetes/kubernetes/issues/30752, there is also something inside kubelet that somehow degrades performance of all this. It's however not exactly known what it is but I'd assume it has something to do with cgroups throttling IO or memory. 

I checked the kernel code for lazy inode table initialization. The nice thing is, that the kernel also does the guaranteed zeroing on discard check. If it is guaranteed, the kernel uses discard for the lazy initialization, which should finish in a just few seconds. If it is not guaranteed, it falls back to using *bio*s, which does not require the use of the write cache. The result is, that free memory is not required and not touched, thus performance is maxed and the system does not suffer.

As the original reason for disabling lazy init was a performance optimization and the kernel already does this optimization by default (and in a much better way), I'd suggest to completely remove these flags and rely on the kernel to do it in the best way.
2017-01-18 09:09:52 -08:00
Sean M. Collins bfb60acc02 [OpenStack-Heat] Add link to the OpenStack CLI install documentation 2017-01-18 11:02:55 -05:00
Kubernetes Submit Queue 16f45aee85 Merge pull request #39925 from appscode/kube-dns-1.11.0
Automatic merge from submit-queue

Use kube-dns:1.11.0

Use [kube-dns:1.11.0](https://github.com/kubernetes/dns/releases/tag/1.11.0)

With: kubernetes/dns#25
Fixes kubernetes/kubernetes#26752
Fixes kubernetes/kubernetes#33470

@bowei @thockin
2017-01-17 10:08:48 -08:00
Kubernetes Submit Queue 685e421b89 Merge pull request #40020 from wojtek-t/really_enable_etcd3
Automatic merge from submit-queue (batch tested with PRs 34763, 38706, 39939, 40020)

Really enable etcd3

Ref #39589

@timothysc @hongchaodeng
2017-01-17 09:14:52 -08:00
sadlil e075e2e633 Use kube-dns:1.11.0 2017-01-17 08:37:24 -08:00
Wojciech Tyczynski 61f2201304 Really enable etcd3 2017-01-17 15:57:43 +01:00
Kubernetes Submit Queue 936a94f0a8 Merge pull request #40012 from Crassirostris/fluentd-liveness-probe-sync
Automatic merge from submit-queue (batch tested with PRs 39911, 40002, 39969, 40012, 40009)

Sync fluentd daemonset liveness probe with static pod liveness probe

Syncing change from https://github.com/kubernetes/kubernetes/pull/39949

Should also be cherry-picked
2017-01-17 06:46:58 -08:00
Mik Vyatskov 83df5b8495 Remove library copying from fluentd image 2017-01-17 15:00:48 +01:00
Kubernetes Submit Queue 002cdfa1ae Merge pull request #39861 from Traum-Ferienwohnungen/hostname_as_nodename
Automatic merge from submit-queue

Use $HOSTNAME as node.name by default

**What this PR does / why we need it**:
Allows to identify elasticsearch instances more easily.
As $HOSTNAME of a pod is unique, this should be no problem.
2017-01-17 04:57:09 -08:00
Mik Vyatskov 5b96233423 Sync fluentd daemonset liveness probe with static pod liveness probe 2017-01-17 13:29:54 +01:00
Alexander Block ee8caaf3e1 Make CACHEBUST for hyperkube build optional 2017-01-17 12:41:26 +01:00
Janis Meybohm 6b3284acd2 Use $HOSTNAME as node.name by default
Allows to identify elasticsearch instances more easily.
As $HOSTNAME of a pod is unique, this should be no problem.
2017-01-17 08:38:53 +01:00
Jordan Liggitt 264dbf0daf
Remove direct kubecfg RBAC grant 2017-01-16 14:12:15 -05:00
Jordan Liggitt 7e98e06e48
Include system:masters group in the bootstrap admin client certificate 2017-01-16 14:01:24 -05:00
Kubernetes Submit Queue 06c610e276 Merge pull request #39949 from Crassirostris/fluentd-liveness-probe-fix
Automatic merge from submit-queue (batch tested with PRs 38592, 39949, 39946, 39882)

Remove fluentd buffers if fluentd is stuck

Fluentd now stores its buffers on disk for the resiliency. However, if buffer is corrupted, fluentd will be restarting forever.

Following change will make fluentd liveness probe delete buffers if fluentd is stuck for more than X minutes (15 by default).
2017-01-16 10:37:40 -08:00
Mik Vyatskov edf1ffc074 Remove fluentd buffers if fluentd is stuck 2017-01-16 13:47:23 +01:00
Jeff Grafton b9e060a630 Update scripts to look for binary artifacts in bazel-bin/ 2017-01-13 16:17:48 -08:00
Jeff Grafton bc4b6ac397 Build release tarballs in bazel and add `make bazel-release` rule 2017-01-13 16:17:44 -08:00
Jordan Liggitt d94bb26776
Conditionally write token file entries 2017-01-13 17:59:46 -05:00
Kubernetes Submit Queue 31483bf546 Merge pull request #39770 from ixdy/ubuntu-slim-base-image
Automatic merge from submit-queue

Update images that use ubuntu-slim base image to :0.6

**What this PR does / why we need it**: `ubuntu-slim:0.4` is somewhat old, being based on Ubuntu 16.04, whereas `ubuntu-slim:0.6` is based on Ubuntu 16.04.1.

**Special notes for your reviewer**: I haven't pushed any of these images yet, so I expect all of the e2e builds to fail. If we're happy with the changes, I can push the images and then re-trigger tests.

**Release note**:

```release-note
NONE
```

cc @aledbf as FYI
2017-01-12 20:39:13 -08:00
Kubernetes Submit Queue ae04755d71 Merge pull request #39827 from MrHohn/addon-manager-v6.2
Automatic merge from submit-queue

Update kubectl to stable version for Addon Manager

Bumps up Addon Manager to v6.2, below images are pushed:
- gcr.io/google-containers/kube-addon-manager:v6.2
- gcr.io/google-containers/kube-addon-manager-amd64:v6.2
- gcr.io/google-containers/kube-addon-manager-arm:v6.2
- gcr.io/google-containers/kube-addon-manager-arm64:v6.2
- gcr.io/google-containers/kube-addon-manager-ppc64le:v6.2
- gcr.io/google-containers/kube-addon-manager-s390x:v6.2

@mikedanese 

cc @ixdy
2017-01-12 15:54:24 -08:00
Kubernetes Submit Queue d50c027d0c Merge pull request #39537 from liggitt/legacy-policy
Automatic merge from submit-queue (batch tested with PRs 39803, 39698, 39537, 39478)

include bootstrap admin in super-user group, ensure tokens file is correct on upgrades

Fixes https://github.com/kubernetes/kubernetes/issues/39532

Possible issues with cluster bring-up scripts:

- [x] known_tokens.csv and basic_auth.csv is not rewritten if the file already exists
  * new users (like the controller manager) are not available on upgrade
  * changed users (like the kubelet username change) are not reflected
  * group additions (like the addition of admin to the superuser group) don't take effect on upgrade
  * this PR updates the token and basicauth files line-by-line to preserve user additions, but also ensure new data is persisted
- [x] existing 1.5 clusters may depend on more permissive ABAC permissions (or customized ABAC policies). This PR adds an option to enable existing ABAC policy files for clusters that are upgrading

Follow-ups:
- [ ] both scripts are loading e2e role-bindings, which only be loaded in e2e tests, not in normal kube-up scenarios
- [ ] when upgrading, set the option to use existing ABAC policy files
- [ ] update bootstrap superuser client certs to add superuser group? ("We also have a certificate that "used to be" a super-user. On GCE, it has CN "kubecfg", on GKE it's "client"")
- [ ] define (but do not load by default) a relaxed set of RBAC roles/rolebindings matching legacy ABAC, and document how to load that for new clusters that do not want to isolate user permissions
2017-01-12 15:06:31 -08:00
Zihong Zheng f62be637c8 Update kubectl to stable version for Addon Manager 2017-01-12 13:49:13 -08:00
Aleksandra Malinowska 043e809b8f update heapster version to 1.3.0-beta.0 2017-01-12 13:42:31 +01:00
Jeff Grafton 1c2ea28080 Update images that use ubuntu-slim base image to :0.6 2017-01-11 15:07:04 -08:00
Jordan Liggitt 968b0b30cf
Update token users if needed 2017-01-11 17:21:12 -05:00
Jordan Liggitt 21b422fccc
Allow enabling ABAC authz 2017-01-11 17:20:51 -05:00
Jordan Liggitt 1fe517e96a
Include admin in super-user group 2017-01-11 17:20:42 -05:00
Kubernetes Submit Queue 12e8271cd3 Merge pull request #33584 from marketlogicsoftware/kayrus/enable_elk_k8s_metadata
Automatic merge from submit-queue

Enable kubernetes_metadata by default for ELK stack

Looks like it was accidentally removed and was not restored back in this PR https://github.com/kubernetes/kubernetes/pull/29883
Because actually this plugin still exists in the image, but new ELK deployment don't allow you to index namespaces, pod names, etc.
2017-01-11 12:19:42 -08:00
Kubernetes Submit Queue 04326905b8 Merge pull request #39721 from euank/rkt-api-restart
Automatic merge from submit-queue (batch tested with PRs 39731, 39662, 39721)

container-linux: restart rkt-api on failure

This works around a flake I saw which had the same root cause as
https://github.com/coreos/rkt/issues/3513.

This will potentially help reduce the impact of such future problems as
well.

```release-note
NONE
```
2017-01-11 11:00:52 -08:00
Kubernetes Submit Queue 9814369ea1 Merge pull request #39662 from rf232/dashboard-v1.5.1
Automatic merge from submit-queue (batch tested with PRs 39731, 39662, 39721)

Update dashboard version to v1.5.1

**What this PR does / why we need it**:
Latest Dashboard developments, including a CSRF issue in the dashboard POST handlers

**Release note**:
```
Set Dashboard UI version to v1.5.1
```
2017-01-11 11:00:50 -08:00
kayrus 8435d19982 Enable kubernetes_metadata by default for ELK stack 2017-01-11 14:08:01 +01:00
Euan Kemp eeef293ee2 container-linux: restart rkt-api on failure
This works around a flake I saw which had the same root cause as
https://github.com/coreos/rkt/issues/3513.

This will potentially help reduce the impact of such future problems as
well.
2017-01-11 00:25:14 -08:00
Kubernetes Submit Queue ebc8e40694 Merge pull request #39691 from yujuhong/bump_timeout
Automatic merge from submit-queue (batch tested with PRs 39694, 39383, 39651, 39691, 39497)

Bump container-linux and gci timeout for docker health check

The command `docker ps` can take longer time to respond under heavy load or
when encountering some known issues. In these cases, the containers are running
fine, so aggressive health check could cause serious disruption. Bump the
timeout to 60s to be consistent with the debian-based containerVM.

This addresses #38588
2017-01-10 21:25:16 -08:00
Kubernetes Submit Queue 3f2a02cf98 Merge pull request #39383 from liggitt/bind-check
Automatic merge from submit-queue (batch tested with PRs 39694, 39383, 39651, 39691, 39497)

Allow rolebinding/clusterrolebinding with explicit bind permission check

Fixes https://github.com/kubernetes/kubernetes/issues/39176
Fixes https://github.com/kubernetes/kubernetes/issues/39258

Allows creating/updating a rolebinding/clusterrolebinding if the user has explicitly been granted permission to perform the "bind" verb against the referenced role/clusterrole (previously, they could only bind if they already had all the permissions in the referenced role via an RBAC role themselves)

```release-note
To create or update an RBAC RoleBinding or ClusterRoleBinding object, a user must:
1. Be authorized to make the create or update API request
2. Be allowed to bind the referenced role, either by already having all of the permissions contained in the referenced role, or by having the "bind" permission on the referenced role.
```
2017-01-10 21:25:13 -08:00
Kubernetes Submit Queue addc6cae4a Merge pull request #38212 from mikedanese/kubeletauth
Automatic merge from submit-queue (batch tested with PRs 38212, 38792, 39641, 36390, 39005)

Generate a kubelet CA and kube-apiserver cert-pair for kubelet auth.

cc @cjcullen
2017-01-10 19:48:09 -08:00
Jeff Grafton 19aafd291c Always --pull in docker build to ensure recent base images 2017-01-10 16:21:05 -08:00
Yu-Ju Hong 4e87973a9b Bump container-linux and gci timeout for docker health check
The command `docker ps` can take longer time to respond under heavy load or
when encountering some known issues. In these cases, the containers are running
fine, so aggressive health check could cause serious disruption. Bump the
timeout to 60s to be consistent with the debian-based containerVM.
2017-01-10 13:07:21 -08:00
Kubernetes Submit Queue d7ce8b80ee Merge pull request #39607 from deads2k/rbac-35-e2e-permission-typos
Automatic merge from submit-queue (batch tested with PRs 39628, 39551, 38746, 38352, 39607)

fix e2e kubelet binding

Fixes #39543

This limits scope of the kubelet.  It was an oversight before.  Hopefully we won't end up chasing permissions again.
2017-01-10 11:54:21 -08:00
Jordan Liggitt 6057a2ca76
Remove kubekins as cluster-admin 2017-01-10 14:34:33 -05:00
Piotr Szczesniak da7b81c4d8 Added owners to monitoring and logging related directories 2017-01-10 12:14:10 +01:00
Rob Franken 59ef8a4739 update dashboard version to v1.5.1 2017-01-10 11:57:21 +01:00
deads2k 60daaa3cca fix e2e kubelet binding 2017-01-09 07:39:10 -05:00
Mik Vyatskov 57ec7b77fd Fix fluentd-gcp image config by avoiding processing its own logs 2017-01-09 10:05:33 +01:00
Bowei Du 75c29adbaa Update DNS readme to point to the new code repository 2017-01-06 13:08:59 -08:00
Bowei Du b5c0fd5837 Update image references to the output of the kubernetes/dns project 2017-01-06 12:57:41 -08:00
Kubernetes Submit Queue 4881341f8c Merge pull request #39520 from shyamjvs/add-etcd-events-log
Automatic merge from submit-queue (batch tested with PRs 39318, 39520)

Added etcd-events to cluster logging

Fixes #38983 

@kubernetes/sig-scalability-misc @wojtek-t @gmarek
2017-01-06 12:22:09 -08:00
Shyam Jeedigunta 9bb636e9f8 Added etcd-events to cluster logging 2017-01-06 10:28:48 +01:00
Kubernetes Submit Queue 3414ec3e13 Merge pull request #39472 from DjangoPeng/docker-fix
Automatic merge from submit-queue

Fix unpack docker.tar.gz failure bug

In centOS, there is a conflict bug of docker. We save docker as [docker.tar.gz](https://github.com/kubernetes/kubernetes/blob/master/cluster/centos/build.sh#L57) in function `download-releases()`, but uncompress [docker.tgz](https://github.com/kubernetes/kubernetes/blob/master/cluster/centos/build.sh#L98) in function `unpack-releases()`.
2017-01-05 22:22:07 -08:00
Jeff Grafton 20d221f75c Enable auto-generating sources rules 2017-01-05 14:14:13 -08:00
Jeff Grafton f8288ed1f3 Create BUILD files in top-level subdirectories 2017-01-05 14:14:12 -08:00
Kubernetes Submit Queue 8ef6902516 Merge pull request #39451 from euank/remove-abac
Automatic merge from submit-queue

cluster/cl: move abac to rbac

See #39092

We based off of GCI in the brief time where it was using abac.

fixes #39395

cc @yifan-gu 

**Release note**:
```release-note
NONE
```
2017-01-05 12:31:17 -08:00
DjangoPeng 61076ede9e fix unpack docker.tar.gz failure bug 2017-01-06 00:17:36 +08:00
Kubernetes Submit Queue f4a8713088 Merge pull request #36229 from wojtek-t/bump_etcd_version
Automatic merge from submit-queue (batch tested with PRs 36229, 39450)

Bump etcd to 3.0.14 and switch to v3 API in etcd.

Ref #20504

**Release note**:

```release-note
Switch default etcd version to 3.0.14.
Switch default storage backend flag in apiserver to `etcd3` mode.
```
2017-01-04 17:36:06 -08:00
Euan Kemp c1afc4a3d8 cluster/cl: move abac to rbac
See #39092

We based off of GCI in the brief time where it was using abac.
2017-01-04 16:10:59 -08:00
Mike Danese 3ab0e37cc6 implement upgrades 2017-01-04 11:45:57 -08:00
Kubernetes Submit Queue b202ebb7a5 Merge pull request #38433 from yuanying/remove-needless-config
Automatic merge from submit-queue (batch tested with PRs 38433, 36245)

Remove needless env var in OpenStack provider

**What this PR does / why we need it**:

If we use openstack provider to set up k8s cluster using kube-up script,
`TENANT_ID` environment variable is needed.
But to configure `TENANT_ID` is very annoying because this value is not static by each env.
This patch uses `TENANT_NAME` instead of `TENANT_ID`

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```

Since `TENANT_NAME` is unique if we use keystone v2 api,
so `TENANT_ID` is not needed if `TENANT_NAME` is provided
to configure OpenStack provider.
And also to set `TENANT_ID` is annoying to develop, because
`TENANT_ID` is not static by each environment.
This patch remove dependency of `TENANT_ID` and simply use
`TENANT_NAME`.
2017-01-03 23:28:07 -08:00
CJ Cullen d0997a3d1f Generate a kubelet CA and kube-apiserver cert-pair for kubelet auth.
Plumb through to kubelet/kube-apiserver on gci & cvm.
2017-01-03 14:30:45 -08:00