Automatic merge from submit-queue
Remove all MAINTAINER statements in the codebase as they are deprecated
**What this PR does / why we need it**:
ref: https://github.com/docker/docker/pull/25466
**Release note**:
```release-note
Remove all MAINTAINER statements in Dockerfiles in the codebase as they are deprecated by docker
```
@ixdy @thockin (who else should be notified?)
Automatic merge from submit-queue
create kuberentes-discovery image
Creates an image for `kubernetes-discovery` since this is the API registration, aggregation, and proxy image.
This update includes significant refactoring. It moves almost all of the
logic into bash scripts, modeled after the `gci` cluster scripts.
The primary differences between the two are the following:
1. Use of the `/opt/kubernetes` directory over `/home/kubernetes`
2. Support for rkt as a runtime
3. No use of logrotate
4. No use of `/etc/default/`
5. No logic related to noexec mounts or gci-specific firewall-stuff
from etcd.sh split the start process into validate fucntion + start function so that the validate piece can be reused elsewhere. the up-cluster script has been changed to remove duplicate docker logic to the one used in buid-tools/common.sh and the validate etcd function is now used here.
moved docker daemon check function to util.sh and made function name changes and upstream changes.
Automatic merge from submit-queue
[Federation][init-11.2] use USE_KUBEFED env var to choose bw old and new federation deployment
This is continuation of #35961
USE_KUBEFED variable is used for deploying federation control plane. if not defined, federation will be brought up using old method i.e scripts.
Have verified that federation comes up using the old method, using following steps
```
$ export FEDERATION=true
$ export E2E_ZONES="asia-east1-c"
$ export FEDERATION_PUSH_REPO_BASE=gcr.io/<my-project>
$ KUBE_RELEASE_RUN_TESTS=n KUBE_FASTBUILD=true go run hack/e2e.go -v -build
$ build-tools/push-federation-images.sh
$ go run hack/e2e.go -v --up
```
Should merge #35961 before this PR
@madhusudancs
Automatic merge from submit-queue
Migrated fluentd addon to daemon set
fix#23224
supersedes #23306
``` release-note
Migrated fluentd addon to daemon set
```
This allows pause to reap zombies in the upcoming Shared PID namespace
(#1615). Uses the better defined sigaction() instead of signal() for all
signals both for consistency (SIGCHLD handler avoids SA_RESTART) and to
avoid the implicit signal()->sigaction() translation of various libc
versions.
Also makes warnings errors and includes a tool to make orphaned zombies
for manual testing.
Automatic merge from submit-queue
Update `gcloud docker` commands to use `gcloud docker -- ARGS`
We can then avoid the following warning:
```
WARNING: The '--' argument must be specified between gcloud specific args on the left and DOCKER_ARGS on the right. IMPORTANT: previously, commands allowed the omission of the --, and unparsed arguments were treated as implementation args. This usage is being deprecated and will be removed in March 2017.
This will be strictly enforced in March 2017. Use 'gcloud beta docker' to see new behavior.
```
If you delete a source file, we want to reflect that in the build container. We
only use --delete going that one way as we don't want to accidentally delete
files in the user's source tree.
We can then avoid the following warning:
```
WARNING: The '--' argument must be specified between gcloud specific args on the left and DOCKER_ARGS on the right. IMPORTANT: previously, commands allowed the omission of the --, and unparsed arguments were treated as implementation args. This usage is being deprecated and will be removed in March 2017.
This will be strictly enforced in March 2017. Use 'gcloud beta docker' to see new behavior.
```
Signed-off-by: Jess Frazelle <acidburn@google.com>
Automatic merge from submit-queue
Check for rsync and give friendlier message
Fixes#34300.
Not sure if #34309 is the same issue. Hopefully it is the same issue.
Automatic merge from submit-queue
Make sure rsync.sh is executable inside the build image
I kept having the build fail:
```console
$ make quick-release
+++ [1006 18:13:44] Verifying Prerequisites....
+++ [1006 18:13:44] Building Docker image kube-build:build-d3c60cf83f-3-v1.6.3-9
+++ [1006 18:13:54] Creating data container kube-build-data-d3c60cf83f-3-v1.6.3-9
+++ [1006 18:13:55] Syncing sources to container
!!! [1006 18:16:01] Could not connect to rsync container. See build/README.md for setting up remote Docker engine.
make: *** [quick-release] Error 1
```
`docker ps` revealed the issue:
```console
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
75c2a3c40cb3 kube-build:build-d3c60cf83f-3-v1.6.3-9 "/rsyncd.sh" 6 seconds ago Exited (126) 5 seconds ago kube-rsync-d3c60cf83f-3-v1.6.3-9
3eb215e41f36 kube-build:build-d3c60cf83f-3-v1.6.3-9 "chown -R 85078.5000 " 8 seconds ago Exited (0) 6 seconds ago kube-build-data-d3c60cf83f-3-v1.6.3-9
5a2707af2ccd 882577c54f67 "/bin/sh -c 'cd ${K8S" 7 days ago Exited (2) 7 days ago stupefied_goldberg
$ docker logs 75c2a3c40cb3
/bin/bash: /rsyncd.sh: Permission denied
```
I'm not sure why this works on Jenkins but not on my machine.
We were using netcat to try and figure out when the rsync container is ready. Now we instead use rsync itself. I suspect that there was a race condition with some versions of Docker where it would accept connections and then close them during container start.
This fixes#34214 (I think)
This was broken by #30787. A stray bash `source` caused an undefined variable reference.
Apparently the federation images have a parallel nad different "release" path
that isn't tested by the pre-checkin tests.
make-generated-{protobuf,runtime}.sh was doing some really nasty stuff with how
the build container was managed in order to copy results out. Since we have
more flexibility to grab results out of the build container, we can now avoid
all of this. Ideally we wouldn't have `hack` calling `build` at all, but we
aren't there yet.
We also add "version" to all docker images and containers
This version is to be incremented manually when we change the shape of the build
image (like changing the golang version or the set of volumes in the data
container). This will delete all older versions of images and containers when
the version is different.
Automatic merge from submit-queue
Use patched golang1.7.1 for cross-builds targeting darwin
This PR extends #32517 to use the patched go1.7.1 introduced by that PR to build all darwin targets (e.g. kubectl).
This is necessary because binaries built with earlier versions of Go regularly segfault on macOS Sierra (see #32999 and #33070).
This solution is somewhat hacky, but we intend to cherry-pick this to 1.4, and switching all of 1.4 to build with go1.7.1 is very high risk.
I haven't yet pushed the cross build image yet, so this will fail to build. Will test locally and update with results.
First step of fixing #33801.
cc @luxas @pwittrock @david-mcmahon @liggitt @smarterclayton @jfrazelle @Starefossen @gerred
Automatic merge from submit-queue
Bump up addon kube-dns to v20 for graceful termination
Below images are built and pushed:
- gcr.io/google_containers/kubedns-amd64:1.8
- gcr.io/google_containers/kubedns-arm:1.8
- gcr.io/google_containers/kubedns-arm64:1.8
- gcr.io/google_containers/kubedns-ppc64le:1.8
Both kubedns and dnsmasq are bumped up in the manifest files.
@thockin @bprashanth
Automatic merge from submit-queue
Add separate build process for node test.
This PR is part of https://github.com/kubernetes/kubernetes/pull/31093. However, because currently node e2e is built on `KUBE_TEST_PLATFORMS`, which includes linux/amd64, darwin/amd64, windows/amd64 and linux/arm, it caused #32251 to fail.
In fact, node e2e is running on the same node with kubelet, and it also has built-in apiserver, etcd and namespace controller. All of them are only built on `KUBE_SERVER_PLATFORMS`, so node e2e should also only be built on those platforms.
```
KUBE_SERVER_PLATFORMS=(
linux/amd64
linux/arm
linux/arm64
)
```
This PR added a separate build process for node e2e to address this.
@vishh Do you need this for v1.4? because this blocks your #32251. /cc @dchen1107
Automatic merge from submit-queue
Use a patched golang version for building linux/arm
Fixes: #29904
Right now, linux/arm is broken because of an internal limitation in Go.
I've filed an issue for it here: https://github.com/golang/go/issues/17028
The affected binaries of this limitation are hyperkube and kube-apiserver, which are the largest binaries.
And when we now have a patched go 1.7.1 version for building "unsupported" but important architectures (ref: https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/multi-platform.md), we should also include the patch for ppc64le and start building ppc64le again.
As soon as @laboger has the patch I need up on Github, I'll include ppc64le to this PR and we'll merge it
TODO:
- [ ] ~~Update the PR with patches for ppc64le at the same time @luxas~~
- [x] Push the new kube-cross image @ixdy
- [x] Run a full `make release` before to verify nothing breaks @luxas + @ixdy
- [ ] Cherrypick into the 1.4 branch @luxas + (who?)
@lavalamp @smarterclayton @ixdy @rsc @davecheney @wojtek-t @jfrazelle @bradfitz @david-mcmahon @pwittrock
Automatic merge from submit-queue
kubectl version -c has been deprecated, use --client instead
```
Flag shorthand -c has been deprecated, please use --client instead.
```
Automatic merge from submit-queue
Deprecate release infrastructure and doc - moved to kubernetes/release
Part 2 of https://github.com/kubernetes/release/pull/1
This PR finalizes the split between the main kubernetes repo and the release tooling now under kubernetes/release.
ref #16529
Automatic merge from submit-queue
Update build docs to include path for scripts.
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
This fix updates the build docs (`build/README.md`) to include the path of `build/` for shell scripts (like `run.sh`, `shell.sh`).
The reason is that while trying to follow the `build/README.md` to build the kubernetes, it is not obvious that all the scripts, e.g., `run.sh make`, `shell.sh`, etc. needs to be executed from the root directory (vs. executed from the `build/` directory).
In other words, the executation should be:
```
build/run.sh make
build/make-clean.sh
...
```
This fix adds `build/` so that it is easy for user to follow the steps.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
```
Automatic merge from submit-queue
Separate federation build.sh into development and deployment scripts.
The idea behind this separation is that it provides a clear distinction
between the dev environment and the prod environment. The
deploy/deploy.sh script will be shipped to the users, but
develop/develop.sh will be purely for development purposes and won't
be part of a release distribution.
Purely for developer convenience, all the deployment functionality is
made available through the develop/develop.sh script.
This change also copies deploy/* files into the release distribution.
cc @kubernetes/sig-cluster-federation @colhom
```release-note
Federation can now be deployed using the `federation/deploy/deploy.sh` script. This script does not depend on any of the development environment shell library/scripts. This is an alternative to the current `federation-up.sh`/`federation-down.sh` scripts. Both the scripts are going to co-exist in this release, but the `federation-up.sh`/`federation-down.sh` scripts might be removed in a future release in favor of `federation/deploy/deploy.sh` script.
```
Automatic merge from submit-queue
[Federation] Downsize the release binary distribution v2.
Second attempt of PR #29632.
There are two things that this PR does:
1. It removes `federation-apiserver` and `federation-controller-manager` from binaries and docker_wrapped_binaries target lists.
2. Build the docker image for `hyperkube` on-the-fly while pushing the federation images.
```release-note
Federation binaries and their corresponding docker images - `federation-apiserver` and `federation-controller-manager` are now folded in to the `hyperkube` binary. If you were using one of these binaries or docker images, please switch to using the `hyperkube` version. Please refer to the federation manifests - `federation/manifests/federation-apiserver.yaml` and `federation/manifests/federation-controller-manager-deployment.yaml` for examples.
```
cc @kubernetes/sig-cluster-federation @colhom
Fixes Issue #28633
Automatic merge from submit-queue
Build and push kube-dns for 1.4 release.
Fix#31355.
Following docker images had been uploaded:
gcr.io/google_containers/kubedns-amd64:1.7
gcr.io/google_containers/kubedns-arm:1.7
gcr.io/google_containers/kubedns-arm64:1.7
Build for ppc64le is disabled by default, and it failed to be built using:
`KUBE_BUILD_PPC64LE=y make release`
I'm still working on making the ppc64le build. Updates will be added following this thread.
@girishkalele @thockin
Also build the hyperkube docker image on-the-fly.
This is only a temporary fix until the proposal in issue
https://github.com/kubernetes/kubernetes/issues/28630 is implemented.
Also, the new build/deployment method completely obviates this step.
We use debian image instead of busybox and do not build hyperkube as a
static binary yet. Wait until PR
https://github.com/kubernetes/kubernetes/pull/26028 is merged to build
static hyperkube binaries.
This fix updates the build docs to include the path of `build/` for
shell scripts. The reason is that while trying to follow the `build/README.md`,
it is not obvious that all the scripts, e.g., `run.sh make`, `shell.sh`,
etc. needs to be executed from the root directory (vs. executed from the
`build/` directory). In other words,
the executation should be:
```
build/run.sh make
build/make-clean.sh
...
```
This fix adds `build/` so that it is easy for user to follow the steps.
The idea behind this separation is that it provides a clear distinction
between the dev environment and the prod environment. The
deploy/deploy.sh script will be shipped to the users, but
develop/develop.sh will be purely for development purposes and won't
be part of a release distribution.
Purely for developer convenience, all the deployment functionality is
made available through the develop/develop.sh script.
This change also copies deploy/* files into the release distribution.
Automatic merge from submit-queue
Disable linux/ppc64le compilation by default
Work-around for #30384.
I'm still testing this locally to see if it actually works. The build is slow. (PR Jenkins won't tell us whether this fixes ppc.)
cc @Random-Liu @spxtr @david-mcmahon @luxas
Automatic merge from submit-queue
Fix subtle build breakage
Repro case:
$ make clean generated_files
$ hack/update-generated-protobuf.sh
This would complain about not finding `fmt`, and it was indicating the wrong
GOROOT. The problem was that the first step built binaries for generating
code, which *embeds* the value of GOROOT into the binary. The whole tree was
bind-mounted into the build container and then JUST the dockerized dir was
mounted over it. The in-container build tried to use the existing binaries,
but GOROOT is wrong.
This change whites-out the whole _output dir.
I first made just an anonymous volume for _output, but docker makes that as
root, which means I can't write to it from our non-root build. So I just put
it in the data container. This seems to work. The biggest change this makes
is that the $GOPATH/bin/ and $GOPATH/pkg/ dirs will persist across dockerized
builds.
NB: this requires a `make clean` to activate.
@lavalamp @jbeda @quinton-hoole @david-mcmahon
Repro case:
$ make clean generated_files
$ hack/update-generated-protobuf.sh
This would complain about not finding `fmt`, and it was indicating the wrong
GOROOT. The problem was that the first step built binaries for generating
code, which *embeds* the value of GOROOT into the binary. The whole tree was
bind-mounted into the build container and then JUST the dockerized dir was
mounted over it. The in-container build tried to use the existing binaries,
but GOROOT is wrong.
This change whites-out the whole _output dir.
I first made just an anonymous volume for _output, but docker makes that as
root, which means I can't write to it from our non-root build. So I just put
it in the data container. This seems to work. The biggest change this makes
is that the $GOPATH/bin/ and $GOPATH/pkg/ dirs will persist across dockerized
builds.
This commit removes a part of common.sh script which copied
contrib/ sources for enabled contribs, which resulted in the
duplicated files inside tarball.
Fixes#30150
Automatic merge from submit-queue
Install go-bindata in cross-build image
Another follow-up to #25584.
We need `go-bindata` to create `test/e2e/generated`, and downloading it with `go get` at build time is painful for a variety of reasons. We can just include it in the cross-build image and not worry about it, especially as it updates very infrequently.
This fixes `hack/update-generated-protobuf.sh` as well.
cc @jayunit100 @soltysh
This allows us to start building real dependencies into Makefile.
Leave old hack/* scripts in place but advise to use 'make'. There are a few
rules that call things like 'go run' or 'build/*' that I left as-is for now.
Automatic merge from submit-queue
build: fixed ${KUBE_ROOT} prefix for build scripts
Running `./make-build-image.sh` command inside the `build` directory doesn't work:
```sh
$ cd build
$ ./make-build-image.sh
./../build/common.sh: line 32: hack/lib/init.sh: No such file or directory
```
This PR adds `${KUBE_ROOT}` prefix for the `source` bash function. Also I added braces to unify the code style.
[]()
If the docker-machine certificates get in a bad state, the current behavior
causes an infinite loop waiting for `docker-machine env` to return. Now it will
echo the certificate error and prompt the user to regenerate.
This logs a false "error" message, so it's time to go. It was needed to ensure
nobody has stale build images laying around, but that was quite a while ago, so
it's probably safe now.
Automatic merge from submit-queue
Bump skydns godeps to latest
Update Godeps for github.com/skynetservices/skydns and miekg/dns.
Bump kubedns version to 1.6 with latest skynetservices/skydns code
Built kube-dns for all architectures and pushed containers to gcr.io.
Automatic merge from submit-queue
Substitute federation_domain_map parameter with its value in node bootstrap scripts.
This PR also removes the substitution code we added to the build scripts.
**Release Note**
```release-note
If you use one of the kube-dns replication controller manifest in `cluster/saltbase/salt/kube-dns`, i.e. `cluster/saltbase/salt/kube-dns/{skydns-rc.yaml.base,skydns-rc.yaml.in}`, either substitute one of `__PILLAR__FEDERATIONS__DOMAIN__MAP__` or `{{ pillar['federations_domain_map'] }}` with the corresponding federation name to domain name value or remove them if you do not support cluster federation at this time. If you plan to substitute the parameter with its value, here is an example for `{{ pillar['federations_domain_map'] }`
pillar['federations_domain_map'] = "- --federations=myfederation=federation.test"
where `myfederation` is the name of the federation and `federation.test` is the domain name registered for the federation.
```
cc @erictune @kubernetes/sig-cluster-federation @MikeSpreitzer @luxas
[]()
Automatic merge from submit-queue
Add upgrade Docker VM
Add an Error Message to upgarde your Docker VM if needed, example output:
```bash
+++ [0622 13:19:48] No docker host is set. Checking options for setting one...
+++ [0622 13:19:49] docker-machine was found.
+++ [0622 13:19:49] A Docker host using docker-machine named 'kube-dev' is ready to go!
Can't connect to 'docker' daemon. please fix and retry.
Possible causes:
- On Mac OS X, DOCKER_HOST hasn't been set. You may need to:
- Create and start your VM using docker-machine or boot2docker:
- docker-machine create -d virtualbox --virtualbox-memory 4096 --virtualbox-cpu-count -1 kube-dev
- boot2docker init && boot2docker start
- Set your environment variables using:
- eval $(docker-machine env kube-dev)
- $(boot2docker shellinit)
- On Linux, user isn't in 'docker' group. Add and relogin.
- Something like 'sudo usermod -a -G docker jscheuermann'
- RHEL7 bug and workaround: https://bugzilla.redhat.com/show_bug.cgi?id=1119282#c8
- On Linux, Docker daemon hasn't been started or has crashed.
!!! Error in hack/../hack/update-generated-protobuf.sh:53
'return 1' exited with status 1
Call stack:
1: hack/../hack/update-generated-protobuf.sh:53 main(...)
Exiting with status 1
Updating generated-protobuf FAILED
$docker info
Error response from daemon: client is newer than server (client API version: 1.24, server API version: 1.23)
```
After running `docker-machine upgrade kube-dev` everything is fine again. So we should add a hint in the error message that this can also happen.
Automatic merge from submit-queue
Add support for Docker for MacOS
With Docker for MacOS [public beta](https://docs.docker.com/docker-for-mac) you don't need docker-machine on MacOS to build kubernetes instead you can use docker "natively". Per Default Docker for MacOS will be installed to `/Applications/Docker.app/Contents/MacOS/Docker` so if Docker for Mac is installed we should use the native version.
I tested it locally with `15.5.0 Darwin Kernel Version 15.5.0` and Docker version `1.12.0-rc2`
Bowei Du
