Pick up some code from https://github.com/heptio/kube-conformance
Fix up build scripts for the new conformance image
Fix Header template and Copyright to make verify job go green
update README and add execute permissions for script
Change-Id: Ib6509acd816cc2fb3a516bfb8e0ff9e32bff8f79
Automatic merge from submit-queue (batch tested with PRs 67950, 68195). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Remove e2e-image-puller
**What this PR does / why we need it**:
A long time ago, We added the image prepulling as a workaround due to
the overwhelming amount of flake caused by pulling during the tests.
This functionality has been broken for a while now when we switched to a
COS image where mounting `docker` binary into `busybox` stopped working.
So we just have dead code we should clean up.
Change-Id: I538171a5c1d9361eee7f9e0a99655b88b1721e3e
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#63355
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
A long time ago, We added the image prepulling as a workaround due to
the overwhelming amount of flake caused by pulling during the tests.
This functionality has been broken for a while now when we switched to a
COS image where mounting `docker` binary into `busybox` stopped working.
So we just have dead code we should clean up.
Change-Id: I538171a5c1d9361eee7f9e0a99655b88b1721e3e
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Add ability to build with runtime coverage instrumentation
**What this PR does / why we need it**:
This PR adds the ability to instrument a subset of kubernetes binaries to report code coverage information. The specific use-case is to help determine coverage of our end-to-end Conformance tests, as well as provide data that can be used to help determine where to focus. This PR focuses on making it possible to build with instrumentation; collecting and using the generated coverage data will be done in later PRs. For more details as to the intent, see the [design doc](https://docs.google.com/document/d/1FKMBFxz7vtA-6ZgUkA47F8m6yR00fwqLcXMVJqsHt0g/edit?usp=sharing) (google doc; requires kubernetes-dev membership).
Specifically, this PR adds a new `KUBE_BUILD_WITH_COVERAGE` make variable, which when set will cause `kube-apiserver`, `kube-controller-manager`, `kube-scheduler`, `kube-proxy` and `kubelet` to be built with coverage instrumentation. These coverage-instrumented binaries will flush coverage information to disk every five seconds, defaulting to a temporary directory unless the `KUBE_COVERAGE_FILE` environment variable is set at launch, in which case it will write to that file instead.
The mechanism used to achieve coverage instrumentation is to build the targeted binaries as "unit tests" with coverage enabled, and then rigging the unit tests to just execute the binary's usual entry point. This is implemented only for the bash build system.
/sig testing
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Add spiffxp to OWNERS files in bash-heavy dirs
I'm comfortable approving changes in hack/, I think I still need
to build up a corpus of reviews in build/ and cluster/ before I'm
comfortable asking for those rights. I'm willing to be voluntold
otherwise if existing approvers feel differently.
Approvers, WDYT?
```release-note
NONE
```
I'm comfortable approving changes in hack/, I think I still need
to build up a corpus of reviews in build/ and cluster/ before I'm
comfortable asking for those rights
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove rescheduler since scheduling DS pods by default scheduler is moving to beta
**What this PR does / why we need it**:
remove rescheduler since scheduling DS pods by default scheduler is moving to beta
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64725
**Special notes for your reviewer**:
**Release note**:
```release-note
Remove rescheduler since scheduling DS pods by default scheduler is moving to beta.
```
Automatic merge from submit-queue (batch tested with PRs 67298, 67518, 67635, 67673). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
build/debian-iptables: make: support arguments for base and tag
**What this PR does / why we need it**:
Change debian-iptables makefile to support arguments for base image and tag.
**Special notes for your reviewer**:
Tested locally.
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 67396, 67097, 67395, 67365, 67099). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update to debian-iptables v10.1 and hyperkube-base 0.10.1
**What this PR does / why we need it**: these images are based on the `debian-base` 0.3.2 images, which include CVE fixes (#67026) and permission fixes of the qemu-ARCH-static helper binary (#67026, #67283).
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 66177, 66185, 67136, 67157, 65065). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Drop kube-aggregator container image from release
**What this PR does / why we need it**:
First version of this was only standalone, now the code is integrated into kube-apiserver. So the standalone binary and
container are no longer useful.
Change-Id: Ib9369de66b4ecb3451f73ba2a252526d6615b96f
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Remove unused binary and container image for kube-aggregator. The functionality is already integrated into the kube-apiserver.
```
First version of this was only standalone, now the code is being
vendored by anyone who wants to use it. So the standalone binary and
container are no longer useful.
Change-Id: Ib9369de66b4ecb3451f73ba2a252526d6615b96f
Automatic merge from submit-queue (batch tested with PRs 66190, 66871, 66617, 66293, 66891). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Unset CDPATH in build scripts
**What this PR does / why we need it**:
Currently trying to build k8s fails if you have CDPATH set in your env. This is a minor change that allows building k8s without manually unsetting the variable.
```release-note
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Adds a build example with KUBE_BUILD_PLATFORMS
**What this PR does / why we need it**:
I didn't manage to find any guides about how to build the specific binary for the specific platform in docs (in this repo and in the community repo). Also it was quite hard for me to follow build scripts. At the end, I found help on the kubernetes-dev channel in Slack:`KUBE_BUILD_PLATFORMS` seems to do what I want. I think, would be great to have it documented somehow: The "Key scripts" section of the build doc seems to be a good place to mention it.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 66341, 66405, 66403, 66264, 66447). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Move api rules list under api-approvers-owned package
Additions to this list should be rare and carefully reviewed
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 66152, 66406, 66218, 66278, 65660). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update crictl to v1.11.1.
Update `crictl` to v1.11.1 to fix several bugs. Release note: https://github.com/kubernetes-incubator/cri-tools/releases/tag/v1.11.1
@kubernetes/sig-node-pr-reviews @kubernetes/sig-cluster-lifecycle-pr-reviews
@kubernetes/sig-gcp-pr-reviews
Signed-off-by: Lantao Liu <lantaol@google.com>
```release-note
Update crictl to v1.11.1.
```
Automatic merge from submit-queue (batch tested with PRs 64181, 65737). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
API linter framework and namesMatch API rule
**What this PR does / why we need it**:
Bump kube-openapi dependency to use the [API linter framework](https://github.com/kubernetes/kube-openapi/pull/83) in k/k OpenAPI spec generation procedure.
Currently one API rule is enforced:
"Go field names must be CamelCase. JSON field names must be camelCase. Other than capitalization of the initial letter, the two should almost always match. No underscores nor dashes in either."
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#65244
**Special notes for your reviewer**:
Most code change in this PR was generated (~1700 lines). Please see commits for detail.
**Release note**:
```release-note
NONE
```
/sig api-machinery
/cc @pwittrock @mbohlool
check in existing API rule violations;
the Make rule fails if generated violation report differs from the
checked-in violation file and prints error message;
add documentation.
Automatic merge from submit-queue (batch tested with PRs 65677, 65711, 65150, 65726). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update to go1.10.3
**What this PR does / why we need it**: updates to build with go1.10.3; see list of changes [here](https://github.com/golang/go/issues?q=milestone%3AGo1.10.3).
We'll probably want to cherrypick this to release-1.11 as well.
**Release note**:
```release-note
Update to use go1.10.3
```
/assign @BenTheElder @cblecker
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
bazel: kubelet deb package postinstall step
**What this PR does / why we need it**:
Presently bazel build .deb (kubelet.deb specifically) does not auto-restart after installation. Adding a postinst control file fixes this.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
kubernetes/kubeadm#944
**Special notes for your reviewer**:
@chuckha
```release-note
bazel deb package bugfix: The kubeadm deb package now reloads the kubelet after installation
```
Automatic merge from submit-queue (batch tested with PRs 65319, 64513, 65474, 65601, 65634). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Allow custom manifests in GCP master setup
Add a hook in GCE setup script to allow using custom manifests on master, so we can decouple some GKE changes from k8s. Note that this PR just adds a hook there is no change in default behavior.
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 65338, 64535). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove cloud-controller-manager deb from releases
**What this PR does / why we need it**:
Stop including cloud controller manager deb from Kubernetes releases.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes # https://github.com/kubernetes/kubernetes/issues/65337
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 65301, 65291, 65307, 63845, 65313). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Adds cri-tools as a dependency to kubeadm deb/rpms
Closeskubernetes/kubeadm#942
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
**What this PR does / why we need it**:
This PR adds cri-tools as a dependency to kubeadm. Kubeadm code now requires crictl to exist on the system or an ignorable error will be thrown during init.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixeskubernetes/kubeadm#942
**Special notes for your reviewer**:
I'm making a similar PR in the release repo as well.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add OWNERS file for rpm packages to mirror the debs
**What this PR does / why we need it**:
Makes the rpm bazel package OWNERS match the deb one
Adds @chuckha as reviewer and @timothysc as an approver for both
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@kubernetes/sig-cluster-lifecycle-pr-reviews @ixdy
Automatic merge from submit-queue (batch tested with PRs 65230, 57355, 59174, 63698, 63659). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Bumps cri-tools version to GA in bazel build
Fixeskubernetes/kubeadm#903
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
**What this PR does / why we need it**:
This PR bumps cri-tools version to appropriate version for k8s 1.11
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixeskubernetes/kubeadm#903
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Overhaul Makefile for generated code
Calculate all go dependencies in a single Go program, rather than a complicated series of `go list` invocations and `Makefile` logic. This simplifies the
Makefile dramatically.
Also adds tests to ensure the dependencies stay correct. This test is pretty slow, though.
Also fixes a few anomalies found by the test.
Fixes#54371
```release-note
NONE
```
Fixeskubernetes/kubeadm#903
The version in the spec file must be hardcoded because the `{filename}`
syntax in the tar command is a substitutaion by bazel which happens before
the spec file does its ownt %{} substitutions. Therefore, if we use %{version}
bazel will not be able to find {crictl-v%{version}-...} in its list of
files so no substituation will take place and the file will not be found.
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kube-proxy and kube-scheduler should not depend on pkg/kubectl
remove dependent on pkg/kubectl/cmd/util in kube-proxy and kube-scheduler
**Release note**:
```release-note
NONE
```
Bazel walks through the Kubernetes repo to add vendor targets for OpenAPI generation. `SkippedPaths` is used to skip the paths such as `_examples`. However, it doesn't work as desired, because it matches for `_` at the beginning of the path, so paths like `vendor\..\_example` are picked up by the generator. This PR fixes this.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix pkg_rpm rules for bazel 0.13+
**What this PR does / why we need it**: next step in addressing https://github.com/kubernetes/kubernetes/issues/63108; we can use the substitutions supported by bazel 0.13+ to get the `pkg_rpm` rules to work properly again.
I've also added a filegroup to allow building all of the RPMs easily with `bazel build //build/rpms`. Note that since these are manual, `bazel build //...` will still skip building them.
**Release note**:
```release-note
NONE
```
/assign @BenTheElder
cc @sigma
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update all script shebangs to use /usr/bin/env interpreter instead of /bin/interpreter
This is required to support systems where bash doesn't reside in /bin (such as NixOS, or the *BSD family) and allow users to specify a different interpreter version through $PATH manipulation.
https://www.cyberciti.biz/tips/finding-bash-perl-python-portably-using-env.html
```release-note
Use /usr/bin/env in all script shebangs to increase portability.
```
Automatic merge from submit-queue (batch tested with PRs 61601, 62881, 63159). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
wire print flags through get.go
**Release note**:
```release-note
NONE
```
Adds the PrintFlags pattern to `get.go`.
Prerequisite to ongoing server-side printing work.
cc @soltysh @deads2k
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Tag pkg_rpm rules as manual
**What this PR does / why we need it**: Mitigation step for #63108. Nothing depends on these rules yet, and tagging as manual prevents `make bazel-build` from building them, too.
**Release note**:
```release-note
NONE
```
/assign @BenTheElder
/cc @sigma
Automatic merge from submit-queue (batch tested with PRs 62495, 63003, 62829, 62151, 62002). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Build cleanup with go 1.10
Simplify our build/test scripts now that go 1.10 is in play. It has caching that seems to actually work.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62495, 63003, 62829, 62151, 62002). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add a GCS mirror to WORKSPACE URLs, along with a script to update them.
**What this PR does / why we need it**:
We get occasional flakes when WORKSPACE downloads fail. This should make that much more unlikely, by having two independent endpoints to try.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63007, 62919, 62669, 62860). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add unit test for configure-helper.sh.
**What this PR does / why we need it**:
Add a framework for unit-testing configure-helper.sh.
configure-helper.sh plays a critical role in initializing clusters both on GCE and GKE. It is currently, over 2K lines of code, yet it has no unit test coverage.
This PR proposes a framework/approach on how to provide test coverage for this component.
Notes:
1. Changes to configure-helper.sh itself were necessary to enable sourcing of this script for the purposes of testing.
2. As POC api_manifest_test.go covers the logic related to the initialization of apiserver when integration with KMS was requested. The hope is that the same approach could be extended to the rest of the script.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add test file for pkg/kubectl/cmd/rollout/rollout_pause.go file
new: pkg/kubectl/cmd/rollout/rollout_pause_test.go
modified: pkg/kubectl/cmd/rollout/BUILD
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
This will help avoid flakes when initializing WORKSPACE, since there are
two sources for each file. Any errors downloading or failed hashes cause
bazel to try the next url in the series.
Change #61633 doesn't seem quite correct, and breaks bazel builds.
This patch makes sure that:
- there is no conflict between directory files and build targets
- rpm files are properly packaged after being installed
Automatic merge from submit-queue (batch tested with PRs 62694, 62569, 62646, 61633, 62433). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Patch for #61632, add `/etc/sysconfig/kublet` and supporting wiring.
**What this PR does / why we need it**:
These are just simple config structure changes to better meet expectations for RHEL users. Helps `KUBELET_EXTRA_ARGS` parameters more easily survive across version upgrades without requiring manual merge of the service file.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61632
**Special notes for your reviewer**:
It may be beneficial to give this treatment to other `systemd`-based systems, but I do not have the intellectual authority to do so.
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 62748, 60536, 62300, 62661, 62731). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
bzl: build --config unit should build with race enabled
This allows preheating unit test runs with:
bazel build -k --config unit -- //... -//vendor/...
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62748, 60536, 62300, 62661, 62731). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Wire PrintFlags through delete, replace, run commands
**Release note**:
```release-note
NONE
```
Wires PrintFlags through the `delete`, `replace`, and `run` commands.
All three commands grouped in this patch as they depend on DeleteOptions.
~~Tagged as WIP for now, as I still need to update tests.~~
cc @soltysh @deads2k
Automatic merge from submit-queue (batch tested with PRs 62436, 62499, 62664, 62685, 62660). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kazel: skip third_party/etcd.*
**What this PR does / why we need it**: fixes the issue encountered in https://github.com/kubernetes/kubernetes/pull/62151#discussion_r179322174.
Basically, we git ignore anything under `third_party/etcd.*`, but if a `BUILD` file ends up in there somehow, it'll confuse kazel and generate invalid entries in top-level `BUILD` files that we do check in. We can avoid this issue by instructing kazel to ignore these directories.
**Release note**:
```release-note
NONE
```
/assign @mikedanese @thockin @BenTheElder
Automatic merge from submit-queue (batch tested with PRs 62192, 61866, 62206, 62360). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove rkt references in the codebase
```release-note
None
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add documentation around build reproducibility and SOURCE_DATE_EPOCH
**What this PR does / why we need it**: makes some of our notes about reproducibility a bit more discoverable.
**Release note**:
```release-note
NONE
```
/assign @BenTheElder
Mac OS X was renamed to OS X in 2012, and then renamed again to macOS in 2016.
I suggest that the current name should be used when referencing the Apple OS.
Automatic merge from submit-queue (batch tested with PRs 61096, 61955, 61542, 60597). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update to use go1.10.1
**What this PR does / why we need it**:
Update to use go1.10.1
**Release note**:
```release-note
Update to use go1.10.1
```
Automatic merge from submit-queue (batch tested with PRs 61848, 61188, 56363, 61357, 61838). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add udev to hyperkube and bump versions
**What this PR does / why we need it**: Adds udev to the hyperkube to fix GCE and OpenStack volume mounts.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61356Fixes#43515
Fixes https://github.com/coreos/bugs/issues/2385
**Special notes for your reviewer**:
/cc @kubernetes/sig-node-bugs @kubernetes/sig-node-pr-reviews
/cc @ixdy
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61378, 60915, 61499, 61507, 61478). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove unused htpasswd
This file is not used anywhere.
Further more, this is an error in this file:
```
$ flake8 third_party/htpasswd/htpasswd.py --select=F
third_party/htpasswd/htpasswd.py:46:13: F821 undefined name 'time'
```
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60759, 60531, 60923, 60851, 58717). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix build scripts to work in case there are spaces in directory names.
**What this PR does / why we need it**:
Fix kubernetes build scripts to work in case the source directory is in a directory path which contains a space.
You can prepare such a directory like this:
```
$ mkdir '/tmp/test dir/'
$ cd '/tmp/test dir/'
$ git clone https://github.com/kubernetes/kubernetes.git
$ cd kubernetes
```
Then, without the fix:
```
$ KUBE_FASTBUILD=true KUBE_RELEASE_RUN_TESTS=n build/release.sh
cat: /tmp/test: No such file or directory
cat: dir/kubernetes/build/build-image/cross/VERSION: No such file or directory
cat: /tmp/test: No such file or directory
cat: dír/kubernetes/build/build-image/VERSION: No such file or directory
+++ [0307 18:10:33] Verifying Prerequisites....
cp: target '/tmp/test dir/kubernetes/_output/images/kube-build:build-7c7cd10a18--/Dockerfile' is not a directory
!!! [0307 18:10:33] Call tree:
!!! [0307 18:10:33] 1: build/release.sh:35
kube::build::build_image(...)
!!! Error in build/../build/common.sh:454
Error in build/../build/common.sh:454. '((i<3-1))' exited with status 1
Call stack:
1: build/../build/common.sh:454 kube::build::build_image(...)
2: build/release.sh:35 main(...)
Exiting with status 1
```
With the fix the compilation succeeds. The fix is done adding double quotes to required places (and also just in case to other places where shellcheck recommended adding them).
Note that this fix doesn't as such help with the official make-based build: it's tricky to make makefiles work with targets with spaces in their names.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
**Special notes for your reviewer**:
This PR needs pretty thorough review since this touches the core build scripts.
**Release note**:
```release-note
NONE
```
Fix kubernetes build scripts to work in case the source directory is in
a directory path which contains a space.
You can prepare such a directory like this:
$ mkdir '/tmp/test dir/'
$ cd '/tmp/test dir/'
$ git clone https://github.com/kubernetes/kubernetes.git
$ cd kubernetes
Then, without the fix:
$ KUBE_FASTBUILD=true KUBE_RELEASE_RUN_TESTS=n build/release.sh
cat: /tmp/test: No such file or directory
cat: dir/kubernetes/build/build-image/cross/VERSION: No such file or directory
cat: /tmp/test: No such file or directory
cat: dír/kubernetes/build/build-image/VERSION: No such file or directory
+++ [0307 18:10:33] Verifying Prerequisites....
cp: target '/tmp/test dir/kubernetes/_output/images/kube-build:build-7c7cd10a18--/Dockerfile' is not a directory
!!! [0307 18:10:33] Call tree:
!!! [0307 18:10:33] 1: build/release.sh:35
kube::build::build_image(...)
!!! Error in build/../build/common.sh:454
Error in build/../build/common.sh:454. '((i<3-1))' exited with status 1
Call stack:
1: build/../build/common.sh:454 kube::build::build_image(...)
2: build/release.sh:35 main(...)
Exiting with status 1
With the fix the compilation succeeds. The fix is done adding double
quotes to required places (and also just in case to other places where
shellcheck recommended adding them).
Note that this fix doesn't help with the official make-based build:
it's tricky to make makefiles work with targets with spaces in their
names.
This file is not used anywhere.
Further more, this is an error in this file:
```
$ flake8 third_party/htpasswd/htpasswd.py --select=F
third_party/htpasswd/htpasswd.py:46:13: F821 undefined name 'time'
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add //test/e2e/... and //test/integration/... to //build/visible_to:COMMON_testing
**What this PR does / why we need it**: adds the common test packages `//test/e2e/...` and `//test/integration/...` to the `COMMON_testing` package group.
The immediate need for this is that #60580 unintentionally broke the integration tests under bazel since it added a new dependency on `//pkg/kubectl/cmd/util` without adding an appropriate entry to `//build/visible_to:COMMON_testing`, and the integration tests aren't currently run under bazel for PRs.
This PR fixes this breakage and proactively fixes future breakages by adding all packages under `//test/e2e` or `//test/integration` to `COMMON_testing`.
**Release note**:
```release-note
NONE
```
/assign @BenTheElder @monopole
/kind bug
/priority important-soon
/sig testing
cc @kad
Automatic merge from submit-queue (batch tested with PRs 60530, 60618). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add support for `make verify WHAT=typecheck`
This will be used to make the typecheck job emit junit and avoids duplicating the rest of the verify logic.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60157, 60337, 60246, 59714, 60467). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
don't (remote) cache release-tars
**What this PR does / why we need it**: Disables Bazel *remote* caching for release tarballs. These are large, low hit rate build outputs. Before we enable remote caching for *builds* we should tag these type of outputs. `pull-kubernetes-bazel-test` already has this enabled and dropped in execution time by 70-80%
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**: See https://github.com/kubernetes/test-infra/issues/6808 for details on the remote caching. This should not affect local builds. See the following for docs on this change: https://docs.bazel.build/versions/master/remote-caching.html#exclude-specific-targets-from-using-the-remote-cache
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 50724, 59025, 59710, 59404, 59958). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Require boilerplate on Bazel Skylark source files
**What this PR does / why we need it**: `.bzl` files are also source code, so they should probably have the boilerplate text too.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add ipset binary for IPVS, context: Fixes#57321
**What this PR does / why we need it**: Add ipset binary in debian-hyperkube-base which fixes issue 57321
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57321
**Special notes for your reviewer**:
**Release note**:
```release-note
Add ipset binary for IPVS to hyperkube docker image
```
We originally seperated build and test so that only 4 integration tests
would be run at a time, but we didn't want to slow down build, however
we didn't know --local_test_jobs existed. This achieves the same result
but more efficiently.
Automatic merge from submit-queue (batch tested with PRs 59705, 59207, 59677). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
build: fix a logic error in shell script.
**What this PR does / why we need it**: It looks like that there's a logic error in `build/common.sh`. The return value of a `docker inspect` command is not checked properly, since the value being assigned is actually the previous command's return value (a `0`, because `local` always returns `0` when used like this).
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57824, 58806, 59410, 59280). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
2nd try at using a vanity GCR name
The 2nd commit here is the changes relative to the reverted PR. Please focus review attention on that.
This is the 2nd attempt. The previous try (#57573) was reverted while we
figured out the regional mirrors (oops).
New plan: k8s.gcr.io is a read-only facade that auto-detects your source
region (us, eu, or asia for now) and pulls from the closest. To publish
an image, push k8s-staging.gcr.io and it will be synced to the regionals
automatically (similar to today). For now the staging is an alias to
gcr.io/google_containers (the legacy URL).
When we move off of google-owned projects (working on it), then we just
do a one-time sync, and change the google-internal config, and nobody
outside should notice.
We can, in parallel, change the auto-sync into a manual sync - send a PR
to "promote" something from staging, and a bot activates it. Nice and
visible, easy to keep track of.
xref https://github.com/kubernetes/release/issues/281
TL;DR:
* The new `staging-k8s.gcr.io` is where we push images. It is literally an alias to `gcr.io/google_containers` (the existing repo) and is hosted in the US.
* The contents of `staging-k8s.gcr.io` are automatically synced to `{asia,eu,us)-k8s.gcr.io`.
* The new `k8s.gcr.io` will be a read-only alias to whichever regional repo is closest to you.
* In the future, images will be promoted from `staging` to regional "prod" more explicitly and auditably.
```release-note
Use "k8s.gcr.io" for pulling container images rather than "gcr.io/google_containers". Images are already synced, so this should not impact anyone materially.
Documentation and tools should all convert to the new name. Users should take note of this in case they see this new name in the system.
```
This is the 2nd attempt. The previous was reverted while we figured out
the regional mirrors (oops).
New plan: k8s.gcr.io is a read-only facade that auto-detects your source
region (us, eu, or asia for now) and pulls from the closest. To publish
an image, push k8s-staging.gcr.io and it will be synced to the regionals
automatically (similar to today). For now the staging is an alias to
gcr.io/google_containers (the legacy URL).
When we move off of google-owned projects (working on it), then we just
do a one-time sync, and change the google-internal config, and nobody
outside should notice.
We can, in parallel, change the auto-sync into a manual sync - send a PR
to "promote" something from staging, and a bot activates it. Nice and
visible, easy to keep track of.
Automatic merge from submit-queue (batch tested with PRs 58562, 56937). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add xfsprogs to hyperkube baseimage
**What this PR does / why we need it**:
adds `xfsprogs` to hyperkube image, so that XFS filesystem can be created on unformatted volumes.
```release-note
NONE
```
/sig node
/sig storage
The right place to assign the "docker inspect" return value is outside
of the subshell. The last return value was coming from something else
than the expected command.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Build using go1.9.3
**What this PR does / why we need it**: more minor bugfixes, which I guess we probably want. https://github.com/golang/go/issues?q=milestone%3AGo1.9.3
**Special notes for your reviewer**: I haven't built/pushed the crossbuild image yet, but will do so if we think this is a good idea. I don't have plans to rebuild the test images.
**Release note**:
```release-note
Build using go1.9.3.
```
Who are good people to review this?
/assign @cblecker @luxas
(for lack of better ideas)
Automatic merge from submit-queue (batch tested with PRs 57322, 57723, 58706, 59004, 58857). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Make the pause image a manifest list
**What this PR does / why we need it**:
Build and push manifest for kubernetes images
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubernetes/issues/57869
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
Davanum Srinivas