Automatic merge from submit-queue (batch tested with PRs 41023, 41031, 40947)
scrub aggregator names to eliminate discovery
Cleanup old uses of `discovery`. Also removes the legacy functionality.
@kubernetes/sig-api-machinery-misc @sttts
Automatic merge from submit-queue (batch tested with PRs 40980, 40985)
added short names for resources which are exposed during discovery
**What this PR does / why we need it**:
The changes add short names for resources. The short names will be delivered to kubectl during discovery.
Automatic merge from submit-queue (batch tested with PRs 40971, 41027, 40709, 40903, 39369)
Promote SubjectAccessReview to v1
We have multiple features that depend on this API:
SubjectAccessReview
- [webhook authorization](https://kubernetes.io/docs/admin/authorization/#webhook-mode)
- [kubelet delegated authorization](https://kubernetes.io/docs/admin/kubelet-authentication-authorization/#kubelet-authorization)
- add-on API server delegated authorization
The API has been in use since 1.3 in beta status (v1beta1) with negligible changes:
- Added a status field for reporting errors evaluating access
- A typo was discovered in the SubjectAccessReviewSpec Groups field name
This PR promotes the existing v1beta1 API to v1, with the only change being the typo fix to the groups field. (fixes https://github.com/kubernetes/kubernetes/issues/32709)
Because the API does not persist data (it is a query/response-style API), there are no data migration concerns.
This positions us to promote the features that depend on this API to stable in 1.7
cc @kubernetes/sig-auth-api-reviews @kubernetes/sig-auth-misc
```release-note
The authorization.k8s.io API group was promoted to v1
```
Automatic merge from submit-queue
local-up-cluster: fix instructions for set-credentials
After launching the services, local-up-cluster.sh tells the
user how to configure kubectl to access it. The instructions
for 'set-credentials' enable plain password auth, but the
services are configured to mandate client certificate auth.
As a result it is not possible to access the cluster with
the instructions printed.
The use of client certs by default was added in
commit a1b17db458
Author: Dr. Stefan Schimanski <sttts@redhat.com>
Date: Sat Nov 12 23:09:04 2016 +0100
Configure client certs in local-cluster-up.sh
and the instructions were correctly updated to refer to
client certificates.
The changed instructions were (mistakenly) reverted though
when the following commit was merged:
commit 72e0e91b5e
Author: xilabao <chenr.fnst@cn.fujitsu.com>
Date: Fri Dec 2 11:04:25 2016 +0800
change prompt for enabling RBAC on local-up-cluster
Fixes: #40192
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
After launching the services, local-up-cluster.sh tells the
user how to configure kubectl to access it. The instructions
for 'set-credentials' enable plain password auth, but the
services are configured to mandate client certificate auth.
As a result it is not possible to access the cluster with
the instructions printed.
The use of client certs by default was added in
commit a1b17db458
Author: Dr. Stefan Schimanski <sttts@redhat.com>
Date: Sat Nov 12 23:09:04 2016 +0100
Configure client certs in local-cluster-up.sh
and the instructions were correctly updated to refer to
client certificates.
The changed instructions were (mistakenly) reverted though
when the following commit was merged:
commit 72e0e91b5e
Author: xilabao <chenr.fnst@cn.fujitsu.com>
Date: Fri Dec 2 11:04:25 2016 +0800
change prompt for enabling RBAC on local-up-cluster
Fixes: #40192
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Automatic merge from submit-queue
Using API_HOST_IP to do apiserver health check.
In `hack/local-up-cluster.sh`, it's better to use `API_HOST_IP` to do apiserver health check.
Automatic merge from submit-queue
[Federation][kubefed] Add option to expose federation apiserver on nodeport service
**What this PR does / why we need it**:
This PR adds an option to kubefed to expose federation api server over nodeport. This can be useful to deploy federation in non-cloud environments. This PR is target to address #39271
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
[Federation] kubefed init learned a new flag, `--api-server-service-type`, that allows service type to be specified for the federation API server.
[Federation] kubefed init also learned a new flag, `--api-server-advertise-address`, that allows specifying advertise address for federation API server in case the service type is NodePort.
```
@kubernetes/sig-federation-misc @madhusudancs
Automatic merge from submit-queue
Add SIG to test owners
**What this PR does / why we need it**:
This PR adds a `sig` column to the test owners file generation script.
A problem experienced with the current owners file is that since members are auto-assigned there are times where tests are assigned to non-active users who don't follow up to notifications to fix flakes. By assigning a SIG to each test we can hold a group we know is active responsible for taking care of flakes it's less likely that flakes will fall through the cracks.
**Special notes for your reviewer**:
* A companion PR will go into *kubernetes/contrib* adding support for mungers parsing this new column.
* Another PR in contrib will add labeling GitHub flake issues with the appropriate SIG
* Currently SIGs are not labeled, this will be added in another PR where SIG determinations can be discussed
@saad-ali @pwittrock
Automatic merge from submit-queue (batch tested with PRs 40289, 40877, 40879, 39972, 40942)
Rename experimental-cgroups-per-pod flag
**What this PR does / why we need it**:
1. Rename `experimental-cgroups-per-qos` to `cgroups-per-qos`
1. Update hack/local-up-cluster to match `CGROUP_DRIVER` with docker runtime if used.
**Special notes for your reviewer**:
We plan to roll this feature out in the upcoming release. Previous node e2e runs were running with this feature on by default. We will default this feature on for all e2es next week.
**Release note**:
```release-note
Rename --experiemental-cgroups-per-qos to --cgroups-per-qos
```
Automatic merge from submit-queue (batch tested with PRs 40906, 40924, 40938, 40902, 40911)
print apiserver log location on apiserver error
**What this PR does / why we need it**:
Improve user experience. Attempt to direct user to logs of failing component.
**Special notes for your reviewer**:
In addition to failure, point to logs so that a user can attempt to self remedy and have more information available to debug immediately. A user may not know that the failing component has logs.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40696, 39914, 40374)
Convert hack/e2e.go to a test-infra/kubetest shim
Replaces `hack/e2e.go` for a shim that passes the args to `k8s.io/test-infra/kubetest`
Adds fejta to `hack/OWNERS`
Adds `e2e_test.go` for unit test coverage of the shim.
`Usage: go run hack/e2e.go [--get=true] [--old=1d] -- KUBETEST_ARGS`
In other words there is are `--get` and `--old` shim flags, which control how we upgrade `kubetest`, and a `--` to separate the shim args from the kubetest args, and the existing kubetest args like `--down` `--up`, etc. If only `KUBETEST_ARGS` are used then you can skip the `--` (although golang will complain about it).
Once this is ready to go I will update the kubekins-e2e image to copy this file from test-infra: https://github.com/kubernetes/test-infra/blob/master/jenkins/e2e-image/Dockerfile#L70
ref https://github.com/kubernetes/test-infra/issues/1475
Automatic merge from submit-queue (batch tested with PRs 40862, 40909)
[Federation][kubefed] Add option to disable persistence storage for etcd
**What this PR does / why we need it**:
This is part of updates to enable deployment of federation on non-cloud environments. This pr enables disabling persistent storage for etcd via kubefed.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#40617
**Special notes for your reviewer**:
**Release note**:
```
[Federation] Add --etcd-persistent-storage flag to kubefed to enable/disable persistent storage for etcd
```
cc: @kubernetes/sig-federation-bugs @madhusudancs
Automatic merge from submit-queue (batch tested with PRs 40864, 40666, 38382, 40874)
apply falls back to generic JSON patch computation if no go struct is registered for the target GVK
This PR is the master version of #40096 which is target 1.4 branch.
This PR is based on #40260
- [x] ensure subkey deletion works in CreateThreeWayJSONMergePatch
- [x] ensure type stomping works in CreateThreeWayJSONMergePatch
- [x] lots of tests for generic json patch computation
- [x] apply falls back to generic 3-way JSON merge patch if no go struct is registered for the target GVK
- [x] prevent generic apply patch computation between different apiVersions and/or kinds
- [x] make pruner generic (apply --prune works with TPR)
```release-note
apply falls back to generic 3-way JSON merge patch if no go struct is registered for the target GVK
```
Automatic merge from submit-queue (batch tested with PRs 40884, 40809, 40845, 40866, 40875)
update repo local config to allow redirects from gopkg.in
closes#39583
Automatic merge from submit-queue (batch tested with PRs 40812, 39903, 40525, 40729)
Add CI coverage for client-go and staging/copy.sh
Client-go master and `staging/copy.sh` keep breaking, leading to frustration of users outside of the inner refactoring circle. This PR adds minimal smoke testing for both.
**Note**: this enforces the split of PRs which change apimachinery and client-go in parallel (via some local "temporary commit" hacking).
- [x] @caesarxuchao ~~do we have to delete the example directory from the upstream repo?~~ merge https://github.com/kubernetes/test-infra/issues/1765
Automatic merge from submit-queue (batch tested with PRs 40812, 39903, 40525, 40729)
test/node_e2e: wire-in cri-enabled local testing
This commit wires-in the pre-existing `--container-runtime` flag for
local node_e2e testing.
This is needed in order to further skip docker specific testing
and validation.
Local CRI node_e2e can now be performed via
`make test-e2e-node RUNTIME=remote REMOTE=false`
which will also take care of passing the appropriate argument to
the kubelet.
This commit wires-in the pre-existing `--container-runtime` flag for
local node_e2e testing.
This is needed in order to further skip docker specific testing
and validation.
Local CRI node_e2e can now be performed via
`make test-e2e-node RUNTIME=remote REMOTE=false`
which will also take care of passing the appropriate arguments to
the kubelet.
Automatic merge from submit-queue
error strings should not end with punctuation
**What this PR does / why we need it**:
Delete the end punctuation of error strings
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
https://github.com/golang/go/wiki/CodeReviewComments#error-strings
**Release note**:
```release-note
```
Signed-off-by: yupeng <yu.peng36@zte.com.cn>
Automatic merge from submit-queue (batch tested with PRs 40529, 40630)
propagate explicit nulls in apply
Rebase of https://github.com/kubernetes/kubernetes/pull/35496 on top of https://github.com/kubernetes/kubernetes/pull/40260
The client-side propagation of the raw value is no longer needed, since the client is preserving the original object in unstructured form (explicit nulls are preserved).
Kept tests and CreateThreeWayMergePatch changes from https://github.com/kubernetes/kubernetes/pull/35496
```release-note
kubectl apply now supports explicitly clearing values not present in the config by setting them to null
```
- [x] Clean up orphaned objects in test-cmd to preserve pre- and post- conditions
- [x] improve CreateThreeWayMergePatch test to not filter based on string comparison to test name
Automatic merge from submit-queue
Add flag to node e2e test specifying location of ssh privkey
**What this PR does / why we need it**: in CI, the ssh private key is not always located at `$HOME/.ssh`, so it's helpful to be able to override it.
@krzyzacy here's my resurrected change. I'm not sure why I neglected to follow-through on it originally.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
genericapiserver: cut off more dependencies – episode 3
Compare commit subjects.
This is the first step to get `apiserver_test.go` and `watch_test.go` in `pkg/genericapiserver/endpoints` cutoff from k8s.io/kubernetes dependencies.
After this we have to sync client-go and then "episode 4" can go in.
approved based on #40363
Automatic merge from submit-queue (batch tested with PRs 39832, 40660)
Complete *-munge-docs.sh deprecation.
**What this PR does / why we need it**:\
Complete *-munge-docs.sh deprecation.
TOC generation now handled by https://github.com/kubernetes/release/pull/247
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
ref #38309
**Special notes for your reviewer**:
cc @bgrant0607 @thockin
Automatic merge from submit-queue
re-enable cascading deployment
I couldn't reproduce #40433 locally sooo trying on the test infra to make sure I'm not crazy
Automatic merge from submit-queue
Improve the multiarch situation; armel => armhf; reenable pcc64le; remove the patched golang
**What this PR does / why we need it**:
- Improves the multiarch situation as described in #38067
- Tries to bump to go1.8 for arm (and later enable ppc64le)
- GOARM 6 => GOARM 7
- Remove the golang 1.7 patch
- armel => armhf
- Bump QEMU version to v2.7.0
**Release note**:
```release-note
Improve the ARM builds and make hyperkube on ARM working again by upgrading the Go version for ARM to go1.8beta2
```
@kubernetes/sig-testing-misc @jessfraz @ixdy @jbeda @david-mcmahon @pwittrock
Automatic merge from submit-queue
run unit tests with go, not just bazel
Follow up to https://github.com/kubernetes/kubernetes/pull/40594, effectively reverts #39105.
I think we should run our unit tests using the official golang infrastructure. I have no objection to also running them in bazel (or running a subset in bazel), but I don't think that bazel should be the primary unit test method.
@smarterclayton @liggitt @ixdy @spxtr
Automatic merge from submit-queue
Temporary disable federation kubectl tests for secrets to unblock merge queue
Fixes https://github.com/kubernetes/kubernetes/issues/40521
cc @kubernetes/sig-federation-misc @deads2k
Automatic merge from submit-queue (batch tested with PRs 39223, 40260, 40082, 40389)
make kubectl generic commands work with unstructured objects
part of making apply, edit, label, annotate, and patch work with third party resources
fixes#35149fixes#34413
prereq of:
https://github.com/kubernetes/kubernetes/issues/35496https://github.com/kubernetes/kubernetes/pull/40096
related to:
https://github.com/kubernetes/kubernetes/issues/39906https://github.com/kubernetes/kubernetes/issues/40119
kubectl is currently decoding any resource it doesn't have compiled-in to a ThirdPartyResourceData struct, which means it computes patches using that struct, and would try to send a ThirdPartyResourceData object to the API server when running `apply`
This PR removes the behavior that decodes unknown objects into ThirdPartyResourceData structs internally, and fixes up the following generic commands to work with unstructured objects
- [x] apply
- [x] decode into runtime.Unstructured objects
- [x] successfully use `--record` with unregistered objects
- [x] patch
- [x] decode into runtime.Unstructured objects
- [x] successfully use `--record` with unregistered objects
- [x] describe
- [x] decode into runtime.Unstructured objects
- [x] implement generic describer
- [x] fix other generic kubectl commands to work with unstructured objects
- [x] label
- [x] annotate
follow-ups for pre-existing issues:
- [ ] `explain` doesn't work with unregistered resources
- [ ] remove special casing of federation group in clientset lookups, etc
- [ ] `patch`
- [ ] doesn't honor output formats when persisting to server (`kubectl patch -f svc.json --type merge -p '{}' -o json` doesn't output json)
- [ ] --local throws exception (`kubectl patch -f svc.json --type merge -p '{}' --local`)
- [ ] `apply`
- [ ] fall back to generic JSON patch computation if no go struct is registered for the target GVK (e.g. https://github.com/kubernetes/kubernetes/pull/40096)
- [ ] ensure subkey deletion works in CreateThreeWayJSONMergePatch
- [ ] ensure type stomping works in CreateThreeWayJSONMergePatch
- [ ] lots of tests for generic json patch computation
- [ ] prevent generic apply patch computation among different versions
- [ ] reconcile treatment of nulls with https://github.com/kubernetes/kubernetes/pull/35496
- [ ] `edit`
- [ ] decode into runtime.Unstructured objects
- [ ] fall back to generic JSON patch computation if no go struct is registered for the target GVK
Automatic merge from submit-queue (batch tested with PRs 40126, 40565, 38777, 40564, 40572)
Print warning about kubelet not running on darwin via local-up-cluster.sh
**What this PR does / why we need it**: As of the time of opening this PR, near the release of v.1.5, using `./hack/local-up-cluster.sh` claims to succeed and run kubelet, but logs, processes, and attempts to hit the service demonstrate that kubelet is NOT running. The issue has been found to be the hard requirements of cadvisor forcing kubelet to force exit when it is found to be unsupported as in the case of DARWIN.
My goal here is to bring the issue to attention to others and also help other devs avoid the issue.
**Release note**:
```NONE```
kubelet.go and kubelet's server.go file has minimal logic that keeps kubelet from breaking just because osx has not cadvisor support.
Automatic merge from submit-queue (batch tested with PRs 40239, 40397, 40449, 40448, 40360)
move the discovery and dynamic clients
Moved the dynamic client, discovery client, testing/core, and testing/cache to `client-go`. Dependencies on api groups we don't have generated clients for have dropped out, so federation, kubeadm, and imagepolicy.
@caesarxuchao @sttts
approved based on https://github.com/kubernetes/kubernetes/issues/40363
Automatic merge from submit-queue (batch tested with PRs 40239, 40397, 40449, 40448, 40360)
CRI: use more gogoprotobuf plugins
Generate marshaler/unmarshaler code should help improve the performance.
This addresses #40098
Automatic merge from submit-queue
Able to quick create a HA cluster by kube-up.sh centos provider
Make `kube-up.sh` `centos provider` support quick create a HA cluster, as I said above [#39430](https://github.com/kubernetes/kubernetes/issues/39430), it's more flexible than `kops` or `kubeadm` for some people in a limited network region.
I'm new to k8s dev, so if this pull request need to change, please let me know.
```release-note
Added support for creating HA clusters for centos using kube-up.sh.
```
Automatic merge from submit-queue (batch tested with PRs 38905, 40421)
Revert "Add the ability to edit fields within a config map."
Fixes#40396
This reverts commit 31eca372c9.
#38445 broke edit for all resources other than configmaps:
```
$ kubectl edit serviceaccount default
Error from server (NotFound): configmaps "serviceaccount" not found
```
also, `edit` is a generic command, we can't add subcommands that claim the `configmap` space and mess with the things resourcebuilder accepts
Fix: cannot get default master advertise address correctly
Set default value of NUM_MASTERS and NUM_NODES by MASTERS and NODES themself
Code cleanup and documented
Using runtime reconfiguration for etcd cluster instead of etcd discovery
Add exceptions for verify-flags
Automatic merge from submit-queue (batch tested with PRs 40335, 40320, 40324, 39103, 40315)
Splitting master/node services into separate charm layers
**What this PR does / why we need it**:
This branch includes a roll-up series of commits from a fork of the
Kubernetes repository pre 1.5 release because we didn't make the code freeze.
This additional effort has been fully tested and has results submit into
the gubernator to enhance confidence in this code quality vs. the single
layer, posing as both master/node.
To reference the gubernator results, please see:
https://k8s-gubernator.appspot.com/builds/canonical-kubernetes-tests/logs/kubernetes-gce-e2e-node/
Apologies in advance for the large commit however, we did not want to
submit without having successful upstream automated testing results.
This commit includes:
- Support for CNI networking plugins
- Support for durable storage provided by Ceph
- Building from upstream templates (read: kubedns - no more template
drift!)
- An e2e charm-layer to make running validation tests much simpler/repeatable
- Changes to support the 1.5.x series of Kubernetes
**Special notes for your reviewer**:
Additional note: We will be targeting -all- future work against upstream
so large pull requests of this magnitude will not occur again.
**Release note**:
```release-note
- Splits Juju Charm layers into master/worker roles
- Adds support for 1.5.x series of Kubernetes
- Introduces a tactic for keeping templates in sync with upstream eliminating template drift
- Adds CNI support to the Juju Charms
- Adds durable storage support to the Juju Charms
- Introduces an e2e Charm layer for repeatable testing efforts and validation of clusters
```
Automatic merge from submit-queue (batch tested with PRs 39260, 40216, 40213, 40325, 40333)
bazel: save git version in kubernetes.tar.gz
**What this PR does / why we need it**: adds a workspace status command for bazel (inspired by #36128) and then uses the saved values to generate the `version` file in `kubernetes.tar.gz`. We need this for `get-kube.sh` to work properly.
**Special notes for your reviewer**: I had to change a few things from #36128 - see comments for explanation.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Make verify-godeps.sh easier to debug
This does 3 things:
1. env var to retain the /tmp/godep
2. env var to use a specified /tmp/godep
3. rework of preload-dep to support more non-github things, like go4.org
This branch includes a rollup series of commits from a fork of the
kubernetes repository pre 1.5 release because we didn't make the code freeze.
This additional effort has been fully tested and has results submit into
the gubernator to enhance confidence in this code quality vs. the single
layer, posing as both master/node.
To reference the gubernator results, please see:
https://k8s-gubernator.appspot.com/builds/canonical-kubernetes-tests/logs/kubernetes-gce-e2e-node/
Apologies in advance for the large commit, however we did not want to
submit without having successful upstream automated testing results.
This commit includes:
- Support for CNI networking plugins
- Support for durable storage provided by ceph
- Building from upstream templates (read: kubedns - no more template
drift!)
- An e2e charm-layer to make running validation tests much simpler/repeatable
- Changes to support the 1.5.x series of kubernetes
Additional note: We will be targeting -all- future work against upstream
so large pull requests of this magnitude will not occur again.
Automatic merge from submit-queue
Optional configmaps and secrets
Allow configmaps and secrets for environment variables and volume sources to be optional
Implements approved proposal c9f881b7bb
Release note:
```release-note
Volumes and environment variables populated from ConfigMap and Secret objects can now tolerate the named source object or specific keys being missing, by adding `optional: true` to the volume or environment variable source specifications.
```
Automatic merge from submit-queue (batch tested with PRs 40328, 40057)
Allows kubectl create -f to filter by selector.
From #32544.
The underlying `seletor` infrastructrue was implemented by #32599. The test is basically copy-paste of the corresponding test for `kubectl apply -f -l`.
```release-note
kubectl create now accepts the label selector flag for filtering objects to create
```
Enforce the following limits:
12kb for total message length in container status
4kb for the termination message path file
2kb or 80 lines (whichever is shorter) from the log on error
Fallback to log output if the user requests it.
Automatic merge from submit-queue (batch tested with PRs 38445, 40292)
Add the ability to edit fields within a config map.
Addresses part of https://github.com/kubernetes/kubernetes/issues/36222
Example command:
```console
$ kubectl edit configmap foo --config-map-data=bar
```
Will open the data element named `bar` in the `ConfigMap` named `foo` in `$EDITOR`, the edited contents are then updated back to the config map.
@kubernetes/sig-cli
```release-note
Add a special purpose tool for editing individual fields in a ConfigMap with kubectl
```
Automatic merge from submit-queue
[Federation] Expose autoscaling apis through federation api server
This PR implements first part of federated pod autoscaler.
The issue to handle the whole feature is https://github.com/kubernetes/kubernetes/issues/38974
cc @kubernetes/sig-cluster-federation
@shashidharatd @kshafiee @deepak-vij
**Release note**:
```
federation users can now use federated autoscaling resources and create federated horizontalpodautoscalers
```
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
kubeadm: add optional self-hosted deployment
**What this PR does / why we need it**: add an optional self-hosted deployment type to `kubeadm`, for master components only, namely `apiserver`, `controller-manager` and `scheduler`.
**Which issue this PR fixes**: closes#38407
**Special notes for your reviewer**: /cc @aaronlevy @luxas @dgoodwin
**Release note**:
```release-note
kubeadm: add optional self-hosted deployment for apiserver, controller-manager and scheduler.
```
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
kubelet: storage: teardown terminated pod volumes
This is a continuation of the work done in https://github.com/kubernetes/kubernetes/pull/36779
There really is no reason to keep volumes for terminated pods attached on the node. This PR extends the removal of volumes on the node from memory-backed (the current policy) to all volumes.
@pmorie raised a concern an impact debugging volume related issues if terminated pod volumes are removed. To address this issue, the PR adds a `--keep-terminated-pod-volumes` flag the kubelet and sets it for `hack/local-up-cluster.sh`.
For consideration in 1.6.
Fixes#35406
@derekwaynecarr @vishh @dashpole
```release-note
kubelet tears down pod volumes on pod termination rather than pod deletion
```
Automatic merge from submit-queue (batch tested with PRs 40168, 40165, 39158, 39966, 40190)
CRI: upgrade protobuf to v3
For #38854, this PR upgrades CRI protobuf version to v3, and also updated related packages for confirming to new api.
**Release note**:
```
CRI: upgrade protobuf version to v3.
```
Automatic merge from submit-queue (batch tested with PRs 39772, 39831, 39481, 40167, 40149)
Add //hack:verify-boilerplate rule.
This pattern is working well in test-infra. I'll add the gofmt and go vet rules next.
Automatic merge from submit-queue
promote certificates api to beta
Mostly posting to see what breaks but also this API is ready to be promoted.
```release-note
Promote certificates.k8s.io to beta and enable it by default. Users using the alpha certificates API should delete v1alpha1 CSRs from the API before upgrading and recreate them as v1beta1 CSR after upgrading.
```
@kubernetes/api-approvers @jcbsmpsn @pipejakob
Automatic merge from submit-queue
Add authorization mode to kubeadm
This PR adds an option in `kubeadm` to allow a user to specify an [authorization plugin](https://kubernetes.io/docs/admin/authorization/). It defaults to RBAC.
Automatic merge from submit-queue
Build release tars using bazel
**What this PR does / why we need it**: builds equivalents of the various kubernetes release tarballs, solely using bazel.
For example, you can now do
```console
$ make bazel-release
$ hack/e2e.go -v -up -test -down
```
**Special notes for your reviewer**: this is currently dependent on 3b29803eb5, which I have yet to turn into a pull request, since I'm still trying to figure out if this is the best approach.
Basically, the issue comes up with the way we generate the various server docker image tarfiles and load them on nodes:
* we `md5sum` the binary being encapsulated (e.g. kube-proxy) and save that to `$binary.docker_tag` in the server tarball
* we then build the docker image and tag using that md5sum (e.g. `gcr.io/google_containers/kube-proxy:$MD5SUM`)
* we `docker save` this image, which embeds the full tag in the `$binary.tar` file.
* on cluster startup, we `docker load` these tarballs, which are loaded with the tag that we'd created at build time. the nodes then use the `$binary.docker_tag` file to find the right image.
With the current bazel `docker_build` rule, the tag isn't saved in the docker image tar, so the node is unable to find the image after `docker load`ing it.
My changes to the rule save the tag in the docker image tar, though I don't know if there are subtle issues with it. (Maybe we want to only tag when `--stamp` is given?)
Also, the docker images produced by bazel have the timestamp set to the unix epoch, which is not great for debugging. Might be another thing to change with a `--stamp`.
Long story short, we probably need to follow up with bazel folks on the best way to solve this problem.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 39625, 39842)
Add RBAC v1beta1
Add `rbac.authorization.k8s.io/v1beta1`. This scrubs `v1alpha1` to remove cruft, then add `v1beta1`. We'll update other bits of infrastructure to code to `v1beta1` as a separate step.
```release-note
The `attributeRestrictions` field has been removed from the PolicyRule type in the rbac.authorization.k8s.io/v1alpha1 API. The field was not used by the RBAC authorizer.
```
@kubernetes/sig-auth-misc @liggitt @erictune
Automatic merge from submit-queue
Remove packages which are now apimachinery
Removes all the content from the packages that were moved to `apimachinery`. This will force all vendoring projects to figure out what's wrong. I had to leave many empty marker packages behind to have verify-godep succeed on vendoring heapster.
@sttts straight deletes and simple adds
Automatic merge from submit-queue (batch tested with PRs 39911, 40002, 39969, 40012, 40009)
kubectl: fix rollback dryrun when version is not specified
@kubernetes/sig-cli-misc
Automatic merge from submit-queue (batch tested with PRs 39609, 39105)
Stop running most unit tests outside of bazel.
Lets not duplicate our efforts. The two I still run here are the two we currently skip in bazel. We should fix those.
This does 3 things:
1. env var to retain the /tmp/godep
2. env var to use a specified /tmp/godep
3. rework of preload-dep to support more non-github things, like go4.org
Automatic merge from submit-queue (batch tested with PRs 39803, 39698, 39537, 39478)
include bootstrap admin in super-user group, ensure tokens file is correct on upgrades
Fixes https://github.com/kubernetes/kubernetes/issues/39532
Possible issues with cluster bring-up scripts:
- [x] known_tokens.csv and basic_auth.csv is not rewritten if the file already exists
* new users (like the controller manager) are not available on upgrade
* changed users (like the kubelet username change) are not reflected
* group additions (like the addition of admin to the superuser group) don't take effect on upgrade
* this PR updates the token and basicauth files line-by-line to preserve user additions, but also ensure new data is persisted
- [x] existing 1.5 clusters may depend on more permissive ABAC permissions (or customized ABAC policies). This PR adds an option to enable existing ABAC policy files for clusters that are upgrading
Follow-ups:
- [ ] both scripts are loading e2e role-bindings, which only be loaded in e2e tests, not in normal kube-up scenarios
- [ ] when upgrading, set the option to use existing ABAC policy files
- [ ] update bootstrap superuser client certs to add superuser group? ("We also have a certificate that "used to be" a super-user. On GCE, it has CN "kubecfg", on GKE it's "client"")
- [ ] define (but do not load by default) a relaxed set of RBAC roles/rolebindings matching legacy ABAC, and document how to load that for new clusters that do not want to isolate user permissions
Automatic merge from submit-queue
Fix kubectl get -f <file> -o <nondefault printer> so it prints all items in the file
**What this PR does / why we need it**: Fix kubectl get -f <file> -o <nondefault printer> so it prints all the objects in the file, instead of just the first one. Also add a test for this feature.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#38907
**Special notes for your reviewer**:
**Release note**:
```release-note
```
cc @AdoHe @deads2k @liggitt @fabianofranz @kubernetes/kubectl @kubernetes/sig-cli-misc
Fix kubectl get -f <file> -o <nondefault printer> so it prints all the
objects in the file, instead of just the first one. Also add a test for
this feature.
Automatic merge from submit-queue (batch tested with PRs 39673, 39536, 39617, 39540, 39686)
Generate stable bindata output
fixes#30621
bindata generation produces spurious diffs when run with different go versions because gzip output is not stable between versions.
It also produces spurious diffs when invoked directly vs via make/go:generate.
This PR:
* adds data to bindata uncompressed (makes translation files human-readable, and the zip files were already compressed... for reference the file was 27.7 KB before, and 20.1 KB with this change)
* runs the generation from KUBE_ROOT, so it doesn't matter whether it is invoked directly or via go:generate
supercedes https://github.com/kubernetes/kubernetes/pull/39631
Automatic merge from submit-queue (batch tested with PRs 39486, 37288, 39477, 39455, 39542)
Allow missing keys in templates by default
Switch to allowing missing keys in jsonpath templates by default.
Add support for allowing/disallowing missing keys in go templates
(default=allow).
Add --allow-missing-template-keys flag to control this behavior (default=true /
allow missing keys).
Fixes#37991
@kubernetes/sig-cli-misc @kubernetes/api-reviewers @smarterclayton @fabianofranz @liggitt @pwittrock
Automatic merge from submit-queue (batch tested with PRs 39648, 38167, 39591, 39415, 39612)
Add hack/verify-readonly-packages.sh
Create a `.readonly` file in a package. Any change between `$KUBE_VERIFY_GIT_BRANCH` and `HEAD` will lead to output like:
```shell
$ hack/verify-readonly-packages.sh
Readonly packages changed compared to "master" branch: pkg/generated
```
This is part of https://github.com/kubernetes/kubernetes/issues/39528
Automatic merge from submit-queue (batch tested with PRs 39628, 39551, 38746, 38352, 39607)
Increasing times on reconciling volumes fixing impact to AWS.
#**What this PR does / why we need it**:
We are currently blocked by API timeouts with PV volumes. See https://github.com/kubernetes/kubernetes/issues/39526. This is a workaround, not a fix.
**Special notes for your reviewer**:
A second PR will be dropped with CLI cobra options in it, but we are starting with increasing the reconciliation periods. I am dropping this without major testing and will test on our AWS account. Will be marked WIP until I run smoke tests.
**Release note**:
```release-note
Provide kubernetes-controller-manager flags to control volume attach/detach reconciler sync. The duration of the syncs can be controlled, and the syncs can be shut off as well.
```
gzip output is not stable between go releases, so bindata.go shows spurious diffs when compressed
also, half the output is human readable, and the other half is already zip compressed
Switch to allowing missing keys in jsonpath templates by default.
Add support for allowing/disallowing missing keys in go templates
(default=allow).
Add --allow-missing-template-keys flag to control this behavior
(default=true / allow missing keys).
Automatic merge from submit-queue
add import cycle checking for staging to verify scripts
Adds an import cycle check to verify scripts since we can't yet use import-boss. See https://github.com/kubernetes/gengo/pull/27 for details, but since the rule is very simple and global, this ought to provide sufficient protection.
@kubernetes/sig-api-machinery-misc
Automatic merge from submit-queue (batch tested with PRs 36229, 39450)
Bump etcd to 3.0.14 and switch to v3 API in etcd.
Ref #20504
**Release note**:
```release-note
Switch default etcd version to 3.0.14.
Switch default storage backend flag in apiserver to `etcd3` mode.
```
Automatic merge from submit-queue
Adding test-federation-cmd.sh to test kubectl with federation apiserver
There are 2 parts to the PR:
* Adding ability to run kubectl tests for a subset of resources.
* Adding test-federation-cmd.sh that runs kubectl tests for resources that are supported by federation-apiserver.
cc @kubernetes/sig-federation @kubernetes/sig-api-machinery
```release-note
Adding kubectl tests for federation
```
Automatic merge from submit-queue (batch tested with PRs 39405, 39371)
hack/local-up-cluster.sh: fix typo in error message
This commit fixes typo in error message and also removes stale comment from `hack/local-up-cluster.sh`.
Automatic merge from submit-queue (batch tested with PRs 39022, 39331, 39070, 39344)
Add a build rule for the boilerplate unit test.
We recently added unit tests that just run whenever you run `hack/verify_boilerplate.sh`, which really isn't the right time to do that. This adds a build rule instead.
Automatic merge from submit-queue
Begin paths for internationalization in kubectl
This is just the first step, purposely simple so we can get the interface correct.
@kubernetes/sig-cli @deads2k
Automatic merge from submit-queue (batch tested with PRs 36751, 38968)
Convert * users/groups to system:authenticated group in ABAC
Part of enabling anonymous auth by default in 1.6 means protecting earlier policies that did not intend to grant access to anonymous users.
This modifies ABAC policies that match `user` or `group` `*` to only match authenticated users.
Docs PR to update examples to use `system:authenticated` or `system:unauthenticated` groups explicitly: https://github.com/kubernetes/kubernetes.github.io/pull/1992
```release-note
ABAC policies using "user":"*" or "group":"*" to match all users or groups will only match authenticated requests. To match unauthenticated requests, ABAC policies must explicitly specify "group":"system:unauthenticated"
```
Automatic merge from submit-queue
create kuberentes-discovery image
Creates an image for `kubernetes-discovery` since this is the API registration, aggregation, and proxy image.
Automatic merge from submit-queue
Coreos kube-up now with less cloud init
This update includes significant refactoring. It moves almost all of the
logic into bash scripts, modeled after the `gci` cluster scripts.
The reason to do this is:
1. Avoid duplicating the saltbase manifests by reusing gci's parsing logic (easier maintenance)
2. Take an incremental step towards sharing more code between gci/trusty/coreos, again for better maintenance
3. Pave the way for making future changes (e.g. improved rkt support, kubelet support) easier to share
The primary differences from the gci scripts are the following:
1. Use of the `/opt/kubernetes` directory over `/home/kubernetes`
2. Support for rkt as a runtime
3. No use of logrotate
4. No use of `/etc/default/`
5. No logic related to noexec mounts or gci-specific firewall-stuff
It will make sense to move 2 over to gci, as well as perhaps a few other small improvements. That will be a separate PR for ease of review.
Ref #29720, this is a part of that because it removes a copy of them.
Fixes#24165
cc @yifan-gu
Since this logic largely duplicates logic from the gci folder, it would be nice if someone closely familiar with that gave an OK or made sure I didn't fall into any gotchas related to that, so cc @andyzheng0831
Automatic merge from submit-queue (batch tested with PRs 38426, 38917, 38891, 38935)
Remove cluster/mesos from hack/verify-flags/exceptions.txt
`cluster/mesos` scripts was removed; so remove it from `hack/verify-flags/exceptions.txt`.
The diff was generated by `hack/verify-flags-underscore.py -e > hack/verify-flags/exceptions.txt`.
Automatic merge from submit-queue
Curating Owners: hack/jenkins
cc @rmmh @spxtr @ixdy @fejta
In an effort to expand the existing pool of reviewers and establish a
two-tiered review process (first someone lgtms and then someone
experienced in the project approves), we are adding new reviewers to
existing owners files.
If You Care About the Process:
------------------------------
We did this by algorithmically figuring out who’s contributed code to
the project and in what directories. Unfortunately, that doesn’t work
well: people that have made mechanical code changes (e.g change the
copyright header across all directories) end up as reviewers in lots of
places.
Instead of using pure commit data, we generated an excessively large
list of reviewers and pruned based on all time commit data, recent
commit data and review data (number of PRs commented on).
At this point we have a decent list of reviewers, but it needs one last
pass for fine tuning.
Also, see https://github.com/kubernetes/contrib/issues/1389.
TLDR:
-----
As an owner of a sig/directory and a leader of the project, here’s what
we need from you:
1. Use PR https://github.com/kubernetes/kubernetes/pull/35715 as an example.
2. The pull-request is made editable, please edit the `OWNERS` file to
remove the names of people that shouldn't be reviewing code in the
future in the **reviewers** section. You probably do NOT need to modify
the **approvers** section. Names asre sorted by relevance, using some
secret statistics.
3. Notify me if you want some OWNERS file to be removed. Being an
approver or reviewer of a parent directory makes you a reviewer/approver
of the subdirectories too, so not all OWNERS files may be necessary.
4. Please use ALIAS if you want to use the same list of people over and
over again (don't hesitate to ask me for help, or use the pull-request
above as an example)
Automatic merge from submit-queue (batch tested with PRs 38525, 38977)
add service status detection to kubernetes-discovery
This adds an inspection of services and endpoints before adding the APIService to kubernetes-discovery. This keeps a single bad API service from killing old clients since proxying will fail.
Automatic merge from submit-queue
conversion-gen: add --skip-unsafe flag
We should expose the SkipUnsafe option, for legacy compatability, so
that conversion-go can be used in other projects, and for platforms
where unsafe is not available.
Make unsafe code generation the default though, and have the help text
hint that the resulting code is sub-optimal.
Automatic merge from submit-queue (batch tested with PRs 37468, 36546, 38713, 38902, 38614)
Remove extensions/v1beta1 Job
Fixes https://github.com/kubernetes/kubernetes/issues/32763. This endpoint was deprecated in 1.5 and was planned to be removed in 1.6.
**Release note**:
```release-note
Remove extensions/v1beta1 Jobs resource, and job/v1beta1 generator.
```
Automatic merge from submit-queue (batch tested with PRs 37468, 36546, 38713, 38902, 38614)
local-up-cluster additions
**What this PR does / why we need it**:
Changes to local-cluster-up: These include: 1) a simple additional help option. 2) additional error message to not being able to run `docker ps`. 3) fail faster when etcd is not found in path. Hopefully these make developing a bit more pleasant.
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 38888, 38895)
Bundle federation/develop directory in the test tarball for federation testing.
cc @kubernetes/sig-federation-misc
Automatic merge from submit-queue
use in-cluster kubeconfig for genericapiserver
Allow the use of the in-cluster config to communicate with the core API server for delegated authn/authz for an addon API server.
@kubernetes/sig-api-machinery @sttts
from etcd.sh split the start process into validate fucntion + start function so that the validate piece can be reused elsewhere. the up-cluster script has been changed to remove duplicate docker logic to the one used in buid-tools/common.sh and the validate etcd function is now used here.
moved docker daemon check function to util.sh and made function name changes and upstream changes.
The current dev-build-*.sh scripts do a full release build which means
running tests and also doing cross-platform builds. This is unnecessary
and after discussing this in Slack it was suggested to either blow away
these files or fix them. This should fix them.
Automatic merge from submit-queue (batch tested with PRs 38783, 36396)
hack/cherry_pick_pull.sh: cleanup patch files
this was driving me nuts.
it will now remove the patch files from /tmp
Automatic merge from submit-queue
Rename root BUILD to BUILD.bazel, and rename build-tools/ back to build/
**What this PR does / why we need it**: undoes #35453. `build-tools/` was intended to be a temporary workaround until https://github.com/bazelbuild/bazel/issues/552 was fixed, which is has been, as of bazel 0.4.1.
**Which issue this PR fixes**: fixes#38126
FYI @thockin
since this is likely to be rebase hell, my list of actions:
```console
$ git mv BUILD BUILD.bazel
$ git commit -a
$ git mv build-tools/ build/
$ sift -l 'build-tools' | xargs sed -i -e 's:build-tools/:build/:g'
$ git checkout HEAD docs/
$ git commit -a
```
Automatic merge from submit-queue (batch tested with PRs 37708, 34410)
Add restclientconfig helper fn for parsing timeout
Related downstream PR: https://github.com/openshift/origin/pull/12062 (example of use-case for this patch)
**Release note**:
```release-note
release-note-none
```
This patch adds a package `pkg/client/unversioned/clientcmd/util` and
defines a `ParseTimeout` helper function for parsing time from a
user-defined string. This allows code re-use in other packages that
require the creation of a new restclient (and therefore must set the
`--global-timeout` flag value manually).
@fabianofranz @kubernetes/cli-review
Automatic merge from submit-queue (batch tested with PRs 38154, 38502)
Rename "release_1_5" clientset to just "clientset"
We used to keep multiple releases in the main repo. Now that [client-go](https://github.com/kubernetes/client-go) does the versioning, there is no need to keep releases in the main repo. This PR renames the "release_1_5" clientset to just "clientset", clientset development will be done in this directory.
@kubernetes/sig-api-machinery @deads2k
```release-note
The main repository does not keep multiple releases of clientsets anymore. Please find previous releases at https://github.com/kubernetes/client-go
```
Automatic merge from submit-queue (batch tested with PRs 38727, 38726, 38347, 38348)
Second pass of renaming kube-dns configure files
Continue work of #38523.
Not sure why cluster/centos/deployAddons.sh was omitted in previous PR. Also deletes the non-use `DNS_REPLICAS` var and changes `-rc` suffix in hack/local-up-cluster.sh.
@thockin @bowei @deads2k
Automatic merge from submit-queue (batch tested with PRs 35436, 37090, 38700)
Significantly speed-up make
- do not call `dirname` for every go file => gives **>8 seconds** on each make run
- do not preserve time during rsync from `_output/local/go/bin` to `_output/local/bin/<os>/<arch>`:
We do a touch after our rebuild heuristic triggers.
If in `_output/local/go/bin` you have an older binary and go decides that it's good enough, the following rsync will "preserve the timestamp" of the binary copy in `_output/local/bin/<os>/<arch>`. This reverts the effect of any previous `touch` (compare `Makefile.generated_files`).
We do a complete rsync of binaries for each go target, not only the one which is rebuilt. This means that later go targets will overwrite the timestamps of freshly touched binaries. Hence, our rebuild heuristic will trigger again and again when running `make`.
This PR remove the "preserve mtime" from the rsync call. Hence, the effect of touch is not reverted anymore by later rsyncs.
Automatic merge from submit-queue (batch tested with PRs 38315, 38624, 38572, 38544)
Fix code coverage in build script on OS X.
Fixes#20223
- Fixes code coverage in build script on BSD systems (including OS X).
- Supersedes #38536.
Successfully tested on two platforms:
```
Darwin <hostname> 16.1.0 Darwin Kernel Version 16.1.0: Thu Oct 13 21:26:57 PDT 2016; root:xnu-3789.21.3~60/RELEASE_X86_64 x86_64
```
```
Linux <hostname> 4.4.14-040414-generic #201606241434 SMP Fri Jun 24 18:36:45 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
```
cc @ixdy @mfanjie @balajismaniam @smarterclayton @soltysh
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 38315, 38624, 38572, 38544)
kubernetes-discovery proxy
The last commit adds an authenticating (but not authorizing) proxy to `kubernetes-discovery`. The other commits are in the submit queue already.
@sttts @cjcullen This is it. After this we can start e2e wiring.
Automatic merge from submit-queue
Leak filling for newline format on benchmark-go.sh
**What this PR does / why we need it**: make hack/benchmark-go.sh work
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**: First commit :D , did I make a wrong modification?
**Release note**:
```release-note
```
Automatic merge from submit-queue
goflags must be after subcommand
**What this PR does / why we need it**:
if GOFLAGS is set when calling make, kubernetes will fail to build
as an example, I often have `GOFLAGS=-v` so I can have some idea of progress during compilation
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
no known issue
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 38277, 36361, 38452)
local-up: resolve sudo -E'd paths better
This problem won't affect everyone, but for people who have `--with-secure-path` configured for their sudo binaries, `sudo -E` won't actually preserve PATH, which means the sub-bash won't necessarily be able to find the binary which `test` found to exist.
This fixes that.
Ubuntu [used to](https://bugs.launchpad.net/ubuntu/+source/sudo/+bug/50797) configure it that way, and they might still.
It's configured as such on my (Gentoo) machine.
Automatic merge from submit-queue (batch tested with PRs 38277, 36361, 38452)
Add options for build container rsync optimization
KUBE_RSYNC_COMPRESS env var sets rsync compression level.
KUBE_RSYNC_GENERATED_TO_BUILD_CONTAINER env var disables rsyncing
generated files to build containers.
Why KUBE_RSYNC_COMPRESS is needed -- from rsync manual on `--compress` option (implied by non-zero `--compress-level`):
> Note that this option typically achieves better compression ratios than can be achieved by using a compressing remote shell or a compressing transport because it takes advantage of the implicit information in the matching data blocks that are not explicitly sent over the connection.
Use case for `KUBE_RSYNC_GENERATED_TO_BUILD_CONTAINER`: when you sometimes build stuff locally (e.g. `make WHAT=cmd/kubectl`) and sometimes do it on remote docker (`build-tools/run.sh make WHAT=cmd/hyperkube`), local builds touch generated files which causes them to be rsynced to the build data container, which may slow down the builds. Still, I'm not sure whether local->remote rsync of generated files is useful (e.g. someone may want to edit generated files for debugging purposes?), so I made not rsyncing these files an option instead of forcing such behavior.
Automatic merge from submit-queue (batch tested with PRs 34002, 38535, 37330, 38522, 38423)
Node E2E: `make test-e2e-node` runs the same test with pr builder by default.
This PR makes `make test-e2e-node` run non-serial, non-flaky, non-slow test by default.
This will make it easier to use.
/cc @timstclair
Automatic merge from submit-queue (batch tested with PRs 37270, 38309, 37568, 34554)
Remove update/verify-munge-docs.sh
These scripts are no long needed because:
1. most docs are moved to other repos
2. the mungers are pre-processing the docs for gh pages, which are created with a whole different process now.
cc @bgrant0607 @thockin
Automatic merge from submit-queue (batch tested with PRs 37860, 38429, 38451, 36050, 38463)
[Part 2] Adding s390x cross-compilation support for gcr.io images in this repo
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**: This PR enables s390x support to kube-dns , pause, addon-manager, etcd, hyperkube, kube-discovery etc. This PR also includes the changes due to which it can be cross compiled on x86 host architecture.
**Which issue this PR fixes#34328
**Special notes for your reviewer**: In existing file "build-tools/build-image/cross/Dockerfile" the repository mentioned for installing cross build tool chains for supporting architecture does not have a tool chain for s390x hence in my PR I am changing the repository so that it will be cross compiled for s390x.
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```
Allows cross compilation of Kubernetes on x86 host for s390x also enables s390x support to kube-dns , pause, addon-manager, etcd, hyperkube, kube-discovery etc
```
Automatic merge from submit-queue (batch tested with PRs 38284, 38403, 38265)
Fix local up cluster dns with RBAC
The DNS server needs permissions to run using RBAC. This does it with a big hammer before we sort out tight permissions.
@fabianofranz for the CLI change
@xilabao for the local-up-cluster change
Automatic merge from submit-queue (batch tested with PRs 38278, 37770)
Refactor REST storage to use generic defaults
This removes the repetition in the REST storage builders by moving the logic to `restoptions.ApplyOptions`. `registry.StorageWithCacher`/`generic.StorageDecorator` no longer assume that they can build the `keyFunc` for arbitrary objects. `restoptions.ApplyOptions` uses the `registry.Store`'s `KeyFunc` for its call to `generic.StorageDecorator`.
```release-note
Cluster federation servers have changed the location in etcd where federated services are stored, so existing federated services must be deleted and recreated. Before upgrading, export all federated services from the federation server and delete the services. After upgrading the cluster, recreate the federated services from the exported data.
```
On machines where sudo is configured with the `--with-secure-path`
option, the `sudo -E /bin/bash` portions of the script would execute
with a different path (e.g. not including $GOPATH/bin) and thus could
fail even though the check for those binaries passed.
We should expose the SkipUnsafe option, for legacy compatability, so
that conversion-go can be used in other projects, and for platforms
where unsafe is not available.
Make unsafe code generation the default though, and have the help text
hint that the resulting code is sub-optimal.
Automatic merge from submit-queue (batch tested with PRs 36310, 37349, 38319, 38402, 38338)
add summarizing discovery controller and handlers
Requires https://github.com/kubernetes/kubernetes/pull/38304 .
This adds discovery support to the new `kubernetes-discovery` that runs based on a controller wired up to the `APIService.apiregistration.k8s.io`. It also adds in plumbing for `local-up-cluster.sh` to register the "normal" kube resources.
@kubernetes/sig-api-machinery
@sttts
Automatic merge from submit-queue
[Federation] Make federation etcd PVC size configurable
This one implements one of the many TODO items pending in the previous set of kubefed PRs.
The design doc PR is at https://github.com/kubernetes/kubernetes/pull/34484
cc @kubernetes/sig-cluster-federation @madhusudancs
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```
[Federation] kubefed init now has a new flag, --etcd-pv-capacity, which can be used to configure the persistent volume capacity for etcd.
```
Automatic merge from submit-queue
kubedns: use initial resource listing as ready signal
Fix#35140.
Set up the ready signal after the first resource listing finished for both endpoints and services instead of listen on kubernetes service.
@bprashanth @bowei @thockin
**Release note**:
```
```
Automatic merge from submit-queue
Fix local-cluster-up on Mac and discovery on all systems
- use cfssl on the host, not from Docker. Solves a number of permission problem with selinux and Mac
- fix discovery startup
Automatic merge from submit-queue (batch tested with PRs 38260, 32811, 28458, 33570, 37096)
Fix support for DNS in local-up-cluster.sh
[]()
The default appears to be gce now, so this tries to spawn a service in
gce... not the desired outcome for a local test setup.
We also need to sync the kubectl config in the script with the instructions for users, since if people don't test with DNS enabled, it bitrots (which this aims to fix).
To test manually, run something like
```
sudo KUBE_ENABLE_CLUSTER_DNS=true API_HOST_IP=0.0.0.0 ./hack/local-up-cluster.sh
```
The API_HOST_IP=0.0.0.0 is required, otherwise the master is not contactable by kubedns, and the dns pod will fail readychecking on the healthz pod.
Automatic merge from submit-queue (batch tested with PRs 38318, 38258)
kernel memcg notification enabled via experimental flag
Kubelet integrates with kernel memcg notification API if and only if enabled via experimental flag.
Automatic merge from submit-queue (batch tested with PRs 36543, 38189, 38289, 38291, 36724)
add authentication/authorization to kubernetes-discovery
Wires authentication and authorization into `kubernetes-discovery` and re-enables the `local-up-cluster.sh` along with proper permission granting for RBAC cases.
@sttts @liggitt
Automatic merge from submit-queue (batch tested with PRs 35101, 38215, 38092)
fix informer generation
Informer generation doesn't work for informers from a different clientset. This updates the generator to generate the internal interfaces required to break the cycle.
@ncdc take a look at the last two commits.
Automatic merge from submit-queue
Disable kubernetes-discovery in local-up-cluster.sh
fix#38257
Fixes local-up-cluster until kubernetes-discovery flags are hooked up
Automatic merge from submit-queue
update local-up-cluster to allow full authentication proxying
Adds group and header information in auth proxy authenticator options for `local-up-cluster.sh`. Must have been missed in the rebase madness.
Automatic merge from submit-queue (batch tested with PRs 38173, 38151, 38197, 38221)
Include resource type headers in diff report
**What this PR does / why we need it**: when checking for leaked resources, we report a failure only if there were additions to the diff - we don't want resources that were deleted during the run to count as a failure.
The logic for this removes the resource type headers from the diff report, however, so it's harder to understand what leaked.
For example, in https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-alpha-features-release-1.5/196/, the test failure shows
```
Error: 4 leaked resources
+NAME REGION IP_ADDRESS IP_PROTOCOL TARGET
+a66878dbaad1f11e695b342010af0002 us-central1 104.198.182.83 TCP us-central1/targetPools/a66878dbaad1f11e695b342010af0002
+NAME REGION SESSION_AFFINITY BACKUP HEALTH_CHECKS
+a66878dbaad1f11e695b342010af0002 us-central1 a66878dbaad1f11e695b342010af0002
```
but the [actual diff](https://storage.googleapis.com/kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-alpha-features-release-1.5/196/artifacts/gcp-resources-diff.txt) contains more information:
```
--- /workspace/_artifacts/gcp-resources-before.txt 2016-12-01 14:39:14.260049037 -0800
+++ /workspace/_artifacts/gcp-resources-after.txt 2016-12-01 14:39:14.260049037 -0800
@@ -34,8 +34 @@ [ addresses ]
-NAME REGION ADDRESS STATUS
-a03d0e020a6bc11e6855f42010af0001 us-central1 104.197.135.88 RESERVED
-a23f20a47a6a011e6829a42010af0004 us-central1 104.197.147.212 RESERVED
-a32291b51a5e611e6b73342010af0002 us-central1 104.198.171.178 RESERVED
-a3c93cf79a65811e6b7c242010af0001 us-central1 104.198.58.57 RESERVED
-a9ac5bd1ba62811e68d1142010af0004 us-central1 104.154.134.135 RESERVED
-adae4c1a7a69511e6b2df42010af0000 us-central1 104.198.182.83 RESERVED
-adb4a80caa60311e689a942010af0001 us-central1 104.154.116.195 RESERVED
+
@@ -64 +57,2 @@ [ forwarding-rules ]
-
+NAME REGION IP_ADDRESS IP_PROTOCOL TARGET
+a66878dbaad1f11e695b342010af0002 us-central1 104.198.182.83 TCP us-central1/targetPools/a66878dbaad1f11e695b342010af0002
@@ -68 +62,2 @@ [ target-pools ]
-
+NAME REGION SESSION_AFFINITY BACKUP HEALTH_CHECKS
+a66878dbaad1f11e695b342010af0002 us-central1 a66878dbaad1f11e695b342010af0002
```
This PR will include the `[ type header ]`s in the report if resources are leaked.
cc @fejta
Automatic merge from submit-queue
add a configuration for kubelet to register as a node with taints
and deprecate --register-schedulable
ref #28687#29178
cc @dchen1107 @davidopp @roberthbailey
Automatic merge from submit-queue (batch tested with PRs 36990, 37494, 38152, 37561, 38136)
join client CA bundles
Last commit grabs client CA bundles from disparate parts of the auth config and makes a pool that contains all of them.
I suspect a rebase broke this because of ordering. I'll keep these separate to make it easier for me to debug.
Automatic merge from submit-queue (batch tested with PRs 36990, 37494, 38152, 37561, 38136)
api federation types
First commit adds types that can back the kubernetes-discovery server with an `kubectl` compatible way of adding federated servers. Second commit is just generated code.
After we have types, I'd like to start splitting `kubernetes-discovery` into a "legacy" mode which will support what we have today and a "normal" mode which will provide an API federation server like this: https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/federated-api-servers.md that includes both discovery and proxy in a single server. Something like this: https://github.com/openshift/kube-aggregator .
@kubernetes/sig-api-machinery @nikhiljindal
Automatic merge from submit-queue
Add "make help" to list all make targets and help info
As discussed in [PR#29320comment](https://github.com/kubernetes/kubernetes/pull/29320#issuecomment-234420145), add a `make help` to make the use of `make` easy. Though it works well on my Ubuntu now (see the output as below, not sure if it still works on other systems), I believe the scripts are somewhat ugly, so, any suggestion for optimization is welcome.
BTW, I intended to list targets by groups, but it's hard to do that automatically. So I just list them in alphabetical order. I think this may be enough.
There are still some TODOs (also mentioned in the Makefile):
1. make EXCLUDE_TARGET auto-generated when there are other files in cmd/
2. should we exclude the target "cmd/libs" but include "cmd/libs/go2idl/*"?
3. should we let `help` be the first/default target? It will show the help when we just type `make`.
1 and 2 are to exclude the invalid targets generated by `$(notdir $(abspath $(wildcard cmd/*/)))`:
- OWNERS is just a file, it can't be a target
- libs itself cannot be built
/cc @thockin @jfrazelle @MHBauer @dims
Output:
```
root@vm:/home/paas/zxp/code/k8s/fork/kubernetes# make help
--------------------------------------------------------------------------------
all
# Build code.
#
# Args:
# WHAT: Directory names to build. If any of these directories has a 'main'
# package, the build will produce executable files under _output/go/bin.
# If not specified, "everything" will be built.
# GOFLAGS: Extra flags to pass to 'go' when building.
# GOLDFLAGS: Extra linking flags passed to 'go' when building.
# GOGCFLAGS: Additional go compile flags passed to 'go' when building.
#
# Example:
# make
# make all
# make all WHAT=cmd/kubelet GOFLAGS=-v
# make all GOGCFLAGS="-N -l"
# Note: Use the -N -l options to disable compiler optimizations an inlining.
# Using these build options allows you to subsequently use source
# debugging tools like delve.
---------------------------------------------------------------------------------
check
# Build and run tests.
#
# Args:
# WHAT: Directory names to test. All *_test.go files under these
# directories will be run. If not specified, "everything" will be tested.
# TESTS: Same as WHAT.
# GOFLAGS: Extra flags to pass to 'go' when building.
# GOLDFLAGS: Extra linking flags to pass to 'go' when building.
# GOGCFLAGS: Additional go compile flags passed to 'go' when building.
#
# Example:
# make check
# make test
# make check WHAT=pkg/kubelet GOFLAGS=-v
---------------------------------------------------------------------------------
clean
# Remove all build artifacts.
#
# Example:
# make clean
#
# TODO(thockin): call clean_generated when we stop committing generated code.
---------------------------------------------------------------------------------
clean_generated
# Remove all auto-generated artifacts.
#
# Example:
# make clean_generated
---------------------------------------------------------------------------------
clean_meta
# Remove make-related metadata files.
#
# Example:
# make clean_meta
---------------------------------------------------------------------------------
cross
# Cross-compile for all platforms
#
# Example:
# make cross
---------------------------------------------------------------------------------
federation-apiserver
federation-controller-manager
genfeddocs
# Add rules for all directories in federation/cmd/
#
# Example:
# make federation-apiserver federation-controller-manager
---------------------------------------------------------------------------------
gendocs
genkubedocs
genman
genswaggertypedocs
genutils
genyaml
hyperkube
kube-apiserver
kube-controller-manager
kubectl
kube-dns
kubelet
kubemark
kube-proxy
kubernetes-discovery
libs
linkcheck
mungedocs
# Add rules for all directories in cmd/
#
# Example:
# make kubectl kube-proxy
---------------------------------------------------------------------------------
generated_files
# Produce auto-generated files needed for the build.
#
# Example:
# make generated_files
---------------------------------------------------------------------------------
ginkgo
# Build ginkgo
#
# Example:
# make ginkgo
---------------------------------------------------------------------------------
help
# Print make targets and help info
#
# Example:
# make help
---------------------------------------------------------------------------------
quick-release
# Build a release, but skip tests
#
# Example:
# make release-skip-tests
---------------------------------------------------------------------------------
release
# Build a release
#
# Example:
# make release
---------------------------------------------------------------------------------
release-skip-tests
# Build a release, but skip tests
#
# Example:
# make release-skip-tests
---------------------------------------------------------------------------------
test
# Build and run tests.
#
# Args:
# WHAT: Directory names to test. All *_test.go files under these
# directories will be run. If not specified, "everything" will be tested.
# TESTS: Same as WHAT.
# GOFLAGS: Extra flags to pass to 'go' when building.
# GOLDFLAGS: Extra linking flags to pass to 'go' when building.
# GOGCFLAGS: Additional go compile flags passed to 'go' when building.
#
# Example:
# make check
# make test
# make check WHAT=pkg/kubelet GOFLAGS=-v
---------------------------------------------------------------------------------
test-cmd
# Build and run cmdline tests.
#
# Example:
# make test-cmd
---------------------------------------------------------------------------------
test-e2e
# Build and run end-to-end tests.
#
# Example:
# make test-e2e
---------------------------------------------------------------------------------
test-e2e-node
# Build and run node end-to-end tests.
#
# Args:
# FOCUS: Regexp that matches the tests to be run. Defaults to "".
# SKIP: Regexp that matches the tests that needs to be skipped. Defaults
# to "".
# RUN_UNTIL_FAILURE: If true, pass --untilItFails to ginkgo so tests are run
# repeatedly until they fail. Defaults to false.
# REMOTE: If true, run the tests on a remote host instance on GCE. Defaults
# to false.
# IMAGES: For REMOTE=true only. Comma delimited list of images for creating
# remote hosts to run tests against. Defaults to a recent image.
# LIST_IMAGES: If true, don't run tests. Just output the list of available
# images for testing. Defaults to false.
# HOSTS: For REMOTE=true only. Comma delimited list of running gce hosts to
# run tests against. Defaults to "".
# DELETE_INSTANCES: For REMOTE=true only. Delete any instances created as
# part of this test run. Defaults to false.
# ARTIFACTS: For REMOTE=true only. Local directory to scp test artifacts into
# from the remote hosts. Defaults to ""/tmp/_artifacts".
# REPORT: For REMOTE=false only. Local directory to write juntil xml results
# to. Defaults to "/tmp/".
# CLEANUP: For REMOTE=true only. If false, do not stop processes or delete
# test files on remote hosts. Defaults to true.
# IMAGE_PROJECT: For REMOTE=true only. Project containing images provided to
# IMAGES. Defaults to "kubernetes-node-e2e-images".
# INSTANCE_PREFIX: For REMOTE=true only. Instances created from images will
# have the name "-". Defaults to "test".
# INSTANCE_METADATA: For REMOTE=true and running on GCE only.
#
# Example:
# make test-e2e-node FOCUS=Kubelet SKIP=container
# make test-e2e-node REMOTE=true DELETE_INSTANCES=true
# make test-e2e-node TEST_ARGS="--cgroups-per-qos=true"
# Build and run tests.
---------------------------------------------------------------------------------
test-integration
# Build and run integration tests.
#
# Example:
# make test-integration
---------------------------------------------------------------------------------
verify
# Runs all the presubmission verifications.
#
# Args:
# BRANCH: Branch to be passed to verify-godeps.sh script.
#
# Example:
# make verify
# make verify BRANCH=branch_x
---------------------------------------------------------------------------------
vet
# Run 'go vet'.
#
# Args:
# WHAT: Directory names to vet. All *.go files under these
# directories will be vetted. If not specified, "everything" will be
# vetted.
#
# Example:
# make vet
# make vet WHAT=pkg/kubelet
---------------------------------------------------------------------------------
```
Automatic merge from submit-queue (batch tested with PRs 38149, 38156, 38150)
Node E2E: Remove setup-node option
This PR removes `setup-node` option, because:
* It is misleading. `setup-node` doesn't really setup the node, test framework will only put current user into docker user group when it is specified.
* It is not necessary anymore. Because we always run node e2e test as root now, we don't need to do this anymore.
This is a minor cleanup preparing for my test framework refactoring work. Will send out the refactor PR later.
/cc @kubernetes/sig-node
Automatic merge from submit-queue (batch tested with PRs 38149, 38156, 38150)
Remove girishkalele from most places
@matchstick you might need to help here. I am doing this because the bot is trying to create an issue assigned to @girishkalele but it cannot be created as he is not a member of the org any longer.
Automatic merge from submit-queue (batch tested with PRs 37328, 38102, 37261, 31321, 38146)
Add federation-openapi-spec to update-all targets
As `make verify` runs all `hack/verify-*` scripts (with some exceptions), `update-all.sh` should be in sync. It is missing all the federation targets right now, but `verify-federation-openapi-spec.sh` exists. Hence, we add that.
@nikhiljindal @mbohlool what about the other federation update scripts?
Automatic merge from submit-queue (batch tested with PRs 38111, 38121)
Prevent SIGPIPE errors causing the version check line to fail
The way we use pipe in hack/lib/etcd.sh when processing the version command can lead to
141 failues on some systems. Using a prefix of tail can prevent this.
Fixes#38109
Automatic merge from submit-queue (batch tested with PRs 37208, 37446, 37420)
Move "-short" to KUBE_TEST_ARGS
fixes#37445
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
NONE
```
"-short" is a argument of "go test", it should be moved to
"KUBE_TEST_ARGS".
Automatic merge from submit-queue
Split inflight requests into read-only and mutating groups
cc @smarterclayton @lavalamp @caesarxuchao
```release-note
API server have two separate limits for read-only and mutating inflight requests.
```
Automatic merge from submit-queue (batch tested with PRs 36816, 37534)
Move pkg/api/unversioned to pkg/apis/meta/v1
This moves code from using pkg/api/unversioned to pkg/apis/meta/v1 with the `metav1` local package name.
Built on top of #37532 (the first three commits related to ExportOptions)
Part of #37530
Automatic merge from submit-queue
plumb in front proxy group header
Builds on https://github.com/kubernetes/kubernetes/pull/36662 and https://github.com/kubernetes/kubernetes/pull/36774, so only the last commit is unique.
This completes the plumbing for front proxy header information and makes it possible to add just the front proxy header authenticator.
WIP because I'm going to assess it in use downstream.
Automatic merge from submit-queue
kubectl top pod|node should handle when Heapster is somewhere else
OpenShift runs Heapster on HTTPS, which means `top node` and `top pod`
are broken because they hardcode 'http' as the scheme. Provide an
options struct allowing users to specify `--heapster-namespace`,
`--heapster-service`, `--heapster-scheme`, and `--heapster-port` to the
commands (leveraging the existing defaults).
@kubernetes/sig-metrics makes top a little more useful in other spots
Automatic merge from submit-queue (batch tested with PRs 35300, 36709, 37643, 37813, 37697)
Add generated informers
Add informer-gen and the informers it generates. We'll do follow-up PRs to convert everything currently using the hand-written informers to the generated ones.
TODO:
- [x] switch to `GroupVersionResource`
- [x] finish godoc
@deads2k @caesarxuchao @sttts @liggitt
Automatic merge from submit-queue (batch tested with PRs 37094, 37663, 37442, 37808, 37826)
Fix the PATH that install-etcd.sh tells you to use
After you run install-etcd.sh, it tells you:
etcd v3.0.14 installed. To use:
export PATH=${PATH}:/home/danw/rh/go/src/k8s.io/kubernetes/third_party/etcd
which doesn't work if you have an older etcd installed in /usr/bin:
danw@w541:kubernetes (master)> PATH=${PATH}:/home/danw/rh/go/src/k8s.io/kubernetes/third_party/etcd etcd --version
etcd Version: 2.2.5
You need to put the local etcd dir first in PATH, not last.
Automatic merge from submit-queue (batch tested with PRs 37094, 37663, 37442, 37808, 37826)
Moved gobindata, refactored ReadOrDie refs
**What this PR does / why we need it**: Having gobindata inside of test/e2e/framework prevents external projects from importing the framework. Moving it out and managing refs fixes this problem.
**Which issue this PR fixes**: fixes#37007
Automatic merge from submit-queue (batch tested with PRs 37997, 37939, 37990, 36700, 37258)
--export flag belongs to kubectl get, not kubectl
**What this PR does / why we need it**:
When running the kubectl via symlink to hyperkube, the ``--export`` flag is not delegated to `kubectl get` but to `kubectl` only. Ending with unrecognized flag.
This patch adds a package `pkg/client/unversioned/clientcmd/util` and
defines a `ParseTimeout` helper function for parsing time from a
user-defined string. This allows code re-use in other packages that
require the creation of a new restclient (and therefore must set the
`--global-timeout` flag value manually).
Automatic merge from submit-queue
Add optional timeout logic to e2e.go
ref https://github.com/kubernetes/test-infra/issues/1250
Add a --timeout=5 flag.
If unset this does not change behavior.
If set this first sends an interrupt signal and then 15m later starts terminating child processes.
Will test out this change in another PR using the .use_head_e2e trick
- Remaining spaghetti untangled
- Missed bazel update and a few hardcoded refs
- New instance of framework.ReadOrDie reference removed post rebase
- Resolve new clientset rebase
- Fixed e2e/generated BUILD dep
- A space
- Missed gobindata ref in golang.sh
Automatic merge from submit-queue
Add kubernetes-anywhere as a new e2e deployment option.
This change adds support for using `kubernetes-anywhere` as a deployment option for hack/e2e.go. This work is toward the larger goal of being able to run e2e tests against `kubeadm` clusters, which `kubernetes-anywhere` supports.
**Release note**:
```release-note
Add kubernetes-anywhere as a new e2e deployment option
```
The configuration in `getConfig()` comes mostly from the defaults in `kubernetes-anywhere`. In the future, we can add more plumbing to override them via CLI flags.
CC @mikedanese
Automatic merge from submit-queue
create service add create ExternalName service implementation
@kubernetes/kubectl create service add ExternalName support, refer #34731 for more detail.
```release-note
kubectl create service externalname
```
Automatic merge from submit-queue
Build vendored copy of go-bindata and use that in go generate step
**What this PR does / why we need it**: as the title says, uses the vendored version of `go-bindata` rather than expecting developers to `go get` it (when building outside docker).
**Which issue this PR fixes**: fixes#34067, partially addresses #36655
**Special notes for your reviewer**: we still call `go generate` far too many times:
```console
~/.../src/k8s.io/kubernetes $ which go-bindata
~/.../src/k8s.io/kubernetes $ make
+++ [1116 17:35:28] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
k8s.io/kubernetes/vendor/github.com/jteeuwen/go-bindata/go-bindata
+++ [1116 17:35:29] Generating bindata:
test/e2e/framework/gobindata_util.go
+++ [1116 17:35:30] Building go targets for linux/amd64:
cmd/libs/go2idl/deepcopy-gen
+++ [1116 17:35:35] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
k8s.io/kubernetes/vendor/github.com/jteeuwen/go-bindata/go-bindata
+++ [1116 17:35:35] Generating bindata:
test/e2e/framework/gobindata_util.go
+++ [1116 17:35:36] Building go targets for linux/amd64:
cmd/libs/go2idl/defaulter-gen
+++ [1116 17:35:41] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
k8s.io/kubernetes/vendor/github.com/jteeuwen/go-bindata/go-bindata
+++ [1116 17:35:41] Generating bindata:
test/e2e/framework/gobindata_util.go
+++ [1116 17:35:42] Building go targets for linux/amd64:
cmd/libs/go2idl/conversion-gen
+++ [1116 17:35:47] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
k8s.io/kubernetes/vendor/github.com/jteeuwen/go-bindata/go-bindata
+++ [1116 17:35:47] Generating bindata:
test/e2e/framework/gobindata_util.go
+++ [1116 17:35:48] Building go targets for linux/amd64:
cmd/libs/go2idl/openapi-gen
+++ [1116 17:35:56] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
k8s.io/kubernetes/vendor/github.com/jteeuwen/go-bindata/go-bindata
+++ [1116 17:35:56] Generating bindata:
test/e2e/framework/gobindata_util.go
```
Fixing that is a separate effort, though.
cc @sebgoa @ZhangBanger
Automatic merge from submit-queue
change prompt for enabling RBAC on local-up-cluster
run `ALLOW_ANY_TOKEN=true ENABLE_RBAC=true hack/local-up-cluster.sh`
Use "--token" instead of "--username/--password", so need to update the prompt
OpenShift runs Heapster on HTTPS, which means `top node` and `top pod`
are broken because they hardcode 'http' as the scheme. Provide an
options struct allowing users to specify `--heapster-namespace`,
`--heapster-service`, `--heapster-scheme`, and `--heapster-port` to the
commands (leveraging the existing defaults).
Automatic merge from submit-queue
Activate client certs in local-cluster-up.sh for RBAC
- set up client certs for each component in local-cluster-up.sh
Automatic merge from submit-queue
build: clean platform envs to prevent cross-contamination
**What this PR does / why we need it**: As I described in https://github.com/kubernetes/kubernetes/issues/37079#issuecomment-263733509, we are leaking platform compilation envs between build stages for different platforms in the non-parallel dockerized cross build. This PR uses a subshell for the non-parallel build, more closely matching the parallel build.
This also adds some logging, which had it existed previously, may have made the bug more immediately obvious.
**Which issue this PR fixes**: fixes#37079
cc @sebgoa @iTagir @saad-ali
Automatic merge from submit-queue
hack/e2e.go: Dump cluster logs in case of Up failure
**What this PR does / why we need it**: A failure in `Up` currently results in no attempt to grab cluster logs. This fixes that hole. (Sigh, a ton of holes for this diagnosis path.)
Automatic merge from submit-queue
Update Stateful Set example files for 1.5
1. Remove initialized annotation from statefulset examples
2. Update storage class annotation to beta in statefulset examples
3. Remove alpha limitation on PetSet in cassandra example
cc @erictune @foxish @kow3ns @enisoc @chrislovecnm @kubernetes/sig-apps
```release-note
NONE
```
Automatic merge from submit-queue
move parts of the mega generic run struct out
This splits the main `ServerRunOptions` into composeable pieces that are bindable separately and adds easy paths for composing servers to run delegating authentication and authorization.
@sttts @ncdc alright, I think this is as far as I need to go to make the composing servers reasonable to write. I'll try leaving it here
Automatic merge from submit-queue
Fix package aliases to follow golang convention
Some package aliases are not not align with golang convention https://blog.golang.org/package-names. This PR fixes them. Also adds a verify script and presubmit checks.
Fixes#35070.
cc/ @timstclair @Random-Liu
Automatic merge from submit-queue
When --grace-period=0 is provided, wait for deletion
The grace-period is automatically set to 1 unless --force is provided, and the client waits until the object is deleted.
This preserves backwards compatibility with 1.4 and earlier. It does not handle scenarios where the object is deleted and a new object is created with the same name because we don't have the initial object loaded (and that's a larger change for 1.5).
Fixes#37117 by relaxing the guarantees provided.
```release-note
When deleting an object with `--grace-period=0`, the client will begin a graceful deletion and wait until the resource is fully deleted. To force deletion, use the `--force` flag.
```
Automatic merge from submit-queue
Node Conformance Test: Final cleanup for node conformance test.
This PR fits node conformance test with recent change.
* Remove `--manifest-path` because the test will get kubelet configuration through `/configz` now. https://github.com/kubernetes/kubernetes/pull/36919
* Add `$TEST_ARGS` so that we can override arguments inside the container.
* Fix a bug in garbage_collector_test.go which will cause the framework tries to connect docker no matter running the test or not. @dashpole
* Add `${REGISTRY}/node-test:${VERSION}` for convenience.
* Bump up the image version to `0.2`. (the one released with v1.4 is `v0.1`)
I've run the test both with `run_test.sh` script and directly `docker run`. Both of them passed.
After this gets merged, I'll build and push the new test image.
@dchen1107
/cc @kubernetes/sig-node
Automatic merge from submit-queue
add wrapper to provide stderr on command errors
The go standard library doesn't include stderr in the error message, but in many cases it is present: https://golang.org/src/os/exec/exec.go#L389 . This adds a wrapper to display that information. I've added in it on spot where the kops test is having trouble. If it works well, we can add it elsewhere.
@wojtek-t ptal
Automatic merge from submit-queue
Node E2E: Fix remote node e2e focus.
Before, we use `'focus'` and `'skip'` in `hack/make-rules/test-e2e-node.sh`.
When we run `make test-e2e-node REMOTE=true FOCUS="Some Thing"`, it will eventually be translated to `-focus='Some Thing'` [here](https://github.com/kubernetes/kubernetes/blob/master/test/e2e_node/remote/remote.go#L284-L285).
However, golang `exec.Command` wraps each argument with single quote, the argument will become `'xx -focus='Some Thing' xx'`, and cause error because of the 2 layer single quote.
Automatic merge from submit-queue
skip test docker if we do not use docker as container runtime.
skip test docker if we do not use docker as container runtime.
Signed-off-by: Xianglin Gao <xlgao@zju.edu.cn>
Automatic merge from submit-queue
Removes shorthand flag -w from kubectl apply
Fixes#37342.
A shorthand flag `-w` was introduced as flag `--prune-whitelist` for kubectl apply two weeks ago. Turned out it is not what we should do. Removing this shorthand flag before 1.5 release to prevent further issues.
@ymqytw @pwittrock
The grace-period is automatically set to 1 unless --force is provided,
and the client waits until the object is deleted.
This preserves backwards compatibility with 1.4 and earlier. It does not
handle scenarios where the object is deleted and a new object is created
with the same name.
Automatic merge from submit-queue
Fixed e2e tests for HA master.
Set of fixes that allows HA master e2e tests to pass for removal/addition master replicas.
The summary of changes:
- fixed host name in etcd certs,
- added cluster validation after kube-down,
- fixed the number of master replicas in cluster validation,
- made MULTIZONE=true required for HA master deployments, ensured we correctly handle MULTIZONE=true when user wants to create HA master but not kubelets in multiple zones,
- extended verification of master replicas in HA master e2e tests.
- Fix unbound variables in hack/local-up-cluster.sh
- explicitly set KUBERNETES_PROVIDER in local-up-cluster.
The default appears to be gce now, so this tries to spawn a service
in gce... not the desired outcome for a local test setup.
- Remove stale creation of kube-system: the kube-system namespace is
built in these days.
- Pickup third party etcd automatically.
- Retab hack/local-up-cluster.sh for consistency
- Update hack-local-up's internal credentials to match the user
instructions we give (which work).
- Note the CA root requirement in local-up-cluster
- Fix a dangling comment in local-up-cluster
To test manually, run something like:
```sudo KUBE_ENABLE_CLUSTER_DNS+true API_HOST_IP=0.0.0.0
./hack/local-up-cluster.sh```
The API_HOST_IP=0.0.0.0 is required otherwise kubedns cannot contact
the master and the dns pod will fail readychecking on the healthz
container.
Automatic merge from submit-queue
[Federation][join-flags] Add flags for cluster context and secret names while joining clusters to federation.
Vast majority of cluster contexts are not RFC 1123 subdomains. Since
cluster and secret names for the API objects are derived from the
cluster context name, there is no way for users to join clusters
with such context names to federation, unless they modify the context
name in their kubeconfigs itself. That's a lot of inconvenience and
entirely goes against the goal and beats the purpose of the `kubefed`
tool. So we are providing these flags to allow users to override these
values.
Also, since users register their clusters with federation, it is makes
sense in terms of user experience to make the cluster name a positional
argument because that feels more natural. Also, specifying cluster name
in the join command as a mandatory positional argument make
`kubefed join` consistent with `kubefed unjoin`. This also means
`--cluster-context` is now made a flag and defaults to cluster name if
unspecified.
`--secret-name` also defaults to the cluster name if unspecified.
Fixes: Issue #35954
cc @kubernetes/sig-cluster-federation @quinton-hoole @irfanurrehman
- Adds command line flags --config-map, --config-map-ns.
- Fixes 36194 (https://github.com/kubernetes/kubernetes/issues/36194)
- Update kube-dns yamls
- Update bazel (hack/update-bazel.sh)
- Update known command line flags
- Temporarily reference new kube-dns image (this will be fixed with
a separate commit when the DNS image is created)
When running the kubectl via symlink to hyperkube, the --export is not delegated to 'kubectl get' but to 'kubectl' only.
Ending with unrecognized flag.
Automatic merge from submit-queue
make kubectl create --edit iterate
`kubectl create --edit` is broken after #36148 merged.
`kubectl create --edit` will fail when a manifest that contains multiple resources.
I guess the root cause is that dynamic typer doesn't support a list of resources currently.
This PR makes `kubectl create --edit` iterate again as `kubectl create`.
Automatic merge from submit-queue
Require BOOTSTRAP_MIGRATION, JENKINS_BUILD_STARTED or JENKINS_BUILD_F…
If we do not do this then we upload finished.json and update latest-build.txt before the build fiinshes, breaking the SQ
Vast majority of cluster contexts are not RFC 1123 subdomains. Since
cluster and secret names for the API objects are derived from the
cluster context name, there is no way for users to join clusters
with such context names to federation, unless they modify the context
name in their kubeconfigs itself. That's a lot of inconvenience and
entirely goes against the goal and beats the purpose of the `kubefed`
tool. So we are providing these flags to allow users to override these
values.
Also, since users register their clusters with federation, it is makes
sense in terms of user experience to make the cluster name a positional
argument because that feels more natural. Also, specifying cluster name
in the join command as a mandatory positional argument make `kubefed
join` consistent with `kubefed unjoin`. This also means `--cluster-
context` is now made a flag and defaults to cluster name if unspecified.
`--secret-name` also defaults to the cluster name if unspecified.
Automatic merge from submit-queue
V2resource fixes
when using kubectl set resources it resets all resource fields that are not being set.
for example
$ kubectl set resources deployments nginx --limits=cpu=100m
followed by
$ kubectl set resources deployments nginx --limits=memory=256Mi
would result in the nginx deployment only limiting memory at 256Mi with the previous
limit placed on the cpu being wiped out. This behavior is corrected so that each invocation
only modifies fields set in that command and changed the testing so that the desired behavior
is checked.
Also a typo:
you must specify an update to requests or limits or (in the form of --requests/--limits)
corrected to
you must specify an update to requests or limits (in the form of --requests/--limits)
Implemented both the dry run and local flags.
Added test cases to show that both flags are operating as intended.
Removed the print statement "running in local mode" as in PR#35112
The original PR associated with these fixes where reverted due to causing a flake in hack/make-rules/test-cmd.sh, I gave the 'kubectl set resources' tests there own deployment and set the terminationGracePeriodSeconds to 0 and have run test-cmd.sh for hours without hitting the flake
Automatic merge from submit-queue
[kubelet]update some --cgroups-per-qos to --experimental-cgroups-per-qos
Follow https://github.com/kubernetes/kubernetes/pull/36767, there are some fields still need update in docs or hack/local-up-cluster.sh
Automatic merge from submit-queue
Add a flag allowing contention profiling of the API server
Useful for performance debugging.
cc @smarterclayton @timothysc @lavalamp
```release-note
Add a flag allowing contention profiling of the API server
```
Automatic merge from submit-queue
Add authz to psp admission
Add authz integration to PSP admission to enable granting access to use specific PSPs on a per-user and per-service account basis. This allows an administrator to use multiple policies in a cluster that grant different levels of access for different types of users.
Builds on https://github.com/kubernetes/kubernetes/pull/32555. Second commit adds authz check to matching policy function in psp admission.
@deads2k @sttts @timstclair
Automatic merge from submit-queue
Better messaging for missing volume binaries on host
**What this PR does / why we need it**:
When mount binaries are not present on a host, the error returned is a generic one.
This change is to check the mount binaries before the mount and return a user-friendly error message.
This change is specific to GCI and the flag is experimental now.
https://github.com/kubernetes/kubernetes/issues/36098
**Release note**:
Introduces a flag `check-node-capabilities-before-mount` which if set, enables a check (`CanMount()`) prior to mount operations to verify that the required components (binaries, etc.) to mount the volume are available on the underlying node. If the check is enabled and `CanMount()` returns an error, the mount operation fails. Implements the `CanMount()` check for NFS.
Sample output post change :
rkouj@rkouj0:~/go/src/k8s.io/kubernetes$ kubectl describe pods
Name: sleepyrc-fzhyl
Namespace: default
Node: e2e-test-rkouj-minion-group-oxxa/10.240.0.3
Start Time: Mon, 07 Nov 2016 21:28:36 -0800
Labels: name=sleepy
Status: Pending
IP:
Controllers: ReplicationController/sleepyrc
Containers:
sleepycontainer1:
Container ID:
Image: gcr.io/google_containers/busybox
Image ID:
Port:
Command:
sleep
6000
QoS Tier:
cpu: Burstable
memory: BestEffort
Requests:
cpu: 100m
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment Variables:
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
Volumes:
data:
Type: NFS (an NFS mount that lasts the lifetime of a pod)
Server: 127.0.0.1
Path: /export
ReadOnly: false
default-token-d13tj:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-d13tj
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
7s 7s 1 {default-scheduler } Normal Scheduled Successfully assigned sleepyrc-fzhyl to e2e-test-rkouj-minion-group-oxxa
6s 3s 4 {kubelet e2e-test-rkouj-minion-group-oxxa} Warning FailedMount Unable to mount volume kubernetes.io/nfs/32c7ef16-a574-11e6-813d-42010af00002-data (spec.Name: data) on pod sleepyrc-fzhyl (UID: 32c7ef16-a574-11e6-813d-42010af00002). Verify that your node machine has the required components before attempting to mount this volume type. Required binary /sbin/mount.nfs is missing
Automatic merge from submit-queue
Add Windows support to kube-proxy
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
This is the first stab at supporting kube-proxy (userspace mode) on Windows
**Which issue this PR fixes** :
fixes#30278
**Special notes for your reviewer**:
The MVP uses `netsh portproxy` to redirect traffic from `ServiceIP:ServicePort` to a `LocalIP:LocalPort`.
For the next version we are expecting to have guidance from Microsoft Container Networking team.
**Limitations**:
Current implementation does not support DNS queries over UDP as `netsh portproxy` currently only supports TCP. We are working with Microsoft to remediate this.
cc: @brendandburns @dcbw
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
```
Automatic merge from submit-queue
Make GCI nodes mount non tmpfs, ext* & bind mounts using an external mounter
This PR downloads the stage1 & gci-mounter ACIs as part of cluster bring up instead of downloading them dynamically from gcr.io, which was the cause for #36206.
I have also optimized the containerized mounter to pre-load the mounter image once to avoid fetch latency while using it.
Original PR which got reverted: https://github.com/kubernetes/kubernetes/pull/35821
```release-note
GCI nodes use an external mounter script to mount NFS & GlusterFS storage volumes
```
@mtaufen Node e2e is not re-enabled in this PR.
cc @jingxu97
Automatic merge from submit-queue
specify custom ca file to verify the keystone server
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
Sometimes the keystone server's certificate is self-signed, mainly used for internal development, testing and etc.
For this kind of ca, we need a way to verify the keystone server.
Otherwise, below error will occur.
> x509: certificate signed by unknown authority
This patch provide a way to pass in a ca file to verify the keystone server when starting `kube-apiserver`.
**Which issue this PR fixes** : fixes#22695, #24984
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
``` release-note
```
Provides an opt-in flag, --experimental-fail-swap-on (and corresponding
KubeletConfiguration value, ExperimentalFailSwapOn), which is false by default.
Automatic merge from submit-queue
Rename experimental-runtime-integration-type to experimental-cri
Also rename the field in the component config to `EnableCRI`
If we have a public & private zone with the same name (which is common
on AWS), this means we can still create records. Also tighten up some
of the logic to allow for zones with duplicate names.
Automatic merge from submit-queue
Add verify script for federation OpenAPI spec generation
We are missing hack/verify-federation-openapi-spec.sh. It should be something similar to hack/verify-openapi-spec.sh.
Automatic merge from submit-queue
Fix hack/local-up by declaring defaults for unbound env vars
The variables were not defaulted and caused hack/local-up-cluster to not work.
Automatic merge from submit-queue
Node Conformance Test: Add system verification
For #30122 and #29081.
This PR introduces system verification test in node e2e and conformance test. It will run before the real test. Once the system verification fails, the test will just fail. The output of the system verification is like this:
```
I0909 23:33:20.622122 2717 validators.go:45] Validating os...
OS: Linux
I0909 23:33:20.623274 2717 validators.go:45] Validating kernel...
I0909 23:33:20.624037 2717 kernel_validator.go:79] Validating kernel version
KERNEL_VERSION: 3.16.0-4-amd64
I0909 23:33:20.624146 2717 kernel_validator.go:93] Validating kernel config
CONFIG_NAMESPACES: enabled
CONFIG_NET_NS: enabled
CONFIG_PID_NS: enabled
CONFIG_IPC_NS: enabled
CONFIG_UTS_NS: enabled
CONFIG_CGROUPS: enabled
CONFIG_CGROUP_CPUACCT: enabled
CONFIG_CGROUP_DEVICE: enabled
CONFIG_CGROUP_FREEZER: enabled
CONFIG_CGROUP_SCHED: enabled
CONFIG_CPUSETS: enabled
CONFIG_MEMCG: enabled
I0909 23:33:20.679328 2717 validators.go:45] Validating cgroups...
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
I0909 23:33:20.679454 2717 validators.go:45] Validating docker...
DOCKER_GRAPH_DRIVER: aufs
```
It verifies the system following a predefined `SysSpec`:
``` go
// DefaultSysSpec is the default SysSpec.
var DefaultSysSpec = SysSpec{
OS: "Linux",
KernelVersion: []string{`3\.[1-9][0-9].*`, `4\..*`}, // Requires 3.10+ or 4+
// TODO(random-liu): Add more config
KernelConfig: KernelConfig{
Required: []string{
"NAMESPACES", "NET_NS", "PID_NS", "IPC_NS", "UTS_NS",
"CGROUPS", "CGROUP_CPUACCT", "CGROUP_DEVICE", "CGROUP_FREEZER",
"CGROUP_SCHED", "CPUSETS", "MEMCG",
},
Forbidden: []string{},
},
Cgroups: []string{"cpu", "cpuacct", "cpuset", "devices", "freezer", "memory"},
RuntimeSpec: RuntimeSpec{
DockerSpec: &DockerSpec{
Version: []string{`1\.(9|\d{2,})\..*`}, // Requires 1.9+
GraphDriver: []string{"aufs", "overlay", "devicemapper"},
},
},
}
```
Currently, it only supports:
- Kernel validation: version validation and kernel configuration validation
- Cgroup validation: validating whether required cgroups subsystems are enabled.
- Runtime Validation: currently, only validates docker graph driver.
The validating framework is ready. The specific validation items could be added over time.
@dchen1107
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Rename PetSet to StatefulSet in docs and examples.
**What this PR does / why we need it**: Addresses some of the pre-code-freeze changes for implementing the PetSet --> StatefulSet rename. (#35534)
**Special notes for your reviewer**: This PR only changes docs and examples, as #35731 hasn't been merged yet and I don't want to create merge conflicts. I'll open another PR for any remaining code changes needed after that PR is merged. /cc @erictune @janetkuo @chrislovecnm
Automatic merge from submit-queue
New command: "kubeadm token generate"
As part of #33930, this PR adds a new top-level command to kubeadm to just generate a token for use with the init/join commands. Otherwise, users are left to either figure out how to generate a token on their own, or let `kubeadm init` generate a token, capture and parse the output, and then use that token for `kubeadm join`.
At this point, I was hoping for feedback on the CLI experience, and then I can add tests. I spoke with @mikedanese and he didn't like the original propose of `kubeadm util generate-token`, so here are the runners up:
```
$ kubeadm generate-token # <--- current implementation
$ kubeadm generate token # in case kubeadm might generate other things in the future?
$ kubeadm init --generate-token # possibly as a subcommand of an existing one
```
Currently, the output is simply the token on one line without any padding/formatting:
```
$ kubeadm generate-token
1087fd.722b60cdd39b1a5f
```
CC: @kubernetes/sig-cluster-lifecycle
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
``` release-note
New kubeadm command: generate-token
```
Automatic merge from submit-queue
Add --force to kubectl delete and explain force deletion
--force is required for --grace-period=0. --now is == --grace-period=1.
Improve command help to explain what graceful deletion is and warn about
force deletion.
Part of #34160 & #29033
```release-note
In order to bypass graceful deletion of pods (to immediately remove the pod from the API) the user must now provide the `--force` flag in addition to `--grace-period=0`. This prevents users from accidentally force deleting pods without being aware of the consequences of force deletion. Force deleting pods for resources like StatefulSets can result in multiple pods with the same name having running processes in the cluster, which may lead to data corruption or data inconsistency when using shared storage or common API endpoints.
```
Automatic merge from submit-queue
make using service account credentials from controllers optional
Adds a flag to make the "SA per controller" behavior optional.
Automatic merge from submit-queue
add script to check for updates to the files for generation
Adds a verify script to help people writing APIs remember to wire all the individual pieces up. Eventually it will be better to avoid this altogether, but this is a start.
@ncdc fyi
Automatic merge from submit-queue
kubectl: make edit work with lists again
@kubernetes/kubectl this is fixing https://github.com/kubernetes/kubernetes/issues/20519 and slightly changes the behavior of --recursive when the directory that is being edited has files with errors. Previously since `edit` was working on an object basis, bad objects would be skipped and the editor would load the next object. We want to load multiple objects in the same list and it's impossible to load invalid objects in a list so --recursive will not work if there is any error in the directory. I think this is an acceptable trade-off.
Review here: https://github.com/kubernetes/kubernetes/pull/36148/files?w=1
Automatic merge from submit-queue
Switch DisruptionBudget api from bool to int allowed disruptions [only v1beta1]
Continuation of #34546. Apparently it there is some bug that prevents us from having 2 different incompatibile version of API in integration tests. So in this PR v1alpha1 is removed until testing infrastructure is fixed.
Base PR comment:
Currently there is a single bool in disruption budget api that denotes whether 1 pod can be deleted or not. Every time a pod is deleted the apiserver filps the bool to false and the disruptionbudget controller sets it to true if more deletions are allowed. This works but it is far from optimal when the user wants to delete multiple pods (for example, by decreasing replicaset size from 10000 to 8000).
This PR adds a new api version v1beta1 and changes bool to int which contains a number of pods that can be deleted at once.
cc: @davidopp @mml @wojtek-t @fgrzadkowski @caesarxuchao
Automatic merge from submit-queue
Disable the test-owners verify step until the merge conflicts are resolved
It's causing more pain than it's preventing currently. There should be some simpler ways to fix this.
Automatic merge from submit-queue
Fix typo in local-up-cluster
Use curly braces instead of round ones for bash variable expansion. The
script complains about unknown ROOT_CA_FILE when running with
KUBE_ENABLE_CLUSTER_DNS=true.
Added test cases to show that both flags are operating as intended.
Removed the print statement "running in local mode" as in PR#35112
The previous attempt at the PR (PR#35050) was reverted for causeing a flake.
I believe that setting the deployments terminationGracePeriodSeconds to 0 should
take care of it. I ran hack/make-rules/test-cmd.sh 50 times in a row without encountering
the flake
Automatic merge from submit-queue
[Kubelet] Use the custom mounter script for Nfs and Glusterfs only
This patch reduces the scope for the containerized mounter to NFS and GlusterFS on GCE + GCI clusters
This patch also enabled the containerized mounter on GCI nodes
Shepherding multiple PRs through the submit queue is painful. Hence I combined them into this PR. Please review each commit individually.
cc @jingxu97 @saad-ali
https://github.com/kubernetes/kubernetes/pull/35652 has also been reverted as part of this PR
Automatic merge from submit-queue
pod and qos level cgroup support
```release-note
[Kubelet] Add alpha support for `--cgroups-per-qos` using the configured `--cgroup-driver`. Disabled by default.
```
Automatic merge from submit-queue
Move Statefulset (previously PetSet) to v1beta1
**What this PR does / why we need it**: #28718
**Which issue this PR fixes** _(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)_: fixes #
**Special notes for your reviewer**: depends on #35663 (PetSet rename)
cc @erictune @foxish @kubernetes/sig-apps
**Release note**:
``` release-note
v1beta1/StatefulSet replaces v1alpha1/PetSet.
```
Use curly braces instead of round ones for bash variable expansion. The
script complains about unknown ROOT_CA_FILE when running with
KUBE_ENABLE_CLUSTER_DNS=true.
Automatic merge from submit-queue
Retry "godep restore" once in verify-dockerized.sh
This should fix#36110. The current flake rate is 0.5%, so with the
unrealistic assumption of uncorrelated flakes, a single retry will bring
it down to <0.01% flake rate.
Automatic merge from submit-queue
fix verify-bazel.sh on mac and windows
mac bash doesn't like |& because mac bash is really old. the formating of wc
is also slightly different then on linux.
```console
$ # on linux
$ echo -n | wc -l
0
$ # on mac
$ echo -n | wc -l
0
```
Automatic merge from submit-queue
[Federation][init] Implement `kubefed init` command that performs federation control plane bootstrap.
Please review only the last commit here. This is based on PR #35593 which will be reviewed independently.
I am intentionally not including the unit tests in this PR to better distribute and parallelize reviews. This PR is already big.
I will add a release note separately for this entire feature, so please don't worry too much about the release note here in the PR.
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @quinton-hoole @nikhiljindal
This should fix#36110. The current flake rate is 0.5%, so with the
unrealistic assumption of uncorrelated flakes, a single retry will bring
it down to <0.01% flake rate.
Fixes#35850.
GitHub's merging doesn't support .gitattributes-- I verified this with
support. There may be a workaround with line padding or making the bot
do merges on its own, which is beneficial for custom merge conflict
resolution in general.
Automatic merge from submit-queue
[Federation][(Un)join-01] Refactor common functions and structs into a util package.
Please review only the last commit here. This is based on PR #35495 which will be reviewed independently.
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @quinton-hoole @nikhiljindal
--force is required for --grace-period=0. --now is == --grace-period=1.
Improve command help to explain what graceful deletion is and warn about
force deletion.
Automatic merge from submit-queue
[Federation][unjoin-00] Implement `kubefed unjoin` command.
Please review only the last commit here. This is based on PR #35493 which will be reviewed independently.
I will add a release note separately for this entire feature, so please don't worry too much about the release note here in the PR.
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @quinton-hoole @nikhiljindal
mac bash doesn't like |& because mac bash is really old. the formating of wc
is also sligthly different then on linux
```
$ # on linux
$ echo -n | wc -l
0
$ # on mac
$ echo -n | wc -l
0
```
Automatic merge from submit-queue
Move .gitattributes annotation to the root, so GitHub will respect them.
This should fix the merge conflicts by letting GitHub use the simpler line-by-line algorithm for this file. Having .gitattributes in a sub-directory would work for local merging, but would show conflicts on the web UI.
Automatic merge from submit-queue
[Federation][join-01] Implement `kubefed join` command.
Supersedes PR #35155.
Please review only the last commit here. This is based on PR #35492 which will be reviewed independently.
I will add a release note separately for this entire feature, so please don't worry too much about the release note here in the PR.
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @quinton-hoole @mwielgus
Automatic merge from submit-queue
Update verify-openapi-spec script to check for extra generated spec
hack/verify-openapi-spec.sh only check for existing spec changes. If for some reason (here most probably I forgot to delete a file in api/openapi-spec folder in #35388 after a rebase) there is an old spec exists in the spec folder, it won't panic but it should. This resulted in an unused out of date v1.spec file in the api/openapi-spec folder that this PR also removes.
Automatic merge from submit-queue
e2e.go/kops: Bump timeout to 20m, fix KUBERNETES_PROVIDER
**What this PR does / why we need it**: I don't have a ton of proof, but I think https://k8s-testgrid.appspot.com/google-aws#kops-aws-updown builds 4045-4047 are just AWS and DNS slowness.
In addition, my original PR was meant to change `KUBERNETES_PROVIDER` based on `KUBERNETES_CONFORMANCE_PROVIDER`.
Along the way: Fix ginkgo-e2e.sh. This change got dropped in the
original PR, but it was meant to allow a conformance-style
kubectl-auth test and still have a legit cloud provider.
Automatic merge from submit-queue
Add tooling to generate listers
Add lister-gen tool to auto-generate listers. So far this PR only demonstrates replacing the manually-written `StoreToLimitRangeLister` with the generated `LimitRangeLister`, as it's a small and easy swap.
cc @deads2k @liggitt @sttts @nikhiljindal @lavalamp @smarterclayton @derekwaynecarr @kubernetes/sig-api-machinery @kubernetes/rh-cluster-infra
Automatic merge from submit-queue
SELinux Overhaul
Overhauls handling of SELinux in Kubernetes. TLDR: Kubelet dir no longer has to be labeled `svirt_sandbox_file_t`.
Fixes#33351 and #33510. Implements #33951.
Automatic merge from submit-queue
Allow apiserver to choose preferred kubelet address type
Follow up to #33718 to stay compatible with clusters using DNS names for master->node communications. Adds the `--kubelet-preferred-address-types` apiserver flag for clusters that prefer a different node address type.
```release-note
The apiserver can now select which type of kubelet-reported address to use for master->node communications, using the --kubelet-preferred-address-types flag.
```
Automatic merge from submit-queue
make localupcluster work with RBAC enabled
When `ENABLE_RBAC=true` is set to true, `local-up-cluster` fails because it can't reach the endpoint (get a 403). This changes to an endpoint that the default policy allows anyone to see.
We can put subdomains into hosted zones (for example,
foo.federation.example.com can be hosted in example.com)
By allowing sharing a common hosted zone, this means the user doesn't
have to do as much setup.
Automatic merge from submit-queue
fixed some issues with kubectl set resources
when using kubectl set resources it resets all resource fields that are not being set.
for example
# kubectl set resources deployments nginx --limits=cpu=100m
followed by
# kubectl set resources deployments nginx --limits=memory=256Mi
would result in the nginx deployment only limiting memory at 256Mi with the previous
limit placed on the cpu being wiped out. This behavior is corrected so that each invocation
only modifies fields set in that command and changed the testing so that the desired behavior
is checked.
Also a typo:
you must specify an update to requests or limits or (in the form of --requests/--limits)
corrected to
you must specify an update to requests or limits (in the form of --requests/--limits)
Implemented both the dry run and local flags.
Added test cases to show that both flags are operating as intended.
Removed the print statement "running in local mode" as in PR#35112
Automatic merge from submit-queue
Verify and update client-go staging area for every PR
We need to keep the staging area up-to-date to prevent PRs from breaking client-go.
It's marked as "WIP" because we need to decide the [versioning strategy](https://github.com/kubernetes/client-go/issues/9) for client-go first. This PR contains breaking changes for client-go.
This is blocking #29934 and potentially #34441
cc @kubernetes/sig-api-machinery
Automatic merge from submit-queue
Let release_1_5 clientset include multiple versions of a group
Fix#35237
This PR make versioned clientset to include multiple versions of a group. Currently only `batch` has `v1` and `v2alpha1`. The clientset interface now looks like:
```go
BatchV2alpha1() v2alpha1batch.BatchV2alpha1Interface
BatchV1() v1batch.BatchV1Interface
// Deprecated: please explicitly pick a version if possible.
Batch() v1batch.BatchV1Interface
```
Commit "update client-gen to say internalversion rather than unversioned" fixes https://github.com/kubernetes/kubernetes/issues/24481.
cc @kubernetes/sig-api-machinery @soltysh @deads2k @nikhiljindal
```release-note
release_1_5 clientset supports multiple versions of a group.
```
Automatic merge from submit-queue
Make overlapping deployments deletable
@kubernetes/deployment ptal
Fixes https://github.com/kubernetes/kubernetes/issues/34466 by 1) not adding the overlapping annotation in the working deployment, 2) updates observedGeneration for overlapping deployments, and 3) updates the kubectl deployment reaper to do non-cascading deletion for deployments with the overlapping annotation.
Automatic merge from submit-queue
support editing before creating resource
Support `kubectl create -f config.yaml --edit`
Support editing before creating resource from files, urls and stdin.
The behavior is similar to `kubectl edit`
It won't create anything when edit make no change.
partial: #18064
Based on: #33686 and #33973
```release-note
Support editing before creating resource from files, urls and stdin, e.g. `kubectl create -f config.yaml --edit`
It won't create anything when edit make no change.
```
Automatic merge from submit-queue
allow authentication through a front-proxy
This allows a front proxy to set a request header and have that be a valid `user.Info` in the authentication chain. To secure this power, a client certificate may be used to confirm the identity of the front proxy
@kubernetes/sig-auth fyi
@erictune per-request
@liggitt you wrote the openshift one, ptal.
when using kubectl set resources it resets all resource fields that are not being set.
for example
# kubectl set resources deployments nginx --limits=cpu=100m
followed by
# kubectl set resources deployments nginx --limits=memory=256Mi
would result in the nginx deployment only limiting memory at 256Mi with the previous
limit placed on the cpu being wiped out. This behavior is corrected so that each invocation
only modifies fields set in that command and changed the testing so that the desired behavior
is checked.
Also a typo:
you must specify an update to requests or limits or (in the form of --requests/--limits)
corrected to
you must specify an update to requests or limits (in the form of --requests/--limits)
changelog:
- fixed a typo in hack/make-rules/test-cmd.sh "effecting" to "affecting"
Automatic merge from submit-queue
remove the non-generated client
Removes the non-generated client from kube. The package has a few methods left, but nothing that needs updating when adding new groups.
@ingvagabund
Automatic merge from submit-queue
Fix migration script to make it sh-friendly
Ref #22448
There is no bash in etcd image, so the script needs to be "sh-friendly".
Also, few fixes to the script.
Automatic merge from submit-queue
rename kubelet flag mounter-path to experimental-mounter-path
```release-note
* Kubelet flag '--mounter-path' renamed to '--experimental-mounter-path'
```
The feature the flag controls is an experimental feature and this renaming ensures that users do not depend on this feature just yet.
Automatic merge from submit-queue
Build: Use checksum to rsync results
If dependencies are touched, the Makefile will detect that tools like
deepcopy-gen need to be rebuilt. It will rebuild them, but if Go decides the
dependencies are not ACTUALLY updated (maybe it was a test file - known
limitation of the current Makefile), it does not touch the resulting binary.
The Makefile touches the result explicitly to mark that the dep change has
been handled. But rsync blows away that change with the older file, and
helpfully preserves the timestamp. This repeats on every build.
Now rsync ignores it if the checksum is the same. Result = faster build.
@wojtek-t @gmarek
If dependencies are touched, the Makefile will detect that tools like
deepcopy-gen need to be rebuilt. It will rebuild them, but Go decides the
dependencies are not actually updated (maybe it was a test file - known
limitation of the current Makefile) and does not touch the resulting binary.
Then the Makefile touches the result explicitly to mark thatthe dep change has
been handled. Then rsync blows away that change with the older file, and
helpfully preserves the timestamp.
Now rsync ignores it if the checksum is the same. Result = faster build.
Automatic merge from submit-queue
Add GroupVersion tags to OpenAPI spec and remove all specs except main one
Tags are used as a grouping mechanism in OpenAPI. We generated one spec per GroupVersion before for this grouping but by adding those tags in this PR, those files have no use. We can always add them back if there were a use-case for them.
**Release note**:
```release-note
Deprecate OpenAPI spec for GroupVersion endpoints in favor of single spec /swagger.json
```
Reference: #13414
hack/verify-boilerplate.sh -v will now print out why the file does
not match along with a diff if possible.
Note: boilerplate.py now has a unit test that is run along with
hack/verify-boilerplate.sh.
Automatic merge from submit-queue
use LC_ALL=C on all sort commands
Most of these don't matter, but this makes it predicable no matter who runs the commands.
Automatic merge from submit-queue
Add `--mounter-path` flag to kubelet that will allow overriding the `mount` command used by kubelet
In order to be able to use new mounter library, this PR adds the
mounterPath flag to kubelet which passes the flag to the mount
interface. If flag is empty, mount uses default mount path.
In order to be able to use new mounter library, this PR adds the
mounterPath flag to kubelet which passes the flag to the mount
interface. If flag is empty, mount uses default mount path.
Lots of places used sort (or sort -u) but didn't set LC_ALL=C. This
means it could be slightly different on different people's systems. Make
it deterministic.
Automatic merge from submit-queue
Retry curl when it failed during download
Fixes#34769
For now I'm adding retry only to etcd installation as I saw it recently to fail in one of my PRs. We may change it also in other places if it also happens.
Generate a kubeconfig for use with controller-manager, kubelet,
scheduler etc. This kubeconfig should use the secure https
port of the api server with appropriate ca cert for the components
to talk to api server.
With this change, one can set API_PORT=0 to completely switch off
insecure access for testing admission controllers etc.
Fixes#33375
Automatic merge from submit-queue
WIP: Remove the legacy networking mode
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
Removes the deprecated configure-cbr0 flag and networking mode to avoid having untested and maybe unstable code in kubelet, see: #33789
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
fixes#30589fixes#31937
**Special notes for your reviewer**: There are a lot of deployments who rely on this networking mode. Not sure how we deal with that: force switch to kubenet or just delete the old deployment?
But please review the code changes first (the first commit)
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
Removed the deprecated kubelet --configure-cbr0 flag, and with that the "classic" networking mode as well
```
PTAL @kubernetes/sig-network @kubernetes/sig-node @mikedanese
Automatic merge from submit-queue
build: Fix version generation.
This PR provides update to version string generation procedure to handle additional case - working on branch delivered from release tag, instead of working on `-(alpha|beta).no` suffixed tag.
Reason of it is that actually if someone would start working on branch forked from release tag (`v1.4.1` for example), will add some commits and then would try to build k8s/run conformance tests - will end up with broken version string in form similar to `v1.4.1.X+YYYY` where `X` is a dictance/number of commits from base tag and `YYYY` will be first 14 characters from commit hash.
Such version - containing four dotted parts is rejected during conformance tests with error similar to:
```
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubectl.go:843
Oct 11 12:38:33.726: Failed to get server version: Unable to parse server version "v1.4.1.5+somecommithash": Invalid character(s) found in patch number "1.5"
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:413
```
This PR provides a cure for this situation creating version string in form `v1.4.1-X+YYYY` which then should be readable for version validation checks.
Release note:
```release-note
Fix version string generation for local version different from release and not based on `-alpha.no` or `-beta.no` suffixed tag.
```
Automatic merge from submit-queue
add optional mutation checks for shared informer cache
We need to make sure that no one is mutating caches if they're using a shared informer. It is important that whatever is tracking those changes gets the object *before* anyone else possibly could.
This adds the ability to track the original objects in the cache and their current values. Go doesn't have an exit hook or a way to say "wait for non-daemon go-funcs to complete before exit", so this runs a gofunc on a loop that can panic the entire process. It's gated behind an env var.
@derekwaynecarr did I get the right spots to make sure that e2e runs with this flag?
@smarterclayton @kubernetes/rh-cluster-infra
Automatic merge from submit-queue
docs generation: Use macOS compatible copying method
Replace option unknown for bsd version of `cp` used on macOS with compatible between systems replacement.
Closes#34890
```release-note
NONE
```
Automatic merge from submit-queue
hack/: update cherry-pick script to have settings to override remote names
**What this PR does / why we need it**:
it allows you to override the remote names for the cherry-pick script in case you have it named differently
Automatic merge from submit-queue
Improvements to CLI usability and maintainability
Improves `kubectl` from an usability perspective by
1. Fixing how we handle terminal width in help. Some sections like the flags use the entire available width, while others like long descriptions breaks lines but don't follow a well established max width (screenshot below). This PR adds a new responsive writer that will adjust to terminal width and set 80, 100, or 120 columns as the max width, but not more than that given POSIX best practices and recommendations for better readability.
2. Adds our own normalizers for long descriptions and cmd examples which allows us better control about how things like lists, paragraphs, line breaks, etc are printed. Features markdown support. Looks like `templates.LongDesc` and `templates.Examples` instead of `dedent.Dedend`.
3. Allows simple reordering and reuse of help and usage sections.
3. Adds `verify-cli-conventions.sh` which intends to run tests to make sure cmd developers are using what we propose as [kubectl conventions](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/kubectl-conventions.md). Just a couple simple tests for now but the framework is there and it's easy to extend.
4. Update [kubectl conventions](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/kubectl-conventions.md) to use our own normalizers instead of `dedent.Dedent`.
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
Improves how 'kubectl' uses the terminal size when printing help and usage.
```
@kubernetes/kubectl
Automatic merge from submit-queue
local-up: Add option to guess binary path
This adds a `-O` flag which guesses the right output directory.
The reason for having two flags, not one, is because bash's `getopt` doesn't let you do optional arguments easily, so having both makes the code simpler.
I also removed the redundant empty check; the bash argument check meant that was never hit.
cc @jayunit100 and @jbeda (arbitrary people from the git log)
Automatic merge from submit-queue
add ownerref permission checks
Adds an admission plugin that ensures that anyone adding an `ownerReference` to a resource has delete rights on the resource they're setting up a delete for.
@caesarxuchao example admission plugin that tests for ownerReference diffs and uses an authorizer to drive the decision.
@liggitt @ncdc we've talked about this before
Automatic merge from submit-queue
Add flags for alternative API and discovery ports
**What this PR does / why we need it**:
We had many issues reported due to the fact that we are using port 443 by default, and we should allow users to override whatever defaults we pick. This doesn't touch on `localhost:8080` yet, which should just generally get rid of.
**Which issue this PR fixes**: fixes#34311#34307#33638
**Special notes for your reviewer**: cc @pires
**Release note**:
```release-note
Add `kubeadm` flags `--api-port` and `--discovery-port`, change default API port to 6443
```
Automatic merge from submit-queue
clientgen: allow to pass custom apiPath when generating client sets
This PR allow to pass the `--clientset-api-path` parameter to clientgen that allows to customize the default API path set in clients. This allows projects like OpenShift to re-use the client with different API path (`/oapi` in our case).
Automatic merge from submit-queue
kubectl: apply prune should fallback to basic delete when a resource has no reaper
Fixes#34790
cc @kubernetes/kubectl @MrHohn
Automatic merge from submit-queue
Add global timeout flag
**Release note**:
```release-note
Add a new global option "--request-timeout" to the `kubectl` client
```
UPSTREAM: https://github.com/kubernetes/client-go/pull/10
This patch adds a global timeout flag (viewable with `kubectl -h`) with
a default value of `0s` (meaning no timeout).
The timeout value is added to the default http client, so that zero
values and default behavior are enforced by the client.
Adding a global timeout ensures that user-made scripts won't hang for an
indefinite amount of time while performing remote calls (right now, remote
calls are re-tried up to 10 times when each attempt fails, however, there is
no option to set a timeout in order to prevent any of these 10 attempts from
hanging indefinitely).
**Example**
```
$ kubectl get pods # no timeout flag set - default to 0s (which means no
timeout)
NAME READY STATUS RESTARTS AGE
docker-registry-1-h7etw 1/1 Running 1 2h
router-1-uv0f9 1/1 Running 1 2h
$ kubectl get pods --request-timeout=0 # zero means no timeout no timeout flag set
NAME READY STATUS RESTARTS AGE
docker-registry-1-h7etw 1/1 Running 1 2h
router-1-uv0f9 1/1 Running 1 2h
$kubectl get pods --request-timeout=1ms
Unable to connect to the server: net/http: request canceled while
waiting for connection (Client.Timeout exceeded while awaiting headers)
```
This patch adds a global timeout flag (viewable with `kubectl -h`) with
a default value of `0s` (meaning no timeout).
The timeout value is added to the default http client, so that zero
values and default behavior are enforced by the client.
**Example**
```
$ kubectl get pods # no timeout flag set - default to 0s (which means no
timeout)
NAME READY STATUS RESTARTS AGE
docker-registry-1-h7etw 1/1 Running 1 2h
router-1-uv0f9 1/1 Running 1 2h
$ kubectl get pods --timeout=0 # zero means no timeout no timeout flag set
NAME READY STATUS RESTARTS AGE
docker-registry-1-h7etw 1/1 Running 1 2h
router-1-uv0f9 1/1 Running 1 2h
$kubectl get pods --timeout=1ms
Unable to connect to the server: net/http: request canceled while
waiting for connection (Client.Timeout exceeded while awaiting headers)
```
Add a way to set resource limits/requests on running pods
Ref: https://github.com/kubernetes/kubernetes/issues/21648
I squashed the commits to make rebasing easier
Change log:
- fixed a typo that caused the command to be run with kubectl set set instead of the correct kubectl set limit
- added a ResourcesWithPodTemplates to pkg/kubectl/cmd/util/factory.go
instead of hardcoding these resources move there description all in one place
- Fixing some of the flow control in kubectl set limit
- update the help info
- changed the name of ResourcesWithPodTemplates to ResourcesWithPodSpecs to more accuratly describe what it is doing
and changed the variable names to lower case to conform to go's variable naming convention
- changing the name of the command from 'set limit' to 'set resources'
- Adding the new file pkg/kubectl/cmd/set/set_resources.go
- changes to the test cases to reflect the change from 'kubectl set limit' to 'kubectl set resources'
- comment removed
- adding the man page to the git repository attempting to fix Jenkins tests
- adding the user guide
- fixed a few typos
- typo in hack/cmd-test.sh
- implamenting suggestions for command help text
- adding the dry-run flag
- removing the "remove" option in favor of zeroing out request/limits in order to remove them
- changed limits/requests to requests/limit
- changing ResourcesWithPodSpec
- updated generated docs and removed whitespace
- change priint on success message from "resource limits/requests updated" to "resource requirements updated"
- minor rebasing issues - 'hack/test-cmd.sh' now passes
- cmdutil.PrintSuccess added another argument
- fixing mungedocs failure
- removed whitespace from hack/make-rules/test-cmd.sh and an erroneous entry from pkg/cloudprovider/providers/openstack/MAINTAINERS.md
- fixed typo in Short: field of the cobra command
- rebased
- Creating a new factory in the ResourcesWithPodSpecs() so that the testing will pass
- changing ResourcesWithPodSpecs, it doesn't need to be a method of factory
Automatic merge from submit-queue
add UpdateRuntimeConfig interface
Expose UpdateRuntimeConfig interface in RuntimeService for kubelet to pass a set of configurations to runtime. Currently it only takes PodCIDR.
The use case is for kubelet to pass configs to runtime. Kubelet holds some config/information which runtime does not have, such as PodCIDR. I expect some of kubelet configurations will gradually move to runtime, but I believe cases like PodCIDR, which dynamically assigned by k8s master, need to stay for a while.
Automatic merge from submit-queue
Increase build verbosity for verify-generated-{protobuf,runtime}.sh
Print out more details of the build process to help with debugging #34675.
Automatic merge from submit-queue
kubeadm implement preflight checks
Checks that user running kubeamd init and join is root and will only execute
command if user is root. Moved away from using kubectl error handling to
having kubeadm handle its own errors. This should allow kubeadm to have
more meaningful errors, exit codes, and logging for specific kubeadm use
cases.
fixes#33908
Automatic merge from submit-queue
add flag for enabling RBAC on local-up-cluster
Enables cases like `ALLOW_ANY_TOKEN=true ENABLE_RBAC=true hack/local-up-cluster.sh ` to startup an api server so you can test RBAC against it.
@pweil- You can try this using `oc login localhost:6443 --token=username/group1,group2,system:masters`. Then you should have full rights. You can use `oc login --token=username` or `kubectl create -f - --token=otheruser/othergroup` to simulate other users.
Add skip-preflight-checks to known flags.
Fix bug with preflight checks not returning system is-active as errors.
Fix error handling to use correct function.
Automatic merge from submit-queue
kube-dns: Add --dns-bind-address flag
Currently, `kube-dns` always binds to `0.0.0.0` when starting the SkyDNS server. Sometimes it's useful to have it bind to a different address for testing. So add a new `--dns-bind-address` flag that can be used to configure this (default is `0.0.0.0`).
Automatic merge from submit-queue
Generate unique Operation IDs for root OpenAPI spec
This PR adds a customization method GetOperationID to OpenAPI spec generation and then use it to make sure root spec has unique operation IDs by mostly adding GroupVersion to the start of operation ID.
Automatic merge from submit-queue
Add gcl cluster logging test
This PR changes default logging destination for tests to gcp and adds test for cluster logging using google cloud logging
Fix#20760
Automatic merge from submit-queue
Add default cluster role bindings
Add default cluster roles bindings to rbac bootstrapping. Also adds a case for allowing escalation when you have no authenticator.
@liggitt I expect you may need to make peace with this.
Automatic merge from submit-queue
log-dump.sh: Add a LOG_DUMP_USE_KUBECTL mode, refactor
**What this PR does / why we need it**: This refactors log-dump.sh so that it can optionally just use `"kubectl get node"` output, for dumping logs off a cluster that was brought up using something other than kube-up.sh (e.g. kops or kubeadm).
Note that for obvious reasons, if the cluster is fried, it'll probably fail to hit the apiserver and all bets are off, but for the 95% case where you're debugging tests, this is good. To cover the remaining 5%, you could actually squirrel away the relevant external IPs after the e2e.go:Up stage (because we better be able to hit the apiserver then) and fetch them from cache later.
**Special notes for your reviewer**: Sorry for so many changes. :)
This refactors log-dump.sh so that it can optionally just use "kubectl
get node" output, for dumping logs off a cluster that was brought up
using sometihng other than kube-up.sh (e.g. kops or kubeadm).
Note that for obvious reasons, if the cluster is fried, it'll probably
fail to hit the apiserver and all bets are off, but for the 95% case
where you're debugging tests, this is good. To cover the remaining 5%,
you could actually squirrel away the relevant external IPs after the
e2e.go:Up stage (because we better be able to hit the apiserver then)
and fetch them from cache later.
Automatic merge from submit-queue
hack: update cherry-pick script to show subject when patch is split
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
This is a continuation of #34049, which worked, but then I realized some patch files have more than one subject, see example: https://github.com/kubernetes/kubernetes/pull/34228
This cleans the output so it looks like this:
```
Automated cherry pick of #32593
Cherry pick of #32593 on release-1.4.
#32593: Fix audit_test regex for iso8601 timestamps
```
pretty!!!
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
Signed-off-by: Jess Frazelle <acidburn@google.com>
Automatic merge from submit-queue
hack/e2e.go: listen for Interrupt signal and attempt cleanup if run with --down
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**: adds a listener on ^C, so if `hack/e2e.go` is run with `--down` and the user ^C, then it will attempt to cleanup the resources.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
NONE
```
Signed-off-by: Jess Frazelle <me@jessfraz.com>
Automatic merge from submit-queue
Generate and store federation OpenAPI spec in source tree
A follow up of #33628 to also add federation spec to source tree.
Automatic merge from submit-queue
controller: save older revisions for Deployment's replica sets
@jwforres the only usable way I could find for multiple old revisions for a single replica set is to stuff them as comma-separated values.
@kubernetes/deployment this retains old revisions served by a replica set inside an annotation.
Fixes https://github.com/kubernetes/kubernetes/issues/33844
Automatic merge from submit-queue
user-parameter support for API_BIND_ADDRESS in local-up-cluster.sh
**What this PR does / why we need it**:
Fixes#34262.
user-parameter for the secure api bind address when running a local dev cluster.
Default bind-address is `0.0.0.0` if not specified by the user. I'm guessing this was the intended behaviour of the original code.
Allow to reach the secure API on different interfaces/IPs when running a dev local-up-cluster.
Automatic merge from submit-queue
Add cgroup-driver and cgroups-per-qos flags to kubelet
Add the flags needed to support pod-level cgroups to kubelet.
/cc @vishh @dchen1107 @dubstack
Automatic merge from submit-queue
add delete-namespace-on-failure flag
I have been doing this for a while.
Setting `--delete-namespace=false --clean-start=true` only works if you have only one e2e test running in a loop.
This PR lets someone to set `delete-namespace-on-failure=false` and run multiple tests in parallel and preserve the crime scene. It makes it easier to reproduce failures.
Let me know if this is worth it or there are some other tricks I am not aware.
Automatic merge from submit-queue
Ignore troublesome paths that cause coverage to fail
**What this PR does / why we need it**:
`KUBE_COVER=y make check` currently fails, this patch fixes it.
**Which issue this PR fixes**
fixes#31691
**Special notes for your reviewer**:
None
**Release note**:
```release-note
NONE
```
This avoids the whole command failing because of errors like the following:
```
# cover k8s.io/kubernetes/pkg/client/restclient
cover: internal error: block 268 overlaps block 270
```
Automatic merge from submit-queue
Update client config invalid option errors to be more specific
This patch adds better error handling for cases where a global option (such as --context or --cluster) causes an invalid config to be returned.
```release-note
release-note-none
```
This patch provides a more relevant error message when a client
configuration option is passed with an invalid or non-existent value.
`$ kubectl get pods --cluster="non-existent"`
```
error: No configuration file found, please login or point to an existing
file
```
`$ kubectl get pods --cluster="non-existent"`
```
error: cluster "non-existent" does not exist
```
Automatic merge from submit-queue
Fix a typo in openapi spec generation
A typo in util.sh resulted in the wrong spec stored in source tree. The commit should be explanatory.
Kubernetes Submit Queue
