Automatic merge from submit-queue
make kube-aggregator run as static pod for local-up-cluster
Runs the kube-aggregator as a static pod for local-up-cluster. Looks like someone broke kubectl negotiation again, so I'll fix that up separately.
@kubernetes/sig-api-machinery-misc
@lavalamp you're probably looking to run kube-aggregator as a static pod, here's an example.
@jwforres I'll make a secure variant for wiring up to openshift.
Automatic merge from submit-queue (batch tested with PRs 41360, 41423, 41430, 40647, 41352)
move kubeadm api group testing to kubeadm package
I think this is sufficient to at least preserve round trip testing.
Automatic merge from submit-queue (batch tested with PRs 40297, 41285, 41211, 41243, 39735)
cluster/gce: Add env var to enable apiserver basic audit log.
For now, this is focused on a fixed set of flags that makes the audit
log show up under /var/log/kube-apiserver-audit.log and behave similarly
to /var/log/kube-apiserver.log. Allowing other customization would
require significantly more complex changes.
Audit log rotation is handled the same as for `kube-apiserver.log`.
**What this PR does / why we need it**:
Add a knob to enable [basic audit logging](https://kubernetes.io/docs/admin/audit/) in GCE.
**Which issue this PR fixes**:
**Special notes for your reviewer**:
We would like to cherrypick/port this to release-1.5 also.
**Release note**:
```release-note
The kube-apiserver [basic audit log](https://kubernetes.io/docs/admin/audit/) can be enabled in GCE by exporting the environment variable `ENABLE_APISERVER_BASIC_AUDIT=true` before running `cluster/kube-up.sh`. This will log to `/var/log/kube-apiserver-audit.log` and use the same `logrotate` settings as `/var/log/kube-apiserver.log`.
```
Automatic merge from submit-queue (batch tested with PRs 41378, 41413, 40743, 41155, 41385)
move kube-aggregator to staging
Straight move of kube-aggregator to staging. The scripts are the interesting bits.
@sttts @lavalamp
This change makes kubelet to use the CRI implementation by default,
unless the users opt out explicitly by using --enable-cri=false.
For the rkt integration, the --enable-cri flag will have no effect
since rktnetes does not use CRI.
Also, mark the original --experimental-cri flag hidden and deprecated,
so that we can remove it in the next release.
For now, this is focused on a fixed set of flags that makes the audit
log show up under /var/log/kube-apiserver-audit.log and behave similarly
to /var/log/kube-apiserver.log. Allowing other customization would
require significantly more complex changes.
Audit log rotation is handled externally by the wildcard /var/log/*.log
already configured in configure-helper.sh.
Automatic merge from submit-queue (batch tested with PRs 41357, 41178, 41280, 41184, 41278)
Add verify-gofmt as a Bazel test.
Also, moved `--flaky_test_attempts=3` out of `.bazelrc` since it was annoying. I think this verify-all pattern is working well in test-infra, we should try and make it happen here.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
copy pkg/util/logs to apiserver
This is a copy, not a move. API servers need to be able to init the logs, but so do clients. It would be weird to have the client-side commands depending on the server side logs utilities.
I updated all the server side references, but left the client-side ones.
@sttts @kubernetes/sig-api-machinery-pr-reviews acceptable?
Automatic merge from submit-queue
Added kubectl create role command
Added `kubectl create role` command.
Fixed part of #39596
**Release note**:
```
Added one new command `kubectl create role` to help user create a single role from command line.
```
Automatic merge from submit-queue
Add pod manifest path to local cluster
Added `POD_MANIFEST_PATH` to local cluster up because we are frequently using this flag when testing static & mirror pod during local dev.
Automatic merge from submit-queue (batch tested with PRs 41259, 41260)
remove hardcoded ansi color coding for `make help`
to tput so that color coding works in multiple platforms including osx.
**What this PR does / why we need it**:
should try not to use hardcoded ansi escape characters.
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 38252, 41122, 36101, 41017, 41264)
BootstrapSigner and TokenCleaner controllers
This is part of https://github.com/kubernetes/features/issues/130 and is an implementation of https://github.com/kubernetes/community/pull/189.
Work that needs to be done yet in this PR:
* [ ] ~~e2e tests~~ Will come in new PR.
* [x] flag to disable this by default
```release-note
Native support for token based bootstrap flow. This includes signing a well known ConfigMap in the `kube-public` namespace and cleaning out expired tokens.
```
@kubernetes/sig-cluster-lifecycle @dgoodwin @roberthbailey @mikedanese
Automatic merge from submit-queue (batch tested with PRs 39418, 41175, 40355, 41114, 32325)
TaintController
```release-note
This PR adds a manager to NodeController that is responsible for removing Pods from Nodes tainted with NoExecute Taints. This feature is beta (as the rest of taints) and enabled by default. It's gated by controller-manager enable-taint-manager flag.
```
Automatic merge from submit-queue (batch tested with PRs 41112, 41201, 41058, 40650, 40926)
Promote TokenReview to v1
Peer to https://github.com/kubernetes/kubernetes/pull/40709
We have multiple features that depend on this API:
- [webhook authentication](https://kubernetes.io/docs/admin/authentication/#webhook-token-authentication)
- [kubelet delegated authentication](https://kubernetes.io/docs/admin/kubelet-authentication-authorization/#kubelet-authentication)
- add-on API server delegated authentication
The API has been in use since 1.3 in beta status (v1beta1) with negligible changes:
- Added a status field for reporting errors evaluating the token
This PR promotes the existing v1beta1 API to v1 with no changes
Because the API does not persist data (it is a query/response-style API), there are no data migration concerns.
This positions us to promote the features that depend on this API to stable in 1.7
cc @kubernetes/sig-auth-api-reviews @kubernetes/sig-auth-misc
```release-note
The authentication.k8s.io API group was promoted to v1
```
Automatic merge from submit-queue (batch tested with PRs 40796, 40878, 36033, 40838, 41210)
HPA v2 (API Changes)
**Release note**:
```release-note
Introduces an new alpha version of the Horizontal Pod Autoscaler including expanded support for specifying metrics.
```
Implements the API changes for kubernetes/features#117.
This implements #34754, which is the new design for the Horizontal Pod Autoscaler. It includes improved support for custom metrics (and/or arbitrary metrics) as well as expanded support for resource metrics. The new HPA object is introduces in the API group "autoscaling/v1alpha1".
Note that the improved custom metric support currently is limited to per pod metrics from Heapster -- attempting to use the new "object metrics" will simply result in an error. This will change once #34586 is merged and implemented.
Automatic merge from submit-queue (batch tested with PRs 40917, 41181, 41123, 36592, 41183)
fix scheduler performance test script
**What this PR does / why we need it**:
`test-performance.sh` is in dir `kubernetes/test/integration/scheduler_perf`
the dir `kubernetes/test/component/scheduler/perf` does not exist
Thanks.
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 40917, 41181, 41123, 36592, 41183)
[Federation] Add override flags options to kubefed init
**What this PR does / why we need it**:
Allows modification of startup flags (of apiserver and controller manager) through kubefed
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
https://github.com/kubernetes/kubernetes/issues/40398
**Special notes for your reviewer**:
I haven't removed the existing redundant flags now (for example --dns-zone-name) intentionally to avoid breaking any existing tests that might use them.
I guess that would be better done as a follow up PR.
@madhusudancs @marun @nikhiljindal
**Release note**:
```
It is now possible for the user to modify any startup flag of federation-apiserver and federation-controller-manager when deployed through kubefed.
There are two new options introduced in kubefed:
--apiserver-arg-overrides and --controllermanager-arg-overrides
Any number of actual federation-apiserver or federation-controller-manager flags can be specified using these options.
Example:
kubefed init "-other options-" ----apiserver-arg-overrides "--flag1=value1,--flag2=value2"
```
Automatic merge from submit-queue (batch tested with PRs 41074, 41147, 40854, 41167, 40045)
Add debug logging to eviction manager
**What this PR does / why we need it**:
This PR adds debug logging to eviction manager.
We need it to help users understand when/why eviction manager is/is not making decisions to support information gathering during support.
Change the name of the client cert used by the controller manager from
system:controller to system:kube-controller-manager, so the appropriate
RBAC rules are applied. Also remove the system:masters group from this
client cert so it doesn't have super powers.
Also, always enable using service account credentials in the controller
manager.
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
add k8s.io/sample-apiserver to demonstrate how to build an aggregated API server
builds on https://github.com/kubernetes/kubernetes/pull/41093
This creates a sample API server is a separate staging repo to guarantee no cheating with `k8s.io/kubernetes` dependencies. The sample is run during integration tests (simple tests on it so far) to ensure that it continues to run.
@sttts @kubernetes/sig-api-machinery-misc ptal
@pwittrock @pmorie @kris-nova an aggregated API server example that will stay up to date.
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
Match etcd process name exactly
A process named, e.g., etcd-operator, should not match
**What this PR does / why we need it**: Matches etcd process name exactly
**Which issue this PR fixes**: fixes#40499
**Special notes for your reviewer**: 👀 @lavalamp
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
Remove deprecated kubelet flags that look safe to remove
Removes:
```
--config
--auth-path
--resource-container
--system-container
```
which have all been marked deprecated since at least 1.4 and look safe to remove.
```release-note
The deprecated flags --config, --auth-path, --resource-container, and --system-container were removed.
```
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
[Federation] Wait after cleanup only if the cleanup succeeds.
It is a waste of time to wait for the resources to cleanup if the
cleanup fails.
cc @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 40873, 40948, 39580, 41065, 40815)
Default target storage in etcd
To make etcd v2->v3 upgrade work correctly, we need to correctly set the "TARGET_STORAGE" env var. Since in head we are defaulting to etcd v3, this PR is defaulting also that env var to etcd3, so that by default upgrade works fine.
@fgrzadkowski @gmarek
Automatic merge from submit-queue (batch tested with PRs 40175, 41107, 41111, 40893, 40919)
remove second CA used for kubelet auth in favor of webhook auth
partial fixes upgrade test.
Automatic merge from submit-queue (batch tested with PRs 40175, 41107, 41111, 40893, 40919)
kubeadm: skip integration tests if kubeadm-cmd-skip flag passed
Will skip integration tests for token generation if it can't find a file by the given --kubeadm-path or default value.
**What this PR does / why we need it**: Tests would fail if just running `go test` in the dir because it expects to have more values. This won't change the behavior of `make test-cmd` which gets run here:
https://github.com/kubernetes/kubernetes/blob/master/Makefile#L258
**Which issue this PR fixes**: fixes#40155
**Special notes for your reviewer**: /cc @pires @pipejakob @liggitt
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41061, 40888, 40664, 41020, 41085)
Cleanup federation components and wait for it only once in a test cycle.
cc @kubernetes/sig-federation-pr-reviews
After today's SIG meeting, it was discussed how to proceed with these
types of test-cmd tests. They will live in kubeamd/test/cmd and will
provide a flag that will allow you to skip them (--kubeadm-cmd-skip) and
by default will fail if kubeadm binary is not present
Automatic merge from submit-queue (batch tested with PRs 40345, 38183, 40236, 40861, 40900)
remove the create-external-load-balancer flag in cmd/expose.go
**What this PR does / why we need it**:
In cmd/expose.go there is a todo "remove create-external-load-balancer in code on or after Aug 25, 2016.", and now it's been a long time past. So I remove this flag and modify the test cases.
Please check for this, thanks!
**Release note**:
```
remove the deprecated flag "create-external-load-balancer" and use --type="LoadBalancer" instead.
```
Automatic merge from submit-queue (batch tested with PRs 41023, 41031, 40947)
scrub aggregator names to eliminate discovery
Cleanup old uses of `discovery`. Also removes the legacy functionality.
@kubernetes/sig-api-machinery-misc @sttts
Automatic merge from submit-queue (batch tested with PRs 40980, 40985)
added short names for resources which are exposed during discovery
**What this PR does / why we need it**:
The changes add short names for resources. The short names will be delivered to kubectl during discovery.
Automatic merge from submit-queue (batch tested with PRs 40971, 41027, 40709, 40903, 39369)
Promote SubjectAccessReview to v1
We have multiple features that depend on this API:
SubjectAccessReview
- [webhook authorization](https://kubernetes.io/docs/admin/authorization/#webhook-mode)
- [kubelet delegated authorization](https://kubernetes.io/docs/admin/kubelet-authentication-authorization/#kubelet-authorization)
- add-on API server delegated authorization
The API has been in use since 1.3 in beta status (v1beta1) with negligible changes:
- Added a status field for reporting errors evaluating access
- A typo was discovered in the SubjectAccessReviewSpec Groups field name
This PR promotes the existing v1beta1 API to v1, with the only change being the typo fix to the groups field. (fixes https://github.com/kubernetes/kubernetes/issues/32709)
Because the API does not persist data (it is a query/response-style API), there are no data migration concerns.
This positions us to promote the features that depend on this API to stable in 1.7
cc @kubernetes/sig-auth-api-reviews @kubernetes/sig-auth-misc
```release-note
The authorization.k8s.io API group was promoted to v1
```
Automatic merge from submit-queue
local-up-cluster: fix instructions for set-credentials
After launching the services, local-up-cluster.sh tells the
user how to configure kubectl to access it. The instructions
for 'set-credentials' enable plain password auth, but the
services are configured to mandate client certificate auth.
As a result it is not possible to access the cluster with
the instructions printed.
The use of client certs by default was added in
commit a1b17db458
Author: Dr. Stefan Schimanski <sttts@redhat.com>
Date: Sat Nov 12 23:09:04 2016 +0100
Configure client certs in local-cluster-up.sh
and the instructions were correctly updated to refer to
client certificates.
The changed instructions were (mistakenly) reverted though
when the following commit was merged:
commit 72e0e91b5e
Author: xilabao <chenr.fnst@cn.fujitsu.com>
Date: Fri Dec 2 11:04:25 2016 +0800
change prompt for enabling RBAC on local-up-cluster
Fixes: #40192
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
After launching the services, local-up-cluster.sh tells the
user how to configure kubectl to access it. The instructions
for 'set-credentials' enable plain password auth, but the
services are configured to mandate client certificate auth.
As a result it is not possible to access the cluster with
the instructions printed.
The use of client certs by default was added in
commit a1b17db458
Author: Dr. Stefan Schimanski <sttts@redhat.com>
Date: Sat Nov 12 23:09:04 2016 +0100
Configure client certs in local-cluster-up.sh
and the instructions were correctly updated to refer to
client certificates.
The changed instructions were (mistakenly) reverted though
when the following commit was merged:
commit 72e0e91b5e
Author: xilabao <chenr.fnst@cn.fujitsu.com>
Date: Fri Dec 2 11:04:25 2016 +0800
change prompt for enabling RBAC on local-up-cluster
Fixes: #40192
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Automatic merge from submit-queue
Using API_HOST_IP to do apiserver health check.
In `hack/local-up-cluster.sh`, it's better to use `API_HOST_IP` to do apiserver health check.
Automatic merge from submit-queue
[Federation][kubefed] Add option to expose federation apiserver on nodeport service
**What this PR does / why we need it**:
This PR adds an option to kubefed to expose federation api server over nodeport. This can be useful to deploy federation in non-cloud environments. This PR is target to address #39271
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
[Federation] kubefed init learned a new flag, `--api-server-service-type`, that allows service type to be specified for the federation API server.
[Federation] kubefed init also learned a new flag, `--api-server-advertise-address`, that allows specifying advertise address for federation API server in case the service type is NodePort.
```
@kubernetes/sig-federation-misc @madhusudancs
Automatic merge from submit-queue
Add SIG to test owners
**What this PR does / why we need it**:
This PR adds a `sig` column to the test owners file generation script.
A problem experienced with the current owners file is that since members are auto-assigned there are times where tests are assigned to non-active users who don't follow up to notifications to fix flakes. By assigning a SIG to each test we can hold a group we know is active responsible for taking care of flakes it's less likely that flakes will fall through the cracks.
**Special notes for your reviewer**:
* A companion PR will go into *kubernetes/contrib* adding support for mungers parsing this new column.
* Another PR in contrib will add labeling GitHub flake issues with the appropriate SIG
* Currently SIGs are not labeled, this will be added in another PR where SIG determinations can be discussed
@saad-ali @pwittrock
Automatic merge from submit-queue (batch tested with PRs 40289, 40877, 40879, 39972, 40942)
Rename experimental-cgroups-per-pod flag
**What this PR does / why we need it**:
1. Rename `experimental-cgroups-per-qos` to `cgroups-per-qos`
1. Update hack/local-up-cluster to match `CGROUP_DRIVER` with docker runtime if used.
**Special notes for your reviewer**:
We plan to roll this feature out in the upcoming release. Previous node e2e runs were running with this feature on by default. We will default this feature on for all e2es next week.
**Release note**:
```release-note
Rename --experiemental-cgroups-per-qos to --cgroups-per-qos
```
Automatic merge from submit-queue (batch tested with PRs 40906, 40924, 40938, 40902, 40911)
print apiserver log location on apiserver error
**What this PR does / why we need it**:
Improve user experience. Attempt to direct user to logs of failing component.
**Special notes for your reviewer**:
In addition to failure, point to logs so that a user can attempt to self remedy and have more information available to debug immediately. A user may not know that the failing component has logs.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40696, 39914, 40374)
Convert hack/e2e.go to a test-infra/kubetest shim
Replaces `hack/e2e.go` for a shim that passes the args to `k8s.io/test-infra/kubetest`
Adds fejta to `hack/OWNERS`
Adds `e2e_test.go` for unit test coverage of the shim.
`Usage: go run hack/e2e.go [--get=true] [--old=1d] -- KUBETEST_ARGS`
In other words there is are `--get` and `--old` shim flags, which control how we upgrade `kubetest`, and a `--` to separate the shim args from the kubetest args, and the existing kubetest args like `--down` `--up`, etc. If only `KUBETEST_ARGS` are used then you can skip the `--` (although golang will complain about it).
Once this is ready to go I will update the kubekins-e2e image to copy this file from test-infra: https://github.com/kubernetes/test-infra/blob/master/jenkins/e2e-image/Dockerfile#L70
ref https://github.com/kubernetes/test-infra/issues/1475
Automatic merge from submit-queue (batch tested with PRs 40862, 40909)
[Federation][kubefed] Add option to disable persistence storage for etcd
**What this PR does / why we need it**:
This is part of updates to enable deployment of federation on non-cloud environments. This pr enables disabling persistent storage for etcd via kubefed.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#40617
**Special notes for your reviewer**:
**Release note**:
```
[Federation] Add --etcd-persistent-storage flag to kubefed to enable/disable persistent storage for etcd
```
cc: @kubernetes/sig-federation-bugs @madhusudancs
Automatic merge from submit-queue (batch tested with PRs 40864, 40666, 38382, 40874)
apply falls back to generic JSON patch computation if no go struct is registered for the target GVK
This PR is the master version of #40096 which is target 1.4 branch.
This PR is based on #40260
- [x] ensure subkey deletion works in CreateThreeWayJSONMergePatch
- [x] ensure type stomping works in CreateThreeWayJSONMergePatch
- [x] lots of tests for generic json patch computation
- [x] apply falls back to generic 3-way JSON merge patch if no go struct is registered for the target GVK
- [x] prevent generic apply patch computation between different apiVersions and/or kinds
- [x] make pruner generic (apply --prune works with TPR)
```release-note
apply falls back to generic 3-way JSON merge patch if no go struct is registered for the target GVK
```
Automatic merge from submit-queue (batch tested with PRs 40884, 40809, 40845, 40866, 40875)
update repo local config to allow redirects from gopkg.in
closes#39583
Automatic merge from submit-queue (batch tested with PRs 40812, 39903, 40525, 40729)
Add CI coverage for client-go and staging/copy.sh
Client-go master and `staging/copy.sh` keep breaking, leading to frustration of users outside of the inner refactoring circle. This PR adds minimal smoke testing for both.
**Note**: this enforces the split of PRs which change apimachinery and client-go in parallel (via some local "temporary commit" hacking).
- [x] @caesarxuchao ~~do we have to delete the example directory from the upstream repo?~~ merge https://github.com/kubernetes/test-infra/issues/1765
Automatic merge from submit-queue (batch tested with PRs 40812, 39903, 40525, 40729)
test/node_e2e: wire-in cri-enabled local testing
This commit wires-in the pre-existing `--container-runtime` flag for
local node_e2e testing.
This is needed in order to further skip docker specific testing
and validation.
Local CRI node_e2e can now be performed via
`make test-e2e-node RUNTIME=remote REMOTE=false`
which will also take care of passing the appropriate argument to
the kubelet.
This commit wires-in the pre-existing `--container-runtime` flag for
local node_e2e testing.
This is needed in order to further skip docker specific testing
and validation.
Local CRI node_e2e can now be performed via
`make test-e2e-node RUNTIME=remote REMOTE=false`
which will also take care of passing the appropriate arguments to
the kubelet.
Automatic merge from submit-queue
error strings should not end with punctuation
**What this PR does / why we need it**:
Delete the end punctuation of error strings
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
https://github.com/golang/go/wiki/CodeReviewComments#error-strings
**Release note**:
```release-note
```
Signed-off-by: yupeng <yu.peng36@zte.com.cn>
Automatic merge from submit-queue (batch tested with PRs 40529, 40630)
propagate explicit nulls in apply
Rebase of https://github.com/kubernetes/kubernetes/pull/35496 on top of https://github.com/kubernetes/kubernetes/pull/40260
The client-side propagation of the raw value is no longer needed, since the client is preserving the original object in unstructured form (explicit nulls are preserved).
Kept tests and CreateThreeWayMergePatch changes from https://github.com/kubernetes/kubernetes/pull/35496
```release-note
kubectl apply now supports explicitly clearing values not present in the config by setting them to null
```
- [x] Clean up orphaned objects in test-cmd to preserve pre- and post- conditions
- [x] improve CreateThreeWayMergePatch test to not filter based on string comparison to test name
Automatic merge from submit-queue
Add flag to node e2e test specifying location of ssh privkey
**What this PR does / why we need it**: in CI, the ssh private key is not always located at `$HOME/.ssh`, so it's helpful to be able to override it.
@krzyzacy here's my resurrected change. I'm not sure why I neglected to follow-through on it originally.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
genericapiserver: cut off more dependencies – episode 3
Compare commit subjects.
This is the first step to get `apiserver_test.go` and `watch_test.go` in `pkg/genericapiserver/endpoints` cutoff from k8s.io/kubernetes dependencies.
After this we have to sync client-go and then "episode 4" can go in.
approved based on #40363
Automatic merge from submit-queue (batch tested with PRs 39832, 40660)
Complete *-munge-docs.sh deprecation.
**What this PR does / why we need it**:\
Complete *-munge-docs.sh deprecation.
TOC generation now handled by https://github.com/kubernetes/release/pull/247
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
ref #38309
**Special notes for your reviewer**:
cc @bgrant0607 @thockin
Automatic merge from submit-queue
re-enable cascading deployment
I couldn't reproduce #40433 locally sooo trying on the test infra to make sure I'm not crazy
Automatic merge from submit-queue
Improve the multiarch situation; armel => armhf; reenable pcc64le; remove the patched golang
**What this PR does / why we need it**:
- Improves the multiarch situation as described in #38067
- Tries to bump to go1.8 for arm (and later enable ppc64le)
- GOARM 6 => GOARM 7
- Remove the golang 1.7 patch
- armel => armhf
- Bump QEMU version to v2.7.0
**Release note**:
```release-note
Improve the ARM builds and make hyperkube on ARM working again by upgrading the Go version for ARM to go1.8beta2
```
@kubernetes/sig-testing-misc @jessfraz @ixdy @jbeda @david-mcmahon @pwittrock
Automatic merge from submit-queue
run unit tests with go, not just bazel
Follow up to https://github.com/kubernetes/kubernetes/pull/40594, effectively reverts #39105.
I think we should run our unit tests using the official golang infrastructure. I have no objection to also running them in bazel (or running a subset in bazel), but I don't think that bazel should be the primary unit test method.
@smarterclayton @liggitt @ixdy @spxtr
Automatic merge from submit-queue
Temporary disable federation kubectl tests for secrets to unblock merge queue
Fixes https://github.com/kubernetes/kubernetes/issues/40521
cc @kubernetes/sig-federation-misc @deads2k
Automatic merge from submit-queue (batch tested with PRs 39223, 40260, 40082, 40389)
make kubectl generic commands work with unstructured objects
part of making apply, edit, label, annotate, and patch work with third party resources
fixes#35149fixes#34413
prereq of:
https://github.com/kubernetes/kubernetes/issues/35496https://github.com/kubernetes/kubernetes/pull/40096
related to:
https://github.com/kubernetes/kubernetes/issues/39906https://github.com/kubernetes/kubernetes/issues/40119
kubectl is currently decoding any resource it doesn't have compiled-in to a ThirdPartyResourceData struct, which means it computes patches using that struct, and would try to send a ThirdPartyResourceData object to the API server when running `apply`
This PR removes the behavior that decodes unknown objects into ThirdPartyResourceData structs internally, and fixes up the following generic commands to work with unstructured objects
- [x] apply
- [x] decode into runtime.Unstructured objects
- [x] successfully use `--record` with unregistered objects
- [x] patch
- [x] decode into runtime.Unstructured objects
- [x] successfully use `--record` with unregistered objects
- [x] describe
- [x] decode into runtime.Unstructured objects
- [x] implement generic describer
- [x] fix other generic kubectl commands to work with unstructured objects
- [x] label
- [x] annotate
follow-ups for pre-existing issues:
- [ ] `explain` doesn't work with unregistered resources
- [ ] remove special casing of federation group in clientset lookups, etc
- [ ] `patch`
- [ ] doesn't honor output formats when persisting to server (`kubectl patch -f svc.json --type merge -p '{}' -o json` doesn't output json)
- [ ] --local throws exception (`kubectl patch -f svc.json --type merge -p '{}' --local`)
- [ ] `apply`
- [ ] fall back to generic JSON patch computation if no go struct is registered for the target GVK (e.g. https://github.com/kubernetes/kubernetes/pull/40096)
- [ ] ensure subkey deletion works in CreateThreeWayJSONMergePatch
- [ ] ensure type stomping works in CreateThreeWayJSONMergePatch
- [ ] lots of tests for generic json patch computation
- [ ] prevent generic apply patch computation among different versions
- [ ] reconcile treatment of nulls with https://github.com/kubernetes/kubernetes/pull/35496
- [ ] `edit`
- [ ] decode into runtime.Unstructured objects
- [ ] fall back to generic JSON patch computation if no go struct is registered for the target GVK
Automatic merge from submit-queue (batch tested with PRs 40126, 40565, 38777, 40564, 40572)
Print warning about kubelet not running on darwin via local-up-cluster.sh
**What this PR does / why we need it**: As of the time of opening this PR, near the release of v.1.5, using `./hack/local-up-cluster.sh` claims to succeed and run kubelet, but logs, processes, and attempts to hit the service demonstrate that kubelet is NOT running. The issue has been found to be the hard requirements of cadvisor forcing kubelet to force exit when it is found to be unsupported as in the case of DARWIN.
My goal here is to bring the issue to attention to others and also help other devs avoid the issue.
**Release note**:
```NONE```
kubelet.go and kubelet's server.go file has minimal logic that keeps kubelet from breaking just because osx has not cadvisor support.
Automatic merge from submit-queue (batch tested with PRs 40239, 40397, 40449, 40448, 40360)
move the discovery and dynamic clients
Moved the dynamic client, discovery client, testing/core, and testing/cache to `client-go`. Dependencies on api groups we don't have generated clients for have dropped out, so federation, kubeadm, and imagepolicy.
@caesarxuchao @sttts
approved based on https://github.com/kubernetes/kubernetes/issues/40363
Automatic merge from submit-queue (batch tested with PRs 40239, 40397, 40449, 40448, 40360)
CRI: use more gogoprotobuf plugins
Generate marshaler/unmarshaler code should help improve the performance.
This addresses #40098
Automatic merge from submit-queue
Able to quick create a HA cluster by kube-up.sh centos provider
Make `kube-up.sh` `centos provider` support quick create a HA cluster, as I said above [#39430](https://github.com/kubernetes/kubernetes/issues/39430), it's more flexible than `kops` or `kubeadm` for some people in a limited network region.
I'm new to k8s dev, so if this pull request need to change, please let me know.
```release-note
Added support for creating HA clusters for centos using kube-up.sh.
```
Automatic merge from submit-queue (batch tested with PRs 38905, 40421)
Revert "Add the ability to edit fields within a config map."
Fixes#40396
This reverts commit 31eca372c9.
#38445 broke edit for all resources other than configmaps:
```
$ kubectl edit serviceaccount default
Error from server (NotFound): configmaps "serviceaccount" not found
```
also, `edit` is a generic command, we can't add subcommands that claim the `configmap` space and mess with the things resourcebuilder accepts
Fix: cannot get default master advertise address correctly
Set default value of NUM_MASTERS and NUM_NODES by MASTERS and NODES themself
Code cleanup and documented
Using runtime reconfiguration for etcd cluster instead of etcd discovery
Add exceptions for verify-flags
Automatic merge from submit-queue (batch tested with PRs 40335, 40320, 40324, 39103, 40315)
Splitting master/node services into separate charm layers
**What this PR does / why we need it**:
This branch includes a roll-up series of commits from a fork of the
Kubernetes repository pre 1.5 release because we didn't make the code freeze.
This additional effort has been fully tested and has results submit into
the gubernator to enhance confidence in this code quality vs. the single
layer, posing as both master/node.
To reference the gubernator results, please see:
https://k8s-gubernator.appspot.com/builds/canonical-kubernetes-tests/logs/kubernetes-gce-e2e-node/
Apologies in advance for the large commit however, we did not want to
submit without having successful upstream automated testing results.
This commit includes:
- Support for CNI networking plugins
- Support for durable storage provided by Ceph
- Building from upstream templates (read: kubedns - no more template
drift!)
- An e2e charm-layer to make running validation tests much simpler/repeatable
- Changes to support the 1.5.x series of Kubernetes
**Special notes for your reviewer**:
Additional note: We will be targeting -all- future work against upstream
so large pull requests of this magnitude will not occur again.
**Release note**:
```release-note
- Splits Juju Charm layers into master/worker roles
- Adds support for 1.5.x series of Kubernetes
- Introduces a tactic for keeping templates in sync with upstream eliminating template drift
- Adds CNI support to the Juju Charms
- Adds durable storage support to the Juju Charms
- Introduces an e2e Charm layer for repeatable testing efforts and validation of clusters
```
Automatic merge from submit-queue (batch tested with PRs 39260, 40216, 40213, 40325, 40333)
bazel: save git version in kubernetes.tar.gz
**What this PR does / why we need it**: adds a workspace status command for bazel (inspired by #36128) and then uses the saved values to generate the `version` file in `kubernetes.tar.gz`. We need this for `get-kube.sh` to work properly.
**Special notes for your reviewer**: I had to change a few things from #36128 - see comments for explanation.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Make verify-godeps.sh easier to debug
This does 3 things:
1. env var to retain the /tmp/godep
2. env var to use a specified /tmp/godep
3. rework of preload-dep to support more non-github things, like go4.org
This branch includes a rollup series of commits from a fork of the
kubernetes repository pre 1.5 release because we didn't make the code freeze.
This additional effort has been fully tested and has results submit into
the gubernator to enhance confidence in this code quality vs. the single
layer, posing as both master/node.
To reference the gubernator results, please see:
https://k8s-gubernator.appspot.com/builds/canonical-kubernetes-tests/logs/kubernetes-gce-e2e-node/
Apologies in advance for the large commit, however we did not want to
submit without having successful upstream automated testing results.
This commit includes:
- Support for CNI networking plugins
- Support for durable storage provided by ceph
- Building from upstream templates (read: kubedns - no more template
drift!)
- An e2e charm-layer to make running validation tests much simpler/repeatable
- Changes to support the 1.5.x series of kubernetes
Additional note: We will be targeting -all- future work against upstream
so large pull requests of this magnitude will not occur again.
Automatic merge from submit-queue
Optional configmaps and secrets
Allow configmaps and secrets for environment variables and volume sources to be optional
Implements approved proposal c9f881b7bb
Release note:
```release-note
Volumes and environment variables populated from ConfigMap and Secret objects can now tolerate the named source object or specific keys being missing, by adding `optional: true` to the volume or environment variable source specifications.
```
Automatic merge from submit-queue (batch tested with PRs 40328, 40057)
Allows kubectl create -f to filter by selector.
From #32544.
The underlying `seletor` infrastructrue was implemented by #32599. The test is basically copy-paste of the corresponding test for `kubectl apply -f -l`.
```release-note
kubectl create now accepts the label selector flag for filtering objects to create
```
Enforce the following limits:
12kb for total message length in container status
4kb for the termination message path file
2kb or 80 lines (whichever is shorter) from the log on error
Fallback to log output if the user requests it.
Automatic merge from submit-queue (batch tested with PRs 38445, 40292)
Add the ability to edit fields within a config map.
Addresses part of https://github.com/kubernetes/kubernetes/issues/36222
Example command:
```console
$ kubectl edit configmap foo --config-map-data=bar
```
Will open the data element named `bar` in the `ConfigMap` named `foo` in `$EDITOR`, the edited contents are then updated back to the config map.
@kubernetes/sig-cli
```release-note
Add a special purpose tool for editing individual fields in a ConfigMap with kubectl
```
Automatic merge from submit-queue
[Federation] Expose autoscaling apis through federation api server
This PR implements first part of federated pod autoscaler.
The issue to handle the whole feature is https://github.com/kubernetes/kubernetes/issues/38974
cc @kubernetes/sig-cluster-federation
@shashidharatd @kshafiee @deepak-vij
**Release note**:
```
federation users can now use federated autoscaling resources and create federated horizontalpodautoscalers
```
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
kubeadm: add optional self-hosted deployment
**What this PR does / why we need it**: add an optional self-hosted deployment type to `kubeadm`, for master components only, namely `apiserver`, `controller-manager` and `scheduler`.
**Which issue this PR fixes**: closes#38407
**Special notes for your reviewer**: /cc @aaronlevy @luxas @dgoodwin
**Release note**:
```release-note
kubeadm: add optional self-hosted deployment for apiserver, controller-manager and scheduler.
```
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
kubelet: storage: teardown terminated pod volumes
This is a continuation of the work done in https://github.com/kubernetes/kubernetes/pull/36779
There really is no reason to keep volumes for terminated pods attached on the node. This PR extends the removal of volumes on the node from memory-backed (the current policy) to all volumes.
@pmorie raised a concern an impact debugging volume related issues if terminated pod volumes are removed. To address this issue, the PR adds a `--keep-terminated-pod-volumes` flag the kubelet and sets it for `hack/local-up-cluster.sh`.
For consideration in 1.6.
Fixes#35406
@derekwaynecarr @vishh @dashpole
```release-note
kubelet tears down pod volumes on pod termination rather than pod deletion
```
Automatic merge from submit-queue (batch tested with PRs 40168, 40165, 39158, 39966, 40190)
CRI: upgrade protobuf to v3
For #38854, this PR upgrades CRI protobuf version to v3, and also updated related packages for confirming to new api.
**Release note**:
```
CRI: upgrade protobuf version to v3.
```
Automatic merge from submit-queue (batch tested with PRs 39772, 39831, 39481, 40167, 40149)
Add //hack:verify-boilerplate rule.
This pattern is working well in test-infra. I'll add the gofmt and go vet rules next.
Automatic merge from submit-queue
promote certificates api to beta
Mostly posting to see what breaks but also this API is ready to be promoted.
```release-note
Promote certificates.k8s.io to beta and enable it by default. Users using the alpha certificates API should delete v1alpha1 CSRs from the API before upgrading and recreate them as v1beta1 CSR after upgrading.
```
@kubernetes/api-approvers @jcbsmpsn @pipejakob
Automatic merge from submit-queue
Add authorization mode to kubeadm
This PR adds an option in `kubeadm` to allow a user to specify an [authorization plugin](https://kubernetes.io/docs/admin/authorization/). It defaults to RBAC.
Automatic merge from submit-queue
Build release tars using bazel
**What this PR does / why we need it**: builds equivalents of the various kubernetes release tarballs, solely using bazel.
For example, you can now do
```console
$ make bazel-release
$ hack/e2e.go -v -up -test -down
```
**Special notes for your reviewer**: this is currently dependent on 3b29803eb5, which I have yet to turn into a pull request, since I'm still trying to figure out if this is the best approach.
Basically, the issue comes up with the way we generate the various server docker image tarfiles and load them on nodes:
* we `md5sum` the binary being encapsulated (e.g. kube-proxy) and save that to `$binary.docker_tag` in the server tarball
* we then build the docker image and tag using that md5sum (e.g. `gcr.io/google_containers/kube-proxy:$MD5SUM`)
* we `docker save` this image, which embeds the full tag in the `$binary.tar` file.
* on cluster startup, we `docker load` these tarballs, which are loaded with the tag that we'd created at build time. the nodes then use the `$binary.docker_tag` file to find the right image.
With the current bazel `docker_build` rule, the tag isn't saved in the docker image tar, so the node is unable to find the image after `docker load`ing it.
My changes to the rule save the tag in the docker image tar, though I don't know if there are subtle issues with it. (Maybe we want to only tag when `--stamp` is given?)
Also, the docker images produced by bazel have the timestamp set to the unix epoch, which is not great for debugging. Might be another thing to change with a `--stamp`.
Long story short, we probably need to follow up with bazel folks on the best way to solve this problem.
**Release note**:
```release-note
NONE
```
Kubernetes Submit Queue