Automatic merge from submit-queue (batch tested with PRs 41360, 41423, 41430, 40647, 41352)
move kubeadm api group testing to kubeadm package
I think this is sufficient to at least preserve round trip testing.
Automatic merge from submit-queue (batch tested with PRs 40297, 41285, 41211, 41243, 39735)
cluster/gce: Add env var to enable apiserver basic audit log.
For now, this is focused on a fixed set of flags that makes the audit
log show up under /var/log/kube-apiserver-audit.log and behave similarly
to /var/log/kube-apiserver.log. Allowing other customization would
require significantly more complex changes.
Audit log rotation is handled the same as for `kube-apiserver.log`.
**What this PR does / why we need it**:
Add a knob to enable [basic audit logging](https://kubernetes.io/docs/admin/audit/) in GCE.
**Which issue this PR fixes**:
**Special notes for your reviewer**:
We would like to cherrypick/port this to release-1.5 also.
**Release note**:
```release-note
The kube-apiserver [basic audit log](https://kubernetes.io/docs/admin/audit/) can be enabled in GCE by exporting the environment variable `ENABLE_APISERVER_BASIC_AUDIT=true` before running `cluster/kube-up.sh`. This will log to `/var/log/kube-apiserver-audit.log` and use the same `logrotate` settings as `/var/log/kube-apiserver.log`.
```
Automatic merge from submit-queue (batch tested with PRs 41378, 41413, 40743, 41155, 41385)
move kube-aggregator to staging
Straight move of kube-aggregator to staging. The scripts are the interesting bits.
@sttts @lavalamp
This change makes kubelet to use the CRI implementation by default,
unless the users opt out explicitly by using --enable-cri=false.
For the rkt integration, the --enable-cri flag will have no effect
since rktnetes does not use CRI.
Also, mark the original --experimental-cri flag hidden and deprecated,
so that we can remove it in the next release.
For now, this is focused on a fixed set of flags that makes the audit
log show up under /var/log/kube-apiserver-audit.log and behave similarly
to /var/log/kube-apiserver.log. Allowing other customization would
require significantly more complex changes.
Audit log rotation is handled externally by the wildcard /var/log/*.log
already configured in configure-helper.sh.
Automatic merge from submit-queue (batch tested with PRs 41357, 41178, 41280, 41184, 41278)
Add verify-gofmt as a Bazel test.
Also, moved `--flaky_test_attempts=3` out of `.bazelrc` since it was annoying. I think this verify-all pattern is working well in test-infra, we should try and make it happen here.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
copy pkg/util/logs to apiserver
This is a copy, not a move. API servers need to be able to init the logs, but so do clients. It would be weird to have the client-side commands depending on the server side logs utilities.
I updated all the server side references, but left the client-side ones.
@sttts @kubernetes/sig-api-machinery-pr-reviews acceptable?
Automatic merge from submit-queue
Added kubectl create role command
Added `kubectl create role` command.
Fixed part of #39596
**Release note**:
```
Added one new command `kubectl create role` to help user create a single role from command line.
```
Automatic merge from submit-queue
Add pod manifest path to local cluster
Added `POD_MANIFEST_PATH` to local cluster up because we are frequently using this flag when testing static & mirror pod during local dev.
Automatic merge from submit-queue (batch tested with PRs 41259, 41260)
remove hardcoded ansi color coding for `make help`
to tput so that color coding works in multiple platforms including osx.
**What this PR does / why we need it**:
should try not to use hardcoded ansi escape characters.
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 38252, 41122, 36101, 41017, 41264)
BootstrapSigner and TokenCleaner controllers
This is part of https://github.com/kubernetes/features/issues/130 and is an implementation of https://github.com/kubernetes/community/pull/189.
Work that needs to be done yet in this PR:
* [ ] ~~e2e tests~~ Will come in new PR.
* [x] flag to disable this by default
```release-note
Native support for token based bootstrap flow. This includes signing a well known ConfigMap in the `kube-public` namespace and cleaning out expired tokens.
```
@kubernetes/sig-cluster-lifecycle @dgoodwin @roberthbailey @mikedanese
Automatic merge from submit-queue (batch tested with PRs 39418, 41175, 40355, 41114, 32325)
TaintController
```release-note
This PR adds a manager to NodeController that is responsible for removing Pods from Nodes tainted with NoExecute Taints. This feature is beta (as the rest of taints) and enabled by default. It's gated by controller-manager enable-taint-manager flag.
```
Automatic merge from submit-queue (batch tested with PRs 41112, 41201, 41058, 40650, 40926)
Promote TokenReview to v1
Peer to https://github.com/kubernetes/kubernetes/pull/40709
We have multiple features that depend on this API:
- [webhook authentication](https://kubernetes.io/docs/admin/authentication/#webhook-token-authentication)
- [kubelet delegated authentication](https://kubernetes.io/docs/admin/kubelet-authentication-authorization/#kubelet-authentication)
- add-on API server delegated authentication
The API has been in use since 1.3 in beta status (v1beta1) with negligible changes:
- Added a status field for reporting errors evaluating the token
This PR promotes the existing v1beta1 API to v1 with no changes
Because the API does not persist data (it is a query/response-style API), there are no data migration concerns.
This positions us to promote the features that depend on this API to stable in 1.7
cc @kubernetes/sig-auth-api-reviews @kubernetes/sig-auth-misc
```release-note
The authentication.k8s.io API group was promoted to v1
```
Automatic merge from submit-queue (batch tested with PRs 40796, 40878, 36033, 40838, 41210)
HPA v2 (API Changes)
**Release note**:
```release-note
Introduces an new alpha version of the Horizontal Pod Autoscaler including expanded support for specifying metrics.
```
Implements the API changes for kubernetes/features#117.
This implements #34754, which is the new design for the Horizontal Pod Autoscaler. It includes improved support for custom metrics (and/or arbitrary metrics) as well as expanded support for resource metrics. The new HPA object is introduces in the API group "autoscaling/v1alpha1".
Note that the improved custom metric support currently is limited to per pod metrics from Heapster -- attempting to use the new "object metrics" will simply result in an error. This will change once #34586 is merged and implemented.
Automatic merge from submit-queue (batch tested with PRs 40917, 41181, 41123, 36592, 41183)
fix scheduler performance test script
**What this PR does / why we need it**:
`test-performance.sh` is in dir `kubernetes/test/integration/scheduler_perf`
the dir `kubernetes/test/component/scheduler/perf` does not exist
Thanks.
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 40917, 41181, 41123, 36592, 41183)
[Federation] Add override flags options to kubefed init
**What this PR does / why we need it**:
Allows modification of startup flags (of apiserver and controller manager) through kubefed
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
https://github.com/kubernetes/kubernetes/issues/40398
**Special notes for your reviewer**:
I haven't removed the existing redundant flags now (for example --dns-zone-name) intentionally to avoid breaking any existing tests that might use them.
I guess that would be better done as a follow up PR.
@madhusudancs @marun @nikhiljindal
**Release note**:
```
It is now possible for the user to modify any startup flag of federation-apiserver and federation-controller-manager when deployed through kubefed.
There are two new options introduced in kubefed:
--apiserver-arg-overrides and --controllermanager-arg-overrides
Any number of actual federation-apiserver or federation-controller-manager flags can be specified using these options.
Example:
kubefed init "-other options-" ----apiserver-arg-overrides "--flag1=value1,--flag2=value2"
```
Automatic merge from submit-queue (batch tested with PRs 41074, 41147, 40854, 41167, 40045)
Add debug logging to eviction manager
**What this PR does / why we need it**:
This PR adds debug logging to eviction manager.
We need it to help users understand when/why eviction manager is/is not making decisions to support information gathering during support.
Change the name of the client cert used by the controller manager from
system:controller to system:kube-controller-manager, so the appropriate
RBAC rules are applied. Also remove the system:masters group from this
client cert so it doesn't have super powers.
Also, always enable using service account credentials in the controller
manager.
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
add k8s.io/sample-apiserver to demonstrate how to build an aggregated API server
builds on https://github.com/kubernetes/kubernetes/pull/41093
This creates a sample API server is a separate staging repo to guarantee no cheating with `k8s.io/kubernetes` dependencies. The sample is run during integration tests (simple tests on it so far) to ensure that it continues to run.
@sttts @kubernetes/sig-api-machinery-misc ptal
@pwittrock @pmorie @kris-nova an aggregated API server example that will stay up to date.
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
Match etcd process name exactly
A process named, e.g., etcd-operator, should not match
**What this PR does / why we need it**: Matches etcd process name exactly
**Which issue this PR fixes**: fixes#40499
**Special notes for your reviewer**: 👀 @lavalamp
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
Remove deprecated kubelet flags that look safe to remove
Removes:
```
--config
--auth-path
--resource-container
--system-container
```
which have all been marked deprecated since at least 1.4 and look safe to remove.
```release-note
The deprecated flags --config, --auth-path, --resource-container, and --system-container were removed.
```
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
[Federation] Wait after cleanup only if the cleanup succeeds.
It is a waste of time to wait for the resources to cleanup if the
cleanup fails.
cc @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 40873, 40948, 39580, 41065, 40815)
Default target storage in etcd
To make etcd v2->v3 upgrade work correctly, we need to correctly set the "TARGET_STORAGE" env var. Since in head we are defaulting to etcd v3, this PR is defaulting also that env var to etcd3, so that by default upgrade works fine.
@fgrzadkowski @gmarek
Automatic merge from submit-queue (batch tested with PRs 40175, 41107, 41111, 40893, 40919)
remove second CA used for kubelet auth in favor of webhook auth
partial fixes upgrade test.
Automatic merge from submit-queue (batch tested with PRs 40175, 41107, 41111, 40893, 40919)
kubeadm: skip integration tests if kubeadm-cmd-skip flag passed
Will skip integration tests for token generation if it can't find a file by the given --kubeadm-path or default value.
**What this PR does / why we need it**: Tests would fail if just running `go test` in the dir because it expects to have more values. This won't change the behavior of `make test-cmd` which gets run here:
https://github.com/kubernetes/kubernetes/blob/master/Makefile#L258
**Which issue this PR fixes**: fixes#40155
**Special notes for your reviewer**: /cc @pires @pipejakob @liggitt
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41061, 40888, 40664, 41020, 41085)
Cleanup federation components and wait for it only once in a test cycle.
cc @kubernetes/sig-federation-pr-reviews
After today's SIG meeting, it was discussed how to proceed with these
types of test-cmd tests. They will live in kubeamd/test/cmd and will
provide a flag that will allow you to skip them (--kubeadm-cmd-skip) and
by default will fail if kubeadm binary is not present
Automatic merge from submit-queue (batch tested with PRs 40345, 38183, 40236, 40861, 40900)
remove the create-external-load-balancer flag in cmd/expose.go
**What this PR does / why we need it**:
In cmd/expose.go there is a todo "remove create-external-load-balancer in code on or after Aug 25, 2016.", and now it's been a long time past. So I remove this flag and modify the test cases.
Please check for this, thanks!
**Release note**:
```
remove the deprecated flag "create-external-load-balancer" and use --type="LoadBalancer" instead.
```
Automatic merge from submit-queue (batch tested with PRs 41023, 41031, 40947)
scrub aggregator names to eliminate discovery
Cleanup old uses of `discovery`. Also removes the legacy functionality.
@kubernetes/sig-api-machinery-misc @sttts
Automatic merge from submit-queue (batch tested with PRs 40980, 40985)
added short names for resources which are exposed during discovery
**What this PR does / why we need it**:
The changes add short names for resources. The short names will be delivered to kubectl during discovery.
Automatic merge from submit-queue (batch tested with PRs 40971, 41027, 40709, 40903, 39369)
Promote SubjectAccessReview to v1
We have multiple features that depend on this API:
SubjectAccessReview
- [webhook authorization](https://kubernetes.io/docs/admin/authorization/#webhook-mode)
- [kubelet delegated authorization](https://kubernetes.io/docs/admin/kubelet-authentication-authorization/#kubelet-authorization)
- add-on API server delegated authorization
The API has been in use since 1.3 in beta status (v1beta1) with negligible changes:
- Added a status field for reporting errors evaluating access
- A typo was discovered in the SubjectAccessReviewSpec Groups field name
This PR promotes the existing v1beta1 API to v1, with the only change being the typo fix to the groups field. (fixes https://github.com/kubernetes/kubernetes/issues/32709)
Because the API does not persist data (it is a query/response-style API), there are no data migration concerns.
This positions us to promote the features that depend on this API to stable in 1.7
cc @kubernetes/sig-auth-api-reviews @kubernetes/sig-auth-misc
```release-note
The authorization.k8s.io API group was promoted to v1
```
Automatic merge from submit-queue
local-up-cluster: fix instructions for set-credentials
After launching the services, local-up-cluster.sh tells the
user how to configure kubectl to access it. The instructions
for 'set-credentials' enable plain password auth, but the
services are configured to mandate client certificate auth.
As a result it is not possible to access the cluster with
the instructions printed.
The use of client certs by default was added in
commit a1b17db458
Author: Dr. Stefan Schimanski <sttts@redhat.com>
Date: Sat Nov 12 23:09:04 2016 +0100
Configure client certs in local-cluster-up.sh
and the instructions were correctly updated to refer to
client certificates.
The changed instructions were (mistakenly) reverted though
when the following commit was merged:
commit 72e0e91b5e
Author: xilabao <chenr.fnst@cn.fujitsu.com>
Date: Fri Dec 2 11:04:25 2016 +0800
change prompt for enabling RBAC on local-up-cluster
Fixes: #40192
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
After launching the services, local-up-cluster.sh tells the
user how to configure kubectl to access it. The instructions
for 'set-credentials' enable plain password auth, but the
services are configured to mandate client certificate auth.
As a result it is not possible to access the cluster with
the instructions printed.
The use of client certs by default was added in
commit a1b17db458
Author: Dr. Stefan Schimanski <sttts@redhat.com>
Date: Sat Nov 12 23:09:04 2016 +0100
Configure client certs in local-cluster-up.sh
and the instructions were correctly updated to refer to
client certificates.
The changed instructions were (mistakenly) reverted though
when the following commit was merged:
commit 72e0e91b5e
Author: xilabao <chenr.fnst@cn.fujitsu.com>
Date: Fri Dec 2 11:04:25 2016 +0800
change prompt for enabling RBAC on local-up-cluster
Fixes: #40192
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Automatic merge from submit-queue
Using API_HOST_IP to do apiserver health check.
In `hack/local-up-cluster.sh`, it's better to use `API_HOST_IP` to do apiserver health check.
Automatic merge from submit-queue
[Federation][kubefed] Add option to expose federation apiserver on nodeport service
**What this PR does / why we need it**:
This PR adds an option to kubefed to expose federation api server over nodeport. This can be useful to deploy federation in non-cloud environments. This PR is target to address #39271
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
[Federation] kubefed init learned a new flag, `--api-server-service-type`, that allows service type to be specified for the federation API server.
[Federation] kubefed init also learned a new flag, `--api-server-advertise-address`, that allows specifying advertise address for federation API server in case the service type is NodePort.
```
@kubernetes/sig-federation-misc @madhusudancs
Automatic merge from submit-queue
Add SIG to test owners
**What this PR does / why we need it**:
This PR adds a `sig` column to the test owners file generation script.
A problem experienced with the current owners file is that since members are auto-assigned there are times where tests are assigned to non-active users who don't follow up to notifications to fix flakes. By assigning a SIG to each test we can hold a group we know is active responsible for taking care of flakes it's less likely that flakes will fall through the cracks.
**Special notes for your reviewer**:
* A companion PR will go into *kubernetes/contrib* adding support for mungers parsing this new column.
* Another PR in contrib will add labeling GitHub flake issues with the appropriate SIG
* Currently SIGs are not labeled, this will be added in another PR where SIG determinations can be discussed
@saad-ali @pwittrock
Automatic merge from submit-queue (batch tested with PRs 40289, 40877, 40879, 39972, 40942)
Rename experimental-cgroups-per-pod flag
**What this PR does / why we need it**:
1. Rename `experimental-cgroups-per-qos` to `cgroups-per-qos`
1. Update hack/local-up-cluster to match `CGROUP_DRIVER` with docker runtime if used.
**Special notes for your reviewer**:
We plan to roll this feature out in the upcoming release. Previous node e2e runs were running with this feature on by default. We will default this feature on for all e2es next week.
**Release note**:
```release-note
Rename --experiemental-cgroups-per-qos to --cgroups-per-qos
```
Automatic merge from submit-queue (batch tested with PRs 40906, 40924, 40938, 40902, 40911)
print apiserver log location on apiserver error
**What this PR does / why we need it**:
Improve user experience. Attempt to direct user to logs of failing component.
**Special notes for your reviewer**:
In addition to failure, point to logs so that a user can attempt to self remedy and have more information available to debug immediately. A user may not know that the failing component has logs.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40696, 39914, 40374)
Convert hack/e2e.go to a test-infra/kubetest shim
Replaces `hack/e2e.go` for a shim that passes the args to `k8s.io/test-infra/kubetest`
Adds fejta to `hack/OWNERS`
Adds `e2e_test.go` for unit test coverage of the shim.
`Usage: go run hack/e2e.go [--get=true] [--old=1d] -- KUBETEST_ARGS`
In other words there is are `--get` and `--old` shim flags, which control how we upgrade `kubetest`, and a `--` to separate the shim args from the kubetest args, and the existing kubetest args like `--down` `--up`, etc. If only `KUBETEST_ARGS` are used then you can skip the `--` (although golang will complain about it).
Once this is ready to go I will update the kubekins-e2e image to copy this file from test-infra: https://github.com/kubernetes/test-infra/blob/master/jenkins/e2e-image/Dockerfile#L70
ref https://github.com/kubernetes/test-infra/issues/1475
Automatic merge from submit-queue (batch tested with PRs 40862, 40909)
[Federation][kubefed] Add option to disable persistence storage for etcd
**What this PR does / why we need it**:
This is part of updates to enable deployment of federation on non-cloud environments. This pr enables disabling persistent storage for etcd via kubefed.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#40617
**Special notes for your reviewer**:
**Release note**:
```
[Federation] Add --etcd-persistent-storage flag to kubefed to enable/disable persistent storage for etcd
```
cc: @kubernetes/sig-federation-bugs @madhusudancs
Automatic merge from submit-queue (batch tested with PRs 40864, 40666, 38382, 40874)
apply falls back to generic JSON patch computation if no go struct is registered for the target GVK
This PR is the master version of #40096 which is target 1.4 branch.
This PR is based on #40260
- [x] ensure subkey deletion works in CreateThreeWayJSONMergePatch
- [x] ensure type stomping works in CreateThreeWayJSONMergePatch
- [x] lots of tests for generic json patch computation
- [x] apply falls back to generic 3-way JSON merge patch if no go struct is registered for the target GVK
- [x] prevent generic apply patch computation between different apiVersions and/or kinds
- [x] make pruner generic (apply --prune works with TPR)
```release-note
apply falls back to generic 3-way JSON merge patch if no go struct is registered for the target GVK
```
Automatic merge from submit-queue (batch tested with PRs 40884, 40809, 40845, 40866, 40875)
update repo local config to allow redirects from gopkg.in
closes#39583
Automatic merge from submit-queue (batch tested with PRs 40812, 39903, 40525, 40729)
Add CI coverage for client-go and staging/copy.sh
Client-go master and `staging/copy.sh` keep breaking, leading to frustration of users outside of the inner refactoring circle. This PR adds minimal smoke testing for both.
**Note**: this enforces the split of PRs which change apimachinery and client-go in parallel (via some local "temporary commit" hacking).
- [x] @caesarxuchao ~~do we have to delete the example directory from the upstream repo?~~ merge https://github.com/kubernetes/test-infra/issues/1765
Automatic merge from submit-queue (batch tested with PRs 40812, 39903, 40525, 40729)
test/node_e2e: wire-in cri-enabled local testing
This commit wires-in the pre-existing `--container-runtime` flag for
local node_e2e testing.
This is needed in order to further skip docker specific testing
and validation.
Local CRI node_e2e can now be performed via
`make test-e2e-node RUNTIME=remote REMOTE=false`
which will also take care of passing the appropriate argument to
the kubelet.
deads2k