Automatic merge from submit-queue (batch tested with PRs 42128, 42064, 42253, 42309, 42322)
kubeadm: Rename some flags for beta UI and fixup some logic
**What this PR does / why we need it**:
In this PR:
- `--api-advertise-addresses` becomes `--apiserver-advertise-address`
- The API Server's logic here is that if the address is `0.0.0.0`, it chooses the host's default interface's address. kubeadm here uses exactly the same logic. This arg is then passed to `--advertise-address`, and the API Server will advertise that one for the service VIP.
- `--api-port` becomes `--apiserver-bind-port` for clarity
ref the meeting notes: https://docs.google.com/document/d/1deJYPIF4LmhGjDVaqrswErIrV7mtwJgovtLnPCDxP7U/edit#
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
@jbeda @dmmcquay @pires @lukemarsden @dgoodwin @mikedanese
This commit switches over the HPA controller to use the custom metrics
API. It also converts the HPA controller to use the generated client
in k8s.io/metrics for the resource metrics API.
In order to enable support, you must enable
`--horizontal-pod-autoscaler-use-rest-clients` on the
controller-manager, which will switch the HPA controller's MetricsClient
implementation over to use the standard rest clients for both custom
metrics and resource metrics. This requires that at the least resource
metrics API is registered with kube-aggregator, and that the controller
manager is pointed at kube-aggregator. For this to work, Heapster
must be serving the new-style API server (`--api-server=true`).
Automatic merge from submit-queue
Extend experimental support to multiple Nvidia GPUs
Extended from #28216
```release-note
`--experimental-nvidia-gpus` flag is **replaced** by `Accelerators` alpha feature gate along with support for multiple Nvidia GPUs.
To use GPUs, pass `Accelerators=true` as part of `--feature-gates` flag.
Works only with Docker runtime.
```
1. Automated testing for this PR is not possible since creation of clusters with GPUs isn't supported yet in GCP.
1. To test this PR locally, use the node e2e.
```shell
TEST_ARGS='--feature-gates=DynamicKubeletConfig=true' FOCUS=GPU SKIP="" make test-e2e-node
```
TODO:
- [x] Run manual tests
- [x] Add node e2e
- [x] Add unit tests for GPU manager (< 100% coverage)
- [ ] Add unit tests in kubelet package
Automatic merge from submit-queue (batch tested with PRs 41921, 41695, 42139, 42090, 41949)
kubeadm: join ux changes
**What this PR does / why we need it**: Update `kubeadm join` UX according to https://github.com/kubernetes/community/pull/381
**Which issue this PR fixes**: fixes # https://github.com/kubernetes/kubeadm/issues/176
**Special notes for your reviewer**: /cc @luxas @jbeda
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42058, 41160, 42065, 42076, 39338)
New command for stand-alone GKE certificates controller
New stand-alone certificates controller for GKE. Rather than requiring the CA's private key on disk, this allows making external calls to GKE in order to sign cluster certificates.
**Which issue this PR fixes**: fixes#39761
**Release note**:
```release-note
New GKE certificates controller.
```
CC @mikedanese @jcbsmpsn
Automatic merge from submit-queue (batch tested with PRs 42044, 41694, 41927, 42050, 41987)
Add apply set-last-applied subcommand
implement part of https://github.com/kubernetes/community/pull/287, will rebase after https://github.com/kubernetes/kubernetes/pull/41699 got merged, EDIT: since bug output format has been confirmed, will update the behavior of output format soon
cc @kubernetes/sig-cli-pr-reviews @AdoHe @pwittrock
```release-note
Support kubectl apply set-last-applied command to update the applied-applied-configuration annotation
```
Automatic merge from submit-queue (batch tested with PRs 41701, 41818, 41897, 41119, 41562)
kubeadm: Secure the control plane communication and add the kubeconfig phase command
**What this PR does / why we need it**:
This generates kubeconfig files for the controller-manager and the scheduler, ref: https://github.com/kubernetes/kubeadm/issues/172
The second commit adds the `kubeadm alpha phase kubeconfig` command as described in the design doc: https://github.com/kubernetes/kubeadm/pull/156
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
@dmmcquay What kind of tests would you like for the kubeconfig phase command?
**Release note**:
```release-note
```
@jbeda @mikedanese @dmmcquay @pires @liggitt @deads2k @errordeveloper
Automatic merge from submit-queue (batch tested with PRs 41814, 41922, 41957, 41406, 41077)
add kubectl can-i to see if you can perform an action
Adds `kubectl auth can-i <verb> <resource> [<name>]` so that a user can see if they are allowed to perform an action.
@kubernetes/sig-cli-pr-reviews @fabianofranz
This particular command satisfies the immediate need of knowing if you can perform an action without trying that action. When using RBAC in a script that is adding permissions, there is a lag between adding the permission and the permission being realized in the RBAC cache. As a user on the CLI, you almost never see it, but as a script adding a binding and then using that new power, you hit it quite often.
There are natural follow-ons to the same area (hence the `auth` subcommand) to figure out if someone else can perform an action, what actions you can perform in total, and who can perform a given action. Someone else is an API we have already, what-can-i-do was a proposed API a while back and a very useful one for interfaces, and who-can is common question if someone is administering a namespace.
Automatic merge from submit-queue (batch tested with PRs 40665, 41094, 41351, 41721, 41843)
kubeadm: Add a --ca-cert-path flag to kubeadm join
**What this PR does / why we need it**:
This PR makes it possible to customize where the CA file is written
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
@pires @mikedanese @dmmcquay @jbeda @errordeveloper
This adds a new stand-alone certificates controller for use on GKE. It
allows calling GKE to sign certificates instead of requiring the CA
private key locally.
It does not aim for 100% feature parity with kube-controller-manager
yet, so for instance, leader election support is omitted.
Automatic merge from submit-queue (batch tested with PRs 41812, 41665, 40007, 41281, 41771)
kube-apiserver: add a bootstrap token authenticator for TLS bootstrapping
Follows up on https://github.com/kubernetes/kubernetes/pull/36101
Still needs:
* More tests.
* To be hooked up to the API server.
- Do I have to do that in a separate PR after k8s.io/apiserver is synced?
* Docs (kubernetes.io PR).
* Figure out caching strategy.
* Release notes.
cc @kubernetes/sig-auth-api-reviews @liggitt @luxas @jbeda
```release-notes
Added a new secret type "bootstrap.kubernetes.io/token" for dynamically creating TLS bootstrapping bearer tokens.
```
Automatic merge from submit-queue
Added a basic monitor for providing etcd version related info
Fixes#41071
This tool scrapes metrics partly from etcd's /version and /metrics endpoints and partly using etcdctl and exposes them as prometheus metrics at `http://localhost:9101/metrics` endpoint on the master. Here is a summary of the metrics it exposes (self-explanatory from the code):
- etcdVersionFetchCount = prometheus.NewCounterVec(
prometheus.CounterOpts{
Namespace: "etcd",
Name: "version_info_fetch_count",
Help: "Number of times etcd's version info was fetched, labeled by etcd's server binary and cluster version",
},
[]string{"serverversion", "clusterversion"})
- etcdGRPCRequestsTotal = prometheus.NewCounterVec(
prometheus.CounterOpts{
Namespace: namespace,
Name: "grpc_requests_total",
Help: "Counter of received grpc requests, labeled by grpc method and grpc service names",
},
[]string{"grpc_method", "grpc_service"})
For further info on how to run this as a binary/docker-container/kubernetes-pod and checking the metrics, have a look at the README.md file.
cc @fgrzadkowski @wojtek-t @piosz
This change makes kubelet to use the CRI implementation by default,
unless the users opt out explicitly by using --enable-cri=false.
For the rkt integration, the --enable-cri flag will have no effect
since rktnetes does not use CRI.
Also, mark the original --experimental-cri flag hidden and deprecated,
so that we can remove it in the next release.
Automatic merge from submit-queue
Added kubectl create role command
Added `kubectl create role` command.
Fixed part of #39596
**Release note**:
```
Added one new command `kubectl create role` to help user create a single role from command line.
```
Automatic merge from submit-queue (batch tested with PRs 39418, 41175, 40355, 41114, 32325)
TaintController
```release-note
This PR adds a manager to NodeController that is responsible for removing Pods from Nodes tainted with NoExecute Taints. This feature is beta (as the rest of taints) and enabled by default. It's gated by controller-manager enable-taint-manager flag.
```
Automatic merge from submit-queue (batch tested with PRs 40917, 41181, 41123, 36592, 41183)
[Federation] Add override flags options to kubefed init
**What this PR does / why we need it**:
Allows modification of startup flags (of apiserver and controller manager) through kubefed
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
https://github.com/kubernetes/kubernetes/issues/40398
**Special notes for your reviewer**:
I haven't removed the existing redundant flags now (for example --dns-zone-name) intentionally to avoid breaking any existing tests that might use them.
I guess that would be better done as a follow up PR.
@madhusudancs @marun @nikhiljindal
**Release note**:
```
It is now possible for the user to modify any startup flag of federation-apiserver and federation-controller-manager when deployed through kubefed.
There are two new options introduced in kubefed:
--apiserver-arg-overrides and --controllermanager-arg-overrides
Any number of actual federation-apiserver or federation-controller-manager flags can be specified using these options.
Example:
kubefed init "-other options-" ----apiserver-arg-overrides "--flag1=value1,--flag2=value2"
```
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
Remove deprecated kubelet flags that look safe to remove
Removes:
```
--config
--auth-path
--resource-container
--system-container
```
which have all been marked deprecated since at least 1.4 and look safe to remove.
```release-note
The deprecated flags --config, --auth-path, --resource-container, and --system-container were removed.
```
Automatic merge from submit-queue (batch tested with PRs 40175, 41107, 41111, 40893, 40919)
kubeadm: skip integration tests if kubeadm-cmd-skip flag passed
Will skip integration tests for token generation if it can't find a file by the given --kubeadm-path or default value.
**What this PR does / why we need it**: Tests would fail if just running `go test` in the dir because it expects to have more values. This won't change the behavior of `make test-cmd` which gets run here:
https://github.com/kubernetes/kubernetes/blob/master/Makefile#L258
**Which issue this PR fixes**: fixes#40155
**Special notes for your reviewer**: /cc @pires @pipejakob @liggitt
```release-note
NONE
```
After today's SIG meeting, it was discussed how to proceed with these
types of test-cmd tests. They will live in kubeamd/test/cmd and will
provide a flag that will allow you to skip them (--kubeadm-cmd-skip) and
by default will fail if kubeadm binary is not present
Automatic merge from submit-queue (batch tested with PRs 40345, 38183, 40236, 40861, 40900)
remove the create-external-load-balancer flag in cmd/expose.go
**What this PR does / why we need it**:
In cmd/expose.go there is a todo "remove create-external-load-balancer in code on or after Aug 25, 2016.", and now it's been a long time past. So I remove this flag and modify the test cases.
Please check for this, thanks!
**Release note**:
```
remove the deprecated flag "create-external-load-balancer" and use --type="LoadBalancer" instead.
```
Automatic merge from submit-queue
[Federation][kubefed] Add option to expose federation apiserver on nodeport service
**What this PR does / why we need it**:
This PR adds an option to kubefed to expose federation api server over nodeport. This can be useful to deploy federation in non-cloud environments. This PR is target to address #39271
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
[Federation] kubefed init learned a new flag, `--api-server-service-type`, that allows service type to be specified for the federation API server.
[Federation] kubefed init also learned a new flag, `--api-server-advertise-address`, that allows specifying advertise address for federation API server in case the service type is NodePort.
```
@kubernetes/sig-federation-misc @madhusudancs
Automatic merge from submit-queue (batch tested with PRs 40862, 40909)
[Federation][kubefed] Add option to disable persistence storage for etcd
**What this PR does / why we need it**:
This is part of updates to enable deployment of federation on non-cloud environments. This pr enables disabling persistent storage for etcd via kubefed.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#40617
**Special notes for your reviewer**:
**Release note**:
```
[Federation] Add --etcd-persistent-storage flag to kubefed to enable/disable persistent storage for etcd
```
cc: @kubernetes/sig-federation-bugs @madhusudancs
Automatic merge from submit-queue (batch tested with PRs 40812, 39903, 40525, 40729)
test/node_e2e: wire-in cri-enabled local testing
This commit wires-in the pre-existing `--container-runtime` flag for
local node_e2e testing.
This is needed in order to further skip docker specific testing
and validation.
Local CRI node_e2e can now be performed via
`make test-e2e-node RUNTIME=remote REMOTE=false`
which will also take care of passing the appropriate argument to
the kubelet.
This commit wires-in the pre-existing `--container-runtime` flag for
local node_e2e testing.
This is needed in order to further skip docker specific testing
and validation.
Local CRI node_e2e can now be performed via
`make test-e2e-node RUNTIME=remote REMOTE=false`
which will also take care of passing the appropriate arguments to
the kubelet.
Automatic merge from submit-queue
Add flag to node e2e test specifying location of ssh privkey
**What this PR does / why we need it**: in CI, the ssh private key is not always located at `$HOME/.ssh`, so it's helpful to be able to override it.
@krzyzacy here's my resurrected change. I'm not sure why I neglected to follow-through on it originally.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 38445, 40292)
Add the ability to edit fields within a config map.
Addresses part of https://github.com/kubernetes/kubernetes/issues/36222
Example command:
```console
$ kubectl edit configmap foo --config-map-data=bar
```
Will open the data element named `bar` in the `ConfigMap` named `foo` in `$EDITOR`, the edited contents are then updated back to the config map.
@kubernetes/sig-cli
```release-note
Add a special purpose tool for editing individual fields in a ConfigMap with kubectl
```
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
kubeadm: add optional self-hosted deployment
**What this PR does / why we need it**: add an optional self-hosted deployment type to `kubeadm`, for master components only, namely `apiserver`, `controller-manager` and `scheduler`.
**Which issue this PR fixes**: closes#38407
**Special notes for your reviewer**: /cc @aaronlevy @luxas @dgoodwin
**Release note**:
```release-note
kubeadm: add optional self-hosted deployment for apiserver, controller-manager and scheduler.
```
Automatic merge from submit-queue (batch tested with PRs 39486, 37288, 39477, 39455, 39542)
Allow missing keys in templates by default
Switch to allowing missing keys in jsonpath templates by default.
Add support for allowing/disallowing missing keys in go templates
(default=allow).
Add --allow-missing-template-keys flag to control this behavior (default=true /
allow missing keys).
Fixes#37991
@kubernetes/sig-cli-misc @kubernetes/api-reviewers @smarterclayton @fabianofranz @liggitt @pwittrock
Switch to allowing missing keys in jsonpath templates by default.
Add support for allowing/disallowing missing keys in go templates
(default=allow).
Add --allow-missing-template-keys flag to control this behavior
(default=true / allow missing keys).
Automatic merge from submit-queue
conversion-gen: add --skip-unsafe flag
We should expose the SkipUnsafe option, for legacy compatability, so
that conversion-go can be used in other projects, and for platforms
where unsafe is not available.
Make unsafe code generation the default though, and have the help text
hint that the resulting code is sub-optimal.
We should expose the SkipUnsafe option, for legacy compatability, so
that conversion-go can be used in other projects, and for platforms
where unsafe is not available.
Make unsafe code generation the default though, and have the help text
hint that the resulting code is sub-optimal.
Automatic merge from submit-queue
[Federation] Make federation etcd PVC size configurable
This one implements one of the many TODO items pending in the previous set of kubefed PRs.
The design doc PR is at https://github.com/kubernetes/kubernetes/pull/34484
cc @kubernetes/sig-cluster-federation @madhusudancs
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```
[Federation] kubefed init now has a new flag, --etcd-pv-capacity, which can be used to configure the persistent volume capacity for etcd.
```
Automatic merge from submit-queue
kubedns: use initial resource listing as ready signal
Fix#35140.
Set up the ready signal after the first resource listing finished for both endpoints and services instead of listen on kubernetes service.
@bprashanth @bowei @thockin
**Release note**:
```
```
Automatic merge from submit-queue
add a configuration for kubelet to register as a node with taints
and deprecate --register-schedulable
ref #28687#29178
cc @dchen1107 @davidopp @roberthbailey
Automatic merge from submit-queue
Split inflight requests into read-only and mutating groups
cc @smarterclayton @lavalamp @caesarxuchao
```release-note
API server have two separate limits for read-only and mutating inflight requests.
```
Automatic merge from submit-queue
plumb in front proxy group header
Builds on https://github.com/kubernetes/kubernetes/pull/36662 and https://github.com/kubernetes/kubernetes/pull/36774, so only the last commit is unique.
This completes the plumbing for front proxy header information and makes it possible to add just the front proxy header authenticator.
WIP because I'm going to assess it in use downstream.
Automatic merge from submit-queue
kubectl top pod|node should handle when Heapster is somewhere else
OpenShift runs Heapster on HTTPS, which means `top node` and `top pod`
are broken because they hardcode 'http' as the scheme. Provide an
options struct allowing users to specify `--heapster-namespace`,
`--heapster-service`, `--heapster-scheme`, and `--heapster-port` to the
commands (leveraging the existing defaults).
@kubernetes/sig-metrics makes top a little more useful in other spots
Automatic merge from submit-queue (batch tested with PRs 35300, 36709, 37643, 37813, 37697)
Add generated informers
Add informer-gen and the informers it generates. We'll do follow-up PRs to convert everything currently using the hand-written informers to the generated ones.
TODO:
- [x] switch to `GroupVersionResource`
- [x] finish godoc
@deads2k @caesarxuchao @sttts @liggitt
Automatic merge from submit-queue
Add kubernetes-anywhere as a new e2e deployment option.
This change adds support for using `kubernetes-anywhere` as a deployment option for hack/e2e.go. This work is toward the larger goal of being able to run e2e tests against `kubeadm` clusters, which `kubernetes-anywhere` supports.
**Release note**:
```release-note
Add kubernetes-anywhere as a new e2e deployment option
```
The configuration in `getConfig()` comes mostly from the defaults in `kubernetes-anywhere`. In the future, we can add more plumbing to override them via CLI flags.
CC @mikedanese
Automatic merge from submit-queue
create service add create ExternalName service implementation
@kubernetes/kubectl create service add ExternalName support, refer #34731 for more detail.
```release-note
kubectl create service externalname
```
OpenShift runs Heapster on HTTPS, which means `top node` and `top pod`
are broken because they hardcode 'http' as the scheme. Provide an
options struct allowing users to specify `--heapster-namespace`,
`--heapster-service`, `--heapster-scheme`, and `--heapster-port` to the
commands (leveraging the existing defaults).
Automatic merge from submit-queue
move parts of the mega generic run struct out
This splits the main `ServerRunOptions` into composeable pieces that are bindable separately and adds easy paths for composing servers to run delegating authentication and authorization.
@sttts @ncdc alright, I think this is as far as I need to go to make the composing servers reasonable to write. I'll try leaving it here
Automatic merge from submit-queue
[Federation][join-flags] Add flags for cluster context and secret names while joining clusters to federation.
Vast majority of cluster contexts are not RFC 1123 subdomains. Since
cluster and secret names for the API objects are derived from the
cluster context name, there is no way for users to join clusters
with such context names to federation, unless they modify the context
name in their kubeconfigs itself. That's a lot of inconvenience and
entirely goes against the goal and beats the purpose of the `kubefed`
tool. So we are providing these flags to allow users to override these
values.
Also, since users register their clusters with federation, it is makes
sense in terms of user experience to make the cluster name a positional
argument because that feels more natural. Also, specifying cluster name
in the join command as a mandatory positional argument make
`kubefed join` consistent with `kubefed unjoin`. This also means
`--cluster-context` is now made a flag and defaults to cluster name if
unspecified.
`--secret-name` also defaults to the cluster name if unspecified.
Fixes: Issue #35954
cc @kubernetes/sig-cluster-federation @quinton-hoole @irfanurrehman
Lucas Käldström