Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Bump to etcd 3.1.12 to pick up critical fix
etcd [3.1.12](https://github.com/coreos/etcd/releases/tag/v3.1.12) (as well as 3.2.17 and 3.3.2) was released yesterday to fix a bug critical to kubernetes:
Fix [mvcc "unsynced" watcher restore operation](https://github.com/coreos/etcd/pull/9297).
- "unsynced" watcher is watcher that needs to be in sync with events that have happened.
- That is, "unsynced" watcher is the slow watcher that was requested on old revision.
- "unsynced" watcher restore operation was not correctly populating its underlying watcher group.
- Which possibly causes [missing events from "unsynced" watchers](https://github.com/coreos/etcd/issues/9086).
This will be backported to 1.9 as well.
Release note:
```release-note
Upgrade the default etcd server version to 3.1.12 to pick up critical etcd "mvcc "unsynced" watcher restore operation" fix.
```
cc @gyuho @wojtek-t @shyamjvs @timothysc @jdumars
Automatic merge from submit-queue (batch tested with PRs 60891, 60935). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Rollback etcd server version to 3.1.11 due to #60589
Ref https://github.com/kubernetes/kubernetes/issues/60589#issuecomment-371171837
The dependencies were a bit complex (so many things relying on it) + the version was updated to 3.2.16 on top of the original bump.
So I had to mostly make manual reverting changes on a case-by-case basis - so likely to have errors :)
/cc @wojtek-t @jpbetz
```release-note
Downgrade default etcd server version to 3.1.11 due to #60589
```
(I'm not sure if we should instead remove release-notes of the original PRs)
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Reverting the anti-affinity from CoreDNS pods
**What this PR does / why we need it**:
Following #54164 and #59357, removing the anti-affinity from CoreDNS.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add support for binaries to run as Windows services
**What this PR does / why we need it**:
Add support for binaries to run as Windows services
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59562
**Special notes for your reviewer**:
**Release note**:
```release-note
kubelet and kube-proxy can now be ran as Windows services
```
This patch adds support for kubernetes to integrate
with Windows SCM.
As a first step both `kubelet` and `kube-proxy` can be registered as a service.
To create the service:
PS > sc.exe create <component_name> binPath= "<path_to_binary> --service <other_args>"
CMD > sc create <component_name> binPath= "<path_to_binary> --service <other_args>"
Please note that if the arguments contain spaces, it must be escaped.
Example:
PS > sc.exe create kubelet binPath= "C:\kubelet.exe --service --hostname-override 'minion' <other_args>"
CMD > sc create kubelet binPath= "C:\kubelet.exe --service --hostname-override 'minion' <other_args>"
Example to start the service:
PS > Start-Service kubelet; Start-Service kube-proxy
CMD > net start kubelet && net start kube-proxy
Example to stop the service:
PS > Stop-Service kubelet (-Force); Stop-Service kube-proxy (-Force)
CMD > net stop kubelet && net stop kube-proxy
Example to query the service:
PS > Get-Service kubelet; Get-Service kube-proxy;
CMD > sc.exe queryex kubelet && sc qc kubelet && sc.exe queryex kube-proxy && sc.exe qc kube-proxy
Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com>
Signed-off-by: Alin Gabriel Serdean <aserdean@ovn.org>
Co-authored-by: Alin Gabriel Serdean <aserdean@ovn.org>
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add mTLS to kubeadm etcd liveness probe.
**What this PR does / why we need it**:
We switched etcd over to using mTLS, but the liveness probe is still using http.
Disabling the liveness probe allows etcd to continue operating.
The real fix isn't simple, because we need to generate a client certificate for healthchecking and update the probe to exec `etcdctl` like so:
https://sourcegraph.com/github.com/coreos/etcd-operator/-/blob/pkg/util/k8sutil/pod_util.go#L71-89
~Working on patching this now.~
This PR now generates the healthcheck identity and updates the liveness probe to use it.
**Which issue(s) this PR fixes**
Fixes#59766Fixeskubernetes/kubeadm#720
**Special notes for your reviewer**:
We should generate a client cert specifically for etcd health checks so that the apiserver certs can be revoked independently.
This will be stored in `/etc/kubernetes/pki/etcd/` so that we don't have to change the pod's hostMount.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Kube-dns configmap translation to CoreDNS
**What this PR does / why we need it**:
Translation of kube-dns ConfigMap (StubDomains and UpstreamNameserver) to equivalent Proxy in CoreDNS.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixeskubernetes/kubeadm#662
**Special notes for your reviewer**:
Currently, the translation is done as follows:
Example ConfigMap of kube-dns.
```
apiVersion: v1
data:
federations: |
{"foo" : "foo.feddomain.com"}
stubDomains: |
{"abc.com" : ["1.2.3.4"], "my.cluster.local" : ["2.3.4.5"]}
upstreamNameservers: |
["8.8.8.8", "8.8.4.4"]
kind: ConfigMap
metadata:
creationTimestamp: 2018-01-22T20:21:56Z
name: kube-dns
namespace: kube-system
```
CoreDNS Corefile after translation.
```
.:53 {
errors
health
kubernetes cluster.local in-addr.arpa ip6.arpa {
upstream 8.8.8.8 8.8.4.4
pods insecure
fallthrough in-addr.arpa ip6.arpa
}
federation cluster.local {
foo foo.feddomain.com
}
prometheus :9153
proxy . 8.8.8.8 8.8.4.4
cache 30
}
abc.com:53 {
errors
cache 30
proxy . 1.2.3.4
}
my.cluster.local:53 {
errors
cache 30
proxy . 2.3.4.5
}
```
**Release note**:
```release-note
Kubeadm: CoreDNS supports migration of the kube-dns configuration to CoreDNS configuration when upgrading the service discovery from kube-dns to CoreDNS as part of Beta.
```
Allow CNI-related network plugin drivers (kubenet, cni) to search a list of
directories for plugin binaries instead of just one. This allows using an
administrator-provided path and fallbacks to others (like the previous default
of /opt/cni/bin) for backwards compatibility.
It's only used for the test code and after talking with Rajat, the
vendor stuff was never really used anyway. So convert the vendor
code into a plain array of plugin binary search paths, which is all
the vendor code was doing anyway.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm 710 Switch to a dedicated CA for kubeadm etcd identities
**What this PR does / why we need it**:
On `kubeadm init`/`kubeadm upgrade`, this PR generates an etcd specific CA for signing the following certs:
- etcd serving cert
- etcd peer cert
- apiserver etcd client cert
These certs were previously signed by the kubernetes CA.
The etcd static pod in `local.go` has also been updated to only mount the `/etcd` subdir of `cfg.CertificatesDir`.
New phase command:
```
kubeadm alpha phase certs etcd-ca
```
See the linked issue for details on why this change is an important security feature.
**Which issue(s) this PR fixes**
Fixes https://github.com/kubernetes/kubeadm/issues/710
**Special notes for your reviewer**:
#### on the master
this should still fail:
```bash
curl localhost:2379/v2/keys # no output
curl --cacert /etc/kubernetes/pki/etcd/ca.crt https://localhost:2379/v2/keys # handshake error
```
this should now fail: (previously would succeed)
```
cd /etc/kubernetes/pki
curl --cacert etcd/ca.crt --cert apiserver-kubelet-client.crt --key apiserver-kubelet-client.key https://localhost:2379/v2/keys
# curl: (35) error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate
```
this should still succeed:
```
cd /etc/kubernetes/pki
curl --cacert etcd/ca.crt --cert apiserver-etcd-client.crt --key apiserver-etcd-client.key https://localhost:2379/v2/keys
```
**Release note**:
```release-note
On cluster provision or upgrade, kubeadm generates an etcd specific CA for all etcd related certificates.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Temporary fix for LeaderElect for kube-scheduler
**What this PR does / why we need it**:
kube-controller-manager defaults --leader-elect to true. We should
do the same for kube-scheduler. kube-scheduler used to have this
set to true, but it got lost during refactoring:
efb2bb71cd
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59729
**Special notes for your reviewer**:
**Release note**:
```release-note
kube-scheduler: restores default leader election behavior. leader-elect command line parameter should "true"
```
Appearance of defer keyword inside a loop structure may caused resource leaks, it's not recommended to do it although it is in an unit test. Releasing a resource just after finishing using it is the highest effective solution, so remove defer is just OK.
Automatic merge from submit-queue (batch tested with PRs 60376, 55584, 60358, 54631, 60291). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
cloud-controller-manager get /healthz to wait for apiserver to be healthy
**What this PR does / why we need it**:
currently cloud-controller-manager use `restclient.ServerAPIVersions()` to wait for apiserver to be healthy.
Remove ServerAPIVersions and make use of /healthz as all other components do.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60288
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 53689, 56880, 55856, 59289, 60249). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add test/typecheck, a fast typecheck for all build platforms.
Add test/typecheck, a fast typecheck for all build platforms.
Most of the time spent compiling is spent optimizing and linking
binary code. Most errors occur at the syntax or semantic (type) layers.
Go's compiler is importable as a normal package, so we can do fast
syntax and type checking for the 10 platforms we build on.
This currently takes ~6 minutes of CPU time (parallelized).
This makes presubmit cross builds superfluous, since it should catch
most cross-build breaks (generally Unix and 64-bit assumptions).
Example output:
```$ time go run test/typecheck/main.go
type-checking: linux/amd64, windows/386, darwin/amd64, linux/arm,
linux/386, windows/amd64, linux/arm64, linux/ppc64le, linux/s390x, darwin/386
ERROR(windows/amd64) pkg/proxy/ipvs/proxier.go:1708:27: ENXIO not declared by package unix
ERROR(windows/386) pkg/proxy/ipvs/proxier.go:1708:27: ENXIO not declared by package unix
real 0m45.083s
user 6m15.504s
sys 1m14.000s
```
```release-note
NONE
```
kube-controller-manager defaults --leader-elect to true. We should
do the same for kube-scheduler. kube-scheduler used to have this
set to true, but it got lost during refactoring:
efb2bb71cd
Automatic merge from submit-queue (batch tested with PRs 60430, 60115, 58052, 60355, 60116). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Make nodeport ip configurable
**What this PR does / why we need it**:
By default, kube-proxy accepts everything from NodePort without any filter. It can be a problem for nodes which has both public and private NICs, and people only want to provide a service in private network and avoid exposing any internal service on the public IPs.
This PR makes nodeport ip configurable.
**Which issue(s) this PR fixes**:
Closes: #21070
**Special notes for your reviewer**:
Design proposal see: https://github.com/kubernetes/community/pull/1547
Issue in feature repo: https://github.com/kubernetes/features/issues/539
**Release note**:
```release-note
Make NodePort IP addresses configurable
```
Automatic merge from submit-queue (batch tested with PRs 59674, 60059, 60220, 58916, 60336). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm token create using config file
**What this PR does / why we need it**:
Extends `kubeadm token create` adding `--config` flag. Using a config file keeps the token off of bash history.
**Which issue(s) this PR fixes**:
Fixes [#461](https://github.com/kubernetes/kubeadm/issues/461)
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
code-gen: output golint compliant 'Generated by' comment
New PR instead of reopening #58115 because /reopen did not work.
This won't be ready to merge until the upstream https://github.com/kubernetes/gengo/pull/94 merges. Once that merges, the second commit will be changed to godep-save.sh and update-staging-godeps.sh, and the last commit will be changed to update-all.sh
The failing test is due to the upstream changes not being merged yet
```devel-release-note
Go code generated by the code generators will now have a comment which allows them to be easily identified by golint
```
Fixes#56489
Automatic merge from submit-queue (batch tested with PRs 59723, 60379, 60329). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Partial revert to fix local-up-cluster.sh
**What this PR does / why we need it**:
25e110dffc broke hyperkube and local-up-cluster.sh. We should revert just the changes in proxy.go to get back to a working hyperkube
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60378
**Special notes for your reviewer**:
/assign @thockin
/assign @stewart-yu
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59159, 60318, 60079, 59371, 57415). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Feature/kubeadm 594 etcd TLS on init/upgrade
**What this PR does / why we need it**:
On `kubeadm init`/`kubeadm upgrade`, this PR generates certificates for securing local etcd:
- etcd serving cert
- etcd peer cert
- apiserver etcd client cert
Flags and hostMounts are added to the etcd and apiserver static-pods to load these certs.
For connections to etcd, `https` is now used in favor of `http` and tests have been added/updated.
Etcd only listens on localhost, so the serving cert SAN defaults to `DNS:localhost,IP:127.0.0.1`.
The etcd peer cert has SANs for `<hostname>,<api-advertise-address>`, but is unused.
New kubeadm config options, `Etcd.ServerCertSANs` and `Etcd.PeerCertSANs`, are used for user additions to the default certificate SANs for the etcd server and peer certs.
This feature continues to utilize the existence of `MasterConfiguration.Etcd.Endpoints` as a feature gate for external-etcd.
If the user passes flags to configure `Etcd.{CAFile,CertFile,KeyFile}` but they omit `Endpoints`, these flags will be unused, and a warning is printed.
New phase commands:
```
kubeadm alpha phase certs etcd-server
kubeadm alpha phase certs etcd-peer
kubeadm alpha phase certs apiserver-etcd-client
```
**Which issue(s) this PR fixes**
Fixes https://github.com/kubernetes/kubeadm/issues/594
**Special notes for your reviewer**:
#### on the master
these should fail:
```bash
curl localhost:2379/v2/keys # no output
curl --cacert /etc/kubernetes/pki/ca.crt https://localhost:2379/v2/keys # handshake error
```
these should succeed:
```
cd /etc/kubernetes/pki
curl --cacert ca.crt --cert apiserver-etcd-client.crt --key apiserver-etcd-client.key https://localhost:2379/v2/keys
```
**Release note**:
```release-note
On cluster provision or upgrade, kubeadm now generates certs and secures all connections to the etcd static-pod with mTLS.
```
Automatic merge from submit-queue (batch tested with PRs 59159, 60318, 60079, 59371, 57415). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: use localhost for API server liveness probe
**What this PR does / why we need it**:
The current liveness probe does not work with an HA cluster created with kubeadm. The probe's `host` value will be set to the IP address of the machine where `kubeadm --init` is run. For other master nodes, the IP address will be wrong.
**Special notes for your reviewer**:
None
**Release note**:
```release-note
NONE
```
/cc @timothysc
Automatic merge from submit-queue (batch tested with PRs 60324, 60269, 59771, 60314, 59941). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
expunge the word 'manifest' from Kubelet's config API
The word 'manifest' technically refers to a container-group specification
that predated the Pod abstraction. We should avoid using this legacy
terminology where possible. Fortunately, the Kubelet's config API will
be beta in 1.10 for the first time, so we still had the chance to make
this change.
I left the flags alone, since they're deprecated anyway.
I changed a few var names in files I touched too, but this PR is the
just the first shot, not the whole campaign
(`git grep -i manifest | wc -l -> 1248`).
```release-note
Some field names in the Kubelet's now v1beta1 config API differ from the v1alpha1 API: PodManifestPath is renamed to PodPath, ManifestURL is renamed to PodURL, ManifestURLHeader is renamed to PodURLHeader.
```
Automatic merge from submit-queue (batch tested with PRs 60324, 60269, 59771, 60314, 59941). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Implement a stub server printer for CRDs
This wires up TableConvertor to CRDs and puts a basic implementation in place for custom paths. However, since our OpenAPISchema can't store OpenAPI extension fields there is no way to expose the custom column piece that get.go supports today (`x-kubernetes-print-columns`). That piece can be implemented separately and needs discussion.
As this is purely exposing the default interface, very low risk. Will add an e2e test that covers this under a registered CRD.
@soltysh @sttts @kubernetes/sig-api-machinery-pr-reviews
A couple of options for wiring up the actual definition:
1. add a new "extensions" map to spec.validation
1. Downside: won't handle future child nested fields, not the correct schema
2. try to change the OpenAPISchema3 field to support extensions
1. Would require a breaking protobuf change, is also very difficult
2. Could store the entire schema as opaque JSON and then parse on load (might be the right thing anyway)
3. Support this as an annotation in 1.11 - `alpha.customresource.k8s.io/x-kubernetes-print-columns` like the CLI
Part of #58536
Automatic merge from submit-queue (batch tested with PRs 59882, 59434, 57722, 60320, 51249). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Expose default service IP CIDR in apiserver
**What this PR does / why we need it**:
The `--service-cluster-ip-range` parameter of API server is very important for deploying Kubernetes on some clouds. The default CIDR "10.0.0.0/24" should be exposed at least from the API server's help message so that users have a better idea whether they need to change it.
This patch exposes this default value in API server's help message.
**Which issue this PR fixes** : fixes#51248
**Release note**:
```
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: Demote controlplane passthrough flags to phases alpha
After a discussion in sig cluster lifecycle we agreed that the passthrough flags should live in phases alpha, and not be 1st class flags. They already exist in the alpha command, so just removing from here.
**What this PR does / why we need it**:
We introduced some flags as 1st class flags in #58080 and decided as a sig that the flags should only live in the `alpha` command. This PR removes the flags from the `init` command so they only exist in the `alpha` command
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
relates to kubernetes/kubernetes/pull/58080
fixes kubernetes/kubeadm/issues/676
**Special notes for your reviewer**:
This is a cosmetic change, and doesn't alter any functionality of the program, only the avenue in which a user access functionality in the program.
**Release note**:
```release-note
kubeadm: Demote controlplane passthrough flags to alpha flags
```
Automatic merge from submit-queue (batch tested with PRs 60054, 60202, 60219, 58090, 60275). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixes for HTTP/2 max streams per connection setting
**What this PR does / why we need it**:
This PR makes two changes. One is to introduce a parameter
for the HTTP/2 setting that an api-server sends to its clients
telling them how many streams they may have concurrently open in
an HTTP/2 connection. If left at its default value of zero,
this means to use the default in golang's HTTP/2 code (which
is currently 250; see https://github.com/golang/net/blob/master/http2/server.go).
The other change is to make the recommended options for an aggregated
api-server set this limit to 1000. The limit of 250 is annoyingly low
for the use case of many controllers watching objects of Kinds served
by an aggregated api-server reached through the main api-server (in
its mode as a proxy for the aggregated api-server, in which it uses a
single HTTP/2 connection for all calls proxied to that aggregated
api-server).
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60042
**Special notes for your reviewer**:
**Release note**:
```release-note
Introduced `--http2-max-streams-per-connection` command line flag on api-servers and set default to 1000 for aggregated API servers.
```
Automatic merge from submit-queue (batch tested with PRs 57672, 60299, 59757, 60283, 60265). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add a metric exposing number of objects per type
Fix#51953
Adds a goroutine that periodically checks the count of objects in etcd and publishes a metric with this data.
```release-note
APIserver backed by etcdv3 exports metric showing number of resources per kind
```
Automatic merge from submit-queue (batch tested with PRs 57672, 60299, 59757, 60283, 60265). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[kube-proxy]enhance kubeproxy init flag
**What this PR does / why we need it**:
remove `TODO: once we switch everything over to Cobra commands, we can go back to calling
utilflag.InitFlags() (by removing its pflag.Parse() call). For now, we have to set the
normalize func and add the go flag set by hand.`
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59286, 59743, 59883, 60190, 60165). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
DaemonSet Controller and tests to apps/v1
**What this PR does / why we need it**:
Updates the DaemonSet controller, its integration tests, and its e2e tests to use the apps/v1 API.
**Release note**:
```release-note
The DaemonSet controller, its integration tests, and its e2e tests, have been updated to use the apps/v1 API.
```
- Place etcd server and peer certs & keys into pki subdir
- Move certs.altName functions to pkiutil + add appendSANstoAltNames()
Share the append logic for the getAltName functions as suggested by
@jamiehannaford.
Move functions/tests to certs/pkiutil as suggested by @luxas.
Update Bazel BUILD deps
- Warn when an APIServerCertSANs or EtcdCertSANs entry is unusable
- Add MasterConfiguration.EtcdPeerCertSANs
- Move EtcdServerCertSANs and EtcdPeerCertSANs under MasterConfiguration.Etcd
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
cmd/controller-manager: add OWNERS for generic controller-manager code
This should have the same owners as kube-controller-manager.
- Generate Server and Peer cert for etcd
- Generate Client cert for apiserver
- Add flags / hostMounts for etcd static pod
- Add flags / hostMounts for apiserver static pod
- Generate certs on upgrade of static-pods for etcd/kube-apiserver
- Modify logic for appending etcd flags to staticpod to be safer for external etcd
Automatic merge from submit-queue (batch tested with PRs 60302, 57921, 59042, 60126, 59561). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Removing Flexvolume feature tag in e2e tests because Flexvolume is now GA
**What this PR does / why we need it**: Flexvolume e2e tests should be run in serial test suite after the feature was marked GA.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57466
/release-note-none
/sig storage
/cc @wongma7
The word 'manifest' technically refers to a container-group specification
that predated the Pod abstraction. We should avoid using this legacy
terminology where possible. Fortunately, the Kubelet's config API will
be beta in 1.10 for the first time, so we still had the chance to make
this change.
I left the flags alone, since they're deprecated anyway.
I changed a few var names in files I touched too, but this PR is the
just the first shot, not the whole campaign
(`git grep -i manifest | wc -l -> 1248`).
Automatic merge from submit-queue (batch tested with PRs 59463, 59719, 60181, 58283, 59966). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
components pprof profiling make use of existing genericapiserver's
**What this PR does / why we need it**:
fix#60278
Instead of writing private pprof, all components make use of generic apiserver existing profiling.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60214, 58762, 59898, 59897, 60204). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add CRI container log rotation support
Fixes https://github.com/kubernetes/kubernetes/issues/58823.
This PR:
1) Added `pkg/kubelet/logs/container_log_manager.go` which manages and rotates container logs.
2) Added a feature gate `CRIContainerLogRotation` to enable the alpha feature. And 2 kubelet flags `--container-log-max-size` and `--container-log-max-files` to configure the rotation behavior.
3) Added unit test and node e2e test for container log rotation.
Note that:
1) Container log manager only starts when the container runtime is `remote` (not docker), because we can't implement `ReopenContainerLog` for docker.
2) Rotated logs are compressed with `gzip`.
2) The latest rotated log is not compressed. Because fluentd may still be reading the file right after rotation.
3) `kubectl logs` still doesn't support log rotation. This is not a regression anyway, it doesn't support log rotation for docker log today. We'll probably fix this in the future. (Issue: https://github.com/kubernetes/kubernetes/issues/59902)
An example of container log directory with `--container-log-max-files=3`:
```console
$ ls -al /var/log/pods/57146449-11ec-11e8-90e1-42010af00002
total 592
drwxr-xr-x 2 root root 4096 Feb 15 01:07 .
drwxr-xr-x 3 root root 12288 Feb 15 01:06 ..
-rw-r----- 1 root root 176870 Feb 15 01:07 log-container_0.log
-rw-r--r-- 1 root root 40239 Feb 15 01:07 log-container_0.log.20180215-010737.gz
-rw-r----- 1 root root 365996 Feb 15 01:07 log-container_0.log.20180215-010747
```
/assign @mtaufen for the config change.
/assign @dashpole @crassirostris for the log change.
/assign @feiskyer for CRI related change.
/cc @yujuhong @feiskyer @abhi @mikebrow @mrunalp @runcom
/cc @kubernetes/sig-node-pr-reviews @kubernetes/sig-instrumentation-pr-reviews
**Release note**:
```release-note
[Alpha] Kubelet now supports container log rotation for container runtime which implements CRI(container runtime interface).
The feature can be enabled with feature gate `CRIContainerLogRotation`.
The flags `--container-log-max-size` and `--container-log-max-files` can be used to configure the rotation behavior.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Migrate FeatureGates type of kube-proxy from string to map[string]bool
**What this PR does / why we need it**:
Migration of FeatureGates type. This is a follow-up of #53025.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
ref: #53025https://github.com/kubernetes/kubernetes/pull/57754#discussion_r160023416
**Special notes for your reviewer**:
/cc @luxas @mtaufen @ncdc
**Release note**:
```release-note
action required: kube-proxy: feature gates are now specified as a map when provided via a JSON or YAML KubeProxyConfiguration, rather than as a string of key-value pairs.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
apiextensions: add subresources for custom resources
Fixes#38113Fixes#58778
**Related**:
- Proposal: https://github.com/kubernetes/community/pull/913
- For custom resources to work with `kubectl scale`: https://github.com/kubernetes/kubernetes/pull/58283
**Add types**:
- Add `CustomResourceSubResources` type to CRD.
- Fix proto generation for `CustomResourceSubResourceStatus`: https://github.com/kubernetes/kubernetes/pull/55970.
- Add feature gate for `CustomResourceSubResources`.
- Update CRD strategy: if feature gate is disabled, this feature is dropped (i.e. set to `nil`).
- Add validation for `CustomResourceSubResources`:
- `SpecReplicasPath` should not be empty and should be a valid json path under `.spec`. If there is no value under the given path in the CustomResource, the `/scale` subresource will return an error on GET.
- `StatusReplicasPath` should not be empty and should be a valid json path under `.status`. If there is no value under the given path in the CustomResource, the status replica value in the /scale subresource will default to 0.
- If present, `LabelSelectorPath` should be a valid json path. If there is no value under `LabelSelectorPath` in the CustomResource, the status label selector value in the `/scale` subresource will default to the empty string.
- `ScaleGroupVersion` should be `autoscaling/v1`.
- If `CustomResourceSubResources` is enabled, only `properties` is allowed under the root schema for CRD validation.
**Add status and scale subresources**:
- Use helper functions from `apimachinery/pkg/apis/meta/v1/unstructured/helpers.go`.
- Improve error handling: https://github.com/kubernetes/kubernetes/pull/56563, https://github.com/kubernetes/kubernetes/pull/58215.
- Introduce Registry interface for storage.
- Update storage:
- Introduce `CustomResourceStorage` which acts as storage for the custom resource and its status and scale subresources. Note: storage for status and scale is only enabled when the feature gate is enabled _and_ the respective fields are enabled in the CRD.
- Introduce `StatusREST` and its `New()`, `Get()` and `Update()` methods.
- Introduce `ScaleREST` and its `New()`, `Get()` and `Update()` methods.
- Get and Update use the json paths from the CRD and use it to return an `autoscaling/v1.Scale` object.
- Update strategy:
- In `PrepareForCreate`,
- Clear `.status`.
- Set `.metadata.generation` = 1
- In `PrepareForUpdate`,
- Do not update `.status`.
- If both the old and new objects have `.status` and it is changed, set it back to its old value.
- If the old object has a `.status` but the new object doesn't, set it to the old value.
- If old object did not have a `.status` but the new object does, delete it.
- Increment generation if spec changes i.e. in the following cases:
- If both the old and new objects had `.spec` and it changed.
- If the old object did not have `.spec` but the new object does.
- If the old object had a `.spec` but the new object doesn't.
- In `Validate` and `ValidateUpdate`,
- ensure that values at `specReplicasPath` and `statusReplicasPath` are >=0 and < maxInt32.
- make sure there are no errors in getting the value at all the paths.
- Introduce `statusStrategy` with its methods.
- In `PrepareForUpdate`:
- Do not update `.spec`.
- If both the old and new objects have `.spec` and it is changed, set it back to its old value.
- If the old object has a `.spec` but the new object doesn't, set it to the old value.
- If old object did not have a `.spec` but the new object does, delete it.
- Do not update `.metadata`.
- In `ValidateStatusUpdate`:
- For CRD validation, validate only under `.status`.
- Validate value at `statusReplicasPath` as above. If `labelSelectorPath` is a path under `.status`, then validate it as well.
- Plug into the custom resource handler:
- Store all three storage - customResource, status and scale in `crdInfo`.
- Use the storage as per the subresource in the request.
- Use the validator as per the subresource (for status, only use the schema for `status`, if present).
- Serve the endpoint as per the subresource - see `serveResource`, `serveStatus` and `serveScale`.
- Update discovery by adding the `/status` and `/scale` resources, if enabled.
**Add tests**:
- Add unit tests in `etcd_test.go`.
- Add integration tests.
- In `subresources_test.go`, use the [polymporphic scale client](https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/client-go/scale) to get and update `Scale`.
- Add a test to check everything works fine with yaml in `yaml_test.go`.
**Release note**:
```release-note
`/status` and `/scale` subresources are added for custom resources.
```
This PR makes two changes. One is to introduce a parameter
for the HTTP/2 setting that an api-server sends to its clients
telling them how many streams they may have concurrently open in
an HTTP/2 connection. If left at its default value of zero,
this means to use the default in golang's HTTP/2 code (which
is currently 250).
The other change is to make the recommended options for an aggregated
api-server set this limit to 1000. The limit of 250 is annoyingly low
for the use case of many controllers watching objects of Kinds served
by an aggregated api-server reached through the main api-server (in
its mode as a proxy for the aggregated api-server, in which it uses a
single HTTP/2 connection for all calls proxied to that aggregated
api-server).
Fixes#60042
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add configuration item to allow kubeadm join to use a dns name pointing to control plane
This adds a new flag (`--apiserver-advertise-dns-address`) to kubeadm which is used in node kubelet.confg to point to API server allowing users to define a DNS entry instead of an IP address.
Fixeskubernetes/kubeadm#411
```release-note
Adds new flag `--apiserver-advertise-dns-address` which is used in node kubelet.confg to point to API server
```
// @timothysc @craigtracey
Automatic merge from submit-queue (batch tested with PRs 60158, 60156, 58111, 57583, 60055). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add support for /token subresource in serviceaccount registry
I'm planning on implementing the registry bits (this) in one PR and followup with an authenticator that supports new id tokens.
https://github.com/kubernetes/kubernetes/issues/58790
@kubernetes/sig-auth-pr-reviews
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kube-proxy: Fix flag validation for healthz-bind-address and metrics-bind-address
**What this PR does / why we need it**: `--healthz-bind-address` and `--metrics-bind-address` are broken for kube-proxy as they do not allow `ip:port` format, though they claim to support it.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: Fixes#53754
**Special notes for your reviewer**:
cc @ncdc
**Release note**:
```release-note
Fix kube-proxy flags validation for --healthz-bind-address and --metrics-bind-address to allow specifying ip:port.
```
Automatic merge from submit-queue (batch tested with PRs 60148, 60022, 59125, 60068, 60154). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Refactor kubeadm join command generation
**What this PR does / why we need it**:
Creation of the `kubeadm join` command is implemented in three different points of kubeadm:
- `kubeadm init`
- `kubeadm token create`
- `kubeadm phase bootstrap-token create`
This PR refactor above points in order to share a common function for creating the `kubeadm join` command.
**Which issue(s) this PR fixes**:
Fixes [#567](https://github.com/kubernetes/kubeadm/issues/567)
**Special notes for your reviewer**:
While implementing the PR, I changed the order of parameters in `kubeadm join` from:
```
kubeadm join --token 8df4zm.5jyv2nrxb18y84jq 172.31.0.101:6443 --discovery-token-ca-cert-hash sha256:b62e1f70c1c6afebe36bc971b15b90f7e453f1c0fe2ddc4d92e07512f1143194
```
to
```
kubeadm join 172.31.0.101:6443 --token 8df4zm.5jyv2nrxb18y84jq --discovery-token-ca-cert-hash sha256:b62e1f70c1c6afebe36bc971b15b90f7e453f1c0fe2ddc4d92e07512f1143194
```
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60148, 60022, 59125, 60068, 60154). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Deprecate KubeletConfiguration flags
This is a follow-up to #53833 to deprecate the old flags in favor of the
config file.
```release-note
Flags that can be set via the Kubelet's --config file are now deprecated in favor of the file.
```
Automatic merge from submit-queue (batch tested with PRs 58716, 59977, 59316, 59884, 60117). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Cap how long the kubelet waits when it has no client cert
If we go a certain amount of time without being able to create a client
cert and we have no current client cert from the store, exit. This
prevents a corrupted local copy of the cert from leaving the Kubelet in a
zombie state forever. Exiting allows a config loop outside the Kubelet
to clean up the file or the bootstrap client cert to get another client
cert.
Five minutes is a totally arbitary timeout, judged to give enough time for really slow static pods to boot.
@mikedanese
```release-note
Set an upper bound (5 minutes) on how long the Kubelet will wait before exiting when the client cert from disk is missing or invalid. This prevents the Kubelet from waiting forever without attempting to bootstrap a new client credentials.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
oidc authentication: switch to v2 of coreos/go-oidc
Switch to v2 of [coreos/go-oidc](https://github.com/coreos/go-oidc), which uses square/go-jose to verify tokens and supports more signing algorithms.
Most of this PR removes dependencies used by the older version of github.com/coreos/go-oidc, and updates vendor files.
This PR has been tested against tokens issued by Okta, Google, and CoreOS's dex.
Closes https://github.com/kubernetes/kubernetes/issues/57806
```release-note
kube-apiserver: the OpenID Connect authenticator can now verify ID Tokens signed with JOSE algorithms other than RS256 through the --oidc-signing-algs flag.
kube-apiserver: the OpenID Connect authenticator no longer accepts tokens from the Google v3 token APIs, users must switch to the "https://www.googleapis.com/oauth2/v4/token" endpoint.
```
cc @rithujohn191 @liggitt
cc @kubernetes/sig-auth-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 59901, 59302, 59928). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Rename StorageProtection to StorageObjectInUseProtection
Rename StorageProtection to StorageObjectInUseProtection
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59639
**Special notes for your reviewer**:
**Release note**:
```release-note
Rename StorageProtection to StorageObjectInUseProtection
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add criSocket to kubeadm MasterConfiguration manifest
**What this PR does / why we need it**:
Adds a criSocket field to the MasterConfiguration manifest used by kubeadm. This field configures the cri socket that kubeadm uses during preflight checks.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes # kubernetes/kubeadm#679
**Special notes for your reviewer**:
kubeadm does not allow the use of --config and the --cri-socket flag together. When using kubeadm to create a cluster that will not be using docker, the preflight checks fail since this is not configurable. This PR adds the criSocket to the MasterConfiguration manifest and uses that value within the MasterConfiguration if it was provided.
Storing the value of the criSocket within the MasterConfiguration manifest will also make joining additional masters with the proposed 'kubeadm join --master' command by not requiring operators to remember to include an additional flag. This may not be the case if we instead relaxed the constraint of using additional flags when using the --config flag is set.
**Release note**:
/area kubeadm
/assign @luxas
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews
```release-note
kubeadm: add criSocket field to MasterConfiguration manifiest
```
Automatic merge from submit-queue (batch tested with PRs 59292, 59600). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add criSocket to kubeadm NodeConfiguration manifest
**What this PR does / why we need it**:
Adds a criSocket field to the NodeConfiguration manifest used by kubeadm. This field configures the cri socket that kubeadm uses during preflight checks.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes # kubernetes/kubeadm#679
**Special notes for your reviewer**:
This is a follow up PR, as requested, to https://github.com/kubernetes/kubernetes/pull/59057. The NodeConfiguration manifest now has a criSocket field that can be used when using the config manifest to join a node to the cluster.
**Release note**:
/area kubeadm
/assign @luxas
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews
```release-note
kubeadm: add criSocket field to NodeConfiguration manifiest
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Introduce PodSecurityPolicy in the policy/v1beta1 API group
Types/constants are completely the same as in `extensions/v1beta1` except that they are located outside of the `extensions` API group.
**What this PR does / why we need it**:
This is the first step for migrating PSP-related stuff away of `extensions` group. See #43214 for more information.
Also it related to https://github.com/kubernetes/features/issues/5
**Example**:
```console
$ cat restricted2.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: restricted2
...
$ kubectl create -f restricted.yaml
podsecuritypolicy "restricted2" created
$ kubectl get psp restricted2 -o yaml
apiVersion: extensions/v1beta1
kind: PodSecurityPolicy
...
```
**Release note**:
```release-note
The `PodSecurityPolicy` API has been moved to the `policy/v1beta1` API group. The `PodSecurityPolicy` API in the `extensions/v1beta1` API group is deprecated and will be removed in a future release.
```
Automatic merge from submit-queue (batch tested with PRs 59391, 58097, 60057, 60090). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add IPv6 to command options.
**What this PR does / why we need it**:
This PR adds IPv6 information to the descriptions for several command options.
[Related PR in kubernetes/website](https://github.com/kubernetes/website/pull/6498)
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update bazelbuild/rules_go, kubernetes/repo-infra, and gazelle dependencies
**What this PR does / why we need it**: updates our bazelbuild/rules_go dependency in order to bump everything to go1.9.4. I'm separating this effort into two separate PRs, since updating rules_go requires a large cleanup, removing an attribute from most build rules.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Deprecate kubelet flag for cadvisor port
**Which issue(s) this PR fixes**:
Issue: #56523
TL;DR the Kubelet's `stats/summary` API is the preferred way of monitoring the node. If you need additional metrics from cAdvisor, it can be run as a daemonset.
**Release note**:
```release-note
Deprecate the kubelet's cadvisor port
```
/assign @mtaufen @tallclair
cc @kubernetes/sig-node-pr-reviews
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Log the command line flags
**What this PR does / why we need it**:
With d7ddcca231, we lost the logging
of the flags. We should at least log what the command line flags
were used to start processes as those incredibly useful for trouble shooting.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
/assign @deads2k
/assign @liggitt
**Release note**:
```release-note
NONE
```
With d7ddcca231, we lost the logging
of the flags. We should at least log what the command line flags
were used to start processes as those incredibly useful for trouble shooting.
After a discussion in sig cluster lifecycle we agreed that the passthrough flags should live in phases alpha, and not be 1st class flags.
Relates to kubernetes/kubernetes/pull/58080
Closes kubernetes/kubeadm/issues/676
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Secure Kubelet's componentconfig defaults while maintaining CLI compatibility
This updates the Kubelet's componentconfig defaults, while applying the legacy defaults to values from options.NewKubeletConfiguration(). This keeps defaults the same for the command line and improves the security of defaults when you load config from a file.
See: https://github.com/kubernetes/kubernetes/issues/53618
See: https://github.com/kubernetes/kubernetes/pull/53833#discussion_r166669931
Also moves EnableServer to KubeletFlags, per @tallclair's comments on #53833.
We should find way of generating documentation for config file defaults, so that people can easily look up what's different from flags.
```release-note
Action required: Default values differ between the Kubelet's componentconfig (config file) API and the Kubelet's command line. Be sure to review the default values when migrating to using a config file.
```
Automatic merge from submit-queue (batch tested with PRs 59800, 59817, 59711). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Kubemark using cobra commands
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove CSI plugin from ProbeExpandableVolumePlugins
Add CSI plugin when feature gate is enabled
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig storage
/assign @vladimirvivien
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove experimental keystone authenticator
**What this PR does / why we need it**:
experimental-keystone-url and experimental-keystone-ca-file were always
experimental. So we don't need a deprecation period.
KeystoneAuthenticator was on the server side and needed userid/password
to be passed in and used that to authenticate with Keystone. We now
have authentication and authorization web hooks that can be used. There
is a external repo with a webook for keystone which works fine along
with the kubectl auth provider that was added in:
a0cebcb559
So we don't need this older style / hard coded / experimental code
anymore.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
kube-apiserver: the experimental in-tree Keystone password authenticator has been removed in favor of extensions that enable use of Keystone tokens.
```
Automatic merge from submit-queue (batch tested with PRs 59653, 58812, 59582, 59665, 59511). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
controller-manager: switch to options+config pattern and add https+auth
This PR switch the {kube,cloud}-controller-managers to use the Options+Config struct pattern for bootstrapping, as we use it throughout all apiservers. This allows us to easily plug in https and authn/z support.
Fixes parts of https://github.com/kubernetes/kubernetes/issues/59483
This is equivalent to https://github.com/kubernetes/kubernetes/pull/59408 after squashing.
```release-note
Deprecate insecure HTTP port of kube-controller-manager and cloud-controller-manager. Use `--secure-port` and `--bind-address` instead.
```
Automatic merge from submit-queue (batch tested with PRs 59653, 58812, 59582, 59665, 59511). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add HTTPProxyCheck to preflight checks for 'kubeadm join' subcommand
**What this PR does / why we need it:**
Add HTTPProxyCheck for API servers
It makes sense to check API servers and print warnings if they're
going to be accessed through proxy. This is similar to what's
already done for 'kubeadm init'.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove bootstrap kubelet config on reset
**What this PR does / why we need it**:
Remove /etc/kubernetes/bootstrap-kubelet.conf when running 'kubeadm reset'
to ensure it will not be reused when joining the cluster next time.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove /ui/ redirect
The existing kube-apiserver hard-codes `/ui` to redirect to an optional add-on, which is not appropriate. It does not work in the following scenarios:
* https-enabled dashboards
* the dashboard is deployed to a different namespace or service name
* the dashboard is not installed at all
* authorization is enabled and does not allow access to /ui
This PR removes the hard-coded `/ui` redirect.
```release-note
apiserver: the /ui kube-dashboard redirect has been removed. Follow instructions specific to your deployment to access kube-dashboard
```
Automatic merge from submit-queue (batch tested with PRs 59767, 56454, 59237, 59730, 55479). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: add configuration option to not taint master
**What this PR does / why we need it**:
Although tainting the master is normally a good and proper thing to do in some situations (docker for mac in our case, but I suppose minikube and such as well) having a single host configuration is desirable.
In linuxkit we have a [workaround](443e47c408/projects/kubernetes/kubernetes/kubeadm-init.sh (L19...L22)) to remove the taint after initialisation. With the change here we could simply populate `/etc/kubeadm/kubeadm.yaml` with `noTaintMaster: true` instead and have it never be tainted in the first place.
I have only added this to the config file and not to the CLI since AIUI the latter is somewhat deprecated.
The code also arranges to _remove_ an existing taint if it is unwanted. I'm unsure if this behaviour is correct or desirable, I think a reasonable argument could be made for leaving an existing taint in place too.
Signed-off-by: Ian Campbell <ijc@docker.com>
**Release note**:
Since the requirement for this option is rather niche and not best practice in the majority of cases I'm not sure if it warrants mentioning in the release notes? If it were then perhaps
```release-note
`kubeadm init` can now omit the tainting of the master node if configured to do so in `kubeadm.yaml`.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove controller-manager --service-sync-period flag
**What this PR does / why we need it**:
This PR removes controller manager --service-sync-period flag which is not used anywhere in the code and is causing confusion
**Which issue(s) this PR fixes**
https://github.com/kubernetes/kubernetes/issues/58776
**Special notes for your reviewer**:
@deads2k this remove the flag as per the discussion on #58776
2 commits
1. one for code change
2. one for auto generated code
**Release note**:
```release-note
1. Controller-manager --service-sync-period flag is removed (was never used in the code).
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Feature Gate - Kubeadm Audit Logging
Fixeskubernetes/kubeadm#623
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
**What this PR does / why we need it**:
This PR enables [Auditing](https://kubernetes.io/docs/tasks/debug-application-cluster/audit/) behind a featureGate. A user can supply their own audit policy with configuration option as well as a place for the audit logs to live. If no policy is supplied a default policy will be provided. The default policy will log all Metadata level policy logs. It is the example provided in the documentation.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixeskubernetes/kubeadm#623
**Special notes for your reviewer**:
**Release note**:
```release-note
kubeadm: Enable auditing behind a feature gate.
```
Although tainting the master is normally a good and proper thing to do in some
situations (docker for mac in our case, but I suppose minikube and such as
well) having a single host configuration is desirable.
In linuxkit we have a [workaround](443e47c408/projects/kubernetes/kubernetes/kubeadm-init.sh (L19...L22))
to remove the taint after initialisation. With the change here we could simply
populate /etc/kubeadm/kubeadm.yaml` with `noTaintMaster: true` instead and have
it never be tainted in the first place.
I have only added this to the config file and not to the CLI since AIUI the
latter is somewhat deprecated.
The code also arranges to _remove_ an existing taint if it is unwanted. I'm
unsure if this behaviour is correct or desirable, I think a reasonable argument
could be made for leaving an existing taint in place too.
Signed-off-by: Ian Campbell <ijc@docker.com>
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: imagePullPolicy option in init config
**What this PR does / why we need it**:
This PR adds `imagePullPolicy` option to the `kubeadm init` configuration file.
The new `imagePullPolicy` option is forwarded to the generated kubelet static pods for etcd, kube-apiserver, kube-controller-manager and kube-scheduler. This option allows for precise image pull policy specification for master nodes and thus for more tight control over images. It is useful in CI environments and in environments, where the user has total control over master VM templates (thus, the master VM templates can be preloaded with the required Docker images for the control plane services).
**Special notes for your reviewer**:
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews
/area kubeadm
/assign @luxas
**Release note**:
```release-note
kubeadm: New "imagePullPolicy" option in the init configuration file, that gets forwarded to kubelet static pods to control pull policy for etcd and control plane images.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix all the typos across the project
**What this PR does / why we need it**:
There are lots of typos across the project. We should avoid small PRs on fixing those annoying typos, which is time-consuming and low efficient.
This PR does fix all the typos across the project currently. And with #59463, typos could be avoided when a new PR gets merged.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
/sig testing
/area test-infra
/sig release
/cc @ixdy
/assign @fejta
**Release note**:
```release-note
None
```
Audit logs are configurable via the MasterConfiguration file.
All options are ignored unless the FeatureGate is enabled.
Fixeskubernetes/kubeadm#623
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
It makes sense to check all API servers mentioned in the command
line and print warnings if they're going to be accessed through proxy.
This is similar to what's already done for 'kubeadm init'.
Automatic merge from submit-queue (batch tested with PRs 59344, 59595, 59598). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix kubeadm typo
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 59580, 58854). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Prefer apps/v1 storage for daemonsets, deployments, replicasets, statefulsets
The workload API objects went GA in 1.9. This means we can safely begin persisting them in etcd in apps/v1 format in 1.10.
xref #43214
```release-note
DaemonSet, Deployment, ReplicaSet, and StatefulSet objects are now persisted in etcd in apps/v1 format
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove CAdvisorPort from KubeletConfiguration struct
See: #56523, cAdvisor is becoming an implementation detail of
Kubernetes, and we should not canonize its knobs on the
KubeletConfiguration.
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Extract instantiation of cloud provider
**What this PR does / why we need it**:
Add a separate method in a new file for creating cloud providers.
Currently the code is all mixed into the controller manager. We
should actively control what is made available to the cloud provider
so list explicitly the parms needed and move the code out. This will
avoid linkages to sneak in as we will catch it better during reviews.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59190, 59360). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Adding benchmarks to envelop encryption integration tests
**What this PR does / why we need it**:
Adding benchmarks for envelop encryption integration tests.
Allows to estimate how envelop encryption may impact the performance of KubeAPI server.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Reimplement 2 tests using fakeexec
**What this PR does / why we need it**:
Used fakeexec API from utils/exec/testing to test
GetKubeletVersion and KubeletVersionCheck APIs without running kubelet.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59054, 59515, 59577). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add 'none' option to EnforceNodeAllocatable
This lets us use omitempty on `EnforceNodeAllocatable`. We shouldn't treat
`nil` as different from `[]T{}`, because this can play havoc with
serializers (a-la #43203).
See: https://github.com/kubernetes/kubernetes/pull/53833#discussion_r166672137
```release-note
'none' can now be specified in KubeletConfiguration.EnforceNodeAllocatable (--enforce-node-allocatable) to explicitly disable enforcement.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove port from HTTPProxyCheck
**What this PR does / why we need it**:
HTTPProxyCheck doesn't use port. It uses proxy related variables
(HTTP_PROXY, NO_PROXY, etc) that only operate with protocol
and host.
Removing port from the check should make it more clear for
user that port is not used as it will not be present in
the check output.
**Release note**:
```release-note
NONE
```
Add a separate method in a new file for creating cloud providers.
Currently the code is all mixed into the controller manager. We
should actively control what is made available to the cloud provider
so list explicitly the parms needed and move the code out. This will
avoid linkages to sneak in as we will catch it better during reviews.
Automatic merge from submit-queue (batch tested with PRs 57824, 58806, 59410, 59280). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
2nd try at using a vanity GCR name
The 2nd commit here is the changes relative to the reverted PR. Please focus review attention on that.
This is the 2nd attempt. The previous try (#57573) was reverted while we
figured out the regional mirrors (oops).
New plan: k8s.gcr.io is a read-only facade that auto-detects your source
region (us, eu, or asia for now) and pulls from the closest. To publish
an image, push k8s-staging.gcr.io and it will be synced to the regionals
automatically (similar to today). For now the staging is an alias to
gcr.io/google_containers (the legacy URL).
When we move off of google-owned projects (working on it), then we just
do a one-time sync, and change the google-internal config, and nobody
outside should notice.
We can, in parallel, change the auto-sync into a manual sync - send a PR
to "promote" something from staging, and a bot activates it. Nice and
visible, easy to keep track of.
xref https://github.com/kubernetes/release/issues/281
TL;DR:
* The new `staging-k8s.gcr.io` is where we push images. It is literally an alias to `gcr.io/google_containers` (the existing repo) and is hosted in the US.
* The contents of `staging-k8s.gcr.io` are automatically synced to `{asia,eu,us)-k8s.gcr.io`.
* The new `k8s.gcr.io` will be a read-only alias to whichever regional repo is closest to you.
* In the future, images will be promoted from `staging` to regional "prod" more explicitly and auditably.
```release-note
Use "k8s.gcr.io" for pulling container images rather than "gcr.io/google_containers". Images are already synced, so this should not impact anyone materially.
Documentation and tools should all convert to the new name. Users should take note of this in case they see this new name in the system.
```
This is the 2nd attempt. The previous was reverted while we figured out
the regional mirrors (oops).
New plan: k8s.gcr.io is a read-only facade that auto-detects your source
region (us, eu, or asia for now) and pulls from the closest. To publish
an image, push k8s-staging.gcr.io and it will be synced to the regionals
automatically (similar to today). For now the staging is an alias to
gcr.io/google_containers (the legacy URL).
When we move off of google-owned projects (working on it), then we just
do a one-time sync, and change the google-internal config, and nobody
outside should notice.
We can, in parallel, change the auto-sync into a manual sync - send a PR
to "promote" something from staging, and a bot activates it. Nice and
visible, easy to keep track of.
experimental-keystone-url and experimental-keystone-ca-file were always
experimental. So we don't need a deprecation period.
KeystoneAuthenticator was on the server side and needed userid/password
to be passed in and used that to authenticate with Keystone. We now
have authentication and authorization web hooks that can be used. There
is a external repo with a webook for keystone which works fine along
with the kubectl auth provider that was added in:
a0cebcb559
So we don't need this older style / hard coded / experimental code
anymore.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Do not recycle volumes that are used by pods
**What this PR does / why we need it**:
Recycler should wait until all pods that use a volume are finished.
Consider this scenario:
1. User creates a PVC that's bound to a NFS PV.
2. User creates a pod that uses the PVC
3. User deletes the PVC.
Now the PV gets `Released` (the PVC does not exists) and recycled, however the PV is still mounted to a running pod. PVC protection won't help us, because it puts finalizers on PVC that is under user's control and user can remove it.
This PR checks that there is no pod that uses a PV before it recycles it.
**Release note**:
```release-note
NONE
```
/sig storage
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update kubeadm supported etcd version to 3.2.14 in 1.10
**What this PR does / why we need it**:
Kubernetes will upgrade to etcd server 3.2.14 in 1.10 cycle (#58645), update DefaultEtcdVersion in kubeadm to this version.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
relevant PR: #57480#58645
fixes: https://github.com/kubernetes/kubeadm/issues/621
**Special notes for your reviewer**:
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews
**Release note**:
```release-note
NONE
```
kubeadm don't need to advertise this in release notes.
Timothy St. Clair