Merge pull request #64187 from luxas/kubeadm_kubelet_improve_security

Automatic merge from submit-queue (batch tested with PRs 64174, 64187, 64216, 63265, 64223). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

kubeadm: Improve the kubelet default configuration security-wise

**What this PR does / why we need it**:
 - Disables the readonly port for the kubelets in the cluster
 - Enables delegated SA token authentication for the secure kubelet port (GCE also did this ref: https://github.com/kubernetes/kubernetes/pull/58178)
 - Follows up https://github.com/kubernetes/kubernetes/pull/63912 to move the last flag from the system dropin to the ComponentConfig

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubeadm/issues/732
Fixes https://github.com/kubernetes/kubeadm/issues/650
Replaces https://github.com/kubernetes/kubernetes/pull/57997

**Special notes for your reviewer**:
In order to make sure this actually works, or that clusters actually are secure, we're adding e2e tests for this: https://github.com/kubernetes/kubeadm/issues/838 & https://github.com/kubernetes/kubernetes/pull/64140
Depends on https://github.com/kubernetes/kubernetes/pull/63912

**Release note**:

```release-note
[action required] kubeadm: kubelets in kubeadm clusters now disable the readonly port (10255). If you're relying on unauthenticated access to the readonly port, please switch to using the secure port (10250). Instead, you can now use ServiceAccount tokens when talking to the secure port, which will make it easier to get access to e.g. the `/metrics` endpoint of the kubelet securely.
```
@kubernetes/sig-cluster-lifecycle-pr-reviews 
@kubernetes/sig-auth-pr-reviews FYI

cmd/kubeadm/app/apis/kubeadm/fuzzer/fuzzer.go

@@ -79,6 +79,7 @@ func Funcs(codecs runtimeserializer.CodecFactory) []interface{} {
 							Enabled: utilpointer.BoolPtr(false),
 						},
 					},
+					RotateCertificates: true,
 				},
 			}
 			kubeletconfigv1beta1.SetDefaults_KubeletConfiguration(obj.KubeletConfiguration.BaseConfig)

cmd/kubeadm/app/apis/kubeadm/v1alpha1/defaults.go

@@ -247,12 +247,13 @@ func SetDefaults_KubeletConfiguration(obj *MasterConfiguration) {
 	obj.KubeletConfiguration.BaseConfig.Authorization.Mode = kubeletconfigv1beta1.KubeletAuthorizationModeWebhook
 
 	// Let clients using other authentication methods like ServiceAccount tokens also access the kubelet API
-	// TODO: Enable in a future PR
-	// obj.KubeletConfiguration.BaseConfig.Authentication.Webhook.Enabled = utilpointer.BoolPtr(true)
+	obj.KubeletConfiguration.BaseConfig.Authentication.Webhook.Enabled = utilpointer.BoolPtr(true)
 
 	// Disable the readonly port of the kubelet, in order to not expose unnecessary information
-	// TODO: Enable in a future PR
-	// obj.KubeletConfiguration.BaseConfig.ReadOnlyPort = 0
+	obj.KubeletConfiguration.BaseConfig.ReadOnlyPort = 0
+
+	// Enables client certificate rotation for the kubelet
+	obj.KubeletConfiguration.BaseConfig.RotateCertificates = true
 
 	// Serve a /healthz webserver on localhost:10248 that kubeadm can talk to
 	obj.KubeletConfiguration.BaseConfig.HealthzBindAddress = "127.0.0.1"

cmd/kubeadm/app/apis/kubeadm/v1alpha2/defaults.go

@@ -210,12 +210,13 @@ func SetDefaults_KubeletConfiguration(obj *MasterConfiguration) {
 	obj.KubeletConfiguration.BaseConfig.Authorization.Mode = kubeletconfigv1beta1.KubeletAuthorizationModeWebhook
 
 	// Let clients using other authentication methods like ServiceAccount tokens also access the kubelet API
-	// TODO: Enable in a future PR
-	// obj.KubeletConfiguration.BaseConfig.Authentication.Webhook.Enabled = utilpointer.BoolPtr(true)
+	obj.KubeletConfiguration.BaseConfig.Authentication.Webhook.Enabled = utilpointer.BoolPtr(true)
 
 	// Disable the readonly port of the kubelet, in order to not expose unnecessary information
-	// TODO: Enable in a future PR
-	// obj.KubeletConfiguration.BaseConfig.ReadOnlyPort = 0
+	obj.KubeletConfiguration.BaseConfig.ReadOnlyPort = 0
+
+	// Enables client certificate rotation for the kubelet
+	obj.KubeletConfiguration.BaseConfig.RotateCertificates = true
 
 	// Serve a /healthz webserver on localhost:10248 that kubeadm can talk to
 	obj.KubeletConfiguration.BaseConfig.HealthzBindAddress = "127.0.0.1"

cmd/kubeadm/app/util/config/testdata/conversion/master/internal.yaml

@@ -130,6 +130,7 @@ KubeletConfiguration:
     registryBurst: 10
     registryPullQPS: 5
     resolvConf: /etc/resolv.conf
+    rotateCertificates: true
     runtimeRequestTimeout: 2m0s
     serializeImagePulls: true
     staticPodPath: /etc/kubernetes/manifests

cmd/kubeadm/app/util/config/testdata/conversion/master/v1alpha2.yaml

@@ -123,6 +123,7 @@ kubeletConfiguration:
     registryBurst: 10
     registryPullQPS: 5
     resolvConf: /etc/resolv.conf
+    rotateCertificates: true
     runtimeRequestTimeout: 2m0s
     serializeImagePulls: true
     staticPodPath: /etc/kubernetes/manifests

cmd/kubeadm/app/util/config/testdata/defaulting/master/defaulted.yaml

@@ -118,6 +118,7 @@ kubeletConfiguration:
     registryBurst: 10
     registryPullQPS: 5
     resolvConf: /etc/resolv.conf
+    rotateCertificates: true
     runtimeRequestTimeout: 2m0s
     serializeImagePulls: true
     staticPodPath: /etc/kubernetes/manifests