Automatic merge from submit-queue (batch tested with PRs 60363, 59208, 59465, 60581, 60702). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix: remove keyword defer in the loop to avoid resource leaks
Appearance of defer keyword inside a loop structure may caused resource leaks, it's not recommended to do it although it is in an unit test. Releasing a resource just after finishing using it is the most safe and effective solution, so remove defer is just OK.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Updates kubeadm default to use 1.10
**What this PR does / why we need it**:
In line with https://github.com/kubernetes/kubeadm/blob/master/docs/release-cycle.md, the default branch for kubeadm to deploy should be bumped right before the rc.1.
This can even be manually merged by the someone from the release team.
**Which issue(s) this PR fixes** :
Fixes#60608
**Special notes for your reviewer**:
**Release note**:
```
Updates default deployment for kubeadm to 1.10
```
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews @kubernetes/sig-release-members @dims @jberkus @jdumars
Automatic merge from submit-queue (batch tested with PRs 61129, 60359). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Cleanup old upgrading code that is v1.8->v1.9-specific
**What this PR does / why we need it**:
Cleanup old upgrading code that is v1.8->v1.9-specific
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubeadm/issues/622
This will finish the task in the issue.
**Special notes for your reviewer**:
/cc @luxas @vbmade2000
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60891, 60935). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Rollback etcd server version to 3.1.11 due to #60589
Ref https://github.com/kubernetes/kubernetes/issues/60589#issuecomment-371171837
The dependencies were a bit complex (so many things relying on it) + the version was updated to 3.2.16 on top of the original bump.
So I had to mostly make manual reverting changes on a case-by-case basis - so likely to have errors :)
/cc @wojtek-t @jpbetz
```release-note
Downgrade default etcd server version to 3.1.11 due to #60589
```
(I'm not sure if we should instead remove release-notes of the original PRs)
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Reverting the anti-affinity from CoreDNS pods
**What this PR does / why we need it**:
Following #54164 and #59357, removing the anti-affinity from CoreDNS.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add mTLS to kubeadm etcd liveness probe.
**What this PR does / why we need it**:
We switched etcd over to using mTLS, but the liveness probe is still using http.
Disabling the liveness probe allows etcd to continue operating.
The real fix isn't simple, because we need to generate a client certificate for healthchecking and update the probe to exec `etcdctl` like so:
https://sourcegraph.com/github.com/coreos/etcd-operator/-/blob/pkg/util/k8sutil/pod_util.go#L71-89
~Working on patching this now.~
This PR now generates the healthcheck identity and updates the liveness probe to use it.
**Which issue(s) this PR fixes**
Fixes#59766Fixeskubernetes/kubeadm#720
**Special notes for your reviewer**:
We should generate a client cert specifically for etcd health checks so that the apiserver certs can be revoked independently.
This will be stored in `/etc/kubernetes/pki/etcd/` so that we don't have to change the pod's hostMount.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Kube-dns configmap translation to CoreDNS
**What this PR does / why we need it**:
Translation of kube-dns ConfigMap (StubDomains and UpstreamNameserver) to equivalent Proxy in CoreDNS.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixeskubernetes/kubeadm#662
**Special notes for your reviewer**:
Currently, the translation is done as follows:
Example ConfigMap of kube-dns.
```
apiVersion: v1
data:
federations: |
{"foo" : "foo.feddomain.com"}
stubDomains: |
{"abc.com" : ["1.2.3.4"], "my.cluster.local" : ["2.3.4.5"]}
upstreamNameservers: |
["8.8.8.8", "8.8.4.4"]
kind: ConfigMap
metadata:
creationTimestamp: 2018-01-22T20:21:56Z
name: kube-dns
namespace: kube-system
```
CoreDNS Corefile after translation.
```
.:53 {
errors
health
kubernetes cluster.local in-addr.arpa ip6.arpa {
upstream 8.8.8.8 8.8.4.4
pods insecure
fallthrough in-addr.arpa ip6.arpa
}
federation cluster.local {
foo foo.feddomain.com
}
prometheus :9153
proxy . 8.8.8.8 8.8.4.4
cache 30
}
abc.com:53 {
errors
cache 30
proxy . 1.2.3.4
}
my.cluster.local:53 {
errors
cache 30
proxy . 2.3.4.5
}
```
**Release note**:
```release-note
Kubeadm: CoreDNS supports migration of the kube-dns configuration to CoreDNS configuration when upgrading the service discovery from kube-dns to CoreDNS as part of Beta.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm 710 Switch to a dedicated CA for kubeadm etcd identities
**What this PR does / why we need it**:
On `kubeadm init`/`kubeadm upgrade`, this PR generates an etcd specific CA for signing the following certs:
- etcd serving cert
- etcd peer cert
- apiserver etcd client cert
These certs were previously signed by the kubernetes CA.
The etcd static pod in `local.go` has also been updated to only mount the `/etcd` subdir of `cfg.CertificatesDir`.
New phase command:
```
kubeadm alpha phase certs etcd-ca
```
See the linked issue for details on why this change is an important security feature.
**Which issue(s) this PR fixes**
Fixes https://github.com/kubernetes/kubeadm/issues/710
**Special notes for your reviewer**:
#### on the master
this should still fail:
```bash
curl localhost:2379/v2/keys # no output
curl --cacert /etc/kubernetes/pki/etcd/ca.crt https://localhost:2379/v2/keys # handshake error
```
this should now fail: (previously would succeed)
```
cd /etc/kubernetes/pki
curl --cacert etcd/ca.crt --cert apiserver-kubelet-client.crt --key apiserver-kubelet-client.key https://localhost:2379/v2/keys
# curl: (35) error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate
```
this should still succeed:
```
cd /etc/kubernetes/pki
curl --cacert etcd/ca.crt --cert apiserver-etcd-client.crt --key apiserver-etcd-client.key https://localhost:2379/v2/keys
```
**Release note**:
```release-note
On cluster provision or upgrade, kubeadm generates an etcd specific CA for all etcd related certificates.
```
Appearance of defer keyword inside a loop structure may caused resource leaks, it's not recommended to do it although it is in an unit test. Releasing a resource just after finishing using it is the highest effective solution, so remove defer is just OK.
Automatic merge from submit-queue (batch tested with PRs 59674, 60059, 60220, 58916, 60336). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm token create using config file
**What this PR does / why we need it**:
Extends `kubeadm token create` adding `--config` flag. Using a config file keeps the token off of bash history.
**Which issue(s) this PR fixes**:
Fixes [#461](https://github.com/kubernetes/kubeadm/issues/461)
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59159, 60318, 60079, 59371, 57415). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Feature/kubeadm 594 etcd TLS on init/upgrade
**What this PR does / why we need it**:
On `kubeadm init`/`kubeadm upgrade`, this PR generates certificates for securing local etcd:
- etcd serving cert
- etcd peer cert
- apiserver etcd client cert
Flags and hostMounts are added to the etcd and apiserver static-pods to load these certs.
For connections to etcd, `https` is now used in favor of `http` and tests have been added/updated.
Etcd only listens on localhost, so the serving cert SAN defaults to `DNS:localhost,IP:127.0.0.1`.
The etcd peer cert has SANs for `<hostname>,<api-advertise-address>`, but is unused.
New kubeadm config options, `Etcd.ServerCertSANs` and `Etcd.PeerCertSANs`, are used for user additions to the default certificate SANs for the etcd server and peer certs.
This feature continues to utilize the existence of `MasterConfiguration.Etcd.Endpoints` as a feature gate for external-etcd.
If the user passes flags to configure `Etcd.{CAFile,CertFile,KeyFile}` but they omit `Endpoints`, these flags will be unused, and a warning is printed.
New phase commands:
```
kubeadm alpha phase certs etcd-server
kubeadm alpha phase certs etcd-peer
kubeadm alpha phase certs apiserver-etcd-client
```
**Which issue(s) this PR fixes**
Fixes https://github.com/kubernetes/kubeadm/issues/594
**Special notes for your reviewer**:
#### on the master
these should fail:
```bash
curl localhost:2379/v2/keys # no output
curl --cacert /etc/kubernetes/pki/ca.crt https://localhost:2379/v2/keys # handshake error
```
these should succeed:
```
cd /etc/kubernetes/pki
curl --cacert ca.crt --cert apiserver-etcd-client.crt --key apiserver-etcd-client.key https://localhost:2379/v2/keys
```
**Release note**:
```release-note
On cluster provision or upgrade, kubeadm now generates certs and secures all connections to the etcd static-pod with mTLS.
```
Automatic merge from submit-queue (batch tested with PRs 59159, 60318, 60079, 59371, 57415). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: use localhost for API server liveness probe
**What this PR does / why we need it**:
The current liveness probe does not work with an HA cluster created with kubeadm. The probe's `host` value will be set to the IP address of the machine where `kubeadm --init` is run. For other master nodes, the IP address will be wrong.
**Special notes for your reviewer**:
None
**Release note**:
```release-note
NONE
```
/cc @timothysc
Automatic merge from submit-queue (batch tested with PRs 60324, 60269, 59771, 60314, 59941). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
expunge the word 'manifest' from Kubelet's config API
The word 'manifest' technically refers to a container-group specification
that predated the Pod abstraction. We should avoid using this legacy
terminology where possible. Fortunately, the Kubelet's config API will
be beta in 1.10 for the first time, so we still had the chance to make
this change.
I left the flags alone, since they're deprecated anyway.
I changed a few var names in files I touched too, but this PR is the
just the first shot, not the whole campaign
(`git grep -i manifest | wc -l -> 1248`).
```release-note
Some field names in the Kubelet's now v1beta1 config API differ from the v1alpha1 API: PodManifestPath is renamed to PodPath, ManifestURL is renamed to PodURL, ManifestURLHeader is renamed to PodURLHeader.
```
Automatic merge from submit-queue (batch tested with PRs 60324, 60269, 59771, 60314, 59941). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Implement a stub server printer for CRDs
This wires up TableConvertor to CRDs and puts a basic implementation in place for custom paths. However, since our OpenAPISchema can't store OpenAPI extension fields there is no way to expose the custom column piece that get.go supports today (`x-kubernetes-print-columns`). That piece can be implemented separately and needs discussion.
As this is purely exposing the default interface, very low risk. Will add an e2e test that covers this under a registered CRD.
@soltysh @sttts @kubernetes/sig-api-machinery-pr-reviews
A couple of options for wiring up the actual definition:
1. add a new "extensions" map to spec.validation
1. Downside: won't handle future child nested fields, not the correct schema
2. try to change the OpenAPISchema3 field to support extensions
1. Would require a breaking protobuf change, is also very difficult
2. Could store the entire schema as opaque JSON and then parse on load (might be the right thing anyway)
3. Support this as an annotation in 1.11 - `alpha.customresource.k8s.io/x-kubernetes-print-columns` like the CLI
Part of #58536
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: Demote controlplane passthrough flags to phases alpha
After a discussion in sig cluster lifecycle we agreed that the passthrough flags should live in phases alpha, and not be 1st class flags. They already exist in the alpha command, so just removing from here.
**What this PR does / why we need it**:
We introduced some flags as 1st class flags in #58080 and decided as a sig that the flags should only live in the `alpha` command. This PR removes the flags from the `init` command so they only exist in the `alpha` command
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
relates to kubernetes/kubernetes/pull/58080
fixes kubernetes/kubeadm/issues/676
**Special notes for your reviewer**:
This is a cosmetic change, and doesn't alter any functionality of the program, only the avenue in which a user access functionality in the program.
**Release note**:
```release-note
kubeadm: Demote controlplane passthrough flags to alpha flags
```
- Place etcd server and peer certs & keys into pki subdir
- Move certs.altName functions to pkiutil + add appendSANstoAltNames()
Share the append logic for the getAltName functions as suggested by
@jamiehannaford.
Move functions/tests to certs/pkiutil as suggested by @luxas.
Update Bazel BUILD deps
- Warn when an APIServerCertSANs or EtcdCertSANs entry is unusable
- Add MasterConfiguration.EtcdPeerCertSANs
- Move EtcdServerCertSANs and EtcdPeerCertSANs under MasterConfiguration.Etcd
- Generate Server and Peer cert for etcd
- Generate Client cert for apiserver
- Add flags / hostMounts for etcd static pod
- Add flags / hostMounts for apiserver static pod
- Generate certs on upgrade of static-pods for etcd/kube-apiserver
- Modify logic for appending etcd flags to staticpod to be safer for external etcd
The word 'manifest' technically refers to a container-group specification
that predated the Pod abstraction. We should avoid using this legacy
terminology where possible. Fortunately, the Kubelet's config API will
be beta in 1.10 for the first time, so we still had the chance to make
this change.
I left the flags alone, since they're deprecated anyway.
I changed a few var names in files I touched too, but this PR is the
just the first shot, not the whole campaign
(`git grep -i manifest | wc -l -> 1248`).
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Migrate FeatureGates type of kube-proxy from string to map[string]bool
**What this PR does / why we need it**:
Migration of FeatureGates type. This is a follow-up of #53025.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
ref: #53025https://github.com/kubernetes/kubernetes/pull/57754#discussion_r160023416
**Special notes for your reviewer**:
/cc @luxas @mtaufen @ncdc
**Release note**:
```release-note
action required: kube-proxy: feature gates are now specified as a map when provided via a JSON or YAML KubeProxyConfiguration, rather than as a string of key-value pairs.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add configuration item to allow kubeadm join to use a dns name pointing to control plane
This adds a new flag (`--apiserver-advertise-dns-address`) to kubeadm which is used in node kubelet.confg to point to API server allowing users to define a DNS entry instead of an IP address.
Fixeskubernetes/kubeadm#411
```release-note
Adds new flag `--apiserver-advertise-dns-address` which is used in node kubelet.confg to point to API server
```
// @timothysc @craigtracey
Automatic merge from submit-queue (batch tested with PRs 60148, 60022, 59125, 60068, 60154). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Refactor kubeadm join command generation
**What this PR does / why we need it**:
Creation of the `kubeadm join` command is implemented in three different points of kubeadm:
- `kubeadm init`
- `kubeadm token create`
- `kubeadm phase bootstrap-token create`
This PR refactor above points in order to share a common function for creating the `kubeadm join` command.
**Which issue(s) this PR fixes**:
Fixes [#567](https://github.com/kubernetes/kubeadm/issues/567)
**Special notes for your reviewer**:
While implementing the PR, I changed the order of parameters in `kubeadm join` from:
```
kubeadm join --token 8df4zm.5jyv2nrxb18y84jq 172.31.0.101:6443 --discovery-token-ca-cert-hash sha256:b62e1f70c1c6afebe36bc971b15b90f7e453f1c0fe2ddc4d92e07512f1143194
```
to
```
kubeadm join 172.31.0.101:6443 --token 8df4zm.5jyv2nrxb18y84jq --discovery-token-ca-cert-hash sha256:b62e1f70c1c6afebe36bc971b15b90f7e453f1c0fe2ddc4d92e07512f1143194
```
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add criSocket to kubeadm MasterConfiguration manifest
**What this PR does / why we need it**:
Adds a criSocket field to the MasterConfiguration manifest used by kubeadm. This field configures the cri socket that kubeadm uses during preflight checks.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes # kubernetes/kubeadm#679
**Special notes for your reviewer**:
kubeadm does not allow the use of --config and the --cri-socket flag together. When using kubeadm to create a cluster that will not be using docker, the preflight checks fail since this is not configurable. This PR adds the criSocket to the MasterConfiguration manifest and uses that value within the MasterConfiguration if it was provided.
Storing the value of the criSocket within the MasterConfiguration manifest will also make joining additional masters with the proposed 'kubeadm join --master' command by not requiring operators to remember to include an additional flag. This may not be the case if we instead relaxed the constraint of using additional flags when using the --config flag is set.
**Release note**:
/area kubeadm
/assign @luxas
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews
```release-note
kubeadm: add criSocket field to MasterConfiguration manifiest
```
Automatic merge from submit-queue (batch tested with PRs 59292, 59600). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add criSocket to kubeadm NodeConfiguration manifest
**What this PR does / why we need it**:
Adds a criSocket field to the NodeConfiguration manifest used by kubeadm. This field configures the cri socket that kubeadm uses during preflight checks.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes # kubernetes/kubeadm#679
**Special notes for your reviewer**:
This is a follow up PR, as requested, to https://github.com/kubernetes/kubernetes/pull/59057. The NodeConfiguration manifest now has a criSocket field that can be used when using the config manifest to join a node to the cluster.
**Release note**:
/area kubeadm
/assign @luxas
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews
```release-note
kubeadm: add criSocket field to NodeConfiguration manifiest
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update bazelbuild/rules_go, kubernetes/repo-infra, and gazelle dependencies
**What this PR does / why we need it**: updates our bazelbuild/rules_go dependency in order to bump everything to go1.9.4. I'm separating this effort into two separate PRs, since updating rules_go requires a large cleanup, removing an attribute from most build rules.
**Release note**:
```release-note
NONE
```
Kubernetes Submit Queue