Automatic merge from submit-queue (batch tested with PRs 63151, 63795, 63553, 64068, 64113). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: Remove .AuthorizationModes in the v1alpha2 API
**What this PR does / why we need it**:
Now that we have https://github.com/kubernetes/kubernetes/pull/63879, we don't actually need to have `:AuthorizationModes` in our API anymore. This PR removes support for `.AuthorizationModes` in the v1alpha2 API, but keeps an upgrade path available (automatic conversion) from the v1alpha1 version.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of kubernetes/community#2131
**Special notes for your reviewer**:
Depends on:
- [x] https://github.com/kubernetes/kubernetes/pull/63879
- [x] https://github.com/kubernetes/kubernetes/pull/63917
**Release note**:
```release-note
[action required] kubeadm: Support for `.AuthorizationModes` in the kubeadm v1alpha2 API has been removed. Instead, you can use the `.APIServerExtraArgs` and `.APIServerExtraVolumes` fields to achieve the same effect. Files using the v1alpha1 API and setting this field will be automatically upgraded to this v1alpha2 API and the information will be preserved.
```
@kubernetes/sig-cluster-lifecycle-pr-reviews @liztio
Automatic merge from submit-queue (batch tested with PRs 63881, 64046, 63409, 63402, 63221). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Kubelet responds to ConfigMap mutations for dynamic Kubelet config
This PR makes dynamic Kubelet config easier to reason about by leaving less room for silent skew scenarios. The new behavior is as follows:
- ConfigMap does not exist: Kubelet reports error status due to missing source
- ConfigMap is created: Kubelet starts using it
- ConfigMap is updated: Kubelet respects the update (but we discourage this pattern, in favor of incrementally migrating to a new ConfigMap)
- ConfigMap is deleted: Kubelet keeps using the config (non-disruptive), but reports error status due to missing source
- ConfigMap is recreated: Kubelet respects any updates (but, again, we discourage this pattern)
This PR also makes a small change to the config checkpoint file tree structure, because ResourceVersion is now taken into account when saving checkpoints. The new structure is as follows:
```
- dir named by --dynamic-config-dir (root for managing dynamic config)
| - meta
| - assigned (encoded kubeletconfig/v1beta1.SerializedNodeConfigSource object, indicating the assigned config)
| - last-known-good (encoded kubeletconfig/v1beta1.SerializedNodeConfigSource object, indicating the last-known-good config)
| - checkpoints
| - uid1 (dir for versions of object identified by uid1)
| - resourceVersion1 (dir for unpacked files from resourceVersion1)
| - ...
| - ...
```
fixes: #61643
```release-note
The dynamic Kubelet config feature will now update config in the event of a ConfigMap mutation, which reduces the chance for silent config skew. Only name, namespace, and kubeletConfigKey may now be set in Node.Spec.ConfigSource.ConfigMap. The least disruptive pattern for config management is still to create a new ConfigMap and incrementally roll out a new Node.Spec.ConfigSource.
```
Automatic merge from submit-queue (batch tested with PRs 63881, 64046, 63409, 63402, 63221). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Kubelet config: Validate new config against future feature gates
This fixes an issue with KubeletConfiguration validation, where the
feature gates set by the new config were not taken into account.
Also fixes a validation issue with dynamic Kubelet config, where flag
precedence was not enforced prior to dynamic config validation in the
controller; this prevented rejection of dynamic configs that don't merge
well with values set via legacy flags.
Fixes#63305
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63881, 64046, 63409, 63402, 63221). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Disable the public cadvisor port by default
**What this PR does / why we need it**:
Quoting @tallclair in #56523:
> We should add the deprecation warning in 1.10 along with a release note, but not change the default. The notes should warn that the default will change in 1.11. We disable the flag by default in 1.11, and remove it entirely in 1.12 or 1.13.
> If you currently depend on the UI or the API, speak up! Going forward, the recommended way of taking advantage of those features will be to run cAdvisor as a DaemonSet.
Disabling the publicly-available cAdvisor port is beneficial for security, as you might not want to expose the UI with lots of information about what your system is doing. We already did this for all kubeadm deployments in v1.7, and haven't recieved any issues for that. This should be okay to do at this stage, as this flag was deprecated in v1.10. Given we need to support this flag for one more release (v1.11), it makes perfect sense to instead switch it off in preparation for v1.12 when we can delete it (see the [deprecation policy](https://kubernetes.io/docs/reference/deprecation-policy/#deprecating-a-flag-or-cli))
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of #56523
**Special notes for your reviewer**:
I removed the e2e test that expects cAdvisor to be running, as we don't expect it to be anymore.
**Release note**:
```release-note
[action required] The formerly publicly-available cAdvisor web UI that the kubelet ran on port 4194 by default is now turned off by default. The flag configuring what port to run this UI on `--cadvisor-port` was deprecated in v1.10. Now the default is `--cadvisor-port=0`, in other words, to not run the web server. The recommended way to run cAdvisor if you still need it, is via a DaemonSet. The `--cadvisor-port` will be removed in v1.12
```
cc @kubernetes/sig-cluster-lifecycle-pr-reviews @kubernetes/sig-auth-pr-reviews @kubernetes/sig-node-pr-reviews
Add the file diff_test.go, which has a single test:
TestRunDiff
The test covers most error cases for the runDiff() function,
and also performs a valid diff.
A couple of test files are added in:
cmd/kubeadm/app/cmd/upgrade/testdata/
1) Store the io.Writer and pass it to sub-commands in upgrade.go
2) Check if the manifest path is an empty string in diff.go:runDiff()
3) Use the io.Writer that upgrade.go defines instead of writing to
os.Stdout directly.
This fixes an issue with KubeletConfiguration validation, where the
feature gates set by the new config were not taken into account.
Also fixes a validation issue with dynamic Kubelet config, where flag
precedence was not enforced prior to dynamic config validation in the
controller; this prevented rejection of dynamic configs that don't merge
well with values set via legacy flags.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: Fix how kubeadm handles `.AuthorizationModes` and `.APIServerExtraArgs`
**What this PR does / why we need it**:
If _authorization-mode_ is configured as `--apiserver-extra-args` for kubeadm, than
_authorization-mode_ argument gets duplicated in the static pod manifest file.
```
$ kubeadm alpha phase controlplane apiserver --apiserver-extra-args authorization-mode=AlwaysAllow
$ grep authorization-mode /etc/kubernetes/manifests/kube-apiserver.yaml
- --authorization-mode=AlwaysAllow
- --authorization-mode=Node,RBAC
```
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig cluster-lifecycle
/assign @fabriziopandini
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add roundtrip, defaulting, upgrading and validation unit tests for the kubeadm API types
**What this PR does / why we need it**:
Follows up from https://github.com/kubernetes/kubernetes/pull/63799, as well as net-new unit testing for our serialization/deserialization package. This tests our API machinery pretty much end to end.
This is more important now given we now support two external types: https://github.com/kubernetes/kubernetes/pull/63788
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of kubernetes/community#2131
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@kubernetes/sig-cluster-lifecycle-pr-reviews @liztio
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix little bug in kube-scheduler options
fix some little bugs in kube-scheduler options.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60012, 63692, 63977, 63960, 64008). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
uses a more resilient way to get branch name from version
**What this PR does / why we need it**:
Currently `getBranchFromVersion` uses a hard coded way to extract branch name from the version string. This is error prone, especially when bumping the minimum supported k8s version from v1.9 to v1.10.
This follow-up PR tries to use a more resilient way to handle this.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
xref #63920
**Special notes for your reviewer**:
/cc luxas timothysc
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 60012, 63692, 63977, 63960, 64008). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use Dial with context
**What this PR does / why we need it**:
`net/http/Transport.Dial` field is deprecated:
```go
// DialContext specifies the dial function for creating unencrypted TCP connections.
// If DialContext is nil (and the deprecated Dial below is also nil),
// then the transport dials using package net.
DialContext func(ctx context.Context, network, addr string) (net.Conn, error)
// Dial specifies the dial function for creating unencrypted TCP connections.
//
// Deprecated: Use DialContext instead, which allows the transport
// to cancel dials as soon as they are no longer needed.
// If both are set, DialContext takes priority.
Dial func(network, addr string) (net.Conn, error)
```
This PR switches all `Dial` usages to `DialContext`. Fixes#63455.
**Special notes for your reviewer**:
Also related: https://github.com/kubernetes/kubernetes/pull/59287https://github.com/kubernetes/kubernetes/pull/58532https://github.com/kubernetes/kubernetes/issues/815https://github.com/kubernetes/community/pull/1166https://github.com/kubernetes/kubernetes/pull/58677https://github.com/kubernetes/kubernetes/pull/57932
**Release note**:
```release-note
HTTP transport now uses `context.Context` to cancel dial operations. k8s.io/client-go/transport/Config struct has been updated to accept a function with a `context.Context` parameter. This is a breaking change if you use this field in your code.
```
/sig api-machinery
/kind enhancement
/cc @sttts
Automatic merge from submit-queue (batch tested with PRs 63969, 63902, 63689, 63973, 63978). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: Remove dependency on pkg/client/clientset_generated/internalclientset
**What this PR does / why we need it**:
This PR removes dependency in kubeadm on `pkg/client/clientset_generated/internalclientset` and blacklists it in `.import-restrictions`.
The above package is pulled in transitively by the usage of the following:
- [GetFlagString](a481f4bbe8/cmd/kubeadm/app/cmd/version.go (L63)) from `k8s.io/kubernetes/pkg/kubectl/cmd/util`
- [LeaseEndpointReconcilerType](a481f4bbe8/cmd/kubeadm/app/phases/controlplane/manifests.go (L205)) from `k8s.io/kubernetes/pkg/master/reconcilers`
Dropping the unneeded dependencies from the build shaves off 4-5MB from the amd64 executable and reduces the dependency hell in kubeadm.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixeskubernetes/kubeadm#806
**Special notes for your reviewer**:
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews
/area kubeadm
/assign @luxas
/assign @timothysc
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63969, 63902, 63689, 63973, 63978). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add a 'kubeadm config print-default' command
**What this PR does / why we need it**:
Improves the UX around creating config files.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubeadm/issues/829
**Special notes for your reviewer**:
**Release note**:
```release-note
kubeadm: A `kubeadm config print-default` command has now been added that you can use as a starting point when writing your own kubeadm configuration files
```
@kubernetes/sig-cluster-lifecycle-pr-reviews @liztio
Automatic merge from submit-queue (batch tested with PRs 63569, 63918, 63980, 63295, 63989). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm - fix upgrades with static pod etcd
**What this PR does / why we need it**:
This PR fixes a regression introduced by https://github.com/kubernetes/kubernetes/pull/63495 which broke kubeadm upgrades for installations using a static pod etcd.
**Release note**:
```release-note
NONE
```
kubeadm uses LeaseEndpointReconcilerType as import from
k8s.io/kubernetes/pkg/master/reconcilers. However, this pull a huge
load of extra dependencies (among which
pkg/client/clientset_generated/internalclientset). The solution is
to copy this string constant locally in kubeadm.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Automatic merge from submit-queue (batch tested with PRs 63920, 63716, 63928, 60553, 63946). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update kubeadm's minimum supported Kubernetes in v1.11.x to 1.10
**What this PR does / why we need it**:
This PR does,
* Restricts supported Kubernetes version in 1.11 cycle;
* Removes useless v190-specific variables, since the minimum version is v1.10;
* Bumps etcd version based on k8s version to 1.10.X => 3.1.12, 1.11.X => 3.2.18, 1.12.X => 3.2.18;
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixeskubernetes/kubeadm#815
**Special notes for your reviewer**:
/cc kubernetes/sig-cluster-lifecycle-pr-reviews
/cc luxas
**Release note**:
```release-note
Update kubeadm's minimum supported kubernetes in v1.11.x to 1.10
```
Using GetFlagString from k8s.io/kubernetes/pkg/kubectl/cmd/util
causes pkg/client/clientset_generated/internalclientset to be pulled
as an external dependency, thus producing larger executable and
increasing the maintainance hell.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Automatic merge from submit-queue (batch tested with PRs 63865, 57849, 63932, 63930, 63936). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Implement `kubeadm init diff`
**What this PR does / why we need it**:
Some users want to see the changes `kubeadm` woulda apply before actually running `kubeadm upgrade apply`. This shows the changes that will be made to the static pod manifests before applying them. This is a narrower case than `kubeadm upgrade apply --dry-run`, which specifically focuses on the static pod manifests.
**Which issue(s) this PR fixes**:
Part of [kubeadm/489](https://github.com/kubernetes/kubeadm/issues/489#issuecomment-388974795)
**Special notes for your reviewer**:
**Release note**:
```release-note
adds the `kubeadm upgrade diff` command to show how static pod manifests will be changed by an upgrade.
```
Automatic merge from submit-queue (batch tested with PRs 63865, 57849, 63932, 63930, 63936). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Minor fixes for kubeadm reset
This set of fixes was proposed by @neolit123 in [PR 63849](https://github.com/kubernetes/kubernetes/pull/63849) review
comments:
- Capitalized output after [reset]
- Used %v format for Go errors
- Fixed spelling: Trying using -> Trying to use
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: Remove the `.CloudProvider` and `.PrivilegedPods` configuration option
**What this PR does / why we need it**:
Removes the `.CloudProvider` option, it has been experimental for a long time. People should now use external cloud providers, which is beta in v1.11. Most importantly, you can get the exact same behavior in the API by utilizing the `.*ExtraArgs` and `.*ExtraVolumes` fields.
Removes `.PrivilegedPods` as that serves a super small edge case with the legacy cloud provider, and only for openstack.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of https://github.com/kubernetes/community/pull/2131
**Special notes for your reviewer**:
Depends on PRs:
- [x] #63799
- [x] #63788
**Release note**:
```release-note
[action required] In the new v1alpha2 kubeadm Configuration API, the `.CloudProvider` and `.PrivilegedPods` fields don't exist anymore.
Instead, you should use the out-of-tree cloud provider implementations which are beta in v1.11.
If you have to use the legacy in-tree cloud providers, you can rearrange your config like the example below.
If you need to use the `.PrivilegedPods` functionality, you can still edit the manifests in
`/etc/kubernetes/manifests/`, and set `.SecurityContext.Privileged=true` for the apiserver
and controller manager.
---
kind: MasterConfiguration
apiVersion: kubeadm.k8s.io/v1alpha2
apiServerExtraArgs:
cloud-provider: "{cloud}"
cloud-config: "{path}"
apiServerExtraVolumes:
- name: cloud
hostPath: "{path}"
mountPath: "{path}"
controllerManagerExtraArgs:
cloud-provider: "{cloud}"
cloud-config: "{path}"
controllerManagerExtraVolumes:
- name: cloud
hostPath: "{path}"
mountPath: "{path}"
---
```
@kubernetes/sig-cluster-lifecycle-pr-reviews @dims @liztio
This command will use crictl or docker to pull images locally.
The dockerfall back is needed because in some cases the kubelet is not
yet running so there is no CRI dockershim socket available.
Fixeskubernetes/kubeadm#812
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
Automatic merge from submit-queue (batch tested with PRs 63314, 63884, 63799, 63521, 62242). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: Restructure internal config usage and fix bugs
**What this PR does / why we need it**:
- Moves the generic LoadYAML function from the versioned, external API package to a helper library so it can be consumed more easily
- Makes the upgrading code use the internal version of the API (which always should be used anyway)
- Moves all config-loading code to `configutil`, together with the migration code needed. This way we have everything in one centralized place, instead of duplicating that logic N times.
- Makes `kubeadm init` use `configutil` for the reasons mentioned above.
This PR is needed in order to support multiple external API groups (like v1alpha2)
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of https://github.com/kubernetes/community/pull/2131
**Special notes for your reviewer**:
This PR depends on:
- https://github.com/kubernetes/kubernetes/pull/63782
- https://github.com/kubernetes/kubernetes/pull/63783
**Please review only the last (third) commit**
**Release note**:
```release-note
NONE
```
@kubernetes/sig-cluster-lifecycle-pr-reviews @liztio
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Move to a structured status for dynamic kubelet config
This PR updates dynamic Kubelet config to use a structured status, rather than a node condition. This makes the status machine-readable, and thus more useful for config orchestration.
Fixes: #56896
```release-note
The status of dynamic Kubelet config is now reported via Node.Status.Config, rather than the KubeletConfigOk node condition.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
pass stop channel to node-controller
node controller run with stop channel passed from outside.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63589, 63644, 63861, 63872, 63847). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm preflight check for IPv4 and IPv6 forwarding
**What this PR does / why we need it**: adds preflight check for IP forwarding
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixeskubernetes/kubeadm#580
**Special notes for your reviewer**:
**Release note**:
```release-note
kubeadm now checks that IPv4/IPv6 forwarding is enabled
```
Automatic merge from submit-queue (batch tested with PRs 63658, 63509, 63800, 63586, 63840). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm - default CoreDNS FeatureGate to true
**What this PR does / why we need it**:
This PR updates kubeadm to deploy CoreDNS rather than KubeDNS by default for new installs.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Addresses part of https://github.com/kubernetes/kubeadm/issues/782
**Release note**:
```release-note
kubeadm will now deploy CoreDNS by default instead of KubeDNS
```
Currently, this does not effect upgrades. Also, documentation updates will need to be coordinated with this change.
Updates dynamic Kubelet config to use a structured status, rather than a
node condition. This makes the status machine-readable, and thus more
useful for config orchestration.
Fixes: #56896
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
scheduler: remove nested retry loops and fix serving metrics under healthz server port
This PR fixes the issues of #63404:
- The Serve func for metrics and healthz does not block. Inside, there is a retry loop. This PR fixes this and gets rid of the error messages every 5 seconds.
- The separate metrics server is only to be started if it is configured on another port. Before this PR we were wrongly checking for the healthz service to be activated and then launched the metrics server instead. Because both server startups are in a go routine, they were racing against each other. If you were unlucky, the metrics endpoint was winning and it returned 404 on /healthz (while it should not have been active in the first place). The kubemark tests run the scheduler with a liveness probe which failed, restarting the scheduler after some minutes.
This set of fixes was proposed by @neolit123 in PR 63849 review
comments:
- Capitalized output after [reset]
- Used %v format for Go errors
- Fixed spelling: Trying using -> Trying to use
Automatic merge from submit-queue (batch tested with PRs 63272, 63782, 63715, 63811, 63803). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Refactor kubeadm config list-images
In preparation for creating a `kubeadm config images pull`
this commit refactors `kubeadm config list-images` into
`kubeadm config images list`.
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
**What this PR does / why we need it**:
Prepares the `kubeadm config images` subcommand so we can add more functionality to it.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Related to kubernetes/kubeadm#812
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63272, 63782, 63715, 63811, 63803). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Stop installing kubeadm types in the generic, legacy scheme
**What this PR does / why we need it**:
Right now the kubeadm API types are (wrongly from a correctness perspective) installed in the "catch-all" and now legacy `pkg/api/legacyscheme`. Instead, we should use our own, kubeadm-specific `scheme` where our API types are registered. Doing this also cuts one dependency on the internal core API types.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of https://github.com/kubernetes/community/pull/2131
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@kubernetes/sig-cluster-lifecycle-pr-reviews @sttts @liztio
Automatic merge from submit-queue (batch tested with PRs 57536, 63812). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: Contact the kubelet on its healthz port instead of its readonly port
**What this PR does / why we need it**:
In order for us to disable the kubelet's readonly port in v1.11 (https://github.com/kubernetes/kubeadm/issues/732), we need to cut the dependency on that port being open. Instead, we can use the dedicated healthz port (using the defaults `--healthz-bind-address=127.0.0.1` and `--healthz-port=10248`, xref: https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/)
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of https://github.com/kubernetes/kubeadm/issues/732
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@kubernetes/sig-cluster-lifecycle-pr-reviews @liztio
Currently kubeadm only performs an upgrade if the etcd cluster is
colocated with the control plane node. As this is only one possible
configuration, kubeadm should support upgrades with etcd clusters
that are not local to the node.
Signed-off-by: Craig Tracey <craigtracey@gmail.com>
Automatic merge from submit-queue (batch tested with PRs 63492, 62379, 61984, 63805, 63807). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Sort arguments before joining them, for reproducible return string
**What this PR does / why we need it**:
This PR makes kubeadm static pod manifest generation consistent. Right now when `kubeadm init` is called repeatedly, the generated pod manifest files under /etc/kubernetes/manifest/ are changing. Its really hard to test how a configuration change effects the manifest files.
The current implementation is ranging over a map[string]string which will be happening in a random order, generating different pod manifests even without changing any configuration.
The suggested solution makes pom manifest generation idempotent. It opens up integration test possibilities, like testing whole yaml result of `kubeadm alpha phase controlplane`.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig cluster-lifecycle
/assign @luxas
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
track/close kubelet->API connections on heartbeat failure
xref #48638
xref https://github.com/kubernetes-incubator/kube-aws/issues/598
we're already typically tracking kubelet -> API connections and have the ability to force close them as part of client cert rotation. if we do that tracking unconditionally, we gain the ability to also force close connections on heartbeat failure as well. it's a big hammer (means reestablishing pod watches, etc), but so is having all your pods evicted because you didn't heartbeat.
this intentionally does minimal refactoring/extraction of the cert connection tracking transport in case we want to backport this
* first commit unconditionally sets up the connection-tracking dialer, and moves all the cert management logic inside an if-block that gets skipped if no certificate manager is provided (view with whitespace ignored to see what actually changed)
* second commit plumbs the connection-closing function to the heartbeat loop and calls it on repeated failures
follow-ups:
* consider backporting this to 1.10, 1.9, 1.8
* refactor the connection managing dialer to not be so tightly bound to the client certificate management
/sig node
/sig api-machinery
```release-note
kubelet: fix hangs in updating Node status after network interruptions/changes between the kubelet and API server
```
Automatic merge from submit-queue (batch tested with PRs 55511, 63372, 63400, 63100, 63769). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Create pkg/scheduling/apis/v1beta1 and move priorityClass to beta
**What this PR does / why we need it**:
This is for creating pkg/apis/scheduling/v1beta1 so that priorityClasses could be moved to beta.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of #57471
**Special notes for your reviewer**:
/cc @bsalamat @aveshagarwal
**Release note**:
```release-note
The `PriorityClass` API is promoted to `scheduling.k8s.io/v1beta1`
```
Automatic merge from submit-queue (batch tested with PRs 55511, 63372, 63400, 63100, 63769). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Improve kubeadm upgrade path for CoreDNS
**What this PR does / why we need it**:
This PR achieves the following:
1. Retains the CoreDNS ConfigMap defined by the user during `kubeadm upgrade`.
2. Removes the CoreDNS deployment when user decides to install kube-dns during `kubeadm upgrade`.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
In preparation for creating a `kubeadm config images pull`
this commit refactors `kubeadm config list-images` into
`kubeadm config images list`.
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
Automatic merge from submit-queue (batch tested with PRs 63787, 62003). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: Use the API machinery for marshalling
**What this PR does / why we need it**:
Similar to https://github.com/kubernetes/kubernetes/pull/63723, we should not use a generic `yaml`
package for marshalling, but use the API machinery we have already in place. This now correctly includes `apiVersion` and `kind` when printing the configuration in the upgrade procedure.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of https://github.com/kubernetes/community/pull/2131
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@kubernetes/sig-cluster-lifecycle-pr-reviews @liztio
Automatic merge from submit-queue (batch tested with PRs 63588, 63806). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm - do not generate etcd ca/certs for external etcd
**What this PR does / why we need it**:
Currently we generate an etcd CA and certificates even if we are specifying an external etcd cluster when running `kubeadm init`, this PR changes this behavior to skip generating the etcd CA and certificates if configured for an external etcd cluster.
**Which issue(s) this PR fixes**
Fixes https://github.com/kubernetes/kubeadm/issues/807
**Release note**:
```release-note
kubeadm will no longer generate an unused etcd CA and certificates when configured to use an external etcd cluster.
```
Automatic merge from submit-queue (batch tested with PRs 63735, 62807). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Kubeadm: Update comments and UT to remove /66 restriction
**What this PR does / why we need it**:
The comments in cmd/kubeadm/app/phases/controlplane/manifests.go mention the
IPv6 /66 restriction, and the UT also refers to this.
This restriction was removed in PR#60089
This removes those comments and updates the UT
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62806
**Special notes for your reviewer**:
**Release note**:
```release-note-none
```
Automatic merge from submit-queue (batch tested with PRs 63735, 62807). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm-init: update note about failing containers
**What this PR does / why we need it**:
This PR was merged before all requested changes
were applied:
kubernetes/pull/59731
Update the note about failing containers as requested here:
https://github.com/kubernetes/kubernetes/pull/59731#pullrequestreview-119517427
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
NONE
**Special notes for your reviewer**:
NONE
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63783, 63734). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Kubeadm marshal tests
**What this PR does / why we need it**:
Adds tests for the marshalling utilities in kubeadm.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes: kubernetes/kubeadm#802
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63783, 63734). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Rename `kubeadmapiext` to the more explicit `kubeadmapiv1alpha1`
**What this PR does / why we need it**:
`kubeadmext` is somewhat confusing to those who read the code (although it means "the external API of kubeadm", which to some degree makes sense), so I'm swapping all references to it to the more explicit `kubeadmapiv1alpha1`. This change is needed given that we will support multiple external APIs.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of https://github.com/kubernetes/community/pull/2131
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@kubernetes/sig-cluster-lifecycle-pr-reviews @liztio
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add necessary explanation for container log rotation.
**What this PR does / why we need it**:
#59898
> Container log manager only starts when the container runtime is remote (not docker), because we can't implement ReopenContainerLog for docker.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Control exactly what we use in kubeadm
**What this PR does / why we need it**:
we recently got rid of extra arg validation for preflight check:
9f21f5dd1e
the import-boss configuration will help us control exactly what
gets added/used in kubeadm. In this PR, we add a white-list of
the exact packages we use currently.
We will need to review these, make new PR(s) to eliminate things
we don't want to be depending on (just like the cloud provider related code).
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
we recently got rid of extra arg validation for preflight check:
9f21f5dd1e
the import-boss configuration will help us control exactly what
gets added/used in kubeadm. In this PR, we add a white-list of
the exact packages we use currently.
We will need to review these, make new PR(s) to eliminate things
we don't want to be depending on (just like the cloud provider related code).
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Revisit the OWNERS file for kubeadm
**What this PR does / why we need it**:
The OWNERS file for kubeadm is getting a little bit stale. As discussed in today's SIG Cluster Lifecycle meeting, we're gonna update it with the currently active contributors.
**Special notes for your reviewer**:
Every person that is involved here, please ACK and LGTM the change.
@jbeda removed from approvers
@krousey removed from approvers/reviewers
@fabriziopandini graduated to an approver
@dmmcquay removed from reviewers
@jamiehannaford removed from reviewers
@kargakis removed from reviewers
@liztio added to reviewers
@chuckha added to reviewers
@detiber added to reviewers
@stealthybox added to reviewers
@dixudx added to reviewers
Thank you everyone for your contributions 👏 (no one can't maintain something forever), and congratulations and welcome everyone with a new role, happy to have you here 👍!
**Release note**:
```release-note
NONE
```
cc @kubernetes/sig-cluster-lifecycle-pr-reviews
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: add test coverage to completion.go
**What this PR does / why we need it**:
Add `completion_test.go` with the following tests:
- TestNewCmdCompletion
- TestRunCompletion
A separate commit exports the function GetSupportedShells() to obtain the list of supported shells.
Test coverage is at 96%. The only untested bit is an `io.Writer.Write()` call in `RunCompletion()`. in the case of `bytes.Buffer` it would panic and/or always return `nil` for `error`.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
NONE
**Special notes for your reviewer**:
NONE
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63367, 63718, 63446, 63723, 63720). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add kind and version to Kubeadm MasterConfiguration configmaps
**What this PR does / why we need it**:
Right now Kubeadm serializes its configuration objects with just `yaml.Marshal`. By switching to `runtime.Encode`, we guarantee that version information will be added.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
**Special notes for your reviewer**:
First part of KEP0008
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 63367, 63718, 63446, 63723, 63720). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
finish new dynamic client and deprecate old dynamic client
Builds on a couple other pulls. This completes the transition to the new dynamic client.
@kubernetes/sig-api-machinery-pr-reviews
@caesarxuchao @sttts
```release-note
The old dynamic client has been replaced by a new one. The previous dynamic client will exist for one release in `client-go/deprecated-dynamic`. Switch as soon as possible.
```
Automatic merge from submit-queue (batch tested with PRs 63049, 59731). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add instructions on how to debug a crashed pod
When I was using `kubeadm init`, I ran into an issue where I had passed an
invalid flag through the kubeadm config file. The flag was being passed into
apiserver and preventing it from launching with a "unknown flag" error.
The flag in question is (other flags elided for clarity):
```
admission-control: ...,GenericAdmissionWebhook,...
```
Since this prevented the apiserver from coming up, the setup timed out
and gave me the error message I just modified.
It would be better if the config was vetted more thoroughly, but I think
documenting the backup strategy for viewing logs in case of failure is
also valuable.
**What this PR does / why we need it**:
This PR extends an error message in kubeadm to make it clear a potential step forward for new cluster users.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59731
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63049, 59731). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
re-enable nodeipam in kube-controller-manager
**What this PR does / why we need it**:
Re-enables nodeipam controller for external clouds. Also does a small refactor so that we don't need to pass in `allocateNodeCidr` into the controller.
In v1.10 we made a change (9187b343e1 (diff-f11913dc67d80d36b3d06a93f61c49cf) in https://github.com/kubernetes/kubernetes/pull/57492) where nodeipam would be disabled for any cluster that sets `--cloud-provider=external`. The original intention behind this was that the nodeipam controller is cloud specific for some clouds (only GCE at the moment) so it should be moved to the CCM (cloud controller manager). After some discussions with wg-cloud-provider it makes sense to re-enable nodeipam controller in KCM and have GCE CCM enable its own cloud-specific IPAM controller as part of [Initialize()](https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/cloud.go#L33-L35). This would allow for GCE to run nodeipam in both KCM (by setting --cloud-provider=gce and --allocate-node-cidr) and in the CCM (once implemented in `Initialize()`) without disabling nodeipam in the KCM for all external clouds and avoids having to implement nodeipam in CCM.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Re-enable nodeipam controller for external clouds.
```
Automatic merge from submit-queue (batch tested with PRs 63673, 63712, 63691, 63684). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add 'system-cluster-critical' to kubeadm control plane pods.
**What this PR does / why we need it**:
Add 'system-cluster-critical' to kubeadm control plane pods.
**Which issue(s) this PR fixes**:
Fixes https://github.com/kubernetes/kubeadm/issues/395
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 63673, 63712, 63691, 63684). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm - add preflight warning when using systemd-resolved
**What this PR does / why we need it**:
This PR adds a preflight warning when the host is running systemd-resolved.
Newer Ubuntu releases (artful and bionic in particular) run systemd-resolved by default and in the dfeault configuration have an /etc/resolv.conf file that references 127.0.0.53 which is not accessible from containers running on the host. We will now provide a warning to the user to tell them that the kubelet args should include `--resolv-conf=/run/systemd/resolve/resolv.conf`. `/run/systemd/resolve/resolv.conf`.
**Which issue(s) this PR fixes**:
This does not resolve the following issues, but it does provide better output to the users affected by the issues: https://github.com/kubernetes/kubeadm/issues/273https://github.com/kubernetes/kubeadm/issues/787
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63673, 63712). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove ExtraArgs kubeadm preflight check
This check pulled in a number of dependencies that bloated the dep graph.
The feature itself was not worth an extra 500 dependencies so we decided
to remove the feature.
Closeskubernetes/kubeadm#497
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
**What this PR does / why we need it**:
This PR removes a check that was pulling in a lot of external dependencies. We decided the check was not worth the extra dependencies.
**Special notes for your reviewer**:
We might want to keep the first part of the check and only delete the second part, but it was easier to delete the whole thing.
**Release note**:
```release-note
Removes a preflight check for kubeadm that validated custom kube-apiserver, kube-controller-manager and kube-scheduler arguments.
```
Automatic merge from submit-queue (batch tested with PRs 63669, 63511, 63561, 63289). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
move resource builder to generic options
This pull snips the last kube/kube links from the resource builder and moves it to the genericclioptions to be built out into flags from there.
@kubernetes/sig-cli-maintainers
```release-note
NONE
```
The comments in cmd/kubeadm/app/phases/controlplane/manifests.go mention the
IPv6 /66 restriction, and the UT also refers to this.
This restriction was removed in PR#60089
Automatic merge from submit-queue (batch tested with PRs 62665, 62194, 63616, 63672, 63450). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Adds kubeadm images command
Closeskubernetes/kubeadm#388
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
**What this PR does / why we need it**:
This PR adds a `list-images` subcommand to `kubeadm config`. We need this to make installing kubernetes on air-gapped environments a little easier. This command will print out a list of images it expects to use for the master node.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixeskubernetes/kubeadm#388
**Special notes for your reviewer**:
**Release note**:
```release-note
Adds a list-images subcommand to kubeadm that lists required images for a kubeadm install.
```
Automatic merge from submit-queue (batch tested with PRs 60200, 63623, 63406). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm-init: add details about --token
**What this PR does / why we need it**:
Add details about the required format when passing
a bootstrap token using 'kubeadm init --token'.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixeskubernetes/kubeadm#768
**Special notes for your reviewer**:
NONE
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Optimize admission plugin API requests
xref https://github.com/kubernetes/kubernetes/issues/63030#issuecomment-387774934
* resource changes are typically slow-moving, and full discovery can be slightly expensive, so this reduces the refresh to a 30 second interval and bumps QPS on the admission client (which wasn't done when the client started getting used for discovery in #62659)
* a large consumer of API requests in scale tests was the node restriction plugin pod lookups during pod deletion. this switches to use the same informer that is feeding the node authorizer graph to avoid those lookups entirely.
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
change deprecated Kubelet --allow-privileged flag default to true
This enables a smooth transition to PSP. Today, users would have to
manually set --allow-privileged to true before transitioning to PSP,
which isn't a smooth deprecation path for the flag (we want people
to *stop* setting it). This PR makes the default behavior isomorphic
with what will happen after the flag is removed.
Defaulting --allow-privileged to true should be safe, because it simply
allows a superset of Pods to run (all workloads continue to work).
WRT https://github.com/kubernetes/kubernetes/issues/58010#issuecomment-383264473
the --allow-privileged flag is effectively useless for security, so this
shouldn't be a concern from that perspective.
I also bumped the deprecation timeline in the comment to 1.13.0, so that
we give people the full period of time to stop setting
--allow-privileged, now that the behavior makes it possible to do so.
```release-note
The Kubelet's deprecated --allow-privileged flag now defaults to true. This enables users to stop setting --allow-privileged in order to transition to PodSecurityPolicy. Previously, users had to continue setting --allow-privileged, because the default was false.
```
Automatic merge from submit-queue (batch tested with PRs 63624, 59847). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
explicit kubelet config key in Node.Spec.ConfigSource.ConfigMap
This makes the Kubelet config key in the ConfigMap an explicit part of
the API, so we can stop using magic key names.
As part of this change, we are retiring ConfigMapRef for ConfigMap.
```release-note
You must now specify Node.Spec.ConfigSource.ConfigMap.KubeletConfigKey when using dynamic Kubelet config to tell the Kubelet which key of the ConfigMap identifies its config file.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[kubeadm] Support `HostPathType` of ExtraVolumes in the kubeadm configuration file.
**What this PR does / why we need it**:
Now we use `DirectoryOrCreate ` as a default `HostPathType` in the kubeadm configuration file, when we create user's extra volumes(like `apiServerExtraVolumes`). So, user can't use other `HostPathType`. In order to let users use other types of `HostPath`(like `File`), I think we should support `HostPathType` of ExtraVolumes in the kubeadm configuration file.
**Which issue(s) this PR fixes**
ref [kubernetes/kubeadm#788](https://github.com/kubernetes/kubeadm/issues/788)
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62850, 63504). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Improve where we load builds from for kubeadm upgrade jobs
**What this PR does / why we need it**:
For 1.8,1.9,1.10 branches, we need to load the latest
version from ci/latest*.txt.
For master, we need to load the version number from
ci-cross/latest.txt
try these to verify:
```
gsutil cat gs://kubernetes-release-dev/ci/latest-1.9.txt
gsutil cat gs://kubernetes-release-dev/ci-cross/latest.txt
```
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubernetes/issues/61483
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixed where we get latest builds for stable branches
```
Automatic merge from submit-queue (batch tested with PRs 62850, 63504). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm-token: search for existing kubeconfig files
**What this PR does / why we need it**:
Add means to search the current user home path and
the environment variable KUBECONFIG for existing files if the
user does not provide a --kubeconfig flag.
If the user provides a --kubeconfig flag respect it.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubeadm/issues/198
**Special notes for your reviewer**:
i'm not 100% sure if that's the correct approach and if that's what's requested. so let's discuss it.
**Release note**:
```release-note
Search standard KubeConfig file locations when using `kubeadm token` without `--kubeconfig`.
```
This check pulled in a number of dpendencies that bloated the dep graph.
The feature itself was not worth an extra 500 dependencies so we decided
to remove the feature.
Closeskubernetes/kubeadm#497
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
We should stop special casing "ci-cross" and just use the
configuration in test-infra to dictate where we pick up
the builds from.
For 1.8,1.9,1.10 branches, we need to load the latest
version from ci/latest*.txt.
For master, 1.11 etc, we need to load from ci-cross/latest*.txt.
We need to update test-infra configs if things fail.
try these to verify:
```
gsutil cat gs://kubernetes-release-dev/ci/latest-1.9.txt
gsutil cat gs://kubernetes-release-dev/ci-cross/latest.txt
```
Automatic merge from submit-queue (batch tested with PRs 63297, 61883). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix commands running crictl
**What this PR does / why we need it**:
Running "kubeadm reset --cri-socket unix:///var/run/crio/crio.sock"
fails with this error:
[reset] Cleaning up running containers using crictl with socket unix:///var/run/crio/crio.sock
[reset] Failed to list running pods using crictl. Trying using docker instead.
The actual error returned by underlying API os/exec is:
fork/exec /usr/bin/crictl -r /var/run/crio/crio.sock info: no such file or directory
This is caused by passing full command line instead of executable
path as a first parameter to the Command API.
Fixed by passing correct parameters to the Command API.
Improved error output.
**Special notes for your reviewer**:
This issue was caused by breaking crictl command execution in [PR 58802](https://github.com/kubernetes/kubernetes/pull/58802)
**Release note**:
```release-note
NONE
```
This makes the Kubelet config key in the ConfigMap an explicit part of
the API, so we can stop using magic key names.
As part of this change, we are retiring ConfigMapRef for ConfigMap.
Automatic merge from submit-queue (batch tested with PRs 63526, 60371, 63444). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
update garbage collection to use the new dynamic client
Update GC to use the new and easy to use dynamic client. This is one of two remaining stragglers.
@kubernetes/sig-api-machinery-pr-reviews
@caesarxuchao @ironcladlou
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63373, 63338). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Wire ConfigFlags struct through cmdutil.Factory
**Release note**:
```release-note
NONE
```
Continues the Flags struct pattern established with PrintFlags.
Wires ConfigFlags through the client_access_factory
cc @deads2k @soltysh
Automatic merge from submit-queue (batch tested with PRs 62914, 63431). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
use new dynamic client
Updates kubeadm to use the new dynamic client that simplifies call patterns.
@kubernetes/sig-cluster-lifecycle-pr-reviews
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add dynamic RESTMapper to ControllerContext and make it generic
Add RESTMapper to ControllerContext and make it generic.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63315, 63383, 63318, 63439). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Re-enable lease reconciler, fix shutdown race
Fixes#63378Fixes#57617
* Fixes the openapi script to wait for the apiserver on shutdown (like all the other scripts do)
* Fixes the apiserver shutdown to not hang forever if the kubernetes service reconciler cannot persist to etcd
* Readds https://github.com/kubernetes/kubernetes/pull/58474 to make the default the lease reconciler
```release-note
kube-apiserver: the default `--endpoint-reconciler-type` is now `lease`. The `master-count` endpoint reconciler type is deprecated and will be removed in 1.13.
```
This enables a smooth transition to PSP. Today, users would have to
manually set --allow-privileged to true before transitioning to PSP,
which isn't a smooth deprecation path for the flag (we want people
to *stop* setting it). This PR makes the default behavior isomorphic
with what will happen after the flag is removed.
Defaulting --allow-privileged to true should be safe, because it simply
allows a superset of Pods to run (all workloads continue to work).
WRT https://github.com/kubernetes/kubernetes/issues/58010#issuecomment-383264473
the --allow-privileged flag is effectively useless for security, so this
shouldn't be a concern from that perspective.
I also bumped the deprecation timeline in the comment to 1.13.0, so that
we give people the full period of time to stop setting
--allow-privileged, now that the behavior makes it possible to do so.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
check for commands in kubelet command line
**What this PR does / why we need it**:
Kubelet doesn't support subcommands and silently ignores them.
This causes a lot of confusion among users, for example:
Running 'kubelet logs' can produce this output:
```
I0206 16:20:40.010949 5712 feature_gate.go:220] feature gates: &{{} map[]}
I0206 16:20:40.011054 5712 controller.go:114] kubelet config controller: starting controller
...
error: failed to run Kubelet: failed to create kubelet:
misconfiguration: kubelet cgroup driver: "cgroupfs" is different from
docker cgroup driver: "systemd"
```
The output in this case looks like a valid log output. The error is not
caused by incorrectly configured cgroup driver though. It's caused by
the fact that kubelet ignores 'logs' subcommand, so 'kubelet logs' is
essentially the same as 'kubelet': it runs kubelet with the default set
of parameters, including default cgroup driver.
**The fix**:
Added check to prevent kubelet from running if user specifies
subcommands in the command line.
**Which issue(s) this PR fixes**:
Fixes: kubeadm issue 639
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63138, 63091, 63201, 63341). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Enable bypassing online checks in kubeadm upgrade plan
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
**What this PR does / why we need it**:
This PR makes `kubeadm upgrade plan` a little nicer to use in an air gapped environment. `kubeadm upgrade plan` now accepts a version and returns that instead of checking the internet.
**Which issue(s) this PR fixes**:
Fixeskubernetes/kubeadm#698
**Special notes for your reviewer**:
I also cleaned up the tests for this section of code by adding formal names for table tests and using `t.Run`.
**Release note**:
```release-note
`kubeadm upgrade plan` now accepts a version which improves the UX nicer in air-gapped environments.
```
Automatic merge from submit-queue (batch tested with PRs 63138, 63091, 63201, 63341). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Api server extra args
**What this PR does / why we need it**:
This PR will correctly override the APIServer command with the values found in APIServerExtraArgs. If none are passed in a default set of flags will be used.
This also includes cleaning up tests by using `t.Run` and naming test cases so we know what they are doing. `t.Run` also provides better testing output when coupled with a name.
At some point we stopped using the k8s version to generate this command. It is trivial to add back if we need it later, but since we don't use it now it's best for it to go. This also exposed several test that were identical. Those have been removed.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixeskubernetes/kubeadm#760
**Special notes for your reviewer**:
I split this into two commits since it's a bit easier to think about like that, but would be fine squashing.
**Release note**:
```release-note
NONE
```
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews
Running "kubeadm reset --cri-socket unix:///var/run/crio/crio.sock"
fails with this error:
[reset] Cleaning up running containers using crictl with socket unix:///var/run/crio/crio.sock
[reset] Failed to list running pods using crictl. Trying using docker instead.
The actual error returned by underlying API os/exec is:
fork/exec /usr/bin/crictl -r /var/run/crio/crio.sock info: no such file or directory
This is caused by passing full command line instead of executable
path as a first parameter to the Command API.
Fixed by passing correct parameters to the Command API.
Improved error output.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
apiserver: master count and lease endpoint test
**What this PR does / why we need it**: Adds a test to make sure master count and lease endpoint reconcilers work well together, so we can bump LeaseEndpoint to beta. Based on Jordan's comment https://github.com/kubernetes/kubernetes/pull/58474#issuecomment-369954890.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Issue: #57617
Followup PR: #58474
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/cc @kubernetes/sig-cluster-lifecycle-api-reviews @kubernetes/sig-cluster-lifecycle-api-reviews
Automatic merge from submit-queue (batch tested with PRs 63251, 59166, 63250, 63180, 63169). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Break a generic package dependency to core/api/v1
It is not necessary for this package to depend on core/v1.
https://github.com/kubernetes/kubernetes/pull/62913 switched from using a client pool, where each groupVersionResource got its own rest client, to a single client.
This increases the QPS to account for increased requests using a single rest client rate limiter.
Use clientcmd.NewDefaultClientConfigLoadingRules()
to search the standard locations for existing KubeConfig
files if the user did not provide a --kubeconfig flag.
If the user provides a --kubeconfig flag respect it.
Also do the following:
- Add a unit test in token_test.go for the
NewDefaultClientConfigLoadingRules() functionality.
- Rename the TestConfig and related constants to make
them unique to this file.
Fixeskubernetes/kubeadm#198
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove KUBE_API_VERSIONS
Fixes https://github.com/kubernetes/kubernetes/issues/63102
KUBE_API_VERSIONS is an attempt to control the available serialization of types. It pre-dates the idea that we'll have separate schemes, so it's not a thing that makes sense anymore.
Server-side we've had a very clear message about breaks in the logs for a year "KUBE_API_VERSIONS is only for testing. Things will break.".
Client-side it became progressively more broken as we moved to generic types for CRUD more than a year ago. What is registered doesn't matter when everything is unstructured.
We should remove this piece of legacy since it doesn't behave predictable server-side or client-side.
@smarterclayton @lavalamp
@kubernetes/sig-api-machinery-bugs
```release-note
KUBE_API_VERSIONS is no longer respected. It was used for testing, but runtime-config is the proper flag to set.
```
`kubeadm upgrade plan <version>` is now supported. If no
version is supplied then the original behavior remains.
If a version is supplied there will be no pause when figuring out
versions. Kubeadm will assume the version you pass in is the latest
stable version.
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
Removes an unused parameter in getAPIServerCommand
Cleans up tests by:
* Naming the tests
* Using t.Run for better test output
* Removing duplicates
Fixeskubernets/kubeadm#760
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
Automatic merge from submit-queue (batch tested with PRs 59965, 59115, 63076, 63059). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Prepull etcd before an upgrade
If kubeadm ever has to upgrade etcd it should prepull the image so
there is less downtime during the upgrade when etcd versions change.
Fixeskubernetes/kubeadm#669
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
**What this PR does / why we need it**:
This PR Prepulls the etcd image during a `kubeadm upgrade apply`.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixeskubernetes/kubeadm#669
**Special notes for your reviewer**:
constants.MasterComponents was not changed because it is used in many places where etcd does not need to be nor should it be a part of this slice.
**Release note**:
```release-note
NONE
```
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 59965, 59115, 63076, 63059). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: prompt for confirmation when resetting a master
Signed-off-by: Alexander Brand <alexbrand09@gmail.com>
**What this PR does / why we need it**:
This PR implements a confirmation prompt when running `kubeadm reset` on a master node. This is to prevent users from mistakenly resetting a master node.
**Which issue(s) this PR fixes**:
Fixes https://github.com/kubernetes/kubeadm/issues/673
**Special notes for your reviewer**:
I am somewhat torn on the approach on how to detect that kubeadm is running on a master node. I went with checking for the apiserver manfiest file on the local filesystem, as it seems like a simpler approach when compared to getting a k8s client, getting a list of nodes, finding the current node, and checking if it has the master taint. I am happy to rework if the latter is more desirable.
Sample runs:
```
# ./kubeadm reset
[Warning] Are you sure you want to reset this master node? Type the word "confirm" to continue: no
Aborted reset operation on master node
# ./kubeadm reset
[Warning] Are you sure you want to reset this master node? Type the word "confirm" to continue: confirm
[preflight] Running pre-flight checks.
[reset] Stopping the kubelet service.
[reset] WARNING: The kubelet service could not be stopped by kubeadm: [exit status 1]
[reset] WARNING: Please ensure kubelet is stopped manually.
[reset] Unmounting mounted directories in "/var/lib/kubelet"
........
# ./kubeadm reset
[Warning] Are you sure you want to reset this master node? Type the word "confirm" to continue:
Aborted reset operation on master node
# ./kubeadm reset --confirm
[preflight] Running pre-flight checks.
[reset] Stopping the kubelet service.
[reset] WARNING: The kubelet service could not be stopped by kubeadm: [exit status 1]
[reset] WARNING: Please ensure kubelet is stopped manually.
[reset] Unmounting mounted directories in "/var/lib/kubelet"
........
```
**Release note**:
```release-note
kubeadm: prompt the user for confirmation when resetting a master node
```
Automatic merge from submit-queue (batch tested with PRs 59965, 59115, 63076, 63059). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix help message of kubeconfig-dir option(kubeadm alpha phase kubeconfig all)
**What this PR does / why we need it**:
This patch will fix wrong help message.
The command is kubeadm alpha phase kubeconfig [command]
The help message is for --kubeconfig-dir option.
kubeconfig-dir is not port.(It is directory)
So, I fixed the message.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
`NONE`
/sig cluster-lifecycle
Automatic merge from submit-queue (batch tested with PRs 61601, 62881, 63159). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
support simultaneous kubeadm --v and --config
**What this PR does / why we need it**:
Providing -v and --config parameters to increase verbosity while providing a kubeadm.config results in an error rather than providing the requested verbosity.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubeadm/issues/765
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61601, 62881, 63159). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: add test coverage to join.go
**What this PR does / why we need it**:
Add test coverage to `join.go`.
A separate commit exports the function `ValidateJoinCommandLine()` from `join.go` so that testing this file is more flexible.
Test coverage is at 76%. One untested part is successfully running `Join.Run()` without errors, but that requires a valid HTTPS API server running and a valid config. i got this partially working but gave up because i faced some cert / config blockers. suggestions on how to get that to work easily are welcome.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
NONE
**Special notes for your reviewer**:
NONE
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62982, 63075, 63067, 62877, 63141). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
eliminate indirection from type registration
Some years back there was a partial attempt to revamp api type registration, but the effort was never completed and this was before we started splitting schemes. With separate schemes, the idea of partial registration no longer makes sense. This pull starts removing cruft from the registration process and pulls out a layer of indirection that isn't needed.
@kubernetes/sig-api-machinery-pr-reviews
@lavalamp @cheftako @sttts @smarterclayton
Rebase cost is fairly high, so I'd like to avoid this lingering.
/assign @sttts
/assign @cheftako
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62982, 63075, 63067, 62877, 63141). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add warnings on using pod-infra-container-image for remote container runtime
**What this PR does / why we need it**:
We should warn on using `--pod-infra-container-image` to avoid confusions, when users are using remote container runtime.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #55676,#62388,#62732
**Special notes for your reviewer**:
/cc @kubernetes/sig-node-pr-reviews
**Release note**:
```release-note
add warnings on using pod-infra-container-image for remote container runtime
```
Instead of using kubeadmutil.CheckErr() in every single
phase of cmd.Run(), use a new helper function
NewValidJoin() that returns a single error.
This would improve the unit testing options for this file.
Otherwise any error in cmd.Run() will trigger an os.Exit()
as kubeadmutil.CheckErr() does that.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Support nsenter in non-systemd environments
**What this PR does / why we need it**:
In our CI, we run kubekins image for most of the jobs. This is a
debian image with upstart and does not enable systemd. So we should
* Bailout if any binary is missing other than systemd-run.
* SupportsSystemd should check the binary path to correctly
identify if the systemd-run is present or not
* Pass the errors back to the callers so kubelet is forced to
fail early when there is a problem. We currently assume
that all binaries are in the root directory by default which
is wrong.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59220, 62927, 63084, 63090, 62284). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix hpa-use-rest-clients help text
The help text erroneously says "WARNING: alpha feature" when it
shouldn't have. When we moved to beta, this should have been removed.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59220, 62927, 63084, 63090, 62284). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: add better test coverage to token.go
**What this PR does / why we need it**:
a PR for adding some more tests in `kubeadm/cmd` for `token.go`.
some areas of the `token.go` like listing, creating and deleting tokens can present challenges.
coverage was increased to around 87%.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
please, link issue # if you know of such.
**Special notes for your reviewer**:
none
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62655, 61711, 59122, 62853, 62390). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: Make kube-proxy tolerate all taints
**What this PR does / why we need it**:
As a essential core component, kube-proxy should generally run on all
nodes even if the cluster operator taints nodes for special purposes.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixeskubernetes/kubeadm#699
**Release note**:
```release-note
kubeadm creates kube-proxy with a toleration to run on all nodes, no matter the taint.
```
Automatic merge from submit-queue (batch tested with PRs 62655, 61711, 59122, 62853, 62390). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: Mount additional paths inside apiserver/controller-manager for working CA root
This is required for a working CA root, as /etc/ssl/certs on a few
Linux distributions just contains a bunch of symlinks.
Container Linux and Debian have symlinks pointing to
/usr/share/ca-certificates, ArchLinux has symlinks pointing
to /etc/ca-certificates.
On Debian /etc/ssl/certs can also include symlinks pointing
to /usr/local/share/ca-certificates for local CA certificates.
Fix: kubeadm/#671
---
**What this PR does / why we need it**:
Without this PR, `controller-manager` and `apiserver` would lack a CA root on some Linux distro (ex: Container Linux) which for example break flexplugins which require a CA root [[1]](https://github.com/kubernetes-incubator/external-storage/issues/571#issuecomment-360155462).
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubeadm/issues/671
**Special notes for your reviewer**:
**Release note**:
```release-note
Mount additional paths required for a working CA root, for setups where /etc/ssl/certs doesn't contains certificates but just symlink.
```
/sig sig-kubeadm
Automatic merge from submit-queue (batch tested with PRs 62655, 61711, 59122, 62853, 62390). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Implemented truncating audit backend
Fixes https://github.com/kubernetes/kubernetes/issues/60432
Introduces an optional truncating backend, disabled by default, that estimates the size of audit events and truncates events/split batches based on the configuration.
/cc @sttts @tallclair @CaoShuFeng @ericchiang
```release-note
Introduce truncating audit backend that can be enabled for existing backend to limit the size of individual audit events and batches of events.
```
Automatic merge from submit-queue (batch tested with PRs 62655, 61711, 59122, 62853, 62390). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Modify the kubeadm upgrade DAG for the TLS Upgrade
**What this PR does / why we need it**:
This adds the necessary utilities to detect Etcd TLS on static pods from the file system and query Etcd.
It modifies the upgrade logic to make it support the APIServer downtime.
Tests are included and should be passing.
```bash
bazel test //cmd/kubeadm/... \
&& bazel build //cmd/kubeadm --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64 \
&& issue=TLSUpgrade ~/Repos/vagrant-kubeadm-testing/copy_kubeadm_bin.sh
```
These cases are working consistently for me
```bash
kubeadm-1.9.6 reset \
&& kubeadm-1.9.6 init --kubernetes-version 1.9.1 \
&& kubectl apply -f https://git.io/weave-kube-1.6
/vagrant/bin/TLSUpgrade_kubeadm upgrade apply 1.9.6 # non-TLS to TLS
/vagrant/bin/TLSUpgrade_kubeadm upgrade apply 1.10.0 # TLS to TLS
/vagrant/bin/TLSUpgrade_kubeadm upgrade apply 1.10.1 # TLS to TLS
/vagrant/bin/TLSUpgrade_kubeadm upgrade apply 1.9.1 # TLS to TLS /w major version downgrade
```
This branch is based on top of #61942, as resolving the hash race condition is necessary for consistent behavior.
It looks to fit in pretty well with @craigtracey's PR: #62141
The interfaces are pretty similar
/assign @detiber @timothysc
**Which issue(s) this PR fixes**
Helps with https://github.com/kubernetes/kubeadm/issues/740
**Special notes for your reviewer**:
278b322a1c
[kubeadm] Implement ReadStaticPodFromDisk
c74b56372d
Implement etcdutils with Cluster.HasTLS()
- Test HasTLS()
- Instrument throughout upgrade plan and apply
- Update plan_test and apply_test to use new fake Cluster interfaces
- Add descriptions to upgrade range test
- Support KubernetesDir and EtcdDataDir in upgrade tests
- Cover etcdUpgrade in upgrade tests
- Cover upcoming TLSUpgrade in upgrade tests
8d8e5fe33b
Update test-case, fix nil-pointer bug, and improve error message
97117fa873
Modify the kubeadm upgrade DAG for the TLS Upgrade
- Calculate `beforePodHashMap` before the etcd upgrade in anticipation of
KubeAPIServer downtime
- Detect if pre-upgrade etcd static pod cluster `HasTLS()==false` to switch
on the Etcd TLS Upgrade if TLS Upgrade:
- Skip L7 Etcd check (could implement a waiter for this)
- Skip data rollback on etcd upgrade failure due to lack of L7 check
(APIServer is already down unable to serve new requests)
- On APIServer upgrade failure, also rollback the etcd manifest to
maintain protocol compatibility
- Add logging
**Release note**:
```release-note
kubeadm upgrade no longer races leading to unexpected upgrade behavior on pod restarts
kubeadm upgrade now successfully upgrades etcd and the controlplane to use TLS
kubeadm upgrade now supports external etcd setups
kubeadm upgrade can now rollback and restore etcd after an upgrade failure
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add --ipvs-exclude-cidrs flag to kube-proxy.
**What this PR does / why we need it**:
Add a flag to kube-proxy called --ipvs-exclude-cidrs. This flag allows a user to specify a list of CIDR ranges that should not be included in the cleanup of IPVS rules.
Fixes: #59507
**Release note**:
```
Use --ipvs-exclude-cidrs to specify a list of CIDR's which the IPVS proxier should not touch when cleaning up IPVS rules.
```
/assign @m1093782566
Fix `rollbackEtcdData()` to return error=nil on success
`rollbackEtcdData()` used to always return an error making the rest of the
upgrade code completely unreachable.
Ignore errors from `rollbackOldManifests()` during the rollback since it
always returns an error.
Success of the rollback is gated with etcd L7 healthchecks.
Remove logic implying the etcd manifest should be rolled back when
`upgradeComponent()` fails
- Calculate `beforePodHashMap` before the etcd upgrade in anticipation of KubeAPIServer downtime
- Detect if pre-upgrade etcd static pod cluster `HasTLS()==false` to switch on the Etcd TLS Upgrade
if TLS Upgrade:
- Skip L7 Etcd check (could implement a waiter for this)
- Skip data rollback on etcd upgrade failure due to lack of L7 check (APIServer is already down unable to serve new requests)
- On APIServer upgrade failure, also rollback the etcd manifest to maintain protocol compatibility
- Add logging
- Test HasTLS()
- Instrument throughout upgrade plan and apply
- Update plan_test and apply_test to use new fake Cluster interfaces
- Add descriptions to upgrade range test
- Support KubernetesDir and EtcdDataDir in upgrade tests
- Cover etcdUpgrade in upgrade tests
- Cover upcoming TLSUpgrade in upgrade tests
If kubeadm ever has to upgrade etcd it should prepull the image so
there is less downtime during the upgrade when etcd versions change.
Fixeskubernetes/kubeadm#669
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
Automatic merge from submit-queue (batch tested with PRs 62495, 63003, 62829, 62151, 62002). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix scheduler informers to receive events for all the pods in the cluster
**What this PR does / why we need it**:
This PR has an important change to fix scheduler informers. More information in #63002.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#63002
**Special notes for your reviewer**:
This should be back-ported to 1.10 and 1.9.
**Release note**:
```release-note
Fix scheduler informers to receive events for all the pods in the cluster.
```
Automatic merge from submit-queue (batch tested with PRs 62464, 62947). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
make API.ControlPlaneEndpoint accept IP
**What this PR does / why we need it**:
This PR implements one of the actions defined by https://github.com/kubernetes/kubeadm/issues/751 (checklist form implementing HA in kubeadm).
With this PR, the `API.ControlPlaneEndpoint` value in the kubeadm MasterConfiguration file now accepts both DNS and IP.
The `API.ControlPlaneEndpoint` should be used to set a stable IP address for the control plane; in an HA configuration, this should be the load balancer address (no matter if identified by a DNS name or by a stable IP).
**Special notes for your reviewer**:
/CC @timothysc
This PR is the same of https://github.com/kubernetes/kubernetes/pull/62667, that I closed by error 😥
**Release note**:
```release-note
NONE
```
Nb. first https://github.com/kubernetes/kubernetes/pull/62667 already has the release note
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove request context mapper
http.Request now allows setting/retrieving a per-request context, which removes the need for plumbing a request-context mapper throughout the stack
In addition to being way simpler, this has the benefit of removing a potentially contentious lock object from the handling path
This PR:
* removes RequestContextMapper
* converts context fetchers to use `req.Context()`
* converts context setters to use `req = req.WithContext(...)`
* updates filter plumbing in two places (audit and timeout) to properly return the request with modified context
* updates tests that used a fake context mapper to set the context in the request instead
Fixes https://github.com/kubernetes/kubernetes/issues/62796
```release-note
NONE
```
In our CI, we run kubekins image for most of the jobs. This is a
debian image with upstart and does not enable systemd. So we should:
* Bailout if any binary is missing other than systemd-run.
* SupportsSystemd should check the binary path to correctly
identify if the systemd-run is present or not
* Pass the errors back to the callers so kubelet is forced to
fail early when there is a problem. We currently assume
that all binaries are in the root directory by default which
is wrong.
- Update kubeadm static pod upgrades to use the
kubetypes.ConfigHashAnnotationKey annotation on the mirror pod rather
than generating a hash from the full object info. Previously, a status
update for the pod would allow the upgrade to proceed before the
new static pod manifest was actually deployed.
Signed-off-by: Jason DeTiberus <detiber@gmail.com>
Automatic merge from submit-queue (batch tested with PRs 61324, 62880, 62765). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Always Start pvc-protection-controller and pv-protection-controller
**What this PR does / why we need it**:
After K8s 1.10 is upgraded to K8s 1.11 finalizer `[kubernetes.io/pvc-protection]` is added to PVCs
because `StorageObjectInUseProtection` feature will be GA in K8s 1.11.
However, when K8s 1.11 is downgraded to K8s 1.10 and the `StorageObjectInUseProtection` feature is disabled the finalizers remain in the PVCs and as `pvc-protection-controller` is not started in K8s 1.10 finalizers are not removed automatically from deleted PVCs and that's why deleted PVC are not removed from the system but remain in `Terminating` phase.
The same applies to `pv-protection-controller` and `[kubernetes.io/pvc-protection]` finalizer in PVs.
That's why `pvc-protection-controller` is always started because the `pvc-protection-controller` removes finalizers from PVCs automatically when a PVC is not in active use by a pod.
Also the `pv-protection-controller` is always started to remove finalizers from PVs automatically when a PV is not `Bound` to a PVC.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes N/A
This issue https://github.com/kubernetes/kubernetes/issues/60764 is for downgrade from K8s 1.10 to K8s 1.9.
This PR fixes the same problem but for downgrade from K8s 1.11 to K8s 1.10.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 50899, 62649). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Ensure webhook service routing resolves kubernetes.default.svc correctly
Going through the normal endpoint resolve path isn't correct in multi-master scenarios
The auth wrapper is pulling from LoopbackClientConfig, the service resolver should do the same
```release-note
Fixes the kubernetes.default.svc loopback service resolution to use a loopback configuration.
```
After K8s 1.10 is upgraded to K8s 1.11 finalizer [kubernetes.io/pvc-protection] is added to PVCs
because StorageObjectInUseProtection feature will be GA in K8s 1.11.
However, when K8s 1.11 is downgraded to K8s 1.10 and the StorageObjectInUseProtection feature is disabled
the finalizers remain in the PVCs and as pvc-protection-controller is not started in K8s 1.10 finalizers
are not removed automatically from deleted PVCs and that's why deleted PVC are not removed from the system
but remain in Terminating phase.
The same applies to pv-protection-controller and [kubernetes.io/pvc-protection] finalizer in PVs.
That's why pvc-protection-controller is always started because the pvc-protection-controller removes finalizers
from PVCs automatically when a PVC is not in active use by a pod.
Also the pv-protection-controller is always started to remove finalizers from PVs automatically when a PV is not
Bound to a PVC.
Related issue: https://github.com/kubernetes/kubernetes/issues/60764
Automatic merge from submit-queue (batch tested with PRs 61962, 58972, 62509, 62606). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: move QOSReserved from experimental to alpha feature gate
Fixes https://github.com/kubernetes/kubernetes/issues/61665
**Release note**:
```release-note
The --experimental-qos-reserve kubelet flags is replaced by the alpha level --qos-reserved flag or QOSReserved field in the kubeletconfig and requires the QOSReserved feature gate to be enabled.
```
/sig node
/assign @derekwaynecarr
/cc @mtaufen
Automatic merge from submit-queue (batch tested with PRs 59592, 62308, 62523, 62635, 62243). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Separate pod priority from preemption
**What this PR does / why we need it**:
Users request to split priority and preemption feature gate so they can use priority separately.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62068
**Special notes for your reviewer**:
~~I kept use `ENABLE_POD_PRIORITY` as ENV name for gce cluster scripts for backward compatibility reason. Please let me know if other approach is preffered.~~
~~This is a potential **break change** as existing clusters will be affected, we may need to include this in 1.11 maybe?~~
TODO: update this doc https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/
[Update] Usage: in config file for scheduler:
```yaml
apiVersion: componentconfig/v1alpha1
kind: KubeSchedulerConfiguration
...
disablePreemption: true
```
**Release note**:
```release-note
Split PodPriority and PodPreemption feature gate
```
Automatic merge from submit-queue (batch tested with PRs 62726, 60085, 62583). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Refactor kubeadm api validation.
**What this PR does / why we need it**:
This PR refactor kubeadm api validation to use field json name. When users get a validation error, they can easily know which field is invalid in their config files.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
@fabriziopandini I remember we mentioned this refactoring in some comment. I didn't see this change was done, so I send this PR to address this.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62781, 62815). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update kubeadm upgrade message
**What this PR does / why we need it**:
Updates kubeadm upgrade message per user request
**Which issue(s) this PR fixes**:
Fixes https://github.com/kubernetes/kubeadm/issues/672
**Special notes for your reviewer**:
none
**Release note**:
```
NONE
```
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews @kargakis
Automatic merge from submit-queue (batch tested with PRs 62481, 62643, 61877, 62515). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add write-config-to to scheduler
**What this PR does / why we need it**:
Scheduler should be able to write its default configure to file. This actually applies to all components which claims options other than `--config` will be deprecated.
Otherwise, users will be super confused to find out how to write a proper config file to these components.
See: https://stackoverflow.com/questions/47966440/how-to-create-a-config-file-for-kube-scheduler-to-use-by-the-config-argument
ref: #52562
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#58805
Usage:
```bash
./_output/bin/kube-scheduler --write-config-to /tmp/kube-scheduler.yaml
```
**Special notes for your reviewer**:
This should have been fixed several releases ago, so lets include it in 1.11
**Release note**:
```release-note
Add write-config-to to scheduler
```
Kubernetes Submit Queue
