Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove rescheduler and corresponding tests from master
**What this PR does / why we need it**:
This is to remove rescheduler from master branch as we are promoting priority and preemption to beta.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of #57471
**Special notes for your reviewer**:
/cc @bsalamat @aveshagarwal
**Release note**:
```release-note
Remove rescheduler from master.
```
Automatic merge from submit-queue (batch tested with PRs 64338, 64219, 64486, 64495, 64347). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove unused status per TODO
This should have been deleted in #63221, as it is now unused.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57082, 64325, 64016, 64443, 64403). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
small nit in the annotations of pkg/kubelet/container/os.go
**What this PR does / why we need it**:
just a small nit in the annotations of container/os.go, but, it looks quite uncomfortable cause others all get right.
Automatic merge from submit-queue (batch tested with PRs 61803, 64305, 64170, 64361, 64339). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add a flag to control the cap on images reported in node status
While I normally try to avoid adding flags, this is a short term
scalability fix for v1.11, and there are other long-term solutions in
the works, so we shouldn't commit to this in the v1beta1 Kubelet config.
Flags are our escape hatch here.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58920, 58327, 60577, 49388, 62306). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Dynamic env in subpath - Fixes Issue 48677
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#48677
**Special notes for your reviewer**:
**Release note**:
```release-note
Adds the VolumeSubpathEnvExpansion alpha feature to support environment variable expansion
Sub-paths cannot be mounted with a dynamic volume mount name.
This fix provides environment variable expansion to sub paths
This reduces the need to manage symbolic linking within sidecar init containers to achieve the same goal
```
While I normally try to avoid adding flags, this is a short term
scalability fix for v1.11, and there are other long-term solutions in
the works, so we shouldn't commit to this in the v1beta1 Kubelet config.
Flags are our escape hatch.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add metadata to kubelet eviction event annotations
**What this PR does / why we need it**:
Add annotations to kubelet eviction events. Annotations include
"offending_containers" : comma-seperated list of containers.
"offending_containers_usage": comma-seperated list of usage.
"starved_resource": v1.ResourceName of the starved resource
**Special notes for your reviewer**:
Adding annotations to events required changing the `EventRecorder` interface to add a `AnnotatedEventf` function, which can add annotations to an event.
**Release note**:
```release-note
NONE
```
/assign @dchen1107
cc @mwielgus @schylek @kgrygiel
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
improve test: verify kubelet.config.Restore only happen once
**What this PR does / why we need it**:
This patch is to add additional test coverage of pod config restore,
it verifies that restore can only happen once.
in the second restore attempt, we should expect no error and no channel update.
**Which issue(s) this PR fixes**:
this is a test improvement based on test been added at https://github.com/kubernetes/kubernetes/pull/63553
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
/sig node
/cc @rphillips @jiayingz @vikaschoudhary16 @anfernee @Random-Liu @dchen1107 @derekwaynecarr
@vishh @yujuhong @tallclair
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Validate cgroups-per-qos for Windows
**What this PR does / why we need it**:
cgroups-per-qos and enforce-node-allocatable is not supported on Windows, but kubelet allows it on Windows. And then Pods may stuck in terminating state because of it. Refer #61716.
This PR adds validation for them and make kubelet refusing to start in this case.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61716
**Special notes for your reviewer**:
**Release note**:
```release-note
Fail fast if cgroups-per-qos is set on Windows
```
The goal of this change is to remove the registration of signal
handling from pkg/kubelet. We now pass in a stop channel.
If you register a signal handler in `main()` to aid in a controlled
and deliberate exit then the handler registered in `pkg/kubelet` often
wins and the process exits immediately. This means all other signal
handler registrations are currently racy if `DockerServer.Start()` is
directly or indirectly invoked.
This change also removes another signal handler registration from
`NewAPIServerCommand()`; a stop channel is now passed to this
function.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix typo in volume_stats.go
**What this PR does / why we need it**:
While reviewing the implementation details I came across a typo in volume_stats.go
sed/volumeStatsCollecotr/volumeStatsCollector/
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64013, 63896, 64139, 57527, 62102). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add dynamic config metrics
This PR exports config-releated metrics from the Kubelet.
The Guages for active, assigned, and last-known-good config can be used
to identify config versions and produce aggregate counts across several
nodes. The error-reporting Gauge can be used to determine whether a node
is experiencing a config-related error, and to prodouce an aggregate
count of nodes in an error state.
https://github.com/kubernetes/features/issues/281
```release-note
The Kubelet now exports metrics that report the assigned (node_config_assigned), last-known-good (node_config_last_known_good), and active (node_config_active) config sources, and a metric indicating whether the node is experiencing a config-related error (node_config_error). The config source metrics always report the value 1, and carry the node_config_name, node_config_uid, node_config_resource_version, and node_config_kubelet_key labels, which identify the config version. The error metric reports 1 if there is an error, 0 otherwise.
```
Automatic merge from submit-queue (batch tested with PRs 59851, 64114, 63912, 64156, 64191). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: Move RotateCertificates to the KubeletConfiguration struct
**What this PR does / why we need it**:
Moves `.RotateCertificates` to the `KubeletConfiguration` struct, so it can be configured via the Config file smoothly.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubernetes/issues/63878
Fixes https://github.com/kubernetes/kubernetes/issues/61653
**Special notes for your reviewer**:
Pretty similar to https://github.com/kubernetes/kubernetes/pull/62352
**Release note**:
```release-note
The kubelet certificate rotation feature can now be enabled via the `.RotateCertificates` field in the kubelet's config file. The `--rotate-certificates` flag is now deprecated, and will be removed in a future release.
```
@kubernetes/sig-node-pr-reviews @kubernetes/sig-cluster-lifecycle-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 63914, 63887, 64116, 64026, 62933). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Enable SELinux relabeling in CSI volumes
**What this PR does / why we need it**:
CSI volume plugin should provide correct information in `GetAttributes` call so kubelet can ask container runtime to relabel the volume. Therefore CSI volume plugin needs to check if a random volume mounted by a CSI driver supports SELinux or not by checking for "seclabel" mount or superblock option.
**Which issue(s) this PR fixes**
Fixes#63965
**Release note**:
```release-note
NONE
```
@saad-ali @vladimirvivien @davidz627
@cofyc, FYI, I'm changing `struct mountInfo`.
This PR exports config-releated metrics from the Kubelet.
The Guages for active, assigned, and last-known-good config can be used
to identify config versions and produce aggregate counts across several
nodes. The error-reporting Gauge can be used to determine whether a node
is experiencing a config-related error, and to prodouce an aggregate
count of nodes in an error state.
Automatic merge from submit-queue (batch tested with PRs 63151, 63795, 63553, 64068, 64113). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: fix checkpoint manager logic bug on restore
**What this PR does / why we need it**:
I am testing the new checkpoint logic within the kubelet and ran across a logic bug on API server restores.
Initial PR: https://github.com/kubernetes/kubernetes/pull/56040
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
**Special notes for your reviewer**:
/cc @vikaschoudhary16
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63151, 63795, 63553, 64068, 64113). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Implement watch-based secret manager
Initial experiments on 5000-node Kubemark show that apiserver is handling those with no real issues.
That said, we shouldn't enable it in prod without much more extensive scalability tests (so most probably not in 1.11), but having that in would enable easier testing.
@liggitt
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix typo: peirodically->periodically
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63881, 64046, 63409, 63402, 63221). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Kubelet responds to ConfigMap mutations for dynamic Kubelet config
This PR makes dynamic Kubelet config easier to reason about by leaving less room for silent skew scenarios. The new behavior is as follows:
- ConfigMap does not exist: Kubelet reports error status due to missing source
- ConfigMap is created: Kubelet starts using it
- ConfigMap is updated: Kubelet respects the update (but we discourage this pattern, in favor of incrementally migrating to a new ConfigMap)
- ConfigMap is deleted: Kubelet keeps using the config (non-disruptive), but reports error status due to missing source
- ConfigMap is recreated: Kubelet respects any updates (but, again, we discourage this pattern)
This PR also makes a small change to the config checkpoint file tree structure, because ResourceVersion is now taken into account when saving checkpoints. The new structure is as follows:
```
- dir named by --dynamic-config-dir (root for managing dynamic config)
| - meta
| - assigned (encoded kubeletconfig/v1beta1.SerializedNodeConfigSource object, indicating the assigned config)
| - last-known-good (encoded kubeletconfig/v1beta1.SerializedNodeConfigSource object, indicating the last-known-good config)
| - checkpoints
| - uid1 (dir for versions of object identified by uid1)
| - resourceVersion1 (dir for unpacked files from resourceVersion1)
| - ...
| - ...
```
fixes: #61643
```release-note
The dynamic Kubelet config feature will now update config in the event of a ConfigMap mutation, which reduces the chance for silent config skew. Only name, namespace, and kubeletConfigKey may now be set in Node.Spec.ConfigSource.ConfigMap. The least disruptive pattern for config management is still to create a new ConfigMap and incrementally roll out a new Node.Spec.ConfigSource.
```
Automatic merge from submit-queue (batch tested with PRs 63881, 64046, 63409, 63402, 63221). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Kubelet config: Validate new config against future feature gates
This fixes an issue with KubeletConfiguration validation, where the
feature gates set by the new config were not taken into account.
Also fixes a validation issue with dynamic Kubelet config, where flag
precedence was not enforced prior to dynamic config validation in the
controller; this prevented rejection of dynamic configs that don't merge
well with values set via legacy flags.
Fixes#63305
```release-note
NONE
```
This fixes an issue with KubeletConfiguration validation, where the
feature gates set by the new config were not taken into account.
Also fixes a validation issue with dynamic Kubelet config, where flag
precedence was not enforced prior to dynamic config validation in the
controller; this prevented rejection of dynamic configs that don't merge
well with values set via legacy flags.
Automatic merge from submit-queue (batch tested with PRs 60012, 63692, 63977, 63960, 64008). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
pkg: kubelet: remote: increase grpc client default size
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
**What this PR does / why we need it**:
when running lots and lots of containers and having tons of images on a given node, we started seeing this in the logs (with docker):
```
Unable to retrieve pods: rpc error: code = ResourceExhausted desc = grpc: received message larger than max (4208374 vs. 4194304)
```
That's because the grpc client is defaulting to a 4MB response size.
This patch increases the resp size to 8MB to avoid such issue.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
increase grpc client default response size
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix e2e "When checkpoint file is corrupted should complete pod sandbo…
…x clean up"
**What this PR does / why we need it**:
This PR fixes the e2e-node test, "When checkpoint file is corrupted should complete pod sandbox clean up"
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62738
Related #62937
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
/cc @dashpole @derekwaynecarr
/sig node
Kubernetes Submit Queue