Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update documentation for azure-shared-securityrule
**What this PR does / why we need it**:
Azure augmented rules for NSGs has been GA https://azure.microsoft.com/en-us/updates/agumented-rules-ga-nsg/. This PR updates documentation for "service.beta.kubernetes.io/azure-shared-securityrule" to reflect this.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix show-all option description
**What this PR does / why we need it**:
The default value of kubectl show-all option has been changed from false to true, but its description didn't change accordingly. This patch fix it.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixes the races around devicemanager Allocate() and endpoint deletion.
There is a race in predicateAdmitHandler Admit() that getNodeAnyWayFunc()
could get Node with non-zero deviceplugin resource allocatable for a
non-existing endpoint. That race can happen when a device plugin fails,
but is more likely when kubelet restarts as with the current registration
model, there is a time gap between kubelet restart and device plugin
re-registration. During this time window, even though devicemanager could
have removed the resource initially during GetCapacity() call, Kubelet
may overwrite the device plugin resource capacity/allocatable with the
old value when node update from the API server comes in later. This
could cause a pod to be started without proper device runtime config set.
To solve this problem, introduce endpointStopGracePeriod. When a device
plugin fails, don't immediately remove the endpoint but set stopTime in
its endpoint. During kubelet restart, create endpoints with stopTime set
for any checkpointed registered resource. The endpoint is considered to be
in stopGracePeriod if its stoptime is set. This allows us to track what
resources should be handled by devicemanager during the time gap.
When an endpoint's stopGracePeriod expires, we remove the endpoint and
its resource. This allows the resource to be exported through other channels
(e.g., by directly updating node status through API server) if there is such
use case. Currently endpointStopGracePeriod is set as 5 minutes.
Given that an endpoint is no longer immediately removed upon disconnection,
mark all its devices unhealthy so that we can signal the resource allocatable
change to the scheduler to avoid scheduling more pods to the node.
When a device plugin endpoint is in stopGracePeriod, pods requesting the
corresponding resource will fail admission handler.
Tested:
Ran GPUDevicePlugin e2e_node test 100 times and all passed now.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubernetes/issues/60176
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixes the races around devicemanager Allocate() and endpoint deletion.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Task 2: Schedule DaemonSet Pods by default scheduler.
Signed-off-by: Da K. Ma <klaus1982.cn@gmail.com>
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
part of #59194https://github.com/kubernetes/features/issues/548
**Release note**:
```release-note
When ScheduleDaemonSetPods is enabled, the DaemonSet controller will delegate Pods scheduling to default scheduler.
```
There is a race in predicateAdmitHandler Admit() that getNodeAnyWayFunc()
could get Node with non-zero deviceplugin resource allocatable for a
non-existing endpoint. That race can happen when a device plugin fails,
but is more likely when kubelet restarts as with the current registration
model, there is a time gap between kubelet restart and device plugin
re-registration. During this time window, even though devicemanager could
have removed the resource initially during GetCapacity() call, Kubelet
may overwrite the device plugin resource capacity/allocatable with the
old value when node update from the API server comes in later. This
could cause a pod to be started without proper device runtime config set.
To solve this problem, introduce endpointStopGracePeriod. When a device
plugin fails, don't immediately remove the endpoint but set stopTime in
its endpoint. During kubelet restart, create endpoints with stopTime set
for any checkpointed registered resource. The endpoint is considered to be
in stopGracePeriod if its stoptime is set. This allows us to track what
resources should be handled by devicemanager during the time gap.
When an endpoint's stopGracePeriod expires, we remove the endpoint and
its resource. This allows the resource to be exported through other channels
(e.g., by directly updating node status through API server) if there is such
use case. Currently endpointStopGracePeriod is set as 5 minutes.
Given that an endpoint is no longer immediately removed upon disconnection,
mark all its devices unhealthy so that we can signal the resource allocatable
change to the scheduler to avoid scheduling more pods to the node.
When a device plugin endpoint is in stopGracePeriod, pods requesting the
corresponding resource will fail admission handler.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Change regional PD cloud provider references to use the beta API
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59988
**Special notes for your reviewer**: Depends on a version of the GCP Go beta compute client that is not yet available. Also need to rebase with #60337 once it's merged.
/hold
/cc @abgworrall
/assign @saad-ali
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
update Mount propagation version in comment
**What this PR does / why we need it**:
Mount propagation feature was moved to beta in PR [#59252](https://github.com/kubernetes/kubernetes/pull/59252)， so update the comment.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60657
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add support for binaries to run as Windows services
**What this PR does / why we need it**:
Add support for binaries to run as Windows services
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59562
**Special notes for your reviewer**:
**Release note**:
```release-note
kubelet and kube-proxy can now be ran as Windows services
```
This patch adds support for kubernetes to integrate
with Windows SCM.
As a first step both `kubelet` and `kube-proxy` can be registered as a service.
To create the service:
PS > sc.exe create <component_name> binPath= "<path_to_binary> --service <other_args>"
CMD > sc create <component_name> binPath= "<path_to_binary> --service <other_args>"
Please note that if the arguments contain spaces, it must be escaped.
Example:
PS > sc.exe create kubelet binPath= "C:\kubelet.exe --service --hostname-override 'minion' <other_args>"
CMD > sc create kubelet binPath= "C:\kubelet.exe --service --hostname-override 'minion' <other_args>"
Example to start the service:
PS > Start-Service kubelet; Start-Service kube-proxy
CMD > net start kubelet && net start kube-proxy
Example to stop the service:
PS > Stop-Service kubelet (-Force); Stop-Service kube-proxy (-Force)
CMD > net stop kubelet && net stop kube-proxy
Example to query the service:
PS > Get-Service kubelet; Get-Service kube-proxy;
CMD > sc.exe queryex kubelet && sc qc kubelet && sc.exe queryex kube-proxy && sc.exe qc kube-proxy
Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com>
Signed-off-by: Alin Gabriel Serdean <aserdean@ovn.org>
Co-authored-by: Alin Gabriel Serdean <aserdean@ovn.org>
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove "scale job" from help info
Remove "scale job" from help info since it's deprecated
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @deads2k @soltysh
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Volume deletion should be idempotent
- Describe* calls should return `aws.Error` so caller can handle individual errors. `aws.Error` already has enough context (`"InvalidVolume.NotFound: The volume 'vol-0a06cc096e989c5a2' does not exist"`)
- Deletion of already deleted volume should succeed.
**Release note**:
Fixes: #60778
```release-note
NONE
```
/sig storage
/sig aws
/assign @justinsb @gnufied
Automatic merge from submit-queue (batch tested with PRs 60159, 60731, 60720, 60736, 60740). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Promote LocalStorageCapacityIsolation feature to beta
The LocalStorageCapacityIsolation feature added a new resource type ResourceEphemeralStorage "ephemeral-storage" so that this resource can be allocated, limited, and consumed as the same way as CPU/memory. All the features related to resource management (resource request/limit, quota, limitrange) are available for local ephemeral storage.
This local ephemeral storage represents the storage for root file system, which will be consumed by containers' writtable layer and logs. Some volumes such as emptyDir might also consume this storage.
Fixes issue #60160
This PR also fixes data race issues discovered after open the feature gate. Basically setNodeStatus function in kubelet could be called by multiple threads so the data needs lock protection. Put the fix with this PR for easy testing.
**Release note**:
```release-note
ACTION REQUIRED: LocalStorageCapacityIsolation feature is beta and enabled by default.
```
The LocalStorageCapacityIsolation feature added a new resource type
ResourceEphemeralStorage "ephemeral-storage" so that this resource can
be allocated, limited, and consumed as the same way as CPU/memory. All
the features related to resource management (resource request/limit, quota, limitrange) are avaiable for local ephemeral storage.
This local ephemeral storage represents the storage for root file system, which will be consumed by containers' writtable layer and logs. Some volumes such as emptyDir might also consume this storage.
Automatic merge from submit-queue (batch tested with PRs 60732, 60689, 60648, 60704). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix a grammatical error in a comment
Fix a grammatical error in a comment in scheduler's code. We should use a word's plural form after "one of".
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60732, 60689, 60648, 60704). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Do not count failed pods as unready in HPA controller
**What this PR does / why we need it**:
Currently, when performing a scale up, any failed pods (which can be present for example in case of evictions performed by kubelet) will be treated as unready. Unready pods are treated as if they had 0% utilization which will slow down or even block scale up.
After this change, failed pods are ignored in all calculations. This way they do not influence neither scale up nor scale down replica calculations.
@MaciekPytel @DirectXMan12
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#55630
**Special notes for your reviewer**:
**Release note**:
```
Stop counting failed pods as unready in HPA controller to avoid failed pods incorrectly affecting scale up replica count calculation.
```
Automatic merge from submit-queue (batch tested with PRs 60683, 60386). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Added unschedulabe predicate.
Signed-off-by: Da K. Ma <madaxa@cn.ibm.com>
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60163
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 60623, 60625, 60520). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix azure file plugin failure issue on Windows after node restart
**What this PR does / why we need it**:
azure file plugin on Windows does not work after node restart, this is due to New-SmbGlobalMapping powershell cmdlet has lost account name/key after reboot, we should remove the invalid link and do the mount again after kubelet restart.
add remount logic for azure file plugin in this PR
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60624
**Special notes for your reviewer**:
**Release note**:
```
fix azure file plugin failure issue on Windows after node restart
```
/sig azure
/sig windows
/assign @karataliu
@feiskyer pls mark this PR as v1.10 milestone, thanks
Automatic merge from submit-queue (batch tested with PRs 60623, 60625, 60520). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Set default vmtype to standard if not set
**What this PR does / why we need it**:
When using IMDS, kubelet reports Failed to extract vmssVMName even when vmtype is not set:
```
E0301 05:30:57.515306 3124 azure_vmss_cache.go:52] Failed to extract vmssVMName "77890k8s9010"
E0301 05:30:57.515306 3124 kubelet_node_status.go:79] Unable to construct v1.Node object for kubelet: failed to get external ID from cloud provider: not a vmss instance
```
azure cloud provider shouldn't call vmss logic in this case. It should assume stardard vmtype if it is not set.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60622
**Special notes for your reviewer**:
**Release note**:
```release-note
Set Azure vmType to standard if it is not set in azure cloud config.
```
Automatic merge from submit-queue (batch tested with PRs 60362, 60629, 60572). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
improve get description
ref: #60210
/assign @deads2k
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60362, 60629, 60572). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix warning info format
Before this change:
```
$ kubectl exec -p nginx-aaaaaaaaaaaaa-67b58bf687-r68b9 ls
-p POD_NAME is DEPRECATED and will be removed in a future version. Use exec POD_NAME instead.bin
boot
dev
...
$ kubectl scale --replicas=2 job/pi
kubectl scale job is DEPRECATED and will be removed in a future version.job.batch "pi" scaled
```
After this change:
```
$ kubectl exec -p nginx-aaaaaaaaaaaaa-67b58bf687-r68b9 ls
-p POD_NAME is DEPRECATED and will be removed in a future version. Use exec POD_NAME instead.
bin
boot
dev
...
$ kubectl scale --replicas=2 job/pi
kubectl scale job is DEPRECATED and will be removed in a future version.
job.batch "pi" scaled
```
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60362, 60629, 60572). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add TestUpdateStatus for horizontalpodautoscaler
**What this PR does / why we need it**:
add `TestUpdateStatus` for horizontalpodautoscaler， and remove the relevant `TODO TestUpdateStatus
`
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Currently, when performing a scale up, any failed pods (which can be present for example in case of evictions performed by kubelet) will be treated as unready. Unready pods are treated as if they had 0% utilization which will slow down or even block scale up.
After this change, failed pods are ignored in all calculations. This way they do not influence neither scale up nor scale down replica calculations.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add missing table converters for server side printing
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60488
/assign @smarterclayton @juanvallejo
/kind bug
**Release note**:
```release-note
None
```
Before this change:
```
$ kubectl exec -p nginx-aaaaaaaaaaaaa-67b58bf687-r68b9 ls
-p POD_NAME is DEPRECATED and will be removed in a future version. Use exec POD_NAME instead.bin
boot
dev
...
$ kubectl scale --replicas=2 job/pi
kubectl scale job is DEPRECATED and will be removed in a future version.job.batch "pi" scaled
```
After this change:
```
$ kubectl exec -p nginx-aaaaaaaaaaaaa-67b58bf687-r68b9 ls
-p POD_NAME is DEPRECATED and will be removed in a future version. Use exec POD_NAME instead.
bin
boot
dev
...
$ kubectl scale --replicas=2 job/pi
kubectl scale job is DEPRECATED and will be removed in a future version.
job.batch "pi" scaled
```
Automatic merge from submit-queue (batch tested with PRs 58171, 58036, 60540). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Enable IPVS feature gateway by default since it's already beta
**What this PR does / why we need it**:
Per #60501, kubeadm init doesn't support the --feature-gates=SupportIPVSProxyMode=true flag anymore since IPVS is in beta and not alpha anymore. We should enable IPVS feature gateway by default to make kubeadm set up IPVS proxy properly.
**Which issue(s) this PR fixes**:
Fixes#60543
**Special notes for your reviewer**:
**Release note**:
```release-note
Enable IPVS feature gateway by default
```
Kubernetes Submit Queue