Automatic merge from submit-queue (batch tested with PRs 67709, 67556). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Fix volume scheduling issue with pod affinity and anti-affinity
**What this PR does / why we need it**:
The previous design of the volume scheduler had volume assume + bind done before pod assume + bind. This causes issues when trying to evaluate future pods with pod affinity/anti-affinity because the pod has not been assumed while the volumes have been decided.
This PR changes the design so that volume and pod are assumed first, followed by volume and pod binding. Volume binding waits (asynchronously) for the operations to complete or error. This eliminates the subsequent passes through the scheduler to wait for volume binding to complete (although pod events or resyncs may still cause the pod to run through scheduling while binding is still in progress). This design also aligns better with the scheduler framework design, so will make it easier to migrate in the future.
Many changes had to be made in the volume scheduler to handle this new design, mostly around:
* How we cache pending binding operations. Now, any delayed binding PVC that is not fully bound must have a cached binding operation. This also means bind API updates may be repeated.
* Waiting for the bind operation to fully complete, and detecting failure conditions to abort the bind and retry scheduling.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#65131
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixes issue where pod scheduling may fail when using local PVs and pod affinity and anti-affinity without the default StatefulSet OrderedReady pod management policy
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Kubelet: only apply default hard evictions of nodefs.inodesFree on Linux
**What this PR does / why we need it**:
Kubelet sets default hard evictions of `nodefs.inodesFree ` for all platforms today. This will cause errors on Windows and a lot `no observation found for eviction signal nodefs.inodesFree` errors will be logs for kubelet.
```
kubelet.err.log:4961:W0711 22:21:12.378789 2872 helpers.go:808] eviction manager: no observation found for eviction signal nodefs.inodesFree
kubelet.err.log:4967:W0711 22:21:30.411371 2872 helpers.go:808] eviction manager: no observation found for eviction signal nodefs.inodesFree
kubelet.err.log:4974:W0711 22:21:48.446456 2872 helpers.go:808] eviction manager: no observation found for eviction signal nodefs.inodesFree
kubelet.err.log:4978:W0711 22:22:06.482441 2872 helpers.go:808] eviction manager: no observation found for eviction signal nodefs.inodesFree
```
This PR updates the default hard eviction value and only apply nodefs.inodesFree on Linux.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#66088
**Special notes for your reviewer**:
**Release note**:
```release-note
Kubelet only applies default hard evictions of nodefs.inodesFree on Linux
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Add kubelet stats for windows system container "pods"
**What this PR does / why we need it**:
This PR adds kubelet stats for windows system container "pods". Without this, kubelet will always logs error:
```
kubelet.err.log:4832:E0711 22:12:49.241358 2872 helpers.go:735] eviction manager: failed to construct signal: "allocatableMemory.available" error: system container "pods" not found
```
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#66087
**Special notes for your reviewer**:
/sig windows
/sig node
**Release note**:
```release-note
Add kubelet stats for windows system container "pods"
```
Automatic merge from submit-queue (batch tested with PRs 66840, 68159). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
CSI Cluster Registry and Node Info CRDs Improvements
**What this PR does / why we need it**:
https://github.com/kubernetes/kubernetes/pull/67803 merged before I could address @lavalamp's feedback. This PR addresses his feedback
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Follow up on PR https://github.com/kubernetes/kubernetes/pull/67803
**Special notes for your reviewer**:
**Release note**:
```release-note
```
/assign @lavalamp
/assign @thockin
CC @jsafrane @vladimirvivien @verult @gnufied @childsb
Automatic merge from submit-queue (batch tested with PRs 66840, 68159). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
TTL for cleaning up Jobs after they finish
**What this PR does / why we need it**: https://github.com/kubernetes/features/issues/592
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64470
For https://github.com/kubernetes/features/issues/592
**Special notes for your reviewer**: @kubernetes/sig-apps-pr-reviews
**Release note**:
```release-note
Add a TTL machenism to clean up Jobs after they finish.
```
* FindPodVolumes
* Prebound PVCs are treated like unbound immediate PVCs and will error
* Always check for fully bound PVCs and cache bindings for not fully
bound PVCs
* BindPodVolumes
* Retry API updates for not fully bound PVCs even if the assume cache
already marked it
* Wait for PVCs to be fully bound after making the API updates
* Error when detecting binding/provisioning failure conditions
1. If TTLAfterFinished feature is enabled, the value should be non-negative.
2. If TTLAfterFinished feature is disabled, the field value should not
be kept.
Automatic merge from submit-queue (batch tested with PRs 67736, 68123, 68138). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Added support to get clusters in gce cloud provider.
**What this PR does / why we need it**:
Implemented the call to get all cluster objects in a zone for a project.
Also added code to allow the container api to be set in the gce.conf
file.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 67555, 68196). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Not split nodes when searching for nodes but doing it all at once
**What this PR does / why we need it**:
Not split nodes when searching for nodes but doing it all at once.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
@bsalamat
This is a follow up PR of #66733.
https://github.com/kubernetes/kubernetes/pull/66733#discussion_r205932531
**Release note**:
```release-note
Not split nodes when searching for nodes but doing it all at once.
```
Automatic merge from submit-queue (batch tested with PRs 63011, 68089, 67944, 68132). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Start deprecating all generators in run except for run-pod/v1
**What this PR does / why we need it**:
This was discussed during SIG-CLI meetings over several past months. The direction is that we want to move away from `kubectl run` because it's over bloated and complicated for both users and developers. We want to mimic `docker run` with `kubectl run` so that it *only* creates a pod, and if you're interested in other resources `kubectl create` is the intended replacement.
This PR starts with deprecating all of the generator except for the pod one.
/assign @juanvallejo
/sig cli
/milestone v1.12
**Release note**:
```release-note
Deprecate kubectl run generators, except for run-pod/v1
```
Automatic merge from submit-queue (batch tested with PRs 63011, 68089, 67944, 68132). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Support both directory and block device for local volume plugin FileSystem VolumeMode
Support both directory and block device for local volume plugin FileSystem VolumeMode
xref: [local storage dynamic provisioning design #1914](https://github.com/kubernetes/community/pull/1914)
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Support both directory and block device for local volume plugin FileSystem VolumeMode
```
Automatic merge from submit-queue (batch tested with PRs 65074, 67469). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Move kube-controller-manager ComponentConfig external types to `k8s.io/kube-controller-manager`
**What this PR does / why we need it**:
As the title describe:
split `kube-controller-manager` component api into their own packages:
- external component api located in `k8s.io/kube-controller-manager/config/v1alpha/types.go`;
- internal component api located in `pkg/controller/apis/config/types.go`
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
ref: [kubernetes/community#2354](https://github.com/kubernetes/community/pull/2354)
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 65074, 67469). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Add CSI volume attributes for kubectl describe pv
**What this PR does / why we need it**:
/king feature
kubectl describe pv doesn't show the volume attributes.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#65021
**Special notes for your reviewer**:
/cc @saad-ali
**Release note**:
```release-note
Add CSI volume attributes for kubectl describe pv.
```
Automatic merge from submit-queue (batch tested with PRs 67986, 68210, 67817). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Fix waiting in kubectl rollout status
**What this PR does / why we need it**:
By `kubectl rollout status` being based on `UntilWithoutRetry` it will fail whenever the watcher is closed - cased by the closing the underlying connection - e.g. on API timeout, LB timeout, ...
Using UntilWithSync (based on informer) allows it to recover from all the failures and truly work trough unlimited timeouts.
(Split from https://github.com/kubernetes/kubernetes/pull/50102)
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubernetes/issues/40224
**Special notes for your reviewer**:
**Release note**:
```release-note
`kubectl rollout status` now works for unlimited timeouts.
```
Automatic merge from submit-queue (batch tested with PRs 67986, 68210, 67817). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Fix panic when processing http response
**What this PR does / why we need it**:
When Azure ARM API gets something wrong, kube-controller-manager may panic because of azure cloud provider:
```
/usr/local/go/src/runtime/asm_amd64.s:2361
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1d4cad9]
goroutine 1386 [running]:
k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:58 +0x107
panic(0x44468c0, 0x8b76a30)
/usr/local/go/src/runtime/panic.go:502 +0x229
k8s.io/kubernetes/pkg/cloudprovider/providers/azure.processHTTPRetryResponse(0x0, 0x64ffec0, 0xc4229fd1f0, 0xc422ed05b0, 0x2, 0x2)
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/providers/azure/azure_backoff.go:364 +0x69
k8s.io/kubernetes/pkg/cloudprovider/providers/azure.(*Cloud).CreateOrUpdatePIPWithRetry.func1(0xc422ed0600, 0x0, 0x0)
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/providers/azure/azure_backoff.go:205 +0x298
```
This PR fixes that.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#68209
**Special notes for your reviewer**:
Should cherry pick to old releases.
**Release note**:
```release-note
Fix panic when processing Azure HTTP response.
```
Automatic merge from submit-queue (batch tested with PRs 67986, 68210, 67817). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
add mixed protocol support for azure load balancer
**What this PR does / why we need it**:
If user specify `service.beta.kubernetes.io/azure-load-balancer-mixed-protocols: "true"`, azure cloud provider will create both TCP and UDP lb rules, for more details, could refer to https://github.com/kubernetes/kubernetes/issues/66887
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#66887
**Special notes for your reviewer**:
original `reconcileLoadBalancer` func is too big, I move part of code implementation to a standalone func `createLoadBalancerRule`
example service config:
```
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/azure-load-balancer-mixed-protocols: "true"
name: web
namespace: default
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: web
sessionAffinity: None
type: LoadBalancer
```
**Release note**:
```
add mixed protocol support for azure load balancer
```
/kind feature
/sig azure
/assign @feiskyer @khenidak
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Replace scale down window
**What this PR does / why we need it**:
Replace scale down forbidden window with scale down stabilization window.
This allows scale down based on more than one sample, to avoid rapidly changing size up and down for controllers with fluctuating load.
A bit more in https://docs.google.com/document/d/1IdG3sqgCEaRV3urPLA29IDudCufD89RYCohfBPNeWIM
This PR is copy of #67771 with resolved comments.
**Release note**:
```release-note
Replace scale down forbidden window with scale down stabilization window. Rather than waiting a fixed period of time between scale downs HPA now scales down to the highest recommendation it during the scale down stabilization window.
```
Automatic merge from submit-queue (batch tested with PRs 68135, 68188). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Fixup openstack cloud provider loadbalancer deletion error
This change enables ```getLoadBalancer``` to return the loadbalancer even if no floating ip is associated to the VIP port of the loadbalancer.
Signed-off-by: Eunsoo Park <esevan.park@gmail.com>
**What this PR does / why we need it**:
This PR fixes the bug like below.
```
Warning CreatingLoadBalancerFailed 17m (x3445 over 12d) service-controller Error creating load balancer (will retry): error getting LB for service default/influxdb: error getting floating ip for port 81253cae-acd6-4bed-8006-814c8729be8c: failed to find object
```
**Special notes for your reviewer**:
Refer to following bug description for better understanding of this PR.
When k8s service type is changed from LoadBalancer to others, there's a deleting loadbalancer process in kubernetes cloud provider as below.
https://github.com/kubernetes/kubernetes/blob/master/pkg/controller/service/service_controller.go#L284
```
if !wantsLoadBalancer(service) {
_, exists, err := s.balancer.GetLoadBalancer(context.TODO(), s.clusterName, service)
if err != nil {
return fmt.Errorf("error getting LB for service %s: %v", key, err)
}
if exists {
glog.Infof("Deleting existing load balancer for service %s that no longer needs a loadbalancer.", key)
s.eventRecorder.Event(service, v1.EventTypeNormal, "DeletingLoadBalancer", "Deleting loadbalancer")
if err := s.balancer.EnsureLoadBalancerDeleted(context.TODO(), s.clusterName, service); err != nil {
return err
}
s.eventRecorder.Event(service, v1.EventTypeNormal, "DeletedLoadBalancer", "Deleted loadbalancer")
}
```
Openstack cloud provider returns error even though LB exists since there's no associated floating IP with VIP port.
https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/openstack/openstack_loadbalancer.go#L476
```
floatIP, err := getFloatingIPByPortID(lbaas.network, portID)
if err != nil {
return nil, false, fmt.Errorf("error getting floating ip for port %s: %v", portID, err)
}
```
This caused ```GetLoadBalancer()``` failed and retrying over and over in ```processServiceUpdate``` of ```k8sServiceController```.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Add validation for kube-scheduler configuration options
**What this PR does / why we need it**: This adds validation to the kube-scheduler so that we're not accepting bogus values to the kube-scheduler. As requested by @bsalamat in issue https://github.com/kubernetes/kubernetes/issues/66743
**Which issue(s) this PR fixes**:
Fixes#66743
**Special notes for your reviewer**:
- Not sure if this validation is too heavy handed. Would love some feedback.
- I started working on this before I realized @islinwb was also working on this same problem... https://github.com/kubernetes/kubernetes/pull/66787 I put this PR up anyways since I'm sure good code exists in both. I wasn't aware of the /assign command so didn't assign myself before starting work.
- I didn't have time to work on adding validation to deprecated cli options. If the rest of this looks ok, I can finish that up.
- I hope the location of IsValidSocketAddr is correct. Lmk if it isn't.
**Release note**:
```release-note
Adding validation to kube-scheduler at the API level
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Affinity/Anti-Affinity Optimization of Pod Being Scheduled
**What this PR does / why we need it**:
Following #66948, it was noticed that the applied optimizations for anti-affinity rules lookup of existing pods could be further applied to checking affinity and anti-affinity terms of the Pod being scheduled. This is done by mapping topology pairs to pods that potentially match the pod being scheduled instead of mapping nodes to matching pods, and accordingly the search space is reduced.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#67738
**Special notes for your reviewer**:
/sig scheduling
/sig scalability
**Release note**:
```release-note
Improve performance of Pod affinity/anti-affinity in the scheduler
```
Automatic merge from submit-queue (batch tested with PRs 67397, 68019). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Fix conversion for autoscaling/v1 ObjectMetricSource and add fuzzer
**What this PR does / why we need it**:
Selectors in ObjectMetricSource's weren't being persisted through roundtrip conversions, and this wasn't caught because we had no fuzzer testing MetricIdentifier selectors
**Which issue(s) this PR fixes**:
none
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Increase Horizontal Pod Autoscaler update frequency to every 15s
**What this PR does / why we need it**:
PR increases Horizontal Pod Autoscaler default update interval (30s -> 15s). It will improve HPA reaction time for metric changes.
**Release note**:
```release-note
Increase Horizontal Pod Autoscaler default update interval (30s -> 15s). It will improve HPA reaction time for metric changes.
```
This change enables ```getLoadBalancer``` to return the loadbalancer even if no floating ip is associated to the VIP port of the loadbalancer.
Signed-off-by: Eunsoo Park <esevan.park@gmail.com>
Kubernetes Submit Queue