Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Fix gce localssd pv tests
**What this PR does / why we need it**:
When running local PV tests against GCE local SSD, it directly uses the disk so doesn't need to create a tmp dir like the other test formats. Fsgroup tests do not create test-file so don't error on cleanup if the file doesn't exist.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#68308
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Kubelet: only sync iptables on linux
**What this PR does / why we need it**:
Iptables is only supported on Linux, kubelet should only sync NAT rules on Linux.
Without this PR, Kubelet on Windows would logs following errors on each `syncNetworkUtil()`:
```
kubelet.err.log:4692:E0711 22:03:42.103939 2872 kubelet_network.go:102] Failed to ensure that nat chain KUBE-MARK-DROP exists: error creating chain "KUBE-MARK-DROP": executable file
```
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#65713
**Special notes for your reviewer**:
**Release note**:
```release-note
Kubelet now only sync iptables on Linux.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Update bazel deps
**What this PR does / why we need it**: updates relevant bazel dependencies to pull in minor feature enhancements and bug fixes:
* rules_go 0.15.3: some speed improvements and better compatibility on Windows and Mac, among other bugfixes / enhancements
* repo-infra: fixes needed to support rules_go 0.15.1, and also a fix for https://github.com/kubernetes/kubernetes/pull/65501#issuecomment-400761696
* rules_docker v0.5.1: various enhancements/fixes, but we need this to support new manifest lists from upstream
* busybox latest: we last updated this a while ago. since Docker hub now uses manifest lists for core images like this, we needed the rules_docker update
* gazelle 0.14.0: among other enhancements, adds a feature to remove the need to build protoc for every build, which we aren't even using.
**Special notes for your reviewer**:
Depends on https://github.com/kubernetes/repo-infra/pull/81. I believe we also need bazel 0.16+.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 68171, 67945, 68233). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Move the CloudControllerManagerConfiguration to an API group in `cmd/`
**What this PR does / why we need it**:
This PR is the last piece of https://github.com/kubernetes/kubernetes/issues/67233.
It moves the `CloudControllerManagerConfiguration` to its own `cloudcontrollermanager.config.k8s.io` config API group, but unlike the other components this API group is "private" (only available in `k8s.io/kubernetes`, which limits consumer base), as it's located entirely in `cmd/` vs a staging repo.
This decision was made for now as we're not sure what the story for the ccm loading ComponentConfig files is, and probably a "real" file-loading ccm will never exist in core, only helper libraries. Eventually the ccm will only be a library in any case, and implementors will/can use the base types the ccm library API group provides. It's probably good to note that there is no practical implication of this change as the ccm **cannot** read ComponentConfig files. Hencec the code move isn't user-facing.
With this change, we're able to remove `pkg/apis/componentconfig`, as this was the last consumer. That is hence done in this PR as well (so the move is easily visible in git, vs first one "big add" then a "big remove"). The only piece of code that was used was the flag helper structs, so I moved them to `pkg/util/flag` that I think makes sense for now.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
ref: kubernetes/community#2354
**Special notes for your reviewer**:
This PR builds on top of (first two commits, marked as `Co-authored by: @stewart-yu`) https://github.com/kubernetes/kubernetes/pull/67689
**Release note**:
```release-note
NONE
```
/assign @liggitt @sttts @thockin @stewart-yu
Automatic merge from submit-queue (batch tested with PRs 68171, 67945, 68233). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
CSI pod information in NodePublish
**What this PR does / why we need it**:
This is implementation of https://github.com/kubernetes/community/pull/2439. It brings CSI closer to Flex volume features, CSI driver can (optionally) get name/namespace/UID/ServiceAccount of the pod that requires the CSI volume mounted. This allows CSI drivers to either do AAA before the volume is mounted or tailor content of volume to the pod.
Work in progress:
* contains #67803 to get `CSIDriver` API. Ignore the first commit.
Related to #64984#66362 (fixes only part of these issues)
/sig storage
cc: @saad-ali @vladimirvivien @verult @msau42 @gnufied
**Release note**:
```release-note
CSI NodePublish call can optionally contain information about the pod that requested the CSI volume.
```
Automatic merge from submit-queue (batch tested with PRs 68171, 67945, 68233). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
[e2e] verifying LimitRange update is effective before creating new pod
**What this PR does / why we need it**:
Refer to the flaky test mentioned in #68170, LimitRange updating should be verified before creating new pod.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#68170
**Special notes for your reviewer**:
/cc bsalamat k82cn
/sig scheduling
**Release note**:
```release-note
[e2e] verifying LimitRange update is effective before creating new pod
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Fix an issue about generation of secret key with invalid key size in gce
Replace /dev/random to /dev/urandom to avoid generation of secret key with invalid key size.
**What this PR does / why we need it**:
In GCE env, AESGCM encryption of secrets by default generates a secret key with /dev/random which sometime generates a key with invalid size.
This cause the cluster/kube-up.sh to fail in gce environment.
This PR replaces /dev/random with /dev/urandom to have a secret key generated consistently with right size.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#67091
**Special notes for your reviewer**:
**Release note**:
```release-note
cluster/gce: generate consistent key sizes in config-default.sh using /dev/urandom instead of /dev/random
```
Automatic merge from submit-queue (batch tested with PRs 68161, 68023, 67909, 67955, 67731). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
CSI: skip attach for non-attachable drivers
**What this PR does / why we need it**:
This is implementation of https://github.com/kubernetes/community/pull/2523. CSI volumes that don't need attach/detach now don't need external attacher running.
WIP:
* contains #67803 to get CSIDriver API. Ignore the first commit.
* ~~missing e2e test~~
/sig storage
cc: @saad-ali @vladimirvivien @verult @msau42 @gnufied @davidz627
**Release note**:
```release-note
CSI volume plugin does not need external attacher for non-attachable CSI volumes.
```
Automatic merge from submit-queue (batch tested with PRs 68161, 68023, 67909, 67955, 67731). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Replace git volume with configmap in emptydir wrapper conflict test
**What this PR does / why we need it**: GitRepoVolumeSource is deprecated, use a ConfigMap instead. (This test is part of the conformance suite, so it would be good to allow downstreams to disable/not support gitRepo)
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 68161, 68023, 67909, 67955, 67731). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Register RuntimeClass CRD as an addon
**What this PR does / why we need it**:
Register the RuntimeClass CRD when the RuntimeClass feature gate is enabled. This is done in through the addon manager.
This is an alternative approach to https://github.com/kubernetes/kubernetes/pull/67924
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
For https://github.com/kubernetes/features/issues/585
**Release note**:
Covered by #67737
```release-note
NONE
```
/sig node
/kind feature
/priority important-soon
/milestone v1.12
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
fix typo in ipvs RADME.md
**What this PR does / why we need it**:
fix typos in ipvs README.md
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
no
**Release note**:
```release-note
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Implement semantic comparison of VolumeNodeAffinity for unit tests
**What this PR does / why we need it**:
Implements a semantic comparison function for VolumeNodeAffinity that is not sensitive to ordering of various members. Previous reflect.DeepEqual was sensitive to ordering causing it to be flaky.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes # https://github.com/kubernetes/kubernetes/issues/67852
**Special notes for your reviewer**:
We want to able to successfully match `VolumeNodeAffinity{Required:&NodeSelector{NodeSelectorTerms:[{[{a In [1]} {b In [2 3]}] []}],},}` and `VolumeNodeAffinity{Required:&NodeSelector{NodeSelectorTerms:[{[{b In [3 2]} {a In [1]}] []}],},}` without being sensitive to the ordering of requirements with key `a` and `b` or the order of values with key `b`. This fix enables such semantic comparison of VolumeNodeAffinity
We can move `volumeNodeAffinitiesEqual` to volume/utils post code freeze
**Release note**:
```release-note
NONE
```
/sig storage
cc @msau42
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Add yue9944882 as subproject reviewer (core admission/apiserver)
extend reviewer bandwidth 😃am a super careful reviewer
i had contributed series of refactors for core admission controllers and apiserver launch flow.
/assign @deads2k
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Deprecation notice of the storage-versions flag
This PR deprecates the `--storage-versions` flag of kube-apiserver. The flag allows user to change the API version the content in etcd will be serialized to.
The flag provides unnecessary flexibility, with the side-effect of making kube-apiserver upgrades/downgrades hard to reason about. Specifically, Kubernetes follows the 4-step API version deprecation [policy](https://kubernetes.io/docs/reference/using-api/deprecation-policy/), which makes upgrades/downgrades across minor versions safe as long as all the data in etcd is encoded to the default storage versions. However, if users can specify their own storage versions, then the safety of each upgrade/downgrade needs to be analyzed case-by-case.
```release-note
Action required: The --storage-versions flag of kube-apiserver is deprecated. Please omit this flag to ensure the default storage versions are used. Otherwise the cluster is not safe to upgrade to a version newer than 1.12. This flag will be removed in 1.13.
```
/assign @deads2k @liggitt @lavalamp @enj
/sig api-machinery
Automatic merge from submit-queue (batch tested with PRs 67691, 68147). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Update etcd client to 3.2.24 for latest release
**What this PR does / why we need it**:
Updates etcd client to 3.2.24 which is the latest in the 3.2 series. See https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.2.md for details.
**Special notes for your reviewer**:
This is only the client, in order to update the server components it requires a googler to push the 3.2.24 image.
**Release note**:
```
Update etcd client interface to 3.2.24
```
/assign @jpbetz @wojtek-t
/cc @liggitt @kubernetes/sig-cluster-lifecycle @kubernetes/sig-scalability-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 67691, 68147). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Bump versions of components with latest security patches.
**What this PR does / why we need it**:
Upgrade versions of monitoring components used on GCP, to include latest security patches.
**Release note**:
```release-note
[fluentd-gcp-scaler addon] Bump fluentd-gcp-scaler to 0.4 to pick up security fixes.
[prometheus-to-sd addon] Bump prometheus-to-sd to 0.3.1 to pick up security fixes, bug fixes and new features.
[event-exporter addon] Bump event-exporter to 0.2.3 to pick up security fixes.
```
Automatic merge from submit-queue (batch tested with PRs 68265, 68273). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
apiserver: make InClusterConfig errs for delegated authn/z non-fatal
Fixes https://github.com/kubernetes/kubernetes/issues/68246:
Background:
In gci e2e tests the kube-controller-manager is started as static pod. When it first starts, there is no kubernetes service and the KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT env vars are not set inside the container.
When the kube-controller-manager is restarted, the KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT env vars are set, but the static pod has no service account, i.e. /var/run/secrets/kubernetes.io/serviceaccount/token does not exist. We made the later fatal in rest.InClusterConfig and its use to setup delegated authn/z.
Automatic merge from submit-queue (batch tested with PRs 68265, 68273). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
gke: fix failing e2e tests
```
/home/kubernetes/bin/configure-helper.sh: line 2032: ENABLE_NODE_TERMINATION_HANDLER: unbound variable
```
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 67709, 67556). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Fix volume scheduling issue with pod affinity and anti-affinity
**What this PR does / why we need it**:
The previous design of the volume scheduler had volume assume + bind done before pod assume + bind. This causes issues when trying to evaluate future pods with pod affinity/anti-affinity because the pod has not been assumed while the volumes have been decided.
This PR changes the design so that volume and pod are assumed first, followed by volume and pod binding. Volume binding waits (asynchronously) for the operations to complete or error. This eliminates the subsequent passes through the scheduler to wait for volume binding to complete (although pod events or resyncs may still cause the pod to run through scheduling while binding is still in progress). This design also aligns better with the scheduler framework design, so will make it easier to migrate in the future.
Many changes had to be made in the volume scheduler to handle this new design, mostly around:
* How we cache pending binding operations. Now, any delayed binding PVC that is not fully bound must have a cached binding operation. This also means bind API updates may be repeated.
* Waiting for the bind operation to fully complete, and detecting failure conditions to abort the bind and retry scheduling.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#65131
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixes issue where pod scheduling may fail when using local PVs and pod affinity and anti-affinity without the default StatefulSet OrderedReady pod management policy
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Kubelet: only apply default hard evictions of nodefs.inodesFree on Linux
**What this PR does / why we need it**:
Kubelet sets default hard evictions of `nodefs.inodesFree ` for all platforms today. This will cause errors on Windows and a lot `no observation found for eviction signal nodefs.inodesFree` errors will be logs for kubelet.
```
kubelet.err.log:4961:W0711 22:21:12.378789 2872 helpers.go:808] eviction manager: no observation found for eviction signal nodefs.inodesFree
kubelet.err.log:4967:W0711 22:21:30.411371 2872 helpers.go:808] eviction manager: no observation found for eviction signal nodefs.inodesFree
kubelet.err.log:4974:W0711 22:21:48.446456 2872 helpers.go:808] eviction manager: no observation found for eviction signal nodefs.inodesFree
kubelet.err.log:4978:W0711 22:22:06.482441 2872 helpers.go:808] eviction manager: no observation found for eviction signal nodefs.inodesFree
```
This PR updates the default hard eviction value and only apply nodefs.inodesFree on Linux.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#66088
**Special notes for your reviewer**:
**Release note**:
```release-note
Kubelet only applies default hard evictions of nodefs.inodesFree on Linux
```
Lars Ekman