Automatic merge from submit-queue (batch tested with PRs 65449, 65373, 49410). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add kernel config locations for fedora and atomic
**What this PR does / why we need it**:
* Fedora stores its kernel configs in /usr/lib/modules/$(uname -r)/config
* Fedora/CentOS/RHEL atomic hosts use /usr/lib/ostree-boot/$(uname -r), though this location is deprecated
* The lack of these locations in the validator is causing kubeadm to hang on "failed to parse kernel config" in its preflight checking on fedora and atomic host
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 65449, 65373, 49410). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
OpenStack LBaaS fix: must use ID, not name, of the node security group
This is a bugfix for the OpenStack LBaaS cloud provider security group management.
A bit of context: When creating a load balancer for a given `type: LoadBalancer` service, the provider will try to:
(see `pkg/cloudprovider/providers/openstack/openstack_loadbalancer.go`/`EnsureLoadBalancer`)
1. create a load balancer (LB) in Openstack with listeners corresponding to the service's ports
2. attach a floating IP to the LB's network port
If `manage-security-groups` is enabled in controller-manager's cloud.conf:
3. create a security group with ingress rules corresponding to the LB's listeners, and attach it to the LB's network port
4. for all nodes of the cluster, pick an existing security group for the nodes ("node security group") and add ingress rules to it exposing the service's NodePorts to the security group created in step 3.
In the current upstream master, steps 1 through 3 work fine, step 4 fails, leading to a service that's not accessible via the LB without further manual intervention.
The bug is in the "pick an existing security group" operation (func `getNodeSecurityGroupIDForLB`), which, contrary to its name, will return the security group's name rather than its ID (actually it returns a list of names rather than IDs, apparently to cover some corner cases where you might have more than one node security group, but anyway). This will then be used when trying to add the ingress rules to the group, which the Openstack API will reject with a 404 (at least on our (fairly standard) Openstack Ocata installation) because we're giving it a name where it expects an ID.
The PR adds a "get ID given a name" lookup to the `getNodeSecurityGroupIDForLB` function, so it actually returns IDs. That's it. I'm not sure if the upstream code wasn't really tested, or maybe other people use other Openstacks with more lenient APIs. The bug and the fix is always reproducible on our installation.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#58145
**Special notes for your reviewer:**
Should we turn `getNodeSecurityGroupIDForLB` into a method with the lbaas as its receiver because it now requires two of the lbaas's attributes? I'm not sure what the conventions are here, if any.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 65449, 65373, 49410). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix run-in-gopath issue with symlink'd gopath
**What this PR does / why we need it**:
Fixes `hack/update-bazel.sh` so that it can be run in a symlink'd GOPATH, (such as using `hack/run-in-gopath.sh`).
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#65403.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add user-agent to audit-logging
**What this PR does / why we need it**:
Add User-Agent to audit event.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64791
**Special notes for your reviewer**:
**Release note**:
```release-note
Add user-agent to audit-logging.
```
Automatic merge from submit-queue (batch tested with PRs 59214, 65330). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Migrate cpumanager to use checkpointing manager
**What this PR does / why we need it**:
This PR migrates `cpumanager` to use new kubelet level node checkpointing feature (#56040) to decrease code redundancy and improve consistency.
**Which issue(s) this PR fixes**:
Fixes#58339
**Notes**:
At point of submitting PR the most straightforward approach was used - `state_checkpoint` implementation of `State` interface was added. However, with checkpointing implementation there might be no point to keep `State` interface and just use single implementation with checkpoint backend and in case of different backend than filestore needed just supply `cpumanager` with custom `CheckpointManager` implementation.
/kind feature
/sig node
cc @flyingcougar @ConnorDoyle
Automatic merge from submit-queue (batch tested with PRs 65156, 65448). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
lint fixes for goal state checks
**What this PR does / why we need it**:
Lint fixes for long lines introduced in #65187
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
**Special notes for your reviewer**:
We could also fix this by setting a longer flake8 line length in something like a local `setup.cfg`
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 65156, 65448). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[GCE] move NEG out of featuregate
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes # https://github.com/kubernetes/ingress-gce/issues/274
**Release note**:
-->
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 65164, 65258). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Query candidate zones for EBS when zone/zones not passed
**What this PR does / why we need it**:
This PR skips invoking `getCandidateZonesForDynamicVolume` to query EC2 zones of instances when zone/zones is present.
/sig storage
**Release note**:
```
none
```
Automatic merge from submit-queue (batch tested with PRs 65164, 65258). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add output to give user awareness of how long timeouts are expected to be
**What this PR does / why we need it**:
Add output to give user awareness of how long manifest upgrade timeout is expected to be.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
ref [kubernetes/kubeadm/#914](https://github.com/kubernetes/kubeadm/issues/914)
**Special notes for your reviewer**:
**Release note**:
```release-note
kubeadm: notify the user of manifest upgrade timeouts
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add myself to cmd/[cloud-]controller-manage reviewer
**What this PR does / why we need it**:
add myself to `cmd/cloud-controller-manage` reviewer
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
deployment: remove unused parameter 'podMap'
Signed-off-by: zhangjie <iamkadisi@163.com>
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix scheduler config decoding
Fixes#65413
Implements a custom unmarshaler for a single scheduler config type which did not correctly specify JSON tags until https://github.com/kubernetes/kubernetes/issues/65414 is resolved
Adds missing compatibility tests for scheduler extenders back to 1.7
```release-note
Fixes incompatibility with custom scheduler extender configurations specifying `bindVerb`
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
apiextensions: enable CoreAPI options needed for admission
Admission webhooks need the client and the shared informers for kube resources. The comment is invalid and we have to enable the CoreAPI options.
This PR is important to run apiextensions-apiserver in a standalone integration test setup.
Automatic merge from submit-queue (batch tested with PRs 63773, 65411). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
client-go/examples/fake-client: add doc.go to fix go build warnings
The publishing bot is currently broken (https://github.com/kubernetes/kubernetes/issues/56876#issuecomment-399763681). This is due to the following error:
```
[24 Jun 18 15:08 UTC]: Running smoke tests for branch master
[24 Jun 18 15:08 UTC]: /bin/bash -xec "godep restore\ngo build ./...\ngo test $(go list ./... | grep -v /vendor/)\n"
+ godep restore
+ go build ./...
go build k8s.io/client-go/examples/fake-client: no non-test Go files in /go-workspace/src/k8s.io/client-go/examples/fake-client
[24 Jun 18 15:09 UTC]: exit status 1
+ godep restore
+ go build ./...
go build k8s.io/client-go/examples/fake-client: no non-test Go files in /go-workspace/src/k8s.io/client-go/examples/fake-client
[24 Jun 18 15:09 UTC]: exit status 1```
```
The fakeclient package does not have any non-test go files. The test file was added in https://github.com/kubernetes/kubernetes/pull/65291 2 days ago.
This causes `go build` to give a warning: `no non-test Go files in /go-workspace/src/k8s.io/client-go/examples/fake-client`, which breaks the publishing bot. This PR adds a dummy doc.go file in the package to avoid this warning and fix the publishing bot.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63773, 65411). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Removed unused vars.
Signed-off-by: Da K. Ma <klaus1982.cn@gmail.com>
**Release note**:
```release-note
None
```
The fakeclient package does not have any non-test go files. This
causes `go build` to give a warning: `no non-test Go files in ...`.
This also causes the publishing bot to fail.
This PR adds a dummy doc.go file in the package to avoid this warning.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
When splitting `snake_case` words, omit the underscore
While we require camelCase by convention for official APIs, CRDs may use
`snake_case`, and the generic describer prints this as `Snake _ Case`. We should print `Snake Case`
Prow is impacted by this for the ProwJob CRD:
```
Decoration _ Config:
Gcs _ Configuration:
Bucket: origin-ci-test
Default _ Org: openshift
Default _ Repo: origin
Path _ Strategy: single
Gcs _ Credentials _ Secret: gcs-publisher-credentials
```
@kubernetes/sig-cli-pr-reviews
```release-note
Using `kubectl describe` on CRDs that use underscores will be prettier.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Improve scheduler's performance by eliminating sorting of nodes by their score
**What this PR does / why we need it**:
Profiling scheduler, I noticed that scheduler spends a significant amount of time in sorting the nodes after we score them to find nodes with the highest score. Finding nodes with the highest score does not need sorting the array. This PR replaces the sort with a linear scan.
Eliminating the sort results in over 10% improvement in throughput of the scheduler.
Before (3 runs for 5000 nodes, scheduling 1000 pods in a cluster running 2000 pods):
BenchmarkScheduling/5000Nodes/2000Pods-12 1000 20682552 ns/op
BenchmarkScheduling/5000Nodes/2000Pods-12 1000 20464729 ns/op
BenchmarkScheduling/5000Nodes/2000Pods-12 1000 21188906 ns/op
After:
BenchmarkScheduling/5000Nodes/2000Pods-12 1000 18485866 ns/op
BenchmarkScheduling/5000Nodes/2000Pods-12 1000 18457749 ns/op
BenchmarkScheduling/5000Nodes/2000Pods-12 1000 18418200 ns/op
**Release note**:
```release-note
Improve scheduler's performance by eliminating sorting of nodes by their score.
```
Automatic merge from submit-queue (batch tested with PRs 65388, 64995). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add more conditions to the list of predicate failures that won't be resolved by preemption
**What this PR does / why we need it**:
Adds more conditions to the list of predicate failures that won't be resolved by preemption. This change can potentially improve performance of preemption by avoiding the nodes that won't be able to schedule the pending pod no matter how many other pods are removed from them.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Add more conditions to the list of predicate failures that won't be resolved by preemption.
```
/sig scheduling
Automatic merge from submit-queue (batch tested with PRs 65388, 64995). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Always create kubeClusterIPSet in ipvs proxier
**What this PR does / why we need it**:
This PR creates the kubeClusterIPSet ipset even if kube-proxy is started without masqueradeAll and clusterCIDR.
This is necessary to masquerade traffic sent to a clusterIP from the host network namespace. The code to do so is actually already present here: https://github.com/kubernetes/kubernetes/blob/master/pkg/proxy/ipvs/proxier.go#L1220-L1244
However the second else (neither masqueradeAll nor clusterCIDR are set) cannot be used because, before this PR, the initial test `if !proxier.ipsetList[kubeClusterIPSet].isEmpty()` can never return true when masqueradeAll and clusterCIDR are not set because kubeClusterIPSet is empty.
**Which issue(s) this PR fixes**
Fixes#65158
```release-note
Allow access to ClusterIP from the host network namespace when kube-proxy is started in IPVS mode without either masqueradeAll or clusterCIDR flags
```
**Additional comment**
Issue #65158 is closed because ClusterIP access from the host has already fixed in master, except for the case described here (no masquerade flag). More detail in the issue.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add /home/kubernetes/bin into sudoers path, so that `sudo crictl` works.
Add `/home/kubernetes/bin` to sudoers path, so that user can call `sudo crictl` directly.
Without this fix, user has to either use the full path `sudo /home/kubernetes/bin/crictl` or switch to root, which is not a good user experience.
/cc @yujuhong @feiskyer @filbranden @kubernetes/sig-node-pr-reviews @kubernetes/sig-gcp-pr-reviews
**Release note**:
```release-note
User can now use `sudo crictl` on GCE cluster.
```
While we require camelCase by convention for official APIs, CRDs may use
`snake_case`, and the generic describer prints this as `Snake _ Case`.
Prow is impacted by this for the ProwJob CRD:
```
Decoration _ Config:
Gcs _ Configuration:
Bucket: origin-ci-test
Default _ Org: openshift
Default _ Repo: origin
Path _ Strategy: single
Gcs _ Credentials _ Secret: gcs-publisher-credentials
```
Automatic merge from submit-queue (batch tested with PRs 65338, 64535). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove cloud-controller-manager deb from releases
**What this PR does / why we need it**:
Stop including cloud controller manager deb from Kubernetes releases.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes # https://github.com/kubernetes/kubernetes/issues/65337
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64122, 64936, 65288, 65383). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
update github.com/pelletier/go-toml to 1.2.0
**What this PR does / why we need it**:
Rationale: github.com/pelletier/go-toml is the only package that currently
prevents the future vendoring of github.com/golang/dep as it depends on
functions introduced in 1.1.0.
The only consumers of this package are github.com/spf13/viper (used to run e2e
tests) and github.com/bazelbuild/bazel-gazelle (bazel helper), so that's a
pretty low-risk change.
**Special notes for your reviewer**:
This should help reducing the noise when #64731 lands
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64122, 64936, 65288, 65383). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Consume watch event for all versions of CRD
The new test code to fix flaky test were considered CRD version but failed to consume watch cache event in all versions.
Fixes#64571
@sttts @liggitt
Automatic merge from submit-queue (batch tested with PRs 64122, 64936, 65288, 65383). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix integer divide by zero panic
**What this PR does / why we need it**:
/kind bug
fix integer divide by zero panic when time.Since(start) < 1s
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64935
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64122, 64936, 65288, 65383). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update to rules_go 0.12.1 and gazelle 0.12.0 and perform related cleanups
**What this PR does / why we need it**: my initial intent was to simply update to rules_go 0.12.1 and gazelle 0.12.0.
A few internal changes / deprecations meant that I finally needed to clean up some technical debt. This also fixes#64122.
I've attempted to keep the steps as separate commits to make it easier to review:
1. Disable gazelle proto rule generation; legacy proto rules are deprecated, and we don't (currently) build protos at build time anyway, instead generating them with `hack/update-generated-protobuf.sh` and then checking them in. We can revisit this in the future if we'd like.
2. Remove the legacy `go_default_library_protos` filegroups using [buildozer](https://github.com/bazelbuild/buildtools/tree/master/buildozer). We don't use these, anyway.
3. Update the rules_go bazel workspace dependency to 0.12.1.
4. Vendor gazelle 0.12.0 and update BUILD files with `hack/update-bazel.sh`. This causes a lot of diffs, because `select()`s are no longer used in `srcs` attributes, external tests are folded into non-external tests, and vendored targets get an `importmap` attribute.
5. Set `gazelle:prefix` on `staging/src/BUILD` to make gazelle treat these correctly(ish). This allows us to remove the sed rewrite hack in `hack/update-bazel.sh`.
6. Explicitly set `# gazelle:importmap_prefix k8s.io/kubernetes/vendor` on `vendor/`, so that all vendored dependencies get the right importmap. gazelle 0.12.0 uses the bazel workspace name + `vendor/` as a prefix, which doesn't work with native go. Newer gazelle will use the go prefix (https://github.com/bazelbuild/bazel-gazelle/pull/207), but it's not released yet. Setting this correctly now also fixes later `BUILD` churn.
7. Re-run `hack/update-bazel.sh`. This causes a bunch of diffs, since anything under `staging/src` now uses the `staging/src/` path instead of `vendor/`. (Both would work for bazel, but gazelle uses the former, since `vendor/` uses symlinks.) Also `importmap`s under `vendor/` are fixed.
8. Reformat a few files (using [buildifier](https://github.com/bazelbuild/buildtools/tree/master/buildifier)) to make later diffs easier to read.
9. Rework the `go_genrule` rules to use the new `go_genrule` from https://github.com/kubernetes/repo-infra/pull/72, which is more bazely, since it uses the rules_go `go_path` rule instead of lots of shell.
10. Remove the deprecated `go_prefix` rule from the root BUILD.bazel file.
11. Set `# gazelle:importmap_prefix k8s.io/kubernetes/vendor` on `staging/src` as well, which ensures that these repos are treated as vendored dependencies. (It's basically the bazel-y way of doing the `vendor/k8s.io` symlinks.)
12. Run `hack/update-bazel.sh` one last time to fix all of the `importmap`s under `staging/src`.
Note re: point 6 above - we're pretty much ignoring the `vendor/k8s.io` symlinks entirely now under bazel. Using the `gazelle:prefix` directive ensures these get mapped into the right go importpath, and the `go_path` rule installs these correctly now too.
**Special notes for your reviewer**: this should not be submitted before https://github.com/kubernetes/repo-infra/pull/72, obviously.
**Release note**:
```release-note
NONE
```
/assign @BenTheElder @fejta @thockin
cc @cblecker @jayconrod
Kubernetes Submit Queue