Automatic merge from submit-queue (batch tested with PRs 63417, 64249, 64242, 64128, 64275). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
graduate DynamicKubeletConfig feature to beta
Everything in the https://github.com/kubernetes/features/issues/281 beta target except a few documentation-related items is complete. Docs should be done very soon, so I'm sending this PR to graduate to beta before freeze.
```release-note
The dynamic Kubelet config feature is now beta, and the DynamicKubeletConfig feature gate is on by default. In order to use dynamic Kubelet config, ensure that the Kubelet's --dynamic-config-dir option is set.
```
/cc @luxas
Automatic merge from submit-queue (batch tested with PRs 63417, 64249, 64242, 64128, 64275). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
conformance testdata: remove tags and trim spaces
**What this PR does / why we need it**:
To detect whether conformance tests have been changed, the test
argument with the `ConformanceIt` is recorded to conformance.txt.
This change remove additional tags (e.g., [Feature:]) and trim spaces,
so that the detection is less-prone to noise.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63417, 64249, 64242, 64128, 64275). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix bugs that break processing when printing errors occur in kubectl
**What this PR does / why we need it**:
The process should continue when error happens, this patch fix the problem
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63417, 64249, 64242, 64128, 64275). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
resourcequota return StatusError when timeout
Fixes#64244
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63319, 64248, 64250, 63890, 64233). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
move Describer from factory
**Release note**:
```release-note
NONE
```
Moves `Describer` out of the object_mapping_factory
cc @soltysh @deads2k
Automatic merge from submit-queue (batch tested with PRs 63319, 64248, 64250, 63890, 64233). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Additional test coverage for kubectl/cmd/cp
This commit adds some additional test coverage
for the kubectl cp command.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63319, 64248, 64250, 63890, 64233). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
replace `__internal` with runtime.APIVersionInternal
**What this PR does / why we need it**:
Replace `__internal` with runtime.APIVersionInternal
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63319, 64248, 64250, 63890, 64233). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add reason message logs for non-exist Azure resources
**What this PR does / why we need it**:
In azure_wrap.go the function checkResourceExistsFromError() looks at a received error and determines it to be a ResourceNotFound if the HTTP status is http.StatusNotFound. However, the HTTP status also equals http.StatusNotFound in case of SubscriptionNotFound.
this PR adds logs to indict the real error messages for such case.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64220
**Special notes for your reviewer**:
**Release note**:
```release-note
Add reason message logs for non-exist Azure resources
```
Automatic merge from submit-queue (batch tested with PRs 63319, 64248, 64250, 63890, 64233). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Always masquerade node-originating traffic with a service VIP source ip
**What this PR does / why we need it**:
This is a follow up to make IPVS work on systems without cluster-cidr or masquerade-all.
On these systems the best matching network / source IP to reach the service VIP is the service VIP itself - at least for the host network.
The workaround is simple: Everything originating on the host (OUTPUT nat chain) with a source IP that is the VIP should be masqueraded.
The relevant rule change is the first rule in `KUBE-SERVICES`:
```
Chain KUBE-SERVICES (2 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-MARK-MASQ all -- * * 0.0.0.0/0 0.0.0.0/0 match-set KUBE-CLUSTER-IP src,dst
104 6240 KUBE-MARK-MASQ tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp match-set KUBE-NODE-PORT-TCP dst
```
The matching rule could be stricter by matching src(ip),dst(ip),dst(port) but the src ip will only be selected if the VIP should be reached.
**Which issue(s) this PR fixes**
Fixes#63241
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64060, 63904, 64218, 64208, 64247). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
apimachinery: remove unused UnstructuredObjectConverter
This is not in apiextensions-apiserver for CR conversion.
Automatic merge from submit-queue (batch tested with PRs 64060, 63904, 64218, 64208, 64247). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
prevent zero for leader election timeouts
Add conditional checks for a stupid error that took me 20 minutes to find.
I don't see a reason to allow zero for any of the leader election tuning params
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64060, 63904, 64218, 64208, 64247). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Revert enable PodPreset admission and also enable settings.k8s.io/v1a…
…lpha1 api resource
**What this PR does / why we need it**:
Enable PodPreset admission for there are alpha feature test cases covering it. Simultaneously enable sttings.k8s.io/v1alpha1 api resource.
Fixes#63843
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64060, 63904, 64218, 64208, 64247). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove knob of equiv class in perf test
**What this PR does / why we need it**:
The ecache knob is controlled by feature gate now:
```go
// createSchedulerConfigurator create a configurator for scheduler with given informer factory and default name.
func createSchedulerConfigurator(
clientSet clientset.Interface,
informerFactory informers.SharedInformerFactory,
) scheduler.Configurator {
// Enable EnableEquivalenceClassCache for all integration tests.
utilfeature.DefaultFeatureGate.Set("EnableEquivalenceClassCache=true")
```
We don't need to pass a flag (and it is not used).
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63434, 64172, 63975, 64180, 63755). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Dump Stack when docker fails on healthcheck
Save stack dump of docker daemon in order to be able to
investigate why docker daemon was unresposive to `docker ps`
See https://github.com/moby/moby/blob/master/daemon/daemon.go on
how docker sets up a trap for SIGUSR1 with `setupDumpStackTrap()`
**What this PR does / why we need it**:
This allows us to investigate why docker daemon was unresponsive to "docker ps" command.
**Special notes for your reviewer**:
Manually tested on Ubuntu and COS.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63434, 64172, 63975, 64180, 63755). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Check SessionAffinity for Azure load balancer
**What this PR does / why we need it**:
Currently azure load balance rule comparing is based on name which does not have affinity information. So during updating, the loadDistribution property is not updated.
This PR fixes this and adds some verbose logs for better understanding what's happening.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#63908
**Special notes for your reviewer**:
Cherry pick to 1.10.
**Release note**:
```release-note
Fix SessionAffinity not updated issue for Azure load balancer
```
Automatic merge from submit-queue (batch tested with PRs 63434, 64172, 63975, 64180, 63755). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Optimize the lock which in the RunPredicate
**What this PR does / why we need it**:
Enhance the performance of scheduler
- Change the lock in the RunPredicate from lock to rlock
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Could solve part of #63784
**Special notes for your reviewer**:
_Run benchmark test by scheduler_perf_:
`Before` BenchmarkScheduling/1000Nodes/0Pods-32 1000 11689758 ns/op
`After` BenchmarkScheduling/1000Nodes/0Pods-32 1000 5951510 ns/op
_Run integration (density) test by scheduler_perf_:
Schedule 3000 Pods On 3000 Nodes
`Before` rate 19 per second on average
`After` rate 58 per second on average
_Cpu profile test result_:
`Before` [click](https://cdn.rawgit.com/godliness/files/master/63784_before.svg)
`After` [click](https://cdn.rawgit.com/godliness/files/master/63784_after.svg)
**Release note**:
```release-note
`None`
```
/sig scheduling
/cc @misterikkit
/cc @bsalamat
/cc @ravisantoshgudimetla
/cc @resouer
Automatic merge from submit-queue (batch tested with PRs 63434, 64172, 63975, 64180, 63755). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
re-reorder authorizers (RBAC before Webhook).
Sort of reverts #57426
We aren't relying on the ordering for anything, and it's not worth the additional latency/traffic at this point.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix typo in volume_stats.go
**What this PR does / why we need it**:
While reviewing the implementation details I came across a typo in volume_stats.go
sed/volumeStatsCollecotr/volumeStatsCollector/
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64127, 63895, 64066, 64215, 64202). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix documentation of GA is in APIService's PriorityVersion and CRD's Versions
Explain what GA is in APIService's PriorityVersion and CRD's Versions.
ref #64136
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64127, 63895, 64066, 64215, 64202). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: Refactor the .Etcd substruct in the v1alpha2 API
**What this PR does / why we need it**:
Splits the monolithic `.Etcd` struct with all the options as fields to a more modular and clear design with two sub-structs for the different modes of hosting etcd we support.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of kubernetes/community#2131
**Special notes for your reviewer**:
Depends on:
- [x] https://github.com/kubernetes/kubernetes/pull/63917
Follows up: https://github.com/kubernetes/kubernetes/pull/63871
TODO: I still need to write unit tests for this.
**Release note**:
```release-note
[action required] kubeadm: The `:Etcd` struct has been refactored in the v1alpha2 API. All the options now reside under either `.Etcd.Local` or `.Etcd.External`. Automatic conversions from the v1alpha1 API are supported.
```
@kubernetes/sig-cluster-lifecycle-pr-reviews @liztio
Automatic merge from submit-queue (batch tested with PRs 64127, 63895, 64066, 64215, 64202). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add warnings about cache invalidation.
Part of https://github.com/kubernetes/kubernetes/pull/63040 is the
assumption that scheduler cache updates must happen before equivalence
cache updates for any given informer event.
The reason for this is that the equivalence cache implementation checks
the main cache for staleness while holding the equiv. cache write lock.
case 1: If an informer invalidates an equiv. cache entry before the
staleness check, then we know that the main cache update completed.
case 2: If an informer blocks trying to grab the equiv. cache lock, then
invalidation will occur right after the potentially stale update is
written.
This patch adds a note to places where we invalidate the equivalence
cache so that hopefully nobody violates this invariant.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/kind cleanup
/sig scheduling
Automatic merge from submit-queue (batch tested with PRs 64127, 63895, 64066, 64215, 64202). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
convert Duration into seconds by go library function
**What this PR does / why we need it**:
Here we try convert time.Duration into int64 seconds. I think it's better to use golang library function,
https://golang.org/pkg/time/#Duration.Round
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64174, 64187, 64216, 63265, 64223). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Moving Regional PD e2e tests to regular test suites
**What this PR does / why we need it**: Regional PD e2e tests previously only run on alpha suites. Need to move them to the regular test suites
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of #59988
**Special notes for your reviewer**: Will be cherry-picked to 1.10 once merged.
**Release note**:
```release-note
NONE
```
/sig gcp
/sig storage
/assign @msau42 @saad-ali
Automatic merge from submit-queue (batch tested with PRs 64174, 64187, 64216, 63265, 64223). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Extend dns configmap tests to include CoreDNS
**What this PR does / why we need it**:
This PR extends the DNS configmap e2e tests to include testing the CoreDNS ConfigMap.
The tests now test the equivalent `stubdomain`, `federation` and `upstreamnameserver` configuration of kube-dns for CoreDNS, when CoreDNS is the installed DNS server.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62865
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64174, 64187, 64216, 63265, 64223). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Do not use DeepEqual to compare slices in test.
This wraps DeepEqual with a helper that considers nil slices and empty
slices to be equal.
Scheduler code might use a nil slice or empty slice to represent an
empty list, so tests should not be sensitive to the difference. Tests
could fail because DeepEqual considers nil to be different from an empty
slice.
**What this PR does / why we need it**:
Avoid breaking tests in cases where application behavior is not changed.
**Special notes for your reviewer**:
This brittle test keeps breaking in a number of my PRs. Hoping to get this fix merged independently.
**Release note**:
```release-note
NONE
```
/sig scheduling
/kind cleanup
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Move unrelated methods from the factory to helper
**What this PR does / why we need it**:
Move unrelated methods from the factory to helper
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64106
**Special notes for your reviewer**:
/cc @juanvallejo
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64013, 63896, 64139, 57527, 62102). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Setup fsGroup for local volumes correctly
**What this PR does / why we need it**:
This pr fixes fsGroup check in local volume in containerized kubelet. Except this, it also fixes fsGroup check when volume source is a normal directory whether kubelet is running on the host or in a container.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61741
**Special notes for your reviewer**:
Bind mounts are detected in `/proc/mounts`, but it does not contain root of mount for bind mounts. So `mount.GetMountRefsByDev()` cannot get all references if source is a normal directory. e.g.
```
# mkdir /tmp/src /mnt/dst
# mount --bind /tmp/src /tmp/src # required by local-volume-provisioner, see https://github.com/kubernetes-incubator/external-storage/pull/499
# mount --bind /tmp/src /mnt/dst
# grep -P 'src|dst' /proc/mounts
tmpfs /tmp/src tmpfs rw,nosuid,nodev,noatime,size=4194304k 0 0
tmpfs /mnt/dst tmpfs rw,nosuid,nodev,noatime,size=4194304k 0 0
# grep -P 'src|dst' /proc/self/mountinfo
234 409 0:42 /src /tmp/src rw,nosuid,nodev,noatime shared:30 - tmpfs tmpfs rw,size=4194304k
235 24 0:42 /src /mnt/dst rw,nosuid,nodev,noatime shared:30 - tmpfs tmpfs rw,size=4194304k
```
We need to compare root of mount and device in this case.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64013, 63896, 64139, 57527, 62102). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add dynamic config metrics
This PR exports config-releated metrics from the Kubelet.
The Guages for active, assigned, and last-known-good config can be used
to identify config versions and produce aggregate counts across several
nodes. The error-reporting Gauge can be used to determine whether a node
is experiencing a config-related error, and to prodouce an aggregate
count of nodes in an error state.
https://github.com/kubernetes/features/issues/281
```release-note
The Kubelet now exports metrics that report the assigned (node_config_assigned), last-known-good (node_config_last_known_good), and active (node_config_active) config sources, and a metric indicating whether the node is experiencing a config-related error (node_config_error). The config source metrics always report the value 1, and carry the node_config_name, node_config_uid, node_config_resource_version, and node_config_kubelet_key labels, which identify the config version. The error metric reports 1 if there is an error, 0 otherwise.
```
Automatic merge from submit-queue (batch tested with PRs 64013, 63896, 64139, 57527, 62102). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Simplify the volume util by v1helper.
Signed-off-by: Da K. Ma <klaus1982.cn@gmail.com>
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 64013, 63896, 64139, 57527, 62102). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Refactor test utils that deal with Kubelet metrics for clarity
I found these functions hard to understand, because the names did not
accurately reflect their behavior. For example, GetKubeletMetrics
assumed that all of the metrics passed in were measuring latency.
The caller of GetKubeletMetrics was implicitly making this assumption,
but it was not obvious at the call site.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64013, 63896, 64139, 57527, 62102). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubectl uses its own pkg/kubectl/util/logs
kubectl uses its own logs instead of `staging/src/k8s.io/apiserver/pkg/util/logs`
**Release note**:
```release-note
NONE
```
Kubernetes Submit Queue
