Automatic merge from submit-queue
Avoid unnecessary memory allocations
Low-hanging fruits in saving memory allocations. During our 5000-node kubemark runs I've see this:
ControllerManager:
- 40.17% k8s.io/kubernetes/pkg/util/system.IsMasterNode
- 19.04% k8s.io/kubernetes/pkg/controller.(*PodControllerRefManager).Classify
Scheduler:
- 42.74% k8s.io/kubernetes/plugin/pkg/scheduler/algrorithm/predicates.(*MaxPDVolumeCountChecker).filterVolumes
This PR is eliminating all of those.
Automatic merge from submit-queue
Begin paths for internationalization in kubectl
This is just the first step, purposely simple so we can get the interface correct.
@kubernetes/sig-cli @deads2k
Automatic merge from submit-queue (batch tested with PRs 38920, 38090)
Improve error message for name/label validation.
Instead of just providing regex in name/label validation error output, we need to add the naming rules of the name/label, which is more end-user readable.
Fixed#37654
Automatic merge from submit-queue (batch tested with PRs 36888, 38180, 38855, 38590)
Fix variable shadowing in exponential backoff when deleting volumes
While https://github.com/kubernetes/kubernetes/pull/38339 implemented exponential backoff on
volume deletion, that PR suffers from a minor bug when error thrown on volume deletion is anything other than `VolumeInUse` errors - in which case exponential backoff will not work.
This PR fixes that. This PR also makes unit tests more deterministic because exponential backoff changed the way operations are permitted.
CC @jsafrane @childsb @wongma7
Automatic merge from submit-queue (batch tested with PRs 38525, 38977)
Prevent json decoder panic on invalid input
Related downstream issue: https://github.com/openshift/origin/issues/12132
```
# Can be replicated on kubectl with:
$ cat panic.json
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "",
"labels": {
"name": ""
},
"generateName": "",
"namespace": "",
"annotations": []
},
"spec": {}
},
$ kubectl create -f panic.json --validate=false
```
**Release note**:
```release-note
release-note-none
```
This patch handles cases where `ioutil.ReadAll` will return a single
character output on an invalid json input, causing the `Decode` method
to panic when it tries to calculate the line number for the syntax
error. The example below would cause a panic due to the trailing comma
at the end:
```
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "",
"labels": {
"name": ""
},
"generateName": "",
"namespace": "",
"annotations": []
},
"spec": {}
},
```
@kubernetes/cli-review @fabianofranz
Automatic merge from submit-queue
Add a package for handling version numbers (including non-"Semantic" versions)
As noted in #32401, we are using Semantic Version-parsing libraries to parse version numbers that aren't necessarily "Semantic". Although, contrary to what I'd said there, it turns out that this wasn't actually currently a problem for the iptables code, because the regexp used to extract the version number out of the "iptables --version" output only pulled out three components, so given "iptables v1.4.19.1", it would have extracted just "1.4.19". Still, it could be a problem if they later release "1.5" rather than "1.5.0", or if we eventually need to _compare_ against a 4-digit version number.
Also, as noted in #23854, we were also using two different semver libraries in different parts of the code (plus a wrapper around one of them in pkg/version).
This PR adds pkg/util/version, with code to parse and compare both semver and non-semver version strings, and then updates kubernetes to use it everywhere (including getting rid of a bunch of code duplication in kubelet by making utilversion.Version implement the kubecontainer.Version interface directly).
Ironically, this does not actually allow us to get rid of either of the vendored semver libraries, because we still have other dependencies that depend on each of them. (cadvisor uses blang/semver and etcd uses coreos/go-semver)
fixes#32401, #23854
Automatic merge from submit-queue
Fixed a typo of wildcard DNS regex variable name.
Happened to see the typo while reading code, fixed the typo and refined the code.
Automatic merge from submit-queue
fix duplicate validation/field/errors
**Release note**:
``` release-note
release-note-none
```
Related PR: https://github.com/kubernetes/kubernetes/pull/30313
PR #30313 fixed duplicate errors for invalid aggregate errors in
https://github.com/kubernetes/kubernetes/blob/master/pkg/kubectl/cmd/util/helpers.go
However, duplicate aggregate errors that went through
https://github.com/kubernetes/kubernetes/blob/master/pkg/util/validation/field/errors.go
were not affected by that patch.
This patch adds duplicate aggregate error checking to
`pkg/util/validation/field/errors.go`
##### Before
`$ kubectl set env rc/idling-echo-1 test-abc=1234`
```
error: ReplicationController "idling-echo-1" is invalid:
[spec.template.spec.containers[0].env[0].name: Invalid value:
"test-abc": must be a C identifier (matching regex
[A-Za-z_][A-Za-z0-9_]*): e.g. "my_name" or "MyName",
spec.template.spec.containers[1].env[0].name: Invalid value: "test-abc":
must be a C identifier (matching regex [A-Za-z_][A-Za-z0-9_]*): e.g.
"my_name" or "MyName", spec.template.spec.containers[0].env[0].name:
Invalid value: "test-abc": must be a C identifier (matching regex
[A-Za-z_][A-Za-z0-9_]*): e.g. "my_name" or "MyName",
spec.template.spec.containers[1].env[0].name: Invalid value: "test-abc":
must be a C identifier (matching regex [A-Za-z_][A-Za-z0-9_]*): e.g.
"my_name" or "MyName"]
```
`$ kubectl set env rc/node-1 test-abc=1234`
```
error: ReplicationController "idling-echo-1" is invalid:
[spec.template.spec.containers[0].env[0].name: Invalid value:
"test-abc": must be a C identifier (matching regex
[A-Za-z_][A-Za-z0-9_]*): e.g. "my_name" or "MyName",
spec.template.spec.containers[1].env[0].name: Invalid value: "test-abc":
must be a C identifier (matching regex [A-Za-z_][A-Za-z0-9_]*): e.g.
"my_name" or "MyName"]
```
##### After
`$ kubectl set env rc/idling-echo-1 test-abc=1234`
```
error: ReplicationController "idling-echo-1" is invalid:
[spec.template.spec.containers[0].env[0].name: Invalid value:
"test-abc": must be a C identifier (matching regex
[A-Za-z_][A-Za-z0-9_]*): e.g. "my_name" or "MyName",
spec.template.spec.containers[1].env[0].name: Invalid value: "test-abc":
must be a C identifier (matching regex [A-Za-z_][A-Za-z0-9_]*): e.g.
"my_name" or "MyName"]
```
`$ kubectl set env rc/node-1 test-abc=1234`
```
error: ReplicationController "node-1" is invalid:
spec.template.spec.containers[0].env[0].name: Invalid value: "test-abc":
must be a C identifier (matching regex [A-Za-z_][A-Za-z0-9_]*): e.g.
"my_name" or "MyName"
```
This patch handles cases where `ioutil.ReadAll` will return a single
character output on an invalid json input, causing the `Decode` method
to panic when it tries to calculate the line number for the syntax
error. The example below would cause a panic due to the trailing comma
at the end:
```
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "",
"labels": {
"name": ""
},
"generateName": "",
"namespace": "",
"annotations": []
},
"spec": {}
},
```
Automatic merge from submit-queue (batch tested with PRs 36736, 35956, 35655, 37713, 38316)
Optimize port_split_test test case.
The `normalized` field doesn't take affect in current test case.
This PR:
1. initializes valid and normalized cases with normalized=true.
2. adds some invalid cases.
@resouer Thanks!
This is a fix on top #38124. In this fix, we move the logic to filter
out shared mount references into operation_executor's UnmountDevice
function to avoid this part is being used by other types volumes such as
rdb, azure etc. This filter function should be only needed during
unmount device for GCI image.
Automatic merge from submit-queue (batch tested with PRs 35939, 38381, 37825, 38306, 38110)
Add test for multi-threaded use of ratelimiter
Adds a test to help prevent #38273 from occurring again
Related PR: https://github.com/kubernetes/kubernetes/pull/30313
PR #30313 fixed duplicate errors for invalid aggregate errors in
https://github.com/kubernetes/kubernetes/blob/master/pkg/kubectl/cmd/util/helpers.go
However, duplicate aggregate errors that went through
https://github.com/kubernetes/kubernetes/blob/master/pkg/util/validation/field/errors.go
were not affected by that patch.
This patch adds duplicate aggregate error checking to
`pkg/util/validation/field/errors.go`
\##### Before
`$ kubectl set env rc/idling-echo-1 test-abc=1234`
```
error: ReplicationController "idling-echo-1" is invalid:
[spec.template.spec.containers[0].env[0].name: Invalid value:
"test-abc": must be a C identifier (matching regex
[A-Za-z_][A-Za-z0-9_]*): e.g. "my_name" or "MyName",
spec.template.spec.containers[1].env[0].name: Invalid value: "test-abc":
must be a C identifier (matching regex [A-Za-z_][A-Za-z0-9_]*): e.g.
"my_name" or "MyName", spec.template.spec.containers[0].env[0].name:
Invalid value: "test-abc": must be a C identifier (matching regex
[A-Za-z_][A-Za-z0-9_]*): e.g. "my_name" or "MyName",
spec.template.spec.containers[1].env[0].name: Invalid value: "test-abc":
must be a C identifier (matching regex [A-Za-z_][A-Za-z0-9_]*): e.g.
"my_name" or "MyName"]
```
`$ kubectl set env rc/node-1 test-abc=1234`
```
error: ReplicationController "idling-echo-1" is invalid:
[spec.template.spec.containers[0].env[0].name: Invalid value:
"test-abc": must be a C identifier (matching regex
[A-Za-z_][A-Za-z0-9_]*): e.g. "my_name" or "MyName",
spec.template.spec.containers[1].env[0].name: Invalid value: "test-abc":
must be a C identifier (matching regex [A-Za-z_][A-Za-z0-9_]*): e.g.
"my_name" or "MyName"]
```
\##### After
`$ kubectl set env rc/idling-echo-1 test-abc=1234`
```
error: ReplicationController "idling-echo-1" is invalid:
[spec.template.spec.containers[0].env[0].name: Invalid value:
"test-abc": must be a C identifier (matching regex
[A-Za-z_][A-Za-z0-9_]*): e.g. "my_name" or "MyName",
spec.template.spec.containers[1].env[0].name: Invalid value: "test-abc":
must be a C identifier (matching regex [A-Za-z_][A-Za-z0-9_]*): e.g.
"my_name" or "MyName"]
```
`$ kubectl set env rc/node-1 test-abc=1234`
```
error: ReplicationController "node-1" is invalid:
spec.template.spec.containers[0].env[0].name: Invalid value: "test-abc":
must be a C identifier (matching regex [A-Za-z_][A-Za-z0-9_]*): e.g.
"my_name" or "MyName"
```
Automatic merge from submit-queue
Added support for HOME environment variable on Windows
**What this PR does / why we need it**:
On Windows the HOME environment variable should be taken in account when trying to find the home directory.
Several tools already support the HOME environment variable, notably git-bash. It would be very convenient to have the kubernete tools (including minikube) to also support the environment variable.
The current situation
**Special notes for your reviewer**:
**Release note**:
```
```
Automatic merge from submit-queue
add a configuration for kubelet to register as a node with taints
and deprecate --register-schedulable
ref #28687#29178
cc @dchen1107 @davidopp @roberthbailey
Automatic merge from submit-queue (batch tested with PRs 38194, 37594, 38123, 37831, 37084)
Better compat with very old iptables (e.g. CentOS 6)
Fixes reported issue with CentOS6 iptables 1.4.7 (ancient)
Older iptables expanded things like 0x4000 into 0x00004000, which defeats the
fallback "check" logic.
Fixes#37416
this is a workaround for the unmount device issue caused by gci mounter. In GCI cluster, if gci mounter is used for mounting, the container started by mounter script will cause additional mounts created in the container. Since these mounts are irrelavant to the original mounts, they should be not considered when checking the mount references. By comparing the mount path prefix, those additional mounts can be filtered out.
Plan to work on better approach to solve this issue.
This change is to only enable containerized mounter for nfs and
glusterfs types. For other types such as tmpfs, ext2/3/4 or empty type,
we should still use mount from $PATH
Automatic merge from submit-queue
Mention overflows when mistakenly call function FromInt
**What this PR does / why we need it**:
When mistakenly call this method with a value that overflows int32 will causes strange behavior in some environment (maybe in amd64 system, i'm not sure but my test shows that).
For example, call FromInt(93333333333) would result in -1155947179 and not mention overflows.
Automatic merge from submit-queue
Fix kubectl Stratigic Merge Patch compatibility
As @smarterclayton pointed out in [comment1](https://github.com/kubernetes/kubernetes/pull/35647#pullrequestreview-8290820) and [comment2](https://github.com/kubernetes/kubernetes/pull/35647#pullrequestreview-8290847) in PR #35647,
we cannot assume the API servers publish version and they shares the same version.
This PR removes all the calls of GetServerSupportedSMPatchVersion().
Change the behavior of `apply` and `edit` to:
Retrying with the old patch version, if the new version fails.
Default other usage of SMPatch to the new version, since they don't update list of primitives.
fixes#36916
cc: @pwittrock @smarterclayton
This PR is to fix the issue in converting aws volume id from mount
paths. Currently there are three aws volume id formats supported. The
following lists example of those three formats and their corresponding
global mount paths:
1. aws:///vol-123456
(/var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/vol-123456)
2. aws://us-east-1/vol-123456
(/var/lib/kubelet/plugins/kubernetes.io/mounts/aws/us-est-1/vol-123455)
3. vol-123456
(/var/lib/kubelet/plugins/kubernetes.io/mounts/aws/us-est-1/vol-123455)
For the first two cases, we need to check the mount path and convert
them back to the original format.
Automatic merge from submit-queue
Fix strategic patch for list of primitive type with merge sementic
Fix strategic patch for list of primitive type when the patch strategy is `merge`.
Before: we cannot replace or delete an item in a list of primitive, e.g. string, when the patch strategy is `merge`. It will always append new items to the list.
This patch will generate a map to update the list of primitive type.
The server with this patch will accept either a new patch or an old patch.
The client will found out the APIserver version before generate the patch.
Fixes#35163, #32398
cc: @pwittrock @fabianofranz
``` release-note
Fix strategic patch for list of primitive type when patch strategy is `merge` to remove deleted objects.
```
Automatic merge from submit-queue
Expand documentation and TODOs in a few packages
I was reading through unfamiliar code and mostly added TODOs and expanded and clarified documentations.
There are a couple of things that are real code changes:
- Removed some unused constants
- Changed `workqueue.Parallize` to clamp the number of worker goroutines to the number of items to be processed.
- Added another unit test to `workqueue.queue`. I thought I found a bug (I was wrong) and wrote a unit test to isolate. I figure the extra test is worth keeping.
Automatic merge from submit-queue
Add Windows support to kube-proxy
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
This is the first stab at supporting kube-proxy (userspace mode) on Windows
**Which issue this PR fixes** :
fixes#30278
**Special notes for your reviewer**:
The MVP uses `netsh portproxy` to redirect traffic from `ServiceIP:ServicePort` to a `LocalIP:LocalPort`.
For the next version we are expecting to have guidance from Microsoft Container Networking team.
**Limitations**:
Current implementation does not support DNS queries over UDP as `netsh portproxy` currently only supports TCP. We are working with Microsoft to remediate this.
cc: @brendandburns @dcbw
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
```
Automatic merge from submit-queue
Restore old apiserver cert CN
This patch got lost during rebase of https://github.com/kubernetes/kubernetes/pull/35109:
- set `host@<unix-timestamp>` as CN in self-signed apiserver certs
- skip non-domain CN in getNamedCertificateMap
Automatic merge from submit-queue
Handle redirects in apiserver proxy handler
Overview:
1. Peek at the HTTP response from the proxied backend
2. If it is a redirect response (302/3), redo the request to the redirect location
3. If it's not a redirect, forward the response to the client and then set up the proxy as before
This change is required for implementing streaming requests in the Container Runtime Interface (CRI). See [design](https://docs.google.com/document/d/1OE_QoInPlVCK9rMAx9aybRmgFiVjHpJCHI9LrfdNM_s/edit).
For https://github.com/kubernetes/kubernetes/issues/29579
/cc @yujuhong
Automatic merge from submit-queue
update port validation message
Related Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1355703#c2
Port validation that results in a multi-line message:
```
* spec.template.spec.containers[0].livenessProbe.httpGet.port: Invalid value: "": must contain only alpha-numeric characters (a-z, 0-9), and hyphens (-)
* spec.template.spec.containers[0].livenessProbe.httpGet.port: Invalid value: "": must contain at least one letter (a-z)
```
suggests that ports can only be at minimum one letter.
Per [this bugzilla comment](https://bugzilla.redhat.com/show_bug.cgi?id=1355703#c2), this patch updates the second bullet point on the error message to be clearer:
```
* spec.template.spec.containers[0].livenessProbe.httpGet.port: Invalid value: "": must contain only alpha-numeric characters (a-z, 0-9), and hyphens (-)
* spec.template.spec.containers[0].livenessProbe.httpGet.port: Invalid value: "": must contain at least one letter or number (a-z, 0-9)
```
**Release note**:
```release-note
release-note-none
```
Automatic merge from submit-queue
[Federation][init-08] Refactor the tests by pulling the common utilities into a testing package.
Please review only the last commit here. This is based on PR #35864 which will be reviewed independently.
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @nikhiljindal
Mike Danese