Automatic merge from submit-queue
Allow mounts to run in parallel for non-attachable volumes
This PR:
* Fixes https://github.com/kubernetes/kubernetes/issues/28616
* Enables mount volume operations to run in parallel for non-attachable volume plugins.
* Enables unmount volume operations to run in parallel for all volume plugins.
* Renames `GoRoutineMap` to `GoroutineMap`, resolving a long outstanding request from @thockin: `"Goroutine" is a noun`
Allow mount volume operations to run in parallel for non-attachable
volume plugins.
Allow unmount volume operations to run in parallel for all volume
plugins.
Automatic merge from submit-queue
Fix port range checking, port should not be greater than 65535.
When passing flag `--proxy-port-range` to kube-proxy with an invalid range which is greater than 65535, the proxy doesn't exit. That's not what we want.
Should we fix this in v1.3?
/cc @thockin @mikedanese @resouer
Before fixing:
```
root@vm:/home/paas/zxp# kube-proxy --master=172.16.1.11:8080 --logtostderr=false --log-dir=/home/user/log/kube --proxy-port-range=65536-65599 &
[6] 6671
root@vm:/home/paas/zxp# ps -ef | grep kube-proxy
root 6671 13507 0 03:48 pts/1 00:00:00 kube-proxy --master=172.16.1.11:8080 --logtostderr=false --log-dir=/home/user/log/kube --proxy-port-range=65536-65599
```
After:
```
root@vm:/home/paas/zxp# kube-proxy --master=172.16.1.11:8080 --logtostderr=false --log-dir=/home/user/log/kube --proxy-port-range=65536-65599 &
[6] 6725
root@vm:/home/paas/zxp# invalid argument "65536-65599" for --proxy-port-range=65536-65599: "65536-65599" is not a valid port range: the port range cannot be greater than 65535: 65536-65599
..............
[6]+ Exit 2 kube-proxy --master=172.16.1.11:8080 --logtostderr=false --log-dir=/home/user/log/kube --proxy-port-range=65536-65599
```
```
root@vm:/home/paas/zxp# kube-proxy --master=172.16.1.11:8080 --logtostderr=false --log-dir=/home/user/log/kube --proxy-port-range=6000-65599 &
[6] 6732
root@vm:/home/paas/zxp# invalid argument "6000-65599" for --proxy-port-range=6000-65599: "6000-65599" is not a valid port range: the port range cannot be greater than 65535: 6000-65599
..............
[6]+ Exit 2 kube-proxy --master=172.16.1.11:8080 --logtostderr=false --log-dir=/home/user/log/kube --proxy-port-range=6000-65599
```
Automatic merge from submit-queue
Reexport term.IsTerminal
Reexport term.IsTerminal so downstream consumers (e.g. OpenShift) can use it.
@smarterclayton @sttts
Automatic merge from submit-queue
Support terminal resizing for exec/attach/run
```release-note
Add support for terminal resizing for exec, attach, and run. Note that for Docker, exec sessions
inherit the environment from the primary process, so if the container was created with tty=false,
that means the exec session's TERM variable will default to "dumb". Users can override this by
setting TERM=xterm (or whatever is appropriate) to get the correct "smart" terminal behavior.
```
Fixes#13585
Add support for terminal resizing for exec, attach, and run. Note that for Docker, exec sessions
inherit the environment from the primary process, so if the container was created with tty=false,
that means the exec session's TERM variable will default to "dumb". Users can override this by
setting TERM=xterm (or whatever is appropriate) to get the correct "smart" terminal behavior.
Automatic merge from submit-queue
controller: wait for synced old replica sets on Recreate
Partially fixes https://github.com/kubernetes/kubernetes/issues/27362
Any other work on it should be handled in the replica set level (and/or kubelet if it's required)
@kubernetes/deployment PTAL
Automatic merge from submit-queue
Add meta field to predicate signature to avoid computing the same things multiple times
This PR only uses it to avoid computing QOS of a pod for every node from scratch.
Ref #28590
Automatic merge from submit-queue
Fixes bad heuristic when calling "tc show" to check interface
`tc` sometimes returns stuff that has more than 12 words in its response. The heuristic is bad, but this at least fixes the case when `tc` is returning too much.
Fixes#28571.
This drives most of the logic of deep-copy generation from tags like:
// +deepcopy-gen=package
..rather than hardcoded lists of packages. This will make it possible to
subsequently generate code ONLY for packages that need it *right now*, rather
than all of them always.
Also remove pkgs that really do not need deep-copies (no symbols used
anywhere).
His PR cam during the middle of this development cycle, and it was easier to
burn it down and recreate it than try to patch it into an existing series and
re-test every assumption. This behavior will be re-introduced in subsequent
commits.
Automatic merge from submit-queue
Validation: Make validation func return error strings
Part of an ongoing series of validation cleanups.
This centralizes the error strings next to the code that checks the error conditions. Future commits will refine the messages further and provide more utility validators.
I'm OK if this doesn't go into 1.2, but I am tired of rebasing :) I suggest commit-by-commit review, which should go pretty quickly. This was largely mechanical.
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/21240)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Cleanup third party (pt 2)
Move forked-and-hacked golang code to the forked/ directory. Remove ast/build/parse code that is now in stdlib. Remove unused shell2junit
Automatic merge from submit-queue
dedup workqueue requeuing
Updates `workqueue.AddAfter` to only perform the add for the earliest requested requeue operation. An earlier time inserts in the earlier slot and removes the old one. A later time is ignored.
When using this conjunction with an `AddRateLimited` method, you get charged for the additional retry even though you're only queue once.
This keeps requeues from multiplying for every add.
@liggitt
Automatic merge from submit-queue
Reorganize volume controllers and manager
* Move both PV and attach/detach volume controllers to `controllers/volume` (closes#26222)
* Rename `kubelet/volume` to `kubelet/volumemanager`
* Add/update OWNER files
Automatic merge from submit-queue
Add MinReadySeconds to rolling updater
Add MinReadySeconds support to RollingUpdater that allows to specify the number of seconds to wait on top of the pod is "ready" because its readiness probe passed.
Automatic merge from submit-queue
Fixed goroutinemap race on Wait()
sync.WaitGroup produces data races when a GoroutineMap is empty and Wait() and Run() are called at the same time.
From sync.WaitGroup:
> Note that calls with a positive delta that occur when the counter is zero must happen before a Wait.
Fixes#28128
Note that this issue affects only PersistentVolume unit tests.
@saad-ali, PTAL
Automatic merge from submit-queue
Enable HTTP2 by default
Update to enable HTTP2 by default, with the option to disable.
This is a continuation of #25280 for the 1.4 release. This should provide ample time for vetting.
/cc @krousey
sync.WaitGroup produces data races when a GoroutineMap is empty and Wait() and
Run() are called at the same time.
From sync.WaitGroup:
Note that calls with a positive delta that occur when the counter is zero
must happen before a Wait.
Fixes#28128
Automatic merge from submit-queue
Proportionally scale paused and rolling deployments
Enable paused and rolling deployments to be proportionally scaled.
Also have cleanup policy work for paused deployments.
Fixes#20853Fixes#20966Fixes#20754
@bgrant0607 @janetkuo @ironcladlou @nikhiljindal
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/20273)
<!-- Reviewable:end -->
Automatic merge from submit-queue
GCE provider: Create TargetPool with 200 instances, then update with rest
GCE provider: Create TargetPool with 200 instances, then update with rest
Tested with 2000 nodes, this actually meets the GCE API specifications (which is nutty). Previous PR (#25178) was based on a mistaken understanding of a poorly documented set of limitations, and even poorer testing, for which I am embarassed.
Also includes the revert of #25178 (review commits separately).
[]()
Modify attach/detach controller to keep track of volumes to report
attached in Node VolumeToAttach status.
Modify kubelet volume manager to wait for volume to show up in Node
VolumeToAttach status.
Implement exponential backoff for errors in volume manager and attach
detach controller
Automatic merge from submit-queue
Fix bug in isLikelyNotMountPoint function
In nsenter_mount.go/isLikelyNotMountPoint function, the returned output
from findmnt command misses the last letter. Modify the code to use
String.contains instead of string matching. fixes#26421fixes#25056fixes#22911
- improve restoreInternal implementation in iptables
- add SetStdin and SetStdout functions to Cmd interface
- modify kubelet/prober and some tests in order to work with Cmd interface
In nsenter_mount.go/isLikelyNotMountPoint function, the returned output
from findmnt command misses the last letter. Modify the code to make sure
that output has the full target path. fix#26421#25056#22911
Automatic merge from submit-queue
Listing pods only once when getting pods for RS in deployment
Fixes#26834
1. Avoid ranging over RSes and then `List` pods of each RS. Instead, `List` pods of the deployment once, and then filter pods of each RS.
2. Avoid using clientset to `List` pods in deployment controller. Use podStore instead. (TODO in some functions because the unit tests don't have podStore.)
@kubernetes/deployment
[]()
Automatic merge from submit-queue
rkt: Fix incomplete selinux context string when the option is partial.
Fix "EmptyDir" e2e tests failures caused by #https://github.com/kubernetes/kubernetes/pull/24901
As mentioned in https://github.com/kubernetes/kubernetes/pull/24901#discussion_r61372312
We should apply the selinux context of the rkt data directory (/var/lib/rkt) when users do not specify all the selinux options.
Due to my fault, the change was missed during rebase, thus caused the regression.
After applying this PR, the e2e tests passed.
```
$ go run hack/e2e.go -v -test --test_args="--ginkgo.dryRun=false --ginkgo.focus=EmptyDir"
...
Ran 19 of 313 Specs in 199.319 seconds
SUCCESS! -- 19 Passed | 0 Failed | 0 Pending | 294 Skipped PASS
```
BTW, the test is removed because the `--no-overlay=true` flag will only be there on non-coreos distro.
cc @euank @kubernetes/sig-node
Automatic merge from submit-queue
Add specific error type for "operation already exists" error.
PersistentVolume controller needs to know why scheduling a new operation has failed - if the operation was already running or some other error happened.
Automatic merge from submit-queue
Sort revisions in rollout history as integers
Previously keys were sorted as strings, thus it was possible to see such order as 1, 10, 2, 3, 4, 5.
fixes: #25788
Automatic merge from submit-queue
Make UnsafeConversion fast by inlining copies
Not ready yet (need to add a copy to "safe" conversion and add mutation tests to roundtrip api/serialization_test).
Cuts another 10% off decode and encode.
Automatic merge from submit-queue
Increase goroutinemap unit test timeouts.
50ms is flaky in Jenkins. This makes the test to take at least 0.5s to check that things that should block and wait for something really do block and wait (was 100ms before).
Fixes#25825
Previously keys were sorted as strings, thus it was possible
to see such order as 1, 10, 2, 3, 4, 5.
Ints64 helper implemented in util/slice module to sort []int64
Automatic merge from submit-queue
Cache Webhook Authentication responses
Add a simple LRU cache w/ 2 minute TTL to the webhook authenticator.
Kubectl is a little spammy, w/ >= 4 API requests per command. This also prevents a single unauthenticated user from being able to DOS the remote authenticator.
Automatic merge from submit-queue
Only expose top N images in `NodeStatus`
Fix#25209
Sorted the image and only pick set top 50 sized images in node status.
cc @vishh
Automatic merge from submit-queue
Fixes panic on round tripper when TLS under a proxy
When under a proxy with a valid cert from a trusted authority, the `SpdyRoundTripper` will likely not have a `*tls.Config` (no cert verification nor `InsecureSkipVerify` happened), which will result in a panic. So we have to create a new `*tls.Config` to be able to create a TLS client right after. If `RootCAs` in that new config is nil, the system pool will be used.
@ncdc PTAL
[]()
PersistentVolume controller needs to know why scheduling a new operation has
failed - if the operation was already running or some other error happened.
Automatic merge from submit-queue
Make IsQualifiedName return error strings
Part of the larger validation PR, broken out for easier review and merge.
@lavalamp FYI, but I know you're swamped, too.
Automatic merge from submit-queue
Rate limitting requeue
I think this will do what I want in almost every case. I'm going to try it out for a single writer quota evaluation.
`NewRateLimitingQueue(NewControllerRateLimiter(qps, burst))` gives an `AddRateLimited(item)` that backs off based on the max of a bucket and exponential backoff per item.
@liggitt Want to see if the interface works for you too?
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/24052)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Upgrades tests use chaosmonkey package and ServiceTestJig
Introduce the `chaosmonkey` e2e package for doing disruptive testing (e.g. upgrade testing) more easily.
- [x] `chaosmonkey` package
- [x] migrate upgrade tests to `chaosmonkey` (using WIP `serviceJig`)
- [x] migrate upgrade tests to use `ServiceTestJig` and `chaosmonkey`
Deferred:
- [ ] make `ServiceTestJig` implement `chaosmonkey.Interface`
- [ ] migrate disruptive services tests to use `ServiceTestJig` and `chaosmonkey`
This provides the extensible framework for #15131. We should now easily be able to add tests (e.g. #6084, #23189).
This is a rewrite of #22446.
cc @mikedanese @quinton-hoole @roberthbailey
Assigning to @thockin, who wrote `ServiceTestJig`.
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/24014)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Display line number on JSON errors
Related issue: https://github.com/kubernetes/kubernetes/issues/12231
This PR will introduce line numbers for all JSON errors in the CLI:
(this is existing error reporting for YAML)
```console
$ kubectl create -f broken.yaml
yaml: line 8: mapping values are not allowed in this context
```
(this is error reporting proposed in this PR for JSON)
```console
$ kubectl create -f broken.json
json: line 35: invalid character '{' after object key
```
(and this is the current reporting:)
```console
$ kubectl create -f broken.json
invalid character '{' after object key
```
[]()
Automatic merge from submit-queue
Add data structure for managing go routines by name
This PR introduces a data structure for managing go routines by name. It prevents the creation of new go routines if an existing go routine with the same name exists. This will enable parallelization of the designs in https://github.com/kubernetes/kubernetes/issues/20262 and https://github.com/kubernetes/kubernetes/issues/21931 with sufficient protection to prevent starting multiple operations on the same volume.
Automatic merge from submit-queue
Kubelet: Cleanup with new engine api
Finish step 2 of #23563
This PR:
1) Cleanup go-dockerclient reference in the code.
2) Bump up the engine-api version.
3) Cleanup the code with new engine-api.
Fixes#24076.
Fixes#23809.
/cc @yujuhong
Scott Creeley
