Automatic merge from submit-queue (batch tested with PRs 67362, 67256, 67809). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Dry run integration
Implement an integration test for dry-run. Also, this turns on the knob to allow dry-run requests, so let's be careful.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
This extends the Kubelet to create and periodically update leases in a
new kube-node-lease namespace. Based on [KEP-0009](https://github.com/kubernetes/community/blob/master/keps/sig-node/0009-node-heartbeat.md),
these leases can be used as a node health signal, and will allow us to
reduce the load caused by over-frequent node status reporting.
- add NodeLease feature gate
- add kube-node-lease system namespace for node leases
- add Kubelet option for lease duration
- add Kubelet-internal lease controller to create and update lease
- add e2e test for NodeLease feature
- modify node authorizer and node restriction admission controller
to allow Kubelets access to corresponding leases
Automatic merge from submit-queue (batch tested with PRs 67707, 62420). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
svcacct: support jose.OpaqueSigner and push errors to token generator creation
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Externalize serviceaacount admission controller
ref: #66680
this pull externalizes serviceaccount admission controller in which secret & serviceaccount informers will be completely replaced.
/sig api-machinery
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Create cli-runtime staging repository
**What this PR does / why we need it**:
This PR creates a designated repository for CLI helpers, which are used for:
- kubectl itself
- kubectl plugins
- commands that want to mimic kubectl behavior
/assign @deads2k @juanvallejo
@seans3 @pwittrock fyi
**Release note**:
```release-note
Create cli-runtime staging repository
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add Labels to various OWNERS files
**What this PR does / why we need it**:
Will reduce the burden of manually adding labels. Information pulled
from:
https://github.com/kubernetes/community/blob/master/sigs.yaml
Change-Id: I17e661e37719f0bccf63e41347b628269cef7c8b
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 67661, 67497, 66523, 67622, 67632). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Externalize node informers for node authz
the pull will completely externalize node authz together with #67194
ref: #66680
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add a feature to the scheduler to score fewer than all nodes in every scheduling cycle
**What this PR does / why we need it**:
Today, the scheduler scores all the nodes in the cluster in every scheduling cycle (every time a posd is attempted). This feature implements a mechanism in the scheduler that allows scoring fewer than all nodes in the cluster. The scheduler stops searching for more nodes once the configured number of feasible nodes are found. This can help improve the scheduler's performance in large clusters (several hundred nodes and larger).
This PR also adds a new structure to the scheduler's cache, called NodeTree, that allows scheduler to iterate over various nodes in different zones in a cluster. This is needed to avoid scoring the same set of nodes in every scheduling cycle.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#66627
**Special notes for your reviewer**:
This is a large PR, but broken into a few logical commits. Reviewing would be easier if you review by commits.
**Release note**:
```release-note
Add a feature to the scheduler to score fewer than all nodes in every scheduling cycle. This can improve performance of the scheduler in large clusters.
```
Automatic merge from submit-queue (batch tested with PRs 66920, 67316, 67363, 67528, 66963). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
integration test run PreShutdownHooks before close.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#66962
**Special notes for your reviewer**:
Run `PreShutdownHooks` before close master.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62441, 66702, 67254, 67421, 65309). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add backoff for DS's pod deletion to limit fighting with kubelet failing the pod repeatedly
**What this PR does / why we need it**:
Limits consequences of DS controller on hot loop fighting with kubelet.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubernetes/issues/65240
**Release note**:
```release-note
DaemonSet controller is now using backoff algorithm to avoid hot loops fighting with kubelet on pod recreation when a particular DaemonSet is misconfigured.
```
TODO:
- [x] Export the backoff settings as args or constants
- [x] Add test a case
/cc @mfojtik
(Will add more folks when it's ready, to avoid spamming them.)
Automatic merge from submit-queue (batch tested with PRs 67294, 67320, 67335, 67334, 67325). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add wait to discovery integration test to fix flakiness
**What this PR does / why we need it**:
Reduce the flakiness of the discovery test that was introduced in #66932 (see cherry-pick PR #67154 for report on flaky failure and suggestions).
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 67071, 66906, 66722, 67276, 67039). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
#50102 Task 1: Move apimachinery/pkg/watch.Until into client-go/tools/watch.UntilWithoutRetry
**What this PR does / why we need it**:
This is a split off from https://github.com/kubernetes/kubernetes/pull/50102 to go in smaller pieces.
Moves `apimachinery/pkg/watch.Until` into `client-go/tools/watch.UntilWithoutRetry` and adds context so it is cancelable.
**Release note**:
```release-note
NONE
```
**Dev release note**:
```dev-release-note
`apimachinery/pkg/watch.Until` has been moved to `client-go/tools/watch.UntilWithoutRetry`.
While switching please consider using the new `client-go/tools/watch.UntilWithSync` or `client-go/tools/watch.Until`.
```
/cc @smarterclayton @kubernetes/sig-api-machinery-pr-reviews
/milestone v1.12
/priority important-soon
/kind bug
(bug after the main PR which is this split from)
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix flakiness in daemonset TestLaunchWithHashCollisiion
**What this PR does / why we need it**: Addresses flakiness in test/integration/daemonset TestLaunchWithHashCollision. It makes sure the test waits for the pod to be added to the daemonset before updating the daemonset. The should avoid the race where the daemonset gets updated in between the calls to get and update in the test.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#67273
**Special notes for your reviewer**:
@janetkuo
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
attachdetach controller: attach volumes immediately when Pod's PVCs are bound
**What this PR does / why we need it**:
Let attachdetach controller to attach volumes immediately when Pod's PVCs are bound.
Current attachdetach controller calls `util.ProcessPodVolume` to add pod volumes into `desiredStateOfWorld` on these events:
- podAdd event
- podUpdate event
- podDelete event
- periodical `desiredStateOfWorldPopulator.findAndAddActivePod`
But if a pod is created with PVCs not bound, no volumes will be added into `desiredStateOfWorld` [because PVCs not bound](https://github.com/kubernetes/kubernetes/blob/v1.12.0-alpha.0/pkg/controller/volume/attachdetach/util/util.go#L99). When pv controller binds PVCs successfully, attachdetach controller will not add pod volumes immediately because it does not watch on PVC events.
It will wait until a pod update event is triggered (normally will not happen because no new status will be reported by kubelet) or `desiredStateOfWorldPopulator.findAndAddActivePod` is called (maybe 0~3 minutes later, see [timer configs](https://github.com/kubernetes/kubernetes/blob/v1.12.0-alpha.0/pkg/controller/volume/attachdetach/attach_detach_controller.go)).
In bad case, pod start time will be very long (~3 minutes + ~2 minutes (kubelet max exponential backoff)), for example: https://github.com/kubernetes/kubernetes/issues/64549#issuecomment-409440546.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64549
**Special notes for your reviewer**:
**Release note**:
```release-note
attachdetach controller attaches volumes immediately when Pod's PVCs are bound
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add integration test for scheduler "on PVC add" event handling
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#67246
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 67090, 67159, 66866, 62111, 66476). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix to handle hash collisions correctly for DaemonSets
**What this PR does / why we need it**: This adds an integration test for the case where there is a hash collision when creating a ControllerRevision for a DaemonSet. It also fixes a shadowed variable that prevented this functionality from working as intended.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62519
**Special notes for your reviewer**:
/sig apps
**Release note**:
```release-note
Fixes issue when updating a DaemonSet causes a hash collision.
```
Automatic merge from submit-queue (batch tested with PRs 65297, 67179, 67116, 67011, 66842). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Externalize node admission controller
**Release note**:
```release-note
NONE
```
**What this PR does / why we need it**:
/sig api-machinery
under unbrella: #66680
We start with injecting external informer/lister into node admission controller for that it's slightly simpler than the others.
Here is the thing I actually did in this pull:
1. Enable plugin initializer to inject external/internal clientset/informers into the admission controllers.
2. Receive external informer in node admission controller.
3. Create a `AdmissionConfig` following how we did in `AuthenticationConfig` and `AuthorizationConfig` and creates the actual plugin initializers and the admission post-start hook in its `New()` method
Automatic merge from submit-queue (batch tested with PRs 66394, 66888, 66932). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Include unavailable apiservices in discovery response
**What this PR does / why we need it**:
Include unavailable apiservices into `apis/` discovery endpoint response to fix namespace deletion https://github.com/kubernetes-incubator/service-catalog/issues/2254
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes-incubator/service-catalog/issues/2254
**Special notes for your reviewer**:
**Release note**:
```release-note
kube-apiserver now includes all registered API groups in discovery, including registered extension API group/versions for unavailable extension API servers.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add watch integration test for dynamic client
**What this PR does / why we need it**:
Add watch to dynamic client integration test
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Use the same pattern everywhere in the e2e test
harness, use busybox (from dockerhub) instead
of using the one from k8s.gcr.io registry.
Change-Id: I57c3b867408c1f9478a8909c26744ea0368ff003
Automatic merge from submit-queue (batch tested with PRs 65570, 65616). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Retry scheduling on StorageClass events
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56163
**Special notes for your reviewer**:
I have taken over #60006.
It's hard to test in e2e, because we cannot know reschedule of pod is triggered by which event (periodically service/node events will move pods to active queue too). ~~I'll add integration tests for this functionality after [this PR](https://github.com/kubernetes/kubernetes/pull/65296) get merged.~~ (already added)
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Move the` k8s.io/kubernetes/pkg/util/pointer` package to` k8s.io/utils/pointer`
**What this PR does / why we need it**:
Move `k8s.io/kubernetes/pkg/util/pointer` to `shared utils` directory, so that we can use it easily.
Close#66010 accidentally, and can't reopen it, so the same as #66010
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
test(scheduler): add more integration tests for TaintNodesByCondition
- refactor test cases to table driven testing style
- more scenarios are covered for TaintNodesByCondition
part of #62109
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
- Add resyncPeriod parameter for setupCluster to make resync period of
scheduler configurable.
- Add test case for static provisioning and delay binding storage class.
Move pods into active queue on PV add/update events.
- Add a stress test with scheduler resync to detect possible race
conditions.
Automatic merge from submit-queue (batch tested with PRs 66212, 66256, 66263, 66193). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove the unused verifyRemainingObjects function
**What this PR does / why we need it**:
erase useless function when reading code
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
none
```
Automatic merge from submit-queue (batch tested with PRs 63877, 64559, 65952). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Typo fix in returned message: formating->formatting
Line 98: formating->formatting
Automatic merge from submit-queue (batch tested with PRs 66094, 65676). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Safe encode template hash value to make it consistent with resource name
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#66093
**Special notes for your reviewer**: It's safe to change the function that generates template hash label, because this value is only used when creating a resource and never updated or compared. Therefore, it won't break existing workloads after k8s upgrade/downgrade. Note that we've changed hash before when introducing hash collision avoidance mechanism.
@kubernetes/sig-apps-pr-reviews
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 65677, 65711, 65150, 65726). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add additional authorization check for create-on-update
**What this PR does / why we need it**:
Currently it is possible for a user who is only authorized to update objects to send a PUT request for an object that doesn't currently exist, and if that resource allows create on update, it will all them to create the object. This PR fixes that bug and adds a test case which fails on master, but succeeds when the additional authorization check is done.
/sig api-machinery
/kind bug
/cc @liggitt @lavalamp
**Release note**:
```release-note
LimitRange and Endpoints resources can be created via an update API call if the object does not already exist. When this occurs, an authorization check is now made to ensure the user making the API call is authorized to create the object. In previous releases, only an update authorization check was performed.
```
Automatic merge from submit-queue (batch tested with PRs 64575, 65120, 65463, 65434, 65522). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add support for Linux Abstract Socket Namespace for KMS provider plugin.
**What this PR does / why we need it**:
Currently, kube-apiserver and kms-plugin interact via a Unix Domain Socket. The current implementation, assumes that such a Domain Socket is supported via a socket file, which is in turn is supported via a volume shared between kube-apiserver and kms-plugin containers.
However, Linux supports Abstract Socket Namespace, where a socket does not need to be back-up by a file. In golang, such sockets are created by prefixing a socket's name with @.
Benefits of using Linux Abstract Socket Namespace:
1. Don't need to worry about possible collisions with existing files.
2. Simpler configuration of master's manifest - no need to setup a shared volume between kube-apiserver and kms-plugin containers.
3. Don't need to remember to unlink the socket when KMS Plugin shuts down.
4. Creates a possibility to run KMS Plugin without access to file system.
This PR adds the ability to define a KMS endpoint as: unix:///@kms-provider.sock
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add limit to the TokenRequest expiration time
**What this PR does / why we need it**:
A new API TokenRequest has been implemented.It improves current serviceaccount model from many ways.
This patch adds limit to TokenRequest expiration time.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#63575
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64122, 64936, 65288, 65383). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix integer divide by zero panic
**What this PR does / why we need it**:
/kind bug
fix integer divide by zero panic when time.Since(start) < 1s
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64935
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 65032, 63471, 64104, 64672, 64427). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Make taint behavior consistent for NoSchedule
**What this PR does / why we need it**:
Make taint behavior consistent.
If `TaintNodesByCondition ` is enable, taint node with `NotReady:NoSchedule`.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#63420
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
David Eads