Automatic merge from submit-queue
Implement Daemonset history
~Depends on #45867 (the 1st commit, ignore it when reviewing)~ (already merged)
Ref https://github.com/kubernetes/community/pull/527/ and https://github.com/kubernetes/community/pull/594
@kubernetes/sig-apps-api-reviews @kubernetes/sig-apps-pr-reviews @erictune @kow3ns @lukaszo @kargakis
---
TODOs:
- [x] API changes
- [x] (maybe) Remove rollback subresource if we decide to do client-side rollback
- [x] deployment controller
- [x] controller revision
- [x] owner ref (claim & adoption)
- [x] history reconstruct (put revision number, hash collision avoidance)
- [x] de-dup history and relabel pods
- [x] compare ds template with history
- [x] hash labels (put it in controller revision, pods, and maybe deployment)
- [x] clean up old history
- [x] Rename status.uniquifier when we reach consensus in #44774
- [x] e2e tests
- [x] unit tests
- [x] daemoncontroller_test.go
- [x] update_test.go
- [x] ~(maybe) storage_test.go // if we do server side rollback~
kubectl part is in #46144
---
**Release note**:
```release-note
```
Automatic merge from submit-queue
Add initializer support to admission and uninitialized filtering to rest storage
Initializers are the opposite of finalizers - they allow API clients to react to object creation and populate fields prior to other clients seeing them.
High level description:
1. Add `metadata.initializers` field to all objects
2. By default, filter objects with > 0 initializers from LIST and WATCH to preserve legacy client behavior (known as partially-initialized objects)
3. Add an admission controller that populates .initializer values per type, and denies mutation of initializers except by certain privilege levels (you must have the `initialize` verb on a resource)
4. Allow partially-initialized objects to be viewed via LIST and WATCH for initializer types
5. When creating objects, the object is "held" by the server until the initializers list is empty
6. Allow some creators to bypass initialization (set initializers to `[]`), or to have the result returned immediately when the object is created.
The code here should be backwards compatible for all clients because they do not see partially initialized objects unless they GET the resource directly. The watch cache makes checking for partially initialized objects cheap. Some reflectors may need to change to ask for partially-initialized objects.
```release-note
Kubernetes resources, when the `Initializers` admission controller is enabled, can be initialized (defaulting or other additive functions) by other agents in the system prior to those resources being visible to other clients. An initialized resource is not visible to clients unless they request (for get, list, or watch) to see uninitialized resources with the `?includeUninitialized=true` query parameter. Once the initializers have completed the resource is then visible. Clients must have the the ability to perform the `initialize` action on a resource in order to modify it prior to initialization being completed.
```
1. Create controllerrevisions (history) and label pods with template
hash for both RollingUpdate and OnDelete update strategy
2. Clean up old, non-live history based on revisionHistoryLimit
3. Remove duplicate controllerrevisions (the ones with the same template)
and relabel their pods
4. Update RBAC to allow DaemonSet controller to manage
controllerrevisions
5. In DaemonSet controller unit tests, create new pods with hash labels
Automatic merge from submit-queue
Add local storage (scratch space) allocatable support
This PR adds the support for allocatable local storage (scratch space).
This feature is only for root file system which is shared by kubernetes
componenets, users' containers and/or images. User could use
--kube-reserved flag to reserve the storage for kube system components.
If the allocatable storage for user's pods is used up, some pods will be
evicted to free the storage resource.
This feature is part of local storage capacity isolation and described in the proposal https://github.com/kubernetes/community/pull/306
**Release note**:
```release-note
This feature exposes local storage capacity for the primary partitions, and supports & enforces storage reservation in Node Allocatable
```
Automatic merge from submit-queue (batch tested with PRs 46239, 46627, 46346, 46388, 46524)
Dynamic webhook admission control plugin
Unit tests pass.
Needs plumbing:
* [ ] service resolver (depends on @wfender PR)
* [x] client cert (depends on ????)
* [ ] hook source (depends on @caesarxuchao PR)
Also at least one thing will need to be renamed after Chao's PR merges.
```release-note
Allow remote admission controllers to be dynamically added and removed by administrators. External admission controllers make an HTTP POST containing details of the requested action which the service can approve or reject.
```
Automatic merge from submit-queue (batch tested with PRs 46239, 46627, 46346, 46388, 46524)
move labels to components which own the APIs
During the apimachinery split in 1.6, we accidentally moved several label APIs into apimachinery. They don't belong there, since the individual APIs are not general machinery concerns, but instead are the concern of particular components: most commonly the kubelet. This pull moves the labels into their owning components and out of API machinery.
@kubernetes/sig-api-machinery-misc @kubernetes/api-reviewers @kubernetes/api-approvers
@derekwaynecarr since most of these are related to the kubelet
Automatic merge from submit-queue
Added unit test for node operation in schedulercache.
Added unit test for node operation in schedulercache.
The code coverage is 62.4% (did not add cases for get/set and util.go which is used by algorithms.)
[combined-coverage.html.gz](https://github.com/kubernetes/kubernetes/files/975427/combined-coverage.html.gz)
Automatic merge from submit-queue (batch tested with PRs 41563, 45251, 46265, 46462, 46721)
Toleration should be `notReady:NoExecute` in defaulttolerationseconds…
… test.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Add support for creating resources that are not immediately visible to
naive clients, but must first be initialized by one or more privileged
cluster agents. These controllers can mark the object as initialized,
allowing others to see them.
Permission to override initialization defaults or modify an initializing
object is limited per resource to a virtual subresource "RESOURCE/initialize"
via RBAC.
Initialization is currently alpha.
This PR adds the check for local storage request when admitting pods. If
the local storage request exceeds the available resource, pod will be
rejected.
This PR adds the support for allocatable local storage (scratch space).
This feature is only for root file system which is shared by kubernetes
componenets, users' containers and/or images. User could use
--kube-reserved flag to reserve the storage for kube system components.
If the allocatable storage for user's pods is used up, some pods will be
evicted to free the storage resource.
As part of https://github.com/kubernetes/community/pull/132, thsi commit
adds a generic webhook admission controller. This plugin allows for a
completely declarative approach for filtering/matching admission requests
and for matching admission requests, calls out to an external webhook for
handling admission requests.
Automatic merge from submit-queue (batch tested with PRs 46076, 43879, 44897, 46556, 46654)
Local storage plugin
**What this PR does / why we need it**:
Volume plugin implementation for local persistent volumes. Scheduler predicate will direct already-bound PVCs to the node that the local PV is at. PVC binding still happens independently.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
Part of #43640
**Release note**:
```
Alpha feature: Local volume plugin allows local directories to be created and consumed as a Persistent Volume. These volumes have node affinity and pods will only be scheduled to the node that the volume is at.
```
Automatic merge from submit-queue
Move hardPodAffinitySymmetricWeight to scheduler policy config
**What this PR does / why we need it**:
Move hardPodAffinitySymmetricWeight to scheduler policy config
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#43845
**Special notes for your reviewer**:
If you like this, will add test later
**Release note**:
```
Move hardPodAffinitySymmetricWeight from KubeSchedulerConfiguration to scheduler Policy config
```
Automatic merge from submit-queue (batch tested with PRs 46552, 46608, 46390, 46605, 46459)
Move admission lifecycle to genericapi
**What this PR does / why we need it**: ends the whole sequence of moving some admission plugins to generic api.
**Release note**:
```release-note
NONE
```
lifecycle plugin: make use of the libraries under k8s.io/client-go/pkg/api and k8s.io/client-go/kubernetes
for the client libraries instead of k8s.io/kubernetes/client/*
move registration to AdmissionOptions
Automatic merge from submit-queue (batch tested with PRs 45891, 46147)
Watching ClusterId from within GCE cloud provider
**What this PR does / why we need it**:
Adds the ability for the GCE cloud provider to watch a config map for `clusterId` and `providerId`.
WIP - still needs more testing
cc @MrHohn @csbell @madhusudancs @thockin @bowei @nikhiljindal
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
remove init blocks from all admission plugins
**What this PR does / why we need it**:
removes init blocks from all admission plugins
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
removing generic_scheduler todo after discussion (#46027)
**What this PR does / why we need it**:
**Which issue this PR fixes** #46027
**Special notes for your reviewer**: just a quick clean cc @wojtek-t
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 45766, 46223)
Scheduler should use a shared informer, and fix broken watch behavior for cached watches
Can be used either from a true shared informer or a local shared
informer created just for the scheduler.
Fixes a bug in the cache watcher where we were returning the "current" object from a watch event, not the historic event. This means that we broke behavior when introducing the watch cache. This may have API implications for filtering watch consumers - but on the other hand, it prevents clients filtering from seeing objects outside of their watch correctly, which can lead to other subtle bugs.
```release-note
The behavior of some watch calls to the server when filtering on fields was incorrect. If watching objects with a filter, when an update was made that no longer matched the filter a DELETE event was correctly sent. However, the object that was returned by that delete was not the (correct) version before the update, but instead, the newer version. That meant the new object was not matched by the filter. This was a regression from behavior between cached watches on the server side and uncached watches, and thus broke downstream API clients.
```
Automatic merge from submit-queue (batch tested with PRs 46201, 45952, 45427, 46247, 46062)
Use shared informers in gc controller if possible
Modify the garbage collector controller to try to use shared informers for resources, if possible, to reduce the number of unique reflectors listing and watching the same thing.
cc @kubernetes/sig-api-machinery-pr-reviews @caesarxuchao @deads2k @liggitt @sttts @smarterclayton @timothysc @soltysh @kargakis @kubernetes/rh-cluster-infra @derekwaynecarr @wojtek-t @gmarek
Automatic merge from submit-queue
Moved qos to api.helpers.
**What this PR does / why we need it**:
The `GetPodQoS` is also used by other components, e.g. kube-scheduler and it's not bound to kubelet; moved it to api helpers so client-go.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #N/A
**Release note**:
```release-note-none
```
Automatic merge from submit-queue
Disambiguate schedule, assume, and bind in functions as well as in
Addresses my comments in #45972 about how these metrics need to be disambiguated.
- separates schedule, assume, and bind.
- renames variables like `dest` to be explicit.
- removes the logging statement to occur outside of the timed portion of the metric measurement.
Generally makes `sheduleOne` a happy function to read :)
Previously, the scheduler created two separate list watchers. This
changes the scheduler to be able to leverage a shared informer, whether
passed in externally or spawned using the new in place method. This
removes the last use of a "special" informer in the codebase.
Allows someone wrapping the scheduler to use a shared informer if they
have more information avaliable.
Automatic merge from submit-queue (batch tested with PRs 45346, 45903, 45958, 46042, 45975)
ResourceQuota admission control injects registry
**What this PR does / why we need it**:
The `ResourceQuota` admission controller works with a registry that maps a GroupKind to an Evaluator. The registry used in the existing plug-in is not injectable, which makes usage of the ResourceQuota plug-in in other API server contexts difficult. This PR updates the code to support late injection of the registry via a plug-in initializer.
Automatic merge from submit-queue (batch tested with PRs 45346, 45903, 45958, 46042, 45975)
Ignored mirror pods in PodPreset admission plugin
**What this PR does / why we need it**:
Ignored mirror pods in PodPreset admission plugin.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#45925
**Release note**:
```release-note
Ignored mirror pods in PodPreset admission plugin.
```
Automatic merge from submit-queue (batch tested with PRs 46028, 46104)
Use name from node object on create
GetName() isn't populated in admission attributes on create unless the rest storage is a NamedCreator (which only specific subresources are today)
Fixes#46085
Automatic merge from submit-queue (batch tested with PRs 45313, 46051)
Change all the existing admission init blocks to call a Register function
**What this PR does / why we need it**:
this is a two stage refactor when done there will be no init block in admission plugins.
Instead all plugins expose Register function which accept admission.Plugins instance.
The registration to global plugin registry happens inside Register function
**Release note**:
```
NONE
```
this is a two stage refactor when done there will be no init block in admission plugins.
Instead all plugins expose Register function which accept admission.Plugins instance.
The registration to global plugin registry happens inside Register func.
Automatic merge from submit-queue (batch tested with PRs 45374, 44537, 45739, 44474, 45888)
Allow kcm and scheduler to lock on ConfigMaps.
**What this PR does / why we need it**:
Plumbs through the ability to lock on ConfigMaps through the kcm and scheduler.
**Which issue this PR fixes**
Fixes: #44857
Addresses issues with: #45415
**Special notes for your reviewer**:
**Release note**:
```
Add leader-election-resource-lock support to kcm and scheduler to allow for locking on ConfigMaps as well as Endpoints(default)
```
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews @jamiehannaford @bsalamat @mikedanese
Automatic merge from submit-queue (batch tested with PRs 44337, 45775, 45832, 45574, 45758)
Tighten validation of mirror pod annotations
Tightens validation for pods with a mirror pod annotation:
1. spec.nodeName must be set
2. makes the mirror pod annotation immutable
3. starts validating pod-specific annotations during pod status update
None of these changes affect usage of the mirror pod annotation by kubelets, which only set it on pod creation (verified this is true back to 1.5.x)
the second commit updates the pod validation tests to look for specific error messages (best reviewed ignoring whitespace changes)
This is the validation portion of https://github.com/kubernetes/community/blob/master/contributors/design-proposals/kubelet-authorizer.md and https://github.com/kubernetes/features/issues/279
```release-note
Mirror pods must now indicate the nodeName they are bound to on creation. The mirror pod annotation is now treated as immutable and cannot be added to an existing pod, removed from a pod, or modified.
```
Automatic merge from submit-queue
Update scheduler reviewers
**What this PR does / why we need it**:
Update Scheduler OWNERS to more accurately reflect reality.
**Special notes for your reviewer**:
Are there others to add here?
**Release note**:
```
NONE
```
/cc @kubernetes/sig-scheduling-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41331, 45591, 45600, 45176, 45658)
Fixing typo in ImagePolicyWebhook admission.go
**What this PR does / why we need it**:
Noticed a typo in kubectl events when working with the ImagePolicyWebhook admission controller
**Special notes for your reviewer**:
first time contributor
Automatic merge from submit-queue (batch tested with PRs 41903, 45311, 45474, 45472, 45501)
Removed old scheduler constructor.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes # N/A
**Release note**:
```release-note-none
```
Automatic merge from submit-queue (batch tested with PRs 44727, 45409, 44968, 45122, 45493)
Total priority overflow check
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#24720
**Special notes for your reviewer**:
@adohe. I have borrowed some parts of your code in the closed PR and created this one.
**Release note**:
```release-note
This fixes the overflow for priorityconfig- valid range {1, 9223372036854775806}.
```
Automatic merge from submit-queue (batch tested with PRs 45100, 45152, 42513, 44796, 45222)
Added InterPodAffinity unit test case with Namespace.
**What this PR does / why we need it**:
Added InterPodAffinity unit test case with Namespace: unit test case for #45098
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes # N/A
**Release note**:
```release-note-none
```
Automatic merge from submit-queue
Align Extender's validation with prioritizers.
**What this PR does / why we need it**:
Align Extender's validation with prioritizers.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes # N/A
**Release note**:
```release-note-none
```
Automatic merge from submit-queue
Quote groups in deny log message
Helps distinguish cases where an external authenticator is incorrectly combining multiple groups into one.
Before:
> RBAC DENY: user "bob" groups [system:masters view system:authenticated] cannot "list" resource "pods" cluster-wide
After:
> RBAC DENY: user "bob" groups ["system:masters view" "system:authenticated"] cannot "list" resource "pods" cluster-wide
Automatic merge from submit-queue (batch tested with PRs 41530, 44814, 43620, 41985)
kube-apiserver: improve bootstrap token authentication error messages
This was requested by @jbeda as a follow up to https://github.com/kubernetes/kubernetes/pull/41281.
cc @jbeda @luxas @kubernetes/sig-auth-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41530, 44814, 43620, 41985)
no need check is nil, because has checked before
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
Improved code coverage for plugin/pkg/scheduler/algorithm
**What this PR does / why we need it**:
Part of #39559 , code coverage improved from 0% to 100%
**Special notes for your reviewer**:
Improved coverage for scheduler/algorithm to 100%
Test cover output:
```
make test WHAT=./plugin/pkg/scheduler/algorithm KUBE_COVER=y
Running tests for APIVersion: v1,apps/v1beta1,authentication.k8s.io/v1,authentication.k8s.io/v1beta1,authorization.k8s.io/v1,authorization.k8s.io/v1beta1,autoscaling/v1,autoscaling/v2alpha1,batch/v1,batch/v2alpha1,certificates.k8s.io/v1beta1,extensions/v1beta1,imagepolicy.k8s.io/v1alpha1,policy/v1beta1,rbac.authorization.k8s.io/v1beta1,rbac.authorization.k8s.io/v1alpha1,storage.k8s.io/v1beta1,federation/v1beta1
+++ [0302 10:43:05] Saving coverage output in '/tmp/k8s_coverage/v1,apps/v1beta1,authentication.k8s.io/v1,authentication.k8s.io/v1beta1,authorization.k8s.io/v1,authorization.k8s.io/v1beta1,autoscaling/v1,autoscaling/v2alpha1,batch/v1,batch/v2alpha1,certificates.k8s.io/v1beta1,extensions/v1beta1,imagepolicy.k8s.io/v1alpha1,policy/v1beta1,rbac.authorization.k8s.io/v1beta1,rbac.authorization.k8s.io/v1alpha1,storage.k8s.io/v1beta1,federation/v1beta1/20170302-104305'
skipped k8s.io/kubernetes/cmd/libs/go2idl/generator
skipped k8s.io/kubernetes/vendor/k8s.io/client-go/1.4/rest
ok k8s.io/kubernetes/plugin/pkg/scheduler/algorithm 0.061s coverage: 100.0% of statements
+++ [0302 10:43:07] Combined coverage report: /tmp/k8s_coverage/v1,apps/v1beta1,authentication.k8s.io/v1,authentication.k8s.io/v1beta1,authorization.k8s.io/v1,authorization.k8s.io/v1beta1,autoscaling/v1,autoscaling/v2alpha1,batch/v1,batch/v2alpha1,certificates.k8s.io/v1beta1,extensions/v1beta1,imagepolicy.k8s.io/v1alpha1,policy/v1beta1,rbac.authorization.k8s.io/v1beta1,rbac.authorization.k8s.io/v1alpha1,storage.k8s.io/v1beta1,federation/v1beta1/20170302-104305/combined-coverage.html
```
Automatic merge from submit-queue
Prepare for move zz_generated_deepcopy.go to k8s.io/api
This is in preparation to move deep copies to with the types to the types repo (see https://github.com/kubernetes/gengo/pull/47#issuecomment-296855818). The init() function is referring the `SchemeBuilder` defined in the register.go in the same packge, so we need to revert the dependency.
This PR depends on https://github.com/kubernetes/gengo/pull/49, otherwise verification will fail.
Kubernetes Submit Queue