Automatic merge from submit-queue
Extend experimental support to multiple Nvidia GPUs
Extended from #28216
```release-note
`--experimental-nvidia-gpus` flag is **replaced** by `Accelerators` alpha feature gate along with support for multiple Nvidia GPUs.
To use GPUs, pass `Accelerators=true` as part of `--feature-gates` flag.
Works only with Docker runtime.
```
1. Automated testing for this PR is not possible since creation of clusters with GPUs isn't supported yet in GCP.
1. To test this PR locally, use the node e2e.
```shell
TEST_ARGS='--feature-gates=DynamicKubeletConfig=true' FOCUS=GPU SKIP="" make test-e2e-node
```
TODO:
- [x] Run manual tests
- [x] Add node e2e
- [x] Add unit tests for GPU manager (< 100% coverage)
- [ ] Add unit tests in kubelet package
- Add a new type PortworxVolumeSource
- Implement the kubernetes volume plugin for Portworx Volumes under pkg/volume/portworx
- The Portworx Volume Driver uses the libopenstorage/openstorage specifications and apis for volume operations.
Changes for k8s configuration and examples for portworx volumes.
- Add PortworxVolume hooks in kubectl, kube-controller-manager and validation.
- Add a README for PortworxVolume usage as PVs, PVCs and StorageClass.
- Add example spec files
Handle code review comments.
- Modified READMEs to incorporate to suggestions.
- Add a test for ReadWriteMany access mode.
- Use util.UnmountPath in TearDown.
- Add ReadOnly flag to PortworxVolumeSource
- Use hostname:port instead of unix sockets
- Delete the mount dir in TearDown.
- Fix link issue in persistentvolumes README
- In unit test check for mountpath after Setup is done.
- Add PVC Claim Name as a Portworx Volume Label
Generated code and documentation.
- Updated swagger spec
- Updated api-reference docs
- Updated generated code under pkg/api/v1
Godeps update for Portworx Volume Driver
- Adds github.com/libopenstorage/openstorage
- Adds go.pedge.io/pb/go/google/protobuf
- Updates Godep Licenses
Automatic merge from submit-queue (batch tested with PRs 41234, 42186, 41615, 42028, 41788)
apimachinery: handle duplicated and conflicting type registration
Double registrations were leading to duplications in `KnownKinds()`. Conflicting registrations with same gvk, but different types were not detected.
Automatic merge from submit-queue (batch tested with PRs 40932, 41896, 41815, 41309, 41628)
Modify CronJob API to add job history limits, cleanup jobs in controller
**What this PR does / why we need it**:
As discussed in #34710: this adds two limits to `CronJobSpec`, to limit the number of finished jobs created by a CronJob to keep.
**Which issue this PR fixes**: fixes#34710
**Special notes for your reviewer**:
cc @soltysh, please have a look and let me know what you think -- I'll then add end to end testing and update the doc in a separate commit. What is the timeline to get this into 1.6?
The plan:
- [x] API changes
- [x] Changing versioned APIs
- [x] `types.go`
- [x] `defaults.go` (nothing to do)
- [x] `conversion.go` (nothing to do?)
- [x] `conversion_test.go` (nothing to do?)
- [x] Changing the internal structure
- [x] `types.go`
- [x] `validation.go`
- [x] `validation_test.go`
- [x] Edit version conversions
- [x] Edit (nothing to do?)
- [x] Run `hack/update-codegen.sh`
- [x] Generate protobuf objects
- [x] Run `hack/update-generated-protobuf.sh`
- [x] Generate json (un)marshaling code
- [x] Run `hack/update-codecgen.sh`
- [x] Update fuzzer
- [x] Actual logic
- [x] Unit tests
- [x] End to end tests
- [x] Documentation changes and API specs update in separate commit
**Release note**:
```release-note
Add configurable limits to CronJob resource to specify how many successful and failed jobs are preserved.
```
Automatic merge from submit-queue (batch tested with PRs 42106, 42094, 42069, 42098, 41852)
Fix availableReplicas validation
An available replica is a ready replica, not the other way around
@kubernetes/sig-apps-bugs caught while testing https://github.com/kubernetes/kubernetes/pull/42097
Automatic merge from submit-queue (batch tested with PRs 41854, 41801, 40088, 41590, 41911)
Add storage.k8s.io/v1 API
v1 API is direct copy of v1beta1 API. This v1 API gets installed and exposed in this PR, I tested that kubectl can create both v1beta1 and v1 StorageClass.
~~Rest of Kubernetes (controllers, examples,. tests, ...) still use v1beta1 API, I will update it when this PR gets merged as these changes would get lost among generated code.~~ Most parts use v1 API now, it would not compile / run tests without it.
**Release note**:
```
Kubernetes API storage.k8s.io for storage objects is now fully supported and is available as storage.k8s.io/v1. Beta version of the API storage.k8s.io/v1beta1 is still available in this release, however it will be removed in a future Kubernetes release.
Together with the API endpoint, StorageClass annotation "storageclass.beta.kubernetes.io/is-default-class" is deprecated and "storageclass.kubernetes.io/is-default-class" should be used instead to mark a default storage class. The beta annotation is still working in this release, however it won't be supported in the next one.
```
@kubernetes/sig-storage-misc
Automatic merge from submit-queue
Add ClassName attributes to PV and PVC
This just adds new attributes to PV/PVC. Real code that uses the attributes instead of beta annotations will follow when we agree on the attribute names / style.
Automatic merge from submit-queue
Include all user.Info data in CSR object
In order to use authorization checks to auto-approve CSRs in the future, we need all the info from the user.Info interface.
This mirrors the API fields in the TokenReview API used to return user info, and in the SubjectAccessReview API we use to check authorization.
```release-note
The CertificateSigningRequest API added the `extra` field to persist all information about the requesting user. This mirrors the fields in the SubjectAccessReview API used to check authorization.
```
Automatic merge from submit-queue (batch tested with PRs 38957, 41819, 41851, 40667, 41373)
Change taints/tolerations to api fields
This PR changes current implementation of taints and tolerations from annotations to API fields. Taint and toleration are now part of `NodeSpec` and `PodSpec`, respectively. The annotation keys: `scheduler.alpha.kubernetes.io/tolerations` and `scheduler.alpha.kubernetes.io/taints` have been removed.
**Release note**:
Pod tolerations and node taints have moved from annotations to API fields in the PodSpec and NodeSpec, respectively. Pod tolerations and node taints that are defined in the annotations will be ignored. The annotation keys: `scheduler.alpha.kubernetes.io/tolerations` and `scheduler.alpha.kubernetes.io/taints` have been removed.
Automatic merge from submit-queue
Convert HPA controller to support HPA v2 mechanics
This PR converts the HPA controller to support the mechanics from HPA v2.
The HPA controller continues to make use of the HPA v1 client, but utilizes
the conversion logic to work with autoscaling/v2alpha1 objects internally.
It is the follow-up PR to #36033 and part of kubernetes/features#117.
**Release note**:
```release-note
NONE
```
There was a bug in the HPA v1 conversion logic that would occur when
a custom metric and a metric that was encoded in v1 as
targetCPUUtilizationPercentage were used at the same time. In this
case, the custom metric could overwrite the CPU metric, or vice versa.
This fixes that bug, and ensures that the fuzzer tests round-tripping
with multiple metrics.
Automatic merge from submit-queue (batch tested with PRs 41401, 41195, 41664, 41521, 41651)
Remove default failure domains from anti-affinity feature
Removing it is necessary to make performance of this feature acceptable at some point.
With default failure domains (or in general when multiple topology keys are possible), we don't have transitivity between node belonging to a topology. And without this, it's pretty much impossible to solve this effectively.
@timothysc
Automatic merge from submit-queue
Remove alpha provisioning
This is the first part of https://github.com/kubernetes/features/issues/36
@kubernetes/sig-storage-misc
**Release note**:
```release-note
Alpha version of dynamic volume provisioning is removed in this release. Annotation
"volume.alpha.kubernetes.io/storage-class" does not have any special meaning. A default storage class
and DefaultStorageClass admission plugin can be used to preserve similar behavior of Kubernetes cluster,
see https://kubernetes.io/docs/user-guide/persistent-volumes/#class-1 for details.
```
Automatic merge from submit-queue (batch tested with PRs 37137, 41506, 41239, 41511, 37953)
Add field to control service account token automounting
Fixes https://github.com/kubernetes/kubernetes/issues/16779
* adds an `automountServiceAccountToken *bool` field to `ServiceAccount` and `PodSpec`
* if set in both the service account and pod, the pod wins
* if unset in both the service account and pod, we automount for backwards compatibility
```release-note
An `automountServiceAccountToken *bool` field was added to ServiceAccount and PodSpec objects. If set to `false` on a pod spec, no service account token is automounted in the pod. If set to `false` on a service account, no service account token is automounted for that service account unless explicitly overridden in the pod spec.
```
Automatic merge from submit-queue
Allow multipe DNS servers as comma-seperated argument for kubelet --dns
This PR explores how kubectls "--dns" could be extended to specify multiple DNS servers for in-cluster PODs. Testing on the local libvirt-coreos cluster shows that multiple DNS server are injected without issues.
Specifying multiple DNS servers increases resilience against
- Packet drops
- Single server failure
I am debugging services that do 50+ DNS requests for a single incoming interactive request, thus highly increase the chance of a slowdown (+5s) due to a single packet drop. Switching to two DNS servers will reduce the impact of the issues (roughly +1s on glibc, 0s on musl, error-rate goes down to error-rate^2).
Note that there is no need to change any runtime related code as far as I know. In the case of "default" dns the /etc/resolv.conf is parsed and multiple DNS server are send to the backend anyway. This only adds the same capability for the clusterFirst case.
I've heard from @thockin that multiple DNS entries are somehow considered. I've no idea what was considered, though. This is what I would like to see for our production use, though.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41378, 41413, 40743, 41155, 41385)
Reconcile bootstrap clusterroles on server start
Currently, on server start, bootstrap roles and bindings are only created if there are no existing roles or rolebindings.
Instead, we should look at each bootstrap role and rolebinding, and ensure it exists and has required permissions and subjects at server start. This allows seamless upgrades to new versions that define roles for new controllers, or add permissions to existing roles.
```release-note
Default RBAC ClusterRole and ClusterRoleBinding objects are automatically updated at server start to add missing permissions and subjects (extra permissions and subjects are left in place). To prevent autoupdating a particular role or rolebinding, annotate it with `rbac.authorization.kubernetes.io/autoupdate=false`.
```
This change makes kubelet to use the CRI implementation by default,
unless the users opt out explicitly by using --enable-cri=false.
For the rkt integration, the --enable-cri flag will have no effect
since rktnetes does not use CRI.
Also, mark the original --experimental-cri flag hidden and deprecated,
so that we can remove it in the next release.
Automatic merge from submit-queue (batch tested with PRs 39418, 41175, 40355, 41114, 32325)
TaintController
```release-note
This PR adds a manager to NodeController that is responsible for removing Pods from Nodes tainted with NoExecute Taints. This feature is beta (as the rest of taints) and enabled by default. It's gated by controller-manager enable-taint-manager flag.
```
Automatic merge from submit-queue (batch tested with PRs 41112, 41201, 41058, 40650, 40926)
Promote TokenReview to v1
Peer to https://github.com/kubernetes/kubernetes/pull/40709
We have multiple features that depend on this API:
- [webhook authentication](https://kubernetes.io/docs/admin/authentication/#webhook-token-authentication)
- [kubelet delegated authentication](https://kubernetes.io/docs/admin/kubelet-authentication-authorization/#kubelet-authentication)
- add-on API server delegated authentication
The API has been in use since 1.3 in beta status (v1beta1) with negligible changes:
- Added a status field for reporting errors evaluating the token
This PR promotes the existing v1beta1 API to v1 with no changes
Because the API does not persist data (it is a query/response-style API), there are no data migration concerns.
This positions us to promote the features that depend on this API to stable in 1.7
cc @kubernetes/sig-auth-api-reviews @kubernetes/sig-auth-misc
```release-note
The authentication.k8s.io API group was promoted to v1
```
Automatic merge from submit-queue (batch tested with PRs 40796, 40878, 36033, 40838, 41210)
StatefulSet hardening
**What this PR does / why we need it**:
This PR contains the following changes to StatefulSet. Only one change effects the semantics of how the controller operates (This is described in #38418), and this change only brings the controller into conformance with its documented behavior.
1. pcb and pcb controller are removed and their functionality is encapsulated in StatefulPodControlInterface. This class modules the design contoller.PodControlInterface and provides an abstraction to clientset.Interface which is useful for testing purposes.
2. IdentityMappers has been removed to clarify what properties of a Pod are mutated by the controller. All mutations are performed in the UpdateStatefulPod method of the StatefulPodControlInterface.
3. The statefulSetIterator and petQueue classes are removed. These classes sorted Pods by CreationTimestamp. This is brittle and not resilient to clock skew. The current control loop, which implements the same logic, is in stateful_set_control.go. The Pods are now sorted and considered by their ordinal indices, as is outlined in the documentation.
4. StatefulSetController now checks to see if the Pods matching a StatefulSet's Selector also match the Name of the StatefulSet. This will make the controller resilient to overlapping, and will be enhanced by the addition of ControllerRefs.
5. The total lines of production code have been reduced, and the total number of unit tests has been increased. All new code has 100% unit coverage giving the module 83% coverage. Tests for StatefulSetController have been added, but it is not practical to achieve greater coverage in unit testing for this code (the e2e tests for StatefulSet cover these areas).
6. Issue #38418 is fixed in that StaefulSet will ensure that all Pods that are predecessors of another Pod are Running and Ready prior to launching a new Pod. This removes the potential for deadlock when a Pod needs to be rescheduled while its predecessor is hung in Pending or Initializing.
7. All reference to pet have been removed from the code and comments.
**Which issue this PR fixes**
fixes #38418,#36859
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixes issue #38418 which, under circumstance, could cause StatefulSet to deadlock.
Mediates issue #36859. StatefulSet only acts on Pods whose identity matches the StatefulSet, providing a partial mediation for overlapping controllers.
```
Automatic merge from submit-queue (batch tested with PRs 40796, 40878, 36033, 40838, 41210)
HPA v2 (API Changes)
**Release note**:
```release-note
Introduces an new alpha version of the Horizontal Pod Autoscaler including expanded support for specifying metrics.
```
Implements the API changes for kubernetes/features#117.
This implements #34754, which is the new design for the Horizontal Pod Autoscaler. It includes improved support for custom metrics (and/or arbitrary metrics) as well as expanded support for resource metrics. The new HPA object is introduces in the API group "autoscaling/v1alpha1".
Note that the improved custom metric support currently is limited to per pod metrics from Heapster -- attempting to use the new "object metrics" will simply result in an error. This will change once #34586 is merged and implemented.
1. pcb and pcb controller are removed and their functionality is
encapsulated in StatefulPodControlInterface.
2. IdentityMappers has been removed to clarify what properties of a Pod are
mutated by the controller. All mutations are performed in the
UpdateStatefulPod method of the StatefulPodControlInterface.
3. The statefulSetIterator and petQueue classes are removed. These classes
sorted Pods by CreationTimestamp. This is brittle and not resilient to
clock skew. The current control loop, which implements the same logic,
is in stateful_set_control.go. The Pods are now sorted and considered by
their ordinal indices, as is outlined in the documentation.
4. StatefulSetController now checks to see if the Pods matching a
StatefulSet's Selector also match the Name of the StatefulSet. This will
make the controller resilient to overlapping, and will be enhanced by
the addition of ControllerRefs.
Automatic merge from submit-queue (batch tested with PRs 40943, 40967)
switch admission serialized config to an apiserver type
Switches the kube admission config from componentconfig to the new `apiserver.k8s.io` group so that all API servers can use the shared configuration. This switch external serialization, but it does not move the code that reads the config. I'd like to do that as a follow-on.
@kubernetes/sig-api-machinery-misc @kubernetes/api-reviewers @smarterclayton
@derekwaynecarr ptal
@sttts
Depending on an exact cluster setup multiple dns may make sense.
Comma-seperated lists of DNS server are quite common as DNS servers
are always plain IPs.
Automatic merge from submit-queue (batch tested with PRs 40392, 39242, 40579, 40628, 40713)
[Federation][kubefed] Create a dedicated service account for federation controller manager in the host cluster and give it appropriate permissions.
Ref: Issue #39555
cc @kubernetes/sig-federation-pr-reviews @kubernetes/sig-auth-misc @kubernetes/sig-auth-pr-reviews
```release-note
kubefed init creates a service account for federation controller manager in the federation-system namespace and binds that service account to the federation-system:federation-controller-manager role that has read and list access on secrets in the federation-system namespace.
```
Automatic merge from submit-queue
genericapiserver: cut off more dependencies – episode 3
Compare commit subjects.
This is the first step to get `apiserver_test.go` and `watch_test.go` in `pkg/genericapiserver/endpoints` cutoff from k8s.io/kubernetes dependencies.
After this we have to sync client-go and then "episode 4" can go in.
approved based on #40363
Automatic merge from submit-queue
Update authn/authz owners and reviewers
Add myself as potential reviewer to several authn/authz-related packages.
Add @deads2k as approver to several packages he wrote.
Automatic merge from submit-queue
Remove proxy-mode annotation from kube-proxy
**What this PR does / why we need it**:
This removes the net.experimental.kubernetes.io/proxy-mode and net.beta.kubernetes.io/proxy-mode annotations from kube-proxy.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#40582
**Special notes for your reviewer**:
**Release note**:
```release-note
Remove outdated net.experimental.kubernetes.io/proxy-mode and net.beta.kubernetes.io/proxy-mode annotations from kube-proxy.
```
Automatic merge from submit-queue (batch tested with PRs 40497, 39769, 40554, 40569, 40597)
NetworkPolicy validation improvements
I noticed while implementing NetworkPolicy that I we weren't validating the "Ports" field at all.
The docs are actually completely silent about what a string-valued Port field is supposed to mean. I had guessed it meant to call `net.LookupPort()` on it (ie, map it from /etc/services) but in every other case where we have an IntOrString-valued Port field in an API struct, it refers to a named ContainerPort. But that would be extremely awkward to implement in this case; a policy specifying a named port could end up mapping to a different numeric port on every container in the namespace... Do other people actually implement string-valued ports that way? Or, for that matter, implement string-valued ports at all? (Related: I hadn't noticed until now that you can leave the Port value unspecified, allowing you to say "allow to all UDP ports, but no TCP ports" or "allow to all TCP ports, but no UDP ports". That seems like something that ended up in the spec just because it was possible, not because it was actually useful...)
@kubernetes/sig-network-misc
Automatic merge from submit-queue (batch tested with PRs 39469, 40557)
Forgiveness api changes
**What this PR does / why we need it**:
Splited from #34825 , contains api changes that are needed to implement forgiveness:
1. update toleration api types to support forgiveness, added a new field forgivenessSeconds to indicate the duration of time it tolerates a taint.
2. update taint api types, added a new field to indicate the time the taint is added.
**Which issue this PR fixes** :
Related issue: #1574
Related PR: #34825
**Special notes for your reviewer**:
**Release note**:
```release-note
forgiveness alpha version api definition
```
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
move apimachinery/announced to apimachinery
This also rewires the announce package to allow installation into multiple groups via multiple calls to `install.Install`. At some point, we'll want to unwire the hardcoded, "install to this scheme" call.
@lavalamp something we've wanted for a while and need for genericapiserver
@seh you've asked related questions
@sttts ptal. First commit is interesting, second commit is a straight move.
Automatic merge from submit-queue
promote certificates api to beta
Mostly posting to see what breaks but also this API is ready to be promoted.
```release-note
Promote certificates.k8s.io to beta and enable it by default. Users using the alpha certificates API should delete v1alpha1 CSRs from the API before upgrading and recreate them as v1beta1 CSR after upgrading.
```
@kubernetes/api-approvers @jcbsmpsn @pipejakob
Automatic merge from submit-queue (batch tested with PRs 39807, 37505, 39844, 39525, 39109)
Admission control support for versioned configuration files
**What this PR does / why we need it**:
Today, the `--admission-control-config-file=` argument takes an opaque file that is shared across all admission controllers to provide configuration. This file is not well-versioned and it's shared across multiple plug-ins. Some plugins take file based configuration (`ImagePolicyWebhook`) and others abuse flags to provide configuration because we lacked a good example (`InitialResources`). This PR defines a versioned configuration format that we can use moving forward to provide configuration input to admission controllers that is well-versioned, and does not require the addition of new flags.
The sample configuration file would look as follows:
```
apiVersion: componentconfig/v1alpha1
kind: AdmissionConfiguration
plugins:
- name: "ImagePolicyWebhook"
path: "image-policy-webhook.json"
```
The general behavior is each plugin that requires additional configuration is enumerated by name. An alternate file location is provided for its specific configuration, or the configuration can be embedded as a raw extension via the configuration section.
**Special notes for your reviewer**:
A follow-on PR will be needed to make `ImagePolicyWebhook` to use versioned configuration. This PR maintains backwards compatibility by ignoring configuration it cannot understand and therefore treating the file as opaque. I plan to make use of this PR to complete https://github.com/kubernetes/kubernetes/pull/36765 which attempts to allow more configuration parameters to the `ResourceQuota` admission plugin.
Automatic merge from submit-queue
replace global registry in apimachinery with global registry in k8s.io/kubernetes
We'd like to remove all globals, but our immediate problem is that a shared registry between k8s.io/kubernetes and k8s.io/client-go doesn't work. Since client-go makes a copy, we can actually keep a global registry with other globals in pkg/api for now.
@kubernetes/sig-api-machinery-misc @lavalamp @smarterclayton @sttts
Automatic merge from submit-queue (batch tested with PRs 39803, 39698, 39537, 39478)
[scheduling] Moved pod affinity and anti-affinity from annotations to api fields #25319
Converted pod affinity and anti-affinity from annotations to api fields
Related: #25319
Related: #34508
**Release note**:
```Pod affinity and anti-affinity has moved from annotations to api fields in the pod spec. Pod affinity or anti-affinity that is defined in the annotations will be ignored.```
Automatic merge from submit-queue (batch tested with PRs 39803, 39698, 39537, 39478)
default a CSR's allowed usage to key encipherment and digital signing
Some pretty safe and sane defaults.
@liggitt
Maciej Szulik