Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove Google Cloud KMS's in-tree integration
Removes the following introduced by #48574 and others:
* `kms.go` which contained the cloudkms-specific code for Google Cloud KMS service.
* Registering the Google Cloud KMS in the KMS plugin registry.
* Google's `cloudkms` API package from `vendor` folder.
The following changes are upcoming:
* Removal of KMSPluginRegistry. This would not be needed anymore, since KMS providers will be out-of-tree from now on (so no need of registering them, an address of the process would be enough).
* A service which allows encrypt/decrypt functionality (satisfies `envelope.Service` interface) if initialized with an IP/Port of an out-of-tree process serving KMS requests. Will tentatively use gRPC requests to talk to this external service.
Reference: https://github.com/kubernetes/kubernetes/pull/54439#issuecomment-340062801 and https://github.com/kubernetes/kubernetes/issues/51965#issuecomment-339333937.
```release-note
Google KMS integration was removed from in-tree in favor of a out-of-process extension point that will be used for all KMS providers.
```
Automatic merge from submit-queue (batch tested with PRs 52880, 52855, 52761, 52885, 52929). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Remove cloud provider rackspace
**What this PR does / why we need it**:
For now, we have to implement functions in both `rackspace` and `openstack` packages if we want to add function for cinder, for example [resize for cinder](https://github.com/kubernetes/kubernetes/pull/51498). Since openstack has implemented all the functions rackspace has, and rackspace is considered deprecated for a long time, [rackspace deprecated](https://github.com/rackspace/gophercloud/issues/592) ,
after talking with @mikedanese and @jamiehannaford offline , i sent this PR to remove `rackspace` in favor of `openstack`
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#52854
**Special notes for your reviewer**:
**Release note**:
```release-note
The Rackspace cloud provider has been removed after a long deprecation period. It was deprecated because it duplicates a lot of the OpenStack logic and can no longer be maintained. Please use the OpenStack cloud provider instead.
```
Automatic merge from submit-queue (batch tested with PRs 52168, 48939, 51889, 52051, 50396). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Add Windows Server Containers Stats and Metrics to Kubelet
**What this PR does / why we need it**:
This PR implements stats for Windows Server Containers. This adds the ability to monitor Windows Server containers via the existing stats/summary endpoint inside the kubelet. Windows metrics can now be ingested into heapster and monitored using existing tools (like Grafana).
Previously, the /stats/summary api would consistently crash the kubelet on Windows server containers. This PR implements a new package "winstats" which reads windows server metrics from a combination of windows specific perf counters as well as docker stats. The "winstats" package exports functions that return CAdvisor data structures, which the existing summary api can read.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#49398
This PR addresses my plan to implement windows server container stats https://github.com/kubernetes/kubernetes/issues/49398 .
**Release note**:
```release-note
Add monitoring of Windows Server containers metrics in the kubelet via the stats/summary endpoint.
```
Automatic merge from submit-queue (batch tested with PRs 51064, 52132). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Kubelet weird output fix
**What this PR does / why we need it**:
All kube binaries print the following message at the top when executed:
```
2017/08/28 21:37:12 proto: duplicate proto type registered: google.protobuf.Any
2017/08/28 21:37:12 proto: duplicate proto type registered: google.protobuf.Duration
2017/08/28 21:37:12 proto: duplicate proto type registered: google.protobuf.Timestamp
```
This PR addresses the above issue
It involves the following two changes:
1. Updating vendor packages
- Removes everything under the repo ``go.pedge.io``
- Updates repo libopenstorage/openstorage (which was using the above package)
2. Updates the portworx volume native driver to adhere to the newly vendor'ed code.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
fixes#51452
**Special notes for your reviewer**:
I have divided the changes into 2 commits
1. Vendor'ed changes
2. Changes in portworx volume driver.
```release-note
Remove duplicate proto errors in kubelet.
```
Automatic merge from submit-queue (batch tested with PRs 51064, 52132). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Fix Kubeproxy to work for Windows Kernel mode
**What this PR does / why we need it**:
Kubeproxy doenst work for with windows kernel mode. This PR adds a Kernel Proxy for windows to use the underlying platform features.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
#49666
**Special notes for your reviewer**:
**Release note**:
```release-note
```
This implements stats for windows nodes in a new package, winstats.
WinStats exports methods to get cadvisor like datastructures, however
with windows specific metrics. WinStats only gets node level metrics and
information, container stats will go via the CRI. This enables the
use of the summary api to get metrics for windows nodes.
Automatic merge from submit-queue (batch tested with PRs 51480, 49616, 50123, 50846, 50404)
Kubectl to use http caching to cache openapi responses from the server
**What this PR does / why we need it**:
This PR is trying to address the problems raised in #50254
> * uses a disk-based cache that is not safe between processes (does not use atomic fs operations)
> * writes get/list responses to disk that should not be cached (like kubectl get secrets)
> * is vulnerable to partially written cache responses being used as responses to future requests
> * breaks uses of the client transport that make use of websockets
> * defaults to enabling the cache for any client builder using RecommendedConfigOverrideFlags or DefaultClientConfig which affects more components than just kubectl
All of these points are addressed by this pull-request:
1. It now uses atomic fs operations
2. Doesn't cache by default, only if requested by the client (and it's only done by openapi client)
3. Fixed because of atomic fs operations
4. Found the reason for the bug: Cache wrapper couldn't be unwrapped. I implemented the `WrappedRoundTripper` interface.
5. Since 2. is fixed, I think that should be fine
@smarterclayton @liggitt
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#50254
**Special notes for your reviewer**:
**Release note**:
```release-note
Allows kubectl to use http caching mechanism for the OpenAPI schema. The cache directory can be configured through `--cache-dir` command line flag to kubectl. If set to empty string, caching will be disabled.
```
Automatic merge from submit-queue
AWS: check validity of KSM key before creating a new encrypted disk.
AWS CreateVolume call does not check if referenced encryption key actually exists and returns a valid new AWS EBS volume even though an invalid key was specified. Later on it removes the EBS silently when its encryption fails.
To work around this buggy behavior we manually check that the key exists before calling CreateVolume.
Fixes#48438
/sig aws
Please review carefully. Can we safely assume that Kubernetes controller-manager can read encryption keys?
```release-note
aws: Kubernetes now checks existence of provided KSM (Key Management Service) key before creating an encrypted AWS EBS.
```
AWS CreateVolume call does not check if referenced encryption key actually
exists and returns a valid new AWS EBS volume even though an invalid key
was specified. Later on it removes the EBS silently when its encryption fails.
To work around this buggy behavior we manually check that the key exists
before calling CreateVolume.
Automatic merge from submit-queue (batch tested with PRs 48292, 48121)
Add Google cloudkms dependency, add cloudkms service to GCE cloud provider
Required to introduce a Google KMS based envelope encryption, which shall allow encrypting secrets at rest using KEK-DEK scheme.
The above requires KMS API to create/delete KeyRings and CryptoKeys, and Encrypt/Decrypt data.
Should target release 1.8
@jcbsmpsn
Update: It appears that Godep only allows dependencies which are in use. We may have to modify this PR to include some Google KMS code.
Progresses #48522
Automatic merge from submit-queue
Use Docker API Version instead of docker version
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Fixes#42492
**Special notes for your reviewer**:
**Release note**:
`Update cadvisor to latest head to use docker APIversion exposed by cadvisor`
Automatic merge from submit-queue
bump(golang.org/x/oauth2): a6bd8cefa1811bd24b86f8902872e4e8225f74c4
As I tackle https://github.com/kubernetes/kubernetes/issues/42654 kubectl's OpenID Connect plugin will start using golang.org/x/oauth2 for refreshing, instead of go-oidc's own hand rolled oauth2 implementation. In preparation, update golang.org/x/oauth2 to include 7374b3f1ec which fixes refreshing with Okta.
We also somehow removed the dependency on `google.golang.org/appengine`. Maybe 8cf58155e4?
cc @kubernetes/sig-auth-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 42202, 40784, 44642, 44623, 44761)
Update CNI plugin to newest version; support ConfigLists
**What this PR does / why we need it**: Updates the CNI network plugin to use the newest version of CNI. This brings with it plugin chaining and support for multiple versions.
**Special notes for your reviewer**: This libcni change is backwards-compatible - older plugins will work without any changes needed.
**Release note**:
```release-note
This adds support for CNI ConfigLists, which permit plugin chaining.
```
** reason for this change **
CNI has recently introduced a new configuration list feature. This
allows for plugin chaining. It also supports varied plugin versions.
Automatic merge from submit-queue (batch tested with PRs 44440, 44038, 44302, 44316, 43876)
Move pkg/api/ref.go and pkg/api/resource_helpers.go to subpackages
First two commits are #44296#44299. (unfortunately, removing these commits results in conflicts)
This PR moves resource_helpers.go to a subpackage. It's mostly a mechanic move, except that:
* i kept the methods of ResourceName and ResourceList in pkg/api/method.go
I'll send one more PR to separate api.Scheme etc. to their own package.
Automatic merge from submit-queue (batch tested with PRs 44424, 44026, 43939, 44386, 42914)
remove defaulting from conversion path
follow up for #42764
* remove call to defaulting from conversion path (defaulting is a separate step from conversion)
* remove non-top-level-object defaulting registration (unused after conversion call is removed)
* generate missing top-level defaults for some api groups:
* autoscaling/v2alpha1
* policy/v1alpha1
* policy/v1beta1
* register top-level defaults for some api groups that were missing them:
* autoscaling/v2alpha1
* settings/v1alpha1
Automatic merge from submit-queue (batch tested with PRs 44406, 41543, 44071, 44374, 44299)
Move pkg/api/ref.go to a subpackage
First commit is #44296. (unfortunately, removing that commit results in conflicts)
This PR moves the pkg/api/ref.go to its own subpackage. It's mostly a mechanic move.
I'll send a few more PRs to make the k8s.io/kubernetes/pkg/api package only contains the code we want in the k8s.io/api repo, then we can run a [script](a0015fd1be (diff-7a2fbb4371972350ee414c6b88aee1c8)) to cut the new repo.
Automatic merge from submit-queue (batch tested with PRs 44406, 41543, 44071, 44374, 44299)
Decouple remotecommand
Refactored unversioned/remotecommand to decouple it from undesirable dependencies:
- term package now is not required, and functionality required to resize terminal size can be plugged in directly in kubectl
- in order to remove dependency on kubelet package - constants from kubelet/server/remotecommand were moved to separate util package (pkg/util/remotecommand)
- remotecommand_test.go moved to pkg/client/tests module
Automatic merge from submit-queue
Move api helpers.go to a subpackage
Part of https://github.com/kubernetes/kubernetes/issues/44065.
This PR moves the pkg/api/helpers.go to its own subpackage. It's mostly a mechanic move, except that
* I removed ConversionError in helpers.go, it's not used by anyone
* I moved the 3 methods of Taint and Toleration to pkg/api/methods.go, and left a TODO saying refactoring these methods to functions.
I'll send a few more PRs to make the k8s.io/kubernetes/pkg/api package only contains the code we want in the k8s.io/api repo, then we can run a [script](a0015fd1be (diff-7a2fbb4371972350ee414c6b88aee1c8)) to cut the new repo.
Automatic merge from submit-queue
Add support for IP aliases for pod IPs (GCP alpha feature)
```release-note
Adds support for allocation of pod IPs via IP aliases.
# Adds KUBE_GCE_ENABLE_IP_ALIASES flag to the cluster up scripts (`kube-{up,down}.sh`).
KUBE_GCE_ENABLE_IP_ALIASES=true will enable allocation of PodCIDR ips
using the ip alias mechanism rather than using routes. This feature is currently
only available on GCE.
## Usage
$ CLUSTER_IP_RANGE=10.100.0.0/16 KUBE_GCE_ENABLE_IP_ALIASES=true bash -x cluster/kube-up.sh
# Adds CloudAllocator to the node CIDR allocator (kubernetes-controller manager).
If CIDRAllocatorType is set to `CloudCIDRAllocator`, then allocation
of CIDR allocation instead is done by the external cloud provider and
the node controller is only responsible for reflecting the allocation
into the node spec.
- Splits off the rangeAllocator from the cidr_allocator.go file.
- Adds cloudCIDRAllocator, which is used when the cloud provider allocates
the CIDR ranges externally. (GCE support only)
- Updates RBAC permission for node controller to include PATCH
```
Automatic merge from submit-queue (batch tested with PRs 43273, 44287, 44281)
Remove ObjectMetaFor
Builds on https://github.com/kubernetes/kubernetes/pull/43767
The second commit removes `ObjectMetaFor`. This was debt we left around after we created the interfaces. Fixing this makes it possible to start running `Unstructured` through generic storage.
@kubernetes/sig-api-machinery-pr-reviews @smarterclayton @lavalamp
Automatic merge from submit-queue (batch tested with PRs 43273, 44287, 44281)
bump(github.com/onsi/ginkgo):v1.2.0-94-g5ca1211
Picks up parallel execution improvements in Ginkgo that distribute jobs more evenly with parallel runs. Opened after observing long tail runs in OpenShift - testing there showed a much more compact run in terms of CPU https://github.com/onsi/ginkgo/issues/333
We'll need some soak time to be sure this has no issues.
Automatic merge from submit-queue
add rancher credential provider
This adds rancher as a credential provider in kubernetes.
@erictune This might be a good opportunity to discuss adding a provision for people to have their own credential providers that is similar to the new cloud provider changes (https://github.com/kubernetes/community/pull/128). WDYT?
```
release-note
Added Rancher Credential Provider to use Rancher Registry credentials when running in a Rancher cluster
```
This PR implements a standard admission plugin initializer for the generic API server.
The initializer accepts external clientset, external informers and the authorizer.
Automatic merge from submit-queue
proxy to IP instead of name, but still use host verification
I think I found a setting that lets us proxy to an IP and still do hostname verification on the certificate.
@liggitt @sttts Can you see if you agree that this knob does what I think it does? Last commit only, still needs tests.
Automatic merge from submit-queue (batch tested with PRs 43681, 40423, 43562, 43008, 43381)
Openstack cinder v1/v2/auto API support
**What this PR does / why we need it**:
It adds support for v2 cinder API + autodetection of available cinder API level (as in LBs).
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#39572
**Special notes for your reviewer**:
Based on work by @anguslees. The first two commits are just rebased from https://github.com/kubernetes/kubernetes/pull/36344 which already had a lgtm by @jbeda
**Release note**:
```
Add support for v2 cinder API for openstack cloud provider. By default it autodetects the available version.
```
Automatic merge from submit-queue (batch tested with PRs 43694, 41262, 42911)
Bump go-systemd to v14
**What this PR does / why we need it**: bump go-systemd to the latest release, which includes a build tag fix so non-cgo systems can compile.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**: go-systemd added an option to unset the `NOTIFY_SOCKET` environment variable as part of the execution of `SdNotify`. I have set that to true, which means that child processes (if any) won't have that set. If anyone thinks it should remain set, I'm happy to change it to false - just let me know.
**Release note**:
```release-note
```
cc @smarterclayton @sdodson @deads2k @sttts @lavalamp @caesarxuchao
Automatic merge from submit-queue (batch tested with PRs 42900, 43044, 42896, 43308, 43621)
require codecfactory
The genericapiserver requires a codec to start. Help new comers to the API by forcing them to set it when they create a new config.
Automatic merge from submit-queue
allow combining API servers
Builds on https://github.com/kubernetes/kubernetes/pull/42886 (already lgtm'ed)
We need to be able to chain multiple API servers together so that a fallthrough case from to another results in delegated handling without double wrapping. We also need to be able to combine shared lists like healthz and poststarthooks so that a single API server start will run all the poststarthooks and present a unified view of health. This creates an interface and methods to provide that wiring.
@kubernetes/sig-api-machinery-misc @ncdc
Automatic merge from submit-queue (batch tested with PRs 43429, 43416, 43312, 43141, 43421)
Create controller to auto register TPRs with the aggregator
Builds on https://github.com/kubernetes/kubernetes/pull/42732 (already lgtmed)
Creates a simple controller to wire TPRs with the API Service autoregistration controller.
@kubernetes/sig-api-machinery-misc @ncdc
Automatic merge from submit-queue (batch tested with PRs 43429, 43416, 43312, 43141, 43421)
remove timed_queue from workqueue
Fix https://github.com/kubernetes/kubernetes/issues/40494.
It's added to gather metrics of the workqueue used in the garbage collector. Now that a regular workqueue has supported the metrics, we can remove timed_queue.
Kubernetes Submit Queue