Automatic merge from submit-queue (batch tested with PRs 41505, 41484, 41544, 41514, 41022)
several issues hit while trying to make it easy to register APIs
I was trying to create a script that would register all API versions on a given server and ended up hitting several problems. These are the fixes.
@sttts I suspect that I won't be able to continue down the host-network approach, since that means I won't be able to use in-cluster DNS without some finagling. It *could* be set up (and we make it work as a for instance), but the simple enablement approach will be hosted on the infrastructure. I'll go back to that.
Automatic merge from submit-queue (batch tested with PRs 41104, 41245, 40722, 41439, 41502)
add sample fuzzing tests
Make fuzzing tests as simple as possible from both the API installer and the scheme, so its easy to add for api groups and so that I can build a scheme and then make sure I got it right.
@kubernetes/sig-api-machinery-pr-reviews @sttts @mikedanese
Automatic merge from submit-queue
make kube-aggregator run as static pod for local-up-cluster
Runs the kube-aggregator as a static pod for local-up-cluster. Looks like someone broke kubectl negotiation again, so I'll fix that up separately.
@kubernetes/sig-api-machinery-misc
@lavalamp you're probably looking to run kube-aggregator as a static pod, here's an example.
@jwforres I'll make a secure variant for wiring up to openshift.
Automatic merge from submit-queue (batch tested with PRs 41378, 41413, 40743, 41155, 41385)
move kube-aggregator to staging
Straight move of kube-aggregator to staging. The scripts are the interesting bits.
@sttts @lavalamp
Automatic merge from submit-queue
copy pkg/util/logs to apiserver
This is a copy, not a move. API servers need to be able to init the logs, but so do clients. It would be weird to have the client-side commands depending on the server side logs utilities.
I updated all the server side references, but left the client-side ones.
@sttts @kubernetes/sig-api-machinery-pr-reviews acceptable?
Automatic merge from submit-queue (batch tested with PRs 41112, 41201, 41058, 40650, 40926)
make round trip testing generic
RoundTrip testing is something associated with a scheme and everyone who writes an API will want to do it. In the end, we should wire each API group separately in a test scheme and have them all call this general function. Once `kubeadm` is out of the main scheme, we'll be able to remove the one really ugly hack.
@luxas @sttts @kubernetes/sig-apimachinery-pr-reviews @smarterclayton
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
Shared informer custom resync periods
**What this PR does / why we need it**: add the ability for event handlers to specify their own distinct resync periods to a shared informer
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#26406
**Special notes for your reviewer**:
This is part of my series of PRs to convert as much of the code base to using generated shared informers as possible. Assuming this merges, we'll also need #40385, and I'll probably close#40097 and replace it with a series of individual PRs (1 per controller change).
Assuming we get close to agreement on the implementation, I can look at adding more unit tests.
**Release note**:
```release-note
```
@smarterclayton @deads2k @liggitt @sttts @derekwaynecarr @pmorie @lavalamp @wojtek-t @gmarek @timothysc @jayunit100 @kubernetes/sig-api-machinery-pr-reviews @kubernetes/sig-scalability-pr-reviews
Add the ability for each event handler of a shared informer to specify
its own resync period. If not specified, a handler will resync at the
informer's default interval.
Automatic merge from submit-queue (batch tested with PRs 38796, 40823, 40756, 41083, 41105)
Add more options to the RecommendedOptions struct.
Builds on https://github.com/kubernetes/kubernetes/pull/41028
Adds `AuditOptions` to the `RecommendedOptions`
@sttts @kubernetes/sig-api-machinery-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41061, 40888, 40664, 41020, 41085)
Allow values to be wrapped prior to serialization in etcd
This adds a new value transformer to the etcd2 store that can transform
the value from etcd on read and write. This will allow the store to
implement encryption at rest or otherwise transform the value prior to
persistence.
* [x] etcd3 store
* [x] example of transformation
* [x] partial error handling
This is in support of #12742
Automatic merge from submit-queue (batch tested with PRs 41023, 41031, 40947)
apiserver command line options lead to config
Logically command line options lead to config, not the other way around. We're clean enough now we can actually do the inversion.
WIP because I have some test cycles to fix, but this is all the meat.
@kubernetes/sig-api-machinery-misc
Automatic merge from submit-queue (batch tested with PRs 35782, 35831, 39279, 40853, 40867)
genericapiserver: cut off more dependencies – episode 7
Follow-up of https://github.com/kubernetes/kubernetes/pull/40822
approved based on #40363
Automatic merge from submit-queue (batch tested with PRs 40810, 40695)
move genericvalidation to apimachinery
Moves genericvalidation to apimachinery. The first commit shows the snip I made.
@sttts
Automatic merge from submit-queue (batch tested with PRs 40638, 40742, 40710, 40718, 40763)
move client/record
An attempt at moving client/record to client-go. It's proving very stubborn and needs a lot manual intervention and near as I can tell, no one actually gets any benefit from the sink and source complexity it adds.
@sttts @caesarchaoxu
Automatic merge from submit-queue (batch tested with PRs 40239, 40397, 40449, 40448, 40360)
move the discovery and dynamic clients
Moved the dynamic client, discovery client, testing/core, and testing/cache to `client-go`. Dependencies on api groups we don't have generated clients for have dropped out, so federation, kubeadm, and imagepolicy.
@caesarxuchao @sttts
approved based on https://github.com/kubernetes/kubernetes/issues/40363
Automatic merge from submit-queue
make client-go more authoritative
Builds on https://github.com/kubernetes/kubernetes/pull/40103
This moves a few more support package to client-go for origination.
1. restclient/watch - nodep
1. util/flowcontrol - used interface
1. util/integer, util/clock - used in controllers and in support of util/flowcontrol
Automatic merge from submit-queue (batch tested with PRs 40038, 40041, 39036)
move admission to genericapiserver
I disconnected the initialization that was type specific for later assessment.
@sttts
Automatic merge from submit-queue
Move PatchType to apimachinery/pkg/types
Fixes https://github.com/kubernetes/kubernetes/issues/39970
`PatchType` is shared by the client and server, they have to agree, and its critical for our API to function.
@smarterclayton @kubernetes/sig-api-machinery-misc
Automatic merge from submit-queue (batch tested with PRs 39466, 39490, 39527)
bump gengo to latest
bumping gengo to limit surprises while working on https://github.com/kubernetes/kubernetes/pull/39475
@kubernetes/sig-api-machinery-misc
Automatic merge from submit-queue
Begin paths for internationalization in kubectl
This is just the first step, purposely simple so we can get the interface correct.
@kubernetes/sig-cli @deads2k
Automatic merge from submit-queue
Update to latest go-restful
Enables us to remap the variable names swagger uses in output so we can avoid duplicates across type names.
Part of #38071
@mbohlool
Automatic merge from submit-queue
Add CoreDNS as Federation DNS provider
This PR contains following
1. DNS provider implementation for CoreDNS with etcd as backend
2. Script for optionally deploying CoreDNS in federation control plane
Still to do:
1. HA for CoreDNS service need to be handled, would be taken up in subsequent PR
2. Need to configure CoreDNS as fallback DNS to in-cluster KubeDNS(Manual Step). Need to consider automation.
Request for review: @quinton-hoole @madhusudancs @nikhiljindal @kshafiee @deepak-vij
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
openstack: Implement the `Routes` provider API
``` release-note
Implement the Routes provider API for OpenStack using Neutron extraroute extension. This removes the need for flannel/etc where supported. To use, ensure all your nodes are on the same Neutron (private) network and specify the router ID in new `[Route]` section of provider config:
[Route]
router-id = <router UUID>
```
Automatic merge from submit-queue (batch tested with PRs 38049, 37823, 38000, 36646)
Test 0-length-arrays in fuzzing tests
While hacking on #37289 I noticed that our fuzzing tests test nil slices and slices of length 1, but not slices of length 0, meaning we aren't testing that 0-length slices get treated the same as nil in all the places we expect them to (and in particular, we aren't ensuring that comparisons always use api.Semantic.DeepEqual rather than reflect.DeepEqual). (Though in fact, changing the fuzzer didn't turn up any bugs, so maybe this effectively gets tested somewhere else...)
`fuzz.New().NilChance(.5).NumElements(0, 1)` means we end up generating `nil` 50% of the time, a length 0 array 25% of the time, and a length 1 array 25% of the time... maybe it should be `fuzz.New().NilChance(.33).NumElements(0, 1)` instead?
The gofuzz rebase is to pull in https://github.com/google/gofuzz/pull/20, and the other fix is just a drive-by.
Automatic merge from submit-queue (batch tested with PRs 36263, 36755, 37357, 37222, 37524)
add other impersonation fields to transport
Adds the group and extra fields to the impersation options in a rest and transport config.
@kubernetes/sig-auth
Automatic merge from submit-queue
Update godep to 3.0.14 to match release recommendations.
Updates godep and regen of bazel build output.
/cc @wojtek-t
Automatic merge from submit-queue
Add validation that detects repeated keys in the labels and annotations maps
Fixes#2965 (a nearly 2 year old feature request!)
@kubernetes/kubectl
@eparis
Automatic merge from submit-queue
Update godeps for libcontainer+cadvisor
Needed to unblock more progress on pod cgroup.
/cc @vishh @dchen1107 @timstclair
Automatic merge from submit-queue
etcd3: use PrevKV to remove additional get
ref: #https://github.com/kubernetes/kubernetes/issues/33653
We are trying to test using PrevKV feature and see if it improves performance.
In order to test this, we will need etcd v3.1 (alpha) image.
Blockers:
- update gcr.io image (version v3.0.12)
Automatic merge from submit-queue
Update golang.org/x/sys/unix package
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
Kubernetes build is failing on ppc64le platform with following error:
```
[root@rhel72 kubernetes]# make all KUBE_BUILD_PPC64LE=y
can't load package: package .: no buildable Go source files in /root/ks_ws/src/k8s.io/kubernetes
can't load package: package .: no buildable Go source files in /root/ks_ws/src/k8s.io/kubernetes
+++ [0926 09:39:53] Generating bindata:
/root/ks_ws/src/k8s.io/kubernetes/test/e2e/framework/gobindata_util.go
+++ [0926 09:39:55] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
+++ [0926 09:39:55] Building go targets for linux/ppc64le:
cmd/libs/go2idl/deepcopy-gen
+++ [0926 09:40:04] Generating bindata:
/root/ks_ws/src/k8s.io/kubernetes/test/e2e/framework/gobindata_util.go
+++ [0926 09:40:05] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
+++ [0926 09:40:05] Building go targets for linux/ppc64le:
cmd/libs/go2idl/conversion-gen
E0926 09:40:11.770782 19065 conversion.go:594] Warning: could not generate autoConvert functions for k8s.io/kubernetes/pkg/apis/apps/v1alpha1.PetSetSpec <-> k8s.io/kubernetes/pkg/apis/apps.PetSetSpec
E0926 09:40:11.888241 19065 conversion.go:594] Warning: could not generate autoConvert functions for k8s.io/kubernetes/pkg/apis/extensions/v1beta1.HorizontalPodAutoscalerSpec <-> k8s.io/kubernetes/pkg/apis/autoscaling.HorizontalPodAutoscalerSpec
E0926 09:40:11.905974 19065 conversion.go:594] Warning: could not generate autoConvert functions for k8s.io/kubernetes/pkg/apis/extensions/v1beta1.JobSpec <-> k8s.io/kubernetes/pkg/apis/batch.JobSpec
E0926 09:40:11.941445 19065 conversion.go:594] Warning: could not generate autoConvert functions for k8s.io/kubernetes/pkg/apis/extensions/v1beta1.RollingUpdateDeployment <-> k8s.io/kubernetes/pkg/apis/extensions.RollingUpdateDeployment
E0926 09:40:11.947455 19065 conversion.go:594] Warning: could not generate autoConvert functions for k8s.io/kubernetes/pkg/apis/extensions/v1beta1.ScaleStatus <-> k8s.io/kubernetes/pkg/apis/extensions.ScaleStatus
+++ [0926 09:40:13] Generating bindata:
/root/ks_ws/src/k8s.io/kubernetes/test/e2e/framework/gobindata_util.go
+++ [0926 09:40:14] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
+++ [0926 09:40:14] Building go targets for linux/ppc64le:
cmd/libs/go2idl/openapi-gen
+++ [0926 09:40:25] Generating bindata:
/root/ks_ws/src/k8s.io/kubernetes/test/e2e/framework/gobindata_util.go
+++ [0926 09:40:26] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
+++ [0926 09:40:26] Building go targets for linux/ppc64le:
cmd/kube-dns
cmd/kube-proxy
cmd/kube-apiserver
cmd/kube-controller-manager
cmd/kubelet
cmd/kubemark
cmd/hyperkube
plugin/cmd/kube-scheduler
cmd/kubectl
cmd/gendocs
cmd/genkubedocs
cmd/genman
cmd/genyaml
cmd/mungedocs
cmd/genswaggertypedocs
cmd/linkcheck
examples/k8petstore/web-server/src
federation/cmd/genfeddocs
vendor/github.com/onsi/ginkgo/ginkgo
test/e2e/e2e.test
vendor/github.com/onsi/ginkgo/ginkgo
test/e2e_node/e2e_node.test
# k8s.io/kubernetes/vendor/github.com/fsnotify/fsnotify
vendor/github.com/fsnotify/fsnotify/inotify.go:39: undefined: unix.InotifyInit
vendor/github.com/fsnotify/fsnotify/inotify_poller.go:48: undefined: unix.Pipe2
make: *** [all] Error 1
[root@rhel72 kubernetes]#
```
Above error already addressed part of https://github.com/golang/go/issues/15393. Current `golang.org/x/sys/unix` is very old so updating the package to the latest version.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
```
Automatic merge from submit-queue
Dynamic provisioning for flocker volume plugin
Refactor flocker volume plugin
* [x] Support provisioning beta (#29006)
* [x] Support deletion
* [x] Use bind mounts instead of /flocker in containers
* [x] support ownership management or SELinux relabeling.
* [x] adds volume specification via datasetUUID (this is guranted to be unique)
I based my refactor work to replicate pretty much GCE-PD behaviour
**Related issues**: #29006#26908
@jsafrane @mattbates @wallrj @wallnerryan
Automatic merge from submit-queue
Update godep for github.com/spf13/pflag and replace StringSlice with String Array in configMap
Update godep for pkg `github.com/spf13/pflag`, because the new flag type `StringArray` within it is needed in issue #27454
Automatic merge from submit-queue
Remove hacks from ScheduledJobs cron spec parsing
Previusly `github.com/robfig/cron` library did not allow passing cron spec without seconds. First commit updates the library, which has additional method ParseStandard which follows the standard cron spec, iow. minute, hour, day of month, month, day of week.
@janetkuo @erictune as promised in #30227 I've updated the library and now I'm updating it in k8s
Automatic merge from submit-queue
Add Viper parametrization as E2E config option.
do-not-merge
Fixes#18099 via viper rather than inis.
Wont build until we remove BurntSushi/ COPYING based deps from upstream viper.
I'll dig into those issues independently and update later, before pushing the updated godeps into this PR.
Automatic merge from submit-queue
Vendor github.com/square/go-jose for kubeadm and kube-discovery
**What this PR does / why we need it**:
This is to be used by `kubeadm` (#31221) and @dgoodwin's `kube-discovery` (PR pending).
***[xref kubernetes/features#11]***
***[cc @kubernetes/sig-cluster-lifecycle]***
**Special notes for your reviewer**:
This is first time I've used `godep`, so please verify if all is quite right about the diff. I'm not 100% sure if there should be secondary dependencies or not.
```release-note
NONE
```
Bump version of golang.org/x/oauth2
Vendor google.golang.org/cloud/
Vendor google.golang.org/api/
Vendor cloud.google.com/go/compute/
Replace google.golang.org/cloud with cloud.google.com/go/
Fixes#30069
Automatic merge from submit-queue
vendor: update github.com/coreos/go-oidc client package
This change updates the github.com/coreos/go-oidc package to it's latest commit
(since we don't version that package).
Notable changes:
- Throw out JWTs with invalid claims early (coreos/go-oidc#97, brougt up in #30457)
- Remove the capnslog dependency (coreos/go-oidc#95)
- Support for Azure AD oddities (coreos/go-oidc#87)
cc @kubernetes/sig-auth
This change updates the github.com/coreos/go-oidc package.
Notable changes:
- Throw out JWTs with invalid claims early (coreos/go-oidc#97, brougt up in #30457)
- Remove the capnslog dependency (coreos/go-oidc#95)
- Support for Azure AD oddities (coreos/go-oidc#87)
Automatic merge from submit-queue
godep: bump etcd to 3.0.6
What?
Bump etcd godep dependency to v3.0.6
Why?
ref: #30843, https://github.com/coreos/etcd/pull/6222
We have some fix to do secure client connection in unit tests.
Automatic merge from submit-queue
Update cAdvisor to 2ed7198
**What this PR does / why we need it**:
Update cAdvisor to 2ed7198 so that we get Prometheus metrics on CPU throttling when pod resource limits are configured. We're flying blind right now.
**Changes**:
* Add container_cpu_cfs_* metrics (CPU throttling due to limits)
* Add container_memory_swap metric
* Ensure minimum kernel version for thin_ls
Diff: c6c06d4...2ed7198
Automatic merge from submit-queue
Bump heapster version
Bump heapster version to v1.2.0-beta.1.
Migrate metrics tests and HPA to use List objects introduced in the new version.
Mik Vyatskov