Automatic merge from submit-queue
allow combining API servers
Builds on https://github.com/kubernetes/kubernetes/pull/42886 (already lgtm'ed)
We need to be able to chain multiple API servers together so that a fallthrough case from to another results in delegated handling without double wrapping. We also need to be able to combine shared lists like healthz and poststarthooks so that a single API server start will run all the poststarthooks and present a unified view of health. This creates an interface and methods to provide that wiring.
@kubernetes/sig-api-machinery-misc @ncdc
Automatic merge from submit-queue (batch tested with PRs 43429, 43416, 43312, 43141, 43421)
Create controller to auto register TPRs with the aggregator
Builds on https://github.com/kubernetes/kubernetes/pull/42732 (already lgtmed)
Creates a simple controller to wire TPRs with the API Service autoregistration controller.
@kubernetes/sig-api-machinery-misc @ncdc
Automatic merge from submit-queue (batch tested with PRs 43429, 43416, 43312, 43141, 43421)
add singular resource names to discovery
Adds the singular resource name to our resource for discovery. This is something we've discussed to remove our pseudo-pluralization library which is unreliable even for english and really has no hope of properly handling other languages or variations we can expect from TPRs and aggregated API servers.
This pull simply adds the information to discovery, it doesn't not re-wire any RESTMappers.
@kubernetes/sig-cli-misc @kubernetes/sig-apimachinery-misc @kubernetes/api-review
```release-note
API resource discovery now includes the `singularName` used to refer to the resource.
```
Automatic merge from submit-queue (batch tested with PRs 43429, 43416, 43312, 43141, 43421)
remove timed_queue from workqueue
Fix https://github.com/kubernetes/kubernetes/issues/40494.
It's added to gather metrics of the workqueue used in the garbage collector. Now that a regular workqueue has supported the metrics, we can remove timed_queue.
Automatic merge from submit-queue (batch tested with PRs 43144, 42671, 43226, 43314, 43361)
use - to indicate audit log goes to system out
When debugging API servers, particularly aggregated API servers, it's really useful to see the audit trail in their normal pod logs. This makes `--audit-log-path=-` direct audit information to stdout.
@kubernetes/sig-api-machinery-misc
Automatic merge from submit-queue (batch tested with PRs 43144, 42671, 43226, 43314, 43361)
start informers as a post-start-hook
Switches the shared informer start to a post start hook to make future API server composition easier. PostStartHooks will have to be unioned for server composition and this ensures that we don't accidentally skip starting them.
Automatic merge from submit-queue (batch tested with PRs 42998, 42902, 42959, 43020, 42948)
Add Host field to TCPSocketAction
Currently, TCPSocketAction always uses Pod's IP in connection. But when a pod uses the host network, sometimes firewall rules may prevent kubelet from connecting through the Pod's IP.
This PR introduces the 'Host' field for TCPSocketAction, and if it is set to non-empty string, the probe will be performed on the configured host rather than the Pod's IP. This gives users an opportunity to explicitly specify 'localhost' as the target for the above situations.
```release-note
Add Host field to TCPSocketAction
```
Automatic merge from submit-queue
allow fallthrough handling from go-restful routes
This sets up the gorestful routes to fall through to a default handler and reorders the API to be ahead of the other endpoints. This makes it possible to cleanly support cases of "match, fail, try this other handler" which we'll need for API server composition.
@kubernetes/sig-api-machinery-pr-reviews @ncdc
Automatic merge from submit-queue
add local option to APIService
APIServices need an option to avoid proxying in cases where the groupversion is handled later in the chain. This will allow a coherent and complete set of APIServices, but won't require extra connections.
@kubernetes/sig-api-machinery-misc @ncdc @cheftako
Automatic merge from submit-queue (batch tested with PRs 42672, 42770, 42818, 42820, 40849)
use separate scheme to serve the kube-aggregator
This removes a hack which used the client scheme to serve the kube-aggregator. This switches it to run from its own scheme.
@kubernetes/sig-api-machinery-pr-reviews
@ncdc
Automatic merge from submit-queue
update kubeconfig document url in comments
**What this PR does / why we need it**:
update kubeconfig document url in comments
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
NONE
**Special notes for your reviewer**:
NONE
**Release note**:
NONE
Many of our core loops rotate around JitterUntil, which means we create
a lot of garbage at steady state for timers. Add proper timer reuse in
this loop.
Automatic merge from submit-queue (batch tested with PRs 43313, 43257, 43271, 43307)
Remove 'all namespaces' meaning of empty list in PodAffinityTerm
Removes the distinction between `null` and `[]` for the PodAffinityTerm#namespaces field (option 4 discussed in https://github.com/kubernetes/kubernetes/issues/43203#issuecomment-287237992), since we can't distinguish between them in protobuf (and it's a less than ideal API)
Leaves the door open to reintroducing "all namespaces" function via a dedicated field or a dedicated token in the list of namespaces
Wanted to get a PR open and tests green in case we went with this option.
Not sure what doc/release-note is needed if the "all namespaces" function is not present in 1.6
Automatic merge from submit-queue
Prevent protobuf storage with etcd2
Prevents accidentally storing protobuf content in etcd2 when upgrading to 1.6
c.f. https://github.com/kubernetes/kubernetes/issues/42976#issuecomment-286537139
```release-note
if kube-apiserver is started with `--storage-backend=etcd2`, the media type `application/json` is used.
```
Automatic merge from submit-queue (batch tested with PRs 40964, 42967, 43091, 43115)
Update hack scripts to use godep v79 and ensure_godep_version
**What this PR does / why we need it**:
Based on #42965 and https://github.com/kubernetes/kubernetes/pull/42958#discussion_r105568318, this pins the godep version at v79, which should fix some issues when running godep in go1.8 local environments.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#42817
**Special notes for your reviewer**:
This should likely get the v1.6 milestone so that it can be merged into master. While I'm setting a default godep version, I'm continuing to use the local pins per this comment: https://github.com/kubernetes/kubernetes/pull/42965#issuecomment-285962723 .
**Release note**:
```release-note
NONE
```
cc: @sttts
Automatic merge from submit-queue (batch tested with PRs 42802, 42927, 42669, 42988, 43012)
update to latest version of coreos/go-oidc
Includes updates that enable OIDC with OKTA as a IDP
**What this PR does / why we need it**:
Updates to the latest version of coreos/go-oidc
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes # TBD
**Special notes for your reviewer**:
Updates coreos/go-oidc module to include fixes for https://github.com/coreos/go-oidc/issues/137 which prevent OKTA being used as an IDP
**Release note**:
```release-note
NONE
```
cc:/ @ericchiang
Automatic merge from submit-queue (batch tested with PRs 41794, 42349, 42755, 42901, 42933)
Fix DefaultTolerationSeconds admission plugin
DefaultTolerationSeconds is not working as expected. It is supposed to add default tolerations (for unreachable and notready conditions). but no pod was getting these toleration. And api server was throwing this error:
```
Mar 08 13:43:57 fedora25 hyperkube[32070]: E0308 13:43:57.769212 32070 admission.go:71] expected pod but got Pod
Mar 08 13:43:57 fedora25 hyperkube[32070]: E0308 13:43:57.789055 32070 admission.go:71] expected pod but got Pod
Mar 08 13:44:02 fedora25 hyperkube[32070]: E0308 13:44:02.006784 32070 admission.go:71] expected pod but got Pod
Mar 08 13:45:39 fedora25 hyperkube[32070]: E0308 13:45:39.754669 32070 admission.go:71] expected pod but got Pod
Mar 08 14:48:16 fedora25 hyperkube[32070]: E0308 14:48:16.673181 32070 admission.go:71] expected pod but got Pod
```
The reason for this error is that the input to admission plugins is internal api objects not versioned objects so expecting versioned object is incorrect. Due to this, no pod got desired tolerations and it always showed:
```
Tolerations: <none>
```
After this fix, the correct tolerations are being assigned to pods as follows:
```
Tolerations: node.alpha.kubernetes.io/notReady=:Exists:NoExecute for 300s
node.alpha.kubernetes.io/unreachable=:Exists:NoExecute for 300s
```
@davidopp @kevin-wangzefeng @kubernetes/sig-scheduling-pr-reviews @kubernetes/sig-scheduling-bugs @derekwaynecarr
Fixes https://github.com/kubernetes/kubernetes/issues/42716
Automatic merge from submit-queue (batch tested with PRs 42786, 42553)
Updated auto generated protobuf codes.
Generated by `./hack/update-generated-protobuf-dockerized.sh` in Mac.
Automatic merge from submit-queue (batch tested with PRs 42786, 42553)
Updated comments for TaintBasedEvictions.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
Automatic merge from submit-queue (batch tested with PRs 42692, 42169, 42173)
DaemonSet: Respect ControllerRef
**What this PR does / why we need it**:
This is part of the completion of the [ControllerRef](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md) proposal. It brings DaemonSet into full compliance with ControllerRef. See the individual commit messages for details.
**Which issue this PR fixes**:
This ensures that DaemonSet does not fight with other controllers over control of Pods.
**Special notes for your reviewer**:
**Release note**:
```release-note
DaemonSet now respects ControllerRef to avoid fighting over Pods.
```
cc @erictune @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 42692, 42169, 42173)
Add pprof trace support
Add support for `/debug/pprof/trace`
Can wait for master to reopen for 1.7.
cc @smarterclayton @wojtek-t @gmarek @timothysc @jeremyeder @kubernetes/sig-scalability-pr-reviews
The DaemonSet Listers still use selectors, because this is the
behavior expected by callers. This clarifies the meaning of the
returned list. Some callers may need to switch to using
GetControllerOf() instead, but that is a separate, case-by-case issue.
Automatic merge from submit-queue
Adding note saying client-go examples only work with the code in the same branch
Adding this note because the problem has confused many users.
It's doc change and only affects client-go examples, so adding the milestone.
Automatic merge from submit-queue (batch tested with PRs 42637, 42648)
Support multiple --feature-gates flags in the command line
Fixes the issue in https://github.com/kubernetes/kubernetes/pull/42647.
Before this change the whole set of gates was replaced with new values. Now values are overridden one by one.
Automatic merge from submit-queue (batch tested with PRs 41890, 42593, 42633, 42626, 42609)
Remove everything that is not new from batch/v2alpha1
Fixes#37166.
@lavalamp you've asked for it
@erictune this is a prereq for moving CronJobs to beta. I initially planned to put all in one PR, but after I did that I figured out it'll be easier to review separately. ptal
@kubernetes/api-approvers @kubernetes/sig-api-machinery-pr-reviews ptal
Automatic merge from submit-queue (batch tested with PRs 42506, 42585, 42596, 42584)
Preserve custom etcd prefix compatibility for etcd3
Fixes#42505
```release-note
restored normalization of custom `--etcd-prefix` when `--storage-backend` is set to etcd3
```
Automatic merge from submit-queue (batch tested with PRs 42080, 41653, 42598, 42555)
Support whitespace in command path for gcp auth plugin
```
External command option on gcp client auth plugin supports whitespace in command path.
```
Splitting on whitespace to get cmd+args breaks when the path the executable contains spaces. Resolve by adding a new "cmd-args" field to config to allow the full string of "cmd-path" to be interpreted as path to executable.
This change is backwards compatible with existing behavior.
Automatic merge from submit-queue (batch tested with PRs 42080, 41653, 42598, 42555)
StatefulSet: Respect ControllerRef
**What this PR does / why we need it**:
This is part of the completion of the [ControllerRef](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md) proposal. It brings StatefulSet into full compliance with ControllerRef. See the individual commit messages for details.
**Which issue this PR fixes**:
Fixes#36859
**Special notes for your reviewer**:
**Release note**:
```release-note
StatefulSet now respects ControllerRef to avoid fighting over Pods. At the time of upgrade, **you must not have StatefulSets with selectors that overlap** with any other controllers (such as ReplicaSets), or else [ownership of Pods may change](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md#upgrading).
```
cc @erictune @kubernetes/sig-apps-pr-reviews
The Deployment Listers still use selectors, because this is the
behavior expected by callers. This clarifies the meaning of the
returned list. Some callers may need to switch to using
GetControllerOf() instead, but that is a separate, case-by-case issue.
Specific use case is GKE users running gcloud/kubectl on Windows
with a cloud sdk installation path containing spaces. Also improving
test coverage using trick borrowed from exec_test.go
Automatic merge from submit-queue (batch tested with PRs 31783, 41988, 42535, 42572, 41870)
Clean user agent to reduce metrics cardinality
**What this PR does / why we need it**:
This PR is an example implementation for my issue #31781.
``` release-note
```
This commit cleans common browser user-agents to reduce the metrics
cardinality in exported prometheus metrics.
Resolveskubernetes/kubernetes#31781
The StatefulSet Listers still use selectors, because this is the
behavior expected by callers. This clarifies the meaning of the
returned list. Some callers may need to switch to using
GetControllerOf() instead, but that is a separate, case-by-case issue.
Automatic merge from submit-queue
make the system:authenticated group adder smarter
Fixes#42437
This prevents the group adder from adding the system:authenticated group when:
1. it's already in the list
2. the user is system:anonymous
3. system:unauthenticated is in the list
Smaller alternative to https://github.com/kubernetes/kubernetes/pull/42421 for 1.6.
@kubernetes/sig-auth-pr-reviews @enj @liggitt
Automatic merge from submit-queue (batch tested with PRs 42443, 38924, 42367, 42391, 42310)
Apply custom defaults to init containers
Adds overridden defaults to init containers. They were not being defaulted the same way normal containers were.
Automatic merge from submit-queue (batch tested with PRs 41306, 42187, 41666, 42275, 42266)
discovery restmapping should always prefer /v1
The core kube API, empty group, version==v1 should always be the most preferred group and resource from a rest mapper. This special cases that. All the others should be based on discovery order as we previously agreed.
@kubernetes/sig-cli-pr-reviews @kubernetes/sig-api-machinery-pr-reviews
@enj
Automatic merge from submit-queue (batch tested with PRs 41306, 42187, 41666, 42275, 42266)
Server timeout returns an incorrect error
Not a valid Status object in JSON
Part of #42163
Automatic merge from submit-queue (batch tested with PRs 41984, 41682, 41924, 41928)
RC/RS: Fully Respect ControllerRef
**What this PR does / why we need it**:
This is part of the completion of the [ControllerRef](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md) proposal. It brings ReplicaSet and ReplicationController into full compliance with ControllerRef. See the individual commit messages for details.
**Which issue this PR fixes**:
Although RC/RS had partially implemented ControllerRef, they didn't use it to determine which controller to sync, or to update expectations. This could lead to instability or controllers getting stuck.
Ref: https://github.com/kubernetes/kubernetes/issues/24433
**Special notes for your reviewer**:
**Release note**:
```release-note
```
cc @erictune @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue
Only set a groupresource on errors if a resource was specified
@deads2k this gets rid of the weird .meta.k8s.io error on unauthorized errors
Automatic merge from submit-queue (batch tested with PRs 42128, 42064, 42253, 42309, 42322)
Fix panic on nil invalid field error
bug fix for validation panic
if a field.Invalid is constructed with a nil badvalue, the Error() method panics, since reflect.TypeOf() returns nil
Automatic merge from submit-queue (batch tested with PRs 42128, 42064, 42253, 42309, 42322)
Add storage.k8s.io/v1 API
This is combined version of reverted #40088 (first 4 commits) and #41646. The difference is that all controllers and tests use old `storage.k8s.io/v1beta1` API so in theory all tests can pass on GKE.
Release note:
```release-note
StorageClassName attribute has been added to PersistentVolume and PersistentVolumeClaim objects and should be used instead of annotation `volume.beta.kubernetes.io/storage-class`. The beta annotation is still working in this release, however it will be removed in a future release.
```
Automatic merge from submit-queue (batch tested with PRs 41672, 42084, 42233, 42165, 42273)
Keep entries of apimachinery in client-go's Godeps.json
1. copy.sh now copies staging/ to a temp dir, creates snapshots for them, and then adds the temp dir to GOPATH before running `godep save`. This way, we don't need the `go list` check added in #41987 so that's removed @sttts.
2. copy.sh fills the entries for k8s.io/apimachinery with dummy SHA1 in client-go's Godeps.json. The publish robot will later fill the actually published one.
3. copy.sh removes the entire `_vendor` from client-go. We cannot keep `_vendor` because it makes hard to use the `git filter-branch` magic for client-go, because the filtered commits might contain changes to `_vendor` while client-go only has `vendor/`.
Some notes on point 3: the robot will restore the vendor/ later, but exclude k8s.io/* and glog, so that users retrieving client-go with `go get` won't end up with multiple copies of them. I think this should be a ***general rule*** for published k8s.io/* repos: every published repos should have complete Godeps.json with latest commit hashes of other k8s.io/* repos, but vendor/k8s.io/* and vendor/glog will be removed from all repos.
@sttts @lavalamp @deads2k
Automatic merge from submit-queue (batch tested with PRs 42126, 42130, 42232, 42245, 41932)
apiserver/pkg/server: include scheme in insecure self client config
Noticed this during bootkube development: https://github.com/kubernetes-incubator/bootkube/issues/325
In Go 1.8's `url.Parse` became more strict, and `url.Parse("127.0.0.1:8080")` now fails.
https://beta.golang.org/doc/go1.8#net_urlhttps://play.golang.org/p/dw_cPeotG4
Accidentally compiled bootkube with 1.8 and tracked a panic down to the loopback client config. Though we're still using the old "genericapiserver", this seems to be translated from the old one.
The actual panic we observed was
```
E0222 19:40:11.364949 5 server.go:254] Failed to create clientset: parse 127.0.0.1:8080: first path segment in URL cannot contain colon
panic: parse 127.0.0.1:8080: first path segment in URL cannot contain colon
goroutine 35 [running]:
github.com/kubernetes-incubator/bootkube/vendor/k8s.io/kubernetes/pkg/client/clientset_generated/internalclientset/typed/core/internalversion.NewForConfigOrDie(0xc420728ea0, 0xc420738e30)
/home/eric/src/github.com/kubernetes-incubator/bootkube/vendor/k8s.io/kubernetes/pkg/client/clientset_generated/internalclientset/typed/core/internalversion/core_client.go:132 +0x62
github.com/kubernetes-incubator/bootkube/vendor/k8s.io/kubernetes/pkg/master.(*Config).Complete(0xc420739db8, 0x0)
/home/eric/src/github.com/kubernetes-incubator/bootkube/vendor/k8s.io/kubernetes/pkg/master/master.go:180 +0x40a
github.com/kubernetes-incubator/bootkube/vendor/k8s.io/kubernetes/cmd/kube-apiserver/app.Run(0xc42007a3c0, 0x0, 0x0)
/home/eric/src/github.com/kubernetes-incubator/bootkube/vendor/k8s.io/kubernetes/cmd/kube-apiserver/app/server.go:347 +0x1e8d
github.com/kubernetes-incubator/bootkube/pkg/bootkube.(*bootkube).Run.func1(0xc4206b01e0, 0xc420164300)
/home/eric/src/github.com/kubernetes-incubator/bootkube/pkg/bootkube/bootkube.go:124 +0x2f
created by github.com/kubernetes-incubator/bootkube/pkg/bootkube.(*bootkube).Run
/home/eric/src/github.com/kubernetes-incubator/bootkube/pkg/bootkube/bootkube.go:124 +0xb0
```
I don't actually know if this is the correct fix or if there should be changes to `NewForConfigOrDie`. Am looking for comments more than anything.
edit: @abourget pointed out over on bootkube that the actual panic was fixed in the internalclient by https://github.com/kubernetes/kubernetes/pull/38519.
cc @deads2k @sttts @kubernetes/sig-api-machinery-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41931, 39821, 41841, 42197, 42195)
Admission Controller: Add Pod Preset
Based off the proposal in https://github.com/kubernetes/community/pull/254
cc @pmorie @pwittrock
TODO:
- [ ] tests
**What this PR does / why we need it**: Implements the Pod Injection Policy admission controller
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Added new Api `PodPreset` to enable defining cross-cutting injection of Volumes and Environment into Pods.
```
Kubernetes Submit Queue