Automatic merge from submit-queue
[Federation][(Un)join-01] Refactor common functions and structs into a util package.
Please review only the last commit here. This is based on PR #35495 which will be reviewed independently.
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @quinton-hoole @nikhiljindal
Automatic merge from submit-queue
[Federation][unjoin-00] Implement `kubefed unjoin` command.
Please review only the last commit here. This is based on PR #35493 which will be reviewed independently.
I will add a release note separately for this entire feature, so please don't worry too much about the release note here in the PR.
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @quinton-hoole @nikhiljindal
Automatic merge from submit-queue
Avoid double decoding all client responses
Fixes#35982
The linked issue uncovered that we were always double decoding the response in restclient for get, list, update, create, and patch. That's fairly expensive, most especially for list. This PR refines the behavior of the rest client to avoid double decoding, and does so while minimizing the changes to rest client consumers.
restclient must be able to deal with multiple types of servers. Alter the behavior of restclient.Result#Raw() to not process the body on error, but instead to return the generic error (which still matches the error checking cases in api/error like IsBadRequest). If the caller uses
.Error(), .Into(), or .Get(), try decoding the body as a Status.
For older servers, continue to default apiVersion "v1" when calling restclient.Result#Error(). This was only for 1.1 servers and the extensions group, which we have since fixed.
This removes a double decode of very large objects (like LIST) - we were trying to DecodeInto status, but that ends up decoding the entire result and then throwing it away. This makes the decode behavior specific to the type of action the user wants.
```release-note
The error handling behavior of `pkg/client/restclient.Result` has changed. Calls to `Result.Raw()` will no longer parse the body, although they will still return errors that react to `pkg/api/errors.Is*()` as in previous releases. Callers of `Get()` and `Into()` will continue to receive errors that are parsed from the body if the kind and apiVersion of the body match the `Status` object.
This more closely aligns rest client as a generic RESTful client, while preserving the special Kube API extended error handling for the `Get` and `Into` methods (which most Kube clients use).
```
Automatic merge from submit-queue
Correct the article in generated documents
**What this PR does / why we need it**:
Fix the article in generated docs for "create/delete [article] [kind]"
**Which issue this PR fixes**
fixes#32305
**Special notes for your reviewer**:
None
**Release note**:
``` release-note
Correct the article in generated documents
```
For example:
"a Ingress" > "an Ingress"
We can put subdomains into hosted zones (for example,
foo.federation.example.com can be hosted in example.com)
By allowing sharing a common hosted zone, this means the user doesn't
have to do as much setup.
Automatic merge from submit-queue
allow authentication through a front-proxy
This allows a front proxy to set a request header and have that be a valid `user.Info` in the authentication chain. To secure this power, a client certificate may be used to confirm the identity of the front proxy
@kubernetes/sig-auth fyi
@erictune per-request
@liggitt you wrote the openshift one, ptal.
Automatic merge from submit-queue
Fix federation-apiserver authentication e2e tests
**What this PR does / why we need it**: Fixes couple of federation e2e tests
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes # partly fixes#35251
Automatic merge from submit-queue
Adding cascading deletion support to federated namespaces
Ref https://github.com/kubernetes/kubernetes/issues/33612
With this change, whenever a federated namespace is deleted with `DeleteOptions.OrphanDependents = false`, then federation namespace controller first deletes the corresponding namespaces from all underlying clusters before deleting the federated namespace.
cc @kubernetes/sig-cluster-federation @caesarxuchao
```release-note
Adding support for DeleteOptions.OrphanDependents for federated namespaces. Setting it to false while deleting a federated namespace also deletes the corresponding namespace from all registered clusters.
```
Automatic merge from submit-queue
Add GroupVersion tags to OpenAPI spec and remove all specs except main one
Tags are used as a grouping mechanism in OpenAPI. We generated one spec per GroupVersion before for this grouping but by adding those tags in this PR, those files have no use. We can always add them back if there were a use-case for them.
**Release note**:
```release-note
Deprecate OpenAPI spec for GroupVersion endpoints in favor of single spec /swagger.json
```
Reference: #13414
Automatic merge from submit-queue
Create restclient interface
Refactoring of code to allow replace *restclient.RESTClient with any RESTClient implementation that implements restclient.RESTClientInterface interface.
Automatic merge from submit-queue
make version an explicit choice so zero config and customized work
Makes `/version` key off of setting the version. This allows composers to add a version that is correct.
Automatic merge from submit-queue
+optional tag for OpenAPI spec
OpenAPI rely on "omitempty" json tag to determine if a field is optional or not. This change will add "+optional" tag to all fields with "omitempty" json tag and support the tag in OpenAPI spec generator.
Automatic merge from submit-queue
Run rbac authorizer from cache
RBAC authorization can be run very effectively out of a cache. The cache is a normal reflector backed cache (shared informer).
I've split this into three parts:
1. slim down the authorizer interfaces
1. boilerplate for adding rbac shared informers and associated listers which conform to the new interfaces
1. wiring
@liggitt @ericchiang @kubernetes/sig-auth
Automatic merge from submit-queue
Change legacy API resource registration
Updates the legacy API resource registration to emphasize its different-ness and to simplify supporting objects. The option has to remain in the genericapiserverconfig for multiple prefixes to enable cases where composers/extenders had composed additional groupless APIs. See OpenShift as an example.
However this is now transparent to "normal" composers.
@ncdc since sttts is out.
Automatic merge from submit-queue
Update `gcloud docker` commands to use `gcloud docker -- ARGS`
We can then avoid the following warning:
```
WARNING: The '--' argument must be specified between gcloud specific args on the left and DOCKER_ARGS on the right. IMPORTANT: previously, commands allowed the omission of the --, and unparsed arguments were treated as implementation args. This usage is being deprecated and will be removed in March 2017.
This will be strictly enforced in March 2017. Use 'gcloud beta docker' to see new behavior.
```
Automatic merge from submit-queue
WantsAuthorizer admission plugin support
The next step of PSP admission is to be able to limit the PSPs used based on user information. To do this the admission plugin would need to make authz checks for the `user.Info` in the request. This code allows a plugin to request the injection of an authorizer to allow it to make the authz checks.
Note: this could be done with a SAR, however since admission is running in the api server using the SAR would incur an extra hop vs using the authorizer directly.
@deads2k @derekwaynecarr
Automatic merge from submit-queue
Generate unique Operation IDs for root OpenAPI spec
This PR adds a customization method GetOperationID to OpenAPI spec generation and then use it to make sure root spec has unique operation IDs by mostly adding GroupVersion to the start of operation ID.
Automatic merge from submit-queue
Federated DaemonSet controller
Based on the secrets controller. E2e tests will come in the next PR.
**Release note**:
```release-note
Federated DaemonSet controller. Supports all the API that regular DaemonSet has.
```
cc: @quinton-hoole @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
[Federation] Fix bash continuation, update charts version and kubeconfig mount permission
```release-note
* New federation deployment mechanism now allows non-GCP clusters.
* Writes the federation kubeconfig to the local kubeconfig file.
```
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Checking function integrated with get from chanel in fed tests
Current federation controller tests assume that each actions results in only one etcd action. However, due to various timing issues, this may not necessary the truth and multiple updates can be generated. Thus tests should try to get all available messages from the debug channels instead failing on the first message, which may come from the previous test scenario.
Applied to deployment tests. Other tests will be updated in the following PR(s).
cc: @quinton-hoole @wojtek-t
Automatic merge from submit-queue
[WIP] AWS compatibility for federation cluster and e2e
I've been testing this and have reached a point where the e2e tests run, and some test failures are popping up which are not overtly related to AWS specific things.
```sh
SSSSSSSSSSSSSSSS
Summarizing 5 Failures:
[Fail] [k8s.io] [Feature:Federation] Federated Services DNS [BeforeEach] should be able to discover a federated service
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/federation-util.go:233
[Fail] [k8s.io] [Feature:Federation] Federated Services Service creation [It] should create matching services in underlying clusters
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/federation-util.go:233
[Fail] [k8s.io] Federated ingresses [Feature:Federation] Federated Ingresses [It] should create and update matching ingresses in underlying clusters
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/federated-ingress.go:289
[Fail] [k8s.io] [Feature:Federation] Federated Services DNS [BeforeEach] non-local federated service [Slow] missing local service should never find DNS entries for a missing local service
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/federation-util.go:233
[Fail] [k8s.io] [Feature:Federation] Federated Services DNS [BeforeEach] non-local federated service should be able to discover a non-local federated service
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/federation-util.go:233
Ran 16 of 383 Specs in 519.872 seconds
FAIL! -- 11 Passed | 5 Failed | 1 Pending | 366 Skipped --- FAIL: TestE2E (519.89s)
```
\cc @quinton-hoole @madhusudancs for advice. Should I investigate further?
Automatic merge from submit-queue
Separate the build recipe in federation Makefile into separate phases.
This is based on PR #30744. Please review only the last commit here.
cc @kubernetes/sig-cluster-federation @colhom
Automatic merge from submit-queue
Include DaemonSets and Deployments in Namespace deletion
More generic deletion should be added next week.
cc: @quinton-hoole
We can then avoid the following warning:
```
WARNING: The '--' argument must be specified between gcloud specific args on the left and DOCKER_ARGS on the right. IMPORTANT: previously, commands allowed the omission of the --, and unparsed arguments were treated as implementation args. This usage is being deprecated and will be removed in March 2017.
This will be strictly enforced in March 2017. Use 'gcloud beta docker' to see new behavior.
```
Signed-off-by: Jess Frazelle <acidburn@google.com>
Automatic merge from submit-queue
Federated deployment controller - part 1
Based on federated replicaset controller (copy + find/replace).
Remaining stuff:
- refacing out common elements to libs
- using owerref in pod analysis
- e2e tests
- renaming concurrency flag for rs and reusing it in deployment
- updating only one cluster at a time if rollingupdate strategy is used.
cc: @quinton-hoole @kubernetes/sig-cluster-federation
**Release note**:
```release-note
Federated deployment controller that supports the same api as the regular kubernetes deployment controller.
```
Automatic merge from submit-queue
Generate and store federation OpenAPI spec in source tree
A follow up of #33628 to also add federation spec to source tree.
Automatic merge from submit-queue
remove testapi.Default.GroupVersion
I'm going to try to take this as a series of mechanicals. This removes `testapi.Default.GroupVersion()` and replaces it with `registered.GroupOrDie(api.GroupName).GroupVersion`.
@caesarxuchao I'm trying to see how much of `pkg/api/testapi` I can remove.
Automatic merge from submit-queue
Add missing argument to log message in federated ingress controller.
Minor fix, to address this:
1006 01:08:50.605184 1 ingress_controller.go:674] No annotation "kubernetes.io/ingress.global-static-ip-name" exists on ingress "e2e-tests-federated-ingress-2787b/federated-ingress" in federation, and index of cluster "federation-e2e-gce-us-central1-f" is 1 and not zero. Not queueing create operation for ingress *%!q(MISSING)* until annotation exists
Automatic merge from submit-queue
update deployment and replicaset listers
Updates the deployment lister to avoid copies and updates the deployment controller to use shared informers.
Pushing WIP to see which tests are broken.
Automatic merge from submit-queue
decouple workqueue metrics from prometheus
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
We want to include the workqueue in client-go, but do not want to having to import Prometheus. This PR decouples the workqueue from prometheus.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Partially address https://github.com/kubernetes/kubernetes/issues/33497
User requested for `workqueue` in client-go: https://github.com/kubernetes/client-go/issues/4#issuecomment-249444848
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
The implicit registration of Prometheus metrics for workqueue has been removed, and a plug-able interface was added. If you were using workqueue in your own binaries and want these metrics, add the following to your imports in the main package: "k8s.io/pkg/util/workqueue/prometheus".
```
Automatic merge from submit-queue
Set deserialization cache size based on target memory usage
**Special notes for your reviewer**:
This is the PR we talked about yesterday.
**Release note**:
```release-note
To reduce memory usage to reasonable levels in smaller clusters, kube-apiserver now sets the deserialization cache size based on the target memory usage.
```
Automatic merge from submit-queue
stop plumbing options to start
The API server should have sufficient information to start itself without relying on an `options` object from a different packages focused on CLI interaction. This provides that separation
@liggitt distinct from other changes
Automatic merge from submit-queue
pass loopback config to poststart hooks
Updates post start hooks to take a clientconfig with the new loopback credentials for bootstrapping.
@ericchiang This is a little bit of plumbing, but mainly auth I think.
Automatic merge from submit-queue
Copy objects in federation test helper
Push clean copies of passed objects in test helper in federation controllers tests.
Fixes: #33667
cc: @gmarek @quinton-hoole @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Allow anonymous API server access, decorate authenticated users with system:authenticated group
When writing authorization policy, it is often necessary to allow certain actions to any authenticated user. For example, creating a service or configmap, and granting read access to all users
It is also frequently necessary to allow actions to any unauthenticated user. For example, fetching discovery APIs might be part of an authentication process, and therefore need to be able to be read without access to authentication credentials.
This PR:
* Adds an option to allow anonymous requests to the secured API port. If enabled, requests to the secure port that are not rejected by other configured authentication methods are treated as anonymous requests, and given a username of `system:anonymous` and a group of `system:unauthenticated`. Note: this should only be used with an `--authorization-mode` other than `AlwaysAllow`
* Decorates user.Info returned from configured authenticators with the group `system:authenticated`.
This is related to defining a default set of roles and bindings for RBAC (https://github.com/kubernetes/features/issues/2). The bootstrap policy should allow all users (anonymous or authenticated) to request the discovery APIs.
```release-note
kube-apiserver learned the '--anonymous-auth' flag, which defaults to true. When enabled, requests to the secure port that are not rejected by other configured authentication methods are treated as anonymous requests, and given a username of 'system:anonymous' and a group of 'system:unauthenticated'.
Authenticated users are decorated with a 'system:authenticated' group.
NOTE: anonymous access is enabled by default. If you rely on authentication alone to authorize access, change to use an authorization mode other than AlwaysAllow, or or set '--anonymous-auth=false'.
```
c.f. https://github.com/kubernetes/kubernetes/issues/29177#issuecomment-244191596
Automatic merge from submit-queue
Update federation controllers to use release 1.5 clientset
This PR is based on #32243, we won't merge it until next week, to make cherry-picks easier.
Automatic merge from submit-queue
Autoconvert cleanups
A few accumulated cleanups to conversion, and then a big change. Please review commit-by-commit. Verified by nuking all generated files and comparing the new results.
cc @lavalamp
Automatic merge from submit-queue
Copy auth plugin to client-go repo
client-go doesn't copy the [auth plugin](https://github.com/kubernetes/kubernetes/blob/master/plugin/pkg/client/auth/plugins.go). This causes user cannot access cluster run by GKE. User will see error "No Auth Provider found for name gcp".
This PR fixes this issue. It's marked as WIP because I'll need to rebase after #32906 gets merged. Also, the fix needs to be cherry-picked into 1.4 branch to update client-go/1.4.
Previously we refused to emit 'autoConvert_*' functions if any field was not
convertible. The way around this was to write manual Conversion functions, but
to do so safely you must handle every fields. Huge opportunity for errors.
This PR cleans up the filtering such that it only operates on types that should
be converted (remove a lot of code) and tracks when fields are skipped. In
that case, it emits an 'autoConvert' function but not a public 'Convert'
function. If there is no manual function, the compile will fail.
This also means that manual conversion functions can call autoConvert functions
and then "patch up" what they need.
Automatic merge from submit-queue
Fix goroutine leak in federation service controller
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**: Fixes a memory leak
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#33186
**Special notes for your reviewer**: Every second new goroutines are created and are getting blocked waiting for the lock in the event queue. only one worker will get a lock when there are some events to process, so all the goroutines which are created every second waits for the lock forever and causes the memory/goroutine leak.
As a fix the new worker will be created only when there is no worker exist. and only one worker per cluster either waits for the event or processes all the events and goes out of existence.
```release-note
Fixes memory/goroutine leak in Federation Service controller.
```
Madhusudan.C.S