Automatic merge from submit-queue
Mount GCP credentials to federation deployment container for GKE clusters.
GKE clusters use GCP credentials and hence require them for deploying federation components on to those clusters.
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Typos and englishify federation
**What this PR does / why we need it**: Typos + "englishify" /federated
**Which issue this PR fixes** None
**Special notes for your reviewer**: Just typos + "englishify" federated and obviously autogenerated stuff.
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
`NONE`
Automatic merge from submit-queue
Updating federation api ref docs
Updating API ref docs to include changes from https://github.com/kubernetes/kubernetes/pull/31925.
PR generated by running `hack/update-federation-api-reference-docs.sh`
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
refactor genericapiserver new to combine initialization
Combines `New` and `init` since the two were inseparable before anyway. `New` now has all the code to create the `GenericAPIServer`.
The rest of the change is a move. I want to refactor the flow more, but I figured that doing it separately would simplify the review.
@sttts how do you feel about looking at this one?
Automatic merge from submit-queue
WatcherDispatcher for federated controller tests
This PR fixes a problem when 2 informers create a watch for the same resource using the same client. Previously only one informer would receive updates.
cc: @quinton-hoole @wojtek-t @kubernetes/sig-cluster-federation
quinton-hoole: To provide some more context to those doing cherrypicking, the bug that this PR fixes makes federated unit tests fail intermittently, and generally behave very poorly.
Automatic merge from submit-queue
fix deliverer dealy constant usage
some delay constants were not correctly used
#29741
@quinton-hoole @deepak-vij @kshafiee @mwielgus
Automatic merge from submit-queue
Adding namespaces/finalizer subresource to federation apiserver
Fixes https://github.com/kubernetes/kubernetes/issues/31077
cc @kubernetes/sig-cluster-federation @mwielgus
Verified manually that I can delete federation namespaces now.
Will update federation-namespace e2e test to verify that namespace is deleted fine
Automatic merge from submit-queue
Use hash in federated replica set planner
Without this all replicaset of size 1 would land in the same cluster if the default preferences (weight=1) are used.
cc: @jianhuiz @quinton-hoole @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Generating API reference docs for federation apiserver
Fixes#30541
Adding a script `update-federation-api-reference-docs.sh` similar to the existing `update-api-reference-docs.sh` for kube-apiserver. Have moved the common parts to `hack/lib/swagger.sh`.
The new script will produce API reference docs for federation-apiserver.
Next step will be to surface these docs at kubernetes.io.
cc @kubernetes/sig-cluster-federation @kubernetes/sig-api-machinery @caesarxuchao
Automatic merge from submit-queue
Clean-up and fixes in federated replica set
* Create and update consistent with other controllers. Previously an annotation update on federated rs would not trigger local rs update.
* Use of federatedUpdater. The previous code use talked to clusters manually, assuming that the stats values in the local rs would be automatically and immediately updated. These stats are updated by controllers so they are not immediately updated and the currently existing stats can be used for building federated rs stats.
* Trigger a rs recheck after some operations are executed.
cc: @quinton-hoole @jianhuiz @wojtek-t @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
EventSink for federated controllers
So that the controllers can reuse k8s.io/kubernetes/pkg/client/record machinery.
cc: @quinton-hoole @nikhiljindal @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Separate federation build.sh into development and deployment scripts.
The idea behind this separation is that it provides a clear distinction
between the dev environment and the prod environment. The
deploy/deploy.sh script will be shipped to the users, but
develop/develop.sh will be purely for development purposes and won't
be part of a release distribution.
Purely for developer convenience, all the deployment functionality is
made available through the develop/develop.sh script.
This change also copies deploy/* files into the release distribution.
cc @kubernetes/sig-cluster-federation @colhom
```release-note
Federation can now be deployed using the `federation/deploy/deploy.sh` script. This script does not depend on any of the development environment shell library/scripts. This is an alternative to the current `federation-up.sh`/`federation-down.sh` scripts. Both the scripts are going to co-exist in this release, but the `federation-up.sh`/`federation-down.sh` scripts might be removed in a future release in favor of `federation/deploy/deploy.sh` script.
```
Automatic merge from submit-queue
[Federation] Downsize the release binary distribution v2.
Second attempt of PR #29632.
There are two things that this PR does:
1. It removes `federation-apiserver` and `federation-controller-manager` from binaries and docker_wrapped_binaries target lists.
2. Build the docker image for `hyperkube` on-the-fly while pushing the federation images.
```release-note
Federation binaries and their corresponding docker images - `federation-apiserver` and `federation-controller-manager` are now folded in to the `hyperkube` binary. If you were using one of these binaries or docker images, please switch to using the `hyperkube` version. Please refer to the federation manifests - `federation/manifests/federation-apiserver.yaml` and `federation/manifests/federation-controller-manager-deployment.yaml` for examples.
```
cc @kubernetes/sig-cluster-federation @colhom
Fixes Issue #28633
Automatic merge from submit-queue
Proper phase/finalizer logic in federated namespace controller
Part 1 of delete logic cleanup.
Part 2 will contain namespace content deletion and tests.
cc: @quinton-hoole @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
return destroy func to clean up internal resources of storage
What?
Provide a destroy func to clean up internal resources of storage.
It changes **unit tests** to clean up resources. (Maybe fix integration test in another PR.)
Why?
Although apiserver is designed to be long running, there are some cases that it's not.
See https://github.com/kubernetes/kubernetes/issues/31262#issuecomment-242208771
We need to gracefully shutdown and clean up resources.
Also build the hyperkube docker image on-the-fly.
This is only a temporary fix until the proposal in issue
https://github.com/kubernetes/kubernetes/issues/28630 is implemented.
Also, the new build/deployment method completely obviates this step.
We use debian image instead of busybox and do not build hyperkube as a
static binary yet. Wait until PR
https://github.com/kubernetes/kubernetes/pull/26028 is merged to build
static hyperkube binaries.
The idea behind this separation is that it provides a clear distinction
between the dev environment and the prod environment. The
deploy/deploy.sh script will be shipped to the users, but
develop/develop.sh will be purely for development purposes and won't
be part of a release distribution.
Purely for developer convenience, all the deployment functionality is
made available through the develop/develop.sh script.
This change also copies deploy/* files into the release distribution.
Automatic merge from submit-queue
ObjectMeta equivalence in federated controller handlers + update in fed secrets
Federated secrets should trigger also on data/type update, not only on object meta.
cc: @quinton-hoole @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Add Cluster field in ObjectMeta
There will be no sub-rs, but add `Cluster` field to the ObjectMeta (for all the objects)
"To distinguish the object at the federation level from it's constituents at the cluster level we will add a "Cluster" field to the metadata of all objects (where the federation itself will also have a cluster identifier). That way it is possible to list, interact with, and distinguish between the objects either at the federation level or at the individual cluster level based on the cluster identifier. "
@quinton-hoole @nikhiljindal @deepak-vij @mfanjie @huangyuqi
[]()
Automatic merge from submit-queue
federation: Adding support for namespace admission controls in federation-apiserver
Now that we have namespaces in federation apiserver, we can support namespace admission controls.
There are 3 of these:
namespace/autoprovision, namespace/exists and namespace/lifecycle.
namespace/autoprovision, namespace/exists should be deprecated in kubernetes(https://github.com/kubernetes/kubernetes/issues/31195). Adding support for namespace/lifecycle to federation-apiserver.
As in kube-apiserver, enabling namespace/lifecycle by default.
```release-note
Action required: If you have a running federation control plane, you will have to ensure that for all federation resources, the corresponding namespace exists in federation control plane.
federation-apiserver now supports NamespaceLifecycle admission control, which is enabled by default. Set the --admission-control flag on the server to change that.
```
cc @kubernetes/sig-cluster-federation @quinton-hoole
Automatic merge from submit-queue
Object meta fixes in federation controllers
This PR adds helper functions for handling object meta structs and applies them to the existing controllers.
cc: @quinton-hoole @jianhuiz @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
pkg/genericapiserver/options: don't import pkg/apiserver
Refactor the authorization options for the API server so
pkg/apiserver isn't directly imported by the options package.
Closes#28544
cc @smarterclayton
@madhusudancs, @nikhiljindal I've updated `federation/cmd/federation-apiserver/app/server.go` to include the RBAC options with this change. I don't know if this was intentionally left out in the first place but would like your feedback.
Automatic merge from submit-queue
Federation informer use kube clientset for target
This is to use kubernetes clientset as the federation informer target clientset as it's used to talking the k8s cluster.
#29939#30669#30207
@mwielgus @quinton-hoole @kshafiee @deepak-vij
Automatic merge from submit-queue
change all PredicateFunc to use SelectionPredicate
What?
- This PR changes all PredicateFunc in registry to return SelectionPredicate instead of Matcher interface.
Why?
- We want to pass SelectionPredicate to storage layer. Matcher interface did not expose enough information for indexing.
Automatic merge from submit-queue
Handle secret namespace and data properly in federated secret controller
This PR fixes what was missed in #30669 review. It uses both namespace and secret name for keying and propagates secret data to underlying clusters.
cc: @quinton-hoole @kshafiee @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
federation: updating cluster name validation to allow DNS labels only
Forked from https://github.com/kubernetes/kubernetes/pull/28921
Summarizing the discussion from that PR:
* Right now, we allow name of the Cluster resource in federation/v1beta1 group version to be a subdomain (group of DNS labels separated by dots). This prevents us from using the cluster name in our dns search paths, since there is a restriction of 6 domain labels there.
* Restricting cluster name to DNS label will give us the flexibility to be able to do that in the future, if we want to. Though we do not have a concrete use case right now, the possibility is attractive.
* There is not a strong argument in favor of allowing subdomains as cluster names right now. If in future, there is one then we can get more permissive but its better to start with a stricter model.
Note that we are breaking a beta API, but it should be fine since we do not expect anyone to be using subdomain as cluster name. Have added release-note-action-required label
```release-note
Action required: federation-only: Please update your cluster name to be a valid DNS label.
Updating federation.v1beta1.Cluster API to disallow subdomains as valid cluster names. Only DNS labels are allowed as valid cluster names now.
```
Automatic merge from submit-queue
Support for preexisting replicas and estimated capacity in federated replicaset controller
With this PR the planer will be able to:
* Keep already existing replicas in their current clusters if rebalance = false and min/max boundaries are met.
* Limit the number of replicas in a cluster to the level that was measured by the count of running and unschedulable pods. And provide an estimate how much more pods would be nice to put in a cluster so that if they are scheduled we will be closer to the desired layout or to schedule the desired number of replicas at all.
cc: @quinton-hoole @jianhuiz @wojtek-t @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Move UTs that block on apiserver to integration tests.
In validating etcd.v3client we had uncovered that a change in the behavior of the client https://github.com/coreos/etcd/issues/6162 , caused a number of unit tests to fail. These test failures were due to the fact that the unit tests were trying to standup a apiserver even though there was no etcd backend stood up.
This PR simply shuffles those tests to integration tests, which is where they should be.
/cc @kubernetes/sig-scalability @wojtek-t @hongchaodeng @xiang90
Chao Xu