When federation controller manager checks cluster status, it will
delete the offline cluster from clusterSet, but do not delete the
corresponding restclient from the map clusterKubeClientMap for
the offline cluster. This patch can fix it.
The unit test for the ingress controller was previously adding
a cluster twice, which resulted in a cluster being deleted and added
back. The deletion was racing the controller shutdown to close
informer channels. This change ensures that the informer clears its
map of informers when Stop() is called to prevent a double close, and
that the test no longer adds the cluster twice.
The Deployment controller was not propagating ReadyReplicas to underlying clusters causing these errors:
```
Error syncing cluster controller: Deployment.apps "federation-deployment" is invalid: status.availableReplicas: Invalid value: 5: cannot be greater than readyReplicas
```
This was caught in e2e testing and is a 1.6 regression for support that was added in #37959. Without this fix, users will be unable to scale up their deployments.
The description of NewReplicaSetController() does not match
its function, and the description of NewDeploymentController()
does not match its function. Let's update their descriptions.
Automatic merge from submit-queue
Add ProviderUid support to Federated Ingress
This PR (along with GLBC support [here](https://github.com/kubernetes/ingress/pull/278)) is a proposed fix for #39989. The Ingress controller uses a configMap reconciliation process to ensure that all underlying ingresses agree on a unique UID. This works for all of GLBC's resources except firewalls which need their own cluster-unique UID. This PR introduces a ProviderUid which is maintained and synchronized cross-cluster much like the UID. We chose to derive the ProviderUid from the cluster name (via md5 hash).
Testing here is augmented to guarantee that configMaps are adequately propagated prior to Ingress creation.
```release-note
Federated Ingress over GCE no longer requires separate firewall rules to be created for each cluster to circumvent flapping firewall health checks.
```
cc @madhusudancs @quinton-hoole
Automatic merge from submit-queue (batch tested with PRs 42369, 42375, 42397, 42435, 42455)
Fixing federation controllers to support controllers flag
Fixes https://github.com/kubernetes/kubernetes/issues/42374
cc @kubernetes/sig-federation-pr-reviews
gives each ingress object a cluster-unique Uid that can be
leveraged by ingress providers.
In the process, supplement the testing of configMap updates to
ensure that the updates are propagated prior to any ingress
object being created. Configmap key/vals for Uid and ProviderUid
must exist at time of Ingress creation.
Automatic merge from submit-queue (batch tested with PRs 41931, 39821, 41841, 42197, 42195)
Revert conditional updates to configmap that breaks ingress
Don't prevent configmap updates to happen on existing ingress-uid configmap despite the lack of ingress objects. Otherwise, ingress objects get created with the wrong name.
Automatic merge from submit-queue (batch tested with PRs 42044, 41694, 41927, 42050, 41987)
federation: use generated listers
**What this PR does / why we need it**: switch federation code to use generated listers
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
cc @smarterclayton @derekwaynecarr @marun @gmarek @wojtek-t @deads2k @liggitt @sttts @timothysc @nikhiljindal @jianhuiz @kubernetes/sig-federation-pr-reviews @kubernetes/sig-scalability-pr-reviews
Automatic merge from submit-queue
error strings should not end with punctuation
**What this PR does / why we need it**:
Delete the end punctuation of error strings
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
https://github.com/golang/go/wiki/CodeReviewComments#error-strings
**Release note**:
```release-note
```
Signed-off-by: yupeng <yu.peng36@zte.com.cn>
These files have been created lately, so we don't have much information
about them anyway, so let's just:
- Remove assignees and make them approvers
- Copy approves as reviewers
Automatic merge from submit-queue (batch tested with PRs 39475, 38666, 39327, 38396, 39613)
Fix a bug in cascading deletion of federation objects
When FinalizerOrphan is present and set to true in federated object we are currently removing the FinalizerOrphan first and then removing FinalizerDeleteFromUnderlyingClusters. if a reconciliation is triggered in between the finalizer removals, it has undesired effect of object deletion in federated cluster.
So we should remove FinalizerDeleteFromUnderlyingClusters first and then remove FinalizerOrphan, when FinalizerOrphan is set to true.
@nikhiljindal, @madhusudancs
Automatic merge from submit-queue (batch tested with PRs 38212, 38792, 39641, 36390, 39005)
Updating federated service controller to support cascading deletion
Ref https://github.com/kubernetes/kubernetes/issues/33612
Service controller is special than other federation controllers because it does not use federatedinformer and updater to sync services (it was written before we had those frameworks).
Updating service controller code to instantiate these frameworks and then use deletion helper to perform cascading deletion.
Note that, I havent changed the queuing logic in this PR so we still dont use federated informer to manage the queue. Will do that in the next PR.
cc @kubernetes/sig-federation-misc @mwielgus @quinton-hoole
```release-note
federation: Adding support for DeleteOptions.OrphanDependents for federated services. Setting it to false while deleting a federated service also deletes the corresponding services from all registered clusters.
```
Automatic merge from submit-queue
Enable update tests in federated namespace controller
Previously it was disabled because of flakiness.
cc: @nikhiljindal @madhusudancs
Automatic merge from submit-queue
Fix typo for federation/*
**What this PR does / why we need it**:
Increase code readability for this new member in v1.5
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
**Special notes for your reviewer**:
Could we develop a typo-fix bot along with a k8s terminology dictionary ?
**Release note**:
```release-note
```
Automatic merge from submit-queue
Add CoreDNS as Federation DNS provider
This PR contains following
1. DNS provider implementation for CoreDNS with etcd as backend
2. Script for optionally deploying CoreDNS in federation control plane
Still to do:
1. HA for CoreDNS service need to be handled, would be taken up in subsequent PR
2. Need to configure CoreDNS as fallback DNS to in-cluster KubeDNS(Manual Step). Need to consider automation.
Request for review: @quinton-hoole @madhusudancs @nikhiljindal @kshafiee @deepak-vij
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Enable updates in federated ingress controller tests
And make channel in test helper non-blocking.
cc: @nikhiljindal @madhusudancs
Automatic merge from submit-queue
Fix package aliases to follow golang convention
Some package aliases are not not align with golang convention https://blog.golang.org/package-names. This PR fixes them. Also adds a verify script and presubmit checks.
Fixes#35070.
cc/ @timstclair @Random-Liu
Automatic merge from submit-queue
Fix infinte loop in federated ingress controller
Previously ingress controller was constantly scheduling reconcilation, even if no updates were needed. That behavior creates a big mess in the logs and consumes resources.
This pr also fixes the stop function for federated ingress controller.
cc: @nikhiljindal @madhusudancs
Automatic merge from submit-queue
Revision handling in federated deployment controller
Deployment controller in regular kubernetes automatically adds an annotation in deployment. This causes a bit of confusion in controller and tests. This PR skips revision annotation in checks. In the next K8S release we will need to have better support for deployment revisions.
Helps with #36588
cc: @nikhiljindal @madhusudancs
Automatic merge from submit-queue
Stop deleting underlying services when federation service is deleted
Fixes https://github.com/kubernetes/kubernetes/issues/36799
Fixing federation service controller to not delete services from underlying clusters when federated service is deleted.
None of the federation controller should do this unless explicitly asked by the user using DeleteOptions. This is the only federation controller that does that.
cc @kubernetes/sig-cluster-federation @madhusudancs
```release-note
federation service controller: stop deleting services from underlying clusters when federated service is deleted.
```
Automatic merge from submit-queue
Fixing the logic to select first cluster in federated ingress controller
Ref https://github.com/kubernetes/kubernetes/issues/36074.
Before this change, ingress controller was using cluster with clusterIndex = 0 as the first cluster to create the ingress in.
But the ordering of clusters can change and hence ingress controller ended up creating the ingress in multiple clusters.
This PR fixes it by using an annotation on federated ingress. Controller now picks up a cluster randomly as the first cluster and creates ingress there. This clusters name is stored as an annotation on the federated ingress. Contoller does not create an ingress in any other cluster if this annotation is set on the federated ingress and IP has not been propagated. Once IP has been propagated, controller creates the ingress in all clusters.
cc @kubernetes/sig-cluster-federation @madhusudancs
Automatic merge from submit-queue
populate ready replicas and aviable replicas to federated replicaset …
populate ready replicas and aviable replicas to federated replicaset status
@nikhiljindal #33312#29481#32771
@deepak-vij
Automatic merge from submit-queue
Fixing the ingress annotation key in federated ingress controller
Fixing a few issues found during debugging https://github.com/kubernetes/kubernetes/issues/36074.
cc @kubernetes/sig-cluster-federation @madhusudancs @bprashanth
Automatic merge from submit-queue
Make desired objects in fed controller reconcilation function fully writable
When creating desired objects for sub-cluster in reconciliate functions we don't make full copies. This may result in unexpected race condition in the future when someone actually adds some object modifications (not needed in most cases but anyway - the cost of extra copy will be relatively small comparing to inter-cluster traffic). In case of ReplicaSet and Deployment the spec copy was a shallow one - ok for setting just replicas but also error-prone in the future.
cc: @quinton-hoole @nikhiljindal @madhusudancs
Automatic merge from submit-queue
Federation test util fix - 1
* Make nil and empty map equivalent when testing ObjectMeta equivalence
* Add a function that waits for a particular content inside federated store
cc: @nikhiljindal @madhusudancs
Automatic merge from submit-queue
Ensure proper serialization of updates and creates in federation test watcher
Fix for finalizer test problems. The issue there was that the updates were coming out of order. It was caused by firing updates in new goroutines in test watcher. The proper solution is to order them in a queue and fire in order on a single goroutine.
Ref: https://github.com/kubernetes/kubernetes/issues/36473#issuecomment-259605870
cc: @nikhiljindal @madhusudancs
Automatic merge from submit-queue
Adding cascading deletion support to federation replicaset and deployments
Forked from https://github.com/kubernetes/kubernetes/pull/36330
Ref #33612
Adding cascading deletion support for federated replicasets and deployments.
```release-note
federation: Adding support for DeleteOptions.OrphanDependents for federated replicasets and deployments. Setting it to false while deleting a federated replicaset or deployment also deletes the corresponding resource from all registered clusters.
```
Automatic merge from submit-queue
Adding cadcading deletion support for federated secrets
Ref https://github.com/kubernetes/kubernetes/issues/33612
Adding cascading deletion support for federated secrets.
The code is same as that for namespaces. Just ensuring that DeletionHelper functions are called at right places in secret_controller.
Also added e2e tests.
cc @kubernetes/sig-cluster-federation @caesarxuchao
```release-note
federation: Adding support for DeleteOptions.OrphanDependents for federated secrets. Setting it to false while deleting a federated secret also deletes the corresponding secrets from all registered clusters.
```
If we have a public & private zone with the same name (which is common
on AWS), this means we can still create records. Also tighten up some
of the logic to allow for zones with duplicate names.
Automatic merge from submit-queue
Adding more e2e tests for federated namespace cascading deletion and fixing bugs
Ref https://github.com/kubernetes/kubernetes/issues/33612
Adding more e2e tests for testing cascading deletion of federated namespace.
New tests are now verifying that cascading deletion happen when DeletionOptions.OrphanDependents=false and it does not happen when DeleteOptions.OrphanDependents=true.
Also updated deletion helper to always add OrphanFinalizer. generic registry will remove it if DeleteOptions.OrphanDependents=false. Also updated namespace registry to do the same.
We need to add the orphan finalizer to keep the orphan by default behavior. We assume that its dependents are going to be orphaned and hence add that finalizer. If user does not want the orphan behavior, he can do so using DeleteOptions and then the registry will remove that finalizer.
cc @kubernetes/sig-cluster-federation @caesarxuchao @derekwaynecarr
Automatic merge from submit-queue
Switch federated secret controller to use NamespacedName
To make it cleaner and consistent with other controllers.
cc: @quinton-hoole
We can put subdomains into hosted zones (for example,
foo.federation.example.com can be hosted in example.com)
By allowing sharing a common hosted zone, this means the user doesn't
have to do as much setup.
Automatic merge from submit-queue
Adding cascading deletion support to federated namespaces
Ref https://github.com/kubernetes/kubernetes/issues/33612
With this change, whenever a federated namespace is deleted with `DeleteOptions.OrphanDependents = false`, then federation namespace controller first deletes the corresponding namespaces from all underlying clusters before deleting the federated namespace.
cc @kubernetes/sig-cluster-federation @caesarxuchao
```release-note
Adding support for DeleteOptions.OrphanDependents for federated namespaces. Setting it to false while deleting a federated namespace also deletes the corresponding namespace from all registered clusters.
```
Automatic merge from submit-queue
Federated DaemonSet controller
Based on the secrets controller. E2e tests will come in the next PR.
**Release note**:
```release-note
Federated DaemonSet controller. Supports all the API that regular DaemonSet has.
```
cc: @quinton-hoole @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Checking function integrated with get from chanel in fed tests
Current federation controller tests assume that each actions results in only one etcd action. However, due to various timing issues, this may not necessary the truth and multiple updates can be generated. Thus tests should try to get all available messages from the debug channels instead failing on the first message, which may come from the previous test scenario.
Applied to deployment tests. Other tests will be updated in the following PR(s).
cc: @quinton-hoole @wojtek-t
Automatic merge from submit-queue
Include DaemonSets and Deployments in Namespace deletion
More generic deletion should be added next week.
cc: @quinton-hoole
Automatic merge from submit-queue
Federated deployment controller - part 1
Based on federated replicaset controller (copy + find/replace).
Remaining stuff:
- refacing out common elements to libs
- using owerref in pod analysis
- e2e tests
- renaming concurrency flag for rs and reusing it in deployment
- updating only one cluster at a time if rollingupdate strategy is used.
cc: @quinton-hoole @kubernetes/sig-cluster-federation
**Release note**:
```release-note
Federated deployment controller that supports the same api as the regular kubernetes deployment controller.
```
Automatic merge from submit-queue
Add missing argument to log message in federated ingress controller.
Minor fix, to address this:
1006 01:08:50.605184 1 ingress_controller.go:674] No annotation "kubernetes.io/ingress.global-static-ip-name" exists on ingress "e2e-tests-federated-ingress-2787b/federated-ingress" in federation, and index of cluster "federation-e2e-gce-us-central1-f" is 1 and not zero. Not queueing create operation for ingress *%!q(MISSING)* until annotation exists
Automatic merge from submit-queue
Copy objects in federation test helper
Push clean copies of passed objects in test helper in federation controllers tests.
Fixes: #33667
cc: @gmarek @quinton-hoole @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Update federation controllers to use release 1.5 clientset
This PR is based on #32243, we won't merge it until next week, to make cherry-picks easier.
Automatic merge from submit-queue
Fix goroutine leak in federation service controller
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**: Fixes a memory leak
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#33186
**Special notes for your reviewer**: Every second new goroutines are created and are getting blocked waiting for the lock in the event queue. only one worker will get a lock when there are some events to process, so all the goroutines which are created every second waits for the lock forever and causes the memory/goroutine leak.
As a fix the new worker will be created only when there is no worker exist. and only one worker per cluster either waits for the event or processes all the events and goes out of existence.
```release-note
Fixes memory/goroutine leak in Federation Service controller.
```
Automatic merge from submit-queue
simplify RC and SVC listers
Make the RC and SVC listers use the common list functions that more closely match client APIs, are consistent with other listers, and avoid unnecessary copies.
Marcin Wielgus