Automatic merge from submit-queue (batch tested with PRs 40234, 45885, 42975)
Fed target cluster by label for sync controller
[use clusterselector w/ federated configmap deploys](667dc77444)
**What this PR does / why we need it**: adds the ability to indicate objects are sent to subsets of federated clusters ref #29887
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 45990, 45544, 45745, 45742, 45678)
[Federation] Add integration testing for cluster addition
This PR adds integration testing of the sync controller for cluster addition. This ensures coverage equivalency between the integration tests and the old controller unit tests, so those tests are removed by this PR.
Resolves#45257
cc: @kubernetes/sig-federation-pr-reviews
fix test error formatting
updates from comments
update gofmt
simplify tests
add to new sync controller
add tests
remove configmap changes due to rebase
updates from review
refactor tests to be based on operations
improvements from review
updates from rebase
rebase to #45374
updates from review
refactor SendToCluster for tests
fix import order
rebase to upstream
Automatic merge from submit-queue (batch tested with PRs 45852, 45965, 45941)
Update k8s.io/gengo dependency
**What this PR does / why we need it**:
Brings in changes to improve type checker and deep copy
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42895, 45940)
[Federation] Automate configuring nameserver in cluster-dns for CoreDNS provider
Addresses issue #42894#42822
**Release note**:
```
[Federation] CoreDNS server will be automatically added to nameserver resolv.conf chain When using CoreDNS as dns provider for federation during federation join.
```
cc @madhusudancs @kubernetes/sig-federation-bugs
Automatic merge from submit-queue (batch tested with PRs 45247, 45810, 45034, 45898, 45899)
[Federation] Segregate DNS related code to separate controller
**What this PR does / why we need it**:
This is the continuation of service controller re-factor work as outlined in #41253
This PR segregates DNS related code from service controller to another controller `service-dns controller` which manages the DNS records on the configured DNS provider.
`service-dns controller` monitors the federated services for the ingress annotations and create/update/delete DNS records accordingly.
`service-dns controller` can be optionally disabled and DNS record management could be done by third party components by monitoring the ingress annotations on federated services. (This would enable something like federation middleware for CoreDNS where federation api server could be used as a backend to CoreDNS eliminating the need for etcd storage.)
**Special notes for your reviewer**:
**Release note**:
```
Federation: A new controller for managing DNS records is introduced which can be optionally disabled to enable third party components to manage DNS records for federated services.
```
cc @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 45247, 45810, 45034, 45898, 45899)
Apiregistration v1alpha1→v1beta1
Promoting apiregistration api from v1alpha1 to v1beta1.
API Registration is responsible for registering an API `Group`/`Version` with
another kubernetes like API server. The `APIService` holds information
about the other API server in `APIServiceSpec` type as well as general
`TypeMeta` and `ObjectMeta`. The `APIServiceSpec` type have the main
configuration needed to do the aggregation. Any request coming for
specified `Group`/`Version` will be directed to the service defined by
`ServiceReference` (on port 443) after validating the target using provided
`CABundle` or skipping validation if development flag `InsecureSkipTLSVerify`
is set. `Priority` is controlling the order of this API group in the overall
discovery document.
The return status is a set of conditions for this aggregation. Currently
there is only one condition named "Available", if true, it means the
api/server requests will be redirected to specified API server.
```release-note
API Registration is now in beta.
```
Automatic merge from submit-queue (batch tested with PRs 45374, 44537, 45739, 44474, 45888)
Update 'More info' links on API types and fields
> **Please just review the first commit, the rest is generated files.**
Recent docs website shuffling during 1.6 caused majority of links in the API
types and fields to break. Since we do not have server-side 301 redirects, user
has to click an extra link, and the #target fragment in the URL will be lost. (This is
because GitHub’s redirect_from feature is not ideal.)
For the time being, I have manually gone through all of them to bring them up to date
and add HTTPS to those missing it. This is a docs-only change and impacts generated
code, generated swaggers, API reference docs etc.
cc: @steveperry-53 @devin-donnelly @chenopis fyi, docs links changes (even small title changes) easily breaks links in API reference, Swagger, kubectl explain, and many other places.
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
Automatic merge from submit-queue (batch tested with PRs 45374, 44537, 45739, 44474, 45888)
[Federation] Refactor sync controller's reconcile method for maintainability
This PR refactors the sync controllers reconcile method for maintainability with the goal of eliminating the need for type-specific controller unit tests. The unit test coverage for reconcile is not complete, but I think it's a good start.
cc: @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 45860, 45119, 44525, 45625, 44403)
coredns: support IPv6 record set
Added support for AAAA record for coredns and included unit test.
Refactored common test code to reduce duplication from added test and
existing tests.
Fixed function names in comments for Google and AWS tests to match
actual test name in this area.
**What this PR does / why we need it**:
Adding IPv6 support to kubernetes, once piece at a time. :)
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#44351
**Special notes for your reviewer**:
In addition to the change and unit test method, I did some minor refactoring (since the UT was a near clone of an existing test). Fixed typos in related test methods' comment lines. Please let me know if this is OK (I was thinking it was a small change, but don't know the protocol here), or if I need to break it into multiple commits.
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 45860, 45119, 44525, 45625, 44403)
[Federation] Move annotations and related parsing code as common code
This PR moves some code, which was duplicate, around as common code.
Changes the names of structures used for annotations to common names.
s/FederatedReplicaSetPreferences/ReplicaAllocationPreferences/
s/ClusterReplicaSetPreferences/PerClusterPreferences/
This can be reused in job controller and hpa controller code.
**Special notes for your reviewer**:
@kubernetes/sig-federation-misc
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 45408, 45355, 45528)
Admission options spits out admission control
**What this PR does / why we need it**:
This PR adds ApplyTo method to AdmissionOptions struct. The method creates and initialises admission control to the server configuration.
**Release note**:
```
NONE
```
Added support for AAAA record for coredns and included unit test.
Fixed function names in comments for Google and AWS tests to match
actual test name in this area.
Recent docs website shuffling during 1.6 caused majority of links in the API
types and fields to break. Since we do not have server-side 301 redirects, user
has to click an extra link and the #target segment in the URL will be lost.
I have manually gone through all of them to bring them up to date and add HTTPS
to those missing it.
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
Automatic merge from submit-queue (batch tested with PRs 41331, 45591, 45600, 45176, 45658)
Updating generic registry to return UID of the deleted resource
Ref https://github.com/kubernetes/kubernetes/issues/42594
cc @kubernetes/sig-api-machinery-pr-reviews @smarterclayton
```release-note
Updating apiserver to return UID of the deleted resource. Clients can use this UID to verify that the resource was deleted or waiting for finalizers.
```
Automatic merge from submit-queue
[Federation] Fix federated service reconcilation issue due to addition of External…
…TrafficPolicy field to v1.Service
**What this PR does / why we need it**:
New fields (ExternalTrafficPolicy) are introduced to v1.Service by this PR #41162. If this field is not specified in service spec, the service controller will assign default and updates the service spec.
In federation, the service spec is not updated and we continuously try to reconcile as the federated service and the service in federated cluster do not match.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#45795
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
cc @kubernetes/sig-federation-bugs @madhusudancs
ApplyTo adds the admission chain to the server configuration the method lazily initializes a generic plugin
that is appended to the list of pluginInitializers.
apiserver.Config will hold an instance of SharedInformerFactory to ensure we only have once instance.
The field will be initialized in apisever.SecureServingOptions
Automatic merge from submit-queue (batch tested with PRs 45623, 45241, 45460, 41162)
Promotes Source IP preservation for Virtual IPs from Beta to GA
Fixes#33625. Feature issue: kubernetes/features#27.
Bullet points:
- Declare 2 fields (ExternalTraffic and HealthCheckNodePort) that mirror the ESIPP annotations.
- ESIPP alpha annotations will be ignored.
- Existing ESIPP beta annotations will still be fully supported.
- Allow promoting beta annotations to first class fields or reversely.
- Disallow setting invalid ExternalTraffic and HealthCheckNodePort on services. Default ExternalTraffic field for nodePort or loadBalancer type service to "Global" if not set.
**Release note**:
```release-note
Promotes Source IP preservation for Virtual IPs to GA.
Two api fields are defined correspondingly:
- Service.Spec.ExternalTrafficPolicy <- 'service.beta.kubernetes.io/external-traffic' annotation.
- Service.Spec.HealthCheckNodePort <- 'service.beta.kubernetes.io/healthcheck-nodeport' annotation.
```
Automatic merge from submit-queue (batch tested with PRs 45684, 45266, 45669, 44787, 44984)
[Federation] Unify the delivery methods in the ReplicaSet controller.
Unifies the federated and non-federated object delivery methods. The reconcile method that's called already checks for existence, and no other controller has a similar paradigm, so it seems reasonable to simplify it here.
cc @marun
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 44626, 45641)
Update Google Cloud DNS provider Rrset.Get(name) method to return a list and change the `Rrset.List()` implementation to perform a paged walk
Some federated service e2e tests and a few ingress tests would become flaky after a few hundred runs. @csbell spent quite a lot of time debugging this and found out that this flakiness was due to a bug in the federated service controller deletion logic. Deletion of a federated service object triggers a logic in the controller to update the DNS records corresponding to that object. This DNS record update logic would return an error in failed runs which would in-turn cause the controller to reschedule the operation. This led to an infinite retry-failure cycle that never gave the API server a chance to garbage collect the deleted service object.
A couple of days ago we started seeing a correlation between the number of resource records in a DNS managed zone and these test failures. If you look at the test runs before and after run 2900 in the test grid - https://k8s-testgrid.appspot.com/cluster-federation#gce, you will notice that the grid became super green at 2900. That's when I deleted all the dangling DNS records from the past runs.
After some investigation yesterday, we found that `ResourceRecordSet.Get()` interface and its implementation, and `ResourceRecordSet.List()` implementation at least for Google Cloud DNS were incorrect.
This PR makes minimal set of changes (read: least invasive) in Google Cloud DNS provider implementation to fix these problems:
1. Modifies DNS provider Rrset.Get(name) interface to return multiple records and updates federated service controller.
There can be multiple DNS resource records for a given name. They can vary by type, ttl, rrdata and a number of various other parameters. It is incorrect to return a single resource record for a given name.
This change updates the Get interface to return multiple records for a given name and uses this list in the federated service controller to perform DNS operations.
2. Update Google Cloud DNS List implementation to perform a paged walk of lists to aggregate all the DNS records.
The current `List()` implementation just lists the DNS resorce records in a given managed zone once and retruns the list. It neither performs a paged walk nor does it consider the `page_token` in the returned response.
This change walks all the pages and aggregates the records in the pages and returns the aggregated list. This is potentially dangerous as it can blow up memory if there are a huge number of records in the given managed zone. But this is the best we can do without changing the provider interface too much.
Next step is to define a new paged list interface and implement it.
**Release note**:
```release-note
NONE
```
/assign @csbell
cc @justinsb @shashidharatd @quinton-hoole @kubernetes/sig-federation-pr-reviews
When we fetch the dns records by name, we get a list of records that match
the given name. As an optimization we look up to see if the new record we
want to create is already in the returned list to avoid performing any updates.
However, when the new record we want to create isn't in the returned list, it
is hard to say if the returned list contains the list of records that we want
to retain. For example, we might get a list of A records and we want to create
a CNAME record. Creating a new CNAME record without removing the A records is
a DNS misconfiguration. So to play safe we just remove all the existing records
in the list and create the new desired record.
**Note**: This is the opposite of what I said here - https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/44626#-Ki9xQOzybryHvsxNrra.
Automatic merge from submit-queue (batch tested with PRs 45556, 45561, 45256)
[Federation] Replace the indexing lister with a regular store in the replicaset controller
This is part of the refactoring work to allow the replicaset controller to use the generic sync controller.
None of the other controllers use a lister, including the deployment controller
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45382, 45384, 44781, 45333, 45543)
[Federation] Provide updater timeout to instance rather than to Update()
This PR changes the federated updater to receive its timeout at construction rather than on every call to Update(). This provides a slight decrease in coupling by removing the need for the deletion handler to be provided the timeout along with the updater.
cc: @kubernetes/sig-federation-pr-reviews @perotinus
Clayton Coleman