Automatic merge from submit-queue (batch tested with PRs 40234, 45885, 42975)
Fed target cluster by label for sync controller
[use clusterselector w/ federated configmap deploys](667dc77444)
**What this PR does / why we need it**: adds the ability to indicate objects are sent to subsets of federated clusters ref #29887
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 45990, 45544, 45745, 45742, 45678)
[Federation] Add integration testing for cluster addition
This PR adds integration testing of the sync controller for cluster addition. This ensures coverage equivalency between the integration tests and the old controller unit tests, so those tests are removed by this PR.
Resolves#45257
cc: @kubernetes/sig-federation-pr-reviews
fix test error formatting
updates from comments
update gofmt
simplify tests
add to new sync controller
add tests
remove configmap changes due to rebase
updates from review
refactor tests to be based on operations
improvements from review
updates from rebase
rebase to #45374
updates from review
refactor SendToCluster for tests
fix import order
rebase to upstream
Automatic merge from submit-queue (batch tested with PRs 45852, 45965, 45941)
Update k8s.io/gengo dependency
**What this PR does / why we need it**:
Brings in changes to improve type checker and deep copy
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42895, 45940)
[Federation] Automate configuring nameserver in cluster-dns for CoreDNS provider
Addresses issue #42894#42822
**Release note**:
```
[Federation] CoreDNS server will be automatically added to nameserver resolv.conf chain When using CoreDNS as dns provider for federation during federation join.
```
cc @madhusudancs @kubernetes/sig-federation-bugs
Automatic merge from submit-queue (batch tested with PRs 45247, 45810, 45034, 45898, 45899)
[Federation] Segregate DNS related code to separate controller
**What this PR does / why we need it**:
This is the continuation of service controller re-factor work as outlined in #41253
This PR segregates DNS related code from service controller to another controller `service-dns controller` which manages the DNS records on the configured DNS provider.
`service-dns controller` monitors the federated services for the ingress annotations and create/update/delete DNS records accordingly.
`service-dns controller` can be optionally disabled and DNS record management could be done by third party components by monitoring the ingress annotations on federated services. (This would enable something like federation middleware for CoreDNS where federation api server could be used as a backend to CoreDNS eliminating the need for etcd storage.)
**Special notes for your reviewer**:
**Release note**:
```
Federation: A new controller for managing DNS records is introduced which can be optionally disabled to enable third party components to manage DNS records for federated services.
```
cc @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 45247, 45810, 45034, 45898, 45899)
Apiregistration v1alpha1→v1beta1
Promoting apiregistration api from v1alpha1 to v1beta1.
API Registration is responsible for registering an API `Group`/`Version` with
another kubernetes like API server. The `APIService` holds information
about the other API server in `APIServiceSpec` type as well as general
`TypeMeta` and `ObjectMeta`. The `APIServiceSpec` type have the main
configuration needed to do the aggregation. Any request coming for
specified `Group`/`Version` will be directed to the service defined by
`ServiceReference` (on port 443) after validating the target using provided
`CABundle` or skipping validation if development flag `InsecureSkipTLSVerify`
is set. `Priority` is controlling the order of this API group in the overall
discovery document.
The return status is a set of conditions for this aggregation. Currently
there is only one condition named "Available", if true, it means the
api/server requests will be redirected to specified API server.
```release-note
API Registration is now in beta.
```
Automatic merge from submit-queue (batch tested with PRs 45374, 44537, 45739, 44474, 45888)
Update 'More info' links on API types and fields
> **Please just review the first commit, the rest is generated files.**
Recent docs website shuffling during 1.6 caused majority of links in the API
types and fields to break. Since we do not have server-side 301 redirects, user
has to click an extra link, and the #target fragment in the URL will be lost. (This is
because GitHub’s redirect_from feature is not ideal.)
For the time being, I have manually gone through all of them to bring them up to date
and add HTTPS to those missing it. This is a docs-only change and impacts generated
code, generated swaggers, API reference docs etc.
cc: @steveperry-53 @devin-donnelly @chenopis fyi, docs links changes (even small title changes) easily breaks links in API reference, Swagger, kubectl explain, and many other places.
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
Automatic merge from submit-queue (batch tested with PRs 45374, 44537, 45739, 44474, 45888)
[Federation] Refactor sync controller's reconcile method for maintainability
This PR refactors the sync controllers reconcile method for maintainability with the goal of eliminating the need for type-specific controller unit tests. The unit test coverage for reconcile is not complete, but I think it's a good start.
cc: @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 45860, 45119, 44525, 45625, 44403)
coredns: support IPv6 record set
Added support for AAAA record for coredns and included unit test.
Refactored common test code to reduce duplication from added test and
existing tests.
Fixed function names in comments for Google and AWS tests to match
actual test name in this area.
**What this PR does / why we need it**:
Adding IPv6 support to kubernetes, once piece at a time. :)
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#44351
**Special notes for your reviewer**:
In addition to the change and unit test method, I did some minor refactoring (since the UT was a near clone of an existing test). Fixed typos in related test methods' comment lines. Please let me know if this is OK (I was thinking it was a small change, but don't know the protocol here), or if I need to break it into multiple commits.
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 45860, 45119, 44525, 45625, 44403)
[Federation] Move annotations and related parsing code as common code
This PR moves some code, which was duplicate, around as common code.
Changes the names of structures used for annotations to common names.
s/FederatedReplicaSetPreferences/ReplicaAllocationPreferences/
s/ClusterReplicaSetPreferences/PerClusterPreferences/
This can be reused in job controller and hpa controller code.
**Special notes for your reviewer**:
@kubernetes/sig-federation-misc
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 45408, 45355, 45528)
Admission options spits out admission control
**What this PR does / why we need it**:
This PR adds ApplyTo method to AdmissionOptions struct. The method creates and initialises admission control to the server configuration.
**Release note**:
```
NONE
```
Added support for AAAA record for coredns and included unit test.
Fixed function names in comments for Google and AWS tests to match
actual test name in this area.
Recent docs website shuffling during 1.6 caused majority of links in the API
types and fields to break. Since we do not have server-side 301 redirects, user
has to click an extra link and the #target segment in the URL will be lost.
I have manually gone through all of them to bring them up to date and add HTTPS
to those missing it.
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
Automatic merge from submit-queue (batch tested with PRs 41331, 45591, 45600, 45176, 45658)
Updating generic registry to return UID of the deleted resource
Ref https://github.com/kubernetes/kubernetes/issues/42594
cc @kubernetes/sig-api-machinery-pr-reviews @smarterclayton
```release-note
Updating apiserver to return UID of the deleted resource. Clients can use this UID to verify that the resource was deleted or waiting for finalizers.
```
Automatic merge from submit-queue
[Federation] Fix federated service reconcilation issue due to addition of External…
…TrafficPolicy field to v1.Service
**What this PR does / why we need it**:
New fields (ExternalTrafficPolicy) are introduced to v1.Service by this PR #41162. If this field is not specified in service spec, the service controller will assign default and updates the service spec.
In federation, the service spec is not updated and we continuously try to reconcile as the federated service and the service in federated cluster do not match.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#45795
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
cc @kubernetes/sig-federation-bugs @madhusudancs
ApplyTo adds the admission chain to the server configuration the method lazily initializes a generic plugin
that is appended to the list of pluginInitializers.
apiserver.Config will hold an instance of SharedInformerFactory to ensure we only have once instance.
The field will be initialized in apisever.SecureServingOptions
Automatic merge from submit-queue (batch tested with PRs 45623, 45241, 45460, 41162)
Promotes Source IP preservation for Virtual IPs from Beta to GA
Fixes#33625. Feature issue: kubernetes/features#27.
Bullet points:
- Declare 2 fields (ExternalTraffic and HealthCheckNodePort) that mirror the ESIPP annotations.
- ESIPP alpha annotations will be ignored.
- Existing ESIPP beta annotations will still be fully supported.
- Allow promoting beta annotations to first class fields or reversely.
- Disallow setting invalid ExternalTraffic and HealthCheckNodePort on services. Default ExternalTraffic field for nodePort or loadBalancer type service to "Global" if not set.
**Release note**:
```release-note
Promotes Source IP preservation for Virtual IPs to GA.
Two api fields are defined correspondingly:
- Service.Spec.ExternalTrafficPolicy <- 'service.beta.kubernetes.io/external-traffic' annotation.
- Service.Spec.HealthCheckNodePort <- 'service.beta.kubernetes.io/healthcheck-nodeport' annotation.
```
Automatic merge from submit-queue (batch tested with PRs 45684, 45266, 45669, 44787, 44984)
[Federation] Unify the delivery methods in the ReplicaSet controller.
Unifies the federated and non-federated object delivery methods. The reconcile method that's called already checks for existence, and no other controller has a similar paradigm, so it seems reasonable to simplify it here.
cc @marun
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 44626, 45641)
Update Google Cloud DNS provider Rrset.Get(name) method to return a list and change the `Rrset.List()` implementation to perform a paged walk
Some federated service e2e tests and a few ingress tests would become flaky after a few hundred runs. @csbell spent quite a lot of time debugging this and found out that this flakiness was due to a bug in the federated service controller deletion logic. Deletion of a federated service object triggers a logic in the controller to update the DNS records corresponding to that object. This DNS record update logic would return an error in failed runs which would in-turn cause the controller to reschedule the operation. This led to an infinite retry-failure cycle that never gave the API server a chance to garbage collect the deleted service object.
A couple of days ago we started seeing a correlation between the number of resource records in a DNS managed zone and these test failures. If you look at the test runs before and after run 2900 in the test grid - https://k8s-testgrid.appspot.com/cluster-federation#gce, you will notice that the grid became super green at 2900. That's when I deleted all the dangling DNS records from the past runs.
After some investigation yesterday, we found that `ResourceRecordSet.Get()` interface and its implementation, and `ResourceRecordSet.List()` implementation at least for Google Cloud DNS were incorrect.
This PR makes minimal set of changes (read: least invasive) in Google Cloud DNS provider implementation to fix these problems:
1. Modifies DNS provider Rrset.Get(name) interface to return multiple records and updates federated service controller.
There can be multiple DNS resource records for a given name. They can vary by type, ttl, rrdata and a number of various other parameters. It is incorrect to return a single resource record for a given name.
This change updates the Get interface to return multiple records for a given name and uses this list in the federated service controller to perform DNS operations.
2. Update Google Cloud DNS List implementation to perform a paged walk of lists to aggregate all the DNS records.
The current `List()` implementation just lists the DNS resorce records in a given managed zone once and retruns the list. It neither performs a paged walk nor does it consider the `page_token` in the returned response.
This change walks all the pages and aggregates the records in the pages and returns the aggregated list. This is potentially dangerous as it can blow up memory if there are a huge number of records in the given managed zone. But this is the best we can do without changing the provider interface too much.
Next step is to define a new paged list interface and implement it.
**Release note**:
```release-note
NONE
```
/assign @csbell
cc @justinsb @shashidharatd @quinton-hoole @kubernetes/sig-federation-pr-reviews
When we fetch the dns records by name, we get a list of records that match
the given name. As an optimization we look up to see if the new record we
want to create is already in the returned list to avoid performing any updates.
However, when the new record we want to create isn't in the returned list, it
is hard to say if the returned list contains the list of records that we want
to retain. For example, we might get a list of A records and we want to create
a CNAME record. Creating a new CNAME record without removing the A records is
a DNS misconfiguration. So to play safe we just remove all the existing records
in the list and create the new desired record.
**Note**: This is the opposite of what I said here - https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/44626#-Ki9xQOzybryHvsxNrra.
Automatic merge from submit-queue (batch tested with PRs 45556, 45561, 45256)
[Federation] Replace the indexing lister with a regular store in the replicaset controller
This is part of the refactoring work to allow the replicaset controller to use the generic sync controller.
None of the other controllers use a lister, including the deployment controller
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45382, 45384, 44781, 45333, 45543)
[Federation] Provide updater timeout to instance rather than to Update()
This PR changes the federated updater to receive its timeout at construction rather than on every call to Update(). This provides a slight decrease in coupling by removing the need for the deletion handler to be provided the timeout along with the updater.
cc: @kubernetes/sig-federation-pr-reviews @perotinus
Automatic merge from submit-queue
[Federation] Improve the logging and user feedback in 'kubefed init'
This is a follow-up to #41849, which added some status information. This PR is based off of that one, and includes its changes as well.
See #41725.
```release-note
None
```
The current `List()` implementation just lists the DNS resorce records in
a given managed zone once and retruns the list. It neither performs a paged
walk nor does it consider the `page_token` in the returned response.
This change walks all the pages and aggregates the records in the pages
and returns the aggregated list. This is potentially dangerous as it can
blow up memory if there are a huge number of records in the given
managed zone. But this is the best we can do without changing the
provider interface too much. Next step is to define a new paged list
interface and implement it.
There can be multiple DNS resource records for a given name. They can
vary by type, ttl, rrdata and a number of various other parameters. It
is incorrect to return a single resource record for a given name.
This change updates the Get interface to return multiple records for a given
name and uses this list in the federated service controller to perform
DNS operations.
Automatic merge from submit-queue
Keep UserAgentName style consistent
Keep using UserAgentName for controllers and add some logs for debugging
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
[Federation] Record events in federated updater
Controllers for federated types were previously recording events when generating the list of operations. This change delegates responsibility for recording events to the federated updater so that events are recorded when the operations are actually executed, and ensures consistency across recording of both operation initiation and failure.
The deletion helper was similarly updated to rely on the federated updater for event recording. To support this change to the deletion helper, controllers have been updated to provide a namespace qualified name via the objNameFunc function to ensure that the updater can record events for deletions with the same detail as for add and update operations.
cc: @kubernetes/sig-federation-pr-reviews @perotinus
Automatic merge from submit-queue (batch tested with PRs 45362, 45159, 45321, 45238)
Remove redundent GetObjectKind() defined on types
Embedding TypeMeta is enough.
Automatic merge from submit-queue (batch tested with PRs 45283, 45289, 45248, 44295)
Use docker_bundle rule from new rules_docker repo
**What this PR does / why we need it**: switched to using the new `docker_bundle` rule from `rules_docker` instead of my patched `docker_build` rule. This also brings in some fixes for the docker rules that were missing from my fork.
Additionally, I switched out the `git_repository` rules for `http_archive` rules, since that seems to be recommended by the bazel docs (and might be faster).
Lastly, I updated the `pkg_tar` rules to use my patch, which doesn't prepend `./` to files inside the tarballs.
This one should likely be merged upstream in the near future.
I think this is the last of the changes necessary to have `bazel run //:ci-artifacts` working properly to support using bazel for e2e in CI.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40544, 44338, 45225)
[Federation]Update event type
Use EventTypeWarning instead of EventTypeNormal when get failure
Left behind after pr #40296
Automatic merge from submit-queue
Fixes broken link in PodSpec
The PodSpec reference to NodeSelector misses
the `.md` extension in the file.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 45100, 45152, 42513, 44796, 45222)
[Federation] Generate the secret name in kubefed join.
Addresses part of #42324. A follow-up PR will address annotating Federation resources.
```release-note
Remove the `--secret-name` flag from `kubefed join`, instead generating the secret name arbitrarily.
```
Automatic merge from submit-queue
Remove GetClientsetForCluster()
The newClusterClientset() has insteaded of GetClientsetForCluster(),
and GetClientsetForCluster() run wrong. Let's remove it.
Automatic merge from submit-queue
README.md: Update outdated links
**What this PR does / why we need it**:
the PR aims to update some links.
Some links with "#" would not redirect to right point of pages.
Other links without "#" can work, but they are outdated. I change them by the way.
**Special notes for your reviewer**:
**Release note**:
```release-note
```
none
Automatic merge from submit-queue
Ignore IsNotFound error
IsNotFound error is fine since that means the object is
deleted already, so we should check err and ignore err
before returning.
Automatic merge from submit-queue
Delete "hard-coded" default value in flags usage.
**What this PR does / why we need it**:
Some flags of kubernetes components have "hard-coded" default values in their usage info. In fact, [pflag pkg](https://github.com/kubernetes/kubernetes/blob/master/vendor/github.com/spf13/pflag/flag.go#L602-L608) has already added a string `(default value)` automatically in the usage info if the flag is initialized. Then we don't need to hard-code the default value in usage info. After this PR, if we want to update the default value of a flag, we only need to update the flag where it is initialized. `pflag` will update the usage info for us. This will avoid inconsistency.
For example:
Before
```
kubelet -h
...
--node-status-update-frequency duration Specifies how often kubelet posts node status to master. Note: be cautious when changing the constant, it must work with nodeMonitorGracePeriod in nodecontroller. Default: 10s (default 10s)
...
```
After
```
kubelet -h
...
--node-status-update-frequency duration Specifies how often kubelet posts node status to master. Note: be cautious when changing the constant, it must work with nodeMonitorGracePeriod in nodecontroller. (default 10s)
...
```
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
This PR doesn't delete some "hard-coded" default values because they are not explicitly initialized. We still need to hard-code them to give users friendly info.
```
--allow-privileged If true, allow containers to request privileged mode. [default=false]
```
**Release note**:
```release-note
None
```
Automatic merge from submit-queue
Prepare for move zz_generated_deepcopy.go to k8s.io/api
This is in preparation to move deep copies to with the types to the types repo (see https://github.com/kubernetes/gengo/pull/47#issuecomment-296855818). The init() function is referring the `SchemeBuilder` defined in the register.go in the same packge, so we need to revert the dependency.
This PR depends on https://github.com/kubernetes/gengo/pull/49, otherwise verification will fail.
Automatic merge from submit-queue (batch tested with PRs 42740, 44980, 45039, 41627, 45044)
[Federation] Convert Daemonset to use the generic sync controller
To be rebased on master when @perotinus's configmaps PR merges.
Tested integration and e2e.
Automatic merge from submit-queue (batch tested with PRs 44942, 41258)
[Federation] Use federated informer for service controller and annotations to store lb ingress
**What this PR does / why we need it**:
This is breaking up of the PR #40296 into smaller one. please refer to #41253
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Handles 2 tasks in #41253
Fixes issues in #27623, #35827
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
cc @quinton-hoole @nikhiljindal @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41287, 41636, 44881, 44826)
[Federation] Only attempt to log information about pods in federation/cluster/log-dump.sh if those pods exist.
Fixes an issue where the script fails, because bash considers an empty array an unbound variable.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41287, 41636, 44881, 44826)
Bump to go1.8 and remove the edge GOROOT
**What this PR does / why we need it**:
Bumps to go1.8; we get:
- performance improvements
- build time improvements
- the possibility to remove the hacky edge-GOROOT for arm and ppc64le that must use go1.8
- all other awesome features that are included in go1.8: https://golang.org/doc/go1.8
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/kubernetes/kubernetes/issues/38228
**Special notes for your reviewer**:
@ixdy Please push the image ASAP so we can see if this passes all tests
**Release note**:
```release-note
Upgrade go version to v1.8
```
cc @ixdy @bradfitz @jessfraz @wojtek-t @timothysc @spxtr @thockin @smarterclayton @bprashanth @gmarek
Automatic merge from submit-queue (batch tested with PRs 44607, 44472, 44482)
[Federation] Fix the comments on FederationNameAnnotation
Follow-up for minor issues raised in #42683
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 44583, 44139, 44753)
Federation: Removing duplicate finalizer manipulation logic in federation controllers
Ref #40989
Using apimachinery's meta accessor to manipulate objects instead of duplicating that logic in each controller.
Just a cleanup, no behavior change.
cc @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 44862, 42241, 42101, 43181, 44147)
[Federation] Build a simple hyperkube image on-the-fly only containing the hyperkube binary for development and testing purposes.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 44837, 44779, 44492)
Default ObjectNameFunc for all REST Stores
All `Store`s in Kubernetes follow the same logic for determining the name of an object. This change makes it so that `CompleteWithOptions` defaults the `ObjectNameFunc` if it is not specified. Thus a user does not need to remember to use `ObjectMeta.Name`. Using the wrong field as the name can lead to an object which has a name that bypasses normal object name validation.
Signed-off-by: Monis Khan <mkhan@redhat.com>
cc @liggitt @soltysh for review
**Release note**:
```
NONE
```
All Stores in Kubernetes follow the same logic for determining the name
of an object. This change makes it so that CompleteWithOptions defaults
the ObjectNameFunc if it is not specified. Thus a user does not need to
remember to use ObjectMeta.Name. Using the wrong field as the name can
lead to an object which has a name that bypasses normal object name
validation.
Signed-off-by: Monis Khan <mkhan@redhat.com>
Automatic merge from submit-queue (batch tested with PRs 42202, 40784, 44642, 44623, 44761)
Split out AdmissionOptions
**What this PR does / why we need it**: The changes in this PR split out AdmissionOptions from ServerRunOptions. In the long term AdmissionOptions will accepts various dependencies
and spit out AdmissionControl
**Release note**:
NONE
Automatic merge from submit-queue (batch tested with PRs 44499, 44674)
Strip "pods/" prefix from the pod names returned by kubectl get pods.
Note that the result returned by kubectl get -o name has plural prefixes "pods". We were already trying to remove the prefix "pod", but that's not how the results are returned unfortunately.
**Release note**:
```release-note
NONE
```
cc @perotinus @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue
Fix ensureDnsRecords comments for federated services
I went to look at the source comments, because the documentation is not exhaustive about what kind of DNS records are created for federated services (and http://blog.kubernetes.io/2016/07/cross-cluster-services.html is wrong...).
It turns out that even the comment is not in sync with the code: two out of three records listed use `.federation`, while the author probably meant `.mydomain.com` (which has less chance of getting mixed up with `myfed`). I fixed those, as well as a few spelling and parenthesis errors. Hopefully this will help others save time and not scratch their heads.
cc @quinton-hoole
Automatic merge from submit-queue (batch tested with PRs 44645, 44639, 43510)
[Federation][kubefed]: Set apiserver to bind securely to 8443 instead of 443
On platforms like OpenShift that don't run containers as root by default, binding to ports < 1000 is not permitted. Having the apiserver bind to a high port means it can run with reduced privileges. The service will still expose the apiserver on 443, so this change shouldn't impact clients of the federation api.
cc: @kubernetes/sig-federation-pr-reviews @perotinus
On platforms like OpenShift that don't run containers as root by
default, binding to ports < 1000 is not permitted. Having the
apiserver bind to a high port means it can run with reduced
privileges. The service will still expose the apiserver on 443, so
this change shouldn't impact clients of the federation api.
Automatic merge from submit-queue
[Federation] Print out status updates while `kubefed init` is running
This is not an ideal final state–it does not address the appearance of hanging during long-running commands, for example–but it provides some level of information when the operations are successful.
See #41725.
**Release note**:
```release-note
Prints out status updates when running `kubefed init`
```
Automatic merge from submit-queue (batch tested with PRs 44469, 44566, 44467, 44526)
[Federation]Fix panic: index out of range
When the number of clusterStatusNew's Conditions is different from
the number of clusterStatusOld's Conditions, clustercontroller
will panic. We should check it before comparing.
When the number of clusterStatusNew's Conditions is different from
the number of clusterStatusOld's Conditions, clustercontroller
will panic. We should check it before comparing.
Automatic merge from submit-queue (batch tested with PRs 44440, 44038, 44302, 44316, 43876)
Extend the gc admission plugin to check ownerReference.blockOwnerDeletion
#Extend the gc admission plugin to prevent user who doesn't have delete permission of the *owner* from changing blockOwnerDeletion field of existing ownerReferences, or adding ownerReference with blockOwnerDeletion=true.
The plugin need a RESTMapper to translate ownerRef.Kind to Resource. It should be using a dynamic one. However, as discussed in https://github.com/kubernetes/kubernetes/pull/42615, such a RESTMapper will be built after watchable discovery API is implemented, so in this PR the plugin is using the `api.Registry.RESTMapper()`, which is also [used](https://github.com/kubernetes/kubernetes/blob/master/cmd/kube-controller-manager/app/core.go#L165-L166) by the garbage collector currently.
```release-note
Extending the gc admission plugin so that a user who doesn't have delete permission of the *owner* cannot modify blockOwnerDeletion field of existing ownerReferences, or add new ownerReference with blockOwnerDeletion=true
```
cc @lavalamp
Automatic merge from submit-queue (batch tested with PRs 44424, 44026, 43939, 44386, 42914)
remove defaulting from conversion path
follow up for #42764
* remove call to defaulting from conversion path (defaulting is a separate step from conversion)
* remove non-top-level-object defaulting registration (unused after conversion call is removed)
* generate missing top-level defaults for some api groups:
* autoscaling/v2alpha1
* policy/v1alpha1
* policy/v1beta1
* register top-level defaults for some api groups that were missing them:
* autoscaling/v2alpha1
* settings/v1alpha1
Automatic merge from submit-queue
dnsprovider: Avoid panic if fields are nil
The aws-sdk has some helper functions which should generally be used
whenever dereferencing an AWS provided pointer, in case the pointer is
nil, which would otherwise be a panic.
Issue https://github.com/kubernetes/kops/issues/2347
```release-note
dnsprovider: avoid panic if route53 fields are nil
```
Automatic merge from submit-queue
Non global admission plugin registry
For testing the global state is a problem. This PR turns the actual registry into a struct that must be instantiated. For the beginning, we do this in `pkg/kubeapiserver/admission`. In some follow-up (where we hunt down all globals some day), we will move this into the genericapiserver.
Automatic merge from submit-queue
Conversionlint
**What this PR does / why we need it**:
This PR adds a comment placeholder for the exported or public functions in conversion generator files. Such functions without a comment results into golint failure in various generated files. The changes in this patch takes care of about 36 related lint failures.
Given below is an example lint error,
zz_generated.conversion.go:91:1: exported function Convert_v1alpha1_Binding_To_servicecatalog_Binding should have comment or be unexported
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Considering minor changes no issue is created.
**Special notes for your reviewer**:
Hello, I am trying to fix golint failures as we see them in the kubernetes-incubator/service-catalog project. I have separate PRs opened for lint issues related to other generator code which are in separate libraries like gengo. Thanks!
**Release note**:
```release-note
```
The aws-sdk has some helper functions which should generally be used
whenever dereferencing an AWS provided pointer, in case the pointer is
nil, which would otherwise be a panic.
Issue https://github.com/kubernetes/kops/issues/2347
I went to look at the source comments, because the documentation is not exhaustive about what kind of DNS records are created for federated services (and http://blog.kubernetes.io/2016/07/cross-cluster-services.html is wrong...).
It turns out that even the comment is not in sync with the code: two out of three records listed use `.federation`, while the author probably meant `.mydomain.com` (which has less chance of getting mixed up with `myfed`). I fixed those, as well as a few spelling and parenthesis errors. Hopefully this will help others save time and not scratch their heads.
The exported or public functions requires a doc comment to pass golint.
This commit has changes of conversion generated code. The actual doc
changes are added into a separate commit for a clean review.
Automatic merge from submit-queue
[Federation][kubefed] Annotate all Federation API objects with the federation name and (if applicable) the cluster name.
Address part of #42324.
```release-note
Adds annotations to all Federation objects created by kubefed.
```
Automatic merge from submit-queue (batch tested with PRs 44286, 44209)
[Federation][Kubefed] Bug fix to enable disabling federation controllers through override args
Targets https://github.com/kubernetes/kubernetes/issues/42761
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
https://github.com/kubernetes/kubernetes/issues/42761
**Special notes for your reviewer**:
@marun @perotinus @nikhiljindal
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 43273, 44287, 44281)
[Federation] Remove deprecated federation-apiserver-kubeconfig secret
federation-apiserver-kubeconfig was deprecated and was supposed to be removed in 1.6.
Removing all references to it as we no longer use it.
**Release note**:
```
[Federation] Deprecated `federation-apiserver-kubeconfig` is not supported anymore. Should use `--kubeconfig` flag to specify Federation API server kubeconfig.
```
cc @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 43866, 42748)
hack/cluster: download cfssl if not present
hack/local-up-cluster.sh uses cfssl to generate certificates and
will exit it cfssl is not already installed. But other cluster-up
mechanisms (GCE) that generate certs just download cfssl if not
present. Make local-up-cluster.sh do that too so users don't have
to bother installing it from somewhere.
Automatic merge from submit-queue (batch tested with PRs 44019, 42225)
federation: Fixing runtime-config support for federation-apiserver
Fixes https://github.com/kubernetes/kubernetes/issues/42587
Ref https://github.com/kubernetes/kubernetes/issues/38593
Fixing the broken `--runtime-config` flag support in federation-apiserver. Fixing the bugs and using it to disable batch and autoscaling groups. Users can enable them by passing `--runtime-config=apis/all=true` to federation-apiserver.
~This also includes a bug fix to kube-apiserver registry that allows users to disable api/v1 resources~
cc @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue
[Federation] Fix Running service controller in federation kubectl tests
Fixes: #42607
cc @nikhiljindal @kubernetes/sig-federation-bugs
Automatic merge from submit-queue (batch tested with PRs 43777, 44121)
Add patchMergeKey and patchStrategy support to OpenAPI
Support generating Open API extensions for strategic merge patch tags in go struct tags
Support `patchStrategy` and `patchMergeKey`.
Also support checking if the Open API extension and struct tags match.
```release-note
Support generating Open API extensions for strategic merge patch tags in go struct tags
```
cc: @pwittrock @ymqytw
(Description mostly copied from #43833)
Automatic merge from submit-queue (batch tested with PRs 44119, 42538, 43802, 42336, 43396)
iSCSI CHAP support
**What this PR does / why we need it**:
To support CHAP authentication in a multi-tenant setup
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Support iSCSI CHAP authentication
```
Automatic merge from submit-queue
[Federation] refactor controller for secrets
First pass of a simple federation controller that can be configured via an adapter to support multiple types. The controller is a strict refactor of the secrets controller.
Targets #40989
cc: @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 44191, 44117, 44072)
Default FEDERATION_KUBE_CONTEXT to FEDERATION_NAME in federation e2e up/down scripts.
This is consistent with how kubefed creates kubeconfig contexts.
**Release note**:
-->
```release-note
NONE
```
This change uses an adapter class to abstracts the interaction of the
secret controller with the secret api type. This is the first step to
creating a generic controller that can target any type for which an
adapter exists.
Automatic merge from submit-queue
[Federation] Add integration test for secrets
This PR adds an integration test for secrets that:
- performs create/read/update/delete on federation resources and validates that the changes are propagated to member clusters.
- uses an abstraction layer (fixture and adapter) to minimize the code required to support each federated type
- It should be possible to replace a test-specific adapter with a runtime adapter in the future (as per #41050)
- reuses fixture (federation api and clusters) across different resource types to minimize setup overhead
- on a fast machine, setup takes ~4s, and validating each type takes ~2s
- uses the [Subtest feature added in Go 1.7](https://blog.golang.org/subtests) to allow the test for a specific controller to be run in isolation
- ``make test-integration WHAT="federation -test.run=TestFederationCRUD/secret"``
Once this PR merges the test can be extended to target other federated types.
This PR targets #40705
cc: @kubernetes/sig-federation-pr-reviews @derekwaynecarr
Automatic merge from submit-queue
[Federation] Remove FEDERATIONS_DOMAIN_MAP references
Remove all references to FEDERATIONS_DOMAIN_MAP as this method is no longer is used and is replaced by adding federation domain map to kube-dns configmap.
cc @madhusudancs @kubernetes/sig-federation-pr-reviews
**Release note**:
```
[Federation] Mechanism of adding `federation domain maps` to kube-dns deployment via `--federations` flag is superseded by adding/updating `federations` key in `kube-system/kube-dns` configmap. If user is using kubefed tool to join cluster federation, adding federation domain maps to kube-dns is already taken care by `kubefed join` and does not need further action.
```
Automatic merge from submit-queue
[Federation] Use cascading deletion when deleting resources in underlying clusters
The Federation control plane issues a delete command unless it wants to orphan the underlying per-cluster resource. When issuing that command, always set the orphanDependents to false.
/release-note-none
/sig-federation
Automatic merge from submit-queue (batch tested with PRs 44084, 42964)
Removing both finalizers in federation controllers in a single update
Fixes https://github.com/kubernetes/kubernetes/issues/43828
There is a bug right now where the controller fails to delete the object if one finalizer is removed and the second isnt.
This updates the code so that both the finalizers are removed in a single API call. Kept the code changes minimum to enable cherrypick in 1.6.x
cc @csbell @kubernetes/sig-federation-bugs
The Federation control plane issues a delete command unless it wants to orphan the underlying per-cluster resource. When issuing that command, always set the orphanDependents to false.
Automatic merge from submit-queue
Support status.hostIP in downward API
**What this PR does / why we need it**:
Exposes pod's hostIP (node IP) via downward API.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
fixes https://github.com/kubernetes/kubernetes/issues/24657
**Special notes for your reviewer**:
Not sure if there's more documentation that's needed, please point me in the right direction and I will add some :)
Brendan Burns